The general point is that you have to robustly compromise and satisfice all the goals. People tend to be rather good at it when taken as a group. (Any particular person may be bad at a given subset of all problems.)

It is a kind of optimality condition on all three goals.

The robustness additionally means that should conditions change, the algorithm usually will become better not worse and should a degradation still happen, it will be graceful and not catastrophical.

It's a hard and open problem in ML and especially ANN, design of robust solutions in the space. Most have really bad problems with it even when debiased.

Hello - do you have a good reference to this area? I bang on about similar ideas whenever allowed, but haven't found good support in the literature.

Not that I mind that too much!

Good presentation