Neural networks are impressive only in that they are able to give any kind of meaningful results at all. In the end, they are only a poor mimicry of real machine intelligence, and not much better, conceptually, than plain old nonlinear regression.
Nobody has been able to determine what the structure of a neural network should look like for any given problem (network type, number of nodes, layers, activation functions), how many iterations of the parameter optimization algorithm are needed to achieve "optimal" results, and how "learning" is actually stored in the network.
Statistical learning methods are obviously still useful, but I think the field is still wide open for something to emerge that is closer to true machine intelligence.
Please, try implementing non-linear regression to understand images. Tell me how it goes.
Also, 'nonbody know hows learning is stored'? You very clearly have never worked with neural nets before. Experience is stored in the form of weight values.
Okay, you've got a neural net that does a really good job on identifying types of animals in pictures. unfortunately, whenever you show it a picture of a horse, it says 'fish'. Everything else, it's great at - marmosets, capybaras, dolphins, kangaroos; but it's got a complete blindspot for horses.
Where's the incorrect data stored? How can you fix it? It's in the weight values, somewhere, but you can't go and change the weight values to fix the horse/fish cascade without breaking everything else it knows.
Yes, we know 'where' the data is stored. But it's diffuse, not discrete, so we can't separate it from other data.
Umm even still, you can train it on more horse photos in order to increase its performance specifically on horses. Furthermore, you can study neron activation levels on said horse training data in order to reverse-engineer the neural "ravines" which the activations settle into. And run comparison tests about those ravines against the ravines for say zebras.
This is something actively being done by nn researches. And it lets us do things like take the low level audio processing part of a neural net trained on english voice data, and use it to train smarter neural nets on Portugese voice data than you couldn've without the English voice recordings.
Nobody has been able to determine what the structure of a neural network should look like for any given problem (network type, number of nodes, layers, activation functions), how many iterations of the parameter optimization algorithm are needed to achieve "optimal" results, and how "learning" is actually stored in the network.
Statistical learning methods are obviously still useful, but I think the field is still wide open for something to emerge that is closer to true machine intelligence.