I think that fear is a very healthy one. I have had an idea for a website for a long time that I haven't gotten around to building yet, but may before too long. It would basically give people such a fear, in hopes that they become better informed and act not to stop the development of such knowledge (it has neutral and positive uses just like any tool which has negative ones) but to restrict its misuse. The site would be a simple one, and decidedly sensationalistic (I am not looking forward to media attention I would hope it would garner, but such is necessary to widely spread an idea these days). It would be machine learning system which has been trained on news articles (or, if I can find such a data set, information about their travel would actually be much better) about politicians and lists of politicians which were later shown to be secretly cheating on their spouses. And it would enable you to select your own politician of choice and see a prediction of how likely that person is to be secretly cheating on their spouse currently.
All of the public discussion around the NSA, and every single last one of the statements by the government about the NSAs practices, centered EXCLUSIVELY on the idea of human analysts at the NSA directly reading the fulltext of peoples communications. Not one single time did they address the (as far as I am concerned) actual danger - the NSAs automated systems having access to peoples communications, providing "summaries" and "ratings" and "predictions." Being worried about some random stranger reading your personal, private communications should maybe be a little bit squicky. Being worried about some random machine learning system teasing out predictions of how likely you are to harbor thoughts that threaten the status quo should be pants-shittingly terrifying.
All of the public discussion around the NSA, and every single last one of the statements by the government about the NSAs practices, centered EXCLUSIVELY on the idea of human analysts at the NSA directly reading the fulltext of peoples communications. Not one single time did they address the (as far as I am concerned) actual danger - the NSAs automated systems having access to peoples communications, providing "summaries" and "ratings" and "predictions." Being worried about some random stranger reading your personal, private communications should maybe be a little bit squicky. Being worried about some random machine learning system teasing out predictions of how likely you are to harbor thoughts that threaten the status quo should be pants-shittingly terrifying.