The machine learning Stanford courses are probably the best open education contributions I've encountered. From Ng's CS229 material to Karpathy's rendition of CS231n. These are some of the best pedagogical materials on machine learning/deep learning available.
Years ago I took Chris Manning and Dan Jurafsky‘s Coursera NLP class and it was excellent. I also own, I think, every book they have written including Jurafsky’s book on food.
It is extremely generous of top universities to make their classes available online. Of course, watching the videos and trying the homework assignments on one’s own is not the experience of going to Stanford, but it is less expensive!
I took this class and can vouch for it. They update the class every year to go over recent research - not an easy task in such a fast moving field. For example, this offering covers the Transformer architecture which has recently been used to obtain state of the art results across a wide range of NLP tasks.
Tangentially related if you're interested in keeping up to date with what's going on in the field:
Sebastian Ruder's blog has many good posts on recent advancements in NLP (literature reviews, conference highlights): http://ruder.io/
The posts are concise and accessible enough that you can skim through them quickly. Then you can go check out the paper directly if something piques your interest.
An attendee, claiming to have read 1.5 books on NLP, called it the broadest coverage of NLP he had ever seen. Some of the attendees told me that I had inspired them to turn their careers to NLP.
I skimmed through your PDF. It's very exhaustive and all-encompassing.
But what really got me excited was that you had included 2 of Steven Pinker's books in your "Suggested References". Few weeks ago, I was going over the Wikipedia entry this book by Steven Pinker - "A Sense of Style", and wondering if it has any implications on linguistics.
I wish I had recorded the session I gave. It was several times more interesting and engaging than what the slides read on their own.
I haven't read this book, but have read a blog by Pinker on the same lines. I just also skimmed through the Wikipedia link you mentioned.
The book would not be a recommendation from NLP algorithms perspective. I am sure though that it would be a good book as Pinker is a mind-opening writer.
I'll nevertheless give you some deep food for thought relating to languages:
Check out slide 14 in my talk, especially the remark on its right side. It is a powerful thought.
There is an equilibrium process involved in the shaping of the language over the time, though language must by definition have at least some standardization for it to work, which means it would resist change.
What Pinker is saying at the high level is that the official rules of grammar sometimes deviate away from the equilibrium point. For example, technical jargon and acronyms are often easier for the speaker than the listener. A poor handwriting is likely less tiring for the writer than easier for the reader.
There would also be deviations which make it harder for both the speaker and the listener.
Yet, in both of the cases above, the language would resist change.
As writing came into being (keep in mind that we have been talking for a few million years, and writing only for a few thousand, so no comparison!), written material stands for much longer than vibrations in air molecules, or our memories, do. That further slows down language velocity as standardization now sits across time too.
We now stand where language is standardizing across the world, and getting frozen in the Internet, there's further velocity reduction involved. (Albeit, newer concepts are adding increasingly faster to the languages...)
How do you, in such cases, make the above deviations from the optimum go away?
There are some rules of language, grammar, that shouldn't be. They are like legacy code.
Pinker is educating us about such rules. He's trying to make style come back to that optimal.
Let's take a simple example. Did you know, comma classically sits inside quotes? Here's an example I picked from [1]:
"Good morning, Frank," said Hal.
Note that the comma after Frank is inside the quotes! Why should it be that way? No wonder putting comma outside is gaining acceptance. :-)
There are even weird rules for what happens when multiple paragraphs are to be included inside a single quote. (Hints: Number of opening and closing quotes is not equal in some English dialects under this scenario. Weird, hmmm.)
These rules should just go away. It's better to choose the optimals for the language and make those cultural, style, shifts happen in our language.
Take care. Good discussion. Feel free to reach out again.
And please feel free to refer others to my slides page as you see fit. :-)
Chris Manning is responsible for so much work in this field. I can't wait to go through this video series. Thanks for all your hard work and I want you to know how much it's appreciated.
