TLDR the standard practice of using 'word vectors' (numeric vector representation of words) may soon be superceded by just using entire pretrained neural nets as is standard in CV, and we have both conceptual and empirical reasons to believe language modeling is how it'll happen.

Helped edit this piece, think it is spot on - exciting times for NLP.