It's not just plugins slow Vim down, a few things that come with Vim are slow too.
The worst is probably language specific support for LaTeX files, particularly syntax highlighting, there is even a tex-slow (http://vimdoc.sourceforge.net/htmldoc/syntax.html#tex-slow) entry in the Vim manual. On my old netbook (haven't heard that word in years!) it was so bad Vim lagged behind my typing, and I'm not particularly fast. I turned off syntax highlighting which fixed the lag, but missed it so much I switched to Emacs instead (and discovered AucTeX which is fantastic, so I wouldn't switch back now).
The worst is probably language specific support for LaTeX files, particularly syntax highlighting, there is even a tex-slow (http://vimdoc.sourceforge.net/htmldoc/syntax.html#tex-slow) entry in the Vim manual. On my old netbook (haven't heard that word in years!) it was so bad Vim lagged behind my typing, and I'm not particularly fast. I turned off syntax highlighting which fixed the lag, but missed it so much I switched to Emacs instead (and discovered AucTeX which is fantastic, so I wouldn't switch back now).