The other major limitation is that forking the encoder state significantly inflates the number of reference buffers you need to keep, which greatly increases memory requirements. That's not much of an issue for software, but it can be a significant problem for hardware (a lot of real-time interactive encoding is still done purely in software, however).