Nah, raster encoders can do a pretty great job on vector-style data, whereas vector encoders can't do anything with raster-style data. In other words what happens when you play a video or view a photograph? Are you going to auto-detect that and switch modes? You'll end up just recreating a modern video codec.

The best thing to do is just use a modern video codec and make sure it works well with text and sharp edges.