This latter bit is exactly what modern 2D display drivers already do, or at least what the NVIDIA binary X driver does. It keeps a glyph cache in video memory and just blits from it to the extent that it can. It's still counting on the software to do the rasterization though, and I don't think there's a way to do the rasterizing on the GPU and then feed that data straight into the glyph cache.