> Why does the successor to the terminal need to be text oriented at all?
Terminals are not "text oriented". They are based on a bidirectional stream of tokens - that can be interpreted as text, or anything else.
That simplicity allows for Unix-style composition. If you make the output something different, then the receiving program will need to be able to parse it. The Amiga OS had some interesting ideas with different data types as system extensions - you'd receive "an image" instead of a JPEG file and you could ask the system to parse it for you. In any case, that's still forcing the receiving program to know what it's receiving.
One way to add some level of complexity is to add JSON output to programs. Then you can push them trough `jq` instead of `grep`, `sed`, or `awk`. Or push it through another tool to make a nice table.
> it’s all legacy invented for a world that is no longer relevant.
I hear that since the Lisa was introduced. Character streams are a pretty common thing today. They are also very useful thanks to their simplicity. Much like Unix, it's an example of the "worse is better" principle. It's simpler, dumber, and, because of that, its uses have evolved over decades with almost no change to the underlying plumbing required - the same tools that worked over serial lines, then multiplexed X.25 channels, then telnet, now work under SSH streams. Apps on both sides only need to know about the token stream.