The Unix Philosophy: A) Tasks should done by combinations of simple tools combined into an elegant pipe. B) Each tools should do three things, one of them at least so-so well: Those three are: 1) Hackily parse the output of the previous tool by flaky assumptions like that a certain delimiter is always present, that spaces will not occur in such and such an item, or that such and such a field is from this column to th…
1) This is not done by every tool. Instead, there are tools built for specific parsing purposes, and each tool is only in charge of interpreting its command-line arguments or, optionally, stdin in some suitable way. Tools are agnostic of the output of previous tools, and only care about processing data that makes sense to them. It's the task of the user to ensure this data is structured correctly.
2) Where are you getting these limits? Shell limits can be controlled by ulimit(1p), but the standard file descriptors function as unlimited streams of data.
3) Again, every process is free to choose the best way to output data. Some have flags that make processing the output easier by another tool, otherwise they default to what makes sense for the user.
You seem to have a bone to pick with the fact that the data shared between processes is unstructured, and that the user must handle this on their own, but this is what enables building independent tools that "do one thing well", yet are still able to work together. Sure, in a tightly controlled environment, tools can share structured data (e.g. objects in PowerShell, JSON in NuShell, Murex, etc.), but this comes at the expense of added complexity, since each tool needs to handle this specific format, encoding, etc. This is difficult to coordinate and scale, and arguably the rich Unix ecosystem wouldn't exist if some arcane format was chosen 40+ years ago, or if a new one needs to be supported whenever something "better" than JSON comes along. Leaving the data unstructured and up to each process and user to handle, ensures both past and future compatibility.