LLM frameworks like LangChain are causing a java-fication or Python . Do you want a banana? You should first create the universe and the jungle and use dependency injection to provide every tree one at a time, then create the monkey that will grab and eat the banana.
It's funny because I was using Langchain recently and found the most confusing part to be the inheritance model and what type was meant to fill which function in the chain. Using Java would make it impossible to mistype an object even while coding. I constantly wonder why the hell the industry decided Python was suitable for this kind of work.
Second reason - to fail fast. No sense in sculpting novel ideas in C++ while you can muddle with Python 3x faster, that's code intended to be used just a few times, on a single computer or cluster. That was an era dominated by research, not deployments.
Llama.cpp was only possible after the neural architecture stabilized and they could focus on a narrow subset of basic functions needed by LLMs for inference.