111–120 of 212 posts
There is absolutely no point of reinventing the wheel to create a generic LLM, spend fortune to run GPUs while there are providers giving this power cheaply
> just prompt engineering This dismisses a lot of actual hard work. The scaffolding required to get SOTA performance is non-trivial! Eg how do you build representative evals and measure forward progress? Also, tool calling, caching, etc is beyond what folks normally call “prompt engineering”. If you think it’s trivial though - go build a startup and raise a seed round, the money is easy to come by if you can show res…
This is rapidly becoming a more extreme version of the classic "what if google does that?" as the foundation model vendors don't necessarily need to target your business or even think about it to eat it.
If an AI company has an AGI, what incentive do they actually have to sell it as a product, especially if it’s a 10x cost/productivity/reliability silicon engineer? Just undercut the competition by building their services from scratch.
But ... what else should they be doing? What's the expectation here? For example, in the 90's, a startup that offered a nice UI for a legacy console based system, would have been a great idea. What's wrong with that?
Prompt engineering isn't as simple as writing prompts in english. It's still engineering data flow, when data is relevant, systems that the AI can access and search, tools that the AI can use, etc.
Nah. There's no such thing as prompt engineering . It doesn't exist. Engineering involves applying scientific principles to solve real world problems. There are no clear scientific principles to apply here. It's all instinct, hunches, educated guesses, and heuristics with maybe some sort of feedback loop. And that's fine, it can still produce useful results. Just don't call it engineering. Maybe artisanal prompt craf…
Not sure if we called it engineering ten years ago.
Earlier quoted context omitted.
Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to explore more options during search - add hybrid BM-25+vector search with proper rank fusion - tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, ho…
"did not hallucinate" Sorry to nitpick, but this is not technically possible no matter how much RAG you throw at it. I assume you just mean "hallucinates a lot less"
What differentiates a product is not the commodity layer it’s built on (databases, programming languages, open source libraries, OS apis, hosting, etc) but how it all gets glued together into something useful and accessible.
It would be a bad strategy for most startups to do anything other than prompt engineering in their AI implementations for the same reason it would be a bad idea for most startups to write low-level database code instead of SQL queries. You need to spend your innovation tokens wisely.
It doesn't have to be this way and it won't be this way forever, but this is the world we live in right now, and it's unclear how many years (or weeks) it'll be until we don't have to do this anymore
> just prompt engineering This dismisses a lot of actual hard work. The scaffolding required to get SOTA performance is non-trivial! Eg how do you build representative evals and measure forward progress? Also, tool calling, caching, etc is beyond what folks normally call “prompt engineering”. If you think it’s trivial though - go build a startup and raise a seed round, the money is easy to come by if you can show res…
And many companies are "just CRUD".