Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

73% of AI startups are just prompt engineering

pub.towardsai.net

Post reply on HN
Print view View original

71–80 of 212 posts

Re: 73% of AI startups are just prompt engineering

#71

Post by hotpaper75 »

yalogin wrote: post #62

That is lower than I expected. There are just a handful of companies that create llms. They are all more ir less similar. So all automation is in using them, which is prompt engineering if you see that way. The bigger question is, this is the same story with apps on mobile phones. Apple and google could easily replicate your app if they wanted to and they did too. That danger is much higher with these ai startups. Th…

Do not forget that a product idea needs to meet a certain ROI to be stolen. Big Tech won't go after opportunities that do not generate billion-level revenue. This leaves some room for applications where you can earn decent money.

Re: 73% of AI startups are just prompt engineering

#72

Post by teiferer »

goranmoomin wrote: post #22

It is beyond annoying that the article is totally generated by AI. I appreciate the author (hopefully) spending effort in trying to figure out the AI systems, but the obviously-LLM non-edited content makes me not trust the article.

What makes you believe that anything in the article is real?

The author seems to not exist and it's unclear where the data underlying the claims is even coming from since you can't just go and capture network traffic wherever you like.

A little due diligence please.

Re: 73% of AI startups are just prompt engineering

#73

Post by cess11 »

InsideOutSanta wrote: post #40
analogpixel wrote: post #30

Where is this guy sitting that he is able to collect all of this data? And why is he able to release it all in a blog post? (my company wouldn't allow me to collect and release customer data like this.)

It sounds like some of these companies call the OpenAI or Anthropic APIs directly from their frontend. Later, the author also mentions "response time patterns for every major AI API," so maybe there's some information about the backend leaking that way even if the API calls are bridged. But I'd like to know an actual answer to this, too, especially since large parts of this post read as if they were written by an LLM…

"I found 12 companies that left API keys in their frontend code. I reported them all. None responded."

They claim to have found that.

Re: 73% of AI startups are just prompt engineering

#74

Post by altcognito »

kgeist wrote: post #60
drowsspa wrote: post #26

Earlier quoted context omitted.

Is it, though? Apparently the current best practice is just to allow the LLM untethered access to everything and try to control access by preventing prompt injection...

Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to explore more options during search - add hybrid BM-25+vector search with proper rank fusion - tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, ho…

whoa, two weeks

Re: 73% of AI startups are just prompt engineering

#75

Post by [deleted] »

kgeist wrote: post #60
drowsspa wrote: post #26

Earlier quoted context omitted.

Is it, though? Apparently the current best practice is just to allow the LLM untethered access to everything and try to control access by preventing prompt injection...

Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to explore more options during search - add hybrid BM-25+vector search with proper rank fusion - tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, ho…

[deleted]

Re: 73% of AI startups are just prompt engineering

#76

Post by mbesto »

kgeist wrote: post #60
drowsspa wrote: post #26

Earlier quoted context omitted.

Is it, though? Apparently the current best practice is just to allow the LLM untethered access to everything and try to control access by preventing prompt injection...

Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to explore more options during search - add hybrid BM-25+vector search with proper rank fusion - tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, ho…

"did not hallucinate"

Sorry to nitpick, but this is not technically possible no matter how much RAG you throw at it. I assume you just mean "hallucinates a lot less"

Re: 73% of AI startups are just prompt engineering

#77

Post by rynn »

indymike wrote: post #5

A long time ago a mentor of mine said, "In tech, often an expert is someone that know one or two things more than everyone else. When things are new, sometimes that's all it takes." It's no surprise it's just prompt engineering. Every new tech goes that way - mainly because innovation is often adding one or two things more the the existing stack.

It’s easy to underestimate the amount of testing “just” prompt/context engineering takes to get above average results.

And then you need to see what variations work best with different models.

My POCs for personal AI projects take time to get this right. It’s not like the API calls are the hard portion of the software.

Re: 73% of AI startups are just prompt engineering

#78

Post by apwell23 »

mettamage wrote: post #46
drowsspa wrote: post #26

Earlier quoted context omitted.

Is it, though? Apparently the current best practice is just to allow the LLM untethered access to everything and try to control access by preventing prompt injection...

You still need to find the correct data, and get it to the LLM. IMO, a lot of it is data engineering work with API calls to an LLM as an extra step. I'm currently doing a lot of ETL work with Airflow (and whatever data {warehouses, lakes, bases} are needed) to get the right data to a prompt engineering flow. The prompt engineering flow is literally a for loop of Google Docs in a Google Drive that non-tech people, but…

can you describe what the usecase is ?

Re: 73% of AI startups are just prompt engineering

#79

Post by rynn »

altcognito wrote: post #74
kgeist wrote: post #60

Earlier quoted context omitted.

Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to explore more options during search - add hybrid BM-25+vector search with proper rank fusion - tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, ho…

whoa, two weeks

@apwell23 while the author didn’t say how s/he measured QA, creating the QA process was literally the first bullet.

Re: 73% of AI startups are just prompt engineering

#80

Post by KineticLensman »

indymike wrote: post #5

A long time ago a mentor of mine said, "In tech, often an expert is someone that know one or two things more than everyone else. When things are new, sometimes that's all it takes." It's no surprise it's just prompt engineering. Every new tech goes that way - mainly because innovation is often adding one or two things more the the existing stack.

I remember being told that the secret of good consultancy is knowing what to read on your way to the meeting
Post reply on HN