Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

73% of AI startups are just prompt engineering

pub.towardsai.net

Post reply on HN
Print view View original

101–110 of 212 posts

Re: 73% of AI startups are just prompt engineering

#101

Post by eddythompson80 »

For me, 2023 was an entire year of weekly demos that now looking back at were basically a "Look at this dank prompt I wrote" followed by thunderous applause from the audience (which was mostly, but not exclusively, upper management)

Hell man, I attended a session at an AWS event last year that was entirely the presenter opening Claud and writing random prompts to help with AWS stuff... Like thanks dude... That was a great use of an hour. I left 15 minutes in.

We have a team that's been working on an "Agent" for about 6 months now. Started as prompt engineering, then they were like "no we need to add more value" developed a ton of tools and integrations and "connectors" and evals etc. The last couple of weeks were a "repivot" going back full circle to "Lets simplify all that by prompt engineering and give it a sandbox environment to run publicly documented CLIs. You know, like Claude Code"

The funny thing is I know where it's going next...

Re: 73% of AI startups are just prompt engineering

#102

Post by hn_throwaway_99 »

InsideOutSanta wrote: post #40
analogpixel wrote: post #30

Where is this guy sitting that he is able to collect all of this data? And why is he able to release it all in a blog post? (my company wouldn't allow me to collect and release customer data like this.)

It sounds like some of these companies call the OpenAI or Anthropic APIs directly from their frontend. Later, the author also mentions "response time patterns for every major AI API," so maybe there's some information about the backend leaking that way even if the API calls are bridged. But I'd like to know an actual answer to this, too, especially since large parts of this post read as if they were written by an LLM…

> It sounds like some of these companies call the OpenAI or Anthropic APIs directly from their frontend.

Which would be a major security hole. And sure, lots of startups have major security holes, but not enough that he could come up with these BS statistics.

I'm a little dismayed at how high up this has been voted given the data is guaranteed to be made up.

Re: 73% of AI startups are just prompt engineering

#103

Post by less_penguiny »

hn_throwaway_99 wrote: post #87
zkmon wrote: post #81

Earlier quoted context omitted.

He seems real. Goes by Teja K. Seems a startup founder.

He may be real, but the article is fake BS. There is simply no way he'd be in a position to intercept the calls, and he never explains it.

There is nothing difficult about monitoring network traffic in this way for desktop or native apps.

Re: 73% of AI startups are just prompt engineering

#105

Post by theptip »

> just prompt engineering

This dismisses a lot of actual hard work. The scaffolding required to get SOTA performance is non-trivial!

Eg how do you build representative evals and measure forward progress?

Also, tool calling, caching, etc is beyond what folks normally call “prompt engineering”.

If you think it’s trivial though - go build a startup and raise a seed round, the money is easy to come by if you can show results.

Re: 73% of AI startups are just prompt engineering

#106

Post by chmod775 »

hn_throwaway_99 wrote: post #59
analogpixel wrote: post #30

Where is this guy sitting that he is able to collect all of this data? And why is he able to release it all in a blog post? (my company wouldn't allow me to collect and release customer data like this.)

Yeah, TBH my BS detector is going off because this article never explains how he is able to intercept these calls. To be able to call the OpenAI directly from the front end, you'd need to include the OpenAI key, which would be a huge security hole. I don't doubt that many of these companies are just wrappers around the big LLM providers, but they'd be calling the APIs from their backend where nothing should be interc…

> Yeah, TBH my BS detector is going off because this article never explains how he is able to intercept these calls.

You mean, except for explaining what he's doing 4-5 times? He was literally repeating himself restating it. Half the article is about the various indicators he used. THERE'S EXAMPLES OF THEM.

There's this bit:

> Monitored their network traffic for 60-second sessions

> Decompiled and analyzed their JavaScript bundles

Also there's this whole explanation:

> The giveaways when I monitored outbound traffic:

> Requests to api.openai.com every time a user interacted with their "AI"

> Request headers containing OpenAI-Organization identifiers

> Response times matching OpenAI’s API latency patterns (150–400ms for most queries)

> Token usage patterns identical to GPT-4’s pricing tiers

> Characteristic exponential backoff on rate limits (OpenAI’s signature pattern)

Also there's these bits:

> The Methodology (Free on GitHub next week):

> - The complete scraping infrastructure

> - API fingerprinting techniques

> - Response time patterns for every major AI AP

One time he even repeats himself by stating what he's doing as playwright pseudocode, in case plain English isn't enough.

This was also really funny:

> One company’s “revolutionary natural language understanding engine” was literally this: [clientside code with prompt + direct openai API call].

And there's also this bit at the end of the article:

> The truth is just an F12 away.

There's more because LITERALLY HALF THE ARTICLE IS HIM DOING THE THING YOU COMPLAIN HE DIDN'T DO.

In case it's still not clear, he was capturing local traffic while automating with playwright as well as analyzing clientside JS.

Re: 73% of AI startups are just prompt engineering

#108

Post by doe88 »

This a kind of global app store all over again, where all these companies are clients of only few true ai companies and try to distinguish themselves in the bounds of the underlying models and apis, just like apps were trying to find niches in the bound of apis and exposed hw of underlying iphones. Apis versions bugs are now models updates. And of course, all are at the mercy of their respective Leviathan.

Re: 73% of AI startups are just prompt engineering

#109

Post by yalogin »

hotpaper75 wrote: post #71
yalogin wrote: post #62

That is lower than I expected. There are just a handful of companies that create llms. They are all more ir less similar. So all automation is in using them, which is prompt engineering if you see that way. The bigger question is, this is the same story with apps on mobile phones. Apple and google could easily replicate your app if they wanted to and they did too. That danger is much higher with these ai startups. Th…

Do not forget that a product idea needs to meet a certain ROI to be stolen. Big Tech won't go after opportunities that do not generate billion-level revenue. This leaves some room for applications where you can earn decent money.

That is not how companies work. What you said may be true for the immediate short term but over time every team in the company needs to show improvement and set yearly milestones. All these startups will then become functionality they want to push that quarter. Yes it doesn’t mean the death of the startup but a struggle

Re: 73% of AI startups are just prompt engineering

#110

Post by saberience »

kgeist wrote: post #60
drowsspa wrote: post #26

Earlier quoted context omitted.

Is it, though? Apparently the current best practice is just to allow the LLM untethered access to everything and try to control access by preventing prompt injection...

Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to: - write an evaluation pipeline to automate quality testing - add a query rewriting step to explore more options during search - add hybrid BM-25+vector search with proper rank fusion - tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, ho…

Sounds like you vibe coded a RAG system in two weeks, which isn't very hard. Any startup can do it.

I've debugged single difficult bugs before for two weeks, a whole feature that takes two weeks is an easy feature to build.

Post reply on HN