Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

73% of AI startups are just prompt engineering

pub.towardsai.net

Post reply on HN
Print view View original

51–60 of 212 posts

Re: 73% of AI startups are just prompt engineering

#51

Post by muppetman »

jmogly wrote: post #35
analogpixel wrote: post #30

Where is this guy sitting that he is able to collect all of this data? And why is he able to release it all in a blog post? (my company wouldn't allow me to collect and release customer data like this.)

Im also wondering how he is able to see calls to AI providers directly in the browser, client side api calls? Thats strange to me. Also how is he able to peer into the rag architectures? I don’t get that, maybe GpT4.1 allows unauthenticated requests? Is there an OAuth setup that allows client side requests to OpenAI?

Yea I just posted a similar comment. I'm sure some websites just skin OpenAI/Claude etc, but ALL of them? It makes no sense.

Re: 73% of AI startups are just prompt engineering

#52

Post by octoberfranklin »

amelius wrote: post #25
asah wrote: post #19

Earlier quoted context omitted.

-1: there's lots of "kingdoms" (openai, anthropic, google, plus open source) - if one king comes for your castle, you can move in minutes.

True, even OpenAI built their castle in nVidia's kingdom. And nVidia built their castle in TSMC's kingdom. And TSMC built their castle in ASML's kingdom.

TSMC bought a huge chunk of ASML's shares before taking the plunge on EUV -- enough to get them a board seat.

Re: 73% of AI startups are just prompt engineering

#53

Post by goostavos »

lomase wrote: post #34
aurareturn wrote: post #6

Prompt engineering isn't as simple as writing prompts in english. It's still engineering data flow, when data is relevant, systems that the AI can access and search, tools that the AI can use, etc.

Imagine you are a top of the line engenier... Engineering data flow... sure, we all like to use big words.

The new 10x engineering is writing "please don't write bugs" in a markdown file.

Re: 73% of AI startups are just prompt engineering

#54

Post by octoberfranklin »

scottiebarnes wrote: post #36
benoau wrote: post #21

Earlier quoted context omitted.

It's worse than that, someone else's models, someone else's smartphone operating systems, it's every conceivable disadvantage.

Every city should have its own municipal chip fabrication plant!

If you break up AT&T the Bell System will collapse!

Re: 73% of AI startups are just prompt engineering

#55

Post by zach_moore »

The reason is because VC needs to show that their flagship investments have "traction" so they manufacture ecosystem interest by funding and encouraging ecosystem product usage. It's a small price to pay. If someone builds a wrapper that gets 100 business users then token use on the foundation layer gets that passed down. Big scheme.

Re: 73% of AI startups are just prompt engineering

#56

Post by Zanfa »

muppetman wrote: post #48

This makes no sense to me? I don't understand why a company, even if it is using GPT or Claude as their true backend, is going to leave API calls in Javascript that anyone can find. Sure maybe a couple would, but 73% of those tested? Surely your browser is going to talk to their webserver, and yup sure it'll then go off and use Claude etc then return the answer to you, but surely they're not all going to just skin an…

It makes perfect sense when you consider that the average Javascript developer does not know that business logic can exist outside of React components.

Re: 73% of AI startups are just prompt engineering

#57

Post by teiferer »

analogpixel wrote: post #30

Where is this guy sitting that he is able to collect all of this data? And why is he able to release it all in a blog post? (my company wouldn't allow me to collect and release customer data like this.)

Another red flag with the article is that the author's LinkedIn profile link at the bottom leads to a non-existing page.

Is Teja Kusireddy a real person? Or is this maybe just an experiment from some AI company (or other actor) to see how far they can push it? A Google search by that name doesn't find anything not related to the article.

The article should be flagged. Otoh, this should get discussed.

Re: 73% of AI startups are just prompt engineering

#58

Post by bradfa »

RC_ITR wrote: post #17

One of the biggest problems frontier models will face going forward is how many tasks require expertise that cannot be achieved through Internet-scale pre-training. Any reasonably informed person realizes that most AI start-ups looking to solve this are not trying to create their own pre-trained models from scratch (they will almost always lose to the hyperscale models). A pragmatic person realizes that they're not f…

Got any links to explanations of why fine tuning open models isn’t a productive solution? Besides renting the GPU time, what other downsides exist on today’s SOTA open models for doing this?

Re: 73% of AI startups are just prompt engineering

#59

Post by hn_throwaway_99 »

analogpixel wrote: post #30

Where is this guy sitting that he is able to collect all of this data? And why is he able to release it all in a blog post? (my company wouldn't allow me to collect and release customer data like this.)

Yeah, TBH my BS detector is going off because this article never explains how he is able to intercept these calls.

To be able to call the OpenAI directly from the front end, you'd need to include the OpenAI key, which would be a huge security hole. I don't doubt that many of these companies are just wrappers around the big LLM providers, but they'd be calling the APIs from their backend where nothing should be interceptable. And sure, I believe a few of them are dumb enough to call OpenAI from the frontend, but that would be a minority.

This whole thing smells fishy, and I call BS unless the author provides more details about how he intercepted the calls.

Re: 73% of AI startups are just prompt engineering

#60

Post by kgeist »

drowsspa wrote: post #26
aurareturn wrote: post #6

Prompt engineering isn't as simple as writing prompts in english. It's still engineering data flow, when data is relevant, systems that the AI can access and search, tools that the AI can use, etc.

Is it, though? Apparently the current best practice is just to allow the LLM untethered access to everything and try to control access by preventing prompt injection...

Well it took me 2 full-time weeks to properly implement a RAG-based system so that it found actually relevant data and did not hallucinate. Had to:

- write an evaluation pipeline to automate quality testing

- add a query rewriting step to explore more options during search

- add hybrid BM-25+vector search with proper rank fusion

- tune all the hyperparameters for best results (like weight bias for bm25 vs. vector, how many documents to retrieve for analysis, how to chunk documents based on semantics)

- parallelize the search pipeline to decrease wait times

- add moderation

- add a reranker to find best candidates

- add background embedding calculation of user documents

- lots of failure cases to iron out so that the prompt worked for most cases

There's no "just give LLM all the data", it's more complex than that, especially if you want best results and also full control of data (we run all of that using open source models because user data is under NDA)

Post reply on HN