Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

151–160 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#151

Post by simonw »

timr wrote: post #147
simonw wrote: post #87

Earlier quoted context omitted.

"Relevant content to your LLM when asking a question" is last year's RAG. If you look at how sophisticated current LLM systems work there is so much more to this. Just one example: Microsoft open sourced VS Code Copilot Chat today (MIT license). Their prompts are dynamically assembled with tool instructions for various tools based on whether or not they are enabled: https://github.com/microsoft/vscode-copilot-chat/bl…

I get what you're saying, but the parent is correct -- most of this stuff is pretty obvious if you spend even an hour thinking about the problem. For example, while the specifics of the prompts you're highlighting are unique to Copilot, I've basically implemented the same ideas on a project I've been working on, because it was clear from the limitations of these models that sooner rather than later it was going to be…

most of this stuff is pretty obvious if you spend even an hour thinking about the problem

I don't think that's true.

Even if it is true, there's a big difference between "thinking about the problem" and spending months (or even years) iteratively testing out different potential prompting patterns and figuring out which are most effective for a given application.

I was hoping "prompt engineering" would mean that.

Re: The new skill in AI is not prompting, it's context engineering

#152

Post by coderatlarge »

bgwalter wrote: post #139

These discussions increasingly remind me of gamers discussing various strategies in WoW or similar. Purportedly working strategies found by trial and error and discussed in a language that is only intelligible to the in-group (because no one else is interested). We are entering a new era of gamification of programming, where the power users force their imaginary strategies on innocent people by selling them to the eq…

i tend to share your view. but then your comment describes a lot of previous cycles of enterprise software selling. it’s just that this time is reaching a little uncomfortably into the builder’s /developer’s traditional areas of influence/control/workflow. how devs feel now is probably how others (ex csr, qa, sre) felt in the past when their managers pushed whatever tooling/practice was becoming popular or sine qua non in previous “waves”.

Re: The new skill in AI is not prompting, it's context engineering

#153

Post by GuinansEyebrows »

simonw wrote: post #46
outofpaper wrote: post #33

Earlier quoted context omitted.

They might be good reads on the topic but Drew makes some significant etymological mistakes. For example loadout doesn't come from gaming but military terminology. It's essentially the same as kit or gear.

Drew isn't using that term in a military context, he's using it in a gaming context. He defines what he means very clearly: > The term “loadout” is a gaming term that refers to the specific combination of abilities, weapons, and equipment you select before a level, match, or round. In the military you don't select your abilities before entering a level.

i think that software engineers using this terminology might be envisioning themselves as generals, not infantry :)

Re: The new skill in AI is not prompting, it's context engineering

#154

Post by benreesman »

The new skill is programming, same as the old skill. To the extent these things are comprehensible, you understand them by writing programs: programs that train them, programs that run inferenve, programs that analyze their behavior. You get the most out of LLMs by knowing how they work in detail.

I had one view of what these things were and how they work, and a bunch of outcomes attached to that. And then I spent a bunch of time training language models in various ways and doing other related upstream and downstream work, and I had a different set of beliefs and outcomes attached to it. The second set of outcomes is much preferable.

I know people really want there to be some different answer, but it remains the case that mastering a programming tool involves implemtenting such, to one degree or another. I've only done medium sophistication ML programming, and my understand is therefore kinda medium, but like compilers, even doing a medium one is the difference between getting good results from a high complexity one and guessing.

Go train an LLM! How do you think Karpathy figured it out? The answer is on his blog!

Re: The new skill in AI is not prompting, it's context engineering

#155

Post by GuinansEyebrows »

mgdev wrote: post #77

If we zoom out far enough, and start to put more and more under the execution umbrella of AI, what we're actually describing here is... product development. You are constructing the set of context, policies, directed attention toward some intentional end, same as it ever was. The difference is you need fewer meat bags to do it, even as your projects get larger and larger. To me this is wholly encouraging. Some projec…

i just hope that, along with imagination, humans can have an economy that supports this shift.

Re: The new skill in AI is not prompting, it's context engineering

#156

Post by pyman »

mgdev wrote: post #77

If we zoom out far enough, and start to put more and more under the execution umbrella of AI, what we're actually describing here is... product development. You are constructing the set of context, policies, directed attention toward some intentional end, same as it ever was. The difference is you need fewer meat bags to do it, even as your projects get larger and larger. To me this is wholly encouraging. Some projec…

That does sound a lot like the role of a software architect. You're setting the direction, defining the constraints, making trade-offs, and stitching different parts together into a working system

Re: The new skill in AI is not prompting, it's context engineering

#157

Post by nyrikki »

storus wrote: post #85
simonw wrote: post #51

Earlier quoted context omitted.

Can you link to the research on millions of different terms and stable long contexts? I haven't come across that yet.

You can look at AnyTool, 2024 (16,000 tools) and start looking at newer research from there. https://arxiv.org/abs/2402.04253 For long contexts start with activation beacons and RoPE scaling.

Thanks for the link. It finally explained why I was getting hit up by recruiters for a job that was for a data broker looking to do what seemed like silly uses.

Cloud API recommender systems must seem like a gift to that industry.

Not my area anyways but I couldn't see a profit model for a human search for an API when what they wanted is well covered by most core libraries in Python etc...

Re: The new skill in AI is not prompting, it's context engineering

#159

Post by timr »

simonw wrote: post #151
timr wrote: post #147

Earlier quoted context omitted.

I get what you're saying, but the parent is correct -- most of this stuff is pretty obvious if you spend even an hour thinking about the problem. For example, while the specifics of the prompts you're highlighting are unique to Copilot, I've basically implemented the same ideas on a project I've been working on, because it was clear from the limitations of these models that sooner rather than later it was going to be…

most of this stuff is pretty obvious if you spend even an hour thinking about the problem I don't think that's true. Even if it is true, there's a big difference between "thinking about the problem" and spending months (or even years) iteratively testing out different potential prompting patterns and figuring out which are most effective for a given application. I was hoping "prompt engineering" would mean that .

>I don't think that's true.

OK, well...maybe I should spend my days writing long blogposts about the next ten things that I know I have to implement, then, and I'll be an AI thought-leader too. Certainly more lucrative than actually doing the work.

Because that's literally what's happening -- I find myself implementing (or having implemented) these trendy ideas. I don't think I'm doing anything special. It certainly isn't taking years, and I'm doing it without reading all of these long posts (mostly because it's kind of obvious).

Again, it very much reminds me of the early days of the web, except there's a lot more people who are just hype-beasting every little development. Linus is over there quietly resolving SMP deadlocks, and some influencer just wrote 10,000 words on how databases are faster if you use indexes.

Re: The new skill in AI is not prompting, it's context engineering

#160

Post by simonw »

coderatlarge wrote: post #158
slavapestov wrote: post #65

I feel like if the first link in your post is a tweet from a tech CEO the rest is unlikely to be insightful.

i don’t disagree with your main point, but is karpathy a tech ceo right now?

I think they meant Tobi Lutke, CEO of Shopify: https://twitter.com/tobi/status/1935533422589399127
Post reply on HN