Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The new skill in AI is not prompting, it's context engineering

philschmid.de

Post reply on HN
Print view View original

541–550 of 550 posts

Re: The new skill in AI is not prompting, it's context engineering

#541

Post by buffzebra »

__loam wrote: post #358
Turskarama wrote: post #294

Earlier quoted context omitted.

Both the context and the prompt are just part of the same input. To the model there is no difference, the only difference is the way the user feeds that input to the model. You could in theory feed the context into the model as one huge prompt.

Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.

“It’s all just tokens in the context window” = “it’s all just fundamental particles,” I think. True, but reductive. Seems key that dude is talking about agentic AI not just chat. I’d revisit the email example in the post.

Re: The new skill in AI is not prompting, it's context engineering

#542

Post by rvz »

CamperBob2 wrote: post #511
rvz wrote: post #499

Earlier quoted context omitted.

> 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. Again, that isn't a reason to never check or write tests for your code because an "AI-generated it" or even assuming that an AI will detect all of them. In fact, it means you NEED to do more reviewing, checking and testing than ever before. > It's not likely that any code that you and I run every day is. S…

Yes, I'm very sure. 99.9999% of the code you are running is not formally proven to be correct, and was not generated by a compiler whose output was formally proven to be correct. Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?

> Yes, I'm very sure.

You do understand that you are proving my entire point? It is still not a reason to *NOT* test or check your code implementation at all or to only rely on an LLM to check it for you.

What it really means is that software testing is extremely more important.

For running formally verified code every day, seL4 runs on the iPhone's security chip (secure enclave) in the hands of billions of users and it is a formally verified microkernel used for cryptographic operations from payments to disk encryption everyday.

This kernel is also used on medical devices, cars and in defense equipment, relied on by hundreds of millions of users.

> Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?

Lots of decades to know that no process developing safety critical system software would allow AI-generated code that isn't checked by a human or is only checked by other LLMs and using that as a substitute to writing tests.

Re: The new skill in AI is not prompting, it's context engineering

#543

Post by 0xdeafcafe »

furyofantares wrote: post #123
root_axis wrote: post #59

Earlier quoted context omitted.

I am not a fan of this banal trend of superficially comparing aspects of machine learning to humans. It doesn't provide any insight and is hardly ever accurate.

I've seen a lot of cases where, if you look at the context you're giving the model and imagine giving it to a human (just not yourself or your coworker, someone who doesn't already know what you're trying to achieve - think mechanical turk), the human would be unlikely to give the output you want. Context is often incomplete, unclear, contradictory, or just contains too much distracting information. Those are all thi…

Totally agree. We've found that a lot of "agent failures" trace back to assumptions, bad agent-decisions, or bloat buried in the context, stuff that makes perfect sense to the dev who built it when following the happy path, but can so easily fall apart in real-world scenarios.

We've been working on a way to test this more systematically by simulating full conversations with agents and surfacing the exact point where things go off the rails. Kind of like unit tests, but for context, behavior, and other ai jank.

Full disclosure, I work at the company building this, but the core library is open source, free to use, etc. https://github.com/langwatch/scenario

Re: The new skill in AI is not prompting, it's context engineering

#544

Post by FeepingCreature »

simonw wrote: post #401
FeepingCreature wrote: post #387

Earlier quoted context omitted.

There's always been a distinction between prompt and data.

LLM's can't distinguish between instruction prompts and data prompts - that's why prompt injection attacks exist.

I agree, and that's a problem. It doesn't mean the distinction doesn't exist, in fact it shows the opposite.

Re: The new skill in AI is not prompting, it's context engineering

#545

Post by kiitos »

ZYbCRq22HbJ2y7 wrote: post #83
storus wrote: post #48

Earlier quoted context omitted.

Those issues are considered artifacts of the current crop of LLMs in academic circles; there is already research allowing LLMs to use millions of different tools at the same time, and stable long contexts, likely reducing the amount of agents to one for most use cases outside interfacing different providers. Anyone basing their future agentic systems on current LLMs would likely face LangChain fate - built for GPT-3,…

How would "a million different tool calls at the same time" work? For instance, MCP is HTTP based, even at low latency in incredibly parallel environments that would take forever.

HTTP is an implementation detail, and doesn't represent any kind of unavoidable bottleneck vs. any other transport protocol one might use to do these kinds of request/response interactions.

Re: The new skill in AI is not prompting, it's context engineering

#546

Post by grumple »

simonw wrote: post #375
grumple wrote: post #346

After a recent conversation here, I spent a few weeks using agents. These agents are just as disappointing as what we had before. Except now I waste more time getting bad results, though I’m really impressed by how these agents manage to fuck things up. My new way of using them is to just go back to writing all the code myself. It’s less of a headache.

Which definition of "agents" are you using there, and which ones did you try?

Cursor, Copilot agent mode, and Windsurf. The agent modes can search repos, modify code, and run code on their own. I thought Cursor's agent was the best. I did like Windsurf's agent plans, but the actual results weren't good. Copilot's agent has been slow and not good. But basically all of them were like a pretty bad junior engineer - sometimes it would hit the right result, but usually not. The code often looked good but rarely even ran, let alone met requirements. They would frequently break things, I'd fix them, they'd break them again. Most of the time this cycle was slower and more frustrating than just writing the code myself. I tried one or two one-shots on lovable - the design was impressive, but functionality and attention to specs were poor.

I've had the most success with extremely small questions rather than asking the agents to write a lot of code. In those cases, the code is still usually wrong, but it's close or small enough that I can quickly fix it.

Don't get me wrong: I find all of these tools to be really impressive and good enough to be useful. But the improvements and huge productivity gains friends claim they or their workers are getting just aren't materializing for me.

Re: The new skill in AI is not prompting, it's context engineering

#547

Post by CamperBob2 »

fumblingness wrote: post #531
CamperBob2 wrote: post #58

Earlier quoted context omitted.

Not really. Got some code you don't understand? Feed it to a model and ask it to add comments. Ultimately humans will never need to look at most AI-generated code, any more than we have to look at the machine language emitted by a C compiler. We're a long way from that state of affairs -- as anyone who struggled with code-generation bugs in the first few generations of compilers will agree -- but we'll get there.

"And at no point does it ever occur to you to demand proof that measures such as this will have the desired effect... or, indeed, that the desired effect is indeed worth achieving at all." - you ( https://news.ycombinator.com/item?id=44439447 )

(Shrug) There's a difference between prescription and prediction. I predict that after 50 years of doing the same old shit the same old way, the practice of programming is about to undergo a series of wrenching changes that amount to nothing less than revolution. Changes powered by radical new insights into the nature and function of language itself.

I'm not initiating these changes, voting for them, or attempting to persuade other people to do so, as the person I replied to in the other thread is doing. I do welcome having something new and interesting to learn and think about, though.

Anyway, always good to hear from a new fan!

Re: The new skill in AI is not prompting, it's context engineering

#549

Post by [deleted] »

v3ss0n wrote: post #334
JohnMakin wrote: post #10

> Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. Ok, I can buy this > It is about the engineering of context and providing the right information and tools, in the right format, at the right time. when the "right" format and "right" time are essentially, and maybe even necessarily, undefined, then aren't you still reaching for a "magic" solution? If the definit…

At this point , due to non-deterministic nature and hallucination context engineering is pretty much magic. But here are our findings. 1 - LLM Tends to pick up and understand contexts that comes at top 7-12 lines.Mostly first 1k token is best understood by llms ( tested on Claude and several opensource models ) so - most important contexts like parsing rules need to be placed there. 2 - Need to keep context short . W…

[deleted]

Re: The new skill in AI is not prompting, it's context engineering

#550

Post by mpazik »

geeewhy wrote: post #104

ive beeen experimenting with this for a while, (im sure in a way, most of us did). Would be good to numerate some examples. When it comes to coding, here's a few: - compile scripts that can grep / compile list of your relevant files as files of interest - make temp symlinks in relevant repos to each other for documentation generation, pass each documentation collected from respective repos to to enable cross-repo ops…

Yeah, I had similar findings. Making good system can be tedious though when it grows and current tools did not provide enough configuration.

I found opencode, and decided to build a fork that allows to setup specialized agent with custom tools and programmatic context building to organize it better https://github.com/mpazik/openagent

Post reply on HN