Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agentic coding notes

danluu.com

Post reply on HN
Print view View original

61–70 of 99 posts

Re: Agentic coding notes

#61

Post by stalfie »

embedding-shape wrote: post #37
bob1029 wrote: post #10

A lot of the crazy ideas seem to have melted away in the face of massive context sizes. Today, I can put roughly a megabyte of utf8 text into my system prompt before things start to get weird. That is a massive amount of information even if we are being sloppy with it. You can read The Hobbit and the first Harry Potter book cover-to-cover and still have room to spare. I would deeply struggle to develop a world model…

> melted away in the face of massive context sizes If only. There is a huge difference between "Gives good responses/can easily spot things within N context size" and "Technically works but sucks within N context size", almost all models basically become cave-people once you go beyond 50% of the "supported" context size, meaning while they may technically work with 1 million output tokens, those last 500K tokens are…

There's at least one benchmark that attempts to measure this, but it has been running for a year plus so it's quite infrequently updated now.

https://fiction.live/stories/Fiction-liveBench-Mar-25-2025/o...

Re: Agentic coding notes

#62

Post by rtpg »

duckmysick wrote: post #38

I'd like to highlight a different part of the article: > In general, when I talk to software folks about testing, I'm coming from such a different place that they immediately look at me like I'm an alien, so let's talk about how we tested at this hardware company I worked for, Centaur, which informs my biases about how I like to work. Some of the things that we did that were or are unorthodox in the software world ar…

I really wonder what "randomixed testing" looks like in practice. What is the measure of success/failure?

I undrestand for fuzzing you have a very basic "doesn't crash" metric. Property based tests.... you gotta write properties for the PBTs to work on. What is the randomized testing hitting?

Re: Agentic coding notes

#63

Post by cognitiveinline »

layer8 wrote: post #57
cognitiveinline wrote: post #54

It's really coming down to "Do we want to subscribe to a human with a salary of many ~$10000(s) ", or "Do we spend 100(s)$ on an AI subscription" Even with it's issues, the latest models are going to disrupt the labor economics.

On the other hand, you have many more humans to choose from than models, and they don’t change their character every few months.

There's a lot people will do for equal quality and faster work at a 100th the cost. Like being OK with character changing and adapting to it. You are not seeing this in your workplace?

Re: Agentic coding notes

#64

Post by kordlessagain »

duckmysick wrote: post #38

I'd like to highlight a different part of the article: > In general, when I talk to software folks about testing, I'm coming from such a different place that they immediately look at me like I'm an alien, so let's talk about how we tested at this hardware company I worked for, Centaur, which informs my biases about how I like to work. Some of the things that we did that were or are unorthodox in the software world ar…

I bake MCP tools into everything now, including doing screenshots. Any LLM can run just about every function, including resizing the window. I just watched Fabel 5 do a full usability test on a new project, copying the release cycle for my agentic terminal, relaunching the app like 20 times as it went, ensuring the move to a built and signed release was working. It installed the program like 5 times (something I do daily multiple times).

I noticed map tiles were not working and started to tell it, but then all of a sudden they reappeared and checking the logs it had found the issue and autocorrected itself.

The key here is feedback loops and systems annealing.

As for Dan, my God I love this guy. Glad someone posted it!

Re: Agentic coding notes

#66

Post by kordlessagain »

rtpg wrote: post #62
duckmysick wrote: post #38

I'd like to highlight a different part of the article: > In general, when I talk to software folks about testing, I'm coming from such a different place that they immediately look at me like I'm an alien, so let's talk about how we tested at this hardware company I worked for, Centaur, which informs my biases about how I like to work. Some of the things that we did that were or are unorthodox in the software world ar…

I really wonder what "randomixed testing" looks like in practice. What is the measure of success/failure? I undrestand for fuzzing you have a very basic "doesn't crash" metric. Property based tests.... you gotta write properties for the PBTs to work on. What is the randomized testing hitting?

Dogfood everything, all the time.

Re: Agentic coding notes

#67

Post by disgruntledphd2 »

kordlessagain wrote: post #64
duckmysick wrote: post #38

I'd like to highlight a different part of the article: > In general, when I talk to software folks about testing, I'm coming from such a different place that they immediately look at me like I'm an alien, so let's talk about how we tested at this hardware company I worked for, Centaur, which informs my biases about how I like to work. Some of the things that we did that were or are unorthodox in the software world ar…

I bake MCP tools into everything now, including doing screenshots. Any LLM can run just about every function, including resizing the window. I just watched Fabel 5 do a full usability test on a new project, copying the release cycle for my agentic terminal, relaunching the app like 20 times as it went, ensuring the move to a built and signed release was working. It installed the program like 5 times (something I do d…

I feel like Dan is one of the most consistently interesting writers in tech at the moment.

This is most likely because we take similar approaches towards things.

Re: Agentic coding notes

#68

Post by fragmede »

mattlondon wrote: post #53
progbits wrote: post #32

Earlier quoted context omitted.

Realizing he's just using it to mean remote place in terms of AI bubble (Vancouver! What does that make all the other places that are not major tech hubs?) was a bummer. Who cares about AI, I wanted to read about living in Galapagos

I visited a decade or so ago. Most of the islands you are forbidden from staying overnight on. No running water. No power. No phone signal. You're going to need a big big solar panel to run local inference during the day, but luckily there is plenty of sun!

Looks like there is Starlink though! https://dplnews.com/cnt-ecuador-y-starlink-conectan-las-4-is...

Re: Agentic coding notes

#70

Post by m_m_carvalho »

kordlessagain wrote: post #64
duckmysick wrote: post #38

I'd like to highlight a different part of the article: > In general, when I talk to software folks about testing, I'm coming from such a different place that they immediately look at me like I'm an alien, so let's talk about how we tested at this hardware company I worked for, Centaur, which informs my biases about how I like to work. Some of the things that we did that were or are unorthodox in the software world ar…

I bake MCP tools into everything now, including doing screenshots. Any LLM can run just about every function, including resizing the window. I just watched Fabel 5 do a full usability test on a new project, copying the release cycle for my agentic terminal, relaunching the app like 20 times as it went, ensuring the move to a built and signed release was working. It installed the program like 5 times (something I do d…

[flagged]
Post reply on HN