Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

Post reply on HN
Print view View original

141–150 of 631 posts

Re: The last six months in LLMs in five minutes

#141

Post by asdff »

angled wrote: post #115
RobinL wrote: post #101

Earlier quoted context omitted.

Interesting. I don't have to use PowerPoint much, but I hate it when I do. I don't want the llm to write the words but I do want it to make things look nice. So does this work well now?

My pipeline for this is vscode + prompts + markdown templates + GitHub copilot -> markdown docs -> pandoc to produce.docx -> copilot in word for “nice” formatting -> copilot in ppt for nice decks. LLMs all the way down. I find it’s easier to version control and diff the .md artefacts, those remain my authoritative source.

Wow. Seems like a headache compared to how I make slides the old fashioned way: copy and paste my figures into blank powerpoint.

Re: The last six months in LLMs in five minutes

#142

Post by stepbeek »

OakNinja wrote: post #123
Timwi wrote: post #103

Earlier quoted context omitted.

> Coding is just one of the task[s] you do in your job But it's by far the most fun part and the only reason to take such a job...

To me, LLM's free up time for me so that I can spend time on the fun parts of coding. Less boilerplate, more focus on the interesting problems. This is no different from using high level languages. The problem domain is less around memory management and garbage collection and closer to the problem you're actually trying to solve.

I agree with this. I feel like there’s a false dichotomy right now in a lot of these discussions where one can only vibe code or only code by hand. It is possible to do both…

Re: The last six months in LLMs in five minutes

#143

Post by piokoch »

Quothling wrote: post #132
LZ_Khan wrote: post #57

I'm curious how the 6 months have looked from a non-programmer's perspective. What kind of co-working tools and similar optimizations have people from other fields experienced?

I think Claude Cowork through the Microsoft thing which was copilot but is now named M365 (or something?) is likely creating every powerpoint resentation within our organisation at this point. We have whatever AI is in teams transcribe every meeting, and it's scaringly good at it. It's also extremely good at sumerizing or finding things from pervious meetings when tasked. One disadvantage in this, is that I can see h…

"I have no idea why it's so hard for people to pick up the Librechat tool we're given access to through our equity fund"

That's because M365 is integrated with the whole Office/Exchange environment, especially in terms of security policies, etc. MS also guarantee that the data are private, this is very important for many companies both from the IP protection perspective and the liability to expose some users/customers data (think of GDPR regulations is Europe).

I don't know who is behind Liberchat, probably some good and friendly folks, but when it comes to privacy/security Microsoft has much more to loose and if shit happens it is easier to sue them than some random VC-financed company from the USA.

Re: The last six months in LLMs in five minutes

#144

Post by emil-lp »

pineapple_opus wrote: post #135

All I see is mention of how various models generate image of "pelican riding bicycle(s)"

Yes, the "pelican riding a bicycle" is the ultimate test of not understanding how LLMs work.

Well, a combination of that and believing that replication of test data is a good measure of progress.

Re: The last six months in LLMs in five minutes

#146

Post by Hfuffzehn »

rTX5CMRXIfFG wrote: post #12

Am I crazy, or are these differences between the best models so marginal that you’d get roughly the same performance if you use the same high-quality harness (ie preloaded instructions from md files, including custom skills)?

You have correctly identified that getting a "high-quality harness (ie preloaded instructions from md files, including custom skills)" is the (or at least a) hard part.

Because you have to adjust the harness to your problem space and provide that so you can say it is high-quality.

Many people will stop that discussion at the claude code vs. codex vs. opencode level and then merge that with discussing model performance.

And that is also why "Generate an SVG of a pelican riding a bicycle" is still a benchmark worth discussing. Because at least it is a defined problem space.

Re: The last six months in LLMs in five minutes

#147

Post by Mashimo »

koonsolo wrote: post #108
simonw wrote: post #80

Earlier quoted context omitted.

Ditching the QA team when the single highest challenge is verifying that vibe-coded systems do what they're meant to is extraordinarily short-sighted. Personally, the more time I spend working with coding agents the least worried I am for my career. Getting the best results out of them is really hard . They amplify existing skills and experience, so the more experience you have the better.

Have you seen the automated tests that QA members deliver? My experience is that they are horrible, and it's not so hard to beat that low quality bar with an LLM. I have a theory: if they were good at writing automated tests, they would have been developers instead of QA engineers. Not saying that there aren't any high quality QA engineers, I worked with some. But LLM's raised the bar in a way that most QA engineers…

Huh, never thought about QA writing unit tests.

In my limited experience they write test cases, test each story, do regression test, verify bugs from customers. All by hand.

At my current job I don't want to miss them.

Re: The last six months in LLMs in five minutes

#148

Post by piva00 »

viccis wrote: post #44
kvakkefly wrote: post #30

Earlier quoted context omitted.

I remember this very clearly myself. Before opus 4.5, I was doing a lot of hand holding and was coding a lot myself, but I have not written code since that day more or less. I did write some stuff myself just to learn how the enigma encryption machine worked, so wrote myself to learn. But professionally, I stopped coding in November.

How do you justify your salary given that you're just using a tool that any of us could use for $20 an hour in your role?

I don't think you understand how programming as a job works, writing code is the final output of the process but it's not the job in itself.

Re: The last six months in LLMs in five minutes

#149

Post by schnitzelstoat »

BOOSTERHIDROGEN wrote: post #112
alexwwang wrote: post #109

Earlier quoted context omitted.

As a former data scientist, I started to use code agent 3 monthes ago. Before that, I use chat completion on web. Now, I nearly do everything which outputs documents with code agent.

Can you give a sanitized example or a hypothetical scenario of what you mean by “output documents with code agents”? Thanks.

I’m not him, but I’ve started using them to do the analysis (SQL, Python etc.) and then output the report as Quarto HTML which can be hosted on GitHub Pages. It works well for this analysis style work.

Once I was going to send some figures to leadership so I checked the queries myself and not only had it done it correctly, but it had also included a lot of sanity checks with other places in the database which as a human I doubt I’d have had the time or inclination to do.

Even for modelling work it can be good to check your ETL queries, or write one itself and then check it etc.

Post reply on HN