Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using LLMs at Oxide

rfd.shared.oxide.computer

Post reply on HN
Print view View original

91–100 of 284 posts

Re: Using LLMs at Oxide

#91

Post by chrisweekly »

frankest wrote: post #74
zackerydev wrote: post #27

Earlier quoted context omitted.

I remember in the very first class I ever took on Web Design the teacher spent an entire semester teaching "first principles" of HTML, CSS and JavaScript by writing it in Notepad. It was only then did she introduce us to the glory that was Adobe Dreamweaver, which (obviously) increased our productivity tenfold.

DreamWeaver absolutely destroyed the code with all kinds of tags and unnecessary stuff. Especially if you used the visual editor. It was fun for brainstorming but plain notepad with clean understandable code was far far better (and with the browser compatibility issues the only option if you were going to production).

The HTML generated by Dreamweaver's WYSIWYG mode might not have been ideal, but it was far superior to the mess produced by MS Front Page. With Dreamweave, it was at least possible to use it as a starting point.

Re: Using LLMs at Oxide

#92

Post by dachris »

thundergolfer wrote: post #4

A measured, comprehensive, and sensible take. Not surprising from Bryan. This was a nice line: > it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open. I think Oxide didn't include this in the RFD because they exclusively hire senior engineers, but in an organization that contains junior engineers I'd add something specific to help junior engineers understand how they shoul…

For the other non-native speakers wondering, "fly" means your trouser zipper.

He surely has his fly closed when cutting through the hype with reflection and pragmatism (without the extreme positions on both sides often seen).

Re: Using LLMs at Oxide

#93

Post by AlexCoventry »

aatd86 wrote: post #81
pests wrote: post #25

Earlier quoted context omitted.

> That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM. Years ago I had to spend many months building nothing but Models (as in MVC) for a huge data import / ingest the company I worked on was rewriting. It was just messy enough that it couldn't be automated. I almost lost my mind from the dull monotony and started even having attendance is…

The issue is that it might look good but an LLM often inserts weird mistakes. Or ellipses. Or overindex on the training data. If someone is not careful it is easy to completely wreck the codebase by piling on seemingly innocuous commits. So far I have developed a good sense for when I need to push the llm to avoid sloppy code. It is all in the details. But a junior engineer would never find/anticipate those issues. I…

> So it seems to me that we are likely to have worse software over time.

IMO, it's already happening. I had to change some personal information on a bunch of online services recently, and two out of seven of them were down. One of them is still down, a week later. This is the website of a major utilities company. When I call them, they acknowledge that it's down, but say my timing is just bad. That combined with all the recent outages has left me with the impression that software has been getting (even more) unreliable, recently.

Re: Using LLMs at Oxide

#94

Post by sudomateo »

AlexCoventry wrote: post #56

I wonder if they would be willing to publish the "LLMs at Oxide" advice, linked in the OP [1], but currently publicly inaccessible. [1] https://github.com/oxidecomputer/meta/tree/master/engineerin...

Disclaimer: Oxide employee here.

To be honest there's really no secret sauce in there. It's primarily how to get started with agents, when to abandon your context and start anew, and advice on models, monitoring cost, and prompting. This is not to diminish the value of the information as it's good information written by great colleagues. I just wanted to note that most of the information can be obtained from the official AI provider documentation and blog posts from AI boosters like Thorsten Ball.

Re: Using LLMs at Oxide

#95

Post by keeda »

Here's the only simple, universal law that should apply:

THOU SHALT OWN THE CODE THAT THOU DOST RENDER.

All other values should flow from that, regardless of whether the code itself is written by you or AI or by your dog. If you look at the values in the article, they make sense even without LLMs in the picture.

The source of workslop is not AI, it's a lack of ownership. This is especially true for Open Source projects, which are seeing a wave of AI slop PR's precisely because the onus of ownership is largely on the maintainers and not the upstart "contributors."

Note also that this does not imply a universal set of values. Different organizations may well have different values for what ownership of code means -- E.g. in the "move fast, break things" era of FaceBook, workslop may have been perfectly fine for Zuck! (I'd bet it may even have hastened the era of "Move fast with stable infrastructure.") But those values must be consistently applied regardless of how the code came to be.

Re: Using LLMs at Oxide

#96

Post by cobertos »

> LLMs are especially good at evaluating documents to assess the degree that an LLM assisted their creation!)

That's a bold claim. Do they have data to back this up? I'd only have confidence to say this after testing this against multiple LLM outputs, but does this really work for, e.g. the em dash leaderboard of HN or people who tell an LLM to not do these 10 LLM-y writing cliches? I would need to see their reasoning on why they think this to believe.

Re: Using LLMs at Oxide

#97

Post by agentultra »

aatd86 wrote: post #81
pests wrote: post #25

Earlier quoted context omitted.

> That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM. Years ago I had to spend many months building nothing but Models (as in MVC) for a huge data import / ingest the company I worked on was rewriting. It was just messy enough that it couldn't be automated. I almost lost my mind from the dull monotony and started even having attendance is…

The issue is that it might look good but an LLM often inserts weird mistakes. Or ellipses. Or overindex on the training data. If someone is not careful it is easy to completely wreck the codebase by piling on seemingly innocuous commits. So far I have developed a good sense for when I need to push the llm to avoid sloppy code. It is all in the details. But a junior engineer would never find/anticipate those issues. I…

They are trained on code people had to make sacrifices for: deadlines, shortcuts, etc. And code people were simply too ignorant to be writing in the first place. Lots of code with hardly any coding standards.

So of course it’s going to generate code that has non-obvious bugs in it.

Ever play the Undefined Behaviour Game? Humans are bad at being compilers and catching mistakes.

I’d hoped… maybe still do, that the future of programming isn’t a shrug and, “good enough.” I hope we’ll keep developing languages and tools that let us better specify programs and optimize them.

Re: Using LLMs at Oxide

#99

Post by tikhonj »

john01dav wrote: post #11

> it is presumed that of the reader and the writer, it is the writer that has undertaken the greater intellectual exertion. (That is, it is more work to write than to read!) This applies to natural language, but, interestingly, the opposite is true of code (in my experience and that of other people that I've discussed it with).

That's because embarrassingly bad writing is useless, while embarrassingly bad code can still make the computer do (roughly) the right thing and lets you tick off a Jira ticket. So we end up having way more room for awful code than for awful prose.

Reading good code can be a better way to learn about something than reading prose. Writing code like that takes some real skill and insight, just like writing clear explanations.

Re: Using LLMs at Oxide

#100

Post by ghurtado »

zackerydev wrote: post #27
thundergolfer wrote: post #4

A measured, comprehensive, and sensible take. Not surprising from Bryan. This was a nice line: > it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open. I think Oxide didn't include this in the RFD because they exclusively hire senior engineers, but in an organization that contains junior engineers I'd add something specific to help junior engineers understand how they shoul…

I remember in the very first class I ever took on Web Design the teacher spent an entire semester teaching "first principles" of HTML, CSS and JavaScript by writing it in Notepad. It was only then did she introduce us to the glory that was Adobe Dreamweaver, which (obviously) increased our productivity tenfold.

> glory that was Adobe Dreamweaver

Dreamweaver was to web development what ...

I just sat here for 5 minutes and I wasn't able to finish that sentence. So I think that's a statement in itself.

Post reply on HN