Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using LLMs at Oxide

rfd.shared.oxide.computer

Post reply on HN
Print view View original

141–150 of 284 posts

Re: Using LLMs at Oxide

#141

Post by _joel »

BobbyTables2 wrote: post #110
frankest wrote: post #74

Earlier quoted context omitted.

DreamWeaver absolutely destroyed the code with all kinds of tags and unnecessary stuff. Especially if you used the visual editor. It was fun for brainstorming but plain notepad with clean understandable code was far far better (and with the browser compatibility issues the only option if you were going to production).

MS FrontPage also went out of its way to do the same.

It might have been pretty horrible but I hold Frontpage 97 with fond memories, it started my IT career, although not for HTML reasons.

The _vti_cnf dir left /etc/passwd downloadable, so I grabbed it from my school website. One Jack the Ripper later and the password was found.

I told the teacher resposible for the IT it was insecure and that ended up getting me some work experience. Ended up working the summer (waiting for my GCSE results) for ICL which immeasurably helped me when it was time to properly start working.

Did think about defacing, often wonder that things could have turned out very much differently!

Re: Using LLMs at Oxide

#142

Post by riffraff »

ghurtado wrote: post #100
zackerydev wrote: post #27

Earlier quoted context omitted.

I remember in the very first class I ever took on Web Design the teacher spent an entire semester teaching "first principles" of HTML, CSS and JavaScript by writing it in Notepad. It was only then did she introduce us to the glory that was Adobe Dreamweaver, which (obviously) increased our productivity tenfold.

> glory that was Adobe Dreamweaver Dreamweaver was to web development what ... I just sat here for 5 minutes and I wasn't able to finish that sentence. So I think that's a statement in itself.

..VB6 was to windows dev?

People with very little competence could and did get things done, but it was a mess underneath.

Re: Using LLMs at Oxide

#143

Post by elAhmo »

cobertos wrote: post #96

> LLMs are especially good at evaluating documents to assess the degree that an LLM assisted their creation!) That's a bold claim. Do they have data to back this up? I'd only have confidence to say this after testing this against multiple LLM outputs, but does this really work for, e.g. the em dash leaderboard of HN or people who tell an LLM to not do these 10 LLM-y writing cliches? I would need to see their reasonin…

I would be surprised they have any data about this. There are so many ways LLMs can be involved, from writing everything, to making text more concise or just "simple proofreading". Detecting all this with certainty is not trivial and probably not possible with the current tools we have.

Re: Using LLMs at Oxide

#144

Post by torben-friis »

gpm wrote: post #63
torben-friis wrote: post #33

The guide is generally very well thought, but I see an issue in this part: It sets the rule that things must be actually read when there’s a social expectation (code interviews for example) but otherwise… remarks that use of LLMs to assist comprehension has little downside. I find two problems with this: - there is incoherence there. If LLMs are flawless in reading and summarization, there is no difference with readi…

> Just yesterday I asked for a comparison of three technical books on a similar topic, and it wrongly guessed the third one rather than follow the three links. I would consider this a failure in their tool use capabilities, not their reading ones. To use them to read things (without relying on their much less reliable tool use) take the thing and put it in the context window yourself. They still aren't perfect of cou…

>Three whole books likely exceeds their context window size of course

This was not “read all three books”, this was “check these three links with the (known) book synopsis/reviews there” and it made up the third one.

>I would consider this a failure in their tool use capabilities, not their reading ones.

Id give it to you if I got an error message, but the text being enhanced with wrong-but-plausible data is clearly a failure of reliability.

Re: Using LLMs at Oxide

#145

Post by philippta »

> LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it.

To extend that: If the LLM is the author and the responsible engineer is the genuine first reviewer, do you need a second engineer at all?

Typically in my experience one review is enough.

Re: Using LLMs at Oxide

#146

Post by discreteevent »

keyle wrote: post #44
thundergolfer wrote: post #4

A measured, comprehensive, and sensible take. Not surprising from Bryan. This was a nice line: > it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open. I think Oxide didn't include this in the RFD because they exclusively hire senior engineers, but in an organization that contains junior engineers I'd add something specific to help junior engineers understand how they shoul…

> That junior engineer possibly hasn't programmed without the tantalizing, even desperately tempting option to be assisted by an LLM. This gives me somewhat of a knee jerk reaction. When I started programming professionally in the 90s, the internet came of age and I remember being told "in my days, we had books and we remembered things" which of course is hilarious because today you can't possibly retain ALL the know…

> "in my days, we had books and we remembered things" which of course is hilarious because today you can't possibly retain ALL the knowledge needed to be software engineer

Reading books was never about knowledge. It was about knowhow. You didn't need to read all the books. Just some. I don't know how many developers I met who would keep asking questions that would be obvious to anyone who had read the book. They never got the big picture and just wasted everyone's time, including their own.

"To know everything, you must first know one thing."

Re: Using LLMs at Oxide

#147

Post by menaerus »

girvo wrote: post #113
keyle wrote: post #89

Earlier quoted context omitted.

> Watching my juniors constantly fight the nonsense auto completion suggestions their LLM editor of choice put in front of them, or worse watching them accept it and proceed to get entirely lost in the sauce, I’m not entirely convinced that the autocompletion part of it is the best one. That's not an LLM problem, they'd do the same thing 10 years ago with stack overflow: argue about which answer is best, or trust the…

No, it is qualitatively different because it happens in-line and much faster. If it’s not correct (which it seems it usually isn’t), they spend more time removing whatever garbage it autocompleted.

People do it with the autocomplete as well so I guess there's not that much of a difference wrt LLMs. It likely depends on the language but people who are inexperienced in C++ would be over-relying on autocomplete to the point that it looks hilarious, if you have a chance to sit next to them helping to debug something for example.

Re: Using LLMs at Oxide

#148

Post by adastra22 »

fastball wrote: post #124
csb6 wrote: post #102

Strange to see no mention of potential copyright violations found in LLM-generated code (e.g. LLMs reproducing code from Github verbatim without respecting the license). I would think that would be a pretty important consideration for any software development company, especially one that produces so much free software.

Has anything like this worked its way through the courts yet?

Yes, training is considered fair use, and output is non-copyrightable / public domain. With many asterix and footnotes, of course.

Re: Using LLMs at Oxide

#149

Post by yard2010 »

cobertos wrote: post #96

> LLMs are especially good at evaluating documents to assess the degree that an LLM assisted their creation!) That's a bold claim. Do they have data to back this up? I'd only have confidence to say this after testing this against multiple LLM outputs, but does this really work for, e.g. the em dash leaderboard of HN or people who tell an LLM to not do these 10 LLM-y writing cliches? I would need to see their reasonin…

I thought about it - a quick way to verify whether something was created with LLM is to feed an LLM half of the text and then let it complete token by token. Every completion, check not just for the next token but the next n-probable tokens. If one of them is the one you have in the text, pick it and continue. This way, I think, you can identify how much the model is "correct" by predicting the text it hasn't yet seen.

I didn't test it and I'm far from an expert, maybe someone can challenge it?

Re: Using LLMs at Oxide

#150

Post by klauserc »

ec109685 wrote: post #76
john01dav wrote: post #20

> Wherever LLM-generated code is used, it becomes the responsibility of the engineer. As part of this process of taking responsibility, self-review becomes essential: LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it. Moreover, once in the loop of peer review, generation should more or less be removed: if code review comments are addressed by wholesale re-g…

Don’t make manual corrections. If you keep all edits to be driven by the LLM, you can use that knowledge later in the session or ask your model to commit the guidelines to long term memory.

The best way to get an LLM to follow style is to make sure that this style is evident in the codebase. Excessive instructions (whether through memories or AGENT.md) do not help as much.

Personally, I absolutely hate instructing agents to make corrections. It's like pushing a wet noodle. If there is lots to correct, fix one or two cases manually and tell the LLM to follow that pattern.

https://www.humanlayer.dev/blog/writing-a-good-claude-md

Post reply on HN