Earlier quoted context omitted.
After 25 or so years doing this, I think there are two kinds of developers: craftsmen and practical “does it get the job done” types. I’m the former. The latter seem to be what makes the world go round.
If you've been doing it for that long (about as long as I have), then surely you remember all the times you had to clean up after the "git 'er done" types. I'm not saying they don't have their place, but without us they would still be making the world go round. Only backwards.
Using LLMs at Oxide
191–200 of 284 posts
Re: Using LLMs at Oxide
#192A measured, comprehensive, and sensible take. Not surprising from Bryan. This was a nice line: > it’s just embarrassing — it’s as if the writer is walking around with their intellectual fly open. I think Oxide didn't include this in the RFD because they exclusively hire senior engineers, but in an organization that contains junior engineers I'd add something specific to help junior engineers understand how they shoul…
For the other non-native speakers wondering, "fly" means your trouser zipper. He surely has his fly closed when cutting through the hype with reflection and pragmatism (without the extreme positions on both sides often seen).
Re: Using LLMs at Oxide
#193Re: Using LLMs at Oxide
#194> LLM-generated code should not be reviewed by others if the responsible engineer has not themselves reviewed it. To extend that: If the LLM is the author and the responsible engineer is the genuine first reviewer, do you need a second engineer at all? Typically in my experience one review is enough.
yes, obviously? anyone who is doing serious enough engineering that they have the rule of "one human writes, one human reviews" wants two humans to actually put careful thought in to a thing, and only one of them is deeply incentivised to just commit the code. your suggestion means less review and worse incentives.
Re: Using LLMs at Oxide
#195I know I'm walking into a den of wolves here and will probably get buried in downvotes, but I have to disagree with the idea that using LLMs for writing breaks some social contract. If you hand me a financial report, I expect you used Excel or a calculator. I don't feel cheated that you didn't do long division by hand to prove your understanding. Writing is no different. The value isn't in how much you sweated while…
Re: Using LLMs at Oxide
#196I know I'm walking into a den of wolves here and will probably get buried in downvotes, but I have to disagree with the idea that using LLMs for writing breaks some social contract. If you hand me a financial report, I expect you used Excel or a calculator. I don't feel cheated that you didn't do long division by hand to prove your understanding. Writing is no different. The value isn't in how much you sweated while…
Totally agree. The output is what matters. At this point, who really cares what the person who sees everything as "AI slop" thinks? I would rather just interact with Gemini anyway. I don't need to read/listen to the "AI slop hunter" regurgitate their social media feed and NY Times headlines back to me like a bad language model.
Re: Using LLMs at Oxide
#197Earlier quoted context omitted.
For the same quality and quantity output, if the cost of using LLMs + the cost of careful oversight is less than the cost of not using LLMs then the rational choice is to use them. Naturally this doesn’t factor in things like human obsolescence, motivation and self-worth.
It seems like this would be a really interesting field to research. Does AI assisted coding result in fewer bugs, or more bugs, vs an unassisted human? I've been thinking about this as I do AoC with Copilot enabled. It's been nice for those "hmm how do I do that in $LANGUAGE again?" moments, but it's also wrote some nice looking snippets that don't do quite what I want it to. And many cases of "hmmm... that would wor…
Re: Using LLMs at Oxide
#198I had trouble getting past the Early Modern English tinge of the language used in this. It’s fun, but it distracts from the comprehension in attempt to just sound epic. It’s fine if you’re writing literature, but it comes off sounding uppity in a practical doc for devs. Writing is not just about conveying something in a mood you wish to set. Study how Richard Feynman and Warren Buffett communicated to their audiences…
Using Large Language Models (LLMs) at Oxide
This document explains how we should think about using LLMs (like ChatGPT or similar tools) at Oxide.
What are LLMs?
LLMs are very advanced computer programs that can understand and generate text. They've become a big deal in the last five years and can change how we work. But, like any powerful tool, they have good and bad sides. They are very flexible, so it’s hard to give strict rules about how to use them. Still, because they are changing so fast, we need to think carefully about when and how we use them at Oxide.
What is Important When Using LLMs
We believe using LLMs should follow our core values:
Responsibility:
We are responsible for the work we produce. Even if we use an LLM to help, a human must make the final decisions. The person using the LLM is responsible for what comes out.
Rigor (Care and Precision):
LLMs can help us think better or find mistakes, but if we use them carelessly, they can cause confusion. We should use them to improve our work, not to cut corners.
Empathy:
Remember, real people read and write what we produce. We should be kind and respectful in our language, whether we are writing ourselves or letting an LLM help.
Teamwork:
We work as a team. Using LLMs should not break trust among team members. If we tell others we used an LLM, it might seem like we’re avoiding responsibility, which can hurt trust.
Urgency (Doing Things Quickly):
LLMs can help us work faster, but we shouldn’t rush so much that we forget responsibility, care, and teamwork. Speed is good, but not at the cost of quality and trust.
How We Use LLMs
LLMs can be used in many ways. Here are some common uses:
1. As Readers
LLMs are great at quickly understanding documents, summaries, or answering questions about texts.
Important: When sharing documents with an LLM, make sure your data is private. Also, remember that uploading files might allow the LLM to learn from your data unless you turn that off.
Note: Use LLMs to help understand documents, but don’t skip reading them yourself. LLMs are tools, not replacements for reading carefully.
2. As Editors
LLMs can give helpful feedback on writing, especially after you’ve written a draft. They can suggest improvements in structure and wording.
Caution: Sometimes, LLMs may flatter your work too much or change your style if used too early. Use them after you’ve done some work yourself.
3. As Writers
LLMs can write text, but their writing can be basic or obvious. Sometimes, they produce text that shows it was made by a machine.
Why be careful? If readers see that the writing is from an LLM, they might think the author didn’t put in enough effort or don’t truly understand the ideas.
Our rule: Usually don’t let LLMs write your final drafts. Use them to help, but own your words and ideas.
4. As Code Reviewers
LLMs can review code and find problems, but they can also miss issues or give bad advice. Use them as a helper, not a replacement for human review.
5. As Debuggers
LLMs can sometimes help find solutions to tricky problems. They might give helpful hints. But don’t rely on them too much—use them as a second opinion.
6. As Programmers
LLMs are very good at writing code, especially simple or experimental code. They can be useful for quick tasks like writing tests or prototypes.
Important: When an LLM writes code, the person responsible must review it carefully. Responsibility for the code stays with the human.
Teamwork: If you use an LLM to generate code, make sure you understand and review it yourself first.
How to Use LLMs Properly
There are detailed guidelines and tips in the internal document called "LLMs at Oxide."
In general:
Using LLMs is encouraged, but always remember your responsibilities—to your product, your customers, and your team.
Re: Using LLMs at Oxide
#199Earlier quoted context omitted.
After 25 or so years doing this, I think there are two kinds of developers: craftsmen and practical “does it get the job done” types. I’m the former. The latter seem to be what makes the world go round.
I am both, I own a small agency when I have to be practical, and have fun crafting code on the hobby side. I think what craftsmen miss is the different goals. Projects fall on a spectrum from long lived app that constantly evolve with a huge team working on it to not opened again after release. In the latter, like movie or music production (or most video games), only the end result matters, the how is not part of the…
Re: Using LLMs at Oxide
#200Earlier quoted context omitted.
...what a remarkable thread.
Right? If this is really true, that some random folk without compiler engineering experience, implemented a completely new feature in ocaml compiler by prompting the LLM to produce the code for him, then I think it really is remarkable.
It seems more like a non experienced guy asked the LLM to implement something and the LLM just output what and experienced guy did before, and it even gave him the credit