341–350 of 376 posts
I am developing my own programming language, but I have no specification written for it. When people tell me that I need a specification, I reply that I already have one - the source code of the language compiler.
A programming language is not the compiler. A programming language is, in fact, not software.
If you think of yourself as a "programmer" or "coder" I think it's fair to say you might have incorrectly defined your role already. Your job is to design and implement software solutions that are reliable and compliant, and make business processes more efficient. Writing code should really be a relatively small piece of the puzzle, and arguably if you're spending a lot of time writing code, you're wasting time. Writ…
Earlier quoted context omitted.
Why is this a weird definition of determinism? Could you please define what you mean when you say deterministic? A C program does not identify a single assembly program. It identifies a set of assembly programs. This fits the pretty standard definition of non-determinism. A difference between natural language and C code is that natural language does not have a formal semantics. Having no formal semantics is a very di…
Determinism implies that the same input will result in the same output. I agree with you that for a single C program there’s a set of assembly code that satisfies it. But by choosing a compiler, an architecture and a set of flags,… you will always get the same assembly code. If you decide to randomize them, then you can no longer guarantee a specific result, but you can still guarantee sets of result. Which is the de…
Hypothetically, if we could guarantee a semantic relationship between the input and output we wouldn’t care if the LLM was deterministic. For instance, if I give the LLM a lean theorem and it instantiates a program and a mechanical proof that the program conforms to the lean theorem, I just don’t care about determinism. Edit: this is equivalent to me not caring very much about which particular conformant C compiler I pick
And my understanding of LLMs is that they actually are functions and the observed randomness is an artifact of how we use them. If you had the weights and the hardware, you could run the frontier models deterministically. But I don’t think you’d be satisfied even if you could do that. Edit: this is maybe analogous to picking a particular C compiler that does not promise conformance
There are valid concerns with LLMs but I’m not convinced non-determinism is the thing we should care about.
Earlier quoted context omitted.
Which is not getting better. I'd pay you 10€ for a TODO app that improved my life meaningfully. It would obviously need to have great UX and be stable. Those are table stakes. I don't have the time to look at all these apps though. If somebody tells me they made a great TODO app, I'm already mentally filtering them out. There's just too much noise here. Does your TODO app solve any meaningful problem beyond the bare…
> It would obviously need to have great UX There is the problem. Todo apps are easy to make. However making one that is actually useful for tracking todo items it hard. Getting the todo into the app is harder than writing it on paper. Getting reminders at a useful time is hard (now is not a great time to fix that broken widget - it needs parts not in the budget, I'm at work, it needs a couple hours of dedicated time…
Of course I also use the AI to refine the spec, but again not to 100% detail, only to ensure I truly cover all that matters to me.
Earlier quoted context omitted.
I've been working with coding LLMs for almost a year. Here's what I've found works best: - Do a brainstorming session with the LLM about your idea. Flesh out the major points of the product, who the stakeholders are, what their motivations and goals are, what their pain points are. Research potential competitors. Find out what people are saying about them, especially the complaints. - Build a high level design docume…
You are basically discovered working in a team. Even that it is an inferior version of that. I have always done that steps with my team and the results are great. If you are a solo developer I understand that the LLM can help somewhat but not replace a real team of developers.
Natural language is fluid and ambiguous while code is rigid and deterministic. Spec-driven development appears to be the best of both worlds. But really, it is the worst of both. LLMs are language models - their breakthrough capability is handling natural language. Code is meant to be unambiguous and deterministic. A spec is neither fluid nor deterministic.
> code is rigid and deterministic I really wish people would start defining what they mean when they say code is deterministic. For instance, code is not deterministic in the sense that it admits a single interpretation when compiled. C code admits many assembly interpretations and the assembly you get will vary depending on your compiler and parameters. When people say code is deterministic I think they mean it has…
There's your own example defining how code is deterministic. Despite being compiled to different assembly interpretations, the code does the same thing.
Confusing the *how* and the *what* is very common when discussing specifications, in my experience. Programmers gravitate toward pseudocode when they have trouble articulating a functional requirement.
> Specifications were never meant to be time-saving devices.
Correct. Anyone selling specifications as a way to save time does not understand the purpose of a specification. Unfortunately, neither does the article's author. The article is based on a false premise.
LLMs experience the same problems as humans when provided with underspecified requirements. That's a specification problem.
[1]: https://en.wikipedia.org/wiki/Software_requirements_specific...