Earlier quoted context omitted.
We will and soon because it does not have to be deterministic like a compiler. It only has to pass all tests.
Who is writing the tests?
In the future users will write the tests
Kotlin creator's new language: talk to LLMs in specs, not English
191–200 of 304 posts
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#192Earlier quoted context omitted.
Functional requirements are known knowns. Out of bounds behavior is sometimes a known unknown, but in the era of generated code is exclusively unknown unknowns. Good luck speccing out all the unanticipated side effects and undefined behaviors. Perhaps you can prompt the agent in a loop a bnumber of times but it's hard to believe that the brute-force throw-more-tokens-at-it approach has the same level of return as a m…
Are you as a developer 100% able to trust that you didn’t miss anything? Your team if you are a team lead who delegates tasks to other developers? If you outsource non business things like Salesforce integrations etc do you know all of the code they wrote? Your library dependencies? Your infrastructure providers?
It seems like ^ and ^^ agree to me. Am I missing something?
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#193Earlier quoted context omitted.
> Eventually, we'll end up in a world where humans don't need to touch code, but we are not there yet. Will we though? Wouldn't AI need to reach a stage where it is a tool, like a compiler, which is 100% deterministic?
Compiler is not 100% deterministic. Its output can change when you upgrade its version, its output can change when you change optimization options. Using profile-guided optimization can also change between runs.
[deleted]
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#194This doesn't make too much sense to me. * This isn't a language, it's some tooling to map specs to code and re-generate * Models aren't deterministic - every time you would try to re-apply you'd likely get different output (without feeding the current code into the re-apply and let it just recommend changes) * Models are evolving rapidly, this months flavour of Codex/Sonnet/etc would very likely generate different co…
I think your objections miss the point. My informal specs to a program are user -focused. I want to dictate what benefits the program will give to the person who is using it, which may include requirements for a transport layer, a philosophy of user interaction, or any number of things. When I know what I want out of a program, I go through the agony of translating that into a spec with database schemas, menu options…
I tried this recently with what I thought was a simple layout, but probably uncommon for CSS. It took an extremely long back and forth to nail it down. It seemingly had no understanding how to achieve what I wanted. A couple sentences would have been clear to a person. Sometimes LLMs are fantastic and sometimes they are brain dead.
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#195> codespeak login
Instant tab close!
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#196As far as I can tell it's not a new language, but rather an alternative workflow for LLM-based development along with a tool that implements it. The idea, IIUC, seems to be that instead of directly telling an LLM agent how to change the code, you keep markdown "spec" files describing what the code does and then the "codespeak" tool runs a diff on the spec files and tells the agent to make those changes; then you chec…
> The limitation seems to be that you can't modify the code yourself if you want the spec to reflect it Eventually, we'll end up in a world where humans don't need to touch code, but we are not there yet. We are looking into ways to "catch up" the specs with whatever changes happen in the code not through CodeSpeak (agents or manual changes or whatever). It's an interesting exercise. In the case of agents, it's very…
Why are we eliminating our own job and maybe hobby so eagerly? Whatever. It is done.
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#197Earlier quoted context omitted.
Let's rephrase: Since nobody involved actually cares whether the code works or not, it doesn't matter whether it's a different wrong thing each time.
You got it completely backwards. The claim is that if the code does exactly what the spec says (which generated tests are supposed to "prove") then the actual code does not matter, even if it's different each time.
The point they are making is the tests are neither necessary nor sufficient alone to prove the code does exactly what the spec says. Looking at the tests isn't enough to prove anything; as an extreme example, if no one involved looks at the code, then the tests can just be static always passing and you wouldn't know either way whether or not the code matches the spec or not.
If anyone cared enough they could look at the code and see the problem immediately and with little effort, but we're encouraging a world where no one cares enough to put even that baseline effort because *gestures at* the tests are passing. Who cares how wrong the code is and in what ways if all the lights are green?
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#198> codespeak login Instant tab close!
Huh? I don't see a login. All the code examples and set up instructions are available to anyone visiting the page.
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#199Feels like writing tests before writing code, but with LLMs :)
Re: Kotlin creator's new language: talk to LLMs in specs, not English
#200This doesn't make too much sense to me. * This isn't a language, it's some tooling to map specs to code and re-generate * Models aren't deterministic - every time you would try to re-apply you'd likely get different output (without feeding the current code into the re-apply and let it just recommend changes) * Models are evolving rapidly, this months flavour of Codex/Sonnet/etc would very likely generate different co…
I think your objections miss the point. My informal specs to a program are user -focused. I want to dictate what benefits the program will give to the person who is using it, which may include requirements for a transport layer, a philosophy of user interaction, or any number of things. When I know what I want out of a program, I go through the agony of translating that into a spec with database schemas, menu options…
[flagged]