Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kotlin creator's new language: talk to LLMs in specs, not English

codespeak.dev

Post reply on HN
Print view View original

231–240 of 304 posts

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#231

Post by siscia »

siscia wrote: post #229
abreslav wrote: post #218

Earlier quoted context omitted.

When you translate spec to tests (if those are traditional unit tests or any automated tests that call the rest of the code), that fixes the API of the code, i.e. the code gets designed implicitly in the test generation step. Is this working well in your experience?

Yes it is passable. Good enough that I don't review it. Granted, it is a personal project that I care only to the point that I want it to work. There are no money on the line. Nothing professional. I believe that part of the secret is that I force CC to run the whole est suites after it change ANY file. Using hooks. It makes iteration slower because it kinda forces it to go from green to green. Or better from red to…

Another trick that I use.

I force the code to be almost 100% dependency injection-able.

It simplifies a lot writing tests and getting the coverage. And I see the LLM being able to handle it very very well.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#232

Post by siscia »

abreslav wrote: post #217
siscia wrote: post #214

What I found more useful is an extra step. Spec to tests, and then red tests to code and green tests. LLMs works on both translation steps. But you end up with an healthy amount of tests. I tagged each tests with the id of the spec so I do get spec to test coverage as well. Beside standard code coverage given by the tests.

Very much agree on coverage. We're actually doing something in that area: https://codespeak.dev/blog/coverage-20260302 For now, it's only about test coverage of the code, but the spec coverage is coming too.

I think you guys are doing pretty much everything right.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#233

Post by WASDx »

lifis wrote: post #38

As far as I can tell it's not a new language, but rather an alternative workflow for LLM-based development along with a tool that implements it. The idea, IIUC, seems to be that instead of directly telling an LLM agent how to change the code, you keep markdown "spec" files describing what the code does and then the "codespeak" tool runs a diff on the spec files and tells the agent to make those changes; then you chec…

I think these limitations could be addressed by allowing trivial manual adjustments to the generated code before committing. And/or allowing for trivial code changes without a spec change. The judgement of "trivial" being that it still follows the spec and does not add functionality mandating a spec change. I haven't checked if they support any of this but I would be frustrated not being allowed to make such a small code change, say to fix an off-by-one error that I recently got from LLM output. The code change would be smaller than the spec change.

Cool idea overall, an incremental psuedocode compiler. Interesting to see how well it scales.

I can also see a hybrid solution with non-specced code files for things where the size of code and spec would be the same, like for enums or mapping tables.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#235

Post by CamperBob2 »

mwarkentin wrote: post #225
CamperBob2 wrote: post #222

Earlier quoted context omitted.

What he misses is that it's much easier to change the spec than the code. And if the cost of regenerating the code is low enough, then the code is not worth talking about.

Is it? If the spec is as detailed as the code would be? If you make a change to one part of the spec do you now have inconsistencies that the LLM is going to have to resolve in some way? Are we going to have a compiler, or type checker type tools for the spec to catch these errors sooner?

It IS a compiler. You might as well ask if the machine-language output of a C compiler is as detailed as the C code was.

To anticipate your objection: you can get over determinism now, or you can get over it later. You will get over it, though, if you intend to stay in this business.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#236

Post by dworks »

the_duke wrote: post #19

This doesn't make too much sense to me. * This isn't a language, it's some tooling to map specs to code and re-generate * Models aren't deterministic - every time you would try to re-apply you'd likely get different output (without feeding the current code into the re-apply and let it just recommend changes) * Models are evolving rapidly, this months flavour of Codex/Sonnet/etc would very likely generate different co…

>I do think there are opportunities in this space, but what I'd like to see is:

>* write text specifications

>* model transforms text into a formal specification

>* then the formal spec is translated into code which can be verified against the spec

This skill does just that: https://github.com/doubleuuser/rlm-workflow

Each stage produces its own output artifact (analysis, implementation plan, implementation summary, etc) and takes the previous phases' outputs as input. The artifact is locked after the stage is done, so there is no drift.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#237

Post by dworks »

the_duke wrote: post #42
davedx wrote: post #35

Earlier quoted context omitted.

My process has organically evolved towards something similar but less strictly defined: - I bootstrap AGENTS.md with my basic way of working and occasionally one or two project specific pieces - I then write a DESIGN.md. How detailed or well specified it is varies from project to project: the other day I wrote a very complete DESIGN.md for a time tracking, invoice management and accounting system I wanted for my free…

I think many have adopted "spec driven development" in the way you describe. I found it works very well in once-off scenarios, but the specs often drift from the implementation. Even if you let the model update the spec at the end, the next few work items will make parts of it obsolete. Maybe that's exactly the goal that "codespeak" is trying to solve, but I'm skeptical this will work well without more formal specifi…

You need to lock the specs and implementation plan and verify the implementation about the previous phase docs.

https://github.com/doubleuuser/rlm-workflow

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#238

Post by jonstaab »

Is it open source? This is a cool idea, but I'm pretty sure it's probably just a thin wrapper around claude. I also couldn't install it on my headless dev box because it relies on a localhost callback. Well, I'm looking forward to the first open source version in about 10 minutes.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#239

Post by James_K »

stephbook wrote: post #221

> It’s one of those things that crackpots keep trying to do, no matter how much you tell them it could never work. If the spec defines precisely what a program will do, with enough detail that it can be used to generate the program itself, this just begs the question: how do you write the spec? Such a complete spec is just as hard to write as the underlying computer program, because just as many details have to be an…

  import Mathlib
  def Goldbach := ∀ x : ℕ, Even x → x > 2 → ∃ (y z: ℕ), Nat.Prime y ∧ Nat.Prime z ∧ x = y + z
A short specification for the proof of the Goldbach conjecture in Lean. Much harder to implement though. Implementation details are always hidden by the interface, which makes it easier to specify than produce. The Curry-Howard correspondence means that Joel's position here is that any question is as hard to ask as answer, and any statement as hard to formulate as it is to prove, which is really just saying that all describable statements are true.

Re: Kotlin creator's new language: talk to LLMs in specs, not English

#240

Post by embedding-shape »

slcjordan wrote: post #230
stephbook wrote: post #221

> It’s one of those things that crackpots keep trying to do, no matter how much you tell them it could never work. If the spec defines precisely what a program will do, with enough detail that it can be used to generate the program itself, this just begs the question: how do you write the spec? Such a complete spec is just as hard to write as the underlying computer program, because just as many details have to be an…

It actually makes sense that code is becoming amorphous and we will no longer scale in terms of building out new features (which has become cheap), but by defining stricter and stricter behavior constraints and structural invariants.

Yeah, "what you're able to build" is no longer one of the most important things, "what you won't build" just became a lot more important.
Post reply on HN