Live data from Hacker News

Trying to Understand Copilot's Type Spaghetti

rtpg.co

61–70 of 89 posts

Re: Trying to Understand Copilot's Type Spaghetti

#61
post #54
post #34

Earlier quoted context omitted.

A big enough quantitative change is a qualitative change. There is a big difference between a bad programmer who banged on that code for three days before finally getting it to do what they wanted it to do, and never went back to try to minimize it and clean it up, and that same programmer pushing a prompt into X-GPT and getting that code in five minutes, then moving on to do it again and again and again dozens of ti…

Based on what you're saying, it seems like in the future people will only choose 1, not 2.

It's my current bet but I'm not excited about bounding the capabilities of future non-LLM-based AIs.

Re: Trying to Understand Copilot's Type Spaghetti

#62
When I read the first sentence

> The other day this snippet of Typescript generated by Github copilot was floating around

I was wondering, did that happen on Twitter/X?

And I was not disappointed. That is the place where these people discuss things and take it for granted. Apparently, if you are not using Twitter, you are not part of the conversation.

Re: Trying to Understand Copilot's Type Spaghetti

#63

Earlier quoted context omitted.

That’s just how type programming looks. Here’s ( https://github.com/unional/type-plus/blob/main/packages/type... ) a human written example that’s well-factored and uses loads of subtypes to clarify what it’s doing - but it’s still going to read like the black tongue of Mordor to you if you’re not familiar with how this kind of type stuff is structured and used. And factoring all that stuff out may help readability bu…

The fact that unclear code is common does not make it clear . Single-letter variable names, ternary operators and long run-on lines, like salt, are delicious in small quantities. But when you're writing code so complicated you need multiple comments, within a single statement? I appreciate that you've got to piss with the cock you've got. I've certainly done things with C++ template metaprogramming, with perl, with T…

Can I ask more about the TLA+ spaghetti you had to write?

Re: Trying to Understand Copilot's Type Spaghetti

#64
post #63

Earlier quoted context omitted.

The fact that unclear code is common does not make it clear . Single-letter variable names, ternary operators and long run-on lines, like salt, are delicious in small quantities. But when you're writing code so complicated you need multiple comments, within a single statement? I appreciate that you've got to piss with the cock you've got. I've certainly done things with C++ template metaprogramming, with perl, with T…

Can I ask more about the TLA+ spaghetti you had to write?

In the defence of TLA+ I was a complete newbie.

I mention it only because certain parts of the experience shared the maths-looking big-sequence-of-ands-and-ors one sometimes sees when dealing with complex type systems. TLA+, very sensibly, lets you break up complex statements into multiple smaller statements and give them names.

I was simulating the behaviour of two ends of a communication channel which had been designed with a bidirectional message number/acknowledge/timeout/resend mechanism. And both a hardware and a software state machine on each end managing the message queue in each direction.

The experience was in equal parts brilliant and frustrating. Brilliant because it did reveal faults in the design that probably couldn't have been found any other way. Frustrating because (for example) if you represent your timeout mechanism as a counter on each end of the link and which can only count from 0 to 2 the check will succeed in a few seconds. But if you change the timeout behaviour to count from 0 to 10 the check will take a lot longer. I don't know exactly how long because I aborted the run after a week. And there was no real indication of precisely why it'd stopped completing - I basically had to undo my changes one line at a time until I figured out what was going on.

TLA+ also seemed to have some sort of TeX integration. So a lot of the documentation is written in TeX - it's a fine way of writing math I'm sure. But sometimes the documentation told you how to do something and showed an example statement with a dot, yet you couldn't copy-and-paste the example from the documentation into your code, to express it in your source code you had to use something like \cdot or \circ instead.

Interesting stuff to try out for sure. And impressive given the size of the team that works on it, which I gather is tiny.

Re: Trying to Understand Copilot's Type Spaghetti

#65

ChatGPT gives me garbage code unless I ask it politely not to. No joke. Usually the first attempt is pure garbage and I have to call it out as such and then it’s like, “you’re right! Here’s the updated code”. No idea why it can basically never get it right the first time. I also find that it can be quite redundant and offer two distinct solutions morphed into one mutant answer which will turn the undiscerning 1x deve…

Are you using 4? I've had great luck with 4-Turbo and Opus

Re: Trying to Understand Copilot's Type Spaghetti

#66
post #63

Earlier quoted context omitted.

Can I ask more about the TLA+ spaghetti you had to write?

In the defence of TLA+ I was a complete newbie. I mention it only because certain parts of the experience shared the maths-looking big-sequence-of-ands-and-ors one sometimes sees when dealing with complex type systems. TLA+, very sensibly, lets you break up complex statements into multiple smaller statements and give them names. I was simulating the behaviour of two ends of a communication channel which had been desi…

Good to know, thank you. I write a lot of teaching material for TLA+ and this will be helpful to think about.

Re: Trying to Understand Copilot's Type Spaghetti

#67
post #50

Earlier quoted context omitted.

Could you help me understand which Copilots are involved in this? Your tweet at https://twitter.com/ataiiam/status/1765089261374914957 mentions "Cursor's copilot". The blog post at https://rtpg.co/2024/03/07/parsing-copilots-type-spaghetti/ talks about GitHub Copilot - did they make a mistake there? And your product is CopilotKit - is that related to the GitHub and Cursor Copilots in some way or is it something diffe…

Hah, too many copilots… let me try to clarify: We are building CopilotKit = a framework + platform for building context-aware AI assistants into any application (not necessarily coding related applications). Part of CopilotKit is about giving the Copilot / AI agents access to the application through a typed "inline realtime API". And to make ergonomics great for _our_ users, CopilotKit ships with hardcore type progra…

The people behind Google's chat product naming could really learn a lot from this situation

Re: Trying to Understand Copilot's Type Spaghetti

#68

from the original tweet linked in the post "ceiling is being raised. cursor's copilot helped us write "superhuman code" for a critical feature. We can read this code, but VERY few engineers out there could write it from scratch." I don't really agree that code is superhuman if VERY few is able to understand it haha..! Code should complex but easy to follow to make it brilliant in my opinion

I think Kernighan said something along the lines of "Because debugging code is twice as hard as writing it, only write code half as smart as you are or you'll never be able to fix it later". AI-assisted code generators seems to make this problem much worse as I can now write code 2x, or 3x as smart as I am. What hope will there ever be in debugging this? A more optimistic take is that maybe such tools will let us wri…

What if I told you that one can write code again from scratch instead of fixing broken one?

Writing replaceable code instead of maintainable code seems to be already working for a lot of projects. With LLMs and all that fast computing we have it seems it will be more of replaceable code in the future.

Of course there are always projects where it will never work.

It already works for infrastructure as nowadays servers are not fixed and treated like important things but you spin up fresh one.

Re: Trying to Understand Copilot's Type Spaghetti

#69
post #68

Earlier quoted context omitted.

I think Kernighan said something along the lines of "Because debugging code is twice as hard as writing it, only write code half as smart as you are or you'll never be able to fix it later". AI-assisted code generators seems to make this problem much worse as I can now write code 2x, or 3x as smart as I am. What hope will there ever be in debugging this? A more optimistic take is that maybe such tools will let us wri…

What if I told you that one can write code again from scratch instead of fixing broken one? Writing replaceable code instead of maintainable code seems to be already working for a lot of projects. With LLMs and all that fast computing we have it seems it will be more of replaceable code in the future. Of course there are always projects where it will never work. It already works for infrastructure as nowadays servers…

> Writing replaceable code instead of maintainable code seems to be already working for a lot of projects.

I contend that these two concepts aren't different. If you have the ability to easily replace a small part of the code and have everything still work, then that's very maintainable. Unless you are talking about throwing out the whole codebase and replacing it, which for sufficiently complex codebases will inevitably lead to the second system effect. If replacing the whole thing is easy, it was probably not that valuable or complex to begin with, but that's not the kind of code where maintainability is paramount.

> It already works for infrastructure as nowadays servers are not fixed and treated like important things but you spin up fresh one.

For that metaphor to work, the programmer would have to be patching the binary output instead of fixing the source code and recompiling.

Re: Trying to Understand Copilot's Type Spaghetti

#70
post #68

Earlier quoted context omitted.

I think Kernighan said something along the lines of "Because debugging code is twice as hard as writing it, only write code half as smart as you are or you'll never be able to fix it later". AI-assisted code generators seems to make this problem much worse as I can now write code 2x, or 3x as smart as I am. What hope will there ever be in debugging this? A more optimistic take is that maybe such tools will let us wri…

What if I told you that one can write code again from scratch instead of fixing broken one? Writing replaceable code instead of maintainable code seems to be already working for a lot of projects. With LLMs and all that fast computing we have it seems it will be more of replaceable code in the future. Of course there are always projects where it will never work. It already works for infrastructure as nowadays servers…

You spin up a fresh server from a cookie cutter image that lets you create a practically unlimited number of identical servers.

That's not even remotely like how rewriting code works. Rewriting code is more comparable to what spinning up a fresh server was like a quarter century ago. So, back in the days where they were important things because spinning up a new one was an unholy PITA and literally never went off without a hitch.

Post reply on HN