Live data from Hacker News

Trying to Understand Copilot's Type Spaghetti

rtpg.co

71–80 of 89 posts

Re: Trying to Understand Copilot's Type Spaghetti

#71
So, that code gets me thinking about premature optimization. In the "verschlimmbessern born of optimizing the wrong thing because you didn't use measurement to guide your efforts" sense.

The rule of thumb I hear is that, in a mature product, reading and maintaining code takes about 10 times as much effort as writing it in the first place. I've never tried to measure this myself, but it doesn't seem to be wildly off from what I see at work, so let's go with it. Let's also assume, for the sake of argument, that these AI tools double the productivity of people writing new code. Or, equivalently, they halve the time it takes to write it. (This is a lot more than what I see people typically claim, but I'm trying to steelman and also it makes the math easier.)

Anyway, this would imply that, speaking purely in terms of raw productivity (not, say, security or correctness) the AI coding assistant is a net win when the code that's written using it is less than 5% more difficult to maintain.

And I'm inclined to think that, if Copilot & friends are enabling more people to write more code that looks like what we see in this article (also not wildly off from what I see at work), then it's hard to see how they could possibly be making codebases where they are used less than 5% more expensive to maintain.

Re: Trying to Understand Copilot's Type Spaghetti

#72
post #50

Earlier quoted context omitted.

Could you help me understand which Copilots are involved in this? Your tweet at https://twitter.com/ataiiam/status/1765089261374914957 mentions "Cursor's copilot". The blog post at https://rtpg.co/2024/03/07/parsing-copilots-type-spaghetti/ talks about GitHub Copilot - did they make a mistake there? And your product is CopilotKit - is that related to the GitHub and Cursor Copilots in some way or is it something diffe…

Hah, too many copilots… let me try to clarify: We are building CopilotKit = a framework + platform for building context-aware AI assistants into any application (not necessarily coding related applications). Part of CopilotKit is about giving the Copilot / AI agents access to the application through a typed "inline realtime API". And to make ergonomics great for _our_ users, CopilotKit ships with hardcore type progra…

I'm going to make a completely fresh alternative written from the ground up. When I'm done you can all find it easily because it's called CoPilot, shouldn't be hard to find.

Re: Trying to Understand Copilot's Type Spaghetti

#73

Earlier quoted context omitted.

Hah, too many copilots… let me try to clarify: We are building CopilotKit = a framework + platform for building context-aware AI assistants into any application (not necessarily coding related applications). Part of CopilotKit is about giving the Copilot / AI agents access to the application through a typed "inline realtime API". And to make ergonomics great for _our_ users, CopilotKit ships with hardcore type progra…

The people behind Google's chat product naming could really learn a lot from this situation

Indeed. For example, they could learn that Copilot is a winning name so lets go with that.

Re: Trying to Understand Copilot's Type Spaghetti

#74

Earlier quoted context omitted.

> It’s well commented and formatted so actually quite readable. Are you looking at the same code I'm looking at? The first block of code from the article? I've seen more readable code in Perl Golf competitions.

That’s just how type programming looks. Here’s ( https://github.com/unional/type-plus/blob/main/packages/type... ) a human written example that’s well-factored and uses loads of subtypes to clarify what it’s doing - but it’s still going to read like the black tongue of Mordor to you if you’re not familiar with how this kind of type stuff is structured and used. And factoring all that stuff out may help readability bu…

> That’s just how type programming looks.

Then that is an argument to do less of it. Or at least to back off and learn to write better types that are not a mess.

Re: Trying to Understand Copilot's Type Spaghetti

#75

Hi, CopiotKit CEO here (I wrote the original viral tweet). This article is great! Thanks for posting. I'd also written an analysis of the code - including announcing a $1000 prize for the best alternative code: https://ai88.substack.com/p/ceiling-has-been-raised-analyzin... We were going to announce the winner this week but if we get a few more submissions we will definitely consider them. Just submit a PR to https:/…

You love swift ??

Re: Trying to Understand Copilot's Type Spaghetti

#77
post #15
post #7

Earlier quoted context omitted.

Even an average programmer can understand it if they bothered to read the TypeScript documentation. “Superhuman” in this case means “has read the manual”.

That's also how you become an extreme outlier at Bash. Actually read the manual.

I read hundreds of the funny chat logs!

Re: Trying to Understand Copilot's Type Spaghetti

#78
Why's these seen as being difficilt to write? It's a giant switch statement that recurses. This is less indicative of AI coming a long way and more of programmers never working on a program that stores types as data, this being the most common and rote pattern that exists.

Re: Trying to Understand Copilot's Type Spaghetti

#79

So, that code gets me thinking about premature optimization. In the "verschlimmbessern born of optimizing the wrong thing because you didn't use measurement to guide your efforts" sense. The rule of thumb I hear is that, in a mature product, reading and maintaining code takes about 10 times as much effort as writing it in the first place. I've never tried to measure this myself, but it doesn't seem to be wildly off f…

Can you elaborate on this part? I am trying to understand.

> in a mature product, reading and maintaining code takes about 10 times as much effort as writing it in the first place

Wouldn't that imply that rewriting a mature codebase from scratch would take ten times less effort than reading and maintaining it?

I only saw two instances where (different) management approved a complete rewrite of a working product and in both cases it turned out to be a disaster, because it is easy to severely underestimate how much effort it will take to match the features, quality and performance of the old codebase. I suspect it is almost universally better to refactor the old codebase incrementally.

Based on that, I take that you mean something else and I didn't get your point.

Re: Trying to Understand Copilot's Type Spaghetti

#80

ChatGPT gives me garbage code unless I ask it politely not to. No joke. Usually the first attempt is pure garbage and I have to call it out as such and then it’s like, “you’re right! Here’s the updated code”. No idea why it can basically never get it right the first time. I also find that it can be quite redundant and offer two distinct solutions morphed into one mutant answer which will turn the undiscerning 1x deve…

maybe ChatGPT crawls stackoverflow, stackoverflow #1 answer old or just garbage, second answer better but not the upvoted,

ChatGPT gives you #1 basically (with some small tweaks but still it is garbage)

you tell it that's garbage

it sees in comments "why is this upvoted answer, answer two is clearly better"

it returns answer two with a few tweaks.

Post reply on HN