Live data from Hacker News

Trying to Understand Copilot's Type Spaghetti

rtpg.co

41–50 of 89 posts

Re: Trying to Understand Copilot's Type Spaghetti

#41
post #39
post #35

Earlier quoted context omitted.

You can't test the function or endpoint from a business definition "does the right thing" perspective?

You cannot because if you're testing the type level wizardry, the failure case will not compile.

Test your functions and endpoints, nobody cares about the internal wizardry. That's an implementation detail.

There's a reason you coded the endpoint, and often times the business logic required is inelegant, self-contradictory and stupid. That's what has to work.

Re: Trying to Understand Copilot's Type Spaghetti

#42
post #30
post #28

The first thing I see when I look at this is "Where are the unit tests?" This is somewhere in the realm of "clever" or "efficient" code: it looks like it could be written in an easier-to-grok way as dozens of lines of if statements, but I assume there is a reason it wasn't (that is better than "just because"). AI-generated or not, someone or team is responsible for making sure the code/app/service is working for cust…

You can scaffold up "unit tests", but honestly for type-level stuff you are working in a different space entirely. Your types are correctness proofs, so your underlying code is either typed correctly or not. There's not really a middle ground that unit tests catch. Having said that, typescript's soundness issues make it easy to drive a truck through a certain kind of issue, but generally speaking if your type-level p…

That's not really true; I see unit-tests for type-level stuff frequently. Here's some: https://github.com/RuyiLi/cursed-typescript/blob/master/type...

Re: Trying to Understand Copilot's Type Spaghetti

#43

Hi, CopiotKit CEO here (I wrote the original viral tweet). This article is great! Thanks for posting. I'd also written an analysis of the code - including announcing a $1000 prize for the best alternative code: https://ai88.substack.com/p/ceiling-has-been-raised-analyzin... We were going to announce the winner this week but if we get a few more submissions we will definitely consider them. Just submit a PR to https:/…

[deleted]

Re: Trying to Understand Copilot's Type Spaghetti

#44
post #39
post #35

Earlier quoted context omitted.

You can't test the function or endpoint from a business definition "does the right thing" perspective?

You cannot because if you're testing the type level wizardry, the failure case will not compile.

Maybe not checked in, but I've made "tests" before where the code really doesn't assert things, but whether it compiles or not is what I'm after.

It's far too easy in Typescript to do something where you actually lose your type safety without realizing. So I could for instance have some dummy code that calls foo.bar() but crashes in runtime, and my goal is to fix the typing throughout some generic functions so that it catches the mistake compile time.

Re: Trying to Understand Copilot's Type Spaghetti

#45

Hi, CopiotKit CEO here (I wrote the original viral tweet). This article is great! Thanks for posting. I'd also written an analysis of the code - including announcing a $1000 prize for the best alternative code: https://ai88.substack.com/p/ceiling-has-been-raised-analyzin... We were going to announce the winner this week but if we get a few more submissions we will definitely consider them. Just submit a PR to https:/…

That's free training/fine-tuning material, right?

Re: Trying to Understand Copilot's Type Spaghetti

#46
post #13

Earlier quoted context omitted.

> It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible I've seen some legacy code and the kind of spaghetti humans are able to generate - especially with multiple layers of "I'll just throw in something to make feature X work/fix that bug" - is pretty bad already. I honestly doubt code generation will m…

I fear it will be a quantity issue more than a quality one. Why refactor some code to make it extensible, when AI can write a whole new module from scratch? Don't understand the 10 AI written classes? Leave them be and have AI write some more. Once we get AI code deletion and refactoring I guess it will be ok, but generative models are a problem because code is a liability.

I spend most of my time with AI assistants simplifying complex code other humans, including me, have written.

They have made me much more efficient at refactoring spaghetti code and removing unnecessary complexity.

It all depends on how you use them. They can be a cause but also a solution to a problem that affects human codebases too.

Re: Trying to Understand Copilot's Type Spaghetti

#47

from the original tweet linked in the post "ceiling is being raised. cursor's copilot helped us write "superhuman code" for a critical feature. We can read this code, but VERY few engineers out there could write it from scratch." I don't really agree that code is superhuman if VERY few is able to understand it haha..! Code should complex but easy to follow to make it brilliant in my opinion

That’s not the claim. It’s well commented and formatted so actually quite readable. The claim is that very few could write it. Though I would say that ‘very few’ is a larger group than they think - there are plenty of people doing metatype programming in TS; I’ve dabbled enough that given the problem I could probably tackle it and I know I learned from seeing others do it (because I am far from a typescript professio…

I don't think that this is well commented. It explains what each line does, but it does not explain the overall technique being applied here and what it achieves. "Ah, if the parameter is optional I include undefined" is not useful. Instead what this wants is a block comment at the top explaining what this type achieves, how it should be used, and the strategy employed to construct this type.

Re: Trying to Understand Copilot's Type Spaghetti

#48
post #28

The first thing I see when I look at this is "Where are the unit tests?" This is somewhere in the realm of "clever" or "efficient" code: it looks like it could be written in an easier-to-grok way as dozens of lines of if statements, but I assume there is a reason it wasn't (that is better than "just because"). AI-generated or not, someone or team is responsible for making sure the code/app/service is working for cust…

Type level unit tests are indeed super helpful, and in my experience they are easier to write than “real” unit tests, because mocking is trivial.

Re: Trying to Understand Copilot's Type Spaghetti

#49
post #33

Earlier quoted context omitted.

[flagged]

Blog author, yes on being a PLT dork. It was mainly very funny to me how I _immediately_ knew what category of code the tweet's snippet was about. Just instantly knew this was a type spec translator. Seen 10 of them, seen them all.

I have `MaybeRequired` serving almost the exact same purpose! You are right it’s a common problem, a useful approach, and I just wish TS had better support for case analysis than chained ternaries.

Re: Trying to Understand Copilot's Type Spaghetti

#50

Hi, CopiotKit CEO here (I wrote the original viral tweet). This article is great! Thanks for posting. I'd also written an analysis of the code - including announcing a $1000 prize for the best alternative code: https://ai88.substack.com/p/ceiling-has-been-raised-analyzin... We were going to announce the winner this week but if we get a few more submissions we will definitely consider them. Just submit a PR to https:/…

Could you help me understand which Copilots are involved in this?

Your tweet at https://twitter.com/ataiiam/status/1765089261374914957 mentions "Cursor's copilot".

The blog post at https://rtpg.co/2024/03/07/parsing-copilots-type-spaghetti/ talks about GitHub Copilot - did they make a mistake there?

And your product is CopilotKit - is that related to the GitHub and Cursor Copilots in some way or is it something different?

Post reply on HN