Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Trying to Understand Copilot's Type Spaghetti

rtpg.co

Post reply on HN
Print view View original

21–30 of 89 posts

Re: Trying to Understand Copilot's Type Spaghetti

#21

Post by swiftlyTyped »

swiftlyTyped wrote: post #17

Hi, CopiotKit CEO here (I wrote the original viral tweet). This article is great! Thanks for posting. I'd also written an analysis of the code - including announcing a $1000 prize for the best alternative code: https://ai88.substack.com/p/ceiling-has-been-raised-analyzin... We were going to announce the winner this week but if we get a few more submissions we will definitely consider them. Just submit a PR to https:/…

By the way - we practice what we preach- Copilots raise the bar (or the ceiling...) on human productivity - in every domain. Which is why we're building infrastructure to make building copilots easier...

What ideas do you think are still missing from today's Copilots?

I.e. suppose we were looking back at today's Copilots 5 years from now- besides better models, what else has changed?

Re: Trying to Understand Copilot's Type Spaghetti

#22

Post by Octoth0rpe »

Sebb767 wrote: post #13
tossandthrow wrote: post #12

The future is going to be a horrible place for people taking over legacy code-bases. It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible, Sometimes you digress to simpler types not because you couldn't write more complex types, but because you realise that the next person who needs to make changes to t…

> It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible I've seen some legacy code and the kind of spaghetti humans are able to generate - especially with multiple layers of "I'll just throw in something to make feature X work/fix that bug" - is pretty bad already. I honestly doubt code generation will m…

I think the difference is that such incidents were previously the exception in a codebase, in the future they will be the rule.

Re: Trying to Understand Copilot's Type Spaghetti

#23

Post by nicklecompte »

jameshart wrote: post #19
blueappconfig wrote: post #4

from the original tweet linked in the post "ceiling is being raised. cursor's copilot helped us write "superhuman code" for a critical feature. We can read this code, but VERY few engineers out there could write it from scratch." I don't really agree that code is superhuman if VERY few is able to understand it haha..! Code should complex but easy to follow to make it brilliant in my opinion

That’s not the claim. It’s well commented and formatted so actually quite readable. The claim is that very few could write it. Though I would say that ‘very few’ is a larger group than they think - there are plenty of people doing metatype programming in TS; I’ve dabbled enough that given the problem I could probably tackle it and I know I learned from seeing others do it (because I am far from a typescript professio…

[flagged]

Re: Trying to Understand Copilot's Type Spaghetti

#24

Post by Octoth0rpe »

blueappconfig wrote: post #4

from the original tweet linked in the post "ceiling is being raised. cursor's copilot helped us write "superhuman code" for a critical feature. We can read this code, but VERY few engineers out there could write it from scratch." I don't really agree that code is superhuman if VERY few is able to understand it haha..! Code should complex but easy to follow to make it brilliant in my opinion

I think Kernighan said something along the lines of "Because debugging code is twice as hard as writing it, only write code half as smart as you are or you'll never be able to fix it later". AI-assisted code generators seems to make this problem much worse as I can now write code 2x, or 3x as smart as I am. What hope will there ever be in debugging this?

A more optimistic take is that maybe such tools will let us write competent code in languages we do NOT specialize in, and in the future either a more competent version of ourselves or some actual expert can fix it if it breaks? That doesn't sound a whole lot better :/

Re: Trying to Understand Copilot's Type Spaghetti

#25

Post by gryn »

Sebb767 wrote: post #13
tossandthrow wrote: post #12

The future is going to be a horrible place for people taking over legacy code-bases. It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible, Sometimes you digress to simpler types not because you couldn't write more complex types, but because you realise that the next person who needs to make changes to t…

> It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible I've seen some legacy code and the kind of spaghetti humans are able to generate - especially with multiple layers of "I'll just throw in something to make feature X work/fix that bug" - is pretty bad already. I honestly doubt code generation will m…

The difference is in the working memory, a human's is much smaller than a computer's, that leaves room for so much art (In the horror genre).

Re: Trying to Understand Copilot's Type Spaghetti

#26

Post by [deleted] »

tossandthrow wrote: post #12

The future is going to be a horrible place for people taking over legacy code-bases. It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible, Sometimes you digress to simpler types not because you couldn't write more complex types, but because you realise that the next person who needs to make changes to t…

[deleted]

Re: Trying to Understand Copilot's Type Spaghetti

#27

Post by websitescenes »

ChatGPT gives me garbage code unless I ask it politely not to. No joke. Usually the first attempt is pure garbage and I have to call it out as such and then it’s like, “you’re right! Here’s the updated code”. No idea why it can basically never get it right the first time. I also find that it can be quite redundant and offer two distinct solutions morphed into one mutant answer which will turn the undiscerning 1x developer into a -10x developer. But hey, it still saves me time. Sometimes..

Re: Trying to Understand Copilot's Type Spaghetti

#28

Post by gregmac »

The first thing I see when I look at this is "Where are the unit tests?"

This is somewhere in the realm of "clever" or "efficient" code: it looks like it could be written in an easier-to-grok way as dozens of lines of if statements, but I assume there is a reason it wasn't (that is better than "just because"). AI-generated or not, someone or team is responsible for making sure the code/app/service is working for customers, and they have to be able to fix bugs, maintain and modify this.

Does it work today? As a reviewer (or coming across this while fixing a bug), I only know by either trying it, or spending quite a long time to understand and analyze it in my head. Unit tests are the easiest way of "trying it", with all the edge cases and then some.

What if I'm faced with modifying this in 6 or 12 months? Even if I wrote it, chances are my internal mental model is gone. I'd like some reasonable assurance I am not breaking anything.

Also, I'd like to not be the only one responsible for this forever, so I want to let other people modify it. Unit tests are the guard rails that let me say "Go ahead and do whatever you'd like, but don't break the tests".

Re: Trying to Understand Copilot's Type Spaghetti

#29

Post by patrickthebold »

Sebb767 wrote: post #13
tossandthrow wrote: post #12

The future is going to be a horrible place for people taking over legacy code-bases. It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible, Sometimes you digress to simpler types not because you couldn't write more complex types, but because you realise that the next person who needs to make changes to t…

> It was difficult enough with other people spaghetti code, but being able to generate vast amounts of hard to decipher spaghetti code it only going to make this horrible I've seen some legacy code and the kind of spaghetti humans are able to generate - especially with multiple layers of "I'll just throw in something to make feature X work/fix that bug" - is pretty bad already. I honestly doubt code generation will m…

I fear it will be a quantity issue more than a quality one.

Why refactor some code to make it extensible, when AI can write a whole new module from scratch?

Don't understand the 10 AI written classes? Leave them be and have AI write some more.

Once we get AI code deletion and refactoring I guess it will be ok, but generative models are a problem because code is a liability.

Re: Trying to Understand Copilot's Type Spaghetti

#30

Post by rtpg »

gregmac wrote: post #28

The first thing I see when I look at this is "Where are the unit tests?" This is somewhere in the realm of "clever" or "efficient" code: it looks like it could be written in an easier-to-grok way as dozens of lines of if statements, but I assume there is a reason it wasn't (that is better than "just because"). AI-generated or not, someone or team is responsible for making sure the code/app/service is working for cust…

You can scaffold up "unit tests", but honestly for type-level stuff you are working in a different space entirely. Your types are correctness proofs, so your underlying code is either typed correctly or not. There's not really a middle ground that unit tests catch.

Having said that, typescript's soundness issues make it easy to drive a truck through a certain kind of issue, but generally speaking if your type-level programming is no good you're not really going to be able to run unit tests, let alone validate them. Your code just won't go anywhere.

Post reply on HN