Live data from Hacker News

Gitlab’s AI-assisted code suggestions

about.gitlab.com

131–140 of 160 posts

Re: Gitlab’s AI-assisted code suggestions

#131
GitLab was rather quick to board the AI hype train this year. Fine - they don't want to look like they're behind the curve, have to please investors, and whatever else.

But IMO there are plenty of other places to add real value across the GitLab product with AI/ML features.

Here, it just looks like they saw GitHub do something and felt a need to copy it. But two years late, and worse.

As a longtime GitLab user (and onetime contributor!), I'm a bit disappointed they're spending so much time on this. I think they're just too far behind.

Re: Gitlab’s AI-assisted code suggestions

#132
It seems I need to have a company to sign up for the trial ?

I have a free gitlab account but can't proceed without entering business information as it seems to be tied to the gitlab premium feature set. Am I doing something wrong ?

Re: Gitlab’s AI-assisted code suggestions

#133

Earlier quoted context omitted.

The LLM's trained parameters are a lossy memory of their training data.

No, they aren't.

Why would you go on HN and say blatantly wrong information

AI is literally 1-1 equivalent to compression. If you don't believe me, you should check out this demo of GPT-2 as a (for awhile SOTA) text compressor.

Oh its down now, but here's the HN thread to prove this exists: https://news.ycombinator.com/item?id=23618465

Re: Gitlab’s AI-assisted code suggestions

#135
post #130

I get that gitlab needs more features to continue to sell. But honestly for me it reduces the attractiveness of gitlab. Just adding half working features does not increase the value for me. Please finish and polish the existing Features before adding hype feature 93939. Maybe add a policy that your open feature and big count should be less than 20K or something. Examples: Security scanning has a god awful UI and UX.…

It's honestly doubly annoying when there's multiple times per month where I run into some issue / straight up bug which I then find an ongoing bug ticket from like 2018 for, that's still open but apparently not being worked on.

I really like Gitlab, I think the CI experience is a lot better than, for instance, Github Actions, but I also don't ever see me using a good chunk of the stuff that's been shoveled on in the last few years and I don't really know anyone that does.

Re: Gitlab’s AI-assisted code suggestions

#136

Crucial: do they train on GPL code? If I am to use this tool, I must abide by the license terms of the training data. Even if it is found that the GPL does not cover LLM responses as derivative works, I would prefer to be on the safe side and refuse to use models trained on software with non-permissive licenses unless I am building Free Software.

Even MIT licensed code requires you to preserve the copyright and permission notice. If a human did what these language models are doing (output derivative works with the copyright and license stripped), it would be a license violation. When humans want to create a new implementation with clean IP, they have one team study the IP-encumbered code and write a spec, then a different team writes a new implementation acco…

Why can't AI do the same: copyrighted code -> spec -> generated code.

... and then execute copyrighted code -> trace resulting values -> tests for new code.

AI could do clean room reimplementation of any code to beef up the training set. It can also make sure the new code is different from the old code at ngram-level, so even by chance it should not look the same.

Would that hold up in court? Is it copyright laundering?

Re: Gitlab’s AI-assisted code suggestions

#137
post #130

I get that gitlab needs more features to continue to sell. But honestly for me it reduces the attractiveness of gitlab. Just adding half working features does not increase the value for me. Please finish and polish the existing Features before adding hype feature 93939. Maybe add a policy that your open feature and big count should be less than 20K or something. Examples: Security scanning has a god awful UI and UX.…

It's honestly doubly annoying when there's multiple times per month where I run into some issue / straight up bug which I then find an ongoing bug ticket from like 2018 for, that's still open but apparently not being worked on. I really like Gitlab, I think the CI experience is a lot better than, for instance, Github Actions, but I also don't ever see me using a good chunk of the stuff that's been shoveled on in the…

At least for python and dotnet, I think github actions are superior.

Gitlab did not support poetry for ages there where bugs with python galore and for dotnet they do not speak the same language - e.g. issues in dotnet are reported as saeif, but gl has it own weird format.

And yes, you are totally right with the bug tracker. It's often funny how long the ticket is just because it was retagged sooo many times.

Re: Gitlab’s AI-assisted code suggestions

#138
post #107

Earlier quoted context omitted.

There’s a clear separation between the training process which looks at code and outputs nothing but weights, and the generation process which takes in weights and prompts and produces code. The weights are an intermediate representation that contains nothing resembling the original code.

But the original content is frequently recoverable. You can't just take copyrighted code, base 64 it, sent it to someone, have them decode it, and claim there was no copyright violation. From my (admittedly vague) understanding copyright law cares about the lineage of data, and I don't see how any reasonable interpretation could consider that the lineage doesn't pass through models. IANAL

> But the original content is frequently recoverable.

What if we train the model on paraphrases of the copyrighted code? The model can't reproduce exactly what it has not seen.

Also consider the size ratio - 1TB of code+text ends up into 1GB of model weights. There is no space to "memorize" the training set, it can only learn basic principles and how to combine them to generate code on demand.

The copyright law in principle should only protect expression, not ideas. As long as the model learns the underlying principles without copying the superficial form, it should be ok. That's my 2c

Re: Gitlab’s AI-assisted code suggestions

#139
post #46

I just canceled my Copilot subscription last night. I definitely never saw the productivity boost that I’ve seen so many claim. I actually feel like the suggestions seemed to get worse during my month of using it for some reason. Towards the end, it started suggesting these large blocks of code (another issue I had with the interface as well) which were very not relevant to what I was attempting to write. All in all,…

This month, for the third time, I evaluated copilot. It has vastly improved. So far that this time I'll probably pay for it.

I think it's biggest benefit isn't for me, though. Copilot is great at churning out predictable or repetitive stuff, much less in writing a abstractions over or around that repetitive or predictable stuff. E.g. it's great at writing yet another CreditcardPayment ActiveRecord model for Rails. But less so for an obscure business domain model used outside of a framework. And great at writing fifteen getters and setters, less for an abstract base class, DSL or macro that introduces all these getters and setters for me.

It's also bad at certain rewrites. A codebase was slowly rewritten from OldStyle (and old libs) to NewStyle. It stubbornly kept suggesting OldStyle snippets.

And last, I find Copilot has a far higher ROI in some languages and frameworks than others. E.g. dynamic languages (like Ruby) have very poor LSP and intellisense support compared to typed and static (like Rust). So the added benefit differs a lot based on where it's used, is my experience.

I guess esp. the latter is why I too am underwhelmed. But also why I'll keep using it, this time, for when I do the inevitable Rails, JavaScript or Panda's gigs. But less so for my weird eventsourced, hexagonal, Rust project.

Re: Gitlab’s AI-assisted code suggestions

#140
post #117

Earlier quoted context omitted.

Okay, that’s an argument from consequences, but is the view factually wrong ?

I mean those consequences are why patent law exists. New technology may require new regulatory frameworks, like we've been doing since railroads. The idea that we could not amend law and that we need to pedantically say "well this isn't illegal now" as an excuse for doing something unethical and harmful to the economy is in my opinion very flawed.

Is it really harmful to the economy, or only to entrenched players? Coding AI should be a benefit to many, like open source is. It opens the source even more, should be a dream come true for the community. It's also good for learning and lowering the entry barrier.

At the same time it does not replace human developers in any application, it might take a long time until we can go on vacation and let AI solve our Jira tickets. Remember the Self Driving task has been under intense research for more than a decade now, and it's still far from L5.

It's a trend that holds in all fields. AI is a tool that stumbles without a human to wield it, it does not replace humans at all. But with each new capability it invites us to launch new products and create jobs. Human empowerment without human replacement is what we want, right?

Post reply on HN