Live data from Hacker News

Gitlab’s AI-assisted code suggestions

about.gitlab.com

91–100 of 160 posts

Re: Gitlab’s AI-assisted code suggestions

#91

Earlier quoted context omitted.

GitLab seems to be a lot more responsive to feedback. It'd be worth asking them if they can publicly document their training set and provide a list of licenses and copyright notices.

GitLab team member here. The training data is documented in https://docs.gitlab.com/ee/user/project/repository/code_sugg... AI Transparency is important, all available AI features provide documentation for training data, and are built with privacy first. The GitLab Duo announcement adds more feature details and plans. https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the... The AI/ML blog series provides insig…

"Codey was fine-tuned on a large dataset of high quality, permissively licensed code from external sources" is not sufficient information to be able to provide attribution and licensing information.

Would it be possible to get a complete list of sources and licenses?

Re: Gitlab’s AI-assisted code suggestions

#92

Earlier quoted context omitted.

> using LLMs to do machine translation is a terrible waste of resources. Could you elaborate on that? LLMs seem perfectly suited for language tasks like translation. They don't seem particularly expensive either, especially compared to hiring a person.

There are already machine-translation services trained and created specifically for that purpose. While it's an amazing realization that LLMs can do this and do it pretty well without having to be trained specifically for this one purpose, training something to do literally all text generation tasks is expensive compared to training something specifically to do language to language translation. For a maybe more obvio…

The original paper[0] that laid the foundation for modern LLMs was demonstrated on machine translation tasks. It's one of the primary use cases these architectures were designed for. What other types of models do you have in mind that outperform them?

[0] "Attention Is All You Need" https://arxiv.org/pdf/1706.03762.pdf

Re: Gitlab’s AI-assisted code suggestions

#93

Earlier quoted context omitted.

Even MIT licensed code requires you to preserve the copyright and permission notice. If a human did what these language models are doing (output derivative works with the copyright and license stripped), it would be a license violation. When humans want to create a new implementation with clean IP, they have one team study the IP-encumbered code and write a spec, then a different team writes a new implementation acco…

There’s a clear separation between the training process which looks at code and outputs nothing but weights, and the generation process which takes in weights and prompts and produces code. The weights are an intermediate representation that contains nothing resembling the original code.

I think this view is incredibly dangerous to any kind of skills mastery. It has the potential to completely destroy the knowledge economy and eventually degrade AI due to a dearth of training data.

Re: Gitlab’s AI-assisted code suggestions

#94
post #49
post #43

Earlier quoted context omitted.

I mean if you ask the tool to produce a specific algorithm by it's very famous name then that's on you. If you ask DALL-E to draw Pikachu you can't be suprised Pikachu when it's a copyright violation.

I don't know, it seems like a bit more of a gray area here than with Pikachu. For example, if I said "write me an implementation of fast inverse square root in rust"... and it did... that's certainly not a copyright violation. And if I said "ok, now port this rust code to C..." and it did, then that's certainly not a violation. But then why should I be penalized because the language I want it in happens to be the ori…

A character for character copy including comments? That should be penalized under current law.

Why not songs, software, entire books and tv shows?

Re: Gitlab’s AI-assisted code suggestions

#95
post #46

I just canceled my Copilot subscription last night. I definitely never saw the productivity boost that I’ve seen so many claim. I actually feel like the suggestions seemed to get worse during my month of using it for some reason. Towards the end, it started suggesting these large blocks of code (another issue I had with the interface as well) which were very not relevant to what I was attempting to write. All in all,…

Are there other people who find that IntelliJ's autocomplete suggestions and shortcuts (including live templates&Co) already do 80% of what Copilot is trying?

Re: Gitlab’s AI-assisted code suggestions

#96
post #70

Earlier quoted context omitted.

The neurons in my brain when I plagiarize are just arrangements of atoms that contain nothing that resembles orginal code/text passages/etc.

The trained weights of a GPT model are a frozen, static, transmissible representation. They’re not equivalent to the live state of a brain.

Pretty equivalent to the snapshot of a live brain. Those inside it are even called neurons and neural network

Re: Gitlab’s AI-assisted code suggestions

#97

Going to be interesting. Github had the advantage that they were offering Copilot into an audience of people who just defaulted to them. Gitlab on the other hand is for people who are into license wars, etc. so the AI product is going to be offered to a hostile audience.

These days Gitlab's main customer is the enterprise. They pushed away a lot of their open source users when they heavily curtailed the free tier.

Re: Gitlab’s AI-assisted code suggestions

#98

Okay, but how much are they charging? It's getting really annoying how many sites force you into a trial just to find out how much it'll cost when it ends. EDIT: Is this even positioned to compete with Copilot? What editors are there plugins for? There is surprisingly little information on the site.

Seems to be $99 per month.

Signing up for the free trial funnels me to the trial of Gitlab Ultimate. So assuming that you need an Ultimate subscription to use it after the trial, that's the price. Pricing is here https://about.gitlab.com/pricing/

In contrast, Copilot is $10 per month https://github.com/features/copilot#pricing

Re: Gitlab’s AI-assisted code suggestions

#99
post #43

Earlier quoted context omitted.

GitHub Copilot copying Quake's fast inverse square root function is a famous example where Copilot copied GPL-licensed code only given the comment // fast inverse square root https://news.ycombinator.com/item?id=27710287

I mean if you ask the tool to produce a specific algorithm by it's very famous name then that's on you. If you ask DALL-E to draw Pikachu you can't be suprised Pikachu when it's a copyright violation.

Doesn't have to be famous. I gave ChatGPT lines 48-67 of drivers/hwmon/sht21.c [1] and asked it to complete the function. Prompt:

    Complete the function for an sht21 driver [code omitted]
What it returned was the copyrighted function, comments and all:

    static inline int sht21_rh_ticks_to_per_cent_mille(int ticks)
    {
    ticks &= ~0x0003; /* clear status bits /
    /* Formula RH = -6 + 125 * SRH / 2^16 from data sheet 6.1,
    * optimized for integer fixed point (3 digits) arithmetic
    */
    return ((15625 * ticks) >> 13) - 6000;
    }
Notice how it even included a comment referencing a specific datasheet!

This driver is hardly "famous" or even "notable", because those aren't things LLMs understand. The prompt simply contains enough context to be distinctive and the sht21.c is an old, stable driver in each of the many kernel trees included in its training set.

Regurgitation isn't a particularly rare thing with LLMs, most cases just aren't this obvious.

[1] https://github.com/torvalds/linux/blob/c6b0271053e7a5ae57511...

Re: Gitlab’s AI-assisted code suggestions

#100
post #54

Earlier quoted context omitted.

TBH, using this as a fancy smart autocomplete doesn't seem like the best idea. I can't believe that I'm suggesting something that sounds like Clippy... but... It really needs to be something where you can rattle off a list of requirements and have it build the code for you. The code context is necessary, but not sufficient.

It’s coming up: https://github.com/features/preview/copilot-x

I don't get the thing with automated PR and commit messages. They mostly describe _what_ was done and not why.

Also, if you enhance your available information with AI, why not just write no message at all (or a short one as always) and use the AI when _looking_ at the commits/PRs? The AI will certainly be better at this in a later stage because they themselves will get better and they will have more context due to being able to look at the commits that followed the PR/commit.

There is no point in generating the commit message when committing.

When auto-generated messages will become a commodity, there will be issues with it. For example, if the message doesn't fit with the actual commit contents (semantically or syntactically), I have to think hard whether this is an AI-message-generator-bug, or the author missed something or wether I am missing something. This is not cool and makes reading commit messages harder.

Post reply on HN