11–20 of 69 posts
So if Microsoft/Github ever offers an on-prem version, what will be the advantage of using your product?
GitHub and other companies like Amazon have the advantage of the scale in terms of dataset. What’s the guarantee that the pre trained model you have that you’ll fine tune on a company’s code base is as good as say Copilot? It makes it even hard to evaluate when you don’t have a demo to try - it’s not that hard to setup a pipeline to run your model in cloud and send invites to potential customers if you want to.
It doesn't make sense to treat Copilot as a competitive benchmark where Copilot isn't even an option for policy reasons, which is what this product is targeting. Does it provide net positive support to developers? Is that support worth the licensing and maintenance costs? Those would indeed benefit from a demo, but it's also hard to demo something whose value hinges on fine-tuning. More likely, they'll need to sell l…
You’re fine-tuning the model. What model are you fine-tuning? I can’t imagine you trained your own LLM from scratch, so how can you possibly guarantee the core model wasn’t trained on non-permissively licensed code?
So if Microsoft/Github ever offers an on-prem version, what will be the advantage of using your product?
Another thing is that since Microsoft/Github are working with OpenAI's closed source Codex model, we think it's unlikely they'll offer something on-prem anytime soon since they would have to reveal the model weights, and thus risk a leak (ex. Meta's LLaMA model weights got leaked within a week)
It's become pretty clear that models aren't a moat. If everyone has Codex-class capability (which is already happening), there's no real risk to them deploying on-prem, because the model itself is a commodity.
Let's celebrate when teams build products to fill big needs, instead of dismissing things because of a potential threat from a big company. If we dismiss new things, then we'll just end up in a world where big companies get complacent and devs get less new stuff. (I know many GitHubbers, and they love seeing new stuff and are cheering for it because they can't do everything.)
GitHub and other companies like Amazon have the advantage of the scale in terms of dataset. What’s the guarantee that the pre trained model you have that you’ll fine tune on a company’s code base is as good as say Copilot? It makes it even hard to evaluate when you don’t have a demo to try - it’s not that hard to setup a pipeline to run your model in cloud and send invites to potential customers if you want to.
Wasn't copilot only trained on public GitHub repos? Presumably there's nothing stopping anyone from training on them.
Any thoughts on an individual "Here you go, but you're on your own" license? Something like this would be massively beneficial to small IT teams who have the same inability to send code to OpenAI, but could use the benefit of a smart auto-complete style system like Copilot. They wouldn't necessarily need the custom integration - even small benefits would be seen from an initial core model. Additional integrations cou…
Training data is only your own data, because that might not achieve a great accuracy?
Does it work with any programming language?
> Permissively Licensed: Trained only on permissively-licensed repos to avoid legal risks You’re fine-tuning the model. What model are you fine-tuning? I can’t imagine you trained your own LLM from scratch, so how can you possibly guarantee the core model wasn’t trained on non-permissively licensed code?
Any thoughts on an individual "Here you go, but you're on your own" license? Something like this would be massively beneficial to small IT teams who have the same inability to send code to OpenAI, but could use the benefit of a smart auto-complete style system like Copilot. They wouldn't necessarily need the custom integration - even small benefits would be seen from an initial core model. Additional integrations cou…