Open Sourcing Cody – Sourcegraph's AI-enabled editor assistant

> There's actually a growing body of evidence that shows the emergent ability of LLMs to reason (the so-called "chain of thought" ability) arises only when LLMs are trained on huge amounts of code, not just natural language. Natural language training data provides the ability to sound human, but it is the programming language training data that provides LLMs with the ability to be logical.

Good on Sourcegraph to contribute back to the open source community. LLMs rely more on open source code than meets the eye.

Any plans for a neovim plugin?

I wonder if there is anything like this for a notebook-style UI? I’m a fan of how Observable does it.

Cody is more like a wrapper on top of GPT API (gpt-3.5-turbo) endpoint, with related code snippets embed in prompt. It’s not competing the low latency scenario that Github Copilot seats in.

There’re recent works show a potential of beating Copilot’s performance (e.g https://arxiv.org/abs/2303.12570) with much smaller models (500M vs 10B+).

Inspired by these work, I’m building Tabby (https://github.com/TabbyML/tabby), a OSS GitHub Copilot alternative. Hopefully it could make low cost AI coding accessible to everyone.