201–210 of 525 posts
TheBloke doesn’t joke around [1]. I’m guessing we’ll have the quantized ones by the end of the day. I’m super excited to use the 34B Python 4 bit quantized one that should just fit on a 3090. [1] https://huggingface.co/TheBloke/CodeLlama-13B-Python-fp16
Earlier quoted context omitted.
But it won't stop there. Why would it stop at some arbitrarily defined boundary? The savings associated with no longer having to pay programmers the amounts of money that they believe they are worth (high enough to result in collusion between employers) are just too tempting.
Some form of AI will eventually take over almost all existing jobs. Whether those jobs evolve or not somehow and new jobs replace them, we will see. But it's definitely not just programmers. And it will take time. Society needs to adjust. Stopping progress would not be a solution and is not possible. However, hopefully we can pause before we create digital animals with hyperspeed reasoning and typical animal instinct…
Humans don’t become obsolete, we become bored. This tech will make us bored. When humans get too bored and need shit to stir up, we’ll start a war. Take US and China, global prosperity is not enough right? We need to stoke the flames of war over Taiwan.
In the next 300 years we’ll wipe out most of each other in some ridiculous war, and then rebuild.
Now we need code quality benchmarks comparing this against GPT-4 and other contenders.
As a complete noob at actually running these models, what kind of hardware are we talking here? Couldn't pick that up from the README. I absolutely love the idea of using one of these models without having to upload my source code to a tech giant.
34B should be able to run on 24GiB consumer graphics card, or 32GiB Mac (M1 / M2 chips) with quantization (5~6bit) (and 7B should be able to run on your smart toaster).
Any "eli5" tutorial on how to do so, if so?
I want to give these models a run but I have no powerful GPU to run them on so don't know where to start.
TheBloke doesn’t joke around [1]. I’m guessing we’ll have the quantized ones by the end of the day. I’m super excited to use the 34B Python 4 bit quantized one that should just fit on a 3090. [1] https://huggingface.co/TheBloke/CodeLlama-13B-Python-fp16
can it be quantised further so it can run locally on a normal laptop of a developer?
TheBloke doesn’t joke around [1]. I’m guessing we’ll have the quantized ones by the end of the day. I’m super excited to use the 34B Python 4 bit quantized one that should just fit on a 3090. [1] https://huggingface.co/TheBloke/CodeLlama-13B-Python-fp16
Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot https://ai.meta.com/blog/code-llama-large-language-model-cod...
[0] https://github.blog/2023-07-28-smarter-more-efficient-coding...
[1] https://github.com/features/preview/copilot-x
[2] https://github.blog/2023-07-20-github-copilot-chat-beta-now-...
I see both vscode and netbeans have a concept of "inference URL" - are there any efforts like language server (lsp) - but for inference?
Earlier quoted context omitted.
Some form of AI will eventually take over almost all existing jobs. Whether those jobs evolve or not somehow and new jobs replace them, we will see. But it's definitely not just programmers. And it will take time. Society needs to adjust. Stopping progress would not be a solution and is not possible. However, hopefully we can pause before we create digital animals with hyperspeed reasoning and typical animal instinct…
Continuing your aside… Humans don’t become obsolete, we become bored. This tech will make us bored. When humans get too bored and need shit to stir up, we’ll start a war. Take US and China, global prosperity is not enough right? We need to stoke the flames of war over Taiwan. In the next 300 years we’ll wipe out most of each other in some ridiculous war, and then rebuild.
"Global prosperity" might be true in a very long-term historical sense, but it's misleading to apply it to the immediate situation.
Taiwan is not just a talking point. Control over Taiwan is critical for maintaining hegemony. When that is no longer assured, there will likely be a bloody battle before China is given the free reign that it desires.
WWIII is likely to fully break out within the next 3-30 years. We don't really have the facilities to imagine what 300 years from now will look like, but it will likely be posthuman.
ollama run codellama "write a python function to add two numbers"