Codestral Mamba

Post by solarkraft » Wed, Jul 17, 2024, 6:06 AM UTC

lolinder wrote: post #98

I know it's just a throwaway line, but the bit about Cleopatra at the top feels in poor taste. It's completely inaccurate in that no one has ever attributed her death to a "mamba", and even the asp that some sources claim has been disputed. But even aside from that, it just feels weird that a human being's death has turned into a random reference you can make in a throwaway joke while advertising a product. They're c…

Post by xoranth » Wed, Jul 17, 2024, 6:10 AM UTC

thot_experiment wrote: post #66

Does anyone have a favorite FIM capable model? I've been using codellama-13b through ollama w/ a vim extension i wrote and it's okay but not amazing, I definitely get better code most of the time out of Gemma-27b but no FIM (and for some reason codellama-34b has broken inference for me)

Post by thot_experiment » Wed, Jul 17, 2024, 6:26 AM UTC

xoranth wrote: post #113

thot_experiment wrote: post #66

Does anyone have a favorite FIM capable model? I've been using codellama-13b through ollama w/ a vim extension i wrote and it's okay but not amazing, I definitely get better code most of the time out of Gemma-27b but no FIM (and for some reason codellama-34b has broken inference for me)

Is the extension you wrote public?

    function! GetSurroundingLines(n)
        let l:current_line = line('.')
        let l:start_line = max([1, l:current_line - a:n])
        let l:end_line = min([line('$'), l:current_line + a:n])
        
        let l:lines_before = getline(l:start_line, l:current_line - 1)
        let l:lines_after = getline(l:current_line + 1, l:end_line)
        
        return [l:lines_before, l:lines_after]
    endfunction
    
    function! AIComplete()
        let l:n = 256
        let [l:lines_before, l:lines_after] = GetSurroundingLines(l:n)
        
        let l:prompt = '
' . join(l:lines_before, "\n") . ' ' . join(l:lines_after, "\n") . ' '
        
        let l:json_data = json_encode({
            \ 'model': 'codellama:13b-code-q6_K',
            \ 'keep_alive': '30m',
            \ 'stream': v:false,
            \ 'prompt': l:prompt
        \ })
        
        let l:response = system('curl -s -X POST -H "Content-Type: application/json" -d ' . shellescape(l:json_data) . ' http://localhost:11434/api/generate')
    
        let l:completion = json_decode(l:response)['response']
        let l:paste_mode = &paste
        set paste
        execute "normal! a" . l:completion
        let &paste = l:paste_mode
    endfunction
    
    nnoremap c :call AIComplete()

Post by terhechte » Wed, Jul 17, 2024, 6:53 AM UTC

solarkraft wrote: post #112

I kinda just want something that can keep up with the original version of Copilot. It was so much better than the crap they’re pumping out now (keeps messing up syntax and only completing a few characters at a time).

Post by karolist » Wed, Jul 17, 2024, 7:31 AM UTC

heeton wrote: post #116

solarkraft wrote: post #112

I kinda just want something that can keep up with the original version of Copilot. It was so much better than the crap they’re pumping out now (keeps messing up syntax and only completing a few characters at a time).

Have you tried supermaven? It replaced copilot for me a couple of months ago.