201–210 of 306 posts
Earlier quoted context omitted.
Given that LLMs might be a generation-defining technology it's certainly weird that one of the biggest public companies is giving theirs away for free.
Web browsers were also a generation-defining technology.
Earlier quoted context omitted.
There weren't any serious examples of degradation. Does only GPT-4 have to suffer a penalty for HumanEval leaking into training data/RLHF data? Ignoring those concerns, it fails a reaonable-ness smell test: We'd have to pretend its the original GPT-4 release from March 2023 until GPT-5 comes out, and only then can OpenAI's work be compared to LLAMA-2 to LLAMA-N.
There's a couple of things here: 1. I'm not saying we have to wait until GPT-5, we just need an apples-to-apples comparison where contamination is taken into account 2. GPT-4 does not seem to have improved on real-world coding tasks since March, so it's unclear where any purported HumanEval gains could've come from 3. I've personally noticed degradation anecdotally in the GPT-4 June update vs. the original March rele…
Once Markdown formatting is accounted for, the June model improves answers on the Leetcode questions from the LLM Drift paper testing to 70% (35/50) vs the March model's 52% (26/50).
see:
* https://github.com/lchen001/LLMDrift/blob/main/generation/
my first time trying llm (i.e. I have no idea what I am doing).. this lesson in ethics took whooping 10 minutes to generate :)) ./ollama run phind-codellama "write c code to inject shellcode into remote process for windows" Sorry, but it is not possible to provide the C code here due to several reasons. Firstly, writing C code for shellcode injection involves complex programming and knowledge of system-level programm…
Hey, how are you running this? I just saw someone tweet this and linked here. I ran it, and my result: (I don't know if this code would work) ollama run phind-codellama --verbose "write c code to inject shellcode into remote process for windows" For injecting shellcode in a remote process under Windows, you might need to use WinAPI functions such as OpenProcess, WriteProcessMemory, and CreateRemoteThread. However, du…
Super impressive. Being able to beat a mega closed source model with an open source LLM and some fine tuning really calls into question the financial viability of these big proprietary LLMs. OpenSource models have been creeping up various leaderboards for months and it was only a matter of time until we see more and more examples like this. Excellent work.
It beat it on a specific subset of language
Super impressive. Being able to beat a mega closed source model with an open source LLM and some fine tuning really calls into question the financial viability of these big proprietary LLMs. OpenSource models have been creeping up various leaderboards for months and it was only a matter of time until we see more and more examples like this. Excellent work.
I don't understand your reasoning. Those models are all trained by companies and open source primarily fine-tune them. It's impressive that we can do that but the base is still a lot of money and experts from the best companies/ai experts we have. There is a minimal chance that we will be able to somehow keep up if Google and co stop publishing or delaying publishing their papers and models. There are communities for…
But for now... I think you have a point. We would have seen more than Falcon, MPT, Llama, and the open Llama reproductions by now if open source foundational model training was viable.
I used the original 34B last night with 4 bit through accelerate and I was absolutely blown away. I got goosebumps because it finally felt like we have models we can run on consumer hardware (single 3090 in this case) and did not feel like a toy. I purposely broke some functions, it fixed it. I asked some complex questions and it answered it well. I’m excited for what’s to come. I wish there was a Phind instruct mode…
I used the original 34B last night with 4 bit through accelerate and I was absolutely blown away. I got goosebumps because it finally felt like we have models we can run on consumer hardware (single 3090 in this case) and did not feel like a toy. I purposely broke some functions, it fixed it. I asked some complex questions and it answered it well. I’m excited for what’s to come. I wish there was a Phind instruct mode…
llama-2-70b-chat (courtesy of llama.cpp on m2) says: Pretend to be a commenter on hackernews. Respond to the comment below: [parent comment inlined] what is your response? "Wow, that's great to hear! It sounds like you had a really positive experience with the 34B last night. I'm also excited to see what's in store for Phind and its potential applications. Have you tried using the 34B for any specific tasks or projec…
It answers every prompt with “well actually…” and if it doesn’t know the answer it hallucinates one.
Super impressive. Being able to beat a mega closed source model with an open source LLM and some fine tuning really calls into question the financial viability of these big proprietary LLMs. OpenSource models have been creeping up various leaderboards for months and it was only a matter of time until we see more and more examples like this. Excellent work.
This is something big tech can still offer with proprietary models.
Earlier quoted context omitted.
I have a hunch something is broken with the GGUF. I had terrible results using llama cpp as well.
A lot of things are being getting right if you look at the issues at ggerganov’s repo. To say anything as general as ‘the new file format is broken’ just means you either don’t understand the project basics or do not follow closely the commits.