71–79 of 79 posts
Earlier quoted context omitted.
My understanding is they legally can't. It was trained used OpenAI, which doesn't allow using their output to train new models. Someone would need to find another data source to fine tune llama.
It’s only a matter of time
Earlier quoted context omitted.
> I'm still amazed that Apple invests so much into every other bit of hardware on a high end phone, yet always gives you the bare minimum amount of RAM they can get away with. Advanced hardware makes bullet points on advertising to sell the device; giving the bare minimum of RAM accelerates the device planned obsolescence, so that user will be forced to upgrade sooner to the next model.
I personally don't buy that it's planned obsolescence. I think most people just don't need that much ram. IOS is really good at loading/unloading stuff as needed, outside of HN I'm not sure most consumers care about the exact amount of ram. Apple still does security updates for IOS - last was 12.5.7 - 23 Jan 2023 - that's back to the iPhone 5S They've literally provided security updates for a 10 year old device, has…
Functionally they're useless
Earlier quoted context omitted.
How is that an iPhone 7 is completely current vs give me a branded Android from the same year of release that has even security updates, not even features.
The problem with iPhones is once updates stop there's nothing you can do. The iPhone 7 isn't current, it's stuck on iOS 15 while the newest is 16. And while the pixel 2 (which is only a month younger than the iPhone 7) only got official support up to Android 11; you actually own the device and can easily unlock the boot-loader to upgrade to Android 13.
Old versions of iOS quickly stop working as apps demand updates, and the updates require a new iOS version.
This is really cool but the output is such garbage at that weight size that you might as well be running a markov chain.
Isn't any LLM mathematically a Markov chain, such that the current state includes the context of the last (finite) n tokens?
With a Markov chain, you're assuming a state machine where each state has independent probabilities on outgoing edges. As the number of states gets larger, you have fewer training samples for each state. When n gets large enough, nearly all states have zero training samples; they've never been seen before. How do you estimate probabilities?
Better to just say it's a stateless function of the input.
Until it thermally throttles 40 seconds later. But yeah, it's really cool how many platforms the vanilla code in llama.cpp can be easily compiled on. And somehow I doubt they did the quantization step on the Pixel itself. My favorite was the person who did it on the rpi4. I know a guy working on getting it going on rpi3 but the ARM7/8 mixing , NEON support, and 64 bit ARM intrinsics are apparently non-trivial to conv…
I'm the original tweet author. Currently typing this from my Pixel after running it countless times :)
Earlier quoted context omitted.
I was replying to a comment that said it “seems fine.” It does not seem fine. It is incomprehensible and doesn’t match the results I’ve seen from 7B through 65B. It is true that RLHF could improve it, and perhaps then this severe of optimization will seem fine.
I've heard a number of people say (from earlier) that the quantization and default sampling parameters is way wacked. Honestly even running that model size alone is the big achievement here and getting the accuracy to actually reach the benchmark is the beeg next step nao, I believe. <3 :'))))
It is not really llama, it is llama quantized to 4bit. Not even the quality of original 7B. I could also quantize it to 1 bit and claim it runs on my RPI3.
i haven't noticed 4bit quantization affecting the quality of LLaMA-7B, it produces very coherent outputs, the trick is having a good example in your prompt so it has a good idea of what's expected of it.