271–280 of 303 posts
Does this really need to be a fork of llama.cpp? It seems like there's no real changes to the code.
Earlier quoted context omitted.
You mean the same neurons that don't really exist, other than as a connection from weight to weight, so quite reasonable to see as the weights themselves, because what else is available for us to reference as neurons in absentia those connections?
I disagree. The metaphor we use in ML is that neurons are the nodes that receive the summed, weighted inputs from other neurons. Yes, the weights are the strengths of the connections between them. However, there are many more weights than neurons, so conflating the two doesn't make sense schematically. Also, neurons can also have other parameters which define their behavior such as bias and activation functions. Furt…
Eg, if a weight can be a synapse, can't a weight be an axon? Axons also "connect" neurons, but their length is more related to the connection strength, so could be considered more analogous to a "weighting".
Yet, axons are not as obtusely "one-to-many" as synapses, but depending on the structure of the ML model, and the view of which aspect of it is more impactful to be highlighting by analogy, either take might be more appropriate.
I suppose it depends on the kind of structure you're working with, and whether you're training and inferring, or just one or the other. In all cases I think a good argument could be made for general neuron analogy abuse.
Earlier quoted context omitted.
I'm trying the 13B one, it's decent, I'd say on par with GPT 3, definitely not 3.5 or 4 yet however. I just ordered 64 GB of RAM sticks to try the 65B model when the sticks arrive.
How are you trying it? Alpaca? Or just straight llama.cpp?
Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?
Runs fine on WSL2: main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291 syst…
Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?
Runs fine on WSL2: main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291 syst…
Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?
Runs fine on WSL2: main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291 syst…
Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?
Runs fine on WSL2: main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291 syst…
Earlier quoted context omitted.
weights refer to the trained model weights like for example stable diffusion's v1.1, v1.2 .. v1.4 v.2.x etc. Same with llama having 13B up to 65B parameters (different weights) LLM refers to large language model, in contrast with diffusion models or GAN models, the text models are the ones that take text and autocomplete it, like the GPT family, open source BLOOM, and now the LlamA from facebook. LoRA is the latest m…
> weights refer to the trained model weights This is what I'm having a hard time understanding. So there's the weights, and also a model somewhere? That the weights are based on? Or that you combine with the model to tune it?
Input (list of numbers) -> (Bunch of math operations) with (other numbers) -> Output (also a list of numbers)
This applies whether you are talking about image classification, image generation, text generation etc.
The model defines what the "(Bunch of math operations)" part is. As in, do these multiplications, then add, then a tanh operation etc.
The weights define what the "(other numbers)" are. Training is the process of figuring out these weights using various methods - some of which involve example inputs/outputs (supervised learning), others don't require examples (unsupervised or self-supervised learning).
Earlier quoted context omitted.
I asked GPT-4 for you: "An "Electron wrapper" refers to a software application created using Electron, a popular open-source framework. Electron allows developers to build cross-platform desktop applications using web technologies like HTML, CSS, and JavaScript. An Electron wrapper essentially "wraps" a web application inside a native desktop application, making it possible to run the web application as a standalone…
What, nothing about Electron's unacceptable code bloat factor? Most disappointing! https://www.hendrik-erz.de/post/electron-bloated-web-and-tra... A GPT-4-on-GPT-4 flame war might be an incredible spectacle...
User1: Hey, have you heard about Electron? It's an awesome framework for creating cross-platform apps using web technologies like HTML, CSS, and JavaScript.
User2: Yeah, I've heard of it. But I think it has some pretty serious shortcomings. Like, it's notorious for consuming a lot of system resources, which can be a major drawback for certain applications.
User1: Well, I think that's an acceptable trade-off for the ease of development and the ability to create one codebase that works across different platforms. Plus, it's backed by a large community, which is always a plus.
User2: Sure, but you can't ignore the fact that it increases the app's size significantly. Even a simple Electron app has a huge size compared to native apps, making it a poor choice for users with limited storage or slow internet connections.
User1: The size issue can be managed with optimization techniques, and honestly, storage is becoming cheaper and more abundant these days. You're blowing it out of proportion.
User2: It's not just about storage. Electron apps tend to have worse performance than native apps because they run on a separate instance of Chromium for each app. That's just bad design!
User1: Look, no framework is perfect, but Electron provides a valuable solution for developers who don't want to learn multiple programming languages and platforms. It's about practicality and productivity.
User2: Practicality? You call it practical to have apps that drain your battery life and hog system resources? I'd rather invest the time in learning native development than creating a subpar user experience.
User1: You're really not giving Electron enough credit here. It's widely used by popular apps like Slack, Visual Studio Code, and Discord! They can't be all that bad.
User2: Just because some popular apps use it doesn't mean it's a good choice. They're actually prime examples of how Electron can lead to bloated and resource-hungry applications. Have you ever noticed how sluggish those apps can be at times?
User1: Oh, come on! You're just nitpicking now. The benefits of Electron far outweigh its drawbacks for most developers and applications.
User2: I'm not nitpicking. I'm pointing out legitimate concerns. If Electron keeps getting a free pass, we'll end up with a sea of bloated, resource-hungry applications, dragging down our devices.
User1: You're just being a native app elitist! Electron has democratized app development, making it more accessible for web developers. It's better to have more apps available, even if they're not perfect, than to have a limited selection because of high barriers to entry.
User2: You Electron fanatics never cease to amaze me. It's like you're blind to its glaring problems. You want everyone to just ignore the elephant in the room!
User1: And you native app snobs are so quick to dismiss any alternative solution that threatens your precious, exclusive club! Get off your high horse!
User2: At least my "exclusive club" doesn't promote lazy development practices and drag down user experience! Enjoy your bloated apps, you Electron zealot!
User1: Have fun in your walled garden, you insufferable native app purist!
Can the moderator change the title to "Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a PC?" It runs great, with no hacking, on my Intel Ubuntu computer. I presume it will work on any system that can run make and has enough RAM. The title implies it works only on a Macbook.