41–50 of 148 posts
Where have I been? What is a “small” language model? Wikipedia just talks about LLMs. Is this a sort of spectrum? Are there medium language models? Or is it a more nuanced classifier?
Looks like it punches way above its weight(s). How far are we from running a GPT-3/GPT-4 level LLM on regular consumer hardware, like a MacBook Pro?
We’re already past that point! MacBooks can easily run models exceeding GPT-3.5, such as Llama 3.1 8B, Qwen 2.5 8B, or Gemma 2 9B. These models run at very comfortable speeds on Apple Silicon. And they are distinctly more capable and less prone to hallucination than GPT-3.5 was. Llama 3.3 70B and Qwen 2.5 72B are certainly comparable to GPT-4, and they will run on MacBook Pros with at least 64GB of RAM. However, I ha…
Where have I been? What is a “small” language model? Wikipedia just talks about LLMs. Is this a sort of spectrum? Are there medium language models? Or is it a more nuanced classifier?
7B vs 70B parameters... I think. The small ones fit in the memory of consumer grade cards. That's what I more or less know (waiting for my new computer to arrive this week)
Earlier quoted context omitted.
The SVG created for the first prompt is valid but is a garbage image.
In general I've had poor results with LLMs generating pictures using text instructions (in my case I've tried to get them to generate pictures using plots in KQL). They work but the pictures are very very basic. I'd be interested for any LLM emitting any kind of text-to-picture instructions to get results that are beyond a kindergartner-cardboard-cutout levels of art.
Earlier quoted context omitted.
7B vs 70B parameters... I think. The small ones fit in the memory of consumer grade cards. That's what I more or less know (waiting for my new computer to arrive this week)
How many parameters did ChatGPT have in Dec 2022 when it first broke into mainstream news?
One of the most exciting trends of the past year has been models getting dramatically smaller while maintaining similar levels of capability.
Earlier quoted context omitted.
7B vs 70B parameters... I think. The small ones fit in the memory of consumer grade cards. That's what I more or less know (waiting for my new computer to arrive this week)
How many parameters did ChatGPT have in Dec 2022 when it first broke into mainstream news?
Earlier quoted context omitted.
That's why I use the SVG pelican riding a bicycle thing as a benchmark: it's a deliberately absurd and extremely difficult task.
Appreciate your rapid analysis of new models, Simon. Have any models you've tested performed well on the pelican SVG task?
Claude 3.5 Sonnet is in second place: https://github.com/simonw/pelican-bicycle?tab=readme-ov-file...
Earlier quoted context omitted.
Compare performance on various Macs here as it gets updated: https://github.com/ggerganov/llama.cpp/discussions/4167 OMM, Llama 3.3 70B runs at ~7 text generation tokens per second on Macbook Pro Max 128GB, while generating GPT-4 feeling text with more in depth responses and fewer bullets. Llama 3.3 70B also doesn't fight the system prompt, it leans in. Consider e.g. LM Studio (0.3.5 or newer) for a Metal (MLX) cente…
What if you have a Macbook Air with 16GB (the bechmarks dont seem to show memory).
Where have I been? What is a “small” language model? Wikipedia just talks about LLMs. Is this a sort of spectrum? Are there medium language models? Or is it a more nuanced classifier?
Earlier quoted context omitted.
We’re already past that point! MacBooks can easily run models exceeding GPT-3.5, such as Llama 3.1 8B, Qwen 2.5 8B, or Gemma 2 9B. These models run at very comfortable speeds on Apple Silicon. And they are distinctly more capable and less prone to hallucination than GPT-3.5 was. Llama 3.3 70B and Qwen 2.5 72B are certainly comparable to GPT-4, and they will run on MacBook Pros with at least 64GB of RAM. However, I ha…
[dead]
May want to double-check your specs. 16" w/128GB & 2TB is $5,400.