141–148 of 148 posts
Earlier quoted context omitted.
What if you have a Macbook Air with 16GB (the bechmarks dont seem to show memory).
You could definitely run an 8B model on that, and some of those are getting very capable now. The problem is that often you can't run anything else. I've had trouble running larger models in 64GB when I've had a bunch of Firefox and VS Code tabs open at the same time.
Earlier quoted context omitted.
This "draw pelican riding on bicycle" is quite deep if you think about it. Phi is all about synthetic training and prompt -> svg -> render -> evaluate image -> feedback loop feels like ideal fit for synthetic learning. You can push it quite far with stuff like basic 2d physics etc with plotting scene after N seconds or optics/rays, magnetic force etc. SVG as LLM window to physical world.
> SVG as LLM window to physical world. What? let’s try not to go full forehead into hype. SVGs would be an awfully poor analogy for the physical world…
Whether it's a good model or a model that the LLMs can manipulate, I have no idea. But it's entirely plausible that I could feed an SVG map and ask it geospatial reasoning questions like how far between A and B, what's the nearest grocery store, etc.
Earlier quoted context omitted.
Saying these models are at GPT-4 level is setting anyone who doesn't place special value on the local aspect up for disappointment. Some people do place value on running locally, and I'm not against then for it, but realistically no 70B class model has the amount of general knowledge or understanding of nuance as any recent GPT-4 checkpoint. That being said these models are still very strong compared to what we had a…
I said GPT-4, not GPT-4o. I'm talking about a model that feels equivalent to the GPT-4 we were using in March of 2023.
The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…
Prompt:
Are you based on OpenAI?
Reply:
No, I am not based on OpenAI's models; I am developed by Microsoft. While both systems are designed for natural language processing and understanding, we may have different architectures, training methodologies, or datasets. However, our shared goal is to provide helpful and informative responses while ensuring user safety and privacy.
Are you similar to ChatGPT?Earlier quoted context omitted.
The coolness of local LLMs is THE only reason I am sadly eyeing upgrading from M1 64GB to M4/5 128+GB.
I'm returning my 96GB m2 max. It can run unquantized llama 3.3 70B but tokens per second is slow as molasses and still I couldn't find any use for it, just kept going back to perplexity when I actually needed to find an answer to something.
Earlier quoted context omitted.
The SVG created for the first prompt is valid but is a garbage image.
In general I've had poor results with LLMs generating pictures using text instructions (in my case I've tried to get them to generate pictures using plots in KQL). They work but the pictures are very very basic. I'd be interested for any LLM emitting any kind of text-to-picture instructions to get results that are beyond a kindergartner-cardboard-cutout levels of art.
Earlier quoted context omitted.
GPT-3 had 175B, and the original ChatGPT was probably just a GPT-3 finetune (although they called it gpt-3.5, so it could have been different). However, it was severely undertrained. Llama-3.1-8B is better in most ways than the original ChatGPT; a well-trained ~70B usually feels GPT-4-level. The latest Llama release, llama-3.3-70b, goes toe-to-toe even with much larger models (albeit is bad at coding, like all Llama…
> However, it was severely undertrained by modern standards. at the time, it was trained according to neural scaling laws oai believed to hold.
Earlier quoted context omitted.
Appreciate your rapid analysis of new models, Simon. Have any models you've tested performed well on the pelican SVG task?
gemini-exp-1206 is my new favorite: https://simonwillison.net/2024/Dec/6/gemini-exp-1206/ Claude 3.5 Sonnet is in second place: https://github.com/simonw/pelican-bicycle?tab=readme-ov-file...
This is what o1-pro yielded: https://gist.github.com/carbocation/8d780ad4c3312693ca9a43c6...