101–110 of 303 posts
Earlier quoted context omitted.
I'm not talking about GPT-4o here - every benchmark I've seen has had the new models from the past ~12 months out-perform the March 2023 GPT-4 model. To pick just the most popular one, https://lmarena.ai/?leaderboard= has GPT-4-0314 ranked 83rd now.
How have you been able to tie benchmark results to better results?
Silly question: how can OpenAI, Claude and all, have a valuation so large considering all the open source models? Not saying they will disappear or be tiny (closed models), but why so so so valuable?
It's user base and brand. Just like with Pepsi and Coca Cola. There's a reason OpenAI ran a Super Bowl ad.
Earlier quoted context omitted.
VRAM Required = Number of Parameters (in billions) × Number of Bytes per Parameter × Overhead[0]. [0]: https://twm.me/posts/calculate-vram-requirements-local-llms/
Don’t forget to add a lot of extra space if you want a usable context size.
Silly question: how can OpenAI, Claude and all, have a valuation so large considering all the open source models? Not saying they will disappear or be tiny (closed models), but why so so so valuable?
I mean all of them.
Silly question: how can OpenAI, Claude and all, have a valuation so large considering all the open source models? Not saying they will disappear or be tiny (closed models), but why so so so valuable?
Earlier quoted context omitted.
is there some reason you cant train a 1b model to just do agentic stuff?
The Berkeley Function Calling Leaderboard [1] might be of interest to you. As of now, it looks like Hammer2.1-3b is the strongest model under 7 billion parameters. Its overall score is ~82% of GPT-4o's. There is also Hammer2.1-1.5b at 1.5 billion parameters that is ~76% of GPT-4o. [1] https://gorilla.cs.berkeley.edu/leaderboard.html
- That'll be 1 turn scores: at multiturn, 4o is 3x as good as the 3b
- BFCL is generally turn natural language into an API call, then multiturn will involve making another API call.
- I hope to inspire work towards an open model that can eat the paid models sooner rather than later
- trained quite specifically on an agent loop with tools read_files and edit_file (you'll also probably do at least read_directory and get_shared_directories, search_filenames and search_files_text are good too), bonus points for cli_command
- IMHO, this is much lower hanging-fruit than ex. training an open computer-vision model, so I beseech thee, intrepid ML-understander, to fill this gap and hear your name resound throughout the age
Earlier quoted context omitted.
I just started self hosting as well on my local machine, been using https://lmstudio.ai/ Locally for now. I think the 32b models are actually good enough that I might stop paying for ChatGPT plus and Claude. I get around 20 tok/second on my m3 and I can get 100 tok/second on smaller models or quantized. 80-100 tok/second is the best for interactive usage if you go above that you basically can’t read as fast as it gen…
Are there any good sources that I can read up on estimiating what would be hardware specs required for 7B, 13B, 32B .. etc size If I need to run them locally? I am grad student on budget but I want to host one locally and trying to build a PC that could run one of these models.
I ran https://huggingface.co/mlx-community/Qwen2.5-VL-32B-Instruct... using uv (so no need to install libraries first) and https://github.com/Blaizzy/mlx-vlm like this:
uv run --with 'numpy