51–60 of 148 posts
Earlier quoted context omitted.
can you expand a bit?
The model performs very poorly in practice, while in the benchmark it is shown to be DeepSeek V3 level. It's not terrible but it's at another level compared to the models it is very close to (a bit better / a bit worse) in the benchmarks.
Suddenly after reasoning models, it looks like OSS models have lost their charm
I have tried a lot of local models. I have 656GB of them on my computer so I have experience with a diverse array of LLMs. Gemma has been nothing to write home about and has been disappointing every single time I have used it. Models that are worth writing home about are; EXAONE-3.5-7.8B-Instruct - It was excellent at taking podcast transcriptions and generating show notes and summaries. Rocinante-12B-v2i - Fun for s…
I see a lot of talk about good and not good here, but (and a question for everyone) what are people using the non-local big boys for that the locals CAN'T do? I mean, IRL tasks?
Earlier quoted context omitted.
From the limited testing I've done, Gemma 3 27B appears to be an incredibly strong model. But I'm not seeing the same performance in Ollama as I'm seeing on aistudio.google.com. So, I'd recommend trying it from the source before you draw any conclusions. One of the downsides of open models is that there are a gazillion little parameters at inference time (sampling strategy, prompt template, etc.) that can easily impa…
At the end of the day it doesn't matter how good it its, it has no system prompt which means no steerability, a sliding window for incredibly slow inference compared to similar sized models because it's too niche and most inference systems have high overhead implementations of it, and Google's psychotic instruct tuning that made Gemma 2 an inconsistent and unreliable glass cannon. I mean hell, even Mistral added syst…
I’ve never seen a case where putting the system prompt in the user prompt would lead to significantly different outcomes though. Would like to see some examples.
(edit: my bad. i stand corrected. it seems the code just prepends the system prompts to the first user prompt.)
I'm curious. Is there any value to do these OSS models? Suddenly after reasoning models, it looks like OSS models have lost their charm
After reading the technical report do the effort of downloading the model and run it against a few prompts. In 5 minutes you understand how broken LLM benchmarking is.
Is "OpenAI" the only AI company that hasn't released any model weights?
I have tried a lot of local models. I have 656GB of them on my computer so I have experience with a diverse array of LLMs. Gemma has been nothing to write home about and has been disappointing every single time I have used it. Models that are worth writing home about are; EXAONE-3.5-7.8B-Instruct - It was excellent at taking podcast transcriptions and generating show notes and summaries. Rocinante-12B-v2i - Fun for s…
Thanks for the overview. > Qwen2.5-Coder-14B-Instruct - Good for simple coding tasks > OpenThinker-7B - Good and fast reasoning Any chance you could be more specific, ie give an example of a concrete coding task or reasoning problem you used them for?
I would be actually happy to see R1 distilled version, it may make it perform better with the less resource usage.