121–130 of 412 posts
Earlier quoted context omitted.
Only in the same way that the plural of 'opinion' is 'fact' ;)
Except, very literally, data is a collection of single points (ie what we call "anecdotes").
Earlier quoted context omitted.
There are a lot of variables here such as your hardware's memory bandwidth, speed at which at processes tensors etc. A basic thing to remember: Any given dense model would require X GB of memory at 8-bit quantization, where X is the number of params (of course I am simplifying a little by not counting context size). Quantization is just 'precision' of the model, 8-bit generally works really well. Generally speaking,…
4 bit is absolutely fine . I know this is crazy to here because the big iron folks still debate 16 vs 32 and 8 vs 16 is near verboten in public conversation. I contribute to llama.cpp and have seen many many efforts to measure evaluation perf of various quants, and no matter which way it was sliced (ranging from subjective volunteers doing A/B voting on responses over months, to objective object perplexity loss) Q4 i…
i mean, deepseek is fp8
Any news on some viable successor of LLMs that could take us to AGI? As I see they still can't solve some fundamental stuff to make it really work in any scenario (halucinations, reasoning, grounding in reality, updating long-term memory, etc.)
AGIs probably comes from neurosymbolic AI. But LLMs could be the neuro-part of that. On the other hand, LLM progress feels like bullshit, gaming benchmarks and other problems occured. So either in two years all hail our AGI/AMI (machine intelligence) overlords, or the bubble bursts.
"Give me a PowerShell script to get all users with an email address, and active license, that have not authed through AD or Azure in the last 30 days. Now take those, compile all the security groups they are members of, and check out the file share to find any root level folders that these members have access to and check the audit logs to see if anyone else has accessed them. If not, dump the paths into a csv at C:\temp\output.csv."
Can I write that myself? Yes. In 20 seconds? Absolutely not. These things are saving me hours daily.
I used to save stuff like this and cobble the pieces together to get things done. I don't save any of them anymore because I can for the most part 1 shot anything I need.
Just because it's not discovering new physics doesn't mean it's not insanely useful or valuable. LLMs have probably 5x'd me.
I haven't tried this model yet and am not in a position to for a couple days, and am wondering if anyone feels that with these.
The new battlefront may be to take reasoning to the level of abstraction and creativity to handle math problems without a numerical answer (for ex: https://arxiv.org/pdf/2503.21934).
I suspect that kind of ability will generalize well to other areas and be a significant step toward human level thinking.
The benchmark results are so incredibly good they are hard to believe. A 30B model that's competitive with Gemini 2.5 Pro and way better than Gemma 27B? Update: I tested "ollama run qwen3:30b" (the MoE) locally and while it thought much it wasn't that smart. After 3 follow up questions it ended up in an infinite loop. I just tried again, and it ended up in an infinite loop immediately, just a single prompt, no follow…
qwen3:30b has the same checksum as https://ollama.com/library/qwen3:30b-a3b-q4_K_M
Earlier quoted context omitted.
This is NOT true. At least on the 1.5B version model on my local machine. It blocks answers when using offline mode. Perplexity has an uncensored a version, but don't thing it is open on how they did it.
Didn't know Perplexity cracked R1's censorship but it is completely uncensored. Anyone can try even without an account: https://labs.perplexity.ai/ . HuggingFace also was working on Open R1 but not sure how far they got.
Sorry, no. It's not.
It can't write about anything "problematic".
Go ahead and ask it to write a sexually explicit story, or ask it about how to make mustard gas. These kinds of queries are not censored in the standard API deepseek R1. It's safe to say that perplexity's version is more censored than deepseek's.
The benchmark results are so incredibly good they are hard to believe. A 30B model that's competitive with Gemini 2.5 Pro and way better than Gemma 27B? Update: I tested "ollama run qwen3:30b" (the MoE) locally and while it thought much it wasn't that smart. After 3 follow up questions it ended up in an infinite loop. I just tried again, and it ended up in an infinite loop immediately, just a single prompt, no follow…
You tried a 4-bit quantized version, not the original. qwen3:30b has the same checksum as https://ollama.com/library/qwen3:30b-a3b-q4_K_M
Earlier quoted context omitted.
You tried a 4-bit quantized version, not the original. qwen3:30b has the same checksum as https://ollama.com/library/qwen3:30b-a3b-q4_K_M
What is the original? The blog post doesn't state the quantization they benchmarked.
You can see it's roughly the same size as the one in the official repo (16 files of 4GB each):