641–650 of 695 posts
Self hosting LLMs will explode in popularity over next 12 months. Open models are made much more interesting and exciting and relevant by new generations of AI focused hardware such as the AMD Strix Halo and Apple Mac Studio M3. GPUs have failed to meet the demands for lower cost and more memory so APUs look like the future for self hosted LLMs.
For single user, maybe. But for small teams GPUs are still the only available option, when considering t/s and concurrency. Nvidia's latest 6000pro series are actually reasonably priced for the amount of vram / wattage you get. A 8x box starts at 75k eur and can host up to DS3 / R1 / Llama4 in 8bit with decent speeds, context and concurrency.
Earlier quoted context omitted.
That particular debate is often a semantics debate, so it isn't in the domain of science at all. The main way I can think of off-hand to try and make it scientific is to ask about correlational clusters. And then you get way more than two genders, but you definitely get some clusters that contain both transwomen and men (e.g. if I hear a video game speed runner or open source software passion projecf maker using she/…
I have noticed certain groups where trans people are relatively over represented and group involvement more correlated with biological gender, but that’s not actually that interesting or meaningful in reality. Trans women having similar interests to men doesn’t make them men any more than me owning a gun makes me a Republican.
And correlational clusters is one of the few ways it's not just semantics.
I guess I have to say thank you Meta? A somewhat sad rant below. Deepseek starts a toxic trend of providing super, super large MoE. And MoE is famous for being parameter-inefficient, which is unfriendly to normal consumer hardware with limited vram. The super large size of LLM also disables nearly every people from doing meaningful development on these models. R1-1776 is the only fine-tune variation of R1 that makes…
Earlier quoted context omitted.
If you were looking for truth you wouldn’t reply like this. I’m not going to do an hour of work to carefully cite this for you, but it’s true nonetheless.
It is yours to provide evidence of your claims, not mine. >If you were looking for truth Except, with this, I don’t expect you to.
This is a common weird mistake people make on HN - I'm not publishing a paper so, no I don't. Really there's minimal rules of engagement here. You could say you think I'm wrong, which I'd be curious to hear why.
It's more productive to first discuss things casually, and then if there's specific disagreements to dig in. If you disagree with my statement, please tell me which countries you think specifically I'm more likely wrong about. You don't need to cite anything, either do I. If we actually do disagree, then we can go off and do our own research, or if we're really motivated bring it back here.
But there's no burden for anything, and it's actually better in many cases to first chat before we dig in and try and out-cite each other.
Earlier quoted context omitted.
It is yours to provide evidence of your claims, not mine. >If you were looking for truth Except, with this, I don’t expect you to.
> It is yours to provide evidence of your claims, not mine. This is a common weird mistake people make on HN - I'm not publishing a paper so, no I don't. Really there's minimal rules of engagement here. You could say you think I'm wrong, which I'd be curious to hear why. It's more productive to first discuss things casually, and then if there's specific disagreements to dig in. If you disagree with my statement, plea…
I don’t think that this thread is worth any more spent energy from either of us.
Earlier quoted context omitted.
yes this is great but I'd like to pick a different voice. the current one feels too robotic
Same, it was using the high quality openai voice until my account ran out of funds.. Now it's using edge-tts which is free. So far it seems like the best option in terms of price/performance, but I'm happy to switch it up if something better comes along.
What about putting the text version that's used to make the audio somewhere on the page? (or better, on a subpage where there's no audio playback)
"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.
Call me crazy, but I don't want an AI that bases its reasoning on politics. I want one that is primarily scientific driven, and if I ask it political questions it should give me representative answers. E.g. "The majority view in [country] is [blah] with the minority view being [bleh]." I have no interest in "all sides are equal" answers because I don't believe all information is equally informative nor equally true.
Earlier quoted context omitted.
It's kinda hilarious to see people claiming that the wall has been hit for the past two years, while evals are creeping up each month, particularly realistic end-to-end SWE-bench. Have you compared GPT-4.5 to 4o? GPT-4.5 just knows things. Some obscure programming language? It knows the syntax. Obviously, that's not sufficient - you also need reasoning, post-training, etc. so quite predictably G2.5P being a large mod…
Ever heard about benchmark contamination? Ever tried to explain a new concept, like a new state management store for web frontend? Most fail spectacularly there, sonnet 3.7 I had reasonable ""success"" with, but not 4.5. It faltered completely. Let’s not get ahead of ourselves. Looking at training efficiency in this now, and all the other factors, it really is difficult to paint a favorable picture atm.