501–510 of 559 posts
Earlier quoted context omitted.
Is there a secure way to use GLM without spending $10K’s for local HW? I “only” have a 128GiB inference machine, and don’t really trust anthropic not to steal my IP over time. I see no reason to trust Z.ai more than other vendors.
You'd need to multiply that $10k by 8 minimum.
Earlier quoted context omitted.
GLM 5.2 is a great model, but if you only want to use the best model available, it isn't there yet. Every lab releases models that memorize benchmark answers, both intentionally and unintentionally. But we consistently find that models from Chinese labs have a wider gap between public benchmarks and our evaluations, which we designed to be less vulnerable to benchmaxxing. In multi-agent coding environments, GLM 5.2 i…
> but if you only want to use the best model available, it isn't there yet I'm trying to wrap my head around exactly why so may people seem to want the best model available when it has recently become clear that most halfway decent models can write damn good code for a fraction of the price. And the frontier models get nerfed constantly so you with open weight you can get something slightly less performant but way mo…
Meanwhile, it's such a cheap model that I've spent not even $25 over 3 weeks.
Earlier quoted context omitted.
> most halfway decent models can write damn good code for a fraction of the price. The difference is how the model is used. With Opus you can give it a long-horizon task (eg build an entire feature) and it will plan it out and implement it and almost always stay on task. This is what people mean when they say "agentic tasks" With the lessor models the code is fine, but they need something else to plan what needs to b…
My whole point is that I don't want it to build an entire feature from one prompt. At most, I want to work with an agent to nail down the spec and then work with an agent that orchestrates the implementation via other agents, same for testing, etc. None of that requires frontier capabilities, it requires a little bit of work on a harness, a little bit more of my input, a little more of my brainpower. I _want_ to buil…
You are free to do you. But you were asking about why others want the best model.
The answer is, clearly, agentic coding (ie multiple agents each cranking through tasks independently) lets you ship A LOT more business value if used correctly.
You can launch GLM-5.2 in Opencode using Nemesis8: https://github.com/DeepBlueDynamics/nemesis8#nemesis-8 After installing, do a `n8 build` to build the image, then `n8 --danger --provider opencode interactive` to launch it in a container. Signup for GLM-5.2 here: https://z.ai
One can also try https://neuralwatt.com using it in opencode. I think they give $5 trail credits to test with any of the open weight models.
Earlier quoted context omitted.
GLM 5.2 is a great model, but if you only want to use the best model available, it isn't there yet. Every lab releases models that memorize benchmark answers, both intentionally and unintentionally. But we consistently find that models from Chinese labs have a wider gap between public benchmarks and our evaluations, which we designed to be less vulnerable to benchmaxxing. In multi-agent coding environments, GLM 5.2 i…
> but if you only want to use the best model available, it isn't there yet I'm trying to wrap my head around exactly why so may people seem to want the best model available when it has recently become clear that most halfway decent models can write damn good code for a fraction of the price. And the frontier models get nerfed constantly so you with open weight you can get something slightly less performant but way mo…
The reason is pretty simple and has to do with statistics: on long-horizon tasks, small errors and deviations from the "good path" compound.
Earlier quoted context omitted.
Hi I'm the CTO of neuralwatt, would love to hear your feedback on what your experience was. Feel free to email me scott@neuralwatt.com. Also for GLM5.2 we run the FP8 quantization at 1M context which is a common deployment target.
Hi Scott! Was just considering signing up, NW looks great (fp8 GLM 5.2 is good !) Standard cached token pricing for GLM 5.2 is pretty high, I'm wondering whether the KV cache for that model actually is that expensive to serve on average, or if Neuralwatt's energy pricing for long-running GLM 5.2 agents is especially competitive? The live energy stats don't break down by token type, would love to see that. And 2/3 of…
We definitely don't have any intention to obfuscate and in fact we actually try and provide more data than any other provider out there about both an individual request, as well as the fleet behavior. Since we tend to focus directly on our energy pricing and optimizing that the issue is likely where the ROI lies on energy optimization versus token optimization (totally correlated but we have other levers to reduce energy while keeping token counts the same).
Earlier quoted context omitted.
What are the challenges you've seen in selling air gapped? Is it the high upfront cost? Challenges with hardware maintenance or something else?
We already use AWS. Everyone else is using AWS, so if there's an issue we can just say we were following industry standards.
Earlier quoted context omitted.
US EVs were also heavily subsidized, but they were all built using Chinese parts.
US EVs were "lightly" subsidized compared to what the Chinese govt has done. In the ballpark of 250 billion dollars by the Chinese vs maybe 10% of that by the US.
In other words, they're not subsidies for Chinese cars being exported abroad. They're not even directly paid to the manufacturers.
Earlier quoted context omitted.
You'd need to multiply that $10k by 8 minimum.
Why? 4 x DGX sparks should be enough. That's way less than $80k.
For real work anything below 60 tokens per second is essentially unusable. That's not taking into account the prompt filling, Llama 3.1. 70b on DGX spark runs at about 800 tps running at that speed prompt filling a 512k context takes like 11 minutes.
Earlier quoted context omitted.
US EVs were "lightly" subsidized compared to what the Chinese govt has done. In the ballpark of 250 billion dollars by the Chinese vs maybe 10% of that by the US.
Note that most of those subsidies are things like sales-tax exemptions for EVs and support for charging infrastructure in China. In other words, they're not subsidies for Chinese cars being exported abroad. They're not even directly paid to the manufacturers.