Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

z.ai

Post reply on HN
Print view View original

131–140 of 540 posts

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#131

Post by vessenes »

NiloCK wrote: post #29

Grey market fast-follow via distillation seems like an inevitable feature of the near to medium future. I've previously doubted that the N-1 or N-2 open weight models will ever be attractive to end users, especially power users. But it now seems that user preferences will be yet another saturated benchmark, that even the N-2 models will fully satisfy. Heck, even my own preferences may be getting saturated already. Op…

Just to say - 4.6 really shines on working longer without input. It feels to me like it gets twice as far. I would not want to go back.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#132

Post by ac29 »

Aurornis wrote: post #19
NiloCK wrote: post #14

Earlier quoted context omitted.

> Didn't expect to go back to macOS but their basically the only feasible consumer option for running large models locally. I presume here you are referring to running on the device in your lap. How about a headless linux inference box in the closet / basement? Return of the home network!

Apple devices have high memory bandwidth necessary to run LLMs at reasonable rates. It’s possible to build a Linux box that does the same but you’ll be spending a lot more to get there. With Apple, a $500 Mac Mini has memory bandwidth that you just can’t get anywhere else for the price.

> a $500 Mac Mini has memory bandwidth that you just can’t get anywhere else for the price.

The cheapest new mac mini is $600 on Apple's US store.

And it has a 128-bit memory interface using LPDDR5X/7500, nothing exotic. The laptop I bought last year for <$500 has roughly the same memory speed and new machines are even faster.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#133

Post by HumanOstrich »

Imustaskforhelp wrote: post #106
sowbug wrote: post #97

Earlier quoted context omitted.

In Silicon Valley we pay PG&E close to 50 cents per kWh. An RTX 6000 PC uses about 1 kW at full load, and renting such a machine from vast.ai costs 60 cents/hour as of this morning. It's very hard for heavy-load local AI to make sense here.

And you are forgetting the fact that things like vast.ai subscriptions would STILL be more expensive than Openrouter's api pricing and even more so in the case of AI subscriptions which actively LOSE money for the company. So I would still point out the GP (Original comment) where yes, it might not make financial sense to run these AI Models [They make sense when you want privacy etc, which are all fair concerns but…

Why did you quote yourself at the end of this comment?

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#134

Post by BeetleB »

amenhotep wrote: post #98
jaccola wrote: post #54

Earlier quoted context omitted.

I have no idea how an LLM company can make any argument that their use of content to train the models is allowed that doesn't equally apply to the distillers using an LLM output. "The distilled LLM isn't stealing the content from the 'parent' LLM, it is learning from the content just as a human would, surely that can't be illegal!"...

When you buy, or pirate, a book, you didn't enter into a business relationship with the author specifically forbidding you from using the text to train models. When you get tokens from one of these providers, you sort of did. I think it's a pretty weak distinction and by separating the concerns, having a company that collects a corpus and then "illegally" sells it for training, you can pretty much exactly reproduce t…

> When you buy, or pirate, a book, you didn't enter into a business relationship with the author specifically forbidding you from using the text to train models.

Nice phrasing, using "pirate".

Violating the TOS of an LLM is the equivalent of pirating a book.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#135

Post by d3k »

zozbot234 wrote: post #60
ingenieroariel wrote: post #50

Earlier quoted context omitted.

With Apple devices you get very fast predictions once it gets going but it is inferior to nvidia precisely during prefetch (processing prompt/context) before it really gets going. For our code assistant use cases the local inference on Macs will tend to favor workflows where there is a lot of generation and little reading and this is the opposite of how many of use use Claude Code. Source: I started getting Mac Studi…

All Apple devices have a NPU which is potentially able to save power for compute bound operations like prefill (at least if you're ok with FP16 FMA/INT8 MADD arithmetic). It's just a matter of hooking up support to the main local AI frameworks. This is not a speedup per se but gives you more headroom wrt. power and thermals for everything else, so should yield higher performance overall.

AFAIK, only CoreML can use Apple's NPU (ANE). Pytorch, MLX and the other kids on the block use MPS (the GPU). I think the limitations you mentioned relate to that (but I might be missing something)

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#136

Post by cbg0 »

Havoc wrote: post #89
CodeCompost wrote: post #68

[flagged]

You're surprised that chinese model makers try to follow chinese law?

This is a classic test to see if the model is censored, as censorship is rarely limited to just one event, which begs the question: what else is censored or outright changed intentionally?

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#138

Post by NitpickLawyer »

AntonyGarand wrote: post #114
NitpickLawyer wrote: post #55

Earlier quoted context omitted.

> It's avaiable in mine Weird, mine (lite plan) says "Only supports GLM-4.7, GLM-4.6, GLM-4.5, and GLM-4.5-Air" and "Get same-tier model updates" ...

The documentation is not updated, but it works if you hardcode the model id to `GLM-5` within your tool

Cool, thanks. Did you try it out, how's the performance? I saw on openrouter that the stealth model was served at ~19t/s. Is it any better on their endpoints?

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#139

Post by BeetleB »

alias_neo wrote: post #95
Mashimo wrote: post #3

Wut? Was glm 4.7 not just a few weeks ago? I wonder if I will be able to use it with my coding plan. Paid just 9 usd for 3 month.

What's the use case for Zai/GLM? I'm currently on Claude Pro, and the Zai looks about 50% more expensive after the first 3 months and according to their chart GLM 4.7 is not quite as capable as Opus 4.5? I'm looking to save on costs because I use it so infrequently, but PAYG seems like it'd cost me more in a single session per month than the monthly cost plan.

If you pay for the whole year, GLM4.7 is only $7/mo for the first year. And until a few days ago, they had a fantastic deal that ran for almost 2 months where it was less than $3/mo for the first year. I grabbed it, and have been using it exclusively for personal coding since. It's good enough for me.

The other claimed benefit is a higher quota of tokens.

Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks

#140

Post by falcor84 »

CodeCompost wrote: post #68

[flagged]

I tried to go about it in a bit of a roundabout way, as a followup question in a longer conversation and was able to get this in the thought process before it froze:

> Step 2: Analyze the Request The user is asking about the events in Tiananmen Square (Beijing, China) in 1989. This refers to the Tiananmen Square protests and subsequent massacre.

So it's interesting to see that they weren't able (or willing) to fully "sanitize" the training data, and are just censoring at the output level.

Post reply on HN