Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Workers Cache

blog.cloudflare.com

Post reply on HN
Print view View original

71–80 of 87 posts

Re: Workers Cache

#71

Post by kentonv »

kentonv wrote: post #67
simonw wrote: post #39

> This is the caching API we've always wanted Workers to have. Here's why it took us this long I was looking forward to the "why it took us this long" explanation but it wasn't explicitly spelled out. Any Cloudflare staff here able to expand on that? (The article does a good job of showing how many different smart design decisions went into this, but given caching is core to what a Cloudflare does I'm still a little…

The honest answer is: We implemented the "standard" Cache API back in the early days because it was a standard, one which was intended to be used together with the Service Workers API, which we were also building around back then. But it was never a good fit. The get/put API was designed for a local browser cache, not a distributed cache like Cloudflare's. We probably should have realized this before implementing it,…

Realizing there's a more fundamental piece I'm not explaining well here.

Our architecture is something like:

ingress -> routing -> security -> workers -> cache -> origin

That is, workers run "in front of" cache. That's usually what you want. Workers run on the edge, so putting them in front of cache doesn't cost anything in terms of latency, and there's a lot of useful stuff you can do only if they run in front, e.g. serving pages from cache but customizing them for specific users.

Putting workers behind cache is an architectural change. All of the logic around routing to them lives in front of the cache. And it makes Workers a lot less useful.

We weren't that excited about it until we had a clearer story for how to run custom logic on both sides of the cache, but it was pretty unclear how to do that in a nice way until the recent developments around ctx.props, ctx.exports, channel tokens, etc.

Re: Workers Cache

#72

Post by pizzafeelsright »

swiftcoder wrote: post #40
topgrain2 wrote: post #17

Earlier quoted context omitted.

Fuck, I spent all these years developing a thoughtful writing style that leaned toward clarity for the reader, even if it meant extra work to achieve precision, or adding affordances like “excessive” hyphenation, and now I guess have to learn to write worse.

It does feel a bit like the LLMs have commoditised correct writing form, and all the plebs are all up in arms about it...

I read the slop and each has a smell. Each model and company does because the humans behind it have their own taste, smell, and perspective.

The human element cannot be recreated because the human element that created the beast becomes further removed and only the beast remains.

I say beast to provide a tell that what I write is 'human'.

Re: Workers Cache

#73

Post by ascorbic »

dbbk wrote: post #58

I was a bit confused what this adds other than just standard CDN-Cache-Control page caching that we do now. Some quirks I've found; - You still get billed per request, whether the request hits cache or not (but don't get billed for CPU time) - You now get billed for static asset requests! This makes no sense to me. "One thing to watch: when caching is enabled, requests that are normally free — static asset requests a…

> I was a bit confused what this adds other than just standard CDN-Cache-Control page caching that we do now.

Until now these cache headers didn't work if you set them in a Worker, because Workers were always run in front of the cache. You could use them if you're running Cloudflare cache in front of another origin, but not if you were e.g. rendering a site in Workers. This changes that.

Re: Workers Cache

#74

Post by lijok »

davidmurdoch wrote: post #48
lijok wrote: post #29

Earlier quoted context omitted.

I don’t have access to my flavor of LLM on the train nor the time or budget to have it do the research and summary for me Why are we all of a sudden pretending like pre-LLM era blogs were these pristinely well written pieces of art or even that effort and care was put into them? In most cases they were significantly less coherent and incomplete. Don’t get me started on the mess that was the communication of this part…

Because we add humans enjoy variety. I read for entertainment, even technical posts like this one that I have no use for. I often trying to think about what the author may have been thinking when writing, why they introduced concepts in a specific order, what ideas might they have omitted, etc. It's personal and enjoyable. But now, when I detect the familiar writing style of what seems to be a gpt 5 model, that "para…

I hear you, and agree with your points, but do have to ask - what’s the point in complaining? See LLM slop as you called it, ignore it and move on. There’s plenty of more intimately authored content out there.

Re: Workers Cache

#75

Post by lijok »

geraneum wrote: post #52
lijok wrote: post #29

Earlier quoted context omitted.

I don’t have access to my flavor of LLM on the train nor the time or budget to have it do the research and summary for me Why are we all of a sudden pretending like pre-LLM era blogs were these pristinely well written pieces of art or even that effort and care was put into them? In most cases they were significantly less coherent and incomplete. Don’t get me started on the mess that was the communication of this part…

>I don’t have access to my flavor of LLM on the train nor the time or budget to have it do the research and summary for me Aren't you gonna let the LLM develop for you anyway? Why bother writing and reading a post at all? > pretending like pre-LLM era blogs were these pristinely well written pieces of art The point is the effort and care that the writer puts which differentiates it from automatically generated text.…

Honestly sounds to me like you have a bone to pick with AI in general. Negative sentiment on a particular LLM authored blog post muddies the waters and misrepresents your valid points. Not the way to do this IMO. Happy to be corrected.

Re: Workers Cache

#76

Post by drdexebtjl »

kentonv wrote: post #71
kentonv wrote: post #67

Earlier quoted context omitted.

The honest answer is: We implemented the "standard" Cache API back in the early days because it was a standard, one which was intended to be used together with the Service Workers API, which we were also building around back then. But it was never a good fit. The get/put API was designed for a local browser cache, not a distributed cache like Cloudflare's. We probably should have realized this before implementing it,…

Realizing there's a more fundamental piece I'm not explaining well here. Our architecture is something like: ingress -> routing -> security -> workers -> cache -> origin That is, workers run "in front of" cache. That's usually what you want. Workers run on the edge, so putting them in front of cache doesn't cost anything in terms of latency, and there's a lot of useful stuff you can do only if they run in front, e.g.…

Does that new architecture make it more expensive to serve worker’s static assets and do worker-to-worker invocations? Or just harder to measure?

This change in billing makes enabling caching not such a straightforward decision, and encourages separating cached and non-cached parts of your workers into two separate workers, which is a bit annoying.

Re: Workers Cache

#77

Post by davidmurdoch »

lijok wrote: post #74
davidmurdoch wrote: post #48

Earlier quoted context omitted.

Because we add humans enjoy variety. I read for entertainment, even technical posts like this one that I have no use for. I often trying to think about what the author may have been thinking when writing, why they introduced concepts in a specific order, what ideas might they have omitted, etc. It's personal and enjoyable. But now, when I detect the familiar writing style of what seems to be a gpt 5 model, that "para…

I hear you, and agree with your points, but do have to ask - what’s the point in complaining? See LLM slop as you called it, ignore it and move on. There’s plenty of more intimately authored content out there.

Why not complain?

Re: Workers Cache

#78

Post by drdexebtjl »

dbbk wrote: post #58

I was a bit confused what this adds other than just standard CDN-Cache-Control page caching that we do now. Some quirks I've found; - You still get billed per request, whether the request hits cache or not (but don't get billed for CPU time) - You now get billed for static asset requests! This makes no sense to me. "One thing to watch: when caching is enabled, requests that are normally free — static asset requests a…

> Yeah that sounds like a bug that just happens to generate them more money.

Yeah. It sounds like they moved the component that counts requests earlier in the pipeline so that it can count cache hits, and now it also counts these.

Re: Workers Cache

#79

Post by kentonv »

drdexebtjl wrote: post #76
kentonv wrote: post #71

Earlier quoted context omitted.

Realizing there's a more fundamental piece I'm not explaining well here. Our architecture is something like: ingress -> routing -> security -> workers -> cache -> origin That is, workers run "in front of" cache. That's usually what you want. Workers run on the edge, so putting them in front of cache doesn't cost anything in terms of latency, and there's a lot of useful stuff you can do only if they run in front, e.g.…

Does that new architecture make it more expensive to serve worker’s static assets and do worker-to-worker invocations? Or just harder to measure? This change in billing makes enabling caching not such a straightforward decision, and encourages separating cached and non-cached parts of your workers into two separate workers, which is a bit annoying.

I wasn't personally involved in pricing here so I can't really say exactly why it is the way it is.

However, I'd note that, in terms of our costs, it's often cheaper for us to run your Worker than to serve from cache. Consider that Workers run locally in whatever edge location the request landed in. We are not saving any bandwidth costs by putting the cache in front of Workers rather than behind, and we've built an architecture such that running a Worker is extremely cheap, almost free.

In that light, making cache hits free would basically be giving away money. To be honest, that is another reason why implementing cache in front of Workers has been held back so long. I think the team finally decided that the only way we can deliver it without giving away too much money is to charge full price for the requests. Workers pricing is already incredibly generous, and we do need to run a business at the end of the day.

(Of course, a cache hit still saves you the CPU pricing. CPU time pricing more directly translates to costs for us, so that makes sense.)

I admit the side effect on static assets is a little weird. But no other provider offers unlimited free static asset serving to start with -- so it's only weird relative to our own unreasonable standards. :) I think the thought here is that if you need caching in front of Workers, that implies you are doing complex work in Workers and getting a lot of value out of them. So a bit of "price segmentation" kicks in here. That said, you can work around it by serving static assets from another hostname, etc.

Re: Workers Cache

#80

Post by sidcool »

dbbk wrote: post #58

I was a bit confused what this adds other than just standard CDN-Cache-Control page caching that we do now. Some quirks I've found; - You still get billed per request, whether the request hits cache or not (but don't get billed for CPU time) - You now get billed for static asset requests! This makes no sense to me. "One thing to watch: when caching is enabled, requests that are normally free — static asset requests a…

CPU time is not billed
Post reply on HN