Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare's AI Platform: an inference layer designed for agents

blog.cloudflare.com

Post reply on HN
Print view View original

31–40 of 106 posts

Re: Cloudflare's AI Platform: an inference layer designed for agents

#31

Post by nubg »

6thbit wrote: post #10
koolba wrote: post #9

Earlier quoted context omitted.

> i presume they wont let you “manage all your AI spend in one place” for free. Of course they will. In return they get to control who they’re routing requests to. I wouldn’t be surprised if this turns I to the LLM equivalent of “paying for order flow”.

i got shivers thinking about a future ai dynamic pricing and automatic gateway choosing the cheapest provider available

shivers? as in it frightens you? i believe there is no way around tokens being prices like gasoline at the gas station - it changes every hour. Any other system means you are either over- or underspending.

Re: Cloudflare's AI Platform: an inference layer designed for agents

#33

Post by james2doyle »

mbtrucks wrote: post #19

Can I set a hard cost limit ? Else I'm not interested, don't be like googles mess of billing.

Seems like it. I just added some credits to my account. You can set a daily $ spend limit as well as add credits without auto-refill

Re: Cloudflare's AI Platform: an inference layer designed for agents

#34

Post by james2doyle »

wahnfrieden wrote: post #11

No spending limit / no ability to set a budget, unlike Google or OpenAI. Be prepared for an eye-watering invoice if you have a bug or get hacked. edit: Why downvote? It's correct, and it's a risk that competitors handle better, including for their CDN products (compared to Bunny CDN). Maybe you are just used to the risk and haven't felt the burn yourself yet. Or you have the mistaken notion that there is no price at…

I just added some credits to my account. You can set a daily $ spend limit as well as add credits without auto-refill

Re: Cloudflare's AI Platform: an inference layer designed for agents

#35

Post by james2doyle »

I find it really confusing that the worker AI models on here: https://developers.cloudflare.com/workers-ai/models/ do not have full overlap with the ones on here: https://developers.cloudflare.com/ai/models/

Yes, you can see the same "hosted" ones on there, but when you look at the models endpoint, there are much less options at the "workers-ai/*" namespace. Is that intentional?

Re: Cloudflare's AI Platform: an inference layer designed for agents

#37

Post by BoorishBears »

whereistejas wrote: post #13

This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.

> For those who don’t use Workers, we’ll be releasing REST API support in the coming weeks, so you can access the full model catalog from any environment.

Cloudflare seems to be building for lock-in and I don't love it. I especially don't understand how you build an OpenRouter and only have bindings for your custom runtime at launch.

Re: Cloudflare's AI Platform: an inference layer designed for agents

#38

Post by james2doyle »

james2doyle wrote: post #35

I find it really confusing that the worker AI models on here: https://developers.cloudflare.com/workers-ai/models/ do not have full overlap with the ones on here: https://developers.cloudflare.com/ai/models/ Yes, you can see the same "hosted" ones on there, but when you look at the models endpoint, there are much less options at the "workers-ai/*" namespace. Is that intentional?

To better clarify, I don’t see "workers-ai/@cf/google/gemma-4-26b-a4b-it" in the /models enpoint in gateway.ai.cloudflare.com but it does seem to exist as a hosted model. Same with "workers-ai/@cf/nvidia/nemotron-3-120b-a12b" which I would expect to see

Re: Cloudflare's AI Platform: an inference layer designed for agents

#39

Post by eis »

whereistejas wrote: post #13

This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.

D1 reliability has been bad in our experience. We've had queries hanging on their internal network layer for several seconds, sometimes double digits over extended periods (on the order of weeks). Recently I've seen a few times plain network exceptions - again, these are internal between their worker and the D1 hosts. And many of the hung queries wouldn't even show up under traces in their observability dashboard so unless you have your own timeout detection you wouldn't even know things are not working. It was hard to get someone on their side to take a look and actually acknowledge and understand the problem.

But even without network issues that have plagued it I would hesitate to build anything for production on it because it can't even do transactions and the product manager for D1 openly stated they wont implement them [0]. Your only way to ensure data consistency is to use a Durable Object which comes with its own costs and tradeoffs.

https://github.com/cloudflare/workers-sdk/issues/2733#issuec...

The basic idea of D1 is great. I just don't trust the implementation.

For a hobby project it's a neat product for sure.

Re: Cloudflare's AI Platform: an inference layer designed for agents

#40

Post by allthetime »

stult wrote: post #20

A few weeks ago, I ran into a bug with Cloudflare's DNS server not detecting when I updated the records with the registrar. The bug was 100% on their end, entirely unsolvable by me, yet they have made it literally impossible to contact them to file a bug report. Their standard user help workflow dead-ended by forcing me to talk to their absolutely useless AI help chatbot, which proceeded to regurgitate their FAQ (ina…

What was the bug? I configure DNS for both public and private networks on cloudflare semi-frequently and always see changes in minutes or less.
Post reply on HN