Can I set a hard cost limit per day ? With no drift, else I'm not interested.
I think you should look at OpenRouter. It has budget controls
Cloudflare's AI Platform: an inference layer designed for agents
41–50 of 106 posts
Re: Cloudflare's AI Platform: an inference layer designed for agents
#42Not seeing any pricing info on the models[1] page. Wonder how much of a lift this is over paying providers directly. Perhaps Cloudflare is doing this at cost? Also interesting that zero data retention is not on by default, and is not supported with all providers[2]. Finally, would be great if this could return OpenAI AND Anthropic style completions. [1] https://developers.cloudflare.com/ai/models/ [2] https://develop…
Hey! I'm one of the engineers who built this :) We'll be adding prices to the docs and the model catalog in the dashboard shortly. In short: currently the pricing matches whatever the provider charges. You can buy unified billing credits [1] which charges a small processing fee. > Finally, would be great if this could return OpenAI AND Anthropic style completions. Agreed! This will be coming shortly. Currently we'll…
excellent! please make sure to include rate limit details as well.
Re: Cloudflare's AI Platform: an inference layer designed for agents
#43This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.
> For those who don’t use Workers, we’ll be releasing REST API support in the coming weeks, so you can access the full model catalog from any environment. Cloudflare seems to be building for lock-in and I don't love it. I especially don't understand how you build an OpenRouter and only have bindings for your custom runtime at launch.
Workers runtime is open source and permissively licensed fwiw
Re: Cloudflare's AI Platform: an inference layer designed for agents
#44This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.
Yeah but the 10GB limit for D1 is crazy, can you really start building on that? Other than toy projects?
Re: Cloudflare's AI Platform: an inference layer designed for agents
#45I find it really confusing that the worker AI models on here: https://developers.cloudflare.com/workers-ai/models/ do not have full overlap with the ones on here: https://developers.cloudflare.com/ai/models/ Yes, you can see the same "hosted" ones on there, but when you look at the models endpoint, there are much less options at the "workers-ai/*" namespace. Is that intentional?
To better clarify, I don’t see "workers-ai/@cf/google/gemma-4-26b-a4b-it" in the /models enpoint in gateway.ai.cloudflare.com but it does seem to exist as a hosted model. Same with "workers-ai/@cf/nvidia/nemotron-3-120b-a12b" which I would expect to see
Hey James.
Thanks for the feedback, and good catch. Looks like that endpoint is pulling from a slightly out of date data source. The docs/dashboard currently are the best resources for the full catalog, but we'll update that API to match.
Re: Cloudflare's AI Platform: an inference layer designed for agents
#46This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.
Agreed -- except that all of their docs and marketing pitches it for use cases like "per-user, per-tenant or per-entity databases" -- which would be SO great. But in practice, it's basically impossible to use that way in conjunctions with workers, since you have to bind every database you want to use to the worker and binding a new database requires redeploying the worker.
If you want to dynamically create sqlite databases, then moving to durable objects which are each backed by an sqlite database seems to be the way to go currently.
Re: Cloudflare's AI Platform: an inference layer designed for agents
#47This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.
Yeah but the 10GB limit for D1 is crazy, can you really start building on that? Other than toy projects?
Really depends on what you’re putting in the DB. Cloudflare is clear that these are supposed to be very localized DBs. Per user or tenant.
Re: Cloudflare's AI Platform: an inference layer designed for agents
#48So, is this similar to openrouter?
Re: Cloudflare's AI Platform: an inference layer designed for agents
#49So it's basically just openrouter with cloudflare argo networking? I feel like they could do some much more interesting stuff with their replicate acquisition. Application specific RL is getting so good but there's no good way to deploy these models in a scalable way. Even the providers like fireworks which claim to let you deploy LORAs in a scalable way can't do it. For now I literally have to host base load on my application on a rack of 3090s in my garage which seems silly but it saves me $1k a month.
Re: Cloudflare's AI Platform: an inference layer designed for agents
#50So, is this similar to openrouter?
with Argo networking