Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare's AI Platform: an inference layer designed for agents

blog.cloudflare.com

Post reply on HN
Print view View original

71–80 of 106 posts

Re: Cloudflare's AI Platform: an inference layer designed for agents

#71

Post by brikym »

whereistejas wrote: post #13

This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.

There is always one thing that bites you because Cloudflare is different. I just built an AI game (sleuththetruth.com) and the primary reason it's so slow to prompt a new board is actually not because of AI latency. It's because CF workers have a limit of 6 connections (including spawned workers). There is no way to gulp down all the wiki images I want all at once. If I had put the backend on Railway I don't think I'd have this issue.

Re: Cloudflare's AI Platform: an inference layer designed for agents

#72

Post by handfuloflight »

bryden_cruz wrote: post #70
mips_avatar wrote: post #49

So it's basically just openrouter with cloudflare argo networking? I feel like they could do some much more interesting stuff with their replicate acquisition. Application specific RL is getting so good but there's no good way to deploy these models in a scalable way. Even the providers like fireworks which claim to let you deploy LORAs in a scalable way can't do it. For now I literally have to host base load on my a…

Running a rack of 3090s in your garage to avoid provider lock-in/costs is the most Hacker News thing. Out of curiosity, what are you doing for uptime/failover? If you are running production traffic to that garage rack, does your app just degrade gracefully if your home internet drops, or do you have a cloud fallback?

[flagged]

Re: Cloudflare's AI Platform: an inference layer designed for agents

#74

Post by jillesvangurp »

rs_rs_rs_rs_rs wrote: post #44
whereistejas wrote: post #13

This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.

Yeah but the 10GB limit for D1 is crazy, can you really start building on that? Other than toy projects?

Most website content management systems would never get close to that size. If you need a bigger database, D1 is probably the wrong solution to begin with. 10GB can be millions of records depending on your table structure. But if you are gathering some survey data, running a CMS, etc. you probably should be fine with even just a few MB of data; which is probably the sweet spot for D1.

Re: Cloudflare's AI Platform: an inference layer designed for agents

#77

Post by hemangjoshi37a »

The interesting question isn't "can CF run agent inference" — it's what the routing layer needs to look like for multi-turn workflows. Shipping agent systems to enterprise clients the last year, the bottleneck is never raw tokens/sec. It's (a) state checkpointing betweentool calls, (b) cold-start latency on embedding/rerank models, (c) rate-limit coordination across concurrent agent loops. Does CF expose per-session state, or still stateless-per-request? Without that, you end up building the interesting part yourself.

Re: Cloudflare's AI Platform: an inference layer designed for agents

#78

Post by ncrmro »

whereistejas wrote: post #13

This actually looks very useful. Cloudflare seems to be brining together a great set of tools. Not to mention, D2 is literally the only sqlite-as-a-service solution out there whose reliability is great and free tier limits are generous.

Turso/libsql has been great for poc project so far

Re: Cloudflare's AI Platform: an inference layer designed for agents

#80

Post by __jonas »

strimoza wrote: post #79

Interesting timing — I've been using Bunny CDN for video delivery and considering moving parts to Cloudflare. Anyone have experience comparing the two for media streaming specifically?

Wondering why you're considering this move, I also use Bunny for some embedded videos and am considering fully moving my websites away from Cloudflare's CDN to Bunny
Post reply on HN