A bit offtopic: could you guys move the "please enable javascript"/captcha page to a separate, but cloudflare owned domain, so at least people who don't want to enable javascript for random websites behind cloudflare could enable it for your domain and pass all your browser checks?
Introducing Workers KV
81–90 of 103 posts
Re: Introducing Workers KV
#82Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?
Can you implement IPC with this system? It seems wrong to relate this to tuple spaces otherwise.
Re: Introducing Workers KV
#83Earlier quoted context omitted.
Indeed. This is just stage 1 of our storage plans. We have some really cool stuff in the works but we wanted to get basic KV functionality out there for people to start using ASAP.
OT: we need to read more about your website-in-kv-thingy setup.
It was pretty trivial. I just stored each page into a value and served them out of there, with some stupid code to infer content-type from file extension. I meant it as a dumb demo because I wasn't feeling creative. I didn't know John would mention it in his blog. :)
Currently this isn't a great way to serve content because the values are limited to 64k, you have to infer content-type (which makes your URLs ugly), etc. So I don't recommend it. We'll come up with something better in the future. :)
Re: Introducing Workers KV
#84Earlier quoted context omitted.
The ultimate goal is to let you do all of that within our network! When you say overhead limit, are you talking about latency? Our goal is to keep reads on the order of 5ms in the 90th percentile.
I mean compute/memory for the workers... just a rough comparison of what that might look like. Will probably sign up for the early access and play around with it soon.
We limit you to 50ms of CPU and 128 MB of memory now, but we're working on ways of raising those limits.
Re: Introducing Workers KV
#85Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?
Can you implement IPC with this system? It seems wrong to relate this to tuple spaces otherwise.
Put the target channel name and a message id at the front of each key and you can use a tuple space for IPC.
Re: Introducing Workers KV
#86Earlier quoted context omitted.
Can you talk more about your specific use case?
The simplest example is liveblogging - someone is sitting at the next iPhone launch, and sending out photos and commentary that's being updated as they send on a large number of readers' screens worldwide. The Verge has a good system that they do with polling S3 for new data, but there's really much more effective ways to do it. The more specific cases include opening ticket sales for movies in India - for some big m…
> Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity.
Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss.
I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. IMO connection management belongs as a separate layer in front of the worker, so workers are only woken when things happen.
And on that note, of course you can use Fanout together with Cloudflare Workers today. At the moment this requires making raw API calls but we plan to update our server libs for CF compatibility soon.
Re: Introducing Workers KV
#87Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?
For anyone curious about this, it probably all comes down to an atomic compare and swap request:
https://en.wikipedia.org/wiki/Compare-and-swap
It might be straightforward to do this with an ETag like Firebase:
https://firebase.googleblog.com/2017/07/introducing-conditio...
Not sure what this would require on their backend though.
Re: Introducing Workers KV
#88Earlier quoted context omitted.
The simplest example is liveblogging - someone is sitting at the next iPhone launch, and sending out photos and commentary that's being updated as they send on a large number of readers' screens worldwide. The Verge has a good system that they do with polling S3 for new data, but there's really much more effective ways to do it. The more specific cases include opening ticket sales for movies in India - for some big m…
> Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity. Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. IMO connection mana…
> Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss.
Hi! Wasn't expecting you to show up like this. Re. pricing for high volume, I'm sure enterprises and publishers could afford it, I'm trying to see if it's possible to offer free or very cheap service to individuals with large audiences. So not so much that I want a high volume discount, I'm trying to figure out a way to make it one or two orders of magnitude cheaper, enough that pricing is no longer relevant. Self hosting Pushpin is an option, but CF might be easier to work with.
> I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too.
The CF edges already hold / passthrough websockets - whether the Workers API is conducive to controlling them is the question, but we know it already allows streaming - doesn't seem like big jump from there. And for my purposes one-way streaming is enough, so might not even need to change the API. And based on the current billing model it seems like only CPU time is billed (?) - so if the socket is waiting most of the time it's still cost effective. And connection management itself would still be outside the scope of the worker - I'm saying the worker's event loop should behave exactly the way it would if I using the streaming feature right now to make a 1000 slow network requests from the worker and streamed the responses out as the worker received them. Only the protocol would change.
Re: Introducing Workers KV
#89Earlier quoted context omitted.
> Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity. Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. I’m not sure you’d want to maintain a WebSocket inside of a CF worker. At least I can’t see that helping with cost. Execution time limits would get you too. IMO connection mana…
> Hi! Fanout founder here. I’m glad you like our way of doing things. If you need better pricing for high volume we are always willing to discuss. Hi! Wasn't expecting you to show up like this. Re. pricing for high volume, I'm sure enterprises and publishers could afford it, I'm trying to see if it's possible to offer free or very cheap service to individuals with large audiences. So not so much that I want a high vo…
I actually have an open PR to add WebSocket endpoint support to CF Workers (so a Worker can act as a WebSocket client or server, instead of just a proxy). The main hangup though is that we need to figure out how to charge for long-lived connections. We obviously don't want to apply the normal time limits in this case. Hopefully we'll find time to work through this soon. :)
Re: Introducing Workers KV
#90Earlier quoted context omitted.
Can you implement IPC with this system? It seems wrong to relate this to tuple spaces otherwise.
Put the target channel name and a message id at the front of each key and you can use a tuple space for IPC.
What I mean is: can a process block until a particular key appears, then resume execution after receiving the associated value? That's the main differentiating feature of a tuple space, and it wasn't clear to me whether this system supports that or not (or even has a concept of a process).