Introducing Workers KV • Hacker News Board

Re: Introducing Workers KV

#61

Post by tracker1 » Fri, Sep 28, 2018, 5:42 PM UTC

zackbloom wrote: post #54

tyingq wrote: post #53

Is there an API method that can list existing/matching keys in a namespace with a glob or prefix or similar? Like, "show me all keys that match xyz∗ in this namespace"

We're planning on adding support for range requests (which becomes equivalent to prefixes a la CouchDB). Is there a reason you need glob specifically?

well, for one example off the top of my head would be using geohashes for proximity in records... if you could glob every geohash with the same first 6-8 characters, you could do range queries pretty quickly.

I've considered similar for shard keys in terms of proximity related information.

Re: Introducing Workers KV

#62

Post by zackbloom » Fri, Sep 28, 2018, 5:47 PM UTC

tracker1 wrote: post #59

This is very cool, but how do you limit an end user to only being able to read a given set of values. It seems to me that this scheme puts all credentials for access in the browser, and they'll be able to just run roughshod all over your data without constraints in place. Can you create data spaces/keys for ones that are read-write per-user and others that are read-only for a given user?

You write the Worker code which runs on our infrastructure, not in a browser. You have complete control over who can access which piece of data, as it's your code doing the accessing.

Re: Introducing Workers KV

#63

Post by jhgg » Fri, Sep 28, 2018, 5:48 PM UTC

tracker1 wrote: post #59

This is very cool, but how do you limit an end user to only being able to read a given set of values. It seems to me that this scheme puts all credentials for access in the browser, and they'll be able to just run roughshod all over your data without constraints in place. Can you create data spaces/keys for ones that are read-write per-user and others that are read-only for a given user?

Access to these values are programmed in your cloudflare worker script. It's up to you to decide how they can be accessed.

Re: Introducing Workers KV

#64

Post by zackbloom » Fri, Sep 28, 2018, 5:48 PM UTC

sudhirj wrote: post #57

Great job - was trying to something similar by setting up three etcd raft nodes on each continent, but this is like 150+ nodes. Mind sharing of you’re using good clocks, paxos or raft to coordinate? Would also be great if CF could do websocket / sse fanout at the edges - if I have a couple million websockets connected on ws.cf.com/key1 and I update the value, would be great if the new value could be broadcast. That w…

Can you talk more about your specific use case?

Re: Introducing Workers KV

#65

Post by tracker1 » Fri, Sep 28, 2018, 5:55 PM UTC

zackbloom wrote: post #62

tracker1 wrote: post #59

This is very cool, but how do you limit an end user to only being able to read a given set of values. It seems to me that this scheme puts all credentials for access in the browser, and they'll be able to just run roughshod all over your data without constraints in place. Can you create data spaces/keys for ones that are read-write per-user and others that are read-only for a given user?

You write the Worker code which runs on our infrastructure, not in a browser. You have complete control over who can access which piece of data, as it's your code doing the accessing.

Thanks, I misunderstood that part... very cool indeed. What are the overhead limits for workers?

Aside, it seems like this could be really interesting combined to make an entire API with static delivery from S3 or Azure Blobs, with a backend on a cloud hosted database. With a lot of flexibility in between.

Re: Introducing Workers KV

#66

Post by zackbloom » Fri, Sep 28, 2018, 5:56 PM UTC

tracker1 wrote: post #65

zackbloom wrote: post #62

Earlier quoted context omitted.

You write the Worker code which runs on our infrastructure, not in a browser. You have complete control over who can access which piece of data, as it's your code doing the accessing.

Thanks, I misunderstood that part... very cool indeed. What are the overhead limits for workers? Aside, it seems like this could be really interesting combined to make an entire API with static delivery from S3 or Azure Blobs, with a backend on a cloud hosted database. With a lot of flexibility in between.

The ultimate goal is to let you do all of that within our network!

When you say overhead limit, are you talking about latency? Our goal is to keep reads on the order of 5ms in the 90th percentile.

Re: Introducing Workers KV

#67

Post by zhobbs » Fri, Sep 28, 2018, 5:59 PM UTC

ryanworl wrote: post #2

Are you planning on releasing any of the architectural details behind this service? If you’re truly replicating to every PoP that’s quite a fan-out and I can see why you’re limited to 1 write per second per key!

I was wondering if it is lazily replicated to PoPs via KV gets. Would be interested in clarification on that though.

Re: Introducing Workers KV

#68

Post by BHSPitMonkey » Fri, Sep 28, 2018, 6:02 PM UTC

webjames wrote: post #9

With the shopping cart example, I'm not sure I understand how session identity can be preserved if I clear my local storage/cookies?

If the user signs in, move their shopping cart data from a key derived from their session ID to a key derived from their account ID. When a user is signed in, data is read from the latter.

Re: Introducing Workers KV

#69

Post by zackbloom » Fri, Sep 28, 2018, 6:07 PM UTC

adrianmonk wrote: post #58

zackbloom wrote: post #13

Earlier quoted context omitted.

It's moving that shopping cart data from being stored at a single origin, to being stored in the network all around the world. The advantage of that in that use-case is you can render your site just as quickly as if it was a static website, but it can contain the customer's personal shopping cart data.

The shopping cart is still stored somewhere, but when the user has cleared their cookies, etc., what information do you have that will let you it's their shopping cart so you can find it again? There's a (K,V) pair somewhere, but in order to get V, you need K, which you've lost.

It would work just as most shopping carts do I imagine. You could store a session id in a cookie, and then use the data in KV to map that session id to a user account (if they're logged in) and their cart. So you would have a namespace full of sessions, and a namespace full of carts.

You could also store carts by session id until they're logged in, and then store the carts by account id when they are.

Re: Introducing Workers KV

#70

Post by skunkworker » Fri, Sep 28, 2018, 6:17 PM UTC

With this and the changes to their worker caching API it's now completely possible to write a custom pull origin CDN service that you have complete control over.