Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cap'n Web: a new RPC system for browsers and web servers

blog.cloudflare.com

Post reply on HN
Print view View original

121–130 of 305 posts

Re: Cap'n Web: a new RPC system for browsers and web servers

#121

Post by mythmon_ »

kentonv wrote: post #69
ianbicking wrote: post #65

Couple random thoughts: I'm trying to see if there's something specifically for streaming/generators. I don't think so? Of course you can use callbacks, but you have to implement your own sentinel to mark the end, and other little corner cases. It seems like you can create a callback to an anonymous function, but then the garbage collector probably can't collect that function? --- I don't see anything about exception…

You can define a stream like: interface Stream extends RpcTarget { write(chunk): void; end(): void; [Symbol.dispose](): void; } Note that the dispose method will be called automatically when the caller disposes the stub or when they disconnect the RPC session. The `end()` method is still useful as a way to distinguish a clean end vs. an abort. In any case, you implement this interface, and pass it over the RPC connec…

I'll definitely be watching out for more built-in streaming support. Being able to throw the standard types directly over the wire and trust that the library will handle optimally utilizing the connection would make this the RPC library that I've been looking for all year.

Re: Cap'n Web: a new RPC system for browsers and web servers

#122

Post by divan »

afiori wrote: post #89
chao- wrote: post #30

Earlier quoted context omitted.

Reminds me of the old "MongoDB is Web Scale" series of comedy videos: https://youtu.be/bzkRVzciAZg Some friends and I still jokingly troll each other in the vein of these, interjecting with "When async programming was discovered in 2008...", or "When memory safe compiled languages were invented in 2012..." and so forth.

Async/await became ergonomic and widespread only recently, I am sure there were async systems in the '80 but for example nodejs focus on non blocking I/O changed how a lot of people thought about servers and concurrency (whether node was first is almost irrelevant) Often when something is discovered or invented is far less influential[1] than when it jumps on and hype train. [1] the discovery is very important for hi…

I actually hate async/await approach to concurrency and avoid it as much as I can.

My mental model is that it's a caller who decides how call should be executed (synchroniously or asynchroniously). Synchronious call is when caller waits till completion/error, asynchronious - is when caller puts the call in the background (whatever it means in that language/context) and handle return results later. CSP concurrency model [1] is the closest fit here.

It's not a property of the function to decide how the caller should deal with it. This frustration was partly described in the viral article "What color is your function?" [2], but my main rant about this concurrency approach is that it doesn't match well how we think and reason about concurrent processes, and requires mental cognitive gymnastics to reason about relatively simple code.

Seeing "async/await/Promises/Futures" being a justification of a "protocol" makes little sense to me. I can totally get that they reimagined how to do RPC with first-class async/await primitives, but that doesn't make it a network "protocol".

[1] https://en.wikipedia.org/wiki/Communicating_sequential_proce...

[2] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

Re: Cap'n Web: a new RPC system for browsers and web servers

#123

Post by kentonv »

samwillis wrote: post #120
mtrovo wrote: post #57

The section on how they solved arrays is fascinating and terrifying at the same time https://blog.cloudflare.com/capnweb-javascript-rpc-library/#... . > .map() is special. It does not send JavaScript code to the server, but it does send something like "code", restricted to a domain-specific, non-Turing-complete language. The "code" is a list of instructions that the server should carry out for each member of the arra…

This record and replay trick is very similar to what I recently used to implement the query DSL for Tanstack DB ( https://tanstack.com/db/latest/docs/guides/live-queries ). We pass a RefProxy object into the where/select/join callbacks and use it to trace all the props and expressions that are performed. As others have noted you can't use js operators to perform actions, so we built a set of small functions that we c…

Yes, in principle, any sort of remote compute we want to support, we could accomplish by having a custom function you have to call for it. Then the calls can be captured into the record.

But also, apps can already do this themselves. Since the record/replay mechanism already intercepts any RPC calls, the server can simply provide a library of operations as part of its RPC API. And now the mapper callback can take advantage of those.

I think this is the approach I prefer: leave it up to servers to provide these ops if they want to. Don't extend the protocol with a built-in library of ops.

Re: Cap'n Web: a new RPC system for browsers and web servers

#124

Post by stavros »

I was really excited when I saw the headline, but I was kind of disappointed to see it doesn't support schemas natively. I know you can specify them via zod, but I'm really not looking forward to any more untyped APIs, given that the lack of strict API typing has been by far the #1 reason for the bugs I've had to deal with in my career.

Re: Cap'n Web: a new RPC system for browsers and web servers

#125

Post by kentonv »

svieira wrote: post #117
kentonv wrote: post #107

Earlier quoted context omitted.

I don't think you could make filter() work with the same approach, because it seems like you'd actually have to do computation on the result. map() works for cases where you don't need to compute anything in the callback, you just want to pipeline the elements into another RPC, which is actually a common case with map(). If you want to filter server-side, you could still accomplish it by having the server explicitly…

But you might want to compose various methods on the server in order to filter, just like you might want to compose various methods on the server in order to transform. Why is `collection.map(server.lookupByInternalizedId)` a special case that doesn't require `server.lookupCollectionByInternalizedId(collection)`, but `collection.filter(server.isOperationSensibleForATuesday)` is a bridge too far and for that you need…

I agree that, in the abstract, it's inconsistent.

But in the concrete:

* Looking up some additional data for each array element is a particularly common thing to want to do.

* We can support it nicely without having to create a library of operations baked into the protocol.

I really don't want to extend the protocol with a library of operations that you're allowed to perform. It seems like that library would just keep growing and add a lot of bloat and possibly security concerns.

(But note that apps can actually do so themselves. See: https://news.ycombinator.com/item?id=45339577 )

Re: Cap'n Web: a new RPC system for browsers and web servers

#126

Post by samwillis »

kentonv wrote: post #123
samwillis wrote: post #120

Earlier quoted context omitted.

This record and replay trick is very similar to what I recently used to implement the query DSL for Tanstack DB ( https://tanstack.com/db/latest/docs/guides/live-queries ). We pass a RefProxy object into the where/select/join callbacks and use it to trace all the props and expressions that are performed. As others have noted you can't use js operators to perform actions, so we built a set of small functions that we c…

Yes, in principle, any sort of remote compute we want to support, we could accomplish by having a custom function you have to call for it. Then the calls can be captured into the record. But also, apps can already do this themselves . Since the record/replay mechanism already intercepts any RPC calls, the server can simply provide a library of operations as part of its RPC API. And now the mapper callback can take ad…

Ah, yes, obviously. This is all very cool!

Re: Cap'n Web: a new RPC system for browsers and web servers

#127

Post by Yoric »

kentonv wrote: post #113
Yoric wrote: post #112

Earlier quoted context omitted.

Ah, thanks, I wasn't aware of that. That won't prevent me from bragging, as we released Opalang 1.0 in 2011 :)

I think Mark S Miller beat us both by like 10-20 years with CapTP. ;)

Not on multi-tier web applications, though :)

(but yeah, I was passingly familiar with E! when I designed the capability system of Opalang, so I definitely don't get full bragging rights)

Re: Cap'n Web: a new RPC system for browsers and web servers

#128

Post by kentonv »

stavros wrote: post #124

I was really excited when I saw the headline, but I was kind of disappointed to see it doesn't support schemas natively. I know you can specify them via zod, but I'm really not looking forward to any more untyped APIs, given that the lack of strict API typing has been by far the #1 reason for the bugs I've had to deal with in my career.

I really want to bake in some sort of support for generating type checks based on TypeScript types... so then your schemas are just TypeScript. Not sure why this doesn't seem to be a common practice TBH, might be missing something.

Re: Cap'n Web: a new RPC system for browsers and web servers

#130

Post by kentonv »

meindnoch wrote: post #119

What happens if I pass a recursive function to map()? let traverse = (dir) => { name: dir.name, files: api.ls(dir).map(traverse) // api.ls() returns [] for files }; let tree = api.ls("/").map(traverse);

I believe this will stack-overflow on the client side. The callback is invoked in recording mode synchronously when you call `.map()`. Nested maps are allowed, but this case ends up being infinitely nested, so eventually you're going to hit a stack overflow while trying to do the recording.
Post reply on HN