Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cap'n Web: a new RPC system for browsers and web servers

blog.cloudflare.com

Post reply on HN
Print view View original

101–110 of 305 posts

Re: Cap'n Web: a new RPC system for browsers and web servers

#101

Post by qcnguy »

kentonv wrote: post #43
benpacker wrote: post #12

This seems great and I'm really excited to try it in place of trpc/orpc. Although it seems to solve one of the problems that GraphQL solved that trpc doesn't (the ability to request nested information from items in a list or properties of an object without changes to server side code), there is no included solution for the server side problem that creates that the data loader pattern was intended to solve, where a na…

I generally agree that the .map() trick doesn't actually replace GraphQL without some sort of server-side optimizations to avoid turning this into N+1 selects. However, if your database is sqlite in a Cloudflare Durable Object, and the RPC protocol is talking directly to it, then N+1 selects are actually just fine. https://www.sqlite.org/np1queryprob.html

I wonder if there's a way to process the RPC encoding inside a stored procedure, using the various JS-in-DB features out there.

Re: Cap'n Web: a new RPC system for browsers and web servers

#102

Post by drysart »

mdasen wrote: post #90
mtrovo wrote: post #57

The section on how they solved arrays is fascinating and terrifying at the same time https://blog.cloudflare.com/capnweb-javascript-rpc-library/#... . > .map() is special. It does not send JavaScript code to the server, but it does send something like "code", restricted to a domain-specific, non-Turing-complete language. The "code" is a list of instructions that the server should carry out for each member of the arra…

In C#, there's expression trees which handle things like this and it's how Entity Framework is able to convert the lambdas it's given into SQL. This means that you can pass around code that can be inspected or transformed instead of being executed. Take this EntityFramework snippet: db.People.Where(p => p.Name == "Joe") `Where` takes an `Expression > predicate`. It isn't taking the `Func` itself, but an `Expression`…

There are inherent limitations with the "execute it once and see what happens" approach; namely that any conditional logic that might be in the mapping function is going to silently get ignored. For example, `db.people.map(p => p.IsPerson ? (p.FirstName + ' ' + p.LastName) : p.EntityName)` would either be seen as reading `(IsPerson, FirstName, LastName)` or `(p.IsPerson, p.EntityName)` depending on the specific behavior of the placeholder value ... and neither of those sets is fully correct.

I wonder why they don't just do `.toString()` on the mapping function and then parse the resulting Javascript into an AST and figure out property accesses from that. At the very least, that'd allow the code to properly throw an error in the event the callback contains any forbidden or unsupported constructs.

Re: Cap'n Web: a new RPC system for browsers and web servers

#103

Post by svieira »

kentonv wrote: post #75
tonyg wrote: post #70

Earlier quoted context omitted.

Is .map specialcased or do user functions accepting callbacks work the same way? Because you could do the Scott-Mogensen thing of #ifTrue:ifFalse: if so, dualizing the control-flow decision making, offering a menu of choices/continuations.

.map() is totally special-cased. For any other function accepting a callback, the function on the server will receive an RPC stub, which, when called, makes an RPC back to the caller, calling the original version of the function. This is usually what you want, and the semantics are entirely normal. But for .map(), this would defeat the purpose, as it'd require an additional network round-trip to call the callback.

Doesn't this apply for _all_ the combinators on `Array.prototype` though? Why special-case `.map` only?

Re: Cap'n Web: a new RPC system for browsers and web servers

#104

Post by kentonv »

drysart wrote: post #102
mdasen wrote: post #90

Earlier quoted context omitted.

In C#, there's expression trees which handle things like this and it's how Entity Framework is able to convert the lambdas it's given into SQL. This means that you can pass around code that can be inspected or transformed instead of being executed. Take this EntityFramework snippet: db.People.Where(p => p.Name == "Joe") `Where` takes an `Expression > predicate`. It isn't taking the `Func` itself, but an `Expression`…

There are inherent limitations with the "execute it once and see what happens" approach; namely that any conditional logic that might be in the mapping function is going to silently get ignored. For example, `db.people.map(p => p.IsPerson ? (p.FirstName + ' ' + p.LastName) : p.EntityName)` would either be seen as reading `(IsPerson, FirstName, LastName)` or `(p.IsPerson, p.EntityName)` depending on the specific behav…

The placeholder value is an RpcPromise. Which means that all its properties are also RpcPromises. So `p.IsPerson` is an RpcPromise. I guess that's truthy, so the expression will always evaluate to `(p.FirstName + ' ' + p.LastName)`. But that's going to evaluate to '[object Object] [object Object]'. So your mapper function will end up not doing anything with the input at all, and you'll get back an array full of '[object Object] [object Object]'.

Unfortunately, "every object is truthy" and "every object can be coerced to a string even if it doesn't have a meaningful stringifier" are just how JavaScript works and there's not much we can do about it. If not for these deficiencies in JS itself, then your code would be flagged by the TypeScript compiler as having multiple type errors.

Re: Cap'n Web: a new RPC system for browsers and web servers

#105

Post by kentonv »

drysart wrote: post #102
mdasen wrote: post #90

Earlier quoted context omitted.

In C#, there's expression trees which handle things like this and it's how Entity Framework is able to convert the lambdas it's given into SQL. This means that you can pass around code that can be inspected or transformed instead of being executed. Take this EntityFramework snippet: db.People.Where(p => p.Name == "Joe") `Where` takes an `Expression > predicate`. It isn't taking the `Func` itself, but an `Expression`…

There are inherent limitations with the "execute it once and see what happens" approach; namely that any conditional logic that might be in the mapping function is going to silently get ignored. For example, `db.people.map(p => p.IsPerson ? (p.FirstName + ' ' + p.LastName) : p.EntityName)` would either be seen as reading `(IsPerson, FirstName, LastName)` or `(p.IsPerson, p.EntityName)` depending on the specific behav…

> I wonder why they don't just do `.toString()` on the mapping function and then parse the resulting Javascript into an AST and figure out property accesses from that.

That sounds incredibly complicated, and not something we could do in a <10kB library!

Re: Cap'n Web: a new RPC system for browsers and web servers

#106

Post by losvedir »

Babe get in here, a new kentonv library just dropped!

I'm surprised how little code is actually involved here, just looking at the linked GitHub repo. Is that really all there is to it? In theory, it shouldn't be too hard to port the server side to another language, right? I'm interested in using it in an Elixir server for a JS/TS frontend.

For that matter, the language porting seems like a pretty good LLM task. Did you use much LLM-generated code for this repo? I seem to recall kentonv doing an entirely AI-generated (though human-reviewed, of course) proof of concept a few months ago.

Re: Cap'n Web: a new RPC system for browsers and web servers

#107

Post by kentonv »

qcnguy wrote: post #98
kentonv wrote: post #75

Earlier quoted context omitted.

.map() is totally special-cased. For any other function accepting a callback, the function on the server will receive an RPC stub, which, when called, makes an RPC back to the caller, calling the original version of the function. This is usually what you want, and the semantics are entirely normal. But for .map(), this would defeat the purpose, as it'd require an additional network round-trip to call the callback.

What about filter? Seems useful also.

I don't think you could make filter() work with the same approach, because it seems like you'd actually have to do computation on the result.

map() works for cases where you don't need to compute anything in the callback, you just want to pipeline the elements into another RPC, which is actually a common case with map().

If you want to filter server-side, you could still accomplish it by having the server explicitly expose a method that takes an array as input, and performs the desired filter. The server would have to know in advance exactly what filter predicates are needed.

Re: Cap'n Web: a new RPC system for browsers and web servers

#108

Post by kentonv »

svieira wrote: post #103
kentonv wrote: post #75

Earlier quoted context omitted.

.map() is totally special-cased. For any other function accepting a callback, the function on the server will receive an RPC stub, which, when called, makes an RPC back to the caller, calling the original version of the function. This is usually what you want, and the semantics are entirely normal. But for .map(), this would defeat the purpose, as it'd require an additional network round-trip to call the callback.

Doesn't this apply for _all_ the combinators on `Array.prototype` though? Why special-case `.map` only?

See cousin comment: https://news.ycombinator.com/item?id=45338969

Re: Cap'n Web: a new RPC system for browsers and web servers

#109

Post by kentonv »

losvedir wrote: post #106

Babe get in here, a new kentonv library just dropped! I'm surprised how little code is actually involved here, just looking at the linked GitHub repo. Is that really all there is to it? In theory, it shouldn't be too hard to port the server side to another language, right? I'm interested in using it in an Elixir server for a JS/TS frontend. For that matter, the language porting seems like a pretty good LLM task. Did…

Some of the tests are LLM-generated, but none of the library itself is.

I don't think LLMs would be capable of writing this library (at least at present). The pieces fit together like a very intricate puzzle. I spent a lot more time thinking about how to do it right, than actually coding.

Very different from my workers-oauth-provider library, where it was just implementing a well-known spec with a novel (yet straightforward) API.

The code might port nicely to another dynamic language, like Python, but I think you'd have a hard time porting it to a statically-typed language. There's a whole lot of iterating over arbitrary objects without knowing their types.

Post reply on HN