Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cap'n Web: a new RPC system for browsers and web servers

blog.cloudflare.com

Post reply on HN
Print view View original

151–160 of 305 posts

Re: Cap'n Web: a new RPC system for browsers and web servers

#151

Post by ziml77 »

javier2 wrote: post #115
mdasen wrote: post #90

Earlier quoted context omitted.

In C#, there's expression trees which handle things like this and it's how Entity Framework is able to convert the lambdas it's given into SQL. This means that you can pass around code that can be inspected or transformed instead of being executed. Take this EntityFramework snippet: db.People.Where(p => p.Name == "Joe") `Where` takes an `Expression > predicate`. It isn't taking the `Func` itself, but an `Expression`…

It dont think C# looks at the code? I suspect it can track that you called p.Name, then generate sql with this information?

The C# compiler is looking at that code. It sees that the lambda is being passed into a function which accepts an Expression as a parameter, so it compiles the lambda as an expression tree rather than behaving like it's a normal delegate.

Re: Cap'n Web: a new RPC system for browsers and web servers

#152

Post by kentonv »

stavros wrote: post #135
kentonv wrote: post #128

Earlier quoted context omitted.

I really want to bake in some sort of support for generating type checks based on TypeScript types... so then your schemas are just TypeScript. Not sure why this doesn't seem to be a common practice TBH, might be missing something.

That would be great, would these checks run at deserialization time? They'd probably need to, as you wouldn't want to assume that the stuff coming through the network is of a specific type.

I'm thinking the ideal would be if I could feed in a TypeScript interface, and have some tool generate a wrapper around that interface which type-checks all inputs.

This tool could actually be totally independent from the RPC implementation.

I don't think it's necessary to bake type checks into the deserialization itself, since the RPC system already doesn't make any assumptions about the payloads it is moving around (other than that they are composed of only the types that the deserialization supports, which is a fixed list).

Re: Cap'n Web: a new RPC system for browsers and web servers

#153

Post by btown »

kentonv wrote: post #66
spankalee wrote: post #60

Earlier quoted context omitted.

I presume conditionals are banned - sort of like the rules of hooks - but how?

The input to the map function (when it is called in "record" mode on the client) is an RpcPromise for the eventual value. That means you can't actually inspect the value, you can only queue pipelined calls on it. Since you can't inspect the value, you can't do any computation or branching on it. So any computation and branching you do perform must necessarily have the same result every time the function runs, and so…

> your function needs to have no side effects

I'm trying to understand how well this no-side-effects footgun is defended against.

https://github.com/cloudflare/capnweb/blob/main/src/map.ts#L... seems to indicate that if the special pre-results "record mode" call of the callback raises an error, the library silently bails out (but keeps anything already recorded, if this was a nested loop).

That catches a huge number of things like conditionals on `item.foo` in the map, but (a) it's quite conservative and will fail quite often with things like those conditionals, and (b) if I had `count += 1` in my callback, where count was defined outside the scope, now that's been incremented one extra time, and it didn't raise an error.

React Hooks had a similar problem, with a constraint that hooks couldn't be called conditionally. But they solved their DX by having a convention where every hook would start with `use`, so they could then build linters that would enforce their constraint. And if I recall, their rules-of-hooks eslint plugin was available within days of their announcement.

The problem with `map` is that there are millions of codebases that already use a method called `map`. I'd really, really love to see Cap'n Web use a different method name - perhaps something like `smartMap` or `quickMap` or `rpcMap` - that is more linter-friendly. A method name that doesn't require the linter to have access to strong typing information, to understand that you're mapping over the special RpcPromise rather than a low-level array.

Honestly, it's a really cool engineering solve, with the constraint of not having access to the AST like one has in Python. I do think that with wider adoption, people will find footguns, and I'd like this software to get a reputation for being resilient to those!

Re: Cap'n Web: a new RPC system for browsers and web servers

#154

Post by davexunit »

cyberax wrote: post #118
davexunit wrote: post #93

Earlier quoted context omitted.

What's so magical about it?

Magically tracing dependencies. Imagine debugging it when it breaks because of some subtle differences between environments.

Not sure what you mean by "tracing dependencies". Can you elaborate?

Re: Cap'n Web: a new RPC system for browsers and web servers

#155

Post by skybrian »

kentonv wrote: post #104
drysart wrote: post #102

Earlier quoted context omitted.

There are inherent limitations with the "execute it once and see what happens" approach; namely that any conditional logic that might be in the mapping function is going to silently get ignored. For example, `db.people.map(p => p.IsPerson ? (p.FirstName + ' ' + p.LastName) : p.EntityName)` would either be seen as reading `(IsPerson, FirstName, LastName)` or `(p.IsPerson, p.EntityName)` depending on the specific behav…

The placeholder value is an RpcPromise. Which means that all its properties are also RpcPromises. So `p.IsPerson` is an RpcPromise. I guess that's truthy, so the expression will always evaluate to `(p.FirstName + ' ' + p.LastName)`. But that's going to evaluate to '[object Object] [object Object]'. So your mapper function will end up not doing anything with the input at all, and you'll get back an array full of '[obj…

Another way to screw this up would be to have an index counter and do something different based on the index. I think the answer is "don't do that."

Re: Cap'n Web: a new RPC system for browsers and web servers

#156

Post by actionfromafar »

kentonv wrote: post #105
drysart wrote: post #102

Earlier quoted context omitted.

There are inherent limitations with the "execute it once and see what happens" approach; namely that any conditional logic that might be in the mapping function is going to silently get ignored. For example, `db.people.map(p => p.IsPerson ? (p.FirstName + ' ' + p.LastName) : p.EntityName)` would either be seen as reading `(IsPerson, FirstName, LastName)` or `(p.IsPerson, p.EntityName)` depending on the specific behav…

> I wonder why they don't just do `.toString()` on the mapping function and then parse the resulting Javascript into an AST and figure out property accesses from that. That sounds incredibly complicated, and not something we could do in a <10kB library!

Maybe Fabrice Bellard could spare an afternoon.

Re: Cap'n Web: a new RPC system for browsers and web servers

#157

Post by stavros »

kentonv wrote: post #152
stavros wrote: post #135

Earlier quoted context omitted.

That would be great, would these checks run at deserialization time? They'd probably need to, as you wouldn't want to assume that the stuff coming through the network is of a specific type.

I'm thinking the ideal would be if I could feed in a TypeScript interface, and have some tool generate a wrapper around that interface which type-checks all inputs. This tool could actually be totally independent from the RPC implementation. I don't think it's necessary to bake type checks into the deserialization itself, since the RPC system already doesn't make any assumptions about the payloads it is moving around…

I understand your reasoning, but this is one of those things where people will use the default, and your choice as designer is whether you'll have the default that makes it harder to get started with, but reduces bugs down the line, or whether you'll make it easier to get started with, but much more buggy.

History has shown that, if you want things to be popular, you should choose the latter, but I think the tide has turned enough that the former could be the right choice now. That's also the reason why we use Typescript instead of JS, so mandatory static typing would definitely fit with the zeitgeist.

Honestly, the boundary is the one place where I wouldn't want to not have types.

Re: Cap'n Web: a new RPC system for browsers and web servers

#158

Post by ryanrasti »

kentonv wrote: post #43
benpacker wrote: post #12

This seems great and I'm really excited to try it in place of trpc/orpc. Although it seems to solve one of the problems that GraphQL solved that trpc doesn't (the ability to request nested information from items in a list or properties of an object without changes to server side code), there is no included solution for the server side problem that creates that the data loader pattern was intended to solve, where a na…

I generally agree that the .map() trick doesn't actually replace GraphQL without some sort of server-side optimizations to avoid turning this into N+1 selects. However, if your database is sqlite in a Cloudflare Durable Object, and the RPC protocol is talking directly to it, then N+1 selects are actually just fine. https://www.sqlite.org/np1queryprob.html

Agree, and to add, from what I see, the main issue is that server-side data frameworks (e.g., ORMs) aren't generally built for the combination of security & composability that make them naturally synergize with Cap'n Web. Another way to put it: promise pipelining is a killer feature but if your ORM doesn't support pipelining, then you have to build a complex bridge to support them both.

I've been working on this issue from the other side. Specifically, a TS ORM that has the level of composability to make promise pipelining a killer feature out of the box. And analagous to Cap'n Web's use of classes, it even models tables as classes with methods that return composable SQL expressions.

If curious: https://typegres.com/play/

Re: Cap'n Web: a new RPC system for browsers and web servers

#159

Post by kentonv »

skybrian wrote: post #155
kentonv wrote: post #104

Earlier quoted context omitted.

The placeholder value is an RpcPromise. Which means that all its properties are also RpcPromises. So `p.IsPerson` is an RpcPromise. I guess that's truthy, so the expression will always evaluate to `(p.FirstName + ' ' + p.LastName)`. But that's going to evaluate to '[object Object] [object Object]'. So your mapper function will end up not doing anything with the input at all, and you'll get back an array full of '[obj…

Another way to screw this up would be to have an index counter and do something different based on the index. I think the answer is "don't do that."

Hmm, but I should make it so the map callback can take the index as the second parameter probably. Of course, it would actually be a promise for the index, so you couldn't compute on it, but there might be other uses...

Re: Cap'n Web: a new RPC system for browsers and web servers

#160

Post by drysart »

kentonv wrote: post #104
drysart wrote: post #102

Earlier quoted context omitted.

There are inherent limitations with the "execute it once and see what happens" approach; namely that any conditional logic that might be in the mapping function is going to silently get ignored. For example, `db.people.map(p => p.IsPerson ? (p.FirstName + ' ' + p.LastName) : p.EntityName)` would either be seen as reading `(IsPerson, FirstName, LastName)` or `(p.IsPerson, p.EntityName)` depending on the specific behav…

The placeholder value is an RpcPromise. Which means that all its properties are also RpcPromises. So `p.IsPerson` is an RpcPromise. I guess that's truthy, so the expression will always evaluate to `(p.FirstName + ' ' + p.LastName)`. But that's going to evaluate to '[object Object] [object Object]'. So your mapper function will end up not doing anything with the input at all, and you'll get back an array full of '[obj…

Yeah I'll definitely chalk this up to my not having more than a very very passing idea of the API surface of your library based on a quick read over just the blog post.

On a little less trivial skim over it looks like the intention here isn't to map property-level subsets returned data (e.g., only getting the `FirstName` and `LastName` properties of a larger object); as much as it is to do joins and it's not data entities being provided to the mapping function but RpcPromises so individual property values aren't even available anyway.

So I guess I might argue that map() isn't a good name for the function because it immediately made me think it's for doing a mapping transformation and not for basically just specifying a join (since you can't really transform the data) since that's what map() can do everywhere else in Javascript. But for all I know that's more clear when you're actually using the library, so take what I think with a heaping grain of salt. ;)

Post reply on HN