Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cap'n Web: a new RPC system for browsers and web servers

blog.cloudflare.com

Post reply on HN
Print view View original

261–270 of 305 posts

Re: Cap'n Web: a new RPC system for browsers and web servers

#261

Post by ryanrasti »

kentonv wrote: post #166
ryanrasti wrote: post #158

Earlier quoted context omitted.

Agree, and to add, from what I see, the main issue is that server-side data frameworks (e.g., ORMs) aren't generally built for the combination of security & composability that make them naturally synergize with Cap'n Web. Another way to put it: promise pipelining is a killer feature but if your ORM doesn't support pipelining, then you have to build a complex bridge to support them both. I've been working on this issu…

That is actually pretty interesting! Have you considered making a sqlite version that works in Durable Objects? :)

Thanks, Kenton! Really encouraging to hear you find the idea interesting.

Right now I'm focused on Postgres (biggest market-share for full-stack apps). A sqlite version is definitely possible conceptually.

You're right about the bigger picture, though: Cap'n Web + Typegres (or a "Typesqlite" :) could enable the dream dev stack: a SQL layer in the client that is both sandboxed (via capabilities) and fully-featured (via SQL composability).

Re: Cap'n Web: a new RPC system for browsers and web servers

#262

Post by ryanrasti »

prngl wrote: post #174
ryanrasti wrote: post #168

Earlier quoted context omitted.

Agree -- I think that's a powerful generalization you're making. > We're often nowadays working in dynamic languages, so they become essentially the frontend to new DSLs, and instead of defining new syntax, we embed the AST construction into the scripting language. And I'd say that TypeScript is the real game-changer here. You get the flexibility of the JavaScript runtime (e.g., how Cap'n Web cleverly uses `Proxy`s)…

I do find the pattern interesting and powerful. But at the same time, something feels off about it (just conceptually, not trying to knock your money-making endeavor, godspeed). Some of the issues that all of these hit is: - No printf debugging. Sometimes you want things to be eager so you can immediately see what's happening. If you print and what you see is that's not very helpful. But that's what you'll get when y…

Really nice summary of the core challenges with this DSL/code-as-data pattern.

I've spent a lot of time thinking about this in the database context:

> No printf debugging

Yeah, spot on. The solutions here would be something like a `toSQL` that let's you inspect the compiled output at any step in the AST construction.

Also, if the backend supports it, you could compile a `printf` function all the way to the backend (this isn't supported in SQL though)

> No "natural" control flow in the DSL/tracing context

Agreed -- that can be a source of confusion and subtle bugs.

You could have a build rule that actually compile `if`/`while`/`for` into your AST (instead of evaluate them in the frontend DSL). Or you could have custom lint rules to forbid them in the DSL.

At the same time -- part of what makes query builders so powerful is the ability to dynamically construct queries. Runtime conditionals is what makes that possible.

> No side effects in the DSL/tracing context because that's not a real "running" context

Agreed -- similar to the above: this is something that needs to be forbidden (e.g., by a lint rule) or clearly understood before using it.

> Is this all just necessary complexity? Or is it because we're missing something, not quite seeing it right?

My take is that, at least in the SQL case: 100% the complexity is justified.

Big reasons why: 1. A *huge* impediment to productive engineering is context switching. A DSL in the same language as your app (i.e., an ORM) makes the bridge to your application code also seamless. (This is similar to the argument of having your entire stack be a single language) 2. The additional layer of indirection (building an AST) allows you to dynamically construct expressions in a way that isn't possible in SQL. This is effectively adding a (very useful) macro system on top of SQL. 3. In the case of Typescript, because its type-system is so flexible, you can have stronger typing on your DSL than the backend target.

tl;dr is these DSLs can enable better ergonomics in practice and the indirection can unlock powerful new primitives

Re: Cap'n Web: a new RPC system for browsers and web servers

#263

Post by evntdrvn »

gwbas1c wrote: post #249
squirrellous wrote: post #180

Earlier quoted context omitted.

Is there anything C# _doesn’t_ have? :-) It feels like C# has an answer to every problem I’ve ever had with other languages - dynamic loading, ADTs with pattern matching, functional programming, whatever this expression tree is, reflection, etc etc. Yet somehow it’s still a niche language that isn't widely used (outside of particular ecosystems).

Good abstractions around units (Apologies if there is a specific terminology that I should use.) Specifically, I'd like to be able to have "inches" as a generic type, where it could be an int, long, float, double. Then I'd also like to have "length" as a generic type where it could be inches as a double, millimeters as a long, ect, ect. I know they added generic numbers to the language in C# 7, so maybe there is a wa…

Check out F# "units of measure" ;)

Re: Cap'n Web: a new RPC system for browsers and web servers

#264

Post by kentonv »

jzig wrote: post #257

Hey kentonv, this looks really neat. How would you go about writing API documentation for something like this? I really like writing up OpenAPI YAML documents for consumers of APIs I write so that any time someone asks a question, "How do I get XYZ?" I can just point them to e.g. the SwaggerUI. But I'm struggling to understand how that would work here.

For public-facing APIs, I would strongly recommend writing the TypeScript interfaces in a separate file from any implementation, and using JSDoc comments. There are various documentation generators that can turn that into a web page, though personally as a user I tend to prefer to just look at the actual TS file.

Re: Cap'n Web: a new RPC system for browsers and web servers

#265

Post by sgarland »

pests wrote: post #202
sgarland wrote: post #188

Tangential from a discussion in TFA about GraphQL: > One benefit of GraphQL was to solve the “waterfall” problem of traditional REST APIs by allowing clients to ask for multiple pieces of data in one query. For example, instead of making three sequential HTTP calls: GET /user GET /user/friends GET /user/friends/photos …you can write one GraphQL query to fetch it all at once. Or you could have designed a schema to all…

> a scheme to allow easy tree traversal Huh that sounds a lot like graphql

GraphQL isn’t changing your schema, it’s issuing queries to satisfy your request. The queries are in no way guaranteed to be optimal, or well-suited for your schema or indices.

Re: Cap'n Web: a new RPC system for browsers and web servers

#266

Post by ryanrasti »

benpacker wrote: post #187
ryanrasti wrote: post #158

Earlier quoted context omitted.

Agree, and to add, from what I see, the main issue is that server-side data frameworks (e.g., ORMs) aren't generally built for the combination of security & composability that make them naturally synergize with Cap'n Web. Another way to put it: promise pipelining is a killer feature but if your ORM doesn't support pipelining, then you have to build a complex bridge to support them both. I've been working on this issu…

This seems really cool and I'd be happy to help (I'm currently a pgtyped + Kysely user and community contributor), and I see how this solves n+1 from promise pipelining when fetched "nested" data with a similar approach as Cap'n Web, but I don't we've solved the map problem. If I run, in client side Cap'n Web land (from the post): ``` let friendsWithPhotos = friendsPromise.map(friend => { return {friend, photo: api.g…

> but I don't we've solved the map problem.

Agreed! If we use `map` directly, Cap'n Web is still constrained by the ORM.

The solution would be what you're getting at -- something that directly composes the query builder primitives. In Typegres, that would look like this:

``` let friendsWithPhotos = friendsPromise.select((f) => ({...f, photo: f.photo()}) // `photo()` is a scalar subquery -- it could also be a join ```

i.e., use promise pipelining to build up the query on the server.

The idea is that Cap'n Web would allow you to pipeline the Typegres query builder operations. Note this should be possible in other fluent-based query builders (e.g., Kysely/Drizzle). But where Typegres really synergizes with Cap'n Web is that everything is already expressed as methods on classes, so the architecture is capability-ready.

P.S. Thanks for your generous offer to help! My contact info is in my HN profile. Would love to connect.

Re: Cap'n Web: a new RPC system for browsers and web servers

#268

Post by rjbwork »

brainzap wrote: post #237
rjbwork wrote: post #185

Earlier quoted context omitted.

It's one of the most widely used languages out there actually. But it's primarily used at buttoned up and boring SMB's/enterprise backoffices. We're not out here touting our new framework of the month to kafloogle the whatzit. We're just building systems with a good language and ecosystem that's getting better every year. I've worked only at startups/small businesses since I graduated university and it's all been in…

getting better? many packages we been using did a license swap on us xD fucking nice ecosystem

Fork it. End of the day some guys decided they wanted to make money and the corporations profiting off their labor weren't paying up. These things don't happen in a vacuum. Does your company have a multi-thousand dollar a year budget to make sure your dependencies are sustainable?

Re: Cap'n Web: a new RPC system for browsers and web servers

#269

Post by kentonv »

coreload wrote: post #267

Language specific RPC. At least Cap'n Proto is language agnostic. ConnectRPC is language agnostic and web compatible and a gRPC extended subset. I would have difficulty adopting a language specific RPC implementation.

The protocol itself is really only language-specific to a similar extent that JSON is language-specific. Which you can totally argue it is, but also people have figured out how to use it in lots of languages other than JavaScript.

I think Cap'n Web could work pretty well in Python and other dynamically-typed languages. Statically-typed would be a bit trickier (again, in the same sense that they are harder to use JSON in), but the answer there might just be to bridge to Cap'n Proto...

Re: Cap'n Web: a new RPC system for browsers and web servers

#270

Post by coreload »

kentonv wrote: post #269
coreload wrote: post #267

Language specific RPC. At least Cap'n Proto is language agnostic. ConnectRPC is language agnostic and web compatible and a gRPC extended subset. I would have difficulty adopting a language specific RPC implementation.

The protocol itself is really only language-specific to a similar extent that JSON is language-specific. Which you can totally argue it is, but also people have figured out how to use it in lots of languages other than JavaScript. I think Cap'n Web could work pretty well in Python and other dynamically-typed languages. Statically-typed would be a bit trickier (again, in the same sense that they are harder to use JSON…

Retrofitting to some but not all languages is not nearly the same as an intentionally language agnostic protocol. To the extent that calling this "web" is at best misleading.
Post reply on HN