Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You Are in a Box

jyn.dev

Post reply on HN
Print view View original

111–120 of 138 posts

Re: You Are in a Box

#111

Post by jynelson »

williamcotton wrote: post #77

Just this past week I've been working on a toy/experimental web DSL [0] that uses dynamically loaded shared libraries as middleware that pass per request arena-allocated Jansson json objects between steps in a pipeline. It's extensible in that new middleware can be created. Influenced by bash and OCaml/F#, here is some kind of demo of the syntax: POST /api/users |> validate: ` name: string(3..50) email: email age?: n…

oh hey Will! long time no see lol, it's been ages. small world.

i think this is on a good track! i like that it's designed to be extensible while still keeping some amount of structure, and that the DSL makes things very compact. how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that?

in general i'd love to see a bunch more DSLs, i think using general-purpose languages for everything is most of the time not actually helpful (cc https://jyn.dev/constrained-languages-are-easier-to-optimize...). i have some vague thoughts about how to make interop between DSLs and the host language easier but i need to think about them some more.

Re: You Are in a Box

#112

Post by stickfigure »

packetslave wrote: post #98
stronglikedan wrote: post #88

Earlier quoted context omitted.

Right? Who thinks that is acceptable in 2025?

Someone who is writing on their personal blog and doesn't give a damn what is "acceptable" to some rando on the Internet?

If you're publishing a public blog, by definition your audience is randos on the internet. Also, the author is posting in this thread.

Re: You Are in a Box

#113

Post by williamcotton »

cryptonector wrote: post #106
williamcotton wrote: post #77

Just this past week I've been working on a toy/experimental web DSL [0] that uses dynamically loaded shared libraries as middleware that pass per request arena-allocated Jansson json objects between steps in a pipeline. It's extensible in that new middleware can be created. Influenced by bash and OCaml/F#, here is some kind of demo of the syntax: POST /api/users |> validate: ` name: string(3..50) email: email age?: n…

I've wanted to do just this. I've used libmicrohttpd, and... it's not my favorite. But if you're going to do it in C then libmicrohttpd is probably the best API to use. You get bonus points for using jq :) If you're also using jq you might as well ditch Jansson and use jq's `jv` API. I highly recommend it.

One thing that sold me on Jansson is that you can set custom allocators which work very well with the request arena. Once those allocators are set the json objects continue to use the arena even when used in middleware. This makes memory management a cinch!

From what I can tell jq’s C lib doesn’t yet expose a way to set custom allocators.

Re: You Are in a Box

#114

Post by dnpp123 »

simpaticoder wrote: post #17

Computers are boxes, therefore all software is literally (and figuratively) "in a box", are they not? This might seem like a frivolous jest, but it is not. For example, the author points out that clojure, java, kotlin can interoperate, but notes they are stuck in the same jvm 'box'. This generalizes and recurses, so you must find a specific place to stop, and then motivate that. One likely place to stop is at "proces…

The vocabulary you speak/write every day is a box.

Your brain is a box.

Your body is a box.

/s

Re: You Are in a Box

#115

Post by cryptonector »

williamcotton wrote: post #113
cryptonector wrote: post #106

Earlier quoted context omitted.

I've wanted to do just this. I've used libmicrohttpd, and... it's not my favorite. But if you're going to do it in C then libmicrohttpd is probably the best API to use. You get bonus points for using jq :) If you're also using jq you might as well ditch Jansson and use jq's `jv` API. I highly recommend it.

One thing that sold me on Jansson is that you can set custom allocators which work very well with the request arena. Once those allocators are set the json objects continue to use the arena even when used in middleware. This makes memory management a cinch! From what I can tell jq’s C lib doesn’t yet expose a way to set custom allocators.

You can set a custom allocator, but it's global. You could contribute a feature to set thread-local custom allocators -- it'd be quite easy.

Re: You Are in a Box

#116

Post by williamcotton »

jynelson wrote: post #111
williamcotton wrote: post #77

Just this past week I've been working on a toy/experimental web DSL [0] that uses dynamically loaded shared libraries as middleware that pass per request arena-allocated Jansson json objects between steps in a pipeline. It's extensible in that new middleware can be created. Influenced by bash and OCaml/F#, here is some kind of demo of the syntax: POST /api/users |> validate: ` name: string(3..50) email: email age?: n…

oh hey Will! long time no see lol, it's been ages. small world. i think this is on a good track! i like that it's designed to be extensible while still keeping some amount of structure, and that the DSL makes things very compact. how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that? in general i'd love to see a bunch more DSLs, i think using g…

> how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that?

Interpreting in-process. There's a jq C lib, libjq-dev.

I hope you're doing well! I've very much enjoyed your recent blog posts.

Re: You Are in a Box

#117

Post by zzo38computer »

chubot wrote: post #31

> there is no interop between powershell and nushell FWIW I wrote a post about this design issue: Oils Is Exterior-First (Code, Text, and Structured Data) - https://www.oilshell.org/blog/2023/06/ysh-design.html#survey... That is - Powershell and nushell have an "interior" design (within a process/VM) - while POSIX shell, bash, OSH, and YSH have an "exterior" design (between processes) And I'll claim that the exterior…

> And I'll claim that the exterior design is the glue you need in large, heterogeneous systems.

Yes, but the specific format (and the way that the data transmission between processes and other pieces of the system is working) for the "exterior" design is an effect of the operating system. A different operating system might have a different way.

Whether it is interior or exterior, there is going to be different data types and other differences between structures, used in different ways. (One example of a difference which does not merely involve the interpretation of byte sequences, is transmissions of capabilities in systems that use capability-based security.)

I think that it is helpful to have a combination of interior and exterior functions, although most things will be exterior. However, for some things it is more helpful to do them internally within a process, for various reasons (including efficiency, requirements according to the specific operating system in use, security, and others).

> IMO it's better to have 1 type of glue, than 2 or more types

It might depend on the specific use. For some uses, it might be necessary to do it differently.

I also think that it is a mistake to assume one character set or one operating system for everything, though. However, even then, you probably wouldn't need more than one type of glue for many things, even if this "type of glue" is not the same as you would use in a different system.

> The data sent over the wire can't be interpreted without the schema information compiled into the binary.

A program is unlikely to be able to do much with data that the program is not designed to handle, regardless of how the schema is tramsmitted, so in-band communication of the schema probably wouldn't help much.

However, there can be multiple levels; e.g. with DER format, a program that does not understand the specific schema would still be able to read most of the values (with some exceptions if implicit types are used), sequences, etc, but does not know what any of that stuff means.

(I use DER format in some programs and formats that I do, because I think it is better than JSON in many ways.)

These issues are also things that my own ideas of operating system design would hopefully improve some of this. The command shell has a combination and data format would hopefully be more helpful.

Re: You Are in a Box

#118

Post by zzo38computer »

fellowniusmonk wrote: post #56
jhoechtl wrote: post #8

> Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can. —Zawinski's Law of Software Envelopment Its THE Zawinski of XEmacs so maybe not the best example.

Emacs has it right though, more right than wrong, they just, as a community, hate humans. Data and data collections should have app-tributes, apps shouldn't have data. The problem with most operating systems is that they need to model space time and minds as first class but they don't. I've been using my own personal OS for years now that I call imtropy, once your abstraction maps to reality everything becomes easier…

Do you have some more details about those things?

Re: You Are in a Box

#119

Post by no_wizard »

procaryote wrote: post #109
no_wizard wrote: post #85

Earlier quoted context omitted.

Having used SOAP and GraphQL, I really disagree with this characterization. The problem I have seen at most organizations is they simply want their APIs to reflect their database schema, even if its not a good or useful idea. They throw junk over the wall. They carry this practice over from REST to GraphQL and of course its horrible, its not a good way to use the technology. Now organizations that understand GraphQL…

Why wouldn't you want your database schema to match how you use it?

The whole point is to decouple the two. The gql queries are written for purpose, they’re not suppose to care about the data sources. This becomes even more important when a single query may grab data from more than a single source.

Re: You Are in a Box

#120

Post by cryptonector »

cryptonector wrote: post #115
williamcotton wrote: post #113

Earlier quoted context omitted.

One thing that sold me on Jansson is that you can set custom allocators which work very well with the request arena. Once those allocators are set the json objects continue to use the arena even when used in middleware. This makes memory management a cinch! From what I can tell jq’s C lib doesn’t yet expose a way to set custom allocators.

You can set a custom allocator, but it's global. You could contribute a feature to set thread-local custom allocators -- it'd be quite easy.

Oh right, no, jq has a custom out-of-memory handler. But it should be trivial to extend src/jv_alloc.[ch] to let you set not just a custom nomem handler but also a custom allocator, and then you can use thread-locals in your allocator.
Post reply on HN