Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We rewrote JSONata with AI in a day, saved $500k/year

reco.ai

Post reply on HN
Print view View original

41–50 of 278 posts

Re: We rewrote JSONata with AI in a day, saved $500k/year

#41

Post by hansvm »

Aurornis wrote: post #19

The key point for me was not the rewrite in Go or even the use of AI, it was that they started with this architecture: > The reference implementation is JavaScript, whereas our pipeline is in Go. So for years we’ve been running a fleet of jsonata-js pods on Kubernetes - Node.js processes that our Go services call over RPC. That meant that for every event (and expression) we had to serialize, send over the network, ev…

I mostly agree, but it's more appropriate to weigh contributions against an FTE's output rather than their input. If I have a $10m/yr feature I'm fleshing out now and a few more lined up afterward, it's often not worth the time to properly handle any minor $300k/yr boondoggle. It's only worth comparing to an FTE's fully loaded cost when you're actually able to hire to fix it, and that's trickier since it takes time away from the core team producing those actually valuable features and tends to result in slower progress from large-team overhead even after onboarding. Plus, even if you could hire to fix it, wouldn't you want them to work on those more valuable features first?

Re: We rewrote JSONata with AI in a day, saved $500k/year

#42

Post by aniceperson »

Herring wrote: post #28
torben-friis wrote: post #16

>The approach was the same as Cloudflare’s vinext rewrite: port the official jsonata-js test suite to Go, then implement the evaluator until every test passes. the first question that comes to mind is: who takes care of this now? You had a dependency with an open source project. now your translated copy (fork?) is yours to maintain, 13k lines of go. how do you make sure it stays updated? Is this maintainance factored…

The full translation took 7hrs and $400 in tokens. Applying diffs every quarter using AI is much easier and cheaper. Software engineering has completely changed.

except there are 2 go implementations already, and he burnt 500k per year to have a kubernetes clusters to parse json (???), so the total gain is -500000*year - 400 + 1 (deducting prompt to use existing implementation)

Re: We rewrote JSONata with AI in a day, saved $500k/year

#43

Post by torben-friis »

simonw wrote: post #18
torben-friis wrote: post #16

>The approach was the same as Cloudflare’s vinext rewrite: port the official jsonata-js test suite to Go, then implement the evaluator until every test passes. the first question that comes to mind is: who takes care of this now? You had a dependency with an open source project. now your translated copy (fork?) is yours to maintain, 13k lines of go. how do you make sure it stays updated? Is this maintainance factored…

That's only important if the plan is to stay feature-compatible with the original going forward. For this case, where it's used as an internal filtering engine, I expect the goal is fixing bugs that show up and occasionally adding a feature that's needed by this organization.

>expect the goal is fixing bugs that show up and occasionally adding a feature that's needed by this organization.

Even if we assume a clean and bug free port, and no compatibility required moving forward, and a scope that doesn't involve security risks, that's already non trivial, since it's a codebase no one has context of.

Probably not 500k worth of maintainance (because wtf were they doing in the first place) but I don't buy placing the current cost at 0.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#44

Post by andai »

Aurornis wrote: post #19

The key point for me was not the rewrite in Go or even the use of AI, it was that they started with this architecture: > The reference implementation is JavaScript, whereas our pipeline is in Go. So for years we’ve been running a fleet of jsonata-js pods on Kubernetes - Node.js processes that our Go services call over RPC. That meant that for every event (and expression) we had to serialize, send over the network, ev…

Yeah, it's like those posts "we made it 5,000x faster by actually thinking about what the code is doing."

Re: We rewrote JSONata with AI in a day, saved $500k/year

#45

Post by slopinthebag »

encoderer wrote: post #34
slopinthebag wrote: post #25

Earlier quoted context omitted.

Would it be better or worse if I had that experience and still said it's stupid?

You didn't say it was stupid. If you had, I would have just ignored the comment. But you expressed a level of surprised that led me to believe you're unfamiliar with how much of a pain in the ass JSON parsing is.

I think OP’s point was surprise that a company would spend so much on such inefficient json parsing. I’m agreeing. I get that JSON is not the fastest format to parse, but the overarching point is that you would expect changes to be made well before you’re spending $300k on it. Or in a slightly more ideal world, you wouldn't architect something so inefficient in the first place.

But it's common for engineers to blow insane amounts of money unnecessarily on inefficient solutions for "reasons". Sort of reminds me of saas's offering 100 concurrent "serverless" WS connections for like $50 / month - some devs buy into this nonsense.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#46

Post by Aurornis »

hansvm wrote: post #41
Aurornis wrote: post #19

The key point for me was not the rewrite in Go or even the use of AI, it was that they started with this architecture: > The reference implementation is JavaScript, whereas our pipeline is in Go. So for years we’ve been running a fleet of jsonata-js pods on Kubernetes - Node.js processes that our Go services call over RPC. That meant that for every event (and expression) we had to serialize, send over the network, ev…

I mostly agree, but it's more appropriate to weigh contributions against an FTE's output rather than their input. If I have a $10m/yr feature I'm fleshing out now and a few more lined up afterward, it's often not worth the time to properly handle any minor $300k/yr boondoggle. It's only worth comparing to an FTE's fully loaded cost when you're actually able to hire to fix it, and that's trickier since it takes time a…

They were running a big kubernetes infrastructure to handle all of these RPC calls.

That takes a lot of engineer hours to set up and maintain. This architecture didn't just happen, it took a lot of FTE hours to get it working and keep it that way.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#48

Post by bawolff »

encoderer wrote: post #23
bawolff wrote: post #21

Earlier quoted context omitted.

They got a 1000x speed up just by switching languages. I highly doubt the issue was serialization latency, unless they were doing something stupid like reserializing the same payload over and over again.

Well, for starters, they replace the RPC call with an in-process function call. But my point is anybody who's surprised that working with JSON at scale is expensive (because hey it's just JSON!) shouldn't be surprised.

Well everything is expensive at scale, and any deserialization/serialization step is going to be expensive if you do it enough. However yes i would be surprised. JSON parsing is pretty optimized now, i suspect most "json parsing at scale is expensive" is really the fault of other parts of the stack

Re: We rewrote JSONata with AI in a day, saved $500k/year

#49

Post by crazygringo »

> The approach was the same as Cloudflare’s vinext rewrite: port the official jsonata-js test suite to Go, then implement the evaluator until every test passes.

This makes me wonder, for reimplementation projects like this that aren't lucky enough to have super-extensive test suites, how good are LLM's at taking existing code bases and writing tests for every single piece of logic, every code path? So that you can then do a "cleanish-room" reimplementation in a different language (or even same language) using these tests?

Obviously the easy part is getting the LLM's to write lots of tests, which is then trivial to iterate until they all pass on the original code. The hard parts are how to verify that the tests cover all possible code paths and edge cases, and how to reliably trigger certain internal code paths.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#50

Post by kanbankaren »

amazingamazing wrote: post #17

how many billions of compute are wasted because this industry can't align on some binary format across all languages and APIs and instead keep serializing and deserializing things

ASN.1 and its on the wire format BER and DER have been available for close to 30+ years and it is running on billions of devices(cryptography, SSL, etc) and other critical infrastructures.

but, it is very boring stable, which means I can't tell the world about my wartime stories and write a blog about it.

Post reply on HN