Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We rewrote JSONata with AI in a day, saved $500k/year

reco.ai

Post reply on HN
Print view View original

221–230 of 278 posts

Re: We rewrote JSONata with AI in a day, saved $500k/year

#221

Post by otterley »

nirb89 wrote: post #117

Hey all, I'm the author of the blog post. I'm honestly loving the discussion this is generating (including the less flattering comments here). I'll try to answer some of the assumptions I've seen, hopefully it clears a few things. First off - some numbers. We're a near real-time cybersecurity platform, and we ingest tens of billions of raw events daily from thousands of different endpoints across SaaS. Additionally,…

> Finally, regarding the $300k/year cost (which many here seem to be horrified by) - it seems I wasn't clear enough in the blog. 200 pods was not the entire fleet, and it was not statically set. It was a single cluster at peak time. We have multiple clusters, each with their own traffic patterns and auto-scaling configurations. The total cost was $25k/month when summed as a whole.

So, then, what do you estimate the actual savings of the transition to be, taking into account only the component in question and its actual resource needs? (i.e. not simply projecting based on a linear multiple of peak utilization).

I'm going to be a little harsh here, and please forgive me: intellectual dishonesty, especially when the hard numbers are easily determinable, is something I've denied engineers' promotions for. It's genuinely impressive that you've saved the company money, but $500k/year based on peak projections is a very different number than, say, $100k/year in actual resources saved over the full course of it.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#225

Post by garganzol »

tantalor wrote: post #197

They say "embedding V8 directly into Go (to avoid the network hop)" was only an "incremental improvement" I'm very curious why this didn't help more. That was my first thought. Maybe they didn't get the result they wanted immediately so gave up before evaluating this fully?

JS runtimes are fatty, so embedding one instantly adds at least 30-50 Mb of RAM usage. Imagine that you do this for just for a specific function (JSON processing) and your total RAM budget for a whole pod is around 256 Mb.

No doubt, this approach would work reasonably well for machines with plenty of RAM, but I can see why it can be a bottleneck when scaled to N instances. RAM is expensive, and when you multiply those 50 extra megabytes by N, your total costs quickly climb up.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#226

Post by johnisgood »

stevepotter wrote: post #178
sigmoid10 wrote: post #138

Earlier quoted context omitted.

Eh. If you get into enterprise business, this is the accepted management style. AI will now mix this up a little, but before you basically needed to ask if you want to blow 300k on developer salaries to maybe fix something that is already working and generating money, or add more features to the roadmap you can pin on your chest. Scaling infrastructure is the best choice for 90% of managers, especially since they are…

I used to work for AWS on a service team. I noticed we were spending way too much on provisioned concurrency for dynamo and would benefit from on-demand provisioning. After proving it worked, making the change, deploying, was rather pleased with myself. "Saved $2M in costs by switching to on-demand provisioning" barely made it onto my performance review lol.

In an ideal world, you would have gotten those extra bucks. :P

Re: We rewrote JSONata with AI in a day, saved $500k/year

#227

Post by nirb89 »

otterley wrote: post #221
nirb89 wrote: post #117

Hey all, I'm the author of the blog post. I'm honestly loving the discussion this is generating (including the less flattering comments here). I'll try to answer some of the assumptions I've seen, hopefully it clears a few things. First off - some numbers. We're a near real-time cybersecurity platform, and we ingest tens of billions of raw events daily from thousands of different endpoints across SaaS. Additionally,…

> Finally, regarding the $300k/year cost (which many here seem to be horrified by) - it seems I wasn't clear enough in the blog. 200 pods was not the entire fleet, and it was not statically set. It was a single cluster at peak time. We have multiple clusters, each with their own traffic patterns and auto-scaling configurations. The total cost was $25k/month when summed as a whole. So, then, what do you estimate the a…

200 pods was peak allocation on a specific cluster, not total sustained cost for all of prod. The savings are taken by quite literally looking at the last month's bill on the cloud, compared to the new one after all optimizations applied and resources were aligned.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#228

Post by andersmurphy »

9wzYQbTYsAIc wrote: post #177
andersmurphy wrote: post #134

Earlier quoted context omitted.

Wonder if the real value of LLMs/AI is similar to microservices in that it solves an organisational/culture problem. In this case AI allowed the developer to make a change that the organisation would not have allowed. Regular rewrites don't let you signal to investors that you are AI ready/ascendant/agentic (whatever the latest AI hype term is) so would have been blocked. But, an AI rewrite.

If the only thing LLMs did was clear enterprise technical debt backlogs, the end consumer would still benefit from the technology.

That assumes they don't accelerate the accumulation of technical debt. For each item cleared how many new ones are added. LLMs accelerate your good engineers and your bad ones. So the slop likely increase faster than it can be cleared.

Re: We rewrote JSONata with AI in a day, saved $500k/year

#229

Post by danso »

macNchz wrote: post #216
xnorswap wrote: post #171

Earlier quoted context omitted.

Perhaps it's specific because it's Opus 4.6, released February 5th. https://www.anthropic.com/news/claude-opus-4-6

Opus 4.5 to 4.6 was pretty incremental, I didn't see much of a difference. The big coding model moments in recent recollection, IMO, were something like: - Sonnet 3.5 update in October 2024: ability to generate actually-working code using context from a codebase became genuinely feasible. - Claude 4 release in May 2025: big tool calling improvements meant that agentic editors like Claude Code could operate on a notic…

Maybe they're like me, who didn't spend a lot of time investigating Claude until 4.6 launched and the hype was enough to be the tipping point to invest energy. I do know that I've been having good/great results with Opus 4.6 and the CLI, but after an hour or so, it'll suddenly forget that the codebase has tab-formatted files and burn up my quota trying to figure out how to read text files. And apparently this snafu has been around since at least late last year [0]. Again, I can't complain about the overall speed and quality for my relatively light projects, I'm just fascinated by people who say their agents can get through a whole weekend without supervision, when even 4.6 appears to randomly get tripped up in a very rookie way?

[0] https://github.com/anthropics/claude-code/issues/11447

Re: We rewrote JSONata with AI in a day, saved $500k/year

#230

Post by kjksf »

sarchertech wrote: post #169
raincole wrote: post #130

Earlier quoted context omitted.

> Those same engineers will have to review and understand all of the AI-generated code now and then improve it, which will take time too. Will they? What makes you think so? If no one cared to improve it when it costed $300k/year, no one will care it when it's cheaper now.

They’ll be forced to work on it when then the bugs in the new system are uncovered. If the system is simple enough someone might take enough time to understand and verify the test suite to the point where they can keep adding regression tests to it and maybe mostly call it done. They probably won’t do this though (based on the situation the company was in in the first place) and people will have Claude fix it and wri…

Fixing bugs is the goldilocks zone for ai. Especially if you have a test that the agent can use to test their fix.

AI is not a junior developer, as some analogise, but Rain Man. Ultra autistic entity that can chew through way more logical conditions that you.

As long as you can describe the bug well ai will likely fix it. Logs help.

Let me give you specific example.

Here's a fix made by claude to my SumatraPDF: https://github.com/sumatrapdfreader/sumatrapdf/commit/a571c0...

I have a crash reporting system that sends me crash information in text file: callstack of crashed thread, basic os info and logs of this execution.

The way I (well, claude) fixed this bug is: I said "analyze crash report: " and it does it in under a minute.

Recently I've fixed at least 30 bugs with this process (you can view recent checkins).

Those are crashes that I found hard to fix because even though by human standard I'm both expert developer and expert Windows API developer.

But I'm not an autistic machine that can just connect the dots between how every windows api works and how it ties to the callstack and information from the log, in under a minute.

Post reply on HN