Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: FixBugs – Reproduce production bugs and verify fixes

fixbugs.ai

Post reply on HN
Print view View original

21–30 of 46 posts

Re: Show HN: FixBugs – Reproduce production bugs and verify fixes

#22

Post by kirtivr »

majestic8 wrote: post #15

This looks very useful! What type of sandbox are you using? How does the mocking work?

On Linux, we rely on a chroot-ed workspace at this time - although we are working on a prototype using the new landlock kernel interface (https://github.com/Zouuup/landrun).

OSX is the best, we use the in-built (seatbelt) sandbox via sandbox-exec.

For Windows, we use WSL containers when available.

By default, if a safe sandbox environment is not available, we inform the user that a repro is not possible in the current conditions.

Re: Show HN: FixBugs – Reproduce production bugs and verify fixes

#23

Post by kirtivr »

Bheemaneni wrote: post #21

This seems especially used for teams handling production incidents.

It's really nice to hear that.

FixBugs was built because while investigating prod incidents, I had an epiphany.

SWEs build tools to solve all types of problems, but we ourselves use the flakiest tools.

While working at Google for example I was surprised GDB support for any type of binary debugging was almost non-existent.

Re: Show HN: FixBugs – Reproduce production bugs and verify fixes

#25

Post by kirtivr »

majestic8 wrote: post #15

This looks very useful! What type of sandbox are you using? How does the mocking work?

On how does the mocking work, that's a really interesting question.

We do a lot of AST parsing - for both code and build configuration languages. Even then, we still have to rely on the LLM to figure out a lot of the details.

Making this work reliably for non-frontier models and codebases that don't have existing test harnesses is where a lot of the design work goes in.

Re: Show HN: FixBugs – Reproduce production bugs and verify fixes

#28

Post by kirtivr »

Rishi1238 wrote: post #24

Interesting project. How does it handle services with multiple dependencies (queues, caches, third-party APIs)? Can it recreate enough of the production environment to reproduce intermittent bugs?

We either mock or fake out the interfaces.

I much prefer faking to mocking, because that still preserves a lot of the real-world behavior relevant to prod bugs.

A full prod reproduction would be a holy grail, but probably only attainable for complex distributed systems if we have access to a prebuilt staging-like environment.

We're traversing the bridge between mocks, fakes and staging at this time.

Re: Show HN: FixBugs – Reproduce production bugs and verify fixes

#29

Post by kirtivr »

kumari_b wrote: post #26

Interesting approach. The investigaion phase is usually the most time-consuming part of debugging. Curious how well this works on large, distributed systems.

Nginx, Caddy, Flink and Firefox are some applications where we we've managed to consistently fix and reproduce reported bugs.

Of course, we did not send the PRs to the repos seeing how they're already overloaded with them.

Nginx for example has 181 open PRs right now, but they only merge 2 or 3 in a day.

Re: Show HN: FixBugs – Reproduce production bugs and verify fixes

#30

Post by kumari_b »

kirtivr wrote: post #29
kumari_b wrote: post #26

Interesting approach. The investigaion phase is usually the most time-consuming part of debugging. Curious how well this works on large, distributed systems.

Nginx, Caddy, Flink and Firefox are some applications where we we've managed to consistently fix and reproduce reported bugs. Of course, we did not send the PRs to the repos seeing how they're already overloaded with them. Nginx for example has 181 open PRs right now, but they only merge 2 or 3 in a day.

Thanks for the clarification. Those are some interesting projects to benchmark against.
Post reply on HN