Oh, how I wish I had your scripts (and insights!) when I was analyzing Unix logs in 1986, looking for the footprints of an intruder...
Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
21–30 of 82 posts
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#22Oh, how I wish I had your scripts (and insights!) when I was analyzing Unix logs in 1986, looking for the footprints of an intruder...
You should write about that sometime! /s
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#23Oh, how I wish I had your scripts (and insights!) when I was analyzing Unix logs in 1986, looking for the footprints of an intruder...
I'm kinda glad you didn't; it might have made the book I read as a kid (and again as an adult, and again with my offspring) less interesting somehow.
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#24"This simple tool solves X at my org" is probably the most underrated type of project. There's not enough room to overcomplicate something that isn't a core part of the business, it must be practical to maintain, simple&stupid enough so that onboarding is not a hurdle, etc. I encourage everyone to share your "splunk in 1kloc of Python" projects! Some of my own: - https://github.com/rollcat/judo is Ansible without Pyt…
My org's apps heavily use this simple key-value interface built on sqlite: https://github.com/aaviator42/StorX
There's also a bunch of other purpose-built tiny utilities on that GitHub account: https://github.com/aaviator42?tab=repositories
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#25Earlier quoted context omitted.
Volume. 1GB of data per day is rounding error. If you have tens of thousands of servers, each generating hundreds of gigabytes of data per day, tail -f and grep don't scale especially well.
And I bet a hang glider can't fly from New York to Paris, either! The nerve! Recall that the poster said this was for a small startup. If you're Google, by all means, use Google logging tools. If you aren't, then solve the problem you have, not the problem your résumé needs.
The guy asked
> Twenty years later, I still can't fathom why we're spending so much money on Splunk, DataDog a the like.
And the poster above answered that question
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#26"This simple tool solves X at my org" is probably the most underrated type of project. There's not enough room to overcomplicate something that isn't a core part of the business, it must be practical to maintain, simple&stupid enough so that onboarding is not a hurdle, etc. I encourage everyone to share your "splunk in 1kloc of Python" projects! Some of my own: - https://github.com/rollcat/judo is Ansible without Pyt…
Thanks, based on the dismissive replies to my original comment in the Splunk acquisition discussion, I thought this would get a lot of hostile takes saying that it was dumb, that I reinvented the wheel because I didn't want to spend 2 weeks trying to figure out opentelemetry nonsense and tools X, Y, and Z, that it was trivial, that it wouldn't scale, etc. But people are actually being surprisingly nice and friendly!…
I suggest you sell it to Oracle, get some popcorn and watch the Cisco vs Oracle log war begin!
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#27"This simple tool solves X at my org" is probably the most underrated type of project. There's not enough room to overcomplicate something that isn't a core part of the business, it must be practical to maintain, simple&stupid enough so that onboarding is not a hurdle, etc. I encourage everyone to share your "splunk in 1kloc of Python" projects! Some of my own: - https://github.com/rollcat/judo is Ansible without Pyt…
> There's not enough room to overcomplicate something that isn't a core part of the business, it must be practical to maintain, simple&stupid enough so that onboarding is not a hurdle, etc.
You would think. But no, there is lots of room to make it over complicated without the corresponding efforts to manage the complexity.
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#28A long long time ago, I used a series of tail -f's and unix pipes to aggregate logs, and grep, less and awk to analyse them. There were about 20 different services written in C++, each producing over 1GB of logs each day. Managed to debug some fairly complex algorithmic trading bugs. Twenty years later, I still can't fathom why we're spending so much money on Splunk, DataDog an the like.
What did you use for visualization in that stack? The fact that I can "|" (pipe) my data and make bar and pie charts is what really does it for me. What's really money is trivially being able to see requests coming in overlaid on a world map. I was sold the first time I saw that because it let me fix an issue that would have taken me hours to suss out just grepping around.
More power to you for using sed awk and grep, they're powerful tools and every computer person should know how to use them. But if you're hung up on only using sed awk and grep for emotional reasons, that's self-limiting. We have better tools today, and you don't get hero points for using shittier tools when there are better ones available to you.
https://www.splunk.com/en_us/blog/tips-and-tricks/mapping-wi...
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#29> If I owned CSCO stock I would sell it-- this deal shows incredibly bad judgment." That may be so, but beware that acquisitions usually increase stock price rather than decrease it.
Is that overtime or immediately after acquisition? https://www.google.com/finance/quote/CSCO:NASDAQ?&window=5D
Re: Show HN: My Single-File Python Script I Used to Replace Splunk in My Startup
#30Earlier quoted context omitted.
Volume. 1GB of data per day is rounding error. If you have tens of thousands of servers, each generating hundreds of gigabytes of data per day, tail -f and grep don't scale especially well.
100GB of logs per day? what kind of applications are that chatty?
Yeah, the solution here is to get rid of 98% of the logging.