Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A Better Log Service

txtlog.net

Post reply on HN
Print view View original

1–10 of 109 posts

Show HN: A Better Log Service

#1

Post by williebeek »

Hello everyone, there are many log services available and this is my attempt at a better one.

Most online logging tools feature convoluted UIs, arbitrary mandatory fields, questionable AI/insights, complex pricing, etc. I hope my application fixes most of these issues. It also has some nice features, such as automatic Geo IP checks and public dashboards.

Although I've created lots of software, this is my first open source application (MIT license), the tutorial for selfhosting is hopefully sufficient! Most of my development career has been with C#, NodeJS and PHP. For this project I've used PHP (8.3) which is an absolute joy to work with. The architecture is very scalable, but I've only tested up to a few billion logs. The current version is used in production for a few months now. Hope you enjoy/fork it as you see fit!

Show HN: A Better Log Service
txtlog.net

Re: Show HN: A Better Log Service

#2

Post by that_guy_iain »

This looks very interesting!

My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup.

I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing thousands of requests per second vs PHP doing hundreds of requests per second.

Re: Show HN: A Better Log Service

#4

Post by majkinetor »

that_guy_iain wrote: post #2

This looks very interesting! My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup. I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing tho…

No benefit using go over C#, IMO, and I am also baffled by the switch

Re: Show HN: A Better Log Service

#5

Post by majkinetor »

that_guy_iain wrote: post #2

This looks very interesting! My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup. I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing tho…

It uses Clickhouse, though, which should be xtremelly fast for this.

Re: Show HN: A Better Log Service

#6

Post by withinboredom »

that_guy_iain wrote: post #2

This looks very interesting! My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup. I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing tho…

PHP is arguably the best solution here. If a log ingestion process breaks everything, no other logs are harmed (a default shared-nothing architecture). Using something like Go, C#, etc, it might be "faster" but less resilient -- or more complex to handle the resiliency.

> But for the log processor that's going to need to handle a high throughput which PHP just isn't good at.

I'm sorry, but wut? PHP is probably one of the fastest languages out there if you can ignore frameworks. It's backed by some of the most tuned C code out there and should be just about as fast as C for most tasks. The only reason it is not is due to the function call overhead -- which is by-far the slowest aspect of PHP.

> you can have Go doing thousands of requests per second vs PHP doing hundreds of requests per second.

This is mostly due to nginx and friends ... There is frankenphp (a frontend for php running in caddy which is written in go) which can easily handle 80k+ requests per second.

Re: Show HN: A Better Log Service

#7

Post by bdcravens »

Very nice. A lot of the complexity you described is why I've settled on using CloudWatch logs for anything I have on AWS. I don't need a fancy UI, just a powerful querying language for investigation and debugging. With that said, it would be nice to see at least some mechanism for building aggregates queries (for example, 4* results in the last 24 hours by user) but if it's ClickHouse underneath, I assume that's easy using standard ClickHouse tools.

Re: Show HN: A Better Log Service

#9

Post by dobin »

Pretty unrelated, but i like how it displays large amount of potentially diverse JSON events. Would need some better filtering and sorting, hiding of keys etc. Products which do this well are Elastic and Splunk, but are too heavy for my taste.

Re: Show HN: A Better Log Service

#10

Post by stingraycharles »

bdcravens wrote: post #7

Very nice. A lot of the complexity you described is why I've settled on using CloudWatch logs for anything I have on AWS. I don't need a fancy UI, just a powerful querying language for investigation and debugging. With that said, it would be nice to see at least some mechanism for building aggregates queries (for example, 4* results in the last 24 hours by user) but if it's ClickHouse underneath, I assume that's easy…

I hate how Cloudwatch itself is so fragmented, and they have three different query languages for logs.

It’s all cognitive overhead I don’t want to learn.

Post reply on HN