Don't tell me why, but I've developed an instinct that recognizes solutions that use Clickhouse under the hood :)
Tell us more!
Show HN: A Better Log Service
11–20 of 109 posts
Re: Show HN: A Better Log Service
#12This looks very interesting! My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup. I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing tho…
PHP is arguably the best solution here. If a log ingestion process breaks everything, no other logs are harmed (a default shared-nothing architecture). Using something like Go, C#, etc, it might be "faster" but less resilient -- or more complex to handle the resiliency. > But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. I'm sorry, but wut? PHP is probably one of…
I'm going to have to also reply with, sorry but what?!
PHP is one of the fastest-interpreted languages. But compiled are going to be faster than interpreted pretty much everytime. It loses benchmarks against every language. That's not to mention it's slowed down by the fact it have to rebuild everything per request.
As a PHP developer for 15+ years, I can tell you what PHP is good at and what PHP is not good at. High throughput API endpoints such as log ingestion are not a good fit for PHP.
Your argument that if it breaks it's fine. Yea, who wants a log system that will only log some of your logs? No one. It's not mission critical but it's pretty important to keep working if you want to keep your system working. And in fact, some places it is a legal requirement.
Re: Show HN: A Better Log Service
#13This looks very interesting! My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup. I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing tho…
It uses Clickhouse, though, which should be xtremelly fast for this.
Yes. But PHP still needs to process it before it goes to Clickhouse. PHP is the bottleneck.
Re: Show HN: A Better Log Service
#14This looks very interesting! My suggestion for the self-hosting is to create docker images and use docker-compose. The self-hosting currently is a bit of effort to setup. I also wonder if PHP is a good language for this. For the UI, yea that's fine and makes sense. But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. For the same resources, you can have Go doing tho…
No benefit using go over C#, IMO, and I am also baffled by the switch
I just used go as an example, any compiled language would be better.
Re: Show HN: A Better Log Service
#15Don't tell me why, but I've developed an instinct that recognizes solutions that use Clickhouse under the hood :)
Seems to be using MySQL instead?
https://github.com/WillieBeek/txtlog/blob/master/txtlog/data...
Re: Show HN: A Better Log Service
#16Very nice. A lot of the complexity you described is why I've settled on using CloudWatch logs for anything I have on AWS. I don't need a fancy UI, just a powerful querying language for investigation and debugging. With that said, it would be nice to see at least some mechanism for building aggregates queries (for example, 4* results in the last 24 hours by user) but if it's ClickHouse underneath, I assume that's easy…
I hate how Cloudwatch itself is so fragmented, and they have three different query languages for logs. It’s all cognitive overhead I don’t want to learn.
I will say that the language isn't the most intuitive, and a project like this one with some simply querying with the (presumed) ability to drop down to SQL for power use is probably the ideal solution. (Doable with CloudWatch logs and Athena, but that's another can of complex worms)
Re: Show HN: A Better Log Service
#17Earlier quoted context omitted.
PHP is arguably the best solution here. If a log ingestion process breaks everything, no other logs are harmed (a default shared-nothing architecture). Using something like Go, C#, etc, it might be "faster" but less resilient -- or more complex to handle the resiliency. > But for the log processor that's going to need to handle a high throughput which PHP just isn't good at. I'm sorry, but wut? PHP is probably one of…
I'm going to have to also reply with, sorry but what?! PHP is one of the fastest-interpreted languages. But compiled are going to be faster than interpreted pretty much everytime. It loses benchmarks against every language. That's not to mention it's slowed down by the fact it have to rebuild everything per request. As a PHP developer for 15+ years, I can tell you what PHP is good at and what PHP is not good at. High…
> It loses benchmarks against every language.
Every language loses benchmarks against every other language. That's not surprising. Since you didn't provide a specific benchmark, it's hard to say why it lost.
> High throughput API endpoints such as log ingestion are not a good fit for PHP.
I disagree; but ultimately, it depends on how you're doing it. You can beat or exceed compiled languages in some cases. PHP allows some low-level stuff directly implemented in C and also the high-level stuff you're used to in interpreted languages.
Re: Show HN: A Better Log Service
#18This is nice.
At work, we use Datadog for logging, and I have previously used CloudWatch, Splunk, and Honeycomb. Among these, only Honeycomb makes implementing canonical log lines [1] easier. I want arbitrarily wide, structured logs [2] without paying exorbitant costs for cardinality.
Our Datadog costs are outrageous, and it seems like no one cares at this point. Pydantic Logfire is also doing some good work in Python-specific environments. I use both Python and Go, but Logfire wasn’t as ergonomic in Go.
[1]: https://stripe.com/blog/canonical-log-lines
[2]: https://www.honeycomb.io/blog/structured-events-basis-observ...
Re: Show HN: A Better Log Service
#19Very nice. A lot of the complexity you described is why I've settled on using CloudWatch logs for anything I have on AWS. I don't need a fancy UI, just a powerful querying language for investigation and debugging. With that said, it would be nice to see at least some mechanism for building aggregates queries (for example, 4* results in the last 24 hours by user) but if it's ClickHouse underneath, I assume that's easy…
I hate how Cloudwatch itself is so fragmented, and they have three different query languages for logs. It’s all cognitive overhead I don’t want to learn.
I would be happy to pay a premium for a better cloudwatch. For me it is always not intuitive which I am sure is driven by limited use.