I like how this is mostly based on the Kibana UI. Makes easier to convince other people to move to it.
Honestly that pushes me away from it. I find kibana to be a very frustrating experience.
(not op) curious what you find frustrating about it?
Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
61–70 of 75 posts
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#62Would this also work with something like Plausible ( https://github.com/plausible/analytics ) which uses ClickHouse to store web analytics data, or is it primarily for log data?
Despite the fact that Telescope is focused on application log data, it could be used for any type of data as long as it's stored in ClickHouse and has some time fields. At the moment, I have no plans to support arbitrary data visualization in Telescope, as I believe there are better BI-like tools for that scenario.
Yeah that's fair, thank you.
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#63Looks cool!
If you're looking for this kind of UI also check out Coroot https://github.com/coroot/coroot which has awesome UI for logs and OpenTelemetry traces and also stores data in Clickhouse
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#64Is there any comprehensive guide in building observability stack using otel, clickhouse, and grafana? I think this is a solid stack for logging & tracing, but I've been looking into it but haven't found any authoritative reference for this particular stack (unlike ELK & LGTM stack)
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#65Very cool! Tested it with the demo, very smooth!
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#66It would be great if the logs could describe a bit what exactly one has to do to use this as an alternative to Grafana Loki. How do I get my logs (e.g. local text files from disk like nginx logs, or files that need transformation like systemd journal logs) into ClickHouse in a way that's useful for Telescope? What kind of indices do I have to configure so that queries are fast? Ideally with some examples. How can I m…
As someone who has never worked anywhere that tried it out, what do you not like about loki. I've been stuck in the very expensive splunk and opensearch/kibana mines for many years and I find it an amazingly frustrating place to be. I honestly find that I can better debug via logs using grep than either of those tools.
Loki works fine for what it does; the problem is what it lacks.
It doesn't do full-text search indices. So if you just search for some word across all your logs (to find eg when a rare error happened), it is very slow (it runs the equivalent of grep, at 500 MB/s on my machine). If you have a couple TB, it takes half an hour!
As you say, even plain grep is usually faster for such plain linear search.
I want full-text indices so that such searches take milliseconds, or a couple seconds at most.
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#67It would be great if the logs could describe a bit what exactly one has to do to use this as an alternative to Grafana Loki. How do I get my logs (e.g. local text files from disk like nginx logs, or files that need transformation like systemd journal logs) into ClickHouse in a way that's useful for Telescope? What kind of indices do I have to configure so that queries are fast? Ideally with some examples. How can I m…
Telescope is primarily focused on log visualization, not on log collection or preparing ClickHouse for storage. The system does not currently provide (and I think will not ever) built-in mechanisms for ingesting logs from any sources. I will consider providing a how-to guide on setting up log storage in ClickHouse, but I’m afraid I won’t be able to cover all possible scenarios. This is a highly specific topic that de…
I think such a guide would be great.
My perspective:
A lot of people who operate servers (including me) just want to view and search their logs -- fast and convenient. Your tool provides that. They don't care about whether the backend uses ClickHouse or Postgres or whatever, that's just a pesky detail. They understand they may have to deal with it to some extent, but they don't want to have to become experts at those, and to conclude everything by themselves, just to read their logs.
Also, those things are general-purpose databases, so they don't tell the user how to best set them up so your tool can produce results fast and convenient. So currently, neither side helps the user with that.
That's why it's best if your tool's docs gives some basic tips on how to achieve the most commonly desired goals: Some basic way to get logs into the backend DB (if there's a standard way to do that for text log files and journald, probably fine to just link it), and docs on what indices Telescope needs to be faster than grep for typical log search tasks (ideally with some quick snippet or link on how to set those up, for people who haven't used ClickHouse before).
So overall, it's fine if the tool doesn't do everything. But it should say what it needs to work well.
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#68Earlier quoted context omitted.
Honestly that pushes me away from it. I find kibana to be a very frustrating experience.
(not op) curious what you find frustrating about it?
at the enterprise scale on the backend you end up paying for a bunch of indexing you will likely never use. On top of that you spend a LOT of money in engineering hours setting up indexes for many teams all with different log formats so the whole thing doesn't just melt down.
On the kibana side, their query language is unshared by any other tool, at least any that I use, meaning that in the middle of an outage I end up chasing my tail reading docs on how to query what you want. The returns are often slow and it's very hard to just export the logs you do find to text files so you can ingest them into other tools.
I mean I came up on cat/gerp/awk/sed/less/tail/(more recently jq for json logs) .. it wasn't perfect but it was RESPONSIVE and portable.
I just think that tools like ES/Splunk weren't conceived for dealing with logs (especially if your logs come in many formats) and are both overkill and at the same time underkill for the task. It's like using a ball peen hammer to drive nails, you can certainly DO it, but a claw hammer is cheaper and a more ergonomic experience.
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#69Looks cool I might try it out! I need a central place, something simple where I can actually read the contents of the logs that are generated by the dozen of services that I run for clients, etc… instead of stupidly SSH’ing to every server. Does this fit the use case? I tried Loki once but it was painful to set up and more geared toward aggregating events and stats.
I'd recommend VictoriaLogs and shipping to via Vector
I also recommend to not hesitate to use other log shippers as well as VictoriaLogs support ingestion not only from Vector - see https://docs.victoriametrics.com/victorialogs/data-ingestion...
Re: Show HN: Telescope – an open-source web-based log viewer for logs in ClickHouse
#70Earlier quoted context omitted.
As someone who has never worked anywhere that tried it out, what do you not like about loki. I've been stuck in the very expensive splunk and opensearch/kibana mines for many years and I find it an amazingly frustrating place to be. I honestly find that I can better debug via logs using grep than either of those tools.
Loki works fine for what it does; the problem is what it lacks. It doesn't do full-text search indices. So if you just search for some word across all your logs (to find eg when a rare error happened), it is very slow (it runs the equivalent of grep, at 500 MB/s on my machine). If you have a couple TB, it takes half an hour! As you say, even plain grep is usually faster for such plain linear search. I want full-text…
see to me, having at one point been responsible for maintaining an ES instance for logs (and exporters and all the other bits) I feel like the prices you pay in engineering hours and hardware costs to maintain all those indexes while keeping ES from absolutely melting down is way too high.
I think grep is amazing but yes if you unleash it on 'all the logs' without narrowing yourself down to a time frame first or some other taxonomy is going to be slow. This seems like a skill issue, frankly.
Also full text indexes for all the things are generally FASTER of course, but seconds/milliseconds? How much hardware are you throwing at logs. Most only go to logs in an emergency, during an incident and the like. How much are you paying just to index a bunch of shit that will probably never even be looked at, and how much are you paying for hardware to run queries on those indexes that will be largely idle.
The problems with ES/Splunk for logs is that they were not designed for logs, so they are both, in my view, overkill AND underkill for the task. Full fuzzy text serch is probably overkill, the UI for the task of dealing with log data is underkill. (The cloud bills are certainly overkill)
I'm currently doing platform engineering at a company in the top half of the fortune 500. Honestly, probably about 90-95% of the time when I'm helping a team troubleshoot their service on kubernetes I'm using the kubectl `stern` plugin (shows log streams from all pods that match a label query) and grep/sed/awk/jq if it's ongoing, it's just waaaaay more responsive. If it's a 'weird thing happened last night, investigate' task and I have to go to Kibana it's just a much worse experience overall.