Earlier quoted context omitted.
I have looked into ElasticSearch + Kibana as a solution to aggregate logs. There may be plenty of choice to replace ElasticSearch(ClickHouse, even Postgres, heck even journald), but a nice UI where you can simply search for that random piece of text you need to sift through the logs is the red herring. Until now, I have not seen a web interface to log as powerful as Kibana that can work with anything other than Elast…
Grafana?
I don't think Elasticsearch is a good logging system
41–50 of 117 posts
Re: I don't think Elasticsearch is a good logging system
#42What's worse (and I've seen this trend a lot) is to use Elasticsearch for metrics. My god.
> (and I've seen this trend a lot) any idea why? Seems like an odd approach to me.
Re: I don't think Elasticsearch is a good logging system
#43I believe the complaints here are a case of 'not using it correctly'. The 'Reverse index' (Lucene's inverted index) is a fundamental data structure used to enable very fast search. Other data structures, like KD trees, are used for non-text data types. If you're not doing full text search, don't use `text` fields. If you're not querying the data, why store it in the first place? Full text search for logs is incredibl…
My point was that by the time you filter on keyword fields (and other exact matching fields), the number of logs is small enough that an efficient full text search isn't necessary. That doesn't mean that full text search itself isn't useful, just that maintaining an inverted index is overkill in the logging case
Re: I don't think Elasticsearch is a good logging system
#44I believe the complaints here are a case of 'not using it correctly'. The 'Reverse index' (Lucene's inverted index) is a fundamental data structure used to enable very fast search. Other data structures, like KD trees, are used for non-text data types. If you're not doing full text search, don't use `text` fields. If you're not querying the data, why store it in the first place? Full text search for logs is incredibl…
My point was that by the time you filter on keyword fields (and other exact matching fields), the number of logs is small enough that an efficient full text search isn't necessary. That doesn't mean that full text search itself isn't useful, just that maintaining an inverted index is overkill in the logging case
My gripe with ES is that it won't let you do post-pass filtering at all. If you create an index with a few keyword fields indexed and then some unindexed fields, you can't query the unindexed fields.
Grafana's Loki seems to be exactly what we are looking for, although I haven't played with it.
Re: I don't think Elasticsearch is a good logging system
#45I can't say I agree with this article. Elasticsearch is a bit of a Swiss army knife and definitely needs some setup and configuring. But if you set it up properly, it does logs extremely well. And it does actually come with useful defaults and well integrated tools. For example, if you use the various Beats agents to push data to Elasticsearch, they come with a default schema and out of the box dashboards that they c…
Anyone stuck paying for Splunk will think they probably should be.
Re: I don't think Elasticsearch is a good logging system
#46What's worse (and I've seen this trend a lot) is to use Elasticsearch for metrics. My god.
> (and I've seen this trend a lot) any idea why? Seems like an odd approach to me.
It seems to commonly be perceived as a panacea of databases.
Re: I don't think Elasticsearch is a good logging system
#47Usually when this happens is because the source uses what could be considered user input as keys, for example in access logs with stuff such as headers o query parameter, and you make a new field for each header or query parameter. In the latter case all you need to exhaust your available fields in the index is some nasty bot trying random query strings on your site. This can be easily solved in the ingest/logstash pipeline
It is true that you have to spend time to carefully map and process the documents you're ingesting in elasticsearch, but once you do I don't think there's any other solution with the same performance and features for logs, especially considering the licensed features (like alerting from anomaly detection machine learning jobs)
The elastic stack is complex and might be hard to grasp (their own training courses are a great help though), requires significant computing resources, and managing a big deployment can be a full time job, but once implemented correctly you can do awesome stuff with it.
Loki is a simpler and slower solution which does less things, so if you don't need what elastic provides, it could be a better fit as it's probably easier to manage
Re: I don't think Elasticsearch is a good logging system
#48Re: I don't think Elasticsearch is a good logging system
#49Earlier quoted context omitted.
My point was that by the time you filter on keyword fields (and other exact matching fields), the number of logs is small enough that an efficient full text search isn't necessary. That doesn't mean that full text search itself isn't useful, just that maintaining an inverted index is overkill in the logging case
Completely agree. My gripe with ES is that it won't let you do post-pass filtering at all. If you create an index with a few keyword fields indexed and then some unindexed fields, you can't query the unindexed fields. Grafana's Loki seems to be exactly what we are looking for, although I haven't played with it.
Your complaint really doesn't make sense, how would you query an unindexed field? Elasticsearch is a _search_ engine, which means it needs to index content that is to be discoverable. What you're saying with unindexed fields is you're completely fine with those not being included in any search or filtering.
Re: I don't think Elasticsearch is a good logging system
#50Earlier quoted context omitted.
My point was that by the time you filter on keyword fields (and other exact matching fields), the number of logs is small enough that an efficient full text search isn't necessary. That doesn't mean that full text search itself isn't useful, just that maintaining an inverted index is overkill in the logging case
Completely agree. My gripe with ES is that it won't let you do post-pass filtering at all. If you create an index with a few keyword fields indexed and then some unindexed fields, you can't query the unindexed fields. Grafana's Loki seems to be exactly what we are looking for, although I haven't played with it.
Without knowing for sure I imagine they originally expected the application side to handle this but many of the current solutions don’t do that. And they expose and overload the elastic search query language as the primary search interface with no additional app logic. The elastic search query “is” the search application.
Making some assumptions but might reconcile the different viewpoints on why it does or doesn’t make sense.