21–30 of 32 posts
Does Elasticsearch need to be as complicated as it is? I was surprised to find there wasn't an Elasticsearch + Kibana competitor that is "simpler". I just want to be able to store JSON logs with a timestamp + a bunch of fields then search them in a nice little UI later. Apparently, that's pretty hard to do right.
Does Elasticsearch need to be as complicated as it is? I was surprised to find there wasn't an Elasticsearch + Kibana competitor that is "simpler". I just want to be able to store JSON logs with a timestamp + a bunch of fields then search them in a nice little UI later. Apparently, that's pretty hard to do right.
My team is using Loki + Grafana and we're pretty happy with it. It's pretty basic but it does what you expect it to do just fine. We ditched Elastic as it was a super massive PITA to operate (& a resource hog at that). I'll admit I'm not an expert at ELK at all, but tbh I was absolutely surprised just how bad Elastic + Kibana was for our basic log uses when they tout it as one of their mainstays. Or we were just exce…
I didn't know Ruby, and I didn't know search engines, but I did know Common Lisp. It took me 7 months to create a binary-compatible, pretty dang functional port of Lucene. I called it Montezuma[1], and it's still actually used by people.
About half the time was spent implementing the text analyzer, document store, and indices. The remaining 50% was spent implementing search (and parsing the query language). It was a very rewarding experience--It was one of the largest projects I'd worked on, mostly solo (I got some help near the end), in an area I knew nothing about, working in an extremely test-driven fashion (over 2000 unit tests when I was done).
I did not, however, learn as much about indexing and search as I expected. I learned a lot, yes, but quite a bit of the Ruby code was so easy to translate to Common Lisp that I didn't have to understand everything in order to make it work.
I still recommend writing a small search engine as an interesting exercise, though.
Does Elasticsearch need to be as complicated as it is? I was surprised to find there wasn't an Elasticsearch + Kibana competitor that is "simpler". I just want to be able to store JSON logs with a timestamp + a bunch of fields then search them in a nice little UI later. Apparently, that's pretty hard to do right.
Earlier quoted context omitted.
My team is using Loki + Grafana and we're pretty happy with it. It's pretty basic but it does what you expect it to do just fine. We ditched Elastic as it was a super massive PITA to operate (& a resource hog at that). I'll admit I'm not an expert at ELK at all, but tbh I was absolutely surprised just how bad Elastic + Kibana was for our basic log uses when they tout it as one of their mainstays. Or we were just exce…
How has your experience with Loki been? I keep an eye on it, but haven't deployed it since it seems to be maturing still.
Does Elasticsearch need to be as complicated as it is? I was surprised to find there wasn't an Elasticsearch + Kibana competitor that is "simpler". I just want to be able to store JSON logs with a timestamp + a bunch of fields then search them in a nice little UI later. Apparently, that's pretty hard to do right.
ElasticSearch is really just clustered Lucene with some nice features wrapping it. You can probably get away with something that also wraps lucene. Though elasticsearch has a dominant position precisely because it is quite full featured and easy to run.
Does Elasticsearch need to be as complicated as it is? I was surprised to find there wasn't an Elasticsearch + Kibana competitor that is "simpler". I just want to be able to store JSON logs with a timestamp + a bunch of fields then search them in a nice little UI later. Apparently, that's pretty hard to do right.
> JSON logs with a timestamp + a bunch of fields then search them There is S3+Athena for this with AWS and Google can store/query JSON with BigQuery. The nice little UI doesn't come with it, but at least you don't have to spin up an Elastic cluster.
Back in 2006 I wanted to learn more about how search engines worked, so I started porting Lucene to Common Lisp. Actually, I wrote a Common Lisp port of Ferret. Ferret is a Ruby port of Lucene. Lucene is sort of Doug Cutting's Java version of Text Database (TDB), which he and Jan Pedersen developed at Xerox PARC, and which, to complete the circle, was written in Common Lisp. I didn't know Ruby, and I didn't know sear…
Hey. I wrote that a long time ago. Funny to see it re-surface and glad it's helpful. :) There is a presentation version of it here: https://www.youtube.com/watch?v=PpX7J-G2PEo&feature=youtu.be (I made the presentation first and then wrote the blog posts) I wrote a follow-up at called Elasticsearch from the Top Down here: https://www.elastic.co/blog/found-elasticsearch-top-down
Back in 2006 I wanted to learn more about how search engines worked, so I started porting Lucene to Common Lisp. Actually, I wrote a Common Lisp port of Ferret. Ferret is a Ruby port of Lucene. Lucene is sort of Doug Cutting's Java version of Text Database (TDB), which he and Jan Pedersen developed at Xerox PARC, and which, to complete the circle, was written in Common Lisp. I didn't know Ruby, and I didn't know sear…
[0] https://github.com/apache/lucene-solr/tree/master/lucene
[1] http://blog.mikemccandless.com/2019/10/concurrent-query-exec...