Dejavu – Web UI for Elasticsearch • Hacker News Board

Re: Dejavu – Web UI for Elasticsearch

#41

Post by bvm » Thu, Mar 22, 2018, 9:45 AM UTC

jstarfish wrote: post #26

giancarlostoro wrote: post #15

Ok so maybe someone can tell me what I (we) did wrong at my job we tried using the ELK stack, and it's probably still running but it is such a resource hog. I do not understand why they built Elasticsearch. I've read in a couple places you need like 32GB of RAM[0] just to run this thing to do queries, and having crashed Kibana / Elasticsearch a dozen times I believe it's designed poorly. I had hoped I could drop in M…

> tell me what I (we) did wrong at my job we tried using the ELK stack, and it's probably still running but it is such a resource hog. Part of the problem is what it's promoted for. It's a great drop-in, horizontally-scalable, full-text search engine, that's inexplicably become popular for log ingestion and analysis. To those ends, I hate it, I hate every bit of it, from the atrocious JSON-based query DSL (seriously…

> atrocious JSON-based query DSL (seriously thought it was a joke at first)

I could not agree more, I found this in my codebase https://i.imgur.com/44stiHv.png

I've never seen an odder choice of data structure.

Re: Dejavu – Web UI for Elasticsearch

#42

Post by bestest » Thu, Mar 22, 2018, 11:20 AM UTC

I've stumbled on a couple of bugs with Dejavu, and wanted to see if I can submit a bug fix. And, well, submit and fix them I can, but boy oh boy is the code terrible. Only use this on your own risk and on a throwaway ES instance, as the code quality in this case kind of implies there's major risk of screwing things up.

Re: Dejavu – Web UI for Elasticsearch

#43

Post by AznHisoka » Thu, Mar 22, 2018, 12:54 PM UTC

giancarlostoro wrote: post #15

Ok so maybe someone can tell me what I (we) did wrong at my job we tried using the ELK stack, and it's probably still running but it is such a resource hog. I do not understand why they built Elasticsearch. I've read in a couple places you need like 32GB of RAM[0] just to run this thing to do queries, and having crashed Kibana / Elasticsearch a dozen times I believe it's designed poorly. I had hoped I could drop in M…

Originally the most popular (and only) use case for Elasticsearch was for full text search. Then a bunch of people thought it was good for analyzing logs and decided to hijack the brand, and market it as a logging analyzer.

But in its core, Elasticsearch is meant for full text search and analytics. Not logging. Logging is not even an interestinng use case.

Re: Dejavu – Web UI for Elasticsearch

#44

Post by skingkong » Thu, Mar 22, 2018, 9:23 PM UTC

giancarlostoro wrote: post #15

Ok so maybe someone can tell me what I (we) did wrong at my job we tried using the ELK stack, and it's probably still running but it is such a resource hog. I do not understand why they built Elasticsearch. I've read in a couple places you need like 32GB of RAM[0] just to run this thing to do queries, and having crashed Kibana / Elasticsearch a dozen times I believe it's designed poorly. I had hoped I could drop in M…

We ran into this same problem trying to build and run a data management platform on it... IDK if you want to try what we built, but it's a hell of a lot faster and less cumbersome. We went from 30 ES servers down to 4 Pilosa servers.

https://github.com/pilosa/pilosa