It would be helpful to know the largest data set this has been used with and the performance of some example queries. Ideally > 1TB to qualify as "big data".
Crate.io – Big Data SQL in real time
71–80 of 83 posts
Re: Crate.io – Big Data SQL in real time
#72If you were initially confused like I was, crate.io used to be the domain for an alternative Python Package Index. http://www.reddit.com/r/Python/comments/1wcp93/what_happened...
Confusing, but it seems to be moved to https://warehouse.python.org/ Edit: I'm not sure they are related
Re: Crate.io – Big Data SQL in real time
#73If you were initially confused like I was, crate.io used to be the domain for an alternative Python Package Index. http://www.reddit.com/r/Python/comments/1wcp93/what_happened...
It still is: https://simple.crate.io/ :-)
Re: Crate.io – Big Data SQL in real time
#74Earlier quoted context omitted.
we here at crate were living in the hadoop ecosystem for some time. but when we came across the beautiful architecture of netty.io (async, event-driven) and the way elasticsearch orchestrated it - since that time we know there will be room for newcomers :)
Well sorry my buzzword filter removed half of your sentence. :) I understand that engineers can get excited about async and event driven but these mean absolutely nothing to your endusers. I could implement your service with blocking IO and non-event driven code and still get the same performance. It simply does not matter that much. The key to Hadoop's success is scalability and predictable performance. Don't get me…
Re: Crate.io – Big Data SQL in real time
#75Just a minor detail, the video show the load of the table with 190K rows, but then the distinct shows 194K results.. that is weird, because the table was empty before the load. :P
Re: Crate.io – Big Data SQL in real time
#76Re: Crate.io – Big Data SQL in real time
#77Re: Crate.io – Big Data SQL in real time
#78Re: Crate.io – Big Data SQL in real time
#79Earlier quoted context omitted.
That is bad . I've done a lot of work with Elasticsearch[1] and while it's a great search engine, it is NOT a primary source of truth or something you want to trust not to lose your data. [1]: Built a startup's product on top of it and have written an open source client library for it.
why ? at least i remember they didn't recommend it for database until the backup funcionality was done
The data needs to go somewhere other than ES before you considered it "saved".
Re: Crate.io – Big Data SQL in real time
#80Some initial thoughts from looking at the website and documentation... 1. The main reason I want SQL is for relational data. The lack of joins basically makes this a "NoSQL" database in every respect except the query language being something resembling SQL. I'm fairly sure ANSI SQL requires support for joins. 2. It does auto-sharding, but I don't know how. The documentation doesn't specify how the data is sharded, an…
thanks for your summary and valuable feedback. a few notes from the crate team: 1. JOINs are on our roadmap. we might never reach full JOIN support anytime soon (INNER, LEFT OUTER, RIGHT OUTER, FULL OUTER and CROSS), but simple use-cases (1:n), equal joins shouldn't be too hard. 2. after re-reading our documentation on sharding i need to admit that we need to improve it. we mention it briefly here https://crate.io/do…