Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MemSQL is now free to use for databases with up to 128GB of RAM usage

memsql.com

Post reply on HN
Print view View original

31–40 of 121 posts

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#31

Post by CuriouslyC »

nikita wrote: post #14
quangio wrote: post #8

What is the benefit of using MemSQL over some other free in memory databases like Apache Ignite ? I see that they have better documentation and support (edit: + competition on Codeforces which winners rarely receive their T-Shirts). What about other things?

MemSQL CEO here. There are a few: - MemSQL is transactional and writes transactions on disk - MemSQL has an excellent implementation of SQL with mature query optimization and query execution. And it get better every release. This is from 6.5 https://www.memsql.com/blog/6.5-performance/ - MemSQL has in-memory and on-disk data storage so you can use MemSQL to store petabytes - MemSQL has columnstores and vectorized que…

Those are some nice features. Unfortunately, your DBMS doesn't do some basic things like return consistent results for a simple SQL query with a group by and having clause. I admit this might be a configuration issue on my company's end, but if so, that is a terrible configuration option and should be hidden away, opt in only, with a huge wall of warnings so people don't actually enable it except in extreme circumstances.

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#32

Post by PeterisP »

shin_lao wrote: post #12

I don't understand the high frequency use case they describe. High frequency trading is something very different from "12,000 transactions a minute". What is exactly the use case? Pre-deal checks? Post-deal checks? Book replay? Or is it just a simulation? It's not very clear.

HFT can easily be 12000 transactions a minute with a requirement for each of these transactions to be very fast.

It's not (necessarily) about high throughput, but about low latency.

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#33

Post by CuriouslyC »

setheron wrote: post #2

is memSQL a replacement for a full RDBMS in your stack ? Or just for analytics -- siphoning off the data from your master.

Personally, I would not use memsql as a first line RDBMS. It lacks too many useful features, and it doesn't return consistent results for all queries (I've found inconsistencies when using group by with having, for instance).

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#34

Post by lhorie »

timmorgan wrote: post #25
siscia wrote: post #11

Sort of shameless plug, but if you are looking into such solution you could have a look also at https://redisql.com/ It is a redis module that embed SQLite, bringing on the table a lot of advantages. Extremely fast and with it you can even upgrade your Redis instance to be your only database. There is not a huge company behind it, but I really do my best to support it. I don't believe to have disappointed any of our…

Cool project! I will check this out. Note: you have a typo on your home page: "never loose a bit" 'loose' => 'lose'

Spotted another typo: simplicity in "The power of SQL with the simplicty of Redis" is missing an i

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#36

Post by neildahlke »

CuriouslyC wrote: post #31
nikita wrote: post #14

Earlier quoted context omitted.

MemSQL CEO here. There are a few: - MemSQL is transactional and writes transactions on disk - MemSQL has an excellent implementation of SQL with mature query optimization and query execution. And it get better every release. This is from 6.5 https://www.memsql.com/blog/6.5-performance/ - MemSQL has in-memory and on-disk data storage so you can use MemSQL to store petabytes - MemSQL has columnstores and vectorized que…

Those are some nice features. Unfortunately, your DBMS doesn't do some basic things like return consistent results for a simple SQL query with a group by and having clause. I admit this might be a configuration issue on my company's end, but if so, that is a terrible configuration option and should be hidden away, opt in only, with a huge wall of warnings so people don't actually enable it except in extreme circumsta…

I've seen you mention these inconsistent results twice here in this this thread, but have worked at MemSQL for 5 years and never heard of such an issue. Have you reached out to see if maybe your query / data is not what you expect? I've seen inconsistent results only once, and it was because the default date formats across RDMBSs were different (and was not anticipated).

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#37

Post by hanson5b »

starik36 wrote: post #28
no1youknowz wrote: post #15

I've been using MemSQL in development since March 2017. I've seen it evolve into one of the fastest databases for columnar storage and analytical workloads. Having in-memory rowstore as well removed the need to have Aerospike in my infrastructure and simplified the whole stack. This announcement, that it's now free to 128GB pretty much saved me from having to do a kickstarter to raise funds for my little SaaS project…

It says that he 128GB version comes with High Availability features. Does that mean if I have 2 nodes, each one is limited to 64GB? Or each one can have 128?

A typical setup for HA would be 2 aggregator nodes and 2 leaf nodes. You can allocate the memory however you want. For example, you could give 32GB to all 4. Or give 16GB to each aggregator and 48GB to each leaf. (I'm a MemSQL Product Manager by the way.)

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#38

Post by rickn »

starik36 wrote: post #28
no1youknowz wrote: post #15

I've been using MemSQL in development since March 2017. I've seen it evolve into one of the fastest databases for columnar storage and analytical workloads. Having in-memory rowstore as well removed the need to have Aerospike in my infrastructure and simplified the whole stack. This announcement, that it's now free to 128GB pretty much saved me from having to do a kickstarter to raise funds for my little SaaS project…

It says that he 128GB version comes with High Availability features. Does that mean if I have 2 nodes, each one is limited to 64GB? Or each one can have 128?

I am the Director of Product Management for MemSQL.

The 128 GB limit applies to the whole cluster. So if you have two nodes in the cluster they would each have to be 64GB or less. If you have four nodes they would all have to be 32 GB or less. To have a highly available system we recommend 4 nodes (a master aggregator, a child aggregator and two leaf nodes). You can read more about the cluster architecture here: https://docs.memsql.com/concepts/v6.7/distributed-architectu...

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#39

Post by nikita »

curiousDog wrote: post #30
nikita wrote: post #14

Earlier quoted context omitted.

MemSQL CEO here. There are a few: - MemSQL is transactional and writes transactions on disk - MemSQL has an excellent implementation of SQL with mature query optimization and query execution. And it get better every release. This is from 6.5 https://www.memsql.com/blog/6.5-performance/ - MemSQL has in-memory and on-disk data storage so you can use MemSQL to store petabytes - MemSQL has columnstores and vectorized que…

If you’re hitting the disk, aren’t you losing some of the advantages of using an in-memory database in the first place? Or would it still be more performant than a traditional RDBMS due to optimized in memory data structures?

There is a compound effect in building memory optimized features and being a distributed database that can put a lot more cores at work and cache a lot more data in a cluster:

- In-memory row stores. Super fast for updates and point lookups

- Memory optimized hash joins that minimized cash misses. Great for analytical/reporting use cases

- Vectorization for columnstore query processing. Super fast aggregations that work best when data is cached in memory

Re: MemSQL is now free to use for databases with up to 128GB of RAM usage

#40

Post by ruw1090 »

CuriouslyC wrote: post #33
setheron wrote: post #2

is memSQL a replacement for a full RDBMS in your stack ? Or just for analytics -- siphoning off the data from your master.

Personally, I would not use memsql as a first line RDBMS. It lacks too many useful features, and it doesn't return consistent results for all queries (I've found inconsistencies when using group by with having, for instance).

If you have an example of the query you thought was returning inconsistent results, feel free to make a post over at https://www.memsql.com/forum/ with a repro.
Post reply on HN