Why cannot I use free open source MySQL heap or memory storage engine? It also provides clustering/replication
MemSQL is now free to use for databases with up to 128GB of RAM usage
51–60 of 121 posts
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#52Is there a way on this release to create auto-scale rules like on RDS?
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#53Earlier quoted context omitted.
The best thing to do in this situation would be to set the maximum memory on the MemSQL server via the `maximum_memory` system variable. The server will then internally manage it's memory usage against this upper bound and fail only specific operations which can't be performed without additional memory.
What are the specific operations? Will another batch of INSERTs / UPDATEs work? Does it matter if I use the analytical part of MemSQL ?
I mean that an operation that needs more memory than is available will fail, but the server itself will remain operational. An simple example would be loading more data into an in-memory table (row store) than there is available memory. If you're using the columnstore then your storage won't be limited to memory.
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#54Earlier quoted context omitted.
HFT can easily be 12000 transactions a minute with a requirement for each of these transactions to be very fast. It's not (necessarily) about high throughput, but about low latency.
No real HFT system is doing a * database transaction * in the critical trading path. HFT systems are not built like web applications. They are typically built as a tight event loop, reading market data packets directly from the network card, doing a tiny bit of computation and then writing to a userspace TCP stack for order entry. I guess you could be using MemSQL for post-order or trade analysis but then it would pr…
Additionally 128 GB is just about 20 minutes worth of data.
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#55Earlier quoted context omitted.
MemSQL CEO here. There are a few: - MemSQL is transactional and writes transactions on disk - MemSQL has an excellent implementation of SQL with mature query optimization and query execution. And it get better every release. This is from 6.5 https://www.memsql.com/blog/6.5-performance/ - MemSQL has in-memory and on-disk data storage so you can use MemSQL to store petabytes - MemSQL has columnstores and vectorized que…
Given the availability of open source solutions why would I in 2018 build a critical part of my application on a closed platform? Genuinely not intended as snark, I'm just curious why memsql is so compelling that I would consider it.
I would also like to see a response to this.
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#56Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#57Sort of shameless plug, but if you are looking into such solution you could have a look also at https://redisql.com/ It is a redis module that embed SQLite, bringing on the table a lot of advantages. Extremely fast and with it you can even upgrade your Redis instance to be your only database. There is not a huge company behind it, but I really do my best to support it. I don't believe to have disappointed any of our…
I was just looking into in-memory SQL DBs so this is great. Does it have ODBC support?
No unfortunately it does not support ODBC.
It works on top of Redis so it inheritance its interface. I guess it would be possible to add an ODBC layer but honestly I have to look into it...
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#58Earlier quoted context omitted.
No real HFT system is doing a * database transaction * in the critical trading path. HFT systems are not built like web applications. They are typically built as a tight event loop, reading market data packets directly from the network card, doing a tiny bit of computation and then writing to a userspace TCP stack for order entry. I guess you could be using MemSQL for post-order or trade analysis but then it would pr…
Additionally 128 GB is just about 20 minutes worth of data.
HFT firms have plenty of cash to buy the unlimited version.
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#59Sort of shameless plug, but if you are looking into such solution you could have a look also at https://redisql.com/ It is a redis module that embed SQLite, bringing on the table a lot of advantages. Extremely fast and with it you can even upgrade your Redis instance to be your only database. There is not a huge company behind it, but I really do my best to support it. I don't believe to have disappointed any of our…
This is awesome! Exactly what I was looking for. Basically a SQLite server-node. Replace my homebrew Go thing.
I am very glad you find it interesting!
For any issues don't hesitate to contact me directly or through GitHub.
Also if you need it for some open source/(do the world a better place) kind of project I am more than happy to provide the PRO version free of any charges or obligations.
Of course it apply to anybody reading!
Re: MemSQL is now free to use for databases with up to 128GB of RAM usage
#60Why cannot I use free open source MySQL heap or memory storage engine? It also provides clustering/replication
[deleted]