Ask HN: Going from Redis to MySQL. Good idea? • Hacker News Board

Re: Ask HN: Going from Redis to MySQL. Good idea?

#41

Post by ColinCera » Mon, Oct 20, 2014, 4:22 AM UTC

If you don't want to worry about database/server administration and scaling issues, I'd recommend looking at Cloudant. It's quite inexpensive, takes virtually no administration effort, scales automatically, and is fast and flexible.

You could continue to use Redis as a write cache, although a message queue or something like Kafka might be better for that purpose, and of course as a read cache, while using Cloudant or some other "real" database for permanent storage.

(On a side note, you described your application as "write-heavy" but you also said you get "about 3 Million individual customer orders on monthly basis" — which works out to just slightly more than one record to insert per second, on average, which doesn't seem "write-heavy" to me. I understand that you get orders submitted in batches, rather than evenly distributed, but it still doesn't seem especially write heavy. At any rate, a database like Cloudant can take as many writes as you want to throw at it.)

(Disclaimer: I don't work for Cloudant; I am a satisfied customer; I don't use Cloudant for everything.)

Re: Ask HN: Going from Redis to MySQL. Good idea?

#42

Post by usenewerversion » Mon, Oct 20, 2014, 5:14 AM UTC

imanaccount247 wrote: post #37

misframer wrote: post #22

Earlier quoted context omitted.

Why not MySQL?

Just the small list of flaws that we hit trying to use mysql (there's dozens more, and we're not including all the "mysql silently inserts invalid data" gotchas): no check constraints views with aggregates are too slow to be used no expression indexes triggers don't fire on cascaded actions no window functions can't set default values to be the result of a function no transactional DDL doesn't have multiple databases…

Agree with most of these issues but your last claim is only true for older versions of MySQL. Newer versions do support microsecond resolution[1].

1- http://hackmysql.com/slowlog

Re: Ask HN: Going from Redis to MySQL. Good idea?

#43

Post by sganguly » Mon, Oct 20, 2014, 5:34 PM UTC

Hi Gary4gar, Read the points that you stated in your question. If you focus on growth (clearly your wins are going to bring more data updates as your increase the customer base) you should look into a solution that does not tie you down with data model and also does not expect you to worry about sharding in the future. Having done sharding and when one keeps growing the data volume and having to do shards it becomes painful and hurts the development flow. Been there many time and done it quite a bit myself. (Am a technical guys myself, http://www.linkedin.com/in/sukantag)

Look for a NoSQL solution which will help in the flexi-schema model. Helps you add different types of data models relatively easily. Check out Aerospike (http://www.aerospike.com), the product scales like scaling problems have disappeared from this planet. All of the characteristics that you would need based on what you have stated is in the product. Best of all it is free for startups. Just give it shot. Open Source server and client libraries in multiple programming languages. (Am a big "C" and Python guys so like them client libs a lot). Lots of startups and many large software companies use them.

If you like it and works for your application tell others, if not tell us what did not work.

I ain't sellin and you ain't buyin;

Ciao

Re: Ask HN: Going from Redis to MySQL. Good idea?

#44

Post by itamarhaber » Mon, Oct 20, 2014, 9:31 PM UTC

justinsb wrote: post #29

resca79 wrote: post #28

Earlier quoted context omitted.

>With redis you'll have to fetch every row over the network; >with a SQL database you'll typically only retrieve the final >summary data. Redis supports filter features native, so it basically works as every remote service.

Do you mean HSCAN with MATCH? Thanks for reminding me about that. I think it's so limited that the point still stands though.

You can actually push all the processing to Redis with Lua without having to fetch a single "row" over the network... I'm not recommending this as a rule of thumb - on the contrary, you could easily tie down your (any type of) DB with application logic - but in some cases it makes sense.

Re: Ask HN: Going from Redis to MySQL. Good idea?

#45

Post by itamarhaber » Mon, Oct 20, 2014, 9:42 PM UTC

Regarding 3 - I don't know if you (or anyone for that matter) are wrong or right about this (or anything), but there are ways to use Redis w/o upgrading to bigger machines. Redis cluster, as mentioned in @bendmorris comment, is one way to go about it.

Alternatively, if you want to continue using Redis - with the current or a slimmer version of the data model - without having to worry about machine upgrades or clusters, then I recommend that you look at http://redislabs.com/redis-cloud (disclaimer - I work at Redis Labs). Our managed service is built using production-proven Redis clusters that allow your database to scale well beyond the limits of any single node, without you having do anything or change a line of code.

Re: Ask HN: Going from Redis to MySQL. Good idea?

#46

Post by taf2 » Tue, Oct 21, 2014, 5:09 PM UTC

imanaccount247 wrote: post #37

misframer wrote: post #22

Earlier quoted context omitted.

Why not MySQL?

Just the small list of flaws that we hit trying to use mysql (there's dozens more, and we're not including all the "mysql silently inserts invalid data" gotchas): no check constraints views with aggregates are too slow to be used no expression indexes triggers don't fire on cascaded actions no window functions can't set default values to be the result of a function no transactional DDL doesn't have multiple databases…

Mysql does have a much better durability solution -galera cluster IMO... Does Postgres have something similar?

Re: Ask HN: Going from Redis to MySQL. Good idea?

#47

Post by taf2 » Tue, Oct 21, 2014, 5:12 PM UTC

imanaccount247 wrote: post #36

Killswitch wrote: post #23

Earlier quoted context omitted.

MySQL is a very good database. Especially if you use MariaDB.

No, it isn't. No amount of saying that will make it true. It is a database that is missing vital functionality, and which considers data integrity not to be important.

How is this correct, mysql has synchronous replication. does Postgres have something equivalent to galera?

Re: Ask HN: Going from Redis to MySQL. Good idea?

#48

Post by taf2 » Tue, Oct 21, 2014, 5:15 PM UTC

imanaccount247 wrote: post #38

resca79 wrote: post #19

Earlier quoted context omitted.

Shocked for what? Mysql is a good choice for many cases. Please you should explain why mysql is better than postgres?

>Mysql is a good choice for many cases Name one. Mysql is horribly broken.

Mysql is fast and easy to scale and make durable, what features of database do you really use that is not met by, fast, durable and scalable... Sounds more like perhaps some sort of belief rather than data?

Re: Ask HN: Going from Redis to MySQL. Good idea?

#49

Post by taf2 » Tue, Oct 21, 2014, 5:16 PM UTC

revelation wrote: post #10

I don't have an opinion re redis to MySQL, but it seems like right now the cheapest options is to simply buy a server with 256+ GB of RAM. These are pretty common by now.

I agree and perhaps see if you can shard between multiple servers.

Re: Ask HN: Going from Redis to MySQL. Good idea?

#50

Post by imanaccount247 » Wed, Oct 22, 2014, 5:07 AM UTC

taf2 wrote: post #48

imanaccount247 wrote: post #38

Earlier quoted context omitted.

>Mysql is a good choice for many cases Name one. Mysql is horribly broken.

Mysql is fast and easy to scale and make durable, what features of database do you really use that is not met by, fast, durable and scalable... Sounds more like perhaps some sort of belief rather than data?

That does not answer the question. Postgresql is faster than mysql, easier to scale than mysql, and is already durable without having to do anything to it. Saying "it is almost as good as postgresql" does not mean it has a use case.