NoSQL in SQL
hakkalabs.co
NoSQL in SQL
1–10 of 20 posts
Re: NoSQL in SQL
#2It's always interesting to observe my bad developer practices (such as stuffing JSON in SQL table columns) become flexible "architectural patterns" for building "schema-free, scalable data storage".
Re: NoSQL in SQL
#3It's always interesting to observe my bad developer practices (such as stuffing JSON in SQL table columns) become flexible "architectural patterns" for building "schema-free, scalable data storage".
If you watched the talk, he explains the specific kinds of that should be stuffed into a JSON text column. It makes a difference.
This practice been done by many production sites over the years http://backchannel.org/blog/friendfeed-schemaless-mysql Might not be considered bad practice now.
Re: NoSQL in SQL
#4It's always interesting to observe my bad developer practices (such as stuffing JSON in SQL table columns) become flexible "architectural patterns" for building "schema-free, scalable data storage".
Wait until you hear about "fractal storage" where every value in a table is a blob that contains a SQLite database, which in turn has blobs which contain SQLite databases, until it's turtles all the way down.
Re: NoSQL in SQL
#5It's always interesting to observe my bad developer practices (such as stuffing JSON in SQL table columns) become flexible "architectural patterns" for building "schema-free, scalable data storage".
If you watched the talk, he explains the specific kinds of that should be stuffed into a JSON text column. It makes a difference. This practice been done by many production sites over the years http://backchannel.org/blog/friendfeed-schemaless-mysql Might not be considered bad practice now.
Well I'm saying it tongue in cheek. For a long time I and many others have stuffed JSON in SQL table columns, and I will continue to do so (heck, databases have started supporting JSON as a result).
But every time a developer sees an interesting twist on a piece of technology and goes for it, peers call it a bad practice.
I've been through many cycles like this, and inevitably some time passes, and one day you wake up to see yesterday's bad practices have turned into exciting advancements.
Moral of the story is, ignore the wisdom of the day and go for it, tiger. Stuff that JSON in an SQL table.
Re: NoSQL in SQL
#6I recently just made what I think is a pretty neat pattern for reporting on a project that we need time series data calculations for.
Essentially I already have a giant precalculation service for all the needed calculations on all the data underneath a parent entity. So, I serialize that using ruby marshall and then use lz4 to compress it before storing to db.
Its actually faster and smaller than using json strings in ruby. The whole tree structure for each entity was 200-400k as raw json strings. It took something like 300ms to serialize to json. I was able to do the ruby marshal and the LZ4 HC compression in something like 20-40ms and it drops the size down to more like 15-30k.
JSON is a pretty cool format, but it's a lot slower in ruby than you might realize and it takes up a lot of space.
Re: NoSQL in SQL
#7It's always interesting to observe my bad developer practices (such as stuffing JSON in SQL table columns) become flexible "architectural patterns" for building "schema-free, scalable data storage".
Wait until you hear about "fractal storage" where every value in a table is a blob that contains a SQLite database, which in turn has blobs which contain SQLite databases, until it's turtles all the way down.
My God, it's full of stars. This is a joke, right?
Please?
Re: NoSQL in SQL
#8It's always interesting to observe my bad developer practices (such as stuffing JSON in SQL table columns) become flexible "architectural patterns" for building "schema-free, scalable data storage".
If you watched the talk, he explains the specific kinds of that should be stuffed into a JSON text column. It makes a difference. This practice been done by many production sites over the years http://backchannel.org/blog/friendfeed-schemaless-mysql Might not be considered bad practice now.
We actually use a storage format very similar to the FriendFeed schema in Dari. Dari is a open source Java persistence layer with a full query API that stores all it's data in a JSON blob in one table with a few extra tables for indexing the JSON data.
https://github.com/perfectsense/dari
Here is the SQL schema: https://github.com/perfectsense/dari/blob/master/db/src/main...
We've used this model for almost five years now with great success. It's simplified rolling out "schema changes" since no tables need to be changed. It's also been optimized to a point where it's extremely fast.
Re: NoSQL in SQL
#9Earlier quoted context omitted.
If you watched the talk, he explains the specific kinds of that should be stuffed into a JSON text column. It makes a difference. This practice been done by many production sites over the years http://backchannel.org/blog/friendfeed-schemaless-mysql Might not be considered bad practice now.
Well I'm saying it tongue in cheek. For a long time I and many others have stuffed JSON in SQL table columns, and I will continue to do so (heck, databases have started supporting JSON as a result). But every time a developer sees an interesting twist on a piece of technology and goes for it, peers call it a bad practice. I've been through many cycles like this, and inevitably some time passes, and one day you wake u…
I'm not sure which authority gets to pronounce which code practices are good and which ones are bad (the Vatican?), but one thing that I've observed plainly is that this authority can't make up its mind and contradicts itself with regularity. My thoughts are that the whole project of trying to decide for each possible snippet of code whether it is good or bad is foolish and will never succeed. The fact is that everything in programming is a trade-off. Being able to make decisions that don't lead to disaster comes down to experience and wisdom.
Re: NoSQL in SQL
#10I recently just made what I think is a pretty neat pattern for reporting on a project that we need time series data calculations for. Essentially I already have a giant precalculation service for all the needed calculations on all the data underneath a parent entity. So, I serialize that using ruby marshall and then use lz4 to compress it before storing to db. Its actually faster and smaller than using json strings i…
Interesting. I just deleted two tables from my project instead of spreading the data into two tables I thought storing a "data" json column (I am a PSQL user) would be better (fewer queries to make and more space efficent).
I didn't think about the size before compression.
Meanwhile, this serves well for Python: https://groups.google.com/forum/#!topic/google-appengine/WPf...
I shall go ahead and write some tests to see how much space will be taken up :)