Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cache is the new RAM

blog.memsql.com

Post reply on HN
Print view View original

21–30 of 97 posts

Re: Cache is the new RAM

#21

Post by jobu »

hcarvalhoalves wrote: post #17

> It’s been 65 years since the invention of the integrated circuit, but we still have billions of these guys around, whirring and clicking and breaking. It’s only now that we are on the cusp of the switch to fully solid-state computing. Am I missing something, or should it read "hard disk" rather than "integrated circuit" here?

He's referring to the picture of the hard drive above that line when he says "these guys". It took me a couple reads through that sentence to grasp the meaning = "Why are we still using spinning metal contraptions to store data 65 years after the invention of the integrated circuit?"

Re: Cache is the new RAM

#22

Post by patrickmay »

temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

GigaSpaces XAP (http://www.gigaspaces.com) gives you a big fraction of that. You can create an in-memory distributed grid that acts as both an object store and processing platform, that is both resilient (self-healing) and scalable. The CTO of Elasticsearch used to work for GigaSpaces.

Disclaimer: I work for GigaSpaces.

Re: Cache is the new RAM

#23

Post by temuze »

dsp1234 wrote: post #19
temuze wrote: post #14

Earlier quoted context omitted.

Well, so long as I'm spitballing my dreamDB's "continuous views", then why not have a concept of "foreign documents"? Let's define "foreign documents" as a type of field that is an exact copy of a document in a separate table. This field gets updated when the original document is updated. This could apply to denormalization really well! For example: let's say you have a table called "items", a table called "customers…

Something like this is available in postgres, mssql, etc and is called a "materialized view" [0]. In the postgres case, you have to update the materialized view manually, but in the mssql case, the updates are done automatically when the origin data is updated. [0] - http://en.wikipedia.org/wiki/Materialized_view

Interesting. I knew about PostgreSQL's mviews, but not MSSQL.

Re: Cache is the new RAM

#24

Post by jsmeaton »

temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

(I'm coming from the perspective of having to use Oracle at work - and I'm happy to be corrected if my experiences do not match your wishlist)

> Easy sharding, a la Elasticsearch

Oracle RAC lets you add nodes to the cluster. Helps with CPU - doesn't really help with disk contention

> Fucking SQL. If I want a new feature, then dammit, build on top of SQL the way PostgreSQL has.

There is no good story here for Oracle. Postgres is amazing (and has the potential for more amazing) here. I'm surprised the legions of NoSQL developers aren't contributing to psql.

> For example, I want to make a materialized view that's the result of a query, but that gets updated as new rows get inserted or as the rows it uses gets updated. Let's call it a continuous view or something. Eventual consistency is fine.

This is how (certain) [1] Oracle Materialised Views already work (.. with some effort..) unless I've misunderstood you.

But you have to:

1. Define "logs" on all the included tables (observables)

2. Only define certain types of queries, otherwise they won't dynamically update

3. If not 2, then schedule the updates on a timer/job (eventually consistent)

4. Write your joins with the old syntax (WHERE A.fieldd = (+) B.field)

> I want to be able to choose if a table/db is always in memory or not. I don't care about individual rows - that sounds like someone else's problem

Oracle's In memory feature[0] supposedly offers this though I haven't had a chance to get at it yet.

> While I'm at it, I want a pony, too.

If you've got the money to pay for all the Oracle gear above, then you have more than enough money for a few Ponies ;)

[0] http://www.oracle.com/us/corporate/features/database-in-memo...

[1] https://docs.oracle.com/cd/B28359_01/server.111/b28286/state...

Re: Cache is the new RAM

#25

Post by wyager »

temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

>Fucking SQL. I don't want to learn your stupid DSL... If I want a new feature, then dammit, build on top of SQL

What I hear is "I don't know anything better than the incredibly cut-rate solution I've been taught how to abuse, so I'm opposed to any improvements."

SQL is a human interface language. I absolutely don't understand the desire that people have to use it for IPC. The tremendous set of SQL injection vulnerabilities would never have existed in the first place if people used proper IPC systems for IPC.

Re: Cache is the new RAM

#26

Post by zzleeper »

temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

Quick question about the pony. I mean, about Cstore. What do you think is the best option for doing that nowadays? (By that I mean columnar stores that are hopefully optimized with the usual tricks like inlined compression. I know Vertica works but if are there cheaper (free?) alternatives?

Re: Cache is the new RAM

#27

Post by temuze »

jsmeaton wrote: post #24
temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

(I'm coming from the perspective of having to use Oracle at work - and I'm happy to be corrected if my experiences do not match your wishlist) > Easy sharding, a la Elasticsearch Oracle RAC lets you add nodes to the cluster. Helps with CPU - doesn't really help with disk contention > Fucking SQL. If I want a new feature, then dammit, build on top of SQL the way PostgreSQL has. There is no good story here for Oracle.…

I come from a PSQL background where materialized views are manually refreshed :)

After skimming Oracle's docs, I'd settle for something as simple as:

CREATE MATERIALIZED VIEW my_mview

REFRESH FAST ON COMMIT

THROTTLE 10s

AS ( ... )

Does Oracle use deltas to refresh these mviews? It'd super cool if it did! For example, if the mview is a sum of a bunch of rows, then it could use the logged changes on those rows and its existing results to make the new mview.

Re: Cache is the new RAM

#28

Post by temuze »

zzleeper wrote: post #26
temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

Quick question about the pony. I mean, about Cstore. What do you think is the best option for doing that nowadays? (By that I mean columnar stores that are hopefully optimized with the usual tricks like inlined compression. I know Vertica works but if are there cheaper (free?) alternatives?

Vertica is great, but expensive. I think it's free until 1 TB or something?

Postgres + cstore_fdw looks promising.

Re: Cache is the new RAM

#29

Post by zzleeper »

temuze wrote: post #28
zzleeper wrote: post #26

Earlier quoted context omitted.

Quick question about the pony. I mean, about Cstore. What do you think is the best option for doing that nowadays? (By that I mean columnar stores that are hopefully optimized with the usual tricks like inlined compression. I know Vertica works but if are there cheaper (free?) alternatives?

Vertica is great, but expensive. I think it's free until 1 TB or something? Postgres + cstore_fdw looks promising.

Great, I'll check it out! S

Re: Cache is the new RAM

#30

Post by icelancer »

temuze wrote: post #7

The database I want still doesn't exist. Here's what I want: - Easy sharding, a la Elasticsearch. I want virtual shards that can be moved node to node and an easy to understand primary/replica shard system for write/reads. I want my DB nodes to find each other with an easy discovery system with plugins for AWS/Azure/Digital Ocean etc. - Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQ…

I have a LOT fewer requests than you, but this one is just so irritating:

>- Fucking SQL. I don't want to learn your stupid DSL. I want to give coworkers a SQL client a say "go! You already know how to use this!". If I want a new feature, then dammit, build on top of SQL the way PostgreSQL has. Odds are, regardless if its some JSON API or SQL, my language will have a client for it that will be superior than writing raw queries anyway.

It pisses me off to no end these developers have to out-think SQL and re-invent the whole damn new wheel for "efficiency's" sake. It seems that no one takes into account friction costs and instead just waggles their dick around saying "Look how smart this new language is or how I co-opted [erlang,perl,JS,etc] into being the query language!" Just stop it. Some people making less than $150k/year are going to have to use this and they are good at DB analysis but not bullshit esoteric language writing.

Good lord this makes me so mad. I feel your pain.

Post reply on HN