91–100 of 160 posts
> The “bigger” your data, the more false positives will turn up in it, when you’re looking for correlations I think they are talking about the Sharpshooter Fallacy http://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy
From the Berkeley paper on Facebook: Nonetheless, large jobs are important too. Over 80% of the IO and over 90% of cluster cycles are consumed by less than 10% of the largest jobs (7% of the largest jobs in the Facebook cluster). These large jobs, in the clusters we considered, are typically revenue-generating critical production jobs feeding front-end applications. So MR job characteristics might follow a power law…
And If the proposal to analyse a 1/4 million terms for kelly search had gone live I woudl have been creating over a LOC (Libray of congress) for each run.
Most business leaders are not rational, and we should stop pretending they are.
Earlier quoted context omitted.
I don't have a problem with relational databases. Or rather, I don't have a problem with relational data modelling. I do have a problem with Oracle. Even the Oracle experts at my former job could barely get Oracle to do something sensible, and running it on my own computer was basically a death sentence for getting anything done. I have a problem with MySQL, from a sysadmin perspective. When I had it installed, MySQL…
This is all pretty sane, except the schema-less part. I just don't understand why people get all hung up over schemas. Sure, migrations are a minor inconvenience, but if you just add fields in an ad-hoc fashion over time the data becomes messy and it's hard to determine any invariants about a large dataset. Sure this is avoided through careful code organization, but then aren't you just re-inventing schemas out-of-ba…
Oh but you have to know the schema right? Yes, some other part of the application knows the schema, but this part doesn't have authority over the DB. Also, the schema may be data as well.
NoSQL reduces the work needed for that.
As some one who is currently dealing with these sort of things I can tell this article hits the nail on its head. Most, heck something like 99.99% of all so-called big data I've dealt is something I wouldn't even classify as small data. I've seen data feeds in KB's sent over to be handled in as big data. It happens all the time. A simple data problem sufficient enough to be easily solved on something like a small db…
Just need a good SQL reference? 'The Art of SQL' or 'SQL and Relational Theory'
Are you not entertained? Fine, choke on this : http://en.wikipedia.org/wiki/The_Third_Manifesto
Earlier quoted context omitted.
In some ways, it kind of is a sham. I think it is perpetuated by the blog/youtube style programming knowledge transference paradigm. Those mediums are fine but there seems to be a rallying cry against actually learning anything about computing in anything other than bite size pieces and thus get a lot of fad driven movements and an over population of redundant frameworks and libraries.
Yes, and then because companies start using NoSQL or whatever for problems they could have done fine in mysql they start asking for NoSQL experts when they are hiring. This makes devs think that they need NoSQL experience and therefor they will find ways to shoehorn NoSQL into whatever problems they are currently solving.
As some one who is currently dealing with these sort of things I can tell this article hits the nail on its head. Most, heck something like 99.99% of all so-called big data I've dealt is something I wouldn't even classify as small data. I've seen data feeds in KB's sent over to be handled in as big data. It happens all the time. A simple data problem sufficient enough to be easily solved on something like a small db…
Hanging off this great comment, please anyone wanting to learn about relational databases I strongly suggest reading Database in Depth by C J Date. It's around 200 pages, you will learn a lot about the relational model, database/query/index design in a short amount of time, with a little bit of logic thrown in there too. Just need a good SQL reference? 'The Art of SQL' or 'SQL and Relational Theory' Are you not enter…
Earlier quoted context omitted.
I don't have a problem with relational databases. Or rather, I don't have a problem with relational data modelling. I do have a problem with Oracle. Even the Oracle experts at my former job could barely get Oracle to do something sensible, and running it on my own computer was basically a death sentence for getting anything done. I have a problem with MySQL, from a sysadmin perspective. When I had it installed, MySQL…
This is all pretty sane, except the schema-less part. I just don't understand why people get all hung up over schemas. Sure, migrations are a minor inconvenience, but if you just add fields in an ad-hoc fashion over time the data becomes messy and it's hard to determine any invariants about a large dataset. Sure this is avoided through careful code organization, but then aren't you just re-inventing schemas out-of-ba…
In systems large enough to be running multiple versions of an app at the same time, talking to the same database, you have to do exactly this--but you have to shoehorn it into relations (doing things like having a field where 99.999% of entries in a column are null, and gradually get computed as the row is accessed by the new version of the app.) NoSQL lets you just say what you mean--that in some versions of the app, the schema is X, in some versions it's Y, and the database isn't the arbiter of the schema.
Earlier quoted context omitted.
Hanging off this great comment, please anyone wanting to learn about relational databases I strongly suggest reading Database in Depth by C J Date. It's around 200 pages, you will learn a lot about the relational model, database/query/index design in a short amount of time, with a little bit of logic thrown in there too. Just need a good SQL reference? 'The Art of SQL' or 'SQL and Relational Theory' Are you not enter…
Ha, I knew that looked familiar, Hugh Darwin is a lecturer where I was at university so half of the coursework for a databases module was in Tutorial D.
As some one who is currently dealing with these sort of things I can tell this article hits the nail on its head. Most, heck something like 99.99% of all so-called big data I've dealt is something I wouldn't even classify as small data. I've seen data feeds in KB's sent over to be handled in as big data. It happens all the time. A simple data problem sufficient enough to be easily solved on something like a small db…
Hanging off this great comment, please anyone wanting to learn about relational databases I strongly suggest reading Database in Depth by C J Date. It's around 200 pages, you will learn a lot about the relational model, database/query/index design in a short amount of time, with a little bit of logic thrown in there too. Just need a good SQL reference? 'The Art of SQL' or 'SQL and Relational Theory' Are you not enter…