Live data from Hacker News

SQL: One of the most valuable skills

craigkerstiens.com

381–390 of 390 posts

Re: SQL: One of the most valuable skills

#381
post #267

Earlier quoted context omitted.

Depends a lot on the actual access patterns of your data. Many recent web & mobile apps have a lot of screens where you just want to grab one blob of heterogenous data and format it with the UI toolkit of choice. Or if they do display multiple results, it's O(10) rather than O(1000) or O(1M). Chasing pointers is fine for use-cases like this, because you do it once and you have all the information you're looking for.…

Depends a lot on the actual access patterns of your data. Yup. Thing is: with RDBMSs you are in control of both the storage patterns and the access patterns of your data. That's where a large part of the performance benefit comes from. > 50% of features could get by with read-only datasets that supported only key/value lookup Did you implement a storage pattern that was ordered by key (or hash(key) if you used hashin…

"Did you implement a storage pattern that was ordered by key (or hash(key) if you used hashing)?"

Well, I didn't implement it, Jeff Dean did, but yes, that's what we used for a lot of features.

https://www.quora.com/What-is-an-SSTable-in-Googles-internal...

https://www.igvita.com/2012/02/06/sstable-and-log-structured...

Re: SQL: One of the most valuable skills

#382
post #37
post #22

The more queries I write on Elasticsearch, the more I value SQL.

Me too. I spent years complaining about databases and all the structure and rules and inflexibility. It was initially great to go to NoSQL backends where suddenly you're free. Now I really miss them :(

Well, at least I can do simple query with SQL and Elasticsearch and on Hive.

Re: SQL: One of the most valuable skills

#383

Earlier quoted context omitted.

I understood when OP said the "concept of SQL" that they referred to a pretty broad idea of querying tabular data in rows and columns with some structured language.

I find it hard to accept that ppl are talking about 'querying tabular data in rows and columns with some structured language' when they talk about sql. They are surely talking about the specific syntax. https://en.wikipedia.org/wiki/SQL

Great, maybe you can ask the parent poster, but the following are now factual statements:

1. You believe in your interpretation. This is true.

2. The parent poster has their interpretation. This is true.

3. The statement "They are surely talking about the specific syntax" is true if and only if their interpretation is the same as your interpretation.

4. This is a semantic argument now.

Re: SQL: One of the most valuable skills

#384
post #314

Earlier quoted context omitted.

All I heard from NoSQL proponents was that "relational databases don't scale". These same people were usually dealing with data sets that would fit easily onto a single server.

No, I'm pretty sure you only heard it from your fellow RDBMS friends. I have never argued for scale myself, but plenty of times tried to debunk this exact stereotype.

No.

Re: SQL: One of the most valuable skills

#385

Earlier quoted context omitted.

People behind NoSQL movement understand SQL better, than any people glorifying SQL here. In fact, I think this is a testament of how poorly most people actually understand SQL, they can't even see basic on the surface problems with it.

I think there are people who went with NoSQL because it fit their specific needs better than a regular RDBMS, and people who went NoSQL because they found it easier to work with than a RDBMS (at first). The latter group often didn't want to learn SQL and went with NoSQL's as a shortcut, not because they had an intimate understanding of the tradeoffs between the approaches and decided NoSQL was the better option.

There was the third group of resume driven developers who bought into the MongoDB hype around 6 or 7 years ago, which probably overlapped with the second group a bit.

Re: SQL: One of the most valuable skills

#386

SQL is one the most amazing concepts I've ever experienced. It's nearly 5 decades old and there is no sign of a replacement. We've created countless other technologies to store and process data, and we always seem to try to re-create SQL in those technologies (e.g. Hive, Presto, KSQL, etc). I run a early stage company that builds analytics infrastructure for companies. We are betting very heavily on SQL, and Craigs p…

"we always seem to try to re-create SQL in those languages (e.g. Hive, Presto, KSQL, etc)." This is largely because of the number of non-programmers who know SQL. Add an SQL layer on top of your non-SQL database and you instantly open up a wide variety of reporting & analytics functionality to PMs, data scientists, business analysts, finance people, librarians (seriously! I have a couple librarian-as-in-dead-trees fr…

I agree with you and the parent comment. I run a data science department and we rely on SQL. We even embed SQL queries into R scripts to pull the latest data. It isn't worth working with ORM for an analytics project when a few dozen lines of SQL can get you everything you need.

Re: SQL: One of the most valuable skills

#387

Earlier quoted context omitted.

SQL is wildly powerful and important, but it's also got clear deficiencies for data traversal and manipulation. I see it on a spectrum between declerative and imperative data structures, and where I think most people go wrong with it is trying to create a monolithic solution to a broad spectrum of problems. I think you need a graduated approach where each data layer is simplifying and satisfying the next, so you're u…

I can't agree more with this. As with anything we use in our applications, understanding and leveraging the strengths of our different tools is vitally important. If, for example, I needed a cartesian product of two datasets, SQL is the first thing that comes to mind due to how simple it is to write and the speed with which it will be processed in the database. On the flip side, I would never want to intermix fronten…

I’ve seen queries returning html and JavaScript

Re: SQL: One of the most valuable skills

#388

Earlier quoted context omitted.

I find it hard to accept that ppl are talking about 'querying tabular data in rows and columns with some structured language' when they talk about sql. They are surely talking about the specific syntax. https://en.wikipedia.org/wiki/SQL

Great, maybe you can ask the parent poster, but the following are now factual statements: 1. You believe in your interpretation. This is true. 2. The parent poster has their interpretation. This is true. 3. The statement "They are surely talking about the specific syntax" is true if and only if their interpretation is the same as your interpretation. 4. This is a semantic argument now.

> 4. This is a semantic argument now.

What about wikipedia entry that lists a specific syntax?

Re: SQL: One of the most valuable skills

#389
post #387

Earlier quoted context omitted.

I can't agree more with this. As with anything we use in our applications, understanding and leveraging the strengths of our different tools is vitally important. If, for example, I needed a cartesian product of two datasets, SQL is the first thing that comes to mind due to how simple it is to write and the speed with which it will be processed in the database. On the flip side, I would never want to intermix fronten…

I’ve seen queries returning html and JavaScript

It completely blows my mind when I see stuff like this. We're not all perfect programmers, I'm sure, but still. An alarm should go off in your head when the thought occurs to put front-end code into a query. The only exception I've encountered is if I want to inject some HTML formatting into a string for emails sent from SQL Server, but even then it's extremely limited in use, and I still think to myself, there must be a better way..

Re: SQL: One of the most valuable skills

#390
post #387

Earlier quoted context omitted.

I’ve seen queries returning html and JavaScript

It completely blows my mind when I see stuff like this. We're not all perfect programmers, I'm sure, but still. An alarm should go off in your head when the thought occurs to put front-end code into a query. The only exception I've encountered is if I want to inject some HTML formatting into a string for emails sent from SQL Server, but even then it's extremely limited in use, and I still think to myself, there must…

The worst thing is that the guilty programmer probably felt very clever when he wrote the code.
Post reply on HN