Live data from Hacker News

SQL: One of the most valuable skills

craigkerstiens.com

81–90 of 390 posts

Re: SQL: One of the most valuable skills

#81

SQL is a mind bender for me. I do a lot of work on Data, use python and pandas to do a lot of data magic, but the problem with me is my mind is too procedural in thinking. - Step 1 - Step 2 - Loop through results in Step 2 - Curate and finish output. I try very hard to transform the above steps into an SQL statement spanning multiple tables, but always fail and I usually fallback to python for manually extracting and…

I took a course in prolog, really seemed to help get my mind in the right place.

Basically, instead of thinking in terms of how to get to the result, think instead of the result and figure out how to get there. For example, let's say I need to get a mapping of students to their classes. One way would be to get the students, then loop through that to get classes for each one. Another way to do it would be to ask the database to combine those sets of data for us instead and return the result (join students and class tables on the student ID).

Basically, find the data you want from each table, tell the database about the relationship between them (join on ), and set up your constraints. I guess you could think about the query as a function, and you're just writing the unit tests for it (when I call X with parameters Y, I expect Z).

Re: SQL: One of the most valuable skills

#82

SQL is a mind bender for me. I do a lot of work on Data, use python and pandas to do a lot of data magic, but the problem with me is my mind is too procedural in thinking. - Step 1 - Step 2 - Loop through results in Step 2 - Curate and finish output. I try very hard to transform the above steps into an SQL statement spanning multiple tables, but always fail and I usually fallback to python for manually extracting and…

I think what your are doing is fine. SQL is strongest at answering questions not processing data. I think metabase has the right approach: What question do you want to ask your data? If you want to process and transform your data, I think your tools you are using are great for that.

Re: SQL: One of the most valuable skills

#84
post #78

SQL is a mind bender for me. I do a lot of work on Data, use python and pandas to do a lot of data magic, but the problem with me is my mind is too procedural in thinking. - Step 1 - Step 2 - Loop through results in Step 2 - Curate and finish output. I try very hard to transform the above steps into an SQL statement spanning multiple tables, but always fail and I usually fallback to python for manually extracting and…

I like the Ben Forta SQL in Minutes book. Just the basics, very clear and easy to understand.

Seconded.

Re: SQL: One of the most valuable skills

#85

SQL is a mind bender for me. I do a lot of work on Data, use python and pandas to do a lot of data magic, but the problem with me is my mind is too procedural in thinking. - Step 1 - Step 2 - Loop through results in Step 2 - Curate and finish output. I try very hard to transform the above steps into an SQL statement spanning multiple tables, but always fail and I usually fallback to python for manually extracting and…

Try some functional programming. It will liberate your brain from thinking operationally. You will soon no longer think in terms of steps, loops or anything like that. You will get used to thinking about data transformations, which is what SQL is good at.

Re: SQL: One of the most valuable skills

#86
post #31

I spent a year in a role where 50% of my duties was writing sql reports. These reports where usually between 500 and 1000 lines of sql a pop. Sometimes the runtime of the report was measured in hours, so learning efficient sql was important. The company had a lot of people that had been writing sql for awhile, and there were lots of cool code snippets floating around. I learned a lot in that year. I've moved to writi…

From a performance perspective, most ORMs are trash if used naively.

Re: SQL: One of the most valuable skills

#87
post #41

Reading the thread so far, I see SQL and bash suggested as being valuable skills. Can you suggest more skills and tools like these that are timeless and valuable?

A few months ago vogre@ suggested SQL, bash, and HTML, and I concurred.

https://news.ycombinator.com/item?id=18001267

HTML is the quickest way to share something permanently. And its client is the most widely available (the browser).

Many people seem to use something like Dropbox or Google Docs now, but my opinion is that HTML still has many benefits, once you know it. I rsync my HTML to a shared hosting account.

In addition to bash, I'd suggest learning more about Unix in general.

I saw this page a few weeks ago. I haven't gone through this particular course, and I don't necessarily endorse it, but I'll just say that the subject matter could have been largely the same 20 years ago. Unix has proven to be an effective way of working and collaborating for a long time.

https://news.ycombinator.com/item?id=19078281

EDIT: I also suggest knowing some basic analysis outside of SQL, using data frames:

https://news.ycombinator.com/item?id=19150543

The need for that skill isn't going away anytime soon. You could argue that a large portion of jobs are munging Excel spreadsheets, and data frames are more of less the programmer's way of accomplishing the same tasks.

You could be a snob and "delegate" those tasks to others, but IME if you really want to get something done that other people can't, you have to roll up your sleeves a bit. (Just like this article suggests, e.g. SQL is useful for both programmers and product managers.)

Re: SQL: One of the most valuable skills

#88
post #69
post #48

Earlier quoted context omitted.

Communication. Particularly the ability to cross the boundary between technical talk and non-technical talk. People like other people who are easy to talk with. Also you will be more likely to build what is actually needed, rather than what someone thought they needed.

One of the most useful books I've ever read was "Thanks for the Feedback" [1] I've read a lot of the more standard books like "How to Win Friends and Influence People", etc. I feel like this book is similarly valuable, but a lot less well known. [1] https://www.goodreads.com/book/show/18114120-thanks-for-the-...

Thanks, will have a look.

Crucial Conversations was recommended to me on HN, it's also super-useful - on how to talk about difficult/important things, whether in relationships or organizations or everyday life, where most people would be afraid and say nothing. How to speak difficult truths and ensure everyone feels good about it. Not many people would be advanced enough to not get a lot from that book, I think.

Re: SQL: One of the most valuable skills

#89
You can get to the opposite conclusion with some perspective. Just two decades ago SQL was pretty much a mandatory skill for new programmers. Everyone was learning it, it was in every programming job requirement in any way related to storing some data, etc. Basically databases and SQL were synonymous. But today a Citus guy writes a post attempting to convince people to learn some SQL (Citus is an SQL database company). And last time I used SQL was when Chromium dropped my cookies after an update and I was too lazy to relogin to all the websites (it stores them in an SQLite database). I guess things are not looking so promising for SQL.

Maybe we'll get to a better query language eventually. Distributed systems have some fundamental incompatibilities with SQL and theoretically could standardize on a different language, one that is not just about the data anymore, but that also lets users express and choose various performance and consistency trade-offs. I mean a query shouldn't, for example, invoke a transaction and consensus algorithm when all you need is to increment a view counter or give a one star rating or insert a log entry, those things can be propagated eventually without destroying latency and throughput and without complicating anything with transactional semantics.

Re: SQL: One of the most valuable skills

#90
There's not much going on in this post, so I assume the overall high rating is due to SQL being so damned fantastic.

I commented to a friend the other day that my favorite thing to do on any project is project-wide query optimization. It's like pulling weeds or power washing.

Arguably _every_ application is simply a means of manipulating data. All the other parts are important, of course, but the data comes first. And SQL is a hell of a UI for data. I've tried quite a few other query languages (or structures), even inventing my own once or twice, but none come close.

Post reply on HN