Live data from Hacker News

SQL: One of the most valuable skills

craigkerstiens.com

211–220 of 390 posts

Re: SQL: One of the most valuable skills

#211
Can't agree more. SQL is the one skill that I continue to use across a 15-year span while many others have come and gone - SAS, SPSS, Clementine and name-your-latest-plug-and-play-analytics platform.

It is a skill with the highest bang for buck and is a great way to introduce non-programmers to a basic querying language. It is intuitive, easy-to-read and unbelievably versatile.

Re: SQL: One of the most valuable skills

#212
I consider myself lucky to have begun my career in earnest at a time just before ORMs became the norm in web development. I wouldn’t say I am an expert, but wow has the sql experience developed early on paid dividends over and over throughout my career. I feel like postgresql in particular is one of the most powerful pieces of software I have ever worked with. You really don’t need much more to build a robust backend!

Re: SQL: One of the most valuable skills

#213
post #41

Reading the thread so far, I see SQL and bash suggested as being valuable skills. Can you suggest more skills and tools like these that are timeless and valuable?

Regular expressions

Invaluable for cleaning up raw data before it gets into the database.

Re: SQL: One of the most valuable skills

#214

SQL is one the most amazing concepts I've ever experienced. It's nearly 5 decades old and there is no sign of a replacement. We've created countless other technologies to store and process data, and we always seem to try to re-create SQL in those technologies (e.g. Hive, Presto, KSQL, etc). I run a early stage company that builds analytics infrastructure for companies. We are betting very heavily on SQL, and Craigs p…

"we always seem to try to re-create SQL in those languages (e.g. Hive, Presto, KSQL, etc)." This is largely because of the number of non-programmers who know SQL. Add an SQL layer on top of your non-SQL database and you instantly open up a wide variety of reporting & analytics functionality to PMs, data scientists, business analysts, finance people, librarians (seriously! I have a couple librarian-as-in-dead-trees fr…

SQL certainly does not suck as a language, unfortunately this is a very common perception, almost a (false) consensus within the developer community. I changed careers from Finance - I was a ACA (akin to CPA) , wizard at Excel VBA etc - to software development, after 10 years learning, I’m now finally also proficient with SQL. I have also learnt other langauges, but SQL is by far my favourite, Yes it has some mistakes in it’s design - the biggest being SELECT before FROM, Yes some key words, esp windows functions, have a silly length - this creates an impression of verbosity but SQL is actually the opposite, it is far more expressive than any Object Oriented language I can think of. 1 line of code, say a window function with a filter clause in Postgres is the equivalent of pages of Java code. Lateral Joins enable chaining, Views CTEs and Functions provide simple safely scoped composability. Postgres has incredible aggregation and analysis ability, built in out of the box. It’s String manipulation capabilities are wondrous, a 1 line StringAgg function can achieve the same as pages of VBA code - you may scoff but VBA with it’s ancient horrible editor is still necessary, if you are in Excel, is Javascript that much less verbose though? Pure SQL Postgres Functions are amazing, you can chain them, you can easily make them pure, they can include fast recursion and conditional logic. I never delete data, just append, a bitemporal immutable functional db is so powerful, enabling time travel and rock solid data integrity. Mat Views or Indexes or summary tables are easily created and often automatically. They provide an efficient reliable cache layer, which can address most performance issues that may arise

Re: SQL: One of the most valuable skills

#215
SQL is for data systems what IP protocol is for networks: it is the neck of the hourglass. You can build plenty of various things on it (the top of the hourglass: applications, reporting frameworks and so on) using various underlying technologies (the bottom: storage engines etc.) but you can't remove the neck without breaking the hourglass.

This is why SQL language and IP protocol are two most valuable things in computer world.

Re: SQL: One of the most valuable skills

#216

Earlier quoted context omitted.

I'd disagree -- I (and some other SQL/postgres zealots), would say that allowing users to run SQL on your own server is only a bad thing when you are running a database with insufficient configuration and/or security features. We just might happen to live in a world where all the RDBMSes have that same failing, but I'd argue Postgres doesn't fail as bad in the are with things like row level security[0]. [0]: https://…

It's near trivial to exhaust resources when you have SQL access. That's not information disclosure, but it's darn painful if the set of users includes a sizable enough group of bad faith actors.

Some people will even do it in good faith. They’ll tank performance and for some reason it will never cross their mind that they could be causing the issue. Happens with any shared system without rate limiting.

Re: SQL: One of the most valuable skills

#217

Earlier quoted context omitted.

What don’t you like about GraphQL? I haven’t used it enough to form a strong opinion but it seems ok so far.

I would 1000% not take my opinion as gospel on GraphQL -- I know how it works, but I have not written an implementation myself, and have not even spun up Apollo. With that heavy grain of salt, here's how I feel about GraphQL: - It doesn't feel very interoperable nor flexible (it's a different "ecosystem") with regards to other software -- meaning that if it's much less "integrate with GraphQL" and more "build on top…

Write a trivial app with it. I thought "oh yeah, looks ok, but not a huge benefit over REST", now I've started playing with it, I'm saying REST is the new SOAP.

Re: SQL: One of the most valuable skills

#218

Earlier quoted context omitted.

I don't disagree. I just don't think we've figured out the next step. Other paradigms (MapReduce and graph databases are perfect examples) have introduced very interesting and clever ways to querying data, but nothing has replaced SQL. If anything, it seems like it will be additive on top of what SQL has already done. I know a lot about data, but I can't solve that problem. To anyone out there, much smarter than me:…

I think graph databases with time will eventually be eating some of the RDBMs market (used for the same purpose), but they haven't had the time to mature as SQL/RDBMs.

I’m dubious, Postgres added OO features when that was the fashion, then XML, then JSON. To my mind Graphs are still relations just indirect one, I use graphs a lot within Postgres no problem, it has native recursive CTEs which are surprisingly fast for most Graph queries. For v large graphs I cache the graph with an automatically generated transitive closure table or mat view. With Postgres you can do union distinct inside the Recursive CTE to prevent cycles and it’s remarkably fast, millions of rows under 1 minute.

Re: SQL: One of the most valuable skills

#219

Earlier quoted context omitted.

I understand your perspective, but I look at it a different way. SQL is troublesome to some programming types because it seems alien to ask what you want instead of telling the computer what to do and I find most programmers, especially ASD-types (who I think have an edge for some situations, like writing certain code in a huge org like Google) find this an unfamiliar and strange way of thinking. You're right about s…

Most programmers who are at all familiar with functional programming, DSLs, configuration languages, or optimizing compilers are very well versed with asking the computer for what you want rather than telling it what to do . At least when I was there, this was a very large percentage of Googlers. My issue with SQL is that a programming language should allow you to compose and name building blocks, and then recombine…

True, if you modulo all the features like views, stored procedures, functions, foreign keys, triggers there is no reusability in SQL...

On the serious side: It's not a bad idea to contain critical business logic in the database. It's shared by any app using the database. Foreign keys esp link and let you cascade changes with no extra code.

Less overall code. Higher guarantees.

Everyone agrees it's a good idea to use a `datetime` field for a `created` field instead of `varchar` and letting the application parse/extract it's own format in there.

It's also a good idea to add triggers that do specific things to keep your db in a consistent state upon actions.

Maybe adding/deleting a row needs to update a counter in a statistics table for quick access because count(*) on that table takes too long.

You could do that it in your app - but what happens if someone changes the data without going through your app.

Instead you add it as a trigger. Now you can be sure that the number in the stats table is always the same as the actual number.

Post reply on HN