It still surprises me, although it shouldn't, that many developers try to 'hide' SQL through (in mi opinion, convoluted) ORMs. Although I see the apparent beauty in the simplicity they provide to front-end developers, to me, additional layers will always add complications down the line.
SQL: One of the most valuable skills
301–310 of 390 posts
Re: SQL: One of the most valuable skills
#302Earlier quoted context omitted.
Two Things. (1) Think of a database as a "big pile of stuff in a room". Some of it's ordered in a sane way, some of it's not. There is "a person at the door" to the room preventing you from entering - this is the Database Engine, not to be confused with the Database itself. You need to get things out of the room, and you need to put things in the room. You're not allowed to enter. If you want something out of the roo…
Thank you for the Stanford class reference. Will check it out.
I took this one ages ago, before I started programming (in prep for a bootcamp) and really enjoyed it.
Re: SQL: One of the most valuable skills
#303While I find SQL useful too, there are other things I value even more than my SQL skills.
Being able to use a POSIX Shell with tools like curl is certainly one of them as it enables me to connect different technologies on a very practical level.
Furthermore, understanding the basic functional programming principles is invaluable if you want to build clean algorithms (doesn't matter if you use Excel, C or Lisp).
Re: SQL: One of the most valuable skills
#304Earlier quoted context omitted.
Can I ask a simple question about efficiency? It seems to me that graph databases are far more efficient than relational ones for most tasks. That’s because all lookups are O(1) instead of O(log N). That adds up. Also, copying a subgraph is far easier, and so is joining. Think about it, when you shard you are essentially approaching graph databases because your hash or range by which you find your shard is basically…
A hash index is O(1) no matter what language it's called from. The reason most people don't use hash indices very often is that they do not allow you to do a lot of useful things that you often end up wanting to do, such as retrieving all values within a given range.
I still don’t think it’s O(1) in the limit. Why not just store pointers to the data, or one indirection if the data can be moved?
Re: SQL: One of the most valuable skills
#305I fell in love again with SQL while working in my current position. What _really_ made me fall in love was the project dbt[0] which is a SQL compiler and executor. You can build a DAG of transformations all the way from your raw data to tables ready for viewing in your BI tool or consumption by your ML model. I'm still amazed at the things I can do with SQL alone without having to bring Python into the picture. Also,…
Same experience here. dbt has been a pleasure to work with, and putting it on Redshift with Redshift Spectrum has proven a great pipeline for our startup.
Re: SQL: One of the most valuable skills
#306Earlier quoted context omitted.
I understand your perspective, but I look at it a different way. SQL is troublesome to some programming types because it seems alien to ask what you want instead of telling the computer what to do and I find most programmers, especially ASD-types (who I think have an edge for some situations, like writing certain code in a huge org like Google) find this an unfamiliar and strange way of thinking. You're right about s…
Most programmers who are at all familiar with functional programming, DSLs, configuration languages, or optimizing compilers are very well versed with asking the computer for what you want rather than telling it what to do . At least when I was there, this was a very large percentage of Googlers. My issue with SQL is that a programming language should allow you to compose and name building blocks, and then recombine…
You may be interested in my project "Binate"; I have some incomplete notes at http://canonical.org/~kragen/binary-relations about it, exploring a possible design space in which we might find a more concise and compositional query language. Of course that's not in a host programming language; for that there do exist some things like SQLAlchemy and SQLObject which, as a bonus, can compile the resulting query to SQL. The most popular of these is Django's "ORM", which unfortunately is very limited.
Re: SQL: One of the most valuable skills
#307Reading the thread so far, I see SQL and bash suggested as being valuable skills. Can you suggest more skills and tools like these that are timeless and valuable?
Regular expressions
I'd add that one should also learn LL and (G(LA))LR grammars. Regex are handy for quick hacks, but anything non-regular becomes hacky and overcomplicated compared with a self-describing grammar. PCREs are over-engineered.
There seems to be an equivalent of the "NoSQL" crowd in parsing which appears to be "NoLALR". I can certainly see that other kinds of parsers have their use cases - but LL and LALR parsing have sound theory which is well understood, and they are certainly not obsolete.
Re: SQL: One of the most valuable skills
#308I spent a year in a role where 50% of my duties was writing sql reports. These reports where usually between 500 and 1000 lines of sql a pop. Sometimes the runtime of the report was measured in hours, so learning efficient sql was important. The company had a lot of people that had been writing sql for awhile, and there were lots of cool code snippets floating around. I learned a lot in that year. I've moved to writi…
Knowing how to optimise SQL (and also database indexes) is a valuable skill. Reducing a highly used query's execution time by several orders of magnitude can be quite gratifying.
Stick the --+ORDERED flag on a random select sometime and look at what happens to the estimated query cost to see how easy it would be to fuck this up if you had to make all the optimisation choices yourself.
Re: SQL: One of the most valuable skills
#309Earlier quoted context omitted.
> My cries to use git were unheeded (would have required upskilling everyone on the team). What is the best practice workflow using git with SQL server views/procedures? Can you actually somehow track changes in the views/procs themselves so that if someone happens to run ALTER VIEW, git diff is going to show something?
Every conversation I've ever had came back to using RedGate SQL Compare to diff databases and TeamCity for CI. You basically shouldn't allow anyone to modify anything without it being scripted (bonus points if it comes with a rollback and is repeatable for testing). Your scripts then all go into Git.
RedGate's SQL Change Automation (formerly ReadyRoll) is pretty slick. DbUp is a good, free alternative.
I second the rollback and repeatability bonus. Every script should leave the database in either the new state or the previous good state no matter how many times it's run.
Re: SQL: One of the most valuable skills
#310Earlier quoted context omitted.
Regular expressions
I'd add that one should also learn LL and (G(LA))LR grammars. Regex are handy for quick hacks, but anything non-regular becomes hacky and overcomplicated compared with a self-describing grammar. PCREs are over-engineered. There seems to be an equivalent of the "NoSQL" crowd in parsing which appears to be "NoLALR". I can certainly see that other kinds of parsers have their use cases - but LL and LALR parsing have soun…
This is just an industry getting bigger, more people gain more experience and start to question all the dogmas. LR was taught dogmatically, but was never realistically any good or even useful in practice, hence "NoLALR" was born.