What SQL Analysts Need to Know About Python (2016)
1–10 of 115 posts
Re: What SQL Analysts Need to Know About Python (2016)
#2SQL+Python is extremely powerful, and the author makes some good points (multivariate regression should be done in python and not SQL for example), but the query example in the blog is not a good one. Every modern DB has aggregate and statistical functions like ntile, percent_Rank, median, min, max, etc. I’d honestly rather run these functions against the database than do it in python. Especially if you're working with billion+ row datasets. In fact if all I'm doing is these kinds of statisical/aggregate functions then I definitely wouldn't use python at all.
Re: What SQL Analysts Need to Know About Python (2016)
#3A few years ago I joined a Rails shop, and one thing that always struck me was how many of the engineers didn't know SQL. Most of them had learned to code on Rails, and had always had SQL abstracted away via ActiveRecord.
I know this is not the point of this article, but as data analyst/scientist roles continue to climb in popularity, I'm curious if there won't be a similar trend with Python.
Re: What SQL Analysts Need to Know About Python (2016)
#4A few years ago I joined a Rails shop, and one thing that always struck me was how many of the engineers didn't know SQL. Most of them had learned to code on Rails, and had always had SQL abstracted away via ActiveRecord. I know this is not the point of this article, but as data analyst/scientist roles continue to climb in popularity, I'm curious if there won't be a similar trend with Python.
Pretty easy to see how, just look at the Python equivalent in Django. You're so far abstracted away from what is actually going on that it's no wonder no one understands it. There is a lot of gotchas, that most probably never investigate, particularly multiple calls to the database for very simple join operations that aren't made apparent in the ORM unless you're watching SQL logs(they aren't).
Re: What SQL Analysts Need to Know About Python (2016)
#5A few years ago I joined a Rails shop, and one thing that always struck me was how many of the engineers didn't know SQL. Most of them had learned to code on Rails, and had always had SQL abstracted away via ActiveRecord. I know this is not the point of this article, but as data analyst/scientist roles continue to climb in popularity, I'm curious if there won't be a similar trend with Python.
Pretty easy to see how, just look at the Python equivalent in Django. You're so far abstracted away from what is actually going on that it's no wonder no one understands it. There is a lot of gotchas, that most probably never investigate, particularly multiple calls to the database for very simple join operations that aren't made apparent in the ORM unless you're watching SQL logs(they aren't).
There’s tools to debug the lowest hanging fruit. You can go a long way with an ORM.
Re: What SQL Analysts Need to Know About Python (2016)
#6I'm worried about the inverse - what Python data analysts should know about SQL. Because I've met tons of analysts who wouldn't be able to even run a basic select.
I've seen tons of (often non-reproducible) code written in place of a simple SQL query. I really wish bootcamps and other learning platforms focused on SQL a bit more.
(I am a Python data analyst who properly learned SQL only after several years in the industry.)
Re: What SQL Analysts Need to Know About Python (2016)
#7I'm worried about the inverse - what Python data analysts should know about SQL. Because I've met tons of analysts who wouldn't be able to even run a basic select. I've seen tons of (often non-reproducible) code written in place of a simple SQL query. I really wish bootcamps and other learning platforms focused on SQL a bit more. (I am a Python data analyst who properly learned SQL only after several years in the ind…
Or worse, they use cursors. Cursors should never be used in SQL. Ever. That's my philosophy.
Re: What SQL Analysts Need to Know About Python (2016)
#8uMatrix prevents it from loading.
Against my better judgement, I turned off uMatrix and loaded the page. Then had a look at uMatrix.
Wow, I think we have a winner for a site with the most scripts, most 3rd party domains, and the sheer number of XHRs.
Sorry for this being off topic.
Re: What SQL Analysts Need to Know About Python (2016)
#9uMatrix prevents it from loading. Against my better judgement, I turned off uMatrix and loaded the page. Then had a look at uMatrix. Wow, I think we have a winner for a site with the most scripts, most 3rd party domains, and the sheer number of XHRs. Sorry for this being off topic.
Do you have a sane setup you can share with uMatrix? Last time I looked at it, I was a bit overwhelmed and shelved it for later.
Re: What SQL Analysts Need to Know About Python (2016)
#10I'm worried about the inverse - what Python data analysts should know about SQL. Because I've met tons of analysts who wouldn't be able to even run a basic select. I've seen tons of (often non-reproducible) code written in place of a simple SQL query. I really wish bootcamps and other learning platforms focused on SQL a bit more. (I am a Python data analyst who properly learned SQL only after several years in the ind…
Or worse, they use cursors. Cursors should never be used in SQL. Ever. That's my philosophy.
Server-side cursors are great if you need to do streaming which I have in the past. It keeps both the client and server memory use in check.