You probably don't need AI/ML. You can make do with well written SQL scripts
threadreaderapp.com
You probably don't need AI/ML. You can make do with well written SQL scripts
1–10 of 331 posts
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#2As someone who sells both of these services, I can only add that it depends, and if you have a good dataset, it's trivial to write either one.
But once you start having to account for noise or seasonality or autoregression or dynamic weights or non linear kernel spaces, pure SQL really starts to fall down on the job.
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#3Companies have a large problem of having their data tucked away or inaccessible to the stakeholders. When people talk about AI / ML what they actually need is their data cleaned to the point where they can communicate to their stakeholders. Also, all of the companies who sell AI / ML as consultants are really good already at cleaning data.
When companies actually hire data scientists what they typically do is clean data for a few months to a year . Then they interpret the data by probably being able to perform linear regression. At that point the data is in a state where it can be easily understood by those stakeholders and then they have created value. Whether or not the linear regression or whatever model has been learned may mean something. But, at the end of the day you need to tell stakeholders how they can create value and guess what SQL and Bash will do 90% of the job.
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#4Who on earth are these people describing ?
I've never heard of anyone hiring expensive Data Scientists, spinning up Spark/H2O clusters, building a data lake, doing a database offload to S3/HDFS all for a "select from orders table where basket size is the biggest" query.
AI/ML doesn't even work like this. It's simply not designed for giving 100% accurate answers to highly structured queries.
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#5Companies have a large problem of having their data tucked away or inaccessible to the stakeholders. When people talk about AI / ML what they actually need is their data cleaned to the point where they can communicate to their stakeholders. Also, all of the companies who sell AI / ML as consultants are really good already at cleaning data. When companies actually hire data scientists what they typically do is clean d…
Agreed. The advertising agency I previously worked at would take the first month and only work on how data is collected and stored. After that first month they would begin addressing goals, plans, etc. About a year later is when true value would be realized because data that previously never existed could be analyzed.
Few things were more enjoyable for me than getting a new client, imagining what analysis I'd like to do, figuring out what data would be necessary, and then implementing the system to make it reality.
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#6Companies have a large problem of having their data tucked away or inaccessible to the stakeholders. When people talk about AI / ML what they actually need is their data cleaned to the point where they can communicate to their stakeholders. Also, all of the companies who sell AI / ML as consultants are really good already at cleaning data. When companies actually hire data scientists what they typically do is clean d…
Agreed. The advertising agency I previously worked at would take the first month and only work on how data is collected and stored. After that first month they would begin addressing goals, plans, etc. About a year later is when true value would be realized because data that previously never existed could be analyzed. Few things were more enjoyable for me than getting a new client, imagining what analysis I'd like to…
how did you convince them to stick on for a year when most clients want to see some results within 2 months or they give up?
according to your schedule they’d ask what I’ve been doing and if I tell them I’ve just been “collecting data” that translates to them as “I haven’t done anything in 2 months”. If I say “keep paying me and you’ll see results in a year” that translates to “I haven’t done anything in 2 months but I want you to continue to pay me for another 10”.
What do you do to qualify a client? How do you know your engagement with them won’t just waste everyone’s time when they quit halfway through and then damage your reputation?
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#7As someone who sells both of these services, I can only add that it depends, and if you have a good dataset, it's trivial to write either one. But once you start having to account for noise or seasonality or autoregression or dynamic weights or non linear kernel spaces, pure SQL really starts to fall down on the job.
Curious.
OP gave a few examples for Ecommerce where SQL will do fine. Can you give a few where ML will do something otherwise impossible or harder with SQL?
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#8[deleted]
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#9Who on earth are these people describing ? I've never heard of anyone hiring expensive Data Scientists, spinning up Spark/H2O clusters, building a data lake, doing a database offload to S3/HDFS all for a "select from orders table where basket size is the biggest" query. AI/ML doesn't even work like this. It's simply not designed for giving 100% accurate answers to highly structured queries.
People who don’t know what AI actually is and buying it anyways. I’ve actually seen this first hand. The developers/data scientists involved simply did what they were asked even though it didn’t make much sense (we had tried and failed to explain why this was a waste too many times and got nowhere)
Although to be fair, this outcome was still an improvement. At least with using machine learning unnecessarily the data actually meant something and wasn’t just arbitrary excel numerology.
Re: You probably don't need AI/ML. You can make do with well written SQL scripts
#10Companies have a large problem of having their data tucked away or inaccessible to the stakeholders. When people talk about AI / ML what they actually need is their data cleaned to the point where they can communicate to their stakeholders. Also, all of the companies who sell AI / ML as consultants are really good already at cleaning data. When companies actually hire data scientists what they typically do is clean d…
Even worse than stakeholders not having data is stakeholders having relatively raw data and building “reports” themselves with Excel that don’t actually mean anything. Sometimes this is just done to provide justification for something someone wants to do, but over time these things get built up and entrenched and the business is actually driving decisions off of completely meaningless metrics.
So while it may be tempting to simply hand data over to the business because “that’s not my job”, that can totally come back to bite you in the ass when your product roadmap is being dictated by ill conceived Excel formulas.