You probably don't need AI/ML. You can make do with well written SQL scripts • Hacker News Board

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#11

Post by philliphaydon » Mon, Apr 23, 2018, 3:10 AM UTC

threeseed wrote: post #4

Who on earth are these people describing ? I've never heard of anyone hiring expensive Data Scientists, spinning up Spark/H2O clusters, building a data lake, doing a database offload to S3/HDFS all for a "select from orders table where basket size is the biggest" query. AI/ML doesn't even work like this. It's simply not designed for giving 100% accurate answers to highly structured queries.

> Who on earth are these people describing ?

I was asked about my thoughts on AI/ML at work, I said it didn't really apply to us. I was told "but with ML we can figure out when deliveries are happening and scale the machines before the deliveries happen based on the peak traffic times". I tried to explain that we could so all that from SQL and looking at our data. We have all the data we just need to formulate it into something that makes sense to predict which times of day, days of week, for each region, where we have more traffic then use that data to pre-scale. I was shot down to "you clearly do not understand ML and should go read up on it".

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#12

Post by smrtinsert » Mon, Apr 23, 2018, 3:10 AM UTC

rokhayakebe wrote: post #7

kthejoker2 wrote: post #2

As someone who sells both of these services, I can only add that it depends, and if you have a good dataset, it's trivial to write either one. But once you start having to account for noise or seasonality or autoregression or dynamic weights or non linear kernel spaces, pure SQL really starts to fall down on the job.

Curious. OP gave a few examples for Ecommerce where SQL will do fine. Can you give a few where ML will do something otherwise impossible or harder with SQL?

Give me all the time frames when there was statistically significant bump in X.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#13

Post by throwawayjava » Mon, Apr 23, 2018, 3:11 AM UTC

rokhayakebe wrote: post #7

kthejoker2 wrote: post #2

As someone who sells both of these services, I can only add that it depends, and if you have a good dataset, it's trivial to write either one. But once you start having to account for noise or seasonality or autoregression or dynamic weights or non linear kernel spaces, pure SQL really starts to fall down on the job.

Curious. OP gave a few examples for Ecommerce where SQL will do fine. Can you give a few where ML will do something otherwise impossible or harder with SQL?

controls and sensing.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#14

Post by hahla » Mon, Apr 23, 2018, 3:13 AM UTC

threeseed wrote: post #4

Who on earth are these people describing ? I've never heard of anyone hiring expensive Data Scientists, spinning up Spark/H2O clusters, building a data lake, doing a database offload to S3/HDFS all for a "select from orders table where basket size is the biggest" query. AI/ML doesn't even work like this. It's simply not designed for giving 100% accurate answers to highly structured queries.

These people are describing 99% of the Fortune 500 companies who have no idea what AI means other than hiring a team of data scientists that will hopefully solve all of their problems in the name of technology.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#15

Post by firasd » Mon, Apr 23, 2018, 3:15 AM UTC

rokhayakebe wrote: post #7

kthejoker2 wrote: post #2

As someone who sells both of these services, I can only add that it depends, and if you have a good dataset, it's trivial to write either one. But once you start having to account for noise or seasonality or autoregression or dynamic weights or non linear kernel spaces, pure SQL really starts to fall down on the job.

Curious. OP gave a few examples for Ecommerce where SQL will do fine. Can you give a few where ML will do something otherwise impossible or harder with SQL?

Product Recommendations. Trending Items (Top items being sold this week as opposed to last week, while filtering out items that are generally popular.) Much easier with Elasticsearch than SQL https://www.elastic.co/blog/significant-terms-aggregation

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#16

Post by rdlecler1 » Mon, Apr 23, 2018, 3:16 AM UTC

zitterbewegung wrote: post #3

Companies have a large problem of having their data tucked away or inaccessible to the stakeholders. When people talk about AI / ML what they actually need is their data cleaned to the point where they can communicate to their stakeholders. Also, all of the companies who sell AI / ML as consultants are really good already at cleaning data. When companies actually hire data scientists what they typically do is clean d…

This is a very good point. Analysis can become quite trivial when you have clean data. With messy data you may need 3+ orders of magnitude of data which is often not possible. Unfortunately VC pays a higher premium for data scientist than data janitor.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#17

Post by reilly3000 » Mon, Apr 23, 2018, 3:19 AM UTC

This is the opposite of AI use cases in marketing. You are declaring a specific timeframe for your message delivery. That is not how a marketer should deploy AI. I haven’t been in any pitch meetings since AI assclownery took hold so I can’t comment on how the term is being abused. What I can say is that a model that used AI would take every parameter it could about each customer and determine the optimal time to sent an email to get a conversion. The only inputs the marketer should provide is raw historical data with clear parameters like order value, order items, estimated revenue, buyer classification, a stream of subsequent etc and date, and the model should solve for the correct timestamp to send the follow up message. I don’t think the AI is writing the message yet, and I don’t think you need a neural net to do a decent job at solving for the right datetime to send. I do think the approach I described would get superior conversion rates than a rule, cost more to make than that rule, and definitely demand a decently huge dataset to add much value.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#18

Post by AndyNemmity » Mon, Apr 23, 2018, 3:20 AM UTC

threeseed wrote: post #4

Who on earth are these people describing ? I've never heard of anyone hiring expensive Data Scientists, spinning up Spark/H2O clusters, building a data lake, doing a database offload to S3/HDFS all for a "select from orders table where basket size is the biggest" query. AI/ML doesn't even work like this. It's simply not designed for giving 100% accurate answers to highly structured queries.

Don't love your tone, but I agree. I have been working in what was called predictive analytics for 16 years.

I've done tons of projects, for tons of companies, and this sort of refrain from people is pretty common when they don't have experience in the field. They think of it like some fad that doesn't make a lick of sense outside of a C-level discussion.

But the reality is, predictive analytics is extremely powerful. One of the last projects I did was to save Trains from derailing. Another was to improve crop yield of a farming company by using satellite imagery to determine when a field was most needed to be harvested. Tons of other examples.

To even explain the particular use cases would take quite awhile because they are domain specific issues. Cron isn't solving these problems.

What the person is really saying is, I don't have experience in these topics, what can be so hard about them?

The same style for people who arm chair sports, or politics, or programming, or any other topic. It all seems easy when you don't know the details.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#19

Post by AndyNemmity » Mon, Apr 23, 2018, 3:22 AM UTC

hahla wrote: post #14

threeseed wrote: post #4

Who on earth are these people describing ? I've never heard of anyone hiring expensive Data Scientists, spinning up Spark/H2O clusters, building a data lake, doing a database offload to S3/HDFS all for a "select from orders table where basket size is the biggest" query. AI/ML doesn't even work like this. It's simply not designed for giving 100% accurate answers to highly structured queries.

These people are describing 99% of the Fortune 500 companies who have no idea what AI means other than hiring a team of data scientists that will hopefully solve all of their problems in the name of technology.

A far greater percentage know what it is, and use it than you are giving credit for.

Re: You probably don't need AI/ML. You can make do with well written SQL scripts

#20

Post by AndyNemmity » Mon, Apr 23, 2018, 3:23 AM UTC

rokhayakebe wrote: post #7

kthejoker2 wrote: post #2

As someone who sells both of these services, I can only add that it depends, and if you have a good dataset, it's trivial to write either one. But once you start having to account for noise or seasonality or autoregression or dynamic weights or non linear kernel spaces, pure SQL really starts to fall down on the job.

Curious. OP gave a few examples for Ecommerce where SQL will do fine. Can you give a few where ML will do something otherwise impossible or harder with SQL?

Pricing.