Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stochastic gradient descent written in SQL

maxhalford.github.io

Post reply on HN
Print view View original

131–140 of 187 posts

Re: Stochastic gradient descent written in SQL

#131

Post by dventimi »

yunwal wrote: post #129
dventimi wrote: post #57

Earlier quoted context omitted.

The resources I'm familiar with are I/O, memory, and CPU. The only one I believe can be spared in the database by using that resource outside of the database, is CPU. When the database is far from saturated on CPU and latency and throughput are determined by I/O and memory, using CPU on some other machine that isn't the database can't possibly have any impact on latency and throughput.

> When the database is far from saturated on CPU The issue here is if you scale enough saturate the database, you'll have to rewrite essentially all your code if you're a typical CRUD webapp. Basically all of your business logic is about data retrieval. There's probably some companies that can get away with this, but it would be way too expensive for most.

If I scale up to saturate the database CPU...by doing data retrieval? Setting aside my skepticism about saturating the CPU with mere data retrieval, how is that solved by moving the data to another host's CPU, when moving the data involves the very data retrieval that's saturating the database's CPU?

Re: Stochastic gradient descent written in SQL

#132

Post by jhd3 »

jackschultz wrote: post #13

>A machine learning algorithm which can be trained using SQL opens a world of possibilities. The model and the data live in the same space. This is as simple as it gets in terms of architecture. Basically, you only need a database which runs SQL. First paragraph of the conclusion, and this very much fits with the mindset that's been growing in me in the data world over the past few years. Databases are much more powe…

> Databases are much more powerful than we think

and data has mass. One example of bringing the work to the data is https://madlib.apache.org/ (works on Postgres and Greenplum)

[Disclaimer - former employee of Pivotal]

Re: Stochastic gradient descent written in SQL

#133

Post by CreRecombinase »

jackschultz wrote: post #13

>A machine learning algorithm which can be trained using SQL opens a world of possibilities. The model and the data live in the same space. This is as simple as it gets in terms of architecture. Basically, you only need a database which runs SQL. First paragraph of the conclusion, and this very much fits with the mindset that's been growing in me in the data world over the past few years. Databases are much more powe…

I spend most of my time in the parallel universe that is scientific computing/HPC. In this alternate reality SQL (not to mention databases) never really took off. Instead of scalable, performant databases, we have only the parallel filesystem. I'm convinced the reason contemporary scientific computing don't involve much SQL is sociological/path-dependency, but there are also very good technical reasons. Optimizing software in scientific computing involves two steps: 1) Think hard about your problem until you can frame it as one or more matrix multiplications 2) Plug that into a numerical linear algebra library The SQL abstraction (in my experience) takes you very much in the opposite direction.

Re: Stochastic gradient descent written in SQL

#134

Post by dogcomplex »

zackmorris wrote: post #71

This is great! Moving away from the proprietary nature of GPUs and complex math gatekeeping should help democratize AI. Has anyone converted stuff like gradient descent to set theory? https://support.unicomsi.com/manuals/soliddb/7/SQL_Guide/2_G... https://www.sqlshack.com/mathematics-sql-server-fast-introdu... https://www.sqlshack.com/learn-sql-set-theory/ Right now AI algorithms kind of look imperative and stateful…

Replying to this to remind myself to research later, as this is a very important question

Re: Stochastic gradient descent written in SQL

#135

Post by spprashant »

jackschultz wrote: post #13

>A machine learning algorithm which can be trained using SQL opens a world of possibilities. The model and the data live in the same space. This is as simple as it gets in terms of architecture. Basically, you only need a database which runs SQL. First paragraph of the conclusion, and this very much fits with the mindset that's been growing in me in the data world over the past few years. Databases are much more powe…

The problem with throwing everything in a database is you end up with brittle stored procedures all over the place, which are painful to debug. There is no good support for version control or testing, which means you end up creating a dozen copies of each function named (sp_v1, sp_v2,.., etc.). It much more harder to practice iterative development which the rest of software development seems implements effectively.

Also traditional relational databases have a way to go before they can support parallelized machine learning workloads. You do not have control or the ability to spin up threads or processes to boost your performance. You rely on the query processor to make those decisions, and depending on your platform the results will be mixed.

Re: Stochastic gradient descent written in SQL

#136

Post by Lemaxoxo »

airstrike wrote: post #123
Lemaxoxo wrote: post #114

Earlier quoted context omitted.

Do you have some data/resources on this? I'm a total snowflake at this, but I'm willing to learn.

In hindsight this is harder than it seems if you don't already have access to data, so I'll try to be as informative as possible in my response It's hard to find this information out there, so here's ~all you need to know. Data is usually behind paywalls, unfortunately. Industry standards are Bloomberg terminal (ridiculously expensive, 5 digits $), FactSet (very expensive, 4 digits), Capital IQ (expensive, not sure).…

Thank you so much for this! It's very generous of you to have taken the time.

Re: Stochastic gradient descent written in SQL

#137

Post by giraffe_lady »

dventimi wrote: post #118
giraffe_lady wrote: post #98

Earlier quoted context omitted.

> I've seen more than one system where the database itself was directly responsible for things like rendering final HTML for use by the end clients. I did this for a side project a few months ago and even used postgrest to serve the page with correct headers for html. It felt simultaneously really cursed and obvious. Shit you could even use plv8 to run mustache or whatever in the db if you really wanted to piss peopl…

I'm doing this right now with a DO droplet built with PostgreSQL, postgrest, nginx, and not much else. Do you have any tips, tricks, or blog posts you can share based on your experience? You should post it to HN. Strike while the iron's hot. With a little luck you'll hit the front page.

Nah nothing that refined. Pretty predictably supabase is doing some weird stuff along these lines and I found some abandoned and semi-active repos associated with them and people working for them that were useful examples of some things.

As for posting to HN absolutely no thanks. These people are so fucking hostile there is no accomplishment too small to tear apart for entertainment here. I have no interest in it.

Re: Stochastic gradient descent written in SQL

#138

Post by airstrike »

Lemaxoxo wrote: post #136
airstrike wrote: post #123

Earlier quoted context omitted.

In hindsight this is harder than it seems if you don't already have access to data, so I'll try to be as informative as possible in my response It's hard to find this information out there, so here's ~all you need to know. Data is usually behind paywalls, unfortunately. Industry standards are Bloomberg terminal (ridiculously expensive, 5 digits $), FactSet (very expensive, 4 digits), Capital IQ (expensive, not sure).…

Thank you so much for this! It's very generous of you to have taken the time.

My pleasure! I've spent the better part of the last decade doing this stuff and I appreciate how hard it is to find resources on it, so thought I'd share since you mentioned you were interested in learning

At the risk of exhausting said interest, here's one real life example of a similar analysis: https://wsp-blog-images.s3.amazonaws.com/uploads/2011/09/171...

See PDF page 14. Note the lines called "Composite P / NTM EPS" which they built as a blend of American Eagle's, Tilly's and Zumiez's P/E multiple, which are companies X, Y and Z in my comment above (for some reason they gave AE double the weight which is unusual) and compared it to Heat's P/E multiple (Heat was the codename for retailer Rue21, or hypothetical company W in my example above)

I got this deck from https://www.wallstreetprep.com/knowledge/investment-banking-... which has a few other examples and a variety of other resources on the topic FYI

Re: Stochastic gradient descent written in SQL

#139

Post by Serow225 »

malnourish wrote: post #110
Serow225 wrote: post #48

Earlier quoted context omitted.

Another is MS SQL Server, which lets you run .NET on the database server :D "you can author stored procedures, triggers, user-defined functions, user-defined types, and user-defined aggregates in managed code"

I have had nothing but bad experiences trying to run .NET in SSIS packages -- is there another way?

I've never had the pleasure(ha) of using SSIS, but this is the stuff that I was talking about: https://learn.microsoft.com/en-us/dotnet/framework/data/adon...

Re: Stochastic gradient descent written in SQL

#140

Post by dventimi »

giraffe_lady wrote: post #137
dventimi wrote: post #118

Earlier quoted context omitted.

I'm doing this right now with a DO droplet built with PostgreSQL, postgrest, nginx, and not much else. Do you have any tips, tricks, or blog posts you can share based on your experience? You should post it to HN. Strike while the iron's hot. With a little luck you'll hit the front page.

Nah nothing that refined. Pretty predictably supabase is doing some weird stuff along these lines and I found some abandoned and semi-active repos associated with them and people working for them that were useful examples of some things. As for posting to HN absolutely no thanks. These people are so fucking hostile there is no accomplishment too small to tear apart for entertainment here. I have no interest in it.

> As for posting to HN absolutely no thanks. These people are so fucking hostile there is no accomplishment too small to tear apart for entertainment here. I have no interest in it.

You make a really good point.

Post reply on HN