Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stochastic gradient descent written in SQL

maxhalford.github.io

Post reply on HN
Print view View original

71–80 of 187 posts

Re: Stochastic gradient descent written in SQL

#71

Post by zackmorris »

This is great! Moving away from the proprietary nature of GPUs and complex math gatekeeping should help democratize AI.

Has anyone converted stuff like gradient descent to set theory?

https://support.unicomsi.com/manuals/soliddb/7/SQL_Guide/2_G...

https://www.sqlshack.com/mathematics-sql-server-fast-introdu...

https://www.sqlshack.com/learn-sql-set-theory/

Right now AI algorithms kind of look imperative and stateful to me, like state machines. But there should be a functional or relational representation, where pattern matching would be used to derive the current state instead.

It's trivial to go from functional to imperative representation, but often nearly impossible to go the other way. That's why monadic (sorry if I'm mincing terms) logic is so troublesome. Mutable variables, futures/promises, async, etc can't be statically analyzed, which is why most code today is difficult to parallelize and stuff like C++ optimizers don't even increase speed to the next order of magnitude. But functional languages have nearly infinite optimization potential through divide and conquer approaches like sharding and scatter-gather arrays that can run internally without side effects. In other words, we can throw hardware at SQL for linear speedup since it's embarrassingly parallel, but might have limited success optimizing something like Python.

Re: Stochastic gradient descent written in SQL

#72

Post by civilized »

In the comments here so far, we see a pattern we've seen before. When someone suggests doing something in SQL, there's a lot of concern about SQL being a very limited programming language where it's hard to do proper engineering.

Here's I would really love to know: why is it that SQL is, to first order, the only language used to interact with databases, and SQL has about the same features as it did in the 70s? It seems analogous to if the general-purpose programming world stopped with C.

Re: Stochastic gradient descent written in SQL

#73

Post by noloblo »

noloblo wrote: post #19
jackschultz wrote: post #13

>A machine learning algorithm which can be trained using SQL opens a world of possibilities. The model and the data live in the same space. This is as simple as it gets in terms of architecture. Basically, you only need a database which runs SQL. First paragraph of the conclusion, and this very much fits with the mindset that's been growing in me in the data world over the past few years. Databases are much more powe…

+1 sql is extremely elegant composable and is under rated Postgres is very powerful. While I sought a short detour in nosql Mongodb land now back to Mysql Postgresql sql territory and glad for it Being able to generate views is and stored procedures is useful as well.having sql Take over more like ml, gradient descent does open up good possibility. Also since sql is declarative it Makes it so it's rather easier than…

Postgres has read write parallism that can scale across millions of read writes ; if ml model is inherent in the Postgres db it is indeed very elegant reminds me of the glory days of tsearch2 to do text search in Postgres for searching our internal document repository using nothing but Postgres and boy was it faster than Java based search systems

Re: Stochastic gradient descent written in SQL

#75

Post by datan3rd »

naasking wrote: post #64
dventimi wrote: post #45

Earlier quoted context omitted.

Is a query not a relation?

Basically, but queries are not first class in SQL. You can't assign a query to a variable, or pass it as a parameter to a stored procedure, for example. This would make SQL composable: declare @x = (select * from Person) select Name from @x where Birthdate

  with persons as (select * from Person)  
  select Name from persons  
  where Birthdate

Re: Stochastic gradient descent written in SQL

#76

Post by maxfurman »

civilized wrote: post #72

In the comments here so far, we see a pattern we've seen before. When someone suggests doing something in SQL, there's a lot of concern about SQL being a very limited programming language where it's hard to do proper engineering. Here's I would really love to know: why is it that SQL is, to first order, the only language used to interact with databases, and SQL has about the same features as it did in the 70s? It see…

Plenty of SQL features have been added since the 70s, notably window functions (which TFA relies on heavily). Most of the major databases are from the 80s, and even new kid on the block MySQL has been around since the mid 90s.

SQL draws a very hard line between the expression of the query and the AST that is used in the actual implementation. Database vendors like this aspect because they are free to implement whichever optimizations they want, but application developers want to build queries programmatically and optimize them themselves before passing them to the db engine, hence the tension in threads like these.

Re: Stochastic gradient descent written in SQL

#77

Post by naasking »

datan3rd wrote: post #75
naasking wrote: post #64

Earlier quoted context omitted.

Basically, but queries are not first class in SQL. You can't assign a query to a variable, or pass it as a parameter to a stored procedure, for example. This would make SQL composable: declare @x = (select * from Person) select Name from @x where Birthdate

with persons as (select * from Person) select Name from persons where Birthdate

Where is the assignment to a variable? Where can you construct a query using a variable in table/query position? That's the whole point of being first class and composable, a query becomes like any other value so you should be able to parameterize any query by another query assigned to a variable that may have been set inside an if-statement, or accepted as a parameter to a stored procedure. You know, the same kinds of composition we see in ordinary programming languages.

Re: Stochastic gradient descent written in SQL

#78

Post by naasking »

noloblo wrote: post #69
naasking wrote: post #64

Earlier quoted context omitted.

Basically, but queries are not first class in SQL. You can't assign a query to a variable, or pass it as a parameter to a stored procedure, for example. This would make SQL composable: declare @x = (select * from Person) select Name from @x where Birthdate

Exactly since it declarative the style Lends itself using stored procedure calls to become composable

You cannot abstract over stored procedures either, so that's still not composable.

Re: Stochastic gradient descent written in SQL

#79

Post by naasking »

Lemaxoxo wrote: post #70
naasking wrote: post #64

Earlier quoted context omitted.

Basically, but queries are not first class in SQL. You can't assign a query to a variable, or pass it as a parameter to a stored procedure, for example. This would make SQL composable: declare @x = (select * from Person) select Name from @x where Birthdate

Isn't that the point of common table expressions (CTEs)?

See: https://news.ycombinator.com/item?id=35058927

Re: Stochastic gradient descent written in SQL

#80

Post by bob1029 »

jackschultz wrote: post #13

>A machine learning algorithm which can be trained using SQL opens a world of possibilities. The model and the data live in the same space. This is as simple as it gets in terms of architecture. Basically, you only need a database which runs SQL. First paragraph of the conclusion, and this very much fits with the mindset that's been growing in me in the data world over the past few years. Databases are much more powe…

> Databases are much more powerful than we think

The older I get the more I agree with this.

There is nothing you cannot build by combining SQL primitives. Side effects can even be introduced - on purpose - by way of UDFs that talk to the outside world.

I've seen more than one system where the database itself was directly responsible for things like rendering final HTML for use by the end clients. You might think this is horrible and indeed many manifestations are. But, there lurk an opportunity for incredible elegance down this path, assuming you have the patience to engage in it.

Post reply on HN