Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who needs MLflow when you have SQLite?

ploomber.io

Post reply on HN
Print view View original

31–40 of 113 posts

Re: Who needs MLflow when you have SQLite?

#31

Post by edublancas »

navbaker wrote: post #19

I work in an environment where there are multiple tech teams developing models for multiple use cases on VMs and GPU clusters spread across our corporate intranet. Once you move beyond a single dev working on a model on their laptop, you absolutely need something that can handle not just metrics tracking, but making the model binaries available and providing a means to ensure reproducibility by the rest of the team.…

Agree. Once you have a team, you need to have a service they can all interact with. This release is a first step, we want to get the user experience right for an individual and then think of how to expand that to teams. Ultimately, the two things we're the most excited about are 1) you don't need to add any extra code (and it works with all libraries, not a pre-defined set) 2) SQL as the query language

Re: Who needs MLflow when you have SQLite?

#32

Post by edublancas »

mostdataisnice wrote: post #21

SQLite is literally a backend for MLflow, so the argument being made really is that you should just use SQL when you can, which is kind of adjacent to any criticisms of MLflow

Is querying the underlying SQL database officially supported in MLflow? Last time I used it, it wasn't documented. I took a look at the database and it wasn't end-user friendly.

Re: Who needs MLflow when you have SQLite?

#33

Post by isoprophlex »

mostdataisnice wrote: post #22
isoprophlex wrote: post #3

Yeah, MLFlow is a shitshow. The docs seem designed to confuse, the API makes Pandas look good and the internal data model is badly designed and exposed, as the article says. But, hordes of architects and managers who almost have a clue have been conditioned to want l and expect mlflow. And it's baked into databricks too, so for most purposes you'll be stuck with it. Props to the author for daring to challenge the sta…

Where does the article say that?

About exposing the data inside MLFlow

> I found the query feature extremely limiting (if my experiments are stored in a SQL table, why not allow me to query them with SQL).

Re: Who needs MLflow when you have SQLite?

#34

Post by jmt_ »

chaxor wrote: post #27
laichzeit0 wrote: post #20

Earlier quoted context omitted.

To be fair, taking R to production is a goddamn nightmare.

R is a trash of a language. It doesn't have any sense of coherency to it at all. They keep trying to fix the underlying problems by ducktaping paradigms on to it over and over (S3, S4, R6, etc). There's never a clear sense of the best way to do anything, but plenty of options to do a thing in a very hacky 'script-kiddy' way. Looking out at the community of different projects it becomes clear that everyone is pretty l…

Trash language is a bit harsh. I'm not sure I would try to put an R project into production or build a huge project with it but, at the very least, R/R Studio was the best scientific calculator I've ever used. Was particularly great during college

Re: Who needs MLflow when you have SQLite?

#35

Post by cameronfraser »

cdong wrote: post #25

I don't get why a lot of people are calling mlflow a shitshow when it has done so much getting data scientist out of recording experiments via CSV. I can log models and parameters and use the UI to track different runs. After comparisons, I can use the registry to register different staging. If you have other model diagnostic charts you can log the artifact as well. I think mlflow v2 has auto logging included so why…

There are those who create and those who complain on the internet about tools they've used one time

Re: Who needs MLflow when you have SQLite?

#37

Post by tomrod »

lordgroff wrote: post #28
chaxor wrote: post #27

Earlier quoted context omitted.

R is a trash of a language. It doesn't have any sense of coherency to it at all. They keep trying to fix the underlying problems by ducktaping paradigms on to it over and over (S3, S4, R6, etc). There's never a clear sense of the best way to do anything, but plenty of options to do a thing in a very hacky 'script-kiddy' way. Looking out at the community of different projects it becomes clear that everyone is pretty l…

Yep, this is a mark of someone that's never used R but has heard a lot of incredibly ill informed criticism around it. One look of dplyr code over pandas would of course disabuse anyone of the notion that R is trash and the tragedy is Python will in the current state never have anything like that. That's the advantage of the language being influenced by Lisp vs not.

I've heavily used R several times.

I agree that it is a trash language and that, outside that many frontier academic ideas are available and some plotting preferences are solidly prescriptive, it should be thrown into the trash bin.

Python, Julia when it gets its druthers for TTFP, Octave, Fortran, C, and eventually Rust. These are the tools I've found in use over and over and over again across business, government, and non-profits.

Everywhere R is used by the org I have seen major gaps in capacity to deliver specifically because R doesn't scale well.

Re: Who needs MLflow when you have SQLite?

#38

Post by isoprophlex »

cameronfraser wrote: post #35
cdong wrote: post #25

I don't get why a lot of people are calling mlflow a shitshow when it has done so much getting data scientist out of recording experiments via CSV. I can log models and parameters and use the UI to track different runs. After comparisons, I can use the registry to register different staging. If you have other model diagnostic charts you can log the artifact as well. I think mlflow v2 has auto logging included so why…

There are those who create and those who complain on the internet about tools they've used one time

Okay that's coming across as a pretty snide remark aimed at me, I'll bite.

Yes, I can understand why you comment that. I don't like blind slagging of free software either.

But there are ALSO those whose day job it is, and has been for the last 2 years, to use a badly designed overcomplex horrorshow of a tool that could be replaced easily by something better ... if it wasn't for the lock-in effects and strong marketing.

So I'm ventilating my frustration and at the same time expressing my gratitude to the person who made something fresh, that shows us things can be better.

I can't build the replacement to MLFlow myself, but I can cheer people on who do, and let them know their efforts are sorely needed.

Re: Who needs MLflow when you have SQLite?

#39

Post by chasil »

benjaminwootton wrote: post #4

The elephant in the room with data is that we don’t need a lot of the fancy and powerful technology. SQL against a relational database gets us extraordinarily far. Add some Python scripts where we need some imperative logic and glue code, and a sprinkle of CI/CD if we really want to professionalise the work of data scientists. I think this covers the vast majority of situations. Despite being around it for some time,…

The article mentions this workflow:

"Let’s now execute the script multiple times, one per set of parameters, and store the results in the experiments.db SQLite database... After finishing executing the experiments, we can initialize our database (experiments.db) and explore the results."

Be warned that issuing queries while DML is in process can result in SQLITE_BUSY, and the default behavior is to abort the transaction, resulting in lost data.

Setting WAL mode for greater concurrency between a writer and reader(s) can lead to corruption if the IPC structures are not visible:

"To accelerate searching the WAL, SQLite creates a WAL index in shared memory. This improves the performance of read transactions, but the use of shared memory requires that all readers must be on the same machine [and OS instance]."

If the database will not be entirely left alone during DML, then the busy handler must be addressed.

Re: Who needs MLflow when you have SQLite?

#40

Post by Fiahil »

LeanderK wrote: post #18

I think MLflow is a good idea (very) badly executed. I would like to have a library that combines: - simple logging of (simple) metrics during and after training - simple logging of all arguments the model was created with - simple logging of a textual representation of the model - simple logging of general architecture details (number of parameters, regularisation hyperparameters, learning rate, number of epochs etc…

> I think MLFlow is a good idea (very) badly executed.

Oh yes, I'm glad to see other with similar opinion.

Post reply on HN