Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who needs MLflow when you have SQLite?

ploomber.io

Post reply on HN
Print view View original

91–100 of 113 posts

Re: Who needs MLflow when you have SQLite?

#91

Post by simonw »

funklute wrote: post #70
imachine1980_ wrote: post #68

Earlier quoted context omitted.

im asking from ignorance, what the difference in effect this context of not having the library itself?

Using the bindings is only possible if the library itself is already installed (since the bindings directly make use of the library, under the hood).

I've never encountered a Python installation on any operating system where `import sqlite3` worked but the underlying libraries were not available.

I imagine this is because SQLite is VERY easy to bundle with Python itself. So on some platforms the OS SQLite is used, but on others it gets shipped as part of the Python installation itself.

It even works in WebAssembly via Pyodide!

Re: Who needs MLflow when you have SQLite?

#92

Post by idomi »

simonw wrote: post #91
funklute wrote: post #70

Earlier quoted context omitted.

Using the bindings is only possible if the library itself is already installed (since the bindings directly make use of the library, under the hood).

I've never encountered a Python installation on any operating system where `import sqlite3` worked but the underlying libraries were not available. I imagine this is because SQLite is VERY easy to bundle with Python itself. So on some platforms the OS SQLite is used, but on others it gets shipped as part of the Python installation itself. It even works in WebAssembly via Pyodide!

+1 I also think it's faster that way on both environment setup and ad hoc rapid experiments, from my experience using the library in a team doesn't scale well, it becomes pretty slow.

Re: Who needs MLflow when you have SQLite?

#93

Post by tomrod »

melagonster wrote: post #89
tomrod wrote: post #61

Earlier quoted context omitted.

Hierarchy on bueracracies, by Jean Tirole. I know because this was the phenomenon I wanted to study in grad school only to find he scooped me (on this an several items) by several decades. Edit: Tirole, Jean. "Hierarchies and bureaucracies: On the role of collusion in organizations." JL Econ. & Org. 2 (1986): 181.

if this research is so old, did world tried any thing to ameliorate this problem? I guess it doesn't happen yet...

36 years is not old in terms of research. 2,223 cites on Google Scholar and many in the past year. Seminal research often identifies the problem but not all solutions.

Re: Who needs MLflow when you have SQLite?

#94

Post by BeefWellington »

citizenpaul wrote: post #53
benjaminwootton wrote: post #4

The elephant in the room with data is that we don’t need a lot of the fancy and powerful technology. SQL against a relational database gets us extraordinarily far. Add some Python scripts where we need some imperative logic and glue code, and a sprinkle of CI/CD if we really want to professionalise the work of data scientists. I think this covers the vast majority of situations. Despite being around it for some time,…

Unless your income is depending on carrying out the exact demands of some money guy that's most common phrase while using a computer is "it won't let me" and they want "big data". Then you just suck it up and build one of the totally unnecessary big data systems that have been excreted all over the business world these days. I don't think the problem is that devs are over-engineering. I wonder what its called, makes…

What gets me is how many companies paid through the nose to push their data into things like Hive and slowed down 99% of their queries to make one "run once a quarter" report run about 25% faster.

At least that was my experience a number of years back.

Re: Who needs MLflow when you have SQLite?

#95

Post by BeefWellington »

whatever1 wrote: post #16
chaps wrote: post #13

Earlier quoted context omitted.

"the API makes Pandas look good" It sparks joy in my heart whenever I see shade cast against pandas.

I have never seen a worse documented library. Initially I thought that they were lazy, now I realize that it cannot be documented because it is a total mess of a library held together with tape. Close second is the plotly library.

I've long suspected that Pandas has taken a similar stance to e-mail scammers. Where e-mail scammers inject all kinds of broken english and bad punctuation to ensure they get their targets of choice, Pandas has broken and often inaccurate documentation in order to get only the chosen ones to work with their software.

However, maybe it makes more sense that it's just a mess that's hard to document.

Re: Who needs MLflow when you have SQLite?

#96

Post by GeneralMayhem »

LeanderK wrote: post #18

I think MLflow is a good idea (very) badly executed. I would like to have a library that combines: - simple logging of (simple) metrics during and after training - simple logging of all arguments the model was created with - simple logging of a textual representation of the model - simple logging of general architecture details (number of parameters, regularisation hyperparameters, learning rate, number of epochs etc…

Yep - totally agree. I respect the attempt to introduce something that is basically an opinionated CRUD app as a central place to put your model metadata. But it's not really ready for large-scale production, or for use by teams bigger than about 5.

It's kind of flaky and slow. It doesn't have namespacing. It's overly opinionated on the workflow (the way that states work, with a model version being in exactly one of dev, staging, prod is super hard to work with).

But beyond that, the biggest problem I have with MLFlow is what I call the "part of this complete breakfast" problem, which the ML/data-science arena is particularly susceptible to these days: the marketing talks a lot about what problems can be solved using the product, but not a lot about what parts of the problem the product actually solves. This is often because an honest answer to the latter question would be "not much". In the case of MLFlow, that would be totally fine, because honestly an opinionated CRUD app is a very useful thing. But it should be a lot more honest about what it does. It's not a system for automatically tracking model metrics, it's a database into which you can write model metrics with a known key structure.

Re: Who needs MLflow when you have SQLite?

#97

Post by ralphc »

edublancas wrote: post #9
guangyeu wrote: post #6

Could you provide context on why SQLite would replace MLflow? From the standpoint of model tracking (record and query experiments), projects (package code for reproducibility on any platform), deploy models in multiple environments, registry for storing and managing models, and now recipes (to simplify model creation and deployment), MLflow helps with the MLOps life cycle.

Fair point. MLflow has a lot of features to cover the end-to-end dev cycle. This SQLite tracker only covers the experiment tracking part. We have another project to cover the orchestration/pipelines aspect: https://github.com/ploomber/ploomber and we have plans to work on the rest of features. For now, we're focusing on those two.

Have you looked into duckdb for the database? I'm hearing for some tasks it's faster than SQLite.

Re: Who needs MLflow when you have SQLite?

#98

Post by anitil »

alexpotato wrote: post #83
bqmjjx0kac wrote: post #81

Earlier quoted context omitted.

If I were looking for bursts, SQL is not the first thing that comes to mind! Could you elaborate on this or sketch out the query?

Basically, doing a group by at millisecond resolution with a sum on the IP packet length to get a rough metric for bandwidth. Once you have that, you can see the milliseconds with the highest bandwidth. Some extra math can also get you to Gigabits/second in a more network engineer friendly format.

I did a histogram-type thing in the same way by using a window function (similarly sqlite table scraped off pcap recordings). I can't remember if it was a fixed-width window (number of samples) or within some time window

Dropped it in datasette with datasette-vega and got a nice little plot

Re: Who needs MLflow when you have SQLite?

#99

Post by kumare3 »

tomrod wrote: post #60
nerdponx wrote: post #43

Earlier quoted context omitted.

Isn't Comet a proprietary SaaS? I like MLFlow because I can run it on my own computer if I want to.

Check out flyte and union.ml. No personal affiliation, just good projects in the vein of airflow/prefect/mlflow/kubeflow

@tomrod, thank you for the callout. By the way we are integrating mlflow into Flyte in a way that you do not need to start the web server to view the logs. They are available Locally and statically in Flyte Ui. Ofcourse you cal also use mlflow server
Post reply on HN