Live data from Hacker News

AutoML-Zero: Evolving machine learning algorithms from scratch

github.com

21–30 of 47 posts

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#21
post #2

Interesting, but how does it perform on standard benchmarks like image net and MNIST?

I am way out of my depth so maybe this is pure nonsense, but presumably the goals aren't so much to evolve better performing models directly for some dataset specifically, but to see what kinds of model families it evolves and whether we've thought of all of them?

Once we've got one we can then presumably train specific models in a more targeted way.

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#22

Earlier quoted context omitted.

Is this an argument in favor of unjustified magic constant arbitrary priors?

A sufficiently large amount of random data contains all the magic constants you could want.

Yeah, but cryptographic hashes have some entropy.

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#23

Earlier quoted context omitted.

> Would be funny but most of those things are already on AutoML Tables, including the carbon offset GCP datacenters are 100% offset with PPAs. Are you referring to different functionality for costing AutoML instructions in terms of carbon? ... I'd add: - Setup a Jupyter Notebook environment > Jupyter Notebooks are one of the most popular development tools for data scientists. They enable you to create interactive, sh…

This sounds like the sort of thing that would be useful outside of data science. Which leads to the question of whether it needs to be generalized, or redone differently for different specializations. Which in turn seems like the sort of question that it's tricky to answer with AI.

> This sounds like the sort of thing that would be useful outside of data science.

The instruction/operation costing or the computational essay/notebook environment setup?

Ethereum ("gas") and EOS have per-instruction costing. SingularityNET is a marketplace for AI solutions hosted on a blockchain, where you pay for AI/ML services with the SingularityNET AGI token. E.g. GridCoin and CureCoin compensate compute resource donations with their own tokens; which also have a floating exchange rate.

TLJH: "The Littlest JupyterHub" describes how to setup multi-user JupyterHub with e.g. Docker spawners that isolate workloads running with shared resources like GPUs and TPUs: http://tljh.jupyter.org/en/latest/

"Zero to BinderHub" describes how to setup BinderHub on a k8s cluster: https://binderhub.readthedocs.io/en/latest/zero-to-binderhub...

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#25

Earlier quoted context omitted.

This sounds like the sort of thing that would be useful outside of data science. Which leads to the question of whether it needs to be generalized, or redone differently for different specializations. Which in turn seems like the sort of question that it's tricky to answer with AI.

> This sounds like the sort of thing that would be useful outside of data science. The instruction/operation costing or the computational essay/notebook environment setup? Ethereum ("gas") and EOS have per-instruction costing. SingularityNET is a marketplace for AI solutions hosted on a blockchain, where you pay for AI/ML services with the SingularityNET AGI token. E.g. GridCoin and CureCoin compensate compute resour…

The notebook/procedure thing. Like, doesn't everybody everywhere operate on a basis of mixed manual/automated procedures, where it needs to fluidly transition from one to another, yet be controlled and recorded and verified and structured?

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#26
post #7

AutoML-Zero aims to automatically discover computer programs that can solve machine learning tasks, starting from empty or random programs and using only basic math operations. If this system is not using human bias, who is it choosing what good program is? Surely, human labeling data involves humans adding their bias to the data? It seems like AlphaGoZero was able to do just end-to-end ML because it was able to use…

> It seems like AlphaGoZero was able to do just end-to-end ML because it was able to use a very clear and "objective" standard, whether a program wins or loses at the game of Go. Just a fun note: winning or losing at the game of Go is actually surprisingly subjective: https://en.wikipedia.org/wiki/Go_(game)#Scoring_rules

Different scoring rules agree on the winner in >99% of cases

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#27

Earlier quoted context omitted.

> This sounds like the sort of thing that would be useful outside of data science. The instruction/operation costing or the computational essay/notebook environment setup? Ethereum ("gas") and EOS have per-instruction costing. SingularityNET is a marketplace for AI solutions hosted on a blockchain, where you pay for AI/ML services with the SingularityNET AGI token. E.g. GridCoin and CureCoin compensate compute resour…

The notebook/procedure thing. Like, doesn't everybody everywhere operate on a basis of mixed manual/automated procedures, where it needs to fluidly transition from one to another, yet be controlled and recorded and verified and structured?

REES is one solution to reproducibility of the computational environment.

> BinderHub ( https://mybinder.org/ ) creates docker containers from {git repos, Zenodo, FigShare,} and launches them in free cloud instances also running JupyterLab by building containers with repo2docker (with REES (Reproducible Execution Environment Specification)). This means that all I have to do is add an environment.yml to my git repo in order to get Binder support so that people can just click on the badge in the README to launch JupyterLab with all of the dependencies installed.

> REES supports a number of dependency specifications: requirements.txt, Pipfile.lock, environment.yml, aptSources, postBuild. With an environment.yml, I can install the necessary CPython/PyPy version and everything else.

REES: https://repo2docker.readthedocs.io/en/latest/specification.h...

REES configuration files: https://repo2docker.readthedocs.io/en/latest/config_files.ht...

Storing a container built with repo2docker in a container registry is one way to increase the likelihood that it'll be possible to run the same analysis pipeline with the same data and get the same results years later.

...

Pachyderm ( https://pachyderm.io/platform/ ) does Data Versioning, Data Pipelines (with commands that each run in a container), and Data Lineage (~ "data provenance"). What other platforms are there for versioning data and recording data provenance?

...

Recording manual procedures is an area where we've somewhat departed from the "write in a lab notebook with a pen" practice. CoCalc records all (collaborative) inputs to the notebook with a timeslider for review.

In practice, people use notebooks for displaying generated charts, manual exploratory analyses (which does introduce bias), for demonstrating APIs, and for teaching.

Is JupyterLab an ideal IDE? Nope, not by a longshot. nbdev makes it easier to write a function in a notebook, sync it to a module, edit it with a more complete data-science IDE (like RStudio, VSCode, Spyder, etc), and then copy it back into the notebook. https://github.com/fastai/nbdev

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#29

This reminds me of John Koza’s Genetic Programming, a technique for evolving small programs. There is an old Common Lisp library to play with it.

My reaction too. They've reinvented genetic/evolutionary programming. They should probably read some of the decades of work that have already been done on it.

Re: AutoML-Zero: Evolving machine learning algorithms from scratch

#30

This reminds me of John Koza’s Genetic Programming, a technique for evolving small programs. There is an old Common Lisp library to play with it.

My reaction too. They've reinvented genetic/evolutionary programming. They should probably read some of the decades of work that have already been done on it.

Same here. When I studied genetic programming, I was hoping that's where problem solving evolve from as it was flawless. But recent events prove otherwise which made me believe we are using the wrong tool for the wrong problem. Here is why.

When AI gets to 100% accuracy, the equation to find the answer becomes 100% accurate. We no longer have to run the AI with heavy resources and equation can be converted to an executable program. This modal of AI will save computing power, and uses resources smartly.

Example.

AI tries to find right equation to add two numbers.

AI finds the equation to add two numbers.

AI outputs the equation as an executable program.

AI discards itself.

Post reply on HN