Show HN: Curated, Pre-trained ML Models for Transfer Learning • Hacker News Board

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#21

Post by minimaxir » Tue, Feb 13, 2018, 8:54 PM UTC

rm999 wrote: post #20

minimaxir wrote: post #11

Two of my projects, textgenrnn ( https://github.com/minimaxir/textgenrnn ) and reactionrnn ( https://github.com/minimaxir/reactionrnn ) are among the pretrained models. While this is allowed by the MIT License (and there is sufficient attribution to the source repos), it might be helpful to more explicitly state that the curated models are forks/modifications. These models also have a dependency on the source package…

Hey, I just installed and played around with reactionRNN; I hate to be negative (especially to people open sourcing models, kudos!), but your model seems to perform quite poorly. It immediately failed my "easy" smell tests: https://i.imgur.com/FvfuZgy.png , and didn't really work on most of my other tests: "This book sucks" is 0% angry, "I'm going to go home and listen to emo music and cry" is 0% sad, "Check out this…

The result is the reaction to a given text; it won't fully be the same as a sentiment analysis, unfortunately.

Additionally, as I put in the README notes, keep in mind that the network is trained on modern (2016-2017) language. As a result, inputting rhetorical/ironic statements will often yield love/wow responses and not sad/angry. That type of systemic bias in text analysis is currently unsolved and there isn't an easy way to account for it.

"I am so angry" and "I'm going to go home and listen to emo music and cry" are phrases that would likely be posted on Facebook ironically, and therefore classifying it tricky.

For the other examples, yes, that result might be overfitting on characters and while setting up the model I had difficulty accounting for that while still getting the model to converge.

I'll admit it's not a perfect model (it was a side project while I was frustrated during a job hunt), but it's a great proof of concept. Unfortunately, since Facebook crippled their /posts endpoint, I can't get more data to improve the model...

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#22

Post by mikeshi42 » Tue, Feb 13, 2018, 8:56 PM UTC

DOsinga wrote: post #19

This is great, I like the inclusion of the notebooks, makes it easy to get started. The download link for the Image Colorization model doesn't seem to work for me though - it gives an Access Denied error.

Hey! Sorry, slight typo on our end, the notebook should still work fine, but the manual download didn't work for a bit.

It's fixed now though! Thanks for the heads up and I hope you've found it useful! Happy to hear any other feedback as well :)

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#23

Post by edraferi » Tue, Feb 13, 2018, 9:26 PM UTC

orliesaurus wrote: post #9

Thanks for submitting! Can you tell me how this differs from something like https://algorithmia.com/ ? I am curious to understand the main differences between their approach and yours! Cheers :)

Algorithmia is a FaaS platform with strong service discovery and a lot of AI functions. Model Depot is a listing of open-source models.

You could find a network on Model Depot and deploy it on Algorithmia. Then you (And others, if you like) can use the model via RPC using HTTP calls or one of the Algorithmia client libraries.

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#24

Post by MayeulC » Tue, Feb 13, 2018, 9:33 PM UTC

mikeshi42 wrote: post #17

badminton1 wrote: post #16

One day there will be a trained model called "software engineer" and we will be out of a job. The training data may come from github, extracting requirements from issues, and mapping it to the code that closed the ticket.

But then we can finally spend _all_ of our time on HN! But we really do think ML, even in the near future, will automate some of our jobs away (and that's a good thing!), if you're interested in some of our thoughts on that: https://medium.com/modeldepot/we-previously-talked-about-how...

I was about to answer roughly the same.

If someone could step up and "take my job", it would free me up for other things. I would be grad to be able to rely on a good "software engineer" model to build the next generation tooling.

All in all, more performance, more efficiency, improving our tools bottom-up, going for the low hanging fruit first, to free us from the less interesting tasks. Isn't it what we always do and always did?

If a task is repetitive or deterministic, I always feel like I could be spending my time on something more interesting, not on the implementation details. The problem is that I have to take care of it myself, if I want it to be done properly (and that's true as well in other fields than software, of course).

Disclaimer: still a student (although nearly graduated). I won't bite you for having more experience than I do :) And the phrasing here might be a little provocative on purpose, don't take it too hard please.

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#25

Post by hsikka » Tue, Feb 13, 2018, 9:38 PM UTC

MayeulC wrote: post #24

mikeshi42 wrote: post #17

Earlier quoted context omitted.

But then we can finally spend _all_ of our time on HN! But we really do think ML, even in the near future, will automate some of our jobs away (and that's a good thing!), if you're interested in some of our thoughts on that: https://medium.com/modeldepot/we-previously-talked-about-how...

I was about to answer roughly the same. If someone could step up and "take my job", it would free me up for other things. I would be grad to be able to rely on a good "software engineer" model to build the next generation tooling. All in all, more performance, more efficiency, improving our tools bottom-up, going for the low hanging fruit first, to free us from the less interesting tasks. Isn't it what we always do a…

Well said!

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#26

Post by DOsinga » Tue, Feb 13, 2018, 10:05 PM UTC

mikeshi42 wrote: post #22

DOsinga wrote: post #19

This is great, I like the inclusion of the notebooks, makes it easy to get started. The download link for the Image Colorization model doesn't seem to work for me though - it gives an Access Denied error.

Hey! Sorry, slight typo on our end, the notebook should still work fine, but the manual download didn't work for a bit. It's fixed now though! Thanks for the heads up and I hope you've found it useful! Happy to hear any other feedback as well :)

Yep, works now. Thanks!

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#27

Post by [deleted] » Wed, Feb 14, 2018, 1:46 AM UTC

MayeulC wrote: post #24

mikeshi42 wrote: post #17

Earlier quoted context omitted.

But then we can finally spend _all_ of our time on HN! But we really do think ML, even in the near future, will automate some of our jobs away (and that's a good thing!), if you're interested in some of our thoughts on that: https://medium.com/modeldepot/we-previously-talked-about-how...

I was about to answer roughly the same. If someone could step up and "take my job", it would free me up for other things. I would be grad to be able to rely on a good "software engineer" model to build the next generation tooling. All in all, more performance, more efficiency, improving our tools bottom-up, going for the low hanging fruit first, to free us from the less interesting tasks. Isn't it what we always do a…

[deleted]

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#28

Post by daniellerommel » Wed, Feb 14, 2018, 3:29 AM UTC

For those using Wolfram Language or Mathematica, there is an unreleased Neural Net Repository that is worth checking out. A few examples:

Caffe: https://resources.wolframcloud.com/NeuralNetRepository/resou...

Keras: https://resources.wolframcloud.com/NeuralNetRepository/resou...

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#29

Post by andreyk » Wed, Feb 14, 2018, 5:43 AM UTC

Definitely a cool project. It's crazy there is no standard model zoo anywhere yet. But, a few thoughts:

-No about page? Not clear at all without cleaning on sign up button that model submission/voting is a thing. And many more details I'd be curious to know seeing this for the first time.

-A few things, like colorization, loaded reallllyyy slowly

-No search/stronger filtering?

-Ideally various project should have weights in ONNX standard, and there should be tabs/selection of different platforms. You'd have five different entries for implementatations of ResNets in each of the major platforms? Quite silly.

-This seems like it should be an open source project , given that it is submission/open source driven

-Although this is aesthetically pleasing, at the end of the day it's just some links to cool Github repos... I feel like it'd make a lot more sense to create this as a CodaLab worksheet (https://worksheets.codalab.org/worksheets/0x818930127c4d47de...) ; the platform there is just more mature for this sort of thing. Or make this a fork of that with the whole Zoo angle.

Re: Show HN: Curated, Pre-trained ML Models for Transfer Learning

#30

Post by badminton1 » Thu, Feb 15, 2018, 1:26 PM UTC

MayeulC wrote: post #24

mikeshi42 wrote: post #17

Earlier quoted context omitted.

But then we can finally spend _all_ of our time on HN! But we really do think ML, even in the near future, will automate some of our jobs away (and that's a good thing!), if you're interested in some of our thoughts on that: https://medium.com/modeldepot/we-previously-talked-about-how...

I was about to answer roughly the same. If someone could step up and "take my job", it would free me up for other things. I would be grad to be able to rely on a good "software engineer" model to build the next generation tooling. All in all, more performance, more efficiency, improving our tools bottom-up, going for the low hanging fruit first, to free us from the less interesting tasks. Isn't it what we always do a…

I strongly disagree.

First, you assume that AI will remain less proficient than humans, and therefore suitable only for grunt work. Artificial intelligence has many advantages over us, and this is often overlooked.

- How much time does it take to train one software engineer? decades worth of time. Once you have one, what does it get another? about the same amount of time and money. In contrast, once you train one AI based engineer, the time to get the second one is about the time of serializing the state of one and deploying it multiple times. You could recruit and discard AI engineers on demand for whatever you need.

- They won't sleep, take breaks for lunch or coffee, get distracted, waste time talking about their car, vacations, wine tasting and other stupid things. They will just work. 24 hours a day. No breaks.

- They won't consume information at a rate of 200 to 1000 words per minute, they won't be limited by their visual field, or how fast can they type. They may not even communicate using words, just transfer thoughts directly among them, or even entire trained neural ensembles. Plus, once they learn something that may affect they collectively rather than individually.

- They will be aggressively supervised, ranked. Then, since their generations are shorter, this will cause them to continuously improve at a much faster pace than humans.

If this happens, the need for having the same amount of human engineers will be lower if not zero.

Some people may argue that the human brain has billions of neurons and synaptic connections. Sure. But which portion of that is actively used in developing software? remove motor functions, autonomous functions like regulating heart rate, respiration and such and you are left with much less.