Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrew Ng: Unbiggen AI

spectrum.ieee.org

Post reply on HN
Print view View original

31–40 of 90 posts

Re: Andrew Ng: Unbiggen AI

#31

Post by NumberCruncher »

overkalix wrote: post #14
whatever1 wrote: post #5

My understanding is that they are trying to automate the data preparation steps that seasoned ML practitioners are doing anyway today. The fact that he tries this in manufacturing makes the case stronger. In most manufacturing companies you do not have access to top ML talent. You have Greg who knows python and recently visualized some production metrics. If we could empower Greg with automated ML libraries that guid…

Greg probably also knows SAS and AMPL, and has a good knowledge of ops research, which is within stone-tossing distance of whatever ML is pretending to be this week.

After 15 years of experience with SAS this sounds to me like saying "knowing how to write and having a pen makes you to a poet". But it depends on how far you can toss a stone...

Re: Andrew Ng: Unbiggen AI

#32

Post by notsag-hn »

I was going to interview at LandingAI. I was asked before the interview to install a spyware browser extension to monitor my traffic to detect if I was cheating during the interview. I respectfully declined and didn't have that interview.

Re: Andrew Ng: Unbiggen AI

#33

Post by pbowyer »

frozenport wrote: post #25
DeathArrow wrote: post #20

Pretty interesting. Mr. Ng claims that for some applications having a small set of quality data can be as good as using huge set of noisy data. I wonder if, assuming the data is of highest quality, with minimal noise, having more data will matter for training or not. And if it matters, on what degree?

This is at the heart of the ML training problem. In general you want to add more variants of data but not so much that the network doesn't get trained by them. Typical practice is to find images whose inclusion causes high variation in final accuracy (under k-fold validation, aka removing/adding the image causes a big difference) and prefer more of those. Now, why not simply add everything? Well in general it takes t…

> Typical practice is to find images whose inclusion causes high variation in final accuracy (under k-fold validation, aka removing/adding the image causes a big difference)

How do you identify these images? It sounds like I'd need to build small models to see the variance but I'm hoping that there's a more scientific way?

Re: Andrew Ng: Unbiggen AI

#35

Post by leobg »

What are some ML data annotation tools that guide you towards those data points where the model gets confused? I hear Prodigy does this. Any others?

Re: Andrew Ng: Unbiggen AI

#36

Post by teruakohatu »

extheat wrote: post #30
teruakohatu wrote: post #11

Earlier quoted context omitted.

That is called transfer learning. You might only need to see one photo of a sign to identify it in real life (although arguably learner drivers take a while to notice signs) but that is only because you have been training on identifying generic objects since you left the womb. You brain already knows how to select the most important features of a sign. The shape, the size and the color. You have also learned how to u…

I'm not sure "training" is the right way to think about it. Children don't train to identify objects, they quickly develop the ability to recognize objects and are able to correlate them with prior information that was retained and learned. Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them.

> Children don't train to identify objects

I have to disagree. They spend an inordinate amount of time trying to understand what they see, taste and hear.

> able to correlate them with prior information that was retained and learned

This is what we call inference.

> Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them.

Not everyone who is legally blind can see absolutely nothing, but people who have recovered from complete vision loss [1] have problems. Mike May [2] lost vision as 3 year old child and regained it in his 40s. Despite seeing for the first three years of his life, years after regaining vision he was unable to see in 3D or recognize people from faces alone.

Blind people do not lack spatial awareness, so being able to recognize objects with context if they regained sight with would not surprise me. There are blind people that can "see" with echo location using parts of the brain associated with visual processing [3] But for example in Mike's case, he was unable to recognize close family by their faces years after regaining vision, he needed additional context.

Many things we take for granted as being innate to the human experience, are in fact learned (trained) behavior.

[1] https://en.wikipedia.org/wiki/Recovery_from_blindness

[2] https://en.wikipedia.org/wiki/Mike_May_(skier)

[3] https://en.wikipedia.org/wiki/Human_echolocation

Re: Andrew Ng: Unbiggen AI

#37

Post by [deleted] »

onethought wrote: post #23
vasco wrote: post #7

Earlier quoted context omitted.

Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.

when you learn to drive you need [approximately 16 years training your visual system hardware that took 6 million years to evolve and] to see approximately one example and you're able to identify them all. FTFY. Yet Tesla have been working on both the hardware and software for 10 years? Amazing progress right?

[deleted]

Re: Andrew Ng: Unbiggen AI

#38

Post by spupe »

atbpaca wrote: post #29

Glad to see the term ML being used more often than AI in the comments as it looks like most "AI" models are trained for image classification. Having said that, the idea of "doing more with less" sounds interesting and I wonder what it means exactly. Does it mean taking a dataset of 50 images and to create 1000s of synthetic images from it?

Yeah I was very interested about that point in particular. I think synthetic data is one of the ideas, but I got the sense that he also means helping to identify what makes a data set good, even if small. It looks like Andrew Ng is developing a platform for automatically detecting whether a dataset is suitable and, if not, what are the steps to improve it. A sort of automated ML consultant, allowing you to sell capabilities much cheaper than if you needed to consult an actual expert.

Re: Andrew Ng: Unbiggen AI

#39

Post by kevsim »

notsag-hn wrote: post #32

I was going to interview at LandingAI. I was asked before the interview to install a spyware browser extension to monitor my traffic to detect if I was cheating during the interview. I respectfully declined and didn't have that interview.

Wow if you can “cheat” during an interview - meaning either that they’re asking trivial, google-able stuff or that they’re so bad at interviewing that they can’t tell if you actually know your stuff - then their hiring process is pretty bad.

Re: Andrew Ng: Unbiggen AI

#40

Post by [deleted] »

riku_iki wrote: post #13
Datenstrom wrote: post #4

Earlier quoted context omitted.

> simply saying "we should make this better thing" isn't enough. Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same resul…

> it is possible to smartly choose the correct few samples for the network that yield the same results. And then, someone is using pretrained 500B model, and fine-tuning your few examples, and getting new SOTA.

[deleted]
Post reply on HN