Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advancements in machine learning for machine learning

blog.research.google

Post reply on HN
Print view View original

141–150 of 162 posts

Re: Advancements in machine learning for machine learning

#141

Post by creer »

ChatGTP wrote: post #97
creer wrote: post #58

Earlier quoted context omitted.

Exactly so. The posts have been moved safely waayyyy over there at AGI, and at "super-human" or at "critical thinking". And several stages or degrees of AGI have been hierarchized. There is a serious reluctance at accepting how dumb an algorithm can be and still compare with humans. But it is also true that numerous ground techniques are issue of the field of AI and generally called AI as they come out. It makes for…

On the other hand, we've passed the turing test, what's changed?

You mean practically? What did this achieve? What did we gain now that the turing test is passed?

For actually already done: Actually believable chat-bots? Summarizers and question answerers? Generative text and graphics actually usable for generation of text, graphics and (mostly) photo-realistic renderings? Architecture brainstorming? (And logos, etc.) Kinda working self-driving cars? New Go playing strategies? A super-human Go champion? AI is on a roll these days.

That's not counting the more proprietary and discreet applications being already used all over the place. I fully expect there are already several.

Re: Advancements in machine learning for machine learning

#142

Post by pixl97 »

euos wrote: post #26
greatpostman wrote: post #6

Earlier quoted context omitted.

I really don’t people will be programming like we do today in five years

I’ve been programming since middle school. That would be 30 years. Nothing really changed much. C++ is incrementally more convenient but fundamentally the same. Code editors are same. Debugger are same. Shell is same. I am certain in 30 years everything will still be the same.

Other than 30 years ago you were writing a whole shitload more buffer/integer overflows. Hell, that's why we've written numerous languages since that point to ensure it's a hell of a lot harder to footgun yourself.

If coding hasn't change much in 30 years, it may mean you have not changed much in 30 years.

Re: Advancements in machine learning for machine learning

#143

Post by NavinF »

staunton wrote: post #123
Palmik wrote: post #120

Earlier quoted context omitted.

This is a naive take. How do you think Google collects or collected data for their safe-search classifiers? Now that's a sludge. Or how do you think Google evaluates search-ranking changes (or gather data for training various ad-ranking & search-ranking models).

I don't know. How do they?

Their instructions for human raters is public info.

Overview: https://blog.google/products/search/overview-our-rater-guide...

Full PDF: https://static.googleusercontent.com/media/guidelines.raterh...

Re: Advancements in machine learning for machine learning

#144

Post by pixl97 »

rybosome wrote: post #86
fjkdlsjflkds wrote: post #34

Earlier quoted context omitted.

Just because you can have a robot/machine that can efficiently churn out 1000 frozen lasagnas a second doesn't necessarily mean that italian restaurants have been "outcompeted" or "left behind" by not using such a machine in their business. Sometimes quality and responsibility matter. Even if a machine is really good at producing bug-free code, often someone is going to have to read/understand the code that the machi…

Ok, but to continue this analogy, industrialization and the ability to create 1000 frozen lasagnas a second had an enormous impact on the world. Not only on the economics of production, but ultimately on human society. Sure, handmade lasagna still exists, but the world looks nothing like it did 200 years ago.

Heh, most cooking these days is like using libraries to build an application.

I don't slaughter an animal, I buy a cut of meat.

It's very rare I make pasta, I rehydrate dried husks.

The cheese comes in some kind of jar or package. The vegetables come from a store.

This has been the general move in applications too. I see companies with very large programs that are just huge sets of node modules joined together with a small amount of code.

Re: Advancements in machine learning for machine learning

#145

Post by ben_w »

djmips wrote: post #100
ben_w wrote: post #74

Earlier quoted context omitted.

We can't all run a YouTube channel for the programming equivalent of Primitive Technology, fun though that would be. 99.99% of us will have to adapt to AI being a coworker, who will probably eventually replace us. Right now we're still OK because the AI isn't good enough; when it gets good enough, doing things manually is as economically sensible as making your own iron by gathering a few times your mass in wood, bur…

Nice analogy! I saw an estimate recently on the cost of programming and they predicted that automated coding will cost 10,000 times less than human coders. It was all back of the envelope and questionable but still it was food for thought. Will we be 10,000 times more productive or will we be out of work? I think a lot of people will be out of work.

Thanks! :)

> Will we be 10,000 times more productive or will we be out of work? I think a lot of people will be out of work.

It can be both. Automation of farming means we've gone from a constant risk of starvation to an epidemic of obesity, while simultaneously reducing the percentage of the workforce in agriculture.

Re: Advancements in machine learning for machine learning

#146

Post by dmarchand90 »

jwuphysics wrote: post #119
dmarchand90 wrote: post #101

Earlier quoted context omitted.

I kinda wonder if maybe it's at least partially due to openai hitting a kind of hyperparameter lottery. When each experiment costs millions it might be that (aside from good/ unique data) they just have a good set of hyperparameters used in training and it's too expensive for a competitor to find equal or better settings

I would be surprised if this is the case. Neural scaling laws are well known and are used by all big industry players to extrapolate experiments.

Are they really "laws" my impression is its all just a bunch of empirical trends.

We cannot know truly how these parameters interact at large scale and also how they interact with each other.

Is it really the case that openai has data that Google doesn't?

Re: Advancements in machine learning for machine learning

#147

Post by AnthonyMouse »

zer00eyz wrote: post #57
AnthonyMouse wrote: post #49

Earlier quoted context omitted.

> In order for an AI to evaluate the effect of a small molecule on the brain, it would have to... simulate the operation of a human brain in a simulated environment. Similarly, to avoid Thalidomide-style disasters, it would have to simulate the conception, development and growth to adulthood of a human. This is how the human doctors who have cured things in the past have done it, is it? The way this is going to work,…

>> The way this is going to work... Google is already doing something like this: https://arstechnica.com/ai/2023/11/googles-deepmind-finds-2-... This is great if you want to use well understood pathways or make new drugs that you can then patent and mark up. New pathways are gonna require feeding data into these models in the first place. Your not getting ozempic out of ML without doing the ground work first: https:/…

> New pathways are gonna require feeding data into these models in the first place. Your not getting ozempic out of ML without doing the ground work first

Sure, but a lot of the ground work has already been done, or is susceptible to simulation. They're getting a lot of results out of simulating protein folding and things like that.

Re: Advancements in machine learning for machine learning

#148

Post by valine »

euos wrote: post #117
valine wrote: post #35

Earlier quoted context omitted.

The way I write code was fundamentally altered in the last year by GPT4 and copilot. Try having GPT4 write your code, you won’t be so certain about the future of programming afterward I guarantee it.

I have free Copilot due to my OSS work. This week I disabled it for C++ because it is chronically incapable to match brackets. I was wasting too much time fixing the messes. I use it for TypeScript/React. But it’s just a more comprehensive code complete. Incremental.

Uh huh, try GPT4 and report back. It’s a generational leap above copilot. I use copilot to auto complete one liners and GPT4 to generate whole methods.

Re: Advancements in machine learning for machine learning

#149

Post by hkmaxpro »

owlbite wrote: post #135
stabbles wrote: post #112

Earlier quoted context omitted.

Take a look at the chess engine Stockfish: they tossed out years and years of human written heuristics in board evaluation, to a small neural net that does the same but better. Now consider all the heuristics for inlining, loop unrolling, vectorization etc in compilers, certainly a neural net can be beneficial and possibly easier to maintain than tons of human written heuristics.

We'll have to see. I could definitely see someone spending a lot of time training for a specific algorithmic kernel and microarchitecture and beating the best human results (by a few percent). I'd be very surprised if that can be extended to a large complex algorithmic system that is amenable to mathematical reformulations (at least within the next 10 years).

Another bitter lesson? http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Re: Advancements in machine learning for machine learning

#150

Post by ChatGTP »

creer wrote: post #141
ChatGTP wrote: post #97

Earlier quoted context omitted.

On the other hand, we've passed the turing test, what's changed?

You mean practically? What did this achieve? What did we gain now that the turing test is passed? For actually already done: Actually believable chat-bots? Summarizers and question answerers? Generative text and graphics actually usable for generation of text, graphics and (mostly) photo-realistic renderings? Architecture brainstorming? (And logos, etc.) Kinda working self-driving cars? New Go playing strategies? A s…

It would be cool if / when we see any of these advancements solve some real world important problems. Realistic chatbots? who cares?
Post reply on HN