Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Francois Chollet is leaving Google

developers.googleblog.com

Post reply on HN
Print view View original

31–40 of 182 posts

Re: Francois Chollet is leaving Google

#31

Post by osm3000 »

I loved Keras at the beginning of my PhD, 2017. But it was just the wrong abstraction: too easy to start with, too difficult to create custom things (e.g., custom loss function).

I really tried to understand TensorFlow, I managed to make a for-loop in a week. Nested for-loop proved to be impossible.

PyTorch was just perfect out of the box. I don't think I would have finished my PhD in time if it wasn't for PyTorch.

I loved Keras. It was an important milestone, and it made me believe deep learning is feasible. It was just...not the final thing.

Re: Francois Chollet is leaving Google

#32

Post by minimaxir »

perturbation wrote: post #28
minimaxir wrote: post #20

Earlier quoted context omitted.

A few things from personal experience: - LLM support with PyTorch is better (both at a tooling level and CUDA level). Hugging Face transformers does have support for both TensorFlow and PyTorch variants of LLMs but... - Almost all new LLMs are in PyTorch first and may or may not be ported to TensorFlow. This most notably includes embeddings models which are the most important area in my work. - Keras's training loop…

I think a lot of these may have improved since your last experience with Keras. It's pretty easy to override the training loop and/or make custom loss. The below is for overriding training / test step altogether, custom loss is easier by making a new loss function/class. https://keras.io/examples/keras_recipes/trainer_pattern/ > - Keras's training loop assumes you can fit all the data in memory and that the data is f…

That's a fair implementation of custom loss. Hugging Face's Trainer with transformers suggests a similar implementation, although their's has less boilerplate.

https://huggingface.co/docs/transformers/main/en/trainer#cus...

Re: Francois Chollet is leaving Google

#33

Post by modeless »

blackeyeblitzar wrote: post #24
modeless wrote: post #9

Earlier quoted context omitted.

Why would you interpret this as Google disliking Keras? Seems a lot more likely he was poached by Anthropic.

Where did you see that he was poached by Anthropic?

I am not suggesting that I know it for a fact. I do recall some speculation on X to that effect but I can't find it now. Maybe just because Anthropic has been getting a lot of people lately.

Re: Francois Chollet is leaving Google

#34

Post by mxwsn »

trott wrote: post #14
max_ wrote: post #3

I wonder what he will be working on? Maybe he figured out a model that beats ARC-AGI by 85%?

> Maybe he figured out a model that beats ARC-AGI by 85%? People have, I think. One of the published approaches (BARC) uses GPT-4o to generate a lot more training data. The approach is scaling really well so far [1], and whether you expect linear scaling or exponential one [2], the 85% threshold can be reached, using the "transduction" model alone, after generating under 2 million tasks ($20K in OpenAI credits). Perh…

My interest was piqued, but the extrapolation in [1] is uh... not the most convincing. If there were more data points then sure, maybe

Re: Francois Chollet is leaving Google

#35

Post by trott »

mxwsn wrote: post #34
trott wrote: post #14

Earlier quoted context omitted.

> Maybe he figured out a model that beats ARC-AGI by 85%? People have, I think. One of the published approaches (BARC) uses GPT-4o to generate a lot more training data. The approach is scaling really well so far [1], and whether you expect linear scaling or exponential one [2], the 85% threshold can be reached, using the "transduction" model alone, after generating under 2 million tasks ($20K in OpenAI credits). Perh…

My interest was piqued, but the extrapolation in [1] is uh... not the most convincing. If there were more data points then sure, maybe

The plot was just showing where the solid lines were trending (see prior messages), and that happened to predict the performance at 400k samples (red dot) very well.

An exponential scaling curve would steer a bit more to the right, but it would still cross the 85% mark before 2000k.

Re: Francois Chollet is leaving Google

#36

Post by fastball »

TheDudeMan wrote: post #25
fastball wrote: post #22

Earlier quoted context omitted.

I like the idea of ARC-AGI and think it was worth a shot. But if someone has already hit the human-level threshold, I think the entire idea can be thrown out. If the ARC-AGI challenge did not actually follow their expected graph[1], I see no reason to believe that any benchmark can be designed in a way where it cannot be gamed. Rather, it seems that the existing SOTA models just weren't well-optimized for that one ta…

What you're calling "gamed" could actually be research and progress in general problem solving.

Almost by definition it is not. If you are "gaming" a specific benchmark, what you have is not progress in general intelligence. The entire premise of the ARC-AGI challenge was that general problem solving would be required. As noted by the GP, one of the top contenders is BARC which performs well by generating a huge amount of training data for this particular problem. That's not general intelligence, that's gaming.

There is no reason to believe that technique would not work for any particular problem. After all, this problem was the best attempt the (very intelligent) challenge designers could come up with, as evidenced by putting $1m on the line.

Re: Francois Chollet is leaving Google

#37

Post by TechDebtDevin »

trott wrote: post #14
max_ wrote: post #3

I wonder what he will be working on? Maybe he figured out a model that beats ARC-AGI by 85%?

> Maybe he figured out a model that beats ARC-AGI by 85%? People have, I think. One of the published approaches (BARC) uses GPT-4o to generate a lot more training data. The approach is scaling really well so far [1], and whether you expect linear scaling or exponential one [2], the 85% threshold can be reached, using the "transduction" model alone, after generating under 2 million tasks ($20K in OpenAI credits). Perh…

I personally think ARC-AGI will be a forgotten, unimportant benchmark that doesn't indicate anything more than a models ability reason, which honestly is just a very small step in the path towards AGI

Re: Francois Chollet is leaving Google

#38

Post by tbalsam »

bearcollision wrote: post #29

I've always wondered how fchollet had authority to force keras into TF... https://github.com/tensorflow/community/pull/24

I remember this post as the day that Keras died. Very strange political powerplay on the part of fchollet, and did immeasurable damage to the community and code that used TF, not just in that PR but also in the precedent it set for other stuff. People legitimately were upset by the attempt to move tensorflow under an unnecessary Keras namespace, and he locked the PR and said that Reddit was brigading it (despite it being pretty consistently disliked as a change, among other changes). People tried to reason with him in the PR thread, but to no avail, the Keras name had to live on, whether or not TF died with it (and it very well did, unfortunately). There were other things working against TF but this one seemed to be the final nail in the coffin, from what I can tell.

I ended up minimizing engagement with the work he's done since as a result.

Re: Francois Chollet is leaving Google

#39

Post by trott »

fastball wrote: post #36
TheDudeMan wrote: post #25

Earlier quoted context omitted.

What you're calling "gamed" could actually be research and progress in general problem solving.

Almost by definition it is not. If you are "gaming" a specific benchmark, what you have is not progress in general intelligence. The entire premise of the ARC-AGI challenge was that general problem solving would be required. As noted by the GP, one of the top contenders is BARC which performs well by generating a huge amount of training data for this particular problem. That's not general intelligence, that's gaming.…

> That's not general intelligence, that's gaming.

In fairness, their approach is non-trivial. Simply asking GPT-4o to fantasize more examples wouldn't have worked very well. Instead, they have it fantasize inputs and programs, and then run the programs on the inputs to compute the outputs.

I think it's a great contribution (although I'm surprised they didn't try making an even bigger dataset -- perhaps they ran out of time or funding)

Re: Francois Chollet is leaving Google

#40

Post by rd11235 »

osm3000 wrote: post #31

I loved Keras at the beginning of my PhD, 2017. But it was just the wrong abstraction: too easy to start with, too difficult to create custom things (e.g., custom loss function). I really tried to understand TensorFlow, I managed to make a for-loop in a week. Nested for-loop proved to be impossible. PyTorch was just perfect out of the box. I don't think I would have finished my PhD in time if it wasn't for PyTorch. I…

> it was just the wrong abstraction: too easy to start with, too difficult to create custom things

Couldn’t agree with this more. I was working on custom RNN variants at the time, and for that, Keras was handcuffs. Even raw TensorFlow was better for that purpose (which in turn still felt a bit like handcuffs after PyTorch was released).

Post reply on HN