Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Vizier: A Service for Black-Box Optimization

ai.google

Post reply on HN
Print view View original

51–60 of 86 posts

Re: Google Vizier: A Service for Black-Box Optimization

#51

Post by Itsdijital »

dekhn wrote: post #16
kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

It's great that they published this - I wasn't sure if I could share this externally. I see they left the ingredient that really improved the scores unexpectedly out, presumbly for proprietary reasons.

They most likely are holding it back for a bigger media event where they will show them off and every tech blog on the internet will run a story about them.

Re: Google Vizier: A Service for Black-Box Optimization

#53

Post by dallen33 »

kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

Seems to be missing the key ingredient of the best tasting chocolate chip cookies: espresso powder.

Re: Google Vizier: A Service for Black-Box Optimization

#55

Post by abhgh »

minimaxir wrote: post #21
usefulcat wrote: post #15

Earlier quoted context omitted.

I know a guy who told me 8 years ago about an idea he had for a "black box" optimization consulting service. You give him a pile of data, he uses ML to optimize for whatever.

That's known nowadays as AutoML. Google recently released AutoML products for image and text classification. https://cloud.google.com/automl/

FYI there is also a different "automl" suite of libraries maintained by Frank Hutters lab at the University of Freiburg - [1]. His group has been leading a lot of research in the area of hyperparameter tuning.

[1] https://www.automl.org/

Re: Google Vizier: A Service for Black-Box Optimization

#57

Post by levesque »

indiesolver wrote: post #28
btown wrote: post #24

Whetlab, a startup founded by some former colleagues of mine, provided a service just like this 4 years ago (in fact, it's referenced in the Vizier paper as its open-source variant Spearmint), but unfortunately, it was acquired and shut down by Twitter: https://venturebeat.com/2015/06/17/twitter-acquires-machine-... Black-box optimization is a hugely important problem to solve, especially when experiments require rea…

Whetlab was the first startup in this domain. Then, followed by SigOpt ( https://sigopt.com/ ) which is doing a great job at popularizing the concept.

Yeah SigOpt clearly has taken the space left by Whetlab. They must not be too pleased about this announcement, I mean generally the worst thing that can happen to your startup is a tech giant launching in your space. Then again the problem of hyperparameter/black-box optimization is so ubiquitous that there should be enough space for them both.

Re: Google Vizier: A Service for Black-Box Optimization

#58

Post by oh-kumudo »

meinstream wrote: post #26
kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

I worked for Google at the time they went through the cookie iterations in the cafes. Personally I disliked all the cookies that ever came out of this project, mainly because they contained spices, I heard the same thing from many others. After you got handed a cookie you were asked to fill out a feedback form on a tablet, which almost always is annoying. But more importantly, the process of optimizing the ingredient…

I think this is exactly the case how ML could go wrong. I would assume the people who hate the cookies probably will not bother to submit a survey and try again. The sample might be biased and not representative of the general population, thus overfitting, performs badly on a boarder test set.

Re: Google Vizier: A Service for Black-Box Optimization

#59

Post by joefkelley »

dekhn wrote: post #16
kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

It's great that they published this - I wasn't sure if I could share this externally. I see they left the ingredient that really improved the scores unexpectedly out, presumbly for proprietary reasons.

Are you talking about orange extract? or cayenne pepper? The post is here: https://ai.google/research/pubs/pub46507 and direct link to paper is here: https://storage.googleapis.com/pub-tools-public-publication-...

And the recipe I found internally is "The Real California Cookie" in that paper:

Bake at 163C (325◦ F) until brown:

• 167 grams of all-purpose flour.

• 245 grams of milk chocolate chips.

• 0.60 tsp. baking soda.

• 0.50 tsp. salt.

• 0.125 tsp. cayenne pepper.

• 127 grams of sugar (31% medium brown, 69% white).

• 25.7 grams of egg.

• 81.3 grams of butter.

• 0.12 tsp. orange extract.

• 0.75 tsp. vanilla extract.

Re: Google Vizier: A Service for Black-Box Optimization

#60

Post by eadmund »

joefkelley wrote: post #59
dekhn wrote: post #16

Earlier quoted context omitted.

It's great that they published this - I wasn't sure if I could share this externally. I see they left the ingredient that really improved the scores unexpectedly out, presumbly for proprietary reasons.

Are you talking about orange extract? or cayenne pepper? The post is here: https://ai.google/research/pubs/pub46507 and direct link to paper is here: https://storage.googleapis.com/pub-tools-public-publication-... And the recipe I found internally is "The Real California Cookie" in that paper: Bake at 163C (325◦ F) until brown: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 0.60 tsp. baking…

What an odd combination of units. Here's a translation:

• 5¼ oz. of all-purpose flour.

• 7 oz. of milk chocolate chips.

• scant ½ + ⅛ tsp. baking soda.

• ½ tsp. salt.

• ⅛ tsp. cayenne pepper.

• 4½ oz. of sugar (31% medium brown, 69% white).

• 1 oz. of egg.

• 2¾ oz. of butter.

• ⅛ tsp. orange extract.

• ¾ tsp. vanilla extract.

Or, equivalently:

• 167 grams of all-purpose flour.

• 245 grams of milk chocolate chips.

• 2.96 mL baking soda.

• 2.46 mL salt.

• 0.61 mL cayenne pepper.

• 127 grams of sugar (31% medium brown, 69% white).

• 25.7 grams of egg.

• 81.3 grams of butter.

• 0.59 mL orange extract.

• 3.7 mL vanilla extract.

Post reply on HN