Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Vizier: A Service for Black-Box Optimization

ai.google

Post reply on HN
Print view View original

61–70 of 86 posts

Re: Google Vizier: A Service for Black-Box Optimization

#61

Post by whazor »

oh-kumudo wrote: post #58
meinstream wrote: post #26

Earlier quoted context omitted.

I worked for Google at the time they went through the cookie iterations in the cafes. Personally I disliked all the cookies that ever came out of this project, mainly because they contained spices, I heard the same thing from many others. After you got handed a cookie you were asked to fill out a feedback form on a tablet, which almost always is annoying. But more importantly, the process of optimizing the ingredient…

I think this is exactly the case how ML could go wrong. I would assume the people who hate the cookies probably will not bother to submit a survey and try again. The sample might be biased and not representative of the general population, thus overfitting, performs badly on a boarder test set.

Our dystopian future where we will be forced fed cookies in order for accurate optimization.

Re: Google Vizier: A Service for Black-Box Optimization

#63

Post by sannee »

kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

Is this possibly related to the cookie machine by Ben Krasnow from several years ago? [1]

[1] https://www.youtube.com/watch?v=8YEdHjGMeho

Re: Google Vizier: A Service for Black-Box Optimization

#64

Post by hammock »

eadmund wrote: post #60
joefkelley wrote: post #59

Earlier quoted context omitted.

Are you talking about orange extract? or cayenne pepper? The post is here: https://ai.google/research/pubs/pub46507 and direct link to paper is here: https://storage.googleapis.com/pub-tools-public-publication-... And the recipe I found internally is "The Real California Cookie" in that paper: Bake at 163C (325◦ F) until brown: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 0.60 tsp. baking…

What an odd combination of units. Here's a translation: • 5¼ oz. of all-purpose flour. • 7 oz. of milk chocolate chips. • scant ½ + ⅛ tsp. baking soda. • ½ tsp. salt. • ⅛ tsp. cayenne pepper. • 4½ oz. of sugar (31% medium brown, 69% white). • 1 oz. of egg. • 2¾ oz. of butter. • ⅛ tsp. orange extract. • ¾ tsp. vanilla extract. Or, equivalently: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 2…

Still no good because what the hell is 1oz of egg?

Re: Google Vizier: A Service for Black-Box Optimization

#65

Post by [deleted] »

hammock wrote: post #64
eadmund wrote: post #60

Earlier quoted context omitted.

What an odd combination of units. Here's a translation: • 5¼ oz. of all-purpose flour. • 7 oz. of milk chocolate chips. • scant ½ + ⅛ tsp. baking soda. • ½ tsp. salt. • ⅛ tsp. cayenne pepper. • 4½ oz. of sugar (31% medium brown, 69% white). • 1 oz. of egg. • 2¾ oz. of butter. • ⅛ tsp. orange extract. • ¾ tsp. vanilla extract. Or, equivalently: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 2…

Still no good because what the hell is 1oz of egg?

[deleted]

Re: Google Vizier: A Service for Black-Box Optimization

#66

Post by btown »

levesque wrote: post #57
indiesolver wrote: post #28

Earlier quoted context omitted.

Whetlab was the first startup in this domain. Then, followed by SigOpt ( https://sigopt.com/ ) which is doing a great job at popularizing the concept.

Yeah SigOpt clearly has taken the space left by Whetlab. They must not be too pleased about this announcement, I mean generally the worst thing that can happen to your startup is a tech giant launching in your space. Then again the problem of hyperparameter/black-box optimization is so ubiquitous that there should be enough space for them both.

I wasn't familiar with SigOpt - this is really cool. I actually think SigOpt will see this as great publicity telling people about the usefulness of the concept - and whereas Google won't lift a finger for (insert hedge fund or pharmaceutical giant here) beyond maintaining uptime, SigOpt can provide those customers with customized advice about how to integrate, what pitfalls to watch for, how to design experiments to maximally take advantage of their technology.

And they're competitive with Spearmint (though not necessarily the closed-source versions of it used at Whetlab), though Vizier remains to be seen: https://arxiv.org/pdf/1603.09441.pdf

Re: Google Vizier: A Service for Black-Box Optimization

#67

Post by archgoon »

whazor wrote: post #61
oh-kumudo wrote: post #58

Earlier quoted context omitted.

I think this is exactly the case how ML could go wrong. I would assume the people who hate the cookies probably will not bother to submit a survey and try again. The sample might be biased and not representative of the general population, thus overfitting, performs badly on a boarder test set.

Our dystopian future where we will be forced fed cookies in order for accurate optimization.

There was an icecream advertisment with that as a theme.

https://www.youtube.com/watch?v=j4IFNKYmLa8

Re: Google Vizier: A Service for Black-Box Optimization

#68

Post by indiesolver »

btown wrote: post #66
levesque wrote: post #57

Earlier quoted context omitted.

Yeah SigOpt clearly has taken the space left by Whetlab. They must not be too pleased about this announcement, I mean generally the worst thing that can happen to your startup is a tech giant launching in your space. Then again the problem of hyperparameter/black-box optimization is so ubiquitous that there should be enough space for them both.

I wasn't familiar with SigOpt - this is really cool. I actually think SigOpt will see this as great publicity telling people about the usefulness of the concept - and whereas Google won't lift a finger for (insert hedge fund or pharmaceutical giant here) beyond maintaining uptime, SigOpt can provide those customers with customized advice about how to integrate, what pitfalls to watch for, how to design experiments to…

Spearmint has quite a few practical drawbacks so I guess that both Vizier and SigOpt are much better by now.

Re: Google Vizier: A Service for Black-Box Optimization

#69

Post by portroyal »

hammock wrote: post #64
eadmund wrote: post #60

Earlier quoted context omitted.

What an odd combination of units. Here's a translation: • 5¼ oz. of all-purpose flour. • 7 oz. of milk chocolate chips. • scant ½ + ⅛ tsp. baking soda. • ½ tsp. salt. • ⅛ tsp. cayenne pepper. • 4½ oz. of sugar (31% medium brown, 69% white). • 1 oz. of egg. • 2¾ oz. of butter. • ⅛ tsp. orange extract. • ¾ tsp. vanilla extract. Or, equivalently: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 2…

Still no good because what the hell is 1oz of egg?

iirc, if avg = 50g, yolk is 20g, white is 30g... still up to the foodmage to interpret which part for 1oz (yolk adds taste/color, white has binding properties).

Re: Google Vizier: A Service for Black-Box Optimization

#70

Post by Corence »

meinstream wrote: post #26
kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

I worked for Google at the time they went through the cookie iterations in the cafes. Personally I disliked all the cookies that ever came out of this project, mainly because they contained spices, I heard the same thing from many others. After you got handed a cookie you were asked to fill out a feedback form on a tablet, which almost always is annoying. But more importantly, the process of optimizing the ingredient…

I've heard this was a problem since the spice level was optimized in an office where they make the cookie dough the day of service, but deployed in an office where they make the dough the night before and cook on demand. The spices in the dough longer made the flavor stronger than it was optimized for.

A good example of training/serving skew.

Post reply on HN