Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Vizier: A Service for Black-Box Optimization

ai.google

Post reply on HN
Print view View original

71–80 of 86 posts

Re: Google Vizier: A Service for Black-Box Optimization

#71

Post by Zephyr314 »

btown wrote: post #66
levesque wrote: post #57

Earlier quoted context omitted.

Yeah SigOpt clearly has taken the space left by Whetlab. They must not be too pleased about this announcement, I mean generally the worst thing that can happen to your startup is a tech giant launching in your space. Then again the problem of hyperparameter/black-box optimization is so ubiquitous that there should be enough space for them both.

I wasn't familiar with SigOpt - this is really cool. I actually think SigOpt will see this as great publicity telling people about the usefulness of the concept - and whereas Google won't lift a finger for (insert hedge fund or pharmaceutical giant here) beyond maintaining uptime, SigOpt can provide those customers with customized advice about how to integrate, what pitfalls to watch for, how to design experiments to…

Thanks! I'm one of the co-founders of SigOpt (YC W15).

You hit the nail on the head. We've been trying to promote more sophisticated optimization approaches since the company formed 4 years ago and are happy to see firms like Google, Amazon, IBM, and SAS enter the space. We definitely feel like the tide of education lifts all boats. Literally everyone doing advanced modeling (ML, AI, simulation, etc) has this problem and we're happy to be the enterprise solution to firms around the world like you mentioned. We provide differentiated support and products from some of these methods via our hosted ensemble of techniques behind a standardized, robust API.

We're active contributors to the field as well via our peer reviewed research [1], sponsorships of academic conferences like NIPS, ICML, AISTATS, and free academic programs [2]. We're super happy to see more people interested in the field and are excited to see where it goes!

[1]: https://sigopt.com/research

[2]: https://sigopt.com/edu

Re: Google Vizier: A Service for Black-Box Optimization

#72

Post by Zephyr314 »

beejiu wrote: post #5

It is referenced in the paper, but it's worth pointing out Yelp's MOE. This is an open source (Apache 2.0) implementation for black-box optimization. It doesn't look like it is well maintained, but it is interesting nonetheless. https://github.com/Yelp/MOE

Hi, I'm one of the co-authors of MOE (it was a fork of my PhD thesis).

We haven't been actively maintaining the open source version of MOE because we built it into a SaaS service 4 years ago as SigOpt (YC W15). Since then we've also had some of the authors of BayesOpt and HyperOpt work with us.

Let me know if you'd like to give it a shot. It is also completely free for academics [1]. If you'd like to see some of the extensions we've made to our original approaches as we've built out our ensemble of optimization algorithms check out our research page [2].

[1]: https://sigopt.com/edu

[2]: https://sigopt.com/research

Re: Google Vizier: A Service for Black-Box Optimization

#73

Post by Zephyr314 »

SigOpt (YC W15) is a related service that performs a superset of these features as a SaaS API (I am one of the co-founders).

We've been solving this problem for customers around the world for the last 4 years and have extended a lot of the original research I started at Yelp with MOE [1]. We employ an ensemble of optimization techniques and provide seamless integration with any pipeline. I'm happy to answer any questions about the product or technology.

We're completely free for academics [2] and publish research like the above at ICML, NIPS, AISTATS regularly [3].

[1]: https://github.com/yelp/MOE

[2]: https://sigopt.com/edu

[3]: https://sigopt.com/research

Re: Google Vizier: A Service for Black-Box Optimization

#74

Post by jrumbut »

hammock wrote: post #64
eadmund wrote: post #60

Earlier quoted context omitted.

What an odd combination of units. Here's a translation: • 5¼ oz. of all-purpose flour. • 7 oz. of milk chocolate chips. • scant ½ + ⅛ tsp. baking soda. • ½ tsp. salt. • ⅛ tsp. cayenne pepper. • 4½ oz. of sugar (31% medium brown, 69% white). • 1 oz. of egg. • 2¾ oz. of butter. • ⅛ tsp. orange extract. • ¾ tsp. vanilla extract. Or, equivalently: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 2…

Still no good because what the hell is 1oz of egg?

Meanwhile I'm feeling pretty good for buying both the 1/10th tsp and the 1/8th tsp scoops right now.

Unfortunately my microliter precision pipette is in the shop otherwise I would be enjoying some of these right now!

Re: Google Vizier: A Service for Black-Box Optimization

#75

Post by antidesitter »

whazor wrote: post #61
oh-kumudo wrote: post #58

Earlier quoted context omitted.

I think this is exactly the case how ML could go wrong. I would assume the people who hate the cookies probably will not bother to submit a survey and try again. The sample might be biased and not representative of the general population, thus overfitting, performs badly on a boarder test set.

Our dystopian future where we will be forced fed cookies in order for accurate optimization.

That already happens :-)

Re: Google Vizier: A Service for Black-Box Optimization

#76

Post by [deleted] »

joshuamorton wrote: post #38
pliny wrote: post #35

Earlier quoted context omitted.

These are not the vizier cookies.

Oh darn you're right (to clarify, I'm pretty sure those are vizier cookies, just not the vizier cookies). I have the recipe sitting on my counter at home. I thought it was public, but it doesn't' seem to exist online anywhere.

[deleted]

Re: Google Vizier: A Service for Black-Box Optimization

#77

Post by sp332 »

joefkelley wrote: post #59
dekhn wrote: post #16

Earlier quoted context omitted.

It's great that they published this - I wasn't sure if I could share this externally. I see they left the ingredient that really improved the scores unexpectedly out, presumbly for proprietary reasons.

Are you talking about orange extract? or cayenne pepper? The post is here: https://ai.google/research/pubs/pub46507 and direct link to paper is here: https://storage.googleapis.com/pub-tools-public-publication-... And the recipe I found internally is "The Real California Cookie" in that paper: Bake at 163C (325◦ F) until brown: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 0.60 tsp. baking…

An article linked in a sibling comment to yours mentioned szechuan pepper but it didn't show up in any of the recipes I've seen so far.

Re: Google Vizier: A Service for Black-Box Optimization

#78

Post by sizzle »

meinstream wrote: post #26
kevlar1818 wrote: post #2

Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box optimization problems arising from physical design or logistical problems. Here we present an example that highlights some additional capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes... We prov…

I worked for Google at the time they went through the cookie iterations in the cafes. Personally I disliked all the cookies that ever came out of this project, mainly because they contained spices, I heard the same thing from many others. After you got handed a cookie you were asked to fill out a feedback form on a tablet, which almost always is annoying. But more importantly, the process of optimizing the ingredient…

I wonder if there was an ethnic bias at play with the population that participated since spice level tolerance is higher in Asian countries.

https://diversity.google/annual-report/

Re: Google Vizier: A Service for Black-Box Optimization

#79

Post by sitkack »

joshuamorton wrote: post #38
pliny wrote: post #35

Earlier quoted context omitted.

These are not the vizier cookies.

Oh darn you're right (to clarify, I'm pretty sure those are vizier cookies, just not the vizier cookies). I have the recipe sitting on my counter at home. I thought it was public, but it doesn't' seem to exist online anywhere.

The secret formula is love!
Post reply on HN