Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

oath.com

Post reply on HN
Print view View original

31–40 of 122 posts

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#31

Post by relyio »

igor_filippov wrote: post #25
madmax108 wrote: post #17

Yahoo time and again releases open source software which is super super helpful to the community at large. But it always makes me wonder why an org with such an amazing engg culture (multiple anecdotes from friends who were at Yahoo, plus the amazing experiences at Yahoo OpenHack each year as a testament to this) could be run into the ground. Really goes to show that engg != business and unless you have a firm busine…

Because often there's no correlation between engineering culture and business.

I haven't had the longest career, but the little I have see empirically confirmed to me that there's actually a strong connection between engineering culture and business success. Of course, a great engineering culture alone is not enough, but it can act as multiplicative factor.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#32

Post by samfisher83 »

relyio wrote: post #31
igor_filippov wrote: post #25

Earlier quoted context omitted.

Because often there's no correlation between engineering culture and business.

I haven't had the longest career, but the little I have see empirically confirmed to me that there's actually a strong connection between engineering culture and business success. Of course, a great engineering culture alone is not enough, but it can act as multiplicative factor.

I would think they aren't all that correlated. Look at apple, Steve job treated some of his engineers like crap and they were successful. Its nice to have a good place to work, but I don't think it ensures success.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#33

Post by cat199 »

samfisher83 wrote: post #32
relyio wrote: post #31

Earlier quoted context omitted.

I haven't had the longest career, but the little I have see empirically confirmed to me that there's actually a strong connection between engineering culture and business success. Of course, a great engineering culture alone is not enough, but it can act as multiplicative factor.

I would think they aren't all that correlated. Look at apple, Steve job treated some of his engineers like crap and they were successful. Its nice to have a good place to work, but I don't think it ensures success.

This doesn't necessarily mean a 'bad culture' -

a) the concept of 'tough love' is applicable

b) were engineers allowed and challenged to innovate? or did their work effort primarily consist of navigating red tape and beurocracy to the point that all creativity was crushed?

etc.

see also: dilbert.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#34

Post by cat199 »

B1FF_PSUVM wrote: post #26
talad wrote: post #22

Earlier quoted context omitted.

Meyer basically destroyed a healthy, but not growing, company in the span of 4 years through short-term number-massaging actions at the cost of their competent engineers and core product development. Nadella isn't making a particularly strong case for MS either with his seeming distaste for any product of theirs that isn't either mobile or cloud-based (like the ones where they actually have a monopoly to build off an…

Microsoft products? "Windows is a service" is the word now, as firmly stated by a recent message demanding a PC reboot. (The first sentence was "Windows is a service and updates are a normal part of keeping it running smoothly.")

to their "credit", this doesn't actually mean anything given the msft ability to misuse language consistently to make it sound more magical like

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#36

Post by weego »

relyio wrote: post #31
igor_filippov wrote: post #25

Earlier quoted context omitted.

Because often there's no correlation between engineering culture and business.

I haven't had the longest career, but the little I have see empirically confirmed to me that there's actually a strong connection between engineering culture and business success. Of course, a great engineering culture alone is not enough, but it can act as multiplicative factor.

In an engineering centric product. Why would a mom and pop email / news / casual gaming portal succeed directly because of good engineering? Time and time again we massively over-play our role in many business stories.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#37

Post by martinp »

cies wrote: post #15
cies wrote: post #4

Wow this project is humongous! https://github.com/vespa-engine/vespa I'm really curious how it compares to Lucene/ElasticSearch/ELK, which is currently my tool of choice for (faceted) search and recommendation.

Found some comparison (on the root page of their project website, my bad). http://vespa.ai/#featurematrix Next thing I'd really like to know is what existing software it builds on top of.

Outside of general purpose libraries that are found in most software projects it's not based on any existing software. It's built from the ground up by Oath, and the companies that preceded it: Yahoo, FAST, Overture since early-mid 2000s.

This article has some more details about the history: https://www.cnbc.com/2017/09/26/yahoo-open-sources-vespa-for...

Disclamer: I work on the Vespa team in Trondheim, Norway.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#38

Post by kumarvvr »

weego wrote: post #36
relyio wrote: post #31

Earlier quoted context omitted.

I haven't had the longest career, but the little I have see empirically confirmed to me that there's actually a strong connection between engineering culture and business success. Of course, a great engineering culture alone is not enough, but it can act as multiplicative factor.

In an engineering centric product. Why would a mom and pop email / news / casual gaming portal succeed directly because of good engineering? Time and time again we massively over-play our role in many business stories.

This is so true. For a website so heavily focussed on content, tech is only a small part of the game. Case point is youtube, though the tech challange with youtube is more complex. Nevertheless, youtube now has to survive on business deals and cannot depend on tech alone, unlike google search.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#39

Post by reacweb »

talad wrote: post #22
reacweb wrote: post #20

Earlier quoted context omitted.

IMHO, yahoo has bad investors. One of the proof is the firing of Marissa Meyer. As soon as she was hired, the reputation of yahoo has started to improve. All the fights of investors against her until they make her leave had the opposite effect. It takes years to build a reputation and it takes more years for this reputation to bring dividends. I have a similar feeling for Microsoft. The reputation has (slowly) improv…

Meyer basically destroyed a healthy, but not growing, company in the span of 4 years through short-term number-massaging actions at the cost of their competent engineers and core product development. Nadella isn't making a particularly strong case for MS either with his seeming distaste for any product of theirs that isn't either mobile or cloud-based (like the ones where they actually have a monopoly to build off an…

I am not in America, but when she arrived, I was very curious to see how she could revived such a dying zombie. Clients and engineers were leaving as hell. IMHO, she has done a lot to reduce the hemorrhage and make it attractive again.

Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine

#40

Post by overcast »

madmax108 wrote: post #17

Yahoo time and again releases open source software which is super super helpful to the community at large. But it always makes me wonder why an org with such an amazing engg culture (multiple anecdotes from friends who were at Yahoo, plus the amazing experiences at Yahoo OpenHack each year as a testament to this) could be run into the ground. Really goes to show that engg != business and unless you have a firm busine…

Juicero had an amazing engineering culture. Look where that went. You need a proper management team to wrangle it in. Clearly Yahoo has made mistakes in that aspect.
Post reply on HN