$('pre:contains("yql=")').each((i, el) => { el.innerText = el.innerText.replace(/\+/g, ' ').replace(/yql=(.+%3B)/, (m, p1) => 'yql=' + decodeURIComponent(p1)) })Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
61–70 of 122 posts
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#62I see it has tensor processing built in - http://docs.vespa.ai/documentation/tensor-intro.html Can this be used as a spark+tensorflow replacement ?
Note: TFoS is also a Yahoo open source project. The teams work together. https://github.com/yahoo/TensorFlowOnSpark
is it used inside Yahoo - because Vespa comes with its own tensor processing engine. I wondered who would use one over the other.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#63$ cloc-git https://github.com/vespa-engine/vespa.git http://cloc.sourceforge.net v 1.60 T=64.10 s (224.5 files/s, 28276.3 lines/s) -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Java 6573 106215 102720 537097 C++ 3209 76542 19178 504855 C/C++ Header 2985 42731 57087 1583…
As an ex-employee, there could not be a better description of Yahoo! development than this.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#64$ cloc-git https://github.com/vespa-engine/vespa.git http://cloc.sourceforge.net v 1.60 T=64.10 s (224.5 files/s, 28276.3 lines/s) -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Java 6573 106215 102720 537097 C++ 3209 76542 19178 504855 C/C++ Header 2985 42731 57087 1583…
As an ex-employee, there could not be a better description of Yahoo! development than this.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#65Earlier quoted context omitted.
2nd point seems valid. 1st one however suggests that people will try to use it to their own benefit - some of them to compete with Yahoo. Why else would they contribute to the project?
Proprietary code is expensive to maintain. Even though there's a dedicated team of a few dozen people that has been working on Vespa over the years there are thousands of developers who have been contributing to ElasticSearch, Lucene, and other projects in the open source world that are in a similar space. Getting contributions to Vespa will help it grow and evolve to make it better -- much like Yahoo did when it evo…
This is not clear to me, can you please explain? I'm still stuck at thinking "if you help your competitors then you give up some of your market share".
EDIT: your pie analogy is really nice, I guess it means you grow the whole market by sharing tools like Vespa so you get a smaller slice of the bigger pie. I still don't get how the "revenue stream comes from ads" part relates to everything else.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#66Earlier quoted context omitted.
When I was there (2003-2005), the big problem already seemed to be that a lot of engineering saw Yahoo as a tech company, while a lot of the business side saw Yahoo as a media company. Especially so after Yahoo conceded Search. So one side wants to pour money into getting technical leverage. The other side largely just wanted more effective ways of publishing and monetizing content. It doesn't really matter which sid…
Yahoo's largest problem while I was there (2004-2011) was that nobody could articulate what y! is and what it does. Actually, at employee orientation in 2004, they had a clear mission: to be a part of everything you do online: either by doing it, or cobranding it, with a goal of being in the top 3 of every internet vertical. Somehow, over time, this message was lost, and it was no longer ok to be #2 or #3 in some ver…
Put another way should your purpose be something customer focused?
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#67Yahoo time and again releases open source software which is super super helpful to the community at large. But it always makes me wonder why an org with such an amazing engg culture (multiple anecdotes from friends who were at Yahoo, plus the amazing experiences at Yahoo OpenHack each year as a testament to this) could be run into the ground. Really goes to show that engg != business and unless you have a firm busine…
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#68Earlier quoted context omitted.
Not sure that Juicero is a good example good engineering, "Do You Need a $400 Juicer?" https://m.youtube.com/watch?v=5lutHF5HhVA
I think you and the parent comment are both partially right - from having watched a number of teardowns, the Juicero was (if anything) massively over-engineered in terms of number of parts, cost of those parts, how they were machined, etc. It was also kind of clear that it went too far, that a more experienced hardware design and engineering team would have found (not a cheaper), but a more effective way to handle th…
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#69I don't understand how such a move is profitable for a business. Can someone please point me to some articles that discuss this?
https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/ The point of Joel's article is that "Smart companies try to commoditize their products’ complements." For what I understand, Yahoo is a media company, and as so it may try to commoditize a natural complement of today's media companies, which is data search.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#70Yahoo time and again releases open source software which is super super helpful to the community at large. But it always makes me wonder why an org with such an amazing engg culture (multiple anecdotes from friends who were at Yahoo, plus the amazing experiences at Yahoo OpenHack each year as a testament to this) could be run into the ground. Really goes to show that engg != business and unless you have a firm busine…
The Register did a good job describing the dichotomy between the product business and the tech side of the business here, related to a previous open source project Yahoo published (disclaimer, I run the open source process at Yahoo) https://www.theregister.co.uk/2017/03/23/yahoo_tensorflow_on... "Over the decades Yahoo! has contributed substantially to the greater good, publishing its own code as open source. Arguabl…
Hopefully Verizon's open source legacy will become even bigger and make it easier for these kinds of innovations to happen.
We've had some success a few cities over in Verizon Labs... https://verizon.github.io/