I don't understand how such a move is profitable for a business.
Can someone please point me to some articles that discuss this?
Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
51–60 of 122 posts
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#52I don't understand how such a move is profitable for a business. Can someone please point me to some articles that discuss this?
My idea is that:
1) It attracts developers who will provide fixes, bug reports, documentation, etc. for free (or even draw potential full-time developers for the project).
2) It makes the project look more "trendy" and appeals to developers who will try not to use proprietary software; this is the same move that .NET did and it seems to have worked very well.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#53The quickstart doesn't seem to work, at lease on macOS. I raised a ticket:
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#54I don't understand how such a move is profitable for a business. Can someone please point me to some articles that discuss this?
My idea is that: 1) It attracts developers who will provide fixes, bug reports, documentation, etc. for free (or even draw potential full-time developers for the project). 2) It makes the project look more "trendy" and appeals to developers who will try not to use proprietary software; this is the same move that .NET did and it seems to have worked very well.
2nd point seems valid.
1st one however suggests that people will try to use it to their own benefit - some of them to compete with Yahoo. Why else would they contribute to the project?
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#55Earlier quoted context omitted.
I think at a glance, it's basically a much more scalable version of something like Elasticsearch, optimized for very quick wide fanout to a large number of leaf nodes. It's a datastore in its own right (just like ES), but I imagine that e.g. you wouldn't use it to handle transactions.
So the upsides of Vespa over Elasticsearch are speeding up the rate at which it scales? Ah, that seems reasonable for a company this size, but is there something in there that's of use for Startups? This blog post shows how Elasticsearch was used to reindex a 136TB dataset with 36B documents[1], so I'm unsure exactly where except for Google/Yahoo Scale companies Vespa is of use. I would like to understand howto utili…
In my experience running machines with Vespa (ended in 2011) and elastic search (which ended earlier this year), Vespa was a lot more stable, even though my elastic search had many times more hardware and fewer documents. At least once a month, elastic search would take a several minute break to do who knows what, even though there was not even any indexing or anything other than searching going on. In case it matters, I was running elastic as a single node cluster (actually several single node cluster), my production Vespa was multinode, but I think we had a single node (or fewer node anyway) cluster for dev/testing.
Anyway, I'm happy that we have more options in this space now.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#56Earlier quoted context omitted.
Meyer basically destroyed a healthy, but not growing, company in the span of 4 years through short-term number-massaging actions at the cost of their competent engineers and core product development. Nadella isn't making a particularly strong case for MS either with his seeming distaste for any product of theirs that isn't either mobile or cloud-based (like the ones where they actually have a monopoly to build off an…
I am not in America, but when she arrived, I was very curious to see how she could revived such a dying zombie. Clients and engineers were leaving as hell. IMHO, she has done a lot to reduce the hemorrhage and make it attractive again.
> IMHO, she has done a lot to reduce the hemorrhage and make it attractive again.
What are you talking about? She's no longer employed and Yahoo has been sold off to Verizon...
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#57Earlier quoted context omitted.
My idea is that: 1) It attracts developers who will provide fixes, bug reports, documentation, etc. for free (or even draw potential full-time developers for the project). 2) It makes the project look more "trendy" and appeals to developers who will try not to use proprietary software; this is the same move that .NET did and it seems to have worked very well.
2nd point seems valid. 1st one however suggests that people will try to use it to their own benefit - some of them to compete with Yahoo. Why else would they contribute to the project?
Proprietary code is expensive to maintain. Even though there's a dedicated team of a few dozen people that has been working on Vespa over the years there are thousands of developers who have been contributing to ElasticSearch, Lucene, and other projects in the open source world that are in a similar space. Getting contributions to Vespa will help it grow and evolve to make it better -- much like Yahoo did when it evolved Hadoop and scaled it out, much like Yahoo did when it help caffe, druid, hive, oozie, openstack, pig, storm, shark, spark, and tensorflow and tons of other projects (that it either created, co-created, or took from others and contributed improvements back to help make better for all).
Sharing code is fine since we use lots of shared code too. We don't sell code. So if someone wants to use Vespa to make an amazing product and make tons of money, we hope they do. We know that sharing Hadoop helped our competitors, but we also know that the revenue stream comes from ads. So we're glad to share code that makes tech better for everyone. As it turns out, many tech companies feel the same way and openly share code with the industry to help us all get to better tech platforms. In the tech space, it's not about grabbing more of the pie, it's about making a bigger pie. The internet revolution is young and the more we built it in the open, the better it will be for all of us.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#58I see it has tensor processing built in - http://docs.vespa.ai/documentation/tensor-intro.html Can this be used as a spark+tensorflow replacement ?
Note: TFoS is also a Yahoo open source project. The teams work together. https://github.com/yahoo/TensorFlowOnSpark
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#59i wonder if this is able to squeeze something useful out of Yahoo Answers.
Re: Open-Sourcing Vespa, Yahoo’s Data Processing and Serving Engine
#60I don't understand how such a move is profitable for a business. Can someone please point me to some articles that discuss this?
https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
The point of Joel's article is that "Smart companies try to commoditize their products’ complements."
For what I understand, Yahoo is a media company, and as so it may try to commoditize a natural complement of today's media companies, which is data search.