Anyscale, from the creators of the Ray distributed computing project, launches
1–10 of 14 posts
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#2Project Ray is specifically more focused on scaling AI systems it seems. That is sort of important but also missing from the article.
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#3another one!
berkeley rise lab is such a powerhouse - spark, mesos, etc. just in the past few years. it really puts some of these big tech companies to shame.
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#4Project Ray is specifically more focused on scaling AI systems it seems. That is sort of important but also missing from the article. https://bair.berkeley.edu/blog/2018/01/09/ray/
Previous discussions on the Ray project:
https://news.ycombinator.com/item?id=15481169
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#5Ray user here. Their adoption is exponential. Much faster than Spark when it was "the same age." It's not limited to AI workloads, that's just its flagship use case with RLlib.
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#6Ray user here. Their adoption is exponential. Much faster than Spark when it was "the same age." It's not limited to AI workloads, that's just its flagship use case with RLlib.
Has Ray grown beyond reinforcement learning? Ray appears to lack a datastore or locality mechanism for at-rest distributed data, which is really important for ETL and SQL workloads. So far Ray looks like it has some nice features that are missing from the Spark RDD API, but for ETL / SQL it looks like you’d only choose Ray there for your own entertainment.
It would be compelling if Ray were to provide Horovod support versus the authors re-applying some of same research in their own thing. Ray programming API + code distribution + Horovod performance primitives is what the community probably wants.
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#7Ray user here. Their adoption is exponential. Much faster than Spark when it was "the same age." It's not limited to AI workloads, that's just its flagship use case with RLlib.
Has Ray grown beyond reinforcement learning? Ray appears to lack a datastore or locality mechanism for at-rest distributed data, which is really important for ETL and SQL workloads. So far Ray looks like it has some nice features that are missing from the Spark RDD API, but for ETL / SQL it looks like you’d only choose Ray there for your own entertainment. It would be compelling if Ray were to provide Horovod support…
Can you compare these to https://github.com/TimelyDataflow/timely-dataflow ?
It seems Spark is rolling on inertia more than being a modular system for low-latency cluster computing.
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#8Ray user here. Their adoption is exponential. Much faster than Spark when it was "the same age." It's not limited to AI workloads, that's just its flagship use case with RLlib.
Has Ray grown beyond reinforcement learning? Ray appears to lack a datastore or locality mechanism for at-rest distributed data, which is really important for ETL and SQL workloads. So far Ray looks like it has some nice features that are missing from the Spark RDD API, but for ETL / SQL it looks like you’d only choose Ray there for your own entertainment. It would be compelling if Ray were to provide Horovod support…
It's still pretty early days for ray yet. That being said, spark never really got the hang of doing machine learning properly. It "works" but not for newer workloads which ray is trying to support.
It's good someone is building a company around it. I could see them building services on top of it and build a SAAS like databricks did with spark.
I'll be curious to see how ray matures.
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#9Ray user here. Their adoption is exponential. Much faster than Spark when it was "the same age." It's not limited to AI workloads, that's just its flagship use case with RLlib.
Has Ray grown beyond reinforcement learning? Ray appears to lack a datastore or locality mechanism for at-rest distributed data, which is really important for ETL and SQL workloads. So far Ray looks like it has some nice features that are missing from the Spark RDD API, but for ETL / SQL it looks like you’d only choose Ray there for your own entertainment. It would be compelling if Ray were to provide Horovod support…
Interesting. What do you mean when you say: "versus the authors re-applying some of same research in their own thing."
And does "code distribution" just mean to release as open source?
Re: Anyscale, from the creators of the Ray distributed computing project, launches
#10another one! berkeley rise lab is such a powerhouse - spark, mesos, etc. just in the past few years. it really puts some of these big tech companies to shame.
I think spark and mesos are about a decade old at this point. I'm not really sure anyone is being put to shame either, there are wildly successful alternatives to all of these.