Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twitter survives election after Ruby-to-JVM move

theregister.co.uk

Post reply on HN
Print view View original

161–170 of 179 posts

Re: Twitter survives election after Ruby-to-JVM move

#161

Post by gadders »

mnutt wrote: post #81
SiVal wrote: post #32

What this says to me, a non-RoR user, is that it's harder to build websites with Java than with RoR, but if you pass a certain (very unusual) level of traffic, you'll wish you'd made the extra effort; otherwise, you'll be glad you didn't. (Extra effort isn't free.) Fine, but what I wish I understood was why RoR is so popular to begin with. The claim is always that Rails is so wonderful that it's worth learning Ruby j…

> but if you pass a certain (very unusual) level of traffic, you'll wish you'd made the extra effort I'm not sure that's the case. Perhaps had they started out by trying to scale to their current levels they would never have gotten off the ground. When you start developing a system for a new company your largest obstacle is almost always lack of product/market fit.

Isn't there some famous quote about premature optimisation? :-)

Re: Twitter survives election after Ruby-to-JVM move

#162

Post by kevinconroy »

pg wrote: post #37
kevinconroy wrote: post #6

The novice programmer says: "My language is better than yours." The wise programmer says: "Use the right tool for the job." Please HN, we're wiser than trending stories would suggest.

Do you think it's impossible for one language to be better than another? If so, it's impossible for a language designer to do a good job, because it's impossible to improve a programming language.

If all you have is a hammer...

I think all languages have strengths and weaknesses that matter based on the context.

For example, I hate manually managing memory in C/C++. Java, Ruby, etc automate all of that for you, but it comes at a cost of using a lot more memory. That's not a big concern for many applications, but if you're doing embedded software or real-time systems it can be a deal breaker.

So in this example, is C++ better or worse than Java? The answer is that it depends what problem you're trying to solve, what environment you're running in, and what resources you have available on your development team. Personal preference matters too in terms of programmer satisfaction, but just because you like one language's constructs more than another doesn't mean it's well suited to every problem space.

As to the original article, I think it's great to learn about how different companies change their software stack, but not because "w00t Twitter hates Ruby - Java rocks"; rather it's interesting to see how business context changes over time and the implications that has on the technology. Seeing how other businesses have dealt with these hurdles can help you keep an eye out for them in your own business. I just wish that this was the lens that it was written in rather than "OMG - Ruby = Fail Whale."

If you're a language designer then it's good to hear about the complaints and preferences of programmers so that you can design a better language (and thus a better tool). And like any tool, there will be times when it's better to use one language rather than another.

Re: Twitter survives election after Ruby-to-JVM move

#163

Post by randall »

vidarh wrote: post #153
timothya wrote: post #8

It's interesting that when it comes time to scale to serve enormous loads, you have to be willing to change fundamental parts of your stack which you've made a huge investment in. Ruby holds up well enough on the majority of the sites that use it, but when you have traffic the size that Twitter does, it's just not good enough. And it turns out that Java provides a nice tradeoff with high performance and high-level co…

No, you don't need to change fundamental parts. You need to get the architecture right. Then you need to determine whether the potential cost savings of switching languages are worth it for you. If things start falling over when you scale, it's an architecture problem, not a language problem. Why do I say that? Because in Twitters case, handling tweets is a "trivial" problem to parallelise, and the potential savings…

Would love to see someone at twitter refute this... seems too simple to be correct. :)

Re: Twitter survives election after Ruby-to-JVM move

#164

Post by elchief »

camus wrote: post #97
elchief wrote: post #3

JVM is a wonderful thing.

Github and the thousands of Rails powered websites have no problem scaling, so i'm pretty sure neither Ruby nor Rails are a problem.

500 million users is more a little more demanding than 2 million users.

Re: Twitter survives election after Ruby-to-JVM move

#165

Post by vidarh »

randall wrote: post #163
vidarh wrote: post #153

Earlier quoted context omitted.

No, you don't need to change fundamental parts. You need to get the architecture right. Then you need to determine whether the potential cost savings of switching languages are worth it for you. If things start falling over when you scale, it's an architecture problem, not a language problem. Why do I say that? Because in Twitters case, handling tweets is a "trivial" problem to parallelise, and the potential savings…

Would love to see someone at twitter refute this... seems too simple to be correct. :)

Keep in mind I gave it as an example that is conceptually simple to demonstrate that scaling it with pretty much any technology is possible. It is still a lot of work, and expensive, and likely far from optimal.

The approach I suggested would create massive message spikes, and there's undoubtably optimizations. E.g. 30 million followers for the top users is a lot, but most of them don't tweet all that often. Perhaps it's better to keep the recent tweet of the top 1000 (or 100,000) users in memory in memcached's and "weave" them into the timelines of their followers instead of trying to immediately stored the in timelines for each user.

The point is scaling is pretty much never a language problem once your needs are large enough that they'll exceed a server or even a rack no matter how efficient the language implementation is.

At that point the language choice is an operational cost vs. productivity issue down to whether or not the CPU usage differences are sufficiently high to cost you enough to offset any productivity advantages you might get from the slower language implementation.

That is a valid concern. It's perfectly possible that Twitter is right in switching in their case. E.g. even a 10% reduction in servers could pay for a lot of developer hours when your server park gets large enough.

If you're on the tipping point where switching to a faster language implementation can keep you from having to scale past a single machine, then it's slightly different. But arguably if you're in that position, you should still plan for growth.

Re: Twitter survives election after Ruby-to-JVM move

#166

Post by Xylakant »

camus wrote: post #129
mercurial wrote: post #127

Earlier quoted context omitted.

> Python is strongely typed Like Ruby. > and compiled into bytecode Like JRuby. Python and Ruby are comparable languages. They are both high-level dynamic languages, with the same sort of tradeoffs (eg, you can do more with less characters than in most statically typed languages, at the expense of safety and performance). There are however significant differences in terms of philosophy, ecosystem and community betwee…

Except you dont need a JVM to get performances in python. Using a JVM destroys any perspective of cheap and easy deveployement. If i have to use a JVM , i'd certainly not use JRuby when i can get scala.

> Using a JVM destroys any perspective of cheap and easy deveployement.

I doubt that. The fact that jruby is close to bug-by-bug compatibility means that for most projects you can go and develop on mri or rubinius for fast development and use jenkins/travis/... to run your testsuite against jruby. You can then do integration tests as well as staging on jruby to catch anything your testsuite didn't catch. I've seen more than one large project take that course of action and do just fine.

Deployment on jruby isn't any harder than deployment on any other ruby. It's still "puma start" or whichever server you chose. You can however package your app in a war use a standard servlet container such as tomcat or jetty to deploy, but that's strictly optional.

Re: Twitter survives election after Ruby-to-JVM move

#167

Post by Xylakant »

diminish wrote: post #142
fizx wrote: post #54

Earlier quoted context omitted.

It seems to me that he is arguing that languages do not have a total ordering. As such, novices will argue that a language is strictly better than another, when a broader perspective will reveal that this is only true for some use cases. The "no free lunch" theorem would indicate to me that there is no ultimate language, merely languages that are better for common (to you) use cases.

pg's argument reminds me of plato's dialogues where socrates confronts sophistry and searches for a universal truth. At first sight, it seems hard to make a primary ordering of programming languages, because they focus on different areas (ease of use, speed of development, performance etc) and ordering those disjunct areas against each other seems to be impossible. However, it is still possible for a programming lang…

> At first sight, it seems hard to make a primary ordering of programming languages, because they focus on different areas (ease of use, speed of development, performance etc) and ordering those disjunct areas against each other seems to be impossible. However, it is still possible for a programming language to focus and excel on all those areas at the same time.

I disagree. There's more than ease of use, speed of development and performance to determine if a language is "better" suited to a problem than others. And they're all trade-offs to some point. So to excel at one means that you fall behind on others. Prolog is still used heavily in some areas since it's a natural fit to logic problems. It's just pure logic theory modeled in a programming language. It's a pityful language to solve most "real world" problems, but it excels at what it was made for. So is it "better" than ruby? I don't think so. Is it possible to write a language that models logic problems so nicely as Prolog and still keep the ease of use of ruby? I very much doubt that.

Re: Twitter survives election after Ruby-to-JVM move

#168

Post by [deleted] »

fizx wrote: post #54
pg wrote: post #37

Earlier quoted context omitted.

Do you think it's impossible for one language to be better than another? If so, it's impossible for a language designer to do a good job, because it's impossible to improve a programming language.

It seems to me that he is arguing that languages do not have a total ordering. As such, novices will argue that a language is strictly better than another, when a broader perspective will reveal that this is only true for some use cases. The "no free lunch" theorem would indicate to me that there is no ultimate language, merely languages that are better for common (to you) use cases.

[deleted]

Re: Twitter survives election after Ruby-to-JVM move

#169

Post by blacksmythe »

irahul wrote: post #134
camus wrote: post #129

Earlier quoted context omitted.

Except you dont need a JVM to get performances in python. Using a JVM destroys any perspective of cheap and easy deveployement. If i have to use a JVM , i'd certainly not use JRuby when i can get scala.

> Except you dont need a JVM to get performances in python. Python's performance is only slightly ahead of Ruby's. Both Python and Ruby are a bad match for CPU intensive tasks(provided we are talking pure python/ruby and not native extensions). Network and IO in general is salvaged by using evented io(or some high level evented framework).

Because numerical processing is a major use case, Python has a number of techniques to easily speed up critical sections of code (numpy, cython, pypy).

I am not aware of comparable features in Ruby.

Re: Twitter survives election after Ruby-to-JVM move

#170

Post by mercurial »

blacksmythe wrote: post #169
irahul wrote: post #134

Earlier quoted context omitted.

> Except you dont need a JVM to get performances in python. Python's performance is only slightly ahead of Ruby's. Both Python and Ruby are a bad match for CPU intensive tasks(provided we are talking pure python/ruby and not native extensions). Network and IO in general is salvaged by using evented io(or some high level evented framework).

Because numerical processing is a major use case, Python has a number of techniques to easily speed up critical sections of code (numpy, cython, pypy). I am not aware of comparable features in Ruby.

Numpy is a dedicated library where the critical code is in C, but that's a library, hardly a language feature. I don't know if there is any numerical processing library in Ruby.

Cython is not intrinsic to Python either. I see there is a ruby2c gem, but I have no idea how it compares to Cython.

As for pypy, that's about switching to a different runtime, the same way Ruby people would go to JRuby.

None of this turns Ruby into apples and Python into oranges, I'm afraid.

Post reply on HN