Live data from Hacker News

How I spent two weeks hunting a memory leak in Ruby (2015)

be9.io

51–60 of 71 posts

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#51
Wouldn't it be useful if the community tested EVERY gem to see if it leaked memory, perhaps each developer stepping up to test one, in an app that just exercised the gem in question?

To test all of the ones we use individually is prohibitive. But thousands of people could do test one.

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#52

Earlier quoted context omitted.

> it's pretty absurd to only have 1GB for a web application in 2016 While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.

Our main proxies (that do the majority of the heavy lifting over here) run nginx and use, what, 50MB tops? If even? I throw them on $5/mo VPS instances and they just blast out IO without any problems. But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing a…

I realised (thanks to the other poster) that my comment may have seemed dismissive, which was not my intent. Just marvelling at it myself :-) But I do wonder about this kind of stuff. I have been wrestling with some legacy systems chewing up gobs and gobs of memory lately. Then wrestling with my own code that was chewing up gobs and gobs of memory :-)

It's pretty easy to make mistakes because we don't really get punished, as you say. But I wonder if we might be doing ourselves a disservice by not restricting ourselves. Generally speaking, better code is better and while it often feels slower to write, my experience has been that it doesn't always work out that way.

I might try working in a memory restricted container to see if it improves my code...

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#53

Interesting post, but I did want to chime in quickly and say that it's pretty absurd to only have 1GB for a web application in 2016, even for a small one. This is why I've been dismissive of using Heroku for my projects, even though I run a lean stack. They have higher RAM options, but they are incredibly expensive. For the cost of a 14GB Heroku Dyno, I can buy a dedicated server off ebay with 32GB ECC every single m…

1GB is an obscene amount of ram for a webapp, it's just that Ruby is a memory hog. You can run an Elixir webapp with much much less memory.

See: http://www.littlelines.com/blog/2014/07/08/elixir-vs-ruby-sh...

tl:dr;

Ruby/Rails: 235.37MB

Elixir/Phoenix: 34.69MB

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#54

Earlier quoted context omitted.

> it's pretty absurd to only have 1GB for a web application in 2016 While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.

Our main proxies (that do the majority of the heavy lifting over here) run nginx and use, what, 50MB tops? If even? I throw them on $5/mo VPS instances and they just blast out IO without any problems. But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing a…

Another thing to look into is to write things in Rust. It will be as fast as C code, but a little nicer to work with. Right now the problem with that is that you don't have the nice libraries for whatever you're doing.

But if people actually wrote fast Rust libraries for everything, they're as easy to embed into your apps as C.

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#55

Earlier quoted context omitted.

> it's pretty absurd to only have 1GB for a web application in 2016 While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.

Web applications don't just serve up web pages, which puts you in the unenviable position of being dismissive or ignorant about the topic.

Though you got voted down, I hadn't really thought that my comment would be interpreted the way you did. Thanks for pointing it out. I will try to be a little more clear in the future.

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#57

Interesting post, but I did want to chime in quickly and say that it's pretty absurd to only have 1GB for a web application in 2016, even for a small one. This is why I've been dismissive of using Heroku for my projects, even though I run a lean stack. They have higher RAM options, but they are incredibly expensive. For the cost of a 14GB Heroku Dyno, I can buy a dedicated server off ebay with 32GB ECC every single m…

> it's pretty absurd to only have 1GB for a web application in 2016 While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.

It's about return on time invested. As an IoT company we got super excited after we decided to bump our chip from one that has 64kB instead of 32kB. However, I wouldn't blink an eye if we would need a few more GB for our cloud servers. Of course if it's just a static website I run it of Github pages.

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#58
Is there a simple way to notice your own subtle mistakes, when writing in a foreign language? The missing articles in texts from Slavic people bug me more than they should and I wonder what kind of (stylistic) errors I myself make.

The English grammar is mostly just simplified German, but there are some constructs, which I know of, but do not naturally use (e.g. "your every …", "to name but two") - they somehow seem twisted to me. Another one are compound words, I have given up on distinguishing between spaces, hypens and actual compounds and just use spaces almost everywhere.

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#59
post #37

Earlier quoted context omitted.

Interpreted languages can have pretty high memory overhead. Let's say we have a page in python or php or ruby or whatever, and it needs to fetch 1,000 rows from a database (100 things here, 20 things there, whatever). Each row in the database can be turned into an object several times, one from the model, one from the controller, one for the template/view. On top of that, each object is a composite key/value object o…

Needing to fetch 1k rows from a database is different from needing to keep all that data in memory, however. Several years ago I had to return 1M rows from a database to the client with a PHP script, and it needed less than the 16MB (yes, sixteen megabytes ) that the environment of the time was configured with as a memory limit. 1G rows would not have consumed any more memory either, because at any one time the scrip…

> Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed

Even the standard PHP MySQL API (mysqli) defaults to MYSQLI_STORE_RESULT which buffers all the results in memory. You have to specify MYSQLI_USE_RESULT to get streaming results.

Re: How I spent two weeks hunting a memory leak in Ruby (2015)

#60

Earlier quoted context omitted.

> it's pretty absurd to only have 1GB for a web application in 2016 While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.

Our main proxies (that do the majority of the heavy lifting over here) run nginx and use, what, 50MB tops? If even? I throw them on $5/mo VPS instances and they just blast out IO without any problems. But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing a…

But one thing that people keep forgetting is that even 50 MB would be a dream to any user on the old Xerox PARC workstations.

So it is mostly an issue of sloppy coding and bad GC implementations than anything else.

Post reply on HN