Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reducing Memory Usage in Ruby

tenderlovemaking.com

Post reply on HN
Print view View original

41–50 of 58 posts

Re: Reducing Memory Usage in Ruby

#41

Post by masklinn »

johncolanduoni wrote: post #11
ad_hominem wrote: post #8

This is orthogonal to tenderlove's GC work, but if you're using MRI and want an easy memory usage/performance win, swap out the standard glibc malloc for jemalloc: https://www.levups.com/en/blog/2017/optimize_ruby_memory_usa...

Kind of a tangent, but does anyone know why glibc’s allocator loses to jemalloc on common workloads? I would expect that would be something worth rectifying unless other requirements tie their hands.

> Kind of a tangent, but does anyone know why glibc’s allocator loses to jemalloc on common workloads?

Jemalloc implements more complex strategies (per-thread caches learned from tcmalloc, preallocated arenas, …).

It tends to have a somewhat higher memory use than simpler allocators, but have faster throughput and less fragmentation (the latter is why Firefox switched to jemalloc by default IIRC).

> I would expect that would be something worth rectifying unless other requirements tie their hands.

Why? If they consider ptmalloc good enough for their purpose, replacing it would be very low priority, and they may value e.g. code simplicity higher.

Incidentally, there was talk at one point to replace glibc's ptmalloc: https://sourceware.org/ml/libc-alpha/2014-10/msg00419.html

Re: Reducing Memory Usage in Ruby

#43

Post by look_lookatme »

eropple wrote: post #21
sam0x17 wrote: post #17

Earlier quoted context omitted.

You can do all of that with really easy to write crystal macros.

You can dynamically build at runtime an HTTP client from an OpenAPI spec (unavailable until runtime, to be clear), method-by-method, using macros? Tell me more.

Yeah use Ruby. I prefer to have well defined interfaces for the objects I use. I love Ruby to death, but it's frustrating to encounter code like this:

  [:thing, :other_thing].each do |msg|
    define_method(msg) do
      @json[msg].to_s
    end
  end
When this is far more clear and, for me, aesthetically pleasing:

  def thing
    @json[:thing].to_s
  end

  def other_thing
    @json[:other_thing].to_s
  end
I'm not sure how you'd use a ruby object that maps to an OpenAPI spec if you don't know what's in the API response until runtime... surely you or the user of your library has an idea what's in the API they are consuming?

Re: Reducing Memory Usage in Ruby

#44

Post by skinnymuch »

ngsayjoe wrote: post #37
mwaitjmp wrote: post #34

Earlier quoted context omitted.

Not only due to the popularity of Rails, Ruby has become popular for sysadmins as config management solutions such as puppet and chef are built on top of it. The resulting syntax is very ruby like, and in some cases you are even required to write a few ruby blocks here and there. It also has an amazing testing framework in the form of rspec, which can be used for much more than just testing Ruby code.

Yeah, and i use Selenium with Ruby as well.

Do you Watir Webdriver, much nicer Ruby wrapper, or straight up Selenium?

Re: Reducing Memory Usage in Ruby

#45

Post by pankajdoharey »

igravious wrote: post #4
sam0x17 wrote: post #2

Or just use Crystal https://crystal-lang.org XD

Crystal is not a replacement for Ruby. In the same way that many Algol-like languages are not a straight swap for each other. Don't get me wrong, Crystal is a nice idea but say goodbye to all that yummy Ruby meta-programming which is one of the key things what makes the language so appealing in the first place.

Meta-programming in Crystal exists through Macros.

Re: Reducing Memory Usage in Ruby

#46

Post by notalex »

eropple wrote: post #13
sam0x17 wrote: post #10

Earlier quoted context omitted.

Have you used a recent (2017) version of Crystal? Quickly building fluent-feeling libraries and tools is exactly my use case for Crystal. For me, everything that Ruby is valuable for is retained. I'm able to do all the things I did in Ruby just fine, and it prevents me from doing a lot of things I shouldn't have been doing. We recently migrated our app from Rails to Crystal (using the Amber framework) and our code ba…

Until I can perform runtime define_method calls and dynamically construct classes, it doesn't matter what Crystal does. It's fundamentally antithetical to my needs. (JavaScript isn't, but JavaScript is kind of clunky.) I have Kotlin, C#, C++, and Rust if I need static typing. Ruby is for when I don't.

Reading well written source code creates a more efficient workflow than reading api docs. For me, the fact that Crystal is written in Crystal gives it a far greater advantage.

Plus it is faster than Go/Java/Elixir. As others have pointed out, Crystal does have meta programming capabilities.

Re: Reducing Memory Usage in Ruby

#47

Post by yxhuvud »

look_lookatme wrote: post #43
eropple wrote: post #21

Earlier quoted context omitted.

You can dynamically build at runtime an HTTP client from an OpenAPI spec (unavailable until runtime, to be clear), method-by-method, using macros? Tell me more.

Yeah use Ruby. I prefer to have well defined interfaces for the objects I use. I love Ruby to death, but it's frustrating to encounter code like this: [:thing, :other_thing].each do |msg| define_method(msg) do @json[msg].to_s end end When this is far more clear and, for me, aesthetically pleasing: def thing @json[:thing].to_s end def other_thing @json[:other_thing].to_s end I'm not sure how you'd use a ruby object th…

The example in question is perfectly possible to do in Crystal during compile-time, using macros. What isn't possible is to create a new class on the fly as a sort of magic lookup table that magic methods is called upon. Good riddance, for that functionality.

Re: Reducing Memory Usage in Ruby

#48

Post by yxhuvud »

sam0x17 wrote: post #28
eropple wrote: post #21

Earlier quoted context omitted.

You can dynamically build at runtime an HTTP client from an OpenAPI spec (unavailable until runtime, to be clear), method-by-method, using macros? Tell me more.

Got me there. First time I've ever in my life heard of a legitimate example where you really do need dynamic methods.

If the problem is stated as you want a dynamic class to create instances where you call dynamic methods, then yes you need Ruby.

However, that is basically a lookup table using string based dispatch to decide what to execute, and such a lookup table can be done in anything.

Re: Reducing Memory Usage in Ruby

#49

Post by eropple »

look_lookatme wrote: post #43
eropple wrote: post #21

Earlier quoted context omitted.

You can dynamically build at runtime an HTTP client from an OpenAPI spec (unavailable until runtime, to be clear), method-by-method, using macros? Tell me more.

Yeah use Ruby. I prefer to have well defined interfaces for the objects I use. I love Ruby to death, but it's frustrating to encounter code like this: [:thing, :other_thing].each do |msg| define_method(msg) do @json[msg].to_s end end When this is far more clear and, for me, aesthetically pleasing: def thing @json[:thing].to_s end def other_thing @json[:other_thing].to_s end I'm not sure how you'd use a ruby object th…

You may have an idea, sure, but the alternative is to codegen a new instance of the client library any time anything changes rather than keeping up to date with the API as it evolves and exists. (The moral equivalent is pulling a WSDL during app startup and making sure that you're presenting a correct interface to it.)

I full-stop don't do codegen in any project under any circumstance that is not done strictly within the workflow of a build tool (i.e., a Maven generated-sources gizmo). OpenAPI's options are pretty much all exactly not that: all the codegen options's happy paths seem to be vendoring generated code. And that makes me itch, too.

Re: Reducing Memory Usage in Ruby

#50

Post by eropple »

sam0x17 wrote: post #39
eropple wrote: post #30

Earlier quoted context omitted.

If you're doing stuff with OpenAPI (as I've been doing over the last few weeks), it becomes a pressing thing. =( Generated clients usually are pretty awful, you usually have to generate your own (because they don't publish gems when their APIs change, which is why I've written online-generated ones!), and once you've completed the bootstrap process for an HTTP client, it's as fast as any other Ruby thing. Which is no…

See GraniteORM or my crystal-mongo-orm. You just have to specify your fields at compile time and you are good-to-go. It works just like active record. https://github.com/amberframework/granite-orm https://github.com/sam0x17/crystal-mongo-orm

"You have to specify your fields at compile time" is exactly not like ActiveRecord, though. That's exactly the downside to it for doing ActiveRecord.

At that point, you may (should) just use an object mapper like ROM that has no model-class hooks at all. (Which I actually like. But the Rails world disagrees.)

Post reply on HN