(Replying to myself for the sake of readability:)
All that being said, Ruby (and Smalltalk; and, oddly enough, Javascript) aren't just object-oriented; they go beyond this, into being fully object-based, which is a different thing that Elixir (because of the Erlang VM's architecture) would have a very hard time being.
Ruby—the whole thing—is a graph of objects in a VM, where everything Ruby code does translates to sending a particular message to an object in that graph. The objects themselves start with some responses-to-messages that involve primitives, but if Ruby had a bytecode, it wouldn't contain those primitives, or any way to define them. It'd just be a set of ops to construct values on the stack, and then a send() op. Ruby code effectively boils down to a wire protocol used to speak to a Ruby VM's object graph server. It's all communication, no "client-side" behavior.
So in Ruby, there are no primitives. From the perspective of the VM, 3 might be an unboxed integer primitive; but from the perspective of your Ruby code, it's an object and you have to send messages to it (i.e. to the object-graph server addressed to it.) Ruby code has no way to "operate on" a primitive 3 without asking the object-graph server to do that operating for it.
This is why Ruby "is slow" (and still suboptimal after JIT, due to never being able to drop runtime type checks), but at the same time, it's also why Ruby lets you override Integer#+, and have that effect apply throughout the whole VM. It's why you can re-open a class and define methods on it, when that class is actually a DRb remote object handle. It's why Ruby's Complex and Rational types don't need "first-class" support in order to work with code that never heard of them. Everything is a black box, and so everything is forced to be message-sends to those black boxes. There are no primitives, and so there's no code that expects primitives, which you could break by defying that expectation.
---
As well, Ruby, like Smalltalk, and like Javascript, is image-based: these are languages without an abstract-machine primitive to "load a module." There are no modules. There is only the object graph.
"Library" code in these languages isn't in a "dead" form that gets dumped into VM memory and linked in with symbolic references; instead, it's the source code to a program, a program that gets evaluated when require()d. This program is much like an SQL migration: it can do anything regular code can do, but it usually sends one or more messages to the object-factory-factory member of the graph (i.e. Ruby's Class) asking it to make a new object-factory; and then sends messages to that object-factory, asking it to add mappings from some passed symbol values, to some passed lambda values (i.e. to define some instance methods.)
Smalltalk is a bit different in that Smalltalk has "image-based persistence", meaning the VM can hibernate, and so there's less infrastructure around "loading a bunch of libraries in a tree at VM startup", and more infrastructure around "purging the effects of old libraries and loading new ones." Less like an OS boot process; more like an OS with a package-manager that can install/uninstall packages that start/stop services. But fundamentally, it's the same idea: Smalltalk source code is a migration that executes on the Smalltalk VM to mutate the [persistent] VM object graph.