This feels less like a framework bug for me than an app bug -- an impressively common app bug, I'll grant you, but an app bug nonetheless. Rails doesn't ship with user management. It's pretty upfront about that, and the docs tell you to roll your own since it is easy, and the community will tell you to use Devise since that will save you time since you'll be doing repetitive work for substantially all apps otherwise.
The typical way beginning Rails developers (including me, on my first Rails apps) would implement logins is stuffing the user ID in session[:user_id] and setting the currently logged in user to that when it is both present and in the DB.
It is true that this approach will make sessions replayable. It will also:
1) Make it impossible to reliably expire session, since the user's browser is not under your control and may treat the expiration date as advisory only.
2) Make it impossible to forcibly expire session, which you'll want to do in event of a user changing or resetting their password, changing their username/email address, etc. (Generic security advice for all web apps, by the way. Otherwise compromise of user accounts, via losing password, XSS, or what have you, is impossible to reverse.)
3) Makes it impossible to implement Sign Me Out Of All Devices, which is a very desirable capability to have for many applications. For example, if you have a more-important-than-cat-videos app, somebody losing their phone should probably not result in their account on your service being irrevocably compromised. You could provide a Dropbox-esque UI to manage their state across multiple devices or, in a pinch, just say "If you hit the log out button on any device, we log you out of all devices."
4) The simplest possible Rails login system is facially non-compliant with a number of security regimes you might be under, including HIPAA among others.
n.b. Switching to ActiveRecordStore doesn't by itself solve anything but #1. You do need app-side logic.