This is entirely about the same politics of safety we're seeing play out with the pandemic. If jaywalking was legal, the sacrifice is that we'd have to drive slower to watch for pedestrians. As Americans our priority has been individual convenience, over collective safety of our neighbors. Its shifting slowly, but for the most part this is a battle we're grappling with as a society on multiple fronts right now.
I'm not too well-versed on the various American speed limits. But I can tell you the Danish ones. Here there's no concept of "jaywalking". There's simply "crossing the street", and that's functionally never illegal.
We drive 50 kph in cities, towns, and villages. That's 31 mph. On wider roads that speed is 60 kph, or 37 mph. Those are still legal to cross as you please as a pedestrian.
Our normal country roads are driven at 80 kph, or 50 mph. These have a single lane in each direction and shared with pedestrians, bicycles, scooters, and every other road-going transportation method.
The next levels of road speeds (90 kph [55 mph], 110 kph [68 mph], and 130 kph [80 mph]) are only found on roads that don't connect directly to anything used by pedestrians. Specifically, on two-lane expressways and motorways. These can only be used by cars and motorcycles.
As I said initially, I'm not too well versed in US speed limits, but it's my understanding that these speeds at least match, if not are higher, than those. In other words, there is no inherent need to slow cars down in the US to make "jaywalking" legal.