It can sort of work if you've got enough spare track capacity, but unlike buses or planes, with trains its quite easy to run out of capacity
very soon, especially on mixed traffic railways (e.g. Karlsruhe – Basel at the moment only has capacity for two long distance trains per hour
and those two trains need to be flighted together in order to leave enough space for local trains and freight). As soon as you reach that point, competition stops being directly about passengers and moves to the level of train paths instead (meaning you start getting some disalignment of incentives between the railways and the passengers), and consequently can't happen without
actively worsening the service offered by the incumbent operator or even "innocent" third parties (local/regional trains as well as freight operators).
Competition also doesn't really work as soon as your journey isn't large metropolis to large metropolis and involves connecting trains, because a) if a connecting train only runs hourly (or at best half-hourly), it's impossible to have attractive connections from that one train to a multitude of competing operators and b) there are usually ticketing problems which start to matter as soon as one of the trains is delayed and you miss a connection.
Spain has, considering its size, economy and population a relatively generous high speed network (to a large extent financed by EU grants) that by necessity (broad gauage vs. standard gauge) is also completely separate from the legacy network and consequently isn't utilised super-heavily. Plus RENFE already sees its AVEs as a sort of plane on wheels and traditionally has a minimum connection time between trains of 60 mins (!) with only some exceptions, so there is still room for running additional trains and things can't get much worse even with competition and incompatible ticketing between operators on top.
France's and Italy's high speed lines are somewhat more heavily utilised, but you can still get comparatively far by staying exclusively on the high speed network. Plus in France the geography favours the simple Paris-to-province model with no complicated connecting services necessary, whereas Italy is favourable to a linear arrangement (Turin - Milan - Bologna - Florence - Rome - Naples are all along the same line).
In Germany on the other hand population centers are somewhat more distributed across the country, the high speed lines are of a more fragmentary nature and you have much more running on mixed traffic lines, plus a lot of network sections are effectively at capacity.
The result is that in Germany competition mostly isn't solely directly about passengers, but often rather about train paths, and almost any new non-DB long distance service that has been introduced in recent years (thankfully there weren't that many) caused varying amounts of havoc among local services running along the same sections of line, or in some cases even with DB's existing long distance trains.
Plus it means that e.g. RENFE and SNCF now see each other as enemies, so good luck booking a journey between Spain and France that involves a change of trains in one country or the other (or even worse, both of them).