Sadly, both the website and the linked codepen render the source quite poorly. The period is in the wrong place! The rendered HTML is fine. I assume this is just the Unicode bidirectional algorithm failing, as it is wont to do. I imagine that a special cased algorithm that understood that HTML tags (the stuff between inclusive) should render as atomic things, internally LTR, but without imposing their direction on su…
The Unicode bidi algorithm isn't failing here, it's working as designed. The source code snippet has base direction LTR, but has mixed-direction content because of the Hebrew text. Punctuation marks have weak directionality, so the period shows up at the end of the RTL run. Since the base direction is LTR, the 'end' of the RTL run means the right side.
If you want to force mixed-direction content to render correctly, you often need to insert bidirectional control characters to specify where a certain directional run begins/ends. That doesn't make sense to add in this case, though, because it would mess up the rendering in the actual input example.