> Exactitude doesn't exist in the universe as we understand it.
Of course exactitude exists. For example, two electrons have exactly the same charge. A photon has exactly 0 charge.
> There's a difference between something not being instantiated in this universe and being unscientific, though.
Well, science is a particular way of studying what exists. Studying something that doesn't exist is unscientific (of course, you can use science to try to determine IF something exists).
But there are also things that are outside the reach of the methods of science, so they are unscientific in this sense. Questions such as "did some god create the universe" are unscientific because it is simply impossible to apply the methods of science to arrive at an answer to this question.
Similarly, asking "is the universe infinite in size" is unscientific, because it is impossible to apply the methods of science and arrive at a definite answer to this question.
> If we produce a model of the universe that doesn't make a single incorrect prediction given all data available, and it predicts infinities to exist in some strange but quite real cases, is it unscientific?
If it predicts actual infinities exist in certain conditions, than it is not going to be a testable theory in those conditions. It may still be a perfectly workable model, just as GR is perfectly workable despite predicting singularities at the center of black holes. That doesn't mean that the singularities exist, it means that GR breaks down at certain points.
But even if you had a physical theory that relied on something like a Banach-Tarski construction, you could never distinguish between an actual infinity of points, leading to two perfectly solid, perfectly identical spheres; and an arbitrarily large number of points, leading either to two perfectly solid but slightly different-sized spheres; or two identically-sized spheres with small holes.
Of course, without some need to specify the number of points, you would be well positioned to use the infinite variant. But if someone asked you if this means that the sphere really has an infinite number of points, the answer would have to be that you can't be sure.