latest is useful for builds where you want to know when something will break. If you only stick to pinned versions, it's common for nobody to ever update the pinned versions for years. Then you're way behind the latest versions and suddenly upgrading becomes a huge pain. You may be stuck in a situation where you're forced to upgrade because of a security hole in your pinned version, but to upgrade to a patched version breaks everything.
".....But that's terrible!", you say. "My build won't be repeatable! I won't be able to perform a roll-back build! My build will constantly break!"
All of that is true, but there are workarounds. For example, if you download, version, and store all artifacts used for each build, you can reuse them later. This isn't hard if you take the time to write some scripts. You can pull a container and export it to a tarball and store it. System packages and language dependencies can be downloaded similarly, and repos mirrored. You can do this once a week and version all artifacts with a datestamp, and make your own app builds pinned to a particular datestamp.
If you always build from those versioned archived datestamps, you can always rebuild or revert to an old working latest build.
As far as builds breaking, they certainly will! You need good testing to catch bugs and regressions. But would you rather learn to adapt quickly to broken builds, or have a sleeping tiger waiting to bite you the one time you finally have to upgrade quickly?
Using a stable branch/tag is the safest hedge against frequently breaking builds while still getting security patches. But stable branches still introduce problems. You will eventually need to revert a stable change, and eventually the stable branch will be End Of Life. So even if you use stable, you should still use the practice of installing from downloaded versioned artifacts.
Since EOL will come eventually, you also have to commit to upgrading to a new stable branch and its breaking changes. You must therefore plan to sunset your own code. Find out when your dependencies will EOL, and plan to completely rebuild your apps using latest before then. It has to be a real commitment and not just a "nice to have", because you will end up being forced to do it by a security vuln or repos that stop carrying an old branch.
Ultimately you need to decide how much risk to take and how much planning is needed to avoid sticky situations. Be aware of the consequences of your design and have contingency plans.