Just to be clear, "Scots" means current denizens of modern Scotland, who are mostly descended from a mixture of Picts and Gaels. It's not the case that the 5th century "Scoti" wiped out everyone else and they're now the only modern "Scots". The Picts and Gaels eventually united under Kenneth MacAlpin as the Kingdom of Alba... which did not include Strathclyde and the Lothians as Scotland has today.
Also to be clear to anyone else reading, "Britons" is referring to https://en.wikipedia.org/wiki/Celtic_Britons rather than all citizens of modern Britain. These people were predominantly in Strathclyde and Cumbria. I'm not sure how many of them were around in Roman times to be kept out.
As you say, the Romans departure from north Britain predates "Scotland". They fought with many different tribes north of Hadrian's Wall and the Antonine Wall. From Ptolemy, we have some of the names they gave these tribes: Taexali, Vacomagi, Caledonii, ... We don't know if the tribes call themselves that. See https://en.wikipedia.org/wiki/Scotland_during_the_Roman_Empi... -> https://commons.wikimedia.org/wiki/File:Britain.north.people... which is based on a 1467 map, which itself is based on Ptolemy's writing.
There was a Roman campaign into northern territories led by Agricola. We know via Tacitus that he (claims to have) soundly defeated "Caledonians" in "Mons Graupius" in 83 AD, which we suspect is one of the Grampian Mountains but we don't know for sure. https://en.wikipedia.org/wiki/Battle_of_Mons_Graupius -> https://commons.wikimedia.org/wiki/File:Agricola.Campaigns.8...
