They say that Economics is the science on mistaking stocks for flows, and this is a very good example.
Interest is denominated in $/month. Loans are denominated in $. Mixing those up is like mixing up miles per hour and miles. They are different units of measurement - the first is a flow, the second is a stock.
Remember that bankers are people too, and they eat just like you do. Therefore interest is nothing more than the wages of bankers. They take those wages and they spend them back with firms in return for food and shelter. The firms then pay the banks with the money they earn from bankers. Round and round the money goes. Bankers earn on the turn as they say.
The same applies to government interest. It is paid on bonds and reserves to financial institutions who pay people a pension from them. Those pensioners then spend that income, which generates additional taxation (because that's how percentages work), which will then balance the amount government paid in the first place.
Therefore the tax that offsets the government interest payments comes from paying the interest payments.
It's just a way of stimulating output, or redistributing it away from the producers to pensioners and other people with money.
In fact all government spending creates the additional tax that offsets it - to the last cent for any positive tax rate. It's a simple geometric progression. The only question is when. If somebody doesn't spend all their income, then taxes are not collected from the spending, earning and re-spending process that would otherwise occur.
And that's what creates the 'deficit' - people deciding not to spend all they earn.
Also known as saving for a rainy day.
There is no need for government to pay interest at all. It's entirely a policy choice. People can then choose to continue to save for no reward, or they can spend the money, which will stimulate economic output.