Yeah, basically. If your federal employment ends your medical benefits do as well. It's not like a professor's tenure or anything. But unlike the private sector, if you served for at least 5 years you can re-enroll after you reach retirement age and the government will pick up the "employer's costs" of your insurance once again. You'd still be paying the premiums out of pocket though (which is better than other jobs or Medicare if you can afford it).
There is still a small guaranteed basic pension that federal employees pay into, but it's not really enough to retire on by itself. It's one part of their 3 part retirement plan. Couple it with Social Security (the second part) and you could probably scrape by and remain reasonably well fed and housed if you relocated somewhere rural and cheap, but if you plan on retiring and staying put you're going to have to save using the TSP (the third part) like anyone else saves with a 401k.
I can't provide a blanket statement for why people are federal employees though, it's probably different in each industry. My wife is part of the scientific and regulatory side of the oil and gas industry and [benefits included] she doesn't earn substantially less than her private industry peers in similar scientific roles. And that industry is notorious for frequent and cut throat boom-bust cycles which brings it even closer to parity over the long haul. As a software developer though, not a chance. The desirability of the roles they have and the compensation even with benefits is not usually very attractive and the federal hiring process is bonkers compared to most companies.