Seems crazy that SEC rules require structured data but these health accounting rules did not. I guess health care has better lobbyists.
Health insurers just published close to a trillion hospital prices
51–60 of 549 posts
Re: Health insurers just published close to a trillion hospital prices
#52Could we rename the title to: "Health insurers just published close to a trillion hospital prices" This post is lot more interesting and important than the current short title would suggest.
Seconded
Re: Health insurers just published close to a trillion hospital prices
#53Could we rename the title to: "Health insurers just published close to a trillion hospital prices" This post is lot more interesting and important than the current short title would suggest.
Sure.
Re: Health insurers just published close to a trillion hospital prices
#54Say someone ingests this data and clean it up make it usable, who's the customer for that service? What would they want to know from it?
A patient wants to know how much a procedure will cost. Now the hospital can look up that data, since apparently (from experience, not hyperbole) no one who works there actually knows.
Re: Health insurers just published close to a trillion hospital prices
#55Anyone else get the feeling this is malicious compliance on behalf of the insurance companies? "Oh, they're going to force us to publish our prices are they? Well we'll publish so much data it'll take a herculean effort to make it readable to anyone that doesn't work in data engineering"
lol, have you ever worked with data from a non-tech company? This is probably the best they have, even inside the company.
Re: Health insurers just published close to a trillion hospital prices
#56Earlier quoted context omitted.
Oh totally. It also probably is the formats they already had -- so they just dumped them into a file -- versus making something more orthogonal and ergonomic.
I'm not sure what format they store their records in, but I have a hunch it's a lot more structured than what we see in the CSV files. The data dumps have to comply with some CMS guidelines set out here: https://github.com/CMSgov/price-transparency-guide
They use relational databases. Then a zillion ETLs to massage that data into every format they need it in, of which this is one of them.
Re: Health insurers just published close to a trillion hospital prices
#57Anyone else get the feeling this is malicious compliance on behalf of the insurance companies? "Oh, they're going to force us to publish our prices are they? Well we'll publish so much data it'll take a herculean effort to make it readable to anyone that doesn't work in data engineering"
The article mentioned CSV files, it seems more like a reflection of what a huge bureaucracy the US healthcare system is. I liked their suggestion that the government should have created the database as part of the law, done the processing on the raw data, and made it more accessible.
import a CSV into Postgres
with open(filepath) as fd:
first_line = fd.readline()
cols = []
for col in first_line.strip().split(','):
col2 = f'''"{col.strip('"')}" text'''
cols.append(col2)
cols2 = ','.join(cols)
print(f"create table {table_name} ({cols2});")
print(f"\copy {table_name} from '{filepath}' csv header;")
run the output in a psql instance connected to your db
(important note: \copy is a psql client command and it is critical to use \copy instead of COPY in many cases where the server process may not have the permission to read your CSV file. with \copy you can read any file the user that launched psql client has permission to read. to make things more confusing it is indeed possible to stream stdin through psql but you use the regular COPY for that instead of \copy)
Re: Health insurers just published close to a trillion hospital prices
#58This seems very much like the episode of Veep'Boxes of lies' where they try to hide their nefarious deeds alongside the real day to day inter-workings of the vice presidents office but inundating the public with data.
This is a common strategy in litigation as well - if someone complains that you have pricked them with a needle, dump haystacks on your opponent's doorstep while also insisting on your right to a speedy trial.
Re: Health insurers just published close to a trillion hospital prices
#59I'm the author. A question I have is: how did so many prices ever get negotiated in the first place? What kind of systems are in place to do this kind of micro-negotiation?
Super curious about this, too. Also! What did they do before they could store 100TB of pricing data? How has pricing (and care quality) changed as a result of being able to do this type of thing?
this would be a great social study. cases where technology has enabled the racketeering and price gouging by corporations with almost no gains in efficiency or output or quality or any metric of value.
Re: Health insurers just published close to a trillion hospital prices
#60I'm the author. A question I have is: how did so many prices ever get negotiated in the first place? What kind of systems are in place to do this kind of micro-negotiation?
The US healthcare system is wildly complicated and inefficient because it is a double-bureaucracy; pubic and private. The government bureaucracy makes a bunch of rules and also provide healthcare through Medicare/Medicaid. The private bureaucracy compete with each other, and hospitals, and pharma companies, ect.
Many of the private health providers are for-profit and lobby against rule changes that would reduce complexity and save the system money. It know this may sound glib, but if you are trying to understand the US healthcare system and something seems strange, usually it's because it makes someone money and they'll fight hard to keep it that way.