With LLM tools being widely available, it's extremely high, imo.
Idea for a website/documentary -- have experts respond to a piece of news, or provide commentary. Put a few expert pieces alongside a few LLM outputs, have people guess/work out which is which. Have the same people tell you why. If on a website, rank the results; present the 'how I worked it out' info for the best spotters (and you could interview them). Keep the answers secret for a few weeks, then reveal them in a…
I love your idea and would enjoy seeing the results of that controlled experiment.
I'm also interested in the broader impact of using LLMs in place of web search for general Q&A when we want 'to know things'. It's pretty clear the way LLMs are being used for knowledge acquisition now is often less accurate while 'feeling' more certain. Even if we set aside explicit hallucinations, I suspect it's still less accurate.
I can easily and cheaply generate nore megabytes per second per dollar by oversampling a heart rate monitor at hundreds of megahertz. Hell, why not hook up a second channel to the same signal and record it twice for double the megabytes? Do you see the problem here? "yeah, but nobody's doing that" Well, then it certainly is odd of them to frame it tgat way, isn't it?
This is a bit of a contrived example. The “megabytes per second per dollar” is clearly in reference to their scanner technology that the say generates terabytes worth of data, with the goal of a scan taking around 60 seconds. So I’m confused about exactly what your point is? That’s a lot of data really fast, so if you want this 3D scan of your body, yes, you do want as much data as fast as possible. 60 seconds sounds…
The problem is that it's not clear how useful those terabytes of information are. Ultrasound is very good for certain types of imaging, but the contrast mechanisms available are very limited - super high resolution images of uniform intensity aren't useful. An imaging method isn't useful if it doesn't help you discriminate what you want to see from what you don't. The reason MR is so useful is that it has so many contrasts available (T1, T2, proton density, flow speed, diffusion coefficient, diffusion direction, chemical composition, tissue elasticity, BOLD activation, and many other more esoteric ones). In an hour long scan, even with rapid acquisitions, you usually only get a few gigabytes of data, but that data has a LOT of information about your tissue - that's the reason the scanner keeps starting and stopping and making different noises, it's taking MANY different types of images with complementary information.
Someone using md in their account name for 14 years is really committing to the bit if they aren’t a doctor. I went back in their comment history before LLMs existed and found comments where they claim to be a doctor and sound like they know what they are taking about. I’m not a doctor but my wife and many of our friends are, so I know what they sound like.
I appreciate your feedback, I did not dig through their comment history to make that discovery, but saw someone showing full trust in a random user on an anonymous forum and wanted to encourage more critical behavior.
I get the intention, but you made an assumption that the person you replied to didn’t spend 30 seconds find some more supporting evidence before they made their post.
But as far as trust goes, Hacker News has historically been a fairly high trust community. LLMs have the potential to change this dynamic, but I don’t think encouraging people to assume that every post is an LLM is helpful. I don’t think a community with that level of distrust is possible, and at that point we should just all walk away.
Almost nobody here is a doctor, and it shows...over diagnosis, over treatment, those are all terms that doctors learn about in medical school. Image segmentation is a real problem, and achieving better precision is a good goal. The "golden" standard these days is likely https://github.com/wasserth/totalsegmentator , if someone can make it even more accurate, that would be very very good. But yet again, there are infi…
Some initial thoughts as a practicing radiologist: - This looks really cool and I hope they keep innovating on this. I love seeing new modalities develop and despite my (many) reservations and criticisms, if even one good use case comes out of it that truly helps people, it's tech money well spent imo. - They show the reconstructed images as though they are a low resolution CT, and promise that quality will improve a…
I'm not a medical doctor at all, just an engineer who works in medical devices and I'm definitely sceptical about this. I'm not totally sure of the value of an imaging system that only gives you very low resolution images if they're not accurate enough to determine anything from. You'd need a secondary CT or MRI anyway so why not skip to that? My real concern is the dependence on external servers to reconstruct the i…
"why not skip to that?" MRIs and CT scans are expensive, require referrals, and you usually can't get them without believing you already have an issue. If this technology can get to a point where it's high enough resolution, cheap enough to just have at spas, and shared across the world then people will be able to know if/when they should get that secondary scan before symptoms start.
I have libertarian enough tendencies to think that if a person wants to self-operate, or pay for an operation that doctors are telling them is not justified given the evidence, then they should have right to do it. But I don't think that's what people normally mean when they say that eager screening causes harmful overdiagnosis. > So your suggestion for indeterminate scans is more scans? The solution to imperfect evi…
The risk estimation is why people aren’t recommended to get scans! There are studies on ‘VIPs’ who get ‘executive MRIs’ and wind up getting treated for things that would never have justified intervention.
Some initial thoughts as a practicing radiologist: - This looks really cool and I hope they keep innovating on this. I love seeing new modalities develop and despite my (many) reservations and criticisms, if even one good use case comes out of it that truly helps people, it's tech money well spent imo. - They show the reconstructed images as though they are a low resolution CT, and promise that quality will improve a…
I do a lot of MRI analysis including segmentation of small structures in the hippocampus called the hippocampal subfields. To collect these segmentations, we collect partial-field-of-view high in-plane resolution T2-weighted images on a 3 or 7 Tesla magnet. These sequences are generally only included in research protocols if the research specifically cares about hippocampal subfields...therefore they are rarely collected. There have been attempts to enable segmentation of these small structures using lower resolution T1-weighted scans, leveraging deep-learning or other models trained on concurrent T2w high resolution scans and the lower resolution scans within the same subject, allowing the model to predict the higher resolution information from the lower resolution inputs. This produces spectacularly beautiful segmentation on shitty data. Data whose resolution is about the same as the thickness of the structures you are segmenting or less. The problem is this: 1. The lower resolution image barely has any information in it on these smaller structures 2. The accuracy of the resulting segmentation depends entirely on how much the person fits the training distribution. But much research is on specific populations: children, autism, etc. 3. Some big names in imaging analysis tools have published these tools, lending their credibility to them. 4. The beautiful segmentations and (3) tend to convince non hippocampal experts that the resulting data is trustworthy, especially to an eager beaver researcher trying to maximize the impact of their already collected datasets.
I've rejected a number of papers for this.
But my point is this. Midjourney Medical might train a model to produce pretty images with this technique, but the more they need to depend on deep-learning models to get usable data, the more that the match between the training distribution and patient will matter.
That's generally exactly what we do, which if we need to follow 2x or 10x incidental lesions in the population, leads to cost and availability problems. A lymphoma patient in remission needs follow up scans too, and I don't want them to have to wait 3 months because thousands of people are now following up their benign adrenal adenomas.
If you could dramatically reduce cost and improve availability, would this still be a problem? What's the limiting factor that prevents medical imaging from getting cheaper and more available?
Excluding the cost of X-ray/CT/MRI machines, operating them, getting people to them and through them, sometimes injecting contrast, and sometimes dealing with side effects of said contrast, radiologists, I think. You can scale all of the above except interpretation. AI is the natural next thought for how to scale that part, but it's been thought that this would happen any moment for over a decade.
The medical industry should NOT be paternalistic towards patients. People have a right to decide for themselves and be treated as adults capable of informed consent. Thankfully that model is starting to change, although there is much inertia. Any test that is approved for use would have a better-than-random outcome distribution. Preconditioned on that, a test result is still useful no matter how uncertain. It is neve…
Well now we’re going down a different path. Telling people to go burn through mountains of cash to get tests they don’t need is not ethical.
I've been reading this website on and off for years now, and I remember one time I read that a silicon valley startup was selling a "smart cup" that would send you detailed statistics of how much water you drank (assuming you used your smart cup for every drink throughout the day). I suspect if I pitched this to doctors, they would say just drink when your thirsty; you don't need all that data. But that's not the poi…
It's funny because these criticisms are similar to the ones that Raymond Damadian faced when he was trying to develop the MRI. He was criticized for just being a doctor and not an NMR specialist, criticized for making bold claims based on the crude initial images that he presented, criticized for being too exuberant about the potential of the technology, criticized for only having a small group of non-expert graduate…
The criticism is also similar to those faced by Theranos. Survivorship bias is always a factor when looking backwards.