Do you think finetuning these LLMS would bring about comparable results to specific models trained for this?
I think so. It feels like there is more to be squeezed from just better prompts but was going to play around with fine-tuning Qwen3
Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
11–20 of 22 posts
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#12very cool, have you tried some of the newer segmenting models to see if they make a difference? I've seen some in the past two weeks that look really effective...I wonder if it could help out the RL environment
Nope I haven't, I can take a look and see if I can fit it in
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#13Wow this is pretty interesting. Excited to see the benchmark!
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#14Fascinating stuff.
For some reason, this reminds me the way video encoders compress video:
https://en.wikipedia.org/wiki/Video_compression_picture_type...
It makes me wonder if you could use a similar technique (iframes, bframes or pframes) to get the diff of a "normal" WSI and then train on pattern recognition of those.
These different frames are used to reduce network transmission costs, but it feels similar to the context window if you squint at it as a throughput problem rather than a context window size problem.
It feels like there would be a lot of tools and codecs you could leverage here.
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#15Do you think finetuning these LLMS would bring about comparable results to specific models trained for this?
I think so. It feels like there is more to be squeezed from just better prompts but was going to play around with fine-tuning Qwen3
fair enough. I wonder if fine-tuning over different modalities like IMC, H&E etc would help it generalize better across all
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#16I wonder if navigation plays a significant role in performance. If you just randomly select 15 frames (presumably with interesting pixels), will the model perform similarly well?
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#17Fascinating stuff. For some reason, this reminds me the way video encoders compress video: https://en.wikipedia.org/wiki/Video_compression_picture_type... It makes me wonder if you could use a similar technique (iframes, bframes or pframes) to get the diff of a "normal" WSI and then train on pattern recognition of those. These different frames are used to reduce network transmission costs, but it feels similar to the…
I've been thinking a bit more about better ways to build the tooling around it, I don't know much about video compression to be fully transparent but will read up on it.
I have been running into some problems with memory management here as each later frame needs to have a degree of context of the previous frames... (currently I just do something simple like pass in the previous frame and the first reference frame into context) maybe I can look into video compression and see if there is any inspiration there
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#18I wonder if navigation plays a significant role in performance. If you just randomly select 15 frames (presumably with interesting pixels), will the model perform similarly well?
Thought about this too. I think there are two broad LLM capabilities here that are kind of currently tangled up in this eval:
1. Can an LLM navigate a slide effectively (i.e find all relevant regions of interest)? 2. Given a region of interest, can an LLM make the correct assessment?
I need to come up with a better test here in general but yep I'm thinking about this
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#19Earlier quoted context omitted.
I think so. It feels like there is more to be squeezed from just better prompts but was going to play around with fine-tuning Qwen3
fair enough. I wonder if fine-tuning over different modalities like IMC, H&E etc would help it generalize better across all
Yeah I think one of the things that would be interesting is to see how well it generalizes across tasks. It seems like the existence of pathology foundation models means there is certainly a degree of generalizability (at least across tissues) but I am not too sure yet about generalizability across different modalities (there are some cool biomarker-prediction models though)
Re: Show HN: Cancer diagnosis makes for an interesting RL environment for LLMs
#20Interesting work. Some thoughts:
First, your business model isn't really clear, as what you've described so far sounds more like a research project than a go-to-market premise. Computational pathology is a crowded market, and the main players all have two things in common: access to huge numbers of labeled whole-slide images, and workflows designed to handle such images. Without the former, your project sounds like a non-starter, and given the latter, the idea you've pitched doesn't seem like an advantage. Notably, some of the existing models even have open weights (e.g. Prov-GigaPath, CTransPath).
Second, you've talked about using this approach to make diagnoses, but it's not clear exactly how this would be pitched as a market solution. The range of possible diagnoses is almost unlimited, so a useful model would need training data for everything (not possible). My understanding is that foundation models solve this problem by focusing on one or a few diagnoses in a restricted scope, e.g. prostate cancer in prostate core biopsies. The other approach is to screen for normal in clearly-defined settings, e.g. Pap smears, so that anything that isn't "normal" is flagged for manual review. Either approach, as you can see, demands a very different training and market positioning strategy.
Finally, do you have pathologists advising you, and have you done any sort of market analysis? Unless you're already a pathologist (and probably even if you were), I suspect that having both would be of immense value in deciding a go-forward plan.
All the best!