To safely deploy generative AI in health care, models must be open source
1–10 of 36 posts
Re: To safely deploy generative AI in health care, models must be open source
#2It would seem to me, that the data is the more important part of the equation, and the health care providers, being the holders of this data, and also needing to find new revenue streams, want to profit from this.
With federated learning and homomorphic encryption, can we satisfy both parties?
Re: To safely deploy generative AI in health care, models must be open source
#3Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
Re: To safely deploy generative AI in health care, models must be open source
#4How does open source improve safety if we simply don't have the analytical tools to intuitively reason about LLMs?
You can't use this to prove that the model will always behave correctly (or desirably). At best, you can build test-suites to empirically check that it kinda-sorta appears to be doing the right thing most of the time. Which you can just as easily do with a black-box model.
It's not that I'm against openness. I just don't see how you can posit that it gets us close enough to safety.
Re: To safely deploy generative AI in health care, models must be open source
#5Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
Yeah it's a pretty obvious misuse of the term. Training data is (part of) the "source"; weights are clearly "binaries". Training is "compiling".
Re: To safely deploy generative AI in health care, models must be open source
#6Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
That's why EU's upcoming AI regulation requires foundational models to have full documentation , including detailed descriptions of training data etc.
Re: To safely deploy generative AI in health care, models must be open source
#7How does open source improve safety if we simply don't have the analytical tools to intuitively reason about LLMs? You can't use this to prove that the model will always behave correctly (or desirably). At best, you can build test-suites to empirically check that it kinda-sorta appears to be doing the right thing most of the time. Which you can just as easily do with a black-box model. It's not that I'm against openn…
Step one is transparency--let's get the black boxes under our control open.
It is not sufficient but it is necessary.
Re: To safely deploy generative AI in health care, models must be open source
#8Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
That's why EU's upcoming AI regulation requires foundational models to have full documentation , including detailed descriptions of training data etc.
I can't fathom why they didn't just require the models to make available the training data itself. Sure you might need to fork some cash so they can ship you hard drives but surely being audited by someone anyone is better than none.
Re: To safely deploy generative AI in health care, models must be open source
#9To deploy generative AI in healthcare someone has to pay for the salaries of a lot of people to do the work. That means there needs to be a business model.
I am not sure who will take an AI through regulatory procedures if it is open source and there is no way to make money from it.
Open source is a useful tool for research yes. More of it would be nice.
But I don’t understand how or why anyone is going to go through all the hurdles of deploying technology if all of it is open source.
Maybe an open source enthusiast can explain to me how that is supposed to work?
Re: To safely deploy generative AI in health care, models must be open source
#10How does open source improve safety if we simply don't have the analytical tools to intuitively reason about LLMs? You can't use this to prove that the model will always behave correctly (or desirably). At best, you can build test-suites to empirically check that it kinda-sorta appears to be doing the right thing most of the time. Which you can just as easily do with a black-box model. It's not that I'm against openn…
Full openness in healthcare also comes with a cost: no training or fine-tuning on patient records or real world interactions which really really ought to not be fully open.