21–30 of 36 posts
Earlier quoted context omitted.
That's why EU's upcoming AI regulation requires foundational models to have full documentation , including detailed descriptions of training data etc.
I can't fathom why they didn't just require the models to make available the training data itself. Sure you might need to fork some cash so they can ship you hard drives but surely being audited by someone anyone is better than none.
How does open source improve safety if we simply don't have the analytical tools to intuitively reason about LLMs? You can't use this to prove that the model will always behave correctly (or desirably). At best, you can build test-suites to empirically check that it kinda-sorta appears to be doing the right thing most of the time. Which you can just as easily do with a black-box model. It's not that I'm against openn…
Step one is transparency--let's get the black boxes under our control open. It is not sufficient but it is necessary .
I wouldn't feel any more comfortable getting diagnosed by an open-source LLM than I would be by a proprietary one made by OpenAI.
To deploy generative AI in healthcare someone has to pay for the salaries of a lot of people to do the work. That means there needs to be a business model. I am not sure who will take an AI through regulatory procedures if it is open source and there is no way to make money from it. Open source is a useful tool for research yes. More of it would be nice. But I don’t understand how or why anyone is going to go through…
Not just biomedical research but all of science. The effort is being managed out of Argonne National Laboratory by Rick Stevens.
https://www.anl.gov/article/new-international-consortium-for...
Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
I suspect a lot of people asking for training data are mainly looking to complain about some aspect of it (bias, copyright, etc etc) instead of actually thinking they can somehow use it to devine how the model will perform.
To deploy generative AI in healthcare someone has to pay for the salaries of a lot of people to do the work. That means there needs to be a business model. I am not sure who will take an AI through regulatory procedures if it is open source and there is no way to make money from it. Open source is a useful tool for research yes. More of it would be nice. But I don’t understand how or why anyone is going to go through…
Only certain classes of healthcare products require regulatory approval. For example, you could likely build and distribute an open source AI tool for summarizing patient charts, and the FDA probably wouldn't object (this is not legal advice).
To deploy generative AI in healthcare someone has to pay for the salaries of a lot of people to do the work. That means there needs to be a business model. I am not sure who will take an AI through regulatory procedures if it is open source and there is no way to make money from it. Open source is a useful tool for research yes. More of it would be nice. But I don’t understand how or why anyone is going to go through…
In progress—-this is a natural for a joint effort by NIH, VA, NSF, DOE, DoD, companies, many universities across the globe. Not just biomedical research but all of science. The effort is being managed out of Argonne National Laboratory by Rick Stevens. https://www.anl.gov/article/new-international-consortium-for...
Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
What would you do with the training data if you had it? I see absolutely no reason why the training data is needed to evaluate a model, or how any kind of guarantees could be made about the model if you did have the training data. With the weights and code it's perfectly possible to interrogate and evaluate it. I suspect a lot of people asking for training data are mainly looking to complain about some aspect of it (…
To deploy generative AI in healthcare someone has to pay for the salaries of a lot of people to do the work. That means there needs to be a business model. I am not sure who will take an AI through regulatory procedures if it is open source and there is no way to make money from it. Open source is a useful tool for research yes. More of it would be nice. But I don’t understand how or why anyone is going to go through…
Redhat, Element, Prusa, Adafruit, Sidero Labs, Arduino... plenty of companies that open source everything or almost everything and have have staying power. Many consumers, myself included, will -only- pay for technology if it is open source. In fact if something is proprietary I feel I am being cheated anyway and I might as well pirate it until I find something open to support. Many of us are willing to pay for time…
This is not a model that can work with regulated medical products. There is a very significant cost to maintaining static artifacts and I don’t see how you can defensively do that if anyone can access the artifacts?
Earlier quoted context omitted.
Step one is transparency--let's get the black boxes under our control open. It is not sufficient but it is necessary .
Right, but the article doesn't make that point. It is full of magical thinking that openness is the one hurdle we need to clear. I wouldn't feel any more comfortable getting diagnosed by an open-source LLM than I would be by a proprietary one made by OpenAI.
The problem with 'Doctor AI' isn't that it's going to make mistakes, we already have doctors doing that and killing 100s of thousand per year. It's that we'll only have 1 doctor AI everywhere and there won't be a thing as a second opinion because "Computer Don't Argue"