He seems to have missed the biggest difference which is the lack of visual information.
What's so special about visual information?
Some Remarks on Large Language Models
71–80 of 87 posts
Re: Some Remarks on Large Language Models
#72I would like to see the section on "Common-yet-boring" arguments cleaned up a bit. There is a whole category of "researchers" who just spend their time criticizing LLMs with common-yet-boring arguments (Emily Bender is the best example) such as "they cost a lot to train" (uhhh have you seen how much enterprise spends on cloud for non-LLM stuff? Or seen the power consumption of an aluminum smelting plant? Or calcuated…
I don't understand why there is such a big group of people who see AI research
like a team sport, where it's "us" vs "them" and "we" are the cheerleaders of
our home team, and "they" are the haters and "we" must do everything to shout
those bad guys down.
Criticism is essential for progress in science, and even in AI research (which is far from science). Get over it. The role of the critic is not to be your enemy, the role of the critic is to help you improve your work. That makes no difference if the critic is a bad person who wants your downfall, or not. What makes a difference is if you can convincingly demonstrate that your critic's criticism does not hold anymore. Then people stop listening to the critic- not when you shout louder than the critic.
Oh and, btw, you do that demonstrating by improving your work, which implies that you need to be one of the researchers whose work is criticised to do that, rather than some random cheerleader of the interwebs. What you propose here, to compose some sort of document to paste all over twitter everytime someone says something critical of the "home team", that's not what researchers do; it's organising an internet mob. And it has exactly 0 chance of being of any use to anyone.
Not to mention the focus on Emily Bender is downright creepy.
Re: Some Remarks on Large Language Models
#73> Finally, RLHF, or "RL with Human Feedback". This is a fancy way of saying that the model now observes two humans in a conversation, one playing the role of a user, and another playing the role of "the AI", demonstrating how the AI should respond in different situations. This clearly helps the model learn how dialogs work, and how to keep track of information across dialog states (something that is very hard to learn from just "found" data). And the instructions to the humans are also the source of all the "It is not appropriate to..." and other formulaic / templatic responses we observe from the model. It is a way to train to "behave nicely" by demonstration.
I think this misses a big component of RLHF (the reinforcement learning). The approach described above is "just" supervised learning on human demonstrations. RLHF uses a reinforcement learning objective to train the model rather than maximizing likelihood of human demonstrations. In fact, you can then take the utterances your model has generated, collect human feedback on those to improve your reward model, and then train a new (hopefully better) model -- you no longer need a human roleplaying as an AI. This changed objective addresses some of the alignment issues that LMs struggle with: Open AI does a pretty good job of summarizing the motivation in https://arxiv.org/abs/2009.01325:
> While [supervised learning] has led to markedly improved performance, there is still a misalignment between this fine-tuning objective—maximizing the likelihood of human-written text—and what we care about—generating high-quality outputs as determined by humans. This misalignment has several causes: the maximum likelihood objective has no distinction between important errors (e.g. making up facts) and unimportant errors (e.g. selecting the precise word from a set of synonyms); models are incentivized to place probability mass on all human demonstrations, including those that are low-quality; and distributional shift during sampling can degrade performance. Optimizing for quality may be a principled approach to overcoming these problems.
where RLHF is one approach to "optimizing for quality".
Re: Some Remarks on Large Language Models
#74I asked ChatGPT if author was still relevant. Apparently so.
> Yoav Goldberg is a computer science professor and researcher in the field of natural language processing (NLP). He is currently a professor at Bar-Ilan University in Israel and a senior researcher at the Allen Institute for Artificial Intelligence (AI2).
Professor Goldberg has made significant contributions to the NLP field, particularly in the areas of syntactic parsing, word embeddings, and multi-task learning. He has published numerous papers in top-tier conferences and journals, and his work has been widely cited by other researchers.
Re: Some Remarks on Large Language Models
#75Earlier quoted context omitted.
We stop this like any other issue with the law. Somebody is going to use LLM and cause harm. They will then get sued and people will have to reconsider the risk of using LLM. It’s just a tool
Or they will be sued and it will be dismissed, or the entire process might be suppressed based on trade secrets, national security concerns, or lobbyist concerns. Then people will have to evaluate the risk of using LLM by making sure they have enough lawyers, enough cash, or enough connections as a contractor to get away with doing whatever benefits them most. I don't know what just a tool is supposed to mean. Pistol…
Not just for killing, bur for area denial, never forget.
Re: Some Remarks on Large Language Models
#76Sometimes I read text like this and really enjoy the deep insights and arguments once I filter out the emotion, attitude, or tone. And I wonder if the core of what they're trying to communicate would be better or more efficiently received if the text was more neutral or positive. E.g. you can be 'bearish' on something and point out 'limitations', or you can say 'this is where I think we are' and 'this is how I think…
> Sometimes I read text like this and really enjoy the deep insights and arguments once I filter out the emotion, attitude, or tone. Curiously enough, I imagine that sort of filtering/translation is the sort of thing a Large Language Model would be pretty good at.
It is. At least in the inverse from what I've tried -- adding emotion into an unemotional thing. It can take very benign topics and turn them into absolute ragefests, or some apparent deep wonder of life, or simple sadness. And through all of that you can have it convey those same emotions on the topic as a bear that can only type in caps, and will wholly frame any rationalization as a primal desire to eat apples.
Re: Some Remarks on Large Language Models
#77Earlier quoted context omitted.
Or they will be sued and it will be dismissed, or the entire process might be suppressed based on trade secrets, national security concerns, or lobbyist concerns. Then people will have to evaluate the risk of using LLM by making sure they have enough lawyers, enough cash, or enough connections as a contractor to get away with doing whatever benefits them most. I don't know what just a tool is supposed to mean. Pistol…
Not just for killing, bur for area denial, never forget.
As in denial of access to an area? Or something else?
Re: Some Remarks on Large Language Models
#78Earlier quoted context omitted.
Conversely, these models open up philosophical questions of "exactly what a human is" beyond language abilities. How much of what we think, do, and perceive comes from the use of language?
I think most intelligence is in the language. We're just carriers, but it doesn't come from us and doesn't end with us. We may be lucky to add one or two original ideas on top. What would a human be without language? Language models feed from the same source. They carry as much claim to intelligence, it's the same intelligence. What makes language models inferior today is the lack of access to feedback signals. They…
> I think most intelligence is in the language. We're just carriers
This is such a profound idea. I’ve been wondering about that for a while. Is there anywhere to read up on it?
Re: Some Remarks on Large Language Models
#79While not unsolvable, I think the author is understating this problem a lot: > Also, let's put things in perspective: yes, it is enviromentally costly, but we aren't training that many of them, and the total cost is miniscule compared to all the other energy consumptions we humans do. Part of the reason LLMs aren't that big in the grand scheme of things is because they haven't been good enough and businesses haven't…
This could, perhaps, become a significant issue if and when such systems achieve commercial viability, but right now, they are research projects, and it seems beside the point to balance their usefulness as such against what would be their energy consumption if their use were scaled up to the level of a major commercial activity.
To add a research-oriented comparison to the others being presented here, the LHC's annual energy budget is about 3,000 times that of training GPT-3.
Re: Some Remarks on Large Language Models
#80Interesting post. I find myself moving away from the sort of "compare/contrast with humans" mode and more "let's figure out exactly what this machine _is_" way of thinking. If we look back at the history of mechanical machines, we see a lot of the same kind of debates happening there that we do around AI today -- comparing them to the abilities of humans or animals, arguing that "sure, this machine can do X, but huma…
Machines don't huse (human) language to communicate with each other, humans do, so comparing LLMs to humans makes sense I think.