81–90 of 97 posts
It's a funny thing to write, like an article in an old newspaper that aged quickly. I suspect that this will be wildly out of date within 2-3 years.
"How do you know the output is good without redoing the work yourself?" Verifying the correctness of solutions is often much easier than finding correct solutions yourself. Examples: Sudoku and most practical problems in just about any field. - "The training doesn't evaluate 'is the answer true' or "is the answer useful.'" Lets pretend RLVF does not exist to give this argument a chance. Then, while the training loop…
"Verifying the correctness of solutions is often much easier than finding correct solutions yourself." Honestly, this has not been my experience at all. Defining what a good solution looks like is most of the battle in Operational Research. But, trying to be constructive, maybe we have identified a sort of diving line between areas where AI is more vs. less helpful.
Earlier quoted context omitted.
AI can do things on its own, without you understanding them yes. But if you are trying to understand something well, there is no better tool for helping you than AI.
I think that AI can sometimes help a lot. But I think doing it correctly is a tightrope and one misstep can easily have terrible results. First issue is this result from reinforcement learning that tells you that you really want to be doing a large fraction of stuff stuff on policy when possible. It's true of RL agents, but I think it's actually just a universal learning result that applies to humans. Sure you could…
Clearly a poor way of studying with AI is: ask the AI to solve a problem, and try to follow through its solution yourself.
A better way is: try to solve a problem yourself, one that is ideally slightly too hard for you but not much more. When you hit a roadblock you can't solve, ask it for help.
In this way you are basically importance sampling information that you misunderstood, but are capable of understanding.
Earlier quoted context omitted.
AI can do things on its own, without you understanding them yes. But if you are trying to understand something well, there is no better tool for helping you than AI.
> But if you are trying to understand something well, there is no better tool for helping you than AI Could not disagree more. The best way to understand something deeply is to practice it. AI is anti-practice. It's like trying to learn something by following a YouTube video step by step. It has an outcome and it feels productive but it's not going to stick in your head at all. It's not practice
As an example, you can ask AI to create problems for you to practice with, tailored for whichever specific difficulty you are having.
Ultimately to understand a thing is to do the thing. And to not understand (which is ok!) is to trust others to, proxy measures or not. Agreed that the future of work is in a precarious place: doing less and trusting more only works up to a point. `simulacrum` is a great word, gotta add that to my vocabulary.
The idea of Simulacrum comes from Baudrillard. His essay "Simulation and Simulacra" is highly recommended for understanding what is so strange about the modern economy.
Earlier quoted context omitted.
Have they though?
Absolutely. Our heuristics for judging human output are useless with LLMs. We can either trust it blindly, or tediously pick over every word (guess which one people do). I've watched this cause havoc over and over at my job (I work with many different teams, one at a time). AI signatures don't mean low quality, they just mean AI. And humans do use them (I have always used the common AI signatures). And yes, humans pr…
We used to call the negative side of these heuristics "code smells", but I see no one has used that term yet in these comments. Code smells are what the post is referring to, what LLMs get rid of.
It's a funny thing to write, like an article in an old newspaper that aged quickly. I suspect that this will be wildly out of date within 2-3 years.
It's already out of date because it makes no sense. If it's true that the superficial signals of quality were once somehow good enough to keep the entire economy on the rails (it's not true), surely you can have an LLM look at given piece of work and extract comparably useful signals of quality or effort.
It's a funny thing to write, like an article in an old newspaper that aged quickly. I suspect that this will be wildly out of date within 2-3 years.
It's already out of date because it makes no sense. If it's true that the superficial signals of quality were once somehow good enough to keep the entire economy on the rails (it's not true), surely you can have an LLM look at given piece of work and extract comparably useful signals of quality or effort.
It was true. The negative signals (we called them "code smells") weren't the be-all-end-all of reviews, they indicated to the reviewer where to spend more effort. It got us 90% of the benefit of an in-depth review with 10% of the effort. But with LLMs eliminating this, we now have to spend all our effort on everything, taking a lot more time and energy overall.
Earlier quoted context omitted.
I think there’s a weaker claim that holds true: we were able to ignore lots of content based on the superficial (and pay proper attention to work that passed this test) and now we are overwhelmed because everything meets the superficial criteria and we can’t pay proper attention to all of it.
That's what I had in mind! The whole post is a claim that evaluating knowledge work got more expensive because cheaper measures stopped correlating well with quality. If someone was already evaluating the work output using a metric closer to the underlying quality then it might not have been a big shift for them (other than having much more work to evaluate).
Earlier quoted context omitted.
It was and still is a negative filter, not a positive one. Meaning it is easy to reject work because there typos and basic factual errors, absence of them is not a good measure of quality. Typically such checks is the first pass not the only criteria. It is valuable to have this, because it the work passes the first check then it easier to identify the actual problems. Same reason we have code quality, lint style fix…
Ironic, you've got some typos but make a good point :)
Perhaps it also conveys different type of meaning by having them in this context.
Errors [1] in community discussion threads like this are positive signals that I am human not a bot. A couple of decades ago, I would be unhappy with myself for it, today accent and idiosyncratic writing are perhaps signals[3] that you are human.
[1] i.e. not proof reading for them, not introducing them deliberately.
[2] I can only see one typographical error (it->if) and many grammar errors, did I miss something ?
[3] Not definitive and not as a personal signature, as it can be easily faked/replicated, but the variations at scale is for now not seen in models. Today's model instances do not get unique personas, accents, idiosyncrasies in writing that would make them unique.