Earlier quoted context omitted.
Except the training data is secret…
You can search github and other open sources to find at least a likely subset of the training data though.
You suggest doing this, by hand, for every suggestion?
Saying Goodbye to GitHub
441–450 of 450 posts
Re: Saying Goodbye to GitHub
#442Earlier quoted context omitted.
You can search github and other open sources to find at least a likely subset of the training data though.
You suggest doing this, by hand, for every suggestion?
Just try it out for some code you have on github where you know yours is the only solution out there. You'll be pleasantly surprised to see that it does not suggest a verbatim copy/paste of your code or anything close to it, unless you try this with a one liner like how to do an fopen(), which would not be a good test, and would not be the only solution out there. And then seeing the result, you can adjust your theory. So, in short, I suggest simply testing your theory, not anything absurd like what you are coming up with.
Re: Saying Goodbye to GitHub
#443Earlier quoted context omitted.
> I fundamentally do not believe there is such a thing as "mimickry of reason". There is only reason, done more or less well. if transaction.amount > MAX_TRANSACTION_VOLUME: transaction.reject() else: transaction.allow() Is this code reasoning? It does, after all, take input and make a decision that is dependent on some context, the transactions amount. It even has a model of the world, albeit a very primitive one. N…
I feel this is mostly going to come down to how we define the word. I suspect we agree that there's no point in differentiating "reasoning" from "mimicked reasoning" if the performed actions are identical in every situation. So let's ask differently: what concrete problem do you think LLMs cannot solve?
> what concrete problem do you think LLMs cannot solve?
From the top of my head:
Drawing novel solutions from existing scientific data for one. Extracting information from incomplete data that is only apparent by reasoning (such as my code-bug example given elsewhere in this thread), aka. assuming hidden factors. Complex math is still beyond them, predictive analysis requiring inference is an issue.
They also still face the problem of, as has been anthropomorphized so well, "fantasizing", especially during longer conversations; which is cute when they pretend that footballs fit in coffee-cups, but not so cute when things like this happens:
https://eu.usatoday.com/story/opinion/columnist/2023/04/03/c...
--
These certainly don't matter for the things I am using them for, of course, and so far, they turn out to be tremendously useful tools.
The trouble, however, is not with the problems I know they cannot, or cannot reliably, solve. The problem is with as of yet unknown problems where humans, me included, might assume they can solve, and suddenly it turns out they can't. What these problems are, time will tell. So far we have barely scratched the surface of introducing LLMs in our tech products. So I think it's valueable to keep in mind that there is, in fact, a difference between actually reasoning, and mimicking it, even if the mimicry is to a high standard. If for nothing else, then only to remind us to be careful in how, and for what, we use them.
Re: Saying Goodbye to GitHub
#444Earlier quoted context omitted.
I feel this is mostly going to come down to how we define the word. I suspect we agree that there's no point in differentiating "reasoning" from "mimicked reasoning" if the performed actions are identical in every situation. So let's ask differently: what concrete problem do you think LLMs cannot solve?
> what concrete problem do you think LLMs cannot solve? From the top of my head: Drawing novel solutions from existing scientific data for one. Extracting information from incomplete data that is only apparent by reasoning (such as my code-bug example given elsewhere in this thread), aka. assuming hidden factors. Complex math is still beyond them, predictive analysis requiring inference is an issue. They also still f…
I mean, do you think a LLM cannot draw a novel solution from existing data, fundamentally, because its reasoning is "of the wrong kind"? That seems potentially disprovable. - Or do you just think current products can't do it? I'd agree with that.
What's the easiest novel scientific solution that AI couldn't find if it wasn't in its training set?
Re: Saying Goodbye to GitHub
#445Earlier quoted context omitted.
> what concrete problem do you think LLMs cannot solve? From the top of my head: Drawing novel solutions from existing scientific data for one. Extracting information from incomplete data that is only apparent by reasoning (such as my code-bug example given elsewhere in this thread), aka. assuming hidden factors. Complex math is still beyond them, predictive analysis requiring inference is an issue. They also still f…
I mean, do you think a LLM cannot draw a novel solution from existing data, fundamentally , because its reasoning is "of the wrong kind"? That seems potentially disprovable. - Or do you just think current products can't do it? I'd agree with that. What's the easiest novel scientific solution that AI couldn't find if it wasn't in its training set?
> because its reasoning is "of the wrong kind"?
No, because it doesn't reason, period. Stochastic analysis of sequence probabilities != Reasoning. I explained my thoughts on the matter in this thread to quite some extend.
> That seems potentially disprovable.
You're welcome to try and disprove it. As for prior research on the matter:
https://www.cnet.com/science/meta-trained-an-ai-on-48-millio...
And afaik, Galactica wasn't even intended to do novel research, it was only intended for the, time consuming but comparably easier, tasks of helping to summarize existing scientific data, ask questions about it in natural language and write "scientific code".
Re: Saying Goodbye to GitHub
#446Earlier quoted context omitted.
I mean, do you think a LLM cannot draw a novel solution from existing data, fundamentally , because its reasoning is "of the wrong kind"? That seems potentially disprovable. - Or do you just think current products can't do it? I'd agree with that. What's the easiest novel scientific solution that AI couldn't find if it wasn't in its training set?
> because its reasoning is "of the wrong kind"? No, because it doesn't reason, period. Stochastic analysis of sequence probabilities != Reasoning. I explained my thoughts on the matter in this thread to quite some extend. > That seems potentially disprovable. You're welcome to try and disprove it. As for prior research on the matter: https://www.cnet.com/science/meta-trained-an-ai-on-48-millio... And afaik, Galactica…
Alright, I'll keep an eye open for instances of networks doing scientific reasoning.
(My own belief is that reasoning is 95% habit and 5% randomness, and that networks don't do it because it hasn't been reflected in their training sets, and they can't acquire the skills because they can't acquire any skills not in the training set.)
Re: Saying Goodbye to GitHub
#447Earlier quoted context omitted.
It shouldn't matter. If your model touched X during training, it should be seen as producing derivative work. This is the reason humans use clean room implementation techniques.
> If your model touched X during training, it should be seen as producing derivative work. I as a programmer touched X during training (learning how to code). Is all my work now derivative because of that?
If you examined the source code of a library for a specific purpose Y, then shortly afterwards went implementing another library for purpose Y, there's a high probability of your code being infringing. That's the entire premise and purpose of clean room design (https://en.wikipedia.org/wiki/Clean_room_design).
Now factor in that machine models don't have a fallible or degrading memory and I think the answer is quite clear.
Re: Saying Goodbye to GitHub
#448Earlier quoted context omitted.
You suggest doing this, by hand, for every suggestion?
Just try it out for some code you have on github where you know yours is the only solution out there. You'll be pleasantly surprised to see that it does not suggest a verbatim copy/paste of your code or anything close to it, unless you try this with a one liner like how to do an fopen(), which would not be a good test, and would not be the only solution out there. And then seeing the result, you can adjust your theor…
What would that prove? I still have no access to all the proprietary code generated from copilot, and no idea if it did copy paste or not in all those cases.
You suggest I try it twice and since it will probably not copy paste in those 2 tries, assume it never copy pastes (despite existing evidence that it does copy paste in some other cases).
What problem would this exercise solve? I can't see it.
Re: Saying Goodbye to GitHub
#449Earlier quoted context omitted.
Why assume it didn't?
I’ve done tests and it passed with flying colors so it’s not an assumption. So the premise of your question is flawed.
It has been shown that github copilot does copy paste.
The fact that you tried it a couple of times (or 10 or 20) means absolutely nothing.
1 copyright infringement is enough for a lawsuit.
Re: Saying Goodbye to GitHub
#450Earlier quoted context omitted.
> but with a way more elaborate model? Simple: I know that humans have intentionality and agency. They want things, they have goals both immediate and long term. Their replies are based not just on the context of their experiences and the conversation but their emotional and physical state, and the applicability of their reply to their goals. And they are capable of coming up with reasoning about topics for which the…
> I know that humans have intentionality and agency. You assume that. You can only maybe know that about yourself. But my question was bit different. How do you know that the ML model doesn't? > about topics for which they have no prior information, by applying reasonable similarities. This is a contradiction. If you have no prior information about a topic you can't know even what topic is similar. > Even if someone…
> You assume that. You can only maybe know that about yourself.
I can also only say with certainty that planetary gravity is an attracting force on the very spot I am standing on. I haven't visited every spot on every planet in the universe after all.
That doesn't make it any more likely that my extrapolation of how gravity works here is wrong somewhere else. Russels Teapot works both ways.
> How do you know that the ML model doesn't?
For the same reason why I know that a Hammer or an Operating System don't. I know how they work. Not in the most minute details, and of course the actual model is essentially a black box, but it's architecture, and MO are not.
It completes sequences. That is all it does. It has no semantic understanding of the things these sequences represent. It has no understanding of true or false. It doesn't know math, it doesn't know who person xyz is, it doesn't know that 1993 already happened and 2221 did not. It cannot have abstract concepts of the things represented by the sequences, because the sequences are the things in its world.
It knows that a sequence is more or less likely to follow another sequence. That's it.
From that limited knowledge however, it can very successfully mimick things like math, logic, and even reasoning to an extend. And it can mimick them well enough to be useful in a lot of areas.
But that mimickry, however useful, is still mimickry. It's still the Chinese-Room thought experiment.