31–40 of 48 posts
Yet another garbage study where “researchers” wasted months of their lives evaluating GPT 3.5 instead of paying $20 to evaluate GPT 4. It’s like drinking the complementary table water at a restaurant and then leaving a one-star review saying “tasteless and flat”. I seriously can’t believe this is maybe the tenth such “paper” I’ve seen making headlines. It brings shame on not only the institution that produced it, but…
LLM are autocomplete with context, I don't know what they were expecting
Earlier quoted context omitted.
If said genetic algorithms were used to directly spew forth symbols that represent or point towards human experience (“language”) as if the underlying processes that ultimately form the foundation of said symbols could be approximated by slowly changing random walks, then, yes, I’d question the usefulness of that approach. If not, what do you mean?
I mean, just because an array of symbols is generated mechanically doesn't inherently make it meaningless, because meaning arises from the execution of those symbols (in case of computer programs; or interpretation, in case of human language). If you read a sentence (whose source is unknown to you) and it has meaning to you and affects you, what difference does it make if the sentence was written by a human or a mach…
I struggled with this too and because I am fond of defective analogies this made me think of the meaning of a personal note to, say, a lover.
Does it matter by whom or indeed what it was written? The experience of reading occurs solely in the recipient’s mind.
Surely it does not affect him/her differently if said note was produced by a romantic slot machine.
Before I meander towards my confused and shaky conclusion, let me clear one thing out of the way: I did not mean to imply that mechanically generated sequences are meaningless.
They absolutely carry meaning, because meaning is ultimately created by us. We are meaning-creation machines. We can find meaning in tea-leaves. Surely tea-leaves carry meaning, but sadly not the will-of-the-gods-kind. Although that doesn’t stop us from thinking they do.
What I am tentatively suggesting is that the meaning of the lover’s note is as much found in the experience of the relationship they share as it is found in the symbols on the paper. The symbols reference that experience.
Because it is ultimately a human that reads and interprets I think failure to take into account the origin of the material will be the source of a subtle and highly impactful, but inexorable error not dissimilar to that of the tasseographer: what you see doesn’t mean what you think it does.
Language used to be a domain dominated by humans only. We are wired for it and biased by it. If it sounds intelligent, it is intelligent, right? This bias could make us defer to the machine quicker than we should.
What’s the difference between GPT4 generating a good piece of code and a human?
Maybe I should shut up now. I am open to interesting reading suggestions by the way.
Yet another garbage study where “researchers” wasted months of their lives evaluating GPT 3.5 instead of paying $20 to evaluate GPT 4. It’s like drinking the complementary table water at a restaurant and then leaving a one-star review saying “tasteless and flat”. I seriously can’t believe this is maybe the tenth such “paper” I’ve seen making headlines. It brings shame on not only the institution that produced it, but…
I still think GPT-4 has an edge but its frustratingly dumb and loses context so often that I'm not sure how much of an edge it really has any more.
GPT4 can run code, so you can ask it to run the code it generated and iterate until it works. Does that make it one shot? Either way, its limitations are becoming apparent, but it's still quite the leap forwards for writing code.
Earlier quoted context omitted.
I'd like to see the stats on how many people use 3.5 over 4 though, I bet you that number is much higher than you think. The price of ChatGPT is being what I'm personally willing to pay for it so if I do use it, it's 3.5 for me. Maybe the logic is to study the most popular product? Would make sense to me.
Many more people drink tap water than eat dinner at a MICHELIN Starred restaurant, but if you're a food critic writing about the latest fancy restaurant, you had better pay for some food and write about that. This is especially painful considering the ludicrous pace of advancement in AI. You really have to aim for where the puck will be, not where the puck is. Within a year GPT4 will be out-of-date. Fundamentally, th…
Earlier quoted context omitted.
I'd like to see the stats on how many people use 3.5 over 4 though, I bet you that number is much higher than you think. The price of ChatGPT is being what I'm personally willing to pay for it so if I do use it, it's 3.5 for me. Maybe the logic is to study the most popular product? Would make sense to me.
If that were the case, I would expect to see an explicit reference to the CGPT version in the introduction, and an explanation as to why the authors chose it for the study. Instead, the only version information appears to be buried in section 3.1.2 ("ChatGPT 3.5 Turbo API is used").
Earlier quoted context omitted.
You can't understand why software that can do in a fraction of a section what a human would do in hours would be valuable?
The software indeed has to do what I do, which is understanding my environment and mentating solutions that are somehow appropriate for either fixing or alleviating any one of the uncountable challenges said environment faces. I cannot even describe what I am doing, only that relatively small parts of the end-result are encoded into symbols called code. This reminds me of what Bohr said to Oppenheimer - in the movie…
Earlier quoted context omitted.
The purpose of code is first and foremost to make a computer do things. Legibility is, strictly speaking, secondary.
I see you've never had to maintain code.
If you realize code and language are ways to communicate asynchronously and conserve information across time and space to enable developing and maintaining systems, the apparent value of generic concatenation of words that looks good and functions but has not meaningful message conveyed drops off precipitously.
Earlier quoted context omitted.
I mean, just because an array of symbols is generated mechanically doesn't inherently make it meaningless, because meaning arises from the execution of those symbols (in case of computer programs; or interpretation, in case of human language). If you read a sentence (whose source is unknown to you) and it has meaning to you and affects you, what difference does it make if the sentence was written by a human or a mach…
That’s an excellent observation. I struggled with this too and because I am fond of defective analogies this made me think of the meaning of a personal note to, say, a lover. Does it matter by whom or indeed what it was written? The experience of reading occurs solely in the recipient’s mind. Surely it does not affect him/her differently if said note was produced by a romantic slot machine. Before I meander towards m…
If there is no inherent message, if it is just a byte array likely to get a desired result in a specific context, you are losing traction of mind over reality and letting an awful lot of noise into our delicate systems.
Isn't letting accidental complexity aka noise into our systems bad? If you don't know the purpose and the context of the message, you already let that happen, and it will only get worse as the subsequent modifications to the systems described by the code or behaviors enacted due to the message starts to become an opaque box due to the fact that there is no meaningful blueprint behind the code or the texts underlying it.