Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The original URL for this prediction will no longer be available in 11 years (2011)

longbets.org

Post reply on HN
Print view View original

51–60 of 106 posts

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#51

Post by tsimionescu »

frotaur wrote: post #28
sublinear wrote: post #23

Earlier quoted context omitted.

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. You don't even have to be that clever. We've all tried using it for work. They fail. In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not p…

I think if you trained an LLM specifically for the purpose of passing the Turing test (instead of being helpful, harmful, and so on), its likely it would pass it. You would have to train it/finetune it on a couple hundred of 'humans chatting in the context of a Turing test'.

I don't think that's true, assuming that the humans are also allowed to strategize and study the problem beforehand. The main weakness of the current models is that they still fail hard at certain kinds of common sense reasoning about real world situations, that no human ever would. Things like the "walk or drive to a car wash that's 2 minutes away" thing from a few months ago (I think the latest models have patched that one in particular, but I'm sure others can be found in a similar vein).

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#52

Post by ofalkaed »

CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

>You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.

While LLMs are very capable of appearing human in general conversation, the test is that they are trying to convince a human that they are human, which is not something that is in their training data and I suspect they would go about things in very odd ways from an actual person, especially when challenged by an actual person. It is still very easy to back LLMs into a corner where they fall into very unnatural patterns and in my experience challenging them on their "humanness" (or anything) backs them into that corner.

LLMs also still can't use context or subtext in natural ways and even people who are terrible with purposefully exploiting context and subtext, generally have a natural ability that they developed to use and parse them. In a short conversation that is not adversarial, this is not an issue but falls apart as the length of the conversation grows or they start getting challenged.

LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump. If we are put into the situation of proving we are human (in an academic test sort of way) we will play with it and have fun with it, and we will find ourselves on that cliff looking into the void, debating if we should expose ourselves and admit things we normally wouldn't. We pick up on this when people do it, even over the internet, their way of stating things changes and their subtext falls away or drastically changes; I have never seen anything approaching this in LLMs.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#53

Post by sigmoid10 »

tsimionescu wrote: post #49
CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.

These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Although with some careful system prompt design, you could probably mask that aspect too.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#54

Post by kvdveer »

tsimionescu wrote: post #49
CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.

In a true competitive Turing test, these reasoning pitfalls could be included in the system prompt, there are not that many of them. The tester would need to find a novel reasoning pitfall, but those aren't uncommon among humans either, so that would not make a very useful differentiator.

Another approach could be to probe the helpfulness of an AI. A human would likely not comply with a request to write a sonnet about tangerines.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#55

Post by tsimionescu »

9dev wrote: post #46
tredre3 wrote: post #21

Earlier quoted context omitted.

I know nothing of Safecloud, but freenet failed for a very valid reasons: 1. Data retention isn't infinite. If the source disappears from the network, peers will prune it shortly. This is exponentially true in our multimedia-heavy world. 2. Despite 1, if you actually want to delete something you published, you can't. Absolutists will see this as a good thing. Normal people will see this as a problem. 3. Peer-to-peer…

Is the p2p problem really still one in our world that is mobile first, but also has ubiquitous internet connectivity pretty much everywhere? I’m almost sure I could host my website from my phone and have at least 99.5% percent of uptime.

Give that it's impossible to even listen on port 443 on a phone, as that would require root privileges, I don't think you can host your website on your phone at all, at least not if you want it to accessible in a normal way. And that's not even going into all of the problems with running background apps on a phone, especially on iOS.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#56

Post by logicallee »

nxpnsv wrote: post #29
sublinear wrote: post #23

Earlier quoted context omitted.

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. You don't even have to be that clever. We've all tried using it for work. They fail. In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not p…

Mannerisms aside, they are easy to spot from having inhuman amounts of trivia knowledge

Humanity is taking an interesting technological arc.

Terminator (1984 film) had a scene showing that in a future 2009, humans would use dogs to try to sniff out whether a robot passing for a human is secretly a machine.[1]

In our real world 2026, there are no humanoid machines that can complete basic generic tasks, like carrying a tray across the stage and holding it for 30 seconds.[2] They move slowly and badly (probably from an LLM like neural network doing very few frames per second of correction and analysis), and are nowhere near lifelike.

When they don't need a body to pass for a human, such as typing online, they do a bit better.

We can tell them apart from humans. As you say, they have stylistic quirks. And you mention that they're easy to spot because they come trained with inhuman amounts of trivial knowledge.

[1] https://www.reddit.com/r/MovieDetails/s/0qUCVPYgjt

[2] https://www.reddit.com/r/LivestreamFail/s/t6ZV0yhgEe

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#58

Post by BoxOfRain »

sublinear wrote: post #23
CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. You don't even have to be that clever. We've all tried using it for work. They fail. In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not p…

I would not call Turing's experiment ill-specified, on the contrary among other factors the paper the game is introduced in earnestly calls for a telepathy-proof room to ensure no side channel leakage. The 1950s was an interesting decade.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#59

Post by applfanboysbgon »

derdi wrote: post #50
CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

The Turing test has been passed. The Turing test had already been passed in 2002. The Turing test isn't about some machine fooling all humans all the time. It's about fooling some humans some of the time. That's an extremely low bar. It has been cleared. Let's put this to rest.

The bar you mentioned was reached in the 1960s with ELIZA. I have no idea why the obsession with the test has persisted into modern generations. It was some guy's thought experiment from 1950. It's not scientific in any fashion. Passing it doesn't prove anything. It's like some kind of authority worship game where people invoke the name of Turing as though doing so bolsters the strength of their arguments in any way whatsoever.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#60

Post by DharmaPolice »

andai wrote: post #30
CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

There was a paper a while ago, where they found a strategy for GPT beating humans on the Turing test 2/3rds of the time. They just prompted it to talk the way a 13 year old girl texts. (Also they prompted it to pretend not to know anything about anything.)

I feel like pretending to be a child is cheating somewhat. Might as well print random key presses and claim it's a toddler.
Post reply on HN