Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The original URL for this prediction will no longer be available in 11 years (2011)

longbets.org

Post reply on HN
Print view View original

81–90 of 105 posts

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#81

Post by joshspankit »

nycdotnet wrote: post #76
iansmith_hn wrote: post #66

Earlier quoted context omitted.

Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue? Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a serv…

There is certainly precedent for this - will executors for example. How about “SCITA? Service continues in the afterlife.

SCIP, in perpetuity?

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#82

Post by bonoboTP »

ofalkaed wrote: post #73
bonoboTP wrote: post #69

Earlier quoted context omitted.

I don't think any frontier lab tried to make an LLM optimized for the literal Turing test where the LLM is made to argue that it's human. This is not in their interest. They make their LLMs insist they aren't human, have no feelings and sentience etc etc. So we don't actually know how good LLMs can be in this.

That is absolutely true of the chatbots we have access to but I suspect there are people working towards these ends, including all the big names in the field, major win for their company if they manage it. But even then, I have yet to see a model that could naturally handle subtext and context, they struggle even when dealing with programming languages where these things are far simpler than they are in human languag…

I'm agnostic on the matter because post training and RLHF on this specific capability could be quite effective. Nobody ever tried because it would cause a backlash and bad PR. Much better PR to solve math problems, help run businesses, cure disease etc than to pretend it's human. The only business case for pretend human is AI boy/girlfriends but those are also not most profitable at their most realistic state.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#83

Post by chuckadams »

sublinear wrote: post #23
CSMastermind wrote: post #18

It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline. The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/ You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. https://longbets.org/9/ is another interesting one depending on whether you think…

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. You don't even have to be that clever. We've all tried using it for work. They fail. In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not p…

I find it more accurate to refer to "A Turing Test" as opposed to "The Turing Test" for that reason. It's just one loosely-defined test; necessary but not sufficient.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#84

Post by tsimionescu »

sigmoid10 wrote: post #53
tsimionescu wrote: post #49

Earlier quoted context omitted.

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.

These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Althou…

> These are usually from a missing context (i.e. specifying where the car is right now).

The reason for the mistake is irrelevant here. All that matters is that you catch the agent saying something that basically no human would, especially in this type of game.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#85

Post by tsimionescu »

kvdveer wrote: post #54
tsimionescu wrote: post #49

Earlier quoted context omitted.

> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities. I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.

In a true competitive Turing test, these reasoning pitfalls could be included in the system prompt, there are not that many of them. The tester would need to find a novel reasoning pitfall, but those aren't uncommon among humans either, so that would not make a very useful differentiator. Another approach could be to probe the helpfulness of an AI. A human would likely not comply with a request to write a sonnet abou…

I don't understand what you mean that "there are not that many of them". The models, even the best ones, still have clear problems with certain kinds of real world reasoning, and it's not a known, closed set. With some prodding and knowing this error mode, I think it's not hard to trip any model with a novel reasoning error like this.

And no, I don't think humans would make the same errors, the key reason for this failure mode is that there are some facts of human day to day life that are so mundane that they are virtually never written about, thus making them an ideal combination of universal human experiences but out of distribution for the LLMs.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#86

Post by AmazingEveryDay »

iansmith_hn wrote: post #66
fragmede wrote: post #17

Earlier quoted context omitted.

Well, and pay registrar and hosting fees for 100 years, and then just hope the hosting company doesn't fold in the interim after your passing. The problem isn't "can you make sure it stays up while you're alive", the problem is, after you're gone, you're no longer there to keep the flame alive and there are no guarantees anyone else picks up the torch.

Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue? Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a serv…

Software after Death as a Service?

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#87

Post by troupo »

mg wrote: post #14

It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place so you can't mess them up accidentally. And convert things that are not changing anymore to static html. One of my first web projects involved a music forum and I have been keeping the URLs alive for 24 years now: https://www.gnoosic.com/discussion/ I feel like I owe it to the people who participated to keep it online…

> It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place

"Just".

Maintaining and keeping these tests (and the oriinal link) running is also probably easy. "Just" keep maintaining them over changes in technology, hosting providers, DNS registrars...

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#88

Post by sigmoid10 »

tsimionescu wrote: post #84
sigmoid10 wrote: post #53

Earlier quoted context omitted.

These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Althou…

> These are usually from a missing context (i.e. specifying where the car is right now). The reason for the mistake is irrelevant here. All that matters is that you catch the agent saying something that basically no human would, especially in this type of game.

If it is missing context, it will be indistinguishable from mistakes that humans could make as well.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#89

Post by ludwik »

munch117 wrote: post #42
ludwik wrote: post #4

Earlier quoted context omitted.

In my mind, the part that is most likely to fail is "return an HTML document that still contains the following text." Nowadays, more and more "HTML documents" do not contain any content at all, just some JavaScript that is responsible for then fetching and displaying the content.

If the text was returned using a staged process, whereby the server first returns an instruction (JavaScript), and the web browser then executes that instruction, which is what transfers the text, then that doesn't actually change anything. Those are just technical details of the transfer. You are viewing an HTML document on-screen; it makes no difference to this bet if it was returned to you by a different mechanism…

I understand that this isn't entirely clear-cut and that people will interpret it differently, but in my opinion, there is a significant difference between "a webpage that contains text X" and "an HTML document that contains text X." "Webpage" and "HTML document" are not synonyms - "HTML document" has a much narrower meaning.

Re: The original URL for this prediction will no longer be available in 11 years (2011)

#90

Post by munch117 »

JoshTriplett wrote: post #62
munch117 wrote: post #42

Earlier quoted context omitted.

If the text was returned using a staged process, whereby the server first returns an instruction (JavaScript), and the web browser then executes that instruction, which is what transfers the text, then that doesn't actually change anything. Those are just technical details of the transfer. You are viewing an HTML document on-screen; it makes no difference to this bet if it was returned to you by a different mechanism…

> or command line tool (like curl) Even if you interpreted the bet generously to allow for a browser to render the content using JavaScript, if I understand the bet correctly, the condition of working with a command-line tool like curl would fail.

I read the or's as giving alternatives, only one of which needs to work.

I can see how other interpretations are possible, but I don't see anything especially generous about mine. I'm not "allowing" anything: To make a judgement you have to pick a web browser to try it with, and you'd probably pick the most popular one, in the most popular configuration, that seems the most fair. If that web browser happens to execute Javascript when you navigate to an URL, then that's just how it is.

Post reply on HN