Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I failed to recreate the 1996 Space Jam website with Claude

j0nah.com

Post reply on HN
Print view View original

381–390 of 483 posts

Re: I failed to recreate the 1996 Space Jam website with Claude

#381

Post by measurablefunc »

jacquesm wrote: post #367
measurablefunc wrote: post #357

Earlier quoted context omitted.

I'm not here to convince you whether Markov chains are helpful for your use cases or not. I know from personal experience that even in cases where I have a logically constrained query I will receive completely nonsensical responses¹. ¹ https://chatgpt.com/share/69367c7a-8258-8009-877c-b44b267a35...

> Here is a correct, standard correction: It does this all the time, but as often as not then outputs nonsense again, just different nonsense, and if you keep it running long enough it starts repeating previous errors (presumably because some sliding window is exhausted).

That's been my general experience and that was the most recent example. People keep forgetting that unless they can independently verify the outputs they are essentially paying OpenAI for the privilige of being very confidently gaslighted.

Re: I failed to recreate the 1996 Space Jam website with Claude

#383

Post by theahura »

I was able to get Claude to do this, though it kinda sorta cheated. Blog post describing the output here: https://theahura.substack.com/p/i-successfully-recreated-the...

TLDR:

"The plan is designed to ‘autoformalize’ the problem by using Test Driven Development (TDD). TDD is incredibly important for getting good outputs from a coding agent, because it helps solve the context rot problem. Specifically, if you can write a good test when the model is most ‘lucid’, it will have an easier time later on because it is just solving the test instead of ‘building a feature’ or whatever high dimensional ask you originally gave it.

From here, Nori chugged away for the better part of half an hour in yolo mode while I went to do other things. And eventually I got a little pop up notification saying that it was done. It had written a playwright test that would open an html file, screenshot it, diff it with the original screenshot, and output the final result...

After trying a few ways to get the stars to line up perfectly, it just gave up and copied the screenshot in as the background image, then overlaid the rest of the HTML elements on top.

I’m tempted to give this a pass for a few reasons.

This obviously covers the original use case that tripped up Jonah.

It also is basically exactly what I asked the model to do — that is, give me a pixel perfect representation — so it’s kind of my fault that I was not clearer.

I’m not sure the model actually can get to pixel perfect any other way. The screengrab has artifacts. After all, I basically just used the default linux screenshot selection tool to get the original output, without even paying much attention to the width of the image.

If you ask the model to loosen the requirements for the exact screengrab, it does the right thing, but the pixel alignment is slightly off. The model included this as index_tiled.html in the repo, and you can see the pixel diff in one of the output images..."

Re: I failed to recreate the 1996 Space Jam website with Claude

#384

Post by fc417fc802 »

martin-t wrote: post #264
nitwit005 wrote: post #250

Earlier quoted context omitted.

Ask for something like "a first person shooter using software rendering", and search github for the function names for the rendering functions. Using Copilot I found code simply lifted from implementations of Doom, except that "int" was replaced with "int32_t" and similar. It's also fun to tell Copilot that the code will violate a license. It will seemingly always tell you it's fine. Safe legal advice.

And this is just the stuff you notice. 1) Verbatin copy is first-order plagiarism. 2a) Second-order plagiarism of written text would be replacing words with synonyms. Or taking a book paragraph by paragraph and for each one of them, rephrasing it in your own words. Yes, it might fool automated checkers but the structure would still be a copy of the original book. And most importantly, it would not contain any new inf…

You make several good points, and I appreciate that they appear well thought out.

> What previously took creative human work now became a mechanical transformation of input data.

At which point I find myself wondering if there's actually a problem. If it was previously permitted due to the presence of creative input, why should automating that process change the legal status? What justifies treating human output differently?

> then free software is dead. No more freedom to inspect and modify.

It seems to me that depends on the ideological framing. Consider a (still entirely hypothetical) world where anyone can receive approximately any software they wish with little more than a Q&A session with an expert AI agent. Rather than free software being dead, such a scenario would appear to obviate the vast majority of needs that free software sets out to serve in the first place.

It seems a bit like worrying that free access to a comprehensive public transportation service would kill off a ride sharing service. It probably would, and the end result would also probably be a net benefit to humanity.

Re: I failed to recreate the 1996 Space Jam website with Claude

#385

Post by vghaisas »

p0w3n3d wrote: post #363

LLM stands for large LANGUAGE models, so I guess you could succeed if you had a correct LANGUAGE. Maybe radial coordinates? Or turtle graphics? I myself tried to generate an SVG with twelve radial dots as in a clock in chatgpt, and failed (a year ago). Now I think it would succeed, however still the question is does it succeed because people trained it to do so. Also I have noticed that AI generates things close to w…

You might enjoy these AI generated clocks: https://clocks.brianmoore.com/

Re: I failed to recreate the 1996 Space Jam website with Claude

#386

Post by subscribed »

handoflixue wrote: post #293
jacquesm wrote: post #178

Earlier quoted context omitted.

That's true. But if we take your implied rebuttal then current level AI would be able to learn from current AI as well as it would learn from humans, just like humans learn from other humans. But so far that does not seem to be the case, in fact, AI companies do everything they can to avoid eating their own tail. They'd love eating their own tail if it was worth it. To me that's proof positive they know their output…

Equally, of course, all six year olds need to be trained by other six year olds; we must stop this crutch of using adult teachers

Beautiful, thank you.

Re: I failed to recreate the 1996 Space Jam website with Claude

#387

Post by falcor84 »

yard2010 wrote: post #371
govping wrote: post #346

Earlier quoted context omitted.

The failure mode here (Claude trying to satisfy rather than saying 'this is impossible with the constraints') shows up everywhere. We use it for security research - it'll keep trying to find exploits even when none exist rather than admit defeat. The key is building external validation (does the POC actually work?) rather than trusting the LLM's confidence.

Ah! I see the problem now! AI can't see shit, it's a statistical model not some form of human. It uses words, so like humans, it can say every shit it wants and it's true until you find out. The number one rule of the internet is don't believe anything you read. This rule was lost in history unfortunately.

When reasoning about sufficiently complex mechanisms, you benefit from adopting the Intentional Stance regardless of whether the thing on the other side is "some form of human". For example, when I'm planning a competitive strategy, I'm reasoning about how $OTHER_FIRM might respond to my pricing changes, without caring whether there's a particular mental process on the other side

Re: I failed to recreate the 1996 Space Jam website with Claude

#388

Post by marcus_holmes »

jacquesm wrote: post #347
marcus_holmes wrote: post #281

Earlier quoted context omitted.

LLMs can definitely produce original other stuff: ask it to create an original poem and on an extremely specific niche subject and it will do so. You can specify the niche subject to the point where it is incredibly unlikely that there is a poem on that subject in its training data, and it will still produce an original poem on that subject [0]. The well-known "otter using wifi on a plane" series of images [1] is ano…

No, it transformed your prompt. Another person giving it the same prompt will get the same result when starting from the same state. f('your prompt here') is a transformation of your prompt based on hidden state.

This is also true of humans, see every debate on free will ever.

The trick, of course, is getting to the exact same starting state.

Re: I failed to recreate the 1996 Space Jam website with Claude

#389

Post by checkmatez »

bsaul wrote: post #135
jacquesm wrote: post #121

Earlier quoted context omitted.

Claude didn't write that code. Someone else did and Claude took that code without credit to the original author(s), adapted it to your use case and then presented it as its own creation to you and you accepted this. If a human did this we probably would have a word for them.

That's an interesting hypothesis : that LLM are fundamentally unable to produce original code. Do you have papers to back this up ? That was also my reaction when i saw some really crazy accurate comments on some vibe coded piece of code, but i couldn't prove it, and thinking about it now i think my intuition was wrong (ie : LLMs do produce original complex code).

> that LLM are fundamentally unable to produce original code.

What about humans? Are humans capable of producing completely original code or ideas or thoughts?

As the saying goes, if you want to create something from scratch, you have to start by inventing the universe.

Human mind works by noticing patterns and applying them in different contexts.

Re: I failed to recreate the 1996 Space Jam website with Claude

#390

Post by meindnoch »

yard2010 wrote: post #371
govping wrote: post #346

Earlier quoted context omitted.

The failure mode here (Claude trying to satisfy rather than saying 'this is impossible with the constraints') shows up everywhere. We use it for security research - it'll keep trying to find exploits even when none exist rather than admit defeat. The key is building external validation (does the POC actually work?) rather than trusting the LLM's confidence.

Ah! I see the problem now! AI can't see shit, it's a statistical model not some form of human. It uses words, so like humans, it can say every shit it wants and it's true until you find out. The number one rule of the internet is don't believe anything you read. This rule was lost in history unfortunately.

You're absolutely right!
Post reply on HN