Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Homework

stratechery.com

Post reply on HN
Print view View original

211–220 of 296 posts

Re: AI Homework

#211

Post by joe_the_user »

otikik wrote: post #102

One of my subjects at highchool was technical drawing. The teacher was a quite old man who was set on his ways. We would have to draw 3D pieces by hand, using our squares, bevels and compasses, and lining everything with an ink pencil afterwards, and he gave us a lot of homework to do, which he examined for imperfections. I was incredibly bad a it. I would start a drawing, only to discover halfway through it that it…

To put it with maximum extremity, should a student be able to turn-in a traced photograph for their homework in a life drawing class?

I mean, human beings have produced a lot of tools for aiding themselves in tasks that were previously done by hand. Whether a given hand-skill should be learned and preserved kind of has to decided on a case-by-case basis. I learned cursive in school and I wouldn't object to that skill being utterly forgotten by humanity.

Which is to say that I don't think we should assume all learning should begin without automatic tools at hand or that we should suppress automatic tools. Rather, we need to look carefully at which hand-skills are worthwhile and "uplifting" and how much use of those skills should be required.

Re: AI Homework

#212

Post by pixl97 »

joe_the_user wrote: post #204
ako wrote: post #169

Earlier quoted context omitted.

I asked ChatGPT how to write texts so it can be identified as human generated. The answer: "One way to make it clear that a text is not generated by AI is to include a joke or riddle that is difficult for a machine to understand or generate. For example, you could include a joke that relies on wordplay or a complex pun. Here is an example: Why was the math book sad? Because it had too many problems."

It's everyone choice - odd, ironic or appropriate - what to think of how GPT's answers to keeps getting quoted on the question of how to detect it. The thing is, I'd see these answers as similar to everything else the program produces. A bunch of claims from the net cobbled together - I've read a number of Sci-fi novels and stories where "inability to understand humor" is the distinguishing quality of an AI (I'm gues…

>A reasonable answer is that the AI's output tends to involve this running-together of common rhetorical devices along with false and/or contradictory claims within them.

The question here is this an actual AI only failure mode. Are we detecting AI, or just bullshittery?

Re: AI Homework

#213

Post by magusdei »

guelo wrote: post #203
micromacrofoot wrote: post #168

Earlier quoted context omitted.

GPT-3 definitely has a writing style that can be detected by AI. But here's the thing: it's really easy to change its writing style. All you have to do is give it different input data and boom, you've got a whole new writing style.

That's not easy, it costs millions of dollars to obtain large amounts of clean data and retrain those models.

No, it would take approximately 3 minutes where you either write a paragraph in the desired style yourself or paste one from the Internet and then ask it to continue in that style. Even if you decided to go the more onerous fine-tuning route, it would require 1-2mb of text, cost very little, and you'd be done in a few hours. It's easy.

Re: AI Homework

#214

Post by hansihe »

guelo wrote: post #203
micromacrofoot wrote: post #168

Earlier quoted context omitted.

GPT-3 definitely has a writing style that can be detected by AI. But here's the thing: it's really easy to change its writing style. All you have to do is give it different input data and boom, you've got a whole new writing style.

That's not easy, it costs millions of dollars to obtain large amounts of clean data and retrain those models.

But you don't really have to retrain the whole model to modify it's writing style, fine tuning would probably be enough

Re: AI Homework

#215

Post by joe_the_user »

sterlind wrote: post #164
pclmulqdq wrote: post #139

Earlier quoted context omitted.

GPT-3 definitely has a writing style. I assume it would be detectable, ironically, by an AI.

if it's detectable by an AI, you can GAN it into erasing its detectable writing style.

I think you could make the GPT's more undetectable by a computer using GAN techniques but that might well make the thing more detectable by a person. The GPT-style is something like rhetorical devices run together to make various claims unfold smoothly. An alternative approach could make the things inconsistency more obvious to a person.

Re: AI Homework

#216

Post by frandroid »

omnimus wrote: post #132
martin-adams wrote: post #121

Earlier quoted context omitted.

How easy do you think it is to spot when AI is being used?

Today? In images? Quite easy. In text? Pretty hard. And if you do some cheking and editing then pretty impossible? In two years? Look where the space was two years ago. I think many things will have to change.

An amusing part with all of the AI-generated graphics is that it's pretty bad at producing the right amount of fingers, among other details. This one really jumps out. I'm assuming that will be ironed out eventually but the devil, as always, remains in the details.

Re: AI Homework

#217

Post by visarga »

adverbly wrote: post #83

Something I've noticed about chat ais vs direct search is that because a chat ai is a blackbox, I can't dig into an answer's source at all. With a search, I can read the wikipedia sources or I know the forum, or the poster. But with an AI, it's a dead end and it sucks. I can ask people how they know something, but AI training data is invisible. It doesn't know how it knows something. There is this oracle with a 10% c…

> AI training data is invisible. It doesn't know how it knows something

You should be accustomed to being surprised by AI. There is of course a new kind of transformer that takes input a query and outputs document IDs. Like a search engine retriever and ranker all packed into a neural net, very fast and efficient. So you can take any paragraph generated by the model and attribute it to the training set. This could be used to implement verification or retrieval augmented generation.

A Neural Corpus Indexer for Document Retrieval

https://arxiv.org/abs/2206.02743

Re: AI Homework

#218

Post by jltsiren »

UncleEntity wrote: post #130
JW_00000 wrote: post #92

Earlier quoted context omitted.

Agreed. In fact, this is the same line of reasoning my teachers used in elementary school when we asked them "Why do we need to learn how to add/subtract/multiply/divide, when calculators can do it automatically for us?" And the answer is still the same: you need to have done it the "manual" way before you can understand what the machine is doing for you. I only understand what a compiler is doing after having writte…

> I only understand what a compiler is doing after having written one (albeit a tiny one) myself during my studies. How many people need to understand how a compiler works to sling some code for a paycheck or just for fun. I learned how compilers work purely for fun, it provides me absolutely no benefit in my day to day life, and before that I taught myself how to code for the exact same reason. If I had to go the op…

> How many people need to understand how a compiler works to sling some code for a paycheck

Only the ones who want a good job.

One of the consequences of automation has been a widening gap between top jobs and ordinary jobs. It's plausible that AI will continue the same process. Many jobs that pay well and are intellectually and socially rewarding today may become similar to airport security checks in the future. You observe what a machine does, double-checking its conclusions and looking for anything out of ordinary. And you often fail at that, because the human brain is poorly suited for such tasks.

When the future is uncertain, its good to have a deeper and wider understanding of things than what your current job requires.

Re: AI Homework

#219

Post by flatline »

drchickensalad wrote: post #202
flatline wrote: post #53

Earlier quoted context omitted.

I do not think it could. What I see GPT doing here is generating a lot of plausible boilerplate. We already have this via things like code snippets. I find them to be useless, like autocorrect on my phone. It gets in the way of my reasoning and does not really enhance it in any way. Sometimes I make mistakes typing but I’d rather them be my honest mistakes than the computer censoring/rewriting my thoughts. Good engin…

didn't we also think that good chess and go play required good reasoning skills? is this just a higher scale version of that?

I don’t think anyone knows. I gave it the famous syllogism:

> All men are mortal > Socrates is a man > Is socrates mortal

To which it gave a very detailed and correct reply. I then tried:

> All cats are white > Sam is a cat > Is sam white?

To which it gave an almost identically worded response that was nonsensical.

I personally do not think it is the size of the model in question, it is that the things it does that appear to reflect the output of human cognition are just an echo or reflection. It is not a generalizable solution: there will always be some novel question it is not trained against and for which it will fall down. If you make those vanishingly small, I don’t know, maybe you will have effectively compressed all human knowledge into the model and have a good-enough solution. That’s one way of looking at an NN. But the problem is fundamentally different than chess.

I think this composed with more specialized models for things like identifying and solving math and logic problems could make something that truly represents what I think people are seeing the potential in this. Something that encodes the structure behind these concepts, is extensible, and has a powerful generative function would be really neat.

Re: AI Homework

#220

Post by DrNosferatu »

ChatGPT is quite constrained now.

Yesterday it wrote me a PSG chip emulator in Python and driver code to play The Good The Bad & The Ugly theme song. The song was initially quite off, but I managed to iteratively get it pretty close. Impossible to have it output even a fraction of that now.

How are you Hackers currently faring?

Post reply on HN