AI Homework • Hacker News Board

Re: AI Homework

#171

Post by somethoughts » Mon, Dec 05, 2022, 7:46 PM UTC

adverbly wrote: post #83

Something I've noticed about chat ais vs direct search is that because a chat ai is a blackbox, I can't dig into an answer's source at all. With a search, I can read the wikipedia sources or I know the forum, or the poster. But with an AI, it's a dead end and it sucks. I can ask people how they know something, but AI training data is invisible. It doesn't know how it knows something. There is this oracle with a 10% c…

I think another important aspect overlooked is that there probably will be AI engine optimization consultants once any AI engine gains popularity - similar to Search Engine Optimziation consultants. The original Google page rank system worked well in the beginning - prior to SEO - but now is largely not used.

AI engine optimization consultants will figure out how to game the system - likely targeting the training data sources.

Re: AI Homework

#172

Post by nashashmi » Mon, Dec 05, 2022, 7:46 PM UTC

borroka wrote: post #158

nashashmi wrote: post #127

Earlier quoted context omitted.

I don't know much about technical drawings by hand. But I am very experienced with AutoCAD. And I cannot appreciate much of what CAD has. On the contrary, having met senior people who took technical drawing, they could appreciate it very much so and were able to teach me concepts that I had a hard time understanding before. Nowadays, when I teach very young people what AutoCAD is, I make them use a mini rolling ruler…

I disagree. Senior professionals might have a better understanding because they have spent a lot of time drawing, but the specific tool is not relevant. It is like saying that playing tennis with a wooden racket gives you a better "feel" for the ball and therefore is preparatory to using modern rackets. No one does this because it is not true. A similar line of thinking was popular in jiu jitsu, where veterans said t…

That is definitely true, that knowing and understanding a tool is not important to actually succeeding with the tool. But it does help a confused person understand why tools are made the way they are.

In Fact, Modern programs in CAD do not follow the tools and principles that AutoCAD emphasizes. And those programs have charted a different course where the history and legacy of technical drawings has less relevance. (Case in point: Visio and maybe Powerpoint.)

Re: AI Homework

#173

Post by cromwellian » Mon, Dec 05, 2022, 7:46 PM UTC

alain94040 wrote: post #107

cromwellian wrote: post #29

Earlier quoted context omitted.

If you ask it about things which require deduction like Math, even simple Math questions like multiply binomials or solve a quadratic it gets it totally wrong, confidently, and even if you correct it, it often still gets it wrong. It’s not even close to something like Wolfram Alpha. I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like…

Sure. But if you combine the understanding that this chatbot has with a Wolfram Alpha backend, you could build an even more amazing system. I'm sure someone is working on hooking up language models to math backends (anywhere from a simple calculator to Wolfram Alpha).

DeepMind published a system that does sort this with a backend theorem prover a year ago. My point is, I don’t think transformer based text prediction systems are the right model here. I could be wrong, but it think about how formal systems work, they seem a far cry from what decoder architectures are doing.

https://www.nature.com/articles/s41586-021-04086-x

Re: AI Homework

#174

Post by darkwizard42 » Mon, Dec 05, 2022, 7:46 PM UTC

TeMPOraL wrote: post #119

moe091 wrote: post #110

Earlier quoted context omitted.

that's a good point, I just made a comment next to yours responding to the same original comment about whether homework should exist at all, and I think you're idea that it is a way to extend the "learning time" of students without requiring more time and resources from teachers is a good counter-point. If you have the time to read my other comment, what would you think about the idea of only making homework mandator…

About your idea, I'm not sure either way. On the surface it makes sense, and there probably is a real correlation between test performance and amount of after-school learning, so it would make sense to make homework mandatory for kids who scored low. It would also work as punishment, but at the same time reinforcing the kids' perception of homework as punishment. What makes me really unsure, however, is parents. Fami…

I resonate a lot with what you said. Homework or in the way it is used today as reinforcement work is most needed for those who have trouble picking up new concepts during limited instruction time. However, those who have trouble picking up concepts quickly will also have overlap with not having much time/resources outside of live instruction time. That just leads to a double whammy in terms of how homework further disadvantages them.

In the reading I've done, I've seen most advocate for smaller class sizes with more educators per class to help provide more 1:1 attention. This is again at odds with how public schools are funded where I am (US) so I don't know if anything will ever change.

My personal experience is homework was an excellent way to reinforce what was taught in school. For me, it felt especially useful in quantitative subjects and languages (French & Spanish) which both felt like they benefitted from having a concept stick. For qualitative subjects like writing, reading comprehension, I actually now look back and see homework as a way for teachers to see samples of your progress. "After reading this passage, did you learn how to construct a summary? Did you learn how to pull the author's main message? Did you learn how to make a convincing argument for or against the passage's main point" and I can't think of a fast way to do this in any kind of live instruction setting.

Re: AI Homework

#175

Post by adamsmith143 » Mon, Dec 05, 2022, 7:47 PM UTC

ghaff wrote: post #69

adamsmith143 wrote: post #65

Earlier quoted context omitted.

Ironically inputting: "This is human generated text." generated an 86% probability of being fake text. But it seems kind of dubious to begin with, I'm not sure why LLM output would be so easily detected.

One short sentence isn't enough. If you play with it a bit, there are definitely patterns. Here's one I did over the weekend: 99.98% fake. The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France. The English arm…

Are there any articles explaining how this works? What exactly is the classifier keying in on that indicates LLM text so reliably?

Re: AI Homework

#176

Post by darkwizard42 » Mon, Dec 05, 2022, 7:48 PM UTC

gattilorenz wrote: post #98

J253 wrote: post #95

As easy solution to this is the just not give any credit for homework that goes toward a final grade. Have a number of in person closed book tests that are difficult enough to prove satisfactory understanding of the topic. Homework is just a mechanism for a teacher/professor to force a student to dig into and really understand a subject. If the student can use these AI tools to subvert that, then instead of wasting e…

I personally prefer open-book exams: they discourage cheating in terms of “smuggling definitions by writing them on a calculator”, and force teachers to ask questions that require reasoning and deeper understanding

Agreed, but there are some levels of study where open-book just doesn't work well.

I think it would be hard to ask any kind of pre-college mathematics questions that don't become incredibly easy with an open book. The other challenge with open-book exams which focus on deeper understanding and reasoning is they still require understanding fundamentals to be successful, so how do you test for fundamentals?

Re: AI Homework

#177

Post by [deleted] » Mon, Dec 05, 2022, 7:50 PM UTC

killerdhmo wrote: post #150

notdang wrote: post #148

After I corrected the chat bot: Q: this is wrong, John Locke believed in separation of powers, Hobbes was a proponent of absolutism. A: I apologize if my response was not accurate. Thomas Hobbes was indeed a proponent of absolutism, which is the belief that a single, central authority should have complete control over a country. Hobbes believed that this central authority should be an absolute monarch, who would have…

You’d think it would be smart enough to know that for this particular question, the details of the answers have not changed since 2021.

[deleted]

Re: AI Homework

#178

Post by BeetleB » Mon, Dec 05, 2022, 7:50 PM UTC

monklu wrote: post #115

IanCal wrote: post #89

Earlier quoted context omitted.

You're not trapped in there, because you're entirely free to go and research yourself. You can look up what it's telling you. It's no more trapping than talking to a stranger who seems to be knowledgeable about a subject but doesn't hand you a list of references.

Not exactly though. With a human stranger, I can still stereotype based on their appearance, background, accents, etc. and apply whatever mental adjustments as taught to me by my societal upbringing. With an "AI" bot, the "strangers" are faceless people who curated the training sets and wrote the obscure statistical algorithms.

> With an "AI" bot, the "strangers" are faceless people who curated the training sets and wrote the obscure statistical algorithms.

I think this is a feature over:

> I can still stereotype based on their appearance, background, accents, etc. and apply whatever mental adjustments as taught to me by my societal upbringing.

Re: AI Homework

#179

Post by benatkin » Mon, Dec 05, 2022, 7:52 PM UTC

Here's a silly answer, where it acts like it's asked a question in a political debate. The first sentence doesn't address the question at all.

"What Cryptocurrencies are there?

Cryptocurrency is a type of digital or virtual currency that uses cryptography for security and is generally decentralized, meaning it is not controlled by a single entity. Some examples of cryptocurrencies include Bitcoin, Ethereum, and Litecoin. There are many different cryptocurrencies that have been developed, and new ones are continually being created. Some cryptocurrencies are designed to function as a medium of exchange, while others are created to provide a specific service or function, such as a platform for smart contracts. Because the field of cryptocurrencies is constantly evolving, it is difficult to provide a comprehensive list of all the cryptocurrencies that exist."

Re: AI Homework

#180

Post by hansvm » Mon, Dec 05, 2022, 7:53 PM UTC

darkwizard42 wrote: post #160

hansvm wrote: post #155

Making homework more secure seems like the wrong approach. The whole purpose of a grade is to certify fitness for the next course/year/job. Homework is exceptionally poor at that task even without AI, and overloading it with grades almost invariably makes it worse at its primary task of better educating the student. Larger projects will still need to reckon with AI somehow, but would it not make more sense to just pu…

I don't think homework is meant to educate the student. It is meant to help push repetition and mastery of concepts. It is likely very difficult for most students to master physics, mathematics, or other quantitative subjects with just 5 hours of instruction a week in person (and the large class sizes in the US and other countries probably don't help).

I disagree slightly (explanation below), but whether you take "educate" to mean some sort of deep exploration or just rote repetition, I think the the point still stands.

The disagreement comes from n=1 anecdata drawn from my own education, but after 8th grade or so nearly all my homework was designed to complement rather than reinforce in-class instruction. Picking on mathematics as one of the things you mentioned, the in-class instruction would introduce a concept, and the homework would give you the opportunity to flounder a bit and probe the bounds of that concept. I particularly enjoyed questions like "the author made a mistake in this question; how would you correct it to be answerable?" In some sense that's repeating a concept, but when you're doing something more than repeating the same technique over and over there's a sense in which it's different too.