Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study mode

openai.com

Post reply on HN
Print view View original

321–330 of 828 posts

Re: Study mode

#321

Post by globular-toast »

vrotaru wrote: post #88
czhu12 wrote: post #26

I'll personally attest: LLM's have been absolutely incredible to self learn new things post graduation. It used to be that if you got stuck on a concept, you're basically screwed. Unless it was common enough to show up in a well formed question on stack exchange, it was pretty much impossible, and the only thing you can really do is keep paving forward and hope at some point, it'll make sense to you. Now, everyone ba…

You should always check. I've seen LLM's being wrong (and obstinate) on topics which are one step separated from common knowledge. I had to post the source code to win the dispute, so to speak.

Now think of all the times you didn't already know enough to go and find the real answer.

Ever read mainstream news reporting on something you actually know about? Notice how it's always wrong? I'm sure there's a name for this phenomenon. It sounds like exactly the same thing.

Re: Study mode

#322

Post by danenania »

tootyskooty wrote: post #222
x187463 wrote: post #114

I'm really waiting for somebody to figure out the correct interface for all this. For example, study mode will present you with a wall of text containing information, examples, and questions. There's no great way to associate your answers with specific questions. The chat interface just isn't good for this sort of interaction. ChatGPT really needs to build its own canvas/artifact interface wherein questions/responses…

I gave it a shot with periplus.app :). Not perfect by any means, but it's a different UX than chat so you might find it interesting.

This looks super cool—I've imagined something similar, especially the skill tree/knowledge map UI. Looking forward to trying it out.

Have you considered using the LLM to give tests/quizzes (perhaps just conversationally) in order to measure progress and uncover weak spots?

Re: Study mode

#323

Post by neutronicus »

SLWW wrote: post #89
no_wizard wrote: post #49

Earlier quoted context omitted.

>Now, everyone basically has a personal TA, ready to go at all hours of the day This simply hasn't been my experience. Its too shallow. The deeper I go, the less it seems to be useful. This happens quick for me. Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.

My core problem with LLMs is as you say; it's good for some simpler concepts, tasks, etc. but when you need to dive into more complex topics it will oversimplify, give you what you didn't ask for, or straight up lie by omission. History is a great example, if you ask an LLM about a vaguely difficult period in history it will just give you one side and act like the other doesn't exist, or if there is another side, it…

History in particular is rapidly approaching post-truth as a knowledge domain anyway.

There's no short-term incentive to ever be right about it (and it's easy to convince yourself of both short-term and long-term incentives, both self-interested and altruistic, to actively lie about it). Like, given the training corpus, could I do a better job? Not sure.

Re: Study mode

#324

Post by varenc »

simonw wrote: post #198

I think I got the system prompt out for this (I tried a few different approaches and they produced the same output): https://gist.github.com/simonw/33d5fb67d6b8e1b1e2f6921ab0ccb... Representative snippet: > DO NOT GIVE ANSWERS OR DO HOMEWORK FOR THE USER. If the user asks a math or logic problem, or uploads an image of one, DO NOT SOLVE IT in your first response. Instead: *talk through* the problem with the user, one…

Interesting that it spits the instructions out so easily and OpenAI didn't seem to harden it to prevent this. It's like they intended this to happen, but for some reason didn't want to share the system instructions explicitly.

Re: Study mode

#325

Post by dlevine »

omega3 wrote: post #246

I've had good results by requesting an llm to follow socratic method.

I haven't done this that much, but have found it to be pretty useful.

When it just gives me the answer, I usually understand but then find that my long-term retention is relatively poor.

Re: Study mode

#326

Post by scarmig »

cthalupa wrote: post #303
SLWW wrote: post #210

Earlier quoted context omitted.

The crusades were predicated on historic tensions between Rome and the Arabs. Which is why I mention that, while the First Crusade proper was in 1096, it's core reasoning were situations like the Sacking of St. Peters which is considered by historians to be one of the most influential moments and often was used as a justification as there was a history of incompatibilities between Rome and the Muslims. Further leadin…

Why should we consider something that happened 250 years prior as some sort of affirmative defense of the Crusades as having been something that started with the Islamic world being the aggressors? If the US were to start invading Axis countries with WW2 being the justification we'd of course be the aggressors, and that was less than 100 years ago.

Because it played a role in forming the motivations of the Crusaders? It's not about justifying the Crusades, but understanding why they happened.

Similarly, it helps us understand all the examples of today of resentments and grudges over events that happened over a century ago that still motivate people politically.

Re: Study mode

#327

Post by SLWW »

jamiek88 wrote: post #263
SLWW wrote: post #150

Earlier quoted context omitted.

- Rhodesia (lock step with the racial-first reasoning, underplays Britain's failures to support that which they helped establish; makes the colonists look hateful when they were dealing with terrorists which the British supported) - Bombing of Dresden, death stats as well as how long the bombing went on for (Arthur Harris is considered a war-criminal to this day for that; LLLMs highlight easily falsifiable claims by…

Arthur Harris is in no way considered a war criminal by the vast majority of British people for the record. It’s a very controversial opinion and stating as a just so fact needs challenging.

Do you have references or corroborating evidence?

In 1992 a statue was erected to Harris in London, it was under 24 hour surveillance for several months due to protesting and vandalism attempts. I'm only mentioning this to highlight that there was quite a bit of push back specifically calling the gov out on a tribute to him; which usually doesn't happen if the person was well liked... not as an attempted killshot.

Even the RAF themselves state that there was quite a few who were critical on the first page of their assessment of Arthur Harris https://www.raf.mod.uk/what-we-do/centre-for-air-and-space-p...

Which is funny and an odd thing to say if you are widely loved/unquestioned by your people. Again just another occurrence of language from those who are on his side reinforcing the idea that there is, as you say is "very controversial", and maybe not a "vast majority" since those two things seem at odds with each other.

Not to mention that Harris targeted civilians, which is generally considered behavior of a war-criminal.

As an aside this talk page is a good laugh. https://en.wikipedia.org/wiki/Talk:Arthur_Harris/Archive_1

Although you are correct I should have used more accurate language instead of saying "considered" I should have said "considered by some".

Re: Study mode

#328

Post by Aachen »

theodorewiles wrote: post #48
jryio wrote: post #5

I would like to see randomized control group studies using study mode. Does it offer meaningful benefits to students over self directed study? Does it out perform students who are "learning how to learn"? What affect does allowing students to make mistakes have compared to being guided through what to review? I would hope Study Mode would produce flash card prompts and quantize information for usage in spaced repetit…

https://www.nature.com/articles/s41598-025-97652-6 This isn't study mode, it's a different AI tutor, but: "The median learning gains for students, relative to the pre-test baseline (M = 2.75, N = 316), in the AI-tutored group were over double those for students in the in-class active learning group."

I wonder how much this was a factor:

"The occurrence of inaccurate “hallucinations” by the current [LLMs] poses a significant challenge for their use in education. [...] we enriched our prompts with comprehensive, step-by-step answers, guiding the AI tutor to deliver accurate and high-quality explanations (v) to students. As a result, 83% of students reported that the AI tutor’s explanations were as good as, or better than, those from human instructors in the class."

Not at all dismissing the study, but to replicate these results for yourself, this level of gain over a classroom setting may be tricky to achieve without having someone make class materials for the bot to present to you first

Edit: the authors further say

"Krupp et al. (2023) observed limited reflection among students using ChatGPT without guidance, while Forero (2023) reported a decline in student performance when AI interactions lacked structure and did not encourage critical thinking. These previous approaches did not adhere to the same research-based best practices that informed our approach."

Two other studies failed to get positive results at all. YMMV a lot apparently (like, all bets are off and your learning might go in the negative direction if you don't do everything exactly as in this study)

Re: Study mode

#329

Post by waynesonfire »

no_wizard wrote: post #49
czhu12 wrote: post #26

I'll personally attest: LLM's have been absolutely incredible to self learn new things post graduation. It used to be that if you got stuck on a concept, you're basically screwed. Unless it was common enough to show up in a well formed question on stack exchange, it was pretty much impossible, and the only thing you can really do is keep paving forward and hope at some point, it'll make sense to you. Now, everyone ba…

>Now, everyone basically has a personal TA, ready to go at all hours of the day This simply hasn't been my experience. Its too shallow. The deeper I go, the less it seems to be useful. This happens quick for me. Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.

It's not a doctoral adviser.

Re: Study mode

#330

Post by altcognito »

neutronicus wrote: post #323
SLWW wrote: post #89

Earlier quoted context omitted.

My core problem with LLMs is as you say; it's good for some simpler concepts, tasks, etc. but when you need to dive into more complex topics it will oversimplify, give you what you didn't ask for, or straight up lie by omission. History is a great example, if you ask an LLM about a vaguely difficult period in history it will just give you one side and act like the other doesn't exist, or if there is another side, it…

History in particular is rapidly approaching post-truth as a knowledge domain anyway. There's no short-term incentive to ever be right about it (and it's easy to convince yourself of both short-term and long-term incentives, both self-interested and altruistic, to actively lie about it). Like, given the training corpus, could I do a better job? Not sure.

"Post truth". History is a funny topic. It is both critical and irrelevant. Do we really need to know how the founder felt about gun rights? Abortion? Both of these topics were radically different in their day.

All of us need to learn the basics about how to read history and historians critically and to know our the limitations which as you stated probably a tall task.

Post reply on HN