Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spaced repetition systems have gotten better

domenic.me

Post reply on HN
Print view View original

141–150 of 538 posts

Re: Spaced repetition systems have gotten better

#141

Post by TheDong »

ix101 wrote: post #138
TheDong wrote: post #124

Earlier quoted context omitted.

> graded by an LLM This seems impossible to me. In anki, there's "hard", "good", and "easy" which are all for "I got this right". For my usage, "hard" is "I got it right, but I was only like 60% sure", "good" is "I had to actively think", and "easy" is "effortlessly correct, no real thought required". There's no way for an AI to tell if my identical input is the result of a 50/50 guess, or a little thought, or effort…

One way it could grade you automatically is by the speed of flipping the card (or entering the correct answer). If it took less than a second to confirm then evidently it was easy.

But conversely, if I alt-tabbed to chat with a friend, or paused studying because the person sitting next to me asked a question, or I took a sip from my coffee mug, that doesn't mean it's hard necessarily. Even though all of those take at least as much time as answering a hard card un-interrupted would.

The AI cannot read my mind, there is no approximation that will work reasonably accurately here for "how confident was I in my answer", unless I input that myself.

Re: Spaced repetition systems have gotten better

#142

Post by david_allison »

sunkcapital wrote: post #132

This is something I’ve been tackling myself in the language app I’m making https://store.steampowered.com/app/3220820/Bilingual_Crosswo... . Right now, I’ve added a set of front loaded intervals: 2M, 5M, 10M, 20M, 40M, 2H, 6H, 1D, 2D, 4D, 8D, and so on eventually stretching to a full year. I’ve always felt this setup was a bit arbitrary and considered it a temporary solution. Thanks for saving me some time on researc…

Take a look at https://github.com/open-spaced-repetition/

As a quick hack, increase the factor to 2.5 once you reach 1 day. That's what Anki's SM-2 used to use (if only pressing 'good')

Re: Spaced repetition systems have gotten better

#143

Post by JackDanMeier »

ix101 wrote: post #138
TheDong wrote: post #124

Earlier quoted context omitted.

> graded by an LLM This seems impossible to me. In anki, there's "hard", "good", and "easy" which are all for "I got this right". For my usage, "hard" is "I got it right, but I was only like 60% sure", "good" is "I had to actively think", and "easy" is "effortlessly correct, no real thought required". There's no way for an AI to tell if my identical input is the result of a 50/50 guess, or a little thought, or effort…

One way it could grade you automatically is by the speed of flipping the card (or entering the correct answer). If it took less than a second to confirm then evidently it was easy.

It should definitely be added as a variable within the calculation, but the current FSRS predicts how likely you are to access the memory (if it's sufficiently available which is defined by its retrieval strength) and speed of retrieval isn't really a factor in this version. The different grades are more to define how well all parts of the memory is retrieved.

Not to say that how quickly you can access it doesn't play a role in real life.

Re: Spaced repetition systems have gotten better

#144

Post by TheDong »

JackDanMeier wrote: post #129
TheDong wrote: post #124

Earlier quoted context omitted.

> graded by an LLM This seems impossible to me. In anki, there's "hard", "good", and "easy" which are all for "I got this right". For my usage, "hard" is "I got it right, but I was only like 60% sure", "good" is "I had to actively think", and "easy" is "effortlessly correct, no real thought required". There's no way for an AI to tell if my identical input is the result of a 50/50 guess, or a little thought, or effort…

Yes, this is also something I have been thinking about, can an LLM really know how well I know something. There is the issue with the grading with again, hard, good and easy that I can cut myself some slack and say "I knew that" even when I didn't(and I have a strong memory of having done this myself). And there is the possibility of bullshitting the LLM and just all you know about the subject rather than the exact d…

Bullshitting the AI maliciously doesn't matter, if you don't want to study effectively, you won't study effectively, and that's not a problem for the app.

> any knowledge rather than specifying that the exact answer should be graded

I don't understand what you mean. The important thing is to feed back into the SRS algorithm "How much does this card need to be studied", and if you mean "any knowledge means we can study it less often", then I doubt the SRS will be able to be effective.

What are you suggesting to feed back into SRS? How will you ensure cards the user knows very well quickly get pushed way back (so the user isn't overwhelmed with a boring slog), and cards they only sorta know bubble up more quickly to start to cement the knowledge?

Re: Spaced repetition systems have gotten better

#145

Post by ix101 »

sunkcapital wrote: post #132

This is something I’ve been tackling myself in the language app I’m making https://store.steampowered.com/app/3220820/Bilingual_Crosswo... . Right now, I’ve added a set of front loaded intervals: 2M, 5M, 10M, 20M, 40M, 2H, 6H, 1D, 2D, 4D, 8D, and so on eventually stretching to a full year. I’ve always felt this setup was a bit arbitrary and considered it a temporary solution. Thanks for saving me some time on researc…

This looks like a fun app!

Re: Spaced repetition systems have gotten better

#146

Post by Muromec »

ix101 wrote: post #138
TheDong wrote: post #124

Earlier quoted context omitted.

> graded by an LLM This seems impossible to me. In anki, there's "hard", "good", and "easy" which are all for "I got this right". For my usage, "hard" is "I got it right, but I was only like 60% sure", "good" is "I had to actively think", and "easy" is "effortlessly correct, no real thought required". There's no way for an AI to tell if my identical input is the result of a 50/50 guess, or a little thought, or effort…

One way it could grade you automatically is by the speed of flipping the card (or entering the correct answer). If it took less than a second to confirm then evidently it was easy.

Whenever I try to use anki I can't figure what those four buttons actually mean, so I end up with 40 cards that I still can't recall and then the thing happily drops another 10 on top and I just delete the deck or the app. Haven't learned the thing I was trying to learn with it ever.

Either I don't understand the algorithm or it doesn't understand me.

Re: Spaced repetition systems have gotten better

#147

Post by nomadpenguin »

InkCanon wrote: post #95
bearjaws wrote: post #6

Spaced repetition has been all the rage for 20 years now. Dozens of apps, thousands of lectures, and it turns out its not really a silver bullet. There's nothing really wrong with it, it's just that people tend to fall off the same way they do on any other education pattern. A couple years ago I was thinking "If Google and Apple really cared about kids they would make a spaced repetition unlock system", where by you…

There's some UX problems of SRS (that I'm working on) that makes it high friction 1) Time taken to create cards 2) Need for self marking 3) Creates a one to one mapping of prompt-answer 4) If you're an autodidact, you have to teach yourself first (alternatively called understanding, scaffolding, etc) More fundamentally, SRS isn't a superpower because it's just very specific to creating a direct prompt retrieval. Gene…

Poor generalization (overtraining on prompts) and loss of context over time are the biggest issues I've found with them. Slow card creation workflows and needing to rate your own reviews are merely UX issues -- losing context and losing generalization make SRS actively harmful when used for some topics.

There's 2 solutions I've thought of but haven't tried implementing:

1. A free-recall based approach. Free recall allows you to operate at a higher level of organization and connect concepts at lower levels. However, how you would schedule SRS with free recall is not clear.

2. Have an LLM generate questions on-the-fly so that you don't overtrain on prompts. You might also instruct the LLM to create questions that connect multiple concepts together. The problem with this approach is that LLMs are still not so good at creating good test questions.

Re: Spaced repetition systems have gotten better

#148

Post by linux2647 »

The folks at SaySomethingIn, that originally started with Welsh and other Celtic languages, have recently expanded to Japanese. I haven’t tried it myself, but I’ve found some decent success with one of their other courses. It’s all about spaced repetition and focuses exclusively on listening and speaking.

https://www.saysomethingin.com/en/

Re: Spaced repetition systems have gotten better

#150

Post by JackDanMeier »

TheDong wrote: post #144
JackDanMeier wrote: post #129

Earlier quoted context omitted.

Yes, this is also something I have been thinking about, can an LLM really know how well I know something. There is the issue with the grading with again, hard, good and easy that I can cut myself some slack and say "I knew that" even when I didn't(and I have a strong memory of having done this myself). And there is the possibility of bullshitting the LLM and just all you know about the subject rather than the exact d…

Bullshitting the AI maliciously doesn't matter, if you don't want to study effectively, you won't study effectively, and that's not a problem for the app. > any knowledge rather than specifying that the exact answer should be graded I don't understand what you mean. The important thing is to feed back into the SRS algorithm "How much does this card need to be studied", and if you mean "any knowledge means we can stud…

My understanding of what is important to feedback into the SRS is, was I able to retrieve the memory, and does the representation in my memory align with what I recalled.

As an example Term: "What is the capital of France and how many inhabitants does it have?" Correct definition: "Paris, which has 2 000 000 inhabitants."

For me there is a difference in not having the answer at all, which falls into "again". But what about if I'm able to retrieve that Paris is the capital, but I remember that the population is 1 500 000. This is where the gray zone begins

Post reply on HN