Live data from Hacker News

Anki and GPT-3

andrewjudson.com

51–60 of 80 posts

Re: Anki and GPT-3

#51

Earlier quoted context omitted.

Anki's key benefits stem from its utilization of two highly supported and extensively researched pedagogical strategies: (1) retrieval practice and (2) spaced repetition. In contrast, the practice of note-taking, which essentially entails summarizing information in one's own words, has been found to have 'low utility' in academic literature. [0] [0]: https://sci-hub.ru/https://pubmed.ncbi.nlm.nih.gov/26173288/

Depends on the note-taking strategy. From the same paper you cited, section 8.4 Practice testing - Issues for implementation, pages 34-35: > Another merit of practice testing is that it can be implemented with minimal training. Students can engage in recall-based self-testing in a relatively straightforward fashion. For example, students can self-test via cued recall by creating flashcards (free and low-cost flashcar…

Founder of StudyWand.com here, who received a 15k grant to develop an AI generating flashcard app in 2020 after an earlier prototype.

We've found students more consistently study ready-made cards that are at desirable difficulty (they get about 80% correct) and which are segmented by topic (e.g. semantic grouping of flashcards to tackle "one lesson at a time" like Duolingo). Students would prefer to use pre-made flashcards by other students in their class, then AI flashcards, then create and use their own.

There is limited evidence by Roediger and Karpicke who are the forefathers of retrieval practise that creating cards is also important. Frank Leeming (2002 study Exam-a-day) also showed that motivation when studying is peaked when you ask just a few questions a day, but every working day.

Now one of the vital benefits of retrieval practise with AI over creating your own cards is foresight bias - not mentioned yet in this thread - the fact that particularly in some subjects like Physics, students don't know what they don't know (watch this amazing Veritasium video, it also explains why misconceptions are so handy for learning physics): https://www.youtube.com/watch?v=eVtCO84MDj8 - basically, if you use AI quizzes (or any prepared subject-specific right/wrong system), you learn quickly where your knowledge sits and what to focus on, and reduce your exam stress. If you just sit their making quizzes, firstly you make questions on things you already know, you overestimate how much you can learn, and you consolidate on your existing strengths, and avoid identifying your own knowledge gaps until later on, which is less effective.

--

To quote from my dissertation experiment on background reading for retrieval practise, the end is about foresight bias a little: Retrieval practice – typically, quizzing - is an exceedingly effective studying mechanism (Roediger & Karpicke, 2006; Roediger & Butler 2011; Bae, Therriault & Redifer, 2017, see Binks 2018 for a review), although underutilized relative to recorded merit, with students vastly preferring to read content (Karpicke & Butler, 2009; Toppino and Cohen, 2009). Notably mature students do engage in practice quizzes more than younger students (Tullis & Maddox, 2020). Undertaking a Quiz (Retrieval practice) can enhance test scores significantly, including web-based quizzes (Daniel & Broida, 2017). Roediger & Karpicke (2006) analysed whether students who solely read content would score differently to students who took a practice quiz, one week after a 5-minute learning session. Students retained information to a higher level in memory after a week with the quiz (56% retained), versus without (42%), despite having read the content less (average 3.4 times) than the control, read-only group (14.2 times). Participants subjectively report preference for regular Quizzing (Leeming, 2002) over final exams, when assessed with the quiz results, with 81% and 83% of participants in two intervention classes recommending Leemings “Exam-a-day” procedure for the next semester, which runs against intuition that students might biases against more exams/quizzes (due to Test Anxiety). Retrieval Practice may increase performance via increasing cognitive load which is generally correlated with score outcomes in (multimedia) learning (Muller et al, 2008). Without adequate alternative stimuli, volume of content could influence results, thus differentiated conditions to control for this possible confound are required when exploring retrieval practice effects (as seen in Renkl 2010 and implemented in Methods). Retrieval practice in middle and high school students can reduce Test Anxiety, when operationalised by “nervousness” (Agarwal et al 2014), though presently no research appears to have analysed the influence of retrieval practice on university students’ Test Anxiety. Quizzing can alleviate foresight bias – overestimation of required studying time – in terms of students appropriately assigning a greater, more realistic study time plan (Soderstrom & Bjork, 2014). Despite the underutilization noted by Karpicke and Butler (2009), quizzing is becoming more common in burgeoning eLearning courses, supported by the research (i.e. Johnson & Johnson, 2006; Leeming, 2002; Glass et al. 2008) demonstrating efficacy in real exam performance.

Re: Anki and GPT-3

#52

This is a great idea. This is what note taking is lacking! When you pile up 200 notes or 2.000 ... you can push them into GPT to generate anki cards so that you can remember and make sense of it all.

As the founder, I would love to hear your thoughts on StudyWand.com, which does this for PDFs/youtube videos, and has greater factual accuracy than base GPT models by quoting the source documents (thanks to a university grant): https://studywand.com/images/splash/getStudyingGIF.gif

Re: Anki and GPT-3

#54
post #17

From my experience, automatically generated flashcards are often disappointing. Recently I've been experimenting with the use of GitHub copilot for accelerating cards creation and I can wholeheartedly recommend this approach, I've noticed at least 2x speed up in the process without sacrificing cards quality

Can you explain your process in more depth?

Re: Anki and GPT-3

#55

Just a side note on Anki - it sounds cool, but how is it possible that there are zero screenshots on the product page? Furthermore, it has a super busy (/intimidating-to-non-techies) download section, and the only real descriptive text is on the linked docs/help site, which is so dense that it is completely overwhelming to anybody who is still in "convince me" mode. The project would probably have 10x the number of u…

This is one of the reasons i made Mochi[0]. I couldn’t understand how this was the best recommendation for new people. Anki can be very useful and powerful but it is so user unfriendly as to be down right hostile.

(Plus Mochi has GPT-3 built in [1])

[0] https://mochi.cards/

[1] https://twitter.com/MochiCardsApp/status/1603569008493711361

Re: Anki and GPT-3

#56
post #46

Just a side note on Anki - it sounds cool, but how is it possible that there are zero screenshots on the product page? Furthermore, it has a super busy (/intimidating-to-non-techies) download section, and the only real descriptive text is on the linked docs/help site, which is so dense that it is completely overwhelming to anybody who is still in "convince me" mode. The project would probably have 10x the number of u…

Anki has been around for a decade, almost two. Several comercial competitors came and went over the years. I suppose it tells you something about the learning tools market.

I initially discovered it ~10 yrs ago as an alternative for Memrise back when all of Memrise courses were still free. UI/UX wise it seemed outdated-ish but overall very powerful. Glad to see this still going strong as FOSS.

Re: Anki and GPT-3

#57
post #55

Just a side note on Anki - it sounds cool, but how is it possible that there are zero screenshots on the product page? Furthermore, it has a super busy (/intimidating-to-non-techies) download section, and the only real descriptive text is on the linked docs/help site, which is so dense that it is completely overwhelming to anybody who is still in "convince me" mode. The project would probably have 10x the number of u…

This is one of the reasons i made Mochi[0]. I couldn’t understand how this was the best recommendation for new people. Anki can be very useful and powerful but it is so user unfriendly as to be down right hostile. (Plus Mochi has GPT-3 built in [1]) [0] https://mochi.cards/ [1] https://twitter.com/MochiCardsApp/status/1603569008493711361

I've been using Mochie for while now, I tried some of the others out there and settled on Mochi as the best.

Re: Anki and GPT-3

#58

As a daily Anki user, I am highly skeptical of this idea. The careful creation of new cards is one of the key parts in learning the material. It promotes a critical thinking step where you have to distill the material down into small but important pieces and (if you want to do it well) forces you to evaluate if you have actually understood what you wrote down. I suspect that forgoing this step will not lead to unders…

What you say is the commonly accepted advice, and the ideal workflow. But the truth is that I have been using a huge premade deck to learn Chinese for the last 3 years, and it works.

Would I learn more if I created my own cards? Sure, but studying the premade deck takes me 30 minutes per day, which I can afford. I cannot afford the time to make and update a custom deck.

Re: Anki and GPT-3

#59
post #55

Just a side note on Anki - it sounds cool, but how is it possible that there are zero screenshots on the product page? Furthermore, it has a super busy (/intimidating-to-non-techies) download section, and the only real descriptive text is on the linked docs/help site, which is so dense that it is completely overwhelming to anybody who is still in "convince me" mode. The project would probably have 10x the number of u…

This is one of the reasons i made Mochi[0]. I couldn’t understand how this was the best recommendation for new people. Anki can be very useful and powerful but it is so user unfriendly as to be down right hostile. (Plus Mochi has GPT-3 built in [1]) [0] https://mochi.cards/ [1] https://twitter.com/MochiCardsApp/status/1603569008493711361

This looks excellent. My one hesitation with moving over is whether or not there is a way to export back to Anki should your tool ever cease to exist.

SRS is a life long game, so I don’t want to lose everything if Mochi shuts down in 5, 10, 20 years (no business lasts forever). Whereas with Anki I know I’ll be able to use it forever.

Re: Anki and GPT-3

#60
post #55

Earlier quoted context omitted.

This is one of the reasons i made Mochi[0]. I couldn’t understand how this was the best recommendation for new people. Anki can be very useful and powerful but it is so user unfriendly as to be down right hostile. (Plus Mochi has GPT-3 built in [1]) [0] https://mochi.cards/ [1] https://twitter.com/MochiCardsApp/status/1603569008493711361

This looks excellent. My one hesitation with moving over is whether or not there is a way to export back to Anki should your tool ever cease to exist. SRS is a life long game, so I don’t want to lose everything if Mochi shuts down in 5, 10, 20 years (no business lasts forever). Whereas with Anki I know I’ll be able to use it forever.

I don't have an export to Anki format specifically, but you can export to JSON (the JSON format is documented on the website, unlike Anki's, which had to be reverse engineered) or markdown, plus all of the data is stored locally on your computer in a SQLite database.
Post reply on HN