Live data from Hacker News

Anki and GPT-3

andrewjudson.com

71–80 of 80 posts

Re: Anki and GPT-3

#71

Earlier quoted context omitted.

Thank you. I am afraid I don't know and couldn't easily find any research. I did find this post, but it looks like SEO spam a little, and doesn't cite the essay claim: https://www.bookishelf.com/the-importance-of-information-ret... I'll ask Prof Roediger as we occasionally communicate and will aim to get back to you. However, I wouldn't be surprised if any correlation was not statistically significant.

One more thing. When you mentioned Test Anxiety it reminded me of frequent comments about anxiety during job interviews. I'm wondering if flashcards (or other retrieval practice) could help there too. Perhaps you can spin it into a product for professionals.

What kind of flashcards would help there for you?

[edit: removed a point about variance which wasn't supported by more recent literate]

High stress (or anxiety) happens only periodically... like those job interviews you mention. It varies massively within individual subjects. There are some ways to reduce anxiety, but I wouldn't say anything has been conclusive yet. Here is an excerpt from my Dissertation about Test Anxiety... I found practise quizzes had a non-significant overall effect on Test Anxiety, and reduced only the "Tenseness" subscale of a relatively old measure sadly. The CTAS scale, referenced below, would probably be most useful for the job interview case.

"Test Anxiety is the additional stress felt when you must provide answers. It is prevalent, affecting students from preschool (McDonald, 2001) to taught university level, and across demographics (Beidel et al 1994; Beer 1991; Mwamwenda,1993). Prevalence is estimated at 20% (Ergene 2003) to 25% (Thomas et al, 2018) in current students. Test Anxiety often involves pressure, highlighting a social, non-genetic nature, and high variance is recorded between subjects (e.g. Keith et al 2010 longitudinal study showed low individual variation, but high between subjects variation). Overwhelming Test Anxiety significantly impairs wellbeing (Steinmayr et al., 2016). Interventions and online course design choices can markedly reduce Test Anxiety (Abdous, 2019). Unfortunately, few online learning studies record Test Anxiety, instead recording grade/score differences. Yet, Test Anxiety interventions are efficacious – e.g. "test-wiseness training" (Kalechstein et al., 1988), group counselling (Alkhawaja, 2013) and mindfulness training (Seidi & Ahmad, 2018). Research often classifies students into High and Low groups. There are disproportionately negative outcomes for High Test Anxiety students, whom more recently have been enrolled automatically in targeted interventions (e.g. Psychoeducation Bedel et al., 2020). Test Anxiety is related to Social Anxiety (Rothman, 2004; Sarason and Sarason, 1990). This manifests in a socialevaluative component, which appears in most scales, such as the Test Anxiety Inventory (TAI; Spielberger, 1980). TAI was designed for assessing children (Ludlow & Guida, 1991). Recent scales, including the Cognitive Test Anxiety Scale (CTAS; Cassady & Johnson, 2002), are adapted to university participants. An example CTAS item is "I feel under a lot of pressure to get good grades on tests.". CTAS has similar correlations with academic performance (Chapell et al., 2005) as those utilizing TAI (such as Hunsley, 1985), without the concerns of TAI applicability discussed in Szafranski (2012). Test Anxiety is typically construed as a trait under Latent state-trait theory, despite short term experimental intervention differences being observed (e.g. "looking ahead" immediately prior to testing in Mavilidi et al., 2014; or taking practice quizzes in Agarwal et al., 2014). Test Anxiety should not be confused with other forms of anxiety, such as General or Social Anxiety Disorder. Prevalence of anxiety is rising in college populations (Zagorski, 2018), as is loneliness, with 1 in 5 young people reporting having no close friends and feeling alone(n=2,522, Mental Health Foundation, 2019). Usage of social media is also associated with greater loneliness (Wohn & LaRose, 2014). However, artificially induced status updates have been shown to reduce it (Deters & Mehl, 2012), which aligns with status updates requesting academic support reducing Test Anxiety as found in Deloatch et al (2017) and findings linking higher wellbeing with greater academic outcomes (Public Health England, 2014). Research on social anxiety and social media is less clear, although recently Erliksson et al (2020) correlated greater internet usage and activity with increased social anxiety. Despite wide research on the negative aspects of social media usage (e.g. for university students, see Odacı & Kalkan, 2010) - little attention has been paid to online helping, which is fundamentally social."

Re: Anki and GPT-3

#72

Just a side note on Anki - it sounds cool, but how is it possible that there are zero screenshots on the product page? Furthermore, it has a super busy (/intimidating-to-non-techies) download section, and the only real descriptive text is on the linked docs/help site, which is so dense that it is completely overwhelming to anybody who is still in "convince me" mode. The project would probably have 10x the number of u…

the thing is, the difficulty associated with actually using the app on a daily basis is very high. you can try to optimize for top of funnel but then you end up with a toy app like Duolingo that can acquire a bunch of users but doesn't actually fulfill the learning goals very well.

Re: Anki and GPT-3

#73

Why would you use LLM to create Anki cards when you can use said LLM to obtain the answer in the first place, eliminating the need to recall the information you are trying to remember with Anki?

why learn anything at all if you can just ask the LLM?

Re: Anki and GPT-3

#74
post #14

As a daily Anki user, I am highly skeptical of this idea. The careful creation of new cards is one of the key parts in learning the material. It promotes a critical thinking step where you have to distill the material down into small but important pieces and (if you want to do it well) forces you to evaluate if you have actually understood what you wrote down. I suspect that forgoing this step will not lead to unders…

It's a classic beginner mistake to load up a 20000 word deck and start learning it when you have no relationship or connection to the cards. Create a card when you first see the word in real life, with that example sentence on the back, and it's so much easier to remember. At least for me anyway.

How do you find the time/energy to get the example sentence on the back of each card? Do you have a process/method to make it easier?

When I encounter new words in real life, it tends to be all at once (i.e. watching a show while making dinner). I scribble the word quickly on a nearby notepad and don't have time to look at it again until I'm bulk-uploading new words later. It bothers me that I'm maybe not getting the full benefit of Anki, because I'm missing the context and/or example sentences.

I would love to hear ideas about how to resolve that timing issue.

I seem to retain more from those 20k word decks you mentioned than my personal decks, likely because they come with example sentences and context that my personal decks are missing.

Re: Anki and GPT-3

#75
post #73

Why would you use LLM to create Anki cards when you can use said LLM to obtain the answer in the first place, eliminating the need to recall the information you are trying to remember with Anki?

why learn anything at all if you can just ask the LLM?

That’s the next logical step in the same argument.

Re: Anki and GPT-3

#76

As a daily Anki user, I am highly skeptical of this idea. The careful creation of new cards is one of the key parts in learning the material. It promotes a critical thinking step where you have to distill the material down into small but important pieces and (if you want to do it well) forces you to evaluate if you have actually understood what you wrote down. I suspect that forgoing this step will not lead to unders…

Anki's key benefits stem from its utilization of two highly supported and extensively researched pedagogical strategies: (1) retrieval practice and (2) spaced repetition. In contrast, the practice of note-taking, which essentially entails summarizing information in one's own words, has been found to have 'low utility' in academic literature. [0] [0]: https://sci-hub.ru/https://pubmed.ncbi.nlm.nih.gov/26173288/

As others already pointed out, the linked article seems to say the reverse of how you interpreted it (though I would agree the terms are a bit ambiguous).

Note taking as parrotting is distinct from note taking as distillation. The latter has much more chance of getting results with proper spaced repetition, since it will help establish a concrete foundation on which to build more knowledge in an easily retrievable manner. The former may or may not benefit from spaced repetition at all.

I would say if GPT distills successfully, while this isnt as good as self-distillation, it may still be useful.

If not, or worse, actively bullshits, it probably won't be much help.

Re: Anki and GPT-3

#77
post #55

Just a side note on Anki - it sounds cool, but how is it possible that there are zero screenshots on the product page? Furthermore, it has a super busy (/intimidating-to-non-techies) download section, and the only real descriptive text is on the linked docs/help site, which is so dense that it is completely overwhelming to anybody who is still in "convince me" mode. The project would probably have 10x the number of u…

This is one of the reasons i made Mochi[0]. I couldn’t understand how this was the best recommendation for new people. Anki can be very useful and powerful but it is so user unfriendly as to be down right hostile. (Plus Mochi has GPT-3 built in [1]) [0] https://mochi.cards/ [1] https://twitter.com/MochiCardsApp/status/1603569008493711361

Your site makes my cpu sweat with Firefox 108 on linux. Seems the problem are the animations on the page. I disable that and there is no problem.

The product does seems good. I may give it a try.

Re: Anki and GPT-3

#78

Earlier quoted context omitted.

Depends on the note-taking strategy. From the same paper you cited, section 8.4 Practice testing - Issues for implementation, pages 34-35: > Another merit of practice testing is that it can be implemented with minimal training. Students can engage in recall-based self-testing in a relatively straightforward fashion. For example, students can self-test via cued recall by creating flashcards (free and low-cost flashcar…

Founder of StudyWand.com here, who received a 15k grant to develop an AI generating flashcard app in 2020 after an earlier prototype. We've found students more consistently study ready-made cards that are at desirable difficulty (they get about 80% correct) and which are segmented by topic (e.g. semantic grouping of flashcards to tackle "one lesson at a time" like Duolingo). Students would prefer to use pre-made flas…

I'm a long term (10 year+) user of Supermemo (and general fan of SRS stuff) and finally got around to checking out StudyWand today. This is the best experience I've had making flashcards and general study material ever. Hands down, nothing I've seen comes close.

It's wild because StudyWand took my sample notes and did everything that I would have done with them if I was going to use them to get a good grade in school. I was expecting some semi-decent cloze generated cards but got much more.

Literally, when I was in college I would take notes in class, and then spend about 20-40 minutes post class doing almost exactly what StudyWand does. The classes that I bothered doing that for, I always got a good grade in, nearly effortlessly. The hardest part was making the notes.

The part of this that I'm actually excited about is that this tool also works with any sort of documentation. For example, I can clean up any reference page from MDN as a PDF and get a usable (like actually well-made flashcards) set of 15-20 flashcards for it. Oh, you also get summaries and multiple essay questions too. The only way this would be better is if it gave you cloze deletions that were actual sample code to fill in the blank with.

I didn't really like your intro so I took a few days longer than I normally would have to look at your software (I normally check out every SRS software I see on HN). This software is insane. The value is so, so, so ridiculous. I half-hearted uploaded one poorly made PDF of a webpage and got flashcards that are comparable to what I would make as a 10+ SRS user. I almost stopped doing the initial reviews halfway through and looked for a way to pay for this.

Outside of Supermemo, this is the only other SRS software that I've seen that's worth my money. The hardest part is going to be convincing all my younger family members to actually use this. I've tried so many times to get people to use Anki (Supermemo won't happen), and they just don't get it. I think StudyWand might be able to bridge that gap. I'm going to try and see.

Re: Anki and GPT-3

#79
post #78

Earlier quoted context omitted.

Founder of StudyWand.com here, who received a 15k grant to develop an AI generating flashcard app in 2020 after an earlier prototype. We've found students more consistently study ready-made cards that are at desirable difficulty (they get about 80% correct) and which are segmented by topic (e.g. semantic grouping of flashcards to tackle "one lesson at a time" like Duolingo). Students would prefer to use pre-made flas…

I'm a long term (10 year+) user of Supermemo (and general fan of SRS stuff) and finally got around to checking out StudyWand today. This is the best experience I've had making flashcards and general study material ever. Hands down, nothing I've seen comes close. It's wild because StudyWand took my sample notes and did everything that I would have done with them if I was going to use them to get a good grade in school…

Whoa, awesome to see a fellow SuperMemo user here! I've been going for 17 years, I absolutely can't imagine life without SuperMemo and I've stopped trying to sell my friends on it. Have you tried integrating Study Wand with SuperMemo at all? I'm always on the lookout for ways to maximize my information intake (Aside from SuperMemo, which already does a nearly perfect job at it).

Re: Anki and GPT-3

#80
post #78

Earlier quoted context omitted.

I'm a long term (10 year+) user of Supermemo (and general fan of SRS stuff) and finally got around to checking out StudyWand today. This is the best experience I've had making flashcards and general study material ever. Hands down, nothing I've seen comes close. It's wild because StudyWand took my sample notes and did everything that I would have done with them if I was going to use them to get a good grade in school…

Whoa, awesome to see a fellow SuperMemo user here! I've been going for 17 years, I absolutely can't imagine life without SuperMemo and I've stopped trying to sell my friends on it. Have you tried integrating Study Wand with SuperMemo at all? I'm always on the lookout for ways to maximize my information intake (Aside from SuperMemo, which already does a nearly perfect job at it).

Nothing automated yet, but I may plan to. I don't regularly add that many new cards to my collection to where it'd be an immediate benefit for me. I may add more with StudyWand since it seems to do a good enough job of creating cards. Incremental reading is the primary way I add new material to my collection, but I don't really fully process articles that much into items these days. Most of my SuperMemo use is using incremental reading to ensure I always have something interesting to read as well as tasklists for planning/ideas. I have a lot of stuff that I like to revisit or review, which I find using SuperMemo great for too.
Post reply on HN