Is the reversal curse in LLMs real?

I doubt MS would put their service in lot of their products.

Conditions I mean: 고양이 => ; cat => ; either [고양이 => cat || cat => 고양이]. Under these conditions 고양이 == cat && cat == 고양이 should be known.

Your conditions: 고양이 ~> ; cat => ; [ 고양이 -> cat && cat ~> ...고양이?] (or something, trying to express you're preferentially learned one direction, usually mapping back to mother tongue while learning)

By no means are we strictly invariant. But I'd say that a good measure of your skills are in fact to be invariant. But also like you are pointing out, we also kinda train ourselves that way because it creates stronger pathways. Training methods are important too! And augmentation is critical to both human learning and machine learning. They're specifically critical to generalization abilities. You'll also see this in humans. I notice test focused educations tend (there's no "always") to create less generalization of concepts as you're over learning a thing presented in a specific way rather than focusing on the concept as a whole. Form of metric hacking if you will.

Side note: if you have tips for an extreme beginner learning Korean I'd really appreciate them. Trying to reduce the burden on my gf having to translate everything for me when we're around other Koreans.

For more information, please see https://people.math.harvard.edu/~ctm/home/text/others/shanno...

If you have learned Hangul, then you can start with Lingodeer(app), TTMIK(talk to me in Korean)’s Grammar books, or Billie's beginner playlist - https://www.youtube.com/watch?v=sx0yyQqkpqo&list=PLbFrQnW0BN....

These are all extremely beginner friendly. You can just look at the options and choose whichever works best for you. I personally tried lingodeer and ttmik both. I did a bit of lingodeer and then ttmik levels 1 to 3. I didn't try Billie's playlist, but he's a great teacher, so I recommend it if you think that would work better for you.

So in the beginning stages, it doesn't really matter so much, but eventually you're going to have to start thinking about how you’ll be learning the big 6, vocabulary and grammar (what I’ve spoken on so far covers grammar and TTMIK takes you pretty far in grammar if you stick with it) then speaking, listening, reading, writing, because believe me, there isn’t as much transfer learning as you would expect or hope. I mean, you don't have to stress over it or anything, just keep in mind that you have to eventually think about how you're going to address those things, and the sooner, reasonably, the better.

So this is kind of where my training diverges from most traditional approaches. I know I used Anki from my example and a lot of learners use that with pre made decks from the community for vocabulary, but honestly, I just couldn't use it then.

Part of it was because at the time, learning Korean words was like drawing blood for stone. For me, it was so hard. Even five words a day could be a challenge. My brain was like, “what the hell are you doing?”. It just wasn't going in.

Come to think of it, you know grokking in NN’s? Learning a language, you quickly realize the same phenomenon can occur in our biological counterparts. It’s like one day I was struggling to learn these new words, I don’t do anything drastic or different and suddenly the next day, learning new words felt so much easier, it’s crazy. Progress isn’t necessarily linear that’s for sure.

Part of it was also because I just didn't like the Anki experience, memorizing hundreds, thousands of words in isolation. It just wasn't really doing much for me. So, I just kind of dropped it. If traditional anki works for you then go for it but gin, you’ll need to figure out the remining 4. No matter how much isolated grammar and vocabulary you learn, you’ll need to practice doing each of those things for it to sink in.

The typical process is to learn a few thousand words and up to intermediate grammar before sinking into the 4. Makes things a lot less frustrating.

What I found was this amazing audiobook series. It's a graded reader series. 100 books with audio, all free(start from the oldest). https://audioclip.naver.com/channels/57.

Here the books are further divided into stories. At first, each story is really a few pages but it does get longer and longer as the series progresses. graded readers are specifically simplified literature created to help language learners progress.

It's also so that you can read in your target language much earlier than you otherwise would have because, well, you might see a lot of people say, “oh, just read, just watch or read children's shows or children's books.” That's a trap because like, children are native speakers man. Don't make the mistake of thinking that they aren't.

The complexity of the stuff they read or watch or whatever, is well beyond the beginner learner. So if you go into children's media and learning some few hundred words thinking, “oh, this should be easy, then, yeah, it may become very demoralizing because it won’t be in fact easy”.

And just to clarify, I mean children’s stuff not baby stuff. Yeah, baby stuff is simple, but you're going to get bored of that very, very fast.

So, yeah, it's a graded reader series and the idea is just to get started. Simplified stories you might find more interesting than baby stuff.

So, I really like reading. And I don't know, I just felt like it would be better because, you see, what Anki does is SRS, spaced repetition system, right? But when you think about reading, it's really a natural form of that. So, I just felt like, okay, this is what I'm going to try.

So, I was even thinking, oh, I could use this to knock almost all the birds with one stone, you know, it’s vocab, grammar, reading, and listening practice at the same time.

But the book series, it starts with a pretty high level. I don't know if you're familiar with the CEFR levels. It starts at about a low B1(bottom end of low intermediate) or so and ends at a high C1(top end of low advanced).

So, yeah, coming from where I was, where I knew at best, maybe 100 or 200 words and jumping in, it was difficult. And honestly, I would not have been able to do it without Mirinae. So, Mirinae is a grammar parser - https://mirinae.io/.

And what it does is, so basically, Korean and English are so far apart that you could know the meaning of every word in a sentence, and you would be absolutely lost as to the meaning of the sentence itself. So, what Mirinae did, or basically took what would have been impossible for me and made it achievable. I learnt a lot of grammar this way. You place the sentence in, and it would basically parse the grammar on the sentence.

You could see the explanations of different grammar parts, different nouns, objects, stuff like that, how this affected this and all that.

It was very, very, incredibly helpful. To to this day, I think that at least for distant languages, a good grammar parser is by far the most useful tool, better than a dictionary, better than just a translator.

So, the first story, I was pasting every sentence. So, it took me about a week to get through it, a few hours every day. The amazing thing was that you could the progress, the reduction in time it takes you to complete things, because the second story took only a few days in comparison. By the third or fourth story, I think it took only a few hours. I mean, I didn't finish it in a day, but I could have if I wasn’t a bit lazy.

So, yeah, that's the benefit of graded readers. Whether you start this as early as I did or not, it’s genuinely an amazing resource. Earlier this year, I came across this Anki deck series with natural audio -https://ankiweb.net/shared/by-author/374470252

What the guy managed to do was, he took a machine learning Korean speech dataset, and he kind of configured a lot of the sentences, which were, you know, a single woman speaking sentences, about 12,000 or so of those sentences. So, he reduced it to about 7,000, and he rearranged, as much as possible to be i+1 comprehensible. i+1 is just basically the idea that you come across a sentence, and there’s only one word you don't know.

So, it's much easier to grasp the meaning of that sentence quickly and easily. Then the next sentence, only one word, then the next sentence, only one word, then, and so on and so on. Now, this deck isn't perfectly i+1. But going through it, it's very good. So, you know, you could also explore that too after the beginner grammar resources, I suggested.

I’ve also noticed these guys, https://umiapp.co/.ly. It’s an app tailored for learning words in context, comprehensible input style. I really liked the looks of what I saw from the other languages. Korean isn’t there yet but is coming soon. By the time you’re ready for it, it’s hopefully there.

Anyway, best of luck.