Launch HN: Issen (YC F24) – Personal AI language tutor

Very interesting, but 20 minutes seems very low to fully try a language learning app. would you consider extending to a few hours?

The cultural/political issues are deeper than you'd think, there are some within Catalonia that would claim all of us Catalan speakers as part of the same cultural umbrella, negating our own cultural background as derivative.

That's why I say that I personally (as a Mallorcan) would much prefer the language to be associated with a Spanish flag than a Catalan flag. Some people would agree and some would disagree – ultimately the best thing is to not link flags to languages anyway.

To give you another example from Issen, they're also using the American flag for English which is probably even more controversial :)

Langua is a nice app with more features than ISSEN. We focus on the voice chats, and I think our tech is better (intelligence, latency, realtime flow, voices, etc), especially for non english/spanish.

We are aiming to create a long term companion/tutor that gets to know you more and more, and can create customized curriculums, lessons, etc.

> We wanted something that would talk with us — realistically, in full conversations — and actually help us improve. So we built it ourselves. The app relies on a custom voice AI pipeline combining STT (speech-to-text), TTS (text-to-speech), LLMs, long term memory, interruptions, turn-taking, etc. Getting speech-to-text to work well for learners was one of the hardest parts — especially with accents, multi-lingual sentences, and noisy environments. We now combine Gemini Flash, Whisper, Scribe, and GPT-4o-transcribe to minimize errors and keep the conversation flowing.

Your prompt can't do this. I know because I've been trying to build something similar and a prompt just isn't enough. You need multiple LLMs and custom code working together to achieve realistic conversations.

I've been thinking and playing slightly with this concept myself. A few thoughts:

1. Using a standard transcription service is pretty tricky because it's going to correct the user's speech. Or make it incorrect! Standard transcription is predicated on the speaker saying things correctly.

2. I've tried sending the audio directly to OpenAI to address this issue. I can't say if it works or not. It's very hard to test or understand a system without a transcript as a source of truth!

3. I'd like to learn a new language as a beginner, and all of these AI systems work poorly for this. It's great to immerse the learner in the language, but if you know NOTHING then it's not that helpful.

4. Language learning needs to be MUCH more multimodal than a standard chat. Especially as a beginner.

5. The AI should be generating translations and explanations alongside its responses. I'd like to be able to inspect everything the AI says (in the language I'm learning) to understand it.

6. Emoji would be another easy way to annotate the text.

7. I think giving the user/AI a subject to talk about would be helpful. Again, a subject that is not language-based would be great, like an image or something.

8. As a very new learner I would like an experience where I respond in my native language and then I'm told how to translate this to the language I'm learning. This should include a pronunciation guide. Then I should repeat the phrase I'm given.

9. I should still be able to ask questions in my native language and probably get a response in my native language. But with some prompting the AI should be able to distinguish these two cases.

10. For low latency it's nice if you produce the spoken text quickly, but you still have the opportunity to get the LLM to produce _more_ material immediately after. This is where things like translations can be produced.

11. You probably don't have timestamps on your TTS, but if you did and could highlight words as they were spoken that would be _great_. Probably worth choosing a TTS provider with that in mind.