Show HN: LunaLora: Multi-LoRA System to Combat Catastrophic Forgetting
1–5 of 5 posts
Re: Show HN: LunaLora: Multi-LoRA System to Combat Catastrophic Forgetting
#2This is more of a conceptual idea from an AI hobbyist; hopefully, the big claims for motivation are not too distracting. After doing too many basic-level tutorials, this could be an interesting intermediate-level project to apply modern AI architectures. What is your opinion?
Re: Show HN: LunaLora: Multi-LoRA System to Combat Catastrophic Forgetting
#3[flagged]
Re: Show HN: LunaLora: Multi-LoRA System to Combat Catastrophic Forgetting
#4[flagged]
Just to understand what you mean, you are worried that there are 2 or more LoRAs that have almost identical content but are distinct only in timestamps. Yeah, that could be a problem...
Hmm..., perhaps the similarity for recency does need to be incorporated through orthogonality, but can somehow be implemented through weight preference, such that the new weight is influenced by the weight history.
Thanks, that is a very good point, will take that into consideration!
Re: Show HN: LunaLora: Multi-LoRA System to Combat Catastrophic Forgetting
#5[flagged]
Just to understand what you mean, you are worried that there are 2 or more LoRAs that have almost identical content but are distinct only in timestamps. Yeah, that could be a problem... Hmm..., perhaps the similarity for recency does need to be incorporated through orthogonality, but can somehow be implemented through weight preference, such that the new weight is influenced by the weight history. Thanks, that is a v…
[flagged]