TinyLoRA – Learning to Reason in 13 Parameters
1–10 of 54 posts
Re: TinyLoRA – Learning to Reason in 13 Parameters
#2With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
Re: TinyLoRA – Learning to Reason in 13 Parameters
#3With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
Except learning to reason is a far cry from curve fitting. Our brains have more than five parameters.
Re: TinyLoRA – Learning to Reason in 13 Parameters
#4With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
[deleted]
Re: TinyLoRA – Learning to Reason in 13 Parameters
#5With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
Except learning to reason is a far cry from curve fitting. Our brains have more than five parameters.
reasoning capability might just be some specific combinations of mirror neurons.
even some advanced math usually evolves applying patterns found elsewhere into new topics
Re: TinyLoRA – Learning to Reason in 13 Parameters
#6[dead]
Re: TinyLoRA – Learning to Reason in 13 Parameters
#7The quality of custom models trained with proper reasoning datasets[0] even with small parameters (3-7B is sweet spot) is incredible now
[0]: cartesien.io or Salesforce's WebscaleRL
Re: TinyLoRA – Learning to Reason in 13 Parameters
#8Re: TinyLoRA – Learning to Reason in 13 Parameters
#9With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
Except learning to reason is a far cry from curve fitting. Our brains have more than five parameters.
After a quick content browse, my understanding is this is more like with a very compressed diff vector, applied to a multi billion parameter model, the models could be 'retrained' to reason (score) better on a specific topic , e.g. math was used in the paper
Re: TinyLoRA – Learning to Reason in 13 Parameters
#10With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
Except learning to reason is a far cry from curve fitting. Our brains have more than five parameters.
I agree, I don't think gradient descent is going to work in the long run for the kind of luxurious & automated communist utopia the technocrats are promising everyone.