Gemma 2 on AWS Lambda with Llamafile
unremarkable.ai
Gemma 2 on AWS Lambda with Llamafile
1–4 of 4 posts
Re: Gemma 2 on AWS Lambda with Llamafile
#2A small experiment to see if we are there yet with highly virtualized CPU compute and Small Language Models (SLM). The answer is a resounding maybe, but most likely not. Huge thanks to Justine for her work on Llamafile supported by Mozilla. Hope folks find this R&D useful.
Re: Gemma 2 on AWS Lambda with Llamafile
#3A small experiment to see if we are there yet with highly virtualized CPU compute and Small Language Models (SLM). The answer is a resounding maybe, but most likely not. Huge thanks to Justine for her work on Llamafile supported by Mozilla. Hope folks find this R&D useful.
Can you expound a bit about why not?
Does it produce bad results? Is it slow to respond? Slow to load?
I've been wanting to play around with llamafile-based edge functions but storing even small models in GitHub (for automated deploys) is a terrible and often impossible experience.
Re: Gemma 2 on AWS Lambda with Llamafile
#4This is great work. Has anyone used it enough to compare the lambda costs with the cost of running a comparable model on, say, OpenAI?