Faster inference won't save you
graphcoder.ai
Faster inference won't save you
1–5 of 5 posts
Re: Faster inference won't save you
#2The latency table says it all. Cloud-to-cloud is 40ms for 20 turns. Hotel Wi-Fi is 16 seconds. You can halve inference time and still have a broken product on bad connections.
Re: Faster inference won't save you
#3The latency table says it all. Cloud-to-cloud is 40ms for 20 turns. Hotel Wi-Fi is 16 seconds. You can halve inference time and still have a broken product on bad connections.
is this an LLM?
Re: Faster inference won't save you
#4does this mean you can disconnect from the internet entirely with the agent loop still running?
Re: Faster inference won't save you
#5does this mean you can disconnect from the internet entirely with the agent loop still running?
yes this is central to our thesis :)