Earlier quoted context omitted.
They can be deterministic. We did this at Groq, if you sent a request with exactly the same input token, seed and temperature value you would get precisely the same result every time. This is harder to do on other architectures that themselves aren't fully deterministic though.
Do you mean that you get the exact same result (as word for word) because from Groq's documentation, it says that determinism is best effort, and also it's not guaranteed across model versions : https://console.groq.com/docs/prompting?_sm_vck=tMW2S7k17245... My understanding is that it's best effort because even if you control the seed/temperature, the model version and hardware specs you still have floating point ma…
The TLDR here is that Groq LPUs are completely deterministic hardware, including for floating point.
Disclaimer: Formerly of Groq, now of NVIDIA, still working on LPUs.