Looking Back at Speculative Decoding
research.google
Looking Back at Speculative Decoding
1–6 of 6 posts
Re: Looking Back at Speculative Decoding
#2Re: Looking Back at Speculative Decoding
#3I've been slightly annoyed by how the Speculative Decoding paper has gotten all the credit for the technique – I first learned about the technique from a paper more than a year older[1], Shallow Aggressive Decoding.
They introduce the same method, but apply it to grammatical error correction, meaning the "draft" output is just the input itself. The Speculative Decoding paper tries to emphasize differences between this and their method, saying that theirs is more general, as they apply it to more domains, allowing the draft to come from a smaller model, and extend it to allow sampling.
All of that is great, and deserves another paper, but doesn't deserve the credit for inventing and rights to rename the method, especially when they were aware of Shallow Aggressive Decoding before uploading their first draft.
Re: Looking Back at Speculative Decoding
#4I've been slightly annoyed by how the Speculative Decoding paper has gotten all the credit for the technique – I first learned about the technique from a paper more than a year older[1], Shallow Aggressive Decoding. They introduce the same method, but apply it to grammatical error correction, meaning the "draft" output is just the input itself. The Speculative Decoding paper tries to emphasize differences between thi…
I first saw the trick in 2009 when Linda Brown Westrick independently discovered it as a way to parallelize sequential sampling of MCMC chains.
Re: Looking Back at Speculative Decoding
#5I've been slightly annoyed by how the Speculative Decoding paper has gotten all the credit for the technique – I first learned about the technique from a paper more than a year older[1], Shallow Aggressive Decoding. They introduce the same method, but apply it to grammatical error correction, meaning the "draft" output is just the input itself. The Speculative Decoding paper tries to emphasize differences between thi…
I first saw the trick in 2009 when Linda Brown Westrick independently discovered it as a way to parallelize sequential sampling of MCMC chains.
Do you have a good reference I can read up on? I'd love to learn a bit more and update my mental "citation"
Re: Looking Back at Speculative Decoding
#6Earlier quoted context omitted.
I first saw the trick in 2009 when Linda Brown Westrick independently discovered it as a way to parallelize sequential sampling of MCMC chains.
Do you have a good reference I can read up on? I'd love to learn a bit more and update my mental "citation"
unpublished, i'm sure the idea has come up many times