I tested speculative decoding on my home GPU cluster. Here's why it didn't help.

· Dev.to

Read full story at source