Is it remarkably difficult? I thought it is expectedly difficult… 🤷

Defeating Nondeterminism in LLM Inference.

Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models.