Contents

White paper covering the most common issues related to NVIDIA GPUs.

• 1. Introduction
• 2. Floating Point
  • 2.1. Formats
  • 2.2. Operations and Accuracy
  • 2.3. The Fused Multiply-Add (FMA)
• 3. Dot Product: An Accuracy Example
  • 3.1. Example Algorithms
  • 3.2. Comparison
• 4. CUDA and Floating Point
  • 4.1. Compute Capability 2.0 and Above
  • 4.2. Rounding Modes
  • 4.3. Controlling Fused Multiply-add
  • 4.4. Compiler Flags
  • 4.5. Differences from x86
• 5. Considerations for a Heterogeneous World
  • 5.1. Mathematical Function Accuracy
  • 5.2. x87 and SSE
  • 5.3. Core Counts
  • 5.4. Verifying GPU Results
• 6. Concrete Recommendations
• 7. Acknowledgements
• 8. References
• 9. Notices
  • 9.1. Notice
  • 9.2. OpenCL
  • 9.3. Trademarks