Quantization FP16, FP8, or INT8

In this article, you can download many papers.

Floating-point arithmetic for AI inference — hit or miss?

• FP8 versus INT8 for efficient deep learning inference
• With Shared Microexponents, A Little Shifting Goes a Long Way
• The case for 4-bit precision: k-bit Inference Scaling Laws