Quantization FP16, FP8, or INT8

In this article, you can download many papers.

Floating-point arithmetic for AI inference — hit or miss?

• FP8 versus INT8 for efficient deep learning inference
• With Shared Microexponents, A Little Shifting Goes a Long Way
• The case for 4-bit precision: k-bit Inference Scaling Laws

About LoRA

LoRA (Low-Rank Adaptation)

- Paper

[Current]

[Before]

• Measuring the Intrinsic Dimension of Objective Landscapes, 2018
• Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning, 2020 

[After]

Towards A Unified View of Parameter-Efficient Transfer Learning, 2022

---- 

- YouTube

 https://www.youtube.com/watch?v=dA-NhCtrrVE

 https://www.youtube.com/watch?v=BJqwmDpa0wM

https://www.youtube.com/watch?v=t509sv5MT0w

----

The case for 4-bit precision: k-bit Interence Scaling Laws

Parameter-Efficient Transfer Learning for NLP

Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning

QLoRA: Efficient Finetuning of Quantized LLMs

https://www.youtube.com/watch?v=X4VvO3G6_vw

----

Intrinsic Dimensionality Ezplanins the Effectiveness of Language Model Fine-Tuning