PyTorch - get the total number of model parameter

Total number of model parameters

• https://stackoverflow.com/questions/49201236/check-the-total-number-of-parameters-in-a-pytorch-model
• https://discuss.pytorch.org/t/how-do-i-check-the-number-of-parameters-of-a-model/4325

1. simple version

pytorch_total_params = sum(p.numel() for p in model.parameters())

2. listed version

def count_parameters(model):

  str_name = "name"

  str_parameter = "parameter"

  print(f"{str_name:50s}: {str_parameter:10s}")

  total_params = 0

  for name, parameter in model.named_parameters():

    if not parameter.requires_grad:

      continue

    params = parameter.numel()

    print(f"{name:50s}: {params:10s}")

    total_params += params

  print(f"Total Trainable Params: {total_params}")

  return total_params

What is the 7B model's 7B?

• https://medium.com/@baicenxiao/some-basic-knowledge-of-llm-parameters-and-memory-estimation-b25c713c3bd8

For instance, the Mistral-7B-BF16 model would need memory equal to the number of parameters multiplied by the size of the type: 7 billion parameters * 2 bytes = 14 billion bytes. Thus, 14 billion bytes = 14 * 1,000 * 1,000 * 1,000 / 1024 / 1024 / 1024 ≈ 13 GB (considering 1000/1024)³ ≈ 0.93.