LLM RAM Calculator

Calculate VRAM/RAM requirements for running large language models

Model Selection

Select Model

Context Length

Batch Size

8B

Parameters

32

Layers

4096

Hidden

Memory Analysis8B params, ~5.5 GB INT4

Memory Requirements

Quant	Model	+KV	Total
FP32 (Full)	32.0 GB	~4.0 GB	~39.6 GB
FP16 (Half)	16.0 GB	~2.0 GB	~19.8 GB
INT8 (8-bit)	8.0 GB	~1.0 GB	~9.9 GB
INT4 (4-bit)	4.0 GB	~1.0 GB	~5.5 GB

Will It Fit?

RTX 3060 12GB

FP16INT8INT4

RTX 3070 8GB

FP16INT8INT4

RTX 3080 10GB

FP16INT8INT4

RTX 3090 24GB

FP16INT8INT4

RTX 4060 8GB

FP16INT8INT4

RTX 4070 12GB

FP16INT8INT4

RTX 4080 16GB

FP16INT8INT4

RTX 4090 24GB

FP16INT8INT4

A100 40GB

FP16INT8INT4

A100 80GB

FP16INT8INT4

H100 80GB

FP16INT8INT4

Apple M1 8GB

FP16INT8INT4

Apple M1 16GB

FP16INT8INT4

Apple M2 Max 32GB

FP16INT8INT4

Apple M2 Max 96GB

FP16INT8INT4

Apple M3 Max 48GB

FP16INT8INT4

Apple M3 Max 128GB

FP16INT8INT4

ClickSelect Model