presetsΒΆ

Convenient factories for common compression recipes. Each preset returns a ready-to-use config that can be further refined by chaining set_module_type(), set_module_name(), only_for(), or without().

Preset

Description

w4()

int4 weight-only quantization, per-channel symmetric.

w4_per_block()

int4 weight-only quantization, per-block symmetric, block_size defaults to 32.

w8()

int8 weight-only quantization, per-channel symmetric.