Quantization of a CNN with 4 bit

claudio137 · February 12, 2025, 1:44pm

Hi, I need to quantize a small cnn. After the training I would like to see weights and bias quantized with 4 bit precision. I’m using Tensorflow model optimization but I always see floating point at the end like many other libraries. With Tensorflow lite I can see 8 bit precision for weights while bias remaining 32 bit.

Can you help me suggesting a way to solve this problem? Any help is welcome.

Thank you so much for your attention.

Kiran_Sai_Ramineni · February 13, 2025, 6:54am

Hi @claudio137, At present 4 bit qutization is not available in tensorflow. Only int8, float16 are minimum quantization supported. Thank You.

Topic		Replies	Views
Quantizing the models using tflite General Discussion models , keras , tflite , post-training	2	334	January 4, 2024
Is it possible to use 8 bit integers instead of floating point numbers? General Discussion model_optimization , help_request	1	1099	April 20, 2022
QAT training: convert input/output to 8 bits instead of float32 General Discussion models , keras , tflite	7	1399	May 16, 2023
How are Conv layer (and others layers) outputs quantized? General Discussion tflite , help_request	1	762	September 5, 2021
Quantization aware training -> In/Output still float32? General Discussion models , keras , model_optimization , help_request	1	1593	May 13, 2022

Quantization of a CNN with 4 bit

Related topics