Quantization of a CNN with 4 bit

claudio137 · February 12, 2025, 1:44pm

Hi, I need to quantize a small cnn. After the training I would like to see weights and bias quantized with 4 bit precision. I’m using Tensorflow model optimization but I always see floating point at the end like many other libraries. With Tensorflow lite I can see 8 bit precision for weights while bias remaining 32 bit.

Can you help me suggesting a way to solve this problem? Any help is welcome.

Thank you so much for your attention.

Kiran_Sai_Ramineni · February 13, 2025, 6:54am

Hi @claudio137, At present 4 bit qutization is not available in tensorflow. Only int8, float16 are minimum quantization supported. Thank You.

Topic		Replies	Views
Quantizing the models using tflite General Discussion models , keras , tflite , post-training	2	338	January 4, 2024
Is it possible to use 8 bit integers instead of floating point numbers? General Discussion model_optimization , help_request	1	1100	April 20, 2022
Post-training quantization. Where to learn more? General Discussion tflite , education , help_request	3	1261	June 5, 2024
Custom quantization aware training with lround during int8 multiplications TensorFlow models	0	575	October 26, 2022
How does full integer quantization work? Micro tflite , help_request	1	1155	March 8, 2024

Quantization of a CNN with 4 bit

Related topics