How to quantize a custom model(not of tf.keras.Model type)?

Felicity_Wang · May 11, 2022, 2:12am

Hi,
I’m trying to perform quantization-aware training on a customized model that’s not of tf.keras.Model type. It’s got its own forward(), loss(), and trainable_variables() functions for the optimizer to apply gradients on. Has anyone had any experience running quantization aware training in such a scenario?

Thank you.

Kiran_Sai_Ramineni · March 6, 2023, 5:08pm

Hi @Felicity_Wang, To perform QAT on custom layers you have to apply tfmot.quantization.keras.quantize_annotate_layer to the CustomLayer and pass in the QuantizeConfig. For more details please refer to this document. Thank You.

Topic		Replies	Views
Quantization Aware Training using tensorflow_model_optimization General Discussion models , tflite , model_optimization	4	1363	May 2, 2024
Model optimization custom layers support General Discussion model_optimization	1	423	October 9, 2024
Transfer learning and Quantization aware training. Subclassed model General Discussion models , keras , learning , model_optimization , help_request	2	3140	December 3, 2021
Custom quantization aware training with lround during int8 multiplications TensorFlow models	0	585	October 26, 2022
Quantization aware training with quantizationConfig -> 4 % Accuracy loss General Discussion models , keras , model_optimization , help_request	1	1715	April 25, 2022

How to quantize a custom model(not of tf.keras.Model type)?

Related topics