Hey everyone,
I was looking at the L2 Normalization routine for TFLite, and noticed the use of GetInvSqrtQuantizedMultiplierExp function here. While I understand the high level goal of this routine is to find a multiplier which acts as 1 / sqrt(sum_of_squared_inputs) normalizing factor, I’m having trouble understanding how exactly is this computation being done, and what is the purpose and formulation for Newton-Raphson in this function? Any help would be appreciated. Thanks in advance.
Hi @Shikhar_Jaiswal ,
The function GetInvSqrtQuantizedMultiplierExp basically computes the approximation of inverse square root of the inputs for L2 normalization. This function normalizes the inputs by scaling them with the computed factor.
The inverse square root function is numerically approximated by an iterative algorithm called Newton_Raphson method. Once the inverse square root is approximated, the function calculates the appropriate shift to apply depending on the magnitude of the input to ensure that the final result is within the representable range of the integer format. The calculated inverse square root and the determined shift are then used to scale the input vector, effectively performing L2 normalization.
The purpose of 'Newton_Raphson`method for TFLite Quantization is to better approximates the inputs for efficient and accurate calibration making it suitable for embedded and mobile platforms.
Thank You