Understanding RUY and Delegates

TensorFlow General Discussion
,
1

Hi

Looking at this table: Build TensorFlow Lite with CMake

I understand that we can switch ON multiple delegates and I would need to switch between them using the delegate provider. However, I noticed that RUY isn’t a delegate and can be switched on while using a delegate eg: XNNPack. When I do this, I see the performance of RUY but I also see that the XNNpack delegate has been created. I’m trying to get an idea what happens in these situations, does that mean I can keep RUY on at all times while switching between delegates?

Cheers,
Uvin

3

Hi @Uvin_Abeysinghe ,

As you mentioned, RUY is not a delegate but it’s a backend library used for optimized matrix multiplication. When TFLite is building with Cmake, enabling RUY means TFLite interpreter use RUY for its internal optimized CPU-based computations. This is different from delegates like XnnPack, GPU, NNAPI and it does not conflict with the delegates. When combined with delegates it improves the overall performance compared to delegates only. We can keep RUY enabled at all times while switching between delegates.

Thank You