How to calculate FLOPs of transformer in tensorflow?

Maxwell_Albert · September 24, 2021, 3:46am

I know that

    flops = tf.profiler.profile(graph, options=tf.profiler.ProfileOptionBuilder.float_operation())

can calculate the FLOPs.
But where can I find the graph of transformer?
Please help me.

Bhack · September 24, 2021, 10:50am

There Is a quite long thread for this in TF 2.x:

github.com/tensorflow/tensorflow

TF 2.0 Feature: Flops calculation

opened 09:54AM - 25 Sep 19 UTC

closed 10:21PM - 18 Jun 22 UTC

pzobel

stat:awaiting response type:feature stale comp:tfdbg

<em>Please make sure that this is a feature request. As per our [GitHub Policy](…https://github.com/tensorflow/tensorflow/blob/master/ISSUES.md), we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template</em> **System information** - TensorFlow version (you are using): TF 2.0 RC2 - Are you willing to contribute it (Yes/No): **Describe the feature and the current behavior/state.** I am missing the opportunity to compute the number of floating point operations of a tf.keras Model in TF 2.0. In TF 1.x tf.profiler was available [see here](https://stackoverflow.com/questions/45085938) but I can find anything equivalent for TF 2.0 yet. **Will this change the current api? How?** **Who will benefit with this feature?** Everbody interested in the computational complexity of a TensorFlow model. **Any Other info.**

Topic		Replies	Views
Calculate Flops in Tensorflow and Pytorch are not equal? General Discussion models , github , python , pytorch	14	9576	December 22, 2023
WHY I can count FLOPs of a model without declare which model to be counted? General Discussion help_request , models , tflite	1	1703	January 13, 2025
CNN net much slower than DNN General Discussion models , gpu , keras , help_request	9	2502	October 30, 2021
How to find out keras model memory size? General Discussion models , keras , help_request	3	9036	October 23, 2021
How to get a coarse-grained op-level graph in tensorflow General Discussion help_request , graph , tfcore	1	884	January 12, 2024

How to calculate FLOPs of transformer in tensorflow?

Related topics