# Tensorflow Operator Computational Graph

**URL:** <https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601>\
**Category:** General Discussion\
**Tags:** tfcore, graph, models, training, help\_request\
**Created:** [August 18, 2021, 11:39am UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601 "2021-08-18T11:39:18Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jaidev\_Shah](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/jaidev_shah/32/7350_2.png) [@Jaidev\_Shah](https://discuss.ai.google.dev/u/Jaidev_Shah)\
**Post date:** [August 18, 2021, 11:39am UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/1 "2021-08-18T11:39:18Z")

</div>

Hi all!

I’d like to be able to obtain the underlying computational graph consisting of the low-level tf operators of a tf2 model. Basically, for any given tf model architecture(along with training script), is there a straightforward way to modify my code to get a list of the tf operators called during training?

---

<div class="post-metadata">

**Author:** ![Sean\_Moriarity](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/sean_moriarity/32/8001_2.png) [@Sean\_Moriarity](https://discuss.ai.google.dev/u/Sean_Moriarity)\
**Post date:** [August 18, 2021, 4:01pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/3 "2021-08-18T16:01:04Z")

</div>

Does this do what you’re looking for?

```auto
import tensorflow as tf
fun = lambda x: tf.nn.softmax(x)
tf_fun = tf.function(fun)
graph = tf_fun.get_concrete_function(tf.constant([1.0])).graph
isinstance(concrete_fun.graph, tf.Graph) # True

graph.get_operations() # returns a list of graph operations

```

Notice the model’s forward pass will need to be a `tf.function` and you’ll need to know at least the input shapes ahead of time. If the model is a Keras model, here’s a somewhat hacky way to do that:

```auto
model = tf.keras.models.Sequential([
  tf.keras.layers.Dense(1, activation='softmax', input_shape=(1,))
])
model.predict(tf.constant([1.0])) # model.predict_function is only populated after 1 call to predict, you can also do the same thing with model.train_function

graph = model.predict_function.get_concrete_function(iter([tf.constant([1.0])])).graph # The concrete function takes an iterator
isinstance(graph, tf.Graph) # True

graph.get_operations()

```

For more info: [Introduction to graphs and tf.function &nbsp;|&nbsp; TensorFlow Core](https://www.tensorflow.org/guide/intro_to_graphs)

---

<div class="post-metadata">

**Author:** ![Jaidev\_Shah](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/jaidev_shah/32/7350_2.png) [@Jaidev\_Shah](https://discuss.ai.google.dev/u/Jaidev_Shah)\
**Post date:** [August 18, 2021, 8:28pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/5 "2021-08-18T20:28:35Z")

</div>

> [@Sean\_Moriarity](#):
>
> ```auto
> model.predict(tf.constant([1.0])) # model.predict_function is only populated after 1 call to predict, you can also do the same thing with model.train_function
> 
> graph = model.predict_function.get_concrete_function(iter([tf.constant([1.0])])).graph # The concrete function takes an iterator
> isinstance(graph, tf.Graph) # True
> 
> graph.get_operations()
> 
> ```

Thanks for the reply! I’m actually looking for the operators that are called during training (not inference) of the model: during backpropagation as well.

---

<div class="post-metadata">

**Author:** ![Bhack](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/bhack/32/6388_2.png) [@Bhack](https://discuss.ai.google.dev/u/Bhack)\
**Post date:** [August 18, 2021, 8:42pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/6 "2021-08-18T20:42:06Z")

</div>

You can see the computational graph with this:

> **[Examining the TensorFlow Graph  |  TensorBoard](https://www.tensorflow.org/tensorboard/graphs)**

---

<div class="post-metadata">

**Author:** ![Jaidev\_Shah](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/jaidev_shah/32/7350_2.png) [@Jaidev\_Shah](https://discuss.ai.google.dev/u/Jaidev_Shah)\
**Post date:** [August 18, 2021, 10:40pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/7 "2021-08-18T22:40:45Z")

</div>

Thanks, this worked! Is there a way to export and store the operators/graph in any format, say JSON?

---

<div class="post-metadata">

**Author:** ![Bhack](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/bhack/32/6388_2.png) [@Bhack](https://discuss.ai.google.dev/u/Bhack)\
**Post date:** [August 18, 2021, 11:20pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/8 "2021-08-18T23:20:47Z")

</div>

With a saved model you could retry the `Metagraph` with something like:

> <https://stackoverflow.com/questions/60154650/how-to-list-all-used-operations-in-tensorflow-savedmodel>

Just as a reminder for profiling we have:

> **[Optimize TensorFlow performance using the Profiler  |  TensorFlow Core](https://www.tensorflow.org/guide/profiler)**

---

<div class="post-metadata">

**Author:** ![Jaidev\_Shah](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/jaidev_shah/32/7350_2.png) [@Jaidev\_Shah](https://discuss.ai.google.dev/u/Jaidev_Shah)\
**Post date:** [August 19, 2021, 2:56am UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/9 "2021-08-19T02:56:52Z")

</div>

Thanks for sharing! Unfortunately, I don’t think the ops from a SavedModel would represent those invoked during training (which is what I’m interested in). To clarify my question: I’d like a way to store the computational graph (op graph) that’s visible on Tensorboard as a JSON file/ some other readable format.

---

<div class="post-metadata">

**Author:** ![Bhack](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/bhack/32/6388_2.png) [@Bhack](https://discuss.ai.google.dev/u/Bhack)\
**Post date:** [August 19, 2021, 10:05am UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/11 "2021-08-19T10:05:43Z")

</div>

The TB summary writer for the Graph is just going to enable this:

> <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/eager/context.py#L1821>

Check also:

> <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/summary_ops_v2.py#L1217>

And see the `RunMetaData` protobuf message if you wan to parse the file:

> <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto#L755>

---

<div class="post-metadata">

**Author:** ![Sean\_Moriarity](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/sean_moriarity/32/8001_2.png) [@Sean\_Moriarity](https://discuss.ai.google.dev/u/Sean_Moriarity)\
**Post date:** [August 19, 2021, 4:42pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/13 "2021-08-19T16:42:19Z")

</div>

What about:

```python
import tensorflow as tf

model = tf.keras.models.Sequential([
  tf.keras.layers.Dense(2, activation='softmax', input_shape=(1,))
])

model.compile(optimizer='rmsprop', loss='categorical_crossentropy')

xs = tf.constant([[1.0]])
ys = tf.constant([[0.2, 0.8]])

model.train_on_batch(xs, ys)

graph = model.train_function.get_concrete_function(iter([(xs, ys)])).graph # The concrete function takes an iterator
isinstance(graph, tf.Graph) # True

graph.get_operations()

```

This will list the operations performed during a single compiled training step. If you want to export that to a readable format you can do `graph.as_graph_def()`.

---

<div class="post-metadata">

**Author:** ![Bhack](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/bhack/32/6388_2.png) [@Bhack](https://discuss.ai.google.dev/u/Bhack)\
**Post date:** [August 19, 2021, 5:08pm UTC](https://discuss.ai.google.dev/t/tensorflow-operator-computational-graph/31601/15 "2021-08-19T17:08:49Z")

</div>

I think that in many cases it will be good enough, you can also iterate over the graph with:

> **[tf.function ile daha iyi performans  |  TensorFlow Core](https://www.tensorflow.org/guide/function?hl=tr)**
