Enable grappler's autoparallel pass

Stonepia · May 21, 2021, 6:28am

Grappler offers an autoparallel optimizer as stated in (TensorFlow graph optimization with Grappler | TensorFlow Core).

But there is no explicit tools for setting the flag, the tf.config.optimizer.set_experimental_options() does not offer this option (auto_parallel) either.

How could I enable this? Thanks in advance

Bhack · May 21, 2021, 11:39am

It is available but not documented. I think you could open a Docoumentation type issue on Github

github.com

tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/rewriter_config.proto#L14-L17


      
          message AutoParallelOptions {
            bool enable = 1;
            int32 num_replicas = 2;
          }

github.com

tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/rewriter_config.proto#L177-L179


      
          // Scheduling will split big ops such as AddN and try to enforce a schedule
          // of the new computations that decreases peak memory usage.
          SCHEDULING_HEURISTICS = 6;

Stonepia · May 24, 2021, 1:09am

Thank you very much. I have submitted the issue at #49507