# Accelerating TensorFlow using Apple M1 Max? **URL:** \ **Category:** General Discussion\ **Tags:** mac\_os, apple\_silicon, pluggable\_device, help\_request\ **Created:** [October 25, 2021, 3:37am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816 "2021-10-25T03:37:26Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![lakshminarasimman](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/lakshminarasimman/32/13796_2.png) [@lakshminarasimman](https://discuss.ai.google.dev/u/lakshminarasimman)\ **Post date:** [October 25, 2021, 3:37am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/1 "2021-10-25T03:37:26Z")
Hello Everyone! I’m planning to buy the M1 Max 32 core gpu MacBook Pro for some Machine Learning (using TensorFlow) like computer vision and some NLP tasks. Is it worth it? Does the TensorFlow use the M1 gpu or the neural engine to accelerate training? I can’t decide what to do? To be transparent I have all Apple devices like the M1 iPad Pro, iPhone 13 Pro, Apple Watch, etc., So I try so hard not to buy other brands with Nvidia gpu for now, because I like the tight integration of Apple eco-system and the M1 Max performance and efficiency. Also, currently I use Google Colab on my 2017 MacBook Air. Desktop is not an option because I travel a lot. Kindly please help me to decide. Thank You All! ---
**Author:** ![Remy\_Wehrung](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/remy_wehrung/32/20714_2.png) [@Remy\_Wehrung](https://discuss.ai.google.dev/u/Remy_Wehrung)\ **Post date:** [October 25, 2021, 4:31am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/3 "2021-10-25T04:31:32Z")
In theory, porting should do this carefree (it’s python), but if Apple was that good at using Keras / TensorFlow, if it was a go-to platform, it would know. I have great doubts, the base M1 equivalent to an RTX 2070, with the performance of a GPU, from AMD, I find that it is less desirable at once. If it is an urgent purchase: either, if you can wait, consider an Nvidia desktop and GPU ---
**Author:** ![Nicko\_van\_Someren](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/nicko_van_someren/32/9359_2.png) [@Nicko\_van\_Someren](https://discuss.ai.google.dev/u/Nicko_van_Someren)\ **Post date:** [October 27, 2021, 1:36am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/5 "2021-10-27T01:36:57Z")
Apple have [released](https://developer.apple.com/metal/tensorflow-plugin/) a [TensorFlow plugin](https://blog.tensorflow.org/2021/06/pluggabledevice-device-plugins-for-TensorFlow.html) (linked below) that allows you to directly use their Metal API to run TensorFlow models on their GPUs. This plugin supports their new M1 chips. While I’ve not tried it on the new M1 Max the general GPU architecture is the same as before and the Metal API remains the same, so in theory this should work. The plugin is being released by Apple themselves so it seems likely that they will make sure that is up to date. Note that the TensorFlow plugin interface needs TF 2.5 or later. ---
**Author:** ![lakshminarasimman](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/lakshminarasimman/32/13796_2.png) [@lakshminarasimman](https://discuss.ai.google.dev/u/lakshminarasimman)\ **Post date:** [October 27, 2021, 7:32am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/7 "2021-10-27T07:32:20Z")
yes, I used it in my friend’s M1 MacBook Air. But few packages was not supported. ---
**Author:** ![anon92429058](https://avatars.discourse-cdn.com/v4/letter/a/2acd7d/32.png) [@anon92429058](https://discuss.ai.google.dev/u/anon92429058)\ **Post date:** [October 27, 2021, 12:26pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/9 "2021-10-27T12:26:45Z")
You can see some benchmarks [here](https://github.com/tlkh/tf-metal-experiments/blob/main/README.md#experiments-and-benchmarks) (credits to the repository author). The experiments I ran so far lead me to think that M1 Max is comparable to my GeForce GTX TITAN X in terms of fp32 deep learning speed performance (of course, M1 Max consumes way less power…). Recent Nvidia cards benefit greatly from fp16 training speedup. ---
**Author:** ![brendank](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/brendank/32/9390_2.png) [@brendank](https://discuss.ai.google.dev/u/brendank)\ **Post date:** [October 28, 2021, 3:42pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/11 "2021-10-28T15:42:14Z")
I have been using the M1 Max 14” for some CNN testing and calibrating and it is very good. For most tasks about 25% better than the Colab Pro P100 instance. I don’t know what the NVIDIA cards can do but it looks to me that you would have to spend quite a lot to get substantially better performance. The tensorflow-metal plugin at 0.2.0 works quite well although I suspect that still want to clean up quite a bit more. Most packages that I have tried work quite well but some are harder to install (such as CatBoost). Using conda to install them has generally worked well for me. If you are happy with the Apple ecosystem then I think you are likely to be happy with an M1 Max machine. ---
**Author:** ![Yingding](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/yingding/32/8043_2.png) [@Yingding](https://discuss.ai.google.dev/u/Yingding)\ **Post date:** [November 1, 2021, 2:17pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/14 "2021-11-01T14:17:24Z")
Can you give some more details, what exactly part of the M1 Max are mostly beneficial for TensorFlow? Is it the GPU or the Neural Engine? Since i am kind of running low on budget, so i thought of go with less GPU cores should only the number of neural core matters. ---
**Author:** ![lakshminarasimman](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/lakshminarasimman/32/13796_2.png) [@lakshminarasimman](https://discuss.ai.google.dev/u/lakshminarasimman)\ **Post date:** [November 1, 2021, 2:21pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/15 "2021-11-01T14:21:36Z")
You can’t use the neural engine for training using TensorFlow. I’ve order the 16 core GPU model and that will be fine for me. Because I planned to build a separate workstation with Nvidia GPU and right now I use cloud services to train my models. So if have a tight budget 16 core GPU model will do just fine. ---
**Author:** ![Yingding](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/yingding/32/8043_2.png) [@Yingding](https://discuss.ai.google.dev/u/Yingding)\ **Post date:** [November 1, 2021, 5:03pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/16 "2021-11-01T17:03:06Z")
Thank you for your clarification. According to metal doc [Tensorflow Plugin - Metal - Apple Developer](https://developer.apple.com/metal/tensorflow-plugin/), only a single GPU can be utilised by metal backend currently. So a larger number of GPU cores on M1 Max doesn’t provide much advantage for TensorFlow, should I do not use distributed training with multiple containers. Am I right? And is it even possible to use multiple metal pods with minikube on M1 Max? ---
**Author:** ![takashi](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/takashi/32/9437_2.png) [@takashi](https://discuss.ai.google.dev/u/takashi)\ **Post date:** [November 2, 2021, 6:39am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/18 "2021-11-02T06:39:57Z")
Hi. I tried a training task of image segmentation using TensorFlow/Keras on GPUs, Apple M1 and nVidia Quadro RTX6000. RTX6000 is 20-times faster than M1(not Max or Pro) SoC, when Automatic Mixed Precision is enabled in RTX… I posted the benchmark in Medium with an estimation of M1 Max (I don’t have an M1 Max machine). > **[Training Speed of TensorFlow in macOS Monterey](https://towardsdatascience.com/training-speed-of-tensorflow-in-macos-monterey-3b8020569be1?source=linkShare-3b467a99d915-1635834746)** > > GPU training in M1 SoC comparing with results in Quadro RTX6000 and estimation in M1 Max SoC ---
**Author:** ![brendank](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/brendank/32/9390_2.png) [@brendank](https://discuss.ai.google.dev/u/brendank)\ **Post date:** [November 3, 2021, 11:22am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/20 "2021-11-03T11:22:25Z")
I don’t have any Nvidia GPUs to compare to so my only comparison is Colab Pro and CPUs. The M1 Max with 32 GPU cores generally performs about 4 times faster than the M1 in my testing - just as you would expect. It is sometimes a closer to 5x in some cases which I am guessing is related to unified memory but that is a guess and I am far from an expert in hardware optimisation for ML. For very small image sizes and very small batch sizes, the M1 Max GPU (and M1) don’t really offer much (but the CPU performs well in those cases). When the batch and image sizes get larger the M1 Max starts to kick in. When compared to Colab Pro (P100 GPU), the M1 Max was 1-1.25x the speed of Colab Pro. For me that works well because it means that I can train models that take a longer time without worrying about timeouts. However, from other posts it sounds like an RTX6000 could make everything significantly faster. However, it is not practical for me to get a separate system with an RTX6000 (as the card itself costs more than my entire M1 Max computer). I guess my summary is that the M1 Max seems like a reasonable medium level system - significantly faster than similarly priced systems but not in the same league as higher cost setups. ---
**Author:** ![Doug](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/doug/32/6398_2.png) [@Doug](https://discuss.ai.google.dev/u/Doug)\ **Post date:** [November 3, 2021, 12:23pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/21 "2021-11-03T12:23:14Z")
Interesting thread. A question to the TensorFlow developers: now that Apple is diving deep into GPU performance, is there any plan to have a native TF support for Metal API (or at least to Apple silicon acceleration)? ---
**Author:** ![Bhack](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/bhack/32/6388_2.png) [@Bhack](https://discuss.ai.google.dev/u/Bhack)\ **Post date:** [November 3, 2021, 12:45pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/22 "2021-11-03T12:45:58Z")
> any plan to have a native TF support for Metal API (or at least to Apple silicon acceleration) I don’t know if the TFRT runtime will still give, at some point, other kind of HW integratoin paths in TF other then pluggable devices that what is currently supported with the `tensorflow-metal` wheel: [https://github.com/tensorflow/community/pull/262#issuecomment-654526062](https://github.com/tensorflow/community/pull/262#issuecomment-654526062) /cc @penporn ---
**Author:** ![takashi](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/takashi/32/9437_2.png) [@takashi](https://discuss.ai.google.dev/u/takashi)\ **Post date:** [November 3, 2021, 1:29pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/23 "2021-11-03T13:29:03Z")
I agree with you. Depending on situations, I will choose M1 Max/Pro. In fact, when I train a regression model, MacBook Air M1 has enough power with reasonable training time. ---
**Author:** ![8bitmp3](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/8bitmp3/32/6397_2.png) [@8bitmp3](https://discuss.ai.google.dev/u/8bitmp3)\ **Post date:** [November 5, 2021, 12:07am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/24 "2021-11-05T00:07:59Z")
> [@takashi](#): > > I posted the benchmark in Medium with an estimation of M1 Max (I don’t have an M1 Max machine). > > ### [Training Speed of TensorFlow in macOS Monterey](https://towardsdatascience.com/training-speed-of-tensorflow-in-macos-monterey-3b8020569be1?source=linkShare-3b467a99d915-1635834746) 👍 Thanks for sharing Here’re an interesting table - M1 8core GPU 16GB RAM on a Macbook Air _vs_ RTX6000 24GB RAM with Xeon 32GB RAM - from @takashi 's post: ![Training speed - 2 epochs](https://d3qe71uytubmmx.cloudfront.net/original/2X/7/76714e8dd0032467781eeb9fbfff3cf0e72322e8.jpeg) ---
**Author:** ![penporn](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/penporn/32/7440_2.png) [@penporn](https://discuss.ai.google.dev/u/penporn)\ **Post date:** [November 5, 2021, 8:15am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/26 "2021-11-05T08:15:22Z")
@Bhack Thank you for the quick answer and for tagging me! [https://tensorflow-prod.ospodiscourse.com/t/accelerating-tensorflow-using-apple-m1-max/5282/21](https://tensorflow-prod.ospodiscourse.com/t/accelerating-tensorflow-using-apple-m1-max/5282/21) @Doug We consider the TensorFlow-Metal plug-in a native support. Future device integrations to the current TensorFlow stack will be in the form of PluggableDevice plug-ins that work with the official TensorFlow packages. ---
**Author:** ![Bhack](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/bhack/32/6388_2.png) [@Bhack](https://discuss.ai.google.dev/u/Bhack)\ **Post date:** [November 5, 2021, 9:45am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/27 "2021-11-05T09:45:16Z")
> [@penporn](#): > > TensorFlow stack will be in the form of PluggableDevice plug-ins that work with the official TensorFlow packages. Will this still valid also when TFRT/MLIR will be more integrated in TF? As one year ago we had this position: > > > Thank you for your interest! > > For the current TensorFlow stack, we recommend …adding the new device and kernels as external modules, connecting to TensorFlow through C APIs. Please see the \[Modular TensorFlow RFC\](https://github.com/tensorflow/community/blob/master/rfcs/20190305-modular-tensorflow.md) for more context. Kernels and ops can be registered through the APIs in this \[ongoing RFC\](https://github.com/tensorflow/community/pull/133). We will share more updates on device addition APIs next week. In addition, there are a few other uncertainties with this approach: > > \- Modular TF cannot guarantee compatibility with the new \[TensorFlow runtime (TFRT)\](https://blog.tensorflow.org/2020/04/tfrt-new-tensorflow-runtime.html) (but we will try to maximize compatibility if possible). TFRT will be modular, but it is fundamentally different from the current runtime and many things are still evolving. > \- The C APIs will initially be subject to changes, we cannot yet guarantee API stability > > What is your integration timeline? TensorFlow is transitioning to a new backend based on \[TFRT\](https://blog.tensorflow.org/2020/04/tfrt-new-tensorflow-runtime.html) and \[MLIR\](https://www.tensorflow.org/mlir), which will not be compatible with the current stack. If your goal is to get your integration productized sometime in 2021, we recommend waiting to integrate with TFRT and MLIR to save throw-away efforts. ---
**Author:** ![Doug](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/doug/32/6398_2.png) [@Doug](https://discuss.ai.google.dev/u/Doug)\ **Post date:** [November 5, 2021, 6:09pm UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/29 "2021-11-05T18:09:10Z")
Many thanks @penporn for clarifying! I meant PluggableDevices were not native because - to the best of my understanding - they are not automatically updated once the “main” TF is updated, and their source code is not open, right? But, in any case, it’s already pretty good to be able to accelerate on Apple gear. ---
**Author:** ![kulin\_seth](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/kulin_seth/32/7572_2.png) [@kulin\_seth](https://discuss.ai.google.dev/u/kulin_seth)\ **Post date:** [November 6, 2021, 6:00am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/30 "2021-11-06T06:00:51Z")
Hi Yingding, Tensorflow Metal plugin utilizes all the core of M1 Max GPU. It appears as a single Device in TF which gets utilized fully to accelerate the training. Distributed training is used for the multi-host scenario. Where different Hosts (with single or multi-gpu) are connected through different network topologies. ---
**Author:** ![Yingding](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/yingding/32/8043_2.png) [@Yingding](https://discuss.ai.google.dev/u/Yingding)\ **Post date:** [November 6, 2021, 10:20am UTC](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816/31 "2021-11-06T10:20:50Z")
Many thanks @kulin_seth for your clarification, I will max out the number of GPU cores on M1 Max and also the RAM probably for TensorFlow training. > [@kulin\_seth](#): > > Tensorflow Metal plugin utilizes all the core of M1 Max GPU. It appears as a single Device in TF which gets utilized fully to accelerate the training Regarding **Federated ML** , which might be unrelated to this thread top, it appears to me Apple M1 chip doesn’t support [JAX with GPU](https://github.com/google/jax/issues/5501) yet and probably also not for FedJAX. Currently, there is also an [issue with TensorFlow Federated on M1](https://github.com/tensorflow/federated/issues/1254). Can TensorFlow team share some timeline regarding the support of TensorFlow Federated on M1 Max/Pro? Is there any way to do Federated ML effectively on M1 Max/Pro with GPU cores? Many thanks in advance. [Next page](https://discuss.ai.google.dev/t/accelerating-tensorflow-using-apple-m1-max/30816.md?page=2)