How to do atomic add on CPU with multithreading

TensorFlow General Discussion
,
1

I’m writing a TF c++ kernel with gradient estimation.
It requires to estimate gradients with respect to multiple input variables of different sizes.

Consequently, i need to update some gradient values multiple times (add gradient value).
For GPU kernel i use GpuAtomicAdd function, but how can i do the same on CPU?

It seems that there is no such function in TF codebase and i don’t want to use large arrays of mutexes due to overhead and code inconsistency

3

Hi @11133 , You can use the std::Atomic library to perform arithmetic operations. To perform add operation you can use fetch_add. Thank You.