# Using numpy function in dataset map function

**URL:** <https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937>\
**Category:** General Discussion\
**Tags:** datasets, help\_request\
**Created:** [August 8, 2022, 10:15am UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937 "2022-08-08T10:15:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jkummert](https://avatars.discourse-cdn.com/v4/letter/j/aeb1de/32.png) [@jkummert](https://discuss.ai.google.dev/u/jkummert)\
**Post date:** [August 8, 2022, 10:15am UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937/1 "2022-08-08T10:15:34Z")

</div>

So I’m calling my map function like this:

```auto
test_dataset = test_dataset.map(lambda x: tf.py_function(func=split,
          inp=[x]))

```

But i still get the error:

```auto
AttributeError: 'Tensor' object has no attribute 'numpy'

```

My split function looks like this:

```auto
def split(data):
    # split off intervention
    splitteded = tf.split(data, [6, 1], axis=0)

    # split into input and target image
    splitted = tf.split(splitteded[0], 2, axis=1)

    # get intervention over full window
    sum = tf.reduce_sum(splitteded[1])
    average_intervention = sum.numpy() / np.shape(splitteded[1])[1]

    # get average heartrate over full window
    peaks, _ = find_peaks([i[0].numpy() for i in tf.split(data, [1, 1, 5], axis=0)[1][0]])
    if peaks[0] == 0:
        peaks = peaks[1:-1]
    start = peaks[0]
    end = peaks[-1]
    average_heartrate = len(peaks) / ((end - start) / 50)
    return (splitted[0], splitted[1], [average_intervention, average_heartrate])

```

Everything I found relating to this says to use tf.py\_function since eager execution does not work otherwise inside the dataset map function. But it is still not working for me. I feel I’m missing something very obvious.  
Any help is greatly appreciated!

---

<div class="post-metadata">

**Author:** ![Babak\_Zahedi](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/babak_zahedi/32/11962_2.png) [@Babak\_Zahedi](https://discuss.ai.google.dev/u/Babak_Zahedi)\
**Post date:** [August 9, 2022, 9:47am UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937/3 "2022-08-09T09:47:41Z")

</div>

I think You can achieve that with the tf.py\_func function, or tf.py\_function (which is the newer version). It does exactly what you want, it will wrap your numpy function that operates on arrays in a tensorflow operation that you can include as part of your dataset graph.

---

<div class="post-metadata">

**Author:** ![Prayag\_Awasthi](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/prayag_awasthi/32/13101_2.png) [@Prayag\_Awasthi](https://discuss.ai.google.dev/u/Prayag_Awasthi)\
**Post date:** [August 10, 2022, 4:41am UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937/4 "2022-08-10T04:41:48Z")

</div>

yes i also same error

---

<div class="post-metadata">

**Author:** ![Kiran\_Sai\_Ramineni](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/kiran_sai_ramineni/32/8859_2.png) [@Kiran\_Sai\_Ramineni](https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni)\
**Post date:** [August 10, 2022, 1:25pm UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937/5 "2022-08-10T13:25:19Z")

</div>

Hi @jkummert , As per the official documentation in [tf.py\_function](https://www.tensorflow.org/api_docs/python/tf/py_function) it is required to pass the Tout argument also. I have reproduced the issue similar to your code and I didn’t get any error. Please refer to this [gist](https://gist.github.com/kiransair/79708d6db5476c8c301eebd5e3484a71). Thank You.

---

<div class="post-metadata">

**Author:** ![jkummert](https://avatars.discourse-cdn.com/v4/letter/j/aeb1de/32.png) [@jkummert](https://discuss.ai.google.dev/u/jkummert)\
**Post date:** [August 12, 2022, 11:03am UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937/6 "2022-08-12T11:03:29Z")

</div>

Adding the tout gives the same error. Could this be related to running tf on cpu instead of gpu?

---

<div class="post-metadata">

**Author:** ![Victor\_GUO](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/victor_guo/32/13563_2.png) [@Victor\_GUO](https://discuss.ai.google.dev/u/Victor_GUO)\
**Post date:** [March 21, 2023, 11:32am UTC](https://discuss.ai.google.dev/t/using-numpy-function-in-dataset-map-function/28937/8 "2023-03-21T11:32:36Z")

</div>

Hi,

I got the same issue and I’ve tried using wrapper functions. But it didn’t work. But I found another solution by myself.

This probelm caused because the “data” is a tensor and when you first time implement the dataset.map() function, the “sum” is a tensor having nothing and of course it dosen’t have “.numpy”.

However, tensorflow allows such calculations if and only if all the calulations are done within the tensorflow framework. Maybe you should try changing “sum.numpy()/np.shape()” into "sum/tf.constant(shape1, shape2, etc…)