Using AsyncGenerator to create a dataset in TF.js

TensorFlow Special Interest Groups TF.js
,
1

I do not see a way to do this:

model.fitDataset(asyncGenerator)

This would be useful when you have to load files asynchronously i.e:

const iteratorParams = (n: number, labels: number[][], paths: string[]) =>
  async function* makeIterator() {
    let i = 0;

    let result;
    while (i < n) {
      let originalImg = tf.node.decodeImage(await readFile(paths[i]), 3);
      const resizedImage = tf.image
        .resizeBilinear(originalImg, [224, 224])
        .div(255);

      const [height, width] = originalImg.shape;
      const scale = tf.tensor1d([1 / width, 1 / height, 1 / width, 1 / height]);
      const label = tf.tensor1d(labels[i]).mul(scale);

      result = {
        xs: resizedImage as tf.Tensor3D,
        ys: label,
      };
      i++;
      yield result;
    }
  };

What do you think @macd ?

3

a few months later, is there any update on it? id also love to see async generators with datasets. would be a huge win.

1 Like