Hi everyone . I am new in this TFJS domain. Actually, I completed the tremendous TFJS EdX course by @Jason sir. The course was excellent and literally ignited a spark of using and exploring this vast field of opportunities.
As I am just getting started with TFJS, I want help regarding the browser.fromPixels funtion. I guess the document object model (DOM) should be in the ready state (not in the loading state) when we run the function. Because when I am running the function inside an event-listener then it is not throwing an error but when I am just using it without any functional block, then it is throwing this error (code’s snippet is also attached):
From the above code snippet, it is clear that the DOM isn’t painted correctly as of now, so the img variable is not holding anything and the error is due to the same. Is it so?
Whereas I have copied an example from the tfjs site and in that case as I have already defined the image data and its attributes, so it is predicting the same itself and not throwing any error!
Can you also tell what the array indexes of the ImageData explain? I have viewed the MDN Web docs for the ImageData() constrictor and the 1,1 is telling the width and height of the image. But I am not able to tell what the index’s values are for.
Also, can you confirm whether providing the HTML img tag’s height and width attribute is important or not, for the browser.fromPixels function? (I don’t think that it should be a factor because if the document object model is in the ready state, then the attributes of the image will be auto-computed and that is why it should not throw this error).
I edited the above post due to some grammatical reasons. But now I am not able to attach the code and error images. It is giving an error. How to resolve that?
After trying sometimes, I found an alternative solution to the problem. I hosted the images on my hosting service and then I was able to upload the images. But that does not seem to be a good way to do so. Dropping images in the discussions are prohibited or is it some sort of glitch on my end?
Welcome @VISHAL_VATS to the TF Forum and thank you for the review of my course! Great to hear from you.
Correct the DOM must exist, and the image must have loaded, before you try and use tf.browser.fromPixels() else you will run into issues. This is the same regardless of library using - trying to load an image that is not loaded will cause issues.
In your code above your code below in 2nd block will execute before the first console.log because the event listener is asynchronous. You must put your other code inside the click event listener call back anonymous function you have defined for it to work.
On my examples in glitch, my JS script import is at the end of the HTML file which means it gets loaded last after the rest of the DOM has already been parsed which is why my example does not have this error and yours does.
Why are you creating an ImageData type? tf.browser.fromPixels returns a Tensor not ImageData object. If you want to paint to canvas you can use toPixels(imageTensor, canvas) instead.
If your image is coming from a different domain to what the HTML / JS is being served on you need to ensure crossorigin attribute of the image is set (see my examples on glitch) to prevent CORS security issues which will prevent access to said image.
Finally its always good to set a width and height of an image to avoid DOM reflows. If you set an image size in the HTML or CSS then it will allocate that space in the DOM visually, vs loading the image and then changing everything on the page around later which gives you a terrible UX as things move suddenly when the image loads. I believe fromPixels can work without setting width / height though regardless. I still recommend setting it though to avoid issues later if you are calculating things for canvas drawing etc which I believe does like to have a defined width height to copy image data correctly.
@Joana is this a bug with the TF Forum? See Vishal’s 2nd post in this thread.
Thanks for the clarification @Jason. Ok, I will set the width and height of the image that has to be converted into pixels data and will ensure that the function runs after the entire Dom is rendered.
By the way, I was just using the ImageData example that was shown in the TfJS Api example. I am good with just the browser.fromPixels()
I am now able to get the 3d Tensor output but I have a small question. If I just do tf.browser.toPixels().print() then it prints the entire tensor (not the entire tensor, to be precise. As it is an array of more than 70k points, so it prints the starting ones and the ending ones) but if I just console.log it, then it prints the object which contains the size, shape and other parameters. But, how can I found the entire tensor in the object? Is there a specific way to extract that or specific property in the output object?
The size, shape and data are understandable but how do I extract the entire array.
I would love to drop an image of the output, but the forum is stopping me from doing so. I hope that thing gets fixed.
For your reference, I am pasting the object which is being shown in the console:
. I am new in this TFJS domain. Actually, I completed the tremendous TFJS EdX course by @Jason sir. The course was excellent and literally ignited a spark of using and exploring this vast field of opportunities.