# Markerless tracking and pose estimation on the web browser with single camera

**URL:** <https://discuss.ai.google.dev/t/markerless-tracking-and-pose-estimation-on-the-web-browser-with-single-camera/32002>\
**Category:** TF.js\
**Tags:** web-ml, tfjs\
**Created:** [June 14, 2024, 2:42am UTC](https://discuss.ai.google.dev/t/markerless-tracking-and-pose-estimation-on-the-web-browser-with-single-camera/32002 "2024-06-14T02:42:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![afasew22](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/afasew22/32/10413_2.png) [@afasew22](https://discuss.ai.google.dev/u/afasew22)\
**Post date:** [June 14, 2024, 2:42am UTC](https://discuss.ai.google.dev/t/markerless-tracking-and-pose-estimation-on-the-web-browser-with-single-camera/32002/1 "2024-06-14T02:42:26Z")

</div>

Hello everyone,

We’re in the process of developing a web application that displays an MJPEG stream. Our goal is to identify specific objects in the video in real-time and perform pose estimation. Ideally, we’d like it to function in a way similar to fiduciary markers, but instead, leverage a machine learning algorithm to enable markerless tracking.

Could anyone share insights on whether this is feasible using TensorFlow.js?

---

<div class="post-metadata">

**Author:** ![Niloy\_Deb\_Barma](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/niloy_deb_barma/32/15009_2.png) [@Niloy\_Deb\_Barma](https://discuss.ai.google.dev/u/Niloy_Deb_Barma)\
**Post date:** [June 18, 2024, 2:35am UTC](https://discuss.ai.google.dev/t/markerless-tracking-and-pose-estimation-on-the-web-browser-with-single-camera/32002/3 "2024-06-18T02:35:47Z")

</div>

Yes, markerless tracking and pose estimation with a single camera in a web browser using TensorFlow.js is feasible. Use models like PoseNet or MoveNet for pose estimation and Coco SSD for object detection. Implement by:

1. Including TensorFlow.js in your project.
2. Loading the appropriate model.
3. Accessing the camera with getUserMedia.
4. Processing video frames through the model.
5. Rendering results on a canvas over the video stream.

These models are efficient and can run in real-time directly in the browser.

---

<div class="post-metadata">

**Author:** ![Jason](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/jason/32/6193_2.png) [@Jason](https://discuss.ai.google.dev/u/Jason)\
**Post date:** [June 18, 2024, 9:27am UTC](https://discuss.ai.google.dev/t/markerless-tracking-and-pose-estimation-on-the-web-browser-with-single-camera/32002/5 "2024-06-18T09:27:25Z")

</div>

+1 to what @Niloy_Deb_Barma said. Just following up with a few links to consider:

Custom object detection blogs:

> **[Real-time SKU detection in the browser using TensorFlow.js](https://blog.tensorflow.org/2022/05/real-time-sku-detection-in-browser.html)**
>
> Realtime and offline SKU detection in the browser using Tensorflow.js

> **[Custom object detection in the browser using TensorFlow.js](https://blog.tensorflow.org/2021/01/custom-object-detection-in-browser.html)**
>
> Train a custom MobileNetV2 using the TensorFlow 2 Object Detection API and Google Colab for object detection, convert the model to TensorFlow.js

Learn how to make a smart camera in TensorFlow.js using COCO-SSD pretrained model:

> **[Crea una cámara web inteligente en JavaScript con un modelo de aprendizaje...](https://codelabs.developers.google.com/codelabs/tensorflowjs-object-detection?hl=es-419)**
>
> En este codelab, aprenderás a cargar y usar uno de los modelos previamente entrenados de TensorFlow.js (COCO-SSD) y a usarlo para reconocer objetos comunes con los que se entrenó.

Video version of codelab:

[![](https://d3qe71uytubmmx.cloudfront.net/original/3X/0/6/06abfde739c79d57bde1ab5b56e56cc23f66837a.jpeg "TensorFlow.js: Make a smart webcam in JS with a pre-trained ML model | Workshop") ](https://www.youtube.com/watch?v=hk-709L3RK8)

MoveNet pose estimation:

> **[Next-Generation Pose Detection with MoveNet and TensorFlow.js](https://blog.tensorflow.org/2021/05/next-generation-pose-detection-with-movenet-and-tensorflowjs.html)**
>
> MoveNet is a human pose detection architecture developed at Google that is ultra fast and accurate. It was designed to detect difficult poses

BlazePose GHUM 3D pose estimation:

> **[3D Pose Detection with MediaPipe BlazePose GHUM and TensorFlow.js](https://blog.tensorflow.org/2021/08/3d-pose-detection-with-mediapipe-blazepose-ghum-tfjs.html)**
>
> 3D Pose Detection with MediaPipe BlazePose GHUM and TensorFlow.js - learn how to use our latest model on images from your camera in the browser to est

Hope that helps!

Learn more about using TensorFlow.js via my course:

https://www.youtube.com/embed/videoseries?list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&wmode=transparent&rel=0&autohide=1&showinfo=1&enablejsapi=1