MedGemma and MedSigLIP for defining severity from clinical notes and fundus image

HAI-DEF
1

Hi everyone,

We’re currently having a project and exploring MedGemma and MedSigLIP for these use cases:

  1. Define diagnosis (eye disease) by analyzing clinical notes (150,000++ datapoints)
  2. Detecting the severity of an eye disease from the fundus image

I’m curious if anyone has similar use cases, a general question about best practices and if anyone has tried them and seen the results so far.

Any recommendations of resources and readings are also very welcome!

Thank you!

2 Likes
2

That’s fantastic! Please keep us in the loop. As you’ve probably seen in the technical report, we evaluated both models on Retina images, but there is so much room for further development. We’ve provided fine-tuning notebooks for both models. Please let me know if you need pointers.

4

I’m having the same idea and evaluate medgemma for zero shot classification on our own fundus dataset, though it is still limited. would love to know more about your progress