TensorFlow port of HF's Paligemma

TensorFlow General Discussion
,
1

Hi,

I assumed many would port such models to TF to learn but I didn’t find any repos. Mine is

It is supposed to be the same as transformers/src/transformers/models/siglip at main · huggingface/transformers · GitHub

The problem is that the tokens are wrong even though they are different for different images.

I did compare weights for all layers and it could be a computation problem that slightly assigns wrong logits to some tokens. Isn’t there a way to debug such complex models ?

Has anyone ported like this ? I thought NNSight/NDIF could help but model may be too complex for that.

Are these Keras layers so tricky to work with ?

‘inference.sh’ is the driver

Thanks

2

I also wanted to ask about debugging techniques. How does one debug this ? This is inference on a laptop.

Thanks