Can BERT be used as an alternative to the Universal Sentence Encoder?

TensorFlow General Discussion
, , ,
1

BERT is available in TFJS for question answering but can also it be used as a sentence encoder? USE is getting pretty old and my impression is that BERT is better for the same task. Is it?

2 Likes
2

I don’t know if you need exactly BERT for your project but if you want something ready we have many USE models available in Google | universal-sentence-encoder | Kaggle

Here you can find conversion commands for TFjs:

3 Likes
3

Thanks. Do you know how those USE models compare to the one in the tfjs-models on TensorFlow GitHub that has already been converted?

1 Like
4

Here are some resources:

1 Like
5

If you click the TFHub link in that Readme you can see that it is the USE light on TFHub

1 Like
6

I’m getting the idea that I should stay with USE (and forget about BERT) but in deciding which TFHub USE version, instead of finding any performance comparisons, I find the sentence “We apply this model to the STS benchmark for semantic similarity, and the results can be seen in the example notebook made available.”

So, if I want to use benchmarks to decide if it is worth trying different sizes of USE do I need to run this notebook to generate my own benchmarks? The paper by Cer et al lists some benchmarks but not for different sizes of USE. I don’t even find a table listing the number of parameters for each of the USE variants.

1 Like
7

This is a great repo for comparison between different embeddings, language models, techniques, etc. for sentence embeddings:

They also have a notebook for USE metrics so you can compare that with the rest of that metrics table.

2 Likes