The first post in what I hope will become a long thread to try to improve the mathematical abilities of Gemini.

**1) Broken mathematical notation**

Gemini lags behind ChatGPT in its ability to correctly format Mathematics, a thing ChatGPT has been able to do since day one. This is putting off several people who use these models for scientific purposes.

**1.1) Gemini Pro**

I asked a question and specified “format the mathematics using latex”. Notice that the first part was fine but then it broke down when using the double dollar sign ($$ $$). Notice also that Gemini does not consistently format mathematics by default.

In this other example I did not specify “format the mathematics using latex”. Nonetheless, it correctly did, but then again it broke down (notice the last line in the screenshot). This time not because of the dollar sign.

Similarly, here it broke down. This time when using the align environment.

Other examples of failures with exponents and square roots:

Here is another example, where it consistently fails to format the equation.

**1.2) Gemini 1.5 Pro**

I was making some tests about information retrieval from a file. Gemini failed to display the formula and raised warnings. I tried to rerun the prompt after deactivating the safety blocks but it still didn’t work. Notice that the information I am asking to retrieve, as well as the uploaded document, are not unsafe.

Here is another attempt.

It seems that Gemini is struggling to display Math notation when using the Latex formatting with double dollar sign.

**1.3) Feedback**

I think math formatting should be set-up like in ChatGPT: always formatting math notation using latex unless explicitly asked to provide the raw latex code. I think it’s more rare to copy and paste math from Gemini into Latex than reading some math directly in Gemini.

**2) Mathematical mistakes**

In general, Gemini 1.5 Pro is slightly worse at Math than gpt-4-turbo-2024-04-09, but not by much; in the sense that when you ask simple questions, Gemini tends to do silly mistakes while ChatGPT gets it all right. However, when the models miss because I ask a “difficult” question, Gemini 1.5 Pro misses by little, still giving you some intuition for the path you should follow; instead, gpt-4-turbo-2024-04-09 completely loses it and gets it 100% wrong.

**2.1) Gemini 1.5 Pro**

Gemini struggles a lot with signs.

Sometimes Gemini refuses to answer a question when it’s too hard. It would be nice if it at least tried or explained why it cannot solve it.

**2.2) Gemini 1.5 Pro**

Here is an example of Gemini getting the wrong sign even after asking to review the calculations multiple times.

**3) Mathematical abilities**

ChatGPT has significantly more Mathematical abilities than Gemini 1.5 Pro when using Wolfram GPT. I think Gemini would significantly benefit from this extension that gives a scientific brain to the LLM. See for example the difficult integral that Gemini refused to solve.