DevX improvements for Gemini API Logs and Datasets

Google AI Studio
1

My main pain point is that it’s it very difficult to find things in the logs:

  1. There is no search option
  2. There is no way to mark requests at API level, to automatically put them in datasets
  3. The pages take a long time to load

I sometimes have to set the option to show 100 requests at a time, and then painstakingly move page by page (sometimes takes 5 minutes) to get to the request I want to fetch or debug.

Why do I need a better developer experience here?

Here are a couple of scenarios where I’m affected:

  1. I want to be able to do A/B testing easier. For a specific feature we are using some prompts internally before releasing them to all the customers. I want to be able to assess the quality of these prompts easier, without relying on user feedback. Specifying the dataset at request level would simplify this process
  2. From time to time we see unexpected errors throughout the system, which are related to the output of the Gemini API. We want to be able to debug these requests in a timely manner without having to store the input and the whole output of the prompt.

Thanks,
John L.