To start, I want to say there were clear instructions about what file types for the multimodal stuff (audio, video) is applicable, but when it comes to other file types (like scripts), there seems to be limitations on what files it can absorb.
I cannot find any docs that talk about what file types (that are not images or video) the Google AI Studio supports.
I am throwing spaghetti at the wall to see what throws an error and what doesn’t. At first, I thought it was file size limitation, but I don’t think that’s what’s going on here. I’ve tried uploading the same file types of varying sizes, and it does not appear to change the result.
What works:
-.txt
-.py
What does not work:
-.json
-.yaml
-.sh
-.go
-.rs
Of course, I am aware that copy/paste can, in theory, resolve this, but this is not something I was expecting to be a problem.
On the other hand, I am still getting used to multi-modal native models, so I understand that data types and files might have to be treated differently. Data is not converted to text content like the known janky methods for giving LLMs more senses.
Please note I did not try this in Vertex AI (yet). I will say, the token counter is quite a handy addition. It lets me assess how many tokens a data type of mine is right off the bat. The only thing I’m unsure of is if there would be a way to count the tokens of an item if it’s longer than 1M tokens. I do not typically have that amount of tokens just lying around to test that yet lol.
I’m not sure how difficult this would be to do, but it would be really nice to have some kind of token-counting tool for just media/files. Or, hell, maybe a token count value next to file size in Google Drive or something. I’m noticing I intuitively just want to check the token count before it’s placed in a prompt.