I am trying to inpaint using Imagen via API with Python. I am able to successfully generate images from a text prompt this way but the inpainting is failing. Specifically, the failure is that while an image is created, it is creating an image from scratch rather than inpainting into the masked area. In my code, I am specifying the base_img (which is a JPG file) and mask_img (which is a PNG file).
I am trying to follow this tutorial: Add image content using mask-based inpainting with Imagen | Generative AI on Vertex AI | Google Cloud
I have uploaded the base image and mask file, as well as a sample output that I get after running the code below. I am new to both Imagen and Python, so any pointers greatly appreciated!
# Based on https://cloud.google.com/vertex-ai/generative-ai/docs/samples/generativeaionvertexai-imagen-edit-image-inpainting-insert-mask?hl=en#generativeaionvertexai_imagen_edit_image_inpainting_insert_mask-python
# Setup environment
output_dir = "Z:\\Temp\\"
# Setup Vertex AI and Imagen 3 model
import vertexai
from vertexai.preview.vision_models import Image, ImageGenerationModel
project_id = "123456789012"
vertexai.init(project=project_id, location="us-west4")
model = ImageGenerationModel.from_pretrained("imagegeneration@006")
# Define base image file and inpaint area
input_file = "testimage.jpg"
mask_file = "testimagemask.png"
base_img = Image.load_from_file(location=output_dir + input_file)
mask_img = Image.load_from_file(location=output_dir + mask_file)
# Define prompt
prompt = "glass vase with purple flowers"
negative_prompt = "people, animals"
#Define parameters
images = model.edit_image(
base_image=base_img,
mask=mask_img,
prompt=prompt,
negative_prompt=negative_prompt,
edit_mode="inpainting-insert",
number_of_images=2,
language="en",
safety_filter_level="block_low_and_above",
person_generation="dont_allow",
)
# Loop through each image and save it with a dynamic filename
for idx, image in enumerate(images):
# Initialize timestamp, location & filename protocol
output_file = f"{output_dir}gen_img_{int(idx)+1}.png"
# Save the image with the dynamic filename
image.save(location=output_file, include_generation_parameters=False)
