Description:
I have a MedGemma 27b text fine-tuned model packaged as a .mar file (TorchServe format). The .mar file is uploaded to Google Cloud Storage and successfully imported into Vertex AI Model Registry.
When I attempt to deploy this model to an endpoint in Vertex AI, the deployment fails with the generic error message:
Model server exited unexpectedly
Additionally, the UI shows an “image not correct” type of message, even though the container image URI I specified exists and is accessible.
Details:
Model format: TorchServe .mar
Deployment target: Vertex AI Endpoint
Container image: Tried multiple images, below
“us-docker.pkg.dev/vertex-ai/prediction/torch-gpu.2-4:latest”
us-docker.pkg.dev/deeplearning-platform-release/vertex-model-garden/pytorch-inference.cu125.0-4.ubuntu2204.py310
“us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:20250430_0916_RC00_maas”
Machine type: a3-highgpu-1g
Region: us-central1
Service account permissions: [confirm if Artifact Registry access is granted]
Issue:The error message is too generic, and I cannot determine the root cause. I need guidance on:
How to get detailed logs or failure reasons for this deployment (e.g., container logs, health probe failures).
Any known compatibility requirements for deploying TorchServe .mar models on Vertex AI (e.g., container entrypoint, health/predict routes, port configuration).
Steps to troubleshoot “image not correct” errors when the image exists in Artifact Registry.
Please advise on how to retrieve detailed error information and confirm the correct configuration for TorchServe-based custom containers on Vertex AI.
Below command to push to model registry then create endpoint
gcloud ai models upload
–display-name=“MedGemma_FineTuned_LoRA_Model_Test”
–artifact-uri=“gs://project-xxx/models/torchserve/fine-tuned-0812”
–container-image-uri=“us-docker.pkg.dev/vertex-ai/training/pytorch-gpu.2-4.py310:latest”
–project=“xxxx-itpr10996”
–region=“us-central1”
–model-id=“medgemma-lora-model-v1-test”
–description=“Fine-tuned MedGemma 27B model with LoRA adapters for obesity stage prediction.”
–labels=“model_type=llm,fine_tuned=true,task=medical_qa”
@tiffanychen Can you please help