A small test program for gauging the efficacy of Vertex AI for doing pool detection inference
  • Go 98.5%
  • Nix 1.3%
  • Shell 0.2%
Find a file
2026-06-18 19:33:46 +00:00
doc Update plan draft after feedback 2026-06-18 15:47:54 +00:00
.gitignore Add full inference pipeline logic 2026-06-18 16:16:08 +00:00
AGENTS.md Initial creation of the overall project files and implementation plan 2026-06-18 15:38:56 +00:00
catalog.go Initial creation of the overall project files and implementation plan 2026-06-18 15:38:56 +00:00
config.go Add -sample flag for testing inference pipeline 2026-06-18 19:24:25 +00:00
flake.lock Update plan draft after feedback 2026-06-18 15:47:54 +00:00
flake.nix Initial README and flake 2026-06-18 15:23:38 +00:00
gcs.go Speed up existence checks with fewer round trips 2026-06-18 19:33:46 +00:00
go.mod Initial implementation of steps 1-3 of the plan 2026-06-18 15:53:35 +00:00
go.sum Initial implementation of steps 1-3 of the plan 2026-06-18 15:53:35 +00:00
inference.go Validate batch model before doing slow work 2026-06-18 17:15:43 +00:00
main.go Don't re-tile if the tile directory already exists 2026-06-18 19:29:11 +00:00
manifest.go Initial creation of the overall project files and implementation plan 2026-06-18 15:38:56 +00:00
README.md Add ability to configure location by env var, update docs on vertex model 2026-06-18 17:22:55 +00:00
scene.go Initial creation of the overall project files and implementation plan 2026-06-18 15:38:56 +00:00
start-vertex-ai-test.sh Add full inference pipeline logic 2026-06-18 16:16:08 +00:00
tile.go Initial creation of the overall project files and implementation plan 2026-06-18 15:38:56 +00:00

Vertex AI Test

Processes a Planet satellite imagery zip file: unzips, validates integrity, slices GeoTIFFs into PNG tiles, and uploads them to Google Cloud Storage. (Vertex AI batch inference coming soon.)

Build

go build ./...

Requires the tiff2rgba and tiffcrop CLI tools (from libtiff) at runtime. If using Nix, nix develop provides them.

GCP prerequisites

The service account used for authentication needs the following IAM roles:

Role Purpose
roles/storage.objectAdmin Read/write objects in the GCS bucket
roles/aiplatform.user Create and manage Vertex AI batch prediction jobs

Grant them via:

gcloud projects add-iam-policy-binding <PROJECT> \
  --member="serviceAccount:<CLIENT_EMAIL>" \
  --role="roles/storage.objectAdmin"

gcloud projects add-iam-policy-binding <PROJECT> \
  --member="serviceAccount:<CLIENT_EMAIL>" \
  --role="roles/aiplatform.user"

The <CLIENT_EMAIL> is the client_email field from your service account JSON key file.

Usage

vertex-ai-test [flags] <input.zip>

Required flags

Flag Env var Description
-bucket GCS_BUCKET GCS bucket name for tile uploads
-project VERTEX_PROJECT or GOOGLE_CLOUD_PROJECT GCP project ID
-model VERTEX_MODEL Vertex AI model ID (numeric ID from the console, not the display name)

Authentication

Supply GCP credentials via one of:

  • -creds flag: path to a GCP service account JSON key file
  • GOOGLE_APPLICATION_CREDENTIALS env var: path to the same JSON key file
  • ADC fallback: if neither is set, the GCE/GKE metadata server is tried

Example using a service account key file:

./vertex-ai-test \
  -bucket my-bucket \
  -project my-gcp-project \
  -model 1234567890123456789 \
  -creds /path/to/service-account.json \
  scene.zip

Or equivalently with env vars:

export GCS_BUCKET=my-bucket
export VERTEX_PROJECT=my-gcp-project
export VERTEX_MODEL=1234567890123456789
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
./vertex-ai-test scene.zip

Optional flags

Flag Default Description
-tilearea 1000.0 Target tile area in square meters
-gcs-prefix "" Optional prefix within the GCS bucket
-location us-central1 GCP region for Vertex AI (or VERTEX_LOCATION env)
-machine-type n1-standard-4 Batch prediction worker machine type
-max-workers 10 Max prediction worker count
-batch-size 64 Batch size per worker
-debug false Enable debug-level logging
-force false Re-upload tiles that already exist in GCS (by default they are skipped)
-j 8 Number of parallel GCS uploads

Output

  • Tiles are extracted to ./tmp/<scene-name>/tiles/<itemID>/<assetType>/<N>px/tile_X_Y.png
  • Uploaded to GCS at gs://<bucket>/[<prefix>/]<catalogID>/<itemID>/<assetType>/<N>px/tile_X_Y.png