Programatically creating new releases in RunPod for serverless endpoints is undocumented (as of writing).
TL;DR copy paste the Python code from here and call update_endpoint_image("ghcr.io/...") with your new image on every deployment. Or use the Github Action.
Their built in workflow for building a Dockerfile straight from a Github repository didn’t work for me. Builds always failed without giving an indication for why they failed. Logs seemed to stop at random points during the build. So I opted for using their mechanism for pulling from a Docker registry. They don’t support ECR as the Docker credentials need to be static in RunPod. So GHCR was my choice.
Looking through first their CLI, their REST API and finally their Python SDK, there was no documented way of creating a new release. Except of course through the UI.
So I had a look at what the UI does and implemented a script based on its GraphQL queries and the RunPod python SDK.
A few simple findings about its structure. The API doesn’t expose “Releases” as one might expect. Each Endpoint has a Template attached to it. The template to endpoint link appears to be immutable (error i got at update: runpod.error.QueryError: This endpoint has a bound template.). Instead you update the template itself. When a “bound” template is updated, it appears that a “Release” is created automatically as a side-effect. The rollout starts immediately after the update. So the workflow is simply this:
[Get Endpoint and its Template] → [Modify Template structure] → [Send update of Template]
Here’s the code:
import os
import json
import runpod
import requests
runpod.api_key = os.environ['RUNPOD_API_KEY']
def update_endpoint_image(endpoint_id, new_image_name):
endpoint_with_template_response = run_graphql_query(
fetch_endpoint_with_template_query,
{'id': endpoint_id})
template = endpoint_with_template_response['data']['myself']['endpoint']['template']
template["imageName"] = new_image_name
run_graphql_query(save_template_query, { 'input': template })
fetch_endpoint_with_template_query = """
query($id: String!) {
myself {
endpoint(id: $id) {
template {
advancedStart
containerDiskInGb
containerRegistryAuthId
dockerArgs
env {
key
value
}
id
imageName
isPublic
isServerless
name
ports
readme
startJupyter
startScript
startSsh
volumeInGb
volumeMountPath
config
category
}
}
}
}
"""
save_template_query = """
mutation saveTemplate($input: SaveTemplateInput) {
saveTemplate(input: $input) {
id
__typename
}
}
"""
# modified version of runpod.api.graphql.run_graphql_query, which sadly sucks.
# it doesn't have a parameter to pass variables through the API. instead they
# use string concatenation to inject variables.
# seems like a horrific idea.
def run_graphql_query(query, variables, api_key = None):
from runpod import api_key as global_api_key
effective_api_key = api_key or global_api_key
api_url_base = os.environ.get("RUNPOD_API_BASE_URL", "https://api.runpod.io")
url = f"{api_url_base}/graphql"
headers = {
"Content-Type": "application/json",
"User-Agent": runpod.user_agent.USER_AGENT,
"Authorization": f"Bearer {effective_api_key}",
}
data = json.dumps({"query": query, "variables": variables})
response = requests.post(url, headers=headers, data=data, timeout=30)
if "errors" in response.json():
raise Exception(response.json()["errors"][0]["message"])
return response.json()
image_name = "${{ inputs.image_name }}"
endpoint_id = "${{ inputs.endpoint_id }}"
update_endpoint_image(endpoint_id, image_name)
EOF
Then I created a reusable Github Action for the deployment itself. You can find it in the halbgut/runpod-serverless-deploy. Additionally, here’s a full example of how i use it:
name: Build and Push Docker Image
on:
push:
branches: [ main ]
workflow_dispatch:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}/runpod-graph-segmentation
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
image-name: ${{ steps.meta.outputs.tags }}
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache,mode=max
deploy:
needs: build-and-push
uses: halbgut/runpod-serverless-deploy/.github/workflows/deploy-runpod.yml@v1
with:
image_name: ${{ fromJSON(steps.meta.outputs.json).tags[2] }}
endpoint_id: "REDACTED"
secrets: inherit