Queue jobs
Large models (e.g. qwen2.5-coder) can take longer than the server’s HTTP timeout to
generate a response. The job queue solves this: you submit a request and get a job ID
back immediately. A worker on the server runs the job in the background. You poll for the
result whenever you like — minutes or hours later, from a completely different session.
This is especially useful for batch workloads: submit hundreds of jobs, disconnect, and collect results the next day.
All queue endpoints require the same Authorization: Bearer YOUR_API_KEY header as the
rest of the API. Each user sees only their own jobs.
—
Submitting a job
Python:
from gseai import GSEAIServer
with GSEAIServer("your-api-token") as server:
job = server.submit_job(
"my-job", # human-readable name
"qwen2.5-coder", # model
"Explain transformers in three paragraphs.",
)
job_id = job["job_id"]
print(job_id) # save this
CLI:
gseai queue submit qwen2.5-coder "Explain transformers in three paragraphs." -n my-job
# a3f1c7d2-...
For jobs that take a file as input (audio transcription, image editing), use
submit_file_job() or gseai queue upload — see
File-in jobs below.
—
Checking status and collecting results
Python — wait in place:
result = server.wait_for_job(job_id) # blocks; polls every 60 s by default
print(result["result"])
Python — poll manually:
import time
while True:
job = server.get_job(job_id)
if job["status"] == "done":
print(job["result"])
break
if job["status"] in ("error", "cancelled"):
print(job["error"])
break
print(f"status: {job['status']} tokens: {job['tokens_generated']}")
time.sleep(60)
CLI — wait in place:
gseai queue wait a3f1c7d2-...
# [14:01:32] running (142 tokens) ...
# [14:18:44] done
# Transformers are a neural network architecture ...
CLI — check status without blocking:
gseai queue status a3f1c7d2-...
Status values: pending, running, done, error, cancelled.
—
Submit and wait in one step
For interactive use, queue run combines submit and wait:
# Python — submit then immediately wait
job = server.submit_job("my-job", "qwen2.5-coder", "Explain transformers.")
result = server.wait_for_job(job["job_id"])
# CLI
gseai queue run qwen2.5-coder "Explain transformers." -n my-job
—
Batch submission
Submit many jobs up front, then collect results later:
from gseai import GSEAIServer
prompts = [
("paper-1", "Summarise: ..."),
("paper-2", "Summarise: ..."),
("paper-3", "Summarise: ..."),
]
with GSEAIServer("your-api-token") as server:
# Submit all jobs
job_ids = {}
for name, prompt in prompts:
job = server.submit_job(name, "qwen2.5-coder", prompt)
job_ids[name] = job["job_id"]
# Come back later and collect
for name, job_id in job_ids.items():
result = server.wait_for_job(job_id)
print(f"{name}: {result['result'][:80]}")
—
File-in jobs
Audio and image jobs require a file upload. Use
submit_file_job() / gseai queue upload:
# Transcribe audio
job = server.submit_file_job("lecture", "transcribe", "whisper-1", "lecture.mp3")
# Translate audio to English
job = server.submit_file_job("talk", "translate", "whisper-1", "talk.mp3")
# Generate a variation of an image
job = server.submit_file_job("variant", "image_variation", "stable-diffusion", "photo.png")
# Edit an image
job = server.submit_file_job(
"edit", "image_edit", "stable-diffusion", "photo.png",
prompt="replace the sky with a sunset",
)
gseai queue upload whisper-1 lecture.mp3 --job-type transcribe -n lecture
gseai queue upload stable-diffusion photo.png --job-type image_variation -n variant
gseai queue upload stable-diffusion photo.png --job-type image_edit \
--prompt "replace the sky with a sunset" -n edit
After submission, poll and collect exactly as for text jobs.
—
Binary results (speech and images)
Jobs that produce audio or image output — speech, image_generate,
image_edit, image_variation — store their result as a file on the server rather
than in the result text field. Use get_job_result() to
download it once the job is done:
# Generate speech
job = server.submit_job("tts", "kokoros", "Hello, world.", job_type="speech")
server.wait_for_job(job["job_id"])
audio = server.get_job_result(job["job_id"])
open("hello.mp3", "wb").write(audio)
# Generate an image
job = server.submit_job("barn", "stable-diffusion", "A red barn.", job_type="image_generate")
server.wait_for_job(job["job_id"])
image = server.get_job_result(job["job_id"])
open("barn.png", "wb").write(image)
# queue run saves the file automatically
gseai queue run kokoros "Hello, world." --job-type speech -o hello.mp3
# Or fetch separately after the job is done
gseai queue submit kokoros "Hello, world." --job-type speech -n tts
# a3f1c7d2-...
gseai queue wait a3f1c7d2-... -o hello.mp3
# Or fetch without polling (job must already be done)
gseai queue fetch a3f1c7d2-... -o hello.mp3
—
Job types summary
|
Submission |
Result location |
Description |
|---|---|---|---|
|
|
|
Chat completion (default) |
|
|
|
Text embeddings |
|
|
|
Text-to-speech (audio/mpeg) |
|
|
|
Image generation (image/png) |
|
|
|
Audio transcription |
|
|
|
Audio translation to English |
|
|
|
Image editing (image/png) |
|
|
|
Image variation (image/png) |
—
Managing jobs
# List all your jobs
server.list_jobs()
# Filter by status or type
server.list_jobs(status="pending")
server.list_jobs(job_type="speech")
# Cancel one job
server.cancel_job(job_id)
# Cancel all pending jobs
server.cancel_all_jobs()
# List all your jobs
gseai queue list
# Filter by status or type
gseai queue list --status pending
gseai queue list --job-type speech
# Cancel a specific pending job
gseai queue cancel a3f1c7d2-...
# Cancel all pending jobs
gseai queue cancel --all
See the CLI reference reference for full option details, and the API reference reference for the Python API.