Vadym/obsidian
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
obsidian/wiki/tech-patterns/kling-veo-video-api.md
Vadym Samoilenko 2ac4192d8e vault backup: 2026-04-24 18:42:47
2026-04-24 18:42:47 +01:00

7.6 KiB
Raw Blame History

title tags updated
Kling AI & Veo 3.1 Video Generation API
kling
veo
video-generation
php
api
2026-04-24

Kling AI & Google Veo 3.1 — API Integration Patterns

Used in: cinema-studio-pro-kling (optical-prod.oliver.solutions/lux-studio/)

Kling AI API

Base URL: https://api-singapore.klingai.com
Auth: HS256 JWT — { iss: accessKey, exp: now+1800, nbf: now-5 } signed with secretKey

Text-to-Video — POST /v1/videos/text2video

{
  "model_name": "kling-v3",
  "prompt": "string (required)",
  "mode": "std",
  "duration": "5",
  "aspect_ratio": "16:9",
  "cfg_scale": 0.5,
  "negative_prompt": "",
  "sound": "on"
}

Valid models: kling-v1, kling-v1-6, kling-v2-master, kling-v2-1-master, kling-v2-5-turbo, kling-v2-6, kling-v3

kling-v3-omni and kling-video-o1 are NOT in the official T2V enum — use with caution.

⚠️ cfg_scale is NOT supported by v2.x models — omit it entirely for kling-v2-*.
duration must be a string: "5", "10" etc. (API example uses string, not integer).

Image-to-Video — POST /v1/videos/image2video

{
  "model_name": "kling-v3",
  "image": "<raw_base64_no_prefix>",
  "image_tail": "<raw_base64_no_prefix>",
  "prompt": "optional",
  "mode": "std",
  "duration": "5",
  "aspect_ratio": "16:9",
  "sound": "off"
}

Valid models: kling-v1, kling-v1-5, kling-v1-6, kling-v2-master, kling-v2-1, kling-v2-1-master, kling-v2-5-turbo, kling-v2-6, kling-v3

⚠️ Base64 format: raw base64 string only — NO data:image/png;base64, prefix.

image_tail, dynamic_masks, and camera_control are mutually exclusive — only one at a time.

Camera Control (T2V and I2V)

Only supported by kling-v1 (STD 5s T2V) and kling-v1-5 (PRO 5s I2V, simple type only).
All v2.x, v1-6, v3 models: no camera_control support → API returns error 1200.

"camera_control": {
  "type": "down_back"
}

type values: simple, down_back, forward_up, right_turn_forward, left_turn_forward

  • Preset types (down_back, forward_up, etc.): config must be absent (not sent).
  • simple type only: provide config with exactly one non-zero value (range −10 to +10): 
    "config": { "horizontal": 5, "vertical": 0, "pan": 0, "tilt": 0, "roll": 0, "zoom": 0 }

Bug history: type: "predefined" is INVALID — API returns error 1200 "camera_control.type value 'predefined' is invalid". The preset name itself IS the type.

Motion Control — POST /v1/videos/motion-control

⚠️ This is a completely separate endpoint — NOT related to camera_control presets.

Takes a reference image + reference video; the character in the generated video follows the reference video's motion (pose transfer).

{
  "model_name": "kling-v2-6",
  "image_url": "https://... or raw_base64_no_prefix",
  "video_url": "https://...",
  "prompt": "optional",
  "character_orientation": "image",
  "mode": "pro"
}

Supported models: kling-v2-6, kling-v3 (std/pro only, not 4k)
Status polling: GET /v1/videos/motion-control/{task_id}

Model Capability Map (key features)

Model T2V I2V camera_control image_tail sound duration
kling-v1 ✅ ✅ ✅ STD 5s T2V only ✅ STD/PRO 5s - 5s/10s
kling-v1-5 ❌ ✅ ✅ PRO 5s I2V (simple) ✅ PRO - 5s/10s
kling-v1-6 ✅ ✅ ❌ ✅ PRO - 5s/10s
kling-v2-master ✅ ✅ ❌ ❌ - 5s/10s
kling-v2-1 ❌ ✅ ❌ ✅ PRO - 5s/10s
kling-v2-1-master ✅ ✅ ❌ ❌ - 5s/10s
kling-v2-5-turbo ✅ ✅ ❌ ✅ PRO - 5s/10s
kling-v2-6 ✅ ✅ ❌ ✅ PRO (no audio) ✅ PRO 5s/10s
kling-v3 ✅ ✅ ❌ ✅ ❌ 3–15s
kling-v3-omni ✅ ✅ ❌ ✅ ❌ 3–15s
kling-video-o1 ✅ ✅ ❌ ✅ ❌ 5s/10s

cfg_scale: supported only by v1.x models — omit for all v2.x models.
sound: kling-v2-6 PRO mode only; all other v1/v2 models do not have sound generation.

Status Polling — GET /v1/videos/{type}/{taskId}

  • task_status: submitted → processing → succeed / failed
  • On succeed: task_result.videos[0].url — CDN URL (temporary, download immediately)
  • Downloaded videos stored: /var/www/html/lux-studio/api/generated_videos/
  • URL returned to frontend: /lux-studio/api/generated_videos/<filename>

Lip Sync — POST /v1/videos/lip-sync

{
  "input": {
    "mode": "audio2video",
    "audio_type": "file",
    "audio_file": "<raw_base64>",
    "video_id": "task_id_of_kling_video"
  }
}

Status polling: GET /v1/videos/lip-sync/{task_id}

Error Codes

Code Meaning
1000–1004 Auth failure — check access/secret keys
1200–1201 Invalid parameters — check field names and types
1302–1304 Rate limit / concurrent task limit / daily limit
HTTP 429 Rate limit

Google Veo 3.1 API (via Gemini)

Base URL: https://generativelanguage.googleapis.com/v1beta/models
Models: veo-3.1-generate-preview, veo-3.1-fast-generate-preview
Auth: ?key=GEMINI_API_KEY query param
Endpoint: POST /{model}:predictLongRunning (async, returns operation ID)

Request Structure

{
  "instances": [{
    "prompt": "string",
    "image": {
      "bytesBase64Encoded": "<raw_base64>",
      "mimeType": "image/jpeg"
    },
    "lastFrame": {
      "bytesBase64Encoded": "<raw_base64>",
      "mimeType": "image/jpeg"
    }
  }],
  "parameters": {
    "aspectRatio": "16:9",
    "durationSeconds": 4,
    "resolution": "720p",
    "sampleCount": 1,
    "personGeneration": "allow_adult"
  }
}

⚠️ Critical format details:

  • Uses bytesBase64Encoded (predict-style API), NOT inlineData (generateContent style)
  • durationSeconds is an integer (4, 6, 8), NOT a string
  • personGeneration: T2V → allow_all, I2V → allow_adult
  • image = start frame, lastFrame = end frame (interpolation, requires 8s duration)

Polling — GET https://generativelanguage.googleapis.com/v1beta/{operationId}

  • Response done: true → check response.generateVideoResponse.generatedSamples[0].video.uri
  • Video URL is temporary — download immediately and store locally

Resolution Constraints

Resolution Requirement
720p Any duration
1080p Requires durationSeconds: 8
4k Requires durationSeconds: 8

Docker / PHP Notes

post_max_size and upload_max_filesize cannot be changed with ini_set() at runtime in php-fpm. Must be set in a conf.d/*.ini file in the Dockerfile:

RUN printf "post_max_size = 100M\nupload_max_filesize = 100M\nmemory_limit = 512M\n" \
    > /usr/local/etc/php/conf.d/uploads.ini

nginx client_max_body_size must also be set — default is 1MB, which 413s any base64 image payload:

http {
    client_max_body_size 100M;
    ...
}

GD extension required for Veo I2V image resizing. Fails with a fatal (not catchable) error if missing — use function_exists('imagecreatefromstring') guard:

RUN apk add --no-cache libpng-dev libjpeg-turbo-dev zlib-dev \
    && docker-php-ext-configure gd --with-jpeg \
    && docker-php-ext-install -j$(nproc) gd

Prompt optimization (VideoGenTab): must run server-side via video_api.php?action=optimize_prompt — direct browser calls to Gemini exhaust the API key's rate limits fast. Backend uses gemini-2.0-flash.