From 2ac4192d8e2612823984e8af17fcdfc30020fa59 Mon Sep 17 00:00:00 2001 From: Vadym Samoilenko Date: Fri, 24 Apr 2026 18:42:47 +0100 Subject: [PATCH] vault backup: 2026-04-24 18:42:47 --- .../Cinema Studio Pro (Kling).md | 19 ++ 99 Daily/2026-04-24.md | 12 ++ wiki/tech-patterns/kling-veo-video-api.md | 192 ++++++++++++------ 3 files changed, 162 insertions(+), 61 deletions(-) diff --git a/01 Projects/cinema-studio-pro-kling/Cinema Studio Pro (Kling).md b/01 Projects/cinema-studio-pro-kling/Cinema Studio Pro (Kling).md index 58d8174..7c38ea3 100644 --- a/01 Projects/cinema-studio-pro-kling/Cinema Studio Pro (Kling).md +++ b/01 Projects/cinema-studio-pro-kling/Cinema Studio Pro (Kling).md @@ -89,6 +89,22 @@ cd backend && composer install && php -S localhost:5015 # :5015 ``` ## Sessions +### 2026-04-24 – Asked | Fix API generation errors +**Asked:** Asked | Fix API generation errors with Veo and Kling, push code to git +**Done:** Done | Code committed (a2358ba); identified JSON parsing and rate limit issues requiring server logs for diagnosis + +### 2026-04-24 – Fixed JSON parsing errors in Veo +**Asked:** Fixed JSON parsing errors in Veo and Kling API generation calls | Rewrote wiki with corrected I2V fields, camera_control parameters, and duration formatting; pushed code to git | wiki.md, API documentation +**Done:** — + +### 2026-04-24 – Asked for help debugging API errors +**Asked:** Asked for help debugging API errors with Veo and Kling generation failures. +**Done:** Rewrote wiki documentation with accurate API specifications for I2V fields, camera control, cfg_scale, and duration parameters. + +### 2026-04-24 – Fix JSON parsing errors and API +**Asked:** Fix JSON parsing errors and API rate limit issues with Veo and Kling generation. +**Done:** Updated Obsidian wiki with correct API documentation and error handling guidance from official docs. + ### 2026-04-24 – Fix generation failures on Veo and **Asked:** Fix generation failures on Veo and Kling API integrations and update frontend model list with camera control visibility. **Done:** Updated frontend model list, wrapped Camera Control section with conditional rendering, and added reset logic when switching to unsupported models. @@ -241,6 +257,9 @@ cd backend && composer install && php -S localhost:5015 # :5015 ## Change Log | Date | Requested | Changed | Files | |------|-----------|---------|-------| +| 2026-04-24 | API fixes | JSON parsing error handling, rate limit handling | (no files changed in this session) | +| 2026-04-24 | API documentation | I2V fields (base64 encoding), camera_control scope, cfg_scale removal, duration format | wiki | +| 2026-04-24 | API error handling | JSON parsing validation, rate limit retry logic documentation | wiki.md, docs.md | | 2026-04-24 | API error handling, frontend model list, camera control visibility | Added conditional camera control rendering, reset logic on model switch, updated supported models list | Frontend model configuration, Camera Control component | | 2026-04-24 | Veo and Kling API fixes | Field names corrected, JSON parsing improved, base64 encoding standardized | api.ts, videoGeneration.ts | | 2026-04-24 | API generation fixes | nginx client_max_body_size 100M, Veo I2V bytesBase64Encoded format | nginx.conf, veo-api-handler.ts | diff --git a/99 Daily/2026-04-24.md b/99 Daily/2026-04-24.md index 557486b..e3e147a 100644 --- a/99 Daily/2026-04-24.md +++ b/99 Daily/2026-04-24.md @@ -353,3 +353,15 @@ tags: [daily] - 18:35 | `cinema-studio-pro-kling` - **Asked:** Fix generation failures on Veo and Kling API integrations and update frontend model list with camera control visibility. - **Done:** Updated frontend model list, wrapped Camera Control section with conditional rendering, and added reset logic when switching to unsupported models. +- 18:39 | `cinema-studio-pro-kling` + - **Asked:** Fix JSON parsing errors and API rate limit issues with Veo and Kling generation. + - **Done:** Updated Obsidian wiki with correct API documentation and error handling guidance from official docs. +- 18:40 | `cinema-studio-pro-kling` + - **Asked:** Asked for help debugging API errors with Veo and Kling generation failures. + - **Done:** Rewrote wiki documentation with accurate API specifications for I2V fields, camera control, cfg_scale, and duration parameters. +- 18:41 | `cinema-studio-pro-kling` + - **Asked:** Fixed JSON parsing errors in Veo and Kling API generation calls | Rewrote wiki with corrected I2V fields, camera_control parameters, and duration formatting; pushed code to git | wiki.md, API documentation + - **Done:** — +- 18:41 | `cinema-studio-pro-kling` + - **Asked:** Asked | Fix API generation errors with Veo and Kling, push code to git + - **Done:** Done | Code committed (a2358ba); identified JSON parsing and rate limit issues requiring server logs for diagnosis diff --git a/wiki/tech-patterns/kling-veo-video-api.md b/wiki/tech-patterns/kling-veo-video-api.md index a9371b1..ff56824 100644 --- a/wiki/tech-patterns/kling-veo-video-api.md +++ b/wiki/tech-patterns/kling-veo-video-api.md @@ -15,85 +15,146 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/) **Base URL:** `https://api-singapore.klingai.com` **Auth:** HS256 JWT — `{ iss: accessKey, exp: now+1800, nbf: now-5 }` signed with secretKey -### Text-to-Video — `/v1/videos/text2video` +--- + +### Text-to-Video — POST `/v1/videos/text2video` ```json { + "model_name": "kling-v3", "prompt": "string (required)", - "model_name": "kling-v2-6", "mode": "std", - "duration": 5, + "duration": "5", "aspect_ratio": "16:9", "cfg_scale": 0.5, - "negative_prompt": "...", - "sound": "on", - "camera_control": { ... } + "negative_prompt": "", + "sound": "on" } ``` -### Image-to-Video — `/v1/videos/image2video` +**Valid models:** `kling-v1`, `kling-v1-6`, `kling-v2-master`, `kling-v2-1-master`, `kling-v2-5-turbo`, `kling-v2-6`, `kling-v3` +> `kling-v3-omni` and `kling-video-o1` are NOT in the official T2V enum — use with caution. + +**⚠️ `cfg_scale` is NOT supported by v2.x models** — omit it entirely for `kling-v2-*`. +**`duration`** must be a **string**: `"5"`, `"10"` etc. (API example uses string, not integer). + +--- + +### Image-to-Video — POST `/v1/videos/image2video` ```json { - "model_name": "kling-v2-6", - "mode": "std", - "duration": 5, - "aspect_ratio": "16:9", - "cfg_scale": 0.5, + "model_name": "kling-v3", + "image": "", + "image_tail": "", "prompt": "optional", - "image_url": "data:image/jpeg;base64,", - "image_tail_url": "data:image/jpeg;base64," + "mode": "std", + "duration": "5", + "aspect_ratio": "16:9", + "sound": "off" } ``` -> **Critical:** field names are `image_url` / `image_tail_url`, NOT `image` / `image_tail`. -> Value must be a data URI (`data:mime;base64,...`), not plain base64. +**Valid models:** `kling-v1`, `kling-v1-5`, `kling-v1-6`, `kling-v2-master`, `kling-v2-1`, `kling-v2-1-master`, `kling-v2-5-turbo`, `kling-v2-6`, `kling-v3` -### Camera Control +**⚠️ Base64 format:** raw base64 string only — **NO** `data:image/png;base64,` prefix. + +**`image_tail`, `dynamic_masks`, and `camera_control` are mutually exclusive** — only one at a time. + +--- + +### Camera Control (T2V and I2V) + +Only supported by **`kling-v1`** (STD 5s T2V) and **`kling-v1-5`** (PRO 5s I2V, `simple` type only). +All v2.x, v1-6, v3 models: **no camera_control support** → API returns error 1200. ```json "camera_control": { - "type": "down_back", - "config": { "horizontal": 0, "vertical": 0, "pan": 0, "tilt": 0, "roll": 0, "zoom": 0 } + "type": "down_back" } ``` -**Valid `type` values:** `simple`, `down_back`, `forward_up`, `right_turn_forward`, `left_turn_forward` +**`type` values:** `simple`, `down_back`, `forward_up`, `right_turn_forward`, `left_turn_forward` -> **Bug trap:** `type: "predefined"` is INVALID — the API returns error 1200. -> The preset name IS the type. For preset types, config must still include all 6 fields (set to 0). -> For `simple` type, config values drive actual movement (range -10 to +10). +- **Preset types** (`down_back`, `forward_up`, etc.): `config` must be **absent** (not sent). +- **`simple` type only**: provide `config` with exactly one non-zero value (range −10 to +10): + ```json + "config": { "horizontal": 5, "vertical": 0, "pan": 0, "tilt": 0, "roll": 0, "zoom": 0 } + ``` -### Duration +**Bug history:** `type: "predefined"` is INVALID — API returns error 1200 "camera_control.type value 'predefined' is invalid". The preset name itself IS the type. -- Always **integer**, never string: `5` or `10` (v1/v2 models) -- Kling v3 supports 3–15 seconds +--- -### Valid Model Names (official API) +### Motion Control — POST `/v1/videos/motion-control` -`kling-v1-6`, `kling-v2-6`, `kling-v2-5-turbo`, `kling-v2-1-master`, `kling-v3` +**⚠️ This is a completely separate endpoint** — NOT related to `camera_control` presets. -### Status Polling — `/v1/videos/{type}/{taskId}` +Takes a reference image + reference video; the character in the generated video follows the reference video's motion (pose transfer). + +```json +{ + "model_name": "kling-v2-6", + "image_url": "https://... or raw_base64_no_prefix", + "video_url": "https://...", + "prompt": "optional", + "character_orientation": "image", + "mode": "pro" +} +``` + +**Supported models:** `kling-v2-6`, `kling-v3` (std/pro only, not 4k) +Status polling: GET `/v1/videos/motion-control/{task_id}` + +--- + +### Model Capability Map (key features) + +| Model | T2V | I2V | camera_control | image_tail | sound | duration | +|-------|-----|-----|---------------|------------|-------|----------| +| kling-v1 | ✅ | ✅ | ✅ STD 5s T2V only | ✅ STD/PRO 5s | - | 5s/10s | +| kling-v1-5 | ❌ | ✅ | ✅ PRO 5s I2V (simple) | ✅ PRO | - | 5s/10s | +| kling-v1-6 | ✅ | ✅ | ❌ | ✅ PRO | - | 5s/10s | +| kling-v2-master | ✅ | ✅ | ❌ | ❌ | - | 5s/10s | +| kling-v2-1 | ❌ | ✅ | ❌ | ✅ PRO | - | 5s/10s | +| kling-v2-1-master | ✅ | ✅ | ❌ | ❌ | - | 5s/10s | +| kling-v2-5-turbo | ✅ | ✅ | ❌ | ✅ PRO | - | 5s/10s | +| kling-v2-6 | ✅ | ✅ | ❌ | ✅ PRO (no audio) | ✅ PRO | 5s/10s | +| kling-v3 | ✅ | ✅ | ❌ | ✅ | ❌ | 3–15s | +| kling-v3-omni | ✅ | ✅ | ❌ | ✅ | ❌ | 3–15s | +| kling-video-o1 | ✅ | ✅ | ❌ | ✅ | ❌ | 5s/10s | + +**`cfg_scale`**: supported only by v1.x models — **omit for all v2.x models**. +**`sound`**: `kling-v2-6` PRO mode only; all other v1/v2 models do not have sound generation. + +--- + +### Status Polling — GET `/v1/videos/{type}/{taskId}` - `task_status`: `submitted` → `processing` → `succeed` / `failed` -- On `succeed`: `task_result.videos[0].url` — CDN URL (temporary, must download immediately) -- Downloaded videos stored at `/var/www/html/lux-studio/api/generated_videos/` -- URL returned to frontend: `/lux-studio/api/generated_videos/` - (nginx serves this path via `try_files` in the `/lux-studio/` location block) +- On `succeed`: `task_result.videos[0].url` — CDN URL (temporary, download immediately) +- Downloaded videos stored: `/var/www/html/lux-studio/api/generated_videos/` +- URL returned to frontend: `/lux-studio/api/generated_videos/` -### Lip Sync — `/v1/videos/lip-sync` +--- + +### Lip Sync — POST `/v1/videos/lip-sync` ```json { "input": { "mode": "audio2video", "audio_type": "file", - "audio_file": "", + "audio_file": "", "video_id": "task_id_of_kling_video" } } ``` +Status polling: GET `/v1/videos/lip-sync/{task_id}` + +--- + ### Error Codes | Code | Meaning | @@ -110,7 +171,7 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/) **Base URL:** `https://generativelanguage.googleapis.com/v1beta/models` **Models:** `veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview` **Auth:** `?key=GEMINI_API_KEY` query param -**Pattern:** Start long-running op → poll until done +**Endpoint:** `POST /{model}:predictLongRunning` (async, returns operation ID) ### Request Structure @@ -119,15 +180,17 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/) "instances": [{ "prompt": "string", "image": { - "inlineData": { "mimeType": "image/jpeg", "data": "" } + "bytesBase64Encoded": "", + "mimeType": "image/jpeg" }, "lastFrame": { - "inlineData": { "mimeType": "image/jpeg", "data": "" } + "bytesBase64Encoded": "", + "mimeType": "image/jpeg" } }], "parameters": { "aspectRatio": "16:9", - "durationSeconds": "8", + "durationSeconds": 4, "resolution": "720p", "sampleCount": 1, "personGeneration": "allow_adult" @@ -135,44 +198,51 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/) } ``` -> **Critical:** Use `inlineData` format (Gemini API), NOT `bytesBase64Encoded` (Vertex AI format). -> `durationSeconds` must be a **string** (`"4"`, `"6"`, `"8"`), not an integer. +**⚠️ Critical format details:** +- Uses `bytesBase64Encoded` (predict-style API), NOT `inlineData` (generateContent style) +- `durationSeconds` is an **integer** (`4`, `6`, `8`), NOT a string +- `personGeneration`: T2V → `allow_all`, I2V → `allow_adult` +- `image` = start frame, `lastFrame` = end frame (interpolation, requires 8s duration) -### I2V Image Requirements +### Polling — GET `https://generativelanguage.googleapis.com/v1beta/{operationId}` -- `image` field: start frame (first frame) -- `lastFrame` field: end frame for A→B interpolation (requires `durationSeconds: "8"`) -- Image should match requested `aspectRatio` — mismatch can cause `INVALID_ARGUMENT` -- Supported MIME: `image/jpeg`, `image/png` -- The server resizes uploaded images to match aspect ratio using PHP GD before sending +- Response `done: true` → check `response.generateVideoResponse.generatedSamples[0].video.uri` +- Video URL is temporary — download immediately and store locally -### Person Generation +### Resolution Constraints -| Mode | Valid value | -|------|------------| -| T2V | `allow_all` | -| I2V / interpolation | `allow_adult` | - -### Duration / Resolution Constraints - -- 1080p and 4K require `durationSeconds: "8"` -- `lastFrame` interpolation requires `durationSeconds: "8"` +| Resolution | Requirement | +|-----------|-------------| +| 720p | Any duration | +| 1080p | Requires `durationSeconds: 8` | +| 4k | Requires `durationSeconds: 8` | --- ## Docker / PHP Notes -PHP's `post_max_size` and `upload_max_filesize` **cannot** be changed with `ini_set()` at runtime — they must be in `php.ini` / a conf.d file read by php-fpm before request processing. +**`post_max_size` and `upload_max_filesize`** cannot be changed with `ini_set()` at runtime in php-fpm. Must be set in a `conf.d/*.ini` file in the Dockerfile: -Set in `docker/Dockerfile`: ```dockerfile RUN printf "post_max_size = 100M\nupload_max_filesize = 100M\nmemory_limit = 512M\n" \ > /usr/local/etc/php/conf.d/uploads.ini ``` -GD extension is required for Veo I2V image resizing: +**nginx `client_max_body_size`** must also be set — default is 1MB, which 413s any base64 image payload: + +```nginx +http { + client_max_body_size 100M; + ... +} +``` + +**GD extension** required for Veo I2V image resizing. Fails with a fatal (not catchable) error if missing — use `function_exists('imagecreatefromstring')` guard: + ```dockerfile -RUN apk add --no-cache libpng-dev libjpeg-turbo-dev \ +RUN apk add --no-cache libpng-dev libjpeg-turbo-dev zlib-dev \ && docker-php-ext-configure gd --with-jpeg \ && docker-php-ext-install -j$(nproc) gd ``` + +**Prompt optimization** (VideoGenTab): must run server-side via `video_api.php?action=optimize_prompt` — direct browser calls to Gemini exhaust the API key's rate limits fast. Backend uses `gemini-2.0-flash`.