Vadym/obsidian
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

vault backup: 2026-04-24 18:42:47

Browse source
This commit is contained in:
Vadym Samoilenko 2026-04-24 18:42:47 +01:00
parent 62506c7456
commit 2ac4192d8e
3 changed files with 162 additions and 61 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
01 Projects/cinema-studio-pro-kling/Cinema Studio Pro (Kling).md
View file

@ -89,6 +89,22 @@ cd backend && composer install && php -S localhost:5015 # :5015
```
## Sessions
### 2026-04-24 – Asked | Fix API generation errors
**Asked:** Asked | Fix API generation errors with Veo and Kling, push code to git
**Done:** Done | Code committed (a2358ba); identified JSON parsing and rate limit issues requiring server logs for diagnosis
### 2026-04-24 – Fixed JSON parsing errors in Veo
**Asked:** Fixed JSON parsing errors in Veo and Kling API generation calls | Rewrote wiki with corrected I2V fields, camera_control parameters, and duration formatting; pushed code to git | wiki.md, API documentation
**Done:** —
### 2026-04-24 – Asked for help debugging API errors
**Asked:** Asked for help debugging API errors with Veo and Kling generation failures.
**Done:** Rewrote wiki documentation with accurate API specifications for I2V fields, camera control, cfg_scale, and duration parameters.
### 2026-04-24 – Fix JSON parsing errors and API
**Asked:** Fix JSON parsing errors and API rate limit issues with Veo and Kling generation.
**Done:** Updated Obsidian wiki with correct API documentation and error handling guidance from official docs.
### 2026-04-24 – Fix generation failures on Veo and
**Asked:** Fix generation failures on Veo and Kling API integrations and update frontend model list with camera control visibility.
**Done:** Updated frontend model list, wrapped Camera Control section with conditional rendering, and added reset logic when switching to unsupported models.
@ -241,6 +257,9 @@ cd backend && composer install && php -S localhost:5015 # :5015
## Change Log
| Date | Requested | Changed | Files |
|------|-----------|---------|-------|
| 2026-04-24 | API fixes | JSON parsing error handling, rate limit handling | (no files changed in this session) |
| 2026-04-24 | API documentation | I2V fields (base64 encoding), camera_control scope, cfg_scale removal, duration format | wiki |
| 2026-04-24 | API error handling | JSON parsing validation, rate limit retry logic documentation | wiki.md, docs.md |
| 2026-04-24 | API error handling, frontend model list, camera control visibility | Added conditional camera control rendering, reset logic on model switch, updated supported models list | Frontend model configuration, Camera Control component |
| 2026-04-24 | Veo and Kling API fixes | Field names corrected, JSON parsing improved, base64 encoding standardized | api.ts, videoGeneration.ts |
| 2026-04-24 | API generation fixes | nginx client_max_body_size 100M, Veo I2V bytesBase64Encoded format | nginx.conf, veo-api-handler.ts |

12
99 Daily/2026-04-24.md
View file

@ -353,3 +353,15 @@ tags: [daily]
- 18:35 | `cinema-studio-pro-kling`
- **Asked:** Fix generation failures on Veo and Kling API integrations and update frontend model list with camera control visibility.
- **Done:** Updated frontend model list, wrapped Camera Control section with conditional rendering, and added reset logic when switching to unsupported models.
- 18:39 | `cinema-studio-pro-kling`
- **Asked:** Fix JSON parsing errors and API rate limit issues with Veo and Kling generation.
- **Done:** Updated Obsidian wiki with correct API documentation and error handling guidance from official docs.
- 18:40 | `cinema-studio-pro-kling`
- **Asked:** Asked for help debugging API errors with Veo and Kling generation failures.
- **Done:** Rewrote wiki documentation with accurate API specifications for I2V fields, camera control, cfg_scale, and duration parameters.
- 18:41 | `cinema-studio-pro-kling`
- **Asked:** Fixed JSON parsing errors in Veo and Kling API generation calls | Rewrote wiki with corrected I2V fields, camera_control parameters, and duration formatting; pushed code to git | wiki.md, API documentation
- **Done:** —
- 18:41 | `cinema-studio-pro-kling`
- **Asked:** Asked | Fix API generation errors with Veo and Kling, push code to git
- **Done:** Done | Code committed (a2358ba); identified JSON parsing and rate limit issues requiring server logs for diagnosis

192
wiki/tech-patterns/kling-veo-video-api.md
View file

@ -15,85 +15,146 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/)
**Base URL:** `https://api-singapore.klingai.com`
**Auth:** HS256 JWT — `{ iss: accessKey, exp: now+1800, nbf: now-5 }` signed with secretKey
### Text-to-Video — `/v1/videos/text2video`
---
### Text-to-Video — POST `/v1/videos/text2video`
```json
{
"model_name": "kling-v3",
"prompt": "string (required)",
"model_name": "kling-v2-6",
"mode": "std",
"duration": 5,
"duration": "5",
"aspect_ratio": "16:9",
"cfg_scale": 0.5,
"negative_prompt": "...",
"sound": "on",
"camera_control": { ... }
"negative_prompt": "",
"sound": "on"
}
```
### Image-to-Video — `/v1/videos/image2video`
**Valid models:** `kling-v1`, `kling-v1-6`, `kling-v2-master`, `kling-v2-1-master`, `kling-v2-5-turbo`, `kling-v2-6`, `kling-v3`
> `kling-v3-omni` and `kling-video-o1` are NOT in the official T2V enum — use with caution.
**⚠️ `cfg_scale` is NOT supported by v2.x models** — omit it entirely for `kling-v2-*`.
**`duration`** must be a **string**: `"5"`, `"10"` etc. (API example uses string, not integer).
---
### Image-to-Video — POST `/v1/videos/image2video`
```json
{
"model_name": "kling-v2-6",
"mode": "std",
"duration": 5,
"aspect_ratio": "16:9",
"cfg_scale": 0.5,
"model_name": "kling-v3",
"image": "<raw_base64_no_prefix>",
"image_tail": "<raw_base64_no_prefix>",
"prompt": "optional",
"image_url": "data:image/jpeg;base64,<base64>",
"image_tail_url": "data:image/jpeg;base64,<base64>"
"mode": "std",
"duration": "5",
"aspect_ratio": "16:9",
"sound": "off"
}
```
> **Critical:** field names are `image_url` / `image_tail_url`, NOT `image` / `image_tail`.
> Value must be a data URI (`data:mime;base64,...`), not plain base64.
**Valid models:** `kling-v1`, `kling-v1-5`, `kling-v1-6`, `kling-v2-master`, `kling-v2-1`, `kling-v2-1-master`, `kling-v2-5-turbo`, `kling-v2-6`, `kling-v3`
### Camera Control
**⚠️ Base64 format:** raw base64 string only — **NO** `data:image/png;base64,` prefix.
**`image_tail`, `dynamic_masks`, and `camera_control` are mutually exclusive** — only one at a time.
---
### Camera Control (T2V and I2V)
Only supported by **`kling-v1`** (STD 5s T2V) and **`kling-v1-5`** (PRO 5s I2V, `simple` type only).
All v2.x, v1-6, v3 models: **no camera_control support** → API returns error 1200.
```json
"camera_control": {
"type": "down_back",
"config": { "horizontal": 0, "vertical": 0, "pan": 0, "tilt": 0, "roll": 0, "zoom": 0 }
"type": "down_back"
}
```
**Valid `type` values:** `simple`, `down_back`, `forward_up`, `right_turn_forward`, `left_turn_forward`
**`type` values:** `simple`, `down_back`, `forward_up`, `right_turn_forward`, `left_turn_forward`
> **Bug trap:** `type: "predefined"` is INVALID — the API returns error 1200.
> The preset name IS the type. For preset types, config must still include all 6 fields (set to 0).
> For `simple` type, config values drive actual movement (range -10 to +10).
- **Preset types** (`down_back`, `forward_up`, etc.): `config` must be **absent** (not sent).
- **`simple` type only**: provide `config` with exactly one non-zero value (range −10 to +10):
```json
"config": { "horizontal": 5, "vertical": 0, "pan": 0, "tilt": 0, "roll": 0, "zoom": 0 }
```
### Duration
**Bug history:** `type: "predefined"` is INVALID — API returns error 1200 "camera_control.type value 'predefined' is invalid". The preset name itself IS the type.
- Always **integer**, never string: `5` or `10` (v1/v2 models)
- Kling v3 supports 3–15 seconds
---
### Valid Model Names (official API)
### Motion Control — POST `/v1/videos/motion-control`
`kling-v1-6`, `kling-v2-6`, `kling-v2-5-turbo`, `kling-v2-1-master`, `kling-v3`
**⚠️ This is a completely separate endpoint** — NOT related to `camera_control` presets.
### Status Polling — `/v1/videos/{type}/{taskId}`
Takes a reference image + reference video; the character in the generated video follows the reference video's motion (pose transfer).
```json
{
"model_name": "kling-v2-6",
"image_url": "https://... or raw_base64_no_prefix",
"video_url": "https://...",
"prompt": "optional",
"character_orientation": "image",
"mode": "pro"
}
```
**Supported models:** `kling-v2-6`, `kling-v3` (std/pro only, not 4k)
Status polling: GET `/v1/videos/motion-control/{task_id}`
---
### Model Capability Map (key features)
| Model | T2V | I2V | camera_control | image_tail | sound | duration |
|-------|-----|-----|---------------|------------|-------|----------|
| kling-v1 | ✅ | ✅ | ✅ STD 5s T2V only | ✅ STD/PRO 5s | - | 5s/10s |
| kling-v1-5 | ❌ | ✅ | ✅ PRO 5s I2V (simple) | ✅ PRO | - | 5s/10s |
| kling-v1-6 | ✅ | ✅ | ❌ | ✅ PRO | - | 5s/10s |
| kling-v2-master | ✅ | ✅ | ❌ | ❌ | - | 5s/10s |
| kling-v2-1 | ❌ | ✅ | ❌ | ✅ PRO | - | 5s/10s |
| kling-v2-1-master | ✅ | ✅ | ❌ | ❌ | - | 5s/10s |
| kling-v2-5-turbo | ✅ | ✅ | ❌ | ✅ PRO | - | 5s/10s |
| kling-v2-6 | ✅ | ✅ | ❌ | ✅ PRO (no audio) | ✅ PRO | 5s/10s |
| kling-v3 | ✅ | ✅ | ❌ | ✅ | ❌ | 3–15s |
| kling-v3-omni | ✅ | ✅ | ❌ | ✅ | ❌ | 3–15s |
| kling-video-o1 | ✅ | ✅ | ❌ | ✅ | ❌ | 5s/10s |
**`cfg_scale`**: supported only by v1.x models — **omit for all v2.x models**.
**`sound`**: `kling-v2-6` PRO mode only; all other v1/v2 models do not have sound generation.
---
### Status Polling — GET `/v1/videos/{type}/{taskId}`
- `task_status`: `submitted` → `processing` → `succeed` / `failed`
- On `succeed`: `task_result.videos[0].url` — CDN URL (temporary, must download immediately)
- Downloaded videos stored at `/var/www/html/lux-studio/api/generated_videos/`
- URL returned to frontend: `/lux-studio/api/generated_videos/<filename>`
(nginx serves this path via `try_files` in the `/lux-studio/` location block)
- On `succeed`: `task_result.videos[0].url` — CDN URL (temporary, download immediately)
- Downloaded videos stored: `/var/www/html/lux-studio/api/generated_videos/`
- URL returned to frontend: `/lux-studio/api/generated_videos/<filename>`
### Lip Sync — `/v1/videos/lip-sync`
---
### Lip Sync — POST `/v1/videos/lip-sync`
```json
{
"input": {
"mode": "audio2video",
"audio_type": "file",
"audio_file": "<base64>",
"audio_file": "<raw_base64>",
"video_id": "task_id_of_kling_video"
}
}
```
Status polling: GET `/v1/videos/lip-sync/{task_id}`
---
### Error Codes
| Code | Meaning |
@ -110,7 +171,7 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/)
**Base URL:** `https://generativelanguage.googleapis.com/v1beta/models`
**Models:** `veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview`
**Auth:** `?key=GEMINI_API_KEY` query param
**Pattern:** Start long-running op → poll until done
**Endpoint:** `POST /{model}:predictLongRunning` (async, returns operation ID)
### Request Structure
@ -119,15 +180,17 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/)
"instances": [{
"prompt": "string",
"image": {
"inlineData": { "mimeType": "image/jpeg", "data": "<base64>" }
"bytesBase64Encoded": "<raw_base64>",
"mimeType": "image/jpeg"
},
"lastFrame": {
"inlineData": { "mimeType": "image/jpeg", "data": "<base64>" }
"bytesBase64Encoded": "<raw_base64>",
"mimeType": "image/jpeg"
}
}],
"parameters": {
"aspectRatio": "16:9",
"durationSeconds": "8",
"durationSeconds": 4,
"resolution": "720p",
"sampleCount": 1,
"personGeneration": "allow_adult"
@ -135,44 +198,51 @@ Used in: **cinema-studio-pro-kling** (optical-prod.oliver.solutions/lux-studio/)
}
```
> **Critical:** Use `inlineData` format (Gemini API), NOT `bytesBase64Encoded` (Vertex AI format).
> `durationSeconds` must be a **string** (`"4"`, `"6"`, `"8"`), not an integer.
**⚠️ Critical format details:**
- Uses `bytesBase64Encoded` (predict-style API), NOT `inlineData` (generateContent style)
- `durationSeconds` is an **integer** (`4`, `6`, `8`), NOT a string
- `personGeneration`: T2V → `allow_all`, I2V → `allow_adult`
- `image` = start frame, `lastFrame` = end frame (interpolation, requires 8s duration)
### I2V Image Requirements
### Polling — GET `https://generativelanguage.googleapis.com/v1beta/{operationId}`
- `image` field: start frame (first frame)
- `lastFrame` field: end frame for A→B interpolation (requires `durationSeconds: "8"`)
- Image should match requested `aspectRatio` — mismatch can cause `INVALID_ARGUMENT`
- Supported MIME: `image/jpeg`, `image/png`
- The server resizes uploaded images to match aspect ratio using PHP GD before sending
- Response `done: true` → check `response.generateVideoResponse.generatedSamples[0].video.uri`
- Video URL is temporary — download immediately and store locally
### Person Generation
### Resolution Constraints
| Mode | Valid value |
|------|------------|
| T2V | `allow_all` |
| I2V / interpolation | `allow_adult` |
### Duration / Resolution Constraints
- 1080p and 4K require `durationSeconds: "8"`
- `lastFrame` interpolation requires `durationSeconds: "8"`
| Resolution | Requirement |
|-----------|-------------|
| 720p | Any duration |
| 1080p | Requires `durationSeconds: 8` |
| 4k | Requires `durationSeconds: 8` |
---
## Docker / PHP Notes
PHP's `post_max_size` and `upload_max_filesize` **cannot** be changed with `ini_set()` at runtime — they must be in `php.ini` / a conf.d file read by php-fpm before request processing.
**`post_max_size` and `upload_max_filesize`** cannot be changed with `ini_set()` at runtime in php-fpm. Must be set in a `conf.d/*.ini` file in the Dockerfile:
Set in `docker/Dockerfile`:
```dockerfile
RUN printf "post_max_size = 100M\nupload_max_filesize = 100M\nmemory_limit = 512M\n" \
> /usr/local/etc/php/conf.d/uploads.ini
```
GD extension is required for Veo I2V image resizing:
**nginx `client_max_body_size`** must also be set — default is 1MB, which 413s any base64 image payload:
```nginx
http {
client_max_body_size 100M;
...
}
```
**GD extension** required for Veo I2V image resizing. Fails with a fatal (not catchable) error if missing — use `function_exists('imagecreatefromstring')` guard:
```dockerfile
RUN apk add --no-cache libpng-dev libjpeg-turbo-dev \
RUN apk add --no-cache libpng-dev libjpeg-turbo-dev zlib-dev \
&& docker-php-ext-configure gd --with-jpeg \
&& docker-php-ext-install -j$(nproc) gd
```
**Prompt optimization** (VideoGenTab): must run server-side via `video_api.php?action=optimize_prompt` — direct browser calls to Gemini exhaust the API key's rate limits fast. Backend uses `gemini-2.0-flash`.