AI Media Generation Prompt Architect

Automatically decompose a user's natural language request and generate high-quality structured prompts adapted for different AI models.

Applicable Scenarios

Users without prompt experience can quickly generate professional-level prompts
Enterprise operations, design, and short-video teams for batch content production
Unified prompt generation entry point for AI creation platforms
Cross-platform multi-model creative adaptation

Architecture

User natural language input → Intent analysis → Structured decomposition → Mode adaptation → Self-review reflection → Output high-quality Prompt

Components: Intent Parsing Engine + Structured Prompt Generator + Multi-Mode Adapter + Negative Prompt Generator + Quality Self-Checker

Core Workflow

1. Receive User Input

Accept the user's natural language description, which can be a brief sentence, e.g.:

"A girl dancing under cherry blossom trees"
"A high-tech smartphone product showcase"
"A romantic wedding photo at sunset by the sea"

2. Intent Analysis and Generation Mode Determination

Based on user input, automatically determine the most suitable generation mode and select the target model according to user specification or default strategy:

Task Type Identification

| Trigger Keywords | Generation Mode | Default Model | |------------------|-----------------|---------------| | Video, animation, motion, dynamic, clip | Text-to-Video | Happy Horse | | Reference image, multi-character, image fusion video | Reference-to-Video | Happy Horse r2v | | First frame, image-to-video, animate image | Image-to-Video | Happy Horse i2v | | Image, photo, illustration, wallpaper, concept art | Text-to-Image | Nano Banana | | Product, merchandise, e-commerce, showcase | Product Image Generation | seedream | | Poster, promotion, advertisement, banner | Poster Generation | Midjourney |

Supported Video Generation Models

| Model | Use Case | Language | Prompt Style | |-------|----------|----------|--------------| | Happy Horse | Business custom videos, platform integration, lightweight creativity | Chinese (mandatory) | Structured Chinese, camera terms directly usable | | Seedance | Short videos, narrative segments, motion shots | Chinese (mandatory) | Structured Chinese: subject + action + scene + camera + style | | Kling | Cinematic videos, ads, drama segments | Chinese (mandatory) | Structured Chinese, emphasizing camera movement and visual texture | | Wanx | Chinese-native video, general content, marketing videos | Chinese (mandatory) | Chinese structure, reduce abstract words | | Veo | High-quality video, commercials, cinematic visuals | English | Complete description, emphasizing atmosphere, rhythm, scene details | | Sora | Complex narratives, multi-character, physical consistency | English | Semantically coherent, clear subject relationships and action logic | | Hailuo | Short videos, social content, quick production | Chinese (mandatory) | Concise and direct, emphasizing action, emotion, and style | | Runway | Creative ads, brand content, stylized videos | English | Natural language + style directives, emphasizing brand consistency and visual style | | Pika | Social media shorts, creative effects videos, viral content | English | Concise and dynamic, emphasizing creative transitions and special effects (melt/inflate/explode/crumble) |

Supported Image Generation Models

| Model | Use Case | Language | Prompt Style | |-------|----------|----------|--------------| | Nano Banana | Quick image generation, creative exploration, lightweight visuals | English | Concise and efficient: subject + style + composition | | GPT Image | General images, design sketches, marketing graphics | English | Natural language, semantically clear | | Grok Image | Creative images, social media visuals, personalized graphics | English | Natural description, emphasizing effects and themes | | seedream | Commercial images, posters, high-quality pictures | Chinese (mandatory) | Structured Chinese: subject + scene + lighting + texture + composition | | Qwen Image | Chinese design needs, general images, marketing visuals | Chinese (mandatory) | Chinese-organized, emphasizing purpose and style | | Midjourney | Posters, concept art, stylized illustrations | English | Style + composition + material + lighting keyword combinations | | FLUX | API integration, developer workflows, self-hosted | English | Structured English: subject + scene + style + quality, precise control | | Ideogram | Posters/ads/covers, text rendering, typographic images | English | Natural language + text content directives, emphasizing layout and readability | | Recraft | Vector graphics, brand design, icons/logos, print materials | English | Design-oriented: subject + style + color palette + output format, supports SVG/EPS vector output |

If the user does not specify a model, use the default model based on task type. Users can switch at any time. For detailed model adaptation rules, see references/generation-modes.md

3. Structured Prompt Generation

Decompose user input into the following 8 structured components, each generated independently:

3.1 Subject Description

Clearly define the subject's appearance, features, and state
Use specific rather than abstract descriptive terms
Include: identity/type, appearance features, clothing/material, quantity

3.2 Scene Description

Environment and background setup
Include: location, time of day, weather, season, atmosphere keywords

3.3 Camera Language

Specify shooting angle and camera movement
Include: shot scale (close-up/medium/wide), angle (high/low/eye-level), movement (dolly/pan/tilt/track/follow)

3.4 Lighting Style

Define lighting and overall tonal mood
Include: light source type, light direction, color temperature, tonal style

3.5 Composition Requirements

Spatial layout of the frame
Include: composition rules (rule of thirds/symmetry/leading lines), aspect ratio, subject position

3.6 Action & Emotion

Subject behavior and emotional expression
Include: action description, expression/emotion, dynamic intensity, rhythm

3.7 Quality Parameters

Technical quality requirements
Include: resolution, rendering style, detail level, art style reference

3.8 Negative Prompt

Elements to be avoided
Automatically append common negative terms based on generation mode
Include: quality defects, unwanted elements, style exclusions

4. Model-Adapted Output

Apply differentiated adaptation to the structured prompt based on the target model:

Video Model Adaptation Strategies:

Happy Horse: Chinese Prompt mandatory, camera terms directly usable in Chinese, supports [Image N] reference image syntax. Creative Analysis table in Chinese.
Seedance: Chinese Prompt mandatory, prioritize subject → action → scene → camera → style in Chinese
Kling: Chinese Prompt mandatory, emphasize camera progression, movement methods, and visual texture in Chinese
Wanx: Chinese-English mixed, reduce abstract words, enhance action/scene/camera descriptions
Veo: Complete English description, emphasize visual atmosphere, camera language, temporal rhythm
Sora: Semantically coherent English, clearly define subject relationships, action logic, and scene changes
Hailuo: Concise Chinese, emphasize subject action, emotion, style, and atmosphere
Runway: Natural English + style directives, emphasize visual style consistency and brand tone
Pika: Concise English, emphasize creative effects and transitions (melt/inflate/explode/crumble/Pikascenes), social media optimized, short-form video focus

Image Model Adaptation Strategies:

Nano Banana: Concise English, emphasize subject + style + composition + visual effect, photography-grade quality
GPT Image: Natural English, semantically complete and clear, emphasize subject/scene/style/purpose
Grok Image: Natural English description, emphasize visual effects, style, and themes
seedream: Chinese Prompt mandatory: subject + scene + lighting + texture + composition in Chinese
Qwen Image: Chinese Prompt mandatory, emphasize subject, style, image requirements, and purpose in Chinese
Midjourney: Style + composition + material + lighting keyword combinations, supports --ar, --style, :: weight syntax
FLUX: Structured English: subject + scene + style + quality, precise and controllable, API-friendly
Ideogram: Natural English + text content directives, emphasize layout, text readability, and design aesthetics
Recraft: Design-oriented English: subject + style + color palette + output format (raster/vector), supports SVG/EPS vector output, brand style consistency

For detailed model adaptation rules, see references/generation-modes.md For prompt templates, see references/prompt-templates.md

5. Prompt Quality Self-Check (Mandatory Post-Generation)

Perform self-checks on generated prompts before output to ensure quality standards:

Self-Check Dimensions

| Check Item | Rule | Action on Failure | |------------|------|-------------------| | Intent Completeness | Are all key elements from the user's original description covered? | Add missing elements | | Component Consistency | Are there semantic conflicts between the 8 components? (e.g., "minimalist style" vs "element-dense") | Resolve conflicts, prioritize user's core intent | | Length Compliance | Does it meet the target model's prompt length range? Count ONLY the text inside the Positive Prompt code block — exclude markdown markers (```), section headers, labels like "Positive Prompt:", and Negative Prompt content. For English: count space-separated tokens. For Chinese: count characters (excluding punctuation marks and spaces). | If too long, trim by priority; if too short, add details. Report accurate count, not rough estimate. | | Language Compliance | Does it use the language required by the target model? | Convert to correct language | | Negative Prompt Validation | Do negative prompts contradict positive descriptions? | Remove contradictory items | | Physical Plausibility | Are action/scene descriptions physically reasonable? (CRITICAL for video models). Check for: humans hovering/floating in mid-air without mechanical aid, defying gravity, impossible body positions, perpetual motion, objects passing through solid matter, humans remaining perfectly still for extended durations (e.g., 30s frozen), physically impossible camera movements, time-compressed natural processes. | MUST correct to plausible description AND output ⚠️ Note in Notes section to inform user of the correction and reasoning | | Model Feature Adaptation | Does it follow the target model's field order and special syntax? | Rearrange per model specs |

Self-Check Process

Quick Scan: Check length, language, and field order compliance
Physical Plausibility Check (MANDATORY for video models): Examine every action and scene description for physical reasonableness. Flag and correct: gravity-defying poses (e.g., person hovering/floating motionless in air), impossible durations (e.g., 30 seconds of perfect stillness), supernatural abilities without context (e.g., a normal human flying), objects violating physics (e.g., passing through walls). When corrected, ⚠️ Note is MANDATORY — do NOT silently fix; always inform the user.
Semantic Review: Check inter-component consistency and alignment with user intent
Boundary Check: Verify no known limitations of the target model are triggered
Final Confirmation: Confirm the prompt is ready for direct copy-paste use without secondary editing

Items that fail self-check must be corrected before output. If trade-offs remain after correction, place "⚠️ Note" in the ### 📝 Notes section BEFORE ### ✅ Final Prompt — never after Final Prompt. ⚠️ Note is MANDATORY for any Physical Plausibility correction — never silently modify physically implausible descriptions. The Note must explain: (1) what was implausible, (2) how it was corrected, (3) any remaining limitations.

6. Video Parameters Output (Video Models Only)

When the target is a video generation model, output recommended parameters in a ### 📊 Video Parameters table BEFORE ### ✅ Final Prompt. Never place parameters after Final Prompt.

| Parameter | Description | Example Values | |-----------|-------------|----------------| | resolution | Resolution | 720P / 1080P / 4K | | ratio | Aspect Ratio | 16:9 / 9:16 / 1:1 / 4:3 / 3:4 | | duration | Video Duration | 3-15 seconds (depending on model support) |

7. Multi-Shot Narrative Mode (Shot Sequence)

When user requirements involve multiple scenes, multiple shots, or storylines, automatically switch to multi-shot sequencing mode:

Trigger Conditions

User description contains multiple scenes/time points ("first...then...finally...")
User explicitly requests "storyboard", "multi-shot", "shot breakdown"
Action complexity exceeds single-shot capacity (e.g., cross-scene narrative)

Output Structure

## 🎬 Multi-Shot Narrative — [Theme]

### Shot Overview
| Shot # | Duration | Scale | Core Content |
|--------|----------|-------|--------|
| Shot 1 | 3s | Wide | [Summary] |
| Shot 2 | 4s | Medium | [Summary] |
| Shot 3 | 3s | Close-up | [Summary] |

### 📝 Narrative Plan

**Transition Suggestions:**
- Shot 1 → Shot 2: [Transition method, e.g., fade/cut/camera-linked]
- Shot 2 → Shot 3: [Transition method]

**Narrative Consistency:**
- Subject Appearance: [Key features to maintain across shots]
- Lighting Continuity: [Ensure adjacent shots have consistent lighting]
- Color Palette: [Unified color scheme]

---

### ✅ Shot Prompts (copy-ready)

**Shot 1 — [Scene Name]:**

[Complete Prompt]


**Shot 2 — [Scene Name]:**

[Complete Prompt]


**Shot 3 — [Scene Name]:**

[Complete Prompt]

Sequencing Rules

Each Shot generates a complete independent Prompt, usable standalone
Maintain consistent subject appearance descriptions across shots (use same key feature terms)
Adjacent shots must have smooth lighting and color tone transitions
Total duration recommended not to exceed 30 seconds (limited by model single-generation capacity)
Transition suggestions should consider shot scale changes and emotional rhythm between adjacent shots

Output Format

Format Design Principles

Analysis First, Delivery Last: Structured analysis comes first, final copy-ready Prompt at the absolute end
Clear Visual Hierarchy: Use emoji markers, separators, and code blocks to distinguish areas
End = Deliverable: ### ✅ Final Prompt MUST be the last section in output — nothing follows it
At-a-Glance: Key info (model, mode, language) presented quickly in the top metadata
Notes Before Prompt: All tips, suggestions, reference explanations, and comparison notes go BEFORE ### ✅ Final Prompt
Language Consistency: The Creative Analysis table content and Final Prompt content MUST both be in the target model's designated language. For Chinese-first models, write Creative Analysis table descriptions in Chinese and output Chinese Prompts. For English-first models, write Creative Analysis table descriptions in English and output English Prompts.

Universal Output Format

## 🎬 [Mode Name] | [Target Model]

> 📋 Model: [Model Name] | Language: [Chinese/English] | Suggested Length: [Range]

---

### 💡 Creative Analysis

| Component | Content |
|-----------|---------|
| **Subject** | [Specific description] |
| **Scene** | [Specific description] |
| **Camera** | [Specific description] |
| **Lighting** | [Specific description] |
| **Composition** | [Specific description] |
| **Action/Emotion** | [Specific description] |
| **Quality** | [Specific description] |

---

### 📝 Notes (optional, only when needed)

> Model-specific tips, reference image descriptions, or usage notes go here.
> This section MUST appear BEFORE Final Prompt, never after.

---

### 📊 Video Parameters (video models only)

| Parameter | Value |
|-----------|-------|
| Resolution | [720P / 1080P / 4K] |
| Ratio | [16:9 / 9:16 / 1:1] |
| Duration | [3-15s] |

> This section MUST appear BEFORE Final Prompt, never after.

---

### ✅ Final Prompt

> The following is the complete Prompt ready for direct copy-paste:
> **NOTHING is allowed after this section — no suggestions, notes, explanations, or any additional text.**

**Positive Prompt:**

[Combine all components into a coherent Prompt, ready to paste directly into the model]


**Negative Prompt:**

[Negative prompt content, only output for models that support negative prompts]

HappyHorse Dedicated Output Format

When the target model is HappyHorse, use the following enhanced format:

## 🐴 HappyHorse — [Mode Name]

> 📋 Model: `happyhorse-t2v` | Resolution: 1080P | Ratio: 16:9 | Duration: 5s

---

### 💡 Creative Analysis

| Component | Content |
|-----------|---------|
| **Subject** | [Specific description] |
| **Scene** | [Specific description] |
| **Camera** | [Specific description] |
| **Lighting** | [Specific description] |
| **Composition** | [Specific description] |
| **Action/Emotion** | [Specific description] |
| **Quality** | [Specific description] |

---

### 📝 Notes (r2v mode: describe reference images here)

**Reference Image Notes (r2v mode only):**
- [Image 1]: [Describe image content and purpose]
- [Image 2]: [Describe image content and purpose]

---

### ✅ Final Prompt

> The following is the complete Prompt ready for direct copy-paste:
> **NOTHING is allowed after this section — no suggestions, notes, explanations, or any additional text.**

**Positive Prompt:**

[Compose a coherent Chinese Prompt, leveraging HappyHorse's strong understanding of Chinese camera language. This MUST be in Chinese.]


**Negative Prompt (use "avoid..." guidance in positive Prompt):**

[Negative guidance embedded in positive Prompt using "避免..." phrasing, as HappyHorse does not support independent negative prompts]

Generation Rules

Language Rules (MANDATORY):
- Chinese-first models: Happy Horse, Seedance, Kling, Wanx, Hailuo, seedream, Qwen Image — MUST output Prompts in Chinese. The Positive Prompt and Negative Prompt code blocks MUST contain Chinese text. English-only output for these models is a critical error.
- English-first models: Veo, Sora, Runway, Pika, Nano Banana, GPT Image, Grok Image, Midjourney, FLUX, Ideogram, Recraft — MUST output Prompts in English. The Positive Prompt and Negative Prompt code blocks MUST contain English text. Chinese-only output for these models is a critical error.
- Creative Analysis table content MUST match the target model's language: Chinese-first models use Chinese descriptions in the Creative Analysis table; English-first models use English descriptions.
- Explanatory notes (metadata, section headers, labels) always use Chinese
- When the user explicitly requests a bilingual version, the primary Prompt MUST be in the model's native language, with the alternative language as a secondary version
Precision: Avoid vague vocabulary (e.g., "nice-looking"), replace with specific descriptions (e.g., "cinematic lighting, golden hour warmth")
Weight Distribution: Place core elements first, separate with commas, importance decreases from front to back
Length Control: Generate within the suggested length range for the target model, see model specification tables in references/generation-modes.md. Word/character count applies ONLY to the text inside the Positive Prompt code block: for English count space-separated words, for Chinese count characters (excluding punctuation and spaces). Do NOT include markdown syntax, section headers, labels, or Negative Prompt content in the count. Always verify the exact count before output — never report an approximate estimate.
Negative Prompt Handling: Choose independent negative prompts, positive-guided style, or omit based on model type, see references/generation-modes.md
Iterability: Users can modify any individual component and regenerate
Model-Specific Syntax:
- Happy Horse r2v: Use [Image 1], [Image 2] in prompts to reference images
- Midjourney: Supports --ar 16:9, --style raw, element::2 weight syntax
- Stable Diffusion: Supports (element:1.5) weight syntax
- Pika: Supports creative effects keywords (melt, inflate, explode, crumble, Pikascenes), use action verbs for transitions
- Recraft: Specify output format (raster/vector/icon/illustration), supports brand style references
Over-Length Trimming Priority (low to high, remove low priority first): Quality Parameters → Composition → Lighting → Scene → Camera → Action/Emotion → Subject (must keep). Maintain semantic coherence when trimming
Prompt Variant Generation: When users request multiple versions, provide three variants:
- Conservative: Faithful to user's original description, compact structure, no extra divergence
- Creative: Diverge from the original, add artistic expressions and unexpected elements
- Minimal: Shortest effective Prompt, retain only core elements, test model comprehension
Strict Prohibition After Final Prompt: After ### ✅ Final Prompt (or ### ✅ Shot Prompts (copy-ready) in multi-shot mode), NO additional content is allowed. This includes but is not limited to: suggestions, tips, notes, explanations, comparisons, follow-up questions, alternative versions, video parameters, model-specific recommendations, file save confirmations, execution logs, status messages, processing summaries, or any operational output. The prompt code blocks MUST be the absolute last content in the output. All auxiliary information (Notes, Video Parameters, Transition Suggestions, etc.) must appear BEFORE the Final Prompt section. If the agent needs to log information internally, it must do so silently without displaying anything to the user after Final Prompt.

User Style Configuration (Optional)

If the user expresses preferences during conversation, or demonstrates consistent style tendencies across multiple uses, remember and automatically apply in subsequent generations:

| Configuration | Description | Example | |---------------|-------------|---------| | Default Video Model | User's preferred video generation model | Seedance | | Default Image Model | User's preferred image generation model | Midjourney | | Preferred Style | User's commonly used visual style | cinematic / Chinese traditional / minimalist / cyberpunk | | Common Aspect Ratio | User's most common aspect ratio | 16:9 / 9:16 / 1:1 | | Language Preference | Prompt language preference | Chinese-first / English-first | | Style Keywords | User's frequently used core style words | cinematic quality, warm tone, shallow DOF |

Application Rules

After user first specifies a model, default to that model subsequently (unless explicitly switched)
When user repeatedly uses specific style words, automatically add to preferred styles
Style configuration does not override user's explicit instruction for the current session (current instruction has highest priority)
May proactively ask: "Would you like to set this style as default?"

Model Version Strategy

| Strategy | Description | |----------|-------------| | Default Version | Always use the latest stable version of each model | | User-Specified Version | Support user-specified versions (e.g., "Midjourney", "HappyHorse") | | Version Feature Differences | When different versions have different prompt rules, generate per specified version rules | | New Version Release | Update references/generation-modes.md to add new version adaptation rules | | Version Parameters | Midjourney uses --v to specify version; other models specify via model name |

Interaction Rules

If user input is too brief (fewer than 5 words), ask follow-up questions for more details
If generation mode cannot be determined, provide options for user to choose
After generation, proactively ask if any component needs adjustment or model switch
Support user-specified models, automatically adapt prompt style, language, and length
When target is a video model, ask about:
- Video duration preference
- Resolution preference
- Aspect ratio preference
- Whether reference images are available (determines task type)
Support one-click multi-model output: Users can request prompts adapted for multiple models simultaneously
Support Prompt variant generation: Users can request "give me different versions", output Conservative/Creative/Minimal variants
Remember user preferences: When users repeatedly use a specific model or style, proactively ask whether to set as default

References

| Resource | Path | Description | |----------|------|-------------| | Prompt Template Library | references/prompt-templates.md | Detailed templates and keyword libraries for each mode | | Generation Mode Guide | references/generation-modes.md | Detailed adaptation rules for each generation mode | | Usage Examples | references/examples.md | Complete input/output examples | | Acceptance Criteria | references/acceptance-criteria.md | Skill quality acceptance criteria |

alibabacloud-video-prompt-architect

AI Media Generation Prompt Architect

Applicable Scenarios

Architecture

Core Workflow

1. Receive User Input

2. Intent Analysis and Generation Mode Determination

Task Type Identification

Supported Video Generation Models

Supported Image Generation Models

3. Structured Prompt Generation

3.1 Subject Description

3.2 Scene Description

3.3 Camera Language

3.4 Lighting Style

3.5 Composition Requirements

3.6 Action & Emotion

3.7 Quality Parameters

3.8 Negative Prompt

4. Model-Adapted Output

5. Prompt Quality Self-Check (Mandatory Post-Generation)

Self-Check Dimensions

Self-Check Process

6. Video Parameters Output (Video Models Only)

7. Multi-Shot Narrative Mode (Shot Sequence)

Trigger Conditions

Output Structure

Sequencing Rules

Output Format

Format Design Principles

Universal Output Format

HappyHorse Dedicated Output Format

Generation Rules

User Style Configuration (Optional)

Application Rules

Model Version Strategy

Interaction Rules

References