Script Planner Assistant▸ 
AI Apps 
AI News 
LumenX Studio: Alibaba's Open-Source One-Stop AI Short Drama Production Platform
LumenX Studio: Alibaba's Open-Source One-Stop AI Short Drama Production PlatformIntroduction
LumenX Studio Introduction
LumenX Studio is an AI short comic drama production platform open-sourced by Alibaba, led in development and maintenance by StarLotus. It can convert novel text into dynamic video, covering the complete creation pipeline from script analysis, character customization, storyboard drawing to video composition. It supports local deployment and is open-sourced under the MIT license.
LumenX Studio interface screenshot
Core Features
Script Analysis: Automatically extracts characters, scenes, and props based on LLM, generating professional storyboard scripts
Art Direction: Supports custom visual styles to maintain consistent art style throughout the entire piece
Visual Storyboarding: Drag-and-drop storyboard editor for WYSIWYG composition of characters, backgrounds, and effects
Multimodal Generation: Integrates models such as Tongyi Wanxi (Wanx), supporting text-to-image and image-to-video
Intelligent Audio-Visual Composition: Automatically generates character voiceover (TTS), sound effects (SFX), and synthesizes the final video
LumenX Studio Complete Creation Pipeline
The platform integrates the following complete workflow:
Asset Extraction → Style Setting → Asset Generation → Storyboard Script Construction → Storyboard Image Generation → Storyboard Video Generation
Specific process:
Script Editing and Entity Extraction: Click "Extract Entities" in the script editor on the left, and Qwen-Plus will automatically extract characters, scenes, and props
Style Setting: Use Qwen-Plus to analyze the suitable style for the script, or use preset styles (composed of positive/negative prompts)
Asset Generation: Generate background-less full-body images, three-view images, and avatar close-ups for characters; generate reference images or reference videos for scenes and props
Storyboard Image Storyboarding: Extract storyboard scripts based on the script, support secondary editing, and integrate AI prompt polishing
Storyboard Video Generation: Supports first-frame driven (i2v) and character action driven (r2v) modes, with multi-batch card drawing
Storyboard Video Stitching: Review and select the best storyboard videos, and stitch them together with one click
LumenX Studio Technical Architecture
Frontend: Next.js 14 + React 18 + TypeScript + Tailwind CSS
Backend: FastAPI + Python 3.11+
AI Core: Alibaba Cloud Qwen (Logic) + Wanx (Vision)
Rendering: Three.js (Canvas) + FFmpeg (Video Processing)
Model Integration and Operating Modes
A built-in workflow for model integration is provided, supporting new model integration, version updates, and parameter adjustments. It adopts a local-first media storage logic. Four operating modes are supported:
DashScope-only (without OSS)
DashScope + OSS
DashScope + Kling Original Factory
DashScope + Vidu Original Factory
LumenX Studio vs Toonflow vs Huobao Drama (Drama)
| Comparison Dimension | LumenX Studio | Toonflow | Huobao Drama (Drama) |
|---|---|---|---|
| Developer | Alibaba / StarLotus | HBAI-Ltd | chatfire-AI |
| GitHub | alibaba/lumenx | HBAI-Ltd/Toonflow-app | chatfire-AI/huobao-drama |
| Core Positioning | One-stop AI short comic drama production platform | AI workstation for short drama production | Full-stack AI short drama automated production platform |
| Tech Stack | Next.js + FastAPI + Python + Qwen/Wanx | Node.js + Express + Electron | Go + Vue3 + Gin |
| Open Source License | MIT | Apache-2.0 | Open Source |
| Core Workflow | Asset Extraction → Style Setting → Asset Generation → Storyboard Script → Storyboard Image → Storyboard Video | Planning → Scriptwriting → Storyboard → Output (Infinite Canvas) | Character Generation → Storyboard Design → Video Generation |
| Character Consistency | Generate background-less full-body, three-view, avatar close-up writing as asset locking | Chapter Event Graph + Persistent Agent Memory | AI painting unified style character image generation |
| Script/Storyboard Processing | LLM automatic entity extraction + visual drag-and-drop storyboard editor | Three-layer Agent collaboration (Decision/Execution/Supervision) + Event graph driven | Intelligent storyboard design (scene description, shot design, matching images) |
| Video Generation | First-frame driven (i2v) + character action driven (r2v), supports multi-batch card drawing | Integrates GPT-4o, Sora, Doubao Video, Nano Banana Pro, etc. | Text-to-video + Image-to-video |
| Audio Processing | Automatic TTS voiceover + SFX sound effects generation | Not specified | Not specified |
| Deployment Method | Local deployment (frontend 3008/backend 8000), supports Docker | Windows/Linux/macOS installer, Docker, Cloud | Docker Compose (recommended), development mode, traditional deployment |
| Deployment Difficulty | Medium | Low | Low |
| Key Highlights | Local-first media storage; repository-level model integration; DashScope/Kling/Vidu multi-mode selectable | Infinite canvas production workstation; programmable vendor system; Skill file-based configuration; batch hundred-episode production | DDD domain-driven architecture; one-sentence short drama generation; asset library + task tracking |
| Applicable Scenarios | Alibaba ecosystem users, need flexible multi-model switching, local-first storage | Batch novel promotion videod> | Quick start, full-stack automation, lightweight deployment with good support in China |
Open Source Information
GitHub Repository: https://github.com/alibaba/lumenx
License: MIT License
Development Ports: Frontend default 3008, Backend 8000
Contact: GitHub Issues / Discussions, Email [email protected]
Related Articles:
