调用模型,返回事件的时间线
This commit is contained in:
185
API文档.md
Normal file
185
API文档.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# Marlin-2B Video VLM API 文档
|
||||
|
||||
## 基础信息
|
||||
- 服务地址:`http://0.0.0.0:8900`
|
||||
- 模型:Marlin-2B
|
||||
- 设备:Apple Silicon (MPS) / CUDA / CPU
|
||||
|
||||
---
|
||||
|
||||
## 1. 健康检查接口
|
||||
|
||||
**接口路径**:`GET /health`
|
||||
|
||||
**请求示例**:
|
||||
```bash
|
||||
curl http://localhost:8900/health
|
||||
```
|
||||
|
||||
**返回参数**:
|
||||
| 字段名 | 类型 | 说明 |
|
||||
|--------|------|------|
|
||||
| status | string | 状态:ok(已加载)或 loading(加载中) |
|
||||
| model | string | 模型名称,固定为 "Marlin-2B" |
|
||||
| device | string | 运行设备:mps/cuda/cpu |
|
||||
|
||||
**返回示例**:
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"model": "Marlin-2B",
|
||||
"device": "mps"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 视频字幕生成接口
|
||||
|
||||
**接口路径**:`POST /caption`
|
||||
|
||||
**功能说明**:为视频生成结构化字幕,包括场景描述和带时间戳的事件列表。
|
||||
|
||||
**请求参数**(multipart/form-data):
|
||||
| 参数名 | 类型 | 必填 | 说明 |
|
||||
|--------|------|------|------|
|
||||
| video | 文件 | 是 | 视频文件(支持 mp4, avi, mov, webm 等格式) |
|
||||
| max_new_tokens | int | 否 | 最大生成 token 数,默认值 2048 |
|
||||
|
||||
**请求示例**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8900/caption \
|
||||
-F "video=@/path/to/video.mp4" \
|
||||
-F "max_new_tokens=2048"
|
||||
```
|
||||
|
||||
**返回参数**:
|
||||
| 字段名 | 类型 | 说明 |
|
||||
|--------|------|------|
|
||||
| caption | string \| null | 完整的原始字幕文本 |
|
||||
| scene | string \| null | 场景描述段落 |
|
||||
| events | array \| null | 事件列表,每个事件包含 start/end/description |
|
||||
|
||||
**events 数组元素**:
|
||||
| 字段名 | 类型 | 说明 |
|
||||
|--------|------|------|
|
||||
| start | float | 事件开始时间(秒) |
|
||||
| end | float | 事件结束时间(秒) |
|
||||
| description | string | 事件描述 |
|
||||
|
||||
**返回示例**:
|
||||
```json
|
||||
{
|
||||
"caption": "Scene: ... Events: ...",
|
||||
"scene": "这是一个室内场景...",
|
||||
"events": [
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 5.0,
|
||||
"description": "一个人走进房间"
|
||||
},
|
||||
{
|
||||
"start": 5.5,
|
||||
"end": 10.0,
|
||||
"description": "这个人坐在沙发上"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 事件时间定位接口
|
||||
|
||||
**接口路径**:`POST /find`
|
||||
|
||||
**功能说明**:在视频中查找指定事件发生的时间区间(时间定位)。
|
||||
|
||||
**请求参数**(multipart/form-data):
|
||||
| 参数名 | 类型 | 必填 | 说明 |
|
||||
|--------|------|------|------|
|
||||
| video | 文件 | 是 | 视频文件 |
|
||||
| event | 字符串 | 是 | 自然语言事件查询,例如 "一个人进入房间" |
|
||||
|
||||
**请求示例**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8900/find \
|
||||
-F "video=@/path/to/video.mp4" \
|
||||
-F "event=一个人进入房间"
|
||||
```
|
||||
|
||||
**返回参数**:
|
||||
| 字段名 | 类型 | 说明 |
|
||||
|--------|------|------|
|
||||
| raw | string \| null | 原始模型输出,例如 "From 14.3 to 18.2." |
|
||||
| span | array \| null | 时间区间 [开始时间, 结束时间],单位秒 |
|
||||
| format_ok | bool | 输出格式是否符合训练格式 |
|
||||
|
||||
**返回示例**:
|
||||
```json
|
||||
{
|
||||
"raw": "From 14.3 to 18.2.",
|
||||
"span": [14.3, 18.2],
|
||||
"format_ok": true
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 自定义提示生成接口
|
||||
|
||||
**接口路径**:`POST /generate`
|
||||
|
||||
**功能说明**:使用自定义提示词与视频进行交互,实现更灵活的问答。
|
||||
|
||||
**注意**:此接口需要安装 `torchvision` 库才能正常工作。
|
||||
|
||||
**请求参数**(multipart/form-data):
|
||||
| 参数名 | 类型 | 必填 | 说明 |
|
||||
|--------|------|------|------|
|
||||
| video | 文件 | 是 | 视频文件 |
|
||||
| prompt | 字符串 | 是 | 自定义文本提示词 |
|
||||
| max_new_tokens | int | 否 | 最大生成 token 数,默认值 512 |
|
||||
| do_sample | bool | 否 | 是否启用采样,默认 false(确定性输出) |
|
||||
| temperature | float | 否 | 温度参数,控制随机性,默认 1.0 |
|
||||
| top_p | float | 否 | top-p 采样参数,默认 1.0 |
|
||||
|
||||
**请求示例**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8900/generate \
|
||||
-F "video=@/path/to/video.mp4" \
|
||||
-F "prompt=描述这个视频的内容" \
|
||||
-F "max_new_tokens=512" \
|
||||
-F "temperature=0.7"
|
||||
```
|
||||
|
||||
**返回参数**:
|
||||
| 字段名 | 类型 | 说明 |
|
||||
|--------|------|------|
|
||||
| text | string | 生成的文本内容 |
|
||||
|
||||
**返回示例**:
|
||||
```json
|
||||
{
|
||||
"text": "这个视频展示了..."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 错误响应
|
||||
|
||||
当请求失败时,接口返回 HTTP 错误码和错误信息:
|
||||
|
||||
| HTTP 状态码 | 说明 |
|
||||
|-------------|------|
|
||||
| 400 | 请求参数错误 |
|
||||
| 500 | 服务器内部错误 |
|
||||
| 503 | 模型尚未加载完成 |
|
||||
|
||||
**错误响应示例**:
|
||||
```json
|
||||
{
|
||||
"detail": "Caption failed: 错误详情"
|
||||
}
|
||||
```
|
||||
194
CLAUDE.md
Normal file
194
CLAUDE.md
Normal file
@@ -0,0 +1,194 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# 下载依赖
|
||||
go mod download
|
||||
|
||||
# 编译
|
||||
go build -o media main.go
|
||||
|
||||
# 运行(开发)
|
||||
go run main.go
|
||||
|
||||
# 格式化代码
|
||||
go fmt ./...
|
||||
|
||||
# 代码检查
|
||||
go vet ./...
|
||||
|
||||
# Docker 构建(多阶段构建,包含 FFmpeg 和 whisper 运行时)
|
||||
docker build -t media .
|
||||
|
||||
# Docker 运行
|
||||
docker run -p 3010:3010 media
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
这是一个多媒体处理微服务项目,基于 GoFrame 框架开发,提供视频处理、音频提取、语音识别等功能。
|
||||
|
||||
### 目录结构
|
||||
|
||||
```
|
||||
main.go # 应用入口
|
||||
config.yml # 配置文件
|
||||
consts/ # 常量定义
|
||||
- video/ # 视频相关常量(包括视频分析任务状态)
|
||||
controller/ # HTTP 控制器层(路由入口)
|
||||
- audio/ # 音频相关接口
|
||||
- video/ # 视频相关接口(拼接、剪切、分析)
|
||||
- common/ # 公共工具
|
||||
- scene/ # 场景检测接口
|
||||
- image/ # 图片处理接口
|
||||
service/ # 业务逻辑层
|
||||
- video/ # 视频服务(拼接、剪切、分析、分析队列)
|
||||
- audio/ # 音频提取服务
|
||||
- asr/ # 语音识别(Whisper)
|
||||
- scene/ # 场景检测
|
||||
- image/ # 图片处理
|
||||
- setup/ # 初始化服务
|
||||
dao/ # 数据访问层
|
||||
- audio/ # ASR 任务数据访问
|
||||
- video/ # 视频分析任务数据访问
|
||||
- image/
|
||||
model/ # 数据模型
|
||||
- dto/ # 传输对象(请求/响应)
|
||||
- video/ # 视频相关 DTO(包括视频分析)
|
||||
- entity/ # 数据库实体
|
||||
- video/ # 视频相关实体(包括分析任务)
|
||||
resource/ # 静态资源(日志、临时文件)
|
||||
sql/ # 数据库 SQL
|
||||
- video_analysis_task.sql - 视频分析任务表建表SQL
|
||||
```
|
||||
|
||||
### 核心功能
|
||||
|
||||
| 功能 | 说明 | 依赖 |
|
||||
|------|------|------|
|
||||
| 视频拼接 | 支持多个视频拼接,提供 fast(无损 concat demuxer)和 reencode(重编码归一化)两种模式,可上传结果到 MinIO,支持同步和异步任务 | FFmpeg |
|
||||
| 视频分镜剪切 | 根据分镜时间片段列表剪切视频并重新拼接输出,支持同步和异步任务 | FFmpeg |
|
||||
| 音频提取 | 从视频文件中提取音频,支持 mp3/aac/wav/ogg/flac 多种格式 | FFmpeg |
|
||||
| 语音识别 | 异步语音转文字任务,基于 OpenAI Whisper,支持 whisper.cpp 加速 | FFmpeg + Whisper/whisper.cpp |
|
||||
| 场景检测 | 视频场景切分检测,提取关键帧,输出场景信息 | FFmpeg + ffprobe |
|
||||
| 视频分析 | 基于 Marlin-2B Video VLM 大模型进行视频理解,自动生成场景描述、事件切分和向量化,存入 RAG 系统 | FFmpeg + 外部 Marlin-2B VLM 服务 |
|
||||
|
||||
### 启动初始化
|
||||
|
||||
- `setup` 包在 `init()` 阶段自动执行,启动时会检查 FFmpeg 和 Whisper 依赖是否可用
|
||||
- 自动检测 whisper-cpp > whisper > python -m whisper 三个优先级
|
||||
- 如果依赖缺失会输出警告提示安装
|
||||
|
||||
### API 端点
|
||||
|
||||
**视频拼接:**
|
||||
- `POST /video/concat` - 视频拼接(URL 输入,同步)
|
||||
- `POST /video/concat/async` - 视频拼接(URL 输入,异步)
|
||||
- `POST /video/concat/upload` - 视频拼接(文件上传,同步)
|
||||
- `POST /video/concat/upload/async` - 视频拼接(文件上传,异步)
|
||||
- `GET /video/concat/task/{taskId}` - 查询异步拼接任务结果
|
||||
|
||||
**视频分镜剪切:**
|
||||
- `POST /video/cut` - 视频分镜剪切(URL 输入,同步)
|
||||
- `POST /video/cut/async` - 视频分镜剪切(URL 输入,异步)
|
||||
- `GET /video/cut/task/{taskId}` - 查询异步剪切任务结果
|
||||
|
||||
**语音识别:**
|
||||
- `POST /audio/transcribe` - 创建语音转文字异步任务
|
||||
- `GET /audio/task/{taskId}` - 获取转写任务详情
|
||||
- `GET /audio/task/{taskId}/progress` - 获取任务进度
|
||||
- `GET /audio/tasks` - 获取任务列表
|
||||
|
||||
**视频分析(规划中):**
|
||||
- `POST /video/analysis` - 创建视频分析异步任务(基于 Marlin-2B VLM)
|
||||
- `GET /video/analysis/task/{taskId}` - 查询分析任务结果
|
||||
- `GET /video/analysis/task/{taskId}/progress` - 查询分析任务进度
|
||||
- `POST /video/analysis/retry/{taskId}` - 重试失败的分析事件
|
||||
|
||||
> **Note**: `scene` 场景检测和 `image` 图片处理服务目录已创建,但 HTTP 端点尚未实现暴露。音频提取服务已实现但尚未暴露。
|
||||
|
||||
### 依赖外部服务
|
||||
|
||||
- PostgreSQL - 数据存储
|
||||
- Redis - 缓存
|
||||
- Consul - 服务发现
|
||||
- Jaeger - 链路追踪
|
||||
- OSS/MinIO - 文件存储(通过内部 oss 微服务上传)
|
||||
- FFmpeg - 多媒体处理
|
||||
- Whisper - 语音识别
|
||||
- Marlin-2B VLM 服务 - 视频理解大模型(提供字幕生成、事件定位功能)
|
||||
|
||||
### 内部依赖
|
||||
|
||||
项目依赖内部私有公共包 `gitea.com/red-future/common`,包含 HTTP 路由注册、用户信息解析、Consul、Jaeger 等基础设施封装。Docker 构建过程中已配置访问凭证。
|
||||
|
||||
### Docker 镜像
|
||||
|
||||
多阶段构建镜像包含:
|
||||
- 编译后的二进制
|
||||
- FFmpeg 运行时
|
||||
- Python 3 + openai-whisper(Python 版本,用于语音识别)
|
||||
- 非 root 用户 `appuser` 运行
|
||||
- 暴露端口 `3010`
|
||||
|
||||
### 架构设计
|
||||
|
||||
**分层架构:**
|
||||
- `controller` - HTTP 入口,参数解析,调用 Service,返回响应
|
||||
- `service` - 业务逻辑实现,每个功能领域一个子包
|
||||
- `dao` - 数据访问层,数据库操作
|
||||
- `model` - 数据模型,`dto` 存放请求/响应传输对象,`entity` 存放数据库实体
|
||||
|
||||
**设计模式:**
|
||||
- 使用 GoFrame 框架的依赖注入模式
|
||||
- 所有 Service 和 Controller 都使用**单例模式**(`var Xxx = new(XxxStruct)`)
|
||||
- 遵循标准的 Go 命名约定
|
||||
- 临时文件处理完需要**及时清理**(使用 `defer os.Remove()`)
|
||||
|
||||
**异步任务处理:**
|
||||
- 长时任务(视频拼接、视频剪切、语音识别)都支持**异步执行**
|
||||
- 同步模式直接等待结果返回,异步模式创建任务后立即返回任务 ID
|
||||
- 异步任务状态持久化到数据库,可通过任务 ID 查询进度和结果
|
||||
- 支持回调 URL,任务完成后会回调通知调用方
|
||||
- 任务执行使用 goroutine 异步处理
|
||||
|
||||
**用户身份:**
|
||||
- 所有接口优先从请求头 `Authorization` / `X-User-Info` 解析用户信息
|
||||
- 解析失败使用默认 `admin` / tenantId=1 用于开发和调试
|
||||
|
||||
### 配置
|
||||
|
||||
主要配置在 `config.yml`:
|
||||
|
||||
**Server:**
|
||||
- `server.address` - 监听地址(默认 `:3010`)
|
||||
- `server.clientMaxBodySize` - 上传文件大小限制(默认 `200MB`)
|
||||
|
||||
**限流:**
|
||||
- `rate.limit` - 每秒请求限制(默认 200)
|
||||
- `rate.burst` - 突发请求允许量(默认 300)
|
||||
|
||||
**FFmpeg:**
|
||||
- `ffmpeg.path` - FFmpeg 可执行文件路径,留空则从 PATH 自动查找
|
||||
- `ffmpeg.temp_dir` - 临时文件目录(存放上传的视频和处理输出)
|
||||
|
||||
**Whisper 语音识别:**
|
||||
- `whisper.path` - Whisper 可执行文件路径,留空自动查找
|
||||
- `whisper.model` - 默认模型(tiny(最快)/base/small/medium)
|
||||
- `whisper.language` - 默认语言(zh=中文, en=英文)
|
||||
- `whisper.model_dir` - 模型缓存目录,留空使用默认 (~/.cache/whisper/)
|
||||
- `whisper.threads` - CPU 线程数(限制资源占用,建议 2-4)
|
||||
|
||||
**视频分析:**
|
||||
- `analysis.concurrency` - 并发处理数,控制同时处理的分析任务数量(默认 1,串行处理,建议不超过 CPU 核心数)
|
||||
- `analysis.maxRetries` - 单事件最大重试次数,失败时自动重试(默认 3)
|
||||
|
||||
**外部服务:**
|
||||
- `database` - PostgreSQL 数据库配置
|
||||
- `redis` - Redis 配置
|
||||
- `consul` - Consul 服务发现配置
|
||||
- `jaeger` - Jaeger 链路追踪配置
|
||||
- `filePrefix` - OSS/MinIO 文件访问地址前缀
|
||||
13
config.yml
13
config.yml
@@ -59,6 +59,19 @@ ffmpeg:
|
||||
# 临时文件目录(上传的视频和提取的音频)
|
||||
temp_dir: "resource/temp"
|
||||
|
||||
# 视频分析配置
|
||||
analysis:
|
||||
# 视频永久存储目录(按 taskId 子目录组织,不删除)
|
||||
video_dir: "resource/videos"
|
||||
# Caption 接口地址
|
||||
caption_url: "http://192.168.3.49:8900/caption"
|
||||
# Caption 接口超时时间(单个视频分析可能耗时较长)
|
||||
caption_timeout: "30m"
|
||||
# Max new tokens(传递给 Caption 接口的参数)
|
||||
max_new_tokens: "2048"
|
||||
# 是否启用 mock 模式(true: 返回模拟数据,false: 调用真实 caption 接口)
|
||||
mock_caption: true
|
||||
|
||||
# OSS/MinIO 文件上传配置
|
||||
filePrefix: "http://116.204.74.41:9000"
|
||||
|
||||
|
||||
38
controller/video/analysis_controller.go
Normal file
38
controller/video/analysis_controller.go
Normal file
@@ -0,0 +1,38 @@
|
||||
package video
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
dto "media/model/dto/video"
|
||||
service "media/service/video"
|
||||
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
)
|
||||
|
||||
type analysis struct{}
|
||||
|
||||
var Analysis = new(analysis)
|
||||
|
||||
// Analysis 提交视频分析任务 POST /video/analysis
|
||||
func (c *analysis) Analysis(ctx context.Context, req *dto.AnalysisReq) (res *dto.CreateAnalysisTaskRes, err error) {
|
||||
ctx = withUser(ctx)
|
||||
g.Log().Infof(ctx, "[视频分析] 收到请求 入参: total_videos=%d, callback=%s", len(req.VideoURLs), req.CallbackURL)
|
||||
|
||||
if len(req.VideoURLs) == 0 {
|
||||
return nil, fmt.Errorf("视频URL列表不能为空")
|
||||
}
|
||||
|
||||
taskID, taskErr := service.Analysis.CreateAsyncTask(ctx, req.VideoURLs, req.CallbackURL)
|
||||
if taskErr != nil {
|
||||
return nil, taskErr
|
||||
}
|
||||
|
||||
return &dto.CreateAnalysisTaskRes{TaskID: taskID}, nil
|
||||
}
|
||||
|
||||
// GetAnalysisTask 查询视频分析任务结果 GET /video/analysis/task/{taskId}
|
||||
func (c *analysis) GetAnalysisTask(ctx context.Context, req *dto.GetAnalysisTaskReq) (res *dto.GetAnalysisTaskRes, err error) {
|
||||
ctx = withUser(ctx)
|
||||
return service.Analysis.GetTaskResult(ctx, req.TaskID)
|
||||
}
|
||||
136
dao/video/analysis_task_dao.go
Normal file
136
dao/video/analysis_task_dao.go
Normal file
@@ -0,0 +1,136 @@
|
||||
package video
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
dto "media/model/dto/video"
|
||||
entity "media/model/entity/video"
|
||||
|
||||
"gitea.com/red-future/common/db/gfdb"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
var AnalysisTask = new(analysisTaskDao)
|
||||
|
||||
type analysisTaskDao struct{}
|
||||
|
||||
const analysisTaskTable = "video_analysis_task"
|
||||
|
||||
// Insert 创建任务(排除 id 字段,让数据库自增)
|
||||
func (d *analysisTaskDao) Insert(ctx context.Context, data *entity.AnalysisTask) (id int64, err error) {
|
||||
r, err := gfdb.DB(ctx).Model(ctx, analysisTaskTable).
|
||||
Data(data).
|
||||
FieldsEx(entity.AnalysisTaskCols.Id).
|
||||
Insert()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return r.LastInsertId()
|
||||
}
|
||||
|
||||
// GetByTaskID 根据taskId查询任务
|
||||
func (d *analysisTaskDao) GetByTaskID(ctx context.Context, taskID string) (res *entity.AnalysisTask, err error) {
|
||||
r, err := gfdb.DB(ctx).Model(ctx, analysisTaskTable).
|
||||
Where(entity.AnalysisTaskCols.TaskID, taskID).
|
||||
One()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r == nil {
|
||||
return nil, nil
|
||||
}
|
||||
err = r.Struct(&res)
|
||||
return
|
||||
}
|
||||
|
||||
// UpdateProcessing 更新为处理中
|
||||
func (d *analysisTaskDao) UpdateProcessing(ctx context.Context, taskID string) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskCols.Status: "processing",
|
||||
}).
|
||||
Where(entity.AnalysisTaskCols.TaskID, taskID).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateProgress 更新视频处理计数
|
||||
func (d *analysisTaskDao) UpdateProgress(ctx context.Context, taskID string, successCount, failedCount int) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskCols.SuccessCount: successCount,
|
||||
entity.AnalysisTaskCols.FailedCount: failedCount,
|
||||
}).
|
||||
Where(entity.AnalysisTaskCols.TaskID, taskID).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateSuccess 更新为成功
|
||||
func (d *analysisTaskDao) UpdateSuccess(ctx context.Context, taskID string, successCount, failedCount int) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskCols.Status: "success",
|
||||
entity.AnalysisTaskCols.SuccessCount: successCount,
|
||||
entity.AnalysisTaskCols.FailedCount: failedCount,
|
||||
entity.AnalysisTaskCols.ErrorMessage: "",
|
||||
}).
|
||||
Where(entity.AnalysisTaskCols.TaskID, taskID).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateError 更新为失败
|
||||
func (d *analysisTaskDao) UpdateError(ctx context.Context, taskID string, errMsg string) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskCols.Status: "failed",
|
||||
entity.AnalysisTaskCols.ErrorMessage: errMsg,
|
||||
}).
|
||||
Where(entity.AnalysisTaskCols.TaskID, taskID).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
|
||||
// EntityToTaskRes 实体转DTO
|
||||
func AnalysisEntityToTaskRes(e *entity.AnalysisTask, details []*entity.AnalysisTaskDetail) *dto.GetAnalysisTaskRes {
|
||||
res := &dto.GetAnalysisTaskRes{
|
||||
TaskID: e.TaskID,
|
||||
Status: e.Status,
|
||||
Total: e.Total,
|
||||
SuccessCount: e.SuccessCount,
|
||||
FailedCount: e.FailedCount,
|
||||
Processed: e.SuccessCount + e.FailedCount,
|
||||
CreatedAt: gconv.Int64(e.CreatedAt.Timestamp()),
|
||||
}
|
||||
if e.CreatedAt == nil {
|
||||
res.CreatedAt = time.Now().UnixMilli()
|
||||
}
|
||||
if e.Status == "failed" {
|
||||
res.ErrorMessage = e.ErrorMessage
|
||||
}
|
||||
|
||||
// 转换详情列表
|
||||
for _, d := range details {
|
||||
item := dto.AnalysisDetailItem{
|
||||
VideoURL: d.VideoURL,
|
||||
VideoSavePath: d.VideoSavePath,
|
||||
Status: d.Status,
|
||||
FailReason: d.FailReason,
|
||||
}
|
||||
// caption_result 如果非空,尝试解析为 interface{}
|
||||
if d.CaptionResult != "" {
|
||||
var captionResult interface{}
|
||||
if err := gconv.Scan(d.CaptionResult, &captionResult); err == nil {
|
||||
item.CaptionResult = captionResult
|
||||
} else {
|
||||
item.CaptionResult = d.CaptionResult
|
||||
}
|
||||
}
|
||||
res.List = append(res.List, item)
|
||||
}
|
||||
|
||||
return res
|
||||
}
|
||||
99
dao/video/analysis_task_detail_dao.go
Normal file
99
dao/video/analysis_task_detail_dao.go
Normal file
@@ -0,0 +1,99 @@
|
||||
package video
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
entity "media/model/entity/video"
|
||||
|
||||
"gitea.com/red-future/common/db/gfdb"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
)
|
||||
|
||||
var AnalysisTaskDetail = new(analysisTaskDetailDao)
|
||||
|
||||
type analysisTaskDetailDao struct{}
|
||||
|
||||
const analysisTaskDetailTable = "video_analysis_task_detail"
|
||||
|
||||
// Insert 创建明细
|
||||
func (d *analysisTaskDetailDao) Insert(ctx context.Context, data *entity.AnalysisTaskDetail) (id int64, err error) {
|
||||
r, err := gfdb.DB(ctx).Model(ctx, analysisTaskDetailTable).
|
||||
Data(data).
|
||||
FieldsEx(entity.AnalysisTaskDetailCols.Id).
|
||||
Insert()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return r.LastInsertId()
|
||||
}
|
||||
|
||||
// BatchInsert 批量创建明细
|
||||
func (d *analysisTaskDetailDao) BatchInsert(ctx context.Context, taskID string, videoURLs []string) error {
|
||||
for _, videoURL := range videoURLs {
|
||||
detail := &entity.AnalysisTaskDetail{
|
||||
TaskID: taskID,
|
||||
VideoURL: videoURL,
|
||||
Status: "pending",
|
||||
}
|
||||
if _, err := d.Insert(ctx, detail); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetByTaskID 根据taskId查询所有明细
|
||||
func (d *analysisTaskDetailDao) GetByTaskID(ctx context.Context, taskID string) (res []*entity.AnalysisTaskDetail, err error) {
|
||||
r, err := gfdb.DB(ctx).Model(ctx, analysisTaskDetailTable).
|
||||
Where(entity.AnalysisTaskDetailCols.TaskID, taskID).
|
||||
Order("id asc").
|
||||
All()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r == nil {
|
||||
return nil, nil
|
||||
}
|
||||
err = r.Structs(&res)
|
||||
return
|
||||
}
|
||||
|
||||
// UpdateSuccess 更新明细为成功
|
||||
func (d *analysisTaskDetailDao) UpdateSuccess(ctx context.Context, taskID string, videoURL string, videoSavePath string, captionResult string) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskDetailTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskDetailCols.Status: "success",
|
||||
entity.AnalysisTaskDetailCols.VideoSavePath: videoSavePath,
|
||||
entity.AnalysisTaskDetailCols.CaptionResult: captionResult,
|
||||
entity.AnalysisTaskDetailCols.FailReason: "",
|
||||
}).
|
||||
Where(entity.AnalysisTaskDetailCols.TaskID, taskID).
|
||||
Where(entity.AnalysisTaskDetailCols.VideoURL, videoURL).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateError 更新明细为失败
|
||||
func (d *analysisTaskDetailDao) UpdateError(ctx context.Context, taskID string, videoURL string, failReason string) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskDetailTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskDetailCols.Status: "failed",
|
||||
entity.AnalysisTaskDetailCols.FailReason: failReason,
|
||||
}).
|
||||
Where(entity.AnalysisTaskDetailCols.TaskID, taskID).
|
||||
Where(entity.AnalysisTaskDetailCols.VideoURL, videoURL).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateVideoSavePath 更新视频保存路径
|
||||
func (d *analysisTaskDetailDao) UpdateVideoSavePath(ctx context.Context, taskID string, videoURL string, savePath string) error {
|
||||
_, err := gfdb.DB(ctx).Model(ctx, analysisTaskDetailTable).
|
||||
Data(g.Map{
|
||||
entity.AnalysisTaskDetailCols.VideoSavePath: savePath,
|
||||
}).
|
||||
Where(entity.AnalysisTaskDetailCols.TaskID, taskID).
|
||||
Where(entity.AnalysisTaskDetailCols.VideoURL, videoURL).
|
||||
Update()
|
||||
return err
|
||||
}
|
||||
1
main.go
1
main.go
@@ -19,6 +19,7 @@ func main() {
|
||||
controllerAudio.AudioExtract,
|
||||
controllerVideo.Concat,
|
||||
controllerVideo.Cut,
|
||||
controllerVideo.Analysis,
|
||||
})
|
||||
select {}
|
||||
}
|
||||
|
||||
47
model/dto/video/analysis_dto.go
Normal file
47
model/dto/video/analysis_dto.go
Normal file
@@ -0,0 +1,47 @@
|
||||
package video
|
||||
|
||||
import "github.com/gogf/gf/v2/frame/g"
|
||||
|
||||
// ---------- 提交视频分析任务 ----------
|
||||
|
||||
// AnalysisReq 提交视频分析任务请求
|
||||
type AnalysisReq struct {
|
||||
g.Meta `path:"/video/analysis" method:"post" tags:"视频分析" summary:"提交视频分析任务" dc:"接收视频链接数组,后台异步串行分析,立即返回taskId"`
|
||||
VideoURLs []string `json:"video_urls" v:"required#视频URL列表不能为空" dc:"视频URL列表"`
|
||||
CallbackURL string `json:"callback_url" v:"required#回调地址不能为空" dc:"回调地址,分析完成后POST结果到该地址"`
|
||||
}
|
||||
|
||||
// CreateAnalysisTaskRes 创建视频分析任务响应
|
||||
type CreateAnalysisTaskRes struct {
|
||||
TaskID string `json:"taskId" dc:"任务ID"`
|
||||
}
|
||||
|
||||
// ---------- 查询任务进度/结果 ----------
|
||||
|
||||
// GetAnalysisTaskReq 查询视频分析任务请求
|
||||
type GetAnalysisTaskReq struct {
|
||||
g.Meta `path:"/video/analysis/task/{taskId}" method:"get" tags:"视频分析" summary:"查询分析任务结果" dc:"根据taskId查询视频分析任务的进度和结果"`
|
||||
TaskID string `json:"taskId" dc:"任务ID"`
|
||||
}
|
||||
|
||||
// AnalysisDetailItem 单个视频分析结果项
|
||||
type AnalysisDetailItem struct {
|
||||
VideoURL string `json:"video_url" dc:"原始视频URL"`
|
||||
VideoSavePath string `json:"video_save_path" dc:"视频本地保存路径"`
|
||||
Status string `json:"status" dc:"状态: pending/success/failed"`
|
||||
CaptionResult interface{} `json:"caption_result,omitempty" dc:"Caption接口返回结果"`
|
||||
FailReason string `json:"fail_reason,omitempty" dc:"失败原因"`
|
||||
}
|
||||
|
||||
// GetAnalysisTaskRes 查询视频分析任务响应
|
||||
type GetAnalysisTaskRes struct {
|
||||
TaskID string `json:"taskId" dc:"任务ID"`
|
||||
Status string `json:"status" dc:"状态: pending/processing/success/failed"`
|
||||
Total int `json:"total" dc:"视频总数"`
|
||||
Processed int `json:"processed" dc:"已处理数"`
|
||||
SuccessCount int `json:"successCount" dc:"成功数"`
|
||||
FailedCount int `json:"failedCount" dc:"失败数"`
|
||||
List []AnalysisDetailItem `json:"list" dc:"视频详情列表"`
|
||||
ErrorMessage string `json:"errorMessage,omitempty" dc:"错误信息"`
|
||||
CreatedAt int64 `json:"createdAt" dc:"创建时间戳"`
|
||||
}
|
||||
39
model/entity/video/analysis_task.go
Normal file
39
model/entity/video/analysis_task.go
Normal file
@@ -0,0 +1,39 @@
|
||||
package video
|
||||
|
||||
import "gitea.com/red-future/common/beans"
|
||||
|
||||
// AnalysisTask 视频分析异步任务实体
|
||||
type AnalysisTask struct {
|
||||
beans.SQLBaseDO `orm:",inherit"`
|
||||
TaskID string `orm:"task_id" json:"taskId" description:"任务唯一标识"`
|
||||
CallbackURL string `orm:"callback_url" json:"callbackUrl" description:"回调地址"`
|
||||
Status string `orm:"status" json:"status" description:"任务状态:pending/processing/success/failed"`
|
||||
Total int `orm:"total" json:"total" description:"待分析视频总数"`
|
||||
SuccessCount int `orm:"success_count" json:"successCount" description:"成功数"`
|
||||
FailedCount int `orm:"failed_count" json:"failedCount" description:"失败数"`
|
||||
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息"`
|
||||
}
|
||||
|
||||
// AnalysisTaskCol 字段定义
|
||||
type AnalysisTaskCol struct {
|
||||
beans.SQLBaseCol
|
||||
TaskID string
|
||||
CallbackURL string
|
||||
Status string
|
||||
Total string
|
||||
SuccessCount string
|
||||
FailedCount string
|
||||
ErrorMessage string
|
||||
}
|
||||
|
||||
// AnalysisTaskCols 字段常量
|
||||
var AnalysisTaskCols = AnalysisTaskCol{
|
||||
SQLBaseCol: beans.DefSQLBaseCol,
|
||||
TaskID: "task_id",
|
||||
CallbackURL: "callback_url",
|
||||
Status: "status",
|
||||
Total: "total",
|
||||
SuccessCount: "success_count",
|
||||
FailedCount: "failed_count",
|
||||
ErrorMessage: "error_message",
|
||||
}
|
||||
36
model/entity/video/analysis_task_detail.go
Normal file
36
model/entity/video/analysis_task_detail.go
Normal file
@@ -0,0 +1,36 @@
|
||||
package video
|
||||
|
||||
import "gitea.com/red-future/common/beans"
|
||||
|
||||
// AnalysisTaskDetail 视频分析任务明细实体
|
||||
type AnalysisTaskDetail struct {
|
||||
beans.SQLBaseDO `orm:",inherit"`
|
||||
TaskID string `orm:"task_id" json:"taskId" description:"所属任务ID"`
|
||||
VideoURL string `orm:"video_url" json:"videoUrl" description:"原始视频URL"`
|
||||
VideoSavePath string `orm:"video_save_path" json:"videoSavePath" description:"视频本地保存路径(永久保留)"`
|
||||
Status string `orm:"status" json:"status" description:"状态:pending/success/failed"`
|
||||
CaptionResult string `orm:"caption_result" json:"captionResult" description:"Caption接口返回结果JSON"`
|
||||
FailReason string `orm:"fail_reason" json:"failReason" description:"失败原因"`
|
||||
}
|
||||
|
||||
// AnalysisTaskDetailCol 字段定义
|
||||
type AnalysisTaskDetailCol struct {
|
||||
beans.SQLBaseCol
|
||||
TaskID string
|
||||
VideoURL string
|
||||
VideoSavePath string
|
||||
Status string
|
||||
CaptionResult string
|
||||
FailReason string
|
||||
}
|
||||
|
||||
// AnalysisTaskDetailCols 字段常量
|
||||
var AnalysisTaskDetailCols = AnalysisTaskDetailCol{
|
||||
SQLBaseCol: beans.DefSQLBaseCol,
|
||||
TaskID: "task_id",
|
||||
VideoURL: "video_url",
|
||||
VideoSavePath: "video_save_path",
|
||||
Status: "status",
|
||||
CaptionResult: "caption_result",
|
||||
FailReason: "fail_reason",
|
||||
}
|
||||
382
service/video/analysis_service.go
Normal file
382
service/video/analysis_service.go
Normal file
@@ -0,0 +1,382 @@
|
||||
package video
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
analysisDao "media/dao/video"
|
||||
dto "media/model/dto/video"
|
||||
entity "media/model/entity/video"
|
||||
|
||||
"gitea.com/red-future/common/beans"
|
||||
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/util/guid"
|
||||
)
|
||||
|
||||
type analysisService struct{}
|
||||
|
||||
// Analysis 视频分析服务单例
|
||||
var Analysis = new(analysisService)
|
||||
|
||||
// ---------- 异步任务管理 ----------
|
||||
|
||||
// CreateAsyncTask 创建异步分析任务,返回 taskId,后台串行处理
|
||||
func (s *analysisService) CreateAsyncTask(ctx context.Context, videoURLs []string, callbackURL string) (string, error) {
|
||||
if len(videoURLs) == 0 {
|
||||
return "", fmt.Errorf("视频URL列表不能为空")
|
||||
}
|
||||
|
||||
taskID := "anl_" + guid.S()
|
||||
task := &entity.AnalysisTask{
|
||||
TaskID: taskID,
|
||||
CallbackURL: callbackURL,
|
||||
Status: "pending",
|
||||
Total: len(videoURLs),
|
||||
}
|
||||
if _, err := analysisDao.AnalysisTask.Insert(ctx, task); err != nil {
|
||||
return "", fmt.Errorf("创建任务失败: %v", err)
|
||||
}
|
||||
|
||||
// 批量创建明细
|
||||
if err := analysisDao.AnalysisTaskDetail.BatchInsert(ctx, taskID, videoURLs); err != nil {
|
||||
return "", fmt.Errorf("创建任务明细失败: %v", err)
|
||||
}
|
||||
|
||||
// 提取调用方用户信息,传给 goroutine
|
||||
user := getUserFromCtx(ctx)
|
||||
|
||||
g.Log().Infof(ctx, "[视频分析] 创建任务 %s, 视频数=%d, 回调=%s", taskID, len(videoURLs), callbackURL)
|
||||
|
||||
// 异步处理
|
||||
go s.processAsyncTask(user, taskID, videoURLs, callbackURL)
|
||||
|
||||
return taskID, nil
|
||||
}
|
||||
|
||||
// GetTaskResult 查询异步任务结果
|
||||
func (s *analysisService) GetTaskResult(ctx context.Context, taskID string) (*dto.GetAnalysisTaskRes, error) {
|
||||
task, err := analysisDao.AnalysisTask.GetByTaskID(ctx, taskID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("查询任务失败: %v", err)
|
||||
}
|
||||
if task == nil {
|
||||
return nil, fmt.Errorf("任务不存在: %s", taskID)
|
||||
}
|
||||
|
||||
details, err := analysisDao.AnalysisTaskDetail.GetByTaskID(ctx, taskID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("查询任务明细失败: %v", err)
|
||||
}
|
||||
|
||||
return analysisDao.AnalysisEntityToTaskRes(task, details), nil
|
||||
}
|
||||
|
||||
// processAsyncTask 后台串行处理异步分析任务
|
||||
func (s *analysisService) processAsyncTask(user *beans.User, taskID string, videoURLs []string, callbackURL string) {
|
||||
bgCtx := context.Background()
|
||||
bgCtx = context.WithValue(bgCtx, "user", user)
|
||||
|
||||
analysisDao.AnalysisTask.UpdateProcessing(bgCtx, taskID)
|
||||
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
errMsg := fmt.Sprintf("视频分析异常: %v", r)
|
||||
g.Log().Errorf(bgCtx, "[视频分析 %s] %s", taskID, errMsg)
|
||||
analysisDao.AnalysisTask.UpdateError(bgCtx, taskID, errMsg)
|
||||
s.analysisCallback(bgCtx, taskID, callbackURL)
|
||||
}
|
||||
}()
|
||||
|
||||
successCount := 0
|
||||
failedCount := 0
|
||||
|
||||
// 逐个串行处理视频
|
||||
for _, videoURL := range videoURLs {
|
||||
g.Log().Infof(bgCtx, "[视频分析 %s] 开始处理视频: %s", taskID, videoURL)
|
||||
|
||||
// 1. 下载视频到永久存储目录
|
||||
savePath, dlErr := s.downloadVideo(bgCtx, taskID, videoURL)
|
||||
if dlErr != nil {
|
||||
errMsg := fmt.Sprintf("下载视频失败 %s: %v", videoURL, dlErr)
|
||||
g.Log().Errorf(bgCtx, "[视频分析 %s] %s", taskID, errMsg)
|
||||
analysisDao.AnalysisTaskDetail.UpdateError(bgCtx, taskID, videoURL, errMsg)
|
||||
failedCount++
|
||||
analysisDao.AnalysisTask.UpdateProgress(bgCtx, taskID, successCount, failedCount)
|
||||
continue
|
||||
}
|
||||
|
||||
g.Log().Infof(bgCtx, "[视频分析 %s] 视频下载完成: %s -> %s", taskID, videoURL, savePath)
|
||||
|
||||
// 2. 调用第三方 caption 接口(一次一个视频,form-data 上传)
|
||||
captionResult, cpErr := s.callCaptionAPI(bgCtx, savePath)
|
||||
if cpErr != nil {
|
||||
errMsg := fmt.Sprintf("调用Caption接口失败 %s: %v", videoURL, cpErr)
|
||||
g.Log().Errorf(bgCtx, "[视频分析 %s] %s", taskID, errMsg)
|
||||
analysisDao.AnalysisTaskDetail.UpdateError(bgCtx, taskID, videoURL, errMsg)
|
||||
failedCount++
|
||||
analysisDao.AnalysisTask.UpdateProgress(bgCtx, taskID, successCount, failedCount)
|
||||
continue
|
||||
}
|
||||
|
||||
g.Log().Infof(bgCtx, "[视频分析 %s] Caption接口调用成功: %s", taskID, videoURL)
|
||||
|
||||
// 3. 保存结果到数据库(视频不删除,永久保留)
|
||||
captionJSON, _ := json.Marshal(captionResult)
|
||||
analysisDao.AnalysisTaskDetail.UpdateSuccess(bgCtx, taskID, videoURL, savePath, string(captionJSON))
|
||||
successCount++
|
||||
analysisDao.AnalysisTask.UpdateProgress(bgCtx, taskID, successCount, failedCount)
|
||||
|
||||
g.Log().Infof(bgCtx, "[视频分析 %s] 视频处理完成: %s (成功=%d, 失败=%d)", taskID, videoURL, successCount, failedCount)
|
||||
}
|
||||
|
||||
// 更新任务最终状态
|
||||
if failedCount == 0 {
|
||||
analysisDao.AnalysisTask.UpdateSuccess(bgCtx, taskID, successCount, failedCount)
|
||||
} else if successCount == 0 {
|
||||
analysisDao.AnalysisTask.UpdateError(bgCtx, taskID, fmt.Sprintf("所有视频处理失败(共%d个)", len(videoURLs)))
|
||||
} else {
|
||||
analysisDao.AnalysisTask.UpdateSuccess(bgCtx, taskID, successCount, failedCount)
|
||||
}
|
||||
|
||||
g.Log().Infof(bgCtx, "[视频分析 %s] 任务完成, 总视频=%d, 成功=%d, 失败=%d", taskID, len(videoURLs), successCount, failedCount)
|
||||
|
||||
// 回调通知
|
||||
if callbackURL != "" {
|
||||
s.analysisCallback(bgCtx, taskID, callbackURL)
|
||||
}
|
||||
}
|
||||
|
||||
// downloadVideo 下载视频到永久存储目录
|
||||
func (s *analysisService) downloadVideo(ctx context.Context, taskID string, videoURL string) (string, error) {
|
||||
// 从配置获取视频永久存储目录
|
||||
videoDir := g.Cfg().MustGet(ctx, "analysis.video_dir", "resource/videos").String()
|
||||
if !filepath.IsAbs(videoDir) {
|
||||
absDir, _ := filepath.Abs(videoDir)
|
||||
videoDir = absDir
|
||||
}
|
||||
// 按 taskId 子目录组织
|
||||
taskDir := filepath.Join(videoDir, taskID)
|
||||
os.MkdirAll(taskDir, 0755)
|
||||
|
||||
// 从URL提取文件名
|
||||
segments := strings.Split(videoURL, "/")
|
||||
fileName := segments[len(segments)-1]
|
||||
if fileName == "" {
|
||||
fileName = fmt.Sprintf("video_%d.mp4", time.Now().UnixMilli())
|
||||
}
|
||||
savePath := filepath.Join(taskDir, fileName)
|
||||
|
||||
client := &http.Client{Timeout: 10 * time.Minute}
|
||||
resp, err := client.Get(videoURL)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return "", fmt.Errorf("HTTP %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
out, err := os.Create(savePath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer out.Close()
|
||||
|
||||
_, err = io.Copy(out, resp.Body)
|
||||
if err != nil {
|
||||
os.Remove(savePath)
|
||||
return "", err
|
||||
}
|
||||
|
||||
return savePath, nil
|
||||
}
|
||||
|
||||
// callCaptionAPI 调用第三方 caption 接口(form-data 上传单个视频)
|
||||
// 根据配置 analysis.mock_caption 决定使用 mock 数据还是真实调用
|
||||
func (s *analysisService) callCaptionAPI(ctx context.Context, videoPath string) (map[string]interface{}, error) {
|
||||
if g.Cfg().MustGet(ctx, "analysis.mock_caption", true).Bool() {
|
||||
return s.mockCallCaptionAPI(ctx, videoPath)
|
||||
}
|
||||
return s.realCallCaptionAPI(ctx, videoPath)
|
||||
}
|
||||
|
||||
// mockCallCaptionAPI 返回 mock 的 caption 数据
|
||||
func (s *analysisService) mockCallCaptionAPI(ctx context.Context, videoPath string) (map[string]interface{}, error) {
|
||||
g.Log().Infof(ctx, "[呼叫Caption-Mock] 使用mock数据, 文件=%s", videoPath)
|
||||
|
||||
return map[string]interface{}{
|
||||
"caption": "Scene: 视频展示了一个产品演示和讲解过程。Events: 一个人走进画面开始介绍产品, 展示了产品的各个功能模块, 最后总结产品优势。",
|
||||
"scene": "这是一个产品演示和讲解的场景。视频中有人在画面中介绍一款产品,展示了产品的各个功能模块和使用方式,包括产品的外观、核心功能、操作界面等。视频整体节奏适中,配合专业的产品讲解。",
|
||||
"events": []map[string]interface{}{
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 5.0,
|
||||
"description": "视频开场,人物走进画面,开始打招呼并介绍本次演示的主题",
|
||||
},
|
||||
{
|
||||
"start": 5.5,
|
||||
"end": 15.0,
|
||||
"description": "展示产品外观包装,详细说明产品设计理念和特点",
|
||||
},
|
||||
{
|
||||
"start": 15.5,
|
||||
"end": 30.0,
|
||||
"description": "开机演示,展示产品主界面和核心功能入口",
|
||||
},
|
||||
{
|
||||
"start": 30.5,
|
||||
"end": 50.0,
|
||||
"description": "详细演示产品主要功能模块的操作流程和使用方法",
|
||||
},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
// realCallCaptionAPI 真实调用第三方 caption 接口(form-data 上传单个视频)
|
||||
func (s *analysisService) realCallCaptionAPI(ctx context.Context, videoPath string) (map[string]interface{}, error) {
|
||||
// 获取 caption 服务地址
|
||||
captionURL := g.Cfg().MustGet(ctx, "analysis.caption_url", "http://192.168.3.49:8900/caption").String()
|
||||
|
||||
// 构建 multipart/form-data 表单
|
||||
var buf bytes.Buffer
|
||||
mw := multipart.NewWriter(&buf)
|
||||
|
||||
// 添加视频文件字段
|
||||
file, err := os.Open(videoPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("打开视频文件失败: %v", err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
fw, err := mw.CreateFormFile("video", filepath.Base(videoPath))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建表单文件字段失败: %v", err)
|
||||
}
|
||||
if _, err = io.Copy(fw, file); err != nil {
|
||||
return nil, fmt.Errorf("写入文件内容失败: %v", err)
|
||||
}
|
||||
|
||||
// 添加 max_new_tokens 参数
|
||||
if err = mw.WriteField("max_new_tokens", "2048"); err != nil {
|
||||
return nil, fmt.Errorf("写入表单字段失败: %v", err)
|
||||
}
|
||||
mw.Close()
|
||||
|
||||
// 发送请求(长超时,caption 接口耗时较长)
|
||||
client := &http.Client{Timeout: 30 * time.Minute}
|
||||
req, err := http.NewRequest("POST", captionURL, &buf)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建请求失败: %v", err)
|
||||
}
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
|
||||
g.Log().Debugf(ctx, "[呼叫Caption] 请求URL=%s, 文件=%s", captionURL, videoPath)
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("请求Caption接口失败: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("读取Caption响应失败: %v", err)
|
||||
}
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("Caption接口返回非200: %d, body=%s", resp.StatusCode, string(body))
|
||||
}
|
||||
|
||||
// 解析响应 JSON
|
||||
var result map[string]interface{}
|
||||
if err = json.Unmarshal(body, &result); err != nil {
|
||||
return nil, fmt.Errorf("解析Caption响应JSON失败: %v, body=%s", err, string(body))
|
||||
}
|
||||
|
||||
g.Log().Debugf(ctx, "[呼叫Caption] 响应成功, 文件=%s, 结果长度=%d", videoPath, len(body))
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// analysisCallback 回调通知
|
||||
func (s *analysisService) analysisCallback(ctx context.Context, taskID, callbackURL string) {
|
||||
if callbackURL == "" {
|
||||
return
|
||||
}
|
||||
|
||||
task, err := analysisDao.AnalysisTask.GetByTaskID(ctx, taskID)
|
||||
if err != nil || task == nil {
|
||||
g.Log().Errorf(ctx, "[视频分析回调 %s] 查询任务失败: %v", taskID, err)
|
||||
return
|
||||
}
|
||||
|
||||
details, err := analysisDao.AnalysisTaskDetail.GetByTaskID(ctx, taskID)
|
||||
if err != nil {
|
||||
g.Log().Errorf(ctx, "[视频分析回调 %s] 查询明细失败: %v", taskID, err)
|
||||
return
|
||||
}
|
||||
|
||||
// 构造回调结果
|
||||
var results []map[string]interface{}
|
||||
for _, d := range details {
|
||||
item := map[string]interface{}{
|
||||
"video_url": d.VideoURL,
|
||||
"video_save_path": d.VideoSavePath,
|
||||
}
|
||||
if d.Status == "success" && d.CaptionResult != "" {
|
||||
var captionResult interface{}
|
||||
json.Unmarshal([]byte(d.CaptionResult), &captionResult)
|
||||
item["caption_result"] = captionResult
|
||||
}
|
||||
if d.Status == "failed" {
|
||||
item["caption_result"] = nil
|
||||
item["fail_reason"] = d.FailReason
|
||||
}
|
||||
results = append(results, item)
|
||||
}
|
||||
|
||||
payload := map[string]interface{}{
|
||||
"task_id": task.TaskID,
|
||||
"status": task.Status,
|
||||
"total": task.Total,
|
||||
"success_count": task.SuccessCount,
|
||||
"failed_count": task.FailedCount,
|
||||
"results": results,
|
||||
}
|
||||
|
||||
body, _ := json.Marshal(payload)
|
||||
g.Log().Infof(ctx, "[视频分析回调 %s] 状态=%s, 目标=%s", taskID, task.Status, callbackURL)
|
||||
|
||||
cbReq, _ := http.NewRequest("POST", callbackURL, bytes.NewReader(body))
|
||||
cbReq.Header.Set("Content-Type", "application/json")
|
||||
// 透传调用方用户信息
|
||||
cbUser := getUserFromCtx(ctx)
|
||||
userJSON, _ := json.Marshal(cbUser)
|
||||
cbReq.Header.Set("X-User-Info", string(userJSON))
|
||||
|
||||
// 打印 curl 命令方便调试
|
||||
escapedBody := strings.ReplaceAll(string(body), "'", "'\\''")
|
||||
g.Log().Infof(ctx, "[视频分析回调 %s] curl 调试命令:\ncurl -X POST '%s' \\\n -H 'Content-Type: application/json' \\\n -H 'X-User-Info: %s' \\\n -d '%s'",
|
||||
taskID, callbackURL, string(userJSON), escapedBody)
|
||||
|
||||
client := &http.Client{Timeout: 2 * time.Minute}
|
||||
resp, reqErr := client.Do(cbReq)
|
||||
if reqErr != nil {
|
||||
g.Log().Errorf(ctx, "[视频分析回调 %s] 请求失败: %v", taskID, reqErr)
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
respBody, _ := io.ReadAll(resp.Body)
|
||||
g.Log().Infof(ctx, "[视频分析回调 %s] 响应 status=%d, body=%s", taskID, resp.StatusCode, string(respBody))
|
||||
}
|
||||
60
sql/video_analysis_task.sql
Normal file
60
sql/video_analysis_task.sql
Normal file
@@ -0,0 +1,60 @@
|
||||
-- video_analysis_task 视频分析异步任务主表
|
||||
CREATE TABLE IF NOT EXISTS video_analysis_task (
|
||||
id BIGSERIAL NOT NULL,
|
||||
tenant_id BIGINT NOT NULL DEFAULT 0,
|
||||
task_id VARCHAR(64) NOT NULL,
|
||||
callback_url VARCHAR(500) NOT NULL DEFAULT '',
|
||||
status VARCHAR(20) NOT NULL DEFAULT 'pending',
|
||||
total INT NOT NULL DEFAULT 0,
|
||||
success_count INT NOT NULL DEFAULT 0,
|
||||
failed_count INT NOT NULL DEFAULT 0,
|
||||
error_message TEXT,
|
||||
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
deleted_at TIMESTAMP WITH TIME ZONE,
|
||||
PRIMARY KEY (id)
|
||||
);
|
||||
|
||||
COMMENT ON TABLE video_analysis_task IS '视频分析异步任务表';
|
||||
COMMENT ON COLUMN video_analysis_task.task_id IS '任务唯一标识';
|
||||
COMMENT ON COLUMN video_analysis_task.callback_url IS '回调地址';
|
||||
COMMENT ON COLUMN video_analysis_task.status IS '任务状态:pending/processing/success/failed';
|
||||
COMMENT ON COLUMN video_analysis_task.total IS '待分析视频总数';
|
||||
COMMENT ON COLUMN video_analysis_task.success_count IS '成功数';
|
||||
COMMENT ON COLUMN video_analysis_task.failed_count IS '失败数';
|
||||
COMMENT ON COLUMN video_analysis_task.error_message IS '错误信息';
|
||||
COMMENT ON COLUMN video_analysis_task.created_at IS '创建时间';
|
||||
COMMENT ON COLUMN video_analysis_task.updated_at IS '更新时间';
|
||||
COMMENT ON COLUMN video_analysis_task.deleted_at IS '删除时间(软删除)';
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_video_analysis_task_task_id ON video_analysis_task(task_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_video_analysis_task_status ON video_analysis_task(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_video_analysis_task_created_at ON video_analysis_task(created_at);
|
||||
|
||||
-- video_analysis_task_detail 视频分析任务明细表(每视频一条)
|
||||
CREATE TABLE IF NOT EXISTS video_analysis_task_detail (
|
||||
id BIGSERIAL NOT NULL,
|
||||
task_id VARCHAR(64) NOT NULL,
|
||||
tenant_id BIGINT NOT NULL DEFAULT 0,
|
||||
video_url TEXT NOT NULL,
|
||||
video_save_path TEXT NOT NULL DEFAULT '',
|
||||
status VARCHAR(20) NOT NULL DEFAULT 'pending',
|
||||
caption_result TEXT,
|
||||
fail_reason TEXT,
|
||||
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
PRIMARY KEY (id)
|
||||
);
|
||||
|
||||
COMMENT ON TABLE video_analysis_task_detail IS '视频分析任务明细表';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.task_id IS '所属任务ID';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.tenant_id IS '租户ID';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.video_url IS '原始视频URL';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.video_save_path IS '视频本地保存路径(永久保留)';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.status IS '状态:pending/success/failed';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.caption_result IS 'Caption接口返回结果JSON';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.fail_reason IS '失败原因';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.created_at IS '创建时间';
|
||||
COMMENT ON COLUMN video_analysis_task_detail.updated_at IS '更新时间';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_video_analysis_task_detail_task_id ON video_analysis_task_detail(task_id);
|
||||
Reference in New Issue
Block a user