CANN计算机视觉算子库ops-cv的图像处理与特征提取优化实践

前言

计算机视觉是人工智能应用最广泛的领域之一，涵盖图像分类、目标检测、语义分割、视频分析等多种任务。这些任务的基础是高效的图像处理和特征提取算子。CANN的ops-cv库提供了丰富的计算机视觉算子，针对NPU硬件进行了深度优化，能够显著提升视觉应用的性能。

相关链接：

• CANN组织链接：https://atomgit.com/cann
• ops-cv仓库链接：https://atomgit.com/cann/ops-cv

---

一、ops-cv库概述

1.1 设计理念

ops-cv（Computer Vision Operators）是CANN框架中专门用于计算机视觉任务的算子库，其设计理念包括：

• 硬件感知优化：针对NPU的向量计算单元和内存层次结构优化
• 算子融合支持：支持多算子融合减少内存访问
• 多格式支持：支持RGB、BGR、GRAY、YUV等多种图像格式
• 精度灵活：支持FP32、FP16、INT8等多种精度计算

1.2 核心算子分类

ops-cv/
├── 图像预处理算子
│   ├── 图像缩放 (Resize)
│   ├── 图像裁剪 (Crop)
│   ├── 图像翻转 (Flip)
│   ├── 图像旋转 (Rotate)
│   └── 填充 (Pad)
├── 颜色空间转换
│   ├── RGB2BGR
│   ├── RGB2GRAY
│   ├── RGB2YUV
│   └── YUV2RGB
├── 特征提取算子
│   ├── 卷积 (Conv2D)
│   ├── 池化 (Pool2D)
│   ├── 激活 (Activation)
│   └── 归一化 (Normalize)
├── 图像增强算子
│   ├── 亮度调整
│   ├── 对比度调整
│   ├── 饱和度调整
│   └── 直方图均衡化
└── 视频处理算子
    ├── 光流计算
    ├── 帧差分
    └── 视频稳定

---

二、核心API详解

2.1 图像预处理API

图像缩放

图像缩放是计算机视觉中最常用的预处理操作，ops-cv提供了多种插值算法：

/**
 * @brief 图像缩放算子
 * @param input 输入图像张量 [N, C, H, W]
 * @param output 输出图像张量
 * @param output_size 目标尺寸 (height, width)
 * @param mode 插值模式: 0=最近邻, 1=双线性, 2=双三次, 3=面积插值
 * @param align_corners 是否对齐角点
 * @param workspace 工作空间指针
 * @param workspace_size 工作空间大小
 * @return 状态码
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_resize(
    const aclTensor* input,
    aclTensor* output,
    const int64_t output_size[2],
    ResizeMode mode,
    bool align_corners,
    void* workspace,
    uint64_t workspace_size);

// 插值模式枚举
enum ResizeMode {
    RESIZE_NEAREST_NEIGHBOR = 0,    // 最近邻插值 - 速度快
    RESIZE_BILINEAR = 1,            // 双线性插值 - 平衡
    RESIZE_BICUBIC = 2,             // 双三次插值 - 质量高
    RESIZE_AREA = 3                 // 面积插值 - 缩小图像效果好
};

图像裁剪

/**
 * @brief 图像裁剪算子
 * @param input 输入图像 [N, C, H, W]
 * @param output 输出图像
 * @param top 裁剪起始行
 * @param left 裁剪起始列
 * @param height 裁剪高度
 * @param width 裁剪宽度
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_crop(
    const aclTensor* input,
    aclTensor* output,
    int64_t top,
    int64_t left,
    int64_t height,
    int64_t width);

图像翻转

/**
 * @brief 图像翻转算子
 * @param input 输入图像
 * @param output 输出图像
 * @param flip_code 翻转模式
 *                >0: 水平翻转 (沿y轴)
 *                =0: 垂直翻转 (沿x轴)
 *                <0: 水平垂直同时翻转
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_flip(
    const aclTensor* input,
    aclTensor* output,
    int flip_code);

2.2 颜色空间转换API

/**
 * @brief 颜色空间转换算子
 * @param input 输入图像
 * @param output 输出图像
 * @param src_format 源颜色格式
 * @param dst_format 目标颜色格式
 * @param workspace 工作空间
 * @param workspace_size 工作空间大小
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_cvtColor(
    const aclTensor* input,
    aclTensor* output,
    ColorFormat src_format,
    ColorFormat dst_format,
    void* workspace,
    uint64_t workspace_size);

// 颜色格式枚举
enum ColorFormat {
    COLOR_RGB = 0,      // RGB格式
    COLOR_BGR = 1,      // BGR格式
    COLOR_GRAY = 2,     // 灰度格式
    COLOR_YUV = 3,      // YUV格式
    COLOR_HSV = 4,      // HSV格式
    COLOR_LAB = 5       // LAB格式
};

2.3 特征提取API

/**
 * @brief 特征金字塔算子 - 用于多尺度特征提取
 * @param input 输入特征图
 * @param outputs 输出特征金字塔数组
 * @param num_levels 金字塔层数
 * @param pyramid_mode 金字塔模式
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_feature_pyramid(
    const aclTensor* input,
    aclTensor** outputs,
    int num_levels,
    PyramidMode pyramid_mode);

// 金字塔模式
enum PyramidMode {
    PYRAMID_FPN = 0,    // Feature Pyramid Network
    PYRAMID_PAN = 1,    // Path Aggregation Network
    PYRAMID_BIFPN = 2   // Bidirectional FPN
};

/**
 * @brief 非极大值抑制 - 用于目标检测后处理
 * @param boxes 检测框 [N, 4] (x1, y1, x2, y2)
 * @param scores 置信度分数 [N]
 * @param output_boxes 输出检测框
 * @param output_scores 输出分数
 * @param iou_threshold IoU阈值
 * @param max_output_size 最大输出数量
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_nms(
    const aclTensor* boxes,
    const aclTensor* scores,
    aclTensor* output_boxes,
    aclTensor* output_scores,
    float iou_threshold,
    int max_output_size);

2.4 高级视觉API

/**
 * @brief RoI Align算子 - 用于实例分割
 * @param input 输入特征图
 * @param rois RoI坐标 [N, 5] (batch_index, x1, y1, x2, y2)
 * @param output 输出RoI特征
 * @param pooled_height 输出高度
 * @param pooled_width 输出宽度
 * @param spatial_scale 空间缩放因子
 * @param sampling_ratio 采样率
 * @param aligned 是否对齐
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_roi_align(
    const aclTensor* input,
    const aclTensor* rois,
    aclTensor* output,
    int pooled_height,
    int pooled_width,
    float spatial_scale,
    int sampling_ratio,
    bool aligned);

/**
 * @brief 光流计算算子 - 用于视频分析
 * @param images1 第一帧图像
 * @param images2 第二帧图像
 * @param flow 输出光流场 (u, v)
 * @param flow_method 光流计算方法
 */
ACL_FUNC_VISIBILITY aclError aclnn_cv_optical_flow(
    const aclTensor* images1,
    const aclTensor* images2,
    aclTensor* flow,
    OpticalFlowMethod flow_method);

// 光流计算方法
enum OpticalFlowMethod {
    FLOW_FARNEBACK = 0,     // Farneback算法
    FLOW_PYR_LK = 1,        // 金字塔Lucas-Kanade
    FLOW_RAFT = 2           // RAFT深度学习算法
};

---

三、应用实践

3.1 图像预处理流水线

以下是一个完整的图像预处理流水线示例，展示如何组合多个ops-cv算子：

#include "acl/acl.h"
#include "aclnnops/aclnn_cv.h"

class ImagePreprocessor {
public:
    ImagePreprocessor(int device_id) : device_id_(device_id) {
        // 初始化设备
        aclError ret = aclrtSetDevice(device_id_);
        CHECK_RET(ret == ACL_SUCCESS, "aclrtSetDevice failed");

        // 创建stream
        ret = aclrtCreateStream(&stream_);
        CHECK_RET(ret == ACL_SUCCESS, "aclrtCreateStream failed");
    }

    ~ImagePreprocessor() {
        aclrtDestroyStream(stream_);
        aclrtResetDevice(device_id_);
    }

    // 完整的预处理流水线
    void Process(const cv::Mat& input_image,
                 aclTensor** output_tensor,
                 int target_height = 224,
                 int target_width = 224) {
        // 1. 将OpenCV Mat转换为ACL张量
        aclTensor* input_tensor = MatToACLTensor(input_image);

        // 2. 颜色空间转换 BGR -> RGB
        aclTensor* rgb_tensor = nullptr;
        aclnn_cv_cvtColor(input_tensor, rgb_tensor,
                          COLOR_BGR, COLOR_RGB,
                          workspace_, workspace_size_);

        // 3. 调整图像大小
        int64_t output_size[2] = {target_height, target_width};
        aclTensor* resized_tensor = nullptr;
        aclnn_cv_resize(rgb_tensor, resized_tensor, output_size,
                       RESIZE_BILINEAR, true, workspace_, workspace_size_);

        // 4. 归一化
        aclTensor* normalized_tensor = Normalize(resized_tensor);

        // 5. 数据布局转换 HWC -> CHW
        aclTensor* chw_tensor = nullptr;
        aclnn_cv_transpose(normalized_tensor, chw_tensor,
                          {0, 3, 1, 2}, workspace_, workspace_size_);

        *output_tensor = chw_tensor;

        // 清理中间张量
        DestroyTensor(input_tensor);
        DestroyTensor(rgb_tensor);
        DestroyTensor(resized_tensor);
        DestroyTensor(normalized_tensor);
    }

private:
    aclTensor* MatToACLTensor(const cv::Mat& image) {
        // 获取图像尺寸
        int height = image.rows;
        int width = image.cols;
        int channels = image.channels();

        // 创建张量描述
        int64_t dims[4] = {1, height, width, channels};
        aclDataType dtype = ACL_FLOAT32;
        aclFormat format = ACL_FORMAT_NHWC;

        aclTensorDesc* desc = aclCreateTensorDesc(dtype, 4, dims, format);

        // 分配设备内存
        void* dev_ptr = nullptr;
        size_t data_size = height * width * channels * sizeof(float);
        aclrtMalloc(&dev_ptr, data_size, ACL_MEM_MALLOC_HUGE_FIRST);

        // 拷贝数据到设备
        std::vector<float> data;
        cv::Mat float_image;
        image.convertTo(float_image, CV_32F);
        data.assign((float*)float_image.data,
                    (float*)float_image.data + height * width * channels);

        aclrtMemcpy(dev_ptr, data_size, data.data(), data_size,
                    ACL_MEMCPY_HOST_TO_DEVICE);

        // 创建张量
        return aclCreateTensor(desc, dev_ptr);
    }

    aclTensor* Normalize(aclTensor* input) {
        // 定义归一化参数 (ImageNet标准化)
        float mean[3] = {0.485f, 0.456f, 0.406f};
        float std[3] = {0.229f, 0.224f, 0.225f};

        // 创建输出张量
        aclTensor* output = CloneTensorShape(input);

        // 执行归一化: (x - mean) / std
        aclnn_cv_normalize(input, output, mean, std, 3,
                          workspace_, workspace_size_);

        return output;
    }

    void DestroyTensor(aclTensor* tensor) {
        if (tensor) {
            void* dev_ptr = aclGetTensorBuffer(tensor);
            aclTensorDesc* desc = aclGetTensorDesc(tensor);
            aclrtFree(dev_ptr);
            aclDestroyTensorDesc(desc);
            aclDestroyTensor(tensor);
        }
    }

    int device_id_;
    aclrtStream stream_;
    void* workspace_;
    uint64_t workspace_size_;
};

3.2 目标检测后处理流水线

class DetectionPostProcessor {
public:
    struct DetectionResult {
        cv::Rect box;        // 检测框
        float score;         // 置信度
        int class_id;        // 类别ID
        std::string label;   // 类别标签
    };

    // 完整的目标检测后处理
    std::vector<DetectionResult> Process(
        const aclTensor* class_predictions,  // [N, num_classes]
        const aclTensor* box_predictions,    // [N, 4]
        const aclTensor* box_confidences,    // [N]
        const std::vector<std::string>& class_labels,
        float conf_threshold = 0.5f,
        float nms_threshold = 0.45f) {

        // 1. 应用置信度阈值过滤
        std::vector<int> valid_indices;
        FilterByConfidence(box_confidences, conf_threshold, valid_indices);

        // 2. 获取类别预测
        std::vector<int> class_ids;
        std::vector<float> max_scores;
        GetMaxClassScores(class_predictions, valid_indices, class_ids, max_scores);

        // 3. 解码检测框
        std::vector<cv::Rect> boxes;
        DecodeBoxes(box_predictions, valid_indices, boxes);

        // 4. 对每个类别分别应用NMS
        std::vector<DetectionResult> results;
        for (int cls_id = 0; cls_id < class_labels.size(); ++cls_id) {
            std::vector<int> class_indices;
            for (size_t i = 0; i < class_ids.size(); ++i) {
                if (class_ids[i] == cls_id) {
                    class_indices.push_back(i);
                }
            }

            if (class_indices.empty()) continue;

            // 提取当前类别的检测框和分数
            std::vector<cv::Rect> class_boxes;
            std::vector<float> class_scores;
            for (int idx : class_indices) {
                class_boxes.push_back(boxes[idx]);
                class_scores.push_back(max_scores[idx]);
            }

            // 应用NMS
            std::vector<int> nms_indices;
            ApplyNMS(class_boxes, class_scores, nms_threshold, nms_indices);

            // 保存结果
            for (int idx : nms_indices) {
                DetectionResult result;
                result.box = class_boxes[idx];
                result.score = class_scores[idx];
                result.class_id = cls_id;
                result.label = class_labels[cls_id];
                results.push_back(result);
            }
        }

        return results;
    }

private:
    void FilterByConfidence(const aclTensor* confidences,
                           float threshold,
                           std::vector<int>& valid_indices) {
        // 获取置信度数据
        void* conf_ptr = aclGetTensorBuffer(confidences);
        size_t conf_size = GetTensorSize(confidences);
        std::vector<float> conf_data(conf_size);
        aclrtMemcpy(conf_data.data(), conf_size * sizeof(float),
                   conf_ptr, conf_size * sizeof(float),
                   ACL_MEMCPY_DEVICE_TO_HOST);

        // 过滤
        for (size_t i = 0; i < conf_data.size(); ++i) {
            if (conf_data[i] >= threshold) {
                valid_indices.push_back(i);
            }
        }
    }

    void GetMaxClassScores(const aclTensor* predictions,
                          const std::vector<int>& indices,
                          std::vector<int>& class_ids,
                          std::vector<float>& max_scores) {
        // 获取预测数据
        void* pred_ptr = aclGetTensorBuffer(predictions);
        int num_preds = indices.size();
        int num_classes = GetTensorDimension(predictions, 1);

        std::vector<float> pred_data(num_preds * num_classes);
        aclrtMemcpy(pred_data.data(), pred_data.size() * sizeof(float),
                   pred_ptr, pred_data.size() * sizeof(float),
                   ACL_MEMCPY_DEVICE_TO_HOST);

        // 对每个预测找到最大类别
        for (size_t i = 0; i < indices.size(); ++i) {
            float max_score = -std::numeric_limits<float>::max();
            int max_class = 0;

            for (int c = 0; c < num_classes; ++c) {
                float score = pred_data[i * num_classes + c];
                if (score > max_score) {
                    max_score = score;
                    max_class = c;
                }
            }

            class_ids.push_back(max_class);
            max_scores.push_back(max_score);
        }
    }

    void DecodeBoxes(const aclTensor* box_predictions,
                    const std::vector<int>& indices,
                    std::vector<cv::Rect>& boxes) {
        // 获取预测框数据
        void* box_ptr = aclGetTensorBuffer(box_predictions);

        for (int idx : indices) {
            float box_coords[4];
            aclrtMemcpy(box_coords, sizeof(box_coords),
                       (float*)box_ptr + idx * 4, sizeof(box_coords),
                       ACL_MEMCPY_DEVICE_TO_HOST);

            boxes.push_back(cv::Rect(
                cv::Point(box_coords[0], box_coords[1]),
                cv::Point(box_coords[2], box_coords[3])
            ));
        }
    }

    void ApplyNMS(const std::vector<cv::Rect>& boxes,
                 const std::vector<float>& scores,
                 float iou_threshold,
                 std::vector<int>& keep_indices) {
        // 按分数降序排序
        std::vector<int> sorted_indices(scores.size());
        std::iota(sorted_indices.begin(), sorted_indices.end(), 0);
        std::sort(sorted_indices.begin(), sorted_indices.end(),
                 [&scores](int a, int b) { return scores[a] > scores[b]; });

        std::vector<bool> suppressed(scores.size(), false);

        for (size_t i = 0; i < sorted_indices.size(); ++i) {
            int idx = sorted_indices[i];
            if (suppressed[idx]) continue;

            keep_indices.push_back(idx);

            // 抑制重叠框
            for (size_t j = i + 1; j < sorted_indices.size(); ++j) {
                int other_idx = sorted_indices[j];
                if (suppressed[other_idx]) continue;

                float iou = ComputeIoU(boxes[idx], boxes[other_idx]);
                if (iou > iou_threshold) {
                    suppressed[other_idx] = true;
                }
            }
        }
    }

    float ComputeIoU(const cv::Rect& a, const cv::Rect& b) {
        cv::Rect intersection = a & b;
        cv::Rect union_rect = a | b;

        float intersection_area = intersection.area();
        float union_area = union_rect.area();

        return intersection_area / union_area;
    }
};

3.3 特征金字塔网络实现

class FeaturePyramidNetwork {
public:
    FeaturePyramidNetwork(int num_levels = 5)
        : num_levels_(num_levels) {}

    // 构建特征金字塔
    std::vector<aclTensor*> BuildFPN(
        const std::vector<aclTensor*>& bottom_up_features) {

        // bottom_up_features: [C3, C4, C5]
        // 返回: [P2, P3, P4, P5, P6]

        std::vector<aclTensor*> pyramid(num_levels_);

        // 1. 自顶向下路径
        // P5来自C5的1x1卷积
        pyramid[4] = Apply1x1Conv(bottom_up_features[2]);

        // 2. 横向连接和上采样
        // P4: Upsample(P5) + 1x1Conv(C4)
        aclTensor* p5_upsampled = Upsample(pyramid[4],
                                           bottom_up_features[1]->shape);
        aclTensor* c4_proj = Apply1x1Conv(bottom_up_features[1]);
        pyramid[3] = ElementwiseAdd(p5_upsampled, c4_proj);

        // P3: Upsample(P4) + 1x1Conv(C3)
        aclTensor* p4_upsampled = Upsample(pyramid[3],
                                           bottom_up_features[0]->shape);
        aclTensor* c3_proj = Apply1x1Conv(bottom_up_features[0]);
        pyramid[2] = ElementwiseAdd(p4_upsampled, c3_proj);

        // 3. P6: P5的3x3 stride-2池化
        pyramid[5] = MaxPool3x3Stride2(pyramid[4]);

        // 4. P2: P3的上采样
        aclTensor* p3_upsampled = Upsample(pyramid[2],
                                          bottom_up_features[0]->shape * 2);
        pyramid[1] = p3_upsampled;

        // 5. 对所有层级应用3x3卷积
        for (int i = 1; i < num_levels_; ++i) {
            pyramid[i] = Apply3x3Conv(pyramid[i]);
        }

        return pyramid;
    }

    // 双向特征金字塔网络
    std::vector<aclTensor*> BuildBiFPN(
        std::vector<aclTensor*>& features,
        int num_repeat = 2) {

        for (int repeat = 0; repeat < num_repeat; ++repeat) {
            // 自顶向下
            for (int i = features.size() - 2; i >= 0; --i) {
                aclTensor* upsampled = Upsample(features[i + 1],
                                                features[i]->shape);
                features[i] = FastNormalizedConcat(
                    {features[i], upsampled});
                features[i] = Apply3x3Conv(features[i]);
            }

            // 自底向上
            for (size_t i = 1; i < features.size(); ++i) {
                aclTensor* downsampled = Downsample(features[i - 1],
                                                    features[i]->shape);
                features[i] = FastNormalizedConcat(
                    {features[i], downsampled});
                features[i] = Apply3x3Conv(features[i]);
            }
        }

        return features;
    }

private:
    aclTensor* Apply1x1Conv(aclTensor* input) {
        int out_channels = 256;
        aclTensor* output = nullptr;
        aclnn_cv_conv2d(input, nullptr,  // input, bias
                       nullptr,          // filter (自动创建1x1卷积核)
                       output, out_channels,
                       {1, 1},           // kernel_size
                       {0, 0},           // padding
                       {1, 1},           // stride
                       {1, 1},           // dilation
                       1,                // groups
                       false,            // bias
                       workspace_, workspace_size_);
        return output;
    }

    aclTensor* Apply3x3Conv(aclTensor* input) {
        int out_channels = 256;
        aclTensor* output = nullptr;
        aclnn_cv_conv2d(input, nullptr, nullptr, output,
                       out_channels, {3, 3},
                       {1, 1}, {1, 1}, {1, 1},
                       1, false, workspace_, workspace_size_);
        return output;
    }

    aclTensor* Upsample(aclTensor* input,
                       const std::vector<int64_t>& target_shape) {
        aclTensor* output = nullptr;
        int64_t size[2] = {target_shape[2], target_shape[3]};
        aclnn_cv_resize(input, output, size, RESIZE_NEAREST_NEIGHBOR,
                       false, workspace_, workspace_size_);
        return output;
    }

    aclTensor* Downsample(aclTensor* input,
                         const std::vector<int64_t>& target_shape) {
        aclTensor* output = nullptr;
        int64_t size[2] = {target_shape[2], target_shape[3]};
        aclnn_cv_resize(input, output, size, RESIZE_AREA,
                       false, workspace_, workspace_size_);
        return output;
    }

    aclTensor* ElementwiseAdd(aclTensor* a, aclTensor* b) {
        aclTensor* output = nullptr;
        aclnn_cv_add(a, b, output, workspace_, workspace_size_);
        return output;
    }

    aclTensor* FastNormalizedConcat(const std::vector<aclTensor*>& inputs) {
        // 计算权重
        std::vector<float> weights(inputs.size(), 1.0f / inputs.size());

        // 加权求和
        aclTensor* output = nullptr;
        aclnn_cv_weighted_sum(inputs.data(), weights.data(),
                             inputs.size(), output,
                             workspace_, workspace_size_);
        return output;
    }

    aclTensor* MaxPool3x3Stride2(aclTensor* input) {
        aclTensor* output = nullptr;
        aclnn_cv_maxpool2d(input, output,
                          {3, 3},      // kernel_size
                          {1, 1},      // padding
                          {2, 2},      // stride
                          false,       // ceil_mode
                          workspace_, workspace_size_);
        return output;
    }

    int num_levels_;
    void* workspace_;
    uint64_t workspace_size_;
};

3.4 视频分析应用

class VideoAnalyzer {
public:
    // 光流跟踪
    std::vector<cv::Point2f> TrackOpticalFlow(
        const cv::Mat& prev_frame,
        const cv::Mat& curr_frame,
        const std::vector<cv::Point2f>& prev_points) {

        // 转换为ACL张量
        aclTensor* prev_tensor = MatToTensor(prev_frame);
        aclTensor* curr_tensor = MatToTensor(curr_frame);

        // 计算稠密光流
        aclTensor* flow_tensor = nullptr;
        aclnn_cv_optical_flow(prev_tensor, curr_tensor, flow_tensor,
                             FLOW_RAFT, workspace_, workspace_size_);

        // 在关键点处提取光流
        std::vector<cv::Point2f> curr_points;
        for (const auto& pt : prev_points) {
            int x = static_cast<int>(pt.x);
            int y = static_cast<int>(pt.y);

            // 获取光流向量
            float flow_uv[2];
            ExtractFlowAtPoint(flow_tensor, x, y, flow_uv);

            // 计算新位置
            curr_points.push_back(cv::Point2f(
                pt.x + flow_uv[0],
                pt.y + flow_uv[1]
            ));
        }

        // 清理
        DestroyTensor(prev_tensor);
        DestroyTensor(curr_tensor);
        DestroyTensor(flow_tensor);

        return curr_points;
    }

    // 背景建模与运动检测
    cv::Mat DetectMotion(const cv::Mat& frame, int history = 500) {
        static cv::Ptr<cv::BackgroundSubtractorMOG2> bg_subtractor =
            cv::createBackgroundSubtractorMOG2(history, 16, true);

        cv::Mat fg_mask;
        bg_subtractor->apply(frame, fg_mask);

        // 使用形态学操作去噪
        cv::Mat kernel = cv::getStructuringElement(
            cv::MORPH_ELLIPSE, cv::Size(5, 5));
        cv::morphologyEx(fg_mask, fg_mask, cv::MORPH_CLOSE, kernel);

        return fg_mask;
    }

    // 视频稳定
    cv::Mat StabilizeFrame(const cv::Mat& frame,
                          const cv::Mat& prev_gray,
                          std::vector<cv::Point2f>& prev_points) {

        cv::Mat gray;
        cv::cvtColor(frame, gray, cv::COLOR_BGR2GRAY);

        if (prev_points.empty()) {
            // 初始化关键点
            cv::goodFeaturesToTrack(gray, prev_points, 200, 0.01, 30);
            return frame.clone();
        }

        // 计算光流
        std::vector<cv::Point2f> curr_points;
        std::vector<uchar> status;
        std::vector<float> err;

        cv::calcOpticalFlowPyrLK(prev_gray, gray, prev_points,
                                curr_points, status, err);

        // 过滤有效点
        std::vector<cv::Point2f> valid_prev, valid_curr;
        for (size_t i = 0; i < status.size(); ++i) {
            if (status[i]) {
                valid_prev.push_back(prev_points[i]);
                valid_curr.push_back(curr_points[i]);
            }
        }

        if (valid_prev.size() < 10) {
            return frame.clone();
        }

        // 估计仿射变换
        cv::Mat transform = cv::estimateAffinePartial2D(
            valid_curr, valid_prev);

        // 应用变换进行稳定
        cv::Mat stabilized;
        if (!transform.empty()) {
            cv::warpAffine(frame, stabilized, transform, frame.size());
        } else {
            stabilized = frame.clone();
        }

        // 更新状态
        prev_points = curr_points;
        return stabilized;
    }

private:
    void ExtractFlowAtPoint(aclTensor* flow, int x, int y, float* flow_uv) {
        int width = GetTensorDimension(flow, 3);
        int height = GetTensorDimension(flow, 2);

        void* flow_ptr = aclGetTensorBuffer(flow);
        int offset = (y * width + x) * 2;

        aclrtMemcpy(flow_uv, 2 * sizeof(float),
                   (float*)flow_ptr + offset, 2 * sizeof(float),
                   ACL_MEMCPY_DEVICE_TO_HOST);
    }

    aclTensor* MatToTensor(const cv::Mat& image) {
        // 简化版本，实际需要完整实现
        return nullptr;
    }

    void DestroyTensor(aclTensor* tensor) {
        if (tensor) {
            aclDestroyTensor(tensor);
        }
    }

    void* workspace_;
    uint64_t workspace_size_;
};

---

四、性能优化技巧

4.1 算子融合优化

// 预处理融合算子: Resize + Normalize + Transpose
class FusedPreprocess {
public:
    aclError Execute(const cv::Mat& input,
                    aclTensor* output,
                    int target_height,
                    int target_width) {
        // 调用融合算子，一次完成所有预处理
        return aclnn_cv_fused_preprocess(
            input.data, input.cols, input.rows, input.channels(),
            output, target_height, target_width,
            mean_, std_, workspace_, workspace_size_);
    }

private:
    float mean_[3] = {0.485f, 0.456f, 0.406f};
    float std_[3] = {0.229f, 0.224f, 0.225f};
    void* workspace_;
    uint64_t workspace_size_;
};

4.2 批处理优化

// 批量图像预处理
class BatchImagePreprocessor {
public:
    void ProcessBatch(const std::vector<cv::Mat>& images,
                     std::vector<aclTensor*>& outputs) {
        // 分配批处理内存
        size_t batch_size = images.size();
        size_t batch_bytes = batch_size * image_size_ * sizeof(float);

        void* batch_buffer;
        aclrtMalloc(&batch_buffer, batch_bytes, ACL_MEM_MALLOC_HUGE_FIRST);

        // 并行处理
        #pragma omp parallel for
        for (size_t i = 0; i < images.size(); ++i) {
            void* img_buffer = (char*)batch_buffer + i * image_size_ * sizeof(float);
            ProcessSingle(images[i], img_buffer);
        }

        // 创建批处理张量
        int64_t dims[4] = {static_cast<int64_t>(batch_size), 3, 224, 224};
        outputs[0] = CreateBatchTensor(batch_buffer, dims);

        // 释放临时内存
        aclrtFree(batch_buffer);
    }

private:
    void ProcessSingle(const cv::Mat& image, void* output) {
        // 单个图像处理逻辑
    }

    aclTensor* CreateBatchTensor(void* buffer, const int64_t* dims) {
        // 创建批处理张量
        return nullptr;
    }

    size_t image_size_ = 3 * 224 * 224;
};

4.3 内存池优化

class TensorMemoryPool {
public:
    TensorMemoryPool(size_t pool_size = 1024 * 1024 * 1024)  // 1GB
        : pool_size_(pool_size) {
        aclrtMalloc(&pool_base_, pool_size_, ACL_MEM_MALLOC_HUGE_FIRST);
        offset_ = 0;
    }

    ~TensorMemoryPool() {
        aclrtFree(pool_base_);
    }

    void* Allocate(size_t size) {
        if (offset_ + size > pool_size_) {
            Reset();
        }
        void* ptr = (char*)pool_base_ + offset_;
        offset_ += size;
        return ptr;
    }

    void Reset() {
        offset_ = 0;
    }

private:
    void* pool_base_;
    size_t pool_size_;
    size_t offset_;
};

---

五、总结

ops-cv库为计算机视觉应用提供了全面的算子支持，从基础的图像预处理到高级的特征提取和视频分析。通过合理使用这些算子，结合算子融合、批处理和内存优化等技巧，可以构建高效的视觉处理流水线，充分发挥NPU硬件的计算能力。

在实际应用中，建议：

1. 优先使用融合算子减少内存访问
2. 合理组织数据布局提高缓存利用率
3. 根据精度需求选择合适的数据类型
4. 使用异步执行实现计算与传输重叠

相关链接：

• CANN组织链接：https://atomgit.com/cann
• ops-cv仓库链接：https://atomgit.com/cann/ops-cv