Java（五）人脸识别（ArcSoft）

前言

示例 Demo 在 itboyst 的基础上进行修改编写。

背景

虹软视觉开放平台注册账号，创建应用并记录下 APP_ID、SDK_KEY。
将三个引擎库 libarcsoft_face、libarcsoft_face_engine、libarcsoft_face_engine_jni 文件拷贝到 java.library.path 所包含的路径下。例如我的平台是 Windows 10 x64，安装的 jdk 位置为 C:\Program Files\Java\jdk1.8.0_231，所以将这三个 dll 文件放在此路径下。
背景 2 被删除的原因是：会在配置文件中指定引擎库的位置。

步骤

人脸识别步骤：

对图片中的人脸进行识别。
抽取人脸特征值，存入特征库。
抽取图片中人脸的特征值，与特征库中的匹配获取相似度。相似度介于 0 ~ 1 之间，越接近 1 相似度越高。

项目代码

application.properties 配置

# 上传文件 最大值限制
spring.servlet.multipart.max-file-size=100MB
# 请求 最大值限制
spring.servlet.multipart.max-request-size=100MB

# ourbatis 模板所在 classpath 下的相对路径
ourbatis.template-locations=ourbatis.xml
# ourbatis 扫描 domain 包名
ourbatis.domain-locations=com.caroly.arcsoft.facedemo.beans.pojo
spring.freemarker.suffix=.ftl

# 开发环境
server.port=8000
# 引擎库位置
config.arcface-sdk.sdk-lib-path=E:\\ArcSoft\\3.0\\WIN64
# APP_ID、SDK_KEY
config.freesdk.app-id=xxxxxxxxxxxxx
config.freesdk.sdk-key=xxxxxxxxxxxxxxxxxxxxxxxxxx
# 线程数量
config.freesdk.thread-pool-size=5

# druid
# mysql 驱动
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.druid.url=jdbc:mysql://192.168.1.30:3306/arcsoft_face_demo?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=UTC
spring.datasource.druid.username=root
spring.datasource.druid.password=KwX9Yvv8VK9uDgMcxbMpDj+QFmmX+6Rw1LVzjp9qfa/jExpYIZOimoyygNaD8Lwk61MyD3s2f3toOA8fysDPJw==
#生成的公钥
public-key=MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAKVSksd3AfcDOICUEcmik5F/Hc2Dz4t0hlZ3y/K69ZNVFH8Ii6L8BYjyXw3qNdB8dPTQiyIDdBSoKKY75oeBy28CAwEAAQ==
spring.datasource.druid.filters=stat
spring.datasource.druid.connection-properties=config.decrypt=true;config.decrypt.key=${public-key}
# 启用 ConfigFilter
spring.datasource.druid.filter.config.enabled=true
#连接池 Druid 阿里巴巴
spring.datasource.type=com.alibaba.druid.pool.DruidDataSource
# 初始化时建立物理连接的个数
spring.datasource.druid.initial-size=5
# 最小连接池数量
spring.datasource.druid.min-idle=5
# 最大连接池数量 maxIdle 已经不再使用
spring.datasource.druid.max-active=20
# 既作为检测的间隔时间又作为 testWhileIdel 执行的依据
spring.datasource.druid.time-between-eviction-runs-millis=60000
# 销毁线程时检测当前连接的最后活动时间和当前时间差大于该值时，关闭当前连接
spring.datasource.druid.min-evictable-idle-time-millis=600000
# 用来检测连接是否有效的 sql 必须是一个查询语句
spring.datasource.druid.validation-query=SELECT 'x'
# 是否缓存 preparedStatement，也就是 PSCache。PSCache 对支持游标的数据库性能提升巨大，比如说 Oracle。在 mysql 下建议关闭。
spring.datasource.druid.pool-prepared-statements=true
# 要启用 PSCache，必须配置大于 0，当大于 0 时，poolPreparedStatements 自动触发修改为 true。
spring.datasource.druid.max-pool-prepared-statement-per-connection-size=20

Druid 密码加密

使用的 jar 版本为：druid-1.1.21

DOS 窗口中跳转到 jar 包所在路径，执行如下命令：

java -cp druid-1.1.21.jar com.alibaba.druid.filter.config.ConfigTools password

生成三个密钥：privateKey、publicKey、password。

pom.xml 主要配置

<dependencies>
    ... ...
    <!-- Ourbatis -->
    <dependency>
        <groupId>com.smallnico</groupId>
        <artifactId>ourbatis-spring-boot-starter</artifactId>
        <version>1.0.6</version>
    </dependency>
    <!-- Hutool 工具类库 -->
    <dependency>
        <groupId>cn.hutool</groupId>
        <artifactId>hutool-all</artifactId>
        <version>4.1.14</version>
    </dependency>
    <!-- Guava -->
    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>26.0-jre</version>
    </dependency>
    <!-- ArcSoft -->
    <dependency>
        <groupId>com.arcsoft.face</groupId>
        <artifactId>arcsoft-sdk-face</artifactId>
        <version>3.0.0.0</version>
        <scope>system</scope>
        <systemPath>${basedir}/lib/arcsoft-sdk-face-3.0.0.0.jar</systemPath>
    </dependency>
    ... ...
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <includeSystemScope>true</includeSystemScope>
                <fork>true</fork>
            </configuration>
        </plugin>
    </plugins>
</build>

上传人脸图片

处理上传的图片转为 RGB 数据，并提取人脸特征。将结果存入特征库中。

@RequestMapping(value = "/uploadFaceImage", method = RequestMethod.POST)
@ResponseBody
public Result<Object> faceAdd(@RequestParam("file") MultipartFile file, @RequestParam("faceId") Integer faceId, @RequestParam("name") String name) {
    try {
        if (file == null|| faceId== null|| name== null) {
            return Results.newFailedResult(ErrorCodeEnum.INVALID_PARAM);
        }

        InputStream inputStream = file.getInputStream();
        ImageInfo imageInfo = ImageUtil.getRGBData(inputStream);

        // 人脸特征获取
        byte[] bytes = faceEngineService.extractFaceFeature(imageInfo);
        if (bytes == null) {
            return Results.newFailedResult(ErrorCodeEnum.NO_FACE_DETECTED);
        }

        UserFaceInfo userFaceInfo = new UserFaceInfo();
        userFaceInfo.setName(name);
        userFaceInfo.setfaceId(faceId);
        userFaceInfo.setFaceFeature(bytes);
        userFaceInfo.setFaceId(RandomUtil.randomString(10));
        // 人脸特征插入到数据库
        userFaceInfoService.insertSelective(userFaceInfo);

        return Results.newSuccessResult("Successfully entered the face");
    } catch (Exception e) {
        logger.error("", e);
    }
    return Results.newFailedResult(ErrorCodeEnum.UNKNOWN);
}

从构建的对象池中获取对象，进行特征的提取。此处开启的功能为：人脸检测、人脸识别、年龄检测、性别检测、活体检测。

/**
 * 人脸特征
 * @param imageInfo
 * @return
 */
@Override
public byte[] extractFaceFeature(ImageInfo imageInfo) throws InterruptedException {
    FaceEngine faceEngine = null;
    try {
        // 获取引擎对象
        faceEngine = extractFaceObjectPool.borrowObject();

        // 人脸检测得到人脸列表
        List<FaceInfo> faceInfoList = new ArrayList<FaceInfo>();

        // 人脸检测
        faceEngine.detectFaces(imageInfo.getRgbData(), imageInfo.getWidth(), imageInfo.getHeight(), ImageFormat.CP_PAF_BGR24, faceInfoList);

        if (CollectionUtil.isNotEmpty(faceInfoList)) {
            FaceFeature faceFeature = new FaceFeature();
            // 提取人脸特征
            faceEngine.extractFaceFeature(imageInfo.getRgbData(), imageInfo.getWidth(), imageInfo.getHeight(), ImageFormat.CP_PAF_BGR24, faceInfoList.get(0), faceFeature);

            return faceFeature.getFeatureData();
        }
    } catch (Exception e) {
        logger.error("", e);
    } finally {
        if (faceEngine != null) {
            // 释放引擎对象
            extractFaceObjectPool.returnObject(faceEngine);
        }
    }
    return null;
}

人脸检测对比

初始化缓存，将特征库中的特征值放入缓存中。

private LoadingCache<Integer, List<FaceUserInfo>> faceGroupCache;
/**
     * 初始化缓存
     */
private void initCache() {
    this.faceGroupCache = CacheBuilder
        .newBuilder()
        .maximumSize(1000)	// 最大 1000 条
        .expireAfterAccess(2, TimeUnit.HOURS) // 设置时效时间，2 个小时过期
        .build(new CacheLoader<Integer, List<FaceUserInfo>>() {
            @Override
            public List<FaceUserInfo> load(Integer faceId) throws Exception {
                UserFaceInfo userFaceInfo = new UserFaceInfo();
                userFaceInfo.setGroupId(faceId);
                List<UserFaceInfo> userFaceInfoList = userFaceInfoMapper.selectList(userFaceInfo);

                List<FaceUserInfo> userFaceInfoListTarget = Lists.newLinkedList();
                userFaceInfoList.forEach(k -> {
                    FaceUserInfo info = new FaceUserInfo();
                    BeanUtil.copyProperties(k, info);
                    userFaceInfoListTarget.add(info);
                });
                return userFaceInfoListTarget;
            }
        });
}

上传新的人脸图片，提取人脸特征值与特征库中的对比。

@RequestMapping(value = "/faceSearch", method = RequestMethod.POST)
@ResponseBody
public Result<FaceSearchResInfo> faceSearch(MultipartFile file, Integer faceId) throws Exception {
    if (faceId == null) {
        return Results.newFailedResult("faceId is null");
    }

    InputStream inputStream = file.getInputStream();
    BufferedImage bufImage = ImageIO.read(inputStream);
    ImageInfo imageInfo = ImageUtil.bufferedImage2ImageInfo(bufImage);
    inputStream.close();

    // 人脸特征获取
    byte[] bytes = faceEngineService.extractFaceFeature(imageInfo);

    if (bytes == null) {
        return Results.newFailedResult(ErrorCodeEnum.NO_FACE_DETECTED);
    }
    // 人脸比对，获取比对结果
    List<FaceUserInfo> userFaceInfoList = faceEngineService.compareFaceFeature(bytes, faceId);

    if (CollectionUtil.isNotEmpty(userFaceInfoList)) {
        FaceUserInfo faceUserInfo = userFaceInfoList.get(0);
        FaceSearchResInfo faceSearchResDto = new FaceSearchResInfo();
        BeanUtil.copyProperties(faceUserInfo, faceSearchResDto);
        List<ProcessInfo> processInfoList = faceEngineService.process(imageInfo);
        if (CollectionUtil.isNotEmpty(processInfoList)) {
            //人脸检测    将检测到的人脸用红框框出
            List<FaceInfo> faceInfoList = faceEngineService.detectFaces(imageInfo);
            int left = faceInfoList.get(0).getRect().getLeft();
            int top = faceInfoList.get(0).getRect().getTop();
            int width = faceInfoList.get(0).getRect().getRight() - left;
            int height = faceInfoList.get(0).getRect().getBottom()- top;

            Graphics2D graphics2D = bufImage.createGraphics();
            graphics2D.setColor(Color.RED);//红色
            BasicStroke stroke = new BasicStroke(5f);
            graphics2D.setStroke(stroke);
            graphics2D.drawRect(left, top, width, height);

            ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
            ImageIO.write(bufImage, "jpg", outputStream);
            byte[] bytes1 = outputStream.toByteArray();
            faceSearchResDto.setImage("data:image/jpeg;base64," + Base64Utils.encodeToString(bytes1));
            faceSearchResDto.setAge(processInfoList.get(0).getAge());
            faceSearchResDto.setGender(processInfoList.get(0).getGender().equals(1) ? "女" : "男");
            faceSearchResDto.setLiveness(processInfoList.get(0).getLiveness().equals(1)? "活体": "非活体");
        }
        return Results.newSuccessResult(faceSearchResDto);
    }
    return Results.newFailedResult(ErrorCodeEnum.FACE_DOES_NOT_MATCH);
}

特征库中的特征值数量较多，使用多线程来处理。

@Override
public List<FaceUserInfo> compareFaceFeature(byte[] faceFeature, Integer faceId) throws InterruptedException, ExecutionException {
    List<FaceUserInfo> resultFaceInfoList = Lists.newLinkedList();//识别到的人脸列表

    FaceFeature targetFaceFeature = new FaceFeature();
    targetFaceFeature.setFeatureData(faceFeature);
    List<FaceUserInfo> faceInfoList = faceGroupCache.get(faceId);//从缓存中提取人脸库

    List<List<FaceUserInfo>> faceUserInfoPartList = Lists.partition(faceInfoList, 1000);    // 分成1000一组，多线程处理
    CompletionService<List<FaceUserInfo>> completionService = new ExecutorCompletionService(executorService);
    for (List<FaceUserInfo> part : faceUserInfoPartList) {
        completionService.submit(new CompareFaceTask(part, targetFaceFeature));
    }
    for (int i = 0; i < faceUserInfoPartList.size(); i++) {
        List<FaceUserInfo> faceUserInfoList = completionService.take().get();
        if (CollectionUtil.isNotEmpty(faceInfoList)) {
            resultFaceInfoList.addAll(faceUserInfoList);
        }
    }
    resultFaceInfoList.sort((h1, h2) -> h2.getSimilarValue().compareTo(h1.getSimilarValue()));//从大到小排序
    return resultFaceInfoList;
}

此处定义的相似值为：0.8。获取到所有相似值大于 0.8 的数据进行倒叙排列，相似值最大的置于第一个。

private class CompareFaceTask implements Callable<List<FaceUserInfo>> {

    private List<FaceUserInfo> faceUserInfoList;
    private FaceFeature targetFaceFeature;

    public CompareFaceTask(List<FaceUserInfo> faceUserInfoList, FaceFeature targetFaceFeature) {
        this.faceUserInfoList = faceUserInfoList;
        this.targetFaceFeature = targetFaceFeature;
    }

    @Override
    public List<FaceUserInfo> call() throws Exception {
        FaceEngine faceEngine = null;
        List<FaceUserInfo> resultFaceInfoList = Lists.newLinkedList();//识别到的人脸列表
        try {
            faceEngine = compareFaceObjectPool.borrowObject();
            for (FaceUserInfo faceUserInfo : faceUserInfoList) {
                FaceFeature sourceFaceFeature = new FaceFeature();
                sourceFaceFeature.setFeatureData(faceUserInfo.getFaceFeature());
                FaceSimilar faceSimilar = new FaceSimilar();
                faceEngine.compareFaceFeature(targetFaceFeature, sourceFaceFeature, faceSimilar);
                Integer similarValue = plusHundred(faceSimilar.getScore());//获取相似值
                if (similarValue > passRate) {// 相似值大于配置预期，加入到识别到人脸的列表

                    FaceUserInfo info = new FaceUserInfo();
                    info.setName(faceUserInfo.getName());
                    info.setFaceId(faceUserInfo.getFaceId());
                    info.setSimilarValue(similarValue);
                    resultFaceInfoList.add(info);
                }
            }
        } catch (Exception e) {
            logger.error("", e);
        } finally {
            if (faceEngine != null) {
                compareFaceObjectPool.returnObject(faceEngine);
            }
        }
        return resultFaceInfoList;
    }
}

获取匹配后的人脸列表，需要对相似度最高的进行数据处理：提取性别、年龄、活体结果。

@Override
public List<ProcessInfo> process(ImageInfo imageInfo){
    FaceEngine faceEngine = null;
    try {
        // 获取引擎对象
        faceEngine = extractFaceObjectPool.borrowObject();
        // 人脸检测得到人脸列表
        List<FaceInfo> faceInfoList = new ArrayList<FaceInfo>();
        // 人脸检测
        faceEngine.detectFaces(imageInfo.getRgbData(), imageInfo.getWidth(), imageInfo.getHeight(), ImageFormat.CP_PAF_BGR24, faceInfoList);
        faceEngine.process(imageInfo.getRgbData(), imageInfo.getWidth(), imageInfo.getHeight(), ImageFormat.CP_PAF_BGR24, faceInfoList, FunctionConfiguration.builder().supportAge(true).supportGender(true).supportLiveness(true).build());
        List<ProcessInfo> processInfoList=Lists.newLinkedList();

        List<GenderInfo> genderInfoList = new ArrayList<GenderInfo>();
        // 性别提取
        faceEngine.getGender(genderInfoList);
        // 年龄提取
        List<AgeInfo> ageInfoList = new ArrayList<AgeInfo>();
        faceEngine.getAge(ageInfoList);
        //活体结果列表
        List<LivenessInfo> livenessInfoList = new ArrayList<LivenessInfo>();
        faceEngine.getLiveness(livenessInfoList);
        for (int i = 0; i <genderInfoList.size() ; i++) {
            ProcessInfo processInfo=new ProcessInfo();
            processInfo.setGender(genderInfoList.get(i).getGender());
            processInfo.setAge(ageInfoList.get(i).getAge());
            processInfo.setLiveness(livenessInfoList.get(i).getLiveness());
            processInfoList.add(processInfo);
        }
        return processInfoList;
    } catch (Exception e) {
        logger.error("", e);
    } finally {
        if (faceEngine != null) {
            //释放引擎对象
            extractFaceObjectPool.returnObject(faceEngine);
        }
    }
    return null;
}

Java Spring Boot

更新时间：2021-04-29 14:45:13

本文由 caroly 创作，如果您觉得本文不错，请随意赞赏
采用知识共享署名4.0 国际许可协议进行许可
本站文章除注明转载 / 出处外，均为本站原创或翻译，转载前请务必署名
原文链接：https://caroly.fun/archives/人脸识别arcsoft
最后更新：2021-04-29 14:45:13