一 dlib本地安装与编译

1.1 dlib源码下载

下载地址：https://github.com/davisking/dlib （当前最新版dlib 19.15）

为了区分版本，将下载目录命名为dlib-19-15，如上图所示。

1.2 dlib C++编译示例程序

1.2.1 dlib库编译

编译需要安装VS，在此安装的是最新版Visual Studio 15 2017版本。

进入dlib-master/dlib-19-15目录，运行：

# mkdir build
# cd build
# cmake .. 
# cmake --build .

指定运行环境及模式：

# cmake .. -G "Visual Studio 15 2017 Win64" -T host=x64

在上图的目录下能看到生成的.lib依赖项，则代表dllib库成功编译。

1.2.2 C++示例程序配置、运行

以examples/train_shape_predictor_ex.cpp为例，其他示例代码操作相同。
1、创建ConsoleApplication1.cpp和source.cpp来源
首先，打开VS新建一个C++控制台工程，将train_shape_predictor_ex.cpp的代码复制到ConsoleApplication1.cpp，以添加现有项的方式加入source.cpp文件，source.cpp文件在dlib-master/dlib-19-15/dlib/all目录下。

2、修改stadfx属性
进入项目-属性进行以下修改，避免预编译头带来的error。

3、 加入目录

4、 加入生成的依赖项.lib的路径

5、图形处理类配置
加入DLIB_JPG_SUPPORT、DLIB_JPEG_ SUPPORT、DLIB_JPEG_STATIC

项目配置完成后，点击生成-生成解决方案，工程目录下将会生成ConsoleApplication1.exe文件。以命令行的方式运行ConsoleApplication1.exe文件，或者在VS上点击调试-开始执行即可。有参数输入的需要输入命令行参数。

1.3 dlib python API编译

方法一：
进入目录，运行：

# python setup.py install

之后进入python_examples便可运行python示例程序。

方法二：

# pip3 install dlib

这种方法目前本地dlib19.15版本不能成功安装，只能安装低版本的dlib，这样python示例中的某些函数调用可能不能正常运行。

二 dlib库的主要功能及准确率评估

dlib库中的主要功能包括人脸检测、人脸关键点检测、人脸识别三部分。此处研究python_examples示例代码部分，C++程序示例类似。这里的评估实现主要是参考2.1节中示例代码的二次开发代码。

2.1 代码功能简介

主要代码在dlib库的python_examples目录下，其中需要用到的模型文件下载地址为http://dlib.net/files：

• face_detector.py
  人脸正面检测器，主要使用dlib.get_frontal_face_detector()。
• cnn_face_detector.py
  人脸检测器，主要使用dlib.cnn_face_detection_model_v1 (‘mmod_human_face_detector.dat’)，官方指出比dlib.get_frontal_face_detector()准确率高。
• face_landmark.py
  人脸关键点检测，主要使用dlib.get_frontal_face_detector()和dlib.shape_predictor(‘shape_predictor_68_face_landmarks.dat’)。
• face_recognition.py
  人脸识别，主要使用dlib.get_frontal_face_detector()和dlib.shape_predictor(‘shape_predictor_5_face_landmarks.dat’)和dlib.face_recognition_model_v1(‘dlib_face_recognition_resnet_model_v1.dat’)。
• opencv_webcam_face_detection.py
  人脸检测的视频使用，主要使用dlib.get_frontal_face_detector()和cv2.VideoCapture()。
• train_object_detector.py
  人脸正面检测器的训练部分，训练生成detector.svm文件。
• train_shape_predictor.py
  人脸关键点检测器的训练部分，训练生成predictor.dat文件。

2.2 人脸检测和人脸关键点

2.2.1 数据集、代码准备

使用参考代码：examples/face_landmark_detection.py，为了进行人脸准确率统计，将其改写并命名为face_landmark.py，目前只能统计图片中含单个人脸的准确率（每张图片含多个人脸难以统计总的准确率）。

需要的模型文件：shape_predictor_68_face_landmarks.dat是训练好的人脸关键点检测器。

待测图像数据集：LFW数据集。

face_landmark.py代码如下：

import os
import dlib
from skimage import io# 待测人脸数据集
faces_folder_path = "lfwdata"# 第一步，人脸检测器和人脸关键点检测器加载
# 人脸检测器
detector = dlib.get_frontal_face_detector()
# 人脸关键点检测器
predictor = dlib.shape_predictor("../shape_predictor_68_face_landmarks.dat")# 第二步，遍历图片，使用人脸检测器和人脸关键点检测器，并显示
# 窗口
win = dlib.image_window()
# 统计检测正确数
tol = ans = 0
# 遍历文件夹中的jpg图片
for (path, dirnames, filenames) in os.walk(faces_folder_path):for filename in filenames:if filename.endswith('.jpg') or filename.endswith('.png'):tol += 1img_path = path + '/' + filenameprint("Processing file: {}".format(img_path))# 读取图片img = io.imread(img_path)win.clear_overlay()win.set_image(img)# 人脸检测器的使用dets = detector(img, 1)# 统计每张图片人脸个数>0判断是否检测成功face_num = len(dets)print("Number of faces detected: {}".format(len(dets)))if face_num > 0:ans += 1else:print("fail")for k, d in enumerate(dets):print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(k, d.left(), d.top(), d.right(), d.bottom()))# 人脸关键点检测器的使用shape = predictor(img, d)print("Part 0: {}, Part 1: {} ...".format(shape.part(0), shape.part(1)))win.add_overlay(shape)win.add_overlay(dets)# 鼠标控制下一张# dlib.hit_enter_to_continue()# 第三步，计算准确率
# 打印准确率
print("correct:{},total{}".format(ans, tol))
print("correct:{}".format(ans/tol))

2.2.2 测试效果图

2.2.3 准确率

检测总共13234张图片，检测到有人脸的有13172张照片，准确率为：99.53%。

测试失败的图像中，人像多为半脸、侧脸、曝光或有遮挡。这与代码中使用的是正脸检测器dlib.get_frontal_face_detector()有很大关系，检测失败的部分图片如下：

2.3 人脸识别

2.3.1 数据集、代码准备

使用参考代码：face_recognition.py，为了进行准确率统计，将其改写并命名为face_recog.py。

需要的模型文件：shape_predictor_68_face_landmarks.dat是训练好的人脸关键点检测器。dlib_face_recognition_resnet_model_v1.dat是训练好的ResNet人脸识别模型。

数据集：lfw数据集挑选候选人脸398张正脸（每人一张图片），待测人脸525张正脸（每个人可能含有多张图片）。

face_recog.py代码如下：

import os
import dlib
import glob
import numpy
from skimage import io# 训练人脸文件夹
faces_folder_path = "recog_train"
# 待测人脸文件夹
img_folder_path = "recog_test"# 第二步，生成训练人脸标签和描述子，供人脸识别使用
# 对文件夹下的每一个人脸进行:
# 1.人脸检测
# 2.关键点检测
# 3.描述子提取
# 训练人脸标签和描述子list
def train(faces_folder_path):trainlabel = []train_descriptors = []for file in glob.glob(os.path.join(faces_folder_path, "*.jpg")):labelName = file.split('_0')[0].split('\\')[1]trainlabel.append(labelName)# print("Processing file: {}".format(labelName))face = io.imread(file)# 1.人脸检测dets = detector(face, 1)# print("Number of faces detected: {}".format(len(dets)))for k, d in enumerate(dets):# 2.关键点检测shape = predictor(face, d)# 3.描述子提取，128D向量face_descriptor = facerec.compute_face_descriptor(face, shape)# 转换为numpy arrayface_vector = numpy.array(face_descriptor)train_descriptors.append(face_vector)return trainlabel, train_descriptors# 第三步，识别待测人脸是哪个人
def recognition(trainlabel, train_descriptors):ans_right = 0ans_wrong = 0# 对需识别人脸进行同样处理for file in glob.glob(os.path.join(img_folder_path, "*.jpg")):img = io.imread(file)# 人脸检测dets = detector(img, 1)# 待测人脸与所有训练人脸的距离dists = []for k, d in enumerate(dets):# 关键点检测shape = predictor(img, d)# 提取描述子test_descriptor = facerec.compute_face_descriptor(img, shape)d_test = numpy.array(test_descriptor)# 计算欧式距离for d_train in train_descriptors:dist = numpy.linalg.norm(d_train-d_test)dists.append(dist)# 待测人脸和所有训练人脸的标签、距离组成一个dictc_d = dict(zip(trainlabel, dists))cd_sorted = sorted(c_d.items(), key=lambda d:d[1])nametest = file.split('_0')[0].split('\\')[1]print(cd_sorted[0][1])# 设置阈值判断是哪个人if cd_sorted[0][1] < 0.6:namepredict = cd_sorted[0][0]else:namepredict = "Unknown"print(nametest, namepredict)# 判断识别是否正确识别if(namepredict == nametest) or (namepredict == "Unknown" and nametest not in trainlabel):print("right")ans_right += 1else:print("wrong")ans_wrong += 1# dlib.hit_enter_to_continue()print("total:", ans_right + ans_wrong, "\nright:", ans_right, "\nwrong:", ans_wrong)if  __name__ == '__main__':# 第一步，三种检测器的加载# 1.加载正脸检测器detector = dlib.get_frontal_face_detector()# 2.加载人脸关键点检测器predictor = dlib.shape_predictor("../shape_predictor_68_face_landmarks.dat")# 3. 加载人脸识别模型facerec = dlib.face_recognition_model_v1("../dlib_face_recognition_resnet_model_v1.dat")# 第二步，生成训练人脸标签和描述子，供人脸识别使用trainlabel, train_descriptors = train(faces_folder_path)# 第三步，识别待测人脸是哪个人并统计正确率recognition(trainlabel, train_descriptors)

2.3.2 人脸识别步骤

首先，先将候选人脸文件夹中的人脸进行：
1.人脸检测
2.关键点检测，画出人脸区域和和关键点
3.描述子提取，128D向量，转换为numpy array
4.将候选人图像的文件名提取出来，作为候选人名单

然后，对待测人脸进行同样的处理：
1.人脸检测，关键点检测，描述子提取
2.计算待测人脸描述子和候选人脸描述子之间的欧氏距离
3.将所有候选人与待测人脸描述子的距离组成一个dict
4.排序
5.距离最小者且阈值小于0.6，判定为同一个人

2.3.3 准确率

检测的525张图片中，有503张检测成功，准确率为：503/525=95.81%。

2.4 视频中的人脸检测、人脸识别

2.4.1 摄像头读入检测时间测试

代码命名为face_detector_video.py。代码如下：

import cv2
import dlib
import time# 初始化dlib人脸检测器
detector = dlib.get_frontal_face_detector()# 初始化显示窗口
win = dlib.image_window()
# opencv加载视频文件
# cap = cv2.VideoCapture(r'../test.mp4')
cap = cv2.VideoCapture(0) #加载摄像头while True:start = time.time()ret, cv_img = cap.read()if cv_img is None:break# 缩小图像至1/4cv_img = cv2.resize(cv_img, (0, 0), fx=0.25, fy=0.25)# OpenCV默认是读取为RGB图像，而dlib需要的是BGR图像，因此这一步转换不能少img = cv2.cvtColor(cv_img, cv2.COLOR_RGB2BGR)# 检测人脸dets = detector(img, 1)print("Number of faces detected: {}".format(len(dets)))for i, d in enumerate(dets):print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(i, d.left(), d.top(), d.right(), d.bottom()))print(time.time() - start)win.clear_overlay()win.set_image(img)win.add_overlay(dets)cap.release()

dlib的人脸检测精度比OpenCV自带的高很多，因此本文采用dlib的人脸检测器。从摄像头读入数据，结合OpenCV将视频流截成图像帧，使用正脸检测器dlib.get_frontal_face_detector()进行检测。

测试效果图：

测试时的输出：

测试速度：
0.09s~0.11s/帧。

2.4.2 mp4文件读入检测时间测试

将2.4.1节的代码中加载摄像头语句更改为加载mp4文件。然后同样将视频截成图像，使用正脸检测器dlib.get_frontal_face_detector()进行检测。
测试效果图：

测试时的输出：

测试速度：
0.09s~0.11s/帧。

注意：视频文件中的人脸检测的速度跟文件的大小（帧高、帧宽）有很大关系。

2.4.3 视频中的人脸识别

分别使用dlib中的人脸识别功能，代码命名为face_recogn_video.py；和dlib二次开发包face_recognition中的人脸识别功能，代码命名为face_recognition_video.py。

face_recogn_video.py代码如下：

import dlib
import numpy as np
import cv2
import json
import os
import glob# 候选人数据集
faces_folder_path = r'../train_person'
video_path = r'../test.mp4'# 获取训练集标签和人脸识别描述子
def train(faces_folder_path):trainlabel = []train_descriptors = []for file in glob.glob(os.path.join(faces_folder_path, "*.jpg")):labelName = file.split('.jpg')[0].split('\\')[1]trainlabel.append(labelName)print("Processing file: {}".format(labelName))face = cv2.imread(file)# 1.人脸检测dets = detector(face, 1)# print("Number of faces detected: {}".format(len(dets)))for k, d in enumerate(dets):# 2.关键点检测shape = predictor(face, d)# 3.描述子提取，128D向量face_descriptor = facerec.compute_face_descriptor(face, shape)# 转换为numpy arrayface_vector = np.array(face_descriptor)train_descriptors.append(face_vector)return trainlabel, train_descriptors# 识别确定哪个人
def findNearestClassForImage(face_descriptor, trainlabel, train_descriptors):train_descriptors = np.array(train_descriptors)dist = np.linalg.norm(face_descriptor - train_descriptors, axis=1, keepdims=True)min_distance = dist.min()print('distance: ', min_distance)if min_distance > threshold:return 'Unknown'index = np.argmin(dist)return trainlabel[index]# 人脸识别
def recognition(img, trainlabel, train_descriptors):# 人脸检测dets = detector(img, 1)for k, d in enumerate(dets):print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(k, d.left(), d.top(), d.right(), d.bottom()))# 人脸关键点检测器shape = predictor(img, d)# 人脸识别描述子face_descriptor = facerec.compute_face_descriptor(img, shape)# 识别确定哪个人class_pre = findNearestClassForImage(face_descriptor, trainlabel, train_descriptors)print(class_pre)cv2.rectangle(img, (d.left(), d.top() + 10), (d.right(), d.bottom()), (0, 255, 0), 2)cv2.putText(img, class_pre, (d.left(), d.top()), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2, cv2.LINE_AA)cv2.imshow('image', img)if  __name__ == '__main__':# 加载网络模型detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor('../shape_predictor_68_face_landmarks.dat')facerec = dlib.face_recognition_model_v1('../dlib_face_recognition_resnet_model_v1.dat')# 设置识别阈值threshold = 0.6# 训练标签及人脸识别描述子trainlabel, train_descriptors = train(faces_folder_path)# cap = cv2.VideoCapture(0)cap = cv2.VideoCapture(video_path)# 保存视频# fps = 10# size = (640, 480)# fourcc = cv2.VideoWriter_fourcc(*'XVID')# videoWriter = cv2.VideoWriter('video.MP4', fourcc, fps, size)while (1):ret, frame = cap.read()# 缩小图像至1/4frame = cv2.resize(frame, (0,0), fx=0.25, fy=0.25)# 人脸识别recognition(frame, trainlabel, train_descriptors)# videoWriter.write(frame)if cv2.waitKey(1) & 0xFF == ord('q'):breakcap.release()videoWriter.release()
cv2.destroyAllWindows()face_recognition_video.py代码如下：
import face_recognition
import cv2
import os
import glob# 视频路径和已知人脸文件夹
video_path = r'../test.mp4'
faces_folder_path = '../train_person'# 读取训练集人脸姓名和人脸识别编码
def train(faces_folder_path):known_face_names = []known_face_encodings = []for file in glob.glob(os.path.join(faces_folder_path, "*.jpg")):labelName = file.split('.jpg')[0].split('\\')[1]known_face_names.append(labelName)image = face_recognition.load_image_file(file)face_encoding = face_recognition.face_encodings(image)[0]known_face_encodings.append(face_encoding)return known_face_names, known_face_encodingsdef recognition(rgb_small_frame, known_face_names, known_face_encodings):# 根据encoding来判断是不是同一个人，是就输出true，不是为flaseface_locations = face_recognition.face_locations(rgb_small_frame)face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)face_names = []for face_encoding in face_encodings:# 默认为unknownmatches = face_recognition.compare_faces(known_face_encodings, face_encoding)name = "Unknown"if True in matches:first_match_index = matches.index(True)name = known_face_names[first_match_index]face_names.append(name)return face_locations, face_namesdef main():face_locations = []face_names = []# 设置显示窗口wnd = 'OpenCV Video'cv2.namedWindow(wnd, flags=0)cv2.resizeWindow(wnd, 1920, 1080)known_face_names, known_face_encodings = train(faces_folder_path)# 读取视频# video_capture = cv2.VideoCapture(0)video_capture = cv2.VideoCapture(video_path)# 隔几帧显示process_this_frame = 0while True:# 读取摄像头画面ret, frame = video_capture.read()# 改变摄像头图像的大小，图像小，所做的计算就少small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)# opencv的图像是BGR格式的，而我们需要是的RGB格式的，因此需要进行一个转换。rgb_small_frame = small_frame[:, :, ::-1]process_this_frame += 1if process_this_frame % 5 == 0:# 位置，姓名face_locations, face_names = recognition(rgb_small_frame, known_face_names, known_face_encodings)# 将捕捉到的人脸显示出来for (top, right, bottom, left), name in zip(face_locations, face_names):# 放大至真实值top *= 4right *= 4bottom *= 4left *= 4# 矩形框cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)#加上标签cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)font = cv2.FONT_HERSHEY_DUPLEXcv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)# 显示cv2.imshow(wnd, frame)# 按Q退出if cv2.waitKey(1) & 0xFF == ord('q'):breakvideo_capture.release()cv2.destroyAllWindows()if  __name__ == '__main__':
main()

测试效果图：

测试结果：
face_recognition中的人脸识别功能比dlib中的人脸识别功能识别速度较快。

2.4.4 优化部分

在face_recognition中的人脸识别功能代码中，加入了两点优化：
1、识别时缩小图像至1/4，显示时扩大至图像原大小；
2、每5帧进行一次人脸识别。
最后达到的人脸识别速度接近实时识别（即接近达到正常播放视频的速度）。

三 python训练自己的模型

用python_examples的示例代码训练自己的模型较简单，为了加快训练，将在linux服务器上运行python_examples的示例代码。关于linux服务器dlib库的安装参考1.3节。此处采用的方法是pip3 install dib。

3.1 数据集标注

3.1.1 imglab简介

imglab是dlib提供用来制作数据集的工具，通过给图片打标签，最后会生成一个xml文件。

3.1.2 imglab使用方法

在dlib官方源码中提供了这个工具，文件路径为：tools/imglab。
使用前要先安装好cmake。

• 使用步骤：

1. 打开cmd
2. 进入tools/imglab目录
3. 新建一个build文件夹，进入build
4. 输入：cmake …
5. 输入：cmake --build . --config Release
6. 进入Release
7. 新建一个image文件夹，将训练集所有图片复制进去
8. 在Release目录下，输入：imglab -c mydataset.xml image，将会创建一个mydataset.xml文件
9. 输入：imglab mydataset.xml

出现imglab标注软件了，可以自己进行标注了。

• 标注方法如下：

11. 按Shift+左键进行画框。先松开左键，框就画上去了；先松开Shift键，则取消画人脸框。
12. 对框双击左键，按delete键可删除。
13. 对框双击左键，按i键可将物体标注为ignore，即是不明物体，进行忽略。
14. 按e键，会曝光图片，效果如下。
    
15. 按Ctrl键加滚轮，可以缩放图片加标签。
16. 双击选中框后，按shift+左键可画关键点。
    
17. 画完人脸框和关键点之后，点filesave保存，然后exit退出，就可以在mydataset.xml文件中看到人脸检测的数据集了。
    

3.2 训练自己的人脸关键点检测器

3.2.1 数据集

使用imglab工具，给训练的图片和测试的图片标注人脸框和关键点（5个关键点：眼睛、鼻子、嘴巴），训练图片7张，测试图片5张。生成标注文件train_landmarks.xml和test_landmarks.xml。目录如下，train、test文件夹中存放训练、测试图片。

3.2.2 训练部分

训练代码参考python_examples/train_shape_predictor.py，如下：

import os
import sys
import glob
import dliboptions = dlib.shape_predictor_training_options()
# Now make the object responsible for training the model.
# This algorithm has a bunch of parameters you can mess with.  The
# documentation for the shape_predictor_trainer explains all of them.
# You should also read Kazemi's paper which explains all the parameters
# in great detail.  However, here I'm just setting three of them
# differently than their default values.  I'm doing this because we
# have a very small dataset.  In particular, setting the oversampling
# to a high amount (300) effectively boosts the training set size, so
# that helps this example.
options.oversampling_amount = 300
# I'm also reducing the capacity of the model by explicitly increasing
# the regularization (making nu smaller) and by using trees with
# smaller depths.
options.nu = 0.05
options.tree_depth = 2
options.be_verbose = True# dlib.train_shape_predictor() does the actual training.  It will save the
# final predictor to predictor.dat.  The input is an XML file that lists the
# images in the training dataset and also contains the positions of the face
# parts.
training_xml_path = ' /home/users/chenzhuo/program/dlib-19-15/python_test/mytest/train_landmarks.xml '
dlib.train_shape_predictor(training_xml_path, "predictor.dat", options)# Now that we have a model we can test it.  dlib.test_shape_predictor()
# measures the average distance between a face landmark output by the
# shape_predictor and where it should be according to the truth data.
print("\nTraining accuracy: {}".format(dlib.test_shape_predictor(training_xml_path, "predictor.dat")))
# The real test is to see how well it does on data it wasn't trained on.  We
# trained it on a very small dataset so the accuracy is not extremely high, but
# it's still doing quite good.  Moreover, if you train it on one of the large
# face landmarking datasets you will obtain state-of-the-art results, as shown
# in the Kazemi paper.
testing_xml_path = ‘/home/users/chenzhuo/program/dlib-19-15/python_test/mytest/test_landmarks.xml’
print("Testing accuracy: {}".format(
dlib.test_shape_predictor(testing_xml_path, "predictor.dat")))

将上述代码命名为shape_predictor_train.py，将代码中training_xml_path改为自己的数据集xml文件路径，进入.py文件所在目录，执行

# python3 shape_predictor_train.py

3.2.3 测试部分

测试代码训练代码参考python_examples/shape_predictor_test.py，如下：

import os
import sys
import glob
import cv2
import dlibif len(sys.argv) != 2:print("Give the path to the examples/faces directory as the argument to this ""program. For example, if you are in the python_examples folder then ""execute this program by running:\n""    ./train_shape_predictor.py ../examples/faces")exit()
faces_folder = sys.argv[1]# Now let's use it as you would in a normal application.  First we will load it
# from disk. We also need to load a face detector to provide the initial
# estimate of the facial location.
predictor = dlib.shape_predictor("predictor.dat")
detector = dlib.get_frontal_face_detector()# Now let's run the detector and shape_predictor over the images in the faces
# folder and display the results.
print("Showing detections and predictions on the images in the faces folder...")
win = dlib.image_window()
for f in glob.glob(os.path.join(faces_folder, "*.jpg")):print("Processing file: {}".format(f))# img = dlib.load_rgb_image(f)img = cv2.imread(f)win.clear_overlay()win.set_image(img)# Ask the detector to find the bounding boxes of each face. The 1 in the# second argument indicates that we should upsample the image 1 time. This# will make everything bigger and allow us to detect more faces.dets = detector(img, 1)print("Number of faces detected: {}".format(len(dets)))for k, d in enumerate(dets):print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(k, d.left(), d.top(), d.right(), d.bottom()))# Get the landmarks/parts for the face in box d.shape = predictor(img, d)print("Part 0: {}, Part 1: {} ...".format(shape.part(0),shape.part(1)))# Draw the face landmarks on the screen.win.add_overlay(shape)win.add_overlay(dets)dlib.hit_enter_to_continue()

将上述代码命名为shape_predictor_test.py，打开VNC客户端，进入.py文件所在目录，执行

# python3 shape_predictor_test.py /home/users/chenzhuo/program/dlib-19-15/examples/faces

3.2.4 优化部分

训练时可以用多姿态的训练数据，比如正脸、左侧脸、右侧脸的标注数据集进行训练。

3.3 训练自己的人脸检测器

3.3.1 数据集

# wget http://dlib.net/files/data/dlib_face_detector_training_data.tar.gz

这是dlib训练使用的数据集，里面有数千张人脸的标注数据集，此处仅使用frontal_faces.xml，如下图。

3.3.2 训练部分

训练代码参考python_examples/train_object_detection.py，如下：

import os
import sys
import glob
import dlib# Now let's do the training.  The train_simple_object_detector() function has a
# bunch of options, all of which come with reasonable default values.  The next
# few lines goes over some of these options.
# 超参数
options = dlib.simple_object_detector_training_options()
# Since faces are left/right symmetric we can tell the trainer to train a
# symmetric detector.  This helps it get the most value out of the training
# data.
# 对称检测器
options.add_left_right_image_flips = True
# The trainer is a kind of support vector machine and therefore has the usual
# SVM C parameter.  In general, a bigger C encourages it to fit the training
# data better but might lead to overfitting.  You must find the best C value
# empirically by checking how well the trained detector works on a test set of
# images you haven't trained on.  Don't just leave the value set at 5.  Try a
# few different C values and see what works best for your data.
options.C = 5
# Tell the code how many CPU cores your computer has for the fastest training.
options.num_threads = 4
options.be_verbose = Truetraining_xml_path = '/home/users/chenzhuo/program/dlib-19-15/python_test/dlib_face_detector_training_data/frontal_faces.xml'
# testing_xml_path = '/home/users/chenzhuo/program/dlib-19-15/python_test/cats/cats_test/cat_test.xml'
# This function does the actual training.  It will save the final detector to
# detector.svm.  The input is an XML file that lists the images in the training
# dataset and also contains the positions of the face boxes.  To create your
# own XML files you can use the imglab tool which can be found in the
# tools/imglab folder.  It is a simple graphical tool for labeling objects in
# images with boxes.  To see how to use it read the tools/imglab/README.txt
# file.  But for this example, we just use the training.xml file included with
# dlib.
dlib.train_simple_object_detector(training_xml_path, "detector.svm", options)# Now that we have a face detector we can test it.  The first statement tests
# it on the training data.  It will print(the precision, recall, and then)
# average precision.
print("")  # Print blank line to create gap from previous output
print("Training accuracy: {}".format(dlib.test_simple_object_detector(training_xml_path, "detector.svm")))
# However, to get an idea if it really worked without overfitting we need to
# run it on images it wasn't trained on.  The next line does this.  Happily, we
# see that the object detector works perfectly on the testing images.
# print("Testing accuracy: {}".format(
#    dlib.test_simple_object_detector(testing_xml_path, "detector.svm")))

将上述代码命名为object_detection_train.py，将代码中training_xml_path改为自己的数据集xml文件路径，进入.py文件所在目录，执行

# python3 object_detection_train.py

3.3.3 测试部分

测试代码训练代码参考python_examples/train_object_detection.py，如下：

import os
import sys
import glob
import dlib
import cv2if len(sys.argv) != 2:print("Give the path to the examples/faces directory as the argument to this ""program. For example, if you are in the python_examples folder then ""execute this program by running:\n""    ./train_object_detector.py ../examples/faces")exit()
faces_folder = sys.argv[1]# Now let's use the detector as you would in a normal application.  First we
# will load it from disk.
detector = dlib.simple_object_detector("detector.svm")# We can look at the HOG filter we learned.  It should look like a face.  Neat!
win_det = dlib.image_window()
win_det.set_image(detector)# Now let's run the detector over the images in the faces folder and display the
# results.
print("Showing detections on the images in the faces folder...")
win = dlib.image_window()
for f in glob.glob(os.path.join(faces_folder, "*.jpg")):print("Processing file: {}".format(f))# img = dlib.load_rgb_image(f)img = cv2.imread(f)dets = detector(img)print("Number of faces detected: {}".format(len(dets)))for k, d in enumerate(dets):print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(k, d.left(), d.top(), d.right(), d.bottom()))win.clear_overlay()win.set_image(img)win.add_overlay(dets)dlib.hit_enter_to_continue()

将上述代码命名为object_detection_test.py，打开VNC客户端，进入.py文件所在目录，执行

# python3 object_detection_test.py /home/users/chenzhuo/program/dlib-19-15/examples/faces 

3.3.4 优化部分

目前训练好的人脸检测器为正脸检测器，对侧脸的检测效果较差。
为了提高人脸检测的准确性，可以训练多个人脸检测器进行人脸预测，比如训练正脸检测器、左侧脸检测器、右侧脸检测器等多个检测器进行组合，使用关键操作如下：

image = dlib.load_rgb_image(faces_folder + '/2008_002506.jpg')
detector1 = dlib.fhog_object_detector("detector.svm")
detector2 = dlib.fhog_object_detector("detector.svm")
detectors = [detector1, detector2][boxes, confidences, detector_idxs] = dlib.fhog_object_detector.run_multiple (detectors, image, upsample_num_times=1, adjust_threshold=0.0)
for i in range(len(boxes)):print("detector {} found box {} with confidence {}.".format(detector_idxs[i], boxes[i], confidences[i]))

3.4 总结

从上面的训练操作流程看，dlib库不仅可以做人脸检测、识别，还可以做其他物体的检测、识别等功能。

四 C++训练自己的模型

4.1 训练自己的人脸关键点检测器

每一个代码的程序配置参见1.2.2节。选择在Release模式下进行项目配置并运行，加快运行速度。

4.1.1 数据集

使用imglab工具，给训练的图片和测试的图片标注人脸框和关键点（5个关键点：眼睛、鼻子、嘴巴），训练图片7张，测试图片5张。生成标注文件train_landmarks.xml和test_landmarks.xml。目录如下，train、test文件夹中存放训练、测试图片。

4.1.2 训练部分

使用examples/train_shape_predictor_ex.cpp代码进行项目配置后，命令参数中输入标注xml文件所在的目录，点击调试-开始执行，进行模型的训练，生成模型文件sp.dat。

load_image_dataset(images_train, face_boxes_train,faces_directory+"\\***.xml");
load_image_dataset(images_test,face_boxes_test, faces_directory+"\\***.xml");

测试误差：

4.1.3 测试

使用examples/face_landmark_detection_ex.cpp代码进行项目配置后，在命令参数中输入生成的模型文件sp.dat的路径和待检测的图片路径，点击调试-开始执行，测试结果如下：

4.1.4 优化部分

训练时可以用多姿态的训练数据，比如正脸、左侧脸、右侧脸的标注数据集进行训练。

4.2 训练自己的人脸检测器

4.2.1 数据集

使用imglab工具，给训练的图片和测试的图片标注人脸框，训练图片7张，测试图片5张。生成标注文件train.xml和test.xml。

4.2.2 训练部分

使用examples/ fhog_object_detector_ex.cpp代码进行项目配置后，在代码里修改以下语句，将自己标注的xml文件名写入代码相应位置中。

load_image_dataset(images_train, face_boxes_train,faces_directory+"\\***.xml");
load_image_dataset(images_test,face_boxes_test, faces_directory+"\\***.xml");

点击调试-开始执行，训练效果图如下，结果会生成face_predictor.svm模型文件：

4.2.3 测试

示例中没提供测试代码，该部分为自写代码，命名为face_object_detection：

/*
人脸检测器测试
*/#include <dlib/svm_threaded.h>
#include <dlib/gui_widgets.h>
#include <dlib/image_processing.h>
#include <dlib/data_io.h>#include <iostream>
#include <fstream>using namespace std;
using namespace dlib;// ----------------------------------------------------------------------------------------int main(int argc, char** argv)
{try{// In this example we are going to train a face detector based on the// small faces dataset in the examples/faces directory.  So the first// thing we do is load that dataset.  This means you need to supply the// path to this faces folder as a command line argument so we will know// where it is.if (argc == 1){cout << "Call this program like this:" << endl;cout << "./face_detector.svm faces/*.jpg" << endl;return 0;}//定义scanner类型，用于扫描图片并提取特征（HOG）typedef scan_fhog_pyramid<pyramid_down<6> > image_scanner_type;// 加载模型object_detector<image_scanner_type> detector;deserialize(argv[1]) >> detector;//显示hogimage_window hogwin(draw_fhog(detector), "Learned fHOG detector");// 显示测试集的人脸检测结果image_window win;// Loop over all the images provided on the command line.for (int i = 2; i < argc; ++i){cout << "processing image " << argv[i] << endl;array2d<rgb_pixel> img;// 读取图片数据load_image(img, argv[i]);// Make the image larger so we can detect small faces.pyramid_up(img);// Now tell the face detector to give us a list of bounding boxes// around all the faces in the image.// 人脸预测std::vector<rectangle> dets = detector(img);cout << "Number of faces detected: " << dets.size() << endl;win.clear_overlay();win.set_image(img);win.add_overlay(dets, rgb_pixel(255, 0, 0));cout << "Hit enter to process the next image..." << endl;cin.get();} }catch (exception& e){cout << "\nexception thrown!" << endl;cout << e.what() << endl;}system("pause");
}

在命令参数中输入生成的模型文件face_predictor.svm的路径和待检测的图片路径，点击调试-开始执行，测试结果如下：

4.2.4 优化部分

目前训练好的人脸检测器为正脸检测器，对侧脸的检测效果较差：

frontal_face_detector detector = get_frontal_face_detector();

训练多个人脸检测器进行人脸预测，比如训练正脸检测器、左侧脸检测器、右侧脸检测器等多个检测器进行组合，使用关键操作如下：

std::vector<object_detector<image_scanner_type> > my_detectors;
my_detectors.push_back(detector);
std::vector<rectangle> dets = evaluate_detectors(my_detectors, image);