FaceEmbeddings Pipeline:

Overview

FaceEmbeddings model is supposed to create embeddings (512-d float32 vector) for a cropped and aligned face. This would provide users with mathematical representation(a vector) which then can be compared with other such embeddings mathematically using something like dot-product. Architecture is based on mobileNetV2 architecture.

Our pipeline handles all the steps starting from detecting a face,detecting corresponding landmarks,followed by aligning the face and then final embeddings (512-d vector). Model handles all the preprocessing and postprocessing calculations so user is expected only to provide rawImage data (from any source),so truly plug&play model.

Features

• Pipeline even after including a number of steps runs in super real-time.
• Work with all versions of python3 (tested with python3 only.)
• Highly portable being a compiled python module.
• Minimal dependencies (all dependencies are standard DLLs.)
• Only expected python dependencies are numpy and opencv(for reading/displaying images purpose).

Limitations

• Works fastest only if a single face is present because for each face detected, embeddings would have to be calculated separately.

Benchmarks (show me the stats!):

• *Time measured in python runtime averaging over 100 loops.
• *Time measured with inputs upto full-HD resolutions with a single face present.
• (All resolutions are resized to fixed input-size specified by model before further processing.)

Usage

import numpy as np
import cv2

import faceEmbeddingsPipeline_python_module as pipeline  #for CPU.
pipeline.load_model("weights.bin")    #load model and initialize weights

#test_1
frame = cv2.imread("./test_1.jpg")   #Uint8 BGR format data.
face_bboxes_1,embeddings_1 = pipeline.detect_embedding(frame,conf_threshold=0.9)
assert face_bboxes_1.shape[0] == 1, "For now single face expected in the specified frame"

#test_2
frame = cv2.imread("./test_2.jpg")   #Uint8 BGR format data.
face_bboxes_2,embeddings_2 = pipeline.detect_embedding(frame,conf_threshold=0.9)
assert faces_bboxes_2.shape[0] == 1, "For now single face expected in the specified frame"   

#calculate a metric for measuring similarty among embeddings, i.e square of EUCLEDIAN NORM.
distance =  np.sum(np.square(embeddings_1 - embeddings_2),axis=1)
print("Empirical Threshold can be taken as 1.32
Calculated Distance: {}".format(distance))

How to install

Once SDK has been downloaded into a local directory. Follow these steps.

• cd into directory. i.e at the root of directory.

• Make sure all the requirements have been fulfilled as stated in requirements.

• On <code>Windows</code> run following command at the root of the directory :
  

  pip install .

  On Linux run following command at the root of the directory :

  pip install .

Functions

def detect_embedding(frame, conf_threshold=0.85, nms_threshold=0.45, max_count=50)

Inputs:

frame: [H,W,3] Uint8 data, BGR data preferred over RGB data, generally resulting from cv2.imread("test.jpg")
conf_threshold:float  confidence threshold to suppress less confident predictions.
nms_threshold:float Used by Non Maximum suppression to suppress bounding boxes based on this threshold.
max_count:int  Number of maximum faces to be detected in a given frame/image.

Returns:

tuple (face_bboxes,embeddings)

where:
    face_bboxes: Numpy array float32 data of shape [num_faces_detected,4] , each row format (x1,y1,x2,y2) aka (left,top,right,bottom) corners.
    embeddings: Numpy array float32 data of shape [num_faces_detected,512] , embedding for each face-detected.

def load_model(weightsPath: str)

Initialize the model and load weights from the path specified by weightsPath argument.

Inputs:

weightsPath:str  /path/to/weights  ,path to file to load weights from ,generally a .bin extension