Li S.Z., Jain A.K. (eds.) Encyclopedia of Biometrics

Подождите немного. Документ загружается.

Heterogeneous Face Biometrics

STAN Z. LI

Biometrics and Security Research & National

Laboratory of Pattern Recognition Institute of

Automation, Chinese Academy of Sciences, Beijing,

China

Synonyms

Cross-modality face biometrics; Heterogenous face

image matching

Definition

Face images can be captured in different spectral

bands, e.g., Visual (VIS), near infrared (NIR), or

thermal infrared (TIR), or as measurements of 3D

facial shape. These different image types, due to differ-

ent image formation characteristics , are said to be

▶ heterogeneous. More generally, even within the VIS

type, the face images can come from different image

sensors, such as charge coupled device (CCD) and

complementary metal oxide semiconductor (CMOS)

cameras, photo scans, face sketches, under different

illumination conditions, with different image resolu-

tions and different image quality. These are heteroge-

neities in the broad sense. Although heterogeneous

face images of a given person differ by pixel values,

the identity of the face should be classiﬁed as the same.

The processing and matching of these diverse face

images is collectively referred to as

▶ heterogeneous

face biometrics (HFBs).

Introduction

Different types of face biometrics have been developed,

including visual (VIS) (see a survey in [1]), near infra-

red (NIR) [2], thermal infrared (TIR) [3], and 3D [4]

image based. In each case, it is assumed that both the

enrollment and query face images are of the same type.

This assumption results in the homogeneous processing

and matching of face images. Two different image

types are said to be heterogeneous if they have different

image formation characteristics. Although heteroge-

neous face images of a given person may

differ signiﬁcantly by pixel values, the identity of the

face should be classiﬁed as the same notwithstanding

the source, illumination, and quality of the image. The

processing and matching of these diverse face images

is collectively referred to as heterogeneous face

biometrics (HFBs).

HFBs provide engines and systems for matching

across different imaging systems, either in different

spectral bands or modalities. These are considered

HFBs (in the true sense). These involve comparison of

face images between VIS, NIR, TIR, and 3D face

images. Heterogeneities in the same type of imaging

system are, in the broad sense, such as variations dealt

with by the conventional VIS face recognition, includ-

ing photo scans and face sketches. Arguably, the true

sense HFBs are more challenging than the broad sense

HFBs of VIS-VIS, NIR-NIR, and 3D-3D comparisons,

because the former contain more heterogeneous fac-

tors and larger variations than the latter.

Recent developments have led to several proposals

of HFBs that include matching between VIS and face

sketch [5], VIS and NIR [6], and 3D and NIR [7], and

also the reconstruction of the facial shape from an NIR

image [8]. The MBGC (Multiple Biometric Grand

Challenge) tests organized by NIST (National Institute

of Standards and Technology) has devised the scenario

of matching between VIS and NIR face images as one of

its experiments [9], with the purpose of examining the

feasibility of NIR-VIS face matching and determining

how its fusion with VIS-VIS face biometric could im-

prove the overall performance. In the following sec-

tions, the heterogeneities encountered in HFBs are

discussed and related research issues are identiﬁed.

Heterogeneities in Face Recognition

Images used in face recognition are related to facial

shape, skin, and hair. A 3D face image is related to the

shape only. It is captured by a range-measuring system

usually made from a laser range system or stereo vision

system. Represent a range image taken from a view-

point by z(x, y). The pixel values measure the distances

of the sensor to the facial surface points.

In developing face biometric engines and systems

using spectral images, researchers and engineers have

identiﬁed intrinsic and extrinsic factors that affect face

recognition. The

▶ Lambertian law provides an image

formation model, relating a spectral image with the 3D

700

Heterogeneous Face Biometrics

shape of the sensed object, the object surface proper-

ties, and the illumination source:

Iðx; yÞ¼rðx; yÞnðx; yÞs ð1Þ

where I(x, y) is the spectral image, r(x,y)isthealbedo

of the facial surface material at point (x, y) (also a func-

tion of the illumination wa velength), n ¼ (n

, n

)

is the surface normal (a unit row vector) at the 3D

surface point z(x, y), and s¼(s

, s

) is the point

lighting direction (a column vector, with magnitude).

The normal directions n(x, y) may be derived from the

range image z(x, y), but not vice versa.

In developing face biometric engines and systems

using spectral images, researchers and engineers have

identiﬁed intrinsic and extrinsic factors that affect face

recognition. The facial albedo and the surface shape are

intrinsic factors pertinent to the face identity. These

should be the most important information to be used

for face recognition. On the other hand, extrinsic factors

include illumination, facial ware, hairstyle, expression,

and posture. Since they are irrelevant to the identity of

the face, their inﬂuence on face recognition should be

mitigated. Much research and development effort has

been spent to minimize the impact of extrinsic factors,

but the problems still persist and are difﬁcult to solve [1].

Heterogeneous spectral face images have different

albedos, and hence, encode intrinsic factors in different

ways even if extrinsic factors are not accounted for.

Set aside extrinsic factors and focus on the intrinsic

ones. Given a still, frontal face under a ﬁxed illumina-

tion, heterogeneous image formation processes produce

face images of different image conﬁgurations of pixel

values. The pixel values have different properties and

interrelationships across heterogeneous face images.

While the above heterogeneities in HFBs are con-

sidered in the true sense, HFBs in the broad sense deals

with heterogeneities in homogeneous face images cap-

tured under heterogeneous conditions. The VIS type of

face images, for example, can be captured



under different illumination conditions,



by different types of image sensors, such as CCD

and CMOS, or sensor brands,



in different image resolutions,



in different image quality, and



by photo scanning or face sketching,

These cause heterogeneities in image formation and

pixel conﬁguration. Among these, face sketches may

have different image styles and contain more

heterogeneities. Also, heterogeneities due to image

resolutions and different image quality also count.

Quality control by imposing constraints on image ac-

quisition conditions is thus sug gested, for example, in

the ISO/ICAO (International Organization for Stan-

dardization/International Civil Aviation Organization)

standard [10].

Research Issues

Despite the heterogeneities, it is desired to perform

face biometric identiﬁcation and veriﬁcation with

whatever types of face images available. Research pro-

blems in HFBs include the following:



Understanding heterogeneous image formation

models: This provides a physical basis for modeling

properties of heterogeneous face images.



Discovering relationships between heterogeneous

images: Relations or correlations between hetero-

geneous images of faces or sets of features derived

thereafter may be discovered using heterogeneous

image formation models.



Formulating transformation of one ty pe to ano-

ther: With latent correlations discovered, one

could construct a transformation or mapping

from one type to another.



Extracting common features: Discovered latent

correlations could also be used for extracting com-

mon features for characte rizing face identities in

heterogeneous images.



Matching across heterogeneous images: Matching

algorithms should be developed based on extracted

features that associate heterogeneous face image

properties.



Fusion of heterogeneous information: HFBs can take

advantage of heterogeneous information in face

images and fuse them to improve the performance.

Statistical learning can be used to develop algo-

rithms for solving these problems. For example, for

the recovery of face shape from a single NIR face image

[8], for matching between VIS face and face sketch [5],

VIS face and NIR face [6], and between 3D face and

NIR face [7].

HFBs require the extraction of features common

across heterogeneous types of faces so as to create a

common ground for things to be compared. The ex-

traction of common features is the most distinct issue

Heterogeneous Face Biometrics

701

among all in HFBs, while the other issues have been

researched in homogeneous face biometrics, and their

results can be applied herein. Methods differ in how

and where to extract the common features – wheth er it

is done at one end of the two types as in the

▶ analysis-

by-synthesis approach, or somewhere in the middle as

in the

▶ common feature approach.

For example, in [6, 7], common features are

extracted using canonical correlation analysis (CCA)

[?] and are in the middle of the two ends rather than at

a single end. In [5], VIS face images (at one end) are

synthesized from face sketches (the other end) explic-

itly, and the matching is performed using features

extracted from the VIS end of face images. Possible

solutions to HFBs can therefore be categorized under

two classes: common feature based and analysis-

by-synthesis based.

Summary

Heterogeneous face biometrics (HFBs) perform bio-

metric matching across heterogeneous face images.

This article has discussed an analysis of problems

in HFBs, identiﬁed issues therein, and point out re-

search directions. HFBs could be used as a standalone

module for biometric authentication or work as an

added module to improve face recognition w ith

homogeneous face images. HFBs are not only new

directions for face-based biometrics, but also address

the underlying issues in conventional homogeneous

face biometrics in the broad sense of HFBs. Research

and development of HFBs, with investigation into

problems caused by hete rogeneities in homogeneous

face biometrics, may lead to better solu tions.

Related Entries

▶ 3D Based Face Recognition

▶ Face Recognition, Near-Infrared

▶ Face Recognition, Thermal

▶ Hyperspectral and Multispectral Biometrics

▶ Skin Spectroscopy

References

1. Zhao, W., Chellappa, R., Phillips, P., Rosenfeld, A.: ‘‘Face recog-

nition: A literature survey’’. ACM Computing Surveys 399–458

(2003)

2. Li, S.Z., Chu, R., Liao, S., Zhang., L.: Illumination invariant face

recognition using near-infrared images. IEEE Trans. Pattern

Anal. Mach. Intell. 26 (Special issue on Biometrics: Progress

and Directions), 627–639 (2007)

3. Kong, S.G., Heo, J., Abidi, B., Paik, J., Abidi, M.: Recent advances

in visual and infrared face recognition - A review. Comput. Vis.

Image Underst. 97(1), 103–135 (2005)

4. Bowyer, K.W., Chang, Flynn, P.J.: A survey of 3D and multi-

modal 3d+2d face recognition. In: Proceedings of International

Conference on Pattern Recognition, 358–361 (2004)

5. Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circuits

Syst. Video Technol. 14(1), 50–57 (2004)

6. Yi, D., Liu, R., Chu, R., Lei, Z., Li, S.Z.: Face matching between

near infrared and visible light images. In: Proceedings of IAPR

International Conference on Biometric, Seoul, Korea (2007)

7. Yang, W., Yi, D., Lei, Z., Sang, J., Li, S.Z.: 2D-3D face matching

using cca. In: Proceedings of the IEEE International Conference

on Automatic Face and Gesture Recognition, Amsterdam,

The Netherlands (2008)

8. Lei, Z., Bai, Q., He, R., Li, S.Z.: Face shape recovery from a single

image using cca mapping between tensor spaces. In: Proceedings

of IEEE Computer Society Conference on Computer Vision and

Pattern Recognition (2008)

9. NIST: Multiple Biometric Grand Challenge (MBGC). http://

face.nist.gov/mbgc (2008)

10. ANSI/INCITS/ISO/IEC 19794-5: Information Technology -

Biometric Data Interchange Formats - Part 5: Face Image

Data [S] (2007)

Heterogenous Face Image Matching

▶ Heterogeneous Face Biometrics

Hidden Markov Models

JAVI E R HERNANDO

Technical University of Catalonia, Barcelona, Spain

Synonym

HMM

Definition

A Hidden Markov Model is a twofold stochastic pro-

cess composed by a ﬁrst-order Markov chain, which is

702

Heterogenous Face Image Matching

a ﬁnite state machine ruled by state transition prob-

abilities that solely depend on the immediate predeces-

sor, and an associated probabilistic function. In a

regular Markov model, the state is visible to any ob-

server external to the model, whereas in a Hidden

Markov Model (HMM), the state is not observable,

that is, hidden, but each state has associated output

probabilities over the possible observable tokens.

Introduction

Hidden Markov Models (HMMs) [1] were introduced

by L.E. Baum in the late 1960s [2], and since the

mid-1970s [3–5], they have become po pular to model

the statistical variation of the spectral features in

speech recognition research. In the late 1980s, HMMs

were applied to the analysis of DNA and other

biological sequences. Nowadays, they are ubiquitous

in bioinformatics, and in par ticular, in biometrics.

Architecture and Types

An HMM is characterized by the following

components:



A set of N states of a ﬁrst-order Markov chain

S ={S

}, i =1,..., N. Denoting the instants of

time regularly spaced associated with the state tran-

sitions by t =1,2,..., T, the state in time t is

denoted by q



The set of transition probabilities between the

states. Assuming the ﬁrst-order Markov chain con-

dition, they can be represented by the matrix

A={a

}, i, j =1,..., N, where

¼ Pðq

tþ1

¼ S

Þ 1  i; j  N



The initial state probability matrix, which will be

denoted by the vector

¼ Pðq

¼ S

Þ 1  i  N



The output probabilities B ={b

)}, j ¼ 1, ..., N,

where

ðO

Þ¼Pð O

¼ S

Þ j ¼ 1; ...; N;

being the probability distribution corresponding

to the state j, assumed to be independent of time,

and O

the value of the observati on at instant t,

corresponding to the observation sequence O ={O

t =1,..., T.

Therefore, the HMM can be represented as

l ¼ (A, B, p). The architecture of a three state

HMM is illustrated in Fig. 1. When any state is reach-

able from any other state, as it is represented in the

ﬁgure, the model is called ergodic. However, in text-

dependent speaker recognition, as well as in spe ech

recognition, the models are left-to-right, that is, the

states are only reach able from the state itself of lower

index states.

The probability distributions of the output prob-

abilities are discrete in the so-called Discrete Hidden

Markov Models (DHMM) and continuous in the so-

called Continuous Hidden Markov Models (CHMM).

When the original observation data are continuous,

they must be quantized if the DHMM is preferred.

In the case of speech and speaker recognition, where

the observation data are a sequence of acoustic param-

eter vectors, vector quantization techniques were used

when the computational efﬁciency and amount were

crucial. However, nowadays CHMMs have become

most popular in speech and speaker recognition.

In the DHMMs, the output probabilities take

values in a ﬁnite set of symbols called alphabet

V ={v

}, k =1,..., M, M being the alphabet size.

Hence, they can be denoted with the matrix B ={b

(k)},

where

ðkÞ¼Pðv

en t jq

¼ S

Þ;

j ¼ 1; ...; N; k ¼ 1; ...; M:

Hidden Markov Models. Figure 1 Representation

of a Hidden Markov Model.

Hidden Markov Models

703

In the CHMMs, generally, the output probabilities

take values in multidimensional, continuous space,

and they are modeled by parametric multivariate prob-

ability densit y functions. A Gaussian mixture is the

most used [6, 7], because it can approximate any

probability density functio n with an adequate number

of mixtures. In this case,

ðO

Þ¼

m¼1

@ðO

; m

; S

Þ; j ¼ 1; ...; N

that is, a linear combination with weight c

of M ℵ

multivariate Gaussian probability density functions

with mean vector m

and covariance matrix S

In both cases, once the numbe r of states and the

permitted transitions between them (topology) and

the parameter tying between states are predeﬁned,

these three main problems must be solved in order to

use HMMs in real applications.

Evaluation

Given the observation sequence O ={O

}, t =1,..., T

and a model l =(A, B, p), the evaluation pro-

blem consists in efﬁciently computing P(O|l), the

probability of the observation sequence, given the

model. This probability can be used to classify obser-

vation sequences in recognition applications.

The brute force solution for this problem is to

enumerate all possible state sequences and calculate

their score directly. For a ﬁxed state sequence

Q ={q

}, t =1,..., T and assuming independence

between the observations O

, the probability of the

observation sequence O can be written as

PðOjO; lÞ¼

t¼1

PðO

; lÞ

¼ b

ðO

Þb

ðO

Þb

ðOTÞ:

On the other hand, the probability of the state se-

quence Q can be written as

PðQjlÞ¼a

q1q2

q2q3

a

qT1qT

Finally, the joint probability of both the observation

and a ﬁxed state sequences given the model P( O, Q|l)

is simply the product of the terms shown earlier, and

the probability of the observation sequence given the

model can be obtained summing this product over all

possible state sequences

PðOjlÞ¼

q1q2qT

ðO

Þa

q1q2

ðO

Þ

qT1qT

ðO

Þ:

This procedure has a time complexity of o(2TN

and therefore it is not tractable in real application,

even for moderate values of N and T. Fortun ately,

there is an alternative recursive procedure called

Forward–Backward [2] that computes this probability

in an efﬁcient way, which is summarized as follows.

The forward variable be a

(i) is given by

ðiÞ¼PðO

O

; q

¼ S

jlÞ:

It is easy to show that it can be computed in the

following inductive way:

1. Initialization a

ðiÞ¼p

ðO

Þi ¼ 1; ...; N

2. Induction a

tþ1

ðjÞ¼

i¼1

ðiÞa



ðO

tþ1

Þ;t ¼1; ...

T 1;j ¼1;...; N

3. Termination PðOjlÞ¼

i¼1

ðiÞ

Theses computations can be organized in the

observations and state lattice as shown in Fig. 2, and

have a time complexity of o(N

T), which is acceptable

for real application.

Alternatively, the backward variable

ðiÞ¼PðO

tþ1

; O

tþ2

; ...; O

¼ S

; lÞ:

It can be computed in the following inductive way:

1. Initialization b

ðiÞ¼1; i ¼ 1; ...; N

2. Induction (see ure 3) b

ðiÞ¼

j¼1

ðO

tþ1

ðjÞ; t ¼ T  1; T  2; ...; 1 i ¼ 1; ...; N

3. Termination PðOjlÞ¼

i¼1

ðO

Þb

ðiÞ

Hidden Markov Models. Figure 2 Computation lattice of

the Forward algorithm.

704

Hidden Markov Models

Decoding

Given the observation sequence O ={O

}, t =1,..., T

and a model l = (A, B, p), the decoding problem

consists in choosing the opitmal state sequence

Q ={q

}, t ¼ 1, .... T, which best explains the observa-

tions. The solution of this problem permits to

obtain information about the hidden process and is

also a good and efﬁcient approximation of the evalua-

tion problem.

The most popular criterion is to select the state

sequence that maximizes the conditional probability



¼ arg max

fPðQjO; lg¼arg max

fPðQ; OjlÞg:

Again, the brute force method cannot be tackled, even

for moderate values of N and T, and in this case, it is

replaced by the

▶ Viterbi algorithm, which recursively

solves the problem in an efﬁcient way.

The Viterbi algorithm deﬁnes the variables

ðiÞ¼ max

q1;q2;...;qt

fPðq

q

¼ i; O

O

jlÞg

and c

(j) in order to recursively maximize the con-

ditional probability and to retrieve the corresponding

optimal state sequence respectively. Both of them are

updated by using the following procedure:

1. Initialization: d

(i)=p

) i =1,..., N c

(j)=0

2. Recursion: d

ðjÞ¼i ¼ 1; ...; N

max

t1

ðiÞa

Þgb

ðO

Þ;

t ¼ 2; ...; T; j ¼ 1; ...; NctðjÞ¼i ¼ 1 ...N

agr max

t1

ðiÞa

Þg t ¼ 2; ...; T; j ¼ 1; ...; N

3. Termination: P



¼ i ¼ 1 ...N

agr max

t1

ðiÞa

Þg

4. State sequence backtracking: qt



¼ i ¼ 1 ...N

agr max

ðd

ðiÞg q



¼ c

tþ1

ðq



tþ1

Þ t ¼ T 1; T  2; ...; 1

In this case also, the lattice structure implements the

computations in an efﬁcient way (see Fig. 3). It is

worth noting that not all the state transitions are con-

sidered, but only the ones that lead to the maximum

probability.

Estimation

Given the observation sequence O ={O

}, t =1,..., T,

the estimation problem consists in adjusting the para-

meters of the model l =(A , B, p) so as to maximize the

probability of observation of this sequence, given

the model P(O|l). The solution to this problem per-

mits to develop a method to train self-learning

classiﬁers.

This is the most challenging of the three problems.

In fact, given a ﬁnite sequence of observations, it is not

possible to optimally estimate the model parameters.

However, the model parameters can be chosen to lo-

cally maximize the probability P(O|l) by using the

▶ Baum–Welch algorithm [8], which is summarized

in the following section.

For the DHMM case, the variable

ði; jÞ¼Pðq

¼ S

; q

tþ1

¼ S

jO; lÞ¼

Pðq

¼ S

; q

tþ1

¼ S

; OjlÞ

PðOjlÞ

;

which is the probability of being in the state S

at time

t and in state S

at time t + 1, given the model and the

observation sequence. This value can be written in

terms of the forward and backward variables an the

parameter models as

ði; jÞ¼

iðÞa

tþ1

ðÞb

tþ1

jðÞ

i;j¼1

iðÞa

tþ1

ðÞb

tþ1

jðÞ

The variable

ðiÞ¼Pðq

¼ S

jO; lÞ¼

Pðq

¼ S

; OjlÞ

PðOjlÞ

which is the probability of being in the in the state S

time t, given the model and the observation sequence,

and can also be written as

Hidden Markov Models. Figure 3 Computation lattice of

the Viterbi algorithm.

Hidden Markov Models

705

ðiÞ¼

iðÞb

iðÞ

i¼1

iðÞb

iðÞ

The sum of x

(i, j)overt =1,..., T1 can be inter-

preted as the expected value (over time) of the number

of transitions from the state S

to the state S

. Further-

more, the sum of g

(i)overt =1,..., T is the expected

number of times that state S

is visited, the sum of g

(i)

over t =1,..., T1 is the expected number of transi-

tions made from the state S

, and the sum of g

(i)over

t =1,..., T, when the symbol is v

observed, is the

expected number of times that the model generates the

symbol v

in the state S

, Therefore, reasonable re-

estimation formulas for p, A and B are



¼ l

ðiÞ i ¼ 1; ...; N



T1

t¼1

i; jðÞ

T1

t¼1

iðÞ

; i; j ¼ 1; ...; N



ðkÞ¼

t¼1;Ot¼vk

jðÞ

t¼1

jðÞ

; j ¼ 1; ...N; k ¼ 1; ...M

For the CHMMS, the variable g

(j, k) is given by

ðj; kÞ¼

jðÞb

jðÞ

j¼1

jðÞb

jðÞ

@ O

; m

; S



m¼1

@ O

; m

; S



which is the probability of being in the state j at time

t with the kth mixture accounting for O

. Following the

analogous reasoning done for DHMMS, a reasonable

re-estimation formula for B is (formulas for p and A

are the same)



t¼1

j; kðÞ

t¼1

k¼1

j; kðÞ

; j ¼ 1; ...N; k ¼ 1; ...M



t¼1

j; kðÞO

t¼1

k¼1

j; kðÞ

; j ¼ 1; ...; N; k ¼ 1; ...; M



t¼1

ðj; kÞ O

 m



 m



t¼1

k¼1

j; kðÞ

;

j ¼ 1; ...; N ; k ¼ 1; ...; M

where prime denotes vector transpose.

It can be shown that using this re-estimation for-

mulae uniformly converge to Maximum Likelihood

estimation, but they only lead to local maxima and

that, in most applications, the optimization surface is

complex, and hence initialization is an issue.

On the other hand, these formulae can also be

obtained directly by maximizing the Baum auxiliary

function

Qðl; l

Þ¼

PðQ

O; lÞlog PQ

O; l

ðÞ½

and also they can be interpreted as an implementation

of the Expectation-Modiﬁcation algorithm [9].

Finally, it is worth noting that this optimiz ation

problem has been solved by conventional gradient

techniques. This approach permits the use of other

optimization criteria like MMI (Maximal Mutual In-

formation) [10]. Furthermore, less formalized algo-

rithms like corrective training [11] have been applied.

Summary

Hidden Markov Models composed by a hidden Mar-

kov chain and an observable associated probabilistic

function have become ubiquitous in biometric in the

recent years. Once the topology has been designed, the

three conventional procedures, which solve the main

problems for the application of the Hidden Markov

Models in the real world, have been discussed in this

paper: the

▶ Forward-Backward algorithm for the

evaluation of the model, the Viterbi algorithm for

the decoding of the optimal sequence of the states,

and the Baum–Welch algo rithm to adjust the model

parameters to the observations.

Related Entries

▶ Biometric Algorithm

▶ Classiﬁer Design

▶ Gaussian Mixture Models

706

Hidden Markov Models

▶ Machine-Learning

▶ Probability Distribution

References

1. Rabiner, L.R.: A tutorial on hidden markov models and selected

applications in speech recognition. Proc. IEEE 77, 257–286

(1989)

2. Baum, L.E., Egon, J.A.: An inequality with applications to statis-

tical estimation for probabilistic functions of a markov process

and to a model for ecology. Bull. Am. Meteorol. Soc. 73, 360–363

(1967)

3. Baker, J.K.: The DRAGON system – an overview. IEEE Acoust.

Speech Signal Process. Mag. 23, 24–29 (1975)

4. Bakis, R.: Continuous speech recognition via centisecond acous-

tic states. In: Proceedings of the ASA Meeting (1976)

5. Jelinek, F.: Continuous speech recognition by statistical meth-

ods. Proc. IEEE 64, 532–556 (1976)

6. Rabiner, L.R., Juang, B.H.: An introduction to hidden markov

models. IEEE Acoust. Speech Signal Process. Mag. 3, 4–16 (1986)

7. Juang, B.H., Levinson, S.E., Sondhi, M.M.: Maximum

likelihood estimations for multivariate mixture observations

of markov chains. IEEE Trans. Inf. Technol. 32, 307–309 (1986)

8. Baum, L.E.: An inequality and associated maximization tech-

nique in statistical estimation of probabilistic functions of

markov processes. Inequal. 3, 1–8 (1972)

9. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood

from incomplete data via the EM algorithm. J. R. Stat. Soc. 39,

1–38 (1977)

10. Merialdo, B.: Phonetic recognition using hidden markov models

and maximum mutual information training. In: Proceeding of

the International Conference on Acoustics, Speech and Signal

Processing pp. 111–114 (1988)

11. Bahl, Brown, P.F., de Souza, P.V., Mercer, R.L.: Estimating

hidden markov model parameters so as to maximize speech

recognition accuracy. IEEE Tran Speech Audio Processing

1, 77–83 (1993)

Hill-Climbing Attack

Security attacks based on generating artiﬁcial data,

injecting it in the system and after analyzing the out-

put, modify such data, as to improve the output. This

is done recursively till the output is the desired result.

In biometrics this attack can be used to generate a

synthetic sample, by analyzing the matching score

returned by the system.

▶ Tamper-proof Operating System

Histogram Equalization

Histogram equalization is a common technique for

adjusting the pixel intensities of images. In histogram

equalization, a monotonic transformation is applied

to the intensities of the pixels in an image. The

transformations are chosen so that the resulting images

have a standard or speciﬁed histogram. This is some-

times performed to enhance the contrast or reduce

the effect of lighting variation on the appearance of

a scene.

▶ Face Variation

▶ Image Pattern Recognition

▶ Illumination Compensation

▶ Pre-Processing

HMM

▶ Hidden Markov Models

Human Computing

Human computing is the merging of mobile commu-

nications and sensing technologies, with the aim of

enabling a per vasive and unobtrusive intelligence in

the surrounding environment supporting the activities

and interactions of the users in a human-centered

manner. Speciﬁcally, human-centered interfaces sup-

port detection of subtleties of and changes in the user’s

behavior, and initiate inter actions based on this infor-

mation rather than simply responding to the user’s

commands. Technologies like machine analysis of fa-

cial expressions and affective computing are inherent

human-computing technologies.

▶ Facial Expression Recognition

Human Computing

707

Human Dental Atlas

Human dental atlas is a model for describing shape

and relative position s of teeth in a fully developed

adult dentition. The model categorizes a complete set

of 32 teeth into 6 different classes according to tooth

shape and position.

▶ Dental Biometrics

Human Detection and Tracking

JAMES W. DAV IS ,VINAY SHARMA,AMBRISH TYAGI,

ARK KECK

Ohio State University, Columbus, OH, USA

Synonyms

Association; Correspondence; Localization; Pedestrian

detection; Target detection; Video surveillance

Definition

Human detection and tracking are tasks of computer

vision systems for locating and following people in

video imagery. Human detection is the task of locating

all instances of human beings present in an image, and

it has been most widely accomplished by searching all

locations in the image, at all possible scales, and com-

paring a small area at each location with known tem-

plates or patterns of people. Human tracking is the

process of temporally associating the human detec -

tions within a video sequence to generate persistent

paths, or trajectories, of the people. Human detectio n

and tracking are generally considered the ﬁrst two

processes in a video surveillance pipeline, and can

feed into higher-level reasoning modules such as ac-

tion recognition and dynamic scene analysis.

Introduction

In relation to large-scale biometric and video surveil-

lance systems, there is an increasing need for persistent,

autonomous sensing of people in video using compu-

ter vision algorithms. Biometric systems can be

employed in close proximity (using ﬁngerprint or iris

scans) or remotely (employing face or gait patterns) to

identify or conﬁrm a person’s identity. Hence knowing

‘‘when’’ and ‘‘where’’ to attempt a remote biometric

signature can greatly aid the task in terms of reliability

and efﬁciency. Being able to detect and track people

from a distance using video cameras can be used to

reliably cue remote biometric sensors to engage only

when appropriate. Furthermore, the desire for (semi‐ )

autonomous video surveillance necessitates computer

vision systems to detect, track, and analyze the beha-

viors of people in video. For example, a large airport

security system could use compu ter vision systems to

simultaneously monitor multiple video camera feeds

and could present only those camera feeds showing

suspicious activity to security personnel.

Fundamental to the afor ementioned remote bio-

metrics and video surveillance applications is the ability

to automatically detect and track people in video . The

purpose of human detection is to ﬁnd the location of

every person in each video image, while producing as few

false detections as possible. The most common methods

used for detection are template or pattern matching

algorithms. The detections themselves can be used for a

variety of applications, such as person counting or deter-

mining if a person is located in an area where nobody

should be present. Human tracking is the process which

associates the human detections over time to create a

consistent path trajectory for each person. Tracking sys-

tems typically retain some history of the past detections

and use this information along with the current detection

to match and update the trajectories. The output of the

human detection and tracking systems can be further

used in higher-level reasoning modules such as activity

analysis and rec ognition.

Human Detection

Detection, in general, refers to the task of determining

whether or not an instance of a speciﬁc object class

is present in the scene. With the growing emphasis on

biometrics and surveillance, special attention has been

paid to the task of detecting humans in images. Smart

urban surveillance systems are typically designed to

monitor people, both to ensure safety and to recognize

unlawful acts. The success of such sur veillance system s

708

Human Dental Atlas

is critically dependent on their ability to ﬁrst reliably

detect and localize all humans in the scene. The task of

human detection is especially challenging due to the

non‐rigid, articulate nature of the human body.

The typical detection output is a bounding box

surrounding each person in the image (Fig. 1). Most

researchers approach human detection as a pattern

classiﬁcation problem, where the emphasis is on

learning characteristic appearance features of humans

from specially designed training datasets. These data-

sets generally consist of hundreds of labeled images

containing people (positive examples) and a larger

collection of images not containing people (negative

examples). Often the positive examples included in the

datasets are constrained to speciﬁc poses or camera

views such that the varia bility of the human class is

limited to some extent. Within such a training frame-

work, several researchers in human detection focus on

devising the most appropriate set of features that

would encompass the variations exhibited naturally

by humans (e.g., changes in pose, size, appearance,

non‐rigid motion, etc.) and at the same time adequ-

ately differentiate the human class from the rest of the

objects in the scene. Image features such as Haar wave-

lets [1], histograms of oriented gradients [2], and

covariance matrices [3], have shown to provide

promising results.

Research in human detection also focuses on efﬁ-

cient learning algorithms and classiﬁcation techniques

that provide high detection accuracy with low false

positive rates. For example, the use of AdaBoost to eff-

ectively combine hundreds of weak, computationally

inexpensive classiﬁers was introduced for human de-

tection in [4]. In [3] a classiﬁcation scheme w as pro-

posed that takes advantage of the geometry of the

manifold on which the extracted features reside.

Other approaches adopt statistical tools such as ﬁeld

models for modeling human shape and motion [5]

and support vector machines for human classiﬁcation

[2]. Certain approaches to human detection also

make explicit use of the structure and

▶ kinematics

of the human body, utilizing some form of human

body model [6]. Such techniques often detect the

human body by its smaller components (arms,

legs, torso) instead of training a classiﬁer for the

whole-body form.

Human Detection and Tracking. Figure 1 Typical human detection output showing the location and scale of people.

Human Detection and Tracking

709