BS ISO IEC 29197:2015 pdf free download – Information technology – Evaluation methodology forenvironmental influence inbiometric system performance
5.2Scenario evaluations
Scenario evaluations consist of defining one or more specific evaluation conditions as well as a referenceevaluation environment and carrying out a similar scenario test in each of them.In these evaluations,thebiometricsystem andenvironmental parameters will be recorded at the sametime in orderto determineerror rates and throughput rates for the specific evaluation conditions. This approach provides insightsinto the degree to which biometric systems are influenced by one or more environmental parameters.Aschema of this evaluation methodology is shown in Figure 1.
5.3Operational evaluations
Operational evaluations consist of determining a specific environment and selecting one or moreenvironmental parameters to assess. In these evaluations, the relevant biometric system performanceand the selected environmental parameters to assess will be recorded at the same time.This approachprovides insights into the degree to which biometric system performance is influenced by the assessedparameters of the operational environment. Operational evaluations are carried out in an uncontrolled environment. When performing this type ofevaluation, it may be difficult to isolate the impact of environmental parameters on users, algorithms,or equipment; the impact is measured for the system as a whole.
Environmental conditions should reflect those typically encountered when the system is not underevaluation. Thus, an operational test may permit operationally viable practices such as use of a canopyfor outdoor collection of facial images.Therefore,for this evaluation, the operational environmentincluding physical layout as well as the environmental parameters that are going to be assessed shall bedescribed.
ln addition, it will be necessary that evaluators specify which evaluation conditions are of interest. Itwill require conducting a preliminary analysis of the operational environment for knowing the potentialvalues of the environmental parameters.
6.1 Introduction
Biometric systems can be tested in different evaluation conditions in order to analyse whether theirperformance is influenced by environmental conditions or not. This Clause defines requirements fordefining and measuring such evaluation conditions for all potential environmental parameters that canbe tested during this kind of evaluations.
6.3.1 General
The selection of the evaluation conditions entails two activities. Firstly, the parameters to be assessedand/or controlled shall be defined and reported depending on the type of biometric evaluation (e.g.temperature, humidity).This decision is mainly based on the biometric modality of the system undertest as well as the type of technology used by its capture sensor. These parameters shall be selected byparties involved in the evaluation.
NOTE ISO/IEC/TR 19795-3[1] lists environmental factors that can impact performance on a modality-by-modality basis.
Then, the measuring points and set points shall be specified for the selected environmental parametersto assess and/or control respectively. This specification shall consider two aspects: the types ofbiometric performance evaluation (i.e.scenario or operational evaluation) and the different phases ofthe recognition process (i.e.enrolment and verification).
This subclause establishes requirements to select the measuring and set points for both types ofenvironments (i.e. target evaluation environment and reference evaluation environment) consideringalso the aspects above mentioned.
NOTE Since it is mandatory for scenario test methodology to carry out evaluations to obtain the baselineperformance apart from any other evaluation performed in the target evaluation conditions, these values can bespecified for the reference evaluation environment and the target evaluation environments.