Face Recognition for Access Control Without Storing Faces
You can change the password in a minute. Your face, never. Why would your company store thousands of them on the same server?
Illustrative scenario based on common operational problems. The company remains anonymous.
The Case
A financial services company installed a continuous face recognition system in its offices and data center to know who enters and moves around. An internal audit found three problems: the system performs differently depending on pose and partial occlusion, it shows bias between demographic groups because of unbalanced training data, and it could be fooled with AI-generated fake videos.
It also stores each employee's face data on a central server and compares whoever arrives against that entire database (1:N identification). If someone attacks that server, the biometrics of the entire workforce are exposed, and a password can be replaced but a face cannot. Data protection laws (such as the GDPR or Mexican personal data law) treat biometric data as sensitive and can impose severe penalties. The exact scope must be validated with compliance.
The architecture decision is between reinforcing the system with infrared and liveness detection, or going back to physical controls. Cards can be lent, lost, and cloned, and they don't verify that the person carrying one is the right person. The goal is verifiable access that checks the person without becoming the custodian of thousands of faces.
This article is technical and is not legal advice. Data protection obligations must be validated with the compliance team.
The Solution Architecture
Figure 1 places the AI component inside a typical financial services company. The gray boxes (PACS, IAM, SIEM, privacy office) are illustrative examples that show where the solution fits, and the blue boxes are the AI architecture. All biometric processing happens on the access terminal, and the central backend only receives decisions. A fairness gate controls when the pilot can scale. This is a technical diagram and not legal advice.
Figure 1. Solution architecture: privacy-first access control at a financial services company (illustrative).
The table summarizes the layers of Figure 1.
Layer | Components | What it does |
Sources | Terminal sensor (RGB + NIR or thermal, signed firmware), employee credentials | Capture the employee and carry their encrypted template |
Integration | Terminal gateway, HR system, enrollment station, consent registry | Enroll with consent and write the template to the credential |
AI on the terminal | Channel integrity, liveness detection, ArcFace, 1:1 verification, discard, rejection with reason, PIN path | Verify the person and discard the image and the vector |
AI central and offline | Access backend (decisions only), decision log, fairness audit, threshold calibration, pilot gate | Record decisions without biometrics and control when to scale |
Company systems (illustrative) | PACS, IAM, SIEM, compliance dashboard, privacy office (DPO) | Open the door, log events, and follow up on compliance |
Today (as-is): RGB camera → continuous 1:N face recognition → central database of facial data → access decision. It fails because there is centralized custody of data that cannot be changed, performance differs by pose, partial occlusion, and demographic group (Grother et al., 2019), with no liveness detection a photo, a screen, or a mask can get through, and with no channel integrity a fake video injected directly into the stream never passes through the sensor.
Proposal (to-be): sensor (visible and infrared) that signs every image → terminal: verifies the signature → liveness detection → ArcFace (Deng et al., 2019) → 1:1 verification against the encrypted template the employee carries on their credential → the image is discarded → only the decision reaches the central system. Without consent, the employee uses a credential plus a PIN.
The as-is architecture is a typical starting point, built from the problem statement and the baselines of the proof of concept (PoC).
Figure 2 details the privacy design. The top strip shows the weaknesses of today's system in red (presentation attacks, injected deepfakes, differences by pose and group, and a central 1:N face database) and the choice the committee faces. The bottom shows the target design: the employee consents and carries an encrypted template on the credential, the terminal verifies 1:1 and discards the image and the vector, and the central system stores only the decision. Physical control stays as the alternative path.
Figure 2. AI architecture: 1:1 verification on the device with privacy by design.
Architecture Decisions
Decision | Why | Alternative discarded | What is given up |
From 1:N identification to 1:1 verification | Eliminates the central database and continuous surveillance | Continuous recognition of the whole workforce | A person isn't "searched for": only whoever carries their credential is verified |
Encrypted template (AES-GCM) on the credential, tied to the employee ID | Biometric data never leaves the device | Templates stored on a central server | The credential and its reissue become part of the system |
Defense in depth: channel integrity, liveness detection, and verification | Infrared doesn't stop a fake video that never passes through the sensor, and no single measure is enough | Liveness detection alone | Hardware with attestation and calibration |
A backend that only stores decisions | Minimizes data: ID, time, method, decision, and reason | Storing the image or the vector for auditing | An event's image cannot be reconstructed |
Consent and an alternative path with a PIN | Voluntary use | Making use mandatory | A path with a weaker identity guarantee to operate |
Threshold calibrated by group and a pilot with a fairness gate | NIST found differences by demographic group in most algorithms (Grother et al., 2019) | Deploying with a single global threshold | Requires data with self-declared attributes and consent |
Why not go back to cards. It is the original problem:
Criterion | Multimodal with the proposed controls | Physical control |
Cybersecurity | Liveness detection and channel integrity, with residual risk from high-quality masks | Credentials can be lent or cloned, and they don't verify the person |
Operational viability | Contactless and auditable access, but requires hardware, calibration, and audits | Simple and cheap, but with queues and credential replacement |
Privacy | High risk if centralized, mitigated with local 1:1 and an encrypted template | No biometric data |
Overall | Recommended, conditional on a pilot with an audit | Kept as an alternative path |
These vectors are not anonymous, they are sensitive biometric data. Express consent, a privacy notice, retention policies, and an impact assessment are required.
Validation with a Proof of Concept (PoC)
The PoC used 30 identities from LFW (a public face dataset) and a simulated thermal channel. 29 were enrolled and one tested the no-consent path. The as-is was not measured: the 1:N system was not built, so there is no baseline of our own. What does exist are the metrics that would need to be collected before deciding (overall and per-group FNMR and FMR, APCER, BPCER, and the accepted injection rate). APCER and BPCER are the metrics of the ISO/IEC 30107-3 standard (International Organization for Standardization [ISO], 2023).
Control | Result |
Impostors accepted (FMR) | 0% (0 of 87 attempts) |
Legitimate users rejected (FNMR) | 1.1% (1 rejection in 87 genuine attempts) |
FNMR at a threshold calibrated for FMR ≤ 1% (0.13 instead of 0.35) | 0% |
Photos and screens accepted (APCER) | 0% (20 attempts each) |
Legitimate users rejected by liveness detection (BPCER) | 0% (0 of 87 attempts) |
Injected images accepted | 0% (0 of 20 attempts) |
High-quality masks accepted | 5% (1 of 20 attempts) |
Enrollment without consent | Rejected, redirected to PIN (correct PIN allowed, wrong PIN denied) |
Data in the central system | Only identifier, time, method, decision, and reason, plus salted PIN hashes for the fallback route |
Risks and Pilot Acceptance Criteria
Proposed criterion (working assumption) | PoC result | Status |
APCER ≤ 1% | 0% on photos and screens, but with only 20 attempts each | [?] |
APCER on masks ≤ 1% | 5% (1 of 20) | [-] |
BPCER ≤ 5% | 0% (0 of 87, below 3.4%) | [+] |
FNMR ratio between groups ≤ 1.5 | Not computable with a single rejection | [?] |
[+] met, [-] not met, [?] cannot be confirmed with this data. The thresholds are working assumptions that must be validated with operations and compliance.
Risks and what to validate in the pilot:
✓ Zero is not a guarantee. The FMR of 0% comes from 87 impostor attempts, which at 95% confidence only shows an FMR below about 3.4%. Photos, screens, and injected frames were each tested 20 times, an upper bound of about 14%. Backing an APCER of 1% or less takes hundreds of attempts per attack type. The calibrated threshold was fit on the same impostor attempts, so its 0% FNMR is optimistic.
✓ The 5% of masks is a small sample. It equals 1 in 20 attempts (a 95% interval from about 0.1% to 25%). The residual risk exists and must be measured with real hardware and masks.
✓ Nothing can be concluded about bias. The single genuine rejection fell in the frontal-pose and normal-lighting subgroups, which is why they show 1.5% and 2.3% while the others show 0%. The ratio of 22.7 between subgroups in the metrics file is an artifact of dividing by a subgroup with zero rejections, so it must not be read as evidence of bias. A real audit requires data with self-declared attributes and consent (El Fadel, 2025).
✓ The sensor signature is simulated with a shared key. In production it needs hardware attestation, and the real thermal channel and key management remain to be validated.
How to Adapt It to Your Company
✓ If your PACS already accepts decisions from an external system, the terminal can send it only the open or deny decision, without sharing images or vectors.
✓ If you already have an IAM, use it for roles and employee status, and let the HR system feed enrollment.
✓ Involve the privacy office from the design stage: consent, privacy notice, retention, and impact assessment.
✓ Start with a voluntary pilot and scale only if FMR and FNMR by group stay within the limits agreed with compliance.
Scope and Out of Scope
In scope | Out of scope |
1:1 verification on the device with ArcFace | A real demographic bias audit |
Encrypted template carried on the credential | A real infrared camera (the thermal channel is simulated) |
Signing every image to detect injection | Hardware attestation |
Liveness detection with synthetic thermal maps | Real high-quality 3D masks |
A central system that only receives decisions | Key management in production |
Consent and an alternative path with a PIN | Impact assessment and privacy notice |
The PoC used public faces (LFW) and a simulated thermal channel, not a real installation. The business case is the framing.
Code
The code of the proof of concept (PoC) is published on GitHub:
https://github.com/rodolfo-casanova/lab-04-face-access-privacy
References
Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). ArcFace: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4690–4699). https://doi.org/10.1109/CVPR.2019.00482
El Fadel, N. (2025). Facial recognition algorithms: A systematic literature review. Journal of Imaging, 11(2), 58. https://doi.org/10.3390/jimaging11020058
Grother, P., Ngan, M., & Hanaoka, K. (2019). Face recognition vendor test (FRVT) Part 3: Demographic effects (NISTIR 8280). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8280
International Organization for Standardization. (2023). Information technology — Biometric presentation attack detection — Part 3: Testing and reporting (ISO/IEC 30107-3:2023). ISO. https://www.iso.org/standard/79520.html
#ComputerVision #Privacy #Cybersecurity #Biometrics #DataProtection
