two layer security system using face recognition and time attendance integration

MISSIONSImplement Anti-Proxy Measures: Utilize embedded computing and artificialintelligence technologies to introduce features that specifically target and prevent instancesof proxy att

INTRODUCTION

PROBLEM STATEMENT

In the contemporary business landscape, the effective tracking of employee attendance is a pivotal concern, with many enterprises, including both small and some larger organizations, facing challenges related to fraudulent practices such as proxy attendance Instances where employees hand over their access cards to colleagues,allowing them to clock out on their behalf and receive full compensation for unworked hours, contribute to financial losses and undermine the integrity of attendance records.Current single-layer security measures, often relying on traditional methods like access cards, are proving insufficient to address these deceptive practices, necessitating the development of a more sophisticated and secure system.

OBJECTIVES

Prevent Proxy Attendance: Develop a two-layer security system incorporating embedded computing and artificial intelligence to effectively prevent deceptive practices like proxy attendance, where employees misuse access cards to manipulate clock-in and clock-out records.

Ensure Individual Accountability: Implement features that discourage the sharing of access cards by providing a personalized identification method for each employee, reducing the potential for fraudulent activities and enhancing individual accountability.

Improve Overall Attendance Integrity: Establish a secure and adaptable attendance tracking system that not only mitigates proxy attendance but also contributes to maintaining accurate and trustworthy attendance records for financial and organizational transparency.

Integrate Web Application: Develop a user-friendly web application that not only facilitates attendance monitoring but also serves as an additional layer of security, allowing users to access the system through a secure credential pin code.

MISSIONS

Implement Anti-Proxy Measures: Utilize embedded computing and artificial intelligence technologies to introduce features that specifically target and prevent instances of proxy attendance, ensuring that only the authorized employee can clock in and out.

Individualized Access Methods: Develop a system that allows employees to choose between access cards and pin codes for the first layer of security, ensuring a personalized and secure identification process that discourages the sharing of access credentials.

Enhance Data Accuracy: Implement advanced artificial intelligence models to accurately verify employee identities through facial recognition, contributing to the overall integrity of attendance data and reducing the risk of fraudulent activities.

Web Application Integration: Incorporate a user-friendly web application accessible through secure credential pin codes, providing employees with a convenient platform for attendance monitoring and reinforcing the multi-layered security approach.

LIMITATIONS

Technology Constraints: The effectiveness of the system may be influenced by the inherent limitations of embedded computing and artificial intelligence technologies, potentially impacting the overall performance and accuracy.

Cost Implications: The adoption of sophisticated technologies, particularly embedded computing and artificial intelligence, may entail significant upfront costs. Smaller enterprises with limited budgets may find it challenging to invest in and maintain such systems.

Maintenance and Upkeep: Ensuring the consistent functionality of the system requires regular maintenance and updates The need for timely software updates, security patches, and hardware maintenance might pose challenges in environments where IT resources are limited.

Scalability Challenges: Adapting the system to accommodate the growing needs of expanding organizations may present scalability challenges As the number of users increases, the system’s ability to maintain performance levels and handle a larger volume of data must be carefully considered.

Environmental Factors: The system’s reliance on hardware components may be susceptible to environmental factors such as temperature fluctuations, humidity, and physical wear and tear Also lighting could effect the accuracy of feature extraction.Ensuring the system’s resilience under varying conditions is essential for sustained reliability.

OUTLINE

The outline of the thesis is divided into five chapters as follows:

 Chapter 1 Introduction: This chapter presents the background research, the purpose, the research tasks, the limitations, and the outline of the report.

 Chapter 2 Literature review: This chapter introduces the theoretical background of the methods used in this study such as MERN Stack, ArcFace, MobileNetv2, and some others foundational theories related to the hardware aspect of the system.

 Chapter 3 System design: This chapter illustrates the steps to finding the solution to the problem and setting requirements for the proposed system The method is described from an overview to details of the algorithms inside.

 Chapter 4 Results: This chapter shows the results of system implementation.

 Chapter 5 Conclusions and Recommendations: This chapter provides conclusions and future work.

LITERATURE REVIEW

MERN STACK

MERN Stack is a JavaScript Stack that is used for easier and faster deployment of full-stack web applications MERN Stack comprises of four technologies namely: MongoDB, Express, React and Node.js It is designed to make the development process smoother and easier [1]

MongoDB, the leading NoSQL database, distinguishes itself as an open-source, document-oriented platform that diverges from traditional relational databases by dispensing with fixed schemas and structured tables Instead, MongoDB employs BSON as its storage format, encoding data in a binary structure that facilitates swift parsing This database excels in scalability and flexibility, offering a document structure that is schema less and adaptable Unlike relational databases, MongoDB operates without the constraints of table formations or relationships, making it particularly efficient for developers Its efficiency is further underscored by streamlined storage and indexing techniques, contributing to faster processing compared to traditional RDBMS While MongoDB lacks complex join operations and does not support intricate transactions, it compensates by allowing scalability through the addition of servers [2]

Express, a server-side JavaScript framework that operates within Node.js, stands out as one of the premier frameworks for backend development Offering developers a powerful platform, Express facilitates the creation and maintenance of resilient servers Its significance is particularly notable in the realm of web and mobile application

Figure 2.1 The MERN Stack work methodology development, where it streamlines the process for building and designing applications with speed and ease Express serves a crucial role in providing server-side logic for a myriad of applications, demonstrating its versatility across various domains Developers leverage Express to effortlessly establish robust APIs and web servers, simplifying the overall development process The framework excels in organizing application functionality through efficient routing and middleware, enhancing the creation of robust web servers.

As a fundamental component of both the MERN and MEAN stacks, Express is instrumental in constructing fast, maintainable, and robust production web applications, cementing its role as a cornerstone in modern web development [3]

React, a highly popular open-source front-end JavaScript library, is widely employed for building web applications, particularly single-page applications Before diving into React, prerequisites include downloading Node packages and possessing a foundational understanding of HTML, CSS, and JavaScript Developed by Facebook, React is not a framework but a library aimed at addressing challenges in web and mobile application development It excels in constructing user interfaces and offers reusability through UI components Originating from the work of software engineer Jordan Walke at Facebook, React made its debut in the Facebook news feed Key attributes of React include its ability to enable the creation of large, dynamic web applications that update data without requiring page reloads [4]

Node.js, an open-source server environment, serves as a cross-platform runtime for executing JavaScript code beyond the confines of a web browser It distinguishes itself as neither a programming language nor a framework but rather as a versatile runtime environment Notably, Node.js finds extensive application in the development of backend services for web and mobile applications, with major corporations like Uber, PayPal, andNetflix leveraging its capabilities in production [5]

ARCFACE AND MOBILENETV2 INTEGRATION

Facial recognition technology has witnessed significant advancements in recent years, driven by breakthroughs in deep learning and neural network architectures Two prominent contributors to this progress are ArcFace, an innovative face recognition algorithm, and MobileNetV2, a lightweight neural network designed for mobile and edge devices Comparing to other model like Facenet512 or Facenet, Arcface archive lower precision compare to the two other models, but recognizable for it speed and portable for edge device like raspberry pi [6]

ArcFace, introduced by Deng et al in the paper “ArcFace: Additive Angular Margin

Loss for Deep Face Recognition” (2019), addresses challenges in face recognition, particularly regarding discriminative feature learning The authors proposed a novel loss function, the additive angular margin loss, which enhances the discriminative power of deep face recognition models ArcFace introduces a margin-based approach that enforces intra-class compactness and inter-class separability, leading to improved face recognition accuracy [7]

The central idea of ArcFace is to map the features of each face to a hypersphere, ensuring that the angular distance between features corresponds to their similarity The algorithm has demonstrated state-of-the-art performance on benchmark face recognition datasets, making it a compelling choice for applications where accuracy and reliability are paramount.

More detail into the ArcFace loss function, the ArcFace loss combines the standard softmax loss with an angular margin term.

The softmax loss is a standard classification loss that measures the dissimilarity between the predicted class scores and the ground truth labels For a given training sample

𝑥 𝑖 with ground truth label𝑦 𝑖 , the softmax loss is calculated asEq 1, where𝑊 𝑗 is the weight vector associated with class𝑗, and𝑁 is the total number of classes.

To introduce the angular margin, the cosine of the angle between the input embedding vector 𝑥 𝑖 and the weight vector𝑊 𝑦 𝑖 (associated with the true class) is considered [6] The angular margin 𝑚 is added to this cosine term as Eq 2, where𝜃 𝑦 𝑖 is the angle between𝑥 𝑖 and 𝑊 𝑦 𝑖 , andm is the angular margin. cos 𝜃 𝑦 𝑖 +𝑚 (𝐸𝑞 2)

The ArcFace loss combines the softmax loss and the angular margin term, and it is formulated as Eq 3:

 𝑥 𝑖 : The embedding vector for the input face image corresponding to the i-th training sample.

 𝑦 𝑖 : The ground truth label for the𝑖 − 𝑡ℎ training sample, representing the true identity of the face.

 𝑊 𝑗 : The weight vector associated with class 𝑗 In the context of face recognition, each class represents a different individual or identity.

 𝑁: The total number of classes or identities in the training dataset.

 𝜃 𝑦 𝑖 : The angle between the embedding vector 𝑥 𝑖 and the weight vector 𝑊 𝑦 𝑖 corresponding to the true class of the𝑖 − 𝑡ℎ 𝑠𝑎𝑚𝑝𝑙𝑒.

 𝑚: The angular margin, a hyperparameter that defines the minimum angular separation between the embeddings of different classes It encourages the model to learn more discriminative features.

 𝑠: The scaling parameter, another hyperparameter that controls the scale of the embeddings Adjusting s can affect the magnitude of the cosine similarities and is often experimentally tuned for optimal performance.

The main steps of training of the pre-trained weight follows:

 Normalization: Normalize the weights and feature vectors.

 Cosine Similarity: Calculatecos𝜃 for each class𝑗 using the normalized feature vectors and weights of class𝑗.

 Calculate Angle 𝜃: Compute 𝜃 𝑗 – cos −1 (𝜃 𝑗 ), the angle between the actual weight 𝑊 𝑦 𝑖 and the feature vector𝑥 𝑖

 Calculate Loss Function: Use the formula𝑠cos(𝜃 𝑦 𝑖 +𝑚) within the range[0,𝜋], where

 Softmax and Probability: Apply the softmax function to obtain the probability distribution of the labels.

 Cross Entropy Loss: Compare the ground truth vector (one-hot labels) with the predicted probability and compute the cross entropy loss.

In the realm of image processing and facial recognition, employing cosine similarity among facial feature representation vectors is a robust method for measuring similarity. Thus, throughout the execution of this project, cosine similarity is utilized to compute the similarity between facial feature vectors The formula for calculating cosine similarity is illustrated inEp 4.

MobileNetV2, introduced by Sandler et al in “MobileNetV2: Inverted Residuals and

Linear Bottlenecks” (2018), focuses on addressing the computational challenges associated with deploying deep neural networks on mobile and edge devices MobileNetV2 builds upon the success of its predecessor, MobileNetV1, by introducing inverted residuals and linear bottlenecks These architectural enhancements contribute to increased efficiency, allowing for faster inference while maintaining competitive accuracy.

Figure 2.2 Training flow supervised by ArcFace loss

Figure 2.3 The difference between residual block (a) and inverted residual block (b)

MobileNetV2 employs inverted residuals, a novel design using lightweight depthwise separable convolutions This design choice significantly reduces the computational cost by separating spatial filtering (depthwise convolution) from pointwise mixing (1 × 1 convolution), making the model well-suited for resource-constrained devices.

Linear bottlenecks are integrated into each inverted residual block to enhance information flow within the network The1 × 1 convolution in the linear bottleneck serves to increase the non-linearity of the network, promoting feature reuse and mitigating the risk of information loss This contributes to the overall efficiency of the model.

The combination of inverted residuals and linear bottlenecks allows MobileNetV2 to strike a balance between computational efficiency and model accuracy This is particularly advantageous for applications on devices with constrained resources, where efficient feature extraction is crucial.

Let𝑥 𝑖 represent the input feature vector to an inverted residual block, and𝐹 denotes the combination of operations within the block The output of the block, denoted as𝑦 𝑖 , can be expressed as Eq 4, where 𝐹(𝑥 𝑖 ) is the operations involving lightweight depthwise separable convolutions and linear bottlenecks.

𝑦 𝑖 =𝑆𝑘𝑖𝑝_𝐶𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛 𝐹(𝑥 𝑖 ) + 𝑥 𝑖 (𝐸𝑞 5) MobileNetV2 achieves a harmonious balance between computational efficiency and model accuracy, making it an ideal choice for applications with limited resources.

This architecture, with its emphasis on efficiency and feature reuse, has proven to be effective for various computer vision tasks on devices with restricted computational capabilities [8]

Integration in Facial Recognition Systems:

The combination of ArcFace and MobileNetV2 in facial recognition systems presents a compelling synergy ArcFace provides a robust solution for discriminative feature learning, enhancing the accuracy of facial recognition When coupled with MobileNetV2, the overall system gains computational efficiency, making it suitable for deployment on resource-constrained devices such as mobile phones and edge devices.

Recent research and applications have explored this integration, showcasing its potential in real-world scenarios By leveraging the strengths of both ArcFace and MobileNetV2, facial recognition systems can achieve a balance between accuracy, efficiency, and suitability for diverse deployment environments.

In conclusion, the literature surrounding ArcFace and MobileNetV2 highlights their individual contributions to face recognition and mobile deployment, respectively As researchers and practitioners continue to explore novel architectures and optimizations, the integration of these technologies remains a promising avenue for advancing the capabilities of facial recognition systems in various domains.

RASPBERRY PI 4 MODEL B

Raspberry Pi is a line of small, affordable computers developed by the Raspberry Pi Foundation in the UK Introduced in 2012, Raspberry Pi was initially aimed at promoting programming and IT education However, it has evolved into a versatile and powerful tool across various fields, from IoT to robotics and embedded systems.

The Raspberry Pi 4 Model B offers a significant performance boost, powered by the Broadcom BCM2711 quad-core ARM Cortex-A72 64-bit processor running at1.5𝐺𝐻𝑧 Its expandable RAM options of 2𝐺𝐵,4𝐺𝐵, and8𝐺𝐵 LPDDR4-3200 SDRAM cater to varied project needs.

Figure 2.4 Map of Raspberry Pi 4 Model B

Equipped with Gigabit Ethernet, USB 3.0, USB 2.0, and Micro HDMI for 4K video output, it ensures fast data transfer and diverse connectivity Wi-Fi 802.11ac and Bluetooth 5.0 enable seamless network connections and remote control.

This model’s advanced cooling system ensures stable operation even under heavy loads In summary, the Raspberry Pi 4 Model B’s versatility and robust performance make it an ideal choice for a wide array of projects, from IoT applications to custom PCs, fostering innovation and experimentation.

The GPIO pins on the Raspberry Pi 4 Model B play a vital role in interacting with peripherals and controlling various functions With a total of 40 GPIO pins, they offer high flexibility for users to create and manage electronic projects.

Each GPIO pin can be configured to perform several functions:

 Digital Input Function: These pins can receive data from peripherals like motion sensors, buttons, or other signal sources They are capable of reading digital signals, receiving values of 0 (LOW) or 1 (HIGH) corresponding to the connected device’s state.

 Digital Output Function: These pins can also control devices such as LEDs, relays, or motors by providing electrical signals By adjusting the signal value (0 or 1), users can manage the operational status of these devices.

 Analog Input Function: Some GPIO pins support reading analog values from sensors with analog outputs This capability allows the Raspberry Pi to collect more detailed

Figure 2.5 GPIO of Raspberry Pi 4 Model B data about signal levels, suitable for applications demanding high measurement accuracy.

 Communication Protocol: GPIO pins also support communication protocols like I2C,

SPI, UART This opens up possibilities for connecting to other devices such as sensors, displays, or expansion boards to extend the functionality of the Raspberry Pi.

Wireless Connectivity and Gigabit Ethernet:

The Raspberry Pi 4 Model B brings Wi-Fi 802.11ac and Bluetooth 5.0 for wireless connectivity, enabling remote access, data transfer, and device communication without physical cables This enhances flexibility for mobility and remote control in diverse applications, connecting easily to existing wireless networks and a wide array of peripherals via Bluetooth 5.0.

Additionally, it features a Gigabit Ethernet port, offering high-speed and stable wired connections, advantageous for data-intensive tasks, media streaming, or applications requiring low latency This combination empowers users to select between wired or wireless networking, catering to specific project needs and enhancing adaptability across varied applications.

The Raspberry Pi 4 Model B offers versatile display options through its HDMI output and DSI Featuring dual Micro HDMI ports supporting 4K video output, it connects seamlessly to monitors or TVs, delivering high-quality visuals and enabling touchscreen interactions Additionally, its DSI supports various display types, from non-touch LCDs to high-resolution screens, expanding possibilities for graphics, data visualization, and information display in diverse projects.

This integration of HDMI output and DSI opens avenues for creativity, allowing users to connect touch-enabled LCD displays for interactive applications and leverage diverse display options for projects across IoT, robotics, and embedded systems.

The USB ports on the Raspberry Pi 4 Model B offer not only versatile peripheral connectivity but also enable various practical applications, including the use of USB- connected cameras.

The Raspberry Pi 4 Model B features a total of 4 USB 2.0 ports and 2 USB 3.0 ports. These ports facilitate connections to a wide range of peripheral devices such as keyboards, mice, hard drives, or USB storage devices for convenient data access and sharing The USB 3.0 ports provide faster data transfer speeds compared to USB 2.0, suitable for handling large datasets and transmitting high-resolution videos.

Raspberry Pi supports multiple types of USB-connected cameras By connecting aUSB camera, users can harness the power of the Pi for imaging and video projects This opens up applications ranging from security surveillance to capturing imagery for virtual reality or artificial intelligence projects For those interested in experimenting with image- based projects, connecting a USB camera to the Raspberry Pi 4 Model B via the USB ports offers a flexible and convenient option.

RADIO FREQUENCY IDENTIFICATION

RFID technology harnesses radio waves to retrieve data from electronic tags, finding wide applications across industries like inventory management, logistics, and access control Its swift and accurate identification abilities streamline operations, from enhancing retail inventory to optimizing logistics and improving safety in healthcare RFID’s adaptability with various tag types underscores its scalability and efficiency in diverse operational settings This technology’s non-contact data retrieval has revolutionized efficiency and automation across industries.

An RFID system comprises essential components:

 Tags: Carriers of unique identification data, ranging from passive tags relying on reader energy to active tags with their own power source for longer-range communication.

 Readers: Transmit and receive signals to communicate with tags, capturing data for processing in computer systems.

 Antennas: Integral for communication between readers and tags, emitting and receiving radio waves to determine communication range and effectiveness.

 Middleware: Software that filters and manages data collected by readers before transmitting it to central databases or software applications.

 Software and Database: Applications managing collected RFID data, integrating it with existing business processes.

 Network Infrastructure: Essential for seamless communication and data transfer between RFID system components, ensuring efficient functionality across industries.

Communication Protocols of RFID Modules:

RFID modules rely on distinct connection protocols, categorized as passive or active. Passive RFID commonly incorporates simple protocols like UART, SPI, or occasionally I2C In contrast, active RFID primarily employs wireless protocols such as Bluetooth or Wi-Fi for transmitting data over extended distances.

SPI holds prominence in passive RFID systems due to its efficiency and simplicity in facilitating data exchange between the reader and tag Operating in a master-slave setup, SPI utilizes key communication lines: MOSI, MISO, SCK, and SS, providing control and data transfer between devices.

The directness and rapid data transmission capabilities of SPI position it as a preferred protocol for RFID systems, particularly in scenarios necessitating swift and straightforward communication between RFID tags and readers.

RTC INTEGRATED CIRCUIT

Overview of RTC Integrated Circuit:

An RTC module is an essential electronic component within systems, maintaining and tracking real-time clock and calendar functions It consists of several key components:

 RTC Chip: The core component, responsible for tracking time in various units and managing leap years and shorter months.

 Backup Power Pin: This offers uninterrupted timekeeping during power interruptions through a secondary power source like a lithium battery.

 Interface Connections: Incorporates interfaces (like I2C, SPI, or UART) to enable data transmission with other system components.

 Crystal Oscillator Resistor: Ensures high accuracy in time measurement by generating a stable signal.

 Control Logic Circuitry: Manages time setting, data exchange, and backup power supply oversight.

RTC modules are utilized in devices requiring precise timekeeping, such as digital clocks, real-time measurement systems, IoT, and embedded systems Their stability and capability to maintain timekeeping during power interruptions make them highly advantageous.

Communication Protocols of RTC Modules:

RTC modules utilize different protocols to connect with microcontrollers or circuits for managing real-time data Commonly, the I2C protocol enables seamless data exchange

Figure 2.8 I2C communication protocol through dedicated SDA and SCL lines Some RTC modules favor the SPI protocol, utilizing specific pins like MOSI, MISO, SCK, and SS for transmission Additionally, certain modules employ the UART protocol, often using TX and RX pins for communication.

The selection of protocol for RTC modules relies on design and application requirements, prioritizing compatibility and flexibility with connected devices I2C, popular among these modules, offers a flexible communication method with minimal pins. Its two-pin system enables multiple devices on the same communication bus, with SDA handling data transmission and SCL managing data synchronization.

Operating on a master/slave mechanism, the I2C protocol allows the microcontroller to direct communication as the master, while the RTC device functions as the slave,responding to read or write requests for real-time data Its flexibility in connecting multiple devices on a single bus makes I2C widely preferred for interfacing microcontrollers withRTC modules in digital clocks, real-time measurement devices, and diverse embedded applications.

SYSTEM DESIGN

REQUIREMENTS AND BLOCK DIAGRAM OF SYSTEM

To meet the design requirements of a dual-layer attendance system using RFID, PIN codes, and facial recognition, it needs to satisfy specifications for both hardware (Raspberry Pi 4 Model B with peripherals) and software (website, server).

 Role Limitation: The system is dedicated solely to managing user attendance and does not encompass administrative functions meant for administrators This segregation ensures that the system’s functionalities remain focused on attendance tracking and user- related operations.

 User-Friendly GUI: The GUI of the system should prioritize user-friendliness It must be intuitive, simple to navigate, and visually clear It should present accurate information regarding attendance, and notifications seamlessly The interface should facilitate easy interaction and understanding for users engaging with the attendance system.

 Precise Input Processing: The system must possess the capability to accurately capture and process various input methods, including RFID, PIN codes, and facial recognition. Each method of input needs to be reliably interpreted and authenticated to ensure precise attendance records for users.

 API Data Handling: The system should have the functionality to retrieve data from an

API and transmit data back to the API This communication should occur over Wi-Fi or the Internet, following a predetermined schedule This feature allows seamless integration with external systems or databases, enabling data exchange for comprehensive attendance management and record-keeping.

 Responsive Design: This requirement involves structuring the web application’s layout and functionality to adapt seamlessly to various screen sizes and devices It ensures that whether accessed from a desktop, tablet, or mobile phone, users experience consistent and optimal usability without sacrificing functionality or clarity.

 User-Friendly Interface: The focus is on crafting an interface that is intuitive, easy to comprehend, and navigable It involves employing UI components that facilitate a smooth user experience, considering factors like button placement, layout simplicity, and clear information presentation to enhance user engagement and understanding.

 Integration of Authentication and User Management: This includes incorporating multiple authentication methods like RFID, PIN codes, and facial recognition into the web interface for user access Additionally, the system should provide administrators with tools to manage user accounts effectively, allowing functionalities such as adding, editing, and deleting user profiles Moreover, it involves designing interfaces to view attendance records and providing straightforward login mechanisms for users.

 Communication with API: The system will establish connections with an external API to fetch attendance data and send it securely to the server This involves setting up secure data transmission protocols to ensure that data integrity and confidentiality are maintained during interactions with the API.

 User Data Security: Security measures will be implemented to protect sensitive user information Techniques such as data encryption will safeguard user data against unauthorized access or breaches Access rights will be strictly regulated, and user authentication mechanisms will be put in place to ensure that only authorized individuals can access, modify, or view specific data, thereby upholding data integrity and ensuring user privacy and security.

As depicted inFigure 3.1, the project’s system is designed into 9 main blocks:

The central processing unit of the system uses Raspberry Pi 4 Model B, playing a pivotal role in receiving and processing data from various sources such as RFID, PIN input, and information from facial recognition sent by RFID, TOUCH SCREEN, and CAMERA blocks Its crucial task involves processing the data according to pre-programmed control sequences, subsequently sending control signals to blocks including TOUCH SCREEN block, NOTIFICATION block, and CONTROL LOCK block Simultaneously, the central processing unit is responsible for communicating with an API, transmitting and receiving data from it, as well as storing the information sent down from the API Its most crucial role lies in establishing and maintaining coherent interaction between the various components within the system, ensuring smooth and efficient operations of the entire system Furthermore, this block also serves as the power source provider for the REAL- TIME, RFID, CAMERA, and TOUCH SCREEN blocks.

Utilizing the DS1307 RTC module, it serves as a clock providing real-time information to the CENTRAL PROCESSING block in cases where there’s no internet

Figure 3.1 Block diagram connection available upon booting up This block also incorporates a dedicated power supply to mitigate power loss scenarios that could otherwise hinder the CENTRAL PROCESSING block from providing power.

Utilizing the RC522 RFID module to receive analog signals from RFID cards/tags, subsequently converting them into digital signals and transmitting these signals to the CENTRAL PROCESSING block.

Utilizing the Xiaomi Xiaovv XVV-6320S-USB 1080P webcam to receive analog signals from user faces, subsequently converting them into digital signals and sending them to the CENTRAL PROCESSING block, becoming the input data for the facial extraction and recognition process.

Utilizing a 7-inch1024 × 600𝑝𝑥 touchscreen LCD display to showcase the GUI and essential information from the CENTRAL PROCESSING block, simultaneously receiving and transmitting touch signals from user interactions to the CENTRAL PROCESSING block for processing.

It functions by receiving control signals from the CENTRAL PROCESSING block and displaying visual notifications for users through the use of light from LEDs and sound from buzzers.

Its function is receiving control signals from the CENTRAL PROCESSING block to lock or unlock according to pre-programmed sequences This block is essentially a relay module that receives signals to command locking/unlocking actions from the CENTRAL PROCESSING block.

Comprising two sections, a12𝑉𝐷𝐶/2𝐴 adapter power supply for the load - normally closed electromagnetic latch lock, and a5𝑉𝐷𝐶/3𝐴 adapter power supply for the Raspberry

HARDWARE

The system requires a processing and control unit powerful enough to handle signal processing from peripheral devices, provide control signals based on programmed sequences, store databases, and have internet connectivity for real-time data retrieval from an API and sending data to the API Additionally, the system needs image processing capabilities, including reading input image data, processing it for facial feature extraction models, and running a suitable facial feature extraction model to meet real-time requirements Therefore, the system requires an embedded computer with sufficient memory and high processing speed to fulfill the functions and requirements specified for the project.

In addition to memory and processing speed requirements, the project calls for the embedded computer to have specific connectivity ports These include an Ethernet port for controlling and programming the embedded computer from a PC or laptop, an HDMI port for a touchscreen LCD display, USB ports to power the touchscreen LCD and a USB camera, and to establish a connection between the USB camera and the embedded computer Additionally, the embedded computer must feature GPIO ports that support theI2C and SPI communication protocols.

The Raspberry Pi 4 Model B 4𝐺𝐵 not only meets the previously mentioned requirements but also demonstrates advantages in energy efficiency during prolonged operation, compact size, ease of accessibility for handling, and cost-effectiveness. Considering these advantages alongside the project requirements, the authors have decided to choose the Raspberry Pi 4 Model B4𝐺𝐵 as the CENTRAL PROCESSING block. The CENTRAL PROCESSING block, driven by the Raspberry Pi 4 Model B4𝐺𝐵, serves as the nerve center of the system This core unit facilitates seamless communication and data processing, harnessing the Raspberry Pi’s versatile connectivity options With Ethernet for remote access and programming, HDMI for interfacing with a touchscreen display, USB ports catering to the power needs of peripherals like the touchscreen LCD and USB camera, and GPIO pins accommodating protocols such as I2C and SPI, this block intricately weaves together various connections essential for cohesive system operation It embodies a powerful yet accessible hub, ensuring efficient processing, control, and interaction among the system’s components.

The RTC DS1307 module fulfills critical requirements: accurate timekeeping, seamless interfacing with the Raspberry Pi, low power consumption, and affordability Its swift connectivity, coupled with precise timekeeping capabilities even under minimal power usage, makes it an ideal choice Additionally, its backup power support ensures uninterrupted functionality when disconnected from the Raspberry Pi All these features are packaged within a compact and cost-effective design, highlighting the DS1307 as an optimal solution for the task at hand.

Table 3.1 Connect pins between Raspberry Pi 4 and DS1307

For the RFID block, similar requirements to those of the REAL TIME block include convenient interfacing, communication, and programming with the Raspberry Pi, low

DS1307 SCL SDA VCC GND

RASPBERRY 5 3 1 9 power consumption, as well as a small form factor and reasonable cost However, a distinctive requirement for the RFID block is the need for fast signal reception, processing, and transmission to the Raspberry Pi While various RFID modules exist, the RC522 module closely aligns with these specifications, making it the choice for the RFID block.

Table 3.2 Connect pins between Raspberry Pi 4 and RC522

Recognizing and extracting facial features are crucial aspects of the system, relatively unaffected by recording devices Hence, the choice of a camera with high resolution and the ability to capture stable, high-quality color images is vital Ensuring consistent brightness is essential for the CAMERA block to acquire and process images accurately and reliably, safeguarding image quality from external factors.

The Xiaomi Xiaovv XVV-6320S-USB 1080P webcam, boasting Full HD 1080P resolution, enables crisp, clear, and color-accurate image capture Equipped with a USB- 2.0 connection, it allows for easy connectivity and programming with the Raspberry Pi. Given its advantages and alignment with the outlined requirements, the Xiaomi Xiaovv XVV-6320S-USB 1080P webcam stands as a fitting choice for the CAMERA block of the system.

The TOUCH SCREEN block requires the ability to display crucial system information clearly and intuitively, with a highly responsive touch interface for real-time responsiveness Additionally, it needs to seamlessly communicate and program with the Raspberry Pi, consume low power, and be reasonably sized and priced.

Based on these requirements, a 7-inch capacitive touch screen has been chosen The 7-inch HDMI capacitive touch LCD screen with IPS panel ensures wide and clear viewing angles, providing real-time touch responsiveness Moreover, this screen exhibits excellent compatibility with the Raspberry Pi, operating reliably while conserving energy.

RC522 SS SCK MOSI MISO GND RST VCC

NOTIFICATION and LOCK CONTROL block:

The NOTIFICATION block consists of two identical parts comprising a LED, a 5𝑉𝐷𝐶 buzzer, a C1815 transistor, and 3 resistors The distinction between the two segments, TH1 and TH2, lies in the color of the LEDs – TH1 utilizes a red LED, whereas TH2 employs a green LED Each LED is triggered by a high-level signal of 3.3𝑉𝐷𝐶 from the Raspberry Pi and limited by a 100Ω resistor, drawing a current of approximately 13− 15𝑚𝐴 Each 5V buzzer is powered by the Raspberry Pi’s5𝑉𝐷𝐶 supply and amplified by a C1815 transistor, triggered by a 3.3𝑉𝐷𝐶 high-level signal from the Raspberry Pi With this design, the current consumption of a 5𝑉𝐷𝐶 buzzer is around 21.7𝑚𝐴, and the base current of a C1815 transistor is about0.87𝑚𝐴.

The LOCK CONTROL block also comprises two segments, TH3 and TH4 TH3 consists of a simple LED circuit similar to TH1 and TH2, comprising a red LED limited by a 100Ω resistor, triggered by a3.3𝑉𝐷𝐶 high-level signal On the other hand, TH4 is a

Figure 3.2 Schematic circuit of NOTIFICATION block

Figure 3.3 Schematic circuit of LOCK CONTROL block relay control circuit utilizing a 5𝑉𝐷𝐶 relay, a diode in parallel with the relay for C1815 transistor protection, and the C1815 transistor amplifying the relay drive current. Additionally, TH4 includes a green LED in parallel with the relay, connected through a 220Ω resistor from the 5𝑉𝐷𝐶 source The purpose of the LED in TH4 segment is to provide a visual signal to users when triggering the relay, equivalent to toggling the lock. According to theoretical calculations based on the design, the LOCK CONTROL block is estimated to consume a current of approximately 95.6𝑚𝐴 from the Raspberry Pi.

Table 3.3 illustrates the current consumption of the system hardware, including the

Raspberry Pi 4 Model B and peripheral devices In a scenario where both the Raspberry Pi

4 Model B and peripheral devices consume maximum power simultaneously, the total maximum current draw reaches approximately 2484𝑚𝐴 Among this, the Raspberry Pi 4 Model B’s maximum current draw is 1300𝑚𝐴, while the maximum current draw of peripheral devices is around 1184𝑚𝐴, nearing the 1200𝑚𝐴 limit of peripheral devices’ current draw However, it’s essential to note that the peripheral devices in the system do not consume their maximum power simultaneously, thus staying within the limit of peripheral current draw.

Additionally, the system includes a normally closed electromagnetic latch lock with a maximum current draw of approximately 1000𝑚𝐴, hence powered by a adapter with output specification of 12𝑉𝐷𝐶/2𝐴.

Table 3.3 Typical current and typycal voltage of hardware

DEVICES QUANTITY TYPICAL CURRENT TYPICAL VOLATGE

Table 3.4 illustrates the hardware components and their quantities used in the system.

3 Xiaomi Xiaovv XVV-6320S-USB 1080P webcam 1

According to Table 3.4, hardware components numbered from 7 to 22 will be soldered onto an electronic circuit with the schematic circuit illustrated inFigure 3.4, the printed circuit board depicted inFigure 3.5, and the implemented circuit board showcased in Figure 3.6.

According to the schematic circuit of the hardware in Figure 3.4, the headers serve the purpose of connecting peripheral devices to the Raspberry Pi 4 Model B using male- to-male breadboard cables, except for the RTC DS1307 module, which connects to the implemented circuit board using a 4-pin straight female header The headers, labeled from

Figure 3.4 Schematic circuit of hardware

Figure 3.5 Printed circuit board of hardware

CONTROL PROGRAMS

3.3.1.1 GET DATA AND SEND DATA PROGRAMS

The system hardware is controlled by three programs: GET DATA, SEND DATA, and ATTENDANCE The GET DATA program is responsible for sending requests, fetching and processing data downloaded from the API, then establishing the database from the processed data Next, the SEND DATA program runs within a specific timeframe, tasked with sending user login timestamp data stored in the system in cases of no internet connection or connectivity to the API Lastly, ATTENDANCE, the primary program of the hardware, receives and processes data from user interactions and communication with the hardware.

Figure 3.6 Implemented circuit board of hardware

 The program checks the connection to the Google website ‘google.com’ using the ‘ping’ command.

 If a connection is established, the program proceeds with further operations.

 If there’s no connection, it waits for 5 minutes ‘sleep(300)’ and then restarts the program from the beginning.

 Using the ‘requests’ library, the program sends a GET request to an API specified by the ‘url’ and authentication token.

 If the response code is ‘200’, it extracts JSON data from the response This data contains

‘user_id’ and ‘rfid_data’ of users.

 The API data is saved into a CSV file named ‘id.csv’ Additionally, it creates separate CSV files in the ‘rfid’ directory, each file named after the ‘rfid_data’ present in the

Figure 3.7 Flowchart of GET DATA program

‘id.csv’ file If a file ‘rfid/{rfid_data}.csv’ already exists, it remains unchanged (no editing, overwriting, or creation).

 Files and directories named after ‘user_id’ and ‘rfid_data’ not present in the new data saved in the ‘id.csv’ file will be deleted.

Error Handling and Loop Management:

 In case of errors while sending requests to the API, or no connection to the API (‘requests.Timeout’, ‘requests.RequestException’), the program waits for 5 minutes

‘sleep(300)’ and restarts from the beginning.

 The program repeats the process of fetching data from the API every 5 minutes using

‘sleep(300)’ within an infinite loop ‘while True’.

This program automates the process of fetching data from an API, processing, and storing it in CSV files and corresponding directories It also manages old data to ensure accuracy and efficiency in handling new data.

Figure 3.8 Flowchart of SEND DATA program

The SEND DATA program is scheduled on a Raspberry Pi 4 Model B to start running at a specific programmed time Consequently, it checks the current time, and if it falls outside the predetermined operational timeframe, the program immediately halts.

 If the current time is within the specified time range (‘start_time’ and ‘end_time’), continue program execution.

 If the current time is outside the specified time range (‘start_time’ and ‘end_time’), stop the program.

 Read data from the file ‘id.csv’ and extract ‘rfid’ data.

 Check and read time data from corresponding ‘rfid/{rfid}.csv’ files.

 Create a list of data ‘data_list’ that contains ‘rfid_data’, ‘clock_in_timestamp’, and

 If ‘data_list’ contains data, continue program execution.

 If ‘data_list’ doesn’t contain data, stop the program.

 Use the ‘ping’ command to check the connection to the Google website ‘google.com’.

 If a connection is established, continue program execution.

 If there’s no connection, wait for 30 seconds ‘sleep(30)’ and then restart the program.

Sending Requests and Data to API:

 Send a POST request containing data from ‘data_list’ to the specified API.

 If there’s a connection to the API and data is sent successfully (response code ‘200’), continue program execution.

 If there’s a connection to the API but data sending fails (response code ‘404’), wait for

30 seconds ‘sleep(30)’ and then restart the program.

 If there’s no connection to the API (‘requests.Timeout’, ‘requests.RequestException’),wait for 30 seconds ‘sleep(30)’ and then restart the program.

 Perform after successfully sending data to the API.

 Delete all time data in the ‘rfid/{rfid}.csv’ files.

 Finish execution and stop the program.

This detailed sequence outlines the steps involved in checking the time, aggregating data, validating it, checking internet connectivity, sending data to an API, and finally, clearing the data upon successful transmission.

The ATTENDANCE program comprises two primary operational threads The first thread executes the attendance logging function by passing through two security levels: RFID or PIN and facial scanning The second thread performs facial scanning and creates a database of facial features for users after passing through a security level using RFID method.

The first operational thread consists of two sub-program modules: FACE ADD and FACE ADD ADD FACE ADD reads and validates the RFID, while FACE ADD ADD performs facial scanning, extracts facial features, and creates a database from these extracted features.

Figure 3.9 Flowchart of ATTENDACE program

In the second operational thread, the first sub-program module is divided into two scenarios, using either RFID or a PIN code to pass through the first security level Next is the second sub-program module, responsible for scanning and extracting the user’s face. After facial scanning, this sub-program module compares the extracted facial features with those in the database The final sub-program module of the second operational thread manages the login process based on the data processed and acquired by the system.

 Reading RFID card/tag information from RC522.

 Executing the verification of RFID existence (via API or on Raspberry Pi 4).

 The program checks the connection to the Google website ‘google.com’ using the ‘ping’ command.

 If a connection is established to Internet, the program proceeds with further operations.

 If there’s no Internet connection, the program proceeds to check the RFID through the RFID database on Raspberry Pi 4.

Requesting RFID Check from API:

 Send a POST request with the provided RFID data to the designated API.

 If the API connection is established, the program continues with subsequent operations.

 If there’s no connection to the API (‘requests.Timeout’, ‘requests.RequestException’), the program performs an RFID check through the RFID database on Raspberry Pi 4.

 In case there’s a connection to the API, if the input RFID data exists within the API, retrieve the ‘user_id’ data from the API, and then execute the FACE ADD ADD sub- program.

 In the scenario where there’s no connection to the Internet or API, proceed to check the input RFID with the RFID database stored on Raspberry Pi 4 If the input RFID data exists in the database, extract the corresponding ‘user_id’ linked to the input RFID, and then execute the FACE ADD ADD sub-program.

 In both scenarios, if the input RFID data doesn’t exist, return to perform the ATTENDANCE program from the beginning.

This versatile process ensures RFID data validation by leveraging an API or the local Raspberry Pi 4 database, dynamically adjusting according to internet connectivity It initiates API verification and seamlessly transitions to local validation as required Upon RFID data existence, it extracts the ‘user_id’ for FACE ADD ADD sub-program execution; otherwise, it gracefully reboots the ATTENDANCE program This adaptable strategy optimizes validation methods through varied data sources and connection pathways, ensuring robustness in the verification process.

 Initialize the compressed facial feature extraction model ‘model.tflite’ derived from the pre-trained ArcFace facial feature extraction model.

 Initialize the pre-trained face detection model, utilizing the default OpenCV model

 Initialize the webcam to capture facial image data in video format.

Face Detection and Feature Extraction:

Figure 3.10 Flowchart of FACE ADD sub-program

 Extract a frame from the webcam video data, converting it to grayscale.

 Configure Cascade classifier parameters for face detection from the grayscale image.

 Detect faces in the frame to obtain face images in color format.

 Adjust the size of the cropped face image to100 × 100 to match the input requirements of the facial feature extraction model.

 Convert the resized face image tofloat32 format to meet the input data requirement of the feature extraction model.

 The facial feature extraction model receives pre-processed input data and performs feature extraction, generating a1 × 512 vector (1 row and 512 columns).

 Store the output feature vector in a ‘vectors’ list, checking the list length; if less than 60 vectors, continue executing ‘Face Detection and Feature Extraction’.

Saving Facial Features to the Database:

 Retrieve the ‘user_id’ from the FACE ADD sub-program.

 Save the ‘vectors’ list of facial features to a file named ‘fd/{user_id}.csv’, named after the retrieved ‘user_id’.

 Upon completing the storage of feature data in the database, return to the beginning to re-execute the ATTENDANCE program.

Figure 3.11 Flowchart of FACE ADD ADD sub-program

The FACE ADD ADD sub-program sets up facial feature extraction and detection using pre-trained models It captures webcam images, extracts features, and stores them based on ‘user_id’ in a database If fewer than 60 features are extracted, it continues until reaching the quota, then restarts the ATTENDANCE program.

 Acquiring PIN input from the GUI keyboard.

 Conducting PIN existence verification via API.

 The program checks the connection to the Google website ‘google.com’ using the ‘ping’ command.

 If Internet connection is available, the program proceeds with subsequent operations.

 If no Internet connection is detected, return to initiate the ATTENDANCE program from the start.

Requesting PIN Check from API:

 Sending a POST request with the provided PIN data to the designated API.

Figure 3.12 Flowchart of PIN INPUT sub-program

 Upon successful API connection, the program proceeds with further operations.

 If there’s no connection to the API (‘requests.Timeout’, ‘requests.RequestException’), return to initiate the ATTENDANCE program from the start.

 If the PIN doesn’t exist (response code ‘200’), return to initiate the ATTENDANCE program from the start.

 If the PIN exists (response code ‘200’), extract the ‘user_id’ from the data returned by the API, then proceed to execute the FACE SCAN sub-program.

 Reading RFID card/tag information from RC522.

 Executing the verification of RFID existence (via API or on Raspberry Pi 4).

 The program checks the connection to the Google website ‘google.com’ using the ‘ping’ command.

 If a connection is established to Internet, the program proceeds with further operations.

Figure 3.13 Flowchart of RFID INPUT sub-program

 If there’s no Internet connection, the program proceeds to check the RFID through the RFID database on Raspberry Pi 4.

Requesting RFID Check from API:

 Send a POST request with the provided RFID data to the designated API.

 If the API connection is established, the program continues with subsequent operations.

 If there’s no connection to the API (‘requests.Timeout’, ‘requests.RequestException’), the program performs an RFID check through the RFID database on Raspberry Pi 4.

 In case there’s a connection to the API, if the input RFID data exists within the API, retrieve the ‘user_id’ data from the API, and then execute the FACE SCAN sub- program.

 In the scenario where there’s no connection to the Internet or API, proceed to check the input RFID with the RFID database stored on Raspberry Pi 4 If the input RFID data exists in the database, extract the corresponding ‘user_id’ linked to the input RFID, and then execute the FACE SCAN sub-program.

 In both scenarios, if the input RFID data doesn’t exist, return to perform the ATTENDANCE program from the beginning.

The RFID CHECKING sub-program resembles the FACE ADD sub-program, yet it shifts to execute FACE SCAN instead of FACE ADD ADD FACE SCAN captures a single facial feature during login, whereas FACE ADD ADD collects 60 features to build a user database for login comparisons.

 Initialize the compressed facial feature extraction model ‘model.tflite’ derived from the pre-trained ArcFace facial feature extraction model.

 Initialize the pre-trained face detection model, utilizing the default OpenCV model

 Initialize the webcam to capture facial image data in video format.

Face Detection and Feature Extraction:

 Extract a frame from the webcam video data, converting it to grayscale.

 Configure Cascade classifier parameters for face detection from the grayscale image.

 Detect faces in the frame to obtain face images in color format.

 Adjust the size of the cropped face image to100 × 100 to match the input requirements of the facial feature extraction model.

 Convert the resized face image tofloat32 format to meet the input data requirement of the feature extraction model.

 The facial feature extraction model receives pre-processed input data and performs feature extraction, generating a1 × 512 vector (1 row and 512 columns).

 Store the output feature vector; then check the condition, if a vector has been extracted, executing ‘Save Facial Features to the Database’.

Saving Facial Features to the Database:

 Retrieve the ‘user_id’ from the PIN INPUT or RFID INPUT sub-program.

 Save the a vector of facial features to a file named ‘fs/{user_id}.csv’, named after the retrieved ‘user_id’.

Figure 3.14 Flowchart of FACE SCAN sub-program

 Opens and reads the CSV files containing facial feature vectors (‘fs/{user_id}.csv’ and

 Converts the read rows (vectors) into numpy arrays offloat32 data type (‘fs_features’ and ‘fd_features’).

 Iterates through each vector in ‘fs_features’ and ‘fd_features’ to compute the similarity between corresponding feature vectors.

 Calculates the similarity using the dot product of vectors and normalizes the result (cosine similarity).

 Tracks maximum and minimum similarities, total similarity, number of comparisons, and the count of similarities above80%.

Figure 3.15 Flowchart of FACE CHECK sub-program

 If the similarity exceeding 80% accounts for more than 80% of all comparisons, it triggers the ACCESS HANDLING sub-program.

WEBSITE DESIGN

Formatting and producing replies for the client is the responsibility of the View, which stands for the presentation layer in the MVC design The frontend framework used is React.js, which allows for the building of responsive and dynamic user interfaces using reusable components.

The login page serves as the entry point for safe access, not just an interface Robust security is ensured by using industry-standard authentication protocols such as OAuth or JWT Pin codes and other two-factor authentication techniques improve security posture by striking a balance between user convenience and data security The pin code information

Figure 3.17 Web application’s architecture page, which is closely linked to the login procedure, requires a design that communicates important security information without being overly intrusive to the user.

For basic use case, use can only access the pincode dashboard for their pin code information, and checking the attendance time For advance user such as admin, they are responsible for new user’s identity creation and manage a list of their user through frontend interface.

An easy-to-use design flow is necessary for the registration page, which is where admin can form a package of user information and save it to database throught API calling to server function Entire error handling, validation prompts, and unambiguous input forms all help ensure a seamless registration procedure Accessibility is ensured via a responsive design that works with different devices, and progressive disclosure strategies can lead users step-by-step through the registration process.

When logged in, the dashboard will be shown to user, providing them the 6-digit pin code used for access the system An summary of attendance data should be clearly shown on the attendance sheet page, which serves as a hub for both administrators and users. Making use of data visualization tools, including graphs and charts, improves the readability of attendance trends Users may easily browse through large datasets with the help of intuitive filters and sorting tools, which provide a thorough picture of attendance records.

In a day where people use a variety of devices, responsive design is essential Pages need to adjust smoothly to different screen sizes so that users can interact with them on

Figure 3.18 User and admin use case interface

PCs, tablets, and smartphones Furthermore, inclusion depends on accessibility factors like keyboard navigation and screen reader compatibility.

MongoDB is a NoSQL database that stores data in flexible, JSON-like BSON documents It’s schema-less, allowing for dynamic and easy data manipulation This flexibility is advantageous for projects with evolving requirements, likeattendance system.

JavaScript server-side code may be run in a runtime environment at the backend thanks to Node.js Because it is non-blocking and event-driven, it can manage several concurrent connections A RESTful API facilitates communication between the frontend and backend operating in the Nojdejs environment The online application framework Express.js aids in the definition and management of these APIs Express is a Node.js framework for creating simple online applications By offering a range of capabilities including routing, middleware, and HTTP utility methods, it makes the process of creating reliable and scalable APIs easier.

With the frontend communicating with users and the backend managing server functionality, the design is simple and adheres to the client-server paradigm The security of online applications is improved by the use of middleware for authentication.

Admin routes handle actions related to administrative tasks in web app system, such as user management, role changes, and data deletion.

The Figure 3.19 show the main api route for requesting backend function process.

These routes also represent for functions in the Controller, as part of the MVC pattern, acts as an intermediary between the Model and the View It handles user inputs, processes them,and updates the Model accordingly This cohesive structure ensures a seamless flow of data and logic, connecting the frontend’s dynamic user interface with the backend’s robust data handling and business logic.

An essential component of a project’s smooth execution of user-related functionality is the User Management API Implementing an authorization mechanism is a crucial component of this system as it guarantees safe access and the efficient performance of designated tasks This study explores the several features offered by the API and emphasizes the integrated procedures for authorization and authentication.

The API has a wide range of features that address many facets of user management. These features include the ability to register, deactivate, activate, change roles, get user data, receive RFID updates, and control attendance Every function may be accessible via unique API endpoints and related routes, which adds to the system’s overall resilience.

A crucial component of the API’s design is authorization, which makes sure that only users who have been verified and given permission may carry out sensitive actions In functions where security and secrecy are critical, such as user registration, deactivation, activation, role change, and data retrieval, permission is especially necessary.

The “admin” position is of great significance in the role-based access control mechanism implemented by the API Some tasks, like adding a new user, can only be carried out by users who hold the “admin” position The middleware is essential in confirming the administrator’s legitimacy since it keeps track of the administrator’s position at all times throughout requests The middleware verifies that the person trying to do the action is, in fact, authorized as an administrator in order to authenticate

Figure 3.19 Admin routes control administrators The authentication process is imperative for maintaining the integrity of the user management system.

Mainly in the project, various function only can be called and request with authorizaton, such as request register for a new user The adminitrator will need to provide necessary information in order to register for a new user, the administrator is indicate as authorized when his current role is “admin” and being verified by the middleware The authorizaton will need to be process for every end route that have the authenticate config. When register for new user, be side all user basic information in store in the database, the pin-code and attendance template data will also be created for user the 6-digits pin code is base on a secret key The secret key in TOTP is a random, cryptographic key that is shared between the server (your backend) and the client (the user’s device) It serves as the basis for generating the one-time passwords The secret key is generated by these steps:

 Random Generation: The secret key is usually generated randomly and is kept secret between the server and the user’s device.

 Length: The secret key is typically a byte sequence with a recommended length of 16 bytes (128 bits) or more Longer keys provide increased security.

Method API End Route Function

Post register Registers a new user

Post deactivate-user Deactivates a user

Post activate-user Activates a user

Post change-role Changes the role of a user

Post get-all-user Retrieves information about all users

Post update-rifd Updates RFID information for a user

Post get-user-hardware-credential-by-user-id Retrieves hardware credentials for a user

Post get-all-rfid-data Retrieves all RFID data

Post delete-user-by-id Deletes a user by user ID

Post reset-attendance Resets attendance data without saving to local data

Post get-attendance Retrieves attendance data

 Base32 Encoding: The secret key is often encoded in Base32 for usability, as it results in a more compact representation using a subset of the ASCII character set.

After generate the secret key, the pin code will be generated based on the secret key by these steps:

 Time Dependency: TOTP generates one-time passwords based on the current time The time is usually divided into fixed-size intervals, often 30 seconds or 60 seconds.

 HMAC Algorithm: TOTP uses HMAC-SHA-1 or other HMAC-based hash functions to combine the secret key and the current time interval into a hash.

 Truncation: The resulting hash is truncated to obtain a dynamic, fixed-length code The truncation involves taking a specific number of bits from the hash result.

 Conversion to Decimal: The truncated hash is then converted to a numerical value, usually in decimal form.

 Modulo Operation: The numerical value is often subjected to a modulo operation to ensure that the PIN code fits within the desired range (e.g., a 6-digit PIN code).

 Authentication: Uses authentication middleware to secure sensitive endpoints.

 Hardware Request Routes: Hardware request routes handle actions related to hardware requests and RFID data.

 Authentication: Uses authentication middleware to secure sensitive endpoints.

Hardware request routes handle actions related to hardware requests and RFID data. Building upon the comprehensive functionalities of the Hardware Request Controller API, this delves into two critical functions aimed at fortifying security measures within the system The functions “check-for-l1” and “check-for-l1-backup” focus on verifying L1

COMPRESSED FACIAL FEATURES EXTRACTION MODEL

ArcFace is a CNN based face recognition algorithm that utilizes a classification network with additive angular margin loss to learn highly discriminative features Its architecture can be divided into three main components:

The input image passes through a series of convolutional layers through backbone layer and head layer, including:

 Depthwise separable convolutions: These perform efficient feature extraction by applying separate filters to each input channel and then combining the outputs through pointwise convolutions.

Figure 3.24 Workflow of feature extraction model

 Inverted residual blocks: These expand the input feature dimension before applying depthwise separable convolutions and use shortcut connections for efficient information flow.

These layers extract increasingly high-level features, culminating in a high- dimensional feature vector representing the image.

This is the key component that distinguishes ArcFace from other algorithms and it aims to:

 Increase inter-class distances: Features from different classes should have large angles between them, making them easily distinguishable.

 Promote intra-class compactness: Features from the same class should have small angles between them, indicating their similarity.

The loss function achieves this by adding an angular margin to the standard softmax loss This margin effectively pushes features from the same class closer and those from different classes further apart in the embedding space.

This layer uses the softmax function to predict the class probabilities for each input image This allows the model to classify faces based on their extracted features.

The input of ArcFace is a RGB image of a face, typically resized to a specific size (e.g.,112 × 112 or224 × 224) Each image is represented as a 3-channel tensor and the data type is typicallyfloat32.

Arcface model is chosen to be the head layer of model for it face recognition algorithm since feature vector are the expected output for the project’s face feature extraction task ArcFace has demonstrated state-of-the-art performance on benchmark face recognition datasets Its ability to learn highly discriminative features has contributed to its popularity in the field.

MobileNetV2 is a CNN architecture specifically designed for efficient inference on mobile and embedded devices with limited computational resources It achieves significant reductions in model size and computational cost compared to traditional CNNs while maintaining good accuracy MobileNetV2 utilizes several key techniques to achieve its efficiency.

The initial convolutional layer is responsible for processing the input image It typically uses a3 × 3 convolution with a stride of 2 to reduce the spatial dimensions of the input The goal is to capture basic features and patterns from the input image.

Stacks of Inverted Residual Blocks with skip connections and residual connections. Inverted Residual Blocks are the fundamental building blocks of MobileNetV2 Each block consists of the following operations:

 Linear bottleneck: First, the1 × 1 Convolution (Linear Expansion), linearly increases channel numbers, enhancing the network’s capacity to grasp intricate features Next, the utilization of Depthwise Separable Convolution slashes computational expenses by decomposing convolutions across depth Finally, the subsequent 1 × 1 Convolution (Linear Projection), downsizes feature maps to a lower-dimensional space.

 Skip Connection: A skip connection is added between the input and output of the block to facilitate the flow of gradients during training.

 Residual Connection: The output of the linear bottleneck is added to the skip connection, creating a residual connection This helps address the vanishing gradient problem.

The expansion layer follows the stacks of inverted residual blocks It uses a 1x1 convolution to increase the number of channels in the feature maps This expansion allows the network to capture more complex patterns before applying depthwise separable convolutions.

Depthwise separable convolutions split the standard convolution into a depthwise convolution and a pointwise (1x1) convolution This reduces the computational cost while maintaining expressive power It is applied after the expansion layer.

Global Average Pooling is used to reduce the spatial dimensions of the feature maps to a fixed size It computes the average value for each channel across the entire spatial dimensions This results in a compact representation of the features.

The output of the Global Average Pooling layer represents a compact and informative feature vector that captures the essential characteristics of the input image This feature vector is then used for downstream tasks such as classification.

For backbone layer, the MobileNetV2 is a relatively lightweight neural network architecture designed for efficiency, making it a suitable choice for deployment on resource-constrained devices like the Raspberry Pi 4 The Raspberry Pi 4 is a compact and affordable single-board computer, and while it has improved computational capabilities compared to earlier Raspberry Pi models, it still has limitations in terms of processing power and memory.

Since the computation resource is limited, we are using a pre-trained weight which has been trained on5,822,653 samples of85,742classes.

The dataset used for training the pre-train model is shown in theTable 3.8:

Table 3.8 Dataset for tranining pre-trained model

Since we can not directly use the weight on Raspberry Pi 4 because of it size, we use TFLite to convert the weight into TFLite model TFLite models are optimized for mobile and edge devices, and they typically have a smaller model size compared to their full- precision counterparts This reduction in size is crucial when dealing with devices that have limited storage capacity, and also optimal for edge device like Raspberry Pi 4 These optimizations include model size reduction, quantization, hardware acceleration support, and compatibility with the TensorFlow Lite runtime This allows neural network models to be deployed and run efficiently on resource-constrained devices while maintaining acceptable levels of accuracy.

Database Version Num of Classes Images

The input for compressed TFLite model is image that have100 × 100 in wide and height, with 3 channel RGB and convert tofloat32.

To revify the weight result is reliable on our system, we create a central face-cropped to work as a testing database, the expected result will be the feature vector of each image that we put in the model to extrac The dataset is face cropped from bounding box, with total of162 images and 4 classes.

The pre-processing process include resizing each image to100 × 100 (𝑀 × 𝑁) then convert them into grayscale format By the end of the pre-processing step, the average size of image is 4.3𝐾𝑏 The feature extraction of each image in saved in form of binary file format in NumPy for persisting a single arbitrary NumPy array on disk Shape of one file is a single dimension array and512 elements ([1,512]) with average size about2.2𝐾𝑏

The result is verify by comparing each class to each other, one by one image By the end, the result of26,244 times of comparing is divined in to4 types:

The accuracy, percision of the model are calculated by following the formula:

Figure 3.25 Output facial feature extraction files

Based on the feature comparison results, the overall accuracy is approximately 72.6% The positive accuracy is high (99.7%), but the negative accuracy is low (25.4%). For False Positives, the cases where this system mistakenly identifies one person as another have a low occurrence rate.

For False Negatives, these cases occur when comparing two features with a similarity below 80%, even though they belong to the same person These instances often happen when comparing images of the same person from different angles (e.g., looking straight vs. looking sideways, looking up, etc.) Since gathering database on Raspberry Pi will only include the case the user is looking straight at the camera only, the False Negatives should be reduced.

Figure 3.26 Output precision of model

RESULTS

WEB APPLICATION INTERFACE RESULTS

The web application is design in form of MVC, the View is one of there main element to represent the information of user in the database, the frontend of the web application will need to be straightforward and easy to understand for the user The objective of the login page prioritizes accessibility, ensuring it is user friendly and usable by user, clear contrasts between text and background, keyboard accessibility, and the inclusion of descriptive alt text for images contribute to a more inclusive user experience The login page implements robust password security measures, including password masking and adherence to best practices for encryption.

For basic user and administrator, the home page serves as the central hub of the web application, providing users with a snapshot of essential information and navigation to key features The navigation of the main web application is display on the side bar on the left of the page, this side bar will include user full name and avatar, navigator to different page. The information display is concise and well-organized content ensures users can quickly grasp the status of key insights without feeling overwhelmed, which is the 6-digit access code display This 6-digit code is re-generated every 60 seconds, ensure that each user will have their own code with no duplication with other users Beside all the main feature, all the page will also include utility features such as changing between light theme and dark theme, and toggle the side bar.

The main dashboard will satisfy the mission of providing user with their access code as it right in front of them when logged in.

The Clock Information page contain clock in time, clock out time, click in date and clock out date for user to keep track of their attendance, this will be easy to review when payslip come in and user need to trace back if the payslip working hours is wrong or not, as the server also calculate the working hours of each day The status for the attendance of user will be “Present” if user have a valid clock in and clock out time during the day The column of clock information also come with utility like filter, choose with column to shown and download export file.

The Manage Users page is accessible only to authorized personnel, typically administrators or users with specific privileges This page serves as a central hub for user management, displaying a comprehensive list of all users in the system For each user, administrators can view their status (active or inactive), email address, assigned role, and other pertinent details This feature empowers administrators to efficiently oversee user accounts, manage permissions, and maintain the overall integrity of the user database.

The All User Info page is an extended view available exclusively to administrators.

It provides in-depth details about each user, offering insights beyond basic information. This includes user ID in the database, RFID number, and other relevant identifiers Having

Figure 4.5 Detail Users Information Page access to this comprehensive information streamlines administrative tasks, such as troubleshooting user-specific issues.

For both Manage Users and All User Info, administrators have the capability to update user details, modify roles, changing password as needed The pages are designed with a user-friendly interface, ensuring that administrators can efficiently navigate and execute actions without unnecessary complexity The Create New User page is a dedicated section within the web application accessible exclusively to administrators Its primary purpose is to facilitate the addition of new users to the system with ease and accuracy The page includes a comprehensive form to capture essential user details This may encompass fields such as full name, email address, role assignment, profile picture and an option to set an initial password Administrators can assign specific roles to the new user based on their responsibilities within the system Role options may include basic user, manager, or other customized roles with varying levels of access that will be update in the future Before finalizing the creation of the new user, administrators are prompted to review the entered information This step helps prevent errors and ensures that all necessary details are accurate Each action taken on the Create New User page is logged in an audit trail This feature helps maintain accountability, providing administrators with a record of when new users were added and by whom.

Figure 4.6 Update Information for User

FACIAL FEATURES EXTRACTION MODEL RESULTS

The facial extraction model on Raspberry Pi 4 is a compressed model intflite format derived from the ArcFace facial extraction model trained and executed on Google Colab with similar configurations.

4.2.1 SIMILARITY OF COMPRESSED AND ORIGINAL MODELS

To examine the similarity between two models, the compressed and the original, a test was devised It involved studying the distribution of facial feature extraction vectors from these models across different platforms, specifically on Google Colab and on the embedded system Raspberry Pi Model B4𝐺𝐵, in the context of this project.

The results of this test not only provide a bar chart illustrating the distribution of the extracted vector values but also offer descriptive statistics providing a summary and description of the main features of a dataset These outcomes help in understanding the distribution, central tendency, and variability of the data.

The verification was conducted using datasets containing feature extraction vectors obtained from the same set of images of the same person, consisting of an equal number of

60 vectors The images within the dataset were captured under relatively ideal conditions: frontal face capture with a fixed distance, good lighting conditions (adequate ambient light and direct light on the face, eliminating backlighting issues), clean facial appearance, and minimal changes in facial position or posture.

Figure 4.7 Create New User page

Some key descriptive statistics metrics to consider:

 Mean across all files:is calculate with the formulaEq 6 below, where𝑛 is the number of data points, and𝑥 𝑖 is each individual data point.

Standard deviation across all files: is calculate with the formulaEq 7 below, where

𝑛 is the number of data points,𝑥 𝑖 is each individual data point, and 𝑀𝑒𝑎𝑛 is the value of

 Media across all files:For an odd number of data points (𝑛), the median is the middle value For an even number of data points (𝑛), the median is the average of the two middle values.

Compressed and Original Models on Google Colab:

The distribution charts of the vectors values from the original and compressed models are respectively depicted inFigure 4.8 andFigure 4.9.

The descriptive statistics results from the original and compressed models are presented in Figure 4.10 andFigure 4.11, respectively.

Figure 4.8 Distribution of vectors - orginal model on Colab

According to Figure 4.8 and Figure 4.9, they respectively illustrate the distribution of values in the feature extraction vectors obtained from the original and compressed models executed on Google Colab, revealing significant differences However, to assess the similarity between the two models, it is necessary to analyze and compare the data from the descriptive statistics of both models, depicted through Figure 4.10 andFigure 4.11.

The data from two sets of face vector extractions, one from the original model and another from the compressed model run on Colab, exhibit significantly different statistical parameters, indicating substantial dissimilarity between them The compressed model showcases a negative mean(−0.0235), whereas the original model maintains a mean close to zero (−0.0014), highlighting a notable contrast.

Particularly, the standard deviation of the compressed model (2.2922) is much higher than that of the original model (0.0442), signifying a significantly greater variability in the

Figure 4.9 Distribution of vectors - compressed model on Colab

Figure 4.10 Statistics of vectors - orginal model on Colab

Figure 4.11 Statistics of vector - compressed model on Colab compressed model’s data compared to the original The minimum and maximum values of the compressed model also reflect considerable differences compared to the original model, with a substantially wider range of values.

The substantial disparity isn’t limited to basic statistical parameters such as mean and standard deviation but is also evident in the median This substantial dissimilarity between the two datasets suggests that the compressed model may have undergone significant alterations in the data extraction process compared to the original model, resulting in notable discrepancies in their features and statistical measures.

Compressed Models on Colab and on Raspberry Pi 4:

Next, the similarity between the compressed model running on Google Colab and on the embedded device Raspberry Pi 4 Model B 4𝐺𝐵 needs to be examined and evaluated. The method of examination and evaluation mirrors the approach applied to assess the similarity between the compressed and original models executed on Google Colab.

The data used to evaluate the compressed model running on Colab is reused from

Figure 4.9 and Figure 4.11 For the compressed model running on the Raspberry Pi, the evaluation data is presented throughFigure 4.12 andFigure 4.13 below.

The data from two different sources - a compression model running on Colab and on a Raspberry Pi 4 Model B 4𝐺𝐵 - have been analyzed using basic statistical measures. Firstly, the comparison between the two datasets revealed several notable points.

The first and second datasets have mean values of positive (0.0506) and negative (−0.0235) respectively, indicating the direction of the data in both cases While the means differ, the variance suggests that the central tendencies of the data are not significantly different The standard deviation, a measure reflecting data dispersion, is relatively close for both datasets This indicates that the variability of the data does not differ significantly. However, a significant difference lies in the minimum and maximum values of the two datasets The second dataset has larger minimum and maximum values compared to the first dataset, suggesting a higher level of variability in the second set.

Overall, despite minor differences in statistical parameters, feature extraction from the compression model on both Colab and Raspberry Pi demonstrates similarity in central tendencies and variability of the data The differences could be explained by variations in the processes or environments while running the model However, the crucial aspect is the consistency in feature extraction, implying that the compression model is capable of generating similar features on both platforms.

The results obtained from the data, verifying the similarity between the original and compressed models, indicate a significant level of dissimilarity between the two models. However, these outcomes do not definitively prove that the compressed model cannot extract features for facial recognition and differentiation beyond the acceptable 80% threshold.

Thus, further experiments are vital to comprehensively validate the compressed model’s capabilities for facial recognition and differentiation when operating on the Raspberry Pi 4 These experiments will offer deeper insights into the model’s efficiency and performance in real-world applications, ensuring its suitability for effective facial recognition tasks.

This verification aims to demonstrate that the compressed model used on the Raspberry Pi 4 Model is capable of extracting features from input image data, achieving a similarity level below80% when comparing the extractions of different people The input

Figure 4.12 Distribution of vectors - compressed model on Raspberry

HARDWARE RESULTS

The hardware of the system operates according to the specified functions and tasks for each block Moreover, these blocks function consistently and reliably, demonstrating that the hardware design and assembly stages have been error-free, encompassing the control programs as well.

The hardware components include the RTC DS1307 module, RFID module RC522, electronic lock, and various other electronic components (LEDs, buzzers, transistors, relay, resistors) functioning according to the design principles, ensuring high stability and reliability.

The Raspberry Pi 4 Model B mini-computer is used for stable and reliable system operation, quick execution of functions, processing, and rapid responses However, initiating the webcam or USB camera takes time during the initial boot-up of the Raspberry

The ATTENDANCE program essentially meets the system requirements, exhibiting fast startup times and error-free operation based on testing and subsequent bug fixes The user interface is simple and user-friendly, adhering to the design principles accurately The following content will provide a detailed presentation of the execution results of the control programs with the hardware components through commentary and visuals.

Below is Figure 4.15, illustrating the MAIN window of the ATTENDANCE program The MAIN window includes the current time (day of the week, date, and time),followed by user-friendly headings for system navigation Lastly, there are four buttons:two modes, one to create a database of facial feature extractions - the ADD FACE button,the other to access the dual-security mode for attendance marking - RFID and PIN buttons,and the final red button - RESTART, aimed at re-running the ATTENDANCE program.The entire GUI of the ATTENDANCE program allows users to close the program by clicking the ‘X’ button on the interface window.

If the user selects the database creation mode and clicks the ADD FACE button, they access the initial step of this mode: scanning the RFID The interface for this step is presented in Figure 4.16.

Figure 4.17 illustrates the user interface for capturing the user’s facial features for feature extraction Once the RFID is confirmed to exist in the database (online/offline), the facial capture interface appears, allowing the user 5 seconds to adjust their face before the system proceeds with facial capture and feature extraction.

Figure 4.16 RFID - FACE ADD window

Figure 4.17 FACE ADD - facial extraction window

Figures 4.18 depict the interface windows when selecting the attendance mode via

Figure 4.19 and Figure 4.20 display the interface windows for the attendance mode using a PIN code.

Figure 4.21 and Figure 4.22 depict the interface windows of the security level 2 system, facial recognition In this stage, the system captures a facial frame of the user, extracts features, and compares them with the database.

Figure 4.20 PIN INPUT - LEVEL 1 window

As outlined in 3.2.3 CONTROL PROGRAMS, the system automatically returns to the MAIN window in cases where the subprograms complete execution, specific errors occur as defined in the control program, or when the BACK button is pressed.

GET DATA and SEND DATA Programs:

The programs GET DATA and SEND DATA are executed without a GUI and user interaction; they are scheduled tasks on the Raspberry Pi 4 Specifically, GET DATA starts upon Raspberry Pi 4 boot-up, similar to the ATTENDANCE program However, GET DATA automatically executes data retrieval from the API and database creation on the embedded system through programmed cycles every 5 minutes On the other hand, the SEND DATA program is scheduled to run at specific time intervals, autonomously performing programmed functions and being terminated under specific conditions.

Figure 4.21 FACE SCAN - LEVEL 2 window

Figure 4.22 FACE SCAN - facial extraction window

The methods for security level 1, RFID and PIN, are chosen due to their high reliability, resulting in absolute accuracy in testing Additionally, the precisely programmed control software contributes significantly to this accuracy Beyond reliability and accuracy, both methods exhibit swift processing In cases where there’s a connection to an API, the authentication time for RFID or PIN is under 1 second However, in scenarios lacking API connectivity (utilizing only the RFID method), the system takes time to authenticate the absence of API connectivity before verifying the existence of the RFID code in the database of the Raspberry Pi 4 embedded system, hence resulting in an authentication time of around from more than 1 second to under 2 seconds.

The facial feature extraction method, chosen for security level 2, was selected due to its swiftness and reasonably high accuracy demonstrated through testing The rapidity of the method employing facial feature extraction has been substantiated through various testing parameters Firstly, the time taken to build the database from 60 face images captured by the camera under similar conditions as described in section 4.2.1 is approximately 10 seconds Subsequently, the time required to scan a user’s face, extract a facial vector, ranges from just above 0.1 to under 0.2 seconds Lastly, the time taken to match the scanned facial vector with the user database comprising 60 vectors falls within the range of over 0.1 to under 0.2 seconds.

To validate the reasonably high accuracy of this method, the tables presenting the experimental results will be outlined below The experimental method was conducted with four faces, identified as person1, person2, person3, and person4 Among them, person1 and person3 are males, while person2 and person4 are females, but person1 and person2 are siblings with similar facial features The experimental procedure involved using the facial scan vector of person1 for comparison with a database containing sets of feature- extracted vectors from the faces of person1, person2, person3, and person4 The criterion to determine facial similarity aligns with the standards presented in section4.2.2.

Table 4.1 Comparison person1 scan vector with person1 database

Table 4.2 Comparison person1 scan vector with person2 database

Percentage of ratios greater than or equal to80%:96.67%

Percentage of ratios greater than or equal to80%:100.0%

Percentage of ratios greater than or equal to80%:100.0%

Percentage of ratios greater than or equal to80%:96.67%

Percentage of ratios greater than or equal to80%:15.0%

Percentage of ratios greater than or equal to80%: 0.0%

Percentage of ratios greater than or equal to80%:50.0%

Percentage of ratios greater than or equal to80%:1.67%

Table 4.3 Comparison person1 scan vector with person3 database

Table 4.4 Comparison person1 scan vector with person4 database

Percentage of ratios greater than or equal to80%:15.0%

Percentage of ratios greater than or equal to80%: 0.0%

Percentage of ratios greater than or equal to80%:0.0%

Percentage of ratios greater than or equal to80%:0.0%

Percentage of ratios greater than or equal to80%: 0 0%

Percentage of ratios greater than or equal to80%: 0.0%

Percentage of ratios greater than or equal to80%:0.0%

Percentage of ratios greater than or equal to80%:0.0%

In this experiment, the scanned face of person1 showed relative changes in position and posture However, the results revealed no mistaken identifications of person1’s face with person2, person3, and person4, following the criteria defined in section 4.2.2.

Notably, the similarity ratios when comparing the vectors were quite high, nearing the80% standard threshold, especially in the comparison of person1’s scan vector with person2’s and person3’s databases, reaching approximately ~83.64% and~82.69%, respectively. Moreover, the highest Percentage of ratios greater than or equal to 80% in comparing person1’s scan vector with person2’s and person3’s databases were recorded at50% and 15%, respectively Through experiments, it has been shown that the feature extraction vectors of the compressed model are significantly affected by external factors such as the distance from the face to the camera, lighting, background, etc.

Figure 4.23 and Figure 4.24 respectively depict the completed hardware with its enclosure.

Figure 4.23 Complete hardware with enclosure (1)

Figure 4.24 Complete hardware with enclosure (2)

CONCLUSIONS AND RECOMMENDATIONS

CONCLUSIONS

In conclusion, the web application has been meticulously designed to provide users and administrators with a robust, secure, and user-friendly environment The Dashboard serves as a dynamic hub, offering real-time insights and customizable widgets for personalized user experiences The Clock Information feature enhances transparency and accountability by timestamping user activities, enabling administrators to track behavior effectively.

The user management functionalities, including the Manage Users and All User Info pages, empower administrators to oversee the user ecosystem comprehensively The Manage Users page acts as a central command center, facilitating efficient user management, role assignments, and permissions control The All User Info page offers an extended view exclusively for administrators, providing in-depth details crucial for troubleshooting and maintaining accurate user records.

The addition of the Create New User page further strengthens the user management capabilities Accessible only to administrators, this page streamlines the onboarding process, allowing for precise role assignments, permissions configuration, and adherence to security best practices The incorporation of an audit trail ensures accountability and transparency in user creation activities.

The combination of ArcFace and MobileNetV2 for face recognition has proven to be a compelling choice, particularly in the realm of real-time applications The integration of ArcFace and MobileNetV2 technologies represents a significant advancement in the capabilities of the system ArcFace, with its state-of-the-art face recognition algorithm, enhances the security and identity verification aspects of the application Its ability to generate discriminative face features contributes to a more accurate and reliable authentication process.

Complementing ArcFace, the incorporation of MobileNetV2 as a lightweight neural network architecture optimizes the efficiency of facial recognition on resource-constrained devices This is particularly beneficial for mobile applications, where computational resources are often limited MobileNetV2 strikes a balance between accuracy and computational efficiency, ensuring a seamless and responsive user experience.

After the process of exploration and research, the hardware has been constructed to a relatively complete extent, meeting the set requirements The functional blocks operate swiftly, accurately, and sustainably over extended periods without encountering errors. Particularly, utilizing the system is also very straightforward and secure.

The hardware components of the system are designed to be compact and systematic, enabling them to be housed within a casing, ensuring aesthetics, convenience, and flexibility, while contributing to creating a hardware assembly that is easily installed, replaced, and repaired.

The hardware GUI is designed to be simple, clear, and sophisticated, a significant advantage that makes the system more user-friendly With a user-friendly GUI, system usage is streamlined, making it easily accessible and operable for users Additionally, the hardware includes visual and auditory notification components such as LEDs and buzzers, enhancing the system’s intuitive nature.

Using facial recognition through the method of feature extraction vectors is a factor that facilitates the system’s swift, accurate, and reliable operation With feature extraction vectors, the system achieves a high level of security along with rapid processing time and a high degree of accuracy in human face recognition and classification Employing feature extraction vectors from faces is a novel and pioneering aspect of the system.

Despite the positive outcomes and advantages, the hardware also exhibits limitations and shortcomings One of its constraints is the use of the OpenCV facial recognition model, coupled with a compressed model for facial feature extraction that significantly differs from the original model This presents a considerable drawback: a notably high similarity ratio among the extracted feature vectors of different faces with many discrepancies. However, fortunately, based on the standard for facial differentiation mentioned in section

4.2.2, the hardware has managed to distinguish faces with a good ratio meeting the 80% threshold as per the defined standard.

RECOMMENDATIONS

Implement a feedback mechanism to gather insights from users and administrators. Regular feedback will aid in identifying areas for improvement, ensuring the web application remains responsive to evolving user needs.

Conduct periodic security audits to identify and address potential vulnerabilities. Proactive measures, such as penetration testing, can enhance the overall security posture of the web application.

Anticipate future growth and plan for scalability As the user base expands, ensuring that the web application infrastructure can seamlessly accommodate increased demand is vital for sustained performance.

Provide comprehensive training materials and documentation for administrators to maximize their efficiency in utilizing the various user management features This will contribute to a smoother onboarding process for new administrators.

Ensure that the web application adheres to accessibility standards, making it inclusive and usable for individuals with diverse needs This involves testing for compatibility with assistive technologies and providing alternative text for multimedia content.

Regularly update and retrain the ArcFace model to adapt to changing user demographics and improve recognition accuracy over time Incorporate diverse datasets to ensure the model remains robust across various facial characteristics.

Explore optimization techniques for MobileNetV2 to further enhance its efficiency. Techniques such as model quantization and pruning can be employed to reduce the model size and improve inference speed without compromising accuracy.

Implement robust security measures to safeguard the facial recognition data and models Additionally, prioritize user privacy by adopting privacy-preserving techniques, such as on-device processing or secure data transmission protocols.

With the aim of efficiently applying facial feature extraction vectors with relatively good accuracy and fast processing times, lightweight models for facial recognition and feature extraction have been implemented into the hardware However, while the use of these lightweight models has somewhat met the specified requirements, they still present limitations and drawbacks Therefore, to enhance facial recognition, extraction, and differentiation, the hardware needs to either employ better models or improve the existing ones.

To improve and enhance the level 2 security of the system, it’s crucial to research and apply superior facial recognition models, allowing the hardware to precisely cut facial features and minimize environmental factors Additionally, it’s necessary to research and fine-tune a new compressed facial feature extraction model that meets two requirements: lightweight enough to efficiently run on embedded devices like the Raspberry Pi 4 Model-

B 4𝐺𝐵 and significantly enhance facial feature extraction, minimizing the impact of environmental factors.

In addition to researching model improvements for recognition and differentiation of faces, it’s essential to explore and implement facial scanning and data collection from various angles into the system This facilitates user convenience and enhances the system’s user-friendliness This research relates to studying the workflow of the control program,defining face differentiation standards to ensure the system’s efficiency and reliability.

[1] P Porter, S Yang and X Xi, The Design and Implementation of a RESTful IoT Service Using the MERN Stack, 2019 IEEE 16th International Conference on Mobile

Ad Hoc and Sensor Systems Workshops (MASSW), Monterey, CA, USA, 2019, pp. 140-145, doi: 10.1109/MASSW.2019.00035.

[2] Anjali Chauhan, A Review on Various Aspects of MongoDb Databases, May 2019,

[3] StrongLoop, IBM, and other expressjs.com contributors,Express documentation.

[4] Alok Kumar Srivastava , Vaishnavi Laxmi , Payal Singh , Km Pratima , Vibha Kirti,

React JS (Open Source JavaScript Library), Feburary 2022, ISSN: 2349-6002 Vol 8

[5] X Huang,Research and Application of Node.js Core Technology, 2020 International

Conference on Intelligent Computing and Human-Computer Interaction (ICHCI), Sanya, China, 2020, pp 1-4, doi: 10.1109/ICHCI51889.2020.00008.

[6] A Firmansyah, T F Kusumasari and E N Alam,Comparison of Face Recognition Accuracy of ArcFace, Facenet and Facenet512 Models on Deepface Framework,

2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Jakarta, Indonesia, 2023, pp 535-539, doi: 10.1109/ICCoSITE57641.2023.10127799.

[7] J Deng, J Guo, J Yang, N Xue, I Kotsia and S Zafeiriou, ArcFace: Additive Angular Margin Loss for Deep Face Recognition, IEEE Transactions on Pattern

Analysis and Machine Intelligence, Octorber 2022, Vol 44, no 10, pp 5962-5979, doi: 10.1109/TPAMI.2021.3087709.

[8] M Sandler, A Howard, M Zhu, A Zhmoginov & L-C, Chen,MobileNetV2: Inverted residuals and linear bottlenecks, March 2019, https://arxiv.org/abs/1801.04381.[9] peteryuX,arcface-tf2,https://github.com/peteryuX/arcface-tf2.

Codes of Website and API: https://github.com/foxxy219/CP_Project

Codes of Control Programes of Hardware: https://github.com/duykhanh2k1321/CP