Integrated Face Recognition and IoT-Based Electronic Equipment Management System

The advent of the Internet of Things (IoT) has ushered in a transformative approach to asset management and security, particularly in educational institutions where equipment lending and tracking are vital. This research aimed to develop a secure, IoT-based "Smart Box Security System" that leverages facial recognition for enhanced equipment management. The methodology included a comprehensive analysis involving a literature review, interviews, and direct observations, which led to the creation of a system that integrates both hardware components, such as the ESP32-CAM


INTRODUCTION
The Internet's evolution has progressed from the static information exchange of the Internet of Content (WWW) to the dynamic user interactions in the Internet of Service (Web 2.0), and further to the Internet of People, marked by the rise of social media.The current phase, the Internet of Things (IoT), features interconnected devices enhancing efficiency and intelligence across various life aspects.
The IoT ecosystem, comprising connected physical objects ('things'), is experiencing significant growth, with estimates of global IoT devices tripling by 2030.IoT revolutionizes daily interactions and activities by transmitting data to cloud or on-premise servers, impacting both small-scale environments like buildings and larger areas like cities (Paolone et al., 2022).(Wang et al., 2021) studied IoT's development, identifying three distinct phases from 2000 to 2019, marked by gradual to sharp increases in research and publications.
Architecture of the Internet of Things (IoT) discussing the Conceptual Model (CM), IoT entities, and domains (ISO/IEC 30141, 2018).It covers IoT's identity concepts, relationships between services, networks, IoT devices, and gateways, as well as the interactions of IoT users.The IoT Reference Model (RM) integrates entity and domain-based approaches.It explores IoT's computing models, including cloud (Ali et al., 2022; Alli & Alam, 2020), fog (IEEE Communications Society, 2018), and edge computing (Aslanpour et al., 2020;Gill, 2022), highlighting their structures and functionalities.Middleware, serving as a core layer in IoT architecture, ensures connectivity between applications and smart devices, adapting to various requirements and domains (Kassab & Darabkh,  2020; Zhang et al., 2021).The IoT system's layered structure supports efficient data processing and communication across different network types.
Face recognition, a crucial area in computer vision and image understanding, involves processing facial images through detection, alignment, and feature extraction.Subsequent comparison with gallery images facilitates either face verification (determining if two images belong to the same subject) or face identification (recognizing a person from a group of gallery images).Face recognition assumes the subject is already registered in a gallery; a scenario termed as 'closed set' identification.However, 'watch-list' identification does not guarantee all subjects are in the gallery (G.Guo & Zhang,   2019).
Advancements in hardware and imaging technology have expanded face recognition applications, such as access control and video surveillance.Nonetheless, the technology faces challenges like low resolution, varying poses, complex lighting, and motion blur, affecting accuracy.Traditional algorithms like Eigenfaces (L.Machidon et al., 2019), Fisherfaces (Ahsan et al., 2021), Bayesian face (Zafar et al., 2019), Metaface (J.Guo et al., 2020), based on Support Vector Machine (SVM) (Pan et al., 2020), and Boosting (Sevastopolsky et al., 2022) offer resource efficiency but limited accuracy in unrestricted environments.In contrast, deep learning algorithms, particularly Artificial Neural Networks (ANNs), excel in learning capacity and robustness, achieving impressive results with substantial training data and computing resources (Isaac Abiodun et al., 2018; Lee & Kim,   2023).
The ESP32-CAM, a small camera module featuring the ESP32-S chip and OV2640 camera, supports face recognition capabilities.It's equipped with a 32-bit CPU 520 KB SRAM, and additional 4M PSRAM, integrating Wi-Fi and Bluetooth.The module's GPIO pins facilitate peripheral connections, and its SD card support enables image storage Utilizing algorithms like Haar Cascades for face detection and Eigenfaces, Fisherfaces, or LBPH for recognition, the ESP32-CAM effectively identifies facial features and patterns, offering a compact yet powerful tool for face recognition applications in various settings.
This research develops a "Smart Box Security System" for electronic equipment storage, integrating face recognition and IoT-based lending management.Previous research in this area includes the development of face recognition systems for automatic door access (Ikponmwosa et al., 2023), IoT applications in asset management using RFID technology (Darwin & Budiyanta, 2021), and IoT-based security systems using face recognition (Putra et al., 2023).Addressing challenges like uncertain equipment availability and return delays, this system uses IoT for automatic record-keeping and real-time tracking, while face recognition ensures secure access.Drawing on prior developments in face recognition and IoT in asset management, this system aims to enhance efficiency, reduce return delays, and improve equipment security and tracking.The system's design includes a return box for borrowed equipment, combining security and automated management to minimize human error and equipment loss.

METHODS
The methodology for this research encompasses various data collection techniques, such as literature review, interviews, and direct observations.This is followed by data analysis techniques to evaluate the effectiveness of the developed IoT tool.This includes analyzing the face recognition system's accuracy in identifying authorized users, assessing the efficiency of equipment lending by examining lending patterns and return times, and analyzing security risks associated with face recognition and equipment storage systems.These analyses are vital in ensuring the system's reliability and effectiveness in managing equipment lending and returns, maintaining security, and minimizing potential vulnerabilities.
The work procedure involves system analysis, system design, and algorithm development for face recognition, software design, system integration, testing for functionality, integration, and reporting of the project.The system analysis stage, the first step in this project, comprises two main components: problem analysis and needs analysis.Problem analysis aims to gather information on existing issues in electronic equipment management.Needs analysis then identifies the tools and materials required to build a new system.This determination is based on the problems identified during the problem analysis phase.This approach ensures that the solution developed is tailored to address specific challenges in electronic equipment management effectively.
In the design phase of the project, the proposed tool is conceptualized through a series of block diagrams and flowcharts.These diagrams aim to visualize the system's functions and overall workflow, while the flowcharts detail the operational process step-by-step.This phase also involves designing the electronic circuitry and PCB, determining the placement and connections of electronic components, and mechanical design, which outlines the physical and ergonomic aspects of the tool.  1) representing the electronic tool management system's architecture.This system harnesses a Camera for image acquisition, an ESP32 Cam module for initial processing connected via Wi-Fi, indicating a wireless setup.The central processing unit, a computer, analyzes the captured image data, making access control decisions.A Relay, controlled by the system, operates a Magnetic Solenoid to manage the locking mechanism of the tool storage, allowing access only to authorized users verified through facial recognition.This integration is essential for ensuring that the system's reliability and effectiveness in managing equipment lending and returns are maintained, with a focus on security and minimizing potential vulnerabilities.The design phase aims to encapsulate the system's functionality and workflow visually, providing a clear blueprint for the electronic and mechanical aspects of the proposed solution.After the design phase, constructing a breadboard prototype marks a pivotal step in the validation process.This initial circuit includes a 9V battery powering the ESP32-CAM module, the principal component for the system's face recognition capabilities.A relay module interfaces with the ESP32-CAM, modulating the magnetic solenoid to regulate access in response to the recognition of authorized users.Integrating LEDs to denote system status is essential for real-time monitoring.This stage assures the system's functional integrity prior to finalizing the PCB design (Figure2), verifying the correct functioning of all electrical components from the power source to the signal processing tasks.
The development of the face recognition algorithm involved the use of the Python programming language.The script began by importing necessary packages for system operations, including numpy for array operations, cv2 for image processing with OpenCV, as well as other modules for thread management and network communication.Global variables were configured to set parameters such as PORT and connection configurations, as well as the path to the XML Haar Cascade file used for face detection.
Two threads operate concurrently within the algorithm.The 'capture' function, running on the first thread, is responsible for setting up a socket to listen for data and, upon receipt, reassembling the image and executing face detection.This involves receiving fragmented image data, reconstructing it, and using the facial recognition algorithm to determine if an identified face matches an authorized user.
The 'packet_mgmt' function, on the second thread, manages the image reconstruction from the received data packets.It employs OpenCV's Cascade Classifier to transform the images into grayscale for face detection.Any detected faces are then highlighted with rectangles on the frame, and if video recording is enabled, these frames are sent to the 'ffmpeg' process for recording.This dual-thread approach allows the system to process visual inputs in real-time and carry out face recognition tasks effectively, a critical component for system functionality that underpins the entire electronic tool management system.
Both threads were initialized and executed, allowing the system to process visual input in real-time and efficiently perform face recognition tasks.The algorithm's processes are represented in the following code: Software design focuses on creating an intuitive user interface and developing use cases to encompass all system usage scenarios, with thorough testing for usability.The user interface design was developed using Figma and Flask API, an interactive design tool, to ensure flexibility in adapting to the chosen template.The provided visual representations serve as an initial conceptual overview of the user interface that will be further developed.This interface is designed to include four main pages: the sign-in page, the item list page, the borrowing process, and the return page.Below is a visual representation of these use cases:  Flask API is configured to manage user data, products, and loan transactions.This API is designed to handle requests from the ESP32 Cam and send appropriate responses based on user data verification and loan status.The integration process between the ESP32 Cam and Flask API is then detailed, explaining how the ESP32 Cam sends facial recognition data to the Flask API and receives instructions to control access based on user status verification.The structure of the Flask API code is as follows: The system design integrates modern technology with functional security to provide a system that manages equipment lending through a sophisticated and automated process.Face recognition mechanism integrated with the camera is mounted on the top surface of the smart box, which contains the ESP32 Cam module with various other components already assembled.The camera is responsible for image acquisition.It is a key component in the facial recognition process, capturing images of individuals trying to access the smart box.The cover of the smart box is equipped with a magnetic solenoid for physical security of the system.This lock is operated by a relay controlled by the output of the facial recognition process.Only upon positive identification of the authorized user will the relay trigger the solenoid to open the lock, granting access to the compartment inside.This ensures that only authorized personnel can access the secure storage, thereby improving the overall safety and efficiency of electronic equipment management.

RESULTS AND DISCUSSION
The research incorporates a thorough analysis of system requirements and issues in electronic equipment management, leading to the design of an IoT-based security system with ESP32-Cam.In system development, it is important to conduct a system analysis stage.This stage is the basis for designing the system to be built.This analysis process is divided into two main aspects, namely problem analysis and needs analysis.These two aspects are important in identifying and understanding the problems that the designed system will solve.
Problem analysis is the initial stage for the manufacturing process, IoT-based tool management security system with ESP32-Cam.Requirement analysis involves the selection of hardware and software.Each component has a specific function that supports the entire process from design conception to physical realization of the tool.Required software includes Arduino IDE, Chrome, Eagle PCB, Fritzing, Fusion 360, and Visual Studio Code.The hardware selection facilitates the physical design and integration of components.The ESP32 acts as a central controller, processing data from the OV2640 Camera Sensor, which captures facial recognition images.The FTDI programmer is critical for programming the ESP32-CAM, while the relay module controls the power to the solenoid lock, a key component in the smart door lock that activates upon successful face recognition.The 7805 regulator ensures stable voltage supply to the ESP32-CAM and other system components.The following is a picture of the system that has been created: Referring to the image provided, there is an aperture for the camera and an LED indicator.The aperture ensures that the camera can capture an image of the user's face for the purpose of returning the tool.Along the side, there are designated ports for adapter cables as well as connections that link the PCB to the relay and magnetic solenoid.The magnetic solenoid will be placed on the smart door lock as security.
Beyond the hardware aspect, the creation of a website plays a pivotal role as it functions as a hub for the borrowing and return of equipment.The seamless integration of user data into the website is crucial, streamlining the user experience by simplifying transaction management and equipment tracking.This digital interface provides a direct and uncomplicated means for users to interact with the system.

DASHBOARD USER PAGE
LOAN PAGE RETURN PAGE Figure 6.All Features on The Website The provided image showcases a comprehensive web interface designed for administrative control over equipment lending and returns within an educational or corporate setting.The interface includes a dashboard for a quick overview of transactions and system alerts, a user management page to handle user data and privileges, and dedicated pages for tracking and managing loans and returns.Admins have the capability to add, edit, or delete user and product data, as well as update the status of loans and returns, directly impacting the operation of the associated IoT-enabled smart box.This system ensures a streamlined process for equipment management, keeping the lending service efficient and secure.It ensures that all user interactions, from initial authentication to the final stages of equipment return, are efficiently logged and processed, thereby simplifying the entire procedure.
The testing phase includes a series of tests designed to identify and fix bugs, evaluate performance, and verify that the system meets all established requirements.Testing is conducted in a variety of use scenarios that include standard cases of operation as well as edge or extreme conditions.The testing process that will be carried out is functional, integration, and security testing.The table below presents the testing data for the developed system:

Pass
The research demonstrates the system's potential in enhancing the management and security of equipment lending in educational settings.By automating the process and incorporating advanced face recognition, the system addresses key challenges such as equipment availability and return delays.Its design and implementation show a significant step forward in using IoT and face recognition technologies for practical, real-world applications.

CONCLUSION
This research successfully developed an Integrated Face Recognition and IoT-Based Electronic Equipment Management System, demonstrating a significant advancement in managing and securing equipment lending processes.The smart box system, with its automated record-keeping and real-time tracking capabilities, not only improves operational efficiency but also ensures secure access to equipment.The system's design and implementation mark a significant stride in the practical application of IoT and facial recognition technologies, showcasing their potential to revolutionize asset management and security.The research's findings and developed system present a promising direction for future innovations in IoT-based security solutions.

Figure
Figure 1.Block Diagram Block diagram (Figure1) representing the electronic tool management system's architecture.This system harnesses a Camera for image acquisition, an ESP32 Cam module for initial processing connected via Wi-Fi, indicating a wireless setup.The central processing unit, a computer, analyzes the captured image data, making access control decisions.A Relay, controlled by the system, operates a Magnetic Solenoid to manage the locking mechanism of the tool storage, allowing access only to authorized users verified through facial recognition.This integration is essential for ensuring that the system's reliability and effectiveness in managing equipment lending and returns are maintained, with a focus on security and minimizing potential vulnerabilities.The design phase aims to encapsulate the system's functionality and workflow visually, providing a clear blueprint for the electronic and mechanical aspects of the proposed solution.

Figure 3 .
Figure 3. User Interface Design System integration entails connecting and configuring hardware and software components to ensure stable communication.The development process begins with programming the ESP32 Cam.This includes initializing the camera, setting up the Wi-Fi connection, and configuring the GPIO for relay control and LED indicators.The ESP32 Cam is programmed to run a camera server capable of capturing images and processing facial recognition data.The corresponding code structure is outlined as follows:

Figure 4 .
Figure 4. Mechanical Design of The System

Figure 5 .
Figure 5. Physical Image of The System

Table 1 .
List of Components and Specifications

Table 2 .
Test Cases for The Smart Box Door Mechanism