Webcam model face recognition. ESP32-CAM Video Streaming and Face Recognition with Arduino IDE

Face Detection and Tracking Using Live Video Acquisition

This example shows how to automatically detect and track a face in a live video stream, using the KLT algorithm.

Overview

Object detection and tracking are important in many computer vision applications including activity recognition, automotive safety, and surveillance. In this example you will develop a simple system for tracking a single face in a live video stream captured by a webcam. MATLAB provides webcam support through a Hardware Support Package, which you will need to download and install in order to run this example. The support package is available via the Support Package Installer.

The face tracking system in this example can be in one of two modes: detection or tracking. In the detection mode you can use a vision.CascadeObjectDetector object to detect a face in the current frame. If a face is detected, then you must detect corner points on the face, initialize a vision.PointTracker object, and then switch to the tracking mode.

In the tracking mode, you must track the points using the point tracker. As you track the points, some of them will be lost because of occlusion. If the number of points being tracked falls below a threshold, that means that the face is no longer being tracked. You must then switch back to the detection mode to try to re-acquire the face.

Setup

Create objects for detecting faces, tracking points, acquiring and displaying video frames.

% Create the face detector object. faceDetector = vision.CascadeObjectDetector; % Create the point tracker object. pointTracker = vision.PointTracker(‘MaxBidirectionalError’, 2); % Create the webcam object. cam = webcam; % Capture one frame to get its size. videoFrame = snapshot(cam); frameSize = size(videoFrame); % Create the video player object. videoPlayer = vision.VideoPlayer(‘Position’, [100 100 [frameSize(2), frameSize(1)]30]);

Detection and Tracking

Capture and process video frames from the webcam in a loop to detect and track a face. The loop will run for 400 frames or until the video player window is closed.

runLoop = true; numPts = 0; frameCount = 0; while runLoop frameCount 400 % Get the next frame. videoFrame = snapshot(cam); videoFrameGray = im2gray(videoFrame); frameCount = frameCount 1; if numPts 10 % Detection mode. bbox = faceDetector.step(videoFrameGray); if ~isempty(bbox) % Find corner points inside the detected region. points = detectMinEigenFeatures(videoFrameGray, ‘ROI’, bbox(1, :)); % Re-initialize the point tracker. xyPoints = points.Location; numPts = size(xyPoints,1); release(pointTracker); initialize(pointTracker, xyPoints, videoFrameGray); % Save a copy of the points. oldPoints = xyPoints; % Convert the rectangle represented as [x, y, w, h] into an % M-by-2 matrix of [x,y] coordinates of the four corners. This % is needed to be able to transform the bounding box to display % the orientation of the face. bboxPoints = bbox2points(bbox(1, :)); % Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4] % format required by insertShape. bboxPolygon = reshape(bboxPoints’, 1, []); % Display a bounding box around the detected face. videoFrame = insertShape(videoFrame, ‘Polygon’, bboxPolygon, ‘LineWidth’, 3); % Display detected corners. videoFrame = insertMarker(videoFrame, xyPoints, ”, ‘Color’, ‘white’); end else % Tracking mode. [xyPoints, isFound] = step(pointTracker, videoFrameGray); visiblePoints = xyPoints(isFound, :); oldInliers = oldPoints(isFound, :); numPts = size(visiblePoints, 1); if numPts = 10 % Estimate the geometric transformation between the old points % and the new points. [xform, inlierIdx] = estgeotform2d(. oldInliers, visiblePoints, ‘similarity’, ‘MaxDistance’, 4); oldInliers = oldInliers(inlierIdx, :); visiblePoints = visiblePoints(inlierIdx, :); % Apply the transformation to the bounding box. bboxPoints = transformPointsForward(xform, bboxPoints); % Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4] % format required by insertShape. bboxPolygon = reshape(bboxPoints’, 1, []); % Display a bounding box around the face being tracked. videoFrame = insertShape(videoFrame, ‘Polygon’, bboxPolygon, ‘LineWidth’, 3); % Display tracked points. videoFrame = insertMarker(videoFrame, visiblePoints, ”, ‘Color’, ‘white’); % Reset the points. oldPoints = visiblePoints; setPoints(pointTracker, oldPoints); end end % Display the annotated video frame using the video player object. step(videoPlayer, videoFrame); % Check whether the video player window has been closed. runLoop = isOpen(videoPlayer); end % Clean up. clear cam; release(videoPlayer); release(pointTracker); release(faceDetector);

References

Viola, Paul A. and Jones, Michael J. Rapid Object Detection using a Boosted Cascade of Simple Features, IEEE CVPR, 2001.

Bruce D. Lucas and Takeo Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision. International Joint Conference on Artificial Intelligence, 1981.

Carlo Tomasi and Takeo Kanade. Detection and Tracking of Point Features. Carnegie Mellon University Technical Report CMU-CS-91-132, 1991.

Jianbo Shi and Carlo Tomasi. Good Features to Track. IEEE Conference on Computer Vision and Pattern Recognition, 1994.

Zdenek Kalal, Krystian Mikolajczyk and Jiri Matas. Forward-Backward Error: Automatic Detection of Tracking Failures. International Conference on Pattern Recognition, 2010

Open Example

You have a modified version of this example. Do you want to open this example with your edits?

ESP32-CAM Video Streaming and Face Recognition with Arduino IDE

This article is a quick getting started guide for the ESP32-CAM board. We’ll show you how to setup a video streaming web server with face recognition and detection in less than 5 minutes with Arduino IDE.

Note: in this tutorial we use the example from the arduino-esp32 library. This tutorial doesn’t cover how to modify the example.

Related project: ESP32-CAM Video Streaming Web Server (works with Home Assistant and Node-Red)

Watch the Video Tutorial

You can watch the video tutorial or keep reading this page for the written instructions.

Parts Required

To follow this tutorial you need the following components:

You can use the preceding links or go directly to MakerAdvisor.com/tools to find all the parts for your projects at the best price!

Introducing the ESP32-CAM

The ESP32-CAM is a very small camera module with the ESP32-S chip that costs approximately 10. Besides the OV2640 camera, and several GPIOs to connect peripherals, it also features a microSD card slot that can be useful to store images taken with the camera or to store files to serve to clients.

The ESP32-CAM doesn’t come with a USB connector, so you need an FTDI programmer to upload code through the U0R and U0T pins (serial pins).

Features

Here is a list with the ESP32-CAM features:

• The smallest 802.11b/g/n Wi-Fi BT SoC module
• Low power 32-bit CPU,can also serve the application processor
• Up to 160MHz clock speed, summary computing power up to 600 DMIPS
• Built-in 520 KB SRAM, external 4MPSRAM
• Supports UART/SPI/I2C/PWM/ADC/DAC
• Support OV2640 and OV7670 cameras, built-in flash lamp
• Support image Wi-Fi upload
• Support TF card
• Supports multiple sleep modes
• Embedded Lwip and FreeRTOS
• Supports STA/AP/STAAP operation mode
• Support Smart Config/AirKiss technology
• Support for serial port local and remote firmware upgrades (FOTA)

ESP32-CAM Pinout

The following figure shows the ESP32-CAM pinout (AI-Thinker module).

There are three GND pins and two pins for power: either 3.3V or 5V.

GPIO 1 and GPIO 3 are the serial pins. You need these pins to upload code to your board. Additionally, GPIO 0 also plays an important role, since it determines whether the ESP32 is in flashing mode or not. When GPIO 0 is connected to GND. the ESP32 is in flashing mode.

The following pins are internally connected to the microSD card reader:

• GPIO 14: CLK
• GPIO 15: CMD
• GPIO 2: Data 0
• GPIO 4: Data 1 (also connected to the on-board LED)
• GPIO 12: Data 2
• GPIO 13: Data 3

Video Streaming Server

Follow the next steps to build a video streaming web server with the ESP32-CAM that you can access on your local network.

Important: Make sure you have your Arduino IDE updated as well as the latest version of the ESP32 add-on.

Install the ESP32 add-on

In this example, we use Arduino IDE to program the ESP32-CAM board. So, you need to have Arduino IDE installed as well as the ESP32 add-on. Follow one of the next tutorials to install the ESP32 add-on, if you haven’t already:

CameraWebServer Example Code

In your Arduino IDE, go to File Examples ESP32 Camera and open the CameraWebServer example.

The following code should load.

Before uploading the code, you need to insert your network credentials in the following variables:

const char ssid = REPLACE_WITH_YOUR_SSID; const char password = REPLACE_WITH_YOUR_PASSWORD;

Then, make sure you select the right camera module. In this case, we’re using the AI-THINKER Model.

So, comment all the other models and uncomment this one:

// Select camera model //#define CAMERA_MODEL_WROVER_KIT //#define CAMERA_MODEL_ESP_EYE //#define CAMERA_MODEL_M5STACK_PSRAM //#define CAMERA_MODEL_M5STACK_WIDE #define CAMERA_MODEL_AI_THINKER

If none of these correspond to the camera you’re using, you need to add the pin assignment for your specific board in the camera_pins.h tab.

Now, the code is ready to be uploaded to your ESP32.

ESP32-CAM Upload Code

Connect the ESP32-CAM board to your computer using an FTDI programmer. Follow the next schematic diagram:

Many FTDI programmers have a jumper that allows you to select 3.3V or 5V. Make sure the jumper is in the right place to select 5V.

Important: GPIO 0 needs to be connected to GND so that you’re able to upload code.

To upload the code, follow the next steps:

1) Go to Tools Board and select AI-Thinker ESP32-CAM.

2) Go to Tools Port and select the COM port the ESP32 is connected to.

3) Then, click the upload button to upload the code.

4) When you start to see these dots on the debugging window as shown below, press the ESP32-CAM on-board RST button.

After a few seconds, the code should be successfully uploaded to your board.

Getting the IP address

After uploading the code, disconnect GPIO 0 from GND.

Open the Serial Monitor at a baud rate of 115200. Press the ESP32-CAM on-board Reset button.

The ESP32 IP address should be printed in the Serial Monitor.

Accessing the Video Streaming Server

Now, you can access your camera streaming server on your local network. Open a browser and type the ESP32-CAM IP address. Press the Start Streaming button to start video streaming.

You also have the option to take photos by clicking the Get Still button. Unfortunately, this example doesn’t save the photos, but you can modify it to use the on board microSD Card to store the captured photos.

There are also several camera settings that you can play with to adjust the image settings.

Finally, you can do face recognition and detection.

First, you need to enroll a new face. It will make several attempts to save the face. After enrolling a new user, it should detect the face later on (subject 0).

And that’s it. Now you have your video streaming web server up and running with face detection and recognition with the example from the library.

[eBook] Build ESP32-CAM Projects using Arduino IDE

Learn how to program and build 17 projects with the ESP32-CAM using Arduino IDE DOWNLOAD »

Learn how to program and build 17 projects with the ESP32-CAM using Arduino IDE DOWNLOAD »

Wrapping Up

The ESP32-CAM provides an inexpensive way to build more advanced home automation projects that feature video, taking photos, and face recognition.

In this tutorial we’ve tested the CameraWebServer example to test the camera functionalities. Now, the idea is to modify the example or write a completely new code to build other projects. For example, take photos and save them to the microSD card when motion is detected, integrate video streaming in your home automation platform (like Node-RED or Home Assistant), and much more.

We hope you’ve find this tutorial useful. If you don’t have an ESP32-CAM yet, you can grab it here.

If you like this project, you may also like other projects with the ESP32-CAM:

[eBook] Build Web Servers with ESP32 and ESP8266 (2nd Edition)

Build Web Server projects with the ESP32 and ESP8266 boards to control outputs and monitor sensors remotely. Learn HTML, CSS, JavaScript and client-server communication protocols DOWNLOAD »

Build Web Server projects with the ESP32 and ESP8266 boards to control outputs and monitor sensors remotely. Learn HTML, CSS, JavaScript and client-server communication protocols DOWNLOAD »

Recommended Resources

Build a Home Automation System from Scratch » With Raspberry Pi, ESP8266, Arduino, and Node-RED.

Home Automation using ESP8266 eBook and video course » Build IoT and home automation projects.

Arduino Step-by-Step Projects » Build 25 Arduino projects with our course, even with no prior experience!

Getting Started with ESP32 Bluetooth Low Energy (BLE) on Arduino IDE

Enjoyed this project? Stay updated by subscribing our newsletter!

401 thoughts on “ESP32-CAM Video Streaming and Face Recognition with Arduino IDE”

TF card 4GB limit. Will larger capacity cards, i.e. 8GB work, but only 4GB will be usable? Smaller cards are getting harder to find. FAT-16 format required? Reply

Hi Bruce. I haven’t tested with 8GB SD cards. I’ll need to check if those work too. It needs to beet FAT-32 format. Regards, Sara Reply

Can you help me because my ESP 32cam keeps showing this again and again and it doesnt show the IP address… Also my ESP 32 CAM doesnt have AI-THINKER marked in it. Do you have any ideas about this? I badly need your help. Carst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1216 ho 0 tail 12 room 4 load:0x40078000,len:9720 ho 0 tail 12 room 4 load:0x40080400,len:6352 entry 0x400806b8 Reply

Hi. It seems that your board is booting constantly. Try pressing the RST button several times to see if it solves the problem. If it doesn’t, double-check that you’re powering the ESP32-CAM with 5V on the 5V pin (not VCC). It may also help taking a look at our troubleshooting guide bullet 2 and 3: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ I hope this helps. Regards, Sara Reply

Hi, i am planning to create a attendance system using esp32. Do u have suggestion that i might follow? i really appreciate the help thanks

Probably because of to low voltage under upload, you have to use 5 volt not 3.3 volt on your ftdi uploader. I made the same error and got same result as you did. After setting ftdi to 5 volt it worked fine. Reply

I don’t understand how it is possible to use 5V with the ESP 32. As far as I know is the maximum voltage for the GPIO-Pins 3.3 – 3.6V. How is it still possible that the ESP32 doesn’t get fried?

im getting the same error.but i have try so many times clicking the Rst button still the same.how to fix it.have u fix the issue. please help me. Reply

ok. the more simple question is: if your SSID network name and password is correct. this was my error on the past…:-( Reply

Hi Sara, can you tell me what are the variable of x, y, w, and h? and how to get those variable? Reply

do we really need micro SD card for face recognition, cause im not using it and i cant enroll my face Reply

Thanks very much for this ESP32-CAM project, I am looking forward to learning the camera applications, it is my first. Unfortunatly I am getting the following error returned to the serial monitor after reset: SCCB_Write [ff]=01 failed SCCB_Write [12]=80 failed [E][camera.c:1085] esp_camera_init: Camera probe failed with error 0x20001 Camera init failed with error 0x20001 I have updated the arduino IDE to 1.8.9 and ESP32 boards as per instructions, but cant find the problem. If you have any ideas I really appriecate it. Reply

Hi James. Did you select the right camera module in the code? Please double check that your camera is well connected to the board. I also found this issue: github.com/espressif/esp32-camera/issues/5 It seems the same as yours, so it might help. Regards, Sara Reply

Hi James. Did you ever get a resolution on this problem? I purchased two units and they are responding the same. Reply

Hi Dan, yes I took Sara’s advice and selected the correct camera module in the code but commenting out the ones that don’t apply. I did also find reducing the upload speed made things more stable. I think my programmer is not the best. Very happy it works very well. Thanks again Reply

can you help me how to modify code to push stream to my public proxy url on the internet? I want to make a page which accessible publicly and don’t want to have public IP for my local network. so needs esp to stream to my proxy url itself Reply

Hi Dan, did you found the solution. I also purchase two units with different brand with same issue. (the first one have successed before but when retry to reupload the issue came). Try all suggestions here by changing board selection, changging cable, changging programmer device, changging pins selection, try with different PC and all have same problem. Reply

I Had the Same Camera init failed with error 0x20004. I powered the ESP32 with 5v and works great. May try the 5V to see if it goes away. THANKS For this great site and tutorials. Reply

Any update on card sizes. Brand name 4 GB cards are special order. When I find 4GB they are almost as expensive as 16/32GB sizes. Ebay takes forever anymore, and then you don’t know what you are getting. No name brand on Ali Express or Banggood. Reply

Hi Sarah, do you have a code that use esp32 cam as qr code scanner and send to webpage with Database?? Please for my school project purpose Reply

Hi. Great tutorial; worked like a charm once used a separate 5V supply. Any way you know of to see the video stream or stills via a TFT display on another ESP through web browser or otherwise? I’ve used ESPNow between ESP12’s or 32’s for display of thermal cam images but they’re much smaller. Avoids need for phone or laptop tied up…. Thanks Mel Reply

Hi, thanks for the tutorial, but I’m getting 2 problems with the code : 1. I can’t include the zip file through “Add.ZIP library” from Arduino IDE 2. When I put it manually through extracting the zip file and moved it to my Arduino libraries folder, then compile the code, I got “no headers files (.h) found” error Any help would be appreciated, thanks again for the tutorial. Reply

Hi Mario. You don’t need to install any library. You just need to have the ESP32 add-on installed. The zip file that we provide contains all the code that you need. You just need to unzip the file, open the CameraWebServer folder and open the CameraWebServer.ino. Your arduino IDE should open the code and you’ll see three tabs at the top. Then, you just need to upload the code to your board. Alternatively, if you have the latest updated ESP32 add-on, you should have the code in your examples. Go to File Examples ESP32 Camera and open the CameraWebServer example. I hope this helps. Regards, Sara Reply

i got this error when upload the code on esp32-cam using ftdt programer A fatal error occurred: Invalid head of packet (0x65) Reply

Hi, nice tutorial, Can I with ESP32CAM store not manually pictures and for example check if have a car in picture? Reply

I was looking for something like this for my recent project, Thanks! Great tutorial! But I think ESP32-CAM is “unofficial” combination of ESP32 with a camera. I think Espressif themselves released a dedicated “official” ESP32camera board called ESP-EYE with their own “official” software library called ESP-WHO. Well I got all the information from here: https://www.ebay.com/itm/254177708782 Have not tried that board myself. Can you make a tutorial on that as well since that is the “official” hardware and software and would have longer support from Espressif itself. Also a comparison between the 2 would be great too. I follow a lot of Random Nerd Tutorials. You guys make easy to follow guides. Cheers! Keep it up! Thanks! Reply

Hi Ryan. Thank you for your nice words. The ESP-EYE is an Espressif release. We haven’t fully tested the ESP-EYE yet. We’ve played with the example firmware that they provide and we made a blog post about it that you can read here: https://makeradvisor.com/esp-eye-new-esp32-based-board/ At the moment, we don’t have any more tutorials with the ESP-EYE. Thank you for your interest in our content. Regards, Sara Reply

Brownout detector was triggered ets Jun 8 2016 00:22:57 rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:10088 load:0x40080400,len:6380 entry 0x400806a4 What happened? ¿Qué ocurre? Thx Reply

Hi David. That’s usually a power issue. Please see our troubleshooting guide bullet 8: https://randomnerdtutorials.com/esp32-troubleshooting-guide/ Regards, Sara Reply

Hello. I faced the same problem with a recent FTDI. Replacing it by an older one (with a large USB connector), the problem has been fixed. Could be it is the same for you. Regards. F.Thomart Reply

Hi David Same thing happened to me. The brownout reset occurs when the power supply to the ESP32 processor is below a minimum level. Try to check the cable, connections, power source … etc. If you can, try to measure the voltage that goes directly to the pins on the ESP32CAM board. It should be the closest to 5 Volt. Reply

Thanks, it will be of great help, recently I was able to integrate my esp32 cam into an MQTT client library, every face detected a publisher is sent to the broker Reply

That’s a great project. Will you publish your project somewhere? Many people here might be interested. Regards, Sara Reply

I have not shared it yet, but I can post here if you wish, I used the pubsubclient library to transform esp32cam into a Mqtt client

It would be very useful if you share your project, also trying to do the same. Thanks in advance Felipe! Reply

Greetings and congratulations for the tutorial. You are a very nice couple. Is it possible to take this captured image to a server on the internet? Can I have this camera in my house and see what happens from my work? Thank you. Reply

Hi Rui Sarah, How do you set up face recognition ? I have the whole thing working as expected, however the Enroll face button does nothing ? Reply

It seems that face recognition is no longer working (at least with the example program) when using the 1.02 ESP core. Rolling back to the 1.01 core and using the example program belonging to that core, will ‘fix’ it (currently that is the program that Sara and Rui have on their Github Reply

Hi, how do i roll back to the 1.01 core ? i have the same problem of Neil, the Enroll face button does nothing. can you help? Reply

Hi Guys, I purchased two units and both fail with the following: rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:10088 load:0x40080400,len:6380 entry 0x400806a4 [D][esp32-hal-psram.c:47] psramInit: PSRAM enabled [E][camera.c:1085] esp_camera_init: Camera probe failed with error 0x20001 Camera init failed with error 0x20001 I’ve selected AI Thinker in the code and reduced the upload to 115200. Anyone have some insights? I have a M5Stack Camera which works pretty well with the code but these two are dead. Thanks Dan Reply

Hi! good tutorial!, I need to put the upload speed 115200 and the flash frequency in 40Mhz to avoid a Guru Meditation Error: Core 0 panic’ed (InstrFetchProhibited) error if someone have the same problem Reply

Hi Leonardo. thank you for sharing. It will definitely be useful for many people. Regards, Sara Reply

Heyy hai how to count number of faces it is getting detected ?? like the count should be displayed Reply

Got my cameras today – your tutorial above works perfectly! Any idea how to turn on the “flash light” LED? Thanks much, ben Reply

There is a small red led (GPIO33 inverted). The main led is controlled by GPIO4. In the example CamWebServer program there is AFAIK no possibility in the webserver to switch the main LED. Should be possible though to program Switching the LED and control it via say HASSio or OpenHab, with an MQTT command or something. If there are any unused pins, you could add a switch Reply

How about: arduinodiy.wordpress.com/2019/08/03/turning-led-off-and-on-on-the-esp32-camera-module-using-bluetooth/ Reply

I am having problems getting errors: camera_probe: Detected camera not supported. esp_camera_init: Camera probe failed with error 0x20003. That occurs selecting AI Thinker. The other two options give me the 0x20001 error. I bought the esp camera from DIYMORE.CC. The description in their ad prints AI Thinker on the chip, but my actual device does not have AI Thinker printed. It just has DM-ESP32-S. Any ideas? Reply

Did you find a solution or the correct IDE setting for your DM ESP32? I have the same modules but haven’t used them yet. I’d appreciate your input. Reply

i have the same DM board, used the same IDE settings as mentioned here, no problem with the arduino sample, except must use 5v power otherwise will keep getting brownout error Reply

Thanks Great Job But i have almost the same problem as Neil. face recognition works very bad i get almost no yellow square how to ficks that? Reply

Hi. Face recognition is a bit slow, however we managed to make it work fine. Please make sure that you have proper lighting to make the face recognition process easier and more efficient. Also, when enrolling a new face, you need to be steady and don’t move much, so that it properly saves your face features and can recognize it in the future. Regards, Sara Reply

Hi, Got my hardware last week from banggood. It had the issue “Brownout detector was triggered”. Seaching the web i found this video where they say to feed by 5v not 3.3v. https://www.YouTube.com/watch?v=tzmcXZ-irIc ~2:30 This solved the brownout issue for me. Then the web service did not appear in google chrome browser. Error message was something about too much header lines or so. In MS Edge it was ok. But i have no image from the cam. Cam must be broken. So i have to wait another month to get this as spare part. Have also ordered another ESP board with an external antenna hoping to get better connection to the router. Reply

Hi Patrick. I’m sorry you’re getting trouble using your ESP32-CAM. The brownout detector error usually means that the ESP32 is not being powered properly. You can read more about this on our troubleshooting guide, bullet 8: https://randomnerdtutorials.com/esp32-troubleshooting-guide/ Our camera worked flawlessly following the steps we describe in our tutorial. The ESP32-CAM should work fine being powered either with 3.3V through the 3.3V pin or 5V through the 5V pin. You’re probably not providing enough current. Also, we didn’t have any trouble accessing the web server on Google Chrome. After you get a new camera, let us know how it went. Regards, Sara Reply

Any ideas what would cause a 20003 error? I have tried all three camera types. The AI Thinker gives 20003. The other two cause a 20001 error Reply

Hi John. I’m sorry you’re having that issue. Those errors usually mean that the camera is not properly connected. So, or your camera module is faulty or it is not properly connected. If these are not the reasons, it is very difficult for us to understand what is going on. Can you try using a new camera probe? Regards, Sara Reply

Thanks. The camera came installed. I bought 2 of them, and they both fail. I decided to buy from another source and see if that works. I am not sure what you are referring to regarding a new camera probe. Reply

Dear ALL ESP32 doesn´t connect with mit Network and no text in Serial Monitor is being printed. SID and PW changed in coding. Any Ideas? Message in Arduino 1.8.8: Der Sketch verwendet 2233514 Bytes (71%) des Programmspeicherplatzes. Das Maximum sind 3145728 Bytes. Globale Variablen verwenden 50692 Bytes (15%) des dynamischen Speichers, 276988 Bytes für lokale Variablen verbleiben. Das Maximum sind 327680 Bytes. esptool.py v2.6-beta1 Serial port COM9 Connecting……. Chip is ESP32D0WDQ6 (revision 1) Features: Wi-Fi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None MAC: cc:50:e3:b6:e5:90 Uploading stub… Running stub… Stub running… Configuring flash size… Auto-detected Flash size: 4MB Compressed 8192 bytes to 47… Writing at 0x0000e000… (100 %) Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 4096.1 kbit/s)… Hash of data verified. Compressed 17664 bytes to 11528… Writing at 0x00001000… (100 %) Wrote 17664 bytes (11528 compressed) at 0x00001000 in 1.0 seconds (effective 138.4 kbit/s)… Hash of data verified. Compressed 2233680 bytes to 1788374… Wrote 2233680 bytes (1788374 compressed) at 0x00010000 in 158.5 seconds (effective 112.7 kbit/s)… Hash of data verified. Compressed 3072 bytes to 134… Writing at 0x00008000… (100 %) Wrote 3072 bytes (134 compressed) at 0x00008000 in 0.0 seconds (effective 768.0 kbit/s)… Hash of data verified. Leaving… Hard resetting via RTS pin… Reply

Hi. It seems that your code was uploaded successfully. Make sure you open the serial monitor at a baud rate of 115200, so that you can see the text on the serial monitor. After uploading the code, you should disconnect GPIO from GND. Open the Serial monitor, and then press the ESP on-board reset button. Please make sure you’ve inserted the right network credentials. Can you access the web server when you insert the IP address on your browser? Regards, Sara Reply

Dear Sara, may I ask you please to advise on the issue below. I purchased an AI Thinker, but it is not printed on the chip. This product contains the OV2640 Camera Module. Can you please advise on which Camera Model to use? The use of #define CAMERA_MODEL_AI_THINKER refers to the error. Also the other led to issues. Thanks Brownout detector was triggered ets Jun 8 2016 00:22:57 rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6400 entry 0x400806a8 [E][camera.c:1049] camera_probe: Detected camera not supported. [E][camera.c:1249] esp_camera_init: Camera probe failed with error 0x20004 Reply

Hi Christian. That’s probably a power issue. Please read bullet 8 of our ESP32 troubleshooting guide: https://randomnerdtutorials.com/esp32-troubleshooting-guide/ Regards, Sara Reply

Check the camera pinout here: github.com/m5stack/m5stack-cam-psram/blob/master/README.md I had to change pins 22 and 25 in camera_pins.h for the M5STACK_PSRAM Reply

Hi! I had same issue and that fixed but does not understand why. if I set it like this: #define CAMERA_MODEL_AI_THINKER why should it g to the case: #elif defined(CAMERA_MODEL_M5STACK_PSRAM). Can you explain? thanks! Reply

Thank you very much for sharing. Using M5STACKcam I didn’t had image. After troubleshooting and comparing with other codes I changed setting for Y2_GPIO_NUM to 17. Now it works like a sharm using ESP32 DevModule with Huge APP for partition scheme. Reply

I faced different problems getting the module working. Since I am using the 5V-supply pin (instead of the 3.3V on the CAMERA_MODEL_AI_THINKER) everything is OK. Reply

Hi. I have an esp32-cam and i went throught all the process to program the board and everything was going fine. At the end i’ve got the message telling me the IP adress to connect my board so i did in my browser and i ‘ve got the viewer that appeared in the screen but but when i press start stream or get still i don’t have any image on the screen ! I tried with 2 boards and still the same problem. The only things in common is the software … Any idea ? Thanks. Reply

Hello Patrick, unfortunately I can’t replicate that error on my end… The default CameraWebServer scripts works fine for me out of the box. Regards, Rui Reply

Patrick, I have the same problem. After I hit the “Start Stream” button, no image shown on the screen. Have you resolved the problem ? Regards, Ong Kheok Chin Reply

Hi Patrick, I have the same problem. I have a Windows machine using Windows 10. I think the problem might be in the Windows firewall. My camera streams fine on my Android phone. Maybe someone can help us setup the Windows firewall. If I can get it figured out, i’ll let you know. Reply

Hi, i’m stuck right at the beginning with Arduino IDE 1.8.9 I have to select the board before i see any ESP32 examples – chose ESP32 Wrover module, however examples do not include Camera – any ideas? Thanks Reply

Hi Mike. I’m sorry you’re facing that problem. I don’t know why that is happening. But you can try to download the example from our repository: https://github.com/RuiSantosdotme/arduino-esp32-CameraWebServer Regards, Sara Reply

Just for anybody else having the same problem: Choose board ‘AI thinker ESP32-Cam’ Then in Examples go to “ESP32” and then “Camera” After that you can alter the selected board again Reply

Nice tutorial, everything worked. Could you please show us how we can broadcast the video stream to the internet (so that we can see the video from any computer)? Maybe using port forwarding of the ESP32-cam or using a dedicated service? It would also be great to have an example working offline to record the video on a SD card (I haven’t managed to do that). Thanks! Reply

Hi Oli. At the moment, we don’t have any tutorial about that subject. We’ve also been trying to use the SD card to save photos and record video, but at the moment, without success. Regards, Sara Reply

Howdy Folks, I am getting this major bug in my serial monitor after disconnecting the GPIO0 cable and resetting it: Brownout detector was triggered ets Jun 8 2016 00:22:57 rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6400 entry 0x400806a8 Guru Meditation Error: Core 0 panic’ed (LoadProhibited). Exception was unhandled. Core 0 register dump: PC : 0x4012fea1 PS : 0x00060031 A0 : 0xca400000 A1 : 0x3ffe3ac0 A2 : 0x3ffaff7c A3 : 0x00000080 A4 : 0x3ffbf0ec A5 : 0x40090858 A6 : 0x02ffffff A7 : 0x00000c00 A8 : 0x4008f290 A9 : 0x3ffe3a90 A10 : 0x3ffbf0ec A11 : 0x000000fe A12 : 0x00000001 A13 : 0x00000000 A14 : 0x00000000 A15 : 0x00000000 SAR : 0x0000001d EXCCAUSE: 0x0000001c EXCVADDR: 0x03000283 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0xffffffff Backtrace: 0x4012fea1:0x3ffe3ac0 0x4a3ffffd:0x3ffe3ae0 0x400dea6d:0x3ffe3ba0 0x400de992:0x3ffe3bc0 0x40083ec3:0x3ffe3bf0 0x400840f4:0x3ffe3c20 0x40078f2b:0x3ffe3c40 0x40078f91:0x3ffe3c70 0x40078f9c:0x3ffe3ca0 0x40079165:0x3ffe3cc0 0x400806da:0x3ffe3df0 0x40007c31:0x3ffe3eb0 0x4000073d:0x3ffe3f20 Rebooting… unhandled. Guru Meditation Error: Core 0 panic’ed (StoreProhibited). Exception was unhandled. Guru Meditation E⸮ets Jun 8 2016 00:22:57 rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6400 entry 0x400806a8 Any Ideas? Reply

Hello Kurt, here’s what the error: “Brownout detector was triggered” means: When you open your Arduino IDE Serial monitor and the error message “Brownout detector was triggered” is constantly being printed over and over again. It means that there’s some sort of hardware problem. It’s often related to one of the following issues: – Poor quality USB cable; – USB cable is too long; – Board with some defect (bad solder joints); – Bad computer USB port; – Or not enough power provided by the computer USB port. Solution: try a different shorter USB cable (with data wires), try a different computer USB port or use a USB hub with an external power supply. Reply

Rui, I am using a USB CH340 and also a USB FTDI serial boards that connect directly to a Computer USB port, there are no cables, other than the Jumper wires. I have tried this on 3 different computers and about 3 to 4 USB ports on each one. I have also tested 2 CAM boards with the exact same results. The Brownout is the only thing listed on my previous post, there’s also the: “Guru Meditation Error: Core 0 panic’ed (StoreProhibited). Exception was unhandled. ” Which spawn 60 TO 100 Messages before it Reboots. Reply

Hi. Some of our readers reported that when they power the ESP32-CAM with 5V, they don’t have the brownout error or guru meditation error anymore. Regards, Sara Reply

When I powered either one of them with 5V through the USB Serial dongle the LED on the ESP board lights up and stays on, while the Serial monitor shows nothing.

HI all, I purchased a ESP32-Cam. I have had a lot of problems trying to get it to work. I could nbot get the sketch to upload and a couple of other small issues. What I found was ( Its all to do with the voltages…. and the pin configuration is different on my USB-TTl compared to the pics on the web. ) – 1. Set the USB-TTl to 3.3V. 2. connect it to the ESP32-CAM as shown in all the diagrams, (but put the 3.3V from the USB-Tl to 3.3V on the ESP32-CAM.) 3. Strap the Io0 and gnd make sure the pins you have cables on are correct… very important. 4. Power up and upload the sketch. Now to test the ESP32-CAM. 1. Remove the IO0 and gnd jumper. 2. Change the USB-TTl to 5v (changing the pin) 3. Change the voltage on the ESP32-CAM to 5V pin. 4. Power up. 5. Open up the serial monitor. 6. Press the reset button on the ESP32-CAM. 7. get the IP address. Enter the IP address in your browser. Go to the bottom to Start streaming data. And It works like a charm. If I do not change the voltage on the pins (3.3v for uploading sketch and %v for operating then I could not get anything to work. I hope this helps other people who are having Issues. Reply

Wonderful tutorial, quick set up….I have 1 little issue…Stills OK, Steaming NOT OK…. Everything seems to work well and good but when I press Start Steam, nothing streams. I can tell through the Monitor, and TTL connection that the Steaming mode is going, and when I stop the monitoring shifts down to lower FPS. Still captures work just fine. Am I missing something? Do I need an SD card installed to allow streaming? Arduino 1.8.9, ESP32 Espressif v1.0.2 Reply

Hi. You don’t need SD card to see the streaming. I don’t known what can be the problem. Please note that you can only see the streaming on one client at a time. So, make sure that you don’t have any other browser tab making requests to the streaming URL. I’m sorry that I can’t help much. Regards, Sara Reply

I am facing the following error while uploading code. Please help A fatal error occurred: Failed to connect to ESP32 cam: Timed out waiting for packet header Reply

You probably don’t have the right connections to the FTDI programmer. Also GPIO 0 needs to be connected to GND while uploading the code. Regards, Sara Reply

I edit code to use esp32 as accesspoint. on serial monitor show: IP address: 192.168.4.1 Starting web server on port: ’80’ Starting stream server on port: ’81’ Camera Ready! Use ‘http://192.168.4.1’ to connect E (5687) Wi-Fi: addba response cb: ap bss deleted Reply

Hi. Unfortunately, I don’t know what that message means. If you find out, please share with us. Regards, Sara Reply

Hola a alguien le ha dado el siguiente error [E][sccb.c:154] SCCB_Write: SCCB_Write Failed addr:0x30, reg:0x23, data:0x00, ret:-1 20:59:56.233. [E][camera.c:1215] camera_init: Failed to set frame size 20:59:56.233. [E][camera.c:1270] esp_camera_init: Camera init failed with error 0x20002 No se como solucionarlo, agradesco vuestra ayuda, saludos. Reply

Hi Arturo. Next time, post your questions in english so that everyone can understand. Which camera board are you using? Reply

Hello, sorry for my previous message in Spanish. the problem is generated on a model plate ESP32-S Al-Thinker. Reply

Hi ARturo. That error you were referring to usually means that the camera is not properly connected or the ESP32 is not able to recognize the camera. That can be due to the following issues: – Camera not connected properly: the camera has a tiny connector and you must ensure it’s connected in the the right away and with a secure fit, otherwise it will fail to establish a connection – Not enough power through USB source: Some ESP32-CAM boards required 5V power supply to work properly. We’ve tested all our examples with 3.3V and they worked fine. However, some of our readers reported that this issue was fixed when they power the ESP32-CAM with 5V. – Faulty FTDI programmer: Some readers also reported this problem was solved by replacing their actual FTDI programmer with this one: https://makeradvisor.com/tools/ftdi-programmer-board/ – The camera/connector is broken: If you get this error, it might also mean that your camera or the camera ribbon is broken. If that is the case, you may get a new OV2640 camera probe. Also, sometimes, unplugging and plugging the FTDI programmer multiple times or restart the board multiple times, might solve the issue. I hope this helps. regards, Sara Reply

Hi I did everything as explained and if I get IP and I can enter and start the camera but when selecting the face dectector does not work does not happen nothing does not detect the faces, I have remained still to see if it detects the face and does not work. esp32 I have it connected to the 5v pin because when I tried it with 3.3v I did not want to load the code Reply

I had the same problem and the camera needs to be in good lighting conditions to get it to do any of the recognition functions……. Reply

hello I did everything as established, I charge the code and it gives me the IP and the entry in my browser and if it enters the platform of the camera and I can start the camera only that when selecting for the face detector it does not work I have been still to see if it detects but nothing appears, and if you notice that the quality of the camera is somewhat low and I do not know if that could be the cause, there is no way to turn on the led that includes the esp32 cam to work as flash Reply

Hi Jesus. What is the camera module that you’re using? If the camera board doens’t have PSRAM, it won’t be able to do face recognition and detection. Regards, Sara Reply

The face recognition and detection should work with that camera. Did you follow Neil suggestions? You really need to have good lighting, otherwise it won’t be able to recognize faces. Regards, Sara Reply

Hello! Excellent tutorial, got me started real easy with the ESP32-Cam. I did got a bit stuck though: – First time I uploaded the CameraWebServer sample sketch, the upload process worked fine, though I could not see any traces back in the serial, even removing the GPIO 0 to GND jumper and resetting. – Then I tried to upload again and got only an error back: esptool.py v2.6 Serial port COM5 Connecting……._…._…._…._…._…._…._ A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header – Since I had a second ESP32-CAM, I repeated the steps above, and the results were the same: first upload from the IDE succeeded, the next one failed with the error above. – I did try to change the upload speed to 115200 bps, but it did not change the results – I did not (yet) try pressing the ‘reset’ button in the board, because it is in the back side and I have the module in a protoboard. Since the first upload worked without pressing reset, I’m not sure I need to do it this time, but I’m open for suggestions Thoughts? Reply

Update: it was indeed the ‘reset’ button underneath; if anyone is facing the same problem, just remember to briefly hit the reset button as you’re about to upload the compiled firmware. Everything is working fine here now, thanks again for this nice tutorial. Reply

Hi Claudio, Yes, you need to press the reset button, otherwise you won’t be able to upload code. Regards, Sara Reply

Hi, I want to thank you for all your articles, I learned a lot on this site. Following this tutorial my ESP32 Cam worked the first try. Now the part where I have some problems: I would like to connect some device through I2C like a BME280, a stepper motor and 2 relay but I have some difficult to locate the right pins (if available). Could you help me? TIA, Vince Reply

Hi Vince. If you intend to use the SD card, there aren’t pins left (at least accessible pins). If you don’t use the SD card, there are some pins available, but I haven’t experimented with those yet. You can see the datasheet to check the internal connections to the pins: https://loboris.eu/ESP32/ESP32-CAM%20Product%20Specification.pdf (page 4) You can see the pinout here: https://randomnerdtutorials.com/wp-content/uploads/2019/03/ESP32-CAM-pinout-1.png I hope this helps. Regards, Sara Reply

Thank you for your reply. My need is to understand how many pins are left unused. It seems that GPIO0.4 and 12.16 are already used by the cam so no other device could be used. Maybe some other GPIO could be used connecting directly to ESP32 pins. Regrds, Vince Reply

Hi again. GPIO16, GPIO 2 and GPIO 3 are not being used by the camera. However, GPIO 2 and GPIO 3 are used for serial communication. But you can try with those. Regards, Sara Reply

I have 2 boards and cams and with both i have the same problem: rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6400 entry 0x400806a8 ………………………………………………………. I get endless dots. that’s it. Camera does not init. If I remove the cams from the boards that is detected and an error is printed. Reply

Hi Mirko. That usually happens when people forget to insert their network credentials or don’t insert the credentials properly. Please make sure that you’ve inserted your network credentials and double-check that they’re correct. Also, make sure that the ESP is relatively close to your router so that it is able to catch Wi-Fi signal. Some readers reported that powering the ESP32-CAM board with 5V solved the problem. Regards, Sara Reply

Thank you. That solved the problem. I was always thinking the ESP32 is opening up a own Wi-Fi hotspot and so inserted credentials for that. I did not realize that it wants to connect to my Wi-Fi and needs that credentials. I did not even think about that, because I thought that the ……. is a part of the camera initialisation So again, thanx for the hint. Mirko Reply

hello I still have the same problem that facial recognition does not work when I start it in the arduino ide serial monitor I start marking this MJPG: 8205B 209ms (4.8fps), AVG: 210ms (4.8fps), 1346100=196 0 MJPG: 8220B 208ms (4.8fps), AVG: 210ms (4.8fps), 1336100=195 0 MJPG: 8234B 207ms (4.8fps), AVG: 210ms (4.8fps), 1336100=195 0 MJPG: 8253B 208ms (4.8fps), AVG: 210ms (4.8fps), 1336100=195 0 MJPG: 8258B 239ms (4.2fps), AVG: 211ms (4.7fps), 1366200=198 0 MJPG: 8244B 282ms (3.5fps), AVG: 215ms (4.7fps), 1346200=196 0 but nothing appears in the view of the camera and if I give it in enrroll face sometimes I throw this error Guru Meditation Error: Core 0 panic’ed (LoadProhibited). Exception was unhandled. Core 0 register dump: PC : 0x40132f33 PS : 0x00060c30 A0 : 0x801333fb A1 : 0x3ffd5090 A2 : 0x3ffc73fc A3 : 0x00000000 A4 : 0x00000000 A5 : 0x00000000 A6 : 0x00000008 A7 : 0x00600002 A8 : 0x80132ea4 A9 : 0x3ffd5070 A10 : 0x00000000 A11 : 0x0000000b A12 : 0x00000005 A13 : 0x00000020 A14 : 0x00000020 A15 : 0x3ffbe140 SAR : 0x00000020 EXCCAUSE: 0x0000001c EXCVADDR: 0x00000001 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0xffffffff Backtrace: 0x40132f33:0x3ffd5090 0x401333f8:0x3ffd50c0 0x401334a0:0x3ffd50f0 0x40133755:0x3ffd5120 0x40094c89:0x3ffd5150 0x4008dae1:0x3ffd5190 Reply

It is also worth to say, that powering the Unit just from the Serial Converter leads to problems (at me) because the Module needs more/quicker Power than my Serial-Converter Module is able to deliver as you can see sometimes on the Serial-Monitor if there is “Brownout Detection ….” I just power it from any other “good” Source to work against the “inrush current” that the Module aparently needs to kick in with Wi-Fi. Reply

Hello, thank you for posting this material, it is very explanatory. I would like to report a problem with the ESP32-CAM I’m using. The image was stuck and locked. So I switched the voltage to 5V and now it works fine. Thank you Reply

Hi Tiago. Thanks for sharing. We now have a troubleshooting guide with the most common problems and how to fix them: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

Hi guys. Thanks a lot for this tutorial. I’m using the esp32-cam without problems. The only question i have for you is: is there any way to rotate the image in 90º? Thanks again! Reply

Hello, I am having trouble with my diymore esp32 cam. I believe it is a dev module so this is what I pick under boards (there is nothing that says diymore). I am getting connection timeouts using my adafruit programmer friend wired up the same way as the diagram. Using 3v3. Any suggestions? Reply

Hi David. Can you try powering your board through the 5V pin and see if it solves the problem? Regards, Sara Reply

I was able to get my sketch uploaded to the DIY more board using 3v and 40mhz. The 5v to run the sketch Reply

Has anyone had any luck in integrating this tutorial with MQTT? I’d like to be able to publish a notification via MQTT to a topic when a recognised face is detected so I can integrate this with my Home Automation System – Thanks Reply

Hi Jonathan. We intend to work on something like that in the future. But at the moment we haven’t experimented with it yet. Meanwhile you can take a look at our MQTT tutorial: https://randomnerdtutorials.com/esp32-mqtt-publish-subscribe-arduino-ide/ Regards, Sara Reply

Thanks for your post! i´m from Brazil and i trying using a board ESP32CAM of DIY but its no work…my first projeto with ESP32CAM was a AI-Thinker and works fine… But when i using ESP32CAM DIY not work. Maybe ESP32CAM DIY its a difrent pinout? Reply

Hi Stefano. I have no idea why one works and the other doesn’t. Some users reported that some boards required 5V to operate. That can be the case. You can also take a look at our troubleshooting guide and see if it helps: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

my problem as that : A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header any solutions pleaseee. thanks Reply

Hi! can you help me how to modify code to push stream to my public proxy url on the internet? I want to make a page which accessible publicly and don’t want to have public IP for my local network. so needs esp to stream to my proxy url itself Reply

Hello, My board is behaving little strange. Did anybody have this kind of message: sptool.py v2.6 Serial port /dev/ttyUSB0 Connecting…. Chip is ESP32D0WDQ6 (revision 1) Features: Wi-Fi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None MAC: cc:50:e3:b6:db:fc Uploading stub… Running stub… Stub running… Changing baud rate to 921600 Changed. Configuring flash size… Warning: Could not auto-detect Flash size (FlashID=0x0, SizeID=0x0), defaulting to 4MB Compressed 8192 bytes to 47… Writing at 0x0000e000… (100 %) Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 4134.8 kbit/s)… A fatal error occurred: Timed out waiting for packet header A fatal error occurred: Timed out waiting for packet header Best Regards, Milan Reply

Hi Milan. That errors means that your ESP32-CAM is not in flashing mode. Please read our troublehsooting guide bullet 1: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ I hope this helps. Regards, Sara Reply

Hi Guys ! Thanks very much for this tutorial. pretty straight forward and concise. I’ve got my cameras from Aliexpress, they look very much alike to AI’s one. DM instead of AI is the brand that appears on the rfshield. I’ve got a Raspberry Pi to serve as a Wi-Fi HotSpot, assign the same IP to the ESP’s MAC address and from my mobile accessing the streaming. A bonus: Checking the schematics, I saw that it operates with 3.3v, so the 5v go to a LM1117-3.3v voltage regulator, and this 3.3v regulator is rated up to 15V input. Long story short, I’ve cramped 4 AAA batteries (6v) and the ESP32-CAM inside a GoPro-like waterproof enclosure and VOILAit worked… Underwater. at least surrounded by 3 ft of water :-). I had to lower the res down to 320×240 to keep the 23fps but still Guys, you’re awesome ! thanks again Gabriel Reply

Hi Gabriel. That’s awesome! Thank you for sharing your project! It would be great if you could send us some photos of your setup as well as how the images look underwater. Use our contact page and just say that you want to send your photos: https://randomnerdtutorials.com/contact/ Regards, Sara Reply

Hi, Excellent Tutorial, but I can start the camera only that when selecting for the face detector it does not work. some Idea? thank. I use 5V/2A, the image is very good, I use 320×240, and I use ESP32 CAM-module, this: https://loboris.eu/ESP32/ESP32-CAM%20Product%20Specification.pdf. but face detector dont work. thank you. Reply

Hi fabian. The example should work with your board. To be able to get face recognition, you should have good lighting conditions so that it can detect the faces. Without further information, it is very difficult to understand what might be the issue. You can also take a look at our troubleshooting guide and see if it helps in some way: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

it is my understanding that face recognition does not work in the 1.02 core of the ESP32. It does work in the 1.01 core. If you revert back to the 1.01 core, make sure you also use the Camera example that comes with that core Reply

For this problem: rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6400 entry 0x400806a8 [E][camera.c:1049] camera_probe: Detected camera not supported. [E][camera.c:1249] esp_camera_init: Camera probe failed with error 0x20004 solution apply 5V to the card, to the 5v pin Reply

Hi Leo. Thanks for sharing that tip. We’ve made a compilation with the most common problems and how to fix them: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ And that is included in our guide. Regards, Sara Reply

Hi, mine won’t detect faces for some reason. Do you have to install a MicroSD card for facial recognition? Reply

Is there any other way without port forwarding? I need some help about this. Can u help me about this? Reply

Can I send images from ESP 32 CAM to smartphone via bluetooth or USB so that I don’t have to connect to a network? Reply

Can you suggest where I should start in order to send image from ESP 32 CAM to smartphone via bluetooth or USB? Reply

Is there any other way without port forwarding? I need some help about this. Can u help me about this? Reply

Hello Rui and Sara, there is a litte led on the board. Do you know if it is possible to put it ON via gpio ? The camera will be installed in birdhouse (almost dark) and I woul’d like to have a little bit more light inside. Otherwise I wil use other ports to lit external leds. Thank you for your great job and in advance for your answer. Reply

Hi Bernard. The LED is connected to GPIO 4. So, you just need to make the usual procedures to put a GPIO on. pinMode(4, OUTPUT); digitalWrite(4, HIGH); Regards, Sara Reply

Thanks a lot, but this little led have not enough power to give good light. I used extra leds strips to do the job via a wemos d1. Regards, Bernard Reply

Hi Bernard. Yes, that LED is not enough for a good light. That’s a good idea. Thank you for sharing. Regards, Sara Reply

Hi everyone. Nice tutorial you’ve got here. I’m working on a door security system that would require a cam to take a picture of a face, compare it with already registered images on a database and have it trigger a lock mechanism on successfull validation. (without streaming or accessing via Wi-Fi.) Would this be possible with esp32 cam? Thanks. Reply

Hello everyone, Can any one help i am getting following error while uploading the code. Arduino: 1.8.9 (Windows 10), Board: “ESP32 Wrover Module, Huge APP (3MB No OTA), QIO, 80MHz, 921600, None” Sketch uses 2241942 bytes (71%) of program storage space. Maximum is 3145728 bytes. Global variables use 52696 bytes (16%) of dynamic memory, leaving 274984 bytes for local variables. Maximum is 327680 bytes. esptool.py v2.6 Serial port COM8 Connecting…. Chip is ESP32D0WDQ5 (revision 1) Features: Wi-Fi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None MAC: 24:0a:c4:bb:65:c4 Uploading stub… Running stub… Stub running… Changing baud rate to 921600 Changed. Configuring flash size… Warning: Could not auto-detect Flash size (FlashID=0x0, SizeID=0x0), defaulting to 4MB Compressed 8192 bytes to 47… Writing at 0x0000e000… (100 %) Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 4369.0 kbit/s)… A fatal error occurred: Timed out waiting for packet header A fatal error occurred: Timed out waiting for packet header Reply

Hi. It seems that your board is not in flashing mode, so it is not able to upload the code. Please take a look at our troubleshooting guide, bullet 1: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ I hope this helps. Regards, Sara Reply

Hello, your site and your instructions are amazing,i believe that i do everything like you said in the video but it stops at this point. It doesent show me that it connects to the internet,i tried either with an antenna or without one. Please help me get through this if you can ets Jun 8 2016 00:22:57 rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6400 entry 0x400806a8 Reply

Hi Ilias. Please take a look at our troubleshooting guide and see if it helps: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

Hi guys… i have tired facing this problem.can you anyone please help me for solving this problem. I have got espressif ESP32-CAM two module. but i am unable to connect Camera,and i did not get any IP address with this module. thanks in advanced manu ets Jun 8 2016 00:22:57 rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6412 entry 0x400806a8 Camera init failed with error 0x20004ets Jun 8 2016 00:22:57 rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6412 entry 0x400806a8 ets Jun 8 2016 00:22:57 rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6412 entry 0x400806a8 ets Jun 8 2016 00:22:57 rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:9232 load:0x40080400,len:6412 entry 0x400806a8 Camera init failed with error 0x20004 Reply

Hi. Please take a look at our ESP32-CAM troubleshooting guide and see if it helps: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

Hi Sara Santos. In a comment from may you mention that you have tried taking photos and saving them to the SD card, but failed. I managed to do this. Do you want me to dig out the code and show it to you? I also managed to take photos when an “intruder” is detected from a sensor. The only problem with that is that I did not manage to connect the sensor directly to the camera module. I had to use an auxiliary Arduino board with the sensor, and make it then send a command to the ESP32 module to make it take a picture. I am pretty sure there are much better ways of doing this, ideally without needing an arduino board. Reply

Hi Antonio. Thank you so much for taking the time to read and answer to our Комментарии и мнения владельцев. Actually, one of our readers also shared a solution for that, and we end up writing a new tutorial about it. Here is the tutorial: https://randomnerdtutorials.com/esp32-cam-pir-motion-detector-photo-capture/ Regards, Sara Reply

Hi all. On a DIY Esp32-cam all I get from Arduino is board esp32 (platform esp32 package esp32) is unknown I installed the Esp32 addon, and tryed all the Esp32 boards on Arduino with the same result. I am missing something, I am sure. Thanks. Reply

Hi Federico. I’ve never faced that issue. I’ve found this discussion: github.com/espressif/arduino-esp32/issues/2388 See if some of the suggestions can help with your issue. Regards, Sara Reply

Hi Sara! Solved by removing all the esp32 stuff and reinstalling. Thanks. Have a nice WE. Federico Reply

Hi All, Nice Tutorial. Have not seen this issue posted anywhere. So here Goes: Followed tutorial, all worked perfectly until ESP32-Cam was removed from power. Then it acted like it had never been Flashed when power was restored. Even tried RST button, nothing shows up in the Serial Monitor. Can set back up to Flash and all goes well (all works) until power is removed then restored, again acts like it had never been Flashed. Bought 2 of these and both act the exact same way. Any help would be great. Thank You in Advance. CharlieBob Reply

Did you remove GPIO 0 from GND? If you leave that connection, the ESP32-CAM starts in FLASHING mode and it will not run your code… Reply

Hi Sara I connect esp32 cam with Lora but it cann’t be initiatized. It seems that deinit; of the esp32_cam doesn’t work as commented in esp32_camera.h. Please kindly suggest how to coexist cam and lora on this esp32_cam module. Many thanks, PP Reply

Hi PP. Most of the GPIOs exposed on the ESP32-CAM are either being used by the camera or by the microSD card. So, it will be very difficult to interface a LoRa module with this board. Regards, Sara Reply

Hi, Where i should buy this product? I am living in Denmark. I could not find suppliers for this product in my contry. Kind regards Salam Reply

Hi Salam. I have no idea. We usually buy our electronics components and boards from stores like eBay, Banggood, Aliexpress, Amazon, etc… https://makeradvisor.com/tools/esp32-cam/ Regards Sara Reply

i can’t run esp32cam, i tried to define all modules and nothing, help. My module has nothing written on the board, what manufacturer it is, how to detect what module it is Reply

Hi. That should light up the flash. Have you defined the pin as an output? pinMode(4, OUTPUT); digitalWrite(4, HIGH); Regards, Sara Reply

Oups, you’re right, I forgot setting the pin as output jejeje It’s been a long time since the last time I used an Arduino … Reply

How can i iuse that URL on other network? i want to access that camera on my mobile network, how can i do that. please answer. Reply

You’ll need to do some router port forwarding. Search for “router port forwarding” and you’ll find how to make a web server accessible from anywhere. Reply

Arduino IDE 1.8.10 The hardware (ESP, USB serial etc.) is the same as yours. Linux Mint (Xfce) I followed each and every step and this is what I get: Arduino: 1.8.10 (Linux), Board: “ESP32 Wrover Module, Huge APP (3MB No OTA), QIO, 80MHz, 921600, None” Traceback (most recent call last): File “/home/swift/.arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py”, line 37, in import serial ImportError: No module named serial Multiple libraries were found for “Wi-Fi.h” Used: /home/swift/.arduino15/packages/esp32/hardware/esp32/1.0.3/libraries/Wi-Fi Not used: /opt/arduino-1.8.10/libraries/Wi-Fi exit status 1 Error compiling for board ESP32 Wrover Module. This report would have more information with “Show verbose output during compilation” option enabled in File. Preferences. Reply

I’ve installed “pyserial” and I don’t get the error “No module named serial” but I get this: Arduino: 1.8.10 (Linux), Board: “ESP32 Wrover Module, Huge APP (3MB No OTA), QIO, 80MHz, 921600, None” Sketch uses 2097154 bytes (66%) of program storage space. Maximum is 3145728 bytes. Global variables use 53516 bytes (16%) of dynamic memory, leaving 274164 bytes for local variables. Maximum is 327680 bytes. esptool.py v2.6 Traceback (most recent call last): File “/home/swift/.arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py”, line 2959, in _main File “/home/swift/.arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py”, line 2952, in _main main File “/home/swift/.arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py”, line 2652, in main esp = chip_class(each_port, initial_baud, args.trace) File “/home/swift/.arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py”, line 222, in init Serial port /dev/ttyUSB0 self._port = serial.serial_for_url(port) File “/home/swift/.local/lib/python2.7/site-packages/serial/init.py”, line 88, in serial_for_url instance.open File “/home/swift/.local/lib/python2.7/site-packages/serial/serialposix.py”, line 268, in open raise SerialException(msg.errno, “could not open port : ”.format(self._port, msg)) serial.serialutil.SerialException: [Errno 13] could not open port /dev/ttyUSB0: [Errno 13] Permission denied: ‘/dev/ttyUSB0’ An error occurred while uploading the sketch When I run python.m serial.tools.list_ports in terminal I get this: /dev/ttyUSB0 1 ports found Reply

Just simply run it as root and it worked. Now I’ve got a problem with Brownout detector but nothing seems to be working :/ Reply

The FTDI programmer wasn’t able to supply 3,3V (only 2,7V), and 5V seemed to be too much for the ESP32. Now it works. Reply

The dl_lib.h is related to the face recognition capabilities 2 (esp-face), and it was removed in version 1.0.3 of the Arduino core. That said, just comment it out and it should compile and work perfectly either if you are using the Arduino IDE. Other option is to revert to version 1.0.2 of the arduino core. Regards Reply

In app_httpd.cpp #include “fb_gfx.h” #include “fd_forward.h” //#include “dl_lib.h” #include “fr_forward.h” Reply

Hi, open the webserver, but I press the “start stream” button and the failure to open the image, this message appears in serial ,, …… already tried in 3 browsers, can anyone help me? [E][camera.c:1344] esp_camera_fb_get: Failed to get the frame on time! Camera capture failed Reply

Raphael i added a solution over in https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide, but here it is in case you dont see it… A solution to the “esp_camera_fb_get: Failed to get the frame on time!” message…. Im using the ESP32-CAM Module 2MP OV2640 Camera sensor Module Type-C USB module from Aliexpress. Although not mentioned It doesn’t have the extra PSRAM the other M5 models do AND the camera has one changed IO pin. See here… https://github.com/m5stack/m5stack-cam-psram/blob/master/README.md and scroll down to Interface Comparison. The CameraWebServer Arduino example we’re probably all using doesnt have this ESP32-CAM model defined. You need to add it yourself eg in the main tab add #define CAMERA_MODEL_M5STACK_NO_PSRAM. and in the camera_pins.h tab add… #elif defined(CAMERA_MODEL_M5STACK_NO_PSRAM) #define PWDN_GPIO_NUM.1 #define RESET_GPIO_NUM 15 #define XCLK_GPIO_NUM 27 #define SIOD_GPIO_NUM 25 #define SIOC_GPIO_NUM 23 #define Y9_GPIO_NUM 19 #define Y8_GPIO_NUM 36 #define Y7_GPIO_NUM 18 #define Y6_GPIO_NUM 39 #define Y5_GPIO_NUM 5 #define Y4_GPIO_NUM 34 #define Y3_GPIO_NUM 35 #define Y2_GPIO_NUM 17 #define VSYNC_GPIO_NUM 22 #define HREF_GPIO_NUM 26 #define PCLK_GPIO_NUM 21 And you’re good to go. Also note that the max resolution of the bare ESP32-CAM Module is XGA1024x768 i assume also because of the lack of PSRAM. Reply

Hello Sara and Rui. Tried the Esp32 camera for the first time today. Sketch upload is only possible with 5 Volts. Also works very nicely and reliably. Thank you for your work. Greetings from the Netherlands from Bert. https://www.YouTube.com/user/nuonnuon?gl=NLhl=nlapp=desktop Reply

hello Sara and Rui …. i m finished this beautiful project.Everything working well but when i forwarding port and conect camera via internet,GET STLL working but VIDEO STREAM not ,maybe you know whats the problem. thanks in advance ,73 de 9a3xz Mikele Croatia Reply

Hi Mikele. I don’t know what can be the problem. In this example, video streaming only works on one client at a time. This means that if you have the web server opened in another tab, it will not work. Just one tab at a time. Thanks for following our work. Regards, Sara Reply

Hi again. Great turtorial again. My Hiletgo ESP32-Cam runs as a Ai-thinker. Noticed the image is mirror image (reversed right to left). Module design should have had the reset button on the camera’s side or a reset pin available. so it can work in a breadboard. Reply

Is it possible for facial recognition to send a signal to turn on a servo / LED? depending if it is intruder or subject Reply

Hi. Yes, it is possible. However, at the moment, we don’t have any tutorial about that. Regards, Sara Reply

Hi Sara, I am getting below error, please help…! Sketch uses 2100647 bytes (66%) of program storage space. Maximum is 3145728 bytes. Global variables use 53552 bytes (16%) of dynamic memory, leaving 274128 bytes for local variables. Maximum is 327680 bytes. esptool.py v2.6 Serial port COM12 Connecting…. Chip is ESP32D0WDQ5 (revision 1) Features: Wi-Fi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None MAC: 24:6f:28:46:97:64 Uploading stub… Running stub… Stub running… Configuring flash size… Warning: Could not auto-detect Flash size (FlashID=0x0, SizeID=0x0), defaulting to 4MB Compressed 8192 bytes to 47… Writing at 0x0000e000… (100 %) Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 5041.2 kbit/s)… A fatal error occurred: Timed out waiting for packet header A fatal error occurred: Timed out waiting for packet header Reply

Just received my ESP32-CAM Ai-Thinker board. Everything works fine except no ‘Toggle settings’ pane on the webpage. Perhaps I received a hacked firmware in mine or did I do something wrong? I’ve backed up the firmware with esptool. Does anyone have a.bin file from a board that shows the toggle settings pane? Thanks Rui and Sara for your work. Reply

I installed esp-idf and esp-who (https://github.com/espressif/esp-who), then built the example code ‘camera web server’ demo. I now have the settings pane. Reply

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1216 ho 0 tail 12 room 4 load:0x40078000,len:9720 ho 0 tail 12 room 4 load:0x40080400,len:6352 entry 0x400806b8 [E][sccb.c:154] SCCB_Write: SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:263 [E][sccb.c:154] SCCB_Write: SCCB_Write Failed addr:0x30, reg:0x12, data:0x80, ret:263 [E][sccb.c:119] SCCB_Read: SCCB_Read Failed addr:0x30, reg:0x0a, data:0x00, ret:263 [E][sccb.c:119] SCCB_Read: SCCB_Read Failed addr:0x30, reg:0x0b, data:0x00, ret:263 [E][camera.c:1049] camera_probe: Detected camera not supported. [E][camera.c:1249] esp_camera_init: Camera probe failed with error 0x20004 I get this error, any solution? Reply

Hi. The 0x2004 error means the camera is not supported. On your camera ribbon, which label do you have? Ours is LA AF2569 0927XA What label do you have in your camera model? Regards, Sara Reply

Hi Sara, I have two ESP32-CAM, both of then was working perfectly untill some time ago. Now, everytime that I upload a sketch to the modules, I get the this error message which is recurrently appearing here: [E][camera.c:1049] camera_probe: Detected camera not supported. [E][camera.c:1249] esp_camera_init: Camera probe failed with error 0x20004 I don’t know what is going on, because it happens in my two ESP32-CAM modules and they were working perfectly here in the last time I used then. The labels n the ribbon cable in both OV2560 cameras are: XRZ00D1.V240 V2.0 1903 Any ideas? Reply

Hi Antonio. It may be the Wi.Fi signal. Are you using the on-board antenna or an external antenna. If you’re using the on-board antenna, you need to be close to your router. The best way is to have an external antenna. Regards, Sara Reply

Hi Sara, I cannot get the IP address. Here is what I am getting at 115200. I moved close to the router, and added an external antenna. Please help. Reply

I had a similar problem. The fact that you get occasional words suggests that the baud rate is right. I forget which problems that I had were solved by what, but I started out powering the module on the Vin pin and having a power line connected from the TTL and I ended up cutting all the power lines to the TTL and powering the module on the 5V pin. Reply

Hi Antonio, Thanks for this example. I would want to know how to capture and send a base64 encoded image to external server. Reply

Hey. Have you tried this camera as an IP cam (softcam)? With own name and IP address? I tried a sketch but I don’t get a video image but also no IP address provided via the Serial monitor. This sketch is in IDE. Do you know him? Greetings (old) Bert Reply

Thanks, worked like a charm. Bought 4 of these clones for 15€, took me 10 minutes to set all of them up. Reply

Hello Mr Rui Santos Iam using in Example Arduino IDE – CameraWebServer Camera OV2640 – stack with high resolution camera resolution UXGA (1280 x 1024) Please select CIF or lower resolution before enabling this feature! then when iam try to get low resolution QVGA (320 x 240) face detection and face recognition done 1. how to make camera OV2640 with high resolution using face detection and face recognition ? when iam opnened the web IP Adress ESP32 Cam – button get still then iam check on program i cant find the button is 2. how to make button “Get Still” – save to SD Card ? then the code checked the image when false detected 0 3. how to read the image then checked the image scan detected=true ? like this web, when camera detect (show on micro SD Card – intruder alert) Reply

Hi. I’m sorry but I don’t have answers to all your questions. I recommend taking a look at all our ESP32-CAM projects and see if you find something that you can modify to use in your own projects. See all the projects here: https://randomnerdtutorials.com/category/esp32-cam/ Regards, Sara Reply

Hi, thanks for your tutorial. I followed all the steps above and get the IP address, but when I did it on browser it just can’t be reached. Whats going wrong? I have no idea whats happened. Reply

Hi. Does your ESP32-CAM have an external antenna? Or are you close to your router? If you don’t have an external antenna, the ESP32-CAM needs to be close to your router, so that it is able to catch the Wi-Fi signal. Read the section about the antenna “7. Weak Wi-Fi Signal” on our troubleshooting guide: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

Is it necessary to connect to the ESP32 Cam using the FTDI Programmer? I have a USB/TLS cable, meaning one side is USB, and on the other are red (VCC). black (gnd), white (TX), and green (RX) which I use frequently to connect upload to ESP8266 w/o the FTDI programmer. With the ESP32 Cam I tried connecting the USB/TLS cables as follows: Red VCC Black Gnd White U0T Green U0R No luck yet: there’s still the issue of getting the ESP32 into bootloader mode. There’s the Reset button, but I’m used the the ESP32 and ESP8266 where you need two buttons I’ve read you can get ESP into bootloader mode by grounding certain pins. Overall the question is: Can I flash the the ESP32 Cam using a USB/TLS? Reply

Answering my own question: ESP32 Cam can be flashed without the FTDI Programmer using a USB/TLS cable wiring as described above with one change: Red 5V (Thanks to RandomNerdTutorial diagram above / link below) Black Gnd White U0T Green U0R Getting into Boot mode, thanks to the same diagram was about grounding GPIO0, taping Reset, then releasing GPIO0. Great tutorial! Reply

Hi. That’s right! It doesn’t have to be an FTDI programmer. It can be a USB/TLS cable, as long as you have the right wiring. Regards, Sara Reply

Hey Sara i want Arduino to take pictures when i am out for walks for example and send them to a web server do you think this is possible? Reply

Hi. The FTDI programmer we’re using has a mini-USB port. So, we just connect a mini-USB to USB cable to the FTDI programmer and then to the computer. Regards, Sara Reply

Hellow sara santos I have problem can you help me please that when I connected esp32 with my arduino and upload the code from arduino ide to arduino and it uploaded but when I opened the serial monitor that the IP address not appear that write camera_probe: detected camera not supported esp_camera_init: camera probe failed with error 0×20004 So ,I can’t solved this problem can any one help me please … Reply

Hi. Please take a look at our troubleshooting guide. I’m sure it will help: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

Hello i have the ftdi i have the esp32-cam and females jumper wires my problem is how to i connect the ftdi programmer with my pc ? Reply

Hi Sara, After uploading the file [CameraWebServer] to the ESP32-CAM board, the following message is shown on the Serial Monitor. After the ESP32-CAM IP address is typed on the browser, no any video. Pressing the Start Streaming button, also no video. I try it on Win10 Win7 machine, same! I follow all your steps; AI Thinker board and ‘CAMERA_MODEL_AI_THINKER’ in the file are chosen. How to solve it ? message on Serial Monitor: (an SC card is inserted) rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1216 ho 0 tail 12 room 4 load:0x40078000,len:9720 ho 0 tail 12 room 4 load:0x40080400,len:6352 entry 0x400806b8 [E][sccb.c:154] SCCB_Write: SCCB_Write Failed addr:0x30, reg:0x91, data:0xa3, ret:-1 [E][sccb.c:154] SCCB_Write: SCCB_Write Failed addr:0x30, reg:0xff, data:0x00, ret:263 [E][sccb.c:154] SCCB_Write: SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1. Wi-Fi connected Starting web server on port: ’80’ Starting stream server on port: ’81’ Camera Ready! Use ‘http://192.168.1.110’ to connect Reply

Hello. I recently started using this ESP32cam programmer. This works a lot better and faster than the loose wires. I have no connection with this company. https://www.tindie.com/products/bitluni/cam-prog/ Greetings Bert. Reply

Hello. When I Verify/Compile I get this space error. I’ve not even connected my ESP32. Sketch uses 2053883 bytes (156%) of program storage space. Maximum is 1310720 bytes. Your trouble shooting mentions (for a space different error) Tools Partition Scheme, select “Huge APP (3MB No OTA)“. but I don’t have this in Tools. I’m using IDE 1.8.12 Thanks. Reply

Solved: I uninstalled the board and reinstalled it and this time I saw Huge APP… and it has now compiled. Reply

Thanks for the good tutorial, very helpful! I read that the logic level for the ESP should be 3.3 V! Not sure if you can fry your board with 5V logic levels. I had success using a FT232RL USB to TTL Serial Converter, using 5V from the side to power the ESP with cam and having the jumper set to 3.3V for the logic levels. Reply

Hi Thomas. You are right that you should use 3.3V with the ESP32-CAM. However, many of our readers had troubles when using 3.3V and those were solved when they used 5V instead. We didn’t have any problems when using one option or the other. Regards, Sara Reply

Hey. Have you tried this camera as an IP cam (softcam)? With own name and IP address? I tried a sketch but I don’t get a video image but also no IP address provided via the Serial monitor. This sketch is in IDE. Do you know him? Greetings (old) Bert Its stil not working, need help. Reply

Dear Sir very good work, it works for me, but the problem it is not saving recognized faces to microsd-card. Each time power on we have to start recognition again. To make sure there is no problem with the microsd-card or the board I try this “https://randomnerdtutorials.com/esp32-cam-take-photo-save-microsd-card/” and it works and save photos to microsd-card. So where is the problem. Reply

Ok thanks for your concern, do you know any other code to save and read faces to microSD card ?? or web help to do that ?? Reply

Ok I need a small help in web server I need to know where is reference to button “Start Stream” in code Why. = I need to start Stream with Device Startup without clicking on “Start Stream” button How to do this ?? Reply

TRY putting http://ipaddress:port/stream ipaddress – Your IP ADDRESS port – Stream port which is normally 81

Hi, the installation went fine and when I enter the IP address (in chrome) I had video; also the integration with Home assistant went smoothly – so far so good. I however don’t have the “camera streaming server” with the config buttons and sliders and I can figure out how to fix that. Ideas / clues would be very appreciated. Best Regards, Ko (Netherlands, The Hague) Reply

Just done with this project without any error. You people are doing great. Keep it up. One issue my eap32 cam is getting heated (the esp 32 side) so much, can you suggest why it’s happening. I am supplying power directly from a 12000mah power bank. Reply

Hi. That can happen if the ESP32-CAM is continuously streaming, specially during face detection and recognition. Regards, Sara Reply

Hi… How can I use face recognition without Wi-Fi? I mean: I’ld like to open/close a box with ESP32-CAM and Servo… Maybe I’ll need SD-card of course… Thanks and have a nice day. Reply

Hi, thank you very much for your tutorials. My second project has been the the esp32 camera. Just for knowing: your tip of connecting GPIO0 to GND when flashing is very important and solved my problem of “fatal error…” I would ask you if you have a tutorial for saving “faces” after a power loss, perhaps in your book? Thanks Reply

I encountered A HORRIBLE problem and would like to share it with you folks with a solution of course Problem is: If you notice every time we have to press the RESET (RST) button to start our ESP32 CAM so that your program works…Now what if for some reason the power to ESP32 CAM board goes down? And say in 5 mins the power is back ON to the board. Will you go and press the RESET (RST) button over and over to get your program started ? How to let your ESP32 CAM board auto reload (call setup function) when powered on without having to press the RESET (RST) button to get your code up and running. Answer: After you upload your code on the ESP32 CAM board, connect the GND wire to the GND pin which is near the 5V pin of the board.Voila. This will solve your problem. This seems like a hardware bug linked to C15 on the ESP32 board. Hope this tip helps all folks… Reply

I have already tried this but the esp32cam remains unstable. Sometimes no signal if the power is oke, but only again after a reset. Too often after start-up a reset must be given first. I have no idea how to fix this. I have 3 cam’s with the same problems. The power supplies are stable 5 volts / 2 amp. (al thinker cam) Reply

Hi. Your problem may be the antenna. Take a look at this article and see if it helps: https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/ Regards, Sara Reply

Last weekend we discovered that our router was no longer working properly. There is a new one and now the ESP32cam problems are suddenly gone. We hadn’t thought of that yet. Thanks for your comment. Reply

Hi i reset my ESP32-CAM after uploading,but to show this error. rst:0x1 (POWERON_RESET),boot:0x3 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_REO_V2)) waiting for download Reply

I have had a lot of trouble programming some ESP32 cameras which are not on the list of cameras in the software. I found that I can program those boards if I use an Arduino, instead of the FTDI cable. Doing this I can select the AI Thinker board. I have always had time outs when trying to use the FTDI cable. I just wanted to mention this for anyone else who is having trouble with the programming cable method. Reply

Hi Guys, I have two ESP32-CAM, both of then was working perfectly untill some time ago. Now, everytime that I upload a sketch to the modules, I get the this error message which is recurrently appearing here: [E][camera.c:1049] camera_probe: Detected camera not supported. [E][camera.c:1249] esp_camera_init: Camera probe failed with error 0x20004 I don’t know what is going on, because it happens in my two ESP32-CAM modules and they were working perfectly here in the last time I used then. The labels n the ribbon cable in both OV2560 cameras are: XRZ00D1.V240 V2.0 1903 I tested here my ESP32-CAM modules (conecting to the Wi-Fi, without initializing the camera) and they are working fine. I tryed many things here, including powering the ESP32-CAM with an external 5V source, but nothing worked so far. Is there any way to test if the camera is working (without the ESP32-CAM module)? Any ideas? Reply Reply

Hi Sara. Your tutorials are awesome! Keep going. I would like to know how to implement the same solution in this tutorial (and in this: https://randomnerdtutorials.com/esp32-cam-video-streaming-web-server-camera-home-assistant/) but using micropython. Can you help me? Thanks in advance. Best regards. Reply

Hi. Thanks for your comment. Unfortunatley, at the moment, we don’t have any tutorials about the ESP32-CAM with MicroPython. Regards, Sara Reply

Going to try this with Blynk. Can’t find ESP32 in my Arduino examples. Will try to update. I need an open GPpin Which one is open for use. Need to output HIGH and LOW. Don’t really care to store images only stream video. Reply

Got the ESP32 installed so can see board and examples. Can I use this with Blynk do you think and is there on open GPIO? Reply

Hi. Unfortunately, we don’t have any examples with the Blynk app. Thanks for your interest in our work. Regards, Sara Reply

Can a TFT display be connected to the ESP32 CAM to show Face Detection either instead of the web server, or both at the same time? Reply

Hi. It should be possible. The TTGO T-Camera Plus comes with an Example that displays the video streaming on the display: https://makeradvisor.com/ttgo-t-camera-plus-esp32-review-pinout/ However, we don’t have the code for that. Regards, Sara Reply

Could this be connected to Windows (7/10/etc)? I mean, I know that it can set the Wi-Fi config in order to attach to any newtwork but I don’t know if it could be detected for Windows as IP Cam without problems. Reply

Hallo Sara, Rui, Can camera OV5640 be included in the camera_index.h tab ? Groeten, Regards Tom Reply

And it works with OV3660. Why not with OV5640 ? Is there another sketch to start this camera ? Would be nice if you could help me. regards, Tom Reply

Hi from Italt. I use your example and it works like a charme ut I don’t understand how the esp32 cam streams video. Do not exist an url to se video stream or to have a snapshot? the only solution is to press button from web server? that’s all? I need if possible a simple url for video strem and snapshot. Thanks so much from Italy Reply

I answer myself: these are the two url i was searching for: – video stream: http://192.168.0.74:81/stream – snapshot: http://192.168.0.74:81/capture Thanks so much Reply

Hi Guys, I have two ESP32-CAM, both of then was working perfectly untill some time ago. Now, everytime that I upload a sketch to the modules, I get the this error message which is recurrently appearing here: [E][camera.c:1049] camera_probe: Detected camera not supported. [E][camera.c:1249] esp_camera_init: Camera probe failed with error 0x20004 I don’t know what is going on, because it happens in my two ESP32-CAM modules and they were working perfectly here in the last time I used then. The labels n the ribbon cable in both OV2560 cameras are: XRZ00D1.V240 V2.0 1903 I tested here my ESP32-CAM modules (conecting to the Wi-Fi, without initializing the camera) and they are working fine. I tryed many things here, including powering the ESP32-CAM with an external 5V source, but nothing worked so far. Is there any way to test if the camera is working (without the ESP32-CAM module)? This camera is making me mad. Any ideas? Reply

Hi, this is cool project. Can we train the esp32cam so can detect face with mask and no mask?would be interisting if we can do with small board. Reply

Hi. We don’t have any tutorial for that project. However, I’ve already seen some people doing it. Regards, Sara Reply

Hi sara, thanks for reply. Could you please give the link of other people project. So I can learn from it. Regards. Reply

Hi. I’m sorry, but I don’t have any links to show you. I’ve seen someone doing it on Instagram, I think, using an ESP32-CAM. However, with a quick search I couldn’t find any code. Probably, they haven’t shared the code online. I’m sorry that I can’t help much. Regards, Sara Reply

When it is all programmed and you are getting the control interface page but no video feed… do not forget to turn off NOSCRIPT in your browser Reply

Hello sara, nice work u done. I want to use same camera but in offline mode, I don’t have internet connection, can this work. Regards. Reply

Hi sara, I want to use this module (camera) for one application, that will be use in environment where internet connection is not available, can camera use in without internet connection. Regards. Reply

Have you seen the new esp32cam motherboard which is being sold on eBay very cheaply (search for esp32cam mb”? This is the only info. I have managed to find about it: hpcba.com/en/latest/source/DevelopmentBoard/HK-ESP32-CAM-MB.html It makes using the esp32cam so much easier as it functions just like any other esp32 development board with no wires, linking pins, removing power etc. The esp32cam does not have a reset pin and it seems the esp32cam supplied with the motherboard is a modified version where one of the GND pins has been changed to a reset when low pin despite still being labelled as GND. You can still use it with other esp32cam boards but you have to connect power whilst holding the program button to upload code. Mine will only work on slower serial upload speeds and the Wi-Fi signal is very poor whilst on the mother board (I am guessing this is why many of the suppliers offer the external antenna option) but well worth a look especially for first time users especially for the price. Reply

Hi. Thanks for the suggestion. It can make uploading the code a much easier task. Regards, Sara Reply

Dear Sara, Your link doesn’t “work”. this one does ! aliexpress.com/i/1005001727033068.html regards and Happy Save Holidays Reply

Hello, I have followed this tutorial multiple times using two separate board sets, but I continue to receive this message in the Serial Monitor window when pressing the RST button. rst:0x1 (POWERON_RESET),boot:0x3 (DOWNLOAD_BOOT(UART0/UART1/SDIO_REI_REO_V2)) waiting for download The compilation and uploading process appears to be successful; Sketch uses 2100647 bytes (66%) of program storage space. Maximum is 3145728 bytes. Global variables use 53552 bytes (16%) of dynamic memory, leaving 274128 bytes for local variables. Maximum is 327680 bytes. esptool.py v2.6 Serial port COM4 Connecting…. Chip is ESP32D0WDQ6 (revision 1) Features: Wi-Fi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None MAC: 10:52:1c:5d:94:4c Uploading stub… Running stub… Stub running… Changing baud rate to 460800 Changed. Configuring flash size… Auto-detected Flash size: 4MB Compressed 8192 bytes to 47… Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 4369.0 kbit/s)… Hash of data verified. Compressed 17392 bytes to 11186… Wrote 17392 bytes (11186 compressed) at 0x00001000 in 0.3 seconds (effective 515.3 kbit/s)… Hash of data verified. Compressed 2100768 bytes to 1661717… Wrote 2100768 bytes (1661717 compressed) at 0x00010000 in 39.8 seconds (effective 422.1 kbit/s)… Hash of data verified. Compressed 3072 bytes to 119… Wrote 3072 bytes (119 compressed) at 0x00008000 in 0.0 seconds (effective 1536.0 kbit/s)… Hash of data verified. Leaving… Hard resetting via RTS pin… Hopefully the problem is glaringly obvious and simple to correct! Your help would be much appreciated. Reply

Okay – I just remembered to remove the GND to IO0 jumper after programming – and the board runs. Reply

compiles and loads, however when it attempts to start the following message appears; “Invalid library found in C:\Users\Bruce\Documents\Arduino\hardware\espressif\esp32\libraries\AzureIoT: no headers files (.h) found in C:\Users\Bruce\Documents\Arduino\hardware\espressif\esp32\libraries\AzureIoT” Sure enough, the AzureIoT folder is empty. Your thoughts? Reply

Hi! I can’t find the FTDI programmer part in my country. Can i replace that part with arduino uno for example, or with something else? Thanks for answering! Reply

hello, very tempted by this device, I bought 3 ESP32-CAM set up on my Wi-Fi on which I already have standard webcams. The installation works, I get many images of a very honorable quality. The problem is that the ESP32 heat up a lot and stop. So I can’t use them as 24/7 surveillance devices. I tried to lower the resolution from SXGA, to XGA, then SVGA but it’s not satisfactory. Am I the only one with this type of problem? Is there any way to stop these overheats? Thank you for your feedback. Reply

Running Arduino 1.8.12 on Windows and I don’t see what you specify below CameraWebServer Example Code In your Arduino IDE, go to File Examples ESP32 Camera and open the CameraWebServer example Reply

Olá Tenho uma camara Ov2640 a funcionar num ESP32-CAM-MB Tentei substituir por uma camara ov5640, pois tem mais definição. Obtive erro: camara não compatível. Há maneira de instar a camara ov5640 num ESP32-CAM-MB? Reply

If anyone is having issues with the esp-32 CAM not coming out of sleep mode you have to change the 10K out for a smaller resistor size. There are some errors not talked about though that I don’t know how to fix like Timeout waiting for VSYNC happens randomly which requires turning the whole system off and on. Also one time my SD card was corrupted and I had to reformat it losing all the photos. Lastly the photos sometimes don’t come out right, like only the top of the photo comes in or the entire photo is unviewable. I think this issue may be caused by the camera going into sleep mode before the picture is done being taken though, so maybe increasing the delay time will fix it. Reply

Hi, ESP32-Cam is wonderful. It can be programmed with FTDI programmers of several kinds, however if I use an Arduino as a programmer (Tx to Tx and Rx to Rx) I have the following problem: With Arduino Duemilanove (FTDI driver) it works, Arduino Uno (CH340 driver) does not charge – does not communicate. I mention that: – ESP power supply is at 5V, – In series with Tx and Rx I put 1K resistors due to the voltage difference (3.3 – 5V) on the terminals. Does anyone have any idea. Thanks! Reply

good day I apologize for using the English language compiler. In any such manual there is the use of an SD card and I ask how to do it without an SD card? Thank you Reply

Hi. I’m sorry, but I’m not sure that I understood your question. This example works without the SD card. You don’t need to insert an SD card to make it work. Regards, Sara Reply

Hi everythings good but my program in serial monitor stops here itself rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:10088 load:0x40080400,len:6392 entry 0x400806a4 After this nothing appears neither the connection status nor the IP address Reply

Hi! Video Streaming works fine for me, but Face Recognition and Detection don’t work. When I click the ‘Face Recognition’ button, the video freezes. The ESP32 crashes and reboots, according to the serial monitor error message: No Match Found CORRUPT HEAP: Bad head at 0x3ffe29ec. Expected 0xabba1234 got 0x00000008 abort was called at PC 0x400889a1 on core 1 ELF file SHA256: 0000000000000000 Backtrace: 0x4008df7c:0x3ffe1bc0 0x4008e1f5:0x3ffe1be0 0x400889a1:0x3ffe1c00 0x40088acd:0x3ffe1c30 0x400d9eaf:0x3ffe1c50 0x400d6141:0x3ffe1f10 0x400d60d0:0x3ffe1f60 0x40093521:0x3ffe1f90 0x400890e2:0x3ffe1fb0 0x40088899:0x3ffe1fd0 0x4000bec7:0x3ffe1ff0 0x400d1df9:0x3ffe2010 0x40108619:0x3ffe21c0 0x40108d21:0x3ffe21f0 0x40108dd9:0x3ffe2280 0x401091ac:0x3ffe22a0 0x40107898:0x3ffe22c0 0x401078ef:0x3ffe2300 0x4008fe76:0x3ffe2320 Rebooting… ets Jun 8 2016 00:22:57 rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1216 ho 0 tail 12 room 4 load:0x40078000,len:10944 load:0x40080400,len:6388 entry 0x400806b4. Wi-Fi connected Starting web server on port: ’80’ Starting stream server on port: ’81’

Do you have any suggestions to get it working? Thanks in advance! Reply

Hi. What’s the board that you’re using? Does your board have PSRAM? What’s your ESP32 boards version installed in your Arduino IDE? Regards, Sara Reply

I use the ESP32-CAM Module with OV2640 Camera. For me, the problem was solved by downgrading the ESP32 Library version from 1.0.5 to 1.0.4. No idea why face recognition doesn’t work in 1.0.5. Reply

The error causes by the codes as shown below. static esp_err_t capture_handler static esp_err_t stream_handler free(net_boxes-score); free(net_boxes-box); free(net_boxes-landmark); free(net_boxes); Please find out the codes and modify as shown below. / free(net_boxes-score); free(net_boxes-box); free(net_boxes-landmark); free(net_boxes); / net_boxes = NULL; I modified the codes and face recognition worked well in 1.0.6.

You mention that the board supports both the OV2640 and the OV7670 cameras, but it appears that the software example you used here does not support the OV7670. Do you know of a software example for this board that uses the OV75670? Reply

Dont forget to turn off “Enable Client Isolation” in your access point / wireless router setting. If u checklist / enable those settings, your acess point won’t let you access another IP in your network Reply

esptool.py v2.6 Serial port COM4 Traceback (most recent call last): File “esptool.py”, line 2959, in File “esptool.py”, line 2952, in _main File “esptool.py”, line 2652, in main File “esptool.py”, line 222, in init File “site-packages\serialinit.py”, line 88, in serial_for_url File “site-packages\serial\serialwin32.py”, line 62, in open serial.serialutil.SerialException: could not open port ‘COM4’: WindowsError(5, ‘Access is denied.’) Failed to execute script esptool the selected serial port Failed to execute script esptool does not exist or your board is not connected Reply

or please make sure no other application is opening the same port – happened to me a few times Reply

Hi, face detection and face recognition dont work. Should we have SD card inserted when we do face recognition? Do you have an example of complete code for face detection and recognition? Ty!! Reply

Hi. You need to use the ESP32 Boards add-on version 1.0.4. It doesn’t work with earlier versions. At least, at the moment. Regards, Sara Reply

Hi, I might have found another issue with this. Here is the output of my Arduino. Not sure why this is happening. BTW, the instead on using the ground pin next to UOT pin, I.m using the ground pin next to the 5V pin instead, Arduino: 1.8.15 (Mac OS X), Board: “AI Thinker ESP32-CAM, 240MHz (Wi-Fi/BT), QIO, 80MHz” Sketch uses 2594390 bytes (82%) of program storage space. Maximum is 3145728 bytes. Global variables use 56256 bytes (17%) of dynamic memory, leaving 271424 bytes for local variables. Maximum is 327680 bytes. esptool.py v3.0-dev Serial port /dev/cu.usbserial-1410 Connecting……. Chip is ESP32-D0WDQ6 (revision 1) Features: Wi-Fi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None WARNING: Detected crystal freq 41.47MHz is quite different to normalized freq 40MHz. Unsupported crystal in use? Crystal is 40MHz MAC: 9c:9c:1f:ea:5d:d0 Uploading stub… Running stub… Stub running… Changing baud rate to 460800 Changed. Configuring flash size… A fatal error occurred: Timed out waiting for packet header A fatal error occurred: Timed out waiting for packet header couldn’t find what baud rate works unfortunately Any ideas or anybody is having the same issue? Regards Azrin Reply

Thank you for this tutorial! I bought a pair of ESP32-CAM from the same vendor. Packing says DFROBOT. Chip says ATHINKER, so I used ATHINKER successfully. Following the steps on “Getting started with the ESP32-CAM” I finally got 1 to work, one to kind of work. Couple of troubleshooting tips: 1) Putting the jumper in the 5V position and connecting to the 5V pin allowed me to load the sketch 2) If, after loading your sketch, removing the programming jumper, reattaching the USB cable, the initialization just ends with an endless line of “………” every few seconds, your SSID or password is wrong. SSID is case sensitive. My problem: I have 2 ATHINKER ESP32-CAMs. My first one will not stream, but will take a still shot. When streaming it shows a small blank square with a broken icon. The second one will stream just fine. Same sketch, wiring, computer, cable. Any ideas why? Reply

Hi. That could be related to a weak Wi-Fi signal because of the antenna selection. It may be different on each board. Check the antenna position for your cameras. This tutorial explains it all: https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/ This tutorial also explains something about the antenna, see bullet number bullet 7: https://randomnerdtutorials.com/esp32-cam-troubleshooting-guide/ Regards, Sara Reply

That’s it! Just laying on the other side of the table was apparently enough to lose the signal. Thank you! Reply

Hi, Would you happen to have a tutorial on how to send data or commands from the ESP32-Cam to a Nano? I have a pan/tilt setup that controls two servos with an ESP32-Cam and I would like the Nano to control the servos instead by getting commands from the ESP32-Cam object detection to move the servos. Is this possible? Thanks. Reply

Hey hi thr, I am new too ESP-32 cam. I have been following your tutorials and those are really amazing. I did study both your codes and driver architect. Now i wonder esp_camera_fb_get; how to use this function to read the exact frame rate (fps) or just give me an brief idea to read the frame rate (fps) of the ESP32-Cam with (ov264 module). I have a necessity of Video analysis and Photo analysis. Reply

Hi there, Can u tell how to read the frame rate (fps) of the ESP32-cam module. I am using one ov264 cam ( AI Thinker CAM ). Reply

hello Santos, please add to troubleshooting, if use ESP-CAM-MB with ESP32-CAM, ESP.getPsramSize report 0 PSRAM and stream is incredibly slow. MJPG: 3972B 8169ms (0.1fps), AVG: 4106ms (0.2fps) Reply

Hi, Thank you so much for all your efforts! I have bought several ESP32-CAMS and when I load (the same) CamWebServer_Example sketch all works well on each but some show the image in portrait mode and others in landscape. Why is this happening with the same code and configuration? Are the boards different in some way or is the camera module itself wired up differently? I would like all to operate in landscape mode – is there any way to make this happen or are these boards permanently different and how can I tell which one I’m buying? Thank you for any help you can give me – it would be much appreciated. Reply

Could one use the example code for the esp-cam and use it on a different MCU say an arduino with a different camera? Reply

Hello, Thanks for this project, I have the camera up and running! I do have a question about integrating it with Home Assistant. I tried adding the picture card in my ui-lovelace.yaml file but the stream is not coming up. Here’s what I have: – icon: mdi:CCTV title: Cameras cards: – type: picture tap_action: action: none hold_action: action: none image: ‘http://192.168.1.140/’ Reply

Hi. The stream is only accessible to one client at a time. Close any other clients that may be connected to the ESP32-CAM video streaming. Regards, Sara Reply

Hello, Thanks again for your excellent tutorial. Is it possible to rotate the streaming image 90 degrees? Reply

Hi. There isn’t a command to rotate 90 degrees. There are functions to mirror and vertical flip the image. See in this tutorial: https://randomnerdtutorials.com/esp32-cam-ov2640-camera-settings/ Regards, Sara Reply

Hello from France Sara, I just found this to, among other things, rotate the streaming image: https://projetsdiy.fr/5-astuces-esp32-cam-adresse-ip-fixe-mode-ap-rotation-image-recuperation-automatique-connexion-Wi-Fi-stockage-code-html/#rotationimage I am convinced that this will interest some people … I have developed and formatted it for my own need. To be able to modify the “camera_index.h” html file, I used “CyberChef” which works very well, there are tutorials on the web to use it. I hope this will interest a few people. Regards gg Reply

Hello, I’m thinking in a new project. I want to build a time lapse camera long term to record my house building. I think that programming the work hours and take any photos for a day. I want no attending camera or battery and catch the camera passing 8/9months. I read in the forum maybe don’t Support 32gb SD card… Isn’t it? My project…. it’s possible? Thanks to all Reply

It is possible when u connect ur esp32cam with arduino board and supply the power supply like solar pannel. But not sure for long term since the esp32cam get hot when long term stream? Reply

Hello Sara, I have one question to the ESP32-Cam: How can I rotate the picture 90 degrees? I cannot rotate the Cam in the case. Thanks and Merry Christmas. Ulli Reply

Hi. Unfortunately, there isn’t a method to rotate the camera 90 degrees. There is a method for mirror and horizontal flip. See this tutorial: https://randomnerdtutorials.com/esp32-cam-ov2640-camera-settings/ Regards, Sara Reply

I have purchased your CAM32 book – after payment (PayPal) I haven’t received a download link … how to get it? BR Reply

Hi. You should receive an email with the instructions to access the RNTLAB.com website and get access to the eBook. If you’re having trouble accessing it, please send an email to our support: https://randomnerdtutorials.com/support/ Regards, Sara Reply

Use a ESP32-CAM Module to Stream HD Video Over Local Network

You have written a great introduction. Thanks! Could the face recognition feature be trained to recognize a stationary logo (not a face) on a TV screen? Is there any gpio output when a face is recognized? Reply

I’m wondering if the face recognition system could be used to recognise moods: can it tell a happy face from a grumpy one? I imagine that that would be easier than recognising different faces, but I have no idea how it works, so I wouldn’t know where to start. Reply

Hi, Wi-Fi connection of the ESP32 CAM disconnects when I switched on face recognition. No face recognition. Pls. help. Reply

Hi. Make sure you have a strong Wi-Fi connection. The board should be relatively close to your router. Also, check if your board needs an external antenna: https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/ It should also be powered with a good/stable power supply. Regards, Sara Reply

I have got face recognition working after implementing a suggestion to download the ESP32 library to 1.04, the other suggestion to use 1.06 and edit the code did not work. Have you any plans to extend the tutorial to store and then load the data required for face recognition to the SD card? At the moment the information is lost when the board is re-powered? It is certainly something I will be looking into. Great tutorials. Reply

I want to get rid of the text output giving info about the video produced. It comes out in the serial monitor and I want to put other info there without it being cluttered by the video info. Do you know where the code is which is producing this. I can’t see it in the camera library files. Reply

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1216 ho 0 tail 12 room 4 load:0x40078000,len:10944 load:0x40080400,len:6360 entry 0x400806b4 [E][camera.c:1113] camera_probe: Detected camera not supported. [E][camera.c:1379] esp_camera_init: Camera probe failed with error 0x20004 Reply

Hello, I’m trying this module and my question is if it can be done digital zoom streaming? Thanks Reply

Hello, I always have the same problem, whether I compile ‘CameraWebServer’ on my Mac or my PC, I get the following error: CameraWebServerJB:65:12: error: ‘struct camera_config_t’ has no member named ‘fb_location’ config.fb_location = CAMERA_FB_IN_DRAM; ^ CameraWebServerJB:65:26: error: ‘CAMERA_FB_IN_DRAM’ was not declared in this scope config.fb_location = CAMERA_FB_IN_DRAM; Any idea? Reply

Hello. I am a true newbie in this but learning a lot with you both. May you please tell me if it is possible to make this project conecting the arduino board to the ESP32-CAM Board with that same code? Thank you in advance. Regards from Lisboa, Portugal. Reply

My camera got connected to the Wi-Fi and I got the IP address as well. The IP address cannot be reached through browser Reply

Hi Sara, i having my project using esp32cam and the dht22 on the io port 15 of the esp32cam. When it start streaming online, the dht22 on the serial monitor will failed to read the humidity and temperature data. How this happen? Reply

It had to be the the same Wi-Fi connection as your ESP32CAM connected. So it can stream the video Reply

Hi Sara, i having my project using esp32cam and the dht22 on the io port 15 of the esp32cam. When it start streaming online, the dht22 on the serial monitor will failed to read the humidity and temperature data. How this happen? Reply

Hi Sara, I tried this “mounting” with ESP32-cam, it worked almost perfectly. I say this because although I followed all the steps, it does not connect to my Wi-Fi network. The connection data are the correct ones (SSID and Password). It generates its own Wi-Fi network with the address 192.168.4.1 to which I connected both with my phone and with my laptop. Am I wrong somewhere? In the serial monitor after reset I receive the following: rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0030,len:4 load:0x3fff0034,len:6968 load:0x40078000,len:13072 ho 0 tail 12 room 4 load:0x40080400,len:3896 entry 0x40080688 [0;32mI (30) boot: ESP-IDF v4.1-dirty 2nd stage bootloader[0m [0;32mI (30) boot: compile time 16:15:01[0m [0;32mI (30) boot: chip revision: 1[0m [0;32mI (33) boot_comm: chip revision: 1, min. bootloader chip revision: 0[0m [0;32mI (40) boot.esp32: SPI Speed : 40MHz[0m [0;32mI (45) boot.esp32: SPI Mode : DIO[0m [0;32mI (49) boot.esp32: SPI Flash Size : 4MB[0m [0;32mI (54) boot: Enabling RNG early entropy source…[0m [0;32mI (59) boot: Partition Table:[0m [0;32mI (63) boot: ## Label Usage Type ST Offset Length[0m [0;32mI (70) boot: 0 nvs Wi-Fi data 01 02 00009000 00005000[0m [0;32mI (78) boot: 1 otadata OTA data 01 00 0000e000 00002000[0m [0;32mI (85) boot: 2 app0 OTA app 00 10 00010000 00300000[0m [0;32mI (93) boot: 3 spiffs Unknown data 01 82 00310000 000f0000[0m [0;32mI (100) boot: End of partition table[0m [0;32mI (104) boot_comm: chip revision: 1, min. application chip revision: 0[0m [0;32mI (112) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x1d2048 (1908808) map[0m [0;32mI (847) esp_image: segment 1: paddr=0x001e2070 vaddr=0x3ffbdb60 size=0x04d3c ( 19772) load[0m [0;32mI (856) esp_image: segment 2: paddr=0x001e6db4 vaddr=0x40080000 size=0x00400 ( 1024) load[0m [0;32mI (857) esp_image: segment 3: paddr=0x001e71bc vaddr=0x40080400 size=0x08e54 ( 36436) load[0m [0;32mI (879) esp_image: segment 4: paddr=0x001f0018 vaddr=0x400d0018 size=0x9df74 (647028) map[0m [0;32mI (1126) esp_image: segment 5: paddr=0x0028df94 vaddr=0x40089254 size=0x0b6a0 ( 46752) load[0m [0;32mI (1158) boot: Loaded app from partition at offset 0x10000[0m [0;32mI (1158) boot: Disabling RNG early entropy source…[0m ⸮ SD Size: 7624MB OK PSRAM OK ESP32-CAM-MB Reply

Hi there to all of you, Hi Sara, Nice tutorial. I made a “system” with ESP32 cam and one USB programmer, uploaded the code and surprise…. Won’t connect to my Wi-Fi network even if I gave the right credentials. It’s working on my phone and on my laptop (after I connect my laptop to the Wi-Fi network generated by esp32). That was yesterday, today at work, I realized that the ESP32 have a resident software inside and my code doesn’t upload. Why I am saying that? Simply because I forget to put IO0 to the ground. I soldered 7 pin socket 5 for USB programmer and 2 for a jumper, but I didn’t connect to ESP32…. My fault…. This evening I will make all connection “by the book” and try again. So the idea is that ESP32 have some program resident “by default”, which will be over written by us in the uploading process from Arduino IDE. Reply

Hi to All, This is regarding ESP32 CAM module. When upload the Camera Web Server programme to my ESP 32 CAM, always get basic adjustment menu and the picture. How to get ADVANCE SETTINGS on the menu that appearing below scan start buttonAnything to do with Version of the IDE or ESP 32 version. I tried but failed I can see In some utube demos this feature is visible. Please help to get going. Thanks Ranraj Reply

Hi Sara, I had the same problem and I solved it with just 3 lines from the forum (https://github.com/nkolban/esp32-snippets/issues/168) ” I have the same problem. I found that solution in a forum : #include “soc/soc.h” #include “soc/rtc_cntl_reg.h” void setup WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector “ Reply

15:44:32.105. #⸮⸮ 15:44:32.105. ⸮⸮2⸮⸮b66⸮⸮3⸮2⸮⸮”c⸮⸮’ the output on serial monitor is this why please help me Reply

Hi, Worked well. I had a new ESP32 cam with USB. Very good tutorial, appreciate your work. Many thanks. Reply

sir can we program esp32cam for gesture recognization and use it with hardware mobile or laptop Reply

Hi, there are few lines used ‘free’ like: free(net_boxes-score); may I know which library come out the ‘free’ variables please? Thanks Summer Reply

Does the esp32 cam video streaming uses any video compression technique? If not can we implement something…. Reply

Hi Sara Thanks for the great tutorials and projects. I can get the code uploaded and obtain the IP address for CameraWebServer, but when I try to connect via the browser with the IP address, there is no connection. There is no error message, just unable to connect to the CameraWebServer. After I uploaded the code, I got the following msg on the Serial monitor below, but there is no “Starting web server on port: ’80’ Starting stream server on port: ’81’ ” msg in between, does that mean the CameraWebServer never got started and hence the no connection problem? If that is the case, what could be the source of the problem. “…. Wi-Fi connected Camera Ready! Use ‘http://XXX.XX.X.XXX’ to connect ” Thanks and Happy Thanksgiving. Reply

Hello, I cannot enroll the face. If i click on the button nothing happen. Can you help me? thank you in advance. Manuel Reply

Facial recognition: top 7 trends (tech, vendors, use cases)

Equally, its arrival prompted profound concerns and surprising reactions in recent years.

But more about that later.

In this web dossier, you will discover the seven face recognition facts and trends that shape the landscape in 2021.

• Top technologies and providers
• AI impact. Getting better all the time.
• 2019-2024 markets and dominant use-cases
• Face recognition in China, India, the United States, the EU, the UK, Brazil, Russia.
• Privacy vs security: laissez-faire or freeze, regulate, or ban?
• Latest hacks: can facial recognition be fooled?
• Towards hybridized solutions.

How facial recognition works

Facial recognition is the process of identifying or verifying a person’s identity using their face. It captures, analyzes, and compares patterns based on the person’s facial details.

• The face detection process is an essential step in detecting and locating human faces in images and videos.
• The face capture process transforms analog information (a face) into digital information (data or vectors) based on the person’s facial features.
• The face match process verifies if two faces belong to the same person.

A student from the greater Washington DC area used an open-source facial extraction app to detect and deduplicate over 6,000 images of faces from 827 videos posted on Parler during the 6 January event outside and inside the Capitol building (source: Wired 20 January 2021.) He created a website called Faces of the Riot, displaying these portraits.

• Demonstrators, rioters, and journalists have done part of the face capture step with their smartphones (analog face to digital picture).
• He used facial detection to extract faces from 200K images.
• It’s up to the FBI to investigate, transform the portraits (digital pixels to vectors) and potentially do the face match with existing databases and identify the individuals (with an AFIS / ABIS system).

Today it’s considered to be the most natural of all biometric measurements.

And for a good reason – we recognize ourselves not by looking at our fingerprints or irises, for example, but by looking at our faces.

Before we go any further, let’s quickly define two keywords: identification and authentication.

Face recognition data to identify and verify

Biometrics are used to identify and authenticate a person using recognizable and verifiable data unique and specific to that person.

For more on biometrics definition. visit our web dossier on biometrics.

Identification answers the question: Who are you?

Authentication answers the question: Are you really who you say you are?

Stay with us. Here are some examples :

• In the case of facial biometrics, a 2D or 3D sensor captures a face. It then transforms it into digital data by applying an algorithm before comparing the image captured to those held in a database.
• These automated systems can be used to identify or check an individual’s identity in just a few seconds based on their facial features (geometry): spacing of the eyes, bridge of the nose, the contour of the lips, ears, chin, etc. They can even do this in a crowd and dynamic and unstable environments.

• Owners of the iPhone X have already been introduced to facial recognition technology.

Of course, other signatures via the human body also exist, such as fingerprints, iris scans, voice recognition, digitization of veins in the palm, and behavioral measurements.

Why face recognition, then?

Facial biometrics continues to be the preferred biometric benchmark.

That’s because it’s easy to deploy and implement. There is no physical interaction with the end user.

over, face detection and face match processes for verification/identification are speedy.

So, what is the best face recognition software?

Top facial recognition technologies

Several projects are vying for the top spot in the race for biometric innovation.

Google, Apple. Amazon, and Microsoft (GAFAM) are also very much in the mix.

All the software web giants now regularly publish their theoretical discoveries in artificial intelligence, image recognition, and face analysis to further our understanding as rapidly as possible.

Academia

The GaussianFace algorithm developed in 2014 by researchers at The Chinese University of Hong Kong achieved facial identification scores of 98.52% compared with the 97.53% achieved by humans. An excellent rating, despite weaknesses regarding memory capacity required and calculation times.

and Google

In 2014, announced its DeepFace program, which can determine whether two photographed faces belong to the same person, with an accuracy rate of 97.25%. When taking the same test, humans answer correctly in 97.53% of cases, or just 0.28% better than the program.

In June 2015, Google went one better with FaceNet. On the widely used Labeled Faces in the Wild (LFW) dataset, FaceNet achieved a new record accuracy of 99.63% (0.9963 ± 0.0009). Using an artificial neural network and a new algorithm, the company from Mountain View has managed to link a face to its owner with almost perfect results.

This technology is incorporated into Google Photos and used to sort pictures and automatically tag them based on the people recognized.

Proving its importance in the biometrics landscape, it was quickly followed by the online release of an unofficial open-source version, OpenFace.

Microsoft, IBM, and Megvii

A study done by MIT researchers in February 2018 found that Microsoft, IBM, and China-based Megvii (FACE) tools had high error rates when identifying darker-skin women compared to lighter-skin men.

At the end of June 2018, Microsoft announced that it had substantially improved its biased facial recognition technology in a blog post.

Amazon

In May 2018, Ars Technica reported that Amazon is already actively promoting its Cloud-based face recognition service named, Rekognition, to law enforcement agencies. The solution could recognize as many as 100 people in a single image and perform face matches against databases containing tens of millions of faces.

In July 2018, Newsweek reported that Amazon’s facial recognition technology falsely identified 28 US Congress members as people arrested for crimes.

Key biometric matching technology providers

At the end of May 2018, the US Homeland Security Science and Technology Directorate published the results of sponsored tests at the Maryland Test Facility (MdTF). These real-life tests measured the performance of 12 face recognition systems in a corridor measuring 2 m by 2.5 m.

Thales’ solution utilizing Facial recognition software (LFIS) achieved excellent results with a face acquisition rate of 99.44% in less than 5 seconds (against an average of 68%), a Vendor True Identification Rate of 98% in less than 5 seconds compared with an average 66%. It also achieved an error rate of 1% compared with an average of 32%.

March 2018 – The live testing using more than 300 volunteers identified the best-performing facial recognition technologies.

on performance benchmarks: The NIST (National Institute of Standards and Technology) report, published in November 2018, details recognition accuracy for 127 algorithms and associates performance with participant names.

The NIST Ongoing Face Recognition Vendor Test (FRVT) 3, performed at the end of 2019, provides additional results. See the NIST report.

NIST also demonstrated that the best facial recognition algorithms have no racial or sex bias, as reported in January 2020 by ITIF. Critics were wrong.

In NIST’s reports (August 2020 and March 2021) entitled Face recognition accuracy with face masks using post-COVID-19 algorithms, we see how algorithms are increasing their performance in less than a year.

Facial Emotion Recognition (FER)

Facial Emotion Recognition (from real-time or static images)is the process of mapping facial expressions to identify emotions such as disgust, joy, anger, surprise, fear, or sadness. or compound emotion such as sadness, anger. on a human face with image processing software.

There are also three steps in the recognition or interpretation of human emotions:

• 1) Face detection
• 2) Face expression detection
• 3) Assignment of expression to a specific emotional state.

Facial emotion detection’s popularity comes from the vast areas of potential applications.

It’s different from facial recognition, which aims to identify a person, not an emotion.

Face expression may be represented by geometric or appearance features, parameters extracted from transformed images such as eigenfaces, dynamic, and 3D models.

Providers include Kairos (face and emotion recognition for brand marketing), Noldus, Affectiva, or Sightcorp.

Learning to learn through deep learning

The feature common to all these disruptive technologies is Artificial Intelligence (A.I.) and, more precisely, deep learning, where a system can learn from data.

It’s a central component of the latest-generation algorithms developed by Thales and other key players. It holds the secret to face detection, face tracking, face match, and real-time translation of conversations.

Face recognition systems are getting better all the time.

According to a recent NIST report, massive gains in recognition accuracy have been made in the last five years (2013- 2018) and exceed the 2010-2013 period.

Most of the face recognition algorithms in 2018 outperformed the most accurate algorithm from late 2013.

In its 2018 test, NIST found that 0.2% of searches in a database of 26.6 million photos failed to match the correct image, compared with a 4% failure rate in 2014.

In NIST’s 2020 tests, the best facial identification algorithm has an error rate of 0.08%. that’s less than one error for 1.000 images. (source: How accurate are facial recognition systems, CSIS)

Yes, you understand that, right?

Artificial neural network algorithms are helping face recognition algorithms to be more accurate.

Facial recognition markets

Face recognition markets

A study published in June 2019 estimates that by 2024, the global facial recognition market would generate 7billion of revenue, supported by a compound annual growth rate (CAGR) of 16% over 2019-2024.

For 2019, the market was estimated at 3.2 billion.

The two most significant drivers of this growth are surveillance in the public sector and numerous other applications in diverse market segments.

According to the study, the top facial recognition vendors include :

Accenture, Aware, BioID, Certibio, Fujitsu, Fulcrum Biometrics, Thales, HYPR, Idemia, Leidos, M2SYS, NEC, Nuance, Phonexia, and Smilepass.

The main facial recognition applications can be grouped into three principal categories.

What is facial recognition used for?

Here are the top three application categories where facial recognition is being used.

Security. law enforcement

Forensic specialists can use Automated Biometric Identification Systems (ABIS) to compare multiple types of biometrics.

This market is led by increased activity to combat crime and terrorism.

The benefits of facial recognition systems for policing are evident: detection and prevention of crime.

• Facial recognition is used when issuing identity documents and, most often, combined with other biometric technologies such as fingerprints (preventing I.D. fraud and identity theft).
• Face match is used at border checks to compare the portrait on a digitized biometric passport with the holder’s face. In 2017, Thales supplied the new automated control gates for the PARAFE system (Automated Fast Track Crossing at External Borders) at Roissy Charles de Gaulle Airport in Paris. This solution has been devised to facilitate the evolution from fingerprint recognition to facial recognition in 2018.
• Face biometrics can also be employed in police checks, although it is rigorously controlled in Europe. In 2016, the man in the hat responsible for the Brussels terror attacks was identified thanks to FBI facial recognition software. The South Wales Police implemented it at the UEFA Champions League Final 2017.
• In the United States, 26 states (and probably as many as 30 ) allow law enforcement to run searches against their databases of driver’s licenses and I.D. photos. The FBI has access to driver’s license photos from 18 states.
• ​Drones and aerial cameras offer an exciting combination of facial recognition applied to large areas during mass events. According to the Keesing Journal of Documents and Identity of June 2018, some hovering drone systems can carry a 10-kilo camera lens that can identify a suspect from 800 meters to a height of 100 meters. The drone can be connected to the ground via a power cable with an unlimited power supply. The communication to ground control can’t be intercepted as it also uses a line.
• Facial recognition CCTV systems can improve performance in carrying out public security missions. Let’s illustrate this with four examples:

• Find missing children and disoriented adults.
• Identify and find exploited children.
• Identify and track criminals.
• Support and accelerate investigations.

Find Missing children and disoriented adults.

Face recognition CCTV systems can significantly accelerate operators’ efforts by enabling them to add a reference photo provided by the missing child’s parents and match it with past appearances of that face captured on video. Police can use face recognition to search video sequences (aka video analytics) of the estimated location and time the child has been declared missing.

Read more on how Delhi Police used a facial recognition system to trace 3,000 missing children in 4 days.

Police officers can better figure out the child’s movements before going missing and locate where he/she was last seen. A real-time alert can trigger an alarm whenever there’s a match.Police can then confirm its accuracy and do what’s necessary to recover the missing children. The same process can be applied for disoriented missing adults (e.g., with dementia, amnesia, epilepsy, or Alzheimer’s disease).

Identify and find exploited children.

Isolating the appearances of specific individuals in a video sequence is critical. It can accelerate investigators’ jobs in child exploitation cases as well.

Video analytics can help build chronologies, track activity on a map, reveal details, and discover non-obvious connections among the players in a case.

Identify and track criminals.

Face recognition CCTV can be used to enable police to track and identify past criminals suspected of perpetrating an additional infraction. Police can also take preventive actions. By using an image of a known criminal from a video or an external picture (or a database), operators can detect matches in live video and react before it’s too late.

Support and accelerate investigations.

Facial recognition CCTV systems can be used to support investigators searching for video evidence in the aftermath of an incident.

The ability to isolate suspects and individuals’ appearances is critical for accelerating investigators’ review of video evidence for relevant details. They can better understand how situations developed.

Health

Significant advances have been made in this area.

Thanks to deep learning and face analysis, it is already possible to:

• track a patient’s use of medication more accurately
• detect genetic diseases such as DiGeorge syndrome with a success rate of 96.6%
• support pain management procedures.

Banking and retail

This area is undoubtedly the one where facial recognition was least expected. And yet, quite possibly, it promises the most.

Know Your Customer (KYC) with facial recognition online will be a hot topic in 2021.

Because 64% of primary checking account openings were done online in Q2 2020 ( and 36% in branches) in the United States alone.

Besides, increased mobile usage urges businesses to FOCUS on mobile-first and develop fully mobile user-friendly onboarding experiences.

During the selfie process, the technology shall provide a liveness detection to avoid fraud using a static image.

Liveness detection proves that the selfie taken comes from a live person.

Adapting to current customer preferences, financial institutions (F.I.s) invest in digital onboarding through online and mobile channels.

Facial recognition with liveness detection simplifies online onboarding and KYC procedures. Thales is a major provider of identity verification solutions, including this feature.

According to Forbes, digital account opening (DAO) was the most popular technology in banking for the third consecutive year. Nearly 80% of financial institutions add new DAO systems or enhance existing ones in 2020 and 2021.

This important trend is combined with the latest marketing advances in customer experience.

By placing cameras in retail outlets, it is now possible to analyze shoppers’ behavior and improve the customer purchase process.

Like the system recently designed by. sales staff are provided with customer information taken from their social media profiles to produce expertly customized responses.

The American department store Saks Fifth Avenue is already using such a system. Amazon Go stores are reportedly using it.

How long before the selfie payment?

Since 2017, KFC, the American king of fried chicken, and Chinese retail and tech giant Alibaba have been testing a face recognition payment solution in Hangzhou, China.

In March 2021, 52 Perekrestok stores (Перекрёсток) from X5 retail group launched touchless payment by face for self-service checkout terminals with Visa Payment System and Sberbank.

The facial recognition payment system would be used in 3,000 stores by yearend, according to Yahoo!

According to Interfax, muscovites can pay for metro rides at the end of 2021.

Mapping of new users

While the United States currently offers the largest market for face recognition opportunities, the Asia-Pacific region sees the fastest growth in the sector. China and India lead the field.

Face recognition in China

Face recognition technology is the new hot topic in China, from banks and airports to police.

Now authorities are expanding the facial recognition sunglasses program as police are beginning to use them in Beijing’s outskirts.

China is also setting up and perfecting a video surveillance network countrywide.

According to CNBC, over 200 million surveillance cameras were used in 2018; over 500 million are expected by 2021.

The facial recognition towers in Chinese cities are emblematic of this move.

This is linked to the social credit system the Chinese government is developing.

In the TOP 10 cities with the most street cameras per person, Chongqing, Shenzhen, Shanghai, Tianjin, and Ji’nan lead the pack.

London is #6 and Atlanta #10, according to the Guardian of 2 December 2019.

Chinese police are working with artificial intelligence companies such as Yitu, Megvii (in partnership with Huawei), SenseTime, and CloudWalk, according to The New York Times on 14 April 2019.

China’s ambitions in A.I. (and facial recognition technology) are high. The country aims to become a world leader in A.I. by 2030.

Surprisingly, China provides strong biometric data protection against private entities AND increases the government’s access to personal information.

This paradox is evidenced by privacy expert Emmanuel Pernot- Leplay’s report dated 2 November 2020.

Facial recognition in Asia

Facial recognition will be an important topic for the 2020 Olympic Games in Tokyo (postponed to September 2021).

This technology will automatically identify authorized persons and grant them access, enhancing their experience and safety. It’s also being used in Japan for easier mobile banking access.

In Sydney, face recognition is undergoing trials at airports to help move people through security much faster and safer.

In India, the Aadhaar project is the largest biometric database in the world. It already provides a unique digital identity number to 1.29 billion residents as of the end of March 2021.

UIDAI, the authority in charge, announced that facial authentication would be launched in a phased roll-out.

It’s presently being tested for financial services (October 2020.)

Face authentication will be available as an add-on service in fusion mode and another authentication factor like a fingerprint, Iris, or TOTP.

India could also roll out the world’s most extensive face recognition system in 2021.

The National Crime Records Bureau (NCRB) has issued an RFP inviting bids to develop a nationwide facial recognition system.

According to the 160-page document, the system will be a centralized web application hosted at the NCRB Data Center in Delhi. It will be available for access to all the police stations.

It will automatically identify people from CCTV videos and images. The Bureau states it will help police catch criminals, find missing people, and identify dead bodies.

Other large projects

The Superior Electoral Court (Tribunal Superior Eleitoral) is involved in Brazil’s nationwide biometric data collection project. The aim is to create a biometric database and unique I.D. cards, recording the information of 140 million citizens.

In Africa, Gabon, Cameroon, and Burkina Faso have chosen Thales to meet the challenges of biometric identity to identify voters uniquely.

Russia’s Central Bank has been deploying a countrywide program since 2017 designed to collect faces, voices, iris scans, and fingerprints.

But the process is progressing very slowly, according to the Biometricupdate website of 13 March 2019.

Moscow claims one of the world’s largest networks of 160,000 surveillance cameras by the end of 2019 and is fitted with facial recognition technology for public safety.

The roll-out started in January 2020.

Russian law does not regulate non-consensual face detection and analysis.

When face recognition strengthens the legal system

Facial recognition technologies radically affect the ethical and societal challenges data protection poses.

Do these technological feats, worthy of science-fiction novels, genuinely threaten our freedom?

And with it, our anonymity?

E.U. and U.K. biometric data protection

The General Data Protection Regulation (GDPR) provides a rigorous framework for these practices in Europe and the U.K.

Any investigations into a citizen’s private life or business travel habits are out of the question, and any such invasions of privacy carry severe penalties.

Applicable from May 2018, the GDPR supports the principle of a harmonized European framework, particularly protecting the right to be forgotten and giving consent through explicit affirmative action.

Yes, you read it well. There’s now one law for 500 million people.

This directive is bound to have international repercussions.

U.S. biometric data protection landscape

Without a federal law, cities and states are filling the gap.

The State of Washington was the third U.S. state (after Illinois and Texas) to formally protect biometric data through a new law introduced in June 2017.

California was the fourth state as of January 2020.

The California Consumer Privacy Act (CCPA ), passed in June 2018 and effective as of 1 January 2020, will severely impact privacy rights and consumer protection for residents of California and the whole nation.

The law is frequently presented as a model for a federal data privacy law.

In that sense, the CCPA can potentially become as consequential as the GDPR.

In July 2018, Bradford L. Smith, Microsoft’s president, compared the face recognition technology to products like highly regulated medicines, and he urged Congress to study it and oversee its use.

In May 2019, U.S. Rep. Alexandria Ocasio-Cortez voiced her absolute concerns in a Committee Hearing on facial recognition Technology (Impact on our Civil Rights and Liberties).

A New York State law called the Stop Hacks and Improve Electronic Data Security (SHIELD) became effective on 21 March 2020. It requires implementing a cybersecurity program and protective measures for N.Y. State residents.

The act applies to businesses that collect the personal information of N.Y. residents.

With the act, New York now stands beside California.

Facial recognition bans (San Francisco, Somerville, Oakland, San Diego, Boston, Portland)

Privacy and civil rights concerns have escalated in the country as face recognition gains traction as a law enforcement tool, and on 6 May 2019, San Francisco voted to ban facial recognition.

The anti-surveillance ordinance signed by San Francisco’s Board of Supervisors bars city agencies, including San Francisco PD, from using the technology as of June 2019.

Yes, this includes law enforcement.

As reported by the Boston Globe on 27 June 2019, the Somerville City Council (Massachusetts) voted to ban facial recognition, making the city the second community to make such a decision.

• On 16 July 2019, Oakland (California) took the same decision and became the third U.S. city to ban face recognition technology. It is interesting to note that the Oakland Police Department is not using this technology and was not planning to use it.
• San Diego took the same decision at the end of December 2019 before the new Californian law. This new law (Assembly Bill 215) about facial recognition and other biometric surveillance) specifically prohibits the use of police body cameras in California. The ban is in place for three years as of 1 January 2020.
• On 24 June 2020, Boston voted to ban face surveillance technology by police, as reported by the Boston Herald.
• Portland (Oregon) decided its ban on 9 September 2020 (effective 1 January 2021.) The city is the first to extend it to private entities in places of public accommodation, such as private stores. (CNN).
• Massachusetts passed a reform bill in December 2020 restricting the use of facial recognition. It’s applicable as of May 2021.
• Virginia legislature passed (April 2021) a new bill (H.B. 2031) that prohibits law enforcement agencies from continuing to use facial recognition software after 1 July 2021.

Since the San Francisco, Sommerville, Oakland, and now San Diego, Boston, and Portland rulings, the debate has gotten louder in many cities and states, not just in the U.S.

In Europe, a t the end of August 2019, Sweden’s Data Protection Authority decided to ban facial recognition technology in schools. It fined a local high school (the first GDPR penalty in the country).

How to better regulate emerging technologies?

• Should other cities or countries follow this example?
• Is the ban just a pause button to better assess risks?
• Is this a step backwards for public safety?
• Is there a policy vacuum? At which level?

Stay tuned for the outcome of all these discussions as the U.S. Congress is getting pressure from activists to ban the technology and from providers) to regulate.

But there’s still no Federal legal framework to address the issue as of May 2021.

The E.U. Commission plans to act on the indiscriminate use of facial identifier technology. The European Commission president Ursula von der Leyen wants a coordinated approach to artificial intelligence’s human and ethical implications. She has pledged to publish an A.I. legislation blueprint very soon.

The final version of the European Commission whitepaper is available online. The European Commission presented tough draft rules in April 2021. But according to Reuters, it could take years before the regulations come into force.

Similarly, in June 2021, the E.U.’s two privacy watchdogs (EDPB and EDPS) called for a ban on facial recognition in publicly accessible spaces.

Again the questions of privacy, consent, and function creep (data collected for one purpose being used for another)are central to the debate.

In our biometric data dossier, find more on biometric data protection laws(E.U., U.K., and U.S. perspectives).

India and its national biometric identification scheme, Aadhaar

In India, thanks to the Puttaswamy judgment delivered on 27 August 2017, the Supreme Court has enshrined the right to privacy in the country’s constitution. This decision has rebalanced the relationship between citizens and the state and posed a new challenge to expanding the Aadhaar project.

The Indian government, however, approved the use of the country’s biometric EID program by private entities on 28 February 2019.

Rebound effect: the legal system and its professions get even stronger.

As ambassadors and guardians of data protection regulation, data protection officers have become necessary for businesses and have a much sought-after role.

The rebels – facial recognition hackers

Despite this technical and legal arsenal designed to protect data, citizens, and their anonymity, critical voices have still been raised.

Some parties are concerned and alarmed by these developments. Some have taken action.

But can facial recognition be fooled?

• Grigory Bakunov in Russia has invented a solution to escape proper face detection and confuse face detection devices. He has developed an algorithm that creates special makeup to fool the software. However, he has not brought his product to market after realizing how easily criminals could use it.

• In Germany, Berlin artist Adam Harvey has developed a similar device known as CV Dazzle. He is now working on clothing featuring patterns to prevent detection. The hyperface camouflage includes patterns in fabric, such as eyes and mouths, to fool the face recognition system.

• In late 2017, a Vietnamese company successfully used a mask to hack the Face ID face recognition function of Apple’s iPhone X. However, the hack is too complicated to implement for large-scale exploitation.

• Around the same time, researchers from a German company revealed a hack that allowed them to bypass the facial authentication of Windows 10 Hello by printing a facial image in infrared.

• ​Forbes announced in May 2018 that researchers from the University of Toronto had developed an algorithm to disrupt facial recognition software (aka privacy filter).

• In August 2020, the Verge detailed a cloaking app named Fawkes. The software gradually distorts your selfies and other pics you may leave on social media. The tool comes from the University of Chicago’s Sand Lab.

In short, a user could apply a filter that modifies​ specific pixels in an image before putting it on the web. These changes are imperceptible to the human eye but are very confusing for facial recognition algorithms.

• In November 2020, a tool named Anonymizer was made available by Generated Media. The software creates a series of synthetic portraits from a picture you can upload. According to the website, the images are mathematically similar to your face and look like you but will trick facial recognition software. It could be an interesting solution to fool systems like Clearview A.I. that are scrapping millions of faces from social media (learn more on the Clearview A.I. controversy).

We tested Anonymizer on 27 November 2020. But the 40 doppelgangers we got were far from looking like the original portrait uploaded.

An interesting experiment by Thomas Smith, published on 28 January 2021, revealed a simple technique to make you invisible.

According to his tests, wearing a disposable mask and opaque sunglasses is a powerful combination to render you invisible.

In that case, the F.R. systems are denied too much valuable information (mouth, nose, eyes, eyebrows) to make a precise facial comparison.

The industry is working on anti-spoofing mechanisms, and standardization groups have specifically identified two topics:

• Make sure the captured image has been done from a person and not from a photograph (2D), a video screen (2D), or a mask (3D) (liveness check or liveness detection)
• Ensure that two or more individuals’ facial images (morphed portraits) have not been joined into a reference document, such as a passport.

Further together – towards hybridized solutions

Future identification and authentication solutions will borrow from all aspects of biometrics.

This will lead to a biometric mix capable of guaranteeing total security and privacy for all stakeholders in the ecosystem.

It’s very much the spirit of Thales Gemalto IdCloud Fraud Prevention, a risk assessment and fraud detection software for payments.

In this solution, geolocation, I.P. addresses (the device being used), and keying patterns can create a solid combination to authenticate users for online banking or egovernment services securely.

This seventh trend belongs to us.

It’s our job to envisage it together and make it happen through high-added-value biometric projects.

Thales has specialized in biometric technologies for almost 30 years. The company has always collaborated with the best research, ethics, and biometric application players.

Face recognition and you.

The months hold many changes in store.

Indeed, we can’t claim to predict all the essential topics that will emerge in the short-term future.

Can you fill in some of the gaps?

If you’ve something to say on facial recognition, tech, trends, a question to ask, or have found this article useful, please leave a comment in the box below.

We’d also welcome any suggestions on how it could be improved or proposals for future articles.

We look forward to hearing from you.

I’ll introduce a simple way to build and use any custom face recognition model with my custom framework

The Most Advanced Data Science Roadmaps You’ve Ever Seen! Comes with Thousands of Free Learning Resources and ChatGPT Integration! https://aigents.co/learn/roadmaps/intro

In this article, I’ll introduce a simple way to build and use any custom face recognition model with my custom framework. After completing this tutorial, you will learn how to use pre-trained models to create a real-time face recognition system with any CPU.

Officially, Face Recognition is defined as the problem of verifying or identifying faces in an image. But usually, there is a question, how exactly do we recognize a human face in an image?

The face recognition process can be separated into multiple steps. In the image below, there is an example of the face recognition pipeline:

The Face recognition pipeline steps:

• Face detection. Detect one or more faces in an image;
• Feature extraction. Extracting the essential features from the faces detected in the picture;
• Face matching. Match faces to anchor face features.

There are many ways to implement each step in a face recognition pipeline. In this article, we will FOCUS on a popular deep learning technique for face detection using the MediaPipe face detection model covered in the first tutorial, feature extraction using FaceNet, and face matching with cosine or euclidean distance.

Let’s tackle this task one step at a time. In each step, we will learn about different machine-learning algorithms. I’m not going to explain each algorithm fully so it won’t become an endless article. Still, I’ll show you the main ideas behind each one. You’ll learn to build your facial recognition system in Python using the MediaPipe face detector and FaceNet face feature extractor.

Face detection with MediaPipe:

The first step in our face recognition pipeline is face detection. We need to localize the faces in the image before we can try to tell them apart! Even though I covered the MediaPipe face detector in detail in my previous tutorial, we still need to mention it. If you used any cameras in the last decade, you probably noticed an evolution of face detection in action.

Face Recognition Based Attendance System using ESP32 CAM & OpenCV Visual Studio

Face detection is a great feature used for most cameras nowadays. For example, when the camera can automatically localize faces, it can ensure that all the faces are focused before it takes the picture. But now we’ll use it for a different purpose. finding the areas of the face that we’ll pass to another step in our pipeline, Face Feature Extractor.

This post will use my favorite, the ultra-light MedaPipe detector. But suppose you are interested in applying any of the other detection methods mentioned. In that case, you can refer to my previous tutorial, where I demonstrated how you could easily create your Object to run on my Engine. I have explored multiple face detectors. These include MTCNN, Dlib, MediaPipe, pure CNN, and a few other face detectors released recently.

But let’s continue what we do with located faces.

FaceNet face feature extraction:

FaceNet, similar to other facial recognition models, is a deep neural network to extract a human facial image’s features. It was published by Google researchers Schroff et al. in 2015.

The FaceNet takes an image of a human face as input, and the output is a vector of 128 numbers representing the essential features of the face. This vector is called embedding because all relevant information from the face image is embedded in this vector. Basically, FaceNet takes a human face and compresses it into a 128-number vector. But if we look at this vector, it would be impossible for us humans to tell what that person’s face is like.

Associating high dimensional data (such as images) with low dimensional images (vector embeddings) in machine learning has become a fairly common practice these days.

One common way to identify a person in an unseen image would be to get their embedding and calculate the distances to pictures of known people. We usually calculate the distance using the Cosine or Euclidean distance math formulas. If the calculated face distance is close enough to the embeddings of person A, we assume that this face is of the same person.

It sounds easy, right? We feed the image through FaceNet, get a magic vector number, and check that the calculated distance is sufficiently close to any known faces we have. Still, there is a question, how does FaceNet know what to extract from a face image? And what do these numbers in the embedding vector even mean? This is not the purpose of this tutorial, but we can cover it briefly so that we are familiar with it.

To train a face recognizer, we need many images of faces. Like every machine learning problem, training typically requires several thousand different images. When we start the training process, the model generates random vectors for each image, which means that the images are randomly distributed. But here are the learning steps:

• Randomly selects an anchor image;
• Randomly selects a positive sample image of the same person as the base image;
• Randomly selects a negative sample image of a person that is different from the main image;
• Adjusts the FaceNet neural network parameters so that the positive sample is closer to the anchor than the negative sample.

We repeat these four steps until there are no more changes to be done or these changes are so tiny that there is no impact. When the training is completed, all the faces of the same person are close to each other and far away from different ones in terms of distance.

You may ask, what do the numbers in the embedding vector mean? In fact, we don’t know what these numbers represent, and it’s tough to interpret them. We know that these features seem essential for face recognition, and it works.

If you are a developer, you can think of face matcher as a hash function. It maps the faces of the same person to (approximately) the same place in the coordinate system where embedding is the hashcode.

But it’s enough of this theory. Let’s go to our code stuff. As before, I wrote the entire Object specifically for face recognition, which we’ll use together with an engine object. This Object has everything it needs to do, from running inference, loading anchor images, measuring distance, drawing Recognition on a frame, and a little more; you can always add the functionality you need. Here is the complete faceNet.py object code:

# faceNet.py import cv2import cv2 import stow import typing import numpy as np import onnxruntime as ort class FaceNet: FaceNet class object, which can be used for simplified face recognition def init( self, detector: object, onnx_model_path: str = models/faceNet.onnx, anchors: typing.Union[str, dict] = ‘faces’, force_cpu: bool = False, threshold: float = 0.5, color: tuple = (255, 255, 255), thickness: int = 2, ). None: Object for face recognition Params: detector: (object). detector object to detect faces in image onnx_model_path: (str). path to onnx model force_cpu: (bool). if True, onnx model will be run on CPU anchors: (str or dict). path to directory with faces or dictionary with anchor names as keys and anchor encodings as values threshold: (float). threshold for face recognition color: (tuple). color of bounding box and text thickness: (int). thickness of bounding box and text if not stow.exists(onnx_model_path): raise Exception(fModel doesn’t exists in ) self.detector = detector self.threshold = threshold self.color = color self.thickness = thickness providers = [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] providers = providers if ort.get_device GPU and not force_cpu else providers[::-1] self.ort_sess = ort.InferenceSession(onnx_model_path, providers=providers) self.input_shape = self.ort_sess._inputs_meta[0].shape[1:3] self.anchors = self.load_anchors(anchors) if isinstance(anchors, str) else anchors def normalize(self, img: np.ndarray). np.ndarray: Normalize image Args: img: (np.ndarray). image to be normalized Returns: img: (np.ndarray). normalized image mean, std = img.mean, img.std return (img. mean) / std def l2_normalize(self, x: np.ndarray, axis: int =.1, epsilon: float = 1e-10). np.ndarray: l2 normalization function Args: x: (np.ndarray). input array axis: (int). axis to normalize epsilon: (float). epsilon to avoid division by zero Returns: x: (np.ndarray). normalized array output = x / np.sqrt(np.maximum(np.sum(np.square(x), axis=axis, keepdims=True), epsilon)) return output def detect_save_faces(self, image: np.ndarray, output_dir: str = faces): Detect faces in given image and save them to output_dir Args: image: (np.ndarray). image to be processed output_dir: (str). directory where faces will be saved Returns: bool: (bool). True if faces were detected and saved face_crops = [image[t:b, l:r] for t, l, b, r in self.detector(image, return_tlbr=True)] if face_crops []: return False stow.mkdir(output_dir) for index, crop in enumerate(face_crops): output_path = stow.join(output_dir, fface_.png) cv2.imwrite(output_path, crop) print(Crop saved to:, output_path) self.anchors = self.load_anchors(output_dir) return True def load_anchors(self, faces_path: str): Generate anchors for given faces path Args: faces_path: (str). path to directory with faces Returns: anchors: (dict). dictionary with anchor names as keys and anchor encodings as values anchors = if not stow.exists(faces_path): return for face_path in stow.ls(faces_path): anchors[stow.basename(face_path)] = self.encode(cv2.imread(face_path.path)) return anchors def encode(self, face_image: np.ndarray). np.ndarray: Encode face image with FaceNet model Args face_image: (np.ndarray). face image to be encoded Returns: face_encoding: (np.ndarray). face encoding face = self.normalize(face_image) face = cv2.resize(face, self.input_shape).astype(np.float32) encode = self.ort_sess.run(None, )[0][0] normalized_encode = self.l2_normalize(encode) return normalized_encode def cosine_distance(self, a: np.ndarray, b: typing.Union[np.ndarray, list]). np.ndarray: Cosine distance between wectors a and b Args: a: (np.ndarray). first vector b: (np.ndarray). second list of vectors Returns: distance: (float). cosine distance if isinstance(a, list): a = np.array(a) if isinstance(b, list): b = np.array(b) return np.dot(a, b.T) / (np.linalg.norm(a) np.linalg.norm(b)) def draw(self, image: np.ndarray, face_crops: dict): Draw face crops on image Args: image: (np.ndarray). image to be drawn on face_crops: (dict). dictionary with face crops as values and face names as keys Returns: image: (np.ndarray). image with drawn face crops for value in face_crops.values: t, l, b, r = value[tlbr] cv2.rectangle(image, (l, t), (r, b), self.color, self.thickness) cv2.putText(image, stow.name(value[‘name’]), (l, t. 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, self.color, self.thickness) return image def call(self, frame: np.ndarray). np.ndarray: Face recognition pipeline Args: frame: (np.ndarray). image to be processed Returns: frame: (np.ndarray). image with drawn face recognition results face_crops = for index, tlbr in enumerate(self.detector(frame, return_tlbr=True)) for key, value in face_crops.items: t, l, b, r = value[tlbr] face_encoding = self.encode(frame[t:b, l:r]) distances = self.cosine_distance(face_encoding, list(self.anchors.values)) if np.max(distances) self.threshold: face_crops[key][name] = list(self.anchors.keys)[np.argmax(distances)] frame = self.draw(frame, face_crops) return frame

I am not going step by step with this Object; I am covering this in my YouTube video tutorial. But I must mention that here we are loading the.onnx model, which we can convert with faceNet/convert_to_onnx.py script:

# faceNet/convert_to_onnx.py import OS import tensorflow as tf import tf2onnx from architecture import InceptionResNetV2 if name ‘main’: weights can be downloaded from https://drive.google.com/drive/folders/1scGoVCQp-cNwKTKOUqevCP1N2LlyXU3l?usp=sharing Put facenet_keras_weights.h5 file in model folder facenet_weights_path = models/facenet_keras_weights.h5 onnx_model_output_path = models/faceNet.onnx if not OS.path.exists(facenet_weights_path): raise Exception(fModel doesn’t exists in. download weights from \ https://drive.google.com/drive/folders/1scGoVCQp-cNwKTKOUqevCP1N2LlyXU3l?usp=sharing) faceNet = InceptionResNetV2 faceNet.load_weights(facenet_weights_path) spec = (tf.TensorSpec(faceNet.inputs[0].shape, tf.float32, name=image_input),) tf2onnx.convert.from_keras(faceNet, output_path=onnx_model_output_path, input_signature=spec)

First, you must download weights from the given link in a code and place them in the models folder. Then run the following code with python faceNet/convert_to_onnx.py. which should convert the model to.onnx format.

It’s as simple as that. When we have our model, we can open the main.py script and use the following code to run webcam real-time face recognition:

# main.py from utils import FPSmetric from engine import Engine from faceDetection import MPFaceDetection from faceNet.faceNet import FaceNet if name ‘main’: facenet = FaceNet( detector = MPFaceDetection, onnx_model_path = models/faceNet.onnx, anchors = faces, force_cpu = True, ) engine = Engine(webcam_id=0, show=True, custom_objects=[facenet, FPSmetric]) # save first face crop as anchor, otherwise don’t use while not facenet.detect_save_faces(engine.process_webcam(return_frame=True), output_dir=faces): continue engine.run

As I mentioned, to extract face embeddings, first, while defining the FaceNet Object, we need to localize the face on a frame. For this, we are using my MPFaceDetection object; more details you can find in my second tutorial about that. We then tell the model the path of our saved.onnx model. Then we give it the path where the anchors are saved; it must be images with face crop. For example, I have mine on the image Rokas.png where my cropped face is. This means that model will load this anchor and going to display my name if it finds a match with it.

Ok, nice. Next, we need to create an engine object that will be responsible for processing images, video, or webcam stream; I am choosing to process a webcam. With the show argument, I am telling the Object to display this frame for me on a screen. Also, I am adding an FPSmetric to know how fast face recognition works. Finally, I must pass my facenet object to the custom_objects argument. Either here, we can add even more, the pencil sketch, background removal or other entities we want.

Now, if we don’t have the crop saved on a disc, I created a function to grab a first webcam frame, and if It finds a face in it, it crops and saves it:

while not facenet.detect_save_faces(engine.process_webcam(return_frame=True), output_dir=faces): continue

After saving the anchor face, it moves to the engine.run function, which processed each frame, and we should see similar results to my following image:

We clearly can see FPS at the left top corner and a square on my face with the name of the anchor image. That’s amazing!

Conclusion:

Finally, we have created a system that can perform real-time face recognition on our CPU, which runs at around 30 fps, which is enough for us!

I found this tutorial to be fascinating with the results I was able to achieve. We could further work on this project by placing it on raspberry pi for real-time face recognition to unlock some entrances for us. Or we can put it at our house entrance to know whether someone unfamiliar is on our door. There are many possible ways to use it!

Thanks for reading! As always, all the code given in this tutorial can be found on my GitHub page and is free to use!

Using Deep Learning to Design Real-time Face Detection and Recognition Systems

Deep learning is a subset of machine learning and is solely concerned with complex algorithms. It has helped develop impressive features like automatic parking in cars, image analytics in healthcare, virtual assistance, and many more. It has also expanded its applications in visual recognition. This article will look at how deep learning is used in face detection and recognition systems.

Face detection vs. face recognition: what’s the difference?

Face detection

In a psychological process, face detection means locating and attending to faces in a visual scene. But in deep learning, it consists of detecting human faces by identifying the features of a human face from images or video streams.

Face recognition

This involves uniquely identifying a person’s face from a digital image or a video source. Face recognition has been in development since the 1960s, but thanks to advancements in technologies like IoT, AI, ML, etc., it has quickly become popular amongst the masses.

Face recognition technology has reached such advanced heights that it is now used in multiple places/situations. Some of them are:

Steps involved in face recognition

Face detection: This is the basic step, where the face is detected and bounding boxes are drawn around it. 2. Face alignment: The detected face is normalized to speed up training. Experiments show that alignment alone increases face recognition accuracy by almost 1%. 3. Feature extraction: Local features are extracted from the image with the help of algorithms. 4. Face recognition: This is the last stage and involves matching the input face with images present in the dataset to identify who it belongs to.

Image source

Deep learning methods for face detection and recognition

DeepFace

• A lightweight face recognition framework for Python.
• A hybrid method that wraps state-of-the-art models like VGGFace, FaceNet, DeepFace, Dlib, etc.
• Mainly uses three modules: DeepFace, OpenCV (for reading input images), and Matplotlib (for visualization).

Implementation

Install all the required libraries in your Python environment:

pip install deepface pip install opencv-python pip install matplotlib

Import the installed libraries:

from deepface import DeepFace import cv2 import matplotlib.pyplot as plt

Function to read and verify the images:

This function verifies face pairs as the same or different persons. It passes image paths as inputs and returns a dictionary with the results.

Output: They are the same.

This requires applying face verification many times. DeepFace finds the identity of the input image in the database path and then returns a Pandas dataframe as the output.

df = DeepFace.find(img_path = img1.jpg, db_path = C:/workspace/my_db)

Image source

Face recognition models

As discussed, DeepFace is a hybrid face recognition package that wraps state-of-the-art models such as VGG-Face, FaceNet, OpenFace, DeepFace, DeepID, ArcFace, Dlib, and SFace.

By default, DeepFace uses the VGG-Face model.

Code source

Similarly, you can loop on the models list and test other models.

Accuracy of the models

All of the models listed above produce an accuracy of almost 99% on the Labeled Faces in the Wild benchmark.

Real-time analysis

DeepFace can be employed for real-time videos as well. A stream function can be used to access a webcam and apply face recognition and analysis.

Face_recognition library

The face_recognition library is built on deep learning techniques and uses only a single training image.

How face_recognition works

• Passes a person’s picture with their name to the model.
• The model takes every picture and after converting them to numerical encoding, stores them in a list with the labels in another list.
• When making a prediction, the model again converts the input image to an encoding.
• This encoded image is used to find similar encodings based on the distance. The least distant encodings will be the closest match for the person.
• Once the closest match is found, the final detected person’s name is produced.

Implementation

pip install dlib pip install face recognition pip install opencv

Dlib: This is a modern C toolkit that has machine learning-related algorithms and tools. OpenCV: This is required for image preprocessing.

Import libraries after installation:

import cv2 import numpy as np import face_recognition

Load sample image to face_recognition library:

import cv2 import numpy as np import face_recognition img_bgr = face_recognition.load_image_file(‘student_images/modi.jpg’) img_rgb = cv2.cvtColor(img_bgr,cv2.COLOR_BGR2RGB) cv2.imshow(‘bgr’, img_bgr) cv2.imshow(‘rgb’, img_rgb) cv2.waitKey

cv2.cvtColor(img_bgr,cv2.COLOR_BGR2RGB). OpenCV is used here because face_recogniton supports only BGR format images. In order to print images, we need to convert them back to RGB using OpenCV.

Detecting and locating faces: Face_recognition quickly locates faces on its own without the use of haar cascade or any other similar technique.

Face detection is performed successfully by face_recognition library.

Face_recogniton can detect a person in a single shot, i.e., it only needs one image because it is based on deep learning.

Code source

The output is True because both input images were of the same person.

This library can be used for real-time face recognition with the following code:

Code source

Challenges in face recognition systems

The accuracy of facial recognition systems depends upon certain factors. And these factors themselves present a challenge.

Aside from DeepFace and the face_recognition module, there are many more methods that can be used to implement real-time face recognition systems using deep learning. Python Tkinter is one such and can be implemented to design GUI-based real-time applications.

FAQs

Once face recognition software is developed, it will need upgrades in time to maintain consistency in performance. Developing such systems in the first place relies on two common deep learning methods:

a. Pre-trained models: One method is to use already trained models, such as DeepFace by. FaceNet by Google, Dlib, etc. They have a set of algorithms required for face recognition. They are trained on very big datasets and usually do not require any more training. However, if fine-tuning is needed, it can be done and the model can be used in the software.

b. Developing neural networks from scratch: Creating complex face recognition systems requires a lot more time and effort as well as millions of images in the training dataset. unlike with pre-trained models that need only a few images. Once the model is ready with a minimal percentage of errors, it is set to be used in face recognition software.

If major changes occur in a model’s parameters or training parameters, such trained models may require re-training on the new images collected. This can easily be done at the backend as the models are usually deployed as APIs.

ArcFace is currently considered to be the best recognition model with respect to the challenges discussed in this article as well as other scenarios such as:

• Constrained and unconstrained environments
• Low-resolution, blurry, pose invariant, illumination issues
• Age-invariant or cross-age face recognition (adult, child)
• Mask-wearing face recognition.