Facial Recognition Security for <$80

The simplest questions can precipitate some of the most interesting creations. When my friend found a wireless handicap door button, I asked myself, "*What can I do with this?*"

handicap

After some research I found out that it transmits a wireless signal between 300-390 MHz.

"Ok, now how can I detect the signal?"

Answer: using software-defined radio.

So I can detect the signal on my computer, but that’s not much use. What can I do with the signal?

Let’s make the signal trigger a facial recognition system on a Raspberry Pi.

This is pretty cool, but how can I convey the results to the user? This Pi has no headphone jack.

Well, build a headphone jack.

Technologies used: Raspberry Pi Zero W, RTL-SDR (software-defined radio) dongle, walkie-talkie antenna, RC low-pass filter, text-to-speech conversion, Python facial recognition, Python-Flask server with uWSGI and Nginx, AWS EC2, and lots of networking know-how

This project was awesome because it was the first time I felt like an "engineer," able to combine existing technologies from multiple subject areas into something new. Ex nihilo—creating something out of nothing. On a high level, you press the button, then the Raspberry Pi takes a photo and says either "Hello Joe!" or "I’m sorry, I didn’t recognize you."

On a lower level, the handicap button sends a signal at 390 MHz, the RTL-SDR dongle attached to the Raspberry Pi picks up the signal, a Python program interprets the signal and uses espeak’s text-to-speech conversion to ask the user for a photo, then the photo is sent to a Python-Flask server in an AWS EC2 instance of Ubuntu 16.04. This server, using a database of "face data," converts the received image into data points and compares these data points to existing faces. If anyone is recognized, their names are returned to the Python program in the Raspberry Pi, which outputs a greeting message. Since the Raspberry Pi Zero W lacks an audio jack, I constructed a RC low-pass filter connected to GPIO pins 13 and 18 of the Pi and piped to an audio jack. This is connected to a nearby speaker, allowing the Raspberry Pi to produce sound.

RTL-SDR.com did a short article about this project, as did UConn Engineering.

Interpreting the Handicap Button Signal

First step, how to detect that the button was pressed. For this, I used an RTL-SDR dongle which can detect frequencies between 24-1766 MHz. I used Gqrx (installed via MacPorts) to visualize the frequencies. Of course, not knowing what I was doing, I didn’t realize that an antenna is needed so at first I was picking up nothing. After realizing about the antenna, I built one with a stripped coaxial cable and a coat hanger courtesy of this link. Finally, we can see the RF spectrum! Below is what the handicap button signal looks like.

gqrx

You can also pick up AM and FM radio stations, car key fabs, wireless speakers, and all kinds of interesting things with the RTL-SDR. Anyway, later I switched from a homemade antenna to a higher-quality antenna from a walkie-talkie. To enable the Raspberry Pi to interpret the RTL-SDR, I used this Python wrapper. It was necessary to tweak some settings on the dongle, but these settings seem to work well:

import numpy as np
from rtlsdr import *

class RTLSDRListener(object):
  def __init__(self):
  # Initialize RTL-SDR settings
  self.sdr = RtlSdr()
  self.sdr.set_manual_gain_enabled(False)
  self.sdr.set_sample_rate(2.4e6)
  self.sdr.set_center_freq(388 * 1000000.0) # in Hz
  self.sdr.set_bandwidth(0) # auto
  self.sample_size = 1024
  self.count = 0

  def get_data(self):
    samples = self.sdr.read_samples(self.sample_size)[:512]
    # Run an FFT and take the absolute value to get frequency magnitudes.
    freqs = np.absolute(np.fft.fft(samples))
    # Ignore the mean/DC values at the ends.
    freqs = freqs[1:-1]
    # Shift FFT result positions to put center frequency in center.
    freqs = np.fft.fftshift(freqs)
    # Convert to decibels.
    freqs = 20.0*np.log10(freqs)
    # Return frequency intensities.
    self.count = (self.count + 1) % 1000000
    return freqs

  def button_pressed(self):
    data = self.get_data()
    m = np.max(data)
    return m > 40 and self.count > 25

The listener gets data from the RTL-SDR dongle, uses numpy to transform it, and then finds the max value out of a subset of the data. During a button press, there is generally a spike in the signal, hence a larger maximum value.

Facial Recognition

Now the interesting bit: facial recognition. This library was used for facial recognition, with some slight modifications. The application is somewhat heavy-duty to run on the Raspberry Pi, so I ended up creating a Python-Flask server on an Amazon Web Services EC2 instance of Ubuntu 16.04. The server also uses uWSGI and Nginx. The Raspberry Pi takes a picture using PiCamera and sends it to the server. The server has various endpoints for recognizing an unknown face, saving a new face, and deleting a person from the database.

This link has a great explanation of how facial recognition works. When the server receives a photo, it will convert it to 128 data points and compare the data points to those of other people in the database. If there is a match, the server returns the name of the recognized person (or people) in its HTTP response.

And yes, I seeded the SQL database manually by sending the server photos and names of my friends. It could easily be scaled up by using a web scraper which scans Facebook and just keeps adding people though! (hello CIA)

Audio Output

There was one unfortunate aspect of the project which still needed to be dealt with—the Zero W model of the Raspberry Pi is super lightweight and lacks an audio output, so I had to make one. Luckily this is well documented here. First things first, solder pins onto the GPIO board of the Pi (GPIO = general purpose input/output pins). Second step, configure pins 13 and 18 to use PWM (pulse-width modulation). With the tutorial linked above, you can use the simplest option by modifying the device tree overlay in /boot/config.txt

dtoverlay=pwm-2chan,pin=18,func=2,pin2=13,func2=4

Third, create a circuit with an RC low-pass filter and a cut-off frequency of about 20 KHz. 1N4148 silicon diodes are used to prevent damage to the GPIO pins. My circuit is shown below:

filter

Now the Raspberry Pi can use the text-to-speech program espeak to ask the user for a photo and to output the user’s name. I piped the results of espeak into aplay (default is pulseaudio but this was causing problems… memory issues, I think).

espeak 'test sentence' --stdout | aplay

Wrapping Up

Wow that felt like a lot to cover. There were some other less interesting hurdles, such as enabling network access for the Raspberry Pi on my university’s WPA2 Enterprise network, properly configuring wpa_supplicant.conf, and getting radius certificates. I ended up having the university tech department assign the Pi a static IP on a guest network. Creating a hotspot on your phone and connecting both the Raspberry Pi and your computer to this hotspot will also allow for an SSH connection if you need it.

Shown below, the full setup. I hope you enjoyed reading about this project as much as I enjoyed creating it!

full_setup