Demystifying Clearview AI
Vehicle Tracking with Public CCTV and Deep Learning
Demystifying Clearview AI is a multi-part blog series based on the namesake csv,conf,v5 lightning talk. The purpose of the presentation was to demonstrate some of the technologies behind Clearview AI and, most importantly, discuss such technologies’ privacy implications.
If you’re primarily interested in the higher-level privacy implications, feel free to jump to Part 8 at any time. I’d recommend carefully reading the sections starting at Warning: ReID Data is Personal Data.
My first csv,conf,v4 in 2019 was such fun, and I learned so much that I wanted to participate again and contribute in any way I could. When the call for speakers went out for csv,conf,v5 I felt a great way to contribute would be to cover Clearview AI, a topic generating a good deal of interest at the time.
Various news outlets were covering Clearview’s threat to privacy and the questionable ethics behind its methods. Still, no coverage presented the underlying technology in a very digestible way. This series aims to fill that gap and, in the process, explore some possible defenses to the privacy problems engendered by Clearview.
Many thanks to the csv,conf,v5 Program Committee for accepting the talk, and to the csv,conf,v5 participants for being such a receptive and engaging audience.
Note: This talk was also subsequently accepted to All Things Open!
You can check out a recording of the talk on the All Things Open YouTube channel.
For Those with Some Programming Experience
At the end of this series, you will know:
- All the tools and technologies you’d need to build your own Clearview.
More than that, you’ll also hopefully appreciate why we don’t need another Clearview, and the dangers such mass surveillance applications engender.
I won’t be getting into code, such as explaining Python syntax or explaining how to use Conda to install specific libraries on your machine. However, you can find all the programs, tools, and data used throughout this series at github.com/samdbrice/cctview to play with.
For Those with No Programming Experience
At the end of this series, you will know:
- How “unintelligent” much of this technology is.
- Questions to ask to understand better what’s going on.
- Some practical ways to protect yourself and your privacy.
You’ll also know where to start (or end up) if you ever wanted to understand better how this technology works.
I won’t be getting into code, programming, or math and won’t put you through having to read much of that. At a high level, I’ll discuss the different technical options available for addressing specific technical challenges along the way but feel free to skim those parts.
For Those Interested In Building Deep Learning Applications
My motivation for using Deep Learning with public CCTV came from attending the Zero to Deep Learning Bootcamp hosted by Francesco Mosconi. Everything I’m teaching here I learned from Francesco, I cannot recommend his Bootcamp enough.
What I Won’t Be Getting Into
Code — If you don’t already know how to code, that won’t likely change at the end of this series. I won’t be getting into code, such as explaining Python syntax or explaining how to use Conda to install specific libraries on your machine. At a high level, I’ll discuss the different technical options available for addressing particular technical challenges along the way but feel free to skim those parts.
However, you can find all the programs, tools, and data used throughout this series at github.com/samdbrice/cctview to play with.
Table of Contents
Introduction: The End of Privacy as We Know It
Part 1: A Short History of Facial Recognition
- From Panoramic to the World Wide Web
- From the FBI to Clearview
Part 2: Collecting Data for Deep Learning Development
- The Deep Learning Development Lifecycle
- Collecting Publicly Available Data
- Scaling a Data Streaming Pipeline in the Cloud
Part 3: Choosing the Right Model for Object Detection
- Faster R-CNNs, YOLO, and Single Shot Detectors
- Using ImageAI, RetinaNet, and ResNet-50 with COCO Weights
Part 4: Running Object Detection and Structuring Data
- Scaling an Object Detection Pipeline in the Cloud
- Structuring and Naming Data for Processing
- Leveraging Amazon Machine Images
- CPUs vs. GPUs
- Restructuring Object Detection Output
Part 5: Tracking Vehicles with Deep Learning
- Face Identification and Re-Identification
- Large Scale Urban Vehicle Identification and Re-Identification
- FDR Drive Vehicle Identification and Re-Identification
- Vehicle Re-Identification Using Track-to-Track Distance
Part 6: Designing a Deep Learning App
- The Application Development Lifecycle
- FDR Drive Tracking Information Architecture Overview
- Tracker Implementation
- Demo Application Architecture Overview
Part 8: Protecting Your Privacy from Facial Recognition
- Spoiler Alert: “Adversarial Glasses” Work 90% of the Time
- Universal Adversarial Perturbations
- Practical Personal Protection from Facial Recognition
- Legal Protection from Facial Recognition
- Warning: ReID Data is Considered Personal Data
- Fatal: ReID Model Data is Considered Personal Data
Part 9: Privacy Implications of Facial Recognition
- What He’d Helped to Destroy
- Abuse of Due Process and Erosion of Equal Protection
- Fake News: Your Privacy is Not At Risk
- Real News: Your Privacy is Endangered
- First they came…
- Cloning and Instaling from GitHub
- Running the Example Script
- Playing with Jupyter Notebook
- Detect Module — Residual Network
- Extract Module — Dense Convolutional Network
- Match Module — Euclidian Distance Metric
- Raw Footage, Detections, and Features Data
- Raw CCTV Footage
