Computer Vision (CV) is a term used in AI industry in the field of understanding images by a machine. In other words a machine is able to detect and recognise a specified image pattern or even is able to create one. From autonomous cars, security systems through text recognition CV is used widely.
Probably a leader, top 1 and most commonly used package in the CV field is OpenCV package.
What’s OpenCV and why it’s TOP 1?
Open Source Computer Vision Library (OpenCV) is an open source computer vision and machine learning software library — as described on the official webpage.
OpenCV is completely free to use for both non-commercial and commercial purposes (currently under Apache 2 licence).
OpenCV can be easily incorporated into applications written in C++, Python, JAVA and MATLAB through interfaces. It supports Windows, Linux, Android and Mac OS X.
OpenCV has almost 50 thousand people of user community and estimated number of downloads close to 19 million.
Ok, let’s start and build a home security system in 30 minutes of coding.
Note: the emphasis here is to learn OpenCV — not to build a complete, professional and bug free program (even though that the code works and may really help you to increase safety).
What we need:
- Python > 3.7
- PC with Windows 10
- Camera (such that can display image on a screen)
- Email account with SMTP (e.g.: gmail, yahoo)
Security system program that we’re going to write needs three functionalities:
- Grab an image from a camera
- Detect objects on an image from the camera
- Send those images if anything suspicious detected
Grab an image from a camera:
There are many possibilities on how to get an image from a camera, but we’re going with the simplest approach of taking a screenshot from a camera windows on the desktop (background mode does not work with all camera windows so implementation is for simple screenshot functionality as of now). In a later “versions” I will create a tutorial on other possibilities.
There is a pyautogui.py package for taking a simple screenshot in Python. And we can simply use it as:
p = pyautogui.screenshot()
Then we can crop an image to delete windows menu etc… from the picture like this:
Note: this you will have to change depending on your camera window size and screen resolution.
p = p.crop((230, 80, 1900, 920))
Below we have the complete function with just one parameter — secs — that is used to tell a program on the frequency of screenshots taking.
def take_screenshot(secs): now = datetime.now().strftime(“%H_%M_%S”)
time.sleep(secs) Path(os.getcwd() + PICTURES_FOLDER).mkdir(parents=True, exist_ok=True) p = pyautogui.screenshot() p = p.crop((230, 80, 1900, 920))p.save(os.getcwd() + PICTURES_FOLDER + IMG_SUBNAME + str(now) + PNG)
return os.getcwd() + PICTURES_FOLDER + IMG_SUBNAME + str(now) + PNG
Detect objects on an image from the camera:
For detecting objects we’ll off-course use OpenCV. Basics steps are:
- read image:
img = cv2.imread(path_img)
2. load OpenCV trained classifier:
_tracker = cv2.CascadeClassifier(os.getcwd() + XMLS_FOLDER+ classifier + XML)
Trained classifiers are provided as a xml file directly on the OpenCV github repo.
Note: download them and place them under xml folder
Then we can just detect objects through:
Note: gray_img is just a grayed version of original image to enhance speed
Complete detector.py module below:
import cv2import osPICTURES_FOLDER = ‘\\Pictures\\’PROCESSED_PNG = ‘processed.png’fullbody_classifier = ‘fullbody’upperbody_classifier = ‘upperbody’face1_classifier = ‘face1’face2_classifier = ‘face2’face3_classifier = ‘face3’face4_classifier = ‘face4’face5_classifier = ‘face5’XMLS_FOLDER = ‘\\xmls\\’XML = ‘.xml’classifiers_list = [fullbody_classifier, upperbody_classifier,face1_classifier,face2_classifier, face3_classifier,face4_classifier, face5_classifier]def detect_objects(path_img): save = False img = cv2.imread(path_img) gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)for classifier in classifiers_list: _tracker = cv2.CascadeClassifier(os.getcwd() + XMLS_FOLDER + classifier + XML) objects = _tracker.detectMultiScale(gray_img)if len(objects) > 0: save = Truefor (x, y, w, h) in objects: cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 2) cv2.putText(img, classifier, (x, y — 10), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)if save: cv2.imwrite(path_img[:-4] + PROCESSED_PNG, img) return path_img[:-4] + PROCESSED_PNGreturn ‘no_objects’
Sending images if anything suspicious detected:
For sending out emails we are going to use a library email in python.
In order to send out an email you’ll have to give your login credentials (login and password) in the code. That’s why it’s very important to store it in separate file and just import the credentials as:
from src.credentials import (login, password)
Here’s a complete send_email.py module:
import smtplibfrom email.mime.multipart import MIMEMultipartfrom email.mime.base import MIMEBasefrom email.mime.text import MIMETextfrom email import encodersfrom threading import Threadfrom src.credentials import (login, password)class EmailThread(Thread): def __init__(self, source): self.source = source super().__init__() def run(self): msg = MIMEMultipart() msg[‘From’] = login msg[‘To’] = login msg[‘Subject’] = ‘New objects detected in the garden!’ body = ‘Attached new images’ msg.attach(MIMEText(body, ‘plain’)) attachment = open(self.source, ‘rb’) part = MIMEBase(‘application’, “octet-stream”) part.set_payload(attachment.read()) encoders.encode_base64(part) part.add_header(‘Content-Disposition’, “attachment; “ “filename=” + ‘screenshot’) msg.attach(part) server = smtplib.SMTP(‘smtp.gmail.com’, 587) server.ehlo() server.starttls() server.ehlo() server.login(login, password) server.send_message(msg)
One important thing of a way of implementing it here is wrapping email sending functionality in a Thread class. Separate thread will make the program to run smoothly, even when sending out emails is taking very long.
Below is the complete main.py module.
Note: find out what’s pyton main function
from datetime import datetime, timedeltafrom pathlib import Pathimport pygetwindow as gwimport osimport pyautoguiimport timefrom src.detector import detect_objectsfrom src.send_mail import EmailThreadPICTURES_FOLDER = ‘\\Pictures\\’PNG = ‘_.png’IMG_SUBNAME = ‘a_’SECONDS_TIME = 6000 # 10 minutesWINDOW_TITLE = ‘camera_window_title’def take_screenshot(secs): now = datetime.now().strftime(“%H_%M_%S”) time.sleep(secs) Path(os.getcwd() + PICTURES_FOLDER).mkdir(parents=True, exist_ok=True) p = pyautogui.screenshot() p = p.crop((230, 80, 1900, 920)) p.save(os.getcwd() + PICTURES_FOLDER + IMG_SUBNAME + str(now) + PNG) return os.getcwd() + PICTURES_FOLDER + IMG_SUBNAME + str(now) + PNGif __name__ == “__main__”:
verbose = False
seconds = 1 number_of_screenshots = 3 print(“Started script: “ + datetime.now().strftime(“%H:%M:%S”)) end_time = datetime.now() + timedelta(seconds=number_of_screenshots) print(“Script will end at: “ + end_time.strftime(“%H:%M:%S”))if verbose: z1 = gw.getAllTitles() print(z1) this = gw.getWindowsWithTitle(WINDOW_TITLE) this.maximize()for a in range(number_of_screenshots): path = take_screenshot(seconds) processed_path = detect_objects(path) if processed_path != ‘no_objects’: def thread_function(processed_path_image): email_sender = EmailThread(processed_path_image) email_sender.start() print(‘Detected objects, sending mail’) thread_function(processed_path) this.minimize()
How to run the program:
- Make sure your camera is on
- Run below code and find a window title of the camera (you need to run it only once to find out the camera window title)
import pygetwindow as gw
z1 = gw.getAllTitles()
3. Adjust cropping points of the screenshot to align to your monitor
p = p.crop((230, 80, 1900, 920))
4. Create credentials.py module with your login and password for mail
Note: in this link you’ll find on how to increase security of credential file.
from src.credentials import (login, password)
login = ‘your login’
password = ‘your password’
5. run program and wait for mails :)
Hope it works for you. Cheers.