How To Create A Virtual Painting App Using OpenCV?
Computer vision is a science of teaching computers to see. With the state of the art algorithms, this technology is behind many applications like self-driving cars, image recognition, medical diagnosis etc. The best part of computer vision is these techniques are used for detecting cancerous cells which helps in saving lives by adding filters to your face to entertain you. Similarly, it has a wide variety of interesting and useful applications. In this article, we will do hands-on implementation on how to create a virtual painting app without touching the keyboard, and just by drawing on-air that will be displayed on the screen. Steps for this implementation:- Importing libraries, assigning BGR values for color.Creating a window that provides various options.Capturing a video using a webcam and grabbing frames.Using different morphing techniques Displaying output So, now let’s begin with this interesting implementation. Step-1 –  Importing required libraries import numpy as np import cv2 from collections import deque Using the below code snippet, we will declare the BGR values of the red color, which detects the red color and helps us to draw. Lower_red = np.array([161, 155, 84]) Upper_red = np.array([179, 255, 255]) Here i used red colour lower and upper ranges, you can try with your own color by setting BGR values Using the below code snippet, we will use a deque function, which is used to store separate colors that helps in creating buttons on the window. bpoints = [deque(maxlen=512)] gpoints = [deque(maxlen=512)] rpoints = [deque(maxlen=512)] ypoints = [deque(maxlen=512)] bindex = 0,gindex = 0,rindex = 0,yindex = 0 colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255), (0, 255, 255)] colorIndex = 0 Step-2 – Creating a window, that provides various options. Using the below code snippet, we will create a window that displays the different color options to draw and also it creates the buttons using cv2.putText. Window = np.zeros((471,636,3)) + 255 Window = cv2.rectangle(paintWindow, (40,1), (140,65), (0,0,0), 2) Window = cv2.rectangle(paintWindow, (160,1), (255,65), colors[0], -1) Window = cv2.rectangle(paintWindow, (275,1), (370,65), colors[1], -1) Window = cv2.rectangle(paintWindow, (390,1), (485,65), colors[2], -1) Window = cv2.rectangle(paintWindow, (505,1), (600,65), colors[3], -1) cv2.putText(Window, "CLEAR ALL", (49, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 2, cv2.LINE_AA) cv2.putText(Window, "BLUE", (185, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA) cv2.putText(Window, "GREEN", (298, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA) cv2.putText(Window, "RED", (420, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA) cv2.putText(Window, "YELLOW", (520, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (150,150,150), 2, cv2.LINE_AA) cv2.namedWindow('Paint', cv2.WINDOW_AUTOSIZE) As you can see in the output we created a normal window now we need to set the webcam and place this template on the webcam video. Step-3 – Capturing a video using a webcam and grabbing frames. Here we are capturing the video, using a webcam so we set videoCapture to zero. camera = cv2.VideoCapture(0) Using the below code snippet, we will run a loop to grab the frames from the webcam and to display the dynamic results. while True: (grabbed, frame) = camera.read() frame = cv2.flip(frame, 1) hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) if not grabbed: Break Step-4 – Using different morphing techniques Using the below code snippet, we will preprocess using the input( the text drawn on-air) using morphing techniques erode, morphologyEx, dilate. kernel = np.ones((5, 5), np.uint8) Mask = cv2.inRange(hsv, Lower_red, Upper_red) Mask = cv2.erode(Mask, kernel, iterations=2) Mask = cv2.morphologyEx(Mask, cv2.MORPH_OPEN, kernel) Mask = cv2.dilate(Mask, kernel, iterations=1) Using the below code snippets, we will use counters, it holds the data in an unordered collection it allows you to count the items in the iterable list. (cnts, _) = cv2.findContours(blueMask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) center = None Using the below code snippet, when it detects the red color and around that color, we create a bounding circle. if len(cnts) > 0: cnt = sorted(cnts, key = cv2.contourArea, reverse = True)[0] ((x, y), radius) = cv2.minEnclosingCircle(cnt) cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2) M = cv2.moments(cnt) center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00']) In the below code snippet, if counters are stored then it runs the loop. using the deque function. if center[1] <= 65: if 40 <= center[0] <= 140: # Clear All bpoints = [deque(maxlen=512)] gpoints = [deque(maxlen=512)] rpoints = [deque(maxlen=512)] ypoints = [deque(maxlen=512)] bindex = 0, gindex = 0, rindex = 0,yindex = 0 paintWindow[67:,:,:] = 255 elif 160 <= center[0] <= 255: colorIndex = 0 elif 275 <= center[0] <= 370: colorIndex = 1 elif 390 <= center[0] <= 485: colorIndex = 2 elif 505 <= center[0] <= 600: colorIndex = 3 else : if colorIndex == 0: bpoints[bindex].appendleft(center) elif colorIndex == 1: gpoints[gindex].appendleft(center) elif colorIndex == 2: rpoints[rindex].appendleft(center) elif colorIndex == 3: ypoints[yindex].appendleft(center) Step-5 – Displaying output In the below code snippet, we are implementing the draw option by taking our designed window and applying it to the webcam video. points = [bpoints, gpoints, rpoints, ypoints] for i in range(len(points)): for j in range(len(points[i])): for k in range(1, len(points[i][j])): if points[i][j][k - 1] is None or points[i][j][k] is None: continue cv2.line(frame, points[i][j][k - 1], points[i][j][k], colors[i], 2) cv2.line(paintWindow, points[i][j][k - 1], points[i][j][k], colors[i], 2) In the below code snippet, we are displaying the output video cv2.imshow("Web_cam_video", frame) cv2.imshow("Paint", paintWindow) Output video. Although my drawings are bad,  it’s fun to implement. Conclusion In the above implementation, we demonstrated how to create a virtual paint application using OpenCV tool. We had given lower and upper ranges of red color so we can use red color as a pencil to draw virtually and it reflects on your system screen. Although my drawings are bad,  it’s fun to implement.
Computer vision is a science of teaching computers to see. With the state of the art algorithms, this technology is behind many applications like self-driving cars, image recognition, medical diagnosis etc. The best part of computer vision is these techniques are used for detecting cancerous cells which helps in saving lives by adding filters to […]
hands-on implementation on how to create a virtual painting app without touching the keyboard, just by drawing on-air will be displayed
[]
["Go", "NumPy", "TPU", "programming_languages:R", "AI", "image recognition", "computer vision", "OpenCV", "Ray", "R"]
["AI", "computer vision", "Ray", "OpenCV", "NumPy", "image recognition", "TPU", "R", "Go", "programming_languages:R"]
AI enthusiast, Currently working with Analytics India Magazine. I have experience of working with  Machine learning, Deep learning real-time problems, Neural networks, structuring and machine learning projects. I am a Computer Vision researcher and I am Interested in solving real-time computer vision problems.