How can I detect an object in image frame using OpenCV?

Question

I am developing a rover using Raspberry Pi , that will sweep a room and pick up objects fallen on the ground . To detect the object , I am using a reference Image , that is taken right at the start of the rover's operation , and an Image (new Image) that is clicked every 10 seconds . To determine if there is a change in the image frame , I do a image subtraction between the reference image and the new image . If any difference is found , it'll draw a contour around it , and if the contour area is greater than a certain threshold (cautionary step) , it concludes that there exists an object .

I am using the following code -

import numpy as np
import cv2,time

img=cv2.imread("object1.jpg")
img1=cv2.imread("object2.jpg")
sub=cv2.subtract(img,img1)

gray=cv2.cvtColor(sub,cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(3,3),0)
_, contours, _= cv2.findContours(blur,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
c=max(contours,key=cv2.contourArea)
print(cv2.contourArea(c))

if cv2.contourArea>20000:
   print("Object detected !")

The above code just uses 2 images to calculate their difference and determine if there is an object present .Please note that I have not posted the original code here that I will be using in my project .

Now , the above code works fine for very controlled situations, say ,when the image background is very constant or there is no presence of shadow in it . But considering the fact that the rover will be moving around in the room , and there are chances that the lighting variations might trigger a false object detection , even if there is no real object in the frame . The difference might be triggered due to false contouring from shadow effects .

I want to know , if there is any other way of achieving this object detection without doing foreground/background image subtraction . I have also considered using a ultrasonic sensor to detect the object's presence , however that is not a very reliable option . I would prefer a Image processing based solution .

Thank you .

==========================================================================

EDIT 1 -

So , I decided to change the algorithm a bit . I have done thresholding on both the foreground and background Images and then performed absdiff between the binary images , to obtain any frame change(object) . The code is as follows -

import numpy as np
import cv2,time

img1=cv2.imread("back.jpeg")
blur1 = cv2.GaussianBlur(img1,(5,5),0)
gray1=cv2.cvtColor(blur1,cv2.COLOR_BGR2GRAY)
ret,thresh1 = cv2.threshold(gray1,65,255,cv2.THRESH_BINARY_INV)

img2=cv2.imread("front.jpeg")
blur2 = cv2.GaussianBlur(img2,(5,5),0)
gray2=cv2.cvtColor(blur2,cv2.COLOR_BGR2GRAY)
ret,thresh2 = cv2.threshold(gray2,65,255,cv2.THRESH_BINARY_INV)

diff=cv2.absdiff(thresh2,thresh1)
diff=cv2.bitwise_xor(diff,thresh1)

kernel = np.ones((2,2),np.uint8)
diff=cv2.erode(diff,kernel,iterations = 1)
diff=cv2.dilate(diff,kernel,iterations = 8)

_, contours, _= cv2.findContours(diff,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
c=max(contours,key=cv2.contourArea)
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(diff,(x,y),(x+w,y+h),(125,125,125),2)


cv2.imshow("thresh",diff)
cv2.waitKey(0)

"absdiff" is followed by Erosion and Dilation . After that , I find the largest contour and determine if there is an object . The images used in the algorithm are as follows -

Background Image - Background Image
Foreground Image - Foreground image
Foreground Threshold - Foreground threshold Image
Background Threshold - Background threshold Image
Difference Image - Final Image with contour and its boundary .

As you can see , the detection works fine . I have few other foreground images which I have used to test the algorithm . They have given satisfactory results .I wanted to know , if there is any other way of achieving the same result with better efficiency .

PS- All of the foreground Images have been taken with Flash ON . I have tried with Flash OFF but there seems to exist a lot of noise in the image .

=============================================================

EDIT 2-

Performance of the algo using other pictures -

Note :- The Background Image has remained same .

Object 1 - Foreground Image 1
Object 1 Detection - Foreground Image 1 Result

with object template images you can use keypoint matching like sift/surf/orb but you should have a template at least from each side of the object. — Micka
– Micka, Commented Jan 6, 2018 at 16:57
@Micka Well the object is going to differ from time to time . It can be a bottle, plastic or anything else . If your solution covers the case I mentioned , then please elaborate . I am really not aware of sift/orb method . — Boudhayan Dev
– Boudhayan Dev, Commented Jan 6, 2018 at 17:14
you must provide templates for each of the objects. In one of the original sift papers there is a description on how to use keypoint matching for object detection. But if there isnt enough texture (transparent bottle without label?) keypoint matching will probably not work. — Micka
– Micka, Commented Jan 6, 2018 at 17:44

flamelite · Accepted Answer · 2018-01-06 18:42:22Z

8

I doubt if this problem is that much simple as you have described in the question, it will be get very complex when we move in real world scenario.

But anyway assuming you have small objects only present in the room then you identify them by identifying connected components in the binary image captured and choose them based on their relative pixels sizes.

Here is Python implementation for the same:

img = cv2.imread('D:/Image/objects.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# binarize the image
ret, bw = cv2.threshold(gray, 128, 255, 
cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

# find connected components
connectivity = 4
nb_components, output, stats, centroids = 
cv2.connectedComponentsWithStats(bw, connectivity, cv2.CV_32S)
sizes = stats[1:, -1]; nb_components = nb_components - 1
min_size = 250 #threshhold value for objects in scene
img2 = np.zeros((img.shape), np.uint8)
for i in range(0, nb_components+1):
    # use if sizes[i] >= min_size: to identify your objects
    color = np.random.randint(255,size=3)
    # draw the bounding rectangele around each object
    cv2.rectangle(img2, (stats[i][0],stats[i][1]),(stats[i][0]+stats[i][2],stats[i][1]+stats[i][3]), (0,255,0), 2)
    img2[output == i + 1] = color

Image containing objects:

Detected objects using connected components labels:

edited Jan 6, 2018 at 18:42

answered Jan 6, 2018 at 17:35

flamelite

2,8744 gold badges27 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Boudhayan Dev Over a year ago

The objects in my image are not going to be entirely black or white . I know I can do a gray image conversion but it will still not result into a complete binary image . I will need to do a threshold to achieve a complete binary image. And If I do thresholding , wouldn't it be better to just find contours and find the contour with a certain size , to detect and object ? What is the use of connected components here ?

Boudhayan Dev Over a year ago

Please check the EDIT. @flamelite

flamelite Over a year ago

ok you captured those images by yourself but how would you make your rover to get the foreground and background image of the same place? Image subtraction would work for the image frames of the same position only.

Boudhayan Dev Over a year ago

Yes I am aware of that shortcoming . But the working assumption is that the background Image is not going to change abruptly . As you can see in my background and foreground Images , the background ,i.e the white tile is almost constant , except probably the tile edges . I know frame subtraction will not give a completely clean result . But it will be enough to detect the foreground object . Also , the background Image will be clicked on rover startup and it will be swapped and replaced every 10 sec with the foreground image (if object is not present) .

flamelite Over a year ago

i am not sure about that, may be you should prepare a prototype based on both the ideas and test which one works then you would get more short comings of the algorithm and change/modify accordingly.

|

fireant · Accepted Answer · 2018-01-10 07:35:03Z

0

Another approach you can consider is to use structure from motion, reconstruct the environment / point cloud, and the point cluster above the floor surface belongs to your object. Also could combine this with your background subtraction method to remove false detection like shadows that are causing issue for that method.

answered Jan 10, 2018 at 7:35

fireant

14.6k4 gold badges43 silver badges49 bronze badges

Collectives™ on Stack Overflow

How can I detect an object in image frame using OpenCV?

2 Answers 2

9 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

9 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related