Webcam IR mouse control

Reply

•
•
Join Date: May 2009
Posts: 104
Reputation: Tech B is an unknown quantity at this point 
Solved Threads: 9
Tech B's Avatar
Tech B Tech B is offline Offline
Junior Poster

Webcam IR mouse control

 
0
  #1
Jul 27th, 2009
I cam up with this when I seen what Johnny Lee can do with a wiimote. Youtube it to check it out, pretty amazing.
I realized not everybody has a wiimote or a bluetooth dongle to connect it to the PC, so I used my webcam and python.
The webcam needs to have a filter on it to rid everything but the IRLED. A few layers of film negative, the black ends of them, is placed over the lens of the webcam to only allow IR light to come through. I don't have a way to click with the mouse, but use voice recognition instead to do that.
You will need VideoCapture and PIL.

Most camera shops will give you waste negatives.
Hope this helps someone.

Python Syntax (Toggle Plain Text) 
  1. #Webcam IR mouse control
  2. #K.B. Carte 07/27/2009
  3. #Works only with an IR filter on the webcam.
  4. #To do this simply cut out the black part of a film
  5. #negitave, and place two or three layers over the lens
  6. #of the webcam. 
  7.  
  8.  
  9.  
  10. from VideoCapture import Device
  11. import Image
  12. from ctypes import *
  13.  
  14. cam = Device()
  15. user = windll.user32
  16. res = (1440,900)      #set to the resolution of the screen
  17.  
  18. def xy(im):
  19.   imxy = im.getprojection()
  20.   imx = imxy[0]
  21.   imy = imxy[1]
  22.   x = imx.index(1)
  23.   y = imy.index(1)
  24.   return (x,y)
  25.  
  26.  
  27. while 1:
  28.   try:
  29.     im = cam.getImage()
  30.     im = im.resize(res)
  31.     x,y = xy(im)
  32.     user.SetCursorPos(x,y)
  33.   except ValueError:   #A value error is raised when it can't see 
  34.     pass               #the IRLED
TECH-B
Reply With Quote Quick reply to this message  
•
•
Join Date: May 2009
Posts: 104
Reputation: Tech B is an unknown quantity at this point 
Solved Threads: 9
Tech B's Avatar
Tech B Tech B is offline Offline
Junior Poster

Re: Webcam IR mouse control

 
0
  #2
Jul 27th, 2009
I don't know how to edit a post after I posted it, but i mirrored the image instance with ImageOps module so the X axis would no longer be inverted. And I also found out I can use a lighter instead of an IRLED. It amazed me and my wife's family =)
TECH-B
Reply With Quote Quick reply to this message  
•
•
Join Date: May 2009
Posts: 104
Reputation: Tech B is an unknown quantity at this point 
Solved Threads: 9
Tech B's Avatar
Tech B Tech B is offline Offline
Junior Poster

Re: Webcam IR mouse control

 
0
  #3
Aug 10th, 2009
I wanted to see what the webcam was seeing, so i used pygame.

Python Syntax (Toggle Plain Text) 
  1.  #Webcam IR mouse control 1.0
  2. #K.B. Carte Aug. 10, 2009
  3. #Works only with an IR filter on the webcam.
  4. #To do this simply cut out the black part of a film
  5. #negitave, and place two or three layers over the lens
  6. #of the webcam.
  7. ######################################################
  8.  
  9.  
  10.  
  11. #----------imports and global variabl assignment------
  12. ######################################################
  13. from VideoCapture import Device
  14. import Image, sys, pygame
  15. from ctypes import *
  16. import ImageOps
  17. from PIL import ImageEnhance
  18. from pygame.locals import *
  19. cam = Device()
  20. user = windll.user32
  21. res = (1440,900) #set to the resolution of the screen
  22. pygame.init()
  23. screen = pygame.display.set_mode((640,480))
  24. pygame.display.set_caption('IR Mouse Control')
  25. font = pygame.font.SysFont("Curier",26)
  26.  
  27. #---------------functions-----------------------
  28. ################################################
  29. #Returns the xy cordinets of the ir dot.
  30. #Doesn't return exact dot, only the first TRUE pixel value
  31. def xy(im):
  32.   imxy = im.getprojection()
  33.   imx = imxy[0]
  34.   imy = imxy[1]
  35.   x = imx.index(1)
  36.   y = imy.index(1)
  37.   return (x,y)
  38.  
  39. #Decides if a 'click' was called.
  40. #Returns 1 or 0 and size in pixels of the ir dot
  41. #Still in testing stage =/
  42. def irclk(im):
  43.   yn = 0
  44.   xi = 0
  45.   yi = 0
  46.   irxy = im.getprojection()
  47.   irx = irxy[0]
  48.   iry = irxy[1]
  49.  
  50.   for i in irx:
  51.     if i == 1:
  52.       xi +=1
  53.   for i in iry:
  54.     if i == 1:
  55.       yi += 1
  56.   xyi = xi + yi
  57.   if xyi >= 100:#***
  58.     yn = 1
  59.   else: yn = 0
  60.   return (yn,xyi)
  61.  
  62.  
  63. #-----------Main loop--------------------
  64. #########################################
  65. while 1:
  66.  
  67.   for event in pygame.event.get():
  68.     if event.type == pygame.QUIT: sys.exit()
  69.  
  70.   try:
  71.     imt = cam.getImage()
  72.     im = imt.resize(res)
  73.     im = ImageOps.mirror(im)
  74.     im1 = ImageEnhance.Contrast(imt).enhance(1.0)
  75.     im1 = ImageEnhance.Brightness(imt).enhance(1.5)
  76.     x,y = xy(im)
  77.     user.SetCursorPos(x,y)
  78.     name = font.render('By K.B. Carte', True, (250,250,250))
  79.     web = font.render('Webcam IR mouse control : 1.0.1', True, (250,250,250))
  80.     fil = font.render('Works only with an IR filter on the webcam lens', True, (250,250,250))
  81.     yn = irclk(im1)
  82.     xyi = yn[1]
  83.     num = font.render("IR intensity:" + str(xyi), True, (250,250,250))
  84.     if yn[0] ==1:
  85.       cl = 'click'
  86.     else: cl = '' 
  87.     clik = font.render(cl, True, (250,250,250))
  88.     im1 = pygame.image.frombuffer(im1.tostring(), (640,480), "RGB")
  89.  
  90.     screen.blit(im1, (0,0))
  91.     screen.blit(name,(0,26))
  92.     screen.blit(web,(0,0))
  93.     screen.blit(fil,(0,52))
  94.     screen.blit(clik,(0,104))
  95.     screen.blit(num, (0,78))
  96.     pygame.display.flip()
  97.  
  98.   except ValueError:   #A value error is raised when it can't see the IRLED.
  99.     pass               #So we ignor it.
  100.                        #I'm trying to maybe control the 'click' with the except
  101.                        #So the click will be a lack of the ir light
  102.                        #Still testing...
TECH-B
Reply With Quote Quick reply to this message  
•
•
Join Date: May 2009
Posts: 104
Reputation: Tech B is an unknown quantity at this point 
Solved Threads: 9
Tech B's Avatar
Tech B Tech B is offline Offline
Junior Poster

Re: Webcam IR mouse control

 
0
  #4
Sep 9th, 2009
This code has the click event.

Python Syntax (Toggle Plain Text) 
  1.  #Webcam IR mouse control 1.6
  2. #K.B. Carte Sept. 9, 2009
  3. #Works only with an IR filter on the webcam.
  4. #To do this simply cut out the black part of a film
  5. #negitave, and place two or three layers over the lens
  6. #of the webcam.
  7. ######################################################
  8.  
  9.  
  10.  
  11. #----------imports and global variabl assignment------
  12. ######################################################
  13. from VideoCapture import Device
  14. import Image, sys, pygame
  15. from ctypes import *
  16. import ImageOps
  17. from PIL import ImageEnhance
  18. from pygame.locals import *
  19. cam = Device()
  20. user = windll.user32
  21. res = (1440,900) #set to the resolution of the screen
  22. pygame.init()
  23. screen = pygame.display.set_mode((640,480))
  24. pygame.display.set_caption('IR Mouse Control')
  25. font = pygame.font.SysFont("Curier",26)
  26.  
  27. #---------------functions-----------------------
  28. ################################################
  29. #Returns the xy cordinets of the ir dot.
  30. #Doesn't return exact dot, only the first TRUE pixel value
  31. def xy(im):
  32.   imxy = im.getprojection()
  33.   imx = imxy[0]
  34.   imy = imxy[1]
  35.   x = imx.index(1)
  36.   y = imy.index(1)
  37.   return (x,y)
  38.  
  39. #Returns size in pixels of the ir dot
  40. #Was going to be used for click event
  41. #No longer needed, but I think its a cool feature =]
  42. def irint(im):
  43.   xyi = 0
  44.   irxy = im.getprojection()
  45.   irx = irxy[0]
  46.   iry = irxy[1]
  47.  
  48.   for i in irx:
  49.     if i == 1:
  50.       xyi +=1
  51.   for i in iry:
  52.     if i == 1:
  53.       xyi += 1
  54.   return xyi
  55.  
  56. ################################################################################
  57. #Used for "click" event
  58. ################################################################################
  59. #from ctypes import *
  60. user32 = windll.user32
  61. import time
  62.  
  63. PUL = POINTER(c_ulong)
  64. class KeyBdInput(Structure):
  65.     _fields_ = [("wVk", c_ushort),
  66.              ("wScan", c_ushort),
  67.              ("dwFlags", c_ulong),
  68.              ("time", c_ulong),
  69.              ("dwExtraInfo", PUL)]
  70.  
  71. class HardwareInput(Structure):
  72.     _fields_ = [("uMsg", c_ulong),
  73.              ("wParamL", c_short),
  74.              ("wParamH", c_ushort)]
  75.  
  76. class MouseInput(Structure):
  77.     _fields_ = [("dx", c_long),
  78.              ("dy", c_long),
  79.              ("mouseData", c_ulong),
  80.              ("dwFlags", c_ulong),
  81.              ("time",c_ulong),
  82.              ("dwExtraInfo", PUL)]
  83.  
  84. class Input_I(Union):
  85.     _fields_ = [("ki", KeyBdInput),
  86.               ("mi", MouseInput),
  87.               ("hi", HardwareInput)]
  88.  
  89. class Input(Structure):
  90.     _fields_ = [("type", c_ulong),
  91.              ("ii", Input_I)]
  92.  
  93. class POINT(Structure):
  94.     _fields_ = [("x", c_ulong),
  95.              ("y", c_ulong)]
  96. # END SENDINPUT TYPE DECLARATIONS
  97.  
  98. FInputs = Input * 2
  99. extra = c_ulong(0)
  100.  
  101. click = Input_I()
  102. click.mi = MouseInput(0, 0, 0, 2, 0, pointer(extra))
  103. release = Input_I()
  104. release.mi = MouseInput(0, 0, 0, 4, 0, pointer(extra))
  105.  
  106. #x = FInputs( (0, click), (0, release) )
  107. #user32.SendInput(2, pointer(x), sizeof(x[0]))
  108.  
  109. ###################################################################
  110. #Work cited: Case Nelson                                          # 
  111. #http://mail.python.org/pipermail/python-list/2005-May/320761.html#
  112. ###################################################################
  113.  
  114.  
  115. def clk():
  116.   x = FInputs((0, click))
  117.   user32.SendInput(2, pointer(x), sizeof(x[0]))
  118. def rel():
  119.   a = FInputs((0, release))
  120.   user32.SendInput(2, pointer(a), sizeof(a[0]))
  121.  
  122.  
  123.  
  124. #-----------Main loop--------------------
  125. #########################################
  126. while 1:
  127.  
  128.   for event in pygame.event.get():
  129.     if event.type == pygame.QUIT: sys.exit()
  130.  
  131.   try:
  132.     rel()
  133.     imt = cam.getImage()
  134.     im = imt.resize(res)
  135.     im = ImageOps.mirror(im)
  136.     im1 = ImageEnhance.Contrast(imt).enhance(1.0)
  137.     im1 = ImageEnhance.Brightness(imt).enhance(1.5)
  138.     x,y = xy(im)
  139.     xyi = irint(im1)
  140.     user.SetCursorPos(x,y)
  141.     irpix = font.render("IR intensity in pixels:" + str(xyi), True, (250,250,250))
  142.     name = font.render('By K.B. Carte', True, (250,250,250))
  143.     web = font.render('Webcam IR mouse control : 1.0.1', True, (250,250,250))
  144.     fil = font.render('Works only with an IR filter on the webcam lens', True, (250,250,250))
  145.     im1 = pygame.image.frombuffer(im1.tostring(), (640,480), "RGB") 
  146.     screen.blit(im1, (0,0))
  147.     screen.blit(name,(0,26))
  148.     screen.blit(web,(0,0))
  149.     screen.blit(fil,(0,52))
  150.     screen.blit(irpix,(0,78))
  151.     pygame.display.flip()
  152.  
  153.   except ValueError:   #A value error is raised when it can't see the IRLED.
  154.     clk()             #I use it to register a click.
  155.     pass               #Not practical, thinking just hack a mouse to mount on head
  156.                        #and click with an momentary swtich or something.
TECH-B
Reply With Quote Quick reply to this message  
•
•
Join Date: May 2009
Posts: 104
Reputation: Tech B is an unknown quantity at this point 
Solved Threads: 9
Tech B's Avatar
Tech B Tech B is offline Offline
Junior Poster
 
0
  #5
Nov 25th, 2009
After I got this working, I wanted to map two IR dots to mimic the WII Mote a little better. http://www.daniweb.com/forums/thread240812.html

The mouse movement will be mapped to the slope of the line.
And the whole click thing doesn't work as smoothly, so I have rigged a mouse to a hat and extended the buttons to run down to the mouth.
It seems to work ALLOT better.
TECH-B
Reply With Quote Quick reply to this message  
Reply

Quick Reply to Thread
This thread is more than three months old.
Perhaps start a new thread instead?
•
•
•
•
•
•
•
•
Message:
•
•
•
•



Similar Threads
Other Threads in the Python Forum


Views: 1932 | Replies: 4
Thread Tools Search this Thread



Tag cloud for Python
application array beginner c++ c/c++ change character class client code command convert count create csv ctypes curved database dictionary django dll error examples excel exe extensions fdlib file float format framework ftp function graphics gui homework image images import input launcher library line linux list lists logging loop loops microcontroller mouse movingimageswithpygame mysql mysqldb number numbers output parse parsing path port prime processing program programming py2exe pygame pygtk pyqt python random raw_input recursion recursive redirect remote scrolledtext server socket ssh stdout string strings table terminal text thread threading tkinter transparency trick tuple tutorial ubuntu unicode variable variables web windows wxpython
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2010 DaniWeb® LLC