941,504 Members | Top Members by Rank

Ad:
  • Python Discussion Thread
  • Unsolved
  • Views: 4204
  • Python RSS
Jul 27th, 2009
0

Webcam IR mouse control

Expand Post »
I cam up with this when I seen what Johnny Lee can do with a wiimote. Youtube it to check it out, pretty amazing.
I realized not everybody has a wiimote or a bluetooth dongle to connect it to the PC, so I used my webcam and python.
The webcam needs to have a filter on it to rid everything but the IRLED. A few layers of film negative, the black ends of them, is placed over the lens of the webcam to only allow IR light to come through. I don't have a way to click with the mouse, but use voice recognition instead to do that.
You will need VideoCapture and PIL.

Most camera shops will give you waste negatives.
Hope this helps someone.

Python Syntax (Toggle Plain Text)
  1. #Webcam IR mouse control
  2. #K.B. Carte 07/27/2009
  3. #Works only with an IR filter on the webcam.
  4. #To do this simply cut out the black part of a film
  5. #negitave, and place two or three layers over the lens
  6. #of the webcam.
  7.  
  8.  
  9.  
  10. from VideoCapture import Device
  11. import Image
  12. from ctypes import *
  13.  
  14. cam = Device()
  15. user = windll.user32
  16. res = (1440,900) #set to the resolution of the screen
  17.  
  18. def xy(im):
  19. imxy = im.getprojection()
  20. imx = imxy[0]
  21. imy = imxy[1]
  22. x = imx.index(1)
  23. y = imy.index(1)
  24. return (x,y)
  25.  
  26.  
  27. while 1:
  28. try:
  29. im = cam.getImage()
  30. im = im.resize(res)
  31. x,y = xy(im)
  32. user.SetCursorPos(x,y)
  33. except ValueError: #A value error is raised when it can't see
  34. pass #the IRLED
Similar Threads
Reputation Points: 59
Solved Threads: 33
Posting Whiz in Training
Tech B is offline Offline
268 posts
since May 2009
Jul 27th, 2009
0

Re: Webcam IR mouse control

I don't know how to edit a post after I posted it, but i mirrored the image instance with ImageOps module so the X axis would no longer be inverted. And I also found out I can use a lighter instead of an IRLED. It amazed me and my wife's family =)
Reputation Points: 59
Solved Threads: 33
Posting Whiz in Training
Tech B is offline Offline
268 posts
since May 2009
Aug 10th, 2009
0

Re: Webcam IR mouse control

I wanted to see what the webcam was seeing, so i used pygame.

Python Syntax (Toggle Plain Text)
  1. #Webcam IR mouse control 1.0
  2. #K.B. Carte Aug. 10, 2009
  3. #Works only with an IR filter on the webcam.
  4. #To do this simply cut out the black part of a film
  5. #negitave, and place two or three layers over the lens
  6. #of the webcam.
  7. ######################################################
  8.  
  9.  
  10.  
  11. #----------imports and global variabl assignment------
  12. ######################################################
  13. from VideoCapture import Device
  14. import Image, sys, pygame
  15. from ctypes import *
  16. import ImageOps
  17. from PIL import ImageEnhance
  18. from pygame.locals import *
  19. cam = Device()
  20. user = windll.user32
  21. res = (1440,900) #set to the resolution of the screen
  22. pygame.init()
  23. screen = pygame.display.set_mode((640,480))
  24. pygame.display.set_caption('IR Mouse Control')
  25. font = pygame.font.SysFont("Curier",26)
  26.  
  27. #---------------functions-----------------------
  28. ################################################
  29. #Returns the xy cordinets of the ir dot.
  30. #Doesn't return exact dot, only the first TRUE pixel value
  31. def xy(im):
  32. imxy = im.getprojection()
  33. imx = imxy[0]
  34. imy = imxy[1]
  35. x = imx.index(1)
  36. y = imy.index(1)
  37. return (x,y)
  38.  
  39. #Decides if a 'click' was called.
  40. #Returns 1 or 0 and size in pixels of the ir dot
  41. #Still in testing stage =/
  42. def irclk(im):
  43. yn = 0
  44. xi = 0
  45. yi = 0
  46. irxy = im.getprojection()
  47. irx = irxy[0]
  48. iry = irxy[1]
  49.  
  50. for i in irx:
  51. if i == 1:
  52. xi +=1
  53. for i in iry:
  54. if i == 1:
  55. yi += 1
  56. xyi = xi + yi
  57. if xyi >= 100:#***
  58. yn = 1
  59. else: yn = 0
  60. return (yn,xyi)
  61.  
  62.  
  63. #-----------Main loop--------------------
  64. #########################################
  65. while 1:
  66.  
  67. for event in pygame.event.get():
  68. if event.type == pygame.QUIT: sys.exit()
  69.  
  70. try:
  71. imt = cam.getImage()
  72. im = imt.resize(res)
  73. im = ImageOps.mirror(im)
  74. im1 = ImageEnhance.Contrast(imt).enhance(1.0)
  75. im1 = ImageEnhance.Brightness(imt).enhance(1.5)
  76. x,y = xy(im)
  77. user.SetCursorPos(x,y)
  78. name = font.render('By K.B. Carte', True, (250,250,250))
  79. web = font.render('Webcam IR mouse control : 1.0.1', True, (250,250,250))
  80. fil = font.render('Works only with an IR filter on the webcam lens', True, (250,250,250))
  81. yn = irclk(im1)
  82. xyi = yn[1]
  83. num = font.render("IR intensity:" + str(xyi), True, (250,250,250))
  84. if yn[0] ==1:
  85. cl = 'click'
  86. else: cl = ''
  87. clik = font.render(cl, True, (250,250,250))
  88. im1 = pygame.image.frombuffer(im1.tostring(), (640,480), "RGB")
  89.  
  90. screen.blit(im1, (0,0))
  91. screen.blit(name,(0,26))
  92. screen.blit(web,(0,0))
  93. screen.blit(fil,(0,52))
  94. screen.blit(clik,(0,104))
  95. screen.blit(num, (0,78))
  96. pygame.display.flip()
  97.  
  98. except ValueError: #A value error is raised when it can't see the IRLED.
  99. pass #So we ignor it.
  100. #I'm trying to maybe control the 'click' with the except
  101. #So the click will be a lack of the ir light
  102. #Still testing...
Reputation Points: 59
Solved Threads: 33
Posting Whiz in Training
Tech B is offline Offline
268 posts
since May 2009
Sep 9th, 2009
0

Re: Webcam IR mouse control

This code has the click event.

Python Syntax (Toggle Plain Text)
  1. #Webcam IR mouse control 1.6
  2. #K.B. Carte Sept. 9, 2009
  3. #Works only with an IR filter on the webcam.
  4. #To do this simply cut out the black part of a film
  5. #negitave, and place two or three layers over the lens
  6. #of the webcam.
  7. ######################################################
  8.  
  9.  
  10.  
  11. #----------imports and global variabl assignment------
  12. ######################################################
  13. from VideoCapture import Device
  14. import Image, sys, pygame
  15. from ctypes import *
  16. import ImageOps
  17. from PIL import ImageEnhance
  18. from pygame.locals import *
  19. cam = Device()
  20. user = windll.user32
  21. res = (1440,900) #set to the resolution of the screen
  22. pygame.init()
  23. screen = pygame.display.set_mode((640,480))
  24. pygame.display.set_caption('IR Mouse Control')
  25. font = pygame.font.SysFont("Curier",26)
  26.  
  27. #---------------functions-----------------------
  28. ################################################
  29. #Returns the xy cordinets of the ir dot.
  30. #Doesn't return exact dot, only the first TRUE pixel value
  31. def xy(im):
  32. imxy = im.getprojection()
  33. imx = imxy[0]
  34. imy = imxy[1]
  35. x = imx.index(1)
  36. y = imy.index(1)
  37. return (x,y)
  38.  
  39. #Returns size in pixels of the ir dot
  40. #Was going to be used for click event
  41. #No longer needed, but I think its a cool feature =]
  42. def irint(im):
  43. xyi = 0
  44. irxy = im.getprojection()
  45. irx = irxy[0]
  46. iry = irxy[1]
  47.  
  48. for i in irx:
  49. if i == 1:
  50. xyi +=1
  51. for i in iry:
  52. if i == 1:
  53. xyi += 1
  54. return xyi
  55.  
  56. ################################################################################
  57. #Used for "click" event
  58. ################################################################################
  59. #from ctypes import *
  60. user32 = windll.user32
  61. import time
  62.  
  63. PUL = POINTER(c_ulong)
  64. class KeyBdInput(Structure):
  65. _fields_ = [("wVk", c_ushort),
  66. ("wScan", c_ushort),
  67. ("dwFlags", c_ulong),
  68. ("time", c_ulong),
  69. ("dwExtraInfo", PUL)]
  70.  
  71. class HardwareInput(Structure):
  72. _fields_ = [("uMsg", c_ulong),
  73. ("wParamL", c_short),
  74. ("wParamH", c_ushort)]
  75.  
  76. class MouseInput(Structure):
  77. _fields_ = [("dx", c_long),
  78. ("dy", c_long),
  79. ("mouseData", c_ulong),
  80. ("dwFlags", c_ulong),
  81. ("time",c_ulong),
  82. ("dwExtraInfo", PUL)]
  83.  
  84. class Input_I(Union):
  85. _fields_ = [("ki", KeyBdInput),
  86. ("mi", MouseInput),
  87. ("hi", HardwareInput)]
  88.  
  89. class Input(Structure):
  90. _fields_ = [("type", c_ulong),
  91. ("ii", Input_I)]
  92.  
  93. class POINT(Structure):
  94. _fields_ = [("x", c_ulong),
  95. ("y", c_ulong)]
  96. # END SENDINPUT TYPE DECLARATIONS
  97.  
  98. FInputs = Input * 2
  99. extra = c_ulong(0)
  100.  
  101. click = Input_I()
  102. click.mi = MouseInput(0, 0, 0, 2, 0, pointer(extra))
  103. release = Input_I()
  104. release.mi = MouseInput(0, 0, 0, 4, 0, pointer(extra))
  105.  
  106. #x = FInputs( (0, click), (0, release) )
  107. #user32.SendInput(2, pointer(x), sizeof(x[0]))
  108.  
  109. ###################################################################
  110. #Work cited: Case Nelson #
  111. #http://mail.python.org/pipermail/python-list/2005-May/320761.html#
  112. ###################################################################
  113.  
  114.  
  115. def clk():
  116. x = FInputs((0, click))
  117. user32.SendInput(2, pointer(x), sizeof(x[0]))
  118. def rel():
  119. a = FInputs((0, release))
  120. user32.SendInput(2, pointer(a), sizeof(a[0]))
  121.  
  122.  
  123.  
  124. #-----------Main loop--------------------
  125. #########################################
  126. while 1:
  127.  
  128. for event in pygame.event.get():
  129. if event.type == pygame.QUIT: sys.exit()
  130.  
  131. try:
  132. rel()
  133. imt = cam.getImage()
  134. im = imt.resize(res)
  135. im = ImageOps.mirror(im)
  136. im1 = ImageEnhance.Contrast(imt).enhance(1.0)
  137. im1 = ImageEnhance.Brightness(imt).enhance(1.5)
  138. x,y = xy(im)
  139. xyi = irint(im1)
  140. user.SetCursorPos(x,y)
  141. irpix = font.render("IR intensity in pixels:" + str(xyi), True, (250,250,250))
  142. name = font.render('By K.B. Carte', True, (250,250,250))
  143. web = font.render('Webcam IR mouse control : 1.0.1', True, (250,250,250))
  144. fil = font.render('Works only with an IR filter on the webcam lens', True, (250,250,250))
  145. im1 = pygame.image.frombuffer(im1.tostring(), (640,480), "RGB")
  146. screen.blit(im1, (0,0))
  147. screen.blit(name,(0,26))
  148. screen.blit(web,(0,0))
  149. screen.blit(fil,(0,52))
  150. screen.blit(irpix,(0,78))
  151. pygame.display.flip()
  152.  
  153. except ValueError: #A value error is raised when it can't see the IRLED.
  154. clk() #I use it to register a click.
  155. pass #Not practical, thinking just hack a mouse to mount on head
  156. #and click with an momentary swtich or something.
Reputation Points: 59
Solved Threads: 33
Posting Whiz in Training
Tech B is offline Offline
268 posts
since May 2009
Nov 25th, 2009
0
Re: Webcam IR mouse control
After I got this working, I wanted to map two IR dots to mimic the WII Mote a little better. http://www.daniweb.com/forums/thread240812.html

The mouse movement will be mapped to the slope of the line.
And the whole click thing doesn't work as smoothly, so I have rigged a mouse to a hat and extended the buttons to run down to the mouth.
It seems to work ALLOT better.
Reputation Points: 59
Solved Threads: 33
Posting Whiz in Training
Tech B is offline Offline
268 posts
since May 2009

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Python Forum Timeline: ide with advanced code complation
Next Thread in Python Forum Timeline: Macro, need help





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC