Synchronizing .wav with other code?

[Hagbard_Durden]

Hello. I'm new here, and new to Python. I have a little experience with C++, and assembly, and just started learning Python about 2 weeks ago.

I'm trying to write a program that will control servo motors in sync with an audio file (probably .wav). So far I've written programs moving the a servo in sawtooth, triangular, and sine wave patterns, and I have .wav files playing through winsound.

I tried chopping the .wav file using Wave_read.readframes(n) from the wave module and passing that to winsound.PlaySound(sound, flags), but it's giving me a "RuntimeError: Failed to play sound" error.

import winsound
import sys
import wave
import winsound
import time

def play_sound(file):
    if sys.platform.startswith('win'):
       
       from winsound import PlaySound, SND_FILENAME, SND_ASYNC, SND_MEMORY
       winsound.PlaySound(file,SND_MEMORY)
    elif platform.find('linux')>-1:
       from wave import open as waveOpen
       from ossaudiodev import open as ossOpen
       s = waveOpen('tada.wav','rb')
       (nc,sw,fr,nf,comptype, compname) = s.getparams( )
       dsp = ossOpen('/dev/dsp','w')
       try:
         from ossaudiodev import AFMT_S16_NE
       except ImportError:
         if byteorder == "little":
           AFMT_S16_NE = ossaudiodev.AFMT_S16_LE
         else:
           AFMT_S16_NE = ossaudiodev.AFMT_S16_BE
       dsp.setparameters(AFMT_S16_NE, nc, fr)
       data = s.readframes(nf)
       s.close()
       dsp.write(data)
       dsp.close()


a = wave.open('tada.wav','rb')
b = a.getnframes()
c = a.readframes(b)

play_sound(c)

The idea is to chop the audio into 0.05 second intervals, moving the motors 1 step in between each time, thus keeping the motors in sync with the sound.

I suspect that even if I get Python to do this, there will be other problems, such as the sound file not sounding right.

I'm open to any suggestions about either getting this code to work, or completely different approaches to the synchronization problem.

Thanks.

[Hagbard_Durden]

Ok, I'm actually starting to lean more toward using Pygame to control audio playback and timing now.

I'm still curious why the above code didn't work, though.

[Hagbard_Durden]

Well, so far so good. I have a functioning program that I have used to train a routine. I decided to go with pyglet instead of pygame since it's more portable, and, at to me at least, easier to understand.

Here's a video of the routine I trained:

http://www.youtube.com/watch?v=0WB6OVAfUw0

and here's the code:

import pyglet
import serial
import time
import shelve

#set up the serial port for action
ser=serial.Serial(0)
ser.baudrate=9600
print ser.portstr
framerate = 0.02


window = pyglet.window.Window(fullscreen=False)

class Timer(object):
    def __init__(self):
        self.label = pyglet.text.Label('00:00', font_size=36, 
                                       x=window.width//2, y=window.height//2,
                                       anchor_x='center', anchor_y='center')
        self.reset()
        self.x = 0
        self.y = 0
        self.frame = 0
        #self.framelist = []
    def reset(self):
        self.i = 0.0
        self.time = 0
        self.running = True
        self.label.text = '00:00'
        self.label.color = (255, 255, 255, 255)

    def update(self, dt):
        
        if self.running:
            self.time += dt
            m, s = divmod(self.time, 60)
            self.label.text = '%02d:%02d' % (m, s)
            if m > 5:
                self.label.color = (180, 0, 0, 255)
                
    def wut(self, dt, motornum):
        if self.x > 254:
            self.x = 254
        
        
        if self.y > 254:
            self.y = 254


        #self.poslist = self.framelist[self.frame]
        #self.framelist.append([])
        if self.frame < len(self.framelist):
            if motornum > 0:
                self.framelist[self.frame].append(self.x)
            if motornum > 1:
                self.framelist[self.frame].append(self.y)
            #motx=chr(255)+chr(0)+chr(self.x)
            #moty=chr(255)+chr(1)+chr(self.y)
            j = 0
            for i in self.framelist[self.frame]:
                position = chr(255)+chr(j)+chr(i)
                ser.write(position)
                j += 1
            #ser.write(moty)
            #ser.write(motx)
            self.frame += 1
        
@window.event    
def on_mouse_motion(x, y, dx, dy):
    timer.x = x
    timer.y = y

@window.event
def on_key_press(symbol, modifiers):
    
    if symbol == pyglet.window.key.SPACE:
        if timer.running:
            timer.running = False
            
        else:
            if timer.time > 0:
                timer.reset()
                
            else:
                timer.running = True
                
    elif symbol == pyglet.window.key.ESCAPE:
        window.close()

@window.event
def on_draw():
    window.clear()
    timer.label.draw()

timer = Timer()


while True:
    neworold = raw_input('Are you starting a (n)ew routine or (c)ontinuing an old one?')
    if neworold == 'n' or neworold =='N':
        framepersec = raw_input('How many frames per second?')
        duration = raw_input('What is the length of the routine in seconds?')
        audio = raw_input('What is the name of the sound file you want to use?')
        motornum = raw_input('How many motors would you like to train this session \(0-2\)')
        routinename = raw_input('What would you like to name the new routine file?')
        timer.framelist = []
        framerate = int(framepersec)
        for i in range(framerate * int(duration)):
            timer.framelist.append([])
        routine = shelve.open(routinename)
        routine['framerate'] = framerate
        routine['audio'] = audio
        routine['framelist'] = timer.framelist
        break
    elif neworold == 'c' or neworold == 'C':
        routinename = raw_input('Which routine file would you like to continue?')
        motornum = raw_input('How many motors would you like to train this session \(0-2\)')
        routine = shelve.open(routinename)
        framerate = routine['framerate']
        audio = routine['audio']
        timer.framelist = routine['framelist']
        break
    else:
        print 'Please enter n or c.'
        continue


motornum = int(motornum)

bug = (1.0/framerate)

#print timer.framelist

#pyglet.clock.schedule(clock.update, 1)
pyglet.clock.schedule_interval(timer.wut, (1.0/framerate), motornum)
pyglet.clock.schedule_interval(timer.update, 1)

player = pyglet.media.Player()
source = pyglet.media.load(audio)
player.queue(source)
player.play()

pyglet.app.run()
routine['framelist'] = timer.framelist
routine.close()


ser.close()

Still plenty of work to be done, but I'm happy with the results so far.

[sneekula]

You are way ahead of me on this. Looks like a very interesting project!
What are you using to control the motor with the serial port?

[Hagbard_Durden]

I have a mini ssc ii servo controller. It works at either 2400 or 9600 baud. Using Pyserial, I just send a 3 byte command consisting of a sync marker (255) motor number (0-254) and motor position (0-254).

I can control up to 8 servos with each mini ssc ii, and they can (theoretically) be strung together to control up to 256 servos. At 9600 baud, I could realistically control up to (maybe) 26 simultaneously moving motors being updated 15 times per second.

So far I have only tried 4 signals at 15 updates per second (with only one motor attatched to the card) and it still works. I tried it with 2 signals at 30 updates per second, and the servo started lagging behind the audio after only a few seconds. By the end of the 2 minute routine, the servo was about 4 seconds behind.