Fast way to convert a buffer of unicode values to a (numpy) array

[craffel]

Hi all, long time lurker and first time poster. I'm writing a Python program which does analysis on mp3 files, so I'd like to read in the mp3 data as a numpy array. I'm using pymad, which seems stable and works well. When you read in a frame of data from pymad, it gets returned as a buffer of unicode hex values and characters, eg,

'\x86\x01\x02\x01\xe4\x01@\x01\xb4\x00'

The format of these values is Left channel MSB, LSB, Right channel MSB, LSB, ... and so on. So, to convert this into a numpy array, here's my function:

def bufToNumpy( inBuf ):
  bufLength = len(inBuf)
  outBuf = np.zeros(bufLength/2)
  for i in range(bufLength/2):
    outBuf[i] = struct.unpack('h', inBuf[2*i] + inBuf[2*i+1])[0]
  return outBuf

As you can see, I am using struct's unpack function to grab each pair of MSB, LSB and converting it into a 16 bit int, which is stored in a numpy array. However, because I'm doing this per-number, it takes a LONG time (for a typical song I call unpack about 10 million times). Is there a way I can vectorize this? In other words, I'd like to just call struct.unpack once and get out an array (of some kind) of 16 bit integers. Thanks for any tips!

-Colin

[craffel]

Nevermind, figured it out! You can just use numpy's built-in frombuffer() function. Eg, my function above can be replaced by

np.frombuffer(buffy, 'h')

That function is about 2000 times faster than the one I posted above, for the buffer size I was dealing with (4000 values).

-Colin