Before you choose a language first decide what you plan to support in your IVR application. For one, any device drivers (e.g. Dialogic interface code) is typically done in C/C++. Also, a IVR system needs to be real-time so this against is best served by C/C++ ... for the core components. What features: TTS, STT, ASR, FAX, and what kind of signalling? North American or European too?
In short, VB6 is not a very good choice for much more than scripting support, but VB is fine if you switch to .NET. The reason is using .NET as your development platform is a far better choice because 1) VB6 and VC6 are not going to be supported much longer, 2) .NET has a much improved I/O model in C#,VB or J#. Pick the language you like best . However you can still write you system programs in C/C++. There is the added benefit of ASP .NET can make for some slick administrative and scripting programs. Finally a .NET compliant IVR with .NET Framework security is a compelling sell point to IT customers.