We've just noticed a glitch in our software that allows undeliverable emails to be bouncing around in our servers, is there a way to catch these undeliverable emails after x amount of time?

We use resin for our applications, and I've read a little about their catch setBounceAddress method, but never used it.

Does anyone know the best way to handle this situation?

nice isn't it.
You'd need a mailclient which never bounces anything and deliberately accepts all bounced messages.
How to detect whether a message is a bounce message is up to you.