If it had been me doing this, I'd redirect to a self-created webpage that talks about how stealing WiFi is wrong, and that the FBI has been sent to the person's location. For added legitimacy, you could write a PHP script that says "Your IP address is: xxx.xxx.xxx.xxx and has been recorded."
Yeah. I can imagine how the neighbors stealing the guy's connection must be flustered. Imagine them coming to this guy and saying "Er, excuse me, but your internet connection's gone wonkers. Can you fix it fast please? I need to chat to my GF urgently, and I'm too cheap to renew my account."