System Restore is the best option when you face such snags. Instead of tottally delete cut it and paste it somewhere. As per Caper Jack, if it is spyware, after delete system will complain that because it have registry entry of the file. Just in that case use revouninstaller's startup utility to delete the registry entry (Or Do it manually if you can) and reboot the machine.
NOTE: BEFORE THAT MAKE BACKUP OF REGISTRY USING EITHER ERRUNT OR ANY BACKUP UTILITY!!!