It's a stupid remark but who knows it helps...
Maybe the CPU chip and the heatsink above it don't touch each other very well so the temperature rises massively when you boot the computer. If you really unplugged everything already, i think it's worth trying to take off the CPU fan + heatsink and check the cooling paste ( maybe add a bit if necessary).

There are no special beeps that could alert you?

I had a similar problem once and the cause of that was high mobo + CPU temperatures. But in your case i would also think it's the AGP card tbh.