You're playing with the temperature and pressure relief valve. It's a safety system, you're meant to pull the trigger on it every few months (releasing air in the top valve and water out of a waste pipe.)
They're meant to be replaced periodically (2-5 years depending on water hardness). My guess based on the way you're stopping the noise - the valve needs replacing.