Strong disagree.
Question: if you had to interact with the IRS due to a problem or an audit, do you want the person to be competent?
Alternate proposal: take the job at the IRS. Be fantastic at your job. Change the culture of the organization from within.