I think the way ancaps freak out about Everyone Taking My Precious puts the lie to the idea that they are an inherently charitable people. Love of money is the opposite of charity (by very definition), so their instinctive responses don't pass the charity smell-test.
People who are inclined to charity, by temperament, are less singularly-concerned with controlling all of their money, or angered at some of that money going to the public purse, than with the effective and efficient application of their contributions toward helping those most in need.
I think it's very difficult for people who have this sort of monetary religion to even understand that lots of us don't mind paying taxes. We don't even mind paying a lot of taxes. (You have to earn a lot of money to pay a lot of taxes, after all.) We mind the amount of our taxes we think is misapplied or wasted, the general inefficiency of government sprawl, and the lack of subsidiarity.