Defense labor rates are similarly bloated. 15 years ago, my company billed the DoD $180/hr for programmer labor. Our typical programmer pay was $90K/yr, or less than $45/hr.
As for contracting... my life got better once I decided to go for direct contracting with my customer. It sucks setting up invoices and that stuff with the customer's accounts payable, and it royally sucks calculating monthly/quarterly self-employment and other taxes.
But once you get it set up... so much better. Dealing with your customer and their finance folks is SO MUCH BETTER than dealing with some agency's HR folks.
Even you raise your rates to cover the accounting and billing and extra taxes and health insurance and 401K... you're still a cheaper option for your customer than an agency. Most people have no idea how much money they give up for the convenience of working for an agency.