I think you're right that it trickles down to the shareholders and employees, but the same is true for buying it direct from Wal-mart.
The rich are rich because they own things: stocks, businesses, etc. That money is not sitting around doing nothing; it's the pool you are borrowing from.