It's actually the other way round.
In the office, on-site, you validated that you are working by our presence.
So you can literally spend as much time as you want doing non-work stuff, gossip, …
Extra points if you manage to take naps.
Now in a remote working situation, obviously, that status if you are working is evaluated by your work output. So yeah, you can still do other things, but you have to self-police, so at the end of the day, you can show plausible output.