I do accidentally hit the scroll wheel on the desktop from time to time, mainly by missing the intended window. In XFCE, which I use for the time being, this switches Workspace, but the big thing is I can scroll the other way to undo.
A UX principle is that if some action triggers some behavior, the same or similar behavior should undo it, so if scrolling the mouse opens a menu, you shouldn't have to hit escape to close it, for instance. Still doable, but possibly awkward.