Unix is a compromise, like everything else.
There are operating systems that use message passing as a means of passing data between applications. Many microkernel OSes use that mechanism, including Microsoft Windows NT oddly enough (which is a "layered" kernel, not a true microkernel… although the concept got polluted quite heavily with NT 4.0 and later).
Having a shell that does everything could work, but then if you encounter a function your shell doesn't provide, you then have to obtain a new shell. The Unix way would be you install a tool the shell can call to perform that function.
If the shell, applications and kernel were baked into one monolithic codebase, you've got basically what we had in the 8-bit computer era: a ROM cartridge that combined the OS and the application, that physically plugged into the computer's memory bus. To change any part, you replace the cartridge, and combining applications is impossible.