Shutting down (not just mute) the music without shutting down the infotainment system

I feel stupid for even asking this..... but how do I turn off the radio/music/Pandora without turning off the infotainment system? I know about the mute button will stop the noise, but when I check my phone it is still pulling either my songs from the memory or linking to Pandora.

Every time I start the car it immediately starts playing tunes via the Bluetooth connection, or the USB connection. It also happens with audio books. If I was listening to a book when I last used the car, it automatically starting playing the book again as soon as I start the car. If I shut down the whole system, I lose the access to car play for maps, calls and texts.

I feel like I am in a constant fight with the system to get it do what I want. There has to be something I am missing.