My wife has complained off and on about this and today I really experienced it for the first time myself. The weather was extremely hot and humid here today so we were using the AC on maximum, but at one point a phone call came in and the HVAC fan speed reduced probably 2/3'rds during the call.
Because of the heat, humidity, and sun...the car started to heat up.
Sometimes my wife talks with friends for 45+ minutes during long commutes, so I can understand how this bothers her.
I know that this is normal behaviour, but can it be overridden? I looked in the config menus but didn't see anything obvious.