I think the issue is that there is no way to turn off the audio system and still get the voice directions from the Nav system. On my 2019 I have found two solutions:
1. Get into the Settings menu and crank the volume for the Nav way up. That way if you are listening to the radio or music at a reasonable level you will be able to hear the voice directions above the sound of the music.
2. If you don't want any background sound at all and just want the Nav audio, select a source like USB or Aux that doesn't have anything connected to it. Then the steering wheel or console volume control will only be for the Nav sound.
At least I think that should work. I pretty much always have SiriusXM or music from a USB stick on when driving with some occasional AM radio for local news or talk shows.