I think I might have figured it out.
I have tried the above and I think the trick is the muting of the audio all together.
I tried the "turn down nav voice the extra notch" but that didn't seem to do it all by itself - I the tried to press the mute audio-button while the nav spoke (or in this...