3" on each side should be plenty if you're just pulling straight in/out? If you do happen to clip the garage door frame with a mirror, it should not cause too much, if any, damage as the mirrors fold forward and backwards so that they don't simply snap off when pushed on in a parking lot or in...