We took the 20s off of our 4runner for 17s and some tires with more sidewall and the ride got much smoother. The tires are definitely louder because of the aggressive tread, but the actual ride got better. It soaks up potholes and bumps in the road much more smoothly. It doesn't corner as confidently as before on smooth surfaces, but the 4runner was never exactly a touge machine anyway.
Not to mention it flies across the deep sand now without even airing down the tires.