Clearly, as times goes by and your principal drops, the tax advantage (being able to deduct the interest paid) will lessen. The bottom line is (when compared to an apt for example) it's nice to be able to deduct that amount... For most home owners, it's the largest deduction they'll take each year so it is significant. One of the secrets of the rich... pay as little tax as possible.
As far as the "little impact on your finances" goes... That's not necessarily true.
To keep things as simple as possible, let's say you go out and buy a brand new MS3 and it costs $25k. Maybe you have $25k in the bank so the thought of just buying it outright comes to mind. Granted, you'd then be left with 0 but at least you wouldn't have a payment to worry about.
Buying it outright might offer the most peace of mind... but that's purely emotional and, as we all know, sometimes the higher our emotions get the further away logic seems to be. By throwing all $25k into a car, you're no longer able to invest that $25k. If you're savvy enough, you should be able to (by utilizing the many tricks of the trade, including diversification) gain a solid 10% over the next 5 years. This would net you roughly $16k in profit thus increasing your funds to $41k As most should know by now, the more you have the more you can make. Sort of like the whole rich get richer thing.
Now if you elect to instead pay for the car all at once... You obviously won't be able to invest that money and make yourself 10%. Instead, you'll essentially be saving the money you would have paid in interest.
Assuming a 5 year loan at 8% (not unheard of these days) you're paying roughly $5k towards interest.
By financing the car (even though you could afford it outright) and making wise investments, (in this example) you're able to net $11k. Lose $5k on interest but make $16k on your investments.
For most, they either don't realize this... or feel it's too risky. Perhaps it's simply not even worth it! This is why I said it's up for debate.
The numbers (if you did this sort of thing for your house however) would blow you away. Talk about a no-brainer! That's why most wealthy people carry mortgages (even though they could easily pay them off.) They know their money is better off invested.