Your are correct and it can be spent on other things. I can use this as a comparison.
Golfers. I love to golf! I never understood why people spend 3-400 dollars on a nice driver. They may use that 1 wood 10-12 times in one round. So to me its not the most important club in your arsenal. Now the putter. I golf with a guy who has a 400 dollar driver and a 20 dollar putter. WTF! You will use your putter at the minumum 1 time a hole, but usually 2-3 times a hole. So why would someone put 400 dollars into a driver when they use it 10-12 times but 20 dollars in a putter they may use 25-35 times! I think the putter is the most important club in your bag. Now all clubs are important but getting on the green is only half of it putting is the other half.
Another comparison- Computers, I would put most my money into the monitor. Its the interface that you use the most. A good keyboard and mouse should be next. I can go on and on!
Have you ever seen a nice car from the factory but it had bad looking wheels. The wheels to me are one of the most important asthetic tools of making a nice looking car into a great looking car. Skimping on the wheels will make or break a car!
Good luck with your decision but in the end you need to be happy. You being happy with your car is most important! Oh and I have a CAI on order. I had to wait a extra 2 weeks to buy it because I payed more for my wheels. I had to save up a extra 2 weeks to get the CAI! It didnt kill me to wait. Money is revolving, it always comes back if you wait long enough.