2.5 years ago, I sold my '04 Mazda3 to buy an S2000 for the same reasons as you. I'm really glad I did.
It's a real fun car to drive, and usable as a daily driver if you don't carry a lot of passengers and stuff. But it is a little noisy and cramped. Definately take it for an extended drive and after you drive the hell out of it, drive it normally and think about whether or not you could live with it as a daily.
Rob