It could be a bad synchro. It could be the clutch is out of alignment and 2nd gear is picky about it.
My opinion (and what I practice) is to rarely downshift into 2nd and almost never into first. If it is grinding when upshifting, that's a problem. My dad would complain I was wearing out my...