Max power is created by optimizing all airflow devices (exhaust, intake, head, valves, cams) for the same flow. Broad powerbands are created by optimizing each airflow device for different flow. The exhaust is commonly the most restricted (optimized for lowest flow/RPM) because that also makes the car quiet.
Heads, intakes, and cams I'm not completely sure about, but usually if you modify the exhaust first, you'll make the most gains.
I've found that the exhaust is by far the easiest, and cheapest part of a car to improve. I don't mean slapping a straight-through muffler with a 5" tip to an otherwise stock exhaust for that "fart" sound. Getting a header-back exhaust custom bent by a muffler shop can be really cheap. Crush bends and ugly mild-steel piping can be ugly and less than ideal, but it's really cheap compared to head work, intake porting, and other big gain mods.