Overvolting anything (GPUs, CPUs and RAM) can increase your overclocking potential, so depending on how hard you wanna push your equipment, eventually you'll hit a point where adding a little more juice will get you there, or have it be more stable at higher speeds.
The trade-offs are, there's more heat disappated, so you'd want to make sure you AS5'd up your GPU before doing it... you might shorten the life of the equipment (not always true, depends on how hard you're pushing it), and in the case of notebooks, decreased battery life (but if you're over-volting you're probably not worrying about it anyway.
If you're going to overvolt, it's always good to do it in increments to 'burn it in'... 0.1v at a time for a few days to let the hardware adjust... this may mean more reliability later (again, only if you're pushin your hardware really hard).
Undervolting can decrease temps if you're really worried about it, but can cause the system to be unreliable at times. It's usually best to do the opposite of burning in... -0.1v at a time, and then testing with some benchmark programs to make sure your system is still stable.
Personally I don't see the point of undervolting... either I'm doin 2d stuff and my 9300 is cool and quiet anyway, or I'm doin 3d stuff and it's gonna warm up either way.