Removing labels can be even more harmful than sticking them - on CD-R at least, delamination is a distinct possibility - the label takes part of the lacquer and silvering with it.
The problems with labels:
1. Impairment of balance, though with competent use of an applicator, this should not be too bad
2. Increased drag, a paper surface is not as smooth as the normal top surface, and the rotation speeds for 52x CD or 16x DVD are pretty fast
3. Adhesive attacking the top coating and silvering - with better top treatments, this should be less of a problem, but old silver media with single lacquering could deteriorate very quickly.
4. Peeling, the mosty serious problem, as this can jam slot loaders - Ideally, you should NEVER insert adhesive labelled media into a slot loader.
For archival use, it's better to mark only in the clear area around the hub, even when using a "safe" pen.
Best advice - never label your only copy!