A heavily fragmented hard drive makes reading and writing data slower. Is is also often necessary to defragment in order to split a partition into two partitions without losing data (for example, with FIPS, Partition Magic, or ntfsresize).
Defragmentation can be impeded by the presense of immovable system files (or files that the defragmenter will not move in order to simplfy its task), especially a swap file.
Certain file systems are more susceptible to fragmentation than others, for example, a FAT file system becomes fragmented more quickly than NTFS or ext3.
Defragmentation programs often are included with an operating system (although Windows NT 4 notably did not include one) and usually it is recommended that they be run periodically to keep hard drive access as fast as possible.