Then I found detailed documentation on how the defraggers are implemented here. The site also provided a cheap defragger and source code. Unfortunately, it suffered the same basic design flaw as Diskeeper - if there wasn't enough free space to hold the whole file, you weren't going to get any better.
So, I got to work. Here is the result. My defragger employs the tried and true defragmenting algorithm of "evacuate and place". It first evacuates a large portion of contiguous free space, then starts placing files in the new empty hole. Repeat until defragged. However, in order to do this, it needs to repeatedly iterate over every file on your disk. This makes the overall process quite slow if you have many files, but the algorithm never fails to completely defrag your disk, no matter how full it is (well, you need some free space). My machine at work has a disk full of about 112,000 files and folders taking up about 15GB and it takes less than 8 hours to do a total disk defrag.
It works with any OS-supported file system* on any OS-supported writable media. It will only work on Windows XP and Windows Server 2003. This program will not work on any other line of Windows (2000, 9x, Win 3.1, etc). For those of you playing with Vista (which I presently am not), I'll be working on getting things working there. There is no 64-bit version at this time. I highly recommend against trying to run this on any 64-bit version of Windows.
* Can be run by any user (service needs to be installed by an administrator).
* Complete whole-disk defragging, including private operating system files.
* Multiple instances can be run simultaneously (if you have multiple disks).
* Individual file defragging, with wildcards and recursion. This mode is an opportunistic defragger, marginally better than Diskeeper in that it will at least defrag the file as best it can rather than not at all.
* Defrags NTFS, FAT** and FAT32** filesystems on any media (HD, Flash, ZipTM/JazTM, Floppy, etc).
* A "shotgun" mode, where it severely fragments files. Useful for testing or really annoying someone.
* Individual file fragmentation analysis. Find out where on disk a file resides, and how many fragments it's in.
* 100% safe and interruptable. This program doesn't actually move the files - the OS does it. Even if you suffer a power failure while it's working, you run no higher risk of file system corruption than you would if you had happened to be saving a file from any application.
* Boot-time defragmentation.
Copy defrag.exe and defragps.dll to a directory on your path which appears before your windows system directory. This isn't strictly necessary, but makes it easier to invoke from a command line.
Copy bootdfrg.exe to your windows\system32 directory.
Log in as a machine administrator (this is necessary to install the service).
From a command prompt, type: defrag -i
you should get back: "The Disk Defragmenter service was successfully installed."
The service is now installed. To later uninstall, type at a command prompt: net stop defragsvc & defrag -u
type defrag -? for general usage.
The defragger is disallowed from moving any part of the hiberfil.sys, pagefile.sys or memory.dmp files. Also, it is presently unable to move any of the NTFS metadata files (Master File Table, volume usage bitmap, etc). These leave "data islands" just like FAT and FAT32 leave in the case of directories. This means that *total* defragmention is not possible. You can defrag those three special files with the boot-time defragger. Until someone sends me information on how to access the NTFS metadata files, those remain untouchable and undefraggable.
To deregister any boot-time defrag job, simply run "defrag -B" without any more parameters.
Only last registered boot-time defrag job will be run.
The boot-time defragger is not interruptable at this time. You must wait for it to finish before the system will complete booting. Depending on what you've asked it to do, this could take a significant amount of time.
defrag -? to bring up the following list of commands:
-d drivespec [drivespec ...] defragment Drive(s).
-f filespec [filespec ...] defragment/relocate File(s).
-F filespec [filespec ...] defragment only fragmented File(s)
-v filespec [filespec ...] View statistics of file(s).
-b filespec [filespec ...] Blast file(s) all over the disk.
-l number diagnostic Logging level.
-s recurse into Subdirectories.
-B Execute at next system reboot.
-i Install defrag service.
-u Uninstall defrag service.
Precisely one of the b, d, f, F, or v switches may be present.
Logging levels range from 0 to 3.
Filespecs may include wildcards.
Multiple file/drive specs will execute sequentially. However, multiple instances may be started at once. Do not start multiple defrags on the same disk at the same time - it will defeat the purpose.
Whole-disk defragmentation takes a LONG time. Be patient. The process may be safely interrupted at any time.
The suggested command of defrag -B doesn't appear to get the program to defrag at boot time. The only way to do it is manually enter the data.
To do a boot time defrag you need to do the following:
There you will see the following key
In the value data box add:
bootdfrg -d C
Click OK and at the next reboot the boot time defrag will run.
If you want to defrag another drive alter C accordingly.
Alternatively add the following in notepad:
Windows Registry Editor Version 5.00
Save the file as boot.reg and then right click on the file and merge into the registry.
Note this will only defrag the C drive, if you want to do other drives you will need to manually change the drive via reg edit.
This process may take a long time to complete, please be patient.
So far not had any problems with it.