Basically, you backup anything that you don't want to lose if your computer decides it doesn't want to start.
You cannot backup programs, but you can backup the data created with the programs. You can backup documents, pictures, emails, videos, etc. For example, I hadn't backed up my wedding photos before my hard drive crashed. The only reason that I'm still alive (my wife would've killed me!

is because I hadn't deleted them from the camera - so that was (accidentally) my backup.
As for how often, it depends on how current you want this to be. Systems may die without giving you notice - so the more often you backup, the less chance there is of losing the most recent additions to your files system.
There's a list of free applications for doing this located in this topic:
http://www.bleepingcomputer.com/forums/topic3616.html
What should you backup to? Well, that depends on a lot of different things, but the size of the backup is generally the deciding factor. If it's only a little bit, you can backup to a CD/DVD or a flash drive. Large backups may require an external hard drive to contain all the data.
Finally, the most important part of a backup is verification. It doesn't help to run a backup if you can't use it later on. While many programs contain verification routines, you just can't be sure until you actually try to recover the backup. So I suggest checking the backup after it's done to see if you can open those files. You'll get peace of mind knowing that your stuff is saved and accessible in the event of your system not booting.