How to backup my data the right way

From time to time people ask me how to deal with cases of data loss. Usually I tell them about a tutorial that was written for Private Disk - the subtleties of backing up encrypted data. The problem is that data loss can happen to anyone, not only a Private Disk user, so I decided to write another guide, which is more generic, and is about data safety in general. It is true that there are more ways in which things can go wrong with encrypted data (because if you forgot the encryption key you actually lost all the data), there are still plenty of scenarios which can make someone unhappy even if they don't use encryption.

This is a story for those who are aware of the fact that we live in a world where Murphy's laws rule, those who understand the importance of backups, and are looking for a good backup strategy.

If you don't know why backups are important and why you need them, check out the reading material section in the end, or read about Murphy's laws when you have some time.

A good backup mechanism is

  • Automatic - copies must be made even if we are asleep, or tired, or not in the mood to do copies. A backup strategy that depends on our not forgetting to do an action is doomed to fail sooner or later, because forgetting is a feature we all have;
  • Reliable - a copy that was made must be the equivalent of the original; i.e. if things go wrong, we must be certain that the copy will be able to replace the original (not "this copy will do", but "this copy is as good as the original and I will not feel the difference after restoring");
  • Invisible - the copy must be made in the background so that the "files are being copied" window doesn't get in the way. If it does, we will feel tempted to close it because "ah, it will never happen to me anyway" or "just this one time I will close it". By having the copy being made in the background we exclude the possibility that a human will accidentally interrupt the process, or interfere with it.

Additional requirements

These are not mandatory, but having them is a bonus:

  • Keeping several old copies, so that you can revert to the state in which the files were at different points in time;
  • Limiting the file copy speed, so that the system stays responsive if the copy is made while you are using the computer;
  • Compressing the backup, so that less space is used on the storage device where the copy is kept;
  • Integrity checks - after the copy is made, it is compared with the original (using a checksum or hash), and you are warned if there is a difference between the two. Such checks will take more time, and most of us will do fine without them; but if you're dealing with mission-critical data, such a check can save lives or millions.

Prerequisites

  • Make a list of files and directories that you need to backup.


    If you have plenty of storage, you will probably want to backup the entire partition or hard disk, but this is not a good idea because:

    • it takes more time to do such a backup
    • if the data is on the system disk you will most likely have to reboot the system in a special mode (which is against the lazy nature of mankind)
    • the backup will contain a lot of data you do not really need (operating system modules, temporary files, programs that you can always download from the Internet, etc)
  • Decide where you will keep the backups, these rules must be taken into account:

    • it is better to keep the backup on another partition
    • even better: on another hard disk
    • even better: on another computer
    • even better: on another computer in another physical location (preferably on a different planet, to make sure that your data will survive the impact of Earth with an asteroid)
  • Estimate how fast your data grows and how much space you will need.


    This is very important, because if it is not done right you might be tempted to interrupt the backup processes in the future because you ran out of space.


    If your original hard disk is 60 GB in size, it is safe to assume the backups will never be larger than 60 GB in size (unless the backup storage device also contains backups from other computers). Take into account that you will most likely compress your backups too - so that will take even less space.

Practical scenario

Here is a sample environment:

  • Lappie - a laptop which contains the following data I care about

    • D:\Soft\Trillian - my instant messaging program, I want to keep my conversation history and the
    • D:\Soft\TheBat - my email client and the email archive for several accounts
    • D:\Stuff\TXTs - a directory with various texts (essays, poems); this is what I use instead of the standard My Documents folder (the rationale behind this decision is given in one of the discussions in the reading material section)
    • D:\Stuff\MyNotes - my OneNote repository (again, notice that it does not reside in My Documents)
    • D:\Soft\Palm\Alex - my Palm desktop profile
  • Servo - a desktop computer at home, which has a shared folder to which I can write, available as \\Servo\backups. In this folder there are sub-folders for each type of data I am backing up (MyNotes, TXTs, etc).
  • Hive - the server at work, which has a shared directory \\Hive\Alex\ to which only I have access; I will store my work-related data there.
  • The backup at home must be made once a week, every Sunday at 20:00; and the one at work is done every Monday at 09:30.
  • If the target computer is not turned on at the time, the backup will be made automatically next time there is an opportunity (ex: instead of waiting for the next Sunday 20:00, attempts will be made to copy the files every now and then, until the backup succeeds).

    The availability of \\Servo\backups can be used as a test: if the share is accessible, it means that both machines are on and the network is up; otherwise the backup is postponed.
  • I want to keep several old backups too, to make sure I can revert to them in case I want to take a look at the older versions of the files, see the image below.
Backup queue

Each time a backup is made, the old backups are pushed down, the fresh backup becomes #1, and the oldest backup is deleted. Note that even though the image says "new files", the backup will include the old ones too. Keeping three older copies is more than enough for the average user, but if you want to be able to travel back in time and see how your files looked like back in '45, you will obviously have to keep more than three previous copies.

The tools

To get the 3 features a good mechanism must have, only 2 tools are needed:

  • task scheduler
  • backup program

Windows comes with a task scheduling mechanism, you can access it via Control panel\Scheduled tasks. There are alternative programs which offer more features, but you will probably want to start with the standard task scheduler because you have it for free and it is already installed. There are reasons that can convince you to choose a different program for this purpose, they will be discussed later.

A backup program can be any program that does what you need. Many of us can be happy with XCopy or Robocopy (which also has a GUI), although this tutorial will focus on another tool.

The tandem is nnBackup and nnCron, both programs come from the same company, they are light and very flexible, providing a broad range of features. They are not free, though there is a special offer for ex-USSR folks, they can get it for free; therefore you might want to take a look at some of the programs described above, or look for alternatives elsewhere.

nnBackup is the program that does the actual copying. You can read about its many settings in the manual, be prepared to make notes on a paper, or in a temporary text file. Once you are done, you will end up with a set of command line arguments that do what you want, for example:

nnbackup.exe verz -n 2 -sdn "onenote" -i D:\Stuff\MyNotes -o \\Servo\backups\Onenote -s -e -sa -zip -v -pc
nnbackup.exe verz -n 2 -sdn "Documents" -i D:\Stuff\TXTs -o \\Servo\backups\Documents -s -e -sa -zip -v -pc

nnbackup.exe verz -n 2 -sdn "trillian" -i D:\Soft\Trillian -o \\Servo\backups\Trillian -s -e -sa -zip -v -pc

And so on... as you can see, all the lines are identical, the only part that varies is the one that concerns the path of the source (where files are copied from) and the target path (where the files will be copied).

For the curious minds, here is what the command line arguments mean in the examples above:

  • -i: input directory
  • -o: output directory
  • -verz: keep several versions of the backup, in compressed files
  • -n 2: two backups will be kept
  • -s: include the subdirectories too
  • -e: include empty directories too
  • -sa: copy the access rights (ex: you have NTFS access rights set for your directories, and you want them to be preserved on the target machine)
  • -pc: add a new backup only if differences were found between the current one and the old one
  • -v: verbose, this will show you which files are being copied - you might be interested in watching what is going on; + it usually impresses the non-tech savvy folk that happens to be around ;-)

In the same manner, I wrote the commands that will backup my other folders. Whenever I have a new type of data I want to backup, I can copy/paste an existing line and alter it accordingly. All these commands are saved in a BAT file, thus they will be executed one after another. All we need now is to launch this BAT file automatically on a weekly basis.

Watch out! some programs lock the files they use, so the files cannot be accessed by other processes (such as nnBackup, trying to make the copy). In this cases you have to make sure that the application is not running (ex: the mail client must be closed before the backup process is started, otherwise the mail archive cannot be read). To counter this, see if the program in question provides command line arguments (or any other mechanism) that allows you to close it correctly. Once you find out how to do that, perform that action before calling the backup script. If you don't know how to do that, then just close the programs by hand - but note that this goes against our philosophy - the backup must not require human intervention of any kind, because we can't trust humans...

nnCron comes into action now, this program will take care of running the backup script at the right time, re-run it if necessary, check if the network is active, etc. Creating a new task with nnCron is very easy, the screenshots below should be more than enough.

scheduled task actions
scheduled task settings

You can play with the other settings too, their names are self-explanatory. You will probably want to use the "host exists" feature, to verify whether the target backup machine is online; there are also various plugins that make it possible to use other conditions when evaluating whether a task has to run or not.

nnCron can keep track of multiple tasks; in this scenario, you will want two different scripts (one for backing stuff up on \\Servo, the other one for \\Hive), each script will have different settings for the time it should be run.

You don't necessarily need another computer for the backups, if you have an external disk, you can use it as the target path (i.e. instead of \\Servo\backups use F:\backups, replacing 'F' with the letter that corresponds to the external disk once it is mounted).

In the beginning you will probably not want to run the tasks in the background, because you want to see the progress of the transfer process, spot errors (if any). But after you do this a couple of times and you're sure everything works as you think it does, you can trust the system and let it work in the background.

Conclusions

The tips above are a set of general guidelines that are supposed to help you understand that good backups are a lot more than just copying and pasting files by hand in Windows Explorer.

A good backup mechanism must be thoroughly analyzed and tested before you can actually trust it. Do not let the apparent complexity dampen your spirits (I refer to finding the right command line arguments), once you get it figured out it is easy; but the most important part is that it is worth it. You will realize that when the first crisis comes and you get over it with no pain, trust me on that one.

Feel free to experiment with other similar tools (I will greatly appreciate it if you leave a comment and share your impressions), there are many of them out there.

Happy backups!


One liners

  • Do not backup that what can be easily replaced (ex: program installers can be downloaded from the Internet).
  • Another computer > another HDD > another partition > another directory.
  • Keep system data and user data separate.
  • Never send a human to do a machine's job.

Other reading material

Note: all the computer names were made up, coincidences with real world entities are just that - coincidences.

Share/Save/Bookmark
Tags: backup, tips

The anatomy of the restart=shutdown problem

It has been reported that in certain circumstances the system will shutdown instead of rebooting itself when the user restarts it while Private Disk is running and an encrypted disk is mounted.

This was a problem difficult to trace; while it repeats itself 10/10 times on a "problematic" machine, on "non-problematic" ones everything is working correctly and it is impossible to simulate the problem.

This is what makes it of reason to make an educated guess that this is caused by a third-party component present on the system, which somehow alters the standard behaviour of Windows. The tough part is that even when you think you have disabled all the non-standard programs, there is a myriad of low-level components that one can't see with the naked eye.

Full story »

Share/Save/Bookmark

How to move Voyager to a bigger USB disk

Voyager is a bundle that includes a portable email client, and Private Disk. The latter is used to encrypt the email archive, to make sure no one can read your messages if you lose the disk, or if it ends up being stolen. Ther are two flavours of Voyager, the 256 MB one and the 2 GB one; no matter which one you have, sooner or later you will need more space. That can be done easily by migrating your email archive to a bigger removable disk.
  1. It is assumed that both USB flash disks are ready; the original one is E:, and the new one is F:
  2. Start Private Disk
  3. Do not mount your image, instead go to Recovery and press Backup, to create a "compressed, encrypted, password protected backup copy of the Private Disk"
  4. Choose a backup file and an encryption password (this password can be, and should be different from the password of the original image; although nothing will break if you use the same password)
  5. Create a new encrypted disk, it should be located in F:\image.dpd
    • Naturally, the new image must be larger than the original one, so that there is plenty of room for new emails
    • The new image should not occupy the flash disk entirely, leave at least 10 MB of space for other data
  6. Go back to the Recovery tab and press Restore to "restore the data from a previously made copy of the Private Disk"
  7. Select the destination image (F:\image.dpd) and enter the password
  8. Choose the backup file made at step #4 and enter its password
  9. After the process is done, copy all the files and directories (except image.dpd) from E: to F:
If you did everything correctly, you will see the following in F:
  • PD, directory
  • autorun.inf
  • RunMe.exe
  • image.dpd
Done! You will probably want to go through an additional step, and customize these settings of the encrypted image:
  • Autorun, to launch Z:\Voyager\Voyager.exe automatically
  • Autofinish, to run Z:\Voyager\tbExit.exe automatically when the disk is dismounted
  • Disk Firewal - enable it and add the trusted applications (if any) to the white list. Remember that every program inside the virtual disk is trusted by default
Share/Save/Bookmark

Second public beta of Logon for Vista

A new beta of Logon for Vista is now available for download: http://files.dekart.com/beta/Logon-Vista-2230.msi It is not yet a final, and it does not provide all the features that are available in the non-Vista version. Here are some highlights that will be useful to you if you're planning to play with it:
  • You must logon using a key once, in order to make the Dekart credential provider the default one;
  • Press Other credential providers to choose key authentication;
  • This version does not provide support for biometric authentication as a third factor, but the feature will be available in the next release;
  • A restart after the installation is not needed;
  • The current version cannot use keys created by the older version, therefore you should make a backup of your sensitive data.
The GUI of the key administration tool is not yet final, there will be changes here and there. Share/Save/Bookmark

How to protect myself from identity theft

A report published recently by IC3 (Internet Crime Complaint Center), provides a lot of insightful tips to those who often engage in Internet commerce. The study was carried out in cooperation with the FBI, the National White Collar Crime Center, and the Bureau of Justice Assistance.

The study is extremely useful, as modern life is a path that will inevitably intersect with the Internet, whether we want it or not. The web helps us - consumers, do things faster and easier; the problem is that fraudsters get the same benefits. As a result, if you fall for an Internet scam, the damage can be of a greater magnitude, and it can be inflicted upon you so quickly that you won't even notice it happened. Here are some numbers that put things in perspective:

  • From January 1, 2007 to December 31, 2007, the IC3 website received 206,884 complaint submissions. This is a 0.3% decrease when compared to 2006 when 207,492 complaints were received.
  • The total dollar loss from all referred cases of fraud was $239.09 million with a median dollar loss of $680.00 per complaint.
  • This was an increase from $198.44 million in total reported losses in 2006.
  • Email (73.6%) and web pages (32.7%) were the two primary mechanisms by which the fraudulent contact took place.

Internet crime losses

It is highly recommended that you look through it and study the charts, and the recommendations section, which explains how to deal with such cases, as well as prevent them from happening. The remaining part of this article focuses on the issue of identity theft, which sadly was not given enough attention in the survey.

It is interesting that the study concludes that identity theft is one of the smaller troubles, as shown in the chart below.

Types of Internet fraud

Such a state of things is quite strange, because another study (the Computer Security Institute survey for 2007) found identity theft a much more serious problem. Could it be so that the victims of identity theft are not yet aware of their status?

Another possible explanation is that the scope of the IC3 report is simply different, it focuses on issues that occur after a transaction is complete (i.e. it is assumed that everything was ok before the final click in the process), while the truth is that identity theft has much more serious consequences. There is no need to use fake cheques, there is no need to engage in a long conversation with a "Nigerian scammer", nor there is a need to get involved in auction bidding. With your data in their pocket, a fraudster can do anything in a clean way - the sellers will not suspect that something is wrong, because from their point of view, they are dealing with an honest person, and everything is legal.

Identity theft occurs when someone else uses your personally identifying information without your knowledge or permission, to obtain credit cards, loans and mortgages, buy various products on your behalf, leaving you responsible for the consequences.

To minimize the risk of identity theft, you have to make sure that all the ways in which an identity can be stolen (attack vectors) are taken care of.

  • If you use a public computer for online banking transactions (ex: buy merchandise or purchase tickets for travel, concerts, or other services):
    • First of all, avoid using public computers, perform all the tasks that involve dealing with sensitive data on your home PC;
    • If you are forced to use a public computer, you can be the target of a keylogger, or the target of malware running on that workstation. There is no guarantee that the computer does not have any harmful programs installed there on purpose. You need a tool such as Password Carrier, which will automatically fill in the forms on the web-sites and in Windows programs - thus keyloggers won't capture your passwords and other personal information, because you don't have to type anything by hand.
  • If you store personal information in your home computer, there is a chance that it will be compromised (ex: if your antivirus or firewall failed), or that someone who uses the computer inadvertently ran an unknown (and malicious) program or an attachment that came with an email.
    • Make sure that all the sensitive files on the system are stored in encrypted form, so that they cannot be copied by someone who connects to the computer remotely. Use Private Disk to encrypt your files;
    • Use additional protection offered by Disk Firewall, to ensure that trusted but compromised programs won't allow an attacker to access private data;
    • As in the previous case, it is a good idea to use Password Carrier, because it takes a lot of expertise to thoroughly study a system and say "this system is 100% clean, no viruses, no spyware, no malware of any kind". If you are not one of those who can check their own computer and guarantee that it is clean, then Password Carrier will definitely help you.
  • Social engineering is another instrument an attacker can use to steal your identity. Why install various malicious tools and risk getting caught, when you can just go ahead and directly ask what you want? Due to the way our brains are wired, this approach is very often effective!
    • Be careful when someone asks you for personal information; it is good to be suspicious, so do not be afraid to question what they intend to do with this information;
    • Always double-check the information you are about to submit, sometimes a detail that seems unimportant can actually make a difference. If you don't know whether some data are sensitive or not, treat them as sensitive and do not disclose them;
    • Examine the privacy policy of the services that are used, in order to find out how they store and apply your data. In addition, if you are dealing with an intermediate party, they might request other, apparently not important data; if you know which details are needed by the service that does the actual processing, you will be able to find out whether the intermediate party requested data which they don't normally need;
    • Cautiously share your personal information with your friends and colleagues. You may trust them, but are you sure they won't accidentally (or even intentionally) share your details with other parties? Is your friend aware of the existing threats? If you are not sure, then you should think twice before handing out passport numbers, addresses, phone numbers, etc.

Conclusions

  • The Internet is a dangerous place, don't forget that.
  • It is a good thing to be a little bit paranoid, when not sure whether you really understand what is going on, take your time to ask someone in the know, or read the available documentation.
  • Software can assist you in protecting your privacy, programs such as Private Disk and Password Carrier will make your life safer, and easier.
  • Keep track of your expenses, to find out if you are already a victim of identity theft before it is too late.
Share/Save/Bookmark

Private Disk Light for Windows Vista

It is not known to the wide public, but the truth is that for quite some time a version of Private Disk Light for Windows Vista, as well as for 64-bit versions of Windows XP has been available.

It can be downloaded: http://files.dekart.com/beta/PrvDiskLight-Vista.exe

It is unofficially called Private Disk Light 1.23, and here are the changes:

  • Support for Windows Vista (32-bit and 64-bit)
  • Support for Windows XP 64-bit
  • Prettier icon
  • Surprise - a new feature that hasn't been present in the earlier versions of Private Disk; it is not likely to get noticed. But if you do notice it, you can get a free license for the full version of Private Disk, and I am absolutely serious about this one ;-) Good luck!
Share/Save/Bookmark

How to find out which files are open on my disk?

I am on Private Disk v 2.09. I close my files and every time I try to disconnect a drive letter I get the message:

There are files currently opened on disk Z:\

I feel that my data is not closing correctly. My question is how do I find out what file(s) are still open and how do I close them?

Quite often a volume can be used by a service, or another process that is running in the background - which makes it difficult to detect. In such cases, the best approach is to use a tool that monitors all the file activity that goes on in the system, and examine the list of processes that interfere with files located on the specific volume.

One such tool is Process Monitor; among many things, it can show which programs are working with data on a specific volume.

  • Start Process Monitor and make sure the File System Activity monitor is enabled
  • To simplify the task, create a new filter that will only show the activity that involves the volume you are interested in. The screenshot below illustrates how to add a rule that will only show which files are open on disk D:\
  • As you can see, in this example the non-system programs that are using files on disk D:\ are TheBat, Trillian.

proc-mon-add-filter

All you have to do is close these programs, and try to dismount the volume again.

If you see an unknown program accessing the volume, and you don't know how to close it (or you are not sure whether "killing the process" will have any serious consequences or not), look up the name of the program in a search engine and that will give you enough details to make a correct decision.

Share/Save/Bookmark

<< 1 2 3 4 5 6 7 8 9 10 11 12 >>