External drive out of space

Have questions about Backup4all, or having problems using it? Ask here for help.
MikeD
Posts: 6
Joined: Tue Feb 01, 2011 4:31 pm

External drive out of space

Post by MikeD »

We are having a problem with our backups. Here is the set-up:

1. Daily backups are incremental and running on to 3 external USB drives which are switched daily, except Thursday.
2. Weekly backups are full and running onto 5 external drives which are being switched each Thursday according to a modified Tower of Hanoi schedule.
3. Local catalog is in use.
4. Temporary file is set to be on a local hard drive, not the external.

The plan was that the weekly backups would be able to get through 5 iterations of the 16 week hanoi schedule before we would have to deal with the drives filling. Unfortunately, as soon as you give your users more space they consume it and the backups have grown to almost double their original size. As a result, several of the weekly backups failed before the administrator noticed it. I got a backup run by adding a new external drive to get an immediate full backup to run then manually deleted backups to make space on the latest drive.

Wrong thing to do as I now realize! So I set the weekly full backup to only keep 8 versions as I knew this would ensure enough space on the external drive and would ensure we still had the backup versions we desired (i.e. 1 year). When last night's backup ran it tried to delete backup 32 as it was running backup 40. It failed (fortunately) as this drive did not have backup 32 on it. What I expected it to do was to delete the oldest backup on this drive as that was number 8 here. I also noticed that it seems to know about only the last 8 weekly backups now!

It seems to me that the way to get it to do version deletion the way I thought it would is to not use the local catalog, i.e. for each external drive to keep its own catalog. For this to work I would have to start using the Archive bit on the files so the Daily backups will only include changed files since any backup, rather than all files not in the catalog, as they will no longer know what has been backed up onto other drives. Is this correct? I am guessing that an issue with this would be that my current catalog would become invalid - what would be required to handle this in a useful manner?

Note that in actuality the proposal in the previous paragraph is not at all what I really want to do. I would prefer to keep using the local catalog but to have the backup delete the oldest files on the disk as required to make room for the new backup to be copied (the exact size will be known as I am using a temp folder on a local hard drive). Is there any way to do this? Or perhaps to set a size cap/percentage for a drive? (this would preferably apply to both the weekly and the daily backups)

Thanks
MikeD

Adrian (Softland)
Posts: 1914
Joined: Wed Dec 16, 2009 12:46 pm

Re: External drive out of space

Post by Adrian (Softland) »

hi,

If you use the local catalog, Backup4all won't know you changed the external hard drives.
When the limit is reached, it will try to delete the oldest backup from destination and catalog too.
If that backup is stored on another drive, it fails to delete it and it will removed only from catalog.
This is not recommended as you will have different backup versions in catalog and in destination.

If you want to switch the external drives, you must uncheck the "Use local catalogs" option.
Do you know you can monitor your backups remotely with Backup4all Monitor? You can read more here: https://www.backup4all.com/backup4all-monitor.html

MikeD
Posts: 6
Joined: Tue Feb 01, 2011 4:31 pm

Re: External drive out of space

Post by MikeD »

Thanks - that is what I suspected.

What about my last 2 questions though?

1. Since I have local catalogs already, how can I transition to catalogs on the external hard drives without losing information about what backups exist? I.e., is there a way to rebuild catalogs from the backups on a disk?

2. Is there a setting in the software to allow me to delete the oldest backups on a disk to make room for the new backup? This way I can keep the disk filled to capacity and not have to worry about the number of backups.

Thanks

Adrian (Softland)
Posts: 1914
Joined: Wed Dec 16, 2009 12:46 pm

Re: External drive out of space

Post by Adrian (Softland) »

Hi,

1. You cannot recreate the backup catalog from the existing files in destination.
2. To automatically delete the older backups, please go to Backup Properties->Type
There you can set the number of backup versions to be stored in destination.
Do you know you can monitor your backups remotely with Backup4all Monitor? You can read more here: https://www.backup4all.com/backup4all-monitor.html

MikeD
Posts: 6
Joined: Tue Feb 01, 2011 4:31 pm

Re: External drive out of space

Post by MikeD »

1. It is unfortunate that there is no way to scan a backup drive/disk and build a catalogue from that - this means a lot of manual intervention on my part to ensure that drives do not overfill until only catalogued backups are on it.

2. Yes, I understand this (see 1 for the issue). I wanted to know if there was a way to set a space limit (occupied or remaining, absolute or percentage) instead of a number limit (i.e. 8 regardless of how much space is remaining on the drive). The number limit requires that you know the size of each backup and allow for the maximum when setting the limit.

If neither of these options is available, you might want to consider adding them to a future release. Either way, I need confirmation on these questions ASAP as I have already had an issue because the weekly did not run again this week as the drive was too full and the person running the backup did not understand that they were responsible for monitoring the situation (not the software, as would be desired).

Thanks
MikeD

Adrian (Softland)
Posts: 1914
Joined: Wed Dec 16, 2009 12:46 pm

Re: External drive out of space

Post by Adrian (Softland) »

Hi,

We already added these features on our To-Do list but there is a lot of work to implement them in Backup4all, so they won't be available in the near future.
Do you know you can monitor your backups remotely with Backup4all Monitor? You can read more here: https://www.backup4all.com/backup4all-monitor.html

MikeD
Posts: 6
Joined: Tue Feb 01, 2011 4:31 pm

Re: External drive out of space

Post by MikeD »

Thanks for the update - too bad neither of these features is available already. :(

I guess I will have to work something else out in the meantime to ensure that the backups contiune to run smoothly - as they had for about 1 year.

MikeD
Posts: 6
Joined: Tue Feb 01, 2011 4:31 pm

Re: External drive out of space

Post by MikeD »

Well, over 1 year has passed since I implemented the solution below for the daily backups (archive bit reset, limit to 125 backups, do not merge) and it did not work as expected or desired!

When one of the daily drives hit backup # 126 it proceeded to delete backups 2 to 125 and create a new backup 126 that is 50GB in size. I am not sure what it contains but what I expected to happen was that backup # 2 would be deleted and anything that was only in that backup would be included on #126 (or just lost as it was now over 1 year old and I only expected the dailies to hold 1 year of backups).

This looks looks like a bug as it does not do what the help says will happen.

BTW, 2 of 3 daily backup drives have now done this and I would like to know how to get the behaviour I desire before the third one hits the 125 backup limit in 10 -20 days.

Thanks
MD

Adrian (Softland)
Posts: 1914
Joined: Wed Dec 16, 2009 12:46 pm

Re: External drive out of space

Post by Adrian (Softland) »

Depending of the backup type used and the configured settings, the versions will be preserved or not.

In Backup Properties->Type, there is the option to "Merge backups". That will merge the old deleted backup with the new created backup.
If you used the Incremental backup type, each increment depends on the previous one. You cannot delete an incremental and keep all its descendents as that won't be good for restore as the chain is broken. If an incremental is deleted, all its descendents will be deleted too.
Do you know you can monitor your backups remotely with Backup4all Monitor? You can read more here: https://www.backup4all.com/backup4all-monitor.html

MikeD
Posts: 6
Joined: Tue Feb 01, 2011 4:31 pm

Re: External drive out of space

Post by MikeD »

Yes, it is an incremental backup - forgot to include this information before. Maybe merge backups is what I want to do - if it will include any files that were only in backup #2 and not in any subsequent backup.

However, why did backups 2 to 125 ALL get deleted? That is not at all what I expected, which was for only the 125th oldest backup to be deleted and this would happen each day that external drive was used from now on.

Thanks
MD

Post Reply