Forum Home
Press F1
Thread ID: 150521	2022-03-10 01:01:00	Particular files that drastically slow down my backup to external hard drive	Misty (368)	Press F1

Post ID	Timestamp	Content	User
1484838	2022-03-10 01:01:00	I am currently doing a backup from "Pictures" on my desktop to one of my external hard drives using Windows 10 and USB3. Having over 32,000 photos it can take a while, which is okay. However, apart from my photos, in Pictures I also have my Adobe Lightroom 6 backup sets (not photos). These will be very numerous small files and it seems that they drastically slow down the process. You can see from my snip under that the speed is much slower. Is this normal and should I just grin and bear it? :illogical 11272	Misty (368)
1484839	2022-03-10 02:11:00	If the light room are compressed or zipped files ( backups) then yep, they would slow things down . The Actual Size is not that great, my own backups to a External USB drive, which is one that I do every so often is the Whole Drive at approx 100GB, takes about 30-45 minutes, where as over the LAN depending on whats happening can take a few hours . What I find often happens is if you try to copy basically the Whole folder in one go it WILL slow transfers down . Suggest you try selecting a batch of folders , Lets say 10 at a time ( no idea how many you have) and do them in smaller batches, - will go quicker . AND leave the Adobe Backup files alone and do then one at a time . Another possibility is what type of drive the Backup is and what condition its in, if its an older Mechanical drive could be a factor as well . ALSO what format is the Backup drive in ?? If its not NTFS then any files over 4GB will cause problems . Here's an article that has several options, that may speed transfers . . easeus . com/computer-instruction/speed-up-wd-external-hard-drive-slow-transfer-rate . html" target="_blank">www . easeus . com Usually I find backup files often slow things, or to much at once .	wainuitech (129)
1484840	2022-03-10 03:14:00	88KBs is a horrendously slow transfer rate. KB/s is too slow . Should be MB/s so many thing could be causing that. could be - failing PC HD . Ive seen lots of PC/laptops HDs that have a fault where the HD often runs extremely slow . - slow USB HD . USB3 means nothing. Its the actual drive speed that matters. Try a new USB HD . - lots of small fragmented files . Defrag the PC HD (this usually doesnt help anything though) - disable USB power save , disable HD sleep (for now) - try another cable, another USB port - format the USB HD as NTFS (mentioned above) - format the USB HD regardless , just to wipe it clean. - pause winupadates , Win updating can really slow thing down - disable AV & see if that helps If its a one off backup, just let it run through & finish , for now . Then run a HD speed test on the PC & USB drives . Compare the speeds to see if a HD is the bottleneck .	1101 (13337)
1484841	2022-03-10 04:24:00	Windows SMB/file copy always slows down when you are dealing with files which are small and numerous in volume... read/write speeds tank because the "process" invokes the reads and writes for each file... so when copying larger files (single thread for a copy) the reads, writes and thus transfer speed can be maintained/sustained at higher speeds for longer due to the file size, because the system isn't having to start, stop and write the file which is what you'd get with the small files. the start, stop, write functions are "interrupts" so that the system can process each file copy. This is common to desktop Windows OS editions, server OS platforms support multi-threaded file copying natively now.... I think since Server 2012. Something you could try is using the built in Robocopy program which comes included with Windows 10 - note this is a command line driven file copy program which support multi-threaded copying and mirror - it has some cool functionality but is mostly considered an "advanced tool". I think your use case may suit Robocopy as you are copying an entire folder on a periodic basis so you could setup a script/bat file which you can then schedule to run. Have a look at this article (social.technet.microsoft.com)	chiefnz (545)
1484842	2022-03-10 05:10:00	Another commonly used copy program which I used with W10 ( as Cheif pointed out is slow) is Teracopy. More stable, and faster than windows. www.codesector.com Just been transferring some "files" to a server in the workshop over the LAN - Rock steady at 11MB/s 11273	wainuitech (129)
1484843	2022-03-10 05:30:00	Another option is EaseUS File Sync (www.easeus.com) - which will allow you to sync your files from the computer to an external storage device. I am a big fan of their backup software but have not used their File Sync product... might pay to run a trial of File Sync and see if it suits your needs. The way I understand it your first "sync" will take the longest time and then subsequent syncs will only write changes made since the last sync to the external drive.	chiefnz (545)
1484844	2022-03-10 06:01:00	Just adding to my post above relating to Teracopy -- Just moving 35GB worth of Files, to an external WD Elements Drive, 3TB capacity (USB3), average transfer speed is steady at 59-60MB/s only took a few minutes. Cheifs post I am a big fan of their backup software but have not used their File Sync product. I have, used it for several years, as well as on a couple of customers sites, works Fine. AND correct, the 1st run takes the longest after that only transfers changes (or new files). It also has the ability to transfer / sync 3 ways. You can ( or used to) use it for free - The Free version after trial, allows only 1 folder sync, but as I found out you can have as many sub folders in that 1 folder. (not to sure if it still allows that or not). Another backup program ( YES I have tried and use a FEW) LOL :D that's also in a customers Server is Handybackup. https://www.handybackup.net/	wainuitech (129)
1484845	2022-03-11 02:30:00	My Media PC backs itself up monthly to my NAS using Aomei backupper, never seems to take long but the files on that pc don't change all that much. I haven't checked on it in a couple years though, I'm sure it was all working great when I set it up :) One comment about file sync, it's a great way to backup files on a regular basis but real time sync where you leave the backup drive connected and it all happens automatically is IMHO almost worthless. The reason is the same as why RAID is not considered a backup, you are protected from drive failure and nothing else. Any corruption, accidental deletion, virus infection, etc is immediately synced to the backup. I am not much for backing things up to be honest, very little I have can't be replaced or reinstalled and I'm too worried about doing that. Some photos and my music collection are duplicated in a few places and that's it.	dugimodo (138)
1484846	2022-03-11 21:17:00	The key phrase here is "numerous small files" When you have a ton of tiny (few kilobytes or less) files, the files copy so fast, that the filesystem has to update the MFT, FAT, Journal etc almost continuously, the overhead of which slows things down. Also on a mechanical drive, this means the heads needing to seek back and forth continuously from MFT to file area and back again, which also slows things down.	Agent_24 (57)
1484847	2022-03-11 21:35:00	Thanks guys, I am REALLY appreciative of all of the ideas, resources, etc that you have suggested. At the moment, some of it appears above my limited expertise, but I will work my way gradually through it. In the meantime, instead of choosing a group of all of the folders that needed backing up I backed up the Lightroom folder on its own. That did so at between 9.5 to 14 Mgs a second - hugely faster (see snip), so I have certainly taken that concept on board. 11274	Misty (368)
1 2