AnimeSuki Forums

Register Forum Rules FAQ Members List Social Groups Search Today's Posts Mark Forums Read

Go Back   AnimeSuki Forum > Support > Tech Support

Notices

Reply
 
Thread Tools
Old 2007-08-11, 22:01   Link #1
hobbes_fan
You could say.....
 
 
Join Date: Apr 2007
Defrag: A myth?

Not so much a myth but rather having overblown benefits. I've seen a number of published articles recently (I'll check with mods if I scan them, might breach copyright) about defrag being such an awesome way to speed up a system, particularly with the FAT file systems but less effective with NTFS file systems. Basically, the articles say that the testing indicates that the pre and post defrag performance was basically the same in terms of boot up and app run times.

Now I defrag maybe all my drives (NTFS) once month or less. ( I only do it on upon installing apps or adding or deleting files of a significant size). But really I don't see any significant performance increase from defragging.

Is it blown out of proportion?
hobbes_fan is offline   Reply With Quote
Old 2007-08-11, 22:58   Link #2
jpwong
Senior Member
 
 
Join Date: Mar 2004
I think defragging has always been blown way out. But then I see my family's computer where windows defragmenter is complainning about the file system being 87% fragmented, and programs take over 5 seconds to open, I can see where these articles are comming from.

Basically, if you put it off for a really really long time, the preformance can degrade a lot, and you would see a large boost in preformance. But I don't see it as being the big preformance enhancer people claim it to be otherwise.
__________________
jpwong is offline   Reply With Quote
Old 2007-08-11, 23:22   Link #3
CandyVanMan
Part Time Hikikomori
 
Join Date: May 2007
Location: In the state of a deep trance
Age: 46
Defrag is a maintenance tool, not a performance booster. In a car, you won't gain 50 horsepower by changing the oil, but if you don't you'll lose more than horsepower with the damage done.

Also, consider a Bittorent file fragmented by default. It's best to consider moving the file to another drive or defragging on a weekly basis if you're constantly downloading torrents.
CandyVanMan is offline   Reply With Quote
Old 2007-08-11, 23:29   Link #4
Vexx
Obey the Darkly Cute ...
*Author
 
 
Join Date: Dec 2005
Location: On the whole, I'd rather be in Kyoto ...
Age: 66
aye, I'll frequently dump an entire folder of torrents to another drive as a "cheap defrag" before burning them to disk.

It probably does more good to defrag the system drive (C periodically. If you have one drive that has both the OS and a mess of data files - it won't hurt to do it more often (once a week or so).
__________________
Vexx is offline   Reply With Quote
Old 2007-08-11, 23:41   Link #5
WanderingKnight
Gregory House
*IT Support
 
 
Join Date: Jun 2006
Location: Buenos Aires, Argentina
Age: 35
Send a message via MSN to WanderingKnight
Go ext3 (and UNIX filesystems in general) and its almost nonexistent fragmentation
__________________


Place them in a box until a quieter time | Lights down, you up and die.
WanderingKnight is offline   Reply With Quote
Old 2007-08-12, 00:56   Link #6
grey_moon
Yummy, sweet and unyuu!!!
 
 
Join Date: Dec 2004
NTFS was billed as a FS that did not require defragging, hence NT not coming with a defragging tool. Unfortunately in practice fragmentation is really bad on it and has some bad performance overheads*. It's been a while since I've read up on it so numbers and references escapes me. But the main evidence is the defrag tool makes a return in later versions of the OS.

Now what is a myth and actually is bad is defragging a SD card (and other types of its family). They perform something called wear levelling which gives the media longer life and basically fragment on purpose. Defragging these beauties lower the life expectancy of them. Although on a million writes per cell it will take some hard core file writing to bugger up the media, unless you decided to swap to it *cough cough*.

*To do with use of FS and how it is configured (cluster sizes yada yada)

*Edit*
To actually answer hobbes_fan, I would say that it depends on what you use the disk for. One of fragmentations biggest overheads is seek time, so ofc the drive itself makes a difference, but also what the drive is being used for. Let me use my old gaming pc as an example....

Personally I have a drive which has 2 partitions on it, one for windows only and one for applications. Now if I install something big sometimes it can become slightly fragmented, but this accumulates over time (service packs are a bad one for the windows partition). I normally don't need to fragment these drives, but if they get fragmented it should affect the overall performance of my system and a benchmark test would show it. But the thing is I don't normally notice it unless I do something stupid and my partitions start to fill up. I actually keep my swap file on a fast but small disk, I use that disk as the systems temporary file drive and for me copying stuff to. Now if this does get fragmented and it does I do notice the effect on my system. Finally my data drive is fragmented to buggery and normally I don't notice it as I don't really use any of my data for time dependant stuff, the only time that it did make a difference was when I was running VMware from it.
__________________

Last edited by grey_moon; 2007-08-12 at 01:17.
grey_moon is offline   Reply With Quote
Old 2007-08-12, 02:12   Link #7
Tiberium Wolf
Senior Member
 
 
Join Date: Dec 2004
Location: Portugal
Age: 44
Quote:
Originally Posted by CandyVanMan View Post
Also, consider a Bittorent file fragmented by default. It's best to consider moving the file to another drive or defragging on a weekly basis if you're constantly downloading torrents.
Quote:
Originally Posted by Vexx View Post
aye, I'll frequently dump an entire folder of torrents to another drive as a "cheap defrag" before burning them to disk.

It probably does more good to defrag the system drive (C periodically. If you have one drive that has both the OS and a mess of data files - it won't hurt to do it more often (once a week or so).
It won't be fragmented much if the client allocates space for the entire file instead of doing small parts at a time.


Quote:
Originally Posted by grey_moon View Post
NTFS was billed as a FS that did not require defragging, hence NT not coming with a defragging tool. Unfortunately in practice fragmentation is really bad on it and has some bad performance overheads*. It's been a while since I've read up on it so numbers and references escapes me. But the main evidence is the defrag tool makes a return in later versions of the OS.
You should see how windows xp with ntfs can fragment a file that you have copied to HDD that had 90% of free space. All that free space and he still had to break up the file.


Quote:
Originally Posted by grey_moon View Post
Personally I have a drive which has 2 partitions on it, one for windows only and one for applications. Now if I install something big sometimes it can become slightly fragmented, but this accumulates over time (service packs are a bad one for the windows partition). I normally don't need to fragment these drives, but if they get fragmented it should affect the overall performance of my system and a benchmark test would show it. But the thing is I don't normally notice it unless I do something stupid and my partitions start to fill up. I actually keep my swap file on a fast but small disk, I use that disk as the systems temporary file drive and for me copying stuff to. Now if this does get fragmented and it does I do notice the effect on my system. Finally my data drive is fragmented to buggery and normally I don't notice it as I don't really use any of my data for time dependant stuff, the only time that it did make a difference was when I was running VMware from it.
Don't use swap file.




You should always do defrag if the HDD, better partition, you use is having constant modification. But if it's the DL drive then don't bother. For example if you download from the mIRC(or other irc client) several files at the same time you end up with files fragmented into thousand of pieces. In this case don't waste time defraging. If you wanted to get rid of the framentation you just move everything out to another HDD and then move back. Believe me it takes less time. Anyway in these files you will notice that it takes time to copy it.

If a file data is all contiguous you minimize the time is takes to read. If it split up in several fragments the HDD read here and had to jump(waste of time) to another place, read, jump, read, jump... you get the point, don't you?

Anyway. It's nice to go to the windows defrag utility and see all blue bars in the HDD image.
__________________
Tiberium Wolf is offline   Reply With Quote
Old 2007-08-12, 02:55   Link #8
hobbes_fan
You could say.....
 
 
Join Date: Apr 2007
See I use 2 drives on my gaming pc: a 74gig WD Raptor for game installs and o/s and your ordinary run of the mill 160gig seagate barracuda for other apps/files. the raptor is blisteringly fast, it's designed to be. But cleaning my registry has a more tangible effect than defrag particularly in boot sequences.

I guess I keep hearing defrag your system it will be super quick, when in reality it won't help as much as its made out to. It's used too often as a cure all to improve efficiency, but yeah, I probably won't be defragging anymore than once a year it seems to provide marginal if any positive outcomes. Everything so far said in this thread pretty much indicates it isn't what it's made out to be and supports what I've read so far.
hobbes_fan is offline   Reply With Quote
Old 2007-08-12, 02:57   Link #9
Vexx
Obey the Darkly Cute ...
*Author
 
 
Join Date: Dec 2005
Location: On the whole, I'd rather be in Kyoto ...
Age: 66
Even when a client allocates space for an entire file, it has to depend on the OS for where and in how many pieces. A badly fragmented drive just reeks of entropy if there's a lot of file addition and deletion activity.

One thing I've found amusing is that in win9X, win2K, winXP - whether commercial, or MSDN, or OEM .... I've found that any iniitial install invariably needs a defrag right off the bat. The damned installer OS CD splatters system file chunks all over the disk.

My working machine (as opposed to my gaming machine) has several disks. I try to keep "things Windows" on the boot drive and as little else as possible. A second disk holds most of the applications, a third for data (like torrent files), and a fourth just because, um, its just nice to have an extra place for cleaning and emergencies. I also try to make sure each disk keeps about 20% spare.

If you're really careful, you can probably get away with defragging a few times a year... as always YMMV. Of course, if you run it as a background automatically scheduled process none of this matters.

Welll.... "super quick" is only true if you're Joe Sixpack "whats defrag" --- when I maintenance the occasional neighbor, a defrag usually has a substantial effect (especially after I delete all the crap AOL multiple installations and open up some space on the drive so it can even BE defragged). Cleaning the registry is certainly very good.... I *hate* looking at the registry and seeing what crap gets put in there.... actually I hate the registry concept except for OS purposes. Apps should keep everything in their folder.
__________________
Vexx is offline   Reply With Quote
Old 2007-08-12, 04:12   Link #10
grey_moon
Yummy, sweet and unyuu!!!
 
 
Join Date: Dec 2004
@hobbes_fan - Fast disks with big caches will minimise the noticeable effects of fragmentation, but the best way to make sure is to do a benchmark on them.

I've noticed too if I restore a disk from backup that it normally ends up fragmented (just straight data copy not imaging) *shudder*
__________________
grey_moon is offline   Reply With Quote
Old 2007-08-12, 06:26   Link #11
toru310
Senior Member
 
Join Date: Apr 2006
Location: Philippines
How long will it take to defrag a harddrive 5months old? with every day torrent downloading and deleting..? This is a seagate 40gb NTFS? this is also a system hard drive?
Thanks in advance.
toru310 is offline   Reply With Quote
Old 2007-08-12, 06:51   Link #12
hobbes_fan
You could say.....
 
 
Join Date: Apr 2007
depends on your processor and other components as it is essentially a read/write process. My best estimate? Maybe 1.5 hrs max, for a full 40gig.
hobbes_fan is offline   Reply With Quote
Old 2007-08-12, 07:03   Link #13
toru310
Senior Member
 
Join Date: Apr 2006
Location: Philippines
I see well before defraging I will be transferring 15gb files to the storage drive so my hard drive will have 20gb of space I think..will the time estimated decrease?
toru310 is offline   Reply With Quote
Old 2007-08-12, 07:20   Link #14
TakutoKun
Mew Member
*IT Support
 
Join Date: Aug 2007
Location: Ontario, Canada
Age: 39
Quote:
Originally Posted by Migufuchi Fusutsu View Post
I see well before defraging I will be transferring 15gb files to the storage drive so my hard drive will have 20gb of space I think..will the time estimated decrease?
If you are transferring away from the drive, then yes, the time to defrag will decrease.
--

I will also address the reasons for defragmenting and why fragmenting occurs. As Grey_Moon mentioned, as you add/remove large files, your drive will quickly become fragemented. Fragmentation occurs with the consistent add/remove of information accross a hard disks platter(s). As the read/write heads write across the megnetic platter, information, over time, becomes scattered across the platter. So, if you are removing or adding files (large ones), your disk's performance will drop exponentially. In fact, as most of you know, when you delete information off of a FAT or NTFS disk, the information remains there until written over or your hard disk is defragmented. This leads into performance issues compared between FAT and NTFS.

FAT (File Allocation Table) had its place in time. FAT was meant to be used with DOS compatible systems in order to utilize hard disks up until roughly 32 GB with the implementation of FAT32. With the creation of NT based Microsoft Operating Systems, the use of NTFS came into play. Of course, with NT 4.0, it was not unitl SP 6a where NT 4.0 could even address NTFS. At any rate, FAT does not address fragmentation very well. As you write to a FAT disk, over time it does not write very sequentially. Instead, however, the information on a FAT disk becomes scattered across the platter. This causes you to defrag often with the older Operating Systems and, usually, it took a lot longer to defrag. With NTFS, it usually writes information sequentially often decreasing the amount of time it takes to defrag a volume. Here is a great WikiPedia article involving "Defragmentation" and why fragmentation occurs across a hard disk - http://en.wikipedia.org/wiki/Defragmentation . Here is a quick overview of the performance/options you get with NTFS and FAT - http://www.ntfs.com/ntfs_vs_fat.htm .
TakutoKun is offline   Reply With Quote
Old 2007-08-12, 10:08   Link #15
Ledgem
Love Yourself
 
 
Join Date: Mar 2003
Location: Northeast USA
Age: 38
Quote:
Originally Posted by hobbes_fan View Post
See I use 2 drives on my gaming pc: a 74gig WD Raptor for game installs and o/s and your ordinary run of the mill 160gig seagate barracuda for other apps/files. the raptor is blisteringly fast, it's designed to be. But cleaning my registry has a more tangible effect than defrag particularly in boot sequences.
Not surprising. As Greymoon mentioned, fast drives with a large cache will cut the performance loss from fragmentation. For those who don't know, the Raptor is a 10,000 RPM hard drive. Standard hard drives are 7200 RPM; before that, they were 5400 RPM (which seems to be the new standard in laptops now). Laptops used to be in the 4000's. The difference is massively noticable, and if you're using a very slow drive, fragmentation only makes it worse. It means that your drive has to spend more time searching for the fragments. With an extremely fast drive, this added time is barely anything. With a very slow drive, you'll notice it, especially if you're trying to multitask. Case in point, on my old laptop, during heavy multitasking I used to get a performance boost with my 4000 RPM HD by stopping any music I was listening to.

Quote:
Originally Posted by Migufuchi Fusutsu
How long will it take to defrag a harddrive 5months old? with every day torrent downloading and deleting..? This is a seagate 40gb NTFS? this is also a system hard drive?
Thanks in advance.
hobbes_fan said that it depends on your processor, but I'm going to challenge that. Unless you're using an extremely old processor, I've never seen defragging to be a processor-intensive process. Logically, it isn't either, because it's just a matter of the drive shuffling data around. In that case, it depends on how fast your hard drive is (seek speeds/read speeds/write speeds, which are impacted by the RPM and cache size). The less fragmented data that there is to shuffle around, the faster the process will be.

On the subject of defragging, the new solid state hard drives will not need to be defragged - supposedly they experience no performance loss during fragmentation. It makes sense, since there's no physical seeking going on.
__________________
Ledgem is offline   Reply With Quote
Old 2007-08-12, 10:31   Link #16
hobbes_fan
You could say.....
 
 
Join Date: Apr 2007
fair enough, I did make a massive jump from a low end celeron to x2's. (i reused the 250 gig from the celeron system in the new system), and personal observations were something like 4hrs (celly) to 1.5(x2 3800) for a 150 - 200 gig defrag.

Hybrid drives seem to be the next logical step, and right now they seem ok value in comparison to pure SS drives. magnetic is around .30c-50c/ gig, while the only samsung SS HDD I've seen is hovering around $7.00-$10.00/gig. But as an added bonus they're silent and have minimal wattage requirements. But in time they'll be a better buy.
hobbes_fan is offline   Reply With Quote
Old 2007-08-12, 11:41   Link #17
Jinto
Asuki-tan Kairin ↓
 
 
Join Date: Feb 2004
Location: Fürth (GER)
Age: 43
Quote:
Originally Posted by CandyVanMan View Post
Defrag is a maintenance tool, not a performance booster. In a car, you won't gain 50 horsepower by changing the oil, but if you don't you'll lose more than horsepower with the damage done.

Also, consider a Bittorent file fragmented by default. It's best to consider moving the file to another drive or defragging on a weekly basis if you're constantly downloading torrents.
I agree with almost everything except the fragmentation issue of torrent files. This depends on whether you use preallocation or not.

Quote:
Originally Posted by Vexx View Post
aye, I'll frequently dump an entire folder of torrents to another drive as a "cheap defrag" before burning them to disk.

It probably does more good to defrag the system drive (C periodically. If you have one drive that has both the OS and a mess of data files - it won't hurt to do it more often (once a week or so).
Once a week sounds like out of proportion imo. At least with NTFS. I have not defragmented my system drive for years and currently it has a fragmentation of overall 25% with filefragmentation of 50%. Well thats 2x70ms or so more access time for every 4th file and 70ms more access time for every 2nd file -statistically (I think I can live with that )

Quote:
Originally Posted by WanderingKnight View Post
Go ext3 (and UNIX filesystems in general) and its almost nonexistent fragmentation
Works only then well, if you have lots of free space available.

Quote:
Originally Posted by grey_moon View Post
NTFS was billed as a FS that did not require defragging, hence NT not coming with a defragging tool. Unfortunately in practice fragmentation is really bad on it and has some bad performance overheads*. It's been a while since I've read up on it so numbers and references escapes me. But the main evidence is the defrag tool makes a return in later versions of the OS.
Strange can't one access the defragmenter by default with the drive properties?

Quote:
Originally Posted by Tiberium Wolf View Post
You should see how windows xp with ntfs can fragment a file that you have copied to HDD that had 90% of free space. All that free space and he still had to break up the file.
Which is due to NTFS file systems write from the partition borders to the partition center. Especially large files will likely get splitted in such an write event.


What is most important to have defragmented is the MFT (Master File Table) because each file access will also trigger at least one access to the MFT.
__________________
Folding@Home, Team Animesuki
Jinto is offline   Reply With Quote
Old 2007-08-12, 12:34   Link #18
TakutoKun
Mew Member
*IT Support
 
Join Date: Aug 2007
Location: Ontario, Canada
Age: 39
Quote:
Originally Posted by Ledgem View Post
Not surprising. As Greymoon mentioned, fast drives with a large cache will cut the performance loss from fragmentation. For those who don't know, the Raptor is a 10,000 RPM hard drive. Standard hard drives are 7200 RPM; before that, they were 5400 RPM (which seems to be the new standard in laptops now). Laptops used to be in the 4000's. The difference is massively noticable, and if you're using a very slow drive, fragmentation only makes it worse. It means that your drive has to spend more time searching for the fragments. With an extremely fast drive, this added time is barely anything. With a very slow drive, you'll notice it, especially if you're trying to multitask. Case in point, on my old laptop, during heavy multitasking I used to get a performance boost with my 4000 RPM HD by stopping any music I was listening to.
...
On the subject of defragging, the new solid state hard drives will not need to be defragged - supposedly they experience no performance loss during fragmentation. It makes sense, since there's no physical seeking going on.
The speed of a hard disk also depends on the technology it uses. I mean, a hard disk is great even if it is running between 5400 - 10 000 RPM. However, if you are using PATA we are talking a serious performance dip issue her (100 - ~ 133 MBs). The speed of the disk, however, does not lead directly to fragmention. It does, however, cause data to be written erratically when you are trying to access multiple parts of the HDD platter. However, the speed of SATA 1/2/3 (1.5 G/s, 3.0 G/s, 8 G/s) has corrected the speed/bus issue. However, we are still seeing fragmentation.

Solid-state drives, as mentioned, are being moved towards as an alternative. The speed, reduced power levels, and so fourth are major advantages of the SS drive. I have seen some pretty big SS drives in development. This will go well with the fiberoptic motherboard that is in production. Since fiber is is not effected by EMI and crosstalk, we can see some major improvements over speed/bus issues very soon.

Quote:
Originally Posted by Jinto Lin View Post
I agree with almost everything except the fragmentation issue of torrent files. This depends on whether you use preallocation or not.
Fragmentation is generally occurs when a file is written to multiple sectors are written to. The more the information spans across the sectors and even into clusters, the more chance the file will be fragemented.

Quote:
Originally Posted by Jinto Lin View Post
Works only then well, if you have lots of free space available.
Journalizing filesystems are great to work with. You do lose performance because of the journalizing, but fault tolerance is much better in case of a loss. Free space is a concern with a Journalizing FS. I believe WinFS is supposed to be used in Windows Vienna (2009).
TakutoKun is offline   Reply With Quote
Old 2007-08-12, 12:36   Link #19
grey_moon
Yummy, sweet and unyuu!!!
 
 
Join Date: Dec 2004
Quote:
Originally Posted by Jinto Lin View Post
Strange can't one access the defragmenter by default with the drive properties?
"The first defragmentation support for Windows NT was introduced by Executive Software. Their Diskeeper product was initially released for Windows NT 3.51, and because the NTFS and FAT file systems for NT 3.51 have no native functions that provide for cluster movement, Executive Software was forced to purchase a source license for NT and to create and ship custom versions of NTFS and FAT, as well as NT itself, along with their defragmentation code."

http://www.microsoft.com/technet/sys...agmenting.mspx

Bit of history hee hee, I still remember when we first automatically rolled it out on over 50 machines using drive image and a sidchanger script. We were so proud of ourselves
__________________
grey_moon is offline   Reply With Quote
Old 2007-08-12, 12:49   Link #20
jpwong
Senior Member
 
 
Join Date: Mar 2004
Quote:
Originally Posted by grey_moon View Post
NTFS was billed as a FS that did not require defragging, hence NT not coming with a defragging tool. Unfortunately in practice fragmentation is really bad on it and has some bad performance overheads*. It's been a while since I've read up on it so numbers and references escapes me. But the main evidence is the defrag tool makes a return in later versions of the OS.
Hehe, one of the OS teachers at my school was discussing that one day. When it turned out that the disk did fragment, MS's response at the time was "Backup your files to tape drive, wipe the hard disk and then restore the drive from your tape backup"
__________________
jpwong is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 00:00.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
We use Silk.