NetAdminTools.com
 
Categories:
GNU/Linux | Homebrew designs | Perl | Administration | Backup/Recovery | Bugs/Fixes | Certification | Database | Email | File/Print | Hardware | Information Grab Bag | Interoperability | GNU/Linux ABCs | Monitoring | Name Resolution | Network Services | Networking | Remote Control | Security | Desktop | Web | BSD | Solaris | GIAGD | REALbasic

Last 30 Days | Last 60 Days | Last 90 Days | All Articles | RSS | Hail Support


Categories:
·GNU/Linux
·Homebrew designs
·Perl
·Administration
·Backup/Recovery
·Bugs/Fixes
·Certification
·Database
·Email
·File/Print
·Hardware
·Information Grab Bag
·Interoperability
·GNU/Linux ABCs
·Monitoring
·Name Resolution
·Network Services
·Networking
·Remote Control
·Security
·Desktop
·Web
·BSD
·Solaris
·GIAGD
·REALbasic
·All Categories


D is for df, du, dd
Topic: GNU/Linux ABCs   Posted:2002-05-29
Printer Friendly: Print

spacerspacer
These three classic Unix utilities like to get their hands dirty with file systems.

****************************************************
df - display disk space usage on mounted filesystems
****************************************************

Every newbie sysadmin learns df right off the bat; it displays mounted file systems and the disk space usage on each. By default (in linux) it displays the statistics as measured in 1k blocks. For the less mathematically inclined, try the -h (human readable) option:

[usr-3@srv-3 usr-3]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1             1.9G  108M  1.6G   6% /
/dev/hda7              10G  2.4G  7.2G  25% /home
/dev/hda5              99M  435k   93M   1% /tmp
/dev/hda6             2.9G  1.8G  972M  66% /usr
/dev/hda3             243M   93M  138M  40% /var
none                  251M     0  251M   0% /dev/shm
eel:/share          169G   74G   87G  46% /share
You'll notice that both local and nfs filesystems are included. Sometimes you'll get wacky messages that a device is full but df shows plenty of space. If your filesystem holds many small files, chances are you're out of inodes. To check this use the -i option:
[usr-3@srv-3 usr-3]$ df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/hda1             251392   17692  233700    8% /
/dev/hda7            1359872   63108 1296764    5% /home
/dev/hda5              26104      50   26054    1% /tmp
/dev/hda6             384768  100644  284124   27% /usr
/dev/hda3              64512     629   63883    1% /var
none                   64304       1   64303    1% /dev/shm
eel:/share               0       0       0    -  /share
The number of inodes is set at filesystem creation time, and cannot be changed once the filesystem is made. So if you're creating a file system which you know will be holding many small files of 1-10K, use the options in mke2fs to increase the number of inodes beyond the default. Finally, use the -T option to include file system types in the listing:
[usr-3@srv-3 usr-3]$ df -T
Filesystem    Type   1k-blocks      Used Available Use% Mounted on
/dev/hda1     ext3     1976492    110504   1765584   6% /
/dev/hda7     ext3    10689180   2509896   7636292  25% /home
/dev/hda5     ext2      101089       435     95435   1% /tmp
/dev/hda6     ext3     3028080   1878800    995460  66% /usr
/dev/hda3     ext2      248895     94291    141752  40% /var
none         tmpfs      257216         0    257216   0% /dev/shm
eel:/share   nfs   177265344  77013672  91247124  46% /share
************************************************
du - estimate disk space usage by directory/file
************************************************

Now that you've determined your disk space usage, how will you figure out who or what is hogging up all that disk space? One way is to use du to summarize disk space usage by directory/file. For a quick summary of the space usage in a directory:

[root@srv-3 home]# du -h --max-depth=1
16k     ./lost+found
2.2G    ./usr-3
188M    ./eore
21M     ./web
2.4G    .
Again, -h makes it human readable, and the --max-depth option tells du
to summarize directories which are only n or fewer levels down from the
command line argument (in this case /home, our current working directory
because we did not specify a directory on the command line.)
If we forgo the -h option, we can pipe it through sort and find the 
piggiest files:
[root@srv-3 usr-3]# du | sort -g
the biggest directories show up at the bottom:
429812  ./code/blh/base/src
513876  ./music/zz
532640  ./code/blh/base
541740  ./music
612196  ./tools/general
781756  ./tools
823384  ./code/blh
862448  ./code
2264696 .
So let's investigate that pesky music directory -- that doesn't look legit for a user on one of *my* systems! Use the -a option to list counts for files as well as directories:
[root@srv-3 music]# du -a | sort -n
28      ./songlists
4240    ./shake-your-ass/Bodyrock.mp3
4552    ./shake-your-ass/11_Southern_girls.mp3
4924    ./shake-your-ass/Chemical Brothers - Block Rockin' Beat.mp3
5400    ./shake-your-ass/VIOLENTLY_HAPPY_Fluke(Well_Tempered).mp3
24300   ./zz/6girlinformme.wav
27828   ./shake-your-ass
28420   ./zz/95rottenworldblues.wav
34100   ./zz/91musiqueautomatic.wav
34276   ./zz/92ratormole.wav
34524   ./zz/3caringiscreepy.wav
513876  ./zz
541740  .
Uh huh. I could go have a talk with this user! Especially if I'm backing her music files to tape with my file server backup! Or perhaps I should just mount her home directory on my workstation.

******************************************************************
dd - copy and convert files
******************************************************************

The dd utility is used to copy and convert files block by block. It reads from standard in by default and writes to standard out.

[usr-3@srv-3 tmp]$ cat upper
I AM A BUNCH OF UPPER CASE CHARACTERS.
[usr-3@srv-3 tmp]$ dd conv=lcase < upper > lower
0+1 records in
0+1 records out
[usr-3@srv-3 tmp]$ cat lower 
i am a bunch of upper case characters.
One can also specify input files and output files at the command line:
[usr-3@srv-3 tmp]$ dd conv=lcase if=upper of=lower
0+1 records in
0+1 records out
There are several conversion options which can be discovered by reading the fine dd man page; to be honest I've never used dd for this much at all. I use it to make images of file systems. example: make a Red Hat boot floppy.
[root@srv-3 images]# dd if=bootnet.img of=/dev/fd0
2880+0 records in
2880+0 records out
[root@srv-3 images]# mount /dev/fd0 /mnt/floppy -t msdos
[root@srv-3 images]# cd /mnt/floppy/
[root@srv-3 floppy]# ls
boot.msg     initrd.img   param.msg   snake.msg     vmlinuz
general.msg  ldlinux.sys  rescue.msg  syslinux.cfg
And there you have it. A bootable Red Hat install floppy. Add a kickstart config file and you're on your way to having unattended automated installs. It's useful for larger file systems as well (this takes a few minutes and works the system):
[root@srv-3 home]# dd if=/dev/hda3 of=/home/tmp/var.img
514080+0 records in
514080+0 records out
[root@srv-3 tmp]# mount var.img var -o loop
[root@srv-3 tmp]# cd var
[root@srv-3 var]# ls
cache  gdm  local  log         mail  opt       run    tmp  yp
db     lib  lock   lost+found  nis   preserve  spool  www
[root@srv-3 tmp]# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda1              1976492    110508   1765580   6% /
/dev/hda7             10689180   2767220   7378968  28% /home
/dev/hda5               101089       435     95435   1% /tmp
/dev/hda6              3028080   1878800    995460  66% /usr
/dev/hda3               248895     94291    141752  40% /var
none                    257216         0    257216   0% /dev/shm
eel:/share         177265344  77013676  91247120  46% /share
/home/tmp/var.img       248895     94291    141752  40% /home/tmp/var
The -o loop option to mount allows us to mount files which contain file system images as filesystems. It is handy, let me tell you! This article has been brought to you by the letter D.




Please read our Terms of Use
Microsoft, Windows, Windows XP, Windows 2003, Windows 2000, and NT are either trademarks or registered trademarks of Microsoft Corporation. NetAdminTools.com is not affiliated with Microsoft Corporation. Linux is a registered trademark of Linus Torvalds, and refers to the Linux kernel. The operating system of most distributions that contain the Linux kernel is GNU/Linux. All logos and trademarks in this site are property of their respective owner. Copyright 1997-2008 NetAdminTools.com