UnixNotes - Tech - The Caboteria

   * DebianTips
   * MandrakeTips
   * FedoraNotes
   * UnixPerformanceTips
   * PostfixNotes
   * PostfixComcastAol
   * PluggableAuthenticationModules


---
 At some point the nightly man-db update process started going into an endless loop using 100% CPU. I'm not sure what the problem was, but blowing away the database ( =/var/cache/man=) fixed it. 
---

I don't know why, but the mouse (trackpoint) on the IBM A30 stopped working when I upgrade from kernel 2.4 to 2.6. I had to explicitly (i.e. =/etc/modules=) load =mousedev= and =psmouse= and then change gpm's mouse type from =autops2= to =ps2= and then things worked OK.
---

I had a problem once trying to ssh from work to my machine at home. Everything worked fine until I left the connection idle for a while. When I hit a key I got

<pre>Read from remote host www.caboteria.org: Connection reset by peer Connection to www.caboteria.org closed. </pre>

Setting ssh's =KeepAlive= parameter on the client and server didn't seem to do much good. I found a patch that helped, though:

http://www.sc.isc.tohoku.ac.jp/~hgot/sources/openssh-watchdog.html

The "contributed" patch at the bottom is close to the version that ships with Debian Potato, and although it gives errors when you try to apply it it's pretty small and relatively easy to cut-n-paste by hand.

---
 This appeared in the RISKS digest:

http://catless.ncl.ac.uk/Risks/21.79.html#subj3 - Risks of the space character in Unix filenames

---

Info on diskless workstations:

http://www.ltsp.org/ - the linux terminal server project, folks who have a Linux distro that turns an old PC into a diskless X-terminal. It worked very well for me on an old Pentium 133.

http://lists.debian.org/debian-devel/2001/debian-devel-200104/msg00647.html I wasn't able to get reliable performance using the user-space nfs server (many "stale NFS handle" errors in apt-get, probably caused by moving files and trying to use the same handle) but it seems to be reliable using the kernel nfs server.

If you're using NFS you should sweep your filesystems periodically looking for old files named =.nfs*=. These get left behind when a process has a file open, another process deletes it, and the first process crashes. See http://www.sunmanagers.org/archives/1998/0229.html

http://www.jukie.net/~bart/blog/20070316092236 - a good overview of the PXE boot process, and howto set up Debian machines to boot diskless.<br/>
http://www.linuxquestions.org/questions/debian-26/how-i-did-it-diskless-netboot-with-debian-etch-468870 - how to make a net-bootable initrd

If you're serving more than a couple of workstations then the default NFS parameters will probably cause bottlenecks. The NFS howto at http://nfs.sourceforge.net/nfs-howto/ has a page about performance tuning. The number of instances of the server daemon is key.

I used to have to build my own kernels because the stock ones wouldn't work on diskless nodes, but as of Debian Lenny I no longer have to - the stock Debian kernels work for diskless.  When a new kernel comes out I just need to copy the modules directory into the diskless node's tree, copy the kernel to the tftpserver root, make the initrd and copy that to the tftpserver root, and edit the pxelinux config file.

---

*chroot*

Two HOWTO's that I found useful for setting up =chroot= jails are: http://www.tjw.org/chroot-login-HOWTO/, http://kegel.com/crosstool/current/doc/chroot-login-howto.html

---
 *grub*

site - http://www.gnu.org/software/grub/

manual - http://www.gnu.org/software/grub/manual/

burn a CD - http://www.gnu.org/software/grub/manual/html_node/Making-a-GRUB-bootable-CD_002dROM.html#Making-a-GRUB-bootable-CD_002dROM

a good way to scan the partitions is =find /grub/stage1= or =find /boot/grub/stage1= since that's about the shortest filename you can reliably look for in a grub installation.

The last time I messed with grub it was to get a USB drive to boot Linux. I had a lot of problems, from grub streaming "GRUB GRUB GRUB GRUB" messages infinitely to it just printing "GRUB" and then stopping. In my case it looks like the problem was that the disk order was different depending on whether the USB drive was being booted from or whether some other drive was. The fix was to *not* use the "d" flag on the grub [[http://www.gnu.org/software/grub/manual/html_node/install.html#install][install command]].

When I installed Fedora 8 on a USB drive I had to boot from the grub cdrom and then =setup (hd1,0) (hd1,0)=.

if you're like me you'll probably blow away the windows MBR in the process, so here's how to fix it: http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/bootcons_fixmbr.mspx

---
 *backup*

rdiff-backup - http://www.nongnu.org/rdiff-backup/docs.html

Python program that uses rdiff to generate efficient differential backups. Straightforward command line. Seems to use a lot of CPU.

rsnapshot - http://www.rsnapshot.org/

Similar idea, written in perl. Seems to be more config-file oriented less command-line oriented.

http://www.mikerubel.org/computers/rsync_snapshots/<br /> http://blog.interlinked.org/tutorials/rsync_time_machine.html

---
 *Hot-plug devices*

When I was logged into =warthog= and plugged Tory's ipod into its cradle, it would get mounted automagically. When Tory tried this it didn't work. It seems that the set of programs to manage things like ipods is pretty complicated, but there's some useful info at http://www.freedesktop.org/wiki/Software/HalFAQ and especially http://www.mythic-beasts.com/~mark/random/hal/.

In our case it turned out that Tory wasn't a member of the =plugdev= group.

---
 *Hard Disk Spindown*

I got a usb-attached disk drive to make backups to, and it spins the drive down after a while to save power. That's cool, but the problem is that Linux barfs when it tries to access the drive:

<verbatim>
Apr  9 06:19:01 voom kernel: sd 0:0:0:0: Device not ready: <6>: Current: sense key: Not Ready
Apr  9 06:19:01 voom kernel:     Additional sense: Logical unit not ready, initializing command required
Apr  9 06:19:01 voom kernel: end_request: I/O error, dev sda, sector 12375
</verbatim>

This might help: http://www.nslu2-linux.org/wiki/FAQ/DealWithAutoSpinDownOnSeagateFreeAgent

After living with this for a while and not figuring it out I upgraded to Debian Lenny which fixed the problem.  Now when you access the drive you can hear it spinning up, and whatever program caused the access waits until the drive it ready.

---
*Mounting USB Devices*

HAL is responsible for mounting USB drives when they're plugged in.  It will use the partition's label as a mount point if it can or it will fall back to something like "disk".  http://www.debuntu.org/device-partition-labeling talks about how to set the labels on ext2 filesystems.  For FAT filesystems use the mlabel command.

_At Boot Time_

Here's an article that talks about how to assign labels to partitions to make booting from USB devices more flexible:
http://www.ibm.com/developerworks/linux/library/l-boot-rootfs/

---
*Network Interfaces Change Names*

I had to swap a motherboard in my little home server when the old one went south, and when I booted the new one I got a message about =udev: renamed network interface eth0 to eth1= which seemed kind of peculiar.  It turns out that udev tries to make sure that a given network interface (identified by its MAC address) will always have the same name, even if you change the machine's hardware (e.g. add another network card).  It uses a file called =/etc/udev/rules.d/70-persistent-net.rules= for this purpose.  Since it already had some entries for eth0 and it hadn't seen the new motherboard's MAC before, it gave it the name "eth1".  This is reasonable behavior once you figure it out.  A quick edit to that file to change the name to "eth0" and I was all set.

---
*SSH and Rsync are both awesome!*

open a connection from inside a firewall allowing tunnelled logins back through: =ssh -Nf -R 22222:localhost:22 login@hostname=

tunnel back up that tunnel and use a USB disk sneakernet to sync a big directory:
<pre>rsync -rtPv --rsh="ssh -p 22222" --only-write-batch=/usb-drive /source-dir/* login@target-host:/target-dir
</pre>
that command writes both a big blob (the "batch") and a shell script that will apply the batch on the other side.

This runs rsync as root on the local and remote host but uses a non-root login on the remote host which is great for systems like Ubuntu that don't allow root logins.
<pre>
$ sudo rsync --rsync-path "sudo rsync" user@host:'/remotepath' localpath
</pre>

---
*What to sync*

When I moved from one cloud machine to another I had to decide what to sync and what to leave behind.  I was also moving from Debian to Ubuntu so I couldn't just sync everything.

What I took:
<pre>
source                target
----------------------------
/home                 /
/var/mail             /var
/var/lib/squirrelmail /var/lib
/var/www              /var
</pre>

---
*Multiline patterns in sed*

http://docstore.mik.ua/orelly/unix/sedawk/ch06_01.htm#SEDAWK-CH-6-SECT-1

e.g. "unfold" two lines back to a single line:
<verbatim>
sed '/@RolesAllowed($/{                                                
N
s/\n//
}' src
</verbatim>

---
---+++ Hardening Linux Machines
http://www.sans.org/score/checklists/linuxchecklist.pdf <br/>
http://cisecurity.org/en-us/?route=downloads.benchmarks <br/>
http://www.nsa.gov/ia/guidance/security_configuration_guides/operating_systems.shtml#linux2 <br/>
https://grepular.com/Protecting_a_Laptop_from_Simple_and_Sophisticated_Attacks <br/>
---

http://noone.org/blog/English/Computer/Shell/How%20to%20find%20broken%20symlinks.html - How to find broken symlinks
<pre>
$ find -L . -type l
</pre>

---
It's a good idea to enable the SMART daemon which isn't enabled by default, at least on Debian.  It will send an email if the disk reports something is wrong.

HOWTO enable smartd: http://www.cyberciti.biz/tips/monitoring-hard-disk-health-with-smartd-under-linux-or-unix-operating-systems.html

If you find that the disk reports any =Current_Pending_Sector= errors then this will help: http://smartmontools.sourceforge.net/badblockhowto.html

to force a full fsck on the next reboot, touch =/forcefsck=

---

*tcpdump*

Sometimes there's no substitute for looking at what's crossing the wire.  Assuming that it's not encrypted, of course.

<pre>
$ sudo tcpdump -s 0 -vv -A "host 173.76.76.13 and tcp port http" | tee capture.txt
</pre>

That command dumps all HTTP traffic to 173.76.76.13 into capture.txt and displays it on the console.  =-s 0= means no packet size limit, =-A= means dump in ASCII (good for HTTP), and =-vv= means verbose.