Posted on 2005-03-11 21:49:12, modified on 2006-01-09 16:29:23
Tags: Computers, Linux
Owned...
Sooner or later it was bound to happen, and for somebody who has a number of machines under his control it is a nightmare scenario.
Victim: a small box used for testing the OpenGroupware software suit. The box is running FC2 and in my childish innocense for the time being, the root password was.... root. Need to say more?
The next day while trying to figure out why OpenGroupware didn't like what I was trying to do (see http://bugzilla.opengroupware.org/bugzilla/show_bug.cgi?id=1270) and saw:
[ogo@boxter opengroupware.org]$ ps xuaw | grep 32755 Segmentation fault
Err... impressive. Why?
[root@boxter root]# dmesg Segmentation fault
Euhm... this is tricky. Why?
[root@boxter root]# reboot
And the machine didn't come back. The next day I could come to the console and saw it was hanging in "INIT 2.65". Not really skilled in Linux and how to debug it *before* the kernel was fully loaded, I booted the box with a Knoppix CD in the hope I could just restore /boot from a different machine. The box didn't come back, but before I overwrote (instead of saving a copy of it.... don't ask) I realized that the md5 checksum of the initrd-2.6.5-1.358.img and vmlinuz-2.6.5-1.358 where different than the one of the different machine.
Another reboot, still no fish. Knoppix again, and wondering what went wrong. Hardware issue? If so, why has this box worked perfectly for months and now suddenly decided to throw up?
The Knoppix CD contains the chkrootkit command, and for some reason I ran it, just to be sure:
root@0[~]# chkrootkit -r /mnt/hda2 ROOTDIR is `/mnt/hda2/' Checking `basename'... not infected [...] Checking `date'... /bin/sh INFECTED [...]
Oh. Euhm. Aha. That explains some things.
root@0[~]# ls -al /mnt/hda2/bin/date -rwxr-xr-x 1 root root 49520 Mar 3 17:03 /mnt/hda2/bin/date
Yups. That's yesterday, while the box was installed the week before and the binaries on a different FC2 machine which were:
[~] root@tardis>ls -al `which date` -rwxr-xr-x 1 root root 45424 May 5 2004 /bin/date
Let's check some basic facts first. Who has been logging in?
root@0[~]# last -f /mnt/hda2/var/log/wtmp -n 30 root pts/32 edwin-3.int.barn Thu Mar 3 17:00 - down (00:02) root pts/31 edwin-3.int.barn Thu Mar 3 16:42 - down (00:20) root pts/30 edwin-3.int.barn Wed Mar 2 20:11 - 21:31 (01:20) root pts/29 203.85.90.88 Wed Mar 2 16:35 - 16:52 (00:16) root pts/28 edwin-3.int.barn Tue Mar 1 17:38 - 22:33 (04:54) root pts/27 147.46.244.31 Tue Mar 1 05:37 - 05:45 (00:08) root pts/26 202.39.75.131 Tue Mar 1 05:17 - 05:17 (00:00) root pts/25 edwin-3.int.barn Tue Mar 1 00:30 - 22:33 (22:03) root pts/24 edwin-3.int.barn Mon Feb 28 18:24 - 00:03 (05:38) root pts/23 edwin-3.int.barn Mon Feb 28 18:16 - 22:33 (1+04:16) root pts/22 edwin-3.int.barn Mon Feb 28 17:40 - 22:33 (1+04:53)
Oh... Three people have been able to figure out that my root password was root.
Which files were changed?
According to an "ls -laR", the following files were changed:
-rw-r--r-- 1 root root 0 Mar 3 17:02 /mnt/hda2/halt
/mnt/hda2/bin: total 5260 drwxr-xr-x 21 root root 4096 Mar 3 17:02 .. -rwxr-xr-x 1 root root 22468 Mar 3 12:02 cat -rwxr-xr-x 1 root root 40124 Mar 3 17:03 chown -rwxr-xr-x 1 root root 49520 Mar 3 17:03 date -rwxr-xr-x 1 root root 10268 Mar 3 17:03 dmesg -rwxr-xr-x 1 root root 58528 Mar 3 17:03 dumpkeys -rwxr-xr-x 1 root root 17792 Mar 3 17:03 false -rwxr-xr-x 1 root root 260572 Mar 2 16:51 gawk -rwxr-xr-x 1 root root 81392 Mar 3 17:03 grep -rwxr-xr-x 3 root root 61360 Mar 3 17:03 gunzip -rwxr-xr-x 3 root root 61360 Mar 3 17:03 gzip -rwxr-xr-x 1 root root 14904 Mar 3 17:03 hostname -rwxr-xr-x 1 root root 32804 Mar 3 17:03 ipcalc -rwxr-xr-x 1 root root 27404 Mar 3 17:03 login -rwxr-xr-x 1 root root 84784 Mar 3 17:03 ls -rwxr-xr-x 1 root root 27932 Mar 3 17:03 mkdir -rwsr-xr-x 1 root root 33196 Mar 3 17:03 ping6 -rwxr-xr-x 1 root root 19568 Mar 3 17:03 pwd -rwxr-xr-x 1 root root 19564 Mar 3 17:03 rmdir -rwxr-xr-x 1 root root 37556 Mar 3 17:03 setfont -rwxr-xr-x 1 root root 22712 Mar 3 17:03 setserial -rwxr-xr-x 1 root root 52656 Mar 3 17:03 sort -rwxr-xr-x 1 root root 42580 Mar 3 17:03 stty -rwxr-xr-x 1 root root 18368 Mar 3 17:03 sync -rwxr-xr-x 1 root root 157660 Mar 3 17:03 tar -rwxr-xr-x 1 root root 13820 Mar 3 17:03 tracepath -rwsr-xr-x 1 root root 57420 Mar 3 17:03 umount -rwxr-xr-x 3 root root 61360 Mar 3 17:03 zcat
(At this moment I lost interest, the machine got reinstalled and nothing is left of it)