CAINE 0.5 is not forensically sound

[forensics]

CAINE 0.5 recovers Ext3 file systems during the boot process and thus modifies the data on suspect media (including at least one important file system timestamp: "Last Write Time"); the screenshot is attached. The "bug" is in initrd casper scripts that perform unsafe fake read-only mounts of several file system types. The patch for scripts/casper-helpers function get_fstype():

Code:

get_fstype() {
    local FSTYPE
    local FSSIZE
    eval $(fstype < $1)
 
    # Fix: do not allow journal recovery on Ext3
    if [ "$FSTYPE" = "ext3" ]; then
        echo "ext2"
        return 0
    fi
 
    # Fix: do not allow journal recovery on Ext4
    if [ "$FSTYPE" = "ext4" ]; then
        echo "ext2"
        return 0
    fi
 
    if [ "$FSTYPE" != "unknown" ]; then
        echo $FSTYPE
        return 0
    fi
    /lib/udev/vol_id -t $1 2>/dev/null
 }

This will force all possible Ext3/4 mounts during the boot to disable all journal recovery actions.



CAINE also uses "proved-to-be-unsafe" mount policy that relies on file system options to perform "forensically safe" mounts. For example, CAINE will write the data to file systems like ReiserFS or XFS even with "ro" and "norecovery" (XFS) / "nolog" (ReiserFS) options specified. It also uses very buggy scanpartitions script that will ignore block devices like /dev/sdad1 and they will not appear in /etc/fstab with proper "forensic" options.

[nannib]

Thank you for the reporting, now we'll check it and I hope we could give you an answer
Bye

Nanni Bassetti

[nannib]

Can you tell us how we can reproduce the bug?
Thanks

[forensics]

Mount the Ext3 file system r/w on some Linux system and then reboot into CAINE without unmounting it.

[nannib]

We are testing...but from the first results, it seems that the problem is nonexistent, because the hash codes are equals...we are testing also using others Live Forensics Distro.

Just for updating you ;-)
Bye

[forensics]

but from the first results, it seems that the problem is nonexistent, because the hash codes are equals...

Very interesting when do you check the hash codes? Did you check dmesg to see that Ext3 was corrupted "correctly"? Do you conduct tests on HDD or USB Flash? Do you use installed CAINE or a Live CD?

we are testing also using others Live Forensics Distro.

The problem exists in all Ubuntu based forensic Live CDs like DEFT Linux, Helix3 (and Pro too), SMART Linux, BackTrack 4 Pre Release and FCCU GNU/Linux and all other distros that use Casper for booting (except grml that sets all block devices RO before mounting).

[nannib]

As I just written...we are testing
We made a test on an USB Ext3 device, now I'm testing on an Hard Disk...when we'll finish we'll publish our results...we hope that will not be problems

Thanks
bye

[nannib]

So these are some of our test:
1) We mounted, in RW mode, an USB ext3 device (previously hashed), we shoutdown the system
without unmounting the ext3 device (powering off unplugging the computer),
then we boot the system by the Live Caine Distro and we saw the messages you
indicated, after the boot the hash it's equal to the first hash of the
ext3 device.

2) We boot Caine Live on a workstation with 2 HDs Ext3 and on /dev/sda there is a Kubuntu 8.04 installed. Then no Hd has been mounted.
We hash the /dev/sda

We mounted only /dev/sda1 in RO (Read Only) and this is the output:
62dc038a11123ffd90d085bbe65e34a9 /dev/sda
We reboot the PC, by the reset button of the case (no shutting down procedure), then re-boot always using Caine and re-hash the device:
62dc038a11123ffd90d085bbe65e34a9 /dev/sda
So after and before the Read Only mounting the 2 hashes are equals.

3) We mount /dev/sda in RW (Read Write) we hash it:
2e1f0dc99757a6663f224c403540af08 /dev/sda
We reboot the PC, by the reset button of the case (no shutting down procedure), then re-boot always using Caine and re-hash the device:
2e1f0dc99757a6663f224c403540af08 /dev/sda
So after and before the Read and Write mounting the 2 hashes are equals.

We are reviewing again our tests...currently they seem correct...
Have you any suggestion for us?
Thanks a lot
Bye

[forensics]

3) We mount /dev/sda in RW (Read Write) we hash it:
2e1f0dc99757a6663f224c403540af08 /dev/sda
We reboot the PC, by the reset button of the case (no shutting down procedure), then re-boot always using Caine and re-hash the device:
2e1f0dc99757a6663f224c403540af08 /dev/sda
So after and before the Read and Write mounting the 2 hashes are equals.

We are reviewing again our tests...currently they seem correct...
Have you any suggestion for us?

Does dmesg shows that FS was recovered? Does fsstat (TSK) shows "recovery needed" flag after you boot into CAINE? I have conducted many tests on real hardware and in VirtualBox and CAINE always recovers and changes Ext3 file systems on HDDs.

"md5sum" for Ext3 FS after cold reboot with mounted Ext3 FS to forensically sound Live CD (RIPLinux):

6621614df4acf6c5ac3d1aea218d1698 /dev/hda1

and "file -s" output:

/dev/hda1: Linux rev 1.0 ext3 filesystem data, UUID=54d98d70-a4a3-4f8a-96b7-4dd3d20f7a43 (needs journal recovery)

Then I boot CAINE Live CD and rebooted again to RIPLinux:

e5da66391840d0ffcedaaef970c40e7b /dev/hda1

and "file -s" output:

/dev/hda1: Linux rev 1.0 ext3 filesystem data, UUID=54d98d70-a4a3-4f8a-96b7-4dd3d20f7a43

[denis]

Test on usb drive are mine.
I made it on Caine v.0.5, Helix 2008 R1, Deft v.4.2. Fccu v.12.1, all from live-cd.
This is the way used in first test session:
1- boot of system
2- connect usb drive with ext3 file system
3- mount usb drive in rw
4- write file on file system
5- umount drive
6- mount usb drive in rw
7- **pull out usb drive without umount it**
8- clean mount point
9- connect usb-drive without mount and without run cfsk
10- calculate md5
11- pull out usb-drive
12- shutdown system
13- connect usb drive
14- rebbot system with usb drive connected
15- check kernel logs that show whath you reported
16- calculated md5 on usb drive not mounted
17- md5 doesn't change

This is the way used in second test session **using a laptop without battery**:
1- boot of system
2- connect usb drive with ext3 file system
3- mount usb drive in rw
4- write file on file system
5- umount drive
6- mount usb drive in rw
7- ** shutdown system pulling out power cable **
8- extract usb drive
9- rebbot system
10- connect usb-drive without mount and without run cfsk
11- calculate md5
12- pull out usb-drive
13- shutdown system
14- connect usb drive
15- rebbot system with usb drive connected
16- check kernel logs that show whath you reported
17- calculated md5 on usb drive not mounted
18- md5 doesn't change

tests made on 3 Ububtu derived live-cd and 1 debian sidux don't show alteration on md5 of device with ext3 fs, it seems that systems don't upadte journal of ext3 fs, no change seem occours.




could you suggest another kind of test?
Regardes

[denis]

"md5sum" for Ext3 FS after cold reboot with mounted Ext3 FS to forensically sound Live CD (RIPLinux):

6621614df4acf6c5ac3d1aea218d1698 /dev/hda1

and "file -s" output:

/dev/hda1: Linux rev 1.0 ext3 filesystem data, UUID=54d98d70-a4a3-4f8a-96b7-4dd3d20f7a43 (needs journal recovery)

well I've RIPLinux, in next days I re-make tests

[forensics]

Don't make tests with USB Flash drives, instead use HDDs.

PS. When testing Helix3 I noticed that installed system do not use Capser, so they don't have this bug.

[denis]

@forensics
private message for you

[nannib]

Does dmesg shows that FS was recovered? Does fsstat (TSK) shows "recovery needed" flag after you boot into CAINE? I have conducted many tests on real hardware and in VirtualBox and CAINE always recovers and changes Ext3 file systems on HDDs.

These are my output:
caine@caine:~$ sudo fsstat -f ext -o 63 /dev/sda | more
FILE SYSTEM INFORMATION
--------------------------------------------
File System Type: Ext3
Volume Name:
Volume ID: 9d705cf85f90439d3e49bbc52b1519d7

Last Written at: Sat Oct 17 09:52:05 2009
Last Checked at: Mon Jun 23 19:44:55 2008

Last Mounted at: Sat Oct 17 08:57:18 2009
Unmounted properly
Last mounted on:

Source OS: Linux
Dynamic Structure
Compat Features: Journal, Resize Inode, Dir Index
InCompat Features: Filetype,
Read Only Compat Features: Sparse Super, Has Large Files,

Journal ID: 00
Journal Inode: 8

METADATA INFORMATION
...
...

then DMESG:

40.447463] Driver 'sd' needs updating - please use bus_type methods
[ 40.447610] sd 0:0:0:0: [sda] 488397168 512-byte hardware sectors (250059 MB)
[ 40.447780] sd 0:0:0:0: [sda] Write Protect is off
[ 40.447848] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 40.447867] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 40.447996] sd 0:0:0:0: [sda] 488397168 512-byte hardware sectors (250059 MB)
[ 40.448076] sd 0:0:0:0: [sda] Write Protect is off
[ 40.448145] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 40.448168] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 40.448246] sda:sr0: scsi3-mmc drive: 40x/48x writer dvd-ram cd/rw xa/form2 cdda tray
[ 40.449544] Uniform CD-ROM driver Revision: 3.20
[ 40.449656] sr 2:0:0:0: Attached scsi CD-ROM sr0
[ 40.453775] sr1: scsi3-mmc drive: 0x/50x cd/rw xa/form2 cdda tray
[ 40.454599] sr 3:0:1:0: Attached scsi CD-ROM sr1
[ 40.457638] sda1 sda4
[ 40.460306] sd 0:0:0:0: Attached scsi generic sg0 type 0
[ 40.460399] sr 2:0:0:0: Attached scsi generic sg1 type 5
[ 40.460485] scsi 2:0:1:0: Attached scsi generic sg2 type 0
[ 40.460569] sr 3:0:1:0: Attached scsi generic sg3 type 5
[ 40.482834] sd 0:0:0:0: [sda] Attached SCSI disk
[ 40.489336] end_request: I/O error, dev fd0, sector 0
[ 40.537323] usb 2-2: USB disconnect, address 2
[ 40.550619] ACPI: PCI Interrupt 0000:03:07.0[A] -> GSI 16 (level, low) -> IRQ 21
[ 40.550736] ACPI: PCI Interrupt 0000:00:12.0[A] -> GSI 23 (level, low) -> IRQ 20
[ 40.551359] eth0: VIA Rhine II at 0xfcfff800, 00:19:66:35:41:d2, IRQ 20.
[ 40.552139] eth0: MII PHY found at address 1, status 0x7849 advertising 05e1 Link 0000.
[ 40.603524] ohci1394: fw-host0: OHCI-1394 1.0 (PCI): IRQ=[21] MMIO=[feaf7800-feaf7fff] Max Packet=[2048] IR/IT contexts=[8/8]
[ 41.272557] usb 2-2: new low speed USB device using uhci_hcd and address 3
[ 41.446132] usb 2-2: configuration #1 chosen from 1 choice
[ 41.464254] input: Logitech USB Mouse as /devices/pci0000:00/0000:00:10.1/usb2/2-2/2-2:1.0/input/input3
[ 41.484434] input,hidraw0: USB HID v1.10 Mouse [Logitech USB Mouse] on usb-0000:00:10.1-2
[ 41.616248] end_request: I/O error, dev fd0, sector 0
[ 41.880088] ieee1394: Host added: ID:BUS[0-00:1023] GUID[001106000000729b]
[ 42.719192] end_request: I/O error, dev fd0, sector 0
[ 43.822136] end_request: I/O error, dev fd0, sector 0
[ 43.897808] sd 2:0:1:0: [sdb] 488397168 512-byte hardware sectors (250059 MB)
[ 43.898127] sd 2:0:1:0: [sdb] Write Protect is off
[ 43.898199] sd 2:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[ 43.898226] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 43.898366] sd 2:0:1:0: [sdb] 488397168 512-byte hardware sectors (250059 MB)
[ 43.898446] sd 2:0:1:0: [sdb] Write Protect is off
[ 43.898513] sd 2:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[ 43.898533] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 43.898613] sdb: sdb1
[ 43.900654] sd 2:0:1:0: [sdb] Attached SCSI disk
[ 44.929076] end_request: I/O error, dev fd0, sector 0
[ 45.017638] kjournald starting. Commit interval 5 seconds
[ 45.017644] EXT3-fs: mounted filesystem with ordered data mode.
[ 45.127440] kjournald starting. Commit interval 5 seconds
[ 45.127447] EXT3-fs: mounted filesystem with ordered data mode.
[ 47.724362] ISO 9660 Extensions: Microsoft Joliet Level 3
[ 47.753668] ISO 9660 Extensions: RRIP_1991A
[ 47.875598] Registering unionfs 1.4
[ 47.875671] unionfs: debugging is not enabled
[ 47.883886] loop: module loaded
[ 47.974945] squashfs: version 3.3 (2007/10/31) Phillip Lougher

What do you think about these data?
Thanks

[forensics]

The output means that Ext3 FS is not corrupted.

Here is a corrupted (using TSK on RIPLinux):

FILE SYSTEM INFORMATION
--------------------------------------------
File System Type: Ext3
Volume Name:
Volume ID: c80f53f0dc5404b64a4b9ebbbd403f32

Last Written at: Sat Oct 17 14:46:51 2009
Last Checked at: Thu Sep 10 18:53:49 2009

Last Mounted at: Sat Oct 17 14:46:51 2009
Unmounted properly
Last mounted on:

Source OS: Linux
Dynamic Structure
Compat Features: Journal, Ext Attributes, Resize Inode, Dir Index
InCompat Features: Filetype, Needs Recovery,
Read Only Compat Features: Sparse Super, Has Large Files,

Journal ID: 00
Journal Inode: 8

METADATA INFORMATION
--------------------------------------------
Inode Range: 1 - 3661825
Root Directory: 2
Free Inodes: 3336721

CONTENT INFORMATION
--------------------------------------------
Block Range: 0 - 14621175
Block Size: 4096
Free Blocks: 12899303

BLOCK GROUP INFORMATION
--------------------------------------------
Number of Block Groups: 447
Inodes per group: 8192
Blocks per group: 32768

Group: 0:
Inode Range: 1 - 8192
Block Range: 0 - 32767
Layout:
Super Block: 0 - 0
Group Descriptor Table: 1 - 4
Data bitmap: 1025 - 1025
Inode bitmap: 1026 - 1026
Inode Table: 1027 - 1538
Data Blocks: 1539 - 32767
Free Inodes: 8181 (99%)
Free Blocks: 31223 (95%)
Total Directories: 2

[nannib]

So in which way can I corrupt the file system?
I mounted it in RW, then I reboot without unmounting it...so I surrender how can I do?
Thanks

[forensics]

"fsstat" output after booting CAINE:

FILE SYSTEM INFORMATION
--------------------------------------------
File System Type: Ext3
Volume Name:
Volume ID: c80f53f0dc5404b64a4b9ebbbd403f32

Last Written at: Sat Oct 17 18:52:58 2009
Last Checked at: Thu Sep 10 18:53:49 2009

Last Mounted at: Sat Oct 17 14:46:51 2009
Unmounted properly
Last mounted on:

Source OS: Linux
Dynamic Structure
Compat Features: Journal, Ext Attributes, Resize Inode, Dir Index
InCompat Features: Filetype,
Read Only Compat Features: Sparse Super, Has Large Files,

Journal ID: 00
Journal Inode: 8

METADATA INFORMATION
--------------------------------------------

Related "dmesg" output after booting CAINE:

[ 47.709642] PCI: Setting latency timer of device 0000:00:1f.2 to 64
[ 47.709662] ACPI: PCI interrupt for device 0000:00:1f.2 disabled
[ 47.717597] ata_piix 0000:00:1f.2: version 2.12
[ 47.717607] ata_piix 0000:00:1f.2: MAP [ P0 P2 IDE IDE ]
[ 47.717860] ACPI: PCI Interrupt 0000:00:1f.2[b] -> GSI 19 (level, low) -> IRQ 20
[ 47.718020] PCI: Setting latency timer of device 0000:00:1f.2 to 64
[ 47.718464] scsi0 : ata_piix
[ 47.718598] scsi1 : ata_piix
[ 47.718749] ata1: SATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0x18b0 irq 14
[ 47.718816] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0x18b8 irq 15
[ 47.882306] ata1.00: ATA-7: FUJITSU MHV2100BH PL, 00000029, max UDMA/100
[ 47.882379] ata1.00: 195371568 sectors, multi 16: LBA48 NCQ (depth 0/32)
[ 47.898305] ata1.00: configured for UDMA/100
[ 48.216858] ata2.00: ATAPI: PHILIPS DVD+/-RW SDVD8820, AX02, max UDMA/33
[ 48.388575] ata2.00: configured for UDMA/33
[ 48.388878] scsi 0:0:0:0: Direct-Access ATA FUJITSU MHV2100B 0000 PQ: 0 ANSI: 5
[ 48.393595] scsi 1:0:0:0: CD-ROM PHILIPS DVD+-RW SDVD8820 AX02 PQ: 0 ANSI: 5
[ 48.404847] Driver 'sd' needs updating - please use bus_type methods
[ 48.405037] sd 0:0:0:0: [sda] 195371568 512-byte hardware sectors (100030 MB)
[ 48.405121] sd 0:0:0:0: [sda] Write Protect is off
[ 48.405188] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 48.405219] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 48.405373] sd 0:0:0:0: [sda] 195371568 512-byte hardware sectors (100030 MB)
[ 48.405453] sd 0:0:0:0: [sda] Write Protect is off
[ 48.405515] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 48.405545] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 48.405625] sda:<4>Driver 'sr' needs updating - please use bus_type methods
[ 48.429050] Clocksource tsc unstable (delta = -1363799154 ns)
[ 48.433035] Time: hpet clocksource has been installed.
[ 48.495995] usb 3-2: new full speed USB device using uhci_hcd and address 2
[ 48.657855] usb 3-2: configuration #1 chosen from 1 choice
[ 48.751719] sda1 sda2 sda3 sda4
[ 48.805609] sd 0:0:0:0: [sda] Attached SCSI disk
[ 48.814107] sd 0:0:0:0: Attached scsi generic sg0 type 0
[ 48.814203] sr 1:0:0:0: Attached scsi generic sg1 type 5
[ 48.814671] sr0: scsi3-mmc drive: 24x/24x writer cd/rw xa/form2 cdda tray
[ 48.814741] Uniform CD-ROM driver Revision: 3.20
[ 48.814873] sr 1:0:0:0: Attached scsi CD-ROM sr0
[ 48.899469] usb 4-2: new low speed USB device using uhci_hcd and address 2
[ 49.076300] usb 4-2: configuration #1 chosen from 1 choice
[ 49.102765] usbcore: registered new interface driver hiddev
[ 49.115535] input: Logitech USB-PS/2 Optical Mouse as /devices/pci0000:00/0000:00:1d.3/usb4/4-2/4-2:1.0/input/input2
[ 49.139217] input,hidraw0: USB HID v1.10 Mouse [Logitech USB-PS/2 Optical Mouse] on usb-0000:00:1d.3-2
[ 49.139504] usbcore: registered new interface driver usbhid
[ 49.139572] /build/buildd/linux-2.6.24/drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver
[ 49.312889] EXT3-fs: INFO: recovery required on readonly filesystem.
[ 49.312962] EXT3-fs: write access will be enabled during recovery.
[ 49.503602] kjournald starting. Commit interval 5 seconds
[ 49.503636] EXT3-fs: recovery complete.
[ 49.504277] EXT3-fs: mounted filesystem with ordered data mode.
[ 50.300953] ISO 9660 Extensions: Microsoft Joliet Level 3
[ 50.324450] ISO 9660 Extensions: RRIP_1991A
[ 50.457093] Registering unionfs 1.4
[ 50.457162] unionfs: debugging is not enabled
[ 50.471152] loop: module loaded
[ 50.524034] squashfs: version 3.3 (2007/10/31) Phillip Lougher
[ 59.843360] usb 5-1: new high speed USB device using ehci_hcd and address 4
[ 59.981851] usb 5-1: configuration #1 chosen from 1 choice
[ 60.027110] usbcore: registered new interface driver libusual
[ 60.040318] Initializing USB Mass Storage driver...
[ 60.041409] scsi2 : SCSI emulation for USB Mass Storage devices

So in which way can I corrupt the file system?

I'm doing the same as you described. Just check that after mounting the Ext3 r/w you have "Need recovery" flag set (ext3 sets this flag when mounting as R/W and removes it when unmounting).

[denis]

Bug has been reproduced!
In next days I'll write a post about it in my blog.

[nannib]

I'm remastering the distro applying the patches suggested, new mount policies and the casper patch...so maybe we'll be the only one distro, Ubuntu based, to be really forensically sound
Many thanks to Forensics from Russia

[nannib]

CAINE 1.0 is cooking with 3 important patches...stay tuned

[Sponsored content]