Working with Hex

set up alias

Before doing everything, we can manually set up some alias for many commands

nano .zshrc

# some more ls aliases
alias ll='ls -l'
alias la='ls -A'
alias l='ls -CF'

Some good text editor.

nano
leafpad
vim
mousepad

Looking for the usb or other driver in the system

fdisk -l

┌──(root㉿kali)-[~/Desktop]
└─# fdisk -l
Disk /dev/sda: 80.09 GiB, 86000000000 bytes, 167968750 sectors
Disk model: VBOX HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x16d6cb34

Device     Boot Start       End   Sectors  Size Id Type
/dev/sda1  *     2048 167968749 167966702 80.1G 83 Linux

Wipe the USB

First, we need write zero or totally wipe to a USB stick via either one of these tools.

[!NOTE]
Apart from all zero, we also can write all ones or random.

dd
dcfldd
dc3dd

┌──(root㉿kali)-[~/Desktop]
└─# dd if=/dev/zero of=/dev/sdb                                             
dd: writing to '/dev/sdb': No space left on device
245761+0 records in
245760+0 records out
125829120 bytes (126 MB, 120 MiB) copied, 384.179 s, 328 kB/s

┌──(root㉿kali)-[~/Desktop]
└─# dcfldd if=/dev/zero of=/dev/sdb
3840 blocks (120Mb) written.
3841+0 records in
3840+0 records out

┌──(root㉿kali)-[~/Desktop]
└─# dc3dd wipe=/dev/sdb            

dc3dd 7.2.646 started at 2024-09-03 16:04:09 +1200
compiled options:
command line dc3dd wipe=/dev/sdb
device size: 245760 sectors (probed),      125,829,120 bytes
sector size: 512 bytes (probed)
   125829120 bytes ( 120 M ) copied ( 100% ),   42 s, 2.9 M/s                 

input results for pattern `00':
   245760 sectors in

output results for device `/dev/sdb':
   245760 sectors out

dc3dd completed at 2024-09-03 16:04:51 +1200

After that, we can check if all meta data has been written to zero

hexedit

┌──(root㉿kali)-[~/Desktop]
└─# hexedit /dev/sdb

From the information, we know how many bytes and how many sectors is it

┌──(root㉿kali)-[~/Desktop]
└─# fdisk -l                                             
Disk /dev/sda: 80.09 GiB, 86000000000 bytes, 167968750 sectors
Disk model: VBOX HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x16d6cb34

Device     Boot Start       End   Sectors  Size Id Type
/dev/sda1  *     2048 167968749 167966702 80.1G 83 Linux

Disk /dev/sdb: 120 MiB, 125829120 bytes, 245760 sectors
Disk model: Disk            
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

So this USB stick has 120MiB or 125829120 bytes. Each sector is 512 bytes, so it has 245760 sectors in total. Here Linux use standard terminology MiB rather than MB.

[!NOTE]
A megabyte can be stated as 1000 kilobytes or 1024 kilobytes and so on for gigabytes, terabytes etc due to unclear definition.

The standard states that a kilobyte is 1000 bytes, a megabyte is 1000 kilobytes etc. The new terminology introduced adds to kilo, mega etc with kibi, mebi etc (ki for kilo meaning 1000 and bi for bit, so 1 kilobyte is 1000 bytes but 1 kibibyte is 1024 bytes. The new terminology uses a power of 2. So, 1 kibibyte is 2 to the power of 10 or 1024 bytes). Linux uses this terminology and so uses MiB to indicate Mebibyte, 1024 Kilobytes.

Put file into USB

Next, we need format this USB stick and choose the allocation unit size, remember sector is phyical size. While allocation unit size can be one or a power of two sectors, 1024 bytes, 2048 bytes, 4096 bytes, etc.

FAT（File Allocation Table）和 NTFS（New Technology File System）是两种常见的文件系统，主要用于不同的操作系统和用途。以下是它们的主要区别：

1. 文件系统结构:
FAT:
包括 FAT16 和 FAT32 版本。
结构简单，适合小型存储设备。
文件最大支持 4 GB（FAT32），分区最大支持 2 TB。
NTFS:
结构更复杂和先进。
支持更大的文件和分区大小（文件大小理论上限为 16 EB，分区理论上限为 16 EB）。
支持文件和文件夹压缩、加密、以及磁盘配额。
2. 兼容性:
FAT:
广泛兼容于各种操作系统，包括 Windows、macOS、Linux、以及大多数嵌入式系统和设备。
常用于 USB 闪存驱动器、SD 卡等可移动存储设备。
NTFS:
原生支持在 Windows 上，macOS 和 Linux 可以通过第三方工具读取和写入。
主要用于 Windows 操作系统的内部硬盘驱动器。
3. 性能与安全性:
FAT:
不支持文件和文件夹的权限控制。
不支持日志记录机制，数据恢复能力较弱。
性能较好，特别是在小文件和小型存储设备上。
NTFS:
提供文件和文件夹的权限设置和加密功能。
支持日志记录机制，有助于数据恢复和防止数据损坏。
支持磁盘配额管理和文件压缩，有助于高效利用存储空间。
4. 用途:
FAT:
适用于需要跨多个操作系统和设备的可移动存储设备。
NTFS:
适用于需要高安全性、大文件支持的内部存储设备，特别是在 Windows 系统中。

Put one picture into the USB stick

This file is actually 13.8379 kibibyts or 14170 bytes, but just show as 13.8KB. While size on disk is how big this file occupied. It uses two clusters, so some space of bytes are waste, and we alwasy call it slack space..

[!NOTE]
Using smaller cluster sizes is much more efficient but much more slower.

Calculate the file size

How to calculate the size from hexedit.
Hexedit navigating
Shift < or shift > : go to start/end of the file
Right: next character
Left: previous character
Down: next line
Up: previous line
Home: beginning of line
End: end of line
PUp: page forward
PDown: page backward
F2: save file
tab: switch to ACSII window

For example, if the file starts from 0x00 position (jpg file signature is FF D9 FF E0), and stops at 0x375A (jpg file signature is FF D9, and after the last byte).

Hex: 375A
So: 12,288 + 1,792 + 80 + 10 = 14170, which is the size of the file. We use this number and minus the starting number in decimal, which is 0, so our file is 14,170 KB.

In the USB stick, we also can calculate and get same result.

End Hex = 38AC75A is 59,426,650 in decimal
Start Hex = 38A9000 is 59,412,480 in decimal

59,426,650 – 59,412,480 = 14,170

Value the hash

To keep file consistency, we need check hash value in the whole proces.

md5sum
sha256sum
sha512sum

┌──(root㉿kali)-[~/Desktop]
└─# sha256sum /dev/sdb
322528227c50a156dfbaf8d5a994bbbd1299a484b2235e145ed77ac31713fe80  /dev/sdb

Delete the file

We can delete this jpg file, so the hash value will be changed, but actually the file is remain same location until new file overwritten it.

In hexedit, we can use tab to switch to ASCII window and search the deleted file name.

Some information about the deleted file at location 0x2946980 (decimal 43,280,768). The information contains the name of the file that was deleted and the date and time the file was deleted in UTC. The deletion information is vital for the forensic investigator, yet many file recovery tools do not provide information about the file name or deletion date and time.

[!NOTE]
This date and time is calculated from the date and time of the local computer. In this case, the date and time in Kali Linux on VirtualBox may be different from the date and time on the host computer. Therefore, as a forensic investigator, I must ensure that this is clear in the report: the date and time may relate to the host computer or virtual machine and is not necessarily accurate unless I have syncronised all times to the correct time.

Carve or recover the entire file

We can jump to the start and then use mouse to select all the required hex, but this is very differcult with many pages of hex, so we need do it with tools or commands.

dcfldd if=Desktop/lab1nocat.dd of=Desktop/catpic.dd skip=59412480 bs=1 count=14170

if=Desktop/lab1nocat.dd: Specifies the input file, lab1nocat.dd, located in the Desktop directory.

of=Desktop/catpic.dd: Specifies the output file, catpic.dd, which will be created in the Desktop directory.

skip=59412480: Skips the first 59,412,480 bytes of the input file before starting to read data. This is useful if you want to start reading from a specific point in the file.

bs=1: Sets the block size to 1 byte. This means that the command will read and write data one byte at a time.

count=14170: Specifies the number of bytes to read from the input file. In this case, it reads 14,170 bytes.

The file catpic.dd appears on the Kali Desktop and is a thumbnail of the cat picture even though it has a .dd file extension. It is evident that Linux is ascertaining the file type from the file signature, not the file extension as Windows does.

If I try to open the file in Windows, I get a query as to what software I should use to try to open the file. If I change the file extension to .xls for example, Windows will try to open the file in Excel and send an error message that the file is corrupted. If I do this in Linux, it will still open as a picture successfully.

Some carving tools will always look at the start of a sector / cluster for a file signature and so will not identify files beginning after the start of the sector / cluster. Other files may, by coincidence, have hex values that just happen to also be a file signature. For example, when carving a picture, you may come across ‘FF D9’ as a pixel colour, but this is also the end file signature for a jpeg. We want to ignore this ‘false’ end signature and can do so if we spend some time searching manually for end signatures. We may, for example, simply look for the final ‘FF D9’ rather than the first one(s). This is partially why carving tools can produce a lot of false negatives – Scalpel with Zip files for example. There is not enough checking of the start and end signatures so start and end hex that just happens to be a signature is accepted as a real signature.

Replace the hex where the cat picture was with zeros.
Find a suitable location at the start of a 512 KB sector to write the cat picture to.

Hide file

We also can write file to a new place, notice here is seek not skip. And notrunc is required.

dd if=Desktop/catpic.dd of=Desktop/lab1nocatcopy2.dd bs=1 seek=8212480 conv=notrunc

overwritten the original one

dd if=/dev/zero of=Desktop/lab1nocatcopy3.dd bs=1 seek=59412480 count=14170 conv=notrunc