mostlyobvious

Moving Partitions Across Drives

9 min read

One thing you might need to find yourself doing is moving partitions around, resizing them, and the filesystems on them, etc. Note, this can be very error prone, but here are some notes.

Also note, this is more focused on system maintenance, not necessarily for drive cloning/deployment. In the past I’ve used tools like FSArchiver with success. Several of these are included on the Archiso Live USB including FSArchiver and Partclone.

4K Sectors

Most modern NVMe SSDs use 512e (e stands for emulation) where 4K physical sectors present as 512 sectors for backwards compatibility.

If you run exclusively Linux, if you’re drive’s firmware allows it, you can switch to using 4K sectors (4Kn mode, where n stands for native, which match modern file systems’ default 4K block sizes) and get slightly better (~10%?) performance (and may make sector alignment less of an issue? not sure if this is a problem or not).

Switching block size modes will wipe your drive, so if you’re planning on doing this, do this when you first get the drive or if you’re wiping it anyway.

How to do it:

# Check on which drive to select
lsblk -o NAME,SIZE,FSTYPE,UUID,PARTUUID,MOUNTPOINT

# Look for "LBA Format" to see if it's set to 512 or 4K and support
nvme id-ns -H /dev/nvme0n1

# Set to the proper format (1 is whatever the right format is)
# This will delete everything on that drive!
nvme format --lbaf=1 /dev/nvme0n1

Tools

These are all included on the Archiso Live USB, which is as good of an env as any if you’re doing minor disk surgery (there are probably better specialized Live USBs, but I didn’t need it).

lsblk - list your devices
blkid - checking individual volumes

fdisk - classic partitioning tool
cfdisk - modern partitioning tool, easiest to use IMO
sfdisk - partitioning tool for scripting
gdisk - if you want to backup GPT data (to binary)
parted - slightly easier disk resizing

# assuming you're using ext4
e2fsck - filesystem integrity
resize2fs - for resizing filesystem
tune2fs - to get detailed filesystem information

Same Drive Sizes

If you are moving to identical drives with identical sector counts (or a smaller to larger drive), then you’re in luck. All you need to to is to dd from the old to the new drive and it’ll be completely cloned:

dd if=/dev/SOURCE of=/dev/TARGET bs=4M status=progress

If the drives are identical, you’re basically done. Everything (including alll UUIDs/PARTIDs, etc are the same). If you need to expand the drive you could do something like this:

# Expand partition
parted /dev/DEVICE
  p # print partition numbers
  resizepart PARTITION_NUMBER 100%
  quit

# Resize ext4 filesystem to fill the partition
e2fsck -f /dev/DEVICEPARTITION
resize2fs /dev/DEVICEPARTITION

Smaller Drive Sizes

For me, I had bought a replacement 4TB drive for a Maxio MAP 1602 drive that was having issues - I had previously compiled a custom kernel with the original patch from a Chinese forum (looks like there’s a new patch that might properly fix things!), but it recently started acting up again and simply wouldn’t boot. Rather than trying to fight things (do a custom kernel patch/compile from recovery), I decided to just order a new drive since I could get a decent same day replacement for <$200. (I got a 4TB Silicon Power UD90 which uses a Phison E21T controller and Micron 176L TLC NAND). Of course that didn’t work out so simply (and hence why you’re reading this).

While both are 4TB drives, it turns out the Maxio drive is actually 4096GB (3.73TiB) while the Silicon Power one is 4000GB (3.64TiB). I had a 64GiB swap partition that I could have dropped, but sadly, that still wouldn’t have been big enough so I would have to resize my partition. At this point it may have been worth looking at a tool like FSArchiver (especially since I only had a 10Gbps external SSD USB enclosure and had about 2TB to transfer) but I decided to go down the old school route.

The first thing to do when messing with partitions is to make sure you have a proper backup of anything you care about.

Also, it’s important to use lsblk and/or fdisk -l and to really make sure you aren’t making any typos and have proper device names, otherwise you will be hosed. Remember also that /dev/nvmeX device ids are randomly assigned if you have multiple drives (I had a second NVMe drive).

The approach we’ll take is simple. Since the existing disk is only about 1/3 full, what I’ll do is resize the file system and then use dd on a per-partition basis to copy them over.

Resize File System

Here’s how to do it for an ext4 filesystem:

# Detailed list of partitions
lsblk -o NAME,SIZE,FSTYPE,LABEL,UUID,PARTUUID,MOUNTPOINT

# If you want to find your exact block usage, size, and free blocks
# You'll need to do some math if you want exact numbers
tune2fs -l /dev/DEVICEPARTITION

# Alternatively, just mount the drive and do a `df` to get usage

# Necessary to run before resize, also fix any errors
e2fsck -f /dev/DEVICEPARTITION

# Resize - you can use K, M, G, T for respective units
resize2fs /dev/DEVICEPARTITION 2T
# I assume it'll complain if it's too small, but better safe than sorry

Now that your partition is smaller, you have some options to copy the partition over.

Copy via dd of the filesystem

You could just dd the partition as is, but limit it to slightly bigger than the file size:

# 2 TiB = 2 * 1024 * 1024 = 2097152 MiB
# Go over a bit to be safe and make sure that you get the whole file system
dd if=/dev/SOURCEDEVICEPARTITION of=/dev/TARGETDEVICEPARTITION bs=1M count=2098000 status=progress

Note, you can use cfdisk or your preferred partition tool to create the partitions you want on the target drive first.

Copy after resizing the partition

If you want, you could resize the partition to encapsulate the file system and then copy it. This makes sense if you want use sfdisk to clone your drive (eg, if you’re really attached to your UUIDs, but may actualy make life tougher if you ever have the drives plugged in together since you know, the UUIDs are supposed to be unique).

Backup and Restore partition tables

Before touching your partition tables, I highly recommend this since it’ll let you easily restore if you mess up (if you screw up your partitions, your data is still there as long as your partitions are restored, you’re fine).

Backup:

sfdisk -d /dev/DEVICE > partition-backup.txt

Restore:

sfdisk /dev/DEVICE < partition-backup.txt

You may also want to use gdisk and make a backup of your GPT table for extra safety here.

Resize the source partition

For safety you could make the partition slightly larger, although the math should work out to allow your to perfectly size it. You can try to mount or use tune2fs to check after I suppose.

# Expand partition
parted /dev/DEVICE
  p # print partition numbers
  resizepart PARTITION_NUMBER 2.1TB
  quit

Copy the device

Now, you have partitioned space and filesystem that is comfortably smaller than the target. You should be able to use sfdisk or disk now to copy the partition tables and then dd the partitions to the new drive.

How it should work:

# sfdisk
sfdisk -d /dev/SOURCEDEVICE > ptransfer.txt
# It's possible you might need to change the lba-end count, but I didn't check
sfdisk /dev/TARGETDEVICE < ptransfer.txt

# gdisk
gdisk /dev/SOURCDEVICE
  b # backup file

gdisk /dev/TARGETDEVICE
  r # recovery
  l # load from backup file

Then you can dd each partition:

dd if=/dev/SOURCEDEVICEPARTITION of=/dev/TARGETDEVICEPARTITION bs=4M status=progress

Expand Your Filesystem

Once you’ve moved over your filesystem to the cloned drive, you can expand it to the full size you want

# Expand your partition if necessary
parted /dev/DEVICE
  p # print partition numbers
  resizepart PARTITION_NUMBER 100%
  quit

# cfdisk has a resize as well I believe

# Resize ext4 filesystem to fill the partition
e2fsck -f /dev/DEVICEPARTITION
resize2fs /dev/DEVICEPARTITION

Cleanup

  • Doublecheck your fstabfile if you’re using UUIDs for booting (which you should be for NVMe devices!) to make sure everything’s pointing to the right place. Remember, you can use lsblk on a device, or blkid on a partition to get details
  • Double check your /boot/loader/entries/ as well to make sure your root= setting is correct. If you have updates, you will need to run bootctl install or grub-install (or whatever update command for the bootloader you use)

More Sector Notes

Besides the 4K Advanced Format stuff, Some more “fun” notes on sectors:

  • Most tools will start with 2048 (1MiB) for safety. This is fine, but for Advanced Format, you want to make sure it’s a multiple of 8, so 40 would be the minimum (you need 34 for GPT)
  • You will also want to reserve 34 sectors at the end for the backup GPT. Most partition creation tools handle that, although I noticed some tools like cfdisk and fdisk -l differ on whether they output the raw sector count or the subtracted sectors, so if you notice a discrepancy, that might be why. YMMV
  • If you’re slightly OCD about wanting to put your swap drive at the very end at the exact right side, you’ll need to do your own math. Multiple 512-byte sectors and calculate the bytes you need. eg for a 64GB swap, you’ll need: 64 * 1024 * 1024 * 1024 / 512 = 134217728 sectors, subtracted from the end (depending on how low level you are going you might need to account for the GPT backup sectors as well). My recommendation is to just use fdisk or cfdisk to create the largest partition, then use sfdisk to output the partition table, and then use sfdisk and use an editor to double-check your sector math. Remember to not resize2fs until after you’ve finalized your partitions.
  • For 512 sectors, you should probably make all your partition boundaries a multiple of 8 to respect 4K boundaries for Advanced Format. This may also be required for some partition encryption schemes as well.
  • Just in case, regenerating UUIDs

One last interesting note, I found the man and --help files to be largely inadequate (they largely don’t specify how they specify units or other options) and the last time I a lot of hard core sector math was back in the CHS days from decades past (although about 10 years ago I had dug hard into volume cloning and most of that tacit knowledge was still useful). For assistance during my volume transfer, I found ChatGPT4 on my phone to be 95% useful while I was in the recovery shell (and while I’ve been writing this): https://chat.openai.com/share/ea0f85a3-9d31-4389-8790-69b9c6ec7f15