Table of contents
Preface
Instructions below are for Debian Buster (testing) running on Raspberry PI3 Model B Rev 1.2. Base system is preview image dumped to MMC card by instructions given on Debians RPI3 wiki page with additional packages required for kernel compilation.
The compilation is done natively on RPI3-board, no cross compiling. Internet is full of instructions for cross compiling so no instructions for that here. Also, there's no quarantees that these instructions work for you or produce minimal and optimal result for although they are titled with word "minimal". There exists barely any instructions for how to trim down your system to minimal. These are minimal for system to boot from MMC, after success with that you can start breeding it up again.
...which leads to question :
Why?
Why should you run your own kernel?
Personally I like to strip off everything unnecessary. I don't install any packages just in case or because smart guys in internet tell me to do. I want to understand myselves why I need something or - if everyone tells me I need it - is there any way around it. Also with kernel, it gives you a way to omit stuff - so I omit. I have been doing that over 20 years starting from kernel 0.98. (And I think this is (or was) lately even a trend. Something called Kon Mari...)
YES, stripping down kernel doesn't make your CPU run any faster and YES, it calculates the value of PI in just same speed than before. (That would be something the Phoronix would test and afterward judge stripped down kernels useless). But stripping off unneeded protocols, netfilter modules, devices, etc. reduces latencies thus making system more responsive.
At least my system boots up trice while waiting stock Ubuntu reaching it's login.
Also, ask yourself. Why you server needs to have IPX, Appletalk, drivers for multitudes of web cameras, drivers for adapters you don't have or never even heard of? Stripping down everything reduces greatly the attack surface you system have.
Short answers to acute questions
- Do you need to run specific RPI kernel? No.
- Can you use official stock linux kernel? Yes.
- Does it run perfectly? I don't know.
- So why to use it? Because it's original. Others are derivatives.
And important warning
Actions described here does not necessarily work, they might delete your files, spoil you MMC-card, break your PI-board, steal your car, burn your house, get your sick, destroy humanity - in short : it possible they cause ANYTHING between complete success and complete failure (ie. collapse of universe).
So you are on your own and I am not responsible of anything. If you decide to go on after all these warnings, it's your decision. Not mine.
Kernel compilation, 4.14 -series
Packages required: (debian-way).
rpi3:~$ sudo apt-get install build-essential libncurses5-dev bc bison flex
Newer 4.19 series kernels seem to require bison and flex. I recall that these weren't required before. Of course you can always test without to not pollute your system with unrequired packages.
Kernel series 4.14 and 4.19 are tested to boot from MMC with minimal config. Instructions below uses kernel 4.14 -series since it fresh but mature enough with long term support. But of course you can also fetch one from 4.19 -series and use config on that.
The minimal config contains :
- Support for USB-keyboard. If not required, remove support for USB-HID
- But no : (enable them by yourself)
- Support for external removable disk and devices
- Netfilter & iptables
- Support for IPv6
- Support for RPI's electronic interfaces I2C, GPIO, SPI, etc.
- Support for initrd and ramdisks, you just have no reason to mess with them.
If minimal config is enough, here's :
- One with support for external disk so that you can boot from external usb hard drive.
- Another with external disk support, IPv6 and basic Netfilter modules
Also, please note that unlike Rasbian, Debian Buster is 64bit distribution that runs under aarch64 -kernel.
Sources
So, get the sources. Probably there is newer version from 4.14-series available at kernel.org, so check and get newer one. Changes inside series are basically just security or some fatal-flaw patches ie. good ones to get included.
rpi3:~# cd /usr/src/ rpi3:/usr/src# wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.14.83.tar.xz rpi3:/usr/src# tar -xvJf linux-4.14.83.tar.xz
Get minimal configuration :
rpi3:/usr/src# cd linux-4.14.83 rpi3:/usr/src/linux-4.14.83# wget -qO .config http://sub.nanona.fi/bin/rpi3/minimal_kernel_config_4.14 rpi3:/usr/src/linux-4.14.83# md5sum .config ec0f73abd26945b91726b604237f5190 .config
Fine tune configuration for your needs or at least do menuconfig end exit by exit :
rpi3:/usr/src/linux-4.14.83# make menuconfig
Start compile (of course utilizing all 4 cores) and wait about hour to it complete :
rpi3:/usr/src/linux-4.14.83# make clean rpi3:/usr/src/linux-4.14.83# make -j4
Kernel install
The compile should end with success.
.... LD [M] drivers/net/mii.ko LD [M] drivers/net/tun.ko LD [M] drivers/net/usb/smsc95xx.ko LD [M] drivers/net/usb/usbnet.ko LD [M] fs/binfmt_misc.ko LD [M] fs/fuse/fuse.ko rpi3:/usr/src/linux-4.14.83$
So install modules.
rpi3:/usr/src/linux-4.14.83$ sudo make modules_install INSTALL drivers/block/loop.ko INSTALL drivers/block/nbd.ko INSTALL drivers/char/hw_random/bcm2835-rng.ko INSTALL drivers/char/hw_random/rng-core.ko INSTALL drivers/hid/hid-generic.ko INSTALL drivers/hid/hid.ko INSTALL drivers/hid/usbhid/usbhid.ko INSTALL drivers/net/dummy.ko INSTALL drivers/net/mii.ko INSTALL drivers/net/tun.ko INSTALL drivers/net/usb/smsc95xx.ko INSTALL drivers/net/usb/usbnet.ko INSTALL fs/binfmt_misc.ko INSTALL fs/fuse/fuse.ko DEPMOD 4.14.83-minimal0 rpi3:/usr/src/linux-4.14.83$
New kernel resides at path linux/arch/arm64/boot/Image. But since RPI's boot partition is required to be in vfat, there's no symlinking available in boot stage. My recommendation is to copy new kernel always to static generic path /boot/firmware/vmlinux BUT keep one or more backups of old working one in case system doesn't boot and you need to revert back to previous good one.
Of course you can copy them as /boot/firmware/vmlinux-4.14.83-myversion, but in this case you need always change config.txt to point that last one. I'd say it's easier to keep config.txt and cmdline.txt as unchanged and only copy right kernel as /boot/firmware/vmlinux
Install new kernel
rpi3:/usr/src/linux-4.14.83$ sudo cp arch/arm64/boot/Image /boot/firmware/vmlinux rpi3:/usr/src/linux-4.14.83$ sudo cp arch/arm64/boot/Image /boot/firmware/vmlinux-4.14.83-minimal0
The former is what RPI boots, the latter is just for backup and reverting if necessary in future.
Onetime changes to config files
Kernels here does not use initrd. It's completely unnecessary complication in situation here we know perfectly well what's required to boot up until login prompt and can compile them in. Because original distribution kernel boots with help of initrd, it needs to be removed from config files to go on with non-initrd one.
First, back up old. You might need to revert to originals
rpi3:/usr/src/linux-4.14.83$ cd /boot/firmware/ rpi3:/boot/firmware$ sudo cp config.txt config.old rpi3:/boot/firmware$ sudo cp cmdline.txt cmdline.old
Fetch example ones (or modify existing accordingly).
rpi3:/boot/firmware$ sudo wget -qO config.new http://sub.nanona.fi/bin/rpi3/config.txt rpi3:/boot/firmware$ cat config.new # Switch the CPU from ARMv7 into ARMv8 (aarch64) mode arm_control=0x200 enable_uart=1 device_tree=bcm2837-rpi-3-b.dtb kernel=vmlinux
The kernel= is where you tell system to boot the /boot/firmware/vmlinux. It's direct filename, since rpi's boot firmware mounts first partition of MMC as boot device and sees it's contents as root directory.
cmdline.txt is generic lilo / grub kernel command line.
rpi3:/boot/firmware$ sudo wget -qO cmdline.new http://sub.nanona.fi/bin/rpi3/cmdline.txt rpi3:/boot/firmware$ cat cmdline.new console=tty0 console=ttyS1,115200 root=/dev/mmcblk0p2 rw fsck.repair=yes net.ifnames=0 rootwait
Enable new's. You can clean up these later, once purging the distributions own kernel packages. Once distributions own packages are purged config.txt and cmdline.txt will remain unchanged. Having .old's and .new's are just shortcut to revert back to distributions own kernel.
rpi3:/boot/firmware$ sudo cp config.new config.txt rpi3:/boot/firmware$ sudo cp cmdline.new cmdline.txt
And boot with hopes high.
rpi3:/boot/firmware$ sudo /sbin/reboot
In case it failed.
If previous kernel was the distributions own kernel package : Plug MMC-card to some other computer and restore the old config.txt and cmdline.txt.
-whereever-:~$ cp config.old config.txt -whereever-:~$ cp cmdline.old cmdline.txt
Hope you backed the as instructed above. Of course extra step is to cat-ensure they point to that long-named vmlinuz-xxx -file with equal length initrd-cryptic-file.
But if you succeed to boot with own self-compiled kernel and decided to trim it even more (resulting unbootable system), just plug the card to some other computer and restore just last good kernel.
rpi3:/boot/firmware$ sudo cp vmlinux-4.14.83-net0 vmlinux
Hope you made that extra copy of it and also remember what WAS the last working one.
Aftermath.
After you have you own fancy trimmed-down kernel running (and backup of it) you can get rid of the distribution's own bloaty kernel packages.
rpi3:~$ sudo apt-get --purge remove linux-image-* ... The following packages will be REMOVED: linux-image-* 0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded. After this operation, 2 068 kB disk space will be freed. Do you want to continue? [Y/n]
Also plenty of other software gets unneeded :
rpi3:~$ sudo apt-get --purge remove initramfs-tools rpi3:~$ sudo apt-get --purge autoremove rpi3:~$ sudo apt-get --purge autoremove # again until cleaned
After removing native kernel packages it's good idea to check /boot/firmware/config.txt and /boot/firmware/cmdline.txt to verify uninstall did not revenge and install (or remove) some old archived backups.
That all, but one extra hint
RPI3 is actuallly quite fast but runs sluggish from MMC. To utilize it as real server, aquire external USB hard drive and clone MMC's second partition to external's disk (probably) first partition. Best do do that on external machine so that you can copy filesystem in it's shutdown state.
It's also good idea to add second small partition for swap to that new external disk. Swapping to MMC is not good idea, but on USB-harddrive it works.
Something like :
-whereever-:~$ mkdir SRC -whereever-:~$ mkdir DST -whereever-:~$ sudo mount /dev/mmcblk0p2 SRC -whereever-:~$ sudo mount /dev/sdb1 DST -whereever-:~$ cd SRC/ -whereever-:~/SRC$ sudo cp -ax . ../DST/ -whereever-:~/SRC$ cd
Edit external disk's /etc/fstab to find root from sda, not from mmc. (This disk will be your sda)
-whereever-:~$ cat DST/etc/fstab /dev/sda1 / ext4 rw 0 1 /dev/mmcblk0p1 /boot/firmware vfat rw 0 2 /dev/sda2 none swap sw 0 1 proc /proc proc defaults 0 0
And set RPI's kernel commandline to use /dev/sda1 as root device instead of /dev/mmcblk0p2
-whereever-:~$ mkdir BOOT -whereever-:~$ sudo mount /dev/mmcblk0p1 BOOT -whereever-:~$ cat BOOT/cmdline.txt dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty0 root=/dev/sda1 fsck.repair=yes net.ifnames=0 rootwait
By this way RPI's own booloader load the kernel, kernel starts, probes it's devices finding that external USB hard drive, and finally mounts it as root continuing booting and omittin /dev/mmcblk0p2 altogether.
The important thing is to have rootwait= so that external disk has time to settle until kernel boot triest to mount it and continue from it.
Also you need to have USB-mass media support (SCSI-disk and USB-massmedia) compiled in. Not as modules. If they are as modules, they are on the disk you are trying to mount (that you cannot do because you don't have modules to drive that disk). Ie. you need to have at least reference kernel configuration usbscsi_kernel_config_4.14
After changes, remember to unmount before unplugging.
-whereever-:~$ sudo umount BOOT SRC DST