Cloning Debian system in extra partition before update (with EFI boot support)

Note: This is an update for my Cloning Debian system in extra partition before update article so that it takes my efi boot support into account. It also works in a systemd system. The explanation is: Linux has the mount tree by default not shared. systemd changes this default setting while booting up. If you don’t do it this way you will have double mount when using –bind.

My system among others has a home partition, a main Debian system and an extra Debian system. My extra Debian system is for saving current Debian Unstable just before updates or upgrades. That’s what I’m going to describe here.

So my current partition is sda4. The Debian clone partition is /dev/sda6.

The overall process is:

• Copy system data
• Change fstab to reflect new partition as root partition
• Update grub

I do not install grub from Debian clone partition because I’m updating the current system and not the clone one.

In order to copy system data I need to mount the current system in a special directory so that only the sda4 partition is seen and not all of the partitions mounted in it (we do not want to copy /home).

mkdir /mnt/origen
mount -o bind --make-private / /mnt/origen

We also need to mount the destination partition

mkdir /mnt/destino
mount -t ext4 /dev/sda6 /mnt/destino

Now we are going to do a live copy. I’m not stopping any services because I personally do not use any of them. If this was a production server you would need to run the commands twice making sure the last time you run it you stop all the services you want to preserve previously.

rsync -aHK --delete --delete-during /mnt/origen/ /mnt/destino/

This is the most efficient way of copying the files because if the files are the same ones they are not copied… although I think Rsync in local copies does not help too much because usually all the destination file has to be read to decide to overwrite it or not. But I’m not quite sure. Anyway… let’s continue.

Now we are going to edit fstab and reinstall grub

mount -o bind /dev /mnt/destino/dev
mount -o bind /proc /mnt/destino/proc
mount -o bind /sys /mnt/destino/sys
mount -o bind /boot/efi /mnt/destino/boot/efi
chroot /mnt/destino/
vim /etc/fstab

Now inside fstab I’m going to replace:

/dev/sda4 /               ext4    errors=remount-ro 0       1

with:

/dev/sda6 /               ext4    errors=remount-ro 0       1

and save the file. You might probably want to use LABELs or UUIDs instead of me.

Once we have saved it we are going to update grub:

update-grub

Now we just need to exit from chroot and umount all the partitions and folders.

exit
umount /mnt/destino/proc
umount /mnt/destino/boot/efi
umount /mnt/destino/sys
umount /mnt/destino/dev
umount /mnt/destino
umount /mnt/origen

So that’s it! If you boot your system with Super Grub2 Disk or another tool you will be able to load your grub.cfg and boot this SDA6 system without too much effort.

Now I’m ready to upgrade or update my Debian Unstable system in sda4 without fear for loosing my daily functionality because it has been cloned in sda6.

 

Waiting for btrfs to be stable in Debian so that I can use snapshots :).

Anuncio publicitario

Init systems choice in Debian the Debian way

Disclaimer

I am a Debian user. I am not neither a Debian Developer nor a Debian maintainer. I develop a Debian derivative named Rescatux. There is no such a Debian way so please re-read this article title as: Init systems choice in Debian in the way that adrian15 think the Debian way it is.

This article serves as a one place to redirect everyone who asks me about what I think about all this fuzz about Debian making systemd as their init default choice in x86 and amd64 platforms.

I did not want to write a-yet-another-systemd-article but, as I don’t want to keep searching the Internet to find another article which mostly fits my views, I am writing it.

Background

Debian Background

According to wikipedia Debian entry:

Debian is the name for a Linux distribution that is composed primarily of free and open-source software, most of which is under the GNU General Public License, and packaged by a group of individuals known as the Debian project. At each point in time the Debian project offers three branches named «stable», «testing» and «unstable».

The Debian Stable distribution is one of the most popular for personal computers and network servers, and has been used as a base for several other Linux distributions.

…

Debian is known for its manifesto, social contract, and policies. Debian’s policies and team efforts focus on collaborative software development and testing processes. As a result of its policies, a new major release tends to occur every two years with revision releases that fix security issues and important problems.

systemd Background

Once again according to Wikipedia systemd article:

systemd is a suite of system management daemons, libraries, and utilities designed for Linux and programmed exclusively for the Linux API. Systemd authors characterize the software suite as a «basic building block» for an operating system.

For systems using it, the daemon systemd is the first process that is executed in user space during the Linux startup process. Therefore, systemd serves as the root of the user space’s process tree. The name systemd adheres to the Unix convention of making daemons easier to distinguish by having the letter d as the last letter of the filename.

…

Systemd is not just the name of the init daemon but can also refer to the entire software bundle around systemd. This includes the daemons systemd, journald, logind and networkd and many other low-level components such as libraries and utilities. In January 2013, systemd author Lennart Poettering described systemd not as one program, but rather a large software suite that includes 69 individual binaries.

Systemd in Debian Background

After the Debian Technical Committee was asked to vote on default Linux init system for jessie  in February 2014 systemd was the winner.

Ian Jackon proposed a general resolution named: init system coupling in October 2014 which tried to:

This GR seeks to preserve the freedom of our users now to select an init system of their choice, and the project’s freedom to select a different init system in the future. It will avoid Debian becoming accidentally locked in to a particular init system (for example, because so much unrelated software has ended up depending on a particular init system that the burden of effort required to change init system becomes too great). A number of init systems exist, and it is clear that there is not yet broad consensus as to what the best init system might look like. This GR does not make any comment on the relative merits of different init systems; the technical committee has decided upon the default init system forLinux for jessie.

The general resolution was not approved in November 2014.

Problem about systemd controversy

Systemd controversy as I understand it means discussing about systemd without understanding it once and once again. There are many articles about debunking systemd myths so I will not enter into details. You probably know that systemd is not an init per se. One of the systemd components is init yes.

Keeping aside the possible dependencies problem (Gnome pulling out systemd which I think it’s no longer true) what raises my attention, as a sysadmin, is that people are arguing that updating from sysv to systemd leaves their system unbootable.

That’s a pity because Debian is supposed to avoid these problems because it’s stable. These problems should be reported so that next stable Debian Jessie release fixes them if needed. That also means that you cannot draw systemd conclusions from using Debian testing without checking if it’s actually a bug or not. The other concern is that this admin is not worth of his name if he does not know how to get back to the machine previous state (thinking of virtual machine snapshots here). And, also, it’s not worth of his name if he does not learn about systemd and its specific issues.

So, there is an explanation on an Slashdot comment by Tollef Fog Heen which explains a practical view on systemd:

 

It’s a new system, so some things work differently. Many people seem to fail to see the line between bugs and intentional behaviour. If something doesn’t work as before, it might not be because we’re evil bastards who are out to steal your logs. It might just be that there’s a bug in some package which means your logs aren’t correctly forwarded from the journal.

Sometimes it might be that systemd makes other assumptions about what something means and we’re just failing to catch that in an upgrade check that should warn you about this. An example here is missing devices/mount points in fstab: sysvinit will happily ignore them, systemd won’t consider local-fs.target reached and you’ll have to fix your system for it to boot correctly.

Assumptions, as so often before, are the mother of all fuckups. Asking (preferably in a civilized manner) will get you a long way: «Hey, I’m not seeing my logs appear in syslog, is this supposed to be that way, and if not, can you help debug?»

This might not be the greatest misconception, but I think it’s the most common one. The greatest is possibly the conspiracy theory that this is all a takeover attempt from RH to kill other Linux distributions and that people pushing systemd are either shills or just unwittingly working against the distro they’re pushing systemd into. I’m not sure how adopting a free software component (which sure, happens to be largely maintained by RH engineers, like many other free software components we use to build a distro) will turn us all into corporate-loving robots giving up freedom to be near the source of systemd.

 

You can also read the Tollef Fog Heen summary on Systemd and Debian which it’s pretty nice.

systemd controversy among Debian is more than one year old at Debian and keep arising once and once again. Some (pro or con systemd) Debian Developers have been upset about some many discussion and have resigned in several of their tasks.

systemd developers also repeatedly suffer the anger of final users which do not like Systemd. If you do not like Systemd do not use it. No one is forcing you to use it. Not even Gnome and if it’s the case fork Gnome but do not complain on systemd people.

What I want to say is that you own the right for complaining but please do complain but only after you have read and tested all you can about systemd and about how Debian works.

Debian lack of resources

If you take a look at Debconf videos you will see that many projects lack manpower and that help is always welcome. Yes, Debian has a lot of users, and a lot of developers but most of them do not get paid for their work. So don’t assume Debian has manpower for meeting your expectations. More than that when people from Debian are asked about Debian Org paying them there’s concern because it goes against Debian spirit.

Another thing Debian is rejecting code that it’s not easy maintainable or similar. You cannot assume that Debian is going to accept everything you request to.

Debian way

Debian CD should let me opt for sysv

The newest Debian Jessie installer will not have an option for letting you not to install systemd. If you want Debian CD to let you install a systemd free system and presumably with sysv as the default init you need to program yourself it.

Then please contribute back your patch to the debian-cd and debian installer packages as needed.

As you have waited so long at helping Debian people to implement it you will notice that it won’t do it in Jessie. Not in the first release at least.

Too many packages depending on systemd

If you are concerned about so many packages depending on systemd please patch these packages either on upstream or in Debian packages so that they are no so dependent on systemd (There is an init metapackage). Then submit back the patch the Debian

What if my patches do not get accepted

Then fork Debian but not before my previous suggestions are tried. Or, well, if you cannot wait for your changes to be approved please fork but, keep in mind, that your best move is to contribute back to Debian.

Pragmatic steps for avoiding systemd

Just take a look at: Without systemd debian – jessie page. Even if you decide to upgrade to Jessie you can then choose to remove systemd and change your system behaviour so that it makes impossible for systemd to be installed.

First install a good init system

# apt-get install sysvinit-core sysvinit sysvinit-utils

Then reboot your machine and remove all the systemd crap

# apt-get remove --purge --auto-remove systemd

To keep systemd away from your system you should prevent the package from being installed again.

# echo -e "Package: systemd\nPin: origin ""\nPin-Priority: -1" > /etc/apt/preferences.d/systemd

There is also an attempt to release a Debian Jessie installer without systemd but it’s not an official one (as any fork).

What do I think about systemd

As a sysadmin I’m interested in being able to use cgroups so that in a multi web hosting like Virtualmin you can assign resource usage limits to the different webpages. I’m also interested in the declarative ways of setting a system. It seems smarter.

As a Rescatux and Super Grub2 Disk is a new field of opportunities because I can just add an option for bypassing systemd and using sysv at boot (if sysv was not uninstalled) if the former ever fails. But not sure if I will ever implement it.

Conclusions

As per many other open source and free as freedom software projects what really counts is the source code. It’s what your contributions are, not what you think a project should do. So, please, the next time you complain about Debian including systemd by default (only in Linux kernel) just step aside to think carefully about what you can do for Debian and, even, for systemd.

Super Kexec Disk mockup

Yet another mockup.

Kexec let’s you boot into another Linux kernel from a live or running Linux kernel. So that makes things interesting because you can use a live kernel from a live cd in order to boot into another one but, just before it, you have a full GNU/Linux with their tools.

I had already known about kexec but I wasn’t aware that it was so useful. I thought the main purpose would be to update your kernel without having to reboot your machine.

After writing my Kernel box disk idea article I discussed it in Super Grub irc channel. Bfree and Smx explained me about Kexec and what you could do with it.

I haven’t used kexec myself yet and I don’t fully understand how an initrd is built depending on kernel modules you need on boot to mount root’s partition. So, some ideas you see here might be not right.

 

So Super Kexec App scans all your installed distributions thanks to the great variety of modules included in the Super Kexec Disk live cd. You are prompted to choose to the distribution you want to boot into.

Super Kexec Disk mockup – Distros Detected

Once you have selected which distribution you want to boot you are given the opportunity to choose which modules should have your initrd (will be created on the fly for being your kernel companion). Super Kexec app suggests you initially which modules you will need so that the Kernel knows how to mount its root partition from it.

Super Kexec Disk mockup – Modules detected

Once you have decided which modules will needed to be loaded you can fine-tune the kernel boot parametres. Super Kexec App will have selected the current Super Kexec Disk boot options for you. If you already forced Super Kexec Disk to boot into nomodeset it’s highly probable that you want your booted system to have that option too. From this screen you will also be able to blacklist kernel modules (just in case some of them hang your system on boot) or force an specific systemd target (runlevel).

Super Kexec Disk mockup – Boot parametres

Finally a summary about what you are going to do is prompted and you can:

• Boot into your Linux Kernel + Generated Initrd
• Review the settings if you made a mistake
• Exit from the Super Kexec App

Super Kexec Disk mockup – Summary

Finally there should be another option named: Force this boot into the distribution so that:

• Initrd is copied into /boot directory
• /etc/grub.d/08_super_kexec is created
• 08_super_kexec defines an entry to boot your kernel and the generated initrd with the choosen boot parametres

and somehow it should guess that Grub2 might need to be reinstalled into the MBR with enforced LVM+RAID capabilites if needed.

So that’s it, once you have a system that can modify another and at the same you can boot a system from it it sounds a bit like Rescatux and a bit like Super Grub2 Disk. Not everyday we see a system like this.

Am I going to implement Super Kexec App as a Rescatux application (instead of doing an only Super Kexec Disk as this mockup describes)?

Don’t think so. Making a Rescatux option to rebuild an initrd with several modules from your chroot and then add some Grub2 entries to boot into it. Yes, that make sense when you somehow manage to remove LVM support from your initrd while your root partition is in LVM. (Or you copy-and-paste a Distribution from Ext4 to LVM).

Not in the short term I will implement it.

If you think about any useful use case or task that might make worth implementing these option please drop a comment so that I am aware of it.

Kernel boot parametres option for Super Grub2 Disk mockup

Yet another mockup :).

So once you have selected which Kernel you want to from you can choose between different set of Kernel boot parametres:

• Knoppix cheatcodes. Just in case you want to boot your installed Knoppix with those options.
  

  super grub2 disk boot parametres mockup based on knoppix boot cheatcodes

• Blacklist modules. This is very useful when trying to debug which module is making your system to crash or reboot at boot time.
  

  SG2D mockup – Blacklist modules

• Systemd target. Choose an specific systemd ‘runlevel’.
  

  SG2D mockup Systemd targets

• Miscelanea. Any other Kernel boot parametre.
  

  SG2D Kernel boot parametres mockup miscelanea

• Choose root partition. Just in case you want to try to boot your Debian GNU/Linux but trying to use Ubuntu kernel. Usually it might not work 100%, but it’s worth a try in some special circumstances.
  

  SG2D Boot parametre Mockup – Choose root partition

I’m not interested in implementing it because I think not so many few people would use when trying to boot into their distro from Super Grub2 Disk. Maybe the blacklist option is the most useful one.

But it’s something I would maintain if I ever implemented it. Not like the Kernel box idea.

Do you use often use an specific boot parametre? Why?

Kernel box disk idea

This is a thing I’m not going to implement because although it’s very easy from technical point of view having to take care of all the kernel versions and their (so big) source code would easily become a burden.

So, this is an idea or mockup.

What is Kernel box?

An special Super Grub2 Disk which includes several Linux kernels and initrds.

Super Grub2 Disk – Kernel box – Mockup

What can you do with Kernel box?

• Try a new kernel without installing it.
• Being able to boot your distribution because you did not install a kernel when installing it.
• Being able to boot your distribution after realising that you have deleted your /boot partition where not only the grub files were but also the kernel files.

How would it work?

I guess I would detect a Linux root partition by finding /sbin/init (or whatever it’s named in systemd) and probably by checking /etc/issue for a possible name for the distro.

Once the user chooses the distro which he wants to start (not choosing boot verb here on purpose :)) he is prompted about the kernel + initrd that wants to try.

Available initrd

There should be the original kernel and initrd from the most popular GNU/Linux distributions out there available. There should be special initrds for these special cases (RAID5 + LVM and the like).

Advanced mode

I’m not sure if it makes sense technically but you could use a kernel from SuSE 11 with Ubuntu 13.10’s initrd.

Boot parametres improvement

You can add most usual boot parametres from a list. Either you can exclude a module to load, you can turn of acpi or another special boot parametre for your video card. I will write probably write about this idea in another post because it was originally intented for normal Super Grub2 Disk.

About implementing it

I don’t mind spending an afternoon and coding the Super Grub2 Disk part if someone devotes himself on building up the rest of the system:

• Code an script to obtain new kernels and initrds semiautomatically and update Super Grub2 Disk scripts accordingly.
• Keep an updated version monthly
• And most importantly keeping the Kernel and Initrd source code organised so that anyone can download it so that we complain to GPL.
• Not sure if it’s needed. Automate the building of the different kernels from their source code to even ensure we comply GPL.

Feedback

• Any thoughts on the idea?
• Any other tricks around what Super Grub2 Disk does that could improve it like this one?