Friday, 2 June 2017

New functionality and simplification of 'isorespin.sh'

UpdateThis work is superseded by my 'isorespin.sh' script which can respin an official ISO suitable for use on Intel Atom devices.


I've updated my 'isorespin.sh' script to include new functionality and simplified its invocation. Besides uploading files to the ISO I've included the ability to upload directories and conversely I've also added the ability to output both files and directories. The other key functionality I've added is the ability to run commands as part of building the ISO and the significance of this is explained in an example below.


The GUI menu is now simplified through adding a sub-menu for the more advanced options that may not be so frequently used. Also adding persistence no longer requires specifying a boot manager. Finally I recommend installing the script by copying it to '/usr/local/bin' as this makes it readily accessible.

Although more detailed documentation is covered by earlier posts as a reminder you can respin all recognized desktop 64-bit Ubuntu and Ubuntu flavoured ISOs (https://www.ubuntu.com/download/ubuntu-flavours), Linux Mint and LDME ISOs (https://www.linuxmint.com/download.php), Debian 'Live install' ISOs (https://www.debian.org/CD/live), KDE neon ISOs (https://neon.kde.org/download), elementary OS ISOs (https://elementary.io) and Kali ISOs (https://www.kali.org/downloads) and in doing so will automatically add a 32-bit bootloader as well as the 64-bit bootloader making it possible to boot the respun ISO on Intel Atom devices.

The basic requirements are unchanged in that you will need to use a Linux machine (or a Linux virtual machine on Windows) with packages like 'squashfs-tools' and 'xorriso' installed (the script will check what is required and notify you if anything is missing) together with 'zenity' if you want the GUI. You will also need at least 10GB of free space and a working internet connection if you want to update the kernel. And on that note the '-u' or '--update' flag will update an Ubuntu or Linux Mint ISO with the latest mainline kernel but when used with a Debian or LMDE ISO it will update the kernel with the Canonical DKMS patches for HDMI audio and RTL8723BS wifi. As a result the '-k' or '--kernel' option is only allowed for Ubuntu and Linux Mint ISOs and any of the Ubuntu Kernel Team kernel builds found at http://kernel.ubuntu.com/~kernel-ppa/mainline can be specified by using the directory/folder name without the trailing '/' (as in '--kernel v4.12-rc3'). Also if selecting the rEFInd bootmanager (http://www.rodsbooks.com/refind) and it is in the directory from where the script is run it won't be downloaded again from https://sourceforge.net/projects/refind/files but the local copy used instead regardless of its version. And once the ISO has been respun it can be written to a USB using the standard 'dd' command.

For anyone brave enough to try an installation of the respun ISO you may encounter the usual issues with installing a bootloader. First try installing with a working internet connection as this resolves many of the problems. However if is still doesn't work I've written an additional (updated) script 'linuxium-install-bootloader.sh' that you can try running or use the commands within the script as a guide to allow you to install manually. Note that running the script still requires an internet connection and the target Linux partition needs to be already mounted on '/target' with the target boot partition mounted on '/target/boot/efi'.

Currently even after respinning your ISO you may find that some wifi/bluetooth driver and userspace files are still required to get them working. Previously I've provided additional scripts and packages to support RTL8723BS and various Broadcom chipsets as well as the ALSA UCM files for some audio chipsets used by headphones. And whilst they can be included as files when respinning the ISO they still need to be run on booting the ISO as a 'LiveCD' to activate the additional functionality. This new version of the respin script includes the 'command' option to address this which and it takes its argument and runs it as a 'bash' command.


Simple commands are straight forward. However installing the UCM files by just uploading and running the script is slightly more complex. This is because the script requires the internet and therefore a valid host name and the script must be run in the same directory it resides in. Also any quotes used within command arguments need delimiting.


In more complex cases it is simpler to write a shell script to execute the commands required. An additional command can always be run to remove the 'wrapper' script if desired. See my separate post for an additional example of the power of this new functionality.

The new version of the script can be downloaded from 'isorespin.sh' and any issues or improvement suggestions are welcome.

If you find my work useful then please donate using the following link http://goo.gl/nXWSGf as everything helps with development costs.

27 comments:

  1. Issue detected? --> I tried the new version with Lubuntu and LinuxMint-Mate (1st with, 2nd with and without kernel update). Boot-Menue appears but booting comes to a complete stop shortly afterwards. Device is a bay-trail no-name tablet on which distributions done with earlier isorespin-versions did boot.

    ReplyDelete
    Replies
    1. V4.12-rc3 contains a regression bug which prevents booting on BYT and CHT devices so use '-k v4.12-rc2' or wait until Sunday/Monday and try rc4.

      Delete
  2. Ups, I guess I should have done some more reading first... Thanks for your reply; it boots now. Let's see if I can get it installed as well.

    ReplyDelete
  3. It's me again...
    Sorry, maybe I lack some basic skills here. The system installed well until I reached the point with grub. After crash of the installer I tried your script, but it does also not run through. Any advice would be great!

    ..sudo mount /dev/mmcblk1p2 /target/
    ..sudo mount /dev/mmcblk1p1 /target/boot/efi/

    mint@mint ~/Downloads $ sudo ./linuxium-install-bootloader.sh
    Ign:1 cdrom://Linux Mint 18.1 _Serena_ - Release amd64 20161213 xenial InRelease
    Err:2 cdrom://Linux Mint 18.1 _Serena_ - Release amd64 20161213 xenial Release
    Please use apt-cdrom to make this CD-ROM recognized by APT. apt-get update cannot be used to add new CD-ROMs
    Hit:3 http://archive.ubuntu.com/ubuntu xenial InRelease
    Hit:4 http://archive.canonical.com/ubuntu xenial InRelease
    Get:5 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
    Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
    Ign:7 http://packages.linuxmint.com serena InRelease
    Hit:8 http://packages.linuxmint.com serena Release
    Get:10 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
    Reading package lists... Done
    E: The repository 'cdrom://Linux Mint 18.1 _Serena_ - Release amd64 20161213 xenial Release' does not have a Release file.
    N: Updating from such a repository can't be done securely, and is therefore disabled by default.
    N: See apt-secure(8) manpage for repository creation and user configuration details.
    ./linuxium-install-bootloader.sh: 42: [: 32: unexpected operator
    grub-install.real: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist. Please specify --target or --directory.
    /usr/sbin/grub-mkconfig: 252: /usr/sbin/grub-mkconfig: cannot create /boot/grub/grub.cfg.new: Directory nonexistent
    mint@mint ~/Downloads $

    ReplyDelete
    Replies
    1. First did you try performing the installation whilst connected to the internet as this fixes most situations?

      Given the installation has finished and assuming you have done nothing since, try booting again from the LiveUSB and enter these command:

      sudo mount /dev/mmcblk1p2 /target/
      sudo mount /dev/mmcblk1p1 /target/boot/efi/
      sudo chroot /target mount -t proc proc /proc
      sudo chroot /target mount -t sysfs sysfs /sys
      sudo mount --bind /dev /target/dev
      sudo mount --bind /run /target/run
      sudo chroot /target apt update
      sudo chroot /target apt -y purge grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed
      sudo chroot /target apt -y purge shim shim-signed
      sudo chroot /target apt -y purge grub-efi-ia32 grub-efi-ia32-bin
      sudo chroot /target apt -y purge grub-common grub2-common
      sudo chroot /target rm -rf /boot/grub/
      sudo chroot /target rm -rf /boot/efi/EFI/ubuntu/
      sudo chroot /target apt -y install grub-common grub2-common
      sudo chroot /target apt -y install grub2-common grub-efi-ia32 grub-efi-ia32-bin
      sudo chroot /target grub-install --target=i386-efi ${BOOTLOAD_PARTITION} --efi-directory=/boot/efi/ --boot-directory=/boot/
      sudo chroot /target grub-mkconfig -o /boot/grub/grub.cfg
      sudo chroot /target umount /sys
      sudo chroot /target umount /proc
      sudo umount /target/run
      sudo umount /target/dev


      You will need a working internet connection for this to work. If there are no errors, reboot and see what happens (as you may then need to fix up NVRAM). Otherwise please post any error encountered.

      Delete
  4. Hi there,

    Worked Fine!
    - I did had an internet connection on the 1st try
    - On your script, only where you proposed an "apt", I needed to switch to "apt-get", as apt returned only an info about how to use that command...

    --> Thank you very much!!!

    ReplyDelete
    Replies
    1. Thanks. I'll update the 'linuxium-install-bootloader.sh' script with this feedback and post for the benefit of others.

      Delete
  5. Thank you for tour tool. I am having one problem, fetching rEFInd bootmanager is giving error, i am conected to the internet and using the isorespin gui

    ReplyDelete
    Replies
    1. Please provide full details about the error so I can try and reproduce and investigate.

      Delete
  6. I think i've found a bug in the gui and also I'm struggling to produce a valid (recognisable) iso

    Can't fit in 4096 characters
    https://pastebin.com/qJ3DmP1b

    ReplyDelete
    Replies
    1. Thanks. Yes you have found a bug and your error report is excellent and has really helped in fixing the issue. I've posted an updated version (v6.03.0). I've also added additional checking that should help with the 'cp' issue you encountered which I think happens when a download fails (for whatever reason) and is typically unreproduceable as it disappears when the script is re-run.

      Delete
    2. Ok thanks. I have it working now.
      BTW your script can run on Arch Linux if I modify the PATH.
      PATH=/usr/sbin:/sbin:/bin:$PATH ./isorespin.sh
      Without that the many chroots called in the script weren't working as Arch's path is missing a lot of Ubuntu's entries.

      It would be nice if you could put this all on a git repo or similar. ATM the instructions (some conflicting) are spread around your blog pages).

      But anyway good work and I've sent some beer tokens

      Delete
    3. Thanks. I'll write an updated guide with the latest links. Actually my blog is intended to be the 'dynamic' update content partner to my 'static' core content pages on my website Typically I publish my development on my blog then once released and stable I published on my website. This approach is meant to address any conflicting information and centralize it in once place.

      Delete
  7. Hi Linuxium, I have the Lenovo Ideapad 100S 11IBY (Intel Atom). I installed Lubuntu using a bootable USB that I made using your patched kernel iso and it ran perfectly. However, I installed xfce4 desktop environment from within terminal after which the laptop failed to boot. On the boot options menu I get two options (1) "Ubuntu" and (2) "Ubuntu", yes they are both same words. The laptop fails to boot using any of these two options. I tried to boot using more linuxium bootable usbs but they all failed to boot. The laptop is unusable now. Can you please help? Some people suggested that because I updated the lubuntu, the grub might have been replaced new grub which is causing the problem. Any help will be appreciated.

    ReplyDelete
    Replies
    1. Just use 'efibootmgr' from a LiveUSB to remove both of them and then use it to recreate a single entry.

      Delete
    2. My laptop Lenovo Ideapad 100S 11iby has ceased to boot from even UEFI bootable USBs. This happened after I installed an extra desktop environment on existing linuxium installed Lubuntu. Do you have any idea of how to make a USB boot in such a situation. Perhaps some BIOS setting or some way to access the storage without booting?

      Delete
    3. Assuming your laptop's eMMC (storage as in your disk drive) has corrupted try downloading 'gparted-live-0.28.1-1-amd64.iso' from 'http://gparted.org/download.php' and write to a USB with 'dd'. Then see if you can boot from it and if so use 'gparted' to wipe your eMMC by creating a new partition table 'Device->Create Partition Table->msdos->Apply'. You should then be able to boot from an Ubuntu ISO USB.

      Delete
    4. I made a gparted live USB with 'dd' and went to 'EFI USB' in the boot menu but it just says 'boot device missing'. I tried with both secure boot 'on' and 'off'. I also tried with:
      (1) Setup Mode,
      (2) Restore all factory keys,
      (3) Load default settings.
      Neither of these combinations work. I am wondering how is this even possible. Is there some particular UEFI setting I need to setup? Or is there a way a access the storage using a USB cable or something like that. I know that eMMC is soldered onto motherboard.

      Delete
    5. One more thing, I cloned my entire eMMC onto an HDD before installling ubuntu on my laptop. Do you think it is possible to boot from the HDD via a SATA-to-USB cable? (It is proper clone made using CloneZilla.)

      Delete
    6. Try this 'https://forums.lenovo.com/t5/Lenovo-IdeaPad-100-305-500-Edge/Ideapad-100S-bricked-by-Windows-update-Solved/m-p/3377245#M38858'

      Delete
    7. Hi, that link has an eloborate BIOS but my laptop does not (Ideapad 100S 11IBY). I tried making GPT only bootable USB using your respun iso "linuxium-atom-v4.13.0-9-ubuntu-17.10-10SEP17-desktop-amd64" but the laptop refuses to identify the EFI boot USB. It keeps going into a loop "ubuntu boot faile" followed by,
      "Default boot deviceMissing or boot failed. Insert recovery media and hit any key, Then select Boot Manager to choose a new Boot Device or to Boot Recovery Media."

      Delete
    8. Please, I will try anything at this point. The laptop is unusable.

      Delete
    9. I suggest reflashing the BIOS as per the manufacturer's recommended method.

      Delete
    10. Is it possible without booting an OS, I mean directly from the bios?

      Delete
    11. It will depend on your device manufacturer but it is typically possible. Otherwise you can flash the BIOS chip with an EEPROM programmer.

      Delete
  8. so Ubuntu respin works via USB no worries, it seemed to install OK... but on reboot obviously its failing with GRUB on my Apollo Lake Ezybook 3 Pro. I need ReFIND... the section has been removed from the tutorial? I tried efibootmgr and removed the previous windows entry, but it still fails with nothing but a black screen and white cursor in top left. Any tips on how to get refind installed on there so I can actually boot into the OS without a usb stick connected? none of the information seems to be helping anywhere I've looked so far

    ReplyDelete
    Replies
    1. The 'rEFInd' link above includes documentation on installation (see 'http://www.rodsbooks.com/refind/installing.html') so I didn't add anything else to the tutorial. Do you think there is a need for additional instructions?

      Delete