Friday, 24 November 2017

Running and installing Ubuntu and Ubuntu flavours on Apollo Lake devices


Some Intel Apollo Lake devices have an option in the BIOS that allows the selection of a bootable OS. Unfortunately on a lot of mini PCs this option is either not present or doesn't include 'Linux' which is required to boot official Ubuntu and Ubuntu flavoured ISOs. Previously I addressed this through adding an option to my 'isorespin.sh' script allowing the ISO to be respun with the rEFInd boot manager to enabled booting.

Whilst investigating why GRUB fails to boot I've developed an alternative approach which I have incorporated into a new release of my 'isorespin.sh' script. Now when respinning with the '--apollo' option the respun ISO replaces the GRUB EFI files with a single EFI file that corresponds to the entry in NVRAM created by the GRUB installation. This EFI file boots the device directly to Linux similar to how Windows boots.


It uses a script I've written that hooks into the 'initramfs' tools to create a bootable EFI file every time 'update-initramfs' is executed. This means if a new kernel is installed the EFI file is automatically updated to boot the new kernel.

To respin an ISO suitable for Intel Apollo Lake devices first download my latest version of 'isorespin.sh' and then respin using the '--apollo' option.

If you have multiple kernels installed it is still possible to boot from a specific kernel simply by rebuilding the 'initramfs' file for that kernel. So for example if you are currently running the 4.10.0-40-generic kernel and want to revert to the 4.10.0-19-generic then just run 'update-initramfs -u -k 4.10.0-19-generic' and reboot.


One issue that might be encountered is where the EFI partition isn't large enough to store the bootable EFI file. Some Windows installations only create a 60MB partition which after installing the Windows EFI files leaves only around 30MB. With the later Linux kernel releases the size and number of modules have increased and if the 'linux-image-extra' package is installed then the corresponding 'initrd.img' is now quite large (over 200MB). Recent kernels only have the minimum set of kernel modules in the 'linux-image' package with the remaining ones in the 'linux-image-extra' package. Whilst in the past installing just the 'linux-image' package was sufficient when upgrading to a new kernel it is now recommended to install both 'linux-image' and 'linux-image-extra' packages.

As a result I've written an additional script that will shrink the size of '/lib/modules' by removing all unused modules from the combination of the 'linux-image' and 'linux-image-extra' packages for a specified kernel. Simply download the script 'update-modules' and pass the kernel version you want to shrink as the parameter.

For example if you want to shrink the overall size of '/lib/modules/4.10.0-19-generic' then run the command 'update-modules 4.10.0-19-generic'. The script will also automatically regenerates the 'initramfs' file and create the bootable EFI file. So if you are running with a small EFI partition and installing a new kernel fails to create the bootable EFI file then running 'update-modules' will fix this. Note that the script requires an internet connection to temporarily download and examine the modules in the 'linux-image-extra' package.


If having removed unused modules a new peripheral requires a missing driver (i.e. module) simply reinstall the 'linux-image-extra' package. For example connecting a USB Ethernet adaptor for the first time when running a previously reduced 4.10.0-19 kernel does not get recognised. Reinstalling all the drivers/modules using the command 'apt install --reinstall linux-image-extra-4.10.0-19-generic' and the adaptor can now connect. 


The remaining unused modules can then be removed by re-running the 'update-modules' script if space is a consideration.


It is also possible to respin a bootable ISO for Intel Apollo Lake devices by using the '-b Linuxium' option which is actually what the '--apollo' option now does. Adding the 'Linuxium' bootscript (as implied by the '--apollo' option) does not actually restrict the ISO to boot on a specific processor however it does depend on the boot loader used by the device. An ISO respun with the  '-b Linuxium' option will work on any Intel Atom and Apollo Lake devices if they are using a 64-bit boot loader which is the case for Apollo Lake devices. Also note that persistence cannot be added to respinning an ISO in conjunction with the '--apollo' or '-b Linuxium' option. This is because persistence adds the rEFInd boot manager which incidentally also boots on both Intel Atom and Apollo Lake devices using a 64-bit boot loader.

As an example I've respun the Ubuntu 17.04 ISO using the '--apollo' option and included the 'update-modules' script which is available under '/usr/local/bin' and created an ISO suitable for booting on Intel Apollo-based mini PCs.

The ISO can be downloaded from:

Apollo (-i ubuntu-17.04-desktop-amd64.iso --apollo -f update-modules)

Anyone wanting to spin their own can download an ISO and respin using my 'isorespin.sh' script with the above options or after consulting my documentation.

Please donate if you find the ISO or scripts useful using the following link http://goo.gl/nXWSGf.


59 comments:

  1. Hi, I tried to respin the iso, with new isorespin.sh but got this instead
    https://imgur.com/MwoLtrp
    any suggestions?

    ReplyDelete
    Replies
    1. Thanks for reporting this issue. I've fixed this by correcting the support for Ubuntu flavours and released version 8.0.1 of my script 'isorespin.sh'. The download links are the same so all you need to do is download the script again and respinning using the same command.

      Delete
    2. Thank you, you are amazing, it worked, I did respin iso ok, but when I boot elementary os to my laptop (jumper ez book 3 pro) and I want to install it, it don't let me, I am stuck in try version of elementary, I will probably give up soon, but still trying to find solution for this

      Delete
    3. Can you provide more details about the error you encounter during installation? Is it an 'elementary OS' issue or specifically related to the respun ISO?

      Delete
    4. I would like to give you more info, but there seems no errors, just installation will not start automatically, also think that is related to respun of elementary or the type of laptop, I tried that usb stick (same respuned iso) on my old laptop Dell Inspiron, and it worked completely fine, dunno how to get you more info, if there is some script I can run in terminal or something like that I am open to do it, but as far as linu goes I am pretty beginner guy so far

      Delete
    5. What is the download URL for the elementary ISO and what options did you use when respinning it and I'll have a look.

      Delete
    6. downloaded from https://elementary.io (https://fra1.dl.elementary.io/download/MTUxMTcwNTk0Mg==/elementaryos-0.4.1-stable.20170814.iso)
      and i used:
      ./isorespin -i < elementary.iso > -u --apollo -f update-modules

      Delete
    7. Okay I'll run the same command and also try an installation.

      Delete
    8. Also I tried to respun ubuntu gnome, respuning went fine, but on booting phase from usb stick I got this
      https://imgur.com/jKfHFHW
      maybe it will give you more info

      Delete
    9. Regarding the installation of elementary os you are probably running out of space on your EFI partition. Run the following commands from the terminal on the elementary os LiveUSB before starting the installation:

      sudo mount $(blkid | grep mmcblk | grep vfat | sed 's/:.*//') /mnt
      sudo rm -f /mnt/EFI/ubuntu/*
      sudo umount /mnt

      as this will clear out old EFI files ready for the installation.

      Regarding the respin Ubuntu GNOME I ran 'isorespin.sh -i ubuntu-gnome-16.04.3-desktop-amd64.iso --apollo -k v4.14' which worked fine (note: using '-u' option will now install the 'v4.15-rc1' kernel so I used the '-k' option to hopefully replicate what you did). I suggest you first try rebooting using your USB to see if the problem still exists otherwise try respinning and recreating your USB again as it should work.

      Delete
  2. I keep replying to you on the 'customing' section, but the replies never appear. So I'm trying again here.

    I've done this, as you advised, but the resultant USB won't boot. Neither will another stick which worked previously. Is this an EFI space thing do you think?

    isorespin.sh -i ubuntu-mate-17.04-desktop-amd64.iso -u --atom

    ReplyDelete
    Replies
    1. You need to click on 'Load more...' at the bottom of the page (several times on some pages) to display all the comments.

      I've just uploaded a revised version of my script that should fix you boot issue. You will need to download my 'isorespin.sh' script again (see top of page for download link) and respin your ISO.

      Delete
    2. Thanks, Linuxium. I always click on 'Load More'. Doesn't help.
      I'll try downloading your new script.
      Thanks again.

      Delete
    3. Still will not boot.
      BIOS/EFI screen appears; select boot from usb; boots existing installed system. Same as before.

      Delete
    4. Can you boot and post the 'dmesg' using pasteinit or similar?

      Delete
    5. Thanks. It's at https://pastebin.com/zCqHJ7xn

      Delete
    6. Can you post the 'dmesg' after booting from the respun ISO?

      Delete
    7. As I said, it won't boot. The existing system that that dmesg is from was installed from a respun ISO. I cannot boot the new one. The new one was generated from your latest version . . .

      isorespin.sh -i ubuntu-mate-17.04-desktop-amd64.iso -u --atom

      Delete
    8. When you said "select boot from usb; boots existing installed system" can you provide the 'dmesg' from that type of boot so I can compare with the previous 'dmesg'?

      Delete
    9. That IS the dmesg. Maybe I wasn't clear . . .

      I turned the machine on and pressed F2 to enter the BIOS.
      Selected the Boot Device as the USB stick (which was in place).
      It booted from the SSD.

      That's what I've done with every other respun ISO I've used, but they've always booted before.

      You're very patient. Thanks again.

      Delete
    10. When you select the USB to boot do you see a GRUB menu next? If not what specific options do you get for 'Boot Device' after you have entered the BIOS by pressing F2?

      Delete
    11. No GRUB menu.
      Boot option #1 [ubuntu]
      Boot option #2 [UEFI generic flash]

      Delete
    12. As option #1 will boot your installed Ubuntu does option #2 boot your USB?

      Delete
    13. No. That's exactly the problem. It boots the installed Ubuntu no matter which one I pick and save. I've tried many times.
      If only I could get the sound working, this would be an ideal netbook. (Mostly for Skype & YouTube.)

      Delete
    14. Just checking -- is VERSION="8.0.2" the latest one?

      Delete
  3. This is off-topic to apollo, but I thought I'd ask here anyway. We are probably about two months from release of V4.15, as the rc1 came out yesterday. I was wondering what the means for respinning. Does anyone have any insight as to what will still be required as extra packages, what has been integrated into the kernel, etc?

    ReplyDelete
  4. To be more specific, if I was to respin a v4.15-rc1 image, should I change any linuxium packages that are typically included in the respin?

    ReplyDelete
    Replies
    1. The Canonical pre-compiled kernel packages (header and image) are not ready for respinning yet as there were several config changes and they haven't been reflected in the built images (the config is based on v4.14).

      Delete
    2. Thanks for the heads-up. I'll stick with 4.14 for now.

      Delete
  5. Thank you for all your work! I have a obook 11 plus and the bluetooth wifi don't work even with the nvram file also the acpi volume buttons don't work do you have any insight to this? also with the silead touch driver and the es8316 also i cannot seem to get the bosch sensor to work
    paste bin http://paste.ubuntu.com/26067113/

    ReplyDelete
    Replies
    1. Can you try respinning again as I've updated the UCM files to include those for the ES8316 codec and this hopefully will fix your audio.

      Most likely the bluetooth device ID will need adding to the kernel before it works so you can either compile your own or hope/wait for the relevant patch to come along.

      As the sensor is device specific have you asked on forums dedicated to your device?

      Delete
  6. Also can you add bodhi to the supported distro

    ReplyDelete
    Replies
    1. The Bodhi ISO is structured differently meaning my script requires changes. Only if there was sufficient interest would I add support for Bodhi.

      Delete
  7. Hello. I use isorespin with --apoll and kde neon iso. it was established normally. But during the load system system not start, only (initramfs).

    ReplyDelete
    Replies
    1. Fixed. neon created it's own NVRAM entry. You will have to download my script 'isorespin.sh' again and respin your ISO.

      Delete
    2. did not help. again (initramfs).

      Delete
    3. If you can provide more information like device details and a 'dmesg' I can take a look at it as currently it works on my test devices.

      Delete
  8. Is there any way to easily adapt the script for Ubuntu Server or Debian? I want to replace my Raspberry PI3 with a Beelink Mini PC with an SSD and I don't want all the UI stuff of Ubuntu Desktop. Unfortunately, the script doesn't work on the Ubuntu Server/Debian ISOs it seems.

    ReplyDelete
    Replies
    1. My 'isorespin.sh' script is designed specifically around desktop ISOs. You can always remove the desktop once installed if you want to create a server environment.

      Delete
  9. Hello Mr. Linuxium. Thanks for your great work!

    I have trouble with isorespin v. 8.0.2 using "./isorespin.sh --apollo --iso lubuntu-17.10-desktop-amd64.iso --kernel v4.14" for a Beelink AP34. Respinning works well and I can burn it to a USB flash drive. I can also boot from the USB flash drive and the system seems to work well.

    However, if I try to install the Ubuntu, it does not boot. I have also tried with rEFInd, which did not work. If I install Ubuntu using the installer, does it also install your custom bootloader? Or do I need to take a different approach?

    btw. why don't you support Ubuntu Server and Debian ISOs? At least Ubuntu Server would make sense for me and should not be an effort!

    Thanks!

    ReplyDelete
    Replies
    1. You probably don't have enough space on your UEFI partition. Try mounting it from a LiveUSB and cleaning out all the unwanted bootloader/bootmanager files so that you have around 52GB free.

      The build for server ISOs is different and the demand for them isn't there. Debian ISOs need a Debian kernel not an Ubuntu one so again more work when there isn't the demand.

      Delete
  10. Hi team,
    I'm a newbie in apollo devices. (in that case ACEPC AK1)
    Let me to explain easily what i do

    1) I can download your image from https://goo.gl/CH2YKr , installed in a USB (by "dd" directly in other linux), and booted it selecting the USB in the EFI "bios"
    2) i can see the installer and install linux into the SSD new device
    3) i CAN'T BOOT linux, in any way. Always starts in Windows

    What I did?

    FIRST ATTEMPT: adding manually "ubuntu" to the windows boot loader, with no success, windows boot loader ignores it.

    SECOND ATTEMPT: Installing refind . Let it to do the magic and re-boot. Now, i can see the new boot manager with TWO options, "Windows" and "ubuntu". Windows boots ok, but ubuntu get in black screen.

    THIRD ATTEMPT: Booting the live rescue ubuntu and using the boot-repair tool (https://help.ubuntu.com/community/Boot-Repair). This tool says that "boot/efi is not mounted". That's ok , i mounted the internal EFI from /dev/mmcblk1p1 to /boot/efi , and next boot-repair do the magic.

    boot-efi says "ok", and tell me to reboot again. And next, i can see REFIND with a new device "linux". Booting this option says

    "Starting grubx64.efi
    Using load options ''"

    And it gets in this screen all time

    I don't know how many times i ran refind and boot-repair. I can see many entries in the boot loader, but with no success.

    Any idea?

    Blas

    ReplyDelete
    Replies
    1. i forgot to say that the installed system was correct, i can do chroot to the root partition with success.

      This only is something in the boot loader and my no knowledge of uefi system :(

      Delete
    2. Using a LiveUSB and mounting '/dev/mmcblk1p1' as '/boot/efi' how much free space do you have left on the partition? If you delete everything under '/boot/efi/EFI' except 'boot' and 'Microsoft' (i.e. delete the directory 'ubuntu' and its contents) how much space is then available?

      Delete
    3. Hi linuxium!!
      If i mount /dev-mmcblk1p1 as is , not there are used 68 mb, free 29 mb
      The disk has 100 mb

      I'm going to delete all the contents except boot and microsoft..

      Delete
    4. I've just deleted all contents below /boot/efi/EFI , booted from a liveUSB, and leaving only "Boot" and "Microsoft" on the directory.
      Now the space is:

      FIlesystem size used Avail Use% Mounted on
      /dev/mmcblk1p1 96M 44M 53M 45% /mnt

      Despite the "mnt" mountpoint..you know

      Delete
    5. With 53MB free you should just have enough free space. If you reinstall Ubuntu from the LiveUSB it should now work.

      Delete
    6. I tried several times..with no success.
      I'm now going to test with MINT using 'isorespin'.
      Stick around...

      Delete
    7. Ei guys,
      it works with isorespin and mint. No extra configuration, only following the default procedure .
      I use as a referer this guide, https://gist.github.com/pjobson/e0b0c99f5d9affb28c1a2f33503c27c7 .. but i really didn't need it.

      But, it works at the first attempt.

      Why do I using your pre-made iso , doesn't work? Hmmm.. do you have a idea?

      Sorry, I'm in CET timezone, see you tomorrow early ;-)

      Delete
    8. I'd prefer if you refer to my guide 'http://linuxiumcomau.blogspot.com.au/2017/06/customizing-ubuntu-isos-documentation.html' rather than one I'm not responsible for.

      The above ISO was initially created with an earlier version of the script and I've just replaced it with an ISO from the latest version. So the 'pre-made' ISO should also work.

      Delete
    9. Yes, i know. i'm going to repeat it following exactly every step you write instead of using the live iso. Keep in touch

      Delete
    10. This comment has been removed by the author.

      Delete
    11. Hmmm
      More problems.
      I can't do respin with latest kernel because the EFI partition (/dev/mmcblk1p1) gone full (100%) and installer cannot continue.
      The file inside /boot/efi/EFI/linuxmint/ is around 56 Mb and gets the partition full.

      There are no other useless information and files inside /boot/efi , and cannot make it greater.

      How can i solve it?

      Delete
  11. hmmm, hey linuxium.
    Some extrange things around. After a few tests, this is a surprise:

    1) Using --apollo -u (gets the 4.15.rc3), it works perfectly. But "one small thing". This appears to be a buggy kernel, because the installer hangs in live session in different moments...and with i achieve the installation, and the system comes online booting perfect.. it gets hangs randomly. Completely freeze!.

    isorespin system ran in USB, and the live in SSD..so the unique thing that have in common is..kernel? I don't know , but it happens

    2) Using --apollo with no parameters works perfect, but it doesn't install its entry in UEFI. I don't get the option to boot (it looks the same as UBUNTU above). The same problem. The only difference here is that the file created in /boot/efi/EFI/linuxmint is around 50mb, so they put the partition at 99%

    Rescue system works perfect ..but i'm not able to boot with refind neither boot rescue.

    So, using a kernel get the system online and getting other doesn't
    This is extrange, but possible

    Now (i spend so many hours doing here), i'm trying to download specific kernel version in order to check if the kernel affects on the UEFI configuration.

    Sorry for this, i'm going absolutely crazu

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete

  12. Got it!!!
    THe solution was in front of me. So i looked in the wrong direction.
    But, there are a problem with isorespin if it create .efi files more than the size available in "efi" partition.

    First problem: Check if file created below /boot/efi/EFI/LINUXVERSION/ is size enough to fit into the partition

    Next, i saw what in the efi partition were two directories. A directory called "linuxmint" and another called "ubuntu". WTF ? I cleaned every fresh install and, anyway, this file comes again.

    But, the problem with this is that the efi file (\EFI\UBUNTU\shimx64.efi" is small to boot, or damaged, or i don't know.

    Second problem: Remove the innapropiate or unnecesary /boot/efi/EFI/ directories.

    So, after many "isorespin.sh" attempts none of the recent kernels (4.13 to the latest 4.15.rc3) makes a "shimx64.efi" file enough to fit into the 56 free Megabytes that the first EFI partition lets. So, the only kernel that did it is 4.10 (i will test again in the next hours..). This kernel let the "efi partition" up to 99%, but usable and with no errors in installation.

    And now, if you install it .. all looks good, but the booting still the same. Black screen and

    ""Starting grubx64.efi
    Using load options ''"
    "
    BUT... here is the trick:
    (next post)

    ReplyDelete
  13. There are the steps:

    1) Check your efibootmgr -v
    acepc efi # efibootmgr -v
    BootCurrent: 0009
    Timeout: 3 seconds
    BootOrder: 0009,0003,000A,000B,0002,0000
    Boot0000* rEFInd Boot Manager VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
    Boot0002 UEFI: Built-in EFI Shell VenMedia(5023b95c-db26-429b-a648-bd47664c8012)..BO
    Boot0003* Windows Boot Manager HD(1,GPT,751e6cbf-2b71-41a9-a6d5-2e8afb3f7816,0x800,0x32000)/File(\EFI\MICROSOFT\BOOT\BOOTMGFW.EFI)..BO
    Boot0009* Linux MINT3 HD(1,GPT,751e6cbf-2b71-41a9-a6d5-2e8afb3f7816,0x800,0x32000)/File(\EFI\UBUNTU\SHIMX64.EFI)
    Boot000B* ubuntu HD(1,GPT,751e6cbf-2b71-41a9-a6d5-2e8afb3f7816,0x800,0x32000)/File(\EFI\UBUNTU\SHIMX64.EFI)..BO

    WTF?? Check the "EFI" path to SHIMX64.EFI file. It's wrong!!. I ran "boot rescue" and "refind" several times, and all of them makes a mistake with the path? I don't know why, but this is not the correct path. This file exists, but if i try to boot "ubuntu" or "linux mint3"... i got the same black screen.

    But, if you change it in filepath (cd /boot/efi/EFI; mv linuxmint/shimx64 ubuntu/shimx64), it works.
    Or even better , create a new entry by hand:

    $sudo efibootmgr -c -d /dev/mmcblk1 -p 1 -L "Linux MINT" -l "\EFI\LINUXMINT\SHIMX64.EFI"

    And... voila! The new entry is on the Loader, and boot loader gets the file, and starts.

    So, to summarize. All make sense now :-)

    1) Check if "isorespin" makes a non-big grub installation EFI file. You can check it by mounting the ISO file and checking the /boot/grub/efi.img and check if it fit into your EFI partition (in my case ACEPC AK1 , 100 Mb with around 58 Mb free when it's clean

    2) If partition is enought, do the install, and CHECK if the "linux/ubuntu/mint/etc..) entry has the correct size

    You can check it if the installation shows no error, and manually by doing "ls" to the EFI partition. In my case

    $ mkdir /tmp/efi; sudo mount /dev/mmcblk1p1 /tmp/efi ;
    $ ls -la /tmp/efi/shimx64.efi
    $ df -kh |egrep "efi"

    With these parameters you should check if "shimx64.efi" has a good size (around 50-60 mb) and the size of the partition is less than 100%

    3) If the file is correct and the partition isn't full, check if the path to the "shimx64.efi" file is correct doing:

    $ efibootmgr -v

    If you see something like this:

    Boot000A* ubuntu HD(1,GPT,37a202a0-3a4f-49ee-84a1-852a63e64592,0x800,0x8e800)/File(\EFI\Ubuntu\shimx64.efi)..BO

    In my case, this "shimx64.efi" file is less than 1.2 Mb.

    In that case, add a new entry to the good efi file. In my case:

    $sudo efibootmgr -c -d /dev/mmcblk1 -p 1 -L "Linux MINT" -l "\EFI\LINUXMINT\SHIMX64.EFI"

    And that its. It's worked. Tested twice .

    Sorry for the long explanation, but i think is better if someone like me landed here by googling for a solution. I could make a script to check all of them, but first i need to test more.

    Regards and thank you all from Madrid



    ReplyDelete
    Replies
    1. When I fixed the script for respinning 'neon' ISOs it looks like I introduced an error in respinning 'Mint' ISOs. If you are interested in helping to test a new version of 'isorespin.sh' you can access it here 'https://goo.gl/3MtdhX'. Once I've confirmed it works with Ubuntu, Mint, neon and others I'll upload it for everyone to use as a new release.

      Delete
    2. Ei linuxium!
      It looks well, but i'm unable to install due to the space restrictions. Using "isorespin.sh -i linuxmint-18.13-mate-64bit.iso --apollo -k v4.14.6" works well and generate a iso, but when i tried to install the efi is around 60 Mb that cannot fit into the space in EFI partition (around 54 Mb).

      I'm trying with 4.10 again, that its look like the only kernel that the EFI image generated is below 55 Mb. (probably 4.11 and 4.12, but 4.13 in advance exceeds 56 mb)

      Delete