Antix 16.2 USB Persistence Fails To Create rootfs and homefs - Page 2

figosdev · posted: 2017-07-05 02:53

vfat fails, thats interesting.

one of the saddest things that i ever experienced in debian was this year, helping someone install debian 8 (not my recommendation!) and then copying their backed up data from usb...

what? it cant mount vfat? seriously guys?

i looked it up, its a known issue after using the installer... vfat seems to not have the modules needed to mount vfat.

except the modules are there. the solution? reboot. oh i tried everything else-- reboot fixed it. not sure this is related, but id feel remiss without mentioning it.

calinb · posted: 2017-07-05 16:16

figosdev wrote:
<snip>

. not sure this is related, but id feel remiss without mentioning it.

Yes and thanks for mentioning it. I think it is very likely to be related, because vfat did not work on my SD card even (where ext4 DID work). The persistence utility created the vfat partition, but obviously it could not use it (probably because mounting it failed).

In using both the USB boot persistence utility and the Control Centre utility, I believe there are perhaps than one bug (or even two bugs) here!

BitJam · posted: 2017-07-05 16:44

The problem is only in the 32-bit version of antiX-16.2. In that version (and in no other) we get a segfault every time we try to run a program in the main file system from the initrd. If you are not on 32-bit hardware then an easy solution for now might be to use the 64-bit version of antiX-16.2 or if you are on 32-bit hardware then use antiX-16.1 instead.

My guess is we will solve this problem either by further upgrading a library or by downgrading a library until this problem gets fixed. There might be a more clever way to use programs from the main file system but since this is a very isolated problem (from my perspective, not from yours of course) the best fix might to change one of the libraries.

BitJam · posted: 2017-07-05 20:24

I tracked the problem down to a mismatch in libraries in the initrd versus the main file system. I fixed this mismatch which solved the segfault problem.

You can fix an existing antiX-16.2_386 live-usb by downloading the file initrd-16.2.gz from
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"https://www.dropbox.com/sh/gak1bbmba8yog8g/AADFo22UqqhZD2haHNtjLsqMa?dl=0"
linktext was:"this Dropbox folder"
====================================
and then using it to replace the file /antiX/initrd.gz on the live-usb. You can also replace /antiX/initrd.gz.md5 with initrd-16.2.gz.md5 from the Dropbox folder.

calinb · posted: 2017-07-05 20:50

BitJam wrote:I tracked the problem down to a mismatch in libraries in the initrd versus the main file system. I fixed this mismatch which solved the segfault problem.

You can fix an existing antiX-16.2_386 live-usb by downloading the file initrd-16.2.gz from
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"https://www.dropbox.com/sh/gak1bbmba8yog8g/AADFo22UqqhZD2haHNtjLsqMa?dl=0"
linktext was:"this Dropbox folder"
====================================
and then using it to replace the file /antiX/initrd.gz on the live-usb. You can also replace /antiX/initrd.gz.md5 with initrd-16.2.gz.md5 from the Dropbox folder.

Wow--thanks BitJam! I'm downloading your files from my (slow) Internet connection now. I think I should be able to migrate my SD card persistence files onto my ext4 hard drive after installing your fix too.

Yes. I'm on old 32-bit hardware (Samsung netbook) but antiX is really snappy on it and it's the only GNU/Linux that never annoys me with any wifi disconnects of my smartphone hotspot...and I've tried several distros with this netbook. Originally I'd planned to just use persistent antiX to help me fix the MX-16 wifi issue (on hard disk) but I may decide to customize antiX to my liking (maybe install LXDE, which I've liked in the past) and permanently install antiX instead of MX-16.

Thanks for the continuing support of old hardware!

BTW, I love your signature line. I've read a couple of Feynman books but don't remember hearing that quote!

BitJam · posted: 2017-07-06 05:22

When I downgraded the libs in the live initrd to match the libs in antiX-16.2 I neglected to downgrade the ntfs-3g program. I've fixed that in initrd-16.2b.gz in the same
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"https://www.dropbox.com/sh/gak1bbmba8yog8g/AADFo22UqqhZD2haHNtjLsqMa?dl=0"
linktext was:"Dropbox folder"
====================================
. This should not matter to you unless you want to run frugal from an ntfs partition or put persistence files on an ntfs partition.

calinb wrote:I may decide to customize antiX to my liking (maybe install LXDE, which I've liked in the past) and permanently install antiX instead of MX-16.

Despite your initial bad experience, live persistence and remastering in antiX are pretty good. Many people (myself included) almost always run a live system and seldom install.

Thanks for the continuing support of old hardware!

You're welcome. It's what we try to do here. We had a miscommunication when we created antiX-16.2 that caused the bugs you found. We are a small team and we were working on 3 distros at the same time (MX-16.1, antiX-16.2, and antiX-17) and that bug fell through the cracks. Thank you for reporting it and hanging in long enough to give us time to fix it.

BTW, I love your signature line. I've read a couple of Feynman books but don't remember hearing that quote!

Thanks! He was a big influence on my life. I was also influenced by an obscure quote by Einstein that basically says object/scientific reality is a subset of subjective reality.

calinb · posted: 2017-07-09 16:22

BitJam wrote:When I downgraded the libs in the live initrd to match the libs in antiX-16.2 I neglected to downgrade the ntfs-3g program. I've fixed that in initrd-16.2b.gz in the same
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"https://www.dropbox.com/sh/gak1bbmba8yog8g/AADFo22UqqhZD2haHNtjLsqMa?dl=0"
linktext was:"Dropbox folder"
====================================
.

I re-authored my antiX-16.2_386-full.iso file with the latest initrd files you provided. Then I re-flashed my USB stick and copied my /antiX subdirectory with rootfs and homefs files from my SD card to my ext4 hard disk (my MX-16 boot disk) and antiX persistence works great from my hard drive now! I still have to use"pdev=sda1" at boot though.

I have an NTFS usb drive around somewhere that I could try, if you need someone to check it. I remember how hard I worked around the Ubuntu 10.4 time frame (or maybe it was 8.4 or so) to get persistence to work with the unionfs. This was much easier and I'm sure it usually works"right out of the box" with antiX.

I switched my cell phone provider and I am no longer experiencing Internet disconnects from my smartphone hotspot, so I don't have to debug that wifi problem after all. __{{emoticon}}__ It's strange that the drop outs never occurred when booted to antiX but occurred with every other distro I tried.

Thanks again!

BitJam · posted: 2017-07-09 18:11

calinb wrote:I re-authored my antiX-16.2_386-full.iso file with the latest initrd files you provided. Then I re-flashed my [USB stick ...

I'm glad you got it to work! Thanks for letting us know! It probably would have been easier to just copy the new initrd file directly to your existing antiX-16.2_386 live-usb. I'm sorry I didn't make this more clear.

I still have to use"pdev=sda1" at boot though

The"F8 --> save" feature will (should) save this for you so you don't have to keep typing it in. This is one of several advanced live-usb features we offer that I haven't seen in other distros.

If someone wants to simply customize the language and/or timezone (or anything else that can be set via the bootloader) then they only need to use" F8 --> save" without having to set up persistence.
If someone wants to install/remove certain programs and then stick with that configuration, they can use live-remaster without having to enable root persistence. If you have a faster/beefier machine then I suggest doing the remaster on that machine because it will be faster. This feature allows you to easily make your own customized live-usb. This allows you to combine the security of"kiosk mode" with the flexibility of arbitrary customization.
Even without persistence enabled we will remember wireless network names and passwords and a few other things, such as your choice of window manager, across reboots. With a minor configuration change, you can use this to save login passwords across reboots.
There is a ~/Live-usb-storage directory (the name has recently changed a little) for storing files across reboots even if persistence is not enabled. With persistence enabled, this is still the place to store large files such as iso files. It is also a handy way for transferring files between the live-usb and other systems via sneaker-net. When you mount the live-usb on another system, these directories show up at /Live-usb-storage/$USER/.

calinb · posted: 2017-07-09 20:23

BitJam wrote:I'm glad you got it to work! Thanks for letting us know! It probably would have been easier to just copy the new initrd file directly to your existing antiX-16.2_386 live-usb. I'm sorry I didn't make this more clear.

Sure! Thanks for taking the time to explain all the live/persistence features to me, BitJam. I will be using them in the future, I'm certain. antiX is definitely the king of live boot and persistence!

I'm having two problems that might even be related. I had to re-author the .iso image to install your new initrd files, because I could not get my USB stick to mount -o rw. It would only mount read-only. Maybe I just don't know the right mount options (force or something).

BitJam wrote: The"F8 --> save" feature will (should) save this for you so you don't have to keep typing it in. This is one of several advanced live-usb features we offer that I haven't seen in other distros.

The second problem is I setup the boot screen as usual (pdev=sda1, persist full) and use F8 to save, but the next time I boot the USB stick, it doesn't remember the settings. When it boots, it doesn't find my rootfs and homefs files on sda1 and tries to setup persistence again.

As always, thanks for your help!

BitJam · posted: 2017-07-09 22:09

calinb wrote:I'm having two problems that might even be related. I had to re-author the .iso image to install your new initrd files, because I could not get my USB stick to mount -o rw. It would only mount read-only. Maybe I just don't know the right mount options (force or something).

The output of this command should help tell us what is going on:

Code: Select all

sudo lsblk -o name,size,fstype,mountpoint,label

The second problem is I setup the boot screen as usual (pdev=sda1, persist full) and use F8 to save, but the next time I boot the USB stick, it doesn't remember the settings.

Did you use"dd" to create the live-usb? What tool did you use? It sounds like you made might have made a read-only live-usb. But if that were the case then the F8 menu should not appear. If you didn't use live-usb-maker then I can give you instructions on how to. It's very easy! If you want it to work right with all the features then please use live-usb-maker. If you have a 2nd usb-stick and a 2nd usb-slot then it can clone your running live-usb system from within. Just plug the 2nd stick in and run live-usb-maker or live-usb-maker-gui.

When it boots, it doesn't find my rootfs and homefs files on sda1 and tries to setup persistence again.

If a new custom entry didn't show up in the main menu with"pdev=sda1" then you will have to type it in each time until we get this read-only problem resolved.

If you have more than one partition with an antiX live bootloader on it then that could explain why the bootloader didn't change but I agree with you it is more likely related to the read-only problem since we can't change anything on a read-only file system.

calinb · posted: 2017-07-09 23:09

BitJam wrote:Did you use"dd" to create the live-usb? What tool did you use? It sounds like you made might have made a read-only live-usb. But if that were the case then the F8 menu should not appear.

I think you have figured it out, BitJam! I used fdisk and sudo mkfs -t ext4 /dev/sdb1. Then I used UNetbootin to write the .iso image. I know I have used Live-usb maker at least once in the past, but I guess I got suckered by the UNetbootin entry in the program launcher. Both antiX and MX Linux contain UNetbootin and Live-usb maker and both utilities can write the USB with an iso file. I guess UNetbootin must be creating a read-only USB. However, the F8 menu did still appear, as you noted.

I re-flashed the stick using Live-usb maker and now F8 retains my pdev=sda1 and live-full settings. It appears that, without pdev=sda1, it only looks on CDs and USBs for the homefs and rootfs files. At least I think that's what it reported to me when I didn't use the pdev option.

Cool! It's working now and thanks once again.

BitJam · posted: 2017-07-10 01:11

calinb wrote:I think you have figured it out, BitJam! I used fdisk and sudo mkfs -t ext4 /dev/sdb1. Then I used UNetbootin to write the .iso image. I know I have used Live-usb maker at least once in the past, but I guess I got suckered by the UNetbootin entry in the program launcher. Both antiX and MX Linux contain UNetbootin and Live-usb maker and both utilities can write the USB with an iso file. I guess UNetbootin must be creating a read-only USB. However, the F8 menu did still appear, as you noted.

I re-flashed the stick using Live-usb maker and now F8 retains my pdev=sda1 and live-full settings. It appears that, without pdev=sda1, it only looks on CDs and USBs for the homefs and rootfs files. At least I think that's what it reported to me when I didn't use the pdev option.

I'm glad you got it to work!

Your use of UNetbootin is entirely understandable if it was in our menu. Maybe it is time for us to remove it.

Too many notes:
Yes, we won't go poking around your hard drives unless we are invited. For booting (looking for the linuxfs file) the"from" boot parameter is set to"from=usb,cd" by default. You can add"hd" to the list or set it"all" to have us check everywhere. One reason for this is that we are looking so early in the boot process we can't guarantee we have the time or the timezone set correctly but when we access a partition to look for files, the"time" in our system is recorded. This can cause mysterious error message when a Linux system that has been poked this way boots up.

For persistence pdev= or plab= or puuid= will disable the same filter for looking for persistence files. But normally we only look for them on the partition that had the linuxfs file. You can change the persist 'from' value by adding"usb" or"hd" to the persist=xxx cheat.

If persistence is requested on a read-only boot device and no pdev or plab or puuid is given then we default to using"plab=antiX-Persist". This is for the unusual case where someone wants persistence with a live-cd/dvd (or read-only live-usb). If they label their persistence partition as"antiX-Persist" then we will look there for the persistence files so they only need to enable persistence from the F5 menu and they don't have to type in something like"pdev=sda1" on every boot.

Even if you specify a device, label, or uuid you can still enable the"from" filter if you give us a"from=" boot parameter or add"hd" or"usb" to the"persist=" parameter. For example:

Code: Select all

from=usb blab=Live-USB

will cause us to only look for usb partitions with the label"Live-USB". On the persistence side you can do the same thing with:

Code: Select all

persist=usb,... plab=my-stuff

Oh, one more thing, you are allowed to use globs (* and ? and so forth) when specifying a uuid. So"buuid=abc*" will look in all partitions that have a UUID that starts with"abc". If you only want us to only look on usb partitions then add"from=usb".

The"from=" boot parameter actually gets used sometimes. The most common case is when someone wants the speed and/or features of the live-usb but have a machine that only boots from cd/dvd. In that case if you make a live-cd and a matching live-usb then you can use the"from=usb" cheat in the"F4 Options" menu when you've started to boot from the live-cd in order to finish the boot from the live-usb.

re: live-usb-maker vesus other programs
There's not much you can do with a live-cd (but we try). The live-usb-maker program is designed to make the best possible live-usb for the vast majority of people. If you buy one of our live-usbs commercially from
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"https://www.osdisc.com/products/antix"
linktext was:"OSDisc"
====================================
this is the program they use to create it for you. They also provide a lot of support for the physical device so IMO it is well worth the price for people who want to go that route.

One imperfection (I'm sure there are many others) is that Dell added a bug to some of their machines. If we use gpt partitioning (which is much better than the ancient bios/msdos partitioning) then some Dell machines forbid UEFI and BIOS booting on the same device. We have to choose one or the other when we make the live-usb. This is nuts because we don't know beforehand who wants/needs to boot via BIOS and who wants/needs to boot via UEFI. We get around this bug by defaulting to msdos partitioning. In the command line version of live-usb-maker you can use the --gpt flag to tell us to use gpt partitioning instead.

Back in 1981 or so, right after the IBM PC was released, a mentor of mine told me that the IBM PC would go the same route as mainframes and minicomputers before it and it would eventually be running Unix. I took a similar approach to live-usbs (in a way). Eight or so years ago we predicted that usb storage would get larger and faster and cheaper and more reliable so we designed with those future usb devices in mind. In the live-usb-maker program we use a real, journaling filesystem (ext4) just like a real installed system and we periodically run fsck on our live-usbs (if possible) just like a real installed system.

BTW: the segfault problem you reported helped inspire a change in our build process. Starting with antiX-17-beta-2 (I hope) we will automate adding dynamic libraries and linked programs to the initrd so they will always match what's in the actual file system. Up until now we were able to get by with manually updating the programs and libs in the initrd for each major release.

calinb · posted: 2017-07-10 02:29

BitJam wrote:I'm glad you got it to work!
Your use of UNetbootin is entirely understandable if it was in our menu. Maybe it is time for us to remove it.

Yes--it's on both the antiX and MX menus. I just now mounted my antiX live USB stick on my MX-16 system, as I did when I first tried to update the initrd files, but this time it mounted rw, whereas the UNetbootin stick would only mount read-only--even when I used sudo and -o rw on the command line. Yes--perhaps it should be removed from the menu (and the MX Linux menu as well, where its advertised as"Tool for creating Live-usb drives").

BitJam wrote: Too many notes:

If you have time to write them, I certainly have time to read and learn! __{{emoticon}}__

BitJam wrote: Yes, we won't go poking around your hard drives unless we are invited.

That makes sense...and thanks!

I read your notes in your post and, though I'm no longer in the business (I was a hardware engineer at Intel for over 20 years) I can appreciate and still occasionally use such a feature rich live-boot architecture. I still have a friend who is a BIOS/EFI engineer at Intel. He's had uses for persistence in the past so I will forward a link to this thread to him.

I go way back with PCs too, but I was working on the hardware side. One of my first jobs at Intel was the design of a networking glue chip for the ISA bus. What a mess IBM created with the LA line decoding to generate MEMCS16 but honestly, I hardly even remember the incompatibility issues from that long ago. I've never strayed too far from software, however. I wrote the boot code for the N1 (i860 or"Cray on a chip"). Only Gordon Moore was backing the chip and, being RISC, it didn't fit into Intel's marketing strategy and plans. I saved Intel from needing to immediately re-"tape out" the N1 by inventing a crazy workaround for a significant architectural oversight.

BitJam wrote: Back in 1981 or so, right after the IBM PC was released, a mentor of mine told me that the IBM PC would go the same route as mainframes and minicomputers before it and it would eventually be running Unix.

Similarly, many at Intel and even IBM believed that Unix would happen not with x86, but rather with the i860, so I spent a couple of years developing SysV i860 machines and then massively parallel i860 Supercomputers before Intel shut down all i860 efforts.

After the i860, I went mainstream into the P6 (Pentium Pro) but left Intel about three years ago to retire and live in the woods!

Well...I don't want to waste your time with any more"resume" stuff than the above, but it is great to meet you here on the forums. I'm very impressed with antiX and, even with its larger footprint, MX runs quite well on my old hardware too. Being an early Slackware guy and then Gentoo guy (back when I had cheap and fast Internet for the code builds), I'll probably end up installing antiX on my hard drive to tweak it how I please, but I am not in a hurry to do it, because I can tweak antiX persistently and install it later or, as you said is also very common with your peers, never install it!

Thanks and Best Regards,

-Cal

calinb · posted: 2017-07-10 03:05

BitJam wrote: BTW: the segfault problem you reported helped inspire a change in our build process.

Oh...and it's good to hear my plea for help resulted in some degree of validation value to the antiX project!

BitJam · posted: 2017-07-10 08:36

calinb wrote:Well...I don't want to waste your time with any more"resume" stuff than the above,

I'm fascinated by your story and would like to hear more. We have a couple of forums for that sort of thing:
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"antix.freeforu ms.org/welcome-to-antix-f1.html"
linktext was:"Welcome to antiX"
====================================
and
========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"antix.freeforu ms.org/in-a-greek-kafeneio-f7.html"
linktext was:"In a Greek kafeneio"
====================================
come to mind.

but it is great to meet you here on the forums.

likewise

I'm very impressed with antiX and, even with it's larger footprint, MX runs quite well on my old hardware too.

Thanks!

Being an early Slackware guy and then Gentoo guy (back when I had cheap and fast Internet for the code builds),

I started using Gentoo a few months after it first came out and I'm still using it now. I started out using RedHat. I tried Debian before I tried Gentoo but it was a disaster for me. I needed to (or thought I needed to) compile the nvidia driver to get X working so I had to wade through all the apt-cache versus apt-get versus aptitude stuff via text mode browsing. It was a nightmare partly because it was not a well trod path. Gentoo on the other hand was really easy. They gave me clear instructions for doing everything from the command line including compiling a kernel.

I'd been thinking about this because in antiX-17 I've finally solved part of that initial problem I had with Debian by creating a command-line package installer that works like Synaptic. I sometimes get very frustrated with design choices made by the Debian team. Sometimes it feels like they do things obscurely or in the most incompatible way possible on purpose. But it is more likely they just don't know or don't care.

I'll probably end up installing antiX on my hard drive to tweak it how I please, but I am not in a hurry to do it, because I can tweak antiX persistently and install it later or, as you said is also very common with your peers, never install it!

Yes! Exactly! You've got it. You can use your custom live-usb to install your version of antiX on a number of machines if you want. We also have an iso-snapshot program for creating an iso file from a live or installed system.

The live system is still clunky from cd or usb-1.x but it really flies with usb-3.0. I once did a frugal install to an internal ssd that booted to bash in about 5 seconds and got to X-windows in less than 10 seconds (something like that).