Posts: 91
dirkd
Joined: 30 May 2014
#1
There's a lot I liked about Antix when I first gave it a try. But what captured my imagination was the promise to make a snapshot of an installed system, including new applications. I'm sure I read somewhere that the result would be an 'installable image'. But although I managed to make the snapshot and transfer the resulting .iso to a bootable USB-key, I don't see how I can install that image to another computer. The option to 'install', that was present in the original Antix distribution image has disappeared. There's some logic in that, because the snapshot was - of course - made on a laptop where Antix was already installed. Still, I had expected that burning the .iso to an USB-key would restore the 'install'-option.

Am I making a mistake, or do I misunderstand the purpose of snapshots?

Another unexpected behaviour of transferring the snapshot to an USB-key was that some information seems missing in the snapshot. When booting from the new USB-key, I find all new applications I installed. Most of them work from the live USB-key (unfortunately, SAGE is an exception). I made some new users and passwords, which are also preserved when booting from USB. Unfortunately, their desktop environments got lost in the process of making a snapshot. I spent some time in customizing the desktop, with launchers for the new applications, and all these icons are missing when booting from USB.

Any suggestions? Or am I asking for the impossible? It would be a disillusion, when at first everything seemed to work so smoothly.

Still, I have to congratulate the Antix people. It's a distro that really stands out.
Posts: 2,238
dolphin_oracle
Joined: 16 Dec 2007
#2
antiX 13.2 has a commandline installer that will install snapshot images. this is included in the snapshot image. I believe the command is" cli-installer". The graphical installer for antiX-14 will also include this feature, but its not included in the 13 installer at this time. the snapshot stuff is still evolving...

I don't know why your user configurations weren't preserved. on the snapshots I've done, my user environment was preserved, with the exception of my wallpaper, which I forgot were stored on a separate data partition, which is not included in the snapshot. The live fstab is different than the installed fstab in that it gets rebuilt every boot.

the aufs filesystem that the live uses is sometimes incompatible with certain apps. I've run into this with netflix-desktop on my liveUSB keys, although on the liveUSB w/ persistence its fairly simple to overcome with a separate HOME persistence file.

the persistence features on the live cd/usb are really getting a big ease-of-use enhancement in antiX-14. you can see some of the new easy to use features in MX-14.2, as the new stuff has been incorporated into MX already.
Posts: 91
dirkd
Joined: 30 May 2014
#3
Thank you for your reply, but I'm afraid I need some more help.

I tried the cli-installer, but didn't get far. I wanted to test on a virtual machine, but couldn't boot a virtual-box vm from USB. I then copied the snapshot .iso-file and coupled it to the virtual CD in virtual box. I could boot, chose the cli-installer, started it, and then things went wrong. The installer offered to repartition the virtual harddrive, and seemed to do so. It found the drive, labeled it as sda, and partitioned it in root, home and swap partitions. But when installing to sda1 it showed no hard drive activity at all, and complained that it couldn't write/extend/... a whole lot of files. It seemed to me that Antix tried to install itself on the virtual CD-drive, in stead of the hard drive.

I would appreciate it too, if you could elaborate a little bit about home and root persistance. There are quite some options you can choose there, and I'm not always sure what their meaning is. When transferring to USB I chose both home and root persistence, but not sure if that made any difference.
Posts: 2,238
dolphin_oracle
Joined: 16 Dec 2007
#4
It is possible that you did not in fact write the changes to the virtual box drive's mbr/partition tables. I recommend setting up the partitions with gparted (on the regular antix iso) ahead of time, then running the cli-installer. The cli-installer in antiX 13.2 is very basic and originally designed for installing the cli-only antiX-core.

As for persistence options, they aren't really meant for snapshot iso's. What is cool with liveUSB w/persistence is that you can set up all your programs ahead of time on the liveUSB, and then they will carry over to the install when you run it. By default you will lose any user customizatoins that are stored in your home folder, unless you copy them over yourself later or put them in /etc/skel ahead of time.

I've got a couple of videos. The first is from antiX 13, and I state in the video that I just use root persistence because that option will store everything. Since that video, I've switched to having both a root and a home persistence file. Its slightly faster, as home isn't compressed, and since aufs isn't used on the home side, things like Netflix-Desktop will work that way as well (won't work with just root persistence, although the app will carry through to a new install).

Once you have your apps installed, you can then run a remaster on the drive (available in remastercc) which will take the root persistence file and incorporate it into the main linuxfs file (which is where the base antix system is stored). this has the advantage of using less ram, less drive space, and a faster boot time. You don't have to incorporate the home persistence file in the remaster unless you want too. I usually leave it out. I don't have any videos showing the remaster operation, but its pretty easy.

antix 13 - (2 metheds shown here, antix2usb and"setup live persistence" from the control center) around 10 minutes in is where we turn an existing liveUSB made with unetbootin into a liveUSB w/ persistence.

mx 14 (shows the root and home system)
Posts: 1,444
skidoo
Joined: 09 Feb 2012
#5
IIRC, during installation to HDD (yes, because doing so"makes sense") the installer-related files (plus live-init and syslinux booloader) are withheld from the destination.
If so, apt-get install antix-installer would reinsert the installer script. Might need to manually recreate a desktop"install antiX" icon, if desired.
Afterward, performing a snapshot (of the HDD-installed OS) should yield a faithful copy, with the install script intact.
The installer script might not (probably won't) preserve additionally-created user accounts, though.

The"remaster" utility (accessed from antiX ControlCentre GUI) is a bit different from"snapshot".
It might (the docs, or someone else, can clarify) auto-download the missing-after-install utilities and re-inject those into the destination copy.
I've never tested whether the remastering utility preserves extra user accounts, but I doubt it is designed to do so.

If you're willing to quickly experiment:
Instead of first"installing" the OS prior to creating multiple user account and installing apps, try this:

From a liveUSB:
-- setup Root Persistence
-- install SAGE
-- create a couple new user accounts

That working base, as a jumping-off point, will enable you to test whether install-to-HDD preserves extra user accounts
(and whether the the installed SAGE configuration transfers properly. I'd expect it would).
Separately, you're free to (skim the docs and) try out the"snapshot" and"antix2usb" and/or"remaster" utilities.

"Why _root_ persistence?" That's a fine question but, for now, to get started... just do it.
IMO, mixing in variables like separate home partition, multiple user accounts, potential VM quirks... at the outset, just invites extra confusion.
Posts: 91
dirkd
Joined: 30 May 2014
#6
Thank you both. I'm beginning to understand the process of snapshots and remastering much better now. I since discovered that booting to the desktop environment and then running the cli-installer worked better, but still not good enough. The hard disk was used, an image of the OS installed, but after finishing I couldn't boot from the hard drive.

I think I'll start over. I'll watch the video's, then I'll try skidoo's approach.
Posts: 91
dirkd
Joined: 30 May 2014
#7
I watched the YouTube video, and found it very instructive. I also came across the FAQ-page about frugal install, which seemed very promising (speedwise). But just to be sure: how is a frugal install converted back to a bootable USB, without loosing the customizations made? If I understand the video about root persistence correctly, running antix2usb just copies the original iso, without customizations.

In the mean time, I tried booting a virtual-box machine from the original Antix distribution USB stick. It couldn't boot the kernel, with the same error message I got before, when booting the system installed from my snapshot. It seems the virtual hardware of virtualbox is not compatible with the kernel used in Antix. Is that possible? I'm now installing VMware, since that program gives you a choice of different hardware to use with you virtual machines.
Posts: 91
dirkd
Joined: 30 May 2014
#8
Still struggling with Antix Snapshots...

I found the configuration file snapshot_exclude.list, and that mentions explicitly

- home/*/Desktop/*

so, that explains why my Desktop customizations were not copied over to my snapshot.

I'm still puzzled why SAGE is nog working though. I can see that a few hundred files and about 20 folders from the SAGE folder, which is installed in /opt/, are not copied over to the snapshot. But SAGE is a gigantic 4GB folder, and I can't pinpoint exactly which files/folders are not copied. Any suggestions?

I tried to attach the snapshot_exclude.list file, but the forum software doesn't seem to let me do so.

So except for this SAGE puzzle, snapshots seem to work for me. As a warning for others testing with virtual machines: don't use Virtualbox: the kernel used with antix 13.2 doesn't boot on the virtual hardware. Using VMware, you can adapt the virtual hardware to the kernel version, and I had no problems booting from the snapshot .iso file.

@skidoo: I tried to reinstall the graphical antix installer as you suggested ( apt-get install antix-installer ), but the package antix-installer was not found.
Posts: 2,238
dolphin_oracle
Joined: 16 Dec 2007
#9
i'm glad its moving in the right direction!

fyi - after release of 13, anti did discover the bug with the kernel and virtualbox (actually any compiled driver). another kernel was made available. on the news page (antix.mepis.org) check out the post 12 November 2013 for details.
Posts: 91
dirkd
Joined: 30 May 2014
#10
I think I pinpointed the problem. In the file snapshot_exlude.list there's an entry

- media/*

and sure enough, the SAGE folder has buried deep within it a folder ...../media/ that is left empty in the copy of the filesystem. Surely what is meant is that the contents of the folder /media/ should be left out from the new squashfs filesystem. As it is now ANY folder media/ is left empty. I will try editing the snapshot_exclude.list file, but each experiment takes about 4 hours on the old laptop I use.

In the mean time: is there anyone who is knowledgeable about the syntax used in this file? This is the original exlude.list file:

Code: Select all

# rsync excludes file for antixsnapshot and antixsnapshot-gui
# version 0.2.7.7 (23.05.2013)

- dev/*
- cdrom/*
- media/*
- swapfile
- mnt/*
- sys/*
- proc/*
- tmp/*
- /live
- boot/grub/menu.lst
- boot/grub/device.map
- etc/udev/rules.d/70-persistent-cd.rules
- etc/udev/rules.d/70-persistent-net.rules
- etc/fstab
- etc/mtab
- home/snapshot
- root/.bash_history
- var/log/*.log
- var/log/*.gz
- var/log/*.old
- var/cache/apt/archives/*.deb
- var/cache/apt/*.bin
- var/lib/apt/lists/*
- var/cache/debconf/*-old
- var/lib/dpkg/*-old
- lost+found
- run/acpid.socket
- run/dbus/system_bus_socket
- usr/share/doc/*

# Entries below this can be edited by the user. If you have any large
# files or directories, you should exclude them from being copied
# by adding them to this list.

- home/*/.mozilla/firefox/*/Cache/*
- home/*/.mozilla/seamonkey/*/Cache/*
- home/*/.adobe
- home/*/.macromedia
- home/*/.thumbnails/large/*
- home/*/.thumbnails/normal/*
- home/*/.Trash*
- home/*/.local/share/Trash/*
#- home/*/.cache
- home/*/.gvfs
- home/*/.bash_history
- home/*/.recently-used
- home/*/.recently-used.xbel
- home/*/.VirtualBox
- home/*/VirtualBox VMs
- home/*/Desktop/*

# Examples:
#- home/*/Downloads/*
#- home/*/Music/*
#- home/*/Pictures/*
#- home/*/Videos/*
# To exclude all hidden files and directories in your home, uncomment
# the next line. You will lose custom desktop configs if you do.
#- home/*/.[a-z,A-Z,0-9]*
I would like to prefix each entry with an"/", so that all folders would refer to the rootfolders only. Is this permissible? The comments seem to explicitly warn not to edit the first section. But surely the behaviour with the media/ folder in SAGE qualifies as some sort of bug, doesn't it?
Posts: 91
dirkd
Joined: 30 May 2014
#11
Succes! This really opens up many possibilities. Up to now, making a live-CD for SAGE was a job for specialists. With Antix I can roll my own, and complement SAGE with some other math programs, suitable for beginning students. Keeping up to date with new versions of SAGE seems relatively straightforward. Thanks everyone for the advice.



I gather a new version of Antix is in the works. Maybe in that new version, the snapshot_exclude.list file can be corrected. Do I have to contact someone, or does this post in the forum suffice?
Posts: 2,238
dolphin_oracle
Joined: 16 Dec 2007
#12
how did you edit the excludes list? This will be informative for the future.
Posts: 91
dirkd
Joined: 30 May 2014
#13
dolphin_oracle wrote:how did you edit the excludes list? This will be informative for the future.
As I said in the previous post I prefixed each and every entry with a"/", thus

Code: Select all

- /dev/* 
in stead of

Code: Select all

- dev/*
etc., etc.,.. Of course I left the entry

Code: Select all

- /live 
untouched, as it had a leading slash already. Also I commented out the entry

Code: Select all

- home/*/Desktop/*
so as to include the launchers on the desktop in the snapshot, but that is of course inessential.
Posts: 1,444
skidoo
Joined: 09 Feb 2012
#14
For the"excludes list", I've never found a document explaining the reason/rationale for each entry in the default list.
Although I've personalized my list so that it's much more granular, I've learned to respect that the default list reflects sane (safe/necessary) exclusions.

/ media path is listed b/c that location serves as a mountpoint.
Above, with that in mind, I advised avoiding complexity (separate homefs, etc) while learning to use the snapshot utility.
Yeah, if sage files are pathed under /media, or contain ambiguously-named (vs the pattern employed) subdirectories ...luckily you figured that out.

/ dev
This gets populated dynamically and, across boots, we can't (er, shouldn't) presume that devices will be consistently recognized in the same order

/ live
Well, there's"live" and"live-boot" and... to avoid recursion, some paths need to be excluded.
During live session, some paths are seen/mounted twice (read-only, and again in the aufs layer).

The leading forward slash for each of the various line item matching patterns... for certain paths, adding the leading slash could introduce/create problems.
That same exclude list is sourced by multiple scripts (are you/we certain none of the scripts will choke when parsing the leading slash character?)
For instance (memory is is foggy but)"remaster" script sources the same excludes_list and (but) it treats the patterns as paths relative to the script's working directory rather than the filesystem's root.

^----- I'm pointing out the above in response to this:

========= SCRAPER REMOVED AN EMBEDDED LINK HERE ===========
url was:"http://ask.sagemath.org/question/23606/copying-the-sage-installation-directory-with-rsync/"
linktext was:"http://ask.sagemath.org/question/23606/ ... ith-rsync/"
====================================

Although I don't agree with the decision to use SAME excludes list for multiple scripts, I'm explaining that the default patterns are"by design"; they don't reflect"sloppiness".
The devs are pretty slick; in response to your post explaining the problem you encountered, maybe they'll change away from utilizing a single"shared" excludes list.

Likewise (not that you asked) presence of the trailing asterisk in many of the default patterns is"by design".
Result is preservation of existing directory, empty, with its creation timestamp and its permissions intact.

Four hours per run?!? In my usage, creating a snapshot + iso (fs contains about 3Gb files-on-disk) takes only 7 -- 20 minutes.
Does SAGE really add"4Gb worth" of files?!? What is the size of your resultant iso?
I'm wondering whether duplicates (as a byproduct of editing your excludes_list) is bloating your snapshot size.

A related consideration: it's advisable to"perform a cleanup" prior to creating a snapshot.
For starters, install and run the"bleachbit" app (twice -- once as root user, again as normal user)(to purge your browser filecache).
Presence of unwanted debs in /var/cache/apt/archives represents another, potentially huge, bloat item (path).
If bleachbit neglects to clean the deb archive path, you can fallback to using commandline"apt-get clean"
Posts: 91
dirkd
Joined: 30 May 2014
#15
I appreciate that there are many, many things one has to think about before changing configurationfiles. I'm really impressed by the people who are able to setup this snapshot system. So when I mentioned a sloppy configuration file, I didn't mean to be derisive or something. I only wanted to make clear that the system, as it is now, doesn't work as it was intended, when SAGE is involved at least. And I'm sure that other similar mistakes will occur in other setups.

I haven't tried the effect of the edited excluse list on 'remaster', but when I do, I'll report here.

And yes, SAGE is quite a behemoth. 4GB for version 6.0, 4.2GB for version 6.2 and there's no end in sight. Luckily, a big percentage of that is text in one form or another, so it can be squashed well. In fact, it installs itself from an .iso file of about 700Mb, so what I do, ironically, is unpack 700Mb to 4GB, than squashing it again to fuse it with Antix. The resulting image is just over 3GB (14GB used on disk before taking the snapshot). Of course, the old laptop I use is 10 years old, that doesn't help much, speedwise.

Next snapshot I will cleanup, as you suggest. I don't expect much of it though. I think the system as it is is pretty clean, given that I started from scratch.