pibox-installer
as well as ansiblecube
inside it depends on a number of external files which are downloaded and used during the installation process.
This document is a list of such files, their use and how to create/update them when that's available.
Bare copies
Following are files publicly available from their upstream source which we copied into our mirror to prevent network issues during builds (CI or image).
/dev/qemu-w32-setup-20180430.exe
/dev/qemu-w64-setup-20180430.exe
/dev/7z920_extra.7z
/dev/7z1805-extra.7z
/dev/pygi-aio-3.18.2_rev12-setup_549872deadabb77a91efbc56c50fe15f969e5681.exe
/dev/parsoid_0.9.0.tar.gz
/dev/mediawiki-1.31.0-rc.0.tar.gz
/dev/composer_1.6.5.phar
QEMU kernel (pibox)
Faster than the stock kernel on modern hosts.
Support up to 4 cores and 30GiB or RAM.
See README.md for details on how to update it.
Static QEMU (pibox)
Used in .gitlab-ci.yml
to skip the compilation step of QEMU.
Stock QEMU compiled statically for linux x86_64
.
macOS QEMU
Used in .travis.yml
as part of macOS bundle. Travis would frequently fail to build due to timeouts in brew install
or compilation of QEMU.
brew update
brew install qemu
git clone https://github.com/auriamg/macdylibbundler/
cd macdylibbundler/
git checkout 84440587e1b2e4e9e0727cbba7a455b96820e40a
make
cd ..
mkdir qemu
cd qemu
cp /usr/local/bin/qemu-img .
cp /usr/local/bin/qemu-system-arm .
chmod u+w qemu-img
chmod u+w qemu-system-arm
../macdylibbundler/dylibbundler -d . -p @executable_path/ -b -x ./qemu-system-arm -x ./qemu-img
chmod u-w qemu-img
chmod u-w qemu-system-arm
otool -L *
cd ..
tar cf qemu-2.12.0_macOS.tar qemu/
Python 3.6 formulas for brew
On travis, Python needs to be installed via brew (for pygobject3). Brew does not provide a way to pick the python version and always uses the latest stable one (3.7.0 now).
Since pyinstaller is not yet 3.7 compatible, we have to use python 3.6.5 and, for that, change the brew formulas of python and some dependencies.
We used to do a git checkout
into `` but now that travis updated its base macos image, this commit is beyond the limited history.
cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
zip brew-py36.zip Formula/python.rb Formula/pygobject.rb Formula/gtk+.rb Formula/glib.rb Formula/adwaita-icon-theme.rb
GoRC
Stock copy of the binary (GoRC.exe) from http://www.godevtool.com/Gorc.zip
Used to compile resource file for Windows.
ideascube
build (ansiblecube)
Custom
A deb of ideascube
with our patches:
- move zim from
--package-cache
instead of copying. -
--skip-sha256
option. - Debian stretch package (python3.5).
Use rgaudin/ideascube until those patches gets integrated upstream.
xz 5.2.4
Used only in linux's build .travis.yml
: the build system is trusty (14.04) due to the py34
requirement but trust's LZMA library is too old. See #176 (closed).
Stock copy of lzmautils's distribution on sourceforge. SF raises frequent download issues from travis.
Exfat-fuse (ansiblecube)
Stock copy of debian bower package as stretch version is obsolete and fetching directly from archive sometimes fail.
Exfat Kernel modules (ansiblecube)
A pair of exfat-nofuse kernel modules for the raspberryPi.
- Compiled against the same kernel tree as the raspbian kernel in use.
- Archive contains module for the two rPi flavors: regular (Pi2) and
-v7
(Pi3)
Creation steps
You can alternatively compile the modules on raspberry hardware running the same raspbian version you are targetting. You will need to do it twice thought (first on a Pi2 then on a P3) and copy the modules from there.
The following instructions are for a cross-compilation.
-
Get kernel commit from
.info
file of raspbian-listecurl http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-04-19/2018-04-18-raspbian-stretch-lite.info
-
Download the raspberry version of the kernel for that commit
wget https://github.com/raspberrypi/linux/archive/f70eae405b5d75f7c41ea300b9f790656f99a203.zip
unzip f70eae405b5d75f7c41ea300b9f790656f99a203.zip
mv linux-f70eae405b5d75f7c41ea300b9f790656f99a203/ linux-4.14.34
cd linux-4.14.34
- Download exfat-nofuse and place it in tree
wget https://github.com/dorimanx/exfat-nofuse/archive/master.zip
unzip master.zip
mv exfat-nofuse-master/ fs/exfat
- Add exfat to
fs/Makefile
obj-$(CONFIG_EXFAT_FS) += exfat/
- Add exfat to
fs/Kconfig
(in filesystems section)source "fs/exfat/Kconfig"
RaspberryPi 2
mkdir -p ../pi2
kernel="kernel"
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcmrpi_defconfig
Edit .config
CONFIG_EXFAT_FS=m
CONFIG_EXFAT_DISCARD=y
CONFIG_EXFAT_DELAYED_SYNC=n
CONFIG_EXFAT_KERNEL_DEBUG=n
CONFIG_EXFAT_DEBUG_MSG=n
CONFIG_EXFAT_DEFAULT_CODEPAGE=437
CONFIG_EXFAT_DEFAULT_IOCHARSET="utf8"
Compile modules
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules
Install files
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=../pi2 modules_install
RaspberryPi 3
mkdir -p ../pi3
kernel="kernel7"
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig
Edit .config
CONFIG_EXFAT_FS=m
CONFIG_EXFAT_DISCARD=y
CONFIG_EXFAT_DELAYED_SYNC=n
CONFIG_EXFAT_KERNEL_DEBUG=n
CONFIG_EXFAT_DEBUG_MSG=n
CONFIG_EXFAT_DEFAULT_CODEPAGE=437
CONFIG_EXFAT_DEFAULT_IOCHARSET="utf8"
Compile modules
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules
Install files
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=../pi3 modules_install
Prepare archive
cp ../pi2/lib/modules/4.14.34/kernel/fs/exfat/exfat.ko exfat_4.14.34.ko
cp ../pi3/lib/modules/4.14.34-v7/kernel/fs/exfat/exfat.ko exfat_4.14.34-v7.ko
tar cvf exfat_4.14.34.tar exfat_4.14.34*.ko
Imdisk Installer
-
/dev/imdiskinst.exe
368K
Stock copy of ImDisk 2.0.9 installer
Exfat-fuse
Static version of exfat-fuse binaries.
Compilation instructions
sudo apt install autoconf automake pkg-config libfuse-dev gcc make
wget https://github.com/relan/exfat/archive/master.zip
unzip master.zip
cd exfat-master
FUSE_LIBS="/usr/lib/x86_64-linux-gnu/libfuse.a /usr/lib/x86_64-linux-gnu/libulockmgr.a -ldl -pthread" ./configure --prefix=`pwd`/dist
make
make install
cd dist/sbin
tar cf exfat-fuse-1.2.8-linux_x86_64.tar mount.exfat-fuse mkexfatfs exfatlabel exfatfsck dumpexfat
Content-related (ansiblecube)
Most content downloads are available through our mirror.
Note: When creating tar
archive, exclude symlinks (and hard links) as those are not supported on ExFAT.
find . -type l
find . -type l | while read f; do /bin/cp -rf --remove-destination $(readlink -f $(find . -wholename "${f}")) "${f}";done;
/other/kalite
)
Ka-Lite (-
videos_en_0.17.tar
44G
-
videos_fr_0.17.tar
11G
-
videos_es_0.17.tar
21G
-
langpack_en_0.17.zip
887M
-
langpack_fr_0.17.zip
97M
-
langpack_es_0.17.zip
89M
Language packs are copies of the stock KA-Lite version available here.
Videos tar files are archives of the extracted videos (folder) available as torrents.
Language packs and videos archives are tied to specific version of ka-lite.
/other/aflatoun
)
Aflatoun (Aflatoun is only available from this mirror. While it is based on kalite, it includes numerous undocumented changes to the kalite
code, templates and resources.
-
ka-lite-static-0.16.7b3.tar.gz
55M
-
content.tar.gz
3.3G
-
fr.zip
944K
-
en.zip
755K
/other/wikifundi
)
Wikifundi (Wikifundi is based on Kiwix-specific archives for both parsoid and the content. The content package contains the PHP code (mediawiki), configuration and the database.
-
parsoid_2016-08.tar.bz2
48M
-
en.africapack.kiwix.org_2017-01.tar.bz2
4.8G
-
fr.africapack.kiwix.org_2017-01.tar.bz2
3.9G
fr.africapack.kiwix.org_2017-01.tar,bz2
is made from fr.africapack.kiwix.org_2017-01.tar.xz
. It's a bzip2-compressed version which also renames the (14) files/folders which are incompatible with ExFAT.
tar xf fr.africapack.kiwix.org_2017-01.tar.xz
pax -M norm -rw -s '/[?<>\\:*|\"]/_/gp' fr.africapack.kiwix.org target-folder
cd target-folder
tar -cjf fr.africapack.kiwix.org_2017-01.tar.bz2 fr.africapack.kiwix.org
Installation instructions on github.