Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
ARCHIVED Kiwix Hotspot
ARCHIVED Kiwix Hotspot
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 45
    • Issues 45
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • ideascube
  • ARCHIVED Kiwix HotspotARCHIVED Kiwix Hotspot
  • Wiki
  • support files

Last edited by rgaudin Aug 02, 2018
Page history

support files

pibox-installer as well as ansiblecube inside it depends on a number of external files which are downloaded and used during the installation process.

This document is a list of such files, their use and how to create/update them when that's available.

Bare copies

Following are files publicly available from their upstream source which we copied into our mirror to prevent network issues during builds (CI or image).

  • /dev/qemu-w32-setup-20180430.exe
  • /dev/qemu-w64-setup-20180430.exe
  • /dev/7z920_extra.7z
  • /dev/7z1805-extra.7z
  • /dev/pygi-aio-3.18.2_rev12-setup_549872deadabb77a91efbc56c50fe15f969e5681.exe
  • /dev/parsoid_0.9.0.tar.gz
  • /dev/mediawiki-1.31.0-rc.0.tar.gz
  • /dev/composer_1.6.5.phar

QEMU kernel (pibox)

  • /dev/pibox-installer-vexpress-boot.zip

Faster than the stock kernel on modern hosts.

Support up to 4 cores and 30GiB or RAM.

See README.md for details on how to update it.

Static QEMU (pibox)

  • /dev/qemu-2.12.0-linux-x86_64.tar.gz

Used in .gitlab-ci.yml to skip the compilation step of QEMU.

Stock QEMU compiled statically for linux x86_64.

macOS QEMU

  • /dev/qemu-2.12.0_macOS.tar

Used in .travis.yml as part of macOS bundle. Travis would frequently fail to build due to timeouts in brew install or compilation of QEMU.

brew update
brew install qemu
git clone https://github.com/auriamg/macdylibbundler/
cd macdylibbundler/
git checkout 84440587e1b2e4e9e0727cbba7a455b96820e40a
make
cd ..
mkdir qemu
cd qemu
cp /usr/local/bin/qemu-img .
cp /usr/local/bin/qemu-system-arm .
chmod u+w qemu-img
chmod u+w qemu-system-arm
../macdylibbundler/dylibbundler -d . -p @executable_path/ -b -x ./qemu-system-arm  -x ./qemu-img
chmod u-w qemu-img
chmod u-w qemu-system-arm
otool -L *
cd ..
tar cf qemu-2.12.0_macOS.tar qemu/

Python 3.6 formulas for brew

  • /dev/brew-py36.zip

On travis, Python needs to be installed via brew (for pygobject3). Brew does not provide a way to pick the python version and always uses the latest stable one (3.7.0 now).

Since pyinstaller is not yet 3.7 compatible, we have to use python 3.6.5 and, for that, change the brew formulas of python and some dependencies.

We used to do a git checkout into `` but now that travis updated its base macos image, this commit is beyond the limited history.

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
zip brew-py36.zip Formula/python.rb Formula/pygobject.rb Formula/gtk+.rb Formula/glib.rb Formula/adwaita-icon-theme.rb

GoRC

  • /dev/GoRC.exe

Stock copy of the binary (GoRC.exe) from http://www.godevtool.com/Gorc.zip

Used to compile resource file for Windows.

Custom ideascube build (ansiblecube)

  • /dev/ideascube_0.37.2_armhf.deb

A deb of ideascube with our patches:

  • move zim from --package-cache instead of copying.
  • --skip-sha256 option.
  • Debian stretch package (python3.5).

Use rgaudin/ideascube until those patches gets integrated upstream.

xz 5.2.4

  • /dev/xz-5.2.4.tar.gz

Used only in linux's build .travis.yml: the build system is trusty (14.04) due to the py34 requirement but trust's LZMA library is too old. See #176 (closed).

Stock copy of lzmautils's distribution on sourceforge. SF raises frequent download issues from travis.

Exfat-fuse (ansiblecube)

  • /dev/exfat-utils_1.2.8-1_armhf.deb

Stock copy of debian bower package as stretch version is obsolete and fetching directly from archive sometimes fail.

Exfat Kernel modules (ansiblecube)

  • /dev/exfat_4.14.34.tar

A pair of exfat-nofuse kernel modules for the raspberryPi.

  • Compiled against the same kernel tree as the raspbian kernel in use.
  • Archive contains module for the two rPi flavors: regular (Pi2) and -v7 (Pi3)

Creation steps

You can alternatively compile the modules on raspberry hardware running the same raspbian version you are targetting. You will need to do it twice thought (first on a Pi2 then on a P3) and copy the modules from there.

The following instructions are for a cross-compilation.

  • Get kernel commit from .info file of raspbian-liste curl http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2018-04-19/2018-04-18-raspbian-stretch-lite.info

  • Download the raspberry version of the kernel for that commit

wget https://github.com/raspberrypi/linux/archive/f70eae405b5d75f7c41ea300b9f790656f99a203.zip
unzip f70eae405b5d75f7c41ea300b9f790656f99a203.zip
mv linux-f70eae405b5d75f7c41ea300b9f790656f99a203/ linux-4.14.34
cd linux-4.14.34
  • Download exfat-nofuse and place it in tree
wget https://github.com/dorimanx/exfat-nofuse/archive/master.zip
unzip master.zip
mv exfat-nofuse-master/ fs/exfat
  • Add exfat to fs/Makefile obj-$(CONFIG_EXFAT_FS) += exfat/
  • Add exfat to fs/Kconfig (in filesystems section) source "fs/exfat/Kconfig"

RaspberryPi 2

mkdir -p ../pi2
kernel="kernel"
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcmrpi_defconfig

Edit .config

CONFIG_EXFAT_FS=m
CONFIG_EXFAT_DISCARD=y
CONFIG_EXFAT_DELAYED_SYNC=n
CONFIG_EXFAT_KERNEL_DEBUG=n
CONFIG_EXFAT_DEBUG_MSG=n
CONFIG_EXFAT_DEFAULT_CODEPAGE=437
CONFIG_EXFAT_DEFAULT_IOCHARSET="utf8"

Compile modules

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules

Install files

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=../pi2 modules_install

RaspberryPi 3

mkdir -p ../pi3
kernel="kernel7"
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig

Edit .config

CONFIG_EXFAT_FS=m
CONFIG_EXFAT_DISCARD=y
CONFIG_EXFAT_DELAYED_SYNC=n
CONFIG_EXFAT_KERNEL_DEBUG=n
CONFIG_EXFAT_DEBUG_MSG=n
CONFIG_EXFAT_DEFAULT_CODEPAGE=437
CONFIG_EXFAT_DEFAULT_IOCHARSET="utf8"

Compile modules

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules

Install files

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=../pi3 modules_install

Prepare archive

cp ../pi2/lib/modules/4.14.34/kernel/fs/exfat/exfat.ko exfat_4.14.34.ko
cp ../pi3/lib/modules/4.14.34-v7/kernel/fs/exfat/exfat.ko exfat_4.14.34-v7.ko
tar cvf exfat_4.14.34.tar exfat_4.14.34*.ko

Imdisk Installer

  • /dev/imdiskinst.exe 368K

Stock copy of ImDisk 2.0.9 installer

Exfat-fuse

  • /dev/exfat-fuse-1.2.8-linux_x86_64.tar

Static version of exfat-fuse binaries.

Compilation instructions

sudo apt install autoconf automake pkg-config libfuse-dev gcc make
wget https://github.com/relan/exfat/archive/master.zip
unzip master.zip
cd exfat-master
FUSE_LIBS="/usr/lib/x86_64-linux-gnu/libfuse.a /usr/lib/x86_64-linux-gnu/libulockmgr.a -ldl -pthread" ./configure --prefix=`pwd`/dist
make
make install
cd dist/sbin
tar cf exfat-fuse-1.2.8-linux_x86_64.tar mount.exfat-fuse mkexfatfs exfatlabel exfatfsck dumpexfat

Content-related (ansiblecube)

Most content downloads are available through our mirror.

Note: When creating tar archive, exclude symlinks (and hard links) as those are not supported on ExFAT.

find . -type l
find . -type l | while read f; do /bin/cp -rf --remove-destination $(readlink -f $(find . -wholename "${f}")) "${f}";done;

Ka-Lite (/other/kalite)

  • videos_en_0.17.tar 44G
  • videos_fr_0.17.tar 11G
  • videos_es_0.17.tar 21G
  • langpack_en_0.17.zip 887M
  • langpack_fr_0.17.zip 97M
  • langpack_es_0.17.zip 89M

Language packs are copies of the stock KA-Lite version available here.

Videos tar files are archives of the extracted videos (folder) available as torrents.

Language packs and videos archives are tied to specific version of ka-lite.

Aflatoun (/other/aflatoun)

Aflatoun is only available from this mirror. While it is based on kalite, it includes numerous undocumented changes to the kalite code, templates and resources.

  • ka-lite-static-0.16.7b3.tar.gz 55M
  • content.tar.gz 3.3G
  • fr.zip 944K
  • en.zip 755K

Wikifundi (/other/wikifundi)

Wikifundi is based on Kiwix-specific archives for both parsoid and the content. The content package contains the PHP code (mediawiki), configuration and the database.

  • parsoid_2016-08.tar.bz2 48M
  • en.africapack.kiwix.org_2017-01.tar.bz2 4.8G
  • fr.africapack.kiwix.org_2017-01.tar.bz2 3.9G

fr.africapack.kiwix.org_2017-01.tar,bz2 is made from fr.africapack.kiwix.org_2017-01.tar.xz. It's a bzip2-compressed version which also renames the (14) files/folders which are incompatible with ExFAT.

tar xf fr.africapack.kiwix.org_2017-01.tar.xz
pax -M norm -rw -s '/[?<>\\:*|\"]/_/gp' fr.africapack.kiwix.org target-folder
cd target-folder
tar -cjf fr.africapack.kiwix.org_2017-01.tar.bz2 fr.africapack.kiwix.org

Installation instructions on github.

Clone repository
  • Keeping Time
  • System Requirements
  • faq
  • Home
  • support files