Pyinstaller apparently breaks LZMA
Problem:
Unarchiving fr.africapack.kiwix.org_2017-01.tar.bz2 using subprocess.check_call(['/bin/tar', ...])
on linux fails.
--> Processing Wikifundi
xz: /tmp/_MEI4QTM6f/liblzma.so.5: version `XZ_5.2' not found (required by xz)
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Installation failed: Command '['tar', '-C', '/media/reg/data/tmps9u6t1k2', '-x', '-f', './cache/parsoid_2016-08.tar.bz2']' returned non-zero exit status 2
Reason:
-
fr.africapack.kiwix.org_2017-01.tar.bz2
is not bzip2 compressed but LZMA (misleading filename) -
tar
deffer toxz
to uncompress. -
xz
usesliblzma.so.5
- pyinstaller bundles (and loads) its own
liblzma
(named_lzma.so
in the archive). - pyinstaller's
liblzma
is too old (< 5.2) to uncompress this file.
Workaround:
LD_PRELOAD=/lib/x86_64-linux-gnu/liblzma.so.5 ./pibox-installer
.
A possible fix might be to:
- Bundle a recent version ourselves.
- Load this version using
ctypes
so thatxz
uses the already loaded lib in memory.
if sys.platform == "linux" and getattr(sys, "frozen", False):
import ctypes
ctypes.cdll.LoadLibrary(os.path.join(sys._MEIPASS, 'liblzma.so.5'))
WONTFIX now as wikifundi is the only LZMA archive we have and needs to be recreated anyway:
- has incorrect file names for ExFAT (14) which fails extraction.
-
xz
compression has almost no gain compared to plain tarball on this folder.
Creating this issue should we need LZMA in the future.