Package download fails with IncompleteRead
Occasionally, some packages (not the same every time) would fail to download with an IncompleteRead
error.
See https://github.com/kiwix/cardshop/issues/84
It's an urllib3
error raises through requests
that we use.
[09:44:13] [STAGE 2/5: Download contents - 20%]
[09:44:13] --> Starting all content downloads
[09:44:13] --> Retrieving package_wikipedia_ar_all_novid.ar-2018-10-08.zim (8.28 GiB)
[ ] 0%
[..........................................................] 100%
Saved `package_wikipedia_ar_all_novid.ar-2018-10-08.zim` successfuly: 8.28 GiB
[09:49:02] [STAGE 2/5: Download contents - 22%]
[09:49:02] --> Retrieving package_icd10_fr_all.fr-2012-01-09.zim (409.8 KiB)
[ ] 0%
[..........................................................] 100%
Saved `package_icd10_fr_all.fr-2012-01-09.zim` successfuly: 409.8 KiB
[09:49:02] [STAGE 2/5: Download contents - 22%]
[09:49:02] --> Retrieving package_gutenberg_fr_all.fr-2018-10-22.zim (2.72 GiB)
[ ] 0%
[..........................................................] 100%
Saved `package_gutenberg_fr_all.fr-2018-10-22.zim` successfuly: 2.72 GiB
[09:50:56] [STAGE 2/5: Download contents - 22%]
[09:50:56] --> Retrieving package_wiktionary_fr_all_nopic.fr-2018-07-14.zim (1.96 GiB)
[ ] 0%
[..........................................................] 100%
Saved `package_wiktionary_fr_all_nopic.fr-2018-07-14.zim` successfuly: 1.96 GiB
[09:52:22] [STAGE 2/5: Download contents - 22%]
[09:52:22] --> Retrieving package_universcience-tv_fr_all.fr-2015-03-20.zim (13.07 GiB)
[ ] 0%
[..........................................................] 100%
[09:56:08] Error downloading http://download.kiwix.org/zim/other/universcience-tv_fr_all_2015-03.zim to /data/kiwix-hotspot.cache/package_universcience-tv_fr_all.fr-2015-03-20.zim
('Connection broken: IncompleteRead(706 bytes read, 318 more expected)', IncompleteRead(706 bytes read, 318 more expected))
[09:56:08] Installation failed: ('Connection broken: IncompleteRead(706 bytes read, 318 more expected)', IncompleteRead(706 bytes read, 318 more expected))
--- Exception Trace ---
Traceback (most recent call last):
File "site-packages/urllib3/response.py", line 360, in _error_catcher
File "site-packages/urllib3/response.py", line 669, in read_chunked
File "site-packages/urllib3/response.py", line 615, in _handle_chunk
File "http/client.py", line 693, in _safe_read
http.client.IncompleteRead: IncompleteRead(706 bytes read, 318 more expected)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "site-packages/requests/models.py", line 750, in generate
File "site-packages/urllib3/response.py", line 490, in stream
File "site-packages/urllib3/response.py", line 694, in read_chunked
File "contextlib.py", line 77, in __exit__
File "site-packages/urllib3/response.py", line 378, in _error_catcher
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(706 bytes read, 318 more expected)', IncompleteRead(706 bytes read, 318 more expected))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run_installation.py", line 222, in run_installation
File "backend/download.py", line 186, in download_file
File "backend/download.py", line 157, in stream
File "site-packages/requests/models.py", line 753, in generate
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(706 bytes read, 318 more expected)', IncompleteRead(706 bytes read, 318 more expected))
Edited by rgaudin