Skip to content
  • Ludovic Courtès's avatar
    daemon: Do not deduplicate files smaller than 8 KiB. · 472a0e82
    Ludovic Courtès authored
    Files smaller than 8 KiB typically represent ~70% of the entries in
    /gnu/store/.links but only contribute to ~4% of the space savings
    afforded by deduplication.
    
    Not considering these files for deduplication speeds up file insertion
    in the store and, more importantly, leaves 'removeUnusedLinks' with
    fewer entries to traverse, thereby speeding it up proportionally.
    
    Partly fixes <https://issues.guix.gnu.org/24937>.
    
    * config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK
    definition.
    * guix/store/deduplication.scm (%deduplication-minimum-size): New
    variable.
    (deduplicate)[loop]: Do not recurse when FILE's size is below
    %DEDUPLICATION-MINIMUM-SIZE.
    (dump-port): New procedure.
    (dump-file/deduplicate)[hash]: Turn into...
    [dump-and-compute-hash]: ... this thunk.
    Call 'deduplicate' only when SIZE is greater than
    %DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'.
    * nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where
    st.st_size < deduplicationMinSize.
    * nix/libstore/local-store.hh (deduplicationMinSize): New declaration.
    * nix/libstore/optimise-store.cc (deduplicationMinSize): New variable.
    (LocalStore::optimisePath_): Return when PATH is a symlink or smaller
    than 'deduplicationMinSize'.
    * tests/derivations.scm ("identical files are deduplicated"): Produce
    files bigger than %DEDUPLICATION-MINIMUM-SIZE.
    * tests/nar.scm ("restore-file-set with directories (signed, valid)"):
    Likewise.
    * tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"):
    New test.
    ("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than
    %DEDUPLICATION-MINIMUM-SIZE.
    * tests/store.scm ("substitute, deduplication"): Likewise.
    472a0e82