Merged requested to merge better-dedup into master
This commit actually does 3 things and might be a bit controversial:
- it introduces a way to test performance of a given function by adding a decorator factory for measuring performance.
- it introduces a photos cache, to prevent useless network requests to fetch pictures (this locally improved the 3rd pass duration from 570 seconds to 70 seconds for my search).
- it decides that two real estate ads are the same if they either have less than 3 pics and they're all the same, or at least 3 common pics. I don't have data backup this up, but it seemed to work great in my case.
Happy to split this in three smaller commits and/or drop changes that are too controversial to be upstreamed.