It's A Digital Disease!

23 readers

1 users here now

This is a sub that aims at bringing data hoarders together to share their passion with like minded people.

founded 2 years ago

MODERATORS

bOt@zerobytes.monster

Anti-Twin Performs poorly for deduplication. Any better alternatives? (zerobytes.monster)

submitted 2 months ago by bOt@zerobytes.monster to c/datahoarder@zerobytes.monster

0 comments fedilink hide all child comments

The original post: /r/datahoarder by /u/Naernoo on 2025-06-19 19:16:20.

Hi!

I have a large number of images I want to deduplicate. I tried Anti-Twin because it worked out of the box.

However, the performance is really bad. I ran a deduplication scan between two folders and it found about 10 GB of duplicates, which I deleted. Then I ran a second scan, and it found another 2 GB. A third scan found 1 GB, and then another found around 500 MB, and so on.

It seems like it never catches all duplicates in one go. Why is that? I set all limits really high.

Are there better alternatives that don’t have these issues?

I tried using Czkawka a few years ago, but ran into permission errors, missing dependencies, and other problems.

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here