重複ファイルの整理: what it does and where it breaks down

A duplicate scanner reports 18,000 redundant files. Another one, run on the same folder an hour later, reports 63. Neither is broken. They are answering different questions, and the question they answer was decided before the scan started, by whoever wrote the matching rule.

That is the part of duplicate cleanup that rarely gets stated plainly. The hard problem is not finding identical bytes. Standard tools have done that reliably for decades. The hard problem is that "duplicate" names at least four distinct situations on an APFS volume, and deleting each one has a different consequence. Some free space. Some free nothing. One of them deletes a file that was never a copy at all.

Four situations, one word

Before any tool runs, it helps to separate what is actually on disk.

Same name, different content. Two files called invoice.pdf sitting in different folders, covering different months. A name based matcher flags these. Deleting one loses a document.

Same content, different name. The common case. An archive extracted twice, an attachment saved from two separate emails, a project folder copied before a risky edit. Deleting the redundant copy returns its full size.

A clone. APFS can present two independent files that share a single copy of the underlying data. Both look like ordinary files. Deleting one returns almost nothing.

A hard link or an alias. One body with several doors, or a signpost pointing at a body. Applications and backup systems create these on purpose. Deleting them can break whatever was pointing at them.

Hard links and aliases behave differently from each other and the difference matters. A hard link is another name for the same data, so opening any name returns the same content and removing one name leaves the data intact. An alias or symbolic link is a pointer, so it stops working the moment its target is gone. The link count in a file's metadata distinguishes them: a count above one means the data carries more than a single name.

Counting by name and counting by content disagree by two orders of magnitude

The gap between matching rules is not a rounding difference. A Downloads folder holding 31,263 files across 91 GB was measured both ways.

Matching rule	Files flagged
Filename carries a copy marker such as "2", "copy" or a parenthesised number	63
Content hash matches another file exactly	18,054

The name based rule found roughly 0.3 percent of what the content based rule found. The redundant bytes came to 9.53 GB, a little over a tenth of the folder.

The reason is structural. Most of the ways a file gets duplicated do not touch its name. Extract an archive into a second location and every filename inside is unchanged. Copy a project directory and not one name gains a suffix. Sync the same camera card to two folders and the names are identical by design. The "copy" suffix appears only when two files collide inside a single folder, which is the least common way duplication happens.

This has a direct implication. Anyone who has run a name based cleaner and concluded that their disk is reasonably tidy reached that conclusion from the matching rule, not from the disk.

File count and recovered space are different problems

A scan result reports two numbers, and they point in different directions often enough that it is worth separating them at the start.

In the measured folder, 18,054 redundant files accounted for 9.53 GB, which works out to an average of roughly 0.53 MB each. Those files fell into 5,305 groups of matching content, so a typical group held three or four copies of something small. That shape is normal for a Downloads folder: extracted archives contribute thousands of tiny assets, while the handful of large installers and video files contribute most of the recovered bytes.

The consequence is that the two goals need different actions. Reclaiming space means sorting results by size and dealing with the top of the list, which is usually a few dozen decisions. Reducing clutter means dealing with the long tail, which is thousands of decisions and is rarely worth making one at a time. Treating them as one task produces a session that ends with a lot of clicking and very little free space.

It also changes how a result list should be read. A tool that reports "18,054 duplicates found" is stating the clutter figure. A tool that reports "9.53 GB recoverable" is stating the space figure, and that number can itself be optimistic if part of it sits in clones. Neither headline is wrong, but they answer different questions, and the answer worth acting on depends on whether the disk is full or the folder is unusable.

One more distinction sits inside the space figure. Redundant bytes are not the same as recoverable bytes, because whichever copy gets kept still occupies its size. A group of four identical 1 GB files holds 4 GB and offers 3 GB back. Reports that quote total matched size rather than total redundant size overstate the result by the size of one copy per group, which on thousands of small groups is a meaningful gap.

Some duplicates cost nothing to keep

Clones are the reason a cleanup session can report gigabytes recovered while free space barely moves.

A 200 MB file was copied two ways on an APFS volume, and free space was measured before and after each copy.

How the copy was made	Disk consumed
Cloned	4 KB
Ordinary copy	about 200 MB

The awkward part is that nothing in the file listing separates the two results. Both files report a logical size of 209,715,200 bytes. Both report 409,600 allocated blocks. The disk usage command reports 200 MB for the clone as confidently as for the real copy. Only the volume's free space, sampled before and after, tells the truth.

Clones arrive through ordinary actions. The Finder's Duplicate command produces one. So does cp -c. Virtual machine images and some backup tools rely on cloning to keep generations cheap.

There is a further wrinkle. A clone shares storage only until one side is written to, at which point the modified regions separate onto their own blocks. Two files that were clones in January may hold substantial independent data by June. So "it is a clone, deleting it is pointless" is not a rule that holds over time. It is a snapshot of a relationship that drifts.

Local snapshots add a second delay. While a deleted file is still held in a snapshot, its blocks are not released. Free space can stay flat right after a cleanup and rise later, which reads as a failed deletion when it was only a deferred one.

Identical content does not mean interchangeable

Once a tool has proven two files hold the same bytes, it is tempting to treat the choice between them as arbitrary. macOS attaches information to a file that lives outside its content, and that information does not match.

Two files were created with identical content, and one was marked as having arrived through a browser download. The content hashes stayed identical. The extended attributes did not. Opening the marked copy raises a confirmation dialog. Opening the unmarked copy does not. Same bytes, different behaviour.

The same asymmetry shows up in several places:

• Creation and modification timestamps. The later save looks newer, even though the content is the same age.
• Finder tags and colour labels. Tagging applied for organisation belongs to one file, not to its copy.
• Anything holding a path. Scripts, backup exclusion rules and recent-items lists point at a specific location, which may well be the location about to be deleted.

The practical consequence is a reframing. The decision is not which file to delete. It is which file to keep, and that rule has to exist in words before a list of thousands appears on screen. Without it, the choice is made one row at a time by whichever path happens to be highlighted.

The volume already prevents duplicates people go looking for

A common assumption is that filename variation quietly multiplies files. On a default macOS volume, several of those variations cannot produce two files at all.

Creating Report.pdf and then report.pdf in the same folder leaves exactly one file. The second write lands on the first. The default APFS volume is case insensitive, so a case-differing pair never coexists to be found.

Japanese filenames behave the same way. A character carrying a voiced mark as a single code point and the same character written as a base plus a combining mark are different byte sequences, but APFS treats them as one name. Writing the second form opens the existing file rather than creating a sibling.

The takeaway is where not to spend effort. Filename variants are largely handled by the filesystem. The files worth hunting are the ones whose names share nothing at all while their contents match byte for byte.

Similar is a separate question from identical

Photo and music libraries hold a kind of overlap that exact matching will never surface. Burst shots of the same moment, the same image exported twice at different quality settings, the same track at two bitrates. One differing byte changes the hash, so none of these appear in a content-match result.

Answering that question requires comparing perceptual features rather than bytes, and it changes the shape of the decision. Exact matching returns a yes or a no. Similarity matching returns a score, and somebody has to set the threshold at which a score counts as the same picture. That is not a judgement to hand to an unattended deletion pass.

Photo libraries carry a second hazard. The library is a package containing tens of thousands of files, presented as a single item. Scanning inside it surfaces large numbers of thumbnails and derivative renders that are genuinely identical and genuinely required. Deleting them by hand damages the library. Duplicate photos belong to the photo application's own duplicates view, which understands which file is the master. Application bundles have the same structure and the same rule.

What to settle before running anything

Decide the matching rule, the scope and the keep rule first, then pick a tool that can express all three. Doing it the other way round means reacting to a number that was produced by someone else's definition. A comparison of how different tools match and how far they reach is laid out in Compared with other file managers, and the range of file operations available on a result list is covered in Features, which matters because confirming a match and acting on it are the same workflow. If the plan involves photos arriving from a phone, the import paths described in From iPhone and iPad are worth checking, since that is where most near-identical files enter. Being able to hash a candidate and inspect it in the same window as the file list, rather than switching applications, is the practical difference a Atriens style setup makes to that loop.