I’ve been working on a Fuse filesystem project for quite some time. Now that OS X v10.9 is out, one of my major roadblocks has been removed. But I’ve still been experiencing some difficulty getting OS X to treat my filesystem like a first-class citizen. So to debug it I’ve been banging on it with Tuxera’s port of Pawel Jakub Dawidek’s POSIX file system test suite.

During this process, I made some baselines of my non-FUSE HFS+ boot volume and I came across a “fun” HFS+ surprise:

An Unremovable File

On an HFS+ filesystem you can make a symlink that can’t (easily) be removed…

# be root, for example with: sudo -i
str1=$(python -c "print '1' * 255")
str2=$(python -c "print '2' * 255")
str3=$(python -c "print '3' * 255")
str4=$(python -c "print '4' * 253")
mkdir -p  $str1/$str2/$str3/$str4
ln -s ftw $str1/$str2/$str3/$str4/L

Now we’ve created a tree that can’t be removed. None of these commands will work on OS X v10.9:

# still as root...
unlink 1*/2*/3*/4*/L
unlink $str1/$str2/$str3/$str4/L
rm -rf 1*
rm -rf $str1
rm -rf $str1/$str2/$str3/$str4
rm -rf $str1/$str2/$str3/$str4/L
(cd $str1/$str2/$str3/$str4; unlink L)
(cd $str1/$str2/$str3/$str4; rm -rf L)

They all boil down to the following error. (Note that I’m abbreviating the path components with “[ … ]” here for blog readability)

root# pwd
/private/tmp/111[ ... ]111/222[ ... ]222/333[ ... ]333/444[ ... ]444
root# ls
L
root# rm -f L
rm: L: No space left on device
root# df -H
Filesystem      Size   Used  Avail Capacity   iused     ifree %iused  Mounted on
/dev/disk1      250G   108G   142G    44%  26385563  34601956   43%   /
[...]

To make sure this is really happening, I’ve made a quick program to make the direct syscall. I’ll place it at /tmp/fixit.c:

#include <unistd.h>
#include <stdio.h>
#include <errno.h>
	
int main(int argc, char* argv[]) {
	printf("Unlink returned %i\n", unlink("L"));
	perror("Error was");
	return 0;
}

Now to run it:

root# pwd
/private/tmp/111[ ... ]111/222[ ... ]222/333[ ... ]333/444[ ... ]444
root# gcc -o /tmp/fixit /tmp/fixit.c 
root# /tmp/fixit 
Unlink returned -1
Error was: No space left on device

ENOSPC? Guess which error you can’t find mentioned in the OS X unlink(2) man page?

It’s complicated:

  • If a regular user creates this tree, they can easily remove it with rm -rf
  • If a regular user creates the tree, root cannot remove it. Weird!
  • If root creates the tree, root cannot remove it
  • If root creates the tree, typical permissions prevent regular users from removing it
  • If root creates the tree, chown/chmod can change the protection of the containing directories so that a regular user can remove the entire tree
  • If root creates the tree, chmod -h and chown -h on the link return ENOSPC
  • If root creates the tree, this works: mkdir -p some/containing/paths; mv 1111* some/containing/paths/ BUT afterward this doesn’t work: rm -rf some (it returns “directory not empty” and ENOSPC errors)

Workaround

Somehow the path is short enough to create a symlink but too long to remove it. The workaround is just to shorten the path.

root# pwd
/private/tmp/111[ ... ]111/222[ ... ]222/333[ ... ]333/444[ ... ]444
root# ls
L
root# mv /private/tmp/1* /private/tmp/one
root# pwd
/private/tmp/one/222[ ... ]222/333[ ... ]333/444[ ... ]444
root# rm L
root# ls
root# rm -rf /tmp/one
root#

Is The Workaround Enough?

So yeah, there’s a manual workaround, but here are some questions:

  • Is your antivirus program smart enough employ it if malware is stored in this manner? AV programs already have to deal with chflags(2) issues. Imagine a file stored in such a path combined with hostile file names and various applications of chflags.
  • Do you have a replacement tool for rm -rf if a malicious program or individual starts to fill your filesystem with these things?