Vanishing files in NFS mounts

After yesterday’s WWDC session on Network Home Directories, two different people asked me if I had seen an issue with NFS mounts where a user would have a folder open, showing contents of items on an NFS share somewhere, and when they’d click on a item in the Finder window, it would just disappear.

As it turns out, we did see this issue about a year ago. It was quite confusing and frustrating. It seemed to be a Finder issue, because the items were still visible and accessible in the Terminal. We found that relaunching the Finder would cause the folder to regain all its items, but that was rather annoying. Finally, by accident, I discovered the cause and came up with a workaround.

First, a digression. In most places that use NFS shares, they are mounted via an automounter – autofs on most other UNIX-y platforms, and Apple’s automount on OS X Tiger and earlier. Automounters do not mount all the NFS shares they know about; they only mount them on demand – that is, as a path is accessed, it may trigger the automounter to mount the NFS share and make it available to the client. When the mount has been idle for some amount of time, the automounter quietly unmounts the NFS share. If the share is needed again in the future, the automounter simply remounts it. This generally works well.

Note that I mentioned that the automounter will unmount an idle share after some amount of time – this time is configurable. The default time in OS X Tiger is 3600 seconds, which is one hour. I had noticed that our Linux boxes were all configured to umount idle mounts after 900 seconds, or 15 minutes. When possible, I like to configure our Macs to behave similarly to our Linux boxes to make it easier for everyone. So I globally reconfigured all our Macs to unmount idle NFS mounts after 15 minutes.

Within a day, the phone calls to the help desk complaining about disappearing items in Finder windows had increased greatly. I sensed a correlation. I spoke to a few users, and reset their automount timeouts to an hour. They reported that the problem decreased noticably.

With these observations, I surmised that the Tiger Finder simply did not deal well with the automounter unmounting NFS shares. The workaround is to increase the “time-to-live” value for the automount – this is the “-tl” flag. I set it back to the default of 3600 seconds, you could set it higher if you wish.

Note that this workaround won’t actually prevent the problem from occurring – it will just reduce its occurrence.

Vanishing files in NFS mounts

16 thoughts on “Vanishing files in NFS mounts

  1. thegooch says:

    I have seen this issue on serveral machines. I can’s say that ‘time-to-live’ is a universal solution. I have lots of macs that mount NFS shares via /etc/fstab (like unix). Automount is not involved on our systems (nor is ‘mounts’ in NetInfo). The fstab is read at bootup, and all shares are available. After a while of use, I get the vanishing files problem. This is not on all machines, but does happen from time to time.

    Again, the terminal shows all the files normally, but the finder displays vanishing folders after some time of use. At this time, I still don’t have a solution. Machines have up to date software.

  2. Dan says:

    I have had files disappear with users mounting SMB shares off of Windows 2003 server for no reason.

    But the kicker is they get deleted….you can’t see them through the Terminal or on the Windows server showing hidden files, etc.

    I haven’t heard anyone complain about it lately, but I’m not sure whether its still happning or not. Alot of our users have started to use USB Flash drives and have just quite using their storage space provided for them because they have more on the flash drives.

  3. Dylan says:

    We see a similar problem where folders will either disappear but also we get multiple folders with the same name. All is OK in the terminal.

  4. brent says:

    We are also having this problem. So far it seems that it is only happening on intel machines vs PPCs. Can anyone corroborate that?

  5. Ben says:

    I’ve been seeing this a lot with a PPC Mac Mini. This machine is in my son’s bedroom, with his home account on an NFS share. He usually puts it to sleep when he’s not using it – not normally a problem (other than a delay when waking up as the mounts are restored), and I haven’t seen any obvious correspondence between the disappearing files and sleeping. It’s a real pain in the backside though – especially as the most often “invisible” file is the iTunes db. This suggests the problem is not the Finder itself, but rather some library routine/process shared by both iTunes and the Finder. I’ve seen a Word file disappear in the Finder, but be perfectly visible in MS Word’s Open dialog. i.e. Word (and presumably the Terminal) isn’t using the problematic routine.

    I haven’t seen this on other NFS accounts, but they’re not used as heavily.

  6. Ben says:

    The ‘ttl=0’ mount option hasn’t hasn’t made any difference. On further reflection, I don’t believe this is an automounter issue – only certain files disappear, others within the same folder are fine.

  7. Dylan B says:

    Dan,

    Did you find a solution to the files disappearing off Win Sever 2003?
    I have had major dramas with this at one site. Whole folders disappear off the server and a NAS box i have installed.

    Thanks

    Dylan

  8. We have the problem here too: vanishing folders and/or multiplying folders. Sometimes they just vanish when you click on them other times new folders will multiply in the same directory with the exact same folder name; some times both will happen at once in the same directory. And at the same time the affected folders behave normally in the terminal.

    We do not use automount at all. We mount nfs shares from netapp filers via /etc/fstab on all our Tiger Macs.

    The problem reoccurs 2-3 times a month for some users and never for others. All users are more or less using the same directories. Some of the affected directories have been around for 6 months or so, others are brand new. I have not seen any concrete patterns yet for reoccurrence but the affected machines have not been rebooted for at least week between occurrences.

    I have seen the problem on Intel MacMinis, Intel iMacs, and a Mac Pro.

    When the problem is manifest a ‘force quit’ of the Finder doesn’t make the problem go away. In fact, while logged in the terminal as local root I have seen affected machines not able to execute a “umount -a” of the affected file system either (you get a “resource busy” error). Other file systems unmounted with the umount command but the affected nfs share hosting the vanishing/multiplying folder would not unmount. And yes, all apps where quit, all finder windows where closed, and, for good measure, the finder was ‘force quit’ again too. Rebooting was the only way to recover from the vanishing/multipling folder problem. And this should not be surprising since the affected file share was forced to umount for the reboot.

    Franklin

  9. I got into contact with a guy that logged a bug on this issue with via Apple Developer Connection (bug rdar://5089077). This what he told me:

    “They resolved it as fixed in 10.5, which makes sense given that they
    completely replaced automount with autofs. This is a big improvement
    (autofs isn’t single-threaded and hang-prone!) but a bit frustrating
    for us since it forces us into a paid upgrade just to get the system
    to work reliably (10.5 also appears to fix the crashes under heavy
    load and NFS freezes which have plagued our Mac labs).

    The best we’ve been able to do add the “-s -tl 0″ option for automount
    in the NFS StartupItem, which lowers the incidence rate to the point
    where it’s tolerable but not what I would consider truly fixed. This
    meant that we had to put some defensive coding into various scripts
    which basically waits for the link to be recreated – not exactly the
    most satisfying code I’ve ever written but it works. ”

    So, it looks like 10.5 may have fixed this. I have not tested it myself yet but i will post again once I have.

    Franklin

  10. Dan says:

    Dylan,

    I haven’t heard anyone complain about the issue since we moved to Tiger…..but alot of people are using USB flash drives now and use those mostly. But I haven’t heard anyone complain in a while.

    But like the other post…10.5 is supposed to fix the issue.

    -Dan

  11. Sam says:

    Sorry guys, but I’m in an educational establishment and I’m seeing this issue only on 10.5. 10.5.3 and 10.5.4 no less. I’m very disappointed with this. You’d think that considering the complete proliferation of windows servers and unix servers that Finder would be able to deal with them better. Thanks Apple😦

    Actually, screw all that. Just make AFP work RELIABLY without crashing every week and I’ll use that!!

  12. Mike says:

    I’m serving files from an Xserve running 10.5.4 over NFS to a mixed bag of linux and OS X clients. Finder seems to be about 99% reliable, What I’m seeing is completely bizarre: my web server (Linux) mounts user homes over NFS (no automount, /etc/fstab) and serves their html directories. When I make any changes– add a file, remove a file, rename a file– to their html directories, the web server shows the directory as empty in a browser, and no amount of refreshing will show the contents.

    That’s weird. The fix is WEIRDER. If you run ‘ls -al’ inside a user’s html directory in Terminal, the contents appear in a browser appropriately. Running a regular ‘ls’ with no flags in Terminal does not help. Only ‘ls -al’. I’m completely stumped. Anyone have any idea what could be causing this odd behavior?

    1. catherine says:

      Probably because ls -al checks the modification date, ownership and permissions as well. I imagine that the mac system wants these before displaying. Using ls just gets the filenames, not the rest.

Comments are closed.