Master FileSystem Redesign - Architecture Review Opinion
- Issue: 51551
- Submitter: Radek Matous
- Inception Date: 2004/11/24
Final Date: 2005/01/04 - Reviewers: Jaroslav Tulach, Trung Duc Tran, Tomas Pavek, Pavel Buzek
- History: available in CVS
Summary
The master file system (originally reviewed as issue 40958) started to be an important part of NetBeans 4.0. This prominent possion resulted in its increased usage and discovery of certain flaws that should be addressed in order to address different operating system specifics and make it more usable and reliable on various operating systems.
The current flaws include UNC problems on Windows, automount of /net on unix systems, symlinks and other ways how to get access to the same physical path by different logical names. The goal is to fix them for promotion E (while we will need to keep support for VCS filesystem) as well as promotion F (where the VCS filesystem may no longer be needed). Details on the goals can be found in [1].
Decision
Accepted with one TCA.Opinion
The need for masterfs
The original need for masterfs (to plug into itself other filesystems) seems no longer be true or will not be true in near future. ThemountingAPI has friend stability and is being used only by VCS, which will very likely soon switch its functionality to non-filesystem based implementation. Then the mounting API can completely disappear (probably promotion F).
However there still seems to be good reasons for existance of the masterfs module. It cleanly separates the filesystem APIs (org.openide.filesystems) from the implementation and allow us to solve problems that would be hard to address in the org.openide.filesystems part (like possible usage of JNI, less stable SPI interfaces, etc.).
The future direction is to move most of the java.io.File file object wrapping
implementation into the masterfs module and slowly discontinue the use of SPI classes
as AbstractFileSystem, LocalFileSystem, etc.
java.io.File relationship
The goal for masterfs has always been to provide one to one mapping withjava.io.File, this is still a reasonable goal as it fits
into the spirit of filesystem APIs (so we do not need to change it) and satisfies
most of our needs. There however exists a need to provide certain improvements with
respect to file uniquiness to prevent data loss. These should be addressed
by improving the locking mechanism.
The reviewers agreed that the team shall write a new implementation
of a filesystem for local files
independent from AbstractFileSystem that will not share
AbstractFileSystem troubles. This solves problems with /net
and to certain extent addresses the high memory usage
(see advisory information [1]).
At the time of promotion E, the MasterFileSystem implementation
shall remain the same (allow pluging of VCS filesystems), just instead of
the old org.openide.filesystem.LocalFileSystem let it use the
new implementation. In promotion F (when the mounting will no longer be needed)
a merge of the local fileobjects with master fs ones, further lowering the memory
requirements, shall happen.
Windows UNC problems
In some sence the UNC files are special (one cannot list them in advance as A:/, C:/ and other roots), in some ways they just make certain problems more visible (a FileObject created on a laptop in work may not be found on the laptop running at home [advisory section [2]).That is why the UNC roots should appear dynamically as chidren of the root of master filesystem when accessed. They can be represented by the newly implemented filesystem for local files or its subclass, in order to separate the UNC only hacks from the rest of java.io.File wrapping code. However the team is requested to make sure that problems are filled and that the JDK team is properly notified about importance of these issues (talk to Trung about that).
The identity problem
Resolving the symlinks problem by delegation of certain operation to a cannonical FileObject looks like a briliant idea. However in certain environments this is not sufficient as logical paths can lead to the same file even without using symlinks. This can cause possible data loss as the same file can still be opened in editor twice and edited more times at once.
There were a lot of ideas how to solve the problem from relaying on
File.getCanonicalFile (known to be unsuffitient), to using
java.nio.channels.FileLock (could possibly prevent other
applications like ant to access the file - see advisory section
[3]), using JNI (we do not want to do it), exposing
some SPI for others to provide better ways to discover the identity (possible
in future), or simulating what other editors do.
The reviewers suggest to try the last option and simulate emacs (advisory section [4]) for now and see if this is enough. If found unsufficient, other ways can be explored later. The possible problems found till now include the unability to lock a file this way if one does not have permition to create the lock file in a directory (just lock it without the file) or prevent the locking of a file if the file is itself referenced by symlink (try to traverse the symlink, or just do not solve this), but none of them seems to be too critical to not choose this solution.
As a result the file will get opened in editor more times, but when user types in both of them, he will be notified (beep, message in status line) that the file is already locked.
Minority Opinion
In the email discussion two opinions arised:- Isao did not like the fact that arbitrary non-file based implementation
of a filesystem will no longer be allowed to be pluged in the masterfs. After
Jesse's explanation, this issue has not appeared anymore.
See http://www.netbeans.org/servlets/ReadMsg?msgId=867333&listName=nbdev
and http://www.netbeans.org/servlets/ReadMsg?msgId=867804&listName=nbdev - Ivan does not seem to be satisfied by the way masterfs prevents one physical file referenced by two different paths to be opened in editor. But as he has not been able to provide changes to this document, here is just few links to his opinions 1, 2, 3.
Advisory Information
nio FileLock and its suitfulness or unsuitfulness to be used in locking files while
discovering their real identity.FileObject.lock() create a temporary file along the original
(its name shall be changable as a private property to accommodate possible
problems on various platforms) and delete the file on lock release. Check
CLIHandlerTest for a possible ways how to write and test code
that needs to synchronize on creation and modification of files between
more independent applications, however the full crash detection and
strict locking between various programs is not the goal number one. The
goal is to make this working internally in one NetBeans based product.
Appendixes
Appendix A: Technical Changes Required
None.
Appendix B: Technical Changes Advised
53801 - test and measure the amount of memory needed to open one file in editor
