Master FileSystem Redesign - Architecture Review Opinion
- Issue: 51551
- Submitter: Radek Matous
- Inception Date: 2004/11/24
Final Date: 2005/01/04
- Reviewers: Jaroslav Tulach, Trung Duc Tran, Tomas Pavek, Pavel Buzek
- History: available in CVS
- Content
-
The master file system (originally reviewed as
issue 40958) started to be an important part
of NetBeans 4.0. This prominent possion resulted in its increased usage and
discovery of certain flaws that should be addressed in order to address
different operating system specifics and make it more
usable and reliable on various operating systems.
The current flaws include UNC problems on Windows, automount of /net on unix systems,
symlinks and other ways how to get access to the same physical path by different
logical names. The goal is to fix them for promotion E (while we will need
to keep support for VCS filesystem) as well as promotion F
(where the VCS filesystem may no longer be needed). Details on the goals
can be found in [1].
Accepted with one TCA.
The need for masterfs
The original need for masterfs (to plug into itself other filesystems) seems no
longer be true or will not be true in near future. The
mounting
API has
friend stability and is being used only by VCS, which will very likely soon
switch its functionality to non-filesystem based implementation. Then the
mounting API can completely disappear (probably promotion F).
However there still seems to be good reasons for existance of the masterfs
module. It cleanly separates the filesystem APIs (org.openide.filesystems)
from the implementation and allow us to solve problems that would be hard
to address in the org.openide.filesystems part (like possible usage of JNI,
less stable SPI interfaces, etc.).
The future direction is to move most of the java.io.File file object wrapping
implementation into the masterfs module and slowly discontinue the use of SPI classes
as AbstractFileSystem, LocalFileSystem, etc.
java.io.File relationship
The goal for masterfs has always been to provide one to one mapping with
java.io.File, this is still a reasonable goal as it fits
into the spirit of filesystem APIs (so we do not need to change it) and satisfies
most of our needs. There however exists a need to provide certain improvements with
respect to file uniquiness to prevent data loss. These should be addressed
by improving the locking mechanism.
The reviewers agreed that the team shall write a new implementation
of a filesystem for local files
independent from AbstractFileSystem that will not share
AbstractFileSystem troubles. This solves problems with /net
and to certain extent addresses the high memory usage
(see advisory information [1]).
At the time of promotion E, the MasterFileSystem implementation
shall remain the same (allow pluging of VCS filesystems), just instead of
the old org.openide.filesystem.LocalFileSystem let it use the
new implementation. In promotion F (when the mounting will no longer be needed)
a merge of the local fileobjects with master fs ones, further lowering the memory
requirements, shall happen.
Windows UNC problems
In some sence the UNC files are special (one cannot list them in advance
as A:/, C:/ and other roots), in some ways they just make certain problems
more visible (a FileObject created on a laptop in work may not be found on
the laptop running at home [advisory section
[2]).
That is why the UNC roots should appear dynamically as chidren of the root
of master filesystem when accessed. They can be represented by the newly
implemented filesystem for local files or its subclass, in order to separate
the UNC only hacks from the rest of java.io.File wrapping code.
However the team is requested to make sure that problems are filled and
that the JDK team is properly notified
about importance of these issues (talk to Trung about that).
The identity problem
Resolving the symlinks problem by delegation of certain operation to a
cannonical FileObject looks like a briliant idea. However in certain environments
this is not sufficient as logical paths can lead to the same file even
without using symlinks. This can cause possible data loss as the same
file can still be opened in editor twice and edited more times at once.
There were a lot of ideas how to solve the problem from relaying on
File.getCanonicalFile (known to be unsuffitient), to using
java.nio.channels.FileLock (could possibly prevent other
applications like ant to access the file - see advisory section
[3]), using JNI (we do not want to do it), exposing
some SPI for others to provide better ways to discover the identity (possible
in future), or simulating what other editors do.
The reviewers suggest to try the last option and simulate emacs
(advisory section [4]) for now and see if this is enough.
If found unsufficient, other ways can be explored later. The possible problems
found till now include the unability to lock a file this way if one does not have
permition to create the lock file in a directory (just lock it without the file) or
prevent the locking of a file if the file is itself referenced by symlink (try to
traverse the symlink, or just do not solve this), but none of them seems to be
too critical to not choose this solution.
As a result the file will get opened in editor more times, but when user
types in both of them, he will be notified (beep, message in status line)
that the file is already locked.
In the email discussion two opinions arised:
- In order to lower the memory requirements, please test and measure
the amount of memory needed to open one file in editor (without its
siblings being needed) and optimize for this case.
- Implement proper serialization and deserialization.
Make sure the fileobject can be deserialized even its java.io.File does not exists
for proper handling of dynamic environment (laptop working at home/work/home/elsehwhere).
- Check (possibly by having an automated test) the behaviour of
nio FileLock and its suitfulness or unsuitfulness to be used in locking files while
discovering their real identity.
- Emulate the emacs locking - when somebody calls
FileObject.lock() create a temporary file along the original
(its name shall be changable as a private property to accommodate possible
problems on various platforms) and delete the file on lock release. Check
CLIHandlerTest for a possible ways how to write and test code
that needs to synchronize on creation and modification of files between
more independent applications, however the full crash detection and
strict locking between various programs is not the goal number one. The
goal is to make this working internally in one NetBeans based product.
None.
53801 - test and measure the amount of memory needed to open one file in editor
- Submitted
document
(its version
1.4 that was reviewed).
- Discussion on nbdev@netbeans.org
