昨天看到了AbstractFileSystem,也知道应用访问文件是通过FileContext这个类,今天来看这个类的源代码,先看下这个类老长的注释说明

  1 /**
  2  * The FileContext class provides an interface to the application writer for
  3  * using the Hadoop file system.
  4  * It provides a set of methods for the usual operation: create, open, 
  5  * list, etc 
  6  * 
  7  * <p>
  8  * <b> *** Path Names *** </b>
  9  * <p>
 10  * 
 11  * The Hadoop file system supports a URI name space and URI names.
 12  * It offers a forest of file systems that can be referenced using fully
 13  * qualified URIs.
 14  * Two common Hadoop file systems implementations are
 15  * <ul>
 16  * <li> the local file system: file:///path
 17  * <li> the hdfs file system hdfs://nnAddress:nnPort/path
 18  * </ul>
 19  * 
 20  * While URI names are very flexible, it requires knowing the name or address
 21  * of the server. For convenience one often wants to access the default system
 22  * in one's environment without knowing its name/address. This has an
 23  * additional benefit that it allows one to change one's default fs
 24  *  (e.g. admin moves application from cluster1 to cluster2).
 25  * <p>
 26  * 
 27  * To facilitate this, Hadoop supports a notion of a default file system.
 28  * The user can set his default file system, although this is
 29  * typically set up for you in your environment via your default config.
 30  * A default file system implies a default scheme and authority; slash-relative
 31  * names (such as /for/bar) are resolved relative to that default FS.
 32  * Similarly a user can also have working-directory-relative names (i.e. names
 33  * not starting with a slash). While the working directory is generally in the
 34  * same default FS, the wd can be in a different FS.
 35  * <p>
 36  *  Hence Hadoop path names can be one of:
 37  *  <ul>
 38  *  <li> fully qualified URI: scheme://authority/path
 39  *  <li> slash relative names: /path relative to the default file system
 40  *  <li> wd-relative names: path  relative to the working dir
 41  *  </ul>   
 42  *  Relative paths with scheme (scheme:foo/bar) are illegal.
 43  *  
 44  *  <p>
 45  *  <b>****The Role of the FileContext and configuration defaults****</b>
 46  *  <p>
 47  *  The FileContext provides file namespace context for resolving file names;
 48  *  it also contains the umask for permissions, In that sense it is like the
 49  *  per-process file-related state in Unix system.
 50  *  These two properties
 51  *  <ul> 
 52  *  <li> default file system i.e your slash)
 53  *  <li> umask
 54  *  </ul>
 55  *  in general, are obtained from the default configuration file
 56  *  in your environment,  (@see {@link Configuration}).
 57  *  
 58  *  No other configuration parameters are obtained from the default config as 
 59  *  far as the file context layer is concerned. All file system instances
 60  *  (i.e. deployments of file systems) have default properties; we call these
 61  *  server side (SS) defaults. Operation like create allow one to select many 
 62  *  properties: either pass them in as explicit parameters or use
 63  *  the SS properties.
 64  *  <p>
 65  *  The file system related SS defaults are
 66  *  <ul>
 67  *  <li> the home directory (default is "/user/userName")
 68  *  <li> the initial wd (only for local fs)
 69  *  <li> replication factor
 70  *  <li> block size
 71  *  <li> buffer size
 72  *  <li> encryptDataTransfer 
 73  *  <li> checksum option. (checksumType and  bytesPerChecksum)
 74  *  </ul>
 75  *
 76  * <p>
 77  * <b> *** Usage Model for the FileContext class *** </b>
 78  * <p>
 79  * Example 1: use the default config read from the $HADOOP_CONFIG/core.xml.
 80  *   Unspecified values come from core-defaults.xml in the release jar.
 81  *  <ul>  
 82  *  <li> myFContext = FileContext.getFileContext(); // uses the default config
 83  *                                                // which has your default FS 
 84  *  <li>  myFContext.create(path, ...);
 85  *  <li>  myFContext.setWorkingDir(path)
 86  *  <li>  myFContext.open (path, ...);  
 87  *  </ul>  
 88  * Example 2: Get a FileContext with a specific URI as the default FS
 89  *  <ul>  
 90  *  <li> myFContext = FileContext.getFileContext(URI)
 91  *  <li> myFContext.create(path, ...);
 92  *   ...
 93  * </ul> 
 94  * Example 3: FileContext with local file system as the default
 95  *  <ul> 
 96  *  <li> myFContext = FileContext.getLocalFSFileContext()
 97  *  <li> myFContext.create(path, ...);
 98  *  <li> ...
 99  *  </ul> 
100  * Example 4: Use a specific config, ignoring $HADOOP_CONFIG
101  *  Generally you should not need use a config unless you are doing
102  *   <ul> 
103  *   <li> configX = someConfigSomeOnePassedToYou.
104  *   <li> myFContext = getFileContext(configX); // configX is not changed,
105  *                                              // is passed down 
106  *   <li> myFContext.create(path, ...);
107  *   <li>...
108  *  </ul>                                          
109  *    
110  */
111 
112 @InterfaceAudience.Public
113 @InterfaceStability.Evolving /*Evolving for a release,to be changed to Stable */
114 public class FileContext {
View Code

相关文章: