18 September 2002

               !DIR and _DIR: SREhttp/2 directory display utilities

Contents:
   I: Introduction
 IIa: The DIR_EXCLUSION parameter.
 IIb: Options used in the _DIR addon
III.1: Notes on the HEADERFILE and FOOTERFILE options.
III.2: Note on PATTERN= and DIR_EXCLUSION
III.3: Notes on caching

                        ----------------------

I. Introduction 

SREhttp/2 comes packaged with two utilities for displaying & 
accessing the contents of a web-accessible directory:

  1) The !DIR "special feature" 
  2) The _DIR.CMD "addon"

!DIR is fairly simple and quick. It can be used to list files in a directory. 

!DIR is invoked in one of two ways:
 i) By explicitly calling it as a !SPECIAL command.
    Example:  /!DIR?dir1
ii) By the !DIR option of the DEFAULT parameter; which instructs SREhttp/2 
    to generate  a directory listing "when no DEFAULE file is found".
    Example: /dir1/    (assuming there is no "directory specific index file"
                         in the DIR1 subdirectory)


_DIR.CMD is more complicated, but also much more powerful. 
You can use the _DIR.SHT demo (in /SRE2k/SREHTTP2/DEMOS) as a
front-end to _DIR.CMD -- most of the _DIR.CMD options can be specified 
in a fairly compact HTML FORM.

Note: _DIR.CMD also supports caching of directory listings -- which can
      increase throughput when working with complicated directory listings 
      (such as listings that contain automatically generated descriptions).


                --------------------------

IIa. The DIR_EXCLUSION parameter.

Both !DIR and _DIR will read the DIR_EXCLUSION SREhttp/2 parameter. 
DIR_EXCLUSION is set in the (possibly host-specific) SREHTTP2.CFG 
configuration file.

DIR_EXCLUSION should be a space delimited list of files or subdirectory names
-- without any path information. Before listing a file, or a subdirectory, 
both !DIR and _DIR will check this list -- if the file (or subdirectory) 
matches any of these entries, then the file (or subdirectory) is not listed.

Notes:
  *  "matches" means a letter by letter match, or a wild card match. Wild
      card matches are specified by including * characters in a file 
      (or subdirectory names)

  *   File names should always contain a period (even if nothing follows 
      the period)

  *   Subdirectory names should not contain a period

  *   Sorry, for now filenames (or subdirectory names) that contain spaces can
      not be directly included in the DIR_EXCLUSION parameter. You can use *, 
      instead of space, to try and approximate such space-including names.

  *   The appearance of a subdirectory name in DIR_EXCLUSION does NOT suppress 
      display of the entire directory  (should it be the target of its own 
      !DIR). 

Example:
    dir_exclusion=PASSWORD.* PASSWD.* USER*.*  HTACCESS*  PRIVATE SYSTEM

    Note that PRIVATE and SYSTEM would match subdirectories. 
    HTACCESS* would also match subdirectories and files that start 
    with "HTACCESS". If only want HTACCESS to  match files, you 
    should use HTACCESS*.* instead.

Example:
     If DIR_EXCLUSION='PRIVATE '
     If D:\WWW is the default data directory, and it contains directories:
        D:\WWW\JOE.1, D:\WWW\JOE\PRIVATE, D:\WWW\PETER, and  
        D:\WWW\PETER\PRIVATE
     then 
       * a /!DIR?JOE request will list all files and directories under 
         D:\WWW\JOE
       * a /!DIR?PETER request will list all files and directories 
         under D:\WWW\PETER, 
         EXCEPT for the D:\WWW\PETER\PRIVATE directory.
       * a !DIR?PETER/PRIVATE will list the contents of   
         D:\WWW\PETER\PRIVATE

          Thus, a well-informed client can view the contents of 
          an "excluded" directory (but will still be subject to 
          access controls if she tries to download anything from it).

                        ----------------------

IIb: Options used in the _DIR addon


The _DIR addon understands a broad set of options. These include:
   * selecting a set of files to include (everything else excluded).
   * several display formats:  2 kinds of tables, and 3 kinds of lists. 
   * filesize, creation date, and a descriptive icon can be displayed
   * automatically generated descriptions (of text files) can be displayed
   * header and footer files can be displayed

You can specify options in one of two ways:

   i) In a web form (with an ACTION="/_DIR.CMD") you can include elements 
      like: <input type="hidden" name="an_option" value="some_value">
      or, instead of hidden, "text" or "checkbox" to let the client 
      have some control!

        Note:
          The _DIR.SHT file (in SRE2K/SREHTTP2/DEMOS) demonstrates the use 
          of most of these options within an HTML form.

   ii) From a hard-coded link, you can use the form:
           <a href="/_dir.cmd?an_option=some_value&an_option2=some_value2& ...">a message</a>

The following describes the _DIR options. We use the "hard-coded" link 
format in the examples just to simplify presentation.

Notes:  
  *  the most important, and only required, parameter is DIR.
  *  see _DIR.SHT for more details on some of these options.


:: The options:

AUTO_DESCRIBE=nnn
   nnn is either 0, or a positive integer
   nnn=0 : do NOT attempt to automatically create descriptions
   nnn>0 : attempt to automatically create descriptions, and retain the
           first nnn characters of these descriptions.

   If nnn>0, then try to create descriptions for text files, and for 
   .ZIP files. 
      This is done by extracting 
         <TITLE> and  <META NAME="DESCRIPTION" Content="xx">
    headers from HTML files, or the -z comments or  FILE_ID.DIZ files 
    from .ZIP files, and the beginning of plain text files.

   AUTO_DESCRIBE can be used with, or without, a DESCRIP_FILE.
   If used with a DESCRIP_FILE, then the AUTO_DESCRIBE description  
   is ONLY used if there is no description for the file
   in the DESCRIP_FILE.


CACHE=0 or 1
     Enable caching of directory lists. These cached lists are stored in
     SREhttp/2's "file cache". If cache is enabled (CACHE=1), then 
     _DIR will see if a matching cache file (for the requested directory,
     using the requested options) can be used. 

     Notes:
        * The cache is especially useful when AUTO_DESCRIBE is used.

        * _DIR.CMD contains a CACHE parameter, which sets the default 
          state of caching. When installed, the default is to enable 
          the cache.

CELLSPACING=nn
    If nn is an integer > 0, and GRID or TABLE mode is selected, then
    nn is used as extra padding within cells.

DATEFMT=x 
     The date format used; where x is one of B C D E M N O S U W.
      Where, given a date of 27 Aug 1988
                B:    725975
                D:    240
                E:    27/08/88
                L:    27 August 1988
                M:    August
                N:    27 Aug 1988
                O:    88/08/27
                S:    19880827
                U:    08/27/88
                W:    Saturday

           Note that if "x" is not one of these, N is used.


DIR=relative_directory
   This is the only required option. It should point to a directory relative 
   to a (possibly host-specific) "data directory", or to a "virtual directory"
   (as may be defined by a selector specific attribute).


DESCRIP_FILE=filename.ext 
    Specify a directory specific "descriptions file". This file MUST be in
    the directory chosen by the DIR= option.

    Note: If filename.exe is not specified, then (depending on 
           DESCRIP_FILE_TYPE) either _DIR.LST or _DIR.KEY is used.

    There are 2 types of description files: single-line and keyed.

    Single-line: 
       each line starts with a file (or directory name), followed by a 
       description. The lines can be arbitrarily long. 

    Keyed: Keys, 
       consisting of a file (or directory) name enclosed by { } brackets, 
       delimits the start of a description. The description runs to the 
       next { in the file. Thus, descriptions can span many lines; 
       or, you can put several descriptions on a single line. 


DESCRIP_FILE_TYPE=0, 1, or 2

      The type of description file:
           0 = do NOT use a description file
           1 = single line
           2 = keyed

DESCRIP_FILE_ONLY=0,1,2

      Only list files & directories that appear in the DESCRIPTION file,
      and list them in the order they appear.  
        
          0 = suppress
          1 = enable
          2 = enable, but only if a description file exists. If it does 
              not exist,  then display in the natural order (usually
              alphabetical), and  automatically generate descriptions 
              if AUTO_DESCRIBE is enabled).

       You can also have descriptive text included in the response, text embedded within
       file listings.
 
       For example (using a "single line" type of description file):

                FOO.BAR: this is foo
                BAR.FOO  this is bar
                [DESC]: no more foo bar stuff
                READ.ME  the intro

      Note: if DESCRIP_FILE_ONLY is enabled (and a description file exists), 
             then Inclusion-lists (created with  PATTERN options), and the
              DIR_EXCLUSION parameter, are ignored.

DESC_HTML=0 or 1
    If 1, then display descriptions using HTML formatting. If 0, then
    convert HTML elements into text (that is, display the < and > characters)

DMODE= The display mode
     5 types are supported:

       PRE : Fixed width font, column aligned, (descriptions
             will not be displayed)

        UL : Bulleted list

        DL : Outline format, with descriptions in indented paragraph

      GRID : Multiple files/subdirectories per row of a table (descriptions
             will not be displayed)

      TABLE : One file entry per table row (descriptions in last column)


DIR_PAGE=selector
    Selector should be a URI (or a "selector") that points to an HTML page 
    that lets you choose a directory to view. Typically, one would use
        DIR_PAGE=/SRE2K/SREHTTP2/DEMOS/_DIR.SHT
    If you set this parameter, a "List a new directory?" link is placed at the
    top of directory listing pages.

EXPAND_ZIP=0 or 1
    If 1, then .ZIP files will have an "expand & view" link placed next to
    them (either "[vu]" or a plus-sign icon). Clicking on this "expand & view"
    icon instructs _DIR to display the contents of the .ZIP file -- the client
    can then download individual files from this .ZIP file.

    If 0, this "expand and view" link will not be offered. The client will 
    still be able to download the entire .ZIP file, but will not be able to
    select and download individual files from the .ZIP file.

FOOTERILE=filename.ext == 
    A file to use as a "footer".  

GIFS=0 or 1
     If 1, then display small descriptive icons next to each file. These 
     icons are stored in the IMGDIR directory.  
     The displayed icon is based on a file's extension.

HEADERFILE=filename.ext ==
    A file to use as a "header".  


IMGDIR=relative_directory
    Optional. If not specified, the IMGDIR parameter (in _DIR.CMD) is used.
    IMGDIR should be a relative directory -- as with the DIR parameter, 
    relative to the data directory or to a virtual directory. 
    It is where small "descriptive icons" are stored. 
    By default, _DIR.CMD hardcodes this to be   /SRE2K/SREHTTP2/IMGS.

MAXLINES   
   Maximum characters to display of footer and header files

NO_RECURSE
  If NO_RECURSE=1, then directory listings will NOT be recursively processed
  with _DIR. Instead, a normal URL pointing to this directory will be used --
  for example: http://foo.bar.net/dir1/subdir1/
  (i.e.;  /dir1/subdir1/index.htm may be displayed)

PATTERN==a_Pattern
    Include only files that match this (possibly wildcard containing) pattern.     
    You can have many PATTERN= elements.
                   Example: PATTERN=*.HTM&PATTERN=*.TXT

RECORD=0 or 1
    If 1, then clicking on a file link will invoke the SREhttp/2 SENDFILE
    utility. SENDFILE will send the requested file to the client, and will
    augment a selector-specific counter (in the SENDFILE.CNT file).
    Note that SENDFILE sends the file as it -- in particular, it does NOT
    attemp SSIs on SHTML files.

ROOTDIR=!, or a selector
    This is the "root directory" -- the client will not be offered "parent" 
    links below this. You can specify an explicit root directory (using the 
    same syntax as with DIR). However, typically one uses 
        ROOTDIR=!
    This causes _DIR to fill in the current value of DIR in ROOTDIR, and then
    to include this ROOTDIR=dir element when traversing subdirectories. The 
    net effect of ROOTDIR=! is to prevent clients from moving above the 
    selected directory (but allow them to move down, and backup, the directory
    tree beneath the DIR directory).
 
SHOWSIZE=0 or 1
    If 1, display file size (format depends on SIZE_FMT parameter)

SHOWDATE=0 or 1
    If 1, display the file creation date

SHOWTIME=0 or 1
    If 1, display the file creation time. This is ignored if SHOWDATE=0



SIZEFMT=abbrev
    Optional. SIZEFMT controls how file-sizei information is displayed:
        SIZEFMT=ABBREV  :  round to nearest Kbytes. Thus, 52,612 becomes 52k
        SIZEFMT=0       :   display bytes as is

SORTBY=xxx 
  Sort criteria for files. 
  xxx can be one of:
        NAME -- Sort by name 
        DATE -- Sort by date
        SIZE -- Sort by date
         EXT -- Sort by file extension
    DESCRIBE -- Sort using the order of entries in a DESCRIP_FILE file.
                Files not mentioned in DESCRIP_FILE are placed after
                the files that are (in order encountered).
                Note that you MUST also specify a DESCRIP_FILE_TYPE option
        NONE --  Do not sort. SORT=NONE has the same effect as NOSORT



SUBDIR=0, 1, or 2
    Display a list of subdirectories under DIR.
        0 = do NOT display
        1 = display the subdirectory list beneath the file list
        2 = display the subdirectory list above the file list

TABLEBORDER=0 or 1
    If 1, and GRID or TABLE mode is selected, then a border is drawn
    around the table.

TARGET_WINDOW= 0 or 1
    If 1, then clicking on a file link will bring up the file in a seperate 
    window.

TERSE=0 or 1
  If TERSE=1, then some summary info (such as # of files, and a reminder on
  how to download files) will NOT be displayed.


TITLE=title_astring
   The "title_string" is written at the top of the document. If not 
   specified, a simple descriptive title will be automatically generated.

   Title_string may contain spaces (though, if you are hard-coding a link
   to point to _DIR, you'll have to use the + for space URL-encoding rule).

TR_COLOR=0 or 1
    If 1, then use alternating cell-background colors in GRID and TABLE
    For details on cell-background colors, see the TR_COLORS. parameters
    in _DIR.CMD.


TABLECOLS=1 to 8
    Used with GRID mode -- this sets how many cells per table row.

TERSE=0 or 1
    If 1, then output will be terser (fewer explanatory notes)


                    ----------------------    

III.1. Notes on the HEADERFILE and FOOTERFILE options.

  * Header and footer files are placed above and below (respectively) the
    file & directory listings. 
  * If a relative name is used, the header-file and footer-file are assumed 
    to be in (or under) the DIR directory.
  * If an absolute name is used, the usual SREhttp/2 resolution rules (i.e.;
    using virtual directories, etc.) are used. Note that header and footer 
    files, like any other requested resources, are checked for access control
    privileges.

                    ----------------------    

III.2. Note on PATTERN= and DIR_EXCLUSION

   * DIR_EXCLUSION is a "can not match list".  PATTERN= entries, after
     concatenation (nb: you can have multiple PATTERN= entries form a
     "must match list". 
     You can have both -- in which case the file MUST match a PATTERN=, a
     and must NOT match  a DIR_EXCLUSION element.

  * Reminder: DIR_EXCLUSION and PATTERN refers to files (and subdirectories) 
              in a requested directory. 

                   --------------------------

III.3: Notes on caching

  * Caching is most profitably used in conjunction with the AUTO_DESCRIBE
    and DESCRIBE=filename option. If you are not including descriptions,
    then caching will probably not help a lot.

 *  _DIR uses the SREhttp/2 "file-cache" directory. This is shared with the
    SSI and MESSAGE-box caches.  By default, cache entries have a 1 day 
    lifespan.
    However, on busy sites with many SSIs requests, Message-box lookups, 
    files and _DIR listings being cached, the lifespan of these cached-files 
    may be shorter.  If this describes you, and your machine has sufficient
    memory to support a larger  (then 500 element) cache, see the 
    FILECACHE_MAXFILES parameter
    in SREHINI.RXX.

   
  * SREhttp/2 will check the file-stamps of all files and subdirectories
    in the requested directory, and the file-stamps of selected header and
    footer files.  In addition, _DIR will check the options is used to create
    the directory listing.
    Note: actually, a CRC of this information is used; thus, on very rare
          occasions SREhttp/2 will NOT detect a change.

  * To suppress, on a request specific basis, SREhttp/2's use of this
    cache; just include a CACHE=0 option. To permanently suppress it,
    set the CACHE=-1 parameter in _DIR.CMD.

   * Cache entries are identified by both fully qualified directory,
     and by the "option list and exclusion" lists.
     Thus, multiple views of the same directory may be cached.

   * When SRE2003 & SREhttp/2  restarts, all existing _DIR cache files
     (in the FILE_CSH subdirectory) are deleted. 


                   --------------------------

III.3: On using _DIR as a simple BBS system


Although not able to offer the fine level of control of a dedicated
bulletin-board system  for file distribution (BBS), _DIR can give you 
many of the more important features.

  * By using DESCIP_FILE and DESCRIP_FILE_ONLY, you can control just
    what files are available. You can also describe them, order them,
    and insert descriptive material between and betwixt file items.

  * With standard SREhttp/2 access controls, you can control just
    which files can be obtained by different users. In fact, if
    a client can have multiple accounts on your system, _DIR's support
    for "dynamic privileges" allows the client to accumulate privileges,
    via multiple logons, thereby increasing the set of files he  
    can download.

  * To support file uploads, in a title or in a description file you
    can include links to SREhttp/2's file upload utilities (i.e.;
    by using a [DESC]: line in a single-line description file.


:End of DIR.TXT.
