### hst-1.0 README file

    		Hopfield-style network simulator
    	       Arun Jagota (jagota@cs.buffalo.edu)

    If you plan to install to use this simulator, I suggest that you
    drop a line to the above e-mail address. It will help me keep you informed
    about newer developments on the network (like new applications, papers etc..)
    or the simulator (new versions etc) as and when that happens.

    Installation
    ------------
    1. This s/w is all in C and should install without change on
    any 32-bit Unix machine. To create the executable,
    cd src
    make
    -- The executable 'mlhn' will be created in the directory 'src'.
    -- Move it to 'bin' and try out one of the examples as follows
    cd ../examples
    ../bin/mlhn room -c < room.train
    -- You should be in the directory which contains the example.

    2. To install it on a 16-bit machine (Unix or MSDOS), change the value
    of the constants MIN_VIGILANCE and MAX_W (in main.c) from -200000000 to
    -32000 and 2500000 to 25000 respectively. This is quite necessary. These
     are the only required changes and have been tested on an IBM XT.
    A file called Makefile.DOS for the MSDOS make utility has been provided
    in the 'src' directory and might be useful.
    3. It will probably also work on any other non-Unix machine since it is
    all written in standard C.

    Changes
    -------
    Within a year, I will probably announce a newer version with added
    features.  It will make it easier for you to "roll" your changes
    (if any) to this s/w to the new version if you track the changes.

    Usage
    -----
    	mlhn <network> [{-t}] [-c]

    	-t   : Test only. mlhn will read standard input and do
    	       energy-descent.
    	-c   : Assume all units in input are clamped to 1 unless
    	       <unit-name>*0.  mlhn will read the standard input
    	       as containing training data
    	[for other flags/options, see the header of the file main.c]

    <network>.cfg has to be made in the directory from which mlhn is
    invoked. It is a configuration file which must contain 5 integers
    on a single line. The meaning and typical range of the numbers is as
    follows.

    max_units lambda lambda_R vigilance       #size units
    10-1000   5 or 7  1-5   -10 to -4000      1-12

    Eg

    mlhn mynet -t

    Files : mynet.cfg

    mynet.cfg (* the following 1 line in the file mynet.cfg*)

    400     5    1  -4000 10

    Input format (during training)

    { a b c }
    ....

    Input Format (to mlhn, during testing)

    { a b c }

    { a*1 b*1 c*1 }

    { a*0 b*1 c }

    { 90-a 80-b 7-c }   (* Optional confidence values in interval [1-100] *)

    -- The only delimiters in sets are blank[s], tab[s] or end-of-line[s].
    -- So {a b c } would be an illegal set and might cause the symbol 'a' to
    -- be ignored.

    Debugging features (Commands accepted from stdin)
    -------------------------------------------------

    w 1 <first-set> <second-set>
    -- Will display weights in matrix format as follows.
    -- rows are 'first-set' and columns are 'second-set'
    Eg,
    w 1 { a b c } { d e f g }

    -- will display
    num11 num12 num13 num14
    num21 num22 num22 num24
    num31 num32 num33 num34

    -- where the rows and columns correspond to
       d      e     f    g
    a
    b
    c

    r 1 <first-set>
    -- Will display the internal resistance (threshold) of each unit.

    Example
    -------

    1. Training file (mynet.train)
    { a b c }
    { c d e }
    .......

    2. mynet.cfg
    100 7 1 -20 1

    /* Train */
    mlhn mynet -c < mynet.train

    /* Test if network has stored all memories cleanly */

    mlhn mynet -t < mynet.train > mynet.tst

    NOTE : network is proven to store all memories 'cleanly' ONLY when
    the domain has a combinatorial structure (See paper,
    A Hopfield-style network for content-addressable memories - 90-02)

    If your application domain has a combinatorial structure, that's fine
    otherwise, any n-dimensional binary domain can be converted to one with
    a combinatorial structure as follows. Need a unit for each value {0,1}
    at each position. This requires 2n units.

    Example : Store the 3-d vectors 101, 111,  011
    It is recommended you use other symbols (like {y,n}) instead of
    {0,1}. Use numerals to identify positions.
    Based on above, we want to store yny, yyy, nyy

    Store as follows:

    { y1 n2 n3 }
    { y1 y2 y3 }
    { n1 y2 y3 }

    Modifying individual weights
    ----------------------------
    It is also possible to modify individual weights by presenting the
    corresponding two units as a binary set. For eg, if you have to
    modify the weight w_ab, then present { a b } during training. However,
    this should be done USING the -ns flag to mlhn which tells mlhn not
    to automatically include the size unit during training. Otherwise
    { a b } will be considered a binary memory and not just a weight
    in perhaps a larger memory. In some sense, this allows you to
    customise the learning rule (within limits). The script 'unlearn_i',
    for eg, uses this because during unlearning we don't want to unlearn
    all pairs of connections in a spurious state, ONLY those connections
    between units in initial state and units that were switched ON. Without
    the '-ns' option, there would be no way to accomplish this.

    Exploiting order in the net O/P
    -------------------------------
    For any 'set' I/P, the net O/P is another 'set'. Actually this
    'set' has some order information in it that can be exploited if
    necessary. If we know that the network switched k units ON during
    energy descent, then the k left-most elements in O/P set will be in
    the reverse order of the switches, the 1st element (from left) being the
    last switch ON and the kth element being the first switch ON.
    At this time, unfortunately, the O/P set contains no information about
    the order in which units (if any) were switched OFF or which was the
    first element that was switched ON, if k is unknown. The latter can
    be found by subtracting the I/P set from the O/P set but preserving
    the order in the O/P set. If the result is non-empty, then the result
    is a set which contains all and only those units that were switched ON
    and in reverse order. The former (info about switch OFF order) will
    be provided in some subsequent version of simulator.

    Files created while training
    ----------------------------
    Suppose you train a network as follows

    mlhn mynet -c < mynet.train

    where mynet.train contains the training set.

    mlhn will create the following files (in your current directory) in
    which it will store the network parameters.

    mynet.pn1 - file contains all symbols used in the training set.
    mynet.R1  - value of internal resistance for each unit
    mynet.w1  - weight matrix (run-length encoded)
    mynet.wup1- Not used for 1-layer net but should be left untouched

    Numeric details
    ---------------
    No 'real' arithmetic is performed. +ve weight values in [0,1]
    are done in practice in the integer interval 0..SCALE-1, where
    'SCALE' is a constant declared in the file 'std.h'. To change the
    precision, simply change it's value and recompile everything
    (typing 'make' will do it). If 'SCALE' is changed, 'SCALE_POWER'
    should also be changed accordingly. 'SCALE' should always be a power
    of 2.
    Similarly, R_i, the internal resistance of each unit is in the integer
    interval 0..THRESH_SCALE. To change the precision, change THRESH_SCALE
    and THRESH_SCALE_POWER in file 'std.h' and recompile (type 'make').
    Again, THRESH_SCALE should be a power of 2. It is recommended that
    SCALE and THRESH_SCALE should always have the same value.
    Initial values of units (recall units can be continous-valued in test
    i/p) are in the integer range 0..HIGH-1. As before, to change the
    precision, change HIGH (and HIGH_POWER) in std.h. Again, HIGH should
    always be a power of 2.

    Pre/Postprocessing utilities
    ----------------------------
    The following utilities are included that do the following.

    some_sequence -> wrd_in.c -> { s1 o2 m3 e4 _5 s6 e7 q8 u9 c10 e11 }
    { s1 o2 m3 e4 _5 s6 e7 q8 u9 c10 e11 } -> wrd_out.c -> some_sequence

    ....xxx..
    .xxxxxx..  -> vis_in.c -> { .1 .2 .3 .4 x5 x6 ......... }
    xxx.xxx..

                                          ....xxx..
    { .1 .2 .3 x4 x5 ...} -> vis_out.c -> .xxxxxx..
                                          xxx.xxx..

    Additional documentation
    ------------------------
    1) Read the shell scripts train, tst and unlearn_i, especially
    the last one.
    2) Read the headers of the source files main.c, wrd_in.c

    Known Bugs
    ----------
    1. At least 1 size unit has to be put into the configuration file.
    This causes no loss of generality.

    1. Very rarely, when the input sets are too large, the network
    gives segmentation fault after having processed 200-300 of them.
    It may be related to running out of memory at some point in time.
    All results before that are correct. Simply run the network until
    a segmentation fault occurs, and re-run it on the remaining input.
    This will work if the input is test input to the network. If it is
    training input then it won't work. Break up the input into multiple
    parts such that no segmentation fault occurs. This phenomenon is
    quite rare.

    Relevant Papers and Reports
    ---------------------------
    The following papers are relevant (in priority order) and their LaTeX
    sources can be made available by e-mail (send request to
    jagota@cs.buffalo.edu). The first 3 papers have minimal overlap and
    are recommended reading.

    %T A new Hopfield-style network for content-addressable memories
    %A Arun Jagota (1990)
    %P Technical Report 90-02, Dept Of Computer Science, Suny Buffalo, NY
    %X Mathematical description of the model and its properties. A must.

    %T Applying a Hopfield-style network to Degraded Text Recognition
    %A Arun Jagota (1990).
    %P To appear in Proceedings of the International Joint Conference on
       Neural Networks, San Diego, 1990
    %X real-world application

    %T Applying a Hopfield-style network to Degraded Printed Text Restoration
    %A Arun Jagota (1990).
    %P To appear in Proceedings of the 3rd conference on neural networks
       and PDP, Indiana-Purdue university
    %X real-world application

    %T Knowledge representation in a multi-layered Hopfield network
    %A Arun Jagota, Oleg Jakubowicz
    %P Proceedings of the International Joint Conference on
       Neural Networks, Wash DC, June 1989, I-435
    %X multi-layer version of above model. Architecture designed with care
       but properties not fully explored. Simulator has partial implementa-
       tion of above architecture. For info, send e-mail.

    %T A neural lexicon in a Hopfield-style network
    %A Arun Jagota, Y.S. Hung
    %P Proceedings of the International Joint Conference on
       Neural Networks, Wash DC, Jan 1990, II-607
    %X Precursor to application-I (Degraded text recognition).
       Detailed experiments with large word sets. Most, though not all
       the information in this paper can be infered from TR 90-02 or the
       Degraded Text recognition paper.