.tok file specifications
========================

Here are the general specifications for the token (*.tok) files used by CaS.

Overview
========

    The token file basically specifies to CaS what tokens translate to which
`ctoken'.  A ctoken is the internal code used by CaS to refer to every type
of token, and is based on the code used by Casio.

    Casio tokens (see also cas_specs) are referred to by a single number,
which may be shifted.  Shifted opcodes are preceded with 0x7f (127).  ctokens
are similar, but shifted tokens simply have 256 added to them, thus CaS can
refer to every token with a single number.

Display and Program Tokens
~~~~~~~~~~~~~~~~~~~~~~~~~~

    CaS differentiates between two types of token, Display Tokens and Program
tokens.  Display tokens are used by list mode in CaS, whilst program tokens
are used in input and output.  Only the first program token is used by output,
and the display token is also often the first program token.  More on
specifying what is what later.

Format
======

Lines _starting_ with `#' are comments and are ignored by CaS.  You cannot
have comments at the end of token lines.

Lines follow this basic syntax:

ctoken|{flags}DisplayToken|{flags}ProgramToken|{flags}ProgramToken...

so an example:

 161|sinh |sinh|shine|sinch

Thus in list mode, `sinh ' will always be displayed.  It will also, in this
case, be used as the first program token, being used in an output.  Thus
`sinh', and the more whimsical `shine' and `sinch' are only used in input.

You can have more than one line with the same ctoken number, and it is
recommended to keep lines < 80 characters.

Spaces and cases
~~~~~~~~~~~~~~~~
    Tokens in CaS are case sensitive.  This is neccessary due to the
differences between the cases on a casio, and context sensitivity would be
slighly more than a SMoP, I'm afraid :-)

    The ascii.tok file supplied is heavily biased to using lower-case, and
so lower-case will work for every token (except capital letters...).  Often
the display token will have a capitalised first letter, so that can also be
used as input:

 224|Plot |Plot|plot

    Here, the form seen on the calculator itself is allowed specifically.
Note the extra `Plot'.  In display and output, a space is preferred as it
looks closer to the output on the calculator.  However when programming,
one might want to put `Plot0,0', and so CaS will cope with this.

Flags
=====

There are several flags which can subtley change the meaning of a tokeen in
the token file, for example, specifying which calculators support the token
etc.  Flags are specified as single letter arguments, within {braces} before
the token.  For example:

 309|{85}Blue |{v85}Blue|{v85}blue

Here the first token has flags `8' and `5', and the other also have `v'.

Flag meanings:
~~~~~~~~~~~~~~
  {l}

  The l flags specifies that the token should be used for list mode only.
  For example, for several tokens, the meaning is ambiguous but obvious by
  the context, eg 3e4 meaning 3 x 2.71... ^ 4.  Here, `e' could as easily
  mean the letter `e'.  Thus it is defined:

    165|{l}e|e^

  {v}

  When text is within quotes, it is essential that CaS renders it faithfully.
  For example, if one has the text "sin", we don't want CaS interpreting it
  as the tokens `"', `sin ', `"', as the calculator adds a space after the
  letters `sin' in the `sin ' token.  Hence all tokens which should *not*
  be matched in text whould be marked with a {v}:

     129|sin |{v}sin

  Notice that if the user types "sin ", this *will* be matched, and a byte
  saving will be made with no ill-effects.

  {7} {9} {8} {5}

  These specify that the token is only available on certain calculators.  If
  *none* of these are specified, then the default is that the token is
  avilable on all models.  (The same as {7985}.)  The numbers represent:

    7 - 7700
    9 - 9700
    8 - 9800
    5 - 9850, 9950  (often referred to as 9x50)

  Hence colour options are generally marked {85}


Special characters and control codes
====================================

    All characters are possible in the token file (except '\0' :-).  To
specify an ASCII (or other) value for a character, then the `@' character is
used (see the end of this line):

  13|@\n|\n@\n|\n|\n@010|\n@013|\n@013@010|@010|@013|@013@010

    These @nnn sequences are in decimal, and *must* consist of three digits,
padded with zeros if neccesary.  To use hexadecimal or octal numbers, prepend
the three digits with `x' or `o' respectively.

    This line also demonstrates the other feature of the @ character, the
control sequences:

     @\n        newline
     @\a        alert (bell)
     @\$        $ dollar sign
     @\@        @ at sign
     @\p        | pipe sign

The above line demonstrating the newline is designed to cope with input
textfiles from any platform, and spit them out according to the current
platform.


Managing token files
====================

It would be appreciated if you did not change either the cp437 or ascii
standard token files, as these will be changed as little as possible to
maintain compatibility between casio programs.  If you want to
create your own set of tokens, copy one of the standard files, or empty.tok
and edit it to your heart's content :)


TW 96/11/14
