Oblivion Mod:Oblivion Character Classes

The CCharClass class is used to define a single custom class of characters which will be used by the tokenizer when converting the stream of text characters to a stream of tokens. Since Oblivion uses simple ASCII characters we don't need to worry about the complexities of UNICODE or other multi-byte character systems.

CCharClass

The relevant members and methods of the CCharClass are given below:

  class CCharClass {
     bool   m_CharTable[256];   /* Defines which characters match for this class */
     string m_Name;             /* Identifies the character class */

          /* Does a character match this class? */
     bool DoesMatch (const char Value) { return m_CharTable[(byte)Value]; }
  };

Data File Format

A character class is defined by data loaded from a text file. The format of this file is:

   CharClasses
       [ClassName] = [ClassDef1] ... [ClassDefN]
       ...
   End

Each character class is held on a single line and is defined by multiple definitions separated by spaces. Class definitions can be any of the following:

• [Single Character] -- The given character matches.
• [StartChar]..[EndChar] -- A range of matching characters including the start and end.
• ![ClassDef] -- An exclamation point before a character class means that characters will not match that class.
• any -- All characters will match.
• space -- Match the space character.
• null -- Match the null \0 character.
• \t -- Match the tab character.
• \n -- Match the carriage return character.
• \r -- Match the linefeed character.
• \! -- Match the exclamation point character (needed since the ! also negates a class match).

Class definition examples:

   Alpha       = a..z A..Z
   Digit       = 0..9
   Whitespace  = \t \n \r space
   StartString = "
   EndString   = any !\n !"
   NotLower    = any !a..z

Oblivion Character Classes

The following character class definitions are used for Oblivion scripts:

  CharClasses
     Digit      = 0..9
     Alpha      = a..z A..Z
     EndLine    = \n NULL
     Comment    = ;
     WhiteSpace = space \t \r
     Quote      = "
     IDStart    = a..z A..Z _
     ID         = a..z A..Z 0..9 _
     Bracket    = ( )
     EndString  = any !\n !" !NULL
     EndLine    = \n
     Equal      = =
     Decimal    = .
     AddOp      = + -
     MultOp     = * / %
     RelOp       = = < > \!
     Comma      = ,
  End

Notes:

• Lines can either end normally with a linefeed (\n) or be at the very end of a scan source (null).
• Classes that match only a single character are not absolutely needed but will make it easier to change in the future if required.
• The CS compiler seems to allow identifiers starting with or containing the underscore (_) character.