ChatterBean: Flexible Alicebot

Table of Contents

• Latest News
• Introduction
• Development Plan: Path to 1.0
• Developer-Friendly Features
• Supported AIML Tags
• Third-Party Includes
• Download, Building and Usage Information
• Sample Alicebots
• Licensing Information
• Credits

Latest News

• Correct whitespace handling: support for the xml:space attribute has been added. This brings ChatterBean one step closer to full AIML compatibility.
• New applet interface: previous releases of ChatterBean provided an applet class as a proof-of-concept GUI. Version 00.008 improves on this by turning the main bitoflife.chatterbean.ChatterBean class itself into a Java Applet, which encloses an Alicebot. Support for embedding the applet into web pages has been added, through a set of recognized named parameters that allows proper configuration of the underlying bot.
• AJAX-ready: an interesting feature of the applet interface is its ability to interact with Javascript code on the web page through its API. This, combined with the new support for the javascript tag (see below), allows ChatterBean to be easily embedded in an AJAX application. A proof-of-concept application is provided here.
• Support for the <javascript> tag: in the past, I refused to support the <javascript> tag, under the rationale that the BeanShell-backed <system> tag already provided all the scripting power an Alicebot would need. While the argument in favor of BeanShell still holds, I realized that the <javascript> tag could be used as an interface between an applet-enclosed bot and the underlying browser. Current support is still limited, in that an Alicebot can only reach Javascript environment features and top-level user-defined functions, but it is well enough to add a nice AJAX-like responsiveness to a web bot.
• Unified Context properties: up until now, the bitoflife.chatterbean.Context class provided separate support for managing predicates and bot predicates. This support has now been unified around a single system of properties. Where, in order to get the value of the dateFormat predicate, you'd previously call:

  String predicate = context.predicate("dateFormat");
  

  You now must call:

  String predicate = (String) context.property("predicate.dateFormat");
  

  Similarly, to get the value of the id bot predicate, where you'd previously call:

  String id = context.botPredicate("id");
  

  You now must call:

  String id = (String) context.property("bot.id");
  

  What is the advantage? So long you respect the use of predicate. and bot. prefixed names for predicates and bot predicates, any object can be bound to an Alicebot's context through a property. For example, at loading time the bitoflife.chatterbean.ChatterBean class bounds a Javascript interpreter to its enclosing Alicebot through a context property named javascript.interpreter; later, the javascript tag can recover the interpreter instance from the context and use it to interpret its contents. In the near future, this will allow the use of non-string values with predicates and bot predicates as well (currently they must be set to string values only).
• Property change listener interface: another novel feature of the bitoflife.chatterbean.Context class is the ability to register event listeners that fire in response of changings in specific properties. This has been used to provide a more flexible, elegant treatment to the problem of updating the random seed and the converstaion topic in response to changes in context properties.
• Better development-time portability: since the beggining of the ChatterBean project, one of my main goals has been to provide a developer-friendly distribution. I refrained from using IDE's such as Eclipse to avoid locking the project into any specific environment, deciding instead to go with javac's own build files and a bit of BeanShell scripts to ensure platform portability. However, as some people pointed out, this solution was not as platform-independent as I hoped. This version solves those portability issues, providing an improved micro-development environment heavily based on BeanShell.
• Small fixes and code changes: as always, there has been some minor bug fixes and adjustements to the code base:
• The architecture of Category matchers created around the bitoflife.chatterbean.Matcher interface has been dropped in favour of a unique bitoflife.chatterbean.Graphmaster class. The old way was meant to facilitate the implementation of more dynamic category bases, since it was my early understanding that it would be hard to implement category exclusion into the Graphmaster algorithm; however, I've come to realize that my concerns were unjustified, and there was no reason to keep more than one category matcher class.
• When the randomSeed bot predicate is not set, the <random> tag uses a different seed each time the bot is started. This allows for less predictable behaviour.
• The behaviour of the <input> has changed: now index (1) refers to the input to the category being matched, and not the one before it. I reasoned that this would be a more complete, useful interpretation of the AIML standard than the previous one.
• Both the <javascript> and the <system> tags now return the result of the last evaluated expression (if it evaluates to something) if the result variable is set to null. This makes for a cleaner syntax where simple expressions are concerned.
• A new environment variable, match, has been added to the context of the BeanShell interpreter. It contains the bitoflife.chatterbean.Match object representing the current matching operation. Since this object also contains a reference to the Alicebot itself, this allows a bot to manipulate its own interpreter according to the inputs it gets.

Special thanks to Andreas Matthias, Charles Chevallier and Lindsay Steele to their suggestions and reports, and my appologies if I forgot to mention anyone else.

2005.11.07 Version 00.007 is now available. This version implements the transformational elements, making it the first ChatterBean release to support the full set of AIML 1.0.1 tags. A lot of work is yet to be done before the project reaches standard compliance (for one thing, correct whitespace handling is yet to be implemented), let alone fulfill its stated objectives. Still, an important milestone has been reached.

As you probably guessed, another change in this release is a new version number system. From now on, every release will have a series number and a version number. Version numbers are assigned sequentially within a series, while series numbers follow this convention:

• 00: Alpha series;
• 01: Beta series;
• 02: Production series.

Whether the series number will keep advancing after ChatterBean upgrades to production status is not yet decided.

2005.07.17 - Version 0.7.5 Alpha is now available. This version implements the atomic template elements <input>, <thatstar> and <topicstar>; the conditional elements <random> and <condition>; the capture element <gossip>; and the covert element <learn>. This leaves only the transformational elements to implement before ChatterBean supports the full set of AIML 1.0.1 tags. Also, a new approach to filling the contents of matched wildcards has been implemented, which greatly improves the quality and stability of the wildcard matching mechanism.

2005.05.17 - Version 0.7.0 Alpha is now available. This version implements many new tags, including the long-awaited <topic> tag; this makes 07.0 Alpha the first version of ChatterBean to implement the complete match path. For a complete list of implemented tags, together with important notes, see the section on Supported AIML Tags. Also, a customized XML file (context.xml) has been created to store both predicate defaults and bot predicate values.

2005.05.09 - Version 0.6.5 Alpha is now available. Changes in this version include:

• The XML parsing architecture introduced in the previous version has been adopted to the parsing of splitter and substitution files, eliminating the older architecture based on the Loader class (which by the way has been removed);
• The AIML parser has been refactored to employ a new reflexive protocol on deciding what class to create each time it reads a tag from an AIML source;
• Property files used to hold default values for predicates and configuration options for the application now employ the new XML format, instead of the old name/value plain text format;
• The <that> tag, both in its pattern and template-side version, has been implemented;
• A new parser architecture makes it easier to create new chatterbot instances;
• Package architecture has been modified: the bitoflife.chatterbean.test and bitoflife.chatterbean.mother packages have been eliminated, their classes moved to the same packages of their respective framework classes; and the bitoflife.chatterbean.xml package has been renamed bitoflife.chatterbean.parser, to better reflect its current use.

For more details, read the new section on Developer-Friendly Features.

Another change, probably more appealing for AIML end-users, is that from now on ChatterBean will be distributed in two versions: source-code and binary. The later takes the form of a jar file that can be executed from the command-line, opening a very simple graphical user interface. See the section on Download, Building and Usage Information for more details.

2005.04.03 - Version 0.5.0 Alpha is now available. This release gets much closer, feature and architecture-wise, to my vision of a developer-friendly AIML engine. Changes include:

• Full compliance to the Java 5 standards for generics;
• A new architecture for XML parsers;
• A new, more customizable architecture for Responders;
• Implementation of the <get> and <set> tags, together with load-time setting of predicates;
• The creation of the ChatterBean class, a highly customizable class to gather the components of the framework into a coherent application.

By the way, I have decided to leave the "bug" mentioned in the last release -- by which punctuations don't appear in the contents of <star> tags -- unchanged, since there are situations where you don't want the punctuation to be present.

2005.03.25 - Version 0.1.3 Alpha is now available. Along with several bugfixes and some improvements, this release brings a new, platform-independent set of scripts for building, testing and running the application (see the list below for more details). Besides, I have changed the application's name from ProgramJ to ChatterBean, in order to avoid confusion with J-Alice, another AIML engine project.

Change Log

• ChatterBean is now a Java 5 application. It is not yet fully compliant to the new standards, but that will be taken care of in the next release;
• The jUnit jar file has been added to the distribution package, so you won't have to bother about installing it yourself anymore;
• Corrected a misunderstanding about the behaviour of wildcards, which would allow only two different wildcards at a Pattern;
• A new class, CategoryBuilder, has been pulled off the CategoriesLoader class (responsible for loading AIML files). The new architecture makes it easier to add new tag handlers;
• Found and solved a bug that would produce invalid behaviour for templates of the form "some text <srai>something</srai> some more text";
• Implemented the <system> tag, which passes its contents (after doing any appropriate preprocessing) to be executed by a BeanShell interpreter;
• Corrected the order in which input normalisations are done -- the previous version would split the user input before applying substitutions. However this introduced a bug where ending punctuation is lost from the unnormalised input. This too will be dealt with in the next release;
• The set of files that illustrated ChatterBean's use in a JSP page has been removed from the distribution package. They will probably return in the future, but right now there is little point in keeping them, since they are not validated by the test sets, and the next releases are very likely to brake them;
• The old set of Bash scripts to do operations such as compilation an running the application has been substituted by a set of BeanShell scripts. Besides, a small jar executable has been provided, that will allow users to easily run these scripts in any machine. See the section Download, Building and Usage Information for more details.

2005.02.07 - Version 0.1.2 Alpha is now available. This release provides several bugfixes, a new, constant-time implementation for the Responder interface (the HashMaster class) and a new way for passing startup-time configuration parameters, using property files, instead of the old configuration.txt unformatted text format. It still implements a very restricted subset of AIML, and is not as well documented as it should; but at least the architecture got to a point where major changes are unlikely (although the API interface may still change some).

Introduction

ChatterBean is an AIML interpreter (also known as "Alicebot") written in pure Java. Its ultimate objectives are:

• To be fully compliant to the current (1.0.1 at the moment of this writing) AIML standard;
• Implement a simple, easily customisable, JavaBeans-compliant plug-in architecture;
• To be as self-contained as possible. Ideally, a ChatterBean release should compile and run on any machine with an appropriate version of the JDK installed, regardless of the underlying operating system and without depend on anything not provided in the distribution package itself;
• Rely on the Java API as much as possible for writing concise code, easy to understand and modify;
• Provide thorough API documentation through the Javadoc tool, complemented by code comments wherever convenient;
• Provide a robust set of tests, both "white box" (or "unit tests", in the eXtreme Programming jargon) and "black box" ("acceptance tests"), that ensure the program works as expected, and that modifications have not broken it in any known way;
• Provide a highly controlable application framework. It must be possible to construct, configure and control an instance of the AIML interpreter entirely through the API;
• Keep a limited set of features. Because ChatterBean is intended to be convenient to client developers, it cannot avoid providing some sort of user interface, even if only as an example; however, support for remote communication protocols, access to databases and other "advanced" features will be left for the client developer to implement.

ChatterBean is free software, licensed under the GPL. All the usual claims about help wanted and no guarantees apply.

Development Plan: Path to 1.0

Developer-Friendly Features

The black-box ("acceptance") tests are implemented as a BeanShell script that drives ChatterBean's console interface, feeding it with a pre-determined set of requests, recording the responses and comparing them with the expected ones. Should any output differ from what is expected, the expected and actual responses are printed, together with the line of the expected response in the responses file. For example:

      $ java -jar bsh.jar acceptance.bsh
      Testing ChatterBean
      Line 6:
        expected: Yes, I am an AliceBot.
        actual:   Yes, I am an ALICE Bot.
    

If no differences are found, the script simply prints "0k".

      $ java -jar bsh.jar acceptance.bsh
      Testing ChatterBean
      0k
    

The white-box ("unit") tests are implemented with the help of the jUnit unit testing framework. Tests are collected into test classes, which follow a naming convention: FooTest is the name of the test class which contains the tests for the Foo class. Occasionally, there will also be a FooMother class, which is responsible for creating instances of the framework class for the respective test class. Test classes and mother classes are always located in the same package of their respective framework class.

Not all framework classes have test classes: some of them are too simple to warrant tests, or are better tested through their wrappers.

Encapsulation and Easy Instantiation

ChatterBean's framework is conveniently encapsulated within the ChatterBean class. Creating a fully configured AliceBot can be as easy as instantiating a class:

      AliceBot bot = new ChatterBean("relative_path_to/configuration_properties.xml");
      String response = bot.respond("Good night, Alice.");
    

Plug-in Architecture

Because ChatterBean is constituted of many small classes exchanging messages among each other, it is easy to tap into the flow of execution and customize its behaviour, by writing specialized classes and connecting them in place. For example, imagine you want your Alicebot to fetch and persist predicates in a data base. This could be easily accomplished by writing a specialized Context class:

      /**
      This is not intended as a proper example of JDBC use; this class is only
      meant to illustrate the possibilities of customization provided by the
      ChatterBean framework.
      */
      public class DBContext extends Context
      {
        private Connection con = DriverManager.getConnection("jdbc:odbc:con", "user", "pass");
      
        public Object property(String name)
        {
          Statement stmt = con.createStatement();
          ResultSet rs = stmt.executeQuery("SELECT VALUE FROM PROPERTIES WHERE NAME = '" + predicate + "'");
          return rs.getString("VALUE");
        }
      
        public void property(String name, Object value)
        {
          Statement stmt = con.createStatement();
          stmt.executeUpdate("UPDATE PROPERTIES SET VALUE = '" + value + " WHERE NAME = '" + name + "'");
        }
      }
    

Now you can connect an instance of this class to your AliceBot:

      AliceBot bot = new ChatterBean("relative_path_to/configuration_properties.xml");
      bot.setContext(new DBContext());
    

Alternatively, you can create an instance of the ChatterBeanParser class, configure it to use your context class, and then use it to create a new AliceBot:

      ChatterBeanParser parser = new ChatterBeanParser();
      parser.setContextClass(DBContext.class);
      AliceBot bot = parser.parse("relative_path_to/configuration_properties.xml");
    

Reflexive AIML Parsing Framework

In the ChatterBean framework, the result of parsing an AIML file is an object tree, with objects representing each parsed tag and its contents. The classes used to build that tree are located inside the bitoflife.chatterbean.aiml package; every time ChatterBean's AIML parser reads a tag, it uses reflection to create an instance of a class with the same name of the tag, feed it with attributes and, perhaps, add children objects to it. For example, every time the parser reads a <category> tag, it creates an instance of the bitoflife.chatterbean.aiml.Category class. The parser code simply assumes that, for every tag it reads, there will be a class with the same name inside the bitoflife.chatterbean.aiml package; there is no program code telling it what tags are valid or not.

This design makes it possible to easily add new tags to the known set, simply by writing new classes into the bitoflife.chatterbean.aiml package. Valid tag classes follow these guidelines:

• Every tag class has a constructor that takes a single argument, an instance of the org.xml.sax.Attributes class. This constructor must be used to get any attributes the tag might have;
• Tags that appear inside the <template> tag must inherit from bitoflife.chatterbean.aiml.TemplateElement;
• All other tags must implement the bitoflife.chatterbean.aiml.AIMLElement interface.

For examples of this protocol, read the source code of the tag classes inside the bitoflife.chatterbean.aiml package.

Supported AIML Tags

• <aiml> [1]
• <bot>
• <category>
• <condition>
• <date> [2]
• <formal>
• <get>
• <gossip> [3]
• <id> [4]
• <input>
• <javascript> [5]
• <learn>
• <lowercase>
• <pattern>
• <random>
• <sentence>
• <set>
• <size>
• <sr>
• <srai>
• <star>
• <system> [6]
• <template>
• <that> (Pattern-side and Template-side)
• <thatstar>
• <think>
• <topic>
• <topicstar>
• <uppercase>
• <version>

[1] Forward-compatible processing is not yet implemented, so the value of the version attribute won't make any difference to the interpreter. Hopefully this will be fixed before a new version of the AIML standard comes out.

[2] The format of the date is defined by the value of the dateFormat predicate, which must be a valid Java date and time pattern string. This makes it possible to change the date format at runtime (after the bot learns the country the user lives in, for example).

[3] If the output bot predicate contains a file path, the current implementation of this tag writes the results of processing its contents to that file; if the bot predicate contains a directory path, the results will be written into a file named gossip-<value of the <id> tag>.txt, which will be created if it doesn't already exist.

[4] The current implementation of this tag returns the value of the id bot predicate, if it is defined; otherwise, it will return the value of the Context object's hash code.

[5] As implemented, this tag executes code in the context of the window Javascript object of an underlying browser. This limit valid invokations to standard environment features (such as the alert() function) and top-level user-defined functions, but it is well enough to add a nice AJAX-like responsiveness to web bots.

[6] As implemented by ChatterBean, this tag passes the result of processing its contents to a BeanShell script interpreter, and returns the contents of the result enviroment variable (or the value of the last expression, if it evaluates to something and result was not set). Not only this implementation makes the tag more portable, it gives the botmaster access to a powerful Java scripting environment.

Third-Party Includes

• BeanShell - A scripting environment and framework for the Java language.
• jUnit - Java implementation of the xUnit testing framework.

Download, Building and Usage Information

• Download ChatterBean 00.008 Binary Distribution
• Download ChatterBean 00.008 Source Distribution

ChatterBean is currently distributed in two versions: binary and source. The former is more convenient for end-users who just want to play with an AIML chatterbot on their local machines, while the later is aimed at developers who want to study and play with the implementation.

ChatterBean is a Java 5 application, so you'll need release 1.5.0 or later of either the Java Run-Time Environment (if you just intend to run the binary version) or the Java Development Kit (if you want to build it from source). For details, go to Sun's Java Homepage.

Binary Version

If all you want is to run an AIML chatterbot on your local machine, you can download ChatterBean's binary-only version. Unpack the ZIP file to a convenient directory, open a command-line prompt and type:

java -cp bsh.jar -jar chatterbean.bin.00.008.jar path_to/properties.xml

Where path_to/properties.xml is the path to the configuration file of the bot you intend to run. If you need an example, look in the Sample Alicebots section.

Source Version

If you are a developer interested in studying an AIML interpreter or embedding AIML features into your application, then you can download ChatterBean's source version. Create a new directory to host the application, then download and unpack the distribution package into it. The base directory contains some BeanShell scripts that automate the most common tasks related to ChatterBean; you can run these scripts with the help of the bsh.jar executable JAR file. Opening a command prompt inside the program's installation directory, you can type:

• java -jar bsh.jar build.bsh program tests - to compile the source files (you can omit the "tests" keyword to skip the compilation of the test classes);
• java -jar bsh.jar unit.bsh - to run the unit tests;
• java -jar bsh.jar acceptance.bsh - to run the acceptance tests;
• java -jar bsh.jar console.bsh - to run ChatterBean in command-line interactive mode. To exit, simply type ENTER on an empty line;
• java -jar bsh.jar gui.bsh - to run ChatterBean in GUI interactive mode;
• java -jar bsh.jar build.bsh program jar - to compile the source files and build the binary JAR file.

The purpose of bsh.jar is allowing users to run ChatterBean's distribution scripts regardless the OS they use. If you have any problems with it, please contact me.

Sample Alicebots

• Ifurita - A proof-of-concept AJAX-ready Alicebot. Ifurita is a character from a 90's japanese animation series, El Hazard: The Wanderers — a cute, clumsy "demon goddess" (android). Her features include:
• The ability to learn new responses from the user (tell her "this is not a good answer" when you get something unsactisfatory) and save them as a cookie for later retrieval:
• When asked to do so, Ifurita opens an external web page in the underlying inline frame. She can also do so as part of the response for user inputs;
• The picture showing her face can change as part of her response.

Licensing Information

Copyleft © 2005-2006 Hélio Perroni Filho
[email protected]
ICQ: 2490863

ChatterBean is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

ChatterBean is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with ChatterBean (look at the Programj/Alpha/Documents/ directory of the program archive); if not, either write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA, or visit (http://www.gnu.org/licenses/gpl.txt).

Credits

AIML was developed by Dr. Richard S. Wallace and the A.L.I.C.E. Foundation. For more information on AIML, visit the A.L.I.C.E. homepage.

� 2005 H�lio Perroni Filho ([email protected]).