Friday, March 11, 2005

How to use java.io.StreamTokenizer

"You are standing at the end of a road before a small brick building. Around you is a forest. A small stream flows out of the building and down a gully."
(Crowther & Woods 1976)

Introduction

This program converts a string, for example Joe, Go West, then type "hello, world!" on the Acme-Tech computer's keyboard. into words and symbols (Roberts 2002) the same way as a real text adventure program such as TADS would do.

Program

import java.io.StreamTokenizer;
import java.io.StringReader;

class Adventure {
    public static void main(String[] args) {
        String command = "Joe, Go West, then type \"hello, world!\" on the Acme-Tech computer's keyboard.";
        
        StreamTokenizer st = new StreamTokenizer(new StringReader(command));
        st.resetSyntax();
        st.wordChars('a', 'z');
        st.wordChars('A', 'Z');
        st.wordChars(128 + 32, 255);
        st.whitespaceChars(0, ' ');
        st.quoteChar('"');
        st.parseNumbers();
        st.wordChars('\'', '\'');
        st.wordChars('-', '-');
        st.ordinaryChar('.');
        st.lowerCaseMode(true);
        
        try {
            while (st.nextToken() != st.TT_EOF) {
                if (st.ttype == st.TT_WORD) {
                    System.out.println(st.sval);
                } else
                    if (st.ttype == st.TT_NUMBER) {
                    System.out.println(st.nval);
                    } else
                        if ((char) st.ttype == '"') {
                    System.out.println((char) st.ttype + st.sval + (char) st.ttype);
                        } else {
                    System.out.println((char) st.ttype);
                        }
            }
        } catch (java.io.IOException e) {
        }
    }
}

Output

[user]$ javac Adventure.java
[user]$ java Adventure
joe
,
go
west
,
then
type
"hello, world!"
on
the
acme-tech
computer's
keyboard
.
  • The example code performs lexical analysis or tokenization on the text String command
  • that could be used for text adventures.
  • Usually, java.util.StringTokenizer is not powerfull enough for tokenization, so use java.io.StreamTokenizer
  • java.io.StringReader is used to tokenize strings
  • If the current token is a word, then ttype == TT_WORD, and sval contains the word string
  • If the current token is a quoted string token, then the value of ttype is the quote character, and sval contains the body of the string
  • If the current token is a single character token then ttype contains the single character, converted to an integer, and the value of sval is null

References

Author Crowther, Will & Woods, Don
Year 1976
Title Adventure
Place
Publisher
Retrieved 7 april 2006
URL http://www.rickadams.org/adventure/

Author Roberts, Michael J.
Year 2002
Title The TADS 2 Author’s Manual, Version 2.5.5b.
Place
Publisher
Retrieved 7 april 2006
URL http://www.tela.bc.ca/tads-manual/

No comments: