Software May 2016 Tokenizer A tool for creating rule-based tokenizers. Download Tokenizer is a tool for creation rule based tokenizers. LICENSE Download tokenizer Download user manual Download sources