Tokenizing

Compares whitespace, treebank, and sentiment tokenizers. Optional downcasing and Porter stemming. For the sentiment version, optional negation marking and elongation normalization.

Enter your own text, or

tokenize a random tweet instead.

Options: