org.apache.fontbox.ttf.gsub

## Class CompoundCharacterTokenizer

- java.lang.Object
- - org.apache.fontbox.ttf.gsub.CompoundCharacterTokenizer

    public class CompoundCharacterTokenizer
    extends Object

Takes in the given text having compound-glyphs to substitute, and splits it into chunks consisting of parts that should be substituted and the ones that can be processed normally.

Author:
  Palash Ray

### Constructor Summary

Constructors
Constructor and Description
`CompoundCharacterTokenizer(Pattern pattern)` Deprecated. Use `CompoundCharacterTokenizer(java.util.Set)`
`CompoundCharacterTokenizer(Set<String> compoundWords)` Constructor.

### Method Summary

All Methods Instance Methods Concrete Methods
`List<String>` `tokenize(String text)` Tokenize a string into tokens.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### CompoundCharacterTokenizer

    public CompoundCharacterTokenizer(Set<String> compoundWords)

Constructor. Calls getRegexFromTokens which returns strings like (_79_99_)|(_80_99_)|(_92_99_) and creates a regexp assigned to regexExpression. See the code in GlyphArraySplitterRegexImpl on how these strings were created.

It is assumed the compound words are sorted in descending order of length.

Parameters:
  `compoundWords` - A set of strings like _79_99_, _80_99_ or _92_99_ .

#### CompoundCharacterTokenizer

    @Deprecated
    public CompoundCharacterTokenizer(Pattern pattern)

Deprecated. Use `CompoundCharacterTokenizer(java.util.Set)`
Constructor.

Parameters:
  `pattern` -

### Method Detail

#### tokenize

    public List<String> tokenize(String text)

Tokenize a string into tokens.

Parameters:
  `text` - A string like "_66_71_71_74_79_70_"
Returns:
  A list of tokens like "_66_", "_71_71_", "74_79_70_". The "_" is sometimes missing at the beginning or end, this has to be cleaned by the caller.