Class BinaryDictionaryWriter

java.lang.Object
org.apache.lucene.analysis.ja.util.BinaryDictionaryWriter
Direct Known Subclasses:
TokenInfoDictionaryWriter, UnknownDictionaryWriter

abstract class BinaryDictionaryWriter extends Object
  • Field Details

    • ID_LIMIT

      private static final int ID_LIMIT
      See Also:
    • implClazz

      private final Class<? extends BinaryDictionary> implClazz
    • buffer

      protected ByteBuffer buffer
    • targetMapEndOffset

      private int targetMapEndOffset
    • lastWordId

      private int lastWordId
    • lastSourceId

      private int lastSourceId
    • targetMap

      private int[] targetMap
    • targetMapOffsets

      private int[] targetMapOffsets
    • posDict

      private final ArrayList<String> posDict
  • Constructor Details

  • Method Details

    • put

      public int put(String[] entry)
      put the entry in map
      Returns:
      current position of buffer, which will be wordId of next entry
    • isKatakana

      private boolean isKatakana(String s)
    • writeKatakana

      private void writeKatakana(String s)
    • toKatakana

      private String toKatakana(String s)
    • sharedPrefix

      private static int sharedPrefix(String left, String right)
    • addMapping

      void addMapping(int sourceId, int wordId)
    • getBaseFileName

      final String getBaseFileName()
    • write

      public void write(Path baseDir) throws IOException
      Write dictionary in file Dictionary format is: [Size of dictionary(int)], [entry:{left id(short)}{right id(short)}{word cost(short)}{length of pos info(short)}{pos info(char)}], [entry...], [entry...].....
      Throws:
      IOException - if an I/O error occurs writing the dictionary files
    • writeTargetMap

      private void writeTargetMap(Path path) throws IOException
      Throws:
      IOException
    • writePosDict

      private void writePosDict(Path path) throws IOException
      Throws:
      IOException
    • writeDictionary

      private void writeDictionary(Path path) throws IOException
      Throws:
      IOException