-
Update README.md
-
Update README.md
-
vocab_size Increased from 64000 to 128000
-
Update README.md
-
Update README.md
-
Update README.md
-
Delete tokenization_df_arc.py
-
Delete phrase_vocab.json
-
Upload custom Unigram tokenizer (v1)
-
Update README for v1.1 release
-
Release v1.1: PMI Phrase Merging & Smart Morphology
-
Release v1.1: PMI Phrase Merging & Smart Morphology
-
Remove obsolete phrases.json
-
Remove exceptions.txt (logic is now embedded)
-
Release v1.1: PMI Phrase Merging & Smart Morphology
-
Delete exceptions.txt
-
Release v1.1: PMI Phrase Merging & Smart Morphology
-
Release v1.1: PMI Phrase Merging & Smart Morphology
-
Release v1.1: PMI Phrase Merging & Smart Morphology
-
Update README.md
-
Upload folder using huggingface_hub
-
Upload folder using huggingface_hub
-
Upload folder using huggingface_hub
-
Upload folder using huggingface_hub
-
Update README.md
-
Upload folder using huggingface_hub
-
initial commit