Arabic
arabic
tokenizer
morphology
nlp
dialect

Commit History

  1. Update README.md

    Verified signature
    a4db065
  2. Update README.md

    Verified signature
    e675913
  3. vocab_size Increased from 64000 to 128000

    Verified signature
    34fc84b

  1. Update README.md

    Verified signature
    d83bb67
  2. Update README.md

    Verified signature
    f308444
  3. Update README.md

    Verified signature
    073d643
  4. Delete tokenization_df_arc.py

    Verified signature
    7956c14
  5. Delete phrase_vocab.json

    Verified signature
    46d7f48
  6. Upload custom Unigram tokenizer (v1)

    Verified signature
    ea23987

  1. Update README for v1.1 release

    Verified signature
    bedd199
  2. Release v1.1: PMI Phrase Merging & Smart Morphology

    Verified signature
    bc06a6e
  3. Release v1.1: PMI Phrase Merging & Smart Morphology

    Verified signature
    4730d8d
  4. Remove obsolete phrases.json

    Verified signature
    acdecdf
  5. Remove exceptions.txt (logic is now embedded)

    Verified signature
    2a0bf20
  6. Release v1.1: PMI Phrase Merging & Smart Morphology

    Verified signature
    70e75dc
  7. Delete exceptions.txt

    Verified signature
    daeaea7
  8. Release v1.1: PMI Phrase Merging & Smart Morphology

    Verified signature
    b892abd
  9. Release v1.1: PMI Phrase Merging & Smart Morphology

    Verified signature
    f1a695e
  10. Release v1.1: PMI Phrase Merging & Smart Morphology

    Verified signature
    dfdf13b

  1. Update README.md

    Verified signature
    60eb243
  2. Upload folder using huggingface_hub

    Verified signature
    562053c
  3. Upload folder using huggingface_hub

    Verified signature
    37aebb1
  4. Upload folder using huggingface_hub

    Verified signature
    8941194
  5. Upload folder using huggingface_hub

    Verified signature
    6443890
  6. Update README.md

    Verified signature
    1b8fc9e
  7. Upload folder using huggingface_hub

    Verified signature
    3b90e9e
  8. initial commit

    Verified signature
    1376fa6