hypaai/Hypa-SmolLM-135M-Instruct-LoRAs

Hypa SmolLM 135M — Keyboard (LoRA adapter)

The unmerged LoRA adapter behind Hypa's 27-language smart-keyboard model.

This repo holds the adapter weights only. It exists for people who want to merge the adapter onto their own base checkpoint, continue training from it, or inspect what was learned. If you just want to run the model, take the GGUF for on-device use or the BF16 weights for Transformers and vLLM.

The model does the four things a keyboard has to do — predict the next word, complete the word being typed, fix the last word, and clean up a whole sentence — in Igbo, Yorùbá, Hausa, Efik, Tiv, Igede, Eggon and twenty others, alongside English, French, Spanish, Portuguese and Arabic.

Built by Hypa Intelligence. Trained on Hypa-Keyboard-v2.

⚠️ Status: Read Limitations before using this for anything.

---

Which build do I want?

Repo	Format	Size	Use it when
-GGUF	GGUF Q4_K_M	105 MB	On-device, llama.cpp, Ollama, LM Studio, mobile
-16bit	BF16 safetensors	0.1B params	Transformers or vLLM; converting or quantising yourself
→ -LoRAs (this one)	PEFT adapter	—	Merging onto your own base, or continued training

---

What it does

Five tasks, each selected by its system prompt:

Task	System prompt	Behaviour
Next-word prediction	You are Hypa Keyboard. Predict the next word.	Text ends at a word boundary → emit the next word
Word completion	You are Hypa Keyboard. Complete the current word.	Text ends mid-word → finish the word being typed
Last-word correction	You are Hypa Keyboard. Correct the last word.	Fix only the final, just-typed token
Block correction	You are Hypa Keyboard. Correct the text block.	Return a clean version of the whole span
Grammar correction	You are Hypa Keyboard. Correct grammar, missing words, spelling, and sentence errors.	Full grammatical error correction over the span

Correction is trained against a controlled corruption vocabulary that treats tone-mark damage as a first-class error type. Losing diacritics is the single most common failure when typing tonal orthographies on a standard mobile keyboard, so restoring them is a core capability rather than an afterthought.

---

Usage

This is a PEFT adapter, not a standalone model. It loads on top of the base:

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "hypaai/Hypa-SmolLM-135M-Instruct-LoRAs"

model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

The base is a 4-bit bitsandbytes checkpoint, so you'll need bitsandbytes installed and a CUDA device.

Calling a keyboard task

def keyboard(system_prompt, text, max_new_tokens=8):
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user",   "content": text},
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
    ).to(model.device)

    outputs = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False)
    return tokenizer.decode(
        outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True
    )


keyboard("You are Hypa Keyboard. Predict the next word.", "Ndewo, kedu ka ị")
keyboard("You are Hypa Keyboard. Complete the current word.", "Ẹ káàbọ̀ sí ilé ìwé wa, a")
keyboard("You are Hypa Keyboard. Correct the last word.", "I dey go markit")
keyboard("You are Hypa Keyboard. Correct the text block.",
         "Omi Omi kp anya'ami mail ekubo uche r'abo ohigbeli mi.", max_new_tokens=64)

Use greedy decoding (do_sample=False) and keep max_new_tokens low — 4–8 for prediction and completion, 32–64 for the correction tasks.

Merging

merged = model.merge_and_unload()
merged.save_pretrained("hypa-keys-135m-merged")
tokenizer.save_pretrained("hypa-keys-135m-merged")

The already-merged result is published as -16bit, so merge yourself only if you're targeting a different base.

---

Training

Base model	unsloth/smollm-135m-instruct-bnb-4bit (SmolLM-135M-Instruct)
Method	LoRA (PEFT) via TRL SFTTrainer, accelerated with Unsloth
Dataset	hypaai/Hypa-Keyboard-v2 — 409,598 examples
Languages	27
LoRA rank (r)
LoRA alpha
LoRA dropout
Target modules
modules_to_save
Learning rate
Epochs / steps
Effective batch size
Max sequence length
Hardware

Loss curves are under the Training metrics tab.

Framework versions

• TRL 1.9.2
• Transformers 5.13.1
• PyTorch 2.11.0+cu128
• Datasets 3.6.0
• Tokenizers 0.22.2
• PEFT

---

Evaluation

Not yet published. See -16bit for the reference weights used in evaluation.

---

Limitations

• Adapter size.
• No per-language evaluation. The training dataset has no language column, so quality across the 27 languages is unmeasured and certainly uneven. Expect better results in Hausa, Igbo, Yorùbá and Swahili than in Eggon, Igede, Ebira or Nupe.
• Quantised base. The adapter is trained against a 4-bit bnb checkpoint, which constrains merging and redeployment. Merging onto a full-precision SmolLM-135M-Instruct will not reproduce these results exactly.
• Synthetic training noise. Corruptions were programmatically injected. The model has not seen real keyboard-layout adjacency errors (fat-finger typos), swipe-typing failures, or genuine mid-sentence code-switching — all of which dominate actual mobile input.
• Close-relative confusion. Efik, Ibibio and Annang share substantial vocabulary and orthography. A correction valid in one may be applied to text written in another.
• Source-formatting leakage. Some training spans carried Markdown, prompt fragments and JSON punctuation, so the model occasionally treats prompt-like text as ordinary typing.
• Not a chat model. Despite the instruct base, this is trained for five narrow keyboard tasks. Conversation, question answering and translation are out of scope and will produce poor output.

---

Intended use

For: merging onto a compatible base, continued training, and research on low-resource keyboard modelling. For running the model, use the GGUF or BF16 builds.

Not for: general text generation, translation, question answering, or any setting where output is treated as authoritative text in these languages. Corrections must be shown as suggestions the user can reject, never applied silently — a wrong autocorrect in a language the user speaks and the model barely knows is worse than no autocorrect at all.

---

Citation

@misc{hypaai2026hypakeys,
  title        = {Hypa SmolLM 135M: A Compact Multilingual Keyboard Model for African Languages},
  author       = {Hypa Intelligence},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/hypaai/Hypa-SmolLM-135M-Instruct-LoRAs}}
}

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}

License

Apache 2.0, inherited from SmolLM-135M-Instruct.

Contact

Hypa Intelligence • Website • Hugging Face • GitHub • Blog

Trained with Unsloth and Hugging Face TRL.