| # dispatchAI SDK |
|
|
| **Small. Mobile. Free. UAE-built.** |
|
|
| `pip install dispatchai` — Run mobile-optimized LLMs on your phone, edge device, or laptop. 31 verified models, all tested on real Snapdragon hardware, all free. |
|
|
| ## Quick Start |
|
|
| ```bash |
| pip install dispatchai[gguf] |
| ``` |
|
|
| ### Chat with a model |
|
|
| ```python |
| from dispatchai import load_model |
| |
| model = load_model("SmolLM2-135M-Instruct-mobile", backend="gguf") |
| response = model.chat("What is the capital of France?") |
| print(response) |
| # → "The capital of France is Paris." |
| ``` |
|
|
| ## 🌐 Inference API |
|
|
| Use dispatchAI models via REST API (OpenAI-compatible): |
|
|
| ```python |
| import openai |
| |
| client = openai.OpenAI( |
| base_url="https://api.dispatchai.ai/v1", |
| api_key="da-demo-key-0001" |
| ) |
| |
| response = client.chat.completions.create( |
| model="dispatchAI/SmolLM2-135M-Instruct-mobile", |
| messages=[{"role": "user", "content": "What is the capital of France?"}] |
| ) |
| print(response.choices[0].message.content) |
| # → "The capital of France is Paris." |
| ``` |
|
|
| **Pricing:** $0.001/1K input tokens, $0.002/1K output tokens (10x cheaper than OpenAI) |
|
|
| **Endpoint:** `https://api.dispatchai.ai/v1` |
|
|
| **Available Models:** |
| - dispatchAI/SmolLM2-135M-Instruct-mobile (101MB, 46 t/s on phone) |
| - dispatchAI/Qwen2.5-0.5B-Instruct-mobile-int4 (469MB, 23 t/s on phone) |
| - dispatchAI/Llama-3.2-1B-Instruct-Q4-mobile (770MB, 5.4 t/s on phone) |
|
|
| ## Local Inference |
|
|
| ### Find the best model for your phone |
|
|
| ```python |
| from dispatchai import recommend |
| |
| rec = recommend(ram_mb=2048, task="chat") |
| print(f"Best model: {rec['recommended']['name']}") |
| ``` |
|
|
| ### List all models |
|
|
| ```python |
| from dispatchai import list_models |
| |
| for m in list_models(task="chat"): |
| print(f" {m['name']}: {m['size_mb']}MB, {m['speed_tps']} t/s") |
| ``` |
|
|
| ### Estimate latency |
|
|
| ```python |
| from dispatchai import estimate_latency |
| |
| lat = estimate_latency("1B", "Q4_K_M") |
| print(f"{lat['tokens_per_sec']} t/s on Snapdragon 865") |
| ``` |
|
|
| ### Calculate cost savings |
|
|
| ```python |
| from dispatchai import calculate_cost |
| |
| result = calculate_cost(daily_queries=10000, cloud_cost_per_1k=0.50) |
| print(f"Annual savings: ${result['savings']}") |
| ``` |
|
|
| ## Installation Options |
|
|
| ```bash |
| pip install dispatchai # Core (model catalog, recommendations) |
| pip install dispatchai[torch] # + transformers/torch backend |
| pip install dispatchai[gguf] # + llama.cpp GGUF backend |
| pip install dispatchai[full] # + everything |
| ``` |
|
|
| ## Verified Models (June 2026) |
|
|
| - ✅ 31 models fully working (0 broken, 0 partial) |
| - 📱 24 models phone-verified on Snapdragon 865 |
| - All have correct chat formats documented |
|
|
| ## Top 3 Models |
|
|
| | Model | Size | Phone Speed | Use Case | |
| |-------|------|-------------|----------| |
| | SmolLM2-135M | 101MB | 46.0 t/s | Ultra-fast, budget phones | |
| | Qwen2.5-0.5B-int4 | 469MB | 23.2 t/s | Best balance for mobile | |
| | Llama-3.2-1B-Q4 | 770MB | 5.4 t/s | Best quality under 1GB | |
|
|
| ## About |
|
|
| Dispatch AI (FZE) — Sharjah Free Zone, UAE. License No. 10818. |
|
|
| 🌐 [dispatchai.ai](https://www.dispatchai.ai) | 🤗 [huggingface.co/dispatchAI](https://huggingface.co/dispatchAI) | API: [api.dispatchai.ai](https://api.dispatchai.ai) |
|
|
| *I think, therefore I ship.* |
|
|
