view post Post 4,392 ๐ Introducing Halo 1.0Today, we are open-sourcing Halo, the training framework we use to train every model at White Circle.It comes with: ๐ง Full post-training stack: SFT, DPO/KTO/SMPO, reward modeling, GRPO, distillation๐ค Async multi-turn RL with vLLM/SGLang rollouts and sandboxed tool useโก ~2.8ร TRL throughput on 8ร B300 (EP+FSDPv2, FA4, fp8/fp4)๐ค Dense HF models + 15 MoE families (Qwen, GLM, Mistral, DeepSeek-V4โฆ)๐ ๏ธ One halo command, prebuilt Docker images, and docs for humans and agents๐ป https://github.com/whitecircle/haloTry it and tell us what you're training See translation 1 reply ยท ๐ 9 9 โค๏ธ 4 4 + Reply
view post Post 90 43% less answer time in a looped transformer by changing answer supervisioncheck out https://x.com/advprop/status/2098087083470373010?s=20 + full writeup linked there See translation ๐ฅ 1 1 + Reply
Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian Paper โข 2405.13929 โข Published May 22, 2024 โข 55