Hugging Face's logo
luca's picture

luca

lukecage
2
·

AI & ML interests

None yet

Recent Activity

liked a model about 7 hours ago
akatz-ai/MiniMax-H3-Character-Swap-LoRA
updated a model about 2 years ago
lukecage/Phi-3.5-mini-ITA-Q8_0-GGUF
reacted to anakin87's post with 🔥 about 2 years ago
💬 🇮🇹 Phi 3.5 mini ITA: a Small Language Model for Italian Lately, I've spent some time fine-tuning language models. Now I am happy to release Phi 3.5 mini ITA: a fine-tuned version of Phi-3.5-mini-instruct to improve performance on the Italian language 🔹 Small (3.82 B parameters) but capable model 🔹 128k context length Chat with it on 🤗 Spaces: https://huggingface.co/spaces/anakin87/Phi-3.5-mini-ITA Model card: https://huggingface.co/anakin87/Phi-3.5-mini-ITA 🗃️ Data Supervised fine-tuning using a good mix of English and Italian data: - https://huggingface.co/datasets/mlabonne/FineTome-100k by @mlabonne - https://huggingface.co/datasets/efederici/capybara-claude-15k-ita by @efederici 🙏 Thanks to the authors for the datasets. 🎯 Targeted training with Spectrum I used Spectrum, a relatively new technique for parameter-efficient learning. The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and ❄️ freeze the rest. I trained the top 30% of model layers. 📝 Spectrum paper: https://arxiv.org/abs/2406.06623 📊 Vibe check and performance on Italian benchmarks seem encouraging
View all activity

Organizations

None yet