Bilingual
Quark-135M
A lightweight bilingual (Italian + English) language model with 135M parameters.
Features GQA, SwiGLU, RMSNorm, and RoPE. Trained on 50B+ tokens of curated data.
View Model →Our Models
Scaled
Quark-270M
Our scaled small model featuring 270M parameters, 32 layers, and 768 hidden dimensions.
Equipped with a 65K vocabulary. Designed for extended bilingual capabilities.
In ProductionUltra-Compact
Quark-Math-Code ~36M
Deep-thin architecture engineered specifically for STEM, coding, and mathematical reasoning.
14 layers, 65K vocabulary. Actively pre-training on a 5B token target with a hardened Chain-of-Thought (CoT), OpenWebMath, and pure-code mix.
Safety
Quark-Mod
A multi-label moderation model covering 9 categories for safe AI deployment.
Detects: toxic, severe_toxic, obscene, threat, insult, identity_hate, cyberbullying, hate_speech, offensive.
View Model →What We Focus On
⚡
Hyper-Efficient Architectures
Mastering the sub-1B parameter space using GQA, Grouped-Query Attention, and deep-thin layer scaling.
🧠
Embedded Chain-of-Thought (CoT)
Hardcoding step-by-step reasoning tokens into the pre-training phase of tiny models to punch far above their weight class.
🌍
Bilingual & Specialty Data
Multi-source streaming pipelines fusing Italian, English, high-density mathematics, and code.
💻
Open-Source & Deployable
Everything from weights to datasets is open. Tailored to achieve massive throughput on consumer GPUs and edge hardware.
Resources
| Resource | Description |
|---|---|
| 📚 Quark-135M-Bilingual | Our flagship general-purpose bilingual model. |
| 🛡️ Quark-Mod | Multi-label content moderation for production pipelines. |
| Quark-v0.1 ⚡️ | All our released models |
| 💻 Our GitHub | Training scripts, custom multi-source streaming iterators, and deployment tools. |