Krafton releases Raon-1 mini preview with 262K context

Krafton released Raon-1 mini preview on Hugging Face under the Apache 2.0 license. The company describes it as a preview whose specifications and results may change before the final release.

Raon-1 mini uses a sparse mixture of experts architecture with 28.8 billion total parameters and 4.3 billion active parameters per token. It combines 18 linear attention layers with seven full attention layers and supports a context window of up to 262,144 tokens.

The model supports Korean, English, Japanese, Chinese, Spanish, French and Thai, along with tool calling for agent workflows. It runs through Transformers, vLLM, llama.cpp and MLX. Quantized MXFP8 and NVFP4 versions are available, with the NVFP4 version able to run on one NVIDIA RTX 5090 GPU with 32 GB of memory.

Krafton also released a DFlash2 draft model for speculative decoding. Tests on one NVIDIA B300 showed up to 2.18 times higher output throughput than autoregressive decoding, depending on the hardware and workload.