Ambassador

Hunter

📌

Another super impressive multi-modal AI model out of China. The core members of the founding Moonshot.ai team "participated in the research and development of many large models such as Google Gemini, Google Bard, Pangu NLP, and Wudao. Many core technologies have been adopted by mainstream products such as Google PaLM, Meta LLaMa, and Stable Diffusion." Key Technologies of Kimi k1.5 Long Context Scaling Supports reinforcement learning (RL) generation with up to 128k tokens. Improves training efficiency through partial rollout techniques, avoiding the cost of regenerating trajectories from scratch. Improved Policy Optimization Employs the online mirror descent (OMD) algorithm. Combines effective sampling strategies, length penalty, and other optimization methods. Multi-Modalities Supports joint reasoning over text and vision modalities. Model Performance Long-CoT Model: Kimi k1.5's Long-CoT version achieves performance comparable to OpenAI's o1 across multiple benchmarks, including AIME, MATH 500, Codeforces, and MathVista. Short-CoT Model Using the long2short method, Kimi k1.5's Short-CoT version outperforms existing short-CoT models such as GPT-4o and Claude Sonnet 3.5 in benchmarks like AIME, MATH 500, and LiveCodeBench, with performance improvements of up to 550%.

Report

6mo ago