Vibe-Skills

项目详情摘要

English | 中文 Latest release · v4.1.0 Quick start · Measured on SkillsBench Mean verifier reward: +21.12 pp Total tokens: -29.6% · Tool calls: -33.1% SkillsBench** is a benchmark designed to evaluate whether AI agents can effectively use Skills to complete professional tasks across diverse domains. Its purpose is to measure how much a model’s ability to solve complex real-world tasks improves when it is equipped with specialized Skills. To evaluate performance in production-like environments where a large number of Skills are installed simultaneously, we adapted SkillsBench into a more realistic **large-scale multi-Skill setting**. In the original SkillsBench setup, each task is provided only with the specialized Skill associated with that task. In our modified setting, every task is evaluated in a global environment containing all **195 specialized Skills**, while all other experimental conditions remain unchanged. This setting is intended to assess whether an agent can autonomously discover, select, and orchestrate the relevant Skills from a large installed Skill pool, and organize them into an effective workflow for completing complex tasks. vibeskills v4.1.0** was benchmarked on **SkillsBench** (https://www.skillsbench.ai/) using **DeepSeekV4Flash-VE** and **OpenHands** as the baseline evaluation setup. Compared with the baseline without vibeskills, vibeskills increased the **average task score by 21.12%**, while reducing **token consumption by 29.6%** and **tool calls by 33.1%**. Task quality: 39 to 57 full-score tasks; Lean Vibe scored higher on 23 tasks, Native on 4, with 55 ties. Resource use: 491.122M to 345.756M total tokens, with tool calls reduced from 9,954 to 6,664. Analysis of the logs from the original benchmark shows that ***VibeSkills achieves better ta