Abstract: The widespread adoption of open-source software (OSS) necessitates the mitigation of vulnerability risks. Most vulnerability detection (VD) methods are limited by inadequate contextual understanding, restrictive single-round interactions, and coarse-grained evaluations, resulting in undesired model performance and biased evaluation results. To address these challenges, we propose MAVUL, a novel multi-agent VD system that integrates contextual reasoning and interactive refinement. Specifically, a vulnerability analyst agent is designed to flexibly leverage tool-using capabilities and contextual reasoning to achieve cross-procedural code understanding and effectively mine vulnerability patterns. Through iterative feedback and refined decision-making within cross-role agent interactions, the system achieves reliable reasoning and vulnerability prediction. Furthermore, MAVUL introduces multi-dimensional ground truth information for fine-grained evaluation, thereby enhancing evaluation accuracy and reliability.

Extensive experiments conducted on a pairwise vulnerability dataset demonstrate MAVUL's superior performance. Our findings indicate that MAVUL significantly outperforms existing multi-agent systems with over 62% higher pairwise accuracy and single-agent systems with over 600% higher average performance. The system's effectiveness is markedly improved with increased communication rounds between the vulnerability analyst agent and the security architect agent, underscoring the importance of contextual reasoning in tracing vulnerability flows and the crucial feedback role. Additionally, the integrated evaluation agent serves as a critical, unbiased judge, ensuring a more accurate and reliable estimation of the system's real-world applicability by preventing misleading binary comparisons.

This system implements a multi-agent workflow for vulnerability detection:

1. ReAct Agent (Vulnerability Analyst Agent): Performs initial vulnerability analysis using reasoning and action
2. Reflexion Agent (Security Architect Agent): Reviews and critiques the ReAct agent's analysis
3. Evaluation Agent: Evaluates the correctness of analyses

• OpenAI and LANGCHAIN API key (set your api keys in .env)
• Required packages (see environment.yml)

1. Clone the repository:

   git clone https://github.com/youpengl/MAVUL.git

2. Install dependencies:

   conda env create -f environment.yml

1. Get the outputs file (e.g., results_gpt-4o.json)

python main.py

2. Evaluate the agent output using the llm-as-a-judge

python llm_as_a_judge.py

Feel free to contact me via email if you have any questions :)

@misc{li2025mavulmultiagentvulnerabilitydetection,
      title={MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement}, 
      author={Youpeng Li and Kartik Joshi and Xinda Wang and Eric Wong},
      year={2025},
      eprint={2510.00317},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2510.00317}, 
}