ICML 2025 | Multi-Agent ChatGPT in Action? Shanghai Jiao Tong MAS-GPT Achieves One-Click Workflow Generation}

First author: Ye Rui, a third-year PhD student at Shanghai Jiao Tong University, specializing in multi-agent large models, federated learning. Advisor: Associate Professor Chen Siheng.

OpenAI regards “Organizational AI” as the fifth crucial stage toward AGI—aiming for AI to operate like a highly efficient, collaborative organization handling complex tasks and large-scale operations. Multi-Agent Systems (MAS) are key to this vision.

However, building MAS capable of supporting such complex intelligence is challenging, with issues like complex structures, prompt tuning, and general task handling hindering widespread adoption.

Now, a new approach emerges: MAS-GPT, developed jointly by Shanghai Jiao Tong University, Shanghai Artificial Intelligence Laboratory, Oxford University, and others. It introduces a generative MAS design paradigm that, with just one query, can produce a complete, organized MAS!

This means constructing MAS becomes as simple as chatting with ChatGPT—a single question yields a full multi-agent system! MAS-GPT aims to make the path to the fifth stage of AGI smoother and more efficient.

This work, “MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems”, was presented at ICML 2025.

• Paper link: https://arxiv.org/abs/2503.03686
• Code link: https://github.com/MASWorks/MAS-GPT
• Model link: https://huggingface.co/MASWorks/MAS-GPT-32B

Generative MAS: One sentence input, automatic MAS creation

Existing MAS methods (like ChatDev, DyLAN, AFlow) face three core issues:

• Lack of adaptability: Highly dependent on manual prompts, with poor flexibility;
• High costs: Multi-round LLM calls make design expensive;
• Low generalization: Overfitting to specific test sets, limiting applicability.

These problems hinder MAS’s widespread use. For example, in systems like ChatGPT handling massive concurrent requests, current MAS paradigms cannot meet scalability and robustness demands.

How does MAS-GPT break this deadlock? The answer is:

Transform “MAS design” into a language generation task! Input your query, and the output is a ready-to-run multi-agent system!

This MAS is elegantly presented as Python code:

• Agent prompts: clear Python variables
• Agent responses: function calls, core logic
• Agent interactions: string concatenation, efficient
• Agent tools: Python functions, limitless extension

From now on, MAS is no longer “handwritten,” but “model-generated!”

How to teach LLM “to design MAS”?

Training MAS-GPT is not about memorization but involves a sophisticated data construction process that teaches the model “what kind of MAS to design for each query.”

Four steps to build high-quality training data:

1. Data pool construction: Collect diverse queries covering math, code, general QA, and over 40 MAS code structures;

2. Pair evaluation: Automated assessment and annotation of each “Query-MAS” pair;

3. Pair selection: Use inter-consistency principles to match similar queries with the best MAS;

4. Pair refinement: Use large models to rewrite MAS, add reasoning explanations, and ensure logical fit with queries.

Finally, 11,000 high-quality samples are used to fine-tune open-source models, resulting in MAS-GPT.

With MAS-GPT, reasoning in multi-agent systems becomes unprecedentedly simple.

Users submit a query, MAS-GPT generates a dedicated MAS, which executes immediately and returns the answer—step by step.

Multiple experiments confirm: MAS-GPT is not only clever but also powerful!

In tests on 8 benchmarks × 5 mainstream models, compared with over 10 existing methods, MAS-GPT:

• Is more accurate: Outperforms the strongest baselines by an average of 3.89%!
• Has better generalization: Performs robustly even on unseen tasks like GPQA and SciBench!

It also reduces reasoning costs to nearly half of other systems like DyLAN and GPTSwarm, while maintaining top performance.

• Highly compatible: MAS-GPT-generated MAS works consistently across different LLMs, showing excellent versatility and universality.

Further expanding the capabilities of reasoning large models

MAS-GPT generated MAS can be used not only for chatbots but also to assist more powerful reasoners. Using strong reasoning models like OpenAI o1 and DeepSeek-R1 combined with MAS-GPT, on the AIME-2024 math challenge:

• o1 + MAS-GPT improved by 13.3%
• DeepSeek-R1 + MAS-GPT improved by 10.0%

MAS-GPT truly has the ability to organize and coordinate powerful models!

Training scalability and future potential of MAS-GPT!

Besides performance and usability, MAS-GPT’s training parameter space has huge potential for exploration, indicating vast future development prospects!

Beyond template copying: generating new structures!

Deep visual analysis shows that MAS-GPT is capable of:

• Automatically creating novel MAS structures
• Providing reasonable agent roles and cooperation strategies for unseen tasks
• Adding reasoning explanations to each MAS, clarifying “why it was designed this way”

It truly learns to design, not just memorize answers!

Future vision of MAS-GPT

MAS-GPT proposes an unprecedented idea: automatically generate a MAS for each query. In theory, all domain-specific multi-agent systems could be integrated into MAS-GPT’s training data, enabling continuous evolution and more sophisticated MAS generation.

As the development path of LLMs shows, with increasing base model capabilities and richer data, MAS-GPT will keep evolving.

In the near future, your intelligent interaction might not just be with a chatbot, but with a powerful MAS-GPT that understands your questions deeply and customizes the best AI system—be it a simple agent or a complex multi-agent network.

MASWorks: The open-source community for large model multi-agent systems

MAS-GPT is part of the recently launched open-source community MASWorks, dedicated to connecting researchers worldwide, sharing knowledge, and fostering development in the field of multi-agent systems.

The community’s first major event is the ICML 2025 workshop: MAS-2025.

MASWorks invites global researchers and developers to contribute, share code, expand their network, and collaboratively shape the future of MAS!

• Community: https://github.com/MASWorks
• Event: https://mas-2025.github.io/MAS-2025/