Encode each agent
Local observations become agent embeddings that preserve the information needed for interaction.
AAMAS 2021 · Coordination graphs
Deep implicit coordination graphs.
A learned dynamic graph discovers which agent interactions matter, then turns that structure into coordinated action.
Learned graph structure
02
DomainMulti-agent RL
StructureDynamic coordination graph
LearningAttention + graph convolution
VenueAAMAS 2021
01 The problem
Coordinate the team, not every pair.
Centralized control becomes difficult as the joint action space grows, while purely decentralized agents can miss the interactions required to solve a task. Coordination graphs offer a middle ground, but traditional graph structures often depend on domain expertise.
Deep Implicit Coordination Graphs remove that hand-designed bottleneck. The model infers a changing interaction structure from agent observations, then uses graph reasoning to model joint values or actions where coordination is actually needed.
Q / 02
Who needs to coordinate with whom—and how should that structure change with the situation?
02 The architecture
Infer the graph. Reason over it.
Attention weights form an implicit coordination graph over encoded agent observations. Graph convolution propagates information along the learned edges, producing representations that can support both centralized and decentralized execution regimes.
Infer dynamic edges
Attention learns the strength of pairwise relationships instead of relying on a fixed expert-designed graph.
Propagate coordinated context
Graph convolution moves information through the inferred structure before policies produce actions.
03 What it showed
Structure can be learned alongside behavior.
The model learns a useful position between full centralization and independent action.
Resolves relative overgeneralization
DICG addresses the coordination pathology in predator–prey tasks where individually reasonable actions can fail jointly.
Strong across distinct domains
The architecture outperformed multi-agent baselines on SMAC and traffic-junction environments.
Supports two execution regimes
The learned structure applies to centralized-training centralized-execution and centralized-training decentralized-execution.
04 / Paper
Published at AAMAS 2021
Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning
International Conference on Autonomous Agents and Multiagent Systems · 2021