New: NIKA 2.0 has been released

NIKA Break the Network. Put Your AI Agent On Call.

A unified platform for benchmarking AI agents on realistic network troubleshooting tasks.

View on GitHub Read the Paper Leaderboard Join Slack
Scroll to explore
Overview

What is NIKA?

Think about SWE-bench, but for network troubleshooting. Agents interact with live CLIs and telemetry across reproducible incidents on your laptop, cluster, or cloud.

NIKA Architecture Diagram
NIKA architecture: benchmark suite plus orchestrator with MCP tools and evaluation.

Benchmark Suite

NIKA ships 1,098 working-matrix cases across data center, campus, enterprise WAN, ISP, SDN, P4, Kubernetes, and vendor routing labs on Kathará and Containerlab.

# pick a network, a fault, its target
scenario: dc-clos-bgp
problem: bgp-asn-misconfig
inject: { host_name: leaf0 }

Orchestration Platform

A modular platform that deploys labs, injects faults, connects agents through MCP servers for live diagnosis, and scores root-cause submissions.

# deploy, run an agent, evaluate
nika env run dc-clos-bgp
nika agent run --agent claude
nika eval judge
Features

Built for AI + Networking Research

Kathará and Containerlab

Emulated labs on Kathará and Containerlab with real routing stacks (FRR, SR Linux, and more). Run them on a laptop or in the cloud.

Pre-built Scenarios & Faults

Pre-built networks of different sizes, with faults you can introduce for troubleshooting tasks.

MCP Tool Integration

Model Context Protocol servers expose host probes, Pingmesh, packet capture, routing CLIs, and INT telemetry as tools for any MCP-compatible agent.

Reproducible Evaluation

Measure whether agents identify the cause of network incidents, and compare their results on the same benchmark tasks.

Network Scenarios

Live Networks and Workloads

Each incident runs on a concrete topology with realistic services. Catalog: network scenario reference.

Scalable

Data Center Clos

FRR eBGP Clos (dc_clos) on Kathará, or Nokia SR Linux Clos (min3clos) on Containerlab.

Data center Clos: super-spine, spines, and leaves
BGPECMPFRRSR Linux
Scalable

Campus LAN

Hierarchical campus with OSPF, DHCP, DNS, and a server farm (campus_lan).

Campus LAN: users, access, distribution, core, and server farm
OSPFDHCPDNS
Scalable

Enterprise WAN

Multi-site hub-and-spoke WAN with provider underlay, WireGuard, eBGP overlay, and per-role VRFs (enterprise_branch).

Enterprise WAN: HQ and DC hubs, branches, dual ISP underlay, WireGuard overlay
WireGuardeBGPVRF
Scalable

ISP Backbone

SNDlib graphs such as Abilene and France (isp_*), on Kathará (FRR) or Containerlab (SR Linux), with optional IGP and BGP presets.

ISP SNDlib backbone mesh with stub hosts
SNDlibIS-ISBGP
Scalable

SDN and P4 Fabrics

ONOS + OVS L3 Clos (sdn_l3_clos), and BMv2 fabrics under P4Runtime (p4_dc_fabric, p4_dc_gateway with INT-MX).

SDN and P4 Clos: controller over spines and leaves with endpoints
ONOSP4BMv2
Fixed

Kubernetes Networking

k3s labs for multi-tenant services (k8s_lab) and simulated llm-d inference routing (llmd_lab).

Kubernetes lab: k3s cluster on FRR fabric with exit path to client
k3sMetalLBllm-d
Failures

Faults Across Network Domains

75 registered types across nine domains. Catalog: failure reference.

Link & Interface

6 types

link_down, flap, capacity bottleneck, packet corruption.

Routing & Control Plane

8 types

BGP ASN mismatch, max-prefix, OSPF adjacency, daemon crash.

Forwarding, Encapsulation & Policy

26 types

ACLs, static routes, SDN/P4 rules, MTU, WireGuard, NetworkPolicy.

Service Networking

6 types

Kubernetes ClusterIP and load-balancer faults.

Management & Orchestration

4 types

API-server reachability, SDN controller, southbound channels.

Addressing, Neighbor & Naming

13 types

IP/MAC conflicts, ARP, DHCP, DNS data and reachability.

Endpoint & Application

2 types

Endpoint resource contention and application delay.

Traffic, Queueing & Resource

3 types

Offered load, queueing, and host resource constraints.

Security

7 types

DoS, BGP hijack, DHCP/DNS spoofing, ARP poisoning.

Define a Task

Define a Troubleshooting Task

Choose a network, its size, and the fault an agent will investigate.

my_tests.yaml 
cases:
  - scenario: dc_clos
    topo_size: s
    problem: link_down
    inject:
      host_name: client_0
      intf_name: eth0
  - scenario: campus_lan
    topo_size: s
    problem: link_down
    inject:
      host_name: dhcp_server
      intf_name: eth0

What Defines a Task

  • Network

    Choose a network, such as a data center or campus LAN.

  • Network size

    Choose a small, medium, or large topology where supported.

  • Fault

    Choose the incident to reproduce, such as a failed link.

  • Fault location

    Specify where the fault occurs in the network.

See the task authoring guide for definitions and instructions for adding networks and faults.

Agents & Harnesses

Agent Integration

Evaluate built-in or custom agents on the NIKA benchmark.

Custom Agents

Connect your existing agent to NIKA and evaluate it on network troubleshooting tasks.

Coding Assistants

Put general-purpose coding assistants to the test on realistic network incidents, out of the box.

Native Agents

Want to focus on the creation of new tasks, not agent plumbing? Use default agents and go ahead.

Community

Explore agents and skills shared by the community, and contribute your own. See the leaderboard.

Quick Start

Up and Running in Minutes

See the setup guide for installation requirements and configuration details.

1

Install

Clone the repository and run ./scripts/install.sh. It installs Docker (if needed), uv, Kathará, Containerlab, and Python dependencies, then creates .env and config/nika.yaml from the examples when missing.

bash 
# Clone the repository
git clone https://github.com/sands-lab/nika.git
cd nika

# Install Docker, uv, Kathará, Containerlab, and Python deps
./scripts/install.sh
2

Configure

Put API keys in .env. In config/nika.yaml, set agent.type, provider, and model. Run uv run nika config show to confirm.

.env 
# Keys in .env (install.sh creates it from .env.example when missing)
OPENAI_API_KEY=<your-key>
# or ANTHROPIC_API_KEY / DEEPSEEK_API_KEY

# config/nika.yaml — pick a type from `nika agent list`
# agent:
#   type: byo.langgraph
#   provider: openai
#   model: gpt-5-mini

# Confirm merged config
uv run nika config show
3

Run a benchmark

Run a frozen release, then summarize the scores.

bash 
# Frozen release, then summarize
uv run nika benchmark run --release 0.2.0 --split test \
  --result_dir results/my-run --batch-size 4
uv run nika eval summary --result_dir results/my-run
4

Optional: smoke one incident

Run a single problem end-to-end, or deploy a lab yourself and drive it with the CLI.

bash 
# One incident: deploy + inject + agent + eval
uv run nika agent run -a byo.langgraph -p openai -m gpt-5-mini \
  --problem dc_clos_s_link_down

# Or deploy a lab only
uv run nika env list
uv run nika env run dc_clos -s s
MCP Servers & Tools

MCP & Network Telemetry

Agents inspect the emulated network through Model Context Protocol (MCP) tools.

Network Telemetry

  • ping_pair / traceroute: reachability and paths
  • iperf_test: bandwidth between hosts
  • Pingmesh: packet loss and round-trip time
  • int_query_telemetry: per-hop traces in supported P4 networks

Packet Capture

  • packet_capture_start: capture network traffic
  • packet_capture_stop: end a capture
  • packet_capture_inspect: inspect packets and protocols

Routing Diagnostics

  • BGP and OSPF configuration
  • BGP session state
  • Routing tables
Citation

Cite NIKA

If you use NIKA in your research, please cite the following papers.

NIKA Pre-print

ArXiv →
BibTeX 
@misc{nika25long,
  title          = {A Network Arena for Benchmarking AI Agents on Network Troubleshooting},
  author         = {Zhihao Wang and Alessandro Cornacchia and Alessio Sacco and Franco Galante and Marco Canini and Dingde Jiang},
  year           = {2025},
  eprint         = {2512.16381},
  archivePrefix  = {arXiv},
  primaryClass   = {cs.NI},
  url            = {https://arxiv.org/abs/2512.16381},
}

NGNO Workshop at ACM SIGCOMM '25

DOI →
BibTeX 
@inproceedings{nika25ngno,
  title        = {Towards a Playground to Democratize Experimentation and Benchmarking of AI Agents for Network Troubleshooting},
  author       = {Wang, Zhihao and Cornacchia, Alessandro and Galante, Franco and Centofanti, Carlo and Sacco, Alessio and Jiang, Dingde},
  year         = {2025},
  publisher    = {Association for Computing Machinery},
  url          = {https://doi.org/10.1145/3748496.3748990},
  booktitle    = {Proceedings of the 1st Workshop on Next-Generation Network Observability},
  location     = {Coimbra, Portugal},
  series       = {NGNO '25},
}