xtcp2

xtcp2 is a high-performance Linux daemon that streams kernel TCP socket state — the same rich diagnostics you get from ss --info — out of the kernel via the netlink inet_diag interface, across every network namespace on the host, and fans the results out to a configurable destination (Kafka, NATS, NSQ, Valkey, stdout/stderr/file, TCP, HTTP(S), UDP, Unix sockets, S3/Parquet, or /dev/null).

It is a ground-up reimplementation of randomizedcoder/xtcp, rewritten with two goals in mind:

• Performance — zero-copy-friendly netlink parsing, sync.Pool-backed buffers and protobuf messages, parallel netlink readers, and an optional io_uring fast path (Linux 6.1+).
• Container / namespace visibility — xtcp2 discovers namespaces by scanning /proc/<pid>/ns/net, uses bind-mounted namespace paths for stable names when available, spawns a dedicated reader per namespace, and reconciles namespace churn (Kubernetes pods, containers) in real time. You see TCP metrics for every container, not just the host.

The typical deployment streams length-delimited protobuf batches to Kafka/Redpanda for ingestion into ClickHouse, but the destination is pluggable and selectable at build time.

---

Architecture at a glance

  /proc/<pid>/ns/net + bind-mounted names  (namespace discovery + reconcile)
        │
        ▼
  ┌─────────────────┐   one per network namespace
  │ namespace reader │  setns(CLONE_NEWNET) → opens a netlink socket in that ns
  └─────────────────┘
        │
        ▼
  ┌─────────────────┐   N parallel netlinkers per ns (-netlinkers)
  │   netlinker(s)   │  send inet_diag dump, recv raw netlink packets
  └─────────────────┘
        │
        ▼
  ┌─────────────────┐   inet_diag attributes → XtcpFlatRecord
  │   deserialize    │  (tcp_info, congestion, meminfo, BBR, DCTCP, …)
  └─────────────────┘
        │
        ▼
  ┌─────────────────┐   accumulate rows into an Envelope; flush on
  │  poll / batch    │  rows (-envelopeFlushRows) or bytes (-envelopeFlushBytes)
  └─────────────────┘
        │
        ▼
  ┌─────────────────┐   protobufList | protoJson | protoText | msgpack
  │   marshaller     │  jsonl | csv | tsv
  └─────────────────┘
        │
        ▼
  ┌─────────────────┐   stdout | stderr | file | tcp | http(s) | udp | unix
  │   destination    │  unixgram | null | kafka | nats | nsq | valkey | s3parquet
  └─────────────────┘

  Side channels:  gRPC API (:8889)  •  Prometheus metrics (:9088)  •  pprof / Pyroscope
                 TCP or UDS listeners  •  optional bearer / HMAC listener auth

For the full picture see the documentation hub.

---

Quick start

xtcp2 builds and runs with Nix. It is a Linux-only tool and needs CAP_NET_ADMIN (read TCP socket state) and CAP_SYS_ADMIN (enter namespaces) — in practice, run it as root or under sudo. It refuses to start if the hard-required capabilities are missing, printing exactly what it needs. For multi-namespace visibility it also needs CAP_SYS_PTRACE: namespace discovery reads /proc/<pid>/ns/net of other processes, which the kernel gates behind ptrace_may_access even for root. Without it xtcp2 starts (with a warning) but sees only its own namespace — every container/pod netns is invisible.

# 1. Clone
git clone https://github.com/randomizedcoder/xtcp2.git
cd xtcp2

# 2. Build the main daemon (or `nix develop` for a full dev shell)
nix build .#xtcp2

# 3. Run it against the local host, discarding output, with verbose logging
sudo ./result/bin/xtcp2 -dest null -d 333

Stream to Kafka/Redpanda instead:

sudo ./result/bin/xtcp2 -dest kafka:127.0.0.1:9092 -topic xtcp

Inspect TCP records live over gRPC, in a second terminal:

nix build .#xtcp2client
./result/bin/xtcp2client            # streams XtcpFlatRecord from the running daemon

Run xtcp2 -help for the full flag list. Common flags:

Flag	Default	Purpose
-dest	kafka:redpanda-0:9092	Destination, scheme:address (see destinations)
-topic	xtcp	Kafka / NSQ topic
-frequency	10s	Poll interval
-marshal	protobufList	Wire format (protobufList, protoJson, protoText, msgpack, jsonl, csv, tsv)
-netlinkers	4	Parallel netlink readers per namespace
-deserializers	all	Which inet_diag attributes to decode
-promListen	:9088	Prometheus metrics listener
-grpcPort	8889	gRPC API port
-d	111	Debug verbosity (higher = more)

---

Features

Feature	Summary	Docs
Netlink TCP collection	Reads TCP socket state via inet_diag; 13 pluggable attribute decoders (tcp_info, congestion, meminfo, BBR, DCTCP, skmem, cgroup, …).	netlink-collection
Multi-namespace visibility	Discovers live namespaces from /proc, uses bind-mounted namespace names when available, and reconciles one reader set per namespace.	network-namespaces
Polling & batching	Periodic dumps accumulated into protobuf Envelopes, flushed by row-count or byte-size thresholds.	polling-and-batching
Output formats & destinations	Seven marshallers; stdlib destinations are always built, while Kafka, NATS, NSQ, Valkey, and S3/Parquet are build-tagged.	output-and-destinations
gRPC API & listener security	Runtime config get/set, live record streaming, TCP or UDS listeners, and optional bearer/HMAC authentication.	grpc-api
Metadata enrichment	Container, LLDP/NIC, NSID/uplink, ASN, and destination-locality enrichment. ASN/locality require matching build flavors. Current ASN is representative provider ASN from feed-owner metadata; true BGP RIB/MRT origin ASN and next-hop ASN are future work.	build-flavors, ASN enrichment, locality enrichment
Observability	Prometheus metrics, pprof on the metrics listener, Pyroscope continuous profiling, startup capability checks.	observability
Performance	Optional io_uring I/O, typed sync.Pool wrappers, parallel readers, thread-cap tuning.	performance
Testing & quality	Real captured netlink .pcap fixtures across many kernel versions, reflection-free typed parsers, custom audits, unit tests, and microVM lifecycle checks.	testing-and-quality

---

Binaries

The daemon is xtcp2; the repo also ships supporting tools under cmd/:

Binary	Purpose
xtcp2	The main daemon.
xtcp2client	gRPC client; streams live XtcpFlatRecords from a running daemon.
xtcp2ctl	gRPC control CLI; changes a running daemon's config over ConfigService (poll cadence, on-demand polls/bursts, S3 upload timing, envelope flush caps, soft-restart reconfigure).
xtcp2_kafka_client	Kafka consumer that decodes xtcp2's protobufList messages.
ns	Namespace inspector — lists/reads netns state the way the daemon sees it.
nsTest	Namespace churn driver for soak/stress testing.
register_schema	Registers the xtcp_flat_record proto with a Confluent Schema Registry.
kafka_to_clickhouse	Bridge: consumes the Kafka topic and writes to ClickHouse.
clickhouse_protobuflist, clickhouse_protobuflist_db, clickhouse_http_insert_protobuflist	ClickHouse ingest tools for the protobufList format.
grpcurl	Vendored grpcurl for poking the gRPC API.

---

Documentation

• Documentation hub — background, design, and the full feature index.
• CONTRIBUTING.md — Nix build targets, the test suite, linting, and proto regeneration for developers.
• Operations notes — running the ClickHouse / Redpanda pipeline with docker-compose.

License

See the repository for license details.