tailscale-exit-bypass¶

Modules

A NixOS module that selectively diverts chosen egress off a Tailscale exit node and back onto the host's own WAN — per destination CIDR, and optionally per destination port.

The problem¶

When a host is configured to route all its traffic through a Tailscale exit node, everything leaves via the tailnet — including traffic that must not. The classic case is an outer tunnel: a VPN client (WireGuard, etc.) running on the host, or inside a local VM, whose outer UDP packets need to reach the Internet directly. Double-encapsulating that tunnel through the exit node is at best wasteful and at worst broken.

You want a scalpel: "traffic to these destinations (optionally on this port) leaves via my real WAN; everything else keeps using the exit node."

Why it's harder than one `ip rule`¶

The naive approach — mark the packets in an output chain and add a policy routing rule — silently fails for exactly the sockets you care about. There are three traps:

type route hook output, not a plain output chain. Marking a packet does not, by itself, make the kernel re-run the route lookup. Sockets that connect() early and cache their destination route (QEMU/SLIRP user-mode networking and many UDP sockets do this) never see the route the new mark selects — they keep the cached one, still pointing at the exit node. The nftables route hook forces a route re-lookup after the mark is set, so those cached-dest sockets actually get diverted.
MASQUERADE on the marked flow is mandatory, not optional. Those same early-connect() sockets also cache their source address — chosen while the route still pointed at the tailscale interface, so it's a tailnet / CGNAT (RFC 6598 shared address space) address. After the mark flips egress to WAN, the packets would leave with that now-wrong tailnet source and the replies get dropped as bogons. A MASQUERADE (srcnat) on the marked flow rewrites the source to the outgoing WAN interface address.
ip rule priority must sit below Tailscale's window. Tailscale installs its own policy rules in roughly the 5210–5290 priority range. The bypass rule must be consulted first, so its priority has to be numerically lower (checked earlier). It points at the main table, which still holds the real WAN default — because Tailscale's exit default route lives in a separate table (commonly table 52), never in main.

Get any one of these wrong and the bypass appears installed but doesn't take effect (or takes effect and then black-holes the replies).

What the module installs, per named route¶

An nft table inet tailscale-exit-bypass-<name> with:
a type route hook output priority mangle chain that matches <optional dport> ip daddr @bypass_targets and stamps the route's fwmark;
a type nat hook postrouting priority srcnat chain that MASQUERADEs the marked flow.
An ip -4 rule add fwmark <mark> lookup main priority <prio>.
A oneshot systemd service that resolves the destination set and applies the rules atomically (reload is idempotent).
A timer that re-asserts every reassertSeconds (60s default), so the bypass survives tailscale up re-runs, transient config-fetch failures, and manual ip rule fiddling. Apply short-circuits on an unchanged hash of cidrs + fwmark + rule, so steady-state cost is a few syscalls per tick.
An optional probe service/timer that runs ip route get <sample> mark <fwmark> and reports degraded (to a JSON file under /run/tailscale-exit-bypass/) if the egress device still matches the forbidden pattern (default tailscale[0-9]*).

Usage¶

Import default.nix and declare routes:

{
  imports = [ ./tailscale-exit-bypass ];

  services.tailscaleExitBypass = {
    enable = true;

    routes.vpn-outer = {
      # Static destinations to send straight out the WAN.
      cidrs = [ "203.0.113.10" "198.51.100.0/24" ];

      # Optional: only match a specific transport/port. Narrow the bypass
      # to just the outer tunnel's UDP port, for example.
      udpDport = 51820;

      # Mark + rule priority. Keep the mark clear of tailscale's
      # 0x80000/0xff0000 mask; keep the priority below ~5210.
      fwmark = "0x42";
      rulePriority = 4500;
    };
  };
}

For destinations that rotate, use cidrSourceCommand instead of (or alongside) cidrs — any command/script/derivation that prints one IPv4 or CIDR per line. It runs at apply-time and on every reassert; its output is unioned with cidrs, deduped, and re-validated against a strict IPv4 regex before it reaches nft:

services.tailscaleExitBypass.routes.dynamic = {
  cidrSourceCommand = pkgs.writeShellScript "endpoints" ''
    grep -hoE '([0-9]{1,3}\.){3}[0-9]{1,3}' /etc/myapp/peers.conf
  '';
  fwmark = "0x43";
  rulePriority = 4501;
};

Options¶

Option	Default	Notes
`enable`	`false`	Master switch.
`reassertSeconds`	`60`	Reassert-timer interval for every route.
`routes.<name>.enable`	`true`	Per-route switch.
`routes.<name>.cidrs`	`[ ]`	Static IPv4 addresses / CIDRs to bypass.
`routes.<name>.cidrSourceCommand`	`null`	Command/path/derivation printing extra CIDRs, re-run each reassert.
`routes.<name>.udpDport`	`null`	Restrict match to this UDP destination port.
`routes.<name>.tcpDport`	`null`	Restrict match to this TCP destination port.
`routes.<name>.fwmark`	(required)	Hex mark, e.g. `"0x42"`. Avoid tailscale's `0x80000/0xff0000`.
`routes.<name>.rulePriority`	(required)	ip rule priority, e.g. `4500`. Must be below ~5210.
`routes.<name>.probe.enable`	`true`	Periodic health probe.
`routes.<name>.probe.intervalSeconds`	`300`	Probe interval.
`routes.<name>.probe.forbiddenDevPattern`	`"tailscale[0-9]*"`	BRE the egress dev must NOT match to be healthy.

You must set at least one of cidrs or cidrSourceCommand per route (enforced by an assertion).

Caveats¶

Fail-open. If the rules are absent (never installed, or torn down), traffic simply falls back to the host default route — i.e. the exit node. There is no IP leak beyond "it used the tailnet path you were trying to avoid." If you need fail-closed, add your own REJECT rule in a parallel nft table.
IPv4 only. The ruleset, the rule, and the CIDR validation are v4. Add a parallel v6 path if you need it.
MTU. Re-routing off tailscale0 (~1280) onto WAN (~1500) is usually fine; a WireGuard outer socket sets DF and PMTUD applies. If a path black-holes, pin the inner tunnel's MTU lower at the consumer.
fwmark / priority hygiene. The mark and priority are global routing state. Pick values that don't collide with tailscale or with other modules on the host.
CIDR validation is defense-in-depth. Everything (static and command-sourced) is filtered through a strict IPv4 regex before it reaches nft, so a compromised resolver, malformed config, or hostile source line can't inject arbitrary nft syntax.

Source¶

modules/tailscale-exit-bypass/default.nix

# tailscale-exit-bypass
#
# Selectively divert chosen egress (by CIDR, and optionally by dport) OFF a
# Tailscale exit node and back onto the host's own WAN, using an fwmark +
# policy-routing rule plus a small nftables ruleset.
#
# Why this is subtle (the three traps this module exists to solve):
#
#   1. `type route hook output` — a plain `output` mangle chain marks the
#      packet but does NOT force the kernel to re-run the route lookup. Sockets
#      that connect() early and cache their destination route (notably
#      QEMU/SLIRP and other UDP sockets) never see the new mark's route. The
#      `route` hook triggers a route re-lookup after the mark is set, so those
#      cached-dest sockets actually get diverted.
#
#   2. MASQUERADE on the marked flow — the same early-connect() sockets also
#      cache their SOURCE address, chosen while the route still pointed at the
#      tailscale interface (a tailnet/CGNAT src). After we flip egress to WAN,
#      packets would leave with a now-wrong tailnet src and replies get dropped
#      as bogons. MASQUERADE on the marked flow rewrites src to the outgoing
#      interface address.
#
#   3. ip rule priority — the diverting rule must sit BELOW tailscale's rule
#      window (~5210-5290) so it is consulted first. It targets the `main`
#      table, which still holds the real WAN default, because tailscale's exit
#      default route lives in a separate table (commonly 52), not in `main`.
#
# Failure mode is fail-OPEN: if the rules are absent, traffic falls back to the
# host default route (the tailscale exit). No IP leak, just the un-bypassed
# path. For fail-closed behaviour, add your own REJECT in a parallel nft table.
#
# This module is self-contained: import it, set `enable = true`, and declare
# one or more named `routes`.

{
  config,
  lib,
  pkgs,
  ...
}:
let
  cfg = config.services.tailscaleExitBypass;

  enabledRoutes = lib.filterAttrs (_: r: r.enable) cfg.routes;

  buildApplyScript =
    name: route:
    pkgs.writeShellScript "tailscale-exit-bypass-${name}-apply" ''
      set -euo pipefail
      umask 077
      PATH=${
        lib.makeBinPath [
          pkgs.coreutils
          pkgs.gawk
          pkgs.gnugrep
          pkgs.gnused
          pkgs.iproute2
          pkgs.nftables
        ]
      }:$PATH

      state_dir=/run/tailscale-exit-bypass
      state_hash="$state_dir/${name}.hash"
      install -d -m 0700 "$state_dir"

      static_cidrs=${lib.escapeShellArg (lib.concatStringsSep "\n" route.cidrs)}
      raw="$static_cidrs"
      ${lib.optionalString (route.cidrSourceCommand != null) ''
        if extra=$(${route.cidrSourceCommand} 2>&1); then
          raw="$raw"$'\n'"$extra"
        else
          echo "tailscale-exit-bypass[${name}]: cidrSourceCommand failed: $extra" >&2
        fi
      ''}

      cidr_re='^([0-9]{1,3}\.){3}[0-9]{1,3}(/[0-9]{1,2})?$'
      cidrs=$(printf '%s\n' "$raw" | grep -E "$cidr_re" | sort -u || true)

      if [[ -z "$cidrs" ]]; then
        echo "tailscale-exit-bypass[${name}]: no valid CIDRs after validation, refusing to install" >&2
        exit 1
      fi

      new_hash=$(
        {
          printf '%s\n' "$cidrs"
          printf 'fwmark=%s\n' "${route.fwmark}"
          printf 'rule_priority=%s\n' "${toString route.rulePriority}"
          ${lib.optionalString (route.udpDport != null) ''
            printf 'udp_dport=%s\n' "${toString route.udpDport}"
          ''}
          ${lib.optionalString (route.tcpDport != null) ''
            printf 'tcp_dport=%s\n' "${toString route.tcpDport}"
          ''}
        } | sha256sum | awk '{print $1}'
      )
      old_hash=$(cat "$state_hash" 2>/dev/null || true)
      table_present=0
      nft list table inet tailscale-exit-bypass-${name} >/dev/null 2>&1 && table_present=1
      rule_present=0
      ip -4 rule show priority ${toString route.rulePriority} \
        | grep -q "fwmark ${route.fwmark}" && rule_present=1

      if [[ "$new_hash" == "$old_hash" \
            && "$table_present" == "1" \
            && "$rule_present" == "1" ]]; then
        exit 0
      fi

      set_elems=$(printf '%s\n' "$cidrs" | paste -sd, -)
      count=$(printf '%s\n' "$cidrs" | wc -l)

      nft -f - <<NFT
      add table inet tailscale-exit-bypass-${name}
      delete table inet tailscale-exit-bypass-${name}
      table inet tailscale-exit-bypass-${name} {
        set bypass_targets {
          type ipv4_addr
          flags interval
          elements = { $set_elems }
        }
        chain output_mark {
          type route hook output priority mangle; policy accept;
          ${
            if route.udpDport != null then
              "udp dport ${toString route.udpDport} ip daddr @bypass_targets meta mark set ${route.fwmark}"
            else if route.tcpDport != null then
              "tcp dport ${toString route.tcpDport} ip daddr @bypass_targets meta mark set ${route.fwmark}"
            else
              "ip daddr @bypass_targets meta mark set ${route.fwmark}"
          }
        }
        chain postrouting_snat {
          type nat hook postrouting priority srcnat; policy accept;
          meta mark ${route.fwmark} masquerade
        }
      }
      NFT

      while ip -4 rule del priority ${toString route.rulePriority} 2>/dev/null; do :; done
      ip -4 rule add fwmark ${route.fwmark} lookup main priority ${toString route.rulePriority}

      printf '%s\n' "$new_hash" > "$state_hash"
      printf '%s\n' "$cidrs" > "$state_dir/${name}.cidrs"
      echo "tailscale-exit-bypass[${name}]: installed $count CIDRs, mark ${route.fwmark} → main @ prio ${toString route.rulePriority}"
    '';

  buildTeardownScript =
    name: route:
    pkgs.writeShellScript "tailscale-exit-bypass-${name}-teardown" ''
      set -u
      PATH=${
        lib.makeBinPath [
          pkgs.nftables
          pkgs.iproute2
          pkgs.coreutils
        ]
      }:$PATH
      nft delete table inet tailscale-exit-bypass-${name} 2>/dev/null || true
      while ip -4 rule del priority ${toString route.rulePriority} 2>/dev/null; do :; done
      rm -f /run/tailscale-exit-bypass/${name}.hash \
            /run/tailscale-exit-bypass/${name}.cidrs \
            /run/tailscale-exit-bypass/${name}.probe.json 2>/dev/null || true
    '';

  buildProbeScript =
    name: route:
    pkgs.writeShellScript "tailscale-exit-bypass-${name}-probe" ''
      set -uo pipefail
      PATH=${
        lib.makeBinPath [
          pkgs.coreutils
          pkgs.gawk
          pkgs.gnugrep
          pkgs.gnused
          pkgs.iproute2
          pkgs.jq
        ]
      }:$PATH

      state_dir=/run/tailscale-exit-bypass
      cidrs_file="$state_dir/${name}.cidrs"
      out_file="$state_dir/${name}.probe.json"
      now=$(date -u +%Y-%m-%dT%H:%M:%SZ)

      write_status() {
        jq -n \
          --arg ts "$now" \
          --arg status "$1" \
          --arg msg "$2" \
          --arg sample "$3" \
          --arg dev "$4" \
          --arg fwmark "${route.fwmark}" \
          '{ ts: $ts, route: "${name}", status: $status, message: $msg,
             sample: $sample, dev: $dev, fwmark: $fwmark }' \
          > "$out_file"
      }

      if [[ ! -f "$cidrs_file" ]]; then
        write_status "unknown" "cidrs file missing — apply hasn't run" "" ""
        echo "tailscale-exit-bypass-probe[${name}]: cidrs file missing" >&2
        exit 1
      fi

      sample=$(head -n1 "$cidrs_file" | sed 's,/.*,,')
      if [[ -z "$sample" ]]; then
        write_status "unknown" "cidrs file empty" "" ""
        exit 1
      fi

      route_out=$(ip -4 route get "$sample" mark ${route.fwmark} 2>&1 || true)
      dev=$(printf '%s\n' "$route_out" | grep -oE 'dev [^ ]+' | head -n1 | awk '{print $2}')

      if [[ -z "$dev" ]]; then
        write_status "degraded" "ip route get returned no dev: $route_out" "$sample" ""
        echo "tailscale-exit-bypass-probe[${name}]: no dev in route output" >&2
        exit 1
      fi

      if [[ "$dev" =~ ^${route.probe.forbiddenDevPattern}$ ]]; then
        write_status "degraded" "egress dev '$dev' matches forbidden pattern" "$sample" "$dev"
        echo "tailscale-exit-bypass-probe[${name}]: DEGRADED — sample $sample → dev $dev (expected non-tailscale)" >&2
        exit 1
      fi

      write_status "ok" "egress dev '$dev'" "$sample" "$dev"
    '';

  routeOpts =
    { ... }:
    {
      options = {
        enable = lib.mkOption {
          type = lib.types.bool;
          default = true;
          description = "Per-route disable switch (the parent enable flag also gates everything).";
        };

        cidrs = lib.mkOption {
          type = lib.types.listOf lib.types.str;
          default = [ ];
          example = [
            "203.0.113.10"
            "198.51.100.0/24"
          ];
          description = "Static IPv4 addresses or CIDRs to bypass off the exit node.";
        };

        cidrSourceCommand = lib.mkOption {
          type = lib.types.nullOr (lib.types.either lib.types.str lib.types.path);
          default = null;
          example = lib.literalExpression ''pkgs.writeShellScript "harvest" "grep -hoE '[0-9.]+' /etc/endpoints"'';
          description = ''
            Optional shell command, absolute path, or derivation that
            prints one IPv4 address or CIDR per line on stdout. Run at
            apply-time and re-run on every reassert. Output is unioned
            with `cidrs`, deduped, and re-validated against a strict IPv4
            regex before reaching the nft ruleset. Use this for
            destinations that rotate (e.g. parsed from a config file the
            application refreshes).
          '';
        };

        udpDport = lib.mkOption {
          type = lib.types.nullOr lib.types.port;
          default = null;
          description = "If set, only UDP traffic with this destination port matches the bypass.";
        };

        tcpDport = lib.mkOption {
          type = lib.types.nullOr lib.types.port;
          default = null;
          description = "If set, only TCP traffic with this destination port matches the bypass.";
        };

        fwmark = lib.mkOption {
          type = lib.types.str;
          example = "0x42";
          description = ''
            Hex fwmark stamped on matched packets and used by the ip
            rule. Must not collide with tailscale's 0x80000/0xff0000
            mask or other host modules' marks. A small value such as
            0x40-0x4f is a safe convention.
          '';
        };

        rulePriority = lib.mkOption {
          type = lib.types.int;
          example = 4500;
          description = ''
            ip rule priority (lower = checked first). Must be below
            tailscale's window (~5210-5290) for the bypass to take
            effect. A value in the 4500-4599 range is a safe convention.
          '';
        };

        probe = {
          enable = lib.mkOption {
            type = lib.types.bool;
            default = true;
            description = "Run a periodic probe asserting the bypass is in effect.";
          };
          intervalSeconds = lib.mkOption {
            type = lib.types.int;
            default = 300;
            description = "How often to run the probe.";
          };
          forbiddenDevPattern = lib.mkOption {
            type = lib.types.str;
            default = "tailscale[0-9]*";
            description = ''
              Regex (BRE) that the egress device MUST NOT match for the
              bypass to be considered healthy. The probe runs
              `ip route get <sample-cidr> mark <fwmark>` and inspects
              `dev <X>`; if `<X>` matches this pattern, the probe reports
              degraded status.
            '';
          };
        };
      };
    };
in
{
  options.services.tailscaleExitBypass = {
    enable = lib.mkEnableOption "host-side policy routing that diverts marked traffic off the default tailscale exit node and onto the main routing table";

    routes = lib.mkOption {
      type = lib.types.attrsOf (lib.types.submodule routeOpts);
      default = { };
      description = ''
        Named bypass routes. Each becomes its own nft table
        (`inet tailscale-exit-bypass-<name>`), oneshot service,
        reassert timer, and ip rule.
      '';
    };

    reassertSeconds = lib.mkOption {
      type = lib.types.int;
      default = 60;
      description = ''
        How often to re-run the apply script for each route. The script
        short-circuits when the resolved set + nft state + ip rule are
        all unchanged, so the steady-state cost is a few syscalls per
        tick. Reasserting survives `tailscale up` re-runs, transient
        config-fetch failures, and manual `ip rule` fiddling.
      '';
    };
  };

  config = lib.mkIf (cfg.enable && enabledRoutes != { }) {
    assertions = lib.mapAttrsToList (name: route: {
      assertion = (route.cidrs != [ ]) || (route.cidrSourceCommand != null);
      message = "services.tailscaleExitBypass.routes.${name}: must set at least one of cidrs or cidrSourceCommand.";
    }) enabledRoutes;

    systemd.services =
      lib.mapAttrs' (
        name: route:
        lib.nameValuePair "tailscale-exit-bypass-${name}" {
          description = "Tailscale exit-node bypass route '${name}' (mark ${route.fwmark} → main @ prio ${toString route.rulePriority})";
          after = [
            "tailscaled.service"
            "network-online.target"
          ];
          wants = [
            "tailscaled.service"
            "network-online.target"
          ];
          wantedBy = [ "multi-user.target" ];
          serviceConfig = {
            Type = "oneshot";
            RemainAfterExit = true;
            ExecStart = buildApplyScript name route;
            ExecReload = buildApplyScript name route;
            ExecStop = buildTeardownScript name route;
          };
        }
      ) enabledRoutes
      // lib.mapAttrs' (
        name: _:
        lib.nameValuePair "tailscale-exit-bypass-${name}-reassert" {
          description = "Re-assert tailscale-exit-bypass route '${name}' (timer-driven)";
          serviceConfig = {
            Type = "oneshot";
            ExecStart = "${pkgs.systemd}/bin/systemctl try-reload-or-restart tailscale-exit-bypass-${name}.service";
          };
        }
      ) enabledRoutes
      // lib.mapAttrs' (
        name: route:
        lib.nameValuePair "tailscale-exit-bypass-${name}-probe" {
          description = "Probe tailscale-exit-bypass route '${name}' (writes /run/tailscale-exit-bypass/${name}.probe.json)";
          after = [ "tailscale-exit-bypass-${name}.service" ];
          serviceConfig = {
            Type = "oneshot";
            ExecStart = buildProbeScript name route;
          };
        }
      ) (lib.filterAttrs (_: r: r.probe.enable) enabledRoutes);

    systemd.timers =
      lib.mapAttrs' (
        name: _:
        lib.nameValuePair "tailscale-exit-bypass-${name}-reassert" {
          description = "Re-assert tailscale-exit-bypass route '${name}'";
          wantedBy = [ "timers.target" ];
          timerConfig = {
            OnBootSec = "2min";
            OnUnitActiveSec = "${toString cfg.reassertSeconds}s";
            Unit = "tailscale-exit-bypass-${name}-reassert.service";
          };
        }
      ) enabledRoutes
      // lib.mapAttrs' (
        name: route:
        lib.nameValuePair "tailscale-exit-bypass-${name}-probe" {
          description = "Probe tailscale-exit-bypass route '${name}'";
          wantedBy = [ "timers.target" ];
          timerConfig = {
            OnBootSec = "5min";
            OnUnitActiveSec = "${toString route.probe.intervalSeconds}s";
            Unit = "tailscale-exit-bypass-${name}-probe.service";
          };
        }
      ) (lib.filterAttrs (_: r: r.probe.enable) enabledRoutes);
  };
}