15 Commits
Author SHA1 Message Date
justme e2e5e437cb docs(deploy): document QUIC runtime and port requirements [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline failed
ci/woodpecker/push/dev-image unknown status
2026-10-10 03:00:47 +08:00
justme b25b111134 feat(display): add compact QUIC and host health views 2026-10-10 03:00:23 +08:00
justme b964dc2bbc feat(quic): replace probe UDP data plane with QUIC datagrams 2026-10-10 02:59:56 +08:00
justme 4609048865 docs: merge the API guidelines into api.md; drop the one-off trial reports; fix the node-update description (Quadlet restart vs host updater); tidy TODO [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-09 22:31:11 +08:00
justme 900bbde46a ci: run the slow e2e tests after the image builds (a parallel run got the backend AOT compile OOM-killed) 2026-10-09 15:24:07 +08:00
justme 2d31bd091e ci: gate on the fast tests (about 20s); slow end-to-end tests run in parallel with the image build [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/e2e Pipeline was successful
ci/woodpecker/push/dev-image Pipeline failed
2026-10-09 14:33:03 +08:00
justme c9c21f8056 ci: fix the failed-test retry (wildcard filter, ignore zero-test exit code, two attempts) [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-09 14:09:54 +08:00
justme b2b63c065d ci: sync packaging and docs into the test cache; retry failed tests once [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline failed
ci/woodpecker/push/dev-image unknown status
2026-10-09 14:02:42 +08:00
justme 0873acf66a fix(link): ride out short path blackouts: 30s handshake, 180s idle limit, 90s keepalive, faster probe reconnect [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline failed
ci/woodpecker/push/dev-image unknown status
2026-10-09 13:46:59 +08:00
justme be04045c8c ci: incremental backend tests, cached web dependencies, web build in parallel with the tests
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/verify Pipeline failed
2026-10-09 13:39:55 +08:00
justme 006814823b docs: main-only workflow, CI-filled versions, dev version format [CI]
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-09 13:20:30 +08:00
justme 7ff0d0f4e6 feat: print build info at startup; version placeholders filled in by CI [CI SKIP] 2026-10-09 13:20:30 +08:00
justme a9f78f99df ci: version from the tag or the commits since it; dev images from main gated on tests; recognise -dev. tags [CI SKIP] 2026-10-09 13:20:30 +08:00
justme a3338e1cf3 fix(display): background quality as a three-way choice (blurred, sharp, original) [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
2026-10-09 04:57:01 +08:00
justme 044ce0c339 feat(display): background photo groups with blurred, compressed and original versions; veil, sway and blur options in the appearance panel [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
2026-10-09 04:50:11 +08:00
154 changed files with 4262 additions and 1039 deletions

No files matched your search

@@ -7,12 +7,19 @@ clone:
image: plugin-git
settings:
lfs: false
# 要拿到 tag 才算得出开发版本号(ci/version.sh:自最近 tag 的提交数)
tags: true
when:
- event: push
branch: main
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
# 先等后端测试和前端构建都通过再出镜像(提交信息写 [SKIP TESTS] 只跳过后端测试)
depends_on:
- verify
- web-build
concurrency: 1
steps:
@@ -28,7 +35,7 @@ steps:
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
commands:
- bash ci/build-main-amd64.sh backend
- bash ci/build-dev-image.sh backend
depends_on: []
- name: build-linux-amd64-agent
@@ -43,7 +50,7 @@ steps:
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
commands:
- bash ci/build-main-amd64.sh agent
- bash ci/build-dev-image.sh agent
depends_on: []
- name: build-linux-amd64-probe
@@ -58,5 +65,20 @@ steps:
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
commands:
- bash ci/build-main-amd64.sh probe
- bash ci/build-dev-image.sh probe
depends_on: []
# 慢的端到端测试:等三个镜像都构建、推送完再跑,不和 NativeAOT 编译抢内存(同时跑过一次,Backend 的 AOT 编译被 OOM 杀掉了)。
# 它失败不影响已经推出去的镜像,只让流水线变红。没带 [CI] 的推送由 e2e.yml 跑。
- name: backend-e2e-tests
image: bash
when:
- evaluate: 'not (CI_COMMIT_MESSAGE contains "[SKIP TESTS]")'
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
commands:
- bash ci/verify-backend.sh slow
depends_on:
- build-linux-amd64-backend
- build-linux-amd64-agent
- build-linux-amd64-probe
+32
View File
@@ -0,0 +1,32 @@
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: plugin-git
settings:
lfs: false
# 带 [CI] 的推送会出镜像,慢测试放在 dev-image 里、等镜像构建完再跑(不和 AOT 编译抢内存);这个 workflow 管其余的:PR 和没带 [CI] 的推送
when:
- event: pull_request
- event: push
branch: main
evaluate: 'not (CI_COMMIT_MESSAGE contains "[CI]")'
# 等 verify 编译完、快的测试过了再跑(共用它的增量编译缓存,不重新编译)
depends_on:
- verify
steps:
- name: backend-e2e-tests
image: bash
# [SKIP TESTS] 和 verify 一样,跳过后端测试
when:
- evaluate: 'not (CI_COMMIT_MESSAGE contains "[SKIP TESTS]")'
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
commands:
# 标了 Category=Slow 的端到端测试,见 ci/verify-backend.sh
- bash ci/verify-backend.sh slow
+2
View File
@@ -22,6 +22,8 @@ steps:
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
# 版本取自 tag:先写进工作区(仓库里的版本号只是占位),前端才会带上对的版本
- bash ci/version.sh apply
- npm ci --prefix web --prefer-offline
- timeout 600 npm run build --prefix web
depends_on: []
+3 -2
View File
@@ -11,7 +11,7 @@ clone:
when:
- event: pull_request
- event: push
branch: dev
branch: main
# On Gitea PR events, CI_COMMIT_MESSAGE is the PR title.
steps:
@@ -23,4 +23,5 @@ steps:
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
commands:
- cd backend && dotnet test --solution MoShiro.slnx -c Release
# 增量编译 + 快的那批测试(约 20 秒);慢的端到端测试在 e2e workflow 里和出镜像并行跑。见 ci/verify-backend.sh
- bash ci/verify-backend.sh fast
+4 -7
View File
@@ -11,17 +11,14 @@ clone:
when:
- event: pull_request
- event: push
branch: dev
depends_on:
- verify
branch: main
# 和 verify 并行:前端构建不依赖后端测试,两个同时跑
steps:
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- npm run build --prefix web
- npm run lint:css --prefix web
# node_modules 按锁文件哈希缓存,展示端和管理端并行构建,再检查 CSS,见 ci/web-build.sh
- bash ci/web-build.sh
+4 -3
View File
@@ -6,8 +6,9 @@ MoShiro is a self-hosted network path quality monitor. Backend coordinates Agent
## Branch and workspace safety
- Treat `main` as the stable release branch and `dev` as the integration branch. Keep ordinary implementation work on `dev` or a feature branch; integrate into `main` through a pull request.
- A push to `main` builds images only if its commit message contains `[CI]` (for a merged PR, put it in the PR or merge commit title). Without `[CI]` nothing is built; use `[CI SKIP]` to skip the pipeline entirely. When squashing or pushing to `main`, add `[CI]` unless a build is explicitly not wanted.
- `main` is the only working branch (the old `dev` branch is retired and no pipeline uses it). Commit and push to `main` directly; use a pull request when a review is wanted.
- A push to `main` publishes development images only if its commit message contains `[CI]` (for a merged PR, put it in the PR or merge commit title). Without `[CI]` nothing is built; use `[CI SKIP]` to skip the pipeline entirely. The fast Backend tests (`verify`) and the web build run on every `main` push and must pass first; the slow end-to-end tests (`[Trait("Category", "Slow")]`) run after the images are built and pushed (inside `dev-image` for `[CI]` pushes, workflow `e2e` otherwise) and never block them; `[SKIP TESTS]` skips only the Backend tests.
- Versions are filled in by CI (`ci/version.sh`): a `vX.Y.Z` tag builds exactly `X.Y.Z`; a `[CI]` build on `main` is `vX.Y.(Z+1)-dev.<N>-<sha7>` (`N` = commits since the latest tag). The `<Version>` in `backend/Directory.Build.props` and the `version` in the `web` `package.json` files are placeholders (`0.0.0-dev`); never bump them by hand.
- Before editing, inspect `git status`. Preserve pre-existing user changes: do not reset, clean, stash, or include unrelated files in a commit.
- Do not commit credentials, API keys, registry passwords, or generated runtime data. CI credentials belong in Woodpecker secrets (`zot_secret`, `gitea_token`, `release_signing_key` for signing release manifests). CI registers the signed manifests at `/api/release/offer`, which needs no token because the panel verifies the signature. The release signing private key never goes into the repository or the panel; only the public key in `packaging/update/` does.
- Keep development OCI images in the `moshiro-dev/` repository path and release images in `moshiro/`. Do not let a development build overwrite release tags such as `latest`.
@@ -40,4 +41,4 @@ Follow the user's requested validation scope. Do not start a full tag/release bu
CI triggers and image tags can change; inspect `.woodpecker/*.yml` and `docs/ops/deployment.md` before changing or describing them. In particular, check the distinction between `[CI]` main builds, `[SKIP TESTS]`, and version-tag releases in the current branch.
A version tag `vMAJOR.MINOR.PATCH` runs the full cross-platform release pipeline. Development image builds and releases must remain isolated in their respective Zot repository paths.
A version tag `vMAJOR.MINOR.PATCH` runs the full cross-platform release pipeline; it needs no version-bump commit, because CI takes the version from the tag. Development image builds and releases must remain isolated in their respective Zot repository paths.
+2 -2
View File
@@ -58,7 +58,7 @@ MoShiro 是一个自托管的**网络线路质量观测工具**。它在若干 P
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │
│ 有公网 IP,监听 │ 先敲门再 TLS │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
测量会话:TLS 控制通道 + UDP/TCP 数据通道,由 Probe 主动拨出
测量会话:TLS 控制通道 + QUIC Datagram/TCP 数据通道,由 Probe 主动拨出
宿主机更新器(容器节点):用本机公钥验签 → 按 digest 拉镜像 → 重启,失败回滚
```
@@ -75,7 +75,7 @@ Backend 用段文件保存原始数据,用 SQLite 保存索引、指标与配
- **控制链路(Fleet)**:节点到 Backend 的 `wss` 长连接(`/api/agent/v1/rpc`),JSON-RPC 2.0 信封加二进制通道;挑战加签名认证,`hello` 协商版本和能力,心跳;**期望状态同步**(Backend 下发完整快照,节点调和并回报已应用的版本和被拒绝的条目);**可靠发件箱**(原始块至少一次投递,按 id 去重)。Fleet 与业务无关,放在 `backend/fleet/`。
- **业务方法**:`moshiro.*`(按需测试、Probe 断开、事件、主机汇总、更新通知等),状态文档有 `task_set`(Agent 的任务集)和 `probe_agents`(Probe 的 Agent 列表)。
- **测量数据面**:Probe 到 Agent 的 TLS 控制通道、UDP 和 TCP 数据通道(默认 `7443/tcp`、`7443/udp`、`7444/tcp`);会话用 `session_id` 和 MAC 认证、不绑定五元组,NAT 映射变化后自动跟随。
- **测量数据面**:Probe 到 Agent 的 TLS 控制通道、QUIC Datagram 和 TCP 数据通道(默认 `7443/tcp`、`7443/udp`、`7444/tcp`);测量包不走可靠重传,继续由会话 MAC、序号窗口和速率准入校验。
- **REST + SSE**:`/api/public`(展示,可匿名)、`/api/admin`(管理);OpenAPI 在 `backend/openapi/v1.json`,前端据此生成类型。
细节见 [`docs/design/protocol.md`](docs/design/protocol.md)、[`backend/fleet/README.md`](backend/fleet/README.md)、[`docs/api/api.md`](docs/api/api.md)、[`docs/security/enrollment.md`](docs/security/enrollment.md)。
+1 -5
View File
@@ -5,7 +5,7 @@
## 测量
- 路由跳数、路径漂移和 trace
- QUIC 握手耗时:TCP 建连和 TLS 握手已经有了(`connect` 任务)。.NET 的 QUIC 依赖系统里的 msquic 原生库,和节点「只依赖 OpenSSL、NativeAOT 单文件」冲突,要做只能自己写最小的 QUIC Initial 握手探测
- QUIC 握手耗时:TCP 建连和 TLS 握手已经有了(`connect` 任务)。.NET 的 QUIC 依赖系统里的 msquic 原生库,和节点「只依赖 OpenSSL、NativeAOT 单文件」冲突,要做只能自己写最小的 QUIC Initial 握手探测;另外 .NET 还没有 QUIC datagram([dotnet/runtime#123418](https://github.com/dotnet/runtime/issues/123418) 只是提案),`System.Net.Quic` 只有可靠的流,重传会盖住丢包,测不了 UDP 丢包率
- 网页长截图
- pcap / eBPF 采集
@@ -14,10 +14,6 @@
- 单个 Agent 的任务(目前是线路模板加按 Agent 的覆盖,没有只属于某一个 Agent 的独立任务)
- 原生 webhook 投递;告警(基于 webhook)。实现之前先定约束(只允许 HTTPS、禁止私网和回环、不跟随重定向、限制超时和响应大小、解析后再检查地址),见 [`docs/security/security.md`](docs/security/security.md) 第 9.2 节
## 部署
- 后端自动下载前端资源:用 Podman、Docker 部署时,默认拉取最新的前端资源(现在前端要自己构建,镜像里没有)。
[security.md](docs/security/security.md)
## 访客页面
- 人机验证:访客页面要通过验证才能拉数据,前端访问时静默触发。候选 tiago2/cap 和 Cloudflare Turnstile。
+1 -1
View File
@@ -13,7 +13,7 @@
<UseArtifactsOutput>true</UseArtifactsOutput>
<ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath>
<Deterministic>true</Deterministic>
<Version>0.4.0</Version>
<Version>0.0.0-dev</Version>
<NuGetAudit>false</NuGetAudit>
</PropertyGroup>
<ItemGroup>
+1 -1
View File
@@ -7,7 +7,7 @@ Agent 仅开放配置的控制和数据端口。它不提供远程终端、命
## 工作流程
```text
Probe ──TLS 控制 / UDP、TCP 测量──> Agent ──长连接与数据上报──> Backend
Probe ──TLS 控制 / QUIC Datagram、TCP 测量──> Agent ──长连接与数据上报──> Backend
▲
fleet-enroll 注册
```
+17 -15
View File
@@ -33,7 +33,7 @@ public sealed class AgentApp : IDisposable
private readonly Outbox _outbox = new(64 << 20);
private readonly RestartUpdater? _update;
private readonly BackendUplink _uplink;
private readonly UdpMux _udp;
private readonly QuicDatagramTransport _datagrams;
private readonly TcpAcceptor _tcp;
private readonly PeerListener _peers;
private readonly PeerHub _hub;
@@ -62,7 +62,7 @@ public sealed class AgentApp : IDisposable
_ipv6Supported = bind.AddressFamily == System.Net.Sockets.AddressFamily.InterNetworkV6;
_admission = new AgentAdmission(_streams);
_udp = new UdpMux(_sessions, _admission, _clock, _counters.Data, log, bind, config.Listen.UdpPorts);
_datagrams = new QuicDatagramTransport(_sessions, _admission, _clock, _counters.Data, log, bind, config.Listen.EffectiveQuicPorts, identity.Certificate);
// 预握手敲门:控制端口和数据端口共用一个关卡(限速和去重一起算);名册就是期望状态里的 peers
_trust = TrustAnchor.Load(config.Fleet.IdentityDir, log);
_knock = new KnockGate(identity.PeerIdentity, RosterKeys, time);
@@ -74,7 +74,7 @@ public sealed class AgentApp : IDisposable
Clock = _clock,
Log = log,
Session = () => null, // 每个 Probe 会话各用自己绑定的环境(BlockEnv.WithSession)
Udp = _udp,
Datagrams = _datagrams,
TcpListen = _tcp,
Admission = _admission,
Counters = _counters.Data,
@@ -83,14 +83,14 @@ public sealed class AgentApp : IDisposable
};
var chunks = new ChunkAssembler(_streams, _clock, _counters, _outbox, identity.NodeId, time, log);
var targets = new TargetRunners(chunks, identity.NodeId, _clock, time, log);
_tasks = new TaskManager(CreateGuard, _streams, env, targets, () => _udp.BoundPorts, () => _tcp.BoundPorts, _trust);
_tasks = new TaskManager(CreateGuard, _streams, env, targets, () => _datagrams.BoundPorts, () => _tcp.BoundPorts, _trust);
_hub = new PeerHub(_sessions, _clock, chunks, _tasks, _uplink, time, log);
_tests = new TestRunner(_tasks, _sessions, _streams, env, _uplink, config.Limits.MaxConcurrentTests, transport => transport == "udp" ? _udp.BoundPorts : _tcp.BoundPorts, time, log);
_tests = new TestRunner(_tasks, _sessions, _streams, env, _uplink, config.Limits.MaxConcurrentTests, transport => transport == "quic_datagram" ? _datagrams.BoundPorts : _tcp.BoundPorts, time, log);
_tasks.OnLineTasksChanged = (probe, set) => _ = _hub.PushToAsync(probe, set).AsTask();
_tasks.OnPeersChanged = _hub.CloseNotIn;
_hub.OnProbeLost = probe => _tests.ProbeLost(probe);
_admission.OnEndpointChanged = probe => _ = _uplink.EmitEventAsync(AgentEventKinds.EndpointChanged, probe.UdpEndpoint?.ToString()).AsTask();
_admission.OnEndpointChanged = probe => _ = _uplink.EmitEventAsync(AgentEventKinds.EndpointChanged, probe.DatagramRemoteEndPoint?.ToString()).AsTask();
_peers = new PeerListener(
new PeerListenerOptions
@@ -113,13 +113,13 @@ public sealed class AgentApp : IDisposable
public void Dispose()
{
_udp.Dispose();
_datagrams.Dispose();
_ownedRestart?.Dispose();
}
internal StreamRegistry Streams => _streams;
public IReadOnlyList<int> UdpPorts => _udp.BoundPorts;
public IReadOnlyList<int> QuicPorts => _datagrams.BoundPorts;
public IReadOnlyList<int> TcpPorts => _tcp.BoundPorts;
@@ -143,25 +143,26 @@ public sealed class AgentApp : IDisposable
public async Task RunAsync(CancellationToken ct)
{
_log.Info("moshiro agent starting", MoShiro.Protocol.BuildInfo.Fields);
using CancellationTokenSource linked = CancellationTokenSource.CreateLinkedTokenSource(ct);
// 数据面和控制端口先开始监听,再连接 Backend:hello 里上报的是实际端口
_udp.Bind();
_datagrams.Bind();
_tcp.Bind();
Task udp = _udp.RunAsync(linked.Token);
Task quic = _datagrams.RunAsync(linked.Token);
Task tcp = _tcp.RunAsync(linked.Token);
Task peers = _peers.RunAsync(linked.Token);
await Task.WhenAny(_peers.Started.Task, peers);
if (peers.IsFaulted)
{
await linked.CancelAsync();
await Task.WhenAll(udp, tcp, peers); // 抛出监听失败的原因(端口被占用等)
await Task.WhenAll(quic, tcp, peers); // 抛出监听失败的原因(端口被占用等)
}
_ready.TrySetResult();
_log.Info("agent listening", ("control", ControlPort), ("udp", string.Join(',', UdpPorts)), ("tcp", string.Join(',', TcpPorts)));
_log.Info("agent listening", ("control", ControlPort), ("quic", string.Join(',', QuicPorts)), ("tcp", string.Join(',', TcpPorts)));
try
{
await Task.WhenAll(udp, tcp, peers, _link.RunAsync(linked.Token), ReportHostAsync(linked.Token));
await Task.WhenAll(quic, tcp, peers, _link.RunAsync(linked.Token), ReportHostAsync(linked.Token));
}
finally
{
@@ -202,7 +203,7 @@ public sealed class AgentApp : IDisposable
MaxTotalPps = _config.Limits.MaxTotalPps,
AllowedTargetCidrs = _config.Limits.AllowedTargetCidrs,
MaxConcurrentTests = _config.Limits.MaxConcurrentTests,
AllowedUdpPorts = _udp.BoundPorts,
AllowedQuicPorts = _datagrams.BoundPorts,
AllowedTcpPorts = _tcp.BoundPorts,
},
BlockCatalog.AgentKinds);
@@ -309,7 +310,8 @@ public sealed class AgentApp : IDisposable
Platform = MoShiro.Measurement.PlatformDetector.Detect(),
TlsPublicKey = Convert.ToBase64String(_identity.Certificate.PublicKey.ExportSubjectPublicKeyInfo()),
ProbeTcpPort = ControlPort,
ProbeUdpPorts = [.. UdpPorts],
ProbeQuicPorts = [.. QuicPorts],
LegacyProbeUdpPorts = [.. QuicPorts],
LocalLimits = new LocalLimits
{
MaxTotalMbps = _config.Limits.MaxTotalMbps,
@@ -51,8 +51,15 @@ public sealed class ListenSection
[JsonPropertyName("control_port")]
public int ControlPort { get; set; } = 7443;
[JsonPropertyName("quic_ports")]
public List<int> QuicPorts { get; set; } = [7443];
/// <summary>Legacy config key accepted while deployments migrate to QUIC Datagram.</summary>
[JsonPropertyName("udp_ports")]
public List<int> UdpPorts { get; set; } = [7443];
public List<int>? LegacyUdpPorts { get; set; }
[JsonIgnore]
public IReadOnlyList<int> EffectiveQuicPorts => LegacyUdpPorts ?? QuicPorts;
[JsonPropertyName("tcp_ports")]
public List<int> TcpPorts { get; set; } = [7444];
@@ -44,7 +44,7 @@ public sealed class ProbeSession : DataSession
{
private readonly ConcurrentDictionary<uint, ReplayWindow> _windows = new();
private readonly ConcurrentDictionary<ulong, byte> _nonces = new();
private SocketAddress? _udpSource;
private SocketAddress? _datagramSource;
public ProbeSession(PeerSessionInfo info)
: base(info.Credentials.SessionIdValue, info.Credentials.Key)
@@ -71,11 +71,11 @@ public sealed class ProbeSession : DataSession
public RpcSession<PeerContext>? Rpc { get; set; }
/// <summary>最近一次认证通过的 UDP 包的源地址(Agent 把单向数据发往这里)。</summary>
public override IPEndPoint? UdpEndpoint
public override IPEndPoint? DatagramRemoteEndPoint
{
get
{
SocketAddress? source = Volatile.Read(ref _udpSource);
SocketAddress? source = Volatile.Read(ref _datagramSource);
return source is null ? null : (IPEndPoint)new IPEndPoint(source.Family == AddressFamily.InterNetworkV6 ? IPAddress.IPv6Any : IPAddress.Any, 0).Create(source);
}
}
@@ -83,7 +83,7 @@ public sealed class ProbeSession : DataSession
/// <summary>记录 UDP 源地址;NAT 映射变化后地址变了返回 true(上报 endpoint_changed)。第一次看到地址不算变化。</summary>
public bool UpdateSource(SocketAddress source)
{
SocketAddress? previous = Volatile.Read(ref _udpSource);
SocketAddress? previous = Volatile.Read(ref _datagramSource);
if (previous is not null && previous.Equals(source))
{
return false;
@@ -96,7 +96,7 @@ public sealed class ProbeSession : DataSession
copy[i] = source[i];
}
Volatile.Write(ref _udpSource, copy);
Volatile.Write(ref _datagramSource, copy);
return previous is not null;
}
@@ -312,7 +312,7 @@ public sealed class AgentAdmission : IAdmission
/// <summary>NAT 映射变化,源地址变了:上报 <c>endpoint_changed</c>。</summary>
public Action<ProbeSession>? OnEndpointChanged { get; set; }
public Admission AdmitPacket(DataSession session, in UdpHeader header, SocketAddress source, long nowUs)
public Admission AdmitDatagram(DataSession session, in MeasurementDatagramHeader header, SocketAddress source, long nowUs)
{
if (session is not ProbeSession probe || !_streams.TryGet(header.StreamId, out StreamInfo? stream) || !ReferenceEquals(stream.Owner, probe) || !probe.AcceptSequence(header.StreamId, header.Seq))
{
@@ -78,8 +78,9 @@ public sealed class PeerHub
{
private const int DedupeWindow = 1024;
/// <summary>超过这么久没有收到 Probe 的任何帧就认为会话已死(Probe 每 30 秒至少发一次时钟同步)。</summary>
private static readonly TimeSpan IdleLimit = TimeSpan.FromSeconds(75);
/// <summary>超过这么久没有收到 Probe 的任何帧就认为会话已死(Probe 每 30 秒至少发一次时钟同步)。
/// 要比线路可能的中断长得多:差的线路上几十秒甚至一两分钟的黑洞时有发生(TCP RTT 能到 30 秒),75 秒就关会话会把一次中断变成几分钟的数据空白(要重新握手,而握手在差线路上也慢)。</summary>
private static readonly TimeSpan IdleLimit = TimeSpan.FromSeconds(180);
private readonly RpcRouter<PeerContext> _router;
private readonly SessionRegistry _sessions;
@@ -24,7 +24,7 @@ public sealed class TaskManager
private readonly StreamRegistry _streams;
private readonly BlockEnv _env;
private readonly TargetRunners _targets;
private readonly Func<IReadOnlyList<int>> _udpPorts;
private readonly Func<IReadOnlyList<int>> _quicPorts;
private readonly Func<IReadOnlyList<int>> _tcpPorts;
private readonly TrustAnchor _trust;
private readonly Dictionary<ProbeSession, Dictionary<string, LineEntry>> _sessionLines = [];
@@ -33,14 +33,14 @@ public sealed class TaskManager
private long _version;
private BudgetGuard? _guard;
public TaskManager(Func<BudgetGuard> guardFactory, StreamRegistry streams, BlockEnv env, TargetRunners targets, Func<IReadOnlyList<int>> udpPorts, Func<IReadOnlyList<int>> tcpPorts, TrustAnchor? trust = null)
public TaskManager(Func<BudgetGuard> guardFactory, StreamRegistry streams, BlockEnv env, TargetRunners targets, Func<IReadOnlyList<int>> quicPorts, Func<IReadOnlyList<int>> tcpPorts, TrustAnchor? trust = null)
{
_trust = trust ?? TrustAnchor.None;
_guardFactory = guardFactory;
_streams = streams;
_env = env;
_targets = targets;
_udpPorts = udpPorts;
_quicPorts = quicPorts;
_tcpPorts = tcpPorts;
}
@@ -229,7 +229,8 @@ public sealed class TaskManager
{
Version = _version,
Tasks = [.. entries.Values.Select(e => e.Task)],
UdpPorts = [.. _udpPorts()],
QuicPorts = [.. _quicPorts()],
LegacyUdpPorts = [.. _quicPorts()],
TcpPorts = [.. _tcpPorts()],
};
@@ -107,7 +107,7 @@ public sealed class TestRunner
// 按需测试的数据端口由 Agent 决定:Backend 给的端口如果不是本机监听的端口(模板里是出厂默认值,每个 Agent 可以改,
// 而 Backend 并不知道 TCP 数据端口),就用本机监听的第一个端口。块头里记录的是实际用的端口。
IReadOnlyList<int> ports = _ports(kind.Blocks.Transport == "udp" ? "udp" : "tcp");
IReadOnlyList<int> ports = _ports(kind.Blocks.Transport == "quic_datagram" ? "quic_datagram" : "tcp");
bool listening = phase.Params.TryGetValue("port", out System.Text.Json.JsonElement requested) && requested.TryGetInt32(out int port) && ports.Contains(port);
if (!listening)
{
+1 -1
View File
@@ -23,7 +23,7 @@
2. **安全默认值**:Fleet 永远不提供 exec、shell、文件读写、远程下发代码这类方法;节点只响应本地注册过的方法(本地策略 `allow_methods` 还能收窄);调用对方不支持的方法在本地直接返回 `fleet_unsupported`。
3. **权限分离**:注册(`fleet-enroll`,一次性)和常驻业务进程是两个进程;常驻进程不含注册代码,也不使用 `HttpClient`。
4. **NativeAOT**:节点侧只依赖 BCL,泛型扩展点通过产品传入的 `JsonTypeInfo<T>` 序列化,不使用反射。
5. **更新不在 Fleet 里**:节点进程没有自更新能力。容器节点的签名更新由宿主机上的更新器完成,见 [`docs/ops/deployment.md`](../../docs/ops/deployment.md);纯二进制节点的更新是待办,见 [`TODO.md`](../../TODO.md)。
5. **更新不在 Fleet 里**:节点进程没有自更新能力。Quadlet 装的节点由退出后 systemd 重新拉镜像、Backend 和脚本装的节点由宿主机上的更新器验签更新,见 [`docs/ops/deployment.md`](../../docs/ops/deployment.md);纯二进制节点的更新是待办,见 [`TODO.md`](../../TODO.md)。
## 身份
+15 -6
View File
@@ -29,7 +29,8 @@ public sealed class PeerListenerOptions
public int MaxSessions { get; init; } = 8;
public TimeSpan HandshakeTimeout { get; init; } = TimeSpan.FromSeconds(10);
/// <summary>敲门、TLS、挑战应答整个握手的上限。线路差的时候(RTT 好几秒、还丢包)握手本来就要很久,10 秒做不完就会一直连不上;敲门已经挡住了不认识的连接,放宽不会放大被占连接的风险。</summary>
public TimeSpan HandshakeTimeout { get; init; } = PeerTimings.Handshake;
}
/// <summary>
@@ -209,7 +210,8 @@ public sealed class PeerListener
}
/// <summary>发起点对点会话(Probe 一侧):按 SPKI 指纹固定对端证书,用身份私钥签名,派生同样的数据面凭据。</summary>
/// <summary>点对点会话的套接字:开启 TCP keepalive(空闲 15 秒后每 5 秒探测,连续 3 次无响应就断开,约 30 秒),主机丢失或路径被切断时连接不会一直挂着。</summary>
/// <summary>点对点会话的套接字:开启 TCP keepalive(空闲 30 秒后每 10 秒探测,连续 6 次无响应才断开,约 90 秒),主机真的没了的时候连接不会一直挂着;
/// 但线路短暂中断(几十秒的黑洞)不能把会话杀掉——线路中断本来就是要测的东西,会话得扛过去,恢复之后接着测。</summary>
internal static class PeerSockets
{
public static void EnableKeepAlive(Socket socket)
@@ -217,9 +219,9 @@ internal static class PeerSockets
try
{
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive, true);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveTime, 15);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveInterval, 5);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveRetryCount, 3);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveTime, 30);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveInterval, 10);
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveRetryCount, 6);
}
catch (Exception ex) when (ex is SocketException or ArgumentException or PlatformNotSupportedException)
{
@@ -228,6 +230,13 @@ internal static class PeerSockets
}
}
/// <summary>点对点会话的时间常数,Probe 和 Agent 两侧共用。</summary>
public static class PeerTimings
{
/// <summary>建立会话(连接、敲门、TLS、挑战应答)的总时限。</summary>
public static readonly TimeSpan Handshake = TimeSpan.FromSeconds(30);
}
public static class PeerConnector
{
public static async Task<(PeerSessionInfo Info, TlsFrameTransport Transport)> ConnectAsync(
@@ -240,7 +249,7 @@ public static class PeerConnector
PeerSockets.EnableKeepAlive(client.Client);
try
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(10), time);
using var timeout = new CancellationTokenSource(PeerTimings.Handshake, time);
using CancellationTokenSource linked = CancellationTokenSource.CreateLinkedTokenSource(ct, timeout.Token);
await client.ConnectAsync(host, port, linked.Token);
Stream stream = client.GetStream();
+182
View File
@@ -5174,6 +5174,7 @@
"public_host",
"probe_tcp_port",
"probe_udp_ports",
"probe_quic_ports",
"labels",
"public_visible",
"kinds",
@@ -5242,6 +5243,17 @@
"format": "int32"
}
},
"probe_quic_ports": {
"type": "array",
"items": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int32"
}
},
"labels": {
"type": "array",
"items": {
@@ -6974,6 +6986,16 @@
"ipv6_available": {
"type": "boolean",
"default": false
},
"quic_health": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/QuicLineHealthDto"
}
]
}
}
},
@@ -7601,6 +7623,16 @@
"ip_version": {
"type": "string",
"default": "ipv4"
},
"quic_health": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/QuicLineHealthDto"
}
]
}
}
},
@@ -8581,6 +8613,156 @@
}
}
},
"QuicLineHealthDto": {
"required": [
"connected",
"handshake_attempts",
"handshake_successes",
"handshake_failures",
"handshake_timeouts",
"handshake_success_rate",
"handshake_failure_rate",
"handshake_timeout_rate",
"handshake_p95_ms",
"handshake_sample_count",
"transport_packet_loss_rate",
"transport_packets_sent",
"transport_congestion_events",
"transport_ecn_congestion_events",
"path_mtu",
"at"
],
"type": "object",
"properties": {
"connected": {
"type": "boolean"
},
"handshake_attempts": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int64"
},
"handshake_successes": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int64"
},
"handshake_failures": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int64"
},
"handshake_timeouts": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int64"
},
"handshake_success_rate": {
"pattern": "^-?(?:0|[1-9]\\d*)(?:\\.\\d+)?(?:[eE][+-]?\\d+)?$",
"type": [
"null",
"number",
"string"
],
"format": "double"
},
"handshake_failure_rate": {
"pattern": "^-?(?:0|[1-9]\\d*)(?:\\.\\d+)?(?:[eE][+-]?\\d+)?$",
"type": [
"null",
"number",
"string"
],
"format": "double"
},
"handshake_timeout_rate": {
"pattern": "^-?(?:0|[1-9]\\d*)(?:\\.\\d+)?(?:[eE][+-]?\\d+)?$",
"type": [
"null",
"number",
"string"
],
"format": "double"
},
"handshake_p95_ms": {
"pattern": "^-?(?:0|[1-9]\\d*)(?:\\.\\d+)?(?:[eE][+-]?\\d+)?$",
"type": [
"null",
"number",
"string"
],
"format": "double"
},
"handshake_sample_count": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int64"
},
"transport_packet_loss_rate": {
"pattern": "^-?(?:0|[1-9]\\d*)(?:\\.\\d+)?(?:[eE][+-]?\\d+)?$",
"type": [
"null",
"number",
"string"
],
"format": "double"
},
"transport_packets_sent": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"null",
"integer",
"string"
],
"format": "uint64"
},
"transport_congestion_events": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"null",
"integer",
"string"
],
"format": "uint32"
},
"transport_ecn_congestion_events": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"null",
"integer",
"string"
],
"format": "uint32"
},
"path_mtu": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"null",
"integer",
"string"
],
"format": "uint16"
},
"at": {
"format": "date-time"
}
}
},
"RawChunkMetaDto": {
"required": [
"chunk_id",
@@ -30,6 +30,14 @@ public sealed class AgentConnection
private readonly ConnectionEnv _env;
private volatile ConnectionSession? _session;
private int _previousQuicSourcePort;
private QuicDatagramTransport? _activeDatagrams;
private readonly object _handshakeGate = new();
private readonly Queue<double> _handshakeSamplesMs = new();
private long _handshakeAttempts;
private long _handshakeSuccesses;
private long _handshakeFailures;
private long _handshakeTimeouts;
public AgentConnection(KnownAgent endpoint, ConnectionEnv env)
{
@@ -45,9 +53,42 @@ public sealed class AgentConnection
public long ClockOffsetUs => _session?.Clock.OffsetUs ?? 0;
public ProbeQuicLinkHealth QuicHealth()
{
lock (_handshakeGate)
{
double? p95 = null;
if (_handshakeSamplesMs.Count > 0)
{
double[] sorted = [.. _handshakeSamplesMs.Order()];
p95 = sorted[(int)Math.Ceiling(sorted.Length * 0.95) - 1];
}
QuicTransportSnapshot? transport = Volatile.Read(ref _activeDatagrams)?.GetTransportSnapshot();
return new ProbeQuicLinkHealth
{
AgentId = Endpoint.AgentId,
IpVersion = Endpoint.IpVersion,
Connected = IsConnected,
HandshakeAttempts = _handshakeAttempts,
HandshakeSuccesses = _handshakeSuccesses,
HandshakeFailures = _handshakeFailures,
HandshakeTimeouts = _handshakeTimeouts,
HandshakeP95Ms = p95,
HandshakeSampleCount = _handshakeSamplesMs.Count,
TransportPacketLossRate = transport?.PacketLossRate,
TransportPacketsSent = transport?.PacketsSent,
TransportCongestionEvents = transport?.CongestionEvents,
TransportEcnCongestionEvents = transport?.EcnCongestionEvents,
PathMtu = transport?.PathMtu,
};
}
}
public async Task RunAsync(CancellationToken ct)
{
var backoff = new Backoff(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(20));
// 重连的间隔别拉太长:每次尝试本来就要等好几秒(握手上限 30 秒),线路一恢复要尽快接上;带抖动,不会和别的节点撞在一起
var backoff = new Backoff(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(8));
while (!ct.IsCancellationRequested)
{
try
@@ -92,10 +133,22 @@ public sealed class AgentConnection
byte[] agentKey = Convert.FromBase64String(Endpoint.PublicKey);
(PeerSessionInfo info, TlsFrameTransport transport) = await PeerConnector.ConnectAsync(
address.ToString(), Endpoint.TcpPort, agentKey, _env.Identity, _env.ProbeId, ProbeApp.Version, 8 << 20, _env.Time, ct);
var remote = (IPEndPoint)info.Remote!;
var data = new DataSession(info.Credentials.SessionIdValue, info.Credentials.Key);
using var udp = new UdpMux(new SingleSession(data), null, _env.Clock, new DataPlaneCounters(), _env.Log, remote.AddressFamily == AddressFamily.InterNetworkV6 ? IPAddress.IPv6Any : IPAddress.Any, []);
udp.Bind();
var remote = (IPEndPoint)info.Remote!;
IReadOnlyList<int> quicPorts = Endpoint.EffectiveQuicPorts;
int quicPort = quicPorts.Count == 0 ? 7443 : quicPorts[0];
using var datagrams = new QuicDatagramTransport(
data,
_env.Clock,
new DataPlaneCounters(),
_env.Log,
new IPEndPoint(remote.Address, quicPort),
agentKey,
Volatile.Read(ref _previousQuicSourcePort));
Volatile.Write(ref _activeDatagrams, datagrams);
datagrams.Bind();
using CancellationTokenSource cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
var outbox = new Outbox(8 << 20);
var clock = new ClockEstimator();
var shipper = new FragmentShipper(outbox, new NodeId(Endpoint.AgentId), new ProbeId(_env.ProbeId), clock, _env.Time);
@@ -104,7 +157,7 @@ public sealed class AgentConnection
Clock = _env.Clock,
Log = _env.Log,
Session = () => data,
Udp = udp,
Datagrams = datagrams,
PeerAddress = remote.Address,
PeerSpkiSha256 = NodeIdentity.Fingerprint(agentKey),
KnockKey = Knock.DeriveKey(_env.Identity, agentKey),
@@ -116,14 +169,52 @@ public sealed class AgentConnection
var rpc = new RpcSession<ConnectionSession>(transport, s_router, session, _env.Time);
rpc.OnBinary = (_, _) => ValueTask.CompletedTask;
using CancellationTokenSource cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
Task run = rpc.RunAsync(cts.Token);
Task receive = udp.RunAsync(cts.Token);
Task receive = datagrams.RunAsync(cts.Token);
Task pump = PumpAsync(rpc, outbox, cts.Token);
try
{
// 先同步时钟(块头里要带偏移);Agent 随后推送线路任务集
await SyncClockAsync(rpc, clock, rounds: 5, cts.Token);
lock (_handshakeGate)
{
_handshakeAttempts++;
}
try
{
await datagrams.ConnectAsync(cts.Token);
lock (_handshakeGate)
{
_handshakeSuccesses++;
if (datagrams.HandshakeDurationMs is { } elapsed)
{
_handshakeSamplesMs.Enqueue(elapsed);
while (_handshakeSamplesMs.Count > 128)
{
_handshakeSamplesMs.Dequeue();
}
}
}
}
catch (Exception ex) when (!(ex is OperationCanceledException && cts.IsCancellationRequested))
{
lock (_handshakeGate)
{
_handshakeFailures++;
if (ex is TimeoutException)
{
_handshakeTimeouts++;
}
}
throw;
}
if (datagrams.BoundPorts.Count > 0)
{
Volatile.Write(ref _previousQuicSourcePort, datagrams.BoundPorts[0]);
}
backoff.Reset();
_session = session;
_env.Log.Info("agent session up", ("agent", Endpoint.AgentId), ("offset_us", clock.OffsetUs));
@@ -136,6 +227,7 @@ public sealed class AgentConnection
finally
{
_session = null;
Volatile.Write(ref _activeDatagrams, null);
await cts.CancelAsync();
await session.StopAllAsync();
await Quiet(run);
@@ -73,14 +73,15 @@ public sealed class AgentDirectory
_trust.Update(result.TrustChain);
foreach (AgentEndpoint agent in result.Agents)
{
List<int> ports = agent.QuicPorts.Count > 0 ? agent.QuicPorts : agent.LegacyUdpPorts ?? [];
if (!string.IsNullOrWhiteSpace(agent.Host))
{
next.Add(new KnownAgent { AgentId = agent.AgentId.Value, Host = agent.Host, TcpPort = agent.TcpPort, PublicKey = agent.PublicKey });
next.Add(new KnownAgent { AgentId = agent.AgentId.Value, Host = agent.Host, TcpPort = agent.TcpPort, QuicPorts = [.. ports], PublicKey = agent.PublicKey });
}
if (!string.IsNullOrWhiteSpace(agent.Ipv6Host))
{
next.Add(new KnownAgent { AgentId = agent.AgentId.Value, Host = agent.Ipv6Host, TcpPort = agent.TcpPort, PublicKey = agent.PublicKey, IpVersion = IpVersions.V6 });
next.Add(new KnownAgent { AgentId = agent.AgentId.Value, Host = agent.Ipv6Host, TcpPort = agent.TcpPort, QuicPorts = [.. ports], PublicKey = agent.PublicKey, IpVersion = IpVersions.V6 });
}
}
lock (_gate)
@@ -111,5 +112,5 @@ public sealed class AgentDirectory
a.Count == b.Count && a.Zip(b).All(p => Equal(p.First, p.Second));
public static bool Equal(KnownAgent a, KnownAgent b) =>
a.AgentId == b.AgentId && a.IpVersion == b.IpVersion && a.Host == b.Host && a.TcpPort == b.TcpPort && a.PublicKey == b.PublicKey;
a.AgentId == b.AgentId && a.IpVersion == b.IpVersion && a.Host == b.Host && a.TcpPort == b.TcpPort && a.QuicPorts.SequenceEqual(b.QuicPorts) && a.PublicKey == b.PublicKey;
}
+10 -1
View File
@@ -45,7 +45,7 @@ public sealed class ProbeApp : IDisposable
NodeId = identity.NodeId,
NodeVersion = Version,
Capabilities = [MoShiroCapabilities.Base],
HeartbeatExtra = () => JsonSerializer.SerializeToElement(new ProbeHeartbeatExtra { Update = update?.Status(), Ipv6 = Ipv6Diagnostics.Sample(null, Environment.GetEnvironmentVariable("MOSHIRO_NETWORK_SCOPE"), Environment.GetEnvironmentVariable("HOST_PROC")) }, MoShiroJsonContext.Default.ProbeHeartbeatExtra),
HeartbeatExtra = () => JsonSerializer.SerializeToElement(new ProbeHeartbeatExtra { Update = update?.Status(), Ipv6 = Ipv6Diagnostics.Sample(null, Environment.GetEnvironmentVariable("MOSHIRO_NETWORK_SCOPE"), Environment.GetEnvironmentVariable("HOST_PROC")), QuicLinks = SnapshotQuicLinks() }, MoShiroJsonContext.Default.ProbeHeartbeatExtra),
HelloMeta = () => JsonSerializer.SerializeToElement(new ProbeHelloMeta { Owners = [.. trust.Controllers], Platform = PlatformDetector.Detect(), Ipv6 = Ipv6Diagnostics.Sample(null, Environment.GetEnvironmentVariable("MOSHIRO_NETWORK_SCOPE"), Environment.GetEnvironmentVariable("HOST_PROC")) }, MoShiroJsonContext.Default.ProbeHelloMeta),
};
var list = new StateHandler<ProbeAgentList>(
@@ -101,8 +101,17 @@ public sealed class ProbeApp : IDisposable
}
}
private List<ProbeQuicLinkHealth> SnapshotQuicLinks()
{
lock (_connections)
{
return [.. _connections.Values.Select(r => r.Connection.QuicHealth())];
}
}
public async Task RunAsync(CancellationToken ct)
{
_log.Info("moshiro probe starting", MoShiro.Protocol.BuildInfo.Fields);
if (_env.Directory.Agents.Count == 0)
{
_log.Info("no agents known yet, waiting for the panel to send the list");
@@ -57,6 +57,15 @@ public sealed class KnownAgent
[JsonPropertyName("tcp_port")]
public int TcpPort { get; set; }
[JsonPropertyName("quic_ports")]
public List<int> QuicPorts { get; set; } = [7443];
[JsonPropertyName("udp_ports")]
public List<int>? LegacyUdpPorts { get; set; }
[JsonIgnore]
public IReadOnlyList<int> EffectiveQuicPorts => LegacyUdpPorts ?? QuicPorts;
/// <summary>Agent 的 TLS 证书公钥(SPKI,base64):证书按它的 SHA-256 指纹固定,敲门密钥由它派生。</summary>
[JsonPropertyName("public_key")]
public string PublicKey { get; set; } = "";
@@ -26,7 +26,7 @@ public static class BlockCatalog
{
public static readonly IReadOnlyList<BlockPair> All =
[
new("udp_echo", (e, s) => new UdpEchoDialer(e, s), (e, s) => new UdpEchoListener(e, s)),
new("udp_echo", (e, s) => new DatagramEchoDialer(e, s), (e, s) => new DatagramEchoListener(e, s)),
new("tcp_echo", (e, s) => new TcpEchoDialer(e, s), (e, s) => new TcpEchoListener(e, s)),
new("udp_train", (e, s) => new TrainDialer(e, s), (e, s) => new TrainListener(e, s), OnDemandOnly: true, Directions: ["rev"]),
new("tcp_bulk", (e, s) => new TcpBulkDialer(e, s), (e, s) => new TcpBulkListener(e, s), OnDemandOnly: true, Directions: ["fwd", "rev"]),
@@ -11,7 +11,7 @@ namespace MoShiro.Measurement;
internal sealed class EchoRows
{
/// <summary>UDP:没有在这段时间内回来的包就是丢了。</summary>
public const long UdpGraceUs = 2_000_000;
public const long DatagramGraceUs = 2_000_000;
/// <summary>
/// TCP 不会丢包,只会延迟(重传、队头阻塞):一次几秒的中断之后,积压的回显会陆续回来,RTT 就是那几秒。
@@ -65,26 +65,27 @@ internal sealed class EchoRows
/// <summary>
/// UDP 回显,拨号方(Probe):按声明的速率发包,t1 在发出时记录,回复到达时回填 t4。回复丢失的行 t4 缺失(Agent 一侧再补 t2/t3)。
/// </summary>
public sealed class UdpEchoDialer : BlockBase
public sealed class DatagramEchoDialer : BlockBase
{
private readonly EchoRows _rows;
public UdpEchoDialer(BlockEnv env, BlockSpec spec)
public DatagramEchoDialer(BlockEnv env, BlockSpec spec)
: base(env, spec)
{
_rows = new EchoRows(spec.Kind, 16384, EchoRows.UdpGraceUs, settleOnReply: false);
_rows = new EchoRows(spec.Kind, 16384, EchoRows.DatagramGraceUs, settleOnReply: false);
}
public override long Produced => _rows.Count;
public override async Task RunAsync(CancellationToken ct)
{
UdpMux udp = Env.Udp!;
QuicDatagramTransport datagrams = Env.Datagrams!;
DataSession session = Env.Session() ?? throw new BlockFaultException("no data session");
await datagrams.WaitUntilConnectedAsync(ct);
IPEndPoint target = PeerEndpoint(Spec.Port);
int size = (int)Spec.Int("size");
byte[] packet = new byte[size];
udp.Register(Spec.StreamId, OnPacket);
datagrams.Register(Spec.StreamId, OnPacket);
try
{
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1.0 / Spec.Int("rate_hz")), Env.Time);
@@ -92,8 +93,8 @@ public sealed class UdpEchoDialer : BlockBase
{
long t1 = Env.Clock.NowUs();
uint seq = _rows.Add(size, t1);
UdpPacket.Write(packet, new UdpHeader(UdpPacketType.EchoRequest, session.SessionId, Spec.StreamId, seq, t1), session.Key);
udp.Send(target, packet);
MeasurementDatagram.Write(packet, new MeasurementDatagramHeader(MeasurementDatagramType.EchoRequest, session.SessionId, Spec.StreamId, seq, t1), session.Key);
datagrams.Send(session, target, packet);
}
}
catch (OperationCanceledException)
@@ -102,7 +103,7 @@ public sealed class UdpEchoDialer : BlockBase
}
finally
{
udp.Unregister(Spec.StreamId);
datagrams.Unregister(Spec.StreamId);
}
}
@@ -118,9 +119,9 @@ public sealed class UdpEchoDialer : BlockBase
return 1;
}
private void OnPacket(in UdpDelivery delivery)
private void OnPacket(in QuicDatagramDelivery delivery)
{
if (delivery.Header.Type == UdpPacketType.EchoReply)
if (delivery.Header.Type == MeasurementDatagramType.EchoReply)
{
_rows.Reply(delivery.Header.Seq, delivery.ReceiveUs);
}
@@ -128,11 +129,11 @@ public sealed class UdpEchoDialer : BlockBase
}
/// <summary>UDP 回显,监听方(Agent):校验后立即回复一个等长的包,记录 t2(到达)和 t3(回复发出),按序号留给合并。</summary>
public sealed class UdpEchoListener : BlockBase
public sealed class DatagramEchoListener : BlockBase
{
private readonly RowStore _records;
public UdpEchoListener(BlockEnv env, BlockSpec spec)
public DatagramEchoListener(BlockEnv env, BlockSpec spec)
: base(env, spec)
{
_records = new RowStore(spec.Kind.FindStream("msg")!, Side.Agent, 32768);
@@ -142,26 +143,26 @@ public sealed class UdpEchoListener : BlockBase
public override async Task RunAsync(CancellationToken ct)
{
Env.Udp!.Register(Spec.StreamId, OnPacket);
Env.Datagrams!.Register(Spec.StreamId, OnPacket);
try
{
await WaitUntilCancelledAsync(ct);
}
finally
{
Env.Udp.Unregister(Spec.StreamId);
Env.Datagrams.Unregister(Spec.StreamId);
}
}
private void OnPacket(in UdpDelivery delivery)
private void OnPacket(in QuicDatagramDelivery delivery)
{
if (delivery.Header.Type != UdpPacketType.EchoRequest)
if (delivery.Header.Type != MeasurementDatagramType.EchoRequest)
{
return; // Data:探活包,只用来保持 NAT 映射和让准入策略学到源地址
}
long t2 = delivery.ReceiveUs;
delivery.ReplyAs(UdpPacketType.EchoReply);
delivery.ReplyAs(MeasurementDatagramType.EchoReply);
Span<long> row = [t2, Env.Clock.NowUs()];
_records.SetRow(delivery.Header.Seq, row);
}
@@ -30,11 +30,11 @@ public sealed class TrainListener : BlockBase
public override async Task RunAsync(CancellationToken ct)
{
// Probe 开始前发来的探活包(Data)要被接受,准入策略才能据此学到它当前的源地址(NAT 映射);它们本身没有内容
Env.Udp!.Register(Spec.StreamId, static (in UdpDelivery _) => { });
Env.Datagrams!.Register(Spec.StreamId, static (in QuicDatagramDelivery _) => { });
try
{
DataSession session = await WaitForPeerAsync(ct);
IPEndPoint target = session.UdpEndpoint!;
IPEndPoint target = session.DatagramRemoteEndPoint!;
TimeSpan duration = TimeSpan.FromSeconds(Spec.Int("duration_s"));
var done = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var thread = new Thread(() =>
@@ -58,7 +58,7 @@ public sealed class TrainListener : BlockBase
}
finally
{
Env.Udp.Unregister(Spec.StreamId);
Env.Datagrams.Unregister(Spec.StreamId);
}
}
@@ -68,7 +68,7 @@ public sealed class TrainListener : BlockBase
while (true)
{
DataSession? session = Env.Session();
if (session?.UdpEndpoint is not null)
if (session?.DatagramRemoteEndPoint is not null)
{
return session;
}
@@ -120,8 +120,8 @@ public sealed class TrainListener : BlockBase
for (; seq < due; seq++)
{
long sendUs = Env.Clock.NowUs();
UdpPacket.Write(packet, new UdpHeader(UdpPacketType.Data, session.SessionId, Spec.StreamId, seq, sendUs), session.Key);
Env.Udp!.Send(target, packet);
MeasurementDatagram.Write(packet, new MeasurementDatagramHeader(MeasurementDatagramType.Data, session.SessionId, Spec.StreamId, seq, sendUs), session.Key);
Env.Datagrams!.Send(session, target, packet);
Span<long> row = [size, sendUs];
_records.SetRow(seq, row);
Interlocked.Exchange(ref _sent, seq + 1L);
@@ -159,18 +159,19 @@ public sealed class TrainDialer : BlockBase
public override async Task RunAsync(CancellationToken ct)
{
UdpMux udp = Env.Udp!;
QuicDatagramTransport datagrams = Env.Datagrams!;
DataSession session = Env.Session() ?? throw new BlockFaultException("no data session");
await datagrams.WaitUntilConnectedAsync(ct);
_startUs = Env.Clock.NowUs();
udp.Register(Spec.StreamId, OnPacket);
datagrams.Register(Spec.StreamId, OnPacket);
try
{
byte[] hello = new byte[UdpPacket.MinBytes + 20];
byte[] hello = new byte[MeasurementDatagram.MinBytes + 20];
IPEndPoint target = PeerEndpoint(Spec.Port);
for (uint seq = 0; seq < 3; seq++)
{
UdpPacket.Write(hello, new UdpHeader(UdpPacketType.Data, session.SessionId, Spec.StreamId, seq, Env.Clock.NowUs()), session.Key);
udp.Send(target, hello);
MeasurementDatagram.Write(hello, new MeasurementDatagramHeader(MeasurementDatagramType.Data, session.SessionId, Spec.StreamId, seq, Env.Clock.NowUs()), session.Key);
datagrams.Send(session, target, hello);
}
await WaitUntilCancelledAsync(ct);
@@ -178,7 +179,7 @@ public sealed class TrainDialer : BlockBase
finally
{
_endUs = Env.Clock.NowUs();
udp.Unregister(Spec.StreamId);
datagrams.Unregister(Spec.StreamId);
}
}
@@ -228,10 +229,10 @@ public sealed class TrainDialer : BlockBase
private static System.Text.Json.JsonElement Number(long value) =>
System.Text.Json.JsonDocument.Parse(value.ToString(System.Globalization.CultureInfo.InvariantCulture)).RootElement.Clone();
private void OnPacket(in UdpDelivery delivery)
private void OnPacket(in QuicDatagramDelivery delivery)
{
long seq = delivery.Header.Seq;
if (delivery.Header.Type == UdpPacketType.Data && seq < _maxSeq && !_received.IsSet(seq, _tRecv))
if (delivery.Header.Type == MeasurementDatagramType.Data && seq < _maxSeq && !_received.IsSet(seq, _tRecv))
{
_received.Set(seq, _tRecv, delivery.ReceiveUs); // 只记第一次到达(重复的包不覆盖)
}
@@ -11,6 +11,8 @@ namespace MoShiro.Measurement;
/// </summary>
public class DataSession
{
private IDatagramConnection? _datagramConnection;
public DataSession(long sessionId, byte[] key)
{
SessionId = sessionId;
@@ -21,13 +23,26 @@ public class DataSession
public byte[] Key { get; }
public IDatagramConnection? DatagramConnection => Volatile.Read(ref _datagramConnection);
public void BindDatagramConnection(IDatagramConnection connection) => Volatile.Write(ref _datagramConnection, connection);
/// <summary>对端最近一次通过认证的 UDP 源地址(监听方把单向数据发往这里;拨号方不需要)。</summary>
public virtual IPEndPoint? UdpEndpoint => null;
public virtual IPEndPoint? DatagramRemoteEndPoint => null;
/// <summary>TCP 首帧的随机数只能用一次。拨号方不接受连接,所以默认放行。</summary>
public virtual bool UseNonce(ReadOnlySpan<byte> nonce) => true;
}
/// <summary>Unreliable QUIC datagram sender bound to one authenticated measurement session.</summary>
public interface IDatagramConnection
{
bool Send(ReadOnlySpan<byte> datagram);
}
/// <summary>Transport-layer QUIC counters. These are separate from application Datagram delivery/loss.</summary>
public sealed record QuicTransportSnapshot(double? PacketLossRate, ulong PacketsSent, uint CongestionEvents, uint EcnCongestionEvents, ushort PathMtu);
public interface IDataSessions
{
DataSession? Find(long sessionId);
@@ -60,7 +75,7 @@ public enum Admission
public interface IAdmission
{
/// <summary>UDP 包:<paramref name="source"/> 是认证通过的源地址(实现可以据此学习 NAT 重绑定)。</summary>
Admission AdmitPacket(DataSession session, in UdpHeader header, SocketAddress source, long nowUs);
Admission AdmitDatagram(DataSession session, in MeasurementDatagramHeader header, SocketAddress source, long nowUs);
/// <summary>TCP 回显的一条消息(只限速)。</summary>
bool AdmitMessage(DataSession session, uint streamId, long nowUs);
@@ -201,7 +216,7 @@ public sealed class BlockEnv
/// <summary>当前的数据面会话(拨号方固定;监听方是 Agent 当前的 Probe 会话,可能没有)。</summary>
public required Func<DataSession?> Session { get; init; }
public UdpMux? Udp { get; init; }
public QuicDatagramTransport? Datagrams { get; init; }
/// <summary>监听方的准入策略(限速等);拨号方为 null。</summary>
public IAdmission? Admission { get; init; }
@@ -238,7 +253,7 @@ public sealed class BlockEnv
Clock = Clock,
Log = Log,
Session = () => session,
Udp = Udp,
Datagrams = Datagrams,
Admission = Admission,
Counters = Counters,
TcpListen = TcpListen,
@@ -17,7 +17,7 @@ public sealed class NodeLimits
public int MaxConcurrentTests { get; init; } = 1;
/// <summary>任务参数里的 <c>port</c> 只能取这些端口(本机配置的 listen 列表)。</summary>
public IReadOnlyList<int> AllowedUdpPorts { get; init; } = [7443];
public IReadOnlyList<int> AllowedQuicPorts { get; init; } = [7443];
public IReadOnlyList<int> AllowedTcpPorts { get; init; } = [7444];
}
@@ -143,7 +143,7 @@ public sealed class BudgetGuard
}
int port = (int)values.GetInt("port");
return kind.Blocks.Transport == "udp" ? _limits.AllowedUdpPorts.Contains(port) : _limits.AllowedTcpPorts.Contains(port);
return kind.Blocks.Transport == "quic_datagram" ? _limits.AllowedQuicPorts.Contains(port) : _limits.AllowedTcpPorts.Contains(port);
}
private static long EstimatePps(string kindId, ParamValues values) => kindId switch
@@ -0,0 +1,110 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using MoShiro.Protocol;
namespace MoShiro.Measurement;
/// <summary>An authenticated, per-stream QUIC Datagram delivered to a measurement block.</summary>
public readonly ref struct QuicDatagramDelivery
{
private readonly Span<byte> _payload;
private readonly MsQuicConnection _connection;
private readonly DataPlaneCounters _counters;
internal QuicDatagramDelivery(MeasurementDatagramHeader header, Span<byte> payload, long receiveUs, DataSession session, MsQuicConnection connection, DataPlaneCounters counters)
{
Header = header;
_payload = payload;
ReceiveUs = receiveUs;
Session = session;
_connection = connection;
_counters = counters;
}
public MeasurementDatagramHeader Header { get; }
public ReadOnlySpan<byte> Payload => _payload;
public long ReceiveUs { get; }
public DataSession Session { get; }
public void ReplyAs(MeasurementDatagramType type)
{
_payload[3] = (byte)type;
MeasurementDatagram.Seal(_payload, Session.Key);
if (_connection.Send(_payload))
{
_counters.PacketOut();
}
}
}
public delegate void QuicDatagramHandler(in QuicDatagramDelivery delivery);
/// <summary>Authenticates datagrams, applies Agent admission policy, and dispatches by stream.</summary>
internal sealed class MeasurementDatagramRouter
{
private readonly IDataSessions _sessions;
private readonly IAdmission? _admission;
private readonly MonoClock _clock;
private readonly ConcurrentDictionary<uint, QuicDatagramHandler> _handlers = new();
public MeasurementDatagramRouter(IDataSessions sessions, IAdmission? admission, MonoClock clock, DataPlaneCounters counters)
{
_sessions = sessions;
_admission = admission;
_clock = clock;
Counters = counters;
}
public DataPlaneCounters Counters { get; }
public void Register(uint streamId, QuicDatagramHandler handler) => _handlers[streamId] = handler;
public void Unregister(uint streamId) => _handlers.TryRemove(streamId, out _);
public void Dispatch(MsQuicConnection connection, ReadOnlySpan<byte> datagram)
{
Counters.PacketIn();
Span<byte> payload = stackalloc byte[MeasurementDatagram.MaxBytes];
if (datagram.Length > payload.Length)
{
Counters.DropInvalid();
return;
}
datagram.CopyTo(payload);
payload = payload[..datagram.Length];
if (!MeasurementDatagram.TryParseHeader(payload, out MeasurementDatagramHeader header)
|| !_handlers.TryGetValue(header.StreamId, out QuicDatagramHandler? handler)
|| _sessions.Find(header.SessionId) is not { } session
|| !MeasurementDatagram.VerifyMac(payload, session.Key))
{
Counters.DropInvalid();
return;
}
IPEndPoint? remote = connection.RefreshRemoteEndPoint();
if (remote is null)
{
Counters.DropInvalid();
return;
}
long receiveUs = _clock.NowUs();
if (_admission is not null)
{
switch (_admission.AdmitDatagram(session, header, remote.Serialize(), receiveUs))
{
case Admission.Invalid:
Counters.DropInvalid();
return;
case Admission.RateLimited:
Counters.DropRateLimited();
return;
}
}
session.BindDatagramConnection(connection);
handler(new QuicDatagramDelivery(header, payload, receiveUs, session, connection, Counters));
}
}
@@ -0,0 +1,341 @@
using System.Runtime.InteropServices;
namespace MoShiro.Measurement;
/// <summary>Small, versioned view of the MsQuic v2 C API table used by this transport.</summary>
internal static unsafe class MsQuicApi
{
private const int SetContextIndex = 0;
private const int SetCallbackHandlerIndex = 2;
private const int SetParamIndex = 3;
private const int GetParamIndex = 4;
private const int RegistrationOpenIndex = 5;
private const int RegistrationCloseIndex = 6;
private const int ConfigurationOpenIndex = 8;
private const int ConfigurationCloseIndex = 9;
private const int ConfigurationLoadCredentialIndex = 10;
private const int ListenerOpenIndex = 11;
private const int ListenerCloseIndex = 12;
private const int ListenerStartIndex = 13;
private const int ListenerStopIndex = 14;
private const int ConnectionOpenIndex = 15;
private const int ConnectionCloseIndex = 16;
private const int ConnectionShutdownIndex = 17;
private const int ConnectionStartIndex = 18;
private const int ConnectionSetConfigurationIndex = 19;
private const int DatagramSendIndex = 28;
private const uint ConnectionStatisticsV2Parameter = 0x05000016;
private static readonly SetContextDelegate s_setContext = MsQuicRuntime.Function<SetContextDelegate>(SetContextIndex);
private static readonly SetCallbackHandlerDelegate s_setCallbackHandler = MsQuicRuntime.Function<SetCallbackHandlerDelegate>(SetCallbackHandlerIndex);
private static readonly SetParamDelegate s_setParam = MsQuicRuntime.Function<SetParamDelegate>(SetParamIndex);
private static readonly GetParamDelegate s_getParam = MsQuicRuntime.Function<GetParamDelegate>(GetParamIndex);
private static readonly RegistrationOpenDelegate s_registrationOpen = MsQuicRuntime.Function<RegistrationOpenDelegate>(RegistrationOpenIndex);
private static readonly HandleCloseDelegate s_registrationClose = MsQuicRuntime.Function<HandleCloseDelegate>(RegistrationCloseIndex);
private static readonly ConfigurationOpenDelegate s_configurationOpen = MsQuicRuntime.Function<ConfigurationOpenDelegate>(ConfigurationOpenIndex);
private static readonly HandleCloseDelegate s_configurationClose = MsQuicRuntime.Function<HandleCloseDelegate>(ConfigurationCloseIndex);
private static readonly ConfigurationLoadCredentialDelegate s_loadCredential = MsQuicRuntime.Function<ConfigurationLoadCredentialDelegate>(ConfigurationLoadCredentialIndex);
private static readonly ListenerOpenDelegate s_listenerOpen = MsQuicRuntime.Function<ListenerOpenDelegate>(ListenerOpenIndex);
private static readonly HandleCloseDelegate s_listenerClose = MsQuicRuntime.Function<HandleCloseDelegate>(ListenerCloseIndex);
private static readonly ListenerStartDelegate s_listenerStart = MsQuicRuntime.Function<ListenerStartDelegate>(ListenerStartIndex);
private static readonly ListenerStopDelegate s_listenerStop = MsQuicRuntime.Function<ListenerStopDelegate>(ListenerStopIndex);
private static readonly ConnectionOpenDelegate s_connectionOpen = MsQuicRuntime.Function<ConnectionOpenDelegate>(ConnectionOpenIndex);
private static readonly HandleCloseDelegate s_connectionClose = MsQuicRuntime.Function<HandleCloseDelegate>(ConnectionCloseIndex);
private static readonly ConnectionShutdownDelegate s_connectionShutdown = MsQuicRuntime.Function<ConnectionShutdownDelegate>(ConnectionShutdownIndex);
private static readonly ConnectionStartDelegate s_connectionStart = MsQuicRuntime.Function<ConnectionStartDelegate>(ConnectionStartIndex);
private static readonly ConnectionSetConfigurationDelegate s_connectionSetConfiguration = MsQuicRuntime.Function<ConnectionSetConfigurationDelegate>(ConnectionSetConfigurationIndex);
private static readonly DatagramSendDelegate s_datagramSend = MsQuicRuntime.Function<DatagramSendDelegate>(DatagramSendIndex);
public static void SetContext(nint handle, nint context) => s_setContext(handle, context);
public static void SetCallbackHandler<T>(nint handle, T callback, nint context)
where T : Delegate => s_setCallbackHandler(handle, Marshal.GetFunctionPointerForDelegate(callback), context);
public static void SetParam(nint handle, uint parameter, byte value)
{
unsafe
{
uint status;
status = s_setParam(handle, parameter, sizeof(byte), &value);
Check(status, "SetParam");
}
}
public static byte[] GetAddress(nint handle, uint parameter)
{
byte[] address = new byte[128];
unsafe
{
fixed (byte* buffer = address)
{
uint length = (uint)address.Length;
uint status = s_getParam(handle, parameter, &length, buffer);
Check(status, "GetParam(address)");
Array.Resize(ref address, checked((int)length));
}
}
return address;
}
public static QuicStatisticsV2 GetStatisticsV2(nint connection)
{
QuicStatisticsV2 statistics = default;
unsafe
{
uint length = (uint)sizeof(QuicStatisticsV2);
uint status = s_getParam(connection, ConnectionStatisticsV2Parameter, &length, &statistics);
Check(status, "GetParam(connection statistics v2)");
}
return statistics;
}
public static nint RegistrationOpen(string appName)
{
nint utf8 = Marshal.StringToCoTaskMemUTF8(appName);
try
{
var config = new RegistrationConfig { AppName = utf8, ExecutionProfile = 0 };
uint status = s_registrationOpen(ref config, out nint registration);
Check(status, "RegistrationOpen");
return registration;
}
finally
{
Marshal.FreeCoTaskMem(utf8);
}
}
public static void RegistrationClose(nint handle) => s_registrationClose(handle);
public static nint ConfigurationOpen(nint registration, byte[] alpn)
{
unsafe
{
fixed (byte* alpnBytes = alpn)
{
var buffer = new QuicBuffer { Length = (uint)alpn.Length, Buffer = (nint)alpnBytes };
uint status = s_configurationOpen(registration, ref buffer, 1, 0, 0, 0, out nint configuration);
Check(status, "ConfigurationOpen");
return configuration;
}
}
}
public static void ConfigurationClose(nint handle) => s_configurationClose(handle);
public static void ConfigurationLoadCredential(nint configuration, ref CredentialConfig credential)
{
uint status = s_loadCredential(configuration, ref credential);
Check(status, "ConfigurationLoadCredential");
}
public static nint ListenerOpen<T>(nint registration, T callback, nint context)
where T : Delegate
{
uint status = s_listenerOpen(registration, Marshal.GetFunctionPointerForDelegate(callback), context, out nint listener);
Check(status, "ListenerOpen");
return listener;
}
public static void ListenerClose(nint handle) => s_listenerClose(handle);
public static void ListenerStart(nint listener, byte[] alpn, byte[] address)
{
unsafe
{
fixed (byte* alpnBytes = alpn)
fixed (byte* addressBytes = address)
{
var buffer = new QuicBuffer { Length = (uint)alpn.Length, Buffer = (nint)alpnBytes };
uint status = s_listenerStart(listener, ref buffer, 1, (nint)addressBytes);
Check(status, "ListenerStart");
}
}
}
public static void ListenerStop(nint handle) => s_listenerStop(handle);
public static nint ConnectionOpen<T>(nint registration, T callback, nint context)
where T : Delegate
{
uint status = s_connectionOpen(registration, Marshal.GetFunctionPointerForDelegate(callback), context, out nint connection);
Check(status, "ConnectionOpen");
return connection;
}
public static void ConnectionClose(nint handle) => s_connectionClose(handle);
public static void ConnectionShutdown(nint handle, ulong errorCode = 0) => s_connectionShutdown(handle, 0x00000001, errorCode);
public static void ConnectionStart(nint handle, nint configuration, ushort addressFamily, nint serverName, ushort port)
{
uint status = s_connectionStart(handle, configuration, addressFamily, serverName, port);
Check(status, "ConnectionStart");
}
public static void ConnectionBind(nint handle, byte[] address)
{
unsafe
{
fixed (byte* addressBytes = address)
{
uint status = s_setParam(handle, 0x05000001, (uint)address.Length, addressBytes);
Check(status, "SetParam(local address)");
}
}
}
public static void ConnectionSetConfiguration(nint handle, nint configuration)
{
uint status = s_connectionSetConfiguration(handle, configuration);
Check(status, "ConnectionSetConfiguration");
}
public static uint DatagramSend(nint connection, nint buffers, nint clientContext) => s_datagramSend(connection, buffers, 1, 0, clientContext);
public static void Check(uint status, string operation)
{
if (!Succeeded(status))
{
throw new InvalidOperationException($"MsQuic {operation} failed with status 0x{status:X8}.");
}
}
// QUIC_STATUS_PENDING (0xFFFFFFFE) is a successful asynchronous result. Other
// negative HRESULT/NTSTATUS values are failures on both Windows and POSIX.
public static bool Succeeded(uint status) => status == 0 || status == 0xFFFFFFFE;
[StructLayout(LayoutKind.Sequential)]
internal struct QuicBuffer
{
public uint Length;
public nint Buffer;
}
// MsQuic QUIC_STATISTICS_V2 through v2.5. All fields are fixed-width on
// the supported 64-bit RIDs; keeping the field order aligned with msquic.h.
[StructLayout(LayoutKind.Sequential)]
internal struct QuicStatisticsV2
{
public ulong CorrelationId;
public uint Flags;
public uint RttUs;
public uint MinRttUs;
public uint MaxRttUs;
public ulong TimingStart;
public ulong TimingInitialFlightEnd;
public ulong TimingHandshakeFlightEnd;
public uint HandshakeClientFlight1Bytes;
public uint HandshakeServerFlight1Bytes;
public uint HandshakeClientFlight2Bytes;
public ushort SendPathMtu;
public ulong SendTotalPackets;
public ulong SendRetransmittablePackets;
public ulong SendSuspectedLostPackets;
public ulong SendSpuriousLostPackets;
public ulong SendTotalBytes;
public ulong SendTotalStreamBytes;
public uint SendCongestionCount;
public uint SendPersistentCongestionCount;
public ulong RecvTotalPackets;
public ulong RecvReorderedPackets;
public ulong RecvDroppedPackets;
public ulong RecvDuplicatePackets;
public ulong RecvTotalBytes;
public ulong RecvTotalStreamBytes;
public ulong RecvDecryptionFailures;
public ulong RecvValidAckFrames;
public uint KeyUpdateCount;
public uint SendCongestionWindow;
public uint DestCidUpdateCount;
public uint SendEcnCongestionCount;
public byte HandshakeHopLimitTtl;
public uint RttVarianceUs;
}
[StructLayout(LayoutKind.Sequential)]
internal struct RegistrationConfig
{
public nint AppName;
public int ExecutionProfile;
}
[StructLayout(LayoutKind.Sequential)]
internal struct CredentialConfig
{
public uint Type;
public uint Flags;
public nint Certificate;
public nint Principal;
public nint Reserved;
public nint AsyncHandler;
public uint AllowedCipherSuites;
public nint CaCertificateFile;
}
[StructLayout(LayoutKind.Sequential)]
internal struct Pkcs12Certificate
{
public nint Asn1Blob;
public uint Asn1BlobLength;
public nint PrivateKeyPassword;
}
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
internal delegate uint ListenerCallback(nint listener, nint context, nint @event);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
internal delegate uint ConnectionCallback(nint connection, nint context, nint @event);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void SetContextDelegate(nint handle, nint context);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void SetCallbackHandlerDelegate(nint handle, nint handler, nint context);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint SetParamDelegate(nint handle, uint parameter, uint bufferLength, void* buffer);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint GetParamDelegate(nint handle, uint parameter, uint* bufferLength, void* buffer);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint RegistrationOpenDelegate(ref RegistrationConfig config, out nint registration);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void HandleCloseDelegate(nint handle);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ConfigurationOpenDelegate(nint registration, ref QuicBuffer alpn, uint alpnCount, nint settings, uint settingsLength, nint context, out nint configuration);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ConfigurationLoadCredentialDelegate(nint configuration, ref CredentialConfig credential);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ListenerOpenDelegate(nint registration, nint callback, nint context, out nint listener);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ListenerStartDelegate(nint listener, ref QuicBuffer alpn, uint alpnCount, nint address);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void ListenerStopDelegate(nint listener);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ConnectionOpenDelegate(nint registration, nint callback, nint context, out nint connection);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void ConnectionShutdownDelegate(nint connection, uint flags, ulong errorCode);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ConnectionStartDelegate(nint connection, nint configuration, ushort family, nint serverName, ushort serverPort);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint ConnectionSetConfigurationDelegate(nint connection, nint configuration);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate uint DatagramSendDelegate(nint connection, nint buffers, uint bufferCount, uint flags, nint clientContext);
}
@@ -0,0 +1,610 @@
using System.Buffers;
using System.Net;
using System.Net.Sockets;
using System.Runtime.InteropServices;
using System.Diagnostics;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using Fleet.Core;
namespace MoShiro.Measurement;
internal delegate void MsQuicDatagramCallback(MsQuicConnection connection, ReadOnlySpan<byte> datagram);
/// <summary>A single MsQuic connection with connection-bound datagram delivery.</summary>
internal sealed class MsQuicConnection : IAsyncDisposable, IDatagramConnection
{
private const uint DatagramReceiveEnabled = 0x0500000D;
private const uint ClientCredentialFlags = 0x00000001 | 0x00000004 | 0x00000010 | 0x00004000;
private const uint CertificateCredentialNone = 0;
private const uint CertificateCredentialContext = 3;
private const uint CertificateCredentialPkcs12 = 6;
private const uint ConnectionEventConnected = 0;
private const uint ConnectionEventShutdownByTransport = 1;
private const uint ConnectionEventShutdownByPeer = 2;
private const uint ConnectionEventShutdownComplete = 3;
private const uint ConnectionEventPeerAddressChanged = 5;
private const uint ConnectionEventDatagramStateChanged = 10;
private const uint ConnectionEventDatagramReceived = 11;
private const uint ConnectionEventDatagramSendStateChanged = 12;
private const uint ConnectionEventPeerCertificateReceived = 15;
private const int DatagramStateFinal = 3;
private const int QuicAddressBytes = 28;
private static readonly MsQuicApi.ConnectionCallback s_callback = OnEvent;
private readonly object _sendGate = new();
private readonly TaskCompletionSource _connected = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly TaskCompletionSource _shutdown = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly GCHandle _self;
private readonly byte[]? _expectedPeerSpki;
private readonly Action<MsQuicConnection>? _closed;
private readonly Action<IPEndPoint>? _peerAddressChanged;
private readonly INodeLog _log;
private readonly Dictionary<nint, SendState> _sends = [];
private nint _registration;
private nint _configuration;
private nint _handle;
private int _maxDatagramBytes;
private int _sendEnabled;
private int _disposeStarted;
private double? _handshakeDurationMs;
private MsQuicConnection(
nint handle,
nint registration,
nint configuration,
byte[]? expectedPeerSpki,
MsQuicDatagramCallback? datagramHandler,
INodeLog log,
Action<MsQuicConnection>? closed,
Action<IPEndPoint>? peerAddressChanged)
{
_handle = handle;
_registration = registration;
_configuration = configuration;
_expectedPeerSpki = expectedPeerSpki;
DatagramHandler = datagramHandler;
_log = log;
_closed = closed;
_peerAddressChanged = peerAddressChanged;
_self = GCHandle.Alloc(this);
}
public MsQuicDatagramCallback? DatagramHandler { get; set; }
public IPEndPoint? RemoteEndPoint { get; private set; }
public IPEndPoint? LocalEndPoint => Volatile.Read(ref _handle) is nint handle && handle != 0
? ReadEndpoint(MsQuicApi.GetAddress(handle, 0x05000001))
: null;
public nint Handle => Volatile.Read(ref _handle);
public IPEndPoint? RefreshRemoteEndPoint()
{
if (RemoteEndPoint is not null || Handle == 0)
{
return RemoteEndPoint;
}
try
{
RemoteEndPoint = ReadEndpoint(MsQuicApi.GetAddress(Handle, 0x05000002));
}
catch (InvalidOperationException)
{
// Address isn't available until the handshake finishes.
}
return RemoteEndPoint;
}
public bool IsReady => _connected.Task.IsCompletedSuccessfully && Volatile.Read(ref _sendEnabled) != 0;
public int MaxDatagramBytes => Volatile.Read(ref _maxDatagramBytes);
public double? HandshakeDurationMs => _handshakeDurationMs;
public bool TryGetStatistics(out MsQuicApi.QuicStatisticsV2 statistics)
{
nint handle = Volatile.Read(ref _handle);
if (handle == 0)
{
statistics = default;
return false;
}
try
{
statistics = MsQuicApi.GetStatisticsV2(handle);
return true;
}
catch (InvalidOperationException)
{
statistics = default;
return false;
}
}
public static async Task<MsQuicConnection> ConnectAsync(
IPAddress address,
int port,
int localPort,
byte[] expectedPeerSpki,
MsQuicDatagramCallback datagramHandler,
INodeLog log,
CancellationToken ct)
{
nint registration = 0;
nint configuration = 0;
MsQuicConnection? connection = null;
long handshakeStarted = 0;
try
{
log.Info("opening MsQuic client registration");
registration = MsQuicApi.RegistrationOpen("moshiro-probe");
log.Info("opening MsQuic client configuration");
configuration = MsQuicApi.ConfigurationOpen(registration, QuicAlpn.Value);
log.Info("loading MsQuic client credentials");
LoadClientCredentials(configuration);
var endpoint = new IPEndPoint(address.AddressFamily == AddressFamily.InterNetworkV6 ? IPAddress.IPv6Any : IPAddress.Any, localPort);
SocketAddress serialized = endpoint.Serialize();
byte[] localAddress = new byte[QuicAddressBytes];
for (int i = 0; i < serialized.Size; i++)
{
localAddress[i] = serialized[i];
}
ushort nativeFamily = ReadAddressFamily(localAddress);
nint name = Marshal.StringToCoTaskMemUTF8(address.ToString());
try
{
nint handle = MsQuicApi.ConnectionOpen(registration, s_callback, 0);
log.Info("opened MsQuic client connection");
connection = new MsQuicConnection(handle, registration, configuration, expectedPeerSpki, datagramHandler, log, null, null);
registration = 0;
configuration = 0;
MsQuicApi.SetContext(handle, GCHandle.ToIntPtr(connection._self));
log.Info("enabling MsQuic datagram receive");
MsQuicApi.SetParam(handle, DatagramReceiveEnabled, 1);
log.Info("binding MsQuic local address", ("local_port", localPort));
MsQuicApi.ConnectionBind(handle, localAddress);
log.Info("starting MsQuic QUIC handshake", ("remote", address), ("port", port), ("local_port", localPort));
handshakeStarted = Stopwatch.GetTimestamp();
MsQuicApi.ConnectionStart(handle, connection._configuration, nativeFamily, name, checked((ushort)port));
log.Info("MsQuic connection start returned");
}
finally
{
Marshal.FreeCoTaskMem(name);
}
await connection._connected.Task.WaitAsync(TimeSpan.FromSeconds(10), ct);
connection._handshakeDurationMs = Stopwatch.GetElapsedTime(handshakeStarted).TotalMilliseconds;
return connection;
}
catch
{
if (connection is not null)
{
await connection.DisposeAsync();
}
if (configuration != 0)
{
MsQuicApi.ConfigurationClose(configuration);
}
if (registration != 0)
{
MsQuicApi.RegistrationClose(registration);
}
throw;
}
}
public static MsQuicConnection Accept(
nint handle,
nint configuration,
MsQuicDatagramCallback datagramHandler,
INodeLog log,
Action<MsQuicConnection> closed,
Action<IPEndPoint>? peerAddressChanged)
{
var connection = new MsQuicConnection(handle, 0, 0, null, datagramHandler, log, closed, peerAddressChanged);
MsQuicApi.SetContext(handle, GCHandle.ToIntPtr(connection._self));
MsQuicApi.SetCallbackHandler(handle, s_callback, GCHandle.ToIntPtr(connection._self));
MsQuicApi.ConnectionSetConfiguration(handle, configuration);
MsQuicApi.SetParam(handle, DatagramReceiveEnabled, 1);
return connection;
}
public bool Send(ReadOnlySpan<byte> datagram)
{
nint handle = Volatile.Read(ref _handle);
if (handle == 0 || !IsReady || datagram.Length > MaxDatagramBytes)
{
return false;
}
byte[] buffer = ArrayPool<byte>.Shared.Rent(datagram.Length);
datagram.CopyTo(buffer);
GCHandle pinned = GCHandle.Alloc(buffer, GCHandleType.Pinned);
nint nativeBuffer = Marshal.AllocHGlobal(Marshal.SizeOf<MsQuicApi.QuicBuffer>());
var state = new SendState(buffer, pinned, nativeBuffer);
GCHandle stateHandle = GCHandle.Alloc(state);
nint context = GCHandle.ToIntPtr(stateHandle);
lock (_sendGate)
{
if (_handle == 0)
{
stateHandle.Free();
state.Release();
return false;
}
_sends.Add(context, state);
}
var nativeBufferValue = new MsQuicApi.QuicBuffer
{
Length = (uint)datagram.Length,
Buffer = pinned.AddrOfPinnedObject(),
};
Marshal.StructureToPtr(nativeBufferValue, nativeBuffer, false);
uint status = MsQuicApi.DatagramSend(handle, nativeBuffer, context);
if (MsQuicApi.Succeeded(status))
{
return true;
}
ReleaseSend(context);
return false;
}
public async ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposeStarted, 1) == 0 && Volatile.Read(ref _handle) is nint handle && handle != 0)
{
MsQuicApi.ConnectionShutdown(handle);
}
if (Volatile.Read(ref _handle) != 0)
{
await _shutdown.Task.ConfigureAwait(false);
}
}
public Task WaitForShutdownAsync() => _shutdown.Task;
private static void LoadClientCredentials(nint configuration)
{
var credential = new MsQuicApi.CredentialConfig { Type = CertificateCredentialNone, Flags = ClientCredentialFlags };
if (OperatingSystem.IsWindows())
{
credential.Flags &= ~0x00004000u;
}
MsQuicApi.ConfigurationLoadCredential(configuration, ref credential);
}
internal static void LoadServerCredentials(nint configuration, X509Certificate2 certificate)
{
var credential = new MsQuicApi.CredentialConfig();
GCHandle pfxPin = default;
nint pkcs12 = 0;
try
{
if (OperatingSystem.IsWindows())
{
credential.Type = CertificateCredentialContext;
credential.Certificate = certificate.Handle;
}
else
{
byte[] pfxBytes = certificate.Export(X509ContentType.Pkcs12);
pfxPin = GCHandle.Alloc(pfxBytes, GCHandleType.Pinned);
var pfxCertificate = new MsQuicApi.Pkcs12Certificate
{
Asn1Blob = pfxPin.AddrOfPinnedObject(),
Asn1BlobLength = (uint)pfxBytes.Length,
};
pkcs12 = Marshal.AllocHGlobal(Marshal.SizeOf<MsQuicApi.Pkcs12Certificate>());
Marshal.StructureToPtr(pfxCertificate, pkcs12, false);
credential.Type = CertificateCredentialPkcs12;
credential.Flags = 0x00004000;
credential.Certificate = pkcs12;
}
MsQuicApi.ConfigurationLoadCredential(configuration, ref credential);
}
finally
{
if (pkcs12 != 0)
{
Marshal.FreeHGlobal(pkcs12);
}
if (pfxPin.IsAllocated)
{
pfxPin.Free();
}
}
}
private static uint OnEvent(nint handle, nint context, nint eventPointer)
{
try
{
var connection = (MsQuicConnection)GCHandle.FromIntPtr(context).Target!;
uint eventType = unchecked((uint)Marshal.ReadInt32(eventPointer));
switch (eventType)
{
case ConnectionEventConnected:
connection._log.Info("MsQuic QUIC handshake completed");
connection.RefreshRemoteEndPoint();
connection._connected.TrySetResult();
break;
case ConnectionEventShutdownByTransport:
case ConnectionEventShutdownByPeer:
connection._connected.TrySetException(new IOException("QUIC connection closed during handshake."));
break;
case ConnectionEventShutdownComplete:
connection.OnShutdownComplete(handle);
break;
case ConnectionEventPeerAddressChanged:
{
nint address = Marshal.ReadIntPtr(eventPointer, IntPtr.Size);
IPEndPoint? endpoint = ReadEndpoint(address);
if (endpoint is not null)
{
connection.RemoteEndPoint = endpoint;
connection._peerAddressChanged?.Invoke(endpoint);
}
break;
}
case ConnectionEventDatagramStateChanged:
connection.OnDatagramStateChanged(eventPointer);
break;
case ConnectionEventDatagramReceived:
connection.OnDatagramReceived(eventPointer);
break;
case ConnectionEventDatagramSendStateChanged:
connection.OnDatagramSendStateChanged(eventPointer);
break;
case ConnectionEventPeerCertificateReceived:
return connection.ValidatePeerCertificate(eventPointer);
}
return 0;
}
catch (Exception ex)
{
try
{
if (context != 0 && GCHandle.FromIntPtr(context).Target is MsQuicConnection connection)
{
connection._log.Error("MsQuic connection callback failed", ("err", ex.Message));
}
}
catch
{
// Keep exceptions inside the native callback boundary.
}
return OperatingSystem.IsWindows() ? 0x80004005u : 1u;
}
}
private void OnDatagramStateChanged(nint eventPointer)
{
byte sendEnabled = Marshal.ReadByte(eventPointer, IntPtr.Size);
int maxSendLength = (ushort)Marshal.ReadInt16(eventPointer, IntPtr.Size + 1);
Volatile.Write(ref _maxDatagramBytes, maxSendLength);
Volatile.Write(ref _sendEnabled, sendEnabled == 0 ? 0 : 1);
}
private void OnDatagramReceived(nint eventPointer)
{
nint bufferPointer = Marshal.ReadIntPtr(eventPointer, IntPtr.Size);
if (bufferPointer == 0)
{
return;
}
var buffer = Marshal.PtrToStructure<MsQuicApi.QuicBuffer>(bufferPointer);
if (buffer.Buffer != 0 && buffer.Length > 0 && buffer.Length <= 65535)
{
unsafe
{
DatagramHandler?.Invoke(this, new ReadOnlySpan<byte>((void*)buffer.Buffer, checked((int)buffer.Length)));
}
}
}
private void OnDatagramSendStateChanged(nint eventPointer)
{
nint context = Marshal.ReadIntPtr(eventPointer, IntPtr.Size);
int state = Marshal.ReadInt32(eventPointer, IntPtr.Size * 2);
if (state >= DatagramStateFinal)
{
ReleaseSend(context);
}
}
private uint ValidatePeerCertificate(nint eventPointer)
{
nint certificate = Marshal.ReadIntPtr(eventPointer, IntPtr.Size);
if (certificate == 0 || _expectedPeerSpki is null)
{
return 0;
}
byte[] der;
if (OperatingSystem.IsWindows())
{
nint encoded = Marshal.ReadIntPtr(certificate, IntPtr.Size);
int length = Marshal.ReadInt32(certificate, IntPtr.Size * 2);
if (encoded == 0 || length <= 0 || length > 64 * 1024)
{
return 0x80004005u;
}
der = new byte[length];
Marshal.Copy(encoded, der, 0, length);
}
else
{
var nativeBuffer = Marshal.PtrToStructure<MsQuicApi.QuicBuffer>(certificate);
if (nativeBuffer.Buffer == 0 || nativeBuffer.Length == 0 || nativeBuffer.Length > 64 * 1024)
{
return 1;
}
der = new byte[checked((int)nativeBuffer.Length)];
Marshal.Copy(nativeBuffer.Buffer, der, 0, der.Length);
}
using X509Certificate2 peerCertificate = X509CertificateLoader.LoadCertificate(der);
bool valid = ExportSubjectPublicKeyInfo(peerCertificate).AsSpan().SequenceEqual(_expectedPeerSpki);
return valid ? 0 : OperatingSystem.IsWindows() ? 0x80004005u : 1u;
}
private static byte[] ExportSubjectPublicKeyInfo(X509Certificate2 certificate)
{
using ECDsa? ecdsa = certificate.GetECDsaPublicKey();
if (ecdsa is not null)
{
return ecdsa.ExportSubjectPublicKeyInfo();
}
using RSA? rsa = certificate.GetRSAPublicKey();
if (rsa is not null)
{
return rsa.ExportSubjectPublicKeyInfo();
}
using DSA? dsa = certificate.GetDSAPublicKey();
if (dsa is not null)
{
return dsa.ExportSubjectPublicKeyInfo();
}
return [];
}
private void ReleaseSend(nint context)
{
SendState? state;
lock (_sendGate)
{
if (!_sends.Remove(context, out state))
{
return;
}
}
if (GCHandle.FromIntPtr(context).IsAllocated)
{
GCHandle.FromIntPtr(context).Free();
}
state.Release();
}
private void OnShutdownComplete(nint handle)
{
if (Interlocked.Exchange(ref _handle, 0) == 0)
{
return;
}
MsQuicApi.ConnectionClose(handle);
lock (_sendGate)
{
foreach ((nint context, SendState state) in _sends)
{
GCHandle.FromIntPtr(context).Free();
state.Release();
}
_sends.Clear();
}
if (_configuration != 0)
{
MsQuicApi.ConfigurationClose(_configuration);
}
if (_registration != 0)
{
MsQuicApi.RegistrationClose(_registration);
}
_self.Free();
_shutdown.TrySetResult();
_closed?.Invoke(this);
}
private static IPEndPoint? ReadEndpoint(nint address)
{
if (address == 0)
{
return null;
}
ushort family = unchecked((ushort)Marshal.ReadInt16(address));
if (family == 2)
{
byte[] bytes = new byte[4];
Marshal.Copy(address + 4, bytes, 0, bytes.Length);
int port = (Marshal.ReadByte(address, 2) << 8) | Marshal.ReadByte(address, 3);
return new IPEndPoint(new IPAddress(bytes), port);
}
ushort ipv6Family = OperatingSystem.IsWindows() ? (ushort)23 : OperatingSystem.IsMacOS() ? (ushort)30 : (ushort)10;
if (family == ipv6Family)
{
byte[] bytes = new byte[16];
Marshal.Copy(address + 8, bytes, 0, bytes.Length);
int port = (Marshal.ReadByte(address, 2) << 8) | Marshal.ReadByte(address, 3);
long scope = unchecked((uint)Marshal.ReadInt32(address, 24));
return new IPEndPoint(new IPAddress(bytes, scope), port);
}
return null;
}
private static IPEndPoint? ReadEndpoint(byte[] address)
{
unsafe
{
fixed (byte* addressBytes = address)
{
return ReadEndpoint((nint)addressBytes);
}
}
}
private static ushort ReadAddressFamily(byte[] address) => (ushort)(address[0] | (address[1] << 8));
private sealed class SendState(byte[] buffer, GCHandle pin, nint nativeBuffer)
{
public void Release()
{
if (pin.IsAllocated)
{
pin.Free();
}
if (nativeBuffer != 0)
{
Marshal.FreeHGlobal(nativeBuffer);
}
ArrayPool<byte>.Shared.Return(buffer);
}
}
}
@@ -0,0 +1,195 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using System.Runtime.InteropServices;
using System.Security.Cryptography.X509Certificates;
using Fleet.Core;
namespace MoShiro.Measurement;
/// <summary>Agent-side MsQuic listener. Measurement frames arrive only as QUIC DATAGRAM frames.</summary>
internal sealed class MsQuicListener : IAsyncDisposable
{
private const int QuicAddressBytes = 28;
private const uint ListenerEventNewConnection = 0;
private const uint ListenerLocalAddress = 0x04000000;
private static readonly MsQuicApi.ListenerCallback s_callback = OnListenerEvent;
private readonly IPAddress _bindAddress;
private readonly IReadOnlyList<int> _ports;
private readonly MsQuicDatagramCallback _datagramHandler;
private readonly INodeLog _log;
private readonly Action<IPEndPoint>? _peerAddressChanged;
private readonly X509Certificate2 _certificate;
private readonly GCHandle _self;
private readonly ConcurrentDictionary<nint, MsQuicConnection> _connections = new();
private readonly List<nint> _listeners = [];
private nint _registration;
private nint _configuration;
private int _disposed;
public MsQuicListener(
IPAddress bindAddress,
IReadOnlyList<int> ports,
X509Certificate2 certificate,
MsQuicDatagramCallback datagramHandler,
INodeLog log,
Action<IPEndPoint>? peerAddressChanged)
{
_bindAddress = bindAddress;
_ports = ports.Count == 0 ? [7443] : ports;
_certificate = certificate;
_datagramHandler = datagramHandler;
_log = log;
_peerAddressChanged = peerAddressChanged;
_self = GCHandle.Alloc(this);
}
public IReadOnlyList<int> BoundPorts { get; private set; } = [];
public void Start()
{
if (_registration != 0)
{
throw new InvalidOperationException("MsQuic listener is already started.");
}
_registration = MsQuicApi.RegistrationOpen("moshiro-agent");
try
{
_configuration = MsQuicApi.ConfigurationOpen(_registration, QuicAlpn.Value);
MsQuicConnection.LoadServerCredentials(_configuration, _certificate);
var bound = new List<int>(_ports.Count);
foreach (int port in _ports)
{
nint listener = MsQuicApi.ListenerOpen(_registration, s_callback, GCHandle.ToIntPtr(_self));
_listeners.Add(listener);
byte[] address = Serialize(new IPEndPoint(_bindAddress, port));
MsQuicApi.ListenerStart(listener, QuicAlpn.Value, address);
bound.Add(ReadPort(MsQuicApi.GetAddress(listener, ListenerLocalAddress)));
}
BoundPorts = bound;
}
catch
{
foreach (nint listener in _listeners)
{
MsQuicApi.ListenerStop(listener);
MsQuicApi.ListenerClose(listener);
}
_listeners.Clear();
DisposeHandles();
throw;
}
}
public async ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposed, 1) != 0)
{
return;
}
foreach (nint listener in _listeners)
{
MsQuicApi.ListenerStop(listener);
}
foreach (nint listener in _listeners)
{
MsQuicApi.ListenerClose(listener);
}
_listeners.Clear();
MsQuicConnection[] connections = [.. _connections.Values];
await Task.WhenAll(connections.Select(c => c.DisposeAsync().AsTask())).ConfigureAwait(false);
DisposeHandles();
_self.Free();
}
private static uint OnListenerEvent(nint listener, nint context, nint @event)
{
try
{
var owner = (MsQuicListener)GCHandle.FromIntPtr(context).Target!;
uint eventType = unchecked((uint)Marshal.ReadInt32(@event));
if (eventType != ListenerEventNewConnection || Volatile.Read(ref owner._disposed) != 0)
{
return 1;
}
nint connectionHandle = Marshal.ReadIntPtr(@event, IntPtr.Size * 2);
owner._log.Info("accepted MsQuic QUIC connection");
MsQuicConnection? connection = null;
connection = MsQuicConnection.Accept(
connectionHandle,
owner._configuration,
owner._datagramHandler,
owner._log,
closed => owner._connections.TryRemove(connectionHandle, out _),
owner._peerAddressChanged);
if (!owner._connections.TryAdd(connectionHandle, connection))
{
_ = connection.DisposeAsync().AsTask();
return 1;
}
return 0;
}
catch (Exception ex)
{
try
{
if (context != 0 && GCHandle.FromIntPtr(context).Target is MsQuicListener owner)
{
owner._log.Error("MsQuic listener callback failed", ("err", ex.Message));
}
}
catch
{
// Keep exceptions inside the native callback boundary.
}
return 1;
}
}
private void DisposeHandles()
{
if (_configuration != 0)
{
MsQuicApi.ConfigurationClose(_configuration);
_configuration = 0;
}
if (_registration != 0)
{
MsQuicApi.RegistrationClose(_registration);
_registration = 0;
}
}
private static byte[] Serialize(IPEndPoint endpoint)
{
SocketAddress source = endpoint.Serialize();
byte[] address = new byte[QuicAddressBytes];
for (int i = 0; i < source.Size; i++)
{
address[i] = source[i];
}
return address;
}
private static int ReadPort(byte[] address)
{
if (address.Length < 4)
{
throw new InvalidOperationException("MsQuic returned a truncated bound address.");
}
return (address[2] << 8) | address[3];
}
}
@@ -0,0 +1,118 @@
using System.Runtime.InteropServices;
namespace MoShiro.Measurement;
/// <summary>Loads the MsQuic v2 ABI used by the QUIC datagram transport.</summary>
internal static class MsQuicRuntime
{
private const uint ApiVersion = 2;
private static readonly object s_gate = new();
private static nint s_library;
private static nint s_api;
private static OpenVersionDelegate? s_open;
private static CloseDelegate? s_close;
private static readonly List<nint> s_supportLibraries = [];
public static nint Api
{
get
{
lock (s_gate)
{
if (s_api != 0)
{
return s_api;
}
string[] candidates = OperatingSystem.IsWindows()
? ["msquic.dll"]
: OperatingSystem.IsMacOS()
? ["libmsquic.dylib"]
: ["libmsquic.so.2", "libmsquic.so"];
LoadBundledSupportLibraries();
string appDirectory = AppContext.BaseDirectory;
foreach (string candidate in candidates)
{
if (NativeLibrary.TryLoad(Path.Combine(appDirectory, candidate), out s_library)
|| NativeLibrary.TryLoad(candidate, out s_library))
{
break;
}
}
if (s_library == 0)
{
throw new DllNotFoundException($"MsQuic runtime was not found. Expected one of: {string.Join(", ", candidates)}.");
}
s_open = Marshal.GetDelegateForFunctionPointer<OpenVersionDelegate>(NativeLibrary.GetExport(s_library, "MsQuicOpenVersion"));
s_close = Marshal.GetDelegateForFunctionPointer<CloseDelegate>(NativeLibrary.GetExport(s_library, "MsQuicClose"));
int status = s_open(ApiVersion, out s_api);
if (status != 0 || s_api == 0)
{
NativeLibrary.Free(s_library);
s_library = 0;
s_api = 0;
throw new InvalidOperationException($"MsQuic API v{ApiVersion} could not be opened (status {status}).");
}
return s_api;
}
}
}
public static void Close()
{
lock (s_gate)
{
if (s_api != 0)
{
s_close!(s_api);
s_api = 0;
}
if (s_library != 0)
{
NativeLibrary.Free(s_library);
s_library = 0;
}
for (int i = s_supportLibraries.Count - 1; i >= 0; i--)
{
NativeLibrary.Free(s_supportLibraries[i]);
}
s_supportLibraries.Clear();
}
}
private static void LoadBundledSupportLibraries()
{
if (OperatingSystem.IsLinux())
{
TryLoadSupport("libnuma.so.1");
}
else if (OperatingSystem.IsMacOS())
{
TryLoadSupport("libcrypto.3.dylib");
TryLoadSupport("libssl.3.dylib");
}
}
private static void TryLoadSupport(string library)
{
if (NativeLibrary.TryLoad(Path.Combine(AppContext.BaseDirectory, library), out nint handle))
{
s_supportLibraries.Add(handle);
}
}
public static T Function<T>(int index)
where T : Delegate => Marshal.GetDelegateForFunctionPointer<T>(Marshal.ReadIntPtr(Api, checked(index * IntPtr.Size)));
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate int OpenVersionDelegate(uint version, out nint api);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void CloseDelegate(nint api);
}
@@ -0,0 +1,7 @@
namespace MoShiro.Measurement;
/// <summary>The application protocol negotiated by the Agent and Probe QUIC endpoints.</summary>
internal static class QuicAlpn
{
internal static readonly byte[] Value = "h3"u8.ToArray();
}
@@ -0,0 +1,205 @@
using System.Net;
using System.Net.Sockets;
using System.Security.Cryptography.X509Certificates;
using Fleet.Core;
namespace MoShiro.Measurement;
/// <summary>
/// Measurement data transport. Agent instances listen on configured QUIC UDP ports; Probe instances
/// open one QUIC connection from a newly selected ephemeral source port. Measurement frames are sent
/// only with MsQuic DATAGRAM calls; the existing TLS peer connection continues to carry session control.
/// </summary>
public sealed class QuicDatagramTransport : IDisposable
{
private readonly MeasurementDatagramRouter _router;
private readonly INodeLog _log;
private readonly MsQuicDatagramCallback _onDatagram;
private readonly IPAddress _bind;
private readonly IReadOnlyList<int> _ports;
private readonly X509Certificate2? _serverCertificate;
private readonly DataSession? _clientSession;
private readonly IPEndPoint? _remote;
private readonly byte[]? _peerSpki;
private readonly int _previousSourcePort;
private readonly TaskCompletionSource _clientReady = new(TaskCreationOptions.RunContinuationsAsynchronously);
private MsQuicListener? _listener;
private MsQuicConnection? _client;
private int _disposed;
/// <summary>Constructs an Agent-side QUIC listener.</summary>
public QuicDatagramTransport(IDataSessions sessions, IAdmission? admission, MonoClock clock, DataPlaneCounters counters, INodeLog log, IPAddress bind, IReadOnlyList<int> ports, X509Certificate2 serverCertificate)
{
_router = new MeasurementDatagramRouter(sessions, admission, clock, counters);
_log = log;
_onDatagram = _router.Dispatch;
_bind = bind;
_ports = ports;
_serverCertificate = serverCertificate;
}
/// <summary>Constructs a Probe-side QUIC dialer for one Agent session.</summary>
public QuicDatagramTransport(DataSession session, MonoClock clock, DataPlaneCounters counters, INodeLog log, IPEndPoint remote, byte[] peerSpki, int previousSourcePort)
{
_router = new MeasurementDatagramRouter(new SingleSession(session), null, clock, counters);
_log = log;
_onDatagram = _router.Dispatch;
_bind = remote.AddressFamily == AddressFamily.InterNetworkV6 ? IPAddress.IPv6Any : IPAddress.Any;
_ports = [];
_clientSession = session;
_remote = remote;
_peerSpki = peerSpki;
_previousSourcePort = previousSourcePort;
}
public IReadOnlyList<int> BoundPorts { get; private set; } = [];
public DataPlaneCounters Counters => _router.Counters;
public bool IsConnected => _client?.IsReady == true;
public double? HandshakeDurationMs => _client?.HandshakeDurationMs;
public QuicTransportSnapshot? GetTransportSnapshot()
{
if (_client is null || !_client.TryGetStatistics(out MsQuicApi.QuicStatisticsV2 stats))
{
return null;
}
ulong lost = stats.SendSuspectedLostPackets >= stats.SendSpuriousLostPackets
? stats.SendSuspectedLostPackets - stats.SendSpuriousLostPackets
: 0;
double? packetLossRate = stats.SendTotalPackets > 0 ? lost * 100.0 / stats.SendTotalPackets : null;
return new QuicTransportSnapshot(packetLossRate, stats.SendTotalPackets, stats.SendCongestionCount, stats.SendEcnCongestionCount, stats.SendPathMtu);
}
public void Bind()
{
if (_remote is not null)
{
return;
}
if (_listener is not null)
{
throw new InvalidOperationException("QUIC listener is already bound.");
}
_listener = new MsQuicListener(_bind, _ports, _serverCertificate!, _onDatagram, _log, null);
_listener.Start();
BoundPorts = _listener.BoundPorts;
}
public async Task ConnectAsync(CancellationToken ct)
{
if (_remote is null || _clientSession is null || _peerSpki is null)
{
throw new InvalidOperationException("Only Probe-side QUIC transports can connect.");
}
try
{
int sourcePort = SelectSourcePort(_remote.AddressFamily, _previousSourcePort);
_client = await MsQuicConnection.ConnectAsync(
_remote.Address,
_remote.Port,
sourcePort,
_peerSpki,
_onDatagram,
_log,
ct);
_clientSession.BindDatagramConnection(_client);
BoundPorts = _client.LocalEndPoint is { } local ? [local.Port] : [sourcePort];
_clientReady.TrySetResult();
}
catch (Exception ex)
{
_log.Error("QUIC client connection failed", ("err", ex.Message));
_clientReady.TrySetException(ex);
throw;
}
}
public Task WaitUntilConnectedAsync(CancellationToken ct) => _remote is null ? Task.CompletedTask : _clientReady.Task.WaitAsync(ct);
public void Register(uint streamId, QuicDatagramHandler handler) => _router.Register(streamId, handler);
public void Unregister(uint streamId) => _router.Unregister(streamId);
/// <summary>Sends one measurement datagram on the connection for the supplied session.</summary>
public void Send(DataSession session, EndPoint target, ReadOnlySpan<byte> packet)
{
_ = target; // A QUIC connection owns its path; the Agent learns it from authenticated datagrams.
if (session.DatagramConnection?.Send(packet) == true)
{
_router.Counters.PacketOut();
}
}
public async Task RunAsync(CancellationToken ct)
{
if (_remote is not null)
{
await _clientReady.Task.WaitAsync(ct);
await _client!.WaitForShutdownAsync().WaitAsync(ct);
return;
}
try
{
await Task.Delay(Timeout.InfiniteTimeSpan, ct);
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
// Listener lifetime is controlled by AgentApp.
}
}
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) != 0)
{
return;
}
_ = DisposeNativeResourcesAsync();
}
private async Task DisposeNativeResourcesAsync()
{
try
{
if (_client is not null)
{
await _client.DisposeAsync().ConfigureAwait(false);
}
if (_listener is not null)
{
await _listener.DisposeAsync().ConfigureAwait(false);
}
}
catch (Exception ex) when (ex is IOException or InvalidOperationException or ObjectDisposedException)
{
_log.Error("QUIC datagram transport shutdown failed", ("err", ex.Message));
}
}
private static int SelectSourcePort(AddressFamily family, int previousPort)
{
IPAddress local = family == AddressFamily.InterNetworkV6 ? IPAddress.IPv6Any : IPAddress.Any;
for (int attempt = 0; attempt < 8; attempt++)
{
using var socket = new Socket(family, SocketType.Dgram, ProtocolType.Udp);
socket.Bind(new IPEndPoint(local, 0));
int port = ((IPEndPoint)socket.LocalEndPoint!).Port;
if (port != previousPort)
{
return port;
}
}
throw new SocketException((int)SocketError.AddressAlreadyInUse);
}
}
@@ -1,216 +0,0 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using Fleet.Core;
using MoShiro.Protocol;
namespace MoShiro.Measurement;
/// <summary>一个通过校验的 UDP 包,交给按 <c>stream_id</c> 登记的处理函数。处理函数在接收循环里同步执行,不能阻塞。</summary>
public readonly ref struct UdpDelivery
{
private readonly Span<byte> _packet;
private readonly Socket _socket;
private readonly SocketAddress _source;
private readonly DataPlaneCounters _counters;
internal UdpDelivery(UdpHeader header, Span<byte> packet, long receiveUs, DataSession session, Socket socket, SocketAddress source, DataPlaneCounters counters)
{
Header = header;
_packet = packet;
ReceiveUs = receiveUs;
Session = session;
_socket = socket;
_source = source;
_counters = counters;
}
public UdpHeader Header { get; }
public ReadOnlySpan<byte> Packet => _packet;
/// <summary>包到达的时刻(本节点的单调时钟,微秒):从接收返回的那一刻取,不含校验的耗时。</summary>
public long ReceiveUs { get; }
public DataSession Session { get; }
/// <summary>
/// 原地回复:同样的长度,只把 type 改成 <paramref name="type"/> 并重新计算 MAC,发回请求的实际源地址。
/// 回复与请求等长,所以监听方不能被当成放大器。发送失败(源地址已经不可达)不抛出。
/// </summary>
public void ReplyAs(UdpPacketType type)
{
_packet[3] = (byte)type;
UdpPacket.Seal(_packet, Session.Key);
try
{
_socket.SendTo(_packet, SocketFlags.None, _source);
_counters.PacketOut();
}
catch (SocketException)
{
// 回复丢失由对端的缺失体现
}
}
}
public delegate void UdpHandler(in UdpDelivery delivery);
/// <summary>
/// UDP 数据面:一个或多个套接字,按 <c>stream_id</c> 把包分发给积木。校验顺序:长度、magic、version、type →
/// 流和会话都存在 → MAC → (监听方)<see cref="IAdmission"/>(重放窗口、限速)。任何一步失败都静默丢弃并计数,不回复。
/// 接收循环只做校验和分发,不分配对象。两端共用这一份代码:Agent 监听多个端口并带准入策略,Probe 每个 Agent 一个临时端口。
/// </summary>
public sealed class UdpMux : IDisposable
{
private const int WindowsUdpConnReset = -1744830452; // SIO_UDP_CONNRESET
private readonly IDataSessions _sessions;
private readonly IAdmission? _admission;
private readonly MonoClock _clock;
private readonly DataPlaneCounters _counters;
private readonly INodeLog _log;
private readonly IPAddress _bind;
private readonly IReadOnlyList<int> _ports;
private readonly List<Socket> _sockets = [];
private readonly ConcurrentDictionary<uint, UdpHandler> _handlers = new();
/// <param name="ports">要监听的端口(0 表示由系统分配);空表示一个临时端口。</param>
public UdpMux(IDataSessions sessions, IAdmission? admission, MonoClock clock, DataPlaneCounters counters, INodeLog log, IPAddress bind, IReadOnlyList<int> ports)
{
_sessions = sessions;
_admission = admission;
_clock = clock;
_counters = counters;
_log = log;
_bind = bind;
_ports = ports.Count == 0 ? [0] : ports;
}
public IReadOnlyList<int> BoundPorts { get; private set; } = [];
public DataPlaneCounters Counters => _counters;
/// <summary>绑定套接字(端口被占用等错误在这里抛出)。</summary>
public void Bind()
{
var bound = new List<int>();
foreach (int port in _ports)
{
var socket = new Socket(_bind.AddressFamily, SocketType.Dgram, ProtocolType.Udp) { ReceiveBufferSize = 4 << 20, SendBufferSize = 4 << 20 };
if (_bind.AddressFamily == AddressFamily.InterNetworkV6)
{
socket.DualMode = true;
}
if (OperatingSystem.IsWindows())
{
// Windows 上对一个已关闭的端口发送之后,下一次接收会抛 ConnectionReset:关掉这个行为
socket.IOControl(WindowsUdpConnReset, [0], null);
}
socket.Bind(new IPEndPoint(_bind, port));
bound.Add(((IPEndPoint)socket.LocalEndPoint!).Port);
_sockets.Add(socket);
}
BoundPorts = bound;
}
public void Register(uint streamId, UdpHandler handler) => _handlers[streamId] = handler;
public void Unregister(uint streamId) => _handlers.TryRemove(streamId, out _);
/// <summary>从第一个套接字发出一个已经封好的包。发送失败(没有路由、缓冲区满)不抛出:丢包本身就是测量结果。</summary>
public void Send(EndPoint to, ReadOnlySpan<byte> packet)
{
try
{
_sockets[0].SendTo(packet, SocketFlags.None, to);
_counters.PacketOut();
}
catch (SocketException)
{
// 记为丢失
}
}
public async Task RunAsync(CancellationToken ct)
{
try
{
await Task.WhenAll(_sockets.Select(s => ReceiveLoopAsync(s, ct)));
}
catch (ObjectDisposedException)
{
// 套接字已释放:停止
}
}
public void Dispose()
{
foreach (Socket socket in _sockets)
{
socket.Dispose();
}
}
private async Task ReceiveLoopAsync(Socket socket, CancellationToken ct)
{
byte[] buffer = new byte[UdpPacket.MaxBytes + 64];
var source = new SocketAddress(socket.AddressFamily);
while (!ct.IsCancellationRequested)
{
int length;
try
{
length = await socket.ReceiveFromAsync(buffer.AsMemory(), SocketFlags.None, source, ct);
}
catch (OperationCanceledException)
{
return;
}
catch (ObjectDisposedException)
{
return;
}
catch (SocketException ex)
{
// 个别错误(例如对端的 ICMP 不可达)不能让接收循环退出
_log.Warn("udp receive failed", ("err", ex.SocketErrorCode));
continue;
}
Dispatch(socket, buffer, length, source, _clock.NowUs());
}
}
private void Dispatch(Socket socket, byte[] buffer, int length, SocketAddress source, long receiveUs)
{
_counters.PacketIn();
Span<byte> packet = buffer.AsSpan(0, length);
if (!UdpPacket.TryParseHeader(packet, out UdpHeader header)
|| !_handlers.TryGetValue(header.StreamId, out UdpHandler? handler)
|| _sessions.Find(header.SessionId) is not { } session
|| !UdpPacket.VerifyMac(packet, session.Key))
{
_counters.DropInvalid();
return;
}
if (_admission is not null)
{
switch (_admission.AdmitPacket(session, header, source, receiveUs))
{
case Admission.Invalid:
_counters.DropInvalid();
return;
case Admission.RateLimited:
_counters.DropRateLimited();
return;
}
}
handler(new UdpDelivery(header, packet, receiveUs, session, socket, source, _counters));
}
}
@@ -0,0 +1,19 @@
namespace MoShiro.Protocol;
/// <summary>构建信息的人能读的形式(启动时打进日志);常量部分由 MoShiro.Protocol.csproj 的 WriteBuildInfo 在构建时生成。</summary>
public static partial class BuildInfo
{
/// <summary>版本、tag、提交、构建时间的字段,没有的(本地构建不设环境变量)写 unknown。</summary>
public static (string Key, object? Value)[] Fields { get; } =
[
("version", Or(Version)),
("tag", Or(Tag)),
("commit", Or(Commit)),
("built_at", Or(BuiltAt)),
];
/// <summary>一行摘要,例如:0.4.1-dev.2-a3338e1 (tag v0.4.1-dev.2-a3338e1, commit a3338e1b2c, built 2026-10-09T03:00:00Z)。</summary>
public static string Summary => $"{Or(Version)} (tag {Or(Tag)}, commit {Or(Commit)}, built {Or(BuiltAt)})";
private static string Or(string value) => string.IsNullOrWhiteSpace(value) ? "unknown" : value;
}
@@ -27,10 +27,10 @@ public static class Kinds
public static readonly KindSpec UdpEcho = new(
"udp_echo",
"UDP echo",
"Fixed-rate UDP echo with four timestamps per packet.",
"QUIC Datagram echo",
"Fixed-rate QUIC Datagram echo with four timestamps per packet.",
Scope.Line,
new BlockSet("udp", "fixed", "fixed", "echo", []),
new BlockSet("quic_datagram", "fixed", "fixed", "echo", []),
[
P.Int("rate_hz", 10, 1, 200, "Hz"),
P.Int("size", 64, 44, 1400, "B"),
@@ -41,10 +41,10 @@ public static class Kinds
public static readonly KindSpec UdpTrain = new(
"udp_train",
"UDP train",
"Fixed-rate or laddered one-way UDP stream (Agent to Probe) for loss and jitter under load.",
"QUIC Datagram train",
"Fixed-rate or laddered one-way QUIC Datagram stream (Agent to Probe) for loss and delay variation under load.",
Scope.Line,
new BlockSet("udp", "ladder", "fixed", "sink", []),
new BlockSet("quic_datagram", "ladder", "fixed", "sink", []),
[
P.Number("mbps", 10, 0.1, 1000, "Mbit/s"),
P.Int("size", 1250, 200, 1400, "B"),
@@ -18,7 +18,7 @@
<ItemGroup>
<BuildInfoLine Include="namespace MoShiro.Protocol%3B" />
<BuildInfoLine Include="/// &lt;summary&gt;这个程序是什么时候、从哪个版本构建出来的(构建时生成)。&lt;/summary&gt;" />
<BuildInfoLine Include="public static class BuildInfo" />
<BuildInfoLine Include="public static partial class BuildInfo" />
<BuildInfoLine Include="{" />
<BuildInfoLine Include=" public const string Version = &quot;$(Version)&quot;%3B" />
<BuildInfoLine Include=" public const string Tag = &quot;$(MOSHIRO_TAG)&quot;%3B" />
@@ -146,8 +146,11 @@ public sealed class AgentEndpoint
[JsonPropertyName("tcp_port")]
public int TcpPort { get; set; }
[JsonPropertyName("quic_ports")]
public List<int> QuicPorts { get; set; } = [];
[JsonPropertyName("udp_ports")]
public List<int> UdpPorts { get; set; } = [];
public List<int> LegacyUdpPorts { get; set; } = [];
/// <summary>Agent 面向 Probe 的 TLS 证书公钥(SPKI,base64):证书按它的 SHA-256 指纹固定,敲门密钥由它和 Probe 的身份密钥派生。</summary>
[JsonPropertyName("public_key")]
@@ -175,8 +178,11 @@ public sealed class AgentEnrollMeta
[JsonPropertyName("probe_tcp_port")]
public int ProbeTcpPort { get; set; } = 7443;
[JsonPropertyName("probe_quic_ports")]
public List<int> ProbeQuicPorts { get; set; } = [];
[JsonPropertyName("probe_udp_ports")]
public List<int> ProbeUdpPorts { get; set; } = [7443];
public List<int>? LegacyProbeUdpPorts { get; set; }
[JsonPropertyName("public_host")]
public string? PublicHost { get; set; }
@@ -296,6 +302,9 @@ public sealed class UpdateStatus
/// <summary>Probe 的 <c>fleet.heartbeat</c> 附加字段:更新器的状态。</summary>
public sealed class ProbeHeartbeatExtra
{
[JsonPropertyName("quic_links")]
public List<ProbeQuicLinkHealth>? QuicLinks { get; set; }
[JsonPropertyName("ipv6")]
public Ipv6Diagnostic? Ipv6 { get; set; }
@@ -303,6 +312,53 @@ public sealed class ProbeHeartbeatExtra
public UpdateStatus? Update { get; set; }
}
/// <summary>Probe's rolling QUIC connection health snapshot for one Agent/IP path.</summary>
public sealed class ProbeQuicLinkHealth
{
[JsonPropertyName("agent_id")]
public string AgentId { get; set; } = "";
[JsonPropertyName("ip_version")]
public string IpVersion { get; set; } = IpVersions.V4;
[JsonPropertyName("connected")]
public bool Connected { get; set; }
[JsonPropertyName("handshake_attempts")]
public long HandshakeAttempts { get; set; }
[JsonPropertyName("handshake_successes")]
public long HandshakeSuccesses { get; set; }
[JsonPropertyName("handshake_failures")]
public long HandshakeFailures { get; set; }
[JsonPropertyName("handshake_timeouts")]
public long HandshakeTimeouts { get; set; }
[JsonPropertyName("handshake_p95_ms")]
public double? HandshakeP95Ms { get; set; }
[JsonPropertyName("handshake_sample_count")]
public long HandshakeSampleCount { get; set; }
/// <summary>QUIC packet-loss estimate from MsQuic's non-spurious loss counters; not Datagram application loss or retransmission.</summary>
[JsonPropertyName("transport_packet_loss_rate")]
public double? TransportPacketLossRate { get; set; }
[JsonPropertyName("transport_packets_sent")]
public ulong? TransportPacketsSent { get; set; }
[JsonPropertyName("transport_congestion_events")]
public uint? TransportCongestionEvents { get; set; }
[JsonPropertyName("transport_ecn_congestion_events")]
public uint? TransportEcnCongestionEvents { get; set; }
[JsonPropertyName("path_mtu")]
public ushort? PathMtu { get; set; }
}
/// <summary>Probe 的 <c>fleet.hello</c> 附加字段:系统、架构和构建信息(只用于展示和更新)。</summary>
public sealed class ProbeHelloMeta
{
@@ -345,8 +401,11 @@ public sealed class AgentHelloMeta
[JsonPropertyName("probe_tcp_port")]
public int? ProbeTcpPort { get; set; }
[JsonPropertyName("probe_quic_ports")]
public List<int>? ProbeQuicPorts { get; set; }
[JsonPropertyName("probe_udp_ports")]
public List<int>? ProbeUdpPorts { get; set; }
public List<int>? LegacyProbeUdpPorts { get; set; }
[JsonPropertyName("local_limits")]
public LocalLimits? LocalLimits { get; set; }
@@ -482,8 +541,11 @@ public sealed class LineTaskSet
public List<LineTask> Tasks { get; set; } = [];
/// <summary>Agent 的数据面端口(Probe 把包发到这里)。</summary>
[JsonPropertyName("quic_ports")]
public List<int> QuicPorts { get; set; } = [];
[JsonPropertyName("udp_ports")]
public List<int> UdpPorts { get; set; } = [];
public List<int> LegacyUdpPorts { get; set; } = [];
[JsonPropertyName("tcp_ports")]
public List<int> TcpPorts { get; set; } = [];
@@ -3,7 +3,7 @@ using System.Security.Cryptography;
namespace MoShiro.Protocol;
public enum UdpPacketType : byte
public enum MeasurementDatagramType : byte
{
EchoRequest = 1,
EchoReply = 2,
@@ -12,7 +12,7 @@ public enum UdpPacketType : byte
Data = 3,
}
public readonly record struct UdpHeader(UdpPacketType Type, long SessionId, uint StreamId, uint Seq, long SendTsUs);
public readonly record struct MeasurementDatagramHeader(MeasurementDatagramType Type, long SessionId, uint StreamId, uint Seq, long SendTsUs);
/// <summary>
/// 数据面的 UDP 包(Fleet 之外的线格式):
@@ -20,7 +20,7 @@ public readonly record struct UdpHeader(UdpPacketType Type, long SessionId, uint
/// mac = HMAC-SHA256(session_key, 前面全部字节) 截断到 16 字节。所有整数小端。
/// 接收方对不合法的包静默丢弃并计数,不回复(防放大)。
/// </summary>
public static class UdpPacket
public static class MeasurementDatagram
{
public const ushort Magic = 0x4D4E; // "MN"
public const byte Version = 1;
@@ -30,19 +30,19 @@ public static class UdpPacket
public const int MaxBytes = 1472;
/// <summary>检查长度、magic、version、type 并解析头部(不校验 MAC)。</summary>
public static bool TryParseHeader(ReadOnlySpan<byte> packet, out UdpHeader header)
public static bool TryParseHeader(ReadOnlySpan<byte> packet, out MeasurementDatagramHeader header)
{
header = default;
if (packet.Length is < MinBytes or > MaxBytes
|| BinaryPrimitives.ReadUInt16LittleEndian(packet) != Magic
|| packet[2] != Version
|| packet[3] is < (byte)UdpPacketType.EchoRequest or > (byte)UdpPacketType.Data)
|| packet[3] is < (byte)MeasurementDatagramType.EchoRequest or > (byte)MeasurementDatagramType.Data)
{
return false;
}
header = new UdpHeader(
(UdpPacketType)packet[3],
header = new MeasurementDatagramHeader(
(MeasurementDatagramType)packet[3],
BinaryPrimitives.ReadInt64LittleEndian(packet[4..]),
BinaryPrimitives.ReadUInt32LittleEndian(packet[12..]),
BinaryPrimitives.ReadUInt32LittleEndian(packet[16..]),
@@ -51,7 +51,7 @@ public static class UdpPacket
}
/// <summary>写入整个包(长度就是 <paramref name="destination"/> 的长度),填充为零,末尾是 MAC。</summary>
public static void Write(Span<byte> destination, in UdpHeader header, ReadOnlySpan<byte> key)
public static void Write(Span<byte> destination, in MeasurementDatagramHeader header, ReadOnlySpan<byte> key)
{
if (destination.Length is < MinBytes or > MaxBytes)
{
@@ -19,8 +19,8 @@ public static class TcpWire
public static void WriteAuth(Span<byte> destination, long sessionId, uint streamId, ReadOnlySpan<byte> nonce, ReadOnlySpan<byte> key)
{
BinaryPrimitives.WriteUInt16LittleEndian(destination, UdpPacket.Magic);
destination[2] = UdpPacket.Version;
BinaryPrimitives.WriteUInt16LittleEndian(destination, MeasurementDatagram.Magic);
destination[2] = MeasurementDatagram.Version;
BinaryPrimitives.WriteInt64LittleEndian(destination[3..], sessionId);
BinaryPrimitives.WriteUInt32LittleEndian(destination[11..], streamId);
nonce[..NonceBytes].CopyTo(destination[15..]);
@@ -35,7 +35,7 @@ public static class TcpWire
sessionId = 0;
streamId = 0;
nonce = default;
if (frame.Length != AuthBytes || BinaryPrimitives.ReadUInt16LittleEndian(frame) != UdpPacket.Magic || frame[2] != UdpPacket.Version)
if (frame.Length != AuthBytes || BinaryPrimitives.ReadUInt16LittleEndian(frame) != MeasurementDatagram.Magic || frame[2] != MeasurementDatagram.Version)
{
return false;
}
@@ -16,7 +16,7 @@ public sealed class MeasurementTests
Params = parameters.ToDictionary(p => p.Key, p => Json(p.Raw)),
};
private static BudgetGuard Guard(NodeLimits? limits = null) => new(limits ?? new NodeLimits { AllowedUdpPorts = [7443], AllowedTcpPorts = [7444] }, BlockCatalog.AgentKinds);
private static BudgetGuard Guard(NodeLimits? limits = null) => new(limits ?? new NodeLimits { AllowedQuicPorts = [7443], AllowedTcpPorts = [7444] }, BlockCatalog.AgentKinds);
[Fact]
public void BudgetGuard_AcceptsDefaults_RejectsUnknownKindsBadParamsAndForeignPorts()
@@ -34,7 +34,7 @@ public sealed class MeasurementTests
[Fact]
public void BudgetGuard_AccumulatesTheBudget_AndRejectsWhatDoesNotFit()
{
BudgetGuard guard = Guard(new NodeLimits { MaxTotalMbps = 15, AllowedUdpPorts = [7443] });
BudgetGuard guard = Guard(new NodeLimits { MaxTotalMbps = 15, AllowedQuicPorts = [7443] });
double mbps = 0;
long pps = 0;
Assert.True(guard.CheckPhase("udp_train", new Dictionary<string, JsonElement> { ["mbps"] = Json("10") }).Accepted);
@@ -298,7 +298,7 @@ public sealed class MeasurementTests
[Fact]
public void EchoRows_Udp_GivesUpOnAMissingReplyAfterTwoSeconds()
{
var rows = new EchoRows(Kinds.UdpEcho, 64, EchoRows.UdpGraceUs, settleOnReply: false);
var rows = new EchoRows(Kinds.UdpEcho, 64, EchoRows.DatagramGraceUs, settleOnReply: false);
rows.Add(64, 0);
rows.Add(64, 1_000_000);
RawStream cut = rows.Take(2_500_000, all: false)!;
@@ -26,10 +26,18 @@ public static class LinkMetrics
"jitter.down", s_udpMsg, Ops.OwdVar(Dir.Down), "Jitter (down)", "One-way delay variation, offset and drift removed.", Unit.Ms, [Stat.P95], good: 3, poor: 20);
public static readonly MetricDef LossRun = Metric.RunLength(
"loss.run", s_udpMsg, Ops.LossRuns, "Loss runs", "Lengths of consecutive lost packets.",
"loss.run", s_udpMsg, Ops.LossRuns(Dir.Down), "Loss runs (legacy downstream alias)", "Legacy downstream-only consecutive loss metric.",
new MetricExport("loss.run_max", Stat.Max, Unit.Packets, 2, 10),
new MetricExport("loss.run_mean", Stat.Mean, Unit.Packets));
public static readonly MetricDef LossRunUp = Metric.Max(
"loss.run.up", s_udpMsg, Ops.LossRunMax(Dir.Up), "Upstream longest loss run", "Longest consecutive Probe-to-Agent Datagram loss run in each chunk; zero means no loss.",
Unit.Packets, "loss.run.up_max", good: 0, poor: 10);
public static readonly MetricDef LossRunDown = Metric.Max(
"loss.run.down", s_udpMsg, Ops.LossRunMax(Dir.Down), "Downstream longest loss run", "Longest consecutive Agent-to-Probe Datagram loss run in each chunk, conditioned on the request reaching the Agent; zero means no loss.",
Unit.Packets, "loss.run.down_max", good: 0, poor: 10);
public static readonly MetricDef Reorder = Metric.Counter(
"reorder", s_udpMsg, Ops.Reorder, "Reordered", "Replies that arrived after a higher sequence number.", new MetricExport("reorder", Stat.Count, Unit.Count));
@@ -11,7 +11,7 @@ public static class MetricRegistry
public static readonly IReadOnlyList<MetricDef> All =
[
LinkMetrics.Rtt, LinkMetrics.LossUp, LinkMetrics.LossDown, LinkMetrics.LossTotal,
LinkMetrics.JitterUp, LinkMetrics.JitterDown, LinkMetrics.LossRun, LinkMetrics.Reorder, LinkMetrics.Dup, LinkMetrics.SamplesUdp,
LinkMetrics.JitterUp, LinkMetrics.JitterDown, LinkMetrics.LossRun, LinkMetrics.LossRunUp, LinkMetrics.LossRunDown, LinkMetrics.Reorder, LinkMetrics.Dup, LinkMetrics.SamplesUdp,
TcpMetrics.Rtt, TcpMetrics.Slow, TcpMetrics.Stall, TcpMetrics.StallMax, TcpMetrics.Retrans, TcpMetrics.Srtt, TcpMetrics.BulkRate, TcpMetrics.BulkPeak, TcpMetrics.SamplesTcp,
SessionMetrics.ConnectSuccess, SessionMetrics.ConnectTcp, SessionMetrics.ConnectTls,
SessionMetrics.FailTimeout, SessionMetrics.FailReset, SessionMetrics.FailTls,
@@ -22,7 +22,8 @@ public static class MetricRegistry
/// <summary>指标分组:前端按分组展示,新增指标只需要把它加进分组。</summary>
public static readonly IReadOnlyList<PresetDef> Presets =
[
new("udp", "UDP", ["loss.up", "loss.down", "loss.total", "rtt.p50", "rtt.p95", "rtt.p99", "jitter.up.p95", "jitter.down.p95", "loss.run_max", "reorder", "dup"]),
new("quic", "QUIC Datagram", ["loss.up", "loss.down", "loss.total", "rtt.p50", "rtt.p95", "rtt.p99", "jitter.up.p95", "jitter.down.p95", "loss.run.up_max", "loss.run.down_max", "reorder", "dup"]),
new("udp", "QUIC Datagram (legacy alias)", ["loss.up", "loss.down", "loss.total", "rtt.p50", "rtt.p95", "rtt.p99", "jitter.up.p95", "jitter.down.p95", "loss.run_max", "reorder", "dup"]),
new("tcp", "TCP", ["tcp.rtt.p50", "tcp.rtt.p95", "tcp.rtt.p99", "tcp.rtt.p999", "tcp.rtt.max", "tcp.rtt.tail", "tcp.slow", "tcp.stall.per_min", "tcp.stall_max", "tcp.retrans", "tcp.srtt.p95"]),
new("connect", "Connect", ["connect.success", "connect.tcp_ms.p95", "connect.tls_ms.p95", "connect.fail.timeout", "connect.fail.rst", "connect.fail.tls"]),
new("session", "Session", ["hold.resets.per_h", "hold.lifetime_max"]),
@@ -1,6 +1,6 @@
namespace MoShiro.Backend.Analysis;
/// <summary>连接建立、长连接、UDP 压测(udp_train)的指标。</summary>
/// <summary>TCP 建连、长连接、QUIC Datagram 压测(legacy kind udp_train)的指标。</summary>
public static class SessionMetrics
{
private static readonly string[] s_conn = ["connect.conn"];
@@ -33,13 +33,13 @@ public static class SessionMetrics
"hold.lifetime", s_hold, Ops.HoldLifetime, "Longest connection lifetime", "Longest observed lifetime of the held connection (ms).", Unit.Ms, "hold.lifetime_max", Better.Higher);
public static readonly MetricDef TrainLoss = Metric.Ratio(
"train.loss", s_train, Ops.TrainLoss, "Train loss", "Loss of a fixed-rate UDP stream.", good: 1, poor: 10);
"train.loss", s_train, Ops.TrainLoss, "Datagram train loss", "Loss of a fixed-rate QUIC Datagram stream.", good: 1, poor: 10);
public static readonly MetricDef TrainJitter = Metric.Histogram(
"train.jitter", s_train, Ops.TrainJitter, "Train jitter", "One-way delay variation of a fixed-rate UDP stream.", Unit.Ms, [Stat.P95], good: 3, poor: 20);
"train.jitter", s_train, Ops.TrainJitter, "Datagram train delay variation", "One-way delay variation of a fixed-rate QUIC Datagram stream.", Unit.Ms, [Stat.P95], good: 3, poor: 20);
public static readonly MetricDef TrainRate = Metric.Histogram(
"train.rate", s_train, Ops.TrainRate, "Train rate", "Delivered rate of a fixed-rate UDP stream per second.", Unit.Mbps, [Stat.P50, Stat.Mean], Better.Higher);
"train.rate", s_train, Ops.TrainRate, "Datagram train rate", "Delivered rate of a fixed-rate QUIC Datagram stream per second.", Unit.Mbps, [Stat.P50, Stat.Mean], Better.Higher);
public static readonly MetricDef SamplesConnect = Metric.Counter(
"n.connect", s_conn, Ops.RowCount, "connect samples", "Number of connection attempts.", new MetricExport("n.connect", Stat.Count, Unit.Count), Better.None);
@@ -12,9 +12,9 @@ public sealed class BaseModel : ServiceModelBase
public override string Title => "Line quality";
public override int Version => 1;
public override int Version => 2;
public override IReadOnlyList<string> Needs { get; } = ["rtt.p95", "loss.total", "jitter.down.p95", "jitter.up.p95", "loss.run_max"];
public override IReadOnlyList<string> Needs { get; } = ["rtt.p95", "loss.total", "jitter.down.p95", "jitter.up.p95", "loss.run.up_max", "loss.run.down_max"];
public override ModelScore Score(MetricView view) =>
Combine(
@@ -24,7 +24,8 @@ public sealed class BaseModel : ServiceModelBase
Dim(view, "loss.total", "Packet loss", 4, s_loss),
Dim(view, "jitter.down.p95", "Jitter (down)", 1.5, s_jitter),
Dim(view, "jitter.up.p95", "Jitter (up)", 1.5, s_jitter),
Dim(view, "loss.run_max", "Longest loss run", 1, s_run),
Dim(view, "loss.run.up_max", "Longest upstream loss run", 0.5, s_run),
Dim(view, "loss.run.down_max", "Longest downstream loss run", 0.5, s_run),
],
estimated: false);
}
@@ -273,7 +273,14 @@ internal static class JitterEmitter
/// <summary>连续丢失段的长度(按发送顺序)。</summary>
internal sealed class LossRunsOp : MetricOp
{
public override string Signature => "loss_runs/1";
private readonly Dir _direction;
public LossRunsOp(Dir direction)
{
_direction = direction;
}
public override string Signature => $"loss_runs/{_direction}/2";
public override void Evaluate(ChunkView view, RawStream stream, IObservationSink sink)
{
@@ -292,7 +299,13 @@ internal sealed class LossRunsOp : MetricOp
continue;
}
if (c.IsReplyLost(i))
bool lost = _direction switch
{
Dir.Up => c.IsUpLost(i),
Dir.Down => !c.IsUpLost(i) && c.IsReplyLost(i),
_ => c.IsReplyLost(i),
};
if (lost)
{
if (run == 0)
{
@@ -315,6 +328,59 @@ internal sealed class LossRunsOp : MetricOp
}
}
/// <summary>每个块一个最长连续丢包值;无丢包的有效块明确输出 0。</summary>
internal sealed class LossRunMaxOp(Dir direction) : MetricOp
{
public override string Signature => $"loss_run_max/{direction}/1";
public override void Evaluate(ChunkView view, RawStream stream, IObservationSink sink)
{
var c = new EchoColumns(stream);
if (!c.HasBasics || (direction != Dir.Total && c.T2 is null))
{
return;
}
int run = 0;
int maximum = 0;
bool hasSample = false;
long sampleAt = 0;
for (int i = 0; i < c.Count; i++)
{
if (!c.T1!.IsValid(i))
{
continue;
}
bool lost = direction switch
{
Dir.Up => c.IsUpLost(i),
Dir.Down => !c.IsUpLost(i) && c.IsReplyLost(i),
_ => c.IsReplyLost(i),
};
if (direction == Dir.Down && c.IsUpLost(i))
{
// A request lost before reaching the Agent is not a downstream sample.
run = 0;
continue;
}
if (!hasSample)
{
sampleAt = c.SentUtcMs(view, i);
hasSample = true;
}
run = lost ? run + 1 : 0;
maximum = Math.Max(maximum, run);
}
if (hasSample)
{
sink.Emit(sampleAt, maximum);
}
}
}
/// <summary>乱序:按到达顺序,序号比之前见过的最大序号还小。重复的序号不算乱序。</summary>
internal sealed class ReorderOp : MetricOp
{
@@ -11,7 +11,9 @@ public static class Ops
public static MetricOp OwdVar(Dir dir) => new OwdVarOp(dir);
public static MetricOp LossRuns { get; } = new LossRunsOp();
public static MetricOp LossRuns(Dir direction) => new LossRunsOp(direction);
public static MetricOp LossRunMax(Dir direction) => new LossRunMaxOp(direction);
public static MetricOp Reorder { get; } = new ReorderOp();
@@ -31,7 +31,23 @@ public static class Mappers
public static OverviewLineDto ToDto(OverviewLine line) =>
new(
line.Line.Key.Id, line.Line.Agent.Agent.Id.Value, line.Line.Probe.Id, line.Line.Probe.Name, line.Line.Agent.Agent.Name, line.Line.Agent.Agent.Labels,
line.Line.Agent.Online, line.Line.Connected, line.Line.Agent.EndpointChanges, ToDto(line.Summary), Summary(line.LastTest), line.Line.IpVersion);
line.Line.Agent.Online, line.Line.Connected, line.Line.Agent.EndpointChanges, ToDto(line.Summary), Summary(line.LastTest), line.Line.IpVersion, ToDto(line.Line.QuicHealth));
public static QuicLineHealthDto? ToDto(ProbeQuicLinkSnapshot? snapshot)
{
if (snapshot is null)
{
return null;
}
ProbeQuicLinkHealth health = snapshot.Health;
double? Rate(long count) => health.HandshakeAttempts > 0 ? count * 100.0 / health.HandshakeAttempts : null;
return new QuicLineHealthDto(
health.Connected, health.HandshakeAttempts, health.HandshakeSuccesses, health.HandshakeFailures, health.HandshakeTimeouts,
Rate(health.HandshakeSuccesses), Rate(health.HandshakeFailures), Rate(health.HandshakeTimeouts), health.HandshakeP95Ms, health.HandshakeSampleCount,
health.TransportPacketLossRate, health.TransportPacketsSent, health.TransportCongestionEvents,
health.TransportEcnCongestionEvents, health.PathMtu, Time(snapshot.AtMs));
}
public static HostDto ToDto(HostSnapshot h) =>
new(h.CpuPercent, h.MemUsed, h.MemTotal, h.DiskUsed, h.DiskTotal, h.NetRxTotal, h.NetTxTotal, h.NetRxBps, h.NetTxBps, h.NetLinkMbps, h.UptimeSeconds, Time(h.AtUtcMs));
@@ -102,7 +118,7 @@ public static class Mappers
? new HeartbeatDto(h.Sessions, h.PpsIn, h.PpsOut, h.DroppedInvalid, h.DroppedRateLimited, h.QueueBytes, Time(view.HeartbeatAtMs ?? 0))
: null;
return new AgentDto(
a.Id.Value, a.Name, StatusName(a.Status), view.Online, view.ProbeConnected, [.. view.Probes.Select(ToDto)], view.Host is null ? null : ToDto(view.Host), a.PublicHost, a.ProbeTcpPort, a.ProbeUdpPorts, a.Labels, a.PublicVisible,
a.Id.Value, a.Name, StatusName(a.Status), view.Online, view.ProbeConnected, [.. view.Probes.Select(ToDto)], view.Host is null ? null : ToDto(view.Host), a.PublicHost, a.ProbeTcpPort, a.ProbeUdpPorts, a.ProbeUdpPorts, a.Labels, a.PublicVisible,
a.Kinds, a.AgentVersion, limits, a.TaskVersion, a.AppliedTaskVersion, rejected, heartbeat, view.EndpointChanges, Time(a.CreatedAtMs), Time(a.LastSeenAtMs), a.Ipv6Host, a.Ipv6Supported, Ipv6Of(view), !a.PublicHostManual, !a.Ipv6HostManual, a.AddedBy);
}
@@ -42,6 +42,7 @@ public sealed record AgentDto(
string PublicHost,
int ProbeTcpPort,
IReadOnlyList<int> ProbeUdpPorts,
IReadOnlyList<int> ProbeQuicPorts,
IReadOnlyList<string> Labels,
bool PublicVisible,
IReadOnlyList<string> Kinds,
@@ -22,6 +22,7 @@ namespace MoShiro.Backend.Api;
[JsonSerializable(typeof(DistributionDto))]
[JsonSerializable(typeof(HeatmapDto))]
[JsonSerializable(typeof(LineDto))]
[JsonSerializable(typeof(QuicLineHealthDto))]
[JsonSerializable(typeof(TestDto))]
[JsonSerializable(typeof(PageDto<TestDto>))]
[JsonSerializable(typeof(PageDto<KindDto>))]
@@ -28,7 +28,27 @@ public sealed record OverviewLineDto(
int EndpointChanges,
SummaryDto Summary,
TestSummaryDto? LastTest,
string IpVersion = "ipv4");
string IpVersion = "ipv4",
QuicLineHealthDto? QuicHealth = null);
/// <summary>Latest QUIC session health reported by the Probe; cumulative counters reset when the Probe restarts.</summary>
public sealed record QuicLineHealthDto(
bool Connected,
long HandshakeAttempts,
long HandshakeSuccesses,
long HandshakeFailures,
long HandshakeTimeouts,
double? HandshakeSuccessRate,
double? HandshakeFailureRate,
double? HandshakeTimeoutRate,
double? HandshakeP95Ms,
long HandshakeSampleCount,
double? TransportPacketLossRate,
ulong? TransportPacketsSent,
uint? TransportCongestionEvents,
uint? TransportEcnCongestionEvents,
ushort? PathMtu,
DateTimeOffset At);
public sealed record OverviewDto(string Range, DateTimeOffset From, DateTimeOffset To, string Step, string Source, bool Degraded, IReadOnlyList<OverviewLineDto> Lines);
@@ -90,7 +110,8 @@ public sealed record LineDto(
IReadOnlyList<string> Kinds,
TestSummaryDto? LastTest,
string IpVersion = "ipv4",
bool Ipv6Available = false);
bool Ipv6Available = false,
QuicLineHealthDto? QuicHealth = null);
public sealed record DimensionDto(string Id, string Label, double Weight, double? Value, double? Score);
@@ -13,8 +13,8 @@ namespace MoShiro.Backend.Api;
public static class QueryEndpoints
{
private const string Tag = "public";
private const string DefaultOverviewPreset = "udp,tcp,scores";
private const string DefaultSeriesPreset = "udp,tcp";
private const string DefaultOverviewPreset = "quic,tcp,scores";
private const string DefaultSeriesPreset = "quic,tcp";
public static void Map(RouteGroupBuilder group)
{
@@ -102,7 +102,7 @@ public static class QueryEndpoints
return TypedResults.Json(
new LineDto(
line.Key.Id, view.Agent.Id.Value, line.Probe.Id, line.Probe.Name, view.Agent.Name, view.Agent.Labels, view.Online, line.Connected,
Mappers.Time(await query.EarliestAsync(line.Key, ct)), view.Agent.Kinds, Mappers.Summary(last), line.IpVersion, view.Agent.Ipv6Host.Length > 0),
Mappers.Time(await query.EarliestAsync(line.Key, ct)), view.Agent.Kinds, Mappers.Summary(last), line.IpVersion, view.Agent.Ipv6Host.Length > 0, Mappers.ToDto(line.QuicHealth)),
ApiJsonContext.Default.LineDto);
}
@@ -173,7 +173,8 @@ public sealed partial class AgentService
Host = agent.PublicHost,
Ipv6Host = agent.Ipv6Supported && agent.Ipv6Host.Length > 0 ? agent.Ipv6Host : null,
TcpPort = agent.ProbeTcpPort,
UdpPorts = [.. agent.ProbeUdpPorts],
QuicPorts = [.. agent.ProbeUdpPorts],
LegacyUdpPorts = [.. agent.ProbeUdpPorts],
PublicKey = Convert.ToBase64String(agent.TlsPublicKey!),
});
}
@@ -43,7 +43,7 @@ public sealed class MoShiroEnrollmentHook : IEnrollmentHook
// 注册请求带了 public_host 就用它;否则来源地址是公网地址时用来源地址(私网或隧道出口不用,等 hello 里的本机地址)
string host = !string.IsNullOrWhiteSpace(meta.PublicHost) ? meta.PublicHost
: System.Net.IPAddress.TryParse(context.RemoteIp, out System.Net.IPAddress? remote) && PublicAddress.IsPublicIpv4(remote) ? remote.ToString() : "";
await _agents.ApplyEnrollmentAsync(node.Id, new AgentEnrollment(host, meta.ProbeTcpPort, meta.ProbeUdpPorts, meta.Kinds), ct);
await _agents.ApplyEnrollmentAsync(node.Id, new AgentEnrollment(host, meta.ProbeTcpPort, meta.ProbeQuicPorts.Count > 0 ? meta.ProbeQuicPorts : meta.LegacyProbeUdpPorts ?? [7443], meta.Kinds), ct);
if (await _agents.GetAsync(node.Id, ct) is { } enrolled)
{
(_, string? observedV6) = AgentAddressPolicy.Choose(enrolled, [], context.RemoteIp);
@@ -70,10 +70,12 @@ public sealed class MoShiroNodeEventSink : INodeEventSink
private readonly ScheduleService _schedule;
private readonly IProbeStore _probes;
private readonly NodeOwnerService _owners;
private readonly ProbeQuicRuntime _quicRuntime;
public MoShiroNodeEventSink(IAgentStore agents, AgentRuntime runtime, IEventStore events, LiveHub live, AgentService service, NodeInfoService nodes, ReleaseService releases, ScheduleService schedule, IProbeStore probes, NodeOwnerService owners)
public MoShiroNodeEventSink(IAgentStore agents, AgentRuntime runtime, IEventStore events, LiveHub live, AgentService service, NodeInfoService nodes, ReleaseService releases, ScheduleService schedule, IProbeStore probes, NodeOwnerService owners, ProbeQuicRuntime quicRuntime)
{
_owners = owners;
_quicRuntime = quicRuntime;
_releases = releases;
_schedule = schedule;
_probes = probes;
@@ -168,12 +170,17 @@ public sealed class MoShiroNodeEventSink : INodeEventSink
case NodeEventKind.Offline:
_live.Publish(new NodeSignal("probe", id.Value, false, nodeEvent.AtMs));
_releases.RecordStatus(id, null);
_quicRuntime.MarkProbeOffline(id.Value, nodeEvent.AtMs);
break;
case NodeEventKind.Heartbeat when nodeEvent.Extra is { } extra:
try
{
ProbeHeartbeatExtra? heartbeat = extra.Deserialize(MoShiroJsonContext.Default.ProbeHeartbeatExtra);
_releases.RecordStatus(id, heartbeat?.Update);
if (heartbeat?.QuicLinks is { } quicLinks)
{
_quicRuntime.Replace(id.Value, nodeEvent.AtMs, quicLinks);
}
if (heartbeat?.Ipv6 is { } ipv6)
{
await _probes.SetIpv6DiagnosticsAsync(id.Value, JsonSerializer.Serialize(ipv6, MoShiroJsonContext.Default.Ipv6Diagnostic), ct);
@@ -236,7 +243,8 @@ public sealed class MoShiroNodeEventSink : INodeEventSink
AgentRecord? before = await _agents.GetAsync(id, ct);
string? limitsJson = meta.LocalLimits is null ? null : JsonSerializer.Serialize(meta.LocalLimits, LocalLimitsJson.Default.LocalLimits);
string? ipv6Json = meta.Ipv6 is null ? null : JsonSerializer.Serialize(meta.Ipv6, MoShiroJsonContext.Default.Ipv6Diagnostic);
await _agents.ApplyHelloAsync(id, new AgentHello(kinds, hello.NodeVersion, limitsJson, spki is { Length: 32 } ? spki : null, meta.ProbeTcpPort, meta.ProbeUdpPorts, meta.Ipv6Supported, ipv6Json, meta.Addresses, tlsKey), ct);
IReadOnlyList<int>? quicPorts = meta.ProbeQuicPorts is { Count: > 0 } ? meta.ProbeQuicPorts : meta.LegacyProbeUdpPorts;
await _agents.ApplyHelloAsync(id, new AgentHello(kinds, hello.NodeVersion, limitsJson, spki is { Length: 32 } ? spki : null, meta.ProbeTcpPort, quicPorts, meta.Ipv6Supported, ipv6Json, meta.Addresses, tlsKey), ct);
// 自动识别对外地址:Agent 上报的公网地址;手填的不动
if (meta.Addresses is not null && await _agents.GetAsync(id, ct) is { } reported)
@@ -0,0 +1,56 @@
using System.Collections.Concurrent;
using MoShiro.Protocol;
namespace MoShiro.Backend.Application;
public sealed record ProbeQuicLinkSnapshot(ProbeQuicLinkHealth Health, long AtMs);
/// <summary>Latest per-line QUIC health reported by connected Probes. Volatile runtime state, not historical metrics.</summary>
public sealed class ProbeQuicRuntime
{
private readonly ConcurrentDictionary<(string Probe, string Agent, string IpVersion), ProbeQuicLinkSnapshot> _links = new();
public ProbeQuicLinkSnapshot? Get(string probeId, string agentId, string ipVersion) =>
_links.TryGetValue((probeId, agentId, ipVersion), out ProbeQuicLinkSnapshot? snapshot) ? snapshot : null;
public void Replace(string probeId, long atMs, IReadOnlyList<ProbeQuicLinkHealth> links)
{
var current = new HashSet<(string Probe, string Agent, string IpVersion)>();
foreach (ProbeQuicLinkHealth link in links)
{
var key = (probeId, link.AgentId, link.IpVersion);
current.Add(key);
_links[key] = new ProbeQuicLinkSnapshot(link, atMs);
}
foreach ((string Probe, string Agent, string IpVersion) key in _links.Keys.Where(k => k.Probe == probeId && !current.Contains(k)))
{
_links.TryRemove(key, out _);
}
}
public void MarkProbeOffline(string probeId, long atMs)
{
foreach (KeyValuePair<(string Probe, string Agent, string IpVersion), ProbeQuicLinkSnapshot> pair in _links.ToArray().Where(p => p.Key.Probe == probeId))
{
ProbeQuicLinkHealth old = pair.Value.Health;
_links[pair.Key] = new ProbeQuicLinkSnapshot(new ProbeQuicLinkHealth
{
AgentId = old.AgentId,
IpVersion = old.IpVersion,
Connected = false,
HandshakeAttempts = old.HandshakeAttempts,
HandshakeSuccesses = old.HandshakeSuccesses,
HandshakeFailures = old.HandshakeFailures,
HandshakeTimeouts = old.HandshakeTimeouts,
HandshakeP95Ms = old.HandshakeP95Ms,
HandshakeSampleCount = old.HandshakeSampleCount,
TransportPacketLossRate = old.TransportPacketLossRate,
TransportPacketsSent = old.TransportPacketsSent,
TransportCongestionEvents = old.TransportCongestionEvents,
TransportEcnCongestionEvents = old.TransportEcnCongestionEvents,
PathMtu = old.PathMtu,
}, atMs);
}
}
}
@@ -16,6 +16,7 @@ public static class ApplicationServiceCollectionExtensions
services.AddHostedService<StartupTasks>();
services.AddSingleton<AgentRuntime>();
services.AddSingleton<ProbeQuicRuntime>();
services.AddSingleton<LiveHub>();
services.AddSingleton<HotLayer>();
services.AddSingleton<BackendCounters>();
@@ -25,7 +25,7 @@ public readonly record struct LineKey(NodeId Agent, ProbeId Probe, string IpVers
}
/// <summary>一条线路的展示视图:Agent 的状态、Probe 记录,以及这一对现在是否连着。</summary>
public sealed record LineView(AgentView Agent, ProbeRecord Probe, string IpVersion = IpVersions.V4)
public sealed record LineView(AgentView Agent, ProbeRecord Probe, string IpVersion = IpVersions.V4, ProbeQuicLinkSnapshot? QuicHealth = null)
{
public LineKey Key => new(Agent.Agent.Id, new ProbeId(Probe.Id), IpVersion);
@@ -40,12 +40,14 @@ public sealed class LineService
private readonly AgentService _agents;
private readonly IProbeStore _probes;
private readonly LinkService _links;
private readonly ProbeQuicRuntime _quicRuntime;
public LineService(AgentService agents, IProbeStore probes, LinkService links)
public LineService(AgentService agents, IProbeStore probes, LinkService links, ProbeQuicRuntime quicRuntime)
{
_links = links;
_agents = agents;
_probes = probes;
_quicRuntime = quicRuntime;
}
/// <summary>所有线路:每个 Agent 和每个<b>未吊销</b>的 Probe 配成一对,只留下线路开着的(见 <see cref="LinkService"/>);关掉的线路历史数据还在,按 id 仍然查得到。</summary>
@@ -59,10 +61,10 @@ public sealed class LineService
{
foreach (ProbeRecord probe in probes.Where(p => links.IsLinked(p.Id, agent.Agent.Id.Value)))
{
result.Add(new LineView(agent, probe));
result.Add(new LineView(agent, probe, IpVersions.V4, _quicRuntime.Get(probe.Id, agent.Agent.Id.Value, IpVersions.V4)));
if (agent.Agent.Ipv6Host.Length > 0)
{
result.Add(new LineView(agent, probe, IpVersions.V6));
result.Add(new LineView(agent, probe, IpVersions.V6, _quicRuntime.Get(probe.Id, agent.Agent.Id.Value, IpVersions.V6)));
}
}
}
@@ -117,6 +119,6 @@ public sealed class LineService
probe = active[0];
}
return Result<LineView>.Ok(new LineView(agent, probe, ipVersion));
return Result<LineView>.Ok(new LineView(agent, probe, ipVersion, _quicRuntime.Get(probe.Id, agent.Agent.Id.Value, ipVersion)));
}
}
@@ -240,7 +240,7 @@ public sealed class TestService
}
// 没有指定端口的 UDP 阶段使用 Agent 上报的数据端口(目录里的默认值只是出厂默认,每个 Agent 可以改)
if (!merged.ContainsKey("port") && kind.Blocks.Transport == "udp" && kind.Params.Any(s => s.Name == "port") && agent.ProbeUdpPorts.Count > 0)
if (!merged.ContainsKey("port") && kind.Blocks.Transport == "quic_datagram" && kind.Params.Any(s => s.Name == "port") && agent.ProbeUdpPorts.Count > 0)
{
merged["port"] = JsonSerializer.SerializeToElement(agent.ProbeUdpPorts[0], PlanPortJson.Default.Int32);
}
@@ -55,9 +55,14 @@ public sealed class BackendUpdates : IHostedService
return "release";
}
return BuildInfo.Tag is { } tag && (tag.StartsWith("dev-", StringComparison.Ordinal) || tag.StartsWith("main-", StringComparison.Ordinal)) ? "dev" : "release";
return IsDevTag(BuildInfo.Tag) ? "dev" : "release";
}
/// <summary>开发构建的 tag:新格式 v0.4.1-dev.12-abc1234(含 -dev.),以及旧的 dev-&lt;sha&gt;、main-&lt;sha&gt;。</summary>
public static bool IsDevTag(string? tag) =>
tag is not null
&& (tag.Contains("-dev.", StringComparison.Ordinal) || tag.StartsWith("dev-", StringComparison.Ordinal) || tag.StartsWith("main-", StringComparison.Ordinal));
public async ValueTask<BackendUpdateView> GetAsync(CancellationToken ct)
{
string channel = ChannelOf(_options);
+8 -1
View File
@@ -1,3 +1,10 @@
using MoShiro.Backend;
await BackendHost.Build(args).RunAsync();
WebApplication app = BackendHost.Build(args);
// 启动时打一行构建信息:版本、tag、提交、构建时间(和 Agent、Probe 启动时打的是同一组)
if (app.Logger.IsEnabled(LogLevel.Information))
{
app.Logger.LogInformation("moshiro backend starting: version {BuildInfo}", MoShiro.Protocol.BuildInfo.Summary);
}
await app.RunAsync();
@@ -40,8 +40,8 @@ public sealed class ModelAndTestEvaluatorTests
[Fact]
public void BaseModel_PerfectLine_ScoresHigh_AndBadLine_ScoresLow()
{
double good = new BaseModel().Score(View(("rtt.p95", 25), ("loss.total", 0), ("jitter.down.p95", 1), ("jitter.up.p95", 1), ("loss.run_max", 1))).Score!.Value;
double bad = new BaseModel().Score(View(("rtt.p95", 400), ("loss.total", 12), ("jitter.down.p95", 90), ("jitter.up.p95", 90), ("loss.run_max", 40))).Score!.Value;
double good = new BaseModel().Score(View(("rtt.p95", 25), ("loss.total", 0), ("jitter.down.p95", 1), ("jitter.up.p95", 1), ("loss.run.up_max", 1), ("loss.run.down_max", 1))).Score!.Value;
double bad = new BaseModel().Score(View(("rtt.p95", 400), ("loss.total", 12), ("jitter.down.p95", 90), ("jitter.up.p95", 90), ("loss.run.up_max", 40), ("loss.run.down_max", 40))).Score!.Value;
Assert.True(good > 95);
Assert.True(bad < 10);
}
@@ -83,7 +83,7 @@ public sealed class ModelAndTestEvaluatorTests
var ranges = new Dictionary<string, double[]>
{
["rtt.p95"] = [10, 30, 80, 150, 300, 600], ["loss.total"] = [0, 0.5, 1, 3, 10, 30],
["jitter.down.p95"] = [0, 2, 8, 20, 80, 200], ["jitter.up.p95"] = [0, 2, 8, 20, 80, 200], ["loss.run_max"] = [1, 3, 10, 30, 60],
["jitter.down.p95"] = [0, 2, 8, 20, 80, 200], ["jitter.up.p95"] = [0, 2, 8, 20, 80, 200], ["loss.run.up_max"] = [1, 3, 10, 30, 60], ["loss.run.down_max"] = [1, 3, 10, 30, 60],
["tcp.rtt.p95"] = [10, 40, 100, 200, 400, 800], ["tcp.stall.per_min"] = [0, 1, 5, 20, 60, 200], ["tcp.stall_max"] = [50, 150, 500, 2000, 5000],
["tcp.retrans"] = [0, 0.1, 1, 3, 10, 30], ["connect.tls_ms.p95"] = [50, 150, 400, 1000, 3000], ["hold.resets.per_h"] = [0, 1, 4, 12, 30],
["connect.success"] = [100, 99, 95, 80, 50], ["tcp.bulk.peak.max"] = [800, 400, 100, 10, 1], ["hy2.sustainable_mbps"] = [100, 50, 25, 10, 0],
@@ -90,6 +90,28 @@ public sealed class OpTests
Assert.Equal(2, ((RunLengthAcc)EvaluateOneBucket(chunk, "loss.run").Items["loss.run"]).Max);
}
[Fact]
public void LossRuns_AreSeparatedByDirection_AndDownstreamExcludesRequestsLostUpstream()
{
bool[] upLost = [false, true, true, false, false, false, true, true];
bool[] downLost = [false, false, true, true, false, true, false, true];
RawChunk chunk = ChunkFactory.Echo(s_agent, "udp_echo", T0, upLost.Length, 100,
i => new EchoPacket(upLost[i], downLost[i], 20_000, 20_000));
MetricSet set = EvaluateOneBucket(chunk, "loss.run.up", "loss.run.down");
Assert.Equal(2, ((MaxAcc)set.Items["loss.run.up"]).Result);
Assert.Equal(1, ((MaxAcc)set.Items["loss.run.down"]).Result);
}
[Fact]
public void LossRuns_NoLoss_ExportsZeroForBothDirections()
{
RawChunk chunk = ChunkFactory.UdpEchoClean(s_agent, T0, count: 8);
MetricSet set = EvaluateOneBucket(chunk, "loss.run.up", "loss.run.down");
Assert.Equal(0, ((MaxAcc)set.Items["loss.run.up"]).Result);
Assert.Equal(0, ((MaxAcc)set.Items["loss.run.down"]).Result);
}
[Fact]
public void Reorder_CountsLateArrivals_AndIgnoresDuplicates()
{
@@ -21,7 +21,7 @@ public sealed class AgentEndToEndTests
AgentHarness agent = await AgentHarness.StartAsync(_backend);
await agent.OverrideLineTasksAsync(rateHz: 30);
ProbeSimulator probe = await agent.ConnectProbeAsync();
LineTaskSet set = await probe.WaitForTasksAsync(s => s.Tasks.Any(t => t.Kind == "udp_echo" && t.Port == agent.App.UdpPorts[0]));
LineTaskSet set = await probe.WaitForTasksAsync(s => s.Tasks.Any(t => t.Kind == "udp_echo" && t.Port == agent.App.QuicPorts[0]));
return (agent, probe, set.Tasks.Single(t => t.Kind == "udp_echo"));
}
@@ -159,8 +159,8 @@ public sealed class AgentEndToEndTests
// The probe process restarts: a new session with its own streams; sequence numbers start from 0 again
await using ProbeSimulator second = first.Restarted();
await second.ConnectAsync("127.0.0.1", agent.App.ControlPort, agent.TlsPublicKey, agent.App.UdpPorts[0], agent.NodeId);
LineTaskSet set = await second.WaitForTasksAsync(s => s.Tasks.Any(t => t.Kind == "udp_echo" && t.Port == agent.App.UdpPorts[0]));
await second.ConnectAsync("127.0.0.1", agent.App.ControlPort, agent.TlsPublicKey, agent.App.QuicPorts[0], agent.NodeId);
LineTaskSet set = await second.WaitForTasksAsync(s => s.Tasks.Any(t => t.Kind == "udp_echo" && t.Port == agent.App.QuicPorts[0]));
LineTask udp2 = set.Tasks.Single(t => t.Kind == "udp_echo");
Assert.NotEqual(udp.StreamId, udp2.StreamId);
@@ -221,11 +221,11 @@ public sealed class AgentEndToEndTests
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
await agent.CreateProbeAsync();
await using var stranger = new ProbeSimulator("prb_stranger", Fleet.Identity.NodeIdentity.Generate());
await Assert.ThrowsAnyAsync<Exception>(() => stranger.ConnectAsync("127.0.0.1", agent.App.ControlPort, agent.TlsPublicKey, agent.App.UdpPorts[0], agent.NodeId));
await Assert.ThrowsAnyAsync<Exception>(() => stranger.ConnectAsync("127.0.0.1", agent.App.ControlPort, agent.TlsPublicKey, agent.App.QuicPorts[0], agent.NodeId));
// 证书公钥不对(中间人)也不会连上
await using ProbeSimulator probe = await agent.CreateProbeAsync();
await Assert.ThrowsAnyAsync<Exception>(() => probe.ConnectAsync("127.0.0.1", agent.App.ControlPort, Fleet.Identity.NodeIdentity.Generate().PublicKeySpki, agent.App.UdpPorts[0], agent.NodeId));
await Assert.ThrowsAnyAsync<Exception>(() => probe.ConnectAsync("127.0.0.1", agent.App.ControlPort, Fleet.Identity.NodeIdentity.Generate().PublicKeySpki, agent.App.QuicPorts[0], agent.NodeId));
}
[Fact]
@@ -257,7 +257,9 @@ public sealed class AgentEndToEndTests
}
}
// 慢的端到端测试(真实时间的阶梯、要等测量块出来):CI 里单独一批跑(Category=Slow),不拦着出镜像,见 ci/verify-backend.sh
[Fact]
[Trait("Category", "Slow")]
public async Task Hy2Ladder_EndToEnd_AgentSendsTheTrain_ProbeReportsReceipts_AndTheTestSucceeds()
{
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
@@ -333,7 +335,7 @@ public sealed class AgentEndToEndTests
await using AgentHarness agent = await AgentHarness.StartAsync(_backend, config => config.Limits.MaxTotalMbps = 20);
await agent.OverrideLineTasksAsync();
var phase = new TestPhase { Kind = "udp_train", DurationSeconds = 1, Params = { ["port"] = System.Text.Json.JsonSerializer.SerializeToElement(agent.App.UdpPorts[0]) } };
var phase = new TestPhase { Kind = "udp_train", DurationSeconds = 1, Params = { ["port"] = System.Text.Json.JsonSerializer.SerializeToElement(agent.App.QuicPorts[0]) } };
var run = new TestRunParams { TestId = new TestId("tst_x"), ProbeId = new ProbeId("prb_nobody"), Plan = "x", Phases = [phase] };
Assert.Equal("probe_offline", agent.App.Tests.Start(run).Error.Code.Value);
@@ -58,7 +58,7 @@ public sealed class AgentHarness : IAsyncDisposable
var config = new AgentConfig
{
Fleet = { Server = backend.BaseAddress.ToString(), IdentityDir = directory },
Listen = { Bind = "127.0.0.1", ControlPort = 0, UdpPorts = [0], TcpPorts = [0] },
Listen = { Bind = "127.0.0.1", ControlPort = 0, QuicPorts = [0], TcpPorts = [0] },
};
configure?.Invoke(config);
LoadedNodeIdentity identity = LoadedNodeIdentity.Load(directory);
@@ -87,7 +87,7 @@ public sealed class AgentHarness : IAsyncDisposable
public async Task<ProbeSimulator> ConnectProbeAsync()
{
ProbeSimulator probe = await CreateProbeAsync();
await probe.ConnectAsync("127.0.0.1", App.ControlPort, TlsPublicKey, App.UdpPorts[0], NodeId);
await probe.ConnectAsync("127.0.0.1", App.ControlPort, TlsPublicKey, App.QuicPorts[0], NodeId);
return probe;
}
@@ -104,7 +104,7 @@ public sealed class AgentHarness : IAsyncDisposable
}
string kind = task["kind"]!.GetValue<string>();
var parameters = new JsonObject { ["port"] = kind == "udp_echo" ? App.UdpPorts[0] : App.TcpPorts[0] };
var parameters = new JsonObject { ["port"] = kind == "udp_echo" ? App.QuicPorts[0] : App.TcpPorts[0] };
// 默认的 connect 模板在这些用例里不需要(也占预算):对这个 Agent 关掉它
bool enabled = kind != "connect";
if (rateHz is { } hz && kind != "connect")
@@ -30,6 +30,18 @@ public sealed class BackendUpdateTests : IAsyncLifetime
Assert.Equal(expected, BackendUpdates.ChannelOf(new BackendOptions { UpdateChannel = "auto", ImageNamespace = imageNamespace }));
}
[Theory]
[InlineData("v0.4.1-dev.12-044ce0c", true)]
[InlineData("dev-725fe17a7de8", true)]
[InlineData("main-725fe17a7d", true)]
[InlineData("v0.4.0", false)]
[InlineData("v0.4.1-rc.1", false)]
[InlineData(null, false)]
public void IsDevTag_RecognisesDevelopmentBuilds(string? tag, bool expected)
{
Assert.Equal(expected, BackendUpdates.IsDevTag(tag));
}
private async Task<HttpResponseMessage> PublishAsync(long seq, string channel = "dev")
{
string manifest = $$"""{"component":"backend","channel":"{{channel}}","image":"registry.example/moshiro-{{channel}}/moshiro-backend","digest":"{{Digest}}","tag":"{{channel}}-{{seq}}","version":"0.1.1","seq":{{seq}}}""";
@@ -221,6 +221,7 @@ public sealed class HostStatusTests
}
[Fact]
[Trait("Category", "Slow")]
public async Task AProbeThatTurnsReportingOff_SendsNothing()
{
await _backend.SiteMustBePublicAsync();
@@ -9,7 +9,9 @@ using MoShiro.TestSupport;
namespace MoShiro.Backend.Tests;
/// <summary>真实双栈回环:同一身份的两种会话、原始块、查询与测试结果必须各自独立。</summary>
// 慢的端到端测试(真实时间的阶梯、要等测量块出来):CI 里单独一批跑(Category=Slow),不拦着出镜像,见 ci/verify-backend.sh
[Collection("probe-e2e")]
[Trait("Category", "Slow")]
public sealed class Ipv6Tests
{
private readonly BackendFixture _backend;
@@ -67,6 +67,7 @@ public sealed class OnDemandTestsTests
}
[Fact]
[Trait("Category", "Slow")]
public async Task Hy2Ladder_RunsEndToEnd_AndProducesAResultWithScores()
{
await using AgentSimulator agent = await AgentWithProbeAsync();
@@ -4,7 +4,7 @@ using System.Text.RegularExpressions;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 对照 docs/api/api-guidelines.md 检查运行时生成的 OpenAPI 文档:operationId、路径风格、错误响应格式、字段命名。
/// 对照 docs/api/api.md 第 7 节 检查运行时生成的 OpenAPI 文档:operationId、路径风格、错误响应格式、字段命名。
/// 新增端点违反规范时,这里会直接失败(等价于文档里的 <c>eng/check-openapi</c>)。
/// </summary>
[Collection("backend")]
@@ -179,7 +179,9 @@ public sealed class OperationsAndSecurityTests
Assert.Equal("fleet_protocol_violation", (await agent.RequestRawAsync("fleet.exec")).Error!.Data!.Code);
}
// 慢的端到端测试(真实时间的阶梯、要等测量块出来):CI 里单独一批跑(Category=Slow),不拦着出镜像,见 ci/verify-backend.sh
[Fact]
[Trait("Category", "Slow")]
public async Task ARepeatOffender_IsDisconnected_AndANodeCannotInitiateWhatOnlyTheBackendMay()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
@@ -3,7 +3,9 @@ using System.Text.Json.Nodes;
namespace MoShiro.Backend.Tests;
/// <summary>真实的后端 + 真实的 Agent + 真实的 Probe(都在进程内,回环网络)。</summary>
// 慢的端到端测试(真实时间的阶梯、要等测量块出来):CI 里单独一批跑(Category=Slow),不拦着出镜像,见 ci/verify-backend.sh
[Collection("probe-e2e")]
[Trait("Category", "Slow")]
public sealed class ProbeEndToEndTests
{
private readonly BackendFixture _backend;
@@ -97,6 +97,7 @@ public sealed class AgentSimulator : IAsyncDisposable
["meta"] = new JsonObject
{
["probe_tcp_port"] = 7443,
["probe_quic_ports"] = new JsonArray(7443),
["probe_udp_ports"] = new JsonArray(7443),
["public_host"] = publicHost,
["kinds"] = new JsonArray((kinds ?? ["udp_echo", "tcp_echo", "udp_train", "connect", "hold", "tcp_bulk", "icmp_echo"]).Select(k => (JsonNode?)JsonValue.Create(k)).ToArray()),
@@ -154,7 +155,8 @@ public sealed class AgentSimulator : IAsyncDisposable
TlsPublicKey = Convert.ToBase64String(_tlsKey.PublicKeySpki),
Addresses = Addresses is null ? null : [.. Addresses],
ProbeTcpPort = 7443,
ProbeUdpPorts = [7443],
ProbeQuicPorts = [7443],
LegacyProbeUdpPorts = [7443],
LocalLimits = new LocalLimits { MaxTotalMbps = 200, MaxTotalPps = 20_000, MaxConcurrentTests = 1 },
},
MoShiroJsonContext.Default.AgentHelloMeta);
@@ -4,6 +4,7 @@
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\shared\MoShiro.Protocol\MoShiro.Protocol.csproj" />
<ProjectReference Include="..\..\shared\MoShiro.Measurement\MoShiro.Measurement.csproj" />
<ProjectReference Include="..\..\fleet\src\Fleet.Identity\Fleet.Identity.csproj" />
<ProjectReference Include="..\..\fleet\src\Fleet.Link\Fleet.Link.csproj" />
</ItemGroup>
@@ -8,13 +8,14 @@ using Fleet.Core;
using Fleet.Identity;
using Fleet.Link;
using Fleet.Rpc;
using MoShiro.Measurement;
using MoShiro.Protocol;
namespace MoShiro.TestSupport;
/// <summary>
/// 假 Probe:通过真实的点对点握手(TLS 固定证书 + 签名 + ECDH)连接真实的 Agent,接收 <c>line_task_set</c>,
/// 在自己的 UDP 套接字上发回显包、接收单向数据流、估计时钟偏移,并把 Probe 一侧的列块经发件箱通道 17 交给 Agent。
/// 通过 QUIC Datagram 发回显包、接收单向数据流、估计时钟偏移,并把 Probe 一侧的列块经发件箱通道 17 交给 Agent。
/// 它实现 Probe 的协议职责,是将来真实 Probe 的参照。
/// </summary>
public sealed class ProbeSimulator : IAsyncDisposable
@@ -30,8 +31,9 @@ public sealed class ProbeSimulator : IAsyncDisposable
private readonly CancellationTokenSource _stop = new();
private NodeId _agentId;
private RpcSession<ProbeSimulator>? _rpc;
private UdpClient? _udp;
private IPEndPoint? _agentUdp;
private QuicDatagramTransport? _quic;
private DataSession? _dataSession;
private IPEndPoint? _agentQuic;
private PeerSessionInfo? _info;
private Task? _receiver;
private Task? _run;
@@ -72,13 +74,22 @@ public sealed class ProbeSimulator : IAsyncDisposable
(PeerSessionInfo info, TlsFrameTransport transport) = await PeerConnector.ConnectAsync(host, controlPort, agentPublicKey, _identity, ProbeId, "0.0.0-sim", 8 << 20, time ?? TimeProvider.System, ct);
_info = info;
_agentKey = agentPublicKey;
_udp = new UdpClient(new IPEndPoint(IPAddress.Loopback, 0));
_agentUdp = new IPEndPoint(IPAddress.Parse(host), udpPort);
_dataSession = new DataSession(info.Credentials.SessionIdValue, info.Credentials.Key);
_agentQuic = new IPEndPoint(IPAddress.Parse(host), udpPort);
_quic = new QuicDatagramTransport(
_dataSession,
new MonoClock(time ?? TimeProvider.System),
new DataPlaneCounters(),
new TestNodeLog(),
_agentQuic,
agentPublicKey,
0);
_rpc = new RpcSession<ProbeSimulator>(transport, _router, this, time ?? TimeProvider.System);
_rpc.OnBinary = (_, _) => ValueTask.CompletedTask;
_run = _rpc.RunAsync(_stop.Token);
_receiver = Task.Run(ReceiveLoopAsync, _stop.Token);
_receiver = _quic.RunAsync(_stop.Token);
await SyncClockAsync();
await _quic.ConnectAsync(ct);
}
public async Task<LineTaskSet> WaitForTasksAsync(Func<LineTaskSet, bool> predicate, TimeSpan? timeout = null)
@@ -123,19 +134,21 @@ public sealed class ProbeSimulator : IAsyncDisposable
int size = (int)task.Params["size"].GetInt64();
var collector = new EchoCollector();
_echoes[task.StreamId] = collector;
_quic!.Register(task.StreamId, OnDatagram);
long startUtc = TimeProvider.System.GetUtcNow().ToUnixTimeMilliseconds();
byte[] packet = new byte[size];
for (uint seq = 0; seq < count; seq++)
{
long t1 = NowUs();
collector.Sent[seq] = t1;
UdpPacket.Write(packet, new UdpHeader(UdpPacketType.EchoRequest, _info!.Credentials.SessionIdValue, task.StreamId, seq, t1), _info.Credentials.Key);
await _udp!.SendAsync(packet, packet.Length, _agentUdp!);
MeasurementDatagram.Write(packet, new MeasurementDatagramHeader(MeasurementDatagramType.EchoRequest, _info!.Credentials.SessionIdValue, task.StreamId, seq, t1), _info.Credentials.Key);
_quic.Send(_dataSession!, _agentQuic!, packet);
await Task.Delay(interval, ct);
}
await Task.Delay(300, ct); // 等最后的回复
_echoes.TryRemove(task.StreamId, out _);
_quic.Unregister(task.StreamId);
var msg = new StreamBuilder(new StreamSpec("msg", [Col.Seq("seq"), Col.U16("size"), Col.TsProbe("t1"), Col.TsProbe("t4", nullable: true)]));
foreach ((uint seq, long t1) in collector.Sent.OrderBy(p => p.Key))
@@ -191,22 +204,28 @@ public sealed class ProbeSimulator : IAsyncDisposable
}
/// <summary>发送任意字节到 Agent 的 UDP 端口(用来测试 Agent 对垃圾包的处理)。</summary>
public Task SendRawUdpAsync(byte[] datagram) => _udp!.SendAsync(datagram, datagram.Length, _agentUdp!);
public Task SendRawUdpAsync(byte[] datagram)
{
_quic!.Send(_dataSession!, _agentQuic!, datagram);
return Task.CompletedTask;
}
/// <summary>用伪造的密钥发一个回显包。</summary>
public Task SendForgedEchoAsync(LineTask task, uint seq, byte[]? key = null)
{
byte[] packet = new byte[64];
UdpPacket.Write(packet, new UdpHeader(UdpPacketType.EchoRequest, _info!.Credentials.SessionIdValue, task.StreamId, seq, NowUs()), key ?? new byte[32]);
return _udp!.SendAsync(packet, packet.Length, _agentUdp!);
MeasurementDatagram.Write(packet, new MeasurementDatagramHeader(MeasurementDatagramType.EchoRequest, _info!.Credentials.SessionIdValue, task.StreamId, seq, NowUs()), key ?? new byte[32]);
_quic!.Send(_dataSession!, _agentQuic!, packet);
return Task.CompletedTask;
}
/// <summary>重放:用同一个序号、同样的密钥再发一次。</summary>
public Task SendEchoAsync(LineTask task, uint seq)
{
byte[] packet = new byte[64];
UdpPacket.Write(packet, new UdpHeader(UdpPacketType.EchoRequest, _info!.Credentials.SessionIdValue, task.StreamId, seq, NowUs()), _info.Credentials.Key);
return _udp!.SendAsync(packet, packet.Length, _agentUdp!);
MeasurementDatagram.Write(packet, new MeasurementDatagramHeader(MeasurementDatagramType.EchoRequest, _info!.Credentials.SessionIdValue, task.StreamId, seq, NowUs()), _info.Credentials.Key);
_quic!.Send(_dataSession!, _agentQuic!, packet);
return Task.CompletedTask;
}
public int EchoRepliesSeen(uint streamId) => _echoes.TryGetValue(streamId, out EchoCollector? c) ? c.Received.Count : 0;
@@ -227,7 +246,7 @@ public sealed class ProbeSimulator : IAsyncDisposable
public async ValueTask DisposeAsync()
{
await _stop.CancelAsync();
_udp?.Dispose();
_quic?.Dispose();
if (_rpc is not null)
{
await _rpc.DisposeAsync();
@@ -277,41 +296,18 @@ public sealed class ProbeSimulator : IAsyncDisposable
return header;
}
private async Task ReceiveLoopAsync()
private void OnDatagram(in QuicDatagramDelivery delivery)
{
while (!_stop.IsCancellationRequested)
long now = delivery.ReceiveUs;
MeasurementDatagramHeader header = delivery.Header;
if (header.Type == MeasurementDatagramType.EchoReply && _echoes.TryGetValue(header.StreamId, out EchoCollector? echo))
{
UdpReceiveResult result;
try
{
result = await _udp!.ReceiveAsync(_stop.Token);
}
catch (Exception ex) when (ex is OperationCanceledException or ObjectDisposedException or SocketException)
{
if (_stop.IsCancellationRequested || ex is ObjectDisposedException)
{
return;
}
continue;
}
long now = NowUs();
byte[] data = result.Buffer;
if (!UdpPacket.TryParseHeader(data, out UdpHeader header) || !UdpPacket.VerifyMac(data, _info!.Credentials.Key))
{
continue;
}
if (header.Type == UdpPacketType.EchoReply && _echoes.TryGetValue(header.StreamId, out EchoCollector? echo))
{
echo.Received.TryAdd(header.Seq, now);
}
else if (header.Type == UdpPacketType.Data && _trains.TryGetValue(header.StreamId, out TrainCollector? train))
{
train.Received.TryAdd(header.Seq, now);
train.Size = data.Length;
}
echo.Received.TryAdd(header.Seq, now);
}
else if (header.Type == MeasurementDatagramType.Data && _trains.TryGetValue(header.StreamId, out TrainCollector? train))
{
train.Received.TryAdd(header.Seq, now);
train.Size = delivery.Payload.Length;
}
}
@@ -366,33 +362,39 @@ public sealed class ProbeSimulator : IAsyncDisposable
private readonly ConcurrentDictionary<uint, TestPhaseStartParams> _activePhases = new();
private async Task<bool> StartPhaseAsync(TestPhaseStartParams start, CancellationToken ct)
private Task<bool> StartPhaseAsync(TestPhaseStartParams start, CancellationToken ct)
{
Phases.Enqueue(start);
if (RefusePhases)
{
return false;
return Task.FromResult(false);
}
_activePhases[start.StreamId] = start;
if (start.Kind == "udp_train")
{
_trains[start.StreamId] = new TrainCollector { StartUtcMs = TimeProvider.System.GetUtcNow().ToUnixTimeMilliseconds() };
_quic!.Register(start.StreamId, OnDatagram);
// 先发几个探活包,让 Agent 学到(最新的)源地址,再回复 accepted
byte[] hello = new byte[64];
for (uint seq = 0; seq < 3; seq++)
{
UdpPacket.Write(hello, new UdpHeader(UdpPacketType.Data, _info!.Credentials.SessionIdValue, start.StreamId, seq, NowUs()), _info.Credentials.Key);
await _udp!.SendAsync(hello.AsMemory(), _agentUdp!, ct);
MeasurementDatagram.Write(hello, new MeasurementDatagramHeader(MeasurementDatagramType.Data, _info!.Credentials.SessionIdValue, start.StreamId, seq, NowUs()), _info.Credentials.Key);
_quic.Send(_dataSession!, _agentQuic!, hello);
}
}
return true;
_ = ct;
return Task.FromResult(true);
}
private async Task<int> EndPhaseAsync(TestPhaseEndParams end, CancellationToken ct)
{
_activePhases.TryRemove(end.StreamId, out TestPhaseStartParams? start);
if (start is not null)
{
_quic!.Unregister(start.StreamId);
}
if (start is null || DropFragments)
{
_trains.TryRemove(end.StreamId, out _);
@@ -456,4 +458,9 @@ public sealed class ProbeSimulator : IAsyncDisposable
public long StartUtcMs { get; init; }
}
private sealed class TestNodeLog : INodeLog
{
public void Write(NodeLogLevel level, string message, params (string Key, object? Value)[] fields) { }
}
}
@@ -1,13 +1,13 @@
#!/usr/bin/env bash
# Build and push one opt-in main Linux amd64 image to the dev repository in Zot.
# Build and push one opt-in Linux amd64 development image (main push with [CI]) to the moshiro-dev repository in Zot.
set -euo pipefail
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
echo "main amd64 builds require a main push with [CI] in the commit message" >&2
echo "development image builds require a main push with [CI] in the commit message" >&2
exit 2
}
component="${1:?usage: build-main-amd64.sh <backend|agent|probe>}"
component="${1:?usage: build-dev-image.sh <backend|agent|probe>}"
case "$component" in
backend|agent|probe) ;;
*) echo "unsupported component: $component" >&2; exit 2 ;;
@@ -22,20 +22,22 @@ esac
}
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
# 版本:v<下一个补丁版本>-dev.<自最近 tag 的提交数>-<提交前 7 位>(见 ci/version.sh);写进工作区的 Directory.Build.props,镜像标签、构建信息、签名清单都用它
source "$root/ci/version.sh"
moshiro_apply_version
registry="${ZOT_REGISTRY:-registry.oeo.one}"
repository="$registry/moshiro-dev/moshiro-$component"
work="$root/.release/main-amd64-build"
base_home="/home/justme/woodpecker/buildah-home/main-amd64-build-$component"
short_sha="${CI_COMMIT_SHA:0:12}"
dev_tag="dev-$short_sha"
work="$root/.release/dev-image-build"
base_home="/home/justme/woodpecker/buildah-home/dev-image-build-$component"
# 镜像标签不带 v(和正式版一致):latest 和 <版本>;构建信息和清单里的 tag 带 v
image_version="$MOSHIRO_VERSION"
mkdir -p "$work" "$base_home"
if [[ "$component" == backend ]]; then
export MOSHIRO_TAG="$dev_tag"
BUILDAH_HOME="$base_home" \
ZOT_REGISTRY="$registry" \
ZOT_IMAGE_REPOSITORY="$repository" \
IMAGE_TAGS="latest,$dev_tag" \
IMAGE_TAGS="latest,$image_version" \
IMAGE_PUBLISH=true \
bash "$root/ci/release-linux.sh" amd64 backend
exit 0
@@ -45,12 +47,12 @@ export HOME="$base_home"
export BUILDAH_HOME="$base_home"
export REGISTRY_AUTH_FILE="$work/$component-auth.json"
context="$work/$component"
local_image="localhost/moshiro-$component:main-amd64-check"
local_image="localhost/moshiro-$component:dev-amd64-check"
python3 "$root/ci/stage-node-image.py" "$component" "$context"
buildah bud --layers --network host --tls-verify=true --arch amd64 \
--build-arg APP_VERSION=main \
--build-arg "MOSHIRO_TAG=$dev_tag" --build-arg "MOSHIRO_COMMIT=${CI_COMMIT_SHA:0:10}" --build-arg "MOSHIRO_BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--build-arg "APP_VERSION=$image_version" \
--build-arg "MOSHIRO_TAG=$MOSHIRO_TAG" --build-arg "MOSHIRO_COMMIT=${CI_COMMIT_SHA:0:10}" --build-arg "MOSHIRO_BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
-f "$context/Containerfile" \
-t "$local_image" "$context"
@@ -67,7 +69,7 @@ for attempt in 1 2 3 4; do
done
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
for tag in latest "$dev_tag"; do
for tag in latest "$image_version"; do
pushed=false
for attempt in 1 2 3; do
if buildah push --tls-verify=true --digestfile "$work/$component.digest" "$local_image" "docker://$repository:$tag"; then
@@ -86,5 +88,5 @@ done
# 发布清单:给镜像的 digest 签名(私钥只在 CI 密钥里),放进镜像仓库(<仓库>-manifest:dev),面板在线时再登记一份加速。
# 没有配置签名密钥(例如 fork)就只构建镜像、不发清单。
if [[ -n "${RELEASE_SIGNING_KEY:-}" ]]; then
bash "$root/ci/publish-release-offer.sh" "$component" "$repository" "$(cat "$work/$component.digest")" "$dev_tag" dev "$work/$component-release"
bash "$root/ci/publish-release-offer.sh" "$component" "$repository" "$(cat "$work/$component.digest")" "$MOSHIRO_TAG" dev "$work/$component-release"
fi
+4 -4
View File
@@ -10,10 +10,10 @@ if (-not $env:GITEA_TOKEN) {
$version = $Matches[1]
$root = Split-Path -Parent $PSScriptRoot
$backend = Join-Path $root 'backend'
$projectVersion = [regex]::Match((Get-Content (Join-Path $backend 'Directory.Build.props') -Raw), '<Version>([^<]+)</Version>').Groups[1].Value
if ($version -ne $projectVersion) {
throw "Tag version $version does not match project version $projectVersion."
}
# 版本取自 tag:写进工作区的 Directory.Build.props(仓库里的 <Version> 只是占位,发版不用再提交 bump)
$propsPath = Join-Path $backend 'Directory.Build.props'
$props = Get-Content $propsPath -Raw
Set-Content -LiteralPath $propsPath -Value ([regex]::Replace($props, '<Version>[^<]*</Version>', "<Version>$version</Version>")) -NoNewline
$tarProject = Join-Path $PSScriptRoot 'MoShiro.Tar\MoShiro.Tar.csproj'
$tarTool = Join-Path $PSScriptRoot 'MoShiro.Tar\bin\Release\net10.0\MoShiro.Tar.dll'
+5 -5
View File
@@ -20,10 +20,10 @@ if (-not $env:CI_COMMIT_TAG -or $env:CI_COMMIT_TAG -notmatch '^v(\d+\.\d+\.\d+(?
$version = $Matches[1]
$root = Split-Path -Parent $PSScriptRoot
$backend = Join-Path $root 'backend'
$projectVersion = [regex]::Match((Get-Content (Join-Path $backend 'Directory.Build.props') -Raw), '<Version>([^<]+)</Version>').Groups[1].Value
if ($version -ne $projectVersion) {
throw "Tag version $version does not match project version $projectVersion."
}
# 版本取自 tag:写进工作区的 Directory.Build.props(仓库里的 <Version> 只是占位,发版不用再提交 bump)
$propsPath = Join-Path $backend 'Directory.Build.props'
$props = Get-Content $propsPath -Raw
Set-Content -LiteralPath $propsPath -Value ([regex]::Replace($props, '<Version>[^<]*</Version>', "<Version>$version</Version>")) -NoNewline
if ($RuntimeIdentifier -in @('win-x64', 'win-arm64')) {
$vcArchitecture = if ($RuntimeIdentifier -eq 'win-x64') { 'x64' } else { 'arm64' }
@@ -52,7 +52,7 @@ $buildRoot = Join-Path $root ".release\build\$RuntimeIdentifier-$Component"
$null = New-Item -ItemType Directory -Force -Path $output
# 构建信息:程序里显示的 tag、提交、构建时间
$env:MOSHIRO_TAG = if ($env:CI_COMMIT_TAG) { $env:CI_COMMIT_TAG } else { "main-$($env:CI_COMMIT_SHA.Substring(0, 10))" }
$env:MOSHIRO_TAG = $env:CI_COMMIT_TAG
$env:MOSHIRO_COMMIT = $env:CI_COMMIT_SHA.Substring(0, 10)
$env:MOSHIRO_BUILT_AT = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
+6 -16
View File
@@ -6,19 +6,16 @@ set -euo pipefail
arch="${1:?usage: release-linux.sh <amd64|arm64>}"
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
tag="${CI_COMMIT_TAG:-}"
if [[ -n "$tag" ]]; then
[[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || {
echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2
exit 2
}
version="${tag#v}"
else
if [[ -z "$tag" ]]; then
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
echo "untagged backend builds require a main push with [CI] in the commit message" >&2
exit 2
}
version=main
fi
# 版本:tag 发版就是 tag 的版本,开发构建是 v<下一个补丁版本>-dev.<N>-<提交>(ci/version.sh);写进工作区的 Directory.Build.props
source "$root/ci/version.sh"
moshiro_apply_version
version="$MOSHIRO_VERSION"
image_publish="${IMAGE_PUBLISH:-true}"
[[ "$image_publish" == true || "$image_publish" == false ]] || {
echo "IMAGE_PUBLISH must be true or false" >&2; exit 2;
@@ -34,12 +31,6 @@ case "$arch" in
*) echo "unsupported architecture: $arch" >&2; exit 2 ;;
esac
project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
if [[ -n "$tag" && "$version" != "$project_version" ]]; then
echo "tag version $version does not match project version $project_version" >&2
exit 1
fi
registry="${ZOT_REGISTRY:-registry.oeo.one}"
image="${ZOT_IMAGE_REPOSITORY:-$registry/moshiro/moshiro-backend}"
output="$root/.release/linux-$arch/backend"
@@ -47,8 +38,7 @@ build_root="$root/.release/build/linux-$arch-backend"
container_context="$root/.release/container/linux-$arch"
mkdir -p "$output" "$build_root" "$container_context/app" "$container_context/data"
# 构建信息:tag 发布用 tag,main 构建用 main-<提交>
export MOSHIRO_TAG="${MOSHIRO_TAG:-${tag:-main-${CI_COMMIT_SHA:0:10}}}"
# 构建信息:tag 发布用 tag,开发构建用 v<版本>-dev.<N>-<提交>(MOSHIRO_TAG 已由 ci/version.sh 设好)
export MOSHIRO_COMMIT="${MOSHIRO_COMMIT:-${CI_COMMIT_SHA:0:10}}"
export MOSHIRO_BUILT_AT="${MOSHIRO_BUILT_AT:-$(date -u +%Y-%m-%dT%H:%M:%SZ)}"
+2 -1
View File
@@ -14,7 +14,8 @@ repository="${2:?}"; digest="${3:?}"; tag="${4:?}"; channel="${5:?}"; outdir="${
[[ -n "${RELEASE_SIGNING_KEY:-}" ]] || { echo "RELEASE_SIGNING_KEY is required" >&2; exit 2; }
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
source "$root/ci/version.sh"
version="$MOSHIRO_VERSION"
commit="$(git -C "$root" rev-parse --short=10 HEAD 2>/dev/null || true)"
built_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
seq="$(date +%s)"
+3 -5
View File
@@ -15,11 +15,9 @@ tag="${CI_COMMIT_TAG:-}"
version="${tag#v}"
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
if [[ "$version" != "$project_version" ]]; then
echo "tag version $version does not match project version $project_version" >&2
exit 1
fi
# 版本取自 tag:写进工作区的 Directory.Build.props(仓库里的 <Version> 只是占位,发版不用再提交 bump)
source "$root/ci/version.sh"
moshiro_apply_version
gitea_base="${GITEA_BASE:-http://127.0.0.1:3100}"
gitea_base="${gitea_base%/}"
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env bash
# 后端测试,带增量编译:把后端源码同步到一个固定目录,在那里 dotnet build / test。
# 用法:verify-backend.sh [fast|slow]
# fast(默认):除了标了 Category=Slow 的以外的全部测试,约 20 秒,出镜像之前要过;
# slow:只跑标了 Category=Slow 的端到端测试(真实时间的阶梯、要等测量块出来,合起来一百来秒),和出镜像并行跑,失败照样红,但不拦着出镜像。
#
# 为什么:每条流水线的工作区都是新克隆的(路径、文件时间都是新的),MSBuild 没法增量,每次从零编译整个解决方案要两分钟。
# 固定目录里 obj/bin 留着,同步时只更新内容真的变了的文件(按内容比较,没变的文件时间不动),MSBuild 就只重编译受影响的项目;
# 改动小的时候整个测试 20–30 秒。
#
# 同一时间只允许一条流水线用这份缓存(flock);增量编译失败(缓存坏了)就清掉缓存从头编译一次,所以最坏也只是回到原来的速度。
set -euo pipefail
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
mode="${1:-fast}"
case "$mode" in
fast) filter=(--filter-not-trait "Category=Slow") ;;
slow) filter=(--filter-trait "Category=Slow") ;;
*) echo "usage: verify-backend.sh [fast|slow]" >&2; exit 2 ;;
esac
cache="${MOSHIRO_CI_CACHE:-/home/justme/woodpecker/cache/backend-tests}"
src="$cache/src"
mkdir -p "$cache"
# .NET 第一次运行的欢迎信息、遥测、开发证书,在每条流水线的临时 HOME 里都要白做一遍
export DOTNET_CLI_TELEMETRY_OPTOUT=1 DOTNET_NOLOGO=1 DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 DOTNET_GENERATE_ASPNET_CERTIFICATE=false
exec 9> "$cache/.lock"
flock -w 900 9 || { echo "could not get the backend test cache lock" >&2; exit 1; }
sync_sources() {
mkdir -p "$src"
local list
list="$(mktemp)"
# 构建和测试要用的:backend/、docs/(有的测试读协议文档)、packaging/(Backend.Api 把安装脚本、compose、Quadlet 单元和发布公钥当嵌入资源编进去)、根目录的几个文件(Directory.Packages.props、global.json、NuGet 配置)
(cd "$root" && git ls-files -z -- backend packaging docs Directory.Packages.props global.json 'NuGet.*' 'nuget.*') | tr '\0' '\n' | LC_ALL=C sort -u > "$list"
local changed=0 file
while IFS= read -r file; do
[[ -f "$root/$file" ]] || continue
if ! cmp -s "$root/$file" "$src/$file" 2>/dev/null; then
mkdir -p "$(dirname "$src/$file")"
cp "$root/$file" "$src/$file"
changed=$((changed + 1))
fi
done < "$list"
# 仓库里已经删掉的文件,缓存里也要删(构建产物 artifacts 不动)
local stale=0
while IFS= read -r file; do
[[ -n "$file" ]] || continue
rm -f "$src/$file"
stale=$((stale + 1))
done < <(cd "$src" && find backend packaging docs -type f -not -path 'backend/artifacts/*' -not -path '*/bin/*' -not -path '*/obj/*' 2>/dev/null | LC_ALL=C sort | LC_ALL=C comm -23 - <(grep -E '^(backend|packaging|docs)/' "$list"))
rm -f "$list"
echo "synced sources: $changed changed, $stale removed"
}
build() { (cd "$src/backend" && dotnet build MoShiro.slnx -c Release --nologo -v:q); }
sync_sources
if ! build; then
echo "incremental build failed; wiping the cache and building from scratch" >&2
rm -rf "$src"
sync_sources
build
fi
# 测试在缓存目录里跑完整的解决方案(不再重复编译)
log="$cache/last-test.log"
status=0
# 别的程序集里没有匹配的测试会因为「零个测试」返回 8,忽略它
(cd "$src/backend" && dotnet test --solution MoShiro.slnx -c Release --no-build "${filter[@]}" --ignore-exit-code 8) 2>&1 | tee "$log" || status=${PIPESTATUS[0]}
[[ "$status" == 0 ]] && exit 0
# 个别端到端测试会偶发失败(比如 Hy2 阶梯测试里探针掉线):失败的测试单独重跑(最多两次),重跑通过就当通过(日志里留着第一次的失败);
# 失败太多说明是真的坏了,不重跑
mapfile -t failed < <(grep -E '^(failed|失败) ' "$log" | awk '{print $2}' | awk -F. '{print $NF}' | sort -u)
if ((${#failed[@]} == 0 || ${#failed[@]} > 3)); then
echo "${#failed[@]} failed test(s); not retrying" >&2
exit "$status"
fi
for name in "${failed[@]}"; do
passed=false
# 最多重跑两次:这个端到端测试在 CI 主机上不稳,一次重跑不一定够
for attempt in 1 2; do
echo "retrying $name (attempt $attempt of 2)"
# 过滤要带通配符;别的程序集里没有匹配的测试,会因为「零个测试」返回 8,忽略它
if (cd "$src/backend" && dotnet test --solution MoShiro.slnx -c Release --no-build --filter-method "*$name*" --ignore-exit-code 8); then
passed=true
break
fi
done
[[ "$passed" == true ]] || { echo "$name failed again" >&2; exit 1; }
done
echo "the failed tests passed on retry"
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# 算出这次构建的版本号。用法:
# source ci/version.sh # 得到 MOSHIRO_VERSION(不带 v)和 MOSHIRO_TAG(带 v)
# bash ci/version.sh apply # 同上,并把 MOSHIRO_VERSION 写进工作区里的 backend/Directory.Build.props 和三个 web/*/package.json(只改工作区,不提交)
#
# 规则:
# - 打 tag 发版(CI_COMMIT_TAG 是 vMAJOR.MINOR.PATCH):版本就是 tag 的版本。仓库里的 <Version> 只是占位,发版不用再提交一次 bump;
# - 其余(main 上带 [CI] 的开发构建):vMAJOR.MINOR.(PATCH+1)-dev.<N>-<提交前 7 位>,
# 基准是最近一个 v* tag,N 是这个 tag 之后的提交数。开发版本按语义化版本排在下一个正式版之前,同一个提交算出来永远一样。
# 最近的 tag 要拿得到:CI 的克隆要带 tags(见 .woodpecker/dev-image.yml 的 clone 设置);没有任何 tag 时基准是 0.0.0。
_moshiro_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
_moshiro_compute_version() {
local tag="${CI_COMMIT_TAG:-}"
if [[ -n "$tag" ]]; then
[[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || {
echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2
return 2
}
MOSHIRO_VERSION="${tag#v}"
else
local sha="${CI_COMMIT_SHA:-$(git -C "$_moshiro_root" rev-parse HEAD)}"
local last base count major minor patch
last="$(git -C "$_moshiro_root" describe --tags --abbrev=0 --match 'v[0-9]*.[0-9]*.[0-9]*' "$sha" 2>/dev/null || true)"
if [[ -n "$last" ]]; then
base="${last#v}"
base="${base%%-*}"
count="$(git -C "$_moshiro_root" rev-list --count "$last..$sha")"
else
base="0.0.0"
count="$(git -C "$_moshiro_root" rev-list --count "$sha")"
fi
IFS=. read -r major minor patch <<< "$base"
MOSHIRO_VERSION="$major.$minor.$((patch + 1))-dev.$count-${sha:0:7}"
fi
MOSHIRO_TAG="v$MOSHIRO_VERSION"
export MOSHIRO_VERSION MOSHIRO_TAG
}
# 把算出来的版本写进工作区的 backend/Directory.Build.props(所有 .NET 项目、清单、镜像标签都从那里读版本)和三个 package.json
moshiro_apply_version() {
[[ -n "${MOSHIRO_VERSION:-}" ]] || _moshiro_compute_version
sed -i "s#<Version>[^<]*</Version>#<Version>$MOSHIRO_VERSION</Version>#" "$_moshiro_root/backend/Directory.Build.props"
# 前端在构建时从 web/package.json 读版本(web/build-info.mjs),其余两个工作区包的版本也一起对上
local manifest
for manifest in web/package.json web/display/package.json web/admin/package.json; do
sed -i -E '0,/"version": "[^"]*"/s//"version": "'"$MOSHIRO_VERSION"'"/' "$_moshiro_root/$manifest"
done
}
_moshiro_compute_version
if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
if [[ "${1:-}" == apply ]]; then moshiro_apply_version; fi
echo "$MOSHIRO_VERSION"
fi
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
# 前端构建和 CSS 检查,带依赖缓存:node_modules 按「锁文件 + package.json」的哈希缓存起来,同样的依赖不再 npm ci(约 20 秒);
# 展示端和管理端并行构建。缓存里放的是硬链接副本(cp -al),恢复几乎是瞬间;依赖变了才重新安装并更新缓存,只留最近 3 份。
set -euo pipefail
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cache="${MOSHIRO_CI_CACHE:-/home/justme/woodpecker/cache/web-node-modules}"
export NPM_CONFIG_CACHE="${NPM_CONFIG_CACHE:-/home/justme/.npm}"
web="$root/web"
dirs=(node_modules display/node_modules admin/node_modules)
mkdir -p "$cache"
key="$(cat "$web/package-lock.json" "$web/package.json" "$web/display/package.json" "$web/admin/package.json" | sha256sum | cut -c1-16)"
restore() {
[[ -d "$cache/$key/node_modules" ]] || return 1
local d
for d in "${dirs[@]}"; do
[[ -d "$cache/$key/$d" ]] || continue
mkdir -p "$(dirname "$web/$d")"
cp -al "$cache/$key/$d" "$web/$d" 2>/dev/null || cp -a "$cache/$key/$d" "$web/$d"
done
}
save() {
local tmp="$cache/$key.tmp.$$" d
rm -rf "$tmp"
for d in "${dirs[@]}"; do
[[ -d "$web/$d" ]] || continue
mkdir -p "$(dirname "$tmp/$d")"
cp -al "$web/$d" "$tmp/$d" 2>/dev/null || cp -a "$web/$d" "$tmp/$d"
done
rm -rf "$cache/$key"
mv "$tmp" "$cache/$key"
# 只留最近用过的 3 份
(cd "$cache" && ls -1t | grep -v '\.tmp\.' | tail -n +4 | xargs -r rm -rf)
}
if restore; then
echo "restored node_modules from the cache ($key)"
else
echo "no cache for $key; running npm ci"
rm -rf "$web/node_modules" "$web/display/node_modules" "$web/admin/node_modules"
npm ci --prefix "$web" --prefer-offline
save || echo "could not save the node_modules cache" >&2
fi
# 展示端和管理端并行构建,再检查 CSS
pids=()
(cd "$web" && timeout 600 npm run build -w display) & pids+=($!)
(cd "$web" && timeout 600 npm run build -w admin) & pids+=($!)
failed=0
for pid in "${pids[@]}"; do wait "$pid" || failed=1; done
[[ "$failed" == 0 ]] || { echo "web build failed" >&2; exit 1; }
npm run lint:css --prefix "$web"
+1 -1
View File
@@ -11,7 +11,7 @@
| 安全 | [security.md](security/security.md) | 零信任:威胁模型、缺口与状态、协议卡口、监听隐身、extension |
| | [access.md](security/access.md) | 访问控制与入网信任链:能力、API key、passkey、操作员集合、属主、安装的供应链 |
| | [enrollment.md](security/enrollment.md) | 注册与身份:Agent、Probe 的接入,节点本地硬上限 |
| 接口 | [api.md](api/api.md)、[api-guidelines.md](api/api-guidelines.md) | 端点说明;API 规范与约束 |
| 接口 | [api.md](api/api.md) | 端点说明;API 设计约束 |
| 运维 | [deployment.md](ops/deployment.md) | 构建、发布、CI、部署与更新、发布密钥 |
| 开发 | [project-layout.md](dev/project-layout.md)、[coding-style.md](dev/coding-style.md) | 仓库结构与依赖规则;代码风格 |
-141
View File
@@ -1,141 +0,0 @@
# API 设计约束
本文是所有 Backend REST 端点都必须遵守的**规则**;具体端点见 [api.md](api.md)。规则参考了 [Microsoft Azure REST API Guidelines](https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md) 和 RFC 9457,并根据本项目做了取舍。内部协议(Probe↔Agent、Agent↔Backend)的兼容规则见 [protocol.md](../design/protocol.md) 第 4 节。
## 1. 契约生成链路
```text
Minimal API 端点(TypedResults 返回类型 + 显式元数据)
│ dotnet build(Microsoft.Extensions.ApiDescription.Server)
▼
backend/openapi/v1.json(OpenAPI 3.1,提交到仓库)
│ CI:git diff --exit-code,未提交的变更会导致失败
▼
web:openapi-typescript 生成类型,调用方的类型错误会直接暴露
```
- 端点**必须**声明精确的返回类型,例如 `Results<Ok<AgentDto>, NotFound<ProblemDetails>>`,不能返回 `IResult` 或匿名对象,否则 OpenAPI 文档会不准确。
- 每个端点都必须设置 `WithName("agents_list")`(即 operationId,snake_case、`资源_动作`)、`WithSummary`、`WithTags`。
- 请求和响应 DTO 放在 `MoShiro.Backend.Api/Contracts/`,并注册到 Backend 的 JSON 源生成上下文。**不要直接暴露 `MoShiro.Protocol` 中的线协议类型**,因为 REST 与线协议的演进节奏不同。
- **结构强类型,指标弱类型**:资源结构(Agent、任务、测试等)必须是强类型 DTO;而**指标值和评分**统一使用字典 `metrics: { "<metric_id>": number }`、`scores: { "<model_id>": number }`,键的含义、单位、版本由 `/api/public/catalog/*` 给出。这样新增指标或模型不需要改 OpenAPI,也不需要改前端类型。
## 2. 版本与兼容性
- 当前版本为 v1,路径前缀为 `/api/public`、`/api/admin`、`/api/agent/v1`。出现破坏性变更时,新增 `/api/v2/public`、`/api/v2/admin` 等路径。v1 内只允许**非破坏性变更**:
| 非破坏性变更(v1 内允许) | 破坏性变更(需要新建 v2) |
| --- | --- |
| 新增端点 | 删除或重命名端点、字段 |
| 响应中新增字段 | 修改字段类型、单位或语义 |
| 新增可选的请求字段或查询参数 | 可选字段改为必填 |
| 枚举新增值 | 删除枚举值 |
| 放宽校验 | 收紧校验、修改默认值 |
| 新增错误码 | 修改已有错误码对应的 HTTP 状态 |
- 客户端必须忽略未知字段,也必须能处理未知的枚举值。
- 需要 v2 时,v1 和 v2 至少并存一个发布周期;OpenAPI 文档按版本分文件(`v1.json`、`v2.json`)。
## 3. URL 与方法
- **按调用方分层**:`/api/public`(只读展示)、`/api/admin`(管理)、`/api/agent`(节点)。同一个资源在不同层中可以有不同的 DTO,例如 public 层不暴露 `pubkey`、`local_limits`。
- 路径全部小写,资源名用复数名词,多个单词用 `-` 连接:`/api/admin/join-tokens`。
- 嵌套最多一层;子资源一旦有全局唯一 ID,就提供顶层路径(`/probes/{probe_id}`)。
- 查询参数和 JSON 字段使用 snake_case。
| 操作 | 方法 | 成功状态 | 说明 |
| --- | --- | --- | --- |
| 列表 | GET 集合 | 200 | 分页 |
| 读取 | GET 单个 | 200 | |
| 创建 | POST 集合 | 201 + `Location` | 响应体返回完整资源 |
| 部分更新 | PATCH | 200 | `application/merge-patch+json`(RFC 7396) |
| 删除 | DELETE | 204 | 资源不存在时同样返回 204(幂等) |
| 非 CRUD 动作 | POST `/{resource}/{id}/{verb}` | 200 或 204 | 例如 `revoke`、`rotate-token` |
- GET 请求不能产生副作用。PUT 不使用:配置类资源都通过 PATCH 修改。
## 4. 数据表示
| 类别 | 规则 | 示例 |
| --- | --- | --- |
| 时间点 | RFC 3339 UTC,以 `Z` 结尾,精确到毫秒。**唯一例外**:列式时序响应中的时间轴数组 `t` 使用 Unix 毫秒(见 [query.md](../design/query.md)) | `2026-09-27T10:00:00.000Z` |
| 时长 | 整数 + 单位后缀 | `interval_ms: 1000` |
| 延迟 | 带小数的毫秒值(指标字典中的单位见目录) | `rtt_p95_ms: 58.93` |
| 百分比 | 0–100 的数字 + `_pct` | `loss_pct: 0.5` |
| 计数 | 整数,不使用浮点数 | `attempts: 600` |
| ID | 带类型前缀的 ULID 字符串 | `prb_01J9Z...` |
| 枚举 | snake_case 字符串 | `"udp_echo"` |
| 缺失值 | 字段始终存在;未知值写 `null`,不省略字段 | `"last_seen_at": null` |
| 布尔值 | 使用 `is_`/`has_` 前缀,或者用意义明确的形容词 | `online`、`enabled` |
## 5. 错误
所有 4xx/5xx 响应都使用 `application/problem+json`(RFC 9457),并附加扩展字段:
```json
{
"type": "about:blank",
"title": "Validation failed",
"status": 400,
"code": "validation_failed",
"detail": "interval_ms must be >= 200",
"trace_id": "00-4bf92f...-01",
"errors": { "interval_ms": ["must be >= 200"] }
}
```
- `code` 取自共享的 `ErrorCode` 枚举,是客户端做判断的**唯一**依据;`title`/`detail` 只供人阅读,内容可能变化。
- `errors` 只在 400 `validation_failed` 中出现,键名是 JSON 字段路径。
- 5xx 响应不能泄露异常信息或调用栈,只提供 `trace_id`,用于在日志中定位问题。
| 状态码 | 使用场景 |
| --- | --- |
| 400 | 请求格式错误或校验失败 |
| 401 | 未认证 |
| 404 | 资源不存在 |
| 403 | 已认证但作用域不足 |
| 409 | 名称冲突、状态冲突(例如 `probe_offline`) |
| 410 | 资源曾经存在但已过保留期(例如 `raw_expired`) |
| 412 | `If-Match` 不匹配 |
| 413 | 请求体超过上限 |
| 422 | 语义上无法满足(例如 `range_too_large`) |
| 429 | 超过限流,需要带 `Retry-After` |
| 503 | 存储不可用,或正在关闭 |
## 6. 集合与查询
- 分页由服务端驱动:`?limit=`(默认 50,最大 500)和 `?cursor=`(不透明字符串);响应格式为 `{ "items": [...], "next_cursor": string | null }`。排序必须稳定(按 ID 或时间 + ID)。
- 时间范围参数统一使用 `from`/`to`,区间为左闭右开 `[from, to)`。缺省时 `to = now`,`from = to − 1h`。
- 所有查询都有上限(点数、时间跨度、结果条数);超出时返回 422 并在 `detail` 中给出可行的替代方案,不能悄悄截断。
## 7. 并发控制
- 配置类资源(Agent、Probe、任务)在响应中返回 `ETag`,PATCH/DELETE 支持 `If-Match`,不匹配时返回 412。没有带 `If-Match` 的请求按“最后写入者获胜”处理。
## 8. 安全
- 只提供 HTTPS。认证方式有两种:
- 浏览器使用 Cookie 会话:`HttpOnly; Secure; SameSite=Strict`;所有非 GET 请求都必须带 `X-CSRF-Token` 请求头;
- 脚本使用 `Authorization: Bearer <api_token>`,作用域为 `read` 或 `admin`。
- 凭据不能放在 URL 或查询参数中;SSE 依靠 Cookie 或同源令牌认证。
- 登录接口需要限流(例如每个 IP 每分钟 10 次),失败时统一返回 401,不区分“用户不存在”和“密码错误”。
- public 层根据站点设置和 `public_visible` 过滤数据:被隐藏的资源对匿名访客一律返回 404,而不是 403。
- 令牌明文只在创建或轮换的响应中出现一次;这类响应带 `Cache-Control: no-store`。
- 请求体上限为 64 KiB;查询接口有超时(默认 10 秒)。
## 9. 实时推送(SSE)
- 实时数据使用 `text/event-stream`。`event` 名称就是数据类型(`line`、`node`、`point`、`test`);`data` 是单行 JSON,遵守第 4 节的数据表示规则。
- 每 15 秒发送一行注释作为心跳。重连后服务端先推送当前快照,再推送增量,不补发历史。
- 订阅的主题数和连接数都有上限,超出时返回 429 `too_many_streams`。
- 推送的内容必须能由对应的 REST 查询重建:SSE 只负责“更快地获得变化”,不承载独有的数据。
## 10. 新增端点检查清单
1. 符合第 3 节的路径与方法规则,并有 operationId、summary 和 tag;
2. 返回类型精确(`TypedResults`),错误使用 problem+json,错误码来自 `ErrorCode`;
3. DTO 符合第 4 节的数据表示规则,并注册到 JSON 源生成上下文;
4. 列表支持分页,查询有上限;
5. 选择正确的层(public / admin / agent),public 层只暴露可以公开的字段;
6. 重新生成并提交 `openapi/v1.json`,在 PR 中说明是否属于破坏性变更;
7. 测试覆盖:成功路径、校验失败、404、鉴权失败、隐藏资源;
8. 同步更新 [api.md](api.md)。
+143 -1
View File
@@ -1,6 +1,6 @@
# Backend API 规范(v1)
本文列出 Backend 的全部对外接口。通用规则见 [api-guidelines.md](api-guidelines.md);OpenAPI 3.1 文档在运行时由 `/openapi/v1.json` 提供,构建时同时生成快照 `backend/openapi/v1.json`。两者冲突时,以 OpenAPI 为准。
本文列出 Backend 的全部对外接口。通用规则见本文第 7 节「设计约束」;OpenAPI 3.1 文档在运行时由 `/openapi/v1.json` 提供,构建时同时生成快照 `backend/openapi/v1.json`。两者冲突时,以 OpenAPI 为准。
## 1. 分层
@@ -191,3 +191,145 @@
| 422 | `range_too_large`、`invalid_step`、`unknown_metric`、`unknown_preset`、`too_many_metrics` |
| 429 | `too_many_requests`、`too_many_streams` |
| 503 | `storage_unavailable` |
## 7. 设计约束
本文是所有 Backend REST 端点都必须遵守的**规则**;具体端点见上面第 1–6 节。规则参考了 [Microsoft Azure REST API Guidelines](https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md) 和 RFC 9457,并根据本项目做了取舍。内部协议(Probe↔Agent、Agent↔Backend)的兼容规则见 [protocol.md](../design/protocol.md) 第 4 节。
### 7.1 契约生成链路
```text
Minimal API 端点(TypedResults 返回类型 + 显式元数据)
│ dotnet build(Microsoft.Extensions.ApiDescription.Server)
▼
backend/openapi/v1.json(OpenAPI 3.1,提交到仓库)
│ CI:git diff --exit-code,未提交的变更会导致失败
▼
web:openapi-typescript 生成类型,调用方的类型错误会直接暴露
```
- 端点**必须**声明精确的返回类型,例如 `Results<Ok<AgentDto>, NotFound<ProblemDetails>>`,不能返回 `IResult` 或匿名对象,否则 OpenAPI 文档会不准确。
- 每个端点都必须设置 `WithName("agents_list")`(即 operationId,snake_case、`资源_动作`)、`WithSummary`、`WithTags`。
- 请求和响应 DTO 放在 `MoShiro.Backend.Api/Contracts/`,并注册到 Backend 的 JSON 源生成上下文。**不要直接暴露 `MoShiro.Protocol` 中的线协议类型**,因为 REST 与线协议的演进节奏不同。
- **结构强类型,指标弱类型**:资源结构(Agent、任务、测试等)必须是强类型 DTO;而**指标值和评分**统一使用字典 `metrics: { "<metric_id>": number }`、`scores: { "<model_id>": number }`,键的含义、单位、版本由 `/api/public/catalog/*` 给出。这样新增指标或模型不需要改 OpenAPI,也不需要改前端类型。
### 7.2 版本与兼容性
- 当前版本为 v1,路径前缀为 `/api/public`、`/api/admin`、`/api/agent/v1`。出现破坏性变更时,新增 `/api/v2/public`、`/api/v2/admin` 等路径。v1 内只允许**非破坏性变更**:
| 非破坏性变更(v1 内允许) | 破坏性变更(需要新建 v2) |
| --- | --- |
| 新增端点 | 删除或重命名端点、字段 |
| 响应中新增字段 | 修改字段类型、单位或语义 |
| 新增可选的请求字段或查询参数 | 可选字段改为必填 |
| 枚举新增值 | 删除枚举值 |
| 放宽校验 | 收紧校验、修改默认值 |
| 新增错误码 | 修改已有错误码对应的 HTTP 状态 |
- 客户端必须忽略未知字段,也必须能处理未知的枚举值。
- 需要 v2 时,v1 和 v2 至少并存一个发布周期;OpenAPI 文档按版本分文件(`v1.json`、`v2.json`)。
### 7.3 URL 与方法
- **按调用方分层**:`/api/public`(只读展示)、`/api/admin`(管理)、`/api/agent`(节点)。同一个资源在不同层中可以有不同的 DTO,例如 public 层不暴露 `pubkey`、`local_limits`。
- 路径全部小写,资源名用复数名词,多个单词用 `-` 连接:`/api/admin/join-tokens`。
- 嵌套最多一层;子资源一旦有全局唯一 ID,就提供顶层路径(`/probes/{probe_id}`)。
- 查询参数和 JSON 字段使用 snake_case。
| 操作 | 方法 | 成功状态 | 说明 |
| --- | --- | --- | --- |
| 列表 | GET 集合 | 200 | 分页 |
| 读取 | GET 单个 | 200 | |
| 创建 | POST 集合 | 201 + `Location` | 响应体返回完整资源 |
| 部分更新 | PATCH | 200 | `application/merge-patch+json`(RFC 7396) |
| 删除 | DELETE | 204 | 资源不存在时同样返回 204(幂等) |
| 非 CRUD 动作 | POST `/{resource}/{id}/{verb}` | 200 或 204 | 例如 `revoke`、`rotate-token` |
- GET 请求不能产生副作用。PUT 不使用:配置类资源都通过 PATCH 修改。
### 7.4 数据表示
| 类别 | 规则 | 示例 |
| --- | --- | --- |
| 时间点 | RFC 3339 UTC,以 `Z` 结尾,精确到毫秒。**唯一例外**:列式时序响应中的时间轴数组 `t` 使用 Unix 毫秒(见 [query.md](../design/query.md)) | `2026-09-27T10:00:00.000Z` |
| 时长 | 整数 + 单位后缀 | `interval_ms: 1000` |
| 延迟 | 带小数的毫秒值(指标字典中的单位见目录) | `rtt_p95_ms: 58.93` |
| 百分比 | 0–100 的数字 + `_pct` | `loss_pct: 0.5` |
| 计数 | 整数,不使用浮点数 | `attempts: 600` |
| ID | 带类型前缀的 ULID 字符串 | `prb_01J9Z...` |
| 枚举 | snake_case 字符串 | `"udp_echo"` |
| 缺失值 | 字段始终存在;未知值写 `null`,不省略字段 | `"last_seen_at": null` |
| 布尔值 | 使用 `is_`/`has_` 前缀,或者用意义明确的形容词 | `online`、`enabled` |
### 7.5 错误
所有 4xx/5xx 响应都使用 `application/problem+json`(RFC 9457),并附加扩展字段:
```json
{
"type": "about:blank",
"title": "Validation failed",
"status": 400,
"code": "validation_failed",
"detail": "interval_ms must be >= 200",
"trace_id": "00-4bf92f...-01",
"errors": { "interval_ms": ["must be >= 200"] }
}
```
- `code` 取自共享的 `ErrorCode` 枚举,是客户端做判断的**唯一**依据;`title`/`detail` 只供人阅读,内容可能变化。
- `errors` 只在 400 `validation_failed` 中出现,键名是 JSON 字段路径。
- 5xx 响应不能泄露异常信息或调用栈,只提供 `trace_id`,用于在日志中定位问题。
| 状态码 | 使用场景 |
| --- | --- |
| 400 | 请求格式错误或校验失败 |
| 401 | 未认证 |
| 404 | 资源不存在 |
| 403 | 已认证但作用域不足 |
| 409 | 名称冲突、状态冲突(例如 `probe_offline`) |
| 410 | 资源曾经存在但已过保留期(例如 `raw_expired`) |
| 412 | `If-Match` 不匹配 |
| 413 | 请求体超过上限 |
| 422 | 语义上无法满足(例如 `range_too_large`) |
| 429 | 超过限流,需要带 `Retry-After` |
| 503 | 存储不可用,或正在关闭 |
### 7.6 集合与查询
- 分页由服务端驱动:`?limit=`(默认 50,最大 500)和 `?cursor=`(不透明字符串);响应格式为 `{ "items": [...], "next_cursor": string | null }`。排序必须稳定(按 ID 或时间 + ID)。
- 时间范围参数统一使用 `from`/`to`,区间为左闭右开 `[from, to)`。缺省时 `to = now`,`from = to − 1h`。
- 所有查询都有上限(点数、时间跨度、结果条数);超出时返回 422 并在 `detail` 中给出可行的替代方案,不能悄悄截断。
### 7.7 并发控制
- 配置类资源(Agent、Probe、任务)在响应中返回 `ETag`,PATCH/DELETE 支持 `If-Match`,不匹配时返回 412。没有带 `If-Match` 的请求按“最后写入者获胜”处理。
### 7.8 安全
- 只提供 HTTPS。认证方式有两种:
- 浏览器使用 Cookie 会话:`HttpOnly; Secure; SameSite=Strict`;所有非 GET 请求都必须带 `X-CSRF-Token` 请求头;
- 脚本使用 `Authorization: Bearer <api_token>`,作用域为 `read` 或 `admin`。
- 凭据不能放在 URL 或查询参数中;SSE 依靠 Cookie 或同源令牌认证。
- 登录接口需要限流(例如每个 IP 每分钟 10 次),失败时统一返回 401,不区分“用户不存在”和“密码错误”。
- public 层根据站点设置和 `public_visible` 过滤数据:被隐藏的资源对匿名访客一律返回 404,而不是 403。
- 令牌明文只在创建或轮换的响应中出现一次;这类响应带 `Cache-Control: no-store`。
- 请求体上限为 64 KiB;查询接口有超时(默认 10 秒)。
### 7.9 实时推送(SSE)
- 实时数据使用 `text/event-stream`。`event` 名称就是数据类型(`line`、`node`、`point`、`test`);`data` 是单行 JSON,遵守第 7.4 节的数据表示规则。
- 每 15 秒发送一行注释作为心跳。重连后服务端先推送当前快照,再推送增量,不补发历史。
- 订阅的主题数和连接数都有上限,超出时返回 429 `too_many_streams`。
- 推送的内容必须能由对应的 REST 查询重建:SSE 只负责“更快地获得变化”,不承载独有的数据。
### 7.10 新增端点检查清单
1. 符合第 7.3 节的路径与方法规则,并有 operationId、summary 和 tag;
2. 返回类型精确(`TypedResults`),错误使用 problem+json,错误码来自 `ErrorCode`;
3. DTO 符合第 7.4 节的数据表示规则,并注册到 JSON 源生成上下文;
4. 列表支持分页,查询有上限;
5. 选择正确的层(public / admin / agent),public 层只暴露可以公开的字段;
6. 重新生成并提交 `openapi/v1.json`,在 PR 中说明是否属于破坏性变更;
7. 测试覆盖:成功路径、校验失败、404、鉴权失败、隐藏资源;
8. 同步更新本文上面的端点说明。
+18 -18
View File
@@ -30,7 +30,7 @@
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │
│ 有公网 IP,监听 │ 先敲门再 TLS │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
测量会话:TLS 控制通道 + UDP/TCP 数据通道,由 Probe 主动拨出
测量会话:TLS 控制通道 + QUIC Datagram/TCP 数据通道,由 Probe 主动拨出(Agent 固定 QUIC 端口,Probe 本地源端口轮换)
宿主机更新器(容器节点):用本机公钥验签 → 按 digest 拉镜像 → 重启,失败回滚
```
@@ -60,32 +60,32 @@
| --- | --- | --- |
| 节点框架 | 注册、身份、连接、RPC、可靠上报、状态同步、宿主都由与业务无关的 **Fleet** 提供;MoShiro 只写业务,其他产品也能复用 | [../backend/fleet/README.md](../../backend/fleet/README.md) |
| 注册与身份 | 使用 `fleet-enroll` 和加入令牌自动注册,之后用签名认证;Probe 用自己的身份直连面板,Agent 列表由面板下发 | [enrollment.md](../security/enrollment.md) |
| 更新 | 只接受发布密钥签名的清单,更新策略由节点本地决定,失败自动回滚。容器节点(Agent、Probe)已实现:CI 对镜像 digest 签名,宿主机上的 `moshiro-update` 验签、按 digest 拉取、重启;纯二进制节点的更新是待办([TODO.md](../../TODO.md)) | [deployment.md](../ops/deployment.md) |
| 更新 | 只接受发布密钥签名的清单,更新策略由节点本地决定,失败自动回滚。CI 对镜像 digest 签名;Backend 和一行脚本装的节点由宿主机上的 `moshiro-update` 验签、按 digest 拉取、失败回滚;Podman Quadlet 装的 Agent、Probe 更简单:节点有序退出,systemd 用 `--pull always` 重新拉最新镜像,节点自己不下载、不验证清单;纯二进制节点的更新是待办([TODO.md](../../TODO.md)) | [deployment.md](../ops/deployment.md) |
| 任务模型 | `Task = {kind, target, params, schedule}`;持续采集、按需测试、目标任务统一建模;下发“期望任务集”,由节点自行调和 | [measurement.md](measurement.md) |
| 采集扩展 | Kind 由积木组合声明;采集密度靠参数调整;原始块自描述 | [measurement.md](measurement.md) |
| 统计扩展 | 指标注册制:少数几种累加器 + 统计算子;一行 `MetricDef` 就是一个新指标 | [analysis.md](analysis.md) |
| 协议 | 控制链路使用 Fleet(JSON-RPC 2.0 + 二进制通道);MoShiro 只定义 `moshiro.*` 方法、`TaskSet` 状态文档和 UDP/TCP 数据面 | [protocol.md](protocol.md) |
| 协议 | 控制链路使用 Fleet(JSON-RPC 2.0 + 二进制通道);MoShiro 只定义 `moshiro.*` 方法、`TaskSet` 状态文档和 QUIC Datagram/TCP 数据面 | [protocol.md](protocol.md) |
| 存储 | 原始块追加写入段文件,SQLite 记录偏移量;指标窗口通用存储并降采样 | [storage.md](storage.md) |
| 展示查询 | 任意时间段(相对或绝对);自动步长(每条曲线最多约 720 个点);1s 内存热层(1 小时)加上 10s/1m/5m/1h 各层;累加器现场合并,汇总精确;列式响应;SSE 实时跟随 | [query.md](query.md) |
| API | `/api/public`(只读,可设为公开)、`/api/admin`(登录)、`/api/agent`(注册与 RPC);实时数据用 SSE | [api.md](../api/api.md) |
### 4.0 容器节点的更新
### 4.0 节点的更新
发布清单(镜像 digest)由 CI 签名,放进镜像仓库(`<仓库>-manifest` 小镜像的标签里);Backend 定时取、验签后登记(也接受 CI 直接 `POST /api/release/offer`),再原样转发给在线的 Agent / Probe。之后分两条路:
```text
CI ──签名的发布清单(镜像 digest)──▶ 镜像仓库(<仓库>-manifest 小镜像的标签里)
│ │
Backend 定时取、验签后登记(也接受 CI 直接 POST /api/release/offer)
▼ │
Backend ──原样转发(offer)──▶ Agent / Probe 进程 │ 宿主机更新器也可以直接读
│ 只写进数据目录的 update/ │
▼ ▼
宿主机 moshiro-update(root):用本机公钥验签 → 核对组件、仓库和 seq → podman pull 仓库@digest → 重启 → 不健康就回滚
│ 状态写进只读的 update-status/
▼
节点随心跳上报更新状态 ──▶ Backend ──▶ 管理端(有新版本时点「更新」)
Podman Quadlet 节点(Agent、Probe,推荐)
节点比对清单和自己的版本 ──▶ 心跳报「有新版本」──▶ 管理端显示
自动(默认,运行满 10 分钟)或管理员点「更新」──▶ 节点有序退出
──▶ systemd 重启单元(Pull=always)──▶ 新镜像、新容器
节点不下载、不替换任何东西,也不验证清单;最多被请求一次有序重启
Backend 和一行脚本装的节点
宿主机 moshiro-update(root):用本机公钥验签 → 核对组件、仓库和 seq → podman pull 仓库@digest → 重启 → 不健康就回滚
──▶ 状态写进只读的 update-status/ ──▶ 节点随心跳上报 ──▶ Backend ──▶ 管理端(有新版本时点「更新」)
```
细节见 [deployment.md](../ops/deployment.md)「容器节点的更新」。
细节见 [deployment.md](../ops/deployment.md)「Podman(Quadlet)节点的更新」和「宿主机更新器」。
### 4.1 回显的四个时间戳
@@ -155,9 +155,9 @@ Kind 目录编译在二进制中,节点不接受远程下发代码(安全边
| D13 | Probe/Agent 运行时只依赖 BCL;.NET 10 NativeAOT | 轻量 |
| D14 | 节点机制抽成与业务无关的 Fleet 框架,放在同一仓库的 `backend/fleet/` 目录,暂不独立(由用户确认;不设拆分时间表,需要时再评估;不使用 PublicApiAnalyzers) | 可复用;不引用任何 `MoShiro.*` 的约束由 `ArchitectureTests` 在构建时检查 |
| D15 | 注册独立为 `fleet-enroll` 工具;常驻进程不包含注册代码 | 常驻进程去掉 HttpClient,边界更纯 |
| D16 | 更新与业务进程分离:发布密钥签名、本地策略、健康确认与回滚。容器节点已实现(宿主机 `moshiro-update`,清单里是镜像 digest);纯二进制节点的更新见 [TODO.md](../../TODO.md) | 面板被攻破时也无法推送恶意二进制 |
| D16 | 更新与业务进程分离:发布密钥签名、本地策略、健康确认与回滚。已实现:宿主机 `moshiro-update`(Backend、脚本装的节点,清单里是镜像 digest),Quadlet 节点则靠退出后由 systemd 重新拉镜像;纯二进制节点的更新见 [TODO.md](../../TODO.md) | 面板被攻破时也无法推送恶意二进制 |
| D17 | Probe 支持 OpenWrt:正式支持 `linux-musl-x64`、`linux-musl-arm64`;armv7 视工具链验证结果决定;MIPS 不支持,改为在局域网设备上运行 Probe | 由用户确认;NativeAOT 不支持 MIPS |
| D18 | 默认测量端口不使用 443:控制 7443/tcp,UDP 数据 7443/udp,TCP 数据 7444/tcp;443 只在 Agent 本地显式开启时才允许使用 | 由用户确认;避免与 VPS 上的真实服务冲突;结果会记录实际端口 |
| D18 | 默认测量端口不使用 443:控制 7443/tcp,QUIC Datagram 数据 7443/udp,TCP 数据 7444/tcp;443 只在 Agent 本地显式开启时才允许使用 | 由用户确认;避免与 VPS 上的真实服务冲突;结果会记录实际端口 |
| D19 | 展示查询最细 1s:最近 1 小时来自内存热层,更早的数据来自 10s/1m/5m/1h 各层,现场合并;时间段内的汇总由累加器精确合并得出 | 由用户确认;5 分钟、15 分钟这类视图足够精细且快 |
| D20 | 时序响应采用列式格式(`t[]` + `values{}`,时间轴使用 Unix 毫秒) | 由用户确认;体积小,可以直接交给图表库 |
| D21 | 国家和运营商只由 Backend 向 IP 归属服务查询(缺省 `https://ip.007995.xyz`,可关闭),只查公网地址;节点不访问任何外部服务;公开接口不返回 IP、城市、ASN | 客户的防火墙常常要审计节点的出站连接;访客拿不到敏感信息(有测试保证) |
+3 -3
View File
@@ -145,10 +145,10 @@ MoShiro.Measurement/ 依赖:MoShiro.Protocol(仅 BCL,
│ ├─ MonoClock.cs 单调时钟(微秒)与 UTC 锚点
│ └─ RunningBlock.cs 一个运行中的积木(取消令牌 + 后台任务)
├─ Net/
│ ├─ UdpMux.cs UDP 数据面:按 stream 分发;校验顺序固定;准入策略(重放、限速)通过 IAdmission 注入
│ ├─ QuicDatagramTransport.cs、MeasurementDatagramRouter.cs MsQuic 数据面与逐流认证分发
│ └─ Tcp.cs TcpLink(拨号:TLS 按指纹固定 + 认证首帧)、TcpAcceptor(监听:TLS 自动识别 + 认证首帧)
├─ Blocks/ 一个文件 = 一种采集方法的两个角色
│ ├─ Echo.cs UdpEchoDialer/Listener、TcpEchoDialer/Listener
│ ├─ Echo.cs DatagramEchoDialer/Listener、TcpEchoDialer/Listener
│ ├─ Session.cs ConnectDialer、HoldDialer、IcmpEchoSolo(没有对端)、IdleListener
│ ├─ Train.cs TrainDialer(接收,记 t_recv)、TrainListener(按码率发包,记发送行)
│ └─ BlockCatalog.cs Kind → (拨号方, 监听方);capabilities、本地预算的支持列表都从这里推出来
@@ -174,7 +174,7 @@ MoShiro.Measurement/ 依赖:MoShiro.Protocol(仅 BCL,
| 用途 | 默认端口 | 说明 |
| --- | --- | --- |
| Probe 控制通道(TLS,Fleet 点对点会话) | 7443/tcp | |
| UDP 数据(`udp_echo`、`udp_train`) | 7443/udp | |
| QUIC Datagram 数据(`udp_echo`、`udp_train`) | 7443/udp | Agent 固定监听;Probe 每次新连接轮换本地源端口 |
| TCP/TLS 数据(`tcp_echo`、`tcp_bulk`、`connect`、`hold`) | 7444/tcp | 按首帧中的 `stream_id` 分发 |
- **默认不使用 443**,避免与 VPS 上已有的 REALITY 或 Hysteria2 服务冲突,也不需要特权端口权限。
+3 -3
View File
@@ -5,14 +5,14 @@ MoShiro 的两条控制链路都建立在 Fleet 之上:
- **Agent ↔ Backend**:Fleet 连接服务端(wss),见 [../backend/fleet/README.md](../../backend/fleet/README.md);
- **Probe ↔ Agent**:Fleet 节点间点对点会话(TLS),见 [../backend/fleet/README.md](../../backend/fleet/README.md) 的「点对点会话」一节。
认证、心跳、重连、能力协商、可靠发件箱、期望状态同步、JSON-RPC 信封都由 Fleet 负责。本文只定义 **MoShiro 自己的内容**:`moshiro.*` 方法、状态文档、二进制通道,以及 Fleet 之外的数据面(UDP 包和数据连接)。所有类型都放在 `MoShiro.Protocol` 中。
认证、心跳、重连、能力协商、可靠发件箱、期望状态同步、JSON-RPC 信封都由 Fleet 负责。本文只定义 **MoShiro 自己的内容**:`moshiro.*` 方法、状态文档、二进制通道,以及 Fleet 之外的数据面(QUIC Datagram 和 TCP 数据连接)。所有类型都放在 `MoShiro.Protocol` 中。
## 1. Fleet 能力的映射
| MoShiro 需要的能力 | 使用的 Fleet 机制 | MoShiro 提供的内容 |
| --- | --- | --- |
| 下发任务 | 期望状态 `fleet.state.sync` | 文档类型 `TaskSet`,包括 `tasks[]`、`peers`(允许连这个 Agent 的 Probe 公钥,已按线路开关过滤) |
| 下发 Agent 列表给 Probe | 期望状态 `fleet.state.sync`(Probe 直连面板) | 文档类型 `probe_agents`:`version`、`agents[]`(`agent_id`、`host`、`tcp_port`、`udp_ports`、`public_key`),`public_key` 是 Agent 面向 Probe 的 TLS 证书公钥(SPKI,base64):Probe 用它固定证书并派生敲门密钥;没有上报公钥的 Agent 不会下发。已按线路开关过滤;Agent 集合或线路开关变化时重新推送 |
| 下发 Agent 列表给 Probe | 期望状态 `fleet.state.sync`(Probe 直连面板) | 文档类型 `probe_agents`:`version`、`agents[]`(`agent_id`、`host`、`tcp_port`、`quic_ports`、`public_key`),`public_key` 是 Agent 面向 Probe 的 TLS 证书公钥(SPKI,base64):Probe 用它固定证书并派生敲门密钥;没有上报公钥的 Agent 不会下发。已按线路开关过滤;Agent 集合或线路开关变化时重新推送 |
| 定时重启 | 通知 `moshiro.schedule`(B→A、B→P) | `{restart_at}`,形如 `04:30+08:00`,空表示关闭;连上时补发、管理端改了设置就群发。节点本地写了 `restart_at` 或 `MOSHIRO_RESTART_AT` 时以本地为准;节点只是到点自己有序退出,由服务管理器拉起 |
| 更新 | 通知 `moshiro.update.offer` / `moshiro.update.apply`(B→A、B→P),心跳附加字段 `update` | `offer` 带 CI 签名的发布清单原文和签名,节点只写进数据目录;`apply` 请求更新;`update` 是宿主机更新器的状态(`state`、`current`、`available`、`error`、`at`),见 [deployment.md](../ops/deployment.md) |
| 上报原始块 | 发件箱,二进制通道 **16** | `RawChunk`(自描述,见 [measurement.md](measurement.md) 第 4 节),`msg_id = chunk_id` |
@@ -113,5 +113,5 @@ Probe 只记录自己一侧的列,Agent 在数据面记录另一侧,由 Agen
- Fleet 层的兼容规则见 [../backend/fleet/README.md](../../backend/fleet/README.md)。
- `moshiro.*` 方法:新增方法或可选字段不需要升级版本,接收方忽略未知字段。
- RawChunk:块头自描述,新增列或新增 Kind 不需要升级 `container_version`;只有容器格式变化时才升级。
- UDP 包和数据连接首帧的布局变化时,必须升级 `protocol_version`,并更新黄金字节测试。
- QUIC Datagram 和 TCP 数据连接首帧的布局变化时,必须升级 `protocol_version`,并更新黄金字节测试。
- Agent 同时支持当前版本和上一个版本的 Probe;Backend 能解码保留期内所有的 `container_version`。
+1 -1
View File
@@ -123,7 +123,7 @@ id 的格式为 `指标.统计量`,详见 [analysis.md](analysis.md) 第 4 节
}
```
- 时间轴 `t` 使用 Unix 毫秒,只出现一次;`values` 中每个数组的长度都与 `t` 相同,可以直接交给 uPlot 或 ECharts。**这是 [api-guidelines.md](../api/api-guidelines.md) 中“时间用 RFC 3339”规则的明确例外**,理由是减小体积、方便绘图;其余时间字段仍然使用 RFC 3339。
- 时间轴 `t` 使用 Unix 毫秒,只出现一次;`values` 中每个数组的长度都与 `t` 相同,可以直接交给 uPlot 或 ECharts。**这是 [api.md](../api/api.md) 第 7.4 节“时间用 RFC 3339”规则的明确例外**,理由是减小体积、方便绘图;其余时间字段仍然使用 RFC 3339。
- 某个桶没有数据时填 `null`(不填 0),前端据此绘制断线。
- `summary` 是**整个时间段**的精确合并结果。`coverage` = 实际数据量 / 期望数据量,用来提示前端“数据是否完整”。
- `gaps` 和 `events` 用于在曲线上做标注(断线、端点变化、时钟跳变等)。
+1 -1
View File
@@ -108,4 +108,4 @@
- 提交信息使用 `<范围>: <摘要>` 格式,范围取值为 `probe`、`agent`、`backend`、`shared`、`web`、`docs`、`scripts`。
- 修改 `backend/shared/MoShiro.Protocol` 的 PR 必须说明兼容性影响(新增字段、行为变化、需要升级版本)。
- 修改 `backend/openapi/v1.json` 的 PR 必须按照 [api-guidelines.md](../api/api-guidelines.md) 第 2 节判断是否属于破坏性变更。
- 修改 `backend/openapi/v1.json` 的 PR 必须按照 [api.md](../api/api.md) 第 7 节的「版本与兼容性」判断是否属于破坏性变更。
+4 -4
View File
@@ -3,7 +3,7 @@
## 1. 目标
1. **三个独立项目加一个前端**:Probe、Agent、Backend 各自拥有解决方案,可以单独构建、测试和发布;Web 是独立的 Vite 项目(`web/`)。
2. **通用能力只写一次**:TCP/UDP/TLS/ICMP 采集积木(类似 iperf3 的能力,但只记录原始数据)放在 `MoShiro.Measurement` 中,Probe 和 Agent 共用;采集方法(Kind)由积木组合声明而成。
2. **通用能力只写一次**:TCP/QUIC Datagram/TLS/ICMP 采集积木(类似 iperf3 的能力,但只记录原始数据)放在 `MoShiro.Measurement` 中,Probe 和 Agent 共用;采集方法(Kind)由积木组合声明而成。
3. **节点机制通用化**:注册、身份、连接、RPC、可靠上报、状态同步、宿主,由与业务无关的 **Fleet** 框架提供(见 [../backend/fleet/README.md](../../backend/fleet/README.md))。MoShiro 只写业务,其他产品也可以复用 Fleet。
4. **改一处,整条链路受益**:协议、原始数据格式、测试方案、指标、错误码、版本号、编译设置、代码风格都只有**一个真源**。改动之后,下游要么自动更新,要么在编译期或 CI 中直接报错。
5. **模块化**:Probe/Agent 由可插拔的模块组成;Backend 按层分成多个项目,层之间的依赖在编译期强制。
@@ -49,7 +49,7 @@ moshiro/
└─ MoShiro.Backend.slnx
```
节点机器上的发布物是两个独立进程,注册和采集分开:`fleet-enroll`(一次性运行)和 `moshiro-agent`(或 `moshiro-probe`,常驻)。容器节点的更新由宿主机上的 `moshiro-update` 完成,见 [deployment.md](../ops/deployment.md)。
节点机器上的发布物是两个独立进程,注册和采集分开:`fleet-enroll`(一次性运行)和 `moshiro-agent`(或 `moshiro-probe`,常驻)。更新:Quadlet 装的 Agent、Probe 由节点退出后 systemd 重新拉镜像,Backend 和一行脚本装的节点由宿主机上的 `moshiro-update` 完成,见 [deployment.md](../ops/deployment.md)。
## 3. 依赖关系
@@ -111,7 +111,7 @@ MoShiro.Protocol/
│ └─ ParamSpec 参数定义(类型、默认值、上下限),节点与 Backend 共用同一套校验
├─ Tasks/ TaskSet(Fleet 期望状态的文档类型)、调和结果
├─ Rpc/ moshiro.* 方法名、params/result、二进制通道号(从 16 开始)
├─ Wire/ UDP 包、数据连接首帧
├─ Wire/ QUIC Datagram 测量包
└─ Raw/ 自描述 RawChunk:头部、列类型、列编解码、Brotli、crc32c
```
@@ -142,7 +142,7 @@ AgentApp Probe(ProbeApp)
发件箱 RawChunk(通道 16) 心跳带更新器的状态;moshiro.update.* 通知
心跳带更新器的状态;moshiro.update.* AgentDirectory → 每个 Agent 一个 AgentConnection(点对点会话)
PeerListener / PeerHub(Probe 会话) 会话内:线路任务集 line_task_set 的调和、角色、列块发件箱(通道 17)
UdpMux / TcpAcceptor(数据端口)
QuicDatagramTransport / TcpAcceptor(数据端口)
TaskManager / TestRunner / ChunkAssembler
RestartUpdater(只报告有无新版本、请求时退出) RestartUpdater
```
+39 -69
View File
@@ -1,6 +1,6 @@
# 部署
节点(Agent、Probe)是单个 NativeAOT 可执行文件,没有运行时依赖,只需要系统的 OpenSSL 3(`libssl.so.3`)。启动时会自检,缺少时第一步就明确报错。
节点(Agent、Probe)使用 NativeAOT。QUIC 目前通过应用目录优先加载 MsQuic v2;Linux 还需要 OpenSSL 3(`libssl.so.3`)和 MsQuic 依赖的 `libnuma.so.1`。CI 发布包尚需补上这些原生 sidecar;在这项打包接通前,Linux 主机要从系统包安装 MsQuic 和 libnuma,发布包不能视为免依赖安装包。启动时会检查 MsQuic 是否可加载。
## 1. 构建与打包
@@ -12,7 +12,7 @@ Agent 和 Probe 在 hello 初始化及心跳中上报 IPv6 检测结果,包括
升级 Backend、Agent、Probe 后,在管理端的 Agent 编辑页保留 IPv4 地址,并填写 IPv6 地址或带 AAAA 记录的域名。未填写 IPv6 地址的 VPS 只采集 IPv4;Probe 没有 IPv6 路由或 IPv6 连接失败时,IPv4 会话继续工作。
Agent 的新默认监听地址为 `::`(双栈);现有 `agent.json` 如果显式配置了 `0.0.0.0`,需要在宿主机改为 `::` 并重启节点。系统不支持 IPv6 套接字时,新默认值退回 IPv4 监听。防火墙需允许 IPv6 的 `7443/tcp`、`7443/udp`、`7444/tcp`(自定义端口时用实际端口);容器推荐宿主网络。管理端的「IPv6 监听」显示节点实际是否启用。
Agent 的新默认监听地址为 `::`(双栈);现有 `agent.json` 如果显式配置了 `0.0.0.0`,需要在宿主机改为 `::` 并重启节点。系统不支持 IPv6 套接字时,新默认值退回 IPv4 监听。防火墙需允许 IPv6 的 `7443/tcp`、`7443/udp`(QUIC)、`7444/tcp`(自定义端口时用实际端口);容器推荐宿主网络。管理端的「IPv6 监听」显示节点实际是否启用。
原来的 SQLite 数据库自动迁移,旧原始块、指标和测试结果继续属于 IPv4。先升级 Backend,再升级节点,最后填写 IPv6 地址;旧 Agent 不会上报 IPv6 支持,Backend 不会给它安排第二条会话。
@@ -23,7 +23,7 @@ scripts/publish.sh <agent|probe|backend|fleet-enroll> <rid> → backend/art
scripts/package.sh <agent|probe> <rid> → backend/artifacts/packages/moshiro-<role>-<version>-<rid>.tar.gz(附 .sha256)
```
构建信息(程序里显示的 tag、提交、构建时间):`publish.sh` 会用 `git describe --tags --always --dirty`、`git rev-parse` 和当前时间自动设置环境变量 `MOSHIRO_TAG`、`MOSHIRO_COMMIT`、`MOSHIRO_BUILT_AT`,直接用 `dotnet publish` 或 CI 构建时自己设置它们(容器镜像的 `*.NativeBuild.Containerfile` 用同名的 `--build-arg`,CI 脚本会传);不设置就是空,本地构建照常可重复。版本号取自 `backend/Directory.Build.props` 的 `Version`。前端的信息由 `web/build-info.mjs` 在构建时写入(版本取自 `web/package.json`)。
构建信息(程序里显示的 tag、提交、构建时间):`publish.sh` 会用 `git describe --tags --always --dirty`、`git rev-parse` 和当前时间自动设置环境变量 `MOSHIRO_TAG`、`MOSHIRO_COMMIT`、`MOSHIRO_BUILT_AT`,直接用 `dotnet publish` 或 CI 构建时自己设置它们(容器镜像的 `*.NativeBuild.Containerfile` 用同名的 `--build-arg`,CI 脚本会传);不设置就是空,本地构建照常可重复。版本号取自 `backend/Directory.Build.props` 的 `Version`;仓库里它只是占位(`0.0.0-dev`),CI 构建时由 `ci/version.sh` 写进工作区(见下文「版本号」)。前端的信息由 `web/build-info.mjs` 在构建时写入(版本取自 `web/package.json`,同样由 CI 填写)。Agent、Probe、Backend 启动时都会在日志里打印一行构建信息:版本、tag、提交、构建时间(没设置环境变量的本地构建显示 `unknown`)。
| rid | 目标 | 构建方式 |
| --- | --- | --- |
@@ -74,25 +74,44 @@ Woodpecker 在 `vMAJOR.MINOR.PATCH` tag 上,由 Windows runner 使用 AotAnywh
## CI 工作流
### Pull request 与 dev 验证
### Pull request 与 main 验证
Pull request 和 `dev` push 会运行 Backend tests;通过后,独立的 `web-build` workflow 才会构建前端并检查 CSS。提交信息后缀 `[SKIP TESTS]` 只跳过 Backend tests,前端构建仍会运行。Gitea PR 事件的 `CI_COMMIT_MESSAGE` 是 PR 标题,因此在 PR 中跳过测试时,把后缀写入 PR 标题;普通分支 push 则写在 commit message 中。
Pull request 和 `main` 的 push 会跑三个 workflow:
- `verify`:后端的快测试(除了标了 `Category=Slow` 的全部,约 20 秒);
- `web-build`:前端构建和 CSS 检查,和 `verify` 并行(约 15 秒);
- 标了 `Category=Slow` 的端到端测试(真实时间的 Hy2 阶梯、IPv6 和真探针的几条,一百来秒):带 `[CI]` 的推送,在 `dev-image` 里**等三个镜像都构建、推送完**再跑(不和 NativeAOT 编译抢内存——同时跑过一次,Backend 的 AOT 编译被内存不足杀掉了);PR 和没带 `[CI]` 的推送由 `e2e` workflow 在 `verify` 之后跑。失败照样红,但不影响已经推出去的镜像。
后端测试用增量编译(`ci/verify-backend.sh`):源码按内容同步到 CI 主机上一个固定目录,保留 `obj/bin`,只重编译变了的项目;同一时间只有一条流水线用这份缓存。失败的测试会单独重跑(最多两次),重跑通过算通过。给新的慢测试打标记:`[Trait("Category", "Slow")]`。
提交信息后缀 `[SKIP TESTS]` 跳过 `verify` 和 `e2e` 里的后端测试,前端构建仍会运行。Gitea PR 事件的 `CI_COMMIT_MESSAGE` 是 PR 标题,因此在 PR 中跳过测试时,把后缀写入 PR 标题;普通分支 push 则写在 commit message 中。项目只用 `main` 一个工作分支,旧的 `dev` 分支不再被任何流水线使用。
### 版本号
版本都由 CI 填写,仓库里写死的只是占位(`backend/Directory.Build.props` 的 `<Version>` 和三个 `web/**/package.json` 的 `version` 都是 `0.0.0-dev`),发版不需要提交 bump。规则在 `ci/version.sh`:
| 构建 | 版本 | 构建信息里的 tag |
| --- | --- | --- |
| 推 `vX.Y.Z` tag(正式版) | `X.Y.Z` | `vX.Y.Z` |
| `main` 上带 `[CI]` 的 push(开发镜像) | `X.Y.(Z+1)-dev.<N>-<提交前 7 位>` | `v` + 同一个版本 |
开发版本的基准是最近一个 `v*` tag(`X.Y.Z`),`N` 是这个 tag 之后的提交数(`git rev-list --count`),所以同一个提交算出来永远一样,也按语义化版本排在下一个正式版之前,例如 `0.4.1-dev.12-044ce0c`。算版本要拿得到 tag,所以开发镜像的 workflow 克隆时带 `tags: true`。镜像标签不带 `v`(和正式版一致),构建信息和签名清单里的 `tag` 带 `v`。
### main 开发镜像
`main` 上只有提交信息包含 `[CI]` 的 push 才构建镜像。合并 PR 时,请把 `[CI]` 放入 PR 标题或合并提交标题。Backend、Agent、Probe 的 Linux/amd64 镜像会并行构建,并推送到同一个 Zot Registry 下独立的 `moshiro-dev/` 仓库:
`main` 上只有提交信息包含 `[CI]` 的 push 才构建镜像,并且要等上面的 `verify`(快测试)和 `web-build` 都通过(提交信息写 `[SKIP TESTS]` 可以跳过测试);慢的端到端测试在镜像推完之后才跑,不拦着出镜像。合并 PR 时,请把 `[CI]` 放入 PR 标题或合并提交标题。Backend、Agent、Probe 的 Linux/amd64 镜像会并行构建,并推送到同一个 Zot Registry 下独立的 `moshiro-dev/` 仓库:
| 镜像 | 滚动标签 | 固定提交标签 |
| 镜像 | 滚动标签 | 固定版本标签 |
| --- | --- | --- |
| Backend | `registry.oeo.one/moshiro-dev/moshiro-backend:latest` | `registry.oeo.one/moshiro-dev/moshiro-backend:dev-<12位SHA>` |
| Agent | `registry.oeo.one/moshiro-dev/moshiro-agent:latest` | `registry.oeo.one/moshiro-dev/moshiro-agent:dev-<12位SHA>` |
| Probe | `registry.oeo.one/moshiro-dev/moshiro-probe:latest` | `registry.oeo.one/moshiro-dev/moshiro-probe:dev-<12位SHA>` |
| Backend | `registry.oeo.one/moshiro-dev/moshiro-backend:latest` | `registry.oeo.one/moshiro-dev/moshiro-backend:<版本>`(如 `0.4.1-dev.12-044ce0c`) |
| Agent | `registry.oeo.one/moshiro-dev/moshiro-agent:latest` | `registry.oeo.one/moshiro-dev/moshiro-agent:<版本>` |
| Probe | `registry.oeo.one/moshiro-dev/moshiro-probe:latest` | `registry.oeo.one/moshiro-dev/moshiro-probe:<版本>` |
Dev 镜像目前仅支持 amd64;Agent/Probe 在 `sdk:10.0-alpine-aot` 中构建 `linux-musl-x64` 产物,运行基底为 `runtime-deps:10.0-alpine`。它们不创建 Gitea Release,也不修改正式版 `moshiro/` 仓库中的标签。普通 main push 不会构建镜像。
Dev 镜像目前仅支持 amd64;Agent/Probe 在 `sdk:10.0-alpine-aot` 中构建 `linux-musl-x64` 产物,运行基底为 `runtime-deps:10.0-alpine`。它们不创建 Gitea Release,也不修改正式版 `moshiro/` 仓库中的标签。普通 main push 不会构建镜像。节点和 Backend 判断自己是不是开发构建,看构建信息里的 tag 含 `-dev.`(旧的 `dev-<sha>`、`main-<sha>` 也认)。
### 正式版发布
推送匹配项目版本的 `vMAJOR.MINOR.PATCH` tag 会触发完整跨平台构建、正式 OCI 镜像推送和 Gitea Release。正式镜像使用 `registry.oeo.one/moshiro/` 路径;Agent、Probe 提供 amd64 与 arm64 多架构标签,稳定版本更新正式仓库自己的 `latest`。这与 `moshiro-dev/` 的 dev `latest` 相互独立。
在 `main` 上推送 `vMAJOR.MINOR.PATCH` tag 会触发完整跨平台构建、正式 OCI 镜像推送和 Gitea Release,版本取自 tag,不需要先提交版本号。正式镜像使用 `registry.oeo.one/moshiro/` 路径;Agent、Probe 提供 amd64 与 arm64 多架构标签,稳定版本更新正式仓库自己的 `latest`。这与 `moshiro-dev/` 的 dev `latest` 相互独立。
Agent 和 Probe 镜像分别发布为 `registry.oeo.one/moshiro/moshiro-agent:<version>` 与 `registry.oeo.one/moshiro/moshiro-probe:<version>`,包含 amd64、arm64 多架构索引;也可指定 `:<version>-amd64` 或 `:<version>-arm64`。稳定版本同时更新 `latest`。镜像以非 root 用户(UID/GID 1654)运行,默认读取 `/etc/moshiro/agent.json` 或 `/etc/moshiro/probe.json`。使用 `packaging/compose/agent`、`probe` 下的标准 Compose 模板,配置和身份放在命名 volume 中;首次注册/导入、宿主指标挂载与更新器见下文。
@@ -136,7 +155,7 @@ Linux 上比 Windows 大,是因为 .NET 的加密、压缩、网络本机库
以 `moshiro` 用户运行,沙箱:`NoNewPrivileges`、`ProtectSystem=strict`、`ProtectHome`、`PrivateTmp/Devices`、`ProtectKernel*`、`RestrictAddressFamilies=AF_INET AF_INET6`、`RestrictNamespaces`、`MemoryDenyWriteExecute`(NativeAOT 不 JIT)、`SystemCallArchitectures=native`,`MemoryMax`(Agent 128 MB、Probe 64 MB)。
- **Agent** 只保留 `CAP_NET_RAW`(ICMP 目标任务用),`/etc/moshiro` 只读;Probe 没有任何 capability,只对 `agents.json` 所在的目录可写。
- 监听 1024 以下的端口(例如把数据面放到 443)需要在 drop-in 里加 `AmbientCapabilities=CAP_NET_BIND_SERVICE` 和对应的 `CapabilityBoundingSet`。默认端口是 7443/tcp+udp(Probe 控制通道和 UDP 数据面)、7444/tcp。
- 监听 1024 以下的端口(例如把数据面放到 443)需要在 drop-in 里加 `AmbientCapabilities=CAP_NET_BIND_SERVICE` 和对应的 `CapabilityBoundingSet`。默认端口是 7443/tcp(Probe 控制通道)、7443/udp(QUIC Datagram 数据面)、7444/tcp(TCP 测量)。Agent 在 UDP 端口固定监听;Probe 的 QUIC 连接使用轮换的临时本地源端口。
### OpenWrt(`packaging/openwrt/`)
@@ -156,68 +175,19 @@ OpenWrt 默认不带 OpenSSL:`opkg install libopenssl3`(较新的快照和 2
实测常驻内存(RSS,回环,默认模板,稳定后):Linux 上 Probe 约 21 MB、Agent 约 23 MB;Windows 上约 22 MB、25 MB。
### 跨机器的真实链路(Windows Probe → WSL2 里的 Agent 和 Backend)
### 故障注入
`scripts/smoke-wsl.py` 用 Windows 上的 NativeAOT Probe 连 WSL2 里的 Agent(真实的虚拟网络,两端的单调时钟原点相差约 38 小时),并用 `tc netem` 在 WSL 里注入故障。结果(Probe 的 RTT 基线约 0.4 ms):
`scripts/smoke-wsl.py`:Windows 上的 NativeAOT Probe 连 WSL2 里的 Agent 和 Backend(真实的虚拟网络,两端的单调时钟原点不同),用 `tc netem` 注入单向丢包和延迟、8 秒黑洞、Agent / Probe 重启,检查指标落在预期的一侧:上行丢包只出现在 `loss.up`、下行只出现在 `loss.down`;黑洞期间 UDP 连续丢包、TCP 不丢包只在 RTT 里出现几秒的长尾;重启之后几秒内自动恢复。具体数字以脚本的输出为准,不记在这里。
| 注入 | 期望 | 实测 |
| --- | --- | --- |
| Agent→Probe 方向 10% 丢包、30 ms ± 5 ms 延迟(egress netem,只匹配 7443/7444) | 丢包在下行,下行抖动变大 | `loss.down` 8.2%、`loss.up` 0;`jitter.down.p95` 8.7 ms、`jitter.up.p95` 0.18 ms;RTT +31 ms;TCP:`tcp.rtt.p99` 1.2 s、`tcp.slow` 28.8%、`tcp.retrans` 5.9%(Agent 一侧内核计数) |
| Probe→Agent 方向 10% 丢包、20 ms ± 3 ms 延迟(ingress + ifb) | 丢包在上行 | `loss.up` 9.4%、`loss.down` 0;`jitter.up.p95` 5.2 ms、`jitter.down.p95` 0 |
| 8 秒黑洞(tc ingress drop 7443/7444) | UDP 连续丢包 ~80 个;TCP 不丢包,只有几秒的长尾 | UDP 最长连续丢包 80(10 Hz × 8 s);`tcp.rtt.max` 9.5 s、`tcp.stall_max` 9.6 s |
| `hy2_ladder`(10/25/50/100 Mbit/s) | 无丢包 | `train.loss` 0.006%、`train.jitter.p95` 0.64 ms、可持续 100 Mbit/s |
| 杀掉 Agent / 杀掉 Probe 再启动 | 自动恢复 | Agent 重启后 2.1 s 恢复在线并重新连上 Probe;Probe 重启后 0.5 s 重新连上 |
这类验证发现过的几个问题,都已经修掉,改动时要留意它们为什么存在:Probe 重启后序号从 0 开始,Agent 不能沿用上一个会话的记录(否则真正的上行丢失会被算成下行丢失);TCP 回显的几秒中断必须记成延迟而不是丢包(TCP 的行要等回复,最长 30 秒才交出);`hy2_ladder` 的阶梯要把同一阶段的块先合并再算(真实的 Probe 会把一个阶段切成几块)。
验证中发现并修掉的两个问题:
### 家宽机器上的 Backend
- **Probe 重启后,Agent 沿用了上一个会话的记录**:Probe 重启之后序号从 0 开始,Agent 里同一个序号的旧 t2/t3 还在,于是真正的上行丢失被算成了下行丢失(黑洞测试里 `loss.up` 0、`loss.down` 7.4%)。现在新的 Probe 会话开始时 Agent 清空所有流的记录。
- **TCP 回显的几秒中断被记成丢包**:Probe 像 UDP 那样等 2 秒就把没有回复的行交出去,TCP 随后才到的回复没有行可以回填,最严重的长尾反而在 RTT 里看不到(黑洞期间 `tcp.rtt.max` 只有 4 ms)。现在 TCP 的行等回复,30 秒仍然没有才当作连接已断(t4 缺失)。
没有覆盖的:Windows 上的 Agent(需要给 Windows 防火墙加入站规则)、NAT 重绑定。跨公网的线路见下一节。
### 跨公网的真实线路(广东移动家宽 → 日本 IIJ 的 VPS)
2026-10-03 在一条真实线路上试跑过:家宽机器(Debian 13,用户态运行,没有公网 IPv4 入站)跑 Probe 和 Backend,VPS(Debian 12,1 核 470 MB,上面本来就跑着 xray、hysteria2、caddy 等服务)跑 Agent。
```text
浏览器 / 本机前端 ──► https://<公网域名> ──隧道(Cloudflare Tunnel 之类)──► 家宽机器: Backend (:29181)
家宽机器: Probe ──WSS 到 https://<公网域名>──► Backend(身份、Agent 列表)
家宽机器: Probe ──TCP/UDP 7443、TCP 7444──► VPS: Agent
VPS: Agent ──WSS 到 https://<公网域名>──► Backend
```
做法和要点:
- **Backend 通过隧道暴露**:隧道把一个公网域名转到家宽机器的 `29181`;Backend 监听 `http://0.0.0.0:29181`(只写一个监听地址,同时写 `0.0.0.0` 和 `[::]` 会报地址已被占用),家宽路由器的防火墙不放行这个端口,公网只能经隧道进来。Agent 用 `fleet-enroll --server https://<公网域名>` 注册,之后用 WSS 连回 Backend。
- **对外地址自动识别**:Agent 在 hello 里上报本机的公网地址(IPv4 公网地址、IPv6 全球单播地址,排除私网、CGNAT、临时地址),面板自动填 `public_host` 和 `ipv6_host`,不需要手填。经隧道注册时来源地址是隧道出口,面板不会用它。只有 Agent 在 NAT 后面、本机看不到公网地址时才需要在管理端手填;手填的优先,勾选“自动识别”可以撤销。
- **Probe 直连面板**:`probe.json` 里的 `fleet.server` 就是面板地址,Probe 用它连面板,Agent 列表由面板下发;密钥由 Probe 注册时自己生成,身份目录(`node.json`、`node.key`、`tls.pfx`)在 `fleet.identity_dir`。
- **小 VPS 把 `limits.max_total_mbps` 调低**(试跑时用 100);Agent 常驻约 23 MB,VPS 上原有的服务不受影响,结束后删除 `/opt/…`、停掉进程即可,没有留下任何东西。
- 家宽的 IPv6 前缀会在路由器重新拨号(比如停电)后变化,SSH 地址要跟着换;面板走隧道的域名,不受影响。
- 本地电脑没有直连 VPS 的路径时,用家宽机器当跳板(`SSH_ASKPASS` + `SSH_ASKPASS_REQUIRE=force`,密码放环境变量,不写进文件)。
**常驻(开机自启)**:
- 家宽机器(没有 root 时):`~/.config/systemd/user/moshiro-backend.service` 和 `moshiro-probe.service`,`EnvironmentFile` 指向放配置的 `backend.env`,`Restart=always`,`systemctl --user enable --now …`,再用 `sudo loginctl enable-linger <用户>` 让它们不登录也能起来。Probe 的命令行是 `moshiro-probe --config <probe.json 路径>`(默认路径是 `/etc/moshiro/probe.json`)。
- VPS:`/etc/systemd/system/moshiro-agent.service`,`Restart=always`,加 `NoNewPrivileges`、`ProtectSystem=strict`、`ReadWritePaths=/opt/moshiro`、`ProtectHome`、`PrivateTmp`。这是不走安装脚本的手动方式;正式安装用上面的 `packaging/systemd`。
- 日志进 journal:`journalctl -u moshiro-agent`、`journalctl --user -u moshiro-probe`。重启 Backend 用 `systemctl` 而不是手动再起一个进程,否则会端口冲突。
没有公网入站的家宽机器上跑 Backend 时,用隧道(Cloudflare Tunnel 之类)把一个公网域名转到 Backend 的端口;Backend 只写一个监听地址(同时写 `0.0.0.0` 和 `[::]` 会报地址已被占用);Probe 的 `fleet.server` 就是这个面板地址。小 VPS 上把 `limits.max_total_mbps` 调低。家宽的 IPv6 前缀会在路由器重新拨号(比如停电)后变化,用 IPv6 直连时地址要跟着换;走隧道的域名不受影响。
**国家和运营商**:Backend 用 Probe、Agent 的来源地址向 IP 归属服务查一次,展示在国旗和运营商图标上(`MoShiro__GeoLookupUrl`,缺省 `https://ip.007995.xyz`;留空就不查;这个服务只供公益使用,署名和使用条款由部署的人自己遵守)。Probe、Agent 自己不访问外部服务,所以客户的防火墙审计里不会多出出站连接;Backend 发出去的只有公网 IP(内网地址不发)。
**从别处看这个面板**:本机跑前端,在页面里把数据源指向这个域名,用 API 令牌访问,Backend 设 `MoShiro__CorsOrigins` 放行前端的来源(见 [web/README.md](../../web/README.md))。
实测(Probe 在广东移动家宽,Agent 在日本 IIJ):
| 项目 | 结果 |
| --- | --- |
| 延迟 p50 / p95 / p99 | 107 / 107 / 114 ms |
| 丢包 | 上行 0%,下行 8.1%(`hy2_ladder` 在 10、25、50、100 Mbit/s 四档下的丢包都是 8~10%,与速率无关:随机丢包,不是限速) |
| 抖动 p95(上 / 下) | 0.9 / 5.2 ms |
| TCP | 重传 1.9%(批量传输时 5.8%),RTT 最大 412 ms |
| `reality_session` | 建连成功率 100%,TCP 建连 p95 115 ms,TLS 握手 p95 117 ms,30 秒长连接无重置;4 秒批量传输峰值 133 Mbit/s、中位数 90 Mbit/s(Agent 上限 100) |
| 评分 | REALITY 82(含按需测试)、Hy2 42 |
| Agent 主机状态 | CPU 1.5%,内存 274 / 493 MB,已运行约 20.5 天 |
这次试跑发现并修掉的问题:`hy2_ladder` 的阶梯按「块」而不是按「阶段」计算,真实的 Probe 把一个阶段切成几块,4 个阶段出了 7 步(模拟数据不切块,所以之前的测试没有发现);现在同一阶段的块先合并。
**从别处看这个面板**:本机跑前端,在页面里把数据源指向面板的地址,用 API 令牌访问,Backend 设 `MoShiro__CorsOrigins` 放行前端的来源(见 [web/README.md](../../web/README.md))。
## 容器部署(Podman Quadlet 优先,Docker 用 compose)
@@ -261,7 +231,7 @@ chmod 600 .env && curl -fsSL 'https://panel.example.com/install/compose/probe.ya
- 只读根文件系统、丢掉全部能力、`no-new-privileges`、命名卷;Linux 上用宿主网络(Agent 对外暴露端口,也靠它识别本机公网地址,并只读挂载 `/proc`、`/sys`、`/` 读主机指标)。**Agent 的 compose 只适用于 Linux**;Probe 在 macOS 和 Windows(Docker Desktop)上没有宿主网络,面板给的命令会在 `.env` 里写 `MOSHIRO_NETWORK_MODE=bridge`。
- 身份和配置在命名卷 `moshiro-<role>-config` 里:镜像入口脚本(`packaging/container/entrypoint.sh`)第一次启动用 `MOSHIRO_SERVER`、`MOSHIRO_JOIN`(和信任根指纹、属主)注册并写配置,之后重启、换新镜像重建容器都不需要再给参数,所以注册成功之后可以把 `.env` 里的 `MOSHIRO_JOIN` 删掉。缺了信任根指纹或属主就不注册(只有还没有信任根的老面板才用 `MOSHIRO_ALLOW_NO_OWNER=1`)。
- Podman 的开机自启要靠 `podman-restart.service`(`systemctl enable --now podman-restart`;rootless 用 `--user`,并 `loginctl enable-linger`);Docker 用 compose 里的 `restart: unless-stopped`。
- **更新**:`podman compose pull && podman compose up -d`(Docker 用 `docker compose`),只重建镜像变了的容器,身份和配置不动,节点几十秒内重连。想固定版本,在 `.env` 里写 `MOSHIRO_VERSION=0.3.3`;想要更严,写成 `MOSHIRO_VERSION=0.3.3@sha256:<digest>`,digest 取自面板信任页或签名清单(那里有发布签名)。回滚就是改回旧版本再 `up -d`。
- **更新**:`podman compose pull && podman compose up -d`(Docker 用 `docker compose`),只重建镜像变了的容器,身份和配置不动,节点几十秒内重连。想固定版本,在 `.env` 里写 `MOSHIRO_VERSION=<版本号>`;想要更严,写成 `MOSHIRO_VERSION=<版本号>@sha256:<digest>`,digest 取自面板信任页或签名清单(那里有发布签名)。回滚就是改回旧版本再 `up -d`。
- **信任**:compose 来自面板或 CI 渠道,面板被攻破也改不了镜像地址;但客户端不再验发布签名,拉取靠 TLS 加「注册表可信」这个假设(文档里本来就把 CI、Gitea、注册表放在可信范围内)。compose 不会自动更新(见下节),也没有管理端的「更新」列和确认更新按钮。
**一行安装脚本(备用,Docker 或 Podman,Agent 和 Probe 共用)**:管理端的弹窗里折叠着一条旧的一行命令,先核对脚本哈希再执行,脚本向面板取签名清单、用内置发布公钥验签、按 digest 拉镜像:
+5 -5
View File
@@ -10,7 +10,7 @@
| Probe | Fleet 节点身份(连面板)+ 另一把独立的密钥(连 Agent,放在 `tls.pfx`) | `fleet-enroll`,和 Agent 一样用加入令牌(`role=probe`);可以有多个 Probe,每个独立吊销 | 两把公钥 |
| 面板用户 | 账号 + 密码(PBKDF2-SHA256) | 面板 | 密码哈希 |
| API 令牌 | 随机高熵令牌,带一组能力(见 [access.md](access.md)) | 面板 | SHA-256 |
| 发布签名 | ECDSA P-256 发布密钥 | CI(Woodpecker 密钥 `release_signing_key`)或离线环境 | 只在节点宿主机的更新器里配置公钥(容器节点是 `moshiro-update`,见 [deployment.md](../ops/deployment.md);纯二进制节点的更新是待办),**面板不保存私钥**;登记清单时面板用公钥验签,节点宿主机的更新器之后再验一次 |
| 发布签名 | ECDSA P-256 发布密钥 | CI(Woodpecker 密钥 `release_signing_key`)或离线环境 | 只在宿主机的更新器里配置公钥(Backend 和脚本装的节点用 `moshiro-update`,见 [deployment.md](../ops/deployment.md);Quadlet 节点不验证清单,只是退出后重新拉镜像;纯二进制节点的更新是待办),**面板不保存私钥**;登记清单时面板用公钥验签,宿主机的更新器之后再验一次 |
## 2. Agent 接入流程
@@ -25,7 +25,7 @@
├─ 用 Docker 或 Podman 起容器,身份和配置存在命名卷里;节点名字默认取主机名
└─ 容器入口用代码注册:fleet-enroll --server … --join jt_… --genesis … --controllers …
├─ 先验操作员集合链(创世指纹、属主在集合里),验不过就不交出密钥
├─ 注册元数据(MoShiro 定义):{ probe_tcp_port, probe_udp_ports, public_host?, kinds[] }
├─ 注册元数据(MoShiro 定义):{ probe_tcp_port, probe_quic_ports, public_host?, kinds[] }
└─ 写 trust.json(创世指纹、链、属主)和 agent.json(本地硬上限)
下载压缩包的二进制安装(packaging/install.sh)做同样的注册,另外创建系统用户 moshiro、启用 moshiro-agent.service,见 deployment.md。
@@ -52,7 +52,7 @@ probe.json:{ fleet: { server, identity_dir }, restart_at?, limits? } ← lim
Probe 启动
├─ 连接 server:Fleet 长连接(用 probe_id 和私钥认证),fleet.hello 带上系统、架构和构建信息
├─ 面板推送期望状态 probe_agents:{ version, agents: [ {agent_id, host, tcp_port, udp_ports, public_key}... ] }
├─ 面板推送期望状态 probe_agents:{ version, agents: [ {agent_id, host, tcp_port, quic_ports, public_key}... ] }
├─ 调和:新增的 Agent → 建立点对点会话(peer 用途签名);被移除或禁用的 → 断开
└─ 之后列表的任何变化(Agent 注册、禁用、地址或证书变化),面板都重新推送同一种期望状态;
Probe 用 Fleet.Hosting.ConfigStore 把最新列表写回 agents.json(这是配置,不是测量数据;面板暂时连不上时,Probe 靠它继续测量)
@@ -67,7 +67,7 @@ Probe 启动
```json
{
"fleet": { "server": "https://panel.example.com", "identity_dir": "/etc/moshiro" },
"listen": { "control_port": 7443, "udp_ports": [7443], "tcp_ports": [7444] },
"listen": { "control_port": 7443, "quic_ports": [7443], "tcp_ports": [7444] },
"limits": {
"max_total_mbps": 200,
"max_total_pps": 20000,
@@ -80,7 +80,7 @@ Probe 启动
- `limits` 是**本地硬上限**:期望任务集超出这些范围时,调和器拒绝执行,并在 `rejected[]` 中给出原因。
- `allowed_target_cidrs` 默认为空,即不允许执行任何目标任务;需要时由节点所有者在本机显式开启。
- `listen` 默认不包含 443。要用 443 测量,需要在本机把它加入 `udp_ports` 或 `tcp_ports`(端口不能被真实服务占用),见 [measurement.md](../design/measurement.md) 第 8 节。
- `listen` 默认不包含 443。要用 443 测量,需要在本机把它加入 `quic_ports` 或 `tcp_ports`(端口不能被真实服务占用),见 [measurement.md](../design/measurement.md) 第 8 节。
- `fleet_policy.allow_methods` 在 Fleet 默认值的基础上进一步收窄。
## 5. 相关 API
+2 -2
View File
@@ -143,7 +143,7 @@ Agent、Backend、Probe 之间互相不信任。这份文档回答三个问题
- 被攻破的 Probe 只能拿到 Backend 授权的会话,Agent 的速率上限由本地兜底;被攻破的 Agent 发来的数据只会影响它自己的线路。
- **Probe 只拨公网地址**(G3):固定 SPKI 只能防止建立会话,防不了探测本身;要在 Probe 本地拒绝私网、回环、链路本地地址。
- **名册里的主机名解析之后再检查地址**:签名保护的是字符串,不是解析结果;默认只放行公网地址,内网需要节点本地显式允许。
- **UDP 会话迁移要做路径验证**:协议允许会话在 NAT 映射变化后跟随新的源地址,路径上的攻击者可以借此把会话牵到自己的地址。迁移前先做一次挑战应答,验证前限制发送量(不超过收到字节数的 3 倍),类似 QUIC 的 path validation。
- **QUIC 会话迁移要做路径验证**:协议允许会话在 NAT 映射变化后跟随新的源地址,路径上的攻击者可以借此把会话牵到自己的地址。迁移前先做一次挑战应答,验证前限制发送量(不超过收到字节数的 3 倍),由 QUIC path validation 提供。
- **不协商、不降级**:现在固定 ECDSA P-256 和固定的签名串格式,没有算法协商,保持这一点。
- Agent 的 `max_total_mbps` 默认是 1200,对 1 核 1 GB 的小机器没有意义,却是被攻破的 Backend 能让 Agent 向某个 Probe 打出去的上限。默认值应该保守,需要的人再调高;`allowed_target_cidrs` 默认为空是对的。
@@ -181,7 +181,7 @@ Agent 和 Probe 的核心永远是纯采集器。**所有敏感能力**(exec
| 用途 | 现状 |
| --- | --- |
| 身份签名 | ECDSA P-256 加 SHA-256(BCL) |
| 数据面逐包认证 | HMAC-SHA256 截断到 16 字节(`UdpPacket`、`TcpWire`) |
| 数据面逐包认证 | HMAC-SHA256 截断到 16 字节(`MeasurementDatagram`、`TcpWire`) |
| 会话密钥派生 | HKDF-SHA256(`SessionKeys`) |
| 控制链路加密 | AES-256-GCM(每个方向一把密钥,严格递增的计数器) |
| 令牌哈希、SPKI 指纹 | SHA-256 |
+1 -1
View File
@@ -5,7 +5,7 @@
# 把面板给的 .env 内容存成 .env,再下载本文件为 compose.yaml
# podman compose up -d # Docker 用 docker compose
# 更新:podman compose pull && podman compose up -d (身份和配置在命名卷里,不会动)
# 固定版本:在 .env 里写 MOSHIRO_VERSION=0.3.3;想要更严,写成 MOSHIRO_VERSION=0.3.3@sha256:<digest>。
# 固定版本:在 .env 里写 MOSHIRO_VERSION=<版本号>;想要更严,写成 MOSHIRO_VERSION=<版本号>@sha256:<digest>。
# .env 里的 MOSHIRO_SERVER、MOSHIRO_JOIN(只在第一次启动用来注册,之后可以删掉)、MOSHIRO_GENESIS、MOSHIRO_CONTROLLERS、MOSHIRO_NAME 原样传进容器。
# 说明见 docs/ops/deployment.md。
services:
+1 -1
View File
@@ -5,7 +5,7 @@
# 把面板给的 .env 内容存成 .env,再下载本文件为 compose.yaml
# podman compose up -d # Docker 用 docker compose
# 更新:podman compose pull && podman compose up -d (身份和配置在命名卷里,不会动)
# 固定版本:在 .env 里写 MOSHIRO_VERSION=0.3.3;想要更严,写成 MOSHIRO_VERSION=0.3.3@sha256:<digest>。
# 固定版本:在 .env 里写 MOSHIRO_VERSION=<版本号>;想要更严,写成 MOSHIRO_VERSION=<版本号>@sha256:<digest>。
# macOS、Windows 的 Docker Desktop 没有宿主网络:面板给的 .env 里已经写了 MOSHIRO_NETWORK_MODE=bridge。
# .env 里的 MOSHIRO_SERVER、MOSHIRO_JOIN(只在第一次启动用来注册,之后可以删掉)、MOSHIRO_GENESIS、MOSHIRO_CONTROLLERS、MOSHIRO_NAME 原样传进容器。
# 说明见 docs/ops/deployment.md。
+1 -1
View File
@@ -67,7 +67,7 @@ write_config() {
cat >"$CONFIG/agent.json" <<JSON
{
"fleet": { "server": "$2", "identity_dir": "$CONFIG" },
"listen": { "bind": "::", "control_port": 7443, "udp_ports": [7443], "tcp_ports": [7444] },
"listen": { "bind": "::", "control_port": 7443, "quic_ports": [7443], "tcp_ports": [7444] },
"limits": { "max_total_mbps": 200, "max_total_pps": 20000, "allowed_target_cidrs": [], "max_concurrent_tests": 1 }
}
JSON
Loaded 100 of 154 files, more files were not shown because too many files have changed in this diff. Show more