1 Commits
Author SHA1 Message Date
justme ded7c4ef50 build: release v0.1.1 2026-10-05 09:38:05 -07:00
166 changed files with 1045 additions and 9010 deletions

No files matched your search

-63
View File
@@ -1,63 +0,0 @@
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: plugin-git
settings:
lfs: false
when:
- event: push
branch: main
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
concurrency: 1
steps:
- name: build-linux-amd64-backend
image: bash
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
commands:
- bash ci/build-main-amd64.sh backend
depends_on: []
- name: build-linux-amd64-agent
image: bash
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
RELEASE_API_URL: https://mo.oeo.one
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
RELEASE_API_TOKEN:
from_secret: release_api_token
commands:
- bash ci/build-main-amd64.sh agent
depends_on: []
- name: build-linux-amd64-probe
image: bash
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
RELEASE_API_URL: https://mo.oeo.one
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
RELEASE_API_TOKEN:
from_secret: release_api_token
commands:
- bash ci/build-main-amd64.sh probe
depends_on: []
+11 -7
View File
@@ -9,18 +9,22 @@ clone:
lfs: false lfs: false
when: when:
- event: pull_request - event: [push, pull_request]
- event: push
branch: dev
# On Gitea PR events, CI_COMMIT_MESSAGE is the PR title.
steps: steps:
- name: backend-tests - name: backend-tests
image: bash image: bash
# [SKIP TESTS] omits Backend tests only; web-build remains a dependent workflow.
when:
- evaluate: 'not (CI_COMMIT_MESSAGE contains "[SKIP TESTS]")'
environment: environment:
NUGET_PACKAGES: /home/justme/.nuget/packages NUGET_PACKAGES: /home/justme/.nuget/packages
commands: commands:
- cd backend && dotnet test --solution Monitor.slnx -c Release - cd backend && dotnet test --solution Monitor.slnx -c Release
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- npm run build --prefix web
- npm run lint:css --prefix web
depends_on: []
-27
View File
@@ -1,27 +0,0 @@
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: plugin-git
settings:
lfs: false
when:
- event: pull_request
- event: push
branch: dev
depends_on:
- verify
steps:
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- npm run build --prefix web
- npm run lint:css --prefix web
-43
View File
@@ -1,43 +0,0 @@
# Agent Instructions
## Project
Monitor is a self-hosted network path quality monitor. Backend coordinates Agents and Probes, stores measurements, and serves the REST/SSE API. `web/display` is the read-only overview; `web/admin` manages nodes, tests, and settings.
## Branch and workspace safety
- Treat `main` as the stable release branch and `dev` as the integration branch. Keep ordinary implementation work on `dev` or a feature branch; integrate into `main` through a pull request.
- A push to `main` builds images only if its commit message contains `[CI]` (for a merged PR, put it in the PR or merge commit title). Without `[CI]` nothing is built; use `[CI SKIP]` to skip the pipeline entirely. When squashing or pushing to `main`, add `[CI]` unless a build is explicitly not wanted.
- Before editing, inspect `git status`. Preserve pre-existing user changes: do not reset, clean, stash, or include unrelated files in a commit.
- Do not commit credentials, API keys, registry passwords, or generated runtime data. CI credentials belong in Woodpecker secrets (`zot_secret`, `gitea_token`, `release_signing_key` for signing release manifests, `release_api_token` with the `release` scope). The release signing private key never goes into the repository or the panel; only the public key in `packaging/update/` does.
- Keep development OCI images in the `monitor-dev/` repository path and release images in `monitor/`. Do not let a development build overwrite release tags such as `latest`.
## Project layout
- `backend/src/Monitor.Backend/`: service entry point and composition.
- `backend/src/Monitor.Backend.Api/`: REST endpoints, contracts, and OpenAPI.
- `backend/agent/`, `backend/probe/`: node applications and their READMEs.
- `backend/fleet/`, `backend/shared/`: reusable node framework, measurement engine, protocols, and data formats.
- `web/display/`, `web/admin/`: Vite applications; `web/shared/`: API client, generated types, charts, and shared UI.
- `packaging/`, `scripts/`, `.woodpecker/`, `docs/`: container/package builds, CI workflows, and project documentation.
## Local development and builds
Requirements: .NET SDK 10 and Node.js/npm. The SDK selection is defined by the root `global.json`; NuGet package versions are centralized in the root `Directory.Packages.props`.
- All .NET projects: `dotnet build Monitor.slnx -c Release` from the repository root.
- Backend build: `cd backend && dotnet build Monitor.slnx -c Release`
- Backend tests: `cd backend && dotnet test --solution Monitor.slnx -c Release`
- Frontend dependencies/build: `npm ci --prefix web && npm run build --prefix web`
- Frontend style check: `npm run lint:css --prefix web`
- Local backend: from `backend/`, run `dotnet run --project src/Monitor.Backend -- --urls http://localhost:5000`.
- Local web apps: from the repository root, run `npm ci --prefix web` then `npm run dev --prefix web`. The current dev config uses display port 5173 and admin port 5175; `/admin/` is proxied through the display server. Check the Vite configs when working on another branch because ports can differ.
- After changing API contracts, regenerate/check `backend/openapi/v1.json` and run `npm run gen:api --prefix web` so frontend API types stay aligned.
Follow the user's requested validation scope. Do not start a full tag/release build unless requested.
## CI and releases
CI triggers and image tags can change; inspect `.woodpecker/*.yml` and `docs/deployment.md` before changing or describing them. In particular, check the distinction between `[CI]` main builds, `[SKIP TESTS]`, and version-tag releases in the current branch.
A version tag `vMAJOR.MINOR.PATCH` runs the full cross-platform release pipeline. Development image builds and releases must remain isolated in their respective Zot repository paths.
-52
View File
@@ -1,52 +0,0 @@
<Solution>
<Folder Name="/backend/fleet/src/">
<Project Path="backend/fleet/src/Fleet.Core/Fleet.Core.csproj" />
<Project Path="backend/fleet/src/Fleet.Enrollment.Client/Fleet.Enrollment.Client.csproj" />
<Project Path="backend/fleet/src/Fleet.Hosting/Fleet.Hosting.csproj" />
<Project Path="backend/fleet/src/Fleet.Identity/Fleet.Identity.csproj" />
<Project Path="backend/fleet/src/Fleet.Link/Fleet.Link.csproj" />
<Project Path="backend/fleet/src/Fleet.Rpc/Fleet.Rpc.csproj" />
<Project Path="backend/fleet/src/Fleet.Server.Abstractions/Fleet.Server.Abstractions.csproj" />
<Project Path="backend/fleet/src/Fleet.Server.Enrollment/Fleet.Server.Enrollment.csproj" />
<Project Path="backend/fleet/src/Fleet.Server.Link/Fleet.Server.Link.csproj" />
</Folder>
<Folder Name="/backend/fleet/tests/">
<Project Path="backend/fleet/tests/Fleet.Tests/Fleet.Tests.csproj" />
</Folder>
<Folder Name="/backend/fleet/tools/">
<Project Path="backend/fleet/tools/fleet-enroll/fleet-enroll.csproj" />
</Folder>
<Folder Name="/backend/shared/">
<Project Path="backend/shared/Monitor.Measurement/Monitor.Measurement.csproj" />
<Project Path="backend/shared/Monitor.Protocol/Monitor.Protocol.csproj" />
</Folder>
<Folder Name="/backend/shared/tests/">
<Project Path="backend/shared/tests/Monitor.Measurement.Tests/Monitor.Measurement.Tests.csproj" />
<Project Path="backend/shared/tests/Monitor.Protocol.Tests/Monitor.Protocol.Tests.csproj" />
</Folder>
<Folder Name="/backend/agent/src/">
<Project Path="backend/agent/src/Monitor.Agent/Monitor.Agent.csproj" />
</Folder>
<Folder Name="/backend/probe/src/">
<Project Path="backend/probe/src/Monitor.Probe/Monitor.Probe.csproj" />
</Folder>
<Folder Name="/backend/probe/tests/">
<Project Path="backend/probe/tests/Monitor.Probe.Tests/Monitor.Probe.Tests.csproj" />
</Folder>
<Folder Name="/backend/src/">
<Project Path="backend/src/Monitor.Backend.Analysis/Monitor.Backend.Analysis.csproj" />
<Project Path="backend/src/Monitor.Backend.Api/Monitor.Backend.Api.csproj" />
<Project Path="backend/src/Monitor.Backend.Application/Monitor.Backend.Application.csproj" />
<Project Path="backend/src/Monitor.Backend.Gateway/Monitor.Backend.Gateway.csproj" />
<Project Path="backend/src/Monitor.Backend.Storage.Sqlite/Monitor.Backend.Storage.Sqlite.csproj" />
<Project Path="backend/src/Monitor.Backend/Monitor.Backend.csproj" />
</Folder>
<Folder Name="/backend/tests/">
<Project Path="backend/tests/Monitor.Backend.Analysis.Tests/Monitor.Backend.Analysis.Tests.csproj" />
<Project Path="backend/tests/Monitor.Backend.Tests/Monitor.Backend.Tests.csproj" />
<Project Path="backend/tests/Monitor.TestSupport/Monitor.TestSupport.csproj" />
</Folder>
<Folder Name="/ci/">
<Project Path="ci/Monitor.Tar/Monitor.Tar.csproj" />
</Folder>
</Solution>
+8 -46
View File
@@ -10,56 +10,36 @@ Monitor 是一个自托管的网络线路质量观测工具。它在 Probe 与
- **线路分析**:从原始测量数据生成时间序列指标,并比较线路在不同服务模型下的体验。 - **线路分析**:从原始测量数据生成时间序列指标,并比较线路在不同服务模型下的体验。
- **节点管理**:通过 Fleet 节点框架处理身份、注册、连接、任务同步和数据上报。 - **节点管理**:通过 Fleet 节点框架处理身份、注册、连接、任务同步和数据上报。
- **实时面板**:展示端查看线路状态和历史趋势;管理端负责节点、任务和系统配置。 - **实时面板**:展示端查看线路状态和历史趋势;管理端负责节点、任务和系统配置。
- **线路管理**:Backend 决定哪些 Probe 连哪些 Agent(自动连接开关加手动选择),在管理端的线路矩阵里一键开关。
- **节点信息与更新**:节点自报系统、架构和构建信息(版本、tag、提交、构建时间),国家和运营商由 Backend 查询;容器节点的更新由 CI 签名、Backend 转发、节点宿主机上的更新器验签并执行,失败自动回滚,管理端可以看状态和点「更新」。
- **自托管**:Backend 使用段文件和 SQLite 保存数据;提供 Linux、systemd 和 OpenWrt 相关部署文件。 - **自托管**:Backend 使用段文件和 SQLite 保存数据;提供 Linux、systemd 和 OpenWrt 相关部署文件。
## 架构 ## 架构
```text ```text
Web(展示端 · 管理端) Probe <──测量会话──> Agent ──长连接与数据上报──> Backend <──REST / SSE──> Web 展示端与管理端
▲ REST + SSE ├── 段文件:原始数据
┌──────────────────────────────────────────────────────────┐ └── SQLite:索引、指标与配置
│ Backend(面板) │
│ Api · Registry · Planner · Links │
│ Releases · Analysis · LiveHub │
│ Store(段文件 + SQLite) │
└──────────────────────────────────────────────────────────┘
▲ ▲
│ wss 长连接(节点发起) │ wss 长连接(节点发起)
│ 任务、数据、状态、更新通知 │ 身份、Agent 列表、状态、更新通知
┌────────────────────┐ ┌──────────────────────┐
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │
│ 有公网 IP,监听 │ │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
测量会话:TLS 控制通道 + UDP/TCP 数据通道,由 Probe 主动拨出
``` ```
两种连接:**Probe 和 Agent 都用长连接连 Backend**(身份、任务、状态、更新通知),**测量只走 Probe → Agent**(Probe 主动拨出,不需要入站端口)。哪些 Probe 连哪些 Agent 由 Backend 决定:默认自动全连,也可以在管理端手动选择。数据存储:段文件保存原始数据,SQLite 保存索引、指标与配置。
## 快速开始 ## 快速开始
### 环境要求 ### 环境要求
- .NET SDK 10;项目使用 `global.json` 指定 SDK 版本和滚动策略。 - .NET SDK 10;项目使用 `backend/global.json` 指定 SDK 版本和滚动策略。
- Node.js 与 npm,用于构建 Web 前端。 - Node.js 与 npm,用于构建 Web 前端。
### 构建 ### 构建
从仓库根目录构建全部 .NET 项目(Backend、Agent、Probe、Fleet、测试和 CI 工具): 从仓库根目录构建后端:
```sh ```sh
cd backend
dotnet build Monitor.slnx -c Release dotnet build Monitor.slnx -c Release
``` ```
NuGet 包版本统一在根目录 `Directory.Packages.props` 管理,项目中的 `PackageReference` 不写版本号。AotAnywhere 的 MSBuild SDK 版本在根目录 `global.json` 的 `msbuild-sdks` 中管理。`backend/Monitor.slnx` 保留用于只构建后端目录内的项目。
安装前端依赖并构建展示端和管理端(在仓库根目录运行): 安装前端依赖并构建展示端和管理端(在仓库根目录运行):
```sh ```sh
npm ci --prefix web npm ci --prefix web
npm run build --prefix web npm run build --prefix web
npm run gen:api --prefix web # API schema changed after Backend edits
npm run lint:css --prefix web
``` ```
前端产物位于 `web/display/dist/` 和 `web/admin/dist/`。NativeAOT 目标平台和 Windows 上使用 AotAnywhere 交叉构建的说明见 [`docs/deployment.md`](docs/deployment.md)。 前端产物位于 `web/display/dist/` 和 `web/admin/dist/`。NativeAOT 目标平台和 Windows 上使用 AotAnywhere 交叉构建的说明见 [`docs/deployment.md`](docs/deployment.md)。
@@ -82,16 +62,7 @@ npm ci --prefix web
npm run dev --prefix web npm run dev --prefix web
``` ```
打开 <http://localhost:5173/> 使用展示端,或访问 <http://localhost:5173/admin/> 进入管理端。依赖安装和前端配置见 [`web/README.md`](web/README.md);Backend 的首次启动与配置见 [`backend/README.md`](backend/README.md)。 打开 <http://localhost:5173/> 使用展示端,或访问 <http://localhost:5174/admin/> 进入管理端。依赖安装和前端配置见 [`web/README.md`](web/README.md);Backend 的首次启动与配置见 [`backend/README.md`](backend/README.md)。
### CI 构建规则
- Pull request 和 `dev` 分支 push:先跑 Backend tests,通过后再构建前端。
- 在需要跳过 Backend tests 的提交信息后加 `[SKIP TESTS]`;Gitea pull request 需要把标记写进 PR 标题。前端构建仍会运行。
- `main` 上的提交信息包含 `[CI]` 时,并行构建 Backend、Agent、Probe 的 Linux/amd64 镜像,推送到 Zot 的 `monitor-dev/` 仓库;普通 main push 不构建。
- 推送 `vMAJOR.MINOR.PATCH` tag 会触发全平台正式构建和 Release 发布。
完整触发条件、开发镜像标签和发布路径见 [`docs/deployment.md`](docs/deployment.md)。
## 项目布局 ## 项目布局
@@ -107,19 +78,10 @@ npm run dev --prefix web
| `packaging/`、`scripts/` | 部署、构建与维护脚本 | | `packaging/`、`scripts/` | 部署、构建与维护脚本 |
| `docs/` | [架构](docs/architecture.md)、[测量模型](docs/measurement.md)、[API](docs/api.md)、[部署](docs/deployment.md)等文档 | | `docs/` | [架构](docs/architecture.md)、[测量模型](docs/measurement.md)、[API](docs/api.md)、[部署](docs/deployment.md)等文档 |
## 待办
已实现的能力见上面的概览和 `docs/`;还没做的:
- **纯二进制节点的更新**:`fleet-updater`、`fleet-release`、Windows 服务和 OpenWrt(procd)适配器;设计见 [`docs/fleet/update.md`](docs/fleet/update.md)。容器节点(Agent、Probe)的签名发布和宿主机更新已经实现,见 [`docs/deployment.md`](docs/deployment.md)。
- **更新的灰度和自动策略**:按标签分批发布;`POLICY=auto` 的线上验证;后端自身的更新。
- **OpenWrt 打包**:procd 脚本、ipk/apk 包、sysupgrade 保留列表,在 aarch64 路由器上长时间运行验证。
- **安全加固**:对 UDP 包、控制帧、RawChunk、JSON-RPC 做模糊测试;伪造签名和重放的专项测试。
- **以后**:告警(Webhook)、真实 QUIC 握手积木、pcap/eBPF 采集。
## 测试 ## 测试
```sh ```sh
cd backend
dotnet test --solution Monitor.slnx dotnet test --solution Monitor.slnx
``` ```
@@ -1,15 +1,10 @@
<Project> <Project>
<PropertyGroup> <PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally> <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<!-- Native SQLite source is pinned separately from the JIT/test binary package. -->
<MonitorSqliteSourceVersion>3530400</MonitorSqliteSourceVersion>
<MonitorSqliteSourceYear>2026</MonitorSqliteSourceYear>
<MonitorSqliteSourceSha256>1e71ddf93849c6a6ecf58b827c0692073d2dd7ee40196158068f7b29f422e87d</MonitorSqliteSourceSha256>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageVersion Include="Microsoft.CodeAnalysis.BannedApiAnalyzers" Version="5.6.0" /> <PackageVersion Include="Microsoft.CodeAnalysis.BannedApiAnalyzers" Version="5.6.0" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="10.0.12" /> <PackageVersion Include="Microsoft.Data.Sqlite" Version="10.0.12" />
<PackageVersion Include="SQLitePCLRaw.lib.e_sqlite3" Version="3.53.3" />
<PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.12" /> <PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.12" />
<PackageVersion Include="Microsoft.Extensions.ApiDescription.Server" Version="10.0.12" /> <PackageVersion Include="Microsoft.Extensions.ApiDescription.Server" Version="10.0.12" />
<PackageVersion Include="Microsoft.Extensions.TimeProvider.Testing" Version="10.0.0" /> <PackageVersion Include="Microsoft.Extensions.TimeProvider.Testing" Version="10.0.0" />
-4
View File
@@ -8,10 +8,6 @@
<EventSourceSupport>false</EventSourceSupport> <EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport> <DebuggerSupport>false</DebuggerSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport> <HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<Http3Support>false</Http3Support>
<MetricsSupport>false</MetricsSupport>
<UseSizeOptimizedLinq>true</UseSizeOptimizedLinq>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<ServerGarbageCollection>false</ServerGarbageCollection> <ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection> <ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
</PropertyGroup> </PropertyGroup>
+1 -1
View File
@@ -47,7 +47,7 @@ fleet-enroll --server https://<backend> --join <token> --name <agent-name> --ide
monitor-agent --config /etc/monitor/agent.json monitor-agent --config /etc/monitor/agent.json
``` ```
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/deployment.md`](../../docs/deployment.md)。也可使用 OCI 镜像 `registry.oeo.one/monitor/monitor-agent:<version>`,自动选择 Linux amd64 或 arm64 镜像。镜像同时包含 `fleet-enroll`;可通过 `--entrypoint /usr/local/bin/fleet-enroll` 执行首次注册。容器默认读取 `/etc/monitor/agent.json`,需要挂载配置与身份目录;建议使用宿主网络,并按 ICMP 测量需求添加 `NET_RAW` 能力。镜像以 UID/GID 1654 运行,挂载目录必须允许该用户读取配置和身份文件。 也可使用 OCI 镜像 `registry.oeo.one/monitor/monitor-agent:<version>`,自动选择 Linux amd64 或 arm64 镜像。镜像同时包含 `fleet-enroll`;可通过 `--entrypoint /usr/local/bin/fleet-enroll` 执行首次注册。容器默认读取 `/etc/monitor/agent.json`,需要挂载配置与身份目录;建议使用宿主网络,并按 ICMP 测量需求添加 `NET_RAW` 能力。镜像以 UID/GID 1654 运行,挂载目录必须允许该用户读取配置和身份文件。
不传 `--config` 时默认读取 `/etc/monitor/agent.json`。Agent 默认监听控制端口 `7443/tcp`,数据端口 `7443/udp` 和 `7444/tcp`;防火墙和面板中的节点端口配置需与本地配置一致。`reporting.host_stats` 可关闭主机状态上报;启用时只上报资源使用等汇总数据,不包含主机名、进程或 IP。 不传 `--config` 时默认读取 `/etc/monitor/agent.json`。Agent 默认监听控制端口 `7443/tcp`,数据端口 `7443/udp` 和 `7444/tcp`;防火墙和面板中的节点端口配置需与本地配置一致。`reporting.host_stats` 可关闭主机状态上报;启用时只上报资源使用等汇总数据,不包含主机名、进程或 IP。
+10 -25
View File
@@ -12,7 +12,7 @@ namespace Monitor.Agent;
/// <summary>Agent 的组装点:数据面、点对点监听、与 Backend 的链路都在这里接起来。没有全局状态,测试可以在同一进程里启动多个。</summary> /// <summary>Agent 的组装点:数据面、点对点监听、与 Backend 的链路都在这里接起来。没有全局状态,测试可以在同一进程里启动多个。</summary>
public sealed class AgentApp : IDisposable public sealed class AgentApp : IDisposable
{ {
public const string Version = Monitor.Protocol.BuildInfo.Version; public const string Version = "0.1.1";
private readonly AgentConfig _config; private readonly AgentConfig _config;
private readonly LoadedNodeIdentity _identity; private readonly LoadedNodeIdentity _identity;
@@ -25,7 +25,6 @@ public sealed class AgentApp : IDisposable
private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously); private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly AgentAdmission _admission; private readonly AgentAdmission _admission;
private readonly Outbox _outbox = new(64 << 20); private readonly Outbox _outbox = new(64 << 20);
private readonly UpdateBridge _update;
private readonly BackendUplink _uplink; private readonly BackendUplink _uplink;
private readonly UdpMux _udp; private readonly UdpMux _udp;
private readonly TcpAcceptor _tcp; private readonly TcpAcceptor _tcp;
@@ -35,10 +34,8 @@ public sealed class AgentApp : IDisposable
private readonly TestRunner _tests; private readonly TestRunner _tests;
private readonly FleetLink _link; private readonly FleetLink _link;
/// <param name="dataDirectory">配置目录(容器里的 <c>/etc/monitor</c>):宿主机更新器和节点在这里交换清单、请求和状态。</param> public AgentApp(AgentConfig config, LoadedNodeIdentity identity, TimeProvider time, INodeLog log)
public AgentApp(AgentConfig config, LoadedNodeIdentity identity, string dataDirectory, TimeProvider time, INodeLog log)
{ {
_update = new UpdateBridge(dataDirectory, new ConfigStore(dataDirectory).Write, log);
_config = config; _config = config;
_identity = identity; _identity = identity;
_time = time; _time = time;
@@ -203,24 +200,6 @@ public sealed class AgentApp : IDisposable
(_, cancel, _) => ValueTask.FromResult(Result<TestRunResult>.Ok(new TestRunResult { Accepted = _tests.Cancel(cancel.TestId) }))); (_, cancel, _) => ValueTask.FromResult(Result<TestRunResult>.Ok(new TestRunResult { Accepted = _tests.Cancel(cancel.TestId) })));
} }
if (Allow(MonitorMethods.UpdateOffer))
{
router.OnNotify(MonitorMethods.UpdateOffer, MonitorJsonContext.Default.UpdateOfferParams, (_, offer, _) =>
{
_update.WriteOffer(offer);
return ValueTask.CompletedTask;
});
}
if (Allow(MonitorMethods.UpdateApply))
{
router.OnNotify(MonitorMethods.UpdateApply, MonitorJsonContext.Default.UpdateOfferParams, (_, _, _) =>
{
_update.RequestApply();
return ValueTask.CompletedTask;
});
}
if (Allow(MonitorMethods.ProbeDisconnect)) if (Allow(MonitorMethods.ProbeDisconnect))
{ {
router.On( router.On(
@@ -230,6 +209,14 @@ public sealed class AgentApp : IDisposable
async (_, request, _) => Result<TestRunResult>.Ok(new TestRunResult { Accepted = await _hub.DisconnectAsync(request.ProbeId.Value) })); async (_, request, _) => Result<TestRunResult>.Ok(new TestRunResult { Accepted = await _hub.DisconnectAsync(request.ProbeId.Value) }));
} }
if (Allow(MonitorMethods.ProbeAgentsChanged))
{
router.OnNotify(
MonitorMethods.ProbeAgentsChanged,
MonitorJsonContext.Default.ProbeAgentsResult,
(_, list, _) => _hub.RelayAgentsAsync(list));
}
var options = new FleetLinkOptions var options = new FleetLinkOptions
{ {
Server = new Uri(_config.Fleet.Server), Server = new Uri(_config.Fleet.Server),
@@ -257,7 +244,6 @@ public sealed class AgentApp : IDisposable
{ {
var meta = new AgentHelloMeta var meta = new AgentHelloMeta
{ {
Platform = Monitor.Measurement.PlatformDetector.Detect(),
TlsSpki = _identity.File.TlsSpki, TlsSpki = _identity.File.TlsSpki,
ProbeTcpPort = ControlPort, ProbeTcpPort = ControlPort,
ProbeUdpPorts = [.. UdpPorts], ProbeUdpPorts = [.. UdpPorts],
@@ -291,7 +277,6 @@ public sealed class AgentApp : IDisposable
DroppedInvalid = _counters.DroppedInvalid, DroppedInvalid = _counters.DroppedInvalid,
DroppedRateLimited = _counters.DroppedRateLimited, DroppedRateLimited = _counters.DroppedRateLimited,
QueueBytes = _outbox.PendingBytes, QueueBytes = _outbox.PendingBytes,
Update = _update.ReadStatus(),
}; };
_lastIn = packetsIn; _lastIn = packetsIn;
_lastOut = packetsOut; _lastOut = packetsOut;
@@ -6,30 +6,23 @@ namespace Monitor.Agent;
/// <summary> /// <summary>
/// 主机资源占用的只读快照:CPU 占用、内存、根分区、网卡累计流量与瞬时网速、系统运行时长。 /// 主机资源占用的只读快照:CPU 占用、内存、根分区、网卡累计流量与瞬时网速、系统运行时长。
/// 只有汇总的数字:<b>不采集 CPU 型号、主机名、网卡名和任何 IP 地址</b>,所以面板和展示页想泄漏也泄漏不出去。 /// 只有汇总的数字:<b>不采集 CPU 型号、主机名、网卡名和任何 IP 地址</b>,所以面板和展示页想泄漏也泄漏不出去。
/// Linux 读 <c>/proc</c> 和 <c>/sys</c>;容器通过 HOST_PROC、HOST_SYS、HOST_ROOT 指定宿主路径。 /// Linux 读 <c>/proc</c>;读不到的项是 null(例如 Windows 上的 CPU)。不执行任何命令,不写文件。
/// 读不到的项是 null(例如 Windows 上的 CPU)。不执行任何命令,不写文件。
/// </summary> /// </summary>
public sealed class HostSampler public sealed class HostSampler
{ {
private static readonly string[] s_virtualPrefixes = ["lo", "veth", "docker", "podman", "br-", "virbr", "cni", "flannel", "cali", "tun", "tap"]; private static readonly string[] s_virtualPrefixes = ["lo", "veth", "docker", "br-", "virbr", "cni", "flannel", "cali", "tun", "tap"];
private readonly TimeProvider _time; private readonly TimeProvider _time;
private readonly Func<string, string?> _read; private readonly Func<string, string?> _read;
private readonly Func<(long Total, long Used)?> _disk; private readonly Func<(long Total, long Used)?> _disk;
private readonly string _procRoot;
private readonly string _sysRoot;
private (long Total, long Idle)? _previousCpu; private (long Total, long Idle)? _previousCpu;
private (long At, long Rx, long Tx)? _previousNet; private (long At, long Rx, long Tx)? _previousNet;
public HostSampler(TimeProvider time, Func<string, string?>? read = null, Func<(long Total, long Used)?>? disk = null, public HostSampler(TimeProvider time, Func<string, string?>? read = null, Func<(long Total, long Used)?>? disk = null)
string? procRoot = null, string? sysRoot = null, string? diskRoot = null)
{ {
_time = time; _time = time;
_read = read ?? ReadProc; _read = read ?? ReadProc;
_procRoot = Root(procRoot, "HOST_PROC", "/proc"); _disk = disk ?? ReadRootDisk;
_sysRoot = Root(sysRoot, "HOST_SYS", "/sys");
string root = Root(diskRoot, "HOST_ROOT", OperatingSystem.IsWindows() ? Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\" : "/");
_disk = disk ?? (() => ReadRootDisk(root));
} }
public HostSnapshot Sample() public HostSnapshot Sample()
@@ -37,7 +30,7 @@ public sealed class HostSampler
long now = _time.GetTimestamp(); long now = _time.GetTimestamp();
var snapshot = new HostSnapshot { AtUtcMs = _time.GetUtcNow().ToUnixTimeMilliseconds() }; var snapshot = new HostSnapshot { AtUtcMs = _time.GetUtcNow().ToUnixTimeMilliseconds() };
if (_read(Path.Combine(_procRoot, "stat")) is { } stat && ParseCpu(stat) is { } cpu) if (_read("/proc/stat") is { } stat && ParseCpu(stat) is { } cpu)
{ {
if (_previousCpu is { } before && cpu.Total > before.Total) if (_previousCpu is { } before && cpu.Total > before.Total)
{ {
@@ -48,7 +41,7 @@ public sealed class HostSampler
_previousCpu = cpu; _previousCpu = cpu;
} }
if (_read(Path.Combine(_procRoot, "meminfo")) is { } meminfo && ParseMemory(meminfo) is { } memory) if (_read("/proc/meminfo") is { } meminfo && ParseMemory(meminfo) is { } memory)
{ {
snapshot.MemTotal = memory.Total; snapshot.MemTotal = memory.Total;
snapshot.MemUsed = memory.Used; snapshot.MemUsed = memory.Used;
@@ -60,11 +53,10 @@ public sealed class HostSampler
snapshot.DiskUsed = disk.Used; snapshot.DiskUsed = disk.Used;
} }
if (_read(Path.Combine(_procRoot, "net/dev")) is { } dev && ParseNetwork(dev) is { } net) if (_read("/proc/net/dev") is { } dev && ParseNetwork(dev) is { } net)
{ {
snapshot.NetRxTotal = net.Rx; snapshot.NetRxTotal = net.Rx;
snapshot.NetTxTotal = net.Tx; snapshot.NetTxTotal = net.Tx;
snapshot.NetLinkMbps = ReadLinkSpeed(dev);
if (_previousNet is { } before && net.Rx >= before.Rx && net.Tx >= before.Tx) if (_previousNet is { } before && net.Rx >= before.Rx && net.Tx >= before.Tx)
{ {
double seconds = Math.Max(0.001, _time.GetElapsedTime(before.At, now).TotalSeconds); double seconds = Math.Max(0.001, _time.GetElapsedTime(before.At, now).TotalSeconds);
@@ -75,11 +67,7 @@ public sealed class HostSampler
_previousNet = (now, net.Rx, net.Tx); _previousNet = (now, net.Rx, net.Tx);
} }
snapshot.UptimeSeconds = _read(Path.Combine(_procRoot, "uptime")) is { } uptime && ParseUptime(uptime) is { } up ? up : null; snapshot.UptimeSeconds = _read("/proc/uptime") is { } uptime && ParseUptime(uptime) is { } up ? up : Environment.TickCount64 / 1000;
if (!OperatingSystem.IsLinux() && snapshot.UptimeSeconds is null)
{
snapshot.UptimeSeconds = Environment.TickCount64 / 1000;
}
return snapshot; return snapshot;
} }
@@ -177,54 +165,11 @@ public sealed class HostSampler
} }
} }
private static string Root(string? value, string variable, string fallback) private static (long Total, long Used)? ReadRootDisk()
{
value ??= Environment.GetEnvironmentVariable(variable);
return string.IsNullOrWhiteSpace(value) ? fallback : value;
}
private long? ReadLinkSpeed(string dev)
{
long total = 0;
bool any = false;
foreach (string line in dev.Split('\n'))
{
int colon = line.IndexOf(':', StringComparison.Ordinal);
if (colon < 0)
{
continue;
}
string name = line[..colon].Trim();
if (s_virtualPrefixes.Any(p => name.StartsWith(p, StringComparison.Ordinal)))
{
continue;
}
string? speed = _read(Path.Combine(_sysRoot, "class/net", name, "speed"));
if (long.TryParse(speed?.Trim(), CultureInfo.InvariantCulture, out long mbps) && mbps > 0)
{
total += mbps;
any = true;
}
else
{
return null;
}
}
return any ? total : null;
}
private static (long Total, long Used)? ReadRootDisk(string root)
{ {
try try
{ {
if (!Directory.Exists(root)) string root = OperatingSystem.IsWindows() ? Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\" : "/";
{
return null;
}
var drive = new DriveInfo(root); var drive = new DriveInfo(root);
return drive.IsReady ? (drive.TotalSize, drive.TotalSize - drive.AvailableFreeSpace) : null; return drive.IsReady ? (drive.TotalSize, drive.TotalSize - drive.AvailableFreeSpace) : null;
} }
@@ -1,18 +1,12 @@
<Project> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.props" Sdk="StuDev.AotAnywhere"
Condition="'$(UseAotAnywhere)' == 'true'" /> Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>monitor-agent</AssemblyName> <AssemblyName>monitor-agent</AssemblyName>
<RootNamespace>Monitor.Agent</RootNamespace> <RootNamespace>Monitor.Agent</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>full</TrimMode>
<StripSymbols>true</StripSymbols>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<InternalsVisibleTo Include="Monitor.Backend.Tests" /> <InternalsVisibleTo Include="Monitor.Backend.Tests" />
@@ -22,5 +16,5 @@
</ItemGroup> </ItemGroup>
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere"
Condition="'$(UseAotAnywhere)' == 'true'" /> Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" />
</Project> </Project>
@@ -57,6 +57,18 @@ public sealed class BackendUplink
await SafeAsync(() => session.NotifyAsync(MonitorMethods.TestStatus, parameters, MonitorJsonContext.Default.TestStatusParams, CancellationToken.None)); await SafeAsync(() => session.NotifyAsync(MonitorMethods.TestStatus, parameters, MonitorJsonContext.Default.TestStatusParams, CancellationToken.None));
} }
/// <summary>代 Probe 向 Backend 取 Agent 列表(Probe 不认识 Backend)。链路断开时返回 fleet_node_offline。</summary>
public async ValueTask<Result<ProbeAgentsResult>> RequestProbeAgentsAsync(ProbeAgentsParams request, CancellationToken ct)
{
RpcSession<LinkContext>? session = _link?.CurrentSession;
if (session is null)
{
return Result<ProbeAgentsResult>.Fail(FleetErrors.NodeOffline, "the agent has no link to the backend");
}
return await session.RequestAsync(MonitorMethods.ProbeAgents, request, MonitorJsonContext.Default.ProbeAgentsParams, MonitorJsonContext.Default.ProbeAgentsResult, ct);
}
private static async ValueTask SafeAsync(Func<ValueTask> send) private static async ValueTask SafeAsync(Func<ValueTask> send)
{ {
try try
@@ -92,7 +104,7 @@ public sealed class PeerHub
public PeerHub(SessionRegistry sessions, MonoClock clock, ChunkAssembler chunks, TaskManager tasks, BackendUplink uplink, TimeProvider time, INodeLog log) public PeerHub(SessionRegistry sessions, MonoClock clock, ChunkAssembler chunks, TaskManager tasks, BackendUplink uplink, TimeProvider time, INodeLog log)
{ {
_router = BuildRouter(); _router = BuildRouter(uplink);
_sessions = sessions; _sessions = sessions;
_clock = clock; _clock = clock;
_chunks = chunks; _chunks = chunks;
@@ -240,6 +252,27 @@ public sealed class PeerHub
} }
} }
/// <summary>Backend 的 Agent 列表变化时转交给所有 Probe(Probe 据此调整它连接的 Agent)。</summary>
public async ValueTask RelayAgentsAsync(ProbeAgentsResult list)
{
foreach (ProbeSession probe in _sessions.All)
{
if (probe.Rpc is not { } rpc)
{
continue;
}
try
{
await rpc.NotifyAsync(MonitorMethods.ProbeAgentsChanged, list, MonitorJsonContext.Default.ProbeAgentsResult, CancellationToken.None);
}
catch (Exception ex) when (ex is ObjectDisposedException or InvalidOperationException or IOException)
{
// 会话刚好结束
}
}
}
private async ValueTask PushAsync(ProbeSession probe, LineTaskSet set, CancellationToken ct) private async ValueTask PushAsync(ProbeSession probe, LineTaskSet set, CancellationToken ct)
{ {
if (probe.Rpc is null) if (probe.Rpc is null)
@@ -267,8 +300,13 @@ public sealed class PeerHub
} }
} }
private static RpcRouter<PeerContext> BuildRouter() => private static RpcRouter<PeerContext> BuildRouter(BackendUplink uplink) =>
new RpcRouter<PeerContext>() new RpcRouter<PeerContext>()
.On(
MonitorMethods.ProbeAgents,
MonitorJsonContext.Default.ProbeAgentsParams,
MonitorJsonContext.Default.ProbeAgentsResult,
(_, request, ct) => uplink.RequestProbeAgentsAsync(request, ct))
.On( .On(
MonitorMethods.Clock, MonitorMethods.Clock,
MonitorJsonContext.Default.ClockParams, MonitorJsonContext.Default.ClockParams,
+1 -1
View File
@@ -35,7 +35,7 @@ catch (Exception ex) when (ex is IOException or InvalidDataException or InvalidO
using (identity) using (identity)
{ {
using var app = new AgentApp(config, identity, Path.GetDirectoryName(Path.GetFullPath(configPath))!, time, log); using var app = new AgentApp(config, identity, time, log);
await ServiceShell.RunAsync(app.RunAsync, log); await ServiceShell.RunAsync(app.RunAsync, log);
} }
@@ -1,36 +1,17 @@
<Project> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.props" Sdk="StuDev.AotAnywhere"
Condition="'$(UseAotAnywhere)' == 'true'" /> Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>fleet-enroll</AssemblyName> <AssemblyName>fleet-enroll</AssemblyName>
<RootNamespace>FleetEnroll</RootNamespace> <RootNamespace>FleetEnroll</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>full</TrimMode>
<StripSymbols>true</StripSymbols>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
<OptimizationPreference>Size</OptimizationPreference>
<UseSystemResourceKeys>true</UseSystemResourceKeys>
<StackTraceSupport>false</StackTraceSupport>
<EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<Http3Support>false</Http3Support>
<MetricsSupport>false</MetricsSupport>
<UseSizeOptimizedLinq>true</UseSizeOptimizedLinq>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<IlcFoldIdenticalMethodBodies>true</IlcFoldIdenticalMethodBodies>
<ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\..\src\Fleet.Enrollment.Client\Fleet.Enrollment.Client.csproj" /> <ProjectReference Include="..\..\src\Fleet.Enrollment.Client\Fleet.Enrollment.Client.csproj" />
</ItemGroup> </ItemGroup>
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere"
Condition="'$(UseAotAnywhere)' == 'true'" /> Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" />
</Project> </Project>
View File
File renamed without changes.
+26 -883
View File
File diff suppressed because it is too large. Load diff
-4
View File
@@ -8,10 +8,6 @@
<EventSourceSupport>false</EventSourceSupport> <EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport> <DebuggerSupport>false</DebuggerSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport> <HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<Http3Support>false</Http3Support>
<MetricsSupport>false</MetricsSupport>
<UseSizeOptimizedLinq>true</UseSizeOptimizedLinq>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<ServerGarbageCollection>false</ServerGarbageCollection> <ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection> <ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
</PropertyGroup> </PropertyGroup>
+6 -9
View File
@@ -1,19 +1,16 @@
# Monitor Probe # Monitor Probe
Probe 是网络测量的发起端,通常运行在家宽或其他需要观测的网络出口。它连接 Backend(身份、Agent 列表、状态)和 Agent(测量),执行线路任务,并把本端采集的原始数据交给 Agent 汇总上报。Probe 不提供远程命令或文件管理功能。 Probe 是网络测量的发起端,通常运行在家宽或其他需要观测的网络出口。它连接 Agent、执行线路任务,并把本端采集的原始数据交给 Agent 汇总上报。Probe 不直接连接 Backend,也不提供远程命令或文件管理功能。
## 工作流程 ## 工作流程
```text ```text
Backend ──任务──> Agent <──测量会话──> Probe Backend ──任务与节点列表──> Agent <──测量会话──> Probe
│ ▲ │ │ │ │
│ └──汇总并上报数据 <───┘ │ └──汇总并上报数据 <───┘
└─────Agent 列表、状态(Fleet 长连接)───────────┘
``` ```
Probe 使用面板生成的 `probe.json` 读取自身身份、面板地址(`server`)和本地测试限制。连上面板之后,面板下发要连接的 Agent 列表;Probe 将列表缓存到同目录的 `agents.json`,面板暂时连不上时靠它继续测量。`probe.json` 含私钥,请限制文件访问权限,不要提交到版本控制。 Probe 使用面板生成的 `probe.json` 读取自身身份、首次连接用的 Agent 地址和本地测试限制。成功连接后,Agent 会同步可用节点列表;Probe 将列表缓存到同目录的 `agents.json`。`probe.json` 含私钥,请限制文件访问权限,不要提交到版本控制。
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/deployment.md`](../../docs/deployment.md);Probe 也可以用 OCI 镜像 `registry.oeo.one/monitor/monitor-probe:<version>`,容器默认读取 `/etc/monitor/probe.json`,数据目录要允许 UID/GID 1654 读写。
## 构建 ## 构建
@@ -46,6 +43,6 @@ monitor-probe --config /etc/monitor/probe.json
## 设计说明 ## 设计说明
- Probe 只执行编译进程序的测量模块。它不落盘测量数据,也不能执行命令或自行更新(更新由宿主机上的更新器完成,Probe 只写清单和请求文件)。 - Probe 只执行编译进程序的测量模块。它不认识 Backend,不落盘测量数据,也不能执行命令或自行更新。
- 测量模块由 `backend/shared/Monitor.Measurement/` 与 Agent 共用;Probe 负责发起连接、调度任务和记录本端数据。 - 测量模块由 `backend/shared/Monitor.Measurement/` 与 Agent 共用;Probe 负责发起连接、调度任务和记录本端数据。
- 测量模型和协议细节见 [`../../docs/measurement.md`](../../docs/measurement.md) 与 [`../../docs/protocol.md`](../../docs/protocol.md);整体架构见 [`../../docs/architecture.md`](../../docs/architecture.md)。 - 测量模型和协议细节见 [`../../docs/measurement.md`](../../docs/measurement.md) 与 [`../../docs/protocol.md`](../../docs/protocol.md);整体架构见 [`../../docs/architecture.md`](../../docs/architecture.md)。
@@ -15,7 +15,7 @@ namespace Monitor.Probe;
public sealed record ConnectionEnv(NodeIdentity Identity, string ProbeId, ProbeLimits Limits, AgentDirectory Directory, MonoClock Clock, TimeProvider Time, INodeLog Log); public sealed record ConnectionEnv(NodeIdentity Identity, string ProbeId, ProbeLimits Limits, AgentDirectory Directory, MonoClock Clock, TimeProvider Time, INodeLog Log);
/// <summary> /// <summary>
/// 与一个 Agent 的点对点会话:认证握手 → 同步时钟 → 按 Agent 下发的 <c>line_task_set</c> 运行角色 → /// 与一个 Agent 的点对点会话:认证握手 → 同步时钟 → 取 Agent 列表 → 按 Agent 下发的 <c>line_task_set</c> 运行角色 →
/// 每 10 秒把已完成的行切成列块放进发件箱 → 响应按需测试的阶段。会话断开时所有角色停止(凭据属于会话),按退避重连。 /// 每 10 秒把已完成的行切成列块放进发件箱 → 响应按需测试的阶段。会话断开时所有角色停止(凭据属于会话),按退避重连。
/// 会话内的所有状态(套接字、发件箱、角色)随会话一起创建和释放。 /// 会话内的所有状态(套接字、发件箱、角色)随会话一起创建和释放。
/// </summary> /// </summary>
@@ -28,13 +28,13 @@ public sealed class AgentConnection
private readonly ConnectionEnv _env; private readonly ConnectionEnv _env;
private volatile ConnectionSession? _session; private volatile ConnectionSession? _session;
public AgentConnection(KnownAgent endpoint, ConnectionEnv env) public AgentConnection(BootstrapAgent endpoint, ConnectionEnv env)
{ {
Endpoint = endpoint; Endpoint = endpoint;
_env = env; _env = env;
} }
public KnownAgent Endpoint { get; } public BootstrapAgent Endpoint { get; }
public bool IsConnected => _session is not null; public bool IsConnected => _session is not null;
@@ -105,11 +105,12 @@ public sealed class AgentConnection
Task pump = PumpAsync(rpc, outbox, cts.Token); Task pump = PumpAsync(rpc, outbox, cts.Token);
try try
{ {
// 先同步时钟(块头里要带偏移);Agent 随后推送线路任务集 // 先同步时钟(块头里要带偏移),再取 Agent 列表;Agent 随后推送线路任务集
await SyncClockAsync(rpc, clock, rounds: 5, cts.Token); await SyncClockAsync(rpc, clock, rounds: 5, cts.Token);
backoff.Reset(); backoff.Reset();
_session = session; _session = session;
_env.Log.Info("agent session up", ("agent", Endpoint.AgentId), ("offset_us", clock.OffsetUs)); _env.Log.Info("agent session up", ("agent", Endpoint.AgentId), ("offset_us", clock.OffsetUs));
await RefreshDirectoryAsync(rpc, cts.Token);
Task maintenance = MaintainAsync(rpc, session, clock, cts.Token); Task maintenance = MaintainAsync(rpc, session, clock, cts.Token);
Task finished = await Task.WhenAny(run, receive, pump, maintenance); Task finished = await Task.WhenAny(run, receive, pump, maintenance);
await cts.CancelAsync(); await cts.CancelAsync();
@@ -162,6 +163,20 @@ public sealed class AgentConnection
} }
} }
private async Task RefreshDirectoryAsync(RpcSession<ConnectionSession> rpc, CancellationToken ct)
{
Result<ProbeAgentsResult> list = await rpc.RequestAsync(
MonitorMethods.ProbeAgents, new ProbeAgentsParams { HaveVersion = _env.Directory.Version }, MonitorJsonContext.Default.ProbeAgentsParams, MonitorJsonContext.Default.ProbeAgentsResult, ct);
if (list.TryGet(out ProbeAgentsResult? agents, out FleetError? error))
{
_env.Directory.Apply(agents);
}
else
{
_env.Log.Warn("could not fetch the agent list", ("agent", Endpoint.AgentId), ("err", error.Code));
}
}
/// <summary>切块(每 10 秒)和时钟同步(每 30 秒)。</summary> /// <summary>切块(每 10 秒)和时钟同步(每 30 秒)。</summary>
private async Task MaintainAsync(RpcSession<ConnectionSession> rpc, ConnectionSession session, ClockEstimator clock, CancellationToken ct) private async Task MaintainAsync(RpcSession<ConnectionSession> rpc, ConnectionSession session, ClockEstimator clock, CancellationToken ct)
{ {
@@ -214,6 +229,14 @@ public sealed class AgentConnection
MonitorMethods.TestAbort, MonitorMethods.TestAbort,
MonitorJsonContext.Default.TestAbortParams, MonitorJsonContext.Default.TestAbortParams,
async (session, abort, _) => await session.AbortAsync(abort.TestId)) async (session, abort, _) => await session.AbortAsync(abort.TestId))
.OnNotify(
MonitorMethods.ProbeAgentsChanged,
MonitorJsonContext.Default.ProbeAgentsResult,
(session, list, _) =>
{
session.Env.Directory.Apply(list);
return ValueTask.CompletedTask;
})
.OnNotify( .OnNotify(
FleetMethods.OutboxAck, FleetMethods.OutboxAck,
FleetRpcContext.Default.OutboxAckParams, FleetRpcContext.Default.OutboxAckParams,
@@ -6,7 +6,8 @@ using Monitor.Protocol;
namespace Monitor.Probe; namespace Monitor.Probe;
/// <summary> /// <summary>
/// Probe 要连接的 Agent 列表:由面板作为期望状态(<c>probe_agents</c>)直接下发;面板连不上时用上一次保存的 <c>agents.json</c>。 /// Probe 要连接的 Agent 列表。Probe 不认识 Backend:列表先取自 <c>probe.json</c> 的 bootstrap(或上一次保存的 <c>agents.json</c>),
/// 连上任意一个 Agent 之后,由它代为向 Backend 查询(<c>monitor.probe.agents</c>),之后的变化由 Agent 推送(<c>agents_changed</c>)。
/// 列表变化时用 <see cref="ConfigStore"/> 写回 <c>agents.json</c>(这是配置,不是测量数据;只在变化时写,保护路由器的闪存)。 /// 列表变化时用 <see cref="ConfigStore"/> 写回 <c>agents.json</c>(这是配置,不是测量数据;只在变化时写,保护路由器的闪存)。
/// </summary> /// </summary>
public sealed class AgentDirectory public sealed class AgentDirectory
@@ -16,25 +17,29 @@ public sealed class AgentDirectory
private readonly object _gate = new(); private readonly object _gate = new();
private readonly ConfigStore _store; private readonly ConfigStore _store;
private readonly INodeLog _log; private readonly INodeLog _log;
private List<KnownAgent> _agents = []; private List<BootstrapAgent> _agents;
private long _version; private long _version;
public AgentDirectory(ConfigStore store, string configDirectory, INodeLog log) public AgentDirectory(ConfigStore store, string configDirectory, ProbeFile file, INodeLog log)
{ {
_store = store; _store = store;
_log = log; _log = log;
_agents = [.. file.Bootstrap];
string cache = Path.Combine(configDirectory, FileName); string cache = Path.Combine(configDirectory, FileName);
if (File.Exists(cache)) if (File.Exists(cache))
{ {
try try
{ {
AgentsCache saved = NodeConfig.Load(cache, ProbeConfigContext.Default.AgentsCache); AgentsCache saved = NodeConfig.Load(cache, ProbeConfigContext.Default.AgentsCache);
if (saved.Agents.Count > 0)
{
_agents = saved.Agents; _agents = saved.Agents;
_version = saved.Version; _version = saved.Version;
} }
}
catch (Exception ex) when (ex is IOException or InvalidDataException or JsonException or InvalidOperationException) catch (Exception ex) when (ex is IOException or InvalidDataException or JsonException or InvalidOperationException)
{ {
_log.Warn("agents.json is unreadable, waiting for the panel", ("err", ex.Message)); _log.Warn("agents.json is unreadable, using the bootstrap list", ("err", ex.Message));
} }
} }
} }
@@ -53,7 +58,7 @@ public sealed class AgentDirectory
} }
} }
public IReadOnlyList<KnownAgent> Agents public IReadOnlyList<BootstrapAgent> Agents
{ {
get get
{ {
@@ -64,10 +69,10 @@ public sealed class AgentDirectory
} }
} }
/// <summary>应用面板下发的列表。版本不比当前新的忽略(乱序到达的旧推送)。</summary> /// <summary>应用 Agent 转来的列表。版本不比当前新的忽略(乱序到达的旧推送)。</summary>
public void Apply(ProbeAgentList result) public void Apply(ProbeAgentsResult result)
{ {
var next = result.Agents.Select(a => new KnownAgent { AgentId = a.AgentId.Value, Host = a.Host, TcpPort = a.TcpPort, Spki = a.Spki }).ToList(); var next = result.Agents.Select(a => new BootstrapAgent { AgentId = a.AgentId.Value, Host = a.Host, TcpPort = a.TcpPort, Spki = a.Spki }).ToList();
lock (_gate) lock (_gate)
{ {
if (result.Version < _version || (result.Version == _version && Same(next, _agents))) if (result.Version < _version || (result.Version == _version && Same(next, _agents)))
@@ -92,9 +97,9 @@ public sealed class AgentDirectory
Changed?.Invoke(); Changed?.Invoke();
} }
private static bool Same(List<KnownAgent> a, List<KnownAgent> b) => private static bool Same(List<BootstrapAgent> a, List<BootstrapAgent> b) =>
a.Count == b.Count && a.Zip(b).All(p => Equal(p.First, p.Second)); a.Count == b.Count && a.Zip(b).All(p => Equal(p.First, p.Second));
public static bool Equal(KnownAgent a, KnownAgent b) => public static bool Equal(BootstrapAgent a, BootstrapAgent b) =>
a.AgentId == b.AgentId && a.Host == b.Host && a.TcpPort == b.TcpPort && a.Spki == b.Spki; a.AgentId == b.AgentId && a.Host == b.Host && a.TcpPort == b.TcpPort && a.Spki == b.Spki;
} }
@@ -1,18 +1,12 @@
<Project> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.props" Sdk="StuDev.AotAnywhere"
Condition="'$(UseAotAnywhere)' == 'true'" /> Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>monitor-probe</AssemblyName> <AssemblyName>monitor-probe</AssemblyName>
<RootNamespace>Monitor.Probe</RootNamespace> <RootNamespace>Monitor.Probe</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>full</TrimMode>
<StripSymbols>true</StripSymbols>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<InternalsVisibleTo Include="Monitor.Backend.Tests" /> <InternalsVisibleTo Include="Monitor.Backend.Tests" />
@@ -22,5 +16,5 @@
</ItemGroup> </ItemGroup>
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere"
Condition="'$(UseAotAnywhere)' == 'true'" /> Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" />
</Project> </Project>
+8 -50
View File
@@ -1,72 +1,32 @@
using Fleet.Core; using Fleet.Core;
using Fleet.Hosting; using Fleet.Hosting;
using System.Text.Json;
using Fleet.Identity; using Fleet.Identity;
using Fleet.Link;
using Fleet.Rpc;
using Monitor.Measurement; using Monitor.Measurement;
using Monitor.Protocol;
namespace Monitor.Probe; namespace Monitor.Probe;
/// <summary> /// <summary>
/// Probe 的组装点:到面板的长连接(身份、Agent 列表、状态)+ 每个 Agent 一个测量连接(认证、重连、角色)。没有全局状态,测试可以在同一进程里启动多个。 /// Probe 的组装点:Agent 列表 → 每个 Agent 一个连接(认证、重连、角色)。没有全局状态,测试可以在同一进程里启动多个。
/// Agent 列表变化时只调和连接:新增的建立,移除的断开,地址或证书指纹变化的重建,其余不受影响。 /// 列表变化时只调和连接:新增的建立,移除的断开,地址或证书指纹变化的重建,其余不受影响。
/// </summary> /// </summary>
public sealed class ProbeApp : IDisposable public sealed class ProbeApp : IDisposable
{ {
public const string Version = Monitor.Protocol.BuildInfo.Version; public const string Version = "0.1.1";
private readonly ConnectionEnv _env; private readonly ConnectionEnv _env;
private readonly Dictionary<string, Running> _connections = []; private readonly Dictionary<string, Running> _connections = [];
private readonly SemaphoreSlim _changed = new(0); private readonly SemaphoreSlim _changed = new(0);
private readonly INodeLog _log; private readonly INodeLog _log;
private readonly FleetLink _link;
public ProbeApp(ProbeFile file, string configDirectory, TimeProvider time, INodeLog log) public ProbeApp(ProbeFile file, string configDirectory, TimeProvider time, INodeLog log)
{ {
_log = log; _log = log;
var directory = new AgentDirectory(new ConfigStore(configDirectory), configDirectory, log); var directory = new AgentDirectory(new ConfigStore(configDirectory), configDirectory, file, log);
NodeIdentity identity = NodeIdentity.FromPkcs8(Convert.FromBase64String(file.PrivateKey)); NodeIdentity identity = NodeIdentity.FromPkcs8(Convert.FromBase64String(file.PrivateKey));
_env = new ConnectionEnv(identity, file.ProbeId, file.Limits, directory, new MonoClock(time), time, log); _env = new ConnectionEnv(identity, file.ProbeId, file.Limits, directory, new MonoClock(time), time, log);
directory.Changed += () => _changed.Release(); directory.Changed += () => _changed.Release();
var update = new UpdateBridge(configDirectory, new ConfigStore(configDirectory).Write, log);
var options = new FleetLinkOptions
{
Server = new Uri(file.Server),
Identity = identity,
NodeId = new NodeId(file.ProbeId),
NodeVersion = Version,
Capabilities = [MonitorCapabilities.Base],
HeartbeatExtra = () => JsonSerializer.SerializeToElement(new ProbeHeartbeatExtra { Update = update.ReadStatus() }, MonitorJsonContext.Default.ProbeHeartbeatExtra),
HelloMeta = () => JsonSerializer.SerializeToElement(new ProbeHelloMeta { Platform = PlatformDetector.Detect() }, MonitorJsonContext.Default.ProbeHelloMeta),
};
var list = new StateHandler<ProbeAgentList>(
ProbeAgentList.StateType,
MonitorJsonContext.Default.ProbeAgentList,
(version, doc, _) =>
{
directory.Apply(doc);
return ValueTask.FromResult(new StateSyncResult { AppliedVersion = version });
});
var router = new RpcRouter<LinkContext>()
.OnNotify(MonitorMethods.UpdateOffer, MonitorJsonContext.Default.UpdateOfferParams, (_, offer, _) =>
{
update.WriteOffer(offer);
return ValueTask.CompletedTask;
})
.OnNotify(MonitorMethods.UpdateApply, MonitorJsonContext.Default.UpdateOfferParams, (_, _, _) =>
{
update.RequestApply();
return ValueTask.CompletedTask;
});
_link = new FleetLink(options, router, [list], new Outbox(1 << 20), time, log);
} }
/// <summary>到面板的连接(测试和状态查看用)。</summary>
public FleetLink Link => _link;
public AgentDirectory Directory => _env.Directory; public AgentDirectory Directory => _env.Directory;
/// <summary>某个 Agent 的会话是否已经建立(测试和状态查看用)。</summary> /// <summary>某个 Agent 的会话是否已经建立(测试和状态查看用)。</summary>
@@ -90,10 +50,9 @@ public sealed class ProbeApp : IDisposable
{ {
if (_env.Directory.Agents.Count == 0) if (_env.Directory.Agents.Count == 0)
{ {
_log.Info("no agents known yet, waiting for the panel to send the list"); _log.Warn("no agents known yet: probe.json has an empty bootstrap list");
} }
Task link = _link.RunAsync(ct);
try try
{ {
while (!ct.IsCancellationRequested) while (!ct.IsCancellationRequested)
@@ -109,7 +68,6 @@ public sealed class ProbeApp : IDisposable
finally finally
{ {
await ReconcileAsync([], CancellationToken.None); await ReconcileAsync([], CancellationToken.None);
await Task.WhenAny(link, Task.Delay(TimeSpan.FromSeconds(5), CancellationToken.None));
} }
} }
@@ -119,7 +77,7 @@ public sealed class ProbeApp : IDisposable
_changed.Dispose(); _changed.Dispose();
} }
private async Task ReconcileAsync(IReadOnlyList<KnownAgent> agents, CancellationToken ct) private async Task ReconcileAsync(IReadOnlyList<BootstrapAgent> agents, CancellationToken ct)
{ {
var wanted = agents.ToDictionary(a => a.AgentId); var wanted = agents.ToDictionary(a => a.AgentId);
var stopped = new List<Running>(); var stopped = new List<Running>();
@@ -127,7 +85,7 @@ public sealed class ProbeApp : IDisposable
{ {
foreach ((string id, Running running) in _connections.ToList()) foreach ((string id, Running running) in _connections.ToList())
{ {
if (!wanted.TryGetValue(id, out KnownAgent? endpoint) || !AgentDirectory.Equal(endpoint, running.Connection.Endpoint)) if (!wanted.TryGetValue(id, out BootstrapAgent? endpoint) || !AgentDirectory.Equal(endpoint, running.Connection.Endpoint))
{ {
stopped.Add(running); stopped.Add(running);
_connections.Remove(id); _connections.Remove(id);
@@ -147,7 +105,7 @@ public sealed class ProbeApp : IDisposable
lock (_connections) lock (_connections)
{ {
foreach ((string id, KnownAgent endpoint) in wanted) foreach ((string id, BootstrapAgent endpoint) in wanted)
{ {
if (!_connections.ContainsKey(id)) if (!_connections.ContainsKey(id))
{ {
@@ -15,15 +15,15 @@ public sealed class ProbeFile
[JsonPropertyName("private_key")] [JsonPropertyName("private_key")]
public string PrivateKey { get; set; } = ""; public string PrivateKey { get; set; } = "";
/// <summary>面板的地址(http 或 https)。Probe 直接连它:身份、Agent 列表、更新都从这里来。</summary> /// <summary>首次连接用的 Agent;连上之后以 <c>agents.json</c>(最新的列表)为准。</summary>
[JsonPropertyName("server")] [JsonPropertyName("bootstrap")]
public string Server { get; set; } = ""; public List<BootstrapAgent> Bootstrap { get; set; } = [];
[JsonPropertyName("limits")] [JsonPropertyName("limits")]
public ProbeLimits Limits { get; set; } = new(); public ProbeLimits Limits { get; set; } = new();
} }
public sealed class KnownAgent public sealed class BootstrapAgent
{ {
[JsonPropertyName("agent_id")] [JsonPropertyName("agent_id")]
public string AgentId { get; set; } = ""; public string AgentId { get; set; } = "";
@@ -54,12 +54,12 @@ public sealed class ProbeLimits
[JsonSerializable(typeof(AgentsCache))] [JsonSerializable(typeof(AgentsCache))]
public sealed partial class ProbeConfigContext : JsonSerializerContext; public sealed partial class ProbeConfigContext : JsonSerializerContext;
/// <summary><c>agents.json</c>:最近一次从面板同步到的 Agent 列表(配置,不是测量数据);面板连不上时 Probe 靠它继续测量。</summary> /// <summary><c>agents.json</c>:最近一次从 Agent 同步到的列表(配置,不是测量数据)。</summary>
public sealed class AgentsCache public sealed class AgentsCache
{ {
[JsonPropertyName("version")] [JsonPropertyName("version")]
public long Version { get; set; } public long Version { get; set; }
[JsonPropertyName("agents")] [JsonPropertyName("agents")]
public List<KnownAgent> Agents { get; set; } = []; public List<BootstrapAgent> Agents { get; set; } = [];
} }
+2 -2
View File
@@ -24,9 +24,9 @@ ProbeFile file;
try try
{ {
file = NodeConfig.Load(configPath, ProbeConfigContext.Default.ProbeFile); file = NodeConfig.Load(configPath, ProbeConfigContext.Default.ProbeFile);
if (file.ProbeId.Length == 0 || file.PrivateKey.Length == 0 || !Uri.TryCreate(file.Server, UriKind.Absolute, out _)) if (file.ProbeId.Length == 0 || file.PrivateKey.Length == 0)
{ {
throw new InvalidOperationException("probe.json needs probe_id, private_key and server (download it from the panel: Probes → Create)"); throw new InvalidOperationException("probe.json needs probe_id and private_key (download it from the panel: POST /api/admin/probe/identity)");
} }
} }
catch (Exception ex) when (ex is IOException or InvalidDataException or InvalidOperationException or System.Text.Json.JsonException) catch (Exception ex) when (ex is IOException or InvalidDataException or InvalidOperationException or System.Text.Json.JsonException)
@@ -30,22 +30,23 @@ public sealed class ProbeTests
return path; return path;
} }
private static ProbeAgentList List(long version, params string[] ids) => new() private static ProbeAgentsResult List(long version, params string[] ids) => new()
{ {
Version = version, Version = version,
Agents = [.. ids.Select(id => new AgentEndpoint { AgentId = new NodeId(id), Host = id + ".example", TcpPort = 7443, Spki = "c3BraQ==" })], Agents = [.. ids.Select(id => new AgentEndpoint { AgentId = new NodeId(id), Host = id + ".example", TcpPort = 7443, Spki = "c3BraQ==" })],
}; };
[Fact] [Fact]
public void AgentDirectory_StartsEmpty_IgnoresStaleLists_PersistsChanges_AndReloadsThem() public void AgentDirectory_StartsFromBootstrap_IgnoresStaleLists_PersistsChanges_AndReloadsThem()
{ {
string directory = NewDirectory(); string directory = NewDirectory();
try try
{ {
var directoryA = new AgentDirectory(new ConfigStore(directory), directory, NullNodeLog.Instance); var file = new ProbeFile { ProbeId = "prb_x", Bootstrap = [new BootstrapAgent { AgentId = "agt_boot", Host = "boot.example", TcpPort = 7443, Spki = "c3BraQ==" }] };
var directoryA = new AgentDirectory(new ConfigStore(directory), directory, file, NullNodeLog.Instance);
int changes = 0; int changes = 0;
directoryA.Changed += () => changes++; directoryA.Changed += () => changes++;
Assert.Empty(directoryA.Agents); Assert.Equal(["agt_boot"], directoryA.Agents.Select(a => a.AgentId));
directoryA.Apply(List(5, "agt_1", "agt_2")); directoryA.Apply(List(5, "agt_1", "agt_2"));
Assert.Equal(1, changes); Assert.Equal(1, changes);
@@ -56,8 +57,8 @@ public sealed class ProbeTests
Assert.Equal(1, changes); Assert.Equal(1, changes);
Assert.Equal(5, directoryA.Version); Assert.Equal(5, directoryA.Version);
// The next start uses the saved list while the panel is not reachable yet // The next start uses the saved list, not the bootstrap one
var directoryB = new AgentDirectory(new ConfigStore(directory), directory, NullNodeLog.Instance); var directoryB = new AgentDirectory(new ConfigStore(directory), directory, file, NullNodeLog.Instance);
Assert.Equal(["agt_1", "agt_2"], directoryB.Agents.Select(a => a.AgentId)); Assert.Equal(["agt_1", "agt_2"], directoryB.Agents.Select(a => a.AgentId));
Assert.Equal(5, directoryB.Version); Assert.Equal(5, directoryB.Version);
} }
@@ -68,14 +69,15 @@ public sealed class ProbeTests
} }
[Fact] [Fact]
public void AgentDirectory_StartsEmpty_WhenTheSavedFileIsCorrupt() public void AgentDirectory_FallsBackToBootstrap_WhenTheSavedFileIsCorrupt()
{ {
string directory = NewDirectory(); string directory = NewDirectory();
try try
{ {
new ConfigStore(directory).Write(AgentDirectory.FileName, "{ not json"u8.ToArray()); new ConfigStore(directory).Write(AgentDirectory.FileName, "{ not json"u8.ToArray());
var loaded = new AgentDirectory(new ConfigStore(directory), directory, NullNodeLog.Instance); var file = new ProbeFile { Bootstrap = [new BootstrapAgent { AgentId = "agt_boot", Host = "h", TcpPort = 1, Spki = "AA==" }] };
Assert.Empty(loaded.Agents); var loaded = new AgentDirectory(new ConfigStore(directory), directory, file, NullNodeLog.Instance);
Assert.Equal(["agt_boot"], loaded.Agents.Select(a => a.AgentId));
} }
finally finally
{ {
@@ -1,81 +0,0 @@
using System.Runtime.InteropServices;
using Monitor.Protocol;
namespace Monitor.Measurement;
/// <summary>
/// 读出本机的系统名和架构,只用于在面板上展示。故意只取粗粒度的信息:
/// 「Debian 12」「Windows 11」「x64」,不读主机名、内核版本、CPU 型号。
/// </summary>
public static class PlatformDetector
{
public static PlatformInfo Detect()
{
PlatformInfo info = Detect(File.Exists("/etc/os-release") ? File.ReadAllLines("/etc/os-release") : []);
info.Version = Empty(BuildInfo.Version);
info.Tag = Empty(BuildInfo.Tag);
info.Commit = Empty(BuildInfo.Commit);
info.BuiltAt = Empty(BuildInfo.BuiltAt);
return info;
}
private static string? Empty(string value) => value.Length == 0 ? null : value;
/// <summary>`osRelease` 是 /etc/os-release 的各行(Linux 之外不用)。拆出来是为了好测。</summary>
internal static PlatformInfo Detect(IEnumerable<string> osRelease)
{
string arch = RuntimeInformation.OSArchitecture switch
{
Architecture.X64 => "x64",
Architecture.Arm64 => "arm64",
Architecture.Arm => "arm",
Architecture.X86 => "x86",
var other => other.ToString().ToLowerInvariant(),
};
if (OperatingSystem.IsWindows())
{
// Windows 11 的版本号还是 10.0,靠内部版本号(22000 起)区分
string name = Environment.OSVersion.Version.Build >= 22000 ? "Windows 11" : "Windows 10";
return new PlatformInfo { Os = "windows", Name = name, Arch = arch };
}
if (OperatingSystem.IsMacOS())
{
return new PlatformInfo { Os = "macos", Name = "macOS", Arch = arch };
}
if (OperatingSystem.IsLinux())
{
return new PlatformInfo { Os = "linux", Name = LinuxName(osRelease), Arch = arch };
}
return new PlatformInfo { Os = "other", Name = null, Arch = arch };
}
/// <summary>从 os-release 里取发行版名字和主版本:「Debian GNU/Linux 12 (bookworm)」→「Debian 12」,「OpenWrt 24.10.0」→「OpenWrt 24.10」。</summary>
internal static string LinuxName(IEnumerable<string> osRelease)
{
var values = new Dictionary<string, string>(StringComparer.Ordinal);
foreach (string line in osRelease)
{
int eq = line.IndexOf('=', StringComparison.Ordinal);
if (eq > 0)
{
values[line[..eq].Trim()] = line[(eq + 1)..].Trim().Trim('"', '\'');
}
}
string name = values.GetValueOrDefault("NAME") ?? values.GetValueOrDefault("ID") ?? "Linux";
name = name.Replace(" GNU/Linux", string.Empty, StringComparison.Ordinal).Trim();
string? version = values.GetValueOrDefault("VERSION_ID");
if (string.IsNullOrEmpty(version))
{
return name;
}
// 只留 主.次:Alpine 的 3.22.1、OpenWrt 的 24.10.0 都截成两段;Debian 的 12 本来就一段
string[] parts = version.Split('.');
return $"{name} {string.Join('.', parts.Take(2))}";
}
}
@@ -1,100 +0,0 @@
using System.Text;
using System.Text.Json;
using Fleet.Core;
using Monitor.Protocol;
namespace Monitor.Measurement;
/// <summary>
/// 节点进程和宿主机更新器之间的文件交换(容器里 <c>/etc/monitor</c>,也就是数据目录):
/// 节点把面板发来的清单和签名写进 <c>update/</c>,把「更新」请求写成 <c>update/apply.request</c>;更新器验签并更新,状态写在 root 拥有的 <c>update-status/status.json</c>。
/// 节点进程不执行任何命令,也不信任清单的内容(验签在宿主机上)。没有安装更新器时目录写不了或没人读,调用只是返回 false。
/// </summary>
public sealed class UpdateBridge
{
private const int MaxManifestBytes = 4096;
private const int MaxSignatureBytes = 512;
private const int MaxStatusBytes = 4096;
private readonly string _updateDirectory;
private readonly string _statusFile;
private readonly Action<string, byte[]> _write;
private readonly INodeLog _log;
/// <param name="write">写数据目录里的文件(相对路径,原子替换):由节点传入 <c>ConfigStore.Write</c>——常驻进程里只有它可以写文件。</param>
public UpdateBridge(string dataDirectory, Action<string, byte[]> write, INodeLog log)
{
_updateDirectory = Path.Combine(dataDirectory, "update");
_statusFile = Path.Combine(dataDirectory, "update-status", "status.json");
_write = write;
_log = log;
}
/// <summary>放下面板发来的清单和签名(和已有的一样就不动,免得每次重连都惊动更新器)。先写清单、后写签名:更新器监视的是签名文件。</summary>
public bool WriteOffer(UpdateOfferParams offer)
{
byte[] manifest = Encoding.UTF8.GetBytes(offer.Manifest);
byte[] signature = Encoding.UTF8.GetBytes(offer.Signature);
if (manifest.Length is 0 or > MaxManifestBytes || signature.Length is 0 or > MaxSignatureBytes)
{
_log.Warn("ignoring an update offer of an unexpected size");
return false;
}
try
{
string manifestPath = Path.Combine(_updateDirectory, "offer.manifest");
string signaturePath = Path.Combine(_updateDirectory, "offer.sig");
if (Same(manifestPath, manifest) && Same(signaturePath, signature))
{
return true;
}
_write("update/offer.manifest", manifest);
_write("update/offer.sig", signature);
_log.Info("update offer received");
return true;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_log.Warn("could not store the update offer (is the updater installed?)", ("err", ex.Message));
return false;
}
}
/// <summary>请求更新:写一个空的 <c>apply.request</c>(更新器验签通过、清单更新才会真的动手)。</summary>
public bool RequestApply()
{
try
{
_write("update/apply.request", []);
_log.Info("update requested");
return true;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_log.Warn("could not request the update (is the updater installed?)", ("err", ex.Message));
return false;
}
}
/// <summary>更新器的状态;没有安装更新器、文件不存在或内容不对时返回 null。</summary>
public UpdateStatus? ReadStatus()
{
try
{
if (!File.Exists(_statusFile) || new FileInfo(_statusFile).Length > MaxStatusBytes)
{
return null;
}
return JsonSerializer.Deserialize(File.ReadAllBytes(_statusFile), MonitorJsonContext.Default.UpdateStatus);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException)
{
return null;
}
}
private static bool Same(string path, byte[] content) => File.Exists(path) && File.ReadAllBytes(path).AsSpan().SequenceEqual(content);
}
@@ -6,30 +6,4 @@
<ProjectReference Include="..\..\fleet\src\Fleet.Core\Fleet.Core.csproj" /> <ProjectReference Include="..\..\fleet\src\Fleet.Core\Fleet.Core.csproj" />
<ProjectReference Include="..\..\fleet\src\Fleet.Rpc\Fleet.Rpc.csproj" /> <ProjectReference Include="..\..\fleet\src\Fleet.Rpc\Fleet.Rpc.csproj" />
</ItemGroup> </ItemGroup>
<!--
构建信息:版本来自 $(Version),tag、提交、构建时间来自环境变量 MONITOR_TAG / MONITOR_COMMIT / MONITOR_BUILT_AT
(scripts/publish.sh 和 CI 会设置;本地开发不设置就是空,这样本地构建是可重复的、不会每次都重新编译)。
生成一个带常量的源文件,NativeAOT 下不用反射。
-->
<Target Name="WriteBuildInfo" BeforeTargets="CoreCompile">
<PropertyGroup>
<BuildInfoFile>$(IntermediateOutputPath)BuildInfo.g.cs</BuildInfoFile>
</PropertyGroup>
<ItemGroup>
<BuildInfoLine Include="namespace Monitor.Protocol%3B" />
<BuildInfoLine Include="/// &lt;summary&gt;这个程序是什么时候、从哪个版本构建出来的(构建时生成)。&lt;/summary&gt;" />
<BuildInfoLine Include="public static class BuildInfo" />
<BuildInfoLine Include="{" />
<BuildInfoLine Include=" public const string Version = &quot;$(Version)&quot;%3B" />
<BuildInfoLine Include=" public const string Tag = &quot;$(MONITOR_TAG)&quot;%3B" />
<BuildInfoLine Include=" public const string Commit = &quot;$(MONITOR_COMMIT)&quot;%3B" />
<BuildInfoLine Include=" public const string BuiltAt = &quot;$(MONITOR_BUILT_AT)&quot;%3B" />
<BuildInfoLine Include="}" />
</ItemGroup>
<WriteLinesToFile File="$(BuildInfoFile)" Lines="@(BuildInfoLine)" Overwrite="true" WriteOnlyWhenDifferent="true" />
<ItemGroup>
<Compile Include="$(BuildInfoFile)" />
<FileWrites Include="$(BuildInfoFile)" />
</ItemGroup>
</Target>
</Project> </Project>
+11 -117
View File
@@ -13,8 +13,8 @@ public static class MonitorMethods
public const string TestStatus = "monitor.test.status"; public const string TestStatus = "monitor.test.status";
public const string Event = "monitor.event"; public const string Event = "monitor.event";
public const string Host = "monitor.host"; public const string Host = "monitor.host";
public const string UpdateOffer = "monitor.update.offer"; public const string ProbeAgents = "monitor.probe.agents";
public const string UpdateApply = "monitor.update.apply"; public const string ProbeAgentsChanged = "monitor.probe.agents_changed";
// Probe ↔ Agent(点对点会话) // Probe ↔ Agent(点对点会话)
public const string Clock = "monitor.clock"; public const string Clock = "monitor.clock";
@@ -147,11 +147,15 @@ public sealed class AgentEndpoint
public string Spki { get; set; } = ""; public string Spki { get; set; } = "";
} }
/// <summary>期望状态文档(Fleet state type = <c>probe_agents</c>):Backend 直接下发给每个 Probe 的 Agent 列表。</summary> public sealed class ProbeAgentsParams
public sealed class ProbeAgentList
{ {
public const string StateType = "probe_agents"; /// <summary>Probe 目前持有的列表版本(用于调试;服务端总是返回当前列表)。</summary>
[JsonPropertyName("have_version")]
public long HaveVersion { get; set; }
}
public sealed class ProbeAgentsResult
{
[JsonPropertyName("version")] [JsonPropertyName("version")]
public long Version { get; set; } public long Version { get; set; }
@@ -190,108 +194,9 @@ public sealed class LocalLimits
public int MaxConcurrentTests { get; set; } = 1; public int MaxConcurrentTests { get; set; } = 1;
} }
/// <summary>
/// 节点的系统和架构(粗粒度,只用于展示):<c>os</c> 是 linux / windows / macos / other,<c>name</c> 是「Debian 12」「Windows 11」这样的名字,
/// <c>arch</c> 是 x64 / arm64 / arm / x86。<b>没有主机名、内核版本、具体型号</b>。另外带着这个程序自己的构建信息(版本、tag、构建时间)。
/// </summary>
public sealed class PlatformInfo
{
[JsonPropertyName("os")]
public string Os { get; set; } = "other";
[JsonPropertyName("name")]
public string? Name { get; set; }
[JsonPropertyName("arch")]
public string? Arch { get; set; }
/// <summary>程序的版本、tag、提交(短)和构建时间(UTC,ISO 8601):<see cref="BuildInfo"/> 构建时写进去的,没设置就是空。</summary>
[JsonPropertyName("version")]
public string? Version { get; set; }
[JsonPropertyName("tag")]
public string? Tag { get; set; }
[JsonPropertyName("commit")]
public string? Commit { get; set; }
[JsonPropertyName("built_at")]
public string? BuiltAt { get; set; }
}
/// <summary>
/// 面板给节点的更新清单(<c>monitor.update.offer</c>):<c>manifest</c> 是发布清单的原始文本,<c>signature</c> 是 base64 的签名。
/// 节点只是把它们原样放进数据目录;验签、拉取、重启都由宿主机上的更新器完成(节点进程不执行任何命令,也不信任清单的内容)。
/// </summary>
public sealed class UpdateOfferParams
{
[JsonPropertyName("manifest")]
public string Manifest { get; set; } = "";
[JsonPropertyName("signature")]
public string Signature { get; set; } = "";
}
/// <summary>宿主机更新器已经验证过的可用更新。</summary>
public sealed class UpdateAvailable
{
[JsonPropertyName("version")]
public string? Version { get; set; }
[JsonPropertyName("tag")]
public string? Tag { get; set; }
[JsonPropertyName("digest")]
public string? Digest { get; set; }
[JsonPropertyName("seq")]
public long Seq { get; set; }
}
/// <summary>
/// 宿主机更新器的状态(它写在 <c>update-status/status.json</c>,节点读出来随心跳上报):
/// <c>up_to_date</c>、<c>available</c>、<c>applying</c>、<c>failed</c>、<c>rolled_back</c>、<c>invalid</c>(清单验签或校验没过)。
/// 没有安装更新器的节点没有这个字段。
/// </summary>
public sealed class UpdateStatus
{
[JsonPropertyName("state")]
public string State { get; set; } = "";
/// <summary>正在运行的镜像 digest(更新器记录的;还没更新过时为空)。</summary>
[JsonPropertyName("current")]
public string? Current { get; set; }
[JsonPropertyName("available")]
public UpdateAvailable? Available { get; set; }
[JsonPropertyName("error")]
public string? Error { get; set; }
[JsonPropertyName("at")]
public long At { get; set; }
}
/// <summary>Probe 的 <c>fleet.heartbeat</c> 附加字段:更新器的状态。</summary>
public sealed class ProbeHeartbeatExtra
{
[JsonPropertyName("update")]
public UpdateStatus? Update { get; set; }
}
/// <summary>Probe 的 <c>fleet.hello</c> 附加字段:系统、架构和构建信息(只用于展示和更新)。</summary>
public sealed class ProbeHelloMeta
{
[JsonPropertyName("platform")]
public PlatformInfo? Platform { get; set; }
}
/// <summary><c>fleet.hello</c> 的附加字段。</summary> /// <summary><c>fleet.hello</c> 的附加字段。</summary>
public sealed class AgentHelloMeta public sealed class AgentHelloMeta
{ {
[JsonPropertyName("platform")]
public PlatformInfo? Platform { get; set; }
[JsonPropertyName("tls_spki")] [JsonPropertyName("tls_spki")]
public string? TlsSpki { get; set; } public string? TlsSpki { get; set; }
@@ -339,10 +244,6 @@ public sealed class HostSnapshot
[JsonPropertyName("net_tx_bps")] [JsonPropertyName("net_tx_bps")]
public long? NetTxBps { get; set; } public long? NetTxBps { get; set; }
/// <summary>已统计接口的内核报告链路速率之和(Mbit/s);驱动不支持或无法读取时为空,不代表互联网带宽。</summary>
[JsonPropertyName("net_link_mbps")]
public long? NetLinkMbps { get; set; }
[JsonPropertyName("uptime_s")] [JsonPropertyName("uptime_s")]
public long? UptimeSeconds { get; set; } public long? UptimeSeconds { get; set; }
@@ -392,9 +293,6 @@ public sealed class AgentHeartbeatExtra
[JsonPropertyName("queue_bytes")] [JsonPropertyName("queue_bytes")]
public long QueueBytes { get; set; } public long QueueBytes { get; set; }
[JsonPropertyName("update")]
public UpdateStatus? Update { get; set; }
} }
/// <summary>Agent 下发给 Probe 的一条线路任务(期望状态类型 <c>line_task_set</c> 里的一项)。</summary> /// <summary>Agent 下发给 Probe 的一条线路任务(期望状态类型 <c>line_task_set</c> 里的一项)。</summary>
@@ -533,13 +431,9 @@ public static class FragmentKeys
[JsonSerializable(typeof(HostSnapshot))] [JsonSerializable(typeof(HostSnapshot))]
[JsonSerializable(typeof(TestStatusParams))] [JsonSerializable(typeof(TestStatusParams))]
[JsonSerializable(typeof(AgentEventParams))] [JsonSerializable(typeof(AgentEventParams))]
[JsonSerializable(typeof(ProbeAgentList))] [JsonSerializable(typeof(ProbeAgentsParams))]
[JsonSerializable(typeof(UpdateOfferParams))] [JsonSerializable(typeof(ProbeAgentsResult))]
[JsonSerializable(typeof(UpdateStatus))]
[JsonSerializable(typeof(ProbeHeartbeatExtra))]
[JsonSerializable(typeof(ProbeHelloMeta))]
[JsonSerializable(typeof(AgentEnrollMeta))] [JsonSerializable(typeof(AgentEnrollMeta))]
[JsonSerializable(typeof(PlatformInfo))]
[JsonSerializable(typeof(AgentHelloMeta))] [JsonSerializable(typeof(AgentHelloMeta))]
[JsonSerializable(typeof(AgentHeartbeatExtra))] [JsonSerializable(typeof(AgentHeartbeatExtra))]
[JsonSerializable(typeof(RawChunkHeader))] [JsonSerializable(typeof(RawChunkHeader))]
@@ -56,6 +56,9 @@ public sealed class TaskSet
/// <summary>允许连接到这个 Agent 的 Probe 公钥。</summary> /// <summary>允许连接到这个 Agent 的 Probe 公钥。</summary>
[JsonPropertyName("peers")] [JsonPropertyName("peers")]
public List<PeerSpec> Peers { get; set; } = []; public List<PeerSpec> Peers { get; set; } = [];
[JsonPropertyName("probe_agents_version")]
public long ProbeAgentsVersion { get; set; }
} }
public static class TaskSchedule public static class TaskSchedule
@@ -1,27 +0,0 @@
namespace Monitor.Measurement.Tests;
public sealed class PlatformDetectorTests
{
[Theory]
[InlineData("PRETTY_NAME=\"Debian GNU/Linux 12 (bookworm)\"|NAME=\"Debian GNU/Linux\"|VERSION_ID=\"12\"|ID=debian", "Debian 12")]
[InlineData("NAME=\"OpenWrt\"|VERSION_ID=\"24.10.0\"|ID=openwrt", "OpenWrt 24.10")]
[InlineData("NAME=\"Alpine Linux\"|ID=alpine|VERSION_ID=3.22.1", "Alpine Linux 3.22")]
[InlineData("NAME=\"Ubuntu\"|VERSION_ID=\"24.04\"", "Ubuntu 24.04")]
[InlineData("ID=arch", "arch")]
[InlineData("", "Linux")]
public void LinuxName_IsTheDistroAndItsMajorMinorVersion(string osRelease, string expected)
{
Assert.Equal(expected, PlatformDetector.LinuxName(osRelease.Split('|', StringSplitOptions.RemoveEmptyEntries)));
}
[Fact]
public void Detect_ReportsOnlyCoarseInformation()
{
var platform = PlatformDetector.Detect();
Assert.Contains(platform.Os, new[] { "linux", "windows", "macos", "other" });
Assert.Contains(platform.Arch, new[] { "x64", "arm64", "arm", "x86" });
// 没有主机名、内核版本:名字很短,也不含小数点以外的版本细节
Assert.True((platform.Name ?? string.Empty).Length < 40);
Assert.DoesNotContain(Environment.MachineName, platform.Name ?? string.Empty, StringComparison.OrdinalIgnoreCase);
}
}
@@ -43,7 +43,7 @@ public static class AgentEndpoints
.Api("join_tokens_create", "Create a join token; the plaintext is returned only once", TokensTag) .Api("join_tokens_create", "Create a join token; the plaintext is returned only once", TokensTag)
.Returns<CreatedJoinTokenDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403); .Returns<CreatedJoinTokenDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403);
group.MapDelete("/join-tokens/{id}", RevokeTokenAsync) group.MapDelete("/join-tokens/{id}", RevokeTokenAsync)
.Api("join_tokens_revoke", "Revoke a join token; with purge=true delete the record for good", TokensTag) .Api("join_tokens_revoke", "Revoke a join token", TokensTag)
.Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403); .Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403);
group.MapGet("/probes", ListProbesAsync) group.MapGet("/probes", ListProbesAsync)
@@ -130,29 +130,21 @@ public static class AgentEndpoints
http.Response.Headers.CacheControl = "no-store"; http.Response.Headers.CacheControl = "no-store";
http.Response.Headers.Location = "/api/admin/join-tokens"; http.Response.Headers.Location = "/api/admin/join-tokens";
return TypedResults.Json(new CreatedJoinTokenDto(Mappers.ToDto(created.Info), created.Plaintext, new InstallCommandsDto(created.InstallCommands.Linux, created.InstallCommands.Macos, created.InstallCommands.Windows)), ApiJsonContext.Default.CreatedJoinTokenDto, statusCode: StatusCodes.Status201Created); return TypedResults.Json(new CreatedJoinTokenDto(Mappers.ToDto(created.Info), created.Plaintext, created.InstallCommand), ApiJsonContext.Default.CreatedJoinTokenDto, statusCode: StatusCodes.Status201Created);
} }
private static async Task<IResult> RevokeTokenAsync(JoinTokenService tokens, string id, bool? purge, CancellationToken ct) private static async Task<IResult> RevokeTokenAsync(JoinTokenService tokens, string id, CancellationToken ct)
{
if (purge == true)
{
await tokens.DeleteAsync(id, ct);
}
else
{ {
await tokens.RevokeAsync(id, ct); await tokens.RevokeAsync(id, ct);
}
return TypedResults.NoContent(); return TypedResults.NoContent();
} }
private static async Task<IResult> ListProbesAsync(ProbeService probes, CancellationToken ct) => private static async Task<IResult> ListProbesAsync(ProbeService probes, CancellationToken ct) =>
TypedResults.Json(new PageDto<ProbeDto>([.. (await probes.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoProbeDto); TypedResults.Json(new PageDto<ProbeDto>([.. (await probes.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoProbeDto);
private static async Task<IResult> CreateProbeAsync(HttpContext http, ProbeService probes, BackendOptions options, CreateProbeRequest? request, CancellationToken ct) private static async Task<IResult> CreateProbeAsync(HttpContext http, ProbeService probes, CreateProbeRequest? request, CancellationToken ct)
{ {
Result<ProbeIdentityFile> result = await probes.CreateAsync(request?.Name, PanelUrl(http, options), ct); Result<ProbeIdentityFile> result = await probes.CreateAsync(request?.Name, ct);
if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error)) if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error))
{ {
return Problems.From(http, error); return Problems.From(http, error);
@@ -179,9 +171,9 @@ public static class AgentEndpoints
return result.TryGet(out ProbeView? view, out FleetError? error) ? TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto) : Problems.From(http, error); return result.TryGet(out ProbeView? view, out FleetError? error) ? TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto) : Problems.From(http, error);
} }
private static async Task<IResult> RotateProbeAsync(HttpContext http, ProbeService probes, BackendOptions options, string id, CancellationToken ct) private static async Task<IResult> RotateProbeAsync(HttpContext http, ProbeService probes, string id, CancellationToken ct)
{ {
Result<ProbeIdentityFile> result = await probes.RotateAsync(id, PanelUrl(http, options), ct); Result<ProbeIdentityFile> result = await probes.RotateAsync(id, ct);
if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error)) if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error))
{ {
return Problems.From(http, error); return Problems.From(http, error);
@@ -214,9 +206,8 @@ public static class AgentEndpoints
return dropped ? TypedResults.NoContent() : Problems.Create(http, MonitorErrors.ProbeOffline, "this probe has no session with this agent"); return dropped ? TypedResults.NoContent() : Problems.Create(http, MonitorErrors.ProbeOffline, "this probe has no session with this agent");
} }
/// <summary>Probe 要连的面板地址:配置的公开地址,没有就用这次请求的地址。</summary>
private static string PanelUrl(HttpContext http, BackendOptions options) => (options.PublicUrl ?? $"{http.Request.Scheme}://{http.Request.Host}").TrimEnd('/');
private static ProbeIdentityDto ToIdentityDto(ProbeIdentityFile file) => private static ProbeIdentityDto ToIdentityDto(ProbeIdentityFile file) =>
new(file.ProbeId.Value, file.PrivateKey, file.Server, new ProbeLimitsDto(file.Limits.MaxTestMbps, file.Limits.MaxTotalPps)); new(
file.ProbeId.Value, file.PrivateKey, [.. file.Bootstrap.Select(b => new BootstrapAgentDto(b.AgentId.Value, b.Host, b.TcpPort, b.Spki))],
new ProbeLimitsDto(file.Limits.MaxTestMbps, file.Limits.MaxTotalPps));
} }
@@ -44,10 +44,6 @@ public static class AuthEndpoints
return Problems.From(http, error); return Problems.From(http, error);
} }
// 跨域的前端(return_session)不用 Cookie:会话令牌放在响应里,之后用 Bearer 带上
bool returnSession = request.ReturnSession == true;
if (!returnSession)
{
http.Response.Cookies.Append( http.Response.Cookies.Append(
PrincipalResolver.SessionCookie, PrincipalResolver.SessionCookie,
login.SessionToken, login.SessionToken,
@@ -59,20 +55,13 @@ public static class AuthEndpoints
Path = "/", Path = "/",
Expires = DateTimeOffset.FromUnixTimeMilliseconds(login.ExpiresAtMs), Expires = DateTimeOffset.FromUnixTimeMilliseconds(login.ExpiresAtMs),
}); });
}
http.Response.Headers.CacheControl = "no-store"; http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new LoginResponse(login.UserId, login.Username, login.CsrfToken, Mappers.Time(login.ExpiresAtMs), returnSession ? login.SessionToken : null), ApiJsonContext.Default.LoginResponse); return TypedResults.Json(new LoginResponse(login.UserId, login.Username, login.CsrfToken, Mappers.Time(login.ExpiresAtMs)), ApiJsonContext.Default.LoginResponse);
} }
private static async Task<IResult> LogoutAsync(HttpContext http, AuthService auth) private static async Task<IResult> LogoutAsync(HttpContext http, AuthService auth)
{ {
string? bearer = PrincipalResolver.BearerOf(http); if (http.Request.Cookies.TryGetValue(PrincipalResolver.SessionCookie, out string? session))
if (bearer is not null)
{
await auth.LogoutAsync(bearer, http.RequestAborted); // 登录时 return_session 拿到的会话令牌;API 令牌不是会话,这里什么也不会发生
}
else if (http.Request.Cookies.TryGetValue(PrincipalResolver.SessionCookie, out string? session))
{ {
await auth.LogoutAsync(session, http.RequestAborted); await auth.LogoutAsync(session, http.RequestAborted);
} }
@@ -1,63 +0,0 @@
using Fleet.Core;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Api;
/// <summary>构建信息与更新:后端和每个节点的版本、tag、构建时间,CI 签名的发布清单,以及「更新这个节点」的请求。更新本身由节点宿主机上的更新器验签并执行。</summary>
public static class BuildEndpoints
{
private const string Tag = "admin-build";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/build-info", GetAsync)
.Api("build_info_get", "Version, tag, commit, build time and update state of the backend and every agent and probe", Tag)
.Returns<BuildInfoDto>().ProducesProblems(401, 403);
group.MapGet("/releases", ListReleasesAsync)
.Api("releases_list", "The latest signed release manifest of each component", Tag).Returns<PageDto<ReleaseDto>>().ProducesProblems(401, 403);
group.MapPost("/nodes/{id}/update", RequestUpdateAsync)
.Api("nodes_update", "Ask an online agent or probe to install the release it has been offered", Tag).Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403, 409);
}
/// <summary>登记发布清单:挂在 <see cref="ReleaseGuard"/> 下,除了管理员,CI 用的 <c>release</c> 令牌也能调用(它只能做这一件事)。</summary>
public static void MapPublish(RouteGroupBuilder group)
{
group.MapPost("/releases", PublishReleaseAsync)
.Api("releases_publish", "Register a CI-signed release manifest; online nodes get it right away (their updaters verify the signature). Admin or release tokens", Tag)
.Returns<ReleaseDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403, 409);
}
private static async Task<IResult> GetAsync(HttpContext http, NodeInfoService nodes, ReleaseService releases, CancellationToken ct)
{
var items = (await nodes.ListAsync(ct))
.OrderBy(n => n.Kind, StringComparer.Ordinal)
.ThenBy(n => n.NodeId, StringComparer.Ordinal)
.Select(n => new NodeBuildDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Version, n.Tag, n.Commit, n.BuiltAt, ToDto(releases.StatusOf(new NodeId(n.NodeId)))))
.ToList();
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new BuildInfoDto(NodeEndpoints.Backend(withCommit: true), items), ApiJsonContext.Default.BuildInfoDto);
}
private static async Task<IResult> ListReleasesAsync(ReleaseService releases, CancellationToken ct) =>
TypedResults.Json(new PageDto<ReleaseDto>([.. (await releases.ListAsync(ct)).Select(ToDto)], null), ApiJsonContext.Default.PageDtoReleaseDto);
private static async Task<IResult> PublishReleaseAsync(HttpContext http, ReleaseService releases, PublishReleaseRequest request)
{
Result<ReleaseOffer> result = await releases.PublishAsync(request.Manifest ?? "", request.Signature ?? "", http.RequestAborted);
return result.TryGet(out ReleaseOffer? offer, out FleetError? error)
? TypedResults.Json(ToDto(offer), ApiJsonContext.Default.ReleaseDto, statusCode: StatusCodes.Status201Created)
: Problems.From(http, error);
}
private static async Task<IResult> RequestUpdateAsync(HttpContext http, ReleaseService releases, string id, CancellationToken ct) =>
await releases.RequestUpdateAsync(new NodeId(id), ct) ? TypedResults.NoContent() : Problems.Create(http, MonitorErrors.AgentOffline, "the node is not connected");
private static ReleaseDto ToDto(ReleaseOffer offer) => new(offer.Component, offer.Seq, offer.Tag, offer.Version, offer.Digest);
private static NodeUpdateDto? ToDto(UpdateStatus? status) =>
status is null ? null : new NodeUpdateDto(status.State, status.Current, status.Available?.Tag, status.Available?.Version, status.Error, status.At > 0 ? DateTimeOffset.FromUnixTimeSeconds(status.At) : null);
}
@@ -1,59 +0,0 @@
using Fleet.Core;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Api;
/// <summary>线路开关:哪些(Probe,Agent)要连。自动连接的总开关是设置 <c>fleet.auto_link</c>;这里看每一对的结果,并且手动强制开、关或取消手动设置。</summary>
public static class LinkEndpoints
{
private const string Tag = "admin-links";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/links", ListAsync)
.Api("links_list", "Every probe/agent pair: whether it is linked, the manual override and whether it is connected", Tag).Returns<LinkMatrixDto>().ProducesProblems(401, 403);
group.MapPut("/links/{probe_id}/{agent_id}", SetAsync)
.Api("links_set", "Force a link on or off, or give it back to the automatic setting (enabled: null)", Tag).Returns<LinkDto>().ProducesProblems(400, 401, 403, 404);
}
private static async Task<IResult> ListAsync(AgentService agents, ProbeService probes, LinkService links, CancellationToken ct)
{
LinkSnapshot snapshot = await links.SnapshotAsync(ct);
IReadOnlyList<AgentView> agentViews = await agents.ListAsync(ct);
var items = new List<LinkDto>();
foreach (ProbeView probe in (await probes.ListAsync(ct)).Where(p => !p.Revoked).OrderBy(p => p.Probe.Name, StringComparer.Ordinal))
{
foreach (AgentView agent in agentViews.OrderBy(a => a.Agent.Name, StringComparer.Ordinal))
{
items.Add(ToDto(snapshot, probe, agent));
}
}
return TypedResults.Json(new LinkMatrixDto(snapshot.Auto, items), ApiJsonContext.Default.LinkMatrixDto);
}
private static async Task<IResult> SetAsync(HttpContext http, AgentService agents, ProbeService probes, LinkService links, string probe_id, string agent_id, SetLinkRequest request, CancellationToken ct)
{
if (await probes.GetAsync(probe_id, ct) is not { Revoked: false } probe)
{
return Problems.Create(http, MonitorErrors.ProbeNotFound, "probe not found");
}
if (await agents.GetAsync(new NodeId(agent_id), ct) is not { } agent)
{
return Problems.Create(http, MonitorErrors.AgentNotFound, "agent not found");
}
await links.SetAsync(probe_id, agent_id, request.Enabled, ct);
await agents.AfterAgentSetChangedAsync(ct); // 重新下发:Probe 的 Agent 列表、Agent 允许的 Probe
return TypedResults.Json(ToDto(await links.SnapshotAsync(ct), probe, agent), ApiJsonContext.Default.LinkDto);
}
private static LinkDto ToDto(LinkSnapshot snapshot, ProbeView probe, AgentView agent) =>
new(probe.Probe.Id, probe.Probe.Name, agent.Agent.Id.Value, agent.Agent.Name, snapshot.IsLinked(probe.Probe.Id, agent.Agent.Id.Value), snapshot.OverrideOf(probe.Probe.Id, agent.Agent.Id.Value),
agent.Probes.Any(p => p.Id == probe.Probe.Id));
}
@@ -63,14 +63,9 @@ public static class SystemEndpoints
private static async Task<IResult> ListSettingsAsync(SettingsService settings, CancellationToken ct) => private static async Task<IResult> ListSettingsAsync(SettingsService settings, CancellationToken ct) =>
TypedResults.Json(new PageDto<SettingDto>([.. (await settings.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto); TypedResults.Json(new PageDto<SettingDto>([.. (await settings.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto);
private static async Task<IResult> PatchSettingsAsync(HttpContext http, SettingsService settings, AgentService agents, Dictionary<string, JsonElement> changes) private static async Task<IResult> PatchSettingsAsync(HttpContext http, SettingsService settings, Dictionary<string, JsonElement> changes)
{ {
Result<IReadOnlyList<SettingView>> result = await settings.PatchAsync(changes, http.RequestAborted); Result<IReadOnlyList<SettingView>> result = await settings.PatchAsync(changes, http.RequestAborted);
if (result.IsSuccess && changes.ContainsKey(SettingsService.AutoLink))
{
await agents.AfterAgentSetChangedAsync(http.RequestAborted); // 自动连接开关变了:重新下发各 Probe 的 Agent 列表和各 Agent 允许的 Probe
}
return result.TryGet(out IReadOnlyList<SettingView>? views, out FleetError? error) return result.TryGet(out IReadOnlyList<SettingView>? views, out FleetError? error)
? TypedResults.Json(new PageDto<SettingDto>([.. views.Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto) ? TypedResults.Json(new PageDto<SettingDto>([.. views.Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto)
: Problems.From(http, error); : Problems.From(http, error);
@@ -97,22 +97,17 @@ public static class ApiModule
QueryEndpoints.Map(publicGroup); QueryEndpoints.Map(publicGroup);
LiveEndpoints.Map(publicGroup); LiveEndpoints.Map(publicGroup);
HostEndpoints.Map(publicGroup); HostEndpoints.Map(publicGroup);
NodeEndpoints.Map(publicGroup);
CatalogEndpoints.Map(publicGroup); CatalogEndpoints.Map(publicGroup);
RouteGroupBuilder admin = app.MapGroup("/api/admin"); RouteGroupBuilder admin = app.MapGroup("/api/admin");
AuthEndpoints.MapLogin(admin); AuthEndpoints.MapLogin(admin);
BuildEndpoints.MapPublish(admin.MapGroup(string.Empty).AddEndpointFilter(ReleaseGuard.InvokeAsync));
RouteGroupBuilder guarded = admin.MapGroup(string.Empty).AddEndpointFilter(AdminGuard.InvokeAsync); RouteGroupBuilder guarded = admin.MapGroup(string.Empty).AddEndpointFilter(AdminGuard.InvokeAsync);
AuthEndpoints.MapSession(guarded); AuthEndpoints.MapSession(guarded);
AgentEndpoints.Map(guarded); AgentEndpoints.Map(guarded);
TaskEndpoints.Map(guarded); TaskEndpoints.Map(guarded);
DataEndpoints.Map(guarded); DataEndpoints.Map(guarded);
SystemEndpoints.Map(guarded); SystemEndpoints.Map(guarded);
BuildEndpoints.Map(guarded);
LinkEndpoints.Map(guarded);
OpsEndpoints.Map(app); OpsEndpoints.Map(app);
app.MapOpenApi("/openapi/v1.json"); app.MapOpenApi("/openapi/v1.json");
@@ -23,25 +23,14 @@ public static class PrincipalResolver
public const string SessionCookie = "monitor_session"; public const string SessionCookie = "monitor_session";
public const string CsrfHeader = "X-CSRF-Token"; public const string CsrfHeader = "X-CSRF-Token";
public static string? BearerOf(HttpContext http)
{
string? header = http.Request.Headers.Authorization;
return header is not null && header.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase) ? header["Bearer ".Length..].Trim() : null;
}
public static async Task ResolveAsync(HttpContext http, Func<Task> next) public static async Task ResolveAsync(HttpContext http, Func<Task> next)
{ {
AuthService auth = http.RequestServices.GetRequiredService<AuthService>(); AuthService auth = http.RequestServices.GetRequiredService<AuthService>();
Principal? principal = null; Principal? principal = null;
string? bearer = BearerOf(http); string? header = http.Request.Headers.Authorization;
if (bearer is not null) if (header is not null && header.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
{ {
principal = await auth.AuthenticateTokenAsync(bearer, http.RequestAborted); principal = await auth.AuthenticateTokenAsync(header["Bearer ".Length..].Trim(), http.RequestAborted);
if (principal is null && await auth.AuthenticateSessionAsync(bearer, http.RequestAborted) is { } viaSession)
{
// 登录时用 return_session 拿到的会话令牌:和 API 令牌一样走 Bearer,不是 Cookie,所以不需要 CSRF
principal = viaSession with { Kind = PrincipalKind.Token, CsrfToken = null };
}
} }
else if (http.Request.Cookies.TryGetValue(SessionCookie, out string? session) && !string.IsNullOrEmpty(session)) else if (http.Request.Cookies.TryGetValue(SessionCookie, out string? session) && !string.IsNullOrEmpty(session))
{ {
@@ -62,7 +51,7 @@ public static class AdminGuard
Principal principal = http.GetPrincipal(); Principal principal = http.GetPrincipal();
if (!principal.IsAdmin) if (!principal.IsAdmin)
{ {
return principal.Scope != ApiScope.None return principal.CanRead
? Problems.Create(http, MonitorErrors.Forbidden, "the token scope does not allow this operation") ? Problems.Create(http, MonitorErrors.Forbidden, "the token scope does not allow this operation")
: Problems.Create(http, MonitorErrors.Unauthorized, "authentication is required"); : Problems.Create(http, MonitorErrors.Unauthorized, "authentication is required");
} }
@@ -80,13 +69,6 @@ public static class AdminGuard
} }
} }
/// <summary>登记发布清单的守卫:管理员(会话要带 CSRF)或 release 令牌;release 令牌只能做这一件事。</summary>
public static class ReleaseGuard
{
public static async ValueTask<object?> InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next) =>
context.HttpContext.GetPrincipal().Scope == ApiScope.Release ? await next(context) : await AdminGuard.InvokeAsync(context, next);
}
/// <summary>展示层守卫:站点公开时匿名可读;私有时需要登录或只读令牌。</summary> /// <summary>展示层守卫:站点公开时匿名可读;私有时需要登录或只读令牌。</summary>
public static class PublicGuard public static class PublicGuard
{ {
@@ -34,13 +34,13 @@ public static class Mappers
line.Line.Agent.Online, line.Line.Connected, line.Line.Agent.EndpointChanges, ToDto(line.Summary), Summary(line.LastTest)); line.Line.Agent.Online, line.Line.Connected, line.Line.Agent.EndpointChanges, ToDto(line.Summary), Summary(line.LastTest));
public static HostDto ToDto(HostSnapshot h) => public static HostDto ToDto(HostSnapshot h) =>
new(h.CpuPercent, h.MemUsed, h.MemTotal, h.DiskUsed, h.DiskTotal, h.NetRxTotal, h.NetTxTotal, h.NetRxBps, h.NetTxBps, h.NetLinkMbps, h.UptimeSeconds, Time(h.AtUtcMs)); new(h.CpuPercent, h.MemUsed, h.MemTotal, h.DiskUsed, h.DiskTotal, h.NetRxTotal, h.NetTxTotal, h.NetRxBps, h.NetTxBps, h.UptimeSeconds, Time(h.AtUtcMs));
public static ConnectedProbeDto ToDto(ConnectedProbeInfo p) => new(p.Id, p.Version, p.Remote, Time(p.SinceMs)); public static ConnectedProbeDto ToDto(ConnectedProbeInfo p) => new(p.Id, p.Version, p.Remote, Time(p.SinceMs));
public static ProbeDto ToDto(ProbeView v) => public static ProbeDto ToDto(ProbeView v) =>
new( new(
v.Probe.Id, v.Probe.Name, Time(v.Probe.CreatedAtMs), Time(v.Probe.RevokedAtMs), v.Revoked, v.Online, v.Probe.Id, v.Probe.Name, Time(v.Probe.CreatedAtMs), Time(v.Probe.RevokedAtMs), v.Revoked,
[.. v.Connections.Select(c => new ProbeConnectionDto(c.AgentId.Value, c.AgentName, c.AgentOnline, c.Version, c.Remote, Time(c.SinceMs)))]); [.. v.Connections.Select(c => new ProbeConnectionDto(c.AgentId.Value, c.AgentName, c.AgentOnline, c.Version, c.Remote, Time(c.SinceMs)))]);
public static SeriesDto ToDto(SeriesResult r, bool live) => public static SeriesDto ToDto(SeriesResult r, bool live) =>
@@ -2,10 +2,9 @@ using System.Text.Json;
namespace Monitor.Backend.Api; namespace Monitor.Backend.Api;
/// <summary><c>return_session</c>:跨域的前端用——不设 Cookie,把会话令牌放在响应里,之后用 <c>Authorization: Bearer</c> 带上。</summary> public sealed record LoginRequest(string Username, string Password);
public sealed record LoginRequest(string Username, string Password, bool? ReturnSession);
public sealed record LoginResponse(string UserId, string Username, string CsrfToken, DateTimeOffset ExpiresAt, string? SessionToken); public sealed record LoginResponse(string UserId, string Username, string CsrfToken, DateTimeOffset ExpiresAt);
public sealed record MeDto(string Kind, string Scope, string? UserId, string? Username, string? CsrfToken); public sealed record MeDto(string Kind, string Scope, string? UserId, string? Username, string? CsrfToken);
@@ -47,9 +46,7 @@ public sealed record JoinTokenDto(string Id, string? Note, IReadOnlyList<string>
public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool? AutoApprove, int? MaxUses, int? ExpiresInHours); public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool? AutoApprove, int? MaxUses, int? ExpiresInHours);
/// <summary>令牌明文只在创建的响应里出现一次。</summary> /// <summary>令牌明文只在创建的响应里出现一次。</summary>
public sealed record InstallCommandsDto(string Linux, string Macos, string Windows); public sealed record CreatedJoinTokenDto(JoinTokenDto Token, string Plaintext, string InstallCommand);
public sealed record CreatedJoinTokenDto(JoinTokenDto Token, string Plaintext, InstallCommandsDto InstallCommands);
/// <summary>Agent 眼中连着的一个 Probe(Agent 上报)。</summary> /// <summary>Agent 眼中连着的一个 Probe(Agent 上报)。</summary>
public sealed record ConnectedProbeDto(string ProbeId, string? Version, string? Remote, DateTimeOffset ConnectedSince); public sealed record ConnectedProbeDto(string ProbeId, string? Version, string? Remote, DateTimeOffset ConnectedSince);
@@ -57,19 +54,14 @@ public sealed record ConnectedProbeDto(string ProbeId, string? Version, string?
/// <summary>某个 Probe 现在连着的一个 Agent。</summary> /// <summary>某个 Probe 现在连着的一个 Agent。</summary>
public sealed record ProbeConnectionDto(string AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, DateTimeOffset ConnectedSince); public sealed record ProbeConnectionDto(string AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, DateTimeOffset ConnectedSince);
public sealed record ProbeDto(string Id, string Name, DateTimeOffset CreatedAt, DateTimeOffset? RevokedAt, bool Revoked, bool Online, IReadOnlyList<ProbeConnectionDto> Connections); public sealed record ProbeDto(string Id, string Name, DateTimeOffset CreatedAt, DateTimeOffset? RevokedAt, bool Revoked, IReadOnlyList<ProbeConnectionDto> Connections);
/// <summary>一对(Probe,Agent)的线路开关:<c>linked</c> 是最终结果,<c>override</c> 是手动设置(<c>null</c> = 跟随自动连接),<c>connected</c> 是现在有没有连着。</summary> public sealed record BootstrapAgentDto(string AgentId, string Host, int TcpPort, string Spki);
public sealed record LinkDto(string ProbeId, string ProbeName, string AgentId, string AgentName, bool Linked, bool? Override, bool Connected);
public sealed record LinkMatrixDto(bool AutoLink, IReadOnlyList<LinkDto> Items);
public sealed record SetLinkRequest(bool? Enabled);
public sealed record ProbeLimitsDto(double MaxTestMbps, long MaxTotalPps); public sealed record ProbeLimitsDto(double MaxTestMbps, long MaxTotalPps);
/// <summary><c>probe.json</c>:含私钥,只在生成时返回一次。</summary> /// <summary><c>probe.json</c>:含私钥,只在生成时返回一次。</summary>
public sealed record ProbeIdentityDto(string ProbeId, string PrivateKey, string Server, ProbeLimitsDto Limits); public sealed record ProbeIdentityDto(string ProbeId, string PrivateKey, IReadOnlyList<BootstrapAgentDto> Bootstrap, ProbeLimitsDto Limits);
public sealed record CreateProbeRequest(string? Name); public sealed record CreateProbeRequest(string? Name);
@@ -129,16 +121,3 @@ public sealed record CreateUserRequest(string Username, string Password);
public sealed record ChangePasswordRequest(string Password); public sealed record ChangePasswordRequest(string Password);
public sealed record RecomputeDto(string State, IReadOnlyList<string> Metrics, long WindowsDone, long WindowsTotal, DateTimeOffset? From, DateTimeOffset? To, int QueueLength); public sealed record RecomputeDto(string State, IReadOnlyList<string> Metrics, long WindowsDone, long WindowsTotal, DateTimeOffset? From, DateTimeOffset? To, int QueueLength);
/// <summary>管理端看的构建信息:每个节点自报的版本、tag、提交和构建时间,以及系统。</summary>
public sealed record NodeBuildDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Version, string? Tag, string? Commit, string? BuiltAt, NodeUpdateDto? Update = null);
/// <summary>节点宿主机上的更新器报告的状态(<c>up_to_date</c>、<c>available</c>、<c>applying</c>、<c>failed</c>、<c>rolled_back</c>、<c>invalid</c>);没装更新器的节点没有这个字段。</summary>
public sealed record NodeUpdateDto(string State, string? CurrentDigest, string? AvailableTag, string? AvailableVersion, string? Error, DateTimeOffset? At);
/// <summary>面板登记的最新发布清单(CI 签名的;验签在节点上)。</summary>
public sealed record ReleaseDto(string Component, long Seq, string Tag, string Version, string Digest);
public sealed record PublishReleaseRequest(string? Manifest, string? Signature);
public sealed record BuildInfoDto(BuildDto Backend, IReadOnlyList<NodeBuildDto> Nodes);
@@ -37,18 +37,6 @@ namespace Monitor.Backend.Api;
[JsonSerializable(typeof(AgentDto))] [JsonSerializable(typeof(AgentDto))]
[JsonSerializable(typeof(PageDto<AgentDto>))] [JsonSerializable(typeof(PageDto<AgentDto>))]
[JsonSerializable(typeof(AgentPatchRequest))] [JsonSerializable(typeof(AgentPatchRequest))]
[JsonSerializable(typeof(LinkDto))]
[JsonSerializable(typeof(LinkMatrixDto))]
[JsonSerializable(typeof(SetLinkRequest))]
[JsonSerializable(typeof(BuildDto))]
[JsonSerializable(typeof(ReleaseDto))]
[JsonSerializable(typeof(PageDto<ReleaseDto>))]
[JsonSerializable(typeof(PublishReleaseRequest))]
[JsonSerializable(typeof(AboutDto))]
[JsonSerializable(typeof(NodeBuildDto))]
[JsonSerializable(typeof(BuildInfoDto))]
[JsonSerializable(typeof(NodeInfoDto))]
[JsonSerializable(typeof(PageDto<NodeInfoDto>))]
[JsonSerializable(typeof(JoinTokenDto))] [JsonSerializable(typeof(JoinTokenDto))]
[JsonSerializable(typeof(PageDto<JoinTokenDto>))] [JsonSerializable(typeof(PageDto<JoinTokenDto>))]
[JsonSerializable(typeof(CreateJoinTokenRequest))] [JsonSerializable(typeof(CreateJoinTokenRequest))]
@@ -163,20 +163,10 @@ public sealed record HostDto(
long? NetTxTotal, long? NetTxTotal,
long? NetRxBps, long? NetRxBps,
long? NetTxBps, long? NetTxBps,
long? NetLinkMbps,
long? UptimeSeconds, long? UptimeSeconds,
DateTimeOffset At); DateTimeOffset At);
/// <summary>一个 Agent 的最新主机状态;<c>stale</c> 表示超过 15 秒没有更新(离线,或者 Agent 关闭了上报)。</summary> /// <summary>一个 Agent 的最新主机状态;<c>stale</c> 表示超过 15 秒没有更新(离线,或者 Agent 关闭了上报)。</summary>
/// <summary>节点的展示信息:只有系统、架构、国家代码、运营商,没有任何地址;站点打开「显示构建信息」之后还有版本和 tag。</summary>
public sealed record NodeInfoDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Country, string? IspKey, string? IspName, string? Version = null, string? Tag = null);
/// <summary>一个程序的构建信息:版本、tag、提交(短)和构建时间(UTC,ISO 8601)。没设置的是 <c>null</c>。</summary>
public sealed record BuildDto(string Version, string? Tag, string? Commit, string? BuiltAt);
/// <summary>公开的「关于」:站点没有打开「显示构建信息」时只有 <c>show_build: false</c>。</summary>
public sealed record AboutDto(bool ShowBuild, BuildDto? Backend);
public sealed record HostItemDto(string AgentId, string Name, bool Online, bool Stale, DateTimeOffset? LastSeenAt, HostDto? Host); public sealed record HostItemDto(string AgentId, string Name, bool Online, bool Stale, DateTimeOffset? LastSeenAt, HostDto? Host);
public sealed record LiveHostDto(string AgentId, bool Online, HostDto Host); public sealed record LiveHostDto(string AgentId, bool Online, HostDto Host);
@@ -1,69 +0,0 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Monitor.Backend.Application;
namespace Monitor.Backend.Api;
/// <summary>
/// 节点的展示信息:系统、架构、国家、运营商。<b>只有这几项,没有任何 IP 地址、城市、ASN、经纬度</b>——
/// 这些在 Backend 里只在查询的那一刻用到,存起来的记录里也没有,所以访客(和任何别的接口)都拿不到。
/// </summary>
public static class NodeEndpoints
{
private const string Tag = "public-nodes";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/nodes", ListAsync)
.Api("nodes_list", "Display info of the visible agents and probes: system, architecture, country and ISP (never an address)", Tag)
.Returns<PageDto<NodeInfoDto>>().ProducesProblems(403);
group.MapGet("/about", AboutAsync)
.Api("about_get", "The backend's version, tag and build time, if the site shows build info", Tag)
.Returns<AboutDto>().ProducesProblems(403);
}
/// <summary>这个后端自己的构建信息;没有(本地构建)的字段是 <c>null</c>。</summary>
internal static BuildDto Backend(bool withCommit) =>
new(Monitor.Protocol.BuildInfo.Version, Empty(Monitor.Protocol.BuildInfo.Tag), withCommit ? Empty(Monitor.Protocol.BuildInfo.Commit) : null, Empty(Monitor.Protocol.BuildInfo.BuiltAt));
private static string? Empty(string value) => value.Length == 0 ? null : value;
private static async Task<IResult> AboutAsync(HttpContext http, SettingsService settings)
{
bool show = await settings.IsShowBuildAsync(http.RequestAborted);
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new AboutDto(show, show ? Backend(withCommit: false) : null), ApiJsonContext.Default.AboutDto);
}
private static async Task<IResult> ListAsync(HttpContext http, AgentService agents, ProbeService probes, NodeInfoService nodes, SettingsService settings, CancellationToken ct)
{
// 和别的公开接口一样:看得见的 Agent(隐藏的只有登录后才看得见)和没有被吊销的 Probe
var visible = new HashSet<string>(StringComparer.Ordinal);
foreach (AgentView view in await agents.ListAsync(ct))
{
if (view.Agent.Status == Fleet.Server.NodeStatus.Active && (view.Agent.PublicVisible || PublicGuard.CanSeeHidden(http)))
{
visible.Add(view.Agent.Id.Value);
}
}
foreach (ProbeView probe in await probes.ListAsync(ct))
{
if (!probe.Revoked)
{
visible.Add(probe.Probe.Id);
}
}
// 版本和 tag 只在站点打开「显示构建信息」时才给访客;关着就是 null
bool showBuild = await settings.IsShowBuildAsync(ct);
var items = (await nodes.ListAsync(ct))
.Where(n => visible.Contains(n.NodeId))
.OrderBy(n => n.NodeId, StringComparer.Ordinal)
.Select(n => new NodeInfoDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Country, n.IspKey, n.IspName, showBuild ? n.Version : null, showBuild ? n.Tag : null))
.ToList();
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new PageDto<NodeInfoDto>(items, null), ApiJsonContext.Default.PageDtoNodeInfoDto);
}
}
@@ -4,10 +4,7 @@ namespace Monitor.Backend.Application;
public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool AutoApprove, int MaxUses, int ExpiresInHours); public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool AutoApprove, int MaxUses, int ExpiresInHours);
/// <summary>各个系统上的安装命令(Linux 和 macOS 都用 install.sh,Windows 用 install.ps1)。</summary> public sealed record CreatedJoinToken(JoinTokenInfo Info, string Plaintext, string InstallCommand);
public sealed record InstallCommands(string Linux, string Macos, string Windows);
public sealed record CreatedJoinToken(JoinTokenInfo Info, string Plaintext, InstallCommands InstallCommands);
/// <summary>加入令牌:明文只在创建时返回一次,服务端只保存 SHA-256。</summary> /// <summary>加入令牌:明文只在创建时返回一次,服务端只保存 SHA-256。</summary>
public sealed class JoinTokenService public sealed class JoinTokenService
@@ -58,16 +55,11 @@ public sealed class JoinTokenService
request.MaxUses, now + (request.ExpiresInHours * 3_600_000L), now, ct); request.MaxUses, now + (request.ExpiresInHours * 3_600_000L), now, ct);
string baseUrl = (_options.PublicUrl ?? requestBaseUrl ?? "https://panel.example.com").TrimEnd('/'); string baseUrl = (_options.PublicUrl ?? requestBaseUrl ?? "https://panel.example.com").TrimEnd('/');
string shell = $"curl -fsSL {baseUrl}/install.sh | sh -s -- --join {plaintext} --name <node-name>"; string command = $"curl -fsSL {baseUrl}/install.sh | sh -s -- --join {plaintext} --name <node-name>";
// PowerShell 6+(pwsh):在 cmd 或 PowerShell 里都能直接粘贴 return Result<CreatedJoinToken>.Ok(new CreatedJoinToken(info, plaintext, command));
string powershell = $"pwsh -NoProfile -ExecutionPolicy Bypass -Command \"& ([scriptblock]::Create((irm {baseUrl}/install.ps1))) -Join {plaintext} -Name <node-name>\"";
return Result<CreatedJoinToken>.Ok(new CreatedJoinToken(info, plaintext, new InstallCommands(shell, shell, powershell)));
} }
public ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct) => _store.ListAsync(ct); public ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct) => _store.ListAsync(ct);
public ValueTask<bool> RevokeAsync(string id, CancellationToken ct) => _store.RevokeAsync(id, _time.GetUtcNow().ToUnixTimeMilliseconds(), ct); public ValueTask<bool> RevokeAsync(string id, CancellationToken ct) => _store.RevokeAsync(id, _time.GetUtcNow().ToUnixTimeMilliseconds(), ct);
/// <summary>彻底删除这条令牌记录(任何状态都可以);还没用完的令牌删除后立即不能再用。</summary>
public ValueTask<bool> DeleteAsync(string id, CancellationToken ct) => _store.DeleteAsync(id, ct);
} }
@@ -12,8 +12,6 @@ public sealed record SettingView(string Key, string Type, string Value, string D
public sealed class SettingsService public sealed class SettingsService
{ {
public const string SitePublic = "site.public"; public const string SitePublic = "site.public";
public const string SiteShowBuild = "site.show_build";
public const string AutoLink = "fleet.auto_link";
public const string RawDays = "retention.raw_days"; public const string RawDays = "retention.raw_days";
public const string TestRawDays = "retention.test_raw_days"; public const string TestRawDays = "retention.test_raw_days";
public const string Metric10sHours = "retention.metric_10s_hours"; public const string Metric10sHours = "retention.metric_10s_hours";
@@ -42,8 +40,6 @@ public sealed class SettingsService
private static IReadOnlyList<SettingSpec> Known(BackendOptions o) => private static IReadOnlyList<SettingSpec> Known(BackendOptions o) =>
[ [
new(SitePublic, "bool", "false", 0, 1, "Whether anonymous visitors can read the public API."), new(SitePublic, "bool", "false", 0, 1, "Whether anonymous visitors can read the public API."),
new(SiteShowBuild, "bool", "false", 0, 1, "Whether the public pages show the versions, tags and build times of the backend, agents and probes."),
new(AutoLink, "bool", "true", 0, 1, "Whether every probe connects to every agent automatically; when off, only the links chosen by hand are connected."),
new(RawDays, "int", o.RawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of continuous tasks."), new(RawDays, "int", o.RawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of continuous tasks."),
new(TestRawDays, "int", o.TestRawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of on-demand tests."), new(TestRawDays, "int", o.TestRawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of on-demand tests."),
new(Metric10sHours, "int", o.Metric10sRetentionHours.ToString(CultureInfo.InvariantCulture), 1, 720, "Hours to keep the 10 s metric layer."), new(Metric10sHours, "int", o.Metric10sRetentionHours.ToString(CultureInfo.InvariantCulture), 1, 720, "Hours to keep the 10 s metric layer."),
@@ -118,9 +114,6 @@ public sealed class SettingsService
return _sitePublic; return _sitePublic;
} }
/// <summary>访客能不能看到版本、tag 和构建时间(站点公开之后的另一道开关,默认关)。</summary>
public async ValueTask<bool> IsShowBuildAsync(CancellationToken ct) => await _store.GetAsync(SiteShowBuild, ct) == "true";
public async ValueTask<int> GetIntAsync(string key, CancellationToken ct) public async ValueTask<int> GetIntAsync(string key, CancellationToken ct)
{ {
SettingSpec spec = Specs.First(s => s.Key == key); SettingSpec spec = Specs.First(s => s.Key == key);
@@ -9,14 +9,12 @@ public sealed class AgentEventService
private readonly AgentRuntime _runtime; private readonly AgentRuntime _runtime;
private readonly IEventStore _events; private readonly IEventStore _events;
private readonly TimeProvider _time; private readonly TimeProvider _time;
private readonly NodeInfoService _nodes;
public AgentEventService(AgentRuntime runtime, IEventStore events, TimeProvider time, NodeInfoService nodes) public AgentEventService(AgentRuntime runtime, IEventStore events, TimeProvider time)
{ {
_runtime = runtime; _runtime = runtime;
_events = events; _events = events;
_time = time; _time = time;
_nodes = nodes;
} }
public async ValueTask OnEventAsync(NodeId agent, AgentEventParams e, CancellationToken ct) public async ValueTask OnEventAsync(NodeId agent, AgentEventParams e, CancellationToken ct)
@@ -35,7 +33,6 @@ public sealed class AgentEventService
{ {
case AgentEventKinds.ProbeConnected when !string.IsNullOrEmpty(e.Detail): case AgentEventKinds.ProbeConnected when !string.IsNullOrEmpty(e.Detail):
state.AddProbe(e.Probe is { } p ? new ConnectedProbeInfo(e.Detail, p.Version, p.Remote, p.SinceUtcMs > 0 ? p.SinceUtcMs : at) : new ConnectedProbeInfo(e.Detail, null, null, at)); state.AddProbe(e.Probe is { } p ? new ConnectedProbeInfo(e.Detail, p.Version, p.Remote, p.SinceUtcMs > 0 ? p.SinceUtcMs : at) : new ConnectedProbeInfo(e.Detail, null, null, at));
_nodes.ObserveProbeInBackground(e.Detail, e.Probe?.Remote);
break; break;
case AgentEventKinds.ProbeDisconnected when !string.IsNullOrEmpty(e.Detail): case AgentEventKinds.ProbeDisconnected when !string.IsNullOrEmpty(e.Detail):
state.RemoveProbe(e.Detail); state.RemoveProbe(e.Detail);
@@ -20,13 +20,8 @@ public sealed partial class AgentService
private readonly TaskPlanner _planner; private readonly TaskPlanner _planner;
private readonly ISettingsStore _settings; private readonly ISettingsStore _settings;
private readonly NodeInfoService _nodes; public AgentService(IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TaskPlanner planner, ISettingsStore settings)
private readonly LinkService _links;
public AgentService(IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TaskPlanner planner, ISettingsStore settings, NodeInfoService nodes, LinkService links)
{ {
_links = links;
_nodes = nodes;
_agents = agents; _agents = agents;
_runtime = runtime; _runtime = runtime;
_gateway = gateway; _gateway = gateway;
@@ -119,20 +114,17 @@ public sealed partial class AgentService
{ {
await _gateway.DisconnectAsync(id, "deleted", ct); await _gateway.DisconnectAsync(id, "deleted", ct);
await _agents.DeleteAsync(id, ct); await _agents.DeleteAsync(id, ct);
await _nodes.ForgetAsync(id.Value, ct);
await _links.ForgetNodeAsync(id.Value, ct);
_runtime.Remove(id); _runtime.Remove(id);
await AfterAgentSetChangedAsync(ct); await AfterAgentSetChangedAsync(ct);
} }
/// <summary>这个 Probe 要连接的 Agent 列表(已激活、有公开地址,并且线路是开着的)。版本号在列表变化时递增。</summary> /// <summary>Probe 需要连接的 Agent 列表(所有已激活的 Agent)。版本号在列表变化时递增。</summary>
public async ValueTask<ProbeAgentList> ProbeAgentsAsync(string probeId, CancellationToken ct) public async ValueTask<ProbeAgentsResult> ProbeAgentsAsync(CancellationToken ct)
{ {
LinkSnapshot links = await _links.SnapshotAsync(ct);
IReadOnlyList<AgentRecord> agents = await _agents.ListAsync(ct); IReadOnlyList<AgentRecord> agents = await _agents.ListAsync(ct);
long version = long.TryParse(await _settings.GetAsync(TaskPlanner.ProbeAgentsVersionKey, ct), out long v) ? v : 0; long version = long.TryParse(await _settings.GetAsync(TaskPlanner.ProbeAgentsVersionKey, ct), out long v) ? v : 0;
var result = new ProbeAgentList { Version = version }; var result = new ProbeAgentsResult { Version = version };
foreach (AgentRecord agent in agents.Where(a => a.Status == NodeStatus.Active && a.PublicHost.Length > 0 && links.IsLinked(probeId, a.Id.Value))) foreach (AgentRecord agent in agents.Where(a => a.Status == NodeStatus.Active && a.PublicHost.Length > 0))
{ {
result.Agents.Add(new AgentEndpoint result.Agents.Add(new AgentEndpoint
{ {
@@ -148,11 +140,12 @@ public sealed partial class AgentService
return result; return result;
} }
/// <summary>Agent 集合变了:递增列表版本,重算各 Agent 的期望状态,并把新的 Agent 列表推给 Probe(都是期望状态,一起通知)。</summary> /// <summary>Agent 集合变了:递增列表版本、重算各 Agent 的期望状态、通知 Probe。</summary>
public async ValueTask AfterAgentSetChangedAsync(CancellationToken ct) public async ValueTask AfterAgentSetChangedAsync(CancellationToken ct)
{ {
await _planner.BumpProbeAgentsVersionAsync(ct); await _planner.BumpProbeAgentsVersionAsync(ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
await _gateway.NotifyProbeAgentsChangedAsync(await ProbeAgentsAsync(ct), ct);
} }
private async ValueTask<Result<AgentView>> ChangeStatusAsync(NodeId id, NodeStatus status, CancellationToken ct) private async ValueTask<Result<AgentView>> ChangeStatusAsync(NodeId id, NodeStatus status, CancellationToken ct)
@@ -1,68 +0,0 @@
namespace Monitor.Backend.Application;
/// <summary>某一时刻的「线路」开关:自动连接的总开关,加上每一对(Probe,Agent)的手动设置。</summary>
public sealed class LinkSnapshot
{
private readonly Dictionary<(string Probe, string Agent), bool> _overrides;
public LinkSnapshot(bool auto, Dictionary<(string Probe, string Agent), bool> overrides)
{
Auto = auto;
_overrides = overrides;
}
/// <summary>自动连接:开着时,没有手动设置的 Probe 和 Agent 都会连;关着时,只有手动开启的才连。</summary>
public bool Auto { get; }
/// <summary>手动设置:<c>true</c> 强制连、<c>false</c> 强制不连、<c>null</c> 没设置(跟随自动连接)。</summary>
public bool? OverrideOf(string probeId, string agentId) => _overrides.TryGetValue((probeId, agentId), out bool value) ? value : null;
/// <summary>这一对最终连不连:手动设置优先,没有就看自动连接。</summary>
public bool IsLinked(string probeId, string agentId) => OverrideOf(probeId, agentId) ?? Auto;
}
/// <summary>
/// 哪些(Probe,Agent)要连成线路。由面板决定、Probe 主动拨出:自动连接开着时所有 Probe 和 Agent 两两连(默认),
/// 关着时只连手动选中的;每一对还可以手动强制开或关。手动设置存在设置表里(<c>link.&lt;probe&gt;.&lt;agent&gt;</c>,值 1 或 0,空表示没设置)。
/// 判断结果分别用在 Probe 的 Agent 列表、Agent 允许的 Probe(<c>peers</c>)和线路目录里;改动之后要调用 <see cref="AgentService.AfterAgentSetChangedAsync"/> 重新下发。
/// </summary>
public sealed class LinkService
{
private const string Prefix = "link.";
private readonly ISettingsStore _settings;
public LinkService(ISettingsStore settings)
{
_settings = settings;
}
public async ValueTask<LinkSnapshot> SnapshotAsync(CancellationToken ct)
{
IReadOnlyDictionary<string, string> all = await _settings.GetAllAsync(ct);
var overrides = new Dictionary<(string, string), bool>();
foreach ((string key, string value) in all)
{
string[] parts = key.Split('.');
if (parts.Length == 3 && parts[0] + "." == Prefix && value is "1" or "0")
{
overrides[(parts[1], parts[2])] = value == "1";
}
}
return new LinkSnapshot(all.GetValueOrDefault(SettingsService.AutoLink) != "false", overrides);
}
/// <summary>手动设置一对线路:<c>true</c> 强制连、<c>false</c> 强制不连、<c>null</c> 取消手动设置。</summary>
public ValueTask SetAsync(string probeId, string agentId, bool? enabled, CancellationToken ct) =>
_settings.SetAsync($"{Prefix}{probeId}.{agentId}", enabled is null ? "" : enabled.Value ? "1" : "0", ct);
/// <summary>Probe 或 Agent 被删除:清掉所有涉及它的手动设置。</summary>
public async ValueTask ForgetNodeAsync(string nodeId, CancellationToken ct)
{
foreach (string key in (await _settings.GetAllAsync(ct)).Keys.Where(k => k.StartsWith(Prefix, StringComparison.Ordinal) && k.Split('.').Skip(1).Contains(nodeId)))
{
await _settings.SetAsync(key, "", ct);
}
}
}
@@ -51,13 +51,9 @@ public sealed class MonitorNodeEventSink : INodeEventSink
private readonly IEventStore _events; private readonly IEventStore _events;
private readonly LiveHub _live; private readonly LiveHub _live;
private readonly AgentService _service; private readonly AgentService _service;
private readonly NodeInfoService _nodes;
private readonly ReleaseService _releases;
public MonitorNodeEventSink(IAgentStore agents, AgentRuntime runtime, IEventStore events, LiveHub live, AgentService service, NodeInfoService nodes, ReleaseService releases) public MonitorNodeEventSink(IAgentStore agents, AgentRuntime runtime, IEventStore events, LiveHub live, AgentService service)
{ {
_releases = releases;
_nodes = nodes;
_agents = agents; _agents = agents;
_runtime = runtime; _runtime = runtime;
_events = events; _events = events;
@@ -68,12 +64,6 @@ public sealed class MonitorNodeEventSink : INodeEventSink
public async ValueTask OnEventAsync(NodeEvent nodeEvent, CancellationToken ct) public async ValueTask OnEventAsync(NodeEvent nodeEvent, CancellationToken ct)
{ {
NodeId id = nodeEvent.Node.Id; NodeId id = nodeEvent.Node.Id;
if (id.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
await OnProbeEventAsync(id, nodeEvent, ct);
return;
}
switch (nodeEvent.Kind) switch (nodeEvent.Kind)
{ {
case NodeEventKind.Online: case NodeEventKind.Online:
@@ -82,7 +72,6 @@ public sealed class MonitorNodeEventSink : INodeEventSink
break; break;
case NodeEventKind.Offline: case NodeEventKind.Offline:
_runtime.MarkOffline(id); _runtime.MarkOffline(id);
_releases.RecordStatus(id, null);
_live.Publish(new NodeSignal("agent", id.Value, false, nodeEvent.AtMs)); _live.Publish(new NodeSignal("agent", id.Value, false, nodeEvent.AtMs));
await _events.AddAsync(id, "agent_offline", nodeEvent.AtMs, null, ct); await _events.AddAsync(id, "agent_offline", nodeEvent.AtMs, null, ct);
break; break;
@@ -96,58 +85,11 @@ public sealed class MonitorNodeEventSink : INodeEventSink
AgentHeartbeatExtra? beat = extra.Deserialize(MonitorJsonContext.Default.AgentHeartbeatExtra); AgentHeartbeatExtra? beat = extra.Deserialize(MonitorJsonContext.Default.AgentHeartbeatExtra);
state.LastHeartbeat = beat; state.LastHeartbeat = beat;
state.LastHeartbeatAtMs = nodeEvent.AtMs; state.LastHeartbeatAtMs = nodeEvent.AtMs;
_releases.RecordStatus(id, beat?.Update);
if (beat is not null) if (beat is not null)
{ {
state.ReplaceProbes(beat.Probes.Select(p => new ConnectedProbeInfo(p.Id.Value, p.Version, p.Remote, p.SinceUtcMs))); state.ReplaceProbes(beat.Probes.Select(p => new ConnectedProbeInfo(p.Id.Value, p.Version, p.Remote, p.SinceUtcMs)));
foreach (ConnectedProbe p in beat.Probes)
{
_nodes.ObserveProbeInBackground(p.Id.Value, p.Remote);
} }
} }
}
catch (JsonException)
{
// 忽略格式不正确的附加字段
}
break;
}
}
/// <summary>Probe 自己连上来了:hello 里带着它的系统、架构和构建信息;上线下线只推给 SSE,不记事件。</summary>
private async ValueTask OnProbeEventAsync(NodeId id, NodeEvent nodeEvent, CancellationToken ct)
{
switch (nodeEvent.Kind)
{
case NodeEventKind.Online:
_live.Publish(new NodeSignal("probe", id.Value, true, nodeEvent.AtMs));
break;
case NodeEventKind.Hello:
if (nodeEvent.Hello?.Meta is { } raw)
{
try
{
ProbeHelloMeta? meta = raw.Deserialize(MonitorJsonContext.Default.ProbeHelloMeta);
await _nodes.SetPlatformAsync(id.Value, NodeKinds.Probe, meta?.Platform, ct);
}
catch (JsonException)
{
// 格式不对就忽略
}
}
await _releases.OfferToNodeAsync(id, ct);
break;
case NodeEventKind.Offline:
_live.Publish(new NodeSignal("probe", id.Value, false, nodeEvent.AtMs));
_releases.RecordStatus(id, null);
break;
case NodeEventKind.Heartbeat when nodeEvent.Extra is { } extra:
try
{
_releases.RecordStatus(id, extra.Deserialize(MonitorJsonContext.Default.ProbeHeartbeatExtra)?.Update);
}
catch (JsonException) catch (JsonException)
{ {
// 忽略格式不正确的附加字段 // 忽略格式不正确的附加字段
@@ -200,8 +142,6 @@ public sealed class MonitorNodeEventSink : INodeEventSink
// 端口或证书指纹变了,Probe 需要新的列表 // 端口或证书指纹变了,Probe 需要新的列表
AgentRecord? after = await _agents.GetAsync(id, ct); AgentRecord? after = await _agents.GetAsync(id, ct);
_nodes.ObserveAgentInBackground(id.Value, after?.PublicHost, meta.Platform);
await _releases.OfferToNodeAsync(id, ct);
if (before is not null && after is not null if (before is not null && after is not null
&& (!before.TlsSpki.AsSpan().SequenceEqual(after.TlsSpki) || before.ProbeTcpPort != after.ProbeTcpPort || !before.ProbeUdpPorts.SequenceEqual(after.ProbeUdpPorts)) && (!before.TlsSpki.AsSpan().SequenceEqual(after.TlsSpki) || before.ProbeTcpPort != after.ProbeTcpPort || !before.ProbeUdpPorts.SequenceEqual(after.ProbeUdpPorts))
&& after.Status == NodeStatus.Active) && after.Status == NodeStatus.Active)
@@ -6,47 +6,47 @@ using Monitor.Protocol;
namespace Monitor.Backend.Application; namespace Monitor.Backend.Application;
public sealed record BootstrapAgent(NodeId AgentId, string Host, int TcpPort, string Spki);
/// <summary>本地硬上限:写在 Probe 本机的配置里,面板无法修改(在路由器上建议调低)。</summary> /// <summary>本地硬上限:写在 Probe 本机的配置里,面板无法修改(在路由器上建议调低)。</summary>
public sealed record ProbeLimits(double MaxTestMbps, long MaxTotalPps); public sealed record ProbeLimits(double MaxTestMbps, long MaxTotalPps);
/// <summary><c>probe.json</c>:只在生成(或轮换密钥)时返回一次,含私钥;<c>Server</c> 是 Probe 要连的面板地址。</summary> /// <summary><c>probe.json</c>:只在生成(或轮换密钥)时返回一次,含私钥。</summary>
public sealed record ProbeIdentityFile(ProbeId ProbeId, string PrivateKey, string Server, ProbeLimits Limits); public sealed record ProbeIdentityFile(ProbeId ProbeId, string PrivateKey, IReadOnlyList<BootstrapAgent> Bootstrap, ProbeLimits Limits);
/// <summary>某个 Probe 现在连着的一个 Agent(Agent 上报)。</summary> /// <summary>某个 Probe 现在连着的一个 Agent(Agent 上报)。</summary>
public sealed record ProbeConnection(NodeId AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, long SinceMs); public sealed record ProbeConnection(NodeId AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, long SinceMs);
public sealed record ProbeView(ProbeRecord Probe, bool Online, IReadOnlyList<ProbeConnection> Connections) public sealed record ProbeView(ProbeRecord Probe, IReadOnlyList<ProbeConnection> Connections)
{ {
public bool Revoked => Probe.RevokedAtMs is not null; public bool Revoked => Probe.RevokedAtMs is not null;
} }
/// <summary> /// <summary>
/// Probe 身份(Backend 生成,私钥只返回一次):可以有多个,每个都能改名、轮换密钥、吊销、删除。 /// Probe 身份(Backend 生成,私钥只返回一次):可以有多个,每个都能改名、轮换密钥、吊销、删除。
/// 所有未吊销的 Probe 的公钥都会写进每个 Agent 的期望状态(<c>peers</c>);吊销、删除、轮换密钥之后,Agent 立刻断开对应会话,Probe 到面板的连接也会被拒绝。 /// 所有未吊销的 Probe 的公钥都会写进每个 Agent 的期望状态(<c>peers</c>);吊销、删除、轮换密钥之后,Agent 立刻断开对应会话。
/// 一条线路是一对(Probe,Agent)。 /// 一条线路是一对(Probe,Agent)。
/// </summary> /// </summary>
public sealed partial class ProbeService public sealed partial class ProbeService
{ {
private readonly IProbeStore _probes; private readonly IProbeStore _probes;
private readonly IAgentStore _agents; private readonly IAgentStore _agents;
private readonly AgentService _agentService;
private readonly AgentRuntime _runtime; private readonly AgentRuntime _runtime;
private readonly IAgentGateway _gateway; private readonly IAgentGateway _gateway;
private readonly TimeProvider _time; private readonly TimeProvider _time;
private readonly NodeInfoService _nodes;
private readonly LinkService _links;
public ProbeService(IProbeStore probes, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TimeProvider time, NodeInfoService nodes, LinkService links) public ProbeService(IProbeStore probes, IAgentStore agents, AgentService agentService, AgentRuntime runtime, IAgentGateway gateway, TimeProvider time)
{ {
_links = links;
_nodes = nodes;
_probes = probes; _probes = probes;
_agents = agents; _agents = agents;
_agentService = agentService;
_runtime = runtime; _runtime = runtime;
_gateway = gateway; _gateway = gateway;
_time = time; _time = time;
} }
public async ValueTask<Result<ProbeIdentityFile>> CreateAsync(string? name, string server, CancellationToken ct) public async ValueTask<Result<ProbeIdentityFile>> CreateAsync(string? name, CancellationToken ct)
{ {
string finalName = string.IsNullOrWhiteSpace(name) ? "probe" : name.Trim(); string finalName = string.IsNullOrWhiteSpace(name) ? "probe" : name.Trim();
if (!NamePattern().IsMatch(finalName)) if (!NamePattern().IsMatch(finalName))
@@ -63,7 +63,7 @@ public sealed partial class ProbeService
var probe = new ProbeRecord(Ulid.NewPrefixed(ProbeId.Prefix, _time), finalName, identity.PublicKeySpki, _time.GetUtcNow().ToUnixTimeMilliseconds(), null); var probe = new ProbeRecord(Ulid.NewPrefixed(ProbeId.Prefix, _time), finalName, identity.PublicKeySpki, _time.GetUtcNow().ToUnixTimeMilliseconds(), null);
await _probes.SaveAsync(probe, ct); await _probes.SaveAsync(probe, ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
return Result<ProbeIdentityFile>.Ok(BuildFile(probe, identity, server)); return Result<ProbeIdentityFile>.Ok(await BuildFileAsync(probe, identity, ct));
} }
public async ValueTask<IReadOnlyList<ProbeView>> ListAsync(CancellationToken ct) public async ValueTask<IReadOnlyList<ProbeView>> ListAsync(CancellationToken ct)
@@ -99,7 +99,7 @@ public sealed partial class ProbeService
} }
/// <summary>换一把新密钥(id 不变):旧私钥立即失效,所有 Agent 上这个 Probe 的会话被断开。返回新的 probe.json(只此一次)。</summary> /// <summary>换一把新密钥(id 不变):旧私钥立即失效,所有 Agent 上这个 Probe 的会话被断开。返回新的 probe.json(只此一次)。</summary>
public async ValueTask<Result<ProbeIdentityFile>> RotateAsync(string id, string server, CancellationToken ct) public async ValueTask<Result<ProbeIdentityFile>> RotateAsync(string id, CancellationToken ct)
{ {
if (await _probes.GetAsync(id, ct) is not { } probe) if (await _probes.GetAsync(id, ct) is not { } probe)
{ {
@@ -116,7 +116,7 @@ public sealed partial class ProbeService
await _probes.SaveAsync(updated, ct); await _probes.SaveAsync(updated, ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
await DisconnectEverywhereAsync(id, ct); await DisconnectEverywhereAsync(id, ct);
return Result<ProbeIdentityFile>.Ok(BuildFile(updated, identity, server)); return Result<ProbeIdentityFile>.Ok(await BuildFileAsync(updated, identity, ct));
} }
/// <summary>吊销:保留记录和历史数据,公钥从所有 Agent 的名单里移除,Agent 立即断开它的会话。</summary> /// <summary>吊销:保留记录和历史数据,公钥从所有 Agent 的名单里移除,Agent 立即断开它的会话。</summary>
@@ -138,8 +138,6 @@ public sealed partial class ProbeService
bool deleted = await _probes.DeleteAsync(id, ct); bool deleted = await _probes.DeleteAsync(id, ct);
if (deleted) if (deleted)
{ {
await _nodes.ForgetAsync(id, ct);
await _links.ForgetNodeAsync(id, ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
} }
@@ -182,11 +180,18 @@ public sealed partial class ProbeService
} }
} }
return new ProbeView(probe, _gateway.IsOnline(new NodeId(probe.Id)), [.. connections.OrderBy(c => c.AgentName, StringComparer.Ordinal)]); return new ProbeView(probe, [.. connections.OrderBy(c => c.AgentName, StringComparer.Ordinal)]);
} }
private static ProbeIdentityFile BuildFile(ProbeRecord probe, NodeIdentity identity, string server) => private async ValueTask<ProbeIdentityFile> BuildFileAsync(ProbeRecord probe, NodeIdentity identity, CancellationToken ct)
new(new ProbeId(probe.Id), Convert.ToBase64String(identity.ExportPkcs8()), server, new ProbeLimits(MaxTestMbps: 100, MaxTotalPps: 20_000)); {
ProbeAgentsResult agents = await _agentService.ProbeAgentsAsync(ct);
return new ProbeIdentityFile(
new ProbeId(probe.Id),
Convert.ToBase64String(identity.ExportPkcs8()),
[.. agents.Agents.Select(a => new BootstrapAgent(a.AgentId, a.Host, a.TcpPort, a.Spki))],
new ProbeLimits(MaxTestMbps: 100, MaxTotalPps: 20_000));
}
[GeneratedRegex(@"^[\p{L}\p{M}\p{N}._-]{1,64}$")] [GeneratedRegex(@"^[\p{L}\p{M}\p{N}._-]{1,64}$")]
private static partial Regex NamePattern(); private static partial Regex NamePattern();
@@ -27,16 +27,13 @@ public static class ApplicationServiceCollectionExtensions
services.AddSingleton<AgentService>(); services.AddSingleton<AgentService>();
services.AddSingleton<AgentEventService>(); services.AddSingleton<AgentEventService>();
services.AddSingleton<HostStatusService>(); services.AddSingleton<HostStatusService>();
services.AddSingleton<NodeInfoService>();
services.AddSingleton<ProbeService>(); services.AddSingleton<ProbeService>();
services.AddSingleton<LinkService>();
services.AddSingleton<LineService>(); services.AddSingleton<LineService>();
services.AddSingleton<TestService>(); services.AddSingleton<TestService>();
services.AddSingleton<IngestService>(); services.AddSingleton<IngestService>();
services.AddSingleton<WindowComputer>(); services.AddSingleton<WindowComputer>();
services.AddSingleton<AuthService>(); services.AddSingleton<AuthService>();
services.AddSingleton<SettingsService>(); services.AddSingleton<SettingsService>();
services.AddSingleton<ReleaseService>();
services.AddSingleton<JoinTokenService>(); services.AddSingleton<JoinTokenService>();
services.AddSingleton<RawBrowserService>(); services.AddSingleton<RawBrowserService>();
services.AddSingleton<EventService>(); services.AddSingleton<EventService>();
@@ -7,13 +7,12 @@ public enum PrincipalKind
Token, Token,
} }
/// <summary>访问级别:作用域为 read 的令牌只读,admin 可以管理,release 只能登记发布清单(给 CI 用,什么都读不了)。</summary> /// <summary>访问级别:作用域为 read 的令牌只读,admin 可以管理。</summary>
public enum ApiScope public enum ApiScope
{ {
None = 0, None = 0,
Release = 1, Read = 1,
Read = 2, Admin = 2,
Admin = 3,
} }
public sealed record Principal(PrincipalKind Kind, ApiScope Scope, string? UserId, string? Username, string? CsrfToken, string? TokenId) public sealed record Principal(PrincipalKind Kind, ApiScope Scope, string? UserId, string? Username, string? CsrfToken, string? TokenId)
@@ -23,8 +22,6 @@ public sealed record Principal(PrincipalKind Kind, ApiScope Scope, string? UserI
public bool CanRead => Scope >= ApiScope.Read; public bool CanRead => Scope >= ApiScope.Read;
public bool IsAdmin => Scope >= ApiScope.Admin; public bool IsAdmin => Scope >= ApiScope.Admin;
public bool CanPublishRelease => Scope is ApiScope.Release or ApiScope.Admin;
} }
public sealed record LoginResult(string SessionToken, string CsrfToken, long ExpiresAtMs, string UserId, string Username); public sealed record LoginResult(string SessionToken, string CsrfToken, long ExpiresAtMs, string UserId, string Username);
@@ -77,7 +77,7 @@ public sealed partial class AuthService
await _store.TouchApiTokenAsync(token.Id, now, ct); await _store.TouchApiTokenAsync(token.Id, now, ct);
} }
ApiScope scope = token.Scope switch { "admin" => ApiScope.Admin, "release" => ApiScope.Release, _ => ApiScope.Read }; ApiScope scope = token.Scope == "admin" ? ApiScope.Admin : ApiScope.Read;
return new Principal(PrincipalKind.Token, scope, null, token.Name, null, token.Id); return new Principal(PrincipalKind.Token, scope, null, token.Name, null, token.Id);
} }
@@ -135,9 +135,9 @@ public sealed partial class AuthService
return ValidationError.Fail<CreatedApiToken>("name", "must be 1-64 characters"); return ValidationError.Fail<CreatedApiToken>("name", "must be 1-64 characters");
} }
if (scope is not ("read" or "admin" or "release")) if (scope is not ("read" or "admin"))
{ {
return ValidationError.Fail<CreatedApiToken>("scope", "must be read, admin or release"); return ValidationError.Fail<CreatedApiToken>("scope", "must be read or admin");
} }
string plaintext = TokenHash.NewToken("mon"); string plaintext = TokenHash.NewToken("mon");
@@ -55,18 +55,6 @@ public sealed class BackendOptions
public int MaxCompareAgents { get; set; } = 20; public int MaxCompareAgents { get; set; } = 20;
/// <summary>
/// 测试方案里「固定时长」阶段(没有 <c>duration_s</c> 参数的,例如 connect、hold)的缩放比例,缺省 1。
/// 只给自动化测试用:端到端测试里把 30 秒的阶段缩短,不用真等。
/// </summary>
public double PhaseDurationScale { get; set; } = 1.0;
/// <summary>
/// IP 归属查询服务的地址(`GET {地址}/lookup?ip=…`):Backend 用 Probe、Agent 的来源地址查一次国家和运营商,
/// 只留下国家代码和运营商名,IP 不对外。留空 = 不查。服务的使用条款(公益、不能商用、保留署名)由部署的人自己遵守。
/// </summary>
public string? GeoLookupUrl { get; set; } = "https://ip.007995.xyz";
/// <summary>初始管理员(只在没有任何用户时创建)。</summary> /// <summary>初始管理员(只在没有任何用户时创建)。</summary>
public string? InitialAdminUser { get; set; } public string? InitialAdminUser { get; set; }
@@ -1,86 +0,0 @@
namespace Monitor.Backend.Application;
/// <summary>
/// 把「ASN 和组织名」归成几个常见的国内运营商:mobile / telecom / unicom / broadnet;别的是 other(显示组织名)。
/// 只用来挑展示用的图标和中文名。
/// </summary>
public static class IspClassifier
{
public const string Mobile = "mobile";
public const string Telecom = "telecom";
public const string Unicom = "unicom";
public const string Broadnet = "broadnet";
public const string Other = "other";
private static readonly HashSet<int> s_mobile = [9808, 24400, 24445, 56040, 56041, 56042, 56044, 56046, 56047, 56048, 132525, 138407];
private static readonly HashSet<int> s_telecom = [4134, 4809, 4811, 4812, 4813, 9394, 17638, 23724, 38283, 58466, 58519, 136188, 137693];
private static readonly HashSet<int> s_unicom = [4808, 4837, 9929, 10099, 17621, 17622, 17623, 17816, 136958, 138421];
private static readonly HashSet<int> s_broadnet = [63838, 140061];
public static (string Key, string Name) Classify(int? asn, string? organization)
{
if (asn is { } a)
{
if (s_mobile.Contains(a))
{
return (Mobile, "中国移动");
}
if (s_telecom.Contains(a))
{
return (Telecom, "中国电信");
}
if (s_unicom.Contains(a))
{
return (Unicom, "中国联通");
}
if (s_broadnet.Contains(a))
{
return (Broadnet, "中国广电");
}
}
string org = organization ?? string.Empty;
if (Has(org, "china mobile") || Has(org, "cmnet") || Has(org, "中国移动"))
{
return (Mobile, "中国移动");
}
if (Has(org, "china telecom") || Has(org, "chinanet") || Has(org, "中国电信"))
{
return (Telecom, "中国电信");
}
if (Has(org, "china unicom") || Has(org, "cncgroup") || Has(org, "中国联通"))
{
return (Unicom, "中国联通");
}
if (Has(org, "china broadnet") || Has(org, "中国广电"))
{
return (Broadnet, "中国广电");
}
return (Other, Shorten(org));
}
private static bool Has(string text, string needle) => text.Contains(needle, StringComparison.OrdinalIgnoreCase);
/// <summary>组织名常常很长(「Hangzhou Alibaba Advertising Co.,Ltd.」):去掉公司后缀,最多留 28 个字符。</summary>
internal static string Shorten(string organization)
{
string name = organization.Trim();
foreach (string suffix in new[] { ", Inc.", " Inc.", ", LLC", " LLC", " Co.,Ltd.", " Co., Ltd.", " Co.,Ltd", " Co., Ltd", " Ltd.", " Ltd", " Corporation", " Corp.", " Limited", " GmbH", " B.V.", " S.A." })
{
if (name.EndsWith(suffix, StringComparison.OrdinalIgnoreCase))
{
name = name[..^suffix.Length].TrimEnd(',', ' ');
break;
}
}
return name.Length > 28 ? name[..28].TrimEnd() : name;
}
}
@@ -1,34 +0,0 @@
namespace Monitor.Backend.Application;
/// <summary>
/// 一个节点(Agent 或 Probe)用来展示的信息:系统和架构、国家、运营商,以及它自己上报的构建信息(版本、tag、提交、构建时间)。
/// <b>不含任何 IP 地址、城市、ASN 或经纬度</b>——这些只在查询的那一刻用到,不留下,也不对外。
/// </summary>
public sealed record NodeInfo(
string NodeId,
string Kind,
string? Os,
string? OsName,
string? Arch,
string? Country,
string? IspKey,
string? IspName,
string? Version = null,
string? Tag = null,
string? Commit = null,
string? BuiltAt = null);
/// <summary>
/// 存起来的完整记录(只给 Backend 内部用):多了一个「查询用的主机」,用来判断地址变了没有、要不要重新查。
/// 这个字段不会出现在任何对外的接口里。
/// </summary>
public sealed record NodeMeta(NodeInfo Info, string? GeoHost, long GeoAtMs, long UpdatedAtMs);
public static class NodeKinds
{
public const string Agent = "agent";
public const string Probe = "probe";
}
/// <summary>IP 归属查询的结果(只取需要的几项)。</summary>
public sealed record GeoInfo(string? Country, int? Asn, string? Organization, bool IsPrivate);
@@ -1,219 +0,0 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using Monitor.Protocol;
namespace Monitor.Backend.Application;
/// <summary>
/// 节点的展示信息:系统和架构(节点自报),国家和运营商(Backend 用节点的来源地址查一次)。
/// <list type="bullet">
/// <item>Probe、Agent 不访问任何外部服务(客户的防火墙常常要审计),查询只由 Backend 发起。</item>
/// <item>只查公网地址:回环、内网、CGNAT、链路本地的地址一律不发出去。</item>
/// <item>查询结果只留国家代码和运营商;IP 不对外,也不和展示信息一起返回。</item>
/// </list>
/// </summary>
public sealed class NodeInfoService
{
/// <summary>同一个地址已经查过的结果保留多久,过期后在节点下次出现时重新查(运营商偶尔会变)。</summary>
private static readonly TimeSpan s_geoTtl = TimeSpan.FromDays(7);
/// <summary>查询失败之后多久再试。</summary>
private static readonly TimeSpan s_retryAfter = TimeSpan.FromMinutes(10);
private readonly INodeMetaStore _store;
private readonly IGeoLookup _geo;
private readonly TimeProvider _time;
private readonly ConcurrentDictionary<string, NodeMeta> _cache = new();
private readonly ConcurrentDictionary<string, byte> _inflight = new();
private Task? _load;
public NodeInfoService(INodeMetaStore store, IGeoLookup geo, TimeProvider time)
{
_store = store;
_geo = geo;
_time = time;
}
/// <summary>所有节点的展示信息(Agent 和 Probe)。</summary>
public async ValueTask<IReadOnlyList<NodeInfo>> ListAsync(CancellationToken ct)
{
await EnsureLoadedAsync(ct);
return [.. _cache.Values.Select(m => m.Info)];
}
/// <summary>节点自报的系统、架构和构建信息;没有变化就不写库。</summary>
public async ValueTask SetPlatformAsync(string nodeId, string kind, PlatformInfo? platform, CancellationToken ct)
{
if (platform is null)
{
return;
}
await EnsureLoadedAsync(ct);
NodeMeta current = _cache.TryGetValue(nodeId, out NodeMeta? found) ? found : Empty(nodeId, kind);
string? os = Clean(platform.Os, 16);
string? name = Clean(platform.Name, 40);
string? arch = Clean(platform.Arch, 16);
string? version = Clean(platform.Version, 32);
string? tag = Clean(platform.Tag, 64);
string? commit = Clean(platform.Commit, 40);
string? builtAt = Clean(platform.BuiltAt, 32);
NodeInfo i = current.Info;
if (i.Os == os && i.OsName == name && i.Arch == arch && i.Version == version && i.Tag == tag && i.Commit == commit && i.BuiltAt == builtAt)
{
return;
}
await SaveAsync(current with { Info = i with { Os = os, OsName = name, Arch = arch, Version = version, Tag = tag, Commit = commit, BuiltAt = builtAt }, UpdatedAtMs = Now() }, ct);
}
/// <summary>
/// 节点从某个地址连上来了:查一次它的国家和运营商。已经查过同一个地址(并且没过期)就什么也不做;
/// 私有地址不查;查询失败或服务关着,不影响别的,隔一阵再试。
/// </summary>
public async ValueTask ObserveAddressAsync(string nodeId, string kind, string? host, CancellationToken ct)
{
host = host?.Trim();
if (string.IsNullOrEmpty(host) || !IsPublic(host) || !_inflight.TryAdd(nodeId, 0))
{
return;
}
try
{
await EnsureLoadedAsync(ct);
NodeMeta current = _cache.TryGetValue(nodeId, out NodeMeta? found) ? found : Empty(nodeId, kind);
long now = Now();
bool same = string.Equals(current.GeoHost, host, StringComparison.OrdinalIgnoreCase);
bool fresh = current.Info.Country is not null && now - current.GeoAtMs < s_geoTtl.TotalMilliseconds;
bool recentlyFailed = current.Info.Country is null && now - current.GeoAtMs < s_retryAfter.TotalMilliseconds;
if (same && (fresh || recentlyFailed))
{
return;
}
GeoInfo? geo = await _geo.LookupAsync(host, ct);
if (geo is null || geo.IsPrivate || string.IsNullOrEmpty(geo.Country))
{
// 记下这次没查到(地址和时间),免得每次心跳都去查
await SaveAsync(current with { GeoHost = host, GeoAtMs = now, UpdatedAtMs = now }, ct);
return;
}
(string key, string name) = IspClassifier.Classify(geo.Asn, geo.Organization);
string country = geo.Country.ToUpperInvariant();
await SaveAsync(current with { Info = current.Info with { Country = country, IspKey = key, IspName = Clean(name, 40) }, GeoHost = host, GeoAtMs = now, UpdatedAtMs = now }, ct);
}
finally
{
_inflight.TryRemove(nodeId, out _);
}
}
/// <summary>Agent 报告某个 Probe 连着它:在后台用 Probe 的来源地址查国家和运营商(系统信息是 Probe 自己连上面板时报的)。出错只吞掉。</summary>
public void ObserveProbeInBackground(string probeId, string? remote) =>
_ = Task.Run(async () =>
{
try
{
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
await ObserveAddressAsync(probeId, NodeKinds.Probe, remote, cts.Token);
}
catch (Exception ex) when (ex is OperationCanceledException or InvalidOperationException or IOException)
{
// 展示信息拿不到也没关系,下次再试
}
});
/// <summary>Agent 打招呼(hello):记下它的系统信息,并用它的公开地址在后台查国家和运营商。</summary>
public void ObserveAgentInBackground(string agentId, string? publicHost, PlatformInfo? platform) =>
_ = Task.Run(async () =>
{
try
{
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
await SetPlatformAsync(agentId, NodeKinds.Agent, platform, cts.Token);
await ObserveAddressAsync(agentId, NodeKinds.Agent, publicHost, cts.Token);
}
catch (Exception ex) when (ex is OperationCanceledException or InvalidOperationException or IOException)
{
// 同上
}
});
public async ValueTask ForgetAsync(string nodeId, CancellationToken ct)
{
_cache.TryRemove(nodeId, out _);
await _store.DeleteAsync(nodeId, ct);
}
/// <summary>公网地址才值得查:回环、内网、CGNAT、链路本地、未指定的地址一律不发出去。域名放行(由查询服务自己解析)。</summary>
public static bool IsPublic(string host)
{
if (!IPAddress.TryParse(host, out IPAddress? ip))
{
return host.Contains('.', StringComparison.Ordinal) && !host.EndsWith(".local", StringComparison.OrdinalIgnoreCase) && !host.EndsWith(".internal", StringComparison.OrdinalIgnoreCase);
}
if (IPAddress.IsLoopback(ip) || ip.Equals(IPAddress.Any) || ip.Equals(IPAddress.IPv6Any) || ip.Equals(IPAddress.None))
{
return false;
}
if (ip.IsIPv4MappedToIPv6)
{
ip = ip.MapToIPv4();
}
if (ip.AddressFamily == AddressFamily.InterNetwork)
{
byte[] b = ip.GetAddressBytes();
return !(b[0] == 10
|| (b[0] == 172 && b[1] is >= 16 and <= 31)
|| (b[0] == 192 && b[1] == 168)
|| (b[0] == 169 && b[1] == 254)
|| (b[0] == 100 && b[1] is >= 64 and <= 127)
|| b[0] == 0
|| b[0] >= 224);
}
byte[] v6 = ip.GetAddressBytes();
bool linkLocal = v6[0] == 0xfe && (v6[1] & 0xc0) == 0x80;
bool uniqueLocal = (v6[0] & 0xfe) == 0xfc;
bool multicast = v6[0] == 0xff;
return !(linkLocal || uniqueLocal || multicast);
}
/// <summary>第一次用到时从库里读一次(只读一次,之后都在内存里)。</summary>
private async ValueTask EnsureLoadedAsync(CancellationToken ct)
{
Task load = Volatile.Read(ref _load) ?? Interlocked.CompareExchange(ref _load, LoadAsync(), null) ?? _load!;
await load.WaitAsync(ct);
}
private async Task LoadAsync()
{
foreach (NodeMeta meta in await _store.ListAsync(CancellationToken.None))
{
_cache.TryAdd(meta.Info.NodeId, meta);
}
}
private async ValueTask SaveAsync(NodeMeta meta, CancellationToken ct)
{
_cache[meta.Info.NodeId] = meta;
await _store.SaveAsync(meta, ct);
}
private long Now() => _time.GetUtcNow().ToUnixTimeMilliseconds();
private static NodeMeta Empty(string nodeId, string kind) => new(new NodeInfo(nodeId, kind, null, null, null, null, null, null), null, 0, 0);
/// <summary>节点自报的字符串只用于展示:去掉首尾空白,截断,没有内容就当没有。</summary>
private static string? Clean(string? text, int max)
{
string trimmed = text?.Trim() ?? string.Empty;
return trimmed.Length == 0 ? null : trimmed.Length > max ? trimmed[..max] : trimmed;
}
}
@@ -4,15 +4,12 @@ using Monitor.Protocol;
namespace Monitor.Backend.Application; namespace Monitor.Backend.Application;
/// <summary> /// <summary>
/// Application 通往 Agent 连接的端口(由 Gateway 实现):下发状态变化、发起按需测试。 /// Application 通往 Agent 连接的端口(由 Gateway 实现):下发状态变化、发起按需测试、通知 Probe 列表变化。
/// </summary> /// </summary>
public interface IAgentGateway public interface IAgentGateway
{ {
IReadOnlyCollection<NodeId> OnlineAgents { get; } IReadOnlyCollection<NodeId> OnlineAgents { get; }
/// <summary>所有在线的节点(Agent 和 Probe)。</summary>
IReadOnlyCollection<NodeId> OnlineNodes { get; }
bool IsOnline(NodeId agent); bool IsOnline(NodeId agent);
/// <summary>某个 Agent 的期望状态变了:Gateway 重新计算并推送。</summary> /// <summary>某个 Agent 的期望状态变了:Gateway 重新计算并推送。</summary>
@@ -20,6 +17,9 @@ public interface IAgentGateway
void NotifyAllTaskSetsChanged(); void NotifyAllTaskSetsChanged();
/// <summary>Agent 列表变了:通知所有在线 Agent 转发给 Probe。</summary>
ValueTask NotifyProbeAgentsChangedAsync(ProbeAgentsResult agents, CancellationToken ct);
/// <summary>失败时错误码为 <c>probe_offline</c>、<c>test_busy</c>、<c>agent_offline</c> 等。</summary> /// <summary>失败时错误码为 <c>probe_offline</c>、<c>test_busy</c>、<c>agent_offline</c> 等。</summary>
ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct); ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct);
@@ -29,10 +29,4 @@ public interface IAgentGateway
ValueTask<Result<bool>> DisconnectProbeAsync(NodeId agent, ProbeId probe, CancellationToken ct); ValueTask<Result<bool>> DisconnectProbeAsync(NodeId agent, ProbeId probe, CancellationToken ct);
ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct); ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct);
/// <summary>把发布清单和签名原样交给节点(通知,不等回复)。</summary>
ValueTask NotifyUpdateOfferAsync(NodeId node, string manifest, string signature, CancellationToken ct);
/// <summary>请求节点更新到它手里的清单。节点不在线返回 false。</summary>
ValueTask<bool> NotifyUpdateApplyAsync(NodeId node, CancellationToken ct);
} }
@@ -33,22 +33,6 @@ public interface IAgentStore
ValueTask SetAppliedTaskStateAsync(NodeId id, long appliedVersion, string? rejectedJson, CancellationToken ct); ValueTask SetAppliedTaskStateAsync(NodeId id, long appliedVersion, string? rejectedJson, CancellationToken ct);
} }
/// <summary>节点的展示信息(系统、架构、国家、运营商)。</summary>
public interface INodeMetaStore
{
ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct);
ValueTask SaveAsync(NodeMeta meta, CancellationToken ct);
ValueTask DeleteAsync(string nodeId, CancellationToken ct);
}
/// <summary>查一个 IP(或域名)的归属:国家、ASN、组织。实现负责超时和失败(失败返回 null,不抛)。</summary>
public interface IGeoLookup
{
ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct);
}
public interface IJoinTokenStore public interface IJoinTokenStore
{ {
ValueTask<JoinTokenInfo> CreateAsync(string id, byte[] tokenSha256, string? note, IReadOnlyList<string> labels, bool autoApprove, int maxUses, long expiresAtMs, long nowMs, CancellationToken ct); ValueTask<JoinTokenInfo> CreateAsync(string id, byte[] tokenSha256, string? note, IReadOnlyList<string> labels, bool autoApprove, int maxUses, long expiresAtMs, long nowMs, CancellationToken ct);
@@ -56,8 +40,6 @@ public interface IJoinTokenStore
ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct); ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct);
ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct); ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct);
ValueTask<bool> DeleteAsync(string id, CancellationToken ct);
} }
public interface IProbeStore public interface IProbeStore
@@ -39,22 +39,19 @@ public sealed class LineService
{ {
private readonly AgentService _agents; private readonly AgentService _agents;
private readonly IProbeStore _probes; private readonly IProbeStore _probes;
private readonly LinkService _links;
public LineService(AgentService agents, IProbeStore probes, LinkService links) public LineService(AgentService agents, IProbeStore probes)
{ {
_links = links;
_agents = agents; _agents = agents;
_probes = probes; _probes = probes;
} }
/// <summary>所有线路:每个 Agent 和每个<b>未吊销</b>的 Probe 配成一对,只留下线路开着的(见 <see cref="LinkService"/>);关掉的线路历史数据还在,按 id 仍然查得到。</summary> /// <summary>所有线路:每个 Agent 和每个<b>未吊销</b>的 Probe 配成一对。</summary>
public async ValueTask<IReadOnlyList<LineView>> ListAsync(Func<AgentView, bool> include, CancellationToken ct) public async ValueTask<IReadOnlyList<LineView>> ListAsync(Func<AgentView, bool> include, CancellationToken ct)
{ {
IReadOnlyList<AgentView> agents = await _agents.ListAsync(ct); IReadOnlyList<AgentView> agents = await _agents.ListAsync(ct);
IReadOnlyList<ProbeRecord> probes = [.. (await _probes.ListAsync(ct)).Where(p => p.RevokedAtMs is null)]; IReadOnlyList<ProbeRecord> probes = [.. (await _probes.ListAsync(ct)).Where(p => p.RevokedAtMs is null)];
LinkSnapshot links = await _links.SnapshotAsync(ct); return [.. agents.Where(include).SelectMany(a => probes.Select(p => new LineView(a, p)))];
return [.. agents.Where(include).SelectMany(a => probes.Where(p => links.IsLinked(p.Id, a.Agent.Id.Value)).Select(p => new LineView(a, p)))];
} }
/// <summary> /// <summary>
@@ -20,12 +20,10 @@ public sealed class TaskPlanner : IDisposable
private readonly IAgentStore _agents; private readonly IAgentStore _agents;
private readonly IProbeStore _probes; private readonly IProbeStore _probes;
private readonly ISettingsStore _settings; private readonly ISettingsStore _settings;
private readonly LinkService _links;
private readonly SemaphoreSlim _gate = new(1, 1); private readonly SemaphoreSlim _gate = new(1, 1);
public TaskPlanner(ITaskStore tasks, IAgentStore agents, IProbeStore probes, ISettingsStore settings, LinkService links) public TaskPlanner(ITaskStore tasks, IAgentStore agents, IProbeStore probes, ISettingsStore settings)
{ {
_links = links;
_tasks = tasks; _tasks = tasks;
_agents = agents; _agents = agents;
_probes = probes; _probes = probes;
@@ -43,9 +41,9 @@ public sealed class TaskPlanner : IDisposable
IReadOnlyList<TaskRecord> all = await _tasks.ListAsync(ct); IReadOnlyList<TaskRecord> all = await _tasks.ListAsync(ct);
IReadOnlyList<ProbeRecord> probes = await _probes.ListAsync(ct); IReadOnlyList<ProbeRecord> probes = await _probes.ListAsync(ct);
LinkSnapshot links = await _links.SnapshotAsync(ct); long listVersion = long.TryParse(await _settings.GetAsync(ProbeAgentsVersionKey, ct), out long v) ? v : 0;
var document = new TaskSet(); var document = new TaskSet { ProbeAgentsVersion = listVersion };
foreach (EffectiveTask task in EffectiveTasks.Compute(all, agent, filterByAgentKinds: true)) foreach (EffectiveTask task in EffectiveTasks.Compute(all, agent, filterByAgentKinds: true))
{ {
document.Tasks.Add(new TaskSpec document.Tasks.Add(new TaskSpec
@@ -60,7 +58,7 @@ public sealed class TaskPlanner : IDisposable
}); });
} }
foreach (ProbeRecord probe in probes.Where(p => p.RevokedAtMs is null && links.IsLinked(p.Id, agentId.Value))) foreach (ProbeRecord probe in probes.Where(p => p.RevokedAtMs is null))
{ {
document.Peers.Add(new PeerSpec { ProbeId = new ProbeId(probe.Id), PublicKey = Convert.ToBase64String(probe.PublicKey) }); document.Peers.Add(new PeerSpec { ProbeId = new ProbeId(probe.Id), PublicKey = Convert.ToBase64String(probe.PublicKey) });
} }
@@ -9,7 +9,6 @@ namespace Monitor.Backend.Application;
/// <summary>按需测试(once):校验与排队、下发给 Agent、状态更新、结果查询。</summary> /// <summary>按需测试(once):校验与排队、下发给 Agent、状态更新、结果查询。</summary>
public sealed class TestService public sealed class TestService
{ {
private readonly double _phaseScale;
private readonly ITestStore _tests; private readonly ITestStore _tests;
private readonly IAgentStore _agents; private readonly IAgentStore _agents;
private readonly AgentRuntime _runtime; private readonly AgentRuntime _runtime;
@@ -17,9 +16,8 @@ public sealed class TestService
private readonly LiveHub _live; private readonly LiveHub _live;
private readonly TimeProvider _time; private readonly TimeProvider _time;
public TestService(ITestStore tests, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, LiveHub live, TimeProvider time, BackendOptions options) public TestService(ITestStore tests, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, LiveHub live, TimeProvider time)
{ {
_phaseScale = options.PhaseDurationScale;
_tests = tests; _tests = tests;
_agents = agents; _agents = agents;
_runtime = runtime; _runtime = runtime;
@@ -54,7 +52,7 @@ public sealed class TestService
return Result<TestRecord>.Fail(FleetErrors.Unsupported, $"this agent does not support {unsupported}, which the {plan.Id} plan needs"); return Result<TestRecord>.Fail(FleetErrors.Unsupported, $"this agent does not support {unsupported}, which the {plan.Id} plan needs");
} }
Result<List<TestPhase>> phases = BuildPhases(plan, request.Params, agent, _phaseScale); Result<List<TestPhase>> phases = BuildPhases(plan, request.Params, agent);
if (!phases.TryGet(out List<TestPhase>? phaseList, out FleetError? error)) if (!phases.TryGet(out List<TestPhase>? phaseList, out FleetError? error))
{ {
return Result<TestRecord>.Fail(error); return Result<TestRecord>.Fail(error);
@@ -175,7 +173,7 @@ public sealed class TestService
public static bool IsTerminal(string status) => status is TestStatuses.Succeeded or TestStatuses.Failed or TestStatuses.Aborted; public static bool IsTerminal(string status) => status is TestStatuses.Succeeded or TestStatuses.Failed or TestStatuses.Aborted;
private static Result<List<TestPhase>> BuildPhases(PlanSpec plan, IReadOnlyDictionary<string, JsonElement>? overrides, AgentRecord agent, double phaseScale) private static Result<List<TestPhase>> BuildPhases(PlanSpec plan, IReadOnlyDictionary<string, JsonElement>? overrides, AgentRecord agent)
{ {
if (overrides is not null) if (overrides is not null)
{ {
@@ -247,7 +245,7 @@ public sealed class TestService
return Result<List<TestPhase>>.Fail(MonitorErrors.BudgetExceeded, $"{phase.Kind} needs {bits / 1_000_000.0:0.#} Mbit/s but the agent allows {limitMbps} Mbit/s"); return Result<List<TestPhase>>.Fail(MonitorErrors.BudgetExceeded, $"{phase.Kind} needs {bits / 1_000_000.0:0.#} Mbit/s but the agent allows {limitMbps} Mbit/s");
} }
int duration = resolved.Values!.Values.TryGetValue("duration_s", out JsonElement d) && d.TryGetInt64(out long seconds) ? (int)seconds : Math.Max(1, (int)Math.Round(phase.DurationSeconds * phaseScale)); int duration = resolved.Values!.Values.TryGetValue("duration_s", out JsonElement d) && d.TryGetInt64(out long seconds) ? (int)seconds : phase.DurationSeconds;
phases.Add(new TestPhase { Kind = phase.Kind, Params = new Dictionary<string, JsonElement>(resolved.Values.Values), DurationSeconds = duration }); phases.Add(new TestPhase { Kind = phase.Kind, Params = new Dictionary<string, JsonElement>(resolved.Values.Values), DurationSeconds = duration });
} }
@@ -1,158 +0,0 @@
using System.Collections.Concurrent;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using System.Text.RegularExpressions;
using Fleet.Core;
using Monitor.Protocol;
namespace Monitor.Backend.Application;
/// <summary>面板记下的、CI 签名的发布清单(只留每个组件最新的一份)。<c>Manifest</c> 是签名对象的原始文本,面板一个字节也不改。</summary>
public sealed record ReleaseOffer(string Component, long Seq, string Tag, string Version, string Digest, string Manifest, string Signature);
/// <summary>
/// 发布清单的分发:CI 签名之后推给面板,面板存下来并原样转给在线的 Agent、Probe(节点连上时再补发一次)。
/// <b>面板不验签、也不决定拉什么</b>:节点宿主机上的更新器用本机的发布公钥验签,并且只认本机配置的镜像仓库;
/// 所以面板即使被攻破,也推不出一个能被装上的假镜像。面板只负责「通知」和「请求更新」,状态由节点随心跳报回来。
/// </summary>
public sealed partial class ReleaseService
{
public const int MaxManifestBytes = 4096;
public const int MaxSignatureBytes = 512;
private static readonly string[] s_components = [NodeKinds.Agent, NodeKinds.Probe];
private readonly ISettingsStore _settings;
private readonly IAgentGateway _gateway;
private readonly ConcurrentDictionary<string, UpdateStatus> _status = new();
public ReleaseService(ISettingsStore settings, IAgentGateway gateway)
{
_settings = settings;
_gateway = gateway;
}
/// <summary>登记一份新清单并推给在线的同类节点。只要格式对、<c>seq</c> 比现有的新就收下(签名由节点验证)。</summary>
public async ValueTask<Result<ReleaseOffer>> PublishAsync(string manifest, string signature, CancellationToken ct)
{
if (!TryParse(manifest, signature, out ReleaseOffer? offer, out string? problem))
{
return ValidationError.Fail<ReleaseOffer>("manifest", problem);
}
ReleaseOffer? current = await GetAsync(offer.Component, ct);
if (current is not null && offer.Seq <= current.Seq)
{
return Result<ReleaseOffer>.Fail(MonitorErrors.NameConflict, $"a release with seq {current.Seq} or newer is already registered");
}
await _settings.SetAsync(Key(offer.Component, "manifest"), offer.Manifest, ct);
await _settings.SetAsync(Key(offer.Component, "signature"), offer.Signature, ct);
foreach (NodeId node in OnlineOf(offer.Component))
{
await _gateway.NotifyUpdateOfferAsync(node, offer.Manifest, offer.Signature, ct);
}
return Result<ReleaseOffer>.Ok(offer);
}
public async ValueTask<ReleaseOffer?> GetAsync(string component, CancellationToken ct)
{
string? manifest = await _settings.GetAsync(Key(component, "manifest"), ct);
string? signature = await _settings.GetAsync(Key(component, "signature"), ct);
return manifest is not null && signature is not null && TryParse(manifest, signature, out ReleaseOffer? offer, out _) && offer.Component == component ? offer : null;
}
public async ValueTask<IReadOnlyList<ReleaseOffer>> ListAsync(CancellationToken ct)
{
var list = new List<ReleaseOffer>();
foreach (string component in s_components)
{
if (await GetAsync(component, ct) is { } offer)
{
list.Add(offer);
}
}
return list;
}
/// <summary>节点刚连上:把它这个组件最新的清单补发一份(节点离线时发布的也不会错过)。</summary>
public async ValueTask OfferToNodeAsync(NodeId node, CancellationToken ct)
{
if (ComponentOf(node) is { } component && await GetAsync(component, ct) is { } offer)
{
await _gateway.NotifyUpdateOfferAsync(node, offer.Manifest, offer.Signature, ct);
}
}
/// <summary>请求某个节点更新到它手里那份(验过签的)清单。节点不在线返回 false。</summary>
public ValueTask<bool> RequestUpdateAsync(NodeId node, CancellationToken ct) => _gateway.NotifyUpdateApplyAsync(node, ct);
public void RecordStatus(NodeId node, UpdateStatus? status)
{
if (status is null)
{
_status.TryRemove(node.Value, out _);
}
else
{
_status[node.Value] = status;
}
}
public UpdateStatus? StatusOf(NodeId node) => _status.TryGetValue(node.Value, out UpdateStatus? status) ? status : null;
public void Forget(NodeId node) => _status.TryRemove(node.Value, out _);
private IEnumerable<NodeId> OnlineOf(string component) => _gateway.OnlineNodes.Where(n => ComponentOf(n) == component);
private static string? ComponentOf(NodeId node) =>
node.Value.StartsWith(AgentIds.Prefix, StringComparison.Ordinal) ? NodeKinds.Agent
: node.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal) ? NodeKinds.Probe
: null;
private static string Key(string component, string part) => $"release.{component}.{part}";
private static bool TryParse(string manifest, string signature, [NotNullWhen(true)] out ReleaseOffer? offer, [NotNullWhen(false)] out string? problem)
{
offer = null;
problem = null;
if (manifest.Length is 0 or > MaxManifestBytes || signature.Length is 0 or > MaxSignatureBytes || !Base64().IsMatch(signature))
{
problem = "manifest or signature has an unexpected size or encoding";
return false;
}
try
{
using JsonDocument document = JsonDocument.Parse(manifest);
JsonElement root = document.RootElement;
string component = Text(root, "component");
string digest = Text(root, "digest");
if (root.ValueKind != JsonValueKind.Object || Array.IndexOf(s_components, component) < 0
|| !root.TryGetProperty("seq", out JsonElement seq) || !seq.TryGetInt64(out long seqValue) || seqValue <= 0
|| !Digest().IsMatch(digest) || Text(root, "image").Length is 0 or > 200)
{
problem = "manifest needs a known component, a positive seq, an image and a sha256 digest";
return false;
}
offer = new ReleaseOffer(component, seqValue, Text(root, "tag"), Text(root, "version"), digest, manifest, signature);
return true;
}
catch (JsonException)
{
problem = "manifest is not valid JSON";
return false;
}
}
private static string Text(JsonElement root, string name) =>
root.ValueKind == JsonValueKind.Object && root.TryGetProperty(name, out JsonElement value) && value.ValueKind == JsonValueKind.String ? value.GetString() ?? "" : "";
[GeneratedRegex("^[A-Za-z0-9+/=]+$")]
private static partial Regex Base64();
[GeneratedRegex("^sha256:[0-9a-f]{64}$")]
private static partial Regex Digest();
}
@@ -24,9 +24,7 @@ public sealed class FleetGateway : IAgentGateway
private IStateNotifier Notifier => _services.GetRequiredService<IStateNotifier>(); private IStateNotifier Notifier => _services.GetRequiredService<IStateNotifier>();
public IReadOnlyCollection<NodeId> OnlineAgents => [.. Connections.OnlineNodes.Where(n => n.Value.StartsWith(AgentIds.Prefix, StringComparison.Ordinal))]; public IReadOnlyCollection<NodeId> OnlineAgents => Connections.OnlineNodes;
public IReadOnlyCollection<NodeId> OnlineNodes => Connections.OnlineNodes;
public bool IsOnline(NodeId agent) => Connections.IsOnline(agent); public bool IsOnline(NodeId agent) => Connections.IsOnline(agent);
@@ -34,6 +32,15 @@ public sealed class FleetGateway : IAgentGateway
public void NotifyAllTaskSetsChanged() => Notifier.NotifyAllChanged(); public void NotifyAllTaskSetsChanged() => Notifier.NotifyAllChanged();
public async ValueTask NotifyProbeAgentsChangedAsync(ProbeAgentsResult agents, CancellationToken ct)
{
IFleetConnections connections = Connections;
foreach (NodeId node in connections.OnlineNodes)
{
await connections.NotifyAsync(node, MonitorMethods.ProbeAgentsChanged, agents, MonitorJsonContext.Default.ProbeAgentsResult, ct);
}
}
public async ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct) public async ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct)
{ {
Result<TestRunResult> result = await Connections.RequestAsync( Result<TestRunResult> result = await Connections.RequestAsync(
@@ -63,10 +70,4 @@ public sealed class FleetGateway : IAgentGateway
} }
public ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct) => Connections.DisconnectAsync(agent, reason, ct); public ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct) => Connections.DisconnectAsync(agent, reason, ct);
public async ValueTask NotifyUpdateOfferAsync(NodeId node, string manifest, string signature, CancellationToken ct) =>
await Connections.NotifyAsync(node, MonitorMethods.UpdateOffer, new UpdateOfferParams { Manifest = manifest, Signature = signature }, MonitorJsonContext.Default.UpdateOfferParams, ct);
public async ValueTask<bool> NotifyUpdateApplyAsync(NodeId node, CancellationToken ct) =>
await Connections.NotifyAsync(node, MonitorMethods.UpdateApply, new UpdateOfferParams(), MonitorJsonContext.Default.UpdateOfferParams, ct);
} }
@@ -20,7 +20,6 @@ public static class GatewayServiceCollectionExtensions
services.AddFleetEnrollment(); services.AddFleetEnrollment();
services.AddFleetServer(options => options.NodeIdPrefix = AgentIds.Prefix) services.AddFleetServer(options => options.NodeIdPrefix = AgentIds.Prefix)
.AddState<TaskSetStateProvider>() .AddState<TaskSetStateProvider>()
.AddState<ProbeAgentsStateProvider>()
.AddInbox<RawChunkInbox>() .AddInbox<RawChunkInbox>()
.MapMethods(MapMonitorMethods); .MapMethods(MapMonitorMethods);
return services; return services;
@@ -40,6 +39,11 @@ public static class GatewayServiceCollectionExtensions
.OnNotify( .OnNotify(
MonitorMethods.TestStatus, MonitorMethods.TestStatus,
MonitorJsonContext.Default.TestStatusParams, MonitorJsonContext.Default.TestStatusParams,
(context, parameters, ct) => services.GetRequiredService<TestService>().OnStatusAsync(context.Node.Id, parameters, ct)); (context, parameters, ct) => services.GetRequiredService<TestService>().OnStatusAsync(context.Node.Id, parameters, ct))
.On(
MonitorMethods.ProbeAgents,
MonitorJsonContext.Default.ProbeAgentsParams,
MonitorJsonContext.Default.ProbeAgentsResult,
async (context, parameters, ct) => Result<ProbeAgentsResult>.Ok(await services.GetRequiredService<AgentService>().ProbeAgentsAsync(ct)));
} }
} }
@@ -1,33 +0,0 @@
using System.Text.Json;
using Fleet.Rpc;
using Fleet.Server;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Gateway;
/// <summary>期望状态:为每个 Probe 下发要连接的 Agent 列表(地址、端口、证书指纹)。Agent 集合变化时由 <c>NotifyAllChanged</c> 一起重算推送。</summary>
public sealed class ProbeAgentsStateProvider : IStateProvider
{
private readonly AgentService _agents;
public ProbeAgentsStateProvider(AgentService agents)
{
_agents = agents;
}
public string Type => ProbeAgentList.StateType;
public async ValueTask<StateDocument?> GetAsync(NodeRecord node, CancellationToken ct)
{
if (!node.Id.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
return null;
}
ProbeAgentList list = await _agents.ProbeAgentsAsync(node.Id.Value, ct);
return new StateDocument(list.Version, JsonSerializer.SerializeToElement(list, MonitorJsonContext.Default.ProbeAgentList));
}
public ValueTask OnAppliedAsync(NodeRecord node, long version, StateSyncResult result, CancellationToken ct) => ValueTask.CompletedTask;
}
@@ -26,11 +26,6 @@ public sealed class TaskSetStateProvider : IStateProvider
public async ValueTask<StateDocument?> GetAsync(NodeRecord node, CancellationToken ct) public async ValueTask<StateDocument?> GetAsync(NodeRecord node, CancellationToken ct)
{ {
if (node.Id.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
return null; // Probe 不运行任务集
}
DesiredTaskSet? desired = await _planner.GetDesiredAsync(node.Id, ct); DesiredTaskSet? desired = await _planner.GetDesiredAsync(node.Id, ct);
return desired is null ? null : new StateDocument(desired.Version, JsonSerializer.SerializeToElement(desired.Document, MonitorJsonContext.Default.TaskSet)); return desired is null ? null : new StateDocument(desired.Version, JsonSerializer.SerializeToElement(desired.Document, MonitorJsonContext.Default.TaskSet));
} }
@@ -5,7 +5,7 @@ namespace Monitor.Backend.Storage.Sqlite;
/// <summary>表结构。版本记录在 <c>PRAGMA user_version</c>;以后的迁移是按版本号追加的内置 SQL。</summary> /// <summary>表结构。版本记录在 <c>PRAGMA user_version</c>;以后的迁移是按版本号追加的内置 SQL。</summary>
internal static class Schema internal static class Schema
{ {
public const int CurrentVersion = 5; public const int CurrentVersion = 3;
public static void Migrate(SqliteConnection connection) public static void Migrate(SqliteConnection connection)
{ {
@@ -55,41 +55,6 @@ internal static class Schema
connection.Exec("PRAGMA user_version = 3"); connection.Exec("PRAGMA user_version = 3");
transaction.Commit(); transaction.Commit();
} }
if (version < 4)
{
// 节点的展示信息:系统、架构、国家、运营商;geo_host 是查询用的地址,只在 Backend 内部用
using SqliteTransaction transaction = connection.BeginTransaction();
connection.Exec("""
CREATE TABLE node_meta (
node_id TEXT PRIMARY KEY,
kind TEXT NOT NULL,
os TEXT,
os_name TEXT,
arch TEXT,
country TEXT,
isp_key TEXT,
isp_name TEXT,
geo_host TEXT,
geo_at INTEGER NOT NULL DEFAULT 0,
updated_at INTEGER NOT NULL DEFAULT 0
) WITHOUT ROWID
""");
connection.Exec("PRAGMA user_version = 4");
transaction.Commit();
}
if (version < 5)
{
// 节点自报的构建信息:版本、tag、提交、构建时间
using SqliteTransaction transaction = connection.BeginTransaction();
connection.Exec("ALTER TABLE node_meta ADD COLUMN version TEXT");
connection.Exec("ALTER TABLE node_meta ADD COLUMN tag TEXT");
connection.Exec("ALTER TABLE node_meta ADD COLUMN commit_id TEXT");
connection.Exec("ALTER TABLE node_meta ADD COLUMN built_at TEXT");
connection.Exec("PRAGMA user_version = 5");
transaction.Commit();
}
} }
/// <summary> /// <summary>
@@ -166,17 +166,8 @@ public sealed class SqliteAgentStore : IAgentStore, IFleetStore
return Result<NodeRecord>.Ok(ToNode(GetInternal(c, node.Id)!, node.MetaJson)); return Result<NodeRecord>.Ok(ToNode(GetInternal(c, node.Id)!, node.MetaJson));
}, ct); }, ct);
/// <summary>Agent 在 agents 表里;Probe 的身份在 probes 表里(吊销之后按「禁用」处理,连不上)。</summary>
async ValueTask<NodeRecord?> IFleetStore.GetNodeAsync(NodeId id, CancellationToken ct) => async ValueTask<NodeRecord?> IFleetStore.GetNodeAsync(NodeId id, CancellationToken ct) =>
await GetAsync(id, ct) is { } agent await GetAsync(id, ct) is { } agent ? ToNode(agent, null) : null;
? ToNode(agent, null)
: await _db.ReadAsync(
c => c.QueryOne(
"SELECT id, name, pubkey, created_at, revoked_at FROM probes WHERE id = @id",
r => new NodeRecord(
new NodeId(r.GetString(0)), r.GetString(1), r.IsDBNull(4) ? NodeStatus.Active : NodeStatus.Disabled, r.Bytes(2), [], [], null, null, r.GetInt64(3), null),
("@id", id.Value)),
ct);
public ValueTask<NodeRecord?> FindEstablishedNodeByNameAsync(string name, CancellationToken ct) => public ValueTask<NodeRecord?> FindEstablishedNodeByNameAsync(string name, CancellationToken ct) =>
_db.ReadAsync( _db.ReadAsync(
@@ -34,9 +34,6 @@ public sealed class SqliteJoinTokenStore : IJoinTokenStore
public ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct) => public ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct) =>
_db.WriteAsync(c => c.Exec("UPDATE join_tokens SET revoked_at = @now WHERE id = @id AND revoked_at IS NULL", ("@id", id), ("@now", nowMs)) == 1, ct); _db.WriteAsync(c => c.Exec("UPDATE join_tokens SET revoked_at = @now WHERE id = @id AND revoked_at IS NULL", ("@id", id), ("@now", nowMs)) == 1, ct);
public ValueTask<bool> DeleteAsync(string id, CancellationToken ct) =>
_db.WriteAsync(c => c.Exec("DELETE FROM join_tokens WHERE id = @id", ("@id", id)) == 1, ct);
} }
public sealed class SqliteProbeStore : IProbeStore public sealed class SqliteProbeStore : IProbeStore
@@ -1,41 +0,0 @@
using Microsoft.Data.Sqlite;
using Monitor.Backend.Application;
namespace Monitor.Backend.Storage.Sqlite;
public sealed class SqliteNodeMetaStore : INodeMetaStore
{
private readonly SqliteDatabase _db;
public SqliteNodeMetaStore(SqliteDatabase db)
{
_db = db;
}
public ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct) =>
_db.ReadAsync<IReadOnlyList<NodeMeta>>(
c => c.Query("SELECT node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at FROM node_meta", Read),
ct);
public ValueTask SaveAsync(NodeMeta meta, CancellationToken ct) =>
_db.ExecuteAsync(
c => c.Exec(
"INSERT INTO node_meta (node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at) " +
"VALUES (@id, @kind, @os, @osn, @arch, @cc, @ik, @in, @gh, @ga, @ua, @ver, @tag, @cm, @bt) ON CONFLICT(node_id) DO UPDATE SET " +
"kind = excluded.kind, os = excluded.os, os_name = excluded.os_name, arch = excluded.arch, country = excluded.country, " +
"isp_key = excluded.isp_key, isp_name = excluded.isp_name, geo_host = excluded.geo_host, geo_at = excluded.geo_at, updated_at = excluded.updated_at, " +
"version = excluded.version, tag = excluded.tag, commit_id = excluded.commit_id, built_at = excluded.built_at",
("@id", meta.Info.NodeId), ("@kind", meta.Info.Kind), ("@os", meta.Info.Os), ("@osn", meta.Info.OsName), ("@arch", meta.Info.Arch), ("@cc", meta.Info.Country),
("@ik", meta.Info.IspKey), ("@in", meta.Info.IspName), ("@gh", meta.GeoHost), ("@ga", meta.GeoAtMs), ("@ua", meta.UpdatedAtMs),
("@ver", meta.Info.Version), ("@tag", meta.Info.Tag), ("@cm", meta.Info.Commit), ("@bt", meta.Info.BuiltAt)),
ct);
public ValueTask DeleteAsync(string nodeId, CancellationToken ct) =>
_db.ExecuteAsync(c => c.Exec("DELETE FROM node_meta WHERE node_id = @id", ("@id", nodeId)), ct);
private static NodeMeta Read(SqliteDataReader r)
{
static string? S(SqliteDataReader reader, int i) => reader.IsDBNull(i) ? null : reader.GetString(i);
return new NodeMeta(new NodeInfo(r.GetString(0), r.GetString(1), S(r, 2), S(r, 3), S(r, 4), S(r, 5), S(r, 6), S(r, 7), S(r, 11), S(r, 12), S(r, 13), S(r, 14)), S(r, 8), r.GetInt64(9), r.GetInt64(10));
}
}
@@ -20,7 +20,6 @@ public static class StorageServiceCollectionExtensions
services.AddSingleton<ITestStore, SqliteTestStore>(); services.AddSingleton<ITestStore, SqliteTestStore>();
services.AddSingleton<IEventStore, SqliteEventStore>(); services.AddSingleton<IEventStore, SqliteEventStore>();
services.AddSingleton<IHostStore, SqliteHostStore>(); services.AddSingleton<IHostStore, SqliteHostStore>();
services.AddSingleton<INodeMetaStore, SqliteNodeMetaStore>();
services.AddSingleton<IAuthStore, SqliteAuthStore>(); services.AddSingleton<IAuthStore, SqliteAuthStore>();
services.AddSingleton<IRawStore, SqliteRawStore>(); services.AddSingleton<IRawStore, SqliteRawStore>();
services.AddSingleton<IMetricStore, SqliteMetricStore>(); services.AddSingleton<IMetricStore, SqliteMetricStore>();
@@ -38,16 +38,6 @@ public static class BackendHost
IServiceCollection services = builder.Services; IServiceCollection services = builder.Services;
services.AddSingleton(time ?? TimeProvider.System); services.AddSingleton(time ?? TimeProvider.System);
services.AddMonitorSqlite(options.DataDirectory); services.AddMonitorSqlite(options.DataDirectory);
// 国家和运营商:Backend 用节点的来源地址查一次(Probe、Agent 不访问任何外部服务);地址留空就不查
if (string.IsNullOrWhiteSpace(options.GeoLookupUrl))
{
services.AddSingleton<IGeoLookup, NoGeoLookup>();
}
else
{
services.AddSingleton<IGeoLookup>(new HttpGeoLookup(options.GeoLookupUrl));
}
services.AddMonitorApplication(options); services.AddMonitorApplication(options);
services.AddMonitorGateway(); services.AddMonitorGateway();
services.AddMonitorApi(); services.AddMonitorApi();
@@ -1,73 +0,0 @@
using System.Text.Json;
using Monitor.Backend.Application;
namespace Monitor.Backend;
/// <summary>
/// 用 IP 归属查询服务(<c>GET {地址}/lookup?ip=…</c>)查国家和运营商。
/// 失败(超时、网络不通、服务返回错误)一律返回 null,不往外抛:展示信息没有也不影响别的功能。
/// </summary>
public sealed class HttpGeoLookup : IGeoLookup, IDisposable
{
private readonly HttpClient _http;
public HttpGeoLookup(string baseUrl)
{
_http = new HttpClient { BaseAddress = new Uri(baseUrl.TrimEnd('/') + "/"), Timeout = TimeSpan.FromSeconds(6) };
_http.DefaultRequestHeaders.UserAgent.ParseAdd("MyMonitor");
}
public async ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct)
{
try
{
using HttpResponseMessage response = await _http.GetAsync("lookup?ip=" + Uri.EscapeDataString(host), ct);
if (!response.IsSuccessStatusCode)
{
return null;
}
using JsonDocument doc = await JsonDocument.ParseAsync(await response.Content.ReadAsStreamAsync(ct), cancellationToken: ct);
return Parse(doc.RootElement);
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException or InvalidOperationException)
{
return null;
}
}
/// <summary>只取国家代码、ASN、组织名;其余字段(城市、经纬度、网段……)一概不读。</summary>
public static GeoInfo? Parse(JsonElement root)
{
if (root.TryGetProperty("found", out JsonElement found) && found.ValueKind == JsonValueKind.False)
{
return null;
}
bool isPrivate = root.TryGetProperty("is_private", out JsonElement priv) && priv.ValueKind == JsonValueKind.True;
string? country = root.TryGetProperty("country_region", out JsonElement region) && region.TryGetProperty("iso_code", out JsonElement code) && code.ValueKind == JsonValueKind.String ? code.GetString() : null;
int? asn = null;
string? organization = null;
if (root.TryGetProperty("network", out JsonElement network))
{
if (network.TryGetProperty("asn", out JsonElement asnValue) && asnValue.TryGetInt32(out int number))
{
asn = number;
}
string? isp = network.TryGetProperty("isp", out JsonElement ispValue) && ispValue.ValueKind == JsonValueKind.String ? ispValue.GetString() : null;
string? org = network.TryGetProperty("organization", out JsonElement orgValue) && orgValue.ValueKind == JsonValueKind.String ? orgValue.GetString() : null;
organization = string.IsNullOrWhiteSpace(isp) ? org : isp;
}
return new GeoInfo(country, asn, organization, isPrivate);
}
public void Dispose() => _http.Dispose();
}
/// <summary>不查(<c>Monitor__GeoLookupUrl</c> 留空)。</summary>
public sealed class NoGeoLookup : IGeoLookup
{
public ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct) => ValueTask.FromResult<GeoInfo?>(null);
}
@@ -1,23 +1,9 @@
<Project> <Project Sdk="Microsoft.NET.Sdk.Web">
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk.Web" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" Condition="'$(UseAotAnywhere)' == 'true'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>monitor-backend</AssemblyName> <AssemblyName>monitor-backend</AssemblyName>
<RootNamespace>Monitor.Backend</RootNamespace> <RootNamespace>Monitor.Backend</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<OptimizationPreference>Size</OptimizationPreference>
<StackTraceSupport>false</StackTraceSupport>
<EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<IlcFoldIdenticalMethodBodies>true</IlcFoldIdenticalMethodBodies>
<StripSymbols>true</StripSymbols>
<!-- This service exposes APIs only; there are no static web assets to publish. -->
<StaticWebAssetsEnabled>false</StaticWebAssetsEnabled>
<SqliteStaticLink Condition="'$(SqliteStaticLink)' == '' and '$(PublishAot)' == 'true' and '$(RuntimeIdentifier)' != ''">true</SqliteStaticLink>
<EnableRequestDelegateGenerator>true</EnableRequestDelegateGenerator> <EnableRequestDelegateGenerator>true</EnableRequestDelegateGenerator>
<EnableConfigurationBindingGenerator>true</EnableConfigurationBindingGenerator> <EnableConfigurationBindingGenerator>true</EnableConfigurationBindingGenerator>
<OpenApiGenerateDocuments>true</OpenApiGenerateDocuments> <OpenApiGenerateDocuments>true</OpenApiGenerateDocuments>
@@ -26,13 +12,8 @@
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Microsoft.Extensions.ApiDescription.Server" PrivateAssets="all" /> <PackageReference Include="Microsoft.Extensions.ApiDescription.Server" PrivateAssets="all" />
<PackageReference Include="SQLitePCLRaw.lib.e_sqlite3" Condition="'$(SqliteStaticLink)' != 'true'" />
<PackageReference Include="SQLitePCLRaw.lib.e_sqlite3" ExcludeAssets="native" Condition="'$(SqliteStaticLink)' == 'true'" />
<ProjectReference Include="..\Monitor.Backend.Api\Monitor.Backend.Api.csproj" /> <ProjectReference Include="..\Monitor.Backend.Api\Monitor.Backend.Api.csproj" />
<ProjectReference Include="..\Monitor.Backend.Gateway\Monitor.Backend.Gateway.csproj" /> <ProjectReference Include="..\Monitor.Backend.Gateway\Monitor.Backend.Gateway.csproj" />
<ProjectReference Include="..\Monitor.Backend.Storage.Sqlite\Monitor.Backend.Storage.Sqlite.csproj" /> <ProjectReference Include="..\Monitor.Backend.Storage.Sqlite\Monitor.Backend.Storage.Sqlite.csproj" />
</ItemGroup> </ItemGroup>
<Import Project="StaticSqlite.targets" />
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk.Web" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" Condition="'$(UseAotAnywhere)' == 'true'" />
</Project> </Project>
@@ -1,71 +0,0 @@
<Project>
<PropertyGroup>
<_SqliteToolchainTargets>SetupOSSpecificProps</_SqliteToolchainTargets>
<_SqliteToolchainTargets Condition="'$(UseAotAnywhere)' == 'true'">SetupOSSpecificProps;OverwriteTargetTriple;SetPathToZig</_SqliteToolchainTargets>
</PropertyGroup>
<ItemGroup Condition="'$(SqliteStaticLink)' == 'true'">
<DirectPInvoke Include="e_sqlite3" />
</ItemGroup>
<Target Name="PrepareStaticSqliteSource" Condition="'$(SqliteStaticLink)' == 'true'">
<Error Condition="'$(RuntimeIdentifier)' != 'linux-x64' and '$(RuntimeIdentifier)' != 'linux-arm64' and
'$(RuntimeIdentifier)' != 'linux-musl-x64' and '$(RuntimeIdentifier)' != 'linux-musl-arm64' and
'$(RuntimeIdentifier)' != 'osx-x64' and '$(RuntimeIdentifier)' != 'osx-arm64' and
'$(RuntimeIdentifier)' != 'win-x64' and '$(RuntimeIdentifier)' != 'win-arm64'"
Text="Static SQLite is configured for the eight supported x64/arm64 Linux, musl, macOS and Windows targets. Unsupported RID: $(RuntimeIdentifier)." />
<PropertyGroup>
<_SqliteCache>$([System.IO.Path]::GetFullPath('$(BaseIntermediateOutputPath)sqlite/$(MonitorSqliteSourceVersion)/'))</_SqliteCache>
<_SqliteArchive>$(_SqliteCache)sqlite-amalgamation-$(MonitorSqliteSourceVersion).zip</_SqliteArchive>
<_SqliteSource>$(_SqliteCache)sqlite-amalgamation-$(MonitorSqliteSourceVersion)/sqlite3.c</_SqliteSource>
<_SqliteObjectDirectory>$(_SqliteCache)$(RuntimeIdentifier)/$(UseAotAnywhere)/</_SqliteObjectDirectory>
<_SqliteObject>$(_SqliteObjectDirectory)e_sqlite3.o</_SqliteObject>
<_SqliteObject Condition="$(RuntimeIdentifier.StartsWith('win'))">$(_SqliteObjectDirectory)e_sqlite3.obj</_SqliteObject>
</PropertyGroup>
<MakeDir Directories="$(_SqliteObjectDirectory)" />
<DownloadFile SourceUrl="https://www.sqlite.org/$(MonitorSqliteSourceYear)/sqlite-amalgamation-$(MonitorSqliteSourceVersion).zip"
DestinationFolder="$(_SqliteCache)" Condition="!Exists('$(_SqliteArchive)')" />
<GetFileHash Files="$(_SqliteArchive)" Algorithm="SHA256">
<Output TaskParameter="Hash" PropertyName="_SqliteArchiveHash" />
</GetFileHash>
<Error Condition="'$(_SqliteArchiveHash.ToLowerInvariant())' != '$(MonitorSqliteSourceSha256)'"
Text="SQLite source SHA256 mismatch: $(_SqliteArchiveHash)." />
<Unzip SourceFiles="$(_SqliteArchive)" DestinationFolder="$(_SqliteCache)" Condition="!Exists('$(_SqliteSource)')" />
</Target>
<Target Name="CompileStaticSqliteObject" Condition="'$(SqliteStaticLink)' == 'true'"
DependsOnTargets="PrepareStaticSqliteSource;$(_SqliteToolchainTargets)"
Inputs="$(_SqliteSource);$(MSBuildThisFileFullPath)" Outputs="$(_SqliteObject)">
<PropertyGroup>
<_SqliteUseMsvc>false</_SqliteUseMsvc>
<_SqliteUseMsvc Condition="$(RuntimeIdentifier.StartsWith('win')) and '$(_AotAnywhereWindowsCross)' != 'true'">true</_SqliteUseMsvc>
<_SqliteCompiler>&quot;$(CppCompilerAndLinker)&quot;</_SqliteCompiler>
<_SqliteCompiler Condition="'$(CppCompilerAndLinker)' == ''">cc</_SqliteCompiler>
<_SqliteCompiler Condition="'$(TargetTriple)' != ''">$(_SqliteCompiler) --target=$(TargetTriple)</_SqliteCompiler>
<_SqliteCompiler Condition="'$(SysRoot)' != ''">$(_SqliteCompiler) --sysroot=&quot;$(SysRoot)&quot;</_SqliteCompiler>
<_SqliteCompiler Condition="'$(UseAotAnywhere)' == 'true'">&quot;$(_AotAnywhereZigExe)&quot; cc -target $(TargetTriple)</_SqliteCompiler>
<_SqliteCompiler Condition="'$(_SqliteUseMsvc)' == 'true'">cl</_SqliteCompiler>
</PropertyGroup>
<Message Importance="high" Text="Compiling SQLite $(MonitorSqliteSourceVersion) into the $(RuntimeIdentifier) NativeAOT executable." />
<Exec Condition="'$(_SqliteUseMsvc)' != 'true'"
Command="$(_SqliteCompiler) -Os -fPIC -c &quot;$(_SqliteSource)&quot; -o &quot;$(_SqliteObject)&quot; -DSQLITE_API= -DSQLITE_THREADSAFE=1 -DSQLITE_DEFAULT_MEMSTATUS=0 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_SNAPSHOT -DSQLITE_ENABLE_UNLOCK_NOTIFY" />
<Exec Condition="'$(_SqliteUseMsvc)' == 'true'"
Command="cl /nologo /O1 /MT /c &quot;$(_SqliteSource)&quot; /Fo&quot;$(_SqliteObject)&quot; /DSQLITE_API= /DSQLITE_THREADSAFE=1 /DSQLITE_DEFAULT_MEMSTATUS=0 /DSQLITE_ENABLE_COLUMN_METADATA /DSQLITE_ENABLE_FTS4 /DSQLITE_ENABLE_FTS5 /DSQLITE_ENABLE_MATH_FUNCTIONS /DSQLITE_ENABLE_RTREE /DSQLITE_ENABLE_SNAPSHOT /DSQLITE_ENABLE_UNLOCK_NOTIFY" />
</Target>
<!-- AotAnywhere runs before LinkNative; register the object before either linker starts. -->
<Target Name="LinkStaticSqlite" Condition="'$(SqliteStaticLink)' == 'true'"
BeforeTargets="AotAnywhereDirectLinkNative;AotAnywhereDirectLinkMacNative;AotAnywhereDirectLinkWindows;LinkNative"
DependsOnTargets="CompileStaticSqliteObject">
<ItemGroup>
<NativeLibrary Include="$(_SqliteObject)" />
<LinkerArg Include="&quot;$(_SqliteObject)&quot;" />
</ItemGroup>
</Target>
<Target Name="RemoveStaticSqliteRuntimeFiles" Condition="'$(SqliteStaticLink)' == 'true'" AfterTargets="ComputeFilesToPublish">
<ItemGroup>
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)"
Condition="'%(ResolvedFileToPublish.Filename)' == 'libe_sqlite3' or '%(ResolvedFileToPublish.Filename)' == 'e_sqlite3'" />
</ItemGroup>
</Target>
</Project>
@@ -256,7 +256,7 @@ public sealed class AgentEndToEndTests
string? startFailure = null; string? startFailure = null;
for (int attempt = 0; attempt < 5; attempt++) for (int attempt = 0; attempt < 5; attempt++)
{ {
accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000, ["duration_s"] = 5 } })); accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000 } }));
if (accepted.StatusCode == System.Net.HttpStatusCode.Accepted) if (accepted.StatusCode == System.Net.HttpStatusCode.Accepted)
{ {
break; break;
@@ -291,7 +291,7 @@ public sealed class AgentEndToEndTests
Assert.Equal(4, probe.Phases.Count); Assert.Equal(4, probe.Phases.Count);
Assert.All(probe.Phases, phase => Assert.Equal("udp_train", phase.Kind)); Assert.All(probe.Phases, phase => Assert.Equal("udp_train", phase.Kind));
// 单块最多 50000 条记录:50 和 100 Mbit/s 阶段(5 s、1000 B 的包)里 100 Mbit/s 的那一段有 6 万多条记录,被 Probe 切成两个列块 // 单块最多 50000 条记录:50 和 100 Mbit/s 阶段(10 s、1000 B 的包)各被 Probe 切成多个列块
Assert.True(agent.App.Counters.Fragments > 4); Assert.True(agent.App.Counters.Fragments > 4);
await BackendFixture.EventuallyAsync( await BackendFixture.EventuallyAsync(
async () => (await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == "udp_train") == agent.App.Counters.Fragments, async () => (await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == "udp_train") == agent.App.Counters.Fragments,
@@ -56,7 +56,7 @@ public sealed class AgentHarness : IAsyncDisposable
}; };
configure?.Invoke(config); configure?.Invoke(config);
LoadedNodeIdentity identity = LoadedNodeIdentity.Load(directory); LoadedNodeIdentity identity = LoadedNodeIdentity.Load(directory);
var app = new AgentApp(config, identity, directory, TimeProvider.System, new ConsoleNodeLog(TimeProvider.System, Console.Out)); var app = new AgentApp(config, identity, TimeProvider.System, new ConsoleNodeLog(TimeProvider.System, Console.Out));
var harness = new AgentHarness(backend, directory, identity, app); var harness = new AgentHarness(backend, directory, identity, app);
await app.Ready.WaitAsync(TimeSpan.FromSeconds(10)); await app.Ready.WaitAsync(TimeSpan.FromSeconds(10));
await BackendFixture.EventuallyAsync(() => Task.FromResult(app.Link.IsConnected), because: "the agent should connect to the backend"); await BackendFixture.EventuallyAsync(() => Task.FromResult(app.Link.IsConnected), because: "the agent should connect to the backend");
@@ -35,20 +35,11 @@ public sealed class AotSmokeTests
string token = JsonNode.Parse(await tokenResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>(); string token = JsonNode.Parse(await tokenResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token); http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
using HttpResponseMessage settingsResponse = await http.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["site.public"] = true }, "application/merge-patch+json"));
settingsResponse.EnsureSuccessStatusCode();
using HttpResponseMessage probeResponse = await http.PostAsync("/api/admin/probes", BackendFixture.Json(new JsonObject { ["name"] = "aot-probe-" + Guid.NewGuid().ToString("N")[..6] }));
probeResponse.EnsureSuccessStatusCode();
string probeId = JsonNode.Parse(await probeResponse.Content.ReadAsStringAsync())!["probe_id"]!.GetValue<string>();
// Agent:加入令牌 → 注册(ECDSA 持有证明)→ WebSocket 签名认证 → 期望状态 // Agent:加入令牌 → 注册(ECDSA 持有证明)→ WebSocket 签名认证 → 期望状态
using HttpResponseMessage joinResponse = await http.PostAsync("/api/admin/join-tokens", BackendFixture.Json(new JsonObject { ["max_uses"] = 1 })); using HttpResponseMessage joinResponse = await http.PostAsync("/api/admin/join-tokens", BackendFixture.Json(new JsonObject { ["max_uses"] = 1 }));
joinResponse.EnsureSuccessStatusCode();
string joinToken = JsonNode.Parse(await joinResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>(); string joinToken = JsonNode.Parse(await joinResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
using var anonymous = new HttpClient { BaseAddress = baseAddress }; using var anonymous = new HttpClient { BaseAddress = baseAddress };
await using AgentSimulator agent = await AgentSimulator.EnrollAsync(anonymous, joinToken, "aot-" + Guid.NewGuid().ToString("N")[..6]); await using AgentSimulator agent = await AgentSimulator.EnrollAsync(anonymous, joinToken, "aot-" + Guid.NewGuid().ToString("N")[..6]);
agent.ProbeForChunks = new ProbeId(probeId);
string lineId = $"{agent.NodeId.Value}~{probeId}";
await agent.ConnectAsync(baseAddress); await agent.ConnectAsync(baseAddress);
TaskSet tasks = await agent.WaitForTaskSetAsync(s => s.Tasks.Count >= 2); TaskSet tasks = await agent.WaitForTaskSetAsync(s => s.Tasks.Count >= 2);
Assert.Contains(tasks.Tasks, t => t.Kind == "udp_echo"); Assert.Contains(tasks.Tasks, t => t.Kind == "udp_echo");
@@ -63,7 +54,7 @@ public sealed class AotSmokeTests
// 查询:热层的精确汇总,随后是持久层的分钟窗口 // 查询:热层的精确汇总,随后是持久层的分钟窗口
string range = $"from={Iso(baseMs)}&to={Iso(baseMs + minute)}&metrics=loss.total,rtt.p95"; string range = $"from={Iso(baseMs)}&to={Iso(baseMs + minute)}&metrics=loss.total,rtt.p95";
JsonNode hot = await GetAsync(http, $"/api/public/lines/{lineId}/series?{range}&step=1s"); JsonNode hot = await GetAsync(http, $"/api/public/lines/{agent.NodeId.Value}/series?{range}&step=1s");
Assert.Equal("hot", hot["source"]!.GetValue<string>()); Assert.Equal("hot", hot["source"]!.GetValue<string>());
Assert.Equal(600, hot["summary"]!["samples"]!["udp_echo"]!.GetValue<long>()); Assert.Equal(600, hot["summary"]!["samples"]!["udp_echo"]!.GetValue<long>());
Assert.Equal(10.0, hot["summary"]!["metrics"]!["loss.total"]!.GetValue<double>(), 6); Assert.Equal(10.0, hot["summary"]!["metrics"]!["loss.total"]!.GetValue<double>(), 6);
@@ -71,7 +62,7 @@ public sealed class AotSmokeTests
JsonNode persisted; JsonNode persisted;
do do
{ {
persisted = await GetAsync(http, $"/api/public/lines/{lineId}/series?{range}&step=1m"); persisted = await GetAsync(http, $"/api/public/lines/{agent.NodeId.Value}/series?{range}&step=1m");
if (persisted["summary"]!["samples"]!["udp_echo"]?.GetValue<long>() == 600) if (persisted["summary"]!["samples"]!["udp_echo"]?.GetValue<long>() == 600)
{ {
break; break;
@@ -89,16 +80,16 @@ public sealed class AotSmokeTests
Assert.True(RawChunkCodec.Decode(bytes).IsSuccess); Assert.True(RawChunkCodec.Decode(bytes).IsSuccess);
// SSE:快照 + 增量 // SSE:快照 + 增量
await using SseClient sse = await SseClient.ConnectAsync(http, $"/api/public/live?topics=nodes,series:{lineId}:1s:udp"); await using SseClient sse = await SseClient.ConnectAsync(http, $"/api/public/live?topics=nodes,series:{agent.NodeId.Value}:1s:udp");
await sse.WaitForAsync(e => e.Name == "series"); await sse.WaitForAsync(e => e.Name == "series");
// 按需测试:Probe 事件 → 下发 → 状态 // 按需测试:Probe 事件 → 下发 → 状态
await agent.SendEventAsync(AgentEventKinds.ProbeConnected, probeId); await agent.SendEventAsync(AgentEventKinds.ProbeConnected);
JsonNode started = null!; JsonNode started = null!;
long begun = TimeProvider.System.GetTimestamp(); long begun = TimeProvider.System.GetTimestamp();
while (TimeProvider.System.GetElapsedTime(begun) < TimeSpan.FromSeconds(10)) while (TimeProvider.System.GetElapsedTime(begun) < TimeSpan.FromSeconds(10))
{ {
using HttpResponseMessage response = await http.PostAsync($"/api/admin/lines/{lineId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "reality_session" })); using HttpResponseMessage response = await http.PostAsync($"/api/admin/lines/{agent.NodeId.Value}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "reality_session" }));
if (response.StatusCode == System.Net.HttpStatusCode.Accepted) if (response.StatusCode == System.Net.HttpStatusCode.Accepted)
{ {
started = JsonNode.Parse(await response.Content.ReadAsStringAsync())!; started = JsonNode.Parse(await response.Content.ReadAsStringAsync())!;
@@ -58,39 +58,6 @@ public sealed class ApiBasicsTests
Assert.Equal("no-store", ok.Headers.CacheControl!.ToString()); Assert.Equal("no-store", ok.Headers.CacheControl!.ToString());
} }
[Fact]
public async Task Login_WithReturnSession_GivesABearerSession_WithoutACookie_AndNoCsrfNeeded()
{
using HttpClient client = _backend.NewClient();
using HttpResponseMessage login = await client.PostAsync("/api/admin/auth/login", BackendFixture.Json(new JsonObject { ["username"] = BackendFixture.AdminUser, ["password"] = BackendFixture.AdminPassword, ["return_session"] = true }));
Assert.Equal(HttpStatusCode.OK, login.StatusCode);
Assert.False(login.Headers.Contains("Set-Cookie"));
string session = JsonNode.Parse(await login.Content.ReadAsStringAsync())!["session_token"]!.GetValue<string>();
Assert.False(string.IsNullOrEmpty(session));
// 之后当 API 令牌那样用 Bearer;不是 Cookie,所以不需要 CSRF,也能做非 GET 操作
using var me = new HttpRequestMessage(HttpMethod.Get, "/api/admin/auth/me");
me.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage meResponse = await client.SendAsync(me);
Assert.Equal(HttpStatusCode.OK, meResponse.StatusCode);
Assert.Equal(BackendFixture.AdminUser, JsonNode.Parse(await meResponse.Content.ReadAsStringAsync())!["username"]!.GetValue<string>());
using var create = new HttpRequestMessage(HttpMethod.Post, "/api/admin/join-tokens") { Content = BackendFixture.Json(new JsonObject()) };
create.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage created = await client.SendAsync(create);
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
// 退出之后这个会话令牌就失效
using var logout = new HttpRequestMessage(HttpMethod.Post, "/api/admin/auth/logout");
logout.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage loggedOut = await client.SendAsync(logout);
Assert.Equal(HttpStatusCode.NoContent, loggedOut.StatusCode);
using var after = new HttpRequestMessage(HttpMethod.Get, "/api/admin/auth/me");
after.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage afterResponse = await client.SendAsync(after);
Assert.Equal(HttpStatusCode.Unauthorized, afterResponse.StatusCode);
}
[Fact] [Fact]
public async Task SessionCookie_RequiresCsrfHeaderOnUnsafeMethods() public async Task SessionCookie_RequiresCsrfHeaderOnUnsafeMethods()
{ {
@@ -192,8 +192,6 @@ public class BackendFixture : IAsyncLifetime
options.InitialAdminUser = AdminUser; options.InitialAdminUser = AdminUser;
options.InitialAdminPassword = AdminPassword; options.InitialAdminPassword = AdminPassword;
options.LoginAttemptsPerMinute = LoginAttemptsPerMinute; options.LoginAttemptsPerMinute = LoginAttemptsPerMinute;
options.GeoLookupUrl = string.Empty; // 测试里不访问外部服务
options.PhaseDurationScale = 0.2; // 端到端测试里 connect、hold 这些固定 30 秒的阶段缩成 6 秒
}); });
await App.StartAsync(); await App.StartAsync();
string address = App.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First(); string address = App.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First();
@@ -83,14 +83,6 @@ public sealed class EnrollmentAndTaskSetTests
using HttpResponseMessage revoke = await _backend.Admin.DeleteAsync("/api/admin/join-tokens/" + id); using HttpResponseMessage revoke = await _backend.Admin.DeleteAsync("/api/admin/join-tokens/" + id);
Assert.Equal(HttpStatusCode.NoContent, revoke.StatusCode); Assert.Equal(HttpStatusCode.NoContent, revoke.StatusCode);
await Assert.ThrowsAsync<InvalidOperationException>(() => AgentSimulator.EnrollAsync(_backend.Anonymous, revoked, _backend.UniqueName("revoked"))); await Assert.ThrowsAsync<InvalidOperationException>(() => AgentSimulator.EnrollAsync(_backend.Anonymous, revoked, _backend.UniqueName("revoked")));
// 吊销之后记录还在(显示「已吊销」);purge=true 才是彻底删除
tokens = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/join-tokens");
Assert.Contains(tokens["items"]!.AsArray(), t => t!["id"]!.GetValue<string>() == id);
using HttpResponseMessage purge = await _backend.Admin.DeleteAsync($"/api/admin/join-tokens/{id}?purge=true");
Assert.Equal(HttpStatusCode.NoContent, purge.StatusCode);
tokens = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/join-tokens");
Assert.DoesNotContain(tokens["items"]!.AsArray(), t => t!["id"]!.GetValue<string>() == id);
} }
[Fact] [Fact]
@@ -338,7 +330,7 @@ public sealed class EnrollmentAndTaskSetTests
JsonNode identity = JsonNode.Parse(await created.Content.ReadAsStringAsync())!; JsonNode identity = JsonNode.Parse(await created.Content.ReadAsStringAsync())!;
string id = identity["probe_id"]!.GetValue<string>(); string id = identity["probe_id"]!.GetValue<string>();
using var key1 = Fleet.Identity.NodeIdentity.FromPkcs8(Convert.FromBase64String(identity["private_key"]!.GetValue<string>())); using var key1 = Fleet.Identity.NodeIdentity.FromPkcs8(Convert.FromBase64String(identity["private_key"]!.GetValue<string>()));
Assert.True(Uri.TryCreate(identity["server"]!.GetValue<string>(), UriKind.Absolute, out _)); // Probe 直接连面板:probe.json 里是面板的地址 Assert.Contains(identity["bootstrap"]!.AsArray(), b => b!["agent_id"]!.GetValue<string>() == agent.NodeId.Value);
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == id && p.PublicKey == Convert.ToBase64String(key1.PublicKeySpki))); await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == id && p.PublicKey == Convert.ToBase64String(key1.PublicKeySpki)));
// The panel lists it; two probes can coexist and a duplicate name is refused // The panel lists it; two probes can coexist and a duplicate name is refused
@@ -374,6 +366,34 @@ public sealed class EnrollmentAndTaskSetTests
Assert.Equal(HttpStatusCode.NotFound, gone.StatusCode); Assert.Equal(HttpStatusCode.NotFound, gone.StatusCode);
} }
[Fact]
public async Task ProbeAgentList_IsServedToAgents_AndPushedWhenTheSetOfAgentsChanges()
{
await using AgentSimulator first = await _backend.ConnectAgentAsync();
ProbeAgentsResult list = await first.RequestProbeAgentsAsync();
Assert.Contains(list.Agents, a => a.AgentId == first.NodeId && a.Host == "203.0.113.10" && a.TcpPort == 7443);
Assert.All(list.Agents, a => Assert.Equal(32, Convert.FromBase64String(a.Spki).Length));
await using AgentSimulator second = await _backend.ConnectAgentAsync();
ProbeAgentsResult pushed = await WaitForAsync(first, l => l.Agents.Any(a => a.AgentId == second.NodeId));
Assert.True(pushed.Version > list.Version);
using HttpResponseMessage disable = await _backend.Admin.PostAsync($"/api/admin/agents/{second.NodeId.Value}/disable", null);
Assert.Equal(HttpStatusCode.OK, disable.StatusCode);
await WaitForAsync(first, l => l.Agents.All(a => a.AgentId != second.NodeId));
}
private static async Task<ProbeAgentsResult> WaitForAsync(AgentSimulator agent, Func<ProbeAgentsResult, bool> predicate)
{
ProbeAgentsResult? found = null;
await BackendFixture.EventuallyAsync(() =>
{
found = agent.ProbeAgentLists.ToArray().LastOrDefault(predicate);
return Task.FromResult(found is not null);
});
return found!;
}
private Task<HttpResponseMessage> PostEnrollAsync(JsonNode body) => private Task<HttpResponseMessage> PostEnrollAsync(JsonNode body) =>
_backend.Anonymous.PostAsync("/api/agent/v1/enroll", BackendFixture.Json(body)); _backend.Anonymous.PostAsync("/api/agent/v1/enroll", BackendFixture.Json(body));
} }
@@ -21,7 +21,6 @@ public sealed class HostSamplerTests
Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1)); Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1));
Assert.Equal(3600L, HostSampler.ParseUptime("3600.55 7000.1\n")); Assert.Equal(3600L, HostSampler.ParseUptime("3600.55 7000.1\n"));
Assert.Null(HostSampler.ParseNetwork("nothing useful")); Assert.Null(HostSampler.ParseNetwork("nothing useful"));
Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1 + "podman0: 8888 1 0 0 0 0 0 0 9999 1 0 0 0 0 0 0\n"));
} }
[Fact] [Fact]
@@ -30,7 +29,7 @@ public sealed class HostSamplerTests
var time = new Microsoft.Extensions.Time.Testing.FakeTimeProvider(); var time = new Microsoft.Extensions.Time.Testing.FakeTimeProvider();
string stat = Stat1; string stat = Stat1;
string dev = Dev1; string dev = Dev1;
var sampler = new HostSampler(time, path => path switch { "/proc/stat" => stat, "/proc/net/dev" => dev, "/proc/uptime" => "100.0 1.0", _ => null }, () => (1000, 250), procRoot: "/proc", sysRoot: "/sys"); var sampler = new HostSampler(time, path => path switch { "/proc/stat" => stat, "/proc/net/dev" => dev, "/proc/uptime" => "100.0 1.0", _ => null }, () => (1000, 250));
HostSnapshot first = sampler.Sample(); HostSnapshot first = sampler.Sample();
Assert.Null(first.CpuPercent); // 第一次没有基线 Assert.Null(first.CpuPercent); // 第一次没有基线
Assert.Null(first.NetRxBps); Assert.Null(first.NetRxBps);
@@ -46,54 +45,6 @@ public sealed class HostSamplerTests
Assert.Equal(2000, second.NetTxBps); Assert.Equal(2000, second.NetTxBps);
Assert.Equal(3000, second.NetRxTotal); Assert.Equal(3000, second.NetRxTotal);
} }
[Fact]
public void Sample_UsesMountedHostProcAndSys_InsteadOfContainerPaths()
{
var reads = new List<string>();
var sampler = new HostSampler(TimeProvider.System, path =>
{
reads.Add(path);
return path switch
{
"/host/proc/stat" => Stat1,
"/host/proc/meminfo" => "MemTotal: 8192 kB\nMemAvailable: 6144 kB\n",
"/host/proc/net/dev" => Dev1,
"/host/proc/uptime" => "3600.0 7000.0",
"/host/sys/class/net/eth0/speed" => "10000\n",
_ => throw new InvalidOperationException("unexpected container path: " + path),
};
}, () => (1000, 250), procRoot: "/host/proc", sysRoot: "/host/sys");
HostSnapshot snapshot = sampler.Sample();
Assert.Equal(8L * 1024 * 1024, snapshot.MemTotal);
Assert.Equal(1000, snapshot.NetRxTotal);
Assert.Equal(10000, snapshot.NetLinkMbps);
Assert.Equal(3600, snapshot.UptimeSeconds);
Assert.DoesNotContain(reads, path => path.StartsWith("/proc/", StringComparison.Ordinal));
Assert.DoesNotContain(reads, path => path.Contains("docker0", StringComparison.Ordinal) || path.Contains("/lo/", StringComparison.Ordinal));
}
[Theory]
[InlineData(null)]
[InlineData("-1")]
[InlineData("0")]
[InlineData("not supported")]
public void Sample_DoesNotInventLinkSpeedOrContainerDisk_WhenHostDataIsUnavailable(string? speed)
{
var sampler = new HostSampler(TimeProvider.System,
path => path.EndsWith("net/dev", StringComparison.Ordinal) ? Dev1 : path.EndsWith("/speed", StringComparison.Ordinal) ? speed : null,
procRoot: "/host/proc", sysRoot: "/host/sys", diskRoot: Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")));
HostSnapshot snapshot = sampler.Sample();
Assert.Null(snapshot.NetLinkMbps);
Assert.Null(snapshot.DiskTotal);
Assert.Null(snapshot.DiskUsed);
if (OperatingSystem.IsLinux())
{
Assert.Null(snapshot.UptimeSeconds);
}
}
} }
[Collection("backend")] [Collection("backend")]
@@ -110,7 +61,6 @@ public sealed class HostStatusTests
{ {
CpuPercent = cpu, MemUsed = 2L << 30, MemTotal = 8L << 30, DiskUsed = 10L << 30, DiskTotal = 40L << 30, CpuPercent = cpu, MemUsed = 2L << 30, MemTotal = 8L << 30, DiskUsed = 10L << 30, DiskTotal = 40L << 30,
NetRxTotal = 600L << 30, NetTxTotal = 180L << 30, NetRxBps = 2500, NetTxBps = 4800, UptimeSeconds = 3_000_000, NetRxTotal = 600L << 30, NetTxTotal = 180L << 30, NetRxBps = 2500, NetTxBps = 4800, UptimeSeconds = 3_000_000,
NetLinkMbps = 1000,
}; };
[Fact] [Fact]
@@ -131,7 +81,6 @@ public sealed class HostStatusTests
JsonNode item = list["items"]!.AsArray().Single(i => i!["agent_id"]!.GetValue<string>() == id)!; JsonNode item = list["items"]!.AsArray().Single(i => i!["agent_id"]!.GetValue<string>() == id)!;
Assert.False(item["stale"]!.GetValue<bool>()); Assert.False(item["stale"]!.GetValue<bool>());
Assert.Equal(40L << 30, item["host"]!["disk_total"]!.GetValue<long>()); Assert.Equal(40L << 30, item["host"]!["disk_total"]!.GetValue<long>());
Assert.Equal(1000, item["host"]!["net_link_mbps"]!.GetValue<long>());
// 展示页能拿到的只有汇总数字:整份响应里没有型号、主机名、地址之类的字段 // 展示页能拿到的只有汇总数字:整份响应里没有型号、主机名、地址之类的字段
string text = list.ToJsonString(); string text = list.ToJsonString();
@@ -1,81 +0,0 @@
using System.Net;
using System.Text.Json.Nodes;
using Monitor.TestSupport;
namespace Monitor.Backend.Tests;
[Collection("probe-e2e")]
public sealed class LinkTests
{
private readonly BackendFixture _backend;
public LinkTests(BackendFixture backend)
{
_backend = backend;
}
private async Task<HttpResponseMessage> SetLinkAsync(string probe, string agent, bool? enabled) =>
await _backend.Admin.PutAsync($"/api/admin/links/{probe}/{agent}", BackendFixture.Json(new JsonObject { ["enabled"] = enabled }));
private async Task SetAutoLinkAsync(bool on)
{
using HttpResponseMessage response = await _backend.Admin.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["fleet.auto_link"] = on }, "application/merge-patch+json"));
response.EnsureSuccessStatusCode();
}
private async Task<JsonNode?> LinkAsync(string probe, string agent) =>
(await _backend.GetJsonAsync(_backend.Admin, "/api/admin/links"))["items"]!.AsArray().FirstOrDefault(i => i!["probe_id"]!.GetValue<string>() == probe && i["agent_id"]!.GetValue<string>() == agent);
[Fact]
public async Task Links_FollowTheAutomaticSwitch_AndManualChoicesAlwaysWin()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
string probe = (await _backend.CreateProbeAsync())["probe_id"]!.GetValue<string>();
string agentId = agent.NodeId.Value;
try
{
// 自动连接(默认开):新 Probe 自动进入这个 Agent 允许的名单
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == probe));
JsonNode link = (await LinkAsync(probe, agentId))!;
Assert.True(link["linked"]!.GetValue<bool>());
Assert.Null(link["override"]);
// 手动关掉这一对:自动连接开着也不连
using HttpResponseMessage off = await SetLinkAsync(probe, agentId, false);
Assert.Equal(HttpStatusCode.OK, off.StatusCode);
await agent.WaitForTaskSetAsync(s => s.Peers.All(p => p.ProbeId.Value != probe));
Assert.False((await LinkAsync(probe, agentId))!["linked"]!.GetValue<bool>());
// 取消手动设置:回到自动连接
using HttpResponseMessage reset = await SetLinkAsync(probe, agentId, null);
Assert.Equal(HttpStatusCode.OK, reset.StatusCode);
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == probe));
// 关掉自动连接:没有手动选中的就都不连
await SetAutoLinkAsync(false);
await agent.WaitForTaskSetAsync(s => s.Peers.All(p => p.ProbeId.Value != probe));
Assert.False((await _backend.GetJsonAsync(_backend.Admin, "/api/admin/links"))["auto_link"]!.GetValue<bool>());
// 自动连接关着,手动选中的照样连
await SetLinkAsync(probe, agentId, true);
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == probe));
Assert.True((await LinkAsync(probe, agentId))!["override"]!.GetValue<bool>());
}
finally
{
await SetLinkAsync(probe, agentId, null);
await SetAutoLinkAsync(true); // 共用的后端:恢复默认,别影响别的测试
}
}
[Fact]
public async Task Links_RefuseUnknownNodes()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
string probe = (await _backend.CreateProbeAsync())["probe_id"]!.GetValue<string>();
using HttpResponseMessage noProbe = await SetLinkAsync("prb_missing", agent.NodeId.Value, false);
Assert.Equal(HttpStatusCode.NotFound, noProbe.StatusCode);
using HttpResponseMessage noAgent = await SetLinkAsync(probe, "agt_missing", false);
Assert.Equal(HttpStatusCode.NotFound, noAgent.StatusCode);
}
}
@@ -1,310 +0,0 @@
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Time.Testing;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Tests;
public sealed partial class NodeInfoTests
{
[Theory]
[InlineData(9808, "China Mobile Communications Group", "mobile", "中国移动")]
[InlineData(4134, "Chinanet", "telecom", "中国电信")]
[InlineData(4837, "China Unicom Backbone", "unicom", "中国联通")]
[InlineData(null, "China Mobile Communications Corporation", "mobile", "中国移动")]
[InlineData(null, "CHINANET-BACKBONE", "telecom", "中国电信")]
[InlineData(null, "China Unicom Beijing Province Network", "unicom", "中国联通")]
[InlineData(null, "China Broadnet Corporation", "broadnet", "中国广电")]
public void IspClassifier_RecognizesTheChineseCarriers(int? asn, string org, string key, string name)
{
Assert.Equal((key, name), IspClassifier.Classify(asn, org));
}
[Fact]
public void IspClassifier_OtherCarriersKeepAShortenedOrganizationName()
{
(string key, string name) = IspClassifier.Classify(37963, "Hangzhou Alibaba Advertising Co.,Ltd.");
Assert.Equal("other", key);
Assert.Equal("Hangzhou Alibaba Advertising", name);
Assert.Equal("Google", IspClassifier.Classify(15169, "Google LLC").Name);
Assert.True(IspClassifier.Classify(1, new string('x', 80)).Name.Length <= 28);
}
[Theory]
[InlineData("8.8.8.8", true)]
[InlineData("223.5.5.5", true)]
[InlineData("2001:4860:4860::8888", true)]
[InlineData("example.com", true)]
[InlineData("127.0.0.1", false)]
[InlineData("10.1.2.3", false)]
[InlineData("172.16.0.1", false)]
[InlineData("172.32.0.1", true)]
[InlineData("192.168.1.10", false)]
[InlineData("169.254.1.1", false)]
[InlineData("100.64.0.1", false)]
[InlineData("0.0.0.0", false)]
[InlineData("::1", false)]
[InlineData("fe80::1", false)]
[InlineData("fd00::1", false)]
[InlineData("::ffff:192.168.1.1", false)]
[InlineData("localhost", false)]
[InlineData("printer.local", false)]
public void OnlyPublicAddressesAreEverSentToTheLookupService(string host, bool expected)
{
Assert.Equal(expected, NodeInfoService.IsPublic(host));
}
[Fact]
public async Task Service_LooksAPublicAddressUpOnce_KeepsOnlyCountryAndIsp_AndNeverQueriesPrivateOnes()
{
var geo = new FakeGeo(new GeoInfo("cn", 9808, "China Mobile Communications Group", false));
var store = new MemoryStore();
var time = new FakeTimeProvider(DateTimeOffset.Parse("2026-10-06T00:00:00Z"));
var service = new NodeInfoService(store, geo, time);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "10.0.0.5", CancellationToken.None);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "127.0.0.1", CancellationToken.None);
Assert.Empty(geo.Calls);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.73.168.45", CancellationToken.None);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.73.168.45", CancellationToken.None);
Assert.Single(geo.Calls);
NodeInfo info = Assert.Single(await service.ListAsync(CancellationToken.None));
Assert.Equal(("CN", "mobile", "中国移动"), (info.Country, info.IspKey, info.IspName));
// 展示用的信息里没有地址:整个对象序列化出来,找不到任何 IP 形状的字符串
Assert.DoesNotMatch(Ipv4(), info.ToString());
// 地址变了才重新查;过了一周也重新查
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.5.5.5", CancellationToken.None);
Assert.Equal(2, geo.Calls.Count);
time.Advance(TimeSpan.FromDays(8));
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.5.5.5", CancellationToken.None);
Assert.Equal(3, geo.Calls.Count);
}
[Fact]
public async Task Service_FailedLookups_AreNotRetriedForAWhile_AndNeverThrow()
{
var geo = new FakeGeo(null);
var time = new FakeTimeProvider(DateTimeOffset.Parse("2026-10-06T00:00:00Z"));
var service = new NodeInfoService(new MemoryStore(), geo, time);
await service.ObserveAddressAsync("agt_1", NodeKinds.Agent, "8.8.8.8", CancellationToken.None);
await service.ObserveAddressAsync("agt_1", NodeKinds.Agent, "8.8.8.8", CancellationToken.None);
Assert.Single(geo.Calls);
time.Advance(TimeSpan.FromMinutes(11));
await service.ObserveAddressAsync("agt_1", NodeKinds.Agent, "8.8.8.8", CancellationToken.None);
Assert.Equal(2, geo.Calls.Count);
Assert.Null(Assert.Single(await service.ListAsync(CancellationToken.None)).Country);
}
[Fact]
public async Task Service_PlatformIsSavedAndSurvivesARestart()
{
var store = new MemoryStore();
var first = new NodeInfoService(store, new FakeGeo(null), TimeProvider.System);
await first.SetPlatformAsync("agt_1", NodeKinds.Agent, new PlatformInfo { Os = "linux", Name = "Debian 12", Arch = "arm64" }, CancellationToken.None);
var second = new NodeInfoService(store, new FakeGeo(null), TimeProvider.System);
NodeInfo info = Assert.Single(await second.ListAsync(CancellationToken.None));
Assert.Equal(("linux", "Debian 12", "arm64"), (info.Os, info.OsName, info.Arch));
}
[Fact]
public void GeoParser_ReadsOnlyCountryAsnAndOrganization()
{
using JsonDocument doc = JsonDocument.Parse("""
{"ip":"223.5.5.5","found":true,"is_private":false,
"country_region":{"iso_code":"CN","subdivisions":[{"names":{"zh-CN":"浙江省"}}]},
"city":{"names":{"zh-CN":"杭州市"}},
"location":{"latitude":30.25,"longitude":120.21},
"network":{"cidr":"223.4.0.0/14","asn":37963,"organization":"Hangzhou Alibaba Advertising Co.,Ltd.","isp":""}}
""");
GeoInfo? geo = HttpGeoLookup.Parse(doc.RootElement);
Assert.Equal(new GeoInfo("CN", 37963, "Hangzhou Alibaba Advertising Co.,Ltd.", false), geo);
Assert.Null(HttpGeoLookup.Parse(JsonDocument.Parse("""{"found":false}""").RootElement));
}
[GeneratedRegex(@"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}")]
private static partial Regex Ipv4();
private sealed class FakeGeo : IGeoLookup
{
private readonly GeoInfo? _result;
public FakeGeo(GeoInfo? result)
{
_result = result;
}
public List<string> Calls { get; } = [];
public ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct)
{
Calls.Add(host);
return ValueTask.FromResult(_result);
}
}
private sealed class MemoryStore : INodeMetaStore
{
private readonly Dictionary<string, NodeMeta> _items = [];
public ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct) => ValueTask.FromResult<IReadOnlyList<NodeMeta>>([.. _items.Values]);
public ValueTask SaveAsync(NodeMeta meta, CancellationToken ct)
{
_items[meta.Info.NodeId] = meta;
return ValueTask.CompletedTask;
}
public ValueTask DeleteAsync(string nodeId, CancellationToken ct)
{
_items.Remove(nodeId);
return ValueTask.CompletedTask;
}
}
}
/// <summary>
/// 访客能拿到的东西里不能有 IP 等敏感信息:公开接口的整个响应里不出现 IP 形状的字符串,也没有 ip、city、asn、经纬度这些字段。
/// Agent 和 Probe 这里是真的连上了(Probe 的来源地址是 127.0.0.1),所以如果哪里把地址带出来了,这里一定能发现。
/// </summary>
[Collection("probe-e2e")]
public sealed partial class NodePrivacyTests
{
private static readonly string[] s_forbiddenKeys = ["ip", "city", "asn", "latitude", "longitude", "cidr", "remote", "public_host", "geo_host", "host_name", "hostname"];
private readonly BackendFixture _backend;
public NodePrivacyTests(BackendFixture backend)
{
_backend = backend;
}
[Fact]
public async Task PublicEndpoints_NeverExposeAnAddress_AndNodesCarryOnlyPlatformCountryAndIsp()
{
await _backend.SiteMustBePublicAsync();
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
await agent.OverrideLineTasksAsync();
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
await BackendFixture.EventuallyAsync(async () => (await _backend.GetJsonAsync(_backend.Admin, "/api/admin/agents/" + agent.Id))["probe_connected"]!.GetValue<bool>());
// 两端都自报了系统和架构(Agent 在 hello 里,Probe 随列表同步)
JsonNode? agentNode = null;
JsonNode? probeNode = null;
await BackendFixture.EventuallyAsync(
async () =>
{
JsonNode nodes = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes");
agentNode = nodes["items"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == agent.Id);
probeNode = nodes["items"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == probe.ProbeId);
return agentNode?["os"] is not null && probeNode?["os"] is not null && agentNode["arch"] is not null && probeNode["arch"] is not null;
},
TimeSpan.FromSeconds(40),
"the agent (hello) and the probe (agent list sync) both report their platform");
Assert.Equal("agent", agentNode!["kind"]!.GetValue<string>());
Assert.Equal("probe", probeNode!["kind"]!.GetValue<string>());
string[] paths =
[
"/api/public/nodes",
"/api/public/hosts",
"/api/public/overview?range=1h&metrics=rtt.p50",
$"/api/public/lines/{Uri.EscapeDataString(agent.Id + "~" + probe.ProbeId)}",
"/api/public/catalog/plans",
];
foreach (string path in paths)
{
using HttpResponseMessage response = await _backend.Anonymous.GetAsync(path);
Assert.True(response.IsSuccessStatusCode, $"{path}: {(int)response.StatusCode}");
string body = await response.Content.ReadAsStringAsync();
Assert.DoesNotMatch(Ipv4(), body);
Assert.DoesNotContain("::1", body, StringComparison.Ordinal);
foreach (string key in KeysOf(JsonNode.Parse(body)))
{
Assert.DoesNotContain(key, s_forbiddenKeys);
}
}
}
[Fact]
public async Task BuildInfo_IsHiddenFromVisitorsUntilTheSwitchIsOn_AndAlwaysVisibleToAdmins()
{
await _backend.SiteMustBePublicAsync();
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
// 默认关:访客看不到版本和 tag,「关于」里只有 show_build: false
JsonNode about = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/about");
Assert.False(about["show_build"]!.GetValue<bool>());
Assert.Null(about["backend"]);
JsonNode hidden = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes");
Assert.All(hidden["items"]!.AsArray(), n =>
{
Assert.Null(n!["version"]);
Assert.Null(n["tag"]);
});
// 管理端一直看得到:后端自己的版本和每个节点上报的版本
JsonNode admin = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info");
Assert.False(string.IsNullOrEmpty(admin["backend"]!["version"]!.GetValue<string>()));
await BackendFixture.EventuallyAsync(
async () => (await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info"))["nodes"]!.AsArray().Any(n => n!["id"]!.GetValue<string>() == agent.Id && n["version"] is not null),
TimeSpan.FromSeconds(30),
"the agent reports its version in hello");
using HttpResponseMessage on = await _backend.Admin.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["site.show_build"] = true }, "application/merge-patch+json"));
on.EnsureSuccessStatusCode();
try
{
JsonNode shown = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/about");
Assert.True(shown["show_build"]!.GetValue<bool>());
Assert.False(string.IsNullOrEmpty(shown["backend"]!["version"]!.GetValue<string>()));
Assert.Null(shown["backend"]!["commit"]); // 访客只看版本、tag、构建时间
JsonNode visible = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes");
JsonNode self = visible["items"]!.AsArray().First(n => n!["id"]!.GetValue<string>() == agent.Id)!;
Assert.False(string.IsNullOrEmpty(self["version"]!.GetValue<string>()));
}
finally
{
using HttpResponseMessage off = await _backend.Admin.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["site.show_build"] = false }, "application/merge-patch+json"));
off.EnsureSuccessStatusCode();
}
}
private static IEnumerable<string> KeysOf(JsonNode? node)
{
switch (node)
{
case JsonObject obj:
foreach ((string key, JsonNode? value) in obj)
{
yield return key;
foreach (string inner in KeysOf(value))
{
yield return inner;
}
}
break;
case JsonArray array:
foreach (JsonNode? item in array)
{
foreach (string inner in KeysOf(item))
{
yield return inner;
}
}
break;
}
}
[GeneratedRegex(@"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b")]
private static partial Regex Ipv4();
}
@@ -175,8 +175,9 @@ public sealed class OperationsAndSecurityTests
Assert.Equal("fleet_unsupported", reply.Error!.Data!.Code); Assert.Equal("fleet_unsupported", reply.Error!.Data!.Code);
} }
// 连接仍然可用:路由器照常回复 // 连接仍然可用
Assert.Equal("fleet_unsupported", (await agent.RequestRawAsync("fleet.exec")).Error!.Data!.Code); ProbeAgentsResult list = await agent.RequestProbeAgentsAsync();
Assert.Contains(list.Agents, a => a.AgentId == agent.NodeId);
} }
[Fact] [Fact]
@@ -194,6 +195,9 @@ public sealed class OperationsAndSecurityTests
public async Task PendingNodes_CannotCallProductMethods() public async Task PendingNodes_CannotCallProductMethods()
{ {
await using AgentSimulator agent = await _backend.ConnectAgentAsync(autoApprove: false); await using AgentSimulator agent = await _backend.ConnectAgentAsync(autoApprove: false);
RpcMessage reply = await agent.RequestRawAsync(MonitorMethods.ProbeAgents, JsonSerializer.SerializeToElement(new ProbeAgentsParams(), MonitorJsonContext.Default.ProbeAgentsParams));
Assert.Equal("fleet_node_pending", reply.Error!.Data!.Code);
// 状态与事件也不会被接受:监控数据里没有它 // 状态与事件也不会被接受:监控数据里没有它
await agent.SendEventAsync(AgentEventKinds.ProbeConnected, _backend.DefaultProbeId); await agent.SendEventAsync(AgentEventKinds.ProbeConnected, _backend.DefaultProbeId);
await Task.Delay(200); await Task.Delay(200);
@@ -19,13 +19,13 @@ public sealed class ProbeEndToEndTests
(await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == kind); (await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == kind);
[Fact] [Fact]
public async Task Probe_GetsTheAgentListFromThePanel_RunsTheLineTasks_AndTheBackendStoresMergedChunks() public async Task Probe_ConnectsThroughTheBootstrapList_RunsTheLineTasks_AndTheBackendStoresMergedChunks()
{ {
await using AgentHarness agent = await AgentHarness.StartAsync(_backend); await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
await agent.OverrideLineTasksAsync(rateHz: 30); await agent.OverrideLineTasksAsync(rateHz: 30);
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend); await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.IsConnected(agent.Id)), because: "the panel sends the agent list to the probe, which then connects"); await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.IsConnected(agent.Id)), because: "the probe should connect to the bootstrap agent");
await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>()); await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>());
await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.Connection(agent.Id)!.RoleCount == 2), because: "udp_echo and tcp_echo"); await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.Connection(agent.Id)!.RoleCount == 2), because: "udp_echo and tcp_echo");
@@ -38,26 +38,6 @@ public sealed class ProbeEndToEndTests
Assert.InRange(probe.App.Connection(agent.Id)!.ClockOffsetUs, -20_000, 20_000); Assert.InRange(probe.App.Connection(agent.Id)!.ClockOffsetUs, -20_000, 20_000);
} }
[Fact]
public async Task Probe_IntroducesItselfToThePanel_AndARevokedProbeIsLockedOut()
{
await _backend.SiteMustBePublicAsync();
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
string path = "/api/admin/probes/" + probe.ProbeId;
// 直连面板:在线,hello 里的系统、架构和版本都被记下来了
await BackendFixture.EventuallyAsync(async () => (await _backend.GetJsonAsync(_backend.Admin, path))["online"]!.GetValue<bool>(), because: "the probe links to the panel");
await BackendFixture.EventuallyAsync(
async () => (await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes"))["items"]!.AsArray().Any(n => n!["id"]!.GetValue<string>() == probe.ProbeId && n["os"] is not null && n["arch"] is not null),
because: "the platform comes from the probe's own hello");
// 吊销:面板立刻断开它,之后连不上
using HttpResponseMessage revoked = await _backend.Admin.PostAsync(path + "/revoke", null);
revoked.EnsureSuccessStatusCode();
await BackendFixture.EventuallyAsync(async () => !(await _backend.GetJsonAsync(_backend.Admin, path))["online"]!.GetValue<bool>(), because: "a revoked probe is disconnected and cannot come back");
Assert.False(probe.App.Link.IsConnected);
}
[Fact] [Fact]
public async Task Probe_FollowsTheAgentListThePanelKnows_AndSavesItForTheNextStart() public async Task Probe_FollowsTheAgentListThePanelKnows_AndSavesItForTheNextStart()
{ {
@@ -84,7 +64,7 @@ public sealed class ProbeEndToEndTests
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend); await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>()); await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>());
using HttpResponseMessage accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000, ["duration_s"] = 5 } })); using HttpResponseMessage accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000 } }));
Assert.True(accepted.StatusCode == System.Net.HttpStatusCode.Accepted, await accepted.Content.ReadAsStringAsync()); Assert.True(accepted.StatusCode == System.Net.HttpStatusCode.Accepted, await accepted.Content.ReadAsStringAsync());
string testId = JsonNode.Parse(await accepted.Content.ReadAsStringAsync())!["id"]!.GetValue<string>(); string testId = JsonNode.Parse(await accepted.Content.ReadAsStringAsync())!["id"]!.GetValue<string>();
@@ -98,7 +78,7 @@ public sealed class ProbeEndToEndTests
TimeSpan.FromSeconds(90)); TimeSpan.FromSeconds(90));
Assert.True(test["status"]!.GetValue<string>() == "succeeded", test.ToJsonString()); Assert.True(test["status"]!.GetValue<string>() == "succeeded", test.ToJsonString());
await BackendFixture.EventuallyAsync(async () => await ChunkCountAsync(agent, "udp_train") >= 5, because: "the 100 Mbit/s phase (5 s of 1000 B packets, over 50000 records) is split into two fragments"); await BackendFixture.EventuallyAsync(async () => await ChunkCountAsync(agent, "udp_train") >= 5, because: "50 and 100 Mbit/s phases are split into several fragments");
Assert.Equal(0, agent.App.Counters.FragmentsDiscarded); Assert.Equal(0, agent.App.Counters.FragmentsDiscarded);
} }
@@ -1,130 +0,0 @@
using System.Net;
using System.Text.Json.Nodes;
using Monitor.Protocol;
using Monitor.TestSupport;
namespace Monitor.Backend.Tests;
[Collection("probe-e2e")]
public sealed class UpdateTests
{
private const string Digest = "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
private readonly BackendFixture _backend;
public UpdateTests(BackendFixture backend)
{
_backend = backend;
}
// 面板不验签(节点宿主机上的更新器验),所以测试里签名随便写一个格式对的
private static string Manifest(string component, long seq, string digest = Digest) =>
$$"""{"schema":1,"component":"{{component}}","channel":"dev","image":"registry.example/monitor-dev/monitor-{{component}}","digest":"{{digest}}","tag":"dev-{{seq}}","version":"0.1.1","commit":"abc","built_at":"2026-10-06T00:00:00Z","seq":{{seq}}}""";
private async Task<HttpResponseMessage> PublishAsync(string manifest, string signature = "MEUCIQDc") =>
await _backend.Admin.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = signature }));
[Fact]
public async Task Release_IsForwardedByteForByte_ToOnlineNodesOfThatComponent_AndToNodesThatConnectLater()
{
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000;
await using AgentSimulator online = await _backend.ConnectAgentAsync();
string manifest = Manifest("agent", seq);
using HttpResponseMessage published = await PublishAsync(manifest);
Assert.Equal(HttpStatusCode.Created, published.StatusCode);
JsonNode release = JsonNode.Parse(await published.Content.ReadAsStringAsync())!;
Assert.Equal("agent", release["component"]!.GetValue<string>());
Assert.Equal(Digest, release["digest"]!.GetValue<string>());
await BackendFixture.EventuallyAsync(() => Task.FromResult(online.UpdateOffers.ToArray().Any(o => o.Manifest == manifest && o.Signature == "MEUCIQDc")), because: "the offer reaches online agents unchanged");
// 之后才连上的节点,hello 之后补发
await using AgentSimulator late = await _backend.ConnectAgentAsync();
await BackendFixture.EventuallyAsync(() => Task.FromResult(late.UpdateOffers.ToArray().Any(o => o.Manifest == manifest)), because: "a node that connects later still gets the latest offer");
// 另一个组件(probe)的清单不会发给 Agent
using HttpResponseMessage probeRelease = await PublishAsync(Manifest("probe", seq));
Assert.Equal(HttpStatusCode.Created, probeRelease.StatusCode);
await Task.Delay(300);
Assert.DoesNotContain(online.UpdateOffers.ToArray(), o => o.Manifest.Contains("\"component\":\"probe\"", StringComparison.Ordinal));
}
[Fact]
public async Task Release_Registration_RefusesBadManifests_StaleSeq_AndNeedsAnAdmin()
{
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000 + 5;
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync("{not json")).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("agent", seq, digest: "sha256:short"))).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("backend", seq))).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("agent", seq), signature: "not base64!")).StatusCode);
Assert.Equal(HttpStatusCode.Created, (await PublishAsync(Manifest("agent", seq))).StatusCode);
Assert.Equal(HttpStatusCode.Conflict, (await PublishAsync(Manifest("agent", seq))).StatusCode); // 同一个 seq:重放
Assert.Equal(HttpStatusCode.Conflict, (await PublishAsync(Manifest("agent", seq - 1))).StatusCode); // 更旧的 seq
using HttpResponseMessage anonymous = await _backend.Anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("agent", seq + 1), ["signature"] = "MEUC" }));
Assert.Contains(anonymous.StatusCode, new[] { HttpStatusCode.Unauthorized, HttpStatusCode.Forbidden });
}
[Fact]
public async Task ReleaseToken_CanOnlyRegisterReleases_AndAReadTokenCannotEvenDoThat()
{
async Task<string> NewTokenAsync(string scope)
{
using HttpResponseMessage created = await _backend.Admin.PostAsync("/api/admin/api-tokens", BackendFixture.Json(new JsonObject { ["name"] = _backend.UniqueName("ci"), ["scope"] = scope }));
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
return JsonNode.Parse(await created.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
}
HttpClient Client(string token)
{
var client = new HttpClient { BaseAddress = _backend.BaseAddress };
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
return client;
}
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000 + 50;
using HttpClient release = Client(await NewTokenAsync("release"));
using HttpClient read = Client(await NewTokenAsync("read"));
var body = BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq), ["signature"] = "MEUC" });
using HttpResponseMessage forbidden = await read.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq), ["signature"] = "MEUC" }));
Assert.Equal(HttpStatusCode.Forbidden, forbidden.StatusCode);
using HttpResponseMessage published = await release.PostAsync("/api/admin/releases", body);
Assert.Equal(HttpStatusCode.Created, published.StatusCode);
// release 令牌只能登记发布清单:读不了任何东西,也不能做别的管理操作
Assert.Equal(HttpStatusCode.Forbidden, (await release.GetAsync("/api/admin/agents")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await release.GetAsync("/api/admin/releases")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await release.PostAsync("/api/admin/nodes/agt_x/update", null)).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await _backend.Anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq + 1), ["signature"] = "MEUC" }))).StatusCode);
}
[Fact]
public async Task UpdateRequest_ReachesTheNode_AndItsReportedStatusShowsInBuildInfo()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
using HttpResponseMessage request = await _backend.Admin.PostAsync($"/api/admin/nodes/{agent.NodeId.Value}/update", null);
Assert.Equal(HttpStatusCode.NoContent, request.StatusCode);
await BackendFixture.EventuallyAsync(() => Task.FromResult(agent.UpdateApplies == 1));
// 节点(宿主机更新器)报告状态:面板在构建信息里显示
await agent.SendUpdateStatusAsync(new UpdateStatus { State = "available", Available = new UpdateAvailable { Tag = "dev-9", Version = "0.1.1", Digest = Digest, Seq = 9 }, At = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() });
await BackendFixture.EventuallyAsync(async () =>
{
JsonNode info = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info");
JsonNode? node = info["nodes"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == agent.NodeId.Value);
return node?["update"]?["state"]?.GetValue<string>() == "available" && node["update"]!["available_tag"]!.GetValue<string>() == "dev-9";
}, because: "the update state comes from the node's heartbeat");
// 节点离线之后不能请求更新
string id = agent.NodeId.Value;
await agent.DisposeAsync();
await BackendFixture.EventuallyAsync(async () =>
{
using HttpResponseMessage offline = await _backend.Admin.PostAsync($"/api/admin/nodes/{id}/update", null);
return offline.StatusCode == HttpStatusCode.Conflict || offline.StatusCode == HttpStatusCode.NotFound;
});
}
}
@@ -46,15 +46,10 @@ public sealed class AgentSimulator : IAsyncDisposable
public ConcurrentQueue<TaskSet> TaskSets { get; } = new(); public ConcurrentQueue<TaskSet> TaskSets { get; } = new();
public ConcurrentQueue<ProbeAgentsResult> ProbeAgentLists { get; } = new();
public ConcurrentQueue<TestRunParams> TestRuns { get; } = new(); public ConcurrentQueue<TestRunParams> TestRuns { get; } = new();
/// <summary>面板发来的发布清单(<c>monitor.update.offer</c>)和更新请求(<c>monitor.update.apply</c>)。</summary>
public ConcurrentQueue<UpdateOfferParams> UpdateOffers { get; } = new();
private int _updateApplies;
public int UpdateApplies => Volatile.Read(ref _updateApplies);
public ConcurrentQueue<TestId> TestCancels { get; } = new(); public ConcurrentQueue<TestId> TestCancels { get; } = new();
public IReadOnlyList<RejectedItem> RejectNextState { get; set; } = []; public IReadOnlyList<RejectedItem> RejectNextState { get; set; } = [];
@@ -195,12 +190,11 @@ public sealed class AgentSimulator : IAsyncDisposable
new TestStatusParams { TestId = test, Status = status, Phase = phase, ErrorCode = errorCode }, new TestStatusParams { TestId = test, Status = status, Phase = phase, ErrorCode = errorCode },
MonitorJsonContext.Default.TestStatusParams)); MonitorJsonContext.Default.TestStatusParams));
/// <summary>像节点的更新器报告的那样,随心跳上报更新状态。</summary> public async Task<ProbeAgentsResult> RequestProbeAgentsAsync()
public Task SendUpdateStatusAsync(UpdateStatus status) => {
SendAsync(RpcMessage.Notification( RpcMessage reply = await RequestAsync(MonitorMethods.ProbeAgents, JsonSerializer.SerializeToElement(new ProbeAgentsParams(), MonitorJsonContext.Default.ProbeAgentsParams));
FleetMethods.Heartbeat, return reply.Result!.Value.Deserialize(MonitorJsonContext.Default.ProbeAgentsResult)!;
new HeartbeatParams { Extra = JsonSerializer.SerializeToElement(new AgentHeartbeatExtra { Update = status }, MonitorJsonContext.Default.AgentHeartbeatExtra) }, }
FleetRpcContext.Default.HeartbeatParams));
public async Task<RpcMessage> RequestRawAsync(string method, JsonElement? parameters = null) => await RequestAsync(method, parameters); public async Task<RpcMessage> RequestRawAsync(string method, JsonElement? parameters = null) => await RequestAsync(method, parameters);
@@ -403,12 +397,6 @@ public sealed class AgentSimulator : IAsyncDisposable
case FleetMethods.Revoked: case FleetMethods.Revoked:
Revoked = true; Revoked = true;
break; break;
case MonitorMethods.UpdateOffer:
UpdateOffers.Enqueue(message.Params!.Value.Deserialize(MonitorJsonContext.Default.UpdateOfferParams)!);
break;
case MonitorMethods.UpdateApply:
Interlocked.Increment(ref _updateApplies);
break;
case MonitorMethods.TestRun: case MonitorMethods.TestRun:
TestRunParams run = message.Params!.Value.Deserialize(MonitorJsonContext.Default.TestRunParams)!; TestRunParams run = message.Params!.Value.Deserialize(MonitorJsonContext.Default.TestRunParams)!;
TestRuns.Enqueue(run); TestRuns.Enqueue(run);
@@ -428,6 +416,9 @@ public sealed class AgentSimulator : IAsyncDisposable
TestCancels.Enqueue(cancel.TestId); TestCancels.Enqueue(cancel.TestId);
await SendAsync(RpcMessage.Success(message.Id!.Value, new TestRunResult { Accepted = true }, MonitorJsonContext.Default.TestRunResult)); await SendAsync(RpcMessage.Success(message.Id!.Value, new TestRunResult { Accepted = true }, MonitorJsonContext.Default.TestRunResult));
break; break;
case MonitorMethods.ProbeAgentsChanged:
ProbeAgentLists.Enqueue(message.Params!.Value.Deserialize(MonitorJsonContext.Default.ProbeAgentsResult)!);
break;
} }
} }
} }
@@ -61,6 +61,8 @@ public sealed class ProbeSimulator : IAsyncDisposable
public ConcurrentQueue<TestId> Aborts { get; } = new(); public ConcurrentQueue<TestId> Aborts { get; } = new();
public ConcurrentQueue<ProbeAgentsResult> AgentLists { get; } = new();
public bool IsConnected => _rpc is not null && !_stop.IsCancellationRequested; public bool IsConnected => _rpc is not null && !_stop.IsCancellationRequested;
public long NowUs() => (TimeProvider.System.GetTimestamp() - _origin) * 1_000_000 / TimeProvider.System.TimestampFrequency; public long NowUs() => (TimeProvider.System.GetTimestamp() - _origin) * 1_000_000 / TimeProvider.System.TimestampFrequency;
@@ -345,6 +347,14 @@ public sealed class ProbeSimulator : IAsyncDisposable
probe.Aborts.Enqueue(abort.TestId); probe.Aborts.Enqueue(abort.TestId);
return ValueTask.CompletedTask; return ValueTask.CompletedTask;
}) })
.OnNotify(
MonitorMethods.ProbeAgentsChanged,
MonitorJsonContext.Default.ProbeAgentsResult,
(probe, list, _) =>
{
probe.AgentLists.Enqueue(list);
return ValueTask.CompletedTask;
})
.OnNotify( .OnNotify(
FleetMethods.OutboxAck, FleetMethods.OutboxAck,
FleetRpcContext.Default.OutboxAckParams, FleetRpcContext.Default.OutboxAckParams,
-95
View File
@@ -1,95 +0,0 @@
#!/usr/bin/env bash
# Build and push one opt-in main Linux amd64 image to the dev repository in Zot.
set -euo pipefail
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
echo "main amd64 builds require a main push with [CI] in the commit message" >&2
exit 2
}
component="${1:?usage: build-main-amd64.sh <backend|agent|probe>}"
case "$component" in
backend|agent|probe) ;;
*) echo "unsupported component: $component" >&2; exit 2 ;;
esac
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
echo "Zot credentials are required" >&2
exit 2
}
[[ "${CI_COMMIT_SHA:-}" =~ ^[0-9a-fA-F]{40}$ ]] || {
echo "CI_COMMIT_SHA must be a full commit SHA" >&2
exit 2
}
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
registry="${ZOT_REGISTRY:-registry.oeo.one}"
repository="$registry/monitor-dev/monitor-$component"
work="$root/.release/main-amd64-build"
base_home="/home/justme/woodpecker/buildah-home/main-amd64-build-$component"
short_sha="${CI_COMMIT_SHA:0:12}"
dev_tag="dev-$short_sha"
mkdir -p "$work" "$base_home"
if [[ "$component" == backend ]]; then
BUILDAH_HOME="$base_home" \
ZOT_REGISTRY="$registry" \
ZOT_IMAGE_REPOSITORY="$repository" \
IMAGE_TAGS="latest,$dev_tag" \
IMAGE_PUBLISH=true \
bash "$root/ci/release-linux.sh" amd64 backend
exit 0
fi
export HOME="$base_home"
export BUILDAH_HOME="$base_home"
export REGISTRY_AUTH_FILE="$work/$component-auth.json"
context="$work/$component"
local_image="localhost/monitor-$component:main-amd64-check"
python3 "$root/ci/stage-node-image.py" "$component" "$context"
buildah bud --layers --network host --tls-verify=true --arch amd64 \
--build-arg APP_VERSION=main \
--build-arg "MONITOR_TAG=$dev_tag" --build-arg "MONITOR_COMMIT=${CI_COMMIT_SHA:0:10}" --build-arg "MONITOR_BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
-f "$context/Containerfile" \
-t "$local_image" "$context"
logged_in=false
for attempt in 1 2 3 4; do
if buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$registry" <<< "$ZOT_SECRET" >/dev/null; then
logged_in=true
break
fi
if [[ "$attempt" -lt 4 ]]; then
echo "Zot login attempt $attempt failed; retrying in 3 seconds" >&2
sleep 3
fi
done
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
for tag in latest "$dev_tag"; do
pushed=false
for attempt in 1 2 3; do
if buildah push --tls-verify=true --digestfile "$work/$component.digest" "$local_image" "docker://$repository:$tag"; then
pushed=true
break
fi
if [[ "$attempt" -lt 3 ]]; then
echo "Zot push for $component:$tag attempt $attempt failed; retrying in 5 seconds" >&2
sleep 5
fi
done
[[ "$pushed" == true ]] || { echo "could not push $repository:$tag after 3 attempts" >&2; exit 1; }
echo "pushed development image $repository:$tag"
done
# 发布清单:给镜像的 digest 签名(私钥只在 CI 密钥里),再登记到面板;面板把它原样转给在线的节点,节点宿主机上的更新器用本机的公钥验签后才会更新。
# 没有配置签名密钥(例如 fork)就只构建镜像、不发清单。
if [[ -n "${RELEASE_SIGNING_KEY:-}" ]]; then
digest="$(cat "$work/$component.digest")"
release_dir="$work/$component-release"
bash "$root/ci/release-manifest.sh" "$component" "$repository" "$digest" "$dev_tag" dev "$release_dir"
if [[ -n "${RELEASE_API_URL:-}" && -n "${RELEASE_API_TOKEN:-}" ]]; then
python3 -c 'import json,sys; print(json.dumps({"manifest": open(sys.argv[1]).read(), "signature": open(sys.argv[2]).read()}))' "$release_dir/manifest.json" "$release_dir/manifest.sig" | curl --fail-with-body --silent --show-error --retry 3 -X POST "${RELEASE_API_URL%/}/api/admin/releases" -H "Authorization: Bearer $RELEASE_API_TOKEN" -H 'Content-Type: application/json' --data-binary @- >/dev/null
echo "registered the signed $component release with the panel"
fi
fi
-6
View File
@@ -51,11 +51,6 @@ $output = Join-Path $root ".release\publish\$RuntimeIdentifier\$Component"
$buildRoot = Join-Path $root ".release\build\$RuntimeIdentifier-$Component" $buildRoot = Join-Path $root ".release\build\$RuntimeIdentifier-$Component"
$null = New-Item -ItemType Directory -Force -Path $output $null = New-Item -ItemType Directory -Force -Path $output
# 构建信息:程序里显示的 tag、提交、构建时间
$env:MONITOR_TAG = if ($env:CI_COMMIT_TAG) { $env:CI_COMMIT_TAG } else { "main-$($env:CI_COMMIT_SHA.Substring(0, 10))" }
$env:MONITOR_COMMIT = $env:CI_COMMIT_SHA.Substring(0, 10)
$env:MONITOR_BUILT_AT = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
Push-Location $backend Push-Location $backend
try { try {
& dotnet publish $project ` & dotnet publish $project `
@@ -64,7 +59,6 @@ try {
-r $RuntimeIdentifier ` -r $RuntimeIdentifier `
--self-contained true ` --self-contained true `
-p:PublishAot=true ` -p:PublishAot=true `
-p:UseAotAnywhere=true `
-p:StripSymbols=true ` -p:StripSymbols=true `
-p:DebugType=none ` -p:DebugType=none `
-p:DebugSymbols=false ` -p:DebugSymbols=false `
+24 -68
View File
@@ -1,82 +1,52 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build a musl NativeAOT Backend OCI image with statically linked SQLite; publishing is optional. # Publish one framework-dependent Backend OCI image for the requested arch.
set -euo pipefail set -euo pipefail
arch="${1:?usage: release-linux.sh <amd64|arm64>}" arch="${1:?usage: release-linux.sh <amd64|arm64>}"
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
tag="${CI_COMMIT_TAG:-}" tag="${CI_COMMIT_TAG:-}"
if [[ -n "$tag" ]]; then [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || {
[[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || {
echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2 echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2
exit 2 exit 2
}
version="${tag#v}"
else
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
echo "untagged backend builds require a main push with [CI] in the commit message" >&2
exit 2
}
version=main
fi
image_publish="${IMAGE_PUBLISH:-true}"
[[ "$image_publish" == true || "$image_publish" == false ]] || {
echo "IMAGE_PUBLISH must be true or false" >&2; exit 2;
} }
if [[ "$image_publish" == true ]]; then [[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
echo "Zot credentials are required" >&2; exit 2; echo "Zot credentials are required" >&2; exit 2;
} }
fi
case "$arch" in case "$arch" in
amd64) rid=linux-musl-x64 ;; amd64|arm64) ;;
arm64) rid=linux-musl-arm64 ;;
*) echo "unsupported architecture: $arch" >&2; exit 2 ;; *) echo "unsupported architecture: $arch" >&2; exit 2 ;;
esac esac
version="${tag#v}"
project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)" project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
if [[ -n "$tag" && "$version" != "$project_version" ]]; then if [[ "$version" != "$project_version" ]]; then
echo "tag version $version does not match project version $project_version" >&2 echo "tag version $version does not match project version $project_version" >&2
exit 1 exit 1
fi fi
registry="${ZOT_REGISTRY:-registry.oeo.one}" registry="${ZOT_REGISTRY:-registry.oeo.one}"
image="${ZOT_IMAGE_REPOSITORY:-$registry/monitor/monitor-backend}" image="$registry/monitor/monitor-backend"
output="$root/.release/linux-$arch/backend" output="$root/.release/linux-$arch/backend"
build_root="$root/.release/build/linux-$arch-backend" build_root="$root/.release/build/linux-$arch-backend"
container_context="$root/.release/container/linux-$arch" container_context="$root/.release/container/linux-$arch"
mkdir -p "$output" "$build_root" "$container_context/app" "$container_context/data" mkdir -p "$output" "$build_root" "$container_context/app" "$container_context/data"
# 构建信息:tag 发布用 tag,main 构建用 main-<提交>
export MONITOR_TAG="${MONITOR_TAG:-${tag:-main-${CI_COMMIT_SHA:0:10}}}"
export MONITOR_COMMIT="${MONITOR_COMMIT:-${CI_COMMIT_SHA:0:10}}"
export MONITOR_BUILT_AT="${MONITOR_BUILT_AT:-$(date -u +%Y-%m-%dT%H:%M:%SZ)}"
(cd "$root/backend" && dotnet publish src/Monitor.Backend \ (cd "$root/backend" && dotnet publish src/Monitor.Backend \
-m:3 -c Release -r "$rid" --self-contained true \ -m:3 -c Release --no-self-contained \
-p:DebugType=none -p:DebugSymbols=false \ -p:DebugType=none -p:DebugSymbols=false \
-p:PublishAot=true -p:UseAotAnywhere=true -p:SqliteStaticLink=true -p:OpenApiGenerateDocuments=false \ -p:UseAppHost=false -p:OpenApiGenerateDocuments=false \
--artifacts-path "$build_root" -o "$output") --artifacts-path "$build_root" -o "$output")
[[ -x "$output/monitor-backend" ]] || { echo "NativeAOT Backend executable is missing" >&2; exit 1; } find "$output" -type f -name '*.pdb' -delete
[[ "$(wc -c < "$output/monitor-backend")" -gt 2000000 ]] || { echo "Backend output is an apphost, not a NativeAOT executable" >&2; exit 1; } cp -a "$output/." "$container_context/app/"
install -m 0755 "$output/monitor-backend" "$container_context/app/monitor-backend"
: > "$container_context/data/.keep" : > "$container_context/data/.keep"
# Woodpecker's local backend isolates HOME per workflow. Keep Buildah's image # Woodpecker's local backend isolates HOME per workflow. Keep Buildah's image
# store outside that temporary directory so the runtime base image is reusable # store outside that temporary directory so the runtime base image is reusable
# across releases. Separate stores avoid concurrent amd64/arm64 writers. # across releases. Separate stores avoid concurrent amd64/arm64 writers.
buildah_home="${BUILDAH_HOME:-/home/justme/woodpecker/buildah-home/$arch}" buildah_home="/home/justme/woodpecker/buildah-home/$arch"
mkdir -p "$buildah_home" mkdir -p "$buildah_home"
export HOME="$buildah_home" export HOME="$buildah_home"
local_image="localhost/monitor-backend:$version-$arch"
buildah bud --tls-verify=true --arch "$arch" \
--build-arg "APP_VERSION=$version" \
-f "$root/packaging/container/Backend.Containerfile" \
-t "$local_image" "$container_context"
if [[ "$image_publish" == false ]]; then
echo "built local Backend image for $arch; not publishing it"
exit 0
fi
export REGISTRY_AUTH_FILE="$root/.release/auth/zot-linux-$arch.json" export REGISTRY_AUTH_FILE="$root/.release/auth/zot-linux-$arch.json"
mkdir -p "$(dirname "$REGISTRY_AUTH_FILE")" mkdir -p "$(dirname "$REGISTRY_AUTH_FILE")"
logged_in=false logged_in=false
@@ -91,10 +61,14 @@ for attempt in 1 2 3 4; do
fi fi
done done
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; } [[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
push_tag() { local_image="localhost/monitor-backend:$version-$arch"
local target_tag="$1" pushed=false buildah bud --tls-verify=true --arch "$arch" \
for attempt in 1 2 3; do --build-arg "APP_VERSION=$version" \
if buildah push --tls-verify=true "$local_image" "docker://$image:$target_tag"; then -f "$root/packaging/container/Backend.Containerfile" \
-t "$local_image" "$container_context"
pushed=false
for attempt in 1 2 3; do
if buildah push --tls-verify=true "$local_image" "docker://$image:$version-$arch"; then
pushed=true pushed=true
break break
fi fi
@@ -102,24 +76,6 @@ push_tag() {
echo "Zot push attempt $attempt failed; retrying in 5 seconds" >&2 echo "Zot push attempt $attempt failed; retrying in 5 seconds" >&2
sleep 5 sleep 5
fi fi
done done
[[ "$pushed" == true ]] || { echo "could not push image to Zot after 3 attempts" >&2; return 1; } [[ "$pushed" == true ]] || { echo "could not push image to Zot after 3 attempts" >&2; exit 1; }
echo "pushed Zot image $image:$target_tag" echo "pushed Zot image $image:$version-$arch"
}
if [[ -n "${IMAGE_TAGS:-}" ]]; then
IFS=',' read -r -a image_tags <<< "$IMAGE_TAGS"
((${#image_tags[@]} > 0)) || { echo "IMAGE_TAGS must contain at least one tag" >&2; exit 2; }
for image_tag in "${image_tags[@]}"; do
[[ "$image_tag" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]] || {
echo "invalid image tag: $image_tag" >&2
exit 2
}
push_tag "$image_tag"
done
else
push_tag "$version-$arch"
if [[ -z "$tag" ]]; then
push_tag "$version"
fi
fi
-30
View File
@@ -1,30 +0,0 @@
#!/usr/bin/env bash
# 生成并签名一个镜像的发布清单:ci/release-manifest.sh <组件> <镜像仓库> <digest> <tag> <通道> <输出目录>
# 签名私钥从环境变量 RELEASE_SIGNING_KEY(PEM 内容)读取,只写进一个 600 权限的临时文件,用完即删;私钥只放在 CI 密钥里,不进仓库、不上传面板。
# 输出:<输出目录>/manifest.json(扁平 JSON,签名对象是它的原始字节)和 manifest.sig(base64 的 DER ECDSA 签名)。
set -euo pipefail
component="${1:?usage: release-manifest.sh <component> <repository> <digest> <tag> <channel> <outdir>}"
repository="${2:?}"; digest="${3:?}"; tag="${4:?}"; channel="${5:?}"; outdir="${6:?}"
[[ "$component" =~ ^(agent|probe|backend)$ ]] || { echo "unsupported component: $component" >&2; exit 2; }
[[ "$digest" =~ ^sha256:[0-9a-f]{64}$ ]] || { echo "digest must look like sha256:<64 hex>" >&2; exit 2; }
[[ "$repository" =~ ^[a-z0-9./_-]+$ && "$tag" =~ ^[A-Za-z0-9._-]+$ && "$channel" =~ ^[a-z0-9._-]+$ ]] || { echo "unexpected characters in repository, tag or channel" >&2; exit 2; }
[[ -n "${RELEASE_SIGNING_KEY:-}" ]] || { echo "RELEASE_SIGNING_KEY is required" >&2; exit 2; }
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
commit="$(git -C "$root" rev-parse --short=10 HEAD 2>/dev/null || true)"
built_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
seq="$(date +%s)"
mkdir -p "$outdir"
printf '{"schema":1,"component":"%s","channel":"%s","image":"%s","digest":"%s","tag":"%s","version":"%s","commit":"%s","built_at":"%s","seq":%s}' \
"$component" "$channel" "$repository" "$digest" "$tag" "$version" "$commit" "$built_at" "$seq" > "$outdir/manifest.json"
key="$(mktemp)"
trap 'rm -f "$key" "$outdir/manifest.sig.bin"' EXIT
chmod 600 "$key"
printf '%s\n' "$RELEASE_SIGNING_KEY" > "$key"
openssl dgst -sha256 -sign "$key" -out "$outdir/manifest.sig.bin" "$outdir/manifest.json"
base64 -w0 "$outdir/manifest.sig.bin" > "$outdir/manifest.sig" 2>/dev/null || base64 "$outdir/manifest.sig.bin" | tr -d '\n' > "$outdir/manifest.sig"
echo "signed $component $tag ($digest) seq=$seq"
+3 -3
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Publish musl OCI images for the Linux Agent and Probe on amd64 and arm64. # Publish glibc OCI images for the Linux Agent and Probe on amd64 and arm64.
set -euo pipefail set -euo pipefail
tag="${CI_COMMIT_TAG:-}" tag="${CI_COMMIT_TAG:-}"
@@ -113,7 +113,7 @@ publish_index() {
} }
for component in agent probe; do for component in agent probe; do
build_arch_image "$component" amd64 musl-amd64 build_arch_image "$component" amd64 amd64
build_arch_image "$component" arm64 musl-arm64 build_arch_image "$component" arm64 arm64
publish_index "$component" publish_index "$component"
done done
-73
View File
@@ -1,73 +0,0 @@
#!/usr/bin/env python3
"""Stage source and restore files for the Linux amd64 Agent or Probe image build."""
from __future__ import annotations
import argparse
import shutil
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
BACKEND = ROOT / "backend"
IGNORED_PARTS = {".git", ".release", "artifacts", "bin", "obj", "TestResults", "tests"}
GLOBAL_BACKEND_FILES = {
"Directory.Build.props",
"Directory.Build.targets",
"Directory.Packages.props",
"global.json",
"NuGet.Config",
"NuGet.config",
}
# The container stages backend as a standalone tree. Copy the repository-wide
# SDK and package versions into it so restore/publish use the same configuration.
ROOT_BUILD_FILES = {"Directory.Packages.props", "global.json"}
CONTAINERFILES = {
"agent": ROOT / "packaging/container/Agent.NativeBuild.Containerfile",
"probe": ROOT / "packaging/container/Probe.NativeBuild.Containerfile",
}
def stage(component: str, destination: Path) -> None:
if destination.exists():
shutil.rmtree(destination)
restore = destination / "restore/backend"
source = destination / "source/backend"
restore.mkdir(parents=True)
source.mkdir(parents=True)
shutil.copy2(CONTAINERFILES[component], destination / "Containerfile")
for path in BACKEND.rglob("*"):
if not path.is_file() or IGNORED_PARTS.intersection(path.parts):
continue
relative = path.relative_to(BACKEND)
if (
path.suffix in {".csproj", ".props", ".targets", ".slnx"}
or path.name in GLOBAL_BACKEND_FILES
or path.name == "packages.lock.json"
):
target = restore / relative
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(path, target)
target = source / relative
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(path, target)
for name in ROOT_BUILD_FILES:
for directory in (restore, source):
shutil.copy2(ROOT / name, directory / name)
config = destination / "config"
config.mkdir()
(config / ".keep").write_text("", encoding="utf-8")
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("component", choices=tuple(CONTAINERFILES))
parser.add_argument("destination", type=Path)
args = parser.parse_args()
stage(args.component, args.destination)
if __name__ == "__main__":
main()
+4 -21
View File
@@ -7,7 +7,7 @@
```text ```text
/api/public/* 只读。站点设为公开时匿名可访问,设为私有时需要登录或只读令牌; /api/public/* 只读。站点设为公开时匿名可访问,设为私有时需要登录或只读令牌;
public_visible = false 的 Agent 对匿名访客隐藏 public_visible = false 的 Agent 对匿名访客隐藏
/api/admin/* 管理。需要 Cookie 会话(含 CSRF 校验)或 admin 作用域的 API 令牌(`POST /api/admin/releases` 也接受只能登记发布清单的 release 令牌) /api/admin/* 管理。需要 Cookie 会话(含 CSRF 校验)或 admin 作用域的 API 令牌
/api/agent/v1/* 节点专用:注册(使用加入令牌)和 RPC(使用签名认证) /api/agent/v1/* 节点专用:注册(使用加入令牌)和 RPC(使用签名认证)
/healthz /metrics /openapi/v1.json /install.sh /install.ps1 /healthz /metrics /openapi/v1.json /install.sh /install.ps1
``` ```
@@ -20,7 +20,6 @@
├─ live SSE 实时推送 ├─ live SSE 实时推送
├─ lines/{line_id} 线路详情(线路 = Agent 与 Probe 的一对,id 形如 agt_…~prb_…) ├─ lines/{line_id} 线路详情(线路 = Agent 与 Probe 的一对,id 形如 agt_…~prb_…)
├─ lines/{line_id}/series 历史曲线(指标与评分) ├─ lines/{line_id}/series 历史曲线(指标与评分)
├─ nodes about 节点的展示信息(系统、国家、运营商,开启后带版本和 tag);后端的版本信息
├─ compare 多条线路对比 ├─ compare 多条线路对比
├─ tests/{test_id} 测试结果 ├─ tests/{test_id} 测试结果
└─ catalog/{kinds|plans|metrics|models} 目录(由代码注册表生成) └─ catalog/{kinds|plans|metrics|models} 目录(由代码注册表生成)
@@ -30,12 +29,10 @@
├─ agents agents/{id} agents/{id}/{approve|disable|enable|takeover} ├─ agents agents/{id} agents/{id}/{approve|disable|enable|takeover}
├─ join-tokens ├─ join-tokens
├─ probes probes/{id} probes/{id}/{rotate|revoke} Probe 身份(后端生成、吊销、删除) ├─ probes probes/{id} probes/{id}/{rotate|revoke} Probe 身份(后端生成、吊销、删除)
├─ links links/{probe_id}/{agent_id} 线路开关:哪些(Probe,Agent)连成线路(自动连接 + 手动选择)
├─ tasks tasks/{id} 线路模板、针对单个 Agent 的覆盖、目标任务 ├─ tasks tasks/{id} 线路模板、针对单个 Agent 的覆盖、目标任务
├─ lines/{line_id}/tests 在某条线路(某个 Probe)上发起按需测试;tests/{id}/cancel ├─ lines/{line_id}/tests 在某条线路(某个 Probe)上发起按需测试;tests/{id}/cancel
├─ raw/chunks raw/chunks/{id} 原始数据检索与下载 ├─ raw/chunks raw/chunks/{id} 原始数据检索与下载
├─ build-info 后端和每个节点的版本、tag、提交、构建时间、更新状态 ├─ releases 应用版本、灰度、批准(Fleet.Server.Releases)
├─ releases nodes/{id}/update CI 签名的发布清单(登记、列表);请求节点更新
├─ events analysis/recompute api-tokens settings users ├─ events analysis/recompute api-tokens settings users
``` ```
@@ -88,15 +85,6 @@
`host` 只有汇总的数字:CPU 占用、内存与根分区的已用和总量、网卡累计收发字节与瞬时网速、系统运行时长。**没有 CPU 型号、主机名、网卡名,也没有任何 IP**——Agent 不采集这些,公开接口当然也不会返回;有测试断言响应里不出现这类字段。实时更新走 SSE 的 `hosts` 主题,历史每个 Agent 每分钟存一个样本(`retention.host_days`,默认 7 天)。 `host` 只有汇总的数字:CPU 占用、内存与根分区的已用和总量、网卡累计收发字节与瞬时网速、系统运行时长。**没有 CPU 型号、主机名、网卡名,也没有任何 IP**——Agent 不采集这些,公开接口当然也不会返回;有测试断言响应里不出现这类字段。实时更新走 SSE 的 `hosts` 主题,历史每个 Agent 每分钟存一个样本(`retention.host_days`,默认 7 天)。
### 2.3.2 节点的展示信息
`GET /api/public/nodes`:看得见的 Agent 和没被吊销的 Probe 的展示信息,`{id, kind, os, os_name, arch, country, isp_key, isp_name}`——系统(`linux` / `windows` / `macos`)、「Debian 12」这样的系统名、架构(`x64` / `arm64`…)、国家代码(`CN`)、运营商类别(`mobile` / `telecom` / `unicom` / `broadnet` / `other`)和名字。**没有 IP 地址、城市、ASN、经纬度**,有测试保证:公开接口的整个响应里不出现 IP 形状的字符串,也没有 `ip`、`city`、`asn` 这些字段。管理员在设置里打开 `site.show_build` 之后,每项还有 `version` 和 `tag`(节点自己上报的程序版本和 tag);关着时这两个字段是 `null`。
`GET /api/public/about`:`{show_build, backend}`。`show_build` 关着时 `backend` 是 `null`;打开后 `backend` 是 `{version, tag, built_at}`(后端自己的版本、tag、构建时间,不含提交号)。
- 系统和架构是节点自报的:Agent 和 Probe 都在各自连面板时的 `fleet.hello` 里自报。只有粗粒度的名字,没有主机名、内核版本、CPU 型号。
- 国家和运营商由 **Backend** 用节点的来源地址向 IP 归属服务查一次(`Monitor__GeoLookupUrl`,缺省 `https://ip.007995.xyz`,留空就不查):Probe 用 Agent 看到的来源 IP,Agent 用它的公开地址。**Probe 和 Agent 不访问任何外部服务。** 只查公网地址(回环、内网、CGNAT、链路本地一律不发出去);同一个地址一周内只查一次,失败 10 分钟后再试;查到的只留国家代码和运营商,地址只存在 Backend 内部的 `node_meta.geo_host`,不对外。
### 2.4 测试结果与目录 ### 2.4 测试结果与目录
- `GET /api/public/tests/{test_id}`:包括计划、参数、状态、阶段,`metrics` 与 `scores`,`series`(例如码率–丢包曲线),以及置信说明 `caveats`。 - `GET /api/public/tests/{test_id}`:包括计划、参数、状态、阶段,`metrics` 与 `scores`,`series`(例如码率–丢包曲线),以及置信说明 `caveats`。
@@ -133,16 +121,11 @@
| GET | `/api/admin/raw/chunks/{chunk_id}` | 下载原始块(`application/vnd.monitor.rawchunk`);过期返回 410 `raw_expired` | | GET | `/api/admin/raw/chunks/{chunk_id}` | 下载原始块(`application/vnd.monitor.rawchunk`);过期返回 410 `raw_expired` |
| GET | `/api/admin/events?agent_id=&kind=&from=&to=` | 事件 | | GET | `/api/admin/events?agent_id=&kind=&from=&to=` | 事件 |
| GET | `/api/admin/analysis/recompute` | 指标重算进度 | | GET | `/api/admin/analysis/recompute` | 指标重算进度 |
| GET/POST/DELETE | `/api/admin/api-tokens` | 作用域为 `read`、`admin` 或 `release`(只能登记发布清单,给 CI 用,什么都读不了);明文只显示一次 | | GET/POST/DELETE | `/api/admin/api-tokens` | 作用域为 `read` 或 `admin`;明文只显示一次 |
| GET | `/api/admin/releases` | 各应用、各通道的当前版本,以及节点的版本分布(由 `Fleet.Server.Releases` 提供) | | GET | `/api/admin/releases` | 各应用、各通道的当前版本,以及节点的版本分布(由 `Fleet.Server.Releases` 提供) |
| POST | `/api/admin/releases/{app}/{channel}/rollout` | `{version, batches:[{labels?, count?}], soak_minutes}`:分批灰度;只能选择已经签名的清单 | | POST | `/api/admin/releases/{app}/{channel}/rollout` | `{version, batches:[{labels?, count?}], soak_minutes}`:分批灰度;只能选择已经签名的清单 |
| POST | `/api/admin/releases/{app}/approve` | 对 `policy = notify` 的节点批准更新 `{version, agents?}` | | POST | `/api/admin/releases/{app}/approve` | 对 `policy = notify` 的节点批准更新 `{version, agents?}` |
| GET/PATCH | `/api/admin/settings` | 站点公开或私有、是否对访客显示构建信息(`site.show_build`)、保留期等 | | GET/PATCH | `/api/admin/settings` | 站点公开或私有、保留期等 |
| GET | `/api/admin/build-info` | 后端和每个 Agent、Probe 的版本、tag、提交、构建时间、系统,以及节点宿主机更新器报告的更新状态(管理端「系统 → 版本」) |
| GET/POST | `/api/admin/releases` | CI 签名的发布清单:POST 登记(`{manifest, signature}`,面板不验签、原样转给在线的节点和之后连上的节点),GET 看每个组件最新的一份 |
| GET | `/api/admin/links` | 每一对(Probe,Agent)的线路开关:`linked`(最终结果)、`override`(手动设置,`null` 跟随自动连接)、`connected`,以及 `auto_link` |
| PUT | `/api/admin/links/{probe_id}/{agent_id}` | `{enabled: true|false|null}`:手动强制连或不连,`null` 取消手动设置;改完立即重新下发 |
| POST | `/api/admin/nodes/{id}/update` | 请求一个在线的 Agent 或 Probe 更新到它收到的清单(节点不在线返回 409) |
| GET/POST/PATCH | `/api/admin/users` | 首次启动时通过命令行或环境变量创建初始管理员 | | GET/POST/PATCH | `/api/admin/users` | 首次启动时通过命令行或环境变量创建初始管理员 |
任务的请求体示例: 任务的请求体示例:
+28 -49
View File
@@ -4,29 +4,26 @@
一个**只关注网络层**的线路质量面板:单体面板(Backend)+ 轻量节点(Probe、Agent)+ 长连接 RPC + 任务下发。核心能力: 一个**只关注网络层**的线路质量面板:单体面板(Backend)+ 轻量节点(Probe、Agent)+ 长连接 RPC + 任务下发。核心能力:
- 在**若干 Probe**(家宽等需要观测的出口,可以有多个)与**多台海外 VPS Agent**(10 几台)之间,采集**最原始的逐包、逐事件数据**;哪些 Probe 连哪些 Agent 由面板决定(自动全连,也可以手动选择); - 在**一个家宽 Probe** 与**多台海外 VPS Agent**(10 几台)之间,采集**最原始的逐包、逐事件数据**;
- 由 Backend 的分析层推算:**如果在某条线路上部署 REALITY 类、Hysteria2 类或 Minecraft 服务,体验会怎样**,并对多条线路排序和对比; - 由 Backend 的分析层推算:**如果在某条线路上部署 REALITY 类、Hysteria2 类或 Minecraft 服务,体验会怎样**,并对多条线路排序和对比;
- **主机层信息只上报只读的汇总数字**(2026-10 起,由用户确认放开):Agent 读 `/proc` 得到 CPU 占用、内存、根分区、网卡累计流量与瞬时网速、运行时长,不采集 CPU 型号、主机名、网卡名和任何 IP,不采集进程;Probe 不上报主机资源。节点只在连接面板时自报粗粒度的系统、架构和构建信息(版本、tag、提交、构建时间),用于展示和更新;国家和运营商由 Backend 用节点的来源地址查询,节点自己不访问任何外部服务。展示页看不到任何 IP。Agent 和 Probe 仍然是**纯采集器**,不具备 SSH、远程执行、终端、文件读写、自更新等能力:更新由节点宿主机上的更新器完成,节点进程只往数据目录放面板转来的发布清单和更新请求。 - **主机层信息只上报只读的汇总数字**(2026-10 起,由用户确认放开):Agent 读 `/proc` 得到 CPU 占用、内存、根分区、网卡累计流量与瞬时网速、运行时长,不采集 CPU 型号、主机名、网卡名和任何 IP,不采集进程;Probe 不上报。展示页只能拿到这些数字。Agent 和 Probe 仍然是**纯采集器**,不具备 SSH、远程执行、终端、文件读写、远程更新等能力。
## 2. 拓扑 ## 2. 拓扑
```text ```text
Web(展示端 · 管理端) ┌────────────────── Backend(面板)───────────────────┐
▲ REST + SSE 前端(待开发)◀─────────┤ Api(public|admin) │ 注册中心 │ 任务编排 │ 结果管线 │
┌──────────────────────────────────────────────────────────┐ REST + SSE 实时 │ Analysis(指标注册表) │ LiveHub │ 存储(段文件+SQLite) │
│ Backend(面板) │ └────▲──────▲──────▲───────────────▲─────────────────┘
│ Api · Registry · Planner · Links │ wss JSON-RPC │ │ │ │ 每个 Agent 一条长连接(由 Agent 发起)
│ Releases · Analysis · LiveHub │ ┌─────┴┐ ┌───┴──┐ ┌─┴────┐ ┌───┴──┐
│ Store(段文件 + SQLite) │ │ HK-1 │ │ JP-1 │ │ SG-1 │ ... │ US-3 │ 10+ 台 VPS Agent(有公网 IP)
└──────────────────────────────────────────────────────────┘ └──▲───┘ └──▲───┘ └──▲───┘ └──▲───┘
▲ ▲ │ │ │ │ 每个 Agent 一个会话:
│ wss 长连接(节点发起) │ wss 长连接(节点发起) └────────┴───┬────┴─────────────┘ TLS 控制通道 + UDP/TCP 数据通道
│ 任务、数据、状态、更新通知 │ 身份、Agent 列表、状态、更新通知 ┌─────┴──────┐
┌────────────────────┐ ┌──────────────────────┐ │ Probe │ 家宽,NAT/CGNAT 后,唯一的测量起点
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │ └────────────┘ 不认识 Backend,只连 Agent
│ 有公网 IP,监听 │ │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
测量会话:TLS 控制通道 + UDP/TCP 数据通道,由 Probe 主动拨出
``` ```
- **线路(line)** = 一对(Probe,Agent)。可以有多个 Probe(身份由 Backend 生成,可以吊销、删除),每个 Probe 连接多个 Agent,每个 Agent 同时服务多个 Probe;一条线路的数据按 `line:<probe_id>` 单独存放,互不混合。对外的线路 id 是 `<agent_id>~<probe_id>`。 - **线路(line)** = 一对(Probe,Agent)。可以有多个 Probe(身份由 Backend 生成,可以吊销、删除),每个 Probe 连接多个 Agent,每个 Agent 同时服务多个 Probe;一条线路的数据按 `line:<probe_id>` 单独存放,互不混合。对外的线路 id 是 `<agent_id>~<probe_id>`。
@@ -45,17 +42,17 @@
| 角色 | 职责 | 明确不做 | | 角色 | 职责 | 明确不做 |
| --- | --- | --- | | --- | --- | --- |
| Probe | 连接 Backend(身份、Agent 列表、系统与构建信息、更新);连接面板指定的 Agent;执行线路任务中 Probe 一侧的角色;记录本端原始列并交给对应的 Agent | 统计;落盘测量数据;任何主机操作 | | Probe | 连接所有启用的 Agent;执行线路任务中 Probe 一侧的角色;记录本端原始列并交给对应的 Agent | 认识 Backend;统计;落盘测量数据;任何主机操作 |
| Agent | 注册;接受 Probe;执行 Agent 一侧的角色和目标任务;合并两侧的列并上传;上报主机资源的汇总数字 | 统计;落盘测量数据;exec/终端/文件/自更新(更新由宿主机上的更新器完成);采集主机的详细信息 | | Agent | 注册;接受 Probe;执行 Agent 一侧的角色和目标任务;合并两侧的列并上传 | 统计;落盘测量数据;exec/终端/文件/自更新;采集主机信息 |
| Backend | 注册与身份;任务编排;决定哪些 Probe 连哪些 Agent;持久化原始数据;分析;实时汇聚;API;分发 CI 签名的发布清单 | 主动连接 Probe 或 Agent(都是节点拨进来);验签和安装更新(在节点宿主机上) | | Backend | 注册与身份;任务编排;持久化原始数据;分析;实时汇聚;API | 连接 Probe |
## 4. 关键设计(详见各专题文档) ## 4. 关键设计(详见各专题文档)
| 主题 | 要点 | 文档 | | 主题 | 要点 | 文档 |
| --- | --- | --- | | --- | --- | --- |
| 节点框架 | 注册、身份、连接、RPC、可靠上报、状态同步、宿主、更新都由与业务无关的 **Fleet** 提供;Monitor 只写业务,其他产品也能复用 | [fleet/overview.md](fleet/overview.md) | | 节点框架 | 注册、身份、连接、RPC、可靠上报、状态同步、宿主、更新都由与业务无关的 **Fleet** 提供;Monitor 只写业务,其他产品也能复用 | [fleet/overview.md](fleet/overview.md) |
| 注册与身份 | 使用 `fleet-enroll` 和加入令牌自动注册,之后用签名认证;Probe 用自己的身份直连面板,Agent 列表由面板下发 | [enrollment.md](enrollment.md) | | 注册与身份 | 使用 `fleet-enroll` 和加入令牌自动注册,之后用签名认证;Probe 通过引导 Agent 同步 Agent 列表 | [enrollment.md](enrollment.md) |
| 更新 | 只接受发布密钥签名的清单,更新策略由节点本地决定,失败自动回滚。容器节点(Agent、Probe)已实现:CI 对镜像 digest 签名,宿主机上的 `monitor-update` 验签、按 digest 拉取、重启;纯二进制的 `fleet-updater` 还只是设计 | [deployment.md](deployment.md)、[fleet/update.md](fleet/update.md) | | 更新 | 独立的 `fleet-updater` 进程:只接受离线密钥签名的清单,更新策略由本地决定,失败自动回滚 | [fleet/update.md](fleet/update.md) |
| 任务模型 | `Task = {kind, target, params, schedule}`;持续采集、按需测试、目标任务统一建模;下发“期望任务集”,由节点自行调和 | [measurement.md](measurement.md) | | 任务模型 | `Task = {kind, target, params, schedule}`;持续采集、按需测试、目标任务统一建模;下发“期望任务集”,由节点自行调和 | [measurement.md](measurement.md) |
| 采集扩展 | Kind 由积木组合声明;采集密度靠参数调整;原始块自描述 | [measurement.md](measurement.md) | | 采集扩展 | Kind 由积木组合声明;采集密度靠参数调整;原始块自描述 | [measurement.md](measurement.md) |
| 统计扩展 | 指标注册制:少数几种累加器 + 统计算子;一行 `MetricDef` 就是一个新指标 | [analysis.md](analysis.md) | | 统计扩展 | 指标注册制:少数几种累加器 + 统计算子;一行 `MetricDef` 就是一个新指标 | [analysis.md](analysis.md) |
@@ -64,20 +61,6 @@
| 展示查询 | 任意时间段(相对或绝对);自动步长(每条曲线最多约 720 个点);1s 内存热层(1 小时)加上 10s/1m/5m/1h 各层;累加器现场合并,汇总精确;列式响应;SSE 实时跟随 | [query.md](query.md) | | 展示查询 | 任意时间段(相对或绝对);自动步长(每条曲线最多约 720 个点);1s 内存热层(1 小时)加上 10s/1m/5m/1h 各层;累加器现场合并,汇总精确;列式响应;SSE 实时跟随 | [query.md](query.md) |
| API | `/api/public`(只读,可设为公开)、`/api/admin`(登录)、`/api/agent`(注册与 RPC);实时数据用 SSE | [api.md](api.md) | | API | `/api/public`(只读,可设为公开)、`/api/admin`(登录)、`/api/agent`(注册与 RPC);实时数据用 SSE | [api.md](api.md) |
### 4.0 容器节点的更新
```text
CI ──签名的发布清单(镜像 digest)──▶ Backend ──原样转发(offer)──▶ Agent / Probe 进程
│ 只写进数据目录的 update/
▼
宿主机 monitor-update(root):用本机公钥验签 → 核对组件、仓库和 seq → podman pull 仓库@digest → 重启 → 不健康就回滚
│ 状态写进只读的 update-status/
▼
节点随心跳上报更新状态 ──▶ Backend ──▶ 管理端(有新版本时点「更新」)
```
细节见 [deployment.md](deployment.md)「容器节点的更新」。
### 4.1 回显的四个时间戳 ### 4.1 回显的四个时间戳
回显类 Kind 记录 `t1..t4`(Probe 发出、Agent 收到、Agent 回复、Probe 收到)。借此可以扣除 Agent 的处理耗时得到 RTT,可以分别判断上行和下行的丢包,并在去除时钟偏移和漂移后,得到**每个方向各自的抖动**,不需要两端时钟同步。 回显类 Kind 记录 `t1..t4`(Probe 发出、Agent 收到、Agent 回复、Probe 收到)。借此可以扣除 Agent 的处理耗时得到 RTT,可以分别判断上行和下行的丢包,并在去除时钟偏移和漂移后,得到**每个方向各自的抖动**,不需要两端时钟同步。
@@ -90,19 +73,17 @@ CI ──签名的发布清单(镜像 digest)──▶ Backend ──原样
```text ```text
Agent / Probe 能做的 执行编译在二进制中的 Kind;监听和访问本地配置允许的端口与网段;受本地硬上限约束 Agent / Probe 能做的 执行编译在二进制中的 Kind;监听和访问本地配置允许的端口与网段;受本地硬上限约束
不能做的 exec / shell / 终端 / 文件读写(配置目录除外)/ 自更新 / 加载外部代码 / 采集主机标识(型号、主机名、IP、进程) 不能做的 exec / shell / 终端 / 文件读写(配置目录除外)/ 远程更新 / 加载外部代码 / 采集主机标识(型号、主机名、IP、进程)
即使 Backend 被攻破 攻击者最多能在本地硬上限内发测量流量,拿不到主机权限,也无法把 Agent 当成扫描器或放大器; 即使 Backend 被攻破 攻击者最多能在本地硬上限内发测量流量,拿不到主机权限,也无法把 Agent 当成扫描器或放大器;
也推不出恶意更新(更新清单必须由发布密钥签名,节点宿主机上的更新器用本机公钥验签,并且只认本机配置的镜像仓库) 也推不出恶意更新(更新清单必须由离线密钥签名)
``` ```
- **本地硬上限**写在节点本机的配置文件中,Backend 下发的任务如果超出,节点直接拒绝:允许访问的目标网段(默认为空,即不开放目标任务)、总码率和包速率上限、允许监听的端口。 - **本地硬上限**写在节点本机的配置文件中,Backend 下发的任务如果超出,节点直接拒绝:允许访问的目标网段(默认为空,即不开放目标任务)、总码率和包速率上限、允许监听的端口。
- **构建期强制**:常驻节点进程使用 `BannedSymbols.Node.txt`,禁用 `System.Diagnostics.Process`、`Assembly.Load*`、`NativeLibrary.Load`、`HttpClient`,以及 `ConfigStore` 以外的文件写入 API,详见 [project-layout.md](project-layout.md) 第 7 节。 - **构建期强制**:常驻节点进程使用 `BannedSymbols.Node.txt`,禁用 `System.Diagnostics.Process`、`Assembly.Load*`、`NativeLibrary.Load`、`HttpClient`,以及 `ConfigStore` 以外的文件写入 API,详见 [project-layout.md](project-layout.md) 第 7 节。
- **权限分离**:注册(`fleet-enroll`,一次性运行)、采集(`monitor-agent` / `monitor-probe`)、更新是不同的进程。容器节点里,采集进程在容器内以 UID 1654 运行(只读文件系统、不带额外能力),更新由宿主机上 root 运行的 `monitor-update` 完成:它只读取节点放进 `update/` 的普通文件、复制到自己的临时目录后验签,状态写在节点只读的 `update-status/`;采集进程改写不了自己的二进制,更新器也读不到节点身份私钥。纯二进制安装的 `fleet-updater` 还只是设计。 - **三进程权限分离**:注册(`fleet-enroll`,一次性运行)、采集(`monitor-agent` / `monitor-probe`)、更新(`fleet-updater`)是三个独立进程,使用不同的系统用户。采集进程无法改写自己的二进制;更新进程读不到节点身份私钥。
- **容器运行**:Agent、Probe 的镜像基于 Alpine(musl,NativeAOT),宿主机用 rootful podman 加 systemd 运行:UID/GID 1654、只读文件系统、`--cap-drop all`、`no-new-privileges`、`/tmp` 用 tmpfs;配置和身份放在挂载的数据目录里。
- **最小权限运行**:非 root 用户;只在需要时授予 `CAP_NET_RAW`(ICMP);默认端口都高于 1024,不需要 `CAP_NET_BIND_SERVICE`,只有在 Agent 本地显式开启 443 时才授予。 - **最小权限运行**:非 root 用户;只在需要时授予 `CAP_NET_RAW`(ICMP);默认端口都高于 1024,不需要 `CAP_NET_BIND_SERVICE`,只有在 Agent 本地显式开启 443 时才授予。
- **身份**:节点使用本地生成的 ECDSA P-256 密钥对签名认证(Probe 的密钥由面板生成、只返回一次,同一把用来连面板和连 Agent),Backend 只存公钥,数据库泄露也无法冒充节点。 - **身份**:节点使用本地生成的 ECDSA P-256 密钥对签名认证,Backend 只存公钥,数据库泄露也无法冒充节点。
- **防放大**:未认证的包静默丢弃;回复长度与请求相等;每个流按声明速率限速;使用重放窗口。 - **防放大**:未认证的包静默丢弃;回复长度与请求相等;每个流按声明速率限速;使用重放窗口。
- **容器里的主机指标**:容器里的 Agent 通过只读挂载宿主机的 `/proc`、`/sys`、根目录和 `/etc/os-release`(环境变量 `HOST_PROC`、`HOST_SYS`、`HOST_ROOT`)读取宿主机的资源占用和系统名;读不到的项返回空值,不会退回去读容器自己的数据。Probe 只需要挂载 `/etc/os-release`。
- **自身状态**:节点上报进程内的计数(会话数、包速率、队列、丢弃数);Agent 另外每隔几秒用 `monitor.host` 上报一份主机资源占用的汇总数字(可以在 `agent.json` 里用 `reporting.host_stats = false` 整个关掉),最新值实时推给展示页,Backend 每分钟存一个样本(默认保留 7 天,`retention.host_days`)。 - **自身状态**:节点上报进程内的计数(会话数、包速率、队列、丢弃数);Agent 另外每隔几秒用 `monitor.host` 上报一份主机资源占用的汇总数字(可以在 `agent.json` 里用 `reporting.host_stats = false` 整个关掉),最新值实时推给展示页,Backend 每分钟存一个样本(默认保留 7 天,`retention.host_days`)。
## 6. 可扩展性目标 ## 6. 可扩展性目标
@@ -132,7 +113,7 @@ Kind 目录编译在二进制中,节点不接受远程下发代码(安全边
| D1 | 只关注网络层;节点是纯采集器,没有远程执行能力 | 由用户确认;防止权限泄露 | | D1 | 只关注网络层;节点是纯采集器,没有远程执行能力 | 由用户确认;防止权限泄露 |
| D2 | 拓扑为多个 Probe 连接多个 Agent(10+);线路 = (Probe, Agent),数据按 Probe 隔离 | 由用户确认 | | D2 | 拓扑为多个 Probe 连接多个 Agent(10+);线路 = (Probe, Agent),数据按 Probe 隔离 | 由用户确认 |
| D3 | Agent 用加入令牌自动注册,之后用密钥对签名认证 | 批量部署省事;Backend 不保存可用于冒充节点的秘密 | | D3 | Agent 用加入令牌自动注册,之后用密钥对签名认证 | 批量部署省事;Backend 不保存可用于冒充节点的秘密 |
| D4 | Probe 直连 Backend:身份、Agent 列表(期望状态 `probe_agents`)、状态和更新都走这条链路,测量仍然只走 Probe → Agent(2026-10 由「Probe 不认识 Backend、经 Agent 同步列表」改成直连,因为面板本来就要对外暴露,直连让更新和管理少一跳)。哪些 Probe 连哪些 Agent 由面板决定,`fleet.auto_link` 打开时两两连,也可以手动选择,Probe 始终主动拨出 | 新增 Agent 后自动覆盖,不需要改 Probe 配置;Probe 的版本、更新和线路都能在面板上管理 | | D4 | Probe 不认识 Backend,通过引导 Agent 同步 Agent 列表 | 新增 Agent 后自动覆盖,不需要改 Probe 配置 |
| D5 | 只采集逐包、逐事件的原始数据,解释全部放在 Backend | 模型和指标可以事后重算;采集端最轻 | | D5 | 只采集逐包、逐事件的原始数据,解释全部放在 Backend | 模型和指标可以事后重算;采集端最轻 |
| D6 | 统一的任务模型 + 期望任务集调和 | 新增任务类型时协议不变 | | D6 | 统一的任务模型 + 期望任务集调和 | 新增任务类型时协议不变 |
| D7 | Kind 由积木组合声明;原始块自描述 | 调整密度或新增采集方法只改一处 | | D7 | Kind 由积木组合声明;原始块自描述 | 调整密度或新增采集方法只改一处 |
@@ -140,21 +121,19 @@ Kind 目录编译在二进制中,节点不接受远程下发代码(安全边
| D9 | JSON-RPC 2.0 信封 + 二进制帧 | 新增能力只需要加方法 | | D9 | JSON-RPC 2.0 信封 + 二进制帧 | 新增能力只需要加方法 |
| D10 | 原始数据每 10 秒切一块,追加写入段文件,保留 30 天;指标长期保留 | 同一条链路同时服务实时和历史;避免海量小文件 | | D10 | 原始数据每 10 秒切一块,追加写入段文件,保留 30 天;指标长期保留 | 同一条链路同时服务实时和历史;避免海量小文件 |
| D11 | API 分为 public / admin / agent 三层;实时数据用 SSE | 前端只读场景足够;反向代理和 CDN 更友好 | | D11 | API 分为 public / admin / agent 三层;实时数据用 SSE | 前端只读场景足够;反向代理和 CDN 更友好 |
| D12 | 单仓库,三个独立项目加 Web(展示端、管理端);契约以 C# 代码为真源 | 改一处,编译期即可发现不一致 | | D12 | 单仓库,三个独立项目加待开发的 Web;契约以 C# 代码为真源 | 改一处,编译期即可发现不一致 |
| D13 | Probe/Agent 运行时只依赖 BCL;.NET 10 NativeAOT | 轻量 | | D13 | Probe/Agent 运行时只依赖 BCL;.NET 10 NativeAOT | 轻量 |
| D14 | 节点机制抽成与业务无关的 Fleet 框架,放在同一仓库的 `backend/fleet/` 目录,暂不独立(由用户确认;不设拆分时间表,需要时再评估;不使用 PublicApiAnalyzers) | 可复用;`samples/EchoNode` 作为第二个使用者,防止过度设计 | | D14 | 节点机制抽成与业务无关的 Fleet 框架,放在同一仓库的 `backend/fleet/` 目录,暂不独立(由用户确认;不设拆分时间表,需要时再评估;不使用 PublicApiAnalyzers) | 可复用;`samples/EchoNode` 作为第二个使用者,防止过度设计 |
| D15 | 注册独立为 `fleet-enroll` 工具;常驻进程不包含注册代码 | 常驻进程去掉 HttpClient,边界更纯 | | D15 | 注册独立为 `fleet-enroll` 工具;常驻进程不包含注册代码 | 常驻进程去掉 HttpClient,边界更纯 |
| D16 | 更新与业务进程分离:发布密钥签名、本地策略、健康确认与回滚。容器节点已实现(宿主机 `monitor-update`,清单里是镜像 digest);纯二进制的 `fleet-updater` 待做 | 面板被攻破时也无法推送恶意二进制;可以同时管理多个产品 | | D16 | 更新独立为 `fleet-updater`:离线签名、多发布源、本地策略、健康确认与回滚(默认方案,待确认) | 面板被攻破时也无法推送恶意二进制;可以同时管理多个产品 |
| D17 | Probe 支持 OpenWrt:正式支持 `linux-musl-x64`、`linux-musl-arm64`;armv7 视工具链验证结果决定;MIPS 不支持,改为在局域网设备上运行 Probe | 由用户确认;NativeAOT 不支持 MIPS | | D17 | Probe 支持 OpenWrt:正式支持 `linux-musl-x64`、`linux-musl-arm64`;armv7 视工具链验证结果决定;MIPS 不支持,改为在局域网设备上运行 Probe | 由用户确认;NativeAOT 不支持 MIPS |
| D18 | 默认测量端口不使用 443:控制 7443/tcp,UDP 数据 7443/udp,TCP 数据 7444/tcp;443 只在 Agent 本地显式开启时才允许使用 | 由用户确认;避免与 VPS 上的真实服务冲突;结果会记录实际端口 | | D18 | 默认测量端口不使用 443:控制 7443/tcp,UDP 数据 7443/udp,TCP 数据 7444/tcp;443 只在 Agent 本地显式开启时才允许使用 | 由用户确认;避免与 VPS 上的真实服务冲突;结果会记录实际端口 |
| D19 | 展示查询最细 1s:最近 1 小时来自内存热层,更早的数据来自 10s/1m/5m/1h 各层,现场合并;时间段内的汇总由累加器精确合并得出 | 由用户确认;5 分钟、15 分钟这类视图足够精细且快 | | D19 | 展示查询最细 1s:最近 1 小时来自内存热层,更早的数据来自 10s/1m/5m/1h 各层,现场合并;时间段内的汇总由累加器精确合并得出 | 由用户确认;5 分钟、15 分钟这类视图足够精细且快 |
| D20 | 时序响应采用列式格式(`t[]` + `values{}`,时间轴使用 Unix 毫秒) | 由用户确认;体积小,可以直接交给图表库 | | D20 | 时序响应采用列式格式(`t[]` + `values{}`,时间轴使用 Unix 毫秒) | 由用户确认;体积小,可以直接交给图表库 |
| D21 | 国家和运营商只由 Backend 向 IP 归属服务查询(缺省 `https://ip.007995.xyz`,可关闭),只查公网地址;节点不访问任何外部服务;公开接口不返回 IP、城市、ASN | 客户的防火墙常常要审计节点的出站连接;访客拿不到敏感信息(有测试保证) |
| D22 | 构建信息(版本、tag、提交、构建时间)由节点自报,访客是否可见由设置 `site.show_build` 控制(默认关);提交号只在管理端 | 展示页可以选择不暴露版本;管理员能看出哪个节点没升级 |
## 9. 仍待确认 ## 9. 仍待确认
待确认事项: 以下两项是按推荐方案写入的默认值(D16),等待确认:Fleet.Link 的通用范围(当前包含注册、认证、RPC、发件箱、状态同步、点对点会话);更新采用多发布源加离线签名。其余待确认事项:
1. **告警**:首版不包含,是否需要预留 Webhook? 1. **告警**:首版不包含,是否需要预留 Webhook?
2. **与真实服务共用 443**(例如 SNI 分流):首版不做,默认使用非 443 端口;以后如有需要再评估。 2. **与真实服务共用 443**(例如 SNI 分流):首版不做,默认使用非 443 端口;以后如有需要再评估。
+4 -115
View File
@@ -9,8 +9,6 @@ scripts/publish.sh <agent|probe|backend|fleet-enroll> <rid> → backend/art
scripts/package.sh <agent|probe> <rid> → backend/artifacts/packages/monitor-<role>-<version>-<rid>.tar.gz(附 .sha256) scripts/package.sh <agent|probe> <rid> → backend/artifacts/packages/monitor-<role>-<version>-<rid>.tar.gz(附 .sha256)
``` ```
构建信息(程序里显示的 tag、提交、构建时间):`publish.sh` 会用 `git describe --tags --always --dirty`、`git rev-parse` 和当前时间自动设置环境变量 `MONITOR_TAG`、`MONITOR_COMMIT`、`MONITOR_BUILT_AT`,直接用 `dotnet publish` 或 CI 构建时自己设置它们(容器镜像的 `*.NativeBuild.Containerfile` 用同名的 `--build-arg`,CI 脚本会传);不设置就是空,本地构建照常可重复。版本号取自 `backend/Directory.Build.props` 的 `Version`。前端的信息由 `web/build-info.mjs` 在构建时写入(版本取自 `web/package.json`)。
| rid | 目标 | 构建方式 | | rid | 目标 | 构建方式 |
| --- | --- | --- | | --- | --- | --- |
| `linux-x64`、`linux-arm64` | Debian/Ubuntu glibc 服务器 | Windows CI 主机通过 AotAnywhere 交叉编译 | | `linux-x64`、`linux-arm64` | Debian/Ubuntu glibc 服务器 | Windows CI 主机通过 AotAnywhere 交叉编译 |
@@ -24,7 +22,7 @@ macOS 的 Apple Silicon 产物带临时 ad-hoc 签名,Intel 产物未签名。
## Backend 容器 ## Backend 容器
Backend 使用 OCI 标准镜像,通过 `registry.oeo.one/monitor/monitor-backend:<version>` 分发,支持 `linux/amd64` 和 `linux/arm64`。容器使用 musl NativeAOT 单文件程序,基底为 `mcr.microsoft.com/dotnet/runtime-deps:10.0-alpine`,以非 root 用户运行,在 `8080/tcp` 提供 HTTP 服务;数据库和原始数据统一写入 `/data`。程序启用 InvariantGlobalization,无需 .NET 运行时或 ICU;时间查询使用 UTC。SQLite 从固定版本的官方源码编译并静态链接,不再随镜像携带 `libe_sqlite3.so`。将 `/data` 挂载为持久卷,并在反向代理处配置 HTTPS。 Backend 使用 OCI 标准镜像,通过 `registry.oeo.one/monitor/monitor-backend:<version>` 分发,支持 `linux/amd64` 和 `linux/arm64`。容器基于 .NET 10 ASP.NET noble-chiseled-extra 运行时镜像,以非 root 用户运行,在 `8080/tcp` 提供 HTTP 服务;数据库和原始数据统一写入 `/data`。`extra` 变体带 IANA 时区数据,供按时区查询使用。将 `/data` 挂载为持久卷,并在反向代理处配置 HTTPS。
在部署目录保存此仓库的 `compose.yaml`,先通过部署平台的环境变量或密钥管理设置 `MONITOR_ADMIN_PASSWORD`(可选设置 `MONITOR_ADMIN_USER`、`MONITOR_VERSION`),然后启动: 在部署目录保存此仓库的 `compose.yaml`,先通过部署平台的环境变量或密钥管理设置 `MONITOR_ADMIN_PASSWORD`(可选设置 `MONITOR_ADMIN_USER`、`MONITOR_VERSION`),然后启动:
@@ -34,35 +32,11 @@ docker compose up -d
Compose 将容器端口只绑定到宿主机 `127.0.0.1:8080`,适合由同机 HTTPS 反向代理转发;数据保存在 Docker 命名卷 `monitor-data`。首次启动成功创建管理员后,改密应在管理端完成。 Compose 将容器端口只绑定到宿主机 `127.0.0.1:8080`,适合由同机 HTTPS 反向代理转发;数据保存在 Docker 命名卷 `monitor-data`。首次启动成功创建管理员后,改密应在管理端完成。
Backend 的 NativeAOT 发布默认启用 `SqliteStaticLink`,支持 Linux glibc/musl、macOS、Windows 的 x64/arm64 RID;跨平台构建使用 `-p:UseAotAnywhere=true`。SQLite 源码版本、下载年份和 SHA-256 统一配置在根目录 `Directory.Packages.props`,构建会校验下载文件并按 RID 缓存原生对象。普通 JIT 开发构建仍使用 NuGet 原生库。发布 Backend 时设置 `-p:OpenApiGenerateDocuments=false`,API 文档在普通开发构建中生成。
健康检查端点为 `GET /healthz`。容器定义见 [`../packaging/container/Backend.Containerfile`](../packaging/container/Backend.Containerfile)。 健康检查端点为 `GET /healthz`。容器定义见 [`../packaging/container/Backend.Containerfile`](../packaging/container/Backend.Containerfile)。
## 节点二进制发布 ## 节点二进制发布
Woodpecker 在 `vMAJOR.MINOR.PATCH` tag 上,由 Windows runner 使用 AotAnywhere 并行构建 Agent/Probe 的 Linux glibc、Linux musl、macOS 和 Windows 产物。Agent 包还包含一次性注册工具 `fleet-enroll`。Linux 归档带有安装脚本和 systemd/OpenWrt 单元;Windows 产物为 `.zip`,Linux/macOS 产物为 `.tar.gz`。Linux musl 的 Agent 与 Probe 二进制还会打包为 OCI 镜像,基底为 `mcr.microsoft.com/dotnet/runtime-deps:10.0-alpine`,支持 `linux/amd64` 和 `linux/arm64`;所有平台仍提供对应归档。所有平台产物和 OCI 镜像发布成功后,CI 会创建对应的 Gitea Release,并附上压缩包和 SHA-256 校验文件。macOS 产物目前未签名/公证。安装说明见 [`../backend/agent/README.md`](../backend/agent/README.md) 与 [`../backend/probe/README.md`](../backend/probe/README.md)。 Woodpecker 在 `vMAJOR.MINOR.PATCH` tag 上,由 Windows runner 使用 AotAnywhere 并行构建 Agent/Probe 的 Linux glibc、Linux musl、macOS 和 Windows 产物。Agent 包还包含一次性注册工具 `fleet-enroll`。Linux 归档带有安装脚本和 systemd/OpenWrt 单元;Windows 产物为 `.zip`,Linux/macOS 产物为 `.tar.gz`。Linux glibc 的 Agent 与 Probe 二进制还会打包为 OCI 镜像,支持 `linux/amd64` 和 `linux/arm64`;musl、macOS、Windows 仍通过对应平台归档分发。所有平台产物和 OCI 镜像发布成功后,CI 会创建对应的 Gitea Release,并附上压缩包和 SHA-256 校验文件。macOS 产物目前未签名/公证。安装说明见 [`../backend/agent/README.md`](../backend/agent/README.md) 与 [`../backend/probe/README.md`](../backend/probe/README.md)。
## CI 工作流
### Pull request 与 dev 验证
Pull request 和 `dev` push 会运行 Backend tests;通过后,独立的 `web-build` workflow 才会构建前端并检查 CSS。提交信息后缀 `[SKIP TESTS]` 只跳过 Backend tests,前端构建仍会运行。Gitea PR 事件的 `CI_COMMIT_MESSAGE` 是 PR 标题,因此在 PR 中跳过测试时,把后缀写入 PR 标题;普通分支 push 则写在 commit message 中。
### main 开发镜像
`main` 上只有提交信息包含 `[CI]` 的 push 才构建镜像。合并 PR 时,请把 `[CI]` 放入 PR 标题或合并提交标题。Backend、Agent、Probe 的 Linux/amd64 镜像会并行构建,并推送到同一个 Zot Registry 下独立的 `monitor-dev/` 仓库:
| 镜像 | 滚动标签 | 固定提交标签 |
| --- | --- | --- |
| Backend | `registry.oeo.one/monitor-dev/monitor-backend:latest` | `registry.oeo.one/monitor-dev/monitor-backend:dev-<12位SHA>` |
| Agent | `registry.oeo.one/monitor-dev/monitor-agent:latest` | `registry.oeo.one/monitor-dev/monitor-agent:dev-<12位SHA>` |
| Probe | `registry.oeo.one/monitor-dev/monitor-probe:latest` | `registry.oeo.one/monitor-dev/monitor-probe:dev-<12位SHA>` |
Dev 镜像目前仅支持 amd64;Agent/Probe 在 `sdk:10.0-alpine-aot` 中构建 `linux-musl-x64` 产物,运行基底为 `runtime-deps:10.0-alpine`。它们不创建 Gitea Release,也不修改正式版 `monitor/` 仓库中的标签。普通 main push 不会构建镜像。
### 正式版发布
推送匹配项目版本的 `vMAJOR.MINOR.PATCH` tag 会触发完整跨平台构建、正式 OCI 镜像推送和 Gitea Release。正式镜像使用 `registry.oeo.one/monitor/` 路径;Agent、Probe 提供 amd64 与 arm64 多架构标签,稳定版本更新正式仓库自己的 `latest`。这与 `monitor-dev/` 的 dev `latest` 相互独立。
Agent 和 Probe 镜像分别发布为 `registry.oeo.one/monitor/monitor-agent:<version>` 与 `registry.oeo.one/monitor/monitor-probe:<version>`,包含 amd64、arm64 多架构索引;也可指定 `:<version>-amd64` 或 `:<version>-arm64`。稳定版本同时更新 `latest`。镜像以非 root 用户(UID/GID 1654)运行,默认读取 `/etc/monitor/agent.json` 或 `/etc/monitor/probe.json`。运行容器时挂载对应配置目录;Probe 还会在该目录写入 `agents.json`,因此目录需可由 1654 用户写入。Agent、Probe 需使用宿主网络以保持节点端口和测量流量行为,并按需添加 `NET_RAW` 能力以执行 ICMP 测量: Agent 和 Probe 镜像分别发布为 `registry.oeo.one/monitor/monitor-agent:<version>` 与 `registry.oeo.one/monitor/monitor-probe:<version>`,包含 amd64、arm64 多架构索引;也可指定 `:<version>-amd64` 或 `:<version>-arm64`。稳定版本同时更新 `latest`。镜像以非 root 用户(UID/GID 1654)运行,默认读取 `/etc/monitor/agent.json` 或 `/etc/monitor/probe.json`。运行容器时挂载对应配置目录;Probe 还会在该目录写入 `agents.json`,因此目录需可由 1654 用户写入。Agent、Probe 需使用宿主网络以保持节点端口和测量流量行为,并按需添加 `NET_RAW` 能力以执行 ICMP 测量:
@@ -78,11 +52,7 @@ docker run -d --name monitor-probe --restart unless-stopped --network host \
宿主机配置目录应允许 UID/GID 1654 读写;Probe 需要写权限来更新 `agents.json`。也可通过 Docker Compose 配置相同的网络、卷和能力。 宿主机配置目录应允许 UID/GID 1654 读写;Probe 需要写权限来更新 `agents.json`。也可通过 Docker Compose 配置相同的网络、卷和能力。
Agent 和 Probe 的默认发布配置:NativeAOT、完整裁剪、`OptimizationPreference=Size`、符号剥离、相同方法体折叠、InvariantGlobalization;关闭 Server/Concurrent GC、异常栈、EventSource、调试、Hot Reload、HTTP tracing、HTTP/3 和 .NET Metrics。JSON 全部使用源码生成上下文,禁用反射序列化回退;LINQ 使用体积优化实现。HTTP/3 裁剪不影响现有 HTTP/WebSocket/TLS 连接,也不影响程序自身上报的监控指标。 实测体积(`StackTraceSupport=false`、`IlcFoldIdenticalMethodBodies=true`、`OptimizationPreference=Size`):
`fleet-enroll` 使用同一套体积优化开关。Fleet 共享库已声明 AOT 兼容,由 Agent、Probe、Backend 或注册工具的最终发布一起裁剪;GC 和运行时功能开关配置在各可执行项目中。
历史实测体积(后续以各 RID 的 CI 产物为准):
| 程序 | linux-musl-x64 | linux-x64 | win-x64 | | 程序 | linux-musl-x64 | linux-x64 | win-x64 |
| --- | --- | --- | --- | | --- | --- | --- | --- |
@@ -163,7 +133,6 @@ OpenWrt 默认不带 OpenSSL:`opkg install libopenssl3`(较新的快照和 2
```text ```text
浏览器 / 本机前端 ──► https://<公网域名> ──隧道(Cloudflare Tunnel 之类)──► 家宽机器: Backend (:29181) 浏览器 / 本机前端 ──► https://<公网域名> ──隧道(Cloudflare Tunnel 之类)──► 家宽机器: Backend (:29181)
家宽机器: Probe ──WSS 到 https://<公网域名>──► Backend(身份、Agent 列表)
家宽机器: Probe ──TCP/UDP 7443、TCP 7444──► VPS: Agent 家宽机器: Probe ──TCP/UDP 7443、TCP 7444──► VPS: Agent
VPS: Agent ──WSS 到 https://<公网域名>──► Backend VPS: Agent ──WSS 到 https://<公网域名>──► Backend
``` ```
@@ -172,7 +141,7 @@ OpenWrt 默认不带 OpenSSL:`opkg install libopenssl3`(较新的快照和 2
- **Backend 通过隧道暴露**:隧道把一个公网域名转到家宽机器的 `29181`;Backend 监听 `http://0.0.0.0:29181`(只写一个监听地址,同时写 `0.0.0.0` 和 `[::]` 会报地址已被占用),家宽路由器的防火墙不放行这个端口,公网只能经隧道进来。Agent 用 `fleet-enroll --server https://<公网域名>` 注册,之后用 WSS 连回 Backend。 - **Backend 通过隧道暴露**:隧道把一个公网域名转到家宽机器的 `29181`;Backend 监听 `http://0.0.0.0:29181`(只写一个监听地址,同时写 `0.0.0.0` 和 `[::]` 会报地址已被占用),家宽路由器的防火墙不放行这个端口,公网只能经隧道进来。Agent 用 `fleet-enroll --server https://<公网域名>` 注册,之后用 WSS 连回 Backend。
- **经隧道注册之后,检查 Agent 的 `public_host`**:Backend 看到的来源地址是隧道的出口,不是 VPS 的地址,要改成 VPS 的真实地址,Probe 才连得上。 - **经隧道注册之后,检查 Agent 的 `public_host`**:Backend 看到的来源地址是隧道的出口,不是 VPS 的地址,要改成 VPS 的真实地址,Probe 才连得上。
- **Probe 直连面板**:`probe.json` 里的 `server` 就是面板地址(取自 `Monitor__PublicUrl`,没配就用创建它时的请求地址),Probe 用它连面板,Agent 列表由面板下发;先注册还是先创建 Probe 都可以。 - **先注册 Agent,再创建 Probe**:`probe.json` 的 `bootstrap` 里才有这个 Agent;Probe 启动后会通过 Agent 同步列表。
- **小 VPS 把 `limits.max_total_mbps` 调低**(试跑时用 100);Agent 常驻约 23 MB,VPS 上原有的服务不受影响,结束后删除 `/opt/…`、停掉进程即可,没有留下任何东西。 - **小 VPS 把 `limits.max_total_mbps` 调低**(试跑时用 100);Agent 常驻约 23 MB,VPS 上原有的服务不受影响,结束后删除 `/opt/…`、停掉进程即可,没有留下任何东西。
- 家宽的 IPv6 前缀会在路由器重新拨号(比如停电)后变化,SSH 地址要跟着换;面板走隧道的域名,不受影响。 - 家宽的 IPv6 前缀会在路由器重新拨号(比如停电)后变化,SSH 地址要跟着换;面板走隧道的域名,不受影响。
- 本地电脑没有直连 VPS 的路径时,用家宽机器当跳板(`SSH_ASKPASS` + `SSH_ASKPASS_REQUIRE=force`,密码放环境变量,不写进文件)。 - 本地电脑没有直连 VPS 的路径时,用家宽机器当跳板(`SSH_ASKPASS` + `SSH_ASKPASS_REQUIRE=force`,密码放环境变量,不写进文件)。
@@ -183,8 +152,6 @@ OpenWrt 默认不带 OpenSSL:`opkg install libopenssl3`(较新的快照和 2
- VPS:`/etc/systemd/system/monitor-agent.service`,`Restart=always`,加 `NoNewPrivileges`、`ProtectSystem=strict`、`ReadWritePaths=/opt/monitor`、`ProtectHome`、`PrivateTmp`。这是不走安装脚本的手动方式;正式安装用上面的 `packaging/systemd`。 - VPS:`/etc/systemd/system/monitor-agent.service`,`Restart=always`,加 `NoNewPrivileges`、`ProtectSystem=strict`、`ReadWritePaths=/opt/monitor`、`ProtectHome`、`PrivateTmp`。这是不走安装脚本的手动方式;正式安装用上面的 `packaging/systemd`。
- 日志进 journal:`journalctl -u monitor-agent`、`journalctl --user -u monitor-probe`。重启 Backend 用 `systemctl` 而不是手动再起一个进程,否则会端口冲突。 - 日志进 journal:`journalctl -u monitor-agent`、`journalctl --user -u monitor-probe`。重启 Backend 用 `systemctl` 而不是手动再起一个进程,否则会端口冲突。
**国家和运营商**:Backend 用 Probe、Agent 的来源地址向 IP 归属服务查一次,展示在国旗和运营商图标上(`Monitor__GeoLookupUrl`,缺省 `https://ip.007995.xyz`;留空就不查;这个服务只供公益使用,署名和使用条款由部署的人自己遵守)。Probe、Agent 自己不访问外部服务,所以客户的防火墙审计里不会多出出站连接;Backend 发出去的只有公网 IP(内网地址不发)。
**从别处看这个面板**:本机跑前端,在页面里把数据源指向这个域名,用 API 令牌访问,Backend 设 `Monitor__CorsOrigins` 放行前端的来源(见 [web/README.md](../web/README.md))。 **从别处看这个面板**:本机跑前端,在页面里把数据源指向这个域名,用 API 令牌访问,Backend 设 `Monitor__CorsOrigins` 放行前端的来源(见 [web/README.md](../web/README.md))。
实测(Probe 在广东移动家宽,Agent 在日本 IIJ): 实测(Probe 在广东移动家宽,Agent 在日本 IIJ):
@@ -200,81 +167,3 @@ OpenWrt 默认不带 OpenSSL:`opkg install libopenssl3`(较新的快照和 2
| Agent 主机状态 | CPU 1.5%,内存 274 / 493 MB,已运行约 20.5 天 | | Agent 主机状态 | CPU 1.5%,内存 274 / 493 MB,已运行约 20.5 天 |
这次试跑发现并修掉的问题:`hy2_ladder` 的阶梯按「块」而不是按「阶段」计算,真实的 Probe 把一个阶段切成几块,4 个阶段出了 7 步(模拟数据不切块,所以之前的测试没有发现);现在同一阶段的块先合并。 这次试跑发现并修掉的问题:`hy2_ladder` 的阶梯按「块」而不是按「阶段」计算,真实的 Probe 把一个阶段切成几块,4 个阶段出了 7 步(模拟数据不切块,所以之前的测试没有发现);现在同一阶段的块先合并。
## 容器节点的运行(podman + systemd)
Agent 和 Probe 推荐在宿主机上用 root 的 podman 加 systemd 运行,这样宿主机更新器(下一节)才能重启它们。以 Agent 为例(Probe 把名字和目录换成 `monitor-probe`,配置文件换成 `probe.json`):
```ini
# /etc/systemd/system/monitor-agent-container.service
[Unit]
Description=Monitor Agent (podman)
After=network-online.target
Wants=network-online.target
[Service]
EnvironmentFile=/etc/monitor-update/agent.image # IMAGE=<仓库>@sha256:<digest>,更新器更新时改写它
Restart=always
RestartSec=5
TimeoutStopSec=30
ExecStartPre=-/usr/bin/podman rm -f monitor-agent
ExecStart=/usr/bin/podman run --rm --name monitor-agent --network host --read-only --security-opt no-new-privileges --security-opt label=disable --cap-drop all --tmpfs /tmp:rw,noexec,nosuid,size=64m,mode=1777 -v /etc/os-release:/etc/os-release:ro -v /proc:/host/proc:ro -v /sys:/host/sys:ro -v /:/host/root:ro -e HOST_PROC=/host/proc -e HOST_SYS=/host/sys -e HOST_ROOT=/host/root -v /opt/monitor/etc:/etc/monitor ${IMAGE}
ExecStop=/usr/bin/podman stop -t 20 monitor-agent
[Install]
WantedBy=multi-user.target
```
- **数据目录**(上面挂到容器 `/etc/monitor` 的 `/opt/monitor/etc`)归容器用户(UID/GID 1654),里面放 `agent.json`、身份目录(`fleet-enroll` 注册之后的 `agent-id/`),Probe 放 `probe.json` 和 `agents.json`。`agent.json` 里的 `fleet.identity_dir` 写容器里的路径(例如 `/etc/monitor/agent-id`)。
- **首次注册 Agent**:`sudo podman run --rm --network host --security-opt label=disable -v <数据目录>:/etc/monitor --entrypoint /usr/local/bin/fleet-enroll <镜像> --server https://<面板> --join <加入令牌> --name <名字> --identity-dir /etc/monitor/agent-id`,再写 `agent.json`。Probe:在管理端创建(或轮换密钥)之后下载 `probe.json`,放进数据目录。
- **拉取私有镜像**需要 Registry 的登录信息:用 `podman login --authfile /etc/monitor-update/registry-auth.json …` 生成一份只有 root 能读(600)的认证文件,更新器用它拉取,容器本身用不到。
- **挂载宿主机的 `/etc/os-release`(只读)**:节点读系统名展示在面板上,不挂的话读到的是镜像里的 Alpine。
- **ICMP 目标任务**要在 `podman run` 里加 `--cap-add NET_RAW`。
- **经隧道注册之后检查 Agent 的 `public_host`**:面板看到的来源地址是隧道的出口(通常是内网地址),要在管理端改成这台机器自己的公网地址(`PATCH /api/admin/agents/{id}`),Probe 才连得上。
- `packaging/compose/` 里的 compose 文件可用 `sudo podman compose up -d` 或 `docker compose up -d`;Podman Compose 需要安装 Compose provider。更新靠手动改版本再 `sudo podman compose pull && sudo podman compose up -d`,面板的更新按钮对它不起作用。
### Podman 的宿主指标与权限
这里使用 Linux 上的 **rootful Podman**(`sudo podman`),容器内仍以 UID/GID 1654 运行。配置目录需由 1654 用户读写;`label=disable` 不会改变 Unix 文件权限。Rootless Podman 的 UID 映射不同,不能直接沿用 `chown 1654:1654` 的准备步骤。
Agent 的 Compose 模板已经包含以下只读挂载和环境变量,systemd 示例也采用相同配置:
| 宿主来源 | 容器路径 | 配置 | 指标 |
| --- | --- | --- | --- |
| `/proc` | `/host/proc` | `HOST_PROC=/host/proc` | CPU、内存、累计网络流量、瞬时收发速率、运行时长 |
| `/sys` | `/host/sys` | `HOST_SYS=/host/sys` | 网卡驱动报告的链路速率 |
| `/` | `/host/root` | `HOST_ROOT=/host/root` | 宿主根文件系统容量和已用空间 |
| `/etc/os-release` | `/etc/os-release` | 无 | 宿主发行版 |
系统目录不加 `:z` 或 `:Z`,容器统一设置 `--security-opt label=disable`。读取失败的指标返回空值;配置了 `HOST_ROOT` 但没有挂载时不会改读容器 overlay 根分区。未设置这些环境变量时,原生进程继续读取本机默认路径。
磁盘指标是根文件系统容量,不是所有硬盘合计,也不采集磁盘 I/O。链路速率是参与流量统计的接口速率合计,单位 Mbit/s,不是互联网测速结果;任一接口不支持读取速率时显示空值。接口仍按名称排除回环、容器、网桥和部分隧道,复杂 bonding/VLAN 配置可能存在重复计数。主机名、硬件型号、MAC/IP 不作为资源指标上报。
Probe 保持现有功能,只上报系统、架构与节点状态,不上报 CPU、内存、磁盘或网卡指标。因此 Probe 的 Compose 模板只需要配置目录和 `/etc/os-release` 挂载;从 Agent systemd 示例改为 Probe 时,去掉 `/proc`、`/sys`、`/` 挂载及 `HOST_*` 环境变量。
可直接安装的 unit 模板见 `packaging/systemd/monitor-agent-container.service` 和 `packaging/systemd/monitor-probe-container.service`;它们与上述示例使用相同策略,分别读取 `/etc/monitor-update/agent.image` 与 `probe.image`。在 Linux 虚拟机上运行容器时,采集对象是该 Linux 虚拟机。
## 容器节点的更新(Agent、Probe)
面板通知、节点验签、宿主机更新,**面板即使被攻破也装不上假镜像**:
```text
CI 构建并推镜像(Zot)──► ci/release-manifest.sh:对镜像 digest 的清单签名(ECDSA P-256,私钥只在 Woodpecker 密钥 release_signing_key 里)
└─ POST /api/admin/releases ──► 面板:只存下并原样转发(不验签)──► 在线的 Agent、Probe(连上时补发)
节点进程:只把清单和签名写进数据目录的 update/,不执行任何命令
│ systemd.path 监视到 offer.sig
▼
宿主机 monitor-update(root):复制到临时目录 → 用本机的发布公钥验签 → 核对组件和镜像仓库 → seq 必须比已安装的大
→ status:available / invalid(写进 update-status/,节点随心跳报给面板,管理端「版本」页显示)
→ 管理员点「更新」(或本机配置 POLICY=auto)→ podman pull <仓库>@sha256:<digest>(内容寻址,Registry 或链路上换不了)
→ 写 IMAGE=… 到 /etc/monitor-update/<组件>.image,重启容器单元;20 秒内没有健康就换回旧镜像(rolled_back)
```
- **安装更新器**(每个容器节点一次,root):`packaging/update/install-updater.sh agent --data-dir /opt/monitor/etc --image-repo registry.oeo.one/monitor/monitor-agent --unit monitor-agent-container.service [--policy auto] [--authfile <Registry 认证文件>]`。它装好 `monitor-update` 脚本和发布公钥,写 `/etc/monitor-update/<组件>.conf`,启用两个 systemd 路径单元。容器单元要用 `EnvironmentFile=/etc/monitor-update/<组件>.image` 和 `${IMAGE}` 启动容器。
- **镜像仓库、服务名、公钥都在本机配置里**,清单里写什么都不会改变拉取哪个仓库;清单被篡改、签名对不上、组件或仓库不符、`seq` 不比已安装的大(重放旧清单),都会被拒绝,状态是 `invalid`。
- **业务进程不能改二进制,也不执行命令**:它只能往 `update/` 里放清单和请求;状态在 root 拥有的 `update-status/` 里(只读)。
- **CI 的配置**:Woodpecker 密钥 `release_signing_key`(发布私钥,PEM)、`release_api_token`(面板的 API 令牌,作用域 `release`,只能登记清单);环境变量 `RELEASE_API_URL`(面板地址)。没有 `release_signing_key` 就只构建镜像、不发清单。
- **管理端**:「系统 → 版本」页显示最新发布的 tag、每个节点的更新状态(已是最新、有新版本、更新中、已回滚、清单无效),有新版本的节点有「更新」按钮,也可以「全部更新」。更新策略在节点本机配置:`/etc/monitor-update/<组件>.conf` 里 `POLICY=manual`(默认,等管理员请求)或 `auto`(验签通过就更新)。
- **发布公钥**在 `packaging/update/release-key.pub.pem`;私钥不入库,请离线备份。更换密钥:节点上换新公钥之后再用新私钥发布。
- 纯二进制的安装方式(`fleet-updater`、`fleet-release`,见 [fleet/update.md](fleet/update.md))还没有实现;容器方式优先。
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff. Show more