67 Commits
Author SHA1 Message Date
justme ad17170664 docs: redraw the architecture diagrams and update the README [SKIP TESTS]
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-07 01:30:28 +08:00
justme 710aa9e99f ci: sync main into dev [CI SKIP] 2026-10-07 01:28:32 +08:00
justme 71d2866b9c docs: update the architecture [SKIP TESTS] 2026-10-07 01:21:49 +08:00
geruipu 103ef27ed3 feat: use Alpine AOT images, static SQLite and Podman host metrics [CI] 2026-10-07 01:03:31 +08:00
justme 48664c0ada docs: remove the roadmap and bring the docs in line with the current code [SKIP TESTS]
Probes link to the panel directly, container nodes update through signed
releases, the frontend exists and the ports changed. The roadmap is gone; its
open items live in the README.
2026-10-06 23:27:26 +08:00
justme 198ee556d6 ci: sync main into dev [CI SKIP] 2026-10-06 23:22:52 +08:00
justme 6e945657c1 feat: choose probe/agent links from the admin, fix the overview popover [CI]
The panel decides which probes link to which agents (probes dial out): an
auto-link setting plus per-pair manual overrides with a link matrix in the
admin, and the hovered overview card is lifted above its neighbours.
2026-10-06 23:22:45 +08:00
justme da465866d5 fix(web): lift the hovered overview card above its neighbours so the popover is not hidden 2026-10-06 23:22:36 +08:00
justme 3a751a0f89 feat: choose which probes link to which agents from the admin
The panel decides the probe/agent pairs and probes always dial out. A new
fleet.auto_link setting connects every probe to every agent (default); with it
off only the pairs chosen by hand connect, and any pair can be forced on or
off. The Probes page has a link matrix with the switch and per-pair toggles.
2026-10-06 23:19:32 +08:00
justme d69581dcb5 docs: mount the host os-release and fix the agent public host after tunnel enrollment [SKIP TESTS] 2026-10-06 23:11:26 +08:00
justme 8f44e924a4 docs: describe container node deployment and the update flow [SKIP TESTS] 2026-10-06 23:02:16 +08:00
justme 0c39485a78 ci: sync main into dev [CI SKIP] 2026-10-06 22:57:11 +08:00
justme d449e50791 feat: update controls in the admin, release token scope, build info in CI images [CI]
The versions page shows each node's update state with update buttons, API
tokens get a release scope that can only register releases (used by CI), and
the CI image and release builds now pass the tag, commit and build time.
2026-10-06 22:57:04 +08:00
justme 7a71bf57f9 feat: update controls in the admin, release token scope, build info in CI images
The versions page shows each node's update state with update buttons, API
tokens get a release scope that can only register releases (used by CI), and
the CI image and release builds now pass the tag, commit and build time.
2026-10-06 22:56:56 +08:00
justme 22a98f2eb0 ci: sync main into dev [CI SKIP] 2026-10-06 22:42:20 +08:00
justme 8dd3209a65 feat!: probes link to the panel; signed release updates for container nodes [CI]
Probes connect to the panel directly and get the agent list as desired state.
CI signs the image digest of agent and probe releases, the panel forwards the
manifest, and a host updater verifies it with a local public key, pulls by
digest and rolls back when the new version is unhealthy.
2026-10-06 22:42:12 +08:00
justme 0a66f5e07a feat: signed releases and a host-side updater for container nodes
CI signs the image digest of each agent and probe release; the panel only
stores and forwards it. Nodes write the manifest to their data directory and
a host updater verifies the signature with a local public key, pulls by
digest, restarts the unit and rolls back when it is unhealthy. The panel can
list releases, ask a node to update and show the reported update state.
2026-10-06 22:37:26 +08:00
justme 48072737c9 feat!: probes connect to the backend directly
A probe now links to the backend like an agent (probe.json carries the
server address). The backend sends the agent list as desired state and the
probe reports its platform and build in its own hello. The agent no longer
relays the list or the probe platform. Existing probes must be re-created.
2026-10-06 22:25:16 +08:00
justme c645851c0e fix(ci): build native node images and add compose files [CI]
Restore with PublishAot so agent and probe images contain NativeAOT
binaries instead of a launcher that needs a .dll, fail the build when the
binary is too small, add agent and probe compose files, and document the
[CI] marker in AGENTS.md.
2026-10-06 21:43:57 +08:00
justme 604cbe3123 fix(ci): restore with PublishAot so node images are native, add compose files
Agent and probe images were built with a restore step that lacked
PublishAot, so the publish silently produced a trimmed launcher that looks
for a .dll. Restore now passes PublishAot and the build fails if the binary
is smaller than 2 MB. Also add agent and probe compose files and document
the [CI] marker in AGENTS.md.
2026-10-06 21:43:46 +08:00
justme e06462f033 ci: build main images [CI] 2026-10-06 21:25:53 +08:00
justme b242701e9e ci: sync main into dev [CI SKIP] 2026-10-06 21:24:47 +08:00
justme 14b0de3544 feat: node info, build info and acrylic web theme
Squash of recent dev work: return_session login, backend switcher, line
detail timeline, overview strips, node platform/country/ISP info with flags
and icons, build info (version, tag, time) behind site.show_build, and the
acrylic display theme.
2026-10-06 21:24:25 +08:00
justme 83f54978d2 feat(web): acrylic theme, node flags and icons, build info
Acrylic cards and topbar over a blurred background, country flags, ISP and
OS icons, per-node version tags, a build info footer and an admin versions
tab. Overview cards drop the sparkline and widen the hover area.
2026-10-06 21:22:04 +08:00
justme 1e5c660d05 feat(backend): report node platform, geo and build info
Agents and probes report OS, arch and build (version, tag, commit, time).
The backend looks up country and ISP of public addresses only, stores them
in node_meta (schema v5) and exposes display info via /api/public/nodes,
/api/public/about and /api/admin/build-info behind site.show_build.
2026-10-06 21:22:04 +08:00
justme 10634bdd80 docs: add project agent instructions [SKIP TESTS]
Merge PR #14.
2026-10-06 09:14:54 +00:00
justme f8175f5734 docs: add project agent instructions [SKIP TESTS] 2026-10-06 17:13:08 +08:00
justme 2e470607fa docs: add project agent instructions [SKIP TESTS] 2026-10-06 17:11:07 +08:00
justme a70b89b95c feat(web): show UDP and TCP strips on overview cards
Add 3/6/12 hour ranges with status strips, latency sparklines and a side popover for the hovered interval.
2026-10-06 16:59:17 +08:00
justme a9538eae4a feat(web): add backend switcher and connection status
Split the login page into backend list and credentials, show connection status in the sidebar and top bar, and render dialogs in a portal.
2026-10-06 16:59:17 +08:00
justme 91c63d0cf2 feat(api): add return_session login for cross-origin clients
Return the session token in the login response instead of a cookie, and accept it as a Bearer token without CSRF.
2026-10-06 16:59:16 +08:00
justme a2a0eee46a docs: explain development and CI workflow [SKIP TESTS]
Merge PR #13.
2026-10-06 08:46:12 +00:00
justme 929882cb04 docs: explain development and CI workflow [SKIP TESTS] 2026-10-06 16:44:27 +08:00
justme 2def8e69f9 ci: publish amd64 dev images to Zot [CI]
Merge PR #12. Publish development images separately from release images.
2026-10-06 08:28:53 +00:00
justme efa17b4485 ci: use SKIP TESTS marker for backend tests [SKIP TESTS] 2026-10-06 16:26:57 +08:00
justme 56d80e8c80 ci: build web after tests and allow skipping tests [CI SKIP] 2026-10-06 16:23:13 +08:00
justme 1da8d083c1 ci: skip backend tests only with CI SKIP suffix [CI SKIP] 2026-10-06 16:21:04 +08:00
justme e67d3294a2 ci: allow skipping verify with CI SKIP suffix [CI SKIP] 2026-10-06 16:18:59 +08:00
justme 383afa777b ci: publish main amd64 images to dev Zot [CI SKIP] 2026-10-06 16:14:50 +08:00
justme da0a0e3443 ci: parallelize opt-in main amd64 builds [CI]
Merge PR #11. Build Backend, Agent, and Probe concurrently on main.
2026-10-06 08:04:13 +00:00
justme 19c6cab104 ci: run main amd64 image builds in parallel [CI SKIP] 2026-10-06 15:59:54 +08:00
justme dd6ac4258d ci: opt in to main amd64 builds [CI]
Merge PR #10. Main Linux amd64 images are built only when explicitly marked.
2026-10-06 07:53:57 +00:00
justme 193fc3a95c feat: improve agent management and speed up tests 2026-10-06 15:48:46 +08:00
justme 46499b10b6 ci: build amd64 on opt-in main commits [CI SKIP] 2026-10-06 15:40:35 +08:00
justme 0f19d5047d ci: make main fastlane manual only [CI SKIP] 2026-10-06 15:18:12 +08:00
justme 134c06d605 merge: bring line detail timeline into dev [CI SKIP] 2026-10-06 14:52:48 +08:00
justme d1be70cda2 feat(web): 线路详情页的时间轴、EWMA 平滑和加载进度
- 先选日期,再在连续 48 小时(或 24 小时)的丢包缩略图上拖选区;选区可以跨零点,
  点轴或刻度时离得近的端点贴过去,悬停预览会动的端点;没有数据的时段画成斜纹
- 刻度下方的时长快捷档:30 分钟、1 小时、3 小时、6 小时、12 小时、整天
- 指标卡片平铺,丢包等标明 UDP/TCP;图表按 UDP、TCP、评分分组
- 图表数据刷新时原地更新(不再整张重建),毫秒到 1 秒以上显示成秒
- EWMA 平滑开关(关/轻/中/重),原始数据以淡线垫底
- 数据加载时卡片上显示进度条和占位;useQuery 区分定时刷新和用户触发的加载
- 前端可以在页面里指定后端地址和令牌(配合后端 Monitor__CorsOrigins)
2026-10-06 14:48:58 +08:00
justme 5cb64d2436 docs: clarify no-op main fastlane for docs
Describe the successful no-op when only documentation changes.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 18:58:35 +00:00
justme 9dcb95ccd0 fix(ci): trigger fastlane on merged main PRs
Handle merged Gitea PRs as well as direct pushes to main.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 18:42:39 +00:00
justme 4bde12b493 fix(ci): trigger fastlane on merged main PRs
Run the Linux amd64 fast-lane after a PR is merged to main, while leaving unmerged PRs as no-ops.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 18:35:24 +00:00
justme c4f1fd9e89 docs: clarify main fastlane merge trigger
Explain that feature PRs update main images after merging.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 18:08:49 +00:00
justme ee24593b2a fix(ci): emit native apphosts in fastlane images
Force native apphost generation and validate fastlane output files.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 17:43:22 +00:00
justme 672f6e3dcb fix(ci): emit native apphosts in fastlane images [CI SKIP] 2026-10-05 10:42:34 -07:00
justme 7cc3afbd9b fix(build): use native NativeAOT SDK in Linux fast lane
Only use the StuDev AotAnywhere SDK when explicitly enabled by Windows cross-platform builds.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 17:37:50 +00:00
justme 9a375ffd9e fix(build): select AotAnywhere explicitly [CI SKIP] 2026-10-05 10:37:10 -07:00
justme 75d6bd5540 fix(ci): use host networking for native builds
Run NativeAOT Buildah steps with host networking on the Linux agent.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 17:31:24 +00:00
justme 407c09f6a5 fix(ci): use host networking for native builds [CI SKIP] 2026-10-05 10:30:35 -07:00
justme d6b919bc42 fix(ci): handle empty changed-file lists
Use a Git diff fallback when Woodpecker provides no changed-file list.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 17:27:35 +00:00
justme fffee8937c ci: sync main into dev [CI SKIP] 2026-10-05 10:26:53 -07:00
justme 91485db500 fix(ci): fall back to git diff for empty path lists [CI SKIP] 2026-10-05 10:25:59 -07:00
justme dcf096431d ci: add dependency-aware main fastlane
Add Linux amd64 fast-lane OCI builds selected by project dependencies.

Co-authored-by: JustMe <mr.toing@outlook.com>
2026-10-05 17:22:26 +00:00
justme fa34869f18 ci: sync main into dev [CI SKIP] 2026-10-05 10:21:22 -07:00
justme deb5a97802 ci: add dependency-aware main fastlane [CI SKIP] 2026-10-05 10:18:22 -07:00
justme 479f8c25b0 feat(ci): publish agent and probe OCI images [CI SKIP] 2026-10-05 09:33:22 -07:00
justme 2b80bb66d1 ci: use local Gitea endpoint for release uploads 2026-10-05 08:44:19 -07:00
justme 078eacf1cb ci: persist Buildah cache for Linux releases 2026-10-05 08:29:08 -07:00
justme deceeb19f3 feat: add cross-platform release pipeline [CI SKIP] 2026-10-04 14:01:26 -07:00
170 changed files with 9268 additions and 1110 deletions

No files matched your search

+17 -1
View File
@@ -17,11 +17,27 @@ depends_on:
- windows-release - windows-release
steps: steps:
- name: publish-agent-probe-oci
image: bash
environment:
GITEA_BASE: http://127.0.0.1:3100
GITEA_TOKEN:
from_secret: gitea_token
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
commands:
- bash ci/release-node-images.sh
depends_on: []
- name: publish-gitea-release - name: publish-gitea-release
image: bash image: bash
environment: environment:
GITEA_BASE: http://192.168.2.162:3100 GITEA_BASE: http://127.0.0.1:3100
GITEA_TOKEN: GITEA_TOKEN:
from_secret: gitea_token from_secret: gitea_token
commands: commands:
- bash ci/publish-gitea-release.sh - bash ci/publish-gitea-release.sh
depends_on:
- publish-agent-probe-oci
+63
View File
@@ -0,0 +1,63 @@
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: plugin-git
settings:
lfs: false
when:
- event: push
branch: main
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
concurrency: 1
steps:
- name: build-linux-amd64-backend
image: bash
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
commands:
- bash ci/build-main-amd64.sh backend
depends_on: []
- name: build-linux-amd64-agent
image: bash
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
RELEASE_API_URL: https://mo.oeo.one
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
RELEASE_API_TOKEN:
from_secret: release_api_token
commands:
- bash ci/build-main-amd64.sh agent
depends_on: []
- name: build-linux-amd64-probe
image: bash
environment:
NUGET_PACKAGES: /home/justme/.nuget/packages
ZOT_REGISTRY: registry.oeo.one
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
RELEASE_API_URL: https://mo.oeo.one
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
RELEASE_API_TOKEN:
from_secret: release_api_token
commands:
- bash ci/build-main-amd64.sh probe
depends_on: []
+7 -11
View File
@@ -9,22 +9,18 @@ clone:
lfs: false lfs: false
when: when:
- event: [push, pull_request] - event: pull_request
- event: push
branch: dev
# On Gitea PR events, CI_COMMIT_MESSAGE is the PR title.
steps: steps:
- name: backend-tests - name: backend-tests
image: bash image: bash
# [SKIP TESTS] omits Backend tests only; web-build remains a dependent workflow.
when:
- evaluate: 'not (CI_COMMIT_MESSAGE contains "[SKIP TESTS]")'
environment: environment:
NUGET_PACKAGES: /home/justme/.nuget/packages NUGET_PACKAGES: /home/justme/.nuget/packages
commands: commands:
- cd backend && dotnet test --solution Monitor.slnx -c Release - cd backend && dotnet test --solution Monitor.slnx -c Release
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- npm run build --prefix web
- npm run lint:css --prefix web
depends_on: []
+27
View File
@@ -0,0 +1,27 @@
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: plugin-git
settings:
lfs: false
when:
- event: pull_request
- event: push
branch: dev
depends_on:
- verify
steps:
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- npm run build --prefix web
- npm run lint:css --prefix web
+43
View File
@@ -0,0 +1,43 @@
# Agent Instructions
## Project
Monitor is a self-hosted network path quality monitor. Backend coordinates Agents and Probes, stores measurements, and serves the REST/SSE API. `web/display` is the read-only overview; `web/admin` manages nodes, tests, and settings.
## Branch and workspace safety
- Treat `main` as the stable release branch and `dev` as the integration branch. Keep ordinary implementation work on `dev` or a feature branch; integrate into `main` through a pull request.
- A push to `main` builds images only if its commit message contains `[CI]` (for a merged PR, put it in the PR or merge commit title). Without `[CI]` nothing is built; use `[CI SKIP]` to skip the pipeline entirely. When squashing or pushing to `main`, add `[CI]` unless a build is explicitly not wanted.
- Before editing, inspect `git status`. Preserve pre-existing user changes: do not reset, clean, stash, or include unrelated files in a commit.
- Do not commit credentials, API keys, registry passwords, or generated runtime data. CI credentials belong in Woodpecker secrets (`zot_secret`, `gitea_token`, `release_signing_key` for signing release manifests, `release_api_token` with the `release` scope). The release signing private key never goes into the repository or the panel; only the public key in `packaging/update/` does.
- Keep development OCI images in the `monitor-dev/` repository path and release images in `monitor/`. Do not let a development build overwrite release tags such as `latest`.
## Project layout
- `backend/src/Monitor.Backend/`: service entry point and composition.
- `backend/src/Monitor.Backend.Api/`: REST endpoints, contracts, and OpenAPI.
- `backend/agent/`, `backend/probe/`: node applications and their READMEs.
- `backend/fleet/`, `backend/shared/`: reusable node framework, measurement engine, protocols, and data formats.
- `web/display/`, `web/admin/`: Vite applications; `web/shared/`: API client, generated types, charts, and shared UI.
- `packaging/`, `scripts/`, `.woodpecker/`, `docs/`: container/package builds, CI workflows, and project documentation.
## Local development and builds
Requirements: .NET SDK 10 and Node.js/npm. The SDK selection is defined by the root `global.json`; NuGet package versions are centralized in the root `Directory.Packages.props`.
- All .NET projects: `dotnet build Monitor.slnx -c Release` from the repository root.
- Backend build: `cd backend && dotnet build Monitor.slnx -c Release`
- Backend tests: `cd backend && dotnet test --solution Monitor.slnx -c Release`
- Frontend dependencies/build: `npm ci --prefix web && npm run build --prefix web`
- Frontend style check: `npm run lint:css --prefix web`
- Local backend: from `backend/`, run `dotnet run --project src/Monitor.Backend -- --urls http://localhost:5000`.
- Local web apps: from the repository root, run `npm ci --prefix web` then `npm run dev --prefix web`. The current dev config uses display port 5173 and admin port 5175; `/admin/` is proxied through the display server. Check the Vite configs when working on another branch because ports can differ.
- After changing API contracts, regenerate/check `backend/openapi/v1.json` and run `npm run gen:api --prefix web` so frontend API types stay aligned.
Follow the user's requested validation scope. Do not start a full tag/release build unless requested.
## CI and releases
CI triggers and image tags can change; inspect `.woodpecker/*.yml` and `docs/deployment.md` before changing or describing them. In particular, check the distinction between `[CI]` main builds, `[SKIP TESTS]`, and version-tag releases in the current branch.
A version tag `vMAJOR.MINOR.PATCH` runs the full cross-platform release pipeline. Development image builds and releases must remain isolated in their respective Zot repository paths.
@@ -1,10 +1,15 @@
<Project> <Project>
<PropertyGroup> <PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally> <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<!-- Native SQLite source is pinned separately from the JIT/test binary package. -->
<MonitorSqliteSourceVersion>3530400</MonitorSqliteSourceVersion>
<MonitorSqliteSourceYear>2026</MonitorSqliteSourceYear>
<MonitorSqliteSourceSha256>1e71ddf93849c6a6ecf58b827c0692073d2dd7ee40196158068f7b29f422e87d</MonitorSqliteSourceSha256>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageVersion Include="Microsoft.CodeAnalysis.BannedApiAnalyzers" Version="5.6.0" /> <PackageVersion Include="Microsoft.CodeAnalysis.BannedApiAnalyzers" Version="5.6.0" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="10.0.12" /> <PackageVersion Include="Microsoft.Data.Sqlite" Version="10.0.12" />
<PackageVersion Include="SQLitePCLRaw.lib.e_sqlite3" Version="3.53.3" />
<PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.12" /> <PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.12" />
<PackageVersion Include="Microsoft.Extensions.ApiDescription.Server" Version="10.0.12" /> <PackageVersion Include="Microsoft.Extensions.ApiDescription.Server" Version="10.0.12" />
<PackageVersion Include="Microsoft.Extensions.TimeProvider.Testing" Version="10.0.0" /> <PackageVersion Include="Microsoft.Extensions.TimeProvider.Testing" Version="10.0.0" />
+52
View File
@@ -0,0 +1,52 @@
<Solution>
<Folder Name="/backend/fleet/src/">
<Project Path="backend/fleet/src/Fleet.Core/Fleet.Core.csproj" />
<Project Path="backend/fleet/src/Fleet.Enrollment.Client/Fleet.Enrollment.Client.csproj" />
<Project Path="backend/fleet/src/Fleet.Hosting/Fleet.Hosting.csproj" />
<Project Path="backend/fleet/src/Fleet.Identity/Fleet.Identity.csproj" />
<Project Path="backend/fleet/src/Fleet.Link/Fleet.Link.csproj" />
<Project Path="backend/fleet/src/Fleet.Rpc/Fleet.Rpc.csproj" />
<Project Path="backend/fleet/src/Fleet.Server.Abstractions/Fleet.Server.Abstractions.csproj" />
<Project Path="backend/fleet/src/Fleet.Server.Enrollment/Fleet.Server.Enrollment.csproj" />
<Project Path="backend/fleet/src/Fleet.Server.Link/Fleet.Server.Link.csproj" />
</Folder>
<Folder Name="/backend/fleet/tests/">
<Project Path="backend/fleet/tests/Fleet.Tests/Fleet.Tests.csproj" />
</Folder>
<Folder Name="/backend/fleet/tools/">
<Project Path="backend/fleet/tools/fleet-enroll/fleet-enroll.csproj" />
</Folder>
<Folder Name="/backend/shared/">
<Project Path="backend/shared/Monitor.Measurement/Monitor.Measurement.csproj" />
<Project Path="backend/shared/Monitor.Protocol/Monitor.Protocol.csproj" />
</Folder>
<Folder Name="/backend/shared/tests/">
<Project Path="backend/shared/tests/Monitor.Measurement.Tests/Monitor.Measurement.Tests.csproj" />
<Project Path="backend/shared/tests/Monitor.Protocol.Tests/Monitor.Protocol.Tests.csproj" />
</Folder>
<Folder Name="/backend/agent/src/">
<Project Path="backend/agent/src/Monitor.Agent/Monitor.Agent.csproj" />
</Folder>
<Folder Name="/backend/probe/src/">
<Project Path="backend/probe/src/Monitor.Probe/Monitor.Probe.csproj" />
</Folder>
<Folder Name="/backend/probe/tests/">
<Project Path="backend/probe/tests/Monitor.Probe.Tests/Monitor.Probe.Tests.csproj" />
</Folder>
<Folder Name="/backend/src/">
<Project Path="backend/src/Monitor.Backend.Analysis/Monitor.Backend.Analysis.csproj" />
<Project Path="backend/src/Monitor.Backend.Api/Monitor.Backend.Api.csproj" />
<Project Path="backend/src/Monitor.Backend.Application/Monitor.Backend.Application.csproj" />
<Project Path="backend/src/Monitor.Backend.Gateway/Monitor.Backend.Gateway.csproj" />
<Project Path="backend/src/Monitor.Backend.Storage.Sqlite/Monitor.Backend.Storage.Sqlite.csproj" />
<Project Path="backend/src/Monitor.Backend/Monitor.Backend.csproj" />
</Folder>
<Folder Name="/backend/tests/">
<Project Path="backend/tests/Monitor.Backend.Analysis.Tests/Monitor.Backend.Analysis.Tests.csproj" />
<Project Path="backend/tests/Monitor.Backend.Tests/Monitor.Backend.Tests.csproj" />
<Project Path="backend/tests/Monitor.TestSupport/Monitor.TestSupport.csproj" />
</Folder>
<Folder Name="/ci/">
<Project Path="ci/Monitor.Tar/Monitor.Tar.csproj" />
</Folder>
</Solution>
+47 -9
View File
@@ -10,41 +10,61 @@ Monitor 是一个自托管的网络线路质量观测工具。它在 Probe 与
- **线路分析**:从原始测量数据生成时间序列指标,并比较线路在不同服务模型下的体验。 - **线路分析**:从原始测量数据生成时间序列指标,并比较线路在不同服务模型下的体验。
- **节点管理**:通过 Fleet 节点框架处理身份、注册、连接、任务同步和数据上报。 - **节点管理**:通过 Fleet 节点框架处理身份、注册、连接、任务同步和数据上报。
- **实时面板**:展示端查看线路状态和历史趋势;管理端负责节点、任务和系统配置。 - **实时面板**:展示端查看线路状态和历史趋势;管理端负责节点、任务和系统配置。
- **线路管理**:Backend 决定哪些 Probe 连哪些 Agent(自动连接开关加手动选择),在管理端的线路矩阵里一键开关。
- **节点信息与更新**:节点自报系统、架构和构建信息(版本、tag、提交、构建时间),国家和运营商由 Backend 查询;容器节点的更新由 CI 签名、Backend 转发、节点宿主机上的更新器验签并执行,失败自动回滚,管理端可以看状态和点「更新」。
- **自托管**:Backend 使用段文件和 SQLite 保存数据;提供 Linux、systemd 和 OpenWrt 相关部署文件。 - **自托管**:Backend 使用段文件和 SQLite 保存数据;提供 Linux、systemd 和 OpenWrt 相关部署文件。
## 架构 ## 架构
```text ```text
Probe <──测量会话──> Agent ──长连接与数据上报──> Backend <──REST / SSE──> Web 展示端与管理端 Web(展示端 · 管理端)
├── 段文件:原始数据 ▲ REST + SSE
└── SQLite:索引、指标与配置 ┌──────────────────────────────────────────────────────────┐
│ Backend(面板) │
│ Api · Registry · Planner · Links │
│ Releases · Analysis · LiveHub │
│ Store(段文件 + SQLite) │
└──────────────────────────────────────────────────────────┘
▲ ▲
│ wss 长连接(节点发起) │ wss 长连接(节点发起)
│ 任务、数据、状态、更新通知 │ 身份、Agent 列表、状态、更新通知
┌────────────────────┐ ┌──────────────────────┐
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │
│ 有公网 IP,监听 │ │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
测量会话:TLS 控制通道 + UDP/TCP 数据通道,由 Probe 主动拨出
``` ```
两种连接:**Probe 和 Agent 都用长连接连 Backend**(身份、任务、状态、更新通知),**测量只走 Probe → Agent**(Probe 主动拨出,不需要入站端口)。哪些 Probe 连哪些 Agent 由 Backend 决定:默认自动全连,也可以在管理端手动选择。数据存储:段文件保存原始数据,SQLite 保存索引、指标与配置。
## 快速开始 ## 快速开始
### 环境要求 ### 环境要求
- .NET SDK 10;项目使用 `backend/global.json` 指定 SDK 版本和滚动策略。 - .NET SDK 10;项目使用 `global.json` 指定 SDK 版本和滚动策略。
- Node.js 与 npm,用于构建 Web 前端。 - Node.js 与 npm,用于构建 Web 前端。
### 构建 ### 构建
从仓库根目录构建后端: 从仓库根目录构建全部 .NET 项目(Backend、Agent、Probe、Fleet、测试和 CI 工具):
```sh ```sh
cd backend
dotnet build Monitor.slnx -c Release dotnet build Monitor.slnx -c Release
``` ```
NuGet 包版本统一在根目录 `Directory.Packages.props` 管理,项目中的 `PackageReference` 不写版本号。AotAnywhere 的 MSBuild SDK 版本在根目录 `global.json` 的 `msbuild-sdks` 中管理。`backend/Monitor.slnx` 保留用于只构建后端目录内的项目。
安装前端依赖并构建展示端和管理端(在仓库根目录运行): 安装前端依赖并构建展示端和管理端(在仓库根目录运行):
```sh ```sh
npm ci --prefix web npm ci --prefix web
npm run build --prefix web npm run build --prefix web
npm run gen:api --prefix web # API schema changed after Backend edits
npm run lint:css --prefix web
``` ```
前端产物位于 `web/display/dist/` 和 `web/admin/dist/`。NativeAOT 目标平台和 Windows 上使用 AotAnywhere 交叉构建的说明见 [`docs/deployment.md`](docs/deployment.md)。 前端产物位于 `web/display/dist/` 和 `web/admin/dist/`。NativeAOT 目标平台和 Windows 上使用 AotAnywhere 交叉构建的说明见 [`docs/deployment.md`](docs/deployment.md)。
Backend 以 OCI 镜像发布,支持 Linux/amd64 与 Linux/arm64;Agent 和 Probe 发布 Linux glibc/musl x64、arm64,macOS x64/arm64,以及 Windows x64/arm64 二进制包。容器部署配置见 [`compose.yaml`](compose.yaml),发布说明见 [`docs/deployment.md`](docs/deployment.md)。 Backend、Agent 和 Probe 均以 OCI 镜像发布,支持 Linux/amd64 与 Linux/arm64;Agent 和 Probe 另外提供 Linux glibc/musl、macOS 与 Windows 二进制包。容器部署配置见 [`compose.yaml`](compose.yaml),发布说明见 [`docs/deployment.md`](docs/deployment.md)。
### 本地运行 ### 本地运行
@@ -62,7 +82,16 @@ npm ci --prefix web
npm run dev --prefix web npm run dev --prefix web
``` ```
打开 <http://localhost:5173/> 使用展示端,或访问 <http://localhost:5174/admin/> 进入管理端。依赖安装和前端配置见 [`web/README.md`](web/README.md);Backend 的首次启动与配置见 [`backend/README.md`](backend/README.md)。 打开 <http://localhost:5173/> 使用展示端,或访问 <http://localhost:5173/admin/> 进入管理端。依赖安装和前端配置见 [`web/README.md`](web/README.md);Backend 的首次启动与配置见 [`backend/README.md`](backend/README.md)。
### CI 构建规则
- Pull request 和 `dev` 分支 push:先跑 Backend tests,通过后再构建前端。
- 在需要跳过 Backend tests 的提交信息后加 `[SKIP TESTS]`;Gitea pull request 需要把标记写进 PR 标题。前端构建仍会运行。
- `main` 上的提交信息包含 `[CI]` 时,并行构建 Backend、Agent、Probe 的 Linux/amd64 镜像,推送到 Zot 的 `monitor-dev/` 仓库;普通 main push 不构建。
- 推送 `vMAJOR.MINOR.PATCH` tag 会触发全平台正式构建和 Release 发布。
完整触发条件、开发镜像标签和发布路径见 [`docs/deployment.md`](docs/deployment.md)。
## 项目布局 ## 项目布局
@@ -78,10 +107,19 @@ npm run dev --prefix web
| `packaging/`、`scripts/` | 部署、构建与维护脚本 | | `packaging/`、`scripts/` | 部署、构建与维护脚本 |
| `docs/` | [架构](docs/architecture.md)、[测量模型](docs/measurement.md)、[API](docs/api.md)、[部署](docs/deployment.md)等文档 | | `docs/` | [架构](docs/architecture.md)、[测量模型](docs/measurement.md)、[API](docs/api.md)、[部署](docs/deployment.md)等文档 |
## 待办
已实现的能力见上面的概览和 `docs/`;还没做的:
- **纯二进制节点的更新**:`fleet-updater`、`fleet-release`、Windows 服务和 OpenWrt(procd)适配器;设计见 [`docs/fleet/update.md`](docs/fleet/update.md)。容器节点(Agent、Probe)的签名发布和宿主机更新已经实现,见 [`docs/deployment.md`](docs/deployment.md)。
- **更新的灰度和自动策略**:按标签分批发布;`POLICY=auto` 的线上验证;后端自身的更新。
- **OpenWrt 打包**:procd 脚本、ipk/apk 包、sysupgrade 保留列表,在 aarch64 路由器上长时间运行验证。
- **安全加固**:对 UDP 包、控制帧、RawChunk、JSON-RPC 做模糊测试;伪造签名和重放的专项测试。
- **以后**:告警(Webhook)、真实 QUIC 握手积木、pcap/eBPF 采集。
## 测试 ## 测试
```sh ```sh
cd backend
dotnet test --solution Monitor.slnx dotnet test --solution Monitor.slnx
``` ```
+1 -1
View File
@@ -13,7 +13,7 @@
<UseArtifactsOutput>true</UseArtifactsOutput> <UseArtifactsOutput>true</UseArtifactsOutput>
<ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath> <ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath>
<Deterministic>true</Deterministic> <Deterministic>true</Deterministic>
<Version>0.1.0</Version> <Version>0.1.1</Version>
<NuGetAudit>false</NuGetAudit> <NuGetAudit>false</NuGetAudit>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
+4
View File
@@ -8,6 +8,10 @@
<EventSourceSupport>false</EventSourceSupport> <EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport> <DebuggerSupport>false</DebuggerSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport> <HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<Http3Support>false</Http3Support>
<MetricsSupport>false</MetricsSupport>
<UseSizeOptimizedLinq>true</UseSizeOptimizedLinq>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<ServerGarbageCollection>false</ServerGarbageCollection> <ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection> <ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
</PropertyGroup> </PropertyGroup>
+2
View File
@@ -47,6 +47,8 @@ fleet-enroll --server https://<backend> --join <token> --name <agent-name> --ide
monitor-agent --config /etc/monitor/agent.json monitor-agent --config /etc/monitor/agent.json
``` ```
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/deployment.md`](../../docs/deployment.md)。也可使用 OCI 镜像 `registry.oeo.one/monitor/monitor-agent:<version>`,自动选择 Linux amd64 或 arm64 镜像。镜像同时包含 `fleet-enroll`;可通过 `--entrypoint /usr/local/bin/fleet-enroll` 执行首次注册。容器默认读取 `/etc/monitor/agent.json`,需要挂载配置与身份目录;建议使用宿主网络,并按 ICMP 测量需求添加 `NET_RAW` 能力。镜像以 UID/GID 1654 运行,挂载目录必须允许该用户读取配置和身份文件。
不传 `--config` 时默认读取 `/etc/monitor/agent.json`。Agent 默认监听控制端口 `7443/tcp`,数据端口 `7443/udp` 和 `7444/tcp`;防火墙和面板中的节点端口配置需与本地配置一致。`reporting.host_stats` 可关闭主机状态上报;启用时只上报资源使用等汇总数据,不包含主机名、进程或 IP。 不传 `--config` 时默认读取 `/etc/monitor/agent.json`。Agent 默认监听控制端口 `7443/tcp`,数据端口 `7443/udp` 和 `7444/tcp`;防火墙和面板中的节点端口配置需与本地配置一致。`reporting.host_stats` 可关闭主机状态上报;启用时只上报资源使用等汇总数据,不包含主机名、进程或 IP。
## 设计说明 ## 设计说明
+25 -10
View File
@@ -12,7 +12,7 @@ namespace Monitor.Agent;
/// <summary>Agent 的组装点:数据面、点对点监听、与 Backend 的链路都在这里接起来。没有全局状态,测试可以在同一进程里启动多个。</summary> /// <summary>Agent 的组装点:数据面、点对点监听、与 Backend 的链路都在这里接起来。没有全局状态,测试可以在同一进程里启动多个。</summary>
public sealed class AgentApp : IDisposable public sealed class AgentApp : IDisposable
{ {
public const string Version = "0.1.0"; public const string Version = Monitor.Protocol.BuildInfo.Version;
private readonly AgentConfig _config; private readonly AgentConfig _config;
private readonly LoadedNodeIdentity _identity; private readonly LoadedNodeIdentity _identity;
@@ -25,6 +25,7 @@ public sealed class AgentApp : IDisposable
private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously); private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly AgentAdmission _admission; private readonly AgentAdmission _admission;
private readonly Outbox _outbox = new(64 << 20); private readonly Outbox _outbox = new(64 << 20);
private readonly UpdateBridge _update;
private readonly BackendUplink _uplink; private readonly BackendUplink _uplink;
private readonly UdpMux _udp; private readonly UdpMux _udp;
private readonly TcpAcceptor _tcp; private readonly TcpAcceptor _tcp;
@@ -34,8 +35,10 @@ public sealed class AgentApp : IDisposable
private readonly TestRunner _tests; private readonly TestRunner _tests;
private readonly FleetLink _link; private readonly FleetLink _link;
public AgentApp(AgentConfig config, LoadedNodeIdentity identity, TimeProvider time, INodeLog log) /// <param name="dataDirectory">配置目录(容器里的 <c>/etc/monitor</c>):宿主机更新器和节点在这里交换清单、请求和状态。</param>
public AgentApp(AgentConfig config, LoadedNodeIdentity identity, string dataDirectory, TimeProvider time, INodeLog log)
{ {
_update = new UpdateBridge(dataDirectory, new ConfigStore(dataDirectory).Write, log);
_config = config; _config = config;
_identity = identity; _identity = identity;
_time = time; _time = time;
@@ -200,6 +203,24 @@ public sealed class AgentApp : IDisposable
(_, cancel, _) => ValueTask.FromResult(Result<TestRunResult>.Ok(new TestRunResult { Accepted = _tests.Cancel(cancel.TestId) }))); (_, cancel, _) => ValueTask.FromResult(Result<TestRunResult>.Ok(new TestRunResult { Accepted = _tests.Cancel(cancel.TestId) })));
} }
if (Allow(MonitorMethods.UpdateOffer))
{
router.OnNotify(MonitorMethods.UpdateOffer, MonitorJsonContext.Default.UpdateOfferParams, (_, offer, _) =>
{
_update.WriteOffer(offer);
return ValueTask.CompletedTask;
});
}
if (Allow(MonitorMethods.UpdateApply))
{
router.OnNotify(MonitorMethods.UpdateApply, MonitorJsonContext.Default.UpdateOfferParams, (_, _, _) =>
{
_update.RequestApply();
return ValueTask.CompletedTask;
});
}
if (Allow(MonitorMethods.ProbeDisconnect)) if (Allow(MonitorMethods.ProbeDisconnect))
{ {
router.On( router.On(
@@ -209,14 +230,6 @@ public sealed class AgentApp : IDisposable
async (_, request, _) => Result<TestRunResult>.Ok(new TestRunResult { Accepted = await _hub.DisconnectAsync(request.ProbeId.Value) })); async (_, request, _) => Result<TestRunResult>.Ok(new TestRunResult { Accepted = await _hub.DisconnectAsync(request.ProbeId.Value) }));
} }
if (Allow(MonitorMethods.ProbeAgentsChanged))
{
router.OnNotify(
MonitorMethods.ProbeAgentsChanged,
MonitorJsonContext.Default.ProbeAgentsResult,
(_, list, _) => _hub.RelayAgentsAsync(list));
}
var options = new FleetLinkOptions var options = new FleetLinkOptions
{ {
Server = new Uri(_config.Fleet.Server), Server = new Uri(_config.Fleet.Server),
@@ -244,6 +257,7 @@ public sealed class AgentApp : IDisposable
{ {
var meta = new AgentHelloMeta var meta = new AgentHelloMeta
{ {
Platform = Monitor.Measurement.PlatformDetector.Detect(),
TlsSpki = _identity.File.TlsSpki, TlsSpki = _identity.File.TlsSpki,
ProbeTcpPort = ControlPort, ProbeTcpPort = ControlPort,
ProbeUdpPorts = [.. UdpPorts], ProbeUdpPorts = [.. UdpPorts],
@@ -277,6 +291,7 @@ public sealed class AgentApp : IDisposable
DroppedInvalid = _counters.DroppedInvalid, DroppedInvalid = _counters.DroppedInvalid,
DroppedRateLimited = _counters.DroppedRateLimited, DroppedRateLimited = _counters.DroppedRateLimited,
QueueBytes = _outbox.PendingBytes, QueueBytes = _outbox.PendingBytes,
Update = _update.ReadStatus(),
}; };
_lastIn = packetsIn; _lastIn = packetsIn;
_lastOut = packetsOut; _lastOut = packetsOut;
@@ -6,23 +6,30 @@ namespace Monitor.Agent;
/// <summary> /// <summary>
/// 主机资源占用的只读快照:CPU 占用、内存、根分区、网卡累计流量与瞬时网速、系统运行时长。 /// 主机资源占用的只读快照:CPU 占用、内存、根分区、网卡累计流量与瞬时网速、系统运行时长。
/// 只有汇总的数字:<b>不采集 CPU 型号、主机名、网卡名和任何 IP 地址</b>,所以面板和展示页想泄漏也泄漏不出去。 /// 只有汇总的数字:<b>不采集 CPU 型号、主机名、网卡名和任何 IP 地址</b>,所以面板和展示页想泄漏也泄漏不出去。
/// Linux 读 <c>/proc</c>;读不到的项是 null(例如 Windows 上的 CPU)。不执行任何命令,不写文件。 /// Linux 读 <c>/proc</c> 和 <c>/sys</c>;容器通过 HOST_PROC、HOST_SYS、HOST_ROOT 指定宿主路径。
/// 读不到的项是 null(例如 Windows 上的 CPU)。不执行任何命令,不写文件。
/// </summary> /// </summary>
public sealed class HostSampler public sealed class HostSampler
{ {
private static readonly string[] s_virtualPrefixes = ["lo", "veth", "docker", "br-", "virbr", "cni", "flannel", "cali", "tun", "tap"]; private static readonly string[] s_virtualPrefixes = ["lo", "veth", "docker", "podman", "br-", "virbr", "cni", "flannel", "cali", "tun", "tap"];
private readonly TimeProvider _time; private readonly TimeProvider _time;
private readonly Func<string, string?> _read; private readonly Func<string, string?> _read;
private readonly Func<(long Total, long Used)?> _disk; private readonly Func<(long Total, long Used)?> _disk;
private readonly string _procRoot;
private readonly string _sysRoot;
private (long Total, long Idle)? _previousCpu; private (long Total, long Idle)? _previousCpu;
private (long At, long Rx, long Tx)? _previousNet; private (long At, long Rx, long Tx)? _previousNet;
public HostSampler(TimeProvider time, Func<string, string?>? read = null, Func<(long Total, long Used)?>? disk = null) public HostSampler(TimeProvider time, Func<string, string?>? read = null, Func<(long Total, long Used)?>? disk = null,
string? procRoot = null, string? sysRoot = null, string? diskRoot = null)
{ {
_time = time; _time = time;
_read = read ?? ReadProc; _read = read ?? ReadProc;
_disk = disk ?? ReadRootDisk; _procRoot = Root(procRoot, "HOST_PROC", "/proc");
_sysRoot = Root(sysRoot, "HOST_SYS", "/sys");
string root = Root(diskRoot, "HOST_ROOT", OperatingSystem.IsWindows() ? Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\" : "/");
_disk = disk ?? (() => ReadRootDisk(root));
} }
public HostSnapshot Sample() public HostSnapshot Sample()
@@ -30,7 +37,7 @@ public sealed class HostSampler
long now = _time.GetTimestamp(); long now = _time.GetTimestamp();
var snapshot = new HostSnapshot { AtUtcMs = _time.GetUtcNow().ToUnixTimeMilliseconds() }; var snapshot = new HostSnapshot { AtUtcMs = _time.GetUtcNow().ToUnixTimeMilliseconds() };
if (_read("/proc/stat") is { } stat && ParseCpu(stat) is { } cpu) if (_read(Path.Combine(_procRoot, "stat")) is { } stat && ParseCpu(stat) is { } cpu)
{ {
if (_previousCpu is { } before && cpu.Total > before.Total) if (_previousCpu is { } before && cpu.Total > before.Total)
{ {
@@ -41,7 +48,7 @@ public sealed class HostSampler
_previousCpu = cpu; _previousCpu = cpu;
} }
if (_read("/proc/meminfo") is { } meminfo && ParseMemory(meminfo) is { } memory) if (_read(Path.Combine(_procRoot, "meminfo")) is { } meminfo && ParseMemory(meminfo) is { } memory)
{ {
snapshot.MemTotal = memory.Total; snapshot.MemTotal = memory.Total;
snapshot.MemUsed = memory.Used; snapshot.MemUsed = memory.Used;
@@ -53,10 +60,11 @@ public sealed class HostSampler
snapshot.DiskUsed = disk.Used; snapshot.DiskUsed = disk.Used;
} }
if (_read("/proc/net/dev") is { } dev && ParseNetwork(dev) is { } net) if (_read(Path.Combine(_procRoot, "net/dev")) is { } dev && ParseNetwork(dev) is { } net)
{ {
snapshot.NetRxTotal = net.Rx; snapshot.NetRxTotal = net.Rx;
snapshot.NetTxTotal = net.Tx; snapshot.NetTxTotal = net.Tx;
snapshot.NetLinkMbps = ReadLinkSpeed(dev);
if (_previousNet is { } before && net.Rx >= before.Rx && net.Tx >= before.Tx) if (_previousNet is { } before && net.Rx >= before.Rx && net.Tx >= before.Tx)
{ {
double seconds = Math.Max(0.001, _time.GetElapsedTime(before.At, now).TotalSeconds); double seconds = Math.Max(0.001, _time.GetElapsedTime(before.At, now).TotalSeconds);
@@ -67,7 +75,11 @@ public sealed class HostSampler
_previousNet = (now, net.Rx, net.Tx); _previousNet = (now, net.Rx, net.Tx);
} }
snapshot.UptimeSeconds = _read("/proc/uptime") is { } uptime && ParseUptime(uptime) is { } up ? up : Environment.TickCount64 / 1000; snapshot.UptimeSeconds = _read(Path.Combine(_procRoot, "uptime")) is { } uptime && ParseUptime(uptime) is { } up ? up : null;
if (!OperatingSystem.IsLinux() && snapshot.UptimeSeconds is null)
{
snapshot.UptimeSeconds = Environment.TickCount64 / 1000;
}
return snapshot; return snapshot;
} }
@@ -165,11 +177,54 @@ public sealed class HostSampler
} }
} }
private static (long Total, long Used)? ReadRootDisk() private static string Root(string? value, string variable, string fallback)
{
value ??= Environment.GetEnvironmentVariable(variable);
return string.IsNullOrWhiteSpace(value) ? fallback : value;
}
private long? ReadLinkSpeed(string dev)
{
long total = 0;
bool any = false;
foreach (string line in dev.Split('\n'))
{
int colon = line.IndexOf(':', StringComparison.Ordinal);
if (colon < 0)
{
continue;
}
string name = line[..colon].Trim();
if (s_virtualPrefixes.Any(p => name.StartsWith(p, StringComparison.Ordinal)))
{
continue;
}
string? speed = _read(Path.Combine(_sysRoot, "class/net", name, "speed"));
if (long.TryParse(speed?.Trim(), CultureInfo.InvariantCulture, out long mbps) && mbps > 0)
{
total += mbps;
any = true;
}
else
{
return null;
}
}
return any ? total : null;
}
private static (long Total, long Used)? ReadRootDisk(string root)
{ {
try try
{ {
string root = OperatingSystem.IsWindows() ? Path.GetPathRoot(Environment.SystemDirectory) ?? "C:\\" : "/"; if (!Directory.Exists(root))
{
return null;
}
var drive = new DriveInfo(root); var drive = new DriveInfo(root);
return drive.IsReady ? (drive.TotalSize, drive.TotalSize - drive.AvailableFreeSpace) : null; return drive.IsReady ? (drive.TotalSize, drive.TotalSize - drive.AvailableFreeSpace) : null;
} }
@@ -1,12 +1,18 @@
<Project> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.props" Sdk="StuDev.AotAnywhere"
Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" /> Condition="'$(UseAotAnywhere)' == 'true'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>monitor-agent</AssemblyName> <AssemblyName>monitor-agent</AssemblyName>
<RootNamespace>Monitor.Agent</RootNamespace> <RootNamespace>Monitor.Agent</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>full</TrimMode>
<StripSymbols>true</StripSymbols>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<InternalsVisibleTo Include="Monitor.Backend.Tests" /> <InternalsVisibleTo Include="Monitor.Backend.Tests" />
@@ -16,5 +22,5 @@
</ItemGroup> </ItemGroup>
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere"
Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" /> Condition="'$(UseAotAnywhere)' == 'true'" />
</Project> </Project>
@@ -57,18 +57,6 @@ public sealed class BackendUplink
await SafeAsync(() => session.NotifyAsync(MonitorMethods.TestStatus, parameters, MonitorJsonContext.Default.TestStatusParams, CancellationToken.None)); await SafeAsync(() => session.NotifyAsync(MonitorMethods.TestStatus, parameters, MonitorJsonContext.Default.TestStatusParams, CancellationToken.None));
} }
/// <summary>代 Probe 向 Backend 取 Agent 列表(Probe 不认识 Backend)。链路断开时返回 fleet_node_offline。</summary>
public async ValueTask<Result<ProbeAgentsResult>> RequestProbeAgentsAsync(ProbeAgentsParams request, CancellationToken ct)
{
RpcSession<LinkContext>? session = _link?.CurrentSession;
if (session is null)
{
return Result<ProbeAgentsResult>.Fail(FleetErrors.NodeOffline, "the agent has no link to the backend");
}
return await session.RequestAsync(MonitorMethods.ProbeAgents, request, MonitorJsonContext.Default.ProbeAgentsParams, MonitorJsonContext.Default.ProbeAgentsResult, ct);
}
private static async ValueTask SafeAsync(Func<ValueTask> send) private static async ValueTask SafeAsync(Func<ValueTask> send)
{ {
try try
@@ -104,7 +92,7 @@ public sealed class PeerHub
public PeerHub(SessionRegistry sessions, MonoClock clock, ChunkAssembler chunks, TaskManager tasks, BackendUplink uplink, TimeProvider time, INodeLog log) public PeerHub(SessionRegistry sessions, MonoClock clock, ChunkAssembler chunks, TaskManager tasks, BackendUplink uplink, TimeProvider time, INodeLog log)
{ {
_router = BuildRouter(uplink); _router = BuildRouter();
_sessions = sessions; _sessions = sessions;
_clock = clock; _clock = clock;
_chunks = chunks; _chunks = chunks;
@@ -252,27 +240,6 @@ public sealed class PeerHub
} }
} }
/// <summary>Backend 的 Agent 列表变化时转交给所有 Probe(Probe 据此调整它连接的 Agent)。</summary>
public async ValueTask RelayAgentsAsync(ProbeAgentsResult list)
{
foreach (ProbeSession probe in _sessions.All)
{
if (probe.Rpc is not { } rpc)
{
continue;
}
try
{
await rpc.NotifyAsync(MonitorMethods.ProbeAgentsChanged, list, MonitorJsonContext.Default.ProbeAgentsResult, CancellationToken.None);
}
catch (Exception ex) when (ex is ObjectDisposedException or InvalidOperationException or IOException)
{
// 会话刚好结束
}
}
}
private async ValueTask PushAsync(ProbeSession probe, LineTaskSet set, CancellationToken ct) private async ValueTask PushAsync(ProbeSession probe, LineTaskSet set, CancellationToken ct)
{ {
if (probe.Rpc is null) if (probe.Rpc is null)
@@ -300,13 +267,8 @@ public sealed class PeerHub
} }
} }
private static RpcRouter<PeerContext> BuildRouter(BackendUplink uplink) => private static RpcRouter<PeerContext> BuildRouter() =>
new RpcRouter<PeerContext>() new RpcRouter<PeerContext>()
.On(
MonitorMethods.ProbeAgents,
MonitorJsonContext.Default.ProbeAgentsParams,
MonitorJsonContext.Default.ProbeAgentsResult,
(_, request, ct) => uplink.RequestProbeAgentsAsync(request, ct))
.On( .On(
MonitorMethods.Clock, MonitorMethods.Clock,
MonitorJsonContext.Default.ClockParams, MonitorJsonContext.Default.ClockParams,
+1 -1
View File
@@ -35,7 +35,7 @@ catch (Exception ex) when (ex is IOException or InvalidDataException or InvalidO
using (identity) using (identity)
{ {
using var app = new AgentApp(config, identity, time, log); using var app = new AgentApp(config, identity, Path.GetDirectoryName(Path.GetFullPath(configPath))!, time, log);
await ServiceShell.RunAsync(app.RunAsync, log); await ServiceShell.RunAsync(app.RunAsync, log);
} }
@@ -1,17 +1,36 @@
<Project> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.props" Sdk="StuDev.AotAnywhere"
Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" /> Condition="'$(UseAotAnywhere)' == 'true'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>fleet-enroll</AssemblyName> <AssemblyName>fleet-enroll</AssemblyName>
<RootNamespace>FleetEnroll</RootNamespace> <RootNamespace>FleetEnroll</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>full</TrimMode>
<StripSymbols>true</StripSymbols>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
<OptimizationPreference>Size</OptimizationPreference>
<UseSystemResourceKeys>true</UseSystemResourceKeys>
<StackTraceSupport>false</StackTraceSupport>
<EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<Http3Support>false</Http3Support>
<MetricsSupport>false</MetricsSupport>
<UseSizeOptimizedLinq>true</UseSizeOptimizedLinq>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<IlcFoldIdenticalMethodBodies>true</IlcFoldIdenticalMethodBodies>
<ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\..\src\Fleet.Enrollment.Client\Fleet.Enrollment.Client.csproj" /> <ProjectReference Include="..\..\src\Fleet.Enrollment.Client\Fleet.Enrollment.Client.csproj" />
</ItemGroup> </ItemGroup>
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere"
Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" /> Condition="'$(UseAotAnywhere)' == 'true'" />
</Project> </Project>
+885 -28
View File
File diff suppressed because it is too large. Load diff
+4
View File
@@ -8,6 +8,10 @@
<EventSourceSupport>false</EventSourceSupport> <EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport> <DebuggerSupport>false</DebuggerSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport> <HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<Http3Support>false</Http3Support>
<MetricsSupport>false</MetricsSupport>
<UseSizeOptimizedLinq>true</UseSizeOptimizedLinq>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<ServerGarbageCollection>false</ServerGarbageCollection> <ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>false</ConcurrentGarbageCollection> <ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
</PropertyGroup> </PropertyGroup>
+11 -6
View File
@@ -1,16 +1,19 @@
# Monitor Probe # Monitor Probe
Probe 是网络测量的发起端,通常运行在家宽或其他需要观测的网络出口。它连接 Agent、执行线路任务,并把本端采集的原始数据交给 Agent 汇总上报。Probe 不直接连接 Backend,也不提供远程命令或文件管理功能。 Probe 是网络测量的发起端,通常运行在家宽或其他需要观测的网络出口。它连接 Backend(身份、Agent 列表、状态)和 Agent(测量),执行线路任务,并把本端采集的原始数据交给 Agent 汇总上报。Probe 不提供远程命令或文件管理功能。
## 工作流程 ## 工作流程
```text ```text
Backend ──任务与节点列表──> Agent <──测量会话──> Probe Backend ──任务──> Agent <──测量会话──> Probe
│ │ │ ▲ │ │
└──汇总并上报数据 <───┘ │ └──汇总并上报数据 <───┘ │
└─────Agent 列表、状态(Fleet 长连接)───────────┘
``` ```
Probe 使用面板生成的 `probe.json` 读取自身身份、首次连接用的 Agent 地址和本地测试限制。成功连接后,Agent 会同步可用节点列表;Probe 将列表缓存到同目录的 `agents.json`。`probe.json` 含私钥,请限制文件访问权限,不要提交到版本控制。 Probe 使用面板生成的 `probe.json` 读取自身身份、面板地址(`server`)和本地测试限制。连上面板之后,面板下发要连接的 Agent 列表;Probe 将列表缓存到同目录的 `agents.json`,面板暂时连不上时靠它继续测量。`probe.json` 含私钥,请限制文件访问权限,不要提交到版本控制。
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/deployment.md`](../../docs/deployment.md);Probe 也可以用 OCI 镜像 `registry.oeo.one/monitor/monitor-probe:<version>`,容器默认读取 `/etc/monitor/probe.json`,数据目录要允许 UID/GID 1654 读写。
## 构建 ## 构建
@@ -39,8 +42,10 @@ monitor-probe --config /etc/monitor/probe.json
不传 `--config` 时默认读取 `/etc/monitor/probe.json`。配置中的 `limits` 是本地硬上限,Backend 无法远程放宽;路由器等低性能设备可调低测试码率和包速率,避免 Probe 自身成为测量瓶颈。部署时可使用仓库的 `packaging/` 文件和安装脚本。 不传 `--config` 时默认读取 `/etc/monitor/probe.json`。配置中的 `limits` 是本地硬上限,Backend 无法远程放宽;路由器等低性能设备可调低测试码率和包速率,避免 Probe 自身成为测量瓶颈。部署时可使用仓库的 `packaging/` 文件和安装脚本。
也可使用 OCI 镜像 `registry.oeo.one/monitor/monitor-probe:<version>`,自动选择 Linux amd64 或 arm64 镜像。容器默认读取 `/etc/monitor/probe.json`,并在同一目录维护 `agents.json` 缓存,因此挂载目录必须允许 UID/GID 1654 读写。建议使用宿主网络,并按 ICMP 测量需求添加 `NET_RAW` 能力。
## 设计说明 ## 设计说明
- Probe 只执行编译进程序的测量模块。它不认识 Backend,不落盘测量数据,也不能执行命令或自行更新。 - Probe 只执行编译进程序的测量模块。它不落盘测量数据,也不能执行命令或自行更新(更新由宿主机上的更新器完成,Probe 只写清单和请求文件)。
- 测量模块由 `backend/shared/Monitor.Measurement/` 与 Agent 共用;Probe 负责发起连接、调度任务和记录本端数据。 - 测量模块由 `backend/shared/Monitor.Measurement/` 与 Agent 共用;Probe 负责发起连接、调度任务和记录本端数据。
- 测量模型和协议细节见 [`../../docs/measurement.md`](../../docs/measurement.md) 与 [`../../docs/protocol.md`](../../docs/protocol.md);整体架构见 [`../../docs/architecture.md`](../../docs/architecture.md)。 - 测量模型和协议细节见 [`../../docs/measurement.md`](../../docs/measurement.md) 与 [`../../docs/protocol.md`](../../docs/protocol.md);整体架构见 [`../../docs/architecture.md`](../../docs/architecture.md)。
@@ -15,7 +15,7 @@ namespace Monitor.Probe;
public sealed record ConnectionEnv(NodeIdentity Identity, string ProbeId, ProbeLimits Limits, AgentDirectory Directory, MonoClock Clock, TimeProvider Time, INodeLog Log); public sealed record ConnectionEnv(NodeIdentity Identity, string ProbeId, ProbeLimits Limits, AgentDirectory Directory, MonoClock Clock, TimeProvider Time, INodeLog Log);
/// <summary> /// <summary>
/// 与一个 Agent 的点对点会话:认证握手 → 同步时钟 → 取 Agent 列表 → 按 Agent 下发的 <c>line_task_set</c> 运行角色 → /// 与一个 Agent 的点对点会话:认证握手 → 同步时钟 → 按 Agent 下发的 <c>line_task_set</c> 运行角色 →
/// 每 10 秒把已完成的行切成列块放进发件箱 → 响应按需测试的阶段。会话断开时所有角色停止(凭据属于会话),按退避重连。 /// 每 10 秒把已完成的行切成列块放进发件箱 → 响应按需测试的阶段。会话断开时所有角色停止(凭据属于会话),按退避重连。
/// 会话内的所有状态(套接字、发件箱、角色)随会话一起创建和释放。 /// 会话内的所有状态(套接字、发件箱、角色)随会话一起创建和释放。
/// </summary> /// </summary>
@@ -28,13 +28,13 @@ public sealed class AgentConnection
private readonly ConnectionEnv _env; private readonly ConnectionEnv _env;
private volatile ConnectionSession? _session; private volatile ConnectionSession? _session;
public AgentConnection(BootstrapAgent endpoint, ConnectionEnv env) public AgentConnection(KnownAgent endpoint, ConnectionEnv env)
{ {
Endpoint = endpoint; Endpoint = endpoint;
_env = env; _env = env;
} }
public BootstrapAgent Endpoint { get; } public KnownAgent Endpoint { get; }
public bool IsConnected => _session is not null; public bool IsConnected => _session is not null;
@@ -105,12 +105,11 @@ public sealed class AgentConnection
Task pump = PumpAsync(rpc, outbox, cts.Token); Task pump = PumpAsync(rpc, outbox, cts.Token);
try try
{ {
// 先同步时钟(块头里要带偏移),再取 Agent 列表;Agent 随后推送线路任务集 // 先同步时钟(块头里要带偏移);Agent 随后推送线路任务集
await SyncClockAsync(rpc, clock, rounds: 5, cts.Token); await SyncClockAsync(rpc, clock, rounds: 5, cts.Token);
backoff.Reset(); backoff.Reset();
_session = session; _session = session;
_env.Log.Info("agent session up", ("agent", Endpoint.AgentId), ("offset_us", clock.OffsetUs)); _env.Log.Info("agent session up", ("agent", Endpoint.AgentId), ("offset_us", clock.OffsetUs));
await RefreshDirectoryAsync(rpc, cts.Token);
Task maintenance = MaintainAsync(rpc, session, clock, cts.Token); Task maintenance = MaintainAsync(rpc, session, clock, cts.Token);
Task finished = await Task.WhenAny(run, receive, pump, maintenance); Task finished = await Task.WhenAny(run, receive, pump, maintenance);
await cts.CancelAsync(); await cts.CancelAsync();
@@ -163,20 +162,6 @@ public sealed class AgentConnection
} }
} }
private async Task RefreshDirectoryAsync(RpcSession<ConnectionSession> rpc, CancellationToken ct)
{
Result<ProbeAgentsResult> list = await rpc.RequestAsync(
MonitorMethods.ProbeAgents, new ProbeAgentsParams { HaveVersion = _env.Directory.Version }, MonitorJsonContext.Default.ProbeAgentsParams, MonitorJsonContext.Default.ProbeAgentsResult, ct);
if (list.TryGet(out ProbeAgentsResult? agents, out FleetError? error))
{
_env.Directory.Apply(agents);
}
else
{
_env.Log.Warn("could not fetch the agent list", ("agent", Endpoint.AgentId), ("err", error.Code));
}
}
/// <summary>切块(每 10 秒)和时钟同步(每 30 秒)。</summary> /// <summary>切块(每 10 秒)和时钟同步(每 30 秒)。</summary>
private async Task MaintainAsync(RpcSession<ConnectionSession> rpc, ConnectionSession session, ClockEstimator clock, CancellationToken ct) private async Task MaintainAsync(RpcSession<ConnectionSession> rpc, ConnectionSession session, ClockEstimator clock, CancellationToken ct)
{ {
@@ -229,14 +214,6 @@ public sealed class AgentConnection
MonitorMethods.TestAbort, MonitorMethods.TestAbort,
MonitorJsonContext.Default.TestAbortParams, MonitorJsonContext.Default.TestAbortParams,
async (session, abort, _) => await session.AbortAsync(abort.TestId)) async (session, abort, _) => await session.AbortAsync(abort.TestId))
.OnNotify(
MonitorMethods.ProbeAgentsChanged,
MonitorJsonContext.Default.ProbeAgentsResult,
(session, list, _) =>
{
session.Env.Directory.Apply(list);
return ValueTask.CompletedTask;
})
.OnNotify( .OnNotify(
FleetMethods.OutboxAck, FleetMethods.OutboxAck,
FleetRpcContext.Default.OutboxAckParams, FleetRpcContext.Default.OutboxAckParams,
@@ -6,8 +6,7 @@ using Monitor.Protocol;
namespace Monitor.Probe; namespace Monitor.Probe;
/// <summary> /// <summary>
/// Probe 要连接的 Agent 列表。Probe 不认识 Backend:列表先取自 <c>probe.json</c> 的 bootstrap(或上一次保存的 <c>agents.json</c>), /// Probe 要连接的 Agent 列表:由面板作为期望状态(<c>probe_agents</c>)直接下发;面板连不上时用上一次保存的 <c>agents.json</c>。
/// 连上任意一个 Agent 之后,由它代为向 Backend 查询(<c>monitor.probe.agents</c>),之后的变化由 Agent 推送(<c>agents_changed</c>)。
/// 列表变化时用 <see cref="ConfigStore"/> 写回 <c>agents.json</c>(这是配置,不是测量数据;只在变化时写,保护路由器的闪存)。 /// 列表变化时用 <see cref="ConfigStore"/> 写回 <c>agents.json</c>(这是配置,不是测量数据;只在变化时写,保护路由器的闪存)。
/// </summary> /// </summary>
public sealed class AgentDirectory public sealed class AgentDirectory
@@ -17,29 +16,25 @@ public sealed class AgentDirectory
private readonly object _gate = new(); private readonly object _gate = new();
private readonly ConfigStore _store; private readonly ConfigStore _store;
private readonly INodeLog _log; private readonly INodeLog _log;
private List<BootstrapAgent> _agents; private List<KnownAgent> _agents = [];
private long _version; private long _version;
public AgentDirectory(ConfigStore store, string configDirectory, ProbeFile file, INodeLog log) public AgentDirectory(ConfigStore store, string configDirectory, INodeLog log)
{ {
_store = store; _store = store;
_log = log; _log = log;
_agents = [.. file.Bootstrap];
string cache = Path.Combine(configDirectory, FileName); string cache = Path.Combine(configDirectory, FileName);
if (File.Exists(cache)) if (File.Exists(cache))
{ {
try try
{ {
AgentsCache saved = NodeConfig.Load(cache, ProbeConfigContext.Default.AgentsCache); AgentsCache saved = NodeConfig.Load(cache, ProbeConfigContext.Default.AgentsCache);
if (saved.Agents.Count > 0) _agents = saved.Agents;
{ _version = saved.Version;
_agents = saved.Agents;
_version = saved.Version;
}
} }
catch (Exception ex) when (ex is IOException or InvalidDataException or JsonException or InvalidOperationException) catch (Exception ex) when (ex is IOException or InvalidDataException or JsonException or InvalidOperationException)
{ {
_log.Warn("agents.json is unreadable, using the bootstrap list", ("err", ex.Message)); _log.Warn("agents.json is unreadable, waiting for the panel", ("err", ex.Message));
} }
} }
} }
@@ -58,7 +53,7 @@ public sealed class AgentDirectory
} }
} }
public IReadOnlyList<BootstrapAgent> Agents public IReadOnlyList<KnownAgent> Agents
{ {
get get
{ {
@@ -69,10 +64,10 @@ public sealed class AgentDirectory
} }
} }
/// <summary>应用 Agent 转来的列表。版本不比当前新的忽略(乱序到达的旧推送)。</summary> /// <summary>应用面板下发的列表。版本不比当前新的忽略(乱序到达的旧推送)。</summary>
public void Apply(ProbeAgentsResult result) public void Apply(ProbeAgentList result)
{ {
var next = result.Agents.Select(a => new BootstrapAgent { AgentId = a.AgentId.Value, Host = a.Host, TcpPort = a.TcpPort, Spki = a.Spki }).ToList(); var next = result.Agents.Select(a => new KnownAgent { AgentId = a.AgentId.Value, Host = a.Host, TcpPort = a.TcpPort, Spki = a.Spki }).ToList();
lock (_gate) lock (_gate)
{ {
if (result.Version < _version || (result.Version == _version && Same(next, _agents))) if (result.Version < _version || (result.Version == _version && Same(next, _agents)))
@@ -97,9 +92,9 @@ public sealed class AgentDirectory
Changed?.Invoke(); Changed?.Invoke();
} }
private static bool Same(List<BootstrapAgent> a, List<BootstrapAgent> b) => private static bool Same(List<KnownAgent> a, List<KnownAgent> b) =>
a.Count == b.Count && a.Zip(b).All(p => Equal(p.First, p.Second)); a.Count == b.Count && a.Zip(b).All(p => Equal(p.First, p.Second));
public static bool Equal(BootstrapAgent a, BootstrapAgent b) => public static bool Equal(KnownAgent a, KnownAgent b) =>
a.AgentId == b.AgentId && a.Host == b.Host && a.TcpPort == b.TcpPort && a.Spki == b.Spki; a.AgentId == b.AgentId && a.Host == b.Host && a.TcpPort == b.TcpPort && a.Spki == b.Spki;
} }
@@ -1,12 +1,18 @@
<Project> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.props" Sdk="StuDev.AotAnywhere"
Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" /> Condition="'$(UseAotAnywhere)' == 'true'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>monitor-probe</AssemblyName> <AssemblyName>monitor-probe</AssemblyName>
<RootNamespace>Monitor.Probe</RootNamespace> <RootNamespace>Monitor.Probe</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<PublishTrimmed>true</PublishTrimmed>
<TrimMode>full</TrimMode>
<StripSymbols>true</StripSymbols>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<InternalsVisibleTo Include="Monitor.Backend.Tests" /> <InternalsVisibleTo Include="Monitor.Backend.Tests" />
@@ -16,5 +22,5 @@
</ItemGroup> </ItemGroup>
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" /> <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" <Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere"
Condition="'$(RuntimeIdentifier)' == 'linux-x64' Or '$(RuntimeIdentifier)' == 'linux-arm64' Or '$(RuntimeIdentifier)' == 'linux-arm' Or '$(RuntimeIdentifier)' == 'linux-musl-x64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm64' Or '$(RuntimeIdentifier)' == 'linux-musl-arm' Or '$(RuntimeIdentifier)' == 'osx-x64' Or '$(RuntimeIdentifier)' == 'osx-arm64' Or '$(RuntimeIdentifier)' == 'win-x64' Or '$(RuntimeIdentifier)' == 'win-arm64'" /> Condition="'$(UseAotAnywhere)' == 'true'" />
</Project> </Project>
+50 -8
View File
@@ -1,32 +1,72 @@
using Fleet.Core; using Fleet.Core;
using Fleet.Hosting; using Fleet.Hosting;
using System.Text.Json;
using Fleet.Identity; using Fleet.Identity;
using Fleet.Link;
using Fleet.Rpc;
using Monitor.Measurement; using Monitor.Measurement;
using Monitor.Protocol;
namespace Monitor.Probe; namespace Monitor.Probe;
/// <summary> /// <summary>
/// Probe 的组装点:Agent 列表 → 每个 Agent 一个连接(认证、重连、角色)。没有全局状态,测试可以在同一进程里启动多个。 /// Probe 的组装点:到面板的长连接(身份、Agent 列表、状态)+ 每个 Agent 一个测量连接(认证、重连、角色)。没有全局状态,测试可以在同一进程里启动多个。
/// 列表变化时只调和连接:新增的建立,移除的断开,地址或证书指纹变化的重建,其余不受影响。 /// Agent 列表变化时只调和连接:新增的建立,移除的断开,地址或证书指纹变化的重建,其余不受影响。
/// </summary> /// </summary>
public sealed class ProbeApp : IDisposable public sealed class ProbeApp : IDisposable
{ {
public const string Version = "0.1.0"; public const string Version = Monitor.Protocol.BuildInfo.Version;
private readonly ConnectionEnv _env; private readonly ConnectionEnv _env;
private readonly Dictionary<string, Running> _connections = []; private readonly Dictionary<string, Running> _connections = [];
private readonly SemaphoreSlim _changed = new(0); private readonly SemaphoreSlim _changed = new(0);
private readonly INodeLog _log; private readonly INodeLog _log;
private readonly FleetLink _link;
public ProbeApp(ProbeFile file, string configDirectory, TimeProvider time, INodeLog log) public ProbeApp(ProbeFile file, string configDirectory, TimeProvider time, INodeLog log)
{ {
_log = log; _log = log;
var directory = new AgentDirectory(new ConfigStore(configDirectory), configDirectory, file, log); var directory = new AgentDirectory(new ConfigStore(configDirectory), configDirectory, log);
NodeIdentity identity = NodeIdentity.FromPkcs8(Convert.FromBase64String(file.PrivateKey)); NodeIdentity identity = NodeIdentity.FromPkcs8(Convert.FromBase64String(file.PrivateKey));
_env = new ConnectionEnv(identity, file.ProbeId, file.Limits, directory, new MonoClock(time), time, log); _env = new ConnectionEnv(identity, file.ProbeId, file.Limits, directory, new MonoClock(time), time, log);
directory.Changed += () => _changed.Release(); directory.Changed += () => _changed.Release();
var update = new UpdateBridge(configDirectory, new ConfigStore(configDirectory).Write, log);
var options = new FleetLinkOptions
{
Server = new Uri(file.Server),
Identity = identity,
NodeId = new NodeId(file.ProbeId),
NodeVersion = Version,
Capabilities = [MonitorCapabilities.Base],
HeartbeatExtra = () => JsonSerializer.SerializeToElement(new ProbeHeartbeatExtra { Update = update.ReadStatus() }, MonitorJsonContext.Default.ProbeHeartbeatExtra),
HelloMeta = () => JsonSerializer.SerializeToElement(new ProbeHelloMeta { Platform = PlatformDetector.Detect() }, MonitorJsonContext.Default.ProbeHelloMeta),
};
var list = new StateHandler<ProbeAgentList>(
ProbeAgentList.StateType,
MonitorJsonContext.Default.ProbeAgentList,
(version, doc, _) =>
{
directory.Apply(doc);
return ValueTask.FromResult(new StateSyncResult { AppliedVersion = version });
});
var router = new RpcRouter<LinkContext>()
.OnNotify(MonitorMethods.UpdateOffer, MonitorJsonContext.Default.UpdateOfferParams, (_, offer, _) =>
{
update.WriteOffer(offer);
return ValueTask.CompletedTask;
})
.OnNotify(MonitorMethods.UpdateApply, MonitorJsonContext.Default.UpdateOfferParams, (_, _, _) =>
{
update.RequestApply();
return ValueTask.CompletedTask;
});
_link = new FleetLink(options, router, [list], new Outbox(1 << 20), time, log);
} }
/// <summary>到面板的连接(测试和状态查看用)。</summary>
public FleetLink Link => _link;
public AgentDirectory Directory => _env.Directory; public AgentDirectory Directory => _env.Directory;
/// <summary>某个 Agent 的会话是否已经建立(测试和状态查看用)。</summary> /// <summary>某个 Agent 的会话是否已经建立(测试和状态查看用)。</summary>
@@ -50,9 +90,10 @@ public sealed class ProbeApp : IDisposable
{ {
if (_env.Directory.Agents.Count == 0) if (_env.Directory.Agents.Count == 0)
{ {
_log.Warn("no agents known yet: probe.json has an empty bootstrap list"); _log.Info("no agents known yet, waiting for the panel to send the list");
} }
Task link = _link.RunAsync(ct);
try try
{ {
while (!ct.IsCancellationRequested) while (!ct.IsCancellationRequested)
@@ -68,6 +109,7 @@ public sealed class ProbeApp : IDisposable
finally finally
{ {
await ReconcileAsync([], CancellationToken.None); await ReconcileAsync([], CancellationToken.None);
await Task.WhenAny(link, Task.Delay(TimeSpan.FromSeconds(5), CancellationToken.None));
} }
} }
@@ -77,7 +119,7 @@ public sealed class ProbeApp : IDisposable
_changed.Dispose(); _changed.Dispose();
} }
private async Task ReconcileAsync(IReadOnlyList<BootstrapAgent> agents, CancellationToken ct) private async Task ReconcileAsync(IReadOnlyList<KnownAgent> agents, CancellationToken ct)
{ {
var wanted = agents.ToDictionary(a => a.AgentId); var wanted = agents.ToDictionary(a => a.AgentId);
var stopped = new List<Running>(); var stopped = new List<Running>();
@@ -85,7 +127,7 @@ public sealed class ProbeApp : IDisposable
{ {
foreach ((string id, Running running) in _connections.ToList()) foreach ((string id, Running running) in _connections.ToList())
{ {
if (!wanted.TryGetValue(id, out BootstrapAgent? endpoint) || !AgentDirectory.Equal(endpoint, running.Connection.Endpoint)) if (!wanted.TryGetValue(id, out KnownAgent? endpoint) || !AgentDirectory.Equal(endpoint, running.Connection.Endpoint))
{ {
stopped.Add(running); stopped.Add(running);
_connections.Remove(id); _connections.Remove(id);
@@ -105,7 +147,7 @@ public sealed class ProbeApp : IDisposable
lock (_connections) lock (_connections)
{ {
foreach ((string id, BootstrapAgent endpoint) in wanted) foreach ((string id, KnownAgent endpoint) in wanted)
{ {
if (!_connections.ContainsKey(id)) if (!_connections.ContainsKey(id))
{ {
@@ -15,15 +15,15 @@ public sealed class ProbeFile
[JsonPropertyName("private_key")] [JsonPropertyName("private_key")]
public string PrivateKey { get; set; } = ""; public string PrivateKey { get; set; } = "";
/// <summary>首次连接用的 Agent;连上之后以 <c>agents.json</c>(最新的列表)为准。</summary> /// <summary>面板的地址(http 或 https)。Probe 直接连它:身份、Agent 列表、更新都从这里来。</summary>
[JsonPropertyName("bootstrap")] [JsonPropertyName("server")]
public List<BootstrapAgent> Bootstrap { get; set; } = []; public string Server { get; set; } = "";
[JsonPropertyName("limits")] [JsonPropertyName("limits")]
public ProbeLimits Limits { get; set; } = new(); public ProbeLimits Limits { get; set; } = new();
} }
public sealed class BootstrapAgent public sealed class KnownAgent
{ {
[JsonPropertyName("agent_id")] [JsonPropertyName("agent_id")]
public string AgentId { get; set; } = ""; public string AgentId { get; set; } = "";
@@ -54,12 +54,12 @@ public sealed class ProbeLimits
[JsonSerializable(typeof(AgentsCache))] [JsonSerializable(typeof(AgentsCache))]
public sealed partial class ProbeConfigContext : JsonSerializerContext; public sealed partial class ProbeConfigContext : JsonSerializerContext;
/// <summary><c>agents.json</c>:最近一次从 Agent 同步到的列表(配置,不是测量数据)。</summary> /// <summary><c>agents.json</c>:最近一次从面板同步到的 Agent 列表(配置,不是测量数据);面板连不上时 Probe 靠它继续测量。</summary>
public sealed class AgentsCache public sealed class AgentsCache
{ {
[JsonPropertyName("version")] [JsonPropertyName("version")]
public long Version { get; set; } public long Version { get; set; }
[JsonPropertyName("agents")] [JsonPropertyName("agents")]
public List<BootstrapAgent> Agents { get; set; } = []; public List<KnownAgent> Agents { get; set; } = [];
} }
+2 -2
View File
@@ -24,9 +24,9 @@ ProbeFile file;
try try
{ {
file = NodeConfig.Load(configPath, ProbeConfigContext.Default.ProbeFile); file = NodeConfig.Load(configPath, ProbeConfigContext.Default.ProbeFile);
if (file.ProbeId.Length == 0 || file.PrivateKey.Length == 0) if (file.ProbeId.Length == 0 || file.PrivateKey.Length == 0 || !Uri.TryCreate(file.Server, UriKind.Absolute, out _))
{ {
throw new InvalidOperationException("probe.json needs probe_id and private_key (download it from the panel: POST /api/admin/probe/identity)"); throw new InvalidOperationException("probe.json needs probe_id, private_key and server (download it from the panel: Probes → Create)");
} }
} }
catch (Exception ex) when (ex is IOException or InvalidDataException or InvalidOperationException or System.Text.Json.JsonException) catch (Exception ex) when (ex is IOException or InvalidDataException or InvalidOperationException or System.Text.Json.JsonException)
@@ -30,23 +30,22 @@ public sealed class ProbeTests
return path; return path;
} }
private static ProbeAgentsResult List(long version, params string[] ids) => new() private static ProbeAgentList List(long version, params string[] ids) => new()
{ {
Version = version, Version = version,
Agents = [.. ids.Select(id => new AgentEndpoint { AgentId = new NodeId(id), Host = id + ".example", TcpPort = 7443, Spki = "c3BraQ==" })], Agents = [.. ids.Select(id => new AgentEndpoint { AgentId = new NodeId(id), Host = id + ".example", TcpPort = 7443, Spki = "c3BraQ==" })],
}; };
[Fact] [Fact]
public void AgentDirectory_StartsFromBootstrap_IgnoresStaleLists_PersistsChanges_AndReloadsThem() public void AgentDirectory_StartsEmpty_IgnoresStaleLists_PersistsChanges_AndReloadsThem()
{ {
string directory = NewDirectory(); string directory = NewDirectory();
try try
{ {
var file = new ProbeFile { ProbeId = "prb_x", Bootstrap = [new BootstrapAgent { AgentId = "agt_boot", Host = "boot.example", TcpPort = 7443, Spki = "c3BraQ==" }] }; var directoryA = new AgentDirectory(new ConfigStore(directory), directory, NullNodeLog.Instance);
var directoryA = new AgentDirectory(new ConfigStore(directory), directory, file, NullNodeLog.Instance);
int changes = 0; int changes = 0;
directoryA.Changed += () => changes++; directoryA.Changed += () => changes++;
Assert.Equal(["agt_boot"], directoryA.Agents.Select(a => a.AgentId)); Assert.Empty(directoryA.Agents);
directoryA.Apply(List(5, "agt_1", "agt_2")); directoryA.Apply(List(5, "agt_1", "agt_2"));
Assert.Equal(1, changes); Assert.Equal(1, changes);
@@ -57,8 +56,8 @@ public sealed class ProbeTests
Assert.Equal(1, changes); Assert.Equal(1, changes);
Assert.Equal(5, directoryA.Version); Assert.Equal(5, directoryA.Version);
// The next start uses the saved list, not the bootstrap one // The next start uses the saved list while the panel is not reachable yet
var directoryB = new AgentDirectory(new ConfigStore(directory), directory, file, NullNodeLog.Instance); var directoryB = new AgentDirectory(new ConfigStore(directory), directory, NullNodeLog.Instance);
Assert.Equal(["agt_1", "agt_2"], directoryB.Agents.Select(a => a.AgentId)); Assert.Equal(["agt_1", "agt_2"], directoryB.Agents.Select(a => a.AgentId));
Assert.Equal(5, directoryB.Version); Assert.Equal(5, directoryB.Version);
} }
@@ -69,15 +68,14 @@ public sealed class ProbeTests
} }
[Fact] [Fact]
public void AgentDirectory_FallsBackToBootstrap_WhenTheSavedFileIsCorrupt() public void AgentDirectory_StartsEmpty_WhenTheSavedFileIsCorrupt()
{ {
string directory = NewDirectory(); string directory = NewDirectory();
try try
{ {
new ConfigStore(directory).Write(AgentDirectory.FileName, "{ not json"u8.ToArray()); new ConfigStore(directory).Write(AgentDirectory.FileName, "{ not json"u8.ToArray());
var file = new ProbeFile { Bootstrap = [new BootstrapAgent { AgentId = "agt_boot", Host = "h", TcpPort = 1, Spki = "AA==" }] }; var loaded = new AgentDirectory(new ConfigStore(directory), directory, NullNodeLog.Instance);
var loaded = new AgentDirectory(new ConfigStore(directory), directory, file, NullNodeLog.Instance); Assert.Empty(loaded.Agents);
Assert.Equal(["agt_boot"], loaded.Agents.Select(a => a.AgentId));
} }
finally finally
{ {
@@ -0,0 +1,81 @@
using System.Runtime.InteropServices;
using Monitor.Protocol;
namespace Monitor.Measurement;
/// <summary>
/// 读出本机的系统名和架构,只用于在面板上展示。故意只取粗粒度的信息:
/// 「Debian 12」「Windows 11」「x64」,不读主机名、内核版本、CPU 型号。
/// </summary>
public static class PlatformDetector
{
public static PlatformInfo Detect()
{
PlatformInfo info = Detect(File.Exists("/etc/os-release") ? File.ReadAllLines("/etc/os-release") : []);
info.Version = Empty(BuildInfo.Version);
info.Tag = Empty(BuildInfo.Tag);
info.Commit = Empty(BuildInfo.Commit);
info.BuiltAt = Empty(BuildInfo.BuiltAt);
return info;
}
private static string? Empty(string value) => value.Length == 0 ? null : value;
/// <summary>`osRelease` 是 /etc/os-release 的各行(Linux 之外不用)。拆出来是为了好测。</summary>
internal static PlatformInfo Detect(IEnumerable<string> osRelease)
{
string arch = RuntimeInformation.OSArchitecture switch
{
Architecture.X64 => "x64",
Architecture.Arm64 => "arm64",
Architecture.Arm => "arm",
Architecture.X86 => "x86",
var other => other.ToString().ToLowerInvariant(),
};
if (OperatingSystem.IsWindows())
{
// Windows 11 的版本号还是 10.0,靠内部版本号(22000 起)区分
string name = Environment.OSVersion.Version.Build >= 22000 ? "Windows 11" : "Windows 10";
return new PlatformInfo { Os = "windows", Name = name, Arch = arch };
}
if (OperatingSystem.IsMacOS())
{
return new PlatformInfo { Os = "macos", Name = "macOS", Arch = arch };
}
if (OperatingSystem.IsLinux())
{
return new PlatformInfo { Os = "linux", Name = LinuxName(osRelease), Arch = arch };
}
return new PlatformInfo { Os = "other", Name = null, Arch = arch };
}
/// <summary>从 os-release 里取发行版名字和主版本:「Debian GNU/Linux 12 (bookworm)」→「Debian 12」,「OpenWrt 24.10.0」→「OpenWrt 24.10」。</summary>
internal static string LinuxName(IEnumerable<string> osRelease)
{
var values = new Dictionary<string, string>(StringComparer.Ordinal);
foreach (string line in osRelease)
{
int eq = line.IndexOf('=', StringComparison.Ordinal);
if (eq > 0)
{
values[line[..eq].Trim()] = line[(eq + 1)..].Trim().Trim('"', '\'');
}
}
string name = values.GetValueOrDefault("NAME") ?? values.GetValueOrDefault("ID") ?? "Linux";
name = name.Replace(" GNU/Linux", string.Empty, StringComparison.Ordinal).Trim();
string? version = values.GetValueOrDefault("VERSION_ID");
if (string.IsNullOrEmpty(version))
{
return name;
}
// 只留 主.次:Alpine 的 3.22.1、OpenWrt 的 24.10.0 都截成两段;Debian 的 12 本来就一段
string[] parts = version.Split('.');
return $"{name} {string.Join('.', parts.Take(2))}";
}
}
@@ -0,0 +1,100 @@
using System.Text;
using System.Text.Json;
using Fleet.Core;
using Monitor.Protocol;
namespace Monitor.Measurement;
/// <summary>
/// 节点进程和宿主机更新器之间的文件交换(容器里 <c>/etc/monitor</c>,也就是数据目录):
/// 节点把面板发来的清单和签名写进 <c>update/</c>,把「更新」请求写成 <c>update/apply.request</c>;更新器验签并更新,状态写在 root 拥有的 <c>update-status/status.json</c>。
/// 节点进程不执行任何命令,也不信任清单的内容(验签在宿主机上)。没有安装更新器时目录写不了或没人读,调用只是返回 false。
/// </summary>
public sealed class UpdateBridge
{
private const int MaxManifestBytes = 4096;
private const int MaxSignatureBytes = 512;
private const int MaxStatusBytes = 4096;
private readonly string _updateDirectory;
private readonly string _statusFile;
private readonly Action<string, byte[]> _write;
private readonly INodeLog _log;
/// <param name="write">写数据目录里的文件(相对路径,原子替换):由节点传入 <c>ConfigStore.Write</c>——常驻进程里只有它可以写文件。</param>
public UpdateBridge(string dataDirectory, Action<string, byte[]> write, INodeLog log)
{
_updateDirectory = Path.Combine(dataDirectory, "update");
_statusFile = Path.Combine(dataDirectory, "update-status", "status.json");
_write = write;
_log = log;
}
/// <summary>放下面板发来的清单和签名(和已有的一样就不动,免得每次重连都惊动更新器)。先写清单、后写签名:更新器监视的是签名文件。</summary>
public bool WriteOffer(UpdateOfferParams offer)
{
byte[] manifest = Encoding.UTF8.GetBytes(offer.Manifest);
byte[] signature = Encoding.UTF8.GetBytes(offer.Signature);
if (manifest.Length is 0 or > MaxManifestBytes || signature.Length is 0 or > MaxSignatureBytes)
{
_log.Warn("ignoring an update offer of an unexpected size");
return false;
}
try
{
string manifestPath = Path.Combine(_updateDirectory, "offer.manifest");
string signaturePath = Path.Combine(_updateDirectory, "offer.sig");
if (Same(manifestPath, manifest) && Same(signaturePath, signature))
{
return true;
}
_write("update/offer.manifest", manifest);
_write("update/offer.sig", signature);
_log.Info("update offer received");
return true;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_log.Warn("could not store the update offer (is the updater installed?)", ("err", ex.Message));
return false;
}
}
/// <summary>请求更新:写一个空的 <c>apply.request</c>(更新器验签通过、清单更新才会真的动手)。</summary>
public bool RequestApply()
{
try
{
_write("update/apply.request", []);
_log.Info("update requested");
return true;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_log.Warn("could not request the update (is the updater installed?)", ("err", ex.Message));
return false;
}
}
/// <summary>更新器的状态;没有安装更新器、文件不存在或内容不对时返回 null。</summary>
public UpdateStatus? ReadStatus()
{
try
{
if (!File.Exists(_statusFile) || new FileInfo(_statusFile).Length > MaxStatusBytes)
{
return null;
}
return JsonSerializer.Deserialize(File.ReadAllBytes(_statusFile), MonitorJsonContext.Default.UpdateStatus);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException)
{
return null;
}
}
private static bool Same(string path, byte[] content) => File.Exists(path) && File.ReadAllBytes(path).AsSpan().SequenceEqual(content);
}
@@ -6,4 +6,30 @@
<ProjectReference Include="..\..\fleet\src\Fleet.Core\Fleet.Core.csproj" /> <ProjectReference Include="..\..\fleet\src\Fleet.Core\Fleet.Core.csproj" />
<ProjectReference Include="..\..\fleet\src\Fleet.Rpc\Fleet.Rpc.csproj" /> <ProjectReference Include="..\..\fleet\src\Fleet.Rpc\Fleet.Rpc.csproj" />
</ItemGroup> </ItemGroup>
<!--
构建信息:版本来自 $(Version),tag、提交、构建时间来自环境变量 MONITOR_TAG / MONITOR_COMMIT / MONITOR_BUILT_AT
(scripts/publish.sh 和 CI 会设置;本地开发不设置就是空,这样本地构建是可重复的、不会每次都重新编译)。
生成一个带常量的源文件,NativeAOT 下不用反射。
-->
<Target Name="WriteBuildInfo" BeforeTargets="CoreCompile">
<PropertyGroup>
<BuildInfoFile>$(IntermediateOutputPath)BuildInfo.g.cs</BuildInfoFile>
</PropertyGroup>
<ItemGroup>
<BuildInfoLine Include="namespace Monitor.Protocol%3B" />
<BuildInfoLine Include="/// &lt;summary&gt;这个程序是什么时候、从哪个版本构建出来的(构建时生成)。&lt;/summary&gt;" />
<BuildInfoLine Include="public static class BuildInfo" />
<BuildInfoLine Include="{" />
<BuildInfoLine Include=" public const string Version = &quot;$(Version)&quot;%3B" />
<BuildInfoLine Include=" public const string Tag = &quot;$(MONITOR_TAG)&quot;%3B" />
<BuildInfoLine Include=" public const string Commit = &quot;$(MONITOR_COMMIT)&quot;%3B" />
<BuildInfoLine Include=" public const string BuiltAt = &quot;$(MONITOR_BUILT_AT)&quot;%3B" />
<BuildInfoLine Include="}" />
</ItemGroup>
<WriteLinesToFile File="$(BuildInfoFile)" Lines="@(BuildInfoLine)" Overwrite="true" WriteOnlyWhenDifferent="true" />
<ItemGroup>
<Compile Include="$(BuildInfoFile)" />
<FileWrites Include="$(BuildInfoFile)" />
</ItemGroup>
</Target>
</Project> </Project>
+117 -11
View File
@@ -13,8 +13,8 @@ public static class MonitorMethods
public const string TestStatus = "monitor.test.status"; public const string TestStatus = "monitor.test.status";
public const string Event = "monitor.event"; public const string Event = "monitor.event";
public const string Host = "monitor.host"; public const string Host = "monitor.host";
public const string ProbeAgents = "monitor.probe.agents"; public const string UpdateOffer = "monitor.update.offer";
public const string ProbeAgentsChanged = "monitor.probe.agents_changed"; public const string UpdateApply = "monitor.update.apply";
// Probe ↔ Agent(点对点会话) // Probe ↔ Agent(点对点会话)
public const string Clock = "monitor.clock"; public const string Clock = "monitor.clock";
@@ -147,15 +147,11 @@ public sealed class AgentEndpoint
public string Spki { get; set; } = ""; public string Spki { get; set; } = "";
} }
public sealed class ProbeAgentsParams /// <summary>期望状态文档(Fleet state type = <c>probe_agents</c>):Backend 直接下发给每个 Probe 的 Agent 列表。</summary>
public sealed class ProbeAgentList
{ {
/// <summary>Probe 目前持有的列表版本(用于调试;服务端总是返回当前列表)。</summary> public const string StateType = "probe_agents";
[JsonPropertyName("have_version")]
public long HaveVersion { get; set; }
}
public sealed class ProbeAgentsResult
{
[JsonPropertyName("version")] [JsonPropertyName("version")]
public long Version { get; set; } public long Version { get; set; }
@@ -194,9 +190,108 @@ public sealed class LocalLimits
public int MaxConcurrentTests { get; set; } = 1; public int MaxConcurrentTests { get; set; } = 1;
} }
/// <summary>
/// 节点的系统和架构(粗粒度,只用于展示):<c>os</c> 是 linux / windows / macos / other,<c>name</c> 是「Debian 12」「Windows 11」这样的名字,
/// <c>arch</c> 是 x64 / arm64 / arm / x86。<b>没有主机名、内核版本、具体型号</b>。另外带着这个程序自己的构建信息(版本、tag、构建时间)。
/// </summary>
public sealed class PlatformInfo
{
[JsonPropertyName("os")]
public string Os { get; set; } = "other";
[JsonPropertyName("name")]
public string? Name { get; set; }
[JsonPropertyName("arch")]
public string? Arch { get; set; }
/// <summary>程序的版本、tag、提交(短)和构建时间(UTC,ISO 8601):<see cref="BuildInfo"/> 构建时写进去的,没设置就是空。</summary>
[JsonPropertyName("version")]
public string? Version { get; set; }
[JsonPropertyName("tag")]
public string? Tag { get; set; }
[JsonPropertyName("commit")]
public string? Commit { get; set; }
[JsonPropertyName("built_at")]
public string? BuiltAt { get; set; }
}
/// <summary>
/// 面板给节点的更新清单(<c>monitor.update.offer</c>):<c>manifest</c> 是发布清单的原始文本,<c>signature</c> 是 base64 的签名。
/// 节点只是把它们原样放进数据目录;验签、拉取、重启都由宿主机上的更新器完成(节点进程不执行任何命令,也不信任清单的内容)。
/// </summary>
public sealed class UpdateOfferParams
{
[JsonPropertyName("manifest")]
public string Manifest { get; set; } = "";
[JsonPropertyName("signature")]
public string Signature { get; set; } = "";
}
/// <summary>宿主机更新器已经验证过的可用更新。</summary>
public sealed class UpdateAvailable
{
[JsonPropertyName("version")]
public string? Version { get; set; }
[JsonPropertyName("tag")]
public string? Tag { get; set; }
[JsonPropertyName("digest")]
public string? Digest { get; set; }
[JsonPropertyName("seq")]
public long Seq { get; set; }
}
/// <summary>
/// 宿主机更新器的状态(它写在 <c>update-status/status.json</c>,节点读出来随心跳上报):
/// <c>up_to_date</c>、<c>available</c>、<c>applying</c>、<c>failed</c>、<c>rolled_back</c>、<c>invalid</c>(清单验签或校验没过)。
/// 没有安装更新器的节点没有这个字段。
/// </summary>
public sealed class UpdateStatus
{
[JsonPropertyName("state")]
public string State { get; set; } = "";
/// <summary>正在运行的镜像 digest(更新器记录的;还没更新过时为空)。</summary>
[JsonPropertyName("current")]
public string? Current { get; set; }
[JsonPropertyName("available")]
public UpdateAvailable? Available { get; set; }
[JsonPropertyName("error")]
public string? Error { get; set; }
[JsonPropertyName("at")]
public long At { get; set; }
}
/// <summary>Probe 的 <c>fleet.heartbeat</c> 附加字段:更新器的状态。</summary>
public sealed class ProbeHeartbeatExtra
{
[JsonPropertyName("update")]
public UpdateStatus? Update { get; set; }
}
/// <summary>Probe 的 <c>fleet.hello</c> 附加字段:系统、架构和构建信息(只用于展示和更新)。</summary>
public sealed class ProbeHelloMeta
{
[JsonPropertyName("platform")]
public PlatformInfo? Platform { get; set; }
}
/// <summary><c>fleet.hello</c> 的附加字段。</summary> /// <summary><c>fleet.hello</c> 的附加字段。</summary>
public sealed class AgentHelloMeta public sealed class AgentHelloMeta
{ {
[JsonPropertyName("platform")]
public PlatformInfo? Platform { get; set; }
[JsonPropertyName("tls_spki")] [JsonPropertyName("tls_spki")]
public string? TlsSpki { get; set; } public string? TlsSpki { get; set; }
@@ -244,6 +339,10 @@ public sealed class HostSnapshot
[JsonPropertyName("net_tx_bps")] [JsonPropertyName("net_tx_bps")]
public long? NetTxBps { get; set; } public long? NetTxBps { get; set; }
/// <summary>已统计接口的内核报告链路速率之和(Mbit/s);驱动不支持或无法读取时为空,不代表互联网带宽。</summary>
[JsonPropertyName("net_link_mbps")]
public long? NetLinkMbps { get; set; }
[JsonPropertyName("uptime_s")] [JsonPropertyName("uptime_s")]
public long? UptimeSeconds { get; set; } public long? UptimeSeconds { get; set; }
@@ -293,6 +392,9 @@ public sealed class AgentHeartbeatExtra
[JsonPropertyName("queue_bytes")] [JsonPropertyName("queue_bytes")]
public long QueueBytes { get; set; } public long QueueBytes { get; set; }
[JsonPropertyName("update")]
public UpdateStatus? Update { get; set; }
} }
/// <summary>Agent 下发给 Probe 的一条线路任务(期望状态类型 <c>line_task_set</c> 里的一项)。</summary> /// <summary>Agent 下发给 Probe 的一条线路任务(期望状态类型 <c>line_task_set</c> 里的一项)。</summary>
@@ -431,9 +533,13 @@ public static class FragmentKeys
[JsonSerializable(typeof(HostSnapshot))] [JsonSerializable(typeof(HostSnapshot))]
[JsonSerializable(typeof(TestStatusParams))] [JsonSerializable(typeof(TestStatusParams))]
[JsonSerializable(typeof(AgentEventParams))] [JsonSerializable(typeof(AgentEventParams))]
[JsonSerializable(typeof(ProbeAgentsParams))] [JsonSerializable(typeof(ProbeAgentList))]
[JsonSerializable(typeof(ProbeAgentsResult))] [JsonSerializable(typeof(UpdateOfferParams))]
[JsonSerializable(typeof(UpdateStatus))]
[JsonSerializable(typeof(ProbeHeartbeatExtra))]
[JsonSerializable(typeof(ProbeHelloMeta))]
[JsonSerializable(typeof(AgentEnrollMeta))] [JsonSerializable(typeof(AgentEnrollMeta))]
[JsonSerializable(typeof(PlatformInfo))]
[JsonSerializable(typeof(AgentHelloMeta))] [JsonSerializable(typeof(AgentHelloMeta))]
[JsonSerializable(typeof(AgentHeartbeatExtra))] [JsonSerializable(typeof(AgentHeartbeatExtra))]
[JsonSerializable(typeof(RawChunkHeader))] [JsonSerializable(typeof(RawChunkHeader))]
@@ -56,9 +56,6 @@ public sealed class TaskSet
/// <summary>允许连接到这个 Agent 的 Probe 公钥。</summary> /// <summary>允许连接到这个 Agent 的 Probe 公钥。</summary>
[JsonPropertyName("peers")] [JsonPropertyName("peers")]
public List<PeerSpec> Peers { get; set; } = []; public List<PeerSpec> Peers { get; set; } = [];
[JsonPropertyName("probe_agents_version")]
public long ProbeAgentsVersion { get; set; }
} }
public static class TaskSchedule public static class TaskSchedule
@@ -0,0 +1,27 @@
namespace Monitor.Measurement.Tests;
public sealed class PlatformDetectorTests
{
[Theory]
[InlineData("PRETTY_NAME=\"Debian GNU/Linux 12 (bookworm)\"|NAME=\"Debian GNU/Linux\"|VERSION_ID=\"12\"|ID=debian", "Debian 12")]
[InlineData("NAME=\"OpenWrt\"|VERSION_ID=\"24.10.0\"|ID=openwrt", "OpenWrt 24.10")]
[InlineData("NAME=\"Alpine Linux\"|ID=alpine|VERSION_ID=3.22.1", "Alpine Linux 3.22")]
[InlineData("NAME=\"Ubuntu\"|VERSION_ID=\"24.04\"", "Ubuntu 24.04")]
[InlineData("ID=arch", "arch")]
[InlineData("", "Linux")]
public void LinuxName_IsTheDistroAndItsMajorMinorVersion(string osRelease, string expected)
{
Assert.Equal(expected, PlatformDetector.LinuxName(osRelease.Split('|', StringSplitOptions.RemoveEmptyEntries)));
}
[Fact]
public void Detect_ReportsOnlyCoarseInformation()
{
var platform = PlatformDetector.Detect();
Assert.Contains(platform.Os, new[] { "linux", "windows", "macos", "other" });
Assert.Contains(platform.Arch, new[] { "x64", "arm64", "arm", "x86" });
// 没有主机名、内核版本:名字很短,也不含小数点以外的版本细节
Assert.True((platform.Name ?? string.Empty).Length < 40);
Assert.DoesNotContain(Environment.MachineName, platform.Name ?? string.Empty, StringComparison.OrdinalIgnoreCase);
}
}
@@ -43,7 +43,7 @@ public static class AgentEndpoints
.Api("join_tokens_create", "Create a join token; the plaintext is returned only once", TokensTag) .Api("join_tokens_create", "Create a join token; the plaintext is returned only once", TokensTag)
.Returns<CreatedJoinTokenDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403); .Returns<CreatedJoinTokenDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403);
group.MapDelete("/join-tokens/{id}", RevokeTokenAsync) group.MapDelete("/join-tokens/{id}", RevokeTokenAsync)
.Api("join_tokens_revoke", "Revoke a join token", TokensTag) .Api("join_tokens_revoke", "Revoke a join token; with purge=true delete the record for good", TokensTag)
.Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403); .Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403);
group.MapGet("/probes", ListProbesAsync) group.MapGet("/probes", ListProbesAsync)
@@ -130,21 +130,29 @@ public static class AgentEndpoints
http.Response.Headers.CacheControl = "no-store"; http.Response.Headers.CacheControl = "no-store";
http.Response.Headers.Location = "/api/admin/join-tokens"; http.Response.Headers.Location = "/api/admin/join-tokens";
return TypedResults.Json(new CreatedJoinTokenDto(Mappers.ToDto(created.Info), created.Plaintext, created.InstallCommand), ApiJsonContext.Default.CreatedJoinTokenDto, statusCode: StatusCodes.Status201Created); return TypedResults.Json(new CreatedJoinTokenDto(Mappers.ToDto(created.Info), created.Plaintext, new InstallCommandsDto(created.InstallCommands.Linux, created.InstallCommands.Macos, created.InstallCommands.Windows)), ApiJsonContext.Default.CreatedJoinTokenDto, statusCode: StatusCodes.Status201Created);
} }
private static async Task<IResult> RevokeTokenAsync(JoinTokenService tokens, string id, CancellationToken ct) private static async Task<IResult> RevokeTokenAsync(JoinTokenService tokens, string id, bool? purge, CancellationToken ct)
{ {
await tokens.RevokeAsync(id, ct); if (purge == true)
{
await tokens.DeleteAsync(id, ct);
}
else
{
await tokens.RevokeAsync(id, ct);
}
return TypedResults.NoContent(); return TypedResults.NoContent();
} }
private static async Task<IResult> ListProbesAsync(ProbeService probes, CancellationToken ct) => private static async Task<IResult> ListProbesAsync(ProbeService probes, CancellationToken ct) =>
TypedResults.Json(new PageDto<ProbeDto>([.. (await probes.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoProbeDto); TypedResults.Json(new PageDto<ProbeDto>([.. (await probes.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoProbeDto);
private static async Task<IResult> CreateProbeAsync(HttpContext http, ProbeService probes, CreateProbeRequest? request, CancellationToken ct) private static async Task<IResult> CreateProbeAsync(HttpContext http, ProbeService probes, BackendOptions options, CreateProbeRequest? request, CancellationToken ct)
{ {
Result<ProbeIdentityFile> result = await probes.CreateAsync(request?.Name, ct); Result<ProbeIdentityFile> result = await probes.CreateAsync(request?.Name, PanelUrl(http, options), ct);
if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error)) if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error))
{ {
return Problems.From(http, error); return Problems.From(http, error);
@@ -171,9 +179,9 @@ public static class AgentEndpoints
return result.TryGet(out ProbeView? view, out FleetError? error) ? TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto) : Problems.From(http, error); return result.TryGet(out ProbeView? view, out FleetError? error) ? TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto) : Problems.From(http, error);
} }
private static async Task<IResult> RotateProbeAsync(HttpContext http, ProbeService probes, string id, CancellationToken ct) private static async Task<IResult> RotateProbeAsync(HttpContext http, ProbeService probes, BackendOptions options, string id, CancellationToken ct)
{ {
Result<ProbeIdentityFile> result = await probes.RotateAsync(id, ct); Result<ProbeIdentityFile> result = await probes.RotateAsync(id, PanelUrl(http, options), ct);
if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error)) if (!result.TryGet(out ProbeIdentityFile? file, out FleetError? error))
{ {
return Problems.From(http, error); return Problems.From(http, error);
@@ -206,8 +214,9 @@ public static class AgentEndpoints
return dropped ? TypedResults.NoContent() : Problems.Create(http, MonitorErrors.ProbeOffline, "this probe has no session with this agent"); return dropped ? TypedResults.NoContent() : Problems.Create(http, MonitorErrors.ProbeOffline, "this probe has no session with this agent");
} }
/// <summary>Probe 要连的面板地址:配置的公开地址,没有就用这次请求的地址。</summary>
private static string PanelUrl(HttpContext http, BackendOptions options) => (options.PublicUrl ?? $"{http.Request.Scheme}://{http.Request.Host}").TrimEnd('/');
private static ProbeIdentityDto ToIdentityDto(ProbeIdentityFile file) => private static ProbeIdentityDto ToIdentityDto(ProbeIdentityFile file) =>
new( new(file.ProbeId.Value, file.PrivateKey, file.Server, new ProbeLimitsDto(file.Limits.MaxTestMbps, file.Limits.MaxTotalPps));
file.ProbeId.Value, file.PrivateKey, [.. file.Bootstrap.Select(b => new BootstrapAgentDto(b.AgentId.Value, b.Host, b.TcpPort, b.Spki))],
new ProbeLimitsDto(file.Limits.MaxTestMbps, file.Limits.MaxTotalPps));
} }
@@ -44,24 +44,35 @@ public static class AuthEndpoints
return Problems.From(http, error); return Problems.From(http, error);
} }
http.Response.Cookies.Append( // 跨域的前端(return_session)不用 Cookie:会话令牌放在响应里,之后用 Bearer 带上
PrincipalResolver.SessionCookie, bool returnSession = request.ReturnSession == true;
login.SessionToken, if (!returnSession)
new CookieOptions {
{ http.Response.Cookies.Append(
HttpOnly = true, PrincipalResolver.SessionCookie,
Secure = options.SecureCookies, login.SessionToken,
SameSite = SameSiteMode.Strict, new CookieOptions
Path = "/", {
Expires = DateTimeOffset.FromUnixTimeMilliseconds(login.ExpiresAtMs), HttpOnly = true,
}); Secure = options.SecureCookies,
SameSite = SameSiteMode.Strict,
Path = "/",
Expires = DateTimeOffset.FromUnixTimeMilliseconds(login.ExpiresAtMs),
});
}
http.Response.Headers.CacheControl = "no-store"; http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new LoginResponse(login.UserId, login.Username, login.CsrfToken, Mappers.Time(login.ExpiresAtMs)), ApiJsonContext.Default.LoginResponse); return TypedResults.Json(new LoginResponse(login.UserId, login.Username, login.CsrfToken, Mappers.Time(login.ExpiresAtMs), returnSession ? login.SessionToken : null), ApiJsonContext.Default.LoginResponse);
} }
private static async Task<IResult> LogoutAsync(HttpContext http, AuthService auth) private static async Task<IResult> LogoutAsync(HttpContext http, AuthService auth)
{ {
if (http.Request.Cookies.TryGetValue(PrincipalResolver.SessionCookie, out string? session)) string? bearer = PrincipalResolver.BearerOf(http);
if (bearer is not null)
{
await auth.LogoutAsync(bearer, http.RequestAborted); // 登录时 return_session 拿到的会话令牌;API 令牌不是会话,这里什么也不会发生
}
else if (http.Request.Cookies.TryGetValue(PrincipalResolver.SessionCookie, out string? session))
{ {
await auth.LogoutAsync(session, http.RequestAborted); await auth.LogoutAsync(session, http.RequestAborted);
} }
@@ -0,0 +1,63 @@
using Fleet.Core;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Api;
/// <summary>构建信息与更新:后端和每个节点的版本、tag、构建时间,CI 签名的发布清单,以及「更新这个节点」的请求。更新本身由节点宿主机上的更新器验签并执行。</summary>
public static class BuildEndpoints
{
private const string Tag = "admin-build";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/build-info", GetAsync)
.Api("build_info_get", "Version, tag, commit, build time and update state of the backend and every agent and probe", Tag)
.Returns<BuildInfoDto>().ProducesProblems(401, 403);
group.MapGet("/releases", ListReleasesAsync)
.Api("releases_list", "The latest signed release manifest of each component", Tag).Returns<PageDto<ReleaseDto>>().ProducesProblems(401, 403);
group.MapPost("/nodes/{id}/update", RequestUpdateAsync)
.Api("nodes_update", "Ask an online agent or probe to install the release it has been offered", Tag).Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403, 409);
}
/// <summary>登记发布清单:挂在 <see cref="ReleaseGuard"/> 下,除了管理员,CI 用的 <c>release</c> 令牌也能调用(它只能做这一件事)。</summary>
public static void MapPublish(RouteGroupBuilder group)
{
group.MapPost("/releases", PublishReleaseAsync)
.Api("releases_publish", "Register a CI-signed release manifest; online nodes get it right away (their updaters verify the signature). Admin or release tokens", Tag)
.Returns<ReleaseDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403, 409);
}
private static async Task<IResult> GetAsync(HttpContext http, NodeInfoService nodes, ReleaseService releases, CancellationToken ct)
{
var items = (await nodes.ListAsync(ct))
.OrderBy(n => n.Kind, StringComparer.Ordinal)
.ThenBy(n => n.NodeId, StringComparer.Ordinal)
.Select(n => new NodeBuildDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Version, n.Tag, n.Commit, n.BuiltAt, ToDto(releases.StatusOf(new NodeId(n.NodeId)))))
.ToList();
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new BuildInfoDto(NodeEndpoints.Backend(withCommit: true), items), ApiJsonContext.Default.BuildInfoDto);
}
private static async Task<IResult> ListReleasesAsync(ReleaseService releases, CancellationToken ct) =>
TypedResults.Json(new PageDto<ReleaseDto>([.. (await releases.ListAsync(ct)).Select(ToDto)], null), ApiJsonContext.Default.PageDtoReleaseDto);
private static async Task<IResult> PublishReleaseAsync(HttpContext http, ReleaseService releases, PublishReleaseRequest request)
{
Result<ReleaseOffer> result = await releases.PublishAsync(request.Manifest ?? "", request.Signature ?? "", http.RequestAborted);
return result.TryGet(out ReleaseOffer? offer, out FleetError? error)
? TypedResults.Json(ToDto(offer), ApiJsonContext.Default.ReleaseDto, statusCode: StatusCodes.Status201Created)
: Problems.From(http, error);
}
private static async Task<IResult> RequestUpdateAsync(HttpContext http, ReleaseService releases, string id, CancellationToken ct) =>
await releases.RequestUpdateAsync(new NodeId(id), ct) ? TypedResults.NoContent() : Problems.Create(http, MonitorErrors.AgentOffline, "the node is not connected");
private static ReleaseDto ToDto(ReleaseOffer offer) => new(offer.Component, offer.Seq, offer.Tag, offer.Version, offer.Digest);
private static NodeUpdateDto? ToDto(UpdateStatus? status) =>
status is null ? null : new NodeUpdateDto(status.State, status.Current, status.Available?.Tag, status.Available?.Version, status.Error, status.At > 0 ? DateTimeOffset.FromUnixTimeSeconds(status.At) : null);
}
@@ -0,0 +1,59 @@
using Fleet.Core;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Api;
/// <summary>线路开关:哪些(Probe,Agent)要连。自动连接的总开关是设置 <c>fleet.auto_link</c>;这里看每一对的结果,并且手动强制开、关或取消手动设置。</summary>
public static class LinkEndpoints
{
private const string Tag = "admin-links";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/links", ListAsync)
.Api("links_list", "Every probe/agent pair: whether it is linked, the manual override and whether it is connected", Tag).Returns<LinkMatrixDto>().ProducesProblems(401, 403);
group.MapPut("/links/{probe_id}/{agent_id}", SetAsync)
.Api("links_set", "Force a link on or off, or give it back to the automatic setting (enabled: null)", Tag).Returns<LinkDto>().ProducesProblems(400, 401, 403, 404);
}
private static async Task<IResult> ListAsync(AgentService agents, ProbeService probes, LinkService links, CancellationToken ct)
{
LinkSnapshot snapshot = await links.SnapshotAsync(ct);
IReadOnlyList<AgentView> agentViews = await agents.ListAsync(ct);
var items = new List<LinkDto>();
foreach (ProbeView probe in (await probes.ListAsync(ct)).Where(p => !p.Revoked).OrderBy(p => p.Probe.Name, StringComparer.Ordinal))
{
foreach (AgentView agent in agentViews.OrderBy(a => a.Agent.Name, StringComparer.Ordinal))
{
items.Add(ToDto(snapshot, probe, agent));
}
}
return TypedResults.Json(new LinkMatrixDto(snapshot.Auto, items), ApiJsonContext.Default.LinkMatrixDto);
}
private static async Task<IResult> SetAsync(HttpContext http, AgentService agents, ProbeService probes, LinkService links, string probe_id, string agent_id, SetLinkRequest request, CancellationToken ct)
{
if (await probes.GetAsync(probe_id, ct) is not { Revoked: false } probe)
{
return Problems.Create(http, MonitorErrors.ProbeNotFound, "probe not found");
}
if (await agents.GetAsync(new NodeId(agent_id), ct) is not { } agent)
{
return Problems.Create(http, MonitorErrors.AgentNotFound, "agent not found");
}
await links.SetAsync(probe_id, agent_id, request.Enabled, ct);
await agents.AfterAgentSetChangedAsync(ct); // 重新下发:Probe 的 Agent 列表、Agent 允许的 Probe
return TypedResults.Json(ToDto(await links.SnapshotAsync(ct), probe, agent), ApiJsonContext.Default.LinkDto);
}
private static LinkDto ToDto(LinkSnapshot snapshot, ProbeView probe, AgentView agent) =>
new(probe.Probe.Id, probe.Probe.Name, agent.Agent.Id.Value, agent.Agent.Name, snapshot.IsLinked(probe.Probe.Id, agent.Agent.Id.Value), snapshot.OverrideOf(probe.Probe.Id, agent.Agent.Id.Value),
agent.Probes.Any(p => p.Id == probe.Probe.Id));
}
@@ -63,9 +63,14 @@ public static class SystemEndpoints
private static async Task<IResult> ListSettingsAsync(SettingsService settings, CancellationToken ct) => private static async Task<IResult> ListSettingsAsync(SettingsService settings, CancellationToken ct) =>
TypedResults.Json(new PageDto<SettingDto>([.. (await settings.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto); TypedResults.Json(new PageDto<SettingDto>([.. (await settings.ListAsync(ct)).Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto);
private static async Task<IResult> PatchSettingsAsync(HttpContext http, SettingsService settings, Dictionary<string, JsonElement> changes) private static async Task<IResult> PatchSettingsAsync(HttpContext http, SettingsService settings, AgentService agents, Dictionary<string, JsonElement> changes)
{ {
Result<IReadOnlyList<SettingView>> result = await settings.PatchAsync(changes, http.RequestAborted); Result<IReadOnlyList<SettingView>> result = await settings.PatchAsync(changes, http.RequestAborted);
if (result.IsSuccess && changes.ContainsKey(SettingsService.AutoLink))
{
await agents.AfterAgentSetChangedAsync(http.RequestAborted); // 自动连接开关变了:重新下发各 Probe 的 Agent 列表和各 Agent 允许的 Probe
}
return result.TryGet(out IReadOnlyList<SettingView>? views, out FleetError? error) return result.TryGet(out IReadOnlyList<SettingView>? views, out FleetError? error)
? TypedResults.Json(new PageDto<SettingDto>([.. views.Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto) ? TypedResults.Json(new PageDto<SettingDto>([.. views.Select(Mappers.ToDto)], null), ApiJsonContext.Default.PageDtoSettingDto)
: Problems.From(http, error); : Problems.From(http, error);
@@ -97,17 +97,22 @@ public static class ApiModule
QueryEndpoints.Map(publicGroup); QueryEndpoints.Map(publicGroup);
LiveEndpoints.Map(publicGroup); LiveEndpoints.Map(publicGroup);
HostEndpoints.Map(publicGroup); HostEndpoints.Map(publicGroup);
NodeEndpoints.Map(publicGroup);
CatalogEndpoints.Map(publicGroup); CatalogEndpoints.Map(publicGroup);
RouteGroupBuilder admin = app.MapGroup("/api/admin"); RouteGroupBuilder admin = app.MapGroup("/api/admin");
AuthEndpoints.MapLogin(admin); AuthEndpoints.MapLogin(admin);
BuildEndpoints.MapPublish(admin.MapGroup(string.Empty).AddEndpointFilter(ReleaseGuard.InvokeAsync));
RouteGroupBuilder guarded = admin.MapGroup(string.Empty).AddEndpointFilter(AdminGuard.InvokeAsync); RouteGroupBuilder guarded = admin.MapGroup(string.Empty).AddEndpointFilter(AdminGuard.InvokeAsync);
AuthEndpoints.MapSession(guarded); AuthEndpoints.MapSession(guarded);
AgentEndpoints.Map(guarded); AgentEndpoints.Map(guarded);
TaskEndpoints.Map(guarded); TaskEndpoints.Map(guarded);
DataEndpoints.Map(guarded); DataEndpoints.Map(guarded);
SystemEndpoints.Map(guarded); SystemEndpoints.Map(guarded);
BuildEndpoints.Map(guarded);
LinkEndpoints.Map(guarded);
OpsEndpoints.Map(app); OpsEndpoints.Map(app);
app.MapOpenApi("/openapi/v1.json"); app.MapOpenApi("/openapi/v1.json");
@@ -23,14 +23,25 @@ public static class PrincipalResolver
public const string SessionCookie = "monitor_session"; public const string SessionCookie = "monitor_session";
public const string CsrfHeader = "X-CSRF-Token"; public const string CsrfHeader = "X-CSRF-Token";
public static string? BearerOf(HttpContext http)
{
string? header = http.Request.Headers.Authorization;
return header is not null && header.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase) ? header["Bearer ".Length..].Trim() : null;
}
public static async Task ResolveAsync(HttpContext http, Func<Task> next) public static async Task ResolveAsync(HttpContext http, Func<Task> next)
{ {
AuthService auth = http.RequestServices.GetRequiredService<AuthService>(); AuthService auth = http.RequestServices.GetRequiredService<AuthService>();
Principal? principal = null; Principal? principal = null;
string? header = http.Request.Headers.Authorization; string? bearer = BearerOf(http);
if (header is not null && header.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase)) if (bearer is not null)
{ {
principal = await auth.AuthenticateTokenAsync(header["Bearer ".Length..].Trim(), http.RequestAborted); principal = await auth.AuthenticateTokenAsync(bearer, http.RequestAborted);
if (principal is null && await auth.AuthenticateSessionAsync(bearer, http.RequestAborted) is { } viaSession)
{
// 登录时用 return_session 拿到的会话令牌:和 API 令牌一样走 Bearer,不是 Cookie,所以不需要 CSRF
principal = viaSession with { Kind = PrincipalKind.Token, CsrfToken = null };
}
} }
else if (http.Request.Cookies.TryGetValue(SessionCookie, out string? session) && !string.IsNullOrEmpty(session)) else if (http.Request.Cookies.TryGetValue(SessionCookie, out string? session) && !string.IsNullOrEmpty(session))
{ {
@@ -51,7 +62,7 @@ public static class AdminGuard
Principal principal = http.GetPrincipal(); Principal principal = http.GetPrincipal();
if (!principal.IsAdmin) if (!principal.IsAdmin)
{ {
return principal.CanRead return principal.Scope != ApiScope.None
? Problems.Create(http, MonitorErrors.Forbidden, "the token scope does not allow this operation") ? Problems.Create(http, MonitorErrors.Forbidden, "the token scope does not allow this operation")
: Problems.Create(http, MonitorErrors.Unauthorized, "authentication is required"); : Problems.Create(http, MonitorErrors.Unauthorized, "authentication is required");
} }
@@ -69,6 +80,13 @@ public static class AdminGuard
} }
} }
/// <summary>登记发布清单的守卫:管理员(会话要带 CSRF)或 release 令牌;release 令牌只能做这一件事。</summary>
public static class ReleaseGuard
{
public static async ValueTask<object?> InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next) =>
context.HttpContext.GetPrincipal().Scope == ApiScope.Release ? await next(context) : await AdminGuard.InvokeAsync(context, next);
}
/// <summary>展示层守卫:站点公开时匿名可读;私有时需要登录或只读令牌。</summary> /// <summary>展示层守卫:站点公开时匿名可读;私有时需要登录或只读令牌。</summary>
public static class PublicGuard public static class PublicGuard
{ {
@@ -34,13 +34,13 @@ public static class Mappers
line.Line.Agent.Online, line.Line.Connected, line.Line.Agent.EndpointChanges, ToDto(line.Summary), Summary(line.LastTest)); line.Line.Agent.Online, line.Line.Connected, line.Line.Agent.EndpointChanges, ToDto(line.Summary), Summary(line.LastTest));
public static HostDto ToDto(HostSnapshot h) => public static HostDto ToDto(HostSnapshot h) =>
new(h.CpuPercent, h.MemUsed, h.MemTotal, h.DiskUsed, h.DiskTotal, h.NetRxTotal, h.NetTxTotal, h.NetRxBps, h.NetTxBps, h.UptimeSeconds, Time(h.AtUtcMs)); new(h.CpuPercent, h.MemUsed, h.MemTotal, h.DiskUsed, h.DiskTotal, h.NetRxTotal, h.NetTxTotal, h.NetRxBps, h.NetTxBps, h.NetLinkMbps, h.UptimeSeconds, Time(h.AtUtcMs));
public static ConnectedProbeDto ToDto(ConnectedProbeInfo p) => new(p.Id, p.Version, p.Remote, Time(p.SinceMs)); public static ConnectedProbeDto ToDto(ConnectedProbeInfo p) => new(p.Id, p.Version, p.Remote, Time(p.SinceMs));
public static ProbeDto ToDto(ProbeView v) => public static ProbeDto ToDto(ProbeView v) =>
new( new(
v.Probe.Id, v.Probe.Name, Time(v.Probe.CreatedAtMs), Time(v.Probe.RevokedAtMs), v.Revoked, v.Probe.Id, v.Probe.Name, Time(v.Probe.CreatedAtMs), Time(v.Probe.RevokedAtMs), v.Revoked, v.Online,
[.. v.Connections.Select(c => new ProbeConnectionDto(c.AgentId.Value, c.AgentName, c.AgentOnline, c.Version, c.Remote, Time(c.SinceMs)))]); [.. v.Connections.Select(c => new ProbeConnectionDto(c.AgentId.Value, c.AgentName, c.AgentOnline, c.Version, c.Remote, Time(c.SinceMs)))]);
public static SeriesDto ToDto(SeriesResult r, bool live) => public static SeriesDto ToDto(SeriesResult r, bool live) =>
@@ -2,9 +2,10 @@ using System.Text.Json;
namespace Monitor.Backend.Api; namespace Monitor.Backend.Api;
public sealed record LoginRequest(string Username, string Password); /// <summary><c>return_session</c>:跨域的前端用——不设 Cookie,把会话令牌放在响应里,之后用 <c>Authorization: Bearer</c> 带上。</summary>
public sealed record LoginRequest(string Username, string Password, bool? ReturnSession);
public sealed record LoginResponse(string UserId, string Username, string CsrfToken, DateTimeOffset ExpiresAt); public sealed record LoginResponse(string UserId, string Username, string CsrfToken, DateTimeOffset ExpiresAt, string? SessionToken);
public sealed record MeDto(string Kind, string Scope, string? UserId, string? Username, string? CsrfToken); public sealed record MeDto(string Kind, string Scope, string? UserId, string? Username, string? CsrfToken);
@@ -46,7 +47,9 @@ public sealed record JoinTokenDto(string Id, string? Note, IReadOnlyList<string>
public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool? AutoApprove, int? MaxUses, int? ExpiresInHours); public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool? AutoApprove, int? MaxUses, int? ExpiresInHours);
/// <summary>令牌明文只在创建的响应里出现一次。</summary> /// <summary>令牌明文只在创建的响应里出现一次。</summary>
public sealed record CreatedJoinTokenDto(JoinTokenDto Token, string Plaintext, string InstallCommand); public sealed record InstallCommandsDto(string Linux, string Macos, string Windows);
public sealed record CreatedJoinTokenDto(JoinTokenDto Token, string Plaintext, InstallCommandsDto InstallCommands);
/// <summary>Agent 眼中连着的一个 Probe(Agent 上报)。</summary> /// <summary>Agent 眼中连着的一个 Probe(Agent 上报)。</summary>
public sealed record ConnectedProbeDto(string ProbeId, string? Version, string? Remote, DateTimeOffset ConnectedSince); public sealed record ConnectedProbeDto(string ProbeId, string? Version, string? Remote, DateTimeOffset ConnectedSince);
@@ -54,14 +57,19 @@ public sealed record ConnectedProbeDto(string ProbeId, string? Version, string?
/// <summary>某个 Probe 现在连着的一个 Agent。</summary> /// <summary>某个 Probe 现在连着的一个 Agent。</summary>
public sealed record ProbeConnectionDto(string AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, DateTimeOffset ConnectedSince); public sealed record ProbeConnectionDto(string AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, DateTimeOffset ConnectedSince);
public sealed record ProbeDto(string Id, string Name, DateTimeOffset CreatedAt, DateTimeOffset? RevokedAt, bool Revoked, IReadOnlyList<ProbeConnectionDto> Connections); public sealed record ProbeDto(string Id, string Name, DateTimeOffset CreatedAt, DateTimeOffset? RevokedAt, bool Revoked, bool Online, IReadOnlyList<ProbeConnectionDto> Connections);
public sealed record BootstrapAgentDto(string AgentId, string Host, int TcpPort, string Spki); /// <summary>一对(Probe,Agent)的线路开关:<c>linked</c> 是最终结果,<c>override</c> 是手动设置(<c>null</c> = 跟随自动连接),<c>connected</c> 是现在有没有连着。</summary>
public sealed record LinkDto(string ProbeId, string ProbeName, string AgentId, string AgentName, bool Linked, bool? Override, bool Connected);
public sealed record LinkMatrixDto(bool AutoLink, IReadOnlyList<LinkDto> Items);
public sealed record SetLinkRequest(bool? Enabled);
public sealed record ProbeLimitsDto(double MaxTestMbps, long MaxTotalPps); public sealed record ProbeLimitsDto(double MaxTestMbps, long MaxTotalPps);
/// <summary><c>probe.json</c>:含私钥,只在生成时返回一次。</summary> /// <summary><c>probe.json</c>:含私钥,只在生成时返回一次。</summary>
public sealed record ProbeIdentityDto(string ProbeId, string PrivateKey, IReadOnlyList<BootstrapAgentDto> Bootstrap, ProbeLimitsDto Limits); public sealed record ProbeIdentityDto(string ProbeId, string PrivateKey, string Server, ProbeLimitsDto Limits);
public sealed record CreateProbeRequest(string? Name); public sealed record CreateProbeRequest(string? Name);
@@ -121,3 +129,16 @@ public sealed record CreateUserRequest(string Username, string Password);
public sealed record ChangePasswordRequest(string Password); public sealed record ChangePasswordRequest(string Password);
public sealed record RecomputeDto(string State, IReadOnlyList<string> Metrics, long WindowsDone, long WindowsTotal, DateTimeOffset? From, DateTimeOffset? To, int QueueLength); public sealed record RecomputeDto(string State, IReadOnlyList<string> Metrics, long WindowsDone, long WindowsTotal, DateTimeOffset? From, DateTimeOffset? To, int QueueLength);
/// <summary>管理端看的构建信息:每个节点自报的版本、tag、提交和构建时间,以及系统。</summary>
public sealed record NodeBuildDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Version, string? Tag, string? Commit, string? BuiltAt, NodeUpdateDto? Update = null);
/// <summary>节点宿主机上的更新器报告的状态(<c>up_to_date</c>、<c>available</c>、<c>applying</c>、<c>failed</c>、<c>rolled_back</c>、<c>invalid</c>);没装更新器的节点没有这个字段。</summary>
public sealed record NodeUpdateDto(string State, string? CurrentDigest, string? AvailableTag, string? AvailableVersion, string? Error, DateTimeOffset? At);
/// <summary>面板登记的最新发布清单(CI 签名的;验签在节点上)。</summary>
public sealed record ReleaseDto(string Component, long Seq, string Tag, string Version, string Digest);
public sealed record PublishReleaseRequest(string? Manifest, string? Signature);
public sealed record BuildInfoDto(BuildDto Backend, IReadOnlyList<NodeBuildDto> Nodes);
@@ -37,6 +37,18 @@ namespace Monitor.Backend.Api;
[JsonSerializable(typeof(AgentDto))] [JsonSerializable(typeof(AgentDto))]
[JsonSerializable(typeof(PageDto<AgentDto>))] [JsonSerializable(typeof(PageDto<AgentDto>))]
[JsonSerializable(typeof(AgentPatchRequest))] [JsonSerializable(typeof(AgentPatchRequest))]
[JsonSerializable(typeof(LinkDto))]
[JsonSerializable(typeof(LinkMatrixDto))]
[JsonSerializable(typeof(SetLinkRequest))]
[JsonSerializable(typeof(BuildDto))]
[JsonSerializable(typeof(ReleaseDto))]
[JsonSerializable(typeof(PageDto<ReleaseDto>))]
[JsonSerializable(typeof(PublishReleaseRequest))]
[JsonSerializable(typeof(AboutDto))]
[JsonSerializable(typeof(NodeBuildDto))]
[JsonSerializable(typeof(BuildInfoDto))]
[JsonSerializable(typeof(NodeInfoDto))]
[JsonSerializable(typeof(PageDto<NodeInfoDto>))]
[JsonSerializable(typeof(JoinTokenDto))] [JsonSerializable(typeof(JoinTokenDto))]
[JsonSerializable(typeof(PageDto<JoinTokenDto>))] [JsonSerializable(typeof(PageDto<JoinTokenDto>))]
[JsonSerializable(typeof(CreateJoinTokenRequest))] [JsonSerializable(typeof(CreateJoinTokenRequest))]
@@ -163,10 +163,20 @@ public sealed record HostDto(
long? NetTxTotal, long? NetTxTotal,
long? NetRxBps, long? NetRxBps,
long? NetTxBps, long? NetTxBps,
long? NetLinkMbps,
long? UptimeSeconds, long? UptimeSeconds,
DateTimeOffset At); DateTimeOffset At);
/// <summary>一个 Agent 的最新主机状态;<c>stale</c> 表示超过 15 秒没有更新(离线,或者 Agent 关闭了上报)。</summary> /// <summary>一个 Agent 的最新主机状态;<c>stale</c> 表示超过 15 秒没有更新(离线,或者 Agent 关闭了上报)。</summary>
/// <summary>节点的展示信息:只有系统、架构、国家代码、运营商,没有任何地址;站点打开「显示构建信息」之后还有版本和 tag。</summary>
public sealed record NodeInfoDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Country, string? IspKey, string? IspName, string? Version = null, string? Tag = null);
/// <summary>一个程序的构建信息:版本、tag、提交(短)和构建时间(UTC,ISO 8601)。没设置的是 <c>null</c>。</summary>
public sealed record BuildDto(string Version, string? Tag, string? Commit, string? BuiltAt);
/// <summary>公开的「关于」:站点没有打开「显示构建信息」时只有 <c>show_build: false</c>。</summary>
public sealed record AboutDto(bool ShowBuild, BuildDto? Backend);
public sealed record HostItemDto(string AgentId, string Name, bool Online, bool Stale, DateTimeOffset? LastSeenAt, HostDto? Host); public sealed record HostItemDto(string AgentId, string Name, bool Online, bool Stale, DateTimeOffset? LastSeenAt, HostDto? Host);
public sealed record LiveHostDto(string AgentId, bool Online, HostDto Host); public sealed record LiveHostDto(string AgentId, bool Online, HostDto Host);
@@ -0,0 +1,69 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Monitor.Backend.Application;
namespace Monitor.Backend.Api;
/// <summary>
/// 节点的展示信息:系统、架构、国家、运营商。<b>只有这几项,没有任何 IP 地址、城市、ASN、经纬度</b>——
/// 这些在 Backend 里只在查询的那一刻用到,存起来的记录里也没有,所以访客(和任何别的接口)都拿不到。
/// </summary>
public static class NodeEndpoints
{
private const string Tag = "public-nodes";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/nodes", ListAsync)
.Api("nodes_list", "Display info of the visible agents and probes: system, architecture, country and ISP (never an address)", Tag)
.Returns<PageDto<NodeInfoDto>>().ProducesProblems(403);
group.MapGet("/about", AboutAsync)
.Api("about_get", "The backend's version, tag and build time, if the site shows build info", Tag)
.Returns<AboutDto>().ProducesProblems(403);
}
/// <summary>这个后端自己的构建信息;没有(本地构建)的字段是 <c>null</c>。</summary>
internal static BuildDto Backend(bool withCommit) =>
new(Monitor.Protocol.BuildInfo.Version, Empty(Monitor.Protocol.BuildInfo.Tag), withCommit ? Empty(Monitor.Protocol.BuildInfo.Commit) : null, Empty(Monitor.Protocol.BuildInfo.BuiltAt));
private static string? Empty(string value) => value.Length == 0 ? null : value;
private static async Task<IResult> AboutAsync(HttpContext http, SettingsService settings)
{
bool show = await settings.IsShowBuildAsync(http.RequestAborted);
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new AboutDto(show, show ? Backend(withCommit: false) : null), ApiJsonContext.Default.AboutDto);
}
private static async Task<IResult> ListAsync(HttpContext http, AgentService agents, ProbeService probes, NodeInfoService nodes, SettingsService settings, CancellationToken ct)
{
// 和别的公开接口一样:看得见的 Agent(隐藏的只有登录后才看得见)和没有被吊销的 Probe
var visible = new HashSet<string>(StringComparer.Ordinal);
foreach (AgentView view in await agents.ListAsync(ct))
{
if (view.Agent.Status == Fleet.Server.NodeStatus.Active && (view.Agent.PublicVisible || PublicGuard.CanSeeHidden(http)))
{
visible.Add(view.Agent.Id.Value);
}
}
foreach (ProbeView probe in await probes.ListAsync(ct))
{
if (!probe.Revoked)
{
visible.Add(probe.Probe.Id);
}
}
// 版本和 tag 只在站点打开「显示构建信息」时才给访客;关着就是 null
bool showBuild = await settings.IsShowBuildAsync(ct);
var items = (await nodes.ListAsync(ct))
.Where(n => visible.Contains(n.NodeId))
.OrderBy(n => n.NodeId, StringComparer.Ordinal)
.Select(n => new NodeInfoDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Country, n.IspKey, n.IspName, showBuild ? n.Version : null, showBuild ? n.Tag : null))
.ToList();
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new PageDto<NodeInfoDto>(items, null), ApiJsonContext.Default.PageDtoNodeInfoDto);
}
}
@@ -4,7 +4,10 @@ namespace Monitor.Backend.Application;
public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool AutoApprove, int MaxUses, int ExpiresInHours); public sealed record CreateJoinTokenRequest(string? Note, IReadOnlyList<string>? Labels, bool AutoApprove, int MaxUses, int ExpiresInHours);
public sealed record CreatedJoinToken(JoinTokenInfo Info, string Plaintext, string InstallCommand); /// <summary>各个系统上的安装命令(Linux 和 macOS 都用 install.sh,Windows 用 install.ps1)。</summary>
public sealed record InstallCommands(string Linux, string Macos, string Windows);
public sealed record CreatedJoinToken(JoinTokenInfo Info, string Plaintext, InstallCommands InstallCommands);
/// <summary>加入令牌:明文只在创建时返回一次,服务端只保存 SHA-256。</summary> /// <summary>加入令牌:明文只在创建时返回一次,服务端只保存 SHA-256。</summary>
public sealed class JoinTokenService public sealed class JoinTokenService
@@ -55,11 +58,16 @@ public sealed class JoinTokenService
request.MaxUses, now + (request.ExpiresInHours * 3_600_000L), now, ct); request.MaxUses, now + (request.ExpiresInHours * 3_600_000L), now, ct);
string baseUrl = (_options.PublicUrl ?? requestBaseUrl ?? "https://panel.example.com").TrimEnd('/'); string baseUrl = (_options.PublicUrl ?? requestBaseUrl ?? "https://panel.example.com").TrimEnd('/');
string command = $"curl -fsSL {baseUrl}/install.sh | sh -s -- --join {plaintext} --name <node-name>"; string shell = $"curl -fsSL {baseUrl}/install.sh | sh -s -- --join {plaintext} --name <node-name>";
return Result<CreatedJoinToken>.Ok(new CreatedJoinToken(info, plaintext, command)); // PowerShell 6+(pwsh):在 cmd 或 PowerShell 里都能直接粘贴
string powershell = $"pwsh -NoProfile -ExecutionPolicy Bypass -Command \"& ([scriptblock]::Create((irm {baseUrl}/install.ps1))) -Join {plaintext} -Name <node-name>\"";
return Result<CreatedJoinToken>.Ok(new CreatedJoinToken(info, plaintext, new InstallCommands(shell, shell, powershell)));
} }
public ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct) => _store.ListAsync(ct); public ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct) => _store.ListAsync(ct);
public ValueTask<bool> RevokeAsync(string id, CancellationToken ct) => _store.RevokeAsync(id, _time.GetUtcNow().ToUnixTimeMilliseconds(), ct); public ValueTask<bool> RevokeAsync(string id, CancellationToken ct) => _store.RevokeAsync(id, _time.GetUtcNow().ToUnixTimeMilliseconds(), ct);
/// <summary>彻底删除这条令牌记录(任何状态都可以);还没用完的令牌删除后立即不能再用。</summary>
public ValueTask<bool> DeleteAsync(string id, CancellationToken ct) => _store.DeleteAsync(id, ct);
} }
@@ -12,6 +12,8 @@ public sealed record SettingView(string Key, string Type, string Value, string D
public sealed class SettingsService public sealed class SettingsService
{ {
public const string SitePublic = "site.public"; public const string SitePublic = "site.public";
public const string SiteShowBuild = "site.show_build";
public const string AutoLink = "fleet.auto_link";
public const string RawDays = "retention.raw_days"; public const string RawDays = "retention.raw_days";
public const string TestRawDays = "retention.test_raw_days"; public const string TestRawDays = "retention.test_raw_days";
public const string Metric10sHours = "retention.metric_10s_hours"; public const string Metric10sHours = "retention.metric_10s_hours";
@@ -40,6 +42,8 @@ public sealed class SettingsService
private static IReadOnlyList<SettingSpec> Known(BackendOptions o) => private static IReadOnlyList<SettingSpec> Known(BackendOptions o) =>
[ [
new(SitePublic, "bool", "false", 0, 1, "Whether anonymous visitors can read the public API."), new(SitePublic, "bool", "false", 0, 1, "Whether anonymous visitors can read the public API."),
new(SiteShowBuild, "bool", "false", 0, 1, "Whether the public pages show the versions, tags and build times of the backend, agents and probes."),
new(AutoLink, "bool", "true", 0, 1, "Whether every probe connects to every agent automatically; when off, only the links chosen by hand are connected."),
new(RawDays, "int", o.RawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of continuous tasks."), new(RawDays, "int", o.RawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of continuous tasks."),
new(TestRawDays, "int", o.TestRawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of on-demand tests."), new(TestRawDays, "int", o.TestRawRetentionDays.ToString(CultureInfo.InvariantCulture), 1, 365, "Days to keep raw chunks of on-demand tests."),
new(Metric10sHours, "int", o.Metric10sRetentionHours.ToString(CultureInfo.InvariantCulture), 1, 720, "Hours to keep the 10 s metric layer."), new(Metric10sHours, "int", o.Metric10sRetentionHours.ToString(CultureInfo.InvariantCulture), 1, 720, "Hours to keep the 10 s metric layer."),
@@ -114,6 +118,9 @@ public sealed class SettingsService
return _sitePublic; return _sitePublic;
} }
/// <summary>访客能不能看到版本、tag 和构建时间(站点公开之后的另一道开关,默认关)。</summary>
public async ValueTask<bool> IsShowBuildAsync(CancellationToken ct) => await _store.GetAsync(SiteShowBuild, ct) == "true";
public async ValueTask<int> GetIntAsync(string key, CancellationToken ct) public async ValueTask<int> GetIntAsync(string key, CancellationToken ct)
{ {
SettingSpec spec = Specs.First(s => s.Key == key); SettingSpec spec = Specs.First(s => s.Key == key);
@@ -9,12 +9,14 @@ public sealed class AgentEventService
private readonly AgentRuntime _runtime; private readonly AgentRuntime _runtime;
private readonly IEventStore _events; private readonly IEventStore _events;
private readonly TimeProvider _time; private readonly TimeProvider _time;
private readonly NodeInfoService _nodes;
public AgentEventService(AgentRuntime runtime, IEventStore events, TimeProvider time) public AgentEventService(AgentRuntime runtime, IEventStore events, TimeProvider time, NodeInfoService nodes)
{ {
_runtime = runtime; _runtime = runtime;
_events = events; _events = events;
_time = time; _time = time;
_nodes = nodes;
} }
public async ValueTask OnEventAsync(NodeId agent, AgentEventParams e, CancellationToken ct) public async ValueTask OnEventAsync(NodeId agent, AgentEventParams e, CancellationToken ct)
@@ -33,6 +35,7 @@ public sealed class AgentEventService
{ {
case AgentEventKinds.ProbeConnected when !string.IsNullOrEmpty(e.Detail): case AgentEventKinds.ProbeConnected when !string.IsNullOrEmpty(e.Detail):
state.AddProbe(e.Probe is { } p ? new ConnectedProbeInfo(e.Detail, p.Version, p.Remote, p.SinceUtcMs > 0 ? p.SinceUtcMs : at) : new ConnectedProbeInfo(e.Detail, null, null, at)); state.AddProbe(e.Probe is { } p ? new ConnectedProbeInfo(e.Detail, p.Version, p.Remote, p.SinceUtcMs > 0 ? p.SinceUtcMs : at) : new ConnectedProbeInfo(e.Detail, null, null, at));
_nodes.ObserveProbeInBackground(e.Detail, e.Probe?.Remote);
break; break;
case AgentEventKinds.ProbeDisconnected when !string.IsNullOrEmpty(e.Detail): case AgentEventKinds.ProbeDisconnected when !string.IsNullOrEmpty(e.Detail):
state.RemoveProbe(e.Detail); state.RemoveProbe(e.Detail);
@@ -20,8 +20,13 @@ public sealed partial class AgentService
private readonly TaskPlanner _planner; private readonly TaskPlanner _planner;
private readonly ISettingsStore _settings; private readonly ISettingsStore _settings;
public AgentService(IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TaskPlanner planner, ISettingsStore settings) private readonly NodeInfoService _nodes;
private readonly LinkService _links;
public AgentService(IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TaskPlanner planner, ISettingsStore settings, NodeInfoService nodes, LinkService links)
{ {
_links = links;
_nodes = nodes;
_agents = agents; _agents = agents;
_runtime = runtime; _runtime = runtime;
_gateway = gateway; _gateway = gateway;
@@ -114,17 +119,20 @@ public sealed partial class AgentService
{ {
await _gateway.DisconnectAsync(id, "deleted", ct); await _gateway.DisconnectAsync(id, "deleted", ct);
await _agents.DeleteAsync(id, ct); await _agents.DeleteAsync(id, ct);
await _nodes.ForgetAsync(id.Value, ct);
await _links.ForgetNodeAsync(id.Value, ct);
_runtime.Remove(id); _runtime.Remove(id);
await AfterAgentSetChangedAsync(ct); await AfterAgentSetChangedAsync(ct);
} }
/// <summary>Probe 需要连接的 Agent 列表(所有已激活的 Agent)。版本号在列表变化时递增。</summary> /// <summary>这个 Probe 要连接的 Agent 列表(已激活、有公开地址,并且线路是开着的)。版本号在列表变化时递增。</summary>
public async ValueTask<ProbeAgentsResult> ProbeAgentsAsync(CancellationToken ct) public async ValueTask<ProbeAgentList> ProbeAgentsAsync(string probeId, CancellationToken ct)
{ {
LinkSnapshot links = await _links.SnapshotAsync(ct);
IReadOnlyList<AgentRecord> agents = await _agents.ListAsync(ct); IReadOnlyList<AgentRecord> agents = await _agents.ListAsync(ct);
long version = long.TryParse(await _settings.GetAsync(TaskPlanner.ProbeAgentsVersionKey, ct), out long v) ? v : 0; long version = long.TryParse(await _settings.GetAsync(TaskPlanner.ProbeAgentsVersionKey, ct), out long v) ? v : 0;
var result = new ProbeAgentsResult { Version = version }; var result = new ProbeAgentList { Version = version };
foreach (AgentRecord agent in agents.Where(a => a.Status == NodeStatus.Active && a.PublicHost.Length > 0)) foreach (AgentRecord agent in agents.Where(a => a.Status == NodeStatus.Active && a.PublicHost.Length > 0 && links.IsLinked(probeId, a.Id.Value)))
{ {
result.Agents.Add(new AgentEndpoint result.Agents.Add(new AgentEndpoint
{ {
@@ -140,12 +148,11 @@ public sealed partial class AgentService
return result; return result;
} }
/// <summary>Agent 集合变了:递增列表版本、重算各 Agent 的期望状态、通知 Probe。</summary> /// <summary>Agent 集合变了:递增列表版本,重算各 Agent 的期望状态,并把新的 Agent 列表推给 Probe(都是期望状态,一起通知)。</summary>
public async ValueTask AfterAgentSetChangedAsync(CancellationToken ct) public async ValueTask AfterAgentSetChangedAsync(CancellationToken ct)
{ {
await _planner.BumpProbeAgentsVersionAsync(ct); await _planner.BumpProbeAgentsVersionAsync(ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
await _gateway.NotifyProbeAgentsChangedAsync(await ProbeAgentsAsync(ct), ct);
} }
private async ValueTask<Result<AgentView>> ChangeStatusAsync(NodeId id, NodeStatus status, CancellationToken ct) private async ValueTask<Result<AgentView>> ChangeStatusAsync(NodeId id, NodeStatus status, CancellationToken ct)
@@ -0,0 +1,68 @@
namespace Monitor.Backend.Application;
/// <summary>某一时刻的「线路」开关:自动连接的总开关,加上每一对(Probe,Agent)的手动设置。</summary>
public sealed class LinkSnapshot
{
private readonly Dictionary<(string Probe, string Agent), bool> _overrides;
public LinkSnapshot(bool auto, Dictionary<(string Probe, string Agent), bool> overrides)
{
Auto = auto;
_overrides = overrides;
}
/// <summary>自动连接:开着时,没有手动设置的 Probe 和 Agent 都会连;关着时,只有手动开启的才连。</summary>
public bool Auto { get; }
/// <summary>手动设置:<c>true</c> 强制连、<c>false</c> 强制不连、<c>null</c> 没设置(跟随自动连接)。</summary>
public bool? OverrideOf(string probeId, string agentId) => _overrides.TryGetValue((probeId, agentId), out bool value) ? value : null;
/// <summary>这一对最终连不连:手动设置优先,没有就看自动连接。</summary>
public bool IsLinked(string probeId, string agentId) => OverrideOf(probeId, agentId) ?? Auto;
}
/// <summary>
/// 哪些(Probe,Agent)要连成线路。由面板决定、Probe 主动拨出:自动连接开着时所有 Probe 和 Agent 两两连(默认),
/// 关着时只连手动选中的;每一对还可以手动强制开或关。手动设置存在设置表里(<c>link.&lt;probe&gt;.&lt;agent&gt;</c>,值 1 或 0,空表示没设置)。
/// 判断结果分别用在 Probe 的 Agent 列表、Agent 允许的 Probe(<c>peers</c>)和线路目录里;改动之后要调用 <see cref="AgentService.AfterAgentSetChangedAsync"/> 重新下发。
/// </summary>
public sealed class LinkService
{
private const string Prefix = "link.";
private readonly ISettingsStore _settings;
public LinkService(ISettingsStore settings)
{
_settings = settings;
}
public async ValueTask<LinkSnapshot> SnapshotAsync(CancellationToken ct)
{
IReadOnlyDictionary<string, string> all = await _settings.GetAllAsync(ct);
var overrides = new Dictionary<(string, string), bool>();
foreach ((string key, string value) in all)
{
string[] parts = key.Split('.');
if (parts.Length == 3 && parts[0] + "." == Prefix && value is "1" or "0")
{
overrides[(parts[1], parts[2])] = value == "1";
}
}
return new LinkSnapshot(all.GetValueOrDefault(SettingsService.AutoLink) != "false", overrides);
}
/// <summary>手动设置一对线路:<c>true</c> 强制连、<c>false</c> 强制不连、<c>null</c> 取消手动设置。</summary>
public ValueTask SetAsync(string probeId, string agentId, bool? enabled, CancellationToken ct) =>
_settings.SetAsync($"{Prefix}{probeId}.{agentId}", enabled is null ? "" : enabled.Value ? "1" : "0", ct);
/// <summary>Probe 或 Agent 被删除:清掉所有涉及它的手动设置。</summary>
public async ValueTask ForgetNodeAsync(string nodeId, CancellationToken ct)
{
foreach (string key in (await _settings.GetAllAsync(ct)).Keys.Where(k => k.StartsWith(Prefix, StringComparison.Ordinal) && k.Split('.').Skip(1).Contains(nodeId)))
{
await _settings.SetAsync(key, "", ct);
}
}
}
@@ -51,9 +51,13 @@ public sealed class MonitorNodeEventSink : INodeEventSink
private readonly IEventStore _events; private readonly IEventStore _events;
private readonly LiveHub _live; private readonly LiveHub _live;
private readonly AgentService _service; private readonly AgentService _service;
private readonly NodeInfoService _nodes;
private readonly ReleaseService _releases;
public MonitorNodeEventSink(IAgentStore agents, AgentRuntime runtime, IEventStore events, LiveHub live, AgentService service) public MonitorNodeEventSink(IAgentStore agents, AgentRuntime runtime, IEventStore events, LiveHub live, AgentService service, NodeInfoService nodes, ReleaseService releases)
{ {
_releases = releases;
_nodes = nodes;
_agents = agents; _agents = agents;
_runtime = runtime; _runtime = runtime;
_events = events; _events = events;
@@ -64,6 +68,12 @@ public sealed class MonitorNodeEventSink : INodeEventSink
public async ValueTask OnEventAsync(NodeEvent nodeEvent, CancellationToken ct) public async ValueTask OnEventAsync(NodeEvent nodeEvent, CancellationToken ct)
{ {
NodeId id = nodeEvent.Node.Id; NodeId id = nodeEvent.Node.Id;
if (id.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
await OnProbeEventAsync(id, nodeEvent, ct);
return;
}
switch (nodeEvent.Kind) switch (nodeEvent.Kind)
{ {
case NodeEventKind.Online: case NodeEventKind.Online:
@@ -72,6 +82,7 @@ public sealed class MonitorNodeEventSink : INodeEventSink
break; break;
case NodeEventKind.Offline: case NodeEventKind.Offline:
_runtime.MarkOffline(id); _runtime.MarkOffline(id);
_releases.RecordStatus(id, null);
_live.Publish(new NodeSignal("agent", id.Value, false, nodeEvent.AtMs)); _live.Publish(new NodeSignal("agent", id.Value, false, nodeEvent.AtMs));
await _events.AddAsync(id, "agent_offline", nodeEvent.AtMs, null, ct); await _events.AddAsync(id, "agent_offline", nodeEvent.AtMs, null, ct);
break; break;
@@ -85,9 +96,14 @@ public sealed class MonitorNodeEventSink : INodeEventSink
AgentHeartbeatExtra? beat = extra.Deserialize(MonitorJsonContext.Default.AgentHeartbeatExtra); AgentHeartbeatExtra? beat = extra.Deserialize(MonitorJsonContext.Default.AgentHeartbeatExtra);
state.LastHeartbeat = beat; state.LastHeartbeat = beat;
state.LastHeartbeatAtMs = nodeEvent.AtMs; state.LastHeartbeatAtMs = nodeEvent.AtMs;
_releases.RecordStatus(id, beat?.Update);
if (beat is not null) if (beat is not null)
{ {
state.ReplaceProbes(beat.Probes.Select(p => new ConnectedProbeInfo(p.Id.Value, p.Version, p.Remote, p.SinceUtcMs))); state.ReplaceProbes(beat.Probes.Select(p => new ConnectedProbeInfo(p.Id.Value, p.Version, p.Remote, p.SinceUtcMs)));
foreach (ConnectedProbe p in beat.Probes)
{
_nodes.ObserveProbeInBackground(p.Id.Value, p.Remote);
}
} }
} }
catch (JsonException) catch (JsonException)
@@ -99,6 +115,48 @@ public sealed class MonitorNodeEventSink : INodeEventSink
} }
} }
/// <summary>Probe 自己连上来了:hello 里带着它的系统、架构和构建信息;上线下线只推给 SSE,不记事件。</summary>
private async ValueTask OnProbeEventAsync(NodeId id, NodeEvent nodeEvent, CancellationToken ct)
{
switch (nodeEvent.Kind)
{
case NodeEventKind.Online:
_live.Publish(new NodeSignal("probe", id.Value, true, nodeEvent.AtMs));
break;
case NodeEventKind.Hello:
if (nodeEvent.Hello?.Meta is { } raw)
{
try
{
ProbeHelloMeta? meta = raw.Deserialize(MonitorJsonContext.Default.ProbeHelloMeta);
await _nodes.SetPlatformAsync(id.Value, NodeKinds.Probe, meta?.Platform, ct);
}
catch (JsonException)
{
// 格式不对就忽略
}
}
await _releases.OfferToNodeAsync(id, ct);
break;
case NodeEventKind.Offline:
_live.Publish(new NodeSignal("probe", id.Value, false, nodeEvent.AtMs));
_releases.RecordStatus(id, null);
break;
case NodeEventKind.Heartbeat when nodeEvent.Extra is { } extra:
try
{
_releases.RecordStatus(id, extra.Deserialize(MonitorJsonContext.Default.ProbeHeartbeatExtra)?.Update);
}
catch (JsonException)
{
// 忽略格式不正确的附加字段
}
break;
}
}
private async ValueTask ApplyHelloAsync(NodeId id, HelloParams hello, CancellationToken ct) private async ValueTask ApplyHelloAsync(NodeId id, HelloParams hello, CancellationToken ct)
{ {
var kinds = new List<string>(); var kinds = new List<string>();
@@ -142,6 +200,8 @@ public sealed class MonitorNodeEventSink : INodeEventSink
// 端口或证书指纹变了,Probe 需要新的列表 // 端口或证书指纹变了,Probe 需要新的列表
AgentRecord? after = await _agents.GetAsync(id, ct); AgentRecord? after = await _agents.GetAsync(id, ct);
_nodes.ObserveAgentInBackground(id.Value, after?.PublicHost, meta.Platform);
await _releases.OfferToNodeAsync(id, ct);
if (before is not null && after is not null if (before is not null && after is not null
&& (!before.TlsSpki.AsSpan().SequenceEqual(after.TlsSpki) || before.ProbeTcpPort != after.ProbeTcpPort || !before.ProbeUdpPorts.SequenceEqual(after.ProbeUdpPorts)) && (!before.TlsSpki.AsSpan().SequenceEqual(after.TlsSpki) || before.ProbeTcpPort != after.ProbeTcpPort || !before.ProbeUdpPorts.SequenceEqual(after.ProbeUdpPorts))
&& after.Status == NodeStatus.Active) && after.Status == NodeStatus.Active)
@@ -6,47 +6,47 @@ using Monitor.Protocol;
namespace Monitor.Backend.Application; namespace Monitor.Backend.Application;
public sealed record BootstrapAgent(NodeId AgentId, string Host, int TcpPort, string Spki);
/// <summary>本地硬上限:写在 Probe 本机的配置里,面板无法修改(在路由器上建议调低)。</summary> /// <summary>本地硬上限:写在 Probe 本机的配置里,面板无法修改(在路由器上建议调低)。</summary>
public sealed record ProbeLimits(double MaxTestMbps, long MaxTotalPps); public sealed record ProbeLimits(double MaxTestMbps, long MaxTotalPps);
/// <summary><c>probe.json</c>:只在生成(或轮换密钥)时返回一次,含私钥。</summary> /// <summary><c>probe.json</c>:只在生成(或轮换密钥)时返回一次,含私钥;<c>Server</c> 是 Probe 要连的面板地址。</summary>
public sealed record ProbeIdentityFile(ProbeId ProbeId, string PrivateKey, IReadOnlyList<BootstrapAgent> Bootstrap, ProbeLimits Limits); public sealed record ProbeIdentityFile(ProbeId ProbeId, string PrivateKey, string Server, ProbeLimits Limits);
/// <summary>某个 Probe 现在连着的一个 Agent(Agent 上报)。</summary> /// <summary>某个 Probe 现在连着的一个 Agent(Agent 上报)。</summary>
public sealed record ProbeConnection(NodeId AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, long SinceMs); public sealed record ProbeConnection(NodeId AgentId, string AgentName, bool AgentOnline, string? Version, string? Remote, long SinceMs);
public sealed record ProbeView(ProbeRecord Probe, IReadOnlyList<ProbeConnection> Connections) public sealed record ProbeView(ProbeRecord Probe, bool Online, IReadOnlyList<ProbeConnection> Connections)
{ {
public bool Revoked => Probe.RevokedAtMs is not null; public bool Revoked => Probe.RevokedAtMs is not null;
} }
/// <summary> /// <summary>
/// Probe 身份(Backend 生成,私钥只返回一次):可以有多个,每个都能改名、轮换密钥、吊销、删除。 /// Probe 身份(Backend 生成,私钥只返回一次):可以有多个,每个都能改名、轮换密钥、吊销、删除。
/// 所有未吊销的 Probe 的公钥都会写进每个 Agent 的期望状态(<c>peers</c>);吊销、删除、轮换密钥之后,Agent 立刻断开对应会话。 /// 所有未吊销的 Probe 的公钥都会写进每个 Agent 的期望状态(<c>peers</c>);吊销、删除、轮换密钥之后,Agent 立刻断开对应会话,Probe 到面板的连接也会被拒绝。
/// 一条线路是一对(Probe,Agent)。 /// 一条线路是一对(Probe,Agent)。
/// </summary> /// </summary>
public sealed partial class ProbeService public sealed partial class ProbeService
{ {
private readonly IProbeStore _probes; private readonly IProbeStore _probes;
private readonly IAgentStore _agents; private readonly IAgentStore _agents;
private readonly AgentService _agentService;
private readonly AgentRuntime _runtime; private readonly AgentRuntime _runtime;
private readonly IAgentGateway _gateway; private readonly IAgentGateway _gateway;
private readonly TimeProvider _time; private readonly TimeProvider _time;
private readonly NodeInfoService _nodes;
private readonly LinkService _links;
public ProbeService(IProbeStore probes, IAgentStore agents, AgentService agentService, AgentRuntime runtime, IAgentGateway gateway, TimeProvider time) public ProbeService(IProbeStore probes, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TimeProvider time, NodeInfoService nodes, LinkService links)
{ {
_links = links;
_nodes = nodes;
_probes = probes; _probes = probes;
_agents = agents; _agents = agents;
_agentService = agentService;
_runtime = runtime; _runtime = runtime;
_gateway = gateway; _gateway = gateway;
_time = time; _time = time;
} }
public async ValueTask<Result<ProbeIdentityFile>> CreateAsync(string? name, CancellationToken ct) public async ValueTask<Result<ProbeIdentityFile>> CreateAsync(string? name, string server, CancellationToken ct)
{ {
string finalName = string.IsNullOrWhiteSpace(name) ? "probe" : name.Trim(); string finalName = string.IsNullOrWhiteSpace(name) ? "probe" : name.Trim();
if (!NamePattern().IsMatch(finalName)) if (!NamePattern().IsMatch(finalName))
@@ -63,7 +63,7 @@ public sealed partial class ProbeService
var probe = new ProbeRecord(Ulid.NewPrefixed(ProbeId.Prefix, _time), finalName, identity.PublicKeySpki, _time.GetUtcNow().ToUnixTimeMilliseconds(), null); var probe = new ProbeRecord(Ulid.NewPrefixed(ProbeId.Prefix, _time), finalName, identity.PublicKeySpki, _time.GetUtcNow().ToUnixTimeMilliseconds(), null);
await _probes.SaveAsync(probe, ct); await _probes.SaveAsync(probe, ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
return Result<ProbeIdentityFile>.Ok(await BuildFileAsync(probe, identity, ct)); return Result<ProbeIdentityFile>.Ok(BuildFile(probe, identity, server));
} }
public async ValueTask<IReadOnlyList<ProbeView>> ListAsync(CancellationToken ct) public async ValueTask<IReadOnlyList<ProbeView>> ListAsync(CancellationToken ct)
@@ -99,7 +99,7 @@ public sealed partial class ProbeService
} }
/// <summary>换一把新密钥(id 不变):旧私钥立即失效,所有 Agent 上这个 Probe 的会话被断开。返回新的 probe.json(只此一次)。</summary> /// <summary>换一把新密钥(id 不变):旧私钥立即失效,所有 Agent 上这个 Probe 的会话被断开。返回新的 probe.json(只此一次)。</summary>
public async ValueTask<Result<ProbeIdentityFile>> RotateAsync(string id, CancellationToken ct) public async ValueTask<Result<ProbeIdentityFile>> RotateAsync(string id, string server, CancellationToken ct)
{ {
if (await _probes.GetAsync(id, ct) is not { } probe) if (await _probes.GetAsync(id, ct) is not { } probe)
{ {
@@ -116,7 +116,7 @@ public sealed partial class ProbeService
await _probes.SaveAsync(updated, ct); await _probes.SaveAsync(updated, ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
await DisconnectEverywhereAsync(id, ct); await DisconnectEverywhereAsync(id, ct);
return Result<ProbeIdentityFile>.Ok(await BuildFileAsync(updated, identity, ct)); return Result<ProbeIdentityFile>.Ok(BuildFile(updated, identity, server));
} }
/// <summary>吊销:保留记录和历史数据,公钥从所有 Agent 的名单里移除,Agent 立即断开它的会话。</summary> /// <summary>吊销:保留记录和历史数据,公钥从所有 Agent 的名单里移除,Agent 立即断开它的会话。</summary>
@@ -138,6 +138,8 @@ public sealed partial class ProbeService
bool deleted = await _probes.DeleteAsync(id, ct); bool deleted = await _probes.DeleteAsync(id, ct);
if (deleted) if (deleted)
{ {
await _nodes.ForgetAsync(id, ct);
await _links.ForgetNodeAsync(id, ct);
_gateway.NotifyAllTaskSetsChanged(); _gateway.NotifyAllTaskSetsChanged();
} }
@@ -180,18 +182,11 @@ public sealed partial class ProbeService
} }
} }
return new ProbeView(probe, [.. connections.OrderBy(c => c.AgentName, StringComparer.Ordinal)]); return new ProbeView(probe, _gateway.IsOnline(new NodeId(probe.Id)), [.. connections.OrderBy(c => c.AgentName, StringComparer.Ordinal)]);
} }
private async ValueTask<ProbeIdentityFile> BuildFileAsync(ProbeRecord probe, NodeIdentity identity, CancellationToken ct) private static ProbeIdentityFile BuildFile(ProbeRecord probe, NodeIdentity identity, string server) =>
{ new(new ProbeId(probe.Id), Convert.ToBase64String(identity.ExportPkcs8()), server, new ProbeLimits(MaxTestMbps: 100, MaxTotalPps: 20_000));
ProbeAgentsResult agents = await _agentService.ProbeAgentsAsync(ct);
return new ProbeIdentityFile(
new ProbeId(probe.Id),
Convert.ToBase64String(identity.ExportPkcs8()),
[.. agents.Agents.Select(a => new BootstrapAgent(a.AgentId, a.Host, a.TcpPort, a.Spki))],
new ProbeLimits(MaxTestMbps: 100, MaxTotalPps: 20_000));
}
[GeneratedRegex(@"^[\p{L}\p{M}\p{N}._-]{1,64}$")] [GeneratedRegex(@"^[\p{L}\p{M}\p{N}._-]{1,64}$")]
private static partial Regex NamePattern(); private static partial Regex NamePattern();
@@ -27,13 +27,16 @@ public static class ApplicationServiceCollectionExtensions
services.AddSingleton<AgentService>(); services.AddSingleton<AgentService>();
services.AddSingleton<AgentEventService>(); services.AddSingleton<AgentEventService>();
services.AddSingleton<HostStatusService>(); services.AddSingleton<HostStatusService>();
services.AddSingleton<NodeInfoService>();
services.AddSingleton<ProbeService>(); services.AddSingleton<ProbeService>();
services.AddSingleton<LinkService>();
services.AddSingleton<LineService>(); services.AddSingleton<LineService>();
services.AddSingleton<TestService>(); services.AddSingleton<TestService>();
services.AddSingleton<IngestService>(); services.AddSingleton<IngestService>();
services.AddSingleton<WindowComputer>(); services.AddSingleton<WindowComputer>();
services.AddSingleton<AuthService>(); services.AddSingleton<AuthService>();
services.AddSingleton<SettingsService>(); services.AddSingleton<SettingsService>();
services.AddSingleton<ReleaseService>();
services.AddSingleton<JoinTokenService>(); services.AddSingleton<JoinTokenService>();
services.AddSingleton<RawBrowserService>(); services.AddSingleton<RawBrowserService>();
services.AddSingleton<EventService>(); services.AddSingleton<EventService>();
@@ -7,12 +7,13 @@ public enum PrincipalKind
Token, Token,
} }
/// <summary>访问级别:作用域为 read 的令牌只读,admin 可以管理。</summary> /// <summary>访问级别:作用域为 read 的令牌只读,admin 可以管理,release 只能登记发布清单(给 CI 用,什么都读不了)。</summary>
public enum ApiScope public enum ApiScope
{ {
None = 0, None = 0,
Read = 1, Release = 1,
Admin = 2, Read = 2,
Admin = 3,
} }
public sealed record Principal(PrincipalKind Kind, ApiScope Scope, string? UserId, string? Username, string? CsrfToken, string? TokenId) public sealed record Principal(PrincipalKind Kind, ApiScope Scope, string? UserId, string? Username, string? CsrfToken, string? TokenId)
@@ -22,6 +23,8 @@ public sealed record Principal(PrincipalKind Kind, ApiScope Scope, string? UserI
public bool CanRead => Scope >= ApiScope.Read; public bool CanRead => Scope >= ApiScope.Read;
public bool IsAdmin => Scope >= ApiScope.Admin; public bool IsAdmin => Scope >= ApiScope.Admin;
public bool CanPublishRelease => Scope is ApiScope.Release or ApiScope.Admin;
} }
public sealed record LoginResult(string SessionToken, string CsrfToken, long ExpiresAtMs, string UserId, string Username); public sealed record LoginResult(string SessionToken, string CsrfToken, long ExpiresAtMs, string UserId, string Username);
@@ -77,7 +77,7 @@ public sealed partial class AuthService
await _store.TouchApiTokenAsync(token.Id, now, ct); await _store.TouchApiTokenAsync(token.Id, now, ct);
} }
ApiScope scope = token.Scope == "admin" ? ApiScope.Admin : ApiScope.Read; ApiScope scope = token.Scope switch { "admin" => ApiScope.Admin, "release" => ApiScope.Release, _ => ApiScope.Read };
return new Principal(PrincipalKind.Token, scope, null, token.Name, null, token.Id); return new Principal(PrincipalKind.Token, scope, null, token.Name, null, token.Id);
} }
@@ -135,9 +135,9 @@ public sealed partial class AuthService
return ValidationError.Fail<CreatedApiToken>("name", "must be 1-64 characters"); return ValidationError.Fail<CreatedApiToken>("name", "must be 1-64 characters");
} }
if (scope is not ("read" or "admin")) if (scope is not ("read" or "admin" or "release"))
{ {
return ValidationError.Fail<CreatedApiToken>("scope", "must be read or admin"); return ValidationError.Fail<CreatedApiToken>("scope", "must be read, admin or release");
} }
string plaintext = TokenHash.NewToken("mon"); string plaintext = TokenHash.NewToken("mon");
@@ -55,6 +55,18 @@ public sealed class BackendOptions
public int MaxCompareAgents { get; set; } = 20; public int MaxCompareAgents { get; set; } = 20;
/// <summary>
/// 测试方案里「固定时长」阶段(没有 <c>duration_s</c> 参数的,例如 connect、hold)的缩放比例,缺省 1。
/// 只给自动化测试用:端到端测试里把 30 秒的阶段缩短,不用真等。
/// </summary>
public double PhaseDurationScale { get; set; } = 1.0;
/// <summary>
/// IP 归属查询服务的地址(`GET {地址}/lookup?ip=…`):Backend 用 Probe、Agent 的来源地址查一次国家和运营商,
/// 只留下国家代码和运营商名,IP 不对外。留空 = 不查。服务的使用条款(公益、不能商用、保留署名)由部署的人自己遵守。
/// </summary>
public string? GeoLookupUrl { get; set; } = "https://ip.007995.xyz";
/// <summary>初始管理员(只在没有任何用户时创建)。</summary> /// <summary>初始管理员(只在没有任何用户时创建)。</summary>
public string? InitialAdminUser { get; set; } public string? InitialAdminUser { get; set; }
@@ -0,0 +1,86 @@
namespace Monitor.Backend.Application;
/// <summary>
/// 把「ASN 和组织名」归成几个常见的国内运营商:mobile / telecom / unicom / broadnet;别的是 other(显示组织名)。
/// 只用来挑展示用的图标和中文名。
/// </summary>
public static class IspClassifier
{
public const string Mobile = "mobile";
public const string Telecom = "telecom";
public const string Unicom = "unicom";
public const string Broadnet = "broadnet";
public const string Other = "other";
private static readonly HashSet<int> s_mobile = [9808, 24400, 24445, 56040, 56041, 56042, 56044, 56046, 56047, 56048, 132525, 138407];
private static readonly HashSet<int> s_telecom = [4134, 4809, 4811, 4812, 4813, 9394, 17638, 23724, 38283, 58466, 58519, 136188, 137693];
private static readonly HashSet<int> s_unicom = [4808, 4837, 9929, 10099, 17621, 17622, 17623, 17816, 136958, 138421];
private static readonly HashSet<int> s_broadnet = [63838, 140061];
public static (string Key, string Name) Classify(int? asn, string? organization)
{
if (asn is { } a)
{
if (s_mobile.Contains(a))
{
return (Mobile, "中国移动");
}
if (s_telecom.Contains(a))
{
return (Telecom, "中国电信");
}
if (s_unicom.Contains(a))
{
return (Unicom, "中国联通");
}
if (s_broadnet.Contains(a))
{
return (Broadnet, "中国广电");
}
}
string org = organization ?? string.Empty;
if (Has(org, "china mobile") || Has(org, "cmnet") || Has(org, "中国移动"))
{
return (Mobile, "中国移动");
}
if (Has(org, "china telecom") || Has(org, "chinanet") || Has(org, "中国电信"))
{
return (Telecom, "中国电信");
}
if (Has(org, "china unicom") || Has(org, "cncgroup") || Has(org, "中国联通"))
{
return (Unicom, "中国联通");
}
if (Has(org, "china broadnet") || Has(org, "中国广电"))
{
return (Broadnet, "中国广电");
}
return (Other, Shorten(org));
}
private static bool Has(string text, string needle) => text.Contains(needle, StringComparison.OrdinalIgnoreCase);
/// <summary>组织名常常很长(「Hangzhou Alibaba Advertising Co.,Ltd.」):去掉公司后缀,最多留 28 个字符。</summary>
internal static string Shorten(string organization)
{
string name = organization.Trim();
foreach (string suffix in new[] { ", Inc.", " Inc.", ", LLC", " LLC", " Co.,Ltd.", " Co., Ltd.", " Co.,Ltd", " Co., Ltd", " Ltd.", " Ltd", " Corporation", " Corp.", " Limited", " GmbH", " B.V.", " S.A." })
{
if (name.EndsWith(suffix, StringComparison.OrdinalIgnoreCase))
{
name = name[..^suffix.Length].TrimEnd(',', ' ');
break;
}
}
return name.Length > 28 ? name[..28].TrimEnd() : name;
}
}
@@ -0,0 +1,34 @@
namespace Monitor.Backend.Application;
/// <summary>
/// 一个节点(Agent 或 Probe)用来展示的信息:系统和架构、国家、运营商,以及它自己上报的构建信息(版本、tag、提交、构建时间)。
/// <b>不含任何 IP 地址、城市、ASN 或经纬度</b>——这些只在查询的那一刻用到,不留下,也不对外。
/// </summary>
public sealed record NodeInfo(
string NodeId,
string Kind,
string? Os,
string? OsName,
string? Arch,
string? Country,
string? IspKey,
string? IspName,
string? Version = null,
string? Tag = null,
string? Commit = null,
string? BuiltAt = null);
/// <summary>
/// 存起来的完整记录(只给 Backend 内部用):多了一个「查询用的主机」,用来判断地址变了没有、要不要重新查。
/// 这个字段不会出现在任何对外的接口里。
/// </summary>
public sealed record NodeMeta(NodeInfo Info, string? GeoHost, long GeoAtMs, long UpdatedAtMs);
public static class NodeKinds
{
public const string Agent = "agent";
public const string Probe = "probe";
}
/// <summary>IP 归属查询的结果(只取需要的几项)。</summary>
public sealed record GeoInfo(string? Country, int? Asn, string? Organization, bool IsPrivate);
@@ -0,0 +1,219 @@
using System.Collections.Concurrent;
using System.Net;
using System.Net.Sockets;
using Monitor.Protocol;
namespace Monitor.Backend.Application;
/// <summary>
/// 节点的展示信息:系统和架构(节点自报),国家和运营商(Backend 用节点的来源地址查一次)。
/// <list type="bullet">
/// <item>Probe、Agent 不访问任何外部服务(客户的防火墙常常要审计),查询只由 Backend 发起。</item>
/// <item>只查公网地址:回环、内网、CGNAT、链路本地的地址一律不发出去。</item>
/// <item>查询结果只留国家代码和运营商;IP 不对外,也不和展示信息一起返回。</item>
/// </list>
/// </summary>
public sealed class NodeInfoService
{
/// <summary>同一个地址已经查过的结果保留多久,过期后在节点下次出现时重新查(运营商偶尔会变)。</summary>
private static readonly TimeSpan s_geoTtl = TimeSpan.FromDays(7);
/// <summary>查询失败之后多久再试。</summary>
private static readonly TimeSpan s_retryAfter = TimeSpan.FromMinutes(10);
private readonly INodeMetaStore _store;
private readonly IGeoLookup _geo;
private readonly TimeProvider _time;
private readonly ConcurrentDictionary<string, NodeMeta> _cache = new();
private readonly ConcurrentDictionary<string, byte> _inflight = new();
private Task? _load;
public NodeInfoService(INodeMetaStore store, IGeoLookup geo, TimeProvider time)
{
_store = store;
_geo = geo;
_time = time;
}
/// <summary>所有节点的展示信息(Agent 和 Probe)。</summary>
public async ValueTask<IReadOnlyList<NodeInfo>> ListAsync(CancellationToken ct)
{
await EnsureLoadedAsync(ct);
return [.. _cache.Values.Select(m => m.Info)];
}
/// <summary>节点自报的系统、架构和构建信息;没有变化就不写库。</summary>
public async ValueTask SetPlatformAsync(string nodeId, string kind, PlatformInfo? platform, CancellationToken ct)
{
if (platform is null)
{
return;
}
await EnsureLoadedAsync(ct);
NodeMeta current = _cache.TryGetValue(nodeId, out NodeMeta? found) ? found : Empty(nodeId, kind);
string? os = Clean(platform.Os, 16);
string? name = Clean(platform.Name, 40);
string? arch = Clean(platform.Arch, 16);
string? version = Clean(platform.Version, 32);
string? tag = Clean(platform.Tag, 64);
string? commit = Clean(platform.Commit, 40);
string? builtAt = Clean(platform.BuiltAt, 32);
NodeInfo i = current.Info;
if (i.Os == os && i.OsName == name && i.Arch == arch && i.Version == version && i.Tag == tag && i.Commit == commit && i.BuiltAt == builtAt)
{
return;
}
await SaveAsync(current with { Info = i with { Os = os, OsName = name, Arch = arch, Version = version, Tag = tag, Commit = commit, BuiltAt = builtAt }, UpdatedAtMs = Now() }, ct);
}
/// <summary>
/// 节点从某个地址连上来了:查一次它的国家和运营商。已经查过同一个地址(并且没过期)就什么也不做;
/// 私有地址不查;查询失败或服务关着,不影响别的,隔一阵再试。
/// </summary>
public async ValueTask ObserveAddressAsync(string nodeId, string kind, string? host, CancellationToken ct)
{
host = host?.Trim();
if (string.IsNullOrEmpty(host) || !IsPublic(host) || !_inflight.TryAdd(nodeId, 0))
{
return;
}
try
{
await EnsureLoadedAsync(ct);
NodeMeta current = _cache.TryGetValue(nodeId, out NodeMeta? found) ? found : Empty(nodeId, kind);
long now = Now();
bool same = string.Equals(current.GeoHost, host, StringComparison.OrdinalIgnoreCase);
bool fresh = current.Info.Country is not null && now - current.GeoAtMs < s_geoTtl.TotalMilliseconds;
bool recentlyFailed = current.Info.Country is null && now - current.GeoAtMs < s_retryAfter.TotalMilliseconds;
if (same && (fresh || recentlyFailed))
{
return;
}
GeoInfo? geo = await _geo.LookupAsync(host, ct);
if (geo is null || geo.IsPrivate || string.IsNullOrEmpty(geo.Country))
{
// 记下这次没查到(地址和时间),免得每次心跳都去查
await SaveAsync(current with { GeoHost = host, GeoAtMs = now, UpdatedAtMs = now }, ct);
return;
}
(string key, string name) = IspClassifier.Classify(geo.Asn, geo.Organization);
string country = geo.Country.ToUpperInvariant();
await SaveAsync(current with { Info = current.Info with { Country = country, IspKey = key, IspName = Clean(name, 40) }, GeoHost = host, GeoAtMs = now, UpdatedAtMs = now }, ct);
}
finally
{
_inflight.TryRemove(nodeId, out _);
}
}
/// <summary>Agent 报告某个 Probe 连着它:在后台用 Probe 的来源地址查国家和运营商(系统信息是 Probe 自己连上面板时报的)。出错只吞掉。</summary>
public void ObserveProbeInBackground(string probeId, string? remote) =>
_ = Task.Run(async () =>
{
try
{
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
await ObserveAddressAsync(probeId, NodeKinds.Probe, remote, cts.Token);
}
catch (Exception ex) when (ex is OperationCanceledException or InvalidOperationException or IOException)
{
// 展示信息拿不到也没关系,下次再试
}
});
/// <summary>Agent 打招呼(hello):记下它的系统信息,并用它的公开地址在后台查国家和运营商。</summary>
public void ObserveAgentInBackground(string agentId, string? publicHost, PlatformInfo? platform) =>
_ = Task.Run(async () =>
{
try
{
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
await SetPlatformAsync(agentId, NodeKinds.Agent, platform, cts.Token);
await ObserveAddressAsync(agentId, NodeKinds.Agent, publicHost, cts.Token);
}
catch (Exception ex) when (ex is OperationCanceledException or InvalidOperationException or IOException)
{
// 同上
}
});
public async ValueTask ForgetAsync(string nodeId, CancellationToken ct)
{
_cache.TryRemove(nodeId, out _);
await _store.DeleteAsync(nodeId, ct);
}
/// <summary>公网地址才值得查:回环、内网、CGNAT、链路本地、未指定的地址一律不发出去。域名放行(由查询服务自己解析)。</summary>
public static bool IsPublic(string host)
{
if (!IPAddress.TryParse(host, out IPAddress? ip))
{
return host.Contains('.', StringComparison.Ordinal) && !host.EndsWith(".local", StringComparison.OrdinalIgnoreCase) && !host.EndsWith(".internal", StringComparison.OrdinalIgnoreCase);
}
if (IPAddress.IsLoopback(ip) || ip.Equals(IPAddress.Any) || ip.Equals(IPAddress.IPv6Any) || ip.Equals(IPAddress.None))
{
return false;
}
if (ip.IsIPv4MappedToIPv6)
{
ip = ip.MapToIPv4();
}
if (ip.AddressFamily == AddressFamily.InterNetwork)
{
byte[] b = ip.GetAddressBytes();
return !(b[0] == 10
|| (b[0] == 172 && b[1] is >= 16 and <= 31)
|| (b[0] == 192 && b[1] == 168)
|| (b[0] == 169 && b[1] == 254)
|| (b[0] == 100 && b[1] is >= 64 and <= 127)
|| b[0] == 0
|| b[0] >= 224);
}
byte[] v6 = ip.GetAddressBytes();
bool linkLocal = v6[0] == 0xfe && (v6[1] & 0xc0) == 0x80;
bool uniqueLocal = (v6[0] & 0xfe) == 0xfc;
bool multicast = v6[0] == 0xff;
return !(linkLocal || uniqueLocal || multicast);
}
/// <summary>第一次用到时从库里读一次(只读一次,之后都在内存里)。</summary>
private async ValueTask EnsureLoadedAsync(CancellationToken ct)
{
Task load = Volatile.Read(ref _load) ?? Interlocked.CompareExchange(ref _load, LoadAsync(), null) ?? _load!;
await load.WaitAsync(ct);
}
private async Task LoadAsync()
{
foreach (NodeMeta meta in await _store.ListAsync(CancellationToken.None))
{
_cache.TryAdd(meta.Info.NodeId, meta);
}
}
private async ValueTask SaveAsync(NodeMeta meta, CancellationToken ct)
{
_cache[meta.Info.NodeId] = meta;
await _store.SaveAsync(meta, ct);
}
private long Now() => _time.GetUtcNow().ToUnixTimeMilliseconds();
private static NodeMeta Empty(string nodeId, string kind) => new(new NodeInfo(nodeId, kind, null, null, null, null, null, null), null, 0, 0);
/// <summary>节点自报的字符串只用于展示:去掉首尾空白,截断,没有内容就当没有。</summary>
private static string? Clean(string? text, int max)
{
string trimmed = text?.Trim() ?? string.Empty;
return trimmed.Length == 0 ? null : trimmed.Length > max ? trimmed[..max] : trimmed;
}
}
@@ -4,12 +4,15 @@ using Monitor.Protocol;
namespace Monitor.Backend.Application; namespace Monitor.Backend.Application;
/// <summary> /// <summary>
/// Application 通往 Agent 连接的端口(由 Gateway 实现):下发状态变化、发起按需测试、通知 Probe 列表变化。 /// Application 通往 Agent 连接的端口(由 Gateway 实现):下发状态变化、发起按需测试。
/// </summary> /// </summary>
public interface IAgentGateway public interface IAgentGateway
{ {
IReadOnlyCollection<NodeId> OnlineAgents { get; } IReadOnlyCollection<NodeId> OnlineAgents { get; }
/// <summary>所有在线的节点(Agent 和 Probe)。</summary>
IReadOnlyCollection<NodeId> OnlineNodes { get; }
bool IsOnline(NodeId agent); bool IsOnline(NodeId agent);
/// <summary>某个 Agent 的期望状态变了:Gateway 重新计算并推送。</summary> /// <summary>某个 Agent 的期望状态变了:Gateway 重新计算并推送。</summary>
@@ -17,9 +20,6 @@ public interface IAgentGateway
void NotifyAllTaskSetsChanged(); void NotifyAllTaskSetsChanged();
/// <summary>Agent 列表变了:通知所有在线 Agent 转发给 Probe。</summary>
ValueTask NotifyProbeAgentsChangedAsync(ProbeAgentsResult agents, CancellationToken ct);
/// <summary>失败时错误码为 <c>probe_offline</c>、<c>test_busy</c>、<c>agent_offline</c> 等。</summary> /// <summary>失败时错误码为 <c>probe_offline</c>、<c>test_busy</c>、<c>agent_offline</c> 等。</summary>
ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct); ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct);
@@ -29,4 +29,10 @@ public interface IAgentGateway
ValueTask<Result<bool>> DisconnectProbeAsync(NodeId agent, ProbeId probe, CancellationToken ct); ValueTask<Result<bool>> DisconnectProbeAsync(NodeId agent, ProbeId probe, CancellationToken ct);
ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct); ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct);
/// <summary>把发布清单和签名原样交给节点(通知,不等回复)。</summary>
ValueTask NotifyUpdateOfferAsync(NodeId node, string manifest, string signature, CancellationToken ct);
/// <summary>请求节点更新到它手里的清单。节点不在线返回 false。</summary>
ValueTask<bool> NotifyUpdateApplyAsync(NodeId node, CancellationToken ct);
} }
@@ -33,6 +33,22 @@ public interface IAgentStore
ValueTask SetAppliedTaskStateAsync(NodeId id, long appliedVersion, string? rejectedJson, CancellationToken ct); ValueTask SetAppliedTaskStateAsync(NodeId id, long appliedVersion, string? rejectedJson, CancellationToken ct);
} }
/// <summary>节点的展示信息(系统、架构、国家、运营商)。</summary>
public interface INodeMetaStore
{
ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct);
ValueTask SaveAsync(NodeMeta meta, CancellationToken ct);
ValueTask DeleteAsync(string nodeId, CancellationToken ct);
}
/// <summary>查一个 IP(或域名)的归属:国家、ASN、组织。实现负责超时和失败(失败返回 null,不抛)。</summary>
public interface IGeoLookup
{
ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct);
}
public interface IJoinTokenStore public interface IJoinTokenStore
{ {
ValueTask<JoinTokenInfo> CreateAsync(string id, byte[] tokenSha256, string? note, IReadOnlyList<string> labels, bool autoApprove, int maxUses, long expiresAtMs, long nowMs, CancellationToken ct); ValueTask<JoinTokenInfo> CreateAsync(string id, byte[] tokenSha256, string? note, IReadOnlyList<string> labels, bool autoApprove, int maxUses, long expiresAtMs, long nowMs, CancellationToken ct);
@@ -40,6 +56,8 @@ public interface IJoinTokenStore
ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct); ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct);
ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct); ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct);
ValueTask<bool> DeleteAsync(string id, CancellationToken ct);
} }
public interface IProbeStore public interface IProbeStore
@@ -39,19 +39,22 @@ public sealed class LineService
{ {
private readonly AgentService _agents; private readonly AgentService _agents;
private readonly IProbeStore _probes; private readonly IProbeStore _probes;
private readonly LinkService _links;
public LineService(AgentService agents, IProbeStore probes) public LineService(AgentService agents, IProbeStore probes, LinkService links)
{ {
_links = links;
_agents = agents; _agents = agents;
_probes = probes; _probes = probes;
} }
/// <summary>所有线路:每个 Agent 和每个<b>未吊销</b>的 Probe 配成一对。</summary> /// <summary>所有线路:每个 Agent 和每个<b>未吊销</b>的 Probe 配成一对,只留下线路开着的(见 <see cref="LinkService"/>);关掉的线路历史数据还在,按 id 仍然查得到。</summary>
public async ValueTask<IReadOnlyList<LineView>> ListAsync(Func<AgentView, bool> include, CancellationToken ct) public async ValueTask<IReadOnlyList<LineView>> ListAsync(Func<AgentView, bool> include, CancellationToken ct)
{ {
IReadOnlyList<AgentView> agents = await _agents.ListAsync(ct); IReadOnlyList<AgentView> agents = await _agents.ListAsync(ct);
IReadOnlyList<ProbeRecord> probes = [.. (await _probes.ListAsync(ct)).Where(p => p.RevokedAtMs is null)]; IReadOnlyList<ProbeRecord> probes = [.. (await _probes.ListAsync(ct)).Where(p => p.RevokedAtMs is null)];
return [.. agents.Where(include).SelectMany(a => probes.Select(p => new LineView(a, p)))]; LinkSnapshot links = await _links.SnapshotAsync(ct);
return [.. agents.Where(include).SelectMany(a => probes.Where(p => links.IsLinked(p.Id, a.Agent.Id.Value)).Select(p => new LineView(a, p)))];
} }
/// <summary> /// <summary>
@@ -20,10 +20,12 @@ public sealed class TaskPlanner : IDisposable
private readonly IAgentStore _agents; private readonly IAgentStore _agents;
private readonly IProbeStore _probes; private readonly IProbeStore _probes;
private readonly ISettingsStore _settings; private readonly ISettingsStore _settings;
private readonly LinkService _links;
private readonly SemaphoreSlim _gate = new(1, 1); private readonly SemaphoreSlim _gate = new(1, 1);
public TaskPlanner(ITaskStore tasks, IAgentStore agents, IProbeStore probes, ISettingsStore settings) public TaskPlanner(ITaskStore tasks, IAgentStore agents, IProbeStore probes, ISettingsStore settings, LinkService links)
{ {
_links = links;
_tasks = tasks; _tasks = tasks;
_agents = agents; _agents = agents;
_probes = probes; _probes = probes;
@@ -41,9 +43,9 @@ public sealed class TaskPlanner : IDisposable
IReadOnlyList<TaskRecord> all = await _tasks.ListAsync(ct); IReadOnlyList<TaskRecord> all = await _tasks.ListAsync(ct);
IReadOnlyList<ProbeRecord> probes = await _probes.ListAsync(ct); IReadOnlyList<ProbeRecord> probes = await _probes.ListAsync(ct);
long listVersion = long.TryParse(await _settings.GetAsync(ProbeAgentsVersionKey, ct), out long v) ? v : 0; LinkSnapshot links = await _links.SnapshotAsync(ct);
var document = new TaskSet { ProbeAgentsVersion = listVersion }; var document = new TaskSet();
foreach (EffectiveTask task in EffectiveTasks.Compute(all, agent, filterByAgentKinds: true)) foreach (EffectiveTask task in EffectiveTasks.Compute(all, agent, filterByAgentKinds: true))
{ {
document.Tasks.Add(new TaskSpec document.Tasks.Add(new TaskSpec
@@ -58,7 +60,7 @@ public sealed class TaskPlanner : IDisposable
}); });
} }
foreach (ProbeRecord probe in probes.Where(p => p.RevokedAtMs is null)) foreach (ProbeRecord probe in probes.Where(p => p.RevokedAtMs is null && links.IsLinked(p.Id, agentId.Value)))
{ {
document.Peers.Add(new PeerSpec { ProbeId = new ProbeId(probe.Id), PublicKey = Convert.ToBase64String(probe.PublicKey) }); document.Peers.Add(new PeerSpec { ProbeId = new ProbeId(probe.Id), PublicKey = Convert.ToBase64String(probe.PublicKey) });
} }
@@ -9,6 +9,7 @@ namespace Monitor.Backend.Application;
/// <summary>按需测试(once):校验与排队、下发给 Agent、状态更新、结果查询。</summary> /// <summary>按需测试(once):校验与排队、下发给 Agent、状态更新、结果查询。</summary>
public sealed class TestService public sealed class TestService
{ {
private readonly double _phaseScale;
private readonly ITestStore _tests; private readonly ITestStore _tests;
private readonly IAgentStore _agents; private readonly IAgentStore _agents;
private readonly AgentRuntime _runtime; private readonly AgentRuntime _runtime;
@@ -16,8 +17,9 @@ public sealed class TestService
private readonly LiveHub _live; private readonly LiveHub _live;
private readonly TimeProvider _time; private readonly TimeProvider _time;
public TestService(ITestStore tests, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, LiveHub live, TimeProvider time) public TestService(ITestStore tests, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, LiveHub live, TimeProvider time, BackendOptions options)
{ {
_phaseScale = options.PhaseDurationScale;
_tests = tests; _tests = tests;
_agents = agents; _agents = agents;
_runtime = runtime; _runtime = runtime;
@@ -52,7 +54,7 @@ public sealed class TestService
return Result<TestRecord>.Fail(FleetErrors.Unsupported, $"this agent does not support {unsupported}, which the {plan.Id} plan needs"); return Result<TestRecord>.Fail(FleetErrors.Unsupported, $"this agent does not support {unsupported}, which the {plan.Id} plan needs");
} }
Result<List<TestPhase>> phases = BuildPhases(plan, request.Params, agent); Result<List<TestPhase>> phases = BuildPhases(plan, request.Params, agent, _phaseScale);
if (!phases.TryGet(out List<TestPhase>? phaseList, out FleetError? error)) if (!phases.TryGet(out List<TestPhase>? phaseList, out FleetError? error))
{ {
return Result<TestRecord>.Fail(error); return Result<TestRecord>.Fail(error);
@@ -173,7 +175,7 @@ public sealed class TestService
public static bool IsTerminal(string status) => status is TestStatuses.Succeeded or TestStatuses.Failed or TestStatuses.Aborted; public static bool IsTerminal(string status) => status is TestStatuses.Succeeded or TestStatuses.Failed or TestStatuses.Aborted;
private static Result<List<TestPhase>> BuildPhases(PlanSpec plan, IReadOnlyDictionary<string, JsonElement>? overrides, AgentRecord agent) private static Result<List<TestPhase>> BuildPhases(PlanSpec plan, IReadOnlyDictionary<string, JsonElement>? overrides, AgentRecord agent, double phaseScale)
{ {
if (overrides is not null) if (overrides is not null)
{ {
@@ -245,7 +247,7 @@ public sealed class TestService
return Result<List<TestPhase>>.Fail(MonitorErrors.BudgetExceeded, $"{phase.Kind} needs {bits / 1_000_000.0:0.#} Mbit/s but the agent allows {limitMbps} Mbit/s"); return Result<List<TestPhase>>.Fail(MonitorErrors.BudgetExceeded, $"{phase.Kind} needs {bits / 1_000_000.0:0.#} Mbit/s but the agent allows {limitMbps} Mbit/s");
} }
int duration = resolved.Values!.Values.TryGetValue("duration_s", out JsonElement d) && d.TryGetInt64(out long seconds) ? (int)seconds : phase.DurationSeconds; int duration = resolved.Values!.Values.TryGetValue("duration_s", out JsonElement d) && d.TryGetInt64(out long seconds) ? (int)seconds : Math.Max(1, (int)Math.Round(phase.DurationSeconds * phaseScale));
phases.Add(new TestPhase { Kind = phase.Kind, Params = new Dictionary<string, JsonElement>(resolved.Values.Values), DurationSeconds = duration }); phases.Add(new TestPhase { Kind = phase.Kind, Params = new Dictionary<string, JsonElement>(resolved.Values.Values), DurationSeconds = duration });
} }
@@ -0,0 +1,158 @@
using System.Collections.Concurrent;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using System.Text.RegularExpressions;
using Fleet.Core;
using Monitor.Protocol;
namespace Monitor.Backend.Application;
/// <summary>面板记下的、CI 签名的发布清单(只留每个组件最新的一份)。<c>Manifest</c> 是签名对象的原始文本,面板一个字节也不改。</summary>
public sealed record ReleaseOffer(string Component, long Seq, string Tag, string Version, string Digest, string Manifest, string Signature);
/// <summary>
/// 发布清单的分发:CI 签名之后推给面板,面板存下来并原样转给在线的 Agent、Probe(节点连上时再补发一次)。
/// <b>面板不验签、也不决定拉什么</b>:节点宿主机上的更新器用本机的发布公钥验签,并且只认本机配置的镜像仓库;
/// 所以面板即使被攻破,也推不出一个能被装上的假镜像。面板只负责「通知」和「请求更新」,状态由节点随心跳报回来。
/// </summary>
public sealed partial class ReleaseService
{
public const int MaxManifestBytes = 4096;
public const int MaxSignatureBytes = 512;
private static readonly string[] s_components = [NodeKinds.Agent, NodeKinds.Probe];
private readonly ISettingsStore _settings;
private readonly IAgentGateway _gateway;
private readonly ConcurrentDictionary<string, UpdateStatus> _status = new();
public ReleaseService(ISettingsStore settings, IAgentGateway gateway)
{
_settings = settings;
_gateway = gateway;
}
/// <summary>登记一份新清单并推给在线的同类节点。只要格式对、<c>seq</c> 比现有的新就收下(签名由节点验证)。</summary>
public async ValueTask<Result<ReleaseOffer>> PublishAsync(string manifest, string signature, CancellationToken ct)
{
if (!TryParse(manifest, signature, out ReleaseOffer? offer, out string? problem))
{
return ValidationError.Fail<ReleaseOffer>("manifest", problem);
}
ReleaseOffer? current = await GetAsync(offer.Component, ct);
if (current is not null && offer.Seq <= current.Seq)
{
return Result<ReleaseOffer>.Fail(MonitorErrors.NameConflict, $"a release with seq {current.Seq} or newer is already registered");
}
await _settings.SetAsync(Key(offer.Component, "manifest"), offer.Manifest, ct);
await _settings.SetAsync(Key(offer.Component, "signature"), offer.Signature, ct);
foreach (NodeId node in OnlineOf(offer.Component))
{
await _gateway.NotifyUpdateOfferAsync(node, offer.Manifest, offer.Signature, ct);
}
return Result<ReleaseOffer>.Ok(offer);
}
public async ValueTask<ReleaseOffer?> GetAsync(string component, CancellationToken ct)
{
string? manifest = await _settings.GetAsync(Key(component, "manifest"), ct);
string? signature = await _settings.GetAsync(Key(component, "signature"), ct);
return manifest is not null && signature is not null && TryParse(manifest, signature, out ReleaseOffer? offer, out _) && offer.Component == component ? offer : null;
}
public async ValueTask<IReadOnlyList<ReleaseOffer>> ListAsync(CancellationToken ct)
{
var list = new List<ReleaseOffer>();
foreach (string component in s_components)
{
if (await GetAsync(component, ct) is { } offer)
{
list.Add(offer);
}
}
return list;
}
/// <summary>节点刚连上:把它这个组件最新的清单补发一份(节点离线时发布的也不会错过)。</summary>
public async ValueTask OfferToNodeAsync(NodeId node, CancellationToken ct)
{
if (ComponentOf(node) is { } component && await GetAsync(component, ct) is { } offer)
{
await _gateway.NotifyUpdateOfferAsync(node, offer.Manifest, offer.Signature, ct);
}
}
/// <summary>请求某个节点更新到它手里那份(验过签的)清单。节点不在线返回 false。</summary>
public ValueTask<bool> RequestUpdateAsync(NodeId node, CancellationToken ct) => _gateway.NotifyUpdateApplyAsync(node, ct);
public void RecordStatus(NodeId node, UpdateStatus? status)
{
if (status is null)
{
_status.TryRemove(node.Value, out _);
}
else
{
_status[node.Value] = status;
}
}
public UpdateStatus? StatusOf(NodeId node) => _status.TryGetValue(node.Value, out UpdateStatus? status) ? status : null;
public void Forget(NodeId node) => _status.TryRemove(node.Value, out _);
private IEnumerable<NodeId> OnlineOf(string component) => _gateway.OnlineNodes.Where(n => ComponentOf(n) == component);
private static string? ComponentOf(NodeId node) =>
node.Value.StartsWith(AgentIds.Prefix, StringComparison.Ordinal) ? NodeKinds.Agent
: node.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal) ? NodeKinds.Probe
: null;
private static string Key(string component, string part) => $"release.{component}.{part}";
private static bool TryParse(string manifest, string signature, [NotNullWhen(true)] out ReleaseOffer? offer, [NotNullWhen(false)] out string? problem)
{
offer = null;
problem = null;
if (manifest.Length is 0 or > MaxManifestBytes || signature.Length is 0 or > MaxSignatureBytes || !Base64().IsMatch(signature))
{
problem = "manifest or signature has an unexpected size or encoding";
return false;
}
try
{
using JsonDocument document = JsonDocument.Parse(manifest);
JsonElement root = document.RootElement;
string component = Text(root, "component");
string digest = Text(root, "digest");
if (root.ValueKind != JsonValueKind.Object || Array.IndexOf(s_components, component) < 0
|| !root.TryGetProperty("seq", out JsonElement seq) || !seq.TryGetInt64(out long seqValue) || seqValue <= 0
|| !Digest().IsMatch(digest) || Text(root, "image").Length is 0 or > 200)
{
problem = "manifest needs a known component, a positive seq, an image and a sha256 digest";
return false;
}
offer = new ReleaseOffer(component, seqValue, Text(root, "tag"), Text(root, "version"), digest, manifest, signature);
return true;
}
catch (JsonException)
{
problem = "manifest is not valid JSON";
return false;
}
}
private static string Text(JsonElement root, string name) =>
root.ValueKind == JsonValueKind.Object && root.TryGetProperty(name, out JsonElement value) && value.ValueKind == JsonValueKind.String ? value.GetString() ?? "" : "";
[GeneratedRegex("^[A-Za-z0-9+/=]+$")]
private static partial Regex Base64();
[GeneratedRegex("^sha256:[0-9a-f]{64}$")]
private static partial Regex Digest();
}
@@ -24,7 +24,9 @@ public sealed class FleetGateway : IAgentGateway
private IStateNotifier Notifier => _services.GetRequiredService<IStateNotifier>(); private IStateNotifier Notifier => _services.GetRequiredService<IStateNotifier>();
public IReadOnlyCollection<NodeId> OnlineAgents => Connections.OnlineNodes; public IReadOnlyCollection<NodeId> OnlineAgents => [.. Connections.OnlineNodes.Where(n => n.Value.StartsWith(AgentIds.Prefix, StringComparison.Ordinal))];
public IReadOnlyCollection<NodeId> OnlineNodes => Connections.OnlineNodes;
public bool IsOnline(NodeId agent) => Connections.IsOnline(agent); public bool IsOnline(NodeId agent) => Connections.IsOnline(agent);
@@ -32,15 +34,6 @@ public sealed class FleetGateway : IAgentGateway
public void NotifyAllTaskSetsChanged() => Notifier.NotifyAllChanged(); public void NotifyAllTaskSetsChanged() => Notifier.NotifyAllChanged();
public async ValueTask NotifyProbeAgentsChangedAsync(ProbeAgentsResult agents, CancellationToken ct)
{
IFleetConnections connections = Connections;
foreach (NodeId node in connections.OnlineNodes)
{
await connections.NotifyAsync(node, MonitorMethods.ProbeAgentsChanged, agents, MonitorJsonContext.Default.ProbeAgentsResult, ct);
}
}
public async ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct) public async ValueTask<Result<bool>> RunTestAsync(NodeId agent, TestRunParams run, CancellationToken ct)
{ {
Result<TestRunResult> result = await Connections.RequestAsync( Result<TestRunResult> result = await Connections.RequestAsync(
@@ -70,4 +63,10 @@ public sealed class FleetGateway : IAgentGateway
} }
public ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct) => Connections.DisconnectAsync(agent, reason, ct); public ValueTask DisconnectAsync(NodeId agent, string reason, CancellationToken ct) => Connections.DisconnectAsync(agent, reason, ct);
public async ValueTask NotifyUpdateOfferAsync(NodeId node, string manifest, string signature, CancellationToken ct) =>
await Connections.NotifyAsync(node, MonitorMethods.UpdateOffer, new UpdateOfferParams { Manifest = manifest, Signature = signature }, MonitorJsonContext.Default.UpdateOfferParams, ct);
public async ValueTask<bool> NotifyUpdateApplyAsync(NodeId node, CancellationToken ct) =>
await Connections.NotifyAsync(node, MonitorMethods.UpdateApply, new UpdateOfferParams(), MonitorJsonContext.Default.UpdateOfferParams, ct);
} }
@@ -20,6 +20,7 @@ public static class GatewayServiceCollectionExtensions
services.AddFleetEnrollment(); services.AddFleetEnrollment();
services.AddFleetServer(options => options.NodeIdPrefix = AgentIds.Prefix) services.AddFleetServer(options => options.NodeIdPrefix = AgentIds.Prefix)
.AddState<TaskSetStateProvider>() .AddState<TaskSetStateProvider>()
.AddState<ProbeAgentsStateProvider>()
.AddInbox<RawChunkInbox>() .AddInbox<RawChunkInbox>()
.MapMethods(MapMonitorMethods); .MapMethods(MapMonitorMethods);
return services; return services;
@@ -39,11 +40,6 @@ public static class GatewayServiceCollectionExtensions
.OnNotify( .OnNotify(
MonitorMethods.TestStatus, MonitorMethods.TestStatus,
MonitorJsonContext.Default.TestStatusParams, MonitorJsonContext.Default.TestStatusParams,
(context, parameters, ct) => services.GetRequiredService<TestService>().OnStatusAsync(context.Node.Id, parameters, ct)) (context, parameters, ct) => services.GetRequiredService<TestService>().OnStatusAsync(context.Node.Id, parameters, ct));
.On(
MonitorMethods.ProbeAgents,
MonitorJsonContext.Default.ProbeAgentsParams,
MonitorJsonContext.Default.ProbeAgentsResult,
async (context, parameters, ct) => Result<ProbeAgentsResult>.Ok(await services.GetRequiredService<AgentService>().ProbeAgentsAsync(ct)));
} }
} }
@@ -0,0 +1,33 @@
using System.Text.Json;
using Fleet.Rpc;
using Fleet.Server;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Gateway;
/// <summary>期望状态:为每个 Probe 下发要连接的 Agent 列表(地址、端口、证书指纹)。Agent 集合变化时由 <c>NotifyAllChanged</c> 一起重算推送。</summary>
public sealed class ProbeAgentsStateProvider : IStateProvider
{
private readonly AgentService _agents;
public ProbeAgentsStateProvider(AgentService agents)
{
_agents = agents;
}
public string Type => ProbeAgentList.StateType;
public async ValueTask<StateDocument?> GetAsync(NodeRecord node, CancellationToken ct)
{
if (!node.Id.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
return null;
}
ProbeAgentList list = await _agents.ProbeAgentsAsync(node.Id.Value, ct);
return new StateDocument(list.Version, JsonSerializer.SerializeToElement(list, MonitorJsonContext.Default.ProbeAgentList));
}
public ValueTask OnAppliedAsync(NodeRecord node, long version, StateSyncResult result, CancellationToken ct) => ValueTask.CompletedTask;
}
@@ -26,6 +26,11 @@ public sealed class TaskSetStateProvider : IStateProvider
public async ValueTask<StateDocument?> GetAsync(NodeRecord node, CancellationToken ct) public async ValueTask<StateDocument?> GetAsync(NodeRecord node, CancellationToken ct)
{ {
if (node.Id.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
return null; // Probe 不运行任务集
}
DesiredTaskSet? desired = await _planner.GetDesiredAsync(node.Id, ct); DesiredTaskSet? desired = await _planner.GetDesiredAsync(node.Id, ct);
return desired is null ? null : new StateDocument(desired.Version, JsonSerializer.SerializeToElement(desired.Document, MonitorJsonContext.Default.TaskSet)); return desired is null ? null : new StateDocument(desired.Version, JsonSerializer.SerializeToElement(desired.Document, MonitorJsonContext.Default.TaskSet));
} }
@@ -5,7 +5,7 @@ namespace Monitor.Backend.Storage.Sqlite;
/// <summary>表结构。版本记录在 <c>PRAGMA user_version</c>;以后的迁移是按版本号追加的内置 SQL。</summary> /// <summary>表结构。版本记录在 <c>PRAGMA user_version</c>;以后的迁移是按版本号追加的内置 SQL。</summary>
internal static class Schema internal static class Schema
{ {
public const int CurrentVersion = 3; public const int CurrentVersion = 5;
public static void Migrate(SqliteConnection connection) public static void Migrate(SqliteConnection connection)
{ {
@@ -55,6 +55,41 @@ internal static class Schema
connection.Exec("PRAGMA user_version = 3"); connection.Exec("PRAGMA user_version = 3");
transaction.Commit(); transaction.Commit();
} }
if (version < 4)
{
// 节点的展示信息:系统、架构、国家、运营商;geo_host 是查询用的地址,只在 Backend 内部用
using SqliteTransaction transaction = connection.BeginTransaction();
connection.Exec("""
CREATE TABLE node_meta (
node_id TEXT PRIMARY KEY,
kind TEXT NOT NULL,
os TEXT,
os_name TEXT,
arch TEXT,
country TEXT,
isp_key TEXT,
isp_name TEXT,
geo_host TEXT,
geo_at INTEGER NOT NULL DEFAULT 0,
updated_at INTEGER NOT NULL DEFAULT 0
) WITHOUT ROWID
""");
connection.Exec("PRAGMA user_version = 4");
transaction.Commit();
}
if (version < 5)
{
// 节点自报的构建信息:版本、tag、提交、构建时间
using SqliteTransaction transaction = connection.BeginTransaction();
connection.Exec("ALTER TABLE node_meta ADD COLUMN version TEXT");
connection.Exec("ALTER TABLE node_meta ADD COLUMN tag TEXT");
connection.Exec("ALTER TABLE node_meta ADD COLUMN commit_id TEXT");
connection.Exec("ALTER TABLE node_meta ADD COLUMN built_at TEXT");
connection.Exec("PRAGMA user_version = 5");
transaction.Commit();
}
} }
/// <summary> /// <summary>
@@ -166,8 +166,17 @@ public sealed class SqliteAgentStore : IAgentStore, IFleetStore
return Result<NodeRecord>.Ok(ToNode(GetInternal(c, node.Id)!, node.MetaJson)); return Result<NodeRecord>.Ok(ToNode(GetInternal(c, node.Id)!, node.MetaJson));
}, ct); }, ct);
/// <summary>Agent 在 agents 表里;Probe 的身份在 probes 表里(吊销之后按「禁用」处理,连不上)。</summary>
async ValueTask<NodeRecord?> IFleetStore.GetNodeAsync(NodeId id, CancellationToken ct) => async ValueTask<NodeRecord?> IFleetStore.GetNodeAsync(NodeId id, CancellationToken ct) =>
await GetAsync(id, ct) is { } agent ? ToNode(agent, null) : null; await GetAsync(id, ct) is { } agent
? ToNode(agent, null)
: await _db.ReadAsync(
c => c.QueryOne(
"SELECT id, name, pubkey, created_at, revoked_at FROM probes WHERE id = @id",
r => new NodeRecord(
new NodeId(r.GetString(0)), r.GetString(1), r.IsDBNull(4) ? NodeStatus.Active : NodeStatus.Disabled, r.Bytes(2), [], [], null, null, r.GetInt64(3), null),
("@id", id.Value)),
ct);
public ValueTask<NodeRecord?> FindEstablishedNodeByNameAsync(string name, CancellationToken ct) => public ValueTask<NodeRecord?> FindEstablishedNodeByNameAsync(string name, CancellationToken ct) =>
_db.ReadAsync( _db.ReadAsync(
@@ -34,6 +34,9 @@ public sealed class SqliteJoinTokenStore : IJoinTokenStore
public ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct) => public ValueTask<bool> RevokeAsync(string id, long nowMs, CancellationToken ct) =>
_db.WriteAsync(c => c.Exec("UPDATE join_tokens SET revoked_at = @now WHERE id = @id AND revoked_at IS NULL", ("@id", id), ("@now", nowMs)) == 1, ct); _db.WriteAsync(c => c.Exec("UPDATE join_tokens SET revoked_at = @now WHERE id = @id AND revoked_at IS NULL", ("@id", id), ("@now", nowMs)) == 1, ct);
public ValueTask<bool> DeleteAsync(string id, CancellationToken ct) =>
_db.WriteAsync(c => c.Exec("DELETE FROM join_tokens WHERE id = @id", ("@id", id)) == 1, ct);
} }
public sealed class SqliteProbeStore : IProbeStore public sealed class SqliteProbeStore : IProbeStore
@@ -0,0 +1,41 @@
using Microsoft.Data.Sqlite;
using Monitor.Backend.Application;
namespace Monitor.Backend.Storage.Sqlite;
public sealed class SqliteNodeMetaStore : INodeMetaStore
{
private readonly SqliteDatabase _db;
public SqliteNodeMetaStore(SqliteDatabase db)
{
_db = db;
}
public ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct) =>
_db.ReadAsync<IReadOnlyList<NodeMeta>>(
c => c.Query("SELECT node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at FROM node_meta", Read),
ct);
public ValueTask SaveAsync(NodeMeta meta, CancellationToken ct) =>
_db.ExecuteAsync(
c => c.Exec(
"INSERT INTO node_meta (node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at) " +
"VALUES (@id, @kind, @os, @osn, @arch, @cc, @ik, @in, @gh, @ga, @ua, @ver, @tag, @cm, @bt) ON CONFLICT(node_id) DO UPDATE SET " +
"kind = excluded.kind, os = excluded.os, os_name = excluded.os_name, arch = excluded.arch, country = excluded.country, " +
"isp_key = excluded.isp_key, isp_name = excluded.isp_name, geo_host = excluded.geo_host, geo_at = excluded.geo_at, updated_at = excluded.updated_at, " +
"version = excluded.version, tag = excluded.tag, commit_id = excluded.commit_id, built_at = excluded.built_at",
("@id", meta.Info.NodeId), ("@kind", meta.Info.Kind), ("@os", meta.Info.Os), ("@osn", meta.Info.OsName), ("@arch", meta.Info.Arch), ("@cc", meta.Info.Country),
("@ik", meta.Info.IspKey), ("@in", meta.Info.IspName), ("@gh", meta.GeoHost), ("@ga", meta.GeoAtMs), ("@ua", meta.UpdatedAtMs),
("@ver", meta.Info.Version), ("@tag", meta.Info.Tag), ("@cm", meta.Info.Commit), ("@bt", meta.Info.BuiltAt)),
ct);
public ValueTask DeleteAsync(string nodeId, CancellationToken ct) =>
_db.ExecuteAsync(c => c.Exec("DELETE FROM node_meta WHERE node_id = @id", ("@id", nodeId)), ct);
private static NodeMeta Read(SqliteDataReader r)
{
static string? S(SqliteDataReader reader, int i) => reader.IsDBNull(i) ? null : reader.GetString(i);
return new NodeMeta(new NodeInfo(r.GetString(0), r.GetString(1), S(r, 2), S(r, 3), S(r, 4), S(r, 5), S(r, 6), S(r, 7), S(r, 11), S(r, 12), S(r, 13), S(r, 14)), S(r, 8), r.GetInt64(9), r.GetInt64(10));
}
}
@@ -20,6 +20,7 @@ public static class StorageServiceCollectionExtensions
services.AddSingleton<ITestStore, SqliteTestStore>(); services.AddSingleton<ITestStore, SqliteTestStore>();
services.AddSingleton<IEventStore, SqliteEventStore>(); services.AddSingleton<IEventStore, SqliteEventStore>();
services.AddSingleton<IHostStore, SqliteHostStore>(); services.AddSingleton<IHostStore, SqliteHostStore>();
services.AddSingleton<INodeMetaStore, SqliteNodeMetaStore>();
services.AddSingleton<IAuthStore, SqliteAuthStore>(); services.AddSingleton<IAuthStore, SqliteAuthStore>();
services.AddSingleton<IRawStore, SqliteRawStore>(); services.AddSingleton<IRawStore, SqliteRawStore>();
services.AddSingleton<IMetricStore, SqliteMetricStore>(); services.AddSingleton<IMetricStore, SqliteMetricStore>();
@@ -38,6 +38,16 @@ public static class BackendHost
IServiceCollection services = builder.Services; IServiceCollection services = builder.Services;
services.AddSingleton(time ?? TimeProvider.System); services.AddSingleton(time ?? TimeProvider.System);
services.AddMonitorSqlite(options.DataDirectory); services.AddMonitorSqlite(options.DataDirectory);
// 国家和运营商:Backend 用节点的来源地址查一次(Probe、Agent 不访问任何外部服务);地址留空就不查
if (string.IsNullOrWhiteSpace(options.GeoLookupUrl))
{
services.AddSingleton<IGeoLookup, NoGeoLookup>();
}
else
{
services.AddSingleton<IGeoLookup>(new HttpGeoLookup(options.GeoLookupUrl));
}
services.AddMonitorApplication(options); services.AddMonitorApplication(options);
services.AddMonitorGateway(); services.AddMonitorGateway();
services.AddMonitorApi(); services.AddMonitorApi();
@@ -0,0 +1,73 @@
using System.Text.Json;
using Monitor.Backend.Application;
namespace Monitor.Backend;
/// <summary>
/// 用 IP 归属查询服务(<c>GET {地址}/lookup?ip=…</c>)查国家和运营商。
/// 失败(超时、网络不通、服务返回错误)一律返回 null,不往外抛:展示信息没有也不影响别的功能。
/// </summary>
public sealed class HttpGeoLookup : IGeoLookup, IDisposable
{
private readonly HttpClient _http;
public HttpGeoLookup(string baseUrl)
{
_http = new HttpClient { BaseAddress = new Uri(baseUrl.TrimEnd('/') + "/"), Timeout = TimeSpan.FromSeconds(6) };
_http.DefaultRequestHeaders.UserAgent.ParseAdd("MyMonitor");
}
public async ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct)
{
try
{
using HttpResponseMessage response = await _http.GetAsync("lookup?ip=" + Uri.EscapeDataString(host), ct);
if (!response.IsSuccessStatusCode)
{
return null;
}
using JsonDocument doc = await JsonDocument.ParseAsync(await response.Content.ReadAsStreamAsync(ct), cancellationToken: ct);
return Parse(doc.RootElement);
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException or InvalidOperationException)
{
return null;
}
}
/// <summary>只取国家代码、ASN、组织名;其余字段(城市、经纬度、网段……)一概不读。</summary>
public static GeoInfo? Parse(JsonElement root)
{
if (root.TryGetProperty("found", out JsonElement found) && found.ValueKind == JsonValueKind.False)
{
return null;
}
bool isPrivate = root.TryGetProperty("is_private", out JsonElement priv) && priv.ValueKind == JsonValueKind.True;
string? country = root.TryGetProperty("country_region", out JsonElement region) && region.TryGetProperty("iso_code", out JsonElement code) && code.ValueKind == JsonValueKind.String ? code.GetString() : null;
int? asn = null;
string? organization = null;
if (root.TryGetProperty("network", out JsonElement network))
{
if (network.TryGetProperty("asn", out JsonElement asnValue) && asnValue.TryGetInt32(out int number))
{
asn = number;
}
string? isp = network.TryGetProperty("isp", out JsonElement ispValue) && ispValue.ValueKind == JsonValueKind.String ? ispValue.GetString() : null;
string? org = network.TryGetProperty("organization", out JsonElement orgValue) && orgValue.ValueKind == JsonValueKind.String ? orgValue.GetString() : null;
organization = string.IsNullOrWhiteSpace(isp) ? org : isp;
}
return new GeoInfo(country, asn, organization, isPrivate);
}
public void Dispose() => _http.Dispose();
}
/// <summary>不查(<c>Monitor__GeoLookupUrl</c> 留空)。</summary>
public sealed class NoGeoLookup : IGeoLookup
{
public ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct) => ValueTask.FromResult<GeoInfo?>(null);
}
@@ -1,9 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk.Web"> <Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk.Web" />
<Import Project="Sdk.props" Sdk="StuDev.AotAnywhere" Condition="'$(UseAotAnywhere)' == 'true'" />
<PropertyGroup> <PropertyGroup>
<OutputType>Exe</OutputType> <OutputType>Exe</OutputType>
<AssemblyName>monitor-backend</AssemblyName> <AssemblyName>monitor-backend</AssemblyName>
<RootNamespace>Monitor.Backend</RootNamespace> <RootNamespace>Monitor.Backend</RootNamespace>
<IsAotCompatible>true</IsAotCompatible> <IsAotCompatible>true</IsAotCompatible>
<PublishAot>true</PublishAot>
<OptimizationPreference>Size</OptimizationPreference>
<StackTraceSupport>false</StackTraceSupport>
<EventSourceSupport>false</EventSourceSupport>
<DebuggerSupport>false</DebuggerSupport>
<MetadataUpdaterSupport>false</MetadataUpdaterSupport>
<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
<IlcFoldIdenticalMethodBodies>true</IlcFoldIdenticalMethodBodies>
<StripSymbols>true</StripSymbols>
<!-- This service exposes APIs only; there are no static web assets to publish. -->
<StaticWebAssetsEnabled>false</StaticWebAssetsEnabled>
<SqliteStaticLink Condition="'$(SqliteStaticLink)' == '' and '$(PublishAot)' == 'true' and '$(RuntimeIdentifier)' != ''">true</SqliteStaticLink>
<EnableRequestDelegateGenerator>true</EnableRequestDelegateGenerator> <EnableRequestDelegateGenerator>true</EnableRequestDelegateGenerator>
<EnableConfigurationBindingGenerator>true</EnableConfigurationBindingGenerator> <EnableConfigurationBindingGenerator>true</EnableConfigurationBindingGenerator>
<OpenApiGenerateDocuments>true</OpenApiGenerateDocuments> <OpenApiGenerateDocuments>true</OpenApiGenerateDocuments>
@@ -12,8 +26,13 @@
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Microsoft.Extensions.ApiDescription.Server" PrivateAssets="all" /> <PackageReference Include="Microsoft.Extensions.ApiDescription.Server" PrivateAssets="all" />
<PackageReference Include="SQLitePCLRaw.lib.e_sqlite3" Condition="'$(SqliteStaticLink)' != 'true'" />
<PackageReference Include="SQLitePCLRaw.lib.e_sqlite3" ExcludeAssets="native" Condition="'$(SqliteStaticLink)' == 'true'" />
<ProjectReference Include="..\Monitor.Backend.Api\Monitor.Backend.Api.csproj" /> <ProjectReference Include="..\Monitor.Backend.Api\Monitor.Backend.Api.csproj" />
<ProjectReference Include="..\Monitor.Backend.Gateway\Monitor.Backend.Gateway.csproj" /> <ProjectReference Include="..\Monitor.Backend.Gateway\Monitor.Backend.Gateway.csproj" />
<ProjectReference Include="..\Monitor.Backend.Storage.Sqlite\Monitor.Backend.Storage.Sqlite.csproj" /> <ProjectReference Include="..\Monitor.Backend.Storage.Sqlite\Monitor.Backend.Storage.Sqlite.csproj" />
</ItemGroup> </ItemGroup>
<Import Project="StaticSqlite.targets" />
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk.Web" />
<Import Project="Sdk.targets" Sdk="StuDev.AotAnywhere" Condition="'$(UseAotAnywhere)' == 'true'" />
</Project> </Project>
@@ -0,0 +1,71 @@
<Project>
<PropertyGroup>
<_SqliteToolchainTargets>SetupOSSpecificProps</_SqliteToolchainTargets>
<_SqliteToolchainTargets Condition="'$(UseAotAnywhere)' == 'true'">SetupOSSpecificProps;OverwriteTargetTriple;SetPathToZig</_SqliteToolchainTargets>
</PropertyGroup>
<ItemGroup Condition="'$(SqliteStaticLink)' == 'true'">
<DirectPInvoke Include="e_sqlite3" />
</ItemGroup>
<Target Name="PrepareStaticSqliteSource" Condition="'$(SqliteStaticLink)' == 'true'">
<Error Condition="'$(RuntimeIdentifier)' != 'linux-x64' and '$(RuntimeIdentifier)' != 'linux-arm64' and
'$(RuntimeIdentifier)' != 'linux-musl-x64' and '$(RuntimeIdentifier)' != 'linux-musl-arm64' and
'$(RuntimeIdentifier)' != 'osx-x64' and '$(RuntimeIdentifier)' != 'osx-arm64' and
'$(RuntimeIdentifier)' != 'win-x64' and '$(RuntimeIdentifier)' != 'win-arm64'"
Text="Static SQLite is configured for the eight supported x64/arm64 Linux, musl, macOS and Windows targets. Unsupported RID: $(RuntimeIdentifier)." />
<PropertyGroup>
<_SqliteCache>$([System.IO.Path]::GetFullPath('$(BaseIntermediateOutputPath)sqlite/$(MonitorSqliteSourceVersion)/'))</_SqliteCache>
<_SqliteArchive>$(_SqliteCache)sqlite-amalgamation-$(MonitorSqliteSourceVersion).zip</_SqliteArchive>
<_SqliteSource>$(_SqliteCache)sqlite-amalgamation-$(MonitorSqliteSourceVersion)/sqlite3.c</_SqliteSource>
<_SqliteObjectDirectory>$(_SqliteCache)$(RuntimeIdentifier)/$(UseAotAnywhere)/</_SqliteObjectDirectory>
<_SqliteObject>$(_SqliteObjectDirectory)e_sqlite3.o</_SqliteObject>
<_SqliteObject Condition="$(RuntimeIdentifier.StartsWith('win'))">$(_SqliteObjectDirectory)e_sqlite3.obj</_SqliteObject>
</PropertyGroup>
<MakeDir Directories="$(_SqliteObjectDirectory)" />
<DownloadFile SourceUrl="https://www.sqlite.org/$(MonitorSqliteSourceYear)/sqlite-amalgamation-$(MonitorSqliteSourceVersion).zip"
DestinationFolder="$(_SqliteCache)" Condition="!Exists('$(_SqliteArchive)')" />
<GetFileHash Files="$(_SqliteArchive)" Algorithm="SHA256">
<Output TaskParameter="Hash" PropertyName="_SqliteArchiveHash" />
</GetFileHash>
<Error Condition="'$(_SqliteArchiveHash.ToLowerInvariant())' != '$(MonitorSqliteSourceSha256)'"
Text="SQLite source SHA256 mismatch: $(_SqliteArchiveHash)." />
<Unzip SourceFiles="$(_SqliteArchive)" DestinationFolder="$(_SqliteCache)" Condition="!Exists('$(_SqliteSource)')" />
</Target>
<Target Name="CompileStaticSqliteObject" Condition="'$(SqliteStaticLink)' == 'true'"
DependsOnTargets="PrepareStaticSqliteSource;$(_SqliteToolchainTargets)"
Inputs="$(_SqliteSource);$(MSBuildThisFileFullPath)" Outputs="$(_SqliteObject)">
<PropertyGroup>
<_SqliteUseMsvc>false</_SqliteUseMsvc>
<_SqliteUseMsvc Condition="$(RuntimeIdentifier.StartsWith('win')) and '$(_AotAnywhereWindowsCross)' != 'true'">true</_SqliteUseMsvc>
<_SqliteCompiler>&quot;$(CppCompilerAndLinker)&quot;</_SqliteCompiler>
<_SqliteCompiler Condition="'$(CppCompilerAndLinker)' == ''">cc</_SqliteCompiler>
<_SqliteCompiler Condition="'$(TargetTriple)' != ''">$(_SqliteCompiler) --target=$(TargetTriple)</_SqliteCompiler>
<_SqliteCompiler Condition="'$(SysRoot)' != ''">$(_SqliteCompiler) --sysroot=&quot;$(SysRoot)&quot;</_SqliteCompiler>
<_SqliteCompiler Condition="'$(UseAotAnywhere)' == 'true'">&quot;$(_AotAnywhereZigExe)&quot; cc -target $(TargetTriple)</_SqliteCompiler>
<_SqliteCompiler Condition="'$(_SqliteUseMsvc)' == 'true'">cl</_SqliteCompiler>
</PropertyGroup>
<Message Importance="high" Text="Compiling SQLite $(MonitorSqliteSourceVersion) into the $(RuntimeIdentifier) NativeAOT executable." />
<Exec Condition="'$(_SqliteUseMsvc)' != 'true'"
Command="$(_SqliteCompiler) -Os -fPIC -c &quot;$(_SqliteSource)&quot; -o &quot;$(_SqliteObject)&quot; -DSQLITE_API= -DSQLITE_THREADSAFE=1 -DSQLITE_DEFAULT_MEMSTATUS=0 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_SNAPSHOT -DSQLITE_ENABLE_UNLOCK_NOTIFY" />
<Exec Condition="'$(_SqliteUseMsvc)' == 'true'"
Command="cl /nologo /O1 /MT /c &quot;$(_SqliteSource)&quot; /Fo&quot;$(_SqliteObject)&quot; /DSQLITE_API= /DSQLITE_THREADSAFE=1 /DSQLITE_DEFAULT_MEMSTATUS=0 /DSQLITE_ENABLE_COLUMN_METADATA /DSQLITE_ENABLE_FTS4 /DSQLITE_ENABLE_FTS5 /DSQLITE_ENABLE_MATH_FUNCTIONS /DSQLITE_ENABLE_RTREE /DSQLITE_ENABLE_SNAPSHOT /DSQLITE_ENABLE_UNLOCK_NOTIFY" />
</Target>
<!-- AotAnywhere runs before LinkNative; register the object before either linker starts. -->
<Target Name="LinkStaticSqlite" Condition="'$(SqliteStaticLink)' == 'true'"
BeforeTargets="AotAnywhereDirectLinkNative;AotAnywhereDirectLinkMacNative;AotAnywhereDirectLinkWindows;LinkNative"
DependsOnTargets="CompileStaticSqliteObject">
<ItemGroup>
<NativeLibrary Include="$(_SqliteObject)" />
<LinkerArg Include="&quot;$(_SqliteObject)&quot;" />
</ItemGroup>
</Target>
<Target Name="RemoveStaticSqliteRuntimeFiles" Condition="'$(SqliteStaticLink)' == 'true'" AfterTargets="ComputeFilesToPublish">
<ItemGroup>
<ResolvedFileToPublish Remove="@(ResolvedFileToPublish)"
Condition="'%(ResolvedFileToPublish.Filename)' == 'libe_sqlite3' or '%(ResolvedFileToPublish.Filename)' == 'e_sqlite3'" />
</ItemGroup>
</Target>
</Project>
@@ -256,7 +256,7 @@ public sealed class AgentEndToEndTests
string? startFailure = null; string? startFailure = null;
for (int attempt = 0; attempt < 5; attempt++) for (int attempt = 0; attempt < 5; attempt++)
{ {
accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000 } })); accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000, ["duration_s"] = 5 } }));
if (accepted.StatusCode == System.Net.HttpStatusCode.Accepted) if (accepted.StatusCode == System.Net.HttpStatusCode.Accepted)
{ {
break; break;
@@ -291,7 +291,7 @@ public sealed class AgentEndToEndTests
Assert.Equal(4, probe.Phases.Count); Assert.Equal(4, probe.Phases.Count);
Assert.All(probe.Phases, phase => Assert.Equal("udp_train", phase.Kind)); Assert.All(probe.Phases, phase => Assert.Equal("udp_train", phase.Kind));
// 单块最多 50000 条记录:50 和 100 Mbit/s 阶段(10 s、1000 B 的包)各被 Probe 切成多个列块 // 单块最多 50000 条记录:50 和 100 Mbit/s 阶段(5 s、1000 B 的包)里 100 Mbit/s 的那一段有 6 万多条记录,被 Probe 切成两个列块
Assert.True(agent.App.Counters.Fragments > 4); Assert.True(agent.App.Counters.Fragments > 4);
await BackendFixture.EventuallyAsync( await BackendFixture.EventuallyAsync(
async () => (await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == "udp_train") == agent.App.Counters.Fragments, async () => (await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == "udp_train") == agent.App.Counters.Fragments,
@@ -56,7 +56,7 @@ public sealed class AgentHarness : IAsyncDisposable
}; };
configure?.Invoke(config); configure?.Invoke(config);
LoadedNodeIdentity identity = LoadedNodeIdentity.Load(directory); LoadedNodeIdentity identity = LoadedNodeIdentity.Load(directory);
var app = new AgentApp(config, identity, TimeProvider.System, new ConsoleNodeLog(TimeProvider.System, Console.Out)); var app = new AgentApp(config, identity, directory, TimeProvider.System, new ConsoleNodeLog(TimeProvider.System, Console.Out));
var harness = new AgentHarness(backend, directory, identity, app); var harness = new AgentHarness(backend, directory, identity, app);
await app.Ready.WaitAsync(TimeSpan.FromSeconds(10)); await app.Ready.WaitAsync(TimeSpan.FromSeconds(10));
await BackendFixture.EventuallyAsync(() => Task.FromResult(app.Link.IsConnected), because: "the agent should connect to the backend"); await BackendFixture.EventuallyAsync(() => Task.FromResult(app.Link.IsConnected), because: "the agent should connect to the backend");
@@ -35,11 +35,20 @@ public sealed class AotSmokeTests
string token = JsonNode.Parse(await tokenResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>(); string token = JsonNode.Parse(await tokenResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token); http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
using HttpResponseMessage settingsResponse = await http.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["site.public"] = true }, "application/merge-patch+json"));
settingsResponse.EnsureSuccessStatusCode();
using HttpResponseMessage probeResponse = await http.PostAsync("/api/admin/probes", BackendFixture.Json(new JsonObject { ["name"] = "aot-probe-" + Guid.NewGuid().ToString("N")[..6] }));
probeResponse.EnsureSuccessStatusCode();
string probeId = JsonNode.Parse(await probeResponse.Content.ReadAsStringAsync())!["probe_id"]!.GetValue<string>();
// Agent:加入令牌 → 注册(ECDSA 持有证明)→ WebSocket 签名认证 → 期望状态 // Agent:加入令牌 → 注册(ECDSA 持有证明)→ WebSocket 签名认证 → 期望状态
using HttpResponseMessage joinResponse = await http.PostAsync("/api/admin/join-tokens", BackendFixture.Json(new JsonObject { ["max_uses"] = 1 })); using HttpResponseMessage joinResponse = await http.PostAsync("/api/admin/join-tokens", BackendFixture.Json(new JsonObject { ["max_uses"] = 1 }));
joinResponse.EnsureSuccessStatusCode();
string joinToken = JsonNode.Parse(await joinResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>(); string joinToken = JsonNode.Parse(await joinResponse.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
using var anonymous = new HttpClient { BaseAddress = baseAddress }; using var anonymous = new HttpClient { BaseAddress = baseAddress };
await using AgentSimulator agent = await AgentSimulator.EnrollAsync(anonymous, joinToken, "aot-" + Guid.NewGuid().ToString("N")[..6]); await using AgentSimulator agent = await AgentSimulator.EnrollAsync(anonymous, joinToken, "aot-" + Guid.NewGuid().ToString("N")[..6]);
agent.ProbeForChunks = new ProbeId(probeId);
string lineId = $"{agent.NodeId.Value}~{probeId}";
await agent.ConnectAsync(baseAddress); await agent.ConnectAsync(baseAddress);
TaskSet tasks = await agent.WaitForTaskSetAsync(s => s.Tasks.Count >= 2); TaskSet tasks = await agent.WaitForTaskSetAsync(s => s.Tasks.Count >= 2);
Assert.Contains(tasks.Tasks, t => t.Kind == "udp_echo"); Assert.Contains(tasks.Tasks, t => t.Kind == "udp_echo");
@@ -54,7 +63,7 @@ public sealed class AotSmokeTests
// 查询:热层的精确汇总,随后是持久层的分钟窗口 // 查询:热层的精确汇总,随后是持久层的分钟窗口
string range = $"from={Iso(baseMs)}&to={Iso(baseMs + minute)}&metrics=loss.total,rtt.p95"; string range = $"from={Iso(baseMs)}&to={Iso(baseMs + minute)}&metrics=loss.total,rtt.p95";
JsonNode hot = await GetAsync(http, $"/api/public/lines/{agent.NodeId.Value}/series?{range}&step=1s"); JsonNode hot = await GetAsync(http, $"/api/public/lines/{lineId}/series?{range}&step=1s");
Assert.Equal("hot", hot["source"]!.GetValue<string>()); Assert.Equal("hot", hot["source"]!.GetValue<string>());
Assert.Equal(600, hot["summary"]!["samples"]!["udp_echo"]!.GetValue<long>()); Assert.Equal(600, hot["summary"]!["samples"]!["udp_echo"]!.GetValue<long>());
Assert.Equal(10.0, hot["summary"]!["metrics"]!["loss.total"]!.GetValue<double>(), 6); Assert.Equal(10.0, hot["summary"]!["metrics"]!["loss.total"]!.GetValue<double>(), 6);
@@ -62,7 +71,7 @@ public sealed class AotSmokeTests
JsonNode persisted; JsonNode persisted;
do do
{ {
persisted = await GetAsync(http, $"/api/public/lines/{agent.NodeId.Value}/series?{range}&step=1m"); persisted = await GetAsync(http, $"/api/public/lines/{lineId}/series?{range}&step=1m");
if (persisted["summary"]!["samples"]!["udp_echo"]?.GetValue<long>() == 600) if (persisted["summary"]!["samples"]!["udp_echo"]?.GetValue<long>() == 600)
{ {
break; break;
@@ -80,16 +89,16 @@ public sealed class AotSmokeTests
Assert.True(RawChunkCodec.Decode(bytes).IsSuccess); Assert.True(RawChunkCodec.Decode(bytes).IsSuccess);
// SSE:快照 + 增量 // SSE:快照 + 增量
await using SseClient sse = await SseClient.ConnectAsync(http, $"/api/public/live?topics=nodes,series:{agent.NodeId.Value}:1s:udp"); await using SseClient sse = await SseClient.ConnectAsync(http, $"/api/public/live?topics=nodes,series:{lineId}:1s:udp");
await sse.WaitForAsync(e => e.Name == "series"); await sse.WaitForAsync(e => e.Name == "series");
// 按需测试:Probe 事件 → 下发 → 状态 // 按需测试:Probe 事件 → 下发 → 状态
await agent.SendEventAsync(AgentEventKinds.ProbeConnected); await agent.SendEventAsync(AgentEventKinds.ProbeConnected, probeId);
JsonNode started = null!; JsonNode started = null!;
long begun = TimeProvider.System.GetTimestamp(); long begun = TimeProvider.System.GetTimestamp();
while (TimeProvider.System.GetElapsedTime(begun) < TimeSpan.FromSeconds(10)) while (TimeProvider.System.GetElapsedTime(begun) < TimeSpan.FromSeconds(10))
{ {
using HttpResponseMessage response = await http.PostAsync($"/api/admin/lines/{agent.NodeId.Value}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "reality_session" })); using HttpResponseMessage response = await http.PostAsync($"/api/admin/lines/{lineId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "reality_session" }));
if (response.StatusCode == System.Net.HttpStatusCode.Accepted) if (response.StatusCode == System.Net.HttpStatusCode.Accepted)
{ {
started = JsonNode.Parse(await response.Content.ReadAsStringAsync())!; started = JsonNode.Parse(await response.Content.ReadAsStringAsync())!;
@@ -58,6 +58,39 @@ public sealed class ApiBasicsTests
Assert.Equal("no-store", ok.Headers.CacheControl!.ToString()); Assert.Equal("no-store", ok.Headers.CacheControl!.ToString());
} }
[Fact]
public async Task Login_WithReturnSession_GivesABearerSession_WithoutACookie_AndNoCsrfNeeded()
{
using HttpClient client = _backend.NewClient();
using HttpResponseMessage login = await client.PostAsync("/api/admin/auth/login", BackendFixture.Json(new JsonObject { ["username"] = BackendFixture.AdminUser, ["password"] = BackendFixture.AdminPassword, ["return_session"] = true }));
Assert.Equal(HttpStatusCode.OK, login.StatusCode);
Assert.False(login.Headers.Contains("Set-Cookie"));
string session = JsonNode.Parse(await login.Content.ReadAsStringAsync())!["session_token"]!.GetValue<string>();
Assert.False(string.IsNullOrEmpty(session));
// 之后当 API 令牌那样用 Bearer;不是 Cookie,所以不需要 CSRF,也能做非 GET 操作
using var me = new HttpRequestMessage(HttpMethod.Get, "/api/admin/auth/me");
me.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage meResponse = await client.SendAsync(me);
Assert.Equal(HttpStatusCode.OK, meResponse.StatusCode);
Assert.Equal(BackendFixture.AdminUser, JsonNode.Parse(await meResponse.Content.ReadAsStringAsync())!["username"]!.GetValue<string>());
using var create = new HttpRequestMessage(HttpMethod.Post, "/api/admin/join-tokens") { Content = BackendFixture.Json(new JsonObject()) };
create.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage created = await client.SendAsync(create);
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
// 退出之后这个会话令牌就失效
using var logout = new HttpRequestMessage(HttpMethod.Post, "/api/admin/auth/logout");
logout.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage loggedOut = await client.SendAsync(logout);
Assert.Equal(HttpStatusCode.NoContent, loggedOut.StatusCode);
using var after = new HttpRequestMessage(HttpMethod.Get, "/api/admin/auth/me");
after.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", session);
using HttpResponseMessage afterResponse = await client.SendAsync(after);
Assert.Equal(HttpStatusCode.Unauthorized, afterResponse.StatusCode);
}
[Fact] [Fact]
public async Task SessionCookie_RequiresCsrfHeaderOnUnsafeMethods() public async Task SessionCookie_RequiresCsrfHeaderOnUnsafeMethods()
{ {
@@ -192,6 +192,8 @@ public class BackendFixture : IAsyncLifetime
options.InitialAdminUser = AdminUser; options.InitialAdminUser = AdminUser;
options.InitialAdminPassword = AdminPassword; options.InitialAdminPassword = AdminPassword;
options.LoginAttemptsPerMinute = LoginAttemptsPerMinute; options.LoginAttemptsPerMinute = LoginAttemptsPerMinute;
options.GeoLookupUrl = string.Empty; // 测试里不访问外部服务
options.PhaseDurationScale = 0.2; // 端到端测试里 connect、hold 这些固定 30 秒的阶段缩成 6 秒
}); });
await App.StartAsync(); await App.StartAsync();
string address = App.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First(); string address = App.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First();
@@ -83,6 +83,14 @@ public sealed class EnrollmentAndTaskSetTests
using HttpResponseMessage revoke = await _backend.Admin.DeleteAsync("/api/admin/join-tokens/" + id); using HttpResponseMessage revoke = await _backend.Admin.DeleteAsync("/api/admin/join-tokens/" + id);
Assert.Equal(HttpStatusCode.NoContent, revoke.StatusCode); Assert.Equal(HttpStatusCode.NoContent, revoke.StatusCode);
await Assert.ThrowsAsync<InvalidOperationException>(() => AgentSimulator.EnrollAsync(_backend.Anonymous, revoked, _backend.UniqueName("revoked"))); await Assert.ThrowsAsync<InvalidOperationException>(() => AgentSimulator.EnrollAsync(_backend.Anonymous, revoked, _backend.UniqueName("revoked")));
// 吊销之后记录还在(显示「已吊销」);purge=true 才是彻底删除
tokens = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/join-tokens");
Assert.Contains(tokens["items"]!.AsArray(), t => t!["id"]!.GetValue<string>() == id);
using HttpResponseMessage purge = await _backend.Admin.DeleteAsync($"/api/admin/join-tokens/{id}?purge=true");
Assert.Equal(HttpStatusCode.NoContent, purge.StatusCode);
tokens = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/join-tokens");
Assert.DoesNotContain(tokens["items"]!.AsArray(), t => t!["id"]!.GetValue<string>() == id);
} }
[Fact] [Fact]
@@ -330,7 +338,7 @@ public sealed class EnrollmentAndTaskSetTests
JsonNode identity = JsonNode.Parse(await created.Content.ReadAsStringAsync())!; JsonNode identity = JsonNode.Parse(await created.Content.ReadAsStringAsync())!;
string id = identity["probe_id"]!.GetValue<string>(); string id = identity["probe_id"]!.GetValue<string>();
using var key1 = Fleet.Identity.NodeIdentity.FromPkcs8(Convert.FromBase64String(identity["private_key"]!.GetValue<string>())); using var key1 = Fleet.Identity.NodeIdentity.FromPkcs8(Convert.FromBase64String(identity["private_key"]!.GetValue<string>()));
Assert.Contains(identity["bootstrap"]!.AsArray(), b => b!["agent_id"]!.GetValue<string>() == agent.NodeId.Value); Assert.True(Uri.TryCreate(identity["server"]!.GetValue<string>(), UriKind.Absolute, out _)); // Probe 直接连面板:probe.json 里是面板的地址
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == id && p.PublicKey == Convert.ToBase64String(key1.PublicKeySpki))); await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == id && p.PublicKey == Convert.ToBase64String(key1.PublicKeySpki)));
// The panel lists it; two probes can coexist and a duplicate name is refused // The panel lists it; two probes can coexist and a duplicate name is refused
@@ -366,34 +374,6 @@ public sealed class EnrollmentAndTaskSetTests
Assert.Equal(HttpStatusCode.NotFound, gone.StatusCode); Assert.Equal(HttpStatusCode.NotFound, gone.StatusCode);
} }
[Fact]
public async Task ProbeAgentList_IsServedToAgents_AndPushedWhenTheSetOfAgentsChanges()
{
await using AgentSimulator first = await _backend.ConnectAgentAsync();
ProbeAgentsResult list = await first.RequestProbeAgentsAsync();
Assert.Contains(list.Agents, a => a.AgentId == first.NodeId && a.Host == "203.0.113.10" && a.TcpPort == 7443);
Assert.All(list.Agents, a => Assert.Equal(32, Convert.FromBase64String(a.Spki).Length));
await using AgentSimulator second = await _backend.ConnectAgentAsync();
ProbeAgentsResult pushed = await WaitForAsync(first, l => l.Agents.Any(a => a.AgentId == second.NodeId));
Assert.True(pushed.Version > list.Version);
using HttpResponseMessage disable = await _backend.Admin.PostAsync($"/api/admin/agents/{second.NodeId.Value}/disable", null);
Assert.Equal(HttpStatusCode.OK, disable.StatusCode);
await WaitForAsync(first, l => l.Agents.All(a => a.AgentId != second.NodeId));
}
private static async Task<ProbeAgentsResult> WaitForAsync(AgentSimulator agent, Func<ProbeAgentsResult, bool> predicate)
{
ProbeAgentsResult? found = null;
await BackendFixture.EventuallyAsync(() =>
{
found = agent.ProbeAgentLists.ToArray().LastOrDefault(predicate);
return Task.FromResult(found is not null);
});
return found!;
}
private Task<HttpResponseMessage> PostEnrollAsync(JsonNode body) => private Task<HttpResponseMessage> PostEnrollAsync(JsonNode body) =>
_backend.Anonymous.PostAsync("/api/agent/v1/enroll", BackendFixture.Json(body)); _backend.Anonymous.PostAsync("/api/agent/v1/enroll", BackendFixture.Json(body));
} }
@@ -21,6 +21,7 @@ public sealed class HostSamplerTests
Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1)); Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1));
Assert.Equal(3600L, HostSampler.ParseUptime("3600.55 7000.1\n")); Assert.Equal(3600L, HostSampler.ParseUptime("3600.55 7000.1\n"));
Assert.Null(HostSampler.ParseNetwork("nothing useful")); Assert.Null(HostSampler.ParseNetwork("nothing useful"));
Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1 + "podman0: 8888 1 0 0 0 0 0 0 9999 1 0 0 0 0 0 0\n"));
} }
[Fact] [Fact]
@@ -29,7 +30,7 @@ public sealed class HostSamplerTests
var time = new Microsoft.Extensions.Time.Testing.FakeTimeProvider(); var time = new Microsoft.Extensions.Time.Testing.FakeTimeProvider();
string stat = Stat1; string stat = Stat1;
string dev = Dev1; string dev = Dev1;
var sampler = new HostSampler(time, path => path switch { "/proc/stat" => stat, "/proc/net/dev" => dev, "/proc/uptime" => "100.0 1.0", _ => null }, () => (1000, 250)); var sampler = new HostSampler(time, path => path switch { "/proc/stat" => stat, "/proc/net/dev" => dev, "/proc/uptime" => "100.0 1.0", _ => null }, () => (1000, 250), procRoot: "/proc", sysRoot: "/sys");
HostSnapshot first = sampler.Sample(); HostSnapshot first = sampler.Sample();
Assert.Null(first.CpuPercent); // 第一次没有基线 Assert.Null(first.CpuPercent); // 第一次没有基线
Assert.Null(first.NetRxBps); Assert.Null(first.NetRxBps);
@@ -45,6 +46,54 @@ public sealed class HostSamplerTests
Assert.Equal(2000, second.NetTxBps); Assert.Equal(2000, second.NetTxBps);
Assert.Equal(3000, second.NetRxTotal); Assert.Equal(3000, second.NetRxTotal);
} }
[Fact]
public void Sample_UsesMountedHostProcAndSys_InsteadOfContainerPaths()
{
var reads = new List<string>();
var sampler = new HostSampler(TimeProvider.System, path =>
{
reads.Add(path);
return path switch
{
"/host/proc/stat" => Stat1,
"/host/proc/meminfo" => "MemTotal: 8192 kB\nMemAvailable: 6144 kB\n",
"/host/proc/net/dev" => Dev1,
"/host/proc/uptime" => "3600.0 7000.0",
"/host/sys/class/net/eth0/speed" => "10000\n",
_ => throw new InvalidOperationException("unexpected container path: " + path),
};
}, () => (1000, 250), procRoot: "/host/proc", sysRoot: "/host/sys");
HostSnapshot snapshot = sampler.Sample();
Assert.Equal(8L * 1024 * 1024, snapshot.MemTotal);
Assert.Equal(1000, snapshot.NetRxTotal);
Assert.Equal(10000, snapshot.NetLinkMbps);
Assert.Equal(3600, snapshot.UptimeSeconds);
Assert.DoesNotContain(reads, path => path.StartsWith("/proc/", StringComparison.Ordinal));
Assert.DoesNotContain(reads, path => path.Contains("docker0", StringComparison.Ordinal) || path.Contains("/lo/", StringComparison.Ordinal));
}
[Theory]
[InlineData(null)]
[InlineData("-1")]
[InlineData("0")]
[InlineData("not supported")]
public void Sample_DoesNotInventLinkSpeedOrContainerDisk_WhenHostDataIsUnavailable(string? speed)
{
var sampler = new HostSampler(TimeProvider.System,
path => path.EndsWith("net/dev", StringComparison.Ordinal) ? Dev1 : path.EndsWith("/speed", StringComparison.Ordinal) ? speed : null,
procRoot: "/host/proc", sysRoot: "/host/sys", diskRoot: Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")));
HostSnapshot snapshot = sampler.Sample();
Assert.Null(snapshot.NetLinkMbps);
Assert.Null(snapshot.DiskTotal);
Assert.Null(snapshot.DiskUsed);
if (OperatingSystem.IsLinux())
{
Assert.Null(snapshot.UptimeSeconds);
}
}
} }
[Collection("backend")] [Collection("backend")]
@@ -61,6 +110,7 @@ public sealed class HostStatusTests
{ {
CpuPercent = cpu, MemUsed = 2L << 30, MemTotal = 8L << 30, DiskUsed = 10L << 30, DiskTotal = 40L << 30, CpuPercent = cpu, MemUsed = 2L << 30, MemTotal = 8L << 30, DiskUsed = 10L << 30, DiskTotal = 40L << 30,
NetRxTotal = 600L << 30, NetTxTotal = 180L << 30, NetRxBps = 2500, NetTxBps = 4800, UptimeSeconds = 3_000_000, NetRxTotal = 600L << 30, NetTxTotal = 180L << 30, NetRxBps = 2500, NetTxBps = 4800, UptimeSeconds = 3_000_000,
NetLinkMbps = 1000,
}; };
[Fact] [Fact]
@@ -81,6 +131,7 @@ public sealed class HostStatusTests
JsonNode item = list["items"]!.AsArray().Single(i => i!["agent_id"]!.GetValue<string>() == id)!; JsonNode item = list["items"]!.AsArray().Single(i => i!["agent_id"]!.GetValue<string>() == id)!;
Assert.False(item["stale"]!.GetValue<bool>()); Assert.False(item["stale"]!.GetValue<bool>());
Assert.Equal(40L << 30, item["host"]!["disk_total"]!.GetValue<long>()); Assert.Equal(40L << 30, item["host"]!["disk_total"]!.GetValue<long>());
Assert.Equal(1000, item["host"]!["net_link_mbps"]!.GetValue<long>());
// 展示页能拿到的只有汇总数字:整份响应里没有型号、主机名、地址之类的字段 // 展示页能拿到的只有汇总数字:整份响应里没有型号、主机名、地址之类的字段
string text = list.ToJsonString(); string text = list.ToJsonString();
@@ -0,0 +1,81 @@
using System.Net;
using System.Text.Json.Nodes;
using Monitor.TestSupport;
namespace Monitor.Backend.Tests;
[Collection("probe-e2e")]
public sealed class LinkTests
{
private readonly BackendFixture _backend;
public LinkTests(BackendFixture backend)
{
_backend = backend;
}
private async Task<HttpResponseMessage> SetLinkAsync(string probe, string agent, bool? enabled) =>
await _backend.Admin.PutAsync($"/api/admin/links/{probe}/{agent}", BackendFixture.Json(new JsonObject { ["enabled"] = enabled }));
private async Task SetAutoLinkAsync(bool on)
{
using HttpResponseMessage response = await _backend.Admin.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["fleet.auto_link"] = on }, "application/merge-patch+json"));
response.EnsureSuccessStatusCode();
}
private async Task<JsonNode?> LinkAsync(string probe, string agent) =>
(await _backend.GetJsonAsync(_backend.Admin, "/api/admin/links"))["items"]!.AsArray().FirstOrDefault(i => i!["probe_id"]!.GetValue<string>() == probe && i["agent_id"]!.GetValue<string>() == agent);
[Fact]
public async Task Links_FollowTheAutomaticSwitch_AndManualChoicesAlwaysWin()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
string probe = (await _backend.CreateProbeAsync())["probe_id"]!.GetValue<string>();
string agentId = agent.NodeId.Value;
try
{
// 自动连接(默认开):新 Probe 自动进入这个 Agent 允许的名单
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == probe));
JsonNode link = (await LinkAsync(probe, agentId))!;
Assert.True(link["linked"]!.GetValue<bool>());
Assert.Null(link["override"]);
// 手动关掉这一对:自动连接开着也不连
using HttpResponseMessage off = await SetLinkAsync(probe, agentId, false);
Assert.Equal(HttpStatusCode.OK, off.StatusCode);
await agent.WaitForTaskSetAsync(s => s.Peers.All(p => p.ProbeId.Value != probe));
Assert.False((await LinkAsync(probe, agentId))!["linked"]!.GetValue<bool>());
// 取消手动设置:回到自动连接
using HttpResponseMessage reset = await SetLinkAsync(probe, agentId, null);
Assert.Equal(HttpStatusCode.OK, reset.StatusCode);
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == probe));
// 关掉自动连接:没有手动选中的就都不连
await SetAutoLinkAsync(false);
await agent.WaitForTaskSetAsync(s => s.Peers.All(p => p.ProbeId.Value != probe));
Assert.False((await _backend.GetJsonAsync(_backend.Admin, "/api/admin/links"))["auto_link"]!.GetValue<bool>());
// 自动连接关着,手动选中的照样连
await SetLinkAsync(probe, agentId, true);
await agent.WaitForTaskSetAsync(s => s.Peers.Any(p => p.ProbeId.Value == probe));
Assert.True((await LinkAsync(probe, agentId))!["override"]!.GetValue<bool>());
}
finally
{
await SetLinkAsync(probe, agentId, null);
await SetAutoLinkAsync(true); // 共用的后端:恢复默认,别影响别的测试
}
}
[Fact]
public async Task Links_RefuseUnknownNodes()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
string probe = (await _backend.CreateProbeAsync())["probe_id"]!.GetValue<string>();
using HttpResponseMessage noProbe = await SetLinkAsync("prb_missing", agent.NodeId.Value, false);
Assert.Equal(HttpStatusCode.NotFound, noProbe.StatusCode);
using HttpResponseMessage noAgent = await SetLinkAsync(probe, "agt_missing", false);
Assert.Equal(HttpStatusCode.NotFound, noAgent.StatusCode);
}
}
@@ -0,0 +1,310 @@
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Time.Testing;
using Monitor.Backend.Application;
using Monitor.Protocol;
namespace Monitor.Backend.Tests;
public sealed partial class NodeInfoTests
{
[Theory]
[InlineData(9808, "China Mobile Communications Group", "mobile", "中国移动")]
[InlineData(4134, "Chinanet", "telecom", "中国电信")]
[InlineData(4837, "China Unicom Backbone", "unicom", "中国联通")]
[InlineData(null, "China Mobile Communications Corporation", "mobile", "中国移动")]
[InlineData(null, "CHINANET-BACKBONE", "telecom", "中国电信")]
[InlineData(null, "China Unicom Beijing Province Network", "unicom", "中国联通")]
[InlineData(null, "China Broadnet Corporation", "broadnet", "中国广电")]
public void IspClassifier_RecognizesTheChineseCarriers(int? asn, string org, string key, string name)
{
Assert.Equal((key, name), IspClassifier.Classify(asn, org));
}
[Fact]
public void IspClassifier_OtherCarriersKeepAShortenedOrganizationName()
{
(string key, string name) = IspClassifier.Classify(37963, "Hangzhou Alibaba Advertising Co.,Ltd.");
Assert.Equal("other", key);
Assert.Equal("Hangzhou Alibaba Advertising", name);
Assert.Equal("Google", IspClassifier.Classify(15169, "Google LLC").Name);
Assert.True(IspClassifier.Classify(1, new string('x', 80)).Name.Length <= 28);
}
[Theory]
[InlineData("8.8.8.8", true)]
[InlineData("223.5.5.5", true)]
[InlineData("2001:4860:4860::8888", true)]
[InlineData("example.com", true)]
[InlineData("127.0.0.1", false)]
[InlineData("10.1.2.3", false)]
[InlineData("172.16.0.1", false)]
[InlineData("172.32.0.1", true)]
[InlineData("192.168.1.10", false)]
[InlineData("169.254.1.1", false)]
[InlineData("100.64.0.1", false)]
[InlineData("0.0.0.0", false)]
[InlineData("::1", false)]
[InlineData("fe80::1", false)]
[InlineData("fd00::1", false)]
[InlineData("::ffff:192.168.1.1", false)]
[InlineData("localhost", false)]
[InlineData("printer.local", false)]
public void OnlyPublicAddressesAreEverSentToTheLookupService(string host, bool expected)
{
Assert.Equal(expected, NodeInfoService.IsPublic(host));
}
[Fact]
public async Task Service_LooksAPublicAddressUpOnce_KeepsOnlyCountryAndIsp_AndNeverQueriesPrivateOnes()
{
var geo = new FakeGeo(new GeoInfo("cn", 9808, "China Mobile Communications Group", false));
var store = new MemoryStore();
var time = new FakeTimeProvider(DateTimeOffset.Parse("2026-10-06T00:00:00Z"));
var service = new NodeInfoService(store, geo, time);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "10.0.0.5", CancellationToken.None);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "127.0.0.1", CancellationToken.None);
Assert.Empty(geo.Calls);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.73.168.45", CancellationToken.None);
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.73.168.45", CancellationToken.None);
Assert.Single(geo.Calls);
NodeInfo info = Assert.Single(await service.ListAsync(CancellationToken.None));
Assert.Equal(("CN", "mobile", "中国移动"), (info.Country, info.IspKey, info.IspName));
// 展示用的信息里没有地址:整个对象序列化出来,找不到任何 IP 形状的字符串
Assert.DoesNotMatch(Ipv4(), info.ToString());
// 地址变了才重新查;过了一周也重新查
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.5.5.5", CancellationToken.None);
Assert.Equal(2, geo.Calls.Count);
time.Advance(TimeSpan.FromDays(8));
await service.ObserveAddressAsync("prb_1", NodeKinds.Probe, "223.5.5.5", CancellationToken.None);
Assert.Equal(3, geo.Calls.Count);
}
[Fact]
public async Task Service_FailedLookups_AreNotRetriedForAWhile_AndNeverThrow()
{
var geo = new FakeGeo(null);
var time = new FakeTimeProvider(DateTimeOffset.Parse("2026-10-06T00:00:00Z"));
var service = new NodeInfoService(new MemoryStore(), geo, time);
await service.ObserveAddressAsync("agt_1", NodeKinds.Agent, "8.8.8.8", CancellationToken.None);
await service.ObserveAddressAsync("agt_1", NodeKinds.Agent, "8.8.8.8", CancellationToken.None);
Assert.Single(geo.Calls);
time.Advance(TimeSpan.FromMinutes(11));
await service.ObserveAddressAsync("agt_1", NodeKinds.Agent, "8.8.8.8", CancellationToken.None);
Assert.Equal(2, geo.Calls.Count);
Assert.Null(Assert.Single(await service.ListAsync(CancellationToken.None)).Country);
}
[Fact]
public async Task Service_PlatformIsSavedAndSurvivesARestart()
{
var store = new MemoryStore();
var first = new NodeInfoService(store, new FakeGeo(null), TimeProvider.System);
await first.SetPlatformAsync("agt_1", NodeKinds.Agent, new PlatformInfo { Os = "linux", Name = "Debian 12", Arch = "arm64" }, CancellationToken.None);
var second = new NodeInfoService(store, new FakeGeo(null), TimeProvider.System);
NodeInfo info = Assert.Single(await second.ListAsync(CancellationToken.None));
Assert.Equal(("linux", "Debian 12", "arm64"), (info.Os, info.OsName, info.Arch));
}
[Fact]
public void GeoParser_ReadsOnlyCountryAsnAndOrganization()
{
using JsonDocument doc = JsonDocument.Parse("""
{"ip":"223.5.5.5","found":true,"is_private":false,
"country_region":{"iso_code":"CN","subdivisions":[{"names":{"zh-CN":"浙江省"}}]},
"city":{"names":{"zh-CN":"杭州市"}},
"location":{"latitude":30.25,"longitude":120.21},
"network":{"cidr":"223.4.0.0/14","asn":37963,"organization":"Hangzhou Alibaba Advertising Co.,Ltd.","isp":""}}
""");
GeoInfo? geo = HttpGeoLookup.Parse(doc.RootElement);
Assert.Equal(new GeoInfo("CN", 37963, "Hangzhou Alibaba Advertising Co.,Ltd.", false), geo);
Assert.Null(HttpGeoLookup.Parse(JsonDocument.Parse("""{"found":false}""").RootElement));
}
[GeneratedRegex(@"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}")]
private static partial Regex Ipv4();
private sealed class FakeGeo : IGeoLookup
{
private readonly GeoInfo? _result;
public FakeGeo(GeoInfo? result)
{
_result = result;
}
public List<string> Calls { get; } = [];
public ValueTask<GeoInfo?> LookupAsync(string host, CancellationToken ct)
{
Calls.Add(host);
return ValueTask.FromResult(_result);
}
}
private sealed class MemoryStore : INodeMetaStore
{
private readonly Dictionary<string, NodeMeta> _items = [];
public ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct) => ValueTask.FromResult<IReadOnlyList<NodeMeta>>([.. _items.Values]);
public ValueTask SaveAsync(NodeMeta meta, CancellationToken ct)
{
_items[meta.Info.NodeId] = meta;
return ValueTask.CompletedTask;
}
public ValueTask DeleteAsync(string nodeId, CancellationToken ct)
{
_items.Remove(nodeId);
return ValueTask.CompletedTask;
}
}
}
/// <summary>
/// 访客能拿到的东西里不能有 IP 等敏感信息:公开接口的整个响应里不出现 IP 形状的字符串,也没有 ip、city、asn、经纬度这些字段。
/// Agent 和 Probe 这里是真的连上了(Probe 的来源地址是 127.0.0.1),所以如果哪里把地址带出来了,这里一定能发现。
/// </summary>
[Collection("probe-e2e")]
public sealed partial class NodePrivacyTests
{
private static readonly string[] s_forbiddenKeys = ["ip", "city", "asn", "latitude", "longitude", "cidr", "remote", "public_host", "geo_host", "host_name", "hostname"];
private readonly BackendFixture _backend;
public NodePrivacyTests(BackendFixture backend)
{
_backend = backend;
}
[Fact]
public async Task PublicEndpoints_NeverExposeAnAddress_AndNodesCarryOnlyPlatformCountryAndIsp()
{
await _backend.SiteMustBePublicAsync();
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
await agent.OverrideLineTasksAsync();
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
await BackendFixture.EventuallyAsync(async () => (await _backend.GetJsonAsync(_backend.Admin, "/api/admin/agents/" + agent.Id))["probe_connected"]!.GetValue<bool>());
// 两端都自报了系统和架构(Agent 在 hello 里,Probe 随列表同步)
JsonNode? agentNode = null;
JsonNode? probeNode = null;
await BackendFixture.EventuallyAsync(
async () =>
{
JsonNode nodes = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes");
agentNode = nodes["items"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == agent.Id);
probeNode = nodes["items"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == probe.ProbeId);
return agentNode?["os"] is not null && probeNode?["os"] is not null && agentNode["arch"] is not null && probeNode["arch"] is not null;
},
TimeSpan.FromSeconds(40),
"the agent (hello) and the probe (agent list sync) both report their platform");
Assert.Equal("agent", agentNode!["kind"]!.GetValue<string>());
Assert.Equal("probe", probeNode!["kind"]!.GetValue<string>());
string[] paths =
[
"/api/public/nodes",
"/api/public/hosts",
"/api/public/overview?range=1h&metrics=rtt.p50",
$"/api/public/lines/{Uri.EscapeDataString(agent.Id + "~" + probe.ProbeId)}",
"/api/public/catalog/plans",
];
foreach (string path in paths)
{
using HttpResponseMessage response = await _backend.Anonymous.GetAsync(path);
Assert.True(response.IsSuccessStatusCode, $"{path}: {(int)response.StatusCode}");
string body = await response.Content.ReadAsStringAsync();
Assert.DoesNotMatch(Ipv4(), body);
Assert.DoesNotContain("::1", body, StringComparison.Ordinal);
foreach (string key in KeysOf(JsonNode.Parse(body)))
{
Assert.DoesNotContain(key, s_forbiddenKeys);
}
}
}
[Fact]
public async Task BuildInfo_IsHiddenFromVisitorsUntilTheSwitchIsOn_AndAlwaysVisibleToAdmins()
{
await _backend.SiteMustBePublicAsync();
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
// 默认关:访客看不到版本和 tag,「关于」里只有 show_build: false
JsonNode about = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/about");
Assert.False(about["show_build"]!.GetValue<bool>());
Assert.Null(about["backend"]);
JsonNode hidden = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes");
Assert.All(hidden["items"]!.AsArray(), n =>
{
Assert.Null(n!["version"]);
Assert.Null(n["tag"]);
});
// 管理端一直看得到:后端自己的版本和每个节点上报的版本
JsonNode admin = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info");
Assert.False(string.IsNullOrEmpty(admin["backend"]!["version"]!.GetValue<string>()));
await BackendFixture.EventuallyAsync(
async () => (await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info"))["nodes"]!.AsArray().Any(n => n!["id"]!.GetValue<string>() == agent.Id && n["version"] is not null),
TimeSpan.FromSeconds(30),
"the agent reports its version in hello");
using HttpResponseMessage on = await _backend.Admin.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["site.show_build"] = true }, "application/merge-patch+json"));
on.EnsureSuccessStatusCode();
try
{
JsonNode shown = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/about");
Assert.True(shown["show_build"]!.GetValue<bool>());
Assert.False(string.IsNullOrEmpty(shown["backend"]!["version"]!.GetValue<string>()));
Assert.Null(shown["backend"]!["commit"]); // 访客只看版本、tag、构建时间
JsonNode visible = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes");
JsonNode self = visible["items"]!.AsArray().First(n => n!["id"]!.GetValue<string>() == agent.Id)!;
Assert.False(string.IsNullOrEmpty(self["version"]!.GetValue<string>()));
}
finally
{
using HttpResponseMessage off = await _backend.Admin.PatchAsync("/api/admin/settings", BackendFixture.Json(new JsonObject { ["site.show_build"] = false }, "application/merge-patch+json"));
off.EnsureSuccessStatusCode();
}
}
private static IEnumerable<string> KeysOf(JsonNode? node)
{
switch (node)
{
case JsonObject obj:
foreach ((string key, JsonNode? value) in obj)
{
yield return key;
foreach (string inner in KeysOf(value))
{
yield return inner;
}
}
break;
case JsonArray array:
foreach (JsonNode? item in array)
{
foreach (string inner in KeysOf(item))
{
yield return inner;
}
}
break;
}
}
[GeneratedRegex(@"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b")]
private static partial Regex Ipv4();
}
@@ -175,9 +175,8 @@ public sealed class OperationsAndSecurityTests
Assert.Equal("fleet_unsupported", reply.Error!.Data!.Code); Assert.Equal("fleet_unsupported", reply.Error!.Data!.Code);
} }
// 连接仍然可用 // 连接仍然可用:路由器照常回复
ProbeAgentsResult list = await agent.RequestProbeAgentsAsync(); Assert.Equal("fleet_unsupported", (await agent.RequestRawAsync("fleet.exec")).Error!.Data!.Code);
Assert.Contains(list.Agents, a => a.AgentId == agent.NodeId);
} }
[Fact] [Fact]
@@ -195,9 +194,6 @@ public sealed class OperationsAndSecurityTests
public async Task PendingNodes_CannotCallProductMethods() public async Task PendingNodes_CannotCallProductMethods()
{ {
await using AgentSimulator agent = await _backend.ConnectAgentAsync(autoApprove: false); await using AgentSimulator agent = await _backend.ConnectAgentAsync(autoApprove: false);
RpcMessage reply = await agent.RequestRawAsync(MonitorMethods.ProbeAgents, JsonSerializer.SerializeToElement(new ProbeAgentsParams(), MonitorJsonContext.Default.ProbeAgentsParams));
Assert.Equal("fleet_node_pending", reply.Error!.Data!.Code);
// 状态与事件也不会被接受:监控数据里没有它 // 状态与事件也不会被接受:监控数据里没有它
await agent.SendEventAsync(AgentEventKinds.ProbeConnected, _backend.DefaultProbeId); await agent.SendEventAsync(AgentEventKinds.ProbeConnected, _backend.DefaultProbeId);
await Task.Delay(200); await Task.Delay(200);
@@ -19,13 +19,13 @@ public sealed class ProbeEndToEndTests
(await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == kind); (await _backend.GetJsonAsync(_backend.Admin, $"/api/admin/raw/chunks?agent_id={agent.Id}"))["items"]!.AsArray().Count(c => c!["kind"]!.GetValue<string>() == kind);
[Fact] [Fact]
public async Task Probe_ConnectsThroughTheBootstrapList_RunsTheLineTasks_AndTheBackendStoresMergedChunks() public async Task Probe_GetsTheAgentListFromThePanel_RunsTheLineTasks_AndTheBackendStoresMergedChunks()
{ {
await using AgentHarness agent = await AgentHarness.StartAsync(_backend); await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
await agent.OverrideLineTasksAsync(rateHz: 30); await agent.OverrideLineTasksAsync(rateHz: 30);
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend); await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.IsConnected(agent.Id)), because: "the probe should connect to the bootstrap agent"); await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.IsConnected(agent.Id)), because: "the panel sends the agent list to the probe, which then connects");
await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>()); await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>());
await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.Connection(agent.Id)!.RoleCount == 2), because: "udp_echo and tcp_echo"); await BackendFixture.EventuallyAsync(() => Task.FromResult(probe.App.Connection(agent.Id)!.RoleCount == 2), because: "udp_echo and tcp_echo");
@@ -38,6 +38,26 @@ public sealed class ProbeEndToEndTests
Assert.InRange(probe.App.Connection(agent.Id)!.ClockOffsetUs, -20_000, 20_000); Assert.InRange(probe.App.Connection(agent.Id)!.ClockOffsetUs, -20_000, 20_000);
} }
[Fact]
public async Task Probe_IntroducesItselfToThePanel_AndARevokedProbeIsLockedOut()
{
await _backend.SiteMustBePublicAsync();
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
string path = "/api/admin/probes/" + probe.ProbeId;
// 直连面板:在线,hello 里的系统、架构和版本都被记下来了
await BackendFixture.EventuallyAsync(async () => (await _backend.GetJsonAsync(_backend.Admin, path))["online"]!.GetValue<bool>(), because: "the probe links to the panel");
await BackendFixture.EventuallyAsync(
async () => (await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/nodes"))["items"]!.AsArray().Any(n => n!["id"]!.GetValue<string>() == probe.ProbeId && n["os"] is not null && n["arch"] is not null),
because: "the platform comes from the probe's own hello");
// 吊销:面板立刻断开它,之后连不上
using HttpResponseMessage revoked = await _backend.Admin.PostAsync(path + "/revoke", null);
revoked.EnsureSuccessStatusCode();
await BackendFixture.EventuallyAsync(async () => !(await _backend.GetJsonAsync(_backend.Admin, path))["online"]!.GetValue<bool>(), because: "a revoked probe is disconnected and cannot come back");
Assert.False(probe.App.Link.IsConnected);
}
[Fact] [Fact]
public async Task Probe_FollowsTheAgentListThePanelKnows_AndSavesItForTheNextStart() public async Task Probe_FollowsTheAgentListThePanelKnows_AndSavesItForTheNextStart()
{ {
@@ -64,7 +84,7 @@ public sealed class ProbeEndToEndTests
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend); await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend);
await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>()); await BackendFixture.EventuallyAsync(async () => (await AgentViewAsync(agent))["probe_connected"]!.GetValue<bool>());
using HttpResponseMessage accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000 } })); using HttpResponseMessage accepted = await _backend.Admin.PostAsync($"/api/admin/lines/{agent.Id}~{probe.ProbeId}/tests", BackendFixture.Json(new JsonObject { ["plan"] = "hy2_ladder", ["params"] = new JsonObject { ["size"] = 1000, ["duration_s"] = 5 } }));
Assert.True(accepted.StatusCode == System.Net.HttpStatusCode.Accepted, await accepted.Content.ReadAsStringAsync()); Assert.True(accepted.StatusCode == System.Net.HttpStatusCode.Accepted, await accepted.Content.ReadAsStringAsync());
string testId = JsonNode.Parse(await accepted.Content.ReadAsStringAsync())!["id"]!.GetValue<string>(); string testId = JsonNode.Parse(await accepted.Content.ReadAsStringAsync())!["id"]!.GetValue<string>();
@@ -78,7 +98,7 @@ public sealed class ProbeEndToEndTests
TimeSpan.FromSeconds(90)); TimeSpan.FromSeconds(90));
Assert.True(test["status"]!.GetValue<string>() == "succeeded", test.ToJsonString()); Assert.True(test["status"]!.GetValue<string>() == "succeeded", test.ToJsonString());
await BackendFixture.EventuallyAsync(async () => await ChunkCountAsync(agent, "udp_train") >= 5, because: "50 and 100 Mbit/s phases are split into several fragments"); await BackendFixture.EventuallyAsync(async () => await ChunkCountAsync(agent, "udp_train") >= 5, because: "the 100 Mbit/s phase (5 s of 1000 B packets, over 50000 records) is split into two fragments");
Assert.Equal(0, agent.App.Counters.FragmentsDiscarded); Assert.Equal(0, agent.App.Counters.FragmentsDiscarded);
} }
@@ -0,0 +1,130 @@
using System.Net;
using System.Text.Json.Nodes;
using Monitor.Protocol;
using Monitor.TestSupport;
namespace Monitor.Backend.Tests;
[Collection("probe-e2e")]
public sealed class UpdateTests
{
private const string Digest = "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
private readonly BackendFixture _backend;
public UpdateTests(BackendFixture backend)
{
_backend = backend;
}
// 面板不验签(节点宿主机上的更新器验),所以测试里签名随便写一个格式对的
private static string Manifest(string component, long seq, string digest = Digest) =>
$$"""{"schema":1,"component":"{{component}}","channel":"dev","image":"registry.example/monitor-dev/monitor-{{component}}","digest":"{{digest}}","tag":"dev-{{seq}}","version":"0.1.1","commit":"abc","built_at":"2026-10-06T00:00:00Z","seq":{{seq}}}""";
private async Task<HttpResponseMessage> PublishAsync(string manifest, string signature = "MEUCIQDc") =>
await _backend.Admin.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = signature }));
[Fact]
public async Task Release_IsForwardedByteForByte_ToOnlineNodesOfThatComponent_AndToNodesThatConnectLater()
{
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000;
await using AgentSimulator online = await _backend.ConnectAgentAsync();
string manifest = Manifest("agent", seq);
using HttpResponseMessage published = await PublishAsync(manifest);
Assert.Equal(HttpStatusCode.Created, published.StatusCode);
JsonNode release = JsonNode.Parse(await published.Content.ReadAsStringAsync())!;
Assert.Equal("agent", release["component"]!.GetValue<string>());
Assert.Equal(Digest, release["digest"]!.GetValue<string>());
await BackendFixture.EventuallyAsync(() => Task.FromResult(online.UpdateOffers.ToArray().Any(o => o.Manifest == manifest && o.Signature == "MEUCIQDc")), because: "the offer reaches online agents unchanged");
// 之后才连上的节点,hello 之后补发
await using AgentSimulator late = await _backend.ConnectAgentAsync();
await BackendFixture.EventuallyAsync(() => Task.FromResult(late.UpdateOffers.ToArray().Any(o => o.Manifest == manifest)), because: "a node that connects later still gets the latest offer");
// 另一个组件(probe)的清单不会发给 Agent
using HttpResponseMessage probeRelease = await PublishAsync(Manifest("probe", seq));
Assert.Equal(HttpStatusCode.Created, probeRelease.StatusCode);
await Task.Delay(300);
Assert.DoesNotContain(online.UpdateOffers.ToArray(), o => o.Manifest.Contains("\"component\":\"probe\"", StringComparison.Ordinal));
}
[Fact]
public async Task Release_Registration_RefusesBadManifests_StaleSeq_AndNeedsAnAdmin()
{
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000 + 5;
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync("{not json")).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("agent", seq, digest: "sha256:short"))).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("backend", seq))).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("agent", seq), signature: "not base64!")).StatusCode);
Assert.Equal(HttpStatusCode.Created, (await PublishAsync(Manifest("agent", seq))).StatusCode);
Assert.Equal(HttpStatusCode.Conflict, (await PublishAsync(Manifest("agent", seq))).StatusCode); // 同一个 seq:重放
Assert.Equal(HttpStatusCode.Conflict, (await PublishAsync(Manifest("agent", seq - 1))).StatusCode); // 更旧的 seq
using HttpResponseMessage anonymous = await _backend.Anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("agent", seq + 1), ["signature"] = "MEUC" }));
Assert.Contains(anonymous.StatusCode, new[] { HttpStatusCode.Unauthorized, HttpStatusCode.Forbidden });
}
[Fact]
public async Task ReleaseToken_CanOnlyRegisterReleases_AndAReadTokenCannotEvenDoThat()
{
async Task<string> NewTokenAsync(string scope)
{
using HttpResponseMessage created = await _backend.Admin.PostAsync("/api/admin/api-tokens", BackendFixture.Json(new JsonObject { ["name"] = _backend.UniqueName("ci"), ["scope"] = scope }));
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
return JsonNode.Parse(await created.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
}
HttpClient Client(string token)
{
var client = new HttpClient { BaseAddress = _backend.BaseAddress };
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
return client;
}
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000 + 50;
using HttpClient release = Client(await NewTokenAsync("release"));
using HttpClient read = Client(await NewTokenAsync("read"));
var body = BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq), ["signature"] = "MEUC" });
using HttpResponseMessage forbidden = await read.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq), ["signature"] = "MEUC" }));
Assert.Equal(HttpStatusCode.Forbidden, forbidden.StatusCode);
using HttpResponseMessage published = await release.PostAsync("/api/admin/releases", body);
Assert.Equal(HttpStatusCode.Created, published.StatusCode);
// release 令牌只能登记发布清单:读不了任何东西,也不能做别的管理操作
Assert.Equal(HttpStatusCode.Forbidden, (await release.GetAsync("/api/admin/agents")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await release.GetAsync("/api/admin/releases")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await release.PostAsync("/api/admin/nodes/agt_x/update", null)).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await _backend.Anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq + 1), ["signature"] = "MEUC" }))).StatusCode);
}
[Fact]
public async Task UpdateRequest_ReachesTheNode_AndItsReportedStatusShowsInBuildInfo()
{
await using AgentSimulator agent = await _backend.ConnectAgentAsync();
using HttpResponseMessage request = await _backend.Admin.PostAsync($"/api/admin/nodes/{agent.NodeId.Value}/update", null);
Assert.Equal(HttpStatusCode.NoContent, request.StatusCode);
await BackendFixture.EventuallyAsync(() => Task.FromResult(agent.UpdateApplies == 1));
// 节点(宿主机更新器)报告状态:面板在构建信息里显示
await agent.SendUpdateStatusAsync(new UpdateStatus { State = "available", Available = new UpdateAvailable { Tag = "dev-9", Version = "0.1.1", Digest = Digest, Seq = 9 }, At = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() });
await BackendFixture.EventuallyAsync(async () =>
{
JsonNode info = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info");
JsonNode? node = info["nodes"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == agent.NodeId.Value);
return node?["update"]?["state"]?.GetValue<string>() == "available" && node["update"]!["available_tag"]!.GetValue<string>() == "dev-9";
}, because: "the update state comes from the node's heartbeat");
// 节点离线之后不能请求更新
string id = agent.NodeId.Value;
await agent.DisposeAsync();
await BackendFixture.EventuallyAsync(async () =>
{
using HttpResponseMessage offline = await _backend.Admin.PostAsync($"/api/admin/nodes/{id}/update", null);
return offline.StatusCode == HttpStatusCode.Conflict || offline.StatusCode == HttpStatusCode.NotFound;
});
}
}
@@ -46,10 +46,15 @@ public sealed class AgentSimulator : IAsyncDisposable
public ConcurrentQueue<TaskSet> TaskSets { get; } = new(); public ConcurrentQueue<TaskSet> TaskSets { get; } = new();
public ConcurrentQueue<ProbeAgentsResult> ProbeAgentLists { get; } = new();
public ConcurrentQueue<TestRunParams> TestRuns { get; } = new(); public ConcurrentQueue<TestRunParams> TestRuns { get; } = new();
/// <summary>面板发来的发布清单(<c>monitor.update.offer</c>)和更新请求(<c>monitor.update.apply</c>)。</summary>
public ConcurrentQueue<UpdateOfferParams> UpdateOffers { get; } = new();
private int _updateApplies;
public int UpdateApplies => Volatile.Read(ref _updateApplies);
public ConcurrentQueue<TestId> TestCancels { get; } = new(); public ConcurrentQueue<TestId> TestCancels { get; } = new();
public IReadOnlyList<RejectedItem> RejectNextState { get; set; } = []; public IReadOnlyList<RejectedItem> RejectNextState { get; set; } = [];
@@ -190,11 +195,12 @@ public sealed class AgentSimulator : IAsyncDisposable
new TestStatusParams { TestId = test, Status = status, Phase = phase, ErrorCode = errorCode }, new TestStatusParams { TestId = test, Status = status, Phase = phase, ErrorCode = errorCode },
MonitorJsonContext.Default.TestStatusParams)); MonitorJsonContext.Default.TestStatusParams));
public async Task<ProbeAgentsResult> RequestProbeAgentsAsync() /// <summary>像节点的更新器报告的那样,随心跳上报更新状态。</summary>
{ public Task SendUpdateStatusAsync(UpdateStatus status) =>
RpcMessage reply = await RequestAsync(MonitorMethods.ProbeAgents, JsonSerializer.SerializeToElement(new ProbeAgentsParams(), MonitorJsonContext.Default.ProbeAgentsParams)); SendAsync(RpcMessage.Notification(
return reply.Result!.Value.Deserialize(MonitorJsonContext.Default.ProbeAgentsResult)!; FleetMethods.Heartbeat,
} new HeartbeatParams { Extra = JsonSerializer.SerializeToElement(new AgentHeartbeatExtra { Update = status }, MonitorJsonContext.Default.AgentHeartbeatExtra) },
FleetRpcContext.Default.HeartbeatParams));
public async Task<RpcMessage> RequestRawAsync(string method, JsonElement? parameters = null) => await RequestAsync(method, parameters); public async Task<RpcMessage> RequestRawAsync(string method, JsonElement? parameters = null) => await RequestAsync(method, parameters);
@@ -397,6 +403,12 @@ public sealed class AgentSimulator : IAsyncDisposable
case FleetMethods.Revoked: case FleetMethods.Revoked:
Revoked = true; Revoked = true;
break; break;
case MonitorMethods.UpdateOffer:
UpdateOffers.Enqueue(message.Params!.Value.Deserialize(MonitorJsonContext.Default.UpdateOfferParams)!);
break;
case MonitorMethods.UpdateApply:
Interlocked.Increment(ref _updateApplies);
break;
case MonitorMethods.TestRun: case MonitorMethods.TestRun:
TestRunParams run = message.Params!.Value.Deserialize(MonitorJsonContext.Default.TestRunParams)!; TestRunParams run = message.Params!.Value.Deserialize(MonitorJsonContext.Default.TestRunParams)!;
TestRuns.Enqueue(run); TestRuns.Enqueue(run);
@@ -416,9 +428,6 @@ public sealed class AgentSimulator : IAsyncDisposable
TestCancels.Enqueue(cancel.TestId); TestCancels.Enqueue(cancel.TestId);
await SendAsync(RpcMessage.Success(message.Id!.Value, new TestRunResult { Accepted = true }, MonitorJsonContext.Default.TestRunResult)); await SendAsync(RpcMessage.Success(message.Id!.Value, new TestRunResult { Accepted = true }, MonitorJsonContext.Default.TestRunResult));
break; break;
case MonitorMethods.ProbeAgentsChanged:
ProbeAgentLists.Enqueue(message.Params!.Value.Deserialize(MonitorJsonContext.Default.ProbeAgentsResult)!);
break;
} }
} }
} }
@@ -61,8 +61,6 @@ public sealed class ProbeSimulator : IAsyncDisposable
public ConcurrentQueue<TestId> Aborts { get; } = new(); public ConcurrentQueue<TestId> Aborts { get; } = new();
public ConcurrentQueue<ProbeAgentsResult> AgentLists { get; } = new();
public bool IsConnected => _rpc is not null && !_stop.IsCancellationRequested; public bool IsConnected => _rpc is not null && !_stop.IsCancellationRequested;
public long NowUs() => (TimeProvider.System.GetTimestamp() - _origin) * 1_000_000 / TimeProvider.System.TimestampFrequency; public long NowUs() => (TimeProvider.System.GetTimestamp() - _origin) * 1_000_000 / TimeProvider.System.TimestampFrequency;
@@ -347,14 +345,6 @@ public sealed class ProbeSimulator : IAsyncDisposable
probe.Aborts.Enqueue(abort.TestId); probe.Aborts.Enqueue(abort.TestId);
return ValueTask.CompletedTask; return ValueTask.CompletedTask;
}) })
.OnNotify(
MonitorMethods.ProbeAgentsChanged,
MonitorJsonContext.Default.ProbeAgentsResult,
(probe, list, _) =>
{
probe.AgentLists.Enqueue(list);
return ValueTask.CompletedTask;
})
.OnNotify( .OnNotify(
FleetMethods.OutboxAck, FleetMethods.OutboxAck,
FleetRpcContext.Default.OutboxAckParams, FleetRpcContext.Default.OutboxAckParams,
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
# Build and push one opt-in main Linux amd64 image to the dev repository in Zot.
set -euo pipefail
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
echo "main amd64 builds require a main push with [CI] in the commit message" >&2
exit 2
}
component="${1:?usage: build-main-amd64.sh <backend|agent|probe>}"
case "$component" in
backend|agent|probe) ;;
*) echo "unsupported component: $component" >&2; exit 2 ;;
esac
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
echo "Zot credentials are required" >&2
exit 2
}
[[ "${CI_COMMIT_SHA:-}" =~ ^[0-9a-fA-F]{40}$ ]] || {
echo "CI_COMMIT_SHA must be a full commit SHA" >&2
exit 2
}
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
registry="${ZOT_REGISTRY:-registry.oeo.one}"
repository="$registry/monitor-dev/monitor-$component"
work="$root/.release/main-amd64-build"
base_home="/home/justme/woodpecker/buildah-home/main-amd64-build-$component"
short_sha="${CI_COMMIT_SHA:0:12}"
dev_tag="dev-$short_sha"
mkdir -p "$work" "$base_home"
if [[ "$component" == backend ]]; then
BUILDAH_HOME="$base_home" \
ZOT_REGISTRY="$registry" \
ZOT_IMAGE_REPOSITORY="$repository" \
IMAGE_TAGS="latest,$dev_tag" \
IMAGE_PUBLISH=true \
bash "$root/ci/release-linux.sh" amd64 backend
exit 0
fi
export HOME="$base_home"
export BUILDAH_HOME="$base_home"
export REGISTRY_AUTH_FILE="$work/$component-auth.json"
context="$work/$component"
local_image="localhost/monitor-$component:main-amd64-check"
python3 "$root/ci/stage-node-image.py" "$component" "$context"
buildah bud --layers --network host --tls-verify=true --arch amd64 \
--build-arg APP_VERSION=main \
--build-arg "MONITOR_TAG=$dev_tag" --build-arg "MONITOR_COMMIT=${CI_COMMIT_SHA:0:10}" --build-arg "MONITOR_BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
-f "$context/Containerfile" \
-t "$local_image" "$context"
logged_in=false
for attempt in 1 2 3 4; do
if buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$registry" <<< "$ZOT_SECRET" >/dev/null; then
logged_in=true
break
fi
if [[ "$attempt" -lt 4 ]]; then
echo "Zot login attempt $attempt failed; retrying in 3 seconds" >&2
sleep 3
fi
done
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
for tag in latest "$dev_tag"; do
pushed=false
for attempt in 1 2 3; do
if buildah push --tls-verify=true --digestfile "$work/$component.digest" "$local_image" "docker://$repository:$tag"; then
pushed=true
break
fi
if [[ "$attempt" -lt 3 ]]; then
echo "Zot push for $component:$tag attempt $attempt failed; retrying in 5 seconds" >&2
sleep 5
fi
done
[[ "$pushed" == true ]] || { echo "could not push $repository:$tag after 3 attempts" >&2; exit 1; }
echo "pushed development image $repository:$tag"
done
# 发布清单:给镜像的 digest 签名(私钥只在 CI 密钥里),再登记到面板;面板把它原样转给在线的节点,节点宿主机上的更新器用本机的公钥验签后才会更新。
# 没有配置签名密钥(例如 fork)就只构建镜像、不发清单。
if [[ -n "${RELEASE_SIGNING_KEY:-}" ]]; then
digest="$(cat "$work/$component.digest")"
release_dir="$work/$component-release"
bash "$root/ci/release-manifest.sh" "$component" "$repository" "$digest" "$dev_tag" dev "$release_dir"
if [[ -n "${RELEASE_API_URL:-}" && -n "${RELEASE_API_TOKEN:-}" ]]; then
python3 -c 'import json,sys; print(json.dumps({"manifest": open(sys.argv[1]).read(), "signature": open(sys.argv[2]).read()}))' "$release_dir/manifest.json" "$release_dir/manifest.sig" | curl --fail-with-body --silent --show-error --retry 3 -X POST "${RELEASE_API_URL%/}/api/admin/releases" -H "Authorization: Bearer $RELEASE_API_TOKEN" -H 'Content-Type: application/json' --data-binary @- >/dev/null
echo "registered the signed $component release with the panel"
fi
fi
+1 -1
View File
@@ -26,7 +26,7 @@ printf 'machine %s login justme password %s\n' "$gitea_host" "$GITEA_TOKEN" > "$
chmod 600 "$netrc" chmod 600 "$netrc"
trap 'rm -rf "$tmp"' EXIT trap 'rm -rf "$tmp"' EXIT
release_body="Agent and Probe builds: Linux glibc and musl (x64, arm64), macOS (x64, arm64), and Windows (x64, arm64). macOS binaries are not Developer ID signed or notarized. Backend OCI images: registry.oeo.one/monitor/monitor-backend:$version (linux/amd64 and linux/arm64)." release_body="Agent and Probe builds: Linux glibc and musl (x64, arm64), macOS (x64, arm64), and Windows (x64, arm64). Linux glibc Agent and Probe OCI images: registry.oeo.one/monitor/monitor-agent:$version and registry.oeo.one/monitor/monitor-probe:$version (linux/amd64 and linux/arm64). Backend OCI image: registry.oeo.one/monitor/monitor-backend:$version (linux/amd64 and linux/arm64). macOS binaries are not Developer ID signed or notarized."
payload="$(printf '{"tag_name":"%s","target_commitish":"%s","name":"MyMonitor %s","body":"%s","draft":false,"prerelease":false}' \ payload="$(printf '{"tag_name":"%s","target_commitish":"%s","name":"MyMonitor %s","body":"%s","draft":false,"prerelease":false}' \
"$tag" "${CI_COMMIT_SHA:-}" "$tag" "$release_body")" "$tag" "${CI_COMMIT_SHA:-}" "$tag" "$release_body")"
release_status="$(curl --silent --show-error --retry 3 --netrc-file "$netrc" \ release_status="$(curl --silent --show-error --retry 3 --netrc-file "$netrc" \
+6
View File
@@ -51,6 +51,11 @@ $output = Join-Path $root ".release\publish\$RuntimeIdentifier\$Component"
$buildRoot = Join-Path $root ".release\build\$RuntimeIdentifier-$Component" $buildRoot = Join-Path $root ".release\build\$RuntimeIdentifier-$Component"
$null = New-Item -ItemType Directory -Force -Path $output $null = New-Item -ItemType Directory -Force -Path $output
# 构建信息:程序里显示的 tag、提交、构建时间
$env:MONITOR_TAG = if ($env:CI_COMMIT_TAG) { $env:CI_COMMIT_TAG } else { "main-$($env:CI_COMMIT_SHA.Substring(0, 10))" }
$env:MONITOR_COMMIT = $env:CI_COMMIT_SHA.Substring(0, 10)
$env:MONITOR_BUILT_AT = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
Push-Location $backend Push-Location $backend
try { try {
& dotnet publish $project ` & dotnet publish $project `
@@ -59,6 +64,7 @@ try {
-r $RuntimeIdentifier ` -r $RuntimeIdentifier `
--self-contained true ` --self-contained true `
-p:PublishAot=true ` -p:PublishAot=true `
-p:UseAotAnywhere=true `
-p:StripSymbols=true ` -p:StripSymbols=true `
-p:DebugType=none ` -p:DebugType=none `
-p:DebugSymbols=false ` -p:DebugSymbols=false `
+83 -32
View File
@@ -1,45 +1,82 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Publish one framework-dependent Backend OCI image for the requested arch. # Build a musl NativeAOT Backend OCI image with statically linked SQLite; publishing is optional.
set -euo pipefail set -euo pipefail
arch="${1:?usage: release-linux.sh <amd64|arm64>}" arch="${1:?usage: release-linux.sh <amd64|arm64>}"
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
tag="${CI_COMMIT_TAG:-}" tag="${CI_COMMIT_TAG:-}"
[[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || { if [[ -n "$tag" ]]; then
echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2 [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || {
exit 2 echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2
} exit 2
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || { }
echo "Zot credentials are required" >&2; exit 2; version="${tag#v}"
else
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
echo "untagged backend builds require a main push with [CI] in the commit message" >&2
exit 2
}
version=main
fi
image_publish="${IMAGE_PUBLISH:-true}"
[[ "$image_publish" == true || "$image_publish" == false ]] || {
echo "IMAGE_PUBLISH must be true or false" >&2; exit 2;
} }
if [[ "$image_publish" == true ]]; then
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
echo "Zot credentials are required" >&2; exit 2;
}
fi
case "$arch" in case "$arch" in
amd64|arm64) ;; amd64) rid=linux-musl-x64 ;;
arm64) rid=linux-musl-arm64 ;;
*) echo "unsupported architecture: $arch" >&2; exit 2 ;; *) echo "unsupported architecture: $arch" >&2; exit 2 ;;
esac esac
version="${tag#v}"
project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)" project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
if [[ "$version" != "$project_version" ]]; then if [[ -n "$tag" && "$version" != "$project_version" ]]; then
echo "tag version $version does not match project version $project_version" >&2 echo "tag version $version does not match project version $project_version" >&2
exit 1 exit 1
fi fi
registry="${ZOT_REGISTRY:-registry.oeo.one}" registry="${ZOT_REGISTRY:-registry.oeo.one}"
image="$registry/monitor/monitor-backend" image="${ZOT_IMAGE_REPOSITORY:-$registry/monitor/monitor-backend}"
output="$root/.release/linux-$arch/backend" output="$root/.release/linux-$arch/backend"
build_root="$root/.release/build/linux-$arch-backend" build_root="$root/.release/build/linux-$arch-backend"
container_context="$root/.release/container/linux-$arch" container_context="$root/.release/container/linux-$arch"
mkdir -p "$output" "$build_root" "$container_context/app" "$container_context/data" mkdir -p "$output" "$build_root" "$container_context/app" "$container_context/data"
# 构建信息:tag 发布用 tag,main 构建用 main-<提交>
export MONITOR_TAG="${MONITOR_TAG:-${tag:-main-${CI_COMMIT_SHA:0:10}}}"
export MONITOR_COMMIT="${MONITOR_COMMIT:-${CI_COMMIT_SHA:0:10}}"
export MONITOR_BUILT_AT="${MONITOR_BUILT_AT:-$(date -u +%Y-%m-%dT%H:%M:%SZ)}"
(cd "$root/backend" && dotnet publish src/Monitor.Backend \ (cd "$root/backend" && dotnet publish src/Monitor.Backend \
-m:3 -c Release --no-self-contained \ -m:3 -c Release -r "$rid" --self-contained true \
-p:DebugType=none -p:DebugSymbols=false \ -p:DebugType=none -p:DebugSymbols=false \
-p:UseAppHost=false -p:OpenApiGenerateDocuments=false \ -p:PublishAot=true -p:UseAotAnywhere=true -p:SqliteStaticLink=true -p:OpenApiGenerateDocuments=false \
--artifacts-path "$build_root" -o "$output") --artifacts-path "$build_root" -o "$output")
find "$output" -type f -name '*.pdb' -delete [[ -x "$output/monitor-backend" ]] || { echo "NativeAOT Backend executable is missing" >&2; exit 1; }
cp -a "$output/." "$container_context/app/" [[ "$(wc -c < "$output/monitor-backend")" -gt 2000000 ]] || { echo "Backend output is an apphost, not a NativeAOT executable" >&2; exit 1; }
install -m 0755 "$output/monitor-backend" "$container_context/app/monitor-backend"
: > "$container_context/data/.keep" : > "$container_context/data/.keep"
# Woodpecker's local backend isolates HOME per workflow. Keep Buildah's image
# store outside that temporary directory so the runtime base image is reusable
# across releases. Separate stores avoid concurrent amd64/arm64 writers.
buildah_home="${BUILDAH_HOME:-/home/justme/woodpecker/buildah-home/$arch}"
mkdir -p "$buildah_home"
export HOME="$buildah_home"
local_image="localhost/monitor-backend:$version-$arch"
buildah bud --tls-verify=true --arch "$arch" \
--build-arg "APP_VERSION=$version" \
-f "$root/packaging/container/Backend.Containerfile" \
-t "$local_image" "$container_context"
if [[ "$image_publish" == false ]]; then
echo "built local Backend image for $arch; not publishing it"
exit 0
fi
export REGISTRY_AUTH_FILE="$root/.release/auth/zot-linux-$arch.json" export REGISTRY_AUTH_FILE="$root/.release/auth/zot-linux-$arch.json"
mkdir -p "$(dirname "$REGISTRY_AUTH_FILE")" mkdir -p "$(dirname "$REGISTRY_AUTH_FILE")"
logged_in=false logged_in=false
@@ -54,21 +91,35 @@ for attempt in 1 2 3 4; do
fi fi
done done
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; } [[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
local_image="localhost/monitor-backend:$version-$arch" push_tag() {
buildah bud --tls-verify=true --arch "$arch" \ local target_tag="$1" pushed=false
--build-arg "APP_VERSION=$version" \ for attempt in 1 2 3; do
-f "$root/packaging/container/Backend.Containerfile" \ if buildah push --tls-verify=true "$local_image" "docker://$image:$target_tag"; then
-t "$local_image" "$container_context" >/dev/null pushed=true
pushed=false break
for attempt in 1 2 3; do fi
if buildah push --tls-verify=true "$local_image" "docker://$image:$version-$arch" >/dev/null; then if [[ "$attempt" -lt 3 ]]; then
pushed=true echo "Zot push attempt $attempt failed; retrying in 5 seconds" >&2
break sleep 5
fi
done
[[ "$pushed" == true ]] || { echo "could not push image to Zot after 3 attempts" >&2; return 1; }
echo "pushed Zot image $image:$target_tag"
}
if [[ -n "${IMAGE_TAGS:-}" ]]; then
IFS=',' read -r -a image_tags <<< "$IMAGE_TAGS"
((${#image_tags[@]} > 0)) || { echo "IMAGE_TAGS must contain at least one tag" >&2; exit 2; }
for image_tag in "${image_tags[@]}"; do
[[ "$image_tag" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]] || {
echo "invalid image tag: $image_tag" >&2
exit 2
}
push_tag "$image_tag"
done
else
push_tag "$version-$arch"
if [[ -z "$tag" ]]; then
push_tag "$version"
fi fi
if [[ "$attempt" -lt 3 ]]; then fi
echo "Zot push attempt $attempt failed; retrying in 5 seconds" >&2
sleep 5
fi
done
[[ "$pushed" == true ]] || { echo "could not push image to Zot after 3 attempts" >&2; exit 1; }
echo "pushed Zot image $image:$version-$arch"
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# 生成并签名一个镜像的发布清单:ci/release-manifest.sh <组件> <镜像仓库> <digest> <tag> <通道> <输出目录>
# 签名私钥从环境变量 RELEASE_SIGNING_KEY(PEM 内容)读取,只写进一个 600 权限的临时文件,用完即删;私钥只放在 CI 密钥里,不进仓库、不上传面板。
# 输出:<输出目录>/manifest.json(扁平 JSON,签名对象是它的原始字节)和 manifest.sig(base64 的 DER ECDSA 签名)。
set -euo pipefail
component="${1:?usage: release-manifest.sh <component> <repository> <digest> <tag> <channel> <outdir>}"
repository="${2:?}"; digest="${3:?}"; tag="${4:?}"; channel="${5:?}"; outdir="${6:?}"
[[ "$component" =~ ^(agent|probe|backend)$ ]] || { echo "unsupported component: $component" >&2; exit 2; }
[[ "$digest" =~ ^sha256:[0-9a-f]{64}$ ]] || { echo "digest must look like sha256:<64 hex>" >&2; exit 2; }
[[ "$repository" =~ ^[a-z0-9./_-]+$ && "$tag" =~ ^[A-Za-z0-9._-]+$ && "$channel" =~ ^[a-z0-9._-]+$ ]] || { echo "unexpected characters in repository, tag or channel" >&2; exit 2; }
[[ -n "${RELEASE_SIGNING_KEY:-}" ]] || { echo "RELEASE_SIGNING_KEY is required" >&2; exit 2; }
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
commit="$(git -C "$root" rev-parse --short=10 HEAD 2>/dev/null || true)"
built_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
seq="$(date +%s)"
mkdir -p "$outdir"
printf '{"schema":1,"component":"%s","channel":"%s","image":"%s","digest":"%s","tag":"%s","version":"%s","commit":"%s","built_at":"%s","seq":%s}' \
"$component" "$channel" "$repository" "$digest" "$tag" "$version" "$commit" "$built_at" "$seq" > "$outdir/manifest.json"
key="$(mktemp)"
trap 'rm -f "$key" "$outdir/manifest.sig.bin"' EXIT
chmod 600 "$key"
printf '%s\n' "$RELEASE_SIGNING_KEY" > "$key"
openssl dgst -sha256 -sign "$key" -out "$outdir/manifest.sig.bin" "$outdir/manifest.json"
base64 -w0 "$outdir/manifest.sig.bin" > "$outdir/manifest.sig" 2>/dev/null || base64 "$outdir/manifest.sig.bin" | tr -d '\n' > "$outdir/manifest.sig"
echo "signed $component $tag ($digest) seq=$seq"
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env bash
# Publish musl OCI images for the Linux Agent and Probe on amd64 and arm64.
set -euo pipefail
tag="${CI_COMMIT_TAG:-}"
[[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || {
echo "release must run from a vMAJOR.MINOR.PATCH tag" >&2
exit 2
}
[[ -n "${GITEA_TOKEN:-}" ]] || { echo "GITEA_TOKEN is required" >&2; exit 2; }
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
echo "Zot credentials are required" >&2
exit 2
}
version="${tag#v}"
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
project_version="$(sed -n 's#.*<Version>\(.*\)</Version>.*#\1#p' "$root/backend/Directory.Build.props" | head -1)"
if [[ "$version" != "$project_version" ]]; then
echo "tag version $version does not match project version $project_version" >&2
exit 1
fi
gitea_base="${GITEA_BASE:-http://127.0.0.1:3100}"
gitea_base="${gitea_base%/}"
gitea_host="${gitea_base#https://}"
gitea_host="${gitea_host#http://}"
gitea_host="${gitea_host%%/*}"
gitea_host="${gitea_host%%:*}"
registry="${ZOT_REGISTRY:-registry.oeo.one}"
image_root="$registry/monitor"
work="$root/.release/node-images"
netrc="$work/netrc"
buildah_home="/home/justme/woodpecker/buildah-home/node-images"
mkdir -p "$buildah_home"
export HOME="$buildah_home"
mkdir -p "$work"
chmod 700 "$work"
printf 'machine %s login justme password %s\n' "$gitea_host" "$GITEA_TOKEN" > "$netrc"
chmod 600 "$netrc"
export REGISTRY_AUTH_FILE="$work/zot-auth.json"
trap 'rm -rf "$work"' EXIT
logged_in=false
for attempt in 1 2 3 4; do
if buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$registry" <<< "$ZOT_SECRET" >/dev/null; then
logged_in=true
break
fi
if [[ "$attempt" -lt 4 ]]; then
echo "Zot login attempt $attempt failed; retrying in 3 seconds" >&2
sleep 3
fi
done
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
build_arch_image() {
local component="$1" arch="$2" suffix="$3"
local filename="monitor-$component-v$version-linux-$suffix.tar.gz"
local context="$work/$component-$arch"
local archive="$work/$filename"
local containerfile="$root/packaging/container/${component^}.Containerfile"
local image="$image_root/monitor-$component"
local local_image="localhost/monitor-$component:$version-$arch"
curl --fail-with-body --silent --show-error --retry 3 --netrc-file "$netrc" \
"$gitea_base/api/packages/justme/generic/monitor-node/$version/$filename" -o "$archive"
mkdir -p "$context/app" "$context/config"
if [[ "$component" == agent ]]; then
tar -xzf "$archive" -C "$work" monitor-agent fleet-enroll
install -m 0755 "$work/fleet-enroll" "$context/app/fleet-enroll"
else
tar -xzf "$archive" -C "$work" "monitor-$component"
fi
install -m 0755 "$work/monitor-$component" "$context/app/monitor-$component"
: > "$context/config/.keep"
buildah bud --tls-verify=true --arch "$arch" \
--build-arg "APP_VERSION=$version" \
-f "$containerfile" -t "$local_image" "$context" >/dev/null
local pushed=false
for attempt in 1 2 3; do
if buildah push --tls-verify=true "$local_image" "docker://$image:$version-$arch" >/dev/null; then
pushed=true
break
fi
if [[ "$attempt" -lt 3 ]]; then
echo "Zot push for $component/$arch attempt $attempt failed; retrying in 5 seconds" >&2
sleep 5
fi
done
[[ "$pushed" == true ]] || { echo "could not push $component/$arch image after 3 attempts" >&2; return 1; }
echo "pushed Zot image $image:$version-$arch"
}
publish_index() {
local component="$1" image="$image_root/monitor-$component"
local manifest="localhost/monitor-$component:$version-multiarch"
buildah manifest rm "$manifest" >/dev/null 2>&1 || true
buildah manifest create "$manifest" >/dev/null
buildah manifest add "$manifest" "docker://$image:$version-amd64" >/dev/null
buildah manifest add "$manifest" "docker://$image:$version-arm64" >/dev/null
buildah manifest push --all --format oci "$manifest" "docker://$image:$version" >/dev/null
if [[ "$version" != *-* ]]; then
buildah manifest push --all --format oci "$manifest" "docker://$image:latest" >/dev/null
fi
buildah manifest rm "$manifest" >/dev/null
echo "published multi-architecture index $image:$version"
if [[ "$version" != *-* ]]; then
echo "published multi-architecture index $image:latest"
fi
}
for component in agent probe; do
build_arch_image "$component" amd64 musl-amd64
build_arch_image "$component" arm64 musl-arm64
publish_index "$component"
done
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env python3
"""Stage source and restore files for the Linux amd64 Agent or Probe image build."""
from __future__ import annotations
import argparse
import shutil
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
BACKEND = ROOT / "backend"
IGNORED_PARTS = {".git", ".release", "artifacts", "bin", "obj", "TestResults", "tests"}
GLOBAL_BACKEND_FILES = {
"Directory.Build.props",
"Directory.Build.targets",
"Directory.Packages.props",
"global.json",
"NuGet.Config",
"NuGet.config",
}
# The container stages backend as a standalone tree. Copy the repository-wide
# SDK and package versions into it so restore/publish use the same configuration.
ROOT_BUILD_FILES = {"Directory.Packages.props", "global.json"}
CONTAINERFILES = {
"agent": ROOT / "packaging/container/Agent.NativeBuild.Containerfile",
"probe": ROOT / "packaging/container/Probe.NativeBuild.Containerfile",
}
def stage(component: str, destination: Path) -> None:
if destination.exists():
shutil.rmtree(destination)
restore = destination / "restore/backend"
source = destination / "source/backend"
restore.mkdir(parents=True)
source.mkdir(parents=True)
shutil.copy2(CONTAINERFILES[component], destination / "Containerfile")
for path in BACKEND.rglob("*"):
if not path.is_file() or IGNORED_PARTS.intersection(path.parts):
continue
relative = path.relative_to(BACKEND)
if (
path.suffix in {".csproj", ".props", ".targets", ".slnx"}
or path.name in GLOBAL_BACKEND_FILES
or path.name == "packages.lock.json"
):
target = restore / relative
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(path, target)
target = source / relative
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(path, target)
for name in ROOT_BUILD_FILES:
for directory in (restore, source):
shutil.copy2(ROOT / name, directory / name)
config = destination / "config"
config.mkdir()
(config / ".keep").write_text("", encoding="utf-8")
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("component", choices=tuple(CONTAINERFILES))
parser.add_argument("destination", type=Path)
args = parser.parse_args()
stage(args.component, args.destination)
if __name__ == "__main__":
main()
+1 -1
View File
@@ -1,6 +1,6 @@
services: services:
backend: backend:
image: registry.oeo.one/monitor/monitor-backend:${MONITOR_VERSION:-0.1.0} image: registry.oeo.one/monitor/monitor-backend:${MONITOR_VERSION:-0.1.1}
container_name: monitor-backend container_name: monitor-backend
restart: unless-stopped restart: unless-stopped
init: true init: true
Loaded 100 of 170 files, more files were not shown because too many files have changed in this diff. Show more