15 Commits
Author SHA1 Message Date
justme 725fe17a7d chore: release 0.4.0 [CI]
ci/woodpecker/tag/windows-release Pipeline was successful
ci/woodpecker/push/main-amd64-build Pipeline was successful
ci/woodpecker/tag/linux-release Pipeline was successful
ci/woodpecker/tag/gitea-release Pipeline was successful
2026-10-09 03:54:56 +08:00
justme e49990776b feat(display): falling snow in three depth layers with wind, mouse and bullet time; appearance panel; dark glass tinted from the background [CI SKIP] 2026-10-09 03:54:49 +08:00
justme f8e8bb8acb feat(display): pointer-sway background with a dark variant; acrylic glass tokens [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
2026-10-09 03:09:14 +08:00
justme 70f20d38d0 feat(display): site name and avatar in the top bar, content aligned left [CI SKIP] 2026-10-09 03:08:54 +08:00
justme 7ff03a9628 fix(display): a hovered card no longer covers the top bar [CI SKIP] 2026-10-09 03:08:53 +08:00
justme 5dce387c87 ci: build a dev image to test automatic node updates [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
2026-10-09 02:10:38 +08:00
justme 535bb3517d feat: node updates by restart, auto by default; Podman Quadlet install from the panel alongside compose [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
2026-10-09 01:33:22 +08:00
justme 88b1a0a9bb feat: probe host stats, node cards with a separate lines panel grouped by carrier, Windows host sampling, manual carrier override [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
2026-10-08 21:56:19 +08:00
justme bac1c27e72 fix: compose passes .env through env_file; clearer name and channel messages [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
podman-compose does not substitute defaults inside "environment:" (a value
like ${MOSHIRO_NAME:-} reached the node as the literal text), so the node was
enrolled with an invalid name and reported the wrong network scope. The three
compose files now read .env with env_file and only interpolate image and
network_mode; the Backend .env keys are the container's own names
(MoShiro__PublicUrl, MoShiro__ClaimCode).

- The generated .env no longer carries a comment line for MOSHIRO_NAME; the
  dialog says how to choose a name instead.
- The container entrypoint prints the name it enrolls with and defaults
  MOSHIRO_NETWORK_SCOPE to host (Docker Desktop sets container in .env).
- The overview and line pages say "measurement channel not established"
  instead of "not connected": the agent can be reachable and pingable while
  the measurement TCP/UDP path is not.
2026-10-08 20:47:38 +08:00
justme 12be44645f feat: install nodes and the backend from a static compose file; remove dead code [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
Agent, Probe and Backend now start from one static compose.yaml plus a local
.env: the panel (or the release assets, for the very first Backend) provides
the file, the panel's command writes the public values into .env, checks the
file's SHA-256 and runs compose up. Updating is compose pull && compose up -d.
The image is chosen by the CI channel (:latest, or MOSHIRO_VERSION in .env),
not by the panel; the previous one-line script stays as a fallback.

- packaging/compose/{agent,probe,backend}/compose.yaml are the single source,
  embedded in the Backend and served at /install/compose/{file}; the root
  compose.yaml moved there. Releases also carry them and list them in the
  signed SHA256SUMS.
- The container entrypoint defaults the node name to the host name.
- Join token creation returns a compose block (.env, url, hash, per-platform
  commands); the admin dialog leads with it.
- Remove code with no references (HotLayer members, LiveHub.SubscriberCount,
  QueryService.ParseParams, ReleaseService.Forget, a few constants) and reduce
  the install-code lookup to the role it is for.
2026-10-08 20:24:23 +08:00
justme e014a19554 docs: reorganize into design, security, api, ops and dev; drop stale designs; refresh the diagrams [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
- Group docs by category with an index (docs/README.md); fix every link,
  code comment and the protocol conformance test that read docs/protocol.md.
- access.md and security.md rewritten to what is implemented: removed the
  unadopted member-proof chain, roles and recovery key that do not exist,
  the old implementation order and the budgets; gaps now carry a status.
- enrollment, deployment, api: the install flow is the static, hash-checked
  script with environment variables; no default admin.
- README and architecture: new topology and update-flow diagrams (operator
  passkey, CI signing, registry feed, auth and trust modules).
2026-10-08 19:37:56 +08:00
justme 636a784043 fix: the panel polls the registry for its own and the nodes' releases [CI]
ci/woodpecker/push/main-amd64-build Pipeline is running
Registering a manifest with the panel is best effort in CI (an unreachable
panel only warns, the pipeline stays green). The Backend update notice then
never appeared, because the host updater only reads the mailbox the panel
writes. A background poller now fetches the signed manifests of the agent, the
probe and the backend itself from the registry every five minutes, verifies the
release signature and registers them, writing the backend one into the host
updater's mailbox.
2026-10-08 19:09:21 +08:00
justme 0f8f3f9a54 feat: trusted release keys setting; drop the duplicate release registration route [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
The panel now lets the deployer decide which release keys it trusts. The
official CI key is built in and always trusted; an own P-256 public key can be
added (off by default, the private key is rejected) so manifests signed with it
are accepted too. Removing it drops the manifests only it signed. Changes are
audited and need update.confirm.

Admin UI: System > Trust > "Trusted release keys" with a collapsible guide on
generating a key pair and signing a manifest with openssl.

Registration now has one path, POST /api/release/offer (no token, signature
verified). The admin POST /releases route, the release.publish capability and
the ci-release preset are removed.
2026-10-08 18:50:16 +08:00
justme 904ea2b0ae refactor: remove the runtime binary binding; every release registration path verifies the CI signature [CI]
ci/woodpecker/push/main-amd64-build Pipeline was successful
The hash a node reports about itself cannot be trusted (a swapped binary can
copy the old hash), so the runtime binding only caught files that were replaced
by something that did not lie. Provenance stays where it can be verified: the
signed release manifest (digest) and the signed SHA256SUMS.

- Remove BinaryBindingService, the node_bindings table (schema v15 drops it),
  BinaryPolicy, the handshake hash/manifest fields and the admin binary column.
- Drop the per-platform "binaries" list from the release manifest, the hash
  scripts and the updater's installed.manifest copy.
- ReleaseService.PublishAsync now verifies the release signature, so the admin
  POST /releases route is no longer an unsigned path; tests sign with a test key.
- Docs: section 16 explains why the runtime check is gone.
2026-10-08 18:30:05 +08:00
justme a4f3d1892a ci: all-platform node binaries in the signed release manifest; signed SHA256SUMS for the archives; web build once before the parallel backend builds; release 0.3.3 [CI SKIP]
ci/woodpecker/tag/windows-release Pipeline was successful
ci/woodpecker/tag/linux-release Pipeline was successful
ci/woodpecker/tag/gitea-release Pipeline was successful
2026-10-08 16:26:29 +08:00
159 changed files with 4936 additions and 2836 deletions

No files matched your search

+2
View File
@@ -40,6 +40,8 @@ steps:
GITEA_BASE: http://127.0.0.1:3100
GITEA_TOKEN:
from_secret: gitea_token
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
commands:
- bash ci/publish-gitea-release.sh
depends_on:
+14 -3
View File
@@ -15,6 +15,17 @@ when:
concurrency: 1
steps:
# 前端先打包好(几十秒),两个架构的 Backend 构建再并行:它们把 web/*/dist 内嵌进程序,只读这份产物,不再各自 npm ci
# (并行的两个 npm ci 会同时删 node_modules,互相搞坏)。步骤加了超时:卡住就几分钟内失败,不是耗着。
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- timeout 600 npm run build --prefix web
depends_on: []
- name: linux-amd64-backend
image: bash
environment:
@@ -25,7 +36,8 @@ steps:
from_secret: zot_secret
commands:
- bash ci/release-linux.sh amd64 backend
depends_on: []
depends_on:
- web-build
- name: linux-arm64-backend
image: bash
@@ -36,10 +48,9 @@ steps:
ZOT_SECRET:
from_secret: zot_secret
commands:
# 两个架构共用一个工作目录:amd64 先构建并生成前端产物,arm64 在它之后(并行会同时 npm ci,互相删 node_modules)
- bash ci/release-linux.sh arm64 backend
depends_on:
- linux-amd64-backend
- web-build
- name: backend-multiarch-index
image: bash
+1 -1
View File
@@ -38,6 +38,6 @@ Follow the user's requested validation scope. Do not start a full tag/release bu
## CI and releases
CI triggers and image tags can change; inspect `.woodpecker/*.yml` and `docs/deployment.md` before changing or describing them. In particular, check the distinction between `[CI]` main builds, `[SKIP TESTS]`, and version-tag releases in the current branch.
CI triggers and image tags can change; inspect `.woodpecker/*.yml` and `docs/ops/deployment.md` before changing or describing them. In particular, check the distinction between `[CI]` main builds, `[SKIP TESTS]`, and version-tag releases in the current branch.
A version tag `vMAJOR.MINOR.PATCH` runs the full cross-platform release pipeline. Development image builds and releases must remain isolated in their respective Zot repository paths.
+40 -34
View File
@@ -12,45 +12,54 @@ MoShiro 是一个自托管的**网络线路质量观测工具**。它在若干 P
**它不做什么**
- **节点是纯采集器**:Agent 和 Probe 没有终端、命令执行、文件读写和自更新能力,也不加载外部代码。更新由节点宿主机上的更新器完成,节点进程只放面板转来的发布清单。
- **不是主机监控**:Agent 只上报 CPU、内存、磁盘、网卡流量和运行时长这类只读的汇总数字,不采集型号、主机名、进程和任何 IP;Probe 不上报主机资源。
- **节点是纯采集器**:Agent 和 Probe 没有终端、命令执行、文件读写和自更新能力,也不加载外部代码。更新由节点外面的服务管理器完成:Podman Quadlet 单元每次启动都先拉最新镜像,管理端点「更新」时节点只是有序退出(没有 Quadlet 的环境用宿主机更新器),节点自己不下载、不替换任何东西。
- **不是主机监控**:Agent 和 Probe 只上报 CPU、内存、磁盘、网卡流量和运行时长这类只读的汇总数字(Probe 可以在本地配置里关掉),不采集型号、主机名、进程和任何 IP。
- **不暴露地址**:公开接口里没有 IP、城市、ASN;国家和运营商只由 Backend 查询(节点自己不访问任何外部服务)。
- 暂不包含告警(见文末待办)。
## 信任关系
```text
管理员 ── 账号密码(Cookie 会话 + CSRF)或 API 令牌(按能力授权)──▶ Backend
管理员 ── 密码或 passkey(Cookie 会话 + CSRF),或 API 令牌(按能力授权)──▶ Backend
访客 ── 匿名只读,站点设为公开时才可用(没有 IP、城市、ASN)──────────────▶ Backend 公开接口
操作员 ── passkey 给「操作员集合」签名 ──▶ Backend 只转发 ──▶ 节点自己验链
(创世指纹和属主在安装时固定在节点上;集合里还登记了 Backend 的服务器密钥,节点据此认 Backend、加密链路)
节点 ── 加入令牌一次性注册,之后用自己的 ECDSA P-256 密钥签名认证 ───────▶ Backend(只存公钥)
Probe ── 用同一把密钥签名建立测量会话 ───────────────────────────────────▶ Agent(只放行面板名单里的 Probe 公钥)
CI ── 发布私钥对「镜像 digest 清单」签名 ──▶ Backend 原样转发 ──▶ 节点宿主机的更新器用本机公钥验签
Probe ── 先敲门,再用自己的密钥签名建立测量会话 ───────────────────────────▶ Agent(只放行面板名单里的 Probe 公钥)
CI ── 发布私钥对「镜像 digest 清单」签名 ──▶ 镜像仓库 / Backend ──▶ 节点宿主机的更新器用本机公钥验签
```
- **Backend 不信任节点上报的内容**:数据按线路隔离存放;节点的身份靠签名,数据库泄露也无法冒充节点。
- **节点不信任 Backend 的下发**:任务超出节点本机的硬上限(目标网段、码率、包速率、监听端口)就被拒绝;节点不执行任何「执行类」指令。
- **更新不依赖面板的诚实**:清单必须由发布私钥签名;节点宿主机的更新器用本机公钥验签、只认本机配置的镜像仓库、按 digest 拉取、拒绝旧的 `seq`。面板被攻破也推不出恶意镜像。
- **节点不信任 Backend 的下发**:任务超出节点本机的硬上限(目标网段、码率、包速率、监听端口)就被拒绝;节点不执行任何「执行类」指令;改信任(操作员集合)要操作员 passkey 签名,Backend 伪造不了。
- **更新不依赖面板的诚实**:清单必须由发布私钥签名(面板登记时验一次,节点宿主机的更新器用本机公钥再验一次)、只认本机配置的镜像仓库、按 digest 拉取、拒绝旧的 `seq`。面板被攻破也推不出恶意镜像。可信的发布密钥由部署的人决定:官方 CI 密钥始终信任,也可以另外信任自己的公钥。
- **没有默认账户**:第一次打开管理端进认领向导,用后端日志里的认领码创建第一个管理员、注册 passkey、建立信任根。每个节点都必须有属主(passkey),由节点自己强制。
- 运行中的二进制不做校验(节点自报的哈希不可信);「资产是我们的」靠安装时的签名清单和签名的 `SHA256SUMS`。详见 [`docs/security/`](docs/security/security.md)。
## 架构
```text
Web(展示端 · 管理端)
▲ REST + SSE
┌──────────────────────────────────────────────────────────┐
│ Backend(面板) │
│ Api · Registry · Planner · Links │
│ Releases · Analysis · LiveHub │
│ Store(段文件 + SQLite) │
└──────────────────────────────────────────────────────────┘
操作员 passkey ──签名「操作员集合」──┐ CI 发布私钥 ──签名「镜像 digest 清单」──▶ 镜像仓库(清单 + 镜像)
▼ ▲
Web(展示端 · 管理端) │ 定时取清单
▲ REST + SSE │
┌──────────────────────────────────────────────────────────────┐
│ Backend(面板) │──────────────────────────────┘
│ Api · Auth(会话 · passkey · API 令牌 · 审计) │
│ Trust(操作员集合 · 服务器身份)· Registry · Planner · Links │
│ Releases(发布清单 · 可信发布密钥) │
│ Ingest · Analysis · LiveHub │
│ Store(段文件 + SQLite) │
└──────────────────────────────────────────────────────────────┘
▲ ▲
│ wss 长连接(节点发起) │ wss 长连接(节点发起)
│ wss 长连接(节点发起,应用层加密) │ wss 长连接(节点发起,应用层加密)
│ 任务、数据、状态、更新通知 │ 身份、Agent 列表、状态、更新通知
┌────────────────────┐ ┌──────────────────────┐
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │
│ 有公网 IP,监听 │ │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
┌────────────────────┐ ┌──────────────────────┐
│ Agent × 10+ │ ◀── 测量会话 ── │ Probe × N │
│ 有公网 IP,监听 │ 先敲门再 TLS │ 家宽等出口,NAT 后 │
└────────────────────┘ └──────────────────────┘
测量会话:TLS 控制通道 + UDP/TCP 数据通道,由 Probe 主动拨出
宿主机更新器(容器节点):用本机公钥验签 → 按 digest 拉镜像 → 重启,失败回滚
```
两种连接:**Probe 和 Agent 都用长连接连 Backend**(身份、任务、状态、更新通知),**测量只走 Probe → Agent**(Probe 主动拨出,Agent 不需要连 Probe,家宽也不需要入站端口)。哪些 Probe 连哪些 Agent 由 Backend 决定:默认自动全连,也可以在管理端手动选择。
@@ -60,7 +69,7 @@ MoShiro 是一个自托管的**网络线路质量观测工具**。它在若干 P
Probe / Agent 积木 ──▶ 原始列块(10 秒一块)──▶ Backend 段文件 ──▶ 指标与服务模型 ──▶ REST / SSE ──▶ Web
```
Backend 用段文件保存原始数据,用 SQLite 保存索引、指标与配置。**完整的架构、数据链路、安全边界和设计决策见 [`docs/architecture.md`](docs/architecture.md)。**
Backend 用段文件保存原始数据,用 SQLite 保存索引、指标与配置。**完整的架构、数据链路、安全边界和设计决策见 [`docs/design/architecture.md`](docs/design/architecture.md)。**
## 协议
@@ -69,7 +78,7 @@ Backend 用段文件保存原始数据,用 SQLite 保存索引、指标与配
- **测量数据面**:Probe 到 Agent 的 TLS 控制通道、UDP 和 TCP 数据通道(默认 `7443/tcp`、`7443/udp`、`7444/tcp`);会话用 `session_id` 和 MAC 认证、不绑定五元组,NAT 映射变化后自动跟随。
- **REST + SSE**:`/api/public`(展示,可匿名)、`/api/admin`(管理);OpenAPI 在 `backend/openapi/v1.json`,前端据此生成类型。
细节见 [`docs/protocol.md`](docs/protocol.md)、[`backend/fleet/README.md`](backend/fleet/README.md)、[`docs/api.md`](docs/api.md)、[`docs/enrollment.md`](docs/enrollment.md)。
细节见 [`docs/design/protocol.md`](docs/design/protocol.md)、[`backend/fleet/README.md`](backend/fleet/README.md)、[`docs/api/api.md`](docs/api/api.md)、[`docs/security/enrollment.md`](docs/security/enrollment.md)。
## 如何构建
@@ -80,7 +89,7 @@ dotnet build MoShiro.slnx -c Release # Backend、Agent、Probe、Fleet
npm ci --prefix web && npm run build --prefix web # 展示端和管理端,产物在 web/display/dist、web/admin/dist
```
本地运行:先在 `backend/` 里 `dotnet run --project src/MoShiro.Backend -- --urls http://localhost:5000`,再 `npm run dev --prefix web`,展示端是 <http://localhost:5173/>,管理端是 <http://localhost:5173/admin/>。
本地运行:先在 `backend/` 里 `dotnet run --project src/MoShiro.Backend -- --urls http://localhost:5000`,再 `npm run dev --prefix web`,展示端是 <http://localhost:5173/>,管理端是 <http://localhost:5173/admin/>。第一次打开管理端会进认领向导,认领码在后端日志里(或用 `MoShiro__ClaimCode` 指定)。
测试和检查:
@@ -90,22 +99,19 @@ npm run lint:css --prefix web # 前端样式(颜色只能写
npm run gen:api --prefix web # 后端改了接口之后重新生成前端类型
```
Backend、Agent、Probe 以 OCI 镜像发布(Linux amd64、arm64),Agent 和 Probe 另有 Linux glibc/musl、macOS、Windows 的二进制包。NativeAOT 的构建细节、CI 触发规则(`[CI]`、`[SKIP TESTS]`、版本 tag)、镜像路径、容器节点的运行和更新见 [`docs/deployment.md`](docs/deployment.md);前端见 [`web/README.md`](web/README.md),后端见 [`backend/README.md`](backend/README.md)。
Backend、Agent、Probe 以 OCI 镜像发布(Linux amd64、arm64),Agent 和 Probe 另有 Linux glibc/musl、macOS、Windows 的二进制包。NativeAOT 的构建细节、CI 触发规则(`[CI]`、`[SKIP TESTS]`、版本 tag)、镜像路径、容器节点的运行和更新见 [`docs/ops/deployment.md`](docs/ops/deployment.md);前端见 [`web/README.md`](web/README.md),后端见 [`backend/README.md`](backend/README.md)。
## 文档
| 文档 | 内容 |
| --- | --- |
| [`docs/architecture.md`](docs/architecture.md) | 架构、数据链路、安全边界、设计决策 |
| [`docs/security.md`](docs/security.md) | 零信任设计稿(部分已实现):威胁模型、入网信任链、监听隐身、extension 授权 |
| [`docs/access.md`](docs/access.md) | 访问控制与入网信任链设计(尚未实现):账户、角色、能力、API key、提权窗口、passkey 到节点的签名链 |
| [`docs/protocol.md`](docs/protocol.md)、[`backend/fleet/README.md`](backend/fleet/README.md) | 线协议、Fleet 节点框架(身份、注册、连接) |
| [`docs/enrollment.md`](docs/enrollment.md) | 注册、身份、Probe 与 Agent 的接入 |
| [`docs/measurement.md`](docs/measurement.md)、[`docs/analysis.md`](docs/analysis.md) | 采集引擎与任务模型、分析层与指标 |
| [`docs/storage.md`](docs/storage.md)、[`docs/query.md`](docs/query.md) | 存储、展示数据查询 |
| [`docs/api.md`](docs/api.md)、[`docs/api-guidelines.md`](docs/api-guidelines.md) | API 规范与约束 |
| [`docs/deployment.md`](docs/deployment.md) | 构建、发布、CI、部署与更新 |
| [`docs/project-layout.md`](docs/project-layout.md)、[`docs/coding-style.md`](docs/coding-style.md) | 仓库结构与依赖规则、代码风格 |
| [`docs/README.md`](docs/README.md) | 文档索引,按类别组织 |
| [`docs/design/`](docs/design/architecture.md) | 设计:架构与决策、线协议、存储与查询、采集与分析 |
| [`docs/security/`](docs/security/security.md) | 安全:零信任与威胁模型、访问控制与入网信任链、注册与身份 |
| [`docs/api/`](docs/api/api.md) | 接口:端点说明、API 规范 |
| [`docs/ops/deployment.md`](docs/ops/deployment.md) | 运维:构建、发布、CI、部署与更新 |
| [`docs/dev/`](docs/dev/project-layout.md) | 开发:仓库结构与依赖规则、代码风格 |
| [`backend/fleet/README.md`](backend/fleet/README.md) | Fleet 节点框架(身份、注册、连接) |
代码目录:`backend/`(Backend、`fleet/` 节点框架、`shared/` 测量引擎和协议、`agent/`、`probe/`、`tests/`)、`web/`(`display` 展示端、`admin` 管理端、`shared`)、`packaging/` 和 `scripts/`(部署与维护)、`ci/` 和 `.woodpecker/`(流水线)。
+4 -4
View File
@@ -12,12 +12,12 @@
## 任务与面板
- 单个 Agent 的任务(目前是线路模板加按 Agent 的覆盖,没有只属于某一个 Agent 的独立任务)
- 原生 webhook 投递;告警(基于 webhook)。实现之前先定约束(只允许 HTTPS、禁止私网和回环、不跟随重定向、限制超时和响应大小、解析后再检查地址),见 [`docs/security.md`](docs/security.md) 第 9.2 节
- 原生 webhook 投递;告警(基于 webhook)。实现之前先定约束(只允许 HTTPS、禁止私网和回环、不跟随重定向、限制超时和响应大小、解析后再检查地址),见 [`docs/security/security.md`](docs/security/security.md) 第 9.2 节
## 部署
- 后端自动下载前端资源:用 Podman、Docker 部署时,默认拉取最新的前端资源(现在前端要自己构建,镜像里没有)。
[security.md](docs/security.md)
[security.md](docs/security/security.md)
## 访客页面
- 人机验证:访客页面要通过验证才能拉数据,前端访问时静默触发。候选 tiago2/cap 和 Cloudflare Turnstile。
@@ -34,11 +34,11 @@
- 流程:下载 → 校验 → 解包到 `versions/<v>/` → 原子切换 `current`(Linux 符号链接,Windows 目录联结)→ 重启服务 → 等健康信标确认 → 失败切回旧版本并把该版本标记为坏版本。
- 权限分离:更新进程只能写安装目录和重启被管理的服务,读不到节点身份私钥。
- OpenWrt:`keep_versions = 1`,下载前检查闪存剩余空间。
- 容器节点已经实现的那一套(清单签名、`seq` 防重放、按 digest 拉取、回滚)见 [`docs/deployment.md`](docs/deployment.md)。
- 容器节点已经实现的那一套(清单签名、`seq` 防重放、按 digest 拉取、回滚)见 [`docs/ops/deployment.md`](docs/ops/deployment.md)。
## 安全
- **零信任**:设计稿在 [`docs/security.md`](docs/security.md),按其中第 12 节的顺序实现(阶段零「协议方向声明表」已完成):Probe 自己生成密钥、Agent 预握手敲门、操作员凭证和名册、测量数据的签名和归属校验、入库配额和管理端 CSP、Backend 应用层身份、extension 框架(敏感能力都放进可选安装的独立 extension,核心保持 NativeAOT);可选的 UDP 单包授权、UDP 会话迁移的路径验证。
- **零信任**:设计和各缺口的状态在 [`docs/security/security.md`](docs/security/security.md)。已完成:协议方向声明表、Probe 自己生成密钥、Agent 预握手敲门、操作员集合与创世固定、服务器身份和安全链路、属主固定。还没做:测量数据的产生者签名和归属校验、入库配额、Probe 只拨公网地址、管理端 CSP 之外的输入校验、extension 框架(敏感能力都放进可选安装的独立 extension,核心保持 NativeAOT)、吊销表;可选的 UDP 单包授权、UDP 会话迁移的路径验证。
- **extension**(Web SSH、TTY、更新遥测,敏感 operator):设计时必须保住「节点是纯采集器」这条边界,见 [`README.md`](README.md) 的项目边界。
- 对 UDP 包、控制帧、RawChunk、JSON-RPC 做模糊测试;伪造签名和重放的专项测试。
+1 -1
View File
@@ -13,7 +13,7 @@
<UseArtifactsOutput>true</UseArtifactsOutput>
<ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath>
<Deterministic>true</Deterministic>
<Version>0.3.2</Version>
<Version>0.4.0</Version>
<NuGetAudit>false</NuGetAudit>
</PropertyGroup>
<ItemGroup>
+3 -3
View File
@@ -36,7 +36,7 @@ dotnet test --solution MoShiro.slnx
dotnet publish src/MoShiro.Backend -c Release -r <RID> -p:PublishAot=true
```
例如 `<RID>` 可为 `win-x64`、`linux-x64` 或 `linux-arm64`。NativeAOT 发布需要对应平台的 C/C++ 工具链。仓库根目录的 `scripts/publish.sh backend <RID>` 可构建单个平台的 Backend 可执行文件;容器发布由 Woodpecker tag 流水线完成,详见 [`../docs/deployment.md`](../docs/deployment.md)。
例如 `<RID>` 可为 `win-x64`、`linux-x64` 或 `linux-arm64`。NativeAOT 发布需要对应平台的 C/C++ 工具链。仓库根目录的 `scripts/publish.sh backend <RID>` 可构建单个平台的 Backend 可执行文件;容器发布由 Woodpecker tag 流水线完成,详见 [`../docs/ops/deployment.md`](../docs/ops/deployment.md)。
正式发布通过 Woodpecker tag 流水线生成 Linux/amd64 与 Linux/arm64 OCI 镜像,容器内监听 `8080`,数据写入挂载的 `/data` 卷。Agent 与 Probe 的多平台二进制包单独发布,不包含在 Backend 镜像中。
@@ -49,6 +49,6 @@ cd backend
dotnet run --project src/MoShiro.Backend -- --urls http://localhost:5000
```
没有默认管理员:第一次打开面板时按页面引导自己创建管理员、注册 passkey、建立信任根,页面要的认领码写在后端日志里(或用 `MoShiro__ClaimCode` 指定)。`MoShiro__InitialAdminUser` / `MoShiro__InitialAdminPassword` 只给自动化用。数据目录、公开地址、Cookie 和 CORS 等设置通过 `MoShiro` 配置节传入;完整配置说明见 [`../docs/deployment.md`](../docs/deployment.md)。站点默认需要登录,可在管理端修改公开设置。
没有默认管理员:第一次打开面板时按页面引导自己创建管理员、注册 passkey、建立信任根,页面要的认领码写在后端日志里(或用 `MoShiro__ClaimCode` 指定)。`MoShiro__InitialAdminUser` / `MoShiro__InitialAdminPassword` 只给自动化用。数据目录、公开地址、Cookie 和 CORS 等设置通过 `MoShiro` 配置节传入;完整配置说明见 [`../docs/ops/deployment.md`](../docs/ops/deployment.md)。站点默认需要登录,可在管理端修改公开设置。
OpenAPI 文档由构建生成到 `openapi/v1.json`。API 设计和端点说明见 [`../docs/api.md`](../docs/api.md);项目总览见仓库根目录的 [`README.md`](../README.md)。
OpenAPI 文档由构建生成到 `openapi/v1.json`。API 设计和端点说明见 [`../docs/api/api.md`](../docs/api/api.md);项目总览见仓库根目录的 [`README.md`](../README.md)。
+4 -4
View File
@@ -29,7 +29,7 @@ dotnet publish agent/src/MoShiro.Agent -c Release -r <RID> -p:PublishAot=true
dotnet publish fleet/tools/fleet-enroll -c Release -r <RID> -p:PublishAot=true
```
也可在仓库根目录运行 `bash scripts/publish.sh agent <RID>` 和 `bash scripts/publish.sh fleet-enroll <RID>`。部署包和平台说明见 [`../../docs/deployment.md`](../../docs/deployment.md)。
也可在仓库根目录运行 `bash scripts/publish.sh agent <RID>` 和 `bash scripts/publish.sh fleet-enroll <RID>`。部署包和平台说明见 [`../../docs/ops/deployment.md`](../../docs/ops/deployment.md)。
## 注册与运行
@@ -47,7 +47,7 @@ fleet-enroll --server https://<backend> --join <token> --name <agent-name> --ide
moshiro-agent --config /etc/moshiro/agent.json
```
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/deployment.md`](../../docs/deployment.md)。也可使用 OCI 镜像 `registry.oeo.one/moshiro/moshiro-agent:<version>`,自动选择 Linux amd64 或 arm64 镜像。镜像同时包含 `fleet-enroll`;可通过 `--entrypoint /usr/local/bin/fleet-enroll` 执行首次注册。容器默认读取命名 volume 中的 `/etc/moshiro/agent.json` 与身份文件;建议使用宿主网络,Linux ICMP 优先使用普通用户 ping socket,内核 ping_group_range 需允许映射后的 GID。镜像以 UID/GID 1654 运行,新命名 volume 由引擎按镜像目录初始化权限。
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/ops/deployment.md`](../../docs/ops/deployment.md)。也可使用 OCI 镜像 `registry.oeo.one/moshiro/moshiro-agent:<version>`,自动选择 Linux amd64 或 arm64 镜像。镜像同时包含 `fleet-enroll`;可通过 `--entrypoint /usr/local/bin/fleet-enroll` 执行首次注册。容器默认读取命名 volume 中的 `/etc/moshiro/agent.json` 与身份文件;建议使用宿主网络,Linux ICMP 优先使用普通用户 ping socket,内核 ping_group_range 需允许映射后的 GID。镜像以 UID/GID 1654 运行,新命名 volume 由引擎按镜像目录初始化权限。
不传 `--config` 时默认读取 `/etc/moshiro/agent.json`。Agent 默认监听控制端口 `7443/tcp`,数据端口 `7443/udp` 和 `7444/tcp`;防火墙和面板中的节点端口配置需与本地配置一致。`reporting.host_stats` 可关闭主机状态上报;启用时只上报资源使用等汇总数据,不包含主机名、进程或 IP。
@@ -58,6 +58,6 @@ IPv6 线路需要 Agent 实际绑定 IPv6。新配置的默认 `listen.bind` 为
- 每个 Probe 使用独立的 TLS 会话和数据密钥;Probe 断开时对应的会话资源会释放。
- 数据面会校验会话认证、重放窗口和速率限制;无效数据包会静默丢弃。
- 测量引擎和协议由 `backend/shared/` 提供;节点身份、注册和连接由 `backend/fleet/` 提供。
- 详细安全边界与拓扑见 [`../../docs/architecture.md`](../../docs/architecture.md),协议说明见 [`../../docs/protocol.md`](../../docs/protocol.md),部署见 [`../../docs/deployment.md`](../../docs/deployment.md)。
- 详细安全边界与拓扑见 [`../../docs/design/architecture.md`](../../docs/design/architecture.md),协议说明见 [`../../docs/design/protocol.md`](../../docs/design/protocol.md),部署见 [`../../docs/ops/deployment.md`](../../docs/ops/deployment.md)。
Docker 与 rootless/rootful Podman 共用 `packaging/compose/agent/compose.yaml`,配置、身份和缓存放在命名 volume 中,无需宿主配置目录、keep-id 或手动 chown。首次导入和自动更新见部署文档。
Podman 系用 Quadlet(`packaging/quadlet/moshiro-agent.container`,默认自动更新),Docker 用 `packaging/compose/agent/compose.yaml`(手动更新);配置、身份和缓存放在命名 volume 中,无需宿主配置目录、keep-id 或手动 chown。首次导入和自动更新见部署文档。
+5 -7
View File
@@ -31,7 +31,7 @@ public sealed class AgentApp : IDisposable
private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly AgentAdmission _admission;
private readonly Outbox _outbox = new(64 << 20);
private readonly UpdateBridge _update;
private readonly RestartUpdater? _update;
private readonly BackendUplink _uplink;
private readonly UdpMux _udp;
private readonly TcpAcceptor _tcp;
@@ -46,8 +46,7 @@ public sealed class AgentApp : IDisposable
{
_ownedRestart = restart is null ? new RestartController() : null;
_restart = restart ?? _ownedRestart!;
_update = new UpdateBridge(dataDirectory, new ConfigStore(dataDirectory).Write, log);
NodeBinary.Prime(); // 正在运行的二进制的哈希:先在后台算好,握手时用
_update = RestartUpdater.FromEnvironment("agent", _restart.RequestNow, time, log);
_config = config;
_identity = identity;
_time = time;
@@ -236,7 +235,7 @@ public sealed class AgentApp : IDisposable
{
router.OnNotify(MoShiroMethods.UpdateOffer, MoShiroJsonContext.Default.UpdateOfferParams, (_, offer, _) =>
{
_update.WriteOffer(offer);
_update?.Offer(offer);
return ValueTask.CompletedTask;
});
}
@@ -258,7 +257,7 @@ public sealed class AgentApp : IDisposable
{
router.OnNotify(MoShiroMethods.UpdateApply, MoShiroJsonContext.Default.UpdateOfferParams, (_, _, _) =>
{
_update.RequestApply();
_update?.RequestApply();
return ValueTask.CompletedTask;
});
}
@@ -277,7 +276,6 @@ public sealed class AgentApp : IDisposable
Server = new Uri(_config.Fleet.Server),
Identity = _identity.Identity,
ServerVerifier = _trust,
Attestation = _update.Attest,
NodeId = _identity.NodeId,
NodeVersion = Version,
Capabilities = [MoShiroCapabilities.Base, MoShiroCapabilities.KindsPrefix + string.Join(',', BlockCatalog.AgentKinds)],
@@ -343,7 +341,7 @@ public sealed class AgentApp : IDisposable
DroppedInvalid = _counters.DroppedInvalid,
DroppedRateLimited = _counters.DroppedRateLimited,
QueueBytes = _outbox.PendingBytes,
Update = _update.ReadStatus(),
Update = _update?.Status(),
};
_lastIn = packetsIn;
_lastOut = packetsOut;
+1 -10
View File
@@ -1,4 +1,5 @@
using System.Text.Json.Serialization;
using MoShiro.Measurement;
namespace MoShiro.Agent;
@@ -28,16 +29,6 @@ public sealed class AgentConfig
public string? RestartAt { get; set; }
}
public sealed class ReportingSection
{
/// <summary>上报主机资源占用(CPU、内存、磁盘、流量、运行时长,只有汇总数字,没有型号和 IP)。设为 false 就一个字也不上报。</summary>
[JsonPropertyName("host_stats")]
public bool HostStats { get; set; } = true;
[JsonPropertyName("host_interval_s")]
public int HostIntervalSeconds { get; set; } = 2;
}
public sealed class FleetSection
{
[JsonPropertyName("server")]
@@ -213,8 +213,6 @@ public sealed class PeerHub
private static ConnectedProbe Describe(ProbeSession p) => new() { Id = new ProbeId(p.ProbeId), Version = p.Version, Remote = p.RemoteAddress, SinceUtcMs = p.ConnectedAtUtcMs, IpVersion = p.IpVersion };
public IReadOnlyList<string> ConnectedProbeIds() => [.. _sessions.All.Select(p => p.ProbeId)];
/// <summary>向一个 Probe 下发它的线路任务集(Probe 连上时由 <see cref="OnSessionAsync"/> 先发一份)。</summary>
public async ValueTask PushToAsync(ProbeSession probe, LineTaskSet set) => await PushAsync(probe, set, CancellationToken.None);
+2 -2
View File
@@ -23,7 +23,7 @@
2. **安全默认值**:Fleet 永远不提供 exec、shell、文件读写、远程下发代码这类方法;节点只响应本地注册过的方法(本地策略 `allow_methods` 还能收窄);调用对方不支持的方法在本地直接返回 `fleet_unsupported`。
3. **权限分离**:注册(`fleet-enroll`,一次性)和常驻业务进程是两个进程;常驻进程不含注册代码,也不使用 `HttpClient`。
4. **NativeAOT**:节点侧只依赖 BCL,泛型扩展点通过产品传入的 `JsonTypeInfo<T>` 序列化,不使用反射。
5. **更新不在 Fleet 里**:节点进程没有自更新能力。容器节点的签名更新由宿主机上的更新器完成,见 [`docs/deployment.md`](../../docs/deployment.md);纯二进制节点的更新是待办,见 [`TODO.md`](../../TODO.md)。
5. **更新不在 Fleet 里**:节点进程没有自更新能力。容器节点的签名更新由宿主机上的更新器完成,见 [`docs/ops/deployment.md`](../../docs/ops/deployment.md);纯二进制节点的更新是待办,见 [`TODO.md`](../../TODO.md)。
## 身份
@@ -56,7 +56,7 @@
- JSON-RPC 2.0;方法名 `领域.动作`,`fleet.*` 保留给框架,产品用自己的前缀(MoShiro 是 `moshiro.*`)。
- 二进制帧 `u8 channel | 正文`:`0`–`15` 框架保留,产品从 `16` 开始。
- 错误:`error.data.code` 为 snake_case,框架的以 `fleet_` 开头,产品可以扩展。
- **方法必须声明**:每个方法在声明表(`MethodTable`)里写明发起方、请求还是通知、哪些角色可以参与;不存在双向方法,同一个方法名在一个平面里只有一个发起方。`RpcRouter` 在注册、分发和发送三处核对,违规回 `fleet_protocol_violation`,超过上限断开连接。详见 [`docs/security.md`](../../docs/security.md) 第 8 节。
- **方法必须声明**:每个方法在声明表(`MethodTable`)里写明发起方、请求还是通知、哪些角色可以参与;不存在双向方法,同一个方法名在一个平面里只有一个发起方。`RpcRouter` 在注册、分发和发送三处核对,违规回 `fleet_protocol_violation`,超过上限断开连接。详见 [`docs/security/security.md`](../../docs/security/security.md) 第 8 节。
- 与传输无关:同一套路由跑在 wss 上,也跑在 TLS 帧(`u32 长度 | u8 类型 | 正文`)上。
| 保留方法 | 方向 | 说明 |
@@ -1,43 +0,0 @@
using System.Security.Cryptography;
namespace Fleet.Core;
/// <summary>
/// 节点握手时自述的「我跑的是哪个二进制」:正在运行的可执行文件的 SHA-256,以及(容器节点)宿主机更新器验过的 CI 签名清单和签名。
/// 服务端在节点第一次握手成功的那一刻把哈希绑定到这个节点,之后哈希对不上就不让它继续运行。
/// </summary>
public sealed record BinaryAttestation(string Sha256, string? Manifest, string? Signature);
/// <summary>
/// 正在运行的二进制的哈希。Linux 上读 <c>/proc/self/exe</c>:它指向进程启动时映射的那个映像,磁盘上的文件之后被换掉也不影响,
/// 所以「运行期间被替换、重启后换了别的」的情况下一次握手就会对不上。算一次,之后缓存。
/// </summary>
public static class NodeBinary
{
private static readonly Lazy<string?> s_hash = new(Compute);
/// <summary>小写十六进制;读不到可执行文件时为 null(这样的节点不做绑定)。</summary>
public static string? Sha256Hex => s_hash.Value;
/// <summary>在后台先算好,免得第一次握手等它。</summary>
public static void Prime() => _ = Task.Run(() => s_hash.Value);
private static string? Compute()
{
try
{
string? path = OperatingSystem.IsLinux() && File.Exists("/proc/self/exe") ? "/proc/self/exe" : Environment.ProcessPath;
if (path is null)
{
return null;
}
using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete, 1 << 16);
return Convert.ToHexStringLower(SHA256.HashData(stream));
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
return null;
}
}
}
@@ -27,6 +27,4 @@ public static class FleetErrors
public static readonly ErrorCode JoinTokenExpired = new("fleet_join_token_expired");
public static readonly ErrorCode ProofInvalid = new("fleet_proof_invalid");
public static readonly ErrorCode FrameTooLarge = new("fleet_frame_too_large");
/// <summary>节点现在跑的二进制和它第一次握手时绑定的不一致,也不在 CI 签名的更新清单里。</summary>
public static readonly ErrorCode BinaryMismatch = new("fleet_binary_mismatch");
}
-12
View File
@@ -51,16 +51,4 @@ public static class VarInt
return false;
}
public static bool TryReadUInt32LittleEndian(ReadOnlySpan<byte> source, out uint value)
{
if (source.Length < 4)
{
value = 0;
return false;
}
value = BinaryPrimitives.ReadUInt32LittleEndian(source);
return true;
}
}
+8 -1
View File
@@ -181,7 +181,7 @@ public static class ServiceShell
// 先取令牌、再取设置:中间被改动的话令牌会被取消,下一轮重新排
CancellationToken changed = restart.ChangeToken;
RestartSchedule? schedule = restart.Current;
using CancellationTokenSource wait = CancellationTokenSource.CreateLinkedTokenSource(cts.Token, changed);
using CancellationTokenSource wait = CancellationTokenSource.CreateLinkedTokenSource(cts.Token, changed, restart.NowToken);
try
{
if (schedule is null)
@@ -196,6 +196,13 @@ public static class ServiceShell
}
catch (OperationCanceledException)
{
if (restart.NowReason is { } reason)
{
log.Info("stopping", ("result", reason));
await cts.CancelAsync();
return;
}
continue; // 进程在退出,或者设置被改了:循环条件和下一轮会处理
}
@@ -8,7 +8,9 @@ public sealed class RestartController : IDisposable
private readonly RestartSchedule? _local;
private readonly object _gate = new();
private RestartSchedule? _remote;
private readonly CancellationTokenSource _now = new();
private CancellationTokenSource _changed = new();
private string? _nowReason;
public RestartController(RestartSchedule? local = null)
{
@@ -41,6 +43,32 @@ public sealed class RestartController : IDisposable
}
}
/// <summary>应用要求现在就有序退出(比如管理员请求更新);退出的原因,没有就是 null。</summary>
public string? NowReason
{
get
{
lock (_gate)
{
return _nowReason;
}
}
}
/// <summary>应用要求现在就退出的令牌。</summary>
public CancellationToken NowToken => _now.Token;
/// <summary>要求进程现在就有序退出,由服务管理器拉起;不管有没有设置定时重启。</summary>
public void RequestNow(string reason)
{
lock (_gate)
{
_nowReason ??= reason;
}
_now.Cancel();
}
/// <summary>面板下发的设置;为空表示面板关闭了定时重启。返回 false 表示文本不合法(节点忽略它)。</summary>
public bool SetRemote(string? text)
{
@@ -68,6 +96,7 @@ public sealed class RestartController : IDisposable
lock (_gate)
{
_changed.Dispose();
_now.Dispose();
}
}
}
@@ -25,7 +25,7 @@ public sealed partial class TrustFileContext : JsonSerializerContext;
/// <summary>
/// 节点的信任根:安装命令里带来的创世指纹和属主,在注册时固定到本地,之后 Backend 改不了。
/// 操作员集合的新版本随期望状态下发,节点只接受从固定的创世出发、签名完整、版本更新的链。
/// 属主只是被固定和保存:节点现在没有 extension,等做 extension 时由节点本地强制(见 docs/access.md 第 8 节)。
/// 属主只是被固定和保存:节点现在没有 extension,等做 extension 时由节点本地强制(见 docs/security/access.md 第 9 节)。
/// </summary>
public sealed class TrustAnchor : IServerVerifier
{
@@ -20,7 +20,7 @@ public sealed record OperatorKey(string CredentialId, byte[] PublicKeySpki);
public sealed record OperatorSet(int Version, string Previous, string RpId, IReadOnlyList<string> Origins, IReadOnlyList<OperatorKey> Keys, IReadOnlyList<byte[]>? Servers = null);
/// <summary>
/// 信任根:操作员(passkey)→ 操作员集合 → 节点本地固定的属主。设计见 docs/access.md 第 7 节。
/// 信任根:操作员(passkey)→ 操作员集合 → 节点本地固定的属主。设计见 docs/security/access.md 第 8 节。
/// 安装命令里带着创世指纹和属主(passkey 的凭证 id):节点把它们固定在本地,之后只认从创世出发、由操作员 passkey 签过名的集合版本,
/// 属主必须是集合里的凭证。Backend 能转发集合,但签不出新的版本,也改不了节点已经固定的东西。
/// 全是纯函数,只用 BCL,节点和 Backend 共用。语句是 <c>key=value</c> 的文本,第一行是带版本的标题;签名覆盖整段文本。
@@ -7,7 +7,7 @@ namespace Fleet.Identity;
/// <summary>
/// 验证一次 WebAuthn 断言(<c>navigator.credentials.get</c> 的结果)。只支持 ES256(P-256 的 ECDSA),纯函数,只用 BCL:
/// 面板用它验证 passkey 登录,节点以后用它验证 passkey 签出的委托(docs/access.md 第 7 节)。
/// 面板用它验证 passkey 登录,节点验证操作员集合的签名也用它(docs/security/access.md 第 8 节)。
/// 不解析 CBOR——注册时的 attestation 解析只在 Backend 里。
/// </summary>
public static class WebAuthnAssertion
-15
View File
@@ -72,9 +72,6 @@ public sealed class FleetLinkOptions
/// 服务器必须在挑战里证明自己,之后这条链路用应用层加密;否则按旧的方式(没有固定信任根的节点)。
/// </summary>
public IServerVerifier? ServerVerifier { get; init; }
/// <summary>握手时自述正在运行的二进制(哈希和 CI 签名的清单);没有就不上报(旧行为)。</summary>
public Func<BinaryAttestation?>? Attestation { get; init; }
}
/// <summary>
@@ -226,16 +223,6 @@ public sealed class FleetLink
}
/// <summary>挑战 → 签名 → 认证结果。失败返回原因;服务端禁用该节点时不再重试。</summary>
private void Attest(AuthParams auth)
{
if (_options.Attestation?.Invoke() is { } attestation)
{
auth.Binary = attestation.Sha256;
auth.Release = attestation.Manifest;
auth.ReleaseSignature = attestation.Signature;
}
}
private async Task<string?> AuthenticateAsync(SecureTransport transport, CancellationToken ct)
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(15), _time);
@@ -270,7 +257,6 @@ public sealed class FleetLink
Signature = Convert.ToBase64String(_options.Identity.Sign(SignaturePurpose.AuthSecure, _options.NodeId.Value, nonce.Nonce, serverEphemeralText, nodeEphemeralText)),
NodeEphemeral = nodeEphemeralText,
};
Attest(authParams);
RpcMessage secureRequest = RpcMessage.Request(1, FleetMethods.Auth, authParams, FleetRpcContext.Default.AuthParams);
await transport.SendAsync(new RpcFrame(FrameKind.Text, JsonSerializer.SerializeToUtf8Bytes(secureRequest, FleetRpcContext.Default.RpcMessage)), linked.Token);
transport.Enable(LinkCipher.ForNode(ephemeral, serverEphemeral, nonce.Nonce));
@@ -278,7 +264,6 @@ public sealed class FleetLink
else
{
authParams = new AuthParams { NodeId = _options.NodeId.Value, Signature = Convert.ToBase64String(_options.Identity.Sign(SignaturePurpose.Auth, _options.NodeId.Value, nonce.Nonce)) };
Attest(authParams);
RpcMessage request = RpcMessage.Request(1, FleetMethods.Auth, authParams, FleetRpcContext.Default.AuthParams);
await transport.SendAsync(new RpcFrame(FrameKind.Text, JsonSerializer.SerializeToUtf8Bytes(request, FleetRpcContext.Default.RpcMessage)), linked.Token);
}
@@ -40,12 +40,6 @@ public static class FleetMethodTable
]);
}
/// <summary>二进制通道:0–15 保留给框架,产品从 16 开始。</summary>
public static class FleetChannels
{
public const byte FirstProductChannel = 16;
}
public sealed class ChallengeParams
{
[JsonPropertyName("nonce")]
@@ -75,17 +69,6 @@ public sealed class AuthParams
/// <summary>节点的临时 ECDH 公钥(SPKI,base64):有它就是安全链路,签名覆盖双方的临时公钥,认证结果起所有帧都是加密的。</summary>
[JsonPropertyName("node_eph")]
public string? NodeEphemeral { get; set; }
/// <summary>正在运行的二进制的 SHA-256(小写十六进制):服务端在第一次握手时绑定,之后核对。旧节点没有这个字段。</summary>
[JsonPropertyName("binary")]
public string? Binary { get; set; }
/// <summary>安装这个二进制时宿主机更新器验过的 CI 签名清单(原文)和签名(base64):服务端用 CI 公钥验过,才相信它列出的官方哈希。</summary>
[JsonPropertyName("release")]
public string? Release { get; set; }
[JsonPropertyName("release_sig")]
public string? ReleaseSignature { get; set; }
}
public sealed class AuthResult
@@ -19,21 +19,6 @@ public interface IServerIdentity
}
/// <summary>操作员集合的链(passkey 签的,创世在前)。没有初始化信任根时是空的。节点注册时用它确认「这个面板确实是我的操作员的」。</summary>
/// <summary>节点握手时自述的二进制(旧节点没有)。</summary>
public sealed record BinaryEvidence(string? Sha256, string? Manifest, string? Signature);
/// <summary>是否让这个节点继续:不放行时 <c>Reason</c> 会原样告诉节点(这是它自己的信息,不是秘密)。</summary>
public sealed record BinaryDecision(bool Accept, string Reason = "");
/// <summary>
/// 服务端对节点二进制的把关:节点第一次握手成功的那一刻绑定它的二进制哈希,之后哈希对不上、又不在 CI 签名的更新清单里,就不让它继续运行。
/// 没有注册实现时不做任何检查。
/// </summary>
public interface IBinaryAttestor
{
ValueTask<BinaryDecision> CheckAsync(NodeRecord node, BinaryEvidence evidence, CancellationToken ct);
}
public interface IOperatorTrustSource
{
ValueTask<IReadOnlyList<SignedStatementDto>> ChainAsync(CancellationToken ct);
@@ -24,11 +24,9 @@ public sealed class FleetLinkHost : IFleetConnections, IStateNotifier
IEnumerable<IInbox> inboxes,
IEnumerable<INodeEventSink> sinks,
FleetRouter router,
IServerIdentity? serverIdentity = null,
IBinaryAttestor? binaryAttestor = null)
IServerIdentity? serverIdentity = null)
{
ServerIdentity = serverIdentity;
BinaryAttestor = binaryAttestor;
Store = store;
Options = options;
Time = time;
@@ -45,9 +43,6 @@ public sealed class FleetLinkHost : IFleetConnections, IStateNotifier
/// <summary>服务器的身份密钥;没有时节点连接用旧的明文认证。</summary>
internal IServerIdentity? ServerIdentity { get; }
/// <summary>节点二进制的把关;没有时不检查。</summary>
internal IBinaryAttestor? BinaryAttestor { get; }
internal FleetServerOptions Options { get; }
internal TimeProvider Time { get; }
@@ -110,8 +110,7 @@ public static class FleetServerExtensions
provider.GetServices<IInbox>(),
provider.GetServices<INodeEventSink>(),
provider.GetRequiredService<FleetRouter>(),
provider.GetService<IServerIdentity>(),
provider.GetService<IBinaryAttestor>()));
provider.GetService<IServerIdentity>()));
services.AddSingleton<IFleetConnections>(provider => provider.GetRequiredService<FleetLinkHost>());
services.AddSingleton<IStateNotifier>(provider => provider.GetRequiredService<FleetLinkHost>());
return builder;
@@ -88,18 +88,6 @@ internal sealed partial class NodeConnection
return false;
}
// 二进制:第一次握手成功的这一刻绑定,之后哈希对不上又不在 CI 签名的更新清单里,就不让它继续运行
if (_host.BinaryAttestor is { } attestor)
{
BinaryDecision decision = await attestor.CheckAsync(node, new BinaryEvidence(auth!.Binary, auth.Release, auth.ReleaseSignature), ct);
if (!decision.Accept)
{
await SendAsync(RpcMessage.Failure(message.Id, new FleetError(FleetErrors.BinaryMismatch, decision.Reason)), ct);
await RejectAsync(null, "binary mismatch: " + decision.Reason, ct);
return false;
}
}
_session = new NodeSessionInfo(node.Id, _host.Time.GetUtcNow().ToUnixTimeMilliseconds(), node.NodeVersion, [], null);
await SendAsync(
RpcMessage.Success(message.Id.Value, new AuthResult { Status = StatusName(node.Status) }, FleetRpcContext.Default.AuthResult),
+394 -223
View File
@@ -815,7 +815,7 @@
"tags": [
"public"
],
"summary": "Latest host resource usage of every visible agent (no hardware model, no addresses)",
"summary": "Latest host resource usage of every visible agent and probe (no hardware model, no addresses)",
"operationId": "hosts_list",
"responses": {
"200": {
@@ -841,16 +841,16 @@
}
}
},
"/api/public/agents/{agent_id}/host": {
"/api/public/hosts/{node_id}": {
"get": {
"tags": [
"public"
],
"summary": "Host resource usage history of one agent (columnar, averaged per step)",
"summary": "Host resource usage history of one agent or probe (columnar, averaged per step)",
"operationId": "hosts_history",
"parameters": [
{
"name": "agent_id",
"name": "node_id",
"in": "path",
"required": true,
"schema": {
@@ -1369,118 +1369,6 @@
}
}
},
"/api/admin/releases": {
"post": {
"tags": [
"admin-build"
],
"summary": "Register a CI-signed release manifest; online nodes get it right away (their updaters verify the signature). Admin or release tokens",
"operationId": "releases_publish",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PublishReleaseRequest"
}
}
},
"required": true
},
"responses": {
"201": {
"description": "Created",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ReleaseDto"
}
}
}
},
"400": {
"description": "Bad Request",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"409": {
"description": "Conflict",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "release.publish"
},
"get": {
"tags": [
"admin-build"
],
"summary": "The latest signed release manifest of each component",
"operationId": "releases_list",
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PageDtoOfReleaseDto"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "update.read"
}
},
"/api/admin/auth/logout": {
"post": {
"tags": [
@@ -4412,6 +4300,48 @@
"x-moshiro-capability": "update.read"
}
},
"/api/admin/releases": {
"get": {
"tags": [
"admin-build"
],
"summary": "The latest signed release manifest of each component",
"operationId": "releases_list",
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PageDtoOfReleaseDto"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "update.read"
}
},
"/api/admin/nodes/{id}/update": {
"post": {
"tags": [
@@ -4472,56 +4402,6 @@
"x-moshiro-capability": "update.confirm"
}
},
"/api/admin/nodes/{id}/binary": {
"delete": {
"tags": [
"admin-build"
],
"summary": "Accept the node's current binary: clear its binding so the next handshake binds it again",
"operationId": "nodes_binary_clear",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"204": {
"description": "No Content",
"content": {
"application/json": {
"schema": { }
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "fleet.write"
}
},
"/api/admin/backend/update": {
"get": {
"tags": [
@@ -4667,6 +4547,148 @@
"x-moshiro-capability": "update.read"
}
},
"/api/admin/release-key": {
"get": {
"tags": [
"admin-release-key"
],
"summary": "The release keys this panel trusts: the built-in official key and the optional own key",
"operationId": "release_key_get",
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ReleaseKeysDto"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "update.read"
},
"put": {
"tags": [
"admin-release-key"
],
"summary": "Trust your own release public key (P-256 PEM; never the private key) in addition to the official one",
"operationId": "release_key_set",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SetReleaseKeyRequest"
}
}
},
"required": true
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ReleaseKeysDto"
}
}
}
},
"400": {
"description": "Bad Request",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "update.confirm"
},
"delete": {
"tags": [
"admin-release-key"
],
"summary": "Stop trusting your own release key; manifests only it signed are dropped",
"operationId": "release_key_clear",
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ReleaseKeysDto"
}
}
}
},
"401": {
"description": "Unauthorized",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
},
"403": {
"description": "Forbidden",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
},
"x-moshiro-capability": "update.confirm"
}
},
"/api/admin/links": {
"get": {
"tags": [
@@ -4944,6 +4966,74 @@
}
}
},
"/install/compose/{file}": {
"get": {
"tags": [
"ops"
],
"summary": "The static compose file of agent, probe or backend (agent.yaml, probe.yaml, backend.yaml; same bytes for everyone)",
"operationId": "install_compose",
"parameters": [
{
"name": "file",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "OK"
},
"404": {
"description": "Not Found",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
}
}
},
"/install/quadlet/{file}": {
"get": {
"tags": [
"ops"
],
"summary": "The static Podman Quadlet unit of agent or probe (agent.container, probe.container; same bytes for everyone)",
"operationId": "install_quadlet",
"parameters": [
{
"name": "file",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "OK"
},
"404": {
"description": "Not Found",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/ProblemDto"
}
}
}
}
}
}
},
"/install.ps1": {
"get": {
"tags": [
@@ -5774,6 +5864,37 @@
}
}
},
"ComposeInstallDto": {
"required": [
"env",
"url",
"sha256",
"linux",
"macos",
"windows"
],
"type": "object",
"properties": {
"env": {
"type": "string"
},
"url": {
"type": "string"
},
"sha256": {
"type": "string"
},
"linux": {
"type": "string"
},
"macos": {
"type": "string"
},
"windows": {
"type": "string"
}
}
},
"ConfirmBackendUpdateRequest": {
"required": [
"seq"
@@ -6007,7 +6128,9 @@
"required": [
"token",
"plaintext",
"install_commands"
"install_commands",
"compose",
"quadlet"
],
"type": "object",
"properties": {
@@ -6019,6 +6142,12 @@
},
"install_commands": {
"$ref": "#/components/schemas/InstallCommandsDto"
},
"compose": {
"$ref": "#/components/schemas/ComposeInstallDto"
},
"quadlet": {
"$ref": "#/components/schemas/QuadletInstallDto"
}
}
},
@@ -6493,7 +6622,8 @@
},
"HostItemDto": {
"required": [
"agent_id",
"id",
"kind",
"name",
"online",
"stale",
@@ -6502,7 +6632,10 @@
],
"type": "object",
"properties": {
"agent_id": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"name": {
@@ -6531,7 +6664,7 @@
},
"HostSeriesDto": {
"required": [
"agent_id",
"id",
"from",
"to",
"step",
@@ -6540,7 +6673,7 @@
],
"type": "object",
"properties": {
"agent_id": {
"id": {
"type": "string"
},
"from": {
@@ -7139,45 +7272,6 @@
}
}
},
"NodeBinaryDto": {
"required": [
"sha256",
"signed",
"seq",
"bound_at",
"violation",
"violation_at"
],
"type": "object",
"properties": {
"sha256": {
"type": "string"
},
"signed": {
"type": "boolean"
},
"seq": {
"pattern": "^-?(?:0|[1-9]\\d*)$",
"type": [
"integer",
"string"
],
"format": "int64"
},
"bound_at": {
"format": "date-time"
},
"violation": {
"type": [
"null",
"string"
]
},
"violation_at": {
"format": "date-time"
}
}
},
"NodeBuildDto": {
"required": [
"id",
@@ -7250,16 +7344,6 @@
}
]
},
"binary": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/NodeBinaryDto"
}
]
},
"owners": {
"type": [
"null",
@@ -7337,6 +7421,10 @@
"null",
"string"
]
},
"isp_manual": {
"type": "boolean",
"default": false
}
}
},
@@ -8164,6 +8252,12 @@
"null",
"string"
]
},
"isp": {
"type": [
"null",
"string"
]
}
}
},
@@ -8464,6 +8558,29 @@
}
}
},
"QuadletInstallDto": {
"required": [
"env",
"url",
"sha256",
"command"
],
"type": "object",
"properties": {
"env": {
"type": "string"
},
"url": {
"type": "string"
},
"sha256": {
"type": "string"
},
"command": {
"type": "string"
}
}
},
"RawChunkMetaDto": {
"required": [
"chunk_id",
@@ -8674,6 +8791,43 @@
}
}
},
"ReleaseKeyDto": {
"required": [
"fingerprint",
"public_key"
],
"type": "object",
"properties": {
"fingerprint": {
"type": "string"
},
"public_key": {
"type": "string"
}
}
},
"ReleaseKeysDto": {
"required": [
"official",
"custom"
],
"type": "object",
"properties": {
"official": {
"$ref": "#/components/schemas/ReleaseKeyDto"
},
"custom": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/ReleaseKeyDto"
}
]
}
}
},
"ScoreDto": {
"required": [
"score",
@@ -8836,6 +8990,20 @@
}
}
},
"SetReleaseKeyRequest": {
"required": [
"public_key"
],
"type": "object",
"properties": {
"public_key": {
"type": [
"null",
"string"
]
}
}
},
"SettingDto": {
"required": [
"key",
@@ -9442,9 +9610,6 @@
{
"name": "admin-auth"
},
{
"name": "admin-build"
},
{
"name": "admin-agents"
},
@@ -9478,6 +9643,12 @@
{
"name": "admin-users"
},
{
"name": "admin-build"
},
{
"name": "admin-release-key"
},
{
"name": "admin-links"
},
+4 -4
View File
@@ -13,7 +13,7 @@ Backend ──任务──> Agent <──测量会话──> Probe
Probe 用 `probe.json` 读取面板地址(`fleet.server`)、身份目录(`fleet.identity_dir`,由 `fleet-enroll` 注册时生成)和本地测试限制。连上面板之后,面板下发要连接的 Agent 列表;Probe 将列表缓存到同目录的 `agents.json`,面板暂时连不上时靠它继续测量。身份目录里有私钥,请限制访问权限,不要提交到版本控制。
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/deployment.md`](../../docs/deployment.md);Probe 也可以用 OCI 镜像 `registry.oeo.one/moshiro/moshiro-probe:<version>`,容器默认读取 `/etc/moshiro/probe.json`,命名 volume 保存身份配置和缓存,镜像以 UID/GID 1654 运行。
容器节点的运行方式(podman + systemd)和宿主机更新器见 [`../../docs/ops/deployment.md`](../../docs/ops/deployment.md);Probe 也可以用 OCI 镜像 `registry.oeo.one/moshiro/moshiro-probe:<version>`,容器默认读取 `/etc/moshiro/probe.json`,命名 volume 保存身份配置和缓存,镜像以 UID/GID 1654 运行。
## 构建
@@ -30,7 +30,7 @@ dotnet build probe/src/MoShiro.Probe -c Release
dotnet publish probe/src/MoShiro.Probe -c Release -r <RID> -p:PublishAot=true
```
也可在仓库根目录运行 `bash scripts/publish.sh probe <RID>`,再使用 `scripts/package.sh probe <RID>` 生成部署包。支持平台和安装流程见 [`../../docs/deployment.md`](../../docs/deployment.md)。
也可在仓库根目录运行 `bash scripts/publish.sh probe <RID>`,再使用 `scripts/package.sh probe <RID>` 生成部署包。支持平台和安装流程见 [`../../docs/ops/deployment.md`](../../docs/ops/deployment.md)。
## 配置与运行
@@ -54,6 +54,6 @@ moshiro-probe --config /etc/moshiro/probe.json
- Probe 只执行编译进程序的测量模块。它不落盘测量数据,也不能执行命令或自行更新(更新由宿主机上的更新器完成,Probe 只写清单和请求文件)。
- 测量模块由 `backend/shared/MoShiro.Measurement/` 与 Agent 共用;Probe 负责发起连接、调度任务和记录本端数据。
- 测量模型和协议细节见 [`../../docs/measurement.md`](../../docs/measurement.md) 与 [`../../docs/protocol.md`](../../docs/protocol.md);整体架构见 [`../../docs/architecture.md`](../../docs/architecture.md)。
- 测量模型和协议细节见 [`../../docs/design/measurement.md`](../../docs/design/measurement.md) 与 [`../../docs/design/protocol.md`](../../docs/design/protocol.md);整体架构见 [`../../docs/design/architecture.md`](../../docs/design/architecture.md)。
Docker 与 rootless/rootful Podman 共用 `packaging/compose/probe/compose.yaml`,配置、身份和缓存放在命名 volume 中,无需宿主配置目录、keep-id 或手动 chown。首次导入和自动更新见部署文档。
Podman 系用 Quadlet(`packaging/quadlet/moshiro-probe.container`,默认自动更新),Docker 用 `packaging/compose/probe/compose.yaml`(手动更新);配置、身份和缓存放在命名 volume 中,无需宿主配置目录、keep-id 或手动 chown。首次导入和自动更新见部署文档。
+44 -7
View File
@@ -22,28 +22,30 @@ public sealed class ProbeApp : IDisposable
private readonly SemaphoreSlim _changed = new(0);
private readonly INodeLog _log;
private readonly FleetLink _link;
private readonly ReportingSection _reporting;
private readonly TimeProvider _time;
/// <param name="identity">注册时由节点自己生成:<c>Identity</c> 用来连面板,<c>PeerIdentity</c> 是另一把独立的密钥,只用来连 Agent。</param>
public ProbeApp(ProbeFile file, LoadedNodeIdentity identity, string configDirectory, TimeProvider time, INodeLog log, RestartController? restart = null)
{
_log = log;
_time = time;
_reporting = file.Reporting;
var trust = TrustAnchor.Load(file.Fleet.IdentityDir, log);
var directory = new AgentDirectory(new ConfigStore(configDirectory), configDirectory, log, trust);
_env = new ConnectionEnv(identity.PeerIdentity, identity.NodeId.Value, file.Limits, directory, new MonoClock(time), time, log);
directory.Changed += () => _changed.Release();
var update = new UpdateBridge(configDirectory, new ConfigStore(configDirectory).Write, log);
NodeBinary.Prime(); // 正在运行的二进制的哈希:先在后台算好,握手时用
RestartUpdater? update = RestartUpdater.FromEnvironment("probe", reason => restart?.RequestNow(reason), time, log);
var options = new FleetLinkOptions
{
Server = new Uri(file.Fleet.Server),
Identity = identity.Identity,
ServerVerifier = trust,
Attestation = update.Attest,
NodeId = identity.NodeId,
NodeVersion = Version,
Capabilities = [MoShiroCapabilities.Base],
HeartbeatExtra = () => JsonSerializer.SerializeToElement(new ProbeHeartbeatExtra { Update = update.ReadStatus(), Ipv6 = Ipv6Diagnostics.Sample(null, Environment.GetEnvironmentVariable("MOSHIRO_NETWORK_SCOPE"), Environment.GetEnvironmentVariable("HOST_PROC")) }, MoShiroJsonContext.Default.ProbeHeartbeatExtra),
HeartbeatExtra = () => JsonSerializer.SerializeToElement(new ProbeHeartbeatExtra { Update = update?.Status(), Ipv6 = Ipv6Diagnostics.Sample(null, Environment.GetEnvironmentVariable("MOSHIRO_NETWORK_SCOPE"), Environment.GetEnvironmentVariable("HOST_PROC")) }, MoShiroJsonContext.Default.ProbeHeartbeatExtra),
HelloMeta = () => JsonSerializer.SerializeToElement(new ProbeHelloMeta { Owners = [.. trust.Controllers], Platform = PlatformDetector.Detect(), Ipv6 = Ipv6Diagnostics.Sample(null, Environment.GetEnvironmentVariable("MOSHIRO_NETWORK_SCOPE"), Environment.GetEnvironmentVariable("HOST_PROC")) }, MoShiroJsonContext.Default.ProbeHelloMeta),
};
var list = new StateHandler<ProbeAgentList>(
@@ -57,12 +59,12 @@ public sealed class ProbeApp : IDisposable
var router = new RpcRouter<LinkContext>(RpcSide.Node, MoShiroMethodTable.Table, NodeRoles.Probe)
.OnNotify(MoShiroMethods.UpdateOffer, MoShiroJsonContext.Default.UpdateOfferParams, (_, offer, _) =>
{
update.WriteOffer(offer);
update?.Offer(offer);
return ValueTask.CompletedTask;
})
.OnNotify(MoShiroMethods.UpdateApply, MoShiroJsonContext.Default.UpdateOfferParams, (_, _, _) =>
{
update.RequestApply();
update?.RequestApply();
return ValueTask.CompletedTask;
})
.OnNotify(MoShiroMethods.Schedule, MoShiroJsonContext.Default.ScheduleParams, (_, schedule, _) =>
@@ -107,6 +109,7 @@ public sealed class ProbeApp : IDisposable
}
Task link = _link.RunAsync(ct);
Task host = ReportHostAsync(ct);
try
{
while (!ct.IsCancellationRequested)
@@ -122,7 +125,41 @@ public sealed class ProbeApp : IDisposable
finally
{
await ReconcileAsync([], CancellationToken.None);
await Task.WhenAny(link, Task.Delay(TimeSpan.FromSeconds(5), CancellationToken.None));
await Task.WhenAny(Task.WhenAll(link, host), Task.Delay(TimeSpan.FromSeconds(5), CancellationToken.None));
}
}
/// <summary>每隔几秒把主机资源占用的汇总数字发给面板,和 Agent 一样(没有型号、主机名、网卡名和 IP);链路断开时丢弃,重连之后下一次就补上。</summary>
private async Task ReportHostAsync(CancellationToken ct)
{
if (!_reporting.HostStats)
{
return;
}
var sampler = new HostSampler(_time);
sampler.Sample(); // 第一次只用来建立 CPU 和网速的基线
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(Math.Clamp(_reporting.HostIntervalSeconds, 1, 60)), _time);
try
{
while (await timer.WaitForNextTickAsync(ct))
{
if (_link.CurrentSession is { } session)
{
try
{
await session.NotifyAsync(MoShiroMethods.Host, sampler.Sample(), MoShiroJsonContext.Default.HostSnapshot, CancellationToken.None);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_log.Warn($"could not report host status: {ex.Message}");
}
}
}
}
catch (OperationCanceledException)
{
// 停止
}
}
@@ -1,4 +1,5 @@
using System.Text.Json.Serialization;
using MoShiro.Measurement;
using MoShiro.Protocol;
namespace MoShiro.Probe;
@@ -18,6 +19,10 @@ public sealed class ProbeFile
[JsonPropertyName("limits")]
public ProbeLimits Limits { get; set; } = new();
/// <summary>上报主机资源占用,和 Agent 一样(只有汇总数字,没有型号和 IP);默认开,设 <c>host_stats: false</c> 就一个字也不上报。</summary>
[JsonPropertyName("reporting")]
public ReportingSection Reporting { get; set; } = new();
}
public sealed class ProbeFleetSection
@@ -1,6 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsAotCompatible>true</IsAotCompatible>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>
<ItemGroup>
<InternalsVisibleTo Include="MoShiro.Measurement.Tests" />
@@ -1,13 +1,13 @@
using System.Globalization;
using MoShiro.Protocol;
namespace MoShiro.Agent;
namespace MoShiro.Measurement;
/// <summary>
/// 主机资源占用的只读快照:CPU 占用、内存、根分区、网卡累计流量与瞬时网速、系统运行时长。
/// Agent 和 Probe 共用。主机资源占用的只读快照:CPU 占用、内存、根分区、网卡累计流量与瞬时网速、系统运行时长。
/// 只有汇总的数字:<b>不采集 CPU 型号、主机名、网卡名和任何 IP 地址</b>,所以面板和展示页想泄漏也泄漏不出去。
/// Linux 读 <c>/proc</c> 和 <c>/sys</c>;容器通过 HOST_PROC、HOST_SYS、HOST_ROOT 指定宿主路径。
/// 读不到的项是 null(例如 Windows 上的 CPU)。不执行任何命令,不写文件。
/// Windows 用 GetSystemTimes、GlobalMemoryStatusEx 和 NetworkInterface(见 WindowsHost);macOS 等其他系统只有磁盘和运行时长。读不到的项是 null。不执行任何命令,不写文件。
/// </summary>
public sealed class HostSampler
{
@@ -18,6 +18,7 @@ public sealed class HostSampler
private readonly Func<(long Total, long Used)?> _disk;
private readonly string _procRoot;
private readonly string _sysRoot;
private readonly bool _native;
private (long Total, long Idle)? _previousCpu;
private (long At, long Rx, long Tx)? _previousNet;
@@ -25,6 +26,7 @@ public sealed class HostSampler
string? procRoot = null, string? sysRoot = null, string? diskRoot = null)
{
_time = time;
_native = read is null && OperatingSystem.IsWindows(); // 只有默认读法才走 Windows 的系统调用;测试注入 read 时仍按 Linux 的文本解析
_read = read ?? ReadProc;
_procRoot = Root(procRoot, "HOST_PROC", "/proc");
_sysRoot = Root(sysRoot, "HOST_SYS", "/sys");
@@ -37,7 +39,8 @@ public sealed class HostSampler
long now = _time.GetTimestamp();
var snapshot = new HostSnapshot { AtUtcMs = _time.GetUtcNow().ToUnixTimeMilliseconds() };
if (_read(Path.Combine(_procRoot, "stat")) is { } stat && ParseCpu(stat) is { } cpu)
(long Total, long Idle)? cpuNow = _native ? WindowsCpu() : _read(Path.Combine(_procRoot, "stat")) is { } stat ? ParseCpu(stat) : null;
if (cpuNow is { } cpu)
{
if (_previousCpu is { } before && cpu.Total > before.Total)
{
@@ -48,7 +51,8 @@ public sealed class HostSampler
_previousCpu = cpu;
}
if (_read(Path.Combine(_procRoot, "meminfo")) is { } meminfo && ParseMemory(meminfo) is { } memory)
(long Total, long Used)? memoryNow = _native ? WindowsMemory() : _read(Path.Combine(_procRoot, "meminfo")) is { } meminfo ? ParseMemory(meminfo) : null;
if (memoryNow is { } memory)
{
snapshot.MemTotal = memory.Total;
snapshot.MemUsed = memory.Used;
@@ -60,11 +64,21 @@ public sealed class HostSampler
snapshot.DiskUsed = disk.Used;
}
if (_read(Path.Combine(_procRoot, "net/dev")) is { } dev && ParseNetwork(dev) is { } net)
(long Rx, long Tx, long? LinkMbps)? netNow = null;
if (_native)
{
netNow = WindowsNetwork();
}
else if (_read(Path.Combine(_procRoot, "net/dev")) is { } dev && ParseNetwork(dev) is { } parsed)
{
netNow = (parsed.Rx, parsed.Tx, ReadLinkSpeed(dev));
}
if (netNow is { } net)
{
snapshot.NetRxTotal = net.Rx;
snapshot.NetTxTotal = net.Tx;
snapshot.NetLinkMbps = ReadLinkSpeed(dev);
snapshot.NetLinkMbps = net.LinkMbps;
if (_previousNet is { } before && net.Rx >= before.Rx && net.Tx >= before.Tx)
{
double seconds = Math.Max(0.001, _time.GetElapsedTime(before.At, now).TotalSeconds);
@@ -83,6 +97,12 @@ public sealed class HostSampler
return snapshot;
}
private static (long Total, long Idle)? WindowsCpu() => OperatingSystem.IsWindows() ? WindowsHost.Cpu() : null;
private static (long Total, long Used)? WindowsMemory() => OperatingSystem.IsWindows() ? WindowsHost.Memory() : null;
private static (long Rx, long Tx, long? LinkMbps)? WindowsNetwork() => OperatingSystem.IsWindows() ? WindowsHost.Network() : null;
/// <summary>第一行 <c>cpu user nice system idle iowait irq softirq steal …</c>:总时间与空闲时间(idle + iowait)。</summary>
public static (long Total, long Idle)? ParseCpu(string stat)
{
@@ -0,0 +1,14 @@
using System.Text.Json.Serialization;
namespace MoShiro.Measurement;
/// <summary>节点本地配置里的 <c>reporting</c> 段(面板无法修改):Agent 和 Probe 共用。</summary>
public sealed class ReportingSection
{
/// <summary>上报主机资源占用(CPU、内存、磁盘、流量、运行时长,只有汇总数字,没有型号和 IP)。设为 false 就一个字也不上报。</summary>
[JsonPropertyName("host_stats")]
public bool HostStats { get; set; } = true;
[JsonPropertyName("host_interval_s")]
public int HostIntervalSeconds { get; set; } = 2;
}
@@ -0,0 +1,125 @@
using System.Net.NetworkInformation;
using System.Runtime.InteropServices;
using System.Runtime.Versioning;
namespace MoShiro.Measurement;
/// <summary>
/// Windows 上读主机资源占用(<see cref="HostSampler"/> 在 Linux 之外的实现):CPU 时间和物理内存用两个只读的系统调用
/// (<c>GetSystemTimes</c>、<c>GlobalMemoryStatusEx</c>),网卡流量和链路速率用 .NET 自带的 <see cref="NetworkInterface"/>。
/// 和 Linux 一样只有汇总的数字:不读网卡名、MAC、IP,也不启动任何进程。
/// </summary>
[SupportedOSPlatform("windows")]
internal static partial class WindowsHost
{
private static readonly string[] s_virtualWords = ["Virtual", "vEthernet", "Hyper-V", "VMware", "VirtualBox", "TAP-", "Wintun", "WireGuard", "Npcap", "Bluetooth", "Loopback", "isatap", "Teredo"];
[StructLayout(LayoutKind.Sequential)]
private struct FileTime
{
public uint Low;
public uint High;
public readonly long Value => (long)(((ulong)High << 32) | Low);
}
[StructLayout(LayoutKind.Sequential)]
private struct MemoryStatus
{
public uint Length;
public uint MemoryLoad;
public ulong TotalPhys;
public ulong AvailPhys;
public ulong TotalPageFile;
public ulong AvailPageFile;
public ulong TotalVirtual;
public ulong AvailVirtual;
public ulong AvailExtendedVirtual;
}
[LibraryImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static partial bool GetSystemTimes(out FileTime idle, out FileTime kernel, out FileTime user);
[LibraryImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static partial bool GlobalMemoryStatusEx(ref MemoryStatus status);
/// <summary>系统启动以来的 CPU 总时间和空闲时间(和 Linux 的 <c>/proc/stat</c> 第一行同一个意思:两次采样相减得到占用)。</summary>
public static (long Total, long Idle)? Cpu()
{
try
{
if (!GetSystemTimes(out FileTime idle, out FileTime kernel, out FileTime user))
{
return null;
}
// 内核时间里已经包含了空闲时间
return (kernel.Value + user.Value, idle.Value);
}
catch (Exception ex) when (ex is DllNotFoundException or EntryPointNotFoundException)
{
return null;
}
}
/// <summary>物理内存的总量和已用(总量减去可用)。</summary>
public static (long Total, long Used)? Memory()
{
try
{
var status = new MemoryStatus { Length = (uint)Marshal.SizeOf<MemoryStatus>() };
if (!GlobalMemoryStatusEx(ref status) || status.TotalPhys == 0)
{
return null;
}
return ((long)status.TotalPhys, (long)(status.TotalPhys - Math.Min(status.AvailPhys, status.TotalPhys)));
}
catch (Exception ex) when (ex is DllNotFoundException or EntryPointNotFoundException)
{
return null;
}
}
/// <summary>所有物理网卡(在用的,排除回环、隧道、虚拟网卡)的累计收发字节,以及链路速率之和(Mbit/s;有一块读不到速率就是 null)。</summary>
public static (long Rx, long Tx, long? LinkMbps)? Network()
{
try
{
long rx = 0;
long tx = 0;
long mbps = 0;
bool speedKnown = true;
bool any = false;
foreach (NetworkInterface nic in NetworkInterface.GetAllNetworkInterfaces())
{
if (nic.OperationalStatus != OperationalStatus.Up || nic.NetworkInterfaceType is NetworkInterfaceType.Loopback or NetworkInterfaceType.Tunnel
|| s_virtualWords.Any(w => nic.Description.Contains(w, StringComparison.OrdinalIgnoreCase) || nic.Name.Contains(w, StringComparison.OrdinalIgnoreCase)))
{
continue;
}
IPInterfaceStatistics stats = nic.GetIPStatistics();
rx += stats.BytesReceived;
tx += stats.BytesSent;
any = true;
if (nic.Speed > 0)
{
mbps += nic.Speed / 1_000_000;
}
else
{
speedKnown = false;
}
}
return any ? (rx, tx, speedKnown && mbps > 0 ? mbps : null) : null;
}
catch (NetworkInformationException)
{
return null;
}
}
}
@@ -0,0 +1,153 @@
using System.Globalization;
using System.Text.Json;
using Fleet.Core;
using MoShiro.Protocol;
namespace MoShiro.Measurement;
/// <summary>
/// 节点的「更新」:面板建议、请求,节点<b>只负责有序退出</b>,换新版本由外面的服务管理器完成——
/// 例如 Podman 的 Quadlet 单元写了 <c>Pull=always</c>,每次启动都先从镜像仓库拉最新的镜像,所以退出再被拉起就是更新。
/// 节点自己不下载、不安装、不执行任何东西,所以不需要验证什么:最多让面板(管理员)请求一次重启,和定时重启是同一种能力。
/// <list type="bullet">
/// <item>只有设了环境变量 <c>MOSHIRO_UPDATE=restart</c> 才启用(Quadlet 模板里写好了):别的部署方式退出之后不会重新拉取,开了也不会更新,所以没有启用就不报告更新状态。</item>
/// <item>策略由环境变量 <c>MOSHIRO_UPDATE_POLICY</c> 决定:<c>auto</c>(默认)发现新版本就自己退出重启,<c>manual</c> 等管理端请求。自动更新有保护:
/// 进程至少运行 <see cref="MinimumUptime"/> 才会自动退出,新镜像没拉到(标签没变、仓库还没更新)最多每个这么久重启一次,不会陷入重启循环。</item>
/// <item>面板转来的发布清单(面板登记时已验过 CI 签名)只用来显示「有没有新版本」:组件对、比这个程序新(seq 晚于构建时间)、tag 和提交不同。</item>
/// </list>
/// </summary>
public sealed class RestartUpdater
{
public const string EnvironmentVariable = "MOSHIRO_UPDATE";
public const string PolicyVariable = "MOSHIRO_UPDATE_POLICY";
/// <summary>自动更新之前,进程至少要运行这么久。</summary>
public static readonly TimeSpan MinimumUptime = TimeSpan.FromMinutes(10);
private const int MaxManifestBytes = 4096;
private readonly string _component;
private readonly Action<string> _requestRestart;
private readonly TimeProvider _time;
private readonly INodeLog _log;
private readonly string _tag;
private readonly string _commit;
private readonly long _builtAt;
private readonly object _gate = new();
private readonly bool _auto;
private int _autoScheduled;
private readonly long _startedAt;
private UpdateStatus _status;
/// <param name="tag">这个程序的构建标签、提交和构建时间(秒):默认取 <see cref="BuildInfo"/>,测试可以传入。</param>
public RestartUpdater(string component, Action<string> requestRestart, TimeProvider time, INodeLog log, string? tag = null, string? commit = null, long? builtAtEpoch = null, bool auto = false)
{
_auto = auto;
_startedAt = time.GetTimestamp();
_component = component;
_requestRestart = requestRestart;
_time = time;
_log = log;
_tag = tag ?? BuildInfo.Tag;
_commit = commit ?? BuildInfo.Commit;
_builtAt = builtAtEpoch ?? (DateTimeOffset.TryParse(BuildInfo.BuiltAt, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out DateTimeOffset built) ? built.ToUnixTimeSeconds() : 0);
_status = new UpdateStatus { State = "up_to_date", At = Now() };
}
/// <summary>环境变量 <c>MOSHIRO_UPDATE=restart</c> 才启用;否则返回 null,节点不报告更新状态。</summary>
public static RestartUpdater? FromEnvironment(string component, Action<string> requestRestart, TimeProvider time, INodeLog log) =>
string.Equals(Environment.GetEnvironmentVariable(EnvironmentVariable), "restart", StringComparison.Ordinal)
? new RestartUpdater(component, requestRestart, time, log, auto: !string.Equals(Environment.GetEnvironmentVariable(PolicyVariable), "manual", StringComparison.OrdinalIgnoreCase))
: null;
/// <summary>面板转来的发布清单:只用来判断有没有新版本。</summary>
public void Offer(UpdateOfferParams offer)
{
if (offer.Manifest.Length is 0 or > MaxManifestBytes)
{
_log.Warn("ignoring an update offer of an unexpected size");
return;
}
try
{
using JsonDocument document = JsonDocument.Parse(offer.Manifest);
JsonElement root = document.RootElement;
if (root.ValueKind != JsonValueKind.Object || Text(root, "component") != _component || !root.TryGetProperty("seq", out JsonElement seqValue) || !seqValue.TryGetInt64(out long seq))
{
return;
}
string tag = Text(root, "tag");
string commit = Text(root, "commit");
bool same = (_tag.Length > 0 && tag == _tag) || (_commit.Length > 0 && commit.Length > 0 && commit == _commit);
bool available;
lock (_gate)
{
_status = same || seq <= _builtAt
? new UpdateStatus { State = "up_to_date", At = Now() }
: new UpdateStatus
{
State = "available",
Available = new UpdateAvailable { Tag = tag, Version = Text(root, "version"), Digest = Text(root, "digest"), Seq = seq },
At = Now(),
};
available = _status.State == "available";
}
if (available && _auto && Interlocked.Exchange(ref _autoScheduled, 1) == 0)
{
_ = ApplyAfterMinimumUptimeAsync();
}
}
catch (JsonException)
{
_log.Warn("ignoring an update offer that is not valid JSON");
}
}
/// <summary>自动更新:等进程运行够久再退出(刚启动就发现新版本,通常是重启之后标签还没变,立刻再退出会变成重启循环)。</summary>
private async Task ApplyAfterMinimumUptimeAsync()
{
TimeSpan wait = MinimumUptime - _time.GetElapsedTime(_startedAt);
if (wait > TimeSpan.Zero)
{
_log.Info("update available, restarting automatically later", ("in", wait.ToString("c", CultureInfo.InvariantCulture)));
await Task.Delay(wait, _time);
}
RequestApply();
}
/// <summary>面板请求更新:有新版本才退出(由外面的服务管理器拉起并拉取最新镜像)。</summary>
public bool RequestApply()
{
lock (_gate)
{
if (_status.State != "available")
{
return false;
}
_status = new UpdateStatus { State = "applying", Available = _status.Available, At = Now() };
}
_log.Info("update requested, restarting so the supervisor pulls the latest image");
_requestRestart("update");
return true;
}
/// <summary>随心跳上报的状态。</summary>
public UpdateStatus Status()
{
lock (_gate)
{
return new UpdateStatus { State = _status.State, Current = _status.Current, Available = _status.Available, Error = _status.Error, At = _status.At };
}
}
private long Now() => _time.GetUtcNow().ToUnixTimeSeconds();
private static string Text(JsonElement root, string name) =>
root.TryGetProperty(name, out JsonElement value) && value.ValueKind == JsonValueKind.String ? value.GetString() ?? "" : "";
}
@@ -1,129 +0,0 @@
using System.Text;
using System.Text.Json;
using Fleet.Core;
using MoShiro.Protocol;
namespace MoShiro.Measurement;
/// <summary>
/// 节点进程和宿主机更新器之间的文件交换(容器里 <c>/etc/moshiro</c>,也就是数据目录):
/// 节点把面板发来的清单和签名写进 <c>update/</c>,把「更新」请求写成 <c>update/apply.request</c>;更新器验签并更新,状态由宿主更新器写在只读挂载的 <c>update-status/status.json</c>。
/// 节点进程不执行任何命令,也不信任清单的内容(验签在宿主机上)。没有安装更新器时目录写不了或没人读,调用只是返回 false。
/// </summary>
public sealed class UpdateBridge
{
private const int MaxManifestBytes = 4096;
private const int MaxSignatureBytes = 512;
private const int MaxStatusBytes = 4096;
private readonly string _updateDirectory;
private readonly string _statusFile;
private readonly Action<string, byte[]> _write;
private readonly INodeLog _log;
/// <param name="write">写数据目录里的文件(相对路径,原子替换):由节点传入 <c>ConfigStore.Write</c>——常驻进程里只有它可以写文件。</param>
public UpdateBridge(string dataDirectory, Action<string, byte[]> write, INodeLog log)
{
_updateDirectory = Path.Combine(dataDirectory, "update");
_statusFile = Path.Combine(dataDirectory, "update-status", "status.json");
_write = write;
_log = log;
}
/// <summary>放下面板发来的清单和签名(和已有的一样就不动,免得每次重连都惊动更新器)。先写清单、后写签名:更新器监视的是签名文件。</summary>
public bool WriteOffer(UpdateOfferParams offer)
{
byte[] manifest = Encoding.UTF8.GetBytes(offer.Manifest);
byte[] signature = Encoding.UTF8.GetBytes(offer.Signature);
if (manifest.Length is 0 or > MaxManifestBytes || signature.Length is 0 or > MaxSignatureBytes)
{
_log.Warn("ignoring an update offer of an unexpected size");
return false;
}
try
{
string manifestPath = Path.Combine(_updateDirectory, "offer.manifest");
string signaturePath = Path.Combine(_updateDirectory, "offer.sig");
if (Same(manifestPath, manifest) && Same(signaturePath, signature))
{
return true;
}
_write("update/offer.manifest", manifest);
_write("update/offer.sig", signature);
_log.Info("update offer received");
return true;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_log.Warn("could not store the update offer (is the updater installed?)", ("err", ex.Message));
return false;
}
}
/// <summary>请求更新:写一个空的 <c>apply.request</c>(更新器验签通过、清单更新才会真的动手)。</summary>
public bool RequestApply()
{
try
{
_write("update/apply.request", []);
_log.Info("update requested");
return true;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_log.Warn("could not request the update (is the updater installed?)", ("err", ex.Message));
return false;
}
}
/// <summary>更新器的状态;没有安装更新器、文件不存在或内容不对时返回 null。</summary>
public UpdateStatus? ReadStatus()
{
try
{
if (!File.Exists(_statusFile) || new FileInfo(_statusFile).Length > MaxStatusBytes)
{
return null;
}
return JsonSerializer.Deserialize(File.ReadAllBytes(_statusFile), MoShiroJsonContext.Default.UpdateStatus);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException)
{
return null;
}
}
/// <summary>
/// 握手时自述的二进制:正在运行的可执行文件的哈希,加上宿主机更新器验过的、安装这个版本时用的 CI 签名清单(更新器写在只读的 <c>update-status/</c>)。
/// 容器节点以外(没有更新器)没有清单,只有哈希;读不到可执行文件时返回 null(不做绑定)。
/// </summary>
public BinaryAttestation? Attest()
{
if (NodeBinary.Sha256Hex is not { } hash)
{
return null;
}
string? manifest = ReadSmall("installed.manifest", MaxManifestBytes);
string? signature = ReadSmall("installed.sig", MaxSignatureBytes);
return manifest is not null && signature is not null ? new BinaryAttestation(hash, manifest, signature) : new BinaryAttestation(hash, null, null);
}
private string? ReadSmall(string name, int maxBytes)
{
try
{
string path = Path.Combine(Path.GetDirectoryName(_statusFile)!, name);
return File.Exists(path) && new FileInfo(path).Length <= maxBytes ? File.ReadAllText(path, Encoding.UTF8) : null;
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
return null;
}
}
private static bool Same(string path, byte[] content) => File.Exists(path) && File.ReadAllBytes(path).AsSpan().SequenceEqual(content);
}
@@ -3,7 +3,6 @@ namespace MoShiro.Protocol;
public static class ProtocolVersion
{
public const int Container = 1;
public const int Wire = 1;
}
public static class ProtocolLimits
@@ -39,5 +38,4 @@ public static class MoShiroCapabilities
{
public const string Base = "moshiro.v1";
public const string KindsPrefix = "moshiro.kinds:";
public const string BlocksPrefix = "moshiro.blocks:";
}
@@ -34,10 +34,10 @@ public static class MoShiroMethodTable
MethodSpec.Notification(MoShiroMethods.UpdateApply, RpcSide.Server),
MethodSpec.Notification(MoShiroMethods.Schedule, RpcSide.Server),
// 节点面:Agent → Backend
// 节点面:节点 → Backend
MethodSpec.Notification(MoShiroMethods.TestStatus, RpcSide.Node, NodeRoles.Agent),
MethodSpec.Notification(MoShiroMethods.Event, RpcSide.Node, NodeRoles.Agent),
MethodSpec.Notification(MoShiroMethods.Host, RpcSide.Node, NodeRoles.Agent),
MethodSpec.Notification(MoShiroMethods.Host, RpcSide.Node), // Agent 和 Probe 都上报主机资源占用的汇总
// 对等面:Probe(拨号方)→ Agent(监听方)
MethodSpec.Request(MoShiroMethods.Clock, RpcSide.Dialer),
@@ -5,6 +5,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="xunit.v3" />
<PackageReference Include="Microsoft.Extensions.TimeProvider.Testing" />
<ProjectReference Include="..\..\MoShiro.Measurement\MoShiro.Measurement.csproj" />
</ItemGroup>
<ItemGroup>
@@ -0,0 +1,113 @@
using Fleet.Core;
using Microsoft.Extensions.Time.Testing;
using MoShiro.Protocol;
namespace MoShiro.Measurement.Tests;
public sealed class RestartUpdaterTests
{
private readonly List<string> _restarts = [];
private RestartUpdater Create(string tag = "dev-aaa", string commit = "aaa", long builtAt = 1_000) =>
new("agent", _restarts.Add, TimeProvider.System, NullNodeLog.Instance, tag, commit, builtAt);
private static UpdateOfferParams Offer(string tag, long seq, string component = "agent", string commit = "bbb") =>
new() { Manifest = $"{{\"component\":\"{component}\",\"tag\":\"{tag}\",\"version\":\"0.4.0\",\"commit\":\"{commit}\",\"digest\":\"sha256:{new string('a', 64)}\",\"seq\":{seq}}}", Signature = "ignored" };
[Fact]
public void ANewerRelease_IsAvailable_AndOnlyARequestRestartsTheNode()
{
RestartUpdater updater = Create();
updater.Offer(Offer("dev-bbb", 2_000));
UpdateStatus status = updater.Status();
Assert.Equal("available", status.State);
Assert.Equal("dev-bbb", status.Available!.Tag);
Assert.Equal(2_000, status.Available.Seq);
Assert.Empty(_restarts);
Assert.True(updater.RequestApply());
Assert.Equal(["update"], _restarts);
Assert.Equal("applying", updater.Status().State);
}
[Fact]
public void TheReleaseThatIsAlreadyRunning_IsUpToDate_ByTagOrByCommit()
{
RestartUpdater updater = Create();
updater.Offer(Offer("dev-aaa", 2_000));
Assert.Equal("up_to_date", updater.Status().State);
updater.Offer(Offer("dev-other-tag", 2_000, commit: "aaa"));
Assert.Equal("up_to_date", updater.Status().State);
Assert.False(updater.RequestApply());
Assert.Empty(_restarts);
}
[Fact]
public void AnOfferOlderThanThisBuild_OrForAnotherComponent_IsNotAnUpdate()
{
RestartUpdater updater = Create(builtAt: 5_000);
updater.Offer(Offer("dev-bbb", 2_000));
Assert.Equal("up_to_date", updater.Status().State);
updater.Offer(Offer("dev-ccc", 9_000, component: "probe"));
Assert.Equal("up_to_date", updater.Status().State);
Assert.False(updater.RequestApply());
}
[Fact]
public void GarbageOffers_AreIgnored()
{
RestartUpdater updater = Create();
updater.Offer(new UpdateOfferParams { Manifest = "not json", Signature = "x" });
updater.Offer(new UpdateOfferParams { Manifest = new string('x', 5000), Signature = "x" });
updater.Offer(new UpdateOfferParams { Manifest = "{\"component\":\"agent\"}", Signature = "x" });
Assert.Equal("up_to_date", updater.Status().State);
Assert.False(updater.RequestApply());
}
[Fact]
public async Task TheAutoPolicy_RestartsByItself_ButNotBeforeTheMinimumUptime()
{
var time = new FakeTimeProvider(DateTimeOffset.FromUnixTimeSeconds(2_000_000_000));
var updater = new RestartUpdater("agent", _restarts.Add, time, NullNodeLog.Instance, "dev-aaa", "aaa", 1_000, auto: true);
updater.Offer(Offer("dev-bbb", 2_000));
updater.Offer(Offer("dev-bbb", 2_000)); // 重连时面板会再发一次:不能排两个
Assert.Empty(_restarts);
time.Advance(TimeSpan.FromMinutes(9));
await Task.Delay(50);
Assert.Empty(_restarts);
time.Advance(TimeSpan.FromMinutes(2));
await WaitAsync(() => _restarts.Count == 1);
Assert.Equal("applying", updater.Status().State);
await Task.Delay(50);
Assert.Single(_restarts);
}
[Fact]
public async Task TheManualPolicy_NeverRestartsByItself()
{
var time = new FakeTimeProvider(DateTimeOffset.FromUnixTimeSeconds(2_000_000_000));
var updater = new RestartUpdater("agent", _restarts.Add, time, NullNodeLog.Instance, "dev-aaa", "aaa", 1_000, auto: false);
updater.Offer(Offer("dev-bbb", 2_000));
time.Advance(TimeSpan.FromHours(1));
await Task.Delay(50);
Assert.Empty(_restarts);
Assert.Equal("available", updater.Status().State);
}
private static async Task WaitAsync(Func<bool> condition)
{
for (int i = 0; i < 100 && !condition(); i++)
{
await Task.Delay(20);
}
Assert.True(condition());
}
[Fact]
public void WithoutTheEnvironmentVariable_ThereIsNoUpdater()
{
Assert.Null(Environment.GetEnvironmentVariable(RestartUpdater.EnvironmentVariable));
Assert.Null(RestartUpdater.FromEnvironment("agent", _restarts.Add, TimeProvider.System, NullNodeLog.Instance));
}
}
@@ -143,7 +143,7 @@ public static class AgentEndpoints
return Problems.From(http, ownerError);
}
var plan = new InstallPlan(genesis, controllers, InstallScripts.Sha256(InstallScripts.Sh), InstallScripts.Sha256(InstallScripts.Ps1));
var plan = new InstallPlan(genesis, controllers, InstallScripts.Sha256(InstallScripts.Sh), InstallScripts.Sha256(InstallScripts.Ps1), InstallScripts.ComposeSha256(request.Role ?? JoinTokenService.AgentRole), InstallScripts.QuadletSha256(request.Role ?? JoinTokenService.AgentRole));
http.SetDetail($"{request.Role ?? JoinTokenService.AgentRole}: {request.Note}");
Result<CreatedJoinToken> result = await tokens.CreateAsync(
new Application.CreateJoinTokenRequest(request.Note, request.Labels, request.AutoApprove ?? true, request.MaxUses ?? 1, request.ExpiresInHours ?? 24, request.Role ?? JoinTokenService.AgentRole, http.GetPrincipal().Label), baseUrl, plan, http.RequestAborted);
@@ -154,7 +154,7 @@ public static class AgentEndpoints
http.Response.Headers.CacheControl = "no-store";
http.Response.Headers.Location = "/api/admin/join-tokens";
return TypedResults.Json(new CreatedJoinTokenDto(Mappers.ToDto(created.Info), created.Plaintext, new InstallCommandsDto(created.InstallCommands.Linux, created.InstallCommands.Macos, created.InstallCommands.Windows)), ApiJsonContext.Default.CreatedJoinTokenDto, statusCode: StatusCodes.Status201Created);
return TypedResults.Json(new CreatedJoinTokenDto(Mappers.ToDto(created.Info), created.Plaintext, new InstallCommandsDto(created.InstallCommands.Linux, created.InstallCommands.Macos, created.InstallCommands.Windows), new ComposeInstallDto(created.Compose.Env, created.Compose.Url, created.Compose.Sha256, created.Compose.Linux, created.Compose.Macos, created.Compose.Windows), new QuadletInstallDto(created.Quadlet.Env, created.Quadlet.Url, created.Quadlet.Sha256, created.Quadlet.Command)), ApiJsonContext.Default.CreatedJoinTokenDto, statusCode: StatusCodes.Status201Created);
}
private static async Task<IResult> RevokeTokenAsync(JoinTokenService tokens, string id, bool? purge, CancellationToken ct)
@@ -179,15 +179,32 @@ public static class AgentEndpoints
? TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto)
: Problems.Create(http, MoShiroErrors.ProbeNotFound, "probe not found");
private static async Task<IResult> PatchProbeAsync(HttpContext http, ProbeService probes, PatchProbeRequest request, string id, CancellationToken ct)
private static async Task<IResult> PatchProbeAsync(HttpContext http, ProbeService probes, NodeInfoService nodes, PatchProbeRequest request, string id, CancellationToken ct)
{
if (request.Name is null)
if (request.Name is null && request.Isp is null)
{
return Problems.From(http, ValidationError.Single("name", "is required"));
return Problems.From(http, ValidationError.Single("name", "name or isp is required"));
}
Result<ProbeView> result = await probes.RenameAsync(id, request.Name, ct);
return result.TryGet(out ProbeView? view, out FleetError? error) ? TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto) : Problems.From(http, error);
if (request.Isp is not null && request.Isp != "auto" && IspClassifier.NameOf(request.Isp) is null)
{
return Problems.From(http, ValidationError.Single("isp", "must be telecom, unicom, mobile, broadnet, other or auto"));
}
Result<ProbeView> result = request.Name is null
? (await probes.GetAsync(id, ct) is { } existing ? Result<ProbeView>.Ok(existing) : Result<ProbeView>.Fail(MoShiroErrors.ProbeNotFound, "probe not found"))
: await probes.RenameAsync(id, request.Name, ct);
if (!result.TryGet(out ProbeView? view, out FleetError? error))
{
return Problems.From(http, error);
}
if (request.Isp is not null)
{
await nodes.SetIspAsync(id, NodeKinds.Probe, request.Isp == "auto" ? null : request.Isp, ct);
}
return TypedResults.Json(Mappers.ToDto(view), ApiJsonContext.Default.ProbeDto);
}
private static async Task<IResult> RevokeProbeAsync(HttpContext http, ProbeService probes, string id, CancellationToken ct)
@@ -25,9 +25,6 @@ public static class BuildEndpoints
group.MapPost("/nodes/{id}/update", RequestUpdateAsync)
.Api("nodes_update", "Ask an online agent or probe to install the release it has been offered", Tag).Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403, 409)
.Requires(Capability.UpdateConfirm);
group.MapDelete("/nodes/{id}/binary", ClearBinaryAsync)
.Api("nodes_binary_clear", "Accept the node's current binary: clear its binding so the next handshake binds it again", Tag).Returns<object>(StatusCodes.Status204NoContent).ProducesProblems(401, 403)
.Requires(Capability.FleetWrite);
group.MapGet("/backend/update", BackendUpdateAsync)
.Api("backend_update_get", "Current channel and host-verified backend update", Tag).Returns<BackendUpdateDto>().ProducesProblems(401, 403)
.Requires(Capability.UpdateRead);
@@ -39,15 +36,6 @@ public static class BuildEndpoints
.Requires(Capability.UpdateRead);
}
/// <summary>登记发布清单:挂在 <see cref="ReleaseGuard"/> 下,除了管理员,CI 用的 <c>release</c> 令牌也能调用(它只能做这一件事)。</summary>
public static void MapPublish(RouteGroupBuilder group)
{
group.MapPost("/releases", PublishReleaseAsync)
.Api("releases_publish", "Register a CI-signed release manifest; online nodes get it right away (their updaters verify the signature). Admin or release tokens", Tag)
.Returns<ReleaseDto>(StatusCodes.Status201Created).ProducesProblems(400, 401, 403, 409)
.Requires(Capability.ReleasePublish);
}
/// <summary>
/// CI 登记清单的入口,<b>不需要令牌</b>:请求里的清单必须带有效的 CI 发布签名(用 Backend 内嵌的发布公钥验)才收下。
/// 这样登记不依赖面板里的任何凭证(清库、重装、还没有 passkey 的时候照常工作),也没有一把能被偷的长期令牌;
@@ -60,12 +48,12 @@ public static class BuildEndpoints
.Returns<ReleaseDto>(StatusCodes.Status201Created).ProducesProblems(400, 403, 409);
}
private static async Task<IResult> OfferReleaseAsync(HttpContext http, ReleaseService releases, PublishReleaseRequest request)
private static async Task<IResult> OfferReleaseAsync(HttpContext http, ReleaseService releases, ReleaseKeyService keys, PublishReleaseRequest request)
{
string manifest = request.Manifest ?? "";
string signature = request.Signature ?? "";
if (manifest.Length is 0 or > ReleaseService.MaxManifestBytes || signature.Length is 0 or > ReleaseService.MaxSignatureBytes
|| !ReleaseSignature.Verify(InstallScripts.ReleaseKey(), manifest, signature))
|| !await keys.VerifyAsync(manifest, signature, http.RequestAborted))
{
return Problems.Create(http, MoShiroErrors.Forbidden, "the manifest does not carry a valid release signature");
}
@@ -73,14 +61,13 @@ public static class BuildEndpoints
return await PublishReleaseAsync(http, releases, request);
}
private static async Task<IResult> GetAsync(HttpContext http, NodeInfoService nodes, ReleaseService releases, BackendUpdates backendUpdates, BinaryBindingService bindings, NodeOwnerService owners, CancellationToken ct)
private static async Task<IResult> GetAsync(HttpContext http, NodeInfoService nodes, ReleaseService releases, BackendUpdates backendUpdates, NodeOwnerService owners, CancellationToken ct)
{
IReadOnlyDictionary<string, IReadOnlyList<NodeOwner>> ownerOf = await owners.ListAsync(ct);
Dictionary<string, NodeBinding> bound = (await bindings.ListAsync(ct)).ToDictionary(b => b.NodeId, StringComparer.Ordinal);
var items = (await nodes.ListAsync(ct))
.OrderBy(n => n.Kind, StringComparer.Ordinal)
.ThenBy(n => n.NodeId, StringComparer.Ordinal)
.Select(n => new NodeBuildDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Version, n.Tag, n.Commit, n.BuiltAt, ToDto(releases.StatusOf(new NodeId(n.NodeId))), bound.TryGetValue(n.NodeId, out NodeBinding? binding) ? ToDto(binding) : null, ownerOf.TryGetValue(n.NodeId, out IReadOnlyList<NodeOwner>? list) ? [.. list.Select(o => new NodeOwnerDto(o.CredentialId, o.PasskeyName, o.Username))] : null))
.Select(n => new NodeBuildDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Version, n.Tag, n.Commit, n.BuiltAt, ToDto(releases.StatusOf(new NodeId(n.NodeId))), ownerOf.TryGetValue(n.NodeId, out IReadOnlyList<NodeOwner>? list) ? [.. list.Select(o => new NodeOwnerDto(o.CredentialId, o.PasskeyName, o.Username))] : null))
.ToList();
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new BuildInfoDto(NodeEndpoints.Backend(withCommit: true), items, ToDto(await backendUpdates.GetAsync(ct))), ApiJsonContext.Default.BuildInfoDto);
@@ -138,17 +125,6 @@ public static class BuildEndpoints
}
}
private static NodeBinaryDto ToDto(NodeBinding b) =>
new(b.Sha256, b.Signed, b.Seq, DateTimeOffset.FromUnixTimeMilliseconds(b.BoundAtMs), b.Violation, b.ViolationAtMs is { } at ? DateTimeOffset.FromUnixTimeMilliseconds(at) : null);
/// <summary>管理员确认新的二进制是自己要的:清除绑定,节点下一次握手重新绑定(没有绑定可清也返回 204,幂等)。</summary>
private static async Task<IResult> ClearBinaryAsync(HttpContext http, BinaryBindingService bindings, string id)
{
http.SetDetail("binary binding cleared");
await bindings.ClearAsync(id, http.RequestAborted);
return TypedResults.NoContent();
}
private static NodeUpdateDto? ToDto(UpdateStatus? status) =>
status is null ? null : new NodeUpdateDto(status.State, status.Current, status.Available?.Tag, status.Available?.Version, status.Error, status.At > 0 ? DateTimeOffset.FromUnixTimeSeconds(status.At) : null);
}
@@ -0,0 +1,54 @@
using Fleet.Core;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using MoShiro.Backend.Application;
namespace MoShiro.Backend.Api;
/// <summary>可信资产来源:官方发布密钥(只读)和可选的自有发布公钥(默认没有)。改动写审计。</summary>
public static class ReleaseKeyEndpoints
{
private const string Tag = "admin-release-key";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/release-key", GetAsync)
.Api("release_key_get", "The release keys this panel trusts: the built-in official key and the optional own key", Tag)
.Returns<ReleaseKeysDto>().ProducesProblems(401, 403)
.Requires(Capability.UpdateRead);
group.MapPut("/release-key", SetAsync)
.Api("release_key_set", "Trust your own release public key (P-256 PEM; never the private key) in addition to the official one", Tag)
.Returns<ReleaseKeysDto>().ProducesProblems(400, 401, 403)
.Requires(Capability.UpdateConfirm);
group.MapDelete("/release-key", ClearAsync)
.Api("release_key_clear", "Stop trusting your own release key; manifests only it signed are dropped", Tag)
.Returns<ReleaseKeysDto>().ProducesProblems(401, 403)
.Requires(Capability.UpdateConfirm);
}
private static async Task<IResult> GetAsync(ReleaseKeyService keys, CancellationToken ct) => Json(await keys.GetAsync(ct));
private static async Task<IResult> SetAsync(HttpContext http, ReleaseKeyService keys, SetReleaseKeyRequest request)
{
Result<ReleaseKeys> result = await keys.SetCustomAsync(request.PublicKey, http.RequestAborted);
if (!result.TryGet(out ReleaseKeys? view, out FleetError? error))
{
return Problems.From(http, error);
}
http.SetDetail("own release key " + view.Custom!.Fingerprint);
return Json(view);
}
private static async Task<IResult> ClearAsync(HttpContext http, ReleaseKeyService keys, ReleaseService releases)
{
await keys.ClearCustomAsync(http.RequestAborted);
await releases.PurgeUnverifiedAsync(http.RequestAborted);
http.SetDetail("own release key removed");
return Json(await keys.GetAsync(http.RequestAborted));
}
private static Microsoft.AspNetCore.Http.HttpResults.JsonHttpResult<ReleaseKeysDto> Json(ReleaseKeys keys) =>
TypedResults.Json(new ReleaseKeysDto(new ReleaseKeyDto(keys.Official.Fingerprint, keys.Official.PublicKey), keys.Custom is null ? null : new ReleaseKeyDto(keys.Custom.Fingerprint, keys.Custom.PublicKey)), ApiJsonContext.Default.ReleaseKeysDto);
}
@@ -8,7 +8,7 @@ using MoShiro.Protocol;
namespace MoShiro.Backend.Api;
/// <summary>
/// 信任根(docs/access.md 第 7 节):操作员集合由 passkey 签名,节点从安装命令里固定的创世指纹出发验证它;
/// 信任根(docs/security/access.md 第 8 节):操作员集合由 passkey 签名,节点从安装命令里固定的创世指纹出发验证它;
/// 带操作员委托的加入码也在这里创建——委托是 passkey 对「这个码」签的,面板自己签不出来。
/// </summary>
public static class TrustEndpoints
+1 -2
View File
@@ -105,8 +105,6 @@ public static class ApiModule
RouteGroupBuilder admin = app.MapGroup("/api/admin");
AuthEndpoints.MapLogin(admin);
BuildEndpoints.MapPublish(admin.MapGroup(string.Empty).AddEndpointFilter(CapabilityGuard.InvokeAsync));
RouteGroupBuilder guarded = admin.MapGroup(string.Empty).AddEndpointFilter(CapabilityGuard.InvokeAsync);
AuthEndpoints.MapSession(guarded);
AgentEndpoints.Map(guarded);
@@ -116,6 +114,7 @@ public static class ApiModule
TrustEndpoints.Map(guarded);
SystemEndpoints.Map(guarded);
BuildEndpoints.Map(guarded);
ReleaseKeyEndpoints.Map(guarded);
LinkEndpoints.Map(guarded);
OpsEndpoints.Map(app);
@@ -83,6 +83,20 @@ public sealed record SetupRequest(string? ClaimCode, string? Username, string? P
public sealed record InstallCommandsDto(string Linux, string Macos, string Windows);
/// <param name="Env">要存成 <c>.env</c> 的内容(只有公开的值)。</param>
/// <param name="Url">compose 文件的下载地址。</param>
/// <param name="Sha256">compose 文件的 SHA-256:命令下载之后先核对它。</param>
/// <param name="Linux">Linux(Podman):建目录、写 .env、下载、核对哈希、启动,合在一起的一段命令。</param>
/// <param name="Macos">macOS(Docker Desktop);Agent 需要 Linux 宿主网络,所以是空的。</param>
/// <param name="Windows">Windows(Docker Desktop,PowerShell);Agent 同样是空的。</param>
public sealed record ComposeInstallDto(string Env, string Url, string Sha256, string Linux, string Macos, string Windows);
/// <param name="Env">要存成 <c>~/.config/moshiro/&lt;角色&gt;.env</c> 的内容(只有公开的值)。</param>
/// <param name="Url">Quadlet 单元的下载地址。</param>
/// <param name="Sha256">Quadlet 单元的 SHA-256:命令下载之后先核对它。</param>
/// <param name="Command">Linux + Podman(rootless):写 env、下载并核对单元、指定镜像仓库、启动,合在一起的一条命令。</param>
public sealed record QuadletInstallDto(string Env, string Url, string Sha256, string Command);
/// <summary>下一版里新增的 passkey。</summary>
public sealed record ProposedCredentialDto(string Id, string Name, DateTimeOffset CreatedAt);
@@ -100,7 +114,7 @@ public sealed record TrustProposalDto(string Statement, int Version, string Chal
/// <param name="InstallMatchesRelease">和 CI 签名的发布清单里记的哈希是否一致;没有清单或清单里没记时为空。不一致说明这个面板提供的脚本不是 CI 构建出来的那份。</param>
public sealed record TrustStateDto(bool Initialized, string? Genesis, int Version, bool InSync, TrustProposalDto? Proposal, string InstallUrl = "", IReadOnlyList<string>? Operators = null, string InstallShSha256 = "", string InstallPs1Sha256 = "", bool? InstallMatchesRelease = null, string ServerKey = "", bool ServerKeyListed = false, IReadOnlyList<NodeOwnerDto>? OperatorDetails = null);
public sealed record CreatedJoinTokenDto(JoinTokenDto Token, string Plaintext, InstallCommandsDto InstallCommands);
public sealed record CreatedJoinTokenDto(JoinTokenDto Token, string Plaintext, InstallCommandsDto InstallCommands, ComposeInstallDto Compose, QuadletInstallDto Quadlet);
/// <summary>Agent 眼中连着的一个 Probe(Agent 上报)。</summary>
public sealed record ConnectedProbeDto(string ProbeId, string? Version, string? Remote, DateTimeOffset ConnectedSince, string IpVersion = "ipv4");
@@ -117,7 +131,8 @@ public sealed record LinkMatrixDto(bool AutoLink, IReadOnlyList<LinkDto> Items);
public sealed record SetLinkRequest(bool? Enabled);
public sealed record PatchProbeRequest(string? Name);
/// <summary>Isp:telecom / unicom / mobile / broadnet / other 是手动指定,auto 是回到自动识别;不传就不改。</summary>
public sealed record PatchProbeRequest(string? Name, string? Isp = null);
public sealed record TaskDto(
string Id,
@@ -185,13 +200,10 @@ public sealed record ChangePasswordRequest(string Password);
public sealed record RecomputeDto(string State, IReadOnlyList<string> Metrics, long WindowsDone, long WindowsTotal, DateTimeOffset? From, DateTimeOffset? To, int QueueLength);
/// <summary>管理端看的构建信息:每个节点自报的版本、tag、提交和构建时间,以及系统。</summary>
/// <summary>节点二进制的绑定:第一次握手成功时绑定的 SHA-256。<c>Signed</c> 表示它出现在一份 CI 签名的清单里(否则只是首次握手时记下的);<c>Violation</c> 是最近一次不符的原因,下一次握手符合了就清掉。</summary>
public sealed record NodeBinaryDto(string Sha256, bool Signed, long Seq, DateTimeOffset BoundAt, string? Violation, DateTimeOffset? ViolationAt);
/// <summary>节点的一个属主:节点报上来的(安装时固定在节点本地的)passkey 凭证 id,以及它在面板上对应的 passkey 名称和账户(对不上任何已注册的 passkey 时为空)。</summary>
public sealed record NodeOwnerDto(string CredentialId, string? Passkey, string? Account);
public sealed record NodeBuildDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Version, string? Tag, string? Commit, string? BuiltAt, NodeUpdateDto? Update = null, NodeBinaryDto? Binary = null, IReadOnlyList<NodeOwnerDto>? Owners = null);
public sealed record NodeBuildDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Version, string? Tag, string? Commit, string? BuiltAt, NodeUpdateDto? Update = null, IReadOnlyList<NodeOwnerDto>? Owners = null);
/// <summary>节点宿主机上的更新器报告的状态(<c>up_to_date</c>、<c>available</c>、<c>applying</c>、<c>failed</c>、<c>rolled_back</c>、<c>invalid</c>);没装更新器的节点没有这个字段。</summary>
public sealed record NodeUpdateDto(string State, string? CurrentDigest, string? AvailableTag, string? AvailableVersion, string? Error, DateTimeOffset? At);
@@ -201,6 +213,14 @@ public sealed record ReleaseDto(string Component, long Seq, string Tag, string V
public sealed record PublishReleaseRequest(string? Manifest, string? Signature);
/// <summary>一把发布公钥:指纹是 SPKI 的 SHA-256(小写十六进制)。</summary>
public sealed record ReleaseKeyDto(string Fingerprint, string PublicKey);
/// <summary>可信资产来源:官方发布密钥(始终信任)和可选的自有发布公钥(默认没有)。</summary>
public sealed record ReleaseKeysDto(ReleaseKeyDto Official, ReleaseKeyDto? Custom);
public sealed record SetReleaseKeyRequest(string? PublicKey);
public sealed record BackendUpdateDto(string Channel, bool Enabled, string State, string? CurrentTag, string? CurrentDigest, ReleaseDto? Available, string? Error);
public sealed record ConfirmBackendUpdateRequest(long Seq);
@@ -45,9 +45,10 @@ namespace MoShiro.Backend.Api;
[JsonSerializable(typeof(ReleaseDto))]
[JsonSerializable(typeof(PageDto<ReleaseDto>))]
[JsonSerializable(typeof(PublishReleaseRequest))]
[JsonSerializable(typeof(ReleaseKeysDto))]
[JsonSerializable(typeof(SetReleaseKeyRequest))]
[JsonSerializable(typeof(AboutDto))]
[JsonSerializable(typeof(NodeBuildDto))]
[JsonSerializable(typeof(NodeBinaryDto))]
[JsonSerializable(typeof(NodeOwnerDto))]
[JsonSerializable(typeof(BuildInfoDto))]
[JsonSerializable(typeof(BackendUpdateDto))]
@@ -85,6 +86,8 @@ namespace MoShiro.Backend.Api;
[JsonSerializable(typeof(TrustStateDto))]
[JsonSerializable(typeof(SetupStatusDto))]
[JsonSerializable(typeof(SetupRequest))]
[JsonSerializable(typeof(ComposeInstallDto))]
[JsonSerializable(typeof(QuadletInstallDto))]
[JsonSerializable(typeof(ProposedCredentialDto))]
[JsonSerializable(typeof(PasskeyDto))]
[JsonSerializable(typeof(PageDto<PasskeyDto>))]
@@ -177,9 +177,9 @@ public sealed record HostDto(
long? UptimeSeconds,
DateTimeOffset At);
/// <summary>一个 Agent 的最新主机状态;<c>stale</c> 表示超过 15 秒没有更新(离线,或者 Agent 关闭了上报)。</summary>
/// <summary>一个节点(Agent 或 Probe)的最新主机状态;<c>stale</c> 表示超过 15 秒没有更新(离线,或者节点关闭了上报)。</summary>
/// <summary>节点的展示信息:只有系统、架构、国家代码、运营商,没有任何地址;站点打开「显示构建信息」之后还有版本和 tag。</summary>
public sealed record NodeInfoDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Country, string? IspKey, string? IspName, string? Version = null, string? Tag = null);
public sealed record NodeInfoDto(string Id, string Kind, string? Os, string? OsName, string? Arch, string? Country, string? IspKey, string? IspName, string? Version = null, string? Tag = null, bool IspManual = false);
/// <summary>一个程序的构建信息:版本、tag、提交(短)和构建时间(UTC,ISO 8601)。没设置的是 <c>null</c>。</summary>
public sealed record BuildDto(string Version, string? Tag, string? Commit, string? BuiltAt);
@@ -187,9 +187,10 @@ public sealed record BuildDto(string Version, string? Tag, string? Commit, strin
/// <summary>公开的「关于」:站点没有打开「显示构建信息」时只有 <c>show_build: false</c>。</summary>
public sealed record AboutDto(bool ShowBuild, BuildDto? Backend);
public sealed record HostItemDto(string AgentId, string Name, bool Online, bool Stale, DateTimeOffset? LastSeenAt, HostDto? Host);
/// <param name="Kind"><c>agent</c> 或 <c>probe</c>。</param>
public sealed record HostItemDto(string Id, string Kind, string Name, bool Online, bool Stale, DateTimeOffset? LastSeenAt, HostDto? Host);
public sealed record LiveHostDto(string AgentId, bool Online, HostDto Host);
public sealed record LiveHostDto(string Id, string Kind, bool Online, HostDto Host);
/// <summary>主机资源占用的历史(列式,按步长取平均)。</summary>
public sealed record HostSeriesDto(string AgentId, DateTimeOffset From, DateTimeOffset To, string Step, IReadOnlyList<long> T, IReadOnlyDictionary<string, double?[]> Values);
public sealed record HostSeriesDto(string Id, DateTimeOffset From, DateTimeOffset To, string Step, IReadOnlyList<long> T, IReadOnlyDictionary<string, double?[]> Values);
@@ -6,7 +6,7 @@ namespace MoShiro.Backend.Api;
/// <summary>
/// 一行安装脚本:<c>packaging/install/</c> 下的 <c>install.sh</c> 和 <c>install.ps1</c> 作为嵌入资源打进 Backend,<b>原样返回,不替换任何值</b>。
/// 静态,所以它的 SHA-256 在一个版本内固定:CI 构建时算出来写进签名的发布清单,面板给的命令先核对哈希再执行(docs/access.md 第 15 节)。
/// 静态,所以它的 SHA-256 在一个版本内固定:CI 构建时算出来写进签名的发布清单,面板给的命令先核对哈希再执行(docs/security/access.md 第 10 节)。
/// 角色、面板地址、代码都走命令里的环境变量。
/// </summary>
public static class InstallScripts
@@ -19,6 +19,18 @@ public static class InstallScripts
public static string Script(string resource) => Read(resource);
/// <summary>compose 文件的资源名(<c>agent.yaml</c>、<c>probe.yaml</c>、<c>backend.yaml</c> 对应的);不是这三个就是 null。</summary>
public static string? ComposeResource(string file) => file is "agent.yaml" or "probe.yaml" or "backend.yaml" ? "compose-" + file : null;
/// <summary>某个角色(agent、probe)的 compose 文件的 SHA-256;其他角色返回空。</summary>
public static string ComposeSha256(string? role) => role is "agent" or "probe" ? Sha256("compose-" + role + ".yaml") : string.Empty;
/// <summary>Quadlet 单元的资源名(<c>agent.container</c>、<c>probe.container</c>);不是这两个就是 null。</summary>
public static string? QuadletResource(string file) => file is "agent.container" or "probe.container" ? "quadlet-" + file : null;
/// <summary>某个角色(agent、probe)的 Quadlet 单元的 SHA-256;其他角色返回空。</summary>
public static string QuadletSha256(string? role) => role is "agent" or "probe" ? Sha256("quadlet-" + role + ".container") : string.Empty;
/// <summary>脚本内容的 SHA-256(小写十六进制,按 UTF-8 字节)。</summary>
public static string Sha256(string resource) => Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(Read(resource))));
@@ -10,6 +10,11 @@
<EmbeddedResource Include="..\..\..\packaging\install\install.sh" LogicalName="install.sh" />
<EmbeddedResource Include="..\..\..\packaging\install\install.ps1" LogicalName="install.ps1" />
<EmbeddedResource Include="..\..\..\packaging\update\release-key.pub.pem" LogicalName="release-key.pub.pem" />
<EmbeddedResource Include="..\..\..\packaging\compose\agent\compose.yaml" LogicalName="compose-agent.yaml" />
<EmbeddedResource Include="..\..\..\packaging\compose\probe\compose.yaml" LogicalName="compose-probe.yaml" />
<EmbeddedResource Include="..\..\..\packaging\compose\backend\compose.yaml" LogicalName="compose-backend.yaml" />
<EmbeddedResource Include="..\..\..\packaging\quadlet\moshiro-agent.container" LogicalName="quadlet-agent.container" />
<EmbeddedResource Include="..\..\..\packaging\quadlet\moshiro-probe.container" LogicalName="quadlet-probe.container" />
</ItemGroup>
<!-- 前端构建产物(web 目录下 npm run build)内嵌进 Backend;没构建时不嵌入,Backend 只提供 API -->
<ItemGroup Condition="Exists('..\..\..\web\display\dist\index.html')">
@@ -33,6 +33,15 @@ public static class OpsEndpoints
app.MapGet("/install/manifest.sig", (HttpContext http, JoinTokenService tokens, ReleaseService releases, BackendOptions options, string? code) => ManifestAsync(http, tokens, releases, options, code, signature: true))
.Api("install_manifest_signature", "Base64 DER ECDSA P-256 signature of the manifest", Tag)
.Produces(StatusCodes.Status200OK, contentType: "text/plain").ProducesProblems(404);
// compose 文件:Agent、Probe、Backend 各一份,静态(镜像用 :latest 或 .env 里的 MOSHIRO_VERSION,不含任何一次性的值),
// 面板的命令把参数放在本地的 .env 里,下载后先核对 SHA-256。
app.MapGet("/install/compose/{file}", (HttpContext http, string file) => InstallScripts.ComposeResource(file) is { } resource ? Static(http, resource, "text/yaml") : Problems.Create(http, MoShiroErrors.NotFound, "unknown compose file"))
.Api("install_compose", "The static compose file of agent, probe or backend (agent.yaml, probe.yaml, backend.yaml; same bytes for everyone)", Tag)
.Produces(StatusCodes.Status200OK, contentType: "text/yaml").ProducesProblems(404);
// Podman Quadlet 单元(Linux 上推荐,节点的更新靠 Pull=always):Agent、Probe 各一份,静态,命令里先核对 SHA-256。
app.MapGet("/install/quadlet/{file}", (HttpContext http, string file) => InstallScripts.QuadletResource(file) is { } resource ? Static(http, resource, "text/plain") : Problems.Create(http, MoShiroErrors.NotFound, "unknown quadlet file"))
.Api("install_quadlet", "The static Podman Quadlet unit of agent or probe (agent.container, probe.container; same bytes for everyone)", Tag)
.Produces(StatusCodes.Status200OK, contentType: "text/plain").ProducesProblems(404);
app.MapGet("/install.ps1", (HttpContext http) => Static(http, InstallScripts.Ps1, "text/plain"))
.Api("install_ps1", "The static one-line installer for Windows nodes (same bytes for everyone; its SHA-256 is published with each release)", Tag)
.Produces(StatusCodes.Status200OK, contentType: "text/plain");
@@ -40,9 +49,8 @@ public static class OpsEndpoints
private static async Task<IResult> ManifestAsync(HttpContext http, JoinTokenService tokens, ReleaseService releases, BackendOptions options, string? code, bool signature)
{
string baseUrl = options.PublicUrl ?? $"{http.Request.Scheme}://{http.Request.Host}";
InstallTarget? target = string.IsNullOrEmpty(code) ? null : await tokens.ResolveInstallAsync(code, baseUrl, http.RequestAborted);
ReleaseOffer? offer = target is null ? null : await releases.GetOrFetchAsync(target.Role, http.RequestAborted);
string? role = string.IsNullOrEmpty(code) ? null : await tokens.RoleOfCodeAsync(code, http.RequestAborted);
ReleaseOffer? offer = role is null ? null : await releases.GetOrFetchAsync(role, http.RequestAborted);
if (offer is null)
{
return Problems.Create(http, MoShiroErrors.NotFound, "no signed release is registered for this install code");
@@ -7,22 +7,27 @@ using MoShiro.Protocol;
namespace MoShiro.Backend.Api;
/// <summary>Agent 的主机资源占用:最新值(实时的部分走 SSE 的 <c>hosts</c> 主题)和历史曲线。只有汇总的数字,没有型号和 IP。</summary>
/// <summary>
/// Agent 和 Probe 的主机资源占用:最新值(实时的部分走 SSE 的 <c>hosts</c> 主题)和历史曲线。只有汇总的数字,没有型号和 IP。
/// 和别的公开接口一样:Agent 看得见才给(隐藏的只有登录后才看得见),Probe 没被吊销就给。
/// </summary>
public static class HostEndpoints
{
private const string Tag = "public";
private const string AgentKind = "agent";
private const string ProbeKind = "probe";
public static void Map(RouteGroupBuilder group)
{
group.MapGet("/hosts", ListAsync)
.Api("hosts_list", "Latest host resource usage of every visible agent (no hardware model, no addresses)", Tag)
.Api("hosts_list", "Latest host resource usage of every visible agent and probe (no hardware model, no addresses)", Tag)
.Returns<PageDto<HostItemDto>>().ProducesProblems(403);
group.MapGet("/agents/{agent_id}/host", HistoryAsync)
.Api("hosts_history", "Host resource usage history of one agent (columnar, averaged per step)", Tag)
group.MapGet("/hosts/{node_id}", HistoryAsync)
.Api("hosts_history", "Host resource usage history of one agent or probe (columnar, averaged per step)", Tag)
.Returns<HostSeriesDto>().ProducesProblems(403, 404, 422);
}
private static async Task<IResult> ListAsync(HttpContext http, AgentService agents, HostStatusService hosts, CancellationToken ct)
private static async Task<IResult> ListAsync(HttpContext http, AgentService agents, ProbeService probes, HostStatusService hosts, CancellationToken ct)
{
var items = new List<HostItemDto>();
foreach (AgentView view in await agents.ListAsync(ct))
@@ -33,18 +38,39 @@ public static class HostEndpoints
}
(HostSnapshot? host, bool stale) = hosts.Latest(view.Agent.Id);
items.Add(new HostItemDto(view.Agent.Id.Value, view.Agent.Name, view.Online, stale, Mappers.Time(view.Agent.LastSeenAtMs), host is null ? null : Mappers.ToDto(host)));
items.Add(new HostItemDto(view.Agent.Id.Value, AgentKind, view.Agent.Name, view.Online, stale, Mappers.Time(view.Agent.LastSeenAtMs), host is null ? null : Mappers.ToDto(host)));
}
foreach (ProbeView view in await probes.ListAsync(ct))
{
if (view.Revoked)
{
continue;
}
(HostSnapshot? host, bool stale) = hosts.Latest(new NodeId(view.Probe.Id));
items.Add(new HostItemDto(view.Probe.Id, ProbeKind, view.Probe.Name, view.Online, stale, host is null ? null : Mappers.Time(host.AtUtcMs), host is null ? null : Mappers.ToDto(host)));
}
return TypedResults.Json(new PageDto<HostItemDto>(items, null), ApiJsonContext.Default.PageDtoHostItemDto);
}
private static async Task<IResult> HistoryAsync(HttpContext http, AgentService agents, HostStatusService hosts, TimeProvider time, string agent_id, string? range, CancellationToken ct)
private static async Task<IResult> HistoryAsync(HttpContext http, AgentService agents, ProbeService probes, HostStatusService hosts, TimeProvider time, string node_id, string? range, CancellationToken ct)
{
AgentView? view = await agents.GetAsync(new NodeId(agent_id), ct);
if (view is not { Agent.Status: Fleet.Server.NodeStatus.Active } || !(view.Agent.PublicVisible || PublicGuard.CanSeeHidden(http)))
bool visible;
if (node_id.StartsWith(AgentIds.Prefix, StringComparison.Ordinal))
{
return Problems.Create(http, MoShiroErrors.AgentNotFound, "agent not found");
AgentView? view = await agents.GetAsync(new NodeId(node_id), ct);
visible = view is { Agent.Status: Fleet.Server.NodeStatus.Active } && (view.Agent.PublicVisible || PublicGuard.CanSeeHidden(http));
}
else
{
visible = node_id.StartsWith(ProbeId.Prefix, StringComparison.Ordinal) && await probes.GetAsync(node_id, ct) is { Revoked: false };
}
if (!visible)
{
return Problems.Create(http, MoShiroErrors.AgentNotFound, "node not found");
}
if (!RangeResolver.TryParseSpan(range ?? "1h", out long span) || span < Steps.Hour || span > 30 * Steps.Day)
@@ -54,7 +80,7 @@ public static class HostEndpoints
long now = time.GetUtcNow().ToUnixTimeMilliseconds();
long from = now - span;
IReadOnlyList<HostSample> samples = await hosts.HistoryAsync(view.Agent.Id, from, now + 60_000, ct);
IReadOnlyList<HostSample> samples = await hosts.HistoryAsync(new NodeId(node_id), from, now + 60_000, ct);
// 最多约 300 个点:按分钟的整数倍取步长,每个步长内取平均
long stepMs = Math.Max(60_000, (long)Math.Ceiling(span / 300.0 / 60_000) * 60_000);
@@ -92,6 +118,6 @@ public static class HostEndpoints
["rx_bps"] = Average(s => s.NetRxBps),
["tx_bps"] = Average(s => s.NetTxBps),
};
return TypedResults.Json(new HostSeriesDto(view.Agent.Id.Value, Mappers.Time(first), Mappers.Time(now), Steps.Format(stepMs), t, values), ApiJsonContext.Default.HostSeriesDto);
return TypedResults.Json(new HostSeriesDto(node_id, Mappers.Time(first), Mappers.Time(now), Steps.Format(stepMs), t, values), ApiJsonContext.Default.HostSeriesDto);
}
}
@@ -32,7 +32,7 @@ public static class LiveEndpoints
}
private static async Task<IResult> StreamAsync(
HttpContext http, AgentService agents, LineService lineService, TestService tests, QueryService query, LiveHub hub, SseLimiter limiter,
HttpContext http, AgentService agents, ProbeService probes, HostStatusService hosts, LineService lineService, TestService tests, QueryService query, LiveHub hub, SseLimiter limiter,
BackendOptions options, BackendCounters counters, TimeProvider time, IHostApplicationLifetime lifetime, string? ip_version)
{
string ipVersion = ip_version ?? IpVersions.V4;
@@ -84,7 +84,7 @@ public static class LiveEndpoints
http.Response.Headers["X-Accel-Buffering"] = "no";
await http.Response.Body.FlushAsync(http.RequestAborted);
var session = new LiveSession(http, agents, lineService, tests, query, hub, options, time, topics, lineKeys, ipVersion);
var session = new LiveSession(http, agents, probes, hosts, lineService, tests, query, hub, options, time, topics, lineKeys, ipVersion);
using var linked = CancellationTokenSource.CreateLinkedTokenSource(http.RequestAborted, lifetime.ApplicationStopping);
await session.RunAsync(linked.Token);
}
@@ -105,6 +105,8 @@ public static class LiveEndpoints
{
private readonly HttpContext _http;
private readonly AgentService _agents;
private readonly ProbeService _probes;
private readonly HostStatusService _hosts;
private readonly LineService _lines;
private readonly IReadOnlyDictionary<string, LineKey> _lineKeys;
private readonly string _ipVersion;
@@ -124,10 +126,12 @@ public static class LiveEndpoints
private long _lastWriteMs;
private long _lastPeriodicMs;
public LiveSession(HttpContext http, AgentService agents, LineService lines, TestService tests, QueryService query, LiveHub hub, BackendOptions options, TimeProvider time, IReadOnlyList<LiveTopic> topics, IReadOnlyDictionary<string, LineKey> lineKeys, string ipVersion)
public LiveSession(HttpContext http, AgentService agents, ProbeService probes, HostStatusService hosts, LineService lines, TestService tests, QueryService query, LiveHub hub, BackendOptions options, TimeProvider time, IReadOnlyList<LiveTopic> topics, IReadOnlyDictionary<string, LineKey> lineKeys, string ipVersion)
{
_http = http;
_agents = agents;
_probes = probes;
_hosts = hosts;
_lines = lines;
_lineKeys = lineKeys;
_ipVersion = ipVersion;
@@ -171,7 +175,7 @@ public static class LiveEndpoints
_nodeSignals.Enqueue(node);
break;
case HostSignal host when _topics.Any(t => t.Kind == LiveTopicKind.Hosts):
_hostSignals[host.AgentId] = host;
_hostSignals[host.Node] = host;
break;
case ChunkSignal chunk:
_dirtyLines.Add((chunk.AgentId, chunk.Series));
@@ -216,10 +220,20 @@ public static class LiveEndpoints
foreach (HostSignal host in hostSignals)
{
AgentView? view = await _agents.GetAsync(host.AgentId, ct);
if (host.Node.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal))
{
if (await _probes.GetAsync(host.Node.Value, ct) is { Revoked: false } probe)
{
await WriteAsync("host", new LiveHostDto(host.Node.Value, "probe", probe.Online, Mappers.ToDto(host.Host)), ApiJsonContext.Default.LiveHostDto, ct);
}
continue;
}
AgentView? view = await _agents.GetAsync(host.Node, ct);
if (view is not null && IsVisible(view) && view.Agent.Status == Fleet.Server.NodeStatus.Active)
{
await WriteAsync("host", new LiveHostDto(host.AgentId.Value, view.Online, Mappers.ToDto(host.Host)), ApiJsonContext.Default.LiveHostDto, ct);
await WriteAsync("host", new LiveHostDto(host.Node.Value, "agent", view.Online, Mappers.ToDto(host.Host)), ApiJsonContext.Default.LiveHostDto, ct);
}
}
@@ -284,7 +298,15 @@ public static class LiveEndpoints
case LiveTopicKind.Hosts:
foreach (AgentView view in (await _agents.ListAsync(ct)).Where(v => v.Agent.Status == Fleet.Server.NodeStatus.Active && IsVisible(v) && v.Host is not null))
{
await WriteAsync("host", new LiveHostDto(view.Agent.Id.Value, view.Online, Mappers.ToDto(view.Host!)), ApiJsonContext.Default.LiveHostDto, ct);
await WriteAsync("host", new LiveHostDto(view.Agent.Id.Value, "agent", view.Online, Mappers.ToDto(view.Host!)), ApiJsonContext.Default.LiveHostDto, ct);
}
foreach (ProbeView view in await _probes.ListAsync(ct))
{
if (!view.Revoked && _hosts.Latest(new NodeId(view.Probe.Id)).Host is { } host)
{
await WriteAsync("host", new LiveHostDto(view.Probe.Id, "probe", view.Online, Mappers.ToDto(host)), ApiJsonContext.Default.LiveHostDto, ct);
}
}
break;
@@ -61,7 +61,7 @@ public static class NodeEndpoints
var items = (await nodes.ListAsync(ct))
.Where(n => visible.Contains(n.NodeId))
.OrderBy(n => n.NodeId, StringComparer.Ordinal)
.Select(n => new NodeInfoDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Country, n.IspKey, n.IspName, showBuild ? n.Version : null, showBuild ? n.Tag : null))
.Select(n => new NodeInfoDto(n.NodeId, n.Kind, n.Os, n.OsName, n.Arch, n.Country, n.IspKey, n.IspName, showBuild ? n.Version : null, showBuild ? n.Tag : null, n.IspManual))
.ToList();
http.Response.Headers.CacheControl = "no-store";
return TypedResults.Json(new PageDto<NodeInfoDto>(items, null), ApiJsonContext.Default.PageDtoNodeInfoDto);
@@ -14,12 +14,23 @@ public sealed record InstallCommands(string Linux, string Macos, string Windows)
/// 安装命令里除了一次性的码,还带的公开值:信任根的创世指纹(节点把它固定下来)、属主(允许授权这个节点 extension 的 passkey 凭证 id)、
/// 和面板提供的静态安装脚本的 SHA-256(命令先核对哈希再执行)。没有建立信任根时 <c>Genesis</c> 为空,节点不固定信任根。
/// </summary>
public sealed record InstallPlan(string? Genesis, IReadOnlyList<string> Controllers, string ScriptShSha256, string ScriptPs1Sha256);
/// <param name="ComposeSha256">面板提供的这个角色的 compose 文件的 SHA-256(命令下载之后先核对)。</param>
/// <param name="QuadletSha256">面板提供的这个角色的 Podman Quadlet 单元的 SHA-256。</param>
public sealed record InstallPlan(string? Genesis, IReadOnlyList<string> Controllers, string ScriptShSha256, string ScriptPs1Sha256, string ComposeSha256 = "", string QuadletSha256 = "");
public sealed record CreatedJoinToken(JoinTokenInfo Info, string Plaintext, InstallCommands InstallCommands);
/// <summary>
/// compose 安装:<c>Env</c> 是要存成 <c>.env</c> 的内容(只有公开的值),<c>Url</c> 和 <c>Sha256</c> 是 compose 文件的下载地址和哈希,
/// 三条命令(Linux 用 Podman,macOS、Windows 用 Docker Desktop)把 .env、下载、核对哈希和启动合在一起;Agent 的 compose 需要 Linux 宿主网络,所以它的 macOS、Windows 命令是空的。
/// </summary>
public sealed record ComposeInstall(string Env, string Url, string Sha256, string Linux, string Macos, string Windows);
/// <summary>安装脚本里要填的四个值。</summary>
public sealed record InstallTarget(string Role, string Server, string Code, string Image);
/// <summary>
/// Podman Quadlet 安装(Linux 上推荐):<c>Env</c> 存成 <c>~/.config/moshiro/&lt;角色&gt;.env</c>,<c>Url</c> 和 <c>Sha256</c> 是 Quadlet 单元的下载地址和哈希,
/// <c>Command</c> 把 env、单元、镜像仓库(drop-in)、核对哈希和启动合在一起。单元写了 <c>Pull=always</c>,节点退出再被拉起就是更新,默认自动更新。
/// </summary>
public sealed record QuadletInstall(string Env, string Url, string Sha256, string Command);
public sealed record CreatedJoinToken(JoinTokenInfo Info, string Plaintext, InstallCommands InstallCommands, ComposeInstall Compose, QuadletInstall Quadlet);
/// <summary>加入令牌:明文只在创建时返回一次,服务端只保存 SHA-256。</summary>
public sealed class JoinTokenService
@@ -82,7 +93,7 @@ public sealed class JoinTokenService
Ulid.NewPrefixed("jtk", _time), TokenHash.Sha256(plaintext), request.Note, request.Labels ?? [], request.AutoApprove, KindOf(request.Role),
request.MaxUses, now + (request.ExpiresInHours * 3_600_000L), now, request.CreatedBy, ct);
return Result<CreatedJoinToken>.Ok(new CreatedJoinToken(info, plaintext, Commands(BaseUrl(requestBaseUrl), request.Role, plaintext, plan)));
return Result<CreatedJoinToken>.Ok(new CreatedJoinToken(info, plaintext, Commands(BaseUrl(requestBaseUrl), request.Role, plaintext, plan), Compose(BaseUrl(requestBaseUrl), request.Role, plaintext, plan), Quadlet(BaseUrl(requestBaseUrl), request.Role, plaintext, plan)));
}
/// <summary>
@@ -101,13 +112,95 @@ public sealed class JoinTokenService
+ $"$env:MOSHIRO_SERVER='{baseUrl}'; $env:MOSHIRO_ROLE='{role}'; $env:MOSHIRO_CODE='{code}'; {psPin}iex (Get-Content -Raw $f)");
}
/// <summary>
/// 把一次性码和公开的值写进本地的 <c>.env</c>,下载静态的 compose 文件、核对哈希、<c>compose up -d</c>:以后更新就是
/// <c>compose pull &amp;&amp; compose up -d</c>。镜像来自 CI 渠道(<c>:latest</c>,也可以在 .env 里固定版本),不由面板决定。
/// </summary>
public ComposeInstall Compose(string baseUrl, string role, string code, InstallPlan plan)
{
string url = $"{baseUrl}/install/compose/{role}.yaml";
string dir = $"moshiro-{role}";
string Env(bool desktop)
{
var lines = new List<string> { $"MOSHIRO_SERVER={baseUrl}", $"MOSHIRO_JOIN={code}" };
if (plan.Genesis is null)
{
lines.Add("MOSHIRO_ALLOW_NO_OWNER=1");
}
else
{
lines.Add($"MOSHIRO_GENESIS={plan.Genesis}");
lines.Add($"MOSHIRO_CONTROLLERS={string.Join(',', plan.Controllers)}");
}
lines.Add($"MOSHIRO_IMAGE={ImageRepository(role)}");
if (desktop)
{
// Docker Desktop 没有宿主网络
lines.Add("MOSHIRO_NETWORK_MODE=bridge");
lines.Add("MOSHIRO_NETWORK_SCOPE=container");
}
return string.Join('\n', lines);
}
string linux = $"mkdir -p {dir} && cd {dir} && cat > .env <<'EOF'\n{Env(false)}\nEOF\n"
+ $"chmod 600 .env && curl -fsSL '{url}' -o compose.yaml && echo \"{plan.ComposeSha256} compose.yaml\" | sha256sum -c - && podman compose up -d";
string macos = role == ProbeRole
? $"mkdir -p {dir} && cd {dir} && cat > .env <<'EOF'\n{Env(true)}\nEOF\n"
+ $"chmod 600 .env && curl -fsSL '{url}' -o compose.yaml && echo \"{plan.ComposeSha256} compose.yaml\" | shasum -a 256 -c - && docker compose up -d"
: "";
string windows = role == ProbeRole
? $"New-Item -ItemType Directory -Force {dir} | Out-Null; Set-Location {dir}\n@'\n{Env(true)}\n'@ | Set-Content -Encoding ascii .env\n"
+ $"irm '{url}' -OutFile compose.yaml; if ((Get-FileHash compose.yaml -Algorithm SHA256).Hash -ne '{plan.ComposeSha256}') {{ throw 'compose.yaml does not match its published hash' }}\ndocker compose up -d"
: "";
return new ComposeInstall(Env(false), url, plan.ComposeSha256, linux, macos, windows);
}
/// <summary>
/// Podman Quadlet 安装:env 文件、单元文件(静态,核对哈希)、一个把镜像仓库指向面板配置的 drop-in,然后启动用户服务。
/// 自动更新默认开(<c>MOSHIRO_UPDATE_POLICY</c> 缺省是 auto):env 里留了一行注释,去掉 # 就改成在管理端手动点更新。
/// </summary>
public QuadletInstall Quadlet(string baseUrl, string role, string code, InstallPlan plan)
{
string url = $"{baseUrl}/install/quadlet/{role}.container";
var lines = new List<string> { $"MOSHIRO_SERVER={baseUrl}", $"MOSHIRO_JOIN={code}" };
if (plan.Genesis is null)
{
lines.Add("MOSHIRO_ALLOW_NO_OWNER=1");
}
else
{
lines.Add($"MOSHIRO_GENESIS={plan.Genesis}");
lines.Add($"MOSHIRO_CONTROLLERS={string.Join(',', plan.Controllers)}");
}
lines.Add("# 自动更新默认开:面板发现新版本后,节点运行满 10 分钟就自动重启,Podman 启动时拉最新镜像。想改成在管理端手动点「更新」,去掉下一行的 #");
lines.Add("# MOSHIRO_UPDATE_POLICY=manual");
string env = string.Join('\n', lines);
string unit = $"$HOME/.config/containers/systemd/moshiro-{role}.container";
string command = $"mkdir -p ~/.config/moshiro ~/.config/containers/systemd/moshiro-{role}.container.d && cat > ~/.config/moshiro/{role}.env <<'EOF'\n{env}\nEOF\n"
+ $"chmod 600 ~/.config/moshiro/{role}.env && curl -fsSL '{url}' -o \"{unit}\" && echo \"{plan.QuadletSha256} {unit}\" | sha256sum -c - "
+ $"&& printf '[Container]\\nImage={ImageRepository(role)}:latest\\n' > ~/.config/containers/systemd/moshiro-{role}.container.d/10-image.conf "
+ $"&& systemctl --user daemon-reload && systemctl --user start moshiro-{role}";
return new QuadletInstall(env, url, plan.QuadletSha256, command);
}
/// <summary>这个角色的镜像仓库地址(不带 tag):节点镜像仓库加上命名空间。</summary>
private string ImageRepository(string role)
{
string registry = (_options.NodeImageRegistry ?? "").Trim('/');
string ns = _options.ImageNamespace.Trim('/');
return $"{(registry.Length > 0 ? registry + "/" : "")}{(ns.Length > 0 ? ns : "moshiro")}/moshiro-{role}";
}
private string BaseUrl(string? requestBaseUrl) => (_options.PublicUrl ?? requestBaseUrl ?? "https://panel.example.com").TrimEnd('/');
/// <summary>
/// 安装脚本要填的值:由代码找到令牌(不消耗),角色、网关、镜像都在这里决定,Agent 和 Probe 共用同一份脚本。
/// 代码不存在、已吊销、已过期或用完了返回 null。网关和镜像要是含有会破坏脚本引号的字符也返回 null。
/// 这个一次性码是给哪个角色(agent 或 probe)的:安装脚本据此向面板取该角色已签名的发布清单。
/// 代码不存在、已吊销、已过期或用完了返回 null(不消耗它)。
/// </summary>
public async ValueTask<InstallTarget?> ResolveInstallAsync(string code, string? requestBaseUrl, CancellationToken ct)
public async ValueTask<string?> RoleOfCodeAsync(string code, CancellationToken ct)
{
if (code.Length is 0 or > 200)
{
@@ -115,22 +208,11 @@ public sealed class JoinTokenService
}
JoinTokenInfo? token = await _store.FindAsync(TokenHash.Sha256(code), ct);
if (token is null || token.RevokedAtMs is not null || token.ExpiresAtMs <= _time.GetUtcNow().ToUnixTimeMilliseconds() || token.Uses >= token.MaxUses)
{
return null;
}
string role = RoleOf(token.Kind);
string registry = (_options.NodeImageRegistry ?? "").Trim('/');
string ns = _options.ImageNamespace.Trim('/');
string image = $"{(registry.Length > 0 ? registry + "/" : "")}{(ns.Length > 0 ? ns : "moshiro")}/moshiro-{role}:latest";
string server = BaseUrl(requestBaseUrl);
return IsSafe(server) && IsSafe(image) && IsSafe(code) ? new InstallTarget(role, server, code, image) : null;
return token is null || token.RevokedAtMs is not null || token.ExpiresAtMs <= _time.GetUtcNow().ToUnixTimeMilliseconds() || token.Uses >= token.MaxUses
? null
: RoleOf(token.Kind);
}
/// <summary>值会被放进脚本的单引号里:只允许 URL 和镜像名里正常出现的字符。</summary>
private static bool IsSafe(string value) => value.All(c => char.IsAsciiLetterOrDigit(c) || c is '.' or '-' or '_' or ':' or '/' or '[' or ']' or '~' or '%');
public ValueTask<IReadOnlyList<JoinTokenInfo>> ListAsync(CancellationToken ct) => _store.ListAsync(ct);
public ValueTask<bool> RevokeAsync(string id, CancellationToken ct) => _store.RevokeAsync(id, _time.GetUtcNow().ToUnixTimeMilliseconds(), ct);
@@ -26,12 +26,10 @@ public sealed partial class AgentService
private readonly LinkService _links;
private readonly TrustService _trust;
private readonly BinaryBindingService _bindings;
private readonly NodeOwnerService _owners;
public AgentService(BinaryBindingService bindings, NodeOwnerService owners, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TaskPlanner planner, ISettingsStore settings, NodeInfoService nodes, LinkService links, TrustService trust)
public AgentService(NodeOwnerService owners, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TaskPlanner planner, ISettingsStore settings, NodeInfoService nodes, LinkService links, TrustService trust)
{
_bindings = bindings;
_owners = owners;
_trust = trust;
_links = links;
@@ -152,7 +150,6 @@ public sealed partial class AgentService
await _gateway.DisconnectAsync(id, "deleted", ct);
await _agents.DeleteAsync(id, ct);
await _nodes.ForgetAsync(id.Value, ct);
await _bindings.ForgetAsync(id.Value, ct);
_owners.Forget(id.Value);
await _links.ForgetNodeAsync(id.Value, ct);
_runtime.Remove(id);
@@ -1,175 +0,0 @@
using System.Text.Json;
using Fleet.Core;
using Fleet.Server;
namespace MoShiro.Backend.Application;
/// <summary>CI 发布公钥(PEM):验节点握手时带来的签名清单。由 Api 层提供(它就是内嵌在程序里的那一把)。</summary>
public interface IReleaseKey
{
string Pem { get; }
}
/// <summary>
/// 节点二进制的绑定:节点第一次握手成功的那一刻,把它正在运行的二进制的 SHA-256 和这个节点绑在一起;之后哈希对不上,就不让它继续运行。
/// <para>
/// 合法的升级:新哈希出现在一份 <b>CI 签名的清单</b>里(节点握手时带着宿主机更新器验过的清单,这里用内嵌的 CI 公钥再验一遍,
/// 所以不依赖面板有没有收到过这次发布),并且清单的 seq 不低于绑定时的——不许回退到更旧的版本。其余情况(文件被换掉、降级、没人签过的二进制、
/// 手工替换了二进制的非容器节点)都算不符;管理员确认过之后可以清除绑定,下一次握手重新绑定。
/// </para>
/// <para>
/// 边界:哈希是节点自己报的。一个被换进来、故意撒谎的恶意二进制可以照抄原来的哈希;这一层抓的是文件被换、版本被改、降级和起了不对的二进制。
/// 容器节点要堵住撒谎,需要宿主机更新器背书(容器之外),见 docs/access.md。
/// </para>
/// </summary>
public sealed class BinaryBindingService : IBinaryAttestor
{
public const string Enforce = "enforce";
public const string Warn = "warn";
public const string Off = "off";
private readonly INodeBindingStore _store;
private readonly ISettingsStore _settings;
private readonly IReleaseKey _key;
private readonly BackendOptions _options;
private readonly TimeProvider _time;
public BinaryBindingService(INodeBindingStore store, ISettingsStore settings, IReleaseKey key, BackendOptions options, TimeProvider time)
{
_settings = settings;
_store = store;
_key = key;
_options = options;
_time = time;
}
public ValueTask<IReadOnlyList<NodeBinding>> ListAsync(CancellationToken ct) => _store.ListAsync(ct);
/// <summary>管理员确认新的二进制是自己要的(比如手工升级了非容器节点):清除绑定,下一次握手重新绑定。</summary>
public ValueTask<bool> ClearAsync(string nodeId, CancellationToken ct) => _store.DeleteAsync(nodeId, ct);
/// <summary>节点删除之后顺手清掉它的绑定。</summary>
public async ValueTask ForgetAsync(string nodeId, CancellationToken ct) => await _store.DeleteAsync(nodeId, ct);
public async ValueTask<BinaryDecision> CheckAsync(NodeRecord node, BinaryEvidence evidence, CancellationToken ct)
{
string policy = _options.BinaryPolicy;
if (policy == Off || ReleaseService.ComponentOf(node.Id) is not { } component)
{
return new BinaryDecision(true);
}
long now = _time.GetUtcNow().ToUnixTimeMilliseconds();
NodeBinding? binding = await _store.GetAsync(node.Id.Value, ct);
if (evidence.Sha256 is null)
{
// 旧节点不上报:还没绑定过就放行(未绑定);绑定过的不许退回去,否则换个旧版本就绕过了
return binding is null ? new BinaryDecision(true) : await ViolationAsync(binding, "the node stopped reporting its binary hash", policy, now, ct);
}
string hash = evidence.Sha256;
if (!IsHash(hash))
{
return new BinaryDecision(policy != Enforce, "the reported binary hash is malformed");
}
// 节点自己带来的清单(宿主机更新器验过的那份);没有更新器的节点(一键容器安装、二进制安装)没有,就用面板自己手里的:
// 面板登记它们时已经验过 CI 签名,这里再验一遍。只有节点自带的清单才能因为「不在清单里」而被拒绝,面板手里的只用来把绑定标成已签名。
Official? official = VerifyManifest(evidence.Manifest, evidence.Signature, component);
bool carried = official is not null;
official ??= await StoredOfficialAsync(component, ct);
bool listed = official is not null && official.Binaries.Contains(hash);
if (binding is null)
{
if (carried && !listed)
{
// 带着一份有效的签名清单,自己的哈希却不在里面:它不是这份清单对应的官方二进制
return new BinaryDecision(policy != Enforce, "this binary is not in the CI-signed manifest it was installed from");
}
await _store.UpsertAsync(new NodeBinding(node.Id.Value, hash, listed ? official!.Seq : 0, listed, now, null, null), ct);
return new BinaryDecision(true);
}
if (binding.Sha256 == hash)
{
if (listed && (official!.Seq > binding.Seq || !binding.Signed || binding.Violation is not null))
{
await _store.UpsertAsync(binding with { Seq = Math.Max(binding.Seq, official!.Seq), Signed = true, Violation = null, ViolationAtMs = null }, ct);
}
else if (binding.Violation is not null)
{
await _store.UpsertAsync(binding with { Violation = null, ViolationAtMs = null }, ct);
}
return new BinaryDecision(true);
}
// 哈希变了:必须是一份不低于绑定时 seq 的 CI 签名清单里的官方二进制(合法升级),否则不符
if (listed && official!.Seq >= binding.Seq)
{
await _store.UpsertAsync(new NodeBinding(node.Id.Value, hash, official.Seq, true, now, null, null), ct);
return new BinaryDecision(true);
}
return await ViolationAsync(binding, $"the running binary changed (bound {Short(binding.Sha256)}, now {Short(hash)}) and is not in a newer CI-signed manifest", policy, now, ct);
}
private async ValueTask<BinaryDecision> ViolationAsync(NodeBinding binding, string reason, string policy, long now, CancellationToken ct)
{
await _store.UpsertAsync(binding with { Violation = reason, ViolationAtMs = now }, ct);
return new BinaryDecision(policy != Enforce, reason);
}
private sealed record Official(long Seq, HashSet<string> Binaries);
/// <summary>用内嵌的 CI 公钥验清单;验过且是这个组件的才返回它列出的官方二进制哈希。</summary>
private async ValueTask<Official?> StoredOfficialAsync(string component, CancellationToken ct) =>
VerifyManifest(await _settings.GetAsync(ReleaseService.Key(component, "manifest"), ct), await _settings.GetAsync(ReleaseService.Key(component, "signature"), ct), component);
private Official? VerifyManifest(string? manifestText, string? signatureText, string component)
{
if (manifestText is not { Length: > 0 and <= ReleaseService.MaxManifestBytes } manifest
|| signatureText is not { Length: > 0 and <= ReleaseService.MaxSignatureBytes } signature
|| !ReleaseSignature.Verify(_key.Pem, manifest, signature))
{
return null;
}
try
{
using JsonDocument document = JsonDocument.Parse(manifest);
JsonElement root = document.RootElement;
if (root.ValueKind != JsonValueKind.Object
|| !root.TryGetProperty("component", out JsonElement c) || c.GetString() != component
|| !root.TryGetProperty("seq", out JsonElement s) || !s.TryGetInt64(out long seq) || seq <= 0)
{
return null;
}
var binaries = new HashSet<string>(StringComparer.Ordinal);
if (root.TryGetProperty("binaries", out JsonElement list) && list.ValueKind == JsonValueKind.String)
{
foreach (string item in (list.GetString() ?? "").Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
if (IsHash(item))
{
binaries.Add(item);
}
}
}
return new Official(seq, binaries);
}
catch (JsonException)
{
return null;
}
}
private static bool IsHash(string value) => value.Length == 64 && value.All(ch => ch is (>= '0' and <= '9') or (>= 'a' and <= 'f'));
private static string Short(string hash) => hash.Length > 12 ? hash[..12] : hash;
}
@@ -4,8 +4,8 @@ using MoShiro.Protocol;
namespace MoShiro.Backend.Application;
/// <summary>
/// <c>moshiro.host</c>:Agent 每隔几秒上报的主机资源占用快照。最新值放在内存里并实时推给订阅者;每个 Agent 每分钟存一个样本做历史。
/// 快照里没有 CPU 型号、主机名、网卡名和 IP(Agent 根本不采集它们)。
/// <c>moshiro.host</c>:Agent 和 Probe 每隔几秒上报的主机资源占用快照。最新值放在内存里并实时推给订阅者;每个节点每分钟存一个样本做历史。
/// 快照里没有 CPU 型号、主机名、网卡名和 IP(节点根本不采集它们)。
/// </summary>
public sealed class HostStatusService
{
@@ -29,12 +29,10 @@ public sealed partial class ProbeService
private readonly NodeInfoService _nodes;
private readonly LinkService _links;
private readonly BinaryBindingService _bindings;
private readonly NodeOwnerService _owners;
public ProbeService(BinaryBindingService bindings, NodeOwnerService owners, IProbeStore probes, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TimeProvider time, NodeInfoService nodes, LinkService links)
public ProbeService(NodeOwnerService owners, IProbeStore probes, IAgentStore agents, AgentRuntime runtime, IAgentGateway gateway, TimeProvider time, NodeInfoService nodes, LinkService links)
{
_bindings = bindings;
_owners = owners;
_links = links;
_nodes = nodes;
@@ -99,7 +97,6 @@ public sealed partial class ProbeService
if (deleted)
{
await _nodes.ForgetAsync(id, ct);
await _bindings.ForgetAsync(id, ct);
_owners.Forget(id);
await _links.ForgetNodeAsync(id, ct);
_gateway.NotifyAllTaskSetsChanged();
@@ -40,9 +40,8 @@ public static class ApplicationServiceCollectionExtensions
services.AddSingleton<ServerIdentityService>();
services.AddSingleton<SetupService>();
services.AddSingleton<ReleaseFeed>();
services.AddSingleton<BinaryBindingService>();
services.AddSingleton<ReleaseKeyService>();
services.AddSingleton<NodeOwnerService>();
services.AddSingleton<Fleet.Server.IBinaryAttestor>(sp => sp.GetRequiredService<BinaryBindingService>());
services.AddSingleton<Fleet.Server.IServerIdentity>(sp => sp.GetRequiredService<ServerIdentityService>());
services.AddSingleton<TrustService>();
services.AddSingleton<Fleet.Server.IOperatorTrustSource>(sp => sp.GetRequiredService<TrustService>());
@@ -72,6 +71,7 @@ public static class ApplicationServiceCollectionExtensions
services.AddHostedService(provider => provider.GetRequiredService<TestFinalizer>());
services.AddHostedService<HotLayerBackfill>();
services.AddHostedService<RetentionService>();
services.AddHostedService<ReleasePoller>();
return services;
}
}
@@ -2,7 +2,7 @@ namespace MoShiro.Backend.Application;
/// <summary>
/// 能力(capability):一项具体的权限。端点声明自己需要哪一项,<see cref="Principal.Has"/> 是唯一的判断入口,
/// 会话、API key 都走它。设计见 docs/access.md 第 2 节。
/// 会话、API key 都走它。设计见 docs/security/access.md 第 3 节。
/// </summary>
public static class Capability
{
@@ -27,7 +27,6 @@ public static class Capability
public const string UpdateRead = "update.read";
public const string UpdateConfirm = "update.confirm";
public const string ReleasePublish = "release.publish";
public const string KeysRead = "keys.read";
public const string KeysManage = "keys.manage";
@@ -39,7 +38,7 @@ public static class Capability
public const string EventsRead = "events.read";
public const string OpsMetrics = "ops.metrics";
// 预留:外部 webhook 回调和数据拉取(docs/access.md 5.4),目录里先存在,实现放后面
// 预留:外部 webhook 回调和数据拉取(docs/security/access.md 6.3),目录里先存在,实现放后面
public const string DataExport = "data.export";
public const string WebhooksRead = "webhooks.read";
public const string WebhooksManage = "webhooks.manage";
@@ -74,7 +73,6 @@ public static class CapabilityCatalog
new(Capability.UpdateRead, "update"),
new(Capability.UpdateConfirm, "update", false, false, Capability.UpdateRead),
new(Capability.ReleasePublish, "update"),
new(Capability.KeysRead, "keys"),
new(Capability.KeysManage, "keys", false, false, Capability.KeysRead),
@@ -99,14 +97,11 @@ public static class CapabilityCatalog
new("read-only", [.. All.Where(c => !c.Reserved && c.Id.EndsWith(".read", StringComparison.Ordinal)).Select(c => c.Id)]),
new("operations", Expand([Capability.PublicRead, Capability.FleetWrite, Capability.TasksWrite, Capability.DataWrite, Capability.SettingsWrite, Capability.UpdateConfirm, Capability.EventsRead, Capability.AuditRead])),
new("enroll", Expand([Capability.FleetEnroll])),
new("ci-release", [Capability.ReleasePublish]),
new("all", [.. All.Select(c => c.Id)]),
];
public static bool IsKnown(string id) => s_byId.ContainsKey(id);
public static CapabilityInfo? Find(string id) => s_byId.GetValueOrDefault(id);
/// <summary>补上蕴含的能力(写蕴含读),去重并排序,结果稳定。</summary>
public static IReadOnlyList<string> Expand(IEnumerable<string> ids)
{
@@ -22,7 +22,7 @@ public sealed record PasskeyAssertion(string CredentialId, string AuthenticatorD
/// <summary>
/// passkey 的注册与登录(WebAuthn,ES256,attestation 当 none)。挑战只在内存里,一次有效、5 分钟过期;
/// 登录是「可发现凭证」:不需要先输用户名,凭证 id 找到用户。验证断言的纯函数在 <see cref="WebAuthnAssertion"/>(节点以后也用)。
/// 设计见 docs/access.md 第 3 节。
/// 设计见 docs/security/access.md 第 4 节。
/// </summary>
public sealed class PasskeyService
{
@@ -20,7 +20,7 @@ public sealed record TrustState(bool Initialized, string? Genesis, int Version,
/// <summary>
/// 信任根的 Backend 一侧:保存操作员集合的链、给操作员出下一版的语句、验证他签回来的版本。
/// Backend <b>不持有任何签名的能力</b>:每一版都要操作员的 passkey 触碰;节点从安装命令里固定的创世指纹出发自己验链,
/// 所以 Backend 在这里的验证只是不收垃圾,不是信任的来源。设计见 docs/access.md 第 7 节。
/// 所以 Backend 在这里的验证只是不收垃圾,不是信任的来源。设计见 docs/security/access.md 第 8 节。
/// </summary>
public sealed class TrustService : IOperatorTrustSource, IDisposable
{
@@ -9,7 +9,7 @@ public sealed record RegisteredCredential(byte[] CredentialId, byte[] PublicKeyS
/// <summary>
/// 解析并校验 WebAuthn 注册(<c>navigator.credentials.create</c> 的结果)。只接受 ES256;attestation 一律当 <c>none</c> 处理:
/// 不验设备证明,信任浏览器的实现(docs/access.md)。CBOR 解析只在 Backend 里,节点不带。
/// 不验设备证明,信任浏览器的实现(docs/security/access.md)。CBOR 解析只在 Backend 里,节点不带。
/// </summary>
public static class WebAuthnRegistration
{
@@ -21,11 +21,6 @@ public sealed class BackendOptions
/// <summary>宿主机更新器的只读状态目录;留空使用数据目录下的 update-status/。</summary>
public string? BackendUpdateStatusDirectory { get; set; }
/// <summary>
/// 节点二进制绑定的策略:<c>enforce</c>(默认)二进制和绑定不符、又不在 CI 签名的更新清单里就拒绝连接;<c>warn</c> 只记录不拒绝;<c>off</c> 不检查。
/// </summary>
public string BinaryPolicy { get; set; } = "enforce";
/// <summary>
/// 每个 Agent 和 Probe 都必须有属主(passkey):创建加入码时必须已经建立信任根,并且至少选一把操作员集合里的 passkey 作为属主。
/// 属主是节点敏感内容(extension 会话、加密数据)绑定的人,装上时固定在节点本地,面板改不了。默认开;没有信任根的老部署才需要关掉。
@@ -28,8 +28,6 @@ public sealed class HotLayer
public void MarkReady() => _ready = true;
public IEnumerable<(NodeId Agent, string Series)> Keys => _rings.Keys;
/// <summary>评估一个块并合并进热层。同一个块重复合并(例如回填与实时竞争)会被忽略。</summary>
public void Merge(NodeId agent, string series, RawChunk chunk)
{
@@ -94,27 +92,6 @@ public sealed class HotLayer
return result;
}
public int BucketCount(NodeId agent, string series)
{
if (!_rings.TryGetValue((agent, series), out Ring? ring))
{
return 0;
}
lock (ring)
{
return ring.Buckets.Count;
}
}
public void Remove(NodeId agent)
{
foreach ((NodeId Agent, string Series) key in _rings.Keys.Where(k => k.Agent == agent).ToList())
{
_rings.TryRemove(key, out _);
}
}
private static void Prune(Ring ring, long cutoff)
{
while (ring.Buckets.Count > 0)
@@ -17,7 +17,7 @@ public sealed record ChunkSignal(NodeId AgentId, string Series) : LiveSignal;
public sealed record TestSignal(TestId TestId) : LiveSignal;
/// <summary>某个 Agent 的主机资源占用有了新的快照(实时推给展示页)。</summary>
public sealed record HostSignal(NodeId AgentId, HostSnapshot Host) : LiveSignal;
public sealed record HostSignal(NodeId Node, HostSnapshot Host) : LiveSignal;
public sealed class LiveSubscription : IDisposable
{
@@ -45,8 +45,6 @@ public sealed class LiveHub
{
private readonly ConcurrentDictionary<LiveSubscription, byte> _subscriptions = new();
public int SubscriberCount => _subscriptions.Count;
public LiveSubscription Subscribe()
{
var subscription = new LiveSubscription(this);
@@ -66,6 +66,17 @@ public static class IspClassifier
return (Other, Shorten(org));
}
/// <summary>手动指定运营商时的中文名;不认识的取值返回 null。</summary>
public static string? NameOf(string key) => key switch
{
Telecom => "中国电信",
Unicom => "中国联通",
Mobile => "中国移动",
Broadnet => "中国广电",
Other => "其他",
_ => null,
};
private static bool Has(string text, string needle) => text.Contains(needle, StringComparison.OrdinalIgnoreCase);
/// <summary>组织名常常很长(「Hangzhou Alibaba Advertising Co.,Ltd.」):去掉公司后缀,最多留 28 个字符。</summary>
@@ -16,7 +16,8 @@ public sealed record NodeInfo(
string? Version = null,
string? Tag = null,
string? Commit = null,
string? BuiltAt = null);
string? BuiltAt = null,
bool IspManual = false);
/// <summary>
/// 存起来的完整记录(只给 Backend 内部用):多了一个「查询用的主机」,用来判断地址变了没有、要不要重新查。
@@ -42,6 +42,31 @@ public sealed class NodeInfoService
return [.. _cache.Values.Select(m => m.Info)];
}
/// <summary>
/// 手动指定节点的运营商(telecom / unicom / mobile / broadnet / other);传 null 回到自动识别(下次节点出现时重新查)。
/// 返回 false 表示取值不认识。
/// </summary>
public async ValueTask<bool> SetIspAsync(string nodeId, string kind, string? key, CancellationToken ct)
{
await EnsureLoadedAsync(ct);
NodeMeta current = _cache.TryGetValue(nodeId, out NodeMeta? found) ? found : Empty(nodeId, kind);
if (key is null)
{
// 清掉手动值,并让下一次心跳重新查询
await SaveAsync(current with { Info = current.Info with { IspKey = null, IspName = null, IspManual = false }, GeoHost = null, GeoAtMs = 0, UpdatedAtMs = Now() }, ct);
return true;
}
string? name = IspClassifier.NameOf(key);
if (name is null)
{
return false;
}
await SaveAsync(current with { Info = current.Info with { IspKey = key, IspName = name, IspManual = true }, UpdatedAtMs = Now() }, ct);
return true;
}
/// <summary>节点自报的系统、架构和构建信息;没有变化就不写库。</summary>
public async ValueTask SetPlatformAsync(string nodeId, string kind, PlatformInfo? platform, CancellationToken ct)
{
@@ -101,8 +126,15 @@ public sealed class NodeInfoService
return;
}
(string key, string name) = IspClassifier.Classify(geo.Asn, geo.Organization);
string country = geo.Country.ToUpperInvariant();
if (current.Info.IspManual)
{
// 运营商是手动指定的:只更新国家,不覆盖运营商
await SaveAsync(current with { Info = current.Info with { Country = country }, GeoHost = host, GeoAtMs = now, UpdatedAtMs = now }, ct);
return;
}
(string key, string name) = IspClassifier.Classify(geo.Asn, geo.Organization);
await SaveAsync(current with { Info = current.Info with { Country = country, IspKey = key, IspName = Clean(name, 40) }, GeoHost = host, GeoAtMs = now, UpdatedAtMs = now }, ct);
}
finally
@@ -144,20 +144,6 @@ public interface IEventStore
ValueTask<int> DeleteOlderThanAsync(long cutoffMs, CancellationToken ct);
}
/// <summary>节点的二进制绑定:第一次握手成功时绑定的 SHA-256,对应的签名清单 seq(没有签名清单时为 0),以及最近一次不符的原因。</summary>
public sealed record NodeBinding(string NodeId, string Sha256, long Seq, bool Signed, long BoundAtMs, string? Violation, long? ViolationAtMs);
public interface INodeBindingStore
{
ValueTask<NodeBinding?> GetAsync(string nodeId, CancellationToken ct);
ValueTask<IReadOnlyList<NodeBinding>> ListAsync(CancellationToken ct);
ValueTask UpsertAsync(NodeBinding binding, CancellationToken ct);
ValueTask<bool> DeleteAsync(string nodeId, CancellationToken ct);
}
/// <summary>操作员集合的链(passkey 签的版本,按版本号递增)。</summary>
public interface IOperatorSetStore
{
@@ -422,8 +422,4 @@ public sealed class QueryService
return gaps;
}
/// <summary>从 JSON 文本读取参数(供端点之外的调用方使用)。</summary>
public static Dictionary<string, JsonElement> ParseParams(string json) =>
JsonSerializer.Deserialize(json, TestJson.Default.ParamMap) ?? [];
}
@@ -0,0 +1,7 @@
namespace MoShiro.Backend.Application;
/// <summary>CI 发布公钥(PEM):验登记的发布清单。由 Api 层提供(它就是内嵌在程序里的那一把)。</summary>
public interface IReleaseKey
{
string Pem { get; }
}
@@ -0,0 +1,105 @@
using System.Security.Cryptography;
using Fleet.Core;
namespace MoShiro.Backend.Application;
/// <summary>一把发布公钥的指纹(SPKI 的 SHA-256,小写十六进制)和 PEM。</summary>
public sealed record ReleaseKeyInfo(string Fingerprint, string PublicKey);
/// <summary>可信资产来源:官方 CI 发布密钥(内嵌,始终信任)和可选的自有发布公钥(默认没有)。</summary>
public sealed record ReleaseKeys(ReleaseKeyInfo Official, ReleaseKeyInfo? Custom);
/// <summary>
/// 面板信任哪些发布密钥:内嵌的官方 CI 公钥始终信任,不能关;部署的人可以另外导入<b>自己的发布公钥</b>(默认没有),
/// 用自己的私钥给自己构建的清单签名。验签(登记清单、取镜像仓库里的清单)通过官方或自有任意一把即可。
/// 私钥只在部署的人手里,面板只存公钥,拒绝带私钥的 PEM。
/// </summary>
public sealed class ReleaseKeyService
{
internal const string SettingKey = "release.custom_key";
private const int MaxPemBytes = 1024;
private const string P256Oid = "1.2.840.10045.3.1.7";
private readonly IReleaseKey _official;
private readonly ISettingsStore _settings;
public ReleaseKeyService(IReleaseKey official, ISettingsStore settings)
{
_official = official;
_settings = settings;
}
public async ValueTask<ReleaseKeys> GetAsync(CancellationToken ct) =>
new(Describe(_official.Pem)!, Describe(await CustomAsync(ct)));
/// <summary>清单的签名由官方密钥或自有密钥(开启时)验过。</summary>
public async ValueTask<bool> VerifyAsync(string manifest, string signature, CancellationToken ct) =>
ReleaseSignature.Verify(_official.Pem, manifest, signature)
|| (await CustomAsync(ct) is { } custom && ReleaseSignature.Verify(custom, manifest, signature));
public async ValueTask<Result<ReleaseKeys>> SetCustomAsync(string? publicKey, CancellationToken ct)
{
if (!TryNormalize(publicKey, out string? pem, out string? problem))
{
return ValidationError.Fail<ReleaseKeys>("public_key", problem);
}
await _settings.SetAsync(SettingKey, pem, ct);
return Result<ReleaseKeys>.Ok(await GetAsync(ct));
}
public async ValueTask ClearCustomAsync(CancellationToken ct) => await _settings.SetAsync(SettingKey, string.Empty, ct);
private async ValueTask<string?> CustomAsync(CancellationToken ct) =>
await _settings.GetAsync(SettingKey, ct) is { Length: > 0 } pem ? pem : null;
private static ReleaseKeyInfo? Describe(string? pem)
{
if (pem is null)
{
return null;
}
using var key = ECDsa.Create();
key.ImportFromPem(pem);
return new ReleaseKeyInfo(Convert.ToHexStringLower(SHA256.HashData(key.ExportSubjectPublicKeyInfo())), key.ExportSubjectPublicKeyInfoPem().Trim());
}
/// <summary>只收 P-256 的公钥:带私钥的 PEM 直接拒绝(私钥不该离开你的机器)。</summary>
private static bool TryNormalize(string? text, [System.Diagnostics.CodeAnalysis.NotNullWhen(true)] out string? pem, [System.Diagnostics.CodeAnalysis.NotNullWhen(false)] out string? problem)
{
pem = null;
problem = null;
string trimmed = (text ?? string.Empty).Trim();
if (trimmed.Length is 0 or > MaxPemBytes)
{
problem = "must be a PEM public key";
return false;
}
if (trimmed.Contains("PRIVATE", StringComparison.OrdinalIgnoreCase))
{
problem = "must be a public key: never upload the private key, keep it on your own machine";
return false;
}
try
{
using var key = ECDsa.Create();
key.ImportFromPem(trimmed);
if (key.ExportParameters(false).Curve.Oid.Value != P256Oid)
{
problem = "must be an ECDSA P-256 public key";
return false;
}
pem = key.ExportSubjectPublicKeyInfoPem().Trim();
return true;
}
catch (Exception ex) when (ex is CryptographicException or ArgumentException)
{
problem = "is not a valid PEM public key";
return false;
}
}
}
@@ -0,0 +1,58 @@
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using MoShiro.Protocol;
namespace MoShiro.Backend.Application;
/// <summary>
/// 定时去镜像仓库取各组件(Agent、Probe、Backend 自己)最新的签名清单:CI 把清单登记给面板那一步是尽力而为的(网络不通只警告),
/// 面板自己也去仓库取,Backend 的更新通知和节点的更新建议就不依赖那一步。取回来的清单要用发布公钥验过签才登记。
/// </summary>
public sealed partial class ReleasePoller : BackgroundService
{
private static readonly TimeSpan s_startupDelay = TimeSpan.FromSeconds(20);
private static readonly TimeSpan s_interval = TimeSpan.FromMinutes(5);
private static readonly string[] s_components = [NodeKinds.Agent, NodeKinds.Probe, ReleaseService.BackendComponent];
private readonly ReleaseService _releases;
private readonly TimeProvider _time;
private readonly ILogger<ReleasePoller> _log;
public ReleasePoller(ReleaseService releases, TimeProvider time, ILogger<ReleasePoller> log)
{
_releases = releases;
_time = time;
_log = log;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
try
{
await Task.Delay(s_startupDelay, _time, stoppingToken);
using var timer = new PeriodicTimer(s_interval, _time);
do
{
foreach (string component in s_components)
{
try
{
await _releases.GetOrFetchAsync(component, stoppingToken);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
LogFailed(ex, component);
}
}
}
while (await timer.WaitForNextTickAsync(stoppingToken));
}
catch (OperationCanceledException)
{
// 后端正在停止
}
}
[LoggerMessage(Level = LogLevel.Warning, Message = "could not check the registry for the latest {Component} release")]
private partial void LogFailed(Exception exception, string component);
}
@@ -26,14 +26,14 @@ public sealed partial class ReleaseService : IDisposable
private readonly IAgentGateway _gateway;
private readonly IBackendUpdateFiles _backendFiles;
private readonly BackendOptions _options;
private readonly IReleaseKey _key;
private readonly ReleaseKeyService _keys;
private readonly ReleaseFeed _feed;
private readonly ConcurrentDictionary<string, long> _feedChecked = new();
private readonly ConcurrentDictionary<string, UpdateStatus> _status = new();
public ReleaseService(ISettingsStore settings, IAgentGateway gateway, IBackendUpdateFiles backendFiles, BackendOptions options, IReleaseKey key, ReleaseFeed feed)
public ReleaseService(ISettingsStore settings, IAgentGateway gateway, IBackendUpdateFiles backendFiles, BackendOptions options, ReleaseKeyService keys, ReleaseFeed feed)
{
_key = key;
_keys = keys;
_feed = feed;
_settings = settings;
_gateway = gateway;
@@ -45,7 +45,7 @@ public sealed partial class ReleaseService : IDisposable
public void Dispose() => BackendGate.Dispose();
/// <summary>登记一份新清单并推给在线的同类节点。只要格式对、<c>seq</c> 比现有的新就收下(签名由节点验证)。</summary>
/// <summary>登记一份新清单并推给在线的同类节点。格式对、带 CI 发布密钥的有效签名、<c>seq</c> 比现有的新才收下;所有登记入口(CI、镜像仓库、管理员)都走这里,所以没有不验签的路径。节点的更新器之后还会用本机公钥再验一次。</summary>
public async ValueTask<Result<ReleaseOffer>> PublishAsync(string manifest, string signature, CancellationToken ct)
{
if (!TryParse(manifest, signature, out ReleaseOffer? offer, out string? problem))
@@ -53,6 +53,11 @@ public sealed partial class ReleaseService : IDisposable
return ValidationError.Fail<ReleaseOffer>("manifest", problem);
}
if (!await _keys.VerifyAsync(manifest, signature, ct))
{
return Result<ReleaseOffer>.Fail(MoShiroErrors.Forbidden, "the manifest does not carry a valid release signature");
}
await BackendGate.WaitAsync(ct);
try
{
@@ -95,16 +100,17 @@ public sealed partial class ReleaseService : IDisposable
/// </summary>
public async ValueTask<ReleaseOffer?> GetOrFetchAsync(string component, CancellationToken ct)
{
ReleaseOffer? stored = await GetAsync(component, ct);
string channel = BackendUpdates.ChannelOf(_options);
ReleaseOffer? stored = component == BackendComponent ? await GetBackendAsync(channel, ct) : await GetAsync(component, ct);
long now = Environment.TickCount64;
long wait = stored is null ? 30_000 : 600_000; // 没有就 30 秒试一次,已经有了就隔 10 分钟看一眼有没有更新的
long wait = stored is null ? 30_000 : 240_000; // 没有就 30 秒试一次,已经有了就隔 4 分钟看一眼有没有更新的(后台每 5 分钟问一次)
if (_feedChecked.TryGetValue(component, out long last) && now - last < wait)
{
return stored;
}
_feedChecked[component] = now;
if (await _feed.FetchAsync(component, BackendUpdates.ChannelOf(_options), ct) is { } fetched && ReleaseSignature.Verify(_key.Pem, fetched.Manifest, fetched.Signature))
if (await _feed.FetchAsync(component, channel, ct) is { } fetched && await _keys.VerifyAsync(fetched.Manifest, fetched.Signature, ct))
{
Result<ReleaseOffer> published = await PublishAsync(fetched.Manifest, fetched.Signature, ct);
if (published.IsSuccess)
@@ -116,6 +122,21 @@ public sealed partial class ReleaseService : IDisposable
return stored;
}
/// <summary>不再信任某把密钥之后(删除自有发布密钥):已登记的清单里,现在验不过的丢掉,免得继续分发。</summary>
public async ValueTask PurgeUnverifiedAsync(CancellationToken ct)
{
foreach (string key in new[] { NodeKinds.Agent, NodeKinds.Probe, BackendComponent + ".dev", BackendComponent + ".release" })
{
string? manifest = await _settings.GetAsync(Key(key, "manifest"), ct);
string? signature = await _settings.GetAsync(Key(key, "signature"), ct);
if (!string.IsNullOrEmpty(manifest) && !string.IsNullOrEmpty(signature) && !await _keys.VerifyAsync(manifest, signature, ct))
{
await _settings.SetAsync(Key(key, "manifest"), string.Empty, ct);
await _settings.SetAsync(Key(key, "signature"), string.Empty, ct);
}
}
}
public async ValueTask<ReleaseOffer?> GetAsync(string component, CancellationToken ct)
{
string? manifest = await _settings.GetAsync(Key(component, "manifest"), ct);
@@ -173,8 +194,6 @@ public sealed partial class ReleaseService : IDisposable
public UpdateStatus? StatusOf(NodeId node) => _status.TryGetValue(node.Value, out UpdateStatus? status) ? status : null;
public void Forget(NodeId node) => _status.TryRemove(node.Value, out _);
private IEnumerable<NodeId> OnlineOf(string component) => _gateway.OnlineNodes.Where(n => ComponentOf(n) == component);
internal static string? ComponentOf(NodeId node) =>
@@ -182,7 +201,7 @@ public sealed partial class ReleaseService : IDisposable
: node.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal) ? NodeKinds.Probe
: null;
internal static string Key(string component, string part) => $"release.{component}.{part}";
private static string Key(string component, string part) => $"release.{component}.{part}";
private static bool TryParse(string manifest, string signature, [NotNullWhen(true)] out ReleaseOffer? offer, [NotNullWhen(false)] out string? problem)
{
@@ -5,7 +5,7 @@ namespace MoShiro.Backend.Storage.Sqlite;
/// <summary>表结构。版本记录在 <c>PRAGMA user_version</c>;以后的迁移是按版本号追加的内置 SQL。</summary>
internal static class Schema
{
public const int CurrentVersion = 14;
public const int CurrentVersion = 16;
public static void Migrate(SqliteConnection connection)
{
@@ -241,6 +241,24 @@ internal static class Schema
connection.Exec("PRAGMA user_version = 14");
transaction.Commit();
}
if (version < 15)
{
// 运行时二进制绑定整体移除(节点自报的哈希不可信,来源证明在安装时):旧版本留下的表不再使用
using SqliteTransaction transaction = connection.BeginTransaction();
connection.Exec("DROP TABLE IF EXISTS node_bindings");
connection.Exec("PRAGMA user_version = 15");
transaction.Commit();
}
if (version < 16)
{
// 运营商可以手动指定(自动识别会错,例如 CN2 线路):手动的不会被后面的 IP 归属查询覆盖
using SqliteTransaction transaction = connection.BeginTransaction();
connection.Exec("ALTER TABLE node_meta ADD COLUMN isp_manual INTEGER NOT NULL DEFAULT 0");
connection.Exec("PRAGMA user_version = 16");
transaction.Commit();
}
}
/// <summary>
@@ -1,35 +0,0 @@
using MoShiro.Backend.Application;
namespace MoShiro.Backend.Storage.Sqlite;
public sealed class SqliteNodeBindingStore : INodeBindingStore
{
private const string Columns = "node_id, sha256, seq, signed, bound_at, violation, violation_at";
private readonly SqliteDatabase _db;
public SqliteNodeBindingStore(SqliteDatabase db)
{
_db = db;
}
public ValueTask<NodeBinding?> GetAsync(string nodeId, CancellationToken ct) =>
_db.ReadAsync<NodeBinding?>(c => c.QueryOne($"SELECT {Columns} FROM node_bindings WHERE node_id = @id", Read, ("@id", nodeId)), ct);
public ValueTask<IReadOnlyList<NodeBinding>> ListAsync(CancellationToken ct) =>
_db.ReadAsync<IReadOnlyList<NodeBinding>>(c => c.Query($"SELECT {Columns} FROM node_bindings ORDER BY node_id", Read), ct);
public async ValueTask UpsertAsync(NodeBinding binding, CancellationToken ct) =>
await _db.WriteAsync(
c => c.Exec(
$"INSERT INTO node_bindings ({Columns}) VALUES (@id, @sha, @seq, @signed, @at, @violation, @violation_at) "
+ "ON CONFLICT(node_id) DO UPDATE SET sha256 = excluded.sha256, seq = excluded.seq, signed = excluded.signed, bound_at = excluded.bound_at, violation = excluded.violation, violation_at = excluded.violation_at",
("@id", binding.NodeId), ("@sha", binding.Sha256), ("@seq", binding.Seq), ("@signed", binding.Signed ? 1 : 0), ("@at", binding.BoundAtMs), ("@violation", binding.Violation), ("@violation_at", binding.ViolationAtMs)),
ct);
public ValueTask<bool> DeleteAsync(string nodeId, CancellationToken ct) =>
_db.WriteAsync(c => c.Exec("DELETE FROM node_bindings WHERE node_id = @id", ("@id", nodeId)) > 0, ct);
private static NodeBinding Read(Microsoft.Data.Sqlite.SqliteDataReader r) =>
new(r.GetString(0), r.GetString(1), r.GetInt64(2), r.GetInt64(3) != 0, r.GetInt64(4), r.NullableString(5), r.NullableLong(6));
}
@@ -14,20 +14,20 @@ public sealed class SqliteNodeMetaStore : INodeMetaStore
public ValueTask<IReadOnlyList<NodeMeta>> ListAsync(CancellationToken ct) =>
_db.ReadAsync<IReadOnlyList<NodeMeta>>(
c => c.Query("SELECT node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at FROM node_meta", Read),
c => c.Query("SELECT node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at, isp_manual FROM node_meta", Read),
ct);
public ValueTask SaveAsync(NodeMeta meta, CancellationToken ct) =>
_db.ExecuteAsync(
c => c.Exec(
"INSERT INTO node_meta (node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at) " +
"VALUES (@id, @kind, @os, @osn, @arch, @cc, @ik, @in, @gh, @ga, @ua, @ver, @tag, @cm, @bt) ON CONFLICT(node_id) DO UPDATE SET " +
"INSERT INTO node_meta (node_id, kind, os, os_name, arch, country, isp_key, isp_name, geo_host, geo_at, updated_at, version, tag, commit_id, built_at, isp_manual) " +
"VALUES (@id, @kind, @os, @osn, @arch, @cc, @ik, @in, @gh, @ga, @ua, @ver, @tag, @cm, @bt, @im) ON CONFLICT(node_id) DO UPDATE SET " +
"kind = excluded.kind, os = excluded.os, os_name = excluded.os_name, arch = excluded.arch, country = excluded.country, " +
"isp_key = excluded.isp_key, isp_name = excluded.isp_name, geo_host = excluded.geo_host, geo_at = excluded.geo_at, updated_at = excluded.updated_at, " +
"version = excluded.version, tag = excluded.tag, commit_id = excluded.commit_id, built_at = excluded.built_at",
"version = excluded.version, tag = excluded.tag, commit_id = excluded.commit_id, built_at = excluded.built_at, isp_manual = excluded.isp_manual",
("@id", meta.Info.NodeId), ("@kind", meta.Info.Kind), ("@os", meta.Info.Os), ("@osn", meta.Info.OsName), ("@arch", meta.Info.Arch), ("@cc", meta.Info.Country),
("@ik", meta.Info.IspKey), ("@in", meta.Info.IspName), ("@gh", meta.GeoHost), ("@ga", meta.GeoAtMs), ("@ua", meta.UpdatedAtMs),
("@ver", meta.Info.Version), ("@tag", meta.Info.Tag), ("@cm", meta.Info.Commit), ("@bt", meta.Info.BuiltAt)),
("@ver", meta.Info.Version), ("@tag", meta.Info.Tag), ("@cm", meta.Info.Commit), ("@bt", meta.Info.BuiltAt), ("@im", meta.Info.IspManual ? 1 : 0)),
ct);
public ValueTask DeleteAsync(string nodeId, CancellationToken ct) =>
@@ -36,6 +36,6 @@ public sealed class SqliteNodeMetaStore : INodeMetaStore
private static NodeMeta Read(SqliteDataReader r)
{
static string? S(SqliteDataReader reader, int i) => reader.IsDBNull(i) ? null : reader.GetString(i);
return new NodeMeta(new NodeInfo(r.GetString(0), r.GetString(1), S(r, 2), S(r, 3), S(r, 4), S(r, 5), S(r, 6), S(r, 7), S(r, 11), S(r, 12), S(r, 13), S(r, 14)), S(r, 8), r.GetInt64(9), r.GetInt64(10));
return new NodeMeta(new NodeInfo(r.GetString(0), r.GetString(1), S(r, 2), S(r, 3), S(r, 4), S(r, 5), S(r, 6), S(r, 7), S(r, 11), S(r, 12), S(r, 13), S(r, 14), r.GetInt64(15) != 0), S(r, 8), r.GetInt64(9), r.GetInt64(10));
}
}
@@ -24,7 +24,6 @@ public static class StorageServiceCollectionExtensions
services.AddSingleton<IAuthStore, SqliteAuthStore>();
services.AddSingleton<IAuditStore, SqliteAuditStore>();
services.AddSingleton<IOperatorSetStore, SqliteOperatorSetStore>();
services.AddSingleton<INodeBindingStore, SqliteNodeBindingStore>();
services.AddSingleton<IRawStore, SqliteRawStore>();
services.AddSingleton<IMetricStore, SqliteMetricStore>();
services.AddSingleton<IStorageHealth, SqliteStorageHealth>();
+2 -1
View File
@@ -19,7 +19,7 @@ public static class BackendHost
public const string ConfigurationSection = "MoShiro";
public const int MaxRequestBodyBytes = 64 * 1024;
public static WebApplication Build(string[] args, Action<BackendOptions>? configureOptions = null, TimeProvider? time = null)
public static WebApplication Build(string[] args, Action<BackendOptions>? configureOptions = null, TimeProvider? time = null, Action<IServiceCollection>? configureServices = null)
{
WebApplicationBuilder builder = WebApplication.CreateSlimBuilder(args);
@@ -51,6 +51,7 @@ public static class BackendHost
services.AddMoShiroApplication(options);
services.AddMoShiroGateway();
services.AddMoShiroApi();
configureServices?.Invoke(services); // 测试替换发布公钥这类服务
WebApplication app = builder.Build();
// 静默死亡的 Agent(断电、断网)不会关闭连接:15 秒一个 Ping,20 秒没有 Pong 就关闭,这样面板上的在线状态才可信
@@ -3,7 +3,7 @@ using System.Xml.Linq;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 依赖规则(docs/project-layout.md 第 3 节):由测试在构建期强制。
/// 依赖规则(docs/dev/project-layout.md 第 3 节):由测试在构建期强制。
/// 读取仓库里所有 .csproj 和源文件,违反规则就失败。
/// </summary>
public sealed class ArchitectureTests
@@ -264,8 +264,10 @@ public class BackendFixture : IAsyncLifetime
options.LoginAttemptsPerMinute = LoginAttemptsPerMinute;
options.RequireOwner = false; // 大部分测试不建立信任根;属主必选的测试自己打开(OwnerTests)
options.GeoLookupUrl = string.Empty; // 测试里不访问外部服务
options.NodeImageRegistry = string.Empty; // 同理:后台不去真实的镜像仓库取清单
options.PhaseDurationScale = 0.2; // 端到端测试里 connect、hold 这些固定 30 秒的阶段缩成 6 秒
});
},
configureServices: services => services.AddSingleton<MoShiro.Backend.Application.IReleaseKey>(TestReleaseKey.Instance)); // 登记清单要验发布签名:测试用自己的密钥签
await App.StartAsync();
string address = App.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First();
BaseAddress = new Uri(address);
@@ -33,7 +33,7 @@ public sealed class BackendUpdateTests : IAsyncLifetime
private async Task<HttpResponseMessage> PublishAsync(long seq, string channel = "dev")
{
string manifest = $$"""{"component":"backend","channel":"{{channel}}","image":"registry.example/moshiro-{{channel}}/moshiro-backend","digest":"{{Digest}}","tag":"{{channel}}-{{seq}}","version":"0.1.1","seq":{{seq}}}""";
return await _backend.Admin.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = "MEUC" }));
return await _backend.Anonymous.PostAsync("/api/release/offer", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = TestReleaseKey.Sign(manifest) }));
}
private Task<JsonNode> ViewAsync() => _backend.GetJsonAsync(_backend.Admin, "/api/admin/backend/update");
@@ -1,348 +0,0 @@
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json.Nodes;
using Fleet.Core;
using Fleet.Server;
using MoShiro.Backend.Application;
using MoShiro.Protocol;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 节点二进制绑定:第一次握手成功的那一刻绑定正在运行的二进制的哈希,之后对不上又不在 CI 签名的更新清单里,就不让它继续运行。
/// 官方哈希集合来自 CI 签名的清单(节点握手时带着),所以被攻破的 Backend 伪造不了,也不依赖面板有没有收到过这次发布。
/// </summary>
public sealed class BinaryBindingTests : IDisposable
{
private const string A = "a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1";
private const string B = "b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2";
private const string C = "c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3";
private readonly ECDsa _ci = ECDsa.Create(ECCurve.NamedCurves.nistP256);
private readonly ECDsa _other = ECDsa.Create(ECCurve.NamedCurves.nistP256);
private readonly MemoryStore _store = new();
private readonly MemorySettings _settings = new();
private readonly NodeRecord _agent = Node(AgentIds.Prefix + "_test1");
public void Dispose()
{
_ci.Dispose();
_other.Dispose();
}
private static string Hash(string seed) => seed[..2] + new string(seed[1], 62);
private static NodeRecord Node(string id) => new(new NodeId(id), "n", NodeStatus.Active, [1], [2], [], null, null, 0, null);
private BinaryBindingService Service(string policy = "enforce") =>
new(_store, _settings, new TestKey(_ci), new BackendOptions { BinaryPolicy = policy }, TimeProvider.System);
private sealed class TestKey(ECDsa key) : IReleaseKey
{
public string Pem => key.ExportSubjectPublicKeyInfoPem();
}
private static string Sign(ECDsa key, string manifest) =>
Convert.ToBase64String(key.SignData(Encoding.UTF8.GetBytes(manifest), HashAlgorithmName.SHA256, DSASignatureFormat.Rfc3279DerSequence));
/// <summary>节点握手时带的证据:哈希,加上(可选)某把密钥签名的清单。</summary>
private BinaryEvidence Evidence(string hash, long? seq = null, string[]? binaries = null, string component = "agent", ECDsa? signer = null)
{
if (seq is null)
{
return new BinaryEvidence(hash, null, null);
}
string manifest = $"{{\"schema\":1,\"component\":\"{component}\",\"seq\":{seq},\"binaries\":\"{string.Join(',', binaries ?? [hash])}\"}}";
return new BinaryEvidence(hash, manifest, Sign(signer ?? _ci, manifest));
}
private static string H(char c) => new(c, 64);
[Fact]
public async Task TheFirstHandshakeBinds_AndTheSameBinaryKeepsWorking()
{
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
Assert.Equal(H('a'), (await _store.GetAsync(_agent.Id.Value, default))!.Sha256);
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
}
[Fact]
public async Task AReplacedBinary_IsRefused_AndTheReasonIsKept()
{
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a')), default);
BinaryDecision decision = await service.CheckAsync(_agent, Evidence(H('b')), default);
Assert.False(decision.Accept);
Assert.Contains("changed", decision.Reason, StringComparison.Ordinal);
NodeBinding bound = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.Equal(H('a'), bound.Sha256); // 绑定不会因为一次不符而被改写
Assert.NotNull(bound.Violation);
// 原来的二进制回来了:恢复,原因清掉
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
Assert.Null((await _store.GetAsync(_agent.Id.Value, default))!.Violation);
}
[Fact]
public async Task AnUpgrade_NeedsACiSignedManifestThatListsTheNewBinary()
{
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a'), seq: 100, binaries: [H('a')]), default);
// 新二进制在一份更新的 CI 签名清单里:合法升级,绑定跟着改
Assert.True((await service.CheckAsync(_agent, Evidence(H('b'), seq: 200, binaries: [H('b')]), default)).Accept);
NodeBinding upgraded = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.Equal(H('b'), upgraded.Sha256);
Assert.Equal(200, upgraded.Seq);
Assert.True(upgraded.Signed);
// 退回旧版本(清单 seq 更低):降级,拒绝
Assert.False((await service.CheckAsync(_agent, Evidence(H('a'), seq: 100, binaries: [H('a')]), default)).Accept);
// 同一份清单里的另一个官方二进制(比如另一个平台的构建)不算降级
Assert.True((await service.CheckAsync(_agent, Evidence(H('c'), seq: 200, binaries: [H('b'), H('c')]), default)).Accept);
}
[Fact]
public async Task AManifestSignedByAnotherKey_OrForAnotherComponent_ProvesNothing()
{
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a')), default);
Assert.False((await service.CheckAsync(_agent, Evidence(H('b'), seq: 500, binaries: [H('b')], signer: _other), default)).Accept);
Assert.False((await service.CheckAsync(_agent, Evidence(H('b'), seq: 500, binaries: [H('b')], component: "probe"), default)).Accept);
Assert.Equal(H('a'), (await _store.GetAsync(_agent.Id.Value, default))!.Sha256);
}
[Fact]
public async Task ASignedManifestThatDoesNotListTheRunningBinary_IsRefusedAtTheFirstHandshake()
{
BinaryBindingService service = Service();
BinaryDecision decision = await service.CheckAsync(_agent, Evidence(H('a'), seq: 100, binaries: [H('b')]), default);
Assert.False(decision.Accept);
Assert.Null(await _store.GetAsync(_agent.Id.Value, default));
}
[Fact]
public async Task AnUnsignedBinding_BecomesSignedWhenTheManifestArrivesLater()
{
// 容器节点第一次握手时更新器可能还没写出清单:先按首次握手绑定,清单到了再标成已签名
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a')), default);
Assert.False((await _store.GetAsync(_agent.Id.Value, default))!.Signed);
Assert.True((await service.CheckAsync(_agent, Evidence(H('a'), seq: 100, binaries: [H('a')]), default)).Accept);
NodeBinding bound = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.True(bound.Signed);
Assert.Equal(100, bound.Seq);
}
[Fact]
public async Task ALegacyNode_IsTolerated_ButCannotGoBackOnceBound()
{
BinaryBindingService service = Service();
NodeRecord legacy = Node(ProbeId.Prefix + "_old");
Assert.True((await service.CheckAsync(legacy, new BinaryEvidence(null, null, null), default)).Accept);
Assert.Null(await _store.GetAsync(legacy.Id.Value, default));
await service.CheckAsync(_agent, Evidence(H('a')), default);
Assert.False((await service.CheckAsync(_agent, new BinaryEvidence(null, null, null), default)).Accept); // 绑定过的不许退回不上报
}
[Fact]
public async Task MalformedHashes_AreRefused()
{
BinaryBindingService service = Service();
Assert.False((await service.CheckAsync(_agent, new BinaryEvidence("not-a-hash", null, null), default)).Accept);
Assert.False((await service.CheckAsync(_agent, new BinaryEvidence(new string('A', 64), null, null), default)).Accept); // 要小写十六进制
Assert.Null(await _store.GetAsync(_agent.Id.Value, default));
}
[Fact]
public async Task TheAdministratorCanAcceptANewBinary_ByClearingTheBinding()
{
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a')), default);
Assert.False((await service.CheckAsync(_agent, Evidence(H('b')), default)).Accept);
Assert.True(await service.ClearAsync(_agent.Id.Value, default));
Assert.True((await service.CheckAsync(_agent, Evidence(H('b')), default)).Accept);
Assert.Equal(H('b'), (await _store.GetAsync(_agent.Id.Value, default))!.Sha256);
}
[Fact]
public async Task WarnRecordsButDoesNotRefuse_AndOffDoesNothing()
{
BinaryBindingService warn = Service("warn");
await warn.CheckAsync(_agent, Evidence(H('a')), default);
BinaryDecision decision = await warn.CheckAsync(_agent, Evidence(H('b')), default);
Assert.True(decision.Accept);
Assert.NotNull((await _store.GetAsync(_agent.Id.Value, default))!.Violation);
BinaryBindingService off = Service("off");
NodeRecord other = Node(AgentIds.Prefix + "_test2");
Assert.True((await off.CheckAsync(other, Evidence(H('a')), default)).Accept);
Assert.Null(await _store.GetAsync(other.Id.Value, default));
}
[Fact]
public async Task EachNodeHasItsOwnBinding()
{
BinaryBindingService service = Service();
NodeRecord second = Node(AgentIds.Prefix + "_test3");
await service.CheckAsync(_agent, Evidence(H('a')), default);
await service.CheckAsync(second, Evidence(H('b')), default);
Assert.False((await service.CheckAsync(_agent, Evidence(H('b')), default)).Accept); // 别的节点的二进制,不是自己绑定的
Assert.True((await service.CheckAsync(second, Evidence(H('b')), default)).Accept);
}
/// <summary>面板手里的、CI 签名的清单(登记时存下的):没有更新器的节点握手时没有自己的清单,就拿它来认官方二进制。</summary>
private void PanelHolds(string component, long seq, string[] binaries, ECDsa? signer = null)
{
string manifest = $"{{\"schema\":1,\"component\":\"{component}\",\"seq\":{seq},\"binaries\":\"{string.Join(',', binaries)}\"}}";
_settings.Values["release." + component + ".manifest"] = manifest;
_settings.Values["release." + component + ".signature"] = Sign(signer ?? _ci, manifest);
}
[Fact]
public async Task ANodeWithoutAnUpdater_IsMarkedSigned_WhenThePanelHoldsAManifestThatListsItsBinary()
{
// 一键容器安装的节点没有宿主机更新器,握手时带不来清单:面板用自己手里(登记时验过 CI 签名)的那份
PanelHolds("agent", seq: 100, [H('a')]);
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
NodeBinding bound = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.True(bound.Signed);
Assert.Equal(100, bound.Seq);
}
[Fact]
public async Task TheManifestThePanelHolds_NeverRefusesANodeAtFirstBinding_OnlyLabelsIt()
{
// 老一点的官方版本、自己构建的版本:不在面板手里最新的清单里,但没有证据说它不是官方的,所以不拒绝,只是不标成已签名
PanelHolds("agent", seq: 100, [H('b')]);
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
NodeBinding bound = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.False(bound.Signed);
Assert.Equal(0, bound.Seq);
}
[Fact]
public async Task AManifestThePanelHoldsThatIsSignedByAnotherKey_ProvesNothing()
{
PanelHolds("agent", seq: 100, [H('a')], signer: _other);
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
Assert.False((await _store.GetAsync(_agent.Id.Value, default))!.Signed);
}
[Fact]
public async Task AnUpgradeOfANodeWithoutAnUpdater_IsFollowedFromTheManifestThePanelHolds()
{
PanelHolds("agent", seq: 100, [H('a')]);
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a')), default);
// 之后发布了新版,面板登记了新的清单;节点换成新二进制重连
PanelHolds("agent", seq: 200, [H('b')]);
Assert.True((await service.CheckAsync(_agent, Evidence(H('b')), default)).Accept);
Assert.Equal(200, (await _store.GetAsync(_agent.Id.Value, default))!.Seq);
// 没人签过的二进制仍然被拒绝
Assert.False((await service.CheckAsync(_agent, Evidence(H('c')), default)).Accept);
}
private sealed class MemorySettings : ISettingsStore
{
public Dictionary<string, string> Values { get; } = [];
public ValueTask<string?> GetAsync(string key, CancellationToken ct) => new(Values.GetValueOrDefault(key));
public ValueTask SetAsync(string key, string value, CancellationToken ct)
{
Values[key] = value;
return ValueTask.CompletedTask;
}
public ValueTask<IReadOnlyDictionary<string, string>> GetAllAsync(CancellationToken ct) => new(Values);
public ValueTask<long> IncrementAsync(string key, CancellationToken ct) => new(0);
}
private sealed class MemoryStore : INodeBindingStore
{
private readonly Dictionary<string, NodeBinding> _rows = [];
public ValueTask<NodeBinding?> GetAsync(string nodeId, CancellationToken ct) => new(_rows.GetValueOrDefault(nodeId));
public ValueTask<IReadOnlyList<NodeBinding>> ListAsync(CancellationToken ct) => new([.. _rows.Values]);
public ValueTask UpsertAsync(NodeBinding binding, CancellationToken ct)
{
_rows[binding.NodeId] = binding;
return ValueTask.CompletedTask;
}
public ValueTask<bool> DeleteAsync(string nodeId, CancellationToken ct) => new(_rows.Remove(nodeId));
}
}
/// <summary>真实的 Agent 在进程里连上 Backend:第一次握手就绑定了它正在运行的二进制,管理端看得到,也可以清除。</summary>
[Collection("probe-e2e")]
public sealed class BinaryBindingEndToEndTests
{
private readonly BackendFixture _backend;
public BinaryBindingEndToEndTests(BackendFixture backend)
{
_backend = backend;
}
[Fact]
public async Task ARealAgent_IsBoundOnItsFirstHandshake_AndTheAdministratorCanClearIt()
{
await using AgentHarness agent = await AgentHarness.StartAsync(_backend);
string? expected = NodeBinary.Sha256Hex;
Assert.NotNull(expected);
JsonNode? binary = null;
await BackendFixture.EventuallyAsync(
async () =>
{
JsonNode info = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info");
binary = info["nodes"]!.AsArray().FirstOrDefault(n => n!["id"]!.GetValue<string>() == agent.Id)?["binary"];
return binary is not null;
},
TimeSpan.FromSeconds(30),
"the agent is bound on its first handshake");
Assert.Equal(expected, binary!["sha256"]!.GetValue<string>());
Assert.False(binary["signed"]!.GetValue<bool>()); // 测试里的 Agent 没有更新器,也就没有 CI 签名的清单
Assert.Null(binary["violation"]);
using HttpResponseMessage cleared = await _backend.Admin.DeleteAsync("/api/admin/nodes/" + agent.Id + "/binary");
Assert.Equal(HttpStatusCode.NoContent, cleared.StatusCode);
JsonNode after = await _backend.GetJsonAsync(_backend.Admin, "/api/admin/build-info");
Assert.Null(after["nodes"]!.AsArray().First(n => n!["id"]!.GetValue<string>() == agent.Id)!["binary"]);
}
}
@@ -8,7 +8,7 @@ using MoShiro.Backend.Application;
namespace MoShiro.Backend.Tests;
/// <summary>能力模型(docs/access.md 第 2 节):每个管理端点声明能力;密钥只能拿到自己拥有能力的子集;写蕴含读。</summary>
/// <summary>能力模型(docs/security/access.md 第 3 节):每个管理端点声明能力;密钥只能拿到自己拥有能力的子集;写蕴含读。</summary>
[Collection("probe-e2e")]
public sealed class CapabilityTests
{
@@ -121,12 +121,4 @@ public sealed class CapabilityTests
Assert.Contains(catalog["presets"]!.AsArray(), p => p!["id"]!.GetValue<string>() == "all");
Assert.Contains(catalog["groups"]!.AsArray().SelectMany(g => g!["capabilities"]!.AsArray()), c => c!["id"]!.GetValue<string>() == Capability.FleetRevoke && c["trust"]!.GetValue<bool>());
}
[Fact]
public async Task ReleaseKey_CanOnlyPublishReleases()
{
using HttpClient ci = await KeyAsync(Capability.ReleasePublish);
Assert.Equal(HttpStatusCode.Forbidden, (await ci.GetAsync("/api/admin/agents")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await ci.GetAsync("/api/admin/backend/update")).StatusCode);
}
}
@@ -1,5 +1,6 @@
using System.Text.Json.Nodes;
using MoShiro.Agent;
using MoShiro.Measurement;
using MoShiro.Protocol;
using MoShiro.TestSupport;
@@ -24,6 +25,23 @@ public sealed class HostSamplerTests
Assert.Equal((1000L, 5000L), HostSampler.ParseNetwork(Dev1 + "podman0: 8888 1 0 0 0 0 0 0 9999 1 0 0 0 0 0 0\n"));
}
[Fact]
public async Task Sample_OnWindows_ReadsCpuMemoryAndNetworkFromTheSystem()
{
if (!OperatingSystem.IsWindows())
{
return;
}
var sampler = new HostSampler(TimeProvider.System);
sampler.Sample();
await Task.Delay(300, TestContext.Current.CancellationToken);
HostSnapshot snapshot = sampler.Sample();
Assert.NotNull(snapshot.CpuPercent);
Assert.InRange(snapshot.CpuPercent!.Value, 0, 100);
Assert.True(snapshot.MemTotal > 0 && snapshot.MemUsed >= 0 && snapshot.MemUsed <= snapshot.MemTotal);
}
[Fact]
public void Sample_DerivesCpuPercentAndNetworkSpeedFromTheDifferenceBetweenTwoReads()
{
@@ -123,13 +141,15 @@ public sealed class HostStatusTests
await using SseClient sse = await SseClient.ConnectAsync(_backend.Anonymous, "/api/public/live?topics=hosts");
await agent.SendHostAsync(Snapshot(18.5));
SseEvent live = await sse.WaitForAsync(e => e.Name == "host" && e.Data!["agent_id"]!.GetValue<string>() == id);
SseEvent live = await sse.WaitForAsync(e => e.Name == "host" && e.Data!["id"]!.GetValue<string>() == id);
Assert.Equal("agent", live.Data!["kind"]!.GetValue<string>());
Assert.Equal(18.5, live.Data!["host"]!["cpu_percent"]!.GetValue<double>());
Assert.True(live.Data["online"]!.GetValue<bool>());
// REST:最新值,没有过期
JsonNode list = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/hosts");
JsonNode item = list["items"]!.AsArray().Single(i => i!["agent_id"]!.GetValue<string>() == id)!;
JsonNode item = list["items"]!.AsArray().Single(i => i!["id"]!.GetValue<string>() == id)!;
Assert.Equal("agent", item["kind"]!.GetValue<string>());
Assert.False(item["stale"]!.GetValue<bool>());
Assert.Equal(40L << 30, item["host"]!["disk_total"]!.GetValue<long>());
Assert.Equal(1000, item["host"]!["net_link_mbps"]!.GetValue<long>());
@@ -150,7 +170,7 @@ public sealed class HostStatusTests
JsonNode history = null!;
await BackendFixture.EventuallyAsync(async () =>
{
history = await _backend.GetJsonAsync(_backend.Anonymous, $"/api/public/agents/{id}/host?range=1h");
history = await _backend.GetJsonAsync(_backend.Anonymous, $"/api/public/hosts/{id}?range=1h");
return history["values"]!["cpu"]!.AsArray().Any(v => v is not null);
});
Assert.Equal(25.0, history["values"]!["mem"]!.AsArray().Last(v => v is not null)!.GetValue<double>());
@@ -159,8 +179,61 @@ public sealed class HostStatusTests
// 隐藏的 Agent 对匿名访客不可见
await _backend.Admin.PatchAsync($"/api/admin/agents/{id}", BackendFixture.Json(new JsonObject { ["public_visible"] = false }));
JsonNode hidden = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/hosts");
Assert.DoesNotContain(hidden["items"]!.AsArray(), i => i!["agent_id"]!.GetValue<string>() == id);
using HttpResponseMessage denied = await _backend.Anonymous.GetAsync($"/api/public/agents/{id}/host");
Assert.DoesNotContain(hidden["items"]!.AsArray(), i => i!["id"]!.GetValue<string>() == id);
using HttpResponseMessage denied = await _backend.Anonymous.GetAsync($"/api/public/hosts/{id}");
Assert.Equal(System.Net.HttpStatusCode.NotFound, denied.StatusCode);
}
[Fact]
public async Task ARealProbeReportsItsHostStatus_LikeAnAgent_NeverCarriesModelsOrAddresses_AndDisappearsWhenRevoked()
{
await _backend.SiteMustBePublicAsync();
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend, file => file.Reporting.HostIntervalSeconds = 1);
JsonNode? item = null;
await BackendFixture.EventuallyAsync(
async () =>
{
JsonNode list = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/hosts");
item = list["items"]!.AsArray().SingleOrDefault(i => i!["id"]!.GetValue<string>() == probe.ProbeId);
return item?["host"] is not null;
},
TimeSpan.FromSeconds(30),
"the probe's host status shows up");
Assert.Equal("probe", item!["kind"]!.GetValue<string>());
Assert.NotNull(item["host"]!["uptime_seconds"]); // 任何平台都读得到运行时长
foreach (string forbidden in new[] { "model", "hostname", "\"ip\"", "address", "remote", "mac", "iface", "public_host" })
{
Assert.DoesNotContain(forbidden, item.ToJsonString(), StringComparison.OrdinalIgnoreCase);
}
// 历史也按节点 id 取
using HttpResponseMessage history = await _backend.Anonymous.GetAsync($"/api/public/hosts/{probe.ProbeId}?range=1h");
Assert.Equal(System.Net.HttpStatusCode.OK, history.StatusCode);
// 吊销之后展示页就看不到它了
using HttpResponseMessage revoked = await _backend.Admin.PostAsync($"/api/admin/probes/{probe.ProbeId}/revoke", null);
Assert.True(revoked.IsSuccessStatusCode);
JsonNode after = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/hosts");
Assert.DoesNotContain(after["items"]!.AsArray(), i => i!["id"]!.GetValue<string>() == probe.ProbeId);
using HttpResponseMessage gone = await _backend.Anonymous.GetAsync($"/api/public/hosts/{probe.ProbeId}");
Assert.Equal(System.Net.HttpStatusCode.NotFound, gone.StatusCode);
}
[Fact]
public async Task AProbeThatTurnsReportingOff_SendsNothing()
{
await _backend.SiteMustBePublicAsync();
await using ProbeHarness probe = await ProbeHarness.StartAsync(_backend, file =>
{
file.Reporting.HostStats = false;
file.Reporting.HostIntervalSeconds = 1;
});
await Task.Delay(TimeSpan.FromSeconds(4));
JsonNode list = await _backend.GetJsonAsync(_backend.Anonymous, "/api/public/hosts");
JsonNode item = list["items"]!.AsArray().Single(i => i!["id"]!.GetValue<string>() == probe.ProbeId)!;
Assert.Null(item["host"]); // 节点还在,但一个字也没上报
}
}
@@ -48,6 +48,128 @@ public sealed class InstallTests
Assert.DoesNotContain("<", linux + macos + windows, StringComparison.Ordinal); // 没有要用户自己替换的占位符
}
[Theory]
[InlineData("agent")]
[InlineData("probe")]
public async Task ComposeInstall_WritesThePublicValuesToAnEnvFile_AndChecksTheComposeHash(string role)
{
JsonNode created = await CreateAsync(role);
string code = created["plaintext"]!.GetValue<string>();
JsonNode compose = created["compose"]!;
string env = compose["env"]!.GetValue<string>();
string url = compose["url"]!.GetValue<string>();
string linux = compose["linux"]!.GetValue<string>();
Assert.EndsWith($"/install/compose/{role}.yaml", url, StringComparison.Ordinal);
Assert.Contains($"MOSHIRO_JOIN={code}", env, StringComparison.Ordinal);
Assert.Contains("MOSHIRO_SERVER=", env, StringComparison.Ordinal);
Assert.Contains($"/moshiro-{role}", env, StringComparison.Ordinal); // MOSHIRO_IMAGE:仓库和命名空间来自后端的配置,不是写死的
Assert.StartsWith($"mkdir -p moshiro-{role} && cd moshiro-{role} && cat > .env <<'EOF'\n", linux, StringComparison.Ordinal);
Assert.Contains(env, linux, StringComparison.Ordinal);
Assert.Contains("chmod 600 .env", linux, StringComparison.Ordinal);
Assert.Contains($"echo \"{compose["sha256"]!.GetValue<string>()} compose.yaml\" | sha256sum -c -", linux, StringComparison.Ordinal);
Assert.EndsWith("podman compose up -d", linux, StringComparison.Ordinal);
// 面板提供的就是命令里核对的那份
using HttpResponseMessage served = await _backend.Anonymous.GetAsync($"/install/compose/{role}.yaml");
Assert.Equal(HttpStatusCode.OK, served.StatusCode);
Assert.Equal("text/yaml", served.Content.Headers.ContentType?.MediaType);
byte[] body = await served.Content.ReadAsByteArrayAsync();
Assert.Equal(compose["sha256"]!.GetValue<string>(), Convert.ToHexStringLower(System.Security.Cryptography.SHA256.HashData(body)));
if (role == "probe")
{
// Docker Desktop 没有宿主网络:macOS 和 Windows 的 .env 里改成 bridge
string macos = compose["macos"]!.GetValue<string>();
string windows = compose["windows"]!.GetValue<string>();
Assert.Contains("MOSHIRO_NETWORK_MODE=bridge", macos + windows, StringComparison.Ordinal);
Assert.Contains("docker compose up -d", macos, StringComparison.Ordinal);
Assert.Contains("Get-FileHash compose.yaml", windows, StringComparison.Ordinal);
}
else
{
// Agent 需要 Linux 的宿主网络
Assert.Equal(string.Empty, compose["macos"]!.GetValue<string>());
Assert.Equal(string.Empty, compose["windows"]!.GetValue<string>());
}
}
[Theory]
[InlineData("agent")]
[InlineData("probe")]
public async Task QuadletInstall_ChecksTheUnitHash_PointsTheImageAtTheConfiguredRepository_AndLeavesAutoUpdateOn(string role)
{
JsonNode created = await CreateAsync(role);
JsonNode quadlet = created["quadlet"]!;
string env = quadlet["env"]!.GetValue<string>();
string command = quadlet["command"]!.GetValue<string>();
string url = quadlet["url"]!.GetValue<string>();
Assert.EndsWith($"/install/quadlet/{role}.container", url, StringComparison.Ordinal);
Assert.Contains($"MOSHIRO_JOIN={created["plaintext"]!.GetValue<string>()}", env, StringComparison.Ordinal);
Assert.Contains("# MOSHIRO_UPDATE_POLICY=manual", env, StringComparison.Ordinal); // 默认自动更新,env 里只留一行注释说明怎么改成手动
Assert.DoesNotContain("\nMOSHIRO_UPDATE_POLICY=", env, StringComparison.Ordinal);
Assert.Contains($"~/.config/moshiro/{role}.env", command, StringComparison.Ordinal);
Assert.Contains("chmod 600", command, StringComparison.Ordinal);
Assert.Contains($"echo \"{quadlet["sha256"]!.GetValue<string>()} $HOME/.config/containers/systemd/moshiro-{role}.container\" | sha256sum -c -", command, StringComparison.Ordinal);
Assert.Contains($"Image=", command, StringComparison.Ordinal);
Assert.Contains($"/moshiro-{role}:latest", command, StringComparison.Ordinal);
Assert.EndsWith($"systemctl --user daemon-reload && systemctl --user start moshiro-{role}", command, StringComparison.Ordinal);
// 面板提供的就是命令里核对的那份:Pull=always(退出再被拉起就是更新),并告诉节点外面会重新拉取
using HttpResponseMessage served = await _backend.Anonymous.GetAsync($"/install/quadlet/{role}.container");
Assert.Equal(HttpStatusCode.OK, served.StatusCode);
byte[] body = await served.Content.ReadAsByteArrayAsync();
Assert.Equal(quadlet["sha256"]!.GetValue<string>(), Convert.ToHexStringLower(System.Security.Cryptography.SHA256.HashData(body)));
string unit = System.Text.Encoding.UTF8.GetString(body);
Assert.Contains("\nPull=always", unit, StringComparison.Ordinal);
Assert.Contains("Environment=MOSHIRO_UPDATE=restart", unit, StringComparison.Ordinal);
Assert.Contains("ReadOnly=true", unit, StringComparison.Ordinal);
Assert.Contains("DropCapability=ALL", unit, StringComparison.Ordinal);
Assert.Contains("NoNewPrivileges=true", unit, StringComparison.Ordinal);
Assert.DoesNotContain("jt_", unit, StringComparison.Ordinal);
}
[Theory]
[InlineData("evil.container")]
[InlineData("backend.container")]
[InlineData("agent")]
public async Task OnlyTheTwoQuadletUnits_AreServed(string file)
{
using HttpResponseMessage response = await _backend.Anonymous.GetAsync("/install/quadlet/" + file);
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
}
[Theory]
[InlineData("agent.yaml")]
[InlineData("probe.yaml")]
[InlineData("backend.yaml")]
public async Task TheComposeFiles_AreStatic_PinNothingToABuild_AndKeepTheNodeHardening(string file)
{
string body = await _backend.Anonymous.GetStringAsync("/install/compose/" + file);
Assert.Contains("${MOSHIRO_VERSION:-latest}", body, StringComparison.Ordinal); // 更新就是 pull 加 up,想固定版本在 .env 里写
Assert.Contains("read_only: true", body, StringComparison.Ordinal);
Assert.Contains("no-new-privileges:true", body, StringComparison.Ordinal);
Assert.Contains("- ALL", body, StringComparison.Ordinal); // cap_drop
Assert.DoesNotContain("jt_", body, StringComparison.Ordinal); // 没有任何一次性的值
// 参数只从 .env 原样传进容器:podman-compose 不替换环境变量里的默认值,写成 ${变量:-默认值} 会把字面量传给节点
Assert.Contains("env_file:", body, StringComparison.Ordinal);
Assert.DoesNotContain(":-}", body, StringComparison.Ordinal);
}
[Theory]
[InlineData("evil.yaml")]
[InlineData("agent")]
[InlineData("..%2Finstall.sh")]
public async Task OnlyTheThreeComposeFiles_AreServed(string file)
{
using HttpResponseMessage response = await _backend.Anonymous.GetAsync("/install/compose/" + file);
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
}
[Fact]
public async Task TheScripts_AreStatic_TheSameBytesForEveryone_AndNeedNoCode()
{
@@ -85,7 +207,7 @@ public sealed class InstallTests
public async Task TheScript_GetsTheSignedManifestOfItsRole_ByteForByte()
{
string manifest = """{"schema":1,"component":"probe","channel":"dev","image":"registry.example/moshiro-dev/moshiro-probe","digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000","tag":"dev-1","version":"0.1.1","commit":"abc","built_at":"2026-10-06T00:00:00Z","seq":1}""";
using HttpResponseMessage published = await _backend.Admin.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = "MEUCIQDc" }));
using HttpResponseMessage published = await _backend.Anonymous.PostAsync("/api/release/offer", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = TestReleaseKey.Sign(manifest) }));
Assert.True(published.StatusCode is HttpStatusCode.Created or HttpStatusCode.Conflict, await published.Content.ReadAsStringAsync());
string code = (await CreateAsync("probe"))["plaintext"]!.GetValue<string>();
@@ -4,7 +4,7 @@ using System.Text.RegularExpressions;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 对照 docs/api-guidelines.md 检查运行时生成的 OpenAPI 文档:operationId、路径风格、错误响应格式、字段命名。
/// 对照 docs/api/api-guidelines.md 检查运行时生成的 OpenAPI 文档:operationId、路径风格、错误响应格式、字段命名。
/// 新增端点违反规范时,这里会直接失败(等价于文档里的 <c>eng/check-openapi</c>)。
/// </summary>
[Collection("backend")]
@@ -53,7 +53,7 @@ public sealed partial class OpenApiConformanceTests
foreach ((string path, _) in doc["paths"]!.AsObject())
{
string[] prefixes = ["/api/public/", "/api/admin/", "/api/agent/v1/"];
string[] roots = ["/healthz", "/metrics", "/install", "/install.ps1", "/install/manifest", "/install/manifest.sig", "/api/release/offer"];
string[] roots = ["/healthz", "/metrics", "/install", "/install.ps1", "/install/manifest", "/install/manifest.sig", "/install/compose/{file}", "/install/quadlet/{file}", "/api/release/offer"];
Assert.True(prefixes.Any(p => path.StartsWith(p, StringComparison.Ordinal)) || roots.Contains(path), $"{path} is outside the documented layers");
foreach (string segment in path.Split('/', StringSplitOptions.RemoveEmptyEntries))
{
@@ -85,7 +85,7 @@ public sealed class SoftwarePasskey : IDisposable
private static byte[] Text(string text) => [.. Head(3, Encoding.UTF8.GetByteCount(text)), .. Encoding.UTF8.GetBytes(text)];
}
/// <summary>passkey 登录:注册、登录、拒绝各种错误的断言、只能管理自己的 passkey。设计见 docs/access.md 第 3 节。</summary>
/// <summary>passkey 登录:注册、登录、拒绝各种错误的断言、只能管理自己的 passkey。设计见 docs/security/access.md 第 4 节。</summary>
[Collection("probe-e2e")]
public sealed class PasskeyTests
{
@@ -39,7 +39,7 @@ public sealed class ProtocolDeclarationTests
public void EveryDeclaredMethod_IsDocumented()
{
string root = FindRoot();
string docs = File.ReadAllText(Path.Combine(root, "docs", "protocol.md")) + File.ReadAllText(Path.Combine(root, "backend", "fleet", "README.md"));
string docs = File.ReadAllText(Path.Combine(root, "docs", "design", "protocol.md")) + File.ReadAllText(Path.Combine(root, "backend", "fleet", "README.md"));
string[] missing = [.. s_table.Methods.Select(m => m.Name).Distinct().Where(n => !docs.Contains($"`{n}`", StringComparison.Ordinal))];
Assert.Empty(missing);
}
@@ -84,7 +84,7 @@ public sealed class ProtocolDeclarationTests
private static string FindRoot()
{
string? dir = AppContext.BaseDirectory;
while (dir is not null && !File.Exists(Path.Combine(dir, "docs", "protocol.md")))
while (dir is not null && !File.Exists(Path.Combine(dir, "docs", "design", "protocol.md")))
{
dir = Path.GetDirectoryName(dir);
}
@@ -6,7 +6,7 @@ using MoShiro.Protocol;
namespace MoShiro.Backend.Tests;
/// <summary>时间段、步长与数据源的规划:与 docs/query.md 里的表格逐行对应。</summary>
/// <summary>时间段、步长与数据源的规划:与 docs/design/query.md 里的表格逐行对应。</summary>
public sealed class QueryPlanningTests
{
private const long Second = 1000;
@@ -19,6 +19,8 @@ public sealed class ReleaseFeedTests : IAsyncLifetime
private WebApplication _registry = null!;
private string _registryUrl = "";
private string _label = "";
private string _backendLabel = "";
private string _backendSignature = "";
public async ValueTask InitializeAsync()
{
@@ -28,6 +30,10 @@ public sealed class ReleaseFeedTests : IAsyncLifetime
_registry.MapGet("/v2/moshiro-dev/moshiro-agent-manifest/manifests/dev", () => Results.Text(new JsonObject { ["schemaVersion"] = 2, ["config"] = new JsonObject { ["digest"] = ConfigDigest } }.ToJsonString(), "application/vnd.oci.image.manifest.v1+json"));
_registry.MapGet("/v2/moshiro-dev/moshiro-agent-manifest/blobs/" + ConfigDigest, () => Results.Text(
new JsonObject { ["config"] = new JsonObject { ["Labels"] = new JsonObject { ["org.moshiro.manifest"] = _label, ["org.moshiro.signature"] = "c2ln" } } }.ToJsonString(), "application/json"));
const string BackendConfigDigest = "sha256:0000000000000000000000000000000000000000000000000000000000000002";
_registry.MapGet("/v2/moshiro-dev/moshiro-backend-manifest/manifests/dev", () => Results.Text(new JsonObject { ["schemaVersion"] = 2, ["config"] = new JsonObject { ["digest"] = BackendConfigDigest } }.ToJsonString(), "application/vnd.oci.image.manifest.v1+json"));
_registry.MapGet("/v2/moshiro-dev/moshiro-backend-manifest/blobs/" + BackendConfigDigest, () => Results.Text(
new JsonObject { ["config"] = new JsonObject { ["Labels"] = new JsonObject { ["org.moshiro.manifest"] = _backendLabel, ["org.moshiro.signature"] = _backendSignature } } }.ToJsonString(), "application/json"));
await _registry.StartAsync();
_registryUrl = _registry.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First();
}
@@ -71,6 +77,56 @@ public sealed class ReleaseFeedTests : IAsyncLifetime
Assert.Null(await empty.FetchAsync("agent", "dev", default));
}
[Fact]
public async Task ThePanelFindsItsOwnNewRelease_InTheRegistry_AndLeavesItForTheHostUpdater()
{
// CI 把清单登记给面板那一步失败了(只警告):面板自己去仓库取,验过签就登记,并写进宿主机更新器的信箱
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds();
string manifest = $$"""{"schema":1,"component":"backend","channel":"dev","image":"registry.example/moshiro-dev/moshiro-backend","digest":"sha256:2222222222222222222222222222222222222222222222222222222222222222","tag":"dev-new","version":"0.1.1","seq":{{seq}}}""";
_backendLabel = Convert.ToBase64String(Encoding.UTF8.GetBytes(manifest));
_backendSignature = TestReleaseKey.Sign(manifest);
string directory = Path.Combine(Path.GetTempPath(), "moshiro-feed-" + Guid.NewGuid().ToString("N"));
await using WebApplication panel = BackendHost.Build(
["--urls", "http://127.0.0.1:0", "--Logging:LogLevel:Default=Warning"],
options =>
{
options.DataDirectory = directory;
options.Pbkdf2Iterations = 1000;
options.SecureCookies = false;
options.NodeImageRegistry = _registryUrl;
options.UpdateChannel = "dev";
options.GeoLookupUrl = string.Empty;
options.RequireOwner = false;
},
configureServices: services => services.AddSingleton<IReleaseKey>(TestReleaseKey.Instance));
await panel.StartAsync();
try
{
ReleaseService releases = panel.Services.GetRequiredService<ReleaseService>();
Assert.Null(await releases.GetBackendAsync("dev", default));
ReleaseOffer? offer = await releases.GetOrFetchAsync("backend", default);
Assert.Equal(seq, offer!.Seq);
Assert.Equal(seq, (await releases.GetBackendAsync("dev", default))!.Seq);
Assert.Equal(manifest, await File.ReadAllTextAsync(Path.Combine(directory, "update", "offer.manifest")));
}
finally
{
await panel.StopAsync();
await panel.DisposeAsync();
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
try
{
Directory.Delete(directory, recursive: true);
}
catch (IOException)
{
// 系统临时目录里的残留
}
}
}
[Fact]
public async Task AForgedManifestInTheRegistry_IsNeverServedToTheInstaller()
{
@@ -0,0 +1,123 @@
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json.Nodes;
using Microsoft.Extensions.DependencyInjection;
using MoShiro.Backend.Application;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 可信资产来源:官方发布密钥始终信任;部署的人可以另外导入自己的发布公钥(默认没有),用自己的私钥给自己构建的清单签名。
/// 面板只收公钥,拒绝私钥;删除之后只有它签过的清单不再被接受,已登记的也丢掉。
/// </summary>
[Collection("probe-e2e")]
public sealed class ReleaseKeyTests
{
private static long s_seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000 + 500_000;
private readonly BackendFixture _backend;
public ReleaseKeyTests(BackendFixture backend)
{
_backend = backend;
}
private static string Manifest(long seq) =>
$$"""{"schema":1,"component":"backend","channel":"dev","image":"registry.example/moshiro-dev/moshiro-backend","digest":"sha256:1111111111111111111111111111111111111111111111111111111111111111","tag":"own-{{seq}}","version":"0.1.1","seq":{{seq}}}""";
private static string Sign(ECDsa key, string manifest) =>
Convert.ToBase64String(key.SignData(Encoding.UTF8.GetBytes(manifest), HashAlgorithmName.SHA256, DSASignatureFormat.Rfc3279DerSequence));
private async Task<HttpResponseMessage> PutKeyAsync(string? pem) =>
await _backend.Admin.PutAsync("/api/admin/release-key", BackendFixture.Json(new JsonObject { ["public_key"] = pem }));
private async Task<HttpResponseMessage> OfferAsync(string manifest, string signature)
{
using HttpClient anonymous = _backend.NewClient();
return await anonymous.PostAsync("/api/release/offer", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = signature }));
}
[Fact]
public async Task ByDefaultOnlyTheOfficialKeyIsTrusted_AndItCannotBeRemoved()
{
using HttpResponseMessage cleared = await _backend.Admin.DeleteAsync("/api/admin/release-key");
Assert.Equal(HttpStatusCode.OK, cleared.StatusCode);
JsonNode keys = JsonNode.Parse(await cleared.Content.ReadAsStringAsync())!;
Assert.Null(keys["custom"]);
Assert.Equal(64, keys["official"]!["fingerprint"]!.GetValue<string>().Length);
Assert.StartsWith("-----BEGIN PUBLIC KEY-----", keys["official"]!["public_key"]!.GetValue<string>(), StringComparison.Ordinal);
}
[Theory]
[InlineData("")]
[InlineData("not a key")]
[InlineData(null)]
public async Task ABadKey_IsRefused(string? pem)
{
using HttpResponseMessage response = await PutKeyAsync(pem);
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
}
[Fact]
public async Task ThePrivateKey_IsNeverAccepted_AndOnlyP256IsAllowed()
{
using var p256 = ECDsa.Create(ECCurve.NamedCurves.nistP256);
using var p384 = ECDsa.Create(ECCurve.NamedCurves.nistP384);
using HttpResponseMessage privateKey = await PutKeyAsync(p256.ExportPkcs8PrivateKeyPem());
using HttpResponseMessage ecPrivateKey = await PutKeyAsync(p256.ExportECPrivateKeyPem());
using HttpResponseMessage otherCurve = await PutKeyAsync(p384.ExportSubjectPublicKeyInfoPem());
Assert.Equal(HttpStatusCode.BadRequest, privateKey.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, ecPrivateKey.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, otherCurve.StatusCode);
Assert.Contains("private key", await privateKey.Content.ReadAsStringAsync(), StringComparison.OrdinalIgnoreCase);
}
[Fact]
public async Task AManifestSignedByTheOwnKey_IsAccepted_UntilTheKeyIsRemoved()
{
using var own = ECDsa.Create(ECCurve.NamedCurves.nistP256);
long first = Interlocked.Increment(ref s_seq);
long second = Interlocked.Increment(ref s_seq);
// 还没有导入:自己的密钥签的清单不被接受
Assert.Equal(HttpStatusCode.Forbidden, (await OfferAsync(Manifest(first), Sign(own, Manifest(first)))).StatusCode);
using HttpResponseMessage set = await PutKeyAsync(own.ExportSubjectPublicKeyInfoPem());
Assert.Equal(HttpStatusCode.OK, set.StatusCode);
string fingerprint = Convert.ToHexStringLower(SHA256.HashData(own.ExportSubjectPublicKeyInfo()));
Assert.Equal(fingerprint, JsonNode.Parse(await set.Content.ReadAsStringAsync())!["custom"]!["fingerprint"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.Created, (await OfferAsync(Manifest(first), Sign(own, Manifest(first)))).StatusCode);
ReleaseService releases = _backend.Services.GetRequiredService<ReleaseService>();
Assert.Equal(first, (await releases.GetBackendAsync("dev", default))!.Seq);
// 别人的密钥照样不行
using var stranger = ECDsa.Create(ECCurve.NamedCurves.nistP256);
Assert.Equal(HttpStatusCode.Forbidden, (await OfferAsync(Manifest(second), Sign(stranger, Manifest(second)))).StatusCode);
// 删除之后:只有它签过的清单不再被接受,已经登记的也丢掉
using HttpResponseMessage cleared = await _backend.Admin.DeleteAsync("/api/admin/release-key");
Assert.Equal(HttpStatusCode.OK, cleared.StatusCode);
Assert.Null(JsonNode.Parse(await cleared.Content.ReadAsStringAsync())!["custom"]);
Assert.Equal(HttpStatusCode.Forbidden, (await OfferAsync(Manifest(second), Sign(own, Manifest(second)))).StatusCode);
Assert.Null(await releases.GetBackendAsync("dev", default));
}
[Fact]
public async Task OnlyAnAdministrator_MayChangeTheTrustedKeys()
{
using HttpClient anonymous = _backend.NewClient();
using var own = ECDsa.Create(ECCurve.NamedCurves.nistP256);
using HttpResponseMessage read = await anonymous.GetAsync("/api/admin/release-key");
using HttpResponseMessage write = await anonymous.PutAsync("/api/admin/release-key", BackendFixture.Json(new JsonObject { ["public_key"] = own.ExportSubjectPublicKeyInfoPem() }));
Assert.Equal(HttpStatusCode.Unauthorized, read.StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, write.StatusCode);
}
}
@@ -75,11 +75,13 @@ public sealed class ReleaseOfferTests
}
[Fact]
public async Task TheAdminEndpoint_StillNeedsTheCapability()
public async Task OfferWithAValidSignature_IsAccepted_WithoutAnyToken()
{
long seq = TimeProvider.System.GetUtcNow().ToUnixTimeSeconds() * 1000; // backend 的 release 通道:不和别的用例抢 agent、probe 的 seq
string manifest = $$"""{"schema":1,"component":"backend","channel":"release","image":"registry.example/moshiro/moshiro-backend","digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000","tag":"dev-x","version":"0.1.1","seq":{{seq}}}""";
using HttpClient anonymous = _backend.NewClient();
using HttpResponseMessage response = await anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest, ["signature"] = OpenSslSignature }));
Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode);
using HttpResponseMessage offered = await anonymous.PostAsync("/api/release/offer", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = TestReleaseKey.Sign(manifest) }));
Assert.Equal(HttpStatusCode.Created, offered.StatusCode);
}
}
@@ -0,0 +1,21 @@
using System.Security.Cryptography;
using System.Text;
using MoShiro.Backend.Application;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 测试用的发布密钥:真正的 CI 私钥不在仓库里,测试后端用这把公钥验签(<see cref="BackendFixture"/> 替换了内嵌的那把),
/// 登记清单的用例用它签名——和 CI 一样是 DER 编码的 ECDSA P-256 / SHA-256,签名对象是清单的原始字节。
/// </summary>
public sealed class TestReleaseKey : IReleaseKey
{
private static readonly ECDsa s_key = ECDsa.Create(ECCurve.NamedCurves.nistP256);
public static TestReleaseKey Instance { get; } = new();
public string Pem => s_key.ExportSubjectPublicKeyInfoPem();
public static string Sign(string manifest) =>
Convert.ToBase64String(s_key.SignData(Encoding.UTF8.GetBytes(manifest), HashAlgorithmName.SHA256, DSASignatureFormat.Rfc3279DerSequence));
}
@@ -15,7 +15,7 @@ namespace MoShiro.Backend.Tests;
/// <summary>
/// 信任根:操作员(passkey)签操作员集合;安装命令里带着创世指纹和属主(都是公开值),节点注册时验证面板的集合并固定在本地。
/// 设计见 docs/access.md 第 7 节。
/// 设计见 docs/security/access.md 第 8 节。
/// </summary>
[Collection("probe-e2e")]
public sealed class TrustTests
@@ -22,12 +22,15 @@ public sealed class UpdateTests
_backend = backend;
}
// 面板不验签(节点宿主机上的更新器验),所以测试里签名随便写一个格式对的
// 面板验发布签名(节点宿主机上的更新器之后还会再验):测试后端换成了测试密钥,清单用同一把签
private static string Manifest(string component, long seq, string digest = Digest) =>
$$"""{"schema":1,"component":"{{component}}","channel":"dev","image":"registry.example/moshiro-dev/moshiro-{{component}}","digest":"{{digest}}","tag":"dev-{{seq}}","version":"0.1.1","commit":"abc","built_at":"2026-10-06T00:00:00Z","seq":{{seq}}}""";
private async Task<HttpResponseMessage> PublishAsync(string manifest, string signature = "MEUCIQDc") =>
await _backend.Admin.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = signature }));
private async Task<HttpResponseMessage> PublishAsync(string manifest, string? signature = null)
{
signature ??= TestReleaseKey.Sign(manifest);
return await _backend.Anonymous.PostAsync("/api/release/offer", BackendFixture.Json(new JsonObject { ["manifest"] = manifest, ["signature"] = signature }));
}
[Fact]
public async Task Release_IsForwardedByteForByte_ToOnlineNodesOfThatComponent_AndToNodesThatConnectLater()
@@ -35,14 +38,15 @@ public sealed class UpdateTests
long seq = NextSeq();
await using AgentSimulator online = await _backend.ConnectAgentAsync();
string manifest = Manifest("agent", seq);
string signature = TestReleaseKey.Sign(manifest); // ECDSA 每次签出来都不一样:签一次,登记和比较共用
using HttpResponseMessage published = await PublishAsync(manifest);
using HttpResponseMessage published = await PublishAsync(manifest, signature);
Assert.Equal(HttpStatusCode.Created, published.StatusCode);
JsonNode release = JsonNode.Parse(await published.Content.ReadAsStringAsync())!;
Assert.Equal("agent", release["component"]!.GetValue<string>());
Assert.Equal(Digest, release["digest"]!.GetValue<string>());
await BackendFixture.EventuallyAsync(() => Task.FromResult(online.UpdateOffers.ToArray().Any(o => o.Manifest == manifest && o.Signature == "MEUCIQDc")), because: "the offer reaches online agents unchanged");
await BackendFixture.EventuallyAsync(() => Task.FromResult(online.UpdateOffers.ToArray().Any(o => o.Manifest == manifest && o.Signature == signature)), because: "the offer reaches online agents unchanged");
// 之后才连上的节点,hello 之后补发
await using AgentSimulator late = await _backend.ConnectAgentAsync();
@@ -56,54 +60,22 @@ public sealed class UpdateTests
}
[Fact]
public async Task Release_Registration_RefusesBadManifests_StaleSeq_AndNeedsAnAdmin()
public async Task Release_Registration_RefusesBadManifests_BadSignatures_AndStaleSeq()
{
long seq = NextSeq();
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync("{not json")).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("agent", seq, digest: "sha256:short"))).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("unknown", seq))).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await PublishAsync(Manifest("agent", seq), signature: "not base64!")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await PublishAsync(Manifest("agent", seq), signature: "not base64!")).StatusCode);
using var other = System.Security.Cryptography.ECDsa.Create(System.Security.Cryptography.ECCurve.NamedCurves.nistP256);
string foreign = Convert.ToBase64String(other.SignData(System.Text.Encoding.UTF8.GetBytes(Manifest("agent", seq)), System.Security.Cryptography.HashAlgorithmName.SHA256, System.Security.Cryptography.DSASignatureFormat.Rfc3279DerSequence));
Assert.Equal(HttpStatusCode.Forbidden, (await PublishAsync(Manifest("agent", seq), foreign)).StatusCode); // 别人的密钥签的
Assert.Equal(HttpStatusCode.Forbidden, (await PublishAsync(Manifest("agent", seq), "MEUCIQDc")).StatusCode); // 格式对、签名不对
Assert.Equal(HttpStatusCode.Forbidden, (await PublishAsync(Manifest("agent", seq), TestReleaseKey.Sign(Manifest("agent", seq + 5)))).StatusCode); // 签的是另一份清单
Assert.Equal(HttpStatusCode.Created, (await PublishAsync(Manifest("agent", seq))).StatusCode);
Assert.Equal(HttpStatusCode.Conflict, (await PublishAsync(Manifest("agent", seq))).StatusCode); // 同一个 seq:重放
Assert.Equal(HttpStatusCode.Conflict, (await PublishAsync(Manifest("agent", seq - 1))).StatusCode); // 更旧的 seq
using HttpResponseMessage anonymous = await _backend.Anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("agent", seq + 1), ["signature"] = "MEUC" }));
Assert.Contains(anonymous.StatusCode, new[] { HttpStatusCode.Unauthorized, HttpStatusCode.Forbidden });
}
[Fact]
public async Task ReleaseToken_CanOnlyRegisterReleases_AndAReadTokenCannotEvenDoThat()
{
async Task<string> NewTokenAsync(params string[] capabilities)
{
using HttpResponseMessage created = await _backend.Admin.PostAsync("/api/admin/api-tokens", BackendFixture.Json(new JsonObject { ["name"] = _backend.UniqueName("ci"), ["capabilities"] = BackendFixture.Capabilities(capabilities) }));
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
return JsonNode.Parse(await created.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
}
HttpClient Client(string token)
{
var client = new HttpClient { BaseAddress = _backend.BaseAddress };
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
return client;
}
long seq = NextSeq();
using HttpClient release = Client(await NewTokenAsync("release.publish"));
using HttpClient read = Client(await NewTokenAsync("public.read", "fleet.read"));
var body = BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq), ["signature"] = "MEUC" });
using HttpResponseMessage forbidden = await read.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq), ["signature"] = "MEUC" }));
Assert.Equal(HttpStatusCode.Forbidden, forbidden.StatusCode);
using HttpResponseMessage published = await release.PostAsync("/api/admin/releases", body);
Assert.Equal(HttpStatusCode.Created, published.StatusCode);
// release 令牌只能登记发布清单:读不了任何东西,也不能做别的管理操作
Assert.Equal(HttpStatusCode.Forbidden, (await release.GetAsync("/api/admin/agents")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await release.GetAsync("/api/admin/releases")).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await release.PostAsync("/api/admin/nodes/agt_x/update", null)).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await _backend.Anonymous.PostAsync("/api/admin/releases", BackendFixture.Json(new JsonObject { ["manifest"] = Manifest("probe", seq + 1), ["signature"] = "MEUC" }))).StatusCode);
}
[Fact]
-20
View File
@@ -1,20 +0,0 @@
#!/usr/bin/env bash
# 镜像里某个可执行文件的 SHA-256:ci/binary-hash.sh <镜像> <镜像里的路径>
# 节点握手时上报的就是它正在运行的那个文件的哈希,CI 把这个哈希写进签名的清单(binaries),Backend 据此认「官方二进制」。
# 先试在镜像里运行 sha256sum(同架构),不行再挂载镜像文件系统直接读。
set -euo pipefail
image="${1:?usage: binary-hash.sh <image> <path>}"; path="${2:?path in the image}"
# 镜像是本地刚构建的,不需要仓库认证;CI 里 REGISTRY_AUTH_FILE 指向的文件在登录之前还不存在,buildah 会因此报错
unset REGISTRY_AUTH_FILE
container="$(buildah from "$image")"
trap 'buildah rm "$container" >/dev/null 2>&1 || true' EXIT
if out="$(buildah run "$container" -- sha256sum "$path" 2>/dev/null)" && [[ "$out" =~ ^([0-9a-f]{64})[[:space:]] ]]; then
echo "${BASH_REMATCH[1]}"
exit 0
fi
buildah unshare bash -c '
set -euo pipefail
mount="$(buildah mount "$1")"
sha256sum "$mount$2" | cut -d" " -f1
buildah umount "$1" >/dev/null
' _ "$container" "$path"
+1 -6
View File
@@ -54,11 +54,6 @@ buildah bud --layers --network host --tls-verify=true --arch amd64 \
-f "$context/Containerfile" \
-t "$local_image" "$context"
# 镜像里节点二进制的哈希:写进签名的清单(binaries),Backend 凭它认「官方二进制」,节点握手时上报的就是它正在运行的这个文件的哈希
binary_sha256="$(bash "$root/ci/binary-hash.sh" "$local_image" "/usr/local/bin/moshiro-$component")"
[[ "$binary_sha256" =~ ^[0-9a-f]{64}$ ]] || { echo "could not hash the $component binary in the image" >&2; exit 1; }
echo "$component binary sha256: $binary_sha256"
logged_in=false
for attempt in 1 2 3 4; do
if buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$registry" <<< "$ZOT_SECRET" >/dev/null; then
@@ -91,5 +86,5 @@ done
# 发布清单:给镜像的 digest 签名(私钥只在 CI 密钥里),放进镜像仓库(<仓库>-manifest:dev),面板在线时再登记一份加速。
# 没有配置签名密钥(例如 fork)就只构建镜像、不发清单。
if [[ -n "${RELEASE_SIGNING_KEY:-}" ]]; then
RELEASE_BINARIES="$binary_sha256" bash "$root/ci/publish-release-offer.sh" "$component" "$repository" "$(cat "$work/$component.digest")" "$dev_tag" dev "$work/$component-release"
bash "$root/ci/publish-release-offer.sh" "$component" "$repository" "$(cat "$work/$component.digest")" "$dev_tag" dev "$work/$component-release"
fi
Loaded 100 of 159 files, more files were not shown because too many files have changed in this diff. Show more