88 Commits
Author SHA1 Message Date
justme 0e7a8c14fb Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-08 15:30:16 +08:00
justme 4a15dba8de Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline failed
ci/woodpecker/push/web-build unknown status
2026-10-08 06:47:15 +08:00
justme 8410735377 Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline failed
ci/woodpecker/push/web-build unknown status
2026-10-08 05:44:20 +08:00
justme 6310358490 Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-08 05:41:02 +08:00
justme 0dc8c6fe34 Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-08 04:23:16 +08:00
justme 1f70c8d31e Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-08 04:10:29 +08:00
justme 0e0b925c8b Merge branch 'main' into dev
ci/woodpecker/push/verify Pipeline failed
ci/woodpecker/push/web-build unknown status
2026-10-08 03:22:20 +08:00
justme f01bb8fd63 ci: sync main into dev [CI SKIP] 2026-10-07 20:03:38 +08:00
justme 96990c6f4e ci: sync main into dev [CI SKIP] 2026-10-07 19:14:58 +08:00
justme bd35a1a966 ci: sync main into dev [CI SKIP] 2026-10-07 18:13:20 +08:00
justme b47e01d944 ci: sync main into dev [CI SKIP] 2026-10-07 17:21:25 +08:00
justme 7ee860cb27 ci: sync main into dev [CI SKIP] 2026-10-07 16:58:59 +08:00
justme 699ed8c411 ci: sync main into dev [CI SKIP] 2026-10-07 16:20:25 +08:00
justme 0012df7b91 ci: sync main into dev [CI SKIP] 2026-10-07 15:08:26 +08:00
justme d5967b6110 ci: sync main into dev [CI SKIP] 2026-10-07 14:57:08 +08:00
justme 450748f3bc ci: sync main into dev [CI SKIP] 2026-10-07 14:18:03 +08:00
justme 5bd53f8880 ci: sync main into dev [CI SKIP] 2026-10-07 03:36:55 +08:00
justme 643f9e1ea3 ci: sync main into dev [CI SKIP] 2026-10-07 03:30:19 +08:00
justme ac0bcd8e1f docs: merge the Fleet docs into backend/fleet/README.md, add TODO.md, fix stale references [CI SKIP] 2026-10-07 03:30:03 +08:00
justme f4aa3f2482 docs: audit the docs, merge the Fleet docs into backend/fleet/README.md and add TODO.md [CI SKIP]
Drop the unimplemented fleet updater design, the stale project-layout sections and broken references, fix the API table, and rename the four Monitor*.cs files left over from the rename.
2026-10-07 03:29:05 +08:00
justme 180e54cb3c ci: sync main into dev [CI SKIP] 2026-10-07 03:12:48 +08:00
justme a19517a57d docs: rewrite the README: scope, trust model, architecture, protocol and build [SKIP TESTS]
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-07 01:51:54 +08:00
justme 83e5120377 refactor!: rename the project from Monitor to MoShiro
Namespaces, assemblies, projects and directories, the solution, binaries,
images, systemd units, config paths, environment variables, protocol method
names, the media type and the raw chunk magic all move to MoShiro/moshiro.
Existing data and deployments are not compatible and must be recreated.
Also fix the architecture tests after the solution layout change and make
the update tests independent of their order.
2026-10-07 01:49:04 +08:00
justme 17b35223aa ci: sync main into dev [CI SKIP] 2026-10-07 01:46:35 +08:00
justme 4d329f5e16 ci: sync main into dev [CI SKIP] 2026-10-07 01:30:46 +08:00
justme ad17170664 docs: redraw the architecture diagrams and update the README [SKIP TESTS]
ci/woodpecker/push/verify Pipeline was successful
ci/woodpecker/push/web-build Pipeline was successful
2026-10-07 01:30:28 +08:00
justme 710aa9e99f ci: sync main into dev [CI SKIP] 2026-10-07 01:28:32 +08:00
justme 71d2866b9c docs: update the architecture [SKIP TESTS] 2026-10-07 01:21:49 +08:00
justme 48664c0ada docs: remove the roadmap and bring the docs in line with the current code [SKIP TESTS]
Probes link to the panel directly, container nodes update through signed
releases, the frontend exists and the ports changed. The roadmap is gone; its
open items live in the README.
2026-10-06 23:27:26 +08:00
justme 198ee556d6 ci: sync main into dev [CI SKIP] 2026-10-06 23:22:52 +08:00
justme da465866d5 fix(web): lift the hovered overview card above its neighbours so the popover is not hidden 2026-10-06 23:22:36 +08:00
justme 3a751a0f89 feat: choose which probes link to which agents from the admin
The panel decides the probe/agent pairs and probes always dial out. A new
fleet.auto_link setting connects every probe to every agent (default); with it
off only the pairs chosen by hand connect, and any pair can be forced on or
off. The Probes page has a link matrix with the switch and per-pair toggles.
2026-10-06 23:19:32 +08:00
justme d69581dcb5 docs: mount the host os-release and fix the agent public host after tunnel enrollment [SKIP TESTS] 2026-10-06 23:11:26 +08:00
justme 8f44e924a4 docs: describe container node deployment and the update flow [SKIP TESTS] 2026-10-06 23:02:16 +08:00
justme 0c39485a78 ci: sync main into dev [CI SKIP] 2026-10-06 22:57:11 +08:00
justme 7a71bf57f9 feat: update controls in the admin, release token scope, build info in CI images
The versions page shows each node's update state with update buttons, API
tokens get a release scope that can only register releases (used by CI), and
the CI image and release builds now pass the tag, commit and build time.
2026-10-06 22:56:56 +08:00
justme 22a98f2eb0 ci: sync main into dev [CI SKIP] 2026-10-06 22:42:20 +08:00
justme 0a66f5e07a feat: signed releases and a host-side updater for container nodes
CI signs the image digest of each agent and probe release; the panel only
stores and forwards it. Nodes write the manifest to their data directory and
a host updater verifies the signature with a local public key, pulls by
digest, restarts the unit and rolls back when it is unhealthy. The panel can
list releases, ask a node to update and show the reported update state.
2026-10-06 22:37:26 +08:00
justme 48072737c9 feat!: probes connect to the backend directly
A probe now links to the backend like an agent (probe.json carries the
server address). The backend sends the agent list as desired state and the
probe reports its platform and build in its own hello. The agent no longer
relays the list or the probe platform. Existing probes must be re-created.
2026-10-06 22:25:16 +08:00
justme 604cbe3123 fix(ci): restore with PublishAot so node images are native, add compose files
Agent and probe images were built with a restore step that lacked
PublishAot, so the publish silently produced a trimmed launcher that looks
for a .dll. Restore now passes PublishAot and the build fails if the binary
is smaller than 2 MB. Also add agent and probe compose files and document
the [CI] marker in AGENTS.md.
2026-10-06 21:43:46 +08:00
justme b242701e9e ci: sync main into dev [CI SKIP] 2026-10-06 21:24:47 +08:00
justme 83f54978d2 feat(web): acrylic theme, node flags and icons, build info
Acrylic cards and topbar over a blurred background, country flags, ISP and
OS icons, per-node version tags, a build info footer and an admin versions
tab. Overview cards drop the sparkline and widen the hover area.
2026-10-06 21:22:04 +08:00
justme 1e5c660d05 feat(backend): report node platform, geo and build info
Agents and probes report OS, arch and build (version, tag, commit, time).
The backend looks up country and ISP of public addresses only, stores them
in node_meta (schema v5) and exposes display info via /api/public/nodes,
/api/public/about and /api/admin/build-info behind site.show_build.
2026-10-06 21:22:04 +08:00
justme 2e470607fa docs: add project agent instructions [SKIP TESTS] 2026-10-06 17:11:07 +08:00
justme a70b89b95c feat(web): show UDP and TCP strips on overview cards
Add 3/6/12 hour ranges with status strips, latency sparklines and a side popover for the hovered interval.
2026-10-06 16:59:17 +08:00
justme a9538eae4a feat(web): add backend switcher and connection status
Split the login page into backend list and credentials, show connection status in the sidebar and top bar, and render dialogs in a portal.
2026-10-06 16:59:17 +08:00
justme 91c63d0cf2 feat(api): add return_session login for cross-origin clients
Return the session token in the login response instead of a cookie, and accept it as a Bearer token without CSRF.
2026-10-06 16:59:16 +08:00
justme 193fc3a95c feat: improve agent management and speed up tests 2026-10-06 15:48:46 +08:00
justme 134c06d605 merge: bring line detail timeline into dev [CI SKIP] 2026-10-06 14:52:48 +08:00
justme d1be70cda2 feat(web): 线路详情页的时间轴、EWMA 平滑和加载进度
- 先选日期,再在连续 48 小时(或 24 小时)的丢包缩略图上拖选区;选区可以跨零点,
  点轴或刻度时离得近的端点贴过去,悬停预览会动的端点;没有数据的时段画成斜纹
- 刻度下方的时长快捷档:30 分钟、1 小时、3 小时、6 小时、12 小时、整天
- 指标卡片平铺,丢包等标明 UDP/TCP;图表按 UDP、TCP、评分分组
- 图表数据刷新时原地更新(不再整张重建),毫秒到 1 秒以上显示成秒
- EWMA 平滑开关(关/轻/中/重),原始数据以淡线垫底
- 数据加载时卡片上显示进度条和占位;useQuery 区分定时刷新和用户触发的加载
- 前端可以在页面里指定后端地址和令牌(配合后端 Monitor__CorsOrigins)
2026-10-06 14:48:58 +08:00
justme 672f6e3dcb fix(ci): emit native apphosts in fastlane images [CI SKIP] 2026-10-05 10:42:34 -07:00
justme 9a375ffd9e fix(build): select AotAnywhere explicitly [CI SKIP] 2026-10-05 10:37:10 -07:00
justme 407c09f6a5 fix(ci): use host networking for native builds [CI SKIP] 2026-10-05 10:30:35 -07:00
justme fffee8937c ci: sync main into dev [CI SKIP] 2026-10-05 10:26:53 -07:00
justme 91485db500 fix(ci): fall back to git diff for empty path lists [CI SKIP] 2026-10-05 10:25:59 -07:00
justme fa34869f18 ci: sync main into dev [CI SKIP] 2026-10-05 10:21:22 -07:00
justme deb5a97802 ci: add dependency-aware main fastlane [CI SKIP] 2026-10-05 10:18:22 -07:00
justme 479f8c25b0 feat(ci): publish agent and probe OCI images [CI SKIP] 2026-10-05 09:33:22 -07:00
justme 2b80bb66d1 ci: use local Gitea endpoint for release uploads 2026-10-05 08:44:19 -07:00
justme 078eacf1cb ci: persist Buildah cache for Linux releases 2026-10-05 08:29:08 -07:00
justme 3db6d5b159 ci: authenticate Zot with API key for justme [CI SKIP] 2026-10-05 22:24:16 +08:00
justme a7a8212c08 ci: push images to public Zot over HTTPS [CI SKIP] 2026-10-05 16:28:15 +08:00
justme 68406843de ci: retry transient Zot registry errors [CI SKIP] 2026-10-04 13:26:09 -07:00
justme fae8c45c90 ci: use bounded runspaces for artifact uploads [CI SKIP] 2026-10-04 13:11:01 -07:00
justme 3bf12ec73d ci: upload release artifacts concurrently [CI SKIP] 2026-10-04 13:05:17 -07:00
justme a9ee4ec34c build: disable PDB debug symbols in releases [CI SKIP] 2026-10-04 12:54:39 -07:00
justme 6b12049df0 ci: drop 32-bit release targets [CI SKIP] 2026-10-04 12:44:32 -07:00
justme d4b4869501 fix(ci): package cross-platform tarballs on Windows [CI SKIP] 2026-10-04 12:38:13 -07:00
justme 92b72d4443 ci: release without blocking on test workflow [CI SKIP] 2026-10-04 12:31:27 -07:00
justme 67b428991f ci: parallelize NativeAOT target builds 2026-10-04 12:29:35 -07:00
justme 55756f7695 ci: cross-publish full NativeAOT matrix from Windows 2026-10-04 12:10:06 -07:00
justme 17a69c1c7a ci: parallelize release components 2026-10-04 11:44:41 -07:00
justme 50d8c07939 ci: gate parallel releases on verified commits 2026-10-05 02:39:21 +08:00
justme 2e549b230d build: use ASP.NET runtime for backend images 2026-10-05 02:24:15 +08:00
justme ace42e5da7 fix(ci): use POST for Gitea release assets 2026-10-05 02:16:14 +08:00
justme ad32afd038 ci: run release builds in parallel 2026-10-05 02:11:29 +08:00
justme 286c9ff7cd ci: publish release assets to Gitea 2026-10-05 02:06:36 +08:00
justme 8aaa574485 fix(ci): make release retryable 2026-10-05 01:21:37 +08:00
justme ce30828ed3 ci: use runner-local Gitea package endpoints 2026-10-05 01:10:17 +08:00
justme a46f45cce3 ci: order multiarch index after builds 2026-10-05 01:05:24 +08:00
justme ce3ba51c02 test: normalize project reference paths across platforms 2026-10-05 00:49:20 +08:00
justme faf1203f0a ci: disable git lfs in clone steps 2026-10-05 00:43:46 +08:00
justme ee422ca366 ci: run verification for push, pull requests and tags 2026-10-05 00:41:29 +08:00
justme c33003c279 ci: restore public Gitea package endpoint 2026-10-04 23:11:37 +08:00
justme 31af31b96f ci: use internal Gitea endpoint 2026-10-04 23:01:59 +08:00
justme 9b6c4e0fd8 ci: skip host OpenAPI generation for arm64 2026-10-04 22:38:26 +08:00
justme ceef543854 ci: verify development branch 2026-10-04 22:20:58 +08:00
justme 299f86dbee ci: configure Woodpecker release pipelines 2026-10-04 22:12:21 +08:00
20 changed files with 22 additions and 505 deletions

No files matched your search

-5
View File
@@ -27,9 +27,6 @@ steps:
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
RELEASE_API_URL: https://mo.oeo.one
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
commands:
- bash ci/release-node-images.sh
depends_on: []
@@ -40,8 +37,6 @@ steps:
GITEA_BASE: http://127.0.0.1:3100
GITEA_TOKEN:
from_secret: gitea_token
RELEASE_SIGNING_KEY:
from_secret: release_signing_key
commands:
- bash ci/publish-gitea-release.sh
depends_on:
+2 -15
View File
@@ -15,17 +15,6 @@ when:
concurrency: 1
steps:
# 前端先打包好(几十秒),两个架构的 Backend 构建再并行:它们把 web/*/dist 内嵌进程序,只读这份产物,不再各自 npm ci
# (并行的两个 npm ci 会同时删 node_modules,互相搞坏)。步骤加了超时:卡住就几分钟内失败,不是耗着。
- name: web-build
image: bash
environment:
NPM_CONFIG_CACHE: /home/justme/.npm
commands:
- npm ci --prefix web --prefer-offline
- timeout 600 npm run build --prefix web
depends_on: []
- name: linux-amd64-backend
image: bash
environment:
@@ -36,8 +25,7 @@ steps:
from_secret: zot_secret
commands:
- bash ci/release-linux.sh amd64 backend
depends_on:
- web-build
depends_on: []
- name: linux-arm64-backend
image: bash
@@ -49,8 +37,7 @@ steps:
from_secret: zot_secret
commands:
- bash ci/release-linux.sh arm64 backend
depends_on:
- web-build
depends_on: []
- name: backend-multiarch-index
image: bash
+1 -1
View File
@@ -13,7 +13,7 @@
<UseArtifactsOutput>true</UseArtifactsOutput>
<ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath>
<Deterministic>true</Deterministic>
<Version>0.3.3</Version>
<Version>0.2.0</Version>
<NuGetAudit>false</NuGetAudit>
</PropertyGroup>
<ItemGroup>
@@ -42,7 +42,7 @@ public static class OpsEndpoints
{
string baseUrl = options.PublicUrl ?? $"{http.Request.Scheme}://{http.Request.Host}";
InstallTarget? target = string.IsNullOrEmpty(code) ? null : await tokens.ResolveInstallAsync(code, baseUrl, http.RequestAborted);
ReleaseOffer? offer = target is null ? null : await releases.GetOrFetchAsync(target.Role, http.RequestAborted);
ReleaseOffer? offer = target is null ? null : await releases.GetAsync(target.Role, http.RequestAborted);
if (offer is null)
{
return Problems.Create(http, MoShiroErrors.NotFound, "no signed release is registered for this install code");
@@ -29,14 +29,12 @@ public sealed class BinaryBindingService : IBinaryAttestor
public const string Off = "off";
private readonly INodeBindingStore _store;
private readonly ISettingsStore _settings;
private readonly IReleaseKey _key;
private readonly BackendOptions _options;
private readonly TimeProvider _time;
public BinaryBindingService(INodeBindingStore store, ISettingsStore settings, IReleaseKey key, BackendOptions options, TimeProvider time)
public BinaryBindingService(INodeBindingStore store, IReleaseKey key, BackendOptions options, TimeProvider time)
{
_settings = settings;
_store = store;
_key = key;
_options = options;
@@ -74,22 +72,18 @@ public sealed class BinaryBindingService : IBinaryAttestor
return new BinaryDecision(policy != Enforce, "the reported binary hash is malformed");
}
// 节点自己带来的清单(宿主机更新器验过的那份);没有更新器的节点(一键容器安装、二进制安装)没有,就用面板自己手里的:
// 面板登记它们时已经验过 CI 签名,这里再验一遍。只有节点自带的清单才能因为「不在清单里」而被拒绝,面板手里的只用来把绑定标成已签名。
Official? official = VerifyManifest(evidence.Manifest, evidence.Signature, component);
bool carried = official is not null;
official ??= await StoredOfficialAsync(component, ct);
Official? official = VerifyManifest(evidence, component);
bool listed = official is not null && official.Binaries.Contains(hash);
if (binding is null)
{
if (carried && !listed)
if (official is not null && !listed)
{
// 带着一份有效的签名清单,自己的哈希却不在里面:它不是这份清单对应的官方二进制
return new BinaryDecision(policy != Enforce, "this binary is not in the CI-signed manifest it was installed from");
}
await _store.UpsertAsync(new NodeBinding(node.Id.Value, hash, listed ? official!.Seq : 0, listed, now, null, null), ct);
await _store.UpsertAsync(new NodeBinding(node.Id.Value, hash, official?.Seq ?? 0, listed, now, null, null), ct);
return new BinaryDecision(true);
}
@@ -126,13 +120,10 @@ public sealed class BinaryBindingService : IBinaryAttestor
private sealed record Official(long Seq, HashSet<string> Binaries);
/// <summary>用内嵌的 CI 公钥验清单;验过且是这个组件的才返回它列出的官方二进制哈希。</summary>
private async ValueTask<Official?> StoredOfficialAsync(string component, CancellationToken ct) =>
VerifyManifest(await _settings.GetAsync(ReleaseService.Key(component, "manifest"), ct), await _settings.GetAsync(ReleaseService.Key(component, "signature"), ct), component);
private Official? VerifyManifest(string? manifestText, string? signatureText, string component)
private Official? VerifyManifest(BinaryEvidence evidence, string component)
{
if (manifestText is not { Length: > 0 and <= ReleaseService.MaxManifestBytes } manifest
|| signatureText is not { Length: > 0 and <= ReleaseService.MaxSignatureBytes } signature
if (evidence.Manifest is not { Length: > 0 and <= ReleaseService.MaxManifestBytes } manifest
|| evidence.Signature is not { Length: > 0 and <= ReleaseService.MaxSignatureBytes } signature
|| !ReleaseSignature.Verify(_key.Pem, manifest, signature))
{
return null;
@@ -39,7 +39,6 @@ public static class ApplicationServiceCollectionExtensions
services.AddSingleton<AuditService>();
services.AddSingleton<ServerIdentityService>();
services.AddSingleton<SetupService>();
services.AddSingleton<ReleaseFeed>();
services.AddSingleton<BinaryBindingService>();
services.AddSingleton<NodeOwnerService>();
services.AddSingleton<Fleet.Server.IBinaryAttestor>(sp => sp.GetRequiredService<BinaryBindingService>());
@@ -1,90 +0,0 @@
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
namespace MoShiro.Backend.Application;
/// <summary>
/// 镜像仓库里的签名清单:CI 每次构建都把签名清单放在 <c>&lt;命名空间&gt;/moshiro-&lt;组件&gt;-manifest:&lt;通道&gt;</c> 这个没有文件层的小镜像的标签里
/// (<c>org.moshiro.manifest</c> 是 base64 的清单,<c>org.moshiro.signature</c> 是签名)。面板没有登记过这个组件的清单时(清过库、CI 登记那一步网络失败),
/// 直接按 OCI 分发协议匿名读取它——只读两个小 JSON,不拉任何镜像层。取回来的只是候选:调用方要用 CI 的发布公钥验过签才采用,所以仓库被改也只会验签失败。
/// </summary>
public sealed partial class ReleaseFeed : IDisposable
{
private const int MaxBytes = 1 << 20;
private readonly BackendOptions _options;
private readonly HttpClient _http;
public ReleaseFeed(BackendOptions options)
{
_options = options;
_http = new HttpClient { Timeout = TimeSpan.FromSeconds(10) };
}
public void Dispose() => _http.Dispose();
/// <summary>取不到(仓库不可读、没有这个标签、格式不对)返回 null;不抛异常。</summary>
public async ValueTask<(string Manifest, string Signature)?> FetchAsync(string component, string channel, CancellationToken ct)
{
string registry = (_options.NodeImageRegistry ?? "").Trim().TrimEnd('/');
if (registry.Length == 0)
{
return null;
}
string baseUrl = registry.StartsWith("http://", StringComparison.OrdinalIgnoreCase) || registry.StartsWith("https://", StringComparison.OrdinalIgnoreCase) ? registry : "https://" + registry;
string repository = $"{(channel == "dev" ? "moshiro-dev" : "moshiro")}/moshiro-{component}-manifest";
try
{
string manifestJson = await GetAsync($"{baseUrl}/v2/{repository}/manifests/{channel}", "application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json", ct);
using JsonDocument manifest = JsonDocument.Parse(manifestJson);
string? digest = manifest.RootElement.GetProperty("config").GetProperty("digest").GetString();
if (digest is null || !Digest().IsMatch(digest))
{
return null;
}
string configJson = await GetAsync($"{baseUrl}/v2/{repository}/blobs/{digest}", "application/json", ct);
using JsonDocument config = JsonDocument.Parse(configJson);
JsonElement labels = config.RootElement.GetProperty("config").GetProperty("Labels");
string? encoded = labels.GetProperty("org.moshiro.manifest").GetString();
string? signature = labels.GetProperty("org.moshiro.signature").GetString();
return encoded is null || signature is null ? null : (Encoding.UTF8.GetString(Convert.FromBase64String(encoded)), signature);
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException or KeyNotFoundException or InvalidOperationException or FormatException)
{
return null;
}
}
private async Task<string> GetAsync(string url, string accept, CancellationToken ct)
{
using var request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.TryAddWithoutValidation("Accept", accept);
using HttpResponseMessage response = await _http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, ct);
response.EnsureSuccessStatusCode();
if (response.Content.Headers.ContentLength is > MaxBytes)
{
throw new InvalidOperationException("the registry response is too large");
}
await using Stream stream = await response.Content.ReadAsStreamAsync(ct);
using var limited = new MemoryStream();
byte[] buffer = new byte[8192];
int read;
while ((read = await stream.ReadAsync(buffer, ct)) > 0)
{
limited.Write(buffer, 0, read);
if (limited.Length > MaxBytes)
{
throw new InvalidOperationException("the registry response is too large");
}
}
return Encoding.UTF8.GetString(limited.ToArray());
}
[GeneratedRegex("^sha256:[0-9a-f]{64}$")]
private static partial Regex Digest();
}
@@ -26,15 +26,10 @@ public sealed partial class ReleaseService : IDisposable
private readonly IAgentGateway _gateway;
private readonly IBackendUpdateFiles _backendFiles;
private readonly BackendOptions _options;
private readonly IReleaseKey _key;
private readonly ReleaseFeed _feed;
private readonly ConcurrentDictionary<string, long> _feedChecked = new();
private readonly ConcurrentDictionary<string, UpdateStatus> _status = new();
public ReleaseService(ISettingsStore settings, IAgentGateway gateway, IBackendUpdateFiles backendFiles, BackendOptions options, IReleaseKey key, ReleaseFeed feed)
public ReleaseService(ISettingsStore settings, IAgentGateway gateway, IBackendUpdateFiles backendFiles, BackendOptions options)
{
_key = key;
_feed = feed;
_settings = settings;
_gateway = gateway;
_backendFiles = backendFiles;
@@ -89,33 +84,6 @@ public sealed partial class ReleaseService : IDisposable
}
}
/// <summary>
/// 这个组件的清单;面板里没有(清过库、CI 登记那一步网络失败)或者已经放了一阵,就去镜像仓库里取一次最新的:
/// 取回来的清单要用 CI 的发布公钥验过签才登记,所以仓库被改也只会被拒绝。取不到就用手里有的。
/// </summary>
public async ValueTask<ReleaseOffer?> GetOrFetchAsync(string component, CancellationToken ct)
{
ReleaseOffer? stored = await GetAsync(component, ct);
long now = Environment.TickCount64;
long wait = stored is null ? 30_000 : 600_000; // 没有就 30 秒试一次,已经有了就隔 10 分钟看一眼有没有更新的
if (_feedChecked.TryGetValue(component, out long last) && now - last < wait)
{
return stored;
}
_feedChecked[component] = now;
if (await _feed.FetchAsync(component, BackendUpdates.ChannelOf(_options), ct) is { } fetched && ReleaseSignature.Verify(_key.Pem, fetched.Manifest, fetched.Signature))
{
Result<ReleaseOffer> published = await PublishAsync(fetched.Manifest, fetched.Signature, ct);
if (published.IsSuccess)
{
return published.Value;
}
}
return stored;
}
public async ValueTask<ReleaseOffer?> GetAsync(string component, CancellationToken ct)
{
string? manifest = await _settings.GetAsync(Key(component, "manifest"), ct);
@@ -182,7 +150,7 @@ public sealed partial class ReleaseService : IDisposable
: node.Value.StartsWith(ProbeId.Prefix, StringComparison.Ordinal) ? NodeKinds.Probe
: null;
internal static string Key(string component, string part) => $"release.{component}.{part}";
private static string Key(string component, string part) => $"release.{component}.{part}";
private static bool TryParse(string manifest, string signature, [NotNullWhen(true)] out ReleaseOffer? offer, [NotNullWhen(false)] out string? problem)
{
@@ -22,7 +22,6 @@ public sealed class BinaryBindingTests : IDisposable
private readonly ECDsa _ci = ECDsa.Create(ECCurve.NamedCurves.nistP256);
private readonly ECDsa _other = ECDsa.Create(ECCurve.NamedCurves.nistP256);
private readonly MemoryStore _store = new();
private readonly MemorySettings _settings = new();
private readonly NodeRecord _agent = Node(AgentIds.Prefix + "_test1");
public void Dispose()
@@ -36,7 +35,7 @@ public sealed class BinaryBindingTests : IDisposable
private static NodeRecord Node(string id) => new(new NodeId(id), "n", NodeStatus.Active, [1], [2], [], null, null, 0, null);
private BinaryBindingService Service(string policy = "enforce") =>
new(_store, _settings, new TestKey(_ci), new BackendOptions { BinaryPolicy = policy }, TimeProvider.System);
new(_store, new TestKey(_ci), new BackendOptions { BinaryPolicy = policy }, TimeProvider.System);
private sealed class TestKey(ECDsa key) : IReleaseKey
{
@@ -209,86 +208,6 @@ public sealed class BinaryBindingTests : IDisposable
Assert.True((await service.CheckAsync(second, Evidence(H('b')), default)).Accept);
}
/// <summary>面板手里的、CI 签名的清单(登记时存下的):没有更新器的节点握手时没有自己的清单,就拿它来认官方二进制。</summary>
private void PanelHolds(string component, long seq, string[] binaries, ECDsa? signer = null)
{
string manifest = $"{{\"schema\":1,\"component\":\"{component}\",\"seq\":{seq},\"binaries\":\"{string.Join(',', binaries)}\"}}";
_settings.Values["release." + component + ".manifest"] = manifest;
_settings.Values["release." + component + ".signature"] = Sign(signer ?? _ci, manifest);
}
[Fact]
public async Task ANodeWithoutAnUpdater_IsMarkedSigned_WhenThePanelHoldsAManifestThatListsItsBinary()
{
// 一键容器安装的节点没有宿主机更新器,握手时带不来清单:面板用自己手里(登记时验过 CI 签名)的那份
PanelHolds("agent", seq: 100, [H('a')]);
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
NodeBinding bound = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.True(bound.Signed);
Assert.Equal(100, bound.Seq);
}
[Fact]
public async Task TheManifestThePanelHolds_NeverRefusesANodeAtFirstBinding_OnlyLabelsIt()
{
// 老一点的官方版本、自己构建的版本:不在面板手里最新的清单里,但没有证据说它不是官方的,所以不拒绝,只是不标成已签名
PanelHolds("agent", seq: 100, [H('b')]);
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
NodeBinding bound = (await _store.GetAsync(_agent.Id.Value, default))!;
Assert.False(bound.Signed);
Assert.Equal(0, bound.Seq);
}
[Fact]
public async Task AManifestThePanelHoldsThatIsSignedByAnotherKey_ProvesNothing()
{
PanelHolds("agent", seq: 100, [H('a')], signer: _other);
BinaryBindingService service = Service();
Assert.True((await service.CheckAsync(_agent, Evidence(H('a')), default)).Accept);
Assert.False((await _store.GetAsync(_agent.Id.Value, default))!.Signed);
}
[Fact]
public async Task AnUpgradeOfANodeWithoutAnUpdater_IsFollowedFromTheManifestThePanelHolds()
{
PanelHolds("agent", seq: 100, [H('a')]);
BinaryBindingService service = Service();
await service.CheckAsync(_agent, Evidence(H('a')), default);
// 之后发布了新版,面板登记了新的清单;节点换成新二进制重连
PanelHolds("agent", seq: 200, [H('b')]);
Assert.True((await service.CheckAsync(_agent, Evidence(H('b')), default)).Accept);
Assert.Equal(200, (await _store.GetAsync(_agent.Id.Value, default))!.Seq);
// 没人签过的二进制仍然被拒绝
Assert.False((await service.CheckAsync(_agent, Evidence(H('c')), default)).Accept);
}
private sealed class MemorySettings : ISettingsStore
{
public Dictionary<string, string> Values { get; } = [];
public ValueTask<string?> GetAsync(string key, CancellationToken ct) => new(Values.GetValueOrDefault(key));
public ValueTask SetAsync(string key, string value, CancellationToken ct)
{
Values[key] = value;
return ValueTask.CompletedTask;
}
public ValueTask<IReadOnlyDictionary<string, string>> GetAllAsync(CancellationToken ct) => new(Values);
public ValueTask<long> IncrementAsync(string key, CancellationToken ct) => new(0);
}
private sealed class MemoryStore : INodeBindingStore
{
private readonly Dictionary<string, NodeBinding> _rows = [];
@@ -91,9 +91,6 @@ public sealed class OwnerEndToEndTests : IDisposable
string script = await anonymous.GetStringAsync(path);
Assert.Contains("MOSHIRO_ALLOW_NO_OWNER", script, StringComparison.Ordinal);
Assert.Contains("this node has no owner", script, StringComparison.Ordinal);
// 只按容器判断「已经装过」:podman 的 inspect 不分类型,本机残留的旧镜像会被误当成已安装的容器
Assert.Contains("container inspect", script, StringComparison.Ordinal);
Assert.DoesNotContain("inspect \"$container\"", script.Replace("container inspect", ""), StringComparison.Ordinal);
}
}
}
@@ -1,123 +0,0 @@
using System.Net;
using System.Text;
using System.Text.Json.Nodes;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using MoShiro.Backend.Application;
namespace MoShiro.Backend.Tests;
/// <summary>
/// 面板没有登记过清单时(清过库、CI 登记那一步网络失败),直接从镜像仓库里匿名读取签名清单;取回来的要用 CI 的发布公钥验过才采用,
/// 所以仓库(或者路上的人)给一份伪造的,面板拒绝,安装脚本也就得不到它。
/// </summary>
public sealed class ReleaseFeedTests : IAsyncLifetime
{
private WebApplication _registry = null!;
private string _registryUrl = "";
private string _label = "";
public async ValueTask InitializeAsync()
{
WebApplicationBuilder builder = WebApplication.CreateSlimBuilder(["--urls", "http://127.0.0.1:0"]);
_registry = builder.Build();
const string ConfigDigest = "sha256:0000000000000000000000000000000000000000000000000000000000000001";
_registry.MapGet("/v2/moshiro-dev/moshiro-agent-manifest/manifests/dev", () => Results.Text(new JsonObject { ["schemaVersion"] = 2, ["config"] = new JsonObject { ["digest"] = ConfigDigest } }.ToJsonString(), "application/vnd.oci.image.manifest.v1+json"));
_registry.MapGet("/v2/moshiro-dev/moshiro-agent-manifest/blobs/" + ConfigDigest, () => Results.Text(
new JsonObject { ["config"] = new JsonObject { ["Labels"] = new JsonObject { ["org.moshiro.manifest"] = _label, ["org.moshiro.signature"] = "c2ln" } } }.ToJsonString(), "application/json"));
await _registry.StartAsync();
_registryUrl = _registry.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First();
}
public async ValueTask DisposeAsync()
{
await _registry.StopAsync();
await _registry.DisposeAsync();
}
[Fact]
public async Task TheManifestAndSignature_AreReadFromTheLabelsOfTheSmallImage()
{
string manifest = "{\"schema\":1,\"component\":\"agent\",\"seq\":7}";
_label = Convert.ToBase64String(Encoding.UTF8.GetBytes(manifest));
using var feed = new ReleaseFeed(new BackendOptions { NodeImageRegistry = _registryUrl });
(string Manifest, string Signature)? fetched = await feed.FetchAsync("agent", "dev", default);
Assert.Equal(manifest, fetched!.Value.Manifest);
Assert.Equal("c2ln", fetched.Value.Signature);
}
[Theory]
[InlineData("probe", "dev")] // 仓库里没有这个组件的清单
[InlineData("agent", "release")] // 没有这个通道
public async Task NothingThere_MeansNull_NotAnException(string component, string channel)
{
using var feed = new ReleaseFeed(new BackendOptions { NodeImageRegistry = _registryUrl });
Assert.Null(await feed.FetchAsync(component, channel, default));
}
[Fact]
public async Task AnUnreachableOrEmptyRegistry_MeansNull()
{
using var unreachable = new ReleaseFeed(new BackendOptions { NodeImageRegistry = "http://127.0.0.1:1" });
using var empty = new ReleaseFeed(new BackendOptions { NodeImageRegistry = "" });
Assert.Null(await unreachable.FetchAsync("agent", "dev", default));
Assert.Null(await empty.FetchAsync("agent", "dev", default));
}
[Fact]
public async Task AForgedManifestInTheRegistry_IsNeverServedToTheInstaller()
{
// 起一个指向这个假仓库的后端:仓库里是一份没有 CI 签名的清单,安装脚本向面板要清单时必须拿不到
string directory = Path.Combine(Path.GetTempPath(), "moshiro-feed-" + Guid.NewGuid().ToString("N"));
_label = Convert.ToBase64String(Encoding.UTF8.GetBytes("{\"schema\":1,\"component\":\"agent\",\"channel\":\"dev\",\"image\":\"evil/agent\",\"digest\":\"sha256:" + new string('a', 64) + "\",\"tag\":\"x\",\"version\":\"9\",\"seq\":999999999999}"));
await using WebApplication panel = BackendHost.Build(
["--urls", "http://127.0.0.1:0", "--Logging:LogLevel:Default=Warning"],
options =>
{
options.DataDirectory = directory;
options.Pbkdf2Iterations = 1000;
options.SecureCookies = false;
options.ClaimCode = "FEED-TEST-CODE";
options.NodeImageRegistry = _registryUrl;
options.UpdateChannel = "dev";
options.GeoLookupUrl = string.Empty;
options.RequireOwner = false;
});
await panel.StartAsync();
try
{
using var http = new HttpClient { BaseAddress = new Uri(panel.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First()) };
using HttpResponseMessage setup = await http.PostAsync("/api/admin/auth/setup", BackendFixture.Json(new JsonObject { ["claim_code"] = "FEED-TEST-CODE", ["username"] = "owner", ["password"] = "correct horse battery", ["return_session"] = true }));
Assert.Equal(HttpStatusCode.Created, setup.StatusCode);
http.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", JsonNode.Parse(await setup.Content.ReadAsStringAsync())!["session_token"]!.GetValue<string>());
using HttpResponseMessage created = await http.PostAsync("/api/admin/join-tokens", BackendFixture.Json(new JsonObject { ["role"] = "agent" }));
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
string code = JsonNode.Parse(await created.Content.ReadAsStringAsync())!["plaintext"]!.GetValue<string>();
using HttpResponseMessage manifest = await http.GetAsync("/install/manifest?code=" + Uri.EscapeDataString(code));
Assert.Equal(HttpStatusCode.NotFound, manifest.StatusCode); // 没有有效的 CI 签名,面板不采用
}
finally
{
await panel.StopAsync();
await panel.DisposeAsync();
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
try
{
Directory.Delete(directory, recursive: true);
}
catch (IOException)
{
// 系统临时目录里的残留
}
}
}
}
-36
View File
@@ -114,40 +114,4 @@ for pid in "${pids[@]}"; do
wait "$pid"
done
# 所有压缩包的校验和汇成一份 SHA256SUMS,用发布私钥签名(base64 的 DER ECDSA,和签名的发布清单是同一把密钥、同一种格式):
# 不走容器的安装(直接下载压缩包)先验这份签名再核对压缩包的哈希,发布页被改也骗不到;没有配置签名密钥(例如 fork)就跳过。
upload_local_asset() {
local name="$1" file="$2" upload_response="$tmp/$1.upload.json" status
if grep -Eq "\"name\"[[:space:]]*:[[:space:]]*\"$name\"" "$assets_json"; then
echo "release asset $name already exists; keeping the existing asset"
return 0
fi
status="$(curl --silent --show-error --retry 3 --netrc-file "$netrc" \
--request POST --header 'Content-Type: application/octet-stream' --data-binary "@$file" \
--output "$upload_response" --write-out '%{http_code}' "$api/releases/$release_id/assets?name=$name")"
[[ "$status" == 2* ]] || { echo "Gitea Release asset upload failed for $name (HTTP $status)" >&2; cat "$upload_response" >&2; return 1; }
echo "attached $name"
}
if [[ -n "${RELEASE_SIGNING_KEY:-}" ]]; then
sums="$tmp/SHA256SUMS"
: > "$sums"
for archive in "${assets[@]}"; do
curl --fail --silent --show-error --retry 3 --netrc-file "$netrc" \
"$gitea_base/api/packages/justme/generic/moshiro-node/$version/$archive.sha256" -o "$tmp/$archive.sha256"
cat "$tmp/$archive.sha256" >> "$sums"
done
[[ "$(wc -l < "$sums")" -eq "${#assets[@]}" ]] || { echo "SHA256SUMS does not have one line per archive" >&2; exit 1; }
signing_key="$(mktemp)"
chmod 600 "$signing_key"
printf '%s\n' "$RELEASE_SIGNING_KEY" > "$signing_key"
openssl dgst -sha256 -sign "$signing_key" -out "$tmp/SHA256SUMS.sig.bin" "$sums"
rm -f "$signing_key"
base64 -w0 "$tmp/SHA256SUMS.sig.bin" > "$tmp/SHA256SUMS.sig" 2>/dev/null || base64 "$tmp/SHA256SUMS.sig.bin" | tr -d '\n' > "$tmp/SHA256SUMS.sig"
upload_local_asset SHA256SUMS "$sums"
upload_local_asset SHA256SUMS.sig "$tmp/SHA256SUMS.sig"
else
echo "release signing is not configured; SHA256SUMS is not signed"
fi
echo "published Gitea Release $tag: https://git.oeo.one/justme/MoShiro/releases/tag/$tag"
-31
View File
@@ -1,31 +0,0 @@
#!/usr/bin/env bash
# 一个组件在所有平台、所有架构的发布压缩包里的可执行文件的 SHA-256(逗号分隔,输出到标准输出)。
# ci/release-binary-hashes.sh <agent|probe> <版本,不带 v> 环境变量:GITEA_BASE、NETRC(curl 的 netrc 文件)
# 签名的发布清单里的 binaries 就是它:节点握手时上报正在运行的可执行文件的哈希,Backend 凭 CI 签名的清单认「官方二进制」——
# 容器、Linux glibc/musl 压缩包、macOS、Windows 的 x64/arm64 都一样,不只是容器镜像里的那两个。
set -euo pipefail
component="${1:?usage: release-binary-hashes.sh <agent|probe> <version>}"; version="${2:?version}"
[[ "$component" =~ ^(agent|probe)$ ]] || { echo "component must be agent or probe" >&2; exit 2; }
gitea_base="${GITEA_BASE:?GITEA_BASE is required}"; gitea_base="${gitea_base%/}"
netrc=(); [[ -n "${NETRC:-}" ]] && netrc=(--netrc-file "$NETRC")
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
# 和 ci/publish-gitea-release.sh 里发布的那 8 种一样(每个组件)
targets=(linux-amd64 linux-arm64 linux-musl-amd64 linux-musl-arm64 macos-x64 macos-arm64 windows-amd64 windows-arm64)
hashes=()
for target in "${targets[@]}"; do
if [[ "$target" == windows-* ]]; then archive="moshiro-$component-v$version-$target.zip"; member="moshiro-$component.exe"
else archive="moshiro-$component-v$version-$target.tar.gz"; member="moshiro-$component"; fi
curl --fail --silent --show-error --retry 3 ${netrc[@]+"${netrc[@]}"} "$gitea_base/api/packages/justme/generic/moshiro-node/$version/$archive" -o "$work/$archive"
if [[ "$archive" == *.zip ]]; then
hash="$(cd "$work" && python3 -c 'import hashlib,sys,zipfile; print(hashlib.sha256(zipfile.ZipFile(sys.argv[1]).read(sys.argv[2])).hexdigest())' "$archive" "$member")"
else
hash="$(tar -xzOf "$work/$archive" "$member" | sha256sum | cut -d' ' -f1)"
fi
[[ "$hash" =~ ^[0-9a-f]{64}$ ]] || { echo "could not hash $member in $archive" >&2; exit 1; }
hashes+=("$hash")
rm -f "$work/$archive"
done
# 不同平台的二进制哈希不会相同;去重只是防御
printf '%s\n' "${hashes[@]}" | sort -u | paste -sd,
+2 -5
View File
@@ -53,11 +53,8 @@ export MOSHIRO_COMMIT="${MOSHIRO_COMMIT:-${CI_COMMIT_SHA:0:10}}"
export MOSHIRO_BUILT_AT="${MOSHIRO_BUILT_AT:-$(date -u +%Y-%m-%dT%H:%M:%SZ)}"
# 前端先构建:Backend.Api 在 web/*/dist 存在时把它们内嵌进程序(展示端 /,管理端 /admin/),这样 passkey 能绑定在面板自己的域名上
# 发版流水线里有单独的 web-build 步骤先构建好了(并行的两个架构不能同时 npm ci);main 的 [CI] 构建只有一个 Backend 步骤,这里自己构建
if [[ ! -f "$root/web/display/dist/index.html" || ! -f "$root/web/admin/dist/index.html" ]]; then
if [[ -z "${NPM_CONFIG_CACHE:-}" && -d /home/justme/.npm ]]; then export NPM_CONFIG_CACHE=/home/justme/.npm; fi
(cd "$root" && npm ci --prefix web --prefer-offline && npm run build --prefix web)
fi
if [[ -z "${NPM_CONFIG_CACHE:-}" && -d /home/justme/.npm ]]; then export NPM_CONFIG_CACHE=/home/justme/.npm; fi
(cd "$root" && npm ci --prefix web --prefer-offline && npm run build --prefix web)
[[ -f "$root/web/display/dist/index.html" && -f "$root/web/admin/dist/index.html" ]] || { echo "the web build did not produce both apps" >&2; exit 1; }
(cd "$root/backend" && dotnet publish src/MoShiro.Backend \
+1 -9
View File
@@ -98,7 +98,7 @@ publish_index() {
buildah manifest create "$manifest" >/dev/null
buildah manifest add "$manifest" "docker://$image:$version-amd64" >/dev/null
buildah manifest add "$manifest" "docker://$image:$version-arm64" >/dev/null
buildah manifest push --all --format oci --digestfile "$work/$component.digest" "$manifest" "docker://$image:$version" >/dev/null
buildah manifest push --all --format oci "$manifest" "docker://$image:$version" >/dev/null
if [[ "$version" != *-* ]]; then
buildah manifest push --all --format oci "$manifest" "docker://$image:latest" >/dev/null
fi
@@ -113,12 +113,4 @@ for component in agent probe; do
build_arch_image "$component" amd64 musl-amd64
build_arch_image "$component" arm64 musl-arm64
publish_index "$component"
# 发布清单:给多架构镜像的 digest 签名(私钥只在 CI 密钥里),放进镜像仓库(<仓库>-manifest:release),面板在线时再登记一份加速。
# 没有配置签名密钥(例如 fork)就只发镜像、不发清单。
if [[ -n "${RELEASE_SIGNING_KEY:-}" ]]; then
digest="$(cat "$work/$component.digest")"
# 清单里的 binaries:所有平台、所有架构的发布压缩包里的可执行文件(不只是镜像里的 musl 那两个),节点握手时上报的哈希对得上其中任何一个
binaries="$(NETRC="$netrc" GITEA_BASE="$gitea_base" bash "$root/ci/release-binary-hashes.sh" "$component" "$version")"
RELEASE_BINARIES="$binaries" bash "$root/ci/publish-release-offer.sh" "$component" "$image_root/moshiro-$component" "$digest" "$tag" release "$work/$component-release"
fi
done
-43
View File
@@ -1,43 +0,0 @@
#!/usr/bin/env bash
# release-binary-hashes.sh 的测试:用本地的假 Gitea 包仓库(python http.server),造 8 个平台的压缩包(tar.gz 和 zip),
# 核对脚本算出来的哈希就是压缩包里可执行文件的 SHA-256。运行:bash ci/tests/binary-hashes-test.sh
set -eu
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
T="$(mktemp -d)"
SERVER=
trap '[ -z "$SERVER" ] || kill "$SERVER" 2>/dev/null || true; rm -rf "$T" 2>/dev/null || true' EXIT
V=9.9.9
PKG="$T/srv/api/packages/justme/generic/moshiro-node/$V"
mkdir -p "$PKG"
fail=0
expected=()
for component in agent probe; do
for target in linux-amd64 linux-arm64 linux-musl-amd64 linux-musl-arm64 macos-x64 macos-arm64 windows-amd64 windows-arm64; do
stage="$T/stage-$component-$target"; mkdir -p "$stage"
if [[ "$target" == windows-* ]]; then member="moshiro-$component.exe"; else member="moshiro-$component"; fi
printf 'binary of %s for %s\n' "$component" "$target" > "$stage/$member"
printf 'noise' > "$stage/fleet-enroll"; printf 'readme' > "$stage/README.md"
if [[ "$target" == windows-* ]]; then
(cd "$stage" && python3 -c "import zipfile,sys; z=zipfile.ZipFile('out.zip','w'); [z.write(n) for n in sys.argv[1:]]" "$member" fleet-enroll README.md && mv out.zip "$PKG/moshiro-$component-v$V-$target.zip")
else
tar -czf "$PKG/moshiro-$component-v$V-$target.tar.gz" -C "$stage" "$member" fleet-enroll README.md
fi
[[ "$component" == agent ]] && expected+=("$(sha256sum "$stage/$member" | cut -d' ' -f1)")
done
done
want="$(printf '%s\n' "${expected[@]}" | sort -u | paste -sd,)"
(cd "$T/srv" && python3 -u -m http.server 0 --bind 127.0.0.1 >"$T/server.log" 2>&1 & echo $! > "$T/pid")
SERVER="$(cat "$T/pid")"
for _ in $(seq 1 50); do port="$(grep -o 'port [0-9]*' "$T/server.log" | head -1 | cut -d' ' -f2 || true)"; [[ -n "$port" ]] && break; sleep 0.2; done
[[ -n "${port:-}" ]] || { echo "FAIL the fake package server did not start"; cat "$T/server.log"; exit 1; }
got="$(GITEA_BASE="http://127.0.0.1:$port" bash "$ROOT/ci/release-binary-hashes.sh" agent "$V")"
if [[ "$got" == "$want" ]]; then echo "ok agent: all 8 platform binaries hashed ($(echo "$got" | tr ',' '\n' | wc -l) distinct)"; else echo "FAIL agent hashes differ"; echo " want $want"; echo " got $got"; fail=1; fi
[[ "$(echo "$got" | tr ',' '\n' | wc -l)" -eq 8 ]] && echo "ok eight different binaries, none dropped" || { echo "FAIL expected 8 hashes"; fail=1; }
# 缺一个平台的压缩包:整个脚本失败,不能悄悄少签一个
rm "$PKG/moshiro-probe-v$V-macos-arm64.tar.gz"
if GITEA_BASE="http://127.0.0.1:$port" bash "$ROOT/ci/release-binary-hashes.sh" probe "$V" >/dev/null 2>&1; then echo "FAIL a missing archive was ignored"; fail=1; else echo "ok a missing archive fails the script instead of signing fewer binaries"; fi
if bash "$ROOT/ci/release-binary-hashes.sh" other "$V" >/dev/null 2>&1; then echo "FAIL unknown component accepted"; fail=1; else echo "ok unknown component is refused"; fi
exit "$fail"
-5
View File
@@ -335,11 +335,6 @@ CI 给每个镜像的 digest 签名之后,除了(面板在线时)登记给
```
- **更新器**:配置里 `FEED=auto`(默认)时先看仓库里的清单,验签通过才用;取不到或验不过就用面板放进信箱的那份。`MANIFEST_REF` 可以指定清单镜像,`FEED=off` 关掉;通道取配置里的 `CHANNEL`,没有就按仓库路径推断(`moshiro-dev` → dev,`moshiro` → release)。仓库里的清单验签失败而信箱里又没有东西时,状态会报「无效清单」而不是悄悄当作没有更新。
- **面板自己取**:安装脚本向面板要某个组件的签名清单时,面板里没有(清过库、CI 登记那一步网络失败)或者已经放了十分钟以上,面板就按 OCI 分发协议**匿名**读一次仓库里的清单镜像(只读清单和它的配置两个小 JSON,不拉镜像层),用内嵌的 CI 发布公钥验过签才登记;验不过(仓库或路上被改)就当没有。所以清库之后不用手工补登记,也不会被一份伪造的清单骗到。仓库要匿名可读(`moshiro/**`、`moshiro-dev/**`)。
- **没有宿主机更新器的节点**(一键容器安装、二进制安装)握手时带不来清单:面板用自己手里(登记时验过 CI 签名)的那份把节点的二进制标成「已签名」;不在清单里的只是不标「已签名」,不会因此被拒绝。
- **发版**:打 `vMAJOR.MINOR.PATCH` tag 时,节点镜像(Agent、Probe 的多架构镜像)也和 Backend 一样签清单,放到 `moshiro/…-manifest:release`。清单里的 `binaries` 是**所有平台、所有架构**的发布压缩包里可执行文件的 SHA-256(Linux glibc 和 musl、macOS、Windows,x64 和 arm64,每个组件 8 个,`ci/release-binary-hashes.sh`),不只是容器镜像里的那两个:节点握手时上报正在运行的可执行文件的哈希,Backend 凭 CI 签名的清单认「官方二进制」,所以容器和直接下载压缩包安装的节点是一回事。
- **压缩包的校验和也签名**:Release 里除了每个压缩包的 `.sha256`,还有汇总的 `SHA256SUMS` 和 `SHA256SUMS.sig`(用发布私钥签,base64 的 DER ECDSA P-256,和清单同一把密钥)。验证:`base64 -d SHA256SUMS.sig > s.bin && openssl dgst -sha256 -verify packaging/update/release-key.pub.pem -signature s.bin SHA256SUMS`,通过之后再 `sha256sum -c SHA256SUMS`。macOS 的 Developer ID 签名和公证、Windows 的 Authenticode 需要苹果和微软的证书,目前没有做。
- 测试:`bash ci/tests/binary-hashes-test.sh`(本地的假包仓库,8 种平台的 tar.gz 和 zip,缺任何一个压缩包都会失败,不会悄悄少签)。
- **首次部署**:`deploy-node.sh` 默认就走这条路,不需要面板,也不用手动指定镜像(Backend 第一次部署时本来就没有面板)。`--tag` 指定标签时不验签并有警告;`--image` 直接用你给的引用。
- **别人自己部署后端,什么都不用改**:仓库对匿名读开放(`moshiro/*` 和各 `*-manifest` 仓库),然后 `sudo sh deploy-node.sh backend --channel release --public-url https://<自己的域名>`,只是配置,不改代码、不需要自己的 CI;passkey 绑定在自己的域名上。自己 fork 并自建镜像的人要换自己的发布公钥和仓库地址(`--image-repo`)并重新构建,这是 fork 的正常代价。
- 测试:`bash packaging/update/tests/feed-test.sh`(假的 podman + 真的 openssl,覆盖篡改、别的密钥、旧 seq、换钥、仓库不可用回退、首次部署的验签)。
+2 -2
View File
@@ -64,7 +64,7 @@ YOrtO3Vso/QNdSMmepQlx1Q3v+KkN7ro3AY3I6DT2Q2txJu5pS7qzutSQA==
$container = "moshiro-$role"
$volume = "moshiro-$role-config"
docker container inspect $container *> $null
docker inspect $container *> $null
if ($LASTEXITCODE -eq 0) {
docker start $container | Out-Null
Write-Host "$container is already installed and running."
@@ -134,7 +134,7 @@ YOrtO3Vso/QNdSMmepQlx1Q3v+KkN7ro3AY3I6DT2Q2txJu5pS7qzutSQA==
# 等注册完成:第一次启动会用上面的令牌注册,成功之后才算装好
for ($i = 0; $i -lt 30; $i++) {
$running = docker container inspect -f '{{.State.Running}}' $container 2>$null
$running = docker inspect -f '{{.State.Running}}' $container 2>$null
$logs = (docker logs $container 2>&1) -join "`n"
if ($logs -match 'link up') {
Write-Host "done: $role '$name' is online. It shows up in the panel now."
+2 -2
View File
@@ -54,7 +54,7 @@ volume="moshiro-$ROLE-config"
linux=0
[ "$(uname -s)" = Linux ] && linux=1
if [ "${MOSHIRO_DRY_RUN:-}" != 1 ] && eng container inspect "$container" >/dev/null 2>&1 </dev/null; then
if [ "${MOSHIRO_DRY_RUN:-}" != 1 ] && eng inspect "$container" >/dev/null 2>&1 </dev/null; then
eng start "$container" >/dev/null </dev/null
say "$container is already installed and running."
say "reinstall: $engine rm -f $container && $engine volume rm $volume (then run the install command again)"
@@ -141,7 +141,7 @@ fi
# 等注册完成:第一次启动会用上面的令牌注册,成功之后才算装好
i=0
while [ "$i" -lt 30 ]; do
state="$(eng container inspect -f '{{.State.Running}}' "$container" 2>/dev/null </dev/null || echo false)"
state="$(eng inspect -f '{{.State.Running}}' "$container" 2>/dev/null </dev/null || echo false)"
logs="$(eng logs "$container" 2>&1 </dev/null || true)"
case "$logs" in
*"link up"*) say "done: $ROLE '$name' is online. It shows up in the panel now."; say "logs: $engine logs -f $container"; exit 0 ;;
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "moshiro-web",
"private": true,
"version": "0.3.3",
"version": "0.1.0",
"workspaces": [
"display",
"admin"