ci: sync main into dev [CI SKIP]
This commit is contained in:
commit
b242701e9e
13 files changed
+300
-370
No files matched your search
@@ -0,0 +1,53 @@
|
||||
labels:
|
||||
ci: domain-linux-native
|
||||
platform: linux/amd64
|
||||
|
||||
clone:
|
||||
- name: git
|
||||
image: plugin-git
|
||||
settings:
|
||||
lfs: false
|
||||
|
||||
when:
|
||||
- event: push
|
||||
branch: main
|
||||
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
|
||||
|
||||
concurrency: 1
|
||||
|
||||
steps:
|
||||
- name: build-linux-amd64-backend
|
||||
image: bash
|
||||
environment:
|
||||
NUGET_PACKAGES: /home/justme/.nuget/packages
|
||||
ZOT_REGISTRY: registry.oeo.one
|
||||
ZOT_USERNAME: justme
|
||||
ZOT_SECRET:
|
||||
from_secret: zot_secret
|
||||
commands:
|
||||
- bash ci/build-main-amd64.sh backend
|
||||
depends_on: []
|
||||
|
||||
- name: build-linux-amd64-agent
|
||||
image: bash
|
||||
environment:
|
||||
NUGET_PACKAGES: /home/justme/.nuget/packages
|
||||
ZOT_REGISTRY: registry.oeo.one
|
||||
ZOT_USERNAME: justme
|
||||
ZOT_SECRET:
|
||||
from_secret: zot_secret
|
||||
commands:
|
||||
- bash ci/build-main-amd64.sh agent
|
||||
depends_on: []
|
||||
|
||||
- name: build-linux-amd64-probe
|
||||
image: bash
|
||||
environment:
|
||||
NUGET_PACKAGES: /home/justme/.nuget/packages
|
||||
ZOT_REGISTRY: registry.oeo.one
|
||||
ZOT_USERNAME: justme
|
||||
ZOT_SECRET:
|
||||
from_secret: zot_secret
|
||||
commands:
|
||||
- bash ci/build-main-amd64.sh probe
|
||||
depends_on: []
|
||||
@@ -1,31 +0,0 @@
|
||||
labels:
|
||||
ci: domain-linux-native
|
||||
platform: linux/amd64
|
||||
|
||||
clone:
|
||||
- name: git
|
||||
image: plugin-git
|
||||
settings:
|
||||
lfs: false
|
||||
|
||||
when:
|
||||
- event: push
|
||||
branch: main
|
||||
- event: pull_request_closed
|
||||
branch: main
|
||||
|
||||
concurrency: 1
|
||||
|
||||
steps:
|
||||
- name: publish-affected-amd64-images
|
||||
image: bash
|
||||
environment:
|
||||
NUGET_PACKAGES: /home/justme/.nuget/packages
|
||||
GITEA_BASE: http://127.0.0.1:3100
|
||||
ZOT_REGISTRY: registry.oeo.one
|
||||
ZOT_USERNAME: justme
|
||||
ZOT_SECRET:
|
||||
from_secret: zot_secret
|
||||
commands:
|
||||
- bash ci/release-main-images.sh
|
||||
depends_on: []
|
||||
+4
-10
@@ -13,20 +13,14 @@ when:
|
||||
- event: push
|
||||
branch: dev
|
||||
|
||||
# On Gitea PR events, CI_COMMIT_MESSAGE is the PR title.
|
||||
steps:
|
||||
- name: backend-tests
|
||||
image: bash
|
||||
# [SKIP TESTS] omits Backend tests only; web-build remains a dependent workflow.
|
||||
when:
|
||||
- evaluate: 'not (CI_COMMIT_MESSAGE contains "[SKIP TESTS]")'
|
||||
environment:
|
||||
NUGET_PACKAGES: /home/justme/.nuget/packages
|
||||
commands:
|
||||
- cd backend && dotnet test --solution Monitor.slnx -c Release
|
||||
|
||||
- name: web-build
|
||||
image: bash
|
||||
environment:
|
||||
NPM_CONFIG_CACHE: /home/justme/.npm
|
||||
commands:
|
||||
- npm ci --prefix web --prefer-offline
|
||||
- npm run build --prefix web
|
||||
- npm run lint:css --prefix web
|
||||
depends_on: []
|
||||
@@ -0,0 +1,27 @@
|
||||
labels:
|
||||
ci: domain-linux-native
|
||||
platform: linux/amd64
|
||||
|
||||
clone:
|
||||
- name: git
|
||||
image: plugin-git
|
||||
settings:
|
||||
lfs: false
|
||||
|
||||
when:
|
||||
- event: pull_request
|
||||
- event: push
|
||||
branch: dev
|
||||
|
||||
depends_on:
|
||||
- verify
|
||||
|
||||
steps:
|
||||
- name: web-build
|
||||
image: bash
|
||||
environment:
|
||||
NPM_CONFIG_CACHE: /home/justme/.npm
|
||||
commands:
|
||||
- npm ci --prefix web --prefer-offline
|
||||
- npm run build --prefix web
|
||||
- npm run lint:css --prefix web
|
||||
@@ -40,6 +40,8 @@ dotnet build Monitor.slnx -c Release
|
||||
```sh
|
||||
npm ci --prefix web
|
||||
npm run build --prefix web
|
||||
npm run gen:api --prefix web # API schema changed after Backend edits
|
||||
npm run lint:css --prefix web
|
||||
```
|
||||
|
||||
前端产物位于 `web/display/dist/` 和 `web/admin/dist/`。NativeAOT 目标平台和 Windows 上使用 AotAnywhere 交叉构建的说明见 [`docs/deployment.md`](docs/deployment.md)。
|
||||
@@ -64,6 +66,15 @@ npm run dev --prefix web
|
||||
|
||||
打开 <http://localhost:5173/> 使用展示端,或访问 <http://localhost:5174/admin/> 进入管理端。依赖安装和前端配置见 [`web/README.md`](web/README.md);Backend 的首次启动与配置见 [`backend/README.md`](backend/README.md)。
|
||||
|
||||
### CI 构建规则
|
||||
|
||||
- Pull request 和 `dev` 分支 push:先跑 Backend tests,通过后再构建前端。
|
||||
- 在需要跳过 Backend tests 的提交信息后加 `[SKIP TESTS]`;Gitea pull request 需要把标记写进 PR 标题。前端构建仍会运行。
|
||||
- `main` 上的提交信息包含 `[CI]` 时,并行构建 Backend、Agent、Probe 的 Linux/amd64 镜像,推送到 Zot 的 `monitor-dev/` 仓库;普通 main push 不构建。
|
||||
- 推送 `vMAJOR.MINOR.PATCH` tag 会触发全平台正式构建和 Release 发布。
|
||||
|
||||
完整触发条件、开发镜像标签和发布路径见 [`docs/deployment.md`](docs/deployment.md)。
|
||||
|
||||
## 项目布局
|
||||
|
||||
| 路径 | 说明 |
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build and push one opt-in main Linux amd64 image to the dev repository in Zot.
|
||||
set -euo pipefail
|
||||
|
||||
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
|
||||
echo "main amd64 builds require a main push with [CI] in the commit message" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
component="${1:?usage: build-main-amd64.sh <backend|agent|probe>}"
|
||||
case "$component" in
|
||||
backend|agent|probe) ;;
|
||||
*) echo "unsupported component: $component" >&2; exit 2 ;;
|
||||
esac
|
||||
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
|
||||
echo "Zot credentials are required" >&2
|
||||
exit 2
|
||||
}
|
||||
[[ "${CI_COMMIT_SHA:-}" =~ ^[0-9a-fA-F]{40}$ ]] || {
|
||||
echo "CI_COMMIT_SHA must be a full commit SHA" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
registry="${ZOT_REGISTRY:-registry.oeo.one}"
|
||||
repository="$registry/monitor-dev/monitor-$component"
|
||||
work="$root/.release/main-amd64-build"
|
||||
base_home="/home/justme/woodpecker/buildah-home/main-amd64-build-$component"
|
||||
short_sha="${CI_COMMIT_SHA:0:12}"
|
||||
dev_tag="dev-$short_sha"
|
||||
mkdir -p "$work" "$base_home"
|
||||
|
||||
if [[ "$component" == backend ]]; then
|
||||
BUILDAH_HOME="$base_home" \
|
||||
ZOT_REGISTRY="$registry" \
|
||||
ZOT_IMAGE_REPOSITORY="$repository" \
|
||||
IMAGE_TAGS="latest,$dev_tag" \
|
||||
IMAGE_PUBLISH=true \
|
||||
bash "$root/ci/release-linux.sh" amd64 backend
|
||||
exit 0
|
||||
fi
|
||||
|
||||
export HOME="$base_home"
|
||||
export BUILDAH_HOME="$base_home"
|
||||
export REGISTRY_AUTH_FILE="$work/$component-auth.json"
|
||||
context="$work/$component"
|
||||
local_image="localhost/monitor-$component:main-amd64-check"
|
||||
|
||||
python3 "$root/ci/stage-node-image.py" "$component" "$context"
|
||||
buildah bud --layers --network host --tls-verify=true --arch amd64 \
|
||||
--build-arg APP_VERSION=main \
|
||||
-f "$context/Containerfile" \
|
||||
-t "$local_image" "$context"
|
||||
|
||||
logged_in=false
|
||||
for attempt in 1 2 3 4; do
|
||||
if buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$registry" <<< "$ZOT_SECRET" >/dev/null; then
|
||||
logged_in=true
|
||||
break
|
||||
fi
|
||||
if [[ "$attempt" -lt 4 ]]; then
|
||||
echo "Zot login attempt $attempt failed; retrying in 3 seconds" >&2
|
||||
sleep 3
|
||||
fi
|
||||
done
|
||||
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
|
||||
|
||||
for tag in latest "$dev_tag"; do
|
||||
pushed=false
|
||||
for attempt in 1 2 3; do
|
||||
if buildah push --tls-verify=true "$local_image" "docker://$repository:$tag"; then
|
||||
pushed=true
|
||||
break
|
||||
fi
|
||||
if [[ "$attempt" -lt 3 ]]; then
|
||||
echo "Zot push for $component:$tag attempt $attempt failed; retrying in 5 seconds" >&2
|
||||
sleep 5
|
||||
fi
|
||||
done
|
||||
[[ "$pushed" == true ]] || { echo "could not push $repository:$tag after 3 attempts" >&2; exit 1; }
|
||||
echo "pushed development image $repository:$tag"
|
||||
done
|
||||
@@ -1,214 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Find impacted OCI images from changed paths and stage NativeAOT contexts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from xml.etree import ElementTree
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
BACKEND = ROOT / "backend"
|
||||
IMAGE_ROOTS = {
|
||||
"backend": [BACKEND / "src/Monitor.Backend/Monitor.Backend.csproj"],
|
||||
"agent": [
|
||||
BACKEND / "agent/src/Monitor.Agent/Monitor.Agent.csproj",
|
||||
BACKEND / "fleet/tools/fleet-enroll/fleet-enroll.csproj",
|
||||
],
|
||||
"probe": [BACKEND / "probe/src/Monitor.Probe/Monitor.Probe.csproj"],
|
||||
}
|
||||
GLOBAL_BACKEND_FILES = {
|
||||
"Directory.Build.props",
|
||||
"Directory.Build.targets",
|
||||
"Directory.Packages.props",
|
||||
"global.json",
|
||||
"NuGet.Config",
|
||||
"NuGet.config",
|
||||
}
|
||||
IGNORED_PARTS = {".git", ".release", "artifacts", "bin", "obj", "TestResults", "tests"}
|
||||
|
||||
|
||||
def read_changed_files() -> list[str]:
|
||||
raw = os.environ.get("CI_PIPELINE_FILES")
|
||||
if os.environ.get("FASTLANE_FILES_FROM_PR_API") == "true":
|
||||
value = json.loads(raw or "[]")
|
||||
return [str(path).replace("\\", "/") for path in value]
|
||||
if raw:
|
||||
try:
|
||||
value = json.loads(raw)
|
||||
if isinstance(value, list):
|
||||
changed = [str(path).replace("\\", "/") for path in value]
|
||||
if changed:
|
||||
return changed
|
||||
except json.JSONDecodeError:
|
||||
return [path.strip().replace("\\", "/") for path in raw.split(",") if path.strip()]
|
||||
|
||||
previous = os.environ.get("CI_PREV_COMMIT_SHA")
|
||||
try:
|
||||
base = previous if previous else "HEAD^"
|
||||
return subprocess.check_output(
|
||||
["git", "diff", "--name-only", base, "HEAD"],
|
||||
cwd=ROOT,
|
||||
text=True,
|
||||
).splitlines()
|
||||
except (OSError, subprocess.CalledProcessError):
|
||||
return []
|
||||
|
||||
|
||||
def project_graph() -> tuple[dict[Path, set[Path]], dict[Path, Path]]:
|
||||
projects = [path.resolve() for path in BACKEND.rglob("*.csproj") if not (IGNORED_PARTS & set(path.parts))]
|
||||
graph: dict[Path, set[Path]] = {}
|
||||
directories: dict[Path, Path] = {}
|
||||
for project in projects:
|
||||
graph[project] = set()
|
||||
directories[project.parent] = project
|
||||
try:
|
||||
document = ElementTree.parse(project)
|
||||
except ElementTree.ParseError:
|
||||
continue
|
||||
for reference in document.iter():
|
||||
if reference.tag.rsplit("}", 1)[-1] == "ProjectReference":
|
||||
include = reference.attrib.get("Include")
|
||||
if include:
|
||||
graph[project].add((project.parent / include.replace("\\", os.sep)).resolve())
|
||||
return graph, directories
|
||||
|
||||
|
||||
def reachable_projects(graph: dict[Path, set[Path]], roots: list[Path]) -> set[Path]:
|
||||
found: set[Path] = set()
|
||||
pending = [path.resolve() for path in roots]
|
||||
while pending:
|
||||
project = pending.pop()
|
||||
if project in found:
|
||||
continue
|
||||
found.add(project)
|
||||
pending.extend(graph.get(project, ()))
|
||||
return found
|
||||
|
||||
|
||||
def is_within(path: Path, directory: Path) -> bool:
|
||||
try:
|
||||
path.relative_to(directory)
|
||||
return True
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
def affected_images(changed: list[str]) -> list[str]:
|
||||
graph, project_dirs = project_graph()
|
||||
image_dependencies = {
|
||||
name: reachable_projects(graph, roots) for name, roots in IMAGE_ROOTS.items()
|
||||
}
|
||||
selected: set[str] = set()
|
||||
|
||||
def select_for_project(project: Path) -> None:
|
||||
for image, dependencies in image_dependencies.items():
|
||||
if project in dependencies:
|
||||
selected.add(image)
|
||||
|
||||
project_directories = sorted(project_dirs, key=lambda path: len(path.parts), reverse=True)
|
||||
container_map = {
|
||||
"packaging/container/Backend.Containerfile": "backend",
|
||||
"packaging/container/Agent.Containerfile": "agent",
|
||||
"packaging/container/Probe.Containerfile": "probe",
|
||||
"packaging/container/Agent.Fastlane.Containerfile": "agent",
|
||||
"packaging/container/Probe.Fastlane.Containerfile": "probe",
|
||||
}
|
||||
|
||||
for filename in changed:
|
||||
normalized = filename.replace("\\", "/")
|
||||
path = (ROOT / normalized).resolve()
|
||||
if normalized in container_map:
|
||||
selected.add(container_map[normalized])
|
||||
continue
|
||||
if normalized.startswith("ci/") or normalized.startswith(".woodpecker/"):
|
||||
if not normalized.endswith(".md"):
|
||||
selected.update(IMAGE_ROOTS)
|
||||
continue
|
||||
if any(part in IGNORED_PARTS for part in Path(normalized).parts):
|
||||
continue
|
||||
if not normalized.startswith("backend/"):
|
||||
continue
|
||||
if "/README" in normalized or normalized.endswith(".md"):
|
||||
continue
|
||||
|
||||
if path.name in GLOBAL_BACKEND_FILES and path.parent == BACKEND:
|
||||
selected.update(IMAGE_ROOTS)
|
||||
continue
|
||||
|
||||
matching = [directory for directory in project_directories if is_within(path, directory)]
|
||||
if matching:
|
||||
select_for_project(project_dirs[matching[0]])
|
||||
continue
|
||||
|
||||
# A props/targets file above one or more projects affects each project
|
||||
# below it and every image that consumes those projects.
|
||||
descendants = [project for directory, project in project_dirs.items() if is_within(directory, path.parent)]
|
||||
if path.suffix in {".props", ".targets"} and descendants:
|
||||
for project in descendants:
|
||||
select_for_project(project)
|
||||
elif normalized.startswith("backend/agent/"):
|
||||
selected.add("agent")
|
||||
elif normalized.startswith("backend/probe/"):
|
||||
selected.add("probe")
|
||||
elif path.is_file() and path.suffix in {".cs", ".csproj", ".props", ".targets", ".json"}:
|
||||
selected.update(IMAGE_ROOTS)
|
||||
|
||||
return [name for name in ("backend", "agent", "probe") if name in selected]
|
||||
|
||||
|
||||
def stage_context(component: str, destination: Path) -> None:
|
||||
containerfiles = {
|
||||
"agent": ROOT / "packaging/container/Agent.Fastlane.Containerfile",
|
||||
"probe": ROOT / "packaging/container/Probe.Fastlane.Containerfile",
|
||||
}
|
||||
if component not in containerfiles:
|
||||
raise SystemExit(f"NativeAOT context is not available for {component}")
|
||||
if destination.exists():
|
||||
shutil.rmtree(destination)
|
||||
restore = destination / "restore/backend"
|
||||
source = destination / "source/backend"
|
||||
restore.mkdir(parents=True)
|
||||
source.mkdir(parents=True)
|
||||
shutil.copy2(containerfiles[component], destination / "Containerfile")
|
||||
|
||||
for path in BACKEND.rglob("*"):
|
||||
if not path.is_file() or IGNORED_PARTS.intersection(path.parts):
|
||||
continue
|
||||
relative = path.relative_to(BACKEND)
|
||||
if path.suffix in {".csproj", ".props", ".targets", ".slnx"} or path.name in GLOBAL_BACKEND_FILES or path.name == "packages.lock.json":
|
||||
target = restore / relative
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(path, target)
|
||||
|
||||
target = source / relative
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(path, target)
|
||||
|
||||
(destination / "config").mkdir()
|
||||
(destination / "config/.keep").write_text("", encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--stage", choices=("agent", "probe"))
|
||||
parser.add_argument("--destination", type=Path)
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.stage:
|
||||
if args.destination is None:
|
||||
parser.error("--destination is required with --stage")
|
||||
stage_context(args.stage, args.destination)
|
||||
return
|
||||
|
||||
for image in affected_images(read_changed_files()):
|
||||
print(image)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+36
-15
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Publish one framework-dependent Backend OCI image for the requested arch.
|
||||
# Build a framework-dependent Backend OCI image for the requested arch; publishing is optional.
|
||||
set -euo pipefail
|
||||
|
||||
arch="${1:?usage: release-linux.sh <amd64|arm64>}"
|
||||
@@ -12,16 +12,21 @@ if [[ -n "$tag" ]]; then
|
||||
}
|
||||
version="${tag#v}"
|
||||
else
|
||||
[[ ("${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main) || \
|
||||
("${CI_PIPELINE_EVENT:-}" == pull_request_closed && "${CI_COMMIT_BRANCH:-}" == main && "${FASTLANE_MERGED_PR:-}" == true) ]] || {
|
||||
echo "untagged image builds are restricted to main pushes and merged PRs" >&2
|
||||
[[ "${CI_PIPELINE_EVENT:-}" == push && "${CI_COMMIT_BRANCH:-}" == main && "${CI_COMMIT_MESSAGE:-}" == *"[CI]"* ]] || {
|
||||
echo "untagged backend builds require a main push with [CI] in the commit message" >&2
|
||||
exit 2
|
||||
}
|
||||
version=main
|
||||
fi
|
||||
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
|
||||
echo "Zot credentials are required" >&2; exit 2;
|
||||
image_publish="${IMAGE_PUBLISH:-true}"
|
||||
[[ "$image_publish" == true || "$image_publish" == false ]] || {
|
||||
echo "IMAGE_PUBLISH must be true or false" >&2; exit 2;
|
||||
}
|
||||
if [[ "$image_publish" == true ]]; then
|
||||
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
|
||||
echo "Zot credentials are required" >&2; exit 2;
|
||||
}
|
||||
fi
|
||||
case "$arch" in
|
||||
amd64|arm64) ;;
|
||||
*) echo "unsupported architecture: $arch" >&2; exit 2 ;;
|
||||
@@ -34,7 +39,7 @@ if [[ -n "$tag" && "$version" != "$project_version" ]]; then
|
||||
fi
|
||||
|
||||
registry="${ZOT_REGISTRY:-registry.oeo.one}"
|
||||
image="$registry/monitor/monitor-backend"
|
||||
image="${ZOT_IMAGE_REPOSITORY:-$registry/monitor/monitor-backend}"
|
||||
output="$root/.release/linux-$arch/backend"
|
||||
build_root="$root/.release/build/linux-$arch-backend"
|
||||
container_context="$root/.release/container/linux-$arch"
|
||||
@@ -56,6 +61,15 @@ buildah_home="${BUILDAH_HOME:-/home/justme/woodpecker/buildah-home/$arch}"
|
||||
mkdir -p "$buildah_home"
|
||||
export HOME="$buildah_home"
|
||||
|
||||
local_image="localhost/monitor-backend:$version-$arch"
|
||||
buildah bud --tls-verify=true --arch "$arch" \
|
||||
--build-arg "APP_VERSION=$version" \
|
||||
-f "$root/packaging/container/Backend.Containerfile" \
|
||||
-t "$local_image" "$container_context"
|
||||
if [[ "$image_publish" == false ]]; then
|
||||
echo "built local Backend image for $arch; not publishing it"
|
||||
exit 0
|
||||
fi
|
||||
export REGISTRY_AUTH_FILE="$root/.release/auth/zot-linux-$arch.json"
|
||||
mkdir -p "$(dirname "$REGISTRY_AUTH_FILE")"
|
||||
logged_in=false
|
||||
@@ -70,11 +84,6 @@ for attempt in 1 2 3 4; do
|
||||
fi
|
||||
done
|
||||
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
|
||||
local_image="localhost/monitor-backend:$version-$arch"
|
||||
buildah bud --tls-verify=true --arch "$arch" \
|
||||
--build-arg "APP_VERSION=$version" \
|
||||
-f "$root/packaging/container/Backend.Containerfile" \
|
||||
-t "$local_image" "$container_context"
|
||||
push_tag() {
|
||||
local target_tag="$1" pushed=false
|
||||
for attempt in 1 2 3; do
|
||||
@@ -91,7 +100,19 @@ push_tag() {
|
||||
echo "pushed Zot image $image:$target_tag"
|
||||
}
|
||||
|
||||
push_tag "$version-$arch"
|
||||
if [[ -z "$tag" ]]; then
|
||||
push_tag "$version"
|
||||
if [[ -n "${IMAGE_TAGS:-}" ]]; then
|
||||
IFS=',' read -r -a image_tags <<< "$IMAGE_TAGS"
|
||||
((${#image_tags[@]} > 0)) || { echo "IMAGE_TAGS must contain at least one tag" >&2; exit 2; }
|
||||
for image_tag in "${image_tags[@]}"; do
|
||||
[[ "$image_tag" =~ ^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]] || {
|
||||
echo "invalid image tag: $image_tag" >&2
|
||||
exit 2
|
||||
}
|
||||
push_tag "$image_tag"
|
||||
done
|
||||
else
|
||||
push_tag "$version-$arch"
|
||||
if [[ -z "$tag" ]]; then
|
||||
push_tag "$version"
|
||||
fi
|
||||
fi
|
||||
@@ -1,99 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build and push only the linux/amd64 OCI images affected by a main push.
|
||||
set -euo pipefail
|
||||
|
||||
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
registry="${ZOT_REGISTRY:-registry.oeo.one}"
|
||||
work="$root/.release/main-fastlane"
|
||||
buildah_home="/home/justme/woodpecker/buildah-home/main-fastlane"
|
||||
mkdir -p "$work" "$buildah_home"
|
||||
export HOME="$buildah_home"
|
||||
export BUILDAH_HOME="$buildah_home"
|
||||
export REGISTRY_AUTH_FILE="$work/zot-auth.json"
|
||||
|
||||
if [[ "${CI_PIPELINE_EVENT:-}" == pull_request_closed ]]; then
|
||||
[[ "${CI_COMMIT_TARGET_BRANCH:-}" == main && -n "${CI_COMMIT_PULL_REQUEST:-}" ]] || {
|
||||
echo "ignoring closed pull request outside main" >&2
|
||||
exit 0
|
||||
}
|
||||
gitea_base="${GITEA_BASE:-http://127.0.0.1:3100}"
|
||||
api="$gitea_base/api/v1/repos/justme/MyMonitor"
|
||||
pr_json="$(curl --fail-with-body --silent --show-error "$api/pulls/$CI_COMMIT_PULL_REQUEST")"
|
||||
read -r merged base_branch <<< "$(python3 -c 'import json,sys; p=json.load(sys.stdin); print(str(p.get("merged", False)).lower(), p.get("base", {}).get("ref", ""))' <<< "$pr_json")"
|
||||
if [[ "$merged" != true || "$base_branch" != main ]]; then
|
||||
echo "pull request $CI_COMMIT_PULL_REQUEST was closed without merging to main; skipping images"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
export FASTLANE_MERGED_PR=true
|
||||
export FASTLANE_FILES_FROM_PR_API=true
|
||||
export CI_PIPELINE_FILES="$(curl --fail-with-body --silent --show-error \
|
||||
"$api/pulls/$CI_COMMIT_PULL_REQUEST/files?limit=500" \
|
||||
| python3 -c 'import json,sys; print(json.dumps([name for item in json.load(sys.stdin) for name in (item["filename"], item.get("previous_filename")) if name]))')"
|
||||
git fetch "$gitea_base/justme/MyMonitor.git" main
|
||||
git checkout --detach FETCH_HEAD
|
||||
export CI_COMMIT_BRANCH=main
|
||||
elif [[ "${CI_PIPELINE_EVENT:-}" != push || "${CI_COMMIT_BRANCH:-}" != main ]]; then
|
||||
echo "fast-lane images can only be published from a main push or a merged PR into main" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
[[ -n "${ZOT_USERNAME:-}" && -n "${ZOT_SECRET:-}" ]] || {
|
||||
echo "Zot credentials are required" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
targets="$(python3 "$root/ci/main-fastlane.py")"
|
||||
if [[ -z "$targets" ]]; then
|
||||
echo "no runtime image changes detected; skipping container builds"
|
||||
exit 0
|
||||
fi
|
||||
echo "affected linux/amd64 images: ${targets//$'\n'/, }"
|
||||
|
||||
logged_in=false
|
||||
for attempt in 1 2 3 4; do
|
||||
if buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$registry" <<< "$ZOT_SECRET" >/dev/null; then
|
||||
logged_in=true
|
||||
break
|
||||
fi
|
||||
if [[ "$attempt" -lt 4 ]]; then
|
||||
echo "Zot login attempt $attempt failed; retrying in 3 seconds" >&2
|
||||
sleep 3
|
||||
fi
|
||||
done
|
||||
[[ "$logged_in" == true ]] || { echo "could not authenticate to Zot after 4 attempts" >&2; exit 1; }
|
||||
|
||||
publish_node_image() {
|
||||
local component="$1" context="$work/$1" image="$registry/monitor/monitor-$1"
|
||||
local local_image="localhost/monitor-$component:main-amd64"
|
||||
|
||||
python3 "$root/ci/main-fastlane.py" --stage "$component" --destination "$context"
|
||||
buildah bud --layers --network host --tls-verify=true --arch amd64 \
|
||||
--build-arg APP_VERSION=main \
|
||||
-f "$context/Containerfile" -t "$local_image" "$context"
|
||||
|
||||
for tag in main-amd64 main; do
|
||||
local pushed=false
|
||||
for attempt in 1 2 3; do
|
||||
if buildah push --tls-verify=true "$local_image" "docker://$image:$tag"; then
|
||||
pushed=true
|
||||
break
|
||||
fi
|
||||
if [[ "$attempt" -lt 3 ]]; then
|
||||
echo "Zot push for $component:$tag attempt $attempt failed; retrying in 5 seconds" >&2
|
||||
sleep 5
|
||||
fi
|
||||
done
|
||||
[[ "$pushed" == true ]] || { echo "could not push $component:$tag after 3 attempts" >&2; return 1; }
|
||||
echo "pushed Zot image $image:$tag"
|
||||
done
|
||||
}
|
||||
|
||||
while IFS= read -r component; do
|
||||
[[ -n "$component" ]] || continue
|
||||
case "$component" in
|
||||
backend) bash "$root/ci/release-linux.sh" amd64 backend ;;
|
||||
agent|probe) publish_node_image "$component" ;;
|
||||
*) echo "unknown affected image: $component" >&2; exit 2 ;;
|
||||
esac
|
||||
done <<< "$targets"
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Stage source and restore files for the Linux amd64 Agent or Probe image build."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
BACKEND = ROOT / "backend"
|
||||
IGNORED_PARTS = {".git", ".release", "artifacts", "bin", "obj", "TestResults", "tests"}
|
||||
GLOBAL_BACKEND_FILES = {
|
||||
"Directory.Build.props",
|
||||
"Directory.Build.targets",
|
||||
"Directory.Packages.props",
|
||||
"global.json",
|
||||
"NuGet.Config",
|
||||
"NuGet.config",
|
||||
}
|
||||
CONTAINERFILES = {
|
||||
"agent": ROOT / "packaging/container/Agent.NativeBuild.Containerfile",
|
||||
"probe": ROOT / "packaging/container/Probe.NativeBuild.Containerfile",
|
||||
}
|
||||
|
||||
|
||||
def stage(component: str, destination: Path) -> None:
|
||||
if destination.exists():
|
||||
shutil.rmtree(destination)
|
||||
restore = destination / "restore/backend"
|
||||
source = destination / "source/backend"
|
||||
restore.mkdir(parents=True)
|
||||
source.mkdir(parents=True)
|
||||
shutil.copy2(CONTAINERFILES[component], destination / "Containerfile")
|
||||
|
||||
for path in BACKEND.rglob("*"):
|
||||
if not path.is_file() or IGNORED_PARTS.intersection(path.parts):
|
||||
continue
|
||||
relative = path.relative_to(BACKEND)
|
||||
if (
|
||||
path.suffix in {".csproj", ".props", ".targets", ".slnx"}
|
||||
or path.name in GLOBAL_BACKEND_FILES
|
||||
or path.name == "packages.lock.json"
|
||||
):
|
||||
target = restore / relative
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(path, target)
|
||||
target = source / relative
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(path, target)
|
||||
|
||||
config = destination / "config"
|
||||
config.mkdir()
|
||||
(config / ".keep").write_text("", encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("component", choices=tuple(CONTAINERFILES))
|
||||
parser.add_argument("destination", type=Path)
|
||||
args = parser.parse_args()
|
||||
stage(args.component, args.destination)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+21
-1
@@ -40,7 +40,27 @@ Compose 将容器端口只绑定到宿主机 `127.0.0.1:8080`,适合由同机
|
||||
|
||||
Woodpecker 在 `vMAJOR.MINOR.PATCH` tag 上,由 Windows runner 使用 AotAnywhere 并行构建 Agent/Probe 的 Linux glibc、Linux musl、macOS 和 Windows 产物。Agent 包还包含一次性注册工具 `fleet-enroll`。Linux 归档带有安装脚本和 systemd/OpenWrt 单元;Windows 产物为 `.zip`,Linux/macOS 产物为 `.tar.gz`。Linux glibc 的 Agent 与 Probe 二进制还会打包为 OCI 镜像,支持 `linux/amd64` 和 `linux/arm64`;musl、macOS、Windows 仍通过对应平台归档分发。所有平台产物和 OCI 镜像发布成功后,CI 会创建对应的 Gitea Release,并附上压缩包和 SHA-256 校验文件。macOS 产物目前未签名/公证。安装说明见 [`../backend/agent/README.md`](../backend/agent/README.md) 与 [`../backend/probe/README.md`](../backend/probe/README.md)。
|
||||
|
||||
推送到 `main` 或 feature PR 合入 `main` 后会触发 Linux/amd64 快车道;关闭但未合入的 PR 会跳过。CI 根据变更文件所在项目及 `.csproj` 的传递引用,只构建受影响的 Backend、Agent 或 Probe 镜像;Agent 镜像也包含 `fleet-enroll`。例如只改 Probe 时只构建 Probe;改动 `Monitor.Protocol` 等被多个程序引用的库时,会构建所有受影响镜像。结果推送到对应镜像的 `main` 和 `main-amd64` 标签,不改写 release 版本和 `latest`。只改 Web、文档或部署配置时,流程会正常完成但不会构建容器。完整多架构和跨平台构建仍由版本 tag 流水线负责。
|
||||
## CI 工作流
|
||||
|
||||
### Pull request 与 dev 验证
|
||||
|
||||
Pull request 和 `dev` push 会运行 Backend tests;通过后,独立的 `web-build` workflow 才会构建前端并检查 CSS。提交信息后缀 `[SKIP TESTS]` 只跳过 Backend tests,前端构建仍会运行。Gitea PR 事件的 `CI_COMMIT_MESSAGE` 是 PR 标题,因此在 PR 中跳过测试时,把后缀写入 PR 标题;普通分支 push 则写在 commit message 中。
|
||||
|
||||
### main 开发镜像
|
||||
|
||||
`main` 上只有提交信息包含 `[CI]` 的 push 才构建镜像。合并 PR 时,请把 `[CI]` 放入 PR 标题或合并提交标题。Backend、Agent、Probe 的 Linux/amd64 镜像会并行构建,并推送到同一个 Zot Registry 下独立的 `monitor-dev/` 仓库:
|
||||
|
||||
| 镜像 | 滚动标签 | 固定提交标签 |
|
||||
| --- | --- | --- |
|
||||
| Backend | `registry.oeo.one/monitor-dev/monitor-backend:latest` | `registry.oeo.one/monitor-dev/monitor-backend:dev-<12位SHA>` |
|
||||
| Agent | `registry.oeo.one/monitor-dev/monitor-agent:latest` | `registry.oeo.one/monitor-dev/monitor-agent:dev-<12位SHA>` |
|
||||
| Probe | `registry.oeo.one/monitor-dev/monitor-probe:latest` | `registry.oeo.one/monitor-dev/monitor-probe:dev-<12位SHA>` |
|
||||
|
||||
Dev 镜像目前仅支持 amd64。它们不创建 Gitea Release,也不修改正式版 `monitor/` 仓库中的标签。普通 main push 不会构建镜像。
|
||||
|
||||
### 正式版发布
|
||||
|
||||
推送匹配项目版本的 `vMAJOR.MINOR.PATCH` tag 会触发完整跨平台构建、正式 OCI 镜像推送和 Gitea Release。正式镜像使用 `registry.oeo.one/monitor/` 路径;Agent、Probe 提供 amd64 与 arm64 多架构标签,稳定版本更新正式仓库自己的 `latest`。这与 `monitor-dev/` 的 dev `latest` 相互独立。
|
||||
|
||||
Agent 和 Probe 镜像分别发布为 `registry.oeo.one/monitor/monitor-agent:<version>` 与 `registry.oeo.one/monitor/monitor-probe:<version>`,包含 amd64、arm64 多架构索引;也可指定 `:<version>-amd64` 或 `:<version>-arm64`。稳定版本同时更新 `latest`。镜像以非 root 用户(UID/GID 1654)运行,默认读取 `/etc/monitor/agent.json` 或 `/etc/monitor/probe.json`。运行容器时挂载对应配置目录;Probe 还会在该目录写入 `agents.json`,因此目录需可由 1654 用户写入。Agent、Probe 需使用宿主网络以保持节点端口和测量流量行为,并按需添加 `NET_RAW` 能力以执行 ICMP 测量:
|
||||
|
||||
|
||||
File renamed without changes.
File renamed without changes.
Reference in new issue
Block a user