44 Commits
Author SHA1 Message Date
justmeandClaude Sonnet 5.5 4d2a0c089a feat(files): store files on the host when a channel has no file domain
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image-api Pipeline was successful
Chat stays usable when the file service is off or down. Files up to 256 MB are hosted like chat; downloads try the host first and fall back to file-service tickets.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> [CI]
2026-10-19 14:25:25 +08:00
justmeandClaude Sonnet 5.5 4b4ab4ee4d feat(files): store files on the host when a channel has no file domain
Chat stays usable when the file service is off or down. Files up to 256 MB are hosted like chat; downloads try the host first and fall back to file-service tickets.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> [CI]
2026-10-19 14:24:41 +08:00
justmeandClaude Sonnet 5.5 7e6ef5e587 chore(contract): regenerate API contract after AccountProfile comment change
ci/woodpecker/push/web-build Pipeline failed
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image-api unknown status
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> [CI]
2026-10-19 14:19:30 +08:00
justmeandClaude Sonnet 5.5 7274390aa1 perf(host): store avatars as content-addressed files and serve them from disk
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/push/web-build Pipeline failed
ci/woodpecker/push/dev-image-api unknown status
Space info now carries avatar_version instead of the inlined thumbnail; the worker fetches the thumbnail once per version. Legacy base64 rows are migrated to files on startup, and the daily backup uploads avatar files.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> [CI]
2026-10-19 14:16:50 +08:00
justme 8af4f284a0 ci: build and release webdrop-api and webdrop-files as separate images [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image-files Pipeline was successful
ci/woodpecker/push/dev-image-api Pipeline was successful
- dev images are built only when the push touched code that image uses (path filters);
  webdrop-files does not wait for the frontend build
- one v* tag releases both images (:tag and :latest) after the shared checks
- registry build and push live in scripts/ci-publish-image.sh, used by every workflow
- containers: fix the api Containerfile (missing Sync project), add Containerfile.files,
  Quadlet units for the file service (prod and dev), env examples, and registry
  auto-update for the production units so each machine pulls only the image it runs
2026-10-19 13:38:59 +08:00
justme 5489407241 refactor(web): restructure into api/domain/services/worker; capabilities, file domains, video covers
- dissolve core-bridge into api (wire contract), domain (models), services (what the UI
  calls) and worker (everything that runs in the Web Worker, grouped by account, crypto,
  space, files, shares, runtime); regroup features, lib and store by responsibility and
  delete unused components
- the UI only reads backend capabilities (domain/capabilities); one tokenRole picks the
  token, hasCode/isRetryable handle errors; pre-checks that decided permissions on the
  client are gone; tests fail if those rules are reimplemented
- files go straight to the file domain (create, upload, download, delete, commit, cancel);
  the host sees one registration per file; creating a channel allocates a file domain and
  works even when the file service is down
- file panel shows which channel the files belong to and its space usage; unlimited files
  preview in chat; videos get a stored cover; the file service status is shown
- fix: console flooded by Retry-After reads on every upload frame
2026-10-19 13:38:51 +08:00
justme 17638b6e20 refactor(host): single-source ChannelPolicy, space-signed file grants, whole-file registration
- all authorization rules (owner/creator, per-channel read/write grants, who may
  remove whom, delete rules, structure management) live in ChannelPolicy; a test
  fails if they are reimplemented elsewhere
- ChannelInfo carries what the caller may do (can_*), member and invitation lists
  say who can be removed; clients display it and never derive it
- the host no longer stores or calls the file service: it only signs short-lived
  tickets from space permissions (create/put/get/delete/domain), and keeps an opaque
  fileDomain reference on the channel
- a whole file is registered in one request (last frame header + per-frame length and
  digest) instead of one request per 1 MiB frame; the signed manifest still has to
  match every digest
- drop file quota/threshold, receipts, reconcile and delete-queue code from the host
- contract generator emits constants for enums, error codes, token roles, permissions
  and share states
2026-10-19 13:38:40 +08:00
justme ccb2c074d6 feat(files): file service owns file domains, quotas and S3 backup
- file domains are created, queried, sized and deleted by the file service
  (POST/GET/DELETE /file/v1/domains), with per-domain quota enforced on write
- /file/v1/options publishes the quota choices; nothing about files lives on
  the host any more
- tickets carry the issuing space and a unique id: a domain only accepts tickets
  from its own space, and read/delete tickets work once (no replaying a counted
  download)
- uploads stay pending until the client confirms them; abandoned ones are swept
  after 24h, and an upload ticket can cancel its own bytes
- backup target is S3 (SigV4 client, no SDK) instead of WebDAV
- CORS allows the methods the browser really uses (POST, DELETE) and exposes Retry-After
2026-10-19 13:38:29 +08:00
justme d3da3597cc Merge master into feat/file-api 2026-10-19 09:56:30 +08:00
justme d048fedc5e feat: keep every frame in animated avatar thumbnails and raise the thumbnail limit to 800k chars [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-17 21:37:35 +08:00
justme ea7895d85c feat(web): upload large avatars in two steps so the thumbnail applies immediately 2026-10-17 21:37:30 +08:00
justme a27534b2dd chore: regenerate api contract [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-17 21:31:15 +08:00
justme d1ff8bab7d feat(web): role groups, last seen and unread markers in members, channels and spaces; lock channels without keys 2026-10-17 21:31:15 +08:00
justme 5043c798ff feat(server): member last seen and a live per-account space activity stream 2026-10-17 21:31:15 +08:00
justme 78b4e2aa63 fix(server): trim every workspace structure response to the caller's grants and restrict avatars to shared spaces
- structure mutations returned the full structure to any member who could call them (moving an item after sending a message exposed channels the member has no access to)
- moving items now requires write access to the destination channel; new folders require an existing channel
- account avatars are only readable by the owner or accounts sharing a space
2026-10-17 21:31:15 +08:00
justme 72a5e5b839 feat(server): split file storage into a separate file service with tickets, receipts and WebDAV backup
- new projects WebDrop.FileApi (file service), WebDrop.FileProtocol (shared tickets and receipts) and WebDrop.Sync (WebDAV)
- file content of any size goes to the file service; chat, avatars and space structure stay on the main backend
- tickets issued by the main backend, receipts by the file service, deletion queue and daily reconciliation
- daily database snapshot and file sync to external storage (OpenList/WebDAV) with a 30 day retention
- reorganize Core, Storage and Api into feature folders (namespaces unchanged)
- web client uploads file frames to the file service and reads them with tickets
2026-10-17 15:25:16 +08:00
justme d66c93fe98 feat(web): avatar thumbnails with original preview, registration avatar, pure-JS crypto fallback for http [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-17 14:35:42 +08:00
justme 283196938e feat(web): stream video and audio while decrypting, paste images, 16px inputs on touch 2026-10-17 14:35:41 +08:00
justme 31ae3bc39e feat(server): lossy avatar thumbnails, gzip upload and separate avatar endpoints 2026-10-17 14:35:41 +08:00
justme ba0b6c258a feat(web): space avatars with GIF support and live space list sync [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-17 13:44:16 +08:00
justme 96f92c2ba9 feat(web): inline previews in chat and per-upload download limit choice 2026-10-17 13:44:16 +08:00
justme 1a4069576b feat(server): space avatars and full member profiles in the structure response 2026-10-17 13:44:16 +08:00
justme 77c9017129 chore: regenerate API contract after role color doc changes [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-17 13:26:38 +08:00
justme a08aa7568f feat(web): animated GIF avatars that refresh across chat and member list [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/push/dev-image unknown status
2026-10-17 13:25:14 +08:00
justme 3b062687cd feat(web): role colors with picker, Material 3 draw, effects and inline markdown 2026-10-17 13:25:14 +08:00
justme bef8898cd8 feat(server): owner role group, custom role colors with effects and animated avatars 2026-10-17 13:25:13 +08:00
justme 7ef298f587 fix(web): make cancel, delete and chat work while a file is uploading [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-16 19:51:45 +08:00
justme 1245aedfd2 docs: record how to measure upload speed without the Vite proxy [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-16 19:42:33 +08:00
justme a574583943 feat(web): space roles, upload queue with pipelined frames, background resume and faster sending 2026-10-16 19:42:33 +08:00
justme 96c4335cb5 feat(api): count upload frames separately and send Retry-After when rate limited 2026-10-16 19:42:33 +08:00
justme 82d55a97f9 fix(api): let authors and owners delete unfinished uploads 2026-10-16 19:42:33 +08:00
justme 1dd54243ae feat(api): add space roles with delegated member removal and structure management 2026-10-16 19:42:31 +08:00
justme 059a29a8d5 feat(web): ask for a display name when creating an account [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-16 18:30:59 +08:00
justme 915992776d feat(web): show the space validity period in the top bar 2026-10-16 18:30:59 +08:00
justme 6d6490055e feat(web): tidy the channel sidebar and let owners delete groups 2026-10-16 18:30:58 +08:00
justme 6310f93469 fix(web): verify deletions against the signer and read crypto state afresh 2026-10-16 18:30:58 +08:00
justme fc1dc08ebe feat(api): delete workspace groups and record who signed a content event 2026-10-16 18:30:58 +08:00
justme 55f1b70d89 feat(web): crop avatars in the browser with a live message preview [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/dev-image Pipeline was successful
2026-10-16 17:54:32 +08:00
justme 18c871ec85 feat(web): use the Material settings gear and add rotate icons 2026-10-16 17:54:32 +08:00
justme 18185c5bd9 refactor(web): drop the settings entry from the space header 2026-10-16 17:54:32 +08:00
justme b847916a5b feat(web): group space settings into basic, content and join tabs 2026-10-16 17:54:31 +08:00
justme 79bfd560cb feat(web): call the login name the account ID and list sign-up requirements 2026-10-16 17:54:31 +08:00
justme 700be2ebae fix(storage): retry a read transaction that fails to begin under load 2026-10-16 17:54:30 +08:00
justme fb5800f81d fix(web): show the author of incoming messages right away [CI]
ci/woodpecker/push/web-build Pipeline was successful
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/push/dev-image unknown status
Messages from other members or devices used to render as an unknown member
until the space structure refreshed. Use the author account the server
records for the object, together with the loaded profiles, when the item
arrives.
2026-10-16 07:04:31 +08:00
393 changed files with 11663 additions and 2461 deletions

No files matched your search

+51
View File
@@ -0,0 +1,51 @@
# 主后端(含内嵌前端)的开发镜像:提交信息带 [CI],并且这次推送改到了主后端用到的代码时才构建。
# 推送 webdrop-api:dev(开发机器的 podman auto-update 跟踪它)和 webdrop-api:dev-<提交>
when:
- event: push
branch: [master, main]
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
path:
include:
- 'web/**'
- 'packages/core/**'
- 'server/Containerfile'
- 'server/Directory.Build.props'
- 'server/src/WebDrop.Api/**'
- 'server/src/WebDrop.Core/**'
- 'server/src/WebDrop.Storage/**'
- 'server/src/WebDrop.Frontend/**'
- 'server/src/WebDrop.FileProtocol/**'
- 'server/src/WebDrop.Sync/**'
- 'scripts/ci-publish-image.sh'
- '.woodpecker/dev-image-api.yml'
- event: manual
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: woodpeckerci/plugin-git:2.9.3
settings:
remote:
from_secret: gitea_clone_webdrop
lfs: false
depends_on:
- ci
- web-build
concurrency: 1
steps:
publish-dev-image:
image: bash
environment:
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
ZOT_REGISTRY: registry.oeo.one
commands:
- sha="${CI_COMMIT_SHA:0:7}"
- bash scripts/ci-publish-image.sh webdrop-api server/Containerfile "dev-$sha" dev
+46
View File
@@ -0,0 +1,46 @@
# 文件服务(fileAPI)的开发镜像:提交信息带 [CI],并且这次推送改到了文件服务用到的代码时才构建。
# 推送 webdrop-files:dev(放文件的机器上的 podman auto-update 跟踪它)和 webdrop-files:dev-<提交>
when:
- event: push
branch: [master, main]
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
path:
include:
- 'server/Containerfile.files'
- 'server/Directory.Build.props'
- 'server/src/WebDrop.FileApi/**'
- 'server/src/WebDrop.FileProtocol/**'
- 'server/src/WebDrop.Sync/**'
- 'scripts/ci-publish-image.sh'
- '.woodpecker/dev-image-files.yml'
- event: manual
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: woodpeckerci/plugin-git:2.9.3
settings:
remote:
from_secret: gitea_clone_webdrop
lfs: false
# 文件服务不带前端,不用等前端构建
depends_on:
- ci
concurrency: 1
steps:
publish-dev-image:
image: bash
environment:
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
ZOT_REGISTRY: registry.oeo.one
commands:
- sha="${CI_COMMIT_SHA:0:7}"
- bash scripts/ci-publish-image.sh webdrop-files server/Containerfile.files "dev-$sha" dev
-51
View File
@@ -1,51 +0,0 @@
when:
- event: push
branch: master
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
- event: push
branch: main
evaluate: 'CI_COMMIT_MESSAGE contains "[CI]"'
labels:
ci: domain-linux-native
platform: linux/amd64
clone:
- name: git
image: woodpeckerci/plugin-git:2.9.3
settings:
remote:
from_secret: gitea_clone_webdrop
lfs: false
depends_on:
- ci
- web-build
concurrency: 1
steps:
publish-dev-image:
image: bash
environment:
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
ZOT_REGISTRY: registry.oeo.one
commands:
- set -euo pipefail
- ': "${WOODPECKER_BACKEND_LOCAL_TEMP_DIR:=$HOME/woodpecker/work}"'
- buildah_store="$(dirname "$WOODPECKER_BACKEND_LOCAL_TEMP_DIR")/buildah-cache/webdrop"
- mkdir -p "$buildah_store/storage" "$buildah_store/run"
- 'buildah() { command buildah --root "$buildah_store/storage" --runroot "$buildah_store/run" "$@"; }'
- printf '%s' "$ZOT_SECRET" | buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$ZOT_REGISTRY"
- trap 'buildah logout "$ZOT_REGISTRY" >/dev/null 2>&1 || true' EXIT
- sha="${CI_COMMIT_SHA:0:7}"
- build_tag="dev-$sha"
- image="$ZOT_REGISTRY/justme/webdrop-api:dev-$sha"
- dev_image="$ZOT_REGISTRY/justme/webdrop-api:dev"
- build_time="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- buildah bud --layers --network host --format oci --platform linux/amd64 --build-arg "WEBDROP_FRONTEND_COMMIT=$CI_COMMIT_SHA" --build-arg "WEBDROP_FRONTEND_TAG=$build_tag" --build-arg "WEBDROP_BACKEND_COMMIT=$CI_COMMIT_SHA" --build-arg "WEBDROP_BACKEND_TAG=$build_tag" --build-arg "WEBDROP_BUILD_TIME=$build_time" --tag "$image" --file server/Containerfile .
- buildah tag "$image" "$dev_image"
- buildah push "$image" "docker://$image"
- buildah push "$dev_image" "docker://$dev_image"
@@ -1,3 +1,6 @@
# 发布:打 vMAJOR.MINOR.PATCH tag(或手动运行并传 RELEASE_TAG)。
# 一个 tag 同时发布两个镜像:webdrop-api:<tag> / :latest 和 webdrop-files:<tag> / :latest。
# 两边共用票据格式和文件协议,版本号一致最不容易配错;各机器只会拉自己用的那个镜像。
when: when:
- event: tag - event: tag
ref: refs/tags/v* ref: refs/tags/v*
@@ -37,29 +40,26 @@ steps:
- NUGET_PACKAGES="$HOME/.cache/woodpecker/webdrop/nuget" dotnet restore server/WebDrop.slnx - NUGET_PACKAGES="$HOME/.cache/woodpecker/webdrop/nuget" dotnet restore server/WebDrop.slnx
- NUGET_PACKAGES="$HOME/.cache/woodpecker/webdrop/nuget" dotnet test server/WebDrop.slnx --configuration Release --verbosity minimal --no-restore - NUGET_PACKAGES="$HOME/.cache/woodpecker/webdrop/nuget" dotnet test server/WebDrop.slnx --configuration Release --verbosity minimal --no-restore
publish-backend: publish-api:
image: bash image: bash
depends_on: [release-checks]
environment: environment:
ZOT_USERNAME: justme ZOT_USERNAME: justme
ZOT_SECRET: ZOT_SECRET:
from_secret: zot_secret from_secret: zot_secret
ZOT_REGISTRY: registry.oeo.one ZOT_REGISTRY: registry.oeo.one
commands: commands:
- set -euo pipefail
- release_tag="${CI_COMMIT_TAG:-${RELEASE_TAG:-}}" - release_tag="${CI_COMMIT_TAG:-${RELEASE_TAG:-}}"
- test -n "$release_tag" - bash scripts/ci-publish-image.sh webdrop-api server/Containerfile "$release_tag" latest
- ': "${WOODPECKER_BACKEND_LOCAL_TEMP_DIR:=$HOME/woodpecker/work}"'
- buildah_store="$(dirname "$WOODPECKER_BACKEND_LOCAL_TEMP_DIR")/buildah-cache/webdrop"
- mkdir -p "$buildah_store/storage" "$buildah_store/run"
- 'buildah() { command buildah --root "$buildah_store/storage" --runroot "$buildah_store/run" "$@"; }'
- printf '%s' "$ZOT_SECRET" | buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$ZOT_REGISTRY"
- trap 'buildah logout "$ZOT_REGISTRY" >/dev/null 2>&1 || true' EXIT
- image="$ZOT_REGISTRY/justme/webdrop-api:$release_tag"
- latest_image="$ZOT_REGISTRY/justme/webdrop-api:latest"
- build_commit="$(git rev-parse HEAD)"
- build_time="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- buildah bud --layers --network host --format oci --platform linux/amd64 --build-arg "WEBDROP_FRONTEND_COMMIT=$build_commit" --build-arg "WEBDROP_FRONTEND_TAG=$release_tag" --build-arg "WEBDROP_BACKEND_COMMIT=$build_commit" --build-arg "WEBDROP_BACKEND_TAG=$release_tag" --build-arg "WEBDROP_BUILD_TIME=$build_time" --tag "$image" --file server/Containerfile .
- buildah tag "$image" "$latest_image"
- buildah push "$image" "docker://$image"
- buildah push "$latest_image" "docker://$latest_image"
publish-files:
image: bash
depends_on: [release-checks]
environment:
ZOT_USERNAME: justme
ZOT_SECRET:
from_secret: zot_secret
ZOT_REGISTRY: registry.oeo.one
commands:
- release_tag="${CI_COMMIT_TAG:-${RELEASE_TAG:-}}"
- bash scripts/ci-publish-image.sh webdrop-files server/Containerfile.files "$release_tag" latest
+1 -1
View File
@@ -1,3 +1,3 @@
{ {
"version": "8.0.0" "version": "8.2.0"
} }
+373 -10
View File
@@ -1,6 +1,6 @@
{ {
"$schema": "https://json-schema.org/draft/2020-12/schema", "$schema": "https://json-schema.org/draft/2020-12/schema",
"x-backend-contract-sha256": "655e918f3443073c270bdeb9ae4e69d79dacf83baa90dea1a0cdcfe71d2b8bd5", "x-backend-contract-sha256": "89c6298b865197835486f01891acc597802367a7ae1b89f21ba882a52935df3b",
"$defs": { "$defs": {
"AccountChannelBody": { "AccountChannelBody": {
"type": "object", "type": "object",
@@ -98,6 +98,9 @@
"displayName": { "displayName": {
"type": "string" "type": "string"
}, },
"hasAvatar": {
"type": "boolean"
},
"modifiedAt": { "modifiedAt": {
"type": "integer" "type": "integer"
}, },
@@ -107,6 +110,7 @@
}, },
"required": [ "required": [
"displayName", "displayName",
"hasAvatar",
"modifiedAt", "modifiedAt",
"username" "username"
] ]
@@ -124,12 +128,26 @@
} }
] ]
}, },
"avatarThumb": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"displayName": { "displayName": {
"type": "string" "type": "string"
},
"keepAvatar": {
"type": "boolean"
} }
}, },
"required": [ "required": [
"displayName" "displayName",
"keepAvatar"
] ]
}, },
"AccountRegisterRequest": { "AccountRegisterRequest": {
@@ -294,9 +312,43 @@
"ChannelInfo": { "ChannelInfo": {
"type": "object", "type": "object",
"properties": { "properties": {
"avatar_version": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"can_create_channel": {
"type": "boolean"
},
"can_delete_any": {
"type": "boolean"
},
"can_delete_own": {
"type": "boolean"
},
"can_edit_settings": {
"type": "boolean"
},
"can_invite": {
"type": "boolean"
},
"can_manage_folders": {
"type": "boolean"
},
"can_manage_roles": {
"type": "boolean"
},
"can_manage_structure": { "can_manage_structure": {
"type": "boolean" "type": "boolean"
}, },
"can_remove_members": {
"type": "boolean"
},
"can_upload": { "can_upload": {
"type": "boolean" "type": "boolean"
}, },
@@ -319,6 +371,9 @@
"id": { "id": {
"type": "string" "type": "string"
}, },
"is_owner": {
"type": "boolean"
},
"last_seq": { "last_seq": {
"type": "integer" "type": "integer"
}, },
@@ -339,11 +394,20 @@
} }
}, },
"required": [ "required": [
"can_create_channel",
"can_delete_any",
"can_delete_own",
"can_edit_settings",
"can_invite",
"can_manage_folders",
"can_manage_roles",
"can_manage_structure", "can_manage_structure",
"can_remove_members",
"can_upload", "can_upload",
"created_at", "created_at",
"expires_at", "expires_at",
"id", "id",
"is_owner",
"last_seq", "last_seq",
"members_used", "members_used",
"mode", "mode",
@@ -411,6 +475,26 @@
"ChannelSettingsPatch": { "ChannelSettingsPatch": {
"type": "object", "type": "object",
"properties": { "properties": {
"avatar": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"avatar_thumb": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"delete_policy": { "delete_policy": {
"anyOf": [ "anyOf": [
{ {
@@ -796,6 +880,64 @@
"error" "error"
] ]
}, },
"FileGrantBody": {
"type": "object",
"properties": {
"domain": {
"type": "string"
},
"expires_at": {
"type": "integer"
},
"ticket": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"domain",
"expires_at",
"ticket",
"url"
]
},
"FileGrantRequest": {
"type": "object",
"properties": {
"channel": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"from": {
"type": "integer"
},
"item": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"op": {
"type": "string"
}
},
"required": [
"from",
"op"
]
},
"FinishShareRequest": { "FinishShareRequest": {
"type": "object", "type": "object",
"properties": { "properties": {
@@ -830,6 +972,9 @@
"InvitationBody": { "InvitationBody": {
"type": "object", "type": "object",
"properties": { "properties": {
"can_revoke": {
"type": "boolean"
},
"channel_id": { "channel_id": {
"type": "string" "type": "string"
}, },
@@ -844,6 +989,7 @@
} }
}, },
"required": [ "required": [
"can_revoke",
"channel_id", "channel_id",
"created_at", "created_at",
"payload", "payload",
@@ -982,12 +1128,18 @@
"MemberBody": { "MemberBody": {
"type": "object", "type": "object",
"properties": { "properties": {
"can_remove": {
"type": "boolean"
},
"devices": { "devices": {
"type": "integer" "type": "integer"
}, },
"joined_at": { "joined_at": {
"type": "integer" "type": "integer"
}, },
"last_seen": {
"type": "integer"
},
"online": { "online": {
"type": "boolean" "type": "boolean"
}, },
@@ -1002,8 +1154,10 @@
} }
}, },
"required": [ "required": [
"can_remove",
"devices", "devices",
"joined_at", "joined_at",
"last_seen",
"online", "online",
"online_devices", "online_devices",
"role", "role",
@@ -1302,6 +1456,9 @@
"defaults": { "defaults": {
"$ref": "#/$defs/DefaultsBody" "$ref": "#/$defs/DefaultsBody"
}, },
"files": {
"$ref": "#/$defs/FilesInfoBody"
},
"limits": { "limits": {
"$ref": "#/$defs/LimitsBody" "$ref": "#/$defs/LimitsBody"
}, },
@@ -1317,6 +1474,7 @@
"choices", "choices",
"contract_version", "contract_version",
"defaults", "defaults",
"files",
"limits", "limits",
"protocol", "protocol",
"transports" "transports"
@@ -1366,6 +1524,27 @@
"share" "share"
] ]
}, },
"FilesInfoBody": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
},
"url": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
}
},
"required": [
"enabled"
]
},
"LimitsBody": { "LimitsBody": {
"type": "object", "type": "object",
"properties": { "properties": {
@@ -1393,9 +1572,6 @@
"max_workspace_bytes": { "max_workspace_bytes": {
"type": "integer" "type": "integer"
}, },
"max_workspace_channel_bytes": {
"type": "integer"
},
"min_ttl_seconds": { "min_ttl_seconds": {
"type": "integer" "type": "integer"
} }
@@ -1409,7 +1585,6 @@
"max_ttl_seconds", "max_ttl_seconds",
"max_visitors", "max_visitors",
"max_workspace_bytes", "max_workspace_bytes",
"max_workspace_channel_bytes",
"min_ttl_seconds" "min_ttl_seconds"
] ]
}, },
@@ -1616,6 +1791,16 @@
"signature": { "signature": {
"type": "string" "type": "string"
}, },
"signerAccount": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"signerDeviceId": { "signerDeviceId": {
"type": "string" "type": "string"
}, },
@@ -1715,8 +1900,15 @@
"createdBy": { "createdBy": {
"type": "string" "type": "string"
}, },
"fileBytes": { "fileDomain": {
"type": "integer" "anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
}, },
"filesEpoch": { "filesEpoch": {
"type": "integer" "type": "integer"
@@ -1746,7 +1938,6 @@
"chatHistory", "chatHistory",
"createdAt", "createdAt",
"createdBy", "createdBy",
"fileBytes",
"filesEpoch", "filesEpoch",
"filesHistory", "filesHistory",
"id", "id",
@@ -1756,6 +1947,16 @@
"WorkspaceChannelRequest": { "WorkspaceChannelRequest": {
"type": "object", "type": "object",
"properties": { "properties": {
"fileDomain": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"groupId": { "groupId": {
"anyOf": [ "anyOf": [
{ {
@@ -1774,6 +1975,17 @@
"name" "name"
] ]
}, },
"WorkspaceFileDomainRequest": {
"type": "object",
"properties": {
"fileDomain": {
"type": "string"
}
},
"required": [
"fileDomain"
]
},
"WorkspaceFolderRequest": { "WorkspaceFolderRequest": {
"type": "object", "type": "object",
"properties": { "properties": {
@@ -1888,6 +2100,146 @@
"name" "name"
] ]
}, },
"WorkspaceRoleCreatedBody": {
"type": "object",
"properties": {
"role": {
"$ref": "#/$defs/WorkspaceRole"
},
"workspace": {
"$ref": "#/$defs/WorkspaceBody"
}
},
"required": [
"role",
"workspace"
]
},
"WorkspaceRole": {
"type": "object",
"properties": {
"color": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"createdAt": {
"type": "integer"
},
"id": {
"type": "string"
},
"members": {
"type": "array",
"items": {
"type": "string"
}
},
"name": {
"type": "string"
},
"permissions": {
"type": "array",
"items": {
"type": "string"
}
},
"prefix": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"suffix": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
}
},
"required": [
"createdAt",
"id",
"members",
"name",
"permissions"
]
},
"WorkspaceRoleMembersRequest": {
"type": "object",
"properties": {
"members": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"members"
]
},
"WorkspaceRoleRequest": {
"type": "object",
"properties": {
"color": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"name": {
"type": "string"
},
"permissions": {
"type": "array",
"items": {
"type": "string"
}
},
"prefix": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
},
"suffix": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
]
}
},
"required": [
"name",
"permissions"
]
},
"WorkspaceState": { "WorkspaceState": {
"type": "object", "type": "object",
"properties": { "properties": {
@@ -1927,6 +2279,15 @@
}, },
"revision": { "revision": {
"type": "integer" "type": "integer"
},
"roles": {
"type": "array",
"items": {
"$ref": "#/$defs/WorkspaceRole"
}
},
"rolesSeeded": {
"type": "boolean"
} }
}, },
"required": [ "required": [
@@ -1934,7 +2295,9 @@
"folders", "folders",
"groups", "groups",
"locations", "locations",
"revision" "revision",
"roles",
"rolesSeeded"
] ]
}, },
"WorkspaceFolder": { "WorkspaceFolder": {
+1
View File
@@ -10,6 +10,7 @@ PublishPort=127.0.0.1:5001:5000
Environment=ASPNETCORE_URLS=http://+:5000 Environment=ASPNETCORE_URLS=http://+:5000
Environment=WebDrop__DataDir=/data Environment=WebDrop__DataDir=/data
Environment=DOTNET_EnableDiagnostics=0 Environment=DOTNET_EnableDiagnostics=0
EnvironmentFile=-%h/podman/webdrop-dev/webdrop.env
Volume=%h/podman/webdrop-dev/data:/data:Z,U Volume=%h/podman/webdrop-dev/data:/data:Z,U
ReadOnly=true ReadOnly=true
ReadOnlyTmpfs=false ReadOnlyTmpfs=false
+29
View File
@@ -0,0 +1,29 @@
[Unit]
Description=WebDrop file service (fileAPI) development environment from registry
Wants=network-online.target
After=network-online.target
[Container]
ContainerName=webdrop-files-dev
Image=registry.oeo.one/justme/webdrop-files:dev
Pull=missing
AutoUpdate=registry
PublishPort=127.0.0.1:5101:5100
Environment=ASPNETCORE_URLS=http://+:5100
Environment=FileApi__DataDir=/data
Environment=DOTNET_EnableDiagnostics=0
EnvironmentFile=%h/podman/webdrop-files-dev/webdrop-files.env
Volume=%h/podman/webdrop-files-dev/data:/data:Z,U
ReadOnly=true
ReadOnlyTmpfs=false
Tmpfs=/tmp:rw,noexec,nosuid,size=16m
DropCapability=ALL
NoNewPrivileges=true
[Service]
Restart=always
TimeoutStartSec=120
TimeoutStopSec=60
[Install]
WantedBy=default.target
+31
View File
@@ -0,0 +1,31 @@
[Unit]
Description=WebDrop file service (fileAPI)
Wants=network-online.target
After=network-online.target
[Container]
ContainerName=webdrop-files
# 正式环境:跟踪打 v* tag 发布的 latest。放文件的机器(例如家里的 NAS)只拉这个镜像,不需要主后端的
Image=registry.oeo.one/justme/webdrop-files:latest
Pull=missing
AutoUpdate=registry
PublishPort=127.0.0.1:5100:5100
Environment=ASPNETCORE_URLS=http://+:5100
Environment=FileApi__DataDir=/data
Environment=DOTNET_EnableDiagnostics=0
# 和主后端共用的密钥、允许的页面来源、备份设置等,不进仓库。例子见 webdrop-files.env.example
EnvironmentFile=%h/podman/webdrop-files/webdrop-files.env
Volume=%h/podman/webdrop-files/data:/data:Z,U
ReadOnly=true
ReadOnlyTmpfs=false
Tmpfs=/tmp:rw,noexec,nosuid,size=16m
DropCapability=ALL
NoNewPrivileges=true
[Service]
Restart=always
TimeoutStartSec=120
TimeoutStopSec=60
[Install]
WantedBy=default.target
+18
View File
@@ -0,0 +1,18 @@
# 文件服务(家里的 NAS)的机器配置:复制到 ~/podman/webdrop-files/webdrop-files.env 并按实际情况修改。不要提交真实值
# 和主后端共用的密钥,至少 32 个字符。两边必须一致
FileApi__Secret=
# 页面所在的源:浏览器直接把文件传到这里,必须列出,否则浏览器会拦下跨域请求
FileApi__CorsOrigins__0=https://app.example.com
# 创建频道时可选的文件空间档位(字节)和默认值;不配用内置的 1/5/10/50/100 GiB,默认 10 GiB
# FileApi__QuotaChoices__0=1073741824
# FileApi__QuotaChoices__1=10737418240
# FileApi__DefaultQuota=10737418240
# 文件服务最多存多少字节,0 为不限
# FileApi__MaxStoreBytes=0
# 每天把新增的文件备份到 S3 协议的存储
# FileApi__Backup__Enabled=true
# FileApi__Backup__Time=04:30
# FileApi__Backup__S3__Endpoint=https://s3.example.com
# FileApi__Backup__S3__Bucket=webdrop
# FileApi__Backup__S3__AccessKey=
# FileApi__Backup__S3__SecretKey=
+9 -3
View File
@@ -1,15 +1,21 @@
[Unit] [Unit]
Description=WebDrop API with embedded frontend Description=WebDrop API with embedded frontend
Wants=network-online.target
After=network-online.target
[Container] [Container]
ContainerName=webdrop ContainerName=webdrop
Image=localhost/webdrop:local # 正式环境:跟踪打 v* tag 发布的 latest。podman-auto-update.timer 每隔几分钟检查 registry,有新镜像就拉取并重启这个单元
Pull=never Image=registry.oeo.one/justme/webdrop-api:latest
Pull=missing
AutoUpdate=registry
PublishPort=127.0.0.1:5000:5000 PublishPort=127.0.0.1:5000:5000
Environment=ASPNETCORE_URLS=http://+:5000 Environment=ASPNETCORE_URLS=http://+:5000
Environment=WebDrop__DataDir=/data Environment=WebDrop__DataDir=/data
Environment=DOTNET_EnableDiagnostics=0 Environment=DOTNET_EnableDiagnostics=0
Volume=/home/justme/podman/webdrop/data:/data:Z,U # 机器上的配置(文件服务的地址和共用密钥等),不进仓库。例子见 webdrop.env.example
EnvironmentFile=-%h/podman/webdrop/webdrop.env
Volume=%h/podman/webdrop/data:/data:Z,U
ReadOnly=true ReadOnly=true
ReadOnlyTmpfs=false ReadOnlyTmpfs=false
Tmpfs=/tmp:rw,noexec,nosuid,size=16m Tmpfs=/tmp:rw,noexec,nosuid,size=16m
+7
View File
@@ -0,0 +1,7 @@
# 主后端(VPS)的机器配置:复制到 ~/podman/webdrop/webdrop.env 并按实际情况修改。不要提交真实值
# 打开文件服务:文件内容不放主后端,由 fileAPI 保存;主后端只签发访问票据
WebDrop__Files__Enabled=true
# 浏览器访问文件服务的地址(必须 https,和文件服务机器上配的对外地址一致)
WebDrop__Files__PublicUrl=https://files.example.com
# 和文件服务共用的密钥,至少 32 个字符:openssl rand -base64 48。两边必须一致
WebDrop__Files__Secret=
+1 -1
View File
@@ -4,7 +4,7 @@
空间结构使用 `GET /spaces/{id}/structure`,包含 revision、channels、groups、folders、locations。频道与分组分别通过 `/spaces/{id}/channels` 和 `/spaces/{id}/groups` 创建;响应同时包含完整结构快照和服务端新建实体及 ID。其他字段通过 PATCH 重命名;频道归属通过 `PUT /spaces/{id}/channels/{channelId}/group` 修改。文件夹通过 `/spaces/{id}/folders` 创建及 PATCH 重命名,文件归属通过 `PUT /spaces/{id}/locations` 修改。 空间结构使用 `GET /spaces/{id}/structure`,包含 revision、channels、groups、folders、locations。频道与分组分别通过 `/spaces/{id}/channels` 和 `/spaces/{id}/groups` 创建;响应同时包含完整结构快照和服务端新建实体及 ID。其他字段通过 PATCH 重命名;频道归属通过 `PUT /spaces/{id}/channels/{channelId}/group` 修改。文件夹通过 `/spaces/{id}/folders` 创建及 PATCH 重命名,文件归属通过 `PUT /spaces/{id}/locations` 修改。
频道、分组和目录写操作需要账号会话和空间写权限;允许上传的成员可以关联和移动自己的资源,文件移动必须满足每个条目的作者或管理权限,跨空间、跨频道的错误分组/目录关联被拒绝。结构修改后通知订阅设备重新拉取。 频道、分组和目录写操作需要账号会话和空间写权限;允许上传的成员可以关联和移动自己的资源,文件移动必须满足每个条目的作者或管理权限,跨空间、跨频道的错误分组/目录关联被拒绝。结构修改后通知订阅设备重新拉取。
URL 基础路径保留 `/api/v1`,`/info.contract_version` 为 `8.0.0`。v8 让频道强制登录:去掉未登录访客(设置 `allow_guests`、`/guests` 接口、访客终端席位),频道与空间的所有接口都要求账号会话,缺少时返回 `401 account_required`;便捷分享(`/shares`)不受影响。v7 把「设备」收束为账号的属性:成员只按用户管理,`seats` 系列命名改为 `max_members` / `members_used` / `members_full`,去掉设备名与设备清单接口,登录账号不再需要终端密钥。v6 引入账号签名根密钥、设备证书和频道对象签名事件;不兼容的旧主版本客户端会被契约预检拒绝。账号资料与空间结构由服务端管理。创建频道/分享以及所有 owner 管理操作必须携带有效账号 session;频道链接只对登录账号有效,未登录的人先登录再按加入规则加入。 URL 基础路径保留 `/api/v1`,`/info.contract_version` 为 `8.2.0`(8.2 新增身份组 `/spaces/{id}/roles`:结构快照多了 `roles`、`rolesSeeded`,内容事件记录多了 `signerAccount`;8.1 新增 `DELETE /spaces/{id}/groups/{groupId}`:删除分组,组内频道回到未分组)。v8 让频道强制登录:去掉未登录访客(设置 `allow_guests`、`/guests` 接口、访客终端席位),频道与空间的所有接口都要求账号会话,缺少时返回 `401 account_required`;便捷分享(`/shares`)不受影响。v7 把「设备」收束为账号的属性:成员只按用户管理,`seats` 系列命名改为 `max_members` / `members_used` / `members_full`,去掉设备名与设备清单接口,登录账号不再需要终端密钥。v6 引入账号签名根密钥、设备证书和频道对象签名事件;不兼容的旧主版本客户端会被契约预检拒绝。账号资料与空间结构由服务端管理。创建频道/分享以及所有 owner 管理操作必须携带有效账号 session;频道链接只对登录账号有效,未登录的人先登录再按加入规则加入。
`GET /info` 的 `defaults` 包含 `channel`、`share` 创建设置;省略创建请求的 settings 时服务端使用相同默认值。`choices.max_members` 与 `limits.max_members` 表示成员人数范围,不代表设备数。频道设置 JSON 字段为 `max_members`(旧存储里的 `seats` 在启动时迁移)。其他选项按部署限制收束,前端只格式化选项文案,不提供替代业务默认值。 `GET /info` 的 `defaults` 包含 `channel`、`share` 创建设置;省略创建请求的 settings 时服务端使用相同默认值。`choices.max_members` 与 `limits.max_members` 表示成员人数范围,不代表设备数。频道设置 JSON 字段为 `max_members`(旧存储里的 `seats` 在启动时迁移)。其他选项按部署限制收束,前端只格式化选项文案,不提供替代业务默认值。
+14
View File
@@ -28,6 +28,20 @@ systemctl --user start webdrop.service
本机开发前端仍可运行 `cd web && npm run dev`,Vite 会代理 `/api/v1` 到本地 API。公网部署时,在 API 前配置 HTTPS 反向代理;不要把未加密 HTTP 直接暴露到公网。只有当 API 端口仅能由可信代理访问时,才信任 forwarded headers。 本机开发前端仍可运行 `cd web && npm run dev`,Vite 会代理 `/api/v1` 到本地 API。公网部署时,在 API 前配置 HTTPS 反向代理;不要把未加密 HTTP 直接暴露到公网。只有当 API 端口仅能由可信代理访问时,才信任 forwarded headers。
### 开发时测上传速度
经 Vite 开发代理(`npm run dev` 默认)转发上传,速度上限约 20–25 MB/s,这是 Node 代理的限制,和应用本身无关。要测到真实速度,让前端直连后端:
```sh
# 后端:允许前端所在的来源跨域(端口按实际的前端端口填)
WebDrop__CorsOrigins__0=http://127.0.0.1:5173 ASPNETCORE_URLS=http://127.0.0.1:5000 dotnet run --project server/src/WebDrop.Api --no-launch-profile
# 前端:把 API 指向后端,不走代理
cd web && VITE_WEBDROP_API_URL=http://127.0.0.1:5000/api/v1 npm run dev
```
本机实测直连约 90 MB/s(300 MB 随机文件 3.3 秒)。上传帧的限流和普通接口分开计数(`WebDrop:FrameRequestsPerMinute`,默认 6000/分钟),上传请求体用 Blob 发送(浏览器对字节数组请求体要多拷贝几次,会慢 5 倍以上)。
## 其他 Linux 主机 ## 其他 Linux 主机
构建 OCI 镜像: 构建 OCI 镜像:
+173 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.1.0 openapi: 3.1.0
info: info:
title: WebDrop API title: WebDrop API
version: 8.0.0 version: 8.2.0
servers: servers:
- url: /api/v1 - url: /api/v1
@@ -291,6 +291,178 @@ paths:
'403': { $ref: '#/components/responses/Error' } '403': { $ref: '#/components/responses/Error' }
'404': { $ref: '#/components/responses/Error' } '404': { $ref: '#/components/responses/Error' }
'409': { $ref: '#/components/responses/Error' } '409': { $ref: '#/components/responses/Error' }
delete:
tags: [channel]
summary: 删除分组;组内频道回到未分组,频道与内容保留
parameters:
- name: id
in: path
required: true
schema: { type: string }
- name: groupId
in: path
required: true
schema: { type: string, format: uuid }
- name: X-WD-Account
in: header
required: true
schema: { type: string }
- $ref: '#/components/parameters/RoleWriter'
- $ref: '#/components/parameters/Account'
- name: Authorization
in: header
schema: { type: string }
responses:
'200':
description: 服务端权威数据
content:
application/json:
schema:
$ref: '../contracts/api.schemas.json#/$defs/WorkspaceBody'
'401': { $ref: '#/components/responses/Error' }
'403': { $ref: '#/components/responses/Error' }
'404': { $ref: '#/components/responses/Error' }
/spaces/{id}/roles:
post:
tags: [channel]
summary: 新建身份组(只有空间主)
parameters:
- name: id
in: path
required: true
schema: { type: string }
- name: X-WD-Account
in: header
required: true
schema: { type: string }
- $ref: '#/components/parameters/RoleWriter'
- $ref: '#/components/parameters/Account'
- name: Authorization
in: header
schema: { type: string }
requestBody:
required: true
content:
application/json:
schema: { $ref: '../contracts/api.schemas.json#/$defs/WorkspaceRoleRequest' }
responses:
'201':
description: 完整结构快照和新建的身份组
content:
application/json:
schema:
$ref: '../contracts/api.schemas.json#/$defs/WorkspaceRoleCreatedBody'
'400': { $ref: '#/components/responses/Error' }
'401': { $ref: '#/components/responses/Error' }
'403': { $ref: '#/components/responses/Error' }
/spaces/{id}/roles/{roleId}:
patch:
tags: [channel]
summary: 修改身份组的名称、前缀后缀、颜色和权限(只有空间主)
parameters:
- name: id
in: path
required: true
schema: { type: string }
- name: roleId
in: path
required: true
schema: { type: string, format: uuid }
- name: X-WD-Account
in: header
required: true
schema: { type: string }
- $ref: '#/components/parameters/RoleWriter'
- $ref: '#/components/parameters/Account'
- name: Authorization
in: header
schema: { type: string }
requestBody:
required: true
content:
application/json:
schema: { $ref: '../contracts/api.schemas.json#/$defs/WorkspaceRoleRequest' }
responses:
'200':
description: 服务端权威数据
content:
application/json:
schema:
$ref: '../contracts/api.schemas.json#/$defs/WorkspaceBody'
'400': { $ref: '#/components/responses/Error' }
'401': { $ref: '#/components/responses/Error' }
'403': { $ref: '#/components/responses/Error' }
'404': { $ref: '#/components/responses/Error' }
delete:
tags: [channel]
summary: 删除身份组(只有空间主);其成员失去它授予的权限
parameters:
- name: id
in: path
required: true
schema: { type: string }
- name: roleId
in: path
required: true
schema: { type: string, format: uuid }
- name: X-WD-Account
in: header
required: true
schema: { type: string }
- $ref: '#/components/parameters/RoleWriter'
- $ref: '#/components/parameters/Account'
- name: Authorization
in: header
schema: { type: string }
responses:
'200':
description: 服务端权威数据
content:
application/json:
schema:
$ref: '../contracts/api.schemas.json#/$defs/WorkspaceBody'
'400': { $ref: '#/components/responses/Error' }
'401': { $ref: '#/components/responses/Error' }
'403': { $ref: '#/components/responses/Error' }
'404': { $ref: '#/components/responses/Error' }
/spaces/{id}/roles/{roleId}/members:
put:
tags: [channel]
summary: 整体替换身份组的成员(只有空间主);成员必须已加入空间
parameters:
- name: id
in: path
required: true
schema: { type: string }
- name: roleId
in: path
required: true
schema: { type: string, format: uuid }
- name: X-WD-Account
in: header
required: true
schema: { type: string }
- $ref: '#/components/parameters/RoleWriter'
- $ref: '#/components/parameters/Account'
- name: Authorization
in: header
schema: { type: string }
requestBody:
required: true
content:
application/json:
schema: { $ref: '../contracts/api.schemas.json#/$defs/WorkspaceRoleMembersRequest' }
responses:
'200':
description: 服务端权威数据
content:
application/json:
schema:
$ref: '../contracts/api.schemas.json#/$defs/WorkspaceBody'
'400': { $ref: '#/components/responses/Error' }
'401': { $ref: '#/components/responses/Error' }
'403': { $ref: '#/components/responses/Error' }
'404': { $ref: '#/components/responses/Error' }
/spaces/{id}/folders: /spaces/{id}/folders:
post: post:
tags: [channel] tags: [channel]
+5 -5
View File
@@ -9,9 +9,9 @@ const fail = (message) => {
process.exit(1); process.exit(1);
}; };
const dtoSources = ['server/src/WebDrop.Api/ApiJson.cs', 'server/src/WebDrop.Core/Dtos.cs', 'server/src/WebDrop.Core/Settings.cs', 'server/src/WebDrop.Core/Enums.cs', 'server/src/WebDrop.Core/Workspace.cs', 'server/src/WebDrop.Core/AccountProfile.cs', 'server/src/WebDrop.Api/PasskeyDtos.cs', 'server/src/WebDrop.Core/WorkspaceCrypto.cs', 'server/src/WebDrop.Api/Endpoints.WorkspaceCrypto.cs']; const dtoSources = ['server/src/WebDrop.Api/Json/ApiJson.cs', 'server/src/WebDrop.Core/Channels/Dtos.cs', 'server/src/WebDrop.Core/Channels/Settings.cs', 'server/src/WebDrop.Core/Common/Enums.cs', 'server/src/WebDrop.Core/Spaces/Workspace.cs', 'server/src/WebDrop.Core/Accounts/AccountProfile.cs', 'server/src/WebDrop.Api/Passkeys/PasskeyDtos.cs', 'server/src/WebDrop.Core/Spaces/WorkspaceCrypto.cs', 'server/src/WebDrop.Api/Endpoints/Endpoints.WorkspaceCrypto.cs'];
const dtoDigest = createHash('sha256').update(dtoSources.map(path => `${path}\n${read(path).replace(/\r\n/g, '\n')}\n`).join('')).digest('hex'); const dtoDigest = createHash('sha256').update(dtoSources.map(path => `${path}\n${read(path).replace(/\r\n/g, '\n')}\n`).join('')).digest('hex');
const generatedDigest = /backend-contract-sha256: ([a-f0-9]{64})/.exec(read('web/src/core-bridge/generated/api.ts'))?.[1]; const generatedDigest = /backend-contract-sha256: ([a-f0-9]{64})/.exec(read('web/src/api/generated.ts'))?.[1];
if (generatedDigest !== dtoDigest) fail('Backend DTOs changed. Regenerate with dotnet run --project tools/ContractGenerator -- .'); if (generatedDigest !== dtoDigest) fail('Backend DTOs changed. Regenerate with dotnet run --project tools/ContractGenerator -- .');
if (JSON.parse(read('contracts/api.schemas.json'))['x-backend-contract-sha256'] !== dtoDigest) fail('Generated API schemas are stale. Run ContractGenerator.'); if (JSON.parse(read('contracts/api.schemas.json'))['x-backend-contract-sha256'] !== dtoDigest) fail('Generated API schemas are stale. Run ContractGenerator.');
@@ -24,14 +24,14 @@ const openapi = read('docs/openapi.yaml');
const openapiVersion = /^info:\s*\r?\n(?:.*\r?\n)*?\s+version:\s*([^\s#]+)/m.exec(openapi)?.[1]; const openapiVersion = /^info:\s*\r?\n(?:.*\r?\n)*?\s+version:\s*([^\s#]+)/m.exec(openapi)?.[1];
if (openapiVersion !== contract.version) fail(`OpenAPI version ${openapiVersion ?? '(missing)'} differs from ${contract.version}.`); if (openapiVersion !== contract.version) fail(`OpenAPI version ${openapiVersion ?? '(missing)'} differs from ${contract.version}.`);
const backend = read('server/src/WebDrop.Api/ApiContract.cs'); const backend = read('server/src/WebDrop.Api/Hosting/ApiContract.cs');
const backendVersion = /const string Version = "([^"]+)"/.exec(backend)?.[1]; const backendVersion = /const string Version = "([^"]+)"/.exec(backend)?.[1];
if (backendVersion !== contract.version) fail(`backend version ${backendVersion ?? '(missing)'} differs from ${contract.version}.`); if (backendVersion !== contract.version) fail(`backend version ${backendVersion ?? '(missing)'} differs from ${contract.version}.`);
const frontend = read('web/src/core-bridge/contract.ts'); const frontend = read('web/src/api/contract.ts');
const supportedMajor = Number(/SUPPORTED_API_CONTRACT_MAJOR\s*=\s*(\d+)/.exec(frontend)?.[1]); const supportedMajor = Number(/SUPPORTED_API_CONTRACT_MAJOR\s*=\s*(\d+)/.exec(frontend)?.[1]);
if (supportedMajor !== apiMajor) { if (supportedMajor !== apiMajor) {
fail(`backend contract is ${contract.version}; frontend still supports API major ${supportedMajor || '(missing)'}. Update web/src/core-bridge/contract.ts.`); fail(`backend contract is ${contract.version}; frontend still supports API major ${supportedMajor || '(missing)'}. Update web/src/api/contract.ts.`);
} }
const tag = process.argv[2]; const tag = process.argv[2];
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
# 构建并推送一个镜像到 registry。开发镜像和发布镜像、主后端和文件服务都用它,构建和推送的细节只写在这里。
# 用法: ci-publish-image.sh <镜像名> <Containerfile> <版本标签> [额外标签...]
# 例: ci-publish-image.sh webdrop-api server/Containerfile dev-abc1234 dev
# 需要环境变量: ZOT_REGISTRY、ZOT_USERNAME、ZOT_SECRET(Woodpecker 的密钥)
set -euo pipefail
name="$1"; containerfile="$2"; version="$3"; shift 3
: "${ZOT_REGISTRY:?}" "${ZOT_USERNAME:?}" "${ZOT_SECRET:?}"
: "${WOODPECKER_BACKEND_LOCAL_TEMP_DIR:=$HOME/woodpecker/work}"
# 每个镜像一个构建缓存目录,两个镜像并行构建时互不干扰
store="$(dirname "$WOODPECKER_BACKEND_LOCAL_TEMP_DIR")/buildah-cache/$name"
mkdir -p "$store/storage" "$store/run"
buildah() { command buildah --root "$store/storage" --runroot "$store/run" "$@"; }
printf '%s' "$ZOT_SECRET" | buildah login --tls-verify=true --username "$ZOT_USERNAME" --password-stdin "$ZOT_REGISTRY"
trap 'buildah logout "$ZOT_REGISTRY" >/dev/null 2>&1 || true' EXIT
commit="${CI_COMMIT_SHA:-$(git rev-parse HEAD)}"
image="$ZOT_REGISTRY/justme/$name:$version"
buildah bud --layers --network host --format oci --platform linux/amd64 \
--build-arg "WEBDROP_FRONTEND_COMMIT=$commit" --build-arg "WEBDROP_FRONTEND_TAG=$version" \
--build-arg "WEBDROP_BACKEND_COMMIT=$commit" --build-arg "WEBDROP_BACKEND_TAG=$version" \
--build-arg "WEBDROP_BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--tag "$image" --file "$containerfile" .
buildah push "$image" "docker://$image"
for extra in "$@"; do
buildah tag "$image" "$ZOT_REGISTRY/justme/$name:$extra"
buildah push "$ZOT_REGISTRY/justme/$name:$extra" "docker://$ZOT_REGISTRY/justme/$name:$extra"
done
+2
View File
@@ -22,8 +22,10 @@ WORKDIR /src
COPY server/Directory.Build.props ./server/ COPY server/Directory.Build.props ./server/
COPY server/src/WebDrop.Api/WebDrop.Api.csproj ./server/src/WebDrop.Api/ COPY server/src/WebDrop.Api/WebDrop.Api.csproj ./server/src/WebDrop.Api/
COPY server/src/WebDrop.Core/WebDrop.Core.csproj ./server/src/WebDrop.Core/ COPY server/src/WebDrop.Core/WebDrop.Core.csproj ./server/src/WebDrop.Core/
COPY server/src/WebDrop.FileProtocol/WebDrop.FileProtocol.csproj ./server/src/WebDrop.FileProtocol/
COPY server/src/WebDrop.Frontend/WebDrop.Frontend.csproj ./server/src/WebDrop.Frontend/ COPY server/src/WebDrop.Frontend/WebDrop.Frontend.csproj ./server/src/WebDrop.Frontend/
COPY server/src/WebDrop.Storage/WebDrop.Storage.csproj ./server/src/WebDrop.Storage/ COPY server/src/WebDrop.Storage/WebDrop.Storage.csproj ./server/src/WebDrop.Storage/
COPY server/src/WebDrop.Sync/WebDrop.Sync.csproj ./server/src/WebDrop.Sync/
RUN dotnet restore server/src/WebDrop.Api/WebDrop.Api.csproj \ RUN dotnet restore server/src/WebDrop.Api/WebDrop.Api.csproj \
--runtime linux-x64 --runtime linux-x64
COPY server/ ./server/ COPY server/ ./server/
+37
View File
@@ -0,0 +1,37 @@
# 文件服务(fileAPI):单独部署,放在存储所在的机器(例如家里的 NAS)。只依赖 FileProtocol 和 Sync,不带前端和数据库
FROM mcr.microsoft.com/dotnet/sdk:10.0-noble-aot AS build
WORKDIR /src
COPY server/Directory.Build.props ./server/
COPY server/src/WebDrop.FileApi/WebDrop.FileApi.csproj ./server/src/WebDrop.FileApi/
COPY server/src/WebDrop.FileProtocol/WebDrop.FileProtocol.csproj ./server/src/WebDrop.FileProtocol/
COPY server/src/WebDrop.Sync/WebDrop.Sync.csproj ./server/src/WebDrop.Sync/
RUN dotnet restore server/src/WebDrop.FileApi/WebDrop.FileApi.csproj \
--runtime linux-x64
COPY server/src/WebDrop.FileApi/ ./server/src/WebDrop.FileApi/
COPY server/src/WebDrop.FileProtocol/ ./server/src/WebDrop.FileProtocol/
COPY server/src/WebDrop.Sync/ ./server/src/WebDrop.Sync/
RUN dotnet publish server/src/WebDrop.FileApi/WebDrop.FileApi.csproj \
--configuration Release \
--runtime linux-x64 \
--self-contained true \
--no-restore \
--output /out
FROM mcr.microsoft.com/dotnet/runtime-deps:10.0 AS runtime
ARG WEBDROP_BACKEND_COMMIT=unknown
ARG WEBDROP_BACKEND_TAG=unknown
ARG WEBDROP_BUILD_TIME=unknown
LABEL org.opencontainers.image.revision=$WEBDROP_BACKEND_COMMIT \
org.opencontainers.image.version=$WEBDROP_BACKEND_TAG \
org.opencontainers.image.created=$WEBDROP_BUILD_TIME
WORKDIR /app
COPY --from=build /out/webdrop-files /app/webdrop-files
RUN mkdir -p /data \
&& chown 10001:10001 /data \
&& chmod 0750 /data
ENV ASPNETCORE_URLS=http://+:5100 \
FileApi__DataDir=/data \
DOTNET_EnableDiagnostics=0
USER 10001:10001
EXPOSE 5100
ENTRYPOINT ["/app/webdrop-files"]
+12 -2
View File
@@ -1,9 +1,19 @@
<Solution> <Solution>
<Folder Name="/src/"> <!-- 主后端(hostAPI):账号、空间、频道、聊天、实时推送,以及内嵌的前端 -->
<Folder Name="/src/host/">
<Project Path="src/WebDrop.Api/WebDrop.Api.csproj" /> <Project Path="src/WebDrop.Api/WebDrop.Api.csproj" />
<Project Path="src/WebDrop.Core/WebDrop.Core.csproj" />
<Project Path="src/WebDrop.Frontend/WebDrop.Frontend.csproj" /> <Project Path="src/WebDrop.Frontend/WebDrop.Frontend.csproj" />
</Folder>
<!-- 文件服务(fileAPI):另一个进程,只保存超过阈值的大文件 -->
<Folder Name="/src/files/">
<Project Path="src/WebDrop.FileApi/WebDrop.FileApi.csproj" />
<Project Path="src/WebDrop.FileProtocol/WebDrop.FileProtocol.csproj" />
</Folder>
<!-- 两边共用:业务规则、数据库、S3 备份 -->
<Folder Name="/src/shared/">
<Project Path="src/WebDrop.Core/WebDrop.Core.csproj" />
<Project Path="src/WebDrop.Storage/WebDrop.Storage.csproj" /> <Project Path="src/WebDrop.Storage/WebDrop.Storage.csproj" />
<Project Path="src/WebDrop.Sync/WebDrop.Sync.csproj" />
</Folder> </Folder>
<Folder Name="/tests/"> <Folder Name="/tests/">
<Project Path="tests/WebDrop.Api.Tests/WebDrop.Api.Tests.csproj" /> <Project Path="tests/WebDrop.Api.Tests/WebDrop.Api.Tests.csproj" />
@@ -1,62 +0,0 @@
using WebDrop.Core;
using WebDrop.Storage;
namespace WebDrop.Api;
public static partial class Endpoints
{
private static void MapWorkspace(RouteGroupBuilder api, SqliteAccountStore accounts)
{
WorkspaceBody View(WorkspaceState state) => new()
{
Structure = state,
People = state.Locations.Select(l => l.AuthorAccount).Where(u => u is not null).Distinct()
.Select(u => accounts.Profile(u!)).ToList(),
};
api.MapGet("/spaces/{id}/structure", (string id, HttpContext ctx, ChannelService svc) =>
Json(View(svc.GetWorkspace(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false))), ApiJson.Default.WorkspaceBody));
api.MapPost("/spaces/{id}/channels", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceChannelRequest, ct);
var created = svc.CreateWorkspaceChannelResult(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.Name, req.GroupId);
return Json(new WorkspaceChannelCreatedBody { Workspace = View(created.State), Channel = created.Channel }, ApiJson.Default.WorkspaceChannelCreatedBody, 201);
});
api.MapPatch("/spaces/{id}/channels/{channelId}", async (string id, string channelId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
return Json(View(svc.RenameWorkspaceChannel(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), channelId, req.Name)), ApiJson.Default.WorkspaceBody);
});
api.MapPut("/spaces/{id}/channels/{channelId}/group", async (string id, string channelId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceGroupRequest, ct);
return Json(View(svc.GroupWorkspaceChannel(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), channelId, req.GroupId)), ApiJson.Default.WorkspaceBody);
});
api.MapPost("/spaces/{id}/groups", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
var created = svc.CreateWorkspaceGroupResult(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.Name);
return Json(new WorkspaceGroupCreatedBody { Workspace = View(created.State), Group = created.Group }, ApiJson.Default.WorkspaceGroupCreatedBody, 201);
});
api.MapPatch("/spaces/{id}/groups/{groupId}", async (string id, string groupId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
return Json(View(svc.RenameWorkspaceGroup(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), groupId, req.Name)), ApiJson.Default.WorkspaceBody);
});
api.MapPost("/spaces/{id}/folders", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceFolderRequest, ct);
return Json(View(svc.CreateWorkspaceFolder(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.ThreadId, req.ParentId, req.Name)), ApiJson.Default.WorkspaceBody, 201);
});
api.MapPatch("/spaces/{id}/folders/{folderId}", async (string id, string folderId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
return Json(View(svc.RenameWorkspaceFolder(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), folderId, req.Name)), ApiJson.Default.WorkspaceBody);
});
api.MapPut("/spaces/{id}/locations", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceMoveRequest, ct);
if (req.ItemIds is null) throw new WebDropException(ErrorCode.BadRequest);
return Json(View(svc.MoveWorkspaceItems(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.ItemIds, req.ThreadId, req.FolderId)), ApiJson.Default.WorkspaceBody);
});
}
}
@@ -1,3 +1,4 @@
using Microsoft.AspNetCore.Http.Features;
using WebDrop.Core; using WebDrop.Core;
using WebDrop.Storage; using WebDrop.Storage;
@@ -10,15 +11,25 @@ public static partial class Endpoints
api.MapGet("/accounts/profile", (HttpContext ctx) => api.MapGet("/accounts/profile", (HttpContext ctx) =>
{ {
if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired); if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired);
return Json(accounts.Profile(username), ApiJson.Default.AccountProfile); return Json(accounts.Profile(username).WithoutAvatar(), ApiJson.Default.AccountProfile);
});
// 头像单独取:默认给有损的缩略图(列表、聊天用),size=full 才是原图(预览时)。带 ETag,没变化时只回 304。需要登录账号
api.MapGet("/accounts/users/{username}/avatar", (string username, HttpContext ctx, ChannelService svc) =>
{
if (!TryAccount(ctx, accounts, out var viewer)) throw new WebDropException(ErrorCode.AccountRequired);
// 只有和对方同在一个空间(或就是自己)才能看头像,不能靠猜用户名批量抓取
if (!svc.SharesSpace(viewer, NormalizeUsername(username))) throw new WebDropException(ErrorCode.NotFound);
return AvatarResult(ctx, accounts.ProfileAvatar(NormalizeUsername(username), ctx.Request.Query["size"] == "full"));
}); });
api.MapPatch("/accounts/profile", async (HttpContext ctx, ChannelService svc, CancellationToken ct) => api.MapPatch("/accounts/profile", async (HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{ {
if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired); if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired);
var req = await ReadJson(ctx.Request, ApiJson.Default.AccountProfileRequest, ct); // 动图头像比其他请求大得多,单独放宽(全局上限 2 MB)
var profile = accounts.UpdateProfile(username, req.DisplayName, req.Avatar); if (ctx.Features.Get<IHttpMaxRequestBodySizeFeature>() is { IsReadOnly: false } size) size.MaxRequestBodySize = 8 * 1024 * 1024;
var req = await ReadJsonCompressed(ctx.Request, ApiJson.Default.AccountProfileRequest, 12 * 1024 * 1024, ct);
var profile = accounts.UpdateProfile(username, req.DisplayName, req.Avatar, req.AvatarThumb, req.KeepAvatar);
svc.AccountProfileChanged(username); svc.AccountProfileChanged(username);
return Json(profile, ApiJson.Default.AccountProfile); return Json(profile.WithoutAvatar(), ApiJson.Default.AccountProfile);
}); });
api.MapPost("/accounts", async (HttpContext ctx, CancellationToken ct) => api.MapPost("/accounts", async (HttpContext ctx, CancellationToken ct) =>
{ {
@@ -12,6 +12,44 @@ public static partial class Endpoints
if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired); if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired);
return Json(svc.AccountSpaces(username).ToList(), ApiJson.Default.ListChannelInfo); return Json(svc.AccountSpaces(username).ToList(), ApiJson.Default.ListChannelInfo);
}); });
// 空间列表的实时提示:账号所在空间有新内容就推一条 {id,last_seq}。只含序号,不含内容
api.MapGet("/accounts/spaces/events", async (HttpContext ctx, ChannelService svc, ChannelHub hub) =>
{
if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired);
var ct = ctx.RequestAborted;
ctx.Response.ContentType = "text/event-stream";
ctx.Response.Headers["X-Accel-Buffering"] = "no";
await ctx.Response.StartAsync(ct);
var sent = new Dictionary<Id16, long>();
try
{
while (!ct.IsCancellationRequested)
{
// 每次唤醒都重新核对登录:退出登录、被移除设备后,这条流立刻结束
if (!TryAccount(ctx, accounts, out var current) || current != username)
{
await ctx.Response.WriteAsync("event: error\ndata: {\"error\":\"account_required\"}\n\n", ct);
break;
}
var changed = hub.AnyAdvanced;
foreach (var id in svc.AccountSpaceIds(username))
{
var seq = hub.AdvancedSeq(id);
// 第一次只记下当前值:连接建立前的进展客户端已经从列表里拿到了
if (!sent.TryGetValue(id, out var known)) { sent[id] = seq; continue; }
if (seq <= known) continue;
sent[id] = seq;
await ctx.Response.WriteAsync($"event: space\ndata: {{\"id\":\"{id.ToBase64Url()}\",\"last_seq\":{seq}}}\n\n", ct);
}
await ctx.Response.Body.FlushAsync(ct);
var idle = Task.Delay(TimeSpan.FromSeconds(20), ct);
// 久无变化就发心跳,客户端据此判断连接还活着
if (await Task.WhenAny(changed, idle) == idle) { await ctx.Response.WriteAsync("event: ping\ndata: 1\n\n", ct); await ctx.Response.Body.FlushAsync(ct); }
}
}
catch (OperationCanceledException) { }
return Results.Empty;
});
api.MapGet("/accounts/channels", (HttpContext ctx, ChannelService svc) => api.MapGet("/accounts/channels", (HttpContext ctx, ChannelService svc) =>
{ {
if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired); if (!TryAccount(ctx, accounts, out var username)) throw new WebDropException(ErrorCode.AccountRequired);
@@ -40,7 +78,8 @@ public static partial class Endpoints
return Json(InvitationBodies(svc.UserInvitations(username)), ApiJson.Default.ListInvitationBody); return Json(InvitationBodies(svc.UserInvitations(username)), ApiJson.Default.ListInvitationBody);
}); });
api.MapGet("/channels/{id}/invitations", (string id, HttpContext ctx, ChannelService svc) => api.MapGet("/channels/{id}/invitations", (string id, HttpContext ctx, ChannelService svc) =>
Json(InvitationBodies(svc.Invitations(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false))), ApiJson.Default.ListInvitationBody)); Json(svc.InvitationsView(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false)).Select(view => new InvitationBody
{ ChannelId = view.Invitation.ChannelId.ToBase64Url(), Username = view.Invitation.Username, Payload = view.Invitation.Payload, CreatedAt = view.Invitation.CreatedAt, CanRevoke = view.CanRevoke }).ToList(), ApiJson.Default.ListInvitationBody));
api.MapGet("/channels/{id}/invitations/{username}/crypto-plan", (string id, string username, HttpContext ctx, ChannelService svc, SqliteAccountStore accountStore) => api.MapGet("/channels/{id}/invitations/{username}/crypto-plan", (string id, string username, HttpContext ctx, ChannelService svc, SqliteAccountStore accountStore) =>
{ {
username = NormalizeUsername(username); username = NormalizeUsername(username);
@@ -50,12 +89,15 @@ public static partial class Endpoints
api.MapGet("/channels/{id}/members", (string id, HttpContext ctx, ChannelService svc, ChannelHub hub) => api.MapGet("/channels/{id}/members", (string id, HttpContext ctx, ChannelService svc, ChannelHub hub) =>
{ {
var channelId = ParseId(id); var channelId = ParseId(id);
var members = svc.Members(channelId, RequestAuth.FromHeaders(ctx.Request, false)).Select(member => var members = svc.MembersView(channelId, RequestAuth.FromHeaders(ctx.Request, false)).Select(view =>
{ {
var member = view.Member;
var (online, onlineDevices) = hub.Presence(channelId, member.Username); var (online, onlineDevices) = hub.Presence(channelId, member.Username);
// 设备是账号的属性:总数取账号的登录设备,在线数取该账号在本频道有订阅的设备 // 设备是账号的属性:总数取账号的登录设备,在线数取该账号在本频道有订阅的设备
var devices = accounts.Devices(member.Username, null);
return new MemberBody { Username = member.Username, Role = member.Role, JoinedAt = member.JoinedAt, return new MemberBody { Username = member.Username, Role = member.Role, JoinedAt = member.JoinedAt,
Online = online, Devices = Math.Max(accounts.Devices(member.Username, null).Count, onlineDevices), OnlineDevices = onlineDevices }; Online = online, Devices = Math.Max(devices.Count, onlineDevices), OnlineDevices = onlineDevices,
LastSeen = devices.Count == 0 ? 0 : devices.Max(device => device.LastSeen), CanRemove = view.CanRemove };
}).ToList(); }).ToList();
return Json(members, ApiJson.Default.ListMemberBody); return Json(members, ApiJson.Default.ListMemberBody);
}); });
@@ -0,0 +1,58 @@
using WebDrop.Core;
using WebDrop.Storage;
namespace WebDrop.Api;
public static partial class Endpoints
{
/// <summary>
/// 文件服务的票据:权限由空间决定,通过后给客户端一张短期票据,客户端拿着它直接找文件服务。
/// create 是所有者/管理员创建文件域;put/get 是按频道权限上传、下载某个条目。
/// </summary>
private static void MapFiles(RouteGroupBuilder api)
{
// 一次登记整个文件:请求体 = 末帧的 26 字节头部 + 每一帧的 [负载长度 u32 大端][整帧 SHA-256 32 字节]。
// 字节早已存进文件域;逐帧登记要为每个 1 MiB 往返一次,这里一次搞定
api.MapPost("/channels/{id}/files", async (string id, HttpContext ctx, ChannelService svc, SqliteAccountStore accountStore, ServerLimits limits, CancellationToken ct) =>
{
var channelId = ParseId(id);
var creds = RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false);
long? maxDownloads = ParseMaxDownloads(ctx.Request.Headers["X-WD-Max-Downloads"].ToString());
const int EntrySize = 36;
int max = Envelope.HeaderSize + limits.MaxFramesPerItem * EntrySize;
if (ctx.Features.Get<Microsoft.AspNetCore.Http.Features.IHttpMaxRequestBodySizeFeature>() is { IsReadOnly: false } size) size.MaxRequestBodySize = max;
byte[] body = await ReadBody(ctx.Request, max, ct);
if (body.Length < Envelope.HeaderSize + EntrySize || (body.Length - Envelope.HeaderSize) % EntrySize != 0) throw new WebDropException(ErrorCode.BadRequest);
var header = body.AsMemory(0, Envelope.HeaderSize);
var entries = new List<ChannelService.FrameEntry>((body.Length - Envelope.HeaderSize) / EntrySize);
for (int at = Envelope.HeaderSize; at < body.Length; at += EntrySize)
entries.Add(new ChannelService.FrameEntry((int)System.Buffers.Binary.BinaryPrimitives.ReadUInt32BigEndian(body.AsSpan(at, 4)), body.AsSpan(at + 4, 32).ToArray()));
var signedEvent = ParseSignedContentEvent(ctx.Request, accountStore, channelId, creds, header.ToArray());
var result = svc.RegisterFileItem(channelId, creds, header, entries, maxDownloads, signedEvent);
return Json(result, ApiJson.Default.AppendResult, result.Created ? StatusCodes.Status201Created : StatusCodes.Status200OK);
});
api.MapPost("/spaces/{id}/file-grants", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var spaceId = ParseId(id);
var creds = RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false);
var req = await ReadJson(ctx.Request, ApiJson.Default.FileGrantRequest, ct);
FileGrant grant;
if (req.Op == "create") grant = svc.IssueCreateGrant(spaceId, creds);
else if (req.Op == "domain") grant = svc.IssueDomainGrant(spaceId, creds, req.Channel ?? throw new WebDropException(ErrorCode.BadRequest));
else
{
if (req.Item is null || req.From is < 0 or > uint.MaxValue) throw new WebDropException(ErrorCode.BadRequest);
var item = ParseId(req.Item);
grant = req.Op switch
{
"put" => svc.IssueUploadGrant(spaceId, creds, item),
"get" => svc.IssueDownloadGrant(spaceId, creds, item, (uint)req.From),
"delete" => svc.IssueDeleteGrant(spaceId, creds, item),
_ => throw new WebDropException(ErrorCode.BadRequest),
};
}
return Json(new FileGrantBody { Url = grant.Url, Domain = grant.Domain, Ticket = grant.Ticket, ExpiresAt = grant.ExpiresAt }, ApiJson.Default.FileGrantBody);
});
}
}
@@ -0,0 +1,89 @@
using WebDrop.Core;
using WebDrop.Storage;
namespace WebDrop.Api;
public static partial class Endpoints
{
private static void MapWorkspace(RouteGroupBuilder api, SqliteAccountStore accounts)
{
// 成员资料:发过内容的作者加上空间里所有成员(没发过内容的成员也要能显示头像和名字)
WorkspaceBody View(ChannelService service, Id16 space, WorkspaceState state) => new()
{
Structure = state,
People = state.Locations.Select(l => l.AuthorAccount).Where(u => u is not null).Select(u => u!)
.Concat(service.MemberAccounts(space)).Distinct(StringComparer.OrdinalIgnoreCase)
.Select(u => accounts.Profile(u).WithoutAvatar()).ToList(),
};
api.MapGet("/spaces/{id}/structure", (string id, HttpContext ctx, ChannelService svc) =>
Json(View(svc, ParseId(id), svc.GetWorkspace(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false))), ApiJson.Default.WorkspaceBody));
api.MapPost("/spaces/{id}/channels", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceChannelRequest, ct);
var created = svc.CreateWorkspaceChannelResult(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.Name, req.GroupId, req.FileDomain);
return Json(new WorkspaceChannelCreatedBody { Workspace = View(svc, ParseId(id), created.State), Channel = created.Channel }, ApiJson.Default.WorkspaceChannelCreatedBody, 201);
});
api.MapPatch("/spaces/{id}/channels/{channelId}", async (string id, string channelId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
return Json(View(svc, ParseId(id), svc.RenameWorkspaceChannel(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), channelId, req.Name)), ApiJson.Default.WorkspaceBody);
});
api.MapPut("/spaces/{id}/channels/{channelId}/file-domain", async (string id, string channelId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceFileDomainRequest, ct);
return Json(View(svc, ParseId(id), svc.SetWorkspaceChannelFileDomain(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), channelId, req.FileDomain)), ApiJson.Default.WorkspaceBody);
});
api.MapPut("/spaces/{id}/channels/{channelId}/group", async (string id, string channelId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceGroupRequest, ct);
return Json(View(svc, ParseId(id), svc.GroupWorkspaceChannel(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), channelId, req.GroupId)), ApiJson.Default.WorkspaceBody);
});
api.MapPost("/spaces/{id}/groups", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
var created = svc.CreateWorkspaceGroupResult(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.Name);
return Json(new WorkspaceGroupCreatedBody { Workspace = View(svc, ParseId(id), created.State), Group = created.Group }, ApiJson.Default.WorkspaceGroupCreatedBody, 201);
});
api.MapPatch("/spaces/{id}/groups/{groupId}", async (string id, string groupId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
return Json(View(svc, ParseId(id), svc.RenameWorkspaceGroup(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), groupId, req.Name)), ApiJson.Default.WorkspaceBody);
});
api.MapDelete("/spaces/{id}/groups/{groupId}", (string id, string groupId, HttpContext ctx, ChannelService svc) =>
Json(View(svc, ParseId(id), svc.DeleteWorkspaceGroup(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), groupId)), ApiJson.Default.WorkspaceBody));
api.MapPost("/spaces/{id}/roles", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceRoleRequest, ct);
var created = svc.CreateWorkspaceRoleResult(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.Name, req.Prefix, req.Suffix, req.Color, req.Permissions);
return Json(new WorkspaceRoleCreatedBody { Workspace = View(svc, ParseId(id), created.State), Role = created.Role }, ApiJson.Default.WorkspaceRoleCreatedBody, 201);
});
api.MapPatch("/spaces/{id}/roles/{roleId}", async (string id, string roleId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceRoleRequest, ct);
return Json(View(svc, ParseId(id), svc.UpdateWorkspaceRole(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), roleId, req.Name, req.Prefix, req.Suffix, req.Color, req.Permissions)), ApiJson.Default.WorkspaceBody);
});
api.MapDelete("/spaces/{id}/roles/{roleId}", (string id, string roleId, HttpContext ctx, ChannelService svc) =>
Json(View(svc, ParseId(id), svc.DeleteWorkspaceRole(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), roleId)), ApiJson.Default.WorkspaceBody));
api.MapPut("/spaces/{id}/roles/{roleId}/members", async (string id, string roleId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceRoleMembersRequest, ct);
return Json(View(svc, ParseId(id), svc.SetWorkspaceRoleMembers(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), roleId, req.Members)), ApiJson.Default.WorkspaceBody);
});
api.MapPost("/spaces/{id}/folders", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceFolderRequest, ct);
return Json(View(svc, ParseId(id), svc.CreateWorkspaceFolder(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.ThreadId, req.ParentId, req.Name)), ApiJson.Default.WorkspaceBody, 201);
});
api.MapPatch("/spaces/{id}/folders/{folderId}", async (string id, string folderId, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceNameRequest, ct);
return Json(View(svc, ParseId(id), svc.RenameWorkspaceFolder(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), folderId, req.Name)), ApiJson.Default.WorkspaceBody);
});
api.MapPut("/spaces/{id}/locations", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{
var req = await ReadJson(ctx.Request, ApiJson.Default.WorkspaceMoveRequest, ct);
if (req.ItemIds is null) throw new WebDropException(ErrorCode.BadRequest);
return Json(View(svc, ParseId(id), svc.MoveWorkspaceItems(ParseId(id), RequestAuth.FromHeaders(ctx.Request, false), req.ItemIds, req.ThreadId, req.FolderId)), ApiJson.Default.WorkspaceBody);
});
}
}
@@ -1,5 +1,6 @@
using System.Buffers; using System.Buffers;
using System.Diagnostics; using System.Diagnostics;
using System.IO.Compression;
using System.Net.WebSockets; using System.Net.WebSockets;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Text.Json; using System.Text.Json;
@@ -16,7 +17,7 @@ public static partial class Endpoints
private const string OctetStream = "application/octet-stream"; private const string OctetStream = "application/octet-stream";
private static readonly TimeSpan MaxLongPoll = TimeSpan.FromSeconds(25); private static readonly TimeSpan MaxLongPoll = TimeSpan.FromSeconds(25);
public static void MapApi(this IEndpointRouteBuilder app, WebDropOptions options, ICaptchaVerifier captcha, SqliteAccountStore accounts) public static void MapApi(this IEndpointRouteBuilder app, WebDropOptions options, ICaptchaVerifier captcha, SqliteAccountStore accounts, IFileGrants files)
{ {
var api = app.MapGroup("/api/v1"); var api = app.MapGroup("/api/v1");
@@ -30,6 +31,7 @@ public static partial class Endpoints
}, },
Transports = new TransportsBody { Http3 = options.EnableHttp3 }, Transports = new TransportsBody { Http3 = options.EnableHttp3 },
Defaults = new DefaultsBody { Channel = CreationDefaults.Channel(limits), Share = CreationDefaults.Share(limits) }, Defaults = new DefaultsBody { Channel = CreationDefaults.Channel(limits), Share = CreationDefaults.Share(limits) },
Files = new FilesInfoBody { Enabled = files.Enabled, Url = files.Enabled ? files.PublicUrl : null },
Choices = new SettingChoicesBody Choices = new SettingChoicesBody
{ {
MaxMembers = new[] { 2, 3, 5, 10, limits.MaxMembers }.Where(n => n >= 1 && n <= limits.MaxMembers).Distinct().Order().ToArray(), MaxMembers = new[] { 2, 3, 5, 10, limits.MaxMembers }.Where(n => n >= 1 && n <= limits.MaxMembers).Distinct().Order().ToArray(),
@@ -43,7 +45,6 @@ public static partial class Endpoints
MaxFramesPerItem = limits.MaxFramesPerItem, MaxFramesPerItem = limits.MaxFramesPerItem,
MaxChannelBytes = limits.MaxChannelBytes, MaxChannelBytes = limits.MaxChannelBytes,
MaxWorkspaceBytes = limits.MaxWorkspaceBytes, MaxWorkspaceBytes = limits.MaxWorkspaceBytes,
MaxWorkspaceChannelBytes = limits.MaxWorkspaceChannelBytes,
MaxMembers = limits.MaxMembers, MaxMembers = limits.MaxMembers,
MaxTtlSeconds = limits.MaxTtlSeconds, MaxTtlSeconds = limits.MaxTtlSeconds,
MinTtlSeconds = limits.MinTtlSeconds, MinTtlSeconds = limits.MinTtlSeconds,
@@ -56,6 +57,7 @@ public static partial class Endpoints
MapPasskeys(api, accounts); MapPasskeys(api, accounts);
MapWorkspace(api, accounts); MapWorkspace(api, accounts);
MapWorkspaceCrypto(api); MapWorkspaceCrypto(api);
MapFiles(api);
// ───────── 频道 ───────── // ───────── 频道 ─────────
@@ -73,11 +75,20 @@ public static partial class Endpoints
api.MapPatch("/channels/{id}/settings", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) => api.MapPatch("/channels/{id}/settings", async (string id, HttpContext ctx, ChannelService svc, CancellationToken ct) =>
{ {
var patch = await ReadJson(ctx.Request, ApiJson.Default.ChannelSettingsPatch, ct); // 空间头像(动图)比其他设置大得多,单独放宽请求体上限
if (ctx.Features.Get<Microsoft.AspNetCore.Http.Features.IHttpMaxRequestBodySizeFeature>() is { IsReadOnly: false } size) size.MaxRequestBodySize = 4 * 1024 * 1024;
var patch = await ReadJsonCompressed(ctx.Request, ApiJson.Default.ChannelSettingsPatch, 6 * 1024 * 1024, ct);
var info = svc.UpdateSettings(ParseId(id), RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false), patch); var info = svc.UpdateSettings(ParseId(id), RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false), patch);
return Json(info, ApiJson.Default.ChannelInfo); return Json(info, ApiJson.Default.ChannelInfo);
}); });
// 空间头像(信息里只带版本):默认缩略图,size=full 原图。磁盘文件直接发送,带 ETag(内容哈希),没变时回 304
api.MapGet("/channels/{id}/avatar", (string id, HttpContext ctx, ChannelService svc) =>
{
bool thumb = ctx.Request.Query["size"] != "full";
return AvatarResult(ctx, svc.GetChannelAvatarBlob(ParseId(id), RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false), thumb));
});
api.MapDelete("/channels/{id}", (string id, HttpContext ctx, ChannelService svc) => api.MapDelete("/channels/{id}", (string id, HttpContext ctx, ChannelService svc) =>
{ {
svc.DeleteChannel(ParseId(id), RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false)); svc.DeleteChannel(ParseId(id), RequestAuth.FromHeaders(ctx.Request, requireEndpoint: false));
@@ -355,12 +366,12 @@ public static partial class Endpoints
bool any = values.Any(value => value.Length > 0); bool any = values.Any(value => value.Length > 0);
if (!any) if (!any)
{ {
if (Envelope.TryParse(frame, out var legacyEnvelope) && legacyEnvelope.IsFinal && creds.AccountName is not null if ((Envelope.TryParse(frame, out var legacyEnvelope) || Envelope.TryParseHeader(frame, out legacyEnvelope)) && legacyEnvelope.IsFinal && creds.AccountName is not null
&& creds.AccountDeviceId is not null && accounts.DeviceSigningKey(creds.AccountName, creds.AccountDeviceId) is not null) && creds.AccountDeviceId is not null && accounts.DeviceSigningKey(creds.AccountName, creds.AccountDeviceId) is not null)
throw new WebDropException(ErrorCode.Forbidden); throw new WebDropException(ErrorCode.Forbidden);
return null; return null;
} }
if (!Envelope.TryParse(frame, out var envelope) || !envelope.IsFinal || creds.AccountName is null || creds.AccountDeviceId is null if (!(Envelope.TryParse(frame, out var envelope) || Envelope.TryParseHeader(frame, out envelope)) || !envelope.IsFinal || creds.AccountName is null || creds.AccountDeviceId is null
|| values.Take(8).Any(string.IsNullOrEmpty)) throw new WebDropException(ErrorCode.BadRequest); || values.Take(8).Any(string.IsNullOrEmpty)) throw new WebDropException(ErrorCode.BadRequest);
if (!Guid.TryParseExact(values[2], "D", out _) || values[3] is not ("chat" or "files") if (!Guid.TryParseExact(values[2], "D", out _) || values[3] is not ("chat" or "files")
|| !int.TryParse(values[4], out var epoch) || epoch < 0 || !int.TryParse(values[4], out var epoch) || epoch < 0
@@ -452,6 +463,40 @@ public static partial class Endpoints
return long.TryParse(header, out long n) && n >= 0 ? n : throw new WebDropException(ErrorCode.BadRequest); return long.TryParse(header, out long n) && n >= 0 ? n : throw new WebDropException(ErrorCode.BadRequest);
} }
/// <summary>
/// 读 JSON 请求体,支持 Content-Encoding: gzip(头像这类带 base64 的大请求,压缩后能省下 base64 多出来的约四分之一)。
/// 解压后的大小另外限制,防止小体积的压缩炸弹。
/// </summary>
private static async Task<T> ReadJsonCompressed<T>(HttpRequest req, JsonTypeInfo<T> info, int maxBytes, CancellationToken ct) where T : class
{
try
{
Stream body = req.Body;
if (string.Equals(req.Headers.ContentEncoding.ToString(), "gzip", StringComparison.OrdinalIgnoreCase)) body = new GZipStream(req.Body, CompressionMode.Decompress);
using var plain = new MemoryStream();
var buffer = new byte[81920]; int read;
while ((read = await body.ReadAsync(buffer, ct)) > 0)
{
if (plain.Length + read > maxBytes) throw new WebDropException(ErrorCode.BadRequest);
plain.Write(buffer, 0, read);
}
plain.Position = 0;
return await JsonSerializer.DeserializeAsync(plain, info, ct) ?? throw new WebDropException(ErrorCode.BadRequest);
}
catch (Exception error) when (error is JsonException or InvalidDataException) { throw new WebDropException(ErrorCode.BadRequest); }
}
/// <summary>头像文件(带 ETag = 内容哈希,没变时 304)。直接按路径发送,不读进内存</summary>
private static IResult AvatarResult(HttpContext ctx, AvatarBlob? blob)
{
if (blob is null) return Results.NotFound();
var etag = $"\"{blob.Hash}\"";
ctx.Response.Headers.ETag = etag;
ctx.Response.Headers.CacheControl = "private, no-cache";
if (ctx.Request.Headers.IfNoneMatch == etag) return Results.StatusCode(StatusCodes.Status304NotModified);
return Results.File(blob.Path, blob.Mime);
}
private static async Task<T> ReadJson<T>(HttpRequest req, JsonTypeInfo<T> info, CancellationToken ct) where T : class private static async Task<T> ReadJson<T>(HttpRequest req, JsonTypeInfo<T> info, CancellationToken ct) where T : class
{ {
try try
@@ -469,7 +514,21 @@ public static partial class Endpoints
{ {
if (req.ContentLength is { } declared && declared > max) throw new WebDropException(ErrorCode.TooLarge); if (req.ContentLength is { } declared && declared > max) throw new WebDropException(ErrorCode.TooLarge);
using var ms = new MemoryStream(req.ContentLength is { } len ? (int)len : 1024); // 长度已知:直接读进刚好大小的数组,少一份拷贝(帧最大几 MB,多一份就是一次大对象分配)
if (req.ContentLength is { } known)
{
var exact = new byte[(int)known];
int filled = 0;
while (filled < exact.Length)
{
int got = await req.Body.ReadAsync(exact.AsMemory(filled), ct);
if (got == 0) throw new WebDropException(ErrorCode.BadRequest);
filled += got;
}
return exact;
}
using var ms = new MemoryStream(1024);
byte[] buffer = ArrayPool<byte>.Shared.Rent(16 * 1024); byte[] buffer = ArrayPool<byte>.Shared.Rent(16 * 1024);
try try
{ {
@@ -0,0 +1,24 @@
using System.Text;
using WebDrop.Core;
using WebDrop.FileProtocol;
namespace WebDrop.Api;
/// <summary>
/// 主后端这一侧和文件服务的全部关系:拿共用的密钥给客户端签短期票据。
/// 它不调用文件服务、不保存文件服务的任何状态;文件域怎么创建、多大、用了多少,都是客户端直接和文件服务之间的事。
/// </summary>
public sealed class FileGrants(FileApiOptions options) : IFileGrants
{
private readonly byte[] _secret = Encoding.UTF8.GetBytes(options.Secret);
private readonly string _publicUrl = options.PublicUrl.TrimEnd('/');
public bool Enabled => true;
public string PublicUrl => _publicUrl;
public FileGrant Issue(FileOp op, string space, string scope, string item, uint from, long nowMs)
{
long expires = nowMs + Math.Max(30, options.TicketSeconds) * 1000L;
return new FileGrant(_publicUrl, op == FileOp.Create ? "" : scope, FileTokens.Sign(_secret, new FileTicket(op, scope, item, from, expires, space)), expires);
}
}
@@ -6,5 +6,5 @@ public static class ApiContract
// 去掉设备名与设备清单接口。设备只是账号的属性。 // 去掉设备名与设备清单接口。设备只是账号的属性。
// v8: 频道强制登录,去掉未登录访客(allow_guests、/guests、终端席位);便捷分享不受影响。 // v8: 频道强制登录,去掉未登录访客(allow_guests、/guests、终端席位);便捷分享不受影响。
// Keep this aligned with contracts/api-contract.json and docs/openapi.yaml. // Keep this aligned with contracts/api-contract.json and docs/openapi.yaml.
public const string Version = "8.0.0"; public const string Version = "8.2.0";
} }
@@ -0,0 +1,79 @@
using System.IO.Compression;
using System.Globalization;
using WebDrop.Storage;
using WebDrop.Sync;
namespace WebDrop.Api;
/// <summary>
/// 每天把数据库快照(聊天、头像、账号)压缩后上传到外部存储,只在本地留一个临时文件。
/// 外部存储里按日期留 RetainDays 份:用户删掉的聊天,最多再在备份里留这么久。
/// </summary>
public sealed class BackupWorker(WebDropOptions options, SqliteChannelStore store, ILogger<BackupWorker> logger) : BackgroundService
{
private readonly HttpClient _http = new() { Timeout = TimeSpan.FromMinutes(30) };
private const string Prefix = "webdrop-";
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var backup = options.Backup;
if (!backup.Enabled || !backup.S3.Configured) return;
while (!stoppingToken.IsCancellationRequested)
{
try { await Task.Delay(DailySchedule.Next(DateTime.Now, backup.TimeOfDay) - DateTime.Now, stoppingToken).ConfigureAwait(false); } catch (OperationCanceledException) { break; }
try { await RunOnceAsync(DateTime.Now, stoppingToken).ConfigureAwait(false); }
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested) { break; }
catch (Exception e) { logger.LogWarning("数据库备份失败:{Type}", e.GetType().Name); }
}
}
public async Task<(string Name, int Purged)> RunOnceAsync(DateTime now, CancellationToken ct)
{
var backup = options.Backup;
var s3 = new S3Client(_http, backup.S3);
string tmp = Path.Combine(options.DataDir, "v2", "backup-tmp");
Directory.CreateDirectory(tmp);
string name = $"{Prefix}{now:yyyyMMdd-HHmm}.db.gz";
string db = Path.Combine(tmp, $"snapshot-{Guid.NewGuid():N}.db"), gz = db + ".gz";
try
{
store.Snapshot(db);
await using (var source = File.OpenRead(db))
await using (var target = File.Create(gz))
await using (var zip = new GZipStream(target, CompressionLevel.SmallestSize))
await source.CopyToAsync(zip, ct).ConfigureAwait(false);
await using (var upload = File.OpenRead(gz))
await s3.PutAsync($"chat/{name}", upload, ct).ConfigureAwait(false);
}
finally
{
foreach (var path in new[] { db, gz }) try { File.Delete(path); } catch (IOException) { }
}
// 头像是按内容哈希命名的磁盘文件(不在数据库里):内容不会变,已经上传过的跳过
foreach (var kind in new[] { "space", "account" })
{
string directory = Path.Combine(options.DataDir, "v2", "avatars", kind);
if (!Directory.Exists(directory)) continue;
var uploaded = (await s3.ListAsync($"chat/avatars/{kind}", ct).ConfigureAwait(false)).ToHashSet();
foreach (var file in Directory.EnumerateFiles(directory))
{
if (file.EndsWith(".tmp", StringComparison.Ordinal) || uploaded.Contains(Path.GetFileName(file))) continue;
await using var content = File.OpenRead(file);
await s3.PutAsync($"chat/avatars/{kind}/{Path.GetFileName(file)}", content, ct).ConfigureAwait(false);
}
}
// 保留期:按文件名里的日期清理更早的
int purged = 0;
var cutoff = now.Date.AddDays(-Math.Max(0, backup.RetainDays));
foreach (var existing in await s3.ListAsync("chat", ct).ConfigureAwait(false))
{
if (!existing.StartsWith(Prefix, StringComparison.Ordinal) || existing.Length < Prefix.Length + 8) continue;
if (!DateTime.TryParseExact(existing.AsSpan(Prefix.Length, 8), "yyyyMMdd", CultureInfo.InvariantCulture, DateTimeStyles.None, out var date) || date >= cutoff) continue;
if (await s3.DeleteAsync($"chat/{existing}", ct).ConfigureAwait(false)) purged++;
}
if (purged > 0) logger.LogInformation("清理了 {Count} 份过期的数据库备份", purged);
return (name, purged);
}
}
@@ -50,7 +50,14 @@ public static class WebDropApp
builder.Services.AddSingleton<ICaptchaVerifier>(sp => CreateCaptchaVerifier(options.Captcha, sp)); builder.Services.AddSingleton<ICaptchaVerifier>(sp => CreateCaptchaVerifier(options.Captcha, sp));
builder.Services.AddSingleton<ChannelHub>(); builder.Services.AddSingleton<ChannelHub>();
builder.Services.AddSingleton<IChannelEvents>(sp => sp.GetRequiredService<ChannelHub>()); builder.Services.AddSingleton<IChannelEvents>(sp => sp.GetRequiredService<ChannelHub>());
builder.Services.AddSingleton<ChannelService>(); options.Files.Validate();
builder.Services.AddSingleton<IFileGrants>(_ => options.Files.Enabled ? new FileGrants(options.Files) : NoFileGrants.Instance);
builder.Services.AddSingleton<BackupWorker>();
builder.Services.AddHostedService(sp => sp.GetRequiredService<BackupWorker>());
builder.Services.AddSingleton<ChannelService>(sp => new ChannelService(
sp.GetRequiredService<IResourceStore>(),
sp.GetRequiredService<IBlobStore>(),
sp.GetRequiredService<IClock>(), options.Limits, sp.GetRequiredService<IChannelEvents>(), sp.GetRequiredService<IFileGrants>()));
builder.Services.AddSingleton<ShareService>(sp => new ShareService( builder.Services.AddSingleton<ShareService>(sp => new ShareService(
sp.GetRequiredService<IResourceStore>(), sp.GetRequiredService<IBlobStore>(), sp.GetRequiredService<IClock>(), sp.GetRequiredService<IResourceStore>(), sp.GetRequiredService<IBlobStore>(), sp.GetRequiredService<IClock>(),
options.Limits, options.ShareDraftTtlSeconds)); options.Limits, options.ShareDraftTtlSeconds));
@@ -67,8 +74,8 @@ public static class WebDropApp
.WithHeaders("Authorization", "Content-Type", "Last-Event-ID", .WithHeaders("Authorization", "Content-Type", "Last-Event-ID",
"X-WD-Role", "X-WD-Endpoint", "X-WD-Cap", "X-WD-Max-Downloads", "X-WD-Account", "X-WD-Role", "X-WD-Endpoint", "X-WD-Cap", "X-WD-Max-Downloads", "X-WD-Account",
"X-WD-Event-Id", "X-WD-Event-Kind", "X-WD-Event-Channel", "X-WD-Event-Domain", "X-WD-Event-Id", "X-WD-Event-Kind", "X-WD-Event-Channel", "X-WD-Event-Domain",
"X-WD-Event-Epoch", "X-WD-Event-Frames", "X-WD-Event-Manifest", "X-WD-Event-Signature", "X-WD-Event-Target") "X-WD-Event-Epoch", "X-WD-Event-Frames", "X-WD-Event-Manifest", "X-WD-Event-Signature", "X-WD-Event-Target", "X-WD-Receipt")
.WithExposedHeaders("X-WD-Last-Seq") .WithExposedHeaders("X-WD-Last-Seq", "Retry-After")
.SetPreflightMaxAge(TimeSpan.FromHours(1)))); .SetPreflightMaxAge(TimeSpan.FromHours(1))));
} }
@@ -76,13 +83,17 @@ public static class WebDropApp
{ {
r.RejectionStatusCode = StatusCodes.Status429TooManyRequests; r.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
// 限流状态只在内存里,按 IP 分区,不落盘 // 限流状态只在内存里,按 IP 分区,不落盘
r.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(ctx => r.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(ctx => IsFrameUpload(ctx.Request)
RateLimitPartition.GetFixedWindowLimiter(ClientKey(ctx), _ => PerMinute(options.RequestsPerMinute))); ? RateLimitPartition.GetFixedWindowLimiter("frames:" + ClientKey(ctx), _ => PerMinute(options.FrameRequestsPerMinute))
: RateLimitPartition.GetFixedWindowLimiter(ClientKey(ctx), _ => PerMinute(options.RequestsPerMinute)));
r.AddPolicy(Endpoints.CreatePolicy, ctx => r.AddPolicy(Endpoints.CreatePolicy, ctx =>
RateLimitPartition.GetFixedWindowLimiter(ClientKey(ctx), _ => PerMinute(options.ChannelCreatesPerMinute))); RateLimitPartition.GetFixedWindowLimiter(ClientKey(ctx), _ => PerMinute(options.ChannelCreatesPerMinute)));
r.OnRejected = async (context, ct) => r.OnRejected = async (context, ct) =>
{ {
context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests; context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;
// 告诉客户端多久后窗口重置,弱网和上传重试据此等待,不再盲目重试
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
context.HttpContext.Response.Headers.RetryAfter = Math.Max(1, (int)Math.Ceiling(retryAfter.TotalSeconds)).ToString(System.Globalization.CultureInfo.InvariantCulture);
await context.HttpContext.Response.WriteAsJsonAsync( await context.HttpContext.Response.WriteAsJsonAsync(
new ErrorBody { Error = ErrorCode.RateLimited.ToWire() }, ApiJson.Default.ErrorBody, cancellationToken: ct); new ErrorBody { Error = ErrorCode.RateLimited.ToWire() }, ApiJson.Default.ErrorBody, cancellationToken: ct);
}; };
@@ -109,7 +120,7 @@ public static class WebDropApp
app.UseWebSockets(new WebSocketOptions { KeepAliveInterval = LogStreamer.Heartbeat }); app.UseWebSockets(new WebSocketOptions { KeepAliveInterval = LogStreamer.Heartbeat });
app.UseRateLimiter(); app.UseRateLimiter();
EmbeddedFrontend.Use(app, options.Captcha.ApiEndpoint); EmbeddedFrontend.Use(app, options.Captcha.ApiEndpoint);
app.MapApi(options, app.Services.GetRequiredService<ICaptchaVerifier>(), app.Services.GetRequiredService<SqliteAccountStore>()); app.MapApi(options, app.Services.GetRequiredService<ICaptchaVerifier>(), app.Services.GetRequiredService<SqliteAccountStore>(), app.Services.GetRequiredService<IFileGrants>());
return app; return app;
} }
@@ -134,6 +145,9 @@ public static class WebDropApp
} }
} }
private static bool IsFrameUpload(HttpRequest request) =>
HttpMethods.IsPost(request.Method) && request.Path.Value is { } path && path.EndsWith("/frames", StringComparison.Ordinal);
private static string ClientKey(HttpContext ctx) => ctx.Connection.RemoteIpAddress?.ToString() ?? "unknown"; private static string ClientKey(HttpContext ctx) => ctx.Connection.RemoteIpAddress?.ToString() ?? "unknown";
private static FixedWindowRateLimiterOptions PerMinute(int permits) => new() private static FixedWindowRateLimiterOptions PerMinute(int permits) => new()
@@ -1,4 +1,5 @@
using WebDrop.Core; using WebDrop.Core;
using WebDrop.Sync;
namespace WebDrop.Api; namespace WebDrop.Api;
@@ -21,6 +22,11 @@ public sealed class WebDropOptions
public string? PasskeyRpId { get; set; } public string? PasskeyRpId { get; set; }
public int RequestsPerMinute { get; set; } = 600; public int RequestsPerMinute { get; set; } = 600;
/// <summary>
/// 上传帧(POST .../frames)单独计数。每帧最多约 1 MiB,和普通 API 请求放在一个 600/分钟的桶里,
/// 持续上传速度就被锁在约 10 MiB/s;帧的总量另有空间配额和单条上限保护。
/// </summary>
public int FrameRequestsPerMinute { get; set; } = 6000;
public int ChannelCreatesPerMinute { get; set; } = 30; public int ChannelCreatesPerMinute { get; set; } = 30;
public int ShareDraftTtlSeconds { get; set; } = 86_400; public int ShareDraftTtlSeconds { get; set; } = 86_400;
public int SweepIntervalSeconds { get; set; } = 30; public int SweepIntervalSeconds { get; set; } = 30;
@@ -28,6 +34,38 @@ public sealed class WebDropOptions
public ServerLimits Limits { get; set; } = new(); public ServerLimits Limits { get; set; } = new();
public CaptchaOptions Captcha { get; set; } = new(); public CaptchaOptions Captcha { get; set; } = new();
/// <summary>文件服务(fileAPI)。关闭时文件帧仍存在本进程的存储里。</summary>
public FileApiOptions Files { get; set; } = new();
/// <summary>每天把数据库(聊天、头像、账号)做个快照,压缩后上传到外部存储(S3 协议),保留 RetainDays 天</summary>
public DailyBackupOptions Backup { get; set; } = new() { Time = "04:15" };
}
/// <summary>
/// 文件的内容不放主后端,由另一台机器上的文件服务(fileAPI)保存。主后端只保存聊天、头像、空间结构和文件的元数据与摘要,
/// 并按空间里的权限给客户端签发访问文件服务的短期票据;客户端直接和文件服务通信,主后端不调用它。
/// </summary>
public sealed class FileApiOptions
{
public bool Enabled { get; set; }
/// <summary>浏览器访问文件服务的地址(经反向隧道对外的那个),形如 https://files.example.com</summary>
public string PublicUrl { get; set; } = "";
/// <summary>和文件服务共用的密钥,至少 32 个字符。用来给票据签名</summary>
public string Secret { get; set; } = "";
public int TicketSeconds { get; set; } = 600;
public string Validate()
{
if (!Enabled) return "";
if (!Uri.TryCreate(PublicUrl, UriKind.Absolute, out var pub) || pub.Scheme is not ("http" or "https")) throw new InvalidOperationException("WebDrop:Files:PublicUrl 必须是 http(s) 绝对地址");
if (Secret.Length < 32) throw new InvalidOperationException("WebDrop:Files:Secret 至少 32 个字符");
return pub.GetLeftPart(UriPartial.Authority).TrimEnd('/');
}
} }
/// <summary> /// <summary>
@@ -23,6 +23,36 @@ public sealed class ServerInfoBody
[JsonPropertyName("limits")] public required LimitsBody Limits { get; init; } [JsonPropertyName("limits")] public required LimitsBody Limits { get; init; }
[JsonPropertyName("defaults")] public required DefaultsBody Defaults { get; init; } [JsonPropertyName("defaults")] public required DefaultsBody Defaults { get; init; }
[JsonPropertyName("choices")] public required SettingChoicesBody Choices { get; init; } [JsonPropertyName("choices")] public required SettingChoicesBody Choices { get; init; }
[JsonPropertyName("files")] public required FilesInfoBody Files { get; init; }
}
public sealed class FilesInfoBody
{
/// <summary>true 时,超过 threshold_bytes 的文件由客户端直接传给文件服务,主后端只登记元数据</summary>
[JsonPropertyName("enabled")] public bool Enabled { get; init; }
/// <summary>超过这个大小的文件才存到文件服务;未开启时为 0</summary>
[JsonPropertyName("url")] public string? Url { get; init; }
}
public sealed class FileGrantRequest
{
/// <summary>create:创建文件域(空间所有者/管理员);put:上传某个条目;get:下载或播放某个条目;delete:删除某个条目的字节;domain:查询某个频道文件域的容量和用量</summary>
[JsonPropertyName("op")] public required string Op { get; init; }
/// <summary>put/get/delete 时的条目 id(base64url)</summary>
[JsonPropertyName("item")] public string? Item { get; init; }
/// <summary>domain 时的频道 id:查询这个频道文件域的容量和用量</summary>
[JsonPropertyName("channel")] public string? Channel { get; init; }
/// <summary>get 时从第几帧开始(0 才计一次下载)</summary>
[JsonPropertyName("from")] public long From { get; init; }
}
/// <summary>去文件服务用的凭证:地址、文件域 id(create 时为空)和一张短期票据</summary>
public sealed class FileGrantBody
{
[JsonPropertyName("url")] public required string Url { get; init; }
[JsonPropertyName("domain")] public required string Domain { get; init; }
[JsonPropertyName("ticket")] public required string Ticket { get; init; }
[JsonPropertyName("expires_at")] public long ExpiresAt { get; init; }
} }
public sealed class DefaultsBody public sealed class DefaultsBody
@@ -60,7 +90,6 @@ public sealed class LimitsBody
[JsonPropertyName("max_frames_per_item")] public required int MaxFramesPerItem { get; init; } [JsonPropertyName("max_frames_per_item")] public required int MaxFramesPerItem { get; init; }
[JsonPropertyName("max_channel_bytes")] public required long MaxChannelBytes { get; init; } [JsonPropertyName("max_channel_bytes")] public required long MaxChannelBytes { get; init; }
[JsonPropertyName("max_workspace_bytes")] public required long MaxWorkspaceBytes { get; init; } [JsonPropertyName("max_workspace_bytes")] public required long MaxWorkspaceBytes { get; init; }
[JsonPropertyName("max_workspace_channel_bytes")] public required long MaxWorkspaceChannelBytes { get; init; }
[JsonPropertyName("max_members")] public required int MaxMembers { get; init; } [JsonPropertyName("max_members")] public required int MaxMembers { get; init; }
[JsonPropertyName("max_ttl_seconds")] public required int MaxTtlSeconds { get; init; } [JsonPropertyName("max_ttl_seconds")] public required int MaxTtlSeconds { get; init; }
[JsonPropertyName("min_ttl_seconds")] public required int MinTtlSeconds { get; init; } [JsonPropertyName("min_ttl_seconds")] public required int MinTtlSeconds { get; init; }
@@ -122,6 +151,8 @@ public sealed class InvitationBody
[JsonPropertyName("username")] public required string Username { get; init; } [JsonPropertyName("username")] public required string Username { get; init; }
[JsonPropertyName("payload")] public required string Payload { get; init; } [JsonPropertyName("payload")] public required string Payload { get; init; }
[JsonPropertyName("created_at")] public required long CreatedAt { get; init; } [JsonPropertyName("created_at")] public required long CreatedAt { get; init; }
/// <summary>调用者能不能撤销这条邀请(后端按权限判断)</summary>
[JsonPropertyName("can_revoke")] public bool CanRevoke { get; init; }
} }
public sealed class JoinRequest { [JsonPropertyName("password")] public string? Password { get; init; } } public sealed class JoinRequest { [JsonPropertyName("password")] public string? Password { get; init; } }
public sealed class KeyPackageRequest { [JsonPropertyName("payload")] public required string Payload { get; init; } } public sealed class KeyPackageRequest { [JsonPropertyName("payload")] public required string Payload { get; init; } }
@@ -138,6 +169,7 @@ public sealed class AccountDeviceBody
[JsonPropertyName("name")] public required string Name { get; init; } [JsonPropertyName("name")] public required string Name { get; init; }
[JsonPropertyName("created_at")] public required long CreatedAt { get; init; } [JsonPropertyName("created_at")] public required long CreatedAt { get; init; }
[JsonPropertyName("last_seen")] public required long LastSeen { get; init; } [JsonPropertyName("last_seen")] public required long LastSeen { get; init; }
[JsonPropertyName("current")] public required bool Current { get; init; } [JsonPropertyName("current")] public required bool Current { get; init; }
} }
public sealed class AccountChannelBody public sealed class AccountChannelBody
@@ -159,18 +191,45 @@ public sealed class MemberBody
/// <summary>该账号在本频道登记过的设备(终端)数;设备只是账号的属性,不是独立成员。</summary> /// <summary>该账号在本频道登记过的设备(终端)数;设备只是账号的属性,不是独立成员。</summary>
[JsonPropertyName("devices")] public required int Devices { get; init; } [JsonPropertyName("devices")] public required int Devices { get; init; }
[JsonPropertyName("online_devices")] public required int OnlineDevices { get; init; } [JsonPropertyName("online_devices")] public required int OnlineDevices { get; init; }
/// <summary>该账号任一设备最近一次活跃的 Unix 时间戳(登录时是秒,活跃续期后是毫秒,客户端按量级区分);没有登录过设备时为 0。</summary>
[JsonPropertyName("last_seen")] public required long LastSeen { get; init; }
/// <summary>调用者能不能移除这位成员(后端按权限判断)。</summary>
[JsonPropertyName("can_remove")] public bool CanRemove { get; init; }
} }
public sealed class AccountProfileRequest public sealed class AccountProfileRequest
{ {
[JsonPropertyName("displayName")] public required string DisplayName { get; init; } [JsonPropertyName("displayName")] public required string DisplayName { get; init; }
[JsonPropertyName("avatar")] public string? Avatar { get; init; } [JsonPropertyName("avatar")] public string? Avatar { get; init; }
/// <summary>头像的有损小图(data URL)。列表、聊天默认只分发它,原图在预览时才下载</summary>
[JsonPropertyName("avatarThumb")] public string? AvatarThumb { get; init; }
/// <summary>只改名字、不动头像:保留服务端现有的头像(不带 avatar 的请求默认表示移除头像)</summary>
[JsonPropertyName("keepAvatar")] public bool KeepAvatar { get; init; }
} }
public sealed class WorkspaceNameRequest { [JsonPropertyName("name")] public required string Name { get; init; } } public sealed class WorkspaceNameRequest { [JsonPropertyName("name")] public required string Name { get; init; } }
public sealed class WorkspaceChannelRequest public sealed class WorkspaceChannelRequest
{ {
[JsonPropertyName("name")] public required string Name { get; init; } [JsonPropertyName("name")] public required string Name { get; init; }
[JsonPropertyName("groupId")] public string? GroupId { get; init; } [JsonPropertyName("groupId")] public string? GroupId { get; init; }
/// <summary>文件服务里为这个频道创建的文件域 id(不透明字符串,主后端只存不解释)</summary>
[JsonPropertyName("fileDomain")] public string? FileDomain { get; init; }
}
public sealed class WorkspaceFileDomainRequest { [JsonPropertyName("fileDomain")] public required string FileDomain { get; init; } }
public sealed class WorkspaceRoleRequest
{
[JsonPropertyName("name")] public required string Name { get; init; }
[JsonPropertyName("prefix")] public string? Prefix { get; init; }
[JsonPropertyName("suffix")] public string? Suffix { get; init; }
[JsonPropertyName("color")] public string? Color { get; init; }
[JsonPropertyName("permissions")] public required List<string> Permissions { get; init; }
}
public sealed class WorkspaceRoleMembersRequest { [JsonPropertyName("members")] public required List<string> Members { get; init; } }
public sealed class WorkspaceRoleCreatedBody
{
[JsonPropertyName("workspace")] public required WorkspaceBody Workspace { get; init; }
[JsonPropertyName("role")] public required WorkspaceRole Role { get; init; }
} }
public sealed class WorkspaceGroupRequest { [JsonPropertyName("groupId")] public string? GroupId { get; init; } } public sealed class WorkspaceGroupRequest { [JsonPropertyName("groupId")] public string? GroupId { get; init; } }
public sealed class WorkspaceFolderRequest public sealed class WorkspaceFolderRequest
@@ -221,11 +280,17 @@ public sealed class WorkspaceGroupCreatedBody
[JsonSerializable(typeof(WorkspaceGroupCreatedBody))] [JsonSerializable(typeof(WorkspaceGroupCreatedBody))]
[JsonSerializable(typeof(WorkspaceNameRequest))] [JsonSerializable(typeof(WorkspaceNameRequest))]
[JsonSerializable(typeof(WorkspaceChannelRequest))] [JsonSerializable(typeof(WorkspaceChannelRequest))]
[JsonSerializable(typeof(WorkspaceFileDomainRequest))]
[JsonSerializable(typeof(WorkspaceGroupRequest))] [JsonSerializable(typeof(WorkspaceGroupRequest))]
[JsonSerializable(typeof(WorkspaceRoleRequest))]
[JsonSerializable(typeof(WorkspaceRoleMembersRequest))]
[JsonSerializable(typeof(WorkspaceRoleCreatedBody))]
[JsonSerializable(typeof(WorkspaceFolderRequest))] [JsonSerializable(typeof(WorkspaceFolderRequest))]
[JsonSerializable(typeof(WorkspaceMoveRequest))] [JsonSerializable(typeof(WorkspaceMoveRequest))]
[JsonSerializable(typeof(GoneLogBody))] [JsonSerializable(typeof(GoneLogBody))]
[JsonSerializable(typeof(ServerInfoBody))] [JsonSerializable(typeof(ServerInfoBody))]
[JsonSerializable(typeof(FileGrantRequest))]
[JsonSerializable(typeof(FileGrantBody))]
[JsonSerializable(typeof(ChannelInfo))] [JsonSerializable(typeof(ChannelInfo))]
[JsonSerializable(typeof(List<ChannelInfo>))] [JsonSerializable(typeof(List<ChannelInfo>))]
[JsonSerializable(typeof(ShareInfo))] [JsonSerializable(typeof(ShareInfo))]
@@ -12,6 +12,10 @@ public sealed class ChannelHub : IChannelEvents
private readonly object _gate = new(); private readonly object _gate = new();
private readonly Dictionary<Id16, Signal> _signals = []; private readonly Dictionary<Id16, Signal> _signals = [];
// 账号级的「有频道前进了」信号:不管有没有人订阅这个频道都会记录,供空间列表上的未读提示用
private readonly Dictionary<Id16, long> _advanced = [];
private TaskCompletionSource _anyAdvanced = new(TaskCreationOptions.RunContinuationsAsynchronously);
// 在线状态:频道 -> 账号 -> 账号设备 -> 活动订阅数。只记账号。 // 在线状态:频道 -> 账号 -> 账号设备 -> 活动订阅数。只记账号。
private readonly Dictionary<Id16, Dictionary<string, Dictionary<string, int>>> _presence = []; private readonly Dictionary<Id16, Dictionary<string, Dictionary<string, int>>> _presence = [];
@@ -66,7 +70,24 @@ public sealed class ChannelHub : IChannelEvents
public void Dispose() => Interlocked.Exchange(ref _release, null)?.Invoke(); public void Dispose() => Interlocked.Exchange(ref _release, null)?.Invoke();
} }
public void Advanced(Id16 channelId, long seq) => Publish(channelId, s => s.Seq = Math.Max(s.Seq, seq)); public void Advanced(Id16 channelId, long seq)
{
TaskCompletionSource old;
lock (_gate)
{
_advanced[channelId] = Math.Max(_advanced.GetValueOrDefault(channelId), seq);
old = _anyAdvanced;
_anyAdvanced = new(TaskCreationOptions.RunContinuationsAsynchronously);
}
old.TrySetResult();
Publish(channelId, s => s.Seq = Math.Max(s.Seq, seq));
}
/// <summary>先取这个任务、再读 <see cref="AdvancedSeq"/>,最后等待任务:不会漏掉中间的前进。</summary>
public Task AnyAdvanced { get { lock (_gate) return _anyAdvanced.Task; } }
/// <summary>本进程启动以来该频道见到的最大序号;没有见到过为 0。</summary>
public long AdvancedSeq(Id16 channelId) { lock (_gate) return _advanced.GetValueOrDefault(channelId); }
public void SettingsChanged(Id16 channelId) => Publish(channelId, s => s.SettingsVersion++); public void SettingsChanged(Id16 channelId) => Publish(channelId, s => s.SettingsVersion++);
public void MembersChanged(Id16 channelId) => Publish(channelId, s => s.MembersVersion++); public void MembersChanged(Id16 channelId) => Publish(channelId, s => s.MembersVersion++);
public void ChannelDeleted(Id16 channelId) => Publish(channelId, s => s.Deleted = true); public void ChannelDeleted(Id16 channelId) => Publish(channelId, s => s.Deleted = true);
@@ -13,6 +13,7 @@
<ProjectReference Include="..\WebDrop.Core\WebDrop.Core.csproj" /> <ProjectReference Include="..\WebDrop.Core\WebDrop.Core.csproj" />
<ProjectReference Include="..\WebDrop.Frontend\WebDrop.Frontend.csproj" /> <ProjectReference Include="..\WebDrop.Frontend\WebDrop.Frontend.csproj" />
<ProjectReference Include="..\WebDrop.Storage\WebDrop.Storage.csproj" /> <ProjectReference Include="..\WebDrop.Storage\WebDrop.Storage.csproj" />
<ProjectReference Include="..\WebDrop.Sync\WebDrop.Sync.csproj" />
</ItemGroup> </ItemGroup>
</Project> </Project>
@@ -118,6 +118,12 @@ public interface IResourceTx
void SetCompactedThrough(Id16 id, long seq); void SetCompactedThrough(Id16 id, long seq);
long NextSeq(Id16 id); long NextSeq(Id16 id);
void DeleteChannelRows(Id16 id); void DeleteChannelRows(Id16 id);
/// <summary>空间头像(data URL)单独存放,不放进每次请求都要读的设置 JSON</summary>
/// <summary>thumb 为 true 取缩略图(没有就用原图),否则取原图</summary>
AvatarBlob? GetChannelAvatar(Id16 id, bool thumb);
/// <summary>头像的版本标记(头像文件名,内容变了才会变);没有头像为 null</summary>
string? GetChannelAvatarVersion(Id16 id);
void SetChannelAvatar(Id16 id, string? avatar, string? thumb);
IReadOnlyList<Id16> ListExpiredChannels(long now, int limit); IReadOnlyList<Id16> ListExpiredChannels(long now, int limit);
// 成员 // 成员
-11
View File
@@ -1,11 +0,0 @@
using System.Text.Json.Serialization;
namespace WebDrop.Core;
public sealed class AccountProfile
{
[JsonPropertyName("username")] public required string Username { get; init; }
[JsonPropertyName("displayName")] public required string DisplayName { get; init; }
[JsonPropertyName("avatar")] public string? Avatar { get; init; }
[JsonPropertyName("modifiedAt")] public required long ModifiedAt { get; init; }
}
@@ -0,0 +1,18 @@
using System.Text.Json.Serialization;
namespace WebDrop.Core;
public sealed class AccountProfile
{
[JsonPropertyName("username")] public required string Username { get; init; }
[JsonPropertyName("displayName")] public required string DisplayName { get; init; }
/// <summary>头像文件名(服务端存的是磁盘文件)。资料列表、空间成员等接口不再内联头像(动图可能有几 MB),只告诉有没有,头像另用 /accounts/users/{用户名}/avatar 取</summary>
[JsonPropertyName("avatar")] public string? Avatar { get; init; }
[JsonPropertyName("hasAvatar")] public bool HasAvatar { get; init; }
[JsonPropertyName("modifiedAt")] public required long ModifiedAt { get; init; }
/// <summary>有损的小图(列表、聊天里默认分发);没有单独上传时用原图。不随 JSON 输出</summary>
[JsonIgnore] public string? AvatarThumb { get; init; }
public AccountProfile WithoutAvatar() => new() { Username = Username, DisplayName = DisplayName, HasAvatar = HasAvatar, ModifiedAt = ModifiedAt };
}
@@ -0,0 +1,35 @@
namespace WebDrop.Core;
/// <summary>头像(账号头像、空间头像)的 data URL 校验。浏览器里已经裁剪好并重新编码,服务端只做格式、大小和文件头检查</summary>
public static class AvatarData
{
/// <summary>静态头像(256×256 WebP/PNG/JPEG)的长度上限(字符数)</summary>
public const int MaxStaticChars = 350_000;
/// <summary>账号动图头像保留全部帧,约 4.5 MB</summary>
public const int MaxAccountGifChars = 6_000_000;
/// <summary>空间头像会跟着空间信息发给所有成员和空间列表,动图上限比账号头像小,约 1.5 MB</summary>
public const int MaxSpaceGifChars = 2_000_000;
/// <summary>缩略图(列表、聊天里默认分发的有损小图)的上限:静态约 75 KB,动图约 370 KB</summary>
public const int MaxThumbStaticChars = 100_000;
public const int MaxThumbGifChars = 800_000;
private static readonly string[] Prefixes = ["data:image/png;base64,", "data:image/jpeg;base64,", "data:image/webp;base64,", "data:image/gif;base64,"];
public static void ValidateThumb(string thumb) => Validate(thumb, MaxThumbGifChars, MaxThumbStaticChars);
/// <summary>不合法时抛 BadRequest</summary>
public static void Validate(string avatar, int maxGifChars, int maxStaticChars = MaxStaticChars)
{
var prefix = Prefixes.FirstOrDefault(p => avatar.StartsWith(p, StringComparison.Ordinal));
var gif = prefix?.Contains("gif", StringComparison.Ordinal) == true;
if (prefix is null || avatar.Length > (gif ? maxGifChars : maxStaticChars) || avatar.Length <= prefix.Length) throw new WebDropException(ErrorCode.BadRequest);
byte[] bytes;
try { bytes = Convert.FromBase64String(avatar[prefix.Length..]); }
catch (FormatException) { throw new WebDropException(ErrorCode.BadRequest); }
if (gif && !(bytes.Length > 6 && bytes[0] == (byte)'G' && bytes[1] == (byte)'I' && bytes[2] == (byte)'F')) throw new WebDropException(ErrorCode.BadRequest);
}
}
/// <summary>落在磁盘上的头像文件:下载时直接按路径发送,不读进内存。Hash 是内容的 SHA-256,也用作 ETag</summary>
public sealed record AvatarBlob(string Path, string Mime, string Hash);
-26
View File
@@ -1,26 +0,0 @@
namespace WebDrop.Core;
/// <summary>谁能做什么。规则只写在这里,服务和测试都调用它。</summary>
public static class ChannelPolicy
{
public static bool IsOwner(Principal p) => p.Role == Role.Owner;
public static bool CanRead(Principal p) => p.Role >= Role.Read;
/// <summary>成员写入需要频道允许上传,且服务端持有写令牌哈希。</summary>
public static bool CanWrite(Channel channel, Principal p) =>
p.Role == Role.Owner
|| (p.Role == Role.Write && channel.Settings.MembersCanUpload && channel.WriteHash is not null);
public static bool CanDeleteItem(Channel channel, Item item, Principal p)
{
if (p.Role == Role.Owner) return true;
if (p.Role < Role.Read) return false;
return channel.Settings.DeletePolicy switch
{
DeletePolicy.Anyone => true,
DeletePolicy.Own => item.AuthorAccount == p.AccountName,
_ => false,
};
}
}
-143
View File
@@ -1,143 +0,0 @@
namespace WebDrop.Core;
public sealed partial class ChannelService
{
public void AccountProfileChanged(string username)
{
foreach (var id in store.Read(tx => tx.ListAccountSpaces(username, clock.NowMs))) events.SettingsChanged(id);
}
public IReadOnlyList<ChannelInfo> AccountSpaces(string username) => store.Read(tx => tx.ListAccountSpaces(username, clock.NowMs).Select(id =>
{
var channel = tx.GetChannel(id)!;
var role = tx.GetChannelMember(id, username)?.Role == "owner" ? Role.Owner : channel.WriteHash is not null ? Role.Write : Role.Read;
return ToInfo(channel, tx.CountMembers(id), new Principal(role, username));
}).ToList());
private static WorkspaceState NewWorkspace(string username, long now)
{
var id = Guid.NewGuid().ToString();
return new() { Revision = 1, DefaultChannelId = id, Channels = [new WorkspaceChannel { Id = id, Name = "常规", CreatedAt = now, CreatedBy = username }] };
}
public WorkspaceState GetWorkspace(Id16 id, Credentials creds) => store.Read(tx =>
{
var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
var state = tx.GetWorkspace(id) ?? new WorkspaceState();
if (principal.AccountName != space.CreatorAccount)
{
var crypto = tx.GetWorkspaceCrypto(id);
var grants = crypto.Access.Where(a => a.Username == principal.AccountName).ToList();
var ids = grants.Where(a => a.ChatRead || a.FilesRead).Select(a => a.ChannelId).ToHashSet();
state.Channels.RemoveAll(c => !ids.Contains(c.Id));
foreach (var workspaceChannel in state.Channels)
{
var grant = grants.Find(a => a.ChannelId == workspaceChannel.Id);
if (grant is null || !grant.FilesRead && !grant.FilesUpload) workspaceChannel.FileBytes = 0;
}
state.Folders.RemoveAll(f => !ids.Contains(f.ThreadId));
state.Locations.RemoveAll(l => !Id16.TryParse(l.ItemId, out var itemId) || !CanReadObject(tx, space, principal, itemId));
}
state.Locations = state.Locations.Where(l => Id16.TryParse(l.ItemId, out var itemId) && tx.GetItem(id, itemId) is { } item && item.ExpiresAt > clock.NowMs).ToList();
return state;
});
private WorkspaceState ChangeWorkspace(Id16 id, Credentials creds, Action<IResourceTx, WorkspaceState, Principal> change, bool requireAccount = true)
{
var state = store.Write(tx =>
{
var (channel, principal) = Authenticate(tx, id, creds, Membership.Required);
if (requireAccount && principal.Role != Role.Owner) throw new WebDropException(ErrorCode.Forbidden);
if (!ChannelPolicy.CanWrite(channel, principal)) throw new WebDropException(ErrorCode.Forbidden);
var current = tx.GetWorkspace(id) ?? new WorkspaceState();
current.Locations = current.Locations.Where(l => Id16.TryParse(l.ItemId, out var itemId) && tx.GetItem(id, itemId) is not null).ToList();
change(tx, current, principal);
ValidateWorkspace(current);
current.Revision++;
tx.PutWorkspace(id, current);
return current;
});
events.SettingsChanged(id);
return state;
}
public (WorkspaceState State, WorkspaceChannel Channel) CreateWorkspaceChannelResult(Id16 id, Credentials creds, string name, string? groupId)
{
WorkspaceChannel? created = null;
var state = ChangeWorkspace(id, creds, (_, workspace, principal) =>
{
created = new WorkspaceChannel { Id = Guid.NewGuid().ToString(), Name = WorkspaceName(name), GroupId = groupId, CreatedAt = clock.NowMs, CreatedBy = principal.AccountName! };
workspace.Channels.Add(created);
});
return (state, created!);
}
public WorkspaceState CreateWorkspaceChannel(Id16 id, Credentials creds, string name, string? groupId) => CreateWorkspaceChannelResult(id, creds, name, groupId).State;
public WorkspaceState RenameWorkspaceChannel(Id16 id, Credentials creds, string channelId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
FindChannel(state, channelId).Name = WorkspaceName(name));
public WorkspaceState GroupWorkspaceChannel(Id16 id, Credentials creds, string channelId, string? groupId) => ChangeWorkspace(id, creds, (_, state, _) =>
FindChannel(state, channelId).GroupId = groupId);
public (WorkspaceState State, WorkspaceGroup Group) CreateWorkspaceGroupResult(Id16 id, Credentials creds, string name)
{
WorkspaceGroup? created = null;
var state = ChangeWorkspace(id, creds, (_, workspace, _) =>
{
created = new WorkspaceGroup { Id = Guid.NewGuid().ToString(), Name = WorkspaceName(name), CreatedAt = clock.NowMs };
workspace.Groups.Add(created);
});
return (state, created!);
}
public WorkspaceState CreateWorkspaceGroup(Id16 id, Credentials creds, string name) => CreateWorkspaceGroupResult(id, creds, name).State;
public WorkspaceState RenameWorkspaceGroup(Id16 id, Credentials creds, string groupId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
(state.Groups.Find(g => g.Id == groupId) ?? throw new WebDropException(ErrorCode.NotFound)).Name = WorkspaceName(name));
public WorkspaceState CreateWorkspaceFolder(Id16 id, Credentials creds, string threadId, string? parentId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
state.Folders.Add(new WorkspaceFolder { Id = Guid.NewGuid().ToString(), ThreadId = threadId, ParentId = parentId, Name = WorkspaceName(name) }));
public WorkspaceState RenameWorkspaceFolder(Id16 id, Credentials creds, string folderId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
(state.Folders.Find(f => f.Id == folderId) ?? throw new WebDropException(ErrorCode.NotFound)).Name = WorkspaceName(name));
public WorkspaceState MoveWorkspaceItems(Id16 id, Credentials creds, string[] itemIds, string threadId, string? folderId) => ChangeWorkspace(id, creds, (tx, state, principal) =>
{
if (itemIds.Length is < 1 or > 100 || itemIds.Distinct().Count() != itemIds.Length) throw new WebDropException(ErrorCode.BadRequest);
foreach (var itemId in itemIds)
{
if (!Id16.TryParse(itemId, out var parsed)) throw new WebDropException(ErrorCode.BadRequest);
var item = tx.GetItem(id, parsed) ?? throw new WebDropException(ErrorCode.NotFound);
if ((item.Flags & (FrameFlags.Control | FrameFlags.Snapshot)) != 0) throw new WebDropException(ErrorCode.BadRequest);
if (principal.Role != Role.Owner && item.AuthorAccount != principal.AccountName) throw new WebDropException(ErrorCode.Forbidden);
state.Locations.RemoveAll(l => l.ItemId == itemId);
state.Locations.Add(new WorkspaceLocation { ItemId = itemId, ThreadId = threadId, FolderId = folderId, AuthorAccount = item.AuthorAccount });
}
}, requireAccount: false);
private static WorkspaceChannel FindChannel(WorkspaceState state, string id) => state.Channels.Find(c => c.Id == id) ?? throw new WebDropException(ErrorCode.NotFound);
private static string WorkspaceName(string name)
{
if (string.IsNullOrWhiteSpace(name) || name.Trim().Length > 80 || name.Any(char.IsControl)) throw new WebDropException(ErrorCode.BadRequest);
return name.Trim();
}
private static void ValidateWorkspace(WorkspaceState state)
{
if (state.Channels is null || state.Groups is null || state.Folders is null || state.Locations is null
|| state.Channels.Count > 256 || state.Groups.Count > 128 || state.Folders.Count > 2048 || state.Locations.Count > 10_000)
throw new WebDropException(ErrorCode.BadRequest);
if (state.Channels.Any(c => c is null) || state.Groups.Any(g => g is null) || state.Folders.Any(f => f is null) || state.Locations.Any(l => l is null)) throw new WebDropException(ErrorCode.BadRequest);
var allIds = state.Channels.Select(c => c.Id).Concat(state.Groups.Select(g => g.Id)).Concat(state.Folders.Select(f => f.Id)).ToArray();
if (allIds.Any(id => !Guid.TryParseExact(id, "D", out _)) || allIds.Distinct().Count() != allIds.Length) throw new WebDropException(ErrorCode.BadRequest);
foreach (var group in state.Groups) group.Name = WorkspaceName(group.Name);
foreach (var channel in state.Channels)
{
channel.Name = WorkspaceName(channel.Name);
if (channel.GroupId is not null && !state.Groups.Any(g => g.Id == channel.GroupId)) throw new WebDropException(ErrorCode.BadRequest);
}
foreach (var folder in state.Folders)
{
folder.Name = WorkspaceName(folder.Name);
_ = FindChannel(state, folder.ThreadId);
var visited = new HashSet<string> { folder.Id };
for (var parentId = folder.ParentId; parentId is not null;)
{
var parent = state.Folders.Find(f => f.Id == parentId && f.ThreadId == folder.ThreadId);
if (parent is null || !visited.Add(parent.Id)) throw new WebDropException(ErrorCode.BadRequest);
parentId = parent.ParentId;
}
}
if (state.Locations.Select(l => l.ItemId).Distinct().Count() != state.Locations.Count) throw new WebDropException(ErrorCode.BadRequest);
foreach (var location in state.Locations)
{
_ = FindChannel(state, location.ThreadId);
if (location.FolderId is not null && !state.Folders.Any(f => f.Id == location.FolderId && f.ThreadId == location.ThreadId)) throw new WebDropException(ErrorCode.BadRequest);
}
}
}
@@ -30,10 +30,36 @@ public sealed partial class ChannelService
public IReadOnlyList<ChannelInvitation> Invitations(Id16 id, Credentials creds) => store.Read(tx => public IReadOnlyList<ChannelInvitation> Invitations(Id16 id, Credentials creds) => store.Read(tx =>
{ {
var (_, principal) = Authenticate(tx, id, creds, Membership.None); var (_, principal) = Authenticate(tx, id, creds, Membership.None);
RequireOwner(principal); if (!ChannelPolicy.IsOwner(principal)) RequireMemberPermission(tx, id, creds, WorkspacePermission.RemoveMembers);
return tx.ListInvitations(id); return tx.ListInvitations(id);
}); });
/// <summary>成员列表,附带调用者能不能移除每一位(后端判断,前端只显示按钮)</summary>
public IReadOnlyList<MemberView> MembersView(Id16 id, Credentials creds) => store.Read(tx =>
{
var (_, principal) = Authenticate(tx, id, creds, Membership.None);
if (!ChannelPolicy.IsOwner(principal)) _ = Authenticate(tx, id, creds, Membership.Required);
var state = tx.GetWorkspace(id);
return tx.ListChannelMembers(id).Select(m => new MemberView(m, ChannelPolicy.CanRemoveMember(state, principal, m.Username, m.Role == "owner"))).ToList();
});
/// <summary>邀请列表,附带调用者能不能撤销每一条</summary>
public IReadOnlyList<InvitationView> InvitationsView(Id16 id, Credentials creds) => store.Read(tx =>
{
var (_, principal) = Authenticate(tx, id, creds, Membership.None);
if (!ChannelPolicy.IsOwner(principal)) principal = RequireMemberPermission(tx, id, creds, WorkspacePermission.RemoveMembers);
var state = tx.GetWorkspace(id);
return tx.ListInvitations(id).Select(i => new InvitationView(i, ChannelPolicy.CanRemoveMember(state, principal, i.Username, false))).ToList();
});
/// <summary>非空间主的调用方:必须是已加入的成员,且所在身份组被授予该权限</summary>
private Principal RequireMemberPermission(IResourceTx tx, Id16 id, Credentials creds, string permission)
{
var (_, principal) = Authenticate(tx, id, creds, Membership.Required);
if (!ChannelPolicy.HasPermission(tx.GetWorkspace(id), principal, permission)) throw new WebDropException(ErrorCode.Forbidden);
return principal;
}
public IReadOnlyList<ChannelInvitation> UserInvitations(string username) => store.Read(tx => tx.ListUserInvitations(username)); public IReadOnlyList<ChannelInvitation> UserInvitations(string username) => store.Read(tx => tx.ListUserInvitations(username));
public void RevokeInvitation(Id16 id, Credentials creds, string username) public void RevokeInvitation(Id16 id, Credentials creds, string username)
@@ -41,8 +67,9 @@ public sealed partial class ChannelService
store.Write(tx => store.Write(tx =>
{ {
var (_, principal) = Authenticate(tx, id, creds, Membership.None); var (_, principal) = Authenticate(tx, id, creds, Membership.None);
RequireOwner(principal); bool targetIsOwner = tx.GetChannelMember(id, username)?.Role == "owner";
if (tx.GetChannelMember(id, username)?.Role == "owner") throw new WebDropException(ErrorCode.Forbidden); if (!ChannelPolicy.IsOwner(principal)) principal = RequireMemberPermission(tx, id, creds, WorkspacePermission.RemoveMembers);
if (!ChannelPolicy.CanRemoveMember(tx.GetWorkspace(id), principal, username, targetIsOwner)) throw new WebDropException(ErrorCode.Forbidden);
tx.DeleteInvitation(id, username); tx.DeleteInvitation(id, username);
tx.DeleteKeyPackage(id, username); tx.DeleteKeyPackage(id, username);
tx.DeleteChannelMember(id, username); tx.DeleteChannelMember(id, username);
@@ -57,10 +84,13 @@ public sealed partial class ChannelService
events.MembersChanged(id); events.MembersChanged(id);
} }
/// <summary>空间里所有成员的账号(不做鉴权,供已经通过鉴权的接口补全成员资料)</summary>
public IReadOnlyList<string> MemberAccounts(Id16 id) => store.Read(tx => tx.ListChannelMembers(id).Select(m => m.Username).ToList());
public IReadOnlyList<ChannelMember> Members(Id16 id, Credentials creds) => store.Read(tx => public IReadOnlyList<ChannelMember> Members(Id16 id, Credentials creds) => store.Read(tx =>
{ {
var (_, principal) = Authenticate(tx, id, creds, Membership.None); var (_, principal) = Authenticate(tx, id, creds, Membership.None);
if (principal.Role != Role.Owner) _ = Authenticate(tx, id, creds, Membership.Required); if (!ChannelPolicy.IsOwner(principal)) _ = Authenticate(tx, id, creds, Membership.Required);
return tx.ListChannelMembers(id); return tx.ListChannelMembers(id);
}); });
@@ -0,0 +1,103 @@
namespace WebDrop.Core;
/// <summary>谁能做什么。规则只写在这里,服务和测试都调用它。</summary>
public static class ChannelPolicy
{
public static bool IsOwner(Principal p) => p.Role == Role.Owner;
public static bool CanRead(Principal p) => p.Role >= Role.Read;
/// <summary>成员写入需要频道允许上传,且服务端持有写令牌哈希。</summary>
public static bool CanWrite(Channel channel, Principal p) =>
p.Role == Role.Owner
|| (p.Role == Role.Write && channel.Settings.MembersCanUpload && channel.WriteHash is not null);
// ───── 空间所有者与频道访问授权(ChannelAccess):谁能读、写哪个频道的哪个数据域。其他地方只调用这里,不再各写一遍
/// <summary>空间的所有者:持有所有者令牌的人,或者空间的创建账号</summary>
public static bool IsSpaceOwner(Channel space, Principal p) => p.Role == Role.Owner || p.AccountName == space.CreatorAccount;
/// <summary>某个账号在某个频道上的访问授权;没有记录返回 null</summary>
public static ChannelAccess? AccessOf(CryptoState crypto, string? account, string channelId) =>
crypto.Access.Find(a => a.ChannelId == channelId && a.Username == account);
/// <summary>这条授权允许读这个数据域</summary>
public static bool CanReadDomain(ChannelAccess? access, CryptoDomain domain) =>
access is not null && (domain == CryptoDomain.Chat ? access.ChatRead : access.FilesRead);
/// <summary>这条授权允许写这个数据域(聊天域写消息,文件域上传)</summary>
public static bool CanWriteDomain(ChannelAccess? access, CryptoDomain domain) =>
access is not null && (domain == CryptoDomain.Chat ? access.ChatWrite : access.FilesUpload);
/// <summary>读一个对象:能读它所在的数据域,并且对象不早于授权生效的版本(关闭历史可读时,新成员看不到加入前的内容)</summary>
public static bool CanReadObject(ChannelAccess? access, CryptoDomain domain, long objectRevision) =>
CanReadDomain(access, domain) && objectRevision >= (domain == CryptoDomain.Chat ? access!.ChatFromRevision : access!.FilesFromRevision);
/// <summary>频道对这个账号可见:聊天或文件任一域能读</summary>
public static bool CanSeeChannel(ChannelAccess access) => access.ChatRead || access.FilesRead;
/// <summary>能碰这个频道的文件域:能读或能上传</summary>
public static bool CanUseFiles(ChannelAccess access) => access.FilesRead || access.FilesUpload;
/// <summary>能往频道里放东西:聊天或文件任一域能写</summary>
public static bool CanWriteAnyDomain(ChannelAccess? access) => access is not null && (access.ChatWrite || access.FilesUpload);
/// <summary>这个频道里能读某个数据域的账号</summary>
public static IEnumerable<ChannelAccess> Readers(CryptoState crypto, string channelId, CryptoDomain domain) =>
crypto.Access.Where(a => a.ChannelId == channelId && CanReadDomain(a, domain));
/// <summary>
/// 这个空间是否已经按授权记录来约束成员:加密空间有了密钥信封,明文空间有了授权记录或对象登记之后才约束。
/// 之前(刚建好、还没初始化)所有成员的行为和旧版一致。
/// </summary>
public static bool GrantsEnforced(Channel space, CryptoState crypto) =>
space.Mode == ChannelMode.Enc ? crypto.Envelopes.Count > 0 : crypto.Access.Count > 0 || crypto.Objects.Count > 0;
// ───── 空间权限(身份组授予)与结构管理
/// <summary>有没有这项空间权限:所有者拥有全部,其余看身份组授予</summary>
public static bool HasPermission(WorkspaceState? state, Principal principal, string permission) =>
principal.Role == Role.Owner
|| principal.AccountName is { } account && state?.Roles.Any(r => r.Members.Contains(account) && r.Permissions.Contains(permission)) == true;
/// <summary>重命名频道、调整频道所属分组、管理分组(所有者,或被授予「管理频道和分组」)</summary>
public static bool CanManageStructure(WorkspaceState? state, Principal p) => HasPermission(state, p, WorkspacePermission.ManageStructure);
/// <summary>移除成员、撤销邀请的权限(所有者,或被授予「移除成员」)。具体能移除谁见 <see cref="CanRemoveMember"/></summary>
public static bool CanRemoveMembers(WorkspaceState? state, Principal p) => HasPermission(state, p, WorkspacePermission.RemoveMembers);
/// <summary>
/// 谁能移除谁:所有者能移除除自己外的任何人;被授权的成员不能移除自己、空间主,也不能移除同样有移除权限的人。
/// 执行(RevokeInvitation)和给前端的可移除标记(MembersView)都用这一处。
/// </summary>
public static bool CanRemoveMember(WorkspaceState? state, Principal acting, string target, bool targetIsOwner)
{
if (targetIsOwner) return false;
if (acting.Role == Role.Owner) return true;
return acting.AccountName is { } actor && actor != target
&& CanRemoveMembers(state, acting)
&& !CanRemoveMembers(state, new Principal(Role.Write, target));
}
// 下面这几项目前都只有所有者:新建频道(涉及密钥)、文件夹、邀请、改空间设置。要放开时只改这里
public static bool CanCreateChannel(Principal p) => IsOwner(p);
public static bool CanManageFolders(Principal p) => IsOwner(p);
public static bool CanInvite(Principal p) => IsOwner(p);
public static bool CanEditSettings(Principal p) => IsOwner(p);
public static bool CanManageRoles(Principal p) => IsOwner(p);
// ───── 删除条目
/// <summary>能删任何人发的条目:所有者,或空间的删除策略是 anyone</summary>
public static bool CanDeleteAnyItem(Channel channel, Principal p) =>
IsOwner(p) || p.Role >= Role.Read && channel.Settings.DeletePolicy == DeletePolicy.Anyone;
/// <summary>能删自己发的条目:所有者,或删除策略是 own / anyone</summary>
public static bool CanDeleteOwnItem(Channel channel, Principal p) =>
IsOwner(p) || p.Role >= Role.Read && channel.Settings.DeletePolicy is DeletePolicy.Own or DeletePolicy.Anyone;
/// <summary>某个具体条目能不能删:能删任何人的,或者是自己的并且能删自己的</summary>
public static bool CanDeleteItem(Channel channel, Item item, Principal p) =>
CanDeleteAnyItem(channel, p) || item.AuthorAccount == p.AccountName && CanDeleteOwnItem(channel, p);
}
@@ -1,4 +1,5 @@
using System.Security.Cryptography; using System.Security.Cryptography;
using WebDrop.FileProtocol;
namespace WebDrop.Core; namespace WebDrop.Core;
@@ -11,8 +12,18 @@ public sealed partial class ChannelService(
IBlobStore blobs, IBlobStore blobs,
IClock clock, IClock clock,
ServerLimits limits, ServerLimits limits,
IChannelEvents events) IChannelEvents events,
IFileGrants? fileGrants = null)
{ {
private IFileGrants Files => fileGrants ?? NoFileGrants.Instance;
/// <summary>帧内容在文件域里的帧,BlobKey 以它开头;主后端只有头部、长度和摘要</summary>
public const string FilesPrefix = "files:";
/// <summary>这个条目是文件域的对象(不含目录控制帧)。所在频道没有分配文件域(文件服务没开、小文件)时,文件和聊天一样存在主后端</summary>
private static bool IsFileObject(IResourceTx tx, Id16 channelId, Envelope env) =>
!env.IsControl && FileDomainOf(tx, channelId, env.ItemId) is not null;
private static bool IsRemote(string? blobKey) => blobKey is not null && blobKey.StartsWith(FilesPrefix, StringComparison.Ordinal);
/// <summary>Required:必须已是频道成员;None:只验证身份(加入、查看信息等)。</summary> /// <summary>Required:必须已是频道成员;None:只验证身份(加入、查看信息等)。</summary>
private enum Membership { None, Required } private enum Membership { None, Required }
@@ -68,7 +79,14 @@ public sealed partial class ChannelService(
public ChannelInfo GetChannel(Id16 id, Credentials creds) => store.Read(tx => public ChannelInfo GetChannel(Id16 id, Credentials creds) => store.Read(tx =>
{ {
var (channel, principal) = Authenticate(tx, id, creds, Membership.None); var (channel, principal) = Authenticate(tx, id, creds, Membership.None);
return ToInfo(channel, tx.CountMembers(id), principal); return ToInfo(channel, tx.CountMembers(id), principal, tx.GetChannelAvatarVersion(id), tx.GetWorkspace(id));
});
/// <summary>空间头像(thumb 为缩略图,否则原图)。需要能读这个空间</summary>
public AvatarBlob? GetChannelAvatarBlob(Id16 id, Credentials creds, bool thumb) => store.Read(tx =>
{
_ = Authenticate(tx, id, creds, Membership.None);
return tx.GetChannelAvatar(id, thumb);
}); });
public ChannelInfo UpdateSettings(Id16 id, Credentials creds, ChannelSettingsPatch patch) public ChannelInfo UpdateSettings(Id16 id, Credentials creds, ChannelSettingsPatch patch)
@@ -80,6 +98,9 @@ public sealed partial class ChannelService(
var updated = patch.ApplyTo(channel.Settings); var updated = patch.ApplyTo(channel.Settings);
updated.Validate(limits); updated.Validate(limits);
if (patch.Avatar is { Length: > 0 } newAvatar) AvatarData.Validate(newAvatar, AvatarData.MaxSpaceGifChars);
if (patch.AvatarThumb is { Length: > 0 } newThumb) { if (patch.Avatar is not { Length: > 0 }) throw new WebDropException(ErrorCode.BadRequest); AvatarData.ValidateThumb(newThumb); }
if (patch.Avatar is not null) tx.SetChannelAvatar(id, patch.Avatar.Length == 0 ? null : patch.Avatar, patch.Avatar.Length == 0 ? null : patch.AvatarThumb is { Length: > 0 } ? patch.AvatarThumb : null);
long expiresAt = updated.TtlSeconds != channel.Settings.TtlSeconds long expiresAt = updated.TtlSeconds != channel.Settings.TtlSeconds
? clock.NowMs + updated.TtlSeconds * 1000L ? clock.NowMs + updated.TtlSeconds * 1000L
@@ -91,7 +112,7 @@ public sealed partial class ChannelService(
tx.SetJoinPassword(id, hash, salt); tx.SetJoinPassword(id, hash, salt);
channel = channel with { JoinPasswordHash = hash, JoinPasswordSalt = salt }; channel = channel with { JoinPasswordHash = hash, JoinPasswordSalt = salt };
} }
return ToInfo(channel with { Settings = updated, ExpiresAt = expiresAt }, tx.CountMembers(id)); return ToInfo(channel with { Settings = updated, ExpiresAt = expiresAt }, tx.CountMembers(id), avatarVersion: tx.GetChannelAvatarVersion(id));
}); });
events.SettingsChanged(id); events.SettingsChanged(id);
return info; return info;
@@ -150,7 +171,9 @@ public sealed partial class ChannelService(
{ {
var (channel, principal) = Authenticate(tx, channelId, creds, Membership.Required); var (channel, principal) = Authenticate(tx, channelId, creds, Membership.Required);
CheckWritable(channel, principal, env); CheckWritable(channel, principal, env);
CheckObjectWrite(tx, channel, principal, env); CheckObjectWrite(tx, channel, principal, env.ItemId);
// 文件域的对象不存主后端:字节在文件域里,主后端只在整个文件传完后一次登记头部和摘要(见 RegisterFileItem)
if (IsFileObject(tx, channelId, env)) throw new WebDropException(ErrorCode.UseFileApi);
return tx.GetFrame(channelId, env.ItemId, env.FrameIndex); return tx.GetFrame(channelId, env.ItemId, env.FrameIndex);
}); });
if (existing is not null) return Resolve(existing, digest); if (existing is not null) return Resolve(existing, digest);
@@ -182,6 +205,166 @@ public sealed partial class ChannelService(
return outcome.Result; return outcome.Result;
} }
/// <summary>登记整个文件时,每一帧的负载长度和摘要(摘要是整帧封装的 SHA-256)</summary>
public readonly record struct FrameEntry(int PayloadLength, byte[] Digest);
/// <summary>
/// 一次登记整个文件:字节都在文件域里,逐帧登记要为每个 1 MiB 往返一次主后端,大文件的上传速度被它拖死。
/// 请求带的是末帧的头部和每一帧的(长度,摘要)。规则和逐帧登记完全一样:第 0 帧创建条目(含下载次数、日志里的公布),
/// 中间帧只落库,末帧验签名事件(清单摘要要和这里登记的所有摘要对得上)。整个登记在一个事务里,要么全成要么全不成。
/// 重复登记同一个文件(摘要全一致)幂等。
/// </summary>
public AppendResult RegisterFileItem(
Id16 channelId, Credentials creds, ReadOnlyMemory<byte> finalHeader, IReadOnlyList<FrameEntry> frames,
long? maxDownloads, SignedContentEventProof? signedEvent = null)
{
if (!Envelope.TryParseHeader(finalHeader.Span, out var last) || !last.IsFinal || frames.Count < 1
|| last.FrameIndex != frames.Count - 1 || frames[^1].PayloadLength != last.PayloadLength) throw new WebDropException(ErrorCode.BadRequest);
if (frames.Count > limits.MaxFramesPerItem) throw new WebDropException(ErrorCode.TooLarge);
if (frames.Any(f => f.Digest.Length != 32 || f.PayloadLength < 0 || f.PayloadLength > limits.MaxPayloadBytes)) throw new WebDropException(ErrorCode.BadRequest);
// 每一帧的头部由末帧头部推出:条目级标志一致,只有末帧带 FINAL
FrameFlags itemFlags = last.Flags & ~FrameFlags.Final;
Envelope EnvelopeOf(int index) => new(index == frames.Count - 1 ? last.Flags : itemFlags, last.ItemId, (uint)index, frames[index].PayloadLength);
ValidateShape(EnvelopeOf(0)); ValidateShape(last);
var outcomes = store.Write<(AppendOutcome? First, AppendOutcome? Last, long Seq)>(tx =>
{
var (channel, principal) = Authenticate(tx, channelId, creds, Membership.Required);
for (int i = 0; i < frames.Count; i++) CheckWritable(channel, principal, EnvelopeOf(i));
CheckObjectWrite(tx, channel, principal, last.ItemId);
if (!IsFileObject(tx, channelId, last)) throw new WebDropException(ErrorCode.BadRequest);
// 已经登记过同一个文件:摘要全部一致就是重复请求,否则冲突
if (tx.GetItem(channelId, last.ItemId) is not null)
{
var stored = tx.ListFrames(channelId, last.ItemId, 0, limits.MaxFramesPerItem);
if (stored.Count != frames.Count || stored.Select((f, i) => f.Index == i && f.Digest.AsSpan().SequenceEqual(frames[i].Digest)).Any(same => !same))
throw new WebDropException(ErrorCode.Conflict);
return (First: (AppendOutcome?)null, Last: (AppendOutcome?)null, Seq: stored[^1].Seq);
}
long middleBytes = 0;
for (int i = 1; i < frames.Count - 1; i++) middleBytes += EnvelopeOf(i).TotalLength;
if (channel.Bytes + middleBytes > limits.MaxWorkspaceBytes) throw new WebDropException(ErrorCode.TooLarge);
string BlobKeyOf(int i) => $"{FilesPrefix}{channelId.ToHex()}/{last.ItemId.ToHex()}/{i}";
var first = AppendInTx(tx, channelId, creds, EnvelopeOf(0), ReadOnlyMemory<byte>.Empty, frames[0].Digest, BlobKeyOf(0), maxDownloads,
frames.Count == 1 ? signedEvent : null);
if (frames.Count == 1) return (First: (AppendOutcome?)first, Last: null, Seq: first.LastSeq);
// 中间帧(受限条目的中间帧不进日志):只落库,条目计数一次性补上
for (int i = 1; i < frames.Count - 1; i++)
{
var env = EnvelopeOf(i);
tx.InsertFrame(new StoredFrame(channelId, last.ItemId, (uint)i, tx.NextSeq(channelId), env.TotalLength, frames[i].Digest, null, BlobKeyOf(i)));
}
if (frames.Count > 2)
{
var item = tx.GetItem(channelId, last.ItemId)!;
tx.UpdateItem(item with { FrameCount = item.FrameCount + frames.Count - 2, MaxIndex = Math.Max(item.MaxIndex, frames.Count - 2), Bytes = item.Bytes + middleBytes });
tx.AddChannelBytes(channelId, middleBytes);
}
var final = AppendInTx(tx, channelId, creds, last, ReadOnlyMemory<byte>.Empty, frames[^1].Digest, BlobKeyOf(frames.Count - 1), null, signedEvent);
return (First: (AppendOutcome?)first, Last: final, Seq: final.Result.Seq);
});
foreach (var outcome in new[] { outcomes.First, outcomes.Last })
{
if (outcome is null) continue;
foreach (string prefix in outcome.BlobPrefixesToDelete) blobs.DeletePrefix(prefix);
events.Advanced(channelId, outcome.LastSeq);
if (outcome.NotifySettings) events.SettingsChanged(channelId);
}
return new AppendResult { Seq = outcomes.Seq, Created = outcomes.First is not null };
}
/// <summary>条目所属频道的文件域(文件域的对象才有)</summary>
private static string? FileDomainOf(IResourceTx tx, Id16 spaceId, Id16 itemId)
{
var obj = tx.GetWorkspaceCrypto(spaceId).Objects.Find(o => o.ItemId == itemId.ToBase64Url() && o.Domain == CryptoDomain.Files);
return obj is null ? null : tx.GetWorkspace(spaceId)?.Channels.Find(c => c.Id == obj.ChannelId)?.FileDomain;
}
/// <summary>创建文件域的票据:只有空间所有者和有「管理频道和分组」权限的人能要。票据绑定空间 id</summary>
public FileGrant IssueCreateGrant(Id16 spaceId, Credentials creds)
{
if (!Files.Enabled) throw new WebDropException(ErrorCode.NotFound);
store.Read(tx =>
{
var (space, principal) = Authenticate(tx, spaceId, creds, Membership.Required);
if (!ChannelPolicy.CanManageStructure(tx.GetWorkspace(spaceId), principal)) throw new WebDropException(ErrorCode.Forbidden);
return 0;
});
return Files.Issue(FileOp.Create, spaceId.ToHex(), spaceId.ToHex(), "*", 0, clock.NowMs);
}
/// <summary>上传票据:确认能写这个条目所在的频道,然后给客户端文件域的访问凭证。</summary>
public FileGrant IssueUploadGrant(Id16 id, Credentials creds, Id16 itemId)
{
if (!Files.Enabled) throw new WebDropException(ErrorCode.NotFound);
string domain = store.Read(tx =>
{
var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
if (!ChannelPolicy.CanWrite(space, principal)) throw new WebDropException(ErrorCode.Forbidden);
CheckObjectWrite(tx, space, principal, itemId);
return FileDomainOf(tx, id, itemId) ?? throw new WebDropException(ErrorCode.Conflict);
});
return Files.Issue(FileOp.Put, id.ToHex(), domain, itemId.ToHex(), 0, clock.NowMs);
}
/// <summary>
/// 文件域的查询票据(看容量和用量):能读这个频道的文件就行。票据的条目是 "*",只能查域,读不了任何文件。
/// </summary>
public FileGrant IssueDomainGrant(Id16 id, Credentials creds, string threadId)
{
if (!Files.Enabled) throw new WebDropException(ErrorCode.NotFound);
string domain = store.Read(tx =>
{
var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
var workspace = tx.GetWorkspace(id) ?? throw new WebDropException(ErrorCode.NotFound);
var thread = workspace.Channels.Find(c => c.Id == threadId) ?? throw new WebDropException(ErrorCode.NotFound);
var crypto = tx.GetWorkspaceCrypto(id);
bool allowed = ChannelPolicy.IsSpaceOwner(space, principal)
|| crypto.Access.Count == 0 && space.Mode == ChannelMode.Raw
|| ChannelPolicy.AccessOf(crypto, principal.AccountName, threadId) is { } access && ChannelPolicy.CanUseFiles(access);
if (!allowed) throw new WebDropException(ErrorCode.Forbidden);
return thread.FileDomain ?? throw new WebDropException(ErrorCode.NotFound);
});
return Files.Issue(FileOp.Get, id.ToHex(), domain, "*", 0, clock.NowMs);
}
/// <summary>删除票据:确认调用者按原来的规则能删这个条目,再签。客户端先拿票据,再删主后端的条目,最后用票据清文件域里的字节</summary>
public FileGrant IssueDeleteGrant(Id16 id, Credentials creds, Id16 itemId)
{
if (!Files.Enabled) throw new WebDropException(ErrorCode.NotFound);
string domain = store.Read(tx =>
{
var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
var item = tx.GetItem(id, itemId) ?? throw new WebDropException(ErrorCode.NotFound);
if (!ChannelPolicy.CanDeleteItem(space, item, principal)) throw new WebDropException(ErrorCode.Forbidden);
return FileDomainOf(tx, id, itemId) ?? throw new WebDropException(ErrorCode.NotFound);
});
return Files.Issue(FileOp.Delete, id.ToHex(), domain, itemId.ToHex(), 0, clock.NowMs);
}
/// <summary>
/// 下载票据:按原来的规则鉴权并计次(只有 from=0 才计),再签票据。条目没传完时返回冲突,让客户端稍后再来。
/// </summary>
public FileGrant IssueDownloadGrant(Id16 id, Credentials creds, Id16 itemId, uint fromIndex)
{
if (!Files.Enabled) throw new WebDropException(ErrorCode.NotFound);
string domain = store.Read(tx =>
{
var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
if (!ChannelPolicy.CanRead(principal)) throw new WebDropException(ErrorCode.Forbidden);
var item = tx.GetItem(id, itemId) ?? throw new WebDropException(ErrorCode.NotFound);
if (!CanReadObject(tx, space, principal, itemId)) throw new WebDropException(ErrorCode.Forbidden);
if (item.FinalIndex is null) throw new WebDropException(ErrorCode.Conflict);
return FileDomainOf(tx, id, itemId) ?? throw new WebDropException(ErrorCode.NotFound);
});
StartItemDownload(id, creds, itemId, fromIndex, allowRemote: true);
return Files.Issue(FileOp.Get, id.ToHex(), domain, itemId.ToHex(), fromIndex, clock.NowMs);
}
private sealed class AppendOutcome(AppendResult result, long lastSeq, List<string> prefixes, bool notifySettings = false) private sealed class AppendOutcome(AppendResult result, long lastSeq, List<string> prefixes, bool notifySettings = false)
{ {
public AppendResult Result { get; } = result; public AppendResult Result { get; } = result;
@@ -196,7 +379,8 @@ public sealed partial class ChannelService(
{ {
var (channel, principal) = Authenticate(tx, channelId, creds, Membership.Required); var (channel, principal) = Authenticate(tx, channelId, creds, Membership.Required);
CheckWritable(channel, principal, env); CheckWritable(channel, principal, env);
CheckObjectWrite(tx, channel, principal, env); CheckObjectWrite(tx, channel, principal, env.ItemId);
int bodyLength = IsRemote(blobKey) ? env.TotalLength : body.Length;
// 并发下预检之后可能已有同一帧 // 并发下预检之后可能已有同一帧
if (tx.GetFrame(channelId, env.ItemId, env.FrameIndex) is { } dup) if (tx.GetFrame(channelId, env.ItemId, env.FrameIndex) is { } dup)
@@ -210,7 +394,6 @@ public sealed partial class ChannelService(
var workspaceState = objectEnvelope is null ? null : tx.GetWorkspace(channelId); var workspaceState = objectEnvelope is null ? null : tx.GetWorkspace(channelId);
var workspaceChannel = objectEnvelope is null ? null : workspaceState?.Channels.Find(c => c.Id == objectEnvelope.ChannelId); var workspaceChannel = objectEnvelope is null ? null : workspaceState?.Channels.Find(c => c.Id == objectEnvelope.ChannelId);
if (objectEnvelope is not null && workspaceChannel is null) throw new WebDropException(ErrorCode.BadRequest); if (objectEnvelope is not null && workspaceChannel is null) throw new WebDropException(ErrorCode.BadRequest);
bool countsTowardFileQuota = objectEnvelope?.Domain == CryptoDomain.Files && !env.IsControl;
if (signedEvent is not null && (objectEnvelope is null || !env.IsFinal)) throw new WebDropException(ErrorCode.BadRequest); if (signedEvent is not null && (objectEnvelope is null || !env.IsFinal)) throw new WebDropException(ErrorCode.BadRequest);
if (objectEnvelope is not null && env.IsFinal) if (objectEnvelope is not null && env.IsFinal)
{ {
@@ -275,14 +458,12 @@ public sealed partial class ChannelService(
if (item.MaxIndex > env.FrameIndex) throw new WebDropException(ErrorCode.BadRequest); if (item.MaxIndex > env.FrameIndex) throw new WebDropException(ErrorCode.BadRequest);
if (item.FinalIndex is { } f && f != env.FrameIndex) throw new WebDropException(ErrorCode.BadRequest); if (item.FinalIndex is { } f && f != env.FrameIndex) throw new WebDropException(ErrorCode.BadRequest);
} }
if (objectEnvelope is null && channel.Bytes + body.Length > limits.MaxChannelBytes if (objectEnvelope is null && channel.Bytes + bodyLength > limits.MaxChannelBytes
|| objectEnvelope is not null && channel.Bytes + body.Length > limits.MaxWorkspaceBytes) || objectEnvelope is not null && channel.Bytes + bodyLength > limits.MaxWorkspaceBytes)
throw new WebDropException(ErrorCode.TooLarge);
if (countsTowardFileQuota && workspaceChannel!.FileBytes + body.Length > limits.MaxWorkspaceChannelBytes)
throw new WebDropException(ErrorCode.TooLarge); throw new WebDropException(ErrorCode.TooLarge);
long seq = tx.NextSeq(channelId); long seq = tx.NextSeq(channelId);
tx.InsertFrame(new StoredFrame(channelId, env.ItemId, env.FrameIndex, seq, body.Length, digest, tx.InsertFrame(new StoredFrame(channelId, env.ItemId, env.FrameIndex, seq, bodyLength, digest,
blobKey is null ? body.ToArray() : null, blobKey)); blobKey is null ? body.ToArray() : null, blobKey));
if (signedEvent is not null && objectEnvelope is not null) if (signedEvent is not null && objectEnvelope is not null)
@@ -292,7 +473,8 @@ public sealed partial class ChannelService(
tx.PutWorkspaceCrypto(channelId, crypto); tx.PutWorkspaceCrypto(channelId, crypto);
} }
if (!env.IsGated) // 受限帧和放在文件服务上的帧不进日志(日志里的帧自带内容),只在第 0 帧公布一次
if (!env.IsGated && !IsRemote(blobKey))
tx.InsertLog(channelId, new LogEntry(seq, LogKind.Frame, env.ItemId, env.FrameIndex, 0, now)); tx.InsertLog(channelId, new LogEntry(seq, LogKind.Frame, env.ItemId, env.FrameIndex, 0, now));
else if (env.FrameIndex == 0) else if (env.FrameIndex == 0)
tx.InsertLog(channelId, new LogEntry(seq, LogKind.Announce, env.ItemId, 0, 0, now)); tx.InsertLog(channelId, new LogEntry(seq, LogKind.Announce, env.ItemId, 0, 0, now));
@@ -302,9 +484,9 @@ public sealed partial class ChannelService(
FrameCount = item.FrameCount + 1, FrameCount = item.FrameCount + 1,
MaxIndex = Math.Max(item.MaxIndex, env.FrameIndex), MaxIndex = Math.Max(item.MaxIndex, env.FrameIndex),
FinalIndex = env.IsFinal ? env.FrameIndex : item.FinalIndex, FinalIndex = env.IsFinal ? env.FrameIndex : item.FinalIndex,
Bytes = item.Bytes + body.Length, Bytes = item.Bytes + bodyLength,
}); });
tx.AddChannelBytes(channelId, body.Length); tx.AddChannelBytes(channelId, bodyLength);
tx.TouchChannel(channelId, Math.Max(channel.ExpiresAt, now + ttlMs)); tx.TouchChannel(channelId, Math.Max(channel.ExpiresAt, now + ttlMs));
long lastSeq = seq; long lastSeq = seq;
@@ -334,16 +516,7 @@ public sealed partial class ChannelService(
if (pruned > channel.CompactedThrough) tx.SetCompactedThrough(channelId, pruned); if (pruned > channel.CompactedThrough) tx.SetCompactedThrough(channelId, pruned);
} }
if (countsTowardFileQuota) return new AppendOutcome(new AppendResult { Seq = seq, Created = true }, lastSeq, prefixes);
{
var latestWorkspace = tx.GetWorkspace(channelId) ?? throw new WebDropException(ErrorCode.NotFound);
var latestChannel = latestWorkspace.Channels.Find(c => c.Id == objectEnvelope!.ChannelId) ?? throw new WebDropException(ErrorCode.NotFound);
latestChannel.FileBytes += body.Length;
tx.PutWorkspace(channelId, latestWorkspace);
}
return new AppendOutcome(new AppendResult { Seq = seq, Created = true }, lastSeq, prefixes,
countsTowardFileQuota && env.IsFinal);
} }
private static AppendResult Resolve(StoredFrame existing, byte[] digest) => private static AppendResult Resolve(StoredFrame existing, byte[] digest) =>
@@ -425,7 +598,7 @@ public sealed partial class ChannelService(
} }
/// <summary>开始拉取条目:鉴权,并按规则计一次下载。<paramref name="fromIndex"/> 为 0 才计数,大于 0 视为续传。</summary> /// <summary>开始拉取条目:鉴权,并按规则计一次下载。<paramref name="fromIndex"/> 为 0 才计数,大于 0 视为续传。</summary>
public void StartItemDownload(Id16 id, Credentials creds, Id16 itemId, uint fromIndex) public void StartItemDownload(Id16 id, Credentials creds, Id16 itemId, uint fromIndex, bool allowRemote = false)
{ {
store.Write(tx => store.Write(tx =>
{ {
@@ -433,6 +606,7 @@ public sealed partial class ChannelService(
if (!ChannelPolicy.CanRead(principal)) throw new WebDropException(ErrorCode.Forbidden); if (!ChannelPolicy.CanRead(principal)) throw new WebDropException(ErrorCode.Forbidden);
var item = tx.GetItem(id, itemId) ?? throw new WebDropException(ErrorCode.NotFound); var item = tx.GetItem(id, itemId) ?? throw new WebDropException(ErrorCode.NotFound);
if (!CanReadObject(tx, space, principal, itemId)) throw new WebDropException(ErrorCode.Forbidden); if (!CanReadObject(tx, space, principal, itemId)) throw new WebDropException(ErrorCode.Forbidden);
if (!allowRemote && IsRemote(tx.GetFrame(id, itemId, 0)?.BlobKey)) throw new WebDropException(ErrorCode.UseFileApi);
long now = clock.NowMs; long now = clock.NowMs;
if (item.ExpiresAt <= now) throw new WebDropException(ErrorCode.Gone); if (item.ExpiresAt <= now) throw new WebDropException(ErrorCode.Gone);
@@ -494,8 +668,11 @@ public sealed partial class ChannelService(
return new ItemChunk(result, next, complete); return new ItemChunk(result, next, complete);
} }
private async ValueTask<byte[]?> LoadFrameBytesAsync(StoredFrame frame, CancellationToken ct) => private async ValueTask<byte[]?> LoadFrameBytesAsync(StoredFrame frame, CancellationToken ct)
frame.Inline ?? (frame.BlobKey is null ? null : await blobs.ReadAsync(frame.BlobKey, ct).ConfigureAwait(false)); {
if (IsRemote(frame.BlobKey)) throw new WebDropException(ErrorCode.UseFileApi);
return frame.Inline ?? (frame.BlobKey is null ? null : await blobs.ReadAsync(frame.BlobKey, ct).ConfigureAwait(false));
}
// ───────────────────────── 删除 ───────────────────────── // ───────────────────────── 删除 ─────────────────────────
@@ -515,7 +692,9 @@ public sealed partial class ChannelService(
var crypto = tx.GetWorkspaceCrypto(id); var crypto = tx.GetWorkspaceCrypto(id);
var obj = crypto.Objects.Find(o => o.ItemId == itemId.ToBase64Url()); var obj = crypto.Objects.Find(o => o.ItemId == itemId.ToBase64Url());
if (creds.RequireSignedContent && obj is not null) // 没传完的上传(末帧从未入库)没有已验签的清单,也就不可能有删除事件:作者或空间主可以直接清理,否则它会永久占着空间配额
bool unfinished = obj is not null && obj.Event is null;
if (creds.RequireSignedContent && obj is not null && !unfinished)
{ {
if (signedEvent is null || obj?.Event is null || signedEvent.Username != principal.AccountName if (signedEvent is null || obj?.Event is null || signedEvent.Username != principal.AccountName
|| signedEvent.DeviceId != creds.AccountDeviceId || signedEvent.ChannelId != obj.ChannelId || signedEvent.DeviceId != creds.AccountDeviceId || signedEvent.ChannelId != obj.ChannelId
@@ -544,18 +723,6 @@ public sealed partial class ChannelService(
private long DeleteItemCore(IResourceTx tx, Id16 channelId, Item item, TombstoneReason reason, List<string> prefixes) private long DeleteItemCore(IResourceTx tx, Id16 channelId, Item item, TombstoneReason reason, List<string> prefixes)
{ {
var crypto = tx.GetWorkspaceCrypto(channelId);
var objectEnvelope = crypto.Objects.Find(o => o.ItemId == item.ItemId.ToBase64Url());
if (objectEnvelope?.Domain == CryptoDomain.Files && (item.Flags & FrameFlags.Control) == 0)
{
var workspace = tx.GetWorkspace(channelId);
var workspaceChannel = workspace?.Channels.Find(c => c.Id == objectEnvelope.ChannelId);
if (workspace is not null && workspaceChannel is not null)
{
workspaceChannel.FileBytes = Math.Max(0, workspaceChannel.FileBytes - item.Bytes);
tx.PutWorkspace(channelId, workspace);
}
}
tx.DeleteItemRows(channelId, item.ItemId); tx.DeleteItemRows(channelId, item.ItemId);
tx.AddChannelBytes(channelId, -item.Bytes); tx.AddChannelBytes(channelId, -item.Bytes);
long seq = tx.NextSeq(channelId); long seq = tx.NextSeq(channelId);
@@ -568,7 +735,7 @@ public sealed partial class ChannelService(
{ {
EventId = Codec.Encode(signedEvent.EventIdBytes), Kind = signedEvent.EventKind, FrameCount = signedEvent.FrameCount, EventId = Codec.Encode(signedEvent.EventIdBytes), Kind = signedEvent.EventKind, FrameCount = signedEvent.FrameCount,
ManifestHash = Convert.ToBase64String(signedEvent.ManifestHash).TrimEnd('=').Replace('+', '-').Replace('/', '_'), ManifestHash = Convert.ToBase64String(signedEvent.ManifestHash).TrimEnd('=').Replace('+', '-').Replace('/', '_'),
SignerDeviceId = signedEvent.DeviceId, SignerDeviceId = signedEvent.DeviceId, SignerAccount = signedEvent.Username,
Signature = Convert.ToBase64String(signedEvent.Signature).TrimEnd('=').Replace('+', '-').Replace('/', '_'), Signature = Convert.ToBase64String(signedEvent.Signature).TrimEnd('=').Replace('+', '-').Replace('/', '_'),
TargetItemId = signedEvent.TargetItemId, TargetItemId = signedEvent.TargetItemId,
SignerRootPublicKey = signedEvent.RootPublicKey, SignerDevicePublicKey = signedEvent.DevicePublicKey, SignerRootPublicKey = signedEvent.RootPublicKey, SignerDevicePublicKey = signedEvent.DevicePublicKey,
@@ -675,10 +842,24 @@ public sealed partial class ChannelService(
if (!ChannelPolicy.IsOwner(principal)) throw new WebDropException(ErrorCode.Forbidden); if (!ChannelPolicy.IsOwner(principal)) throw new WebDropException(ErrorCode.Forbidden);
} }
private static ChannelInfo ToInfo(Channel channel, int membersUsed, Principal? principal = null) => new() private static ChannelInfo ToInfo(Channel channel, int membersUsed, Principal? principal = null, string? avatarVersion = null, WorkspaceState? workspace = null)
{ {
CanUpload = ChannelPolicy.CanWrite(channel, principal ?? new Principal(Role.Owner, channel.CreatorAccount)), // 没有调用者(创建、改设置时)就是所有者本人
CanManageStructure = principal is null || principal.Role == Role.Owner, var who = principal ?? new Principal(Role.Owner, channel.CreatorAccount);
return new()
{
AvatarVersion = avatarVersion,
IsOwner = ChannelPolicy.IsOwner(who),
CanUpload = ChannelPolicy.CanWrite(channel, who),
CanManageStructure = ChannelPolicy.CanManageStructure(workspace, who),
CanCreateChannel = ChannelPolicy.CanCreateChannel(who),
CanManageFolders = ChannelPolicy.CanManageFolders(who),
CanInvite = ChannelPolicy.CanInvite(who),
CanEditSettings = ChannelPolicy.CanEditSettings(who),
CanManageRoles = ChannelPolicy.CanManageRoles(who),
CanRemoveMembers = ChannelPolicy.CanRemoveMembers(workspace, who),
CanDeleteAny = ChannelPolicy.CanDeleteAnyItem(channel, who),
CanDeleteOwn = ChannelPolicy.CanDeleteOwnItem(channel, who),
Id = channel.Id.ToBase64Url(), Id = channel.Id.ToBase64Url(),
CreatorAccount = channel.CreatorAccount, CreatorAccount = channel.CreatorAccount,
PasswordRequired = channel.JoinPasswordHash is not null, PasswordRequired = channel.JoinPasswordHash is not null,
@@ -690,4 +871,5 @@ public sealed partial class ChannelService(
CreatedAt = channel.CreatedAt, CreatedAt = channel.CreatedAt,
ExpiresAt = channel.ExpiresAt, ExpiresAt = channel.ExpiresAt,
}; };
}
} }
@@ -52,10 +52,27 @@ public sealed class ShareItemInfo
public sealed class ChannelInfo public sealed class ChannelInfo
{ {
// ───── 调用者在这个空间里的能力。前端只读不算:规则只在后端一处
/// <summary>调用者是空间所有者</summary>
[JsonPropertyName("is_owner")] public bool IsOwner { get; init; }
[JsonPropertyName("can_upload")] public bool CanUpload { get; init; } [JsonPropertyName("can_upload")] public bool CanUpload { get; init; }
/// <summary>能重命名频道、管理分组(所有者,或被授予「管理频道和分组」)</summary>
[JsonPropertyName("can_manage_structure")] public bool CanManageStructure { get; init; } [JsonPropertyName("can_manage_structure")] public bool CanManageStructure { get; init; }
[JsonPropertyName("can_create_channel")] public bool CanCreateChannel { get; init; }
[JsonPropertyName("can_manage_folders")] public bool CanManageFolders { get; init; }
[JsonPropertyName("can_invite")] public bool CanInvite { get; init; }
[JsonPropertyName("can_edit_settings")] public bool CanEditSettings { get; init; }
[JsonPropertyName("can_manage_roles")] public bool CanManageRoles { get; init; }
/// <summary>能移除成员、撤销邀请;具体能移除谁见成员列表里每一项的 can_remove</summary>
[JsonPropertyName("can_remove_members")] public bool CanRemoveMembers { get; init; }
/// <summary>能删除任何人的消息和文件(所有者,或删除策略为 anyone)</summary>
[JsonPropertyName("can_delete_any")] public bool CanDeleteAny { get; init; }
/// <summary>能删除自己发的消息和文件</summary>
[JsonPropertyName("can_delete_own")] public bool CanDeleteOwn { get; init; }
[JsonPropertyName("password_required")] public bool PasswordRequired { get; init; } [JsonPropertyName("password_required")] public bool PasswordRequired { get; init; }
[JsonPropertyName("creator_account")] public string? CreatorAccount { get; init; } [JsonPropertyName("creator_account")] public string? CreatorAccount { get; init; }
/// <summary>头像版本标记,变了就要重新取头像(GET /channels/{id}/avatar);没有头像为 null。头像本体不随空间信息下发。</summary>
[JsonPropertyName("avatar_version")] public string? AvatarVersion { get; init; }
[JsonPropertyName("id")] public required string Id { get; init; } [JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("mode")] public required ChannelMode Mode { get; init; } [JsonPropertyName("mode")] public required ChannelMode Mode { get; init; }
[JsonPropertyName("settings")] public required ChannelSettings Settings { get; init; } [JsonPropertyName("settings")] public required ChannelSettings Settings { get; init; }
@@ -9,7 +9,6 @@ public sealed class ServerLimits
public int MaxFramesPerItem { get; set; } = 65_536; public int MaxFramesPerItem { get; set; } = 65_536;
public long MaxChannelBytes { get; set; } = 1L << 30; public long MaxChannelBytes { get; set; } = 1L << 30;
public long MaxWorkspaceBytes { get; set; } = 1L << 40; public long MaxWorkspaceBytes { get; set; } = 1L << 40;
public long MaxWorkspaceChannelBytes { get; set; } = 10L << 30;
/// <summary>最大账号成员数。属性名和 info 字段保持旧线格式兼容。</summary> /// <summary>最大账号成员数。属性名和 info 字段保持旧线格式兼容。</summary>
public int MaxMembers { get; set; } = 64; public int MaxMembers { get; set; } = 64;
public int MaxVisitors { get; set; } = 10_000; public int MaxVisitors { get; set; } = 10_000;
@@ -64,6 +63,10 @@ public sealed class ChannelSettingsPatch
[JsonPropertyName("max_items")] public int? MaxItems { get; init; } [JsonPropertyName("max_items")] public int? MaxItems { get; init; }
[JsonPropertyName("file_max_downloads")] public int? FileMaxDownloads { get; init; } [JsonPropertyName("file_max_downloads")] public int? FileMaxDownloads { get; init; }
[JsonPropertyName("ui_restrict_copy_download")] public bool? UiRestrictCopyDownload { get; init; } [JsonPropertyName("ui_restrict_copy_download")] public bool? UiRestrictCopyDownload { get; init; }
/// <summary>空间头像 data URL;不传表示不改,空字符串表示移除。只有空间主能改。</summary>
[JsonPropertyName("avatar")] public string? Avatar { get; init; }
/// <summary>空间头像的有损小图(data URL):空间列表、标题默认只分发它,原图在预览时才下载</summary>
[JsonPropertyName("avatar_thumb")] public string? AvatarThumb { get; init; }
public ChannelSettings ApplyTo(ChannelSettings s) => new() public ChannelSettings ApplyTo(ChannelSettings s) => new()
{ {
File renamed without changes.
@@ -66,3 +66,10 @@ public enum TombstoneReason : byte
DownloadLimit = 3, DownloadLimit = 3,
Superseded = 4, Superseded = 4,
} }
/// <summary>便捷分享的状态(线格式的取值)。前端从生成的契约里取,不手写字符串</summary>
public static class ShareState
{
public const string Draft = "draft";
public const string Sealed = "sealed";
}
@@ -30,6 +30,18 @@ public readonly record struct Envelope(FrameFlags Flags, Id16 ItemId, uint Frame
return true; return true;
} }
/// <summary>只有 26 字节头部(不带负载)的帧:文件服务存负载,主后端只收头部和回执。长度字段仍要合法。</summary>
public static bool TryParseHeader(ReadOnlySpan<byte> data, out Envelope envelope)
{
envelope = default;
if (data.Length != HeaderSize || data[0] != Version) return false;
if ((data[1] & 0xE0) != 0) return false;
uint payloadLen = BinaryPrimitives.ReadUInt32BigEndian(data[22..]);
if (payloadLen > int.MaxValue) return false;
envelope = new Envelope((FrameFlags)data[1], Id16.FromBytes(data.Slice(2, Id16.Size)), BinaryPrimitives.ReadUInt32BigEndian(data[18..]), (int)payloadLen);
return true;
}
public bool IsFinal => (Flags & FrameFlags.Final) != 0; public bool IsFinal => (Flags & FrameFlags.Final) != 0;
public bool IsGated => (Flags & FrameFlags.Gated) != 0; public bool IsGated => (Flags & FrameFlags.Gated) != 0;
public bool IsSnapshot => (Flags & FrameFlags.Snapshot) != 0; public bool IsSnapshot => (Flags & FrameFlags.Snapshot) != 0;
@@ -22,6 +22,10 @@ public enum ErrorCode
ChannelPasswordRequired, ChannelPasswordRequired,
ChannelPasswordInvalid, ChannelPasswordInvalid,
SpaceNameRequired, SpaceNameRequired,
/// <summary>这个条目的内容在文件服务上:去要一张票据,直接找文件服务</summary>
UseFileApi,
/// <summary>文件服务暂时连不上或已满</summary>
FilesUnavailable,
} }
/// <summary>预期内的业务失败。消息只含错误码,绝不回显用户提交的内容。</summary> /// <summary>预期内的业务失败。消息只含错误码,绝不回显用户提交的内容。</summary>
@@ -63,6 +67,8 @@ public static class ErrorCodes
ErrorCode.ChannelPasswordRequired => "channel_password_required", ErrorCode.ChannelPasswordRequired => "channel_password_required",
ErrorCode.ChannelPasswordInvalid => "channel_password_invalid", ErrorCode.ChannelPasswordInvalid => "channel_password_invalid",
ErrorCode.SpaceNameRequired => "space_name_required", ErrorCode.SpaceNameRequired => "space_name_required",
ErrorCode.UseFileApi => "use_file_api",
ErrorCode.FilesUnavailable => "files_unavailable",
_ => "bad_request", _ => "bad_request",
}; };
@@ -79,6 +85,8 @@ public static class ErrorCodes
ErrorCode.TooLarge => 413, ErrorCode.TooLarge => 413,
ErrorCode.CaptchaRequired => 428, ErrorCode.CaptchaRequired => 428,
ErrorCode.CaptchaUnavailable => 503, ErrorCode.CaptchaUnavailable => 503,
ErrorCode.FilesUnavailable => 503,
ErrorCode.UseFileApi => 409,
ErrorCode.RateLimited => 429, ErrorCode.RateLimited => 429,
ErrorCode.VisitorsFull => 403, ErrorCode.VisitorsFull => 403,
ErrorCode.AccountRequired => 401, ErrorCode.AccountRequired => 401,
@@ -25,6 +25,15 @@ public readonly struct Id16 : IEquatable<Id16>
} }
/// <summary>只接受规范的 22 字符 base64url,拒绝同一值的非规范编码。</summary> /// <summary>只接受规范的 22 字符 base64url,拒绝同一值的非规范编码。</summary>
/// <summary>32 个十六进制字符(文件服务和 blob 键里的写法)</summary>
public static bool TryParseHex(string? hex, out Id16 id)
{
id = default;
if (hex is not { Length: 32 }) return false;
try { id = FromBytes(Convert.FromHexString(hex)); return true; }
catch (FormatException) { return false; }
}
public static bool TryParse(ReadOnlySpan<char> text, out Id16 id) public static bool TryParse(ReadOnlySpan<char> text, out Id16 id)
{ {
id = default; id = default;
@@ -84,6 +84,8 @@ public sealed record Item(
public sealed record ChannelInvitation(Id16 ChannelId, string Username, string Payload, long CreatedAt); public sealed record ChannelInvitation(Id16 ChannelId, string Username, string Payload, long CreatedAt);
public sealed record ChannelMember(string Username, string Role, long JoinedAt); public sealed record ChannelMember(string Username, string Role, long JoinedAt);
public sealed record MemberView(ChannelMember Member, bool CanRemove);
public sealed record InvitationView(ChannelInvitation Invitation, bool CanRevoke);
public sealed record ChannelKeyPackage(Id16 ChannelId, string Username, string Role, string Payload, long CreatedAt); public sealed record ChannelKeyPackage(Id16 ChannelId, string Username, string Role, string Payload, long CreatedAt);
/// <summary>已存储的一帧(完整封装)。Inline 与 BlobKey 二选一。</summary> /// <summary>已存储的一帧(完整封装)。Inline 与 BlobKey 二选一。</summary>
@@ -0,0 +1,27 @@
using WebDrop.FileProtocol;
namespace WebDrop.Core;
/// <summary>给客户端的访问凭证:文件服务地址、文件域 id 和一张短期票据</summary>
public sealed record FileGrant(string Url, string Domain, string Ticket, long ExpiresAt);
/// <summary>
/// 主后端只负责「权限」:判断某人在某个空间里能不能读写某个频道的文件,然后给文件服务(fileAPI)签一张短期票据。
/// 文件域的创建、容量、用量、字节的存取都是文件服务自己的事,主后端既不调用它,也不保存它的任何状态。
/// </summary>
public interface IFileGrants
{
bool Enabled { get; }
/// <summary>文件服务对外的地址,告诉客户端去哪里</summary>
string PublicUrl { get; }
/// <summary>space:签发票据的空间 id(十六进制)。scope:Create 时是空间 id;其余是文件域 id。item 是条目 id(十六进制)或 "*"</summary>
FileGrant Issue(FileOp op, string space, string scope, string item, uint from, long nowMs);
}
public sealed class NoFileGrants : IFileGrants
{
public static readonly NoFileGrants Instance = new();
public bool Enabled => false;
public string PublicUrl => "";
public FileGrant Issue(FileOp op, string space, string scope, string item, uint from, long nowMs) => throw new WebDropException(ErrorCode.NotFound);
}
@@ -355,7 +355,7 @@ public sealed class ShareService(
private static ShareInfo ToInfo(Share share, IReadOnlyList<ShareItem> items) => new() private static ShareInfo ToInfo(Share share, IReadOnlyList<ShareItem> items) => new()
{ {
Id = share.Id.ToBase64Url(), Id = share.Id.ToBase64Url(),
State = share.IsSealed ? "sealed" : "draft", State = share.IsSealed ? ShareState.Sealed : ShareState.Draft,
ExpiresAt = share.ExpiresAt, ExpiresAt = share.ExpiresAt,
CreatedAt = share.CreatedAt, CreatedAt = share.CreatedAt,
SealedAt = share.SealedAt, SealedAt = share.SealedAt,
@@ -27,8 +27,7 @@ public sealed partial class ChannelService
} }
else else
{ {
var recipients = crypto.Access.Where(a => a.ChannelId == channelId && (domain == CryptoDomain.Chat ? a.ChatRead : a.FilesRead) var recipients = ChannelPolicy.Readers(crypto, channelId, domain).Where(a => (a.Username == space.CreatorAccount || tx.GetChannelMember(id, a.Username) is not null || tx.HasInvitation(id, a.Username)))
&& (a.Username == space.CreatorAccount || tx.GetChannelMember(id, a.Username) is not null || tx.HasInvitation(id, a.Username)))
.Select(a => a.Username).Append(space.CreatorAccount!).Append(username).Distinct().Order(StringComparer.Ordinal).ToList(); .Select(a => a.Username).Append(space.CreatorAccount!).Append(username).Distinct().Order(StringComparer.Ordinal).ToList();
if (recipients.Count > 256) throw new WebDropException(ErrorCode.TooLarge); if (recipients.Count > 256) throw new WebDropException(ErrorCode.TooLarge);
batches.Add(new CryptoGrantBatch { Domain = domain, Epoch = currentEpoch + 1, Recipients = recipients, Rotate = true }); batches.Add(new CryptoGrantBatch { Domain = domain, Epoch = currentEpoch + 1, Recipients = recipients, Rotate = true });
@@ -67,8 +66,7 @@ public sealed partial class ChannelService
} }
else else
{ {
var recipients = state.Access.Where(a => a.ChannelId == channelId && (domain == CryptoDomain.Chat ? a.ChatRead : a.FilesRead) var recipients = ChannelPolicy.Readers(state, channelId, domain).Where(a => (a.Username == space.CreatorAccount || tx.GetChannelMember(id, a.Username) is not null || tx.HasInvitation(id, a.Username)))
&& (a.Username == space.CreatorAccount || tx.GetChannelMember(id, a.Username) is not null || tx.HasInvitation(id, a.Username)))
.Select(a => a.Username).Append(space.CreatorAccount!).Append(username).Distinct().ToHashSet(StringComparer.Ordinal); .Select(a => a.Username).Append(space.CreatorAccount!).Append(username).Distinct().ToHashSet(StringComparer.Ordinal);
if (recipients.Count > 256) throw new WebDropException(ErrorCode.TooLarge); if (recipients.Count > 256) throw new WebDropException(ErrorCode.TooLarge);
expected.AddRange(recipients.Select(recipient => (domain, currentEpoch + 1, recipient))); expected.AddRange(recipients.Select(recipient => (domain, currentEpoch + 1, recipient)));
@@ -111,38 +109,34 @@ public sealed partial class ChannelService
if (crypto.Envelopes.Count == 0 && crypto.Objects.Count == 0 && space.Mode == ChannelMode.Enc) return true; if (crypto.Envelopes.Count == 0 && crypto.Objects.Count == 0 && space.Mode == ChannelMode.Enc) return true;
var item = crypto.Objects.Find(o => o.ItemId == itemId.ToBase64Url()); var item = crypto.Objects.Find(o => o.ItemId == itemId.ToBase64Url());
if (item is null) return false; if (item is null) return false;
if (principal.Role == Role.Owner || principal.AccountName == space.CreatorAccount) return true; if (ChannelPolicy.IsSpaceOwner(space, principal)) return true;
var access = crypto.Access.Find(a => a.ChannelId == item.ChannelId && a.Username == principal.AccountName); if (!ChannelPolicy.CanReadObject(ChannelPolicy.AccessOf(crypto, principal.AccountName, item.ChannelId), item.Domain, item.CreatedRevision)) return false;
if (access is null || (item.Domain == CryptoDomain.Chat ? !access.ChatRead || item.CreatedRevision < access.ChatFromRevision : !access.FilesRead || item.CreatedRevision < access.FilesFromRevision)) return false;
if (space.Mode == ChannelMode.Raw) return true; if (space.Mode == ChannelMode.Raw) return true;
return crypto.Envelopes.Exists(e => e.ChannelId == item.ChannelId && e.Domain == item.Domain && e.Epoch == item.Epoch && e.Username == principal.AccountName); return crypto.Envelopes.Exists(e => e.ChannelId == item.ChannelId && e.Domain == item.Domain && e.Epoch == item.Epoch && e.Username == principal.AccountName);
} }
private static void CheckObjectWrite(IResourceTx tx, Channel space, Principal principal, Envelope envelope) private static void CheckObjectWrite(IResourceTx tx, Channel space, Principal principal, Id16 itemId)
{ {
var crypto = tx.GetWorkspaceCrypto(space.Id); var crypto = tx.GetWorkspaceCrypto(space.Id);
if (space.Mode == ChannelMode.Enc && crypto.Envelopes.Count == 0) return; if (!ChannelPolicy.GrantsEnforced(space, crypto)) return;
if (space.Mode == ChannelMode.Raw && crypto.Access.Count == 0 && crypto.Objects.Count == 0) return; var item = crypto.Objects.Find(o => o.ItemId == itemId.ToBase64Url()) ?? throw new WebDropException(ErrorCode.BadRequest);
var item = crypto.Objects.Find(o => o.ItemId == envelope.ItemId.ToBase64Url()) ?? throw new WebDropException(ErrorCode.BadRequest);
if (item.AuthorAccount != principal.AccountName) throw new WebDropException(ErrorCode.Forbidden); if (item.AuthorAccount != principal.AccountName) throw new WebDropException(ErrorCode.Forbidden);
var channel = tx.GetWorkspace(space.Id)?.Channels.Find(c => c.Id == item.ChannelId); var channel = tx.GetWorkspace(space.Id)?.Channels.Find(c => c.Id == item.ChannelId);
if (space.Mode == ChannelMode.Enc && channel is not null && item.Epoch != (item.Domain == CryptoDomain.Chat ? channel.ChatEpoch : channel.FilesEpoch)) throw new WebDropException(ErrorCode.Conflict); if (space.Mode == ChannelMode.Enc && channel is not null && item.Epoch != (item.Domain == CryptoDomain.Chat ? channel.ChatEpoch : channel.FilesEpoch)) throw new WebDropException(ErrorCode.Conflict);
if (principal.Role != Role.Owner) if (!ChannelPolicy.IsOwner(principal))
{ {
var access = crypto.Access.Find(a => a.ChannelId == item.ChannelId && a.Username == principal.AccountName); var access = ChannelPolicy.AccessOf(crypto, principal.AccountName, item.ChannelId);
if (access is null && (space.Mode == ChannelMode.Enc || crypto.Access.Count > 0) if (access is null ? space.Mode == ChannelMode.Enc || crypto.Access.Count > 0 : !ChannelPolicy.CanWriteDomain(access, item.Domain)) throw new WebDropException(ErrorCode.Forbidden);
|| access is not null && (item.Domain == CryptoDomain.Chat ? !access.ChatWrite : !access.FilesUpload)) throw new WebDropException(ErrorCode.Forbidden);
} }
} }
public CryptoState ReadCrypto(Id16 id, Credentials creds) => store.Read(tx => public CryptoState ReadCrypto(Id16 id, Credentials creds) => store.Read(tx =>
{ {
var (space, principal) = Authenticate(tx, id, creds, Membership.Required); var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
var state = tx.GetWorkspaceCrypto(id); var state = tx.GetWorkspaceCrypto(id);
if (principal.Role == Role.Owner || principal.AccountName == space.CreatorAccount) return state; if (ChannelPolicy.IsSpaceOwner(space, principal)) return state;
var access = state.Access.Where(a => a.Username == principal.AccountName).ToList(); var access = state.Access.Where(a => a.Username == principal.AccountName).ToList();
var allowed = access.ToDictionary(a => a.ChannelId); var allowed = access.ToDictionary(a => a.ChannelId);
bool CanRead(string channelId, CryptoDomain domain) => allowed.TryGetValue(channelId, out var grant) bool CanRead(string channelId, CryptoDomain domain) => ChannelPolicy.CanReadDomain(allowed.GetValueOrDefault(channelId), domain);
&& (domain == CryptoDomain.Chat ? grant.ChatRead : grant.FilesRead);
var envelopes = state.Envelopes.Where(e => e.Username == principal.AccountName && CanRead(e.ChannelId, e.Domain)).ToList(); var envelopes = state.Envelopes.Where(e => e.Username == principal.AccountName && CanRead(e.ChannelId, e.Domain)).ToList();
var epochs = envelopes.Select(e => (e.ChannelId, e.Domain, e.Epoch)).ToHashSet(); var epochs = envelopes.Select(e => (e.ChannelId, e.Domain, e.Epoch)).ToHashSet();
return new CryptoState { Envelopes = envelopes, return new CryptoState { Envelopes = envelopes,
@@ -183,10 +177,9 @@ public sealed partial class ChannelService
var workspace = tx.GetWorkspace(id) ?? throw new WebDropException(ErrorCode.NotFound); var workspace = tx.GetWorkspace(id) ?? throw new WebDropException(ErrorCode.NotFound);
var channel = workspace.Channels.Find(c => c.Id == req.ChannelId) ?? throw new WebDropException(ErrorCode.NotFound); var channel = workspace.Channels.Find(c => c.Id == req.ChannelId) ?? throw new WebDropException(ErrorCode.NotFound);
var state = tx.GetWorkspaceCrypto(id); var state = tx.GetWorkspaceCrypto(id);
if (principal.Role != Role.Owner && (space.Mode == ChannelMode.Enc || state.Access.Count > 0)) if (!ChannelPolicy.IsOwner(principal) && (space.Mode == ChannelMode.Enc || state.Access.Count > 0))
{ {
var access = state.Access.Find(a => a.ChannelId == req.ChannelId && a.Username == principal.AccountName); if (!ChannelPolicy.CanWriteDomain(ChannelPolicy.AccessOf(state, principal.AccountName, req.ChannelId), req.Domain)) throw new WebDropException(ErrorCode.Forbidden);
if (access is null || req.Domain == CryptoDomain.Chat && !access.ChatWrite || req.Domain == CryptoDomain.Files && !access.FilesUpload) throw new WebDropException(ErrorCode.Forbidden);
} }
if (space.Mode == ChannelMode.Enc && req.Epoch != (req.Domain == CryptoDomain.Chat ? channel.ChatEpoch : channel.FilesEpoch)) throw new WebDropException(ErrorCode.Conflict); if (space.Mode == ChannelMode.Enc && req.Epoch != (req.Domain == CryptoDomain.Chat ? channel.ChatEpoch : channel.FilesEpoch)) throw new WebDropException(ErrorCode.Conflict);
var existing = state.Objects.Find(o => o.ItemId == req.ItemId); var existing = state.Objects.Find(o => o.ItemId == req.ItemId);
@@ -0,0 +1,256 @@
namespace WebDrop.Core;
public sealed partial class ChannelService
{
public void AccountProfileChanged(string username)
{
foreach (var id in store.Read(tx => tx.ListAccountSpaces(username, clock.NowMs))) events.SettingsChanged(id);
}
/// <summary>两个账号是否在同一个空间里(自己总是可以)。成员资料只对有交集的账号开放</summary>
public bool SharesSpace(string username, string other) => username == other
|| store.Read(tx => tx.ListAccountSpaces(username, clock.NowMs).Any(id => tx.GetChannelMember(id, other) is not null));
public IReadOnlyList<Id16> AccountSpaceIds(string username) => store.Read(tx => tx.ListAccountSpaces(username, clock.NowMs));
public IReadOnlyList<ChannelInfo> AccountSpaces(string username) => store.Read(tx => tx.ListAccountSpaces(username, clock.NowMs).Select(id =>
{
var channel = tx.GetChannel(id)!;
var role = tx.GetChannelMember(id, username)?.Role == "owner" ? Role.Owner : channel.WriteHash is not null ? Role.Write : Role.Read;
return ToInfo(channel, tx.CountMembers(id), new Principal(role, username), tx.GetChannelAvatarVersion(id), tx.GetWorkspace(id));
}).ToList());
private static WorkspaceState NewWorkspace(string username, long now)
{
var id = Guid.NewGuid().ToString();
var state = new WorkspaceState { Revision = 1, DefaultChannelId = id, Channels = [new WorkspaceChannel { Id = id, Name = "常规", CreatedAt = now, CreatedBy = username }] };
EnsureDefaultRole(state, now);
return state;
}
/// <summary>补上默认身份组(只补一次)。读取时只在内存里补,下一次写入才落库</summary>
private static void EnsureDefaultRole(WorkspaceState state, long now)
{
if (!state.RolesSeeded)
{
if (!state.Roles.Any(r => r.Id == WorkspaceRole.DefaultId)) state.Roles.Insert(0, WorkspaceRole.CreateDefault(now));
state.RolesSeeded = true;
}
// Owner 组永远排在最前面
if (!state.Roles.Any(r => r.Id == WorkspaceRole.OwnerId)) state.Roles.Insert(0, WorkspaceRole.CreateOwner(now));
}
public WorkspaceState GetWorkspace(Id16 id, Credentials creds) => store.Read(tx =>
{
var (space, principal) = Authenticate(tx, id, creds, Membership.Required);
var state = tx.GetWorkspace(id) ?? new WorkspaceState();
EnsureDefaultRole(state, clock.NowMs);
return ViewFor(tx, space, principal, state);
});
/// <summary>
/// 空间结构按调用者的授权裁剪:没授权的频道、目录和条目位置一律不下发。
/// 读取和所有修改接口的返回值都必须经过它,否则任何一个成员的一次修改(比如发消息后登记条目位置)
/// 都会把整份结构返回给他。
/// </summary>
private WorkspaceState ViewFor(IResourceTx tx, Channel space, Principal principal, WorkspaceState state)
{
if (principal.AccountName != space.CreatorAccount)
{
var crypto = tx.GetWorkspaceCrypto(space.Id);
var grants = crypto.Access.Where(a => a.Username == principal.AccountName).ToList();
var ids = grants.Where(ChannelPolicy.CanSeeChannel).Select(a => a.ChannelId).ToHashSet();
state.Channels.RemoveAll(c => !ids.Contains(c.Id));
state.Folders.RemoveAll(f => !ids.Contains(f.ThreadId));
state.Locations.RemoveAll(l => !Id16.TryParse(l.ItemId, out var itemId) || !CanReadObject(tx, space, principal, itemId));
}
state.Locations = state.Locations.Where(l => Id16.TryParse(l.ItemId, out var itemId) && tx.GetItem(space.Id, itemId) is { } item && item.ExpiresAt > clock.NowMs).ToList();
return state;
}
private WorkspaceState ChangeWorkspace(Id16 id, Credentials creds, Action<IResourceTx, WorkspaceState, Principal> change, bool requireAccount = true, string? permission = null)
{
var state = store.Write(tx =>
{
var (channel, principal) = Authenticate(tx, id, creds, Membership.Required);
var current = tx.GetWorkspace(id) ?? new WorkspaceState();
EnsureDefaultRole(current, clock.NowMs);
// 被授予对应权限的身份组成员可以代行,不要求频道开放成员上传
bool delegated = permission is not null && !ChannelPolicy.IsOwner(principal) && ChannelPolicy.HasPermission(current, principal, permission);
if (requireAccount && !ChannelPolicy.IsOwner(principal) && !delegated) throw new WebDropException(ErrorCode.Forbidden);
if (!delegated && !ChannelPolicy.CanWrite(channel, principal)) throw new WebDropException(ErrorCode.Forbidden);
current.Locations = current.Locations.Where(l => Id16.TryParse(l.ItemId, out var itemId) && tx.GetItem(id, itemId) is not null).ToList();
change(tx, current, principal);
ValidateWorkspace(current);
current.Revision++;
tx.PutWorkspace(id, current);
// 先落库完整的结构,再按调用者裁剪后返回
return ViewFor(tx, channel, principal, current);
});
events.SettingsChanged(id);
return state;
}
/// <summary>文件域的引用只是个不透明的字符串(文件服务给的域 id),主后端不解释它,只存下来、原样给成员</summary>
private static string? FileDomainRef(string? value)
{
if (string.IsNullOrEmpty(value)) return null;
if (value.Length > 64 || !value.All(c => char.IsAsciiLetterOrDigit(c) || c is '-' or '_')) throw new WebDropException(ErrorCode.BadRequest);
return value;
}
public (WorkspaceState State, WorkspaceChannel Channel) CreateWorkspaceChannelResult(Id16 id, Credentials creds, string name, string? groupId, string? fileDomain = null)
{
string? domain = FileDomainRef(fileDomain);
WorkspaceChannel? created = null;
var state = ChangeWorkspace(id, creds, (_, workspace, principal) =>
{
created = new WorkspaceChannel { Id = Guid.NewGuid().ToString(), Name = WorkspaceName(name), GroupId = groupId, CreatedAt = clock.NowMs, CreatedBy = principal.AccountName!, FileDomain = domain };
workspace.Channels.Add(created);
});
return (state, created!);
}
public WorkspaceState CreateWorkspaceChannel(Id16 id, Credentials creds, string name, string? groupId) => CreateWorkspaceChannelResult(id, creds, name, groupId).State;
/// <summary>给还没有文件域的频道指定一个(创建频道时没带,或旧频道)。指定后不能改:文件已经存在那里了</summary>
public WorkspaceState SetWorkspaceChannelFileDomain(Id16 id, Credentials creds, string channelId, string fileDomain) => ChangeWorkspace(id, creds, (_, state, _) =>
{
var channel = FindChannel(state, channelId);
string domain = FileDomainRef(fileDomain) ?? throw new WebDropException(ErrorCode.BadRequest);
if (channel.FileDomain is not null && channel.FileDomain != domain) throw new WebDropException(ErrorCode.Conflict);
channel.FileDomain = domain;
}, permission: WorkspacePermission.ManageStructure);
public WorkspaceState RenameWorkspaceChannel(Id16 id, Credentials creds, string channelId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
FindChannel(state, channelId).Name = WorkspaceName(name), permission: WorkspacePermission.ManageStructure);
public WorkspaceState GroupWorkspaceChannel(Id16 id, Credentials creds, string channelId, string? groupId) => ChangeWorkspace(id, creds, (_, state, _) =>
FindChannel(state, channelId).GroupId = groupId, permission: WorkspacePermission.ManageStructure);
public (WorkspaceState State, WorkspaceGroup Group) CreateWorkspaceGroupResult(Id16 id, Credentials creds, string name)
{
WorkspaceGroup? created = null;
var state = ChangeWorkspace(id, creds, (_, workspace, _) =>
{
created = new WorkspaceGroup { Id = Guid.NewGuid().ToString(), Name = WorkspaceName(name), CreatedAt = clock.NowMs };
workspace.Groups.Add(created);
}, permission: WorkspacePermission.ManageStructure);
return (state, created!);
}
public WorkspaceState CreateWorkspaceGroup(Id16 id, Credentials creds, string name) => CreateWorkspaceGroupResult(id, creds, name).State;
public WorkspaceState RenameWorkspaceGroup(Id16 id, Credentials creds, string groupId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
(state.Groups.Find(g => g.Id == groupId) ?? throw new WebDropException(ErrorCode.NotFound)).Name = WorkspaceName(name), permission: WorkspacePermission.ManageStructure);
/// <summary>删除分组:组里的频道回到未分组,频道本身和内容都不动</summary>
public WorkspaceState DeleteWorkspaceGroup(Id16 id, Credentials creds, string groupId) => ChangeWorkspace(id, creds, (_, state, _) =>
{
var group = state.Groups.Find(g => g.Id == groupId) ?? throw new WebDropException(ErrorCode.NotFound);
foreach (var channel in state.Channels.Where(c => c.GroupId == groupId)) channel.GroupId = null;
state.Groups.Remove(group);
}, permission: WorkspacePermission.ManageStructure);
// ───── 身份组(只有空间主能增删改、分配成员) ─────
private static string? RoleLabel(string? value)
{
value = value?.Trim();
if (string.IsNullOrEmpty(value)) return null;
if (value.Length > WorkspacePermission.MaxLabelLength || value.Any(char.IsControl)) throw new WebDropException(ErrorCode.BadRequest);
return value;
}
private static void ApplyRole(WorkspaceRole role, string name, string? prefix, string? suffix, string? color, IReadOnlyList<string> permissions)
{
if (color is not null && !WorkspacePermission.IsValidColor(color)) throw new WebDropException(ErrorCode.BadRequest);
if (permissions.Any(p => !WorkspacePermission.All.Contains(p))) throw new WebDropException(ErrorCode.BadRequest);
role.Name = WorkspaceName(name); role.Prefix = RoleLabel(prefix); role.Suffix = RoleLabel(suffix); role.Color = color;
role.Permissions = permissions.Distinct().ToList();
}
public (WorkspaceState State, WorkspaceRole Role) CreateWorkspaceRoleResult(Id16 id, Credentials creds, string name, string? prefix, string? suffix, string? color, IReadOnlyList<string> permissions)
{
WorkspaceRole? created = null;
var state = ChangeWorkspace(id, creds, (_, workspace, _) =>
{
created = new WorkspaceRole { Id = Guid.NewGuid().ToString(), Name = "x", CreatedAt = clock.NowMs };
ApplyRole(created, name, prefix, suffix, color, permissions);
workspace.Roles.Add(created);
});
return (state, created!);
}
public WorkspaceState UpdateWorkspaceRole(Id16 id, Credentials creds, string roleId, string name, string? prefix, string? suffix, string? color, IReadOnlyList<string> permissions) =>
ChangeWorkspace(id, creds, (_, state, _) =>
{
var role = FindRole(state, roleId);
// Owner 组的名称和权限固定,只有前后缀和颜色可改
if (role.Id == WorkspaceRole.OwnerId) ApplyRole(role, role.Name, prefix, suffix, color, WorkspacePermission.All);
else ApplyRole(role, name, prefix, suffix, color, permissions);
});
public WorkspaceState DeleteWorkspaceRole(Id16 id, Credentials creds, string roleId) =>
ChangeWorkspace(id, creds, (_, state, _) =>
{
var role = FindRole(state, roleId);
if (role.Id == WorkspaceRole.OwnerId) throw new WebDropException(ErrorCode.BadRequest);
state.Roles.Remove(role);
});
public WorkspaceState SetWorkspaceRoleMembers(Id16 id, Credentials creds, string roleId, IReadOnlyList<string> members) =>
ChangeWorkspace(id, creds, (tx, state, _) =>
{
var role = FindRole(state, roleId);
if (role.Id == WorkspaceRole.OwnerId) throw new WebDropException(ErrorCode.BadRequest);
var distinct = members.Select(m => m.Trim().ToLowerInvariant()).Distinct().ToList();
if (distinct.Count > 256) throw new WebDropException(ErrorCode.BadRequest);
// 只能把已加入空间的账号放进身份组;空间主本来就有全部权限,不必加入
foreach (var member in distinct)
if (tx.GetChannelMember(id, member) is not { } row || row.Role == "owner") throw new WebDropException(ErrorCode.BadRequest);
role.Members = distinct;
});
private static WorkspaceRole FindRole(WorkspaceState state, string id) => state.Roles.Find(r => r.Id == id) ?? throw new WebDropException(ErrorCode.NotFound);
public WorkspaceState CreateWorkspaceFolder(Id16 id, Credentials creds, string threadId, string? parentId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
{
FindChannel(state, threadId);
state.Folders.Add(new WorkspaceFolder { Id = Guid.NewGuid().ToString(), ThreadId = threadId, ParentId = parentId, Name = WorkspaceName(name) });
});
public WorkspaceState RenameWorkspaceFolder(Id16 id, Credentials creds, string folderId, string name) => ChangeWorkspace(id, creds, (_, state, _) =>
(state.Folders.Find(f => f.Id == folderId) ?? throw new WebDropException(ErrorCode.NotFound)).Name = WorkspaceName(name));
public WorkspaceState MoveWorkspaceItems(Id16 id, Credentials creds, string[] itemIds, string threadId, string? folderId) => ChangeWorkspace(id, creds, (tx, state, principal) =>
{
if (itemIds.Length is < 1 or > 100 || itemIds.Distinct().Count() != itemIds.Length) throw new WebDropException(ErrorCode.BadRequest);
// 目的频道必须存在,而且成员只能往自己有写入授权的频道放条目
if (!state.Channels.Exists(c => c.Id == threadId)) throw new WebDropException(ErrorCode.NotFound);
var space = tx.GetChannel(id)!; var crypto = tx.GetWorkspaceCrypto(id);
// 与 CheckObjectWrite 同一口径:加密空间或已有授权记录时,必须有目的频道的写入授权
if (!ChannelPolicy.IsSpaceOwner(space, principal) && ChannelPolicy.GrantsEnforced(space, crypto)
&& !ChannelPolicy.CanWriteAnyDomain(ChannelPolicy.AccessOf(crypto, principal.AccountName, threadId))) throw new WebDropException(ErrorCode.Forbidden);
foreach (var itemId in itemIds)
{
if (!Id16.TryParse(itemId, out var parsed)) throw new WebDropException(ErrorCode.BadRequest);
var item = tx.GetItem(id, parsed) ?? throw new WebDropException(ErrorCode.NotFound);
if ((item.Flags & (FrameFlags.Control | FrameFlags.Snapshot)) != 0) throw new WebDropException(ErrorCode.BadRequest);
if (!ChannelPolicy.IsOwner(principal) && item.AuthorAccount != principal.AccountName) throw new WebDropException(ErrorCode.Forbidden);
state.Locations.RemoveAll(l => l.ItemId == itemId);
state.Locations.Add(new WorkspaceLocation { ItemId = itemId, ThreadId = threadId, FolderId = folderId, AuthorAccount = item.AuthorAccount });
}
}, requireAccount: false);
private static WorkspaceChannel FindChannel(WorkspaceState state, string id) => state.Channels.Find(c => c.Id == id) ?? throw new WebDropException(ErrorCode.NotFound);
private static string WorkspaceName(string name)
{
if (string.IsNullOrWhiteSpace(name) || name.Trim().Length > 80 || name.Any(char.IsControl)) throw new WebDropException(ErrorCode.BadRequest);
return name.Trim();
}
private static void ValidateWorkspace(WorkspaceState state)
{
if (state.Roles is null || state.Roles.Count > 32 || state.Roles.Any(r => r is null || r.Members is null || r.Permissions is null)) throw new WebDropException(ErrorCode.BadRequest);
if (state.Channels is null || state.Groups is null || state.Folders is null || state.Locations is null
|| state.Channels.Count > 256 || state.Groups.Count > 128 || state.Folders.Count > 2048 || state.Locations.Count > 10_000)
throw new WebDropException(ErrorCode.BadRequest);
if (state.Channels.Any(c => c is null) || state.Groups.Any(g => g is null) || state.Folders.Any(f => f is null) || state.Locations.Any(l => l is null)) throw new WebDropException(ErrorCode.BadRequest);
var allIds = state.Channels.Select(c => c.Id).Concat(state.Groups.Select(g => g.Id)).Concat(state.Folders.Select(f => f.Id)).Concat(state.Roles.Select(r => r.Id)).ToArray();
if (allIds.Any(id => !Guid.TryParseExact(id, "D", out _)) || allIds.Distinct().Count() != allIds.Length) throw new WebDropException(ErrorCode.BadRequest);
foreach (var group in state.Groups) group.Name = WorkspaceName(group.Name);
foreach (var channel in state.Channels)
{
channel.Name = WorkspaceName(channel.Name);
if (channel.GroupId is not null && !state.Groups.Any(g => g.Id == channel.GroupId)) throw new WebDropException(ErrorCode.BadRequest);
}
foreach (var folder in state.Folders)
{
folder.Name = WorkspaceName(folder.Name);
_ = FindChannel(state, folder.ThreadId);
var visited = new HashSet<string> { folder.Id };
for (var parentId = folder.ParentId; parentId is not null;)
{
var parent = state.Folders.Find(f => f.Id == parentId && f.ThreadId == folder.ThreadId);
if (parent is null || !visited.Add(parent.Id)) throw new WebDropException(ErrorCode.BadRequest);
parentId = parent.ParentId;
}
}
if (state.Locations.Select(l => l.ItemId).Distinct().Count() != state.Locations.Count) throw new WebDropException(ErrorCode.BadRequest);
foreach (var location in state.Locations)
{
_ = FindChannel(state, location.ThreadId);
if (location.FolderId is not null && !state.Folders.Any(f => f.Id == location.FolderId && f.ThreadId == location.ThreadId)) throw new WebDropException(ErrorCode.BadRequest);
}
}
}
+101
View File
@@ -0,0 +1,101 @@
using System.Text.Json.Serialization;
namespace WebDrop.Core;
public sealed class WorkspaceChannel
{
[JsonPropertyName("chatEpoch")] public int ChatEpoch { get; set; }
[JsonPropertyName("filesEpoch")] public int FilesEpoch { get; set; }
[JsonPropertyName("chatHistory")] public bool ChatHistory { get; set; } = true;
[JsonPropertyName("filesHistory")] public bool FilesHistory { get; set; } = true;
/// <summary>这个频道的文件域在文件服务里的 id。不透明字符串:创建、容量、用量都归文件服务管,这里只是引用。null 表示还没分配</summary>
[JsonPropertyName("fileDomain")] public string? FileDomain { get; set; }
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
[JsonPropertyName("createdAt")] public required long CreatedAt { get; init; }
[JsonPropertyName("createdBy")] public required string CreatedBy { get; init; }
[JsonPropertyName("groupId")] public string? GroupId { get; set; }
}
public sealed class WorkspaceGroup
{
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
[JsonPropertyName("createdAt")] public required long CreatedAt { get; init; }
}
public sealed class WorkspaceFolder
{
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("threadId")] public required string ThreadId { get; init; }
[JsonPropertyName("parentId")] public string? ParentId { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
}
public sealed class WorkspaceLocation
{
[JsonPropertyName("itemId")] public required string ItemId { get; init; }
[JsonPropertyName("threadId")] public required string ThreadId { get; init; }
[JsonPropertyName("folderId")] public string? FolderId { get; init; }
[JsonPropertyName("authorAccount")] public string? AuthorAccount { get; init; }
}
/// <summary>身份组可授予的权限。空间主拥有全部权限,不属于任何身份组</summary>
public static partial class WorkspacePermission
{
/// <summary>移除成员、撤销邀请</summary>
public const string RemoveMembers = "members.remove";
/// <summary>重命名频道、调整频道所属分组、管理分组(新建频道涉及密钥,仍只有空间主)</summary>
public const string ManageStructure = "structure.manage";
public static readonly IReadOnlyList<string> All = [RemoveMembers, ManageStructure];
/// <summary>前后缀(支持行内 Markdown)的长度上限:界面不限制,这里只防止空间数据被撑大</summary>
public const int MaxLabelLength = 2000;
/// <summary>预设名,或自定义:纯色 "#rrggbb"、渐变 "g:#rrggbb,#rrggbb[,#rrggbb]"(2~5 个色标,从左到右);后面可以接动画效果 "|breathe"(呼吸) "|blink"(闪烁) "|marquee"(跑马灯,仅渐变) "|cycle"(色相循环)</summary>
public static bool IsValidColor(string color) => Colors.Contains(color) || CustomColor().IsMatch(color);
[System.Text.RegularExpressions.GeneratedRegex("^(#[0-9a-fA-F]{6}|g:#[0-9a-fA-F]{6}(,#[0-9a-fA-F]{6}){1,4})(\\|(breathe|blink|marquee|cycle))?$")]
private static partial System.Text.RegularExpressions.Regex CustomColor();
/// <summary>早期版本的预设色名,老数据里可能还有;新的界面只写自定义颜色</summary>
public static readonly IReadOnlyList<string> Colors = ["gray", "blue", "green", "amber", "red", "purple", "sunset", "ocean", "aurora", "candy", "gold", "rainbow"];
}
public sealed class WorkspaceRole
{
/// <summary>空间主所在的「Owner 组」,每个空间都有,id 固定。成员就是空间主本人,权限固定为全部,只能改前后缀和颜色</summary>
public const string OwnerId = "00000000-0000-4000-8000-000000000000";
public static WorkspaceRole CreateOwner(long now) => new()
{
Id = OwnerId, Name = "Owner 组", Suffix = "Owner", Color = "g:#d99a1c,#f2c94c,#c27a0a", CreatedAt = now,
Permissions = [.. WorkspacePermission.All],
};
/// <summary>每个空间都有的默认「管理组」,id 固定,老空间第一次读写时补上</summary>
public const string DefaultId = "00000000-0000-4000-8000-000000000001";
public static WorkspaceRole CreateDefault(long now) => new()
{
Id = DefaultId, Name = "管理组", Suffix = "管理员", Color = "#3b6ea8", CreatedAt = now,
Permissions = [.. WorkspacePermission.All],
};
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
/// <summary>聊天里显示在名字前、后的文字(如「频道主」「管理员」),可只填其一</summary>
[JsonPropertyName("prefix")] public string? Prefix { get; set; }
[JsonPropertyName("suffix")] public string? Suffix { get; set; }
[JsonPropertyName("color")] public string? Color { get; set; }
[JsonPropertyName("permissions")] public List<string> Permissions { get; set; } = [];
[JsonPropertyName("members")] public List<string> Members { get; set; } = [];
[JsonPropertyName("createdAt")] public required long CreatedAt { get; init; }
}
public sealed class WorkspaceState
{
[JsonPropertyName("defaultChannelId")] public string? DefaultChannelId { get; init; }
[JsonPropertyName("revision")] public long Revision { get; set; }
[JsonPropertyName("channels")] public List<WorkspaceChannel> Channels { get; init; } = [];
[JsonPropertyName("groups")] public List<WorkspaceGroup> Groups { get; init; } = [];
/// <summary>旧版本存下来的结构里没有 roles,反序列化后是 null,这里统一当作空列表</summary>
[JsonPropertyName("roles")] public List<WorkspaceRole> Roles { get => _roles ??= []; init => _roles = value; }
private List<WorkspaceRole>? _roles;
/// <summary>默认身份组已经放进来过。之后空间主删掉它不会再自动出现</summary>
[JsonPropertyName("rolesSeeded")] public bool RolesSeeded { get; set; }
[JsonPropertyName("folders")] public List<WorkspaceFolder> Folders { get; init; } = [];
[JsonPropertyName("locations")] public List<WorkspaceLocation> Locations { get; set; } = [];
}
[JsonSerializable(typeof(WorkspaceState))]
[JsonSourceGenerationOptions(DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull)]
public sealed partial class WorkspaceJson : JsonSerializerContext;
@@ -30,6 +30,8 @@ public sealed class SignedContentEventRecord
[JsonPropertyName("frameCount")] public required int FrameCount { get; init; } [JsonPropertyName("frameCount")] public required int FrameCount { get; init; }
[JsonPropertyName("manifestHash")] public required string ManifestHash { get; init; } [JsonPropertyName("manifestHash")] public required string ManifestHash { get; init; }
[JsonPropertyName("signerDeviceId")] public required string SignerDeviceId { get; init; } [JsonPropertyName("signerDeviceId")] public required string SignerDeviceId { get; init; }
/// <summary>签名者的账号。空间主撤回别人的消息时,签名者不是作者;旧记录没有这个字段,按作者处理</summary>
[JsonPropertyName("signerAccount")] public string? SignerAccount { get; init; }
[JsonPropertyName("signature")] public required string Signature { get; init; } [JsonPropertyName("signature")] public required string Signature { get; init; }
[JsonPropertyName("targetItemId")] public string? TargetItemId { get; init; } [JsonPropertyName("targetItemId")] public string? TargetItemId { get; init; }
[JsonPropertyName("signerRootPublicKey")] public required string SignerRootPublicKey { get; init; } [JsonPropertyName("signerRootPublicKey")] public required string SignerRootPublicKey { get; init; }
@@ -6,4 +6,8 @@
<Nullable>enable</Nullable> <Nullable>enable</Nullable>
</PropertyGroup> </PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\WebDrop.FileProtocol\WebDrop.FileProtocol.csproj" />
</ItemGroup>
</Project> </Project>
-50
View File
@@ -1,50 +0,0 @@
using System.Text.Json.Serialization;
namespace WebDrop.Core;
public sealed class WorkspaceChannel
{
[JsonPropertyName("chatEpoch")] public int ChatEpoch { get; set; }
[JsonPropertyName("filesEpoch")] public int FilesEpoch { get; set; }
[JsonPropertyName("chatHistory")] public bool ChatHistory { get; set; } = true;
[JsonPropertyName("filesHistory")] public bool FilesHistory { get; set; } = true;
[JsonPropertyName("fileBytes")] public long FileBytes { get; set; }
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
[JsonPropertyName("createdAt")] public required long CreatedAt { get; init; }
[JsonPropertyName("createdBy")] public required string CreatedBy { get; init; }
[JsonPropertyName("groupId")] public string? GroupId { get; set; }
}
public sealed class WorkspaceGroup
{
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
[JsonPropertyName("createdAt")] public required long CreatedAt { get; init; }
}
public sealed class WorkspaceFolder
{
[JsonPropertyName("id")] public required string Id { get; init; }
[JsonPropertyName("threadId")] public required string ThreadId { get; init; }
[JsonPropertyName("parentId")] public string? ParentId { get; init; }
[JsonPropertyName("name")] public required string Name { get; set; }
}
public sealed class WorkspaceLocation
{
[JsonPropertyName("itemId")] public required string ItemId { get; init; }
[JsonPropertyName("threadId")] public required string ThreadId { get; init; }
[JsonPropertyName("folderId")] public string? FolderId { get; init; }
[JsonPropertyName("authorAccount")] public string? AuthorAccount { get; init; }
}
public sealed class WorkspaceState
{
[JsonPropertyName("defaultChannelId")] public string? DefaultChannelId { get; init; }
[JsonPropertyName("revision")] public long Revision { get; set; }
[JsonPropertyName("channels")] public List<WorkspaceChannel> Channels { get; init; } = [];
[JsonPropertyName("groups")] public List<WorkspaceGroup> Groups { get; init; } = [];
[JsonPropertyName("folders")] public List<WorkspaceFolder> Folders { get; init; } = [];
[JsonPropertyName("locations")] public List<WorkspaceLocation> Locations { get; set; } = [];
}
[JsonSerializable(typeof(WorkspaceState))]
[JsonSourceGenerationOptions(DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull)]
public sealed partial class WorkspaceJson : JsonSerializerContext;
+45
View File
@@ -0,0 +1,45 @@
using System.Security.Cryptography;
using System.Text.Json;
using WebDrop.FileProtocol;
namespace WebDrop.FileApi;
/// <summary>
/// 文件域的登记:一个域一个小 JSON 文件({数据目录}/domains/{id}.json),记录它属于哪个空间、分配了多大。
/// 域 id 是文件服务随机生成的,和主后端的任何 id 无关;用了多少字节由 <see cref="FrameStore"/> 按顶层目录统计。
/// </summary>
public sealed class DomainStore
{
private readonly string _root;
public DomainStore(string root)
{
_root = Path.GetFullPath(root);
Directory.CreateDirectory(_root);
}
private string PathOf(string id) => Path.Combine(_root, id + ".json");
public FileDomainBody Create(string space, long quota, long nowMs)
{
var domain = new FileDomainBody(Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16)), space, quota, 0, nowMs);
string tmp = PathOf(domain.Id) + "." + Guid.NewGuid().ToString("N") + ".tmp";
File.WriteAllText(tmp, JsonSerializer.Serialize(domain, FileProtocolJson.Default.FileDomainBody));
File.Move(tmp, PathOf(domain.Id), overwrite: false);
return domain;
}
public FileDomainBody? Get(string id)
{
if (!FileWire.IsHex32(id)) return null;
try { return JsonSerializer.Deserialize(File.ReadAllText(PathOf(id)), FileProtocolJson.Default.FileDomainBody); }
catch (Exception e) when (e is IOException or JsonException) { return null; }
}
public bool Delete(string id)
{
if (!FileWire.IsHex32(id) || !File.Exists(PathOf(id))) return false;
File.Delete(PathOf(id));
return true;
}
}
+177
View File
@@ -0,0 +1,177 @@
using System.Security.Cryptography;
using System.Text;
using WebDrop.FileProtocol;
namespace WebDrop.FileApi;
public static class FileApiApp
{
public static WebApplication Build(string[] args, Action<WebApplicationBuilder>? configure = null)
{
var builder = WebApplication.CreateSlimBuilder(args);
configure?.Invoke(builder);
var options = builder.Configuration.GetSection("FileApi").Get<FileApiOptions>() ?? new FileApiOptions();
if (options.Secret.Length < 32) throw new InvalidOperationException("FileApi:Secret 至少 32 个字符(和主后端的 WebDrop:Files:Secret 一致)");
if (Array.IndexOf(options.EffectiveChoices, options.DefaultQuota) < 0) throw new InvalidOperationException("FileApi:DefaultQuota 必须是 FileApi:QuotaChoices 里的一档");
builder.WebHost.ConfigureKestrel(k =>
{
k.AddServerHeader = false;
k.Limits.MaxRequestBodySize = options.MaxFrameBytes;
});
builder.Services.AddSingleton(options);
builder.Services.AddSingleton(new FrameStore(Path.Combine(options.DataDir, "frames"), options.MaxStoreBytes, Path.Combine(options.DataDir, "sync")));
var domains = new DomainStore(Path.Combine(options.DataDir, "domains"));
builder.Services.AddSingleton(domains);
builder.Services.AddHostedService<PendingSweeper>();
builder.Services.AddSingleton<SyncWorker>();
builder.Services.AddHostedService(sp => sp.GetRequiredService<SyncWorker>());
if (options.CorsOrigins.Length > 0)
{
builder.Services.AddCors(c => c.AddDefaultPolicy(p => p
.WithOrigins(options.CorsOrigins)
.WithMethods("GET", "PUT", "POST", "DELETE", "OPTIONS")
.WithHeaders(FileWire.TicketHeader, "Content-Type")
.WithExposedHeaders("Retry-After")
.SetPreflightMaxAge(TimeSpan.FromHours(1))));
}
var app = builder.Build();
app.Use(async (ctx, next) =>
{
var h = ctx.Response.Headers;
h["Referrer-Policy"] = "no-referrer";
h["X-Content-Type-Options"] = "nosniff";
h["Cache-Control"] = "no-store";
h["Content-Security-Policy"] = "default-src 'none'; frame-ancestors 'none'";
try { await next(ctx); }
catch (OperationCanceledException) when (ctx.RequestAborted.IsCancellationRequested) { }
catch (BadHttpRequestException e)
{
if (!ctx.Response.HasStarted) { ctx.Response.StatusCode = e.StatusCode; await ctx.Response.WriteAsync("{\"error\":\"bad_request\"}"); }
}
});
if (options.CorsOrigins.Length > 0) app.UseCors();
Map(app, options, app.Services.GetRequiredService<FrameStore>(), domains);
return app;
}
private static void Map(WebApplication app, FileApiOptions options, FrameStore store, DomainStore domains)
{
byte[] secret = Encoding.UTF8.GetBytes(options.Secret);
// 校验票据,规则只在这里:文件域、条目必须和路径一致;票据必须是这个域所属的空间签发的;
// 读取、删除的票据只能用一次(下载次数在主后端签票据时计,重放同一张票据等于绕过计次);管理票据(运维用)对任何条目有效
var usedTickets = new UsedTickets();
FileTicket? Authorize(HttpContext ctx, FileOp op, string channel, string? item)
{
if (!FileTokens.TryVerify(secret, ctx.Request.Headers[FileWire.TicketHeader].ToString(), DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(), out FileTicket ticket)) return null;
if (ticket.Op == FileOp.Admin) return ticket;
if (!(ticket.Op == op && ticket.Channel == channel && (item is null || ticket.Item == item))) return null;
// 票据是哪个空间签的,必须就是域所属的那个空间(别的空间即使拿到域 id 也用不了)
if (domains.Get(channel) is not { } domain || domain.Space != ticket.Space) return null;
if (op is FileOp.Get or FileOp.Delete && !usedTickets.TryUse(ticket.Jti, ticket.ExpiresMs)) return null;
return ticket;
}
IResult Error(int status, string code) => Results.Text($"{{\"error\":\"{code}\"}}", "application/json", Encoding.UTF8, status);
app.MapGet("/healthz", () => Results.Text("ok"));
// 文件域的选项(创建时可选的容量档位):不含任何机密,谁都能读,前端创建文件域前来看
app.MapGet("/file/v1/options", () => Results.Text(System.Text.Json.JsonSerializer.Serialize(options.Domain, FileProtocolJson.Default.FileDomainOptions), "application/json"));
app.MapGet("/file/v1/info", (HttpContext ctx) =>
FileTokens.TryVerify(secret, ctx.Request.Headers[FileWire.TicketHeader].ToString(), DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(), out FileTicket t) && t.Op == FileOp.Admin
? Results.Text($"{{\"used_bytes\":{store.UsedBytes},\"free_bytes\":{store.FreeBytes}}}", "application/json")
: Error(401, "unauthorized"));
// 存一帧:请求体是完整的帧封装。域必须存在,写入受域的容量限制。返回这一帧的摘要,客户端拿它去主后端登记
app.MapPut("/file/v1/domains/{id}/items/{item}/frames/{index:long}", async (string id, string item, long index, HttpContext ctx, CancellationToken ct) =>
{
if (!FileWire.IsHex32(id) || !FileWire.IsHex32(item) || index is < 0 or > uint.MaxValue) return Error(400, "bad_request");
if (Authorize(ctx, FileOp.Put, id, item) is null) return Error(401, "unauthorized");
if (domains.Get(id) is not { } domain) return Error(404, "not_found");
if (ctx.Request.ContentLength is not { } length || length < FrameHeader.Size || length > options.MaxFrameBytes) return Error(413, "too_large");
byte[] frame = new byte[length];
await ctx.Request.Body.ReadExactlyAsync(frame, ct).ConfigureAwait(false);
if (!FrameHeader.TryParse(frame, out var header) || header.TotalLength != frame.Length || header.ItemHex != item || header.Index != index)
return Error(400, "bad_request");
byte[] digest = SHA256.HashData(frame);
switch (await store.PutAsync(id, item, (uint)index, frame, digest, ct, domain.Quota).ConfigureAwait(false))
{
case PutResult.Conflict: return Error(409, "conflict");
case PutResult.Full: return Error(507, "store_full");
case PutResult.OverQuota: return Error(507, "quota_exceeded");
}
return Results.Text($"{{\"digest\":\"{Convert.ToHexStringLower(digest)}\",\"length\":{frame.Length}}}", "application/json", Encoding.UTF8, 200);
});
// 读取:自 from 起把连续的帧原样顺序写出
app.MapGet("/file/v1/domains/{id}/items/{item}/frames", async (string id, string item, HttpContext ctx, CancellationToken ct) =>
{
if (!FileWire.IsHex32(id) || !FileWire.IsHex32(item)) return Error(400, "bad_request");
var ticket = Authorize(ctx, FileOp.Get, id, item);
if (ticket is null) return Error(401, "unauthorized");
if (!uint.TryParse(ctx.Request.Query["from"], out uint from)) from = ticket.From;
if (ticket.Op != FileOp.Admin && from < ticket.From) return Error(403, "forbidden");
ctx.Response.ContentType = "application/octet-stream";
bool any = false;
foreach (var path in store.FramesFrom(id, item, from))
{
any = true;
await ctx.Response.SendFileAsync(path, ct).ConfigureAwait(false);
}
return any ? Results.Empty : Error(404, "not_found");
});
// 删一个条目的所有帧:删除票据(主后端确认调用者有权删除之后签发),或者上传这个条目的票据(取消上传时清掉自己传的字节)
app.MapDelete("/file/v1/domains/{id}/items/{item}", (string id, string item, HttpContext ctx) =>
{
if (!FileWire.IsHex32(id) || !FileWire.IsHex32(item)) return Error(400, "bad_request");
if (Authorize(ctx, FileOp.Delete, id, item) is null && Authorize(ctx, FileOp.Put, id, item) is null) return Error(401, "unauthorized");
return store.DeletePrefix($"{id}/{item}") ? Results.NoContent() : Error(400, "bad_request");
});
// 确认一个条目:整个文件已经登记到主后端。持上传这个条目的票据
app.MapPost("/file/v1/domains/{id}/items/{item}/commit", (string id, string item, HttpContext ctx) =>
{
if (!FileWire.IsHex32(id) || !FileWire.IsHex32(item)) return Error(400, "bad_request");
if (Authorize(ctx, FileOp.Put, id, item) is null) return Error(401, "unauthorized");
return store.Commit(id, item) ? Results.NoContent() : Error(404, "not_found");
});
// ───── 文件域:创建、查询、删除。前端带着空间签发的票据直接来这里
// 创建:票据必须是 Create(空间的所有者/管理员向主后端申请的),Channel 字段是空间 id
app.MapPost("/file/v1/domains", async (HttpContext ctx, CancellationToken ct) =>
{
if (!FileTokens.TryVerify(secret, ctx.Request.Headers[FileWire.TicketHeader].ToString(), DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(), out FileTicket ticket)
|| ticket.Op is not (FileOp.Create or FileOp.Admin)) return Error(401, "unauthorized");
if (!FileWire.IsHex32(ticket.Channel)) return Error(400, "bad_request");
var request = ctx.Request.ContentLength is 0 ? new FileDomainCreate() : await ctx.Request.ReadFromJsonAsync(FileProtocolJson.Default.FileDomainCreate, ct).ConfigureAwait(false) ?? new FileDomainCreate();
long quota = request.Quota == 0 ? options.DefaultQuota : request.Quota;
if (Array.IndexOf(options.EffectiveChoices, quota) < 0) return Error(400, "bad_quota");
var created = domains.Create(ticket.Channel, quota, DateTimeOffset.UtcNow.ToUnixTimeMilliseconds());
return Results.Text(System.Text.Json.JsonSerializer.Serialize(created, FileProtocolJson.Default.FileDomainBody), "application/json", Encoding.UTF8, 201);
});
// 查询:持有这个域的读、写或管理票据都可以
app.MapGet("/file/v1/domains/{id}", (string id, HttpContext ctx) =>
{
if (!FileWire.IsHex32(id)) return Error(400, "bad_request");
if (Authorize(ctx, FileOp.Get, id, null) is null && Authorize(ctx, FileOp.Put, id, null) is null) return Error(401, "unauthorized");
if (domains.Get(id) is not { } domain) return Error(404, "not_found");
return Results.Text(System.Text.Json.JsonSerializer.Serialize(domain with { Used = store.UsedBy(id) }, FileProtocolJson.Default.FileDomainBody), "application/json");
});
// 删除域和里面所有文件:只有管理票据(运维用)
app.MapDelete("/file/v1/domains/{id}", (string id, HttpContext ctx) =>
{
if (!FileWire.IsHex32(id)) return Error(400, "bad_request");
if (Authorize(ctx, FileOp.Admin, "*", null) is null) return Error(401, "unauthorized");
if (!domains.Delete(id)) return Error(404, "not_found");
store.DeletePrefix(id);
return Results.NoContent();
});
}
}
@@ -0,0 +1,38 @@
using WebDrop.FileProtocol;
using WebDrop.Sync;
namespace WebDrop.FileApi;
/// <summary>配置节 "FileApi"。环境变量形式:FileApi__DataDir、FileApi__Secret ……</summary>
public sealed class FileApiOptions
{
public string DataDir { get; set; } = "files-data";
/// <summary>和主后端共用的密钥(主后端的 WebDrop:Files:Secret),至少 32 个字符</summary>
public string Secret { get; set; } = "";
/// <summary>前端所在的源;浏览器直接把文件传给这里,必须列出,否则不发 CORS 头</summary>
public string[] CorsOrigins { get; set; } = [];
/// <summary>创建频道时可以分配的文件空间档位(字节);不配置用内置的 1/5/10/50/100 GiB。默认值不放在数组里:配置绑定会把数组按下标合并进默认值</summary>
public long[] QuotaChoices { get; set; } = [];
/// <summary>创建频道时不选择的默认文件空间,必须是档位之一</summary>
public long DefaultQuota { get; set; } = 10L << 30;
public FileDomainOptions Domain => new(EffectiveChoices, DefaultQuota);
public long[] EffectiveChoices => QuotaChoices.Length == 0 ? FileDomainOptions.Fallback.QuotaChoices : QuotaChoices.Where(n => n > 0).Distinct().Order().ToArray();
/// <summary>上传了字节、却一直没有被确认(没有登记到主后端)的条目,过多少小时删掉。默认 24</summary>
public int PendingRetentionHours { get; set; } = 24;
/// <summary>磁盘上最多存这么多字节,0 表示不限。满了拒绝新的帧(507),已有的不受影响</summary>
public long MaxStoreBytes { get; set; }
/// <summary>一帧的最大字节数:26 字节头部 + 最大负载(1 MiB + 16 字节标签)</summary>
public int MaxFrameBytes { get; set; } = 26 + (1 << 20) + 16;
/// <summary>每天把新增的文件帧同步到外部存储(S3 协议)。已删除的内容在备份里再留 RetainDays 天</summary>
public DailyBackupOptions Backup { get; set; } = new() { Time = "04:30" };
}
+233
View File
@@ -0,0 +1,233 @@
using System.Security.Cryptography;
using WebDrop.FileProtocol;
namespace WebDrop.FileApi;
public enum PutResult { Created, Existing, Conflict, Full, OverQuota }
/// <summary>
/// 文件帧的磁盘存储:{根}/{频道}/{条目}/{帧序号}。一帧一个文件,先写临时文件再改名,读者看不到半个文件。
/// 同一个序号只能存一份内容:再传同样的内容是幂等的,不同的内容拒绝(帧一旦存下就不可变)。
/// </summary>
public sealed class FrameStore
{
private readonly string _root;
private readonly string _syncRoot;
private readonly long _maxBytes;
private long _used;
/// <param name="syncRoot">同步状态:每个已经同步到外部存储的帧在这里有一个空的标记文件,不放在帧目录里(那里的内容就是帧本身)</param>
public FrameStore(string root, long maxBytes, string? syncRoot = null)
{
_root = Path.GetFullPath(root);
_syncRoot = Path.GetFullPath(syncRoot ?? _root + ".sync");
_maxBytes = maxBytes;
Directory.CreateDirectory(_root);
Directory.CreateDirectory(_syncRoot);
_used = Directory.EnumerateFiles(_root, "*", SearchOption.AllDirectories).Where(p => !p.EndsWith(".tmp", StringComparison.Ordinal)).Sum(p => new FileInfo(p).Length);
}
public long UsedBytes => Interlocked.Read(ref _used);
// 每个顶层目录(文件域)已用的字节数,第一次用到时扫一遍磁盘,之后随写入、删除增减
private readonly System.Collections.Concurrent.ConcurrentDictionary<string, long[]> _usedBy = new();
private long[] CounterOf(string channelHex) => _usedBy.GetOrAdd(channelHex, key =>
{
string dir = Path.Combine(_root, key);
return [Directory.Exists(dir) ? Directory.EnumerateFiles(dir, "*", SearchOption.AllDirectories).Where(p => !p.EndsWith(".tmp", StringComparison.Ordinal)).Sum(p => new FileInfo(p).Length) : 0];
});
/// <summary>一个文件域(顶层目录)已经存了多少字节</summary>
public long UsedBy(string channelHex) => Interlocked.Read(ref CounterOf(channelHex)[0]);
public long FreeBytes
{
get
{
try { return new DriveInfo(Path.GetPathRoot(_root)!).AvailableFreeSpace; }
catch (Exception) { return -1; }
}
}
/// <summary>
/// 新条目第一帧落盘时,在条目目录里放一个「待确认」标记。客户端把整个文件登记到主后端之后再来确认(Commit),标记才消失;
/// 一直没确认的(上传到一半放弃、浏览器关了)过了保留期由清理任务删掉,不会永远占着文件域的容量。
/// 在这个机制之前存下的条目没有标记,永远不会被清理。
/// </summary>
private const string PendingMarker = ".pending";
private string PathOf(string channelHex, string itemHex, uint index) => Path.Combine(_root, channelHex, itemHex, index.ToString("D10"));
/// <param name="quota">这个顶层目录(文件域)最多存多少字节,0 表示不限</param>
public async Task<PutResult> PutAsync(string channelHex, string itemHex, uint index, ReadOnlyMemory<byte> frame, byte[] digest, CancellationToken ct, long quota = 0)
{
string path = PathOf(channelHex, itemHex, index);
if (File.Exists(path)) return SameContent(path, digest) ? PutResult.Existing : PutResult.Conflict;
if (_maxBytes > 0 && UsedBytes + frame.Length > _maxBytes) return PutResult.Full;
if (quota > 0 && UsedBy(channelHex) + frame.Length > quota) return PutResult.OverQuota;
string itemDir = Path.GetDirectoryName(path)!;
bool fresh = !Directory.Exists(itemDir);
Directory.CreateDirectory(itemDir);
if (fresh) File.WriteAllBytes(Path.Combine(itemDir, PendingMarker), []);
string tmp = path + "." + Guid.NewGuid().ToString("N") + ".tmp";
await using (var fs = new FileStream(tmp, FileMode.CreateNew, FileAccess.Write, FileShare.None, 4096, FileOptions.Asynchronous))
await fs.WriteAsync(frame, ct).ConfigureAwait(false);
try
{
File.Move(tmp, path, overwrite: false);
}
catch (Exception e) when (e is IOException or UnauthorizedAccessException && File.Exists(path))
{
File.Delete(tmp); // 并发写入同一帧,对方先到了
return SameContent(path, digest) ? PutResult.Existing : PutResult.Conflict;
}
Interlocked.Add(ref _used, frame.Length);
Interlocked.Add(ref CounterOf(channelHex)[0], frame.Length);
return PutResult.Created;
}
private static bool SameContent(string path, byte[] digest)
{
using var fs = File.OpenRead(path);
return CryptographicOperations.FixedTimeEquals(SHA256.HashData(fs), digest);
}
/// <summary>确认一个条目:已经登记到主后端,不再是待确认。条目不存在返回 false;没有标记(旧条目)也算成功</summary>
public bool Commit(string channelHex, string itemHex)
{
string dir = Path.Combine(_root, channelHex, itemHex);
if (!Directory.Exists(dir)) return false;
File.Delete(Path.Combine(dir, PendingMarker));
return true;
}
/// <summary>一直待确认、最后一次写入早于 olderThan 的条目(频道/条目)</summary>
public IEnumerable<string> StalePending(TimeSpan olderThan)
{
var cutoff = DateTime.UtcNow - olderThan;
foreach (var channelDir in Directory.EnumerateDirectories(_root))
{
foreach (var itemDir in Directory.EnumerateDirectories(channelDir))
{
if (!File.Exists(Path.Combine(itemDir, PendingMarker))) continue;
var newest = Directory.EnumerateFiles(itemDir).Select(File.GetLastWriteTimeUtc).DefaultIfEmpty(DateTime.MinValue).Max();
if (newest < cutoff) yield return $"{Path.GetFileName(channelDir)}/{Path.GetFileName(itemDir)}";
}
}
}
/// <summary>自 from 起连续存在的帧(遇到缺口就停)。调用方负责按需读取,这里只给文件路径。</summary>
public IEnumerable<string> FramesFrom(string channelHex, string itemHex, uint from)
{
for (uint index = from; ; index++)
{
string path = PathOf(channelHex, itemHex, index);
if (!File.Exists(path)) yield break;
yield return path;
if (index == uint.MaxValue) yield break;
}
}
/// <summary>前缀是 {频道} 或 {频道}/{条目}</summary>
public bool DeletePrefix(string prefix)
{
var parts = prefix.Split('/');
if (parts.Length is < 1 or > 2 || parts.Any(p => !FileWire.IsHex32(p))) return false;
string dir = Path.Combine(_root, Path.Combine(parts));
if (!Directory.Exists(dir)) return true;
// 这个域的用量计数第一次用到时要扫磁盘:必须在删除之前扫,否则删掉的字节会被先排除、再被减一次
var counter = CounterOf(parts[0]);
long size = Directory.EnumerateFiles(dir, "*", SearchOption.AllDirectories).Where(p => !p.EndsWith(".tmp", StringComparison.Ordinal)).Sum(p => new FileInfo(p).Length);
Directory.Delete(dir, recursive: true);
Interlocked.Add(ref _used, -size);
if (parts.Length == 1) _usedBy.TryRemove(parts[0], out _); else Interlocked.Add(ref counter[0], -size);
ForgetSynced(prefix, parts);
if (parts.Length == 2) TryRemoveEmpty(Path.Combine(_root, parts[0]));
return true;
}
private string TombstoneLog => Path.Combine(_syncRoot, "deleted.log");
private readonly object _syncGate = new();
/// <summary>删除了的内容,如果已经同步到外部存储,记下删除时间,过了保留期再从外部存储清理</summary>
private void ForgetSynced(string prefix, string[] parts)
{
string markers = Path.Combine(_syncRoot, Path.Combine(parts));
if (!Directory.Exists(markers)) return;
lock (_syncGate)
{
File.AppendAllText(TombstoneLog, $"{prefix}\t{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}\n");
Directory.Delete(markers, recursive: true);
}
}
/// <summary>还没同步到外部存储的帧:(外部存储里的相对路径, 本地文件路径)</summary>
public IEnumerable<(string Key, string Path)> Unsynced()
{
foreach (var channelDir in Directory.EnumerateDirectories(_root))
{
string channel = Path.GetFileName(channelDir);
foreach (var itemDir in Directory.EnumerateDirectories(channelDir))
{
string item = Path.GetFileName(itemDir);
if (File.Exists(Path.Combine(itemDir, PendingMarker))) continue; // 还没确认的不同步
foreach (var file in Directory.EnumerateFiles(itemDir).Order(StringComparer.Ordinal))
{
if (file.EndsWith(".tmp", StringComparison.Ordinal) || Path.GetFileName(file).StartsWith('.')) continue;
string key = $"{channel}/{item}/{Path.GetFileName(file)}";
if (!File.Exists(Path.Combine(_syncRoot, channel, item, Path.GetFileName(file)))) yield return (key, file);
}
}
}
}
public void MarkSynced(string key)
{
string marker = Path.Combine(_syncRoot, key.Replace('/', Path.DirectorySeparatorChar));
Directory.CreateDirectory(Path.GetDirectoryName(marker)!);
File.WriteAllBytes(marker, []);
}
/// <summary>保留期已过的删除记录。返回前缀;处理完调用 <see cref="RemoveTombstones"/></summary>
public IReadOnlyList<string> ExpiredTombstones(TimeSpan retain)
{
lock (_syncGate)
{
if (!File.Exists(TombstoneLog)) return [];
long cutoff = DateTimeOffset.UtcNow.Add(-retain).ToUnixTimeMilliseconds();
return File.ReadAllLines(TombstoneLog).Select(l => l.Split('\t')).Where(p => p.Length == 2 && long.TryParse(p[1], out var at) && at <= cutoff).Select(p => p[0]).ToList();
}
}
public void RemoveTombstones(IEnumerable<string> prefixes)
{
var done = prefixes.ToHashSet();
lock (_syncGate)
{
if (!File.Exists(TombstoneLog)) return;
File.WriteAllLines(TombstoneLog, File.ReadAllLines(TombstoneLog).Where(l => !done.Contains(l.Split('\t')[0])));
}
}
private static void TryRemoveEmpty(string dir)
{
try { if (Directory.Exists(dir) && !Directory.EnumerateFileSystemEntries(dir).Any()) Directory.Delete(dir); }
catch (IOException) { /* 并发写入时目录又有内容了 */ }
}
/// <summary>最后一次写入早于 olderThan 的条目(频道/条目)。对账用,正在上传的条目不会出现在里面</summary>
public IEnumerable<string> ItemsOlderThan(TimeSpan olderThan)
{
var cutoff = DateTime.UtcNow - olderThan;
foreach (var channelDir in Directory.EnumerateDirectories(_root))
{
foreach (var itemDir in Directory.EnumerateDirectories(channelDir))
{
var newest = Directory.EnumerateFiles(itemDir).Select(File.GetLastWriteTimeUtc).DefaultIfEmpty(DateTime.MinValue).Max();
if (newest < cutoff) yield return $"{Path.GetFileName(channelDir)}/{Path.GetFileName(itemDir)}";
}
}
}
}
@@ -0,0 +1,28 @@
namespace WebDrop.FileApi;
/// <summary>
/// 清理一直没被确认的条目:字节传进来了,却没有登记到主后端(上传到一半放弃、关掉了浏览器)。
/// 它们会占着文件域的容量,过了保留期就删。已确认的条目、以及没有待确认标记的旧条目不受影响。
/// </summary>
public sealed class PendingSweeper(FileApiOptions options, FrameStore store, ILogger<PendingSweeper> logger) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
using var timer = new PeriodicTimer(TimeSpan.FromMinutes(30));
do
{
try { SweepOnce(); }
catch (Exception e) when (e is IOException or UnauthorizedAccessException) { logger.LogWarning("清理待确认条目失败:{Type}", e.GetType().Name); }
}
while (await timer.WaitForNextTickAsync(stoppingToken).ConfigureAwait(false));
}
public int SweepOnce()
{
int removed = 0;
foreach (var prefix in store.StalePending(TimeSpan.FromHours(Math.Max(1, options.PendingRetentionHours))).ToList())
if (store.DeletePrefix(prefix)) removed++;
if (removed > 0) logger.LogInformation("清理了 {Count} 个一直没有确认的上传", removed);
return removed;
}
}
+4
View File
@@ -0,0 +1,4 @@
using WebDrop.FileApi;
var app = FileApiApp.Build(args);
app.Run();
+70
View File
@@ -0,0 +1,70 @@
using WebDrop.FileProtocol;
using WebDrop.Sync;
namespace WebDrop.FileApi;
public sealed record SyncSummary(int Uploaded, long UploadedBytes, int Failed, int Purged);
/// <summary>
/// 每天定时把新增的文件帧同步到外部存储(S3 协议);删除了的内容在外部存储里再留 RetainDays 天才清理。
/// 帧的文件名里就是 频道/条目/序号,一帧一个远端文件,重复上传同样内容是幂等的,中途失败下次接着传。
/// </summary>
public sealed class SyncWorker(FileApiOptions options, FrameStore store, ILogger<SyncWorker> logger) : BackgroundService
{
private readonly HttpClient _http = new() { Timeout = TimeSpan.FromMinutes(30) };
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var backup = options.Backup;
if (!backup.Enabled || !backup.S3.Configured) return;
while (!stoppingToken.IsCancellationRequested)
{
var wait = DailySchedule.Next(DateTime.Now, backup.TimeOfDay) - DateTime.Now;
try { await Task.Delay(wait, stoppingToken).ConfigureAwait(false); } catch (OperationCanceledException) { break; }
try { await RunOnceAsync(stoppingToken).ConfigureAwait(false); }
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested) { break; }
catch (Exception e) { logger.LogWarning("文件同步失败:{Type}", e.GetType().Name); }
}
}
public async Task<SyncSummary> RunOnceAsync(CancellationToken ct)
{
var backup = options.Backup;
var s3 = new S3Client(_http, backup.S3);
var throttle = new Throttle(backup.MaxBytesPerSecond);
int uploaded = 0, failed = 0, purged = 0; long bytes = 0;
foreach (var (key, path) in store.Unsynced())
{
ct.ThrowIfCancellationRequested();
var started = DateTime.UtcNow;
try
{
await using var file = File.OpenRead(path);
long length = file.Length; // 上传结束后流已经被释放
await s3.PutAsync($"files/{key}", file, ct).ConfigureAwait(false);
store.MarkSynced(key);
uploaded++; bytes += length;
await throttle.AfterAsync(length, DateTime.UtcNow - started, ct).ConfigureAwait(false);
}
catch (Exception e) when (e is HttpRequestException or IOException)
{
// 外部存储连不上或这一帧读不了(正好被删):记一笔,连续失败几次就等下一次再来
if (++failed >= 5) break;
}
}
// 保留期已过的删除记录:从外部存储清掉
var expired = store.ExpiredTombstones(TimeSpan.FromDays(Math.Max(0, backup.RetainDays)));
var cleaned = new List<string>();
foreach (var prefix in expired)
{
try { await s3.DeleteAsync($"files/{prefix}", ct).ConfigureAwait(false); cleaned.Add(prefix); purged++; }
catch (HttpRequestException) { break; }
}
store.RemoveTombstones(cleaned);
if (uploaded + purged + failed > 0) logger.LogInformation("文件同步:上传 {Uploaded} 帧({Bytes} 字节),失败 {Failed},清理 {Purged}", uploaded, bytes, failed, purged);
return new SyncSummary(uploaded, bytes, failed, purged);
}
}
+26
View File
@@ -0,0 +1,26 @@
using System.Collections.Concurrent;
namespace WebDrop.FileApi;
/// <summary>
/// 记下用过的一次性票据(按票据编号),过期后随清理丢掉。放在内存里:重启后在票据有效期内理论上能重放一次,
/// 但票据有效期只有几分钟,而且下载次数的计数在主后端,代价可以接受。
/// </summary>
public sealed class UsedTickets
{
private readonly ConcurrentDictionary<string, long> _used = new();
private long _lastPurge;
/// <summary>第一次用返回 true;同一个编号再用返回 false。编号为空的票据(旧格式)不被接受</summary>
public bool TryUse(string jti, long expiresMs)
{
if (jti.Length == 0) return false;
long now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
if (now - Interlocked.Read(ref _lastPurge) > 60_000)
{
Interlocked.Exchange(ref _lastPurge, now);
foreach (var (key, expiry) in _used) if (expiry < now) _used.TryRemove(key, out _);
}
return _used.TryAdd(jti, expiresMs);
}
}
@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<!-- 文件服务:只保存文件帧的密文,按票据收发。部署在家宽,和主后端(WebDrop.Api)分开,只共用 WebDrop.FileProtocol -->
<PropertyGroup>
<PublishAot>true</PublishAot>
<StripSymbols>true</StripSymbols>
<EnableConfigurationBindingGenerator>true</EnableConfigurationBindingGenerator>
<JsonSerializerIsReflectionEnabledByDefault>false</JsonSerializerIsReflectionEnabledByDefault>
<AssemblyName>webdrop-files</AssemblyName>
<RootNamespace>WebDrop.FileApi</RootNamespace>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\WebDrop.FileProtocol\WebDrop.FileProtocol.csproj" />
<ProjectReference Include="..\WebDrop.Sync\WebDrop.Sync.csproj" />
</ItemGroup>
</Project>
@@ -0,0 +1,16 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"Microsoft.AspNetCore.Hosting.Diagnostics": "None"
}
},
"FileApi": {
"DataDir": "files-data",
"Secret": "",
"CorsOrigins": [],
"MaxStoreBytes": 0,
"MaxFrameBytes": 1048642
}
}
@@ -0,0 +1,29 @@
using System.Text.Json.Serialization;
namespace WebDrop.FileProtocol;
/// <summary>
/// 文件域的选项,由文件服务(fileAPI)决定并对外公布:创建频道时能分配多大的文件空间。
/// </summary>
public sealed record FileDomainOptions(
[property: JsonPropertyName("quota_choices")] long[] QuotaChoices,
[property: JsonPropertyName("default_quota")] long DefaultQuota)
{
/// <summary>没有配置时的内置档位(频道默认 10 GiB)</summary>
public static readonly FileDomainOptions Fallback = new([1L << 30, 5L << 30, 10L << 30, 50L << 30, 100L << 30], 10L << 30);
}
/// <summary>文件域:创建频道时分配的文件空间。由文件服务创建和管理,quota 是容量,used 是已用字节</summary>
public sealed record FileDomainBody(
[property: JsonPropertyName("id")] string Id,
[property: JsonPropertyName("space")] string Space,
[property: JsonPropertyName("quota")] long Quota,
[property: JsonPropertyName("used")] long Used,
[property: JsonPropertyName("created_at")] long CreatedAt);
public sealed record FileDomainCreate([property: JsonPropertyName("quota")] long Quota = 0);
[JsonSerializable(typeof(FileDomainOptions))]
[JsonSerializable(typeof(FileDomainBody))]
[JsonSerializable(typeof(FileDomainCreate))]
public sealed partial class FileProtocolJson : JsonSerializerContext;
@@ -0,0 +1,76 @@
using System.Security.Cryptography;
using System.Text;
namespace WebDrop.FileProtocol;
/// <summary>文件服务(fileAPI)的三种凭据用途</summary>
public enum FileOp : byte
{
/// <summary>上传某个条目的帧</summary>
Put = 1,
/// <summary>读取某个条目自某一帧起的内容</summary>
Get = 2,
/// <summary>主后端对文件服务的管理调用(删除、对账、复制)</summary>
Admin = 3,
/// <summary>在某个空间名下创建文件域(频道的文件空间)。由空间的所有者或管理员向主后端申请</summary>
Create = 4,
/// <summary>删除某个条目的所有帧。主后端确认调用者有权删除之后签发</summary>
Delete = 5,
}
/// <summary>
/// 主后端签发、文件服务校验的短期票据:允许对哪个文件域(Channel)的哪个条目做什么,过期即失效。
/// Space 是签发这张票据的空间:文件服务只认域所属的空间签发的票据,别的空间即使知道域 id 也用不了。
/// Jti 是票据的唯一编号:读取和删除的票据只能用一次,重放无效(每次读取都向空间要新的,计次才有意义)。
/// </summary>
public sealed record FileTicket(FileOp Op, string Channel, string Item, uint From, long ExpiresMs, string Space = "", string Jti = "");
/// <summary>
/// 票据:ASCII 载荷 + HMAC-SHA256,双方共享同一个密钥。
/// 文件服务本来就保管密文,密钥泄露的后果也只限于文件服务这一侧(不涉及账号、频道密钥),所以不需要非对称签名。
/// </summary>
public static class FileTokens
{
public const int MinSecretBytes = 16;
public static string Sign(ReadOnlySpan<byte> secret, FileTicket ticket) =>
Seal(secret, $"t|{(byte)ticket.Op}|{ticket.Channel}|{ticket.Item}|{ticket.From}|{ticket.ExpiresMs}|{ticket.Space}|{(ticket.Jti.Length > 0 ? ticket.Jti : Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(12)))}");
public static bool TryVerify(ReadOnlySpan<byte> secret, string? token, long nowMs, out FileTicket ticket)
{
ticket = null!;
if (!Open(secret, token, out var parts) || parts.Length != 8 || parts[0] != "t") return false;
if (!byte.TryParse(parts[1], out var op) || !Enum.IsDefined((FileOp)op) || !uint.TryParse(parts[4], out var from)
|| !long.TryParse(parts[5], out var expires) || expires < nowMs) return false;
ticket = new FileTicket((FileOp)op, parts[2], parts[3], from, expires, parts[6], parts[7]);
return true;
}
private static string Seal(ReadOnlySpan<byte> secret, string payload)
{
var bytes = Encoding.ASCII.GetBytes(payload);
var mac = HMACSHA256.HashData(secret, bytes);
return $"{Convert.ToBase64String(bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_')}.{Convert.ToBase64String(mac).TrimEnd('=').Replace('+', '-').Replace('/', '_')}";
}
private static bool Open(ReadOnlySpan<byte> secret, string? token, out string[] parts)
{
parts = [];
if (string.IsNullOrEmpty(token) || token.Length > 1024) return false;
int dot = token.IndexOf('.');
if (dot <= 0) return false;
if (!TryFromBase64Url(token[..dot], out var payload) || !TryFromBase64Url(token[(dot + 1)..], out var mac)) return false;
if (!CryptographicOperations.FixedTimeEquals(HMACSHA256.HashData(secret, payload), mac)) return false;
parts = Encoding.ASCII.GetString(payload).Split('|');
return true;
}
private static bool TryFromBase64Url(string text, out byte[] bytes)
{
bytes = [];
var padded = text.Replace('-', '+').Replace('_', '/');
padded = padded.PadRight(padded.Length + (4 - padded.Length % 4) % 4, '=');
try { bytes = Convert.FromBase64String(padded); return true; }
catch (FormatException) { return false; }
}
}
@@ -0,0 +1,38 @@
using System.Buffers.Binary;
namespace WebDrop.FileProtocol;
/// <summary>主后端、文件服务、浏览器三方约定的名字和格式</summary>
public static class FileWire
{
/// <summary>上传、读取时带的票据(主后端签发)</summary>
public const string TicketHeader = "X-WD-Ticket";
/// <summary>存储键:频道/条目/帧序号。帧序号补零到 10 位,按名字排序就是按帧序排序</summary>
public static string FrameKey(string channelHex, string itemHex, uint index) => $"{channelHex}/{itemHex}/{index:D10}";
public static bool IsHex32(string value)
{
if (value.Length != 32) return false;
foreach (char c in value) if (!(c is >= '0' and <= '9' or >= 'a' and <= 'f')) return false;
return true;
}
}
/// <summary>帧封装的 26 字节明文头部(protocol.md §4),文件服务只看这几个字段,负载原样保存</summary>
public readonly record struct FrameHeader(byte Flags, string ItemHex, uint Index, int PayloadLength)
{
public const int Size = 26;
public const byte Version = 1;
public const byte FinalFlag = 0x01;
public int TotalLength => Size + PayloadLength;
public static bool TryParse(ReadOnlySpan<byte> data, out FrameHeader header)
{
header = default;
if (data.Length < Size || data[0] != Version || (data[1] & 0xE0) != 0) return false;
uint payload = BinaryPrimitives.ReadUInt32BigEndian(data[22..]);
if (payload > int.MaxValue) return false;
header = new FrameHeader(data[1], Convert.ToHexStringLower(data.Slice(2, 16)), BinaryPrimitives.ReadUInt32BigEndian(data[18..]), (int)payload);
return true;
}
}
@@ -0,0 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">
<!-- 主后端和文件服务(fileAPI)共用的协议:票据、回执、帧头部。两边分开部署,所以只放这一点点,不依赖任何业务代码 -->
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>
+70
View File
@@ -0,0 +1,70 @@
using System.Security.Cryptography;
using WebDrop.Core;
namespace WebDrop.Storage;
/// <summary>
/// 头像按内容哈希存成磁盘文件,数据库里只记文件名(引用,形如 3f…a1.webp)。
/// 下载时直接把文件交给服务器发送,不用把几 MB 的 base64 读进托管内存再解码。
/// </summary>
public sealed class AvatarFiles
{
private static readonly Dictionary<string, string> MimeByExtension = new()
{
["png"] = "image/png", ["jpg"] = "image/jpeg", ["webp"] = "image/webp", ["gif"] = "image/gif",
};
private readonly string _dir;
public AvatarFiles(string dir)
{
_dir = Path.GetFullPath(dir);
Directory.CreateDirectory(_dir);
}
public string Dir => _dir;
/// <summary>旧数据把整段 data URL 存在数据库里;新数据是文件名</summary>
public static bool IsLegacy(string? value) => value is not null && value.StartsWith("data:", StringComparison.Ordinal);
/// <summary>把已校验的 data URL 解码写成文件(同样内容只写一份),返回引用</summary>
public string Save(string dataUrl)
{
var chars = dataUrl.AsSpan();
int semicolon = chars.IndexOf(';'), comma = chars.IndexOf(',');
if (!dataUrl.StartsWith("data:image/", StringComparison.Ordinal) || semicolon < 0 || comma < semicolon) throw new WebDropException(ErrorCode.BadRequest);
string extension = chars[11..semicolon].ToString() switch { "jpeg" => "jpg", var other => other };
if (!MimeByExtension.ContainsKey(extension)) throw new WebDropException(ErrorCode.BadRequest);
var payload = chars[(comma + 1)..];
var bytes = new byte[payload.Length / 4 * 3 + 3];
if (!Convert.TryFromBase64Chars(payload, bytes, out int written)) throw new WebDropException(ErrorCode.BadRequest);
string name = Convert.ToHexStringLower(SHA256.HashData(bytes.AsSpan(0, written))) + "." + extension;
string path = Path.Combine(_dir, name);
if (!File.Exists(path))
{
string temp = path + "." + Guid.NewGuid().ToString("N") + ".tmp";
File.WriteAllBytes(temp, bytes.AsSpan(0, written));
File.Move(temp, path, overwrite: true);
}
return name;
}
public AvatarBlob? Resolve(string? reference)
{
if (reference is null || IsLegacy(reference)) return null;
int dot = reference.IndexOf('.');
if (dot != 64 || !MimeByExtension.TryGetValue(reference[(dot + 1)..], out var mime)) return null;
foreach (char c in reference.AsSpan(0, 64)) if (!char.IsAsciiHexDigitLower(c)) return null;
string path = Path.Combine(_dir, reference);
return File.Exists(path) ? new AvatarBlob(path, mime, reference[..64]) : null;
}
/// <summary>启动时调用:删掉没有任何记录引用的文件(替换、删除头像后留下的)</summary>
public void Sweep(IReadOnlySet<string> referenced)
{
foreach (var path in System.IO.Directory.EnumerateFiles(_dir))
{
if (referenced.Contains(Path.GetFileName(path))) continue;
try { File.Delete(path); } catch (IOException) { }
}
}
}
@@ -0,0 +1,47 @@
using WebDrop.Core;
namespace WebDrop.Storage;
public sealed partial class SqliteAccountStore
{
/// <summary>头像文件(full 为原图,否则缩略图,没有缩略图就用原图)。没有头像为 null</summary>
public AvatarBlob? ProfileAvatar(string username, bool full)
{
var profile = Profile(username);
return _avatars.Resolve(full ? profile.Avatar : profile.AvatarThumb ?? profile.Avatar);
}
/// <summary>Avatar / AvatarThumb 是磁盘上的头像文件名,不是图片内容</summary>
public AccountProfile Profile(string username)
{
lock (_gate)
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "SELECT display_name,avatar,modified_at,avatar_thumb FROM account_profiles WHERE username=$u";
cmd.Parameters.AddWithValue("$u", username);
using var reader = cmd.ExecuteReader();
return reader.Read() ? new AccountProfile { Username = username, DisplayName = reader.GetString(0), Avatar = reader.IsDBNull(1) ? null : reader.GetString(1), HasAvatar = !reader.IsDBNull(1), ModifiedAt = reader.GetInt64(2), AvatarThumb = reader.IsDBNull(3) ? null : reader.GetString(3) }
: new AccountProfile { Username = username, DisplayName = username, ModifiedAt = 0 };
}
}
public AccountProfile UpdateProfile(string username, string displayName, string? avatar, string? avatarThumb = null, bool keepAvatar = false)
{
if (displayName is null || displayName.Trim().Length > 48 || displayName.Any(char.IsControl)) throw new WebDropException(ErrorCode.BadRequest);
if (keepAvatar) { var current = Profile(username); avatar = current.Avatar; avatarThumb = current.AvatarThumb; }
else if (avatar is not null) AvatarData.Validate(avatar, AvatarData.MaxAccountGifChars);
if (!keepAvatar && avatarThumb is not null) { if (avatar is null) throw new WebDropException(ErrorCode.BadRequest); AvatarData.ValidateThumb(avatarThumb); }
if (!keepAvatar) { avatar = avatar is null ? null : _avatars.Save(avatar); avatarThumb = avatarThumb is null ? null : _avatars.Save(avatarThumb); }
lock (_gate)
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "INSERT INTO account_profiles(username,display_name,avatar,avatar_thumb,modified_at) VALUES($u,$n,$a,$b,$t) ON CONFLICT(username) DO UPDATE SET display_name=excluded.display_name,avatar=excluded.avatar,avatar_thumb=excluded.avatar_thumb,modified_at=MAX(account_profiles.modified_at+1,excluded.modified_at)";
cmd.Parameters.AddWithValue("$u", username);
cmd.Parameters.AddWithValue("$n", displayName.Trim());
cmd.Parameters.AddWithValue("$a", (object?)avatar ?? DBNull.Value);
cmd.Parameters.AddWithValue("$b", (object?)avatarThumb ?? DBNull.Value);
cmd.Parameters.AddWithValue("$t", DateTimeOffset.UtcNow.ToUnixTimeMilliseconds());
cmd.ExecuteNonQuery();
return Profile(username);
}
}
}
@@ -8,10 +8,13 @@ public sealed partial class SqliteAccountStore : IDisposable
{ {
private readonly SqliteConnection _connection; private readonly SqliteConnection _connection;
private readonly object _gate = new(); private readonly object _gate = new();
private readonly AvatarFiles _avatars;
public SqliteAccountStore(string path) public SqliteAccountStore(string path, AvatarFiles? avatars = null)
{ {
Directory.CreateDirectory(Path.GetDirectoryName(Path.GetFullPath(path))!); string directory = Path.GetDirectoryName(Path.GetFullPath(path))!;
Directory.CreateDirectory(directory);
_avatars = avatars ?? new AvatarFiles(Path.Combine(directory, "avatars", "account"));
_connection = new SqliteConnection(new SqliteConnectionStringBuilder { DataSource = path, Pooling = true }.ToString()); _connection = new SqliteConnection(new SqliteConnectionStringBuilder { DataSource = path, Pooling = true }.ToString());
_connection.Open(); _connection.Open();
using var cmd = _connection.CreateCommand(); using var cmd = _connection.CreateCommand();
@@ -27,6 +30,19 @@ public sealed partial class SqliteAccountStore : IDisposable
cmd.ExecuteNonQuery(); cmd.ExecuteNonQuery();
using (var columns = _connection.CreateCommand()) using (var columns = _connection.CreateCommand())
{ {
columns.CommandText = "PRAGMA table_info(account_profiles)";
using (var profileColumns = columns.ExecuteReader())
{
bool hasThumb = false;
while (profileColumns.Read()) if (profileColumns.GetString(1) == "avatar_thumb") hasThumb = true;
profileColumns.Close();
if (!hasThumb)
{
using var alterProfiles = _connection.CreateCommand();
alterProfiles.CommandText = "ALTER TABLE account_profiles ADD COLUMN avatar_thumb TEXT";
alterProfiles.ExecuteNonQuery();
}
}
columns.CommandText = "PRAGMA table_info(accounts)"; columns.CommandText = "PRAGMA table_info(accounts)";
using var reader = columns.ExecuteReader(); using var reader = columns.ExecuteReader();
bool hasSigningKey = false; bool hasSigningKey = false;
@@ -39,6 +55,45 @@ public sealed partial class SqliteAccountStore : IDisposable
alter.ExecuteNonQuery(); alter.ExecuteNonQuery();
} }
} }
MigrateAvatars();
}
/// <summary>旧版把头像的 data URL 整段存在数据库里:搬成磁盘文件(幂等,一个账号一个账号处理),再删掉没人引用的文件</summary>
private void MigrateAvatars()
{
var legacy = new List<string>();
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = "SELECT username FROM account_profiles WHERE avatar LIKE 'data:%' OR avatar_thumb LIKE 'data:%'";
using var reader = cmd.ExecuteReader();
while (reader.Read()) legacy.Add(reader.GetString(0));
}
foreach (var username in legacy)
{
string? avatar, thumb;
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = "SELECT avatar, avatar_thumb FROM account_profiles WHERE username=$u";
cmd.Parameters.AddWithValue("$u", username);
using var reader = cmd.ExecuteReader();
reader.Read();
avatar = reader.IsDBNull(0) ? null : reader.GetString(0); thumb = reader.IsDBNull(1) ? null : reader.GetString(1);
}
using var update = _connection.CreateCommand();
update.CommandText = "UPDATE account_profiles SET avatar=$a, avatar_thumb=$t WHERE username=$u";
update.Parameters.AddWithValue("$u", username);
update.Parameters.AddWithValue("$a", (object?)(AvatarFiles.IsLegacy(avatar) ? _avatars.Save(avatar!) : avatar) ?? DBNull.Value);
update.Parameters.AddWithValue("$t", (object?)(AvatarFiles.IsLegacy(thumb) ? _avatars.Save(thumb!) : thumb) ?? DBNull.Value);
update.ExecuteNonQuery();
}
var referenced = new HashSet<string>();
using (var cmd = _connection.CreateCommand())
{
cmd.CommandText = "SELECT avatar, avatar_thumb FROM account_profiles";
using var reader = cmd.ExecuteReader();
while (reader.Read()) { if (!reader.IsDBNull(0)) referenced.Add(reader.GetString(0)); if (!reader.IsDBNull(1)) referenced.Add(reader.GetString(1)); }
}
_avatars.Sweep(referenced);
} }
private static byte[] HashVerifier(byte[] verifier, byte[] salt) => Rfc2898DeriveBytes.Pbkdf2(verifier, salt, 210_000, HashAlgorithmName.SHA256, 32); private static byte[] HashVerifier(byte[] verifier, byte[] salt) => Rfc2898DeriveBytes.Pbkdf2(verifier, salt, 210_000, HashAlgorithmName.SHA256, 32);
@@ -15,11 +15,13 @@ public sealed class SqliteChannelStore : IResourceStore, IDisposable
private readonly string _connectionString; private readonly string _connectionString;
private readonly SqliteConnection _writeConn; private readonly SqliteConnection _writeConn;
private readonly object _writeLock = new(); private readonly object _writeLock = new();
private readonly AvatarFiles _avatars;
public SqliteChannelStore(string dbPath) public SqliteChannelStore(string dbPath, AvatarFiles? avatars = null)
{ {
string? dir = Path.GetDirectoryName(Path.GetFullPath(dbPath)); string? dir = Path.GetDirectoryName(Path.GetFullPath(dbPath));
if (!string.IsNullOrEmpty(dir)) Directory.CreateDirectory(dir); if (!string.IsNullOrEmpty(dir)) Directory.CreateDirectory(dir);
_avatars = avatars ?? new AvatarFiles(Path.Combine(dir ?? ".", "avatars", "space"));
_connectionString = new SqliteConnectionStringBuilder _connectionString = new SqliteConnectionStringBuilder
{ {
@@ -37,6 +39,7 @@ public sealed class SqliteChannelStore : IResourceStore, IDisposable
Exec(_writeConn, "DELETE FROM channel_join_denials WHERE target LIKE 'endpoint:%';"); Exec(_writeConn, "DELETE FROM channel_join_denials WHERE target LIKE 'endpoint:%';");
// v7: 设置里的成员人数上限由 seats 改名为 max_members,迁移已有频道的设置 JSON(幂等) // v7: 设置里的成员人数上限由 seats 改名为 max_members,迁移已有频道的设置 JSON(幂等)
Exec(_writeConn, "UPDATE channels SET settings = json_remove(json_set(settings, '$.max_members', json_extract(settings, '$.seats')), '$.seats') WHERE json_extract(settings, '$.seats') IS NOT NULL AND json_extract(settings, '$.max_members') IS NULL;"); Exec(_writeConn, "UPDATE channels SET settings = json_remove(json_set(settings, '$.max_members', json_extract(settings, '$.seats')), '$.seats') WHERE json_extract(settings, '$.seats') IS NOT NULL AND json_extract(settings, '$.max_members') IS NULL;");
try { Exec(_writeConn, "ALTER TABLE channel_avatars ADD COLUMN thumb TEXT;"); } catch (SqliteException) { /* 新建的表或已经迁移过 */ }
// Additive table: retain the running v2 database and its existing channel data. // Additive table: retain the running v2 database and its existing channel data.
Exec(_writeConn, """ Exec(_writeConn, """
CREATE TABLE IF NOT EXISTS channel_key_packages ( CREATE TABLE IF NOT EXISTS channel_key_packages (
@@ -46,16 +49,76 @@ public sealed class SqliteChannelStore : IResourceStore, IDisposable
CREATE INDEX IF NOT EXISTS ix_channel_keys_username ON channel_key_packages(username); CREATE INDEX IF NOT EXISTS ix_channel_keys_username ON channel_key_packages(username);
CREATE TABLE IF NOT EXISTS workspace_structures (space_id BLOB PRIMARY KEY, state TEXT NOT NULL) WITHOUT ROWID; CREATE TABLE IF NOT EXISTS workspace_structures (space_id BLOB PRIMARY KEY, state TEXT NOT NULL) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS workspace_crypto (space_id BLOB PRIMARY KEY, state TEXT NOT NULL) WITHOUT ROWID; CREATE TABLE IF NOT EXISTS workspace_crypto (space_id BLOB PRIMARY KEY, state TEXT NOT NULL) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS channel_avatars (space_id BLOB PRIMARY KEY, avatar TEXT NOT NULL, thumb TEXT) WITHOUT ROWID;
"""); """);
MigrateAvatars();
}
/// <summary>旧版把头像的 data URL 整段存在数据库里:搬成磁盘文件(幂等,一行一行处理,不一次读入),再删掉没人引用的文件</summary>
private void MigrateAvatars()
{
var legacy = new List<byte[]>();
using (var cmd = _writeConn.CreateCommand())
{
cmd.CommandText = "SELECT space_id FROM channel_avatars WHERE avatar LIKE 'data:%' OR thumb LIKE 'data:%'";
using var reader = cmd.ExecuteReader();
while (reader.Read()) legacy.Add((byte[])reader[0]);
}
foreach (var key in legacy)
{
string? avatar, thumb;
using (var cmd = _writeConn.CreateCommand())
{
cmd.CommandText = "SELECT avatar, thumb FROM channel_avatars WHERE space_id=@c";
cmd.Parameters.AddWithValue("@c", key);
using var reader = cmd.ExecuteReader();
reader.Read();
avatar = reader.GetString(0); thumb = reader.IsDBNull(1) ? null : reader.GetString(1);
}
using var update = _writeConn.CreateCommand();
update.CommandText = "UPDATE channel_avatars SET avatar=@a, thumb=@t WHERE space_id=@c";
update.Parameters.AddWithValue("@c", key);
update.Parameters.AddWithValue("@a", AvatarFiles.IsLegacy(avatar) ? _avatars.Save(avatar) : avatar);
update.Parameters.AddWithValue("@t", (object?)(AvatarFiles.IsLegacy(thumb) ? _avatars.Save(thumb!) : thumb) ?? DBNull.Value);
update.ExecuteNonQuery();
}
var referenced = new HashSet<string>();
using (var cmd = _writeConn.CreateCommand())
{
cmd.CommandText = "SELECT avatar, thumb FROM channel_avatars";
using var reader = cmd.ExecuteReader();
while (reader.Read()) { referenced.Add(reader.GetString(0)); if (!reader.IsDBNull(1)) referenced.Add(reader.GetString(1)); }
}
_avatars.Sweep(referenced);
}
/// <summary>把整个数据库(含账号表,它们在同一个文件里)做一份一致的快照,不阻塞正在服务的请求</summary>
public void Snapshot(string path)
{
using var conn = Open();
using var cmd = conn.CreateCommand();
cmd.CommandText = "VACUUM INTO $path";
cmd.Parameters.AddWithValue("$path", path);
cmd.ExecuteNonQuery();
} }
public T Read<T>(Func<IResourceTx, T> action) public T Read<T>(Func<IResourceTx, T> action)
{
for (int attempt = 0; ; attempt++)
{ {
using var conn = Open(); using var conn = Open();
// 显式事务让一次 Read 内的多条查询看到同一份快照 // 显式事务让一次 Read 内的多条查询看到同一份快照
// 读事务延迟到首次访问时才取快照,避免并发读预检抢占 SQLite 写锁。 // 读事务延迟到首次访问时才取快照,避免并发读预检抢占 SQLite 写锁。
using var tx = conn.BeginTransaction(deferred: true); SqliteTransaction tx;
return action(new SqliteTx(conn)); try { tx = conn.BeginTransaction(deferred: true); }
// 并发压力下偶发 SQLITE_ERROR(CI 的并行追加测试里出现过):丢弃这条连接和池里的闲置连接,换新连接重试
catch (SqliteException e) when (e.SqliteErrorCode == 1 && attempt < 2)
{
SqliteConnection.ClearPool(conn);
continue;
}
using (tx) return action(new SqliteTx(conn, _avatars));
}
} }
public T Write<T>(Func<IResourceTx, T> action) public T Write<T>(Func<IResourceTx, T> action)
@@ -65,7 +128,7 @@ public sealed class SqliteChannelStore : IResourceStore, IDisposable
using var tx = _writeConn.BeginTransaction(); using var tx = _writeConn.BeginTransaction();
try try
{ {
T result = action(new SqliteTx(_writeConn)); T result = action(new SqliteTx(_writeConn, _avatars));
tx.Commit(); tx.Commit();
return result; return result;
} }
@@ -243,7 +306,7 @@ public sealed class SqliteChannelStore : IResourceStore, IDisposable
CREATE TABLE channel_members(channel_id BLOB NOT NULL,username TEXT NOT NULL,role TEXT NOT NULL,joined_at INTEGER NOT NULL,PRIMARY KEY(channel_id,username)); CREATE TABLE channel_members(channel_id BLOB NOT NULL,username TEXT NOT NULL,role TEXT NOT NULL,joined_at INTEGER NOT NULL,PRIMARY KEY(channel_id,username));
"""; """;
private sealed class SqliteTx(SqliteConnection conn) : IResourceTx private sealed class SqliteTx(SqliteConnection conn, AvatarFiles avatars) : IResourceTx
{ {
private const string ChannelCols = "id, mode, settings, read_hash, write_hash, owner_hash, created_at, expires_at, last_seq, compacted_through, bytes, creator_account, join_password_hash, join_password_salt"; private const string ChannelCols = "id, mode, settings, read_hash, write_hash, owner_hash, created_at, expires_at, last_seq, compacted_through, bytes, creator_account, join_password_hash, join_password_salt";
public WorkspaceState? GetWorkspace(Id16 id) => One("SELECT state FROM workspace_structures WHERE space_id=@id", public WorkspaceState? GetWorkspace(Id16 id) => One("SELECT state FROM workspace_structures WHERE space_id=@id",
@@ -492,10 +555,25 @@ public sealed class SqliteChannelStore : IResourceStore, IDisposable
Run("DELETE FROM channel_key_packages WHERE channel_id=@c", ("@c", key)); Run("DELETE FROM channel_key_packages WHERE channel_id=@c", ("@c", key));
Run("DELETE FROM workspace_structures WHERE space_id=@c", ("@c", key)); Run("DELETE FROM workspace_structures WHERE space_id=@c", ("@c", key));
Run("DELETE FROM workspace_crypto WHERE space_id=@c", ("@c", key)); Run("DELETE FROM workspace_crypto WHERE space_id=@c", ("@c", key));
Run("DELETE FROM channel_avatars WHERE space_id=@c", ("@c", key));
Run("DELETE FROM channel_members WHERE channel_id=@c", ("@c", key)); Run("DELETE FROM channel_members WHERE channel_id=@c", ("@c", key));
Run("DELETE FROM channel_join_denials WHERE channel_id=@c", ("@c", key)); Run("DELETE FROM channel_join_denials WHERE channel_id=@c", ("@c", key));
} }
public AvatarBlob? GetChannelAvatar(Id16 id, bool thumb) => avatars.Resolve(
Scalar(thumb ? "SELECT COALESCE(thumb, avatar) FROM channel_avatars WHERE space_id=@c" : "SELECT avatar FROM channel_avatars WHERE space_id=@c", ("@c", id.ToArray())) as string);
/// <summary>头像文件名(内容哈希)就是版本:内容变了名字才会变</summary>
public string? GetChannelAvatarVersion(Id16 id) =>
Scalar("SELECT avatar || '|' || COALESCE(thumb, '') FROM channel_avatars WHERE space_id=@c", ("@c", id.ToArray())) as string;
public void SetChannelAvatar(Id16 id, string? avatar, string? thumb)
{
if (avatar is null) { Run("DELETE FROM channel_avatars WHERE space_id=@c", ("@c", id.ToArray())); return; }
Run("INSERT INTO channel_avatars (space_id, avatar, thumb) VALUES (@c,@a,@t) ON CONFLICT(space_id) DO UPDATE SET avatar=excluded.avatar, thumb=excluded.thumb",
("@c", id.ToArray()), ("@a", avatars.Save(avatar)), ("@t", (object?)(thumb is null ? null : avatars.Save(thumb)) ?? DBNull.Value));
}
public IReadOnlyList<Id16> ListExpiredChannels(long now, int limit) => public IReadOnlyList<Id16> ListExpiredChannels(long now, int limit) =>
Many("SELECT id FROM channels WHERE expires_at<=@now LIMIT @n", r => IdAt(r, 0), ("@now", now), ("@n", limit)); Many("SELECT id FROM channels WHERE expires_at<=@now LIMIT @n", r => IdAt(r, 0), ("@now", now), ("@n", limit));
@@ -1,41 +0,0 @@
using WebDrop.Core;
namespace WebDrop.Storage;
public sealed partial class SqliteAccountStore
{
public AccountProfile Profile(string username)
{
lock (_gate)
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "SELECT display_name,avatar,modified_at FROM account_profiles WHERE username=$u";
cmd.Parameters.AddWithValue("$u", username);
using var reader = cmd.ExecuteReader();
return reader.Read() ? new AccountProfile { Username = username, DisplayName = reader.GetString(0), Avatar = reader.IsDBNull(1) ? null : reader.GetString(1), ModifiedAt = reader.GetInt64(2) }
: new AccountProfile { Username = username, DisplayName = username, ModifiedAt = 0 };
}
}
public AccountProfile UpdateProfile(string username, string displayName, string? avatar)
{
if (displayName is null || displayName.Trim().Length > 48 || displayName.Any(char.IsControl)) throw new WebDropException(ErrorCode.BadRequest);
if (avatar is not null)
{
string? prefix = new[] { "data:image/png;base64,", "data:image/jpeg;base64,", "data:image/webp;base64," }.FirstOrDefault(p => avatar.StartsWith(p, StringComparison.Ordinal));
if (prefix is null || avatar.Length > 350_000 || avatar.Length <= prefix.Length) throw new WebDropException(ErrorCode.BadRequest);
try { _ = Convert.FromBase64String(avatar[prefix.Length..]); }
catch (FormatException) { throw new WebDropException(ErrorCode.BadRequest); }
}
lock (_gate)
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "INSERT INTO account_profiles(username,display_name,avatar,modified_at) VALUES($u,$n,$a,$t) ON CONFLICT(username) DO UPDATE SET display_name=excluded.display_name,avatar=excluded.avatar,modified_at=MAX(account_profiles.modified_at+1,excluded.modified_at)";
cmd.Parameters.AddWithValue("$u", username);
cmd.Parameters.AddWithValue("$n", displayName.Trim());
cmd.Parameters.AddWithValue("$a", (object?)avatar ?? DBNull.Value);
cmd.Parameters.AddWithValue("$t", DateTimeOffset.UtcNow.ToUnixTimeMilliseconds());
cmd.ExecuteNonQuery();
return Profile(username);
}
}
}
+131
View File
@@ -0,0 +1,131 @@
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Xml.Linq;
namespace WebDrop.Sync;
/// <summary>
/// 只实现备份用得到的 S3 操作:上传、删除、按前缀列出。自己做 SigV4 签名,不引 AWS SDK(NativeAOT 友好)。
/// 上传的流必须可以回退:先扫一遍算摘要再发送,这样不用把大文件读进内存。
/// </summary>
public sealed class S3Client(HttpClient http, S3Options options)
{
private static readonly XNamespace Ns = "http://s3.amazonaws.com/doc/2006-03-01/";
private static readonly string EmptyHash = Convert.ToHexStringLower(SHA256.HashData([]));
private readonly string _base = options.BasePath.Trim('/');
private string KeyOf(string path) => string.Join('/', new[] { _base, path.Trim('/') }.Where(s => s.Length > 0));
public async Task PutAsync(string path, Stream content, CancellationToken ct)
{
string hash = Convert.ToHexStringLower(await SHA256.HashDataAsync(content, ct).ConfigureAwait(false));
content.Position = 0;
using var request = Signed(HttpMethod.Put, KeyOf(path), [], hash);
request.Content = new StreamContent(content);
request.Content.Headers.ContentType = new("application/octet-stream");
using var response = await http.SendAsync(request, ct).ConfigureAwait(false);
if (!response.IsSuccessStatusCode) throw new HttpRequestException($"S3 PUT 失败:{(int)response.StatusCode}");
}
/// <summary>删除 path 这个对象,以及它作为目录时底下的所有对象。什么都没删到返回 false</summary>
public async Task<bool> DeleteAsync(string path, CancellationToken ct)
{
bool any = false;
string key = KeyOf(path);
foreach (var name in await ListKeysAsync(key + "/", null, ct).ConfigureAwait(false)) any |= await DeleteKeyAsync(name, ct).ConfigureAwait(false);
return await DeleteKeyAsync(key, ct).ConfigureAwait(false) | any;
}
private async Task<bool> DeleteKeyAsync(string key, CancellationToken ct)
{
// S3 删除不存在的对象也回 204,所以先 HEAD 才知道有没有
using (var head = Signed(HttpMethod.Head, key, [], EmptyHash))
using (var existing = await http.SendAsync(head, ct).ConfigureAwait(false))
{
if (existing.StatusCode == HttpStatusCode.NotFound) return false;
if (!existing.IsSuccessStatusCode) throw new HttpRequestException($"S3 HEAD 失败:{(int)existing.StatusCode}");
}
using var request = Signed(HttpMethod.Delete, key, [], EmptyHash);
using var response = await http.SendAsync(request, ct).ConfigureAwait(false);
if (!response.IsSuccessStatusCode && response.StatusCode != HttpStatusCode.NotFound) throw new HttpRequestException($"S3 DELETE 失败:{(int)response.StatusCode}");
return true;
}
/// <summary>列出目录下一层的名字(文件和子目录),不含目录自己</summary>
public async Task<IReadOnlyList<string>> ListAsync(string path, CancellationToken ct)
{
string prefix = KeyOf(path) + "/";
var names = new List<string>();
foreach (var key in await ListKeysAsync(prefix, "/", ct).ConfigureAwait(false))
{
string rest = key[prefix.Length..].TrimEnd('/');
if (rest.Length > 0) names.Add(rest);
}
return names;
}
private async Task<List<string>> ListKeysAsync(string prefix, string? delimiter, CancellationToken ct)
{
var result = new List<string>();
string? token = null;
do
{
var query = new List<KeyValuePair<string, string>> { new("list-type", "2"), new("prefix", prefix) };
if (delimiter is not null) query.Add(new("delimiter", delimiter));
if (token is not null) query.Add(new("continuation-token", token));
using var request = Signed(HttpMethod.Get, "", query, EmptyHash);
using var response = await http.SendAsync(request, ct).ConfigureAwait(false);
if (!response.IsSuccessStatusCode) throw new HttpRequestException($"S3 LIST 失败:{(int)response.StatusCode}");
var root = XDocument.Parse(await response.Content.ReadAsStringAsync(ct).ConfigureAwait(false)).Root!;
result.AddRange(root.Elements(Ns + "Contents").Select(e => e.Element(Ns + "Key")!.Value));
result.AddRange(root.Elements(Ns + "CommonPrefixes").Select(e => e.Element(Ns + "Prefix")!.Value));
token = root.Element(Ns + "IsTruncated")?.Value == "true" ? root.Element(Ns + "NextContinuationToken")?.Value : null;
} while (token is not null);
return result;
}
// ───── SigV4
private HttpRequestMessage Signed(HttpMethod method, string key, IReadOnlyList<KeyValuePair<string, string>> query, string payloadHash)
{
var endpoint = new Uri(options.Endpoint.TrimEnd('/') + "/");
string host = endpoint.IsDefaultPort ? endpoint.Host : $"{endpoint.Host}:{endpoint.Port}";
if (!options.PathStyle) host = options.Bucket + "." + host;
string objectPath = key.Length == 0 ? "" : "/" + string.Join('/', key.Split('/').Select(Encode));
string canonicalUri = endpoint.AbsolutePath.TrimEnd('/') + (options.PathStyle ? "/" + Encode(options.Bucket) : "") + objectPath;
if (canonicalUri.Length == 0) canonicalUri = "/";
string canonicalQuery = string.Join('&', query.Select(p => (Key: Encode(p.Key), Value: Encode(p.Value)))
.OrderBy(p => p.Key, StringComparer.Ordinal).ThenBy(p => p.Value, StringComparer.Ordinal).Select(p => $"{p.Key}={p.Value}"));
var now = DateTime.UtcNow;
string amzDate = now.ToString("yyyyMMdd'T'HHmmss'Z'"), day = now.ToString("yyyyMMdd");
const string signedHeaders = "host;x-amz-content-sha256;x-amz-date";
string canonicalHeaders = $"host:{host}\nx-amz-content-sha256:{payloadHash}\nx-amz-date:{amzDate}\n";
string canonical = $"{method.Method}\n{canonicalUri}\n{canonicalQuery}\n{canonicalHeaders}\n{signedHeaders}\n{payloadHash}";
string scope = $"{day}/{options.Region}/s3/aws4_request";
string toSign = $"AWS4-HMAC-SHA256\n{amzDate}\n{scope}\n{Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(canonical)))}";
byte[] signingKey = Hmac(Hmac(Hmac(Hmac(Encoding.UTF8.GetBytes("AWS4" + options.SecretKey), day), options.Region), "s3"), "aws4_request");
string signature = Convert.ToHexStringLower(Hmac(signingKey, toSign));
var request = new HttpRequestMessage(method, new Uri($"{endpoint.Scheme}://{host}{canonicalUri}" + (canonicalQuery.Length > 0 ? "?" + canonicalQuery : "")));
request.Headers.TryAddWithoutValidation("x-amz-content-sha256", payloadHash);
request.Headers.TryAddWithoutValidation("x-amz-date", amzDate);
request.Headers.TryAddWithoutValidation("Authorization", $"AWS4-HMAC-SHA256 Credential={options.AccessKey}/{scope}, SignedHeaders={signedHeaders}, Signature={signature}");
return request;
}
private static byte[] Hmac(byte[] key, string data) => HMACSHA256.HashData(key, Encoding.UTF8.GetBytes(data));
/// <summary>RFC 3986:只有字母数字和 -_.~ 不转义</summary>
private static string Encode(string value)
{
var sb = new StringBuilder();
foreach (byte b in Encoding.UTF8.GetBytes(value))
{
if (b is >= (byte)'A' and <= (byte)'Z' or >= (byte)'a' and <= (byte)'z' or >= (byte)'0' and <= (byte)'9' or (byte)'-' or (byte)'_' or (byte)'.' or (byte)'~') sb.Append((char)b);
else sb.Append('%').Append(b.ToString("X2"));
}
return sb.ToString();
}
}
+43
View File
@@ -0,0 +1,43 @@
namespace WebDrop.Sync;
/// <summary>外部存储:任何兼容 S3 协议的服务(AWS S3、Cloudflare R2、MinIO、Backblaze B2、阿里云 OSS ……)</summary>
public sealed class S3Options
{
/// <summary>服务地址,如 https://s3.us-east-1.amazonaws.com 或 http://minio:9000</summary>
public string Endpoint { get; set; } = "";
/// <summary>签名用的区域;MinIO 等可以保持 us-east-1(R2 用 auto)</summary>
public string Region { get; set; } = "us-east-1";
public string Bucket { get; set; } = "";
public string AccessKey { get; set; } = "";
public string SecretKey { get; set; } = "";
/// <summary>对象键的统一前缀,如 webdrop</summary>
public string BasePath { get; set; } = "webdrop";
/// <summary>路径风格(endpoint/bucket/key),MinIO 等需要;false 为虚拟主机风格(bucket.endpoint/key)</summary>
public bool PathStyle { get; set; } = true;
public bool Configured => Endpoint.Length > 0 && Bucket.Length > 0 && AccessKey.Length > 0 && SecretKey.Length > 0;
}
/// <summary>每天固定时刻运行一次的备份计划。时间用服务器本地时区,形如 04:00</summary>
public sealed class DailyBackupOptions
{
public bool Enabled { get; set; }
public string Time { get; set; } = "04:00";
/// <summary>外部备份保留多少天;被删除的内容在备份里最多再留这么久</summary>
public int RetainDays { get; set; } = 30;
/// <summary>限速(字节/秒),0 表示不限。家宽上行只有 50M,可以限一下,别把聊天上传挤掉</summary>
public long MaxBytesPerSecond { get; set; }
public S3Options S3 { get; set; } = new();
public TimeOnly TimeOfDay => TimeOnly.TryParse(Time, out var t) ? t : new TimeOnly(4, 0);
}
public static class DailySchedule
{
/// <summary>下一次运行的时刻(严格晚于 now)</summary>
public static DateTime Next(DateTime now, TimeOnly at)
{
var today = now.Date + at.ToTimeSpan();
return today > now ? today : today.AddDays(1);
}
}
+12
View File
@@ -0,0 +1,12 @@
namespace WebDrop.Sync;
/// <summary>粗略的限速:每发完一块,按它占的时间睡够。0 表示不限</summary>
public sealed class Throttle(long bytesPerSecond)
{
public async Task AfterAsync(long bytes, TimeSpan took, CancellationToken ct)
{
if (bytesPerSecond <= 0 || bytes <= 0) return;
var target = TimeSpan.FromSeconds((double)bytes / bytesPerSecond);
if (target > took) await Task.Delay(target - took, ct).ConfigureAwait(false);
}
}
@@ -0,0 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">
<!-- 同步到外部存储(OpenList 等,走 S3 协议)。主后端和文件服务各自用它备份自己的数据,所以单独成项目,不依赖任何业务代码 -->
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>
@@ -0,0 +1,72 @@
using System.IO.Compression;
using Microsoft.Data.Sqlite;
using Microsoft.Extensions.DependencyInjection;
using static WebDrop.Api.Tests.ServerFixture;
namespace WebDrop.Api.Tests;
/// <summary>主后端每天把数据库快照压缩后传到外部存储(S3 协议),按保留期清理旧的。</summary>
public sealed class BackupTests : IAsyncLifetime
{
private readonly FakeS3 _s3 = new();
private ServerFixture _server = null!;
public async Task InitializeAsync()
{
await _s3.StartAsync();
_server = ServerFixture.WithSettings(new()
{
["WebDrop:Backup:S3:Endpoint"] = _s3.Url,
["WebDrop:Backup:S3:AccessKey"] = FakeS3.AccessKey,
["WebDrop:Backup:S3:SecretKey"] = FakeS3.SecretKey,
["WebDrop:Backup:S3:BasePath"] = "webdrop",
["WebDrop:Backup:S3:Bucket"] = FakeS3.Bucket,
["WebDrop:Backup:RetainDays"] = "30",
});
await _server.InitializeAsync();
}
public async Task DisposeAsync()
{
await _server.DisposeAsync();
await _s3.DisposeAsync();
}
[Fact]
public async Task TheSnapshotIsAConsistentCompressedDatabaseAndOldOnesAreRetired()
{
var ch = await _server.CreateAsync();
var worker = _server.App.Services.GetServices<Microsoft.Extensions.Hosting.IHostedService>().OfType<WebDrop.Api.BackupWorker>().Single();
// 40 天前做的一份,和今天的一份
var old = await worker.RunOnceAsync(DateTime.Now.AddDays(-40), CancellationToken.None);
Assert.Equal(0, old.Purged);
Assert.True(_s3.Files.ContainsKey($"webdrop/chat/{old.Name}"));
var today = await worker.RunOnceAsync(DateTime.Now, CancellationToken.None);
Assert.Equal(1, today.Purged); // 超过 30 天的那份被清掉
Assert.False(_s3.Files.ContainsKey($"webdrop/chat/{old.Name}"));
Assert.True(_s3.Files.ContainsKey($"webdrop/chat/{today.Name}"));
// 备份能还原成数据库,里面有刚建的频道和账号
string path = Path.Combine(Path.GetTempPath(), "restore-" + Guid.NewGuid().ToString("N") + ".db");
try
{
using (var input = new MemoryStream(_s3.Files[$"webdrop/chat/{today.Name}"]))
using (var zip = new GZipStream(input, CompressionMode.Decompress))
using (var output = File.Create(path))
zip.CopyTo(output);
using var conn = new SqliteConnection($"Data Source={path};Mode=ReadOnly");
conn.Open();
using var count = conn.CreateCommand();
count.CommandText = "SELECT (SELECT COUNT(*) FROM channels), (SELECT COUNT(*) FROM accounts)";
using var reader = count.ExecuteReader();
Assert.True(reader.Read());
Assert.True(reader.GetInt64(0) >= 1 && reader.GetInt64(1) >= 2);
}
finally { SqliteConnection.ClearAllPools(); try { File.Delete(path); } catch (IOException) { } }
// 本地不留临时文件
Assert.Empty(Directory.EnumerateFiles(Path.Combine(_server.DataDir, "v2", "backup-tmp")));
}
}
+89
View File
@@ -0,0 +1,89 @@
using System.Collections.Concurrent;
using System.Security.Cryptography;
using System.Text;
using System.Xml.Linq;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
namespace WebDrop.Api.Tests;
/// <summary>
/// 只实现备份用到的 S3 子集(PUT、HEAD、DELETE、ListObjectsV2),路径风格,数据放在内存里。
/// 检查 SigV4 的凭据字段和 x-amz-content-sha256 与正文是否一致;不重算签名本身(签名算法由真实服务兜底)。
/// </summary>
public sealed class FakeS3 : IAsyncDisposable
{
private readonly WebApplication _app;
/// <summary>键(不含存储桶)→ 内容</summary>
public ConcurrentDictionary<string, byte[]> Files { get; } = new();
public bool Down { get; set; }
public string Url { get; private set; } = "";
public const string AccessKey = "AKIATEST", SecretKey = "secret", Bucket = "bk";
public FakeS3()
{
var builder = WebApplication.CreateSlimBuilder();
builder.Configuration["urls"] = "http://127.0.0.1:0";
builder.Logging.ClearProviders();
_app = builder.Build();
_app.Map("/{bucket}/{**key}", async (HttpContext ctx, string bucket, string? key) =>
{
if (Down) { ctx.Response.StatusCode = 503; return; }
string auth = ctx.Request.Headers.Authorization.ToString();
if (bucket != Bucket || !auth.StartsWith($"AWS4-HMAC-SHA256 Credential={AccessKey}/", StringComparison.Ordinal) || !auth.Contains("Signature=") || !ctx.Request.Headers.ContainsKey("x-amz-date"))
{ ctx.Response.StatusCode = 403; return; }
key = Uri.UnescapeDataString(key ?? "");
switch (ctx.Request.Method)
{
case "PUT":
{
using var ms = new MemoryStream();
await ctx.Request.Body.CopyToAsync(ms);
var body = ms.ToArray();
if (!string.Equals(ctx.Request.Headers["x-amz-content-sha256"], Convert.ToHexStringLower(SHA256.HashData(body)), StringComparison.Ordinal)) { ctx.Response.StatusCode = 400; return; }
Files[key] = body;
ctx.Response.StatusCode = 200;
break;
}
case "HEAD": ctx.Response.StatusCode = Files.ContainsKey(key) ? 200 : 404; break;
case "DELETE": Files.TryRemove(key, out _); ctx.Response.StatusCode = 204; break;
case "GET":
{
string prefix = ctx.Request.Query["prefix"].ToString(), delimiter = ctx.Request.Query["delimiter"].ToString();
var keys = Files.Keys.Where(k => k.StartsWith(prefix, StringComparison.Ordinal)).Order(StringComparer.Ordinal).ToList();
XNamespace ns = "http://s3.amazonaws.com/doc/2006-03-01/";
var root = new XElement(ns + "ListBucketResult", new XElement(ns + "IsTruncated", "false"));
var folders = new SortedSet<string>(StringComparer.Ordinal);
foreach (var k in keys)
{
int cut = delimiter.Length == 0 ? -1 : k.IndexOf(delimiter, prefix.Length, StringComparison.Ordinal);
if (cut < 0) root.Add(new XElement(ns + "Contents", new XElement(ns + "Key", k)));
else folders.Add(k[..(cut + delimiter.Length)]);
}
foreach (var f in folders) root.Add(new XElement(ns + "CommonPrefixes", new XElement(ns + "Prefix", f)));
ctx.Response.StatusCode = 200;
ctx.Response.ContentType = "application/xml";
await ctx.Response.WriteAsync(root.ToString(), Encoding.UTF8);
break;
}
default: ctx.Response.StatusCode = 405; break;
}
});
}
public async Task StartAsync()
{
await _app.StartAsync();
Url = _app.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First().TrimEnd('/');
}
public async ValueTask DisposeAsync()
{
await _app.StopAsync();
await _app.DisposeAsync();
}
}
@@ -0,0 +1,448 @@
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using WebDrop.Core;
using WebDrop.FileApi;
using WebDrop.Storage;
using static WebDrop.Api.Tests.ServerFixture;
namespace WebDrop.Api.Tests;
/// <summary>主后端 + 文件服务两个独立的服务一起跑:文件内容只在文件服务上;上传、登记、下载、计次、删除的完整流程。</summary>
public sealed class FileApiTests : IAsyncLifetime
{
private const string Secret = "files-test-secret-0123456789abcdef-xyz";
private readonly string _dir = Path.Combine(Path.GetTempPath(), "webdrop-files-" + Guid.NewGuid().ToString("N"));
private WebApplication _files = null!;
private ServerFixture _main = null!;
private HttpClient _fileHttp = null!;
private string _fileUrl = "";
private Signer _owner = null!;
private sealed record Signer(string Token, string Username, string DeviceId, byte[] PrivateKey);
public async Task InitializeAsync()
{
_files = FileApiApp.Build([], b =>
{
b.Configuration["FileApi:DataDir"] = _dir;
b.Configuration["FileApi:Secret"] = Secret;
b.Configuration["urls"] = "http://127.0.0.1:0";
b.Logging.ClearProviders();
});
await _files.StartAsync();
_fileUrl = _files.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First().TrimEnd('/');
_fileHttp = new HttpClient { BaseAddress = new Uri(_fileUrl) };
_main = ServerFixture.WithSettings(new()
{
["WebDrop:Files:Enabled"] = "true",
["WebDrop:Files:PublicUrl"] = _fileUrl,
["WebDrop:Files:Secret"] = Secret,
});
await _main.InitializeAsync();
_owner = await RegisterSignerAsync("files_owner");
}
public async Task DisposeAsync()
{
await _main.DisposeAsync();
_fileHttp.Dispose();
await _files.StopAsync();
await _files.DisposeAsync();
try { Directory.Delete(_dir, recursive: true); } catch (IOException) { }
}
// ───── 账号与签名:带设备签名密钥的账号,文件和聊天对象的最后一帧必须带签名事件 ─────
private async Task<Signer> RegisterSignerAsync(string username)
{
var accounts = _main.App.Services.GetRequiredService<SqliteAccountStore>();
var verifier = RandomNumberGenerator.GetBytes(32);
using var invitation = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
using var root = ECDsa.Create(ECCurve.NamedCurves.nistP256);
using var device = ECDsa.Create(ECCurve.NamedCurves.nistP256);
static string Jwk(ECPoint p) => JsonSerializer.Serialize(new { kty = "EC", crv = "P-256", x = B64(p.X!), y = B64(p.Y!) });
var devicePoint = device.ExportParameters(false).Q;
Assert.True(accounts.Register(username, RandomNumberGenerator.GetBytes(16), verifier, new byte[28], Jwk(invitation.ExportParameters(false).Q), Jwk(root.ExportParameters(false).Q)));
string deviceId = Guid.NewGuid().ToString("D");
var token = accounts.Authenticate(username, verifier, deviceId, "Test device")!;
var cert = root.SignData(SignatureCodec.BuildMessage("device.certificate", SignatureCodec.DeviceCertificatePayload(username, deviceId, devicePoint.X!, devicePoint.Y!)),
HashAlgorithmName.SHA256, DSASignatureFormat.IeeeP1363FixedFieldConcatenation);
using var request = new HttpRequestMessage(HttpMethod.Put, "/api/v1/accounts/devices/signing-key")
{ Content = JsonBody(new { device_id = deviceId, public_key = Jwk(devicePoint), signature = Convert.ToBase64String(cert) }) };
request.Headers.Add("X-WD-Account", token);
Assert.Equal(HttpStatusCode.NoContent, (await _main.Http.SendAsync(request)).StatusCode);
return new Signer(token, username, deviceId, device.ExportPkcs8PrivateKey());
}
/// <summary>条目最后一帧的签名事件头:对所有帧的摘要(清单)签名</summary>
private Dictionary<string, string> SignedHeaders(string spaceId, string channelId, string itemId, CryptoDomain domain, string kind, IReadOnlyList<byte[]> frames)
{
Assert.True(Codec.TryDecodeExact(spaceId, 16, out var space)); Assert.True(Codec.TryDecodeExact(itemId, 16, out var item));
var manifest = SignatureCodec.FrameManifestHash(frames.Select(frame => SHA256.HashData(frame)).ToList());
var eventId = RandomNumberGenerator.GetBytes(16);
var payload = SignatureCodec.ContentEventPayload(space, channelId, _owner.Username, _owner.DeviceId, eventId, kind, domain, 1, item, frames.Count, manifest, []);
using var device = ECDsa.Create(); device.ImportPkcs8PrivateKey(_owner.PrivateKey, out _);
var signature = device.SignData(SignatureCodec.BuildMessage("content.event", payload), HashAlgorithmName.SHA256, DSASignatureFormat.IeeeP1363FixedFieldConcatenation);
return new Dictionary<string, string>
{
["X-WD-Event-Id"] = B64(eventId), ["X-WD-Event-Kind"] = kind, ["X-WD-Event-Channel"] = channelId,
["X-WD-Event-Domain"] = domain == CryptoDomain.Chat ? "chat" : "files", ["X-WD-Event-Epoch"] = "1", ["X-WD-Event-Frames"] = frames.Count.ToString(),
["X-WD-Event-Manifest"] = B64(manifest), ["X-WD-Event-Signature"] = B64(signature),
};
}
private Who Owner(Chan ch) => ch.Owner with { AccountToken = _owner.Token };
private async Task<Chan> CreateSpaceAsync()
{
var ch = new Chan(NewId(), ChannelMode.Enc);
var body = new Dictionary<string, object?>
{
["id"] = ch.Id, ["mode"] = "enc", ["owner_hash"] = B64(TokenHash.Compute(ch.OwnerToken)),
["write_hash"] = B64(TokenHash.Compute(ch.WriteToken)), ["read_hash"] = B64(TokenHash.Compute(ch.ReadToken)),
["settings"] = new { name = "Files space", max_members = 2, join_policy = "open" },
};
using var response = await _main.Http.SendAsync(_main.Req(HttpMethod.Post, "/api/v1/channels", Owner(ch), JsonBody(body)));
Assert.Equal(HttpStatusCode.Created, response.StatusCode);
return ch;
}
/// <summary>空间里的默认频道:发布聊天和文件两个域的密钥(客户端创建空间时做的事),并为它创建一个文件域(同样是客户端做的事),之后才能声明对象</summary>
private async Task<string> PrepareAsync(Chan ch)
{
string channelId;
using (var request = _main.Req(HttpMethod.Get, $"/api/v1/spaces/{ch.Id}/structure", Owner(ch)))
using (var response = await _main.Http.SendAsync(request))
channelId = (await ReadJsonAsync(response)).GetProperty("structure").GetProperty("defaultChannelId").GetString()!;
foreach (var domain in new[] { "chat", "files" })
{
using var bootstrap = _main.Req(HttpMethod.Put, $"/api/v1/spaces/{ch.Id}/crypto/domains", Owner(ch),
JsonBody(new { channelId, domain, epoch = 1, envelopes = new[] { new { channelId, domain, epoch = 1, username = _owner.Username, payload = $"owner-{domain}" } } }));
Assert.Equal(HttpStatusCode.NoContent, (await _main.Http.SendAsync(bootstrap)).StatusCode);
}
await CreateDomainAsync(ch, channelId);
return channelId;
}
private async Task DeclareAsync(Chan ch, string channelId, string itemId, string domain = "files")
{
using var declare = _main.Req(HttpMethod.Post, $"/api/v1/spaces/{ch.Id}/crypto/objects", Owner(ch),
JsonBody(new { itemId, channelId, domain, epoch = 1, keyEnvelope = "opaque-object-key" }));
Assert.Equal(HttpStatusCode.Created, (await _main.Http.SendAsync(declare)).StatusCode);
}
// ───── 主后端发票据,文件服务存字节 ─────
private readonly Dictionary<string, string> _domains = [];
private async Task<HttpResponseMessage> GrantRequestAsync(Chan ch, string op, string? item = null, uint from = 0) =>
await _main.Http.SendAsync(_main.Req(HttpMethod.Post, $"/api/v1/spaces/{ch.Id}/file-grants", Owner(ch),
new StringContent(JsonSerializer.Serialize(new { op, item, from }), Encoding.UTF8, "application/json")));
private async Task<(string Url, string Domain, string Ticket)> GrantAsync(Chan ch, string op, string? item = null, uint from = 0)
{
using var response = await GrantRequestAsync(ch, op, item, from);
if (response.StatusCode != HttpStatusCode.OK) throw new HttpRequestException($"grant {(int)response.StatusCode} {await response.Content.ReadAsStringAsync()}");
using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync());
var root = doc.RootElement;
return (root.GetProperty("url").GetString()!, root.GetProperty("domain").GetString()!, root.GetProperty("ticket").GetString()!);
}
/// <summary>前端创建频道时做的事:向空间要「创建」票据,去文件服务建文件域,再把域 id 记到频道上</summary>
private async Task<string> CreateDomainAsync(Chan ch, string channelId)
{
var grant = await GrantAsync(ch, "create");
using var create = new HttpRequestMessage(HttpMethod.Post, "/file/v1/domains") { Content = new StringContent("{}", Encoding.UTF8, "application/json") };
create.Headers.Add("X-WD-Ticket", grant.Ticket);
using var created = await _fileHttp.SendAsync(create);
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
using var doc = JsonDocument.Parse(await created.Content.ReadAsStringAsync());
string domain = doc.RootElement.GetProperty("id").GetString()!;
using var record = _main.Req(HttpMethod.Put, $"/api/v1/spaces/{ch.Id}/channels/{channelId}/file-domain", Owner(ch), JsonBody(new { fileDomain = domain }));
Assert.Equal(HttpStatusCode.OK, (await _main.Http.SendAsync(record)).StatusCode);
return _domains[ch.Id] = domain;
}
/// <summary>把一帧存进文件域,返回文件服务算的摘要</summary>
private async Task<string> StoreAsync(Chan ch, string item, byte[] frame, uint index)
{
var grant = await GrantAsync(ch, "put", item);
using var put = new HttpRequestMessage(HttpMethod.Put, $"/file/v1/domains/{grant.Domain}/items/{Hex(item)}/frames/{index}") { Content = new ByteArrayContent(frame) };
put.Headers.Add("X-WD-Ticket", grant.Ticket);
using var stored = await _fileHttp.SendAsync(put);
Assert.Equal(HttpStatusCode.OK, stored.StatusCode);
using var doc = JsonDocument.Parse(await stored.Content.ReadAsStringAsync());
string digest = doc.RootElement.GetProperty("digest").GetString()!;
Assert.Equal(Convert.ToHexStringLower(SHA256.HashData(frame)), digest);
return digest;
}
/// <summary>到主后端一次登记整个文件:末帧的 26 字节头部 + 每一帧的(负载长度,整帧摘要),不带任何字节</summary>
private async Task<HttpResponseMessage> RegisterAsync(Chan ch, IReadOnlyList<byte[]> frames, string? maxDownloads = null, Dictionary<string, string>? signed = null, IReadOnlyList<string>? digests = null)
{
var body = new byte[26 + frames.Count * 36];
frames[^1][..26].CopyTo(body, 0);
for (int i = 0; i < frames.Count; i++)
{
System.Buffers.Binary.BinaryPrimitives.WriteUInt32BigEndian(body.AsSpan(26 + i * 36), (uint)(frames[i].Length - 26));
Convert.FromHexString(digests?[i] ?? Convert.ToHexStringLower(SHA256.HashData(frames[i]))).CopyTo(body, 26 + i * 36 + 4);
}
var register = _main.Req(HttpMethod.Post, $"/api/v1/channels/{ch.Id}/files", Owner(ch), new ByteArrayContent(body));
if (maxDownloads is not null) register.Headers.Add("X-WD-Max-Downloads", maxDownloads);
foreach (var (name, value) in signed ?? []) register.Headers.Add(name, value);
return await _main.Http.SendAsync(register);
}
/// <summary>上传一个文件:每一帧先存进文件域,全部存好后一次到主后端登记(末帧的签名事件盖着所有帧的摘要)</summary>
private async Task<HttpStatusCode> UploadFileAsync(Chan ch, string channelId, string item, IReadOnlyList<byte[]> frames, string? maxDownloads = null)
{
for (int i = 0; i < frames.Count; i++) await StoreAsync(ch, item, frames[i], (uint)i);
using var response = await RegisterAsync(ch, frames, maxDownloads, SignedHeaders(ch.Id, channelId, item, CryptoDomain.Files, "file.publish", frames));
return response.StatusCode;
}
private async Task<(HttpResponseMessage Response, byte[] Body)> ReadFromFilesAsync(string domain, string item, string ticket, uint from)
{
using var get = new HttpRequestMessage(HttpMethod.Get, $"/file/v1/domains/{domain}/items/{Hex(item)}/frames?from={from}");
get.Headers.Add("X-WD-Ticket", ticket);
var response = await _fileHttp.SendAsync(get);
return (response, await response.Content.ReadAsByteArrayAsync());
}
private static byte[] Small(string item, byte fill = 0x42) => Frame(item, 0, FinalEnc, EncPayload, fill);
private static string Hex(string base64UrlId)
{
Assert.True(Codec.TryDecodeExact(base64UrlId, 16, out var bytes));
return Convert.ToHexStringLower(bytes);
}
// ───── 测试 ─────
[Fact]
public async Task FileBytesLiveInTheFileDomainWhileChatStaysOnTheHost()
{
var info = await _main.Http.GetStringAsync("/api/v1/info");
Assert.Contains("\"files\":{\"enabled\":true", info);
Assert.DoesNotContain("threshold", info);
var ch = await CreateSpaceAsync();
string channelId = await PrepareAsync(ch);
string item = NewId(), tiny = NewId(), chat = NewId();
await DeclareAsync(ch, channelId, item); await DeclareAsync(ch, channelId, tiny); await DeclareAsync(ch, channelId, chat, "chat");
var frames = new[] { Frame(item, 0, GatedEnc, EncPayload, 1), Frame(item, 1, GatedEnc, EncPayload, 2), Frame(item, 2, GatedFinalEnc, EncPayload, 3) };
var tinyFrame = Small(tiny, 7);
// 文件域的对象不能把字节直接交给主后端,不管多大
using (var direct = await _main.AppendAsync(ch, Owner(ch), frames[0], maxDownloads: "1")) Assert.Equal(HttpStatusCode.Conflict, direct.StatusCode);
using (var directSmall = _main.Req(HttpMethod.Post, $"/api/v1/channels/{ch.Id}/frames", Owner(ch), Bytes(tinyFrame)))
{
foreach (var (name, value) in SignedHeaders(ch.Id, channelId, tiny, CryptoDomain.Files, "file.publish", [tinyFrame])) directSmall.Headers.Add(name, value);
using var rejected = await _main.Http.SendAsync(directSmall);
Assert.Equal(HttpStatusCode.Conflict, rejected.StatusCode);
}
Assert.Equal(HttpStatusCode.Created, await UploadFileAsync(ch, channelId, item, frames, "1"));
Assert.Equal(HttpStatusCode.Created, await UploadFileAsync(ch, channelId, tiny, [tinyFrame]));
// 聊天(聊天域的对象)照旧存在主后端,日志里带着内容
var chatFrame = Small(chat, 5);
using (var chatRequest = _main.Req(HttpMethod.Post, $"/api/v1/channels/{ch.Id}/frames", Owner(ch), Bytes(chatFrame)))
{
foreach (var (name, value) in SignedHeaders(ch.Id, channelId, chat, CryptoDomain.Chat, "chat.create", [chatFrame])) chatRequest.Headers.Add(name, value);
using var chatResponse = await _main.Http.SendAsync(chatRequest);
Assert.Equal(HttpStatusCode.Created, chatResponse.StatusCode);
}
var (_, logBody) = await _main.GetBytesAsync($"/api/v1/channels/{ch.Id}/log", Owner(ch));
var log = ParseLog(logBody);
Assert.Equal(2, log.Count(r => r.Kind == LogKind.Announce)); // 两个文件条目都只是「公布」
Assert.Equal(1, log.Count(r => r.Kind == LogKind.Frame)); // 只有聊天帧带内容
await AssertErrorAsync(await _main.Http.SendAsync(_main.Req(HttpMethod.Get, $"/api/v1/channels/{ch.Id}/items/{item}/frames", Owner(ch))), 409, "use_file_api");
// 受限条目:要票据,按原来的规则计次(上限 1 次)
var grant = await GrantAsync(ch, "get", item);
Assert.Equal(_domains[ch.Id], grant.Domain);
var (read, body) = await ReadFromFilesAsync(grant.Domain, item, grant.Ticket, 0);
Assert.Equal(HttpStatusCode.OK, read.StatusCode);
Assert.Equal(frames.SelectMany(f => f).ToArray(), body);
using (var second = await GrantRequestAsync(ch, "get", item)) Assert.Equal(HttpStatusCode.Gone, second.StatusCode);
var resume = await GrantAsync(ch, "get", item, 2);
Assert.Equal(frames[2], (await ReadFromFilesAsync(resume.Domain, item, resume.Ticket, 2)).Body);
// 单帧不限次的文件:想读几次都行
for (int i = 0; i < 3; i++)
{
var t = await GrantAsync(ch, "get", tiny);
Assert.Equal(tinyFrame, (await ReadFromFilesAsync(t.Domain, tiny, t.Ticket, 0)).Body);
}
// 聊天条目没有文件域,不发文件票据
using (var chatGrant = await GrantRequestAsync(ch, "get", chat)) Assert.Equal(HttpStatusCode.NotFound, chatGrant.StatusCode);
}
[Fact]
public async Task TicketsFollowTheSpacePermissionsAndAreBoundToTheirItem()
{
var ch = await CreateSpaceAsync();
string channelId = await PrepareAsync(ch);
string item = NewId(), other = NewId();
await DeclareAsync(ch, channelId, item); await DeclareAsync(ch, channelId, other);
var f0 = Frame(item, 0, GatedEnc, EncPayload, 1);
var f1 = Frame(item, 1, GatedFinalEnc, EncPayload, 2);
// 没有空间里的身份,什么票据都拿不到
var stranger = ch.Owner with { Token = new byte[] { 1, 2, 3 }, AccountToken = null };
using (var denied = await _main.Http.SendAsync(_main.Req(HttpMethod.Post, $"/api/v1/spaces/{ch.Id}/file-grants", stranger,
new StringContent(JsonSerializer.Serialize(new { op = "create" }), Encoding.UTF8, "application/json"))))
Assert.False(denied.IsSuccessStatusCode);
// 别的条目的票据传不进这个条目;没有票据也不行
var otherGrant = await GrantAsync(ch, "put", other);
using (var put = new HttpRequestMessage(HttpMethod.Put, $"/file/v1/domains/{otherGrant.Domain}/items/{Hex(item)}/frames/0") { Content = new ByteArrayContent(f0) })
{
put.Headers.Add("X-WD-Ticket", otherGrant.Ticket);
using var wrongItem = await _fileHttp.SendAsync(put);
Assert.Equal(HttpStatusCode.Unauthorized, wrongItem.StatusCode);
}
using (var noTicket = await _fileHttp.PutAsync($"/file/v1/domains/{otherGrant.Domain}/items/{Hex(item)}/frames/0", new ByteArrayContent(f0))) Assert.Equal(HttpStatusCode.Unauthorized, noTicket.StatusCode);
// 同一个序号不能存不同的内容;同样的内容重传是幂等的
string digest0 = await StoreAsync(ch, item, f0, 0);
var grant = await GrantAsync(ch, "put", item);
using (var put = new HttpRequestMessage(HttpMethod.Put, $"/file/v1/domains/{grant.Domain}/items/{Hex(item)}/frames/0") { Content = new ByteArrayContent(Frame(item, 0, GatedEnc, EncPayload, 9)) })
{
put.Headers.Add("X-WD-Ticket", grant.Ticket);
using var conflict = await _fileHttp.SendAsync(put);
Assert.Equal(HttpStatusCode.Conflict, conflict.StatusCode);
}
Assert.Equal(digest0, await StoreAsync(ch, item, f0, 0));
// 登记整个文件(一次请求):先把第二帧也存进文件域
string digest1 = await StoreAsync(ch, item, f1, 1);
// 末帧的签名清单盖着所有帧的摘要:登记的摘要和清单对不上,整个文件就不被接受
using (var wrongDigest = await RegisterAsync(ch, [f0, f1], "1", SignedHeaders(ch.Id, channelId, item, CryptoDomain.Files, "file.publish", [f0, f1]), [digest0, digest0]))
Assert.Equal(HttpStatusCode.Conflict, wrongDigest.StatusCode);
using (var good = await RegisterAsync(ch, [f0, f1], "1", SignedHeaders(ch.Id, channelId, item, CryptoDomain.Files, "file.publish", [f0, f1])))
Assert.Equal(HttpStatusCode.Created, good.StatusCode);
// 同一个文件重复登记幂等;摘要不同(同一个条目)就是冲突
using (var again = await RegisterAsync(ch, [f0, f1], "1", SignedHeaders(ch.Id, channelId, item, CryptoDomain.Files, "file.publish", [f0, f1]))) Assert.True(again.IsSuccessStatusCode);
using (var different = await RegisterAsync(ch, [f0, f1], "1", SignedHeaders(ch.Id, channelId, item, CryptoDomain.Files, "file.publish", [f0, f1]), [digest0, new string('a', 64)]))
Assert.Equal(HttpStatusCode.Conflict, different.StatusCode);
// 请求体格式不对
using (var malformed = _main.Req(HttpMethod.Post, $"/api/v1/channels/{ch.Id}/files", Owner(ch), new ByteArrayContent(new byte[10])))
using (var bad = await _main.Http.SendAsync(malformed)) Assert.Equal(HttpStatusCode.BadRequest, bad.StatusCode);
// 不是文件域的条目不能走登记
string chat = NewId();
await DeclareAsync(ch, channelId, chat, "chat");
var chatFrame = Small(chat);
using (var notAFile = await RegisterAsync(ch, [chatFrame], null, SignedHeaders(ch.Id, channelId, chat, CryptoDomain.Chat, "chat.create", [chatFrame])))
Assert.Equal(HttpStatusCode.BadRequest, notAFile.StatusCode);
}
[Fact]
public async Task ItemsInAChannelWithoutAFileDomainGetNoTicketsAndNeverRegisteredFilesCannotBeDownloaded()
{
var ch = await CreateSpaceAsync();
string channelId = await PrepareAsync(ch);
string item = NewId();
await DeclareAsync(ch, channelId, item);
// 字节存进了文件域、但没有到主后端登记的文件(上传到一半放弃):主后端不认识它,不发下载票据
await StoreAsync(ch, item, Frame(item, 0, GatedEnc, EncPayload, 1), 0);
using (var notRegistered = await GrantRequestAsync(ch, "get", item)) Assert.Equal(HttpStatusCode.NotFound, notRegistered.StatusCode);
// 第二个空间里的默认频道没有文件域:声明了文件对象也要不到上传票据
var bare = await CreateSpaceAsync();
string bareChannel;
using (var request = _main.Req(HttpMethod.Get, $"/api/v1/spaces/{bare.Id}/structure", Owner(bare)))
using (var response = await _main.Http.SendAsync(request))
bareChannel = (await ReadJsonAsync(response)).GetProperty("structure").GetProperty("defaultChannelId").GetString()!;
foreach (var domain in new[] { "chat", "files" })
{
using var bootstrap = _main.Req(HttpMethod.Put, $"/api/v1/spaces/{bare.Id}/crypto/domains", Owner(bare),
JsonBody(new { channelId = bareChannel, domain, epoch = 1, envelopes = new[] { new { channelId = bareChannel, domain, epoch = 1, username = _owner.Username, payload = $"owner-{domain}" } } }));
Assert.Equal(HttpStatusCode.NoContent, (await _main.Http.SendAsync(bootstrap)).StatusCode);
}
string orphan = NewId();
await DeclareAsync(bare, bareChannel, orphan);
using (var noDomain = await GrantRequestAsync(bare, "put", orphan)) Assert.Equal(HttpStatusCode.Conflict, noDomain.StatusCode);
}
[Fact]
public async Task DeletingAFileTakesAGrantThenDeletesOnTheHostAndInTheFileDomain()
{
var ch = await CreateSpaceAsync();
string channelId = await PrepareAsync(ch);
var store = _files.Services.GetRequiredService<FrameStore>();
string finished = NewId();
await DeclareAsync(ch, channelId, finished);
var finishedFrame = Small(finished, 3);
Assert.Equal(HttpStatusCode.Created, await UploadFileAsync(ch, channelId, finished, [finishedFrame]));
string domain = _domains[ch.Id];
Assert.True(store.UsedBy(domain) > 0);
foreach (var item in new[] { finished })
{
// 先向空间要「删除」票据(主后端确认能删),再删主后端的条目,最后用票据清文件域里的字节
var grant = await GrantAsync(ch, "delete", item);
using (var hostDelete = _main.Req(HttpMethod.Delete, $"/api/v1/channels/{ch.Id}/items/{item}", Owner(ch)))
{
// 删除文件要带作者设备签名的 file.delete 事件(清单和发布时一致)
foreach (var (name, value) in SignedHeaders(ch.Id, channelId, item, CryptoDomain.Files, "file.delete", [finishedFrame])) hostDelete.Headers.Add(name, value);
using var host = await _main.Http.SendAsync(hostDelete);
Assert.Equal(HttpStatusCode.NoContent, host.StatusCode);
}
using var remove = new HttpRequestMessage(HttpMethod.Delete, $"/file/v1/domains/{grant.Domain}/items/{Hex(item)}");
remove.Headers.Add("X-WD-Ticket", grant.Ticket);
using var removed = await _fileHttp.SendAsync(remove);
Assert.Equal(HttpStatusCode.NoContent, removed.StatusCode);
}
Assert.Equal(0, store.UsedBy(domain));
// 取消上传:主后端那边还没有这个条目(整个文件传完才登记),用上传这个条目的票据就能清掉自己传的字节
string cancelled = NewId();
await DeclareAsync(ch, channelId, cancelled);
await StoreAsync(ch, cancelled, Frame(cancelled, 0, GatedEnc, EncPayload, 5), 0);
Assert.True(store.UsedBy(domain) > 0);
var put = await GrantAsync(ch, "put", cancelled);
using var abandon = new HttpRequestMessage(HttpMethod.Delete, $"/file/v1/domains/{put.Domain}/items/{Hex(cancelled)}");
abandon.Headers.Add("X-WD-Ticket", put.Ticket);
using var abandoned = await _fileHttp.SendAsync(abandon);
Assert.Equal(HttpStatusCode.NoContent, abandoned.StatusCode);
Assert.Equal(0, store.UsedBy(domain));
}
[Fact]
public async Task AFileUploadAsTheClientDoesItCompletesWithCatalogAndLocation()
{
var ch = await CreateSpaceAsync();
string channelId = await PrepareAsync(ch);
string item = NewId(), catalog = NewId();
await DeclareAsync(ch, channelId, item); await DeclareAsync(ch, channelId, catalog);
var frames = new[] { Frame(item, 0, GatedEnc, EncPayload, 1), Frame(item, 1, GatedEnc, EncPayload, 2), Frame(item, 2, GatedFinalEnc, EncPayload, 3) };
// 1. 字节进文件域,整个文件一次登记
Assert.Equal(HttpStatusCode.Created, await UploadFileAsync(ch, channelId, item, frames, "0"));
// 2. 目录条目:一个控制帧(小,不受限),走主后端,带 file.catalog 签名事件
var control = Frame(catalog, 0, 16 | 8 | 1, EncPayload, 7);
using (var request = _main.Req(HttpMethod.Post, $"/api/v1/channels/{ch.Id}/frames", Owner(ch), Bytes(control)))
{
foreach (var (name, value) in SignedHeaders(ch.Id, channelId, catalog, CryptoDomain.Files, "file.catalog", [control])) request.Headers.Add(name, value);
using var response = await _main.Http.SendAsync(request);
Assert.True(response.IsSuccessStatusCode, $"目录条目被拒绝:{(int)response.StatusCode} {await response.Content.ReadAsStringAsync()}");
}
// 3. 位置登记:文件放进频道
using (var location = _main.Req(HttpMethod.Put, $"/api/v1/spaces/{ch.Id}/locations", Owner(ch), JsonBody(new { itemIds = new[] { item }, threadId = channelId })))
using (var response = await _main.Http.SendAsync(location))
Assert.True(response.IsSuccessStatusCode, $"位置登记被拒绝:{(int)response.StatusCode} {await response.Content.ReadAsStringAsync()}");
}
}
@@ -0,0 +1,248 @@
using System.Buffers.Binary;
using System.Net;
using System.Net.Http.Json;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using WebDrop.FileApi;
using WebDrop.FileProtocol;
namespace WebDrop.Api.Tests;
/// <summary>文件域是 fileAPI 自己的概念:创建、查询、容量、删除都由它负责,凭空间签发的票据访问。</summary>
public sealed class FileDomainTests : IAsyncLifetime
{
private const string Secret = "domain-test-secret-0123456789abcdef0123";
private static readonly byte[] SecretBytes = Encoding.UTF8.GetBytes(Secret);
private readonly string _dir = Path.Combine(Path.GetTempPath(), "wd-domain-" + Guid.NewGuid().ToString("N"));
private WebApplication _files = null!;
private HttpClient _http = null!;
private readonly string _space = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
public async Task InitializeAsync()
{
_files = FileApiApp.Build([], b =>
{
b.Configuration["FileApi:DataDir"] = _dir;
b.Configuration["FileApi:Secret"] = Secret;
b.Configuration["FileApi:QuotaChoices:0"] = "2048";
b.Configuration["FileApi:QuotaChoices:1"] = "8192";
b.Configuration["FileApi:DefaultQuota"] = "2048";
b.Configuration["FileApi:CorsOrigins:0"] = "http://localhost:5173";
b.Configuration["urls"] = "http://127.0.0.1:0";
b.Logging.ClearProviders();
});
await _files.StartAsync();
_http = new HttpClient { BaseAddress = new Uri(_files.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First()) };
}
public async Task DisposeAsync()
{
_http.Dispose();
await _files.StopAsync();
await _files.DisposeAsync();
try { Directory.Delete(_dir, recursive: true); } catch (IOException) { }
}
/// <summary>票据默认是本测试的空间签发的(管理票据不属于任何空间)</summary>
private string Ticket(FileOp op, string channel, string item = "*", string? space = null) =>
FileTokens.Sign(SecretBytes, new FileTicket(op, channel, item, 0, DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() + 60_000, space ?? (op == FileOp.Admin ? "" : _space)));
private HttpRequestMessage Request(HttpMethod method, string url, string ticket, HttpContent? content = null)
{
var request = new HttpRequestMessage(method, url) { Content = content };
request.Headers.Add(FileWire.TicketHeader, ticket);
return request;
}
private async Task<FileDomainBody> CreateAsync(long quota = 0)
{
using var request = Request(HttpMethod.Post, "/file/v1/domains", Ticket(FileOp.Create, _space), JsonContent.Create(new FileDomainCreate(quota), options: null));
using var response = await _http.SendAsync(request);
Assert.Equal(HttpStatusCode.Created, response.StatusCode);
return JsonSerializer.Deserialize(await response.Content.ReadAsStringAsync(), FileProtocolJson.Default.FileDomainBody)!;
}
private static byte[] Frame(string itemHex, uint index, int payload)
{
var frame = new byte[FrameHeader.Size + payload];
frame[0] = FrameHeader.Version;
frame[1] = FrameHeader.FinalFlag;
Convert.FromHexString(itemHex).CopyTo(frame, 2);
BinaryPrimitives.WriteUInt32BigEndian(frame.AsSpan(18), index);
BinaryPrimitives.WriteUInt32BigEndian(frame.AsSpan(22), (uint)payload);
RandomNumberGenerator.Fill(frame.AsSpan(FrameHeader.Size));
return frame;
}
private Task<HttpResponseMessage> PutAsync(string domain, string item, uint index, byte[] frame, string? ticket = null) =>
_http.SendAsync(Request(HttpMethod.Put, $"/file/v1/domains/{domain}/items/{item}/frames/{index}", ticket ?? Ticket(FileOp.Put, domain, item), new ByteArrayContent(frame)));
[Fact]
public async Task OptionsAreAPublicStatementOfWhatTheFileServiceOffers()
{
var options = JsonSerializer.Deserialize(await _http.GetStringAsync("/file/v1/options"), FileProtocolJson.Default.FileDomainOptions)!;
Assert.Equal([2048L, 8192L], options.QuotaChoices);
Assert.Equal(2048, options.DefaultQuota);
}
[Fact]
public async Task OnlyASpaceIssuedCreateTicketCreatesADomainAndTheQuotaMustBeOneOfTheChoices()
{
var domain = await CreateAsync(8192);
Assert.Equal(8192, domain.Quota);
Assert.Equal(_space, domain.Space);
Assert.True(FileWire.IsHex32(domain.Id));
Assert.Equal(2048, (await CreateAsync()).Quota); // 不选用默认值
using (var odd = Request(HttpMethod.Post, "/file/v1/domains", Ticket(FileOp.Create, _space), JsonContent.Create(new FileDomainCreate(12345))))
using (var rejected = await _http.SendAsync(odd)) Assert.Equal(HttpStatusCode.BadRequest, rejected.StatusCode);
// 读写票据、没有票据都不能创建
using (var wrong = Request(HttpMethod.Post, "/file/v1/domains", Ticket(FileOp.Put, _space, "*"), JsonContent.Create(new FileDomainCreate())))
using (var denied = await _http.SendAsync(wrong)) Assert.Equal(HttpStatusCode.Unauthorized, denied.StatusCode);
using (var anonymous = await _http.PostAsync("/file/v1/domains", JsonContent.Create(new FileDomainCreate()))) Assert.Equal(HttpStatusCode.Unauthorized, anonymous.StatusCode);
}
[Fact]
public async Task WritesAreCountedAgainstTheDomainQuotaAndOverflowIsRefused()
{
var domain = await CreateAsync(); // 2048 字节
string item = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16)), other = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var first = await PutAsync(domain.Id, item, 0, Frame(item, 0, 1000))) Assert.Equal(HttpStatusCode.OK, first.StatusCode); // 1026 字节
using (var query = Request(HttpMethod.Get, $"/file/v1/domains/{domain.Id}", Ticket(FileOp.Get, domain.Id)))
using (var info = await _http.SendAsync(query))
{
var body = JsonSerializer.Deserialize(await info.Content.ReadAsStringAsync(), FileProtocolJson.Default.FileDomainBody)!;
Assert.Equal(1026, body.Used);
Assert.Equal(2048, body.Quota);
}
using (var second = await PutAsync(domain.Id, other, 0, Frame(other, 0, 1000))) Assert.Equal(HttpStatusCode.InsufficientStorage, second.StatusCode); // 再存会超出
using (var small = await PutAsync(domain.Id, other, 0, Frame(other, 0, 500))) Assert.Equal(HttpStatusCode.OK, small.StatusCode);
}
[Fact]
public async Task ATicketOnlyWorksForItsOwnDomainAndUnknownDomainsDoNotExist()
{
var a = await CreateAsync(); var b = await CreateAsync();
string item = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var cross = await PutAsync(b.Id, item, 0, Frame(item, 0, 10), Ticket(FileOp.Put, a.Id, item))) Assert.Equal(HttpStatusCode.Unauthorized, cross.StatusCode);
string ghost = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var missing = await PutAsync(ghost, item, 0, Frame(item, 0, 10))) Assert.Equal(HttpStatusCode.Unauthorized, missing.StatusCode);
using (var read = Request(HttpMethod.Get, $"/file/v1/domains/{ghost}", Ticket(FileOp.Get, ghost)))
using (var notFound = await _http.SendAsync(read)) Assert.Equal(HttpStatusCode.Unauthorized, notFound.StatusCode);
}
[Fact]
public async Task DeletingADomainRemovesItsFilesAndOnlyAnAdminTicketMay()
{
var domain = await CreateAsync();
string item = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var stored = await PutAsync(domain.Id, item, 0, Frame(item, 0, 100))) Assert.Equal(HttpStatusCode.OK, stored.StatusCode);
using (var denied = await _http.SendAsync(Request(HttpMethod.Delete, $"/file/v1/domains/{domain.Id}", Ticket(FileOp.Put, domain.Id, item)))) Assert.Equal(HttpStatusCode.Unauthorized, denied.StatusCode);
using (var removed = await _http.SendAsync(Request(HttpMethod.Delete, $"/file/v1/domains/{domain.Id}", Ticket(FileOp.Admin, "*")))) Assert.Equal(HttpStatusCode.NoContent, removed.StatusCode);
using (var gone = await PutAsync(domain.Id, item, 0, Frame(item, 0, 100))) Assert.Equal(HttpStatusCode.Unauthorized, gone.StatusCode);
using (var read = await _http.SendAsync(Request(HttpMethod.Get, $"/file/v1/domains/{domain.Id}/items/{item}/frames", Ticket(FileOp.Get, domain.Id, item)))) Assert.Equal(HttpStatusCode.Unauthorized, read.StatusCode);
}
[Fact]
public async Task ATicketFromAnotherSpaceIsRefusedEvenIfItNamesTheDomain()
{
var domain = await CreateAsync(); // 属于 _space
string item = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
string otherSpace = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var foreign = await PutAsync(domain.Id, item, 0, Frame(item, 0, 10), Ticket(FileOp.Put, domain.Id, item, otherSpace))) Assert.Equal(HttpStatusCode.Unauthorized, foreign.StatusCode);
using (var own = await PutAsync(domain.Id, item, 0, Frame(item, 0, 10))) Assert.Equal(HttpStatusCode.OK, own.StatusCode);
}
[Fact]
public async Task AReadTicketWorksOnlyOnceSoReplayingItCannotBypassTheDownloadCount()
{
var domain = await CreateAsync();
string item = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var stored = await PutAsync(domain.Id, item, 0, Frame(item, 0, 100))) Assert.Equal(HttpStatusCode.OK, stored.StatusCode);
string ticket = Ticket(FileOp.Get, domain.Id, item);
using (var first = await _http.SendAsync(Request(HttpMethod.Get, $"/file/v1/domains/{domain.Id}/items/{item}/frames", ticket))) Assert.Equal(HttpStatusCode.OK, first.StatusCode);
using (var replay = await _http.SendAsync(Request(HttpMethod.Get, $"/file/v1/domains/{domain.Id}/items/{item}/frames", ticket))) Assert.Equal(HttpStatusCode.Unauthorized, replay.StatusCode);
// 上传票据一个条目的所有帧共用,不受一次性限制
string put = Ticket(FileOp.Put, domain.Id, item);
using (var f1 = await PutAsync(domain.Id, item, 1, Frame(item, 1, 50), put)) Assert.Equal(HttpStatusCode.OK, f1.StatusCode);
using (var f2 = await PutAsync(domain.Id, item, 2, Frame(item, 2, 50), put)) Assert.Equal(HttpStatusCode.OK, f2.StatusCode);
}
[Fact]
public async Task AnUploadStaysPendingUntilCommittedAndAbandonedOnesAreSwept()
{
var domain = await CreateAsync();
var store = _files.Services.GetRequiredService<FrameStore>();
string kept = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16)), abandoned = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16)), old = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
foreach (var item in new[] { kept, abandoned })
using (var stored = await PutAsync(domain.Id, item, 0, Frame(item, 0, 100))) Assert.Equal(HttpStatusCode.OK, stored.StatusCode);
// 都还待确认;确认了一个之后,它不再被当成没人要的
Assert.Contains($"{domain.Id}/{kept}", store.StalePending(TimeSpan.Zero));
using (var commit = await _http.SendAsync(Request(HttpMethod.Post, $"/file/v1/domains/{domain.Id}/items/{kept}/commit", Ticket(FileOp.Put, domain.Id, kept)))) Assert.Equal(HttpStatusCode.NoContent, commit.StatusCode);
Assert.DoesNotContain($"{domain.Id}/{kept}", store.StalePending(TimeSpan.Zero));
Assert.Contains($"{domain.Id}/{abandoned}", store.StalePending(TimeSpan.Zero));
// 读取票据不能确认
using (var denied = await _http.SendAsync(Request(HttpMethod.Post, $"/file/v1/domains/{domain.Id}/items/{abandoned}/commit", Ticket(FileOp.Get, domain.Id, abandoned)))) Assert.Equal(HttpStatusCode.Unauthorized, denied.StatusCode);
// 没确认的过了保留期被清理,确认过的和旧版本留下的(没有标记)不受影响
Directory.CreateDirectory(Path.Combine(_dir, "frames", domain.Id, old));
File.WriteAllBytes(Path.Combine(_dir, "frames", domain.Id, old, "0000000000"), new byte[10]);
var sweeper = _files.Services.GetServices<Microsoft.Extensions.Hosting.IHostedService>().OfType<PendingSweeper>().Single();
foreach (var path in Directory.EnumerateFiles(Path.Combine(_dir, "frames", domain.Id, abandoned))) File.SetLastWriteTimeUtc(path, DateTime.UtcNow.AddDays(-2));
Assert.Equal(1, sweeper.SweepOnce());
Assert.False(Directory.Exists(Path.Combine(_dir, "frames", domain.Id, abandoned)));
Assert.True(Directory.Exists(Path.Combine(_dir, "frames", domain.Id, kept)));
Assert.True(Directory.Exists(Path.Combine(_dir, "frames", domain.Id, old)));
}
[Fact]
public async Task CancellingAnUploadWithTheUploadTicketFreesTheDomainSpace()
{
var domain = await CreateAsync();
var store = _files.Services.GetRequiredService<FrameStore>();
string item = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var stored = await PutAsync(domain.Id, item, 0, Frame(item, 0, 500))) Assert.Equal(HttpStatusCode.OK, stored.StatusCode);
Assert.True(store.UsedBy(domain.Id) > 0);
using (var cancelled = await _http.SendAsync(Request(HttpMethod.Delete, $"/file/v1/domains/{domain.Id}/items/{item}", Ticket(FileOp.Put, domain.Id, item)))) Assert.Equal(HttpStatusCode.NoContent, cancelled.StatusCode);
Assert.Equal(0, store.UsedBy(domain.Id));
// 读取票据不能拿来删
using (var again = await PutAsync(domain.Id, item, 0, Frame(item, 0, 500))) Assert.Equal(HttpStatusCode.OK, again.StatusCode);
using (var denied = await _http.SendAsync(Request(HttpMethod.Delete, $"/file/v1/domains/{domain.Id}/items/{item}", Ticket(FileOp.Get, domain.Id, item)))) Assert.Equal(HttpStatusCode.Unauthorized, denied.StatusCode);
}
[Theory]
[InlineData("POST")]
[InlineData("PUT")]
[InlineData("DELETE")]
[InlineData("GET")]
public async Task BrowserPreflightAllowsEveryMethodTheFrontendUses(string method)
{
// 浏览器直连文件服务:创建文件域、确认条目是 POST,取消和删除是 DELETE,不在预检允许的方法里会被浏览器拦下
using var request = new HttpRequestMessage(HttpMethod.Options, "/file/v1/domains");
request.Headers.Add("Origin", "http://localhost:5173");
request.Headers.Add("Access-Control-Request-Method", method);
request.Headers.Add("Access-Control-Request-Headers", "x-wd-ticket,content-type");
using var response = await _http.SendAsync(request);
Assert.Contains(method, response.Headers.GetValues("Access-Control-Allow-Methods").Single().Split(','));
}
[Fact]
public async Task DomainUsageSurvivesARestartAndStaysCorrectAfterADelete()
{
// 重启之后用量从磁盘重新统计:先删一个条目,剩下的条目不能被算两次扣减
var domain = await CreateAsync();
string a = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16)), b = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(16));
using (var one = await PutAsync(domain.Id, a, 0, Frame(a, 0, 1000))) Assert.Equal(HttpStatusCode.OK, one.StatusCode);
using (var two = await PutAsync(domain.Id, b, 0, Frame(b, 0, 300))) Assert.Equal(HttpStatusCode.OK, two.StatusCode);
var restarted = new FrameStore(Path.Combine(_dir, "frames"), 0, Path.Combine(_dir, "sync-restarted"));
Assert.True(restarted.DeletePrefix($"{domain.Id}/{a}"));
Assert.Equal(26 + 300, restarted.UsedBy(domain.Id));
}
}
@@ -0,0 +1,155 @@
using System.Net;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using WebDrop.Core;
using WebDrop.FileApi;
using WebDrop.FileProtocol;
using static WebDrop.Api.Tests.ServerFixture;
namespace WebDrop.Api.Tests;
/// <summary>文件服务把新增的帧同步到外部存储(S3 协议):增量、可重试、删除后保留一段时间再清理。</summary>
public sealed class FileSyncTests : IAsyncLifetime
{
private const string Secret = "sync-test-secret-0123456789abcdef-xyz-0123";
private readonly string _dir = Path.Combine(Path.GetTempPath(), "webdrop-sync-" + Guid.NewGuid().ToString("N"));
private readonly FakeS3 _s3 = new();
private WebApplication _files = null!;
private HttpClient _http = null!;
private string _channel = "";
public async Task InitializeAsync()
{
await _s3.StartAsync();
_files = FileApiApp.Build([], b =>
{
b.Configuration["FileApi:DataDir"] = _dir;
b.Configuration["FileApi:Secret"] = Secret;
b.Configuration["FileApi:Backup:Enabled"] = "false"; // 定时任务不在测试里跑,测试手动触发
b.Configuration["FileApi:Backup:RetainDays"] = "30";
b.Configuration["FileApi:Backup:S3:Endpoint"] = _s3.Url;
b.Configuration["FileApi:Backup:S3:AccessKey"] = FakeS3.AccessKey;
b.Configuration["FileApi:Backup:S3:SecretKey"] = FakeS3.SecretKey;
b.Configuration["FileApi:Backup:S3:BasePath"] = "webdrop";
b.Configuration["FileApi:Backup:S3:Bucket"] = FakeS3.Bucket;
b.Configuration["urls"] = "http://127.0.0.1:0";
b.Logging.ClearProviders();
});
await _files.StartAsync();
_http = new HttpClient { BaseAddress = new Uri(_files.Services.GetRequiredService<IServer>().Features.Get<IServerAddressesFeature>()!.Addresses.First()) };
// 帧都存在一个文件域里:先用「创建」票据建一个
string space = Convert.ToHexStringLower(System.Security.Cryptography.RandomNumberGenerator.GetBytes(16));
using var create = new HttpRequestMessage(HttpMethod.Post, "/file/v1/domains") { Content = new StringContent("{}", System.Text.Encoding.UTF8, "application/json") };
create.Headers.Add("X-WD-Ticket", FileTokens.Sign(System.Text.Encoding.UTF8.GetBytes(Secret), new FileTicket(FileOp.Create, space, "*", 0, DateTimeOffset.UtcNow.AddMinutes(5).ToUnixTimeMilliseconds())));
using var created = await _http.SendAsync(create);
Assert.Equal(HttpStatusCode.Created, created.StatusCode);
_channel = System.Text.Json.JsonSerializer.Deserialize(await created.Content.ReadAsStringAsync(), FileProtocolJson.Default.FileDomainBody)!.Id;
}
public async Task DisposeAsync()
{
_http.Dispose();
await _files.StopAsync(); await _files.DisposeAsync(); await _s3.DisposeAsync();
try { Directory.Delete(_dir, recursive: true); } catch (IOException) { }
}
private static string Hex(string id) { Codec.TryDecodeExact(id, 16, out var b); return Convert.ToHexStringLower(b); }
private string Admin() => FileTokens.Sign(System.Text.Encoding.UTF8.GetBytes(Secret), new FileTicket(FileOp.Admin, "*", "*", 0, DateTimeOffset.UtcNow.AddMinutes(5).ToUnixTimeMilliseconds()));
private async Task<byte[]> PutAsync(string item, uint index, byte fill)
{
var frame = Frame(item, index, GatedEnc, EncPayload, fill);
using var request = new HttpRequestMessage(HttpMethod.Put, $"/file/v1/domains/{_channel}/items/{Hex(item)}/frames/{index}") { Content = new ByteArrayContent(frame) };
request.Headers.Add("X-WD-Ticket", Admin());
using var response = await _http.SendAsync(request);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
// 确认这个条目(已登记到主后端):没确认的待确认条目不同步
using var commit = new HttpRequestMessage(HttpMethod.Post, $"/file/v1/domains/{_channel}/items/{Hex(item)}/commit");
commit.Headers.Add("X-WD-Ticket", Admin());
using var committed = await _http.SendAsync(commit);
Assert.Equal(HttpStatusCode.NoContent, committed.StatusCode);
return frame;
}
private SyncWorker Worker => _files.Services.GetRequiredService<SyncWorker>();
private string Remote(string item, uint index) => $"webdrop/files/{_channel}/{Hex(item)}/{index:D10}";
[Fact]
public async Task NewFramesAreUploadedOnceAndTheRestResumesAfterAnOutage()
{
string a = NewId(), b = NewId();
var frames = new[] { await PutAsync(a, 0, 1), await PutAsync(a, 1, 2), await PutAsync(a, 2, 3), await PutAsync(b, 0, 4) };
// 外部存储暂时连不上:不会标记已同步,下次接着来
_s3.Down = true;
var down = await Worker.RunOnceAsync(CancellationToken.None);
Assert.Equal(0, down.Uploaded); Assert.True(down.Failed > 0);
Assert.Empty(_s3.Files);
_s3.Down = false;
var first = await Worker.RunOnceAsync(CancellationToken.None);
Assert.Equal(4, first.Uploaded);
Assert.Equal(frames[0], _s3.Files[Remote(a, 0)]);
Assert.Equal(frames[2], _s3.Files[Remote(a, 2)]);
Assert.Equal(frames[3], _s3.Files[Remote(b, 0)]);
// 增量:没有新内容就什么也不传;新来的帧只传新的
Assert.Equal(0, (await Worker.RunOnceAsync(CancellationToken.None)).Uploaded);
var more = await PutAsync(a, 3, 5);
var second = await Worker.RunOnceAsync(CancellationToken.None);
Assert.Equal(1, second.Uploaded);
Assert.Equal(more, _s3.Files[Remote(a, 3)]);
}
[Fact]
public async Task DeletedContentStaysInTheBackupUntilTheRetentionPeriodPasses()
{
string a = NewId(), b = NewId();
await PutAsync(a, 0, 1); await PutAsync(b, 0, 2);
await Worker.RunOnceAsync(CancellationToken.None);
Assert.True(_s3.Files.ContainsKey(Remote(a, 0)));
// 在文件服务上删掉 a:本地没了,外部备份里还在(误删可以找回)
using (var delete = new HttpRequestMessage(HttpMethod.Delete, $"/file/v1/domains/{_channel}/items/{Hex(a)}"))
{
delete.Headers.Add("X-WD-Ticket", Admin());
using var response = await _http.SendAsync(delete);
Assert.Equal(HttpStatusCode.NoContent, response.StatusCode);
}
var early = await Worker.RunOnceAsync(CancellationToken.None);
Assert.Equal(0, early.Purged);
Assert.True(_s3.Files.ContainsKey(Remote(a, 0)));
Assert.True(_s3.Files.ContainsKey(Remote(b, 0)));
// 删除记录满 30 天后,下一次同步才把外部备份里的也清掉(没删的 b 不受影响)
var log = Path.Combine(_dir, "sync", "deleted.log");
long old = DateTimeOffset.UtcNow.AddDays(-31).ToUnixTimeMilliseconds();
File.WriteAllLines(log, File.ReadAllLines(log).Select(l => l.Split('\t')[0] + "\t" + old));
var late = await Worker.RunOnceAsync(CancellationToken.None);
Assert.Equal(1, late.Purged);
Assert.False(_s3.Files.ContainsKey(Remote(a, 0)));
Assert.True(_s3.Files.ContainsKey(Remote(b, 0)));
Assert.Empty(File.ReadAllLines(log));
}
[Fact]
public async Task ContentDeletedBeforeItWasEverSyncedLeavesNothingToPurge()
{
string a = NewId();
await PutAsync(a, 0, 1);
using (var delete = new HttpRequestMessage(HttpMethod.Delete, $"/file/v1/domains/{_channel}/items/{Hex(a)}"))
{
delete.Headers.Add("X-WD-Ticket", Admin());
using var response = await _http.SendAsync(delete);
Assert.Equal(HttpStatusCode.NoContent, response.StatusCode);
}
var run = await Worker.RunOnceAsync(CancellationToken.None);
Assert.Equal(0, run.Uploaded);
Assert.Empty(_s3.Files);
Assert.False(File.Exists(Path.Combine(_dir, "sync", "deleted.log")));
}
}
Loaded 100 of 393 files, more files were not shown because too many files have changed in this diff. Show more