thepra/SocialPub
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

R9 Web Push, then the honest version: VAPID key, push subscriptions, encrypted pushes, 4.5.0 advertised

Browse Source 
The server's VAPID key (PushKeys, made on first need, in Mongo) is the instance API's vapid.public_key and every app's
vapid_key. /api/v1/push/subscription (POST, GET, PUT, DELETE) keeps one subscription per access token, so per persona;
every notification kept (never a filtered one) is queued as a SendPush job for the subscriptions whose alert and policy
admit it, and sent through the `push` client (the safe handler, https only) encrypted as RFC 8291 aes128gcm or the
aesgcm draft with an ES256 VAPID JWT. 404/410 delete the subscription, 429/5xx/timeouts retry, other answers drop the
push. Subscriptions end with their tokens (revocation, sessions ended, suspension, deletion, a restore); backups hold
neither keys nor subscriptions. The walk of every Mastodon 4.3-4.5 client route (docs/MASTODON-API.md) added
conversations/:id/unread, domain_blocks/preview, v2 notifications clear and PUT policy, /oauth/userinfo and stubs, so both
instance APIs now say 4.5.0 (compatible; PrivaPub <release>). The pasture gets a push sink (pushsink.test) and
scenarios/push.sh in batch A.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WpVCoDeM1qUTRp4fXQdfof
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-18 09:18:26 +02:00
1 parent 87529bca49
commit 9b792042e6
43 files changed
+2079 -49
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.gitignore
+1
View File
@@ -407,6 +407,7 @@ PrivaPub/media-store-incoming/
PrivaPub/media-store-backups/
tools/pasture/.publish/
tools/pasture/.flood/
tools/pasture/.pushsink/
.claude/worktrees/
tools/pasture/.ca/
tools/pasture/.state/
CLAUDE.md
+134 -11
View File
@@ -94,6 +94,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Infrastructure/
Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out
Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker
Push/ Web Push (R9): Vapid (the server's key, ES256 JWTs), WebPushCrypto (aes128gcm, aesgcm),
WebPushJob (SendPush, through the `push` client)
Ids/ PrivacyIds (day-only ids for personas and groups, arrival-ordered ids for remote posts)
Data/ Indexes (created at start), EntityMaps.Warm, Migrations/_NNN_*.cs
Cli/ AdminCommands (`PrivaPub admin promote|demote|role|create-root|move-persona|delete-root|reset-totp`,
@@ -130,6 +132,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Moderation/ ModerationLog (the staff's log, AdminAction), AccountModeration (silence, suspend, posts
removed), Restrictions (who is held back, in memory), ModerationFlags (posts' flags again)
Social/ FollowService (local in-process, remote Follow/Accept), Notifications, Announcements,
WebPush (who is pushed what; WebPushDispatcher on Notifications.Told),
FeaturedTags (a persona's featured hashtags, sent and received), Suggestions (who to follow)
Timelines/ Fanout (TimelineEntry rows, Mastodon's home rules), TimelineService
Privacy/ VisibilityPolicy (IsPublic expression, CanSee)
@@ -146,8 +149,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch
Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, lists, filters,
media, polls, reactions, reports, scheduled statuses, streaming, tags, trends, cdns,
provenance, custom emojis, featured tags, suggestions; the stubs are `StubsController`
in `SearchController.cs`
provenance, custom emojis, featured tags, suggestions, Web Push (`PushController`), OAuth
userinfo; the stubs are `StubsController` in `SearchController.cs`
Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex);
OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent)
Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, RootRemoval,
@@ -429,6 +432,87 @@ client's role route never does.
and `ProtectiveMerge` deletes those of a persona deleted since. Mastodon's account archive has no featured tags, notes or
endorsements, so the persona archive leaves them out.
## Web Push (completion program R9)
- **The key** (`Models/PushKeys.cs`, `Infrastructure/Push/Vapid.cs`): one P-256 VAPID key pair (RFC 8292), made the first
time something needs it and kept in Mongo (one document, id `vapid`; two processes at once keep the first one saved).
Its public key, the uncompressed point in URL-safe base64 with its padding (as Mastodon writes `VAPID_PUBLIC_KEY`), is
v2's `configuration.vapid.public_key`, an app's `vapid_key` (`POST /api/v1/apps`, `apps/verify_credentials`) and every
subscription's `server_key`.
- **Subscriptions** (`Models/Social/PushSubscription.cs`, a new collection, unique on `TokenId`): one per Mastodon access
token, as on Mastodon, so one persona's (the token's subject) and never its login's. Each holds the endpoint, the
client's `p256dh` and `auth` as it sent them, `standard`, the alerts turned on, the policy, and the token itself, which
the payload carries back. The alerts are Mastodon 4.5's names (`WebPush.Alerts`: `mention`, `quote`, `status`,
`reblog`, `follow`, `follow_request`, `favourite`, `poll`, `update`, `quoted_update`, `severed_relationships`,
`moderation_warning`, `annual_report`, `admin.sign_up`, `admin.report`) and Pleroma's `pleroma:emoji_reaction`; those
PrivaPub never notifies (the admin ones, `quoted_update`, `severed_relationships`, `annual_report`) are kept and never fire.
- **Routes** (`Api/Mastodon/Controllers/PushController.cs`, the `push` scope, a persona's token; Mastodon's parameters as
a form or JSON):
- `POST /api/v1/push/subscription`: `subscription[endpoint]` (an https URL; plain http only where
`Federation:AllowPlainHttp`, which production refuses), `subscription[keys][p256dh]` (a point on P-256),
`subscription[keys][auth]` (16 bytes), `subscription[standard]`, `data[alerts][<alert>]`, `data[policy]` (`all` by
default, `followed`, `follower`, `none`). It replaces the token's subscription and answers it; 400 without an
endpoint, 422 for an endpoint that is no https URL, keys that are no Web Push keys, or another policy.
- `GET`: the token's subscription, or 404 `Record not found`.
- `PUT`: `data[alerts]` and `data[policy]` replace what it had, as Mastodon's update does (an alert left out is off, a
policy left out is `all`); 404 without one.
- `DELETE`: `{}`, whether there was one or not.
- The answer is Mastodon's `WebPushSubscription`: `id` (a string), `endpoint`, `standard`, `alerts` (every alert, true
or false), `server_key`, `policy`.
- **What is pushed** (`Domain/Social/WebPush.cs`): `Notifications.Add` and `Notifications.Warn` tell `Notifications.Told`
of each notification they keep, never one the notification policy filtered or dropped nor one blocks or mutes hid.
`WebPushDispatcher` (a hosted service: the command line pushes nothing) hangs there and queues one `SendPush` job
(`JobKind.SendPush`, the last member; dedupe `push|<subscription>|<notification>`, its host the endpoint's) for each
subscription of that persona whose alert for the type is on and whose policy admits the sender: `followed`, an account
the persona follows; `follower`, one that follows it; the persona itself (its poll ended, the staff's warning) under
every policy but `none`. Nothing for a persona `Restrictions` holds back (suspended, its login banned).
- **Sending** (`Infrastructure/Push/WebPushJob.cs`). Everything is read again when the job runs: the subscription, its
persona (not held back, not deleted), its token (valid, unexpired, its subject the subscription's persona, or the
subscription goes), the notification (still the persona's, still pushable, still shown to it by
`MastodonMapper.Notifications`). The payload is Mastodon's JSON, at most 3993 bytes (`WebPushCrypto.MaxPayload`):
- `access_token`, the subscription's own token, never another's; `preferred_locale`, the persona's default posting
language, else the server's first; `notification_id`, PrivaPub's id (a string); `notification_type`, its Mastodon
name;
- `icon`, the sender's avatar through the media proxy; `title`, Mastodon's English subjects ("You were mentioned by
…", "… favorited your post", …); `body`, the post's warning, else its words, else the sender's bio, without tags, 140
characters at most.
- **Encryption** (`WebPushCrypto`, System.Security.Cryptography only: ECDiffieHellman on P-256, HKDF-SHA-256, AesGcm),
a fresh key pair and salt per message. A `standard` subscription gets RFC 8291's `aes128gcm` (RFC 8188's header:
salt, record size 4096, our public key as the key id; one record, delimiter 0x02) with `Authorization: vapid
t=<jwt>, k=<key>`. Any other gets the draft Mastodon still sends by default, `aesgcm` (two bytes of padding length
first) with `Encryption: salt=…`, `Crypto-Key: dh=<our key>;p256ecdsa=<the VAPID key>` and `Authorization: WebPush
<jwt>`. Both carry `Content-Type: application/octet-stream`, `TTL: 172800` and `Urgency: normal`, as Mastodon's do.
- **The JWT** (`VapidKey.Token`): ES256, `aud` the endpoint's origin, `exp` twelve hours ahead (RFC 8292 allows a
day), `sub` `mailto:` the contact email the administrator gave, else the server's address (`mailto:privapub@<host>`
for a test server on plain http). Never a persona.
- **Answers:** 2xx is done; 404 and 410 delete the subscription; 408, 429, a 5xx, a timeout or a broken connection retry
on the jobs' backoff (six attempts, about forty minutes); any other answer, a redirect, or an endpoint that does not
resolve to a public address drops the push, logged by the push service's host alone. Four run at a time, four per host.
- **decePub:** the first-party client's exchanged persona tokens may hold `push` too (`PersonaExchange.FirstPartyScopes`),
so it can subscribe like any other client.
- **Lifecycle:** a subscription goes with its token: `/oauth/revoke` (an OpenIddict handler after the revocation),
`RootSessions.Revoke` (a password changed or recovered, two-factor turned on, signing out everywhere, a ban, a deleted
login), `PersonaRemoval`, a suspension (`AccountModeration`), `admin move-persona`, and any token found unusable when a
push is sent. A new subscription made with the same token replaces it.
- **Backups:** `PushSubscription` and `PushKeys` are never in a backup (`ServerBackup.Excluded`). A restore leaves the
live key as it is, so the key the push services know never changes, and deletes every subscription with the tokens
(`ProtectiveMerge`); clients subscribe again once signed in.
- **The rest of the honest version** (`docs/MASTODON-API.md`, the walk of every Mastodon 4.3, 4.4 and 4.5 client route),
built with it:
- `POST /api/v1/conversations/:id/unread` (`ConversationState.MarkedUnread`, until it is read);
- `GET /api/v1/domain_blocks/preview` (`following_count` and `followers_count` the block would take, counted as
`BlockDomain` takes them);
- `POST /api/v2/notifications/clear` and `PUT /api/v2/notifications/policy`;
- `GET`/`POST /oauth/userinfo` (the `profile` scope: `iss`, `sub` the actor, `name`, `preferred_username`, `profile`,
`picture`; `userinfo_endpoint` in the server's metadata) and `verify_credentials` for a `profile` token
(`Scope("read:accounts|profile")`: a scope written `a|b` is either);
- stubs: `GET /api/v1/annual_reports/:year` (none) and `POST …/:year/read` (`{}`), `POST /api/v1/accounts` (403, as
Mastodon with registrations closed: they are cut), `POST /api/v1/emails/confirmations` (403) and
`GET /api/v1/emails/check_confirmation` (`true`), `GET /api/v1/peers/search` (`[]`), and 4.4's
`GET /api/v1_alpha/async_refreshes/:id` (404: PrivaPub never sends `Mastodon-Async-Refresh`).
Both instance APIs then say `4.5.0 (compatible; PrivaPub <release>)` (Mastodon invariant 6).
## Custom emoji (completion program R7)
- **The server's own emoji** (`Models/Post/LocalEmoji.cs`, a collection of its own): a shortcode (`[a-zA-Z0-9_]{2,64}`,
@@ -487,8 +571,12 @@ client's role route never does.
1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only
to public addresses; redirects are followed by hand (three at most, each re-checked); bodies are capped at 1 MB;
only JSON media types are read; a refused URL is not asked again for five minutes. Never create another
`HttpClient` for federation. The only other outbound traffic is SMTP and `GeoUpdater`'s monthly DB-IP Lite download
(its own `geo` client, a fixed HTTPS host, size-capped, the file checked before it is swapped in).
`HttpClient` for federation. The only other outbound traffic is SMTP, `GeoUpdater`'s monthly DB-IP Lite download
(its own `geo` client, a fixed HTTPS host, size-capped, the file checked before it is swapped in), and Web Push (R9):
the named `push` client, on `SafeHttpHandlerFactory`'s handler (it resolves the name itself and connects only to
public addresses, with federation's test-network exceptions; no redirects, cookies or proxy), https only (plain http
only where `Federation:AllowPlainHttp`), 5 s to connect and 15 s a request. It POSTs encrypted payloads to the
endpoints clients subscribed with, and reads nothing back but the status.
2. **Every fetch is signed by the instance actor** (`privapub`), never by a persona; deliveries are signed by the acting
avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies). Inbound,
an RFC 9421 signature (`Signature-Input`) is verified too, its target against our public address. A persona's own
@@ -605,9 +693,14 @@ client's role route never does.
5. Unsupported features answer empty lists or 422 with a message, never 404 or 500, so clients degrade. Any `/api`
error that leaves without a body (a challenge, a 404 from routing, a 429) gets Mastodon's `{"error": ...}` from
`UseMastodonErrorBodies`.
6. Advertise `4.2.0 (compatible; PrivaPub)` until streaming and Web Push exist too; `api_versions.mastodon = 7` stays,
because clients show quotes from it, so everything a 4.3+ client calls because of it must answer: grouped
notifications (`/api/v2/notifications`, its unread count, groups, accounts and dismiss) and the notification policy.
6. **Advertise `4.5.0 (compatible; PrivaPub <release>)`** in both instance APIs (`InstanceController.Version`, since R9;
the release is the build's tag without its `v`, else its ref or commit), with `api_versions.mastodon = 7`, 4.5's own.
The number is the walk's (`docs/MASTODON-API.md`): every client route of Mastodon 4.3, 4.4 and 4.5 is served or
answers as a named stub, and none a client relies on is missing. A route Mastodon adds, or one PrivaPub stops serving,
goes there first, and the version follows the walk, never a wish; `tools/smoke/mastodon-api.sh` checks it on every
deploy. Clients branch on both numbers, so everything a 4.3+ client calls must answer: grouped notifications
(`/api/v2/notifications`, its unread count, groups, accounts, dismiss and clear), quotes, Web Push, streaming and the
notification policy.
The policy is real (`Domain/Social/NotificationPolicies.cs`, applied in `Notifications.Add`): accepted by default,
it filters (into a `NotificationRequest` per account, out of every list and count unless `include_filtered`) or drops
what strangers, accounts that do not follow the persona (or only for three days), accounts newer than 30 days,
@@ -624,6 +717,8 @@ client's role route never does.
private notes (shown to it alone, never federated), dismissed suggestions. Suggestions never offer the persona itself
or a sibling persona of its login, and `familiar_followers` answers no one for every account, since it would tell who
follows whom here.
9. **A Web Push subscription is its token's** (R9): one per access token, read, changed and deleted only with that token,
pushed only its persona's notifications, and its payload carries only that token. It ends when the token does.
## Media invariants
@@ -766,6 +861,12 @@ client's role route never does.
decided in `docs/ROADMAP.md` ("Owner decisions on what PrivaPub reveals"). Anything new that tells another server
something about an avatar gets the same treatment: ask, then record it there.
- **Location-ranged posts never federate.**
- **Web Push tells a push service when, never what** (R9). Every payload is encrypted to the subscribing client's own
keys (RFC 8291, or the aesgcm draft), so the push service (Google's, Mozilla's, Apple's, a UnifiedPush distributor)
learns only that this server has something for that endpoint, when, and how big: never the words, the sender or the
persona. The VAPID JWT names the push service's origin and the server's contact (the contact email, or the server's
address), never a persona. Subscriptions are per token, so per persona: none receives a sibling's notifications, and
nothing stored pairs two personas of one login.
- **Statistics name servers, never people** (owner decisions on statistics, 2026-10-03). Every interaction is recorded
through `IInteractionLedger` (`Infrastructure/Statistics`) as an `InteractionEvent` (90 days), but an event never
holds a root, persona or group id, an activity id, an inbox URL, a remote actor URI or a sender IP:
@@ -868,7 +969,8 @@ client's role route never does.
**Never in a backup** (`ServerBackup.Excluded`): `InteractionSalt` (owner rule), `Job` and `Delivery` (work in flight),
`EmailRecovery` and `EmailVerification`, `openiddict.tokens` and `.authorizations` (sessions), `MaintenanceLock`, `ServerSetup` (a restore
never opens setup again), and `AppConfiguration` (its SMTP password; it is made again from appsettings at boot). One
never opens setup again), `AppConfiguration` (its SMTP password; it is made again from appsettings at boot),
`PushSubscription` (sessions' too, R9) and `PushKeys` (the VAPID key: a restore keeps the live one). One
backup or restore runs at a time
(`MaintenanceLock`, a heartbeat document; a holder silent for 2 minutes is taken over), and the media janitor purges
nothing meanwhile. `BackupScheduler` backs up at `Backups:NightlyAt` (03:30 UTC), or at once when it missed the night;
@@ -895,7 +997,8 @@ client's role route never does.
nothing holds any more goes there too); the staff's log (`AdminAction`) is the union, so a restore never loses an
entry, its own request included, and the server's settings and custom emoji (`LocalEmoji`, R7) are P's: the live
settings and emoji outlive a restore, and an emoji deleted since stays deleted;
5. every session ends (a new `SessionStamp` for each root, persona tokens deleted), circuits close, and a
5. every session ends (a new `SessionStamp` for each root, persona tokens deleted, and every Web Push subscription with
them, R9: clients subscribe again, to the same VAPID key), circuits close, and a
`RestoreRecord` (never in a backup) tells what happened (`admin restore --status`).
Each attempt redoes everything. Refused before anything changed, it is abandoned and recorded, and the server boots as
@@ -1154,6 +1257,25 @@ tools/pasture/run.sh down # removes e
DM; her endorsement of him and her note on him are hers alone (feat_bob, a sibling persona of the same login, sees
neither, and `GET /api/v1/accounts/:id/endorsements` answers nobody else), and feat_bob is never suggested to her; the
stubs (translate 422, annual reports, the admin API 403) answer.
- **Web Push (`scenarios/push.sh`, needs mastodon and pushsink; after featured in batch A):** push_alice subscribes with
the sink's keys; mastouser's mention reaches the sink as aes128gcm, its VAPID JWT by the instance API's key for the
sink's origin with TTL and Urgency, and decrypts to a `mention` with her own access token and her notification's id;
the mention alert turned off stops it while his favourite is pushed; `policy=followed` stops him until she follows him;
`standard=false` is aesgcm (Encryption, Crypto-Key, `WebPush`) and decrypts too, and the new subscription replaces the
old; push_bob, a sibling persona of the same login, never gets hers nor she his; DELETE stops the pushes; the sink
answering 410 removes the subscription; a token revoked at `/oauth/revoke` is pushed nothing; no subscription is left.
Both instance APIs say 4.5.0 and give the key an app registration gives.
- **pushsink (`peers/pushsink.sh`, `pushsink/pushsink.cs`, as `pushsink.test`):** a Web Push service of the pasture's
own, a .NET file-based app published once into `.pushsink` like flood. It makes each subscription's keys itself, checks
what a push service checks (the VAPID JWT: ES256 by the server's key, the instance API's or the one the scenario gave
it; `aud` its own origin; `exp` ahead and at most a day; a `mailto:` or `https:` `sub`; TTL), decrypts `aes128gcm` and
`aesgcm` with its own code, not PrivaPub's, and keeps what it took. Its API, behind Caddy: `POST /subscriptions`
(`{"server_key"}` optional) answers `{id, endpoint, keys: {p256dh, auth}}`; `GET /subscriptions/{id}`;
`POST /subscriptions/{id}/answer` (`{"status": 410, "times": 1}`, for the next pushes); `GET` and `DELETE
/subscriptions/{id}/received` (each push: when, the status answered, `ok` and the errors, headers, the JWT's claims, the
decrypted payload); `POST /push/{id}` is the push service (201, 403 for a VAPID failure, 400 for the rest, 404 for no
such subscription); `/health`. PrivaPub reaches it through Caddy as every peer, since its `push` client honours
`appsettings.Pasture.json`'s Federation exceptions; nothing new is configured. In memory only.
- **Mailpit (1.31.4, `peers/mailpit.sh`, as `mailpit.test`):** the pasture's mail server. `appsettings.Pasture.json`
sends PrivaPub's mail to `pasture-mailpit:1025` (plain SMTP, any login accepted) and its recovery links to
`https://decepub.test`. `scenarios/accounts.sh` reads what arrived through Mailpit's API: an address set with the
@@ -1318,8 +1440,9 @@ the owner's GoToSocial account.**
the server's own pre-deploy backup (`PrivaPub admin backup --kind pre-deploy --db-only`, run from the new build before
its migrations, verified, and checked to hold no statistics salt; no deploy while `restore.json` waits),
stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo,
Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration, client credentials,
discovery, instance, public timeline). Then it checks that what production should be running is running:
Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration with its `vapid_key`,
client credentials, discovery, both instance APIs with the advertised version and the VAPID key, public timeline).
Then it checks that what production should be running is running:
- NodeInfo and the instance API agree on registrations (closed, invitations only);
- `/stargazing` says the crawler is on;
- nothing signs in (owner decision 2026-10-17): the deploy's own login, `deploy-smoke`, gave its @thepra to the
FEDERATION.md
+8
View File
@@ -456,6 +456,14 @@ PrivaPub keeps statistics about servers, never about accounts (see `/stargazing`
- **Opting out:** in robots.txt, the crawler obeys the group `PrivaPub-Stargazer`, then `PrivaPub`, then `*`. A
robots.txt that answers with a server error or times out also keeps it out, as do this server's domain blocks.
## Web Push
Not federation, but traffic an operator may see: when a client signed in as a persona subscribes to Web Push (Mastodon's
`/api/v1/push/subscription`), PrivaPub POSTs its notifications to the push service the client named, encrypted to the
client (RFC 8291 `aes128gcm`, or the older `aesgcm` draft Mastodon still sends) and carrying a VAPID JWT (RFC 8292) signed
by the server's own P-256 key, the one `/api/v2/instance` gives as `configuration.vapid.public_key`. No HTTP signature
and no ActivityPub; https endpoints on public addresses only, no redirects. The push service learns when, never what.
## Local-only posts
Posts with a location (shown to nearby users of this server) never leave the server, in any form.
PrivaPub/Api/Mastodon/Auth/OAuthSetup.cs
+17
View File
@@ -72,6 +72,23 @@ namespace PrivaPub.Api.Mastodon.Auth
options.AddEventHandler<ApplyAuthorizationResponseContext>(builder => builder.UseInlineHandler(OutOfBandCode)
.SetOrder(OpenIddictServerAspNetCoreHandlers.Authentication.ProcessFormPostResponse.Descriptor.Order - 1_000)
.SetType(OpenIddictServerHandlerType.Custom));
// a revoked token takes its Web Push subscription with it (R9), once OpenIddict has revoked it
options.AddEventHandler<HandleRevocationRequestContext>(builder => builder.UseInlineHandler(async context =>
{
if (context.GenericTokenPrincipal?.GetTokenId() is { Length: > 0 } tokenId)
await PrivaPub.Domain.Social.WebPush.ForgetToken(tokenId);
})
.SetOrder(OpenIddictServerHandlers.Revocation.RevokeToken.Descriptor.Order + 1_000)
.SetType(OpenIddictServerHandlerType.Custom));
// Mastodon 4.4's userinfo (Controllers/UserinfoController), named in the authorization server's metadata
options.AddEventHandler<HandleConfigurationRequestContext>(builder => builder.UseInlineHandler(context =>
{
if (context.Issuer != default)
context.UserInfoEndpoint = new Uri(context.Issuer, Controllers.UserinfoController.Path.TrimStart('/'));
return default;
})
.SetOrder(OpenIddictServerHandlers.Discovery.AttachEndpoints.Descriptor.Order + 1_000)
.SetType(OpenIddictServerHandlerType.Custom));
options.AddEventHandler<ApplyTokenResponseContext>(builder => builder.UseInlineHandler(context =>
{
if (context.Response.AccessToken != default)
PrivaPub/Api/Mastodon/Auth/PersonaExchange.cs
+4 -1
View File
@@ -27,6 +27,9 @@ namespace PrivaPub.Api.Mastodon.Auth
public static string FirstPartyClientId(IConfiguration configuration) =>
configuration["ClientApi:FirstPartyClientId"] is { Length: > 0 } clientId ? clientId : "decepub";
// what the first-party client's persona tokens may hold: push since R9, so decePub can subscribe to Web Push
public static readonly string[] FirstPartyScopes = { "read", "write", "follow", "push" };
//the first-party application: public (a browser keeps no secret), token exchange only, no redirect
public static async Task EnsureFirstPartyClient(IServiceProvider services, CancellationToken token)
{
@@ -46,7 +49,7 @@ namespace PrivaPub.Api.Mastodon.Auth
Permissions.GrantTypes.TokenExchange
}
};
foreach (var scopeName in new[] { "read", "write", "follow" })
foreach (var scopeName in FirstPartyScopes)
descriptor.Permissions.Add(Permissions.Prefixes.Scope + scopeName);
var existing = await applications.FindByClientIdAsync(clientId, token);
PrivaPub/Api/Mastodon/Auth/TokenController.cs
+1 -1
View File
@@ -70,7 +70,7 @@ namespace PrivaPub.Api.Mastodon.Auth
var identity = new ClaimsIdentity(TokenValidationParameters.DefaultAuthenticationType, Claims.Name, Claims.Role);
identity.SetClaim(Claims.Subject, avatar.ID);
identity.SetClaim(Claims.Name, avatar.UserName);
identity.SetScopes(MastodonScopes.Parse(request.Scope).Intersect(new[] { "read", "write", "follow" }));
identity.SetScopes(MastodonScopes.Parse(request.Scope).Intersect(PersonaExchange.FirstPartyScopes));
identity.SetDestinations(_ => new[] { Destinations.AccessToken });
return SignIn(new ClaimsPrincipal(identity), OpenIddictServerAspNetCoreDefaults.AuthenticationScheme);
}
PrivaPub/Api/Mastodon/Controllers/AccountsController.cs
+24 -1
View File
@@ -47,7 +47,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
_outbox = outbox;
}
[HttpGet("/api/v1/accounts/verify_credentials"), Scope("read:accounts")]
// Mastodon 4.4: a token with the profile scope alone may read it too
[HttpGet("/api/v1/accounts/verify_credentials"), Scope("read:accounts|profile")]
public async Task<IActionResult> VerifyCredentials(CancellationToken token) => Json(await _mapper.Local(Me, withSource: true, token));
[HttpPatch("/api/v1/accounts/update_credentials"), Scope("write:accounts"), EnableRateLimiting(RateLimiting.Uploads), RequestSizeLimit(20 * 1024 * 1024),
@@ -398,6 +399,28 @@ namespace PrivaPub.Api.Mastodon.Controllers
public async Task<IActionResult> DomainBlocks(CancellationToken token) =>
Json((await DB.Default.Find<AccountDomainBlock>().Match(b => b.AvatarId == MyId).Sort(b => b.Domain, Order.Ascending).ExecuteAsync(token)).Select(b => b.Domain).ToList());
// Mastodon's preview of a server block: how many of the accounts the persona follows, and of those following it, the
// block would take away, counted as BlockDomain takes them (an account on the server or a subdomain, or whose handle
// is)
[HttpGet("/api/v1/domain_blocks/preview"), Scope("write:blocks")]
public async Task<IActionResult> PreviewDomainBlock(CancellationToken token)
{
var domain = Params.Get("domain")?.Trim().ToLowerInvariant();
if (string.IsNullOrEmpty(domain))
return Error(StatusCodes.Status400BadRequest, "Param is missing or the value is empty: domain");
if (Uri.CheckHostName(domain) != UriHostNameType.Dns)
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Domain is invalid");
var followings = await _dbEntities.Followings.Match(f => f.AvatarId == MyId && !f.TargetIsLocal && f.State == FollowState.Accepted)
.ExecuteAsync(token);
var followers = await _dbEntities.Followers.Match(f => f.LocalActorId == MyId && f.IsAccepted).ExecuteAsync(token);
var handles = await Hidden.HandleDomains(followings.Select(f => f.TargetActorURI).Concat(followers.Select(f => f.ActorURI)).ToList(), token);
return Json(new
{
following_count = followings.Count(f => Hidden.Blocks(domain, f.TargetActorURI, handles.GetValueOrDefault(f.TargetActorURI))),
followers_count = followers.Count(f => Hidden.Blocks(domain, f.ActorURI, handles.GetValueOrDefault(f.ActorURI)))
});
}
[HttpPost("/api/v1/domain_blocks"), Scope("write:blocks")]
public async Task<IActionResult> BlockDomain(CancellationToken token)
{
PrivaPub/Api/Mastodon/Controllers/AppsController.cs
+6 -3
View File
@@ -16,10 +16,12 @@ namespace PrivaPub.Api.Mastodon.Controllers
public class AppsController : MastodonController
{
readonly IOpenIddictApplicationManager _applications;
readonly PrivaPub.Infrastructure.Push.IVapid _vapid;
public AppsController(IOpenIddictApplicationManager applications)
public AppsController(IOpenIddictApplicationManager applications, PrivaPub.Infrastructure.Push.IVapid vapid)
{
_applications = applications;
_vapid = vapid;
}
[HttpPost("/api/v1/apps"), AllowAnonymous]
@@ -69,7 +71,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
RedirectUris = redirects,
RedirectUri = string.Join("\n", redirects),
ClientId = descriptor.ClientId,
ClientSecret = descriptor.ClientSecret
ClientSecret = descriptor.ClientSecret,
VapidKey = (await _vapid.Key(token)).ForApi//Web Push's server key, as Mastodon still gives it here
});
}
@@ -88,7 +91,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
website = properties.TryGetValue("website", out var website) ? website.GetString() : default,
scopes = User.GetScopes().ToList(),
redirect_uris = (await _applications.GetRedirectUrisAsync(application, token)).ToList(),
vapid_key = string.Empty
vapid_key = (await _vapid.Key(token)).ForApi
});
}
PrivaPub/Api/Mastodon/Controllers/InstanceController.cs
+18 -3
View File
@@ -28,7 +28,19 @@ namespace PrivaPub.Api.Mastodon.Controllers
// are enforced.
public class InstanceController : MastodonController
{
public const string Version = "4.2.0 (compatible; PrivaPub)";
// The Mastodon version this API answers as, then PrivaPub's own release (its tag, or the commit a build came from).
// 4.5.0 since R9 (2026-10-18): every client route of Mastodon 4.3, 4.4 and 4.5 is served or answers as a documented
// stub (docs/MASTODON-API.md), streaming and Web Push included. api_versions.mastodon is 4.5's own, 7.
public const string MastodonVersion = "4.5.0";
public static readonly string Version = Advertised(string.IsNullOrEmpty(BuildInfo.Ref) ? BuildInfo.Commit : BuildInfo.Ref);
static string Advertised(string release)
{
var own = release?.Trim();
if (own is { Length: > 1 } && own[0] == 'v' && char.IsDigit(own[1]))
own = own[1..];//a tag v1.27.0 is release 1.27.0
return string.IsNullOrEmpty(own) ? $"{MastodonVersion} (compatible; PrivaPub)" : $"{MastodonVersion} (compatible; PrivaPub {own})";
}
const int ActivityWeeks = 12;
static readonly TimeSpan ActivityLifetime = TimeSpan.FromHours(1);
static volatile CachedActivity _activity;
@@ -39,14 +51,16 @@ namespace PrivaPub.Api.Mastodon.Controllers
readonly IOptionsMonitor<RegistrationOptions> _registrations;
readonly IInstanceSettings _settings;
readonly MastodonMapper _mapper;
readonly PrivaPub.Infrastructure.Push.IVapid _vapid;
public InstanceController(ILocalActorService localActors, IOptionsMonitor<RegistrationOptions> registrations, IInstanceSettings settings,
MastodonMapper mapper)
MastodonMapper mapper, PrivaPub.Infrastructure.Push.IVapid vapid)
{
_localActors = localActors;
_registrations = registrations;
_settings = settings;
_mapper = mapper;
_vapid = vapid;
}
bool Open => _registrations.CurrentValue.IsOpen;
@@ -165,7 +179,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
statuses = Statuses,
media_attachments = MediaAttachments,
polls = Polls,
translation = new { enabled = false }
translation = new { enabled = false },
vapid = new { public_key = (await _vapid.Key(token)).ForApi }//Web Push (R9): the key every push is signed with
},
registrations = new { enabled = Open, approval_required = false, message = Open ? default : RegistrationOptions.ClosedMessage },
contact = new { email = Settings.ContactEmail ?? string.Empty, account = await ContactAccount(Settings, token) },
PrivaPub/Api/Mastodon/Controllers/NotificationRequestsController.cs
+1 -1
View File
@@ -69,7 +69,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
[HttpGet("/api/v2/notifications/policy"), Scope("read:notifications")]
public async Task<IActionResult> Policy(CancellationToken token) => Json(await V2(await NotificationPolicies.Of(MyId, token), token));
[HttpPatch("/api/v2/notifications/policy"), Scope("write:notifications")]
[HttpPatch("/api/v2/notifications/policy"), HttpPut("/api/v2/notifications/policy"), Scope("write:notifications")]
public async Task<IActionResult> SetPolicy(CancellationToken token)
{
var policy = await NotificationPolicies.Of(MyId, token);
PrivaPub/Api/Mastodon/Controllers/PushController.cs
+169
View File
@@ -0,0 +1,169 @@
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Options;
using MongoDB.Driver;
using MongoDB.Entities;
using OpenIddict.Abstractions;
using OpenIddict.Validation.AspNetCore;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Social;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Push;
using PrivaPub.Models.Social;
namespace PrivaPub.Api.Mastodon.Controllers
{
// Mastodon's Web Push API (completion program R9): one subscription per access token, so per persona, made, read, changed
// and deleted with the token itself (the `push` scope). Its parameters as Mastodon takes them, as a form or JSON:
// subscription[endpoint], subscription[keys][p256dh], subscription[keys][auth], subscription[standard] (4.4: RFC 8291
// and RFC 8292, else the aesgcm draft), data[alerts][<type>] and data[policy] (all, followed, follower, none). Its answer is
// Mastodon's WebPushSubscription. What is pushed, and how, is WebPushDispatcher's and WebPushJob's.
public class PushController : MastodonController
{
const int MaxEndpointLength = 2048;
readonly IVapid _vapid;
readonly IOptionsMonitor<FederationOptions> _federation;
public PushController(IVapid vapid, IOptionsMonitor<FederationOptions> federation)
{
_vapid = vapid;
_federation = federation;
}
string TokenId => User.GetTokenId();
// a new subscription replaces the one this token had, as on Mastodon
[HttpPost("/api/v1/push/subscription"), Scope("push")]
public async Task<IActionResult> Create(CancellationToken token)
{
var endpoint = Params.Get("subscription[endpoint]")?.Trim();
if (string.IsNullOrEmpty(endpoint))
return Error(StatusCodes.Status400BadRequest, "Param is missing or the value is empty: subscription");
if (endpoint.Length > MaxEndpointLength || !Uri.TryCreate(endpoint, UriKind.Absolute, out var uri) || !WebPushJob.IsAllowed(uri, _federation.CurrentValue))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Endpoint is not a valid URL");
var p256dh = Params.Get("subscription[keys][p256dh]")?.Trim();
var auth = Params.Get("subscription[keys][auth]")?.Trim();
if (string.IsNullOrEmpty(p256dh) || string.IsNullOrEmpty(auth))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Key p256dh can't be blank, Key auth can't be blank");
if (!WebPushCrypto.AreUsable(WebPushCrypto.FromBase64Url(p256dh), WebPushCrypto.FromBase64Url(auth)))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Keys are not valid Web Push keys");
var (alerts, policy, invalid) = Data();
if (invalid != default)
return invalid;
var tokenId = TokenId;
var accessToken = await AccessToken();
if (string.IsNullOrEmpty(tokenId) || string.IsNullOrEmpty(accessToken))
return Error(StatusCodes.Status401Unauthorized, "The access token is invalid");
var subscription = new PushSubscription
{
AvatarId = MyId,
TokenId = tokenId,
AccessToken = accessToken,
Endpoint = endpoint,
P256dh = p256dh,
Auth = auth,
Standard = Params.Bool("subscription[standard]") == true,
Alerts = alerts,
Policy = policy
};
// (two at once with one token: the unique index on the token keeps one, and the later replaces it)
for (var attempt = 0; ; attempt++)
{
await WebPush.ForgetToken(tokenId);
try
{
subscription.ID = default;
await DB.Default.SaveAsync(subscription, token);
break;
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey && attempt < 2)
{
}
}
return Json(await View(subscription, token));
}
[HttpGet("/api/v1/push/subscription"), Scope("push")]
public async Task<IActionResult> Show(CancellationToken token) =>
await Mine(token) is { } subscription ? Json(await View(subscription, token)) : NotFoundError();
// Mastodon's update: the data given replaces what was there (alerts left out are off, the policy left out is all)
[HttpPut("/api/v1/push/subscription"), Scope("push")]
public async Task<IActionResult> Update(CancellationToken token)
{
var subscription = await Mine(token);
if (subscription == default)
return NotFoundError();
var (alerts, policy, invalid) = Data();
if (invalid != default)
return invalid;
subscription.Alerts = alerts;
subscription.Policy = policy;
subscription.UpdatedAt = DateTime.UtcNow;
await DB.Default.Update<PushSubscription>().MatchID(subscription.ID)
.Modify(s => s.Alerts, alerts)
.Modify(s => s.Policy, policy)
.Modify(s => s.UpdatedAt, subscription.UpdatedAt)
.ExecuteAsync(token);
return Json(await View(subscription, token));
}
[HttpDelete("/api/v1/push/subscription"), Scope("push")]
public async Task<IActionResult> Delete(CancellationToken token)
{
await WebPush.ForgetToken(TokenId);
return Json(new { });
}
async Task<PushSubscription> Mine(CancellationToken token)
{
var tokenId = TokenId;
var avatarId = MyId;
return string.IsNullOrEmpty(tokenId) ? default
: await DB.Default.Find<PushSubscription>().Match(s => s.TokenId == tokenId && s.AvatarId == avatarId).ExecuteFirstAsync(token);
}
// data[alerts][<type>] (each true or false; unknown types are ignored, as Mastodon permits only its own) and
// data[policy]
(List<string> Alerts, PushPolicy Policy, IActionResult Invalid) Data()
{
var alerts = WebPush.Alerts.Where(alert => Params.Bool($"data[alerts][{alert}]") == true).ToList();
var given = Params.Get("data[policy]");
if (string.IsNullOrWhiteSpace(given))
return (alerts, PushPolicy.All, default);
return WebPush.ParsePolicy(given) is { } policy
? (alerts, policy, default)
: (alerts, PushPolicy.All, Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Policy is not included in the list"));
}
// the bearer token of this request, wherever the client put it (OpenIddict takes the header, a form field or the
// query): the payload carries it back, as Mastodon's does
async Task<string> AccessToken()
{
var stored = await HttpContext.GetTokenAsync(OpenIddictValidationAspNetCoreDefaults.AuthenticationScheme,
OpenIddictValidationAspNetCoreConstants.Tokens.AccessToken);
if (!string.IsNullOrEmpty(stored))
return stored;
var header = Request.Headers.Authorization.ToString();
if (header.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
return header["Bearer ".Length..].Trim();
return Params.Get("access_token");
}
// Mastodon's WebPushSubscription
async Task<object> View(PushSubscription subscription, CancellationToken token) => new
{
id = subscription.ID,
endpoint = subscription.Endpoint,
standard = subscription.Standard,
alerts = WebPush.Alerts.ToDictionary(alert => alert, alert => subscription.Alerts.Contains(alert)),
server_key = (await _vapid.Key(token)).ForApi,
policy = WebPush.Name(subscription.Policy)
};
}
}
PrivaPub/Api/Mastodon/Controllers/SearchController.cs
+29 -6
View File
@@ -140,9 +140,6 @@ namespace PrivaPub.Api.Mastodon.Controllers
public class StubsController : MastodonController
{
[HttpGet("/api/v1/push/subscription"), Scope("push")]
public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found");
// Routes Mastodon answers that we have nothing behind: an empty or default answer lets clients degrade, where a
// 404 breaks some of them (Mastodon invariant 5)
[HttpGet("/api/v1/timelines/link"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
@@ -163,9 +160,35 @@ namespace PrivaPub.Api.Mastodon.Controllers
[HttpPost("/api/v1/statuses/{id}/translate"), Scope("read:statuses")]
public IActionResult Translate(string id) => Error(StatusCodes.Status422UnprocessableEntity, "Translation is not available on this server");
// Mastodon 4.3's yearly reports (Wrapstodon): none are made here, so there are never any to show
[HttpGet("/api/v1/annual_reports"), Scope("read:accounts")]
public IActionResult AnnualReports() => Json(new { annual_reports = Array.Empty<object>(), accounts = Array.Empty<object>(), statuses = Array.Empty<object>() });
// Mastodon 4.3's yearly reports (Wrapstodon): none are made here, so there are never any to show, for any year (4.4's
// show), and marking one read changes nothing
[HttpGet("/api/v1/annual_reports"), HttpGet("/api/v1/annual_reports/{year}"), Scope("read:accounts")]
public IActionResult AnnualReports(string year) => Json(new { annual_reports = Array.Empty<object>(), accounts = Array.Empty<object>(), statuses = Array.Empty<object>() });
[HttpPost("/api/v1/annual_reports/{year}/read"), Scope("write:accounts")]
public IActionResult ReadAnnualReport(string year) => Json(new { });
// Registration through the API is cut (owner decision 2026-10-17): a login is made by setup or an invitation, on
// /clientapi. Mastodon answers this when its registrations are closed; its e-mail confirmation routes follow, since no
// account here waits for one
[HttpPost("/api/v1/accounts"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Register() => Error(StatusCodes.Status403Forbidden, "This action is not allowed");
[HttpPost("/api/v1/emails/confirmations"), Scope("write:accounts")]
public IActionResult ResendConfirmation() =>
Error(StatusCodes.Status403Forbidden, "This method is only available while the e-mail is awaiting confirmation");
[HttpGet("/api/v1/emails/check_confirmation"), Scope("read:accounts")]
public IActionResult CheckConfirmation() => Json(true);
// the peers' names are not published (/instance/peers stays empty), so a search of them finds none
[HttpGet("/api/v1/peers/search"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult SearchPeers() => Json(Array.Empty<string>());
// Mastodon 4.4's asynchronous refreshes are announced by a Mastodon-Async-Refresh header, which PrivaPub never sends
// (a thread's replies are read again in the background and simply show up): there is never one to ask about
[HttpGet("/api/v1_alpha/async_refreshes/{id}"), Scope("read")]
public IActionResult AsyncRefresh(string id) => NotFoundError();
// the Mastodon admin API is cut: moderation stays on /clientapi (ModeratorController, AdminController), so every
// admin route answers as Mastodon answers an account that may not use it, whoever asks
PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs
+13 -1
View File
@@ -177,6 +177,18 @@ namespace PrivaPub.Api.Mastodon.Controllers
return Json(conversation ?? new Conversation { Id = group.ID });
}
// Mastodon's unread (R9): shown unread again until it is read
[HttpPost("/api/v1/conversations/{id}/unread"), Scope("write:conversations")]
public async Task<IActionResult> UnreadConversation(string id, CancellationToken token)
{
var group = await MyConversation(id, token);
if (group == default)
return NotFoundError();
await ConversationStates.Unread(MyId, group.ID, token);
var conversation = (await ConversationsOf(new List<DmGroup> { group }, token)).FirstOrDefault();
return Json(conversation ?? new Conversation { Id = group.ID, Unread = true });
}
// off the persona's list until a newer message; the messages themselves stay
[HttpDelete("/api/v1/conversations/{id}"), Scope("write:conversations")]
public async Task<IActionResult> RemoveConversation(string id, CancellationToken token)
@@ -410,7 +422,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
return mapped == default ? NotFoundError() : Json(mapped);
}
[HttpPost("/api/v1/notifications/clear"), Scope("write:notifications")]
[HttpPost("/api/v1/notifications/clear"), HttpPost("/api/v2/notifications/clear"), Scope("write:notifications")]
public async Task<IActionResult> Clear(CancellationToken token)
{
await DB.Default.DeleteAsync<NotificationEntity>(n => n.AvatarId == MyId);
PrivaPub/Api/Mastodon/Controllers/UserinfoController.cs
+40
View File
@@ -0,0 +1,40 @@
using Microsoft.AspNetCore.Mvc;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Federation.Actors;
namespace PrivaPub.Api.Mastodon.Controllers
{
// Mastodon 4.4's OAuth userinfo (OpenID Connect's shape, GET or POST, the `profile` scope): who the token's persona is,
// for a site that signs in with it. Only what its public account says already; never its login. The authorization
// server's metadata names it (OAuthSetup).
public class UserinfoController : MastodonController
{
public const string Path = "/oauth/userinfo";
readonly MastodonMapper _mapper;
readonly ILocalActorService _localActors;
public UserinfoController(MastodonMapper mapper, ILocalActorService localActors)
{
_mapper = mapper;
_localActors = localActors;
}
[HttpGet(Path), HttpPost(Path), Scope("profile")]
public async Task<IActionResult> Show(CancellationToken token)
{
var account = await _mapper.Local(Me, false, token);
return Json(new
{
iss = _localActors.BaseAddress + "/",
sub = Me.Uri,
name = account.DisplayName,
preferred_username = account.Username,
profile = account.Url,
picture = account.Avatar
});
}
}
}
PrivaPub/Api/Mastodon/Infrastructure/MastodonController.cs
+2 -1
View File
@@ -48,7 +48,8 @@ namespace PrivaPub.Api.Mastodon.Infrastructure
var scope = context.ActionDescriptor.EndpointMetadata.OfType<ScopeAttribute>().LastOrDefault();
if (User.Identity?.IsAuthenticated == true)
{
if (scope != default && !MastodonScopes.Grants(User.GetScopes(), scope.Scope))
// (a scope written a|b is either)
if (scope != default && !scope.Scope.Split('|').Any(one => MastodonScopes.Grants(User.GetScopes(), one)))
{
context.Result = Error(StatusCodes.Status403Forbidden, "This action is outside the authorized scopes");
return;
PrivaPub/Domain/Moderation/AccountModeration.cs
+1
View File
@@ -154,6 +154,7 @@ namespace PrivaPub.Domain.Moderation
// nothing acts as it from now on: its Mastodon tokens end and its open streams close
await _tokens.RevokeBySubjectAsync(avatar.ID, CancellationToken.None);
await _authorizations.RevokeBySubjectAsync(avatar.ID, CancellationToken.None);
await Social.WebPush.Forget(new[] { avatar.ID });//and its Web Push subscriptions with them (R9): it is pushed nothing
Streams.CloseFor(avatar.ID);
}
if (action is ModerationActions.Silence or ModerationActions.Suspend)
PrivaPub/Domain/Social/Notifications.cs
+21
View File
@@ -7,6 +7,25 @@ namespace PrivaPub.Domain.Social
{
public static class Notifications
{
// told of every notification once it is kept for the persona's lists and streams: never one the notification policy
// filtered or dropped, nor one its blocks and mutes hid. Web Push hangs here (R9, WebPushDispatcher, set while the
// host runs; the command line sets nothing). Whatever it does never fails the notification.
public static Func<Notification, CancellationToken, Task> Told { get; set; }
static async Task Tell(Notification notification, CancellationToken token)
{
if (Told is not { } told)
return;
try
{
await told(notification, token);
}
catch (Exception)
{
//the hook logs its own failures
}
}
public static async Task Add(string avatarId, NotificationType type, string fromAccountId, string fromActorUri, string postId,
CancellationToken token, string emoji = default, string emojiUrl = default, string text = default)
{
@@ -42,6 +61,7 @@ namespace PrivaPub.Domain.Social
var told = new Timelines.Streams.Event("notification", notification.ID);
Timelines.Streams.ToPersona(avatarId, new Timelines.Streams.Key(Timelines.Streams.User), told);
Timelines.Streams.ToPersona(avatarId, new Timelines.Streams.Key(Timelines.Streams.UserNotification), told);
await Tell(notification, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
@@ -73,6 +93,7 @@ namespace PrivaPub.Domain.Social
var told = new Timelines.Streams.Event("notification", notification.ID);
Timelines.Streams.ToPersona(avatarId, new Timelines.Streams.Key(Timelines.Streams.User), told);
Timelines.Streams.ToPersona(avatarId, new Timelines.Streams.Key(Timelines.Streams.UserNotification), told);
await Tell(notification, CancellationToken.None);
}
}
}
PrivaPub/Domain/Social/WebPush.cs
+156
View File
@@ -0,0 +1,156 @@
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Controllers;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Social;
using System.Text.Json;
namespace PrivaPub.Domain.Social
{
public sealed record PushPayload(string SubscriptionId, string NotificationId);
// What Web Push sends, and to whom (completion program R9). A notification is pushed to each subscription of its persona
// whose alert for its type is on and whose policy admits its sender, as Mastodon's Web::PushSubscription#pushable? says;
// a filtered one never is, nor anything for a persona held back (suspended, or its login banned).
public static class WebPush
{
// Mastodon 4.5's alerts (its Notification::TYPES, which data[alerts] takes), then Pleroma's for emoji reactions.
// admin.sign_up and admin.report are kept but never fire: moderation is on /clientapi, and it notifies nobody.
public static readonly IReadOnlyList<string> Alerts = new[]
{
"mention", "quote", "status", "reblog", "follow", "follow_request", "favourite", "poll", "update", "quoted_update",
"severed_relationships", "moderation_warning", "annual_report", "admin.sign_up", "admin.report", "pleroma:emoji_reaction"
};
/// <summary>The alert a notification's type answers to: its Mastodon name.</summary>
public static string AlertOf(NotificationType type) => NotificationsController.Names.TryGetValue(type, out var name) ? name : default;
public static string Name(PushPolicy policy) => policy switch
{
PushPolicy.Followed => "followed",
PushPolicy.Follower => "follower",
PushPolicy.None => "none",
_ => "all"
};
public static PushPolicy? ParsePolicy(string value) => value?.Trim().ToLowerInvariant() switch
{
"all" => PushPolicy.All,
"followed" => PushPolicy.Followed,
"follower" => PushPolicy.Follower,
"none" => PushPolicy.None,
_ => (PushPolicy?)null
};
public static async Task<bool> Pushable(PushSubscription subscription, Notification notification, CancellationToken token)
{
if (notification.Filtered || notification.AvatarId != subscription.AvatarId)
return false;
var alert = AlertOf(notification.Type);
if (alert == default || !subscription.Alerts.Contains(alert))
return false;
return await Admits(subscription, notification, token);
}
// Mastodon's policy_allows_notification?: everyone, nobody, the accounts the persona follows, those that follow it. What
// the persona tells itself (its own poll ended, a warning from the staff, which Mastodon sends from the account itself)
// passes every policy but none.
static async Task<bool> Admits(PushSubscription subscription, Notification notification, CancellationToken token)
{
if (subscription.Policy == PushPolicy.None)
return false;
if (subscription.Policy == PushPolicy.All || notification.FromAccountId == subscription.AvatarId)
return true;
var avatarId = subscription.AvatarId;
var fromId = notification.FromAccountId;
var fromUri = notification.FromActorURI;
if (subscription.Policy == PushPolicy.Followed)
{
if (!string.IsNullOrEmpty(fromId) && await DB.Default.Find<Following>()
.Match(f => f.AvatarId == avatarId && f.FollowerKind == null && f.State == FollowState.Accepted && f.TargetAccountId == fromId).ExecuteAnyAsync(token))
return true;
return !string.IsNullOrEmpty(fromUri) && await DB.Default.Find<Following>()
.Match(f => f.AvatarId == avatarId && f.FollowerKind == null && f.State == FollowState.Accepted && f.TargetActorURI == fromUri).ExecuteAnyAsync(token);
}
if (!string.IsNullOrEmpty(fromUri) && await DB.Default.Find<Follower>()
.Match(f => f.LocalActorId == avatarId && f.LocalActorKind == LocalActorKind.Person && f.IsAccepted && f.ActorURI == fromUri).ExecuteAnyAsync(token))
return true;
return !string.IsNullOrEmpty(fromId) && await DB.Default.Find<Following>()
.Match(f => f.AvatarId == fromId && f.FollowerKind == null && f.State == FollowState.Accepted && f.TargetAccountId == avatarId).ExecuteAnyAsync(token);
}
/// <summary>Every subscription of these personas goes (their tokens ended, or they did).</summary>
public static async Task Forget(IReadOnlyCollection<string> avatarIds)
{
var ids = avatarIds?.Where(id => !string.IsNullOrEmpty(id)).Distinct().ToList() ?? new List<string>();
if (ids.Count > 0)
await DB.Default.DeleteAsync<PushSubscription>(s => ids.Contains(s.AvatarId));//a delete takes no token outside a transaction
}
/// <summary>The subscription made with this token goes (the token was revoked).</summary>
public static async Task ForgetToken(string tokenId)
{
if (!string.IsNullOrEmpty(tokenId))
await DB.Default.DeleteAsync<PushSubscription>(s => s.TokenId == tokenId);
}
}
// Hangs on Notifications.Told while the host runs and queues one SendPush job per subscription the notification goes to
// (WebPushJob sends it, encrypted). Queueing is a few indexed reads and inserts; nothing leaves the server here.
public class WebPushDispatcher : IHostedService
{
readonly IJobQueue _jobs;
readonly ILogger<WebPushDispatcher> _logger;
public WebPushDispatcher(IJobQueue jobs, ILogger<WebPushDispatcher> logger)
{
_jobs = jobs;
_logger = logger;
}
public Task StartAsync(CancellationToken cancellationToken)
{
Notifications.Told = Dispatch;
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
Notifications.Told = default;
return Task.CompletedTask;
}
public async Task Dispatch(Notification notification, CancellationToken token)
{
try
{
if (notification == default || notification.Filtered || Moderation.Restrictions.IsSuspended(notification.AvatarId))
return;
var alert = WebPush.AlertOf(notification.Type);
if (alert == default)
return;
var avatarId = notification.AvatarId;
var subscriptions = await DB.Default.Find<PushSubscription>().Match(s => s.AvatarId == avatarId && s.Alerts.Contains(alert)).ExecuteAsync(token);
var jobs = new List<Job>();
foreach (var subscription in subscriptions)
if (await WebPush.Pushable(subscription, notification, token))
jobs.Add(new Job
{
Kind = JobKind.SendPush,
Payload = JsonSerializer.Serialize(new PushPayload(subscription.ID, notification.ID)),
Host = Uri.TryCreate(subscription.Endpoint, UriKind.Absolute, out var endpoint) ? endpoint.Host : default,
DedupeKey = $"push|{subscription.ID}|{notification.ID}"
});
if (jobs.Count > 0)
await _jobs.EnqueueMany(jobs, token);
}
catch (Exception ex) when (ex is not OperationCanceledException || !token.IsCancellationRequested)
{
_logger.LogWarning(ex, "{Service} could not queue the pushes of a notification", nameof(WebPushDispatcher));
}
}
}
}
PrivaPub/Domain/Statuses/ConversationStates.cs
+29 -3
View File
@@ -21,14 +21,40 @@ namespace PrivaPub.Domain.Statuses
await Read(authorAvatarId, conversationId, postId, token);
}
public static Task Read(string avatarId, string conversationId, string postId, CancellationToken token) =>
Mark(avatarId, conversationId, s => s.LastReadPostId, postId, token);
public static async Task Read(string avatarId, string conversationId, string postId, CancellationToken token)
{
await Mark(avatarId, conversationId, s => s.LastReadPostId, postId, token);
await DB.Default.Update<ConversationState>()
.Match(s => s.AvatarId == avatarId && s.ConversationId == conversationId && s.MarkedUnread)
.Modify(s => s.MarkedUnread, false)
.ExecuteAsync(token);
}
// Mastodon's unread: the conversation shows unread, whoever wrote last, until the persona reads it again (R9)
public static async Task Unread(string avatarId, string conversationId, CancellationToken token)
{
Task Write() => DB.Default.Update<ConversationState>()
.Match(s => s.AvatarId == avatarId && s.ConversationId == conversationId)
.Modify(s => s.MarkedUnread, true)
.Modify(s => s.UpdatedAt, DateTime.UtcNow)
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
try
{
await Write();
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
await Write();
}
}
public static Task Hide(string avatarId, string conversationId, string postId, CancellationToken token) =>
Mark(avatarId, conversationId, s => s.HiddenUpTo, postId, token);
public static bool IsUnread(ConversationState state, string lastPostId, string lastAuthorId, string avatarId) =>
lastPostId != default && lastAuthorId != avatarId && string.CompareOrdinal(lastPostId, state?.LastReadPostId) > 0;
state?.MarkedUnread == true
|| lastPostId != default && lastAuthorId != avatarId && string.CompareOrdinal(lastPostId, state?.LastReadPostId) > 0;
public static bool IsHidden(ConversationState state, string lastPostId) =>
state?.HiddenUpTo != default && string.CompareOrdinal(lastPostId, state.HiddenUpTo) <= 0;
PrivaPub/Infrastructure/Backup/ProtectiveMerge.cs
+3
View File
@@ -168,6 +168,9 @@ namespace PrivaPub.Infrastructure.Backup
.Set("CredentialsChangedAt", now), cancellationToken: token);
foreach (var collection in new[] { "openiddict.tokens", "openiddict.authorizations" })
report.SessionsEnded += (int)(await database.GetCollection<BsonDocument>(collection).DeleteManyAsync(FilterDefinition<BsonDocument>.Empty, token)).DeletedCount;
// and the Web Push subscriptions made with those tokens (R9; never in a backup): clients subscribe again once signed
// in. The VAPID key (PushKeys, never in a backup either) stays the live one.
await database.GetCollection<BsonDocument>("PushSubscription").DeleteManyAsync(FilterDefinition<BsonDocument>.Empty, token);
// what the backup remembered of servers being down is long past
await database.GetCollection<BsonDocument>("RemoteInstance").UpdateManyAsync(FilterDefinition<BsonDocument>.Empty,
PrivaPub/Infrastructure/Backup/ServerBackup.cs
+3 -1
View File
@@ -42,7 +42,9 @@ namespace PrivaPub.Infrastructure.Backup
["AppConfiguration"] = "the configuration's copy holds the SMTP password, and is made again from appsettings at boot",
[nameof(RestoreRecord)] = "what restores did outlives what they restore",
["PersonaArchive"] = "personas' archives being made or imported belong to the moment",
["ServerSetup"] = "the server is set up once: a restore never opens its setup again"
["ServerSetup"] = "the server is set up once: a restore never opens its setup again",
["PushSubscription"] = "Web Push subscriptions belong to sessions: a restore ends every one, and clients subscribe again",
["PushKeys"] = "the VAPID key every push subscription was made with: a restore keeps the live one"
};
static readonly JsonWriterSettings Json = new() { OutputMode = JsonOutputMode.CanonicalExtendedJson, Indent = false };
PrivaPub/Infrastructure/Cli/AdminCommands.cs
+1
View File
@@ -290,6 +290,7 @@ namespace PrivaPub.Infrastructure.Cli
await DB.Default.Update<RootToAvatar>().MatchID(link.ID).Modify(ra => ra.RootId, root.ID).ExecuteAsync();
await services.GetRequiredService<IOpenIddictTokenManager>().RevokeBySubjectAsync(persona.ID);
await services.GetRequiredService<IOpenIddictAuthorizationManager>().RevokeBySubjectAsync(persona.ID);
await Domain.Social.WebPush.Forget(new[] { persona.ID });//its Web Push subscriptions go with its tokens (R9)
await Log(services, AdminActions.PersonaMove, AdminTargets.Persona, persona.ID, "@" + personaName, "moved to another login");
output.WriteLine($"@{personaName}: moved to {login}");
return 0;
PrivaPub/Infrastructure/Data/Indexes.cs
+3
View File
@@ -73,6 +73,9 @@ namespace PrivaPub.Infrastructure.Data
await Plain<PersonaNote>(token, n => n.TargetAccountId);
await DB.Default.Index<SuggestionDismissal>().Key(d => d.AvatarId, KeyType.Ascending).Key(d => d.AccountId, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
// Web Push (a new collection, R9): one subscription per access token, a persona's read when it is notified
await Unique<PushSubscription>(s => s.TokenId, Builders<PushSubscription>.Filter.Gt(s => s.TokenId, ""), token);
await Plain<PushSubscription>(token, s => s.AvatarId);
// the staff's log, read newest first
await DB.Default.Index<Models.AdminAction>().Key(a => a.At, KeyType.Descending).CreateAsync(token);
PrivaPub/Infrastructure/Http/SafeHttpHandlerFactory.cs
+2 -2
View File
@@ -11,7 +11,7 @@ namespace PrivaPub.Infrastructure.Http
public static class SafeHttpHandlerFactory
{
public static SocketsHttpHandler Create(FederationOptions options, IConnectedAddresses connected = default) => new()
public static SocketsHttpHandler Create(FederationOptions options, IConnectedAddresses connected = default, TimeSpan? connectTimeout = default) => new()
{
SslOptions = options.AcceptAnyCertificate
? new SslClientAuthenticationOptions { RemoteCertificateValidationCallback = (_, _, _, _) => true }
@@ -20,7 +20,7 @@ namespace PrivaPub.Infrastructure.Http
UseProxy = false,
UseCookies = false,
AutomaticDecompression = DecompressionMethods.All,
ConnectTimeout = TimeSpan.FromSeconds(10),
ConnectTimeout = connectTimeout ?? TimeSpan.FromSeconds(10),
PooledConnectionLifetime = TimeSpan.FromMinutes(2),
MaxResponseHeadersLength = 64,
ConnectCallback = (context, token) => Connect(context.DnsEndPoint, options.AllowPrivateNetworks, token, connected)
PrivaPub/Infrastructure/Push/Vapid.cs
+116
View File
@@ -0,0 +1,116 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Models;
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
namespace PrivaPub.Infrastructure.Push
{
// The server's VAPID key (RFC 8292, completion program R9) and the JWT every push carries. The key is made the first time
// it is needed and kept in Mongo (PushKeys, one document); every process reads that one.
public interface IVapid
{
Task<VapidKey> Key(CancellationToken token);
}
public sealed class VapidKey
{
// RFC 8292 2: no more than 24 hours ahead; half of it leaves room for a push service's clock
public static readonly TimeSpan TokenLifetime = TimeSpan.FromHours(12);
readonly ECParameters _parameters;
public VapidKey(byte[] publicPoint, byte[] d)
{
PublicPoint = publicPoint;
_parameters = new ECParameters
{
Curve = ECCurve.NamedCurves.nistP256,
D = d,
Q = new ECPoint { X = publicPoint[1..33], Y = publicPoint[33..] }
};
}
public byte[] PublicPoint { get; }
/// <summary>As the APIs give it (the instance's vapid.public_key, an app's vapid_key, a subscription's server_key):
/// URL-safe base64 with its padding, as Mastodon's VAPID_PUBLIC_KEY is written.</summary>
public string ForApi => Convert.ToBase64String(PublicPoint).Replace('+', '-').Replace('/', '_');
/// <summary>As headers take it (RFC 8292's k, the draft's Crypto-Key p256ecdsa): base64url without padding.</summary>
public string ForHeader => WebPushCrypto.Base64Url(PublicPoint);
/// <summary>The ES256 JWT for one push service (aud: its origin), signed by this key.</summary>
public string Token(string audience, string subject, DateTime now)
{
var header = WebPushCrypto.Base64Url("{\"typ\":\"JWT\",\"alg\":\"ES256\"}"u8);
var claims = WebPushCrypto.Base64Url(JsonSerializer.SerializeToUtf8Bytes(new Dictionary<string, object>
{
["aud"] = audience,
["exp"] = new DateTimeOffset(now + TokenLifetime).ToUnixTimeSeconds(),
["sub"] = subject
}));
var signed = header + "." + claims;
using var key = ECDsa.Create(_parameters);
var signature = key.SignData(Encoding.ASCII.GetBytes(signed), HashAlgorithmName.SHA256, DSASignatureFormat.IeeeP1363FixedFieldConcatenation);
return signed + "." + WebPushCrypto.Base64Url(signature);
}
/// <summary>A push service's origin, the JWT's aud: scheme, host and a port that is not the scheme's own.</summary>
public static string Audience(Uri endpoint) => endpoint.GetLeftPart(UriPartial.Authority).ToLower(CultureInfo.InvariantCulture);
}
public class Vapid : IVapid
{
readonly SemaphoreSlim _gate = new(1, 1);
volatile VapidKey _key;
public async Task<VapidKey> Key(CancellationToken token)
{
if (_key is { } known)
return known;
await _gate.WaitAsync(token);
try
{
if (_key is { } meanwhile)
return meanwhile;
var stored = await Stored(token) ?? await Make(token);
_key = new VapidKey(WebPushCrypto.FromBase64Url(stored.PublicKey), WebPushCrypto.FromBase64Url(stored.PrivateKey));
return _key;
}
finally
{
_gate.Release();
}
}
static Task<PushKeys> Stored(CancellationToken token) =>
DB.Default.Find<PushKeys>().Match(k => k.ID == PushKeys.Name).ExecuteFirstAsync(token);
// the first need makes it; two processes at once keep the first one saved
static async Task<PushKeys> Make(CancellationToken token)
{
using var key = ECDsa.Create(ECCurve.NamedCurves.nistP256);
var parameters = key.ExportParameters(true);
var keys = new PushKeys
{
ID = PushKeys.Name,
PublicKey = WebPushCrypto.Base64Url([0x04, .. parameters.Q.X, .. parameters.Q.Y]),
PrivateKey = WebPushCrypto.Base64Url(parameters.D)
};
try
{
await DB.Default.InsertAsync(keys, token);
return keys;
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return await Stored(token);
}
}
}
}
PrivaPub/Infrastructure/Push/WebPushCrypto.cs
+169
View File
@@ -0,0 +1,169 @@
using System.Buffers.Binary;
using System.Security.Cryptography;
using System.Text;
namespace PrivaPub.Infrastructure.Push
{
// Web Push message encryption, on System.Security.Cryptography alone (completion program R9). Two encodings, as Mastodon
// sends them:
// - RFC 8291 (`aes128gcm`, in RFC 8188's record), for a subscription made with `standard` (Mastodon 4.4);
// - the draft before it (`aesgcm`: draft-ietf-webpush-encryption-04 over draft-ietf-httpbis-encryption-encoding-03), for
// every other one, with its salt in the Encryption header and our public key in Crypto-Key.
// Both: a fresh P-256 key pair and a fresh 16-byte salt per message, ECDH with the client's p256dh, HKDF-SHA-256 keyed by
// its auth secret, AES-128-GCM, one record. Only the client can read the message; the push service carries it blind.
public static class WebPushCrypto
{
public const int RecordSize = 4096;
// the largest payload a 4096-byte aes128gcm body holds: 86 bytes of header, the delimiter and the tag
public const int MaxPayload = RecordSize - 86 - 1 - 16;
const int SaltLength = 16;
const int TagLength = 16;
public sealed record Message(byte[] Body, string ContentEncoding, byte[] Salt, byte[] ServerPublicKey);
public static Message Aes128Gcm(ReadOnlySpan<byte> plaintext, byte[] clientPublicKey, byte[] authSecret)
{
using var server = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
return Aes128Gcm(plaintext, clientPublicKey, authSecret, server, RandomNumberGenerator.GetBytes(SaltLength));
}
public static Message Aes128Gcm(ReadOnlySpan<byte> plaintext, byte[] clientPublicKey, byte[] authSecret, ECDiffieHellman server, byte[] salt)
{
if (plaintext.Length > MaxPayload)
throw new ArgumentException($"a push payload holds {MaxPayload} bytes at most", nameof(plaintext));
var serverPublic = PublicPoint(server);
var shared = SharedSecret(server, clientPublicKey);
// RFC 8291 3.3 and 3.4: PRK_key = HMAC-SHA-256(auth, ecdh); IKM = HKDF-Expand(PRK_key, "WebPush: info" 0x00
// ua_public as_public, 32)
var keyInfo = Concat(Encoding.ASCII.GetBytes("WebPush: info\0"), clientPublicKey, serverPublic);
var ikm = HKDF.Expand(HashAlgorithmName.SHA256, HKDF.Extract(HashAlgorithmName.SHA256, shared, authSecret), 32, keyInfo);
// RFC 8188 2.2 and 2.3: the content encryption key and the nonce, from the salt
var prk = HKDF.Extract(HashAlgorithmName.SHA256, ikm, salt);
var cek = HKDF.Expand(HashAlgorithmName.SHA256, prk, 16, Encoding.ASCII.GetBytes("Content-Encoding: aes128gcm\0"));
var nonce = HKDF.Expand(HashAlgorithmName.SHA256, prk, 12, Encoding.ASCII.GetBytes("Content-Encoding: nonce\0"));
// one record, the last: the payload, its delimiter 0x02, no padding
var record = new byte[plaintext.Length + 1];
plaintext.CopyTo(record);
record[^1] = 0x02;
var encrypted = Seal(cek, nonce, record);
// RFC 8188 2.1's header: the salt, the record size, the key id's length and the key id (RFC 8291: our public key)
var body = new byte[SaltLength + 4 + 1 + serverPublic.Length + encrypted.Length];
salt.CopyTo(body, 0);
BinaryPrimitives.WriteUInt32BigEndian(body.AsSpan(SaltLength), RecordSize);
body[SaltLength + 4] = (byte)serverPublic.Length;
serverPublic.CopyTo(body, SaltLength + 5);
encrypted.CopyTo(body, SaltLength + 5 + serverPublic.Length);
return new Message(body, "aes128gcm", salt, serverPublic);
}
public static Message AesGcm(ReadOnlySpan<byte> plaintext, byte[] clientPublicKey, byte[] authSecret)
{
using var server = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
return AesGcm(plaintext, clientPublicKey, authSecret, server, RandomNumberGenerator.GetBytes(SaltLength));
}
public static Message AesGcm(ReadOnlySpan<byte> plaintext, byte[] clientPublicKey, byte[] authSecret, ECDiffieHellman server, byte[] salt)
{
if (plaintext.Length > MaxPayload)
throw new ArgumentException($"a push payload holds {MaxPayload} bytes at most", nameof(plaintext));
var serverPublic = PublicPoint(server);
var shared = SharedSecret(server, clientPublicKey);
// the draft's: IKM = HKDF(salt: auth, ecdh, "Content-Encoding: auth" 0x00, 32), then the key and nonce from the salt
// over a context of both public keys, each after its length
var ikm = HKDF.Expand(HashAlgorithmName.SHA256, HKDF.Extract(HashAlgorithmName.SHA256, shared, authSecret), 32,
Encoding.ASCII.GetBytes("Content-Encoding: auth\0"));
var context = Concat(Encoding.ASCII.GetBytes("P-256\0"), Length16(clientPublicKey), clientPublicKey, Length16(serverPublic), serverPublic);
var prk = HKDF.Extract(HashAlgorithmName.SHA256, ikm, salt);
var cek = HKDF.Expand(HashAlgorithmName.SHA256, prk, 16, Concat(Encoding.ASCII.GetBytes("Content-Encoding: aesgcm\0"), context));
var nonce = HKDF.Expand(HashAlgorithmName.SHA256, prk, 12, Concat(Encoding.ASCII.GetBytes("Content-Encoding: nonce\0"), context));
// two bytes saying how much padding follows (none), then the payload
var padded = new byte[2 + plaintext.Length];
plaintext.CopyTo(padded.AsSpan(2));
return new Message(Seal(cek, nonce, padded), "aesgcm", salt, serverPublic);
}
/// <summary>A client's public key: an uncompressed point on P-256, or a CryptographicException.</summary>
public static ECDiffieHellman ImportPublic(byte[] point)
{
if (point is not { Length: 65 } || point[0] != 0x04)
throw new CryptographicException("not an uncompressed P-256 point");
var key = ECDiffieHellman.Create(new ECParameters
{
Curve = ECCurve.NamedCurves.nistP256,
Q = new ECPoint { X = point[1..33], Y = point[33..] }
});
return key;
}
/// <summary>Whether a subscription's keys can be encrypted to: a point on P-256 and a 16-byte secret.</summary>
public static bool AreUsable(byte[] p256dh, byte[] auth)
{
if (auth is not { Length: 16 })
return false;
try
{
using var key = ImportPublic(p256dh);
using var probe = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
probe.DeriveRawSecretAgreement(key.PublicKey);
return true;
}
catch (CryptographicException)
{
return false;
}
}
public static byte[] PublicPoint(ECAlgorithm key)
{
var parameters = key.ExportParameters(false);
return [0x04, .. parameters.Q.X, .. parameters.Q.Y];
}
public static string Base64Url(ReadOnlySpan<byte> bytes) =>
Convert.ToBase64String(bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_');
/// <summary>base64url or base64, padded or not, as clients send keys; null when it is neither.</summary>
public static byte[] FromBase64Url(string text)
{
if (string.IsNullOrWhiteSpace(text))
return default;
var normal = text.Trim().TrimEnd('=').Replace('-', '+').Replace('_', '/');
normal = normal.PadRight(normal.Length + (4 - normal.Length % 4) % 4, '=');
var bytes = new byte[normal.Length * 3 / 4];
return Convert.TryFromBase64String(normal, bytes, out var written) ? bytes[..written] : default;
}
static byte[] SharedSecret(ECDiffieHellman server, byte[] clientPublicKey)
{
using var client = ImportPublic(clientPublicKey);
return server.DeriveRawSecretAgreement(client.PublicKey);
}
static byte[] Seal(byte[] key, byte[] nonce, byte[] plaintext)
{
var output = new byte[plaintext.Length + TagLength];
using var aes = new AesGcm(key, TagLength);
aes.Encrypt(nonce, plaintext, output.AsSpan(0, plaintext.Length), output.AsSpan(plaintext.Length));
return output;
}
static byte[] Length16(byte[] value)
{
var length = new byte[2];
BinaryPrimitives.WriteUInt16BigEndian(length, (ushort)value.Length);
return length;
}
static byte[] Concat(params byte[][] parts)
{
var all = new byte[parts.Sum(p => p.Length)];
var at = 0;
foreach (var part in parts)
{
part.CopyTo(all, at);
at += part.Length;
}
return all;
}
}
}
PrivaPub/Infrastructure/Push/WebPushJob.cs
+264
View File
@@ -0,0 +1,264 @@
using Microsoft.Extensions.Localization;
using Microsoft.Extensions.Options;
using MongoDB.Entities;
using OpenIddict.Abstractions;
using PrivaPub.Api.Mastodon.Controllers;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Instance;
using PrivaPub.Domain.Moderation;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Resources;
using System.Net;
using System.Net.Http.Headers;
using System.Net.Sockets;
using System.Text.Json;
using System.Text.RegularExpressions;
using static OpenIddict.Abstractions.OpenIddictConstants;
namespace PrivaPub.Infrastructure.Push
{
// Sends one Web Push notification to one subscription (JobKind.SendPush, queued by WebPushDispatcher; completion
// program R9), as Mastodon's Web::PushNotificationWorker does:
// - the payload is Mastodon's (access_token, preferred_locale, notification_id, notification_type, icon, title, body), built
// when it is sent, as the persona sees the notification then (one it can no longer see, or dismissed meanwhile, is not
// sent), its access_token the subscription's own: the token is checked to be valid and the persona's first;
// - encrypted for the client alone (WebPushCrypto): RFC 8291 aes128gcm with RFC 8292's `vapid t=, k=` for a standard
// subscription, otherwise the aesgcm draft with Encryption and Crypto-Key and `WebPush <jwt>`; the JWT (Vapid) names the
// push service's origin (aud), lasts twelve hours (exp) and the server's contact (sub: mailto: the contact email, or
// the server's address; never a persona);
// - TTL two days and Urgency normal, as Mastodon's;
// - through the `push` client: public addresses only, https only, no redirects, small timeouts (the federation client's
// safe handler, with the pasture's exceptions where Federation says so).
// A 404 or 410 means the subscription is gone: it is deleted. 408, 429, a 5xx, a timeout or a broken connection retry on
// the jobs' backoff; any other answer drops the push (logged by the push service's host alone).
public class WebPushJob : IJobHandler
{
public const string ClientName = "push";
public static readonly TimeSpan RequestTimeout = TimeSpan.FromSeconds(15);
public static readonly TimeSpan ConnectTimeout = TimeSpan.FromSeconds(5);
// Mastodon's Web::PushNotificationWorker::TTL and URGENCY
const int TimeToLive = 48 * 3600;
const string Urgency = "normal";
const int BodyLength = 140;
readonly IHttpClientFactory _http;
readonly IVapid _vapid;
readonly IServiceScopeFactory _scopes;
readonly IInstanceSettings _settings;
readonly ILocalActorService _localActors;
readonly IOptionsMonitor<FederationOptions> _federation;
readonly IStringLocalizer<GenericRes> _localizer;
readonly ILogger<WebPushJob> _logger;
public WebPushJob(IHttpClientFactory http, IVapid vapid, IServiceScopeFactory scopes, IInstanceSettings settings, ILocalActorService localActors,
IOptionsMonitor<FederationOptions> federation, IStringLocalizer<GenericRes> localizer, ILogger<WebPushJob> logger)
{
_http = http;
_vapid = vapid;
_scopes = scopes;
_settings = settings;
_localActors = localActors;
_federation = federation;
_localizer = localizer;
_logger = logger;
}
public JobKind Kind => JobKind.SendPush;
public int Concurrency => 4;
public int MaxAttempts => 6;
public int PerHostLimit => 4;
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var payload = JsonSerializer.Deserialize<PushPayload>(job.Payload ?? "null");
if (string.IsNullOrEmpty(payload?.SubscriptionId) || string.IsNullOrEmpty(payload.NotificationId))
return JobOutcome.Done;
var subscriptionId = payload.SubscriptionId;
var subscription = await DB.Default.Find<PushSubscription>().Match(s => s.ID == subscriptionId).ExecuteFirstAsync(token);
if (subscription == default)
return JobOutcome.Done;
// a persona held back (suspended, its login banned) gets no push
if (Restrictions.IsSuspended(subscription.AvatarId))
return JobOutcome.Done;
var notificationId = payload.NotificationId;
var avatarId = subscription.AvatarId;
var notification = await DB.Default.Find<Notification>().Match(n => n.ID == notificationId && n.AvatarId == avatarId).ExecuteFirstAsync(token);
if (notification == default || !await WebPush.Pushable(subscription, notification, token))
return JobOutcome.Done;
if (!Uri.TryCreate(subscription.Endpoint, UriKind.Absolute, out var endpoint) || !IsAllowed(endpoint, _federation.CurrentValue))
return JobOutcome.Dead("endpoint");
byte[] json;
using (var scope = _scopes.CreateScope())
{
// the token the subscription was made with must still be valid, and the persona's: it goes into the payload
if (!await TokenHolds(scope.ServiceProvider.GetRequiredService<IOpenIddictTokenManager>(), subscription, token))
{
await WebPush.ForgetToken(subscription.TokenId);
return JobOutcome.Done;
}
var avatar = await DB.Default.Find<Avatar>().Match(a => a.ID == avatarId && a.DeletionAt == null).ExecuteFirstAsync(token);
if (avatar == default)
{
await WebPush.Forget(new[] { avatarId });
return JobOutcome.Done;
}
var mapped = (await scope.ServiceProvider.GetRequiredService<MastodonMapper>().Notifications(new List<Notification> { notification }, avatarId, token))
.FirstOrDefault();
if (mapped == default)
return JobOutcome.Done;
json = JsonSerializer.SerializeToUtf8Bytes(new Dictionary<string, object>
{
["access_token"] = subscription.AccessToken,
["preferred_locale"] = Locale(avatar, scope.ServiceProvider),
["notification_id"] = notification.ID,
["notification_type"] = WebPush.AlertOf(notification.Type),
["icon"] = mapped.Account?.AvatarStatic ?? mapped.Account?.Avatar,
["title"] = Title(notification, mapped),
["body"] = Body(mapped)
});
}
var key = await _vapid.Key(token);
var p256dh = WebPushCrypto.FromBase64Url(subscription.P256dh);
var auth = WebPushCrypto.FromBase64Url(subscription.Auth);
if (!WebPushCrypto.AreUsable(p256dh, auth))
return JobOutcome.Dead("keys");
var message = subscription.Standard ? WebPushCrypto.Aes128Gcm(json, p256dh, auth) : WebPushCrypto.AesGcm(json, p256dh, auth);
var jwt = key.Token(VapidKey.Audience(endpoint), Subject(), DateTime.UtcNow);
using var request = new HttpRequestMessage(HttpMethod.Post, endpoint) { Content = new ByteArrayContent(message.Body) };
request.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
request.Content.Headers.ContentEncoding.Add(message.ContentEncoding);
request.Headers.TryAddWithoutValidation("TTL", TimeToLive.ToString(System.Globalization.CultureInfo.InvariantCulture));
request.Headers.TryAddWithoutValidation("Urgency", Urgency);
if (subscription.Standard)
request.Headers.TryAddWithoutValidation("Authorization", $"vapid t={jwt}, k={key.ForHeader}");
else
{
request.Headers.TryAddWithoutValidation("Authorization", $"WebPush {jwt}");
request.Headers.TryAddWithoutValidation("Encryption", $"salt={WebPushCrypto.Base64Url(message.Salt)}");
request.Headers.TryAddWithoutValidation("Crypto-Key", $"dh={WebPushCrypto.Base64Url(message.ServerPublicKey)};p256ecdsa={key.ForHeader}");
}
try
{
using var response = await _http.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
var status = (int)response.StatusCode;
if (status is >= 200 and < 300)
return JobOutcome.Done;
if (response.StatusCode is HttpStatusCode.NotFound or HttpStatusCode.Gone)
{
await DB.Default.DeleteAsync<PushSubscription>(subscription.ID);
_logger.LogInformation("{Kind}: {Host} answered {Status}, the subscription is gone", Kind, endpoint.Host, status);
return JobOutcome.Done;
}
if (response.StatusCode is HttpStatusCode.RequestTimeout or HttpStatusCode.TooManyRequests || status >= 500)
return JobOutcome.Retry($"status {status}");
_logger.LogWarning("{Kind}: {Host} refused a push with {Status}; it is dropped", Kind, endpoint.Host, status);
return JobOutcome.Dead($"status {status}");
}
catch (BlockedDestinationException)
{
_logger.LogWarning("{Kind}: {Host} does not resolve to a public address; the push is dropped", Kind, endpoint.Host);
return JobOutcome.Dead("blocked destination");
}
catch (TaskCanceledException) when (!token.IsCancellationRequested)
{
return JobOutcome.Retry("timeout");
}
catch (HttpRequestException ex) when (ex.InnerException is BlockedDestinationException)
{
_logger.LogWarning("{Kind}: {Host} does not resolve to a public address; the push is dropped", Kind, endpoint.Host);
return JobOutcome.Dead("blocked destination");
}
catch (Exception ex) when (ex is HttpRequestException or IOException or SocketException)
{
return JobOutcome.Retry(ex.GetType().Name);
}
}
/// <summary>https, a host, no user information; plain http only on a test network (Federation:AllowPlainHttp, which
/// production refuses at start).</summary>
public static bool IsAllowed(Uri endpoint, FederationOptions options) =>
endpoint != default && endpoint.IsAbsoluteUri && !string.IsNullOrEmpty(endpoint.Host) && string.IsNullOrEmpty(endpoint.UserInfo)
&& (endpoint.Scheme == Uri.UriSchemeHttps || options?.AllowPlainHttp == true && endpoint.Scheme == Uri.UriSchemeHttp);
static async Task<bool> TokenHolds(IOpenIddictTokenManager tokens, PushSubscription subscription, CancellationToken token)
{
if (string.IsNullOrEmpty(subscription.TokenId) || string.IsNullOrEmpty(subscription.AccessToken))
return false;
var stored = await tokens.FindByIdAsync(subscription.TokenId, token);
if (stored == default || !await tokens.HasStatusAsync(stored, Statuses.Valid, token))
return false;
if (await tokens.GetExpirationDateAsync(stored, token) is { } expires && expires < DateTimeOffset.UtcNow)
return false;
return await tokens.GetSubjectAsync(stored, token) == subscription.AvatarId;
}
// RFC 8292's sub: how the push service reaches the server's operator. The contact email the administrator gave, or
// the server's own address; never a persona. (A server on plain http, a test network, has no https: address to give,
// so it names a mailbox at its host: RFC 8292 takes only mailto: and https:.)
string Subject()
{
var email = _settings.Current.ContactEmail?.Trim();
if (!string.IsNullOrEmpty(email) && !email.Any(char.IsWhiteSpace) && email.Contains('@'))
return "mailto:" + email;
var address = _localActors.BaseAddress;
return Uri.TryCreate(address, UriKind.Absolute, out var base_) && base_.Scheme != Uri.UriSchemeHttps
? "mailto:privapub@" + base_.Host
: address;
}
// the persona's language (its default for posts), else the server's first
static string Locale(Avatar avatar, IServiceProvider services)
{
var language = avatar.Settings?.DefaultLanguage;
return string.IsNullOrWhiteSpace(language) ? InstanceController.LanguageCodes(services)[0] : language.Trim();
}
// Mastodon's notification_mailer subjects
string Title(Notification notification, Api.Mastodon.Entities.Notification mapped)
{
var name = string.IsNullOrWhiteSpace(mapped.Account?.DisplayName) ? mapped.Account?.Username ?? string.Empty : mapped.Account.DisplayName;
return notification.Type switch
{
NotificationType.Mention => string.Format(_localizer["You were mentioned by {0}"], name),
NotificationType.Status => string.Format(_localizer["{0} just posted"], name),
NotificationType.Reblog => string.Format(_localizer["{0} boosted your post"], name),
NotificationType.Follow => string.Format(_localizer["{0} is now following you"], name),
NotificationType.FollowRequest => string.Format(_localizer["Pending follower: {0}"], name),
NotificationType.Favourite => string.Format(_localizer["{0} favorited your post"], name),
NotificationType.Poll => string.Format(_localizer["A poll by {0} has ended"], name),
NotificationType.Update => string.Format(_localizer["{0} edited a post"], name),
NotificationType.Quote => string.Format(_localizer["{0} quoted your post"], name),
NotificationType.Reaction => string.Format(_localizer["{0} reacted to your post with {1}"], name, notification.Emoji),
NotificationType.ModerationWarning => _localizer["You have received a moderation warning"],
_ => name
};
}
// Mastodon's: the post's warning, else its words, else the account's bio, without tags, 140 characters at most
static string Body(Api.Mastodon.Entities.Notification mapped)
{
var source = !string.IsNullOrWhiteSpace(mapped.Status?.SpoilerText) ? mapped.Status.SpoilerText
: mapped.Status?.Content ?? mapped.ModerationWarning?.Text ?? mapped.Account?.Note ?? string.Empty;
var text = WebUtility.HtmlDecode(Tags.Replace(Breaks.Replace(source, "\n"), string.Empty)).Trim();
return text.Length <= BodyLength ? text : text[..(BodyLength - 3)] + "...";
}
static readonly Regex Breaks = new(@"<br\s*/?>|</p>\s*<p[^>]*>", RegexOptions.IgnoreCase | RegexOptions.Compiled);
static readonly Regex Tags = new("<[^>]*>", RegexOptions.Compiled);
}
}
PrivaPub/Middleware/SocialPubConfigurations.cs
+14
View File
@@ -57,7 +57,21 @@ namespace PrivaPub.Middleware
.ConfigurePrimaryHttpMessageHandler(provider =>
SafeHttpHandlerFactory.Create(provider.GetRequiredService<IOptions<FederationOptions>>().Value, provider.GetRequiredService<IConnectedAddresses>()))
.AddHttpMessageHandler(provider => new EdgeHintsHandler(provider.GetRequiredService<IEdgeHints>()));
// not federation: Web Push (R9), to the push services clients subscribed with. The same safe handler (public
// addresses only, the pasture's exceptions where Federation says so, no redirects), https only (WebPushJob),
// small timeouts, and no statistics of where it connects
service.AddHttpClient(Infrastructure.Push.WebPushJob.ClientName, (provider, client) =>
{
var baseAddress = provider.GetRequiredService<IOptionsMonitor<AppConfiguration>>().CurrentValue.BackendBaseAddress?.TrimEnd('/');
client.Timeout = Infrastructure.Push.WebPushJob.RequestTimeout;
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref} (+{baseAddress}/)");
})
.ConfigurePrimaryHttpMessageHandler(provider =>
SafeHttpHandlerFactory.Create(provider.GetRequiredService<IOptions<FederationOptions>>().Value, connectTimeout: Infrastructure.Push.WebPushJob.ConnectTimeout));
return service
.AddSingleton<Infrastructure.Push.IVapid, Infrastructure.Push.Vapid>()
.AddSingleton<IJobHandler, Infrastructure.Push.WebPushJob>()
.AddHostedService<Domain.Social.WebPushDispatcher>()
.AddSingleton<IConnectedAddresses, ConnectedAddresses>()
.AddSingleton<IEdgeHints, EdgeHints>()
.AddSingleton<IFederationHttp, FederationHttp>()
PrivaPub/Models/Jobs/Job.cs
+2 -1
View File
@@ -40,7 +40,8 @@ namespace PrivaPub.Models.Jobs
ExportArchive,
ImportArchive,
SyncFeeds,
SendEmailVerification
SendEmailVerification,
SendPush//a Web Push notification to one subscription (R9, WebPushJob)
}
public enum JobState
PrivaPub/Models/PushKeys.cs
+18
View File
@@ -0,0 +1,18 @@
using MongoDB.Entities;
namespace PrivaPub.Models
{
// The server's VAPID key pair (RFC 8292, completion program R9): one P-256 key, made the first time something needs it
// (Infrastructure/Push/Vapid) and kept here, one document with the fixed id `vapid`. Every push carries a JWT it signs,
// and the push services tie each subscription to its public key, which the instance API and app registrations
// advertise. Never in a backup and never dropped by a restore (ServerBackup.Excluded): a restored key would break every
// subscription a client made since.
public class PushKeys : Entity
{
public const string Name = "vapid";
public string PublicKey { get; set; }//the uncompressed point (0x04 || X || Y), base64url without padding
public string PrivateKey { get; set; }//the private scalar D, base64url without padding
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
}
PrivaPub/Models/Social/ConversationState.cs
+1
View File
@@ -10,6 +10,7 @@ namespace PrivaPub.Models.Social
public string ConversationId { get; set; }
public string LastReadPostId { get; set; }
public string HiddenUpTo { get; set; }
public bool MarkedUnread { get; set; }//Mastodon's POST /api/v1/conversations/:id/unread, until it is read again (R9)
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
}
}
PrivaPub/Models/Social/PushSubscription.cs
+34
View File
@@ -0,0 +1,34 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Social
{
// A Web Push subscription (completion program R9). One per Mastodon access token, as on Mastodon (a new one made with the
// same token replaces it), and so one persona's, the token's subject: never its login's, so nothing pairs sibling
// personas. It holds the token it was made with, which the encrypted payload carries back to the client as Mastodon's
// does (`access_token`), so it goes the moment the token does: revoked (/oauth/revoke, its login's sessions ended), its
// persona suspended, moved or deleted, a restore. Never in a backup (ServerBackup.Excluded).
public class PushSubscription : Entity
{
public string AvatarId { get; set; }
public string TokenId { get; set; }//the OpenIddict token's id; unique
public string AccessToken { get; set; }//the bearer token as the client sent it when it subscribed
public string Endpoint { get; set; }//the push service's URL for this client
public string P256dh { get; set; }//the client's ECDH public key: an uncompressed P-256 point, base64url as it sent it
public string Auth { get; set; }//its 16-byte authentication secret, base64url as it sent it
public bool Standard { get; set; }//RFC 8291 aes128gcm and RFC 8292 `vapid` when true (Mastodon 4.4), else the aesgcm draft
public List<string> Alerts { get; set; } = new();//the alerts turned on, by Mastodon's names (WebPush.Alerts)
public PushPolicy Policy { get; set; } = PushPolicy.All;
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
}
// whose notifications are pushed (Mastodon's data[policy]): everyone's, the accounts the persona follows, the accounts
// that follow it, nobody's
public enum PushPolicy
{
All,
Followed,
Follower,
None
}
}
PrivaPub/Services/PersonaRemoval.cs
+1
View File
@@ -95,6 +95,7 @@ namespace PrivaPub.Services
// nothing acts as it while it goes
await _tokens.RevokeBySubjectAsync(avatar.ID, token);
await _authorizations.RevokeBySubjectAsync(avatar.ID, token);
await Domain.Social.WebPush.Forget(new[] { avatar.ID });//its Web Push subscriptions with its tokens (R9)
Streams.CloseFor(avatar.ID);
foreach (var group in await _dbEntities.Groups.Match(g => g.OwnerAvatarId == avatar.ID && !g.DeletionAt.HasValue).ExecuteAsync(token))
PrivaPub/Services/RootSessions.cs
+5 -2
View File
@@ -15,7 +15,7 @@ namespace PrivaPub.Services
}
// Ends everything a root is signed in with: its /clientapi tokens, through a new SessionStamp (checked by JwtEvents),
// and the Mastodon API tokens and authorizations of each of its personas. Used when its password is changed or
// and the Mastodon API tokens and authorizations of each of its personas, with their Web Push subscriptions. Used when its password is changed or
// recovered, when it signs out everywhere and when it is deleted. End signs out one session only: the device that
// asked (its Mastodon tokens it revokes itself).
public class RootSessions : IRootSessions
@@ -40,11 +40,14 @@ namespace PrivaPub.Services
.Modify(u => u.SessionStamp, Guid.NewGuid().ToString("N"))
.Modify(u => u.EndedSessions, new List<EndedSession>())//the new stamp ends those too
.ExecuteAsync(token);
foreach (var link in await _dbEntities.RootToAvatars.Match(ra => ra.RootId == rootId).ExecuteAsync(token))
var links = await _dbEntities.RootToAvatars.Match(ra => ra.RootId == rootId).ExecuteAsync(token);
foreach (var link in links)
{
await _tokens.RevokeBySubjectAsync(link.AvatarId, token);
await _authorizations.RevokeBySubjectAsync(link.AvatarId, token);
}
// their Web Push subscriptions go with their tokens (R9)
await Domain.Social.WebPush.Forget(links.Select(l => l.AvatarId).ToList());
}
// remembered as long as one of its tokens could still be valid; the ones that have expired are forgotten meanwhile
docs/MASTODON-API.md
+70
View File
@@ -0,0 +1,70 @@
# The Mastodon client API: what PrivaPub answers
The walk behind the version PrivaPub advertises (completion program R9, 2026-10-18). Every client route of Mastodon 4.3.0,
4.4.0 and 4.5.0, as their `config/routes/api.rb` and `config/routes.rb` define them, set against PrivaPub's controllers
(`PrivaPub/Api/Mastodon/Controllers`). Since nothing a client relies on is missing, both instance APIs say
`4.5.0 (compatible; PrivaPub <release>)` (`InstanceController.Version`; Mastodon invariant 6 in CLAUDE.md), and
`api_versions.mastodon` is 7, 4.5's own.
- **Served:** the route does what Mastodon's does, within PrivaPub's rules (a token is one persona, per-persona state,
located posts outside the Mastodon API).
- **Stub:** the route answers as Mastodon answers when the feature is off or empty here (an empty list, `{}`, 403, 404 or
422 with Mastodon's message), so clients degrade instead of breaking (Mastodon invariant 5). Each is a decision, named.
- **Missing:** no route. None of these is the client API.
When a route is added to Mastodon, add it here, and keep the advertised version honest.
## Served
| Area | Routes |
|---|---|
| Statuses | `GET /api/v1/statuses` (`id[]`), `POST`, `GET`/`PUT`/`DELETE /api/v1/statuses/:id`, `context`, `history`, `source`, `reblogged_by`, `favourited_by`, `reblog`/`unreblog`, `favourite`/`unfavourite`, `bookmark`/`unbookmark`, `mute`/`unmute`, `pin`/`unpin`; 4.5's `GET …/quotes`, `POST …/quotes/:id/revoke` and `PUT …/interaction_policy` |
| Timelines | `home`, `public`, `tag/:hashtag`, `list/:id` |
| Streaming | `/api/v1/streaming` (WebSocket) and `/api/v1/streaming/*` (server-sent events), `health`: `user`, `user:notification`, `public`, `public:local`, `public:remote`, `hashtag`, `hashtag:local`, `list`, `direct` |
| Accounts | `verify_credentials` (4.4: the `profile` scope too), `update_credentials`, `search`, `lookup`, `relationships`, `familiar_followers` (empty for every account by design: R8), `GET /api/v1/accounts` (`id[]`), `:id`, `:id/statuses`, `followers`, `following`, `lists`, `featured_tags`, `endorsements` (4.4; to the persona itself), `follow`, `unfollow`, `remove_from_followers`, `block`, `unblock`, `mute`, `unmute`, `pin`/`unpin` and 4.4's `endorse`/`unendorse`, `note` |
| Relationships | `blocks`, `mutes`, `domain_blocks` (`GET`, `POST`, `DELETE`), `domain_blocks/preview` (R9: how many follows and followers the block would take), `follow_requests` with `authorize` and `reject`, `endorsements` |
| Notifications | v1 `notifications` (index, `:id`, `clear`, `unread_count`, `:id/dismiss`), `notifications/requests` (index, `:id`, `accept`, `dismiss`, the bulk `accept` and `dismiss`, `merged`), `notifications/policy` (`GET`, `PUT`, `PATCH`); v2 grouped `notifications` (index, `:group_key`, `:group_key/accounts`, `:group_key/dismiss`, `unread_count`, `clear` since R9), `notifications/policy` (`GET`, `PATCH`, `PUT` since R9) |
| Web Push | `POST`, `GET`, `PUT`, `DELETE /api/v1/push/subscription` (R9; CLAUDE.md, Web Push) |
| Conversations | `conversations` (index), `DELETE :id`, `:id/read`, `:id/unread` (R9) |
| Media and polls | `POST /api/v1/media`, `/api/v2/media`, `GET`/`PUT`/`DELETE /api/v1/media/:id`; `polls/:id`, `polls/:id/votes` |
| Lists, filters, tags | `lists` (CRUD, `:id/accounts` `GET`/`POST`/`DELETE`); v1 `filters` and v2 `filters` with `keywords` and `statuses` (both nested and by id); `tags/:name` with `follow`, `unfollow` and 4.4's `feature`, `unfeature`; `followed_tags`; `featured_tags` (index, create, destroy) and `featured_tags/suggestions` |
| Discovery | `/api/v2/search`, `trends` and `trends/tags`, `trends/statuses`, `trends/links`, `directory`, `suggestions` (v1 index and destroy, v2 index) |
| The persona's own | `bookmarks`, `favourites`, `markers` (`GET`, `POST`), `preferences`, `scheduled_statuses` (index, show, update, destroy), `profile/avatar` and `profile/header` (`DELETE`), `reports` |
| The server | v1 and v2 `instance`, `instance/peers` (empty by design: which servers this one talks to is not published), `rules`, `domain_blocks` (the published ones), `extended_description`, `privacy_policy`, 4.4's `terms_of_service` and `terms_of_service/:date`, `languages`, `translation_languages` (none: translation is cut), `activity` (empty unless the administrator publishes it), `custom_emojis`, `announcements` with `dismiss` and `reactions` |
| Apps and OAuth | `POST /api/v1/apps`, `apps/verify_credentials`, `/oauth/authorize`, `/oauth/token`, `/oauth/revoke`, `/.well-known/oauth-authorization-server`, 4.4's `/oauth/userinfo` (`GET` and `POST`, the `profile` scope; R9) |
## Stubs
| Route | Answer | Why |
|---|---|---|
| `POST /api/v1/statuses/:id/translate` | 422 `Translation is not available on this server` | translation is cut (owner decision 2026-10-17) |
| `GET /api/v1/timelines/link` | `[]` | 4.3's timeline of a trending link: not built (trending links themselves are served) |
| `GET /api/v1/annual_reports`, 4.4's `GET /api/v1/annual_reports/:year` | `{"annual_reports": [], "accounts": [], "statuses": []}` | no yearly reports are made |
| 4.4's `POST /api/v1/annual_reports/:year/read` | `{}` | there is none to read |
| `POST /api/v1/accounts` | 403 `This action is not allowed` (Mastodon's answer with registrations closed) | registration through the API is cut: logins are made by setup or invitation on `/clientapi` |
| `POST /api/v1/emails/confirmations` | 403 `This method is only available while the e-mail is awaiting confirmation` | no account here waits for a confirmation |
| `GET /api/v1/emails/check_confirmation` | `true` | the same |
| `GET /api/v1/peers/search` | `[]` | peers are not published, as `instance/peers` |
| 4.4's `GET /api/v1_alpha/async_refreshes/:id` | 404 `Record not found` | PrivaPub never sends `Mastodon-Async-Refresh`: a thread's replies are read in the background and show up |
| `GET /api/v1/accounts/:id/identity_proofs` | `[]` | gone from Mastodon long ago; still asked by old clients |
| every verb under `/api/v1/admin/**`, `GET`/`POST` under `/api/v2/admin/**` | 403 `This action is not allowed`, signed in or not | the Mastodon admin API is cut: moderation is on `/clientapi` |
## Missing, and not the client API
| Route | Why it is not served |
|---|---|
| `GET /api/oembed` | embedding a post in another website; PrivaPub's public pages have no embed view |
| `/api/web/*` (`settings`, `embeds`, `push_subscriptions`) | Mastodon's own web interface, signed in by its session cookie; PrivaPub's client is decePub on `/clientapi` |
| `/oauth/authorized_applications` | Doorkeeper's HTML page of a user's applications, in Mastodon's web interface |
| `/invite/:code` (JSON) | Mastodon's app shows an invitation before registering through the API, which is cut; PrivaPub's invitations are `/clientapi`'s |
| `/.well-known/host-meta` | OStatus-era discovery; every client and server here uses WebFinger directly |
| `POST /api/v1/polls` | in 4.3's route list with no action behind it; gone in 4.4 |
Beyond routes: the `public:media` streams (and their `:local` and `:remote` forms) are not offered; the media-only public
timelines (`only_media`) are, so a client polls them.
## PrivaPub's own, on the same API
`/api/v1/pleroma/statuses/:id/reactions` (Pleroma's emoji reactions), `/api/v1/statuses/:id/downvote` and `undownvote`,
`/api/v1/feeds` and `/api/v1/timelines/feed/:id` (PieFed's and Lemmy's feeds), and `/api/privapub/v1/*` (provenance,
servers, CDNs, events' join and leave): Mastodon invariant 7.
docs/ROADMAP.md
+22 -2
View File
@@ -154,6 +154,24 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
settings form; a `Move` carries the persona's endorsement and note to the new account; featured tags, notes and
endorsements are left out of the persona archive (Mastodon's layout has none), its `following_accounts.csv` now
carries notify and languages.
- R9 Web Push, then the honest version (built 2026-10-18, not yet deployed; `tools/pasture/scenarios/push.sh` with the
pasture's own push service, `tools/pasture/pushsink`): the server's VAPID key (`PushKeys`, made on first need, in
Mongo, never in a backup), Mastodon's `/api/v1/push/subscription` (create, show, update, delete; one subscription
per access token, so per persona), every notification kept (never a filtered one) pushed to the subscriptions whose
alert and policy admit it, as `SendPush` jobs through the `push` client, encrypted as RFC 8291 `aes128gcm` or the
`aesgcm` draft, signed with an ES256 VAPID JWT; subscriptions end with their tokens (revocation, sessions ended,
suspension, deletion, a restore). Then the walk of every Mastodon 4.3, 4.4 and 4.5 client route
(`docs/MASTODON-API.md`), with what it found missing built (`conversations/:id/unread`, `domain_blocks/preview`, v2
`notifications/clear` and `PUT` policy, `/oauth/userinfo` and the `profile` scope on `verify_credentials`) or answered
as stubs (annual reports by year, API registration 403, e-mail confirmation, peer search, async refreshes), and both
instance APIs advertising `4.5.0 (compatible; PrivaPub <release>)`. Chosen while building it, for the owner to
confirm: the payload carries the subscription's own access token, as Mastodon's does (encrypted to that client
alone); what a persona tells itself (its poll ended, the staff's warning) passes the `followed` and `follower` policies
(Mastodon would hold it back); a push service's 4xx other than 404, 410, 408 and 429 drops that push and keeps the
subscription (Mastodon deletes it); the JWT lasts twelve hours (Mastodon's a day); titles are Mastodon's English
subjects and `preferred_locale` is the persona's posting language (no persona has an interface language); six
attempts on the jobs' backoff (Mastodon retries five times); `alerts` answers every alert true or false (Mastodon
echoes those sent); the key lives in Mongo, not in the configuration; `/oauth/userinfo` is served (Mastodon 4.4's).
## Intent
@@ -600,7 +618,8 @@ The first refactor commit is a pure move with namespaces only. Logic changes fol
- `Link` paging on `max_id`, `since_id` and `min_id`; CORS exposes `Link`.
- `Idempotency-Key` honoured on posting statuses.
- **Endpoints:**
- **Instance:** v1 and v2, advertising version `4.2.0 (compatible; PrivaPub)`.
- **Instance:** v1 and v2, advertising version `4.2.0 (compatible; PrivaPub)`. *`4.5.0 (compatible; PrivaPub
<release>)` since R9 (2026-10-18), after the walk in `docs/MASTODON-API.md`.*
- **Accounts:** `verify_credentials`, `update_credentials`, `:id`, `:id/statuses`, `lookup`, `relationships`,
`search`, followers and following, follow and unfollow, `follow_requests`.
- **Statuses:** CRUD, edit, `context`, `history`, `source`, favourite and reblog plus their undos (outbound Like and
@@ -829,7 +848,8 @@ it, raw where it doesn't.
- **Mastodon API:** ~~streaming WebSocket~~ (done 2026-10-05: `/api/v1/streaming` as a WebSocket and as server-sent
events, user, notification, public, hashtag and list streams, each event mapped for its persona; a deletion reaches
only the streams that showed the post), Web Push (gated: outbound traffic to push services; approved 2026-10-17, C R9), grouped
notifications. Then advertise an honest version.
notifications. Then advertise an honest version. *Done in R9 (2026-10-18): Web Push, then `4.5.0 (compatible; PrivaPub
<release>)` once every 4.3, 4.4 and 4.5 client route was served or a named stub (`docs/MASTODON-API.md`).*
- **Backfill:** an author's outbox after following them: **done 2026-10-05** (`OutboxBackfill`, the first page's
public posts, twenty at most, once a day; checked live against Mastodon).
- **Long tail:**
tools/pasture/Caddyfile
+5
View File
@@ -207,6 +207,11 @@ lemmy19.test {
reverse_proxy pasture-lemmy19:8536
}
pushsink.test {
tls internal
reverse_proxy pasture-pushsink:8080
}
flood1.test, flood2.test, flood3.test, flood4.test, flood5.test, flood6.test, flood7.test, flood8.test, flood9.test, flood10.test, flood11.test, flood12.test, flood13.test, flood14.test, flood15.test, flood16.test, flood17.test, flood18.test, flood19.test, flood20.test {
tls internal
reverse_proxy pasture-flood:8080
tools/pasture/lib/interop.sh
+4 -4
View File
@@ -42,14 +42,14 @@ privapub_root() {
echo "$root" | j "print(d['token'])"
}
# privapub_token <persona>: creates the persona under the pasture root if needed and returns a Mastodon token for it,
# through the real OAuth flow (tools/smoke/oauth.sh).
# privapub_token <persona> [scopes]: creates the persona under the pasture root if needed and returns a Mastodon token for
# it, through the real OAuth flow (tools/smoke/oauth.sh); "read write follow" unless other scopes are asked for
privapub_token() {
local persona=$1 jwt
local persona=$1 scopes=${2:-read write follow} jwt
jwt=$(privapub_root)
curl -s -o /dev/null -X POST $P/clientapi/avatar/private/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"userName\":\"$persona\",\"name\":\"$persona of PrivaPub\",\"biography\":\"testing federation\"}"
"$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" "$persona" "read write follow" | cut -d' ' -f1
"$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" "$persona" "$scopes" | cut -d' ' -f1
}
# p_report <authorization header> <account id> <reason> <status id...>: a persona reports an account's posts and asks for
tools/pasture/peers/pushsink.sh
+22
View File
@@ -0,0 +1,22 @@
# pushsink: a Web Push service as pushsink.test, for scenarios/push.sh (tools/pasture/pushsink/pushsink.cs, a .NET
# file-based app published once into .pushsink and run on the runtime image PrivaPub uses). It makes each subscription's
# keys itself, takes PrivaPub's pushes as a push service would (the VAPID JWT checked against the instance API's key, the
# aes128gcm or aesgcm body decrypted) and keeps what it took for the scenario to read: https://pushsink.test:6443/
# subscriptions/<id>/received. It can be told to answer the next push otherwise (/subscriptions/<id>/answer, a 410).
# PrivaPub reaches it through Caddy as it reaches every peer: its push client honours appsettings.Pasture.json's
# Federation exceptions (AcceptAnyCertificate for Caddy's CA, AllowPrivateNetworks), which production refuses at start.
pushsink_up() {
local dotnet=${DOTNET:-$(command -v dotnet || echo ~/.dotnet/dotnet)}
if [ ! -x "$here/.pushsink/pushsink" ] || [ "$here/pushsink/pushsink.cs" -nt "$here/.pushsink/pushsink" ]; then
"$dotnet" publish "$here/pushsink/pushsink.cs" -r linux-x64 --self-contained true -o "$here/.pushsink" -v quiet
fi
podman run -d --replace --name pasture-pushsink --network $net --label pasture=1 -w /app -v "$here/.pushsink:/app:Z,ro" \
-v "$ca/bundle.pem:/etc/ssl/certs/ca-certificates.crt:z,ro" -e SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
-e PUSHSINK_BASE=https://pushsink.test -e PUSHSINK_INSTANCE=https://privapub.test \
mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/pushsink >/dev/null
for _ in $(seq 1 60); do
site pushsink.test -s -o /dev/null -w '%{http_code}' https://pushsink.test:6443/health 2>/dev/null | grep -q 200 && break
sleep 1
done
echo "pushsink: https://pushsink.test:6443"
}
tools/pasture/pushsink/pushsink.cs
+465
View File
@@ -0,0 +1,465 @@
#:sdk Microsoft.NET.Sdk.Web
#:property PublishAot=false
#:property InvariantGlobalization=true
#:property Nullable=disable
// pushsink: a Web Push service for the pasture (tools/pasture/scenarios/push.sh), behind Caddy as pushsink.test, which
// takes PrivaPub's pushes as a browser's push service would and keeps what it took for the scenario to read. It makes the
// keys of each subscription itself, so it can read what PrivaPub encrypted to them, and it checks what a push service
// checks: the VAPID JWT (ES256 by the server's key, the push service's origin as aud, exp in the future and at most a day
// ahead, a mailto: or https: sub) and the encryption (RFC 8291 aes128gcm, or the aesgcm draft with Encryption and
// Crypto-Key). Its own code, not PrivaPub's, so a round trip proves the two agree.
// POST /subscriptions {"server_key": "<the instance's vapid.public_key>"} (optional; without it the
// key is read from $PUSHSINK_INSTANCE/api/v2/instance): a new subscription, its
// keys made here: {"id", "endpoint", "keys": {"p256dh", "auth"}}
// GET /subscriptions/{id} the subscription, with what it answers next
// POST /subscriptions/{id}/answer {"status": 410, "times": 1}: the next push(es) are answered so, after being read
// GET /subscriptions/{id}/received every push it took or refused, oldest first: when, the status it answered,
// whether every check passed (ok) and the errors, the headers, the JWT's claims,
// the decrypted payload (parsed JSON)
// DELETE /subscriptions/{id}/received forgets them
// POST /push/{id} the push service: 201 when every check passes, 403 for the VAPID's failures, 400
// for the rest, 404 for no such subscription, or the status asked for
// GET /health
// Everything is in memory: a new container starts empty.
using System.Buffers.Binary;
using System.Collections.Concurrent;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
var publicBase = (Environment.GetEnvironmentVariable("PUSHSINK_BASE") ?? "https://pushsink.test").TrimEnd('/');
var instance = (Environment.GetEnvironmentVariable("PUSHSINK_INSTANCE") ?? "https://privapub.test").TrimEnd('/');
var listen = Environment.GetEnvironmentVariable("PUSHSINK_LISTEN") ?? "http://0.0.0.0:8080";
var subscriptions = new ConcurrentDictionary<string, Subscription>();
var http = new HttpClient { Timeout = TimeSpan.FromSeconds(10) };
var builder = WebApplication.CreateBuilder();
builder.Logging.SetMinimumLevel(LogLevel.Warning);
builder.WebHost.UseUrls(listen);
builder.WebHost.ConfigureKestrel(k => k.Limits.MaxRequestBodySize = 64 * 1024);
var app = builder.Build();
app.MapGet("/health", () => Results.Json(new { ok = true, subscriptions = subscriptions.Count }));
app.MapPost("/subscriptions", async (HttpContext context) =>
{
var request = context.Request.ContentLength > 0 ? await JsonNode.ParseAsync(context.Request.Body) : null;
var key = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
var subscription = new Subscription(Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(12)), key, RandomNumberGenerator.GetBytes(16),
request?["server_key"]?.GetValue<string>());
subscriptions[subscription.Id] = subscription;
return Results.Json(subscription.View(publicBase));
});
app.MapGet("/subscriptions/{id}", (string id) =>
subscriptions.TryGetValue(id, out var subscription) ? Results.Json(subscription.View(publicBase)) : Results.NotFound());
app.MapPost("/subscriptions/{id}/answer", async (HttpContext context, string id) =>
{
if (!subscriptions.TryGetValue(id, out var subscription))
return Results.NotFound();
var request = await JsonNode.ParseAsync(context.Request.Body);
var status = request?["status"]?.GetValue<int>() ?? 0;
if (status is < 200 or > 599)
return Results.BadRequest(new { error = "status: 200 to 599" });
subscription.Answer(status, request?["times"]?.GetValue<int>() ?? 1);
return Results.Json(subscription.View(publicBase));
});
app.MapGet("/subscriptions/{id}/received", (string id) =>
subscriptions.TryGetValue(id, out var subscription) ? Results.Text(subscription.Received().ToJsonString(), "application/json") : Results.NotFound());
app.MapDelete("/subscriptions/{id}/received", (string id) =>
{
if (!subscriptions.TryGetValue(id, out var subscription))
return Results.NotFound();
subscription.Forget();
return Results.Json(new { });
});
app.MapPost("/push/{id}", async (HttpContext context, string id) =>
{
if (!subscriptions.TryGetValue(id, out var subscription))
return Results.NotFound();
using var buffer = new MemoryStream();
await context.Request.Body.CopyToAsync(buffer);
var check = new Check(context.Request, buffer.ToArray(), subscription, publicBase);
var serverKey = subscription.ServerKey ?? await InstanceKey();
check.Run(serverKey);
var forced = subscription.NextAnswer();
var status = forced ?? (check.Errors.Count == 0 ? 201 : check.VapidFailed ? 403 : 400);
subscription.Keep(check.Record(status));
if (status == 201)
context.Response.Headers.Location = $"{publicBase}/messages/{Guid.NewGuid():N}";
return Results.StatusCode(status);
});
app.Run();
return 0;
// the server's key as its instance API gives it (configuration.vapid.public_key), read once a minute at most
async Task<string> InstanceKey()
{
if (InstanceCache.Key is { } known && DateTime.UtcNow - InstanceCache.At < TimeSpan.FromMinutes(1))
return known;
try
{
var document = JsonNode.Parse(await http.GetStringAsync(instance + "/api/v2/instance"));
InstanceCache.Key = document?["configuration"]?["vapid"]?["public_key"]?.GetValue<string>();
InstanceCache.At = DateTime.UtcNow;
}
catch (Exception)
{
InstanceCache.Key = null;
}
return InstanceCache.Key;
}
static class InstanceCache
{
public static string Key;
public static DateTime At;
}
sealed class Subscription(string id, ECDiffieHellman key, byte[] auth, string serverKey)
{
readonly Lock _gate = new();
readonly List<JsonObject> _received = new();
int _answer;
int _answerTimes;
public string Id => id;
public ECDiffieHellman Key => key;
public byte[] Auth => auth;
public string ServerKey => serverKey;
public byte[] PublicPoint
{
get
{
var parameters = key.ExportParameters(false);
return [0x04, .. parameters.Q.X, .. parameters.Q.Y];
}
}
public JsonObject View(string publicBase)
{
lock (_gate)
return new JsonObject
{
["id"] = id,
["endpoint"] = $"{publicBase}/push/{id}",
["keys"] = new JsonObject { ["p256dh"] = B64.Url(PublicPoint), ["auth"] = B64.Url(auth) },
["server_key"] = serverKey,
["next_answer"] = _answerTimes > 0 ? _answer : null,
["next_answer_times"] = _answerTimes,
["received"] = _received.Count
};
}
public void Answer(int status, int times)
{
lock (_gate)
{
_answer = status;
_answerTimes = Math.Max(1, times);
}
}
public int? NextAnswer()
{
lock (_gate)
{
if (_answerTimes <= 0)
return null;
_answerTimes--;
return _answer;
}
}
public void Keep(JsonObject record)
{
lock (_gate)
_received.Add(record);
}
public JsonArray Received()
{
lock (_gate)
return new JsonArray(_received.Select(r => (JsonNode)r.DeepClone()).ToArray());
}
public void Forget()
{
lock (_gate)
_received.Clear();
}
}
// what a push service checks of one push, and what the sink keeps of it
sealed class Check(HttpRequest request, byte[] body, Subscription subscription, string publicBase)
{
public List<string> Errors { get; } = new();
public bool VapidFailed { get; private set; }
JsonObject _claims;
string _scheme;
string _k;
JsonNode _payload;
string _encoding;
public void Run(string expectedServerKey)
{
_encoding = request.Headers.ContentEncoding.ToString().Trim().ToLowerInvariant();
if (_encoding is not ("aes128gcm" or "aesgcm"))
Errors.Add($"Content-Encoding is '{_encoding}', neither aes128gcm nor aesgcm");
if (!int.TryParse(request.Headers["TTL"].ToString(), out var ttl) || ttl < 0)
Errors.Add("no TTL header (RFC 8030 5.2)");
Vapid(expectedServerKey);
try
{
var plaintext = _encoding switch
{
"aes128gcm" => Decrypt.Aes128Gcm(body, subscription.Key, subscription.PublicPoint, subscription.Auth),
"aesgcm" => Decrypt.AesGcm(body, Param(request.Headers["Encryption"].ToString(), "salt"), Param(request.Headers["Crypto-Key"].ToString(), "dh"),
subscription.Key, subscription.PublicPoint, subscription.Auth),
_ => null
};
if (plaintext != null)
_payload = JsonNode.Parse(plaintext);
}
catch (Exception ex) when (ex is CryptographicException or FormatException or JsonException or ArgumentException or IndexOutOfRangeException)
{
Errors.Add("the body could not be decrypted: " + ex.Message);
}
}
// RFC 8292 3: Authorization: vapid t=<jwt>, k=<key>; the draft's: Authorization: WebPush <jwt> and Crypto-Key's p256ecdsa
void Vapid(string expectedServerKey)
{
var authorization = request.Headers.Authorization.ToString().Trim();
var space = authorization.IndexOf(' ');
_scheme = space > 0 ? authorization[..space] : authorization;
var rest = space > 0 ? authorization[(space + 1)..].Trim() : string.Empty;
string jwt;
if (_scheme.Equals("vapid", StringComparison.OrdinalIgnoreCase))
{
jwt = Param(rest, "t");
_k = Param(rest, "k");
}
else if (_scheme.Equals("WebPush", StringComparison.OrdinalIgnoreCase))
{
jwt = rest;
_k = Param(request.Headers["Crypto-Key"].ToString(), "p256ecdsa");
}
else
{
Fail($"Authorization's scheme is '{_scheme}', neither vapid nor WebPush");
return;
}
if (_encoding == "aes128gcm" && !_scheme.Equals("vapid", StringComparison.OrdinalIgnoreCase))
Fail("an aes128gcm push carries RFC 8292's vapid authorization");
var key = B64.Decode(_k);
if (key is not { Length: 65 } || key[0] != 0x04)
{
Fail("the VAPID key (k or p256ecdsa) is no uncompressed P-256 point");
return;
}
if (string.IsNullOrEmpty(expectedServerKey))
Fail("the server's key is unknown (no server_key given, and the instance API did not say)");
else if (!B64.Decode(expectedServerKey).AsSpan().SequenceEqual(key))
Fail("the VAPID key is not the server's (the instance API's vapid.public_key)");
var parts = jwt?.Split('.') ?? [];
if (parts.Length != 3)
{
Fail("the VAPID token is no JWT");
return;
}
JsonNode header;
try
{
header = JsonNode.Parse(B64.Decode(parts[0]));
_claims = JsonNode.Parse(B64.Decode(parts[1])) as JsonObject;
}
catch (Exception)
{
Fail("the JWT's header or claims are not JSON");
return;
}
if (header?["alg"]?.GetValue<string>() != "ES256")
Fail($"the JWT's alg is '{header?["alg"]}', not ES256");
using (var verifier = ECDsa.Create(new ECParameters { Curve = ECCurve.NamedCurves.nistP256, Q = new ECPoint { X = key[1..33], Y = key[33..] } }))
{
var signature = B64.Decode(parts[2]);
if (signature is not { Length: 64 } || !verifier.VerifyData(Encoding.ASCII.GetBytes(parts[0] + "." + parts[1]), signature, HashAlgorithmName.SHA256,
DSASignatureFormat.IeeeP1363FixedFieldConcatenation))
Fail("the JWT's ES256 signature does not verify with the VAPID key");
}
var audience = new Uri($"{publicBase}/push/{subscription.Id}").GetLeftPart(UriPartial.Authority);
if (_claims?["aud"]?.GetValue<string>() != audience)
Fail($"the JWT's aud is '{_claims?["aud"]}', not the push service's origin {audience}");
var now = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
var exp = _claims?["exp"]?.GetValue<long>() ?? 0;
if (exp <= now)
Fail("the JWT has expired (exp)");
else if (exp > now + 24 * 3600 + 60)
Fail("the JWT's exp is more than 24 hours ahead (RFC 8292 2)");
var sub = _claims?["sub"]?.GetValue<string>() ?? string.Empty;
if (!sub.StartsWith("mailto:", StringComparison.Ordinal) && !sub.StartsWith("https:", StringComparison.Ordinal))
Fail($"the JWT's sub is '{sub}', neither mailto: nor https:");
}
void Fail(string error)
{
VapidFailed = true;
Errors.Add(error);
}
// a parameter of a header written a=b, c=d or a=b;c=d (Crypto-Key may hold several)
static string Param(string header, string name)
{
foreach (var part in (header ?? string.Empty).Split([',', ';'], StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
var equals = part.IndexOf('=');
if (equals > 0 && part[..equals].Trim().Equals(name, StringComparison.OrdinalIgnoreCase))
return part[(equals + 1)..].Trim().Trim('"');
}
return null;
}
public JsonObject Record(int status) => new()
{
["at"] = DateTime.UtcNow.ToString("O"),
["status"] = status,
["ok"] = Errors.Count == 0,
["errors"] = new JsonArray(Errors.Select(e => (JsonNode)e).ToArray()),
["encoding"] = _encoding,
["headers"] = new JsonObject
{
["ttl"] = request.Headers["TTL"].ToString(),
["urgency"] = request.Headers["Urgency"].ToString(),
["content_type"] = request.ContentType,
["content_encoding"] = request.Headers.ContentEncoding.ToString(),
["authorization_scheme"] = _scheme,
["encryption"] = request.Headers["Encryption"].ToString(),
["crypto_key"] = request.Headers["Crypto-Key"].ToString(),
["user_agent"] = request.Headers.UserAgent.ToString()
},
["vapid"] = new JsonObject
{
["k"] = _k,
["aud"] = _claims?["aud"]?.DeepClone(),
["exp"] = _claims?["exp"]?.DeepClone(),
["sub"] = _claims?["sub"]?.DeepClone()
},
["bytes"] = body.Length,
["payload"] = _payload?.DeepClone()
};
}
// the client's side of RFC 8291 and of the aesgcm draft
static class Decrypt
{
public static byte[] Aes128Gcm(byte[] body, ECDiffieHellman client, byte[] clientPublic, byte[] auth)
{
// RFC 8188 2.1: salt (16), record size (4), key id length (1), key id: the sender's public key
var salt = body[..16];
var recordSize = (int)BinaryPrimitives.ReadUInt32BigEndian(body.AsSpan(16, 4));
var idLength = body[20];
var serverPublic = body[21..(21 + idLength)];
if (recordSize < 18)
throw new CryptographicException($"record size {recordSize} is below 18");
var shared = Agree(client, serverPublic);
var keyInfo = Concat(Encoding.ASCII.GetBytes("WebPush: info\0"), clientPublic, serverPublic);
var ikm = HKDF.Expand(HashAlgorithmName.SHA256, HKDF.Extract(HashAlgorithmName.SHA256, shared, auth), 32, keyInfo);
var prk = HKDF.Extract(HashAlgorithmName.SHA256, ikm, salt);
var cek = HKDF.Expand(HashAlgorithmName.SHA256, prk, 16, Encoding.ASCII.GetBytes("Content-Encoding: aes128gcm\0"));
var nonce = HKDF.Expand(HashAlgorithmName.SHA256, prk, 12, Encoding.ASCII.GetBytes("Content-Encoding: nonce\0"));
var content = body[(21 + idLength)..];
var plaintext = new List<byte>();
var records = (content.Length + recordSize - 1) / recordSize;
for (var i = 0; i < records; i++)
{
var record = content[(i * recordSize)..Math.Min(content.Length, (i + 1) * recordSize)];
var recordNonce = (byte[])nonce.Clone();
for (var b = 0; b < 8; b++)//the record's sequence number, XORed into the nonce's last bytes
recordNonce[11 - b] ^= (byte)((long)i >> (8 * b));
var opened = Open(cek, recordNonce, record);
var end = opened.Length - 1;
while (end >= 0 && opened[end] == 0)
end--;
var last = i == records - 1;
if (end < 0 || opened[end] != (last ? 0x02 : 0x01))
throw new CryptographicException("a record's delimiter is wrong");
plaintext.AddRange(opened[..end]);
}
return plaintext.ToArray();
}
public static byte[] AesGcm(byte[] body, string salt, string serverKey, ECDiffieHellman client, byte[] clientPublic, byte[] auth)
{
var saltBytes = B64.Decode(salt) ?? throw new FormatException("no salt in the Encryption header");
var serverPublic = B64.Decode(serverKey) ?? throw new FormatException("no dh in the Crypto-Key header");
var shared = Agree(client, serverPublic);
var ikm = HKDF.Expand(HashAlgorithmName.SHA256, HKDF.Extract(HashAlgorithmName.SHA256, shared, auth), 32, Encoding.ASCII.GetBytes("Content-Encoding: auth\0"));
var context = Concat(Encoding.ASCII.GetBytes("P-256\0"), Length16(clientPublic), clientPublic, Length16(serverPublic), serverPublic);
var prk = HKDF.Extract(HashAlgorithmName.SHA256, ikm, saltBytes);
var cek = HKDF.Expand(HashAlgorithmName.SHA256, prk, 16, Concat(Encoding.ASCII.GetBytes("Content-Encoding: aesgcm\0"), context));
var nonce = HKDF.Expand(HashAlgorithmName.SHA256, prk, 12, Concat(Encoding.ASCII.GetBytes("Content-Encoding: nonce\0"), context));
var opened = Open(cek, nonce, body);
var padding = BinaryPrimitives.ReadUInt16BigEndian(opened);
return opened[(2 + padding)..];
}
static byte[] Agree(ECDiffieHellman client, byte[] serverPublic)
{
if (serverPublic is not { Length: 65 } || serverPublic[0] != 0x04)
throw new CryptographicException("the sender's key is no uncompressed P-256 point");
using var server = ECDiffieHellman.Create(new ECParameters
{
Curve = ECCurve.NamedCurves.nistP256,
Q = new ECPoint { X = serverPublic[1..33], Y = serverPublic[33..] }
});
return client.DeriveRawSecretAgreement(server.PublicKey);
}
static byte[] Open(byte[] key, byte[] nonce, byte[] sealedRecord)
{
if (sealedRecord.Length < 16)
throw new CryptographicException("a record shorter than its tag");
var plaintext = new byte[sealedRecord.Length - 16];
using var aes = new AesGcm(key, 16);
aes.Decrypt(nonce, sealedRecord.AsSpan(0, plaintext.Length), sealedRecord.AsSpan(plaintext.Length), plaintext);
return plaintext;
}
static byte[] Length16(byte[] value)
{
var length = new byte[2];
BinaryPrimitives.WriteUInt16BigEndian(length, (ushort)value.Length);
return length;
}
static byte[] Concat(params byte[][] parts) => parts.SelectMany(p => p).ToArray();
}
static class B64
{
public static string Url(byte[] bytes) => Convert.ToBase64String(bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_');
public static byte[] Decode(string text)
{
if (string.IsNullOrWhiteSpace(text))
return null;
var normal = text.Trim().TrimEnd('=').Replace('-', '+').Replace('_', '/');
normal = normal.PadRight(normal.Length + (4 - normal.Length % 4) % 4, '=');
try
{
return Convert.FromBase64String(normal);
}
catch (FormatException)
{
return null;
}
}
}
tools/pasture/scenarios/push.sh
+168
View File
@@ -0,0 +1,168 @@
# Web Push (completion program R9), with Mastodon and the pushsink. push_alice subscribes with keys the sink made for
# her: a mention from mastouser reaches the sink encrypted (aes128gcm), signed by the server's VAPID key for the sink's
# origin, and decrypts to Mastodon's payload with notification_type mention and her own access token. Turning the mention
# alert off stops it while a favourite still comes; policy=followed stops mastouser's mention until she follows him;
# standard=false is the aesgcm draft and decrypts too, and a new subscription replaces the old; push_bob, a sibling persona
# of the same login, never gets hers, nor she his; DELETE stops the pushes; the sink answering 410 removes the
# subscription; a revoked token takes its subscription with it. Needs the mastodon and pushsink peers (batch A); run
# after featured.
M=https://mastodon.test:6443
mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; }
PS=https://pushsink.test:6443
scurl() { curl -sk --resolve pushsink.test:6443:127.0.0.1 "$@"; }
. "$here/peers/mastodon.sh"
code() { curl -s -o /dev/null -w '%{http_code}' "$@"; }
# sink_new: a subscription of the sink's own, its keys made there, told the server's key to check the VAPID JWT with
sink_new() { scurl -X POST "$PS/subscriptions" -H 'Content-Type: application/json' -d "{\"server_key\":\"$server_key\"}"; }
sink_count() { scurl "$PS/subscriptions/$1" | j "print(d['received'])"; }
# sink_last <id> <python over d, the pushes it took, oldest first>
sink_last() { scurl "$PS/subscriptions/$1/received" | j "$2"; }
sink_waits() { local sid=$1 want=$2; until_true 30 '[ "$(sink_count "$sid")" -ge "$want" ]'; }
# subscribe <authorization header> <sink subscription json> <standard> <alert...> [policy=<policy>]: Mastodon's form
subscribe() {
local auth=$1 sink=$2 standard=$3 args=() a
shift 3
for a in "$@"; do
case $a in policy=*) args+=(-d "data[policy]=${a#policy=}") ;; *) args+=(-d "data[alerts][$a]=true") ;; esac
done
curl -s -X POST -H "$auth" "$P/api/v1/push/subscription" \
--data-urlencode "subscription[endpoint]=$(echo "$sink" | j "print(d['endpoint'])")" \
--data-urlencode "subscription[keys][p256dh]=$(echo "$sink" | j "print(d['keys']['p256dh'])")" \
--data-urlencode "subscription[keys][auth]=$(echo "$sink" | j "print(d['keys']['auth'])")" \
-d "subscription[standard]=$standard" "${args[@]}"
}
# m_mention <handle> <words>: mastouser's public post naming a persona here
m_mention() { mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" --data-urlencode "status=@$1@privapub.test $2" -d visibility=public; }
# notified <authorization header> <words>: the persona holds a mention with these words (it arrived, pushed or not)
notified() { curl -s -H "$1" "$P/api/v1/notifications?types[]=mention&limit=30" | j "print(any('$2' in (n.get('status') or {}).get('content', '') for n in d))"; }
# arrives_unpushed <authorization header> <words> <sink id> <count before>: the mention arrived and the sink got nothing
arrives_unpushed() {
local auth=$1 words=$2 sid=$3 had=$4
until_true 30 '[ "$(notified "$auth" "$words")" = True ]' || return 1
sleep 6
[ "$(sink_count "$sid")" = "$had" ]
}
echo "push"
run=$(date +%s)
AT=$(privapub_token push_alice "read write follow push"); AH="Authorization: Bearer $AT"
BT=$(privapub_token push_bob "read write follow push"); BH="Authorization: Bearer $BT"
MT=$(mastodon_token); MH="Authorization: Bearer $MT"
[ -n "$AT" ] && [ -n "$BT" ] && [ -n "$MT" ] && ok "tokens for push_alice, push_bob (her sibling, the same login) and mastouser" \
|| { ko "a token is missing"; return 1; }
echo " the server's key"
server_key=$(curl -s "$P/api/v2/instance" | j "print(d['configuration']['vapid']['public_key'])")
[ "$(python3 -c "import base64,sys; k=base64.urlsafe_b64decode(sys.argv[1]); print(len(k) == 65 and k[0] == 4)" "$server_key" 2>/dev/null)" = True ] \
&& ok "the instance API gives the VAPID key (an uncompressed P-256 point)" || ko "configuration.vapid: $server_key"
[ "$(curl -s -X POST "$P/api/v1/apps" -d client_name=push-check -d redirect_uris=urn:ietf:wg:oauth:2.0:oob -d scopes=push | j "print(d['vapid_key'])")" = "$server_key" ] \
&& ok "an app's vapid_key is the same key" || ko "the app's vapid_key differs"
[ "$(curl -s "$P/api/v1/instance" | j "print(d['version'].startswith('4.5.0 (compatible; PrivaPub'))")" = True ] \
&& [ "$(curl -s "$P/api/v2/instance" | j "print(d['version'].startswith('4.5.0 (compatible; PrivaPub') and d['api_versions']['mastodon'] == 7)")" = True ] \
&& ok "both instance APIs say 4.5.0 (compatible; PrivaPub ...)" || ko "the version: $(curl -s "$P/api/v2/instance" | j "print(d['version'])")"
echo " a mention, pushed"
m_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=mastouser@mastodon.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
# (a run before may have left the follow the policy check needs gone)
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/unfollow"
for persona in push_alice push_bob; do
mcurl -o /dev/null -H "$MH" "$M/api/v2/search?q=@$persona@privapub.test&resolve=true&type=accounts"
done
sa=$(sink_new); sa_id=$(echo "$sa" | j "print(d['id'])")
made=$(subscribe "$AH" "$sa" true mention favourite policy=all)
[ "$(echo "$made" | j "print(d['standard'] is True and d['alerts']['mention'] is True and d['alerts']['reblog'] is False and d['policy'] == 'all' and d['server_key'] == '$server_key' and d['endpoint'].startswith('https://pushsink.test/push/'))")" = True ] \
&& ok "push_alice subscribes with the sink's keys (standard)" || ko "the subscription: $(echo "$made" | head -c 400)"
[ "$(curl -s -H "$AH" "$P/api/v1/push/subscription" | j "print(d['id'])")" = "$(echo "$made" | j "print(d['id'])")" ] \
&& ok "GET answers it" || ko "GET: $(curl -s -H "$AH" "$P/api/v1/push/subscription" | head -c 300)"
[ "$(code -H "$BH" "$P/api/v1/push/subscription")" = 404 ] && ok "push_bob's token has none" || ko "push_bob's token sees a subscription"
m_mention push_alice "first push $run"
sink_waits "$sa_id" 1 \
&& [ "$(sink_last "$sa_id" "r = d[-1]; p = r['payload'] or {}; print(r['ok'] and r['status'] == 201 and r['encoding'] == 'aes128gcm' and p.get('notification_type') == 'mention' and 'first push $run' in p.get('body', ''))")" = True ] \
&& ok "mastouser's mention reaches the sink, aes128gcm, and decrypts to a mention" || ko "the push: $(scurl "$PS/subscriptions/$sa_id/received" | head -c 800)"
[ "$(sink_last "$sa_id" "print(d[-1]['payload']['access_token'] == '$AT')")" = True ] && ok "it carries push_alice's own access token" \
|| ko "the push's access_token is not hers"
[ "$(sink_last "$sa_id" "r = d[-1]; v = r['vapid']; print(v['aud'] == 'https://pushsink.test' and v['sub'].startswith(('mailto:', 'https:')) and 'push_alice' not in v['sub'] and r['headers']['authorization_scheme'] == 'vapid' and int(r['headers']['ttl']) > 0 and r['headers']['urgency'] == 'normal')")" = True ] \
&& ok "signed for the sink's origin by the server, TTL and Urgency set, no persona in the sub" || ko "the VAPID claims: $(sink_last "$sa_id" "print(d[-1]['vapid'], d[-1]['headers'])")"
pushed_id=$(sink_last "$sa_id" "print(d[-1]['payload']['notification_id'])")
[ "$(curl -s -H "$AH" "$P/api/v1/notifications/$pushed_id" | j "print(d['type'] == 'mention' and 'first push $run' in d['status']['content'])")" = True ] \
&& ok "its notification_id is her notification" || ko "notification $pushed_id: $(curl -s -H "$AH" "$P/api/v1/notifications/$pushed_id" | head -c 200)"
[ "$(sink_last "$sa_id" "p = d[-1]['payload']; print(p['icon'].startswith('https://privapub.test/') and 'mastouser' in p['title'] and p['preferred_locale'] != '')")" = True ] \
&& ok "its icon goes through PrivaPub, its title names mastouser" || ko "icon and title: $(sink_last "$sa_id" "print(d[-1]['payload'])")"
echo " alerts and policy"
changed=$(curl -s -X PUT -H "$AH" "$P/api/v1/push/subscription" -d "data[alerts][favourite]=true")
[ "$(echo "$changed" | j "print(d['alerts']['mention'] is False and d['alerts']['favourite'] is True and d['policy'] == 'all')")" = True ] \
&& ok "PUT turns the mention alert off" || ko "PUT: $(echo "$changed" | head -c 300)"
before=$(sink_count "$sa_id")
m_mention push_alice "unwanted $run"
arrives_unpushed "$AH" "unwanted $run" "$sa_id" "$before" && ok "with the alert off, the mention arrives unpushed" || ko "a mention was pushed with its alert off"
alice_post=$(curl -s -X POST -H "$AH" "$P/api/v1/statuses" --data-urlencode "status=push_alice posts $run" -d visibility=public | j "print(d['uri'])")
m_post=$(mcurl -H "$MH" "$M/api/v2/search?q=$alice_post&resolve=true&type=statuses" | j "print(d['statuses'][0]['id'])")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses/$m_post/favourite"
sink_waits "$sa_id" $((before + 1)) && [ "$(sink_last "$sa_id" "print(d[-1]['ok'] and d[-1]['payload']['notification_type'])")" = favourite ] \
&& ok "his favourite of her post is pushed" || ko "no favourite push: $(sink_last "$sa_id" "print(d[-1])")"
curl -s -o /dev/null -X PUT -H "$AH" "$P/api/v1/push/subscription" -d "data[alerts][mention]=true" -d "data[policy]=followed"
before=$(sink_count "$sa_id")
m_mention push_alice "a stranger $run"
arrives_unpushed "$AH" "a stranger $run" "$sa_id" "$before" && ok "policy=followed: mastouser, whom she does not follow, is not pushed" \
|| ko "a non-followed sender was pushed"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/follow"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/relationships?id[]=$m_on_p" | j "print(d[0][\"following\"])")" = True ]' \
&& ok "push_alice follows mastouser" || ko "her follow was not accepted"
m_mention push_alice "a friend now $run"
sink_waits "$sa_id" $((before + 1)) && [ "$(sink_last "$sa_id" "print('a friend now $run' in d[-1]['payload']['body'])")" = True ] \
&& ok "once followed, his mention is pushed" || ko "a followed sender was not pushed"
echo " the aesgcm draft"
sl=$(sink_new); sl_id=$(echo "$sl" | j "print(d['id'])")
[ "$(subscribe "$AH" "$sl" false mention | j "print(d['standard'])")" = False ] && ok "push_alice subscribes again, standard=false" || ko "the legacy subscription"
before=$(sink_count "$sa_id")
m_mention push_alice "the draft $run"
sink_waits "$sl_id" 1 \
&& [ "$(sink_last "$sl_id" "r = d[-1]; print(r['ok'] and r['encoding'] == 'aesgcm' and r['headers']['authorization_scheme'] == 'WebPush' and 'p256ecdsa=' in r['headers']['crypto_key'] and r['headers']['encryption'].startswith('salt=') and r['payload']['notification_type'] == 'mention' and r['payload']['access_token'] == '$AT')")" = True ] \
&& ok "aesgcm, with Encryption and Crypto-Key, decrypts to her mention" || ko "the aesgcm push: $(scurl "$PS/subscriptions/$sl_id/received" | head -c 800)"
sleep 3
[ "$(sink_count "$sa_id")" = "$before" ] && ok "the subscription it replaced gets nothing" || ko "the replaced subscription is still pushed"
echo " siblings"
sb=$(sink_new); sb_id=$(echo "$sb" | j "print(d['id'])")
subscribe "$BH" "$sb" true mention policy=all >/dev/null
alice_before=$(sink_count "$sl_id")
m_mention push_alice "for alice only $run"
sink_waits "$sl_id" $((alice_before + 1)); sleep 4
[ "$(sink_count "$sb_id")" = 0 ] && ok "push_bob's subscription never gets push_alice's mention" || ko "a sibling's subscription got her push"
m_mention push_bob "for bob only $run"
sink_waits "$sb_id" 1 && [ "$(sink_last "$sb_id" "p = d[-1]['payload']; print(p['access_token'] == '$BT' and 'for bob only $run' in p['body'])")" = True ] \
&& ok "push_bob's own mention, with his own token" || ko "push_bob's push: $(scurl "$PS/subscriptions/$sb_id/received" | head -c 400)"
sleep 3
[ "$(sink_count "$sl_id")" = "$((alice_before + 1))" ] && [ "$(sink_last "$sl_id" "print(all(r['payload']['access_token'] == '$AT' for r in d))")" = True ] \
&& ok "push_alice gets nothing of his, and only her token" || ko "push_alice's subscription got push_bob's"
[ "$(curl -s -X DELETE -H "$BH" "$P/api/v1/push/subscription")" = "{}" ] && ok "push_bob unsubscribes" || ko "push_bob's DELETE"
echo " DELETE, 410, revocation"
[ "$(curl -s -X DELETE -H "$AH" "$P/api/v1/push/subscription")" = "{}" ] && [ "$(code -H "$AH" "$P/api/v1/push/subscription")" = 404 ] \
&& ok "DELETE: the subscription is gone" || ko "DELETE left it"
before=$(sink_count "$sl_id")
m_mention push_alice "after delete $run"
arrives_unpushed "$AH" "after delete $run" "$sl_id" "$before" && ok "nothing is pushed after DELETE" || ko "pushed after DELETE"
sg=$(sink_new); sg_id=$(echo "$sg" | j "print(d['id'])")
subscribe "$AH" "$sg" true mention >/dev/null
scurl -o /dev/null -X POST "$PS/subscriptions/$sg_id/answer" -H 'Content-Type: application/json' -d '{"status": 410}'
m_mention push_alice "gone $run"
sink_waits "$sg_id" 1 && until_true 15 '[ "$(code -H "$AH" "$P/api/v1/push/subscription")" = 404 ]' \
&& ok "the sink answering 410 removes the subscription" || ko "after a 410: $(curl -s -H "$AH" "$P/api/v1/push/subscription" | head -c 200)"
before=$(sink_count "$sg_id")
m_mention push_alice "still gone $run"
arrives_unpushed "$AH" "still gone $run" "$sg_id" "$before" && ok "and nothing more goes there" || ko "pushed after the 410"
oauth=$("$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" push_alice "read write follow push")
RT=$(echo "$oauth" | cut -d' ' -f1); rcid=$(echo "$oauth" | cut -d' ' -f2); rcs=$(echo "$oauth" | cut -d' ' -f3)
sr=$(sink_new); sr_id=$(echo "$sr" | j "print(d['id'])")
subscribe "Authorization: Bearer $RT" "$sr" true mention >/dev/null
curl -s -o /dev/null -X POST "$P/oauth/revoke" -d "token=$RT" -d "client_id=$rcid" -d "client_secret=$rcs"
[ "$(code -H "Authorization: Bearer $RT" "$P/api/v1/push/subscription")" = 401 ] && ok "the revoked token is refused" || ko "the revoked token still works"
m_mention push_alice "revoked $run"
arrives_unpushed "$AH" "revoked $run" "$sr_id" 0 && ok "a revoked token's subscription is pushed nothing" || ko "pushed to a revoked token's subscription"
[ "$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'db.PushSubscription.countDocuments()' 2>/dev/null | tail -1)" = 0 ] \
&& ok "no subscription is left" || ko "subscriptions left: $(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'db.PushSubscription.countDocuments()' 2>/dev/null | tail -1)"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/unfollow"
tools/pasture/sweep.sh
+5 -4
View File
@@ -6,11 +6,12 @@
# PrivaPub runs with SecureMode on, as in production, unless PRIVAPUB_ENV says otherwise; DOTNET and PASTURE_PORT reach
# run.sh as they are. Each batch's output is kept in out/sweep-<batch>.log.
# usage: tools/pasture/sweep.sh [A|B|C|D ...] (default: every batch, in that order)
# A gts mastodon misskey sharkey akkoma, with relay, aoderelay and mailpit; scenarios: those five, threads followsync
# A gts mastodon misskey sharkey akkoma, with relay, aoderelay, mailpit and pushsink; scenarios: those five, threads followsync
# pins moves relay (relay's covers both relays) accounts (mailpit's) instance (the server's own settings: it changes
# them all and puts them back) moderation (gts and mastodon; after instance, whose log check expects only the
# administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation) featured (mastodon: featured
# tags, notify, the direct stream, endorsements and notes per persona; after emoji)
# tags, notify, the direct stream, endorsements and notes per persona; after emoji) push (mastodon and pushsink: Web
# Push, encrypted both ways, alerts, policies, siblings, DELETE, 410 and revocation; after featured)
# B lemmy lemmy19 piefed mbin nodebb
# C peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale
# D friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla
@@ -22,7 +23,7 @@ export PRIVAPUB_ENV=${PRIVAPUB_ENV:-Federation__SecureMode=true}
batch_peers() {
case "$1" in
A) echo "gts mastodon misskey sharkey akkoma relay aoderelay mailpit" ;;
A) echo "gts mastodon misskey sharkey akkoma relay aoderelay mailpit pushsink" ;;
B) echo "lemmy lemmy19 piefed mbin nodebb" ;;
C) echo "peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale" ;;
D) echo "friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla" ;;
@@ -30,7 +31,7 @@ batch_peers() {
}
batch_scenarios() {
case "$1" in
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji featured" ;;
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji featured push" ;;
*) batch_peers "$1" ;;
esac
}
tools/smoke/mastodon-api.sh
+8 -1
View File
@@ -6,12 +6,19 @@ BASE="${1:?base url}"; TOKEN="${2:-}"
fail() { echo "::error::$*"; exit 1; }
json() { python3 -c "import sys,json; d=json.load(sys.stdin); $1"; }
# the version clients branch on (Mastodon invariant 6): Mastodon 4.5's, then PrivaPub's release
version=$(curl -fsS "$BASE/api/v1/instance" | json "print(d['version'])") || fail "instance v1"
curl -fsS "$BASE/api/v2/instance" | json "assert d['configuration']['statuses']['max_characters'] > 0" || fail "instance v2"
case "$version" in "4.5.0 (compatible; PrivaPub"*) ;; *) fail "instance v1 says version '$version', not 4.5.0 (compatible; PrivaPub ...)" ;; esac
v2=$(curl -fsS "$BASE/api/v2/instance") || fail "instance v2"
echo "$v2" | json "assert d['configuration']['statuses']['max_characters'] > 0" || fail "instance v2"
echo "$v2" | json "assert d['version'] == '$version' and d['api_versions']['mastodon'] == 7" || fail "instance v2's version or api_versions"
vapid=$(echo "$v2" | json "import base64; k = d['configuration']['vapid']['public_key']; assert len(base64.urlsafe_b64decode(k)) == 65; print(k)") \
|| fail "instance v2 gives no VAPID key (configuration.vapid.public_key)"
curl -fsS "$BASE/.well-known/oauth-authorization-server" | json "assert d['token_endpoint'].endswith('/oauth/token')" || fail "oauth discovery"
app=$(curl -fsS -X POST "$BASE/api/v1/apps" -d 'client_name=privapub-smoke&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read') || fail "app registration"
id=$(echo "$app" | json "print(d['client_id'])"); secret=$(echo "$app" | json "print(d['client_secret'])")
[ "$(echo "$app" | json "print(d['vapid_key'])")" = "$vapid" ] || fail "the app's vapid_key is not the instance's VAPID key"
app_token=$(curl -fsS -X POST "$BASE/oauth/token" -d "grant_type=client_credentials&client_id=$id&client_secret=$secret&scope=read" | json "print(d['access_token'])") || fail "client credentials"
curl -fsS -H "Authorization: Bearer $app_token" "$BASE/api/v1/apps/verify_credentials" | json "assert d['name'] == 'privapub-smoke'" || fail "app verify_credentials"
code=$(curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer $app_token" "$BASE/api/v1/accounts/verify_credentials")