Compare commits

..
9 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 ad8d353f9a docs: FEDERATION.md tested against six peers with signed fetches, the shipped FEPs; v1.19.1 and v1.20.0 in ROADMAP
Build / Build (push) Successful in 5m14s
Deploy / privapub.thepra.dev (push) Successful in 5m25s
- FEDERATION.md:
  - It said only GoToSocial was tested live. It now lists all six pasture peers, run with signed fetches required as
    production runs, and what is checked both ways.
  - FEP-044f, FEP-9967, FEP-c0e0 and FEP-5feb moved from planned to supported.
  - indexable and discoverable are described as the settings they are.
- ROADMAP: v1.19.1 deployed and verified, v1.20.0 (phase 4).
- Pasture, Akkoma: its like and boost count gets two minutes. In the final clean pass, all six peers with SecureMode on
  gave 245 passed, 1 failed, 4 expected. The failure was this count, which Akkoma's job queue was slow to update
  under the load of six peers; two reruns gave 44/44.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 04:16:16 +02:00
thepraandClaude Opus 5.5 e2f61ede56 Everything on, phase 4b: ids that resolve, a hashtag page, grouped notifications, remote accounts' real counts
- A community's announces resolve, as FEDERATION.md and ROADMAP always said Announce ids do. GroupDistributor keeps
  each one it sends (GroupAnnouncement, unique by id). /grunts serves it while the post it carries is shown and 410
  after, and also serves the announce-{postId} ids the group outbox lists, which now keep a stable `published`
  instead of the time of the fetch.
- A persona's boost points at the boosted post: its `url` in the Mastodon API is the boosted post's page, and a browser
  following the boost's id is redirected there instead of getting JSON.
- /tags/{tag}, where every Hashtag link we send points, is now a public page of this server's public posts with that
  tag, with the same strict CSP and noindex as the profile pages.
- Grouped notifications (/api/v2/notifications, its unread count, a group, its accounts and dismiss). We advertise
  api_versions.mastodon = 7 so clients show quotes, and clients that trust it call these; they answered 404. Likes and
  boosts of one post group together, as do follows within an hour. The version string stays 4.2.0 until streaming
  and Web Push exist.
- A remote account's follower, following and post counts are what its server publishes. AccountCountsJob reads its
  collections' totalItems from its own origin, signed by the instance actor, at most daily and only after the account
  was fetched, never when someone looks. They used to be 0.
- The other ids that do not resolve are documented as such: Update, Delete, EmojiReact, QuoteRequest and its answers,
  Flag, Ignore and poll votes.

676 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:56:18 +02:00
thepraandClaude Opus 5.5 8f25bf056d Everything on, phase 4a: one answer everywhere for counts, search, collections and the instance API
Owner decision 2026-10-04: fix the mismatches and every other mismatch of the same kind.

- One counting rule (Domain/Privacy/Counted), Mastodon's. It is used for a persona's statuses_count, its outbox
  totalItems, NodeInfo localPosts and the instance status_count, which used to count four different things. It
  counts every post that is neither deleted nor a DM, boosts included, and circle and located posts too (owner
  decision). A group's count includes its remote members' posts.
- Users. Personas of banned or deleted roots no longer count, and are not found in search. NodeInfo now gives
  activeMonth and activeHalfyear, and the v2 instance gives active_month instead of a constant 0.
- replies_count counts only public and unlisted replies, so it no longer tells anyone that a private reply exists.
  Migration _012 recounts it.
- A remote account that deletes itself takes everything out of every count (GoneActors): its likes, downvotes,
  reactions and poll votes go and their counters come back, as do its boosts', replies' and quotes' counts, and its
  notifications. Lookups, account lists, search and favourited_by no longer show it. Migration _012 applies this to
  accounts already gone.
- Deleting a post also deletes its pins and the local boosts of it.
- /stalking gives the same total as following_count. Members are still never listed, and hide_collections is now
  always true, since the setting never did anything.
- Joining a community by invitation is following it, so /flock and /groupies agree; leaving unfollows.
- Search. Anyone may search, as on Mastodon; resolve and offset need a sign-in, offset pages, and deleted accounts
  are never found.
- notifications/unread_count counts what the list shows, and the owner's follower and following lists page with
  Link.
- The instance API advertises what is enforced:
  - max_characters, now enforced with a 422;
  - max_pinned_statuses = MaxPins;
  - the media types and limits MediaService and MediaOptions accept;
  - PollService's limits;
  - the configured languages;
  - no streaming URL until streaming exists.

  domain_count counts the servers we have exchanged with; which ones stays unpublished (peers is empty).
- Routes Mastodon answers now answer instead of 404:
  - directory, tags/{name}, timelines/link and identity_proofs;
  - instance/languages, translation_languages, domain_blocks and privacy_policy;
  - the v1 and v2 notification policy, and notification requests.

Also, from phase 3: a recovered password ends /clientapi sessions through a per-root SessionStamp claim instead of
comparing the JWT's whole-second nbf with the change time. That comparison let a token issued in the same second
survive, which made a test flaky.

671 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:48:40 +02:00
thepraandClaude Opus 5.5 123ba39520 A post whose author cannot be shown is left out of a list, never a 500
Build / Build (push) Successful in 4m19s
Deploy / privapub.thepra.dev (push) Successful in 4m50s
The v1.19.0 deploy's tests failed where build.yml's passed. A test stored a public remote post with no author id, and
when the public timeline test ran after it, MastodonMapper.Statuses looked that null up in its accounts and answered
500. The mapper now skips such a post, and the test stores a real author.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:46:48 +02:00
thepraandClaude Opus 5.5 f6dbf71964 Everything on, phase 2 completed: SecureMode on in production, checked by the deploy
Build / Build (push) Successful in 5m5s
Deploy / privapub.thepra.dev (push) Failing after 4m39s
Owner decision 2026-10-04: SecureMode on once the pasture passes with it.

- Both clean pasture passes were run over all six peers:
  - normally: 246 passed, 0 failed;
  - with Federation__SecureMode=true: every federation check passed. The only failures were four checks expecting an
    unsigned GET to get 404 or 410 where SecureMode answers 401. Those checks now go through `unserved` and
    `gone_unsigned` (lib/interop.sh), which expect 401 when SecureMode is on.
- Circle posts now reach their member on GoToSocial and Mastodon, and survive Mastodon's signed refetch, as does a
  followers-only post. The GoToSocial expected failure is gone.
- appsettings.Production.json turns SecureMode on.
- The deploy now checks that an unsigned GET of @thepra answers 401 and that a browser is redirected. It reads
  @thepra's discoverability through the Mastodon API, since the actor is no longer readable unsigned.
- docs/INTEROP.md (Mastodon, GoToSocial), CLAUDE.md and ROADMAP updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:34:48 +02:00
thepraandClaude Opus 5.5 8c2eba6cbb Everything on, phase 3: sign-in and recovery tell nothing, recovered passwords end sessions, deleted roots are gone everywhere
Owner decision 2026-10-04: fix the account privacy findings.

- Sign-in. Every failure answers "That username and password do not match." after the same work: an unknown login
  is hashed against a decoy, and the comparison is constant-time. "Banned" is told only to someone who gave the right
  password. This covers /clientapi/user/login, /invitation/login and /oauth/login.
- Recovery.
  - Every request answers the same sentence and queues a SendRecovery job, whether or not the account exists or has an
    email. The lookup, the code and SMTP move to RecoveryJob, so neither the answer nor its timing says anything.
  - Codes are kept only as a SHA-256 hash, for one hour. Migration _011 drops the plaintext ones, which never expired.
  - A recovered password ends every session of the root. RootSessions sets CredentialsChangedAt, which JwtEvents
    checks against the JWT's issue time, now stamped as nbf, and revokes each persona's OAuth tokens and authorizations.
- Deleting a root (RootRemoval: the admin route, or the restored self-delete at /clientapi/user/delete, which asks for
  the password).
  - Its sessions end.
  - Each persona and each group it owns sends Delete{Actor} to its followers, its members and the accounts it follows.
  - The personas' posts are emptied.
  - /peasants/{name} answers 410 with a Tombstone (formerType Person or Group), as do its inbox and WebFinger, through
    LocalActorService.Gone. The names stay reserved.
  - The root keeps only a unique `deleted-{id}` name; the second deletion on an instance used to collide on
    "Deleted user".

Also, from phase 2's pasture: GoToSocial files a circle post like a DM and shows it only to accounts it mentions. Each
member's copy, and a member's refetch, now also mentions that member silently. The GoToSocial scenario checks circle
posts in conversations, like DMs, and they pass there now, as on Mastodon.

657 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:16:59 +02:00
thepraandClaude Opus 5.5 fcd35f5043 Everything on, phase 2: circle posts for everyone, private posts on signed refetch, browsers past SecureMode
Circles (owner decision 2026-10-04: fix them for compatibility):
- Mastodon 4.7 and GoToSocial drop a post that names none of their accounts, and a circle post named only the circle
  and its /flock. OutboxPublisher.Publish now sends each member a copy that also names that member in `cc`, on the
  activity and on the object, and names no other member. The Create, every Update (edit, poll, quote approval, policy,
  through the new PublishUpdate) and the Delete (StatusService.Remove now uses Publish) all go that way.
- UpdateOf renders with the post's group, so an Update keeps a circle post's `audience` and a community post's `Page`
  and title.
- A reply to a circle post stays in the circle, whichever client wrote it.
- A circle post can no longer quote a post that needs permission: asking would show the circle post to its author.

Posts that are not public, on refetch (SignedFetchAuthorizer.MayRead):
- Followers-only, direct and circle posts are served to a signed request from someone they were for, or from the
  instance actor of a server where one of them lives. That is a follower or an addressed account, an addressed
  account, or a member. Everyone else still gets 404.
- Once deleted they answer those readers 410. Mastodon deletes its copy when a refetch answers 404.
- A circle refetch names the requesting member, or the members on the requesting server, as the delivered copy did.
- /grunts/create-{id} serves the same.
- /peasants/{name}/whispers/{id}, a DM's `context`, was never routed. It is now the conversation's posts, for its
  participants only.

SecureMode lets browsers through to the redirect to the public page, instead of answering them 401.

653 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:01:14 +02:00
thepraandClaude Opus 5.5 5f56681c01 Everything on, phase 1: geolocation fetches itself, the deploy signs in as @thepra, the crawler is on, sign-up by invitation
Build / Build (push) Successful in 5m1s
Deploy / privapub.thepra.dev (push) Successful in 5m39s
Owner decisions (2026-10-04, recorded in docs/ROADMAP.md): production runs everything that is built, and nothing waits
on a person running a command.

- Geolocation updates itself. GeoUpdater, a hosted service, checks daily whether each DB-IP Lite database was built this
  month. If not, it fetches this month's, or last month's early in the month. It installs a file only once it opens as
  the right kind of database, then swaps it in atomically, and the locator reloads at once. Lookups now run under the
  lock, so a reload can no longer dispose a reader mid-lookup. The systemd timer, its script and their setup.sh lines
  are gone: the root step they needed never happened, and none is needed now. /stargazing names the database in use.
- The admin CLI runs after the app is built, with every service and nothing started.
  - `create-root <login> [--admin]` takes the password on stdin; it is how the first login is made while sign-up is
    closed.
  - `smoke <persona>` keeps the root `deploy-smoke` and an undiscoverable persona, and gives the root a new password
    on every run.
- The deploy signs in as @thepra. It runs the CLI, gets a token through the real OAuth flow (tools/smoke/oauth.sh,
  moved out of the pasture's privapub_token, which now uses it), checks the signed-in API and that @thepra is
  undiscoverable, then revokes the token. PRIVAPUB_SMOKE_TOKEN is gone.
- The deploy also fails when:
  - NodeInfo and the instance API disagree about registrations;
  - /stargazing does not say the crawler is on;
  - the geolocation databases are missing or more than 40 days old.
- The crawler is on in production, seeded with ten large servers of different kinds. FEDERATION.md now describes it
  and how to opt out.
- One registrations switch (Registrations:Mode, default Invitations; Open in tests and the pasture). It is read by
  open sign-up (403 when closed), NodeInfo `openRegistrations`, and v1 and v2 of the instance API, so they can no longer
  disagree. Before, NodeInfo said open and the instance API said closed. Group invitations always work, so
  invites_enabled is true.
- A persona edit through /clientapi no longer resets what the Mastodon API set (discoverable, locked, quote policy…):
  the theme is merged into the settings instead of replacing them.

650 tests pass. The deploy's smoke step was rehearsed against the pasture's PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 02:37:38 +02:00
thepraandClaude Opus 5.5 cd5eb25948 docs: v1.17.1 deployed and verified; what the T/M track still needs from the owner
Build / Build (push) Successful in 4m8s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 15:35:14 +02:00
86 changed files with 3036 additions and 545 deletions

No files matched your search

+40 -4
View File
@@ -90,13 +90,16 @@ jobs:
fi fi
- name: Verify what is being served - name: Verify what is being served
env:
SMOKE_TOKEN: ${{ secrets.PRIVAPUB_SMOKE_TOKEN }}
run: | run: |
served=$(curl -fsS "$PUBLIC_URL/build.json" | python3 -c "import json,sys; print(json.load(sys.stdin).get('commit',''))") served=$(curl -fsS "$PUBLIC_URL/build.json" | python3 -c "import json,sys; print(json.load(sys.stdin).get('commit',''))")
[ "$served" = "$BUILD_COMMIT" ] || { echo "::error::served build is '$served', expected '$BUILD_COMMIT'"; exit 1; } [ "$served" = "$BUILD_COMMIT" ] || { echo "::error::served build is '$served', expected '$BUILD_COMMIT'"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/privapub") code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/privapub")
[ "$code" = "200" ] || { echo "::error::the instance actor answered $code"; exit 1; } [ "$code" = "200" ] || { echo "::error::the instance actor answered $code"; exit 1; }
# SecureMode (owner decision 2026-10-04): an unsigned reader gets 401 from a persona, a browser the public page
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/thepra")
[ "$code" = "401" ] || { echo "::error::an unsigned GET of a persona answered $code, not 401: SecureMode is off"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: text/html' "$PUBLIC_URL/peasants/thepra")
[ "$code" = "302" ] || { echo "::error::a browser asking for a persona got $code, not a redirect"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/.well-known/nodeinfo") code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/.well-known/nodeinfo")
[ "$code" = "200" ] || { echo "::error::nodeinfo answered $code"; exit 1; } [ "$code" = "200" ] || { echo "::error::nodeinfo answered $code"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/swagger/index.html") code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/swagger/index.html")
@@ -106,6 +109,39 @@ jobs:
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/activity+json' \ code=$(curl -s -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/activity+json' \
--data '{"type":"Follow","actor":"https://example.org/users/x","object":"'"$PUBLIC_URL"'/peasants/privapub"}' "$PUBLIC_URL/human-centipede") --data '{"type":"Follow","actor":"https://example.org/users/x","object":"'"$PUBLIC_URL"'/peasants/privapub"}' "$PUBLIC_URL/human-centipede")
[ "$code" = "401" ] || { echo "::error::an unsigned inbox POST answered $code"; exit 1; } [ "$code" = "401" ] || { echo "::error::an unsigned inbox POST answered $code"; exit 1; }
tools/smoke/mastodon-api.sh "$PUBLIC_URL" "$SMOKE_TOKEN" tools/smoke/mastodon-api.sh "$PUBLIC_URL"
[ -n "$SMOKE_TOKEN" ] || echo "::warning::PRIVAPUB_SMOKE_TOKEN is not set, so the signed-in API was not checked" open=$(curl -fsS "$PUBLIC_URL/nodeinfo/2.1" | python3 -c "import json,sys; print(json.load(sys.stdin)['openRegistrations'])")
enabled=$(curl -fsS "$PUBLIC_URL/api/v2/instance" | python3 -c "import json,sys; print(json.load(sys.stdin)['registrations']['enabled'])")
[ "$open" = "$enabled" ] || { echo "::error::NodeInfo says registrations are $open, the instance API $enabled"; exit 1; }
curl -fsS "$PUBLIC_URL/stargazing" | grep -q 'The crawler is <strong>on</strong>' \
|| { echo "::error::/stargazing does not say the crawler is on"; exit 1; }
echo "::notice::serving $served" echo "::notice::serving $served"
# @thepra is the deploy's own persona (owner decision, 2026-10-04): the server's CLI makes or keeps it, undiscoverable,
# and gives its root a new password on every deploy, so no secret is stored and nobody has to create anything.
- name: Sign in as @thepra and check the signed-in API
run: |
creds=$(cd "$WEB_ROOT" && ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin smoke thepra | grep -E '^deploy-smoke [^ ]+$' | tail -1)
[ -n "$creds" ] || { echo "::error::the smoke persona could not be prepared"; exit 1; }
read -r login password <<< "$creds"
echo "::add-mask::$password"
read -r token cid cs <<< "$(tools/smoke/oauth.sh "$PUBLIC_URL" "$login" "$password" thepra read)"
echo "::add-mask::$token"
tools/smoke/mastodon-api.sh "$PUBLIC_URL" "$token"
discoverable=$(curl -fsS -H "Authorization: Bearer $token" "$PUBLIC_URL/api/v1/accounts/verify_credentials" | python3 -c "import json,sys; print(json.load(sys.stdin).get('discoverable'))")
[ "$discoverable" = "False" ] || { echo "::error::@thepra is discoverable"; exit 1; }
curl -fsS -o /dev/null -X POST "$PUBLIC_URL/oauth/revoke" --data-urlencode "token=$token" \
--data-urlencode "client_id=$cid" --data-urlencode "client_secret=$cs"
echo "::notice::signed in as @thepra"
# The server fetches DB-IP Lite itself (GeoUpdater) about 30 s after it starts and then daily; the deploy only checks.
- name: Geolocation databases are current
run: |
for kind in city asn; do
db="/var/lib/privapub/geo/dbip-$kind-lite.mmdb"
for i in $(seq 1 60); do [ -s "$db" ] && break; sleep 10; done
[ -s "$db" ] || { echo "::error::$db is missing: the server has not fetched DB-IP Lite"; exit 1; }
days=$(( ($(date +%s) - $(stat -c %Y "$db")) / 86400 ))
[ "$days" -le 40 ] || { echo "::error::$db was installed $days days ago"; exit 1; }
done
curl -fsS "$PUBLIC_URL/stargazing" | grep -o 'DB-IP Lite [0-9-]*' | head -1 | sed 's/^/::notice::locating with /'
+64 -14
View File
@@ -136,18 +136,27 @@ curl-able. Outbound fetches only go to https DNS names resolving to public addre
`Federation__AllowPrivateNetworks=true`, `Federation__AllowPlainHttp=true` and `Federation__AcceptAnyCertificate=true`, `Federation__AllowPrivateNetworks=true`, `Federation__AllowPlainHttp=true` and `Federation__AcceptAnyCertificate=true`,
which startup refuses in Production. which startup refuses in Production.
Promoting an admin on the box (signing up as "admin" grants nothing): The admin CLI runs with every service built but nothing started (no Kestrel, no hosted services), so it can run next to
the live service. Sign-up is closed in production (invitations only), so the first login is made here; promoting is how
an admin is made (signing up as "admin" grants nothing):
```bash ```bash
cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin promote <root> cd /var/www/privapub.thepra.dev
echo '<password>' | ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin create-root <login> --admin # password on stdin
ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin promote|demote <root>
ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin smoke <persona> # the deploy's: prints "deploy-smoke <new password>"
``` ```
The deploy runs them as `build-runner`, which owns the published files, reads `appsettings.Production.json` through
group www-data and reaches the private mongod; `sudo -u www-data` works too.
## Federation invariants ## Federation invariants
1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only 1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only
to public addresses; redirects are followed by hand (three at most, each re-checked); bodies are capped at 1 MB; to public addresses; redirects are followed by hand (three at most, each re-checked); bodies are capped at 1 MB;
only JSON media types are read; a refused URL is not asked again for five minutes. Never create another only JSON media types are read; a refused URL is not asked again for five minutes. Never create another
`HttpClient` for federation. `HttpClient` for federation. The only other outbound traffic is SMTP and `GeoUpdater`'s monthly DB-IP Lite download
(its own `geo` client, a fixed HTTPS host, size-capped, the file checked before it is swapped in).
2. **Every fetch is signed by the instance actor** (`privapub`), never by a persona; deliveries are signed by the acting 2. **Every fetch is signed by the instance actor** (`privapub`), never by a persona; deliveries are signed by the acting
avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies). avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies).
3. **A remote document is believed only from its own address.** `RemoteActorService.FetchObject` requires the 3. **A remote document is believed only from its own address.** `RemoteActorService.FetchObject` requires the
@@ -173,8 +182,17 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
`ContentFormat` says what `Text` holds. Remote names are plain text. `ContentFormat` says what `Text` holds. Remote names are plain text.
8. **Circles federate to members only.** A circle is an undiscoverable Group actor that takes follow requests (the owner 8. **Circles federate to members only.** A circle is an undiscoverable Group actor that takes follow requests (the owner
approves); its posts are addressed to the circle and its `/flock`, delivered to members' personal inboxes, never approves); its posts are addressed to the circle and its `/flock`, delivered to members' personal inboxes, never
announced, and served only to a signed request from a member or a member's instance actor announced. Each member's copy also names that member in `cc` (`OutboxPublisher.Naming`, owner decision
(`SignedFetchAuthorizer`), 404 otherwise. Circles never appear in search, lookups, mentions or profile pages. 2026-10-04), because Mastodon and GoToSocial keep a post only when it names one of their accounts; Create, every
Update and the Delete all go through `OutboxPublisher.Publish` for that. A reply to a circle post stays in the circle,
and a circle post never asks a non-member for a quote. Circles never appear in search, lookups, mentions or profile
pages.
**A post that is not public is served only to a signed request from someone it was for** (`SignedFetchAuthorizer.MayRead`):
a follower or an addressed account for followers-only, an addressed account for a DM, a member for a circle, or the
instance actor of a server where one of them lives; 404 to anyone else, 410 to them once it is deleted. A circle
refetch names the requesting member, or the members on the requesting server. A DM's `context`
(`/peasants/{name}/whispers/{id}`) lists the conversation's posts for its participants. Mastodon deletes its copy
when a refetch answers 404, which is why this matters.
**Communities** are FEP-1b12 groups: `GroupDistributor` announces the whole activity (plus the object for new posts, **Communities** are FEP-1b12 groups: `GroupDistributor` announces the whole activity (plus the object for new posts,
for Mastodon), top-level posts are `Page`s with a `name`, posting follows `Group.PostingPolicy`. for Mastodon), top-level posts are `Page`s with a `name`, posting follows `Group.PostingPolicy`.
Located posts (`LocalGeo`) are the only local-only posts. Located posts (`LocalGeo`) are the only local-only posts.
@@ -183,13 +201,20 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`): 10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`):
leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`). 202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`).
11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`, and 11. **Every number about posts and users goes through `Domain/Privacy/Counted`** (owner decision 2026-10-04: one
answer everywhere): a persona's `statuses_count`, its outbox `totalItems`, NodeInfo `localPosts` and the instance
`status_count` all count Mastodon's way (not deleted, not a DM, boosts included, circle and located posts too);
users exclude personas of banned or deleted roots; `replies_count` counts only public and unlisted replies; a
remote account that deletes itself takes its likes, votes, reactions, boosts and replies out of every count
(`GoneActors`). Follower and following counts are public, their members never are (`hide_collections` is always
true).
12. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`, and
any other read of stored posts filters by `IsShown`. All three hide deleted posts and the posts of a remote account any other read of stored posts filters by `IsShown`. All three hide deleted posts and the posts of a remote account
that deleted itself (`Post.AuthorGone`: kept, hidden everywhere, owner decision). that deleted itself (`Post.AuthorGone`: kept, hidden everywhere, owner decision).
12. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or 13. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is
fetched as context. Followers-only is detected by the author's stored `followers` URL. fetched as context. Followers-only is detected by the author's stored `followers` URL.
13. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and 14. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and
every delete removes its `TimelineEntry` rows. Local deletes are soft (content cleared, 410 Tombstone). every delete removes its `TimelineEntry` rows. Local deletes are soft (content cleared, 410 Tombstone).
## Mastodon client API invariants ## Mastodon client API invariants
@@ -205,7 +230,9 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
error that leaves without a body (a challenge, a 404 from routing, a 429) gets Mastodon's `{"error": ...}` from error that leaves without a body (a challenge, a 404 from routing, a 429) gets Mastodon's `{"error": ...}` from
`UseMastodonErrorBodies`. `NeverFiveHundredTests` walks every route with junk ids, anonymously, as a persona and `UseMastodonErrorBodies`. `NeverFiveHundredTests` walks every route with junk ids, anonymously, as a persona and
with a junk token. with a junk token.
6. Advertise `4.2.0 (compatible; PrivaPub)` until grouped notifications exist. 6. Advertise `4.2.0 (compatible; PrivaPub)` until streaming and Web Push exist too; `api_versions.mastodon = 7` stays,
because clients show quotes from it, so everything a 4.3+ client calls because of it must answer: grouped
notifications (`/api/v2/notifications`, its unread count, groups, accounts and dismiss) and the notification policy.
7. **What a Mastodon `Status` cannot say goes in `Status.privapub`** (`PrivaPubStatus`): object type, title, excerpt, 7. **What a Mastodon `Status` cannot say goes in `Status.privapub`** (`PrivaPubStatus`): object type, title, excerpt,
cover, the author's source, link, video, audio and event details, and up/down votes. Every media URL in it goes cover, the author's source, link, video, audio and event details, and up/down votes. Every media URL in it goes
through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following
@@ -234,6 +261,16 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
- **No root id in federation output, NodeInfo or logs, no IP next to an identity in logs, and no `ex.Message` to a - **No root id in federation output, NodeInfo or logs, no IP next to an identity in logs, and no `ex.Message` to a
client** ("Something went wrong." instead). client** ("Something went wrong." instead).
- **Sign-in and recovery never tell which logins or addresses exist** (owner decision 2026-10-04): every failed sign-in
gets "That username and password do not match." after the same hashing (`RootUsersService.Decoy`, a constant-time
comparison), and only a correct password learns of a ban. Every recovery request answers the same sentence and
queues a `SendRecovery` job, found or not; `RecoveryJob` sends the email and keeps only a SHA-256 of the code, for an
hour. A recovered password ends every session of the root (`RootSessions`: a new `SessionStamp`, which every `/clientapi`
JWT carries, and OpenIddict revocation for each persona).
- **Deleting a root deletes everything public it had** (`RootRemoval`, from the admin route or `/clientapi/user/delete`
with the password): its sessions end, each persona and each group it owns sends `Delete{Actor}` to followers, members
and the accounts it follows, the personas' posts are emptied, and `/peasants/{name}`, its inbox and WebFinger answer
410 (`LocalActorService.Gone`) while the names stay reserved. The root keeps only a `deleted-{id}` name.
- **Never derive a public name from the root username.** Invitation sign-up takes `AvatarUserName` and refuses one - **Never derive a public name from the root username.** Invitation sign-up takes `AvatarUserName` and refuses one
equal to the login. equal to the login.
- **One username space:** personas, groups and the instance reserve their name in `ReservedName` (unique index) before - **One username space:** personas, groups and the instance reserve their name in `ReservedName` (unique index) before
@@ -403,14 +440,17 @@ tools/pasture/run.sh down # removes e
delete is checked as a 404 on `/api/v1/statuses/{id}`. delete is checked as a 404 on `/api/v1/statuses/{id}`.
- It creates its accounts locked, so the scenario approves alice's request through `/api/v1/follow_requests`, which - It creates its accounts locked, so the scenario approves alice's request through `/api/v1/follow_requests`, which
also checks our pending (`requested`) state and the manual Accept. also checks our pending (`requested`) state and the manual Accept.
- 37 checks: discovery and follows both ways; posts and CW; a reply and its notification; likes and boosts both - 55 checks: discovery and follows both ways; posts and CW; a reply and its notification; likes and boosts both
ways; DMs both ways and off public timelines; polls both ways; quote policy; link cards; edits and deletes both ways; DMs both ways and off public timelines; polls both ways; quote policy; link cards; edits and deletes both
ways; unfollow; block and unblock; statistics. ways; communities and circles; locked personas; unfollow; block and unblock; statistics.
- It files a circle post as a direct message and shows it only to the accounts it mentions, so like a DM it is
checked in the member's `/api/v1/conversations`, never by URI.
- **Mastodon (4.7.3):** web and sidekiq on the shared Postgres and Redis, `ALLOWED_PRIVATE_ADDRESSES` for the network. - **Mastodon (4.7.3):** web and sidekiq on the shared Postgres and Redis, `ALLOWED_PRIVATE_ADDRESSES` for the network.
Its token comes from `rails runner` (no password grant). Without Elasticsearch its status search finds nothing, so Its token comes from `rails runner` (no password grant). Without Elasticsearch its status search finds nothing, so
deliveries are checked through `/api/v1/accounts/:id/statuses` of the sender as Mastodon knows them, or deliveries are checked through `/api/v1/accounts/:id/statuses` of the sender as Mastodon knows them, or
`Status.exists?` through `rails runner`. Its actors are numbered (`/ap/users/<id>`), so look URIs up rather than `Status.exists?` through `rails runner`. Its actors are numbered (`/ap/users/<id>`), so look URIs up rather than
build them. 49 checks; circle posts are an expected failure (see `docs/INTEROP.md`, Mastodon). build them. Circle posts and a followers-only post survive its signed refetch (`ActivityPub::FetchRemoteStatusService`
through `rails runner`). Inbound Block is the one expected failure until P7.
- **Misskey (2026.10.0):** one container on the shared Postgres and Redis. A new Misskey federates with nobody - **Misskey (2026.10.0):** one container on the shared Postgres and Redis. A new Misskey federates with nobody
(`federation: none`) until `admin/update-meta` says `all`, which `misskey_up` does. Its API is `POST /api/<endpoint>` (`federation: none`) until `admin/update-meta` says `all`, which `misskey_up` does. Its API is `POST /api/<endpoint>`
with the token as `i` (`mk` in the scenario); `users/relation` answers a list, `users/notes` leaves replies out unless with the token as `i` (`mk` in the scenario); `users/relation` answers a list, `users/notes` leaves replies out unless
@@ -432,6 +472,9 @@ tools/pasture/run.sh down # removes e
its own activity and every bare `Announce{object}` 400 by design, and the echo is still needed (see its own activity and every bare `Announce{object}` 400 by design, and the echo is still needed (see
`docs/INTEROP.md`, Lemmy). A new Lemmy never sends what it queued for a server before its send worker for that `docs/INTEROP.md`, Lemmy). A new Lemmy never sends what it queued for a server before its send worker for that
server started, so the scenario waits for that worker (`lm_worker`) before its first follow. 20 checks; relayed votes and a moderator's removal are expected failures (P7). server started, so the scenario waits for that worker (`lm_worker`) before its first follow. 20 checks; relayed votes and a moderator's removal are expected failures (P7).
- **SecureMode:** `PRIVAPUB_ENV="Federation__SecureMode=true" run.sh up …`, as production runs. A check of what an
unsigned reader sees uses `unserved` and `gone_unsigned` (`lib/interop.sh`), which expect 401 when SecureMode is on and
404 or 410 when it is off.
- **Crawler:** `PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test" - **Crawler:** `PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test"
run.sh up mastodon`, then `interop.sh crawler`. `PRIVAPUB_ENV` passes any setting to the PrivaPub container. run.sh up mastodon`, then `interop.sh crawler`. `PRIVAPUB_ENV` passes any setting to the PrivaPub container.
- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into - `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into
@@ -446,10 +489,17 @@ the owner's GoToSocial account.**
`deploy.yml`: tests, self-contained linux-x64 publish, snapshot and `mongodump` to `/var/backups/privapub.thepra.dev`, `deploy.yml`: tests, self-contained linux-x64 publish, snapshot and `mongodump` to `/var/backups/privapub.thepra.dev`,
stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo, stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo,
Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration, client credentials, Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration, client credentials,
discovery, instance, public timeline; pass a persona token as a second argument to check the signed-in side). discovery, instance, public timeline). Then it checks that what production should be running is running:
- NodeInfo and the instance API agree on registrations (closed, invitations only);
- `/stargazing` says the crawler is on;
- **@thepra signs in**: `PrivaPub admin smoke thepra` gives the root `deploy-smoke` a new password, `tools/smoke/oauth.sh`
runs the real OAuth flow (the pasture's `privapub_token` uses the same script), the signed-in API is checked, the
persona must be undiscoverable, and the token is revoked;
- the DB-IP Lite databases, which the server fetches itself (`GeoUpdater`), exist and are at most 40 days old.
- **The box:** Max (`nuvola.xyz`). Unit `privapub` runs as www-data from `/var/www/privapub.thepra.dev` with - **The box:** Max (`nuvola.xyz`). Unit `privapub` runs as www-data from `/var/www/privapub.thepra.dev` with
`ASPNETCORE_ENVIRONMENT=Production`. `ASPNETCORE_ENVIRONMENT=Production`.
- **One-time root setup:** `deploy/max/setup.sh`, run through `../arasaka.software/tools/max/run.sh`. - **One-time root setup:** `deploy/max/setup.sh`, run through `../arasaka.software/tools/max/run.sh` (directories, the
runner's sudoers line, the units, nginx and the certificate). Nothing that runs later needs it again.
- **Config:** `appsettings.Production.json` is committed and deployed, **secrets included, by the owner's convention** - **Config:** `appsettings.Production.json` is committed and deployed, **secrets included, by the owner's convention**
(the same as arasaka.software). A hand edit on the box is lost at the next deploy. (the same as arasaka.software). A hand edit on the box is lost at the next deploy.
- **Secrets drift:** `AppConfigurationService` copies `AppConfiguration` into Mongo on first boot and reads the stored - **Secrets drift:** `AppConfigurationService` copies `AppConfiguration` into Mongo on first boot and reads the stored
+70 -22
View File
@@ -12,17 +12,29 @@ PrivaPub is an ActivityPub server written in C#. This document follows
## Tested against ## Tested against
- **GoToSocial 0.22.1, end to end.** It runs in a private network on the workstation (`tools/pasture/`) and is driven End to end, in a private network on the workstation (`tools/pasture/`), each peer driven through its own client API
through its own client API. Checked both ways: and every run starting clean, with signed fetches required (as privapub.thepra.dev runs):
- follows, including to a locked account; - **GoToSocial 0.22.1**
- public posts with a content warning; - **Mastodon 4.7.3**
- replies with notifications; - **Misskey 2026.10**
- likes and boosts; - **Sharkey 2025.4.7**
- direct messages; - **Akkoma 3.20.1**
- edits and deletes; - **Lemmy 1.0.0-beta.2**
- unfollow.
- **Mastodon, Misskey, Lemmy and PeerTube, by unit tests only.** The tests feed the parser documents written in each Checked both ways, where the peer has the feature:
server's shape. No live exchange with any of them has run yet. - follows, locked accounts included;
- every visibility, content warnings, replies and their notifications;
- likes, boosts and emoji reactions with their undos;
- direct messages;
- polls;
- quotes;
- images with alt text;
- edits with history and deletes;
- communities and circles;
- blocks and unfollows.
`docs/INTEROP.md` has each peer's evidence and what is still expected to fail. PeerTube, PieFed, Mbin and the others are
covered by unit tests written in their documents' shape.
## Supported FEPs ## Supported FEPs
@@ -30,11 +42,16 @@ PrivaPub is an ActivityPub server written in C#. This document follows
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md) - [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
- [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md) - [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md)
- [FEP-1b12: Group federation](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md) (communities; see "Groups") - [FEP-1b12: Group federation](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md) (communities; see "Groups")
- [FEP-044f: Consent-respecting quote posts](https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md)
(`QuoteAuthorization` at `/parrot-licences/{id}`; checked with Mastodon both ways)
- [FEP-9967: Polls](https://codeberg.org/fediverse/fep/src/branch/main/fep/9967/fep-9967.md)
- [FEP-c0e0: Emoji reactions](https://codeberg.org/fediverse/fep/src/branch/main/fep/c0e0/fep-c0e0.md) (`EmojiReact`, and Misskey's
`Like` with content)
- [FEP-5feb: Search indexing consent](https://codeberg.org/fediverse/fep/src/branch/main/fep/5feb/fep-5feb.md) (`indexable`)
Planned (see `docs/ROADMAP.md`, phases P5 to P8, and the per-platform notes in `docs/INTEROP.md`): FEP-044f (quotes), Planned (see `docs/ROADMAP.md`, phases P7 and P8, and the per-platform notes in `docs/INTEROP.md`): FEP-9098 (custom
FEP-9967 (polls), FEP-c0e0 (emoji reactions), FEP-9098 (custom emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), FEP-8fcf (followers synchronisation), FEP-8967 (link
FEP-8fcf (followers synchronisation), FEP-5feb (`indexable`), FEP-8967 (link attachments), FEP-521a and FEP-8b32 (keys attachments), FEP-521a and FEP-8b32 (keys and integrity proofs), FEP-ae0c (relays).
and integrity proofs), FEP-ae0c (relays).
## Actors ## Actors
@@ -58,7 +75,8 @@ The names are the project's own and are stable; resolve actors through WebFinger
- The key is `{actor}#main-key`, RSA 2048, served as SPKI PEM with `owner` set to the actor. - The key is `{actor}#main-key`, RSA 2048, served as SPKI PEM with `owner` set to the actor.
- `published` on an actor is a whole day, chosen at random up to two weeks before the account was made, so two - `published` on an actor is a whole day, chosen at random up to two weeks before the account was made, so two
personas made on the same day do not share a date. `indexable` is `false`. personas made on the same day do not share a date. `indexable` is `false` unless the persona turns it on, and
`discoverable` is `true` unless it turns that off (the deploy's own persona, @thepra, is undiscoverable).
- The instance actor is `/peasants/privapub` (type `Application`). It signs every fetch PrivaPub makes, so no avatar's key - The instance actor is `/peasants/privapub` (type `Application`). It signs every fetch PrivaPub makes, so no avatar's key
is used to read another server's content. is used to read another server's content.
@@ -73,8 +91,10 @@ A group is either a **community** or a **circle**.
top-level post is a `Page` with a `name`. Members are counted at `/flock`; moderators are listed at `/wardens`, which top-level post is a `Page` with a `name`. Members are counted at `/flock`; moderators are listed at `/wardens`, which
the actor's `attributedTo` points to, with `postingRestrictedToMods` as Lemmy expects. A mention of a community posts into it. the actor's `attributedTo` points to, with `postingRestrictedToMods` as Lemmy expects. A mention of a community posts into it.
- A **circle** is private. Its actor is not discoverable and every follow is a request. Its posts are addressed to the - A **circle** is private. Its actor is not discoverable and every follow is a request. Its posts are addressed to the
circle and its members collection, delivered to each member's own inbox and never announced. They are served only circle and its members collection, delivered to each member's own inbox and never announced. Each member's copy also
to a signed request from a member, or from the instance actor of a member's server; anyone else gets 404. A names that member in `cc`, so servers that keep only posts naming one of their accounts (Mastodon, GoToSocial) keep
it; it names no other member. The posts are served only to a signed request from a member, or from the instance actor
of a member's server, and that copy names the member (or the members on that server); anyone else gets 404. A
Mastodon member's replies reach only the people they mention. Mastodon member's replies reach only the people they mention.
- Announces from **remote** groups (Lemmy communities) are followed through to the activity: the object is fetched - Announces from **remote** groups (Lemmy communities) are followed through to the activity: the object is fetched
from its own origin, never taken from the announce. from its own origin, never taken from the announce.
@@ -162,6 +182,22 @@ The page's OpenGraph and Twitter tags give the title, description and image; the
days and shared by every account on the server. Images are served to clients only through PrivaPub's media proxy. days and shared by every account on the server. Images are served to clients only through PrivaPub's media proxy.
`Federation:FetchLinkPreviews=false` turns page fetching off. `Federation:FetchLinkPreviews=false` turns page fetching off.
## Server descriptions and the crawler
PrivaPub keeps statistics about servers, never about accounts (see `/stargazing` on the server).
- **Describing a server:** a server it exchanges activities with is described at most once a week, from:
- `/.well-known/nodeinfo` and the NodeInfo document it links;
- `/api/v2/instance`, falling back to `/api/v1/instance`.
These requests are unsigned, because they are not ActivityPub documents.
- **The crawler** is on at privapub.thepra.dev. It identifies as
`PrivaPub-Stargazer/<version> (+https://privapub.thepra.dev/stargazing)`.
- **What it reads:** `/robots.txt`, then the documents above and `/api/v1/instance/peers`, and nothing else: no
accounts, posts or directories.
- **How often:** one server a minute, each at most weekly.
- **Opting out:** in robots.txt, the crawler obeys the group `PrivaPub-Stargazer`, then `PrivaPub`, then `*`. A
robots.txt that answers with a server error or times out also keeps it out, as do this server's domain blocks.
## Local-only posts ## Local-only posts
Posts with a location (shown to nearby users of this server) never leave the server, in any form. Posts with a location (shown to nearby users of this server) never leave the server, in any form.
@@ -176,12 +212,22 @@ Posts with a location (shown to nearby users of this server) never leave the ser
deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin. deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin.
- **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three - **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three
redirects and read at most 1 MB. redirects and read at most 1 MB.
- **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like
Mastodon's authorized fetch; the instance actor `/peasants/privapub` is the exception, since its key is needed first.
A browser asking for HTML is redirected to the public page instead.
- **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were
for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted
they answer those same readers 410. A direct message's `context`, `/peasants/{name}/whispers/{id}`, is an
`OrderedCollection` of the conversation's posts for its participants.
- **HTML.** Received HTML is sanitised to Mastodon's allowlist. - **HTML.** Received HTML is sanitised to Mastodon's allowlist.
- **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered - **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered
5xx, the inbox answers 503 with `Retry-After: 300` rather than 401. 5xx, the inbox answers 503 with `Retry-After: 300` rather than 401.
- **Activity ids.** `Create` and `Announce` ids dereference. `Follow`, `Like`, `Block` and the `Accept`, `Reject` and - **Activity ids.** `Create` and `Announce` ids dereference, a community's announces included, until the post they
`Undo` that answer them do not, because serving them would reveal who follows, likes and blocks whom; they are always carry is deleted (then 410); a persona's boost id sends a browser to the boosted post. `Follow`, `Like`, `Block` and
sent with their object embedded. the `Accept`, `Reject` and `Undo` that answer them do not, because serving them would reveal who follows, likes and
blocks whom. Neither do `Update`, `Delete`, `EmojiReact`, `QuoteRequest` and its answers, `Flag`, `Ignore` or poll
votes. All of them are always sent with their object embedded.
- **Hashtags.** A post's `Hashtag` links go to `/tags/{tag}`, a public page of this server's public posts with that tag.
- **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused, - **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
starting at an hour and growing to a week. starting at an hour and growing to a week.
@@ -190,5 +236,7 @@ Posts with a location (shown to nearby users of this server) never leave the ser
- Our own posts carry no custom emoji: emoji are received and shown, not offered. - Our own posts carry no custom emoji: emoji are received and shown, not offered.
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed - Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer. through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
- Collections expose counts, not members. - Collections expose counts, not members: `/groupies` and `/stalking` give the same totals as the Mastodon API's
follower and following counts, and the outbox's `totalItems` is the persona's post count, though only public posts
are listed. `/api/v1/instance/peers` is empty: which servers this one talks to is not published.
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned. - Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
@@ -0,0 +1,14 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.User
{
// Deleting one's own login, and with it every persona and group it owns, asks for the password again.
public class RemoveSelfForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
Display(Name = "Password", ResourceType = typeof(FieldsNameResource))]
public string Password { get; set; }
}
}
@@ -60,6 +60,7 @@ namespace PrivaPub.Tests.Domain
{ {
public string BaseAddress => Base; public string BaseAddress => Base;
public Task<GoneActor> Gone(string userName, CancellationToken token) => Task.FromResult<GoneActor>(default);
public Task<LocalActor> FindByUserName(string userName, CancellationToken token) => public Task<LocalActor> FindByUserName(string userName, CancellationToken token) =>
Task.FromResult(userName.Equals("alice", StringComparison.OrdinalIgnoreCase) Task.FromResult(userName.Equals("alice", StringComparison.OrdinalIgnoreCase)
? new LocalActor { Id = "a1", UserName = "alice", BaseAddress = Base, Kind = LocalActorKind.Person } ? new LocalActor { Id = "a1", UserName = "alice", BaseAddress = Base, Kind = LocalActorKind.Person }
+67 -1
View File
@@ -9,6 +9,7 @@ using PrivaPub.Models.Federation;
using PrivaPub.Models.Group; using PrivaPub.Models.Group;
using PrivaPub.Models.Post; using PrivaPub.Models.Post;
using PrivaPub.Models.Social; using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support; using PrivaPub.Tests.Support;
using System.Text.Json.Nodes; using System.Text.Json.Nodes;
@@ -138,21 +139,86 @@ namespace PrivaPub.Tests.Federation
Assert.Equal(PostVisibility.Circle, outcome.Post.Visibility); Assert.Equal(PostVisibility.Circle, outcome.Post.Visibility);
var create = Assert.Single(await _harness.Outgoing(member.Id + "/inbox")); var create = Assert.Single(await _harness.Outgoing(member.Id + "/inbox"));
Assert.Equal(new[] { circle.Uri, circle.Flock }, create["object"]!["to"]!.AsArray().Select(t => t!.GetValue<string>())); Assert.Equal(new[] { circle.Uri, circle.Flock }, create["object"]!["to"]!.AsArray().Select(t => t!.GetValue<string>()));
// the member's copy names the member, so Mastodon and GoToSocial keep it, and names nobody else
Assert.Equal(new[] { member.Id }, create["cc"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.Equal(new[] { member.Id }, create["object"]!["cc"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.Contains(create["object"]!["tag"]!.AsArray(), t => t!["type"]!.GetValue<string>() == "Mention" && t["href"]!.GetValue<string>() == member.Id);
Assert.DoesNotContain(Addressing.Public, create.ToJsonString()); Assert.DoesNotContain(Addressing.Public, create.ToJsonString());
Assert.Empty(await _harness.Outgoing(follower.SharedInbox)); Assert.Empty(await _harness.Outgoing(follower.SharedInbox));
Assert.Empty((await _harness.Outgoing(member.SharedInbox)).Where(a => a["type"]!.GetValue<string>() == "Announce")); Assert.Empty((await _harness.Outgoing(member.SharedInbox)).Where(a => a["type"]!.GetValue<string>() == "Announce"));
var authorizer = new SignedFetchAuthorizer(_harness.Remote); var authorizer = new SignedFetchAuthorizer(_harness.Remote, new DbEntities());
var path = new Uri(outcome.Post.ObjectURI).AbsolutePath; var path = new Uri(outcome.Post.ObjectURI).AbsolutePath;
var asMember = await authorizer.Requester(member.Get(Harness.Host, path), token); var asMember = await authorizer.Requester(member.Get(Harness.Host, path), token);
var asOutsider = await authorizer.Requester(outsider.Get(Harness.Host, path), token); var asOutsider = await authorizer.Requester(outsider.Get(Harness.Host, path), token);
Assert.True(SignedFetchAuthorizer.MayReadCircle(circleEntity, asMember)); Assert.True(SignedFetchAuthorizer.MayReadCircle(circleEntity, asMember));
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, asOutsider)); Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, asOutsider));
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, default)); Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, default));
Assert.True(await authorizer.MayRead(outcome.Post, asMember, token));
Assert.False(await authorizer.MayRead(outcome.Post, asOutsider, token));
Assert.Equal(new[] { member.Id }, SignedFetchAuthorizer.CircleReaders(circleEntity, asMember));
Assert.True(circle.IsCircle); Assert.True(circle.IsCircle);
Assert.False(circle.Discoverable); Assert.False(circle.Discoverable);
} }
[Fact]
public async Task A_circle_posts_edit_and_delete_reach_each_member_naming_only_that_member()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var first = new RemoteActor(_harness.Peer, "first");
var second = new RemoteActor(_harness.Peer, "second", _harness.Peer.B);
var (_, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id, first.Id, second.Id);
await _harness.Remote.GetActor(first.Id, refresh: false, token);
await _harness.Remote.GetActor(second.Id, refresh: false, token);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
await _harness.Statuses.Edit(alice, outcome.Post.ID, new StatusDraft { Text = "just us, edited" }, token);
await _harness.Statuses.Remove(alice, outcome.Post.ID, token);
foreach (var (member, other) in new[] { (first, second), (second, first) })
{
var sent = await _harness.Outgoing(member.Id + "/inbox");
Assert.Equal(new[] { "Create", "Update", "Delete" }, sent.Select(a => a["type"]!.GetValue<string>()));
Assert.All(sent, a => Assert.Contains(member.Id, a["cc"]!.AsArray().Select(c => c!.GetValue<string>())));
Assert.All(sent, a => Assert.DoesNotContain(other.Id, a.ToJsonString()));
}
}
[Fact]
public async Task A_reply_to_a_circle_post_stays_in_it_and_a_circle_post_asks_nobody_outside_for_a_quote()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var (entity, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id);
entity.Members.Add(new GroupMember { AvatarId = bob.Id });
await DB.Default.SaveAsync(entity, token);
var outsider = new RemoteActor(_harness.Peer, "asked");
var outsiderAccount = await _harness.Remote.GetActor(outsider.Id, refresh: false, token);
var asksFirst = new Post
{
ObjectURI = $"{Origin(outsider)}/notes/{Guid.NewGuid():N}",
ActorURI = outsider.Id,
GroupUserId = outsiderAccount.ID,
AuthorAccountId = outsiderAccount.ID,
IsFederatedCopy = true,
Visibility = PostVisibility.Public,
ContentHtml = "<p>ask me first</p>",
QuotePolicy = new InteractionRule { Manual = new() { Addressing.Public } }
};
await DB.Default.SaveAsync(asksFirst, token);
var root = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
var reply = await _harness.Statuses.Publish(bob, new StatusDraft { Text = "still just us", InReplyTo = root.Post.ID }, token);
var quote = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "look", GroupId = circle.Id, QuotedStatusId = asksFirst.ID }, token);
Assert.Equal(PostVisibility.Circle, reply.Post.Visibility);
Assert.Equal(circle.Id, reply.Post.GroupId);
Assert.Equal(422, quote.Status);
Assert.Empty(await _harness.Outgoing(outsider.Id + "/inbox"));
}
[Fact] [Fact]
public async Task Only_circle_members_can_post_into_a_circle() public async Task Only_circle_members_can_post_into_a_circle()
{ {
+70 -23
View File
@@ -6,6 +6,8 @@ using MongoDB.Entities;
using PrivaPub.ClientModels; using PrivaPub.ClientModels;
using PrivaPub.Models.Group; using PrivaPub.Models.Group;
using PrivaPub.Models.User; using PrivaPub.Models.User;
using PrivaPub.Services;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support; using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host; using PrivaPub.Tests.Support.Host;
@@ -136,9 +138,11 @@ namespace PrivaPub.Tests.Http
Assert.Equal(HttpStatusCode.OK, ok.StatusCode); Assert.Equal(HttpStatusCode.OK, ok.StatusCode);
Assert.Equal(root.Id, (await ok.JsonBody())["userId"]!.GetValue<string>()); Assert.Equal(root.Id, (await ok.JsonBody())["userId"]!.GetValue<string>());
// a wrong password and an unknown login get the same answer, so sign-in tells nobody which logins exist
Assert.Equal(HttpStatusCode.BadRequest, wrong.StatusCode); Assert.Equal(HttpStatusCode.BadRequest, wrong.StatusCode);
Assert.Equal("Wrong password.", await Message(wrong)); Assert.Equal(RootUsersService.NoMatch, await Message(wrong));
Assert.Equal(HttpStatusCode.BadRequest, unknown.StatusCode); Assert.Equal(HttpStatusCode.BadRequest, unknown.StatusCode);
Assert.Equal(await Message(wrong), await Message(unknown));
Assert.Equal(HttpStatusCode.OK, logout.StatusCode); Assert.Equal(HttpStatusCode.OK, logout.StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, anonymousLogout.StatusCode); Assert.Equal(HttpStatusCode.Unauthorized, anonymousLogout.StatusCode);
} }
@@ -290,42 +294,59 @@ namespace PrivaPub.Tests.Http
} }
[Fact] [Fact]
public async Task Recovery_without_an_email_is_refused() public async Task Recovery_answers_the_same_whoever_asks_and_queues_the_work()
{ {
var root = await _host.SignUp("noemail"); var token = TestContext.Current.CancellationToken;
var since = DateTime.UtcNow.AddSeconds(-1);
var withEmail = await _host.SignUp("withemail");
var withoutEmail = await _host.SignUp("noemail");
var email = $"{Guid.NewGuid():N}@example.test";
using (var signedIn = _host.As(withEmail.Jwt))
Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode);
using var client = _host.Client(); using var client = _host.Client();
var response = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName }); var answers = new[]
var unknown = await client.PostJson("/clientapi/user/recover/password", new { userName = Name("nobody") }); {
var neither = await client.PostJson("/clientapi/user/recover/password", new { }); await client.PostJson("/clientapi/user/recover/password", new { userName = withEmail.UserName }),
await client.PostJson("/clientapi/user/recover/password", new { email }),
await client.PostJson("/clientapi/user/recover/password", new { userName = withoutEmail.UserName }),
await client.PostJson("/clientapi/user/recover/password", new { userName = Name("nobody") }),
await client.PostJson("/clientapi/user/recover/password", new { email = $"{Guid.NewGuid():N}@example.test" })
};
var bodies = new List<string>();
foreach (var answer in answers)
{
Assert.Equal(HttpStatusCode.OK, answer.StatusCode);
bodies.Add(await answer.Content.ReadAsStringAsync(token));
}
Assert.Equal(HttpStatusCode.Locked, response.StatusCode); Assert.Single(bodies.Distinct());
Assert.Contains("doesn't have an email", await Message(response)); Assert.Contains("recovery link is on its way", bodies[0]);
Assert.False(await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken)); var queued = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.SendRecovery && j.CreatedAt >= since).ExecuteAsync(token);
Assert.Equal(HttpStatusCode.NotFound, unknown.StatusCode); Assert.True(queued.Count >= answers.Length);
Assert.Equal(HttpStatusCode.BadRequest, neither.StatusCode); Assert.Contains(queued, j => j.Payload.Contains(withEmail.Id));
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostJson("/clientapi/user/recover/password", new { })).StatusCode);
} }
[Fact] [Fact]
public async Task Recovery_through_an_unreachable_mail_server_says_so_and_nothing_more() public async Task The_recovery_job_keeps_only_a_hash_and_an_unreachable_mail_server_says_nothing_to_anyone()
{ {
var token = TestContext.Current.CancellationToken;
var root = await _host.SignUp("smtp"); var root = await _host.SignUp("smtp");
var email = $"{Guid.NewGuid():N}@example.test"; var email = $"{Guid.NewGuid():N}@example.test";
using (var signedIn = _host.As(root.Jwt)) using (var signedIn = _host.As(root.Jwt))
Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode); Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode);
using var client = _host.Client(); using var client = _host.Client();
var byName = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName }); var answer = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName });
var byEmail = await client.PostJson("/clientapi/user/recover/password", new { email }); await _host.Run(j => j.Kind == JobKind.SendRecovery && j.Payload.Contains(root.Id), token);
foreach (var response in new[] { byName, byEmail }) Assert.Equal(HttpStatusCode.OK, answer.StatusCode);
{ var recovery = await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteFirstAsync(token);
var body = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken); Assert.NotNull(recovery);
Assert.Equal(HttpStatusCode.ServiceUnavailable, response.StatusCode); Assert.Null(recovery.RecoveryCode);
Assert.Equal("Failed to send email.", JsonNode.Parse(body)!["errorMessage"]!.GetValue<string>()); Assert.Equal(64, recovery.CodeHash.Length);
foreach (var leak in new[] { "xception", "efused", "ocket", "127.0.0.1", "smtp", "SMTP", " at " }) Assert.InRange(recovery.ExpiresAt, DateTime.UtcNow.AddMinutes(50), DateTime.UtcNow.AddMinutes(61));
Assert.DoesNotContain(leak, body);
}
} }
[Fact] [Fact]
@@ -348,7 +369,10 @@ namespace PrivaPub.Tests.Http
{ {
var root = await _host.SignUp("recover"); var root = await _host.SignUp("recover");
var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N"); var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N");
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, RecoveryCode = code }, TestContext.Current.CancellationToken); await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, CodeHash = RecoveryJob.Hash(code), ExpiresAt = DateTime.UtcNow.AddHours(1) },
TestContext.Current.CancellationToken);
var persona = await _host.Persona(root, "recovered");
var apiToken = await _host.MastodonToken(persona);
using var client = _host.Client(); using var client = _host.Client();
var valid = await client.PostAsync("/clientapi/user/recover/valid", new StringContent($"\"{code}\"", Encoding.UTF8, "application/json"), TestContext.Current.CancellationToken); var valid = await client.PostAsync("/clientapi/user/recover/valid", new StringContent($"\"{code}\"", Encoding.UTF8, "application/json"), TestContext.Current.CancellationToken);
@@ -361,6 +385,29 @@ namespace PrivaPub.Tests.Http
Assert.Equal(HttpStatusCode.BadRequest, (await LogIn(root.UserName, root.Password)).StatusCode); Assert.Equal(HttpStatusCode.BadRequest, (await LogIn(root.UserName, root.Password)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, NewPassword)).StatusCode); Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, NewPassword)).StatusCode);
Assert.False(await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken)); Assert.False(await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken));
// whoever held the old sessions may be why the password was recovered: they end
using (var oldSession = _host.As(root.Jwt))
Assert.Equal(HttpStatusCode.Unauthorized, (await oldSession.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken)).StatusCode);
using (var oldApp = _host.Client())
{
oldApp.DefaultRequestHeaders.Authorization = new("Bearer", apiToken);
Assert.Equal(HttpStatusCode.Unauthorized, (await oldApp.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode);
}
}
[Fact]
public async Task An_expired_recovery_code_changes_nothing()
{
var root = await _host.SignUp("expired");
var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N");
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, CodeHash = RecoveryJob.Hash(code), ExpiresAt = DateTime.UtcNow.AddMinutes(-1) },
TestContext.Current.CancellationToken);
using var client = _host.Client();
var change = await client.PostJson("/clientapi/user/recover/update/password", new { newPassword = NewPassword, repeatedPassword = NewPassword, recoveryCode = code });
Assert.Equal(HttpStatusCode.NotFound, change.StatusCode);
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, root.Password)).StatusCode);
} }
[Fact] [Fact]
@@ -154,6 +154,34 @@ namespace PrivaPub.Tests.Http
Assert.DoesNotContain(persona.Root.UserName, update.ToJsonString()); Assert.DoesNotContain(persona.Root.UserName, update.ToJsonString());
} }
[Fact]
public async Task A_theme_change_here_keeps_what_the_mastodon_api_set()
{
var persona = await _host.Persona(await _host.SignUp("keep"), "keep");
await DB.Default.Update<Avatar>().MatchID(persona.Id)
.Modify(a => a.Settings.IsDiscoverable, false)
.Modify(a => a.Settings.IsLocked, true)
.Modify(a => a.Settings.QuotePolicy, QuotePolicies.Nobody)
.ExecuteAsync(TestContext.Current.CancellationToken);
using var client = _host.As(persona.Root.Jwt);
var response = await client.PostJson("/clientapi/avatar/private/update", new
{
avatarId = persona.Id,
name = "Kept",
biography = "same settings",
settings = new { isDefault = false, darkThemeIndexColour = 100, themeIsDarkMode = true }
});
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var settings = (await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Settings;
Assert.True(settings.ThemeIsDarkMode);
Assert.Equal(100, settings.DarkThemeIndexColour);
Assert.False(settings.IsDiscoverable);
Assert.True(settings.IsLocked);
Assert.Equal(QuotePolicies.Nobody, settings.QuotePolicy);
}
[Fact] [Fact]
public async Task Updating_another_roots_persona_is_refused() public async Task Updating_another_roots_persona_is_refused()
{ {
+64 -3
View File
@@ -142,7 +142,9 @@ namespace PrivaPub.Tests.Http
Assert.Equal(ActivityJson, outbox.MediaType); Assert.Equal(ActivityJson, outbox.MediaType);
Assert.Equal("OrderedCollection", outbox.Json["type"]!.GetValue<string>()); Assert.Equal("OrderedCollection", outbox.Json["type"]!.GetValue<string>());
Assert.Equal(persona.ActorUri() + "/anus", outbox.Json["id"]!.GetValue<string>()); Assert.Equal(persona.ActorUri() + "/anus", outbox.Json["id"]!.GetValue<string>());
Assert.Equal(22, outbox.Json["totalItems"]!.GetValue<int>()); // statuses_count, as Mastodon counts it: everything but the DM, the deleted post and the copy of a remote post,
// the followers-only and located posts and the boost included; only the public posts are listed
Assert.Equal(24, outbox.Json["totalItems"]!.GetValue<int>());
Assert.Null(outbox.Json["orderedItems"]); Assert.Null(outbox.Json["orderedItems"]);
var first = outbox.Json["first"]!.GetValue<string>(); var first = outbox.Json["first"]!.GetValue<string>();
Assert.Equal(persona.ActorUri() + "/anus?page=true", first); Assert.Equal(persona.ActorUri() + "/anus?page=true", first);
@@ -221,7 +223,7 @@ namespace PrivaPub.Tests.Http
Assert.DoesNotContain(pending, followers.Text); Assert.DoesNotContain(pending, followers.Text);
Assert.Equal(HttpStatusCode.OK, following.Status); Assert.Equal(HttpStatusCode.OK, following.Status);
Assert.Equal(persona.ActorUri() + "/stalking", following.Json["id"]!.GetValue<string>()); Assert.Equal(persona.ActorUri() + "/stalking", following.Json["id"]!.GetValue<string>());
Assert.Equal(0, following.Json["totalItems"]!.GetValue<int>()); Assert.Equal(1, following.Json["totalItems"]!.GetValue<int>());//counted, as following_count says, never listed
Assert.DoesNotContain(friend.UserName, following.Text); Assert.DoesNotContain(friend.UserName, following.Text);
} }
@@ -322,11 +324,66 @@ namespace PrivaPub.Tests.Http
Assert.Equal(new[] { circle.Uri, circle.Uri + "/flock" }, Strings(asMember.Json["to"])); Assert.Equal(new[] { circle.Uri, circle.Uri + "/flock" }, Strings(asMember.Json["to"]));
Assert.DoesNotContain(ActivityPubRenderer.Public, asMember.Text); Assert.DoesNotContain(ActivityPubRenderer.Public, asMember.Text);
Assert.Equal(HttpStatusCode.OK, asMemberServer.Status); Assert.Equal(HttpStatusCode.OK, asMemberServer.Status);
// a refetch names the member, as the member's copy did; the instance actor sees the members on its server only
Assert.Equal(new[] { member.Id }, Strings(asMember.Json["cc"]));
Assert.Equal(new[] { member.Id }, Strings(asMemberServer.Json["cc"]));
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path)).Status); Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path)).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path, Browser)).Status); Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path, Browser)).Status);
foreach (var outsider in new[] { neighbour, stranger, strangerServer }) foreach (var outsider in new[] { neighbour, stranger, strangerServer })
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(outsider.SignedGet(path))).Status); Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(outsider.SignedGet(path))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch($"/peasants/{owner.UserName}/grunts/create-{post.ID}")).Status); var create = $"/peasants/{owner.UserName}/grunts/create-{post.ID}";
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(create)).Status);
var createAsMember = await _client.Fetch(member.SignedGet(create));
Assert.Equal(HttpStatusCode.OK, createAsMember.Status);
Assert.Equal(post.ObjectURI, createAsMember.Json["object"]!["id"]!.GetValue<string>());
Assert.Equal(new[] { member.Id }, Strings(createAsMember.Json["object"]!["cc"]));
}
[Fact]
public async Task Followers_only_and_direct_posts_are_served_to_signed_fetches_from_those_they_were_for()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "private");
var follower = new RemoteActor(_peer, "follower");
var followerServer = new RemoteActor(_peer, "instance", type: "Application");
var recipient = new RemoteActor(_peer, "recipient");
var stranger = new RemoteActor(_peer, "stranger", _peer.B);
_peer.WebFinger(recipient);
await DB.Default.SaveAsync(new Follower
{
LocalActorId = owner.Id,
LocalActorKind = LocalActorKind.Person,
ActorURI = follower.Id,
InboxURL = follower.Id + "/inbox"
}, token);
var quiet = await _host.Publish(owner, "for followers", PostVisibility.FollowersOnly);
var direct = await _host.Publish(owner, new StatusDraft { Text = $"@{recipient.Handle()} just you", PlainText = true, Visibility = PostVisibility.Direct });
var quietPath = $"/peasants/{owner.UserName}/scribbles/{quiet.ID}";
var directPath = $"/peasants/{owner.UserName}/scribbles/{direct.ID}";
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(follower.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(followerServer.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status);
var asRecipient = await _client.Fetch(recipient.SignedGet(directPath));
Assert.Equal(HttpStatusCode.OK, asRecipient.Status);
Assert.Contains(recipient.Id, Strings(asRecipient.Json["to"]));
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(follower.SignedGet(directPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(directPath)).Status);
// the DM's context is the conversation, for its participants only
var context = PathOf(asRecipient.Json["context"]!.GetValue<string>());
var conversation = await _client.Fetch(recipient.SignedGet(context));
Assert.Equal(HttpStatusCode.OK, conversation.Status);
Assert.Equal(new[] { direct.ObjectURI }, Strings(conversation.Json["orderedItems"]));
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(context))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(context)).Status);
// once deleted, those it was for learn it is gone; everyone else still learns nothing
await _host.Remove(owner, quiet);
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(follower.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status);
} }
[Fact] [Fact]
@@ -463,6 +520,10 @@ namespace PrivaPub.Tests.Http
Assert.Equal("Application", instance.Json["type"]!.GetValue<string>()); Assert.Equal("Application", instance.Json["type"]!.GetValue<string>());
foreach (var path in paths[..^1]) foreach (var path in paths[..^1])
Assert.True((await client.Fetch(reader.SignedGet(path))).Status == HttpStatusCode.OK, $"{path} refused a signed GET"); Assert.True((await client.Fetch(reader.SignedGet(path))).Status == HttpStatusCode.OK, $"{path} refused a signed GET");
// a browser is not asked for a signature: it is only sent to the public pages
var browser = await client.Fetch($"/peasants/{persona.UserName}", Browser);
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
Assert.Equal(HttpStatusCode.Redirect, (await client.Fetch($"/peasants/{persona.UserName}/scribbles/{post.ID}", Browser)).Status);
} }
} }
} }
+7 -3
View File
@@ -28,7 +28,8 @@ namespace PrivaPub.Tests.Http
Assert.Equal("4.2.0 (compatible; PrivaPub)", v1.Body.Text("version")); Assert.Equal("4.2.0 (compatible; PrivaPub)", v1.Body.Text("version"));
Assert.Equal(InstanceController.Version, v1.Body.Text("version")); Assert.Equal(InstanceController.Version, v1.Body.Text("version"));
Assert.Equal(PrivaPubHost.Host, v1.Body.Text("uri")); Assert.Equal(PrivaPubHost.Host, v1.Body.Text("uri"));
Assert.False(v1.Body.Flag("registrations")); Assert.True(v1.Body.Flag("registrations"));
Assert.True(v1.Body.Flag("invites_enabled"));
Assert.True(v1.Body["stats"].Number("user_count") >= 0); Assert.True(v1.Body["stats"].Number("user_count") >= 0);
Assert.Equal(5000, v1.Body["configuration"]!["statuses"].Number("max_characters")); Assert.Equal(5000, v1.Body["configuration"]!["statuses"].Number("max_characters"));
Assert.Equal(4, v1.Body["configuration"]!["polls"].Number("max_options")); Assert.Equal(4, v1.Body["configuration"]!["polls"].Number("max_options"));
@@ -40,8 +41,11 @@ namespace PrivaPub.Tests.Http
var v2 = (await anonymous.Get("/api/v2/instance")).Ok(); var v2 = (await anonymous.Get("/api/v2/instance")).Ok();
Assert.Equal(InstanceController.Version, v2.Body.Text("version")); Assert.Equal(InstanceController.Version, v2.Body.Text("version"));
Assert.Equal(PrivaPubHost.Host, v2.Body.Text("domain")); Assert.Equal(PrivaPubHost.Host, v2.Body.Text("domain"));
Assert.Equal($"wss://{PrivaPubHost.Host}", v2.Body["configuration"]!["urls"].Text("streaming")); Assert.Null(v2.Body["configuration"]!["urls"]!["streaming"]);//nothing streams yet, so nothing is advertised
Assert.False(v2.Body["registrations"].Flag("enabled")); Assert.Equal(PrivaPub.Api.Mastodon.Controllers.StatusesController.MaxPins, v2.Body["configuration"]!["accounts"].Number("max_pinned_statuses"));
Assert.Contains("image/avif", v2.Body["configuration"]!["media_attachments"]!["supported_mime_types"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.True(v2.Body["registrations"].Flag("enabled"));
Assert.True((await anonymous.Get("/nodeinfo/2.1")).Ok().Body.Flag("openRegistrations"));
Assert.False(v2.Body["configuration"]!["translation"].Flag("enabled")); Assert.False(v2.Body["configuration"]!["translation"].Flag("enabled"));
Assert.Equal(7, v2.Body["api_versions"].Number("mastodon")); Assert.Equal(7, v2.Body["api_versions"].Number("mastodon"));
+263
View File
@@ -0,0 +1,263 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.Host.FederationHelpers;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Http
{
// Two views of the same fact give one answer (owner decision 2026-10-04: fix the mismatches).
[Trait("Category", "Integration")]
public sealed class OneAnswerTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
[Fact]
public async Task Anyone_may_search_and_only_a_signed_in_reader_resolves_or_pages()
{
var reader = await _host.Mastodon("searcher");
Assert.Equal(HttpStatusCode.OK, (await _client.Get("/api/v2/search?q=somebody")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Get("/api/v2/search?q=somebody&resolve=true")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Get("/api/v2/search?q=somebody&offset=5")).Status);
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get("/api/v2/search?q=somebody&offset=5")).Status);
}
[Fact]
public async Task A_remote_account_that_deletes_itself_leaves_nothing_in_the_counts()
{
var token = TestContext.Current.CancellationToken;
var author = await _host.Mastodon("liked");
var post = await author.Status("like me");
var fan = new RemoteActor(_peer, "fickle");
var like = new JsonObject
{
["id"] = $"{fan.Id}#likes/{Guid.NewGuid():N}",
["type"] = "Like",
["actor"] = fan.Id,
["object"] = post.Text("uri")
};
Assert.Equal(HttpStatusCode.Accepted, (await _client.SendAsync(fan.SignedPost("/human-centipede", like), token)).StatusCode);
await _host.RunInbox(like["id"]!.GetValue<string>(), token);
Assert.Equal(1, (await author.Client.Get($"/api/v1/statuses/{post.Text("id")}")).Ok().Body.Number("favourites_count"));
var delete = new JsonObject
{
["id"] = $"{fan.Id}#delete",
["type"] = "Delete",
["actor"] = fan.Id,
["object"] = fan.Id,
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public")
};
Assert.Equal(HttpStatusCode.Accepted, (await _client.SendAsync(fan.SignedPost("/human-centipede", delete), token)).StatusCode);
await _host.RunInbox(delete["id"]!.GetValue<string>(), token);
Assert.Equal(0, (await author.Client.Get($"/api/v1/statuses/{post.Text("id")}")).Ok().Body.Number("favourites_count"));
Assert.Empty((await author.Client.Get($"/api/v1/statuses/{post.Text("id")}/favourited_by")).Ok().Array);
Assert.False(await DB.Default.Find<Favourite>().Match(f => f.ActorURI == fan.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Notification>().Match(n => n.FromActorURI == fan.Id).ExecuteAnyAsync(token));
Assert.Equal(0, (await author.Client.Get("/api/v1/notifications/unread_count")).Ok().Body.Number("count"));
}
[Fact]
public async Task Only_public_and_unlisted_replies_are_counted()
{
var author = await _host.Mastodon("threadstart");
var replier = await _host.Mastodon("replier");
var root = await author.Status("say something");
var id = root.Text("id");
await replier.Status("in public", ("in_reply_to_id", id));
await replier.Status("quietly", ("in_reply_to_id", id), ("visibility", "unlisted"));
await replier.Status("for my followers", ("in_reply_to_id", id), ("visibility", "private"));
await replier.Status($"@{author.UserName} just you", ("in_reply_to_id", id), ("visibility", "direct"));
Assert.Equal(2, (await author.Client.Get($"/api/v1/statuses/{id}")).Ok().Body.Number("replies_count"));
}
[Fact]
public async Task A_status_longer_than_advertised_is_refused()
{
var author = await _host.Mastodon("wordy");
var limit = (await _client.Get("/api/v2/instance")).Ok().Body["configuration"]!["statuses"].Number("max_characters");
var refused = await author.Client.Post("/api/v1/statuses", ("status", new string('a', limit + 1)));
var accepted = await author.Client.Post("/api/v1/statuses", ("status", new string('a', limit)));
Assert.Equal(HttpStatusCode.UnprocessableEntity, refused.Status);
Assert.Equal(HttpStatusCode.OK, accepted.Status);
}
[Fact]
public async Task Joining_a_community_by_invitation_is_following_it()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "commowner");
var joinerRoot = await _host.SignUp("joiner");
var joiner = await _host.Persona(joinerRoot, "joiner");
var group = await _host.Group(owner, community: true);
var groupName = group["userName"]!.GetValue<string>();
using var client = _host.As(joinerRoot.Jwt);
var joined = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode = group["invitationCode"]!.GetValue<string>() });
Assert.Equal(HttpStatusCode.OK, joined.StatusCode);
Assert.Equal(1, (await _client.Fetch($"/peasants/{groupName}/groupies")).Json["totalItems"]!.GetValue<int>());
Assert.True(await DB.Default.Find<Following>().Match(f => f.AvatarId == joiner.Id && f.State == FollowState.Accepted).ExecuteAnyAsync(token));
}
public static TheoryData<string> AnsweredRoutes() => new()
{
"/api/v1/directory",
"/api/v1/tags/cats",
"/api/v1/instance/languages",
"/api/v1/instance/translation_languages",
"/api/v1/instance/domain_blocks",
"/api/v1/instance/privacy_policy",
"/api/v1/timelines/link?url=https%3A%2F%2Fexample.org%2F",
"/api/v1/accounts/000000000000000000000000/identity_proofs"
};
[Theory]
[MemberData(nameof(AnsweredRoutes))]
public async Task Mastodon_routes_we_have_nothing_behind_answer_instead_of_404(string path)
{
Assert.Equal(HttpStatusCode.OK, (await _client.Get(path)).Status);
}
[Fact]
public async Task Likes_of_one_post_come_as_one_group_of_notifications()
{
var author = await _host.Mastodon("grouped");
var first = await _host.Mastodon("fanone");
var second = await _host.Mastodon("fantwo");
var post = await author.Status("like me twice");
var id = post.Text("id");
await first.Client.Post($"/api/v1/statuses/{id}/favourite");
await second.Client.Post($"/api/v1/statuses/{id}/favourite");
var list = (await author.Client.Get("/api/v2/notifications")).Ok().Body;
var group = Assert.Single(list["notification_groups"]!.AsArray(), g => g!.Text("type") == "favourite")!;
var key = group.Text("group_key");
Assert.Equal($"grouped-favourite-{id}", key);
Assert.Equal(2, group.Number("notifications_count"));
Assert.Equal(2, group["sample_account_ids"]!.AsArray().Count);
Assert.Equal(id, group.Text("status_id"));
Assert.Contains(list["statuses"]!.AsArray(), s => s!.Text("id") == id);
Assert.True((await author.Client.Get("/api/v2/notifications/unread_count")).Ok().Body.Number("count") >= 1);
Assert.Equal(key, Assert.Single((await author.Client.Get($"/api/v2/notifications/{key}")).Ok().Body["notification_groups"]!.AsArray())!.Text("group_key"));
Assert.Equal(2, (await author.Client.Get($"/api/v2/notifications/{key}/accounts")).Ok().Array.Count);
Assert.Equal(HttpStatusCode.OK, (await author.Client.Post($"/api/v2/notifications/{key}/dismiss")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await author.Client.Get($"/api/v2/notifications/{key}")).Status);
}
[Fact]
public async Task A_hashtag_link_opens_a_page_of_public_posts_with_that_tag()
{
var author = await _host.Mastodon("tagger");
var tag = $"tag{Guid.NewGuid():N}"[..16];
await author.Status($"about #{tag}");
await author.Status($"quietly about #{tag}", ("visibility", "private"));
var page = await _client.Fetch($"/tags/{tag}", Browser);
Assert.Equal(HttpStatusCode.OK, page.Status);
Assert.Contains("about", page.Text);
Assert.DoesNotContain("quietly", page.Text);
Assert.Contains("default-src 'none'", page.Response.Headers.GetValues("Content-Security-Policy").Single());
}
[Fact]
public async Task A_communitys_announces_resolve_while_the_post_is_shown()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "announcer");
var community = await _host.FederatedGroup(owner, community: true);
var post = await _host.Publish(owner, new PrivaPub.Domain.Statuses.StatusDraft { Text = "into the community", PlainText = true, GroupId = community.Id });
var kept = await DB.Default.Find<GroupAnnouncement>().Match(a => a.GroupId == community.Id && a.ObjectURI == post.ObjectURI).ExecuteAsync(token);
Assert.Equal(2, kept.Count);//the activity, and the object for Mastodon
foreach (var announcement in kept)
{
var fetched = await _client.Fetch(PathOf(announcement.ActivityURI));
Assert.Equal(HttpStatusCode.OK, fetched.Status);
Assert.Equal("Announce", fetched.Json["type"]!.GetValue<string>());
Assert.Equal(announcement.ActivityURI, fetched.Json["id"]!.GetValue<string>());
}
var listed = await _client.Fetch($"/peasants/{community.UserName}/grunts/announce-{post.ID}");
Assert.Equal(HttpStatusCode.OK, listed.Status);
await _host.Remove(owner, post);
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(PathOf(kept[0].ActivityURI))).Status);
}
[Fact]
public async Task A_boost_points_at_the_boosted_posts_page()
{
var author = await _host.Mastodon("boosted");
var booster = await _host.Mastodon("booster");
var post = await author.Status("boost me");
var boost = (await booster.Client.Post($"/api/v1/statuses/{post.Text("id")}/reblog")).Ok().Body;
Assert.Equal(post.Text("url"), boost.Text("url"));
var browser = await _client.Fetch(PathOf(boost.Text("uri")), Browser);
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
}
[Fact]
public async Task A_remote_accounts_counts_are_what_its_server_publishes()
{
var token = TestContext.Current.CancellationToken;
var reader = await _host.Mastodon("counter");
var popular = new RemoteActor(_peer, "popular");
_peer.Serve(new Uri(popular.Id).AbsolutePath + "/followers",
$$"""{"@context":"https://www.w3.org/ns/activitystreams","id":"{{popular.Id}}/followers","type":"OrderedCollection","totalItems":42}""");
var remote = _host.Get<PrivaPub.Federation.Actors.IRemoteActorService>();
var account = await remote.GetActor(popular.Id, refresh: true, token);
await _host.Run(j => j.Kind == PrivaPub.Models.Jobs.JobKind.CountAccount && j.Payload == popular.Id, token);
var shown = (await reader.Client.Get($"/api/v1/accounts/{account.ID}")).Ok().Body;
Assert.Equal(42, shown.Number("followers_count"));
Assert.Equal(0, shown.Number("following_count"));
}
[Fact]
public async Task Notification_policy_and_requests_answer_a_signed_in_reader()
{
var reader = await _host.Mastodon("policy");
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get("/api/v1/notifications/policy")).Status);
Assert.Equal("accept", (await reader.Client.Get("/api/v2/notifications/policy")).Ok().Body.Text("for_not_following"));
Assert.Empty((await reader.Client.Get("/api/v1/notifications/requests")).Ok().Array);
}
}
}
+50
View File
@@ -0,0 +1,50 @@
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using PrivaPub.Infrastructure;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class RegistrationsTests : IAsyncLifetime
{
WebApplicationFactory<Program> _closed;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var shared = await PrivaPubHost.Shared();
_closed = shared.WithWebHostBuilder(builder => builder.UseSetting("Registrations:Mode", nameof(RegistrationMode.Invitations)));
}
public async ValueTask DisposeAsync()
{
if (_closed != default)
await _closed.DisposeAsync();
}
[Fact]
public async Task Invitations_only_refuses_sign_up_and_every_document_says_closed()
{
using var client = _closed.CreateClient(new WebApplicationFactoryClientOptions { BaseAddress = new Uri(PrivaPubHost.Base + "/"), AllowAutoRedirect = false });
client.DefaultRequestHeaders.Add(PrivaPubHost.ClientHeader, $"10.9.{Random.Shared.Next(256)}.{Random.Shared.Next(1, 255)}");
var signUp = await client.PostJson("/clientapi/user/signup", new { userName = $"closed{Guid.NewGuid():N}"[..20], password = "Closed-Door-Pass-1" });
var nodeInfo = (await client.Get("/nodeinfo/2.1")).Ok().Body;
var v1 = (await client.Get("/api/v1/instance")).Ok().Body;
var v2 = (await client.Get("/api/v2/instance")).Ok().Body;
Assert.Equal(HttpStatusCode.Forbidden, signUp.StatusCode);
Assert.Contains("invitation", await signUp.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
Assert.False(nodeInfo.Flag("openRegistrations"));
Assert.False(v1.Flag("registrations"));
Assert.True(v1.Flag("invites_enabled"));
Assert.False(v2["registrations"].Flag("enabled"));
Assert.Equal(RegistrationOptions.ClosedMessage, v2["registrations"].Text("message"));
}
}
}
+118
View File
@@ -0,0 +1,118 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Net.Http.Json;
using GroupEntity = PrivaPub.Models.Group.Group;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Http
{
// Deleting a root deletes its personas and their groups everywhere (owner decision 2026-10-04).
[Trait("Category", "Integration")]
public sealed class RootRemovalTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
async Task<HttpResponseMessage> AdminRemoves(params string[] rootIds)
{
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
using var request = new HttpRequestMessage(HttpMethod.Delete, "/clientapi/admin/remove/users") { Content = JsonContent.Create(new { userIdList = rootIds }) };
return await client.SendAsync(request, TestContext.Current.CancellationToken);
}
[Fact]
public async Task A_removed_roots_personas_and_groups_are_deleted_everywhere_and_say_so()
{
var token = TestContext.Current.CancellationToken;
var root = await _host.SignUp("leaving");
var persona = await _host.Persona(root, "leaving");
var follower = new RemoteActor(_peer, "fan");
var member = new RemoteActor(_peer, "member");
await DB.Default.SaveAsync(new Follower { LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = follower.Id, InboxURL = follower.Id + "/inbox" }, token);
var group = await _host.Group(persona, community: true);
var groupId = group["id"]!.GetValue<string>();
await DB.Default.SaveAsync(new Follower { LocalActorId = groupId, LocalActorKind = LocalActorKind.Group, ActorURI = member.Id, InboxURL = member.Id + "/inbox" }, token);
var post = await _host.Publish(persona, "goodbye soon");
var apiToken = await _host.MastodonToken(persona);
var since = DateTime.UtcNow.AddSeconds(-1);
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(root.Id)).StatusCode);
var toFollower = Assert.Single(await Jobs.Deliveries(follower.Id + "/inbox", since, token), d => d["type"]!.GetValue<string>() == "Delete");
Assert.Equal(persona.ActorUri(), toFollower["object"]!.GetValue<string>());
Assert.Equal(persona.ActorUri(), toFollower["actor"]!.GetValue<string>());
var groupName = group["userName"]!.GetValue<string>();
Assert.Contains(await Jobs.Deliveries(member.Id + "/inbox", since, token),
d => d["type"]!.GetValue<string>() == "Delete" && d["object"]!.GetValue<string>() == $"{PrivaPubHost.Base}/peasants/{groupName}");
var gone = await _client.Fetch($"/peasants/{persona.UserName}");
Assert.Equal(HttpStatusCode.Gone, gone.Status);
Assert.Equal("Person", gone.Json["formerType"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch($"/peasants/{groupName}")).Status);
Assert.Equal(HttpStatusCode.Gone, (await _client.GetAsync($"/.well-known/webfinger?resource=acct:{persona.UserName}@{PrivaPubHost.Host}", token)).StatusCode);
var stored = await DB.Default.Find<PostEntity>().MatchID(post.ID).ExecuteFirstAsync(token);
Assert.NotNull(stored.DeletedAt);
Assert.Null(stored.ContentHtml);
Assert.NotNull((await DB.Default.Find<GroupEntity>().MatchID(groupId).ExecuteFirstAsync(token)).DeletionAt);
var removed = await DB.Default.Find<RootUser>().MatchID(root.Id).ExecuteFirstAsync(token);
Assert.Equal($"deleted-{root.Id}", removed.UserName);
Assert.Null(removed.HashedPassword);
using var app = _host.Client();
app.DefaultRequestHeaders.Authorization = new("Bearer", apiToken);
Assert.Equal(HttpStatusCode.Unauthorized, (await app.GetAsync("/api/v1/accounts/verify_credentials", token)).StatusCode);
}
[Fact]
public async Task A_second_removal_does_not_collide_with_the_first()
{
var first = await _host.SignUp("first");
var second = await _host.SignUp("second");
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(first.Id)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(second.Id)).StatusCode);
Assert.NotNull((await DB.Default.Find<RootUser>().MatchID(second.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).DeletedAt);
}
[Fact]
public async Task A_root_deletes_itself_only_with_its_password()
{
var token = TestContext.Current.CancellationToken;
var root = await _host.SignUp("self");
var persona = await _host.Persona(root, "self");
using var client = _host.As(root.Jwt);
var refused = await client.PostJson("/clientapi/user/delete", new { password = "Not-The-Password-1" });
Assert.Equal(HttpStatusCode.Forbidden, refused.StatusCode);
Assert.Null((await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(token)).DeletionAt);
var accepted = await client.PostJson("/clientapi/user/delete", new { password = root.Password });
Assert.Equal(HttpStatusCode.OK, accepted.StatusCode);
Assert.NotNull((await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(token)).DeletionAt);
Assert.Equal(HttpStatusCode.Unauthorized, (await client.GetAsync("/clientapi/user/sniff/again", token)).StatusCode);
}
}
}
+5 -3
View File
@@ -179,8 +179,10 @@ namespace PrivaPub.Tests.Http
return (await client.Fetch("/nodeinfo/2.1", "application/json")).Json["usage"]!["localPosts"]!.GetValue<long>(); return (await client.Fetch("/nodeinfo/2.1", "application/json")).Json["usage"]!["localPosts"]!.GetValue<long>();
} }
// localPosts is Mastodon's: the sum of statuses_count, every local post but DMs and deleted ones, boosts included,
// and circle and located posts too (owner decision 2026-10-04)
[Fact] [Fact]
public async Task Local_posts_count_only_what_anyone_may_see() public async Task Local_posts_count_as_mastodon_counts_statuses()
{ {
var persona = await _host.Persona(await _host.SignUp(), "usage"); var persona = await _host.Persona(await _host.SignUp(), "usage");
var friend = await _host.Persona(await _host.SignUp(), "usagefriend"); var friend = await _host.Persona(await _host.SignUp(), "usagefriend");
@@ -197,10 +199,10 @@ namespace PrivaPub.Tests.Http
await _host.Reblog(friend, shown); await _host.Reblog(friend, shown);
var circle = await _host.FederatedGroup(persona, community: false); var circle = await _host.FederatedGroup(persona, community: false);
await _host.Publish(persona, new StatusDraft { Text = "circle only", PlainText = true, GroupId = circle.Id }); await _host.Publish(persona, new StatusDraft { Text = "circle only", PlainText = true, GroupId = circle.Id });
Assert.Equal(start + 2, await LocalPosts()); Assert.Equal(start + 6, await LocalPosts());//followers-only, located, the boost and the circle post; not the DM nor the copy
await _host.Remove(persona, shown); await _host.Remove(persona, shown);
Assert.Equal(start + 1, await LocalPosts()); Assert.Equal(start + 4, await LocalPosts());//the post and the boost of it
} }
} }
} }
@@ -1,3 +1,5 @@
using Microsoft.Extensions.DependencyInjection;
using MongoDB.Entities; using MongoDB.Entities;
using PrivaPub.ClientModels; using PrivaPub.ClientModels;
@@ -37,8 +39,8 @@ namespace PrivaPub.Tests.Infrastructure
{ {
var root = await _host.SignUp("promote"); var root = await _host.SignUp("promote");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName.ToUpperInvariant() })); Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName.ToUpperInvariant() }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(new[] { Policies.IsAdmin, Policies.IsModerator, Policies.IsUser }.Order(), await StoredPolicies(root)); Assert.Equal(new[] { Policies.IsAdmin, Policies.IsModerator, Policies.IsUser }.Order(), await StoredPolicies(root));
} }
@@ -47,10 +49,10 @@ namespace PrivaPub.Tests.Infrastructure
public async Task Demote_leaves_a_plain_user() public async Task Demote_leaves_a_plain_user()
{ {
var root = await _host.SignUp("demote"); var root = await _host.SignUp("demote");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(new[] { Policies.IsUser }, await StoredPolicies(root)); Assert.Equal(new[] { Policies.IsUser }, await StoredPolicies(root));
} }
@@ -58,8 +60,8 @@ namespace PrivaPub.Tests.Infrastructure
[Fact] [Fact]
public async Task An_unknown_root_exits_1() public async Task An_unknown_root_exits_1()
{ {
Assert.Equal(1, await AdminCommands.Run(new[] { "promote", $"nobody{Guid.NewGuid():N}" })); Assert.Equal(1, await AdminCommands.Run(new[] { "promote", $"nobody{Guid.NewGuid():N}" }, _host.Services));
Assert.Equal(1, await AdminCommands.Run(new[] { "demote", $"nobody{Guid.NewGuid():N}" })); Assert.Equal(1, await AdminCommands.Run(new[] { "demote", $"nobody{Guid.NewGuid():N}" }, _host.Services));
} }
public static TheoryData<string[]> Misuses() => new() public static TheoryData<string[]> Misuses() => new()
@@ -75,7 +77,7 @@ namespace PrivaPub.Tests.Infrastructure
[MemberData(nameof(Misuses))] [MemberData(nameof(Misuses))]
public async Task Misuse_exits_2(string[] args) public async Task Misuse_exits_2(string[] args)
{ {
Assert.Equal(2, await AdminCommands.Run(args)); Assert.Equal(2, await AdminCommands.Run(args, _host.Services));
} }
[Fact] [Fact]
@@ -84,7 +86,7 @@ namespace PrivaPub.Tests.Infrastructure
var root = await _host.SignUp("policies"); var root = await _host.SignUp("policies");
Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root)); Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(HttpStatusCode.OK, await AdminRoute(root)); Assert.Equal(HttpStatusCode.OK, await AdminRoute(root));
using (var client = _host.As(root.Jwt)) using (var client = _host.As(root.Jwt))
{ {
@@ -92,8 +94,54 @@ namespace PrivaPub.Tests.Infrastructure
Assert.Contains(Policies.IsAdmin, refreshed["policies"]!.AsArray().Select(p => p!.GetValue<string>())); Assert.Contains(Policies.IsAdmin, refreshed["policies"]!.AsArray().Select(p => p!.GetValue<string>()));
} }
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root)); Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root));
} }
async Task<HttpStatusCode> LogIn(string login, string password)
{
using var client = _host.Client();
return (await client.PostJson("/clientapi/user/login", new { userName = login, password })).StatusCode;
}
[Fact]
public async Task Create_root_makes_a_login_from_a_password_on_standard_input()
{
using var scope = _host.Services.CreateScope();
var login = $"made{Guid.NewGuid():N}"[..20];
var output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(new[] { "create-root", login, "--admin" }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-1\n"), output));
Assert.Equal(1, await AdminCommands.Run(new[] { "create-root", login }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-2\n"), new StringWriter()));
Assert.Equal(1, await AdminCommands.Run(new[] { "create-root", $"weak{Guid.NewGuid():N}"[..20] }, scope.ServiceProvider, new StringReader("short\n"), new StringWriter()));
Assert.Equal(2, await AdminCommands.Run(new[] { "create-root", login, "--root" }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-3\n"), new StringWriter()));
Assert.Equal(HttpStatusCode.OK, await LogIn(login, "Typed-On-Stdin-1"));
var stored = await DB.Default.Find<RootUser>().Match(u => u.UserName == login).ExecuteFirstAsync(TestContext.Current.CancellationToken);
Assert.Contains(Policies.IsAdmin, stored.Policies);
}
[Fact]
public async Task Smoke_keeps_one_undiscoverable_persona_and_a_new_password_each_run()
{
using var scope = _host.Services.CreateScope();
var persona = $"smoke{Guid.NewGuid():N}"[..20];
var first = new StringWriter();
var second = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(new[] { "smoke", persona }, scope.ServiceProvider, output: first));
Assert.Equal(0, await AdminCommands.Run(new[] { "smoke", persona }, scope.ServiceProvider, output: second));
var (login, oldPassword) = (first.ToString().Trim().Split(' ')[0], first.ToString().Trim().Split(' ')[1]);
var newPassword = second.ToString().Trim().Split(' ')[1];
Assert.Equal(AdminCommands.SmokeLogin, login);
Assert.NotEqual(oldPassword, newPassword);
Assert.Equal(HttpStatusCode.OK, await LogIn(login, newPassword));
Assert.NotEqual(HttpStatusCode.OK, await LogIn(login, oldPassword));
var avatars = await DB.Default.Find<Avatar>().Match(a => a.UserName == persona).ExecuteAsync(TestContext.Current.CancellationToken);
var avatar = Assert.Single(avatars);
Assert.False(avatar.Settings.IsDiscoverable);
Assert.False(avatar.Settings.IsIndexable);
}
} }
} }
@@ -0,0 +1,125 @@
using MaxMind.Db;
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Tests.Support;
using System.IO.Compression;
namespace PrivaPub.Tests.Infrastructure
{
public sealed class GeoUpdaterTests : IAsyncLifetime
{
static readonly DateTime Now = new(2026, 10, 4, 12, 0, 0, DateTimeKind.Utc);
static readonly DateTime LastMonth = new(2026, 9, 2, 0, 0, 0, DateTimeKind.Utc);
readonly string _directory = Path.Combine(Path.GetTempPath(), $"privapub-geo-{Guid.NewGuid():N}");
Peer _peer;
GeoUpdater _updater;
DbIpLocator _locator;
public async ValueTask InitializeAsync()
{
_peer = await Peer.Start();
var options = new StaticOptions<StatisticsOptions>(new StatisticsOptions
{
GeoDirectory = _directory,
Geo = new GeoOptions { DownloadBase = $"{_peer.A}/free" }
});
_locator = new DbIpLocator(options, NullLogger<DbIpLocator>.Instance);
_updater = new GeoUpdater(new Clients(), options, _locator, NullLogger<GeoUpdater>.Instance);
}
public async ValueTask DisposeAsync()
{
_locator?.Dispose();
await _peer.DisposeAsync();
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
sealed class Clients : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new();
}
static byte[] Gzip(byte[] bytes)
{
using var output = new MemoryStream();
using (var gzip = new GZipStream(output, CompressionLevel.Fastest))
gzip.Write(bytes);
return output.ToArray();
}
void Publish(string kind, DateTime month, byte[] database) =>
_peer.ServeFile($"/free/dbip-{kind}-lite-{month:yyyy-MM}.mmdb.gz", Gzip(database), "application/gzip");
static byte[] City(DateTime built) => MiniMmdb.Build("DBIP-City-Lite", built);
static byte[] Asn(DateTime built) => MiniMmdb.Build("DBIP-ASN-Lite (compat=GeoLite2-ASN)", built);
DateTime Built(string kind)
{
using var reader = new Reader(Path.Combine(_directory, GeoUpdater.FileOf(kind)));
return reader.Metadata.BuildDate;
}
[Fact]
public async Task Missing_databases_are_fetched_checked_and_read_at_once()
{
Publish("city", Now, City(Now));
Publish("asn", Now, Asn(Now));
Assert.Equal(2, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(Now.Date, Built("city").Date);
Assert.Equal(Now.Date, Built("asn").Date);
Assert.Equal("DB-IP Lite 2026-10", _locator.Source);
Assert.Empty(Directory.GetFiles(_directory).Where(f => f.EndsWith(".part") || f.EndsWith(".new")));
}
[Fact]
public async Task Early_in_the_month_last_months_database_is_used_and_not_fetched_again()
{
Publish("city", LastMonth, City(LastMonth));
Publish("asn", LastMonth, Asn(LastMonth));
Assert.Equal(2, await _updater.Update(Now, TestContext.Current.CancellationToken));
var asked = _peer.Requests.Count;
Assert.Equal(0, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(LastMonth.Date, Built("city").Date);
Assert.DoesNotContain(_peer.Requests.Skip(asked), r => r.Path.Contains("2026-09"));
}
[Fact]
public async Task A_current_database_asks_nothing()
{
Publish("city", Now, City(Now));
Publish("asn", Now, Asn(Now));
await _updater.Update(Now, TestContext.Current.CancellationToken);
var asked = _peer.Requests.Count;
Assert.Equal(0, await _updater.Update(Now.AddDays(1), TestContext.Current.CancellationToken));
Assert.Equal(asked, _peer.Requests.Count);
}
[Fact]
public async Task A_broken_or_wrong_database_is_refused_and_the_old_one_kept()
{
Publish("city", LastMonth, City(LastMonth));
Publish("asn", LastMonth, Asn(LastMonth));
await _updater.Update(LastMonth.AddDays(1), TestContext.Current.CancellationToken);
Publish("city", Now, "not a database at all"u8.ToArray());
Publish("asn", Now, City(Now));
Assert.Equal(0, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(LastMonth.Date, Built("city").Date);
Assert.Equal(LastMonth.Date, Built("asn").Date);
Assert.Empty(Directory.GetFiles(_directory).Where(f => f.EndsWith(".part") || f.EndsWith(".new")));
}
}
}
+1 -1
View File
@@ -39,7 +39,7 @@ namespace PrivaPub.Tests.Support.Host
public static async Task<Root> Admin(this PrivaPubHost host) public static async Task<Root> Admin(this PrivaPubHost host)
{ {
var root = await host.SignUp("admin"); var root = await host.SignUp("admin");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName })); Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, host.Services));
return await host.LogIn(root); return await host.LogIn(root);
} }
@@ -73,6 +73,8 @@ namespace PrivaPub.Tests.Support.Host
["Federation:AllowPrivateNetworks"] = "true", ["Federation:AllowPrivateNetworks"] = "true",
["Federation:AllowPlainHttp"] = "true", ["Federation:AllowPlainHttp"] = "true",
["Federation:FetchLinkPreviews"] = "false", ["Federation:FetchLinkPreviews"] = "false",
["Registrations:Mode"] = "Open",
["Statistics:Geo:AutoUpdate"] = "false",
["Media:Root"] = _mediaRoot, ["Media:Root"] = _mediaRoot,
["Logging:LogLevel:Default"] = "Warning", ["Logging:LogLevel:Default"] = "Warning",
["Serilog:MinimumLevel:Default"] = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_LOGS") == "1" ? "Information" : "Fatal" ["Serilog:MinimumLevel:Default"] = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_LOGS") == "1" ? "Information" : "Fatal"
+79
View File
@@ -0,0 +1,79 @@
using System.Text;
namespace PrivaPub.Tests.Support
{
// The smallest MaxMind DB the reader accepts (https://maxmind.github.io/MaxMind-DB/): an IPv4 search tree of one node
// whose two records both mean "no data", an empty data section, and the metadata map, which is all a test of the updater
// needs to tell a city database from an ASN one and to read its build date.
public static class MiniMmdb
{
public static byte[] Build(string databaseType, DateTime built)
{
using var output = new MemoryStream();
const uint nodeCount = 1;
for (var record = 0; record < 2; record++)
output.Write(new byte[] { 0, 0, (byte)nodeCount });//24-bit records equal to node_count: not found
output.Write(new byte[16]);//data section separator; the data section itself is empty
output.Write(new byte[] { 0xAB, 0xCD, 0xEF });
output.Write(Encoding.ASCII.GetBytes("MaxMind.com"));
Map(output,
("binary_format_major_version", w => UInt16(w, 2)),
("binary_format_minor_version", w => UInt16(w, 0)),
("build_epoch", w => UInt64(w, (ulong)new DateTimeOffset(built).ToUnixTimeSeconds())),
("database_type", w => Text(w, databaseType)),
("description", w => Map(w, ("en", x => Text(x, "a test database")))),
("ip_version", w => UInt16(w, 4)),
("languages", w => Array(w, x => Text(x, "en"))),
("node_count", w => UInt32(w, nodeCount)),
("record_size", w => UInt16(w, 24)));
return output.ToArray();
}
// type in the top three bits (0 and an extra byte for the extended types), size below 29 in the low five, or 29 and the
// rest in one more byte (names longer than 28 characters need it)
static void Control(Stream output, int type, int size)
{
var low = size < 29 ? size : 29;
output.WriteByte((byte)((type <= 7 ? type : 0) << 5 | low));
if (type > 7)
output.WriteByte((byte)(type - 7));
if (size >= 29)
output.WriteByte((byte)(size - 29));
}
static void Text(Stream output, string value)
{
var bytes = Encoding.UTF8.GetBytes(value);
Control(output, 2, bytes.Length);
output.Write(bytes);
}
static void Unsigned(Stream output, int type, ulong value)
{
var bytes = BitConverter.GetBytes(value).Reverse().SkipWhile(b => b == 0).ToArray();
Control(output, type, bytes.Length);
output.Write(bytes);
}
static void UInt16(Stream output, ushort value) => Unsigned(output, 5, value);
static void UInt32(Stream output, uint value) => Unsigned(output, 6, value);
static void UInt64(Stream output, ulong value) => Unsigned(output, 9, value);
static void Map(Stream output, params (string Key, Action<Stream> Value)[] entries)
{
Control(output, 7, entries.Length);
foreach (var (key, value) in entries)
{
Text(output, key);
value(output);
}
}
static void Array(Stream output, params Action<Stream>[] items)
{
Control(output, 11, items.Length);
foreach (var item in items)
item(output);
}
}
}
@@ -71,8 +71,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
avatar.Settings.IsDiscoverable = discoverable; avatar.Settings.IsDiscoverable = discoverable;
if (Params.Bool("indexable") is { } indexable) if (Params.Bool("indexable") is { } indexable)
avatar.Settings.IsIndexable = indexable; avatar.Settings.IsIndexable = indexable;
if (Params.Bool("hide_collections") is { } hide) // hide_collections stays true: who follows whom is never listed here, only counted (FEDERATION.md)
avatar.Settings.HideCollections = hide;
if (Params.Get("source[privacy]") is { } privacy && privacy is "public" or "unlisted" or "private") if (Params.Get("source[privacy]") is { } privacy && privacy is "public" or "unlisted" or "private")
avatar.Settings.DefaultVisibility = privacy; avatar.Settings.DefaultVisibility = privacy;
if (Params.Bool("source[sensitive]") is { } sensitive) if (Params.Bool("source[sensitive]") is { } sensitive)
@@ -120,7 +119,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
} }
var userName = parts[0]; var userName = parts[0];
var domain = parts[1].ToLowerInvariant(); var domain = parts[1].ToLowerInvariant();
var foreign = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain).ExecuteFirstAsync(token); var foreign = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain && !f.DeletionAt.HasValue).ExecuteFirstAsync(token);
return foreign == default ? NotFoundError() : Json(_mapper.Foreign(foreign)); return foreign == default ? NotFoundError() : Json(_mapper.Foreign(foreign));
} }
@@ -197,8 +196,9 @@ namespace PrivaPub.Api.Mastodon.Controllers
var (local, _) = await Find(id, token); var (local, _) = await Find(id, token);
if (local == default || local.Id != MyId) if (local == default || local.Id != MyId)
return Json(Array.Empty<Account>()); return Json(Array.Empty<Account>());
var followers = await _dbEntities.Followers.Match(f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted) var followers = await Page.From(Params, Limit(40, 80))
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token); .Fetch(_dbEntities.Followers.Match(f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted), f => f.ID, token);
Link($"/api/v1/accounts/{id}/followers", followers.LastOrDefault()?.ID, followers.FirstOrDefault()?.ID);
return Json(await AccountsFor(followers.Select(f => f.ActorURI), token)); return Json(await AccountsFor(followers.Select(f => f.ActorURI), token));
} }
@@ -208,8 +208,9 @@ namespace PrivaPub.Api.Mastodon.Controllers
var (local, _) = await Find(id, token); var (local, _) = await Find(id, token);
if (local == default || local.Id != MyId) if (local == default || local.Id != MyId)
return Json(Array.Empty<Account>()); return Json(Array.Empty<Account>());
var following = await _dbEntities.Followings.Match(f => f.AvatarId == local.Id && f.State == FollowState.Accepted) var following = await Page.From(Params, Limit(40, 80))
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token); .Fetch(_dbEntities.Followings.Match(f => f.AvatarId == local.Id && f.State == FollowState.Accepted), f => f.ID, token);
Link($"/api/v1/accounts/{id}/following", following.LastOrDefault()?.ID, following.FirstOrDefault()?.ID);
var accounts = await _mapper.Accounts(following.Select(f => f.TargetAccountId), token); var accounts = await _mapper.Accounts(following.Select(f => f.TargetAccountId), token);
return Json(following.Select(f => accounts.GetValueOrDefault(f.TargetAccountId)).Where(a => a != default).ToList()); return Json(following.Select(f => accounts.GetValueOrDefault(f.TargetAccountId)).Where(a => a != default).ToList());
} }
@@ -1,11 +1,13 @@
using Microsoft.AspNetCore.Authorization; using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc; using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Options;
using MongoDB.Entities; using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure; using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Privacy; using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure;
using PrivaPub.Models.User; using PrivaPub.Models.User;
using PostEntity = PrivaPub.Models.Post.Post; using PostEntity = PrivaPub.Models.Post.Post;
@@ -18,34 +20,60 @@ namespace PrivaPub.Api.Mastodon.Controllers
const string Description = "A small ActivityPub server where one private login keeps several unlinkable public personas."; const string Description = "A small ActivityPub server where one private login keeps several unlinkable public personas.";
readonly ILocalActorService _localActors; readonly ILocalActorService _localActors;
readonly IOptionsMonitor<RegistrationOptions> _registrations;
public InstanceController(ILocalActorService localActors) public InstanceController(ILocalActorService localActors, IOptionsMonitor<RegistrationOptions> registrations)
{ {
_localActors = localActors; _localActors = localActors;
_registrations = registrations;
}
bool Open => _registrations.CurrentValue.IsOpen;
// the languages this server is set up for (AppConfiguration:SupportedLanguages), "en" when none are named
public static IReadOnlyList<string> LanguageCodes(IServiceProvider services)
{
var configured = services.GetRequiredService<Microsoft.Extensions.Options.IOptionsMonitor<Models.AppConfiguration>>().CurrentValue.SupportedLanguages;
var codes = (configured ?? new List<string>()).Where(c => !string.IsNullOrWhiteSpace(c)).Select(c => c.Split('-')[0].ToLowerInvariant()).Distinct().ToList();
return codes.Count == 0 ? new[] { "en" } : codes;
} }
string Domain => new Uri(_localActors.BaseAddress).Authority; string Domain => new Uri(_localActors.BaseAddress).Authority;
static object Statuses => new { max_characters = 5000, max_media_attachments = 4, characters_reserved_per_url = 23 }; // every limit is what the server enforces, read from where it is enforced
static object Statuses => new { max_characters = PrivaPub.Domain.Statuses.StatusService.MaxCharacters, max_media_attachments = 4, characters_reserved_per_url = 23 };
static object MediaAttachments => new object MediaAttachments
{ {
supported_mime_types = new[] { "image/jpeg", "image/png", "image/gif", "image/webp", "video/mp4", "video/webm", "audio/mpeg", "audio/ogg" }, get
image_size_limit = 16 * 1024 * 1024, {
image_matrix_limit = 16_777_216, var media = HttpContext.RequestServices.GetRequiredService<IOptionsMonitor<PrivaPub.Domain.Media.MediaOptions>>().CurrentValue;
video_size_limit = 99 * 1024 * 1024, return new
video_frame_rate_limit = 60, {
video_matrix_limit = 2_304_000 supported_mime_types = PrivaPub.Domain.Media.MediaService.SupportedTypes,
}; image_size_limit = media.MaxImageBytes,
image_matrix_limit = media.MaxImageSide * media.MaxImageSide,
video_size_limit = media.MaxVideoBytes,
video_frame_rate_limit = 60,
video_matrix_limit = 2_304_000
};
}
}
static object Polls => new { max_options = 4, max_characters_per_option = 50, min_expiration = 300, max_expiration = 2_629_746 }; static object Polls => new
{
max_options = PrivaPub.Domain.Statuses.PollService.MaxOptions,
max_characters_per_option = PrivaPub.Domain.Statuses.PollService.MaxOptionLength,
min_expiration = PrivaPub.Domain.Statuses.PollService.MinExpiration,
max_expiration = PrivaPub.Domain.Statuses.PollService.MaxExpiration
};
[HttpGet("/api/v1/instance"), AllowAnonymous] [HttpGet("/api/v1/instance"), AllowAnonymous]
public async Task<IActionResult> V1(CancellationToken token) public async Task<IActionResult> V1(CancellationToken token)
{ {
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token); var users = await Counted.Users(token);
var statuses = await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token); var statuses = await Counted.LocalPosts(token);
var domains = (await DB.Default.Find<ForeignAvatar, string>().Project(a => a.Domain).ExecuteAsync(token)).Distinct().Count(); var domains = await KnownServers(token);
return Json(new return Json(new
{ {
uri = Domain, uri = Domain,
@@ -54,21 +82,27 @@ namespace PrivaPub.Api.Mastodon.Controllers
description = Description, description = Description,
email = string.Empty, email = string.Empty,
version = Version, version = Version,
urls = new { streaming_api = $"wss://{Domain}" }, urls = new { },//no streaming API yet, so none is advertised
stats = new { user_count = users, status_count = statuses, domain_count = domains }, stats = new { user_count = users, status_count = statuses, domain_count = domains },
thumbnail = $"{_localActors.BaseAddress}/media/missing-header.png", thumbnail = $"{_localActors.BaseAddress}/media/missing-header.png",
languages = new[] { "en" }, languages = LanguageCodes(HttpContext.RequestServices),
registrations = false, registrations = Open,
approval_required = false, approval_required = false,
invites_enabled = false, invites_enabled = true,//a group invitation always creates an account
configuration = new { accounts = new { max_featured_tags = 0 }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls }, configuration = new { accounts = new { max_featured_tags = 0 }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls },
contact_account = default(object), contact_account = default(object),
rules = Array.Empty<object>() rules = Array.Empty<object>()
}); });
} }
// the servers we have exchanged activities with, as the statistics count them. Which ones is not published
// (/instance/peers stays empty, as on a Mastodon with its peers API off): on a small server it would say who its
// personas talk to.
static async Task<long> KnownServers(CancellationToken token) =>
await DB.Default.CountAsync<Models.Jobs.RemoteInstance>(i => i.Seen == "touched", token);
[HttpGet("/api/v2/instance"), AllowAnonymous] [HttpGet("/api/v2/instance"), AllowAnonymous]
public IActionResult V2() => Json(new public async Task<IActionResult> V2(CancellationToken token) => Json(new
{ {
domain = Domain, domain = Domain,
title = "PrivaPub", title = "PrivaPub",
@@ -76,19 +110,19 @@ namespace PrivaPub.Api.Mastodon.Controllers
api_versions = new { mastodon = 7 }, api_versions = new { mastodon = 7 },
source_url = "https://git.thepra.dev/thepra/SocialPub", source_url = "https://git.thepra.dev/thepra/SocialPub",
description = Description, description = Description,
usage = new { users = new { active_month = 0 } }, usage = new { users = new { active_month = await Counted.ActiveUsers(30, token) } },
thumbnail = new { url = $"{_localActors.BaseAddress}/media/missing-header.png" }, thumbnail = new { url = $"{_localActors.BaseAddress}/media/missing-header.png" },
languages = new[] { "en" }, languages = LanguageCodes(HttpContext.RequestServices),
configuration = new configuration = new
{ {
urls = new { streaming = $"wss://{Domain}" }, urls = new { },//no streaming API yet, so none is advertised
accounts = new { max_featured_tags = 0, max_pinned_statuses = 0 }, accounts = new { max_featured_tags = 0, max_pinned_statuses = StatusesController.MaxPins },
statuses = Statuses, statuses = Statuses,
media_attachments = MediaAttachments, media_attachments = MediaAttachments,
polls = Polls, polls = Polls,
translation = new { enabled = false } translation = new { enabled = false }
}, },
registrations = new { enabled = false, approval_required = false, message = default(string) }, registrations = new { enabled = Open, approval_required = false, message = Open ? default : RegistrationOptions.ClosedMessage },
contact = new { email = string.Empty, account = default(object) }, contact = new { email = string.Empty, account = default(object) },
rules = Array.Empty<object>() rules = Array.Empty<object>()
}); });
@@ -35,11 +35,15 @@ namespace PrivaPub.Api.Mastodon.Controllers
_search = search; _search = search;
} }
[HttpGet("/api/v2/search"), Scope("read:search", requiresUser: false)] // Anyone may search, as on Mastodon; only resolving (which fetches from other servers) and paging need a sign-in.
[HttpGet("/api/v2/search"), Scope("read:search", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Search(CancellationToken token) public async Task<IActionResult> Search(CancellationToken token)
{ {
var q = Params.Get("q")?.Trim(); var q = Params.Get("q")?.Trim();
var type = Params.Get("type"); var type = Params.Get("type");
var offset = Params.Int("offset") ?? 0;
if (MyId == default && (Params.Bool("resolve") == true || offset > 0))
return Error(StatusCodes.Status401Unauthorized, "Search queries that resolve uris or use offset require the read:search scope");
var resolve = Params.Bool("resolve") == true && MyId != default; var resolve = Params.Bool("resolve") == true && MyId != default;
var results = new SearchResults(); var results = new SearchResults();
if (string.IsNullOrEmpty(q)) if (string.IsNullOrEmpty(q))
@@ -66,7 +70,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
if (type is null or "accounts") if (type is null or "accounts")
{ {
results.Accounts = await _search.Find(q, resolve, Limit(), token); results.Accounts = await _search.Find(q, resolve, Limit(), Math.Max(0, offset), token);
} }
if (type is null or "hashtags" && q.TrimStart('#') is { Length: > 0 } tag) if (type is null or "hashtags" && q.TrimStart('#') is { Length: > 0 } tag)
results.Hashtags.Add(new { name = tag.ToLowerInvariant(), url = $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag.ToLowerInvariant())}", history = Array.Empty<object>() }); results.Hashtags.Add(new { name = tag.ToLowerInvariant(), url = $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag.ToLowerInvariant())}", history = Array.Empty<object>() });
@@ -133,5 +137,67 @@ namespace PrivaPub.Api.Mastodon.Controllers
[HttpGet("/api/v1/push/subscription"), Scope("push")] [HttpGet("/api/v1/push/subscription"), Scope("push")]
public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found"); public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found");
// Routes Mastodon answers that we have nothing behind: an empty or default answer lets clients degrade, where a
// 404 breaks some of them (Mastodon invariant 5)
[HttpGet("/api/v1/directory"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Directory() => Json(Array.Empty<object>());
[HttpGet("/api/v1/timelines/link"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult LinkTimeline() => Json(Array.Empty<object>());
[HttpGet("/api/v1/accounts/{id}/identity_proofs"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult IdentityProofs(string id) => Json(Array.Empty<object>());
[HttpGet("/api/v1/tags/{name}"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Tag(string name)
{
var tag = name?.TrimStart('#').ToLowerInvariant() ?? string.Empty;
var baseAddress = HttpContext.RequestServices.GetRequiredService<Federation.Actors.ILocalActorService>().BaseAddress;
return Json(new { name = tag, url = $"{baseAddress}/tags/{Uri.EscapeDataString(tag)}", history = Array.Empty<object>(), following = false });
}
[HttpGet("/api/v1/instance/languages"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Languages() => Json(InstanceController.LanguageCodes(HttpContext.RequestServices)
.Select(code => new { code, name = System.Globalization.CultureInfo.GetCultureInfo(code).EnglishName }));
[HttpGet("/api/v1/instance/translation_languages"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult TranslationLanguages() => Json(new { });
[HttpGet("/api/v1/instance/domain_blocks"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult InstanceDomainBlocks() => Json(Array.Empty<object>());
[HttpGet("/api/v1/instance/privacy_policy"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult PrivacyPolicy() => Json(new
{
updated_at = "2026-10-04T00:00:00.000Z",
content = "<p>One private login owns several public personas, and nobody but this server's admin can tell they belong "
+ "together. What a persona posts is shared as its visibility says; a located post never leaves this server. "
+ "Uploads lose their metadata. Statistics name servers, never people: see /stargazing.</p>"
});
[HttpGet("/api/v1/notifications/policy"), Scope("read:notifications")]
public IActionResult NotificationPolicyV1() => Json(new
{
filter_not_following = false,
filter_not_followers = false,
filter_new_accounts = false,
filter_private_mentions = false,
summary = new { pending_requests_count = 0, pending_notifications_count = 0 }
});
[HttpGet("/api/v2/notifications/policy"), Scope("read:notifications")]
public IActionResult NotificationPolicyV2() => Json(new
{
for_not_following = "accept",
for_not_followers = "accept",
for_new_accounts = "accept",
for_private_mentions = "accept",
for_limited_accounts = "accept",
summary = new { pending_requests_count = 0, pending_notifications_count = 0 }
});
[HttpGet("/api/v1/notifications/requests"), Scope("read:notifications")]
public IActionResult NotificationRequests() => Json(Array.Empty<object>());
} }
} }
@@ -273,7 +273,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.LocalQuotePolicy, policy).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.LocalQuotePolicy, policy).ExecuteAsync(token);
post.LocalQuotePolicy = policy; post.LocalQuotePolicy = policy;
if (!post.IsLocalOnly) if (!post.IsLocalOnly)
await _outbox.Publish(Me, post, ActivityPubRenderer.UpdateOf(post, Me, $"policy-{DateTime.UtcNow.Ticks}"), token); await _outbox.PublishUpdate(Me, post, $"policy-{DateTime.UtcNow.Ticks}", token);
return Json(await _mapper.Status(post, MyId, token)); return Json(await _mapper.Status(post, MyId, token));
} }
@@ -177,8 +177,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
_dbEntities = dbEntities; _dbEntities = dbEntities;
} }
[HttpGet("/api/v1/notifications"), Scope("read:notifications")] Find<NotificationEntity, NotificationEntity> Filtered()
public async Task<IActionResult> List(CancellationToken token)
{ {
var types = Params.List("types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList(); var types = Params.List("types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList();
var excluded = Params.List("exclude_types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList(); var excluded = Params.List("exclude_types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList();
@@ -189,11 +188,132 @@ namespace PrivaPub.Api.Mastodon.Controllers
query.Match(n => !excluded.Contains(n.Type)); query.Match(n => !excluded.Contains(n.Type));
if (Params.Get("account_id") is { } accountId) if (Params.Get("account_id") is { } accountId)
query.Match(n => n.FromAccountId == accountId); query.Match(n => n.FromAccountId == accountId);
var notifications = await Page.From(Params, Limit(15, 30)).Fetch(query, n => n.ID, token); return query;
}
[HttpGet("/api/v1/notifications"), Scope("read:notifications")]
public async Task<IActionResult> List(CancellationToken token)
{
var notifications = await Page.From(Params, Limit(15, 30)).Fetch(Filtered(), n => n.ID, token);
Link("/api/v1/notifications", notifications.LastOrDefault()?.ID, notifications.FirstOrDefault()?.ID); Link("/api/v1/notifications", notifications.LastOrDefault()?.ID, notifications.FirstOrDefault()?.ID);
return Json(await Map(notifications, token)); return Json(await Map(notifications, token));
} }
// Grouped notifications (Mastodon 4.3): api_versions.mastodon says we speak that API, so clients call it. Likes and
// boosts of one post group together, as do follows within an hour; everything else stands alone.
static readonly HashSet<string> DefaultGrouped = new() { "favourite", "reblog", "follow" };
HashSet<string> Grouped() => Params.List("grouped_types") is { Count: > 0 } chosen ? chosen.ToHashSet() : DefaultGrouped;
static string GroupKey(NotificationEntity n, ISet<string> grouped) =>
!grouped.Contains(Names[n.Type]) ? $"ungrouped-{n.ID}"
: n.Type == NotificationType.Follow ? $"grouped-follow-{n.CreatedAt:yyyyMMddHH}"
: string.IsNullOrEmpty(n.PostId) ? $"ungrouped-{n.ID}"
: $"grouped-{Names[n.Type]}-{n.PostId}";
// the notifications a group key stands for
System.Linq.Expressions.Expression<Func<NotificationEntity, bool>> InGroup(string key)
{
var me = MyId;
if (key.StartsWith("ungrouped-", StringComparison.Ordinal))
{
var id = key["ungrouped-".Length..];
return n => n.AvatarId == me && n.ID == id;
}
if (key.StartsWith("grouped-follow-", StringComparison.Ordinal)
&& DateTime.TryParseExact(key["grouped-follow-".Length..], "yyyyMMddHH", System.Globalization.CultureInfo.InvariantCulture,
System.Globalization.DateTimeStyles.AdjustToUniversal | System.Globalization.DateTimeStyles.AssumeUniversal, out var hour))
{
var end = hour.AddHours(1);
return n => n.AvatarId == me && n.Type == NotificationType.Follow && n.CreatedAt >= hour && n.CreatedAt < end;
}
if (key.Split('-') is ["grouped", var name, var postId] && Parse(name) is { } type)
return n => n.AvatarId == me && n.Type == type && n.PostId == postId;
return default;
}
async Task<List<NotificationEntity>> Members(string key, CancellationToken token) =>
InGroup(key) is { } filter
? await _dbEntities.Notifications.Match(filter).Sort(n => n.ID, Order.Descending).Limit(80).ExecuteAsync(token)
: new List<NotificationEntity>();
async Task<object> Groups(List<NotificationEntity> page, ISet<string> grouped, CancellationToken token)
{
var mapped = (await Map(page, token)).ToDictionary(n => n.Id);
var shown = page.Where(n => mapped.ContainsKey(n.ID)).ToList();
var groups = new List<object>();
foreach (var group in shown.GroupBy(n => GroupKey(n, grouped)))
{
var newest = mapped[group.First().ID];
var total = group.Key.StartsWith("ungrouped-", StringComparison.Ordinal) ? 1 : await DB.Default.CountAsync(InGroup(group.Key), token);
groups.Add(new
{
group_key = group.Key,
notifications_count = Math.Max(total, group.Count()),
type = newest.Type,
most_recent_notification_id = newest.Id,
page_min_id = group.Last().ID,
page_max_id = group.First().ID,
latest_page_notification_at = newest.CreatedAt,
sample_account_ids = group.Select(n => mapped[n.ID].Account.Id).Distinct().Take(8).ToList(),
status_id = newest.Status?.Id,
emoji = newest.Emoji,
emoji_url = newest.EmojiUrl
});
}
return new
{
accounts = mapped.Values.Select(n => n.Account).DistinctBy(a => a.Id).ToList(),
statuses = mapped.Values.Where(n => n.Status != default).Select(n => n.Status).DistinctBy(s => s.Id).ToList(),
notification_groups = groups
};
}
[HttpGet("/api/v2/notifications"), Scope("read:notifications")]
public async Task<IActionResult> GroupedList(CancellationToken token)
{
var notifications = await Page.From(Params, Limit(40, 80)).Fetch(Filtered(), n => n.ID, token);
Link("/api/v2/notifications", notifications.LastOrDefault()?.ID, notifications.FirstOrDefault()?.ID);
return Json(await Groups(notifications, Grouped(), token));
}
[HttpGet("/api/v2/notifications/unread_count"), Scope("read:notifications")]
public async Task<IActionResult> GroupedUnreadCount(CancellationToken token)
{
var unread = await Filtered().Match(n => !n.IsRead).Sort(n => n.ID, Order.Descending)
.Limit(Math.Clamp(Params.Int("limit") ?? 100, 1, 1000)).ExecuteAsync(token);
var shown = (await Map(unread, token)).Select(n => n.Id).ToHashSet();
var grouped = Grouped();
return Json(new { count = unread.Where(n => shown.Contains(n.ID)).Select(n => GroupKey(n, grouped)).Distinct().Count() });
}
[HttpGet("/api/v2/notifications/{groupKey}"), Scope("read:notifications")]
public async Task<IActionResult> Group(string groupKey, CancellationToken token)
{
var members = await Members(groupKey, token);
if (members.Count == 0)
return NotFoundError();
return Json(await Groups(members, new HashSet<string>(DefaultGrouped.Append(Names[members[0].Type])), token));
}
[HttpGet("/api/v2/notifications/{groupKey}/accounts"), Scope("read:notifications")]
public async Task<IActionResult> GroupAccounts(string groupKey, CancellationToken token)
{
var members = await Members(groupKey, token);
if (members.Count == 0)
return NotFoundError();
return Json((await Map(members, token)).Select(n => n.Account).DistinctBy(a => a.Id).ToList());
}
[HttpPost("/api/v2/notifications/{groupKey}/dismiss"), Scope("write:notifications")]
public async Task<IActionResult> DismissGroup(string groupKey, CancellationToken token)
{
if (InGroup(groupKey) is not { } filter)
return NotFoundError();
await DB.Default.DeleteAsync(filter);
return Json(new { });
}
[HttpGet("/api/v1/notifications/{id}"), Scope("read:notifications")] [HttpGet("/api/v1/notifications/{id}"), Scope("read:notifications")]
public async Task<IActionResult> Get(string id, CancellationToken token) public async Task<IActionResult> Get(string id, CancellationToken token)
{ {
@@ -216,9 +336,15 @@ namespace PrivaPub.Api.Mastodon.Controllers
return Json(new { }); return Json(new { });
} }
// counts what the list would show (Map drops notifications from gone accounts and about hidden posts), at most
// `limit` of them, as Mastodon does
[HttpGet("/api/v1/notifications/unread_count"), Scope("read:notifications")] [HttpGet("/api/v1/notifications/unread_count"), Scope("read:notifications")]
public async Task<IActionResult> UnreadCount(CancellationToken token) => public async Task<IActionResult> UnreadCount(CancellationToken token)
Json(new { count = await DB.Default.CountAsync<NotificationEntity>(n => n.AvatarId == MyId && !n.IsRead, token) }); {
var unread = await DB.Default.Find<NotificationEntity>().Match(n => n.AvatarId == MyId && !n.IsRead)
.Sort(n => n.ID, Order.Descending).Limit(Math.Clamp(Params.Int("limit") ?? 100, 1, 1000)).ExecuteAsync(token);
return Json(new { count = (await Map(unread, token)).Count });
}
async Task<List<Entities.Notification>> Map(List<NotificationEntity> notifications, CancellationToken token) async Task<List<Entities.Notification>> Map(List<NotificationEntity> notifications, CancellationToken token)
{ {
+18 -8
View File
@@ -19,7 +19,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
_remoteActors = remoteActors; _remoteActors = remoteActors;
} }
public async Task<List<Account>> Find(string q, bool resolve, int limit, CancellationToken token) public async Task<List<Account>> Find(string q, bool resolve, int limit, CancellationToken token) => await Find(q, resolve, limit, 0, token);
// gone accounts (remote ones that deleted themselves, personas of banned or deleted roots) are never found
public async Task<List<Account>> Find(string q, bool resolve, int limit, int offset, CancellationToken token)
{ {
q = q?.Trim().TrimStart('@'); q = q?.Trim().TrimStart('@');
if (string.IsNullOrEmpty(q)) if (string.IsNullOrEmpty(q))
@@ -31,24 +34,31 @@ namespace PrivaPub.Api.Mastodon.Mappers
{ {
var userName = parts[0]; var userName = parts[0];
var domain = parts[1].ToLowerInvariant(); var domain = parts[1].ToLowerInvariant();
var known = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain).ExecuteFirstAsync(token); var known = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain && !f.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (known == default && resolve) if (known == default && resolve)
{ {
var actorUri = await _remoteActors.ResolveHandle(q, token); var actorUri = await _remoteActors.ResolveHandle(q, token);
known = actorUri == default ? default : await _remoteActors.GetActor(actorUri, refresh: false, token); known = actorUri == default ? default : await _remoteActors.GetActor(actorUri, refresh: false, token);
} }
if (known != default) if (known is { DeletionAt: null } && offset == 0)
results.Add(_mapper.Foreign(known)); results.Add(_mapper.Foreign(known));
return results; return results;
} }
var prefix = parts[0].ToLowerInvariant(); var prefix = parts[0].ToLowerInvariant();
foreach (var avatar in await _dbEntities.Avatars.Match(a => a.UserName.StartsWith(prefix) && !a.DeletionAt.HasValue && a.Settings.IsDiscoverable != false) var barred = await Domain.Privacy.Counted.BarredPersonas(token);
.Limit(limit).ExecuteAsync(token)) var locals = await _dbEntities.Avatars
.Match(a => a.UserName.StartsWith(prefix) && !a.DeletionAt.HasValue && a.Settings.IsDiscoverable != false && !barred.Contains(a.ID))
.Sort(a => a.UserName, MongoDB.Entities.Order.Ascending)
.Limit(offset + limit).ExecuteAsync(token);
foreach (var avatar in locals.Skip(offset))
results.Add(await _mapper.Local(_localActors.FromAvatar(avatar), false, token)); results.Add(await _mapper.Local(_localActors.FromAvatar(avatar), false, token));
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => f.UserName.StartsWith(parts[0]) && f.IsDiscoverable) var skipRemote = Math.Max(0, offset - locals.Count);
.Limit(Math.Max(0, limit - results.Count)).ExecuteAsync(token)) if (results.Count < limit)
results.Add(_mapper.Foreign(foreign)); foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => f.UserName.StartsWith(parts[0]) && f.IsDiscoverable && !f.DeletionAt.HasValue)
.Sort(f => f.UserName, MongoDB.Entities.Order.Ascending)
.Skip(skipRemote).Limit(limit - results.Count).ExecuteAsync(token))
results.Add(_mapper.Foreign(foreign));
return results; return results;
} }
} }
+11 -11
View File
@@ -50,15 +50,11 @@ namespace PrivaPub.Api.Mastodon.Mappers
? await DB.Default.CountAsync<Following>(f => f.AvatarId == actor.Id && f.State == FollowState.Accepted, token) ? await DB.Default.CountAsync<Following>(f => f.AvatarId == actor.Id && f.State == FollowState.Accepted, token)
: 0; : 0;
var latest = await _dbEntities.Posts var latest = await _dbEntities.Posts
.Match(p => (actor.Kind == LocalActorKind.Group ? p.GroupId == actor.Id : p.GroupUserId == actor.Id) && !p.IsFederatedCopy .Match(Counted.By(actor))
&& !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct && p.ReblogOfPostId == null)
.Sort(p => p.ID, Order.Descending) .Sort(p => p.ID, Order.Descending)
.Limit(1) .Limit(1)
.ExecuteFirstAsync(token); .ExecuteFirstAsync(token);
var statuses = latest == default var statuses = latest == default ? 0 : await Counted.PostsBy(actor, token);
? 0
: await DB.Default.CountAsync<PostEntity>(p => (actor.Kind == LocalActorKind.Group ? p.GroupId == actor.Id : p.GroupUserId == actor.Id)
&& !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct && p.ReblogOfPostId == null, token);
var account = new Account var account = new Account
{ {
@@ -71,7 +67,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
Discoverable = actor.Discoverable, Discoverable = actor.Discoverable,
Indexable = actor.Indexable, Indexable = actor.Indexable,
Noindex = !actor.Indexable, Noindex = !actor.Indexable,
HideCollections = actor.Settings.HideCollections, HideCollections = true,//follower and following lists are never shown to others, only their counts
Group = actor.Kind == LocalActorKind.Group, Group = actor.Kind == LocalActorKind.Group,
CreatedAt = MastodonJson.Day(actor.Published), CreatedAt = MastodonJson.Day(actor.Published),
Note = ActivityPubRenderer.Html(actor.Summary), Note = ActivityPubRenderer.Html(actor.Summary),
@@ -124,7 +120,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
Avatar = Proxied(foreign.PictureURL) ?? MissingAvatar, Avatar = Proxied(foreign.PictureURL) ?? MissingAvatar,
AvatarStatic = Proxied(foreign.PictureURL) ?? MissingAvatar, AvatarStatic = Proxied(foreign.PictureURL) ?? MissingAvatar,
Header = Proxied(foreign.ThumbnailURL) ?? MissingHeader, Header = Proxied(foreign.ThumbnailURL) ?? MissingHeader,
HeaderStatic = Proxied(foreign.ThumbnailURL) ?? MissingHeader HeaderStatic = Proxied(foreign.ThumbnailURL) ?? MissingHeader,
FollowersCount = foreign.FollowersCount ?? 0,//as its server publishes them (AccountCountsJob); 0 while unknown
FollowingCount = foreign.FollowingCount ?? 0,
StatusesCount = foreign.StatusesCount ?? 0
}; };
public async Task<Account> Account(string id, CancellationToken token) => public async Task<Account> Account(string id, CancellationToken token) =>
@@ -141,7 +140,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
accounts[avatar.ID] = await Local(_localActors.FromAvatar(avatar), false, token); accounts[avatar.ID] = await Local(_localActors.FromAvatar(avatar), false, token);
foreach (var group in await _dbEntities.Groups.Match(g => wanted.Contains(g.ID) && !g.DeletionAt.HasValue && g.Kind == GroupKind.Community).ExecuteAsync(token)) foreach (var group in await _dbEntities.Groups.Match(g => wanted.Contains(g.ID) && !g.DeletionAt.HasValue && g.Kind == GroupKind.Community).ExecuteAsync(token))
accounts[group.ID] = await Local(_localActors.FromGroup(group), false, token); accounts[group.ID] = await Local(_localActors.FromGroup(group), false, token);
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => wanted.Contains(f.ID)).ExecuteAsync(token)) foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => wanted.Contains(f.ID) && !f.DeletionAt.HasValue).ExecuteAsync(token))
accounts[foreign.ID] = Foreign(foreign); accounts[foreign.ID] = Foreign(foreign);
return accounts; return accounts;
} }
@@ -202,8 +201,8 @@ namespace PrivaPub.Api.Mastodon.Mappers
Status Map(PostEntity post) Status Map(PostEntity post)
{ {
if (!accounts.TryGetValue(AuthorOf(post), out var account)) if (AuthorOf(post) is not { } author || !accounts.TryGetValue(author, out var account))
return default; return default;//a post whose author we cannot show is left out, never a 500
var status = new Status var status = new Status
{ {
Id = post.ID, Id = post.ID,
@@ -283,6 +282,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
continue; continue;
Quote(inner, original); Quote(inner, original);
status.Reblog = inner; status.Reblog = inner;
status.Url = inner.Url;//a boost has no page of its own; /grunts would answer JSON to a browser
status.Content = string.Empty; status.Content = string.Empty;
status.Reblogged = reblogged.Contains(original.ID); status.Reblogged = reblogged.Contains(original.ID);
} }
@@ -34,6 +34,7 @@ namespace PrivaPub.Controllers.ClientToServer
readonly ILocalActorService LocalActors; readonly ILocalActorService LocalActors;
readonly AuthTokenManager AuthTokenManager; readonly AuthTokenManager AuthTokenManager;
readonly IOptionsMonitor<AppConfiguration> AppConfiguration; readonly IOptionsMonitor<AppConfiguration> AppConfiguration;
readonly IOptionsMonitor<RegistrationOptions> Registrations;
readonly ILogger<RootUserController> Logger; readonly ILogger<RootUserController> Logger;
readonly IStringLocalizer Localizer; readonly IStringLocalizer Localizer;
@@ -43,9 +44,11 @@ namespace PrivaPub.Controllers.ClientToServer
ILocalActorService localActors, ILocalActorService localActors,
AuthTokenManager authTokenManager, AuthTokenManager authTokenManager,
IOptionsMonitor<AppConfiguration> appConfiguration, IOptionsMonitor<AppConfiguration> appConfiguration,
IOptionsMonitor<RegistrationOptions> registrations,
IStringLocalizer<GenericRes> localizer, IStringLocalizer<GenericRes> localizer,
ILogger<RootUserController> logger) ILogger<RootUserController> logger)
{ {
Registrations = registrations;
UsersService = usersService; UsersService = usersService;
GroupUsersService = groupUsersService; GroupUsersService = groupUsersService;
AvatarUsersService = avatarUsersService; AvatarUsersService = avatarUsersService;
@@ -63,6 +66,8 @@ namespace PrivaPub.Controllers.ClientToServer
{ {
if (User.Identity?.IsAuthenticated ?? false) return Redirect("/"); if (User.Identity?.IsAuthenticated ?? false) return Redirect("/");
var result = new WebResult(); var result = new WebResult();
if (!Registrations.CurrentValue.IsOpen)
return StatusCode(StatusCodes.Status403Forbidden, result.Invalidate(Localizer[RegistrationOptions.ClosedMessage], StatusCodes.Status403Forbidden));
if (!ModelState.IsValid) if (!ModelState.IsValid)
return BadRequest(result.Invalidate(Localizer["Invalid model."])); return BadRequest(result.Invalidate(Localizer["Invalid model."]));
try try
@@ -356,7 +361,7 @@ namespace PrivaPub.Controllers.ClientToServer
if (!result.IsValid) if (!result.IsValid)
return StatusCode(result.StatusCode, result); return StatusCode(result.StatusCode, result);
return Ok(result); return Ok(result.Data);//the same sentence for everyone
} }
catch (Exception ex) catch (Exception ex)
{ {
@@ -409,26 +414,23 @@ namespace PrivaPub.Controllers.ClientToServer
} }
} }
//[HttpDelete, Route("delete"), Authorize(Policy = Policies.IsUser)] [HttpPost, Route("/clientapi/user/delete"), EnableRateLimiting(RateLimiting.Accounts), Authorize(Policy = Policies.IsUser)]
//public async Task<IActionResult> RemoveSelf() public async Task<IActionResult> RemoveSelf(RemoveSelfForm form)
//{ {
// var result = new WebResult(); var result = new WebResult();
// try if (!ModelState.IsValid)
// { return BadRequest(result.Invalidate(Localizer["Invalid model."]));
// //result = await UsersService.RemoveUserAsync(User.GetUserId()); try
// if (!result.IsValid) {
// return StatusCode(result.StatusCode, result); result = await UsersService.RemoveSelfAsync(User.GetUserId(), form.Password);
return result.IsValid ? Ok() : StatusCode(result.StatusCode, result);
// await HttpContext.SignOutAsync(); }
catch (Exception ex)
// return Ok(); {
// } Logger.LogError(ex, $"{nameof(User)}.{nameof(RemoveSelf)}()");
// catch (Exception ex) return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
// { }
// Logger.LogError(ex, $"{nameof(User)}.{nameof(RemoveSelf)}()"); }
// return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
// }
//}
#endregion User endpoints #endregion User endpoints
+3
View File
@@ -42,6 +42,9 @@ namespace PrivaPub.Domain.Media
["audio/mp4"] = "m4a", ["audio/x-m4a"] = "m4a", ["audio/flac"] = "flac", ["audio/webm"] = "webm" ["audio/mp4"] = "m4a", ["audio/x-m4a"] = "m4a", ["audio/flac"] = "flac", ["audio/webm"] = "webm"
}; };
// what an upload may be, as the instance API advertises it
public static IReadOnlyList<string> SupportedTypes => ImageTypes.Concat(AvTypes.Keys).ToList();
readonly IOptionsMonitor<MediaOptions> _options; readonly IOptionsMonitor<MediaOptions> _options;
readonly ILocalActorService _localActors; readonly ILocalActorService _localActors;
readonly IWebHostEnvironment _environment; readonly IWebHostEnvironment _environment;
+73
View File
@@ -0,0 +1,73 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using System.Linq.Expressions;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Privacy
{
// What the numbers count, in one place, so a persona's statuses_count, its outbox's totalItems, NodeInfo and the
// instance API agree (they used to count four different things).
public static class Counted
{
// Mastodon's statuses_count: every post that is neither deleted nor a direct message, reblogs included. Circle and
// located posts count too (owner decision 2026-10-04: "a good balance for the fediverse to work").
public static readonly Expression<Func<PostEntity, bool>> Post = p => !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct;
// a persona's own posts, or every post in a group, its remote members' included
public static Expression<Func<PostEntity, bool>> By(LocalActor actor)
{
var id = actor.Id;
return actor.Kind == LocalActorKind.Group
? p => p.GroupId == id && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct
: p => p.GroupUserId == id && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct;
}
// replies_count counts only public and unlisted replies, as Mastodon does: a count must not tell anyone that a
// followers-only, direct or circle reply exists
public static bool Reply(PostEntity reply) => reply.Visibility is PostVisibility.Public or PostVisibility.Unlisted;
public static async Task<long> PostsBy(LocalActor actor, CancellationToken token) =>
await DB.Default.CountAsync(By(actor), token);
// Personas that do not count as this server's users, and are not found in search: those of a banned or deleted root.
public static async Task<List<string>> BarredPersonas(CancellationToken token)
{
var roots = (await DB.Default.Find<RootUser>().Match(u => u.IsBanned || u.DeletedAt != null).Project(u => u.Include(x => x.ID)).ExecuteAsync(token))
.Select(u => u.ID).ToList();
return roots.Count == 0
? new List<string>()
: (await DB.Default.Find<RootToAvatar>().Match(ra => roots.Contains(ra.RootId)).ExecuteAsync(token)).Select(ra => ra.AvatarId).ToList();
}
public static async Task<long> Users(CancellationToken token)
{
var barred = await BarredPersonas(token);
return await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue && !barred.Contains(a.ID), token);
}
// users who posted within the last days, as NodeInfo's activeMonth and activeHalfyear and the instance API mean it
public static async Task<long> ActiveUsers(int days, CancellationToken token)
{
var barred = await BarredPersonas(token);
var since = DateTime.UtcNow.AddDays(-days);
var authors = await DB.Default.Find<PostEntity, string>()
.Match(p => !p.IsFederatedCopy && p.CreationDate >= since && !barred.Contains(p.GroupUserId))
.Project(p => p.GroupUserId)
.ExecuteAsync(token);
var distinct = authors.Where(a => !string.IsNullOrEmpty(a)).Distinct().ToList();
return distinct.Count == 0 ? 0 : await DB.Default.CountAsync<Avatar>(a => distinct.Contains(a.ID) && !a.DeletionAt.HasValue, token);
}
public static async Task<long> LocalPosts(CancellationToken token)
{
var barred = await BarredPersonas(token);
return await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy && !barred.Contains(p.GroupUserId)) & f.Where(Post), token);
}
}
}
+1 -1
View File
@@ -233,7 +233,7 @@ namespace PrivaPub.Domain.Statuses
return JobOutcome.Done; return JobOutcome.Done;
await Closing(post, author, token); await Closing(post, author, token);
if (!post.IsLocalOnly) if (!post.IsLocalOnly)
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, $"poll-{DateTime.UtcNow.Ticks}"), token); await _outbox.PublishUpdate(author, post, $"poll-{DateTime.UtcNow.Ticks}", token);
return JobOutcome.Done; return JobOutcome.Done;
} }
+1 -1
View File
@@ -239,7 +239,7 @@ namespace PrivaPub.Domain.Statuses
.ExecuteAsync(token); .ExecuteAsync(token);
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
if (!post.IsLocalOnly) if (!post.IsLocalOnly)
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, "quote-approved"), token); await _outbox.PublishUpdate(author, post, "quote-approved", token);
return true; return true;
} }
+26 -3
View File
@@ -95,8 +95,17 @@ namespace PrivaPub.Domain.Statuses
_fanout = fanout; _fanout = fanout;
} }
public const int MaxCharacters = 5000;//what the instance API advertises as max_characters, and enforced here
static StatusOutcome TooLong(StatusDraft draft) =>
(draft.Text?.Length ?? 0) > MaxCharacters
? StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, $"Validation failed: Text character limit of {MaxCharacters} exceeded")
: default;
public async Task<StatusOutcome> Publish(LocalActor author, StatusDraft draft, CancellationToken token) public async Task<StatusOutcome> Publish(LocalActor author, StatusDraft draft, CancellationToken token)
{ {
if (TooLong(draft) is { } tooLong)
return tooLong;
var media = await Media(author, draft.MediaIds, default, token); var media = await Media(author, draft.MediaIds, default, token);
if (media == default) if (media == default)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid"); return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid");
@@ -119,6 +128,11 @@ namespace PrivaPub.Domain.Statuses
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token)) if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token))
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
// a reply in a circle stays in the circle, whichever client wrote it: Mastodon clients know nothing of groups
if (group == default && parent is { Visibility: PostVisibility.Circle } && !string.IsNullOrEmpty(parent.GroupId)
&& await _dbEntities.Groups.MatchID(parent.GroupId).ExecuteFirstAsync(token) is { DeletionAt: null } parentCircle
&& await MayPost(parentCircle, author, token))
group = _localActors.FromGroup(parentCircle);
PostEntity quoted = default; PostEntity quoted = default;
var quotePermission = QuotePermission.Denied; var quotePermission = QuotePermission.Denied;
@@ -161,6 +175,9 @@ namespace PrivaPub.Domain.Statuses
var visibility = located ? PostVisibility.LocalGeo var visibility = located ? PostVisibility.LocalGeo
: group is { IsCircle: true } ? PostVisibility.Circle : group is { IsCircle: true } ? PostVisibility.Circle
: draft.Visibility is PostVisibility.Circle or PostVisibility.LocalGeo ? PostVisibility.Public : draft.Visibility; : draft.Visibility is PostVisibility.Circle or PostVisibility.LocalGeo ? PostVisibility.Public : draft.Visibility;
// asking a quoted post's author for permission would show them the circle post
if (visibility == PostVisibility.Circle && quoted != default && quotePermission != QuotePermission.Granted)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: A circle post can only quote a post that needs no permission");
var post = new PostEntity var post = new PostEntity
{ {
GroupUserId = author.Id, GroupUserId = author.Id,
@@ -231,7 +248,7 @@ namespace PrivaPub.Domain.Statuses
await DB.Default.SaveAsync(post, token); await DB.Default.SaveAsync(post, token);
await Attach(media, post.ID, token); await Attach(media, post.ID, token);
if (parent != default) if (parent != default && Counted.Reply(post))
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
await _fanout.Distribute(post, token); await _fanout.Distribute(post, token);
await _polls.Scheduled(post, token); await _polls.Scheduled(post, token);
@@ -252,6 +269,8 @@ namespace PrivaPub.Domain.Statuses
var post = await Own(author, postId, token); var post = await Own(author, postId, token);
if (post == default) if (post == default)
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
if (TooLong(draft) is { } tooLong)
return tooLong;
var media = draft.MediaIds == default ? default : await Media(author, draft.MediaIds, post.ID, token); var media = draft.MediaIds == default ? default : await Media(author, draft.MediaIds, post.ID, token);
if (draft.MediaIds != default && media == default) if (draft.MediaIds != default && media == default)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid"); return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid");
@@ -334,13 +353,17 @@ namespace PrivaPub.Domain.Statuses
.Modify(p => p.Revisions, new List<PostRevision>()) .Modify(p => p.Revisions, new List<PostRevision>())
.ExecuteAsync(token); .ExecuteAsync(token);
await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID); await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID);
if (!string.IsNullOrEmpty(post.AnsweringToPostId)) // boosts of it end with it, as on Mastodon, so no count keeps them
await DB.Default.Update<PostEntity>().Match(p => p.ReblogOfPostId == post.ID && !p.DeletedAt.HasValue)
.Modify(p => p.DeletedAt, DateTime.UtcNow).ExecuteAsync(token);
if (!string.IsNullOrEmpty(post.AnsweringToPostId) && Counted.Reply(post))
await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token);
await DB.Default.DeleteAsync<Models.Social.Pin>(p => p.PostId == post.ID);//a deleted post no longer takes a pin
if (audience.Count > 0) if (audience.Count > 0)
{ {
var delete = ActivityPubRenderer.Delete(author, post.ObjectURI, $"delete-{post.ID}", var delete = ActivityPubRenderer.Delete(author, post.ObjectURI, $"delete-{post.ID}",
new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()), new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray())); new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()), new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()));
await _delivery.Enqueue(author, audience, delete, token); await _outbox.Publish(author, post, delete, token);//the post in hand still has its group and mentions
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token); var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
if (group is { IsCircle: false }) if (group is { IsCircle: false })
await _groups.Announce(group, delete, post.ObjectURI, isNewPost: false, token); await _groups.Announce(group, delete, post.ObjectURI, isNewPost: false, token);
@@ -0,0 +1,60 @@
using MongoDB.Entities;
using PrivaPub.Federation.Objects;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
namespace PrivaPub.Federation.Actors
{
// A remote account's follower, following and post counts, as its own server publishes them (the totalItems of its
// followers, following and outbox collections). Read when the account is fetched, at most once a day, never when
// someone looks at it; a hidden or missing collection leaves its count unknown. Mastodon and GoToSocial do the same.
public class AccountCountsJob : IJobHandler
{
readonly IRemoteActorService _remoteActors;
readonly DbEntities _dbEntities;
public AccountCountsJob(IRemoteActorService remoteActors, DbEntities dbEntities)
{
_remoteActors = remoteActors;
_dbEntities = dbEntities;
}
public JobKind Kind => JobKind.CountAccount;
public int Concurrency => 2;
public int MaxAttempts => 2;
public int PerHostLimit => 1;
public static string DedupeKey(string actorUri, DateTime day) => $"count|{actorUri}|{day:yyyyMMdd}";
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var actor = await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == job.Payload && !a.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (actor == default)
return JobOutcome.Done;
await DB.Default.Update<ForeignAvatar>().MatchID(actor.ID)
.Modify(a => a.FollowersCount, await Total(actor, actor.FollowersURL, token))
.Modify(a => a.FollowingCount, await Total(actor, actor.FollowingURL, token))
.Modify(a => a.StatusesCount, await Total(actor, actor.OutboxURL, token))
.Modify(a => a.CountedAt, DateTime.UtcNow)
.ExecuteAsync(token);
return JobOutcome.Done;
}
// only from the actor's own server, as everything we believe about it
async Task<int?> Total(ForeignAvatar actor, string collection, CancellationToken token)
{
if (string.IsNullOrEmpty(collection) || !Origin.Same(collection, actor.ActorURI))
return default;
using var scope = HttpScope.For("collection");
using var fetched = await _remoteActors.FetchObject(collection, token);
return fetched != default && fetched.Root.ValueKind == System.Text.Json.JsonValueKind.Object
&& fetched.Root.TryGetProperty("totalItems", out var total) && total.TryGetInt32(out var count) && count >= 0
? count
: default;
}
}
}
@@ -62,6 +62,7 @@ namespace PrivaPub.Federation.Actors
{ {
string BaseAddress { get; } string BaseAddress { get; }
Task<LocalActor> FindByUserName(string userName, CancellationToken token); Task<LocalActor> FindByUserName(string userName, CancellationToken token);
Task<GoneActor> Gone(string userName, CancellationToken token);
Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token); Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token);
Task<LocalActor> FindByUri(string actorUri, CancellationToken token); Task<LocalActor> FindByUri(string actorUri, CancellationToken token);
Task<LocalActor> GetInstanceActor(CancellationToken token); Task<LocalActor> GetInstanceActor(CancellationToken token);
@@ -71,6 +72,9 @@ namespace PrivaPub.Federation.Actors
LocalActor FromGroup(GroupEntity group); LocalActor FromGroup(GroupEntity group);
} }
// a persona or group that was deleted: its name stays reserved, and its documents answer 410 with this
public sealed record GoneActor(string Uri, string FormerType, DateTime DeletedAt);
public class LocalActorService : ILocalActorService public class LocalActorService : ILocalActorService
{ {
public const string InstanceUserName = "privapub"; public const string InstanceUserName = "privapub";
@@ -113,6 +117,18 @@ namespace PrivaPub.Federation.Actors
return group == default ? default : FromGroup(group); return group == default ? default : FromGroup(group);
} }
public async Task<GoneActor> Gone(string userName, CancellationToken token)
{
if (string.IsNullOrEmpty(userName))
return default;
userName = userName.ToLowerInvariant();
var avatar = await _dbEntities.Avatars.Match(a => a.UserName == userName && a.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (avatar != default)
return new GoneActor($"{BaseAddress}/peasants/{avatar.UserName}", "Person", avatar.DeletionAt.Value);
var group = await _dbEntities.Groups.Match(g => g.UserName == userName && g.DeletionAt.HasValue).ExecuteFirstAsync(token);
return group == default ? default : new GoneActor($"{BaseAddress}/peasants/{group.UserName}", "Group", group.DeletionAt.Value);
}
public async Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token) public async Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token)
{ {
switch (kind) switch (kind)
@@ -36,9 +36,12 @@ namespace PrivaPub.Federation.Actors
readonly DbEntities _dbEntities; readonly DbEntities _dbEntities;
readonly IOptionsMonitor<FederationOptions> _options; readonly IOptionsMonitor<FederationOptions> _options;
readonly Infrastructure.Jobs.IJobQueue _jobs;
public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities, public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities,
IOptionsMonitor<FederationOptions> options = default) IOptionsMonitor<FederationOptions> options = default, Infrastructure.Jobs.IJobQueue jobs = default)
{ {
_jobs = jobs;
_http = http; _http = http;
_localActors = localActors; _localActors = localActors;
_cache = cache; _cache = cache;
@@ -89,7 +92,16 @@ namespace PrivaPub.Federation.Actors
return cached; return cached;
var key = cached == default ? default : actor.Key(cached.PublicKeyId); var key = cached == default ? default : actor.Key(cached.PublicKeyId);
return await Upsert(actor, key ?? actor.Keys.FirstOrDefault(), token); return await Counted(await Upsert(actor, key ?? actor.Keys.FirstOrDefault(), token), token);
}
// its counts are read once a day, after it was fetched (AccountCountsJob)
async Task<ForeignAvatar> Counted(ForeignAvatar stored, CancellationToken token)
{
if (_jobs != default && stored is { DeletionAt: null } && Uri.TryCreate(stored.ActorURI, UriKind.Absolute, out var uri))
await _jobs.Enqueue(Models.Jobs.JobKind.CountAccount, stored.ActorURI, uri.Host.ToLowerInvariant(),
AccountCountsJob.DedupeKey(stored.ActorURI, DateTime.UtcNow), token);
return stored;
} }
public async Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token) public async Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token)
@@ -121,7 +133,7 @@ namespace PrivaPub.Federation.Actors
var key = actor?.Key(keyId); var key = actor?.Key(keyId);
if (key == default) if (key == default)
return default; return default;
return await Upsert(actor, key, token); return await Counted(await Upsert(actor, key, token), token);
} }
public bool KeyTemporarilyUnavailable(string keyId) => Origin.Of(keyId) != default && _http.FailedTemporarily(StripFragment(keyId)); public bool KeyTemporarilyUnavailable(string keyId) => Origin.Of(keyId) != default && _http.FailedTemporarily(StripFragment(keyId));
@@ -17,6 +17,7 @@ using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering; using PrivaPub.Federation.Rendering;
using PrivaPub.Federation.Inbox; using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Signing; using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Http; using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Group; using PrivaPub.Models.Group;
@@ -52,6 +53,20 @@ namespace PrivaPub.Federation.Controllers
public async Task<IActionResult> GetActor(string actor, CancellationToken token) public async Task<IActionResult> GetActor(string actor, CancellationToken token)
{ {
var local = await _localActors.FindByUserName(actor, token); var local = await _localActors.FindByUserName(actor, token);
if (local == default && await _localActors.Gone(actor, token) is { } gone)
return new ContentResult
{
Content = new JsonObject
{
["@context"] = ActivityPubRenderer.ActivityStreams,
["id"] = gone.Uri,
["type"] = "Tombstone",
["formerType"] = gone.FormerType,
["deleted"] = ActivityPubRenderer.Timestamp(gone.DeletedAt)
}.ToJsonString(),
ContentType = ActivityContentType,
StatusCode = StatusCodes.Status410Gone
};
if (local is not { IsFederated: true }) if (local is not { IsFederated: true })
return NotFound(); return NotFound();
if (WantsHtml() && local.Kind != LocalActorKind.Application) if (WantsHtml() && local.Kind != LocalActorKind.Application)
@@ -69,9 +84,8 @@ namespace PrivaPub.Federation.Controllers
if (!page) if (!page)
{ {
var total = local.Kind == LocalActorKind.Group // as Mastodon's: statuses_count, though only public posts are listed
? await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupId == local.Id) & f.Where(VisibilityPolicy.IsPublic), token) var total = await Counted.PostsBy(local, token);
: await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupUserId == local.Id && !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token);
return Activity(ActivityPubRenderer.OrderedCollection(local.Outbox, (int)total, default, $"{local.Outbox}?page=true")); return Activity(ActivityPubRenderer.OrderedCollection(local.Outbox, (int)total, default, $"{local.Outbox}?page=true"));
} }
@@ -87,7 +101,7 @@ namespace PrivaPub.Federation.Controllers
{ {
if (local.Kind == LocalActorKind.Group) if (local.Kind == LocalActorKind.Group)
{ {
items.Add(ActivityPubRenderer.Announce(local, post.ObjectURI, $"announce-{post.ID}")); items.Add(ActivityPubRenderer.Announce(local, post.ObjectURI, $"announce-{post.ID}", post.CreationDate));
continue; continue;
} }
if (post.ReblogOfPostId != default) if (post.ReblogOfPostId != default)
@@ -122,7 +136,13 @@ namespace PrivaPub.Federation.Controllers
public async Task<IActionResult> Following(string actor, CancellationToken token) public async Task<IActionResult> Following(string actor, CancellationToken token)
{ {
var local = await _localActors.FindByUserName(actor, token); var local = await _localActors.FindByUserName(actor, token);
return local is not { IsFederated: true } ? NotFound() : Activity(ActivityPubRenderer.OrderedCollection(local.Following, 0, default)); if (local is not { IsFederated: true })
return NotFound();
// the count, as following_count says it; who is followed is never listed (FEDERATION.md)
var following = local.Kind == LocalActorKind.Person
? await DB.Default.CountAsync<Following>(f => f.AvatarId == local.Id && f.State == FollowState.Accepted, token)
: 0;
return Activity(ActivityPubRenderer.OrderedCollection(local.Following, (int)following, default));
} }
[HttpGet, Route("{actor}/flock")] [HttpGet, Route("{actor}/flock")]
@@ -185,11 +205,13 @@ namespace PrivaPub.Federation.Controllers
public async Task<IActionResult> Post(string actor, string postId, CancellationToken token) public async Task<IActionResult> Post(string actor, string postId, CancellationToken token)
{ {
var (local, post) = await PublicPost(actor, postId, token); var (local, post) = await PublicPost(actor, postId, token);
if (post == default && await CirclePost(actor, postId, token) is { } circlePost) if (post == default && await SignedPost(actor, postId, token) is { } signed)
{ {
var circleNote = ActivityPubRenderer.Note(circlePost.Post, circlePost.Author, circlePost.Circle, circlePost.Post.InReplyToURI); if (signed.Post.DeletedAt.HasValue)
circleNote["@context"] = ActivityPubRenderer.Context(); return TombstoneOf(signed.Author, signed.Post);
return Activity(circleNote); var signedNote = (JsonObject)signed.Note.DeepClone();
signedNote["@context"] = ActivityPubRenderer.Context();
return Activity(signedNote);
} }
if (post == default) if (post == default)
return await Tombstone(actor, postId, token) ?? NotFound(); return await Tombstone(actor, postId, token) ?? NotFound();
@@ -236,20 +258,55 @@ namespace PrivaPub.Federation.Controllers
{ {
var booster = await _localActors.FindByUserName(actor, token); var booster = await _localActors.FindByUserName(actor, token);
var uri = booster?.ActivityUri(activityId); var uri = booster?.ActivityUri(activityId);
if (booster is { Kind: LocalActorKind.Group, IsCircle: false })
return await GroupAnnounce(booster, activityId, uri, token);
var reblog = uri == default ? default : await _dbEntities.Posts.Match(p => p.ObjectURI == uri && !p.DeletedAt.HasValue).ExecuteFirstAsync(token); var reblog = uri == default ? default : await _dbEntities.Posts.Match(p => p.ObjectURI == uri && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
var rendered = reblog == default ? default : await AnnounceFor(reblog, token); var rendered = reblog == default ? default : await AnnounceFor(reblog, token);
if (rendered != default && WantsHtml() && await _dbEntities.Posts.MatchID(reblog.ReblogOfPostId).ExecuteFirstAsync(token) is { } boosted)
return Redirect(boosted.Url ?? boosted.ObjectURI);//a boost has no page of its own: the boosted post's
return rendered == default ? NotFound() : Activity(rendered); return rendered == default ? NotFound() : Activity(rendered);
} }
if (!activityId.StartsWith("create-", StringComparison.Ordinal)) if (!activityId.StartsWith("create-", StringComparison.Ordinal))
return NotFound(); return NotFound();
var (local, post) = await PublicPost(actor, activityId["create-".Length..], token); var (local, post) = await PublicPost(actor, activityId["create-".Length..], token);
return post == default ? NotFound() : Activity(await CreateFor(post, local, token)); if (post != default)
return Activity(await CreateFor(post, local, token));
return await SignedPost(actor, activityId["create-".Length..], token) is { Post.DeletedAt: null } signed
? Activity(ActivityPubRenderer.Create(signed.Author, (JsonObject)signed.Note.DeepClone(), activityId))
: NotFound();
}
// A DM's `context`: the conversation's posts, for its participants (or their servers' instance actors) only.
[HttpGet, Route("{actor}/whispers/{conversationId}")]
public async Task<IActionResult> Whispers(string actor, string conversationId, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local is not { IsFederated: true, Kind: LocalActorKind.Person })
return NotFound();
var uri = local.ConversationUri(conversationId);
var conversation = await _dbEntities.DmGroups.Match(g => g.ID == conversationId && g.ConversationURI == uri && !g.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (conversation == default || !SignedFetchAuthorizer.MayReadConversation(conversation, await _fetches.Requester(Request, token)))
return NotFound();
var posts = await _dbEntities.Posts
.Match(p => p.ConversationId == conversationId && p.Visibility == PostVisibility.Direct && !p.DeletedAt.HasValue)
.Sort(p => p.CreationDate, Order.Ascending)
.ExecuteAsync(token);
return Activity(new JsonObject
{
["@context"] = ActivityPubRenderer.ActivityStreams,
["id"] = uri,
["type"] = "OrderedCollection",
["totalItems"] = posts.Count,
["orderedItems"] = new JsonArray(posts.Select(p => (JsonNode)p.ObjectURI).ToArray())
});
} }
[HttpPost, Route("{actor}/mouth"), EnableRateLimiting(RateLimiting.Inbox)] [HttpPost, Route("{actor}/mouth"), EnableRateLimiting(RateLimiting.Inbox)]
public async Task<IActionResult> Inbox(string actor, CancellationToken token) public async Task<IActionResult> Inbox(string actor, CancellationToken token)
{ {
var local = await _localActors.FindByUserName(actor, token); var local = await _localActors.FindByUserName(actor, token);
if (local == default && await _localActors.Gone(actor, token) != default)
return StatusCode(StatusCodes.Status410Gone);
if (local is not { IsFederated: true }) if (local is not { IsFederated: true })
return Answer(_inbox.NoSuchRecipient(Request)); return Answer(_inbox.NoSuchRecipient(Request));
return Answer(await _inbox.Receive(Request, local, token)); return Answer(await _inbox.Receive(Request, local, token));
@@ -275,18 +332,37 @@ namespace PrivaPub.Federation.Controllers
return (local, post); return (local, post);
} }
async Task<(LocalActor Author, LocalActor Circle, PostEntity Post)?> CirclePost(string actor, string postId, CancellationToken token) // A followers-only, direct or circle post (deleted ones included), for a signed request from someone it was for
// (SignedFetchAuthorizer.MayRead). It is rendered as it was delivered: a circle post also names, in cc, the requesting
// member, or the members on the requesting instance actor's server.
async Task<(LocalActor Author, PostEntity Post, JsonObject Note)?> SignedPost(string actor, string postId, CancellationToken token)
{ {
var local = await _localActors.FindByUserName(actor, token); var local = await _localActors.FindByUserName(actor, token);
if (local is not { Kind: LocalActorKind.Person }) if (local is not { IsFederated: true, Kind: LocalActorKind.Person })
return default; return default;
var post = await _dbEntities.Posts var post = await _dbEntities.Posts
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility == PostVisibility.Circle) .Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.ReblogOfPostId == null
&& (p.Visibility == PostVisibility.FollowersOnly || p.Visibility == PostVisibility.Direct || p.Visibility == PostVisibility.Circle))
.ExecuteFirstAsync(token); .ExecuteFirstAsync(token);
var circle = post == default ? default : await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token); if (post == default)
if (circle == default || !SignedFetchAuthorizer.MayReadCircle(circle, await _fetches.Requester(Request, token)))
return default; return default;
return (local, _localActors.FromGroup(circle), post); var requester = await _fetches.Requester(Request, token);
if (!await _fetches.MayRead(post, requester, token))
return default;
if (post.Visibility == PostVisibility.Circle)
{
var circle = await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token);
var note = ActivityPubRenderer.Note(post, local, _localActors.FromGroup(circle), post.InReplyToURI);
var readers = SignedFetchAuthorizer.CircleReaders(circle, requester);
var named = readers.Count == 0 ? new List<Models.User.ForeignAvatar>() : await _dbEntities.ForeignAvatars.Match(a => readers.Contains(a.ActorURI)).ExecuteAsync(token);
return (local, post, OutboxPublisher.Naming(note, named));
}
var rendered = post.Visibility == PostVisibility.Direct
? ActivityPubRenderer.DirectNote(post, local, Array.Empty<(string, string)>(), post.ContextURI)
: ActivityPubRenderer.Note(post, local, default, post.InReplyToURI);
rendered["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
rendered["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
return (local, post, rendered);
} }
async Task<IActionResult> Tombstone(string actor, string postId, CancellationToken token) async Task<IActionResult> Tombstone(string actor, string postId, CancellationToken token)
@@ -298,8 +374,11 @@ namespace PrivaPub.Federation.Controllers
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.DeletedAt.HasValue .Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.DeletedAt.HasValue
&& (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted)) && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted))
.ExecuteFirstAsync(token); .ExecuteFirstAsync(token);
if (deleted == default) return deleted == default ? default : TombstoneOf(local, deleted);
return default; }
ContentResult TombstoneOf(LocalActor local, PostEntity deleted)
{
var tombstone = new JsonObject var tombstone = new JsonObject
{ {
["@context"] = ActivityPubRenderer.ActivityStreams, ["@context"] = ActivityPubRenderer.ActivityStreams,
@@ -311,6 +390,24 @@ namespace PrivaPub.Federation.Controllers
return new ContentResult { Content = tombstone.ToJsonString(), ContentType = ActivityContentType, StatusCode = StatusCodes.Status410Gone }; return new ContentResult { Content = tombstone.ToJsonString(), ContentType = ActivityContentType, StatusCode = StatusCodes.Status410Gone };
} }
// A community's announce: one GroupDistributor sent (kept as sent), or one its outbox lists (announce-{postId}),
// while the post it is about is still shown; 410 once that post is gone.
async Task<IActionResult> GroupAnnounce(LocalActor group, string activityId, string uri, CancellationToken token)
{
if (await DB.Default.Find<GroupAnnouncement>().Match(a => a.ActivityURI == uri && a.GroupId == group.Id).ExecuteFirstAsync(token) is { } kept)
{
var about = await _dbEntities.Posts.Match(p => p.ObjectURI == kept.ObjectURI).ExecuteFirstAsync(token);
if (!VisibilityPolicy.Shown(about) || about.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
return StatusCode(StatusCodes.Status410Gone);
var body = JsonNode.Parse(kept.Body)!.AsObject();
body["@context"] = ActivityPubRenderer.Context();
return Activity(body);
}
var postId = activityId["announce-".Length..];
var post = await _dbEntities.Posts.Match(p => p.ID == postId && p.GroupId == group.Id).Match(VisibilityPolicy.IsPublic).ExecuteFirstAsync(token);
return post == default ? NotFound() : Activity(ActivityPubRenderer.Announce(group, post.ObjectURI, activityId, post.CreationDate));
}
async Task<JsonObject> AnnounceFor(PostEntity reblog, CancellationToken token) async Task<JsonObject> AnnounceFor(PostEntity reblog, CancellationToken token)
{ {
var original = await _dbEntities.Posts.MatchID(reblog.ReblogOfPostId).ExecuteFirstAsync(token); var original = await _dbEntities.Posts.MatchID(reblog.ReblogOfPostId).ExecuteFirstAsync(token);
@@ -349,7 +446,9 @@ namespace PrivaPub.Federation.Controllers
{ {
if (HttpMethods.IsGet(Request.Method)) if (HttpMethods.IsGet(Request.Method))
Response.Headers.Vary = "Accept"; Response.Headers.Vary = "Accept";
if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method) // SecureMode asks every reader of ActivityPub documents for a signature, except for the instance actor, whose key
// peers need first, and except for browsers, which only get redirected to the public pages
if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method) && !WantsHtml()
&& !string.Equals(context.RouteData.Values["actor"] as string, LocalActorService.InstanceUserName, StringComparison.OrdinalIgnoreCase) && !string.Equals(context.RouteData.Values["actor"] as string, LocalActorService.InstanceUserName, StringComparison.OrdinalIgnoreCase)
&& await _fetches.Requester(Request, HttpContext.RequestAborted) == default) && await _fetches.Requester(Request, HttpContext.RequestAborted) == default)
{ {
@@ -1,4 +1,5 @@
using Microsoft.AspNetCore.Mvc; using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Options;
using MongoDB.Entities; using MongoDB.Entities;
@@ -12,6 +13,7 @@ using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post; using PostEntity = PrivaPub.Models.Post.Post;
using PrivaPub.Domain.Privacy; using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure;
namespace PrivaPub.Federation.Controllers namespace PrivaPub.Federation.Controllers
{ {
@@ -20,9 +22,11 @@ namespace PrivaPub.Federation.Controllers
{ {
readonly ILocalActorService _localActors; readonly ILocalActorService _localActors;
readonly DbEntities _dbEntities; readonly DbEntities _dbEntities;
readonly IOptionsMonitor<RegistrationOptions> _registrations;
public WellKnownController(ILocalActorService localActors, DbEntities dbEntities) public WellKnownController(ILocalActorService localActors, DbEntities dbEntities, IOptionsMonitor<RegistrationOptions> registrations)
{ {
_registrations = registrations;
_localActors = localActors; _localActors = localActors;
_dbEntities = dbEntities; _dbEntities = dbEntities;
} }
@@ -45,6 +49,8 @@ namespace PrivaPub.Federation.Controllers
if (parts.Length != 2 || !parts[1].Equals(domain, StringComparison.OrdinalIgnoreCase)) if (parts.Length != 2 || !parts[1].Equals(domain, StringComparison.OrdinalIgnoreCase))
return NotFound(); return NotFound();
actor = await _localActors.FindByUserName(parts[0], token); actor = await _localActors.FindByUserName(parts[0], token);
if (actor == default && await _localActors.Gone(parts[0], token) != default)
return StatusCode(StatusCodes.Status410Gone);
} }
else else
actor = await _localActors.FindByUri(resource, token); actor = await _localActors.FindByUri(resource, token);
@@ -88,8 +94,8 @@ namespace PrivaPub.Federation.Controllers
[HttpGet, Route("/nodeinfo/{version:regex(^2\\.[[01]]$)}")] [HttpGet, Route("/nodeinfo/{version:regex(^2\\.[[01]]$)}")]
public async Task<IActionResult> NodeInfo(string version, CancellationToken token) public async Task<IActionResult> NodeInfo(string version, CancellationToken token)
{ {
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token); var users = await Counted.Users(token);
var posts = await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy && p.ReblogOfPostId == null) & f.Where(VisibilityPolicy.IsPublic), token); var posts = await Counted.LocalPosts(token);
var software = new JsonObject { ["name"] = "privapub", ["version"] = BuildInfo.Ref }; var software = new JsonObject { ["name"] = "privapub", ["version"] = BuildInfo.Ref };
if (version == "2.1") if (version == "2.1")
{ {
@@ -102,10 +108,15 @@ namespace PrivaPub.Federation.Controllers
["software"] = software, ["software"] = software,
["protocols"] = new JsonArray("activitypub"), ["protocols"] = new JsonArray("activitypub"),
["services"] = new JsonObject { ["inbound"] = new JsonArray(), ["outbound"] = new JsonArray() }, ["services"] = new JsonObject { ["inbound"] = new JsonArray(), ["outbound"] = new JsonArray() },
["openRegistrations"] = true, ["openRegistrations"] = _registrations.CurrentValue.IsOpen,
["usage"] = new JsonObject ["usage"] = new JsonObject
{ {
["users"] = new JsonObject { ["total"] = users }, ["users"] = new JsonObject
{
["total"] = users,
["activeMonth"] = await Counted.ActiveUsers(30, token),
["activeHalfyear"] = await Counted.ActiveUsers(180, token)
},
["localPosts"] = posts ["localPosts"] = posts
}, },
["metadata"] = new JsonObject ["metadata"] = new JsonObject
+63
View File
@@ -0,0 +1,63 @@
using MongoDB.Entities;
using PrivaPub.Domain.Privacy;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Inbox
{
// A remote account that deleted itself keeps its posts, hidden (owner decision 2026-10-03), but leaves nothing in our
// numbers: its likes, downvotes, reactions and poll votes go, and the counters they had moved come back, as do the
// reblog, reply and quote counts its own posts had added. Its notifications go too. ForgottenAt makes it run once.
public static class GoneActors
{
public static async Task Forget(ForeignAvatar actor, CancellationToken token)
{
if (actor == default || actor.ForgottenAt.HasValue)
return;
var uri = actor.ActorURI;
foreach (var like in await DB.Default.Find<Favourite>().Match(f => f.ActorURI == uri).ExecuteAsync(token))
{
await DB.Default.Update<PostEntity>().MatchID(like.PostId).Modify(b => b.Inc(p => p.FavouritesCount, -1)).ExecuteAsync(token);
await DB.Default.DeleteAsync<Favourite>(like.ID);
}
foreach (var down in await DB.Default.Find<Downvote>().Match(d => d.ActorURI == uri).ExecuteAsync(token))
{
await DB.Default.Update<PostEntity>().MatchID(down.PostId).Modify(b => b.Inc(p => p.DownvotesCount, -1)).ExecuteAsync(token);
await DB.Default.DeleteAsync<Downvote>(down.ID);
}
await DB.Default.DeleteAsync<Reaction>(r => r.ActorURI == uri);
foreach (var ballot in (await DB.Default.Find<PollVote>().Match(v => v.ActorURI == uri).ExecuteAsync(token)).GroupBy(v => v.PostId))
{
var poll = await DB.Default.Find<PostEntity>().MatchID(ballot.Key).ExecuteFirstAsync(token);
if (poll?.Poll != default)
{
var update = DB.Default.Update<PostEntity>().MatchID(poll.ID);
foreach (var vote in ballot.Where(v => v.Choice >= 0 && v.Choice < poll.Poll.Options.Count))
update = update.Modify(b => b.Inc($"{nameof(PostEntity.Poll)}.{nameof(PostPoll.Options)}.{vote.Choice}.{nameof(PollOption.Votes)}", -1));
if (poll.Poll.VotersCount.HasValue)
update = update.Modify(b => b.Inc($"{nameof(PostEntity.Poll)}.{nameof(PostPoll.VotersCount)}", -1));
await update.ExecuteAsync(token);
}
await DB.Default.DeleteAsync<PollVote>(v => v.PostId == ballot.Key && v.ActorURI == uri);
}
foreach (var post in await DB.Default.Find<PostEntity>().Match(p => p.ActorURI == uri && !p.DeletedAt.HasValue).ExecuteAsync(token))
{
if (!string.IsNullOrEmpty(post.ReblogOfPostId))
await DB.Default.Update<PostEntity>().MatchID(post.ReblogOfPostId).Modify(b => b.Inc(p => p.ReblogsCount, -1)).ExecuteAsync(token);
if (!string.IsNullOrEmpty(post.AnsweringToPostId) && Counted.Reply(post))
await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token);
if (!string.IsNullOrEmpty(post.QuotedPostId) && post.QuoteState == QuoteState.Accepted)
await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
}
await DB.Default.DeleteAsync<Notification>(n => n.FromActorURI == uri);
await DB.Default.Update<ForeignAvatar>().MatchID(actor.ID).Modify(a => a.ForgottenAt, DateTime.UtcNow).ExecuteAsync(token);
}
}
}
@@ -182,7 +182,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
await _previews.Wanted(post, token); await _previews.Wanted(post, token);
await _quotes.Resolve(post, note, token); await _quotes.Resolve(post, note, token);
if (parent != default) if (parent != default && Domain.Privacy.Counted.Reply(post))
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
await _fanout.Distribute(post, token); await _fanout.Distribute(post, token);
if (conversation != default) if (conversation != default)
@@ -71,6 +71,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
await DB.Default.DeleteAsync<Following>(f => f.TargetActorURI == actor.ActorURI); await DB.Default.DeleteAsync<Following>(f => f.TargetActorURI == actor.ActorURI);
await DB.Default.DeleteAsync<TimelineEntry>(e => e.AuthorAccountId == actor.ID); await DB.Default.DeleteAsync<TimelineEntry>(e => e.AuthorAccountId == actor.ID);
await DB.Default.Update<PostEntity>().Match(p => p.ActorURI == actor.ActorURI).Modify(p => p.AuthorGone, true).ExecuteAsync(token); await DB.Default.Update<PostEntity>().Match(p => p.ActorURI == actor.ActorURI).Modify(p => p.AuthorGone, true).ExecuteAsync(token);
await GoneActors.Forget(actor, token);
return; return;
} }
+1 -1
View File
@@ -104,7 +104,7 @@ namespace PrivaPub.Federation.Inbox
await DB.Default.DeleteAsync<ObjectRecord>(r => r.PostId == post.ID || r.ObjectURI == objectUri); await DB.Default.DeleteAsync<ObjectRecord>(r => r.PostId == post.ID || r.ObjectURI == objectUri);
await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID); await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID);
await DB.Default.DeleteAsync<PostEntity>(p => p.ReblogOfPostId == post.ID); await DB.Default.DeleteAsync<PostEntity>(p => p.ReblogOfPostId == post.ID);
if (!string.IsNullOrEmpty(post.AnsweringToPostId)) if (!string.IsNullOrEmpty(post.AnsweringToPostId) && Domain.Privacy.Counted.Reply(post))
await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token);
if (post.QuoteState == QuoteState.Accepted && !string.IsNullOrEmpty(post.QuotedPostId)) if (post.QuoteState == QuoteState.Accepted && !string.IsNullOrEmpty(post.QuotedPostId))
await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token); await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
+29 -1
View File
@@ -1,5 +1,8 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering; using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Text; using System.Text;
@@ -14,6 +17,26 @@ namespace PrivaPub.Federation.Outbox
public class GroupDistributor : IGroupDistributor public class GroupDistributor : IGroupDistributor
{ {
// so the announce's id resolves (PeasantsController.Grunt); a repeat of the same announce keeps the first
static async Task Keep(LocalActor group, JsonObject announce, string objectUri, CancellationToken token)
{
var body = (JsonObject)announce.DeepClone();
body.Remove("@context");
try
{
await DB.Default.SaveAsync(new GroupAnnouncement
{
GroupId = group.Id,
ActivityURI = body["id"]!.GetValue<string>(),
ObjectURI = objectUri,
Body = body.ToJsonString()
}, token);
}
catch (MongoDB.Driver.MongoWriteException ex) when (ex.WriteError?.Category == MongoDB.Driver.ServerErrorCategory.DuplicateKey)
{
}
}
readonly IDeliveryService _delivery; readonly IDeliveryService _delivery;
public GroupDistributor(IDeliveryService delivery) public GroupDistributor(IDeliveryService delivery)
@@ -43,8 +66,13 @@ namespace PrivaPub.Federation.Outbox
// Every follower, the author's own server included: Lemmy 1.0 keeps its user's post pending until the community // Every follower, the author's own server included: Lemmy 1.0 keeps its user's post pending until the community
// announces it back, and clears that before it answers the echo 400 ("Object is not remote"). // announces it back, and clears that before it answers the echo 400 ("Object is not remote").
await _delivery.EnqueueToFollowers(group, announce, token); await _delivery.EnqueueToFollowers(group, announce, token);
await Keep(group, announce, objectUri, token);
if (isNewPost && !string.IsNullOrEmpty(objectUri)) if (isNewPost && !string.IsNullOrEmpty(objectUri))
await _delivery.EnqueueToFollowers(group, ActivityPubRenderer.Announce(group, objectUri, $"announce-object-{key}"), token); {
var boost = ActivityPubRenderer.Announce(group, objectUri, $"announce-object-{key}");
await _delivery.EnqueueToFollowers(group, boost, token);
await Keep(group, boost, objectUri, token);
}
} }
} }
} }
+60 -4
View File
@@ -2,6 +2,7 @@ using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering; using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation; using PrivaPub.Models.Federation;
using PrivaPub.Models.Post; using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices; using PrivaPub.StaticServices;
using System.Text.Json.Nodes; using System.Text.Json.Nodes;
@@ -14,6 +15,7 @@ namespace PrivaPub.Federation.Outbox
{ {
Task<IReadOnlyList<string>> Audience(LocalActor author, PostEntity post, CancellationToken token); Task<IReadOnlyList<string>> Audience(LocalActor author, PostEntity post, CancellationToken token);
Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token); Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token);
Task PublishUpdate(LocalActor author, PostEntity post, string reason, CancellationToken token);
Task PublishProfile(LocalActor actor, CancellationToken token); Task PublishProfile(LocalActor actor, CancellationToken token);
} }
@@ -73,25 +75,79 @@ namespace PrivaPub.Federation.Outbox
return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList(); return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList();
} }
async Task<IReadOnlyList<string>> CircleMembers(string groupId, CancellationToken token) async Task<IReadOnlyList<string>> CircleMembers(string groupId, CancellationToken token) =>
(await CircleRecipients(groupId, token)).Select(r => r.InboxURL).Distinct(StringComparer.Ordinal).ToList();
async Task<IReadOnlyList<ForeignAvatar>> CircleRecipients(string groupId, CancellationToken token)
{ {
var circle = string.IsNullOrEmpty(groupId) ? default : await _dbEntities.Groups.MatchID(groupId).ExecuteFirstAsync(token); var circle = string.IsNullOrEmpty(groupId) ? default : await _dbEntities.Groups.MatchID(groupId).ExecuteFirstAsync(token);
if (circle == default) if (circle == default)
return Array.Empty<string>(); return Array.Empty<ForeignAvatar>();
var remote = circle.Members.Where(m => m.IsForeign).Select(m => m.AvatarId).ToList(); var remote = circle.Members.Where(m => m.IsForeign).Select(m => m.AvatarId).ToList();
if (remote.Count == 0) if (remote.Count == 0)
return Array.Empty<string>(); return Array.Empty<ForeignAvatar>();
return (await _dbEntities.ForeignAvatars.Match(a => remote.Contains(a.ActorURI)).ExecuteAsync(token)) return (await _dbEntities.ForeignAvatars.Match(a => remote.Contains(a.ActorURI)).ExecuteAsync(token))
.Select(a => a.InboxURL).Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList(); .Where(a => !string.IsNullOrEmpty(a.InboxURL))
.ToList();
} }
public async Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token) public async Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token)
{ {
if (post.Visibility == PostVisibility.Circle)
{
foreach (var member in await CircleRecipients(post.GroupId, token))
await _delivery.Enqueue(author, new[] { member.InboxURL }, Naming(activity, new[] { member }), token);
return;
}
var inboxes = await Audience(author, post, token); var inboxes = await Audience(author, post, token);
if (inboxes.Count > 0) if (inboxes.Count > 0)
await _delivery.Enqueue(author, inboxes, activity, token); await _delivery.Enqueue(author, inboxes, activity, token);
} }
public async Task PublishUpdate(LocalActor author, PostEntity post, string reason, CancellationToken token)
{
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
await Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, group, reason), token);
}
// Mastodon keeps a post only when it names one of its own accounts, and GoToSocial shows a post that is neither
// public nor for followers only to the accounts it mentions; a circle post names only the circle. So each member's
// copy, or a member's refetch, also names that member in cc and mentions them, silently, in its tags (owner decision
// 2026-10-04): it tells each member nothing but that they are in the circle.
public static JsonObject Naming(JsonObject activityOrObject, IEnumerable<ForeignAvatar> members)
{
var named = members.ToList();
var copy = (JsonObject)activityOrObject.DeepClone();
if (copy["object"] is JsonObject inner)
{
Name(copy, named, mention: false);
if (inner.ContainsKey("to"))
Name(inner, named, mention: true);
}
else
Name(copy, named, mention: copy.ContainsKey("attributedTo"));
return copy;
}
static void Name(JsonObject node, IReadOnlyList<ForeignAvatar> members, bool mention)
{
var cc = node["cc"] as JsonArray ?? new JsonArray();
var tags = node["tag"] as JsonArray ?? new JsonArray();
foreach (var member in members)
{
if (!cc.Any(c => c?.GetValue<string>() == member.ActorURI))
cc.Add(member.ActorURI);
if (mention && !tags.Any(t => t?["href"]?.GetValue<string>() == member.ActorURI))
tags.Add(new JsonObject { ["type"] = "Mention", ["href"] = member.ActorURI, ["name"] = Handle(member) });
}
node["cc"] = cc;
if (mention)
node["tag"] = tags;
}
static string Handle(ForeignAvatar member) =>
string.IsNullOrEmpty(member.UserName) ? member.ActorURI : $"@{member.UserName}@{new Uri(member.ActorURI).Authority}";
public async Task PublishProfile(LocalActor actor, CancellationToken token) public async Task PublishProfile(LocalActor actor, CancellationToken token)
{ {
var document = ActivityPubRenderer.Actor(actor); var document = ActivityPubRenderer.Actor(actor);
@@ -178,11 +178,11 @@ namespace PrivaPub.Federation.Rendering
return note; return note;
} }
public static JsonObject UpdateOf(PostEntity post, LocalActor author, string reason) public static JsonObject UpdateOf(PostEntity post, LocalActor author, LocalActor group, string reason)
{ {
var note = post.Visibility == PostVisibility.Direct var note = post.Visibility == PostVisibility.Direct
? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI) ? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI)
: Note(post, author, default, post.InReplyToURI); : Note(post, author, group, post.InReplyToURI);
note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()); note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()); note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
return new JsonObject return new JsonObject
@@ -357,13 +357,13 @@ namespace PrivaPub.Federation.Rendering
["object"] = note ["object"] = note
}; };
public static JsonObject Announce(LocalActor group, string objectUri, string activityId) => new() public static JsonObject Announce(LocalActor group, string objectUri, string activityId, DateTime? published = default) => new()
{ {
["@context"] = ActivityStreams, ["@context"] = ActivityStreams,
["id"] = group.ActivityUri(activityId), ["id"] = group.ActivityUri(activityId),
["type"] = "Announce", ["type"] = "Announce",
["actor"] = group.Uri, ["actor"] = group.Uri,
["published"] = Timestamp(DateTime.UtcNow), ["published"] = Timestamp(published ?? DateTime.UtcNow),
["to"] = new JsonArray(Public), ["to"] = new JsonArray(Public),
["cc"] = new JsonArray(group.Followers), ["cc"] = new JsonArray(group.Followers),
["object"] = objectUri ["object"] = objectUri
@@ -1,23 +1,34 @@
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects; using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.User; using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using GroupEntity = PrivaPub.Models.Group.Group; using GroupEntity = PrivaPub.Models.Group.Group;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Signing namespace PrivaPub.Federation.Signing
{ {
public interface ISignedFetchAuthorizer public interface ISignedFetchAuthorizer
{ {
Task<ForeignAvatar> Requester(HttpRequest request, CancellationToken token); Task<ForeignAvatar> Requester(HttpRequest request, CancellationToken token);
Task<bool> MayRead(PostEntity post, ForeignAvatar requester, CancellationToken token);
} }
// Who may refetch a post that is not public (owner decision 2026-10-04). A server that received a followers-only post,
// a DM or a circle post refetches it, signed by the account it was for or by its instance actor, and Mastodon deletes its
// copy when the refetch answers 404. So the post is served to a signed request from someone it was for, or from the
// instance actor of a server where someone it was for lives; everyone else still gets 404.
public class SignedFetchAuthorizer : ISignedFetchAuthorizer public class SignedFetchAuthorizer : ISignedFetchAuthorizer
{ {
readonly IRemoteActorService _remoteActors; readonly IRemoteActorService _remoteActors;
readonly DbEntities _dbEntities;
public SignedFetchAuthorizer(IRemoteActorService remoteActors) public SignedFetchAuthorizer(IRemoteActorService remoteActors, DbEntities dbEntities)
{ {
_remoteActors = remoteActors; _remoteActors = remoteActors;
_dbEntities = dbEntities;
} }
public async Task<ForeignAvatar> Requester(HttpRequest request, CancellationToken token) public async Task<ForeignAvatar> Requester(HttpRequest request, CancellationToken token)
@@ -33,8 +44,48 @@ namespace PrivaPub.Federation.Signing
return actor != default && HttpSignatures.Verify(actor.PublicKey, signingString, parameters.Signature) ? actor : default; return actor != default && HttpSignatures.Verify(actor.PublicKey, signingString, parameters.Signature) ? actor : default;
} }
public async Task<bool> MayRead(PostEntity post, ForeignAvatar requester, CancellationToken token)
{
if (post == default || requester == default)
return false;
switch (post.Visibility)
{
case PostVisibility.Public or PostVisibility.Unlisted:
return true;
case PostVisibility.Circle:
var circle = string.IsNullOrEmpty(post.GroupId) ? default : await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token);
return circle != default && MayReadCircle(circle, requester);
case PostVisibility.Direct:
return Addressed(post).Any(uri => Is(requester, uri));
case PostVisibility.FollowersOnly:
if (Addressed(post).Any(uri => Is(requester, uri)))
return true;
var followers = await _dbEntities.Followers
.Match(f => f.LocalActorId == post.GroupUserId && f.LocalActorKind == LocalActorKind.Person && f.IsAccepted)
.ExecuteAsync(token);
return followers.Any(f => Is(requester, f.ActorURI));
default:
return false;
}
}
static IEnumerable<string> Addressed(PostEntity post) =>
post.To.Concat(post.Cc).Concat(post.Mentions.Select(m => m.ActorURI)).Where(uri => !string.IsNullOrEmpty(uri));
// the account itself, or the instance actor of the server it lives on
static bool Is(ForeignAvatar requester, string actorUri) =>
actorUri == requester.ActorURI || requester.AvatarType == AvatarType.Application && Origin.Same(actorUri, requester.ActorURI);
public static bool MayReadCircle(GroupEntity circle, ForeignAvatar requester) => public static bool MayReadCircle(GroupEntity circle, ForeignAvatar requester) =>
requester != default && circle.Members.Any(m => m.IsForeign requester != default && circle.Members.Any(m => m.IsForeign && Is(requester, m.AvatarId));
&& (m.AvatarId == requester.ActorURI || requester.AvatarType == AvatarType.Application && Origin.Same(m.AvatarId, requester.ActorURI)));
public static bool MayReadConversation(Models.Group.DmGroup conversation, ForeignAvatar requester) =>
requester != default && conversation.Members.Any(m => m.IsForeign && Is(requester, m.AvatarId));
// the members a refetch names in cc: the requesting member, or the members on an instance actor's server
public static IReadOnlyList<string> CircleReaders(GroupEntity circle, ForeignAvatar requester) =>
requester == default
? Array.Empty<string>()
: circle.Members.Where(m => m.IsForeign && Is(requester, m.AvatarId)).Select(m => m.AvatarId).ToList();
} }
} }
+114 -7
View File
@@ -1,22 +1,49 @@
using MongoDB.Entities; using MongoDB.Entities;
using PrivaPub.ClientModels; using PrivaPub.ClientModels;
using PrivaPub.ClientModels.User;
using PrivaPub.ClientModels.User.Avatar;
using PrivaPub.Models.User; using PrivaPub.Models.User;
using PrivaPub.Services;
using PrivaPub.Services.ClientToServer.Private;
using PrivaPub.StaticServices;
using System.ComponentModel.DataAnnotations;
using System.Security.Cryptography;
namespace PrivaPub.Infrastructure.Cli namespace PrivaPub.Infrastructure.Cli
{ {
// `PrivaPub admin ...` runs with the whole server built but not started: no Kestrel, no hosted services.
public static class AdminCommands public static class AdminCommands
{ {
const string Usage = "usage: PrivaPub admin promote|demote <root username>"; public const string SmokeLogin = "deploy-smoke";
public static async Task<int> Run(string[] args) const string Usage = """
usage: PrivaPub admin promote|demote <root>
PrivaPub admin create-root <login> [--admin] the password is read from standard input
PrivaPub admin smoke <persona> prints "<login> <password>" for the deploy's signed-in check
""";
public static async Task<int> Run(string[] args, IServiceProvider services, TextReader input = default, TextWriter output = default)
{ {
if (args is not [("promote" or "demote") and var verb, var userName]) input ??= Console.In;
output ??= Console.Out;
switch (args)
{ {
Console.Error.WriteLine(Usage); case [("promote" or "demote") and var verb, var userName]:
return 2; return await Promote(verb == "promote", userName, output);
case ["create-root", var login, .. var flags] when flags.All(f => f == "--admin"):
return await CreateRoot(services, login, input.ReadLine(), flags.Contains("--admin"), output);
case ["smoke", var persona]:
return await Smoke(services, persona, output);
default:
Console.Error.WriteLine(Usage);
return 2;
} }
}
static async Task<int> Promote(bool promote, string userName, TextWriter output)
{
userName = userName.ToLowerInvariant(); userName = userName.ToLowerInvariant();
var user = await DB.Default.Find<RootUser>().Match(u => u.UserName == userName).ExecuteFirstAsync(); var user = await DB.Default.Find<RootUser>().Match(u => u.UserName == userName).ExecuteFirstAsync();
if (user == default) if (user == default)
@@ -26,14 +53,94 @@ namespace PrivaPub.Infrastructure.Cli
} }
user.Policies.RemoveAll(p => p is Policies.IsAdmin or Policies.IsModerator); user.Policies.RemoveAll(p => p is Policies.IsAdmin or Policies.IsModerator);
if (verb == "promote") if (promote)
user.Policies.AddRange(new[] { Policies.IsAdmin, Policies.IsModerator }); user.Policies.AddRange(new[] { Policies.IsAdmin, Policies.IsModerator });
if (!user.Policies.Contains(Policies.IsUser)) if (!user.Policies.Contains(Policies.IsUser))
user.Policies.Add(Policies.IsUser); user.Policies.Add(Policies.IsUser);
user.UpdatedAt = DateTime.UtcNow; user.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(user); await DB.Default.SaveAsync(user);
Console.WriteLine($"{userName}: {string.Join(", ", user.Policies)}"); output.WriteLine($"{userName}: {string.Join(", ", user.Policies)}");
return 0;
}
// With sign-up closed this is how the first root is made; group invitations make the rest.
static async Task<int> CreateRoot(IServiceProvider services, string login, string password, bool admin, TextWriter output)
{
var form = new LoginForm { UserName = login, Password = password?.Trim() };
var problems = new List<ValidationResult>();
if (!Validator.TryValidateObject(form, new ValidationContext(form), problems, validateAllProperties: true))
{
Console.Error.WriteLine(string.Join(Environment.NewLine, problems.Select(p => p.ErrorMessage)));
return 1;
}
var created = await services.GetRequiredService<IRootUsersService>().SignUpAsync(form);
if (!created.IsValid)
{
Console.Error.WriteLine(created.ErrorMessage);
return 1;
}
return admin ? await Promote(true, login, output) : await Report(login, output);
}
static Task<int> Report(string login, TextWriter output)
{
output.WriteLine($"{login.ToLowerInvariant()}: created");
return Task.FromResult(0);
}
// The deploy's signed-in smoke check: a root nobody signs in to by hand, owning one undiscoverable persona. Every run
// sets a new password and prints it, so no secret is kept anywhere and nothing waits on a person.
static async Task<int> Smoke(IServiceProvider services, string personaName, TextWriter output)
{
personaName = personaName.ToLowerInvariant();
var password = "Smoke-x" + Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(24));
var root = await DB.Default.Find<RootUser>().Match(u => u.UserName == SmokeLogin).ExecuteFirstAsync();
if (root == default)
{
var created = await services.GetRequiredService<IRootUsersService>().SignUpAsync(new LoginForm { UserName = SmokeLogin, Password = password });
if (!created.IsValid)
{
Console.Error.WriteLine(created.ErrorMessage);
return 1;
}
root = await DB.Default.Find<RootUser>().Match(u => u.UserName == SmokeLogin).ExecuteFirstAsync();
}
else if (root.DeletedAt.HasValue || root.IsBanned)
{
Console.Error.WriteLine($"the root '{SmokeLogin}' is deleted or banned");
return 1;
}
else
await DB.Default.Update<RootUser>().MatchID(root.ID)
.Modify(u => u.HashedPassword, services.GetRequiredService<IPasswordHasher>().Hash(password))
.Modify(u => u.UpdatedAt, DateTime.UtcNow)
.ExecuteAsync();
var owned = (await DB.Default.Find<RootToAvatar>().Match(ra => ra.RootId == root.ID).ExecuteAsync()).Select(ra => ra.AvatarId).ToList();
var persona = await DB.Default.Find<Avatar>().Match(a => owned.Contains(a.ID) && a.UserName == personaName && !a.DeletionAt.HasValue).ExecuteFirstAsync();
if (persona == default)
{
var inserted = await services.GetRequiredService<IPrivateAvatarUsersService>().InsertAvatar(new InsertAvatarForm
{
RootId = root.ID,
UserName = personaName,
Name = personaName,
Biography = "The deploy signs in here to check that the Mastodon API works for a signed-in persona."
});
if (!inserted.IsValid)
{
Console.Error.WriteLine(inserted.ErrorMessage);
return 1;
}
persona = await DB.Default.Find<Avatar>().MatchID(((ViewAvatar)inserted.Data).Id).ExecuteFirstAsync();
}
await DB.Default.Update<Avatar>().MatchID(persona.ID)
.Modify(a => a.Settings.IsDiscoverable, false)
.Modify(a => a.Settings.IsIndexable, false)
.ExecuteAsync();
output.WriteLine($"{SmokeLogin} {password}");
return 0; return 0;
} }
} }
+1
View File
@@ -54,6 +54,7 @@ namespace PrivaPub.Infrastructure.Data
await Plain<Group>(token, g => g.InvitationCode); await Plain<Group>(token, g => g.InvitationCode);
await Plain<DmGroup>(token, g => g.ParticipantsKey); await Plain<DmGroup>(token, g => g.ParticipantsKey);
await Unique<GroupAnnouncement>(a => a.ActivityURI, Builders<GroupAnnouncement>.Filter.Type(a => a.ActivityURI, BsonType.String), token);
await Plain<Delivery>(token, d => d.DeliveredAt, d => d.AbandonedAt, d => d.NextAttemptAt); await Plain<Delivery>(token, d => d.DeliveredAt, d => d.AbandonedAt, d => d.NextAttemptAt);
@@ -0,0 +1,14 @@
using MongoDB.Entities;
using PrivaPub.Models.User;
namespace PrivaPub.Infrastructure.Data.Migrations
{
// Recovery codes were stored in plain text and never expired; from now on only a hash with a one-hour expiry is kept.
// The old codes are dropped: anyone mid-recovery asks again.
public class _011_recovery_codes_are_hashed : IMigration
{
public async Task UpgradeAsync() =>
await DB.Default.DeleteAsync<EmailRecovery>(r => r.CodeHash == null);
}
}
@@ -0,0 +1,35 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Federation.Inbox;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Infrastructure.Data.Migrations
{
// replies_count now counts only public and unlisted replies, and accounts that deleted themselves leave nothing in the
// counts (GoneActors). Both are recounted once for what was stored before.
public class _012_counts_follow_one_rule : IMigration
{
public async Task UpgradeAsync()
{
var counted = (await DB.Default.Find<PostEntity, string>()
.Match(p => p.AnsweringToPostId != null && !p.DeletedAt.HasValue && !p.AuthorGone
&& (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted))
.Project(p => p.AnsweringToPostId)
.ExecuteAsync())
.GroupBy(id => id)
.ToDictionary(g => g.Key, g => g.Count());
var stale = await DB.Default.Find<PostEntity>().Match(p => p.RepliesCount > 0).Project(p => p.Include(x => x.ID).Include(x => x.RepliesCount)).ExecuteAsync();
foreach (var post in stale.Where(p => counted.GetValueOrDefault(p.ID) != p.RepliesCount))
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.RepliesCount, counted.GetValueOrDefault(post.ID)).ExecuteAsync();
foreach (var (id, count) in counted.Where(c => stale.All(p => p.ID != c.Key)))
await DB.Default.Update<PostEntity>().MatchID(id).Modify(p => p.RepliesCount, count).ExecuteAsync();
foreach (var gone in await DB.Default.Find<ForeignAvatar>().Match(a => a.DeletionAt != null && a.ForgottenAt == null).ExecuteAsync())
await GoneActors.Forget(gone, CancellationToken.None);
}
}
}
+19 -12
View File
@@ -25,8 +25,8 @@ namespace PrivaPub.Infrastructure.Geo
public sealed class DbIpLocator : IGeoLocator, IDisposable public sealed class DbIpLocator : IGeoLocator, IDisposable
{ {
public const string CityFile = "dbip-city-lite.mmdb"; public static readonly string CityFile = GeoUpdater.FileOf("city");
public const string AsnFile = "dbip-asn-lite.mmdb"; public static readonly string AsnFile = GeoUpdater.FileOf("asn");
static readonly TimeSpan CheckInterval = TimeSpan.FromMinutes(10); static readonly TimeSpan CheckInterval = TimeSpan.FromMinutes(10);
readonly IOptionsMonitor<StatisticsOptions> _options; readonly IOptionsMonitor<StatisticsOptions> _options;
@@ -60,18 +60,17 @@ namespace PrivaPub.Infrastructure.Geo
if (address == default || !IpRangeGuard.IsPublic(address)) if (address == default || !IpRangeGuard.IsPublic(address))
return default; return default;
Refresh(); Refresh();
Reader city, asn; Dictionary<string, object> place, network;
lock (_lock)
{
city = _city;
asn = _asn;
}
if (city == default && asn == default)
return default;
try try
{ {
var place = city?.Find<Dictionary<string, object>>(address); // under the lock, so a reload cannot dispose a reader in the middle of a lookup; lookups are rare
var network = asn?.Find<Dictionary<string, object>>(address); lock (_lock)
{
if (_city == default && _asn == default)
return default;
place = _city?.Find<Dictionary<string, object>>(address);
network = _asn?.Find<Dictionary<string, object>>(address);
}
var location = Section(place, "location"); var location = Section(place, "location");
return new GeoFix( return new GeoFix(
Text(Section(place, "country"), "iso_code"), Text(Section(place, "country"), "iso_code"),
@@ -88,6 +87,14 @@ namespace PrivaPub.Infrastructure.Geo
} }
} }
// Looks at the files again now instead of within the next ten minutes: GeoUpdater has just replaced one.
public void Reload()
{
lock (_lock)
_checkedAt = DateTime.MinValue;
Refresh();
}
void Refresh() void Refresh()
{ {
if (DateTime.UtcNow - _checkedAt < CheckInterval) if (DateTime.UtcNow - _checkedAt < CheckInterval)
+163
View File
@@ -0,0 +1,163 @@
using MaxMind.Db;
using Microsoft.Extensions.Options;
using PrivaPub.Infrastructure.Statistics;
using System.IO.Compression;
using System.Net;
namespace PrivaPub.Infrastructure.Geo
{
// Keeps the DB-IP Lite databases current by itself, so a deploy needs no timer and no root: once a day it checks
// whether each database was built this month and, if not, fetches this month's (or, early in the month, last month's),
// checks it opens as the right kind of database and swaps it in. This and SMTP are the server's only traffic that is not
// federation, which is why it has its own client instead of IFederationHttp.
public sealed class GeoUpdater : BackgroundService
{
public const string ClientName = "geo";
static readonly string[] Kinds = { "city", "asn" };
static readonly TimeSpan FirstWait = TimeSpan.FromSeconds(30);
static readonly TimeSpan Interval = TimeSpan.FromHours(24);
readonly IHttpClientFactory _http;
readonly IOptionsMonitor<StatisticsOptions> _options;
readonly IGeoLocator _locator;
readonly ILogger<GeoUpdater> _logger;
public GeoUpdater(IHttpClientFactory http, IOptionsMonitor<StatisticsOptions> options, IGeoLocator locator, ILogger<GeoUpdater> logger)
{
_http = http;
_options = options;
_locator = locator;
_logger = logger;
}
public static string FileOf(string kind) => $"dbip-{kind}-lite.mmdb";
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
try
{
await Task.Delay(FirstWait, stoppingToken);
while (!stoppingToken.IsCancellationRequested)
{
if (_options.CurrentValue.Geo.AutoUpdate)
await Update(DateTime.UtcNow, stoppingToken);
await Task.Delay(Interval + TimeSpan.FromMinutes(Random.Shared.Next(60)), stoppingToken);
}
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
}
}
// The number of databases replaced. A failure keeps the database already there and is tried again the next day.
public async Task<int> Update(DateTime now, CancellationToken token)
{
var options = _options.CurrentValue;
var replaced = 0;
try
{
Directory.CreateDirectory(options.GeoDirectory);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
_logger.LogWarning(ex, "Cannot create {Directory}; servers are not located", options.GeoDirectory);
return 0;
}
foreach (var kind in Kinds)
{
var path = Path.Combine(options.GeoDirectory, FileOf(kind));
var built = BuiltMonth(path);
foreach (var month in new[] { Month(now), Month(now).AddMonths(-1) })
{
if (built >= month)
break;
try
{
if (await Install(kind, month, path, options, token))
{
replaced++;
_logger.LogInformation("Installed DB-IP Lite {Kind} {Month:yyyy-MM}", kind, month);
break;
}
}
catch (Exception ex) when (ex is HttpRequestException or IOException or InvalidDatabaseException or InvalidDataException
or TaskCanceledException && !token.IsCancellationRequested)
{
_logger.LogWarning(ex, "Could not install DB-IP Lite {Kind} {Month:yyyy-MM}", kind, month);
break;
}
}
}
if (replaced > 0 && _locator is DbIpLocator locator)
locator.Reload();
return replaced;
}
static DateTime Month(DateTime at) => new(at.Year, at.Month, 1, 0, 0, 0, DateTimeKind.Utc);
static DateTime BuiltMonth(string path)
{
if (!File.Exists(path))
return DateTime.MinValue;
try
{
using var reader = new Reader(path, FileAccessMode.MemoryMapped);
return Month(reader.Metadata.BuildDate);
}
catch (Exception ex) when (ex is InvalidDatabaseException or IOException)
{
return DateTime.MinValue;
}
}
// False when that month is not published (yet); throws when it is published but cannot be used.
async Task<bool> Install(string kind, DateTime month, string path, StatisticsOptions options, CancellationToken token)
{
var url = $"{options.Geo.DownloadBase.TrimEnd('/')}/dbip-{kind}-lite-{month:yyyy-MM}.mmdb.gz";
var packed = path + ".gz.part";
var unpacked = path + ".new";
try
{
using (var response = await _http.CreateClient(ClientName).GetAsync(url, HttpCompletionOption.ResponseHeadersRead, token))
{
if (response.StatusCode == HttpStatusCode.NotFound)
return false;
response.EnsureSuccessStatusCode();
await using var source = await response.Content.ReadAsStreamAsync(token);
await using var target = File.Create(packed);
await CopyCapped(source, target, options.Geo.MaxDownloadBytes, token);
}
await using (var source = new GZipStream(File.OpenRead(packed), CompressionMode.Decompress))
await using (var target = File.Create(unpacked))
await CopyCapped(source, target, options.Geo.MaxDatabaseBytes, token);
using (var reader = new Reader(unpacked, FileAccessMode.MemoryMapped))
if (!reader.Metadata.DatabaseType.Contains(kind, StringComparison.OrdinalIgnoreCase))
throw new InvalidDatabaseException($"{url} is a {reader.Metadata.DatabaseType} database, not {kind}");
File.Move(unpacked, path, overwrite: true);
return true;
}
finally
{
File.Delete(packed);
File.Delete(unpacked);
}
}
static async Task CopyCapped(Stream source, Stream target, long limit, CancellationToken token)
{
var buffer = new byte[81920];
long total = 0;
int read;
while ((read = await source.ReadAsync(buffer, token)) > 0)
{
total += read;
if (total > limit)
throw new InvalidDataException($"more than {limit} bytes");
await target.WriteAsync(buffer.AsMemory(0, read), token);
}
}
}
}
@@ -0,0 +1,19 @@
namespace PrivaPub.Infrastructure
{
public enum RegistrationMode
{
Invitations,
Open
}
// Who may make a root login. NodeInfo, both instance API versions and the sign-up endpoint read this one switch, so they
// cannot disagree. A group invitation always creates an account, whatever the mode.
public class RegistrationOptions
{
public RegistrationMode Mode { get; set; } = RegistrationMode.Invitations;//owner decision (2026-10-04): invitations only
public bool IsOpen => Mode == RegistrationMode.Open;
public const string ClosedMessage = "Sign-up is by invitation only: ask someone here for a group invitation.";
}
}
@@ -6,9 +6,19 @@ namespace PrivaPub.Infrastructure.Statistics
public string GeoDirectory { get; set; } = "/var/lib/privapub/geo"; public string GeoDirectory { get; set; } = "/var/lib/privapub/geo";
public int PublicCityMinUsers { get; set; } = 10; public int PublicCityMinUsers { get; set; } = 10;
public CrawlerOptions Crawler { get; set; } = new(); public CrawlerOptions Crawler { get; set; } = new();
public GeoOptions Geo { get; set; } = new();
} }
//owner decision: off by default; when on, one server a minute, each at most weekly, at most MaxHosts servers //owner decision (2026-10-04): the server keeps its DB-IP Lite databases current by itself
public class GeoOptions
{
public bool AutoUpdate { get; set; } = true;
public string DownloadBase { get; set; } = "https://download.db-ip.com/free";
public long MaxDownloadBytes { get; set; } = 300L * 1024 * 1024;
public long MaxDatabaseBytes { get; set; } = 1024L * 1024 * 1024;
}
//owner decision: off by default (on in production since 2026-10-04); when on, one server a minute, each at most weekly, at most MaxHosts servers
public class CrawlerOptions public class CrawlerOptions
{ {
public bool Enabled { get; set; } public bool Enabled { get; set; }
+18 -3
View File
@@ -41,7 +41,8 @@ namespace PrivaPub.Middleware
{ {
return service return service
.Configure<MongoSettings>(configuration.GetSection(nameof(MongoSettings))) .Configure<MongoSettings>(configuration.GetSection(nameof(MongoSettings)))
.Configure<AppConfiguration>(configuration.GetSection(nameof(AppConfiguration))); .Configure<AppConfiguration>(configuration.GetSection(nameof(AppConfiguration)))
.Configure<PrivaPub.Infrastructure.RegistrationOptions>(configuration.GetSection("Registrations"));
} }
public static IServiceCollection PrivaPubWorkersConfiguration(this IServiceCollection service) public static IServiceCollection PrivaPubWorkersConfiguration(this IServiceCollection service)
{ {
@@ -98,6 +99,8 @@ namespace PrivaPub.Middleware
.AddSingleton<IPollService, PollService>() .AddSingleton<IPollService, PollService>()
.AddSingleton<IQuoteService, QuoteService>() .AddSingleton<IQuoteService, QuoteService>()
.AddSingleton<IJobHandler, PollRefreshJob>() .AddSingleton<IJobHandler, PollRefreshJob>()
.AddSingleton<IJobHandler, RecoveryJob>()
.AddSingleton<IJobHandler, Federation.Actors.AccountCountsJob>()
.AddSingleton<IJobHandler, PollCloseJob>() .AddSingleton<IJobHandler, PollCloseJob>()
.AddSingleton<InstanceDescriber>() .AddSingleton<InstanceDescriber>()
.AddSingleton<IJobHandler>(services => services.GetRequiredService<InstanceDescriber>()) .AddSingleton<IJobHandler>(services => services.GetRequiredService<InstanceDescriber>())
@@ -107,8 +110,16 @@ namespace PrivaPub.Middleware
.AddSingleton<IJobHandler, DeliveryJobHandler>() .AddSingleton<IJobHandler, DeliveryJobHandler>()
.AddHostedService<JobWorker>(); .AddHostedService<JobWorker>();
} }
public static IServiceCollection PrivaPubStatisticsConfiguration(this IServiceCollection service, IConfiguration configuration) => public static IServiceCollection PrivaPubStatisticsConfiguration(this IServiceCollection service, IConfiguration configuration)
service {
// not federation: the DB-IP download only (GeoUpdater), plain HTTPS to a fixed host
service.AddHttpClient(GeoUpdater.ClientName, (provider, client) =>
{
var baseAddress = provider.GetRequiredService<IOptionsMonitor<AppConfiguration>>().CurrentValue.BackendBaseAddress?.TrimEnd('/');
client.Timeout = TimeSpan.FromMinutes(15);
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref} (+{baseAddress}/)");
});
return service
.Configure<StatisticsOptions>(configuration.GetSection("Statistics")) .Configure<StatisticsOptions>(configuration.GetSection("Statistics"))
.AddSingleton<InteractionSalts>() .AddSingleton<InteractionSalts>()
.AddSingleton<InteractionLedger>() .AddSingleton<InteractionLedger>()
@@ -116,10 +127,12 @@ namespace PrivaPub.Middleware
.AddHostedService(services => services.GetRequiredService<InteractionLedger>()) .AddHostedService(services => services.GetRequiredService<InteractionLedger>())
.AddSingleton<IJobHandler, RollupJob>() .AddSingleton<IJobHandler, RollupJob>()
.AddSingleton<IGeoLocator, DbIpLocator>() .AddSingleton<IGeoLocator, DbIpLocator>()
.AddHostedService<GeoUpdater>()
.AddSingleton<Domain.Statistics.StatisticsQueries>() .AddSingleton<Domain.Statistics.StatisticsQueries>()
.AddSingleton<IJobHandler, Federation.Crawler.CrawlPlanner>() .AddSingleton<IJobHandler, Federation.Crawler.CrawlPlanner>()
.AddSingleton<IJobHandler, Federation.Crawler.InstanceCrawler>() .AddSingleton<IJobHandler, Federation.Crawler.InstanceCrawler>()
.AddHostedService<StatisticsSchedule>(); .AddHostedService<StatisticsSchedule>();
}
public static IServiceCollection PrivaPubAuthServicesConfiguration(this IServiceCollection service, IConfiguration configuration) public static IServiceCollection PrivaPubAuthServicesConfiguration(this IServiceCollection service, IConfiguration configuration)
{ {
@@ -171,6 +184,8 @@ namespace PrivaPub.Middleware
return service return service
.AddTransient<IDataService, DataService>() .AddTransient<IDataService, DataService>()
.AddTransient<IRootUsersService, RootUsersService>() .AddTransient<IRootUsersService, RootUsersService>()
.AddScoped<IRootSessions, RootSessions>()
.AddScoped<IRootRemoval, RootRemoval>()
.AddTransient<IPublicAvatarUsersService, PublicAvatarUsersService>() .AddTransient<IPublicAvatarUsersService, PublicAvatarUsersService>()
.AddTransient<IPrivateAvatarUsersService, PrivateAvatarUsersService>() .AddTransient<IPrivateAvatarUsersService, PrivateAvatarUsersService>()
.AddTransient<IGroupUsersService, GroupUsersService>() .AddTransient<IGroupUsersService, GroupUsersService>()
@@ -0,0 +1,15 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Federation
{
// An Announce a community sent (GroupDistributor), kept so its id resolves as both docs promise: announces are public,
// unlike follows, likes and blocks. Served only while the post it is about is still shown.
public class GroupAnnouncement : Entity
{
public string GroupId { get; set; }
public string ActivityURI { get; set; }
public string ObjectURI { get; set; }//the post it is about
public string Body { get; set; }//the Announce as sent, without @context
public DateTime PublishedAt { get; set; } = DateTime.UtcNow;
}
}
+3 -1
View File
@@ -29,7 +29,9 @@ namespace PrivaPub.Models.Jobs
FetchPreview, FetchPreview,
RollupDay, RollupDay,
CrawlPlan, CrawlPlan,
CrawlInstance CrawlInstance,
SendRecovery,
CountAccount
} }
public enum JobState public enum JobState
+5
View File
@@ -87,6 +87,11 @@ namespace PrivaPub.Models.User
public DateTime? SuspendedAt { get; set; } public DateTime? SuspendedAt { get; set; }
public DateTime? BannedAt { get; set; } public DateTime? BannedAt { get; set; }
public DateTime? DeletionAt { get; set; } public DateTime? DeletionAt { get; set; }
public DateTime? ForgottenAt { get; set; }//its likes, votes, boosts and replies were taken out of our counts (GoneActors)
public int? FollowersCount { get; set; }//its own collections' totalItems, read at most daily (AccountCountsJob)
public int? FollowingCount { get; set; }
public int? StatusesCount { get; set; }
public DateTime? CountedAt { get; set; }
} }
public class AvatarSettings public class AvatarSettings
+3 -1
View File
@@ -5,7 +5,9 @@ namespace PrivaPub.Models.User
public class EmailRecovery : Entity public class EmailRecovery : Entity
{ {
public string RootUserId { get; set; } public string RootUserId { get; set; }
public string RecoveryCode { get; set; } public string RecoveryCode { get; set; }//plaintext codes before 2026-10-04; migration _011 removed them, never written now
public string CodeHash { get; set; }//sha-256 of the code, hex; the code itself exists only in the email
public DateTime ExpiresAt { get; set; }
public string RequestIP { get; set; } public string RequestIP { get; set; }
public DateTime CreationDate { get; set; } = DateTime.UtcNow; public DateTime CreationDate { get; set; } = DateTime.UtcNow;
} }
+2
View File
@@ -23,6 +23,8 @@ namespace PrivaPub.Models.User
public DateTime CreatedAt { get; set; } = DateTime.UtcNow; public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow; public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
public DateTime? DeletedAt { get; set; } public DateTime? DeletedAt { get; set; }
public DateTime? CredentialsChangedAt { get; set; }//when RootSessions last ended every session
public string SessionStamp { get; set; }//carried by every /clientapi token; a new one ends the tokens issued before
} }
public class ContactItem public class ContactItem
+9 -6
View File
@@ -87,12 +87,6 @@ try
EntityMaps.Warm(); EntityMaps.Warm();
await DB.Default.MigrateAsync<Program>(); await DB.Default.MigrateAsync<Program>();
await Indexes.Create(); await Indexes.Create();
if (args is ["admin", ..])
{
Environment.ExitCode = await AdminCommands.Run(args[1..]);
return;
}
} }
catch (Exception ex) catch (Exception ex)
{ {
@@ -111,6 +105,15 @@ try
throw; throw;
} }
// Commands get every service but start nothing: no Kestrel, no hosted services, no media directory. The deploy runs
// them as its own user, which can read the configuration and reach the private mongod but owns no www-data directory.
if (args is ["admin", ..])
{
using var scope = app.Services.CreateScope();
Environment.ExitCode = await AdminCommands.Run(args[1..], scope.ServiceProvider);
return;
}
try try
{ {
var localizationService = app.Services.GetService<RequestLocalizationOptionsService>(); var localizationService = app.Services.GetService<RequestLocalizationOptionsService>();
@@ -73,7 +73,7 @@ namespace PrivaPub.Services.ClientToServer.Private
if (!await _localActors.TryReserveUserName(userName, LocalActorKind.Person, newAvatar.ID, default)) if (!await _localActors.TryReserveUserName(userName, LocalActorKind.Person, newAvatar.ID, default))
return result.Invalidate(_localizer["The username '{0}' is already take.", userName]); return result.Invalidate(_localizer["The username '{0}' is already take.", userName]);
if (form.Settings is { IsDefault: false }) if (form.Settings is { IsDefault: false })
newAvatar.Settings = ToSettings(form.Settings); Apply(newAvatar.Settings, form.Settings);
var actor = _localActors.FromAvatar(newAvatar); var actor = _localActors.FromAvatar(newAvatar);
newAvatar.Url = actor.Uri; newAvatar.Url = actor.Uri;
@@ -116,7 +116,7 @@ namespace PrivaPub.Services.ClientToServer.Private
avatar.Fields = form.Fields ?? new(); avatar.Fields = form.Fields ?? new();
avatar.PersonalNote = form.PersonalNote; avatar.PersonalNote = form.PersonalNote;
if (form.Settings != default) if (form.Settings != default)
avatar.Settings = ToSettings(form.Settings); Apply(avatar.Settings ??= new(), form.Settings);
avatar.UpdatedAt = DateTime.UtcNow; avatar.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(avatar); await DB.Default.SaveAsync(avatar);
await _outbox.PublishProfile(_localActors.FromAvatar(avatar), default); await _outbox.PublishProfile(_localActors.FromAvatar(avatar), default);
@@ -158,18 +158,20 @@ namespace PrivaPub.Services.ClientToServer.Private
.Match(ru => !ru.DeletedAt.HasValue && !ru.IsBanned) .Match(ru => !ru.DeletedAt.HasValue && !ru.IsBanned)
.ExecuteAnyAsync(); .ExecuteAnyAsync();
static AvatarSettings ToSettings(ViewAvatarSettings settings) => new() // The client API carries only the theme; the rest (discoverable, locked, quote policy…) is set through the Mastodon API
// and must survive an edit made here.
static void Apply(AvatarSettings target, ViewAvatarSettings settings)
{ {
IsDefault = settings.IsDefault, target.IsDefault = settings.IsDefault;
DarkThemeIndexColour = settings.DarkThemeIndexColour, target.DarkThemeIndexColour = settings.DarkThemeIndexColour;
IconsThemeIndexColour = settings.IconsThemeIndexColour, target.IconsThemeIndexColour = settings.IconsThemeIndexColour;
LanguageCode = settings.LanguageCode, target.LanguageCode = settings.LanguageCode;
LightThemeIndexColour = settings.LightThemeIndexColour, target.LightThemeIndexColour = settings.LightThemeIndexColour;
PreferSystemTheming = settings.PreferSystemTheming, target.PreferSystemTheming = settings.PreferSystemTheming;
ThemeIsDarkGray = settings.ThemeIsDarkGray, target.ThemeIsDarkGray = settings.ThemeIsDarkGray;
ThemeIsDarkMode = settings.ThemeIsDarkMode, target.ThemeIsDarkMode = settings.ThemeIsDarkMode;
ThemeIsLightGray = settings.ThemeIsLightGray target.ThemeIsLightGray = settings.ThemeIsLightGray;
}; }
ViewAvatar ToView(Avatar avatar) => new() ViewAvatar ToView(Avatar avatar) => new()
{ {
+34
View File
@@ -210,6 +210,7 @@ namespace PrivaPub.Services
group.UpdatedAt = DateTime.UtcNow; group.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(group, token); await DB.Default.SaveAsync(group, token);
} }
await FollowAsMember(group, form.AvatarId, token);
result.Data = ToView(group, form.AvatarId); result.Data = ToView(group, form.AvatarId);
return result; return result;
@@ -221,6 +222,33 @@ namespace PrivaPub.Services
} }
} }
// Joining by invitation is following, as a local follow of the group would be: the member counts among its
// followers (so /groupies and /flock agree) and gets its posts in the home timeline.
async Task FollowAsMember(GroupEntity group, string avatarId, CancellationToken token)
{
var persona = await _localActors.FindById(LocalActorKind.Person, avatarId, token);
var target = _localActors.FromGroup(group);
if (persona == default)
return;
await DB.Default.Update<Follower>()
.Match(f => f.LocalActorId == group.ID && f.LocalActorKind == LocalActorKind.Group && f.ActorURI == persona.Uri)
.Modify(f => f.InboxURL, persona.Inbox)
.Modify(f => f.SharedInboxURL, persona.SharedInbox)
.Modify(f => f.IsAccepted, true)
.Modify(b => b.SetOnInsert(f => f.CreationDate, DateTime.UtcNow))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
await DB.Default.Update<Models.Social.Following>()
.Match(f => f.AvatarId == persona.Id && f.TargetActorURI == target.Uri)
.Modify(f => f.TargetAccountId, group.ID)
.Modify(f => f.TargetIsLocal, true)
.Modify(f => f.TargetInboxURL, target.Inbox)
.Modify(f => f.State, Models.Social.FollowState.Accepted)
.Modify(b => b.SetOnInsert(f => f.CreatedAt, DateTime.UtcNow))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
}
public async Task<WebResult> LeaveGroup(string rootUserId, GroupMembershipForm form, CancellationToken token) public async Task<WebResult> LeaveGroup(string rootUserId, GroupMembershipForm form, CancellationToken token)
{ {
var result = new WebResult(); var result = new WebResult();
@@ -238,6 +266,12 @@ namespace PrivaPub.Services
group.Members.RemoveAll(m => !m.IsForeign && m.AvatarId == form.AvatarId); group.Members.RemoveAll(m => !m.IsForeign && m.AvatarId == form.AvatarId);
group.UpdatedAt = DateTime.UtcNow; group.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(group, token); await DB.Default.SaveAsync(group, token);
// leaving is unfollowing, as joining was following
var groupUri = _localActors.FromGroup(group).Uri;
var personaUri = (await _localActors.FindById(LocalActorKind.Person, form.AvatarId, token))?.Uri;
await DB.Default.DeleteAsync<Models.Social.Following>(f => f.AvatarId == form.AvatarId && f.TargetActorURI == groupUri);
if (personaUri != default)
await DB.Default.DeleteAsync<Follower>(f => f.LocalActorId == group.ID && f.LocalActorKind == LocalActorKind.Group && f.ActorURI == personaUri);
return result; return result;
} }
catch (Exception ex) catch (Exception ex)
+1
View File
@@ -16,6 +16,7 @@ namespace PrivaPub.Services
Task<WebResult> UnbanUserAsync(UsersIds usersIds); Task<WebResult> UnbanUserAsync(UsersIds usersIds);
Task<WebResult> RemoveUserAsync(UsersIds usersIds); Task<WebResult> RemoveUserAsync(UsersIds usersIds);
Task<WebResult> RemoveSelfAsync(string rootId, string password);
Task<WebResult> UpdateUserAsync(UserForm userEmailForm, string userId); Task<WebResult> UpdateUserAsync(UserForm userEmailForm, string userId);
+6
View File
@@ -33,6 +33,12 @@ namespace PrivaPub.Services
context.Fail("The account can no longer be used."); context.Fail("The account can no longer be used.");
return; return;
} }
// a password recovery ends every session made before it (RootSessions gives the root a new stamp)
if ((root.SessionStamp ?? string.Empty) != (context.Principal.FindFirst(AuthTokenManager.SessionStamp)?.Value ?? string.Empty))
{
context.Fail("The account's sessions were ended.");
return;
}
if (context.Principal.Identity is not ClaimsIdentity identity) if (context.Principal.Identity is not ClaimsIdentity identity)
return; return;
foreach (var policy in PolicyClaims) foreach (var policy in PolicyClaims)
+88
View File
@@ -0,0 +1,88 @@
using MailKit.Net.Smtp;
using Microsoft.Extensions.Localization;
using MimeKit;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using PrivaPub.Resources;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
namespace PrivaPub.Services
{
public sealed record RecoveryPayload(string RootUserId, string Host);
// Sends a password recovery link. Every request queues one of these, for an account that exists or not, so the answer
// to the request and its timing say nothing; SMTP's slowness and failures happen here, where nobody waits on them.
// The code exists only in the email: the database keeps its hash, for an hour.
public class RecoveryJob : IJobHandler
{
public const string Host = "recovery";
public static readonly TimeSpan CodeLifetime = TimeSpan.FromHours(1);
readonly AppConfigurationService _app;
readonly IStringLocalizer<GenericRes> _localizer;
readonly ILogger<RecoveryJob> _logger;
public RecoveryJob(AppConfigurationService app, IStringLocalizer<GenericRes> localizer, ILogger<RecoveryJob> logger)
{
_app = app;
_localizer = localizer;
_logger = logger;
}
public JobKind Kind => JobKind.SendRecovery;
public int Concurrency => 1;
public int MaxAttempts => 3;
public int PerHostLimit => 1;
public static string Hash(string code) => Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(code ?? string.Empty)));
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var payload = JsonSerializer.Deserialize<RecoveryPayload>(job.Payload);
if (string.IsNullOrEmpty(payload?.RootUserId))
return JobOutcome.Done;//asked for an account that does not exist: there is nobody to write to
var user = await DB.Default.Find<RootUser>().MatchID(payload.RootUserId).ExecuteFirstAsync(token);
if (user is not { DeletedAt: null, IsBanned: false } || string.IsNullOrEmpty(user.Email))
return JobOutcome.Done;
var code = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(32));
await DB.Default.DeleteAsync<EmailRecovery>(r => r.RootUserId == user.ID);
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = user.ID, CodeHash = Hash(code), ExpiresAt = DateTime.UtcNow + CodeLifetime }, token);
var email = _app.AppConfiguration.EmailConfiguration;
var message = new MimeMessage();
message.From.Add(new MailboxAddress("PrivaPub", email.SmtpUsername));
message.To.Add(MailboxAddress.Parse(user.Email));
message.Subject = _localizer["PrivaPub - Password recovery link"];
message.Body = new TextPart("plain")
{
Text = string.Format(_localizer[
"Hello,\n\nSomeone asked to reset the password of the PrivaPub login {0}. If it was you, open this link within an hour:\n{1}\n\nIf it was not you, ignore this email: nothing changes."],
user.UserName, $"{payload.Host}/password-recovery?rc={code}")
};
try
{
using var smtp = new SmtpClient();
await smtp.ConnectAsync(email.SmtpServer, email.SmtpPort, email.UseSSL, token);
await smtp.AuthenticateAsync(email.SmtpUsername, email.SmtpPassword, token);
await smtp.SendAsync(message, token);
await smtp.DisconnectAsync(quit: true, token);
return JobOutcome.Done;
}
catch (Exception ex) when (ex is not OperationCanceledException || !token.IsCancellationRequested)
{
_logger.LogWarning(ex, "The recovery email could not be sent");
return JobOutcome.Retry("smtp");
}
}
}
}
+119
View File
@@ -0,0 +1,119 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Services
{
public interface IRootRemoval
{
Task<bool> Remove(string rootId, CancellationToken token);
}
// Deleting a root deletes everything public it had (owner decision 2026-10-04): each persona, and each group a persona
// owns, is announced deleted with a Delete of the actor to everyone who follows it, every member and everyone it
// follows; their posts are emptied; their names stay reserved, and their documents answer 410 from then on. The root's
// sessions end first, so nothing acts as it while it goes.
public class RootRemoval : IRootRemoval
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IDeliveryService _delivery;
readonly IRootSessions _sessions;
public RootRemoval(DbEntities dbEntities, ILocalActorService localActors, IDeliveryService delivery, IRootSessions sessions)
{
_dbEntities = dbEntities;
_localActors = localActors;
_delivery = delivery;
_sessions = sessions;
}
public async Task<bool> Remove(string rootId, CancellationToken token)
{
var root = await _dbEntities.RootUsers.MatchID(rootId).Match(u => u.DeletedAt == null).ExecuteFirstAsync(token);
if (root == default)
return false;
await _sessions.Revoke(root.ID, token);
var now = DateTime.UtcNow;
var avatarIds = (await _dbEntities.RootToAvatars.Match(ra => ra.RootId == root.ID).ExecuteAsync(token)).Select(ra => ra.AvatarId).ToList();
var avatars = await _dbEntities.Avatars.Match(a => avatarIds.Contains(a.ID) && !a.DeletionAt.HasValue).ExecuteAsync(token);
foreach (var avatar in avatars)
{
foreach (var group in await _dbEntities.Groups.Match(g => g.OwnerAvatarId == avatar.ID && !g.DeletionAt.HasValue).ExecuteAsync(token))
{
await Announce(_localActors.FromGroup(group), group.Members.Where(m => m.IsForeign).Select(m => m.AvatarId), token);
await DB.Default.Update<GroupEntity>().MatchID(group.ID).Modify(g => g.DeletionAt, now).ExecuteAsync(token);
}
var persona = _localActors.FromAvatar(avatar);
var followed = (await _dbEntities.Followings.Match(f => f.AvatarId == avatar.ID && !f.TargetIsLocal).ExecuteAsync(token))
.Select(f => f.TargetActorURI);
await Announce(persona, followed, token);
await DB.Default.Update<Avatar>().MatchID(avatar.ID).Modify(a => a.DeletionAt, now).ExecuteAsync(token);
await Empty(avatar.ID, now, token);
}
await DB.Default.Update<RootUser>().MatchID(root.ID)
.Modify(u => u.UserName, $"deleted-{root.ID}")//unique, so a second deletion never collides with the first
.Modify(u => u.Email, null)
.Modify(u => u.HashedPassword, null)
.Modify(u => u.Policies, new List<string>())
.Modify(u => u.IsBanned, false)
.Modify(u => u.IsEmailValidated, false)
.Modify(u => u.DeletedAt, now)
.ExecuteAsync(token);
await DB.Default.DeleteAsync<EmailRecovery>(r => r.RootUserId == root.ID);
return true;
}
// Delete{Actor}: to its followers' (shared) inboxes, and to the inboxes of the other accounts named
async Task Announce(LocalActor actor, IEnumerable<string> others, CancellationToken token)
{
var inboxes = (await _delivery.FollowerInboxes(actor, token)).ToList();
var named = others.Where(uri => !string.IsNullOrEmpty(uri)).Distinct().ToList();
if (named.Count > 0)
inboxes.AddRange((await _dbEntities.ForeignAvatars.Match(a => named.Contains(a.ActorURI)).ExecuteAsync(token))
.Select(a => string.IsNullOrEmpty(a.SharedInboxURL) ? a.InboxURL : a.SharedInboxURL));
await _delivery.Enqueue(actor, inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(), new JsonObject
{
["@context"] = ActivityPubRenderer.Context(),
["id"] = actor.ActivityUri("delete-actor"),
["type"] = "Delete",
["actor"] = actor.Uri,
["object"] = actor.Uri,
["to"] = new JsonArray(ActivityPubRenderer.Public),
["cc"] = new JsonArray(actor.Followers)
}, token);
}
// a persona's posts keep their ids and lose their content, like a single deleted post; a group writes none of its own
static async Task Empty(string avatarId, DateTime now, CancellationToken token)
{
var postIds = (await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == avatarId && !p.IsFederatedCopy && !p.DeletedAt.HasValue)
.Project(p => p.Include(x => x.ID)).ExecuteAsync(token)).Select(p => p.ID).ToList();
if (postIds.Count == 0)
return;
await DB.Default.Update<PostEntity>().Match(p => postIds.Contains(p.ID))
.Modify(p => p.DeletedAt, now)
.Modify(p => p.Text, null)
.Modify(p => p.ContentHtml, null)
.Modify(p => p.Title, null)
.Modify(p => p.SpoilerText, null)
.Modify(p => p.Media, new List<Models.Post.PostMedia>())
.Modify(p => p.Revisions, new List<Models.Post.PostRevision>())
.ExecuteAsync(token);
await DB.Default.DeleteAsync<TimelineEntry>(e => postIds.Contains(e.PostId) || postIds.Contains(e.ReblogOfPostId));
}
}
}
+44
View File
@@ -0,0 +1,44 @@
using MongoDB.Entities;
using OpenIddict.Abstractions;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
namespace PrivaPub.Services
{
public interface IRootSessions
{
Task Revoke(string rootId, CancellationToken token);
}
// Ends everything a root is signed in with: its /clientapi tokens, through a new SessionStamp (checked by JwtEvents),
// and the Mastodon API tokens and authorizations of each of its personas. Used when its password is recovered and when
// it is deleted.
public class RootSessions : IRootSessions
{
readonly DbEntities _dbEntities;
readonly IOpenIddictTokenManager _tokens;
readonly IOpenIddictAuthorizationManager _authorizations;
public RootSessions(DbEntities dbEntities, IOpenIddictTokenManager tokens, IOpenIddictAuthorizationManager authorizations)
{
_dbEntities = dbEntities;
_tokens = tokens;
_authorizations = authorizations;
}
public async Task Revoke(string rootId, CancellationToken token)
{
await DB.Default.Update<RootUser>().MatchID(rootId)
.Modify(u => u.CredentialsChangedAt, DateTime.UtcNow)
.Modify(u => u.SessionStamp, Guid.NewGuid().ToString("N"))
.ExecuteAsync(token);
foreach (var link in await _dbEntities.RootToAvatars.Match(ra => ra.RootId == rootId).ExecuteAsync(token))
{
await _tokens.RevokeBySubjectAsync(link.AvatarId, token);
await _authorizations.RevokeBySubjectAsync(link.AvatarId, token);
}
}
}
}
+78 -181
View File
@@ -14,9 +14,12 @@ using PrivaPub.ClientModels.User;
using PrivaPub.Models; using PrivaPub.Models;
using PrivaPub.Models.User; using PrivaPub.Models.User;
using PrivaPub.Resources; using PrivaPub.Resources;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.StaticServices; using PrivaPub.StaticServices;
using System.Globalization; using System.Globalization;
using System.Text.Json;
#pragma warning disable 8603 #pragma warning disable 8603
#pragma warning disable 8625 #pragma warning disable 8625
@@ -31,8 +34,14 @@ namespace PrivaPub.Services
readonly ILogger<RootUsersService> Logger; readonly ILogger<RootUsersService> Logger;
readonly AppConfigurationService AppConfigurationService; readonly AppConfigurationService AppConfigurationService;
readonly AuthTokenManager AuthTokenManager; readonly AuthTokenManager AuthTokenManager;
readonly IJobQueue Jobs;
readonly IRootSessions Sessions;
readonly IRootRemoval Removal;
public RootUsersService( public RootUsersService(
IJobQueue jobs,
IRootSessions sessions,
IRootRemoval removal,
IStringLocalizer<GenericRes> localizer, IStringLocalizer<GenericRes> localizer,
ILogger<RootUsersService> logger, ILogger<RootUsersService> logger,
IPasswordHasher passwordHasher, IPasswordHasher passwordHasher,
@@ -40,6 +49,9 @@ namespace PrivaPub.Services
AppConfigurationService appConfigurationService, AppConfigurationService appConfigurationService,
AuthTokenManager authTokenManager) AuthTokenManager authTokenManager)
{ {
Jobs = jobs;
Sessions = sessions;
Removal = removal;
DbEntities = dbEntities; DbEntities = dbEntities;
AuthTokenManager = authTokenManager; AuthTokenManager = authTokenManager;
PasswordHasher = passwordHasher; PasswordHasher = passwordHasher;
@@ -48,6 +60,12 @@ namespace PrivaPub.Services
AppConfigurationService = appConfigurationService; AppConfigurationService = appConfigurationService;
} }
public const string NoMatch = "That username and password do not match.";
static string _decoy;
// a hash nobody's password matches, so an unknown login costs the same hashing as a wrong password
string Decoy => _decoy ??= PasswordHasher.Hash(Convert.ToHexString(System.Security.Cryptography.RandomNumberGenerator.GetBytes(16)));
public async Task<WebResult> SignUpAsync(LoginForm signUpForm, string invitationCode = default, public async Task<WebResult> SignUpAsync(LoginForm signUpForm, string invitationCode = default,
bool isPasswordRequired = false) bool isPasswordRequired = false)
{ {
@@ -114,20 +132,16 @@ namespace PrivaPub.Services
var result = new WebResult(); var result = new WebResult();
try try
{ {
// One answer, after the same work, whether the login is unknown, deleted or the password wrong: sign-in must not
// tell anyone which logins exist. Only someone who knows the password learns the login is banned.
loginForm.UserName = loginForm.UserName.ToLower(); loginForm.UserName = loginForm.UserName.ToLower();
if (!await DbEntities.RootUsers.Match(u => u.UserName == loginForm.UserName && u.DeletedAt == null) var user = await DbEntities.RootUsers.Match(u => u.UserName == loginForm.UserName && u.DeletedAt == null).ExecuteFirstAsync();
.ExecuteAnyAsync()) var (verified, needsUpgrade) = PasswordHasher.Check(user?.HashedPassword ?? Decoy, loginForm.Password);
return result.Invalidate(Localizer["Username '{0}' not found.", loginForm.UserName]); if (user?.HashedPassword == default || !verified)
return result.Invalidate(Localizer[NoMatch]);
var user = await DbEntities.RootUsers.Match(u => u.UserName == loginForm.UserName).ExecuteFirstAsync();
if (user.IsBanned) if (user.IsBanned)
return result.Invalidate(Localizer["User '{0}' banned.", user.UserName]); return result.Invalidate(Localizer["User '{0}' banned.", user.UserName]);
var (verified, needsUpgrade) = PasswordHasher.Check(user.HashedPassword, loginForm.Password);
if (!verified)
return result.Invalidate(Localizer["Wrong password."]);
if (needsUpgrade) if (needsUpgrade)
result.ErrorMessage = Localizer["Needs upgrade!"]; result.ErrorMessage = Localizer["Needs upgrade!"];
@@ -261,24 +275,12 @@ namespace PrivaPub.Services
var result = new WebResult(); var result = new WebResult();
try try
{ {
var users = await DbEntities.RootUsers.Match(u => usersIds.UserIdList.Contains(u.ID) && u.DeletedAt == default).ExecuteAsync(); var removed = 0;
if (users == null || users.Count == 0) foreach (var id in usersIds.UserIdList.Distinct())
if (await Removal.Remove(id, CancellationToken.None))
removed++;
if (removed == 0)
return result.Invalidate(Localizer["User already deleted."]); return result.Invalidate(Localizer["User already deleted."]);
foreach (var user in users)
{
user.Email = Localizer["Deleted user"];
user.HashedPassword = null;
user.Policies.Clear();
user.IsBanned = false;
user.IsEmailValidated = false;
//user.TempSecret = null;
user.UserName = Localizer["Deleted user"];
user.DeletedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(user);
}
return result; return result;
} }
catch (Exception ex) catch (Exception ex)
@@ -288,6 +290,26 @@ namespace PrivaPub.Services
} }
} }
// A root deletes itself only with its password, so a stolen session cannot.
public async Task<WebResult> RemoveSelfAsync(string rootId, string password)
{
var result = new WebResult();
try
{
var user = await DbEntities.RootUsers.MatchID(rootId).Match(u => u.DeletedAt == null).ExecuteFirstAsync();
var (verified, _) = PasswordHasher.Check(user?.HashedPassword ?? Decoy, password ?? string.Empty);
if (user?.HashedPassword == default || !verified)
return result.Invalidate(Localizer[NoMatch], StatusCodes.Status403Forbidden);
await Removal.Remove(user.ID, CancellationToken.None);
return result;
}
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(RootUsersService)}.{nameof(RemoveSelfAsync)}()");
return result.Invalidate(Localizer["Something went wrong."], exception: ex);
}
}
public async Task<WebResult> BanUserAsync(UsersIds usersIds) public async Task<WebResult> BanUserAsync(UsersIds usersIds)
{ {
var result = new WebResult(); var result = new WebResult();
@@ -354,141 +376,22 @@ namespace PrivaPub.Services
} }
} }
public const string RecoverySent = "If that account has an email address, a recovery link is on its way. It works for one hour.";
// The same answer, after the same work, whether the account exists, has an email or can be written to: recovery must
// not tell anyone which logins or addresses exist. RecoveryJob does the rest, out of the request.
public async Task<WebResult> SetupAndSendRecoveryEmail(PasswordRecoveryForm passwordRecoveryForm, string host) public async Task<WebResult> SetupAndSendRecoveryEmail(PasswordRecoveryForm passwordRecoveryForm, string host)
{ {
var result = new WebResult(); var result = new WebResult();
try try
{ {
var usernameExists = false; var userName = passwordRecoveryForm.UserName?.ToLowerInvariant();
if (passwordRecoveryForm.IsEmailDisabled) var user = passwordRecoveryForm.IsEmailDisabled
{ ? await DbEntities.RootUsers.Match(u => u.UserName == userName && u.DeletedAt == null).ExecuteFirstAsync()
if (!await DbEntities.RootUsers.Match(u => u.UserName == passwordRecoveryForm.UserName && u.DeletedAt == null) : await DbEntities.RootUsers.Match(u => u.Email == passwordRecoveryForm.Email && u.DeletedAt == null).ExecuteFirstAsync();
.ExecuteAnyAsync()) await Jobs.Enqueue(JobKind.SendRecovery, JsonSerializer.Serialize(new RecoveryPayload(user?.ID, host)), RecoveryJob.Host,
return result.Invalidate(Localizer["Username '{0}' not found.", passwordRecoveryForm.UserName], dedupeKey: default, CancellationToken.None);
StatusCodes.Status404NotFound); result.Data = Localizer[RecoverySent].Value;
usernameExists = true;
}
else
{
if (!await DbEntities.RootUsers.Match(u => u.Email == passwordRecoveryForm.Email && u.DeletedAt == null)
.ExecuteAnyAsync())
return result.Invalidate(Localizer["Username '{0}' not found.", passwordRecoveryForm.UserName],
StatusCodes.Status404NotFound);
}
var user = default(RootUser);
if (usernameExists)
user = await DbEntities.RootUsers.Match(u => u.UserName == passwordRecoveryForm.UserName).ExecuteFirstAsync();
else
user = await DbEntities.RootUsers.Match(u => u.Email == passwordRecoveryForm.Email).ExecuteFirstAsync();
if (string.IsNullOrEmpty(user.Email))
return result.Invalidate(Localizer["This User doesn't have an email, no way to recover."],
StatusCodes.Status423Locked);
var emailRecovery = await DbEntities.EmailRecoveries
.Match(er => er.RootUserId == user.ID).ExecuteFirstAsync();
if (emailRecovery is null)
{
emailRecovery = new()
{
RootUserId = user.ID
};
await DB.Default.SaveAsync(emailRecovery);
}
using var recoveryCodeGenerator = new Password(true, true, true, false, 127);
emailRecovery.RecoveryCode = recoveryCodeGenerator.Next();
await DB.Default.SaveAsync(emailRecovery);
using (var smtpClient = new SmtpClient())
{
try
{
await smtpClient.ConnectAsync(AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpServer, AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpPort,
AppConfigurationService.AppConfiguration.EmailConfiguration.UseSSL);
if (!smtpClient.IsConnected)
{
Logger.LogError($"Failed to connect to the SMTP server({AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpServer}).");
return result.Invalidate(Localizer["Failed to send email."], StatusCodes.Status503ServiceUnavailable);
}
}
catch (Exception ex)
{
Logger.LogError(ex, $"Error at connection to the SMTP server({AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpServer}).");
return result.Invalidate(Localizer["Failed to send email."], StatusCodes.Status503ServiceUnavailable, exception: ex);
}
try
{
await smtpClient.AuthenticateAsync(AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername, AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpPassword);
if (!smtpClient.IsAuthenticated)
{
Logger.LogError($"Failed SMTP authentication of {AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername}.");
return result.Invalidate(Localizer["Failed to send email."],
StatusCodes.Status503ServiceUnavailable);
}
}
catch (Exception ex)
{
Logger.LogError(ex, $"Failed SMTP authentication of {AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername}.");
return result.Invalidate(Localizer["Failed to send email."],
StatusCodes.Status503ServiceUnavailable, exception: ex);
}
try
{
var toParsed = await smtpClient.VerifyAsync(user.Email);
if (toParsed == null)
return result.Invalidate(Localizer["Invalid email."], StatusCodes.Status400BadRequest);
}
catch (OperationCanceledException ex)
{
Logger.LogWarning(
"SMTP operation canceled at email verification: {Error}", ex.Message);
}
catch (SmtpCommandException ex)
{
Logger.LogWarning(
"SMTP command exception at email verification: {Error}", ex.Message);
}
catch (SmtpProtocolException ex)
{
Logger.LogWarning(
"SMTP protocol exception at email verification: {Error}", ex.Message);
}
catch (Exception ex)
{
Logger.LogWarning("Exception at email verification: {Error}", ex.Message);
}
var message = new MimeMessage();
message.From.Add(new MailboxAddress("PrivaPub", AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername));
message.To.Add(MailboxAddress.Parse(user.Email));
message.Subject = Localizer["PrivaPub - Password recovery link"];
message.Body = new TextPart("plain")
{
Text = string.Format(Localizer[@"Hey {0},
Eugene from collAnon, following is the password recovery link:
{1}
-- Eugene"], user.UserName, $"{host}/password-recovery?rc={emailRecovery.RecoveryCode}")
};
try
{
await smtpClient.SendAsync(message);
}
catch (Exception ex)
{
Logger.LogError(ex, $"Error at email sending to {user.Email} from {AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername}.");
return result.Invalidate(Localizer["Failed to send email."], StatusCodes.Status503ServiceUnavailable, exception: ex);
}
await smtpClient.DisconnectAsync(quit: true);
}
return result; return result;
} }
catch (Exception ex) catch (Exception ex)
@@ -498,16 +401,19 @@ Eugene from collAnon, following is the password recovery link:
} }
} }
Task<EmailRecovery> LiveRecovery(string recoveryCode)
{
var hash = RecoveryJob.Hash(recoveryCode);
var now = DateTime.UtcNow;
return DbEntities.EmailRecoveries.Match(er => er.CodeHash == hash && er.ExpiresAt > now).ExecuteFirstAsync();
}
public async Task<WebResult> IsValidRecoveryCode(string recoveryCode) public async Task<WebResult> IsValidRecoveryCode(string recoveryCode)
{ {
var result = new WebResult(); var result = new WebResult();
try try
{ {
var isValidRecoveryCode = await DbEntities.EmailRecoveries result.Data = await LiveRecovery(recoveryCode) != default;
.Match(er => er.RecoveryCode == recoveryCode).ExecuteAnyAsync();
result.Data = isValidRecoveryCode;
return result; return result;
} }
catch (Exception ex) catch (Exception ex)
@@ -516,33 +422,24 @@ Eugene from collAnon, following is the password recovery link:
} }
} }
// A recovered password ends every session the root had: whoever had them may be why it was recovered.
public async Task<WebResult> ChangePassword(NewPasswordForm newPasswordForm) public async Task<WebResult> ChangePassword(NewPasswordForm newPasswordForm)
{ {
var result = new WebResult(); var result = new WebResult();
try try
{ {
var isValidRecoveryCode = await DbEntities.EmailRecoveries var recovery = await LiveRecovery(newPasswordForm.RecoveryCode);
.Match(er => er.RecoveryCode == newPasswordForm.RecoveryCode).ExecuteAnyAsync(); var user = recovery == default ? default
if (!isValidRecoveryCode) : await DbEntities.RootUsers.MatchID(recovery.RootUserId).Match(u => u.DeletedAt == null && !u.IsBanned).ExecuteFirstAsync();
if (user == default)
return result.Invalidate(Localizer["Invalid recovery code."], StatusCodes.Status404NotFound); return result.Invalidate(Localizer["Invalid recovery code."], StatusCodes.Status404NotFound);
var emailRecovery = await DbEntities.EmailRecoveries await DB.Default.Update<RootUser>().MatchID(user.ID)
.Match(er => er.RecoveryCode == newPasswordForm.RecoveryCode) .Modify(u => u.HashedPassword, PasswordHasher.Hash(newPasswordForm.NewPassword))
.Project(er => er.Include(nameof(EmailRecovery.RootUserId))) .Modify(u => u.UpdatedAt, DateTime.UtcNow)
.ExecuteFirstAsync();
if (emailRecovery is null || emailRecovery.RootUserId is null)
return result.Invalidate(Localizer["User not found."]);
var newHashedPassword = PasswordHasher.Hash(newPasswordForm.NewPassword);
_ = await DB.Default.Update<RootUser>()
.Match(u => u.ID == emailRecovery.RootUserId && u.DeletedAt == null)
.Modify(u => u.HashedPassword, newHashedPassword)
.ExecuteAsync(); .ExecuteAsync();
await DB.Default.DeleteAsync<EmailRecovery>(er => er.RootUserId == user.ID);
_ = await DB.Default.DeleteAsync<EmailRecovery>(er => er.RecoveryCode == newPasswordForm.RecoveryCode); await Sessions.Revoke(user.ID, CancellationToken.None);
return result; return result;
} }
catch (Exception ex) catch (Exception ex)
+6 -1
View File
@@ -12,6 +12,9 @@ namespace PrivaPub.StaticServices
{ {
public class AuthTokenManager public class AuthTokenManager
{ {
public const string SessionStamp = "ss";
const string SessionStampClaim = SessionStamp;//the root's RootUser.SessionStamp when issued; JwtEvents refuses a stale one
readonly IConfiguration Configuration; readonly IConfiguration Configuration;
public AuthTokenManager(IConfiguration configuration) public AuthTokenManager(IConfiguration configuration)
@@ -35,7 +38,8 @@ namespace PrivaPub.StaticServices
var claims = new List<Claim> var claims = new List<Claim>
{ {
new(ClaimTypes.UserData, user.ID), new(ClaimTypes.UserData, user.ID),
new(ClaimTypes.Name, user.UserName) new(ClaimTypes.Name, user.UserName),
new(SessionStampClaim, user.SessionStamp ?? string.Empty)
}; };
claims.Add(new(Policies.IsUser, $"{user.Policies.Contains(Policies.IsUser)}".ToLower())); claims.Add(new(Policies.IsUser, $"{user.Policies.Contains(Policies.IsUser)}".ToLower()));
@@ -44,6 +48,7 @@ namespace PrivaPub.StaticServices
var token = new JwtSecurityToken(issuer: Configuration["AppConfiguration:Jwt:Issuer"], audience: Configuration["AppConfiguration:Jwt:Audience"], var token = new JwtSecurityToken(issuer: Configuration["AppConfiguration:Jwt:Issuer"], audience: Configuration["AppConfiguration:Jwt:Audience"],
claims: claims, claims: claims,
notBefore: DateTime.UtcNow,
expires: expiration, expires: expiration,
signingCredentials: new(securityKey, SecurityAlgorithms.HmacSha512) signingCredentials: new(securityKey, SecurityAlgorithms.HmacSha512)
); );
+1 -1
View File
@@ -31,7 +31,7 @@ namespace PrivaPub.StaticServices
using (var algorithm = new Rfc2898DeriveBytes(password, salt, iterations, HashAlgorithmName.SHA512)) using (var algorithm = new Rfc2898DeriveBytes(password, salt, iterations, HashAlgorithmName.SHA512))
{ {
var keyToCheck = algorithm.GetBytes(KeySize); var keyToCheck = algorithm.GetBytes(KeySize);
var verified = keyToCheck.SequenceEqual(key); var verified = CryptographicOperations.FixedTimeEquals(keyToCheck, key);
return (verified, needsUpgrade); return (verified, needsUpgrade);
} }
} }
+48 -1
View File
@@ -92,6 +92,49 @@ namespace PrivaPub.Web.Pages
} }
} }
// The hashtag links our posts carry (/tags/{tag}): this server's public posts with that tag, as a page like a profile's.
public class TagModel : PublicPageModel
{
const int PageSize = 20;
readonly ILocalActorService _localActors;
readonly DbEntities _dbEntities;
public TagModel(ILocalActorService localActors, DbEntities dbEntities)
{
_localActors = localActors;
_dbEntities = dbEntities;
}
public string Tag { get; private set; }
public IReadOnlyList<PostView> Posts { get; private set; } = Array.Empty<PostView>();
public async Task<IActionResult> OnGetAsync(string tag, CancellationToken token)
{
Tag = tag?.TrimStart('#').ToLowerInvariant();
if (string.IsNullOrEmpty(Tag) || Tag.Length > 100)
return NotFound();
Harden();
var posts = await _dbEntities.Posts
.Match(p => p.Tags.Contains(Tag) && !p.IsFederatedCopy && p.ReblogOfPostId == null && p.Visibility == PostVisibility.Public)
.Match(VisibilityPolicy.IsPublic)
.Sort(p => p.ID, Order.Descending)
.Limit(PageSize)
.ExecuteAsync(token);
var authors = new Dictionary<string, LocalActor>();
var views = new List<PostView>();
foreach (var post in posts)
{
if (!authors.TryGetValue(post.GroupUserId, out var author))
authors[post.GroupUserId] = author = await _localActors.FindById(LocalActorKind.Person, post.GroupUserId, token);
if (author is { IsFederated: true })
views.Add(PostView.From(post, author));
}
Posts = views;
return Page();
}
}
public class StatusModel : PublicPageModel public class StatusModel : PublicPageModel
{ {
readonly ILocalActorService _localActors; readonly ILocalActorService _localActors;
@@ -133,14 +176,18 @@ namespace PrivaPub.Web.Pages
readonly Microsoft.Extensions.Options.IOptionsMonitor<Infrastructure.Statistics.StatisticsOptions> _options; readonly Microsoft.Extensions.Options.IOptionsMonitor<Infrastructure.Statistics.StatisticsOptions> _options;
readonly Microsoft.Extensions.Options.IOptionsMonitor<Models.AppConfiguration> _app; readonly Microsoft.Extensions.Options.IOptionsMonitor<Models.AppConfiguration> _app;
readonly Infrastructure.Geo.IGeoLocator _geo;
public StargazingModel(Microsoft.Extensions.Options.IOptionsMonitor<Infrastructure.Statistics.StatisticsOptions> options, public StargazingModel(Microsoft.Extensions.Options.IOptionsMonitor<Infrastructure.Statistics.StatisticsOptions> options,
Microsoft.Extensions.Options.IOptionsMonitor<Models.AppConfiguration> app) Microsoft.Extensions.Options.IOptionsMonitor<Models.AppConfiguration> app, Infrastructure.Geo.IGeoLocator geo)
{ {
_options = options; _options = options;
_app = app; _app = app;
_geo = geo;
} }
public bool CrawlerEnabled => _options.CurrentValue.Crawler.Enabled; public bool CrawlerEnabled => _options.CurrentValue.Crawler.Enabled;
public string GeoSource => _geo.Source;
public string UserAgent => Federation.Crawler.Stargazer.UserAgent(_app.CurrentValue.BackendBaseAddress); public string UserAgent => Federation.Crawler.Stargazer.UserAgent(_app.CurrentValue.BackendBaseAddress);
public void OnGet() => Harden(); public void OnGet() => Harden();
+9
View File
@@ -15,6 +15,15 @@
<p>A server we exchange activities with is described once a week, from its public NodeInfo and, when it has one, its <p>A server we exchange activities with is described once a week, from its public NodeInfo and, when it has one, its
Mastodon instance API. Its location comes from the address we reached, looked up in an offline database: only the Mastodon instance API. Its location comes from the address we reached, looked up in an offline database: only the
country is shown publicly for small servers, and only the CDN for servers behind one.</p> country is shown publicly for small servers, and only the CDN for servers behind one.</p>
@if (Model.GeoSource is { } geo)
{
<p>Locations come from <a href="https://db-ip.com" rel="nofollow noopener">@geo</a>, licensed under
<a href="https://creativecommons.org/licenses/by/4.0/" rel="nofollow noopener">CC BY 4.0</a>.</p>
}
else
{
<p>No location database is loaded yet, so servers are not located.</p>
}
</article> </article>
<article> <article>
<h2>The crawler</h2> <h2>The crawler</h2>
+17
View File
@@ -0,0 +1,17 @@
@page "/tags/{tag}"
@model PrivaPub.Web.Pages.TagModel
@{
ViewData["Title"] = $"#{Model.Tag}";
}
<header>
<h1>#@Model.Tag</h1>
<div class="meta">Public posts on this server with this tag.</div>
</header>
@foreach (var post in Model.Posts)
{
<partial name="_Post" model="post" />
}
@if (Model.Posts.Count == 0)
{
<p class="meta">Nothing public with this tag yet.</p>
}
+23
View File
@@ -2,6 +2,29 @@
"Media": { "Media": {
"Root": "/var/lib/privapub/media" "Root": "/var/lib/privapub/media"
}, },
"Registrations": {
"Mode": "Invitations"
},
"Federation": {
"SecureMode": true
},
"Statistics": {
"Crawler": {
"Enabled": true,
"Seeds": [
"mastodon.social",
"fosstodon.org",
"mas.to",
"lemmy.world",
"lemmy.ml",
"misskey.io",
"pixelfed.social",
"framatube.org",
"piefed.social",
"bookwyrm.social"
]
}
},
"MongoSettings": { "MongoSettings": {
"Database": "PrivaPub", "Database": "PrivaPub",
"LogsDatabase": "logs", "LogsDatabase": "logs",
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env bash
# Fetches the month's DB-IP Lite city and ASN databases (CC BY 4.0, https://db-ip.com) into the directory PrivaPub reads
# them from (Statistics:GeoDirectory). The app swaps to new files on its own; a failed download leaves the old ones.
set -euo pipefail
dir="${GEO_DIR:-/var/lib/privapub/geo}"
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
this=$(date -u +%Y-%m)
last=$(date -u -d "$(date -u +%Y-%m-15) -1 month" +%Y-%m)
for kind in city asn; do
got=""
for month in "$this" "$last"; do
if curl -fsS --max-time 900 -o "$tmp/$kind.gz" "https://download.db-ip.com/free/dbip-$kind-lite-$month.mmdb.gz"; then
got=$month; break
fi
done
[ -n "$got" ] || { echo "no $kind database for $this or $last" >&2; exit 1; }
gunzip -t "$tmp/$kind.gz"
gunzip -c "$tmp/$kind.gz" > "$tmp/dbip-$kind-lite.mmdb"
[ "$(stat -c %s "$tmp/dbip-$kind-lite.mmdb")" -gt 1000000 ] || { echo "the $kind database is too small" >&2; exit 1; }
install -m 640 "$tmp/dbip-$kind-lite.mmdb" "$dir/.dbip-$kind-lite.mmdb.new"
mv -f "$dir/.dbip-$kind-lite.mmdb.new" "$dir/dbip-$kind-lite.mmdb"
echo "$kind: DB-IP Lite $got"
done
+1 -6
View File
@@ -12,7 +12,7 @@ ACME=/root/.acme.sh/acme.sh
echo "== directories" echo "== directories"
install -d -o "$RUNNER" -g www-data -m 755 /var/www/$HOST install -d -o "$RUNNER" -g www-data -m 755 /var/www/$HOST
install -d -o "$RUNNER" -g "$RUNNER" -m 750 /var/backups/$HOST install -d -o "$RUNNER" -g "$RUNNER" -m 750 /var/backups/$HOST
install -d -o www-data -g www-data -m 750 /var/lib/privapub /var/lib/privapub/mongo /var/lib/privapub/geo install -d -o www-data -g www-data -m 750 /var/lib/privapub /var/lib/privapub/mongo
echo "== sudoers" echo "== sudoers"
SUDOERS=/etc/sudoers.d/$RUNNER SUDOERS=/etc/sudoers.d/$RUNNER
@@ -23,15 +23,10 @@ visudo -cf "$SUDOERS"
echo "== units" echo "== units"
install -m 644 "$SRC/systemd/privapub-mongod.service" /etc/systemd/system/privapub-mongod.service install -m 644 "$SRC/systemd/privapub-mongod.service" /etc/systemd/system/privapub-mongod.service
install -m 644 "$SRC/systemd/$UNIT.service" /etc/systemd/system/$UNIT.service install -m 644 "$SRC/systemd/$UNIT.service" /etc/systemd/system/$UNIT.service
install -m 755 "$SRC/max/geo-update.sh" /usr/local/bin/privapub-geo-update
install -m 644 "$SRC/systemd/privapub-geo.service" /etc/systemd/system/privapub-geo.service
install -m 644 "$SRC/systemd/privapub-geo.timer" /etc/systemd/system/privapub-geo.timer
systemctl daemon-reload systemctl daemon-reload
systemctl enable --now privapub-mongod >/dev/null systemctl enable --now privapub-mongod >/dev/null
systemctl enable $UNIT >/dev/null systemctl enable $UNIT >/dev/null
systemctl enable --now privapub-geo.timer >/dev/null
systemctl is-active privapub-mongod systemctl is-active privapub-mongod
[ -f /var/lib/privapub/geo/dbip-city-lite.mmdb ] || systemctl start privapub-geo.service || echo "geolocation databases not fetched yet; the timer retries"
echo "== nginx snippet and bootstrap vhost" echo "== nginx snippet and bootstrap vhost"
install -m 644 "$SRC/nginx/privapub-headers.conf" /etc/nginx/snippets/privapub-headers.conf install -m 644 "$SRC/nginx/privapub-headers.conf" /etc/nginx/snippets/privapub-headers.conf
-15
View File
@@ -1,15 +0,0 @@
[Unit]
Description=PrivaPub: fetch the DB-IP Lite geolocation databases
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
User=www-data
Group=www-data
ExecStart=/usr/local/bin/privapub-geo-update
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/lib/privapub/geo
-10
View File
@@ -1,10 +0,0 @@
[Unit]
Description=PrivaPub: refresh the geolocation databases monthly
[Timer]
OnCalendar=*-*-03 04:00:00
RandomizedDelaySec=6h
Persistent=true
[Install]
WantedBy=timers.target
+11 -4
View File
@@ -169,15 +169,19 @@ Priorities, used throughout:
Findings: Findings:
- **Mastodon 4.7 names its actors by number**: `https://mastodon.test/ap/users/<id>`, inbox `…/ap/users/<id>/inbox`. Nothing here may assume - **Mastodon 4.7 names its actors by number**: `https://mastodon.test/ap/users/<id>`, inbox `…/ap/users/<id>/inbox`. Nothing here may assume
`/users/<name>`. `/users/<name>`.
- **It drops circle posts** (expected failure in the scenario). - **It dropped circle posts** until each member's copy named that member (owner decision 2026-10-04, v1.19.0).
- A post addressed only to `[circle, circle/flock]` parses as `direct` there, and Mastodon keeps a `direct` post - A post addressed only to `[circle, circle/flock]` parses as `direct` there, and Mastodon keeps a `direct` post
only if it names a local account or arrived in a known account's inbox (`Create#addresses_local_accounts?`). only if it names a local account or arrived in a known account's inbox (`Create#addresses_local_accounts?`).
- But `ActivityPub::InboxesController#account_required?` looks only at `params[:account_username]`. A delivery to - But `ActivityPub::InboxesController#account_required?` looks only at `params[:account_username]`. A delivery to
the numeric `/ap/users/:account_id/inbox` it now advertises therefore reaches the worker with no recipient and the numeric `/ap/users/:account_id/inbox` it now advertises therefore reaches the worker with no recipient and
is rejected. is rejected.
- DMs are unaffected because they name the recipient. - DMs are unaffected because they name the recipient.
- Fixing it on our side means naming the member (or that server's members) in each copy's `cc`, which changes what a - Each member's copy now names that member in `cc` and mentions them silently, so Mastodon keeps it, and its signed
circle reveals: **owner decision pending**. Reporting it upstream is the other half. refetch (`ActivityPub::FetchRemoteStatusService`, by its instance actor) gets the post back naming the members on
that server. The same holds for a followers-only post's refetch, which used to answer 404, and a 404 on refetch
makes Mastodon delete its copy. Reporting the numeric-inbox recipient loss upstream is still worth doing.
- **With SecureMode on** (all six peers, 2026-10-04) every federation check passes: Mastodon, GoToSocial, Misskey,
Sharkey, Akkoma and Lemmy all sign their fetches, and fetch our instance actor's key unsigned first.
- Inbound `Block` from Mastodon is not enforced yet (P7). - Inbound `Block` from Mastodon is not enforced yet (P7).
### GoToSocial: 0.22.1 (2026-07-20) ### GoToSocial: 0.22.1 (2026-07-20)
@@ -233,7 +237,10 @@ Findings:
**Pasture evidence (2026-10-03, GoToSocial 0.22.1, `tools/pasture/scenarios/gts.sh`):** 37 checks pass, three runs in a **Pasture evidence (2026-10-03, GoToSocial 0.22.1, `tools/pasture/scenarios/gts.sh`):** 37 checks pass, three runs in a
row. That is the original 33 plus four on statistics: described as gotosocial, inbound and outbound traffic counted, row. That is the original 33 plus four on statistics: described as gotosocial, inbound and outbound traffic counted,
no account named. no account named. Since 2026-10-04 (v1.19.0) circle posts reach a GoToSocial member too. GoToSocial files a post for
neither the public nor the author's followers as a direct message, like our DMs, and shows it only to the accounts it
mentions. Being in `cc` stored it but left it invisible, so each member's copy also mentions that member silently.
Such posts are then found in the member's conversations, never by a search on their URI.
### Misskey family: Misskey 2026.10.0, Sharkey 2025.4.7, Iceshrimp.NET 2026.1.2-beta, CherryPick 4.17 ### Misskey family: Misskey 2026.10.0, Sharkey 2025.4.7, Iceshrimp.NET 2026.1.2-beta, CherryPick 4.17
+43 -6
View File
@@ -17,7 +17,7 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
- daily rollups, every touched server described and located, the admin statistics API, the opt-in crawler and `/stargazing`; - daily rollups, every touched server described and located, the admin statistics API, the opt-in crawler and `/stargazing`;
- the pasture as plugins: GoToSocial 37/37, Mastodon 49 plus 2 expected failures. - the pasture as plugins: GoToSocial 37/37, Mastodon 49 plus 2 expected failures.
T10 and T12–T14: v1.17.1, 2026-10-03. The pasture runs all six peers in one pass: T10 and T12–T14: v1.17.1, deployed and verified 2026-10-03. The pasture runs all six peers in one pass:
- GoToSocial 54 plus 1 expected failure; - GoToSocial 54 plus 1 expected failure;
- Mastodon 49 plus 2; - Mastodon 49 plus 2;
- Misskey 35, Sharkey 40, Akkoma 44; - Misskey 35, Sharkey 40, Akkoma 44;
@@ -29,7 +29,27 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
- followers-only posts arrived as DMs on Pleroma and Akkoma; - followers-only posts arrived as DMs on Pleroma and Akkoma;
- Akkoma's open polls showed as ended and refused votes. - Akkoma's open polls showed as ended and refused votes.
Still open: installing the geolocation timer on Max (`deploy/max/setup.sh`, as root). Both open items were closed without a root step (owner decisions, 2026-10-04): the server fetches its geolocation
databases itself, and the deploy makes and signs in as @thepra.
- [ ] Everything on in production (owner decisions 2026-10-04):
- v1.18.0, deployed and verified 2026-10-04: geolocation that updates itself (DB-IP Lite 2026-10 loaded), the deploy
signing in as @thepra (undiscoverable), the crawler on (1010 servers known within the hour), sign-up by invitation
with one registrations switch;
- v1.19.1, deployed and verified 2026-10-04 (v1.19.0's deploy stopped at a flaky test, fixed in v1.19.1):
- circle posts reach Mastodon and GoToSocial members, each copy naming and mentioning its member;
- followers-only, direct and circle posts are served to signed refetches from those they were for;
- SecureMode is on: a persona answers 401 unsigned, a browser is redirected, and all six pasture peers pass;
- account privacy: sign-in and recovery say nothing, recovery codes are hashed for an hour, a recovered password
ends every session, and a deleted root's personas and groups are deleted everywhere;
- v1.20.0, 2026-10-04: one answer everywhere:
- one counting rule for posts and users;
- gone accounts out of every count;
- replies_count only for public replies;
- search anyone may use;
- the instance API advertising what is enforced;
- Mastodon routes that answer, grouped notifications;
- community announce ids that resolve, a hashtag page;
- remote accounts' real counts.
- [ ] P7 Threads, communities, moderation, the social graph - [ ] P7 Threads, communities, moderation, the social graph
- [ ] P8 Signatures, discovery, the long tail - [ ] P8 Signatures, discovery, the long tail
@@ -158,10 +178,24 @@ and circles (see Owner decisions).
| Local side in statistics | **Only the kind of local actor** (person, group, application), **and only on public and unlisted traffic.** DMs, followers-only and circle traffic are one "private" class, never broken out per server in public. Circles are never named, whether as a kind or as a reason. Fetches of our own documents are counted per day, never per server. | | Local side in statistics | **Only the kind of local actor** (person, group, application), **and only on public and unlisted traffic.** DMs, followers-only and circle traffic are one "private" class, never broken out per server in public. Circles are never named, whether as a kind or as a reason. Fetches of our own documents are counted per day, never per server. |
| Reading-driven traffic | **Counted per day, never logged per event:** the media proxy, lookups a client asks for, and the client API per endpoint group (admin only). Client app names are not recorded. | | Reading-driven traffic | **Counted per day, never logged per event:** the media proxy, lookups a client asks for, and the client API per endpoint group (admin only). Client app names are not recorded. |
| Describing servers | **Every server we exchange activities with is described weekly**, from its NodeInfo (including the user counts it publishes) and its Mastodon instance API, never its contact account. These requests are unsigned, because they are not ActivityPub documents. Never on read. | | Describing servers | **Every server we exchange activities with is described weekly**, from its NodeInfo (including the user counts it publishes) and its Mastodon instance API, never its contact account. These requests are unsigned, because they are not ActivityPub documents. Never on read. |
| Server locations | **City and network (ASN) from the offline DB-IP Lite databases** (CC BY 4.0, attributed), downloaded monthly outside the app. The location comes from the address we connected to; an inbound sender's address is never recorded, and no address is stored. In public: city and network only for servers reporting at least 10 users and not behind a CDN; the country otherwise; only the CDN's name for CDN-fronted servers. The admin sees everything. | | Server locations | **City and network (ASN) from the offline DB-IP Lite databases** (CC BY 4.0, attributed), downloaded monthly (by the server itself since 2026-10-04). The location comes from the address we connected to; an inbound sender's address is never recorded, and no address is stored. In public: city and network only for servers reporting at least 10 users and not behind a CDN; the country otherwise; only the CDN's name for CDN-fronted servers. The admin sees everything. |
| Crawler | **Off by default** (`Statistics:Crawler:Enabled`). When on, it identifies as `PrivaPub-Stargazer/<version> (+https://privapub.thepra.dev/stargazing)`, where `/stargazing` explains it and how to opt out. It honours robots.txt (an unreachable robots.txt means "keep out") and domain blocks. It visits one server a minute, each at most weekly, and at most 5000 servers. It reads only robots.txt, NodeInfo, the instance API and the peers list, never accounts, posts or directories. Crawled servers stay marked as crawled. | | Crawler | **Off by default** (`Statistics:Crawler:Enabled`); **on in production since 2026-10-04**, see below. When on, it identifies as `PrivaPub-Stargazer/<version> (+https://privapub.thepra.dev/stargazing)`, where `/stargazing` explains it and how to opt out. It honours robots.txt (an unreachable robots.txt means "keep out") and domain blocks. It visits one server a minute, each at most weekly, and at most 5000 servers. It reads only robots.txt, NodeInfo, the instance API and the peers list, never accounts, posts or directories. Crawled servers stay marked as crawled. |
| A remote account deletes itself | **Its posts are kept but hidden everywhere** (`Post.AuthorGone`): from timelines, profiles, search and lookups by id. Its follows and timeline rows go, as before. | | A remote account deletes itself | **Its posts are kept but hidden everywhere** (`Post.AuthorGone`): from timelines, profiles, search and lookups by id. Its follows and timeline rows go, as before. |
| Signed-in smoke check in production | **An undiscoverable persona**, whose read-only token is the Gitea secret `PRIVAPUB_SMOKE_TOKEN`; the deploy checks `verify_credentials`, home and notifications with it. The owner creates both. | | Signed-in smoke check in production | **An undiscoverable persona**, the deploy checks `verify_credentials`, home and notifications with it. *Superseded 2026-10-04: the deploy makes and keeps the persona itself, below.* |
### Owner decisions on running everything in production (2026-10-04)
| Question | Decision |
|---|---|
| What runs in production | **Everything that is built is on and checked by the deploy**, and nothing waits on a person running a command. |
| Geolocation | **The server fetches DB-IP Lite itself** (`GeoUpdater`): it checks daily, installs a new month's databases once they are published, refuses a file that does not open as the right kind of database, and keeps the old one when anything fails. No timer and no root step. The deploy fails if the databases are missing or more than 40 days old. |
| Crawler | **On in production**, seeded with a handful of large servers of different kinds (`appsettings.Production.json`); the deploy fails if `/stargazing` does not say it is on. |
| Sign-up | **Closed: invitations only.** A group invitation creates an account; open sign-up answers 403. NodeInfo, `/api/v1/instance` and `/api/v2/instance` read the same switch (`Registrations:Mode`), and the deploy fails if they disagree. The first login on a server is made with `PrivaPub admin create-root`. |
| Signed-in smoke check | **`@thepra`, undiscoverable, made and kept by the deploy**: `PrivaPub admin smoke thepra` creates or keeps the root `deploy-smoke` and the persona and gives the root a new password on every deploy; the deploy signs in through the real OAuth flow, checks the signed-in API and revokes its token. No secret is stored. |
| Signed fetches (SecureMode) | **On in production** once the pasture passes with it on (Phase 2 of the 2026-10-04 plan). |
| Circles on Mastodon and GoToSocial | **Each member's copy names that member** in `cc`; a member's refetch names the member, an instance actor's refetch the members on its server. Nothing new is revealed to anyone outside the circle. |
| What circles and located posts reveal | **Unchanged**: circles still answer WebFinger, and circle and located posts still count in a persona's post count, "a good balance for the fediverse to work". |
| Public `/stargazing` statistics | **Later**, as decided on 2026-10-03; the crawler and the admin API keep collecting meanwhile. |
## Libraries (researched; no maintained .NET ActivityPub library exists, so Letterbook and Iceshrimp.NET both wrote their own) ## Libraries (researched; no maintained .NET ActivityPub library exists, so Letterbook and Iceshrimp.NET both wrote their own)
@@ -480,7 +514,10 @@ it, raw where it doesn't.
so Mastodon 4.7 retries instead of switching to RFC 9421. so Mastodon 4.7 retries instead of switching to RFC 9421.
- **Deliberately not done: dereferenceable `Follow`, `Like`, `Block`, `Accept`, `Reject` and `Undo` ids.** Serving them - **Deliberately not done: dereferenceable `Follow`, `Like`, `Block`, `Accept`, `Reject` and `Undo` ids.** Serving them
would publish who follows, likes and blocks whom, which our collections deliberately hide. Every one of them is sent would publish who follows, likes and blocks whom, which our collections deliberately hide. Every one of them is sent
with its object embedded, which is all Misskey needs. `Create` and `Announce` ids do dereference. with its object embedded, which is all Misskey needs. `Create` and `Announce` ids do dereference, a community's
announces included (kept since 2026-10-04 while the post is shown, 410 after). Update, Delete, EmojiReact,
QuoteRequest and its answers, Flag, Ignore and poll votes do not either: like Mastodon's, they are sent whole and
never fetched.
#### P6 What people see: emoji, polls, quotes, reactions, cards, players #### P6 What people see: emoji, polls, quotes, reactions, cards, players
- **Custom emoji** on posts, names, fields and poll options, proxied (done in v1.10.0), together with fuller remote - **Custom emoji** on posts, names, fields and poll options, proxied (done in v1.10.0), together with fuller remote
+2
View File
@@ -26,6 +26,8 @@
}, },
"Media": { "Root": "/tmp/privapub-media" }, "Media": { "Root": "/tmp/privapub-media" },
"RateLimits": { "AccountsPerMinute": 1000 }, "RateLimits": { "AccountsPerMinute": 1000 },
"Registrations": { "Mode": "Open" },
"Statistics": { "Geo": { "AutoUpdate": false } },
"Kestrel": { "Endpoints": { "Http": { "Url": "http://0.0.0.0:80", "Protocols": "Http1AndHttp2" } } }, "Kestrel": { "Endpoints": { "Http": { "Url": "http://0.0.0.0:80", "Protocols": "Http1AndHttp2" } } },
"Serilog": { "Serilog": {
"MinimumLevel": { "Default": "Information", "Override": { "Microsoft": "Warning", "System": "Warning" } }, "MinimumLevel": { "Default": "Information", "Override": { "Microsoft": "Warning", "System": "Warning" } },
+11 -27
View File
@@ -15,6 +15,13 @@ site() { curl -k --resolve "$1:6443:127.0.0.1" "${@:2}"; }
pfetch() { curl -sk --connect-to privapub.test:443:127.0.0.1:6443 "$@"; } pfetch() { curl -sk --connect-to privapub.test:443:127.0.0.1:6443 "$@"; }
# make_png <path>: an 8x8 red PNG, for uploads # make_png <path>: an 8x8 red PNG, for uploads
# the status an unsigned ActivityPub GET of a PrivaPub document gets
pstatus() { pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$1"; }
# SecureMode (PRIVAPUB_ENV="Federation__SecureMode=true") answers every unsigned GET but the instance actor's 401, so a
# post's author answering 401 means it is on; then 401 is what "not served" and "gone" look like to an unsigned reader
secure_mode() { [ "$(pstatus "${1%%/scribbles/*}")" = "401" ]; }
unserved() { local code; code=$(pstatus "$1"); if secure_mode "$1"; then [ "$code" = 401 ]; else [ "$code" = 404 ]; fi; }
gone_unsigned() { local code; code=$(pstatus "$1"); if secure_mode "$1"; then [ "$code" = 401 ]; else [ "$code" = 410 ]; fi; }
make_png() { python3 -c " make_png() { python3 -c "
import struct,zlib import struct,zlib
w=h=8 w=h=8
@@ -30,37 +37,14 @@ privapub_root() {
echo "$root" | j "print(d['token'])" echo "$root" | j "print(d['token'])"
} }
# privapub_token <persona>: creates the persona under the pasture root if needed and returns a Mastodon token for it. # privapub_token <persona>: creates the persona under the pasture root if needed and returns a Mastodon token for it,
# through the same OAuth flow the deploy's smoke check uses (tools/smoke/oauth.sh).
privapub_token() { privapub_token() {
local persona=$1 jwt cid cs q xt form code local persona=$1 jwt
jwt=$(privapub_root) jwt=$(privapub_root)
curl -s -o /dev/null -X POST $P/clientapi/avatar/private/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ curl -s -o /dev/null -X POST $P/clientapi/avatar/private/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"userName\":\"$persona\",\"name\":\"$persona of PrivaPub\",\"biography\":\"testing federation\"}" -d "{\"userName\":\"$persona\",\"name\":\"$persona of PrivaPub\",\"biography\":\"testing federation\"}"
local app; app=$(curl -s -X POST $P/api/v1/apps -d 'client_name=pasture&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read+write+follow') "$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" "$persona" "read write follow" | cut -d' ' -f1
cid=$(echo "$app" | j "print(d['client_id'])"); cs=$(echo "$app" | j "print(d['client_secret'])")
q="client_id=$cid&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=read+write+follow"
local jar="$work/jar-$persona"
xt=$(curl -s -c "$jar" -b "$jar" "$P/oauth/login?returnUrl=/oauth/authorize?$q" | grep -o 'name="__RequestVerificationToken" type="hidden" value="[^"]*"' | sed 's/.*value="//;s/"//')
curl -s -o /dev/null -c "$jar" -b "$jar" -X POST $P/oauth/login --data-urlencode "returnUrl=/oauth/authorize?$q" --data-urlencode "__RequestVerificationToken=$xt" \
--data-urlencode "userName=$ROOT_USER" --data-urlencode "password=$ROOT_PASS"
curl -s -c "$jar" -b "$jar" "$P/oauth/authorize?$q&signed_in=1" > "$work/choose-$persona.html"
form=$(python3 - "$work/choose-$persona.html" "$persona" <<'PY'
import re,sys,urllib.parse,html
s=open(sys.argv[1]).read()
pairs=[(k,html.unescape(v)) for k,v in re.findall(r'<input type="hidden" name="([^"]*)" value="([^"]*)"',s)]
blocks=re.findall(r'<label[^>]*>(.*?)</label>',s,re.S)
avatar=None
for b in blocks:
if '@'+sys.argv[2]+'@' in b or '@'+sys.argv[2]+'<' in b or '>'+sys.argv[2]+'<' in b:
m=re.search(r'name="avatarId" value="([^"]*)"',b)
if m: avatar=m.group(1)
if avatar is None:
avatar=re.findall(r'name="avatarId" value="([^"]*)"',s)[0]
print(urllib.parse.urlencode(pairs+[("avatarId",avatar),("decision","allow")]))
PY
)
code=$(curl -s -c "$jar" -b "$jar" -X POST $P/oauth/authorize --data "$form" | grep -o '<code>[^<]*</code>' | sed 's/<[^>]*>//g')
curl -s -X POST $P/oauth/token -d "grant_type=authorization_code&code=$code&client_id=$cid&client_secret=$cs&redirect_uri=urn:ietf:wg:oauth:2.0:oob" | j "print(d['access_token'])"
} }
# stats_check <host> <software>: the admin statistics name the peer's software and count traffic both ways. # stats_check <host> <software>: the admin statistics name the peer's software and count traffic both ways.
+2 -1
View File
@@ -57,7 +57,8 @@ until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id/context" | j
echo " likes, boosts and reactions" echo " likes, boosts and reactions"
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/favourite" curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/favourite"
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/reblog" curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/reblog"
until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "1 1" ]' && ok "alice_akkoma's like and boost count on Akkoma" || ko "like or boost not counted on Akkoma" # Akkoma counts them from its own job queue, which can lag behind under the load of a six-peer run: two minutes
until_true 60 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "1 1" ]' && ok "alice_akkoma's like and boost count on Akkoma" || ko "like or boost not counted on Akkoma"
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/unfavourite" curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/unfavourite"
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/unreblog" curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/unreblog"
until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "0 0" ]' && ok "alice_akkoma's unlike and unboost reach Akkoma" || ko "undo of like or boost not applied on Akkoma" until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "0 0" ]' && ok "alice_akkoma's unlike and unboost reach Akkoma" || ko "undo of like or boost not applied on Akkoma"
+6 -6
View File
@@ -165,12 +165,12 @@ until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/accounts/relationships?id[]=$ci
curl -s -o /dev/null -X POST $P/clientapi/post/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ curl -s -o /dev/null -X POST $P/clientapi/post/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle on GoToSocial sees this\",\"groupId\":\"$circle\"}" -d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle on GoToSocial sees this\",\"groupId\":\"$circle\"}"
circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle on GoToSocial/}).ObjectURI)') circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle on GoToSocial/}).ObjectURI)')
if until_true 15 '[ "$(on_gts "$circle_uri" | j "print(len(d[\"statuses\"]))")" = "1" ]'; then # GoToSocial files a post for neither the public nor the author's followers as a direct message (as it does our DMs),
ok "a circle post reaches its GoToSocial member" # shown only to the accounts it mentions; so each member's copy names and silently mentions that member. Like a DM it
else # is then in gtsuser's conversations, never in a search by URI.
xf "a circle post reaches its GoToSocial member (GoToSocial keeps no post addressed only to a collection it does not know)" until_true 15 'gcurl -s -H "$GH" "$G/api/v1/conversations" | grep -q "only the circle on GoToSocial sees this"' \
fi && ok "a circle post, naming its member, reaches its GoToSocial member" || ko "circle post missing on GoToSocial"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$circle_uri")" = "404" ] && ok "the circle post is not served unsigned" || ko "circle post served unsigned" unserved "$circle_uri" && ok "the circle post is not served unsigned" || ko "circle post served unsigned"
echo "locked personas" echo "locked personas"
LT=$(privapub_token locked_alice); LH="Authorization: Bearer $LT" LT=$(privapub_token locked_alice); LH="Authorization: Bearer $LT"
+12 -10
View File
@@ -39,7 +39,10 @@ cw_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=behind a warning
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$cw_uri" | j "print(d and d[\"spoiler_text\"]==\"spoilers\" and d[\"sensitive\"])")" = "True" ]' && ok "a content warning survives to Mastodon" || ko "content warning lost on Mastodon" until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$cw_uri" | j "print(d and d[\"spoiler_text\"]==\"spoilers\" and d[\"sensitive\"])")" = "True" ]' && ok "a content warning survives to Mastodon" || ko "content warning lost on Mastodon"
fo_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=only for followers&visibility=private' | j "print(d['uri'])") fo_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=only for followers&visibility=private' | j "print(d['uri'])")
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$fo_uri" | j "print(d and d[\"visibility\"])")" = "private" ]' && ok "a followers-only post reaches Mastodon as private" || ko "followers-only post missing or widened on Mastodon" until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$fo_uri" | j "print(d and d[\"visibility\"])")" = "private" ]' && ok "a followers-only post reaches Mastodon as private" || ko "followers-only post missing or widened on Mastodon"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$fo_uri")" = "404" ] && ok "the followers-only post is not served unsigned" || ko "followers-only post served unsigned" unserved "$fo_uri" && ok "the followers-only post is not served unsigned" || ko "followers-only post served unsigned"
# Mastodon deletes its copy when a refetch answers 404, so a signed refetch from a follower's server is answered
fo_refetch=$(podman exec pasture-mastodon bin/rails runner "s = ActivityPub::FetchRemoteStatusService.new.call('$fo_uri'); puts(s.present? ? s.visibility : 'lost')" 2>/dev/null | tail -1)
[ "$fo_refetch" = "private" ] && ok "Mastodon's signed refetch of the followers-only post keeps it private" || ko "Mastodon's refetch of the followers-only post: $fo_refetch"
echo " replies" echo " replies"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" -d "status=@alice_masto@privapub.test replying from Mastodon&in_reply_to_id=$a_post_on_m&visibility=public" mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" -d "status=@alice_masto@privapub.test replying from Mastodon&in_reply_to_id=$a_post_on_m&visibility=public"
@@ -112,7 +115,7 @@ cw_on_m=$(m_status_by_uri "$alice_on_m" "$cw_uri" | j "print(d['id'])")
cw_id=$(curl -s -H "$AH" "$P/api/v1/accounts/$a_poll_id/statuses" | j "print(next(s['id'] for s in d if s['uri']=='$cw_uri'))") cw_id=$(curl -s -H "$AH" "$P/api/v1/accounts/$a_poll_id/statuses" | j "print(next(s['id'] for s in d if s['uri']=='$cw_uri'))")
curl -s -o /dev/null -X DELETE -H "$AH" "$P/api/v1/statuses/$cw_id" curl -s -o /dev/null -X DELETE -H "$AH" "$P/api/v1/statuses/$cw_id"
until_true 30 '[ "$(mcurl -o /dev/null -w "%{http_code}" -H "$MH" "$M/api/v1/statuses/$cw_on_m")" = "404" ]' && ok "alice_masto's delete reaches Mastodon" || ko "delete not applied on Mastodon" until_true 30 '[ "$(mcurl -o /dev/null -w "%{http_code}" -H "$MH" "$M/api/v1/statuses/$cw_on_m")" = "404" ]' && ok "alice_masto's delete reaches Mastodon" || ko "delete not applied on Mastodon"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$cw_uri")" = "410" ] && ok "the deleted post answers 410" || ko "deleted post does not answer 410" gone_unsigned "$cw_uri" && ok "the deleted post answers 410" || ko "deleted post does not answer 410"
mcurl -o /dev/null -X DELETE -H "$MH" "$M/api/v1/statuses/$m_poll" mcurl -o /dev/null -X DELETE -H "$MH" "$M/api/v1/statuses/$m_poll"
until_true 30 '! curl -s -H "$AH" "$P/api/v1/timelines/home" | grep -q "Mastodon asks"' && ok "mastouser's delete reaches PrivaPub" || ko "Mastodon's delete not applied on PrivaPub" until_true 30 '! curl -s -H "$AH" "$P/api/v1/timelines/home" | grep -q "Mastodon asks"' && ok "mastouser's delete reaches PrivaPub" || ko "Mastodon's delete not applied on PrivaPub"
@@ -144,20 +147,19 @@ circle_post=$(curl -s -X POST $P/clientapi/post/insert -H 'Content-Type: applica
-d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle sees this\",\"groupId\":\"$circle\"}") -d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle sees this\",\"groupId\":\"$circle\"}")
circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle sees this/},{ObjectURI:1}).ObjectURI)') circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle sees this/},{ObjectURI:1}).ObjectURI)')
# Mastodon 4.7 learns whose personal inbox a delivery reached only from /users/<name>/inbox, never from the numeric # Mastodon 4.7 learns whose personal inbox a delivery reached only from /users/<name>/inbox, never from the numeric
# /ap/users/<id>/inbox it now advertises, and keeps a post naming no local account only for that recipient # /ap/users/<id>/inbox it now advertises, and keeps a post naming no local account of its own only for that recipient
# (InboxesController#account_required?, Create#addresses_local_accounts?). A circle post names only the circle. # (InboxesController#account_required?, Create#addresses_local_accounts?). So each member's copy names that member.
m_stored() { podman exec pasture-mastodon bin/rails runner "puts Status.exists?(uri: '$1')" 2>/dev/null | tail -1; } m_stored() { podman exec pasture-mastodon bin/rails runner "puts Status.exists?(uri: '$1')" 2>/dev/null | tail -1; }
if until_true 15 '[ "$(m_stored "$circle_uri")" = "true" ]'; then until_true 15 '[ "$(m_stored "$circle_uri")" = "true" ]' && ok "a circle post, naming its member, reaches its Mastodon member" || ko "circle post missing on Mastodon"
ok "a circle post reaches its Mastodon member" # a signed refetch, as Mastodon does it, is answered for a member's server and names the member, so the copy stays
else refetched=$(podman exec pasture-mastodon bin/rails runner "s = ActivityPub::FetchRemoteStatusService.new.call('$circle_uri'); puts(s.present? ? 'kept' : 'lost')" 2>/dev/null | tail -1)
xf "a circle post reaches its Mastodon member (Mastodon 4.7 loses the recipient of numeric-inbox deliveries; owner decision pending)" [ "$refetched" = "kept" ] && ok "Mastodon's signed refetch of the circle post keeps it" || ko "Mastodon's refetch of the circle post failed ($refetched)"
fi
mastodon_user outsider mastodon_user outsider
OT=$(mastodon_token outsider) OT=$(mastodon_token outsider)
alice_on_m_o=$(mcurl -H "Authorization: Bearer $OT" "$M/api/v2/search?q=@alice_masto@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") alice_on_m_o=$(mcurl -H "Authorization: Bearer $OT" "$M/api/v2/search?q=@alice_masto@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
[ "$(mcurl -H "Authorization: Bearer $OT" "$M/api/v1/accounts/$alice_on_m_o/statuses?limit=40" | j "print(any(s['uri']=='$circle_uri' for s in d))")" = "False" ] \ [ "$(mcurl -H "Authorization: Bearer $OT" "$M/api/v1/accounts/$alice_on_m_o/statuses?limit=40" | j "print(any(s['uri']=='$circle_uri' for s in d))")" = "False" ] \
&& ok "another Mastodon user does not see the circle post" || ko "the circle post leaked to a non-member" && ok "another Mastodon user does not see the circle post" || ko "the circle post leaked to a non-member"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$circle_uri")" = "404" ] && ok "the circle post is not served unsigned" || ko "circle post served unsigned" unserved "$circle_uri" && ok "the circle post is not served unsigned" || ko "circle post served unsigned"
echo " reports" echo " reports"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/reports" -d "account_id=$masto_on_p&comment=pasture-report-$circle_name&forward=true" curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/reports" -d "account_id=$masto_on_p&comment=pasture-report-$circle_name&forward=true"
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Gets a Mastodon API token for one persona through PrivaPub's real OAuth code flow, as a client would: register an app,
# sign in with the root's password at /oauth/login, choose the persona at /oauth/authorize, read the out-of-band code and
# exchange it. Prints "<access token> <client id> <client secret>" so the caller can revoke the token afterwards.
# usage: tools/smoke/oauth.sh <base url> <root login> <root password> <persona> [scopes]
set -euo pipefail
BASE="${1:?base url}"; LOGIN="${2:?root login}"; PASSWORD="${3:?root password}"; PERSONA="${4:?persona}"; SCOPES="${5:-read}"
jar=$(mktemp -d); trap 'rm -rf "$jar"' EXIT
json() { python3 -c "import sys,json; d=json.load(sys.stdin); print($1)"; }
app=$(curl -fsS -X POST "$BASE/api/v1/apps" --data-urlencode client_name=privapub-oauth \
--data-urlencode redirect_uris=urn:ietf:wg:oauth:2.0:oob --data-urlencode "scopes=$SCOPES")
cid=$(echo "$app" | json "d['client_id']"); cs=$(echo "$app" | json "d['client_secret']")
scope_q=$(python3 -c "import sys,urllib.parse; print(urllib.parse.quote(sys.argv[1]))" "$SCOPES")
q="client_id=$cid&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=$scope_q"
xt=$(curl -fsS -c "$jar/c" -b "$jar/c" "$BASE/oauth/login?returnUrl=/oauth/authorize?$q" \
| grep -o 'name="__RequestVerificationToken" type="hidden" value="[^"]*"' | sed 's/.*value="//;s/"//')
curl -sS -o /dev/null -c "$jar/c" -b "$jar/c" -X POST "$BASE/oauth/login" --data-urlencode "returnUrl=/oauth/authorize?$q" \
--data-urlencode "__RequestVerificationToken=$xt" --data-urlencode "userName=$LOGIN" --data-urlencode "password=$PASSWORD"
curl -fsS -c "$jar/c" -b "$jar/c" "$BASE/oauth/authorize?$q&signed_in=1" > "$jar/choose.html"
form=$(python3 - "$jar/choose.html" "$PERSONA" <<'PY'
import re, sys, urllib.parse, html
page = open(sys.argv[1]).read()
pairs = [(k, html.unescape(v)) for k, v in re.findall(r'<input type="hidden" name="([^"]*)" value="([^"]*)"', page)]
persona = None
for block in re.findall(r'<label[^>]*>(.*?)</label>', page, re.S):
if '@' + sys.argv[2] + '@' in block or '@' + sys.argv[2] + '<' in block or '>' + sys.argv[2] + '<' in block:
found = re.search(r'name="avatarId" value="([^"]*)"', block)
if found:
persona = found.group(1)
if persona is None:
sys.exit("persona '%s' is not offered at /oauth/authorize" % sys.argv[2])
print(urllib.parse.urlencode(pairs + [("avatarId", persona), ("decision", "allow")]))
PY
)
code=$(curl -fsS -c "$jar/c" -b "$jar/c" -X POST "$BASE/oauth/authorize" --data "$form" | grep -o '<code>[^<]*</code>' | sed 's/<[^>]*>//g')
[ -n "$code" ] || { echo "no authorization code" >&2; exit 1; }
token=$(curl -fsS -X POST "$BASE/oauth/token" --data-urlencode grant_type=authorization_code --data-urlencode "code=$code" \
--data-urlencode "client_id=$cid" --data-urlencode "client_secret=$cs" --data-urlencode redirect_uri=urn:ietf:wg:oauth:2.0:oob \
| json "d['access_token']")
echo "$token $cid $cs"