Compare commits

...
21 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 60bf21b019 CLAUDE.md describes the code after P1.1
Build / Build (push) Successful in 1m7s
Deploy / privapub.thepra.dev (push) Successful in 1m1s
The route table lists the themed names now in use, the repo map gains
Jobs, Ids, Moderation, Domain and Web, and the invariants add the job
queue, VisibilityPolicy and the rule for which remote posts are stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:28:37 +02:00
thepraandClaude Opus 5.5 1c78da34a9 One visibility rule for every public read
VisibilityPolicy.IsPublic is the single expression for "anyone may see
this" (Public or Unlisted, not deleted); the outbox, the object and
activity endpoints, the HTML pages and NodeInfo all use it instead of
spelling it out. CanSee answers for a persona: the author, a mentioned
local persona, a conversation member for Direct, a circle member for
Circle; followers-only waits for P1.2's follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:26:51 +02:00
thepraandClaude Opus 5.5 5a13597c00 Persona and group ids say which day, nothing more
An ObjectId carries its creation second and a per-process counter, so two
avatars made one after the other by the same login got ids a few counts
apart: a link between personas that every Mastodon client would have
seen. Avatar and Group now generate ids from the UTC day plus eight random
bytes (still valid ObjectIds), and a remote post's id is made from its
published time with a random tail, so posts page in the order they were
written rather than the order they arrived.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:25:48 +02:00
thepraandClaude Opus 5.5 e6a362c0b8 Domain blocks: suspend, silence, reject media
DomainBlock (domain, severity, reject-media, public and private comment)
covers the domain and its subdomains. Admins manage them under
/clientapi/admin/domainblocks/{list,insert,delete}; the set is kept in
memory, reloaded on every change and at most five minutes stale.

A suspended domain is refused by FederationHttp.IsAllowed, so nothing is
fetched from it and no job delivers to it, and the inbox drops its
activities with a 202 before fetching any key. Reject-media strips the
attachments of posts from that domain. Silence is recorded for the
timelines and notifications that arrive in P1.2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:25:05 +02:00
thepraandClaude Opus 5.5 9ec1f9930b Profile and post pages, NodeInfo 2.1, and FEDERATION.md
/@user and /@user/{id} are Razor pages showing an avatar's or community's
public and unlisted posts: no scripts, a strict CSP, no-referrer, noindex,
and an alternate link to the ActivityPub document. A client asking them
for activity+json is redirected to the actor or note, and the actor and
note redirect browsers here.

NodeInfo answers 2.1 as well as 2.0 (repository, homepage, a link to
FEDERATION.md) and counts only public local posts.

FEDERATION.md (FEP-67ff) lists the protocols, FEPs, route names,
activities and the security rules a peer will notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:23:03 +02:00
thepraandClaude Opus 5.5 e54e17708f Outbound documents speak Mastodon's dialect under PrivaPub's route names
Local text (Domain/Content/ContentRenderer): Markdig with raw HTML off and
autolinks, or plain text for API clients; then @user and @user@host are
resolved (locally or through WebFinger) into Mastodon's h-card markup and
#tags into hashtag links, skipping code, links and e-mail addresses. A
post stores the result, its mentions and its tags, and is delivered to
the mentioned actors and the parent's author as well as to followers.

Renderer:
- Mastodon's @context (toot, schema PropertyValue, discoverable,
  indexable, blurhash, focalPoint, featured, alsoKnownAs, movedTo) and
  FEP-2c59's webfinger;
- an actor's url is its HTML page, its fields are PropertyValue
  attachments, published is cut to the day and indexable is false;
- a note's content is the stored rendering; a title becomes name and a
  bold first line, a content warning without its own text is summarised
  by the title or "Content warning"; Mention and Hashtag tags, contentMap,
  updated, and to/cc by visibility.

Routes take the agreed names: /groupies, /stalking, /scribbles/{id},
/grunts/{id} (a Create resolves), /whispers/{id} for a DM's context. The
outbox is a collection with a first page, paged by max_id. A browser
asking for an actor or a note is redirected to /@user, and WebFinger's
profile-page points there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:21:31 +02:00
thepraandClaude Opus 5.5 e9529310fe One Post model for every visibility, read from notes the way peers write them
Post gains what federation and the Mastodon API need: Visibility (Public,
Unlisted, FollowersOnly, Direct, Circle, LocalGeo), the author's account
id, to/cc, the Create's id, url, context, quote, InReplyToURI and the
parent's author, a separate SpoilerText next to the title, language,
mentions, hashtags, remote attachments (alt text, blurhash, focus, size),
reply/favourite/reblog counters, revisions, EditedAt and DeletedAt.

Direct messages are Posts with Visibility Direct and a ConversationId;
migration _005 copies DmPost rows across with their ids and fills the new
fields of existing posts. DmPost is left in place so a rollback still sees
the old messages.

Inbound:
- NoteParser reads Note, Article, Page, Question and media types: content,
  then contentMap, then _misskey_content; summary as the spoiler and name
  as the title; Mention and Hashtag tags; attachments; a PeerTube-style
  list attribution prefers the person over the channel; quote URIs.
- Addressing classifies like Mastodon, finding followers-only by the
  author's own followers URL (now stored on ForeignAvatar), not by a
  "/followers" suffix.
- Create keeps a post when a local persona is addressed or mentioned, when
  it replies to a local post (the parent's reply count goes up) or when a
  community it follows is addressed; an unsolicited public post is not
  stored. Update keeps the previous version as a revision.

The outbox and object endpoints serve only Public and Unlisted posts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:17:14 +02:00
thepraandClaude Opus 5.5 796f06acf4 The inbox answers once it has verified, and processes from the job queue
InboxService is split the way the roadmap lays out Federation/Inbox:
- InboxReceiver reads and verifies the request exactly as before, runs
  the checks that need no fetch (the activity id's origin, a Follow of a
  missing or local-only actor, an Undo of someone else's activity, an
  embedded object attributed to someone else), queues a ProcessInbox job
  and answers 202. The job's dedupe key is the activity id, so a peer that
  delivers the same activity twice is processed once.
- InboxProcessor (two at a time, eight attempts) loads the verified actor
  and hands the activity to the handler for its type.
- Handlers/{Follow,Undo,Create,Delete,Update}Handler are the old methods,
  unchanged except that they no longer produce status codes; the JSON
  helpers live in Objects/ActivityJson and the group membership helpers in
  Inbox/ForeignMembers.

A slow fetch of an object or a remote actor now delays the job, not the
sender's HTTP request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:12:34 +02:00
thepraandClaude Opus 5.5 d1a91c40c4 Deliveries run on a Mongo job queue with leases, backoff and per-host limits
The single serial DeliveryWorker is replaced by Infrastructure/Jobs:
- Job rows are leased with one FindOneAndUpdate (oldest RunAt first, a
  two-minute lease) and a reaper returns expired leases every 30 s;
- enqueueing wakes the workers, which otherwise poll every five seconds;
- delivery runs eight at a time with at most two per host, so a slow or
  dead server holds two slots, not the queue;
- a failure waits n^4 + 15 + jitter seconds (Mastodon's curve) for up to
  16 attempts; a 4xx other than 408/429 is final, a 429 honours
  Retry-After;
- RemoteInstance is a per-host circuit breaker: ten consecutive failures
  quarantine a host for an hour, doubling to a week, and its jobs wait
  without spending attempts;
- a delivery is queued once per activity and inbox (unique DedupeKey), and
  finished jobs expire after seven days (TTL on FinishedAt).

Migration _004 moves pending Delivery rows into jobs and marks them
abandoned, so a rollback to the old worker cannot send them twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:09:35 +02:00
thepraandClaude Opus 5.5 934b6fe687 P0 is deployed and verified
Build / Build (push) Successful in 25s
v1.1.0 serves the actor, NodeInfo and WebFinger, answers Swagger with 404
and inbox junk with 400 and 401; migrations 1-3 ran and the unique
indexes exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:07:23 +02:00
thepraandClaude Opus 5.5 1a62e8a567 CLAUDE.md describes the code after P0
Build / Build (push) Successful in 47s
Deploy / privapub.thepra.dev (push) Successful in 51s
The repo map gains Infrastructure/ and Objects/, the federation invariants
are the ones the code now enforces instead of a list of known holes, and
Commands, Data, Testing and Deploy cover the test project, the startup
order (warm maps, migrate, index), the admin CLI and the deploy checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:04:20 +02:00
thepraandClaude Opus 5.5 d80cd42a0c Rate limits on accounts and inboxes, a database dump before every deploy
- Sign-up, login, the invitation flows and password recovery allow ten
  requests a minute per client address; the inboxes give each sending
  origin (the keyId's) a bucket of 300 that refills at 300 a minute, and
  answer 429 beyond it, which peers retry.
- deploy.yml dumps the PrivaPub database to /var/backups before it stops
  the service (the last seven are kept), and after the swap checks that
  Swagger answers 404 and that the shared inbox answers junk with 400 and
  an unsigned activity with 401.
- ActivityPubClient and PostBoost, never used, are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:02:21 +02:00
thepraandClaude Opus 5.5 d2f0c7a14e Unique indexes, one username space, and entity maps warmed before use
Infrastructure/Data/Indexes runs at every start, after the migrations:
unique on Post/DmPost ObjectURI, ForeignAvatar ActorURI, the Follower
triple, RootToAvatar, RootUser UserName and ReservedName; plain indexes on
the lookups the services actually make (PublicKeyId, author and group
post listings, ParticipantsKey, the delivery queue).

Migration _001 runs first and removes the duplicates the races could
already have left (keeping the newest actor row, the oldest post, the
accepted follower), then fills ReservedName from every avatar and group.

ReservedName is one username space for personas, groups and the instance:
a name is reserved by an insert the unique index arbitrates, before the
avatar or group is saved, so two simultaneous sign-ups cannot both get it.
A short list of names (admin, support, abuse, postmaster, ...) is never
available.

EntityMaps.Warm touches every entity's collection one at a time before
anything else runs. MongoDB.Entities maps the Entity base class on first
touch, and two types mapped at once throw "An item with the same key has
already been added" and stay broken for the life of the process; the
parallel test run hit it, and the delivery worker racing a request could
have too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:01:08 +02:00
thepraandClaude Opus 5.5 0f85030744 No admin by username, no Swagger in production, no exception text to clients
S14 and the privacy items of P0:
- signing up as "admin" no longer grants admin; `PrivaPub admin promote
  <root>` (and `demote`) does, run on the box against the configured
  database;
- Swagger is served in Development only;
- every service and controller answers "Something went wrong." where it
  used to send ex.Message, and the SMTP warnings no longer log the
  recipient's address;
- sign-up and login no longer log the IP, User-Agent and root id together;
- invitation sign-up takes the persona's own AvatarUserName (and optional
  AvatarName) instead of naming the avatar after the private login, and
  refuses a persona username equal to the login's. Invitation login uses
  the named persona, creating it if it is new;
- recovery mail comes from "PrivaPub", not collAnon's support address name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:58:39 +02:00
thepraandClaude Opus 5.5 2eb2a63f1e Conversations are keyed by who is in them, and circles stay home
S8: a direct message joins a conversation only when its participants are
exactly that conversation's members, found through a new
DmGroup.ParticipantsKey (a hash of the sorted members). A remote context
no longer decides anything: it let anyone who knew a conversation's
context post into it, and joining by context while dropping a participant
would have shown a reply to someone it was not addressed to. A context is
kept only when it is on the author's origin. Sending a DM to the same
people again reuses their conversation instead of opening a new one.

S9: Group.Kind is Circle or Community. A circle is not a federated actor:
its actor, collections, WebFinger and inbox answer 404, a remote Follow is
refused, and posts in it are IsLocalOnly - never delivered, never in an
outbox, never served. Communities keep today's behaviour until P4.
Migration _003 makes every existing group a circle, marks their posts
local-only and backfills the conversation keys.

End-to-end inbox tests sign real deliveries from a fake peer: a context
injection, a forged activity id, a note attributed to someone else, a
cross-origin object, a bad signature, junk bodies and a Follow of a circle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:57:01 +02:00
thepraandClaude Opus 5.5 bf7c88ce71 Remote HTML is sanitized before it is stored
S5 of the roadmap. ContentSanitizer wraps HtmlSanitizer with Mastodon's
allowlist: the inline and list tags Mastodon keeps, href/rel/class and
the list attributes, microformat and mention/hashtag/ellipsis/invisible
classes, Mastodon's link schemes, every link rel=nofollow noopener
noreferrer, relative links unlinked, headings folded to a bold paragraph,
and the contents of script, style, svg, iframe and friends dropped rather
than kept as text.

Post and DmPost gain ContentHtml (what is shown) and ContentFormat
(Markdown for local, Html for remote). Inbound Create and Update, and a
remote actor's biography, are sanitized on the way in; local posts store
their Markdig rendering. Migration _002 does the same to what is already
stored, and migrations now run at startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:54:17 +02:00
thepraandClaude Opus 5.5 611abb5857 Inbound signatures must be fresh and cover the request line and host
S6 of the roadmap. An inbox POST is refused unless its signature covers
(request-target) and host, as well as the digest and a date or (created).
The Date or (created) may be at most an hour old and fifteen minutes ahead
(it was twelve hours either way), and an (expires) in the past is refused.
The request target is read raw from the server, so a percent-encoded path
verifies as the sender signed it.

Tests cover a Mastodon-shaped delivery and each way of tampering with it,
plus a round trip of our own outbound signature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:51:10 +02:00
thepraandClaude Opus 5.5 a060204dd6 A remote actor is believed only from its own origin
S1 and S2 of the roadmap. RemoteActorService:
- FetchObject accepts a document only when its id is the address it was
  served from; a same-origin document naming another address is asked for
  at that address once (how GoToSocial serves its key URIs), anything else
  is dropped;
- GetActorByKeyId accepts a key only when the actor lists it, its owner is
  the actor and it lives on the actor's origin, whether the keyId points at
  the actor or at a key document;
- a refetch for a key or an actor happens at most once per five minutes,
  so a bad signature cannot make us hammer a host;
- the cache row is written by one atomic upsert on ActorURI;
- every fetch is signed by the instance actor, never by the persona that
  happened to receive the activity.

The inbox refuses an activity whose id is not on its actor's origin, and
an Undo of someone else's activity; a Create's object, an Update and a
Delete must be on the actor's origin too, and a cross-origin object is
refetched from its own origin before it is trusted.

Tests: a fake peer on two origins serves forged actors, foreign-owned keys,
cross-origin key documents, aliases and a GoToSocial-style key address
(integration, PRIVAPUB_TEST_MONGOD=1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:49:58 +02:00
thepraandClaude Opus 5.5 ccc3597699 Every outbound federation request goes through one guarded client
Infrastructure/Http adds the client the roadmap's S3 and S4 ask for:
- the connect callback resolves the name itself and refuses loopback,
  private, link-local, CGNAT, documentation, multicast, ULA, NAT64, 6to4,
  Teredo and IPv4-mapped/compatible forms, then connects to the vetted
  address, so DNS rebinding cannot swap it afterwards;
- redirects are followed by hand, at most three, each one re-checked;
- bodies are capped at 1 MB after decompression, only JSON media types are
  read, every request has a 15 s budget, and a refused URL is not asked
  again for five minutes.

Actor and WebFinger fetches and inbox deliveries all use it. Test networks
can switch on Federation:AllowPrivateNetworks/AllowPlainHttp; startup
refuses both in Production.

PrivaPub.Tests (xUnit v3) starts with the address table and the fetcher's
limits against an in-process peer; build.yml and deploy.yml run it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:46:09 +02:00
thepraandClaude Opus 5.5 034b792801 Move the federation code under Federation/, namespaces only
Build / Build (push) Successful in 16s
Services/Federation and Controllers/ServerToServer become
Federation/{Actors,Signing,Inbox,Outbox,Rendering,Controllers}, the first step
of the roadmap's layout. No type, route or behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:39:41 +02:00
thepraandClaude Opus 5.5 31f6015591 CLAUDE.md, a README, and the roadmap to full ActivityPub interop
Build / Build (push) Successful in 25s
CLAUDE.md records what the project is (one private login owning unlinkable
personas, each its own actor), the deliberately odd route names (peasants,
mouth, anus, human-centipede, sniff/again, and the agreed ones still to come),
the federation and privacy invariants, the data and style conventions, and
how it deploys. docs/ROADMAP.md holds the intent reconstructed from the 2023
code, the gap audit, the owner's decisions of 2026-10-01 (Mastodon client
API, personas unlinkable to others, local-only range posts, communities and
circles), the libraries chosen, and phases P0-P5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:38:35 +02:00
111 changed files with 7101 additions and 1390 deletions

No files matched your search

+3
View File
@@ -14,3 +14,6 @@ jobs:
- name: Build
run: dotnet build PrivaPub.sln -c Release
- name: Test
run: dotnet test PrivaPub.sln -c Release --no-build
+19
View File
@@ -12,6 +12,8 @@ env:
BACKUPS: /var/backups/privapub.thepra.dev
LOCAL_URL: http://127.0.0.1:6970
PUBLIC_URL: https://privapub.thepra.dev
MONGO_URI: mongodb://127.0.0.1:27022
MONGO_DB: PrivaPub
jobs:
site:
@@ -26,6 +28,9 @@ jobs:
echo "BUILD_REF=${GITHUB_REF_NAME:-master}" >> "$GITHUB_ENV"
echo "BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_ENV"
- name: Test
run: dotnet test PrivaPub.sln -c Release
- name: Publish
run: |
rm -rf "$GITHUB_WORKSPACE/publish"
@@ -52,6 +57,13 @@ jobs:
echo "SNAPSHOT=$BACKUPS/site-$STAMP" >> "$GITHUB_ENV"
ls -1dt "$BACKUPS"/site-* 2>/dev/null | tail -n +4 | xargs -r rm -rf || true
- name: Dump the database
run: |
DUMP="$BACKUPS/mongo-$(date +%Y%m%d-%H%M%S).archive.gz"
mongodump --quiet --uri "$MONGO_URI" --db "$MONGO_DB" --gzip --archive="$DUMP"
echo "::notice::database dumped to $DUMP ($(du -h "$DUMP" | cut -f1))"
ls -1t "$BACKUPS"/mongo-*.archive.gz 2>/dev/null | tail -n +8 | xargs -r rm -f || true
- name: Stop, sync, start
run: |
sudo systemctl stop "$UNIT"
@@ -85,4 +97,11 @@ jobs:
[ "$code" = "200" ] || { echo "::error::the instance actor answered $code"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/.well-known/nodeinfo")
[ "$code" = "200" ] || { echo "::error::nodeinfo answered $code"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/swagger/index.html")
[ "$code" = "404" ] || { echo "::error::swagger answered $code in production"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/activity+json' --data '{"junk":' "$PUBLIC_URL/human-centipede")
[ "$code" = "400" ] || { echo "::error::a junk inbox POST answered $code"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/activity+json' \
--data '{"type":"Follow","actor":"https://example.org/users/x","object":"'"$PUBLIC_URL"'/peasants/privapub"}' "$PUBLIC_URL/human-centipede")
[ "$code" = "401" ] || { echo "::error::an unsigned inbox POST answered $code"; exit 1; }
echo "::notice::serving $served"
+238
View File
@@ -0,0 +1,238 @@
# CLAUDE.md
Guidance for working in this repository. `docs/ROADMAP.md` holds the owner's decisions and the phased plan to full
ActivityPub interop; read it before changing anything federation-, privacy- or API-shaped.
## What this is
**PrivaPub** (repo name SocialPub) is a self-hosted ActivityPub server in C#, live at https://privapub.thepra.dev.
It is meant as a Pleroma-like microblogging server for privacy-minded people and small communities, federating with
Mastodon, GoToSocial, Pleroma/Akkoma, Misskey and Lemmy.
Its defining idea: **one private login owns several public personas.**
- **`RootUser` is the private login.** Username, password, optional email (used only for recovery), policies.
- **`Avatar` is a public persona.** Each one is an ActivityPub actor with its own RSA keys at `/peasants/{username}`.
Root and avatar are linked only through `RootToAvatar`.
- **Personas must stay unlinkable** to other users and servers; only the instance admin can see the link.
- **`Group` is an ActivityPub Group actor.** It has members, an invitation code and an optional password. Groups are
becoming two kinds: a public **community** (FEP-1b12, Lemmy-compatible) and a private, invitation-only **circle**.
- **`DmGroup` is a direct-message conversation.**
- **`privapub` is the instance actor** (type Application). It signs fetches no persona should be tied to.
Privacy features in the model:
- location-ranged posts (`Post.Location`, `RangeKm`), to become local-only and never federated;
- contacts a persona can share (`Avatar.SharedPersonalContacts`);
- private notes (`RootUserNote`, `PersonalNote`).
The intended client is the Mastodon client API (Tusky, Elk, Phanpy, Ivory), where each avatar logs in as its own
account. The private `/clientapi` covers what Mastodon can't express. `thepra/decePubClient`
(https://decepub.thepra.dev) is the owner's own Pleroma-FE-like PWA. It still shows mock data and is not wired to the
server.
## Route names are deliberate
The odd names are the owner's and part of the project's character. **Never "fix" them to conventional ones.** A new
actor-scoped route gets a name in the same spirit, agreed with the owner, and a name is frozen once it has federated.
| Thing | Route |
|---|---|
| Actor | `/peasants/{name}` (`/users/{name}` 301-redirects here; browsers are sent to `/@{name}`) |
| Inbox | `/peasants/{name}/mouth` |
| Outbox | `/peasants/{name}/anus` (`?page=true[&max_id=]` for pages) |
| Shared inbox | `/human-centipede` (also `/peasants/{name}/human-centipede`) |
| Followers | `/peasants/{name}/groupies` |
| Following | `/peasants/{name}/stalking` |
| Notes | `/peasants/{name}/scribbles/{id}` |
| Activities | `/peasants/{name}/grunts/{id}` (`create-{postId}` resolves) |
| DM context | `/peasants/{name}/whispers/{id}` |
| Token refresh | `/clientapi/user/sniff/again` |
Agreed for later phases: `/gossip`, `/drool`, `/echoes` (replies, likes, shares), `/trophies` (featured), `/tattoos`
(featured tags), `/flock` and `/wardens` (group members and moderators).
Routes other software looks up by name stay conventional:
- `/.well-known/*` and `/nodeinfo/*`;
- `/api/v1/*` and `/oauth/*`;
- `/@{name}` pages;
- `/media/*`.
## Repo map
```
PrivaPub.sln
PrivaPub/ ASP.NET Core Web API, net10.0
Program.cs host, Mongo init (GUIDs Standard), pipeline, /build.json
Middleware/SocialPubConfigurations.cs every DI registration (auth, federation, services, swagger, CORS)
Controllers/ClientToServer/ /clientapi/*: RootUser (signup/login/invitations/recovery), PrivateAvatar,
Group, Post (posts + DMs), Admin, Data
Infrastructure/
Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out
Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker
Ids/ PrivacyIds (day-only ids for personas and groups, published-time ids for remote posts)
Data/ Indexes (created at start), EntityMaps.Warm, Migrations/_NNN_*.cs
Cli/ AdminCommands (`PrivaPub admin promote|demote <root>`)
RateLimiting.cs accounts (per client address) and inbox (per sending origin) policies
Federation/
Controllers/ PeasantsController (actor, outbox, followers, following, posts, inboxes),
WellKnownController (webfinger, nodeinfo), UsersController (redirect)
Actors/ LocalActorService (LocalActor, Keys, ReservedName), RemoteActorService
(authoritative fetch, key verification, WebFinger), ActorDocument (parser)
Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer
Moderation/ DomainBlocks (suspend / silence / reject media)
Signing/ HttpSignatures (draft-cavage sign/verify)
Inbox/ InboxReceiver (verify, queue, 202) → InboxProcessor (job) → Handlers/{Follow,Undo,Create,Update,Delete}
Outbox/ DeliveryService (queues jobs) + DeliveryJobHandler
Rendering/ ActivityPubRenderer (Mastodon @context, actors, notes, collections)
Domain/
Content/ ContentRenderer (Markdown or plain text → HTML with h-card mentions and hashtags)
Privacy/ VisibilityPolicy (IsPublic expression, CanSee)
Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex)
Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, …
Models/ Mongo entities: User/, Group/, Post/, Federation/, Jobs/, AppConfiguration
StaticServices/ DbEntities (Find<T> accessors), AuthTokenManager (JWT), PasswordHasher
Data/InitDb.cs first-run seeding (languages)
PrivaPub.ClientModels/ DTOs + validation resources shared with clients
PrivaPub.Tests/ xUnit v3; Support/ has a fake two-origin peer and a throwaway-database fixture
deploy/ nginx vhost, systemd units (privapub, privapub-mongod), max/setup.sh
.gitea/workflows/ build.yml (push) · deploy.yml (tag v*)
docs/ROADMAP.md decisions + phased plan
```
The roadmap moves code towards `Infrastructure/`, `Federation/`, `Domain/`, `Api/{ClientApi,Mastodon}` and `Web/`,
one pure-move commit at a time.
## Commands
```bash
dotnet build PrivaPub.sln -c Release
dotnet test PrivaPub.sln # unit tests; integration tests skip
PRIVAPUB_TEST_MONGOD=1 dotnet test PrivaPub.sln # all of them, against mongod on 127.0.0.1:27017
# (PRIVAPUB_TEST_MONGO overrides; a fresh database per run, dropped after)
cd PrivaPub && ASPNETCORE_ENVIRONMENT=Development \
Kestrel__Endpoints__Http__Url=http://127.0.0.1:6970 Kestrel__Endpoints__Http__Protocols=Http1AndHttp2 \
AppConfiguration__BackendBaseAddress=http://127.0.0.1:6970 MongoSettings__Database=PrivaPubTest \
dotnet run # needs a local mongod on 27017; swagger at /swagger
```
Development config (`appsettings.Development.json`) binds HTTPS 7195 with HTTP/2 only; the overrides above make it
curl-able. Outbound fetches only go to https DNS names resolving to public addresses; a test network (Pasture) sets
`Federation__AllowPrivateNetworks=true` and `Federation__AllowPlainHttp=true`, which startup refuses in Production.
Promoting an admin on the box (signing up as "admin" grants nothing):
```bash
cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin promote <root>
```
## Federation invariants
1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only
to public addresses; redirects are followed by hand (three at most, each re-checked); bodies are capped at 1 MB;
only JSON media types are read; a refused URL is not asked again for five minutes. Never create another
`HttpClient` for federation.
2. **Every fetch is signed by the instance actor** (`privapub`), never by a persona; deliveries are signed by the acting
avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies).
3. **A remote document is believed only from its own address.** `RemoteActorService.FetchObject` requires the
document's `id` to be the URL it was served from (a same-origin alias is followed once). A key is accepted only if
the actor lists it, its `owner` is the actor and it shares the actor's origin.
4. **Inbound inboxes verify everything before acting:**
- the signature covers `(request-target)`, `host`, `digest` and `date` or `(created)`;
- the Digest matches the body and the date is at most an hour old and fifteen minutes ahead;
- the signature verifies against the key owner's key, and the activity's `actor` is the key owner;
- the activity's `id`, and any object it creates, updates or deletes, is on the actor's origin; a cross-origin
object is refetched from its own origin.
5. **Status codes:**
- bad or missing signature: **401**;
- malformed or forged body: **400**;
- accepted: **202**;
- over the rate limit: **429**;
- **never 500** from `/peasants` or an inbox. Peers retry or "double-knock" based on these codes.
6. **Actor documents:**
- served as `application/activity+json`;
- `publicKey` is an SPKI PEM at `{actor}#main-key`, with `owner` equal to the actor's id;
- the shared inbox is advertised in `endpoints.sharedInbox`.
7. **Remote HTML is sanitized before it is stored** (`ContentSanitizer`); `Post.ContentHtml` is what is shown,
`ContentFormat` says what `Text` holds. Remote names are plain text.
8. **Circles never federate.** A circle's actor, collections, WebFinger and inbox answer 404, and its posts are
`IsLocalOnly`. Only communities are Group actors.
9. **A DM joins a conversation only by `DmGroup.ParticipantsKey`**, the exact set of its participants; a remote
`context` decides nothing. DMs are `Post`s with `Visibility = Direct` and a `ConversationId` (`DmPost` is legacy).
10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`):
leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`).
11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`.
12. **Remote content is stored only when someone here asked for it:** a local persona addressed or mentioned, a reply to
a local post, or a community the author follows. Followers-only is detected by the author's stored `followers` URL.
## Privacy invariants
- **No root id in federation output, NodeInfo or logs, no IP next to an identity in logs, and no `ex.Message` to a
client** ("Something went wrong." instead).
- **Never derive a public name from the root username.** Invitation sign-up takes `AvatarUserName` and refuses one
equal to the login.
- **One username space:** personas, groups and the instance reserve their name in `ReservedName` (unique index) before
they are saved; `LocalActorService.TryReserveUserName` is the only way to claim one.
- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows. Nothing may relate sibling avatars.
- **Location-ranged posts never federate.**
## Data
- MongoDB through **MongoDB.Entities 25.1**, instance API: `DB.Default.Find<T>()`, `.SaveAsync(e)`, `.Update<T>()`,
`.DeleteAsync<T>()`, `.CountAsync<T>()`. `DbEntities` wraps the finds.
- **Pass no cancellation token to `DeleteAsync` outside a transaction;** v25 throws if you do.
- **Collection name = class name, so never rename an entity class.**
- `Entity.ID` is a 24-character lowercase hex string; `GenerateNewID()` returns `object`, so cast it.
- New fields must be additive: a deploy rollback restores the binary, not the database.
- **Startup order:** `EntityMaps.Warm()` (every entity's class map, one at a time; two mapped at once throw "An item
with the same key has already been added" and stay broken), then `MigrateAsync` (`Infrastructure/Data/Migrations`,
`_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a
dedupe migration before it.
- Production runs its **own mongod** on 127.0.0.1:27022 (unit `privapub-mongod`, no auth, data in
`/var/lib/privapub/mongo`). The box's shared mongod needs credentials nobody here has.
## Code style
- Tabs, block-scoped namespaces, Allman braces.
- New services use `readonly _camel` fields and constructor injection. Older services use PascalCase fields; leave
them.
- Services return `WebResult` (`PrivaPub.ClientModels/WebResult.cs`): `result.Invalidate(localizer[...], status)`.
Controllers turn it into a status code.
- Localised strings go through `IStringLocalizer<GenericRes>`.
- Every `Display`/`ErrorMessage` resource key must exist in `FieldsNameResource`/`ErrorsResource`, including the
`Designer.cs`, which the CLI build doesn't regenerate. A missing key throws at validation time.
- ActivityPub output is built with `System.Text.Json.Nodes` in `ActivityPubRenderer`, not typed models. Inbound
documents are read through `InboxService.Id`/`Value`, which handle string, object and array.
- Libraries chosen for the roadmap: HtmlSanitizer, Markdig (`DisableHtml`), NSign (RFC 9421 inbound), OpenIddict +
OpenIddict.MongoDb, NetVips, Blurhash.Core, FFMpegCore. No ImageSharp (licence key enforced), no MassTransit.
## Testing
`PrivaPub.Tests` (xUnit v3). Unit tests need nothing; tests marked `Category=Integration` need a mongod and skip
without `PRIVAPUB_TEST_MONGOD=1`. CI runs the unit tests only (the box's mongods are production).
- `Support/Peer` is an in-process HTTP server answering on two origins (`127.0.0.1` and `localhost`), so origin rules
can be tested; `Support/RemoteActor` signs real deliveries with its own key.
- Inbox scenarios go through `InboxService.Receive` with a signed request, not through the private handlers.
Beyond the tests, verify by building, running locally, and exercising:
- the client API (sign up, create an avatar, a group, a post);
- the ActivityPub endpoints with curl and `Accept: application/activity+json`.
Interop is checked against real servers: Fediverse Pasture with podman on the workstation, verify.funfedi.dev, and
the owner's GoToSocial at social.arasaka.software. **Ask before acting from the owner's GoToSocial account.**
## Deploy
- **CI/CD:** push to `master` runs `build.yml` (build + tests) on the instance-wide `build` runner. A `v*` tag runs
`deploy.yml`: tests, self-contained linux-x64 publish, snapshot and `mongodump` to `/var/backups/privapub.thepra.dev`,
stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo,
Swagger 404, inbox junk 400, unsigned 401).
- **The box:** Max (`nuvola.xyz`). Unit `privapub` runs as www-data from `/var/www/privapub.thepra.dev` with
`ASPNETCORE_ENVIRONMENT=Production`.
- **One-time root setup:** `deploy/max/setup.sh`, run through `../arasaka.software/tools/max/run.sh`.
- **Config:** `appsettings.Production.json` is committed and deployed, **secrets included, by the owner's convention**
(the same as arasaka.software). A hand edit on the box is lost at the next deploy.
- **Secrets drift:** `AppConfigurationService` copies `AppConfiguration` into Mongo on first boot and reads the stored
copy afterwards, so changing those keys needs a Mongo update as well.
- **Logs:** `journalctl -u privapub`, plus the `logs` database on the private mongod.
+95
View File
@@ -0,0 +1,95 @@
# Federation
PrivaPub is an ActivityPub server written in C#. This document follows
[FEP-67ff](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md) and describes how it federates.
## Supported federation protocols and standards
- [ActivityPub](https://www.w3.org/TR/activitypub/) (server-to-server)
- [WebFinger](https://webfinger.net/)
- [HTTP Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures), `rsa-sha256` / `hs2019` with RSA keys
- [NodeInfo](https://nodeinfo.diaspora.software/) 2.0 and 2.1
## Supported FEPs
- [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md)
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
- [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md)
Planned: FEP-1b12 (communities), FEP-8fcf (followers synchronisation), FEP-5feb (`indexable`), FEP-7628 (Move),
FEP-044f (quotes).
## Actors
Every account is an *avatar*: one private login can own several, and they are deliberately unlinkable. Nothing in an
actor document, a collection, NodeInfo or a delivery relates two avatars of the same login.
| Thing | Address |
|---|---|
| Actor (Person, Group, Application) | `/peasants/{name}` (`/users/{name}` redirects) |
| Inbox | `/peasants/{name}/mouth` |
| Outbox | `/peasants/{name}/anus` |
| Shared inbox | `/human-centipede` |
| Followers / following | `/peasants/{name}/groupies`, `/peasants/{name}/stalking` |
| Objects | `/peasants/{name}/scribbles/{id}` |
| Activities | `/peasants/{name}/grunts/{id}` |
| Direct-message context | `/peasants/{name}/whispers/{id}` |
| Profile and post pages | `/@{name}`, `/@{name}/{id}` |
The names are the project's own and are stable; resolve actors through WebFinger, not by guessing a path.
- The key is `{actor}#main-key`, RSA 2048, served as SPKI PEM with `owner` set to the actor.
- `published` on an actor is truncated to the day. `indexable` is `false`.
- The instance actor is `/peasants/privapub` (type `Application`). It signs every fetch PrivaPub makes, so no avatar's key
is used to read another server's content.
## Groups
A group is either a **community** or a **circle**.
- A community is a `Group` actor. It accepts `Follow` and re-shares (`Announce`) posts from its followers that address
it. Full FEP-1b12 behaviour (announcing activities, `audience`, moderation) is planned.
- A circle is private and does not federate yet: its actor, collections and WebFinger answer 404, and its posts are never
delivered.
## Activities
Received:
| Activity | Effect |
|---|---|
| `Follow` | follows an avatar or community; `Accept` is sent unless the community approves members by hand |
| `Undo{Follow}` | unfollows |
| `Create{Note, Article, Page, Question, …}` | stored when it addresses or mentions a local avatar, replies to a local post, or is addressed to a community the author follows |
| `Update{Note}` | replaces the content; the previous version is kept |
| `Update{Person}` | refetches the actor |
| `Delete` | deletes the object, or the actor and its follows |
Sent: `Create{Note}`, `Delete{Tombstone}`, `Accept{Follow}`, `Announce` (communities).
A `Create`'s `Note` carries Mastodon's `content`, `contentMap`, `summary` and `sensitive`, plus `Mention` and `Hashtag`
tags. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
`name`. A content warning without its own text uses the title, or "Content warning".
Visibility is expressed in `to`/`cc` the way Mastodon does it: public, unlisted, followers-only and direct. Inbound
followers-only posts are recognised by the author's own `followers` collection.
## Security rules a peer will notice
- **Signatures.** Inbox POSTs must be signed over `(request-target)`, `host`, `digest` and `date` (or `(created)`). The
date may be at most one hour old and fifteen minutes ahead. A bad signature gets 401, malformed input 400, an accepted
activity 202, too many requests 429. Activities are processed after the 202.
- **Origins.** An actor document is accepted only from the address it names as its `id`. A key only if its actor lists
it with `owner` set to the actor and on the actor's origin. An activity's `id`, and any object it creates, updates or
deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin.
- **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three
redirects and read at most 1 MB.
- **HTML.** Received HTML is sanitised to Mastodon's allowlist.
- **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
starting at an hour and growing to a week.
## Known limitations
- Likes, boosts, follow requests to remote accounts, media uploads and polls are not implemented yet.
- Collections expose counts, not members.
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
@@ -0,0 +1,33 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.Admin
{
public class DomainBlockForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(253, MinimumLength = 3, ErrorMessageResourceName = "StringLengthMinMax", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Domain { get; set; }
public bool Suspend { get; set; } = true;
public bool RejectMedia { get; set; }
[StringLength(1000, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string PublicComment { get; set; }
[StringLength(1000, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string PrivateComment { get; set; }
}
public class ViewDomainBlock
{
public string Id { get; set; }
public string Domain { get; set; }
public string Severity { get; set; }
public bool RejectMedia { get; set; }
public string PublicComment { get; set; }
public string PrivateComment { get; set; }
public DateTime CreatedAt { get; set; }
}
}
@@ -24,6 +24,7 @@ namespace PrivaPub.ClientModels.Group
public string Description { get; set; }
public string InvitationPassword { get; set; }
public bool IsCommunity { get; set; }
public bool IsDiscoverable { get; set; } = true;
public bool ManuallyApprovesMembers { get; set; }
}
+1
View File
@@ -8,6 +8,7 @@ namespace PrivaPub.ClientModels.Group
public string Description { get; set; }
public string Url { get; set; }
public string Handle { get; set; }
public bool IsCommunity { get; set; }
public bool IsDiscoverable { get; set; }
public bool ManuallyApprovesMembers { get; set; }
public bool IsOwner { get; set; }
@@ -10,5 +10,12 @@ namespace PrivaPub.ClientModels
StringLength(Constants.GroupInvitationLength, MinimumLength = Constants.GroupInvitationLength,
ErrorMessageResourceName = "StringLengthMinMax", ErrorMessageResourceType = typeof(ErrorsResource))]
public string InvitationCode { get; set; }
[StringLength(32, MinimumLength = 3, ErrorMessageResourceName = "StringLengthMinMax", ErrorMessageResourceType = typeof(ErrorsResource)),
RegularExpression("^[a-z0-9_]+$", ErrorMessageResourceName = "EmptySpacesNotAllowed", ErrorMessageResourceType = typeof(ErrorsResource))]
public string AvatarUserName { get; set; }
[StringLength(64, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string AvatarName { get; set; }
}
}
+2
View File
@@ -8,9 +8,11 @@ namespace PrivaPub.ClientModels.Post
public string AuthorActorURI { get; set; }
public string GroupId { get; set; }
public string DmGroupId { get; set; }
public string Visibility { get; set; }
public string AnsweringToPostId { get; set; }
public string Title { get; set; }
public string Text { get; set; }
public string ContentHtml { get; set; }
public bool HasContentWarning { get; set; }
public bool IsFederatedCopy { get; set; }
public DateTime CreationDate { get; set; }
@@ -0,0 +1,91 @@
using PrivaPub.Domain.Content;
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Domain
{
public class ContentRendererTests
{
const string Base = "https://privapub.test";
readonly ContentRenderer _renderer = new(new StubLocalActors(), new StubRemoteActors());
[Fact]
public async Task Decorates_local_and_remote_mentions_and_hashtags()
{
var rendered = await _renderer.Markdown("hello @alice and @bob@m.example about #Fedi **now** <script>x</script>", TestContext.Current.CancellationToken);
Assert.Contains("<span class=\"h-card\" translate=\"no\"><a href=\"https://privapub.test/peasants/alice\" class=\"u-url mention\"", rendered.Html);
Assert.Contains("href=\"https://m.example/@bob\"", rendered.Html);
Assert.Contains("href=\"https://privapub.test/tags/fedi\" class=\"mention hashtag\"", rendered.Html);
Assert.Contains("<strong>now</strong>", rendered.Html);
Assert.DoesNotContain("<script>", rendered.Html);
Assert.Equal(new[] { "https://m.example/users/bob", "https://privapub.test/peasants/alice" }, rendered.Mentions.Select(m => m.ActorUri).Order(StringComparer.Ordinal));
Assert.Equal("fedi", Assert.Single(rendered.Tags));
}
[Theory]
[InlineData("`@alice` in code")]
[InlineData("mail me@alice.example")]
[InlineData("@nobody is unknown")]
[InlineData("see https://x.example/page#alice")]
public async Task Leaves_what_is_not_a_mention_alone(string text)
{
var rendered = await _renderer.Markdown(text, TestContext.Current.CancellationToken);
Assert.Empty(rendered.Mentions);
Assert.DoesNotContain("h-card", rendered.Html);
}
[Theory]
[InlineData("issue #123")]
[InlineData("it&#39;s fine")]
[InlineData("[link](https://x.example/#tag)")]
public async Task Leaves_what_is_not_a_hashtag_alone(string text) =>
Assert.Empty((await _renderer.Markdown(text, TestContext.Current.CancellationToken)).Tags);
[Fact]
public async Task Renders_plain_text_like_mastodon()
{
var rendered = await _renderer.PlainText("line one\nline <two>\n\nsee https://x.example/a?b=1. #Tag", TestContext.Current.CancellationToken);
Assert.Equal("<p>line one<br>line &lt;two&gt;</p><p>see <a href=\"https://x.example/a?b=1\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">https://x.example/a?b=1</a>. "
+ "<a href=\"https://privapub.test/tags/tag\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>Tag</span></a></p>", rendered.Html);
}
sealed class StubLocalActors : ILocalActorService
{
public string BaseAddress => Base;
public Task<LocalActor> FindByUserName(string userName, CancellationToken token) =>
Task.FromResult(userName.Equals("alice", StringComparison.OrdinalIgnoreCase)
? new LocalActor { Id = "a1", UserName = "alice", BaseAddress = Base, Kind = LocalActorKind.Person }
: default);
public Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> FindByUri(string actorUri, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> GetInstanceActor(CancellationToken token) => throw new NotSupportedException();
public Task<bool> IsUserNameTaken(string userName, CancellationToken token) => throw new NotSupportedException();
public Task<bool> TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token) => throw new NotSupportedException();
public LocalActor FromAvatar(Avatar avatar) => throw new NotSupportedException();
public LocalActor FromGroup(GroupEntity group) => throw new NotSupportedException();
}
sealed class StubRemoteActors : IRemoteActorService
{
public Task<string> ResolveHandle(string handle, CancellationToken token) =>
Task.FromResult(handle == "bob@m.example" ? "https://m.example/users/bob" : default);
public Task<ForeignAvatar> GetActor(string actorUri, bool refresh, CancellationToken token) =>
Task.FromResult(actorUri == "https://m.example/users/bob"
? new ForeignAvatar { ID = "f1", ActorURI = actorUri, UserName = "bob", Url = "https://m.example/@bob", InboxURL = actorUri + "/inbox" }
: default);
public Task<FetchedJson> FetchObject(string uri, CancellationToken token) => throw new NotSupportedException();
public Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token) => throw new NotSupportedException();
}
}
}
@@ -0,0 +1,42 @@
using MongoDB.Entities;
using PrivaPub.Domain.Privacy;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public class VisibilityPolicyTests
{
[Fact]
public async Task Who_sees_what()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var conversation = new DmGroup { Members = new() { new GroupMember { AvatarId = "alice" }, new GroupMember { AvatarId = "https://r.example/u/bob", IsForeign = true } } };
var circle = new GroupEntity { UserName = $"c{Guid.NewGuid():N}"[..20], Members = new() { new GroupMember { AvatarId = "carol" } } };
await DB.Default.SaveAsync(conversation, token);
await DB.Default.SaveAsync(circle, token);
var direct = new Post { Visibility = PostVisibility.Direct, ConversationId = conversation.ID, GroupUserId = "dave" };
var circlePost = new Post { Visibility = PostVisibility.Circle, GroupId = circle.ID };
var followersOnly = new Post { Visibility = PostVisibility.FollowersOnly, Mentions = new() { new PostMention { IsLocal = true, AccountId = "erin" } } };
var deleted = new Post { Visibility = PostVisibility.Public, DeletedAt = DateTime.UtcNow };
Assert.True(await VisibilityPolicy.CanSee(new Post(), default, token));
Assert.True(await VisibilityPolicy.CanSee(direct, "alice", token));
Assert.True(await VisibilityPolicy.CanSee(direct, "dave", token));
Assert.False(await VisibilityPolicy.CanSee(direct, "carol", token));
Assert.False(await VisibilityPolicy.CanSee(direct, default, token));
Assert.True(await VisibilityPolicy.CanSee(circlePost, "carol", token));
Assert.False(await VisibilityPolicy.CanSee(circlePost, "alice", token));
Assert.True(await VisibilityPolicy.CanSee(followersOnly, "erin", token));
Assert.False(await VisibilityPolicy.CanSee(followersOnly, "alice", token));
Assert.False(await VisibilityPolicy.CanSee(deleted, "alice", token));
}
}
}
@@ -0,0 +1,51 @@
using PrivaPub.Federation.Objects;
namespace PrivaPub.Tests.Federation
{
public class ContentSanitizerTests
{
[Theory]
[InlineData("<script>alert(1)</script><p>hi</p>", "<p>hi</p>")]
[InlineData("<p onclick=\"alert(1)\">hi</p>", "<p>hi</p>")]
[InlineData("<img src=x onerror=alert(1)>", "")]
[InlineData("<a href=\"javascript:alert(1)\">x</a>", "<a rel=\"nofollow noopener noreferrer\" target=\"_blank\">x</a>")]
[InlineData("<a href=\"data:text/html,<script>alert(1)</script>\">x</a>", "<a rel=\"nofollow noopener noreferrer\" target=\"_blank\">x</a>")]
[InlineData("<a href=\"/relative\">x</a>", "<a rel=\"nofollow noopener noreferrer\" target=\"_blank\">x</a>")]
[InlineData("<iframe src=\"https://evil.example\"></iframe>text", "text")]
[InlineData("<style>body{display:none}</style><p>x</p>", "<p>x</p>")]
[InlineData("<svg><script>alert(1)</script></svg>", "")]
[InlineData("<p style=\"position:fixed\">x</p>", "<p>x</p>")]
[InlineData("<form action=\"https://evil.example\"><input name=p></form>ok", "ok")]
[InlineData("<p>a<!-- comment -->b</p>", "<p>ab</p>")]
[InlineData("<div><p>kept</p></div>", "<p>kept</p>")]
[InlineData("<h2>Title</h2>", "<p><strong>Title</strong></p>")]
[InlineData("<span class=\"evil big\">x</span>", "<span>x</span>")]
public void Removes_what_is_not_allowed(string input, string expected) =>
Assert.Equal(expected, ContentSanitizer.Html(input));
[Fact]
public void Keeps_mastodon_mention_and_hashtag_markup()
{
var html = "<p><span class=\"h-card\" translate=\"no\"><a href=\"https://m.example/@alice\" class=\"u-url mention\">@<span>alice</span></a></span> "
+ "<a href=\"https://m.example/tags/fedi\" class=\"mention hashtag\" rel=\"tag\">#<span>fedi</span></a> "
+ "<a href=\"https://example.org/a-long-link\"><span class=\"invisible\">https://</span><span class=\"ellipsis\">example.org/a-long</span></a></p>";
var sanitized = ContentSanitizer.Html(html);
Assert.Contains("class=\"h-card\"", sanitized);
Assert.Contains("class=\"u-url mention\"", sanitized);
Assert.Contains("class=\"mention hashtag\"", sanitized);
Assert.Contains("class=\"invisible\"", sanitized);
Assert.Contains("class=\"ellipsis\"", sanitized);
Assert.Contains("href=\"https://m.example/@alice\"", sanitized);
Assert.DoesNotContain("rel=\"tag\"", sanitized);
}
[Theory]
[InlineData("")]
[InlineData(" ")]
[InlineData(null)]
public void Empty_input_is_empty(string input) =>
Assert.Equal(string.Empty, ContentSanitizer.Html(input));
}
}
@@ -0,0 +1,135 @@
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Features;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Signing;
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
namespace PrivaPub.Tests.Federation
{
public class HttpSignaturesTests
{
const string Host = "privapub.test";
const string KeyId = "https://mastodon.example/users/alice#main-key";
static readonly DateTimeOffset Now = new(2026, 10, 1, 12, 0, 0, TimeSpan.Zero);
static readonly byte[] Body = Encoding.UTF8.GetBytes("""{"type":"Follow","actor":"https://mastodon.example/users/alice"}""");
readonly RSA _key = RSA.Create(2048);
string PublicKey => _key.ExportSubjectPublicKeyInfoPem();
DefaultHttpContext MastodonRequest(string signedTarget = "/peasants/bob/mouth", DateTimeOffset? signedAt = default,
string headers = "(request-target) host date digest content-type", byte[] signedBody = default, string extra = "")
{
var date = (signedAt ?? Now).ToString("r", CultureInfo.InvariantCulture);
var digest = HttpSignatures.Digest(signedBody ?? Body);
var values = new Dictionary<string, string>
{
["(request-target)"] = $"post {signedTarget}",
["host"] = Host,
["date"] = date,
["digest"] = digest,
["content-type"] = "application/activity+json"
};
var signingString = string.Join("\n", headers.Split(' ').Select(h => $"{h}: {values[h]}"));
var signature = Convert.ToBase64String(_key.SignData(Encoding.UTF8.GetBytes(signingString), HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
var context = new DefaultHttpContext();
context.Request.Method = "POST";
context.Request.Host = new HostString(Host);
context.Request.Path = "/peasants/bob/mouth";
context.Features.Get<IHttpRequestFeature>().RawTarget = "/peasants/bob/mouth";
context.Request.Headers["Date"] = date;
context.Request.Headers["Digest"] = digest;
context.Request.Headers["Content-Type"] = "application/activity+json";
context.Request.Headers["Signature"] = $"keyId=\"{KeyId}\",algorithm=\"rsa-sha256\",headers=\"{headers}\",signature=\"{signature}\"{extra}";
return context;
}
string Check(HttpContext context, byte[] body = default)
{
var parameters = HttpSignatures.Parse(context.Request.Headers["Signature"].ToString());
var problem = HttpSignatures.CheckRequest(context.Request, parameters, body ?? Body, Now);
if (problem != default)
return problem;
return HttpSignatures.Verify(PublicKey, HttpSignatures.SigningString(context.Request, parameters), parameters.Signature)
? default
: "does not verify";
}
[Fact]
public void Accepts_a_mastodon_style_delivery() =>
Assert.Null(Check(MastodonRequest()));
[Fact]
public void Refuses_a_tampered_body() =>
Assert.Equal("the digest does not match the body", Check(MastodonRequest(), Encoding.UTF8.GetBytes("""{"type":"Delete"}""")));
[Fact]
public void Refuses_a_signature_for_another_inbox() =>
Assert.Equal("does not verify", Check(MastodonRequest(signedTarget: "/peasants/carol/mouth")));
[Fact]
public void Refuses_a_signature_older_than_an_hour() =>
Assert.Equal("the Date header is outside the allowed window", Check(MastodonRequest(signedAt: Now.AddMinutes(-61))));
[Fact]
public void Accepts_a_clock_slightly_ahead() =>
Assert.Null(Check(MastodonRequest(signedAt: Now.AddMinutes(10))));
[Fact]
public void Refuses_a_clock_far_ahead() =>
Assert.Equal("the Date header is outside the allowed window", Check(MastodonRequest(signedAt: Now.AddMinutes(20))));
[Fact]
public void Refuses_a_signature_without_the_request_target() =>
Assert.Equal("(request-target) is not signed", Check(MastodonRequest(headers: "host date digest")));
[Fact]
public void Refuses_a_signature_without_the_host() =>
Assert.Equal("host is not signed", Check(MastodonRequest(headers: "(request-target) date digest")));
[Fact]
public void Refuses_a_body_whose_digest_is_not_signed() =>
Assert.Equal("the digest is not signed", Check(MastodonRequest(headers: "(request-target) host date")));
[Fact]
public void Refuses_an_expired_signature() =>
Assert.Equal("the signature has expired",
Check(MastodonRequest(extra: $",expires=\"{Now.AddMinutes(-20).ToUnixTimeSeconds()}\"")));
[Fact]
public void Signs_with_the_raw_request_target()
{
var context = MastodonRequest(signedTarget: "/peasants/b%6Fb/mouth");
context.Features.Get<IHttpRequestFeature>().RawTarget = "/peasants/b%6Fb/mouth";
Assert.Null(Check(context));
}
[Fact]
public void Verifies_its_own_outbound_signature()
{
var (privateKey, publicKey) = Keys.NewKeyPair();
var signer = new LocalActor { UserName = "bob", BaseAddress = "https://privapub.test", PrivateKeyPem = privateKey };
using var outbound = new HttpRequestMessage(HttpMethod.Post, "https://mastodon.example/users/alice/inbox");
HttpSignatures.Sign(outbound, signer, Body);
var context = new DefaultHttpContext();
context.Request.Method = "POST";
context.Request.Host = new HostString("mastodon.example");
context.Request.Path = "/users/alice/inbox";
context.Features.Get<IHttpRequestFeature>().RawTarget = "/users/alice/inbox";
foreach (var header in outbound.Headers)
context.Request.Headers[header.Key] = string.Join(", ", header.Value);
var parameters = HttpSignatures.Parse(context.Request.Headers["Signature"].ToString());
Assert.Equal("https://privapub.test/peasants/bob#main-key", parameters.KeyId);
Assert.Null(HttpSignatures.CheckRequest(context.Request, parameters, Body));
Assert.True(HttpSignatures.Verify(publicKey, HttpSignatures.SigningString(context.Request, parameters), parameters.Signature));
}
}
}
@@ -0,0 +1,345 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Models;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class InboxScenarioTests : IAsyncLifetime
{
const string Host = "privapub.test";
const string Base = "https://" + Host;
Peer _peer;
LocalActorService _local;
InboxReceiver _receiver;
InboxProcessor _processor;
DomainBlocks _blocks;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
var cache = new MemoryCache(new MemoryCacheOptions());
_local = new LocalActorService(new DbEntities(), new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = Base }));
var remote = new RemoteActorService(Peer.Http(cache), _local, cache, new DbEntities());
var queue = new JobQueue();
var delivery = new DeliveryService(new DbEntities(), queue);
var db = new DbEntities();
_blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
_receiver = new InboxReceiver(_local, remote, queue, _blocks, NullLogger<InboxReceiver>.Instance);
_processor = new InboxProcessor(remote, new IActivityHandler[]
{
new FollowHandler(db, _local, remote, delivery),
new UndoHandler(db, _local, remote, delivery),
new CreateHandler(db, _local, remote, delivery, _blocks),
new DeleteHandler(db, _local, remote, delivery),
new UpdateHandler(db, _local, remote)
}, NullLogger<InboxProcessor>.Instance);
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
async Task<LocalActor> LocalAvatar(string name)
{
var (privateKey, publicKey) = Keys.NewKeyPair();
var avatar = new Avatar { UserName = $"{name}{Guid.NewGuid():N}"[..20], PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(avatar, TestContext.Current.CancellationToken);
return _local.FromAvatar(avatar);
}
static JsonObject DirectCreate(RemoteActor author, string to, string context = default, string objectOrigin = default, string attributedTo = default)
{
var id = $"{objectOrigin ?? Origin(author.Id)}/notes/{Guid.NewGuid():N}";
var note = new JsonObject
{
["id"] = id,
["type"] = "Note",
["attributedTo"] = attributedTo ?? author.Id,
["content"] = "<p>psst</p>",
["to"] = new JsonArray(to),
["cc"] = new JsonArray()
};
if (context != default)
note["context"] = context;
return new JsonObject
{
["id"] = $"{Origin(author.Id)}/activities/{Guid.NewGuid():N}",
["type"] = "Create",
["actor"] = author.Id,
["to"] = new JsonArray(to),
["object"] = note
};
}
async Task<InboxResult> Deliver(RemoteActor sender, string path, JsonNode activity)
{
var token = TestContext.Current.CancellationToken;
var result = await _receiver.Receive(sender.Post(Host, path, activity), default, token);
var dedupe = "inbox|" + (activity is JsonObject ? activity["id"]?.GetValue<string>() : default);
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == dedupe).ExecuteFirstAsync(token);
if (job != default)
Assert.Equal(JobResult.Done, (await _processor.Handle(job, token)).Result);
return result;
}
static string Origin(string uri) => new Uri(uri).GetLeftPart(UriPartial.Authority);
[Fact]
public async Task A_context_cannot_pull_a_stranger_into_an_existing_conversation()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob");
var mallory = new RemoteActor(_peer, "mallory");
var context = $"{_peer.A}/contexts/{Guid.NewGuid():N}";
var first = await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri, context));
var injected = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, context));
Assert.Equal(202, first.StatusCode);
Assert.Equal(202, injected.StatusCode);
var bobDm = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id && p.Visibility == PostVisibility.Direct).ExecuteSingleAsync(token);
var malloryDm = await DB.Default.Find<Post>().Match(p => p.ActorURI == mallory.Id).ExecuteSingleAsync(token);
Assert.NotEqual(bobDm.ConversationId, malloryDm.ConversationId);
var bobConversation = await DB.Default.Find<DmGroup>().OneAsync(bobDm.ConversationId, token);
Assert.DoesNotContain(bobConversation.Members, m => m.AvatarId == mallory.Id);
}
[Fact]
public async Task Replies_between_the_same_people_land_in_the_same_conversation()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob");
await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri));
await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri));
var dms = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id && p.Visibility == PostVisibility.Direct).ExecuteAsync(token);
Assert.Equal(2, dms.Count);
Assert.Single(dms.Select(d => d.ConversationId).Distinct());
}
[Fact]
public async Task An_activity_id_on_another_origin_is_refused()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
var create = DirectCreate(mallory, alice.Uri);
create["id"] = $"{_peer.B}/activities/{Guid.NewGuid():N}";
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", create);
Assert.Equal(400, result.StatusCode);
}
[Fact]
public async Task A_note_put_in_someone_elses_mouth_is_refused()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
var victim = new RemoteActor(_peer, "victim");
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, attributedTo: victim.Id));
Assert.Equal(400, result.StatusCode);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == victim.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_cross_origin_object_is_fetched_from_its_origin_before_it_is_believed()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, objectOrigin: _peer.B));
Assert.Equal(202, result.StatusCode);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == mallory.Id).ExecuteAnyAsync(token));
}
static JsonObject PublicCreate(RemoteActor author, string inReplyTo = default, params string[] cc)
{
var id = $"{Origin(author.Id)}/notes/{Guid.NewGuid():N}";
var note = new JsonObject
{
["id"] = id,
["type"] = "Note",
["attributedTo"] = author.Id,
["content"] = "<p>hello</p>",
["to"] = new JsonArray(Addressing.Public),
["cc"] = new JsonArray(cc.Prepend(author.Id + "/followers").Select(c => (JsonNode)c).ToArray())
};
if (inReplyTo != default)
note["inReplyTo"] = inReplyTo;
return new JsonObject
{
["id"] = $"{Origin(author.Id)}/activities/{Guid.NewGuid():N}",
["type"] = "Create",
["actor"] = author.Id,
["object"] = note
};
}
[Fact]
public async Task A_public_reply_to_a_local_post_is_kept_and_counted()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var parent = new Post { GroupUserId = alice.Id, AuthorAccountId = alice.Id, ActorURI = alice.Uri, Text = "hi" };
parent.ID = (string)parent.GenerateNewID();
parent.ObjectURI = alice.PostUri(parent.ID);
await DB.Default.SaveAsync(parent, token);
var bob = new RemoteActor(_peer, "bob");
await Deliver(bob, "/human-centipede", PublicCreate(bob, parent.ObjectURI));
var reply = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteSingleAsync(token);
Assert.Equal(PostVisibility.Public, reply.Visibility);
Assert.Equal(parent.ID, reply.AnsweringToPostId);
Assert.Equal(alice.Id, reply.InReplyToAccountId);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(parent.ID, token)).RepliesCount);
}
[Fact]
public async Task A_public_post_nobody_here_asked_for_is_dropped()
{
var token = TestContext.Current.CancellationToken;
var bob = new RemoteActor(_peer, "bob");
await Deliver(bob, "/human-centipede", PublicCreate(bob));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_mention_is_kept_and_linked_to_the_local_persona()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob");
var create = PublicCreate(bob, default, alice.Uri);
create["object"]!["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@" + alice.Handle });
await Deliver(bob, "/human-centipede", create);
var post = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteSingleAsync(token);
var mention = Assert.Single(post.Mentions);
Assert.True(mention.IsLocal);
Assert.Equal(alice.Id, mention.AccountId);
}
[Fact]
public async Task A_suspended_domain_is_dropped_before_its_key_is_fetched()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob", _peer.B);
await DB.Default.SaveAsync(new DomainBlock { Domain = "localhost", Severity = DomainBlockSeverity.Suspend }, token);
try
{
await _blocks.Reload(token);
var before = _peer.Requests.Count;
var result = await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri));
Assert.Equal(202, result.StatusCode);
Assert.Equal(before, _peer.Requests.Count);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteAnyAsync(token));
}
finally
{
await DB.Default.DeleteAsync<DomainBlock>(b => b.Domain == "localhost");
await _blocks.Reload(token);
}
}
[Fact]
public void A_block_covers_subdomains_but_not_lookalikes()
{
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
typeof(DomainBlocks).GetField("_blocks", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance)!
.SetValue(blocks, new Dictionary<string, DomainBlock> { ["evil.example"] = new() { Domain = "evil.example", Severity = DomainBlockSeverity.Silence } });
typeof(DomainBlocks).GetField("_loadedAt", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance)!
.SetValue(blocks, DateTime.UtcNow);
Assert.NotNull(blocks.Find("evil.example"));
Assert.NotNull(blocks.Find("A.Evil.Example."));
Assert.Null(blocks.Find("notevil.example"));
Assert.False(blocks.IsSuspended("evil.example"));
}
[Fact]
public async Task A_bad_signature_is_a_401()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
var request = mallory.Post(Host, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri));
request.Headers["Signature"] = request.Headers["Signature"].ToString().Replace("signature=\"", "signature=\"AAAA");
Assert.Equal(401, (await _receiver.Receive(request, alice, token)).StatusCode);
}
[Fact]
public async Task Junk_is_a_400_never_a_500()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
foreach (var junk in new JsonNode[] { new JsonArray(1, 2), JsonValue.Create("x"), new JsonObject { ["type"] = "Create" } })
Assert.Equal(400, (await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", junk)).StatusCode);
}
[Fact]
public async Task A_circle_is_not_a_federated_actor()
{
var token = TestContext.Current.CancellationToken;
var (privateKey, publicKey) = Keys.NewKeyPair();
var circle = new GroupEntity { UserName = $"circle{Guid.NewGuid():N}"[..20], PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(circle, token);
var bob = new RemoteActor(_peer, "bob");
var actor = _local.FromGroup(circle);
var follow = new JsonObject
{
["id"] = $"{bob.Id}/follows/{Guid.NewGuid():N}",
["type"] = "Follow",
["actor"] = bob.Id,
["object"] = actor.Uri
};
Assert.False(actor.IsFederated);
Assert.Equal(404, (await Deliver(bob, "/human-centipede", follow)).StatusCode);
}
}
}
@@ -0,0 +1,114 @@
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class NoteParserTests
{
const string MastodonNote = """
{
"id": "https://m.example/users/alice/statuses/1",
"type": "Note",
"summary": "spoilers",
"inReplyTo": "https://privapub.test/peasants/bob/posts/abc",
"published": "2026-09-30T10:00:00Z",
"url": "https://m.example/@alice/1",
"attributedTo": "https://m.example/users/alice",
"to": ["https://www.w3.org/ns/activitystreams#Public"],
"cc": ["https://m.example/users/alice/followers", "https://privapub.test/peasants/bob"],
"sensitive": true,
"conversation": "tag:m.example,2026-09-30:objectId=1:objectType=Conversation",
"context": "https://m.example/contexts/1",
"content": "<p><span class=\"h-card\"><a href=\"https://privapub.test/peasants/bob\" class=\"u-url mention\">@<span>bob</span></a></span> hi <a href=\"https://m.example/tags/Fedi\" class=\"mention hashtag\" rel=\"tag\">#<span>Fedi</span></a><script>x</script></p>",
"contentMap": { "en": "<p>hi</p>" },
"attachment": [{
"type": "Document", "mediaType": "image/png", "url": "https://files.m.example/a.png",
"name": "a cat", "blurhash": "UBL_:rOpGG-oBUNG,qRj2so|=eE1w^n4S5NH", "focalPoint": [0.5, -2.0], "width": 640, "height": 480
}],
"tag": [
{ "type": "Mention", "href": "https://privapub.test/peasants/bob", "name": "@bob@privapub.test" },
{ "type": "Hashtag", "href": "https://m.example/tags/fedi", "name": "#Fedi" },
{ "type": "Emoji", "name": ":blob:", "icon": { "type": "Image", "url": "https://files.m.example/blob.png" } }
]
}
""";
[Fact]
public void Reads_a_mastodon_note()
{
var note = NoteParser.Parse(JsonNode.Parse(MastodonNote));
Assert.Equal("https://m.example/users/alice", note.AttributedTo);
Assert.Equal("spoilers", note.SpoilerText);
Assert.True(note.Sensitive);
Assert.Equal("en", note.Language);
Assert.Equal("https://m.example/@alice/1", note.Url);
Assert.Equal("https://m.example/contexts/1", note.Context);
Assert.DoesNotContain("<script>", note.ContentHtml);
Assert.Contains("u-url mention", note.ContentHtml);
Assert.Equal("https://privapub.test/peasants/bob", Assert.Single(note.Mentions).ActorURI);
Assert.Equal("fedi", Assert.Single(note.Tags));
var media = Assert.Single(note.Attachments);
Assert.Equal("a cat", media.Description);
Assert.Equal(new[] { 0.5f, -1f }, media.Focus);
Assert.Equal(new DateTime(2026, 9, 30, 10, 0, 0, DateTimeKind.Utc), note.Published);
}
[Fact]
public void Reads_a_lemmy_page_with_a_title()
{
var note = NoteParser.Parse(JsonNode.Parse("""
{
"id": "https://l.example/post/7", "type": "Page", "name": "A title",
"attributedTo": "https://l.example/u/carol", "audience": "https://l.example/c/cats",
"content": "<p>body</p>", "to": ["https://l.example/c/cats", "https://www.w3.org/ns/activitystreams#Public"],
"published": "2026-09-30T10:00:00.123456+00:00"
}
"""));
Assert.Equal("A title", note.Title);
Assert.Equal("https://l.example/c/cats", note.Audience);
Assert.False(note.Sensitive);
}
[Fact]
public void Falls_back_to_misskey_content_and_takes_the_person_from_a_peertube_attribution()
{
var note = NoteParser.Parse(JsonNode.Parse("""
{
"id": "https://k.example/notes/9", "type": "Note",
"attributedTo": [{ "type": "Group", "id": "https://k.example/c/chan" }, { "type": "Person", "id": "https://k.example/users/9" }],
"_misskey_content": "hello <b>there</b>\nnext", "_misskey_quote": "https://k.example/notes/8"
}
"""));
Assert.Equal("https://k.example/users/9", note.AttributedTo);
Assert.Equal("hello &lt;b&gt;there&lt;/b&gt;<br>next", note.ContentHtml);
Assert.Equal("https://k.example/notes/8", note.QuoteUri);
}
[Theory]
[InlineData("""{ "id": "https://x.example/1", "type": "Person" }""")]
[InlineData("""{ "id": "not a uri", "type": "Note" }""")]
[InlineData("""[1, 2]""")]
public void Refuses_what_is_not_a_note(string json) =>
Assert.Null(NoteParser.Parse(JsonNode.Parse(json)));
}
public class AddressingTests
{
const string Followers = "https://m.example/users/alice/followers";
[Theory]
[InlineData(new[] { Addressing.Public }, new[] { Followers }, PostVisibility.Public)]
[InlineData(new[] { "as:Public" }, new string[0], PostVisibility.Public)]
[InlineData(new[] { Followers }, new[] { Addressing.Public }, PostVisibility.Unlisted)]
[InlineData(new[] { Followers }, new[] { "https://privapub.test/peasants/bob" }, PostVisibility.FollowersOnly)]
[InlineData(new[] { "https://privapub.test/peasants/bob" }, new string[0], PostVisibility.Direct)]
[InlineData(new[] { "https://m.example/users/mallory/followers" }, new string[0], PostVisibility.Direct)]
public void Classifies_like_mastodon(string[] to, string[] cc, PostVisibility expected) =>
Assert.Equal(expected, Addressing.Classify(to, cc, Followers));
}
}
@@ -0,0 +1,244 @@
using Microsoft.Extensions.Caching.Memory;
using PrivaPub.Federation.Actors;
using PrivaPub.Models;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
using System.Text.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class RemoteActorServiceTests : IAsyncLifetime
{
Peer _peer;
RemoteActorService _service;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
var cache = new MemoryCache(new MemoryCacheOptions());
var local = new LocalActorService(new DbEntities(),
new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" }));
_service = new RemoteActorService(Peer.Http(cache), local, cache, new DbEntities());
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static string Actor(string id, string keyId = default, string owner = default, string pem = default, string type = "Person") =>
new JsonObject
{
["id"] = id,
["type"] = type,
["preferredUsername"] = id[(id.LastIndexOf('/') + 1)..],
["inbox"] = id + "/inbox",
["publicKey"] = new JsonObject
{
["id"] = keyId ?? id + "#main-key",
["owner"] = owner ?? id,
["publicKeyPem"] = pem ?? Keys.NewKeyPair().PublicKeyPem
}
}.ToJsonString();
string Unique(string name) => $"/users/{name}{Guid.NewGuid():N}";
[Fact]
public async Task Accepts_an_actor_whose_document_names_its_own_address()
{
var path = Unique("alice");
_peer.Serve(path, Actor("{A}" + path));
var actor = await _service.GetActorByKeyId($"{_peer.A}{path}#main-key", refresh: false, TestContext.Current.CancellationToken);
Assert.NotNull(actor);
Assert.Equal(_peer.A + path, actor.ActorURI);
Assert.Equal($"{_peer.A}{path}#main-key", actor.PublicKeyId);
}
[Fact]
public async Task Refuses_a_document_that_claims_another_origin()
{
var victim = Unique("bob");
var mallory = Unique("mallory");
_peer.Serve(mallory, Actor("{B}" + victim));
var actor = await _service.GetActor(_peer.A + mallory, refresh: false, TestContext.Current.CancellationToken);
var byKey = await _service.GetActorByKeyId($"{_peer.B}{victim}#main-key", refresh: false, TestContext.Current.CancellationToken);
Assert.Null(actor);
Assert.Null(byKey);
}
[Fact]
public async Task Refuses_a_key_owned_by_someone_else()
{
var eve = Unique("eve");
_peer.Serve(eve, Actor("{A}" + eve, owner: "{A}/users/alice"));
Assert.Null(await _service.GetActorByKeyId($"{_peer.A}{eve}#main-key", refresh: false, TestContext.Current.CancellationToken));
}
[Fact]
public async Task Refuses_a_key_document_whose_owner_is_on_another_origin()
{
var alice = Unique("alice");
var key = $"/keys/{Guid.NewGuid():N}";
_peer.Serve(alice, Actor("{B}" + alice));
_peer.Serve(key, new JsonObject { ["id"] = "{A}" + key, ["owner"] = "{B}" + alice, ["publicKeyPem"] = Keys.NewKeyPair().PublicKeyPem }.ToJsonString());
Assert.Null(await _service.GetActorByKeyId(_peer.A + key, refresh: false, TestContext.Current.CancellationToken));
}
[Fact]
public async Task Refuses_a_key_the_actor_does_not_list()
{
var alice = Unique("alice");
_peer.Serve(alice, Actor("{A}" + alice));
Assert.Null(await _service.GetActorByKeyId($"{_peer.A}{alice}#other-key", refresh: false, TestContext.Current.CancellationToken));
}
[Fact]
public async Task Follows_a_same_origin_alias_to_the_actor_it_names()
{
var alice = Unique("alice");
var alias = $"/@alias{Guid.NewGuid():N}";
_peer.Serve(alice, Actor("{A}" + alice));
_peer.Serve(alias, Actor("{A}" + alice));
var actor = await _service.GetActor(_peer.A + alias, refresh: false, TestContext.Current.CancellationToken);
Assert.NotNull(actor);
Assert.Equal(_peer.A + alice, actor.ActorURI);
}
[Fact]
public async Task Resolves_a_gotosocial_style_key_address()
{
var gts = Unique("gts");
var keyPath = gts + "/main-key";
var pem = Keys.NewKeyPair().PublicKeyPem;
_peer.Serve(gts, Actor("{A}" + gts, keyId: "{A}" + keyPath, pem: pem));
_peer.Serve(keyPath, Actor("{A}" + gts, keyId: "{A}" + keyPath, pem: pem));
var actor = await _service.GetActorByKeyId(_peer.A + keyPath, refresh: false, TestContext.Current.CancellationToken);
Assert.NotNull(actor);
Assert.Equal(_peer.A + gts, actor.ActorURI);
Assert.Equal(pem, actor.PublicKey);
}
[Fact]
public async Task A_rotated_key_replaces_the_stored_one_but_only_once_per_interval()
{
var alice = Unique("alice");
var keyId = $"{_peer.A}{alice}#main-key";
var first = Keys.NewKeyPair().PublicKeyPem;
var second = Keys.NewKeyPair().PublicKeyPem;
var third = Keys.NewKeyPair().PublicKeyPem;
_peer.Serve(alice, Actor("{A}" + alice, pem: first));
var token = TestContext.Current.CancellationToken;
Assert.Equal(first, (await _service.GetActorByKeyId(keyId, refresh: false, token)).PublicKey);
_peer.Serve(alice, Actor("{A}" + alice, pem: second));
Assert.Equal(first, (await _service.GetActorByKeyId(keyId, refresh: true, token)).PublicKey);
var fresh = new RemoteActorService(Peer.Http(), new LocalActorService(new DbEntities(),
new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" })),
new MemoryCache(new MemoryCacheOptions()), new DbEntities());
Assert.Equal(second, (await fresh.GetActorByKeyId(keyId, refresh: true, token)).PublicKey);
_peer.Serve(alice, Actor("{A}" + alice, pem: third));
Assert.Equal(second, (await fresh.GetActorByKeyId(keyId, refresh: true, token)).PublicKey);
Assert.Single(await new DbEntities().ForeignAvatars.Match(a => a.ActorURI == _peer.A + alice).ExecuteAsync(token));
}
[Fact]
public async Task Signs_every_fetch()
{
var alice = Unique("alice");
_peer.Serve(alice, Actor("{A}" + alice));
await _service.GetActor(_peer.A + alice, refresh: false, TestContext.Current.CancellationToken);
var request = Assert.Single(_peer.Requests, r => r.Path == alice);
Assert.Contains("keyId=\"https://privapub.test/peasants/privapub#main-key\"", request.Signature);
}
}
public class ActorDocumentTests
{
[Fact]
public void Reads_a_mastodon_actor()
{
var document = ActorDocument.Parse(JsonDocument.Parse("""
{
"@context": ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1"],
"id": "https://mastodon.example/users/alice",
"type": "Person",
"preferredUsername": "alice",
"name": "Alice",
"inbox": "https://mastodon.example/users/alice/inbox",
"outbox": "https://mastodon.example/users/alice/outbox",
"url": "https://mastodon.example/@alice",
"discoverable": false,
"endpoints": { "sharedInbox": "https://mastodon.example/inbox" },
"icon": { "type": "Image", "url": "https://files.mastodon.example/a.png" },
"publicKey": {
"id": "https://mastodon.example/users/alice#main-key",
"owner": "https://mastodon.example/users/alice",
"publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIB\n-----END PUBLIC KEY-----\n"
}
}
""").RootElement);
Assert.Equal("alice", document.PreferredUsername);
Assert.Equal("https://mastodon.example/inbox", document.SharedInbox);
Assert.Equal("https://files.mastodon.example/a.png", document.Icon);
Assert.False(document.Discoverable);
Assert.NotNull(document.Key("https://mastodon.example/users/alice#main-key"));
}
[Fact]
public void Keeps_only_keys_owned_by_the_actor_on_its_origin()
{
var document = ActorDocument.Parse(JsonDocument.Parse("""
{
"id": "https://a.example/users/alice",
"type": "Person",
"publicKey": [
{ "id": "https://a.example/users/alice#main-key", "owner": "https://a.example/users/alice", "publicKeyPem": "x" },
{ "id": "https://b.example/keys/1", "owner": "https://a.example/users/alice", "publicKeyPem": "y" },
{ "id": "https://a.example/users/alice#other", "owner": "https://a.example/users/bob", "publicKeyPem": "z" },
{ "id": "https://a.example/users/alice#unowned", "publicKeyPem": "w" }
]
}
""").RootElement);
Assert.Equal("https://a.example/users/alice#main-key", Assert.Single(document.Keys).Id);
}
[Fact]
public void Refuses_an_inbox_on_another_origin()
{
var document = ActorDocument.Parse(JsonDocument.Parse("""
{ "id": "https://a.example/users/alice", "type": "Person", "inbox": "https://b.example/inbox" }
""").RootElement);
Assert.Null(document.Inbox);
}
[Theory]
[InlineData("Note")]
[InlineData("Tombstone")]
[InlineData("Collection")]
public void Refuses_non_actor_types(string type) =>
Assert.Null(ActorDocument.Parse(JsonDocument.Parse($$"""{ "id": "https://a.example/x", "type": "{{type}}" }""").RootElement));
}
}
+103
View File
@@ -0,0 +1,103 @@
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class RendererTests
{
static readonly LocalActor Alice = new()
{
Id = "a1",
Kind = LocalActorKind.Person,
UserName = "alice",
Name = "Alice",
Summary = "I **like** <b>cats</b>",
PublicKeyPem = Keys.NewKeyPair().PublicKeyPem,
Published = new DateTime(2026, 9, 30, 17, 45, 12, DateTimeKind.Utc),
Fields = new Dictionary<string, string> { ["site"] = "https://alice.example", ["mood"] = "<i>ok</i>" },
BaseAddress = "https://privapub.test"
};
[Fact]
public void An_actor_looks_like_a_mastodon_actor_with_our_route_names()
{
var actor = ActivityPubRenderer.Actor(Alice);
Assert.Equal("https://privapub.test/peasants/alice", actor["id"]!.GetValue<string>());
Assert.Equal("https://privapub.test/@alice", actor["url"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/mouth", actor["inbox"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/anus", actor["outbox"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/groupies", actor["followers"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/stalking", actor["following"]!.GetValue<string>());
Assert.Equal("https://privapub.test/human-centipede", actor["endpoints"]!["sharedInbox"]!.GetValue<string>());
Assert.Equal("2026-09-30T00:00:00Z", actor["published"]!.GetValue<string>());
Assert.Equal("alice@privapub.test", actor["webfinger"]!.GetValue<string>());
Assert.False(actor["indexable"]!.GetValue<bool>());
Assert.Equal("<p>I <strong>like</strong> &lt;b&gt;cats&lt;/b&gt;</p>", actor["summary"]!.GetValue<string>());
var fields = actor["attachment"]!.AsArray();
Assert.Contains("rel=\"nofollow noopener noreferrer me\"", fields[0]!["value"]!.GetValue<string>());
Assert.Equal("&lt;i&gt;ok&lt;/i&gt;", fields[1]!["value"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice#main-key", actor["publicKey"]!["id"]!.GetValue<string>());
Assert.Equal(actor["id"]!.GetValue<string>(), actor["publicKey"]!["owner"]!.GetValue<string>());
}
[Fact]
public void A_titled_post_with_a_warning_keeps_the_title_visible_on_mastodon()
{
var post = new Post
{
ID = "p1",
Title = "On <cats>",
ContentHtml = "<p>body</p>",
HasContentWarning = true,
Language = "en",
Mentions = new() { new PostMention { ActorURI = "https://m.example/users/bob", Handle = "@bob@m.example" } },
Tags = new() { "cats" },
CreationDate = new DateTime(2026, 9, 30, 17, 45, 12, DateTimeKind.Utc)
};
var note = ActivityPubRenderer.Note(post, Alice, default, "https://m.example/users/bob/statuses/1");
Assert.Equal("https://privapub.test/peasants/alice/scribbles/p1", note["id"]!.GetValue<string>());
Assert.Equal("https://privapub.test/@alice/p1", note["url"]!.GetValue<string>());
Assert.Equal("<p><strong>On &lt;cats&gt;</strong></p><p>body</p>", note["content"]!.GetValue<string>());
Assert.Equal("On <cats>", note["summary"]!.GetValue<string>());
Assert.True(note["sensitive"]!.GetValue<bool>());
Assert.Equal("2026-09-30T17:45:12Z", note["published"]!.GetValue<string>());
Assert.Equal(new[] { ActivityPubRenderer.Public }, note["to"]!.AsArray().Select(n => n!.GetValue<string>()));
Assert.Equal(new[] { Alice.Followers, "https://m.example/users/bob" }, note["cc"]!.AsArray().Select(n => n!.GetValue<string>()));
var tags = note["tag"]!.AsArray();
Assert.Equal("@bob@m.example", tags[0]!["name"]!.GetValue<string>());
Assert.Equal("https://privapub.test/tags/cats", tags[1]!["href"]!.GetValue<string>());
Assert.Equal("https://m.example/users/bob/statuses/1", note["inReplyTo"]!.GetValue<string>());
Assert.NotNull(note["contentMap"]!["en"]);
}
[Fact]
public void A_warning_without_a_title_or_spoiler_gets_a_generic_summary()
{
var note = ActivityPubRenderer.Note(new Post { ID = "p2", ContentHtml = "<p>x</p>", HasContentWarning = true }, Alice, default, default);
Assert.Equal(ActivityPubRenderer.ContentWarning, note["summary"]!.GetValue<string>());
}
[Fact]
public void A_direct_note_is_addressed_to_its_recipients_only()
{
var post = new Post { ID = "p3", ContentHtml = "<p>psst</p>", Visibility = PostVisibility.Direct };
var note = ActivityPubRenderer.DirectNote(post, Alice, new[] { ("https://m.example/users/bob", "bob@m.example") },
Alice.ConversationUri("c1"));
Assert.Equal(new[] { "https://m.example/users/bob" }, note["to"]!.AsArray().Select(n => n!.GetValue<string>()));
Assert.Empty(note["cc"]!.AsArray());
Assert.Equal("https://privapub.test/peasants/alice/whispers/c1", note["context"]!.GetValue<string>());
Assert.StartsWith("<p><span class=\"h-card\"", note["content"]!.GetValue<string>());
Assert.Equal("Mention", note["tag"]![0]!["type"]!.GetValue<string>());
}
}
}
@@ -0,0 +1,127 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using PrivaPub.Federation.Moderation;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Federation;
namespace PrivaPub.Tests.Infrastructure
{
public sealed class FederationHttpTests : IAsyncLifetime
{
WebApplication _peer;
string _base;
public async ValueTask InitializeAsync()
{
var builder = WebApplication.CreateSlimBuilder();
builder.WebHost.UseUrls("http://127.0.0.1:0");
_peer = builder.Build();
_peer.MapGet("/actor", () => Results.Text("{\"id\":\"x\"}", "application/activity+json"));
_peer.MapGet("/html", () => Results.Text("<html></html>", "text/html"));
_peer.MapGet("/big", () => Results.Text("{\"a\":\"" + new string('a', FederationHttp.MaxResponseBytes) + "\"}", "application/activity+json"));
_peer.MapGet("/hop/{n:int}", (int n) => Results.Redirect(n == 0 ? "/actor" : $"/hop/{n - 1}"));
_peer.MapGet("/gone", () => Results.StatusCode(410));
await _peer.StartAsync();
_base = _peer.Urls.First();
}
public async ValueTask DisposeAsync() => await _peer.DisposeAsync();
static FederationHttp Client(bool allowTestNetwork = true)
{
var options = new FederationOptions { AllowPrivateNetworks = allowTestNetwork, AllowPlainHttp = allowTestNetwork };
var services = new ServiceCollection();
services.AddHttpClient(FederationHttp.ClientName)
.ConfigurePrimaryHttpMessageHandler(() => SafeHttpHandlerFactory.Create(options));
var provider = services.BuildServiceProvider();
return new FederationHttp(provider.GetRequiredService<IHttpClientFactory>(), new MemoryCache(new MemoryCacheOptions()),
new StaticOptionsMonitor(options), new StaticBlocks(), NullLogger<FederationHttp>.Instance);
}
[Fact]
public async Task Reads_a_json_document()
{
using var fetched = await Client().GetJson($"{_base}/actor", "application/activity+json", default, TestContext.Current.CancellationToken);
Assert.NotNull(fetched);
Assert.Equal("x", fetched.Root.GetProperty("id").GetString());
}
[Fact]
public async Task Follows_up_to_three_redirects_and_reports_the_final_url()
{
using var fetched = await Client().GetJson($"{_base}/hop/2", "application/activity+json", default, TestContext.Current.CancellationToken);
Assert.NotNull(fetched);
Assert.Equal($"{_base}/actor", fetched.FinalUri.ToString());
}
[Fact]
public async Task Refuses_a_fourth_redirect() =>
Assert.Null(await Client().GetJson($"{_base}/hop/3", "application/activity+json", default, TestContext.Current.CancellationToken));
[Fact]
public async Task Refuses_html() =>
Assert.Null(await Client().GetJson($"{_base}/html", "application/activity+json", default, TestContext.Current.CancellationToken));
[Fact]
public async Task Refuses_a_body_over_the_limit() =>
Assert.Null(await Client().GetJson($"{_base}/big", "application/activity+json", default, TestContext.Current.CancellationToken));
[Fact]
public async Task Refuses_an_error_status() =>
Assert.Null(await Client().GetJson($"{_base}/gone", "application/activity+json", default, TestContext.Current.CancellationToken));
[Fact]
public async Task Refuses_a_private_network_in_production_mode() =>
Assert.Null(await Client(allowTestNetwork: false).GetJson($"{_base}/actor", "application/activity+json", default, TestContext.Current.CancellationToken));
[Theory]
[InlineData("https://mastodon.social/users/Gargron", true)]
[InlineData("http://mastodon.social/users/Gargron", false)]
[InlineData("https://user:pass@mastodon.social/", false)]
[InlineData("https://localhost/", false)]
[InlineData("https://printer.local/", false)]
[InlineData("https://metadata.google.internal/", false)]
[InlineData("https://127.0.0.1/", false)]
[InlineData("https://[::1]/", false)]
[InlineData("ftp://example.org/", false)]
public void IsAllowed_takes_https_dns_names_only(string url, bool allowed) =>
Assert.Equal(allowed, Client(allowTestNetwork: false).IsAllowed(new Uri(url)));
[Fact]
public void IsAllowed_refuses_a_suspended_domain_and_its_subdomains()
{
var http = new FederationHttp(new ServiceCollection().AddHttpClient().BuildServiceProvider().GetRequiredService<IHttpClientFactory>(),
new MemoryCache(new MemoryCacheOptions()), new StaticOptionsMonitor(new FederationOptions()), new StaticBlocks("evil.example"),
NullLogger<FederationHttp>.Instance);
Assert.False(http.IsAllowed(new Uri("https://evil.example/users/x")));
Assert.False(http.IsAllowed(new Uri("https://cdn.evil.example/a.png")));
Assert.True(http.IsAllowed(new Uri("https://notevil.example/users/x")));
}
sealed class StaticBlocks : IDomainBlocks
{
readonly string[] _suspended;
public StaticBlocks(params string[] suspended) => _suspended = suspended;
public DomainBlock Find(string host) => _suspended.Any(s => host == s || host.EndsWith("." + s))
? new DomainBlock { Domain = host, Severity = DomainBlockSeverity.Suspend }
: default;
public bool IsSuspended(string host) => Find(host) != default;
public Task Reload(CancellationToken token) => Task.CompletedTask;
}
sealed class StaticOptionsMonitor : IOptionsMonitor<FederationOptions>
{
public StaticOptionsMonitor(FederationOptions value) => CurrentValue = value;
public FederationOptions CurrentValue { get; }
public FederationOptions Get(string name) => CurrentValue;
public IDisposable OnChange(Action<FederationOptions, string> listener) => default;
}
}
}
@@ -0,0 +1,63 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure.Data;
using PrivaPub.Infrastructure.Data.Migrations;
using PrivaPub.Models;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
[Trait("Category", "Integration")]
public class IndexTests
{
[Fact]
public async Task Duplicates_are_removed_and_the_unique_indexes_then_hold()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
await DB.Default.Index<ForeignAvatar>().DropAllAsync(token);
await DB.Default.Index<Follower>().DropAllAsync(token);
var actorUri = $"https://r.example/users/{Guid.NewGuid():N}";
var localId = Guid.NewGuid().ToString("N")[..24];
await DB.Default.SaveAsync(new[]
{
new ForeignAvatar { ActorURI = actorUri, Name = "old", UpdatedAt = DateTime.UtcNow.AddDays(-2) },
new ForeignAvatar { ActorURI = actorUri, Name = "new", UpdatedAt = DateTime.UtcNow }
}, token);
await DB.Default.SaveAsync(new[]
{
new Follower { LocalActorId = localId, ActorURI = actorUri, IsAccepted = false },
new Follower { LocalActorId = localId, ActorURI = actorUri, IsAccepted = true }
}, token);
await new _001_deduplicate_before_unique_indexes().UpgradeAsync();
await Indexes.Create(token);
Assert.Equal("new", (await DB.Default.Find<ForeignAvatar>().Match(a => a.ActorURI == actorUri).ExecuteSingleAsync(token)).Name);
Assert.True((await DB.Default.Find<Follower>().Match(f => f.LocalActorId == localId).ExecuteSingleAsync(token)).IsAccepted);
var duplicate = await Assert.ThrowsAsync<MongoWriteException>(async () =>
await DB.Default.SaveAsync(new ForeignAvatar { ActorURI = actorUri }, token));
Assert.Equal(ServerErrorCategory.DuplicateKey, duplicate.WriteError.Category);
}
[Fact]
public async Task A_username_is_reserved_once_across_personas_and_groups()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var local = new LocalActorService(new DbEntities(), new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" }));
var name = $"n{Guid.NewGuid():N}"[..20];
Assert.True(await local.TryReserveUserName(name, LocalActorKind.Person, "a", token));
Assert.False(await local.TryReserveUserName(name.ToUpperInvariant(), LocalActorKind.Group, "b", token));
Assert.True(await local.IsUserNameTaken(name, token));
Assert.False(await local.TryReserveUserName("admin", LocalActorKind.Person, "c", token));
Assert.False(await local.TryReserveUserName(LocalActorService.InstanceUserName, LocalActorKind.Person, "c", token));
}
}
}
@@ -0,0 +1,86 @@
using System.Net;
using PrivaPub.Infrastructure.Http;
namespace PrivaPub.Tests.Infrastructure
{
public class IpRangeGuardTests
{
[Theory]
[InlineData("1.1.1.1")]
[InlineData("8.8.8.8")]
[InlineData("93.184.215.14")]
[InlineData("172.15.255.255")]
[InlineData("172.32.0.1")]
[InlineData("100.63.255.255")]
[InlineData("100.128.0.0")]
[InlineData("2a01:4f8::1")]
[InlineData("2606:4700:4700::1111")]
[InlineData("::ffff:1.1.1.1")]
public void Public_addresses_are_allowed(string address) =>
Assert.True(IpRangeGuard.IsPublic(IPAddress.Parse(address)));
[Theory]
[InlineData("0.0.0.0")]
[InlineData("10.1.2.3")]
[InlineData("100.64.0.1")]
[InlineData("100.127.255.255")]
[InlineData("127.0.0.1")]
[InlineData("127.255.255.254")]
[InlineData("169.254.169.254")]
[InlineData("172.16.0.1")]
[InlineData("172.31.255.255")]
[InlineData("192.0.0.170")]
[InlineData("192.0.2.1")]
[InlineData("192.88.99.1")]
[InlineData("192.168.1.1")]
[InlineData("198.18.0.1")]
[InlineData("198.51.100.7")]
[InlineData("203.0.113.9")]
[InlineData("224.0.0.251")]
[InlineData("239.255.255.250")]
[InlineData("240.0.0.1")]
[InlineData("255.255.255.255")]
[InlineData("::")]
[InlineData("::1")]
[InlineData("::127.0.0.1")]
[InlineData("::ffff:127.0.0.1")]
[InlineData("::ffff:10.0.0.1")]
[InlineData("::ffff:169.254.169.254")]
[InlineData("64:ff9b::7f00:1")]
[InlineData("64:ff9b::808:808")]
[InlineData("64:ff9b:1::1")]
[InlineData("100::1")]
[InlineData("2001::1")]
[InlineData("2001:0:4136:e378:8000:63bf:3fff:fdd2")]
[InlineData("2001:db8::1")]
[InlineData("2001:10::1")]
[InlineData("2002:7f00:1::1")]
[InlineData("2002:c0a8:101::1")]
[InlineData("3fff::1")]
[InlineData("fc00::1")]
[InlineData("fd12:3456:789a::1")]
[InlineData("fe80::1")]
[InlineData("fec0::1")]
[InlineData("ff02::1")]
public void Private_and_special_addresses_are_refused(string address) =>
Assert.False(IpRangeGuard.IsPublic(IPAddress.Parse(address)));
[Fact]
public async Task Connect_refuses_a_name_that_resolves_to_loopback()
{
await Assert.ThrowsAsync<BlockedDestinationException>(async () =>
await SafeHttpHandlerFactory.Connect(new DnsEndPoint("localhost", 443), allowPrivateNetworks: false, TestContext.Current.CancellationToken));
}
[Theory]
[InlineData("127.0.0.1")]
[InlineData("[::1]")]
[InlineData("169.254.169.254")]
public async Task Connect_refuses_a_private_literal(string host)
{
await Assert.ThrowsAsync<BlockedDestinationException>(async () =>
await SafeHttpHandlerFactory.Connect(new DnsEndPoint(host, 443), allowPrivateNetworks: false, TestContext.Current.CancellationToken));
}
}
}
@@ -0,0 +1,145 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Infrastructure
{
public class BackoffTests
{
[Fact]
public void Grows_like_mastodons()
{
Assert.InRange(Backoff.After(1).TotalSeconds, 16, 16 + 9 * 2);
Assert.InRange(Backoff.After(10).TotalSeconds, 10015, 10015 + 9 * 11);
}
[Fact]
public void Quarantines_a_host_only_past_the_threshold()
{
Assert.Equal(TimeSpan.Zero, Backoff.HostQuarantine(9, 10));
Assert.Equal(TimeSpan.FromHours(1), Backoff.HostQuarantine(10, 10));
Assert.Equal(TimeSpan.FromHours(4), Backoff.HostQuarantine(12, 10));
Assert.Equal(TimeSpan.FromDays(7), Backoff.HostQuarantine(40, 10));
}
}
[Trait("Category", "Integration")]
public sealed class JobQueueTests : IAsyncLifetime
{
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static Job NewJob(string host = default, string dedupe = default) => new()
{
Kind = (JobKind)99,
Host = host,
DedupeKey = dedupe,
Payload = "{}"
};
[Fact]
public async Task A_dedupe_key_is_queued_once()
{
var queue = new JobQueue();
var key = Guid.NewGuid().ToString("N");
var inserted = await queue.EnqueueMany(new[] { NewJob(dedupe: key), NewJob(dedupe: key) }, TestContext.Current.CancellationToken);
Assert.Equal(1, inserted);
}
[Fact]
public async Task A_busy_host_is_skipped_and_a_failed_job_backs_off()
{
var token = TestContext.Current.CancellationToken;
var queue = new JobQueue();
var kind = (JobKind)(100 + Random.Shared.Next(1000));
var busy = $"busy{Guid.NewGuid():N}.example";
var free = $"free{Guid.NewGuid():N}.example";
await queue.EnqueueMany(new[] { new Job { Kind = kind, Host = busy }, new Job { Kind = kind, Host = free } }, token);
var leased = await queue.Lease(kind, new[] { busy }, token);
Assert.Equal(free, leased.Host);
Assert.Equal(1, leased.Attempts);
await queue.Finish(leased, JobOutcome.Retry("boom"), maxAttempts: 3, token);
var after = await DB.Default.Find<Job>().OneAsync(leased.ID, token);
Assert.Equal(JobState.Pending, after.State);
Assert.True(after.RunAt > DateTime.UtcNow.AddSeconds(10));
leased.Attempts = 3;
await queue.Finish(leased, JobOutcome.Retry("boom"), maxAttempts: 3, token);
Assert.Equal(JobState.Dead, (await DB.Default.Find<Job>().OneAsync(leased.ID, token)).State);
}
[Fact]
public async Task An_expired_lease_returns_to_the_queue()
{
var token = TestContext.Current.CancellationToken;
var job = new Job { Kind = (JobKind)98, State = JobState.Running, LeasedUntil = DateTime.UtcNow.AddMinutes(-1) };
await DB.Default.SaveAsync(job, token);
await new JobQueue().Reap(token);
Assert.Equal(JobState.Pending, (await DB.Default.Find<Job>().OneAsync(job.ID, token)).State);
}
[Fact]
public async Task A_dead_host_does_not_hold_up_deliveries_to_live_ones()
{
var token = TestContext.Current.CancellationToken;
var (privateKey, publicKey) = Keys.NewKeyPair();
var avatar = new Avatar { UserName = $"sender{Guid.NewGuid():N}"[..20], PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(avatar, token);
var local = new LocalActorService(new DbEntities(), new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" }));
var sender = local.FromAvatar(avatar);
var queue = new JobQueue();
var delivery = new DeliveryService(new DbEntities(), queue);
_peer.Answer("/dead/inbox", 503, TimeSpan.FromMilliseconds(300));
_peer.Answer("/live/inbox", 202);
var deadInboxes = Enumerable.Range(0, 60).Select(i => $"{_peer.B}/dead/inbox?{i}");
await delivery.Enqueue(sender, deadInboxes, new JsonObject { ["id"] = $"https://privapub.test/a/{Guid.NewGuid():N}", ["type"] = "Create" }, token);
await delivery.Enqueue(sender, new[] { $"{_peer.A}/live/inbox" }, new JsonObject { ["id"] = $"https://privapub.test/a/{Guid.NewGuid():N}", ["type"] = "Create" }, token);
var handler = new DeliveryJobHandler(local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())), NullLogger<DeliveryJobHandler>.Instance);
using var worker = new JobWorker(queue, new IJobHandler[] { handler }, NullLogger<JobWorker>.Instance);
await worker.StartAsync(token);
var deadline = DateTime.UtcNow.AddSeconds(5);
while (DateTime.UtcNow < deadline && !_peer.Requests.Any(r => r.Path == "/live/inbox"))
await Task.Delay(50, token);
var live = Assert.Single(_peer.Requests, r => r.Path == "/live/inbox");
Assert.True(_peer.Requests.Count(r => r.Path == "/dead/inbox") < 10);
while (DateTime.UtcNow < deadline && !await DB.Default.Find<RemoteInstance>().Match(i => i.Host == "localhost").ExecuteAnyAsync(token))
await Task.Delay(50, token);
await worker.StopAsync(token);
Assert.Contains($"keyId=\"{sender.KeyId}\"", live.Signature);
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == "localhost").ExecuteFirstAsync(token);
Assert.True(instance.ConsecutiveFailures > 0);
}
}
}
@@ -0,0 +1,49 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Data.Migrations;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
[Trait("Category", "Integration")]
public class MigrationTests
{
[Fact]
public async Task Stored_remote_content_is_sanitized_and_local_content_rendered()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var remote = new Post { IsFederatedCopy = true, Text = "<p>hi<script>alert(1)</script></p>", ObjectURI = $"https://r.example/{Guid.NewGuid():N}" };
var local = new Post { Text = "**bold** <b>raw</b>", ObjectURI = $"https://privapub.test/{Guid.NewGuid():N}" };
await DB.Default.SaveAsync(new[] { remote, local }, token);
await new _002_sanitize_stored_content().UpgradeAsync();
var migratedRemote = await DB.Default.Find<Post>().OneAsync(remote.ID, token);
var migratedLocal = await DB.Default.Find<Post>().OneAsync(local.ID, token);
Assert.Equal("<p>hi</p>", migratedRemote.Text);
Assert.Equal("<p>hi</p>", migratedRemote.ContentHtml);
Assert.Equal(ContentFormat.Html, migratedRemote.ContentFormat);
Assert.Equal("**bold** <b>raw</b>", migratedLocal.Text);
Assert.Equal("<p><strong>bold</strong> &lt;b&gt;raw&lt;/b&gt;</p>", migratedLocal.ContentHtml);
Assert.Equal(ContentFormat.Markdown, migratedLocal.ContentFormat);
}
[Fact]
public async Task Direct_posts_move_into_posts_with_their_conversation()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var dm = new DmPost { GroupId = "conversation1", Text = "psst", ObjectURI = $"https://privapub.test/{Guid.NewGuid():N}", GroupUserId = "a1" };
await DB.Default.SaveAsync(dm, token);
await new _005_direct_posts_join_posts().UpgradeAsync();
var post = await DB.Default.Find<Post>().OneAsync(dm.ID, token);
Assert.Equal(PostVisibility.Direct, post.Visibility);
Assert.Equal("conversation1", post.ConversationId);
Assert.Equal("a1", post.AuthorAccountId);
}
}
}
@@ -0,0 +1,40 @@
using MongoDB.Bson;
using PrivaPub.Infrastructure.Ids;
using PrivaPub.Models.User;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Infrastructure
{
public class PrivacyIdsTests
{
[Fact]
public void Persona_ids_carry_only_the_day_and_share_nothing_else()
{
var first = (string)new Avatar().GenerateNewID();
var second = (string)new Avatar().GenerateNewID();
Assert.True(ObjectId.TryParse(first, out var a));
Assert.True(ObjectId.TryParse(second, out var b));
Assert.Equal(DateTime.UtcNow.Date, a.CreationTime);
Assert.Equal(a.CreationTime, b.CreationTime);
Assert.NotEqual(first[8..14], second[8..14]);
Assert.Equal(DateTime.UtcNow.Date, ObjectId.Parse((string)new GroupEntity().GenerateNewID()).CreationTime);
}
[Fact]
public void A_remote_post_id_sorts_by_its_published_time()
{
var older = PrivacyIds.At(new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc));
var newer = PrivacyIds.At(new DateTime(2025, 1, 2, 0, 0, 0, DateTimeKind.Utc));
Assert.True(string.CompareOrdinal(older, newer) < 0);
Assert.Equal(new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc), ObjectId.Parse(older).CreationTime);
}
[Fact]
public void A_published_time_in_the_future_is_clamped() =>
Assert.True(ObjectId.Parse(PrivacyIds.At(DateTime.UtcNow.AddYears(50))).CreationTime < DateTime.UtcNow.AddDays(2));
}
}
+29
View File
@@ -0,0 +1,29 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>disable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<OutputType>Exe</OutputType>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.10.1" />
<PackageReference Include="xunit.v3" Version="3.2.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\PrivaPub\PrivaPub.csproj" />
</ItemGroup>
<ItemGroup>
<None Update="Fixtures\**\*" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
</Project>
+44
View File
@@ -0,0 +1,44 @@
using MongoDB.Bson;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Serializers;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Data;
[assembly: AssemblyFixture(typeof(PrivaPub.Tests.Support.MongoFixture))]
namespace PrivaPub.Tests.Support
{
public sealed class MongoFixture : IAsyncLifetime
{
public const string Skip = "set PRIVAPUB_TEST_MONGOD=1 (and optionally PRIVAPUB_TEST_MONGO) to run the tests that need a mongod";
public static bool Enabled => Environment.GetEnvironmentVariable("PRIVAPUB_TEST_MONGOD") == "1";
public string Database { get; } = $"PrivaPubTests_{Guid.NewGuid():N}";
public async ValueTask InitializeAsync()
{
if (!Enabled)
return;
try
{
BsonSerializer.RegisterSerializer(new GuidSerializer(GuidRepresentation.Standard));
}
catch (BsonSerializationException)
{
}
var connection = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_MONGO") ?? "mongodb://127.0.0.1:27017";
await DB.InitAsync(Database, MongoClientSettings.FromConnectionString(connection));
EntityMaps.Warm();
await Indexes.Create();
}
public async ValueTask DisposeAsync()
{
if (Enabled)
await DB.Default.Database().Client.DropDatabaseAsync(Database);
}
}
}
+98
View File
@@ -0,0 +1,98 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using PrivaPub.Federation.Moderation;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Federation;
using System.Collections.Concurrent;
namespace PrivaPub.Tests.Support
{
public sealed class Peer : IAsyncDisposable
{
readonly WebApplication _app;
readonly ConcurrentDictionary<string, string> _documents = new();
readonly ConcurrentDictionary<string, (int Status, TimeSpan Delay)> _answers = new();
public int Port { get; }
public string A => $"http://127.0.0.1:{Port}";
public string B => $"http://localhost:{Port}";
public ConcurrentQueue<HttpRequestRecord> Requests { get; } = new();
Peer(WebApplication app, int port)
{
_app = app;
Port = port;
}
public static async Task<Peer> Start()
{
var builder = WebApplication.CreateSlimBuilder();
builder.WebHost.UseUrls("http://127.0.0.1:0");
var app = builder.Build();
Peer peer = default;
app.Run(async context =>
{
peer.Requests.Enqueue(new(context.Request.Method, context.Request.Path, context.Request.Headers["Signature"].ToString()));
var key = context.Request.Path.Value;
if (peer._answers.TryGetValue(key, out var answer))
{
if (answer.Delay > TimeSpan.Zero)
await Task.Delay(answer.Delay);
context.Response.StatusCode = answer.Status;
return;
}
if (!peer._documents.TryGetValue(key, out var document))
{
context.Response.StatusCode = StatusCodes.Status404NotFound;
return;
}
context.Response.ContentType = "application/activity+json";
await context.Response.WriteAsync(document.Replace("{A}", peer.A).Replace("{B}", peer.B));
});
await app.StartAsync();
peer = new Peer(app, new Uri(app.Urls.First()).Port);
return peer;
}
public void Serve(string path, string json) => _documents[path] = json;
public void Answer(string path, int status, TimeSpan delay = default) => _answers[path] = (status, delay);
public static FederationHttp Http(IMemoryCache cache = default, IDomainBlocks blocks = default)
{
var options = new FederationOptions { AllowPrivateNetworks = true, AllowPlainHttp = true };
var services = new ServiceCollection();
services.AddHttpClient(FederationHttp.ClientName)
.ConfigurePrimaryHttpMessageHandler(() => SafeHttpHandlerFactory.Create(options));
return new FederationHttp(services.BuildServiceProvider().GetRequiredService<IHttpClientFactory>(),
cache ?? new MemoryCache(new MemoryCacheOptions()), new StaticOptions<FederationOptions>(options),
blocks ?? new NoBlocks(), NullLogger<FederationHttp>.Instance);
}
public async ValueTask DisposeAsync() => await _app.DisposeAsync();
}
public sealed class NoBlocks : IDomainBlocks
{
public DomainBlock Find(string host) => default;
public bool IsSuspended(string host) => false;
public Task Reload(CancellationToken token) => Task.CompletedTask;
}
public sealed record HttpRequestRecord(string Method, string Path, string Signature);
public sealed class StaticOptions<T> : IOptionsMonitor<T>
{
public StaticOptions(T value) => CurrentValue = value;
public T CurrentValue { get; }
public T Get(string name) => CurrentValue;
public IDisposable OnChange(Action<T, string> listener) => default;
}
}
+65
View File
@@ -0,0 +1,65 @@
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Features;
using PrivaPub.Federation.Signing;
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Support
{
public sealed class RemoteActor
{
readonly RSA _key = RSA.Create(2048);
public RemoteActor(Peer peer, string name, string origin = default)
{
Name = $"{name}{Guid.NewGuid():N}"[..20];
Id = $"{origin ?? peer.A}/users/{Name}";
peer.Serve($"/users/{Name}", Document().ToJsonString());
}
public string Name { get; }
public string Id { get; }
public string KeyId => Id + "#main-key";
public JsonObject Document() => new()
{
["id"] = Id,
["type"] = "Person",
["preferredUsername"] = Name,
["inbox"] = Id + "/inbox",
["followers"] = Id + "/followers",
["publicKey"] = new JsonObject
{
["id"] = KeyId,
["owner"] = Id,
["publicKeyPem"] = _key.ExportSubjectPublicKeyInfoPem()
}
};
public HttpRequest Post(string host, string path, JsonNode activity)
{
var body = Encoding.UTF8.GetBytes(activity.ToJsonString());
var date = DateTimeOffset.UtcNow.ToString("r", CultureInfo.InvariantCulture);
var digest = HttpSignatures.Digest(body);
var signingString = $"(request-target): post {path}\nhost: {host}\ndate: {date}\ndigest: {digest}";
var signature = Convert.ToBase64String(_key.SignData(Encoding.UTF8.GetBytes(signingString), HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
var context = new DefaultHttpContext();
context.Request.Method = "POST";
context.Request.Host = new HostString(host);
context.Request.Path = path;
context.Features.Get<IHttpRequestFeature>().RawTarget = path;
context.Request.Headers["Date"] = date;
context.Request.Headers["Digest"] = digest;
context.Request.Headers["Content-Type"] = "application/activity+json";
context.Request.Headers["Signature"] = $"keyId=\"{KeyId}\",algorithm=\"rsa-sha256\",headers=\"(request-target) host date digest\",signature=\"{signature}\"";
context.Request.Body = new MemoryStream(body);
context.Request.ContentLength = body.Length;
return context.Request;
}
}
}
+34
View File
@@ -7,20 +7,54 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "PrivaPub", "PrivaPub\PrivaP
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "PrivaPub.ClientModels", "PrivaPub.ClientModels\PrivaPub.ClientModels.csproj", "{5E63C68C-0E60-4418-A53A-FF4EF0007080}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "PrivaPub.Tests", "PrivaPub.Tests\PrivaPub.Tests.csproj", "{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Debug|x64 = Debug|x64
Debug|x86 = Debug|x86
Release|Any CPU = Release|Any CPU
Release|x64 = Release|x64
Release|x86 = Release|x86
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Debug|Any CPU.Build.0 = Debug|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Debug|x64.ActiveCfg = Debug|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Debug|x64.Build.0 = Debug|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Debug|x86.ActiveCfg = Debug|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Debug|x86.Build.0 = Debug|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Release|Any CPU.ActiveCfg = Release|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Release|Any CPU.Build.0 = Release|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Release|x64.ActiveCfg = Release|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Release|x64.Build.0 = Release|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Release|x86.ActiveCfg = Release|Any CPU
{EB2A0BD2-0150-405A-939E-5B0F6F642539}.Release|x86.Build.0 = Release|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Debug|Any CPU.Build.0 = Debug|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Debug|x64.ActiveCfg = Debug|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Debug|x64.Build.0 = Debug|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Debug|x86.ActiveCfg = Debug|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Debug|x86.Build.0 = Debug|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Release|Any CPU.ActiveCfg = Release|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Release|Any CPU.Build.0 = Release|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Release|x64.ActiveCfg = Release|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Release|x64.Build.0 = Release|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Release|x86.ActiveCfg = Release|Any CPU
{5E63C68C-0E60-4418-A53A-FF4EF0007080}.Release|x86.Build.0 = Release|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Debug|x64.ActiveCfg = Debug|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Debug|x64.Build.0 = Debug|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Debug|x86.ActiveCfg = Debug|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Debug|x86.Build.0 = Debug|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Release|Any CPU.Build.0 = Release|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Release|x64.ActiveCfg = Release|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Release|x64.Build.0 = Release|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Release|x86.ActiveCfg = Release|Any CPU
{BA20AFE4-A348-4BDF-A2F4-36E0710D61A2}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
@@ -45,7 +45,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(RemoveUsers)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -73,7 +73,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(BanUsers)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -99,7 +99,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(UnbanUsers)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -122,7 +122,7 @@ namespace PrivaPub.Controllers.ClientToServer
// catch (Exception ex)
// {
// Logger.LogError(ex, $"{nameof(User)}.{nameof(GetUsers)}()");
// return BadRequest(result.Invalidate(ex.Message));
// return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
// }
//}
}
@@ -48,7 +48,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(DataController)}.{nameof(Languages)}()");
return BadRequest(result.Invalidate(ex.Message, exception: ex));
return BadRequest(result.Invalidate("Something went wrong.", exception: ex));
}
}
}
@@ -0,0 +1,74 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Localization;
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.ClientModels.Admin;
using PrivaPub.Federation.Moderation;
using PrivaPub.Models.Federation;
using PrivaPub.Resources;
namespace PrivaPub.Controllers.ClientToServer
{
[ApiController,
Route("clientapi/admin/domainblocks"),
Authorize(Policy = Policies.IsAdmin)]
public class DomainBlockController : ControllerBase
{
readonly IDomainBlocks _domainBlocks;
readonly IStringLocalizer _localizer;
public DomainBlockController(IDomainBlocks domainBlocks, IStringLocalizer<GenericRes> localizer)
{
_domainBlocks = domainBlocks;
_localizer = localizer;
}
[HttpGet, Route("/clientapi/admin/domainblocks/list")]
public async Task<IActionResult> List(CancellationToken token) =>
Ok((await DB.Default.Find<DomainBlock>().Sort(b => b.Domain, Order.Ascending).ExecuteAsync(token)).Select(ToView).ToList());
[HttpPost, Route("/clientapi/admin/domainblocks/insert")]
public async Task<IActionResult> Insert(DomainBlockForm form, CancellationToken token)
{
var domain = DomainBlocks.Normalise(form.Domain);
if (!ModelState.IsValid || Uri.CheckHostName(domain) != UriHostNameType.Dns)
return BadRequest(new WebResult().Invalidate(_localizer["Invalid model."]));
var block = await DB.Default.UpdateAndGet<DomainBlock>()
.Match(b => b.Domain == domain)
.Modify(b => b.Domain, domain)
.Modify(b => b.Severity, form.Suspend ? DomainBlockSeverity.Suspend : DomainBlockSeverity.Silence)
.Modify(b => b.RejectMedia, form.RejectMedia)
.Modify(b => b.PublicComment, form.PublicComment)
.Modify(b => b.PrivateComment, form.PrivateComment)
.Modify(b => b.SetOnInsert(x => x.CreatedAt, DateTime.UtcNow))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
await _domainBlocks.Reload(token);
return Ok(ToView(block));
}
[HttpPost, Route("/clientapi/admin/domainblocks/delete")]
public async Task<IActionResult> Delete([FromQuery] string domain, CancellationToken token)
{
domain = DomainBlocks.Normalise(domain);
await DB.Default.DeleteAsync<DomainBlock>(b => b.Domain == domain);
await _domainBlocks.Reload(token);
return Ok();
}
static ViewDomainBlock ToView(DomainBlock block) => new()
{
Id = block.ID,
Domain = block.Domain,
Severity = block.Severity.ToString(),
RejectMedia = block.RejectMedia,
PublicComment = block.PublicComment,
PrivateComment = block.PrivateComment,
CreatedAt = block.CreatedAt
};
}
}
@@ -52,7 +52,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(PrivateAvatarController)}.{nameof(InsertAvatar)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(_localizer["Something went wrong."]));
}
}
@@ -74,7 +74,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(PrivateAvatarController)}.{nameof(UpdateAvatar)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(_localizer["Something went wrong."]));
}
}
@@ -1,6 +1,7 @@
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.RateLimiting;
using Microsoft.Extensions.Localization;
using Microsoft.Extensions.Options;
@@ -10,6 +11,8 @@ using PrivaPub.ClientModels.Group;
using PrivaPub.ClientModels.User;
using PrivaPub.ClientModels.User.Avatar;
using PrivaPub.Extensions;
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure;
using PrivaPub.Models;
using PrivaPub.Models.User;
using PrivaPub.Resources;
@@ -28,6 +31,7 @@ namespace PrivaPub.Controllers.ClientToServer
readonly IRootUsersService UsersService;
readonly IGroupUsersService GroupUsersService;
readonly IPrivateAvatarUsersService AvatarUsersService;
readonly ILocalActorService LocalActors;
readonly AuthTokenManager AuthTokenManager;
readonly IOptionsMonitor<AppConfiguration> AppConfiguration;
readonly ILogger<RootUserController> Logger;
@@ -36,6 +40,7 @@ namespace PrivaPub.Controllers.ClientToServer
public RootUserController(IRootUsersService usersService,
IGroupUsersService groupUsersService,
IPrivateAvatarUsersService avatarUsersService,
ILocalActorService localActors,
AuthTokenManager authTokenManager,
IOptionsMonitor<AppConfiguration> appConfiguration,
IStringLocalizer<GenericRes> localizer,
@@ -44,6 +49,7 @@ namespace PrivaPub.Controllers.ClientToServer
UsersService = usersService;
GroupUsersService = groupUsersService;
AvatarUsersService = avatarUsersService;
LocalActors = localActors;
AuthTokenManager = authTokenManager;
AppConfiguration = appConfiguration;
Localizer = localizer;
@@ -52,7 +58,7 @@ namespace PrivaPub.Controllers.ClientToServer
#region User endpoints
[HttpPost, Route("/clientapi/user/signup"), Authorize(Policy = Policies.IsUser), AllowAnonymous]
[HttpPost, Route("/clientapi/user/signup"), EnableRateLimiting(RateLimiting.Accounts), Authorize(Policy = Policies.IsUser), AllowAnonymous]
public async Task<IActionResult> SignUp(LoginForm signUpForm)
{
if (User.Identity?.IsAuthenticated ?? false) return Redirect("/");
@@ -67,18 +73,16 @@ namespace PrivaPub.Controllers.ClientToServer
(var user, var userSettings) = ((RootUser, ViewUserSettings))result.Data;
var jwtUser = AuthTokenManager.GenerateToken(user, userSettings);
Logger.LogInformation(
$"{nameof(SignUp)}();IP:[{HttpContext.Connection?.RemoteIpAddress}];\nUser-Agent:[{Request.Headers["User-Agent"]}];\nUserId:[{user.ID}]");
return Ok(jwtUser);
}
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(SignUp)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
[HttpPost, Route("/clientapi/user/login"), Authorize(Policy = Policies.IsUser), AllowAnonymous]
[HttpPost, Route("/clientapi/user/login"), EnableRateLimiting(RateLimiting.Accounts), Authorize(Policy = Policies.IsUser), AllowAnonymous]
public async Task<IActionResult> Login(LoginForm loginForm)
{
if (User.Identity?.IsAuthenticated ?? false) return Redirect("/discussions");
@@ -94,19 +98,17 @@ namespace PrivaPub.Controllers.ClientToServer
var (user, userSettings) =
((RootUser, ViewUserSettings))result.Data;
var jwtUser = AuthTokenManager.GenerateToken(user, userSettings);
Logger.LogInformation(
$"{nameof(Login)}();IP:[{HttpContext.Connection?.RemoteIpAddress}];\nUser-Agent:[{Request.Headers["User-Agent"]}];\nUserId:[{user.ID}]");
return Ok(jwtUser);
}
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(Login)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
[HttpPost, Route("/clientapi/user/invitation/signup"), AllowAnonymous]
[HttpPost, Route("/clientapi/user/invitation/signup"), EnableRateLimiting(RateLimiting.Accounts), AllowAnonymous]
public async Task<IActionResult> InvitationSignUp(InvitationLoginForm signUpForm, CancellationToken token)
{
var result = new WebResult();
@@ -118,6 +120,11 @@ namespace PrivaPub.Controllers.ClientToServer
if (!invitation.IsValid)
return StatusCode(invitation.StatusCode, invitation);
var avatarUserName = signUpForm.AvatarUserName?.Trim().ToLowerInvariant();
var personaProblem = await PersonaProblem(avatarUserName, signUpForm.UserName, token);
if (personaProblem != default)
return BadRequest(result.Invalidate(personaProblem));
result = await UsersService.SignUpAsync(signUpForm, signUpForm.InvitationCode);
if (!result.IsValid)
return StatusCode(result.StatusCode, result);
@@ -126,8 +133,8 @@ namespace PrivaPub.Controllers.ClientToServer
var avatar = await AvatarUsersService.InsertAvatar(new InsertAvatarForm
{
RootId = user.ID,
Name = user.UserName,
UserName = user.UserName,
Name = string.IsNullOrWhiteSpace(signUpForm.AvatarName) ? avatarUserName : signUpForm.AvatarName.Trim(),
UserName = avatarUserName,
Biography = string.Empty
});
if (!avatar.IsValid)
@@ -142,18 +149,16 @@ namespace PrivaPub.Controllers.ClientToServer
if (!joined.IsValid)
return StatusCode(joined.StatusCode, joined);
Logger.LogInformation(
$"{nameof(InvitationSignUp)}();IP:[{HttpContext.Connection?.RemoteIpAddress}];\nUser-Agent:[{Request.Headers["User-Agent"]}];\nUserId:[{user.ID}]");
return Ok(AuthTokenManager.GenerateToken(user, userSettings));
}
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(InvitationSignUp)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
[HttpPost, Route("/clientapi/user/invitation/login"), AllowAnonymous]
[HttpPost, Route("/clientapi/user/invitation/login"), EnableRateLimiting(RateLimiting.Accounts), AllowAnonymous]
public async Task<IActionResult> InvitationLogin(InvitationLoginForm loginForm, CancellationToken token)
{
var result = new WebResult();
@@ -171,14 +176,20 @@ namespace PrivaPub.Controllers.ClientToServer
var (user, userSettings) = ((RootUser, ViewUserSettings))result.Data;
var avatars = await AvatarUsersService.GetRootAvatars(user.ID, token);
var avatar = (avatars.Data as List<ViewAvatar>)?.FirstOrDefault();
var avatarUserName = loginForm.AvatarUserName?.Trim().ToLowerInvariant();
var avatar = string.IsNullOrEmpty(avatarUserName)
? (avatars.Data as List<ViewAvatar>)?.FirstOrDefault()
: (avatars.Data as List<ViewAvatar>)?.FirstOrDefault(a => a.UserName == avatarUserName);
if (avatar == default)
{
var personaProblem = await PersonaProblem(avatarUserName, user.UserName, token);
if (personaProblem != default)
return BadRequest(new WebResult().Invalidate(personaProblem));
var inserted = await AvatarUsersService.InsertAvatar(new InsertAvatarForm
{
RootId = user.ID,
Name = user.UserName,
UserName = user.UserName,
Name = string.IsNullOrWhiteSpace(loginForm.AvatarName) ? avatarUserName : loginForm.AvatarName.Trim(),
UserName = avatarUserName,
Biography = string.Empty
});
if (!inserted.IsValid)
@@ -195,17 +206,26 @@ namespace PrivaPub.Controllers.ClientToServer
if (!joined.IsValid)
return StatusCode(joined.StatusCode, joined);
Logger.LogInformation(
$"{nameof(InvitationLogin)}();IP:[{HttpContext.Connection?.RemoteIpAddress}];\nUser-Agent:[{Request.Headers["User-Agent"]}];\nUserId:[{user.ID}]");
return Ok(AuthTokenManager.GenerateToken(user, userSettings));
}
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(InvitationLogin)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
async Task<string> PersonaProblem(string avatarUserName, string rootUserName, CancellationToken token)
{
if (string.IsNullOrEmpty(avatarUserName))
return Localizer["Choose a username for your persona."];
if (string.Equals(avatarUserName, rootUserName?.Trim(), StringComparison.OrdinalIgnoreCase))
return Localizer["Your persona's username must differ from your login."];
if (await LocalActors.IsUserNameTaken(avatarUserName, token))
return Localizer["The username '{0}' is already take.", avatarUserName];
return default;
}
[HttpGet, Route("/clientapi/user/logout"), Authorize(Policy = Policies.IsUser)]
public IActionResult Logout()
{
@@ -217,7 +237,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(Logout)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -238,7 +258,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(UpdateUser)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -259,7 +279,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(UpdateUserSettings)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -280,7 +300,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(UpdatePassword)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -299,7 +319,7 @@ namespace PrivaPub.Controllers.ClientToServer
// catch (Exception ex)
// {
// Logger.LogError(ex, $"{nameof(User)}.{nameof(GetUser)}()");
// return BadRequest(result.Invalidate(ex.Message));
// return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
// }
//}
@@ -317,11 +337,11 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(GetUserSettings)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
[HttpPost, Route("/clientapi/user/recover/password"), AllowAnonymous]
[HttpPost, Route("/clientapi/user/recover/password"), EnableRateLimiting(RateLimiting.Accounts), AllowAnonymous]
public async Task<IActionResult> RecoverPassword(PasswordRecoveryForm passwordRecoveryForm)
{
var result = new WebResult();
@@ -341,11 +361,11 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(RecoverPassword)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
[HttpPost, Route("/clientapi/user/recover/valid"), AllowAnonymous]
[HttpPost, Route("/clientapi/user/recover/valid"), EnableRateLimiting(RateLimiting.Accounts), AllowAnonymous]
public async Task<IActionResult> IsValidRecoveryCode(
[FromBody,
Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
@@ -364,11 +384,11 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(IsValidRecoveryCode)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
[HttpPost, Route("/clientapi/user/recover/update/password"), AllowAnonymous]
[HttpPost, Route("/clientapi/user/recover/update/password"), EnableRateLimiting(RateLimiting.Accounts), AllowAnonymous]
public async Task<IActionResult> ChangePassword(NewPasswordForm newPasswordForm)
{
var result = new WebResult();
@@ -385,7 +405,7 @@ namespace PrivaPub.Controllers.ClientToServer
catch (Exception ex)
{
Logger.LogError(ex, $"{nameof(User)}.{nameof(ChangePassword)}()");
return BadRequest(result.Invalidate(ex.Message));
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
}
}
@@ -406,7 +426,7 @@ namespace PrivaPub.Controllers.ClientToServer
// catch (Exception ex)
// {
// Logger.LogError(ex, $"{nameof(User)}.{nameof(RemoveSelf)}()");
// return BadRequest(result.Invalidate(ex.Message));
// return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
// }
//}
@@ -1,154 +0,0 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Services.Federation;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Controllers.ServerToServer
{
[ApiController, Route("peasants")]
public class PeasantsController : ControllerBase
{
const string ActivityContentType = "application/activity+json; charset=utf-8";
const int OutboxSize = 20;
readonly ILocalActorService _localActors;
readonly IInboxService _inbox;
readonly DbEntities _dbEntities;
readonly ILogger<PeasantsController> _logger;
public PeasantsController(ILocalActorService localActors, IInboxService inbox, DbEntities dbEntities,
ILogger<PeasantsController> logger)
{
_localActors = localActors;
_inbox = inbox;
_dbEntities = dbEntities;
_logger = logger;
}
[HttpGet, Route("{actor}")]
public async Task<IActionResult> GetActor(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
return local == default ? NotFound() : Activity(ActivityPubRenderer.Actor(local));
}
[HttpGet, Route("{actor}/anus")]
public async Task<IActionResult> Outbox(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local == default)
return NotFound();
var total = local.Kind == LocalActorKind.Group
? await DB.Default.CountAsync<PostEntity>(p => p.GroupId == local.Id, token)
: await DB.Default.CountAsync<PostEntity>(p => p.GroupUserId == local.Id && !p.IsFederatedCopy, token);
var latest = await (local.Kind == LocalActorKind.Group
? _dbEntities.Posts.Match(p => p.GroupId == local.Id)
: _dbEntities.Posts.Match(p => p.GroupUserId == local.Id && !p.IsFederatedCopy))
.Sort(p => p.CreationDate, Order.Descending)
.Limit(OutboxSize)
.ExecuteAsync(token);
var items = new List<JsonNode>();
foreach (var post in latest)
{
if (local.Kind == LocalActorKind.Group)
{
items.Add(ActivityPubRenderer.Announce(local, post.ObjectURI, $"announce-{post.ID}"));
continue;
}
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
var note = ActivityPubRenderer.Note(post, local, group, default);
items.Add(ActivityPubRenderer.Create(local, note, $"create-{post.ID}"));
}
return Activity(ActivityPubRenderer.OrderedCollection(local.Outbox, (int)total, items));
}
[HttpGet, Route("{actor}/followers")]
public async Task<IActionResult> Followers(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local == default)
return NotFound();
var count = await DB.Default.CountAsync<Follower>(
f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted, token);
return Activity(ActivityPubRenderer.OrderedCollection(local.Followers, (int)count, default));
}
[HttpGet, Route("{actor}/following")]
public async Task<IActionResult> Following(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
return local == default ? NotFound() : Activity(ActivityPubRenderer.OrderedCollection(local.Following, 0, Enumerable.Empty<JsonNode>()));
}
[HttpGet, Route("{actor}/posts/{postId}")]
public async Task<IActionResult> Post(string actor, string postId, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local == default || local.Kind != LocalActorKind.Person)
return NotFound();
var post = await _dbEntities.Posts
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy)
.ExecuteFirstAsync(token);
if (post == default)
return NotFound();
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
var replyTo = await ReplyTarget(post.AnsweringToPostId, token);
var note = ActivityPubRenderer.Note(post, local, group, replyTo);
note["@context"] = ActivityPubRenderer.ActivityStreams;
return Activity(note);
}
[HttpPost, Route("{actor}/mouth")]
public async Task<IActionResult> Inbox(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local == default)
return NotFound();
return Answer(await _inbox.Receive(Request, local, token));
}
[HttpPost, Route("{actor}/human-centipede")]
public async Task<IActionResult> ActorSharedInbox(string actor, CancellationToken token) =>
Answer(await _inbox.Receive(Request, default, token));
[HttpPost, Route("/human-centipede")]
public async Task<IActionResult> SharedInbox(CancellationToken token) =>
Answer(await _inbox.Receive(Request, default, token));
async Task<string> ReplyTarget(string answeringToPostId, CancellationToken token)
{
if (string.IsNullOrEmpty(answeringToPostId))
return default;
if (answeringToPostId.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
return answeringToPostId;
return (await _dbEntities.Posts.MatchID(answeringToPostId).ExecuteFirstAsync(token))?.ObjectURI;
}
IActionResult Answer(InboxResult result)
{
if (result.Error != default)
_logger.LogInformation("Inbox refused with {Status}: {Error}", result.StatusCode, result.Error);
return result.Error == default ? StatusCode(result.StatusCode) : StatusCode(result.StatusCode, result.Error);
}
ContentResult Activity(JsonObject document) => new()
{
Content = document.ToJsonString(),
ContentType = ActivityContentType,
StatusCode = StatusCodes.Status200OK
};
}
}
@@ -1,96 +0,0 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Services.Federation;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Controllers.ServerToServer
{
[ApiController]
public class WellKnownController : ControllerBase
{
readonly ILocalActorService _localActors;
readonly DbEntities _dbEntities;
public WellKnownController(ILocalActorService localActors, DbEntities dbEntities)
{
_localActors = localActors;
_dbEntities = dbEntities;
}
[HttpGet, Route("/.well-known/webfinger")]
public async Task<IActionResult> WebFinger([FromQuery] string resource, CancellationToken token)
{
if (string.IsNullOrEmpty(resource))
return BadRequest();
LocalActor actor;
if (resource.StartsWith("acct:", StringComparison.OrdinalIgnoreCase))
{
var parts = resource[5..].TrimStart('@').Split('@');
var domain = new Uri(_localActors.BaseAddress).Authority;
if (parts.Length != 2 || !parts[1].Equals(domain, StringComparison.OrdinalIgnoreCase))
return NotFound();
actor = await _localActors.FindByUserName(parts[0], token);
}
else
actor = await _localActors.FindByUri(resource, token);
if (actor == default)
return NotFound();
var document = new JsonObject
{
["subject"] = $"acct:{actor.Handle}",
["aliases"] = new JsonArray(actor.Uri),
["links"] = new JsonArray(
new JsonObject { ["rel"] = "self", ["type"] = "application/activity+json", ["href"] = actor.Uri },
new JsonObject { ["rel"] = "http://webfinger.net/rel/profile-page", ["type"] = "text/html", ["href"] = actor.Uri })
};
return Content(document.ToJsonString(), "application/jrd+json; charset=utf-8");
}
[HttpGet, Route("/.well-known/nodeinfo")]
public IActionResult NodeInfoLinks()
{
var document = new JsonObject
{
["links"] = new JsonArray(new JsonObject
{
["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.0",
["href"] = $"{_localActors.BaseAddress}/nodeinfo/2.0"
})
};
return Content(document.ToJsonString(), "application/json; charset=utf-8");
}
[HttpGet, Route("/nodeinfo/2.0")]
public async Task<IActionResult> NodeInfo(CancellationToken token)
{
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token);
var posts = await DB.Default.CountAsync<PostEntity>(p => !p.IsFederatedCopy, token);
var document = new JsonObject
{
["version"] = "2.0",
["software"] = new JsonObject { ["name"] = "privapub", ["version"] = BuildInfo.Ref },
["protocols"] = new JsonArray("activitypub"),
["services"] = new JsonObject { ["inbound"] = new JsonArray(), ["outbound"] = new JsonArray() },
["openRegistrations"] = true,
["usage"] = new JsonObject
{
["users"] = new JsonObject { ["total"] = users },
["localPosts"] = posts
},
["metadata"] = new JsonObject()
};
return Content(document.ToJsonString(), "application/json; profile=\"http://nodeinfo.diaspora.software/ns/schema/2.0#\"; charset=utf-8");
}
}
}
+150
View File
@@ -0,0 +1,150 @@
using AngleSharp.Dom;
using AngleSharp.Html.Parser;
using Markdig;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using System.Net;
using System.Text.RegularExpressions;
namespace PrivaPub.Domain.Content
{
public sealed record ResolvedMention(string ActorUri, string UserName, string Domain, string ProfileUrl, bool IsLocal, string AccountId, string Inbox)
{
public string Handle => $"{UserName}@{Domain}";
}
public sealed record RenderedContent(string Html, IReadOnlyList<ResolvedMention> Mentions, IReadOnlyList<string> Tags);
public interface IContentRenderer
{
Task<RenderedContent> Markdown(string text, CancellationToken token);
Task<RenderedContent> PlainText(string text, CancellationToken token);
string TagUrl(string tag);
}
public partial class ContentRenderer : IContentRenderer
{
const int MaxMentions = 20;
static readonly MarkdownPipeline Pipeline = new MarkdownPipelineBuilder().DisableHtml().UseAutoLinks().UseEmphasisExtras().Build();
static readonly HtmlParser Parser = new();
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
public ContentRenderer(ILocalActorService localActors, IRemoteActorService remoteActors)
{
_localActors = localActors;
_remoteActors = remoteActors;
}
public string TagUrl(string tag) => $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag)}";
public Task<RenderedContent> Markdown(string text, CancellationToken token) =>
Decorate(string.IsNullOrWhiteSpace(text) ? string.Empty : Markdig.Markdown.ToHtml(text, Pipeline), token);
public Task<RenderedContent> PlainText(string text, CancellationToken token)
{
if (string.IsNullOrWhiteSpace(text))
return Decorate(string.Empty, token);
var paragraphs = Regex.Split(text.Replace("\r\n", "\n").Trim(), "\n{2,}")
.Select(p => "<p>" + Linkify(WebUtility.HtmlEncode(p)).Replace("\n", "<br>") + "</p>");
return Decorate(string.Concat(paragraphs), token);
}
async Task<RenderedContent> Decorate(string html, CancellationToken token)
{
var document = Parser.ParseDocument($"<body>{html}</body>");
var textNodes = document.Body.Descendants<IText>()
.Where(node => !node.Ancestors<IElement>().Any(e => e.LocalName is "a" or "code" or "pre"))
.ToList();
var handles = textNodes.SelectMany(node => MentionPattern().Matches(node.Data).Select(m => m.Value))
.Distinct(StringComparer.OrdinalIgnoreCase)
.Take(MaxMentions)
.ToList();
var mentions = new Dictionary<string, ResolvedMention>(StringComparer.OrdinalIgnoreCase);
foreach (var handle in handles)
{
var resolved = await Resolve(handle, token);
if (resolved != default)
mentions[handle] = resolved;
}
var tags = new List<string>();
foreach (var node in textNodes)
{
var replaced = Replace(node.Data, mentions, tags);
if (replaced == default)
continue;
var fragment = Parser.ParseFragment(replaced, node.ParentElement);
foreach (var child in fragment.ToList())
node.Parent.InsertBefore(child, node);
node.Parent.RemoveChild(node);
}
return new RenderedContent(
ContentSanitizer.Html(document.Body.InnerHtml),
mentions.Values.DistinctBy(m => m.ActorUri).ToList(),
tags.Distinct().ToList());
}
string Replace(string text, IReadOnlyDictionary<string, ResolvedMention> mentions, List<string> tags)
{
var changed = false;
var encoded = WebUtility.HtmlEncode(text);
encoded = MentionPattern().Replace(encoded, match =>
{
if (!mentions.TryGetValue(WebUtility.HtmlDecode(match.Value), out var mention))
return match.Value;
changed = true;
return $"<span class=\"h-card\" translate=\"no\"><a href=\"{WebUtility.HtmlEncode(mention.ProfileUrl)}\" class=\"u-url mention\">@<span>{WebUtility.HtmlEncode(mention.UserName)}</span></a></span>";
});
encoded = HashtagPattern().Replace(encoded, match =>
{
var tag = match.Groups["tag"].Value;
if (tag.All(char.IsDigit))
return match.Value;
changed = true;
tags.Add(tag.ToLowerInvariant());
return $"{match.Groups["lead"].Value}<a href=\"{WebUtility.HtmlEncode(TagUrl(tag.ToLowerInvariant()))}\" class=\"mention hashtag\" rel=\"tag\">#<span>{tag}</span></a>";
});
return changed ? encoded : default;
}
async Task<ResolvedMention> Resolve(string handle, CancellationToken token)
{
var parts = handle.TrimStart('@').Split('@');
var localDomain = new Uri(_localActors.BaseAddress).Authority;
if (parts.Length == 1 || parts[1].Equals(localDomain, StringComparison.OrdinalIgnoreCase))
{
var local = await _localActors.FindByUserName(parts[0], token);
return local is { IsFederated: true, Kind: not LocalActorKind.Application }
? new ResolvedMention(local.Uri, local.UserName, local.Domain, local.Uri, true, local.Id, local.Inbox)
: default;
}
var actorUri = await _remoteActors.ResolveHandle($"{parts[0]}@{parts[1]}", token);
var actor = actorUri == default ? default : await _remoteActors.GetActor(actorUri, refresh: false, token);
return actor == default
? default
: new ResolvedMention(actor.ActorURI, actor.UserName ?? parts[0], parts[1].ToLowerInvariant(), actor.Url ?? actor.ActorURI, false, actor.ID,
actor.InboxURL);
}
static string Linkify(string encoded) =>
UrlPattern().Replace(encoded, match => $"<a href=\"{match.Value}\">{match.Value}</a>");
[GeneratedRegex(@"(?<![\w@/.])@[a-zA-Z0-9_]+(?:@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)+(?::\d+)?)?(?![\w@])")]
private static partial Regex MentionPattern();
[GeneratedRegex(@"(?<lead>^|[^\w&#/])#(?<tag>[\p{L}\p{N}_]{1,100})(?![\p{L}\p{N}_])")]
private static partial Regex HashtagPattern();
[GeneratedRegex(@"https?://[^\s<]+[^\s<.,;:!?)\]'""]")]
private static partial Regex UrlPattern();
}
}
@@ -0,0 +1,42 @@
using MongoDB.Entities;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using System.Linq.Expressions;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Domain.Privacy
{
public static class VisibilityPolicy
{
public static readonly Expression<Func<Post, bool>> IsPublic = p =>
!p.DeletedAt.HasValue && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted);
static readonly Func<Post, bool> IsPublicCompiled = IsPublic.Compile();
public static async Task<bool> CanSee(Post post, string viewerAvatarId, CancellationToken token)
{
if (post == default || post.DeletedAt.HasValue)
return false;
if (IsPublicCompiled(post))
return true;
if (string.IsNullOrEmpty(viewerAvatarId))
return false;
if (post.GroupUserId == viewerAvatarId || post.Mentions.Any(m => m.IsLocal && m.AccountId == viewerAvatarId))
return true;
return post.Visibility switch
{
PostVisibility.Direct => !string.IsNullOrEmpty(post.ConversationId) && await DB.Default.Find<DmGroup>()
.Match(g => g.ID == post.ConversationId && g.Members.Any(m => !m.IsForeign && m.AvatarId == viewerAvatarId))
.ExecuteAnyAsync(token),
PostVisibility.Circle => !string.IsNullOrEmpty(post.GroupId) && await DB.Default.Find<GroupEntity>()
.Match(g => g.ID == post.GroupId && g.Members.Any(m => !m.IsForeign && m.AvatarId == viewerAvatarId))
.ExecuteAnyAsync(token),
_ => false
};
}
}
}
@@ -0,0 +1,82 @@
using System.Text.Json;
using PrivaPub.Federation.Objects;
namespace PrivaPub.Federation.Actors
{
public sealed record ActorKey(string Id, string Pem);
public sealed class ActorDocument
{
static readonly string[] ActorTypes = { "Person", "Service", "Application", "Group", "Organization" };
public string Id { get; init; }
public string Type { get; init; }
public string PreferredUsername { get; init; }
public string Name { get; init; }
public string Summary { get; init; }
public string Url { get; init; }
public string Inbox { get; init; }
public string Outbox { get; init; }
public string Followers { get; init; }
public string Following { get; init; }
public string SharedInbox { get; init; }
public string Icon { get; init; }
public bool Discoverable { get; init; } = true;
public IReadOnlyList<ActorKey> Keys { get; init; } = Array.Empty<ActorKey>();
public ActorKey Key(string keyId) => Keys.FirstOrDefault(k => k.Id == keyId);
public static ActorDocument Parse(JsonElement root)
{
if (root.ValueKind != JsonValueKind.Object)
return default;
var id = RemoteActorService.Text(root, "id");
var type = RemoteActorService.Text(root, "type");
if (Origin.Of(id) == default || !ActorTypes.Contains(type))
return default;
var inbox = RemoteActorService.Text(root, "inbox");
return new ActorDocument
{
Id = id,
Type = type,
PreferredUsername = RemoteActorService.Text(root, "preferredUsername"),
Name = RemoteActorService.Text(root, "name"),
Summary = RemoteActorService.Text(root, "summary"),
Url = RemoteActorService.Text(root, "url") ?? id,
Inbox = Origin.Same(inbox, id) ? inbox : default,
Outbox = RemoteActorService.Text(root, "outbox"),
Followers = RemoteActorService.Text(root, "followers"),
Following = RemoteActorService.Text(root, "following"),
SharedInbox = root.TryGetProperty("endpoints", out var endpoints) ? RemoteActorService.Text(endpoints, "sharedInbox") : default,
Icon = root.TryGetProperty("icon", out var icon) ? RemoteActorService.Text(icon, "url") : default,
Discoverable = !root.TryGetProperty("discoverable", out var discoverable) || discoverable.ValueKind != JsonValueKind.False,
Keys = ParseKeys(root, id)
};
}
static List<ActorKey> ParseKeys(JsonElement root, string actorId)
{
var keys = new List<ActorKey>();
if (!root.TryGetProperty("publicKey", out var publicKey))
return keys;
var candidates = publicKey.ValueKind switch
{
JsonValueKind.Object => new[] { publicKey },
JsonValueKind.Array => publicKey.EnumerateArray().Where(k => k.ValueKind == JsonValueKind.Object).ToArray(),
_ => Array.Empty<JsonElement>()
};
foreach (var key in candidates)
{
var keyId = RemoteActorService.Text(key, "id");
var owner = RemoteActorService.Text(key, "owner");
var pem = RemoteActorService.Text(key, "publicKeyPem");
if (string.IsNullOrEmpty(pem) || owner != actorId || !Origin.Same(keyId, actorId))
continue;
keys.Add(new ActorKey(keyId, pem));
}
return keys;
}
}
}
@@ -1,9 +1,11 @@
using Microsoft.Extensions.Options;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Models;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
@@ -11,7 +13,7 @@ using System.Security.Cryptography;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Services.Federation
namespace PrivaPub.Federation.Actors
{
public class LocalActor
{
@@ -26,20 +28,25 @@ namespace PrivaPub.Services.Federation
public string PublicKeyPem { get; init; }
public bool Discoverable { get; init; } = true;
public bool ManuallyApprovesFollowers { get; init; }
public bool IsFederated { get; init; } = true;
public DateTime Published { get; init; }
public string BaseAddress { get; init; }
public IReadOnlyDictionary<string, string> Fields { get; init; } = new Dictionary<string, string>();
public string Uri => $"{BaseAddress}/peasants/{UserName}";
public string KeyId => $"{Uri}#main-key";
public string Inbox => $"{Uri}/mouth";
public string Outbox => $"{Uri}/anus";
public string Followers => $"{Uri}/followers";
public string Following => $"{Uri}/following";
public string Followers => $"{Uri}/groupies";
public string Following => $"{Uri}/stalking";
public string SharedInbox => $"{BaseAddress}/human-centipede";
public string Domain => new Uri(BaseAddress).Authority;
public string Handle => $"{UserName}@{Domain}";
public string PostUri(string postId) => $"{Uri}/posts/{postId}";
public string ActivityUri(string activityId) => $"{Uri}/activities/{activityId}";
public string HtmlUrl => $"{BaseAddress}/@{UserName}";
public string PostUri(string postId) => $"{Uri}/scribbles/{postId}";
public string PostHtmlUrl(string postId) => $"{HtmlUrl}/{postId}";
public string ActivityUri(string activityId) => $"{Uri}/grunts/{activityId}";
public string ConversationUri(string conversationId) => $"{Uri}/whispers/{conversationId}";
}
public interface ILocalActorService
@@ -50,6 +57,7 @@ namespace PrivaPub.Services.Federation
Task<LocalActor> FindByUri(string actorUri, CancellationToken token);
Task<LocalActor> GetInstanceActor(CancellationToken token);
Task<bool> IsUserNameTaken(string userName, CancellationToken token);
Task<bool> TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token);
LocalActor FromAvatar(Avatar avatar);
LocalActor FromGroup(GroupEntity group);
}
@@ -58,8 +66,15 @@ namespace PrivaPub.Services.Federation
{
public const string InstanceUserName = "privapub";
static readonly HashSet<string> ReservedByInstance = new(StringComparer.Ordinal)
{
InstanceUserName, "admin", "administrator", "root", "system", "support", "help", "moderator", "mod",
"abuse", "postmaster", "webmaster", "hostmaster", "security", "noreply", "no_reply", "null", "undefined"
};
readonly DbEntities _dbEntities;
readonly IOptionsMonitor<AppConfiguration> _appConfiguration;
InstanceActor _instanceActor;
public LocalActorService(DbEntities dbEntities, IOptionsMonitor<AppConfiguration> appConfiguration)
{
@@ -115,13 +130,7 @@ namespace PrivaPub.Services.Federation
public async Task<LocalActor> GetInstanceActor(CancellationToken token)
{
var instance = await _dbEntities.InstanceActors.ExecuteFirstAsync(token);
if (instance == default)
{
var (privateKey, publicKey) = Keys.NewKeyPair();
instance = new InstanceActor { PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(instance, token);
}
var instance = _instanceActor ??= await LoadInstanceActor(token);
return new LocalActor
{
@@ -138,14 +147,39 @@ namespace PrivaPub.Services.Federation
};
}
async Task<InstanceActor> LoadInstanceActor(CancellationToken token)
{
var instance = await _dbEntities.InstanceActors.Sort(i => i.CreationDate, Order.Ascending).ExecuteFirstAsync(token);
if (instance != default)
return instance;
var (privateKey, publicKey) = Keys.NewKeyPair();
instance = new InstanceActor { PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(instance, token);
return instance;
}
public async Task<bool> IsUserNameTaken(string userName, CancellationToken token)
{
userName = userName?.ToLowerInvariant();
if (string.IsNullOrEmpty(userName) || userName == InstanceUserName)
if (string.IsNullOrEmpty(userName) || ReservedByInstance.Contains(userName))
return true;
if (await _dbEntities.Avatars.Match(a => a.UserName == userName).ExecuteAnyAsync(token))
return await DB.Default.Find<ReservedName>().Match(r => r.Name == userName).ExecuteAnyAsync(token);
}
public async Task<bool> TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token)
{
userName = userName?.ToLowerInvariant();
if (string.IsNullOrEmpty(userName) || ReservedByInstance.Contains(userName))
return false;
try
{
await DB.Default.SaveAsync(new ReservedName { Name = userName, OwnerKind = kind, OwnerId = ownerId }, token);
return true;
return await _dbEntities.Groups.Match(g => g.UserName == userName).ExecuteAnyAsync(token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return false;
}
}
public LocalActor FromAvatar(Avatar avatar) => new()
@@ -160,6 +194,7 @@ namespace PrivaPub.Services.Federation
PrivateKeyPem = avatar.PrivateKey,
PublicKeyPem = avatar.PublicKey,
Published = avatar.CreatedAt,
Fields = avatar.Fields ?? new Dictionary<string, string>(),
BaseAddress = BaseAddress
};
@@ -176,6 +211,7 @@ namespace PrivaPub.Services.Federation
PublicKeyPem = group.PublicKey,
Discoverable = group.IsDiscoverable,
ManuallyApprovesFollowers = group.ManuallyApprovesMembers,
IsFederated = group.Kind == GroupKind.Community,
Published = group.CreationDate,
BaseAddress = BaseAddress
};
@@ -0,0 +1,199 @@
using MongoDB.Entities;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json;
using Microsoft.Extensions.Caching.Memory;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Http;
namespace PrivaPub.Federation.Actors
{
public interface IRemoteActorService
{
Task<FetchedJson> FetchObject(string uri, CancellationToken token);
Task<ForeignAvatar> GetActor(string actorUri, bool refresh, CancellationToken token);
Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token);
Task<string> ResolveHandle(string handle, CancellationToken token);
}
public class RemoteActorService : IRemoteActorService
{
public const string ActivityJson = "application/activity+json";
const string Accept = "application/activity+json, application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"";
static readonly TimeSpan CacheLifetime = TimeSpan.FromDays(1);
static readonly TimeSpan RefetchInterval = TimeSpan.FromMinutes(5);
readonly IFederationHttp _http;
readonly ILocalActorService _localActors;
readonly IMemoryCache _cache;
readonly DbEntities _dbEntities;
public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities)
{
_http = http;
_localActors = localActors;
_cache = cache;
_dbEntities = dbEntities;
}
public async Task<FetchedJson> FetchObject(string uri, CancellationToken token)
{
var signer = await _localActors.GetInstanceActor(token);
var fetched = await Get(uri, signer, token);
if (fetched == default)
return default;
var id = Text(fetched.Root, "id");
if (Origin.IsDocumentAt(id, fetched.FinalUri))
return fetched;
var finalUri = fetched.FinalUri;
fetched.Dispose();
if (!Origin.Same(id, finalUri.AbsoluteUri))
return default;
var named = await Get(id, signer, token);
if (named != default && Origin.IsDocumentAt(Text(named.Root, "id"), named.FinalUri))
return named;
named?.Dispose();
return default;
}
public async Task<ForeignAvatar> GetActor(string actorUri, bool refresh, CancellationToken token)
{
if (string.IsNullOrEmpty(actorUri))
return default;
actorUri = StripFragment(actorUri);
var cached = await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == actorUri).ExecuteFirstAsync(token);
if (cached != default && !refresh && DateTime.UtcNow - cached.UpdatedAt < CacheLifetime)
return cached;
if (!MayFetch(actorUri))
return cached;
using var fetched = await FetchObject(actorUri, token);
var actor = fetched == default ? default : ActorDocument.Parse(fetched.Root);
if (actor == default)
return cached;
var key = cached == default ? default : actor.Key(cached.PublicKeyId);
return await Upsert(actor, key ?? actor.Keys.FirstOrDefault(), token);
}
public async Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token)
{
if (Origin.Of(keyId) == default)
return default;
var cached = await _dbEntities.ForeignAvatars.Match(a => a.PublicKeyId == keyId).ExecuteFirstAsync(token);
if (cached != default && !string.IsNullOrEmpty(cached.PublicKey) && !refresh)
return cached;
if (!MayFetch(keyId))
return cached;
using var fetched = await FetchObject(StripFragment(keyId), token);
if (fetched == default)
return cached;
var actor = ActorDocument.Parse(fetched.Root);
if (actor == default)
{
var owner = Text(fetched.Root, "owner");
if (Text(fetched.Root, "id") != keyId || !Origin.Same(owner, keyId))
return default;
using var ownerDocument = await FetchObject(owner, token);
actor = ownerDocument == default ? default : ActorDocument.Parse(ownerDocument.Root);
}
var key = actor?.Key(keyId);
if (key == default)
return default;
return await Upsert(actor, key, token);
}
public async Task<string> ResolveHandle(string handle, CancellationToken token)
{
var parts = handle?.TrimStart('@').Split('@');
if (parts is not { Length: 2 } || string.IsNullOrEmpty(parts[0]) || string.IsNullOrEmpty(parts[1]))
return default;
var url = $"https://{parts[1]}/.well-known/webfinger?resource={Uri.EscapeDataString($"acct:{parts[0]}@{parts[1]}")}";
using var fetched = await _http.GetJson(url, "application/jrd+json, application/json", sign: default, token);
if (fetched == default)
return default;
var document = fetched.Document;
if (!document.RootElement.TryGetProperty("links", out var links) || links.ValueKind != JsonValueKind.Array)
return default;
foreach (var link in links.EnumerateArray())
{
var type = Text(link, "type") ?? string.Empty;
if (Text(link, "rel") == "self" && (type.Contains("activity+json") || type.Contains("ld+json")))
return Text(link, "href");
}
return default;
}
async Task<FetchedJson> Get(string uri, LocalActor signer, CancellationToken token) =>
await _http.GetJson(uri, Accept, request => HttpSignatures.Sign(request, signer, body: null), token);
bool MayFetch(string uri)
{
var key = "remote-actor:fetched:" + uri;
if (_cache.TryGetValue(key, out _))
return false;
_cache.Set(key, true, RefetchInterval);
return true;
}
static async Task<ForeignAvatar> Upsert(ActorDocument actor, ActorKey key, CancellationToken token)
{
var now = DateTime.UtcNow;
return await DB.Default.UpdateAndGet<ForeignAvatar>()
.Match(a => a.ActorURI == actor.Id)
.Modify(a => a.UserName, actor.PreferredUsername)
.Modify(a => a.Name, actor.Name)
.Modify(a => a.Biography, ContentSanitizer.Html(actor.Summary))
.Modify(a => a.Url, actor.Url)
.Modify(a => a.Domain, new Uri(actor.Id).Authority)
.Modify(a => a.InboxURL, actor.Inbox)
.Modify(a => a.OutboxURL, actor.Outbox)
.Modify(a => a.FollowersURL, actor.Followers)
.Modify(a => a.FollowingURL, actor.Following)
.Modify(a => a.SharedInboxURL, actor.SharedInbox)
.Modify(a => a.PictureURL, actor.Icon)
.Modify(a => a.IsDiscoverable, actor.Discoverable)
.Modify(a => a.AvatarType, Enum.TryParse<AvatarType>(actor.Type, out var type) ? type : AvatarType.Person)
.Modify(a => a.PublicKeyId, key?.Id)
.Modify(a => a.PublicKey, key?.Pem)
.Modify(a => a.UpdatedAt, now)
.Modify(b => b.SetOnInsert(a => a.CreatedAt, now))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
}
public static string StripFragment(string uri)
{
var hash = uri.IndexOf('#');
return hash < 0 ? uri : uri[..hash];
}
public static string Text(JsonElement element, string property)
{
if (element.ValueKind != JsonValueKind.Object || !element.TryGetProperty(property, out var value))
return default;
return value.ValueKind switch
{
JsonValueKind.String => value.GetString(),
JsonValueKind.Object => Text(value, "id") ?? Text(value, "href"),
JsonValueKind.Array => value.EnumerateArray().Select(v => v.ValueKind == JsonValueKind.String ? v.GetString() : Text(v, "id")).FirstOrDefault(v => v != null),
_ => default
};
}
}
}
@@ -0,0 +1,191 @@
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.RateLimiting;
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Federation.Inbox;
using PrivaPub.Infrastructure;
namespace PrivaPub.Federation.Controllers
{
[ApiController, Route("peasants")]
public class PeasantsController : ControllerBase
{
const string ActivityContentType = "application/activity+json; charset=utf-8";
const int OutboxSize = 20;
readonly ILocalActorService _localActors;
readonly IInboxReceiver _inbox;
readonly DbEntities _dbEntities;
readonly ILogger<PeasantsController> _logger;
public PeasantsController(ILocalActorService localActors, IInboxReceiver inbox, DbEntities dbEntities,
ILogger<PeasantsController> logger)
{
_localActors = localActors;
_inbox = inbox;
_dbEntities = dbEntities;
_logger = logger;
}
[HttpGet, Route("{actor}")]
public async Task<IActionResult> GetActor(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local is not { IsFederated: true })
return NotFound();
if (WantsHtml() && local.Kind != LocalActorKind.Application)
return Redirect(local.HtmlUrl);
return Activity(ActivityPubRenderer.Actor(local));
}
[HttpGet, Route("{actor}/anus")]
public async Task<IActionResult> Outbox(string actor, [FromQuery] bool page, [FromQuery(Name = "max_id")] string maxId,
CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local is not { IsFederated: true })
return NotFound();
if (!page)
{
var total = local.Kind == LocalActorKind.Group
? await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupId == local.Id) & f.Where(VisibilityPolicy.IsPublic), token)
: await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupUserId == local.Id && !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token);
return Activity(ActivityPubRenderer.OrderedCollection(local.Outbox, (int)total, default, $"{local.Outbox}?page=true"));
}
var query = local.Kind == LocalActorKind.Group
? _dbEntities.Posts.Match(p => p.GroupId == local.Id).Match(VisibilityPolicy.IsPublic)
: _dbEntities.Posts.Match(p => p.GroupUserId == local.Id && !p.IsFederatedCopy).Match(VisibilityPolicy.IsPublic);
if (!string.IsNullOrEmpty(maxId))
query.Match(f => f.Lt(p => p.ID, maxId));
var latest = await query.Sort(p => p.ID, Order.Descending).Limit(OutboxSize).ExecuteAsync(token);
var items = new List<JsonNode>();
foreach (var post in latest)
{
if (local.Kind == LocalActorKind.Group)
{
items.Add(ActivityPubRenderer.Announce(local, post.ObjectURI, $"announce-{post.ID}"));
continue;
}
items.Add(await CreateFor(post, local, token));
}
var pageId = string.IsNullOrEmpty(maxId) ? $"{local.Outbox}?page=true" : $"{local.Outbox}?page=true&max_id={maxId}";
var next = latest.Count == OutboxSize ? $"{local.Outbox}?page=true&max_id={latest[^1].ID}" : default;
return Activity(ActivityPubRenderer.OrderedCollectionPage(pageId, local.Outbox, items, next, default));
}
[HttpGet, Route("{actor}/groupies")]
public async Task<IActionResult> Followers(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local is not { IsFederated: true })
return NotFound();
var count = await DB.Default.CountAsync<Follower>(
f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted, token);
return Activity(ActivityPubRenderer.OrderedCollection(local.Followers, (int)count, default));
}
[HttpGet, Route("{actor}/stalking")]
public async Task<IActionResult> Following(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
return local is not { IsFederated: true } ? NotFound() : Activity(ActivityPubRenderer.OrderedCollection(local.Following, 0, default));
}
[HttpGet, Route("{actor}/scribbles/{postId}")]
public async Task<IActionResult> Post(string actor, string postId, CancellationToken token)
{
var (local, post) = await PublicPost(actor, postId, token);
if (post == default)
return NotFound();
if (WantsHtml())
return Redirect(local.PostHtmlUrl(post.ID));
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
var note = ActivityPubRenderer.Note(post, local, group, post.InReplyToURI);
note["@context"] = ActivityPubRenderer.Context();
return Activity(note);
}
[HttpGet, Route("{actor}/grunts/{activityId}")]
public async Task<IActionResult> Grunt(string actor, string activityId, CancellationToken token)
{
if (!activityId.StartsWith("create-", StringComparison.Ordinal))
return NotFound();
var (local, post) = await PublicPost(actor, activityId["create-".Length..], token);
return post == default ? NotFound() : Activity(await CreateFor(post, local, token));
}
[HttpPost, Route("{actor}/mouth"), EnableRateLimiting(RateLimiting.Inbox)]
public async Task<IActionResult> Inbox(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local is not { IsFederated: true })
return NotFound();
return Answer(await _inbox.Receive(Request, local, token));
}
[HttpPost, Route("{actor}/human-centipede"), EnableRateLimiting(RateLimiting.Inbox)]
public async Task<IActionResult> ActorSharedInbox(string actor, CancellationToken token) =>
Answer(await _inbox.Receive(Request, default, token));
[HttpPost, Route("/human-centipede"), EnableRateLimiting(RateLimiting.Inbox)]
public async Task<IActionResult> SharedInbox(CancellationToken token) =>
Answer(await _inbox.Receive(Request, default, token));
async Task<(LocalActor Actor, PostEntity Post)> PublicPost(string actor, string postId, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
if (local is not { IsFederated: true, Kind: LocalActorKind.Person })
return default;
var post = await _dbEntities.Posts
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy)
.Match(VisibilityPolicy.IsPublic)
.ExecuteFirstAsync(token);
return (local, post);
}
async Task<JsonObject> CreateFor(PostEntity post, LocalActor author, CancellationToken token)
{
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
var note = ActivityPubRenderer.Note(post, author, group, post.InReplyToURI);
return ActivityPubRenderer.Create(author, note, $"create-{post.ID}");
}
bool WantsHtml()
{
var accept = Request.Headers.Accept.ToString();
return accept.Contains("text/html", StringComparison.OrdinalIgnoreCase)
&& !accept.Contains("activity+json", StringComparison.OrdinalIgnoreCase)
&& !accept.Contains("ld+json", StringComparison.OrdinalIgnoreCase);
}
IActionResult Answer(InboxResult result)
{
if (result.Error != default)
_logger.LogInformation("Inbox refused with {Status}: {Error}", result.StatusCode, result.Error);
return result.Error == default ? StatusCode(result.StatusCode) : StatusCode(result.StatusCode, result.Error);
}
ContentResult Activity(JsonObject document) => new()
{
Content = document.ToJsonString(),
ContentType = ActivityContentType,
StatusCode = StatusCodes.Status200OK
};
}
}
@@ -1,6 +1,6 @@
using Microsoft.AspNetCore.Mvc;
namespace PrivaPub.Controllers.ServerToServer
namespace PrivaPub.Federation.Controllers
{
[ApiController, Route("users")]
public class UsersController : ControllerBase
@@ -0,0 +1,118 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors;
namespace PrivaPub.Federation.Controllers
{
[ApiController]
public class WellKnownController : ControllerBase
{
readonly ILocalActorService _localActors;
readonly DbEntities _dbEntities;
public WellKnownController(ILocalActorService localActors, DbEntities dbEntities)
{
_localActors = localActors;
_dbEntities = dbEntities;
}
[HttpGet, Route("/.well-known/webfinger")]
public async Task<IActionResult> WebFinger([FromQuery] string resource, CancellationToken token)
{
if (string.IsNullOrEmpty(resource))
return BadRequest();
LocalActor actor;
if (resource.StartsWith("acct:", StringComparison.OrdinalIgnoreCase))
{
var parts = resource[5..].TrimStart('@').Split('@');
var domain = new Uri(_localActors.BaseAddress).Authority;
if (parts.Length != 2 || !parts[1].Equals(domain, StringComparison.OrdinalIgnoreCase))
return NotFound();
actor = await _localActors.FindByUserName(parts[0], token);
}
else
actor = await _localActors.FindByUri(resource, token);
if (actor is not { IsFederated: true })
return NotFound();
var document = new JsonObject
{
["subject"] = $"acct:{actor.Handle}",
["aliases"] = actor.Kind == LocalActorKind.Application ? new JsonArray(actor.Uri) : new JsonArray(actor.HtmlUrl, actor.Uri),
["links"] = actor.Kind == LocalActorKind.Application
? new JsonArray(new JsonObject { ["rel"] = "self", ["type"] = "application/activity+json", ["href"] = actor.Uri })
: new JsonArray(
new JsonObject { ["rel"] = "http://webfinger.net/rel/profile-page", ["type"] = "text/html", ["href"] = actor.HtmlUrl },
new JsonObject { ["rel"] = "self", ["type"] = "application/activity+json", ["href"] = actor.Uri })
};
return Content(document.ToJsonString(), "application/jrd+json; charset=utf-8");
}
[HttpGet, Route("/.well-known/nodeinfo")]
public IActionResult NodeInfoLinks()
{
var document = new JsonObject
{
["links"] = new JsonArray(
new JsonObject
{
["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.1",
["href"] = $"{_localActors.BaseAddress}/nodeinfo/2.1"
},
new JsonObject
{
["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.0",
["href"] = $"{_localActors.BaseAddress}/nodeinfo/2.0"
})
};
return Content(document.ToJsonString(), "application/json; charset=utf-8");
}
[HttpGet, Route("/nodeinfo/{version:regex(^2\\.[[01]]$)}")]
public async Task<IActionResult> NodeInfo(string version, CancellationToken token)
{
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token);
var posts = await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token);
var software = new JsonObject { ["name"] = "privapub", ["version"] = BuildInfo.Ref };
if (version == "2.1")
{
software["repository"] = "https://git.thepra.dev/thepra/SocialPub";
software["homepage"] = "https://git.thepra.dev/thepra/SocialPub";
}
var document = new JsonObject
{
["version"] = version,
["software"] = software,
["protocols"] = new JsonArray("activitypub"),
["services"] = new JsonObject { ["inbound"] = new JsonArray(), ["outbound"] = new JsonArray() },
["openRegistrations"] = true,
["usage"] = new JsonObject
{
["users"] = new JsonObject { ["total"] = users },
["localPosts"] = posts
},
["metadata"] = new JsonObject
{
["nodeName"] = "PrivaPub",
["nodeDescription"] = "A small ActivityPub server where one private login keeps several unlinkable public personas.",
["federation"] = new JsonObject { ["document"] = "https://git.thepra.dev/thepra/SocialPub/src/branch/master/FEDERATION.md" }
}
};
return Content(document.ToJsonString(),
$"application/json; profile=\"http://nodeinfo.diaspora.software/ns/schema/{version}#\"; charset=utf-8");
}
}
}
@@ -0,0 +1,37 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Federation.Inbox
{
public static class ForeignMembers
{
public static async Task<bool> IsAcceptedFollower(LocalActor group, string actorUri, CancellationToken token) =>
await DB.Default.Find<Follower>()
.Match(f => f.LocalActorId == group.Id && f.LocalActorKind == LocalActorKind.Group && f.ActorURI == actorUri && f.IsAccepted)
.ExecuteAnyAsync(token);
public static async Task AddForeignMember(string groupId, string actorUri, CancellationToken token)
{
var group = await DB.Default.Find<GroupEntity>().MatchID(groupId).ExecuteFirstAsync(token);
if (group == default || group.Members.Any(m => m.IsForeign && m.AvatarId == actorUri))
return;
group.Members.Add(new GroupMember { AvatarId = actorUri, IsForeign = true });
group.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(group, token);
}
public static async Task RemoveForeignMember(string groupId, string actorUri, CancellationToken token)
{
var group = await DB.Default.Find<GroupEntity>().MatchID(groupId).ExecuteFirstAsync(token);
if (group == default || group.Members.RemoveAll(m => m.IsForeign && m.AvatarId == actorUri) == 0)
return;
group.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync<GroupEntity>(group, token);
}
}
}
@@ -0,0 +1,201 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Ids;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Inbox.ForeignMembers;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Inbox.Handlers
{
public class CreateHandler : IActivityHandler
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IDeliveryService _delivery;
readonly IDomainBlocks _domainBlocks;
public CreateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
IDomainBlocks domainBlocks)
{
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_delivery = delivery;
_domainBlocks = domainBlocks;
}
public string Type => "Create";
public async Task Handle(JsonNode activity, ForeignAvatar author, CancellationToken token)
{
var node = activity["object"];
if (node is not JsonObject || !Origin.Same(Id(node), author.ActorURI))
{
using var fetched = await _remoteActors.FetchObject(Id(node), token);
node = fetched == default ? default : JsonNode.Parse(fetched.Root.GetRawText());
}
var note = NoteParser.Parse(node);
if (note == default || note.AttributedTo != author.ActorURI)
return;
if (await _dbEntities.Posts.Match(p => p.ObjectURI == note.Id).ExecuteAnyAsync(token))
return;
var to = note.To.Concat(Strings(activity["to"])).Distinct(StringComparer.Ordinal).ToList();
var cc = note.Cc.Concat(Strings(activity["cc"])).Distinct(StringComparer.Ordinal).ToList();
var visibility = Addressing.Classify(to, cc, author.FollowersURL);
var addressed = to.Concat(cc)
.Concat(Addresses(activity))
.Concat(note.Mentions.Select(m => m.ActorURI))
.Append(note.Audience)
.Where(a => a != default && !Addressing.IsPublic(a) && a != author.FollowersURL)
.Distinct(StringComparer.Ordinal)
.ToList();
var localTargets = new List<LocalActor>();
foreach (var uri in addressed)
{
var local = await _localActors.FindByUri(uri, token);
if (local is { IsFederated: true } && localTargets.All(l => l.Id != local.Id))
localTargets.Add(local);
}
var persons = localTargets.Where(t => t.Kind == LocalActorKind.Person).ToList();
var parent = string.IsNullOrEmpty(note.InReplyTo)
? default
: await _dbEntities.Posts.Match(p => p.ObjectURI == note.InReplyTo && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
var group = visibility is PostVisibility.Public or PostVisibility.Unlisted
? localTargets.FirstOrDefault(t => t.Kind == LocalActorKind.Group)
: default;
if (group != default && !await IsAcceptedFollower(group, author.ActorURI, token))
group = default;
var repliesToLocal = parent is { IsFederatedCopy: false } && visibility != PostVisibility.Direct;
if (visibility == PostVisibility.Direct ? persons.Count == 0 : group == default && persons.Count == 0 && !repliesToLocal)
return;
var mentions = new List<PostMention>();
foreach (var mention in note.Mentions)
{
var local = await _localActors.FindByUri(mention.ActorURI, token);
mentions.Add(new PostMention
{
ActorURI = mention.ActorURI,
Handle = mention.Handle,
IsLocal = local != default,
AccountId = local?.Id
});
}
DmGroup conversation = default;
if (visibility == PostVisibility.Direct)
{
var participants = persons.Select(p => p.Uri)
.Concat(addressed.Where(a => !IsCollection(a)))
.Append(author.ActorURI)
.ToList();
conversation = await FindOrCreateConversation(participants, Origin.Same(note.Context, author.ActorURI) ? note.Context : default, token);
}
var post = new PostEntity
{
ID = PrivacyIds.At(note.Published),
ObjectURI = note.Id,
ActivityURI = Id(activity),
ActorURI = author.ActorURI,
AuthorAccountId = author.ID,
Visibility = visibility,
To = to,
Cc = cc,
Url = note.Url,
ContextURI = note.Context,
QuoteURI = note.QuoteUri,
GroupId = group?.Id,
ConversationId = conversation?.ID,
Title = note.Title,
SpoilerText = note.SpoilerText,
HasContentWarning = note.Sensitive,
Text = note.ContentHtml,
ContentHtml = note.ContentHtml,
ContentFormat = ContentFormat.Html,
Language = note.Language,
Mentions = mentions,
Tags = note.Tags.ToList(),
Media = _domainBlocks.Find(new Uri(author.ActorURI).Host)?.RejectMedia == true ? new() : note.Attachments.ToList(),
InReplyToURI = note.InReplyTo,
AnsweringToPostId = parent?.ID,
InReplyToAccountId = parent?.AuthorAccountId ?? parent?.GroupUserId,
IsFederatedCopy = true,
CreationDate = note.Published,
UpdateDate = note.Updated ?? note.Published,
EditedAt = note.Updated
};
try
{
await DB.Default.SaveAsync(post, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return;
}
if (parent != default)
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
if (conversation != default)
await DB.Default.Update<DmGroup>().MatchID(conversation.ID).Modify(g => g.UpdatedAt, DateTime.UtcNow).ExecuteAsync(token);
if (group != default)
{
var announce = ActivityPubRenderer.Announce(group, note.Id, $"announce-{post.ID}");
await _delivery.EnqueueToFollowers(group, announce, token);
}
}
async Task<DmGroup> FindOrCreateConversation(List<string> participantUris, string context, CancellationToken token)
{
var members = new List<GroupMember>();
foreach (var uri in participantUris.Distinct(StringComparer.Ordinal))
{
var local = await _localActors.FindByUri(uri, token);
var member = local != default && local.Kind == LocalActorKind.Person
? new GroupMember { AvatarId = local.Id }
: new GroupMember { AvatarId = uri, IsForeign = true };
if (members.All(m => m.AvatarId != member.AvatarId || m.IsForeign != member.IsForeign))
members.Add(member);
}
var key = DmGroup.KeyOf(members);
var match = await _dbEntities.DmGroups.Match(g => g.ParticipantsKey == key && !g.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (match != default)
return match;
var conversation = new DmGroup { Members = members, ConversationURI = context, ParticipantsKey = key };
await DB.Default.SaveAsync(conversation, token);
return conversation;
}
static bool IsCollection(string uri) =>
uri.EndsWith("/followers", StringComparison.Ordinal) || uri.EndsWith("/following", StringComparison.Ordinal);
static IEnumerable<string> Strings(JsonNode node) => node switch
{
JsonArray array => array.Select(Id).Where(id => id != default),
JsonNode single when Id(single) is { } id => new[] { id },
_ => Enumerable.Empty<string>()
};
}
}
@@ -0,0 +1,66 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Inbox.ForeignMembers;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Inbox.Handlers
{
public class DeleteHandler : IActivityHandler
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IDeliveryService _delivery;
public DeleteHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery)
{
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_delivery = delivery;
}
public string Type => "Delete";
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
{
var delete = activity;
var objectUri = Id(delete["object"]);
if (!Origin.Same(objectUri, actor.ActorURI))
return;
if (objectUri == actor.ActorURI)
{
actor.DeletionAt = DateTime.UtcNow;
actor.AccountState = AvatarAccountState.Deleted;
await DB.Default.SaveAsync(actor, token);
var followers = await _dbEntities.Followers.Match(f => f.ActorURI == actor.ActorURI).ExecuteAsync(token);
foreach (var follower in followers)
{
await DB.Default.DeleteAsync<Follower>(follower.ID);
if (follower.LocalActorKind == LocalActorKind.Group)
await RemoveForeignMember(follower.LocalActorId, actor.ActorURI, token);
}
return;
}
await DB.Default.DeleteAsync<PostEntity>(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI);
return;
}
}
}
@@ -0,0 +1,74 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Inbox.ForeignMembers;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Inbox.Handlers
{
public class FollowHandler : IActivityHandler
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IDeliveryService _delivery;
public FollowHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery)
{
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_delivery = delivery;
}
public string Type => "Follow";
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
{
var follow = activity;
var follower = actor;
var target = await _localActors.FindByUri(Id(follow["object"]), token);
if (target is not { IsFederated: true } || target.Kind == LocalActorKind.Application)
return;
var existing = await _dbEntities.Followers
.Match(f => f.LocalActorId == target.Id && f.LocalActorKind == target.Kind && f.ActorURI == follower.ActorURI)
.ExecuteFirstAsync(token);
var record = existing ?? new Follower
{
LocalActorId = target.Id,
LocalActorKind = target.Kind,
ActorURI = follower.ActorURI
};
record.InboxURL = follower.InboxURL;
record.SharedInboxURL = follower.SharedInboxURL;
record.FollowActivityURI = Id(follow);
record.IsAccepted = existing?.IsAccepted == true || !target.ManuallyApprovesFollowers;
await DB.Default.SaveAsync(record, token);
if (!record.IsAccepted)
return;
if (target.Kind == LocalActorKind.Group)
await AddForeignMember(target.Id, follower.ActorURI, token);
var accept = ActivityPubRenderer.Accept(target, follow, $"accept-{record.ID}-{DateTime.UtcNow.Ticks}");
await _delivery.Enqueue(target, new[] { follower.InboxURL }, accept, token);
return;
}
}
}
@@ -0,0 +1,69 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Inbox.ForeignMembers;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Inbox.Handlers
{
public class UndoHandler : IActivityHandler
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IDeliveryService _delivery;
public UndoHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery)
{
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_delivery = delivery;
}
public string Type => "Undo";
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
{
var undo = activity;
var inner = undo["object"];
var innerType = inner is JsonObject ? Value(inner, "type") : default;
var innerId = Id(inner);
if (inner is JsonObject && Id(inner["actor"]) != actor.ActorURI)
return;
if (innerType is not (null or "Follow"))
return;
var followers = await _dbEntities.Followers.Match(f => f.ActorURI == actor.ActorURI).ExecuteAsync(token);
var targetUri = inner is JsonObject ? Id(inner["object"]) : default;
foreach (var follower in followers)
{
var matchesActivity = innerId != default && follower.FollowActivityURI == innerId;
var target = targetUri == default ? default : await _localActors.FindByUri(targetUri, token);
var matchesTarget = target != default && target.Id == follower.LocalActorId && target.Kind == follower.LocalActorKind;
if (!matchesActivity && !matchesTarget)
continue;
await DB.Default.DeleteAsync<Follower>(follower.ID);
if (follower.LocalActorKind == LocalActorKind.Group)
await RemoveForeignMember(follower.LocalActorId, actor.ActorURI, token);
}
return;
}
}
}
@@ -0,0 +1,84 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Inbox.Handlers
{
public class UpdateHandler : IActivityHandler
{
const int MaxRevisions = 20;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
public UpdateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors)
{
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
}
public string Type => "Update";
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
{
var inner = activity["object"];
if (Id(inner) == actor.ActorURI)
{
await _remoteActors.GetActor(actor.ActorURI, refresh: true, token);
return;
}
if (inner is not JsonObject || !Origin.Same(Id(inner), actor.ActorURI))
return;
var note = NoteParser.Parse(inner);
if (note == default || note.AttributedTo != actor.ActorURI)
return;
var post = await _dbEntities.Posts.Match(p => p.ObjectURI == note.Id && p.ActorURI == actor.ActorURI).ExecuteFirstAsync(token);
if (post == default || post.DeletedAt.HasValue)
return;
post.Revisions.Add(new PostRevision
{
Title = post.Title,
SpoilerText = post.SpoilerText,
ContentHtml = post.ContentHtml,
HasContentWarning = post.HasContentWarning,
EditedAt = post.EditedAt ?? post.CreationDate
});
if (post.Revisions.Count > MaxRevisions)
post.Revisions.RemoveRange(0, post.Revisions.Count - MaxRevisions);
var mentions = new List<PostMention>();
foreach (var mention in note.Mentions)
{
var local = await _localActors.FindByUri(mention.ActorURI, token);
mentions.Add(new PostMention { ActorURI = mention.ActorURI, Handle = mention.Handle, IsLocal = local != default, AccountId = local?.Id });
}
post.Title = note.Title;
post.SpoilerText = note.SpoilerText;
post.HasContentWarning = note.Sensitive;
post.Text = note.ContentHtml;
post.ContentHtml = note.ContentHtml;
post.Language = note.Language;
post.Tags = note.Tags.ToList();
post.Mentions = mentions;
post.Media = note.Attachments.ToList();
post.EditedAt = note.Updated ?? DateTime.UtcNow;
post.UpdateDate = DateTime.UtcNow;
await DB.Default.SaveAsync(post, token);
}
}
}
@@ -0,0 +1,54 @@
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using System.Text.Json;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
namespace PrivaPub.Federation.Inbox
{
public interface IActivityHandler
{
string Type { get; }
Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token);
}
public class InboxProcessor : IJobHandler
{
readonly IRemoteActorService _remoteActors;
readonly IReadOnlyDictionary<string, IActivityHandler> _handlers;
readonly ILogger<InboxProcessor> _logger;
public InboxProcessor(IRemoteActorService remoteActors, IEnumerable<IActivityHandler> handlers, ILogger<InboxProcessor> logger)
{
_remoteActors = remoteActors;
_handlers = handlers.ToDictionary(h => h.Type, StringComparer.Ordinal);
_logger = logger;
}
public JobKind Kind => JobKind.ProcessInbox;
public int Concurrency => 2;
public int MaxAttempts => 8;
public int PerHostLimit => 2;
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var payload = JsonSerializer.Deserialize<InboxPayload>(job.Payload);
var activity = payload == default ? default : JsonNode.Parse(payload.Activity);
var type = Value(activity, "type");
if (type == default || !_handlers.TryGetValue(type, out var handler))
return JobOutcome.Done;
var actor = await _remoteActors.GetActor(payload.ActorURI, refresh: false, token);
if (actor == default)
return JobOutcome.Retry("the actor could not be loaded");
await handler.Handle(activity, actor, token);
_logger.LogInformation("Processed {Type} {Id} from {Actor}", type, Id(activity), actor.ActorURI);
return JobOutcome.Done;
}
}
}
+134
View File
@@ -0,0 +1,134 @@
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using System.Text.Json;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
namespace PrivaPub.Federation.Inbox
{
public sealed record InboxResult(int StatusCode, string Error = default);
public sealed record InboxPayload(string ActorURI, string Activity);
public interface IInboxReceiver
{
Task<InboxResult> Receive(HttpRequest request, LocalActor recipient, CancellationToken token);
}
public class InboxReceiver : IInboxReceiver
{
const int MaxBodyBytes = 1024 * 1024;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IJobQueue _queue;
readonly IDomainBlocks _domainBlocks;
readonly ILogger<InboxReceiver> _logger;
public InboxReceiver(ILocalActorService localActors, IRemoteActorService remoteActors, IJobQueue queue, IDomainBlocks domainBlocks,
ILogger<InboxReceiver> logger)
{
_localActors = localActors;
_remoteActors = remoteActors;
_queue = queue;
_domainBlocks = domainBlocks;
_logger = logger;
}
public async Task<InboxResult> Receive(HttpRequest request, LocalActor recipient, CancellationToken token)
{
if (request.ContentLength > MaxBodyBytes)
return new(StatusCodes.Status413PayloadTooLarge);
using var buffer = new MemoryStream();
await request.Body.CopyToAsync(buffer, token);
if (buffer.Length > MaxBodyBytes)
return new(StatusCodes.Status413PayloadTooLarge);
var body = buffer.ToArray();
JsonNode activity;
try
{
activity = JsonNode.Parse(body);
}
catch (JsonException)
{
return new(StatusCodes.Status400BadRequest, "the body is not JSON");
}
if (activity is not JsonObject)
return new(StatusCodes.Status400BadRequest, "the body is not an activity");
var type = Value(activity, "type");
var actorUri = Id(activity["actor"]);
if (string.IsNullOrEmpty(type) || string.IsNullOrEmpty(actorUri))
return new(StatusCodes.Status400BadRequest, "type and actor are required");
var parameters = HttpSignatures.Parse(request.Headers["Signature"].ToString());
if (parameters == default)
return new(StatusCodes.Status401Unauthorized, "missing or unreadable Signature header");
if (_domainBlocks.IsSuspended(HostOf(parameters.KeyId)) || _domainBlocks.IsSuspended(HostOf(actorUri)))
return new(StatusCodes.Status202Accepted);
var requestProblem = HttpSignatures.CheckRequest(request, parameters, body);
if (requestProblem != default)
return new(StatusCodes.Status401Unauthorized, requestProblem);
var signingString = HttpSignatures.SigningString(request, parameters);
var keyOwner = await _remoteActors.GetActorByKeyId(parameters.KeyId, refresh: false, token);
if (keyOwner == default && type == "Delete" && Id(activity["object"]) == actorUri)
return new(StatusCodes.Status202Accepted);
if (keyOwner == default || !HttpSignatures.Verify(keyOwner.PublicKey, signingString, parameters.Signature))
{
keyOwner = await _remoteActors.GetActorByKeyId(parameters.KeyId, refresh: true, token);
if (keyOwner == default || !HttpSignatures.Verify(keyOwner.PublicKey, signingString, parameters.Signature))
return new(StatusCodes.Status401Unauthorized, "the signature does not verify");
}
if (!string.Equals(keyOwner.ActorURI, actorUri, StringComparison.Ordinal))
return new(StatusCodes.Status401Unauthorized, "the activity's actor is not the key's owner");
var shapeProblem = await ShapeProblem(type, activity, actorUri, token);
if (shapeProblem != default)
return shapeProblem;
var activityId = Id(activity);
await _queue.Enqueue(JobKind.ProcessInbox, JsonSerializer.Serialize(new InboxPayload(actorUri, activity.ToJsonString())),
new Uri(actorUri).Host.ToLowerInvariant(), activityId == default ? default : "inbox|" + activityId, token);
_logger.LogInformation("Inbox {Recipient}: {Type} from {Actor} queued", recipient?.Handle ?? "shared", type, actorUri);
return new(StatusCodes.Status202Accepted);
}
static string HostOf(string uri) => Uri.TryCreate(uri, UriKind.Absolute, out var parsed) ? parsed.Host : default;
async Task<InboxResult> ShapeProblem(string type, JsonNode activity, string actorUri, CancellationToken token)
{
var activityId = Id(activity);
if (activityId != default && !Origin.Same(activityId, actorUri))
return new(StatusCodes.Status400BadRequest, "the activity's id is not on its actor's origin");
var inner = activity["object"];
switch (type)
{
case "Follow":
var target = await _localActors.FindByUri(Id(inner), token);
if (target is not { IsFederated: true } || target.Kind == LocalActorKind.Application)
return new(StatusCodes.Status404NotFound, "no such local actor");
break;
case "Undo" when inner is JsonObject && Id(inner["actor"]) != actorUri:
return new(StatusCodes.Status400BadRequest, "an actor can only undo its own activities");
case "Create" or "Update" when inner is JsonObject && Origin.Same(Id(inner), actorUri)
&& inner["attributedTo"] != default && Id(inner["attributedTo"]) != actorUri
&& Value(inner, "type") is not ("Person" or "Service" or "Application" or "Group" or "Organization"):
return new(StatusCodes.Status400BadRequest, "the object is not attributed to the actor");
}
return default;
}
}
}
@@ -0,0 +1,77 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
namespace PrivaPub.Federation.Moderation
{
public interface IDomainBlocks
{
DomainBlock Find(string host);
bool IsSuspended(string host);
Task Reload(CancellationToken token);
}
public class DomainBlocks : IDomainBlocks
{
static readonly TimeSpan Staleness = TimeSpan.FromMinutes(5);
readonly ILogger<DomainBlocks> _logger;
IReadOnlyDictionary<string, DomainBlock> _blocks = new Dictionary<string, DomainBlock>();
DateTime _loadedAt = DateTime.MinValue;
int _reloading;
public DomainBlocks(ILogger<DomainBlocks> logger)
{
_logger = logger;
}
public static string Normalise(string domain) => domain?.Trim().Trim('.').ToLowerInvariant();
public DomainBlock Find(string host)
{
RefreshIfStale();
host = Normalise(host);
var blocks = _blocks;
while (!string.IsNullOrEmpty(host))
{
if (blocks.TryGetValue(host, out var block))
return block;
var dot = host.IndexOf('.');
host = dot < 0 ? default : host[(dot + 1)..];
}
return default;
}
public bool IsSuspended(string host) => Find(host)?.Severity == DomainBlockSeverity.Suspend;
public async Task Reload(CancellationToken token)
{
var blocks = await DB.Default.Find<DomainBlock>().ExecuteAsync(token);
_blocks = blocks.Where(b => !string.IsNullOrEmpty(b.Domain))
.GroupBy(b => Normalise(b.Domain))
.ToDictionary(g => g.Key, g => g.First());
_loadedAt = DateTime.UtcNow;
}
void RefreshIfStale()
{
if (DateTime.UtcNow - _loadedAt < Staleness || Interlocked.Exchange(ref _reloading, 1) == 1)
return;
_ = Task.Run(async () =>
{
try
{
await Reload(CancellationToken.None);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Domain blocks could not be reloaded");
}
finally
{
Interlocked.Exchange(ref _reloading, 0);
}
});
}
}
}
@@ -0,0 +1,39 @@
using System.Text.Json.Nodes;
namespace PrivaPub.Federation.Objects
{
public static class ActivityJson
{
public static string Id(JsonNode node) => node switch
{
JsonValue value when value.TryGetValue<string>(out var text) => text,
JsonObject obj => Value(obj, "id") ?? Value(obj, "href"),
JsonArray array => array.Select(Id).FirstOrDefault(i => i != default),
_ => default
};
public static string Value(JsonNode node, string property) =>
node is JsonObject obj && obj[property] is JsonValue value && value.TryGetValue<string>(out var text) ? text : default;
public static IEnumerable<string> Addresses(JsonNode node)
{
foreach (var field in new[] { "to", "cc", "bto", "bcc", "audience" })
{
switch (node[field])
{
case JsonArray array:
foreach (var item in array)
{
var id = Id(item);
if (id != default)
yield return id;
}
break;
case JsonNode single when Id(single) is { } id:
yield return id;
break;
}
}
}
}
}
+22
View File
@@ -0,0 +1,22 @@
using PrivaPub.Models.Post;
namespace PrivaPub.Federation.Objects
{
public static class Addressing
{
public const string Public = "https://www.w3.org/ns/activitystreams#Public";
public static bool IsPublic(string address) => address is Public or "as:Public" or "Public";
public static PostVisibility Classify(IReadOnlyCollection<string> to, IReadOnlyCollection<string> cc, string authorFollowers)
{
if (to.Any(IsPublic))
return PostVisibility.Public;
if (cc.Any(IsPublic))
return PostVisibility.Unlisted;
if (!string.IsNullOrEmpty(authorFollowers) && (to.Contains(authorFollowers) || cc.Contains(authorFollowers)))
return PostVisibility.FollowersOnly;
return PostVisibility.Direct;
}
}
}
@@ -0,0 +1,80 @@
using AngleSharp.Css.Dom;
using AngleSharp.Dom;
using Ganss.Xss;
using System.Text.RegularExpressions;
namespace PrivaPub.Federation.Objects
{
public static partial class ContentSanitizer
{
static readonly HtmlSanitizer Sanitizer = Build();
public static string Html(string html) =>
string.IsNullOrWhiteSpace(html) ? string.Empty : Sanitizer.Sanitize(html).Trim();
static HtmlSanitizer Build()
{
var sanitizer = new HtmlSanitizer(new HtmlSanitizerOptions
{
AllowedTags = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
"p", "br", "span", "a", "abbr", "del", "s", "pre", "blockquote", "code", "b", "strong", "u", "i", "em",
"sub", "sup", "ul", "ol", "li", "ruby", "rt", "rp", "h1", "h2", "h3", "h4", "h5", "h6"
},
AllowedAttributes = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
"href", "rel", "class", "translate", "start", "reversed", "value", "title"
},
AllowedCssProperties = new HashSet<string>(),
AllowedAtRules = new HashSet<CssRuleType>(),
AllowedSchemes = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
"http", "https", "dat", "dweb", "ipfs", "ipns", "ssb", "gopher", "xmpp", "magnet", "gemini"
},
UriAttributes = new HashSet<string>(StringComparer.OrdinalIgnoreCase) { "href" }
})
{
KeepChildNodes = true
};
foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible" })
sanitizer.AllowedClasses.Add(allowed);
sanitizer.RemovingCssClass += (_, e) => e.Cancel = MicroformatClass().IsMatch(e.CssClass);
sanitizer.RemovingTag += (_, e) =>
{
if (e.Tag.LocalName is "script" or "style" or "template" or "iframe" or "object" or "embed" or "noscript" or "svg" or "math")
e.Tag.InnerHtml = string.Empty;
};
sanitizer.PostProcessNode += (_, e) =>
{
if (e.Node is not IElement element)
return;
switch (element.LocalName)
{
case "a":
if (!SchemePrefix().IsMatch(element.GetAttribute("href") ?? string.Empty))
element.RemoveAttribute("href");
element.SetAttribute("rel", "nofollow noopener noreferrer");
element.SetAttribute("target", "_blank");
break;
case "h1" or "h2" or "h3" or "h4" or "h5" or "h6":
var paragraph = e.Document.CreateElement("p");
var strong = e.Document.CreateElement("strong");
while (element.FirstChild != default)
strong.AppendChild(element.FirstChild);
paragraph.AppendChild(strong);
e.ReplacementNodes.Add(paragraph);
break;
}
};
return sanitizer;
}
[GeneratedRegex("^[a-z][a-z0-9+.-]*:", RegexOptions.IgnoreCase)]
private static partial Regex SchemePrefix();
[GeneratedRegex("^(h|p|u|dt|e)-[a-z0-9-]+$")]
private static partial Regex MicroformatClass();
}
}
+179
View File
@@ -0,0 +1,179 @@
using PrivaPub.Models.Post;
using System.Globalization;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
namespace PrivaPub.Federation.Objects
{
public sealed class NoteDocument
{
public string Id { get; init; }
public string Type { get; init; }
public string AttributedTo { get; init; }
public string ContentHtml { get; init; }
public string Title { get; init; }
public string SpoilerText { get; init; }
public bool Sensitive { get; init; }
public string Language { get; init; }
public string Url { get; init; }
public string InReplyTo { get; init; }
public string Context { get; init; }
public string Audience { get; init; }
public string QuoteUri { get; init; }
public DateTime Published { get; init; }
public DateTime? Updated { get; init; }
public IReadOnlyList<string> To { get; init; } = Array.Empty<string>();
public IReadOnlyList<string> Cc { get; init; } = Array.Empty<string>();
public IReadOnlyList<PostMention> Mentions { get; init; } = Array.Empty<PostMention>();
public IReadOnlyList<string> Tags { get; init; } = Array.Empty<string>();
public IReadOnlyList<PostMedia> Attachments { get; init; } = Array.Empty<PostMedia>();
}
public static class NoteParser
{
static readonly string[] NoteTypes = { "Note", "Article", "Page", "Question", "Event", "Video", "Audio", "Image" };
const int MaxAttachments = 16;
const int MaxTags = 64;
public static NoteDocument Parse(JsonNode node)
{
if (node is not JsonObject note)
return default;
var type = Value(note, "type");
var id = Id(note);
if (!NoteTypes.Contains(type) || Origin.Of(id) == default)
return default;
var (contentHtml, language) = Content(note);
var summary = Value(note, "summary");
return new NoteDocument
{
Id = id,
Type = type,
AttributedTo = Attribution(note["attributedTo"]),
ContentHtml = ContentSanitizer.Html(contentHtml),
Title = Plain(Value(note, "name"), 500),
SpoilerText = Plain(summary, 500),
Sensitive = note["sensitive"] is JsonValue sensitive && sensitive.TryGetValue<bool>(out var s) && s
|| !string.IsNullOrWhiteSpace(summary),
Language = language,
Url = UrlOf(note["url"]) ?? id,
InReplyTo = Id(note["inReplyTo"]),
Context = Id(note["context"]) ?? Value(note, "conversation"),
Audience = Id(note["audience"]),
QuoteUri = Value(note, "quote") ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote"),
Published = Time(Value(note, "published")) ?? DateTime.UtcNow,
Updated = Time(Value(note, "updated")),
To = List(note["to"]),
Cc = List(note["cc"]),
Mentions = Tags(note, "Mention").Select(t => new PostMention { ActorURI = Value(t, "href"), Handle = Value(t, "name") })
.Where(m => Origin.Of(m.ActorURI) != default)
.DistinctBy(m => m.ActorURI)
.Take(MaxTags)
.ToList(),
Tags = Tags(note, "Hashtag").Select(t => Value(t, "name")?.TrimStart('#').ToLowerInvariant())
.Where(t => !string.IsNullOrEmpty(t))
.Distinct()
.Take(MaxTags)
.ToList(),
Attachments = Attachments(note)
};
}
static (string Html, string Language) Content(JsonObject note)
{
var content = Value(note, "content");
string language = default;
if (note["contentMap"] is JsonObject map && map.Count > 0)
{
var first = map.First();
language = first.Key;
if (string.IsNullOrEmpty(content) && first.Value is JsonValue value && value.TryGetValue<string>(out var mapped))
content = mapped;
}
if (string.IsNullOrEmpty(content) && Value(note, "_misskey_content") is { } misskey)
content = System.Net.WebUtility.HtmlEncode(misskey).Replace("\n", "<br>");
return (content, language);
}
static string Attribution(JsonNode node) => node switch
{
JsonArray array => array.OfType<JsonObject>().Where(o => Value(o, "type") is "Person" or "Service" or "Application" or "Organization").Select(Id)
.Concat(array.OfType<JsonValue>().Select(Id))
.Concat(array.OfType<JsonObject>().Where(o => Value(o, "type") == "Group").Select(Id))
.FirstOrDefault(id => id != default),
_ => Id(node)
};
static IEnumerable<JsonObject> Tags(JsonObject note, string type) =>
(note["tag"] switch
{
JsonArray array => array.OfType<JsonObject>(),
JsonObject single => new[] { single },
_ => Enumerable.Empty<JsonObject>()
}).Where(t => Value(t, "type") == type);
static List<PostMedia> Attachments(JsonObject note)
{
var items = note["attachment"] switch
{
JsonArray array => array.OfType<JsonObject>(),
JsonObject single => new[] { single },
_ => Enumerable.Empty<JsonObject>()
};
return items
.Where(a => Value(a, "type") is "Document" or "Image" or "Video" or "Audio")
.Select(a => new PostMedia
{
RemoteURL = UrlOf(a["url"]),
ContentType = Value(a, "mediaType"),
Description = Plain(Value(a, "name") ?? Value(a, "summary"), 1500),
Blurhash = Value(a, "blurhash"),
Width = Int(a["width"]),
Height = Int(a["height"]),
Focus = a["focalPoint"] is JsonArray focus && focus.Count == 2
? focus.Select(f => f is JsonValue v && v.TryGetValue<double>(out var d) ? (float)Math.Clamp(d, -1, 1) : 0f).ToArray()
: default
})
.Where(m => Origin.Of(m.RemoteURL) != default)
.Take(MaxAttachments)
.ToList();
}
static string UrlOf(JsonNode node) => node switch
{
JsonValue value when value.TryGetValue<string>(out var text) => text,
JsonObject obj => Value(obj, "href") ?? Value(obj, "id"),
JsonArray array => array.Select(item => item is JsonObject link && Value(link, "mediaType") is { } mediaType && !mediaType.StartsWith("text/html")
? default
: UrlOf(item))
.FirstOrDefault(url => url != default) ?? array.Select(UrlOf).FirstOrDefault(url => url != default),
_ => default
};
static IReadOnlyList<string> List(JsonNode node) => node switch
{
JsonArray array => array.Select(Id).Where(id => id != default).Distinct(StringComparer.Ordinal).ToList(),
JsonNode single when Id(single) is { } id => new[] { id },
_ => Array.Empty<string>()
};
static DateTime? Time(string value) =>
DateTimeOffset.TryParse(value, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out var parsed)
? parsed.UtcDateTime
: default;
static int? Int(JsonNode node) =>
node is JsonValue value && value.TryGetValue<int>(out var number) && number is > 0 and < 100_000 ? number : default;
static string Plain(string value, int max)
{
if (string.IsNullOrWhiteSpace(value))
return default;
value = value.Trim();
return value.Length <= max ? value : value[..max];
}
}
}
+24
View File
@@ -0,0 +1,24 @@
namespace PrivaPub.Federation.Objects
{
public static class Origin
{
public static string Of(string uri) =>
Uri.TryCreate(uri, UriKind.Absolute, out var parsed) ? Of(parsed) : default;
public static string Of(Uri uri) =>
uri is { IsAbsoluteUri: true } && (uri.Scheme == Uri.UriSchemeHttps || uri.Scheme == Uri.UriSchemeHttp)
? $"{uri.Scheme}://{uri.IdnHost.ToLowerInvariant()}:{uri.Port}"
: default;
public static bool Same(string first, string second)
{
var origin = Of(first);
return origin != default && origin == Of(second);
}
public static bool IsDocumentAt(string id, Uri location) =>
Uri.TryCreate(id, UriKind.Absolute, out var parsed)
&& location is { IsAbsoluteUri: true }
&& Uri.Compare(parsed, location, UriComponents.HttpRequestUrl, UriFormat.UriEscaped, StringComparison.Ordinal) == 0;
}
}
@@ -0,0 +1,155 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.StaticServices;
using System.Net;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json.Nodes;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using System.Text.Json;
namespace PrivaPub.Federation.Outbox
{
public interface IDeliveryService
{
Task Enqueue(LocalActor signer, IEnumerable<string> inboxes, JsonObject activity, CancellationToken token);
Task EnqueueToFollowers(LocalActor signer, JsonObject activity, CancellationToken token, IEnumerable<string> extraInboxes = default);
Task<IReadOnlyList<string>> FollowerInboxes(LocalActor actor, CancellationToken token);
}
public sealed record DeliveryPayload(string SignerId, LocalActorKind SignerKind, string Inbox, string Body);
public class DeliveryService : IDeliveryService
{
readonly DbEntities _dbEntities;
readonly IJobQueue _queue;
public DeliveryService(DbEntities dbEntities, IJobQueue queue)
{
_dbEntities = dbEntities;
_queue = queue;
}
public async Task Enqueue(LocalActor signer, IEnumerable<string> inboxes, JsonObject activity, CancellationToken token)
{
var body = activity.ToJsonString();
var activityId = activity["id"] is JsonValue id && id.TryGetValue<string>(out var text) ? text : default;
var jobs = inboxes
.Where(i => !string.IsNullOrEmpty(i) && !i.StartsWith(signer.BaseAddress + "/", StringComparison.OrdinalIgnoreCase))
.Distinct(StringComparer.Ordinal)
.Select(inbox => Uri.TryCreate(inbox, UriKind.Absolute, out var uri) ? (inbox, uri) : default)
.Where(target => target.uri != default)
.Select(target => new Job
{
Kind = JobKind.Deliver,
Host = target.uri.Host.ToLowerInvariant(),
DedupeKey = activityId == default ? default : $"{activityId}|{target.inbox}",
Payload = JsonSerializer.Serialize(new DeliveryPayload(signer.Id, signer.Kind, target.inbox, body))
})
.ToList();
if (jobs.Count > 0)
await _queue.EnqueueMany(jobs, token);
}
public async Task EnqueueToFollowers(LocalActor signer, JsonObject activity, CancellationToken token, IEnumerable<string> extraInboxes = default)
{
var inboxes = (await FollowerInboxes(signer, token)).Concat(extraInboxes ?? Enumerable.Empty<string>());
await Enqueue(signer, inboxes, activity, token);
}
public async Task<IReadOnlyList<string>> FollowerInboxes(LocalActor actor, CancellationToken token)
{
var followers = await _dbEntities.Followers
.Match(f => f.LocalActorId == actor.Id && f.LocalActorKind == actor.Kind && f.IsAccepted)
.ExecuteAsync(token);
return followers.Select(f => string.IsNullOrEmpty(f.SharedInboxURL) ? f.InboxURL : f.SharedInboxURL)
.Distinct(StringComparer.Ordinal)
.ToList();
}
}
public class DeliveryJobHandler : IJobHandler
{
readonly ILocalActorService _actors;
readonly IFederationHttp _http;
readonly IHostCircuitBreaker _breaker;
readonly ILogger<DeliveryJobHandler> _logger;
public DeliveryJobHandler(ILocalActorService actors, IFederationHttp http, IHostCircuitBreaker breaker, ILogger<DeliveryJobHandler> logger)
{
_actors = actors;
_http = http;
_breaker = breaker;
_logger = logger;
}
public JobKind Kind => JobKind.Deliver;
public int Concurrency => 8;
public int MaxAttempts => 16;
public int PerHostLimit => 2;
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var payload = JsonSerializer.Deserialize<DeliveryPayload>(job.Payload);
if (payload == default || !Uri.TryCreate(payload.Inbox, UriKind.Absolute, out var inbox) || !_http.IsAllowed(inbox))
return JobOutcome.Dead("not a deliverable inbox");
var unavailableUntil = await _breaker.UnavailableUntil(job.Host, token);
if (unavailableUntil.HasValue)
return JobOutcome.Defer(unavailableUntil.Value, "the host is unavailable");
var signer = await _actors.FindById(payload.SignerKind, payload.SignerId, token);
if (signer == default)
return JobOutcome.Dead("the signing actor no longer exists");
var body = Encoding.UTF8.GetBytes(payload.Body);
using var request = new HttpRequestMessage(HttpMethod.Post, inbox) { Content = new ByteArrayContent(body) };
request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(RemoteActorService.ActivityJson);
HttpSignatures.Sign(request, signer, body);
try
{
using var response = await _http.Send(request, token);
var status = (int)response.StatusCode;
if (response.IsSuccessStatusCode)
{
await _breaker.Succeeded(job.Host, token);
return JobOutcome.Done;
}
if (response.StatusCode == HttpStatusCode.TooManyRequests)
{
var retryAfter = response.Headers.RetryAfter?.Delta ?? (response.Headers.RetryAfter?.Date - DateTimeOffset.UtcNow);
return retryAfter > TimeSpan.Zero
? JobOutcome.Defer(DateTime.UtcNow + Min(retryAfter.Value, TimeSpan.FromHours(6)), "429")
: JobOutcome.Retry("429");
}
if (status is >= 400 and < 500 && status != 408)
{
await _breaker.Succeeded(job.Host, token);
_logger.LogInformation("Delivery to {Inbox} refused with {Status}", payload.Inbox, status);
return JobOutcome.Dead($"{status} {response.ReasonPhrase}");
}
await _breaker.Failed(job.Host, $"{status}", token);
return JobOutcome.Retry($"{status} {response.ReasonPhrase}");
}
catch (BlockedDestinationException ex)
{
return JobOutcome.Dead(ex.Message);
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException && !token.IsCancellationRequested)
{
await _breaker.Failed(job.Host, ex.GetType().Name, token);
return JobOutcome.Retry(ex.Message);
}
}
static TimeSpan Min(TimeSpan a, TimeSpan b) => a < b ? a : b;
}
}
@@ -0,0 +1,278 @@
using Markdig;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using System.Globalization;
using System.Net;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Federation.Rendering
{
public static class ActivityPubRenderer
{
public const string ActivityStreams = "https://www.w3.org/ns/activitystreams";
public const string Public = Addressing.Public;
public const string ContentWarning = "Content warning";
static readonly MarkdownPipeline Pipeline = new MarkdownPipelineBuilder().DisableHtml().Build();
public static JsonArray Context() => new(
ActivityStreams,
"https://w3id.org/security/v1",
new JsonObject
{
["manuallyApprovesFollowers"] = "as:manuallyApprovesFollowers",
["sensitive"] = "as:sensitive",
["Hashtag"] = "as:Hashtag",
["alsoKnownAs"] = new JsonObject { ["@id"] = "as:alsoKnownAs", ["@type"] = "@id" },
["movedTo"] = new JsonObject { ["@id"] = "as:movedTo", ["@type"] = "@id" },
["toot"] = "http://joinmastodon.org/ns#",
["featured"] = new JsonObject { ["@id"] = "toot:featured", ["@type"] = "@id" },
["featuredTags"] = new JsonObject { ["@id"] = "toot:featuredTags", ["@type"] = "@id" },
["discoverable"] = "toot:discoverable",
["indexable"] = "toot:indexable",
["blurhash"] = "toot:blurhash",
["focalPoint"] = new JsonObject { ["@container"] = "@list", ["@id"] = "toot:focalPoint" },
["schema"] = "http://schema.org#",
["PropertyValue"] = "schema:PropertyValue",
["value"] = "schema:value",
["webfinger"] = "https://purl.archive.org/socialweb/webfinger#webfinger"
});
public static string Html(string markdown) =>
string.IsNullOrEmpty(markdown) ? string.Empty : Markdown.ToHtml(markdown, Pipeline).Trim();
public static string Timestamp(DateTime value) =>
DateTime.SpecifyKind(value, DateTimeKind.Utc).ToString("yyyy-MM-ddTHH:mm:ssZ", CultureInfo.InvariantCulture);
public static string Day(DateTime value) =>
DateTime.SpecifyKind(value, DateTimeKind.Utc).Date.ToString("yyyy-MM-ddT00:00:00Z", CultureInfo.InvariantCulture);
public static string TagUrl(string baseAddress, string tag) => $"{baseAddress}/tags/{Uri.EscapeDataString(tag)}";
public static JsonObject Actor(LocalActor actor)
{
var document = new JsonObject
{
["@context"] = Context(),
["id"] = actor.Uri,
["type"] = actor.Kind switch
{
LocalActorKind.Group => "Group",
LocalActorKind.Application => "Application",
_ => "Person"
},
["preferredUsername"] = actor.UserName,
["name"] = actor.Name,
["summary"] = Html(actor.Summary),
["url"] = actor.Kind == LocalActorKind.Application ? actor.Uri : actor.HtmlUrl,
["inbox"] = actor.Inbox,
["outbox"] = actor.Outbox,
["followers"] = actor.Followers,
["following"] = actor.Following,
["published"] = Day(actor.Published),
["manuallyApprovesFollowers"] = actor.ManuallyApprovesFollowers,
["discoverable"] = actor.Discoverable,
["indexable"] = false,
["webfinger"] = actor.Handle,
["endpoints"] = new JsonObject { ["sharedInbox"] = actor.SharedInbox },
["publicKey"] = new JsonObject
{
["id"] = actor.KeyId,
["owner"] = actor.Uri,
["publicKeyPem"] = Keys.ToSubjectPublicKeyInfoPem(actor.PublicKeyPem)
},
["attachment"] = new JsonArray(actor.Fields.Select(field => (JsonNode)new JsonObject
{
["type"] = "PropertyValue",
["name"] = field.Key,
["value"] = FieldValue(field.Value)
}).ToArray())
};
if (!string.IsNullOrEmpty(actor.PictureURL))
document["icon"] = new JsonObject { ["type"] = "Image", ["url"] = actor.PictureURL };
if (!string.IsNullOrEmpty(actor.ThumbnailURL))
document["image"] = new JsonObject { ["type"] = "Image", ["url"] = actor.ThumbnailURL };
return document;
}
public static JsonObject Note(PostEntity post, LocalActor author, LocalActor group, string inReplyTo)
{
var mentions = post.Mentions.Select(m => (JsonNode)m.ActorURI).ToArray();
var (to, cc) = post.Visibility switch
{
PostVisibility.Unlisted => (new JsonArray(author.Followers), new JsonArray(mentions.Prepend(Public).ToArray())),
PostVisibility.FollowersOnly => (new JsonArray(author.Followers), new JsonArray(mentions)),
PostVisibility.Direct => (new JsonArray(mentions), new JsonArray()),
_ => (new JsonArray(Public), new JsonArray(mentions.Prepend(author.Followers).ToArray()))
};
if (group != default)
cc.Add(group.Uri);
var note = NoteBody(post, author, to, cc, inReplyTo);
if (group != default)
note["audience"] = group.Uri;
return note;
}
public static JsonObject DirectNote(PostEntity post, LocalActor author, IReadOnlyList<(string Uri, string Handle)> recipients,
string context)
{
var note = NoteBody(post, author, new JsonArray(recipients.Select(r => (JsonNode)r.Uri).ToArray()), new JsonArray(), post.InReplyToURI);
var named = post.Mentions.Select(m => m.ActorURI).ToHashSet();
var tags = note["tag"]!.AsArray();
foreach (var recipient in recipients.Where(r => !named.Contains(r.Uri)))
tags.Add(new JsonObject { ["type"] = "Mention", ["href"] = recipient.Uri, ["name"] = "@" + recipient.Handle });
var unmentioned = recipients.Where(r => !named.Contains(r.Uri)).ToList();
if (unmentioned.Count > 0)
note["content"] = "<p>" + string.Join(" ", unmentioned.Select(r =>
$"<span class=\"h-card\" translate=\"no\"><a href=\"{WebUtility.HtmlEncode(r.Uri)}\" class=\"u-url mention\">@<span>{WebUtility.HtmlEncode(r.Handle.Split('@')[0])}</span></a></span>"))
+ "</p>" + note["content"]!.GetValue<string>();
if (!string.IsNullOrEmpty(context))
note["context"] = context;
return note;
}
static JsonObject NoteBody(PostEntity post, LocalActor author, JsonArray to, JsonArray cc, string inReplyTo)
{
var content = post.ContentHtml ?? Html(post.Text);
if (!string.IsNullOrEmpty(post.Title))
content = $"<p><strong>{WebUtility.HtmlEncode(post.Title)}</strong></p>{content}";
var note = new JsonObject
{
["id"] = author.PostUri(post.ID),
["type"] = "Note",
["attributedTo"] = author.Uri,
["content"] = content,
["published"] = Timestamp(post.CreationDate),
["url"] = author.PostHtmlUrl(post.ID),
["to"] = to,
["cc"] = cc,
["sensitive"] = post.HasContentWarning,
["tag"] = new JsonArray(post.Mentions
.Select(m => (JsonNode)new JsonObject { ["type"] = "Mention", ["href"] = m.ActorURI, ["name"] = MentionName(m) })
.Concat(post.Tags.Select(t => (JsonNode)new JsonObject
{
["type"] = "Hashtag",
["href"] = TagUrl(author.BaseAddress, t),
["name"] = "#" + t
}))
.ToArray())
};
if (!string.IsNullOrEmpty(post.Language))
note["contentMap"] = new JsonObject { [post.Language] = content };
if (!string.IsNullOrEmpty(post.Title))
note["name"] = post.Title;
var summary = post.SpoilerText ?? (post.HasContentWarning ? post.Title ?? ContentWarning : default);
if (!string.IsNullOrEmpty(summary))
{
note["summary"] = summary;
note["sensitive"] = true;
}
if (!string.IsNullOrEmpty(inReplyTo))
note["inReplyTo"] = inReplyTo;
if (post.EditedAt.HasValue)
note["updated"] = Timestamp(post.EditedAt.Value);
return note;
}
static string MentionName(PostMention mention)
{
var handle = mention.Handle?.TrimStart('@');
if (string.IsNullOrEmpty(handle))
return "@" + mention.ActorURI;
return "@" + (handle.Contains('@') ? handle : $"{handle}@{new Uri(mention.ActorURI).Authority}");
}
static string FieldValue(string value)
{
var encoded = WebUtility.HtmlEncode(value ?? string.Empty);
return Uri.TryCreate(value, UriKind.Absolute, out var link) && link.Scheme is "https" or "http"
? $"<a href=\"{encoded}\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\" translate=\"no\">{encoded}</a>"
: encoded;
}
public static JsonObject Create(LocalActor actor, JsonObject note, string activityId) => new()
{
["@context"] = Context(),
["id"] = actor.ActivityUri(activityId),
["type"] = "Create",
["actor"] = actor.Uri,
["published"] = note["published"]?.DeepClone(),
["to"] = note["to"]?.DeepClone(),
["cc"] = note["cc"]?.DeepClone(),
["object"] = note
};
public static JsonObject Announce(LocalActor group, string objectUri, string activityId) => new()
{
["@context"] = ActivityStreams,
["id"] = group.ActivityUri(activityId),
["type"] = "Announce",
["actor"] = group.Uri,
["published"] = Timestamp(DateTime.UtcNow),
["to"] = new JsonArray(Public),
["cc"] = new JsonArray(group.Followers),
["object"] = objectUri
};
public static JsonObject Delete(LocalActor actor, string objectUri, string activityId, JsonArray to, JsonArray cc) => new()
{
["@context"] = ActivityStreams,
["id"] = actor.ActivityUri(activityId),
["type"] = "Delete",
["actor"] = actor.Uri,
["to"] = to,
["cc"] = cc,
["object"] = new JsonObject { ["id"] = objectUri, ["type"] = "Tombstone" }
};
public static JsonObject Accept(LocalActor actor, JsonNode follow, string activityId) => new()
{
["@context"] = ActivityStreams,
["id"] = actor.ActivityUri(activityId),
["type"] = "Accept",
["actor"] = actor.Uri,
["object"] = follow.DeepClone()
};
public static JsonObject OrderedCollection(string id, int totalItems, IEnumerable<JsonNode> items, string first = default)
{
var collection = new JsonObject
{
["@context"] = ActivityStreams,
["id"] = id,
["type"] = "OrderedCollection",
["totalItems"] = totalItems
};
if (first != default)
collection["first"] = first;
if (items != default)
collection["orderedItems"] = new JsonArray(items.ToArray());
return collection;
}
public static JsonObject OrderedCollectionPage(string id, string partOf, IEnumerable<JsonNode> items, string next, string prev)
{
var page = new JsonObject
{
["@context"] = Context(),
["id"] = id,
["type"] = "OrderedCollectionPage",
["partOf"] = partOf,
["orderedItems"] = new JsonArray(items.ToArray())
};
if (next != default)
page["next"] = next;
if (prev != default)
page["prev"] = prev;
return page;
}
}
}
@@ -1,15 +1,19 @@
using Microsoft.AspNetCore.Http.Features;
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using PrivaPub.Federation.Actors;
namespace PrivaPub.Services.Federation
namespace PrivaPub.Federation.Signing
{
public sealed record SignatureParameters(string KeyId, string Algorithm, string[] Headers, byte[] Signature,
string Created, string Expires);
public static class HttpSignatures
{
static readonly TimeSpan AllowedClockSkew = TimeSpan.FromHours(12);
public static readonly TimeSpan MaxAge = TimeSpan.FromHours(1);
public static readonly TimeSpan MaxClockSkew = TimeSpan.FromMinutes(15);
public static string Digest(byte[] body) => "SHA-256=" + Convert.ToBase64String(SHA256.HashData(body));
@@ -104,7 +108,7 @@ namespace PrivaPub.Services.Federation
switch (header)
{
case "(request-target)":
lines.Add($"(request-target): {request.Method.ToLowerInvariant()} {request.PathBase}{request.Path}{request.QueryString}");
lines.Add($"(request-target): {request.Method.ToLowerInvariant()} {RequestTarget(request)}");
break;
case "(created)":
lines.Add($"(created): {parameters.Created}");
@@ -125,10 +129,17 @@ namespace PrivaPub.Services.Federation
return string.Join("\n", lines);
}
public static string CheckRequest(HttpRequest request, SignatureParameters parameters, byte[] body)
public static string CheckRequest(HttpRequest request, SignatureParameters parameters, byte[] body) =>
CheckRequest(request, parameters, body, DateTimeOffset.UtcNow);
public static string CheckRequest(HttpRequest request, SignatureParameters parameters, byte[] body, DateTimeOffset now)
{
if (parameters.Algorithm is not ("rsa-sha256" or "hs2019"))
return $"unsupported signature algorithm '{parameters.Algorithm}'";
if (!parameters.Headers.Contains("(request-target)"))
return "(request-target) is not signed";
if (!parameters.Headers.Contains("host"))
return "host is not signed";
if (body is { Length: > 0 })
{
@@ -141,24 +152,48 @@ namespace PrivaPub.Services.Federation
return "the digest does not match the body";
}
if (parameters.Headers.Contains("date"))
var signsDate = parameters.Headers.Contains("date");
var signsCreated = parameters.Headers.Contains("(created)");
if (!signsDate && !signsCreated)
return "neither date nor (created) is signed";
if (signsDate)
{
if (!DateTimeOffset.TryParse(request.Headers["Date"].ToString(), CultureInfo.InvariantCulture,
DateTimeStyles.AssumeUniversal, out var date))
return "unreadable Date header";
if ((DateTimeOffset.UtcNow - date).Duration() > AllowedClockSkew)
if (!IsFresh(date, now))
return "the Date header is outside the allowed window";
}
else if (!parameters.Headers.Contains("(created)"))
return "neither date nor (created) is signed";
if (!string.IsNullOrEmpty(parameters.Expires) && long.TryParse(parameters.Expires, out var expires)
&& DateTimeOffset.FromUnixTimeSeconds(expires) < DateTimeOffset.UtcNow - AllowedClockSkew)
return "the signature has expired";
if (signsCreated)
{
if (!long.TryParse(parameters.Created, NumberStyles.Integer, CultureInfo.InvariantCulture, out var created))
return "unreadable (created)";
if (!IsFresh(DateTimeOffset.FromUnixTimeSeconds(created), now))
return "(created) is outside the allowed window";
}
if (!string.IsNullOrEmpty(parameters.Expires) || parameters.Headers.Contains("(expires)"))
{
if (!long.TryParse(parameters.Expires, NumberStyles.Integer, CultureInfo.InvariantCulture, out var expires))
return "unreadable (expires)";
if (DateTimeOffset.FromUnixTimeSeconds(expires) < now - MaxClockSkew)
return "the signature has expired";
}
return default;
}
static bool IsFresh(DateTimeOffset signedAt, DateTimeOffset now) =>
signedAt >= now - MaxAge && signedAt <= now + MaxClockSkew;
static string RequestTarget(HttpRequest request)
{
var raw = request.HttpContext.Features.Get<IHttpRequestFeature>()?.RawTarget;
return string.IsNullOrEmpty(raw) || raw[0] != '/' ? $"{request.PathBase}{request.Path}{request.QueryString}" : raw;
}
public static bool Verify(string publicKeyPem, string signingString, byte[] signature)
{
if (string.IsNullOrEmpty(publicKeyPem) || signingString == null)
@@ -0,0 +1,40 @@
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.Models.User;
namespace PrivaPub.Infrastructure.Cli
{
public static class AdminCommands
{
const string Usage = "usage: PrivaPub admin promote|demote <root username>";
public static async Task<int> Run(string[] args)
{
if (args is not [("promote" or "demote") and var verb, var userName])
{
Console.Error.WriteLine(Usage);
return 2;
}
userName = userName.ToLowerInvariant();
var user = await DB.Default.Find<RootUser>().Match(u => u.UserName == userName).ExecuteFirstAsync();
if (user == default)
{
Console.Error.WriteLine($"no root user '{userName}'");
return 1;
}
user.Policies.RemoveAll(p => p is Policies.IsAdmin or Policies.IsModerator);
if (verb == "promote")
user.Policies.AddRange(new[] { Policies.IsAdmin, Policies.IsModerator });
if (!user.Policies.Contains(Policies.IsUser))
user.Policies.Add(Policies.IsUser);
user.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(user);
Console.WriteLine($"{userName}: {string.Join(", ", user.Policies)}");
return 0;
}
}
}
@@ -0,0 +1,18 @@
using MongoDB.Entities;
namespace PrivaPub.Infrastructure.Data
{
public static class EntityMaps
{
public static void Warm()
{
var collection = typeof(DB).GetMethods()
.Single(m => m.Name == nameof(DB.Collection) && m.IsGenericMethodDefinition && m.GetParameters().Length == 0);
var entities = typeof(EntityMaps).Assembly.GetTypes()
.Where(t => t is { IsClass: true, IsAbstract: false, IsGenericTypeDefinition: false } && typeof(IEntity).IsAssignableFrom(t))
.OrderBy(t => t.FullName, StringComparer.Ordinal);
foreach (var entity in entities)
collection.MakeGenericMethod(entity).Invoke(DB.Default, null);
}
}
}
+85
View File
@@ -0,0 +1,85 @@
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
namespace PrivaPub.Infrastructure.Data
{
public static class Indexes
{
public static async Task Create(CancellationToken token = default)
{
await Unique<Post>(p => p.ObjectURI, Builders<Post>.Filter.Type(p => p.ObjectURI, BsonType.String), token);
await Plain<Post>(token, p => p.GroupUserId, p => p.ID);
await Plain<Post>(token, p => p.GroupId, p => p.ID);
await Plain<Post>(token, p => p.ActorURI);
await Plain<Post>(token, p => p.ConversationId, p => p.ID);
await Plain<Post>(token, p => p.InReplyToURI);
await Unique<DmPost>(p => p.ObjectURI, Builders<DmPost>.Filter.Type(p => p.ObjectURI, BsonType.String), token);
await Plain<DmPost>(token, p => p.GroupId, p => p.ID);
await Unique<ForeignAvatar>(a => a.ActorURI, Builders<ForeignAvatar>.Filter.Type(a => a.ActorURI, BsonType.String), token);
await Plain<ForeignAvatar>(token, a => a.PublicKeyId);
await DB.Default.Index<Follower>()
.Key(f => f.LocalActorId, KeyType.Ascending)
.Key(f => f.LocalActorKind, KeyType.Ascending)
.Key(f => f.ActorURI, KeyType.Ascending)
.Option(o => o.Unique = true)
.CreateAsync(token);
await Plain<Follower>(token, f => f.ActorURI);
await DB.Default.Index<RootToAvatar>()
.Key(r => r.RootId, KeyType.Ascending)
.Key(r => r.AvatarId, KeyType.Ascending)
.Option(o => o.Unique = true)
.CreateAsync(token);
await Plain<RootToAvatar>(token, r => r.AvatarId);
await Unique<ReservedName>(r => r.Name, Builders<ReservedName>.Filter.Type(r => r.Name, BsonType.String), token);
await Unique<RootUser>(u => u.UserName, Builders<RootUser>.Filter.Type(u => u.UserName, BsonType.String), token);
await Plain<Avatar>(token, a => a.UserName);
await Plain<Group>(token, g => g.UserName);
await Plain<Group>(token, g => g.InvitationCode);
await Plain<DmGroup>(token, g => g.ParticipantsKey);
await Plain<Delivery>(token, d => d.DeliveredAt, d => d.AbandonedAt, d => d.NextAttemptAt);
await Plain<Job>(token, j => j.Kind, j => j.State, j => j.RunAt);
await Plain<Job>(token, j => j.State, j => j.LeasedUntil);
await Unique<Job>(j => j.DedupeKey, Builders<Job>.Filter.Type(j => j.DedupeKey, BsonType.String), token);
await DB.Default.Index<Job>()
.Key(j => j.FinishedAt, KeyType.Ascending)
.Option(o => o.ExpireAfter = TimeSpan.FromDays(7))
.CreateAsync(token);
await Unique<DomainBlock>(b => b.Domain, Builders<DomainBlock>.Filter.Type(b => b.Domain, BsonType.String), token);
await Unique<RemoteInstance>(i => i.Host, Builders<RemoteInstance>.Filter.Type(i => i.Host, BsonType.String), token);
}
static async Task Unique<T>(System.Linq.Expressions.Expression<Func<T, object>> key, FilterDefinition<T> partial,
CancellationToken token) where T : IEntity =>
await DB.Default.Index<T>()
.Key(key, KeyType.Ascending)
.Option(o =>
{
o.Unique = true;
o.PartialFilterExpression = partial;
})
.CreateAsync(token);
static async Task Plain<T>(CancellationToken token, params System.Linq.Expressions.Expression<Func<T, object>>[] keys) where T : IEntity
{
var index = DB.Default.Index<T>();
foreach (var key in keys)
index.Key(key, KeyType.Ascending);
await index.CreateAsync(token);
}
}
}
@@ -0,0 +1,59 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
namespace PrivaPub.Infrastructure.Data.Migrations
{
public class _001_deduplicate_before_unique_indexes : IMigration
{
public async Task UpgradeAsync()
{
var foreignAvatars = await DB.Default.Find<ForeignAvatar>().Match(a => a.ActorURI != null).ExecuteAsync();
await DeleteAllBut(foreignAvatars.GroupBy(a => a.ActorURI).Select(g => g.OrderByDescending(a => a.UpdatedAt)), DeleteForeignAvatar);
var posts = await DB.Default.Find<Post>().Match(p => p.ObjectURI != null).ExecuteAsync();
await DeleteAllBut(posts.GroupBy(p => p.ObjectURI).Select(g => g.OrderBy(p => p.ID, StringComparer.Ordinal)), DeletePost);
var dmPosts = await DB.Default.Find<DmPost>().Match(p => p.ObjectURI != null).ExecuteAsync();
await DeleteAllBut(dmPosts.GroupBy(p => p.ObjectURI).Select(g => g.OrderBy(p => p.ID, StringComparer.Ordinal)), DeleteDmPost);
var followers = await DB.Default.Find<Follower>().ExecuteAsync();
await DeleteAllBut(followers.GroupBy(f => (f.LocalActorId, f.LocalActorKind, f.ActorURI))
.Select(g => g.OrderByDescending(f => f.IsAccepted).ThenBy(f => f.ID, StringComparer.Ordinal)), DeleteFollower);
var links = await DB.Default.Find<RootToAvatar>().ExecuteAsync();
await DeleteAllBut(links.GroupBy(r => (r.RootId, r.AvatarId)).Select(g => g.OrderBy(r => r.ID, StringComparer.Ordinal)), DeleteLink);
var taken = new HashSet<string>(StringComparer.Ordinal) { LocalActorService.InstanceUserName };
var reservations = new List<ReservedName>
{
new() { Name = LocalActorService.InstanceUserName, OwnerKind = LocalActorKind.Application }
};
foreach (var avatar in (await DB.Default.Find<Avatar>().ExecuteAsync()).OrderBy(a => a.ID, StringComparer.Ordinal))
if (!string.IsNullOrEmpty(avatar.UserName) && taken.Add(avatar.UserName.ToLowerInvariant()))
reservations.Add(new() { Name = avatar.UserName.ToLowerInvariant(), OwnerKind = LocalActorKind.Person, OwnerId = avatar.ID });
foreach (var group in (await DB.Default.Find<Group>().ExecuteAsync()).OrderBy(g => g.ID, StringComparer.Ordinal))
if (!string.IsNullOrEmpty(group.UserName) && taken.Add(group.UserName.ToLowerInvariant()))
reservations.Add(new() { Name = group.UserName.ToLowerInvariant(), OwnerKind = LocalActorKind.Group, OwnerId = group.ID });
await DB.Default.DeleteAsync<ReservedName>(_ => true);
await DB.Default.SaveAsync(reservations);
}
static async Task DeleteAllBut<T>(IEnumerable<IOrderedEnumerable<T>> groups, Func<T, Task> delete)
{
foreach (var group in groups)
foreach (var duplicate in group.Skip(1))
await delete(duplicate);
}
static Task DeleteForeignAvatar(ForeignAvatar a) => DB.Default.DeleteAsync<ForeignAvatar>(a.ID);
static Task DeletePost(Post p) => DB.Default.DeleteAsync<Post>(p.ID);
static Task DeleteDmPost(DmPost p) => DB.Default.DeleteAsync<DmPost>(p.ID);
static Task DeleteFollower(Follower f) => DB.Default.DeleteAsync<Follower>(f.ID);
static Task DeleteLink(RootToAvatar r) => DB.Default.DeleteAsync<RootToAvatar>(r.ID);
}
}
@@ -0,0 +1,40 @@
using MongoDB.Entities;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
namespace PrivaPub.Infrastructure.Data.Migrations
{
public class _002_sanitize_stored_content : IMigration
{
public async Task UpgradeAsync()
{
foreach (var post in await DB.Default.Find<Post>().ExecuteAsync())
{
var html = post.IsFederatedCopy ? ContentSanitizer.Html(post.Text) : ActivityPubRenderer.Html(post.Text);
await DB.Default.Update<Post>().MatchID(post.ID)
.Modify(p => p.Text, post.IsFederatedCopy ? html : post.Text)
.Modify(p => p.ContentHtml, html)
.Modify(p => p.ContentFormat, post.IsFederatedCopy ? ContentFormat.Html : ContentFormat.Markdown)
.ExecuteAsync();
}
foreach (var post in await DB.Default.Find<DmPost>().ExecuteAsync())
{
var html = post.IsFederatedCopy ? ContentSanitizer.Html(post.Text) : ActivityPubRenderer.Html(post.Text);
await DB.Default.Update<DmPost>().MatchID(post.ID)
.Modify(p => p.Text, post.IsFederatedCopy ? html : post.Text)
.Modify(p => p.ContentHtml, html)
.Modify(p => p.ContentFormat, post.IsFederatedCopy ? ContentFormat.Html : ContentFormat.Markdown)
.ExecuteAsync();
}
foreach (var avatar in await DB.Default.Find<ForeignAvatar>().ExecuteAsync())
await DB.Default.Update<ForeignAvatar>().MatchID(avatar.ID)
.Modify(a => a.Biography, ContentSanitizer.Html(avatar.Biography))
.ExecuteAsync();
}
}
}
@@ -0,0 +1,28 @@
using MongoDB.Entities;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
namespace PrivaPub.Infrastructure.Data.Migrations
{
public class _003_conversation_keys_and_circles : IMigration
{
public async Task UpgradeAsync()
{
foreach (var dmGroup in await DB.Default.Find<DmGroup>().ExecuteAsync())
await DB.Default.Update<DmGroup>().MatchID(dmGroup.ID)
.Modify(g => g.ParticipantsKey, DmGroup.KeyOf(dmGroup.Members))
.ExecuteAsync();
await DB.Default.Update<Group>()
.Match(_ => true)
.Modify(g => g.Kind, GroupKind.Circle)
.ExecuteAsync();
await DB.Default.Update<Post>()
.Match(p => p.GroupId != null && !p.IsFederatedCopy)
.Modify(p => p.IsLocalOnly, true)
.ExecuteAsync();
}
}
}
@@ -0,0 +1,43 @@
using MongoDB.Entities;
using PrivaPub.Federation.Outbox;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using System.Text.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Infrastructure.Data.Migrations
{
public class _004_pending_deliveries_become_jobs : IMigration
{
public async Task UpgradeAsync()
{
var pending = await DB.Default.Find<Delivery>()
.Match(d => !d.DeliveredAt.HasValue && !d.AbandonedAt.HasValue)
.ExecuteAsync();
foreach (var delivery in pending)
{
if (Uri.TryCreate(delivery.InboxURL, UriKind.Absolute, out var inbox))
{
var activityId = JsonNode.Parse(delivery.Body)?["id"]?.GetValue<string>();
var job = new Job
{
Kind = JobKind.Deliver,
Host = inbox.Host.ToLowerInvariant(),
DedupeKey = activityId == default ? default : $"{activityId}|{delivery.InboxURL}",
Attempts = delivery.Attempts,
RunAt = delivery.NextAttemptAt,
Payload = JsonSerializer.Serialize(new DeliveryPayload(delivery.SignerId, delivery.SignerKind, delivery.InboxURL, delivery.Body))
};
if (!await DB.Default.Find<Job>().Match(j => j.DedupeKey == job.DedupeKey && job.DedupeKey != null).ExecuteAnyAsync())
await DB.Default.SaveAsync(job);
}
await DB.Default.Update<Delivery>().MatchID(delivery.ID)
.Modify(d => d.AbandonedAt, DateTime.UtcNow)
.Modify(d => d.LastError, "moved to the job queue")
.ExecuteAsync();
}
}
}
}
@@ -0,0 +1,67 @@
using MongoDB.Entities;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
namespace PrivaPub.Infrastructure.Data.Migrations
{
public class _005_direct_posts_join_posts : IMigration
{
public async Task UpgradeAsync()
{
foreach (var dm in await DB.Default.Find<DmPost>().ExecuteAsync())
{
if (await DB.Default.Find<Post>().Match(p => p.ID == dm.ID || p.ObjectURI == dm.ObjectURI).ExecuteAnyAsync())
continue;
await DB.Default.SaveAsync(new Post
{
ID = dm.ID,
GroupUserId = dm.GroupUserId,
AuthorAccountId = dm.IsFederatedCopy ? await ForeignAccountId(dm.ActorURI) : dm.GroupUserId,
ConversationId = dm.GroupId,
Visibility = PostVisibility.Direct,
AnsweringToPostId = Local(dm.AnsweringToPostId),
InReplyToURI = Remote(dm.AnsweringToPostId),
Title = dm.Title,
Text = dm.Text,
ContentHtml = dm.ContentHtml,
ContentFormat = dm.ContentFormat,
Media = dm.Media,
HasContentWarning = dm.HasContentWarning,
IsFederatedCopy = dm.IsFederatedCopy,
ObjectURI = dm.ObjectURI,
ActorURI = dm.ActorURI,
Url = dm.ObjectURI,
CreationDate = dm.CreationDate,
UpdateDate = dm.UpdateDate
});
}
foreach (var post in await DB.Default.Find<Post>().Match(p => p.Visibility != PostVisibility.Direct).ExecuteAsync())
{
var parent = Local(post.AnsweringToPostId) is { } parentId
? await DB.Default.Find<Post>().OneAsync(parentId)
: Remote(post.AnsweringToPostId) is { } parentUri
? await DB.Default.Find<Post>().Match(p => p.ObjectURI == parentUri).ExecuteFirstAsync()
: default;
await DB.Default.Update<Post>().MatchID(post.ID)
.Modify(p => p.Visibility, post.IsLocalOnly ? PostVisibility.Circle : PostVisibility.Public)
.Modify(p => p.AuthorAccountId, post.IsFederatedCopy ? await ForeignAccountId(post.ActorURI) : post.GroupUserId)
.Modify(p => p.AnsweringToPostId, parent?.ID)
.Modify(p => p.InReplyToURI, parent?.ObjectURI ?? Remote(post.AnsweringToPostId))
.Modify(p => p.InReplyToAccountId, parent?.AuthorAccountId ?? parent?.GroupUserId)
.Modify(p => p.Url, post.Url ?? post.ObjectURI)
.ExecuteAsync();
}
}
static string Local(string reference) =>
string.IsNullOrEmpty(reference) || reference.Contains("://") ? default : reference;
static string Remote(string reference) =>
reference != default && reference.Contains("://") ? reference : default;
static async Task<string> ForeignAccountId(string actorUri) =>
(await DB.Default.Find<ForeignAvatar>().Match(a => a.ActorURI == actorUri).ExecuteFirstAsync())?.ID;
}
}
@@ -0,0 +1,170 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Options;
using PrivaPub.Federation.Moderation;
using System.Net;
using System.Text.Json;
namespace PrivaPub.Infrastructure.Http
{
public sealed class FetchedJson : IDisposable
{
public Uri FinalUri { get; init; }
public JsonDocument Document { get; init; }
public JsonElement Root => Document.RootElement;
public void Dispose() => Document?.Dispose();
}
public interface IFederationHttp
{
bool IsAllowed(Uri target);
Task<FetchedJson> GetJson(string url, string accept, Action<HttpRequestMessage> sign, CancellationToken token);
Task<HttpResponseMessage> Send(HttpRequestMessage request, CancellationToken token);
}
public class FederationHttp : IFederationHttp
{
public const string ClientName = "Federation";
public const int MaxResponseBytes = 1024 * 1024;
public static readonly TimeSpan RequestTimeout = TimeSpan.FromSeconds(15);
const int MaxRedirects = 3;
static readonly TimeSpan NegativeCacheLifetime = TimeSpan.FromMinutes(5);
static readonly string[] JsonMediaTypes =
{
"application/activity+json",
"application/ld+json",
"application/jrd+json",
"application/json"
};
readonly IHttpClientFactory _httpClientFactory;
readonly IMemoryCache _cache;
readonly IOptionsMonitor<FederationOptions> _options;
readonly IDomainBlocks _domainBlocks;
readonly ILogger<FederationHttp> _logger;
public FederationHttp(IHttpClientFactory httpClientFactory, IMemoryCache cache, IOptionsMonitor<FederationOptions> options,
IDomainBlocks domainBlocks, ILogger<FederationHttp> logger)
{
_httpClientFactory = httpClientFactory;
_cache = cache;
_options = options;
_domainBlocks = domainBlocks;
_logger = logger;
}
public bool IsAllowed(Uri target)
{
if (target is not { IsAbsoluteUri: true } || !string.IsNullOrEmpty(target.UserInfo))
return false;
if (_domainBlocks?.IsSuspended(target.Host) == true)
return false;
var options = _options.CurrentValue;
if (target.Scheme != Uri.UriSchemeHttps && !(options.AllowPlainHttp && target.Scheme == Uri.UriSchemeHttp))
return false;
if (options.AllowPrivateNetworks)
return true;
return target.HostNameType == UriHostNameType.Dns
&& target.Host.Contains('.')
&& !target.Host.EndsWith(".localhost", StringComparison.OrdinalIgnoreCase)
&& !target.Host.EndsWith(".local", StringComparison.OrdinalIgnoreCase)
&& !target.Host.EndsWith(".internal", StringComparison.OrdinalIgnoreCase);
}
public async Task<FetchedJson> GetJson(string url, string accept, Action<HttpRequestMessage> sign, CancellationToken token)
{
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsAllowed(target))
return default;
var negativeKey = NegativeKey(target);
if (_cache.TryGetValue(negativeKey, out _))
return default;
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(token);
timeout.CancelAfter(RequestTimeout);
try
{
for (var hop = 0; hop <= MaxRedirects; hop++)
{
using var request = new HttpRequestMessage(HttpMethod.Get, target);
request.Headers.Accept.ParseAdd(accept);
sign?.Invoke(request);
using var response = await _httpClientFactory.CreateClient(ClientName)
.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, timeout.Token);
if (IsRedirect(response.StatusCode))
{
var location = response.Headers.Location;
var next = location == default ? default : location.IsAbsoluteUri ? location : new Uri(target, location);
if (!IsAllowed(next))
return Refuse(negativeKey, url, "a redirect to a disallowed location");
target = next;
continue;
}
if (!response.IsSuccessStatusCode)
return Refuse(negativeKey, url, $"status {(int)response.StatusCode}");
var mediaType = response.Content.Headers.ContentType?.MediaType;
if (mediaType == default || !JsonMediaTypes.Contains(mediaType, StringComparer.OrdinalIgnoreCase))
return Refuse(negativeKey, url, $"content type '{mediaType}'");
if (response.Content.Headers.ContentLength > MaxResponseBytes)
return Refuse(negativeKey, url, "a body over the size limit");
var body = await ReadBounded(response.Content, MaxResponseBytes, timeout.Token);
if (body == default)
return Refuse(negativeKey, url, "a body over the size limit");
return new FetchedJson { FinalUri = target, Document = JsonDocument.Parse(body) };
}
return Refuse(negativeKey, url, "too many redirects");
}
catch (OperationCanceledException) when (!token.IsCancellationRequested)
{
return Refuse(negativeKey, url, "a timeout");
}
catch (Exception ex) when (ex is HttpRequestException or JsonException or BlockedDestinationException)
{
return Refuse(negativeKey, url, ex.Message);
}
}
public async Task<HttpResponseMessage> Send(HttpRequestMessage request, CancellationToken token)
{
if (!IsAllowed(request.RequestUri))
throw new BlockedDestinationException(request.RequestUri?.Host);
return await _httpClientFactory.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
}
public static async Task<byte[]> ReadBounded(HttpContent content, int limit, CancellationToken token)
{
await using var stream = await content.ReadAsStreamAsync(token);
using var buffer = new MemoryStream();
var chunk = new byte[16 * 1024];
int read;
while ((read = await stream.ReadAsync(chunk, token)) > 0)
{
if (buffer.Length + read > limit)
return default;
buffer.Write(chunk, 0, read);
}
return buffer.ToArray();
}
static bool IsRedirect(HttpStatusCode status) =>
status is HttpStatusCode.MovedPermanently or HttpStatusCode.Found or HttpStatusCode.SeeOther
or HttpStatusCode.TemporaryRedirect or HttpStatusCode.PermanentRedirect;
static string NegativeKey(Uri target) => "federation-http:refused:" + target.AbsoluteUri;
FetchedJson Refuse(string negativeKey, string url, string reason)
{
_cache.Set(negativeKey, true, NegativeCacheLifetime);
_logger.LogInformation("GET {Url} refused: {Reason}", url, reason);
return default;
}
}
}
@@ -0,0 +1,8 @@
namespace PrivaPub.Infrastructure.Http
{
public class FederationOptions
{
public bool AllowPrivateNetworks { get; set; }
public bool AllowPlainHttp { get; set; }
}
}
@@ -0,0 +1,55 @@
using System.Net;
using System.Net.Sockets;
namespace PrivaPub.Infrastructure.Http
{
public static class IpRangeGuard
{
static readonly IPNetwork[] BlockedV4 =
{
IPNetwork.Parse("0.0.0.0/8"),
IPNetwork.Parse("10.0.0.0/8"),
IPNetwork.Parse("100.64.0.0/10"),
IPNetwork.Parse("127.0.0.0/8"),
IPNetwork.Parse("169.254.0.0/16"),
IPNetwork.Parse("172.16.0.0/12"),
IPNetwork.Parse("192.0.0.0/24"),
IPNetwork.Parse("192.0.2.0/24"),
IPNetwork.Parse("192.88.99.0/24"),
IPNetwork.Parse("192.168.0.0/16"),
IPNetwork.Parse("198.18.0.0/15"),
IPNetwork.Parse("198.51.100.0/24"),
IPNetwork.Parse("203.0.113.0/24"),
IPNetwork.Parse("224.0.0.0/4"),
IPNetwork.Parse("240.0.0.0/4")
};
static readonly IPNetwork GlobalUnicastV6 = IPNetwork.Parse("2000::/3");
static readonly IPNetwork[] BlockedV6 =
{
IPNetwork.Parse("2001::/32"),//Teredo
IPNetwork.Parse("2001:2::/48"),
IPNetwork.Parse("2001:10::/28"),
IPNetwork.Parse("2001:20::/28"),
IPNetwork.Parse("2001:db8::/32"),
IPNetwork.Parse("2002::/16"),//6to4
IPNetwork.Parse("3fff::/20")
};
public static bool IsPublic(IPAddress address)
{
if (address == default)
return false;
if (address.IsIPv4MappedToIPv6)
address = address.MapToIPv4();
return address.AddressFamily switch
{
AddressFamily.InterNetwork => !BlockedV4.Any(range => range.Contains(address)),
AddressFamily.InterNetworkV6 => GlobalUnicastV6.Contains(address) && !BlockedV6.Any(range => range.Contains(address)),
_ => false
};
}
}
}
@@ -0,0 +1,58 @@
using System.Net;
using System.Net.Sockets;
namespace PrivaPub.Infrastructure.Http
{
public sealed class BlockedDestinationException : Exception
{
public BlockedDestinationException(string host) : base($"'{host}' does not resolve to a public address") { }
}
public static class SafeHttpHandlerFactory
{
public static SocketsHttpHandler Create(FederationOptions options) => new()
{
AllowAutoRedirect = false,
UseProxy = false,
UseCookies = false,
AutomaticDecompression = DecompressionMethods.All,
ConnectTimeout = TimeSpan.FromSeconds(10),
PooledConnectionLifetime = TimeSpan.FromMinutes(2),
MaxResponseHeadersLength = 64,
ConnectCallback = (context, token) => Connect(context.DnsEndPoint, options.AllowPrivateNetworks, token)
};
public static async ValueTask<Stream> Connect(DnsEndPoint endPoint, bool allowPrivateNetworks, CancellationToken token)
{
var addresses = await Resolve(endPoint.Host, token);
if (addresses.Length == 0 || !allowPrivateNetworks && !addresses.All(IpRangeGuard.IsPublic))
throw new BlockedDestinationException(endPoint.Host);
var socket = new Socket(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
try
{
await socket.ConnectAsync(addresses, endPoint.Port, token);
return new NetworkStream(socket, ownsSocket: true);
}
catch
{
socket.Dispose();
throw;
}
}
static async Task<IPAddress[]> Resolve(string host, CancellationToken token)
{
if (IPAddress.TryParse(host.Trim('[', ']'), out var literal))
return new[] { literal };
try
{
return await Dns.GetHostAddressesAsync(host, token);
}
catch (SocketException)
{
return Array.Empty<IPAddress>();
}
}
}
}
+20
View File
@@ -0,0 +1,20 @@
using System.Buffers.Binary;
using System.Security.Cryptography;
namespace PrivaPub.Infrastructure.Ids
{
public static class PrivacyIds
{
public static string ForDay(DateTime when) => At(DateTime.SpecifyKind(when, DateTimeKind.Utc).Date);
public static string At(DateTime when)
{
var seconds = Math.Clamp(new DateTimeOffset(DateTime.SpecifyKind(when, DateTimeKind.Utc)).ToUnixTimeSeconds(), 0,
DateTimeOffset.UtcNow.AddDays(1).ToUnixTimeSeconds());
Span<byte> bytes = stackalloc byte[12];
BinaryPrimitives.WriteInt32BigEndian(bytes, (int)seconds);
RandomNumberGenerator.Fill(bytes[4..]);
return Convert.ToHexStringLower(bytes);
}
}
}
+13
View File
@@ -0,0 +1,13 @@
namespace PrivaPub.Infrastructure.Jobs
{
public static class Backoff
{
public static TimeSpan After(int attempt) =>
TimeSpan.FromSeconds(Math.Pow(attempt, 4) + 15 + Random.Shared.Next(10) * (attempt + 1));
public static TimeSpan HostQuarantine(int consecutiveFailures, int threshold) =>
consecutiveFailures < threshold
? TimeSpan.Zero
: TimeSpan.FromHours(Math.Min(Math.Pow(2, consecutiveFailures - threshold), 24 * 7));
}
}
@@ -0,0 +1,76 @@
using Microsoft.Extensions.Caching.Memory;
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
namespace PrivaPub.Infrastructure.Jobs
{
public interface IHostCircuitBreaker
{
Task<DateTime?> UnavailableUntil(string host, CancellationToken token);
Task Succeeded(string host, CancellationToken token);
Task Failed(string host, string error, CancellationToken token);
}
public class HostCircuitBreaker : IHostCircuitBreaker
{
public const int Threshold = 10;
static readonly TimeSpan CacheLifetime = TimeSpan.FromSeconds(30);
readonly IMemoryCache _cache;
public HostCircuitBreaker(IMemoryCache cache)
{
_cache = cache;
}
public async Task<DateTime?> UnavailableUntil(string host, CancellationToken token)
{
var instance = await Instance(host, token);
return instance?.UnavailableUntil > DateTime.UtcNow ? instance.UnavailableUntil : default;
}
public async Task Succeeded(string host, CancellationToken token)
{
var instance = await Instance(host, token);
if (instance is not { ConsecutiveFailures: > 0 } && instance?.LastSuccessAt > DateTime.UtcNow.AddHours(-1))
return;
await DB.Default.Update<RemoteInstance>()
.Match(i => i.Host == host)
.Modify(i => i.ConsecutiveFailures, 0)
.Modify(i => i.UnavailableUntil, null)
.Modify(i => i.LastSuccessAt, DateTime.UtcNow)
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
_cache.Remove(Key(host));
}
public async Task Failed(string host, string error, CancellationToken token)
{
var now = DateTime.UtcNow;
var instance = await DB.Default.UpdateAndGet<RemoteInstance>()
.Match(i => i.Host == host)
.Modify(b => b.Inc(i => i.ConsecutiveFailures, 1))
.Modify(i => i.LastFailureAt, now)
.Modify(i => i.LastError, error)
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
var quarantine = Backoff.HostQuarantine(instance.ConsecutiveFailures, Threshold);
if (quarantine > TimeSpan.Zero)
await DB.Default.Update<RemoteInstance>().MatchID(instance.ID)
.Modify(i => i.UnavailableUntil, now + quarantine)
.ExecuteAsync(token);
_cache.Remove(Key(host));
}
async Task<RemoteInstance> Instance(string host, CancellationToken token) =>
await _cache.GetOrCreateAsync(Key(host), async entry =>
{
entry.AbsoluteExpirationRelativeToNow = CacheLifetime;
return await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteFirstAsync(token);
});
static string Key(string host) => "remote-instance:" + host;
}
}
+142
View File
@@ -0,0 +1,142 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using System.Collections.Concurrent;
namespace PrivaPub.Infrastructure.Jobs
{
public sealed record JobOutcome(JobResult Result, string Error = default, DateTime? RetryAt = default)
{
public static readonly JobOutcome Done = new(JobResult.Done);
public static JobOutcome Retry(string error) => new(JobResult.Retry, error);
public static JobOutcome Dead(string error) => new(JobResult.Dead, error);
public static JobOutcome Defer(DateTime until, string error) => new(JobResult.Defer, error, until);
}
public enum JobResult
{
Done,
Retry,
Dead,
Defer
}
public interface IJobQueue
{
Task<bool> Enqueue(JobKind kind, string payload, string host, string dedupeKey, CancellationToken token);
Task<int> EnqueueMany(IEnumerable<Job> jobs, CancellationToken token);
Task<Job> Lease(JobKind kind, IReadOnlyCollection<string> busyHosts, CancellationToken token);
Task Finish(Job job, JobOutcome outcome, int maxAttempts, CancellationToken token);
Task<long> Reap(CancellationToken token);
Task WaitForWork(JobKind kind, TimeSpan poll, CancellationToken token);
}
public class JobQueue : IJobQueue
{
public static readonly TimeSpan LeaseTime = TimeSpan.FromMinutes(2);
readonly string _owner = $"{Environment.MachineName}:{Environment.ProcessId}";
readonly ConcurrentDictionary<JobKind, SemaphoreSlim> _signals = new();
public async Task<bool> Enqueue(JobKind kind, string payload, string host, string dedupeKey, CancellationToken token) =>
await EnqueueMany(new[] { new Job { Kind = kind, Payload = payload, Host = host, DedupeKey = dedupeKey } }, token) == 1;
public async Task<int> EnqueueMany(IEnumerable<Job> jobs, CancellationToken token)
{
var inserted = 0;
foreach (var job in jobs)
{
try
{
await DB.Default.SaveAsync(job, token);
inserted++;
Signal(job.Kind);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
}
}
return inserted;
}
public async Task<Job> Lease(JobKind kind, IReadOnlyCollection<string> busyHosts, CancellationToken token)
{
var now = DateTime.UtcNow;
var busy = busyHosts?.ToList() ?? new List<string>();
return await DB.Default.UpdateAndGet<Job>()
.Match(j => j.Kind == kind && j.State == JobState.Pending && j.RunAt <= now && !busy.Contains(j.Host))
.Modify(j => j.State, JobState.Running)
.Modify(j => j.LeasedUntil, now + LeaseTime)
.Modify(j => j.LeaseOwner, _owner)
.Modify(b => b.Inc(j => j.Attempts, 1))
.Option(o => o.Sort = Builders<Job>.Sort.Ascending(j => j.RunAt))
.ExecuteAsync(token);
}
public async Task Finish(Job job, JobOutcome outcome, int maxAttempts, CancellationToken token)
{
var now = DateTime.UtcNow;
var update = DB.Default.Update<Job>().MatchID(job.ID)
.Modify(j => j.LeasedUntil, null)
.Modify(j => j.LeaseOwner, null)
.Modify(j => j.LastError, outcome.Error);
switch (outcome.Result)
{
case JobResult.Done:
update.Modify(j => j.State, JobState.Done).Modify(j => j.FinishedAt, now);
break;
case JobResult.Retry when job.Attempts < maxAttempts:
update.Modify(j => j.State, JobState.Pending).Modify(j => j.RunAt, now + Backoff.After(job.Attempts));
break;
case JobResult.Defer:
update.Modify(j => j.State, JobState.Pending)
.Modify(j => j.RunAt, outcome.RetryAt ?? now + Backoff.After(job.Attempts))
.Modify(b => b.Inc(j => j.Attempts, -1));
break;
default:
update.Modify(j => j.State, JobState.Dead).Modify(j => j.FinishedAt, now);
break;
}
await update.ExecuteAsync(token);
}
public async Task<long> Reap(CancellationToken token)
{
var now = DateTime.UtcNow;
var result = await DB.Default.Update<Job>()
.Match(j => j.State == JobState.Running && j.LeasedUntil < now)
.Modify(j => j.State, JobState.Pending)
.Modify(j => j.RunAt, now)
.Modify(j => j.LeasedUntil, null)
.Modify(j => j.LeaseOwner, null)
.ExecuteAsync(token);
return result.ModifiedCount;
}
public async Task WaitForWork(JobKind kind, TimeSpan poll, CancellationToken token)
{
try
{
await SignalFor(kind).WaitAsync(poll, token);
}
catch (OperationCanceledException) when (token.IsCancellationRequested)
{
}
}
void Signal(JobKind kind)
{
try
{
SignalFor(kind).Release();
}
catch (SemaphoreFullException)
{
}
}
SemaphoreSlim SignalFor(JobKind kind) => _signals.GetOrAdd(kind, _ => new SemaphoreSlim(0, 64));
}
}
+137
View File
@@ -0,0 +1,137 @@
using PrivaPub.Models.Jobs;
using System.Collections.Concurrent;
namespace PrivaPub.Infrastructure.Jobs
{
public interface IJobHandler
{
JobKind Kind { get; }
int Concurrency { get; }
int MaxAttempts { get; }
int PerHostLimit { get; }
Task<JobOutcome> Handle(Job job, CancellationToken token);
}
public class JobWorker : BackgroundService
{
static readonly TimeSpan Poll = TimeSpan.FromSeconds(5);
static readonly TimeSpan ReapInterval = TimeSpan.FromSeconds(30);
readonly IJobQueue _queue;
readonly IEnumerable<IJobHandler> _handlers;
readonly ILogger<JobWorker> _logger;
public JobWorker(IJobQueue queue, IEnumerable<IJobHandler> handlers, ILogger<JobWorker> logger)
{
_queue = queue;
_handlers = handlers;
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var loops = _handlers.SelectMany(handler =>
{
var inFlight = new ConcurrentDictionary<string, int>(StringComparer.OrdinalIgnoreCase);
return Enumerable.Range(0, handler.Concurrency).Select(_ => Run(handler, inFlight, stoppingToken));
}).Append(Reap(stoppingToken));
await Task.WhenAll(loops);
}
async Task Run(IJobHandler handler, ConcurrentDictionary<string, int> inFlight, CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
Job job;
try
{
var busy = inFlight.Where(h => h.Value >= handler.PerHostLimit).Select(h => h.Key).ToList();
job = await _queue.Lease(handler.Kind, busy, stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception ex)
{
_logger.LogError(ex, "{Worker} could not lease a {Kind} job", nameof(JobWorker), handler.Kind);
await Delay(Poll, stoppingToken);
continue;
}
if (job == default)
{
await _queue.WaitForWork(handler.Kind, Poll, stoppingToken);
continue;
}
var host = job.Host ?? string.Empty;
inFlight.AddOrUpdate(host, 1, (_, count) => count + 1);
try
{
JobOutcome outcome;
try
{
outcome = await handler.Handle(job, stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception ex)
{
_logger.LogError(ex, "{Kind} job {Id} threw", handler.Kind, job.ID);
outcome = JobOutcome.Retry(ex.GetType().Name);
}
if (outcome.Result != JobResult.Done && job.Attempts >= handler.MaxAttempts && outcome.Result == JobResult.Retry)
_logger.LogWarning("{Kind} job {Id} for {Host} is dead after {Attempts} attempts: {Error}",
handler.Kind, job.ID, job.Host, job.Attempts, outcome.Error);
await _queue.Finish(job, outcome, handler.MaxAttempts, CancellationToken.None);
}
catch (Exception ex)
{
_logger.LogError(ex, "{Kind} job {Id} could not be finished; its lease will expire", handler.Kind, job.ID);
}
finally
{
inFlight.AddOrUpdate(host, 0, (_, count) => count - 1);
}
}
}
async Task Reap(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
var reaped = await _queue.Reap(stoppingToken);
if (reaped > 0)
_logger.LogWarning("{Worker} returned {Count} expired leases to the queue", nameof(JobWorker), reaped);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception ex)
{
_logger.LogError(ex, "{Worker} reaper failed", nameof(JobWorker));
}
await Delay(ReapInterval, stoppingToken);
}
}
static async Task Delay(TimeSpan delay, CancellationToken token)
{
try
{
await Task.Delay(delay, token);
}
catch (OperationCanceledException)
{
}
}
}
}
+39
View File
@@ -0,0 +1,39 @@
using Microsoft.AspNetCore.RateLimiting;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Signing;
using System.Threading.RateLimiting;
namespace PrivaPub.Infrastructure
{
public static class RateLimiting
{
public const string Accounts = "accounts";
public const string Inbox = "inbox";
public static IServiceCollection PrivaPubRateLimiting(this IServiceCollection service) =>
service.AddRateLimiter(options =>
{
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
options.AddPolicy(Accounts, context => RateLimitPartition.GetFixedWindowLimiter(
context.Connection.RemoteIpAddress?.ToString() ?? "unknown",
_ => new FixedWindowRateLimiterOptions { PermitLimit = 10, Window = TimeSpan.FromMinutes(1), QueueLimit = 0 }));
options.AddPolicy(Inbox, context => RateLimitPartition.GetTokenBucketLimiter(
SenderOrigin(context.Request) ?? "unsigned:" + context.Connection.RemoteIpAddress,
_ => new TokenBucketRateLimiterOptions
{
TokenLimit = 300,
TokensPerPeriod = 50,
ReplenishmentPeriod = TimeSpan.FromSeconds(10),
QueueLimit = 0
}));
});
static string SenderOrigin(HttpRequest request)
{
var signature = request.Headers["Signature"].ToString();
return string.IsNullOrEmpty(signature) ? default : Origin.Of(HttpSignatures.Parse(signature)?.KeyId);
}
}
}
+35 -9
View File
@@ -11,7 +11,15 @@ using System.Text.Json.Serialization;
using Microsoft.OpenApi;
using PrivaPub.Services.ClientToServer.Private;
using PrivaPub.Services.ClientToServer.Public;
using PrivaPub.Services.Federation;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Domain.Content;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using Microsoft.Extensions.Options;
namespace PrivaPub.Middleware
{
@@ -30,19 +38,35 @@ namespace PrivaPub.Middleware
//.AddHostedService<GroupsCleanerWorker>()
//.AddHostedService<PoliciesCleanerWorker>();
}
public static IServiceCollection PrivaPubFederationConfiguration(this IServiceCollection service)
public static IServiceCollection PrivaPubFederationConfiguration(this IServiceCollection service, IConfiguration configuration)
{
service.AddHttpClient(RemoteActorService.HttpClientName, client =>
{
client.Timeout = TimeSpan.FromSeconds(20);
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref}");
});
service.Configure<FederationOptions>(configuration.GetSection("Federation"));
service.AddHttpClient(FederationHttp.ClientName, (provider, client) =>
{
var baseAddress = provider.GetRequiredService<IOptionsMonitor<AppConfiguration>>().CurrentValue.BackendBaseAddress?.TrimEnd('/');
client.Timeout = FederationHttp.RequestTimeout;
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref} (+{baseAddress}/)");
})
.ConfigurePrimaryHttpMessageHandler(provider =>
SafeHttpHandlerFactory.Create(provider.GetRequiredService<IOptions<FederationOptions>>().Value));
return service
.AddSingleton<IFederationHttp, FederationHttp>()
.AddSingleton<IDomainBlocks, DomainBlocks>()
.AddSingleton<IContentRenderer, ContentRenderer>()
.AddSingleton<ILocalActorService, LocalActorService>()
.AddSingleton<IRemoteActorService, RemoteActorService>()
.AddSingleton<IDeliveryService, DeliveryService>()
.AddTransient<IInboxService, InboxService>()
.AddHostedService<DeliveryWorker>();
.AddSingleton<IInboxReceiver, InboxReceiver>()
.AddSingleton<IActivityHandler, FollowHandler>()
.AddSingleton<IActivityHandler, UndoHandler>()
.AddSingleton<IActivityHandler, CreateHandler>()
.AddSingleton<IActivityHandler, DeleteHandler>()
.AddSingleton<IActivityHandler, UpdateHandler>()
.AddSingleton<IJobHandler, InboxProcessor>()
.AddSingleton<IJobQueue, JobQueue>()
.AddSingleton<IHostCircuitBreaker, HostCircuitBreaker>()
.AddSingleton<IJobHandler, DeliveryJobHandler>()
.AddHostedService<JobWorker>();
}
public static IServiceCollection PrivaPubAuthServicesConfiguration(this IServiceCollection service, IConfiguration configuration)
{
@@ -120,6 +144,8 @@ namespace PrivaPub.Middleware
[new OpenApiSecuritySchemeReference("Bearer", document)] = []
});
})
.AddRazorPages(options => options.RootDirectory = "/Web/Pages")
.Services
.AddControllers(options => { options.Filters.Add<OperationCancelledExceptionFilter>(); })
.AddJsonOptions(options =>
{
+21
View File
@@ -0,0 +1,21 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Federation
{
public class DomainBlock : Entity
{
public string Domain { get; set; }
public DomainBlockSeverity Severity { get; set; }
public bool RejectMedia { get; set; }
public string PublicComment { get; set; }
public string PrivateComment { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
public enum DomainBlockSeverity
{
None,
Silence,
Suspend
}
}
+8
View File
@@ -1,5 +1,8 @@
using MongoDB.Entities;
using System.Security.Cryptography;
using System.Text;
namespace PrivaPub.Models.Group
{
public class DmGroup : Entity
@@ -7,9 +10,14 @@ namespace PrivaPub.Models.Group
public string Description { get; set; }
public List<GroupMember> Members { get; set; } = new();
public string ConversationURI { get; set; }//context of the federated thread
public string ParticipantsKey { get; set; }
public DateTime CreationDate { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
public DateTime? DeletionAt { get; set; }
public static string KeyOf(IEnumerable<GroupMember> members) =>
Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(string.Join("\n",
members.Select(m => (m.IsForeign ? "remote:" : "local:") + m.AvatarId).Distinct().Order(StringComparer.Ordinal)))));
}
}
+11
View File
@@ -1,5 +1,7 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Ids;
namespace PrivaPub.Models.Group
{
public class Group : Entity
@@ -17,6 +19,7 @@ namespace PrivaPub.Models.Group
public string PrivateKey { get; set; }
public string PublicKey { get; set; }
public GroupKind Kind { get; set; }
public bool IsDiscoverable { get; set; } = true;
public bool ManuallyApprovesMembers { get; set; }
public string OwnerAvatarId { get; set; }
@@ -28,6 +31,8 @@ namespace PrivaPub.Models.Group
public DateTime CreationDate { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
public DateTime? DeletionAt { get; set; }
public override object GenerateNewID() => PrivacyIds.ForDay(DateTime.UtcNow);
}
public class GroupMember
@@ -38,6 +43,12 @@ namespace PrivaPub.Models.Group
public DateTime JoinedAt { get; set; } = DateTime.UtcNow;
}
public enum GroupKind
{
Circle,
Community
}
public enum GroupRole
{
Member,
+34
View File
@@ -0,0 +1,34 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Jobs
{
public class Job : Entity
{
public JobKind Kind { get; set; }
public JobState State { get; set; }
public string Payload { get; set; }
public string Host { get; set; }
public string DedupeKey { get; set; }
public int Attempts { get; set; }
public DateTime RunAt { get; set; } = DateTime.UtcNow;
public DateTime? LeasedUntil { get; set; }
public string LeaseOwner { get; set; }
public string LastError { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime? FinishedAt { get; set; }
}
public enum JobKind
{
Deliver,
ProcessInbox
}
public enum JobState
{
Pending,
Running,
Done,
Dead
}
}
+14
View File
@@ -0,0 +1,14 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Jobs
{
public class RemoteInstance : Entity
{
public string Host { get; set; }
public int ConsecutiveFailures { get; set; }
public DateTime? UnavailableUntil { get; set; }
public DateTime? LastSuccessAt { get; set; }
public DateTime? LastFailureAt { get; set; }
public string LastError { get; set; }
}
}
+8
View File
@@ -0,0 +1,8 @@
namespace PrivaPub.Models.Post
{
public enum ContentFormat
{
Markdown,
Html
}
}
+2
View File
@@ -9,6 +9,8 @@ namespace PrivaPub.Models.Post
public string GroupId { get; set; }
public string Title { get; set; }
public string Text { get; set; }
public string ContentHtml { get; set; }
public ContentFormat ContentFormat { get; set; }
public List<PostMedia> Media { get; set; } = new();
public List<float> Location { get; set; } = new();
public bool HasContentWarning { get; set; } = false;
+55 -2
View File
@@ -4,22 +4,75 @@ namespace PrivaPub.Models.Post
{
public class Post : Entity
{
public string GroupUserId { get; set; }
public string AnsweringToPostId { get; set; }
public string GroupUserId { get; set; }//local author's Avatar.ID
public string AuthorAccountId { get; set; }//Avatar.ID or ForeignAvatar.ID
public string AnsweringToPostId { get; set; }//the parent's Post.ID when we hold it
public string InReplyToURI { get; set; }
public string InReplyToAccountId { get; set; }
public string GroupId { get; set; }
public string ConversationId { get; set; }//DmGroup.ID of a direct post
public string ReblogOfPostId { get; set; }
public PostVisibility Visibility { get; set; }
public string Title { get; set; }
public string SpoilerText { get; set; }
public string Text { get; set; }
public string ContentHtml { get; set; }
public ContentFormat ContentFormat { get; set; }
public string Language { get; set; }
public List<PostMention> Mentions { get; set; } = new();
public List<string> Tags { get; set; } = new();
public List<PostMedia> Media { get; set; } = new();
public List<float> Location { get; set; } = new();
public float RangeKm { get; set; } = 5.0f;
public bool HasContentWarning { get; set; } = false;
public bool IsLocalOnly { get; set; }
public bool IsFederatedCopy { get; set; }
public string ObjectURI { get; set; }//the Note's id
public string ActivityURI { get; set; }//the Create's id
public string ActorURI { get; set; }//attributedTo
public string Url { get; set; }
public string ContextURI { get; set; }
public string QuoteURI { get; set; }
public List<string> To { get; set; } = new();
public List<string> Cc { get; set; } = new();
public int RepliesCount { get; set; }
public int FavouritesCount { get; set; }
public int ReblogsCount { get; set; }
public List<PostRevision> Revisions { get; set; } = new();
public DateTime CreationDate { get; set; } = DateTime.UtcNow;
public DateTime? UpdateDate { get; set; } = DateTime.UtcNow;
public DateTime? EditedAt { get; set; }
public DateTime? DeletedAt { get; set; }
public string UpdateReason { get; set; }
}
public enum PostVisibility
{
Public,
Unlisted,
FollowersOnly,
Direct,
Circle,
LocalGeo
}
public class PostMention
{
public string ActorURI { get; set; }
public string AccountId { get; set; }
public string Handle { get; set; }
public bool IsLocal { get; set; }
}
public class PostRevision
{
public string Title { get; set; }
public string SpoilerText { get; set; }
public string ContentHtml { get; set; }
public bool HasContentWarning { get; set; }
public DateTime EditedAt { get; set; }
}
}
-12
View File
@@ -1,12 +0,0 @@
namespace PrivaPub.Models.Post
{
public class PostBoost
{
public string PostId { get; set; }
public string GroupUserId { get; set; }
public bool IsFederatedCopy { get; set; }
public DateTime CreationDate { get; set; }
}
}
+7 -1
View File
@@ -1,4 +1,4 @@
namespace PrivaPub.Models.Post
namespace PrivaPub.Models.Post
{
public class PostMedia
{
@@ -8,5 +8,11 @@
public string Extension { get; set; }
public string Path { get; set; }
public string URL { get; set; }
public string RemoteURL { get; set; }
public string Description { get; set; }
public string Blurhash { get; set; }
public int? Width { get; set; }
public int? Height { get; set; }
public float[] Focus { get; set; }
}
}
+6
View File
@@ -1,5 +1,7 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Ids;
namespace PrivaPub.Models.User
{
public class Avatar : Entity
@@ -34,6 +36,8 @@ namespace PrivaPub.Models.User
public DateTime? SuspendedAt { get; set; }
public DateTime? BannedAt { get; set; }
public DateTime? DeletionAt { get; set; }
public override object GenerateNewID() => PrivacyIds.ForDay(DateTime.UtcNow);
}
public class ForeignAvatar : Entity
@@ -57,6 +61,8 @@ namespace PrivaPub.Models.User
public string ActorURI { get; set; }//id
public string PublicKeyId { get; set; }
public string FollowersURL { get; set; }
public string FollowingURL { get; set; }
public string SharedInboxURL { get; set; }
public string InboxURL { get; set; }
public string OutboxURL { get; set; }
+14
View File
@@ -0,0 +1,14 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
namespace PrivaPub.Models.User
{
public class ReservedName : Entity
{
public string Name { get; set; }
public LocalActorKind OwnerKind { get; set; }
public string OwnerId { get; set; }
public DateTime CreationDate { get; set; } = DateTime.UtcNow;
}
}
+1
View File
@@ -7,6 +7,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="HtmlSanitizer" Version="9.2.1039" />
<PackageReference Include="MailKit" Version="4.18.0" />
<PackageReference Include="Markdig" Version="1.4.0" />
<PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.9" />
+27 -3
View File
@@ -11,6 +11,10 @@ using Serilog;
using PrivaPub.Data;
using PrivaPub.Extensions;
using PrivaPub.Infrastructure;
using PrivaPub.Infrastructure.Cli;
using PrivaPub.Infrastructure.Data;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Middleware;
using PrivaPub.Models;
using PrivaPub.Services;
@@ -47,8 +51,9 @@ try
.PrivaPubOptimizationConfiguration()
.PrivaPubDataBaseConfiguration()
.PrivaPubServicesConfiguration()
.PrivaPubFederationConfiguration()
.PrivaPubFederationConfiguration(builder.Configuration)
.PrivaPubCORSConfiguration()
.PrivaPubRateLimiting()
.PrivaPubMiddlewareConfiguration();
}
catch (Exception ex)
@@ -57,11 +62,24 @@ try
throw;
}
var federationOptions = builder.Configuration.GetSection("Federation").Get<FederationOptions>() ?? new();
if (builder.Environment.IsProduction() && (federationOptions.AllowPrivateNetworks || federationOptions.AllowPlainHttp))
throw new InvalidOperationException("Federation:AllowPrivateNetworks and Federation:AllowPlainHttp are for test networks and must stay off in Production.");
try
{
BsonSerializer.RegisterSerializer(new GuidSerializer(GuidRepresentation.Standard));
var mongoSettings = builder.Configuration.GetSection(nameof(MongoSettings)).Get<MongoSettings>();
await DB.InitAsync(mongoSettings.Database, MongoClientSettings.FromConnectionString(mongoSettings.ConnectionString));
EntityMaps.Warm();
await DB.Default.MigrateAsync<Program>();
await Indexes.Create();
if (args is ["admin", ..])
{
Environment.ExitCode = await AdminCommands.Run(args[1..]);
return;
}
}
catch (Exception ex)
{
@@ -92,8 +110,11 @@ try
});
}
app.UseSwagger();
app.UseSwaggerUI();
if (app.Environment.IsDevelopment())
{
app.UseSwagger();
app.UseSwaggerUI();
}
app.UseHttpsRedirection();
app.UseCors("DefaultCORS");
@@ -103,6 +124,7 @@ try
app.UseRequestLocalization(await localizationService.Get());
app.UseRouting();
app.UseRateLimiter();
app.UseAuthentication();
app.UseAuthorization();
@@ -117,6 +139,7 @@ try
builtAt = BuildInfo.BuiltAt,
}));
app.MapControllers();
app.MapRazorPages();
//app.MapFallbackToFile("index.html");
}
catch (Exception ex)
@@ -131,6 +154,7 @@ try
var dbClient = app.Services.GetService(typeof(DbEntities)) as DbEntities;
var passwordHasher = app.Services.GetService(typeof(IPasswordHasher)) as IPasswordHasher;
await dbClient.Init(passwordHasher);
await app.Services.GetRequiredService<PrivaPub.Federation.Moderation.IDomainBlocks>().Reload(CancellationToken.None);
}
catch (Exception ex)
{
-12
View File
@@ -1,12 +0,0 @@
namespace PrivaPub.Services
{
public class ActivityPubClient : HttpClient
{
//readonly HttpClient client;
//public ActivityPubClient(HttpClient client)
//{
// this.client = client;
//}
}
}
@@ -5,10 +5,11 @@ using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.ClientModels.User;
using PrivaPub.ClientModels.User.Avatar;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Resources;
using PrivaPub.Services.Federation;
using PrivaPub.StaticServices;
using PrivaPub.Federation.Actors;
namespace PrivaPub.Services.ClientToServer.Private
{
@@ -64,6 +65,9 @@ namespace PrivaPub.Services.ClientToServer.Private
PublicKey = publicKey,
Domain = _localActors.BaseAddress
};
newAvatar.ID = (string)newAvatar.GenerateNewID();
if (!await _localActors.TryReserveUserName(userName, LocalActorKind.Person, newAvatar.ID, default))
return result.Invalidate(_localizer["The username '{0}' is already take.", userName]);
if (form.Settings is { IsDefault: false })
newAvatar.Settings = ToSettings(form.Settings);
@@ -81,7 +85,7 @@ namespace PrivaPub.Services.ClientToServer.Private
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(PrivateAvatarUsersService)}.{nameof(InsertAvatar)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -118,7 +122,7 @@ namespace PrivaPub.Services.ClientToServer.Private
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(PrivateAvatarUsersService)}.{nameof(UpdateAvatar)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -139,7 +143,7 @@ namespace PrivaPub.Services.ClientToServer.Private
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(PrivateAvatarUsersService)}.{nameof(GetRootAvatars)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -1,178 +0,0 @@
using Markdig;
using PrivaPub.Models.Federation;
using System.Globalization;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
using DmPostEntity = PrivaPub.Models.Post.DmPost;
namespace PrivaPub.Services.Federation
{
public static class ActivityPubRenderer
{
public const string ActivityStreams = "https://www.w3.org/ns/activitystreams";
public const string Public = "https://www.w3.org/ns/activitystreams#Public";
static readonly MarkdownPipeline Pipeline = new MarkdownPipelineBuilder().DisableHtml().Build();
public static string Html(string markdown) =>
string.IsNullOrEmpty(markdown) ? string.Empty : Markdown.ToHtml(markdown, Pipeline).Trim();
public static string Timestamp(DateTime value) =>
DateTime.SpecifyKind(value, DateTimeKind.Utc).ToString("yyyy-MM-ddTHH:mm:ssZ", CultureInfo.InvariantCulture);
public static JsonObject Actor(LocalActor actor)
{
var document = new JsonObject
{
["@context"] = new JsonArray(ActivityStreams, "https://w3id.org/security/v1"),
["id"] = actor.Uri,
["type"] = actor.Kind switch
{
LocalActorKind.Group => "Group",
LocalActorKind.Application => "Application",
_ => "Person"
},
["preferredUsername"] = actor.UserName,
["name"] = actor.Name,
["summary"] = Html(actor.Summary),
["url"] = actor.Uri,
["inbox"] = actor.Inbox,
["outbox"] = actor.Outbox,
["followers"] = actor.Followers,
["following"] = actor.Following,
["published"] = Timestamp(actor.Published),
["manuallyApprovesFollowers"] = actor.ManuallyApprovesFollowers,
["discoverable"] = actor.Discoverable,
["endpoints"] = new JsonObject { ["sharedInbox"] = actor.SharedInbox },
["publicKey"] = new JsonObject
{
["id"] = actor.KeyId,
["owner"] = actor.Uri,
["publicKeyPem"] = Keys.ToSubjectPublicKeyInfoPem(actor.PublicKeyPem)
}
};
if (!string.IsNullOrEmpty(actor.PictureURL))
document["icon"] = new JsonObject { ["type"] = "Image", ["url"] = actor.PictureURL };
if (!string.IsNullOrEmpty(actor.ThumbnailURL))
document["image"] = new JsonObject { ["type"] = "Image", ["url"] = actor.ThumbnailURL };
return document;
}
public static JsonObject Note(PostEntity post, LocalActor author, LocalActor group, string inReplyTo)
{
var to = new JsonArray(Public);
var cc = new JsonArray(author.Followers);
if (group != default)
cc.Add(group.Uri);
var note = NoteBody(author.PostUri(post.ID), author, post.Title, post.Text, post.HasContentWarning,
post.CreationDate, to, cc, inReplyTo);
if (group != default)
note["audience"] = group.Uri;
return note;
}
public static JsonObject DirectNote(DmPostEntity post, LocalActor author, IReadOnlyList<(string Uri, string Handle)> recipients,
string context)
{
var note = NoteBody(author.PostUri(post.ID), author, post.Title, post.Text, post.HasContentWarning,
post.CreationDate, new JsonArray(recipients.Select(r => (JsonNode)r.Uri).ToArray()), new JsonArray(), default);
note["tag"] = new JsonArray(recipients.Select(r => (JsonNode)new JsonObject
{
["type"] = "Mention",
["href"] = r.Uri,
["name"] = "@" + r.Handle
}).ToArray());
note["content"] = string.Join(" ", recipients.Select(r => $"<span class=\"h-card\"><a href=\"{r.Uri}\" class=\"u-url mention\">@{r.Handle}</a></span>"))
+ " " + note["content"]!.GetValue<string>();
if (!string.IsNullOrEmpty(context))
note["context"] = context;
return note;
}
static JsonObject NoteBody(string id, LocalActor author, string title, string text, bool sensitive,
DateTime published, JsonArray to, JsonArray cc, string inReplyTo)
{
var note = new JsonObject
{
["id"] = id,
["type"] = "Note",
["attributedTo"] = author.Uri,
["content"] = Html(text),
["published"] = Timestamp(published),
["url"] = id,
["to"] = to,
["cc"] = cc,
["sensitive"] = sensitive
};
if (sensitive && !string.IsNullOrEmpty(title))
note["summary"] = title;
else if (!string.IsNullOrEmpty(title))
note["name"] = title;
if (!string.IsNullOrEmpty(inReplyTo))
note["inReplyTo"] = inReplyTo;
return note;
}
public static JsonObject Create(LocalActor actor, JsonObject note, string activityId) => new()
{
["@context"] = ActivityStreams,
["id"] = actor.ActivityUri(activityId),
["type"] = "Create",
["actor"] = actor.Uri,
["published"] = note["published"]?.DeepClone(),
["to"] = note["to"]?.DeepClone(),
["cc"] = note["cc"]?.DeepClone(),
["object"] = note
};
public static JsonObject Announce(LocalActor group, string objectUri, string activityId) => new()
{
["@context"] = ActivityStreams,
["id"] = group.ActivityUri(activityId),
["type"] = "Announce",
["actor"] = group.Uri,
["published"] = Timestamp(DateTime.UtcNow),
["to"] = new JsonArray(Public),
["cc"] = new JsonArray(group.Followers),
["object"] = objectUri
};
public static JsonObject Delete(LocalActor actor, string objectUri, string activityId, JsonArray to, JsonArray cc) => new()
{
["@context"] = ActivityStreams,
["id"] = actor.ActivityUri(activityId),
["type"] = "Delete",
["actor"] = actor.Uri,
["to"] = to,
["cc"] = cc,
["object"] = new JsonObject { ["id"] = objectUri, ["type"] = "Tombstone" }
};
public static JsonObject Accept(LocalActor actor, JsonNode follow, string activityId) => new()
{
["@context"] = ActivityStreams,
["id"] = actor.ActivityUri(activityId),
["type"] = "Accept",
["actor"] = actor.Uri,
["object"] = follow.DeepClone()
};
public static JsonObject OrderedCollection(string id, int totalItems, IEnumerable<JsonNode> items)
{
var collection = new JsonObject
{
["@context"] = ActivityStreams,
["id"] = id,
["type"] = "OrderedCollection",
["totalItems"] = totalItems
};
if (items != default)
collection["orderedItems"] = new JsonArray(items.ToArray());
return collection;
}
}
}
@@ -1,178 +0,0 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.StaticServices;
using System.Net;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Services.Federation
{
public interface IDeliveryService
{
Task Enqueue(LocalActor signer, IEnumerable<string> inboxes, JsonObject activity, CancellationToken token);
Task EnqueueToFollowers(LocalActor signer, JsonObject activity, CancellationToken token, IEnumerable<string> extraInboxes = default);
Task<IReadOnlyList<string>> FollowerInboxes(LocalActor actor, CancellationToken token);
}
public class DeliveryService : IDeliveryService
{
readonly DbEntities _dbEntities;
public DeliveryService(DbEntities dbEntities)
{
_dbEntities = dbEntities;
}
public async Task Enqueue(LocalActor signer, IEnumerable<string> inboxes, JsonObject activity, CancellationToken token)
{
var body = activity.ToJsonString();
var deliveries = inboxes
.Where(i => !string.IsNullOrEmpty(i) && !i.StartsWith(signer.BaseAddress + "/", StringComparison.OrdinalIgnoreCase))
.Distinct(StringComparer.Ordinal)
.Select(inbox => new Delivery
{
SignerId = signer.Id,
SignerKind = signer.Kind,
InboxURL = inbox,
Body = body
})
.ToList();
if (deliveries.Count > 0)
await DB.Default.SaveAsync(deliveries, token);
}
public async Task EnqueueToFollowers(LocalActor signer, JsonObject activity, CancellationToken token, IEnumerable<string> extraInboxes = default)
{
var inboxes = (await FollowerInboxes(signer, token)).Concat(extraInboxes ?? Enumerable.Empty<string>());
await Enqueue(signer, inboxes, activity, token);
}
public async Task<IReadOnlyList<string>> FollowerInboxes(LocalActor actor, CancellationToken token)
{
var followers = await _dbEntities.Followers
.Match(f => f.LocalActorId == actor.Id && f.LocalActorKind == actor.Kind && f.IsAccepted)
.ExecuteAsync(token);
return followers.Select(f => string.IsNullOrEmpty(f.SharedInboxURL) ? f.InboxURL : f.SharedInboxURL)
.Distinct(StringComparer.Ordinal)
.ToList();
}
}
public class DeliveryWorker : BackgroundService
{
const int MaxAttempts = 8;
static readonly TimeSpan Poll = TimeSpan.FromSeconds(3);
readonly IServiceProvider _services;
readonly IHttpClientFactory _httpClientFactory;
readonly ILogger<DeliveryWorker> _logger;
public DeliveryWorker(IServiceProvider services, IHttpClientFactory httpClientFactory, ILogger<DeliveryWorker> logger)
{
_services = services;
_httpClientFactory = httpClientFactory;
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
await DeliverDue(stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception ex)
{
_logger.LogError(ex, "{Worker} pass failed", nameof(DeliveryWorker));
}
await Task.Delay(Poll, stoppingToken);
}
}
async Task DeliverDue(CancellationToken token)
{
using var scope = _services.CreateScope();
var dbEntities = scope.ServiceProvider.GetRequiredService<DbEntities>();
var actors = scope.ServiceProvider.GetRequiredService<ILocalActorService>();
var now = DateTime.UtcNow;
var due = await dbEntities.Deliveries
.Match(d => !d.DeliveredAt.HasValue && !d.AbandonedAt.HasValue && d.NextAttemptAt <= now)
.Sort(d => d.NextAttemptAt, Order.Ascending)
.Limit(20)
.ExecuteAsync(token);
foreach (var delivery in due)
{
var signer = await actors.FindById(delivery.SignerKind, delivery.SignerId, token);
if (signer == default)
{
delivery.AbandonedAt = DateTime.UtcNow;
delivery.LastError = "the signing actor no longer exists";
await DB.Default.SaveAsync(delivery, token);
continue;
}
await Deliver(delivery, signer, token);
await DB.Default.SaveAsync(delivery, token);
}
}
async Task Deliver(Delivery delivery, LocalActor signer, CancellationToken token)
{
delivery.Attempts++;
try
{
if (!Uri.TryCreate(delivery.InboxURL, UriKind.Absolute, out var inbox) || !RemoteActorService.IsFetchable(inbox))
{
delivery.AbandonedAt = DateTime.UtcNow;
delivery.LastError = "not a deliverable inbox";
return;
}
var body = Encoding.UTF8.GetBytes(delivery.Body);
using var request = new HttpRequestMessage(HttpMethod.Post, inbox)
{
Content = new ByteArrayContent(body)
};
request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(RemoteActorService.ActivityJson);
HttpSignatures.Sign(request, signer, body);
using var response = await _httpClientFactory.CreateClient(RemoteActorService.HttpClientName).SendAsync(request, token);
if (response.IsSuccessStatusCode)
{
delivery.DeliveredAt = DateTime.UtcNow;
delivery.LastError = default;
return;
}
delivery.LastError = $"{(int)response.StatusCode} {response.ReasonPhrase}";
if (response.StatusCode is HttpStatusCode.Gone or HttpStatusCode.NotFound or HttpStatusCode.BadRequest or HttpStatusCode.Forbidden)
{
delivery.AbandonedAt = DateTime.UtcNow;
_logger.LogWarning("Delivery {Id} to {Inbox} abandoned: {Status}", delivery.ID, delivery.InboxURL, delivery.LastError);
return;
}
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException && !token.IsCancellationRequested)
{
delivery.LastError = ex.Message;
}
if (delivery.Attempts >= MaxAttempts)
{
delivery.AbandonedAt = DateTime.UtcNow;
_logger.LogWarning("Delivery {Id} to {Inbox} abandoned after {Attempts} attempts: {Error}",
delivery.ID, delivery.InboxURL, delivery.Attempts, delivery.LastError);
return;
}
delivery.NextAttemptAt = DateTime.UtcNow.AddMinutes(Math.Pow(2, delivery.Attempts));
}
}
}
@@ -1,398 +0,0 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json;
using System.Text.Json.Nodes;
using DmPostEntity = PrivaPub.Models.Post.DmPost;
using GroupEntity = PrivaPub.Models.Group.Group;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Services.Federation
{
public sealed record InboxResult(int StatusCode, string Error = default);
public interface IInboxService
{
Task<InboxResult> Receive(HttpRequest request, LocalActor recipient, CancellationToken token);
}
public class InboxService : IInboxService
{
const int MaxBodyBytes = 1024 * 1024;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IDeliveryService _delivery;
readonly ILogger<InboxService> _logger;
public InboxService(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors,
IDeliveryService delivery, ILogger<InboxService> logger)
{
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_delivery = delivery;
_logger = logger;
}
public async Task<InboxResult> Receive(HttpRequest request, LocalActor recipient, CancellationToken token)
{
if (request.ContentLength > MaxBodyBytes)
return new(StatusCodes.Status413PayloadTooLarge);
using var buffer = new MemoryStream();
await request.Body.CopyToAsync(buffer, token);
if (buffer.Length > MaxBodyBytes)
return new(StatusCodes.Status413PayloadTooLarge);
var body = buffer.ToArray();
JsonNode activity;
try
{
activity = JsonNode.Parse(body);
}
catch (JsonException)
{
return new(StatusCodes.Status400BadRequest, "the body is not JSON");
}
if (activity is not JsonObject)
return new(StatusCodes.Status400BadRequest, "the body is not an activity");
var type = Value(activity, "type");
var actorUri = Id(activity["actor"]);
if (string.IsNullOrEmpty(type) || string.IsNullOrEmpty(actorUri))
return new(StatusCodes.Status400BadRequest, "type and actor are required");
var parameters = HttpSignatures.Parse(request.Headers["Signature"].ToString());
if (parameters == default)
return new(StatusCodes.Status401Unauthorized, "missing or unreadable Signature header");
var requestProblem = HttpSignatures.CheckRequest(request, parameters, body);
if (requestProblem != default)
return new(StatusCodes.Status401Unauthorized, requestProblem);
var signAs = recipient ?? await _localActors.GetInstanceActor(token);
var signingString = HttpSignatures.SigningString(request, parameters);
var keyOwner = await _remoteActors.GetActorByKeyId(parameters.KeyId, signAs, refresh: false, token);
if (keyOwner == default && type == "Delete" && Id(activity["object"]) == actorUri)
return new(StatusCodes.Status202Accepted);
if (keyOwner == default || !HttpSignatures.Verify(keyOwner.PublicKey, signingString, parameters.Signature))
{
keyOwner = await _remoteActors.GetActorByKeyId(parameters.KeyId, signAs, refresh: true, token);
if (keyOwner == default || !HttpSignatures.Verify(keyOwner.PublicKey, signingString, parameters.Signature))
return new(StatusCodes.Status401Unauthorized, "the signature does not verify");
}
if (!string.Equals(keyOwner.ActorURI, actorUri, StringComparison.Ordinal))
return new(StatusCodes.Status401Unauthorized, "the activity's actor is not the key's owner");
_logger.LogInformation("Inbox {Recipient}: {Type} from {Actor}", recipient?.Handle ?? "shared", type, actorUri);
return type switch
{
"Follow" => await Follow(activity, keyOwner, token),
"Undo" => await Undo(activity, keyOwner, token),
"Create" => await Create(activity, keyOwner, signAs, token),
"Delete" => await Delete(activity, keyOwner, token),
"Update" => await Update(activity, keyOwner, signAs, token),
_ => new(StatusCodes.Status202Accepted)
};
}
async Task<InboxResult> Follow(JsonNode follow, ForeignAvatar follower, CancellationToken token)
{
var target = await _localActors.FindByUri(Id(follow["object"]), token);
if (target == default || target.Kind == LocalActorKind.Application)
return new(StatusCodes.Status404NotFound, "no such local actor");
var existing = await _dbEntities.Followers
.Match(f => f.LocalActorId == target.Id && f.LocalActorKind == target.Kind && f.ActorURI == follower.ActorURI)
.ExecuteFirstAsync(token);
var record = existing ?? new Follower
{
LocalActorId = target.Id,
LocalActorKind = target.Kind,
ActorURI = follower.ActorURI
};
record.InboxURL = follower.InboxURL;
record.SharedInboxURL = follower.SharedInboxURL;
record.FollowActivityURI = Id(follow);
record.IsAccepted = existing?.IsAccepted == true || !target.ManuallyApprovesFollowers;
await DB.Default.SaveAsync(record, token);
if (!record.IsAccepted)
return new(StatusCodes.Status202Accepted);
if (target.Kind == LocalActorKind.Group)
await AddForeignMember(target.Id, follower.ActorURI, token);
var accept = ActivityPubRenderer.Accept(target, follow, $"accept-{record.ID}-{DateTime.UtcNow.Ticks}");
await _delivery.Enqueue(target, new[] { follower.InboxURL }, accept, token);
return new(StatusCodes.Status202Accepted);
}
async Task<InboxResult> Undo(JsonNode undo, ForeignAvatar actor, CancellationToken token)
{
var inner = undo["object"];
var innerType = inner is JsonObject ? Value(inner, "type") : default;
var innerId = Id(inner);
if (innerType is not (null or "Follow"))
return new(StatusCodes.Status202Accepted);
var followers = await _dbEntities.Followers.Match(f => f.ActorURI == actor.ActorURI).ExecuteAsync(token);
var targetUri = inner is JsonObject ? Id(inner["object"]) : default;
foreach (var follower in followers)
{
var matchesActivity = innerId != default && follower.FollowActivityURI == innerId;
var target = targetUri == default ? default : await _localActors.FindByUri(targetUri, token);
var matchesTarget = target != default && target.Id == follower.LocalActorId && target.Kind == follower.LocalActorKind;
if (!matchesActivity && !matchesTarget)
continue;
await DB.Default.DeleteAsync<Follower>(follower.ID);
if (follower.LocalActorKind == LocalActorKind.Group)
await RemoveForeignMember(follower.LocalActorId, actor.ActorURI, token);
}
return new(StatusCodes.Status202Accepted);
}
async Task<InboxResult> Create(JsonNode create, ForeignAvatar author, LocalActor signAs, CancellationToken token)
{
var note = create["object"];
if (note is JsonValue)
{
using var fetched = await _remoteActors.Fetch(Id(note), signAs, token);
note = fetched == default ? default : JsonNode.Parse(fetched.RootElement.GetRawText());
}
if (note is not JsonObject || Value(note, "type") is not ("Note" or "Article" or "Page" or "Question"))
return new(StatusCodes.Status202Accepted);
var objectUri = Id(note);
if (string.IsNullOrEmpty(objectUri) || Id(note["attributedTo"]) != author.ActorURI)
return new(StatusCodes.Status400BadRequest, "the object is not attributed to the actor");
var addressed = Addresses(note).Concat(Addresses(create)).Distinct(StringComparer.Ordinal).ToList();
var isPublic = addressed.Contains(ActivityPubRenderer.Public) || addressed.Contains("as:Public") || addressed.Contains("Public");
var inReplyTo = Id(note["inReplyTo"]);
var localTargets = new List<LocalActor>();
foreach (var uri in addressed.Concat(new[] { Id(note["audience"]) }).Where(u => u != default).Distinct())
{
var local = await _localActors.FindByUri(uri, token);
if (local != default && localTargets.All(l => l.Id != local.Id))
localTargets.Add(local);
}
if (isPublic || addressed.Any(a => a == author.ActorURI + "/followers" || a.EndsWith("/followers")))
{
var group = localTargets.FirstOrDefault(t => t.Kind == LocalActorKind.Group);
if (group != default && !await IsAcceptedFollower(group, author.ActorURI, token))
group = default;
if (group == default && localTargets.All(t => t.Kind != LocalActorKind.Person))
return new(StatusCodes.Status202Accepted);
if (await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri).ExecuteAnyAsync(token))
return new(StatusCodes.Status202Accepted);
var post = new PostEntity
{
ObjectURI = objectUri,
ActorURI = author.ActorURI,
GroupId = group?.Id,
Title = Value(note, "summary") ?? Value(note, "name"),
Text = Value(note, "content"),
HasContentWarning = note["sensitive"] is JsonValue sensitive && sensitive.TryGetValue<bool>(out var s) && s,
AnsweringToPostId = await LocalPostId(inReplyTo, token) ?? inReplyTo,
IsFederatedCopy = true,
CreationDate = DateTime.TryParse(Value(note, "published"), out var published) ? published.ToUniversalTime() : DateTime.UtcNow
};
await DB.Default.SaveAsync(post, token);
if (group != default)
{
var announce = ActivityPubRenderer.Announce(group, objectUri, $"announce-{post.ID}");
await _delivery.EnqueueToFollowers(group, announce, token);
}
return new(StatusCodes.Status202Accepted);
}
var recipients = localTargets.Where(t => t.Kind == LocalActorKind.Person).ToList();
if (recipients.Count == 0)
return new(StatusCodes.Status202Accepted);
if (await _dbEntities.DmPosts.Match(p => p.ObjectURI == objectUri).ExecuteAnyAsync(token))
return new(StatusCodes.Status202Accepted);
var participants = addressed.Where(a => a != ActivityPubRenderer.Public).Append(author.ActorURI).ToList();
var dmGroup = await FindOrCreateDmGroup(participants, Value(note, "context") ?? Value(note, "conversation"), token);
var dm = new DmPostEntity
{
ObjectURI = objectUri,
ActorURI = author.ActorURI,
GroupId = dmGroup.ID,
Title = Value(note, "summary"),
Text = Value(note, "content"),
HasContentWarning = note["sensitive"] is JsonValue dmSensitive && dmSensitive.TryGetValue<bool>(out var ds) && ds,
AnsweringToPostId = inReplyTo,
IsFederatedCopy = true,
CreationDate = DateTime.TryParse(Value(note, "published"), out var dmPublished) ? dmPublished.ToUniversalTime() : DateTime.UtcNow
};
await DB.Default.SaveAsync(dm, token);
await DB.Default.Update<DmGroup>().MatchID(dmGroup.ID).Modify(g => g.UpdatedAt, DateTime.UtcNow).ExecuteAsync(token);
return new(StatusCodes.Status202Accepted);
}
async Task<InboxResult> Delete(JsonNode delete, ForeignAvatar actor, CancellationToken token)
{
var objectUri = Id(delete["object"]);
if (string.IsNullOrEmpty(objectUri))
return new(StatusCodes.Status202Accepted);
if (objectUri == actor.ActorURI)
{
actor.DeletionAt = DateTime.UtcNow;
actor.AccountState = AvatarAccountState.Deleted;
await DB.Default.SaveAsync(actor, token);
var followers = await _dbEntities.Followers.Match(f => f.ActorURI == actor.ActorURI).ExecuteAsync(token);
foreach (var follower in followers)
{
await DB.Default.DeleteAsync<Follower>(follower.ID);
if (follower.LocalActorKind == LocalActorKind.Group)
await RemoveForeignMember(follower.LocalActorId, actor.ActorURI, token);
}
return new(StatusCodes.Status202Accepted);
}
await DB.Default.DeleteAsync<PostEntity>(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI);
await DB.Default.DeleteAsync<DmPostEntity>(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI);
return new(StatusCodes.Status202Accepted);
}
async Task<InboxResult> Update(JsonNode update, ForeignAvatar actor, LocalActor signAs, CancellationToken token)
{
var inner = update["object"];
if (Id(inner) == actor.ActorURI)
{
await _remoteActors.GetActor(actor.ActorURI, signAs, refresh: true, token);
return new(StatusCodes.Status202Accepted);
}
if (inner is not JsonObject || Id(inner["attributedTo"]) != actor.ActorURI)
return new(StatusCodes.Status202Accepted);
var objectUri = Id(inner);
var text = Value(inner, "content");
await DB.Default.Update<PostEntity>()
.Match(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI)
.Modify(p => p.Text, text)
.Modify(p => p.UpdateDate, DateTime.UtcNow)
.ExecuteAsync(token);
await DB.Default.Update<DmPostEntity>()
.Match(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI)
.Modify(p => p.Text, text)
.Modify(p => p.UpdateDate, DateTime.UtcNow)
.ExecuteAsync(token);
return new(StatusCodes.Status202Accepted);
}
async Task<bool> IsAcceptedFollower(LocalActor group, string actorUri, CancellationToken token) =>
await _dbEntities.Followers
.Match(f => f.LocalActorId == group.Id && f.LocalActorKind == LocalActorKind.Group && f.ActorURI == actorUri && f.IsAccepted)
.ExecuteAnyAsync(token);
async Task AddForeignMember(string groupId, string actorUri, CancellationToken token)
{
var group = await _dbEntities.Groups.MatchID(groupId).ExecuteFirstAsync(token);
if (group == default || group.Members.Any(m => m.IsForeign && m.AvatarId == actorUri))
return;
group.Members.Add(new GroupMember { AvatarId = actorUri, IsForeign = true });
group.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(group, token);
}
async Task RemoveForeignMember(string groupId, string actorUri, CancellationToken token)
{
var group = await _dbEntities.Groups.MatchID(groupId).ExecuteFirstAsync(token);
if (group == default || group.Members.RemoveAll(m => m.IsForeign && m.AvatarId == actorUri) == 0)
return;
group.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync<GroupEntity>(group, token);
}
async Task<DmGroup> FindOrCreateDmGroup(List<string> participantUris, string context, CancellationToken token)
{
var members = new List<GroupMember>();
foreach (var uri in participantUris.Distinct(StringComparer.Ordinal))
{
var local = await _localActors.FindByUri(uri, token);
members.Add(local != default && local.Kind == LocalActorKind.Person
? new GroupMember { AvatarId = local.Id }
: new GroupMember { AvatarId = uri, IsForeign = true });
}
if (!string.IsNullOrEmpty(context))
{
var byContext = await _dbEntities.DmGroups.Match(g => g.ConversationURI == context).ExecuteFirstAsync(token);
if (byContext != default)
return byContext;
}
var keys = members.Select(m => m.AvatarId).OrderBy(k => k, StringComparer.Ordinal).ToList();
var candidates = await _dbEntities.DmGroups
.Match(g => !g.DeletionAt.HasValue && g.Members.Count == keys.Count)
.ExecuteAsync(token);
var match = candidates.FirstOrDefault(g => g.Members.Select(m => m.AvatarId).OrderBy(k => k, StringComparer.Ordinal).SequenceEqual(keys));
if (match != default)
return match;
var dmGroup = new DmGroup { Members = members, ConversationURI = context };
await DB.Default.SaveAsync(dmGroup, token);
return dmGroup;
}
async Task<string> LocalPostId(string uri, CancellationToken token)
{
if (string.IsNullOrEmpty(uri) || !uri.StartsWith(_localActors.BaseAddress + "/peasants/", StringComparison.OrdinalIgnoreCase))
return default;
var id = uri[(uri.LastIndexOf('/') + 1)..];
return await _dbEntities.Posts.MatchID(id).ExecuteAnyAsync(token) ? id : default;
}
static IEnumerable<string> Addresses(JsonNode node)
{
foreach (var field in new[] { "to", "cc", "bto", "bcc", "audience" })
{
switch (node[field])
{
case JsonArray array:
foreach (var item in array)
{
var id = Id(item);
if (id != default)
yield return id;
}
break;
case JsonNode single when Id(single) is { } id:
yield return id;
break;
}
}
}
public static string Id(JsonNode node) => node switch
{
JsonValue value when value.TryGetValue<string>(out var text) => text,
JsonObject obj => Value(obj, "id") ?? Value(obj, "href"),
JsonArray array => array.Select(Id).FirstOrDefault(i => i != default),
_ => default
};
public static string Value(JsonNode node, string property) =>
node is JsonObject obj && obj[property] is JsonValue value && value.TryGetValue<string>(out var text) ? text : default;
}
}
@@ -1,191 +0,0 @@
using MongoDB.Entities;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Net.Http.Headers;
using System.Text.Json;
namespace PrivaPub.Services.Federation
{
public interface IRemoteActorService
{
Task<JsonDocument> Fetch(string uri, LocalActor signAs, CancellationToken token);
Task<ForeignAvatar> GetActor(string actorUri, LocalActor signAs, bool refresh, CancellationToken token);
Task<ForeignAvatar> GetActorByKeyId(string keyId, LocalActor signAs, bool refresh, CancellationToken token);
Task<string> ResolveHandle(string handle, CancellationToken token);
}
public class RemoteActorService : IRemoteActorService
{
public const string HttpClientName = "ActivityPub";
public const string ActivityJson = "application/activity+json";
const string Accept = "application/activity+json, application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"";
static readonly TimeSpan CacheLifetime = TimeSpan.FromDays(1);
readonly IHttpClientFactory _httpClientFactory;
readonly DbEntities _dbEntities;
readonly ILogger<RemoteActorService> _logger;
public RemoteActorService(IHttpClientFactory httpClientFactory, DbEntities dbEntities, ILogger<RemoteActorService> logger)
{
_httpClientFactory = httpClientFactory;
_dbEntities = dbEntities;
_logger = logger;
}
public async Task<JsonDocument> Fetch(string uri, LocalActor signAs, CancellationToken token)
{
if (!Uri.TryCreate(uri, UriKind.Absolute, out var target) || !IsFetchable(target))
return default;
using var request = new HttpRequestMessage(HttpMethod.Get, target);
request.Headers.Accept.ParseAdd(Accept);
if (signAs != default)
HttpSignatures.Sign(request, signAs, body: null);
using var response = await _httpClientFactory.CreateClient(HttpClientName).SendAsync(request, token);
if (!response.IsSuccessStatusCode)
{
_logger.LogInformation("GET {Uri} answered {Status}", uri, (int)response.StatusCode);
return default;
}
await using var stream = await response.Content.ReadAsStreamAsync(token);
return await JsonDocument.ParseAsync(stream, cancellationToken: token);
}
public async Task<ForeignAvatar> GetActor(string actorUri, LocalActor signAs, bool refresh, CancellationToken token)
{
if (string.IsNullOrEmpty(actorUri))
return default;
actorUri = StripFragment(actorUri);
var cached = await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == actorUri).ExecuteFirstAsync(token);
if (cached != default && !refresh && DateTime.UtcNow - cached.UpdatedAt < CacheLifetime)
return cached;
using var document = await Fetch(actorUri, signAs, token);
if (document == default)
return cached;
return await Upsert(document.RootElement, cached, token);
}
public async Task<ForeignAvatar> GetActorByKeyId(string keyId, LocalActor signAs, bool refresh, CancellationToken token)
{
if (string.IsNullOrEmpty(keyId))
return default;
if (!refresh)
{
var cached = await _dbEntities.ForeignAvatars.Match(a => a.PublicKeyId == keyId).ExecuteFirstAsync(token);
if (cached != default && !string.IsNullOrEmpty(cached.PublicKey))
return cached;
}
using var document = await Fetch(StripFragment(keyId), signAs, token);
if (document == default)
return default;
var root = document.RootElement;
if (root.TryGetProperty("publicKey", out _))
{
var actorUri = Text(root, "id");
var existing = await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == actorUri).ExecuteFirstAsync(token);
return await Upsert(root, existing, token);
}
var owner = Text(root, "owner");
return owner == default ? default : await GetActor(owner, signAs, refresh: true, token);
}
public async Task<string> ResolveHandle(string handle, CancellationToken token)
{
var parts = handle?.TrimStart('@').Split('@');
if (parts is not { Length: 2 } || string.IsNullOrEmpty(parts[0]) || string.IsNullOrEmpty(parts[1]))
return default;
var url = $"https://{parts[1]}/.well-known/webfinger?resource={Uri.EscapeDataString($"acct:{parts[0]}@{parts[1]}")}";
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsFetchable(target))
return default;
using var request = new HttpRequestMessage(HttpMethod.Get, target);
request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/jrd+json"));
request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
using var response = await _httpClientFactory.CreateClient(HttpClientName).SendAsync(request, token);
if (!response.IsSuccessStatusCode)
return default;
await using var stream = await response.Content.ReadAsStreamAsync(token);
using var document = await JsonDocument.ParseAsync(stream, cancellationToken: token);
if (!document.RootElement.TryGetProperty("links", out var links) || links.ValueKind != JsonValueKind.Array)
return default;
foreach (var link in links.EnumerateArray())
{
var type = Text(link, "type") ?? string.Empty;
if (Text(link, "rel") == "self" && (type.Contains("activity+json") || type.Contains("ld+json")))
return Text(link, "href");
}
return default;
}
async Task<ForeignAvatar> Upsert(JsonElement actor, ForeignAvatar existing, CancellationToken token)
{
var actorUri = Text(actor, "id");
if (string.IsNullOrEmpty(actorUri))
return existing;
var avatar = existing ?? new ForeignAvatar { ActorURI = actorUri, CreatedAt = DateTime.UtcNow };
avatar.ActorURI = actorUri;
avatar.UserName = Text(actor, "preferredUsername");
avatar.Name = Text(actor, "name");
avatar.Biography = Text(actor, "summary");
avatar.Url = Text(actor, "url") ?? actorUri;
avatar.Domain = new Uri(actorUri).Authority;
avatar.InboxURL = Text(actor, "inbox");
avatar.OutboxURL = Text(actor, "outbox");
avatar.IsDiscoverable = !actor.TryGetProperty("discoverable", out var discoverable) || discoverable.ValueKind != JsonValueKind.False;
avatar.AvatarType = Enum.TryParse<AvatarType>(Text(actor, "type"), out var type) ? type : AvatarType.Person;
if (actor.TryGetProperty("endpoints", out var endpoints) && endpoints.ValueKind == JsonValueKind.Object)
avatar.SharedInboxURL = Text(endpoints, "sharedInbox");
if (actor.TryGetProperty("publicKey", out var publicKey) && publicKey.ValueKind == JsonValueKind.Object)
{
avatar.PublicKeyId = Text(publicKey, "id");
avatar.PublicKey = Text(publicKey, "publicKeyPem");
}
if (actor.TryGetProperty("icon", out var icon) && icon.ValueKind == JsonValueKind.Object)
avatar.PictureURL = Text(icon, "url");
avatar.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(avatar, token);
return avatar;
}
public static bool IsFetchable(Uri target) =>
target.Scheme == Uri.UriSchemeHttps
&& target.HostNameType == UriHostNameType.Dns
&& !target.IsLoopback
&& !target.Host.Equals("localhost", StringComparison.OrdinalIgnoreCase)
&& target.Host.Contains('.');
public static string StripFragment(string uri)
{
var hash = uri.IndexOf('#');
return hash < 0 ? uri : uri[..hash];
}
public static string Text(JsonElement element, string property)
{
if (element.ValueKind != JsonValueKind.Object || !element.TryGetProperty(property, out var value))
return default;
return value.ValueKind switch
{
JsonValueKind.String => value.GetString(),
JsonValueKind.Object => Text(value, "id") ?? Text(value, "href"),
JsonValueKind.Array => value.EnumerateArray().Select(v => v.ValueKind == JsonValueKind.String ? v.GetString() : Text(v, "id")).FirstOrDefault(v => v != null),
_ => default
};
}
}
}
+15 -8
View File
@@ -7,10 +7,12 @@ using PrivaPub.ClientModels.Group;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Resources;
using PrivaPub.Services.Federation;
using PrivaPub.StaticServices;
using GroupEntity = PrivaPub.Models.Group.Group;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Federation.Outbox;
namespace PrivaPub.Services
{
@@ -67,7 +69,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(GetGroup)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -88,7 +90,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(GetGroups)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -113,6 +115,7 @@ namespace PrivaPub.Services
Domain = _localActors.BaseAddress,
PrivateKey = privateKey,
PublicKey = publicKey,
Kind = form.IsCommunity ? GroupKind.Community : GroupKind.Circle,
IsDiscoverable = form.IsDiscoverable,
ManuallyApprovesMembers = form.ManuallyApprovesMembers,
OwnerAvatarId = form.AvatarId,
@@ -120,6 +123,9 @@ namespace PrivaPub.Services
HashedInvitationPassword = string.IsNullOrEmpty(form.InvitationPassword) ? default : _passwordHasher.Hash(form.InvitationPassword),
Members = new() { new GroupMember { AvatarId = form.AvatarId, Role = GroupRole.Owner } }
};
group.ID = (string)group.GenerateNewID();
if (!await _localActors.TryReserveUserName(userName, LocalActorKind.Group, group.ID, token))
return result.Invalidate(_localizer["The username '{0}' is already take.", userName]);
var actor = _localActors.FromGroup(group);
group.Url = actor.Uri;
group.InboxURL = actor.Inbox;
@@ -132,7 +138,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(InsertGroup)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -168,7 +174,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(UpdateGroup)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -208,7 +214,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(JoinGroup)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -234,7 +240,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(LeaveGroup)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -276,7 +282,7 @@ namespace PrivaPub.Services
catch (Exception ex)
{
_logger.LogError(ex, $"{nameof(GroupUsersService)}.{nameof(ApproveMember)}");
return result.Invalidate(_localizer["Error: {0}", ex.Message], exception: ex);
return result.Invalidate(_localizer["Something went wrong."], exception: ex);
}
}
@@ -324,6 +330,7 @@ namespace PrivaPub.Services
Description = group.Description,
Url = actor.Uri,
Handle = actor.Handle,
IsCommunity = group.Kind == GroupKind.Community,
IsDiscoverable = group.IsDiscoverable,
ManuallyApprovesMembers = group.ManuallyApprovesMembers,
IsOwner = group.OwnerAvatarId == avatarId,
Loaded 100 of 111 files, more files were not shown because too many files have changed in this diff. Show more