Compare commits

..
57 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 90a7e38ce9 M6: provenance fixes, stored features, community edits that are real edits
Build / Build (push) Successful in 2m34s
Deploy / privapub.thepra.dev (push) Successful in 2m53s
- A fetched ObjectRecord no longer wears the signature, key, signed headers and @context
  of the activity that caused the fetch, and its ReceivedAt is the fetch's own time. Its
  activity fields now name the trigger. Provenance answers signature null and
  fetchedBy "instance-actor". Migration _008 clears the old fetched records.
- ObjectRecords store their ObjectFeatures extensions and context namespaces (migration
  _009 fills older records in batches), so statistics can count them. Provenance reads the
  stored lists.
- ForeignAvatar.Features records what an actor document uses (ActorFeatures): featured,
  shared inbox, FEP-521a, FEP-8b32, identity proofs, Misskey fields, indexable,
  undiscoverable, locked, key size, and more.
- RemoteEdits.Apply and RemoteDeletes.Remove are shared by the Update, Delete and Announce
  handlers. A community-wrapped Update is now an edit only when newer, refreshes polls
  and media otherwise, revises the ObjectRecord and re-resolves quotes. A
  community-wrapped Delete now tombstones the object, removes its ObjectRecord, reblogs
  and timeline rows, and lowers the reply count. Any deleted quoting post lowers the
  quoted post's quote count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:15:21 +02:00
thepraandClaude Opus 5.5 d37999970c M5: outbound requests, previews, the media proxy and served traffic
- HttpScope (AsyncLocal) tags each outbound request with a purpose and a trigger:
  - purpose is set by the caller: actor, key, object, webfinger, context, nodeinfo;
  - trigger is set by the job kind, by "verify" during inbox verification, or defaults
    to "request".
- FederationHttp records every JSON, media and stream fetch: status, time, bytes, hops,
  and an outcome of ok, refused or failed, with a reason: disallowed, remembered,
  bad-redirect, too-many-redirects, content-type, too-large, bad-json, private-address,
  timeout, network, or the status. A fetch a reader caused (trigger "request") is only
  counted per server per day.
- Link previews record a 'preview' event: card, no-card or failed.
- The media proxy counts cache hits.
- TrafficMeter counts the client API per endpoint group, method and status class. It
  counts our served documents (actor, outbox, collection, object, activity, licence,
  webfinger, nodeinfo) by kind, status and whether signed, per day and never per server,
  and never names a circle's collections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:10:42 +02:00
thepraandClaude Opus 5.5 e247001bdb M4: every delivery attempt is recorded
DeliveryJobHandler records each attempt as an 'out' event: the activity, object type and
audience read from the body (ActivityShape, shared with the inbox side), the receiving
server, the status, the time the POST took, the wait since it was queued, the bytes, the
attempt number and the signer's kind (none for circles or private traffic). The outcome is
ok, deferred (429 or 503 with Retry-After, or an open breaker: host-unavailable), retry
(5xx, 408, timeout, network), or dead: another 4xx, private-address, not-deliverable,
signer-gone, or a retry at the last attempt. Until now none of this outlived the job's
seven-day TTL, and the last error was overwritten on success.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:06:24 +02:00
thepraandClaude Opus 5.5 ac7cbd136b M3: what each handler did with an activity
Arrival carries a verdict that handlers set with one line before an existing return
(Arrival.Drop, Reject, Accept, About), so no handler signature changes. InboxProcessor
records it as an 'in' event, with the time taken, the wait since the inbox accepted it, the
attempt, the audience (from the post's visibility, or else the activity's addressing), the
local actor kind, the object's age for updates, deletes and reactions, and the FEP features
of a delivered object. It also records types nothing handles (unknown-type), actors that
cannot be loaded (deferred) and handler failures.

Drop reasons: fetch-failed, unparseable, misattributed, duplicate, deleted, not-addressed,
not-followed, not-public, not-visible, not-deleted, unknown-object, unknown-recipient,
cross-origin, unsupported. Accepted sub-reasons: stored, poll-vote, edit, refresh,
actor-refresh, actor-delete, removed, tombstone-only, auto-accepted, pending,
follow-answer, quote-answer, quote-granted, undone, reaction, reported. Rejected: blocked,
ignored, quote-refused.

A circle's traffic is private and kindless, and a stranger posting into one is just
not-addressed, so the event store cannot reveal a circle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:04:42 +02:00
thepraandClaude Opus 5.5 4fa53f63bd M2: every inbox answer is recorded
InboxReceiver records each answer once, in a finally, with a reason: too-large, not-json,
not-activity, missing-type-or-actor, no-signature, the signature check's own codes
(headers-unsigned, digest-mismatch, header-unreadable, date-skew, expired,
algorithm-unsupported), signature-invalid, actor-not-key-owner, key-unavailable,
id-cross-origin, undo-foreign, misattributed, unknown-recipient, and for 202s queued,
duplicate, suspended or self-delete-unknown-key. Each event carries the activity and object
type, the inbox, the signature scheme, the bytes and the time taken. A 404 for an unknown
/mouth and a rate-limited inbox (429, from OnRejected) are recorded too.

Until the signature verifies, the host is only claimed, so it is kept only if the server is
already known. A suspended server is recorded under its own name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:59:55 +02:00
thepraandClaude Opus 5.5 15cd034b29 M1: the interaction ledger
InteractionEvent records one interaction with a remote server: its channel (recv, in, out,
http, preview, crawl), activity and object type, outcome and reason, status, latency, wait,
bytes, attempt, audience, local actor kind, inbox, signature, features and the object's age.
IInteractionLedger.Record never blocks and never throws: events go into a bounded channel
of 10k, a full channel drops and counts, and a hosted service writes batches of up to 1000
every two seconds.

Privacy, as decided by the owner:
- no persona, root, group or activity id, inbox URL, actor URI or sender IP is stored;
- distinct accounts are counted with an HMAC keyed by a per-day salt (InteractionSalt,
  upserted so restarts agree, never created for a past day);
- the local actor kind survives only on public and unlisted traffic;
- a host claimed by an unverified sender is kept only if it is already known.

Traffic caused by reading is only counted per day (InstanceDay.Reads, ServerDay). Indexes:
a 90-day TTL on events, unique day rows, and a TTL safety net on salts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:56:13 +02:00
thepraandClaude Opus 5.5 125a49a1a0 T4: nothing answers 500
Build / Build (push) Successful in 2m24s
Deploy / privapub.thepra.dev (push) Successful in 2m39s
NeverFiveHundredTests walks every endpoint the server maps and fills each route parameter
with junk ('x', zeroed and random ObjectIds, a dot-dot segment, 5000 characters). It calls
each one anonymously, as a persona, and with a junk token, and asserts that nothing
answers 5xx or throws, and that every /api error carries a Mastodon error body.

It found two bugs:
- A junk 'Authorization: Bearer' on any non-/api route, anonymous ones included
  (/build.json, /peasants/*, inboxes), broke the response. JwtEvents.AuthenticationFailed
  wrote a body during authentication and the endpoint then wrote its own. The 401 body now
  comes from the Challenge event, which runs only when an endpoint needs a user.
- /api answered 401, 404 and 429 with no body. UseMastodonErrorBodies gives any /api error
  that leaves without a body Mastodon's {"error": ...}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:46:20 +02:00
thepraandClaude Opus 5.5 5e34517e73 T3: the whole server under test
PrivaPubHost is a WebApplicationFactory<Program> on the fixture's database, configured only
through UseSetting (visible before Build, unlike ConfigureAppConfiguration). It drops the
background workers so tests run the jobs they queue (Jobs.Run, RunInbox), gives each client
its own address for the rate limiter, and has a SecureMode variant. Accounts signs up roots,
adds personas and gets Mastodon tokens through the real /oauth code flow. RemoteActor signs
HttpRequestMessages for the real /peasants routes; Peer records bodies and headers and
serves files with ranges and text pages.

Program registers the Guid serializer with TryRegisterSerializer, so a second host in one
process starts; the fixture runs the migrations in production's order before any test.

HostBootTests: the host shares the fixture database; the harness handles exactly the
activities the server registers; every job kind has one handler; every controller and page
model can be made; the service graph validates with ValidateOnBuild and ValidateScopes;
Swagger is 404 outside Development; a persona's token never names its root; a signed DM
through the real /mouth route is queued, processed and stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:43:03 +02:00
thepraandClaude Opus 5.5 85fdff606d T2: CI runs every test against a throwaway mongod
tools/ci/with-test-mongod.sh starts mongod (or podman's mongo:8) on a random localhost port
with a temporary data directory, runs the command, and removes both. build.yml and deploy.yml
test through it, so the ~85 integration tests stop being skipped in CI. MongoFixture refuses
production's port and data directory, and in CI anything but the wrapper's mongod; with
PRIVAPUB_TEST_REQUIRE_MONGOD=1 a missing mongod fails instead of skipping.

The deploy now passes the PRIVAPUB_SMOKE_TOKEN secret to the Mastodon smoke check, so its
signed-in half runs once the owner creates the persona and the secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:37:42 +02:00
thepraandClaude Opus 5.5 00b2685cf4 T1: tests stop sharing state they don't own
- JobQueue takes an optional scope, so a test's worker leases and reaps only its own jobs.
- The dead-host delivery test runs alone (Exclusive), on its own jobs, and cleans up the
  breaker rows it trips; the breaker has tests of its own on unique hosts.
- Index and migration tests run alone: they drop indexes and rewrite every post.
- DomainBlocks.Load replaces reflection and a database-wide block in tests.
- Harness.Outgoing sees only deliveries queued since the harness started: Peer ports are
  reused within a run, which made the circle test flaky.
- Two pure-logic tests leave Mongo-gated classes, so CI runs them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:36:15 +02:00
thepraandClaude Opus 5.5 7c6fe80f1b docs: owner decisions on statistics, and the sweep and statistics plan
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:30:46 +02:00
thepraandClaude Opus 5.5 e6729c77e7 P6 done: personas can be quoted, under the owner's default of anyone, automatically
Build / Build (push) Successful in 1m7s
Deploy / privapub.thepra.dev (push) Successful in 1m15s
- Our public and unlisted posts state interactionPolicy.canQuote. The policy comes from the post, then the persona
  (`source[quote_policy]`: public, followers or nobody; default public as the owner chose), and is always nobody for
  followers-only posts and DMs.
- QuoteRequests are answered with Accept{result} naming a parrot-licence at /peasants/{name}/parrot-licences/{id}
  (the route name the owner chose), or with Reject. Followers-only checks that the requester really follows.
- A licence is a QuoteAuthorization naming both posts; revoking it (POST /api/v1/statuses/:id/quotes/:quoting_id/revoke)
  marks it 410, sends Delete{licence} to the quoter and the persona's followers, and revokes our own copy of the quote.
- A quote that arrives with one of our licences is accepted only if that licence is ours, unrevoked and names exactly
  that quoting post. One persona quoting another gets a licence too.
- Mastodon API: quote_approval for our posts (automatic, followers, current_user), `quote_approval_policy` when posting,
  PUT /api/v1/statuses/:id/interaction_policy, `source.quote_policy`.

Checked live: GoToSocial still accepts our posts with the policy stated, and leaves likes, replies and boosts open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 19:53:13 +02:00
thepraandClaude Opus 5.5 f9658a8e7a P6: remote video and audio play through the proxy
Build / Build (push) Successful in 37s
Deploy / privapub.thepra.dev (push) Successful in 57s
- The media proxy streams ranged requests from the origin (passing the range on, never caching), downloads and caches
  whole files otherwise, and serves cached files with range support. A PeerTube video is never fetched whole for one
  viewer, and clients still never contact the remote host.
- PeerTube's fragmented MP4 files inside an HLS entry are read as variants, so HLS-only instances play too.
- A remote Video or Audio post becomes one playable Mastodon attachment: the best MP4 up to 720p that carries both
  sound and picture, with its poster and duration; the card is kept only when nothing is playable.
- nginx: /media/proxy/ with proxy_buffering off and a 600 s read timeout (applied on the box, with a backup).

Checked live: a GoToSocial image through the proxy answers 206 with exactly the asked range when streamed, 200 when
cached, and 206 with the right Content-Range from the cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:59:14 +02:00
thepraandClaude Opus 5.5 6f1ab0073c P6: quote posts, received and sent (FEP-044f and the older keys)
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 54s
- Received quotes: read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 Link tag; the quoted post
  is fetched once; a quoteAuthorization stamp is verified field by field on the quoted author's origin; a consent
  quote without a stamp is pending; an older-key quote of a public post is shown; Delete of a stamp revokes. Counts
  and a `quote` notification follow the accepted state. The `quote-inline` fallback survives sanitising and is removed
  from content when the real quote is shown.
- Personas quote through `quoted_status_id`: posts that state a quote policy get a QuoteRequest and stay pending until
  an Accept brings a stamp we can verify, then an Update adds quoteAuthorization; posts that state none are quoted
  the older way, without `quote`; another persona's posts cannot be quoted yet (we issue no stamps). Quoting posts
  are delivered to the quoted author too.
- Mastodon API: Status.quote (with the quoted status one level deep), quotes_count, quote_approval from the remote
  policy, GET /api/v1/statuses/:id/quotes, `quote` notifications, and api_versions.mastodon = 7.

Checked live: GoToSocial's author-only quote policy is respected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:53:28 +02:00
thepraandClaude Opus 5.5 001fdac3be P6: link previews read by the server, as the owner decided
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 56s
- A public post that links to a page gets a preview card: from the post's own data first (FEP-8967 Link preview, the
  object's image, title and summary); otherwise the server reads the page once, 0-60 s after the post arrives, never
  when someone reads it. OpenGraph and Twitter tags give title, description and image; one LinkPreview per address
  is cached for 7 days and shared by the whole server, so a fetch never points at a persona.
- Lemmy link posts, which carry no title or description, get them filled in.
- The page fetch uses the guarded client (public addresses, three redirects, HTML only, first 512 KB).
- Federation:FetchLinkPreviews switches page fetching off.
- Local public posts get cards too.

Checked live: a link to a GoToSocial profile page becomes a card with its title, description and proxied image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:43:39 +02:00
thepraandClaude Opus 5.5 fdcf5176bc P6: emoji reactions in and out
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 57s
- Incoming reactions in all three shapes: Misskey and Sharkey Likes carrying an emoji (`content` or
  `_misskey_reaction`), Pleroma, Akkoma and Iceshrimp.NET EmojiReacts, and their Undo. Unicode reactions are one
  grapheme; custom ones need their Emoji tag and keep its image; `:name@host:` is read as `name`. A Like whose
  content is a heart stays a favourite.
- Reactions are kept per account and emoji on our posts and on remote ones we hold, and the author of a local post
  gets a `pleroma:emoji_reaction` notification with the emoji.
- Personas react through Pleroma's API (PUT/DELETE /api/v1/pleroma/statuses/:id/reactions/:emoji, GET for the list),
  which sends an EmojiReact (or its Undo) to the post's author.
- Statuses carry `emoji_reactions` (read by Phanpy) and `pleroma.emoji_reactions`, with counts and whether the viewer
  reacted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:39:18 +02:00
thepraandClaude Opus 5.5 7be6749a23 P6: custom emoji, fuller remote profiles, and polls both ways
Build / Build (push) Successful in 39s
Deploy / privapub.thepra.dev (push) Successful in 57s
- Custom emoji (Emoji tags) on posts, display names, bios and profile fields, at most 64 per object, proxied, in
  Status.emojis and Account.emojis.
- Remote profiles keep their header, profile fields, locked flag, published date, movedTo, indexable, memorial and
  image descriptions (a locked GoToSocial account no longer shows as open).
- Polls: incoming Questions (Mastodon, Misskey, Pleroma, GoToSocial shapes) with counts, voters, end and closed;
  our own polls from the Mastodon API go out as Questions; votes in are counted once per voter and never become
  replies; personas vote on other servers' polls with one Note per choice; counts refresh with an Update at most every
  three minutes; a poll closes on time and tells its voters and its author. GET /api/v1/polls/:id and POST
  /api/v1/polls/:id/votes.
- An Update without a newer `updated` only refreshes poll, video, audio and event details and leaves no revision.

Checked live against GoToSocial: each side's poll reaches the other as a poll and each side's vote is counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:35:20 +02:00
thepraandClaude Opus 5.5 81de470f64 P5 done: a key we cannot fetch for now gets 503, and follow/like/block ids stay private on purpose
Build / Build (push) Successful in 59s
Deploy / privapub.thepra.dev (push) Successful in 1m11s
- When a sender's key cannot be fetched because its server timed out or answered 5xx, the inbox answers 503 with
  Retry-After: 300 instead of 401, so Mastodon 4.7 retries rather than switching to RFC 9421 signatures we do not
  verify yet. The fetcher's failure cache now remembers whether a failure was temporary.
- Follow, Like, Block, Accept, Reject and Undo ids are deliberately not dereferenceable: serving them would publish
  who follows, likes and blocks whom. They are always sent with their object embedded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:25:18 +02:00
thepraandClaude Opus 5.5 b691c6766d P5: downvotes, private messages from Lemmy, late Creates of deleted posts, and typed details for clients
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 55s
- Dislike and its Undo are kept as downvotes (Lemmy, PieFed, Mbin, Friendica) and shown with favourites as votes.
- ChatMessage (Lemmy 0.19, Mbin, PieFed to those two) arrives as a direct message.
- A deleted object's id is remembered for 90 days, so a Create that arrives after its Delete cannot bring the post
  back; the post's ObjectRecord goes with it.
- A Join of one of our objects is answered with Ignore, as FEP-8a8e asks of a server without RSVP.
- A 503 with Retry-After is waited out like a 429 instead of counting as a failure of the host (GoToSocial throttling,
  Mastodon's temporary key failures).
- Status.privapub carries what a Mastodon Status cannot: object type, title, excerpt, cover, the author's source,
  link, video, audio and event details, and up/down votes, with every media URL proxied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 17:58:22 +02:00
thepraandClaude Opus 5.5 fc111d8c46 P5: remote objects are read in every shape, and videos, events, audio and links keep their own details
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 52s
- ObjectShapes reads `url`, `icon` and `image` as a value, a Link or an array; Markdown `content` (PeerTube) is
  rendered; a missing `mediaType` is inferred from the extension or the attachment type (Bridgy); alt text comes from
  `name`, `summary` or their language maps; thumbnails come from attachment `icon`, object `icon[]`, `image` or a Loops
  `preview`; durations are ISO 8601 or seconds; per-attachment `sensitive` is kept; the language falls back to
  `@context` `@language` (Pleroma); titles and excerpts are plain text; hashtags normalise with NFKC like Mastodon.
- Typed details on Post: Link (Lemmy link posts, Mbin `source` URLs, Mastodon 4.7 Link attachments with an FEP-8967
  publisher preview), Video (PeerTube files with their streams, HLS playlist, poster, captions, chapters, licence,
  channel, live state, comment policy), Audio (Funkwhale), Event (Mobilizon, Gancio, Hubzilla: times, zone, floating
  time, places and addresses, online link, capacity, status), the cover image, and the author's own source text.
- Mastodon API: a card built from those details without fetching anything, an event's "when · where" line, attachment
  thumbnails and durations, and no media file offered as a preview image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 17:53:59 +02:00
thepraandClaude Opus 5.5 dcd024100a Every remote object keeps its raw form and how it reached us, for the client's details view
Build / Build (push) Successful in 36s
- ObjectRecord, one per stored remote object: the raw JSON (up to 256 KB, always hashed), delivered or fetched,
  refetched from origin or not, the activity that brought it (or caused the fetch), shared or personal inbox, the
  signature's key, algorithm and signed headers, received time, published and updated, the delivering activity's
  @context, and up to ten later revisions from Update.
- Delivery details travel from InboxReceiver through the inbox job to the handlers as Arrival.Current.
- A host is described from its NodeInfo when we first hear from it, at most weekly (DescribeInstance job), never when
  someone opens the details view.
- GET /api/privapub/v1/statuses/:id/provenance and /api/privapub/v1/instances/:host, with the extensions an object
  used detected from its raw form (044f quotes, interaction policies, contexts, proofs, Misskey fields, MFM, FEP-8967
  links, emoji, polls, language maps, url variants, Markdown content).

Checked live: a GoToSocial reply shows as delivered to the shared inbox, signed hs2019 with GoToSocial's fragment-less
key id, with its interaction policy detected, and gts.test is described as gotosocial 0.22.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 17:49:22 +02:00
thepraandClaude Opus 5.5 bb9bd71391 P5, first batch: summaries stop hiding articles, personas get reachable names, blocks federate
Build / Build (push) Successful in 50s
Deploy / privapub.thepra.dev (push) Successful in 1m4s
- A remote `summary` is a content warning only on a Note or Question, or when `sensitive` is set. On Articles,
  Events, Videos, Pages and Audio it is an excerpt (WordPress teasers, Mobilizon dates and places, Mbin titles) and
  is now kept as `Post.Excerpt` instead of hiding the post. `Post.ObjectType` records the remote type, and the
  Mastodon API shows a remote non-Note object's title above its body again.
- Persona usernames must match `^[a-z0-9_]+$`, as groups already did; a name outside it was unreachable from
  Mastodon and Misskey.
- `postingRestrictedToMods` is defined in our JSON-LD context (Iceshrimp.NET drops undefined terms).
- `Vary: Accept` on actor and object URLs.
- Owner decision: blocks federate. A block sends `Block`, an unblock `Undo{Block}`; checked live against GoToSocial.
- Owner decision: a persona's and a group's `published`, and the day in new ids, is a random day up to two weeks
  before creation, so personas made the same day no longer share a date. Migration _007 gives existing ones theirs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 17:39:58 +02:00
thepraandClaude Opus 5.5 4c94254502 The owner's six decisions on what PrivaPub reveals: previews, blocks, authorship, views, bridging, reactions
Build / Build (push) Successful in 57s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 17:32:59 +02:00
thepraandClaude Opus 5.5 829eae3ccf Interop research: every major platform's wire shapes, gotchas, and what PrivaPub still drops
Build / Build (push) Successful in 1m22s
docs/INTEROP.md collects five research passes from 2026-10-01: Mastodon 4.7 and GoToSocial 0.22, the Misskey and
Pleroma families, the threadiverse (Lemmy 0.19/1.0, PieFed, Mbin, NodeBB), media and long-form (PeerTube, Loops,
Pixelfed, WordPress, Ghost, events, audio, books, Threads, Flipboard, Bridgy Fed), and cross-cutting FEPs and
signatures. Each claim was checked against source code or live fetches, with dates and versions.

It is checked against our own code: what is already right, three cheap things that are wrong today (summary read as a
CW on every type, unchecked usernames, an undefined context term), what the rich client needs kept (a post kind with
typed payloads, raw capture and provenance for the details view), and the six privacy choices the owner has to make.

The roadmap's P5 becomes P5 to P8, ordered by that evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 14:01:57 +02:00
thepraandClaude Opus 5.5 8f4d6cbdf9 A private fediverse on the workstation: PrivaPub against a real GoToSocial
Build / Build (push) Successful in 57s
Deploy / privapub.thepra.dev (push) Successful in 1m12s
tools/pasture/run.sh starts PrivaPub, GoToSocial and Mongo on one podman network behind Caddy's internal CA, and
interop.sh drives both through their own client APIs: follows (one to a locked account), posts, CW, replies, likes,
boosts, DMs, edits, deletes and unfollow. All 25 checks pass, three fresh runs in a row.

- Federation:AcceptAnyCertificate joins the two test-network switches; startup refuses all three in Production.
- WebFinger falls back to http only when AllowPlainHttp is on.
- A bootstrap logger, so a failure before the host is built is no longer silent.
- P4 is ticked in the roadmap, with what has not been run live (Lemmy, a Mastodon circle member).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 13:19:16 +02:00
thepraandClaude Opus 5.5 31542a181e Remote posts take their timeline place on arrival, and an unedited one has no edit date
Found by the first live run against GoToSocial:

- A remote post's id came from its published second plus random bytes, so a reply arriving in the same second as
  the post it answers could sort under it, and a late arrival landed behind a client's since_id. A post published
  within the last hour now gets an id for its arrival; backfill keeps its published time.
- NoteParser.Time returned default(DateTime) for a missing "updated", so every remote post was stored as edited in
  year one. Migration _006 clears the stored ones.
- published now carries milliseconds, so peers that derive ids from it (GoToSocial, Mastodon) keep our posts in
  order within a second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 13:19:16 +02:00
thepraandClaude Opus 5.5 51f2c10c6f Community actors name their moderators the way Lemmy reads them
Build / Build (push) Successful in 41s
Deploy / privapub.thepra.dev (push) Successful in 1m1s
attributedTo points at /wardens and postingRestrictedToMods reflects the
posting policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:31:13 +02:00
thepraandClaude Opus 5.5 a5d9a89445 Communities follow FEP-1b12, circles federate to their members only
Communities:
- a post addressed to a community (to, cc or audience) is accepted
  according to its posting policy - followers, anyone, or moderators -
  and GroupDistributor announces the whole activity with `audience` to
  the community's followers, plus the object for new posts so Mastodon
  shows them; updates and deletes of community content are announced too;
- top-level posts are Pages with a name (the title, or a headline from
  the text); /flock counts members, /wardens lists moderators;
- a Mastodon client posts into a community by mentioning it, or into a
  remote group, which sets `audience`;
- an Announce of an activity from a remote group a persona follows (Lemmy)
  is followed through: the object is fetched from its own origin, kept with
  its AudienceURI, and fanned out to the group's local followers; updates
  are applied in place and deletes checked against the origin.

Circles stop being local-only: an undiscoverable Group actor whose follows
are all requests the owner approves; posts addressed to the circle and its
/flock and delivered to members' own inboxes, never announced, never
public; a remote member's post into the circle is accepted from members
only. SignedFetchAuthorizer serves circle posts and collections only to a
signed request from a member or a member server's instance actor - 404 for
anyone else. Circles never surface in search, lookups, mentions, account
ids or profile pages.

Federation:SecureMode requires a valid signature on every GET under
/peasants except the instance actor. Group forms take a posting policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:30:57 +02:00
thepraandClaude Opus 5.5 0ccbcd558f P3 is deployed
Build / Build (push) Successful in 32s
v1.5.0 is live; the media directory, libvips and the proxy's signature
check are in place on the box. No upload has been made in production yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:24:30 +02:00
thepraandClaude Opus 5.5 c00865d805 Located posts: local only, rounded, matched against their own radius
A post given a latitude and longitude becomes LocalGeo: its position is
rounded to two decimals (about a kilometre) and stored as a 2dsphere
point, its radius clamped to 1-50 km, and it is local only - no Create,
no delivery, no outbox, never in the Mastodon API or on a profile page.
/clientapi/post/nearby takes the viewer's position for the query only
(it is neither stored nor logged), runs $geoNear and keeps posts within
each post's own radius, minus authors the viewer blocks or mutes. A
located post cannot be direct or in a group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:24:30 +02:00
thepraandClaude Opus 5.5 28b581b6b1 FEDERATION.md and CLAUDE.md describe media, blocks, pins and reports
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 52s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:22:08 +02:00
thepraandClaude Opus 5.5 4a713f3fb6 Media: uploads stripped of metadata, attachments both ways, a remote media proxy
- /api/v1/media and /api/v2/media (and GET/PUT /api/v1/media/:id):
  images go through libvips (NetVips, its native build bundled):
  autorotated, every kind of metadata dropped (EXIF, GPS, XMP, IPTC,
  comments), capped at 4096 px, with a 640 px preview and a blurhash
  (own encoder, the reference algorithm); animated GIFs are re-encoded;
  video and audio are remuxed by ffmpeg with -map_metadata -1, never
  re-encoded, and a video gets a still preview. Files get random names
  under /var/lib/privapub/media, outside the web root deploys replace, and
  are served at /media/files with nosniff and a sandbox CSP.
- media_ids on create and edit (four at most, the persona's own, each used
  once); notes carry them as Document attachments with alt text, blurhash,
  focalPoint and size; inbound attachments were already kept.
- avatar and header uploads in update_credentials, cropped to 400x400 and
  1500x500, federated with Update{Person}.
- Remote media reaches clients only through /media/proxy/{hmac}/{url},
  fetched by the guarded client (no SVG, 40 MB cap) and cached outside the
  served root, trimmed to 5 GB; foreign avatars and headers use it too, so
  a client never contacts another server.
- MediaJanitor deletes uploads left unattached for a day.
- nginx accepts 100 MB bodies on the upload endpoints only (applied on Max).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:22:08 +02:00
thepraandClaude Opus 5.5 8f75317050 Blocks, mutes, bookmarks, pins and reports
- Blocks are per persona and never federate (a Block activity would tell
  the other server who blocked whom): the blocked account is removed as a
  follower, with a Reject{Follow} if it is remote, unfollowed, cleared from
  home and notifications, and refused with Reject if it follows again.
- Mutes (optionally timed, optionally sparing notifications) and per-
  persona domain blocks keep authors out of home timelines, notifications
  and every status list the API returns; the boosts of a hidden author's
  posts are hidden too.
- Bookmarks and pins (at most five, public or unlisted, own posts) with
  their Mastodon endpoints and flags; pinned posts are the actor's
  `featured` collection at /trophies, and `featuredTags` points at
  /tattoos.
- Reports: /api/v1/reports stores the report and, when forwarding to a
  remote account, sends Flag from the instance actor, so the reporting
  persona is never named to the other server. An inbound Flag about a local
  persona or its posts becomes a report; moderators list and resolve them
  under /clientapi/moderator/reports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:13:57 +02:00
thepraandClaude Opus 5.5 525b5368a1 P2 is deployed
Build / Build (push) Successful in 39s
v1.4.0 serves OAuth and the Mastodon client API; tools/smoke/mastodon-api.sh
passes against production. The signed-in side was verified locally with two
personas of one login; no real client has logged into production yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:12:28 +02:00
thepraandClaude Opus 5.5 d8f163b5ce Persona separation is tested, the API is smoke-checked on deploy, OAuth rows are pruned
Build / Build (push) Successful in 59s
Deploy / privapub.thepra.dev (push) Successful in 1m13s
- PersonaSeparationTests: two personas of one login follow the same
  account and post; nothing the API maps for one contains the other's id,
  username or the root id.
- tools/smoke/mastodon-api.sh checks what a client meets first (instance
  v1/v2, discovery, app registration, client credentials, an app token
  refused by a user endpoint, the public timeline, revocation) and, given
  a persona token, verify_credentials, home and notifications. deploy.yml
  runs it after every deploy.
- OAuthPruner removes invalid tokens and authorizations older than two
  weeks, every six hours.
- The consent page no longer sets form-action, which browsers apply to the
  redirect back to the client after the form is posted.

CLAUDE.md gains the Mastodon API layout and invariants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:09:19 +02:00
thepraandClaude Opus 5.5 70b4c45eb0 The dead-host test tolerates a loaded machine
It asserted fewer than ten requests to the dead host by the time the live
one arrived, which a busy workstation can exceed without the per-host
limit being broken; the live delivery arriving at all is the point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:07:44 +02:00
thepraandClaude Opus 5.5 0028e96e76 The Mastodon client API: accounts, statuses, timelines, notifications, search
With a token for a persona, a Mastodon client can now:
- accounts: verify/update credentials (display name, note, fields, locked,
  bot, discoverable, indexable, hide collections, posting defaults; the
  change federates as Update{Person}), get, lookup, statuses (paged, by
  visibility to the viewer), relationships, search, follow and unfollow,
  follow requests (authorize and reject answer remote followers with
  Accept or Reject), remove from followers. Followers and following lists
  are shown to their owner only.
- statuses: post (plain text, mentions, hashtags, replies, content
  warnings, the four visibilities, Idempotency-Key), get, edit (PUT),
  delete returning the source for redrafting, context, history, source,
  favourite, reblog and their undos, favourited_by and reblogged_by.
- timelines: home, public (local or remote), tag; favourites;
  conversations; markers.
- notifications: list with types and exclude_types, get, dismiss, clear,
  unread count.
- /api/v2/search, resolving a handle or a URL to an account or a post.

Media, polls, pins, bookmarks, mutes, blocks, lists, filters, trends and
push answer empty lists or a 422 saying they are not supported yet, so
clients degrade instead of failing. Public reads work without a token.
Persona settings (locked, bot, indexable, discoverable) now also shape
the ActivityPub actor.

Fixed on the way: three conditional expressions whose `default` was the
value type's, so a missing limit became 1, a missing flag became false and
an attachment without dimensions became 0x0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:05:58 +02:00
thepraandClaude Opus 5.5 b54cdf78b2 One StatusService behind both client APIs, with outbound likes and boosts
Domain/Statuses/StatusService takes a persona, not a root, and is what
/clientapi and the Mastodon API share:
- Publish renders Markdown (clientapi) or plain text (Mastodon clients),
  checks the persona may see the post it replies to, opens or reuses the
  conversation for a direct post from its recipients and mentions, fans
  out and hands the Create to the outbox;
- Edit keeps a revision and sends Update{Note}; Remove soft-deletes and
  returns the post, so a client can delete and redraft;
- Favourite and Reblog work on anything the persona can see and send Like,
  Announce and their Undo to the author (and, for boosts, to followers);
  boosts are refused for anything but public and unlisted posts.

PostsService is now the clientapi wrapper that checks the root owns the
persona. A persona's own boost is a local row: the outbox renders it as
Announce, /grunts/announce-{id} resolves, and object endpoints, profile
pages and NodeInfo skip it. ContentFormat gains Plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:55:22 +02:00
thepraandClaude Opus 5.5 4d9be37c1c Mastodon clients can sign in: OAuth with a persona per token
OpenIddict 7.7 (MongoDB stores, keys kept in Mongo so tokens survive
restarts) serves /oauth/authorize, /oauth/token, /oauth/revoke and the
discovery documents, including /.well-known/oauth-authorization-server:
- authorization code (PKCE optional) and client credentials, Mastodon's
  scopes including the granular ones, non-expiring reference tokens,
  `created_at` in the token response, and the urn:ietf:wg:oauth:2.0:oob
  page that shows the code;
- /oauth/login signs the private login into a fifteen-minute cookie that
  only /oauth sees (rate limited, antiforgery-protected); /oauth/authorize
  then asks which persona the application acts as. The token's subject
  is that persona's id and nothing else; no root id reaches a token, an
  authorization or a log line;
- the token exchange refuses a persona whose login is banned or deleted,
  and every API request checks the same.

/api/v1/apps registers applications dynamically, /api/v1/apps/
verify_credentials, /api/v1/instance (v1 and v2, "4.2.0 (compatible;
PrivaPub)") and verify_credentials answer in Mastodon's shapes: snake_case
with explicit nulls, Rails-style parameters from query, form or JSON,
{"error": ...} on failure, Link paging. CORS exposes Link.

/api goes to OpenIddict validation and everything else to the existing
JWT; the JWT failure handler no longer sends the exception and stack trace
to the client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:52:31 +02:00
thepraandClaude Opus 5.5 e6a6fe46d7 P1.2 is deployed and verified
Build / Build (push) Successful in 27s
v1.3.0 is live with follows, timelines, notifications, likes, boosts,
edits, tombstones and thread backfill; its indexes exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:42:16 +02:00
thepraandClaude Opus 5.5 7ff3a61e75 FEDERATION.md and CLAUDE.md describe follows, likes, boosts and timelines
Build / Build (push) Successful in 30s
Deploy / privapub.thepra.dev (push) Successful in 45s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:40:02 +02:00
thepraandClaude Opus 5.5 afd2e727d0 A remote delete clears what it touched
Deleting a remote post now removes its timeline entries and the reblogs
of it and lowers its parent's reply count; deleting a remote actor also
drops the follows pointing at it and its entries in home timelines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:39:45 +02:00
thepraandClaude Opus 5.5 b34c71775c Likes and boosts arrive, count, notify, and can be taken back
- Like: a Favourite per account and post (unique), the post's counter, a
  Favourite notification for a local author; only on posts the liker
  could see (public and unlisted, or followers-only and direct when they
  were addressed).
- Announce: the original is always refetched from its own origin (or is
  ours), and must be public or unlisted. A boost of a local post counts
  and notifies; a boost by an account a persona follows is stored as a
  reblog row and fanned out to its followers' home timelines (as a reblog,
  where reblogs are wanted). A boost by a stranger of a stranger's post is
  ignored. FEP-1b12 group announces of activities wait for P4.
- Undo of a Like or an Announce reverses each of them; an Undo naming only
  an id is matched against follows, likes and boosts in turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:39:10 +02:00
thepraandClaude Opus 5.5 e99b76dbd7 Home timelines, mention notifications, and threads that fetch their parents
Fanout writes a TimelineEntry for every persona a post should reach: the
author, local followers of a local author, local followers of a remote one,
the members of a direct conversation or a circle. Mastodon's home rules
apply when it is written: a reply shows only to followers of both sides
(or to the one replied to), a reblog only where reblogs are wanted. A
mention of a local persona becomes a Mention notification.

Inbound posts are now also kept when a persona follows their author.

RemotePosts holds what CreateHandler and backfill share: building a Post
from a note, and fetching a public parent the first time a reply to it
arrives, so its author is known (a reply to an unknown or non-public
parent stays out of home timelines). Each fetched ancestor queues a
FetchAncestors job for the next one, up to ten deep.

/clientapi/timeline/home and /clientapi/notifications (with
/notifications/read) page by max_id for the persona's own root only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:37:41 +02:00
thepraandClaude Opus 5.5 a76cc34557 Posts reach their audience, edits follow them, deletes leave a tombstone
OutboxPublisher works out who a post goes to, Mastodon's way: followers'
shared inboxes for public, unlisted and followers-only; every mentioned
remote account's own inbox; the recipients only for direct; the parent's
author for a reply; a community's followers for a post in it; nobody for
a circle. Creates, updates and deletes all use it.

- Local posts take a visibility (public, unlisted, followers-only) and a
  spoiler text next to the content-warning flag.
- /clientapi/post/update keeps the previous version as a revision,
  re-renders, and sends Update{Note} with `updated` to the same to/cc.
- Deleting is now soft: the content, title, spoiler, media and revisions
  are cleared, timeline entries removed, the parent's reply count goes
  down, Delete goes to the stored audience, and the object answers 410
  with a Tombstone instead of 404.
- Editing a persona's profile sends Update{Person} to its followers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:34:35 +02:00
thepraandClaude Opus 5.5 61f6ca0676 Personas can follow: locally at once, remotely by Follow and Accept
Following records what a local persona follows, local or remote, with
its state and the Follow activity's id. FollowService (behind
/clientapi/follow, /clientapi/unfollow and /clientapi/following):
- resolves @user, user@host or an actor URI;
- a local account is followed in-process: accepted unless it approves
  followers by hand, a Follower row on the other side, a community gains
  the persona as a member, and a Follow or FollowRequest notification;
- a remote account gets a Follow signed by the persona, at
  /grunts/follow-{id}, and stays Requested until an Accept;
- unfollowing deletes the rows, or sends Undo{Follow} to a remote account.

Inbound Accept and Reject are matched to the Follow by its id (or, for an
embedded Follow without one, by its actor), and only from the account that
was followed. A remote Follow now notifies the persona too.

Notification is the per-persona record P1.2 builds on, deduplicated by
type, persona, sender and post. TimelineEntry and Favourite are created
with their indexes for the next commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:32:33 +02:00
thepraandClaude Opus 5.5 8af50dad9c P1.1 is deployed and verified
Build / Build (push) Successful in 33s
v1.2.0 serves the themed routes, NodeInfo 2.0 and 2.1 and the day-cut
actor; migrations 1-5 ran and the job queue's indexes exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:31:26 +02:00
thepraandClaude Opus 5.5 60bf21b019 CLAUDE.md describes the code after P1.1
Build / Build (push) Successful in 1m7s
Deploy / privapub.thepra.dev (push) Successful in 1m1s
The route table lists the themed names now in use, the repo map gains
Jobs, Ids, Moderation, Domain and Web, and the invariants add the job
queue, VisibilityPolicy and the rule for which remote posts are stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:28:37 +02:00
thepraandClaude Opus 5.5 1c78da34a9 One visibility rule for every public read
VisibilityPolicy.IsPublic is the single expression for "anyone may see
this" (Public or Unlisted, not deleted); the outbox, the object and
activity endpoints, the HTML pages and NodeInfo all use it instead of
spelling it out. CanSee answers for a persona: the author, a mentioned
local persona, a conversation member for Direct, a circle member for
Circle; followers-only waits for P1.2's follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:26:51 +02:00
thepraandClaude Opus 5.5 5a13597c00 Persona and group ids say which day, nothing more
An ObjectId carries its creation second and a per-process counter, so two
avatars made one after the other by the same login got ids a few counts
apart: a link between personas that every Mastodon client would have
seen. Avatar and Group now generate ids from the UTC day plus eight random
bytes (still valid ObjectIds), and a remote post's id is made from its
published time with a random tail, so posts page in the order they were
written rather than the order they arrived.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:25:48 +02:00
thepraandClaude Opus 5.5 e6a362c0b8 Domain blocks: suspend, silence, reject media
DomainBlock (domain, severity, reject-media, public and private comment)
covers the domain and its subdomains. Admins manage them under
/clientapi/admin/domainblocks/{list,insert,delete}; the set is kept in
memory, reloaded on every change and at most five minutes stale.

A suspended domain is refused by FederationHttp.IsAllowed, so nothing is
fetched from it and no job delivers to it, and the inbox drops its
activities with a 202 before fetching any key. Reject-media strips the
attachments of posts from that domain. Silence is recorded for the
timelines and notifications that arrive in P1.2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:25:05 +02:00
thepraandClaude Opus 5.5 9ec1f9930b Profile and post pages, NodeInfo 2.1, and FEDERATION.md
/@user and /@user/{id} are Razor pages showing an avatar's or community's
public and unlisted posts: no scripts, a strict CSP, no-referrer, noindex,
and an alternate link to the ActivityPub document. A client asking them
for activity+json is redirected to the actor or note, and the actor and
note redirect browsers here.

NodeInfo answers 2.1 as well as 2.0 (repository, homepage, a link to
FEDERATION.md) and counts only public local posts.

FEDERATION.md (FEP-67ff) lists the protocols, FEPs, route names,
activities and the security rules a peer will notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:23:03 +02:00
thepraandClaude Opus 5.5 e54e17708f Outbound documents speak Mastodon's dialect under PrivaPub's route names
Local text (Domain/Content/ContentRenderer): Markdig with raw HTML off and
autolinks, or plain text for API clients; then @user and @user@host are
resolved (locally or through WebFinger) into Mastodon's h-card markup and
#tags into hashtag links, skipping code, links and e-mail addresses. A
post stores the result, its mentions and its tags, and is delivered to
the mentioned actors and the parent's author as well as to followers.

Renderer:
- Mastodon's @context (toot, schema PropertyValue, discoverable,
  indexable, blurhash, focalPoint, featured, alsoKnownAs, movedTo) and
  FEP-2c59's webfinger;
- an actor's url is its HTML page, its fields are PropertyValue
  attachments, published is cut to the day and indexable is false;
- a note's content is the stored rendering; a title becomes name and a
  bold first line, a content warning without its own text is summarised
  by the title or "Content warning"; Mention and Hashtag tags, contentMap,
  updated, and to/cc by visibility.

Routes take the agreed names: /groupies, /stalking, /scribbles/{id},
/grunts/{id} (a Create resolves), /whispers/{id} for a DM's context. The
outbox is a collection with a first page, paged by max_id. A browser
asking for an actor or a note is redirected to /@user, and WebFinger's
profile-page points there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:21:31 +02:00
thepraandClaude Opus 5.5 e9529310fe One Post model for every visibility, read from notes the way peers write them
Post gains what federation and the Mastodon API need: Visibility (Public,
Unlisted, FollowersOnly, Direct, Circle, LocalGeo), the author's account
id, to/cc, the Create's id, url, context, quote, InReplyToURI and the
parent's author, a separate SpoilerText next to the title, language,
mentions, hashtags, remote attachments (alt text, blurhash, focus, size),
reply/favourite/reblog counters, revisions, EditedAt and DeletedAt.

Direct messages are Posts with Visibility Direct and a ConversationId;
migration _005 copies DmPost rows across with their ids and fills the new
fields of existing posts. DmPost is left in place so a rollback still sees
the old messages.

Inbound:
- NoteParser reads Note, Article, Page, Question and media types: content,
  then contentMap, then _misskey_content; summary as the spoiler and name
  as the title; Mention and Hashtag tags; attachments; a PeerTube-style
  list attribution prefers the person over the channel; quote URIs.
- Addressing classifies like Mastodon, finding followers-only by the
  author's own followers URL (now stored on ForeignAvatar), not by a
  "/followers" suffix.
- Create keeps a post when a local persona is addressed or mentioned, when
  it replies to a local post (the parent's reply count goes up) or when a
  community it follows is addressed; an unsolicited public post is not
  stored. Update keeps the previous version as a revision.

The outbox and object endpoints serve only Public and Unlisted posts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:17:14 +02:00
thepraandClaude Opus 5.5 796f06acf4 The inbox answers once it has verified, and processes from the job queue
InboxService is split the way the roadmap lays out Federation/Inbox:
- InboxReceiver reads and verifies the request exactly as before, runs
  the checks that need no fetch (the activity id's origin, a Follow of a
  missing or local-only actor, an Undo of someone else's activity, an
  embedded object attributed to someone else), queues a ProcessInbox job
  and answers 202. The job's dedupe key is the activity id, so a peer that
  delivers the same activity twice is processed once.
- InboxProcessor (two at a time, eight attempts) loads the verified actor
  and hands the activity to the handler for its type.
- Handlers/{Follow,Undo,Create,Delete,Update}Handler are the old methods,
  unchanged except that they no longer produce status codes; the JSON
  helpers live in Objects/ActivityJson and the group membership helpers in
  Inbox/ForeignMembers.

A slow fetch of an object or a remote actor now delays the job, not the
sender's HTTP request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:12:34 +02:00
thepraandClaude Opus 5.5 d1a91c40c4 Deliveries run on a Mongo job queue with leases, backoff and per-host limits
The single serial DeliveryWorker is replaced by Infrastructure/Jobs:
- Job rows are leased with one FindOneAndUpdate (oldest RunAt first, a
  two-minute lease) and a reaper returns expired leases every 30 s;
- enqueueing wakes the workers, which otherwise poll every five seconds;
- delivery runs eight at a time with at most two per host, so a slow or
  dead server holds two slots, not the queue;
- a failure waits n^4 + 15 + jitter seconds (Mastodon's curve) for up to
  16 attempts; a 4xx other than 408/429 is final, a 429 honours
  Retry-After;
- RemoteInstance is a per-host circuit breaker: ten consecutive failures
  quarantine a host for an hour, doubling to a week, and its jobs wait
  without spending attempts;
- a delivery is queued once per activity and inbox (unique DedupeKey), and
  finished jobs expire after seven days (TTL on FinishedAt).

Migration _004 moves pending Delivery rows into jobs and marks them
abandoned, so a rollback to the old worker cannot send them twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:09:35 +02:00
thepraandClaude Opus 5.5 934b6fe687 P0 is deployed and verified
Build / Build (push) Successful in 25s
v1.1.0 serves the actor, NodeInfo and WebFinger, answers Swagger with 404
and inbox junk with 400 and 401; migrations 1-3 ran and the unique
indexes exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:07:23 +02:00
240 changed files with 21246 additions and 947 deletions

No files matched your search

+2 -2
View File
@@ -15,5 +15,5 @@ jobs:
- name: Build
run: dotnet build PrivaPub.sln -c Release
- name: Test
run: dotnet test PrivaPub.sln -c Release --no-build
- name: Test (unit and integration, on a throwaway mongod)
run: tools/ci/with-test-mongod.sh dotnet test PrivaPub.sln -c Release --no-build
+6 -2
View File
@@ -28,8 +28,8 @@ jobs:
echo "BUILD_REF=${GITHUB_REF_NAME:-master}" >> "$GITHUB_ENV"
echo "BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_ENV"
- name: Test
run: dotnet test PrivaPub.sln -c Release
- name: Test (unit and integration, on a throwaway mongod)
run: tools/ci/with-test-mongod.sh dotnet test PrivaPub.sln -c Release
- name: Publish
run: |
@@ -90,6 +90,8 @@ jobs:
fi
- name: Verify what is being served
env:
SMOKE_TOKEN: ${{ secrets.PRIVAPUB_SMOKE_TOKEN }}
run: |
served=$(curl -fsS "$PUBLIC_URL/build.json" | python3 -c "import json,sys; print(json.load(sys.stdin).get('commit',''))")
[ "$served" = "$BUILD_COMMIT" ] || { echo "::error::served build is '$served', expected '$BUILD_COMMIT'"; exit 1; }
@@ -104,4 +106,6 @@ jobs:
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/activity+json' \
--data '{"type":"Follow","actor":"https://example.org/users/x","object":"'"$PUBLIC_URL"'/peasants/privapub"}' "$PUBLIC_URL/human-centipede")
[ "$code" = "401" ] || { echo "::error::an unsigned inbox POST answered $code"; exit 1; }
tools/smoke/mastodon-api.sh "$PUBLIC_URL" "$SMOKE_TOKEN"
[ -n "$SMOKE_TOKEN" ] || echo "::warning::PRIVAPUB_SMOKE_TOKEN is not set, so the signed-in API was not checked"
echo "::notice::serving $served"
+4
View File
@@ -398,3 +398,7 @@ FodyWeavers.xsd
# JetBrains Rider
*.sln.iml
PrivaPub/media-store/
PrivaPub/media-store-proxy/
tools/pasture/.publish/
+220 -33
View File
@@ -1,7 +1,9 @@
# CLAUDE.md
Guidance for working in this repository. `docs/ROADMAP.md` holds the owner's decisions and the phased plan to full
ActivityPub interop; read it before changing anything federation-, privacy- or API-shaped.
ActivityPub interop; read it before changing anything federation-, privacy- or API-shaped. `docs/INTEROP.md` is the
per-platform evidence behind phases P5 to P8: what each peer sends, what it expects, and what PrivaPub still drops.
Check the platform's section there before writing a parser or a renderer, and add to it whatever you learn.
## What this is
@@ -37,26 +39,20 @@ actor-scoped route gets a name in the same spirit, agreed with the owner, and a
| Thing | Route |
|---|---|
| Actor | `/peasants/{name}` (`/users/{name}` 301-redirects here) |
| Actor | `/peasants/{name}` (`/users/{name}` 301-redirects here; browsers are sent to `/@{name}`) |
| Inbox | `/peasants/{name}/mouth` |
| Outbox | `/peasants/{name}/anus` |
| Outbox | `/peasants/{name}/anus` (`?page=true[&max_id=]` for pages) |
| Shared inbox | `/human-centipede` (also `/peasants/{name}/human-centipede`) |
| Followers | `/peasants/{name}/groupies` |
| Following | `/peasants/{name}/stalking` |
| Notes | `/peasants/{name}/scribbles/{id}` |
| Activities | `/peasants/{name}/grunts/{id}` (`create-{postId}` resolves) |
| DM context | `/peasants/{name}/whispers/{id}` |
| Quote permission (FEP-044f `QuoteAuthorization`) | `/peasants/{name}/parrot-licences/{id}` |
| Token refresh | `/clientapi/user/sniff/again` |
These are agreed for the roadmap and replace the conventional names still in the code today:
| Thing | Route | Replaces |
|---|---|---|
| Followers | `/groupies` | `/followers` |
| Following | `/stalking` | `/following` |
| Notes | `/scribbles/{id}` | `/posts/{id}` |
| Activities | `/grunts/{id}` | `/activities/{id}` |
| Replies, likes, shares | `/gossip`, `/drool`, `/echoes` | |
| Featured | `/trophies` | |
| Featured tags | `/tattoos` | |
| Group members | `/flock` | |
| Group moderators | `/wardens` | |
| DM context | `/whispers/{id}` | |
Agreed for later phases: `/gossip`, `/drool`, `/echoes` (replies, likes, shares), `/trophies` (featured), `/tattoos`
(featured tags), `/flock` and `/wardens` (group members and moderators).
Routes other software looks up by name stay conventional:
- `/.well-known/*` and `/nodeinfo/*`;
@@ -75,6 +71,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Group, Post (posts + DMs), Admin, Data
Infrastructure/
Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out
Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker
Ids/ PrivacyIds (day-only ids for personas and groups, arrival-ordered ids for remote posts)
Data/ Indexes (created at start), EntityMaps.Warm, Migrations/_NNN_*.cs
Cli/ AdminCommands (`PrivaPub admin promote|demote <root>`)
RateLimiting.cs accounts (per client address) and inbox (per sending origin) policies
@@ -83,13 +81,30 @@ PrivaPub/ ASP.NET Core Web API, net10.0
WellKnownController (webfinger, nodeinfo), UsersController (redirect)
Actors/ LocalActorService (LocalActor, Keys, ReservedName), RemoteActorService
(authoritative fetch, key verification, WebFinger), ActorDocument (parser)
Objects/ Origin (same-origin rules), ContentSanitizer (HtmlSanitizer, Mastodon allowlist)
Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer
Moderation/ DomainBlocks (suspend / silence / reject media)
Signing/ HttpSignatures (draft-cavage sign/verify)
Inbox/ InboxService (Follow/Undo/Create/Delete/Update)
Outbox/ DeliveryService + DeliveryWorker
Rendering/ ActivityPubRenderer (JsonObject builders)
Inbox/ InboxReceiver (verify, queue, 202) → InboxProcessor (job) → Handlers/{Follow,Accept,Reject,
Undo,Create,Update,Delete,Like,Announce}; RemotePosts (build, fetch parents, FetchAncestors)
Outbox/ OutboxPublisher (who a post goes to), DeliveryService (queues jobs) + DeliveryJobHandler
Rendering/ ActivityPubRenderer (Mastodon @context, actors, notes, collections)
Domain/
Content/ ContentRenderer (Markdown or plain text → HTML with h-card mentions and hashtags)
Social/ FollowService (local in-process, remote Follow/Accept), Notifications
Timelines/ Fanout (TimelineEntry rows, Mastodon's home rules), TimelineService
Privacy/ VisibilityPolicy (IsPublic expression, CanSee)
Relationships/ RelationshipService (blocks, mutes, account domain blocks; Hidden), ReportService
Media/ MediaService (libvips, ffmpeg remux, blurhash), MediaProxy, MediaJanitor
Domain/Statuses/ StatusService: publish, edit, remove, favourite, reblog, for a persona (both client APIs use it)
Api/Mastodon/
Auth/ OpenIddict setup (keys in Mongo), MastodonScopes, TokenController, OAuthPruner
Infrastructure/ MastodonController (avatar context, scopes, errors, Link), MastodonParams, MastodonJson, Page
Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch
Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, stubs
Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex);
OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent)
Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, …
Models/ Mongo entities: User/, Group/, Post/, Federation/, AppConfiguration
Models/ Mongo entities: User/, Group/, Post/, Federation/, Jobs/, AppConfiguration
StaticServices/ DbEntities (Find<T> accessors), AuthTokenManager (JWT), PasswordHasher
Data/InitDb.cs first-run seeding (languages)
PrivaPub.ClientModels/ DTOs + validation resources shared with clients
@@ -109,6 +124,7 @@ dotnet build PrivaPub.sln -c Release
dotnet test PrivaPub.sln # unit tests; integration tests skip
PRIVAPUB_TEST_MONGOD=1 dotnet test PrivaPub.sln # all of them, against mongod on 127.0.0.1:27017
# (PRIVAPUB_TEST_MONGO overrides; a fresh database per run, dropped after)
tools/ci/with-test-mongod.sh dotnet test PrivaPub.sln # all of them, on a throwaway mongod, as CI runs them
cd PrivaPub && ASPNETCORE_ENVIRONMENT=Development \
Kestrel__Endpoints__Http__Url=http://127.0.0.1:6970 Kestrel__Endpoints__Http__Protocols=Http1AndHttp2 \
AppConfiguration__BackendBaseAddress=http://127.0.0.1:6970 MongoSettings__Database=PrivaPubTest \
@@ -117,7 +133,8 @@ cd PrivaPub && ASPNETCORE_ENVIRONMENT=Development \
Development config (`appsettings.Development.json`) binds HTTPS 7195 with HTTP/2 only; the overrides above make it
curl-able. Outbound fetches only go to https DNS names resolving to public addresses; a test network (Pasture) sets
`Federation__AllowPrivateNetworks=true` and `Federation__AllowPlainHttp=true`, which startup refuses in Production.
`Federation__AllowPrivateNetworks=true`, `Federation__AllowPlainHttp=true` and `Federation__AcceptAnyCertificate=true`,
which startup refuses in Production.
Promoting an admin on the box (signing up as "admin" grants nothing):
@@ -154,11 +171,62 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
- the shared inbox is advertised in `endpoints.sharedInbox`.
7. **Remote HTML is sanitized before it is stored** (`ContentSanitizer`); `Post.ContentHtml` is what is shown,
`ContentFormat` says what `Text` holds. Remote names are plain text.
8. **Circles never federate.** A circle's actor, collections, WebFinger and inbox answer 404, and its posts are
`IsLocalOnly`. Only communities are Group actors.
8. **Circles federate to members only.** A circle is an undiscoverable Group actor that takes follow requests (the owner
approves); its posts are addressed to the circle and its `/flock`, delivered to members' personal inboxes, never
announced, and served only to a signed request from a member or a member's instance actor
(`SignedFetchAuthorizer`), 404 otherwise. Circles never appear in search, lookups, mentions or profile pages.
**Communities** are FEP-1b12 groups: `GroupDistributor` announces the whole activity (plus the object for new posts,
for Mastodon), top-level posts are `Page`s with a `name`, posting follows `Group.PostingPolicy`.
Located posts (`LocalGeo`) are the only local-only posts.
9. **A DM joins a conversation only by `DmGroup.ParticipantsKey`**, the exact set of its participants; a remote
`context` decides nothing.
10. **Durable delivery:** activities go out through the delivery queue, never inline in a request.
`context` decides nothing. DMs are `Post`s with `Visibility = Direct` and a `ConversationId` (`DmPost` is legacy).
10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`):
leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`).
11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`.
12. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is
fetched as context. Followers-only is detected by the author's stored `followers` URL.
13. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and
every delete removes its `TimelineEntry` rows. Local deletes are soft (content cleared, 410 Tombstone).
## Mastodon client API invariants
1. **A token is one persona.** Its subject is the avatar id; the root id lives only in the fifteen-minute `/oauth`
cookie used while choosing the persona, and never in a token, an authorization or a response.
2. `/api/*` authenticates with OpenIddict validation, everything else with the old JWT (`PrivaPub` policy scheme).
Every `/api` request re-checks that the persona's root is neither banned nor deleted (`MastodonController`).
3. Read parameters through `Params` (query, form and JSON merged Rails-style), never MVC binding. A value type read
from a conditional must say `(int?)null`, not `default`: that bug once made every list one item long.
4. Answer with `Json(...)` (snake_case, explicit nulls) or `Error(status, message)`; page lists with `Page` and `Link`.
5. Unsupported features answer empty lists or 422 with a message, never 404 or 500, so clients degrade. Any `/api`
error that leaves without a body (a challenge, a 404 from routing, a 429) gets Mastodon's `{"error": ...}` from
`UseMastodonErrorBodies`. `NeverFiveHundredTests` walks every route with junk ids, anonymously, as a persona and
with a junk token.
6. Advertise `4.2.0 (compatible; PrivaPub)` until grouped notifications exist.
7. **What a Mastodon `Status` cannot say goes in `Status.privapub`** (`PrivaPubStatus`): object type, title, excerpt,
cover, the author's source, link, video, audio and event details, and up/down votes. Every media URL in it goes
through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following
one is the reader's choice.
## Media invariants
1. **No upload keeps its metadata.** Images are re-encoded by libvips with `keep=none`; audio and video are remuxed with
`-map_metadata -1`. `MediaProcessingTests` checks EXIF and XMP are gone.
2. Files live under `Media:Root` (`/var/lib/privapub/media`), never in the published directory; the proxy cache is the
sibling `media-proxy`, which `/media/files` does not serve.
3. **A client never contacts a remote server for media:** every remote URL the API returns goes through
`IMediaProxy.Wrap`, an HMAC-signed `/media/proxy/` URL fetched by `IFederationHttp.GetMedia`.
4. **The proxy serves three ways:**
- **Cached:** a file already cached is served from disk, ranges included.
- **Downloaded:** a request without a `Range` is downloaded whole, up to `Media:MaxProxiedBytes`, then cached.
- **Streamed:** a ranged request, or anything too big to cache, is streamed from the origin with the range passed on,
and never cached. That is how remote video plays.
nginx has a `/media/proxy/` location with `proxy_buffering off` and a 600 s read timeout for those streams.
5. **Remote video and audio become one playable attachment** in the Mastodon API (`MastodonMapper.Playable`): the best MP4
up to 720p that carries both sound and picture, including PeerTube's fragmented files inside an HLS entry. HLS
playlists themselves are not rewritten.
## Privacy invariants
@@ -168,8 +236,29 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
equal to the login.
- **One username space:** personas, groups and the instance reserve their name in `ReservedName` (unique index) before
they are saved; `LocalActorService.TryReserveUserName` is the only way to claim one.
- **Usernames are `Constants.UserNameRegex`** (`^[a-z0-9_]+$`): the intersection of what Mastodon and Misskey accept. A
name outside it creates a persona nobody on those servers can reach.
- **An actor's `published` is `PublishedOn`**, a random whole day up to two weeks before creation, so personas made the
same day do not share a date. New persona and group ids carry that same day (`GenerateNewID`), because local clients
see ids. Never emit `CreatedAt`.
- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows. Nothing may relate sibling avatars.
- **Blocks federate** (owner decision, 2026-10-01): a block is sent as `Block` from the blocking avatar, an unblock as
`Undo{Block}`. Reports still leave as `Flag` from the instance actor, never from the reporting avatar.
- **What PrivaPub reveals is the owner's call.** Previews, blocks, website authorship, views, bridging and reactions were
decided in `docs/ROADMAP.md` ("Owner decisions on what PrivaPub reveals"). Anything new that tells another server
something about an avatar gets the same treatment: ask, then record it there.
- **Location-ranged posts never federate.**
- **Statistics name servers, never people** (owner decisions on statistics, 2026-10-03). Every interaction is recorded
through `IInteractionLedger` (`Infrastructure/Statistics`) as an `InteractionEvent` (90 days), but an event never
holds a root, persona or group id, an activity id, an inbox URL, a remote actor URI or a sender IP:
- Distinct remote accounts are counted with `ActorHash`, an HMAC keyed by that day's `InteractionSalt`, which the
day's rollup deletes.
- `LocalKind` survives `Interactions.Sanitize` only on public and unlisted traffic, so a circle can show neither by
presence nor by absence, and no reason code names one.
- Traffic a reader causes (the media proxy, lookups, the client API, fetches of our own documents) is only counted
per day (`Count`, `CountServer`), never logged per event.
- A host claimed by an unverified sender is kept only if it is already a `RemoteInstance`.
- `Record` never blocks and never throws: it writes to a bounded channel, and a full channel drops and counts.
## Data
@@ -179,6 +268,50 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
- **Collection name = class name, so never rename an entity class.**
- `Entity.ID` is a 24-character lowercase hex string; `GenerateNewID()` returns `object`, so cast it.
- New fields must be additive: a deploy rollback restores the binary, not the database.
- **Every stored remote object has an `ObjectRecord`** (raw JSON up to 256 KB plus its hash; delivered or fetched; the
activity, inbox, key and signed headers; received time; up to 10 later revisions). Write it right after the post's
save: `CreateHandler` (delivered) and `RemotePosts.StoreContext` (fetched) do, and `UpdateHandler` appends a
revision. Delivery details reach them through `Arrival.Current`, which `InboxProcessor` sets for the handler's
duration. A fetched record has no signature, key or `@context` of the activity that caused the fetch (its activity
fields name that trigger), and every record stores its `Extensions` and `ContextNamespaces` for statistics. The raw form lives outside `Post` so timelines never load it. Read through
`/api/privapub/v1/statuses/:id/provenance` and `/api/privapub/v1/instances/:host`.
- **Remote objects are parsed for every shape in `ObjectShapes`:** `url`/`icon`/`image` as a value, an object or an
array; Markdown `content`; missing `mediaType`s inferred; thumbnails from any of their five places; and the typed
`Link`, `Video`, `Audio` and `Event` details. A Mastodon API card is built from those, never by fetching the linked page
(that fetch is the owner's decision 1, for P6).
- **An `Update` is an edit only when its `updated` is newer than ours** (`RemoteEdits.IsEdit`; plain and community-wrapped
Updates both go through `RemoteEdits.Apply`, and deletes through `RemoteDeletes.Remove`). Otherwise it refreshes
the poll, video, audio and event details and nothing else, and leaves no revision: Mastodon and Misskey refresh poll
counts with bare Updates.
- **A poll vote is a `Note` with a `name`, an `inReplyTo` that is a poll we hold, and no content.** `CreateHandler`
hands it to `PollService.Receive` before anything else, so it never becomes a reply. Our votes on other servers' polls
go only to the poll's author, without `published` (PieFed counts a vote only then).
- **Link previews follow owner decision 1** (`Domain/Content/LinkPreviews.cs`): only public posts, queued on arrival with
0–60 s of jitter, one cached `LinkPreview` per address for the whole server, never fetched when someone reads.
`LinkPreviews.Wanted` is called where posts are saved (`CreateHandler`, `StoreContext`, `StatusService.Publish`).
- **Quote states come from `QuoteService.Resolve`:**
- with a stamp: accepted only if `Verified` (fetched, on the quoted author's origin, naming both posts exactly);
- a FEP-044f `quote` without a stamp: pending;
- older keys only: accepted when the quoted post is public.
`QuotesCount` moves with the accepted state, never with the raw key. When a persona quotes, `QuotePermission` decides:
asking first for posts that state a policy, quoting at once for posts that state none, refusing otherwise. Only `quote`
of a post that asks for consent puts `quote` in our JSON; older-key quotes leave it out, as Sharkey learned they must.
- **Our quote policy is `ActivityPubRenderer.QuotableBy`:** the post's `LocalQuotePolicy`, falling back to the persona's
`Settings.QuotePolicy` (default `public`, owner decision), and always `nobody` for anything but public and unlisted. The
same rule writes `canQuote`, answers `QuoteRequest`s (`QuoteService.ReceiveRequest`) and fills `quote_approval`, so they
cannot disagree. A persona quoting another persona gets a parrot-licence too, so other servers see an approved quote.
- **A server is described on arrival, never on read.** The first record from a host enqueues `DescribeInstance` (its
NodeInfo, at most once a week, into `RemoteInstance`), so opening the details view tells nobody anything.
- **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post
published within the last hour (or in the future) a fresh `ObjectId`, which sorts after every post already stored,
and only backfill keeps a `published`-derived id. With `published` ids a reply arriving in the same second could sort
under the post it answers, and a late arrival landed behind a client's `since_id` and was never seen. `created_at`
comes from `CreationDate`, never from the id.
- **The same ordering problem exists on the other side, so `published` carries milliseconds**
(`ActivityPubRenderer.Timestamp`). GoToSocial (ULIDs) and Mastodon (Snowflakes) derive a remote status's id from
`published` at millisecond resolution. With whole seconds, two of our posts from the same second sorted at random
there.
- **Startup order:** `EntityMaps.Warm()` (every entity's class map, one at a time; two mapped at once throw "An item
with the same key has already been added" and stay broken), then `MigrateAsync` (`Infrastructure/Data/Migrations`,
`_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a
@@ -196,33 +329,87 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
- Localised strings go through `IStringLocalizer<GenericRes>`.
- Every `Display`/`ErrorMessage` resource key must exist in `FieldsNameResource`/`ErrorsResource`, including the
`Designer.cs`, which the CLI build doesn't regenerate. A missing key throws at validation time.
- **`cond ? value : default` with a value-type branch is the type's default, not null:** `false`, `0`, or year one
stored as an edit date on every remote post. Write `(T?)null`. It has shipped four times (`MastodonParams.Bool/Int`,
`NoteParser.Int`, `NoteParser.Time`).
- ActivityPub output is built with `System.Text.Json.Nodes` in `ActivityPubRenderer`, not typed models. Inbound
documents are read through `InboxService.Id`/`Value`, which handle string, object and array.
documents are read through `ActivityJson.Id`/`Value`, which handle string, object and array.
- Libraries chosen for the roadmap: HtmlSanitizer, Markdig (`DisableHtml`), NSign (RFC 9421 inbound), OpenIddict +
OpenIddict.MongoDb, NetVips, Blurhash.Core, FFMpegCore. No ImageSharp (licence key enforced), no MassTransit.
## Testing
`PrivaPub.Tests` (xUnit v3). Unit tests need nothing; tests marked `Category=Integration` need a mongod and skip
without `PRIVAPUB_TEST_MONGOD=1`. CI runs the unit tests only (the box's mongods are production).
without `PRIVAPUB_TEST_MONGOD=1`. CI (`build.yml` and `deploy.yml`) runs all of them through
`tools/ci/with-test-mongod.sh`, which starts a throwaway mongod on a random localhost port and deletes it afterwards.
The box's own mongods are production, so `MongoFixture` refuses port 27022, a data directory under `/var/lib/privapub`,
and in CI anything but the wrapper's mongod. With `PRIVAPUB_TEST_REQUIRE_MONGOD=1`, which the wrapper sets, a missing
mongod fails the run instead of silently skipping half the tests.
- `Support/Peer` is an in-process HTTP server answering on two origins (`127.0.0.1` and `localhost`), so origin rules
can be tested; `Support/RemoteActor` signs real deliveries with its own key.
- Inbox scenarios go through `InboxService.Receive` with a signed request, not through the private handlers.
- Inbox scenarios go through `InboxReceiver.Receive` with a signed request, not through the private handlers.
- `Support/Host/PrivaPubHost` is the whole server under test (`WebApplicationFactory<Program>`, environment `Testing`,
configured only through `UseSetting`, on the fixture's database). `PrivaPubHost.Shared()` boots it once per run;
`SecureModeHost` is the same with `Federation:SecureMode`. Background workers are removed, so a test runs the jobs it
queued with `host.Run(j => ...)` or `host.RunInbox(activityId)`. `Accounts` signs a root up, adds personas, and gets a
Mastodon token through the real `/oauth` code flow; `RemoteActor.SignedPost`/`SignedGet` sign `HttpRequestMessage`s
for the real `/peasants` routes. Each client gets its own `X-Test-Client` address, so rate limits don't collide.
- All test classes share one database and run in parallel, so a test touches only rows it made:
- random names and GUIDs;
- `Harness.Outgoing` sees only deliveries queued since that harness started, because Peer ports are reused;
- a worker gets a scoped `new JobQueue(j => ...)` so it never leases another test's jobs;
- domain blocks are set with `DomainBlocks.Load`, never written to the database.
A test that must change something database-wide (drop indexes, run a migration over every post, let deliveries to
`localhost` fail and trip its breaker) goes in `[Xunit.Collection(nameof(Exclusive))]`, which runs alone, and
cleans up after itself. Pure logic belongs in an unconditional unit test, not in a Mongo-gated class.
Beyond the tests, verify by building, running locally, and exercising:
- the client API (sign up, create an avatar, a group, a post);
- the ActivityPub endpoints with curl and `Accept: application/activity+json`.
Interop is checked against real servers: Fediverse Pasture with podman on the workstation, verify.funfedi.dev, and
the owner's GoToSocial at social.arasaka.software. **Ask before acting from the owner's GoToSocial account.**
Interop is checked against real servers, starting with the workstation's own pasture:
```bash
DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up # podman: PrivaPub + the latest GoToSocial + Mongo, behind Caddy
tools/pasture/interop.sh # 33 checks, each side driven through its own Mastodon API
tools/pasture/run.sh down
```
- **Two sites on one podman network, one Caddy in front.** `privapub.test` and `gts.test` are network aliases of the
Caddy container, which serves both with its internal CA (`tls internal`). Both servers are told to accept any
certificate: PrivaPub through `appsettings.Pasture.json`, GoToSocial through `GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`.
GoToSocial WebFingers and fetches over https only, so plain http between them is not an option.
- From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. GoToSocial is reached as `https://gts.test:6443`
with `curl -k --resolve gts.test:6443:127.0.0.1`, because its sign-in cookie is bound to the host name.
- **GoToSocial's cached home timeline can stop taking new posts after its first read**, its owner's own included, while
a `min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false`
(`on_gts`), which answers from GoToSocial's database and never fetches from us. A home-timeline check there proves
nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a delete
is checked as a 404 on `/api/v1/statuses/{id}`.
- **GoToSocial creates its accounts locked**, so `interop.sh` approves alice's request through
`/api/v1/follow_requests`. That also checks our pending (`requested`) state and the manual Accept.
- The scenario covers:
- discovery and follows both ways;
- posts and CW;
- a reply and its notification;
- likes and boosts both ways;
- DMs both ways, and the DM staying off public timelines;
- edit, delete both ways, and unfollow.
- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into
`tools/pasture/.publish` and `podman restart pasture-privapub`; that is how a migration is tried on dirty data.
After the pasture, verify.funfedi.dev and the owner's GoToSocial at social.arasaka.software. **Ask before acting from
the owner's GoToSocial account.**
## Deploy
- **CI/CD:** push to `master` runs `build.yml` (build + tests) on the instance-wide `build` runner. A `v*` tag runs
`deploy.yml`: tests, self-contained linux-x64 publish, snapshot and `mongodump` to `/var/backups/privapub.thepra.dev`,
stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo,
Swagger 404, inbox junk 400, unsigned 401).
Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration, client credentials,
discovery, instance, public timeline; pass a persona token as a second argument to check the signed-in side).
- **The box:** Max (`nuvola.xyz`). Unit `privapub` runs as www-data from `/var/www/privapub.thepra.dev` with
`ASPNETCORE_ENVIRONMENT=Production`.
- **One-time root setup:** `deploy/max/setup.sh`, run through `../arasaka.software/tools/max/run.sh`.
+194
View File
@@ -0,0 +1,194 @@
# Federation
PrivaPub is an ActivityPub server written in C#. This document follows
[FEP-67ff](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md) and describes how it federates.
## Supported federation protocols and standards
- [ActivityPub](https://www.w3.org/TR/activitypub/) (server-to-server)
- [WebFinger](https://webfinger.net/)
- [HTTP Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures), `rsa-sha256` / `hs2019` with RSA keys
- [NodeInfo](https://nodeinfo.diaspora.software/) 2.0 and 2.1
## Tested against
- **GoToSocial 0.22.1, end to end.** It runs in a private network on the workstation (`tools/pasture/`) and is driven
through its own client API. Checked both ways:
- follows, including to a locked account;
- public posts with a content warning;
- replies with notifications;
- likes and boosts;
- direct messages;
- edits and deletes;
- unfollow.
- **Mastodon, Misskey, Lemmy and PeerTube, by unit tests only.** The tests feed the parser documents written in each
server's shape. No live exchange with any of them has run yet.
## Supported FEPs
- [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md)
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
- [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md)
- [FEP-1b12: Group federation](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md) (communities; see "Groups")
Planned (see `docs/ROADMAP.md`, phases P5 to P8, and the per-platform notes in `docs/INTEROP.md`): FEP-044f (quotes),
FEP-9967 (polls), FEP-c0e0 (emoji reactions), FEP-9098 (custom emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move),
FEP-8fcf (followers synchronisation), FEP-5feb (`indexable`), FEP-8967 (link attachments), FEP-521a and FEP-8b32 (keys
and integrity proofs), FEP-ae0c (relays).
## Actors
Every account is an *avatar*: one private login can own several, and they are deliberately unlinkable. Nothing in an
actor document, a collection, NodeInfo or a delivery relates two avatars of the same login.
| Thing | Address |
|---|---|
| Actor (Person, Group, Application) | `/peasants/{name}` (`/users/{name}` redirects) |
| Inbox | `/peasants/{name}/mouth` |
| Outbox | `/peasants/{name}/anus` |
| Shared inbox | `/human-centipede` |
| Followers / following | `/peasants/{name}/groupies`, `/peasants/{name}/stalking` |
| Objects | `/peasants/{name}/scribbles/{id}` |
| Activities | `/peasants/{name}/grunts/{id}` |
| Quote permissions (`QuoteAuthorization`) | `/peasants/{name}/parrot-licences/{id}` |
| Direct-message context | `/peasants/{name}/whispers/{id}` |
| Profile and post pages | `/@{name}`, `/@{name}/{id}` |
The names are the project's own and are stable; resolve actors through WebFinger, not by guessing a path.
- The key is `{actor}#main-key`, RSA 2048, served as SPKI PEM with `owner` set to the actor.
- `published` on an actor is a whole day, chosen at random up to two weeks before the account was made, so two
personas made on the same day do not share a date. `indexable` is `false`.
- The instance actor is `/peasants/privapub` (type `Application`). It signs every fetch PrivaPub makes, so no avatar's key
is used to read another server's content.
## Groups
A group is either a **community** or a **circle**.
- A **community** follows [FEP-1b12](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md). Posts
addressed to it (in `to`, `cc` or `audience`) are accepted according to its posting policy (followers, anyone, or
moderators only) and the group `Announce`s the whole activity, with `audience` set, to its followers. A new post is
also announced as an object so Mastodon shows it. Updates and deletes of community content are announced too. A
top-level post is a `Page` with a `name`. Members are counted at `/flock`; moderators are listed at `/wardens`, which
the actor's `attributedTo` points to, with `postingRestrictedToMods` as Lemmy expects. A mention of a community posts into it.
- A **circle** is private. Its actor is not discoverable and every follow is a request. Its posts are addressed to the
circle and its members collection, delivered to each member's own inbox and never announced. They are served only
to a signed request from a member, or from the instance actor of a member's server; anyone else gets 404. A
Mastodon member's replies reach only the people they mention.
- Announces from **remote** groups (Lemmy communities) are followed through to the activity: the object is fetched
from its own origin, never taken from the announce.
## Activities
Received:
| Activity | Effect |
|---|---|
| `Follow` | follows an avatar or community; `Accept` is sent unless the community approves members by hand |
| `Accept{Follow}`, `Reject{Follow}` | completes or ends a follow an avatar requested |
| `Undo{Follow, Like, Announce}` | reverses it |
| `Create{Note, Article, Page, Question, Video, Audio, Event, ChatMessage, …}` | stored when a local avatar follows the author, is addressed or mentioned, when it replies to a local post, or when it is addressed to a community the author follows; a public parent is fetched to complete the thread |
| `Update{Note}` | replaces the content; the previous version is kept |
| `Update{Person}` | refetches the actor |
| `Like` | counted and notified, on posts the liker could see |
| `Dislike` | counted as a downvote (Lemmy, PieFed, Mbin, Friendica); `Undo` takes it back |
| `EmojiReact`, `Like` with an emoji `content` | an emoji reaction (FEP-c0e0; Pleroma, Akkoma, Iceshrimp.NET, Misskey, Sharkey), Unicode or a custom emoji from its `tag`; a `Like` whose content is ❤ stays a favourite; `Undo` takes it back |
| `Join` | answered with `Ignore`: PrivaPub hosts no events yet (FEP-8a8e) |
| `Announce` | counted and notified for local posts; shown to followers of the announcer, with the original refetched from its origin |
| `Delete` | deletes the object, or the actor and its follows; a deleted object id is remembered for 90 days, so a late `Create` cannot bring it back |
| `Flag` | becomes a report for this server's moderators |
Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Create{Question}` and poll votes, `EmojiReact` and its `Undo`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`,
`Reject{Follow}`, `Like`, `Announce` and their `Undo`, `Flag`. A deleted post answers 410 with a `Tombstone`.
- **Attachments** are `Document`s with `mediaType`, `name` (alt text), `blurhash`, `focalPoint`, `width` and `height`.
Uploaded files have all metadata removed.
- **Pinned posts** are the actor's `featured` collection (`/trophies`); `featuredTags` is `/tattoos`.
- **Blocks are sent.** A blocked remote account receives `Block` from the blocking account (and `Reject{Follow}` if it
followed); an unblock sends `Undo{Block}`.
- **Reports** are sent as `Flag` by the instance actor, never by the reporting account.
A `Create`'s `Note` carries Mastodon's `content`, `contentMap`, `summary` and `sensitive`, plus `Mention` and `Hashtag`
tags.
**Polls** (FEP-9967) go out as a `Question` with `oneOf` or `anyOf`, each option's count in `replies.totalItems`,
`endTime`, `votersCount`, and `closed` once it has ended. Incoming votes are `Create{Note{name, inReplyTo}}` with no
content, one per choice, counted once per voter. Counts are refreshed with an `Update{Question}` at most every three
minutes. Our own votes on other servers' polls are sent the same way to the poll's author only, without `published`.
An incoming `Update` without a newer `updated` only refreshes counts and details; it is never recorded as an edit.
**Quotes** (FEP-044f, with the older keys):
- **Received.** A quote is read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 `Link` tag.
- With a `quoteAuthorization`, the stamp is fetched and must be a `QuoteAuthorization` on the quoted author's origin,
attributed to them, naming exactly the two posts.
- A FEP-044f `quote` without a stamp is pending until an `Update` brings one.
- A quote made with the older keys only, by software that asks nobody, is shown when the quoted post is public.
- A `Delete` of a stamp revokes the quote.
- **Sent.** Our accounts quote with every key, plus the `RE:` fallback.
- When the quoted post states an `interactionPolicy.canQuote` that allows us, `quote` is included and a `QuoteRequest`
(with the quoting post as `instrument`) goes to the quoted author. The quote stays pending until their `Accept` brings a
stamp we can verify; the post is then updated with `quoteAuthorization`.
- Posts that state no policy are quoted the older way, without `quote`.
- Quoting posts are also delivered to the quoted author.
- **Quoting our accounts.** Our public and unlisted posts state `interactionPolicy.canQuote`. By default anyone may quote,
automatically (as on Mastodon); an account can choose followers only or nobody, and so can each post.
- A `QuoteRequest` is answered with `Accept{result}` naming a stamp at `/peasants/{name}/parrot-licences/{id}`, or
with `Reject`.
- A stamp is a `QuoteAuthorization` naming both posts. A revoked stamp answers 410 and is announced with
`Delete{stamp}`.
- Followers-only posts and DMs can never be quoted.
**Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object.
**Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`,
`memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
`name`. A content warning without its own text uses the title, or "Content warning".
Received `summary` is a content warning only on a `Note` or `Question`, or when `sensitive` is `true`. On an `Article`,
`Page`, `Event`, `Video` or `Audio` it is an excerpt or description, kept as such and never hidden.
Visibility is expressed in `to`/`cc` the way Mastodon does it: public, unlisted, followers-only and direct. Inbound
followers-only posts are recognised by the author's own `followers` collection.
## Link previews
For a public post that links to a page, PrivaPub builds a preview card. It uses, in order:
1. a Link attachment's own `preview` (FEP-8967) or the post's own image, title and summary;
2. otherwise, the linked page, read once by the server between 0 and 60 seconds after the post arrives (never when
someone reads it), with an `Accept: text/html` request that reads at most 512 KB, through the same address checks as
every other fetch.
The page's OpenGraph and Twitter tags give the title, description and image; the result is cached per address for 7
days and shared by every account on the server. Images are served to clients only through PrivaPub's media proxy.
`Federation:FetchLinkPreviews=false` turns page fetching off.
## Local-only posts
Posts with a location (shown to nearby users of this server) never leave the server, in any form.
## Security rules a peer will notice
- **Signatures.** Inbox POSTs must be signed over `(request-target)`, `host`, `digest` and `date` (or `(created)`). The
date may be at most one hour old and fifteen minutes ahead. A bad signature gets 401, malformed input 400, an accepted
activity 202, too many requests 429. Activities are processed after the 202.
- **Origins.** An actor document is accepted only from the address it names as its `id`. A key only if its actor lists
it with `owner` set to the actor and on the actor's origin. An activity's `id`, and any object it creates, updates or
deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin.
- **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three
redirects and read at most 1 MB.
- **HTML.** Received HTML is sanitised to Mastodon's allowlist.
- **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered
5xx, the inbox answers 503 with `Retry-After: 300` rather than 401.
- **Activity ids.** `Create` and `Announce` ids dereference. `Follow`, `Like`, `Block` and the `Accept`, `Reject` and
`Undo` that answer them do not, because serving them would reveal who follows, likes and blocks whom; they are always
sent with their object embedded.
- **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
starting at an hour and growing to a week.
## Known limitations
- Our own posts carry no custom emoji: emoji are received and shown, not offered.
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
- Collections expose counts, not members.
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
@@ -0,0 +1,33 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.Admin
{
public class DomainBlockForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(253, MinimumLength = 3, ErrorMessageResourceName = "StringLengthMinMax", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Domain { get; set; }
public bool Suspend { get; set; } = true;
public bool RejectMedia { get; set; }
[StringLength(1000, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string PublicComment { get; set; }
[StringLength(1000, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string PrivateComment { get; set; }
}
public class ViewDomainBlock
{
public string Id { get; set; }
public string Domain { get; set; }
public string Severity { get; set; }
public bool RejectMedia { get; set; }
public string PublicComment { get; set; }
public string PrivateComment { get; set; }
public DateTime CreatedAt { get; set; }
}
}
+1
View File
@@ -9,5 +9,6 @@
public const int GroupInvitationLength = 64;
public const string PasswordRegex = @"^(?=.*)(?=.*[a-z])(?=.*[A-Z])(?!.*\s).*$";
public const string UserNameRegex = "^[a-z0-9_]+$";
}
}
@@ -12,7 +12,7 @@ namespace PrivaPub.ClientModels.Group
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(32, MinimumLength = 3, ErrorMessageResourceName = "StringLengthMinMax", ErrorMessageResourceType = typeof(ErrorsResource)),
RegularExpression("^[a-z0-9_]+$", ErrorMessageResourceName = "EmptySpacesNotAllowed", ErrorMessageResourceType = typeof(ErrorsResource)),
RegularExpression(Constants.UserNameRegex, ErrorMessageResourceName = "InvalidUserName", ErrorMessageResourceType = typeof(ErrorsResource)),
NoWhiteSpaces]
public string UserName { get; set; }
@@ -25,6 +25,7 @@ namespace PrivaPub.ClientModels.Group
public string InvitationPassword { get; set; }
public bool IsCommunity { get; set; }
public string PostingPolicy { get; set; }//followers (default), anyone, moderators
public bool IsDiscoverable { get; set; } = true;
public bool ManuallyApprovesMembers { get; set; }
}
@@ -20,6 +20,7 @@ namespace PrivaPub.ClientModels.Group
public bool RemoveInvitationPassword { get; set; }
public bool RegenerateInvitationCode { get; set; }
public bool? IsDiscoverable { get; set; }
public string PostingPolicy { get; set; }
public bool? ManuallyApprovesMembers { get; set; }
}
}
+1
View File
@@ -9,6 +9,7 @@ namespace PrivaPub.ClientModels.Group
public string Url { get; set; }
public string Handle { get; set; }
public bool IsCommunity { get; set; }
public string PostingPolicy { get; set; }
public bool IsDiscoverable { get; set; }
public bool ManuallyApprovesMembers { get; set; }
public bool IsOwner { get; set; }
+1 -1
View File
@@ -12,7 +12,7 @@ namespace PrivaPub.ClientModels
public string InvitationCode { get; set; }
[StringLength(32, MinimumLength = 3, ErrorMessageResourceName = "StringLengthMinMax", ErrorMessageResourceType = typeof(ErrorsResource)),
RegularExpression("^[a-z0-9_]+$", ErrorMessageResourceName = "EmptySpacesNotAllowed", ErrorMessageResourceType = typeof(ErrorsResource))]
RegularExpression(Constants.UserNameRegex, ErrorMessageResourceName = "InvalidUserName", ErrorMessageResourceType = typeof(ErrorsResource))]
public string AvatarUserName { get; set; }
[StringLength(64, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
@@ -19,6 +19,41 @@ namespace PrivaPub.ClientModels.Post
public string GroupId { get; set; }
public string AnsweringToPostId { get; set; }
public bool HasContentWarning { get; set; }
[StringLength(500, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string SpoilerText { get; set; }
public string Visibility { get; set; }//public (default), unlisted, followersonly
[Range(-90, 90, ErrorMessageResourceName = "Range", ErrorMessageResourceType = typeof(ErrorsResource))]
public double? Latitude { get; set; }
[Range(-180, 180, ErrorMessageResourceName = "Range", ErrorMessageResourceType = typeof(ErrorsResource))]
public double? Longitude { get; set; }
[Range(1, 50, ErrorMessageResourceName = "Range", ErrorMessageResourceType = typeof(ErrorsResource))]
public double? RangeKm { get; set; }
}
public class UpdatePostForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource))]
public string AvatarId { get; set; }
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource))]
public string PostId { get; set; }
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(5000, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Text { get; set; }
[StringLength(200, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Title { get; set; }
public bool HasContentWarning { get; set; }
[StringLength(500, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string SpoilerText { get; set; }
}
public class InsertDmForm
+4
View File
@@ -8,6 +8,7 @@ namespace PrivaPub.ClientModels.Post
public string AuthorActorURI { get; set; }
public string GroupId { get; set; }
public string DmGroupId { get; set; }
public string Visibility { get; set; }
public string AnsweringToPostId { get; set; }
public string Title { get; set; }
public string Text { get; set; }
@@ -15,6 +16,9 @@ namespace PrivaPub.ClientModels.Post
public bool HasContentWarning { get; set; }
public bool IsFederatedCopy { get; set; }
public DateTime CreationDate { get; set; }
public ViewPost ReblogOf { get; set; }
public double? RangeKm { get; set; }
public double? DistanceKm { get; set; }
}
public class ViewDmGroup
@@ -105,6 +105,15 @@ namespace PrivaPub.ClientModels.Resources {
}
}
/// <summary>
/// Looks up a localized string similar to Use only lower-case letters, digits and underscores..
/// </summary>
public static string InvalidUserName {
get {
return ResourceManager.GetString("InvalidUserName", resourceCulture);
}
}
/// <summary>
/// Looks up a localized string similar to Invalid password, upper-case, lower-case and number characters required..
/// </summary>
@@ -132,6 +132,9 @@
<data name="InvalidEmail" xml:space="preserve">
<value>{0} è invalida.</value>
</data>
<data name="InvalidUserName" xml:space="preserve">
<value>Usa solo lettere minuscole, cifre e trattini bassi.</value>
</data>
<data name="InvalidPassword" xml:space="preserve">
<value>Password non valida, deve avere almeno un carattere grande, un carattere piccolo ed un numero.</value>
</data>
@@ -132,6 +132,9 @@
<data name="InvalidEmail" xml:space="preserve">
<value>{0} is invalid.</value>
</data>
<data name="InvalidUserName" xml:space="preserve">
<value>Use only lower-case letters, digits and underscores.</value>
</data>
<data name="InvalidPassword" xml:space="preserve">
<value>Invalid password, upper-case, lower-case and number characters required.</value>
</data>
@@ -0,0 +1,37 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.Social
{
public class FollowForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource))]
public string AvatarId { get; set; }
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(2048, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Target { get; set; }//user@host, @user, or an actor URI
}
public class ViewFollowing
{
public string Id { get; set; }
public string TargetActorURI { get; set; }
public string TargetAccountId { get; set; }
public bool TargetIsLocal { get; set; }
public string State { get; set; }
public DateTime CreatedAt { get; set; }
}
public class ViewNotification
{
public string Id { get; set; }
public string Type { get; set; }
public string FromAccountId { get; set; }
public string FromActorURI { get; set; }
public string PostId { get; set; }
public bool IsRead { get; set; }
public DateTime CreatedAt { get; set; }
}
}
@@ -20,6 +20,7 @@ namespace PrivaPub.ClientModels.User.Avatar
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(Constants.MaxAvatarNameLength, ErrorMessageResourceName = "StringLength", ErrorMessageResourceType = typeof(ErrorsResource)),
RegularExpression(Constants.UserNameRegex, ErrorMessageResourceName = "InvalidUserName", ErrorMessageResourceType = typeof(ErrorsResource)),
Display(Name = nameof(UserName), ResourceType = typeof(FieldsNameResource))]
public string UserName { get; set; }//preferredUsername
+82
View File
@@ -0,0 +1,82 @@
using Microsoft.AspNetCore.Http;
using PrivaPub.Api.Mastodon.Auth;
using PrivaPub.Api.Mastodon.Infrastructure;
using System.Text;
namespace PrivaPub.Tests.Api
{
public class MastodonParamsTests
{
static async Task<MastodonParams> Read(string query = "", string contentType = default, string body = default)
{
var context = new DefaultHttpContext();
context.Request.QueryString = new QueryString(query);
if (body != default)
{
context.Request.ContentType = contentType;
var bytes = Encoding.UTF8.GetBytes(body);
context.Request.Body = new MemoryStream(bytes);
context.Request.ContentLength = bytes.Length;
}
return await MastodonParams.Read(context.Request, TestContext.Current.CancellationToken);
}
[Fact]
public async Task Absent_values_are_null_not_false_or_zero()
{
var parameters = await Read();
Assert.Null(parameters.Bool("sensitive"));
Assert.Null(parameters.Int("limit"));
Assert.Null(parameters.Get("status"));
Assert.Empty(parameters.List("media_ids"));
}
[Fact]
public async Task Reads_rails_style_arrays_from_the_query()
{
var parameters = await Read("?id[]=1&id[]=2&limit=5&local=true");
Assert.Equal(new[] { "1", "2" }, parameters.List("id"));
Assert.Equal(5, parameters.Int("limit"));
Assert.True(parameters.Bool("local"));
}
[Fact]
public async Task Reads_forms_and_nested_keys()
{
var parameters = await Read(contentType: "application/x-www-form-urlencoded",
body: "status=hi&media_ids[]=a&media_ids[]=b&poll[options][]=x&poll[options][]=y&sensitive=0");
Assert.Equal("hi", parameters.Get("status"));
Assert.Equal(new[] { "a", "b" }, parameters.List("media_ids"));
Assert.Equal(new[] { "x", "y" }, parameters.List("poll[options]"));
Assert.False(parameters.Bool("sensitive"));
}
[Fact]
public async Task Reads_json_bodies_the_same_way()
{
var parameters = await Read(contentType: "application/json",
body: """{"status":"hi","media_ids":["a","b"],"poll":{"options":["x","y"],"expires_in":300},"sensitive":true,"source":{"privacy":"unlisted"}}""");
Assert.Equal(new[] { "a", "b" }, parameters.List("media_ids"));
Assert.Equal(new[] { "x", "y" }, parameters.List("poll[options]"));
Assert.Equal(300, parameters.Int("poll[expires_in]"));
Assert.True(parameters.Bool("sensitive"));
Assert.Equal("unlisted", parameters.Get("source[privacy]"));
}
[Theory]
[InlineData(new[] { "read" }, "read:statuses", true)]
[InlineData(new[] { "read:statuses" }, "read:statuses", true)]
[InlineData(new[] { "read:accounts" }, "read:statuses", false)]
[InlineData(new[] { "write" }, "read:statuses", false)]
[InlineData(new[] { "follow" }, "write:follows", true)]
[InlineData(new[] { "follow" }, "write:statuses", false)]
public void Scopes_grant_like_mastodons(string[] granted, string required, bool expected) =>
Assert.Equal(expected, MastodonScopes.Grants(granted, required));
}
}
@@ -0,0 +1,61 @@
using Microsoft.Extensions.Caching.Memory;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.ClientModels.Post;
using PrivaPub.ClientModels.Social;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json;
namespace PrivaPub.Tests.Api
{
[Trait("Category", "Integration")]
public sealed class PersonaSeparationTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task Nothing_shown_to_one_persona_names_its_sibling()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var (_, sibling) = await _harness.Persona("sibling", root);
var (bobRoot, bob) = await _harness.Persona("bob");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = bob.UserName }, token);
await _harness.Follows.Follow(root, new FollowForm { AvatarId = sibling.Id, Target = bob.UserName }, token);
await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = sibling.Id, Text = "from the sibling" }, token);
await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "from alice" }, token);
await _harness.Posts.InsertPost(bobRoot, new InsertPostForm { AvatarId = bob.Id, Text = "hi both" }, token);
var mapper = new MastodonMapper(_harness.Db, _harness.Local);
var account = await mapper.Local(alice, withSource: true, token);
var homeIds = (await _harness.Db.TimelineEntries.Match(e => e.AvatarId == alice.Id).ExecuteAsync(token)).Select(e => e.PostId).ToList();
var home = await _harness.Db.Posts.Match(p => homeIds.Contains(p.ID)).ExecuteAsync(token);
var statuses = await mapper.Statuses(home, alice.Id, token);
var bobAsSeenByAlice = await mapper.Account(bob.Id, token);
var json = JsonSerializer.Serialize(new object[] { account, statuses, bobAsSeenByAlice }, MastodonJson.Options);
Assert.DoesNotContain(sibling.Id, json);
Assert.DoesNotContain(sibling.UserName, json);
Assert.DoesNotContain(root, json);
Assert.Contains("hi both", json);
Assert.Equal(2, bobAsSeenByAlice.FollowersCount);
}
}
}
+29
View File
@@ -0,0 +1,29 @@
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Models.Post;
namespace PrivaPub.Tests.Api
{
public class StatusContentTests
{
[Fact]
public void A_remote_article_shows_its_title_above_its_body() =>
Assert.Equal("<p><strong>Rain &amp; sun</strong></p><p>body</p>", MastodonMapper.Content(new Post
{
IsFederatedCopy = true, ObjectType = "Article", Title = "Rain & sun", ContentHtml = "<p>body</p>"
}));
[Fact]
public void A_remote_note_keeps_its_content_alone() =>
Assert.Equal("<p>body</p>", MastodonMapper.Content(new Post
{
IsFederatedCopy = true, ObjectType = "Note", Title = "a poll choice", ContentHtml = "<p>body</p>"
}));
[Fact]
public void A_remote_event_without_a_body_shows_its_excerpt() =>
Assert.Equal("<p><strong>Meetup</strong></p><p>Saturday &lt;10:00&gt;</p>", MastodonMapper.Content(new Post
{
IsFederatedCopy = true, ObjectType = "Event", Title = "Meetup", Excerpt = "Saturday <10:00>", ContentHtml = string.Empty
}));
}
}
@@ -0,0 +1,91 @@
using PrivaPub.Domain.Content;
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Domain
{
public class ContentRendererTests
{
const string Base = "https://privapub.test";
readonly ContentRenderer _renderer = new(new StubLocalActors(), new StubRemoteActors());
[Fact]
public async Task Decorates_local_and_remote_mentions_and_hashtags()
{
var rendered = await _renderer.Markdown("hello @alice and @bob@m.example about #Fedi **now** <script>x</script>", TestContext.Current.CancellationToken);
Assert.Contains("<span class=\"h-card\" translate=\"no\"><a href=\"https://privapub.test/peasants/alice\" class=\"u-url mention\"", rendered.Html);
Assert.Contains("href=\"https://m.example/@bob\"", rendered.Html);
Assert.Contains("href=\"https://privapub.test/tags/fedi\" class=\"mention hashtag\"", rendered.Html);
Assert.Contains("<strong>now</strong>", rendered.Html);
Assert.DoesNotContain("<script>", rendered.Html);
Assert.Equal(new[] { "https://m.example/users/bob", "https://privapub.test/peasants/alice" }, rendered.Mentions.Select(m => m.ActorUri).Order(StringComparer.Ordinal));
Assert.Equal("fedi", Assert.Single(rendered.Tags));
}
[Theory]
[InlineData("`@alice` in code")]
[InlineData("mail me@alice.example")]
[InlineData("@nobody is unknown")]
[InlineData("see https://x.example/page#alice")]
public async Task Leaves_what_is_not_a_mention_alone(string text)
{
var rendered = await _renderer.Markdown(text, TestContext.Current.CancellationToken);
Assert.Empty(rendered.Mentions);
Assert.DoesNotContain("h-card", rendered.Html);
}
[Theory]
[InlineData("issue #123")]
[InlineData("it&#39;s fine")]
[InlineData("[link](https://x.example/#tag)")]
public async Task Leaves_what_is_not_a_hashtag_alone(string text) =>
Assert.Empty((await _renderer.Markdown(text, TestContext.Current.CancellationToken)).Tags);
[Fact]
public async Task Renders_plain_text_like_mastodon()
{
var rendered = await _renderer.PlainText("line one\nline <two>\n\nsee https://x.example/a?b=1. #Tag", TestContext.Current.CancellationToken);
Assert.Equal("<p>line one<br>line &lt;two&gt;</p><p>see <a href=\"https://x.example/a?b=1\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">https://x.example/a?b=1</a>. "
+ "<a href=\"https://privapub.test/tags/tag\" class=\"mention hashtag\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">#<span>Tag</span></a></p>", rendered.Html);
}
sealed class StubLocalActors : ILocalActorService
{
public string BaseAddress => Base;
public Task<LocalActor> FindByUserName(string userName, CancellationToken token) =>
Task.FromResult(userName.Equals("alice", StringComparison.OrdinalIgnoreCase)
? new LocalActor { Id = "a1", UserName = "alice", BaseAddress = Base, Kind = LocalActorKind.Person }
: default);
public Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> FindByUri(string actorUri, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> GetInstanceActor(CancellationToken token) => throw new NotSupportedException();
public Task<bool> IsUserNameTaken(string userName, CancellationToken token) => throw new NotSupportedException();
public Task<bool> TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token) => throw new NotSupportedException();
public LocalActor FromAvatar(Avatar avatar) => throw new NotSupportedException();
public LocalActor FromGroup(GroupEntity group) => throw new NotSupportedException();
}
sealed class StubRemoteActors : IRemoteActorService
{
public Task<string> ResolveHandle(string handle, CancellationToken token) =>
Task.FromResult(handle == "bob@m.example" ? "https://m.example/users/bob" : default);
public Task<ForeignAvatar> GetActor(string actorUri, bool refresh, CancellationToken token) =>
Task.FromResult(actorUri == "https://m.example/users/bob"
? new ForeignAvatar { ID = "f1", ActorURI = actorUri, UserName = "bob", Url = "https://m.example/@bob", InboxURL = actorUri + "/inbox" }
: default);
public Task<FetchedJson> FetchObject(string uri, CancellationToken token) => throw new NotSupportedException();
public Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token) => throw new NotSupportedException();
}
}
}
+124
View File
@@ -0,0 +1,124 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Social;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public sealed class FollowTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task A_local_follow_is_accepted_at_once_and_notified()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var result = await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = "@" + bob.UserName }, token);
Assert.True(result.IsValid);
var following = await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id).ExecuteSingleAsync(token);
Assert.Equal(FollowState.Accepted, following.State);
Assert.True(following.TargetIsLocal);
Assert.True((await DB.Default.Find<Follower>().Match(f => f.LocalActorId == bob.Id && f.ActorURI == alice.Uri).ExecuteSingleAsync(token)).IsAccepted);
Assert.Equal(NotificationType.Follow, (await DB.Default.Find<Notification>().Match(n => n.AvatarId == bob.Id).ExecuteSingleAsync(token)).Type);
Assert.Empty(await _harness.Outgoing(bob.Inbox));
}
[Fact]
public async Task Someone_elses_persona_cannot_be_made_to_follow()
{
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var result = await _harness.Follows.Follow("someone-else", new FollowForm { AvatarId = alice.Id, Target = bob.UserName }, TestContext.Current.CancellationToken);
Assert.False(result.IsValid);
Assert.Equal(404, result.StatusCode);
}
[Fact]
public async Task A_remote_follow_is_requested_then_accepted()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = carol.Id }, token);
var following = await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id).ExecuteSingleAsync(token);
Assert.Equal(FollowState.Requested, following.State);
var follow = Assert.Single(await _harness.Outgoing(carol.Id + "/inbox"));
Assert.Equal("Follow", follow["type"]!.GetValue<string>());
Assert.Equal(alice.Uri, follow["actor"]!.GetValue<string>());
Assert.StartsWith(alice.Uri + "/grunts/follow-", follow["id"]!.GetValue<string>());
var accept = new JsonObject
{
["id"] = $"{carol.Id}#accepts/{Guid.NewGuid():N}",
["type"] = "Accept",
["actor"] = carol.Id,
["object"] = follow.DeepClone()
};
Assert.Equal(202, (await _harness.Deliver(carol, $"/peasants/{alice.UserName}/mouth", accept)).StatusCode);
Assert.Equal(FollowState.Accepted, (await DB.Default.Find<Following>().OneAsync(following.ID, token)).State);
}
[Fact]
public async Task An_accept_from_someone_else_changes_nothing_and_a_reject_ends_the_request()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = carol.Id }, token);
var following = await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id).ExecuteSingleAsync(token);
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = $"{mallory.Id}#accepts/{Guid.NewGuid():N}", ["type"] = "Accept", ["actor"] = mallory.Id, ["object"] = following.FollowActivityURI
});
Assert.Equal(FollowState.Requested, (await DB.Default.Find<Following>().OneAsync(following.ID, token)).State);
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{carol.Id}#rejects/{Guid.NewGuid():N}", ["type"] = "Reject", ["actor"] = carol.Id, ["object"] = following.FollowActivityURI
});
Assert.Null(await DB.Default.Find<Following>().OneAsync(following.ID, token));
}
[Fact]
public async Task Unfollowing_a_remote_account_sends_an_undo()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = carol.Id }, token);
await _harness.Follows.Unfollow(root, new FollowForm { AvatarId = alice.Id, Target = carol.Id }, token);
Assert.False(await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id).ExecuteAnyAsync(token));
var undo = (await _harness.Outgoing(carol.Id + "/inbox")).Single(a => a["type"]!.GetValue<string>() == "Undo");
Assert.Equal("Follow", undo["object"]!["type"]!.GetValue<string>());
}
}
}
+43
View File
@@ -0,0 +1,43 @@
using PrivaPub.Domain.Content;
using PrivaPub.Models.Post;
namespace PrivaPub.Tests.Domain
{
public class LinkPreviewTests
{
[Fact]
public void Reads_open_graph_then_twitter_then_the_page_title()
{
var preview = LinkPreviews.Read("https://news.example/a?utm_source=x", new Uri("https://news.example/a"), """
<html><head><title>Fallback title</title>
<meta property="og:title" content="Rain &amp; sun">
<meta name="twitter:description" content="A <b>long</b> story">
<meta property="og:image" content="/img/cover.jpg">
<meta property="og:site_name" content="News">
</head><body></body></html>
""");
Assert.Equal("Rain & sun", preview.Title);
Assert.Equal("A long story", preview.Description);
Assert.Equal("https://news.example/img/cover.jpg", preview.ImageURL);
Assert.Equal("News", preview.SiteName);
Assert.False(preview.Failed);
Assert.True(LinkPreviews.Read("https://x.example", new Uri("https://x.example"), "<html><body>nothing</body></html>").Failed);
}
[Fact]
public void Picks_the_first_real_link_and_skips_mentions_tags_quotes_and_ourselves() =>
Assert.Equal("https://blog.example/post", LinkPreviews.FirstLink(new Post
{
Url = "https://m.example/@a/1",
ContentHtml = """
<p><span class="h-card"><a href="https://m.example/@bob" class="u-url mention">@bob</a></span>
<a href="https://m.example/tags/x" class="mention hashtag" rel="tag">#x</a>
<span class="quote-inline">RE: <a href="https://k.example/notes/1">link</a></span>
<a href="https://privapub.test/@me">me</a>
<a href="https://m.example/@a/1">self</a>
<a href="https://blog.example/post">read this</a></p>
"""
}, "https://privapub.test"));
}
}
+102
View File
@@ -0,0 +1,102 @@
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Memory;
using MongoDB.Entities;
using NetVips;
using PrivaPub.Domain.Media;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Media;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public sealed class MediaFlowTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static byte[] Png(int width, int height)
{
using var image = (Image.Black(width, height, bands: 3) + new double[] { 10, 120, 200 }).Cast(Enums.BandFormat.Uchar);
return image.WriteToBuffer(".png");
}
static IFormFile Upload(byte[] bytes, string contentType) =>
new FormFile(new MemoryStream(bytes), 0, bytes.Length, "file", "picture") { Headers = new HeaderDictionary(), ContentType = contentType };
[Fact]
public async Task An_upload_is_attached_once_and_federated_with_its_alt_text()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, mallory) = await _harness.Persona("mallory");
var upload = await _harness.Media.Upload(alice, Upload(Png(300, 200), "image/png"), "a blue square", "0.25,-0.5", token);
Assert.True(upload.Ok);
Assert.True(File.Exists(Path.Combine(_harness.Media.Root, upload.Attachment.FilePath)));
var stolen = await _harness.Statuses.Publish(mallory, new StatusDraft { Text = "mine", MediaIds = new[] { upload.Attachment.ID } }, token);
Assert.False(stolen.Ok);
var posted = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "look", MediaIds = new[] { upload.Attachment.ID } }, token);
Assert.True(posted.Ok);
Assert.Equal(posted.Post.ID, (await DB.Default.Find<MediaAttachment>().OneAsync(upload.Attachment.ID, token)).PostId);
var note = ActivityPubRenderer.Note(posted.Post, alice, default, default);
var attachment = note["attachment"]![0]!;
Assert.Equal("a blue square", attachment["name"]!.GetValue<string>());
Assert.Equal(300, attachment["width"]!.GetValue<int>());
Assert.Equal(-0.5f, attachment["focalPoint"]![1]!.GetValue<float>());
Assert.StartsWith("https://privapub.test/media/files/", attachment["url"]!.GetValue<string>());
var reused = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "again", MediaIds = new[] { upload.Attachment.ID } }, token);
Assert.False(reused.Ok);
}
[Fact]
public async Task Unsupported_files_are_refused()
{
var (_, alice) = await _harness.Persona("alice");
var outcome = await _harness.Media.Upload(alice, Upload(System.Text.Encoding.UTF8.GetBytes("<svg/>"), "image/svg+xml"), default, default, TestContext.Current.CancellationToken);
Assert.False(outcome.Ok);
Assert.Equal(422, outcome.Status);
}
[Fact]
public async Task The_proxy_serves_signed_remote_media_and_nothing_else()
{
var token = TestContext.Current.CancellationToken;
var path = $"/files/{Guid.NewGuid():N}.png";
_harness.Peer.ServeFile(path, Png(10, 10), "image/png");
var proxy = new MediaProxy(_harness.Local, Peer.Http(), _harness.Media, new StaticOptions<MediaOptions>(new MediaOptions()));
var wrapped = proxy.Wrap(_harness.Peer.A + path);
var parts = new Uri(wrapped).AbsolutePath.Split('/');
var (file, contentType) = await proxy.Fetch(parts[3], parts[4], token);
var (tampered, _) = await proxy.Fetch(parts[3].Replace(parts[3][0], parts[3][0] == 'A' ? 'B' : 'A'), parts[4], token);
Assert.StartsWith("https://privapub.test/media/proxy/", wrapped);
Assert.Equal("image/png", contentType);
Assert.True(File.Exists(file));
Assert.StartsWith(_harness.Media.ProxyRoot, file);
Assert.Null(tampered);
}
}
}
@@ -0,0 +1,72 @@
using NetVips;
using PrivaPub.Domain.Media;
namespace PrivaPub.Tests.Domain
{
public class MediaProcessingTests
{
static byte[] JpegWithMetadata(int width, int height)
{
using var image = (Image.Black(width, height, bands: 3) + new double[] { 200, 40, 90 }).Cast(Enums.BandFormat.Uchar);
using var tagged = image.Mutate(m =>
{
m.Set(GValue.GStrType, "exif-ifd0-ImageDescription", "where I live");
m.Set(GValue.GStrType, "exif-ifd0-Artist", "Alice Smith");
m.Set(GValue.GStrType, "exif-ifd2-UserComment", "at home");
m.Set(GValue.BlobType, "xmp-data", System.Text.Encoding.UTF8.GetBytes("<x:xmpmeta xmlns:x='adobe:ns:meta/'><secret/></x:xmpmeta>"));
});
return tagged.WriteToBuffer(".jpg");
}
[Fact]
public void Uploaded_images_lose_every_kind_of_metadata()
{
var input = JpegWithMetadata(800, 600);
using (var original = Image.NewFromBuffer(input))
Assert.Contains("exif-data", original.GetFields());
var processed = MediaService.ProcessImage(input, 4096, 640, animated: false);
using var output = Image.NewFromBuffer(processed.Bytes);
var fields = output.GetFields();
Assert.DoesNotContain("exif-data", fields);
Assert.DoesNotContain("xmp-data", fields);
Assert.DoesNotContain("iptc-data", fields);
Assert.DoesNotContain(fields, f => f.StartsWith("exif-ifd"));
Assert.Equal((800, 600), (processed.Width, processed.Height));
Assert.Equal("image/jpeg", processed.ContentType);
using var preview = Image.NewFromBuffer(processed.Preview);
Assert.Equal(640, Math.Max(preview.Width, preview.Height));
Assert.DoesNotContain("exif-data", preview.GetFields());
}
[Fact]
public void Large_images_are_capped()
{
var processed = MediaService.ProcessImage(JpegWithMetadata(5000, 2500), 4096, 640, animated: false);
Assert.Equal((4096, 2048), (processed.Width, processed.Height));
}
[Fact]
public void A_blurhash_is_well_formed()
{
var processed = MediaService.ProcessImage(JpegWithMetadata(64, 64), 4096, 640, animated: false);
Assert.Equal(28, processed.Blurhash.Length);
Assert.Equal('L', processed.Blurhash[0]);
}
[Fact]
public void A_white_image_encodes_white_as_its_average()
{
var pixels = Enumerable.Range(0, 16 * 16).SelectMany(_ => new byte[] { 255, 255, 255 }).ToArray();
var hash = Blurhash.Encode(pixels, 16, 16);
Assert.Equal(28, hash.Length);
Assert.Equal("TSUA", hash[2..6]);
}
}
}
+77
View File
@@ -0,0 +1,77 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Outbox;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public sealed class NearbyTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task A_located_post_reaches_only_those_within_its_radius_and_never_leaves_the_server()
{
var token = TestContext.Current.CancellationToken;
var latitude = 10 + Random.Shared.NextDouble() * 50;
var longitude = 10 + Random.Shared.NextDouble() * 50;
var (aliceRoot, alice) = await _harness.Persona("alice");
var (bobRoot, bob) = await _harness.Persona("bob");
var (carolRoot, carol) = await _harness.Persona("carol");
var follower = new RemoteActor(_harness.Peer, "follower");
await _harness.FollowedBy(alice, follower);
var created = await _harness.Posts.InsertPost(aliceRoot, new InsertPostForm
{
AvatarId = alice.Id, Text = "free bread at the bakery", Latitude = latitude, Longitude = longitude, RangeKm = 5
}, token);
var post = await DB.Default.Find<Post>().OneAsync(((ViewPost)created.Data).Id, token);
var three = (List<ViewPost>)(await _harness.Posts.Nearby(bobRoot, bob.Id, latitude + 0.027, longitude, token)).Data;
var eight = (List<ViewPost>)(await _harness.Posts.Nearby(carolRoot, carol.Id, latitude + 0.072, longitude, token)).Data;
Assert.Equal(PostVisibility.LocalGeo, post.Visibility);
Assert.Equal(Math.Round(latitude, 2), post.Geo.Coordinates[1]);
Assert.Contains(three, p => p.Id == post.ID);
Assert.InRange(three.First(p => p.Id == post.ID).DistanceKm.Value, 2.0, 4.0);
Assert.DoesNotContain(eight, p => p.Id == post.ID);
Assert.False(await VisibilityPolicy.CanSee(post, bob.Id, token));
var outgoing = (await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.Deliver).ExecuteAsync(token))
.Select(j => JsonSerializer.Deserialize<DeliveryPayload>(j.Payload))
.Where(p => p.Body.Contains(post.ID));
Assert.Empty(outgoing);
}
[Fact]
public async Task A_located_post_cannot_be_direct_or_in_a_group()
{
var (root, alice) = await _harness.Persona("alice");
var result = await _harness.Statuses.Publish(alice, new PrivaPub.Domain.Statuses.StatusDraft
{
Text = "x", Latitude = 45, Longitude = 7, Visibility = PostVisibility.Direct
}, TestContext.Current.CancellationToken);
Assert.False(result.Ok);
}
}
}
+92
View File
@@ -0,0 +1,92 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.ClientModels;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public sealed class OutboxTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
async Task<ForeignAvatar> Known(RemoteActor actor) =>
await _harness.Remote.GetActor(actor.Id, refresh: false, TestContext.Current.CancellationToken);
[Fact]
public async Task The_audience_follows_mastodons_table()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var follower = new RemoteActor(_harness.Peer, "follower");
var mentioned = await Known(new RemoteActor(_harness.Peer, "mentioned"));
var parentAuthor = await Known(new RemoteActor(_harness.Peer, "parent"));
await _harness.FollowedBy(alice, follower);
var mention = new PostMention { ActorURI = mentioned.ActorURI };
async Task<IReadOnlyList<string>> Audience(PostVisibility visibility, string replyTo = default) =>
await _harness.Outbox.Audience(alice, new Post { Visibility = visibility, Mentions = new() { mention }, To = new() { mentioned.ActorURI }, InReplyToAccountId = replyTo }, token);
Assert.Equal(new[] { follower.SharedInbox, mentioned.InboxURL }, await Audience(PostVisibility.Public));
Assert.Equal(new[] { follower.SharedInbox, mentioned.InboxURL, parentAuthor.InboxURL }, await Audience(PostVisibility.Unlisted, parentAuthor.ID));
Assert.Equal(new[] { follower.SharedInbox, mentioned.InboxURL }, await Audience(PostVisibility.FollowersOnly));
Assert.Equal(new[] { mentioned.InboxURL }, await Audience(PostVisibility.Direct, parentAuthor.ID));
Assert.Empty(await Audience(PostVisibility.Circle));
}
[Fact]
public async Task A_post_is_created_edited_and_deleted_for_its_audience()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var follower = new RemoteActor(_harness.Peer, "follower");
await _harness.FollowedBy(alice, follower);
var created = await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "first", Visibility = "followersonly" }, token);
var postId = ((ViewPost)created.Data).Id;
await _harness.Posts.UpdatePost(root, new UpdatePostForm { AvatarId = alice.Id, PostId = postId, Text = "second", SpoilerText = "cw" }, token);
await _harness.Posts.DeletePost(root, new DeletePostForm { AvatarId = alice.Id, PostId = postId }, token);
var sent = await _harness.Outgoing(follower.SharedInbox);
Assert.Equal(new[] { "Create", "Update", "Delete" }, sent.Select(a => a["type"]!.GetValue<string>()));
Assert.DoesNotContain(sent[0]["to"]!.AsArray(), n => n!.GetValue<string>().EndsWith("#Public"));
Assert.Equal("cw", sent[1]["object"]!["summary"]!.GetValue<string>());
Assert.NotNull(sent[1]["object"]!["updated"]);
Assert.Equal(sent[0]["to"]!.ToJsonString(), sent[2]["to"]!.ToJsonString());
var stored = await DB.Default.Find<Post>().OneAsync(postId, token);
Assert.NotNull(stored.DeletedAt);
Assert.Null(stored.ContentHtml);
Assert.Empty(stored.Revisions);
}
[Fact]
public async Task Someone_elses_post_cannot_be_edited()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var (otherRoot, bob) = await _harness.Persona("bob");
var created = await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "mine" }, token);
var result = await _harness.Posts.UpdatePost(otherRoot, new UpdatePostForm { AvatarId = bob.Id, PostId = ((ViewPost)created.Data).Id, Text = "yours" }, token);
Assert.False(result.IsValid);
Assert.Equal(404, result.StatusCode);
}
}
}
+146
View File
@@ -0,0 +1,146 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.ClientModels.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public sealed class RelationshipTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
[Fact]
public async Task A_block_cuts_follows_both_ways_and_hides_the_account()
{
var token = TestContext.Current.CancellationToken;
var (aliceRoot, alice) = await _harness.Persona("alice");
var (bobRoot, bob) = await _harness.Persona("bob");
await _harness.Follows.Follow(aliceRoot, new FollowForm { AvatarId = alice.Id, Target = bob.UserName }, token);
await _harness.Follows.Follow(bobRoot, new FollowForm { AvatarId = bob.Id, Target = alice.UserName }, token);
await _harness.Relationships.Block(alice, bob.Uri, bob.Id, token);
await _harness.Posts.InsertPost(bobRoot, new InsertPostForm { AvatarId = bob.Id, Text = $"hey @{alice.UserName}" }, token);
Assert.False(await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id || f.AvatarId == bob.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id || f.LocalActorId == bob.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id && e.AuthorAccountId == bob.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Notification>().Match(n => n.AvatarId == alice.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_blocked_remote_account_is_refused_when_it_follows()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await _harness.Relationships.Block(alice, mallory.Id, default, token);
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(mallory)}/follows/{Guid.NewGuid():N}", ["type"] = "Follow", ["actor"] = mallory.Id, ["object"] = alice.Uri
});
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id).ExecuteAnyAsync(token));
var sent = (await _harness.Outgoing(mallory.Id + "/inbox")).Select(a => a["type"]!.GetValue<string>()).ToList();
Assert.Equal(new[] { "Block", "Reject" }, sent.Order());
}
[Fact]
public async Task A_block_is_told_to_the_blocked_server_and_so_is_the_unblock()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(mallory)}/follows/{Guid.NewGuid():N}", ["type"] = "Follow", ["actor"] = mallory.Id, ["object"] = alice.Uri
});
await _harness.Relationships.Block(alice, mallory.Id, default, token);
await _harness.Relationships.Unblock(alice, mallory.Id, token);
var sent = await _harness.Outgoing(mallory.Id + "/inbox");
var block = Assert.Single(sent, a => a["type"]!.GetValue<string>() == "Block");
Assert.Equal(mallory.Id, block["object"]!.GetValue<string>());
Assert.Equal(alice.Uri, block["actor"]!.GetValue<string>());
var undo = Assert.Single(sent, a => a["type"]!.GetValue<string>() == "Undo");
Assert.Equal(block["id"]!.GetValue<string>(), undo["object"]!["id"]!.GetValue<string>());
Assert.False(await DB.Default.Find<Block>().Match(b => b.AvatarId == alice.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_muted_account_stays_out_of_home_and_notifications()
{
var token = TestContext.Current.CancellationToken;
var (aliceRoot, alice) = await _harness.Persona("alice");
var (bobRoot, bob) = await _harness.Persona("bob");
await _harness.Follows.Follow(aliceRoot, new FollowForm { AvatarId = alice.Id, Target = bob.UserName }, token);
await _harness.Relationships.Mute(alice, bob.Uri, bob.Id, hideNotifications: true, default, token);
await _harness.Posts.InsertPost(bobRoot, new InsertPostForm { AvatarId = bob.Id, Text = $"hey @{alice.UserName}" }, token);
Assert.False(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Notification>().Match(n => n.AvatarId == alice.Id && n.Type == NotificationType.Mention).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_report_is_forwarded_by_the_instance_not_the_persona()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var foreign = await _harness.Remote.GetActor(mallory.Id, refresh: false, token);
var report = await _harness.Reports.File(alice, foreign.ID, Array.Empty<string>(), "spam everywhere", "spam", forward: true, token);
Assert.True(report.Forwarded);
var flag = Assert.Single(await _harness.Outgoing(mallory.Id + "/inbox"));
Assert.Equal("Flag", flag["type"]!.GetValue<string>());
Assert.EndsWith("/peasants/privapub", flag["actor"]!.GetValue<string>());
Assert.DoesNotContain(alice.UserName, flag.ToJsonString());
Assert.DoesNotContain(alice.Id, flag.ToJsonString());
}
[Fact]
public async Task An_inbound_flag_becomes_a_report_for_moderators()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var reporter = new RemoteActor(_harness.Peer, "reporter");
await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "bad post" }, token);
var post = await DB.Default.Find<PrivaPub.Models.Post.Post>().Match(p => p.GroupUserId == alice.Id).ExecuteSingleAsync(token);
await _harness.Deliver(reporter, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(reporter)}/flags/{Guid.NewGuid():N}", ["type"] = "Flag", ["actor"] = reporter.Id,
["content"] = "please look", ["object"] = new JsonArray(alice.Uri, post.ObjectURI)
});
var report = await DB.Default.Find<Report>().Match(r => r.TargetAccountId == alice.Id).ExecuteSingleAsync(token);
Assert.Equal(reporter.Id, report.ReporterActorURI);
Assert.Equal(new[] { post.ID }, report.PostIds);
Assert.Equal("please look", report.Comment);
}
}
}
+162
View File
@@ -0,0 +1,162 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.ClientModels.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public sealed class TimelineTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
async Task<List<ViewPost>> Home(string root, string avatarId) =>
(List<ViewPost>)(await _harness.Timelines.Home(root, avatarId, default, 40, TestContext.Current.CancellationToken)).Data;
async Task Accepted(string root, string avatarId, RemoteActor target)
{
await _harness.Follows.Follow(root, new FollowForm { AvatarId = avatarId, Target = target.Id }, TestContext.Current.CancellationToken);
await DB.Default.Update<Following>().Match(f => f.AvatarId == avatarId && f.TargetActorURI == target.Id)
.Modify(f => f.State, FollowState.Accepted).ExecuteAsync(TestContext.Current.CancellationToken);
}
static JsonObject Create(RemoteActor author, string inReplyTo = default, string text = "hello", params (string Uri, string Name)[] mentions)
{
var origin = new Uri(author.Id).GetLeftPart(UriPartial.Authority);
var note = new JsonObject
{
["id"] = $"{origin}/notes/{Guid.NewGuid():N}",
["type"] = "Note",
["attributedTo"] = author.Id,
["content"] = $"<p>{text}</p>",
["to"] = new JsonArray(Addressing.Public),
["cc"] = new JsonArray(mentions.Select(m => (JsonNode)m.Uri).Prepend(author.Id + "/followers").ToArray()),
["tag"] = new JsonArray(mentions.Select(m => (JsonNode)new JsonObject { ["type"] = "Mention", ["href"] = m.Uri, ["name"] = m.Name }).ToArray())
};
if (inReplyTo != default)
note["inReplyTo"] = inReplyTo;
return new JsonObject { ["id"] = $"{origin}/activities/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Id, ["object"] = note };
}
[Fact]
public async Task A_local_post_reaches_the_author_and_local_followers_only()
{
var token = TestContext.Current.CancellationToken;
var (aliceRoot, alice) = await _harness.Persona("alice");
var (bobRoot, bob) = await _harness.Persona("bob");
var (carolRoot, carol) = await _harness.Persona("carol");
await _harness.Follows.Follow(bobRoot, new FollowForm { AvatarId = bob.Id, Target = alice.UserName }, token);
await _harness.Posts.InsertPost(aliceRoot, new InsertPostForm { AvatarId = alice.Id, Text = "for my followers" }, token);
Assert.Single(await Home(aliceRoot, alice.Id));
Assert.Equal("<p>for my followers</p>", Assert.Single(await Home(bobRoot, bob.Id)).ContentHtml);
Assert.Empty(await Home(carolRoot, carol.Id));
Assert.Equal(404, (await _harness.Timelines.Home(carolRoot, bob.Id, default, 20, token)).StatusCode);
}
[Fact]
public async Task A_followed_remote_account_is_kept_and_its_replies_follow_mastodons_rule()
{
var (bobRoot, bob) = await _harness.Persona("bob");
var carol = new RemoteActor(_harness.Peer, "carol");
var dave = new RemoteActor(_harness.Peer, "dave");
await Accepted(bobRoot, bob.Id, carol);
var top = Create(carol, text: "top");
await _harness.Deliver(carol, "/human-centipede", top);
await _harness.Deliver(carol, "/human-centipede", Create(carol, top["object"]!["id"]!.GetValue<string>(), "self reply"));
await _harness.Deliver(dave, "/human-centipede", Create(dave, text: "dave alone"));
var daveRoot = Create(dave, text: "unfollowed root");
await _harness.Deliver(carol, "/human-centipede", Create(carol, $"{new Uri(dave.Id).GetLeftPart(UriPartial.Authority)}/notes/unknown", "reply to a stranger"));
var home = (await Home(bobRoot, bob.Id)).Select(p => p.ContentHtml).ToList();
Assert.Contains("<p>top</p>", home);
Assert.Contains("<p>self reply</p>", home);
Assert.DoesNotContain("<p>dave alone</p>", home);
Assert.DoesNotContain("<p>reply to a stranger</p>", home);
}
[Fact]
public async Task A_reply_brings_its_public_parent_and_the_grandparent_is_queued()
{
var token = TestContext.Current.CancellationToken;
var (bobRoot, bob) = await _harness.Persona("bob");
var carol = new RemoteActor(_harness.Peer, "carol");
var dave = new RemoteActor(_harness.Peer, "dave");
await Accepted(bobRoot, bob.Id, carol);
var origin = new Uri(dave.Id).GetLeftPart(UriPartial.Authority);
var grandparentId = $"{origin}/notes/{Guid.NewGuid():N}";
var parentId = $"{origin}/notes/{Guid.NewGuid():N}";
_harness.Peer.Serve(new Uri(parentId).AbsolutePath, new JsonObject
{
["id"] = parentId, ["type"] = "Note", ["attributedTo"] = dave.Id, ["content"] = "<p>parent</p>",
["to"] = new JsonArray(Addressing.Public), ["inReplyTo"] = grandparentId
}.ToJsonString());
await _harness.Deliver(carol, "/human-centipede", Create(carol, parentId, "reply to dave"));
var parent = await DB.Default.Find<PrivaPub.Models.Post.Post>().Match(p => p.ObjectURI == parentId).ExecuteSingleAsync(token);
var reply = await DB.Default.Find<PrivaPub.Models.Post.Post>().Match(p => p.InReplyToURI == parentId).ExecuteSingleAsync(token);
Assert.Equal(parent.ID, reply.AnsweringToPostId);
Assert.Equal(parent.AuthorAccountId, reply.InReplyToAccountId);
Assert.True(await DB.Default.Find<PrivaPub.Models.Jobs.Job>().Match(j => j.DedupeKey == "ancestors|" + parent.ID).ExecuteAnyAsync(token));
Assert.DoesNotContain("<p>reply to dave</p>", (await Home(bobRoot, bob.Id)).Select(p => p.ContentHtml));
Assert.DoesNotContain("<p>parent</p>", (await Home(bobRoot, bob.Id)).Select(p => p.ContentHtml));
}
[Fact]
public async Task A_remote_delete_takes_the_post_out_of_home()
{
var (bobRoot, bob) = await _harness.Persona("bob");
var carol = new RemoteActor(_harness.Peer, "carol");
await Accepted(bobRoot, bob.Id, carol);
var create = Create(carol, text: "soon gone");
await _harness.Deliver(carol, "/human-centipede", create);
Assert.Single(await Home(bobRoot, bob.Id));
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{new Uri(carol.Id).GetLeftPart(UriPartial.Authority)}/deletes/{Guid.NewGuid():N}", ["type"] = "Delete", ["actor"] = carol.Id,
["object"] = new JsonObject { ["id"] = create["object"]!["id"]!.GetValue<string>(), ["type"] = "Tombstone" }
});
Assert.Empty(await Home(bobRoot, bob.Id));
Assert.False(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == bob.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken));
}
[Fact]
public async Task A_mention_notifies_the_persona()
{
var token = TestContext.Current.CancellationToken;
var (bobRoot, bob) = await _harness.Persona("bob");
var carol = new RemoteActor(_harness.Peer, "carol");
await _harness.Deliver(carol, "/human-centipede", Create(carol, mentions: (bob.Uri, "@" + bob.Handle)));
var notifications = (List<ViewNotification>)(await _harness.Timelines.Notifications(bobRoot, bob.Id, default, 20, token)).Data;
var mention = Assert.Single(notifications);
Assert.Equal("Mention", mention.Type);
Assert.Equal(carol.Id, mention.FromActorURI);
await _harness.Timelines.MarkNotificationsRead(bobRoot, bob.Id, token);
Assert.True(((List<ViewNotification>)(await _harness.Timelines.Notifications(bobRoot, bob.Id, default, 20, token)).Data).Single().IsRead);
}
}
}
@@ -0,0 +1,42 @@
using MongoDB.Entities;
using PrivaPub.Domain.Privacy;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Domain
{
[Trait("Category", "Integration")]
public class VisibilityPolicyTests
{
[Fact]
public async Task Who_sees_what()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var conversation = new DmGroup { Members = new() { new GroupMember { AvatarId = "alice" }, new GroupMember { AvatarId = "https://r.example/u/bob", IsForeign = true } } };
var circle = new GroupEntity { UserName = $"c{Guid.NewGuid():N}"[..20], Members = new() { new GroupMember { AvatarId = "carol" } } };
await DB.Default.SaveAsync(conversation, token);
await DB.Default.SaveAsync(circle, token);
var direct = new Post { Visibility = PostVisibility.Direct, ConversationId = conversation.ID, GroupUserId = "dave" };
var circlePost = new Post { Visibility = PostVisibility.Circle, GroupId = circle.ID };
var followersOnly = new Post { Visibility = PostVisibility.FollowersOnly, Mentions = new() { new PostMention { IsLocal = true, AccountId = "erin" } } };
var deleted = new Post { Visibility = PostVisibility.Public, DeletedAt = DateTime.UtcNow };
Assert.True(await VisibilityPolicy.CanSee(new Post(), default, token));
Assert.True(await VisibilityPolicy.CanSee(direct, "alice", token));
Assert.True(await VisibilityPolicy.CanSee(direct, "dave", token));
Assert.False(await VisibilityPolicy.CanSee(direct, "carol", token));
Assert.False(await VisibilityPolicy.CanSee(direct, default, token));
Assert.True(await VisibilityPolicy.CanSee(circlePost, "carol", token));
Assert.False(await VisibilityPolicy.CanSee(circlePost, "alice", token));
Assert.True(await VisibilityPolicy.CanSee(followersOnly, "erin", token));
Assert.False(await VisibilityPolicy.CanSee(followersOnly, "alice", token));
Assert.False(await VisibilityPolicy.CanSee(deleted, "alice", token));
}
}
}
@@ -0,0 +1,50 @@
using PrivaPub.Federation.Actors;
using System.Security.Cryptography;
using System.Text.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class ActorFeaturesTests
{
[Fact]
public void A_rich_actor_reports_the_extensions_it_uses()
{
using var rsa = RSA.Create(4096);
var actor = new JsonObject
{
["id"] = "https://social.example/users/a",
["type"] = "Person",
["featured"] = "https://social.example/users/a/featured",
["alsoKnownAs"] = new JsonArray("https://old.example/users/a"),
["endpoints"] = new JsonObject { ["sharedInbox"] = "https://social.example/inbox" },
["assertionMethod"] = new JsonArray(),
["attachment"] = new JsonArray(new JsonObject { ["type"] = "PropertyValue", ["name"] = "x", ["value"] = "y" }),
["manuallyApprovesFollowers"] = true,
["discoverable"] = false,
["indexable"] = true,
["isCat"] = true,
["_misskey_summary"] = "hi",
["publicKey"] = new JsonObject { ["id"] = "https://social.example/users/a#main-key", ["publicKeyPem"] = rsa.ExportSubjectPublicKeyInfoPem() }
};
var features = ActorFeatures.Detect(JsonDocument.Parse(actor.ToJsonString()).RootElement);
Assert.Equal(new[]
{
"featured", "also-known-as", "shared-inbox", "fep-521a-assertion-method", "property-value", "misskey", "is-cat",
"indexable", "undiscoverable", "locked", "key:rsa-4096"
}, features);
}
[Fact]
public void A_bare_actor_reports_nothing_and_junk_is_survived()
{
Assert.Empty(ActorFeatures.Detect(JsonDocument.Parse("{\"id\":\"https://a.example/u\",\"type\":\"Person\"}").RootElement));
Assert.Equal(new[] { "public-key-array" },
ActorFeatures.Detect(JsonDocument.Parse("{\"publicKey\":[{\"publicKeyPem\":\"not a key\"}]}").RootElement));
Assert.Empty(ActorFeatures.Detect(JsonDocument.Parse("[1,2]").RootElement));
}
}
}
@@ -0,0 +1,41 @@
using PrivaPub.Federation.Actors;
using System.Text.Json;
namespace PrivaPub.Tests.Federation
{
public class ActorProfileTests
{
[Fact]
public void Reads_a_mastodon_profile_with_its_header_fields_emoji_and_lock()
{
using var json = JsonDocument.Parse("""
{
"id": "https://m.example/users/ann", "type": "Person", "preferredUsername": "ann", "name": "Ann :sparkle:",
"inbox": "https://m.example/users/ann/inbox", "manuallyApprovesFollowers": true, "indexable": true, "memorial": false,
"published": "2023-04-01T00:00:00Z", "movedTo": "https://n.example/users/ann",
"icon": { "type": "Image", "url": "https://m.example/a.png", "summary": "a cat" },
"image": { "type": "Image", "url": "https://m.example/h.png", "name": "mountains" },
"tag": [{ "type": "Emoji", "name": ":sparkle:", "icon": { "type": "Image", "url": "https://m.example/emoji/sparkle.png" } }],
"attachment": [
{ "type": "PropertyValue", "name": "Site", "value": "<a href=\"https://ann.example\" rel=\"me\">ann.example</a><script>x</script>" },
{ "type": "PropertyValue", "name": "Pronouns", "value": "she/her" }
]
}
""");
var actor = ActorDocument.Parse(json.RootElement);
Assert.True(actor.Locked);
Assert.True(actor.Indexable);
Assert.Equal("https://m.example/h.png", actor.Header);
Assert.Equal("a cat", actor.IconDescription);
Assert.Equal("mountains", actor.HeaderDescription);
Assert.Equal("https://n.example/users/ann", actor.MovedTo);
Assert.Equal(new DateTime(2023, 4, 1, 0, 0, 0, DateTimeKind.Utc), actor.Published);
Assert.Equal("sparkle", Assert.Single(actor.Emojis).Shortcode);
Assert.Equal(new[] { "Site", "Pronouns" }, actor.Fields.Keys);
Assert.DoesNotContain("<script>", actor.Fields["Site"]);
}
}
}
@@ -0,0 +1,131 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class CommunityEditsTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
async Task<(RemoteActor Community, RemoteActor Poster, JsonObject Note, Post Post)> Announced()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "cats", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = community.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
var note = new JsonObject
{
["id"] = $"{Origin(poster)}/notes/{Guid.NewGuid():N}",
["type"] = "Page",
["name"] = "a title",
["attributedTo"] = poster.Id,
["content"] = "<p>first</p>",
["to"] = new JsonArray(community.Id, Addressing.Public),
["audience"] = community.Id,
["published"] = DateTime.UtcNow.AddMinutes(-5).ToString("O")
};
_harness.Peer.Serve(new Uri(note["id"]!.GetValue<string>()).AbsolutePath, note.ToJsonString());
await Announce(community, new JsonObject { ["id"] = $"{Origin(poster)}/create/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = poster.Id, ["object"] = note.DeepClone() });
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == note["id"]!.GetValue<string>()).ExecuteSingleAsync(token);
return (community, poster, note, post);
}
Task Announce(RemoteActor community, JsonObject inner) =>
_harness.Deliver(community, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(community)}/announce/{Guid.NewGuid():N}",
["type"] = "Announce",
["actor"] = community.Id,
["to"] = new JsonArray(Addressing.Public),
["object"] = inner
});
[Fact]
public async Task A_community_edit_is_an_edit_only_when_newer_and_keeps_its_history()
{
var token = TestContext.Current.CancellationToken;
var (community, poster, note, post) = await Announced();
var refresh = (JsonObject)note.DeepClone();
_harness.Peer.Serve(new Uri(note["id"]!.GetValue<string>()).AbsolutePath, refresh.ToJsonString());
await Announce(community, new JsonObject { ["id"] = $"{Origin(poster)}/update/{Guid.NewGuid():N}", ["type"] = "Update", ["actor"] = poster.Id, ["object"] = note["id"]!.GetValue<string>() });
var afterRefresh = await DB.Default.Find<Post>().OneAsync(post.ID, token);
var edit = (JsonObject)note.DeepClone();
edit["content"] = "<p>second</p>";
edit["updated"] = DateTime.UtcNow.ToString("O");
_harness.Peer.Serve(new Uri(note["id"]!.GetValue<string>()).AbsolutePath, edit.ToJsonString());
await Announce(community, new JsonObject { ["id"] = $"{Origin(poster)}/update/{Guid.NewGuid():N}", ["type"] = "Update", ["actor"] = poster.Id, ["object"] = note["id"]!.GetValue<string>() });
var afterEdit = await DB.Default.Find<Post>().OneAsync(post.ID, token);
var record = await DB.Default.Find<ObjectRecord>().Match(r => r.ObjectURI == post.ObjectURI).ExecuteSingleAsync(token);
Assert.Empty(afterRefresh.Revisions);
Assert.Null(afterRefresh.EditedAt);
Assert.Equal("<p>second</p>", afterEdit.ContentHtml);
Assert.Equal("a title", afterEdit.Title);
Assert.Single(afterEdit.Revisions);
Assert.NotNull(afterEdit.EditedAt);
Assert.Equal(2, record.Revisions.Count);
Assert.Equal(new[] { "refresh", "edit" }, _harness.Ledger.Of("in").Where(e => e.Activity == "Announce" && e.Object == "Update").Select(e => e.Reason));
}
[Fact]
public async Task A_community_delete_leaves_a_tombstone_and_no_trace()
{
var token = TestContext.Current.CancellationToken;
var (community, poster, note, post) = await Announced();
_harness.Peer.Serve(new Uri(note["id"]!.GetValue<string>()).AbsolutePath, new JsonObject { ["id"] = note["id"]!.GetValue<string>(), ["type"] = "Tombstone" }.ToJsonString());
await Announce(community, new JsonObject { ["id"] = $"{Origin(poster)}/delete/{Guid.NewGuid():N}", ["type"] = "Delete", ["actor"] = poster.Id, ["object"] = note["id"]!.GetValue<string>() });
Assert.False(await DB.Default.Find<Post>().Match(p => p.ID == post.ID).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<ObjectRecord>().Match(r => r.ObjectURI == post.ObjectURI).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<TimelineEntry>().Match(e => e.PostId == post.ID).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<DeletedObject>().Match(d => d.ObjectURI == post.ObjectURI).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_fetched_object_does_not_wear_the_signature_of_the_activity_that_caused_the_fetch()
{
var token = TestContext.Current.CancellationToken;
var (_, _, _, post) = await Announced();
var record = await DB.Default.Find<ObjectRecord>().Match(r => r.ObjectURI == post.ObjectURI).ExecuteSingleAsync(token);
Assert.Equal(ObjectPath.Fetched, record.Path);
Assert.Null(record.KeyId);
Assert.Null(record.SignatureScheme);
Assert.Empty(record.SignedHeaders);
Assert.Null(record.ActivityContext);
Assert.Equal("Announce", record.ActivityType);
Assert.InRange(record.ReceivedAt, DateTime.UtcNow.AddMinutes(-1), DateTime.UtcNow.AddSeconds(1));
Assert.Contains("fep-1b12-audience", record.Extensions);
Assert.NotNull(record.ContextNamespaces);
}
}
}
@@ -0,0 +1,37 @@
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Federation.Moderation;
using PrivaPub.Models.Federation;
namespace PrivaPub.Tests.Federation
{
public class DomainBlocksTests
{
[Fact]
public void A_block_covers_subdomains_but_not_lookalikes()
{
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
blocks.Load(new[] { new DomainBlock { Domain = "evil.example", Severity = DomainBlockSeverity.Silence } });
Assert.NotNull(blocks.Find("evil.example"));
Assert.NotNull(blocks.Find("A.Evil.Example."));
Assert.Null(blocks.Find("notevil.example"));
Assert.False(blocks.IsSuspended("evil.example"));
}
[Fact]
public void A_suspension_is_found_from_any_subdomain()
{
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
blocks.Load(new[]
{
new DomainBlock { Domain = " Spam.Example. ", Severity = DomainBlockSeverity.Suspend },
new DomainBlock { Domain = "", Severity = DomainBlockSeverity.Suspend }
});
Assert.True(blocks.IsSuspended("deep.inside.spam.example"));
Assert.False(blocks.IsSuspended("example"));
Assert.Null(blocks.Find(default));
}
}
}
+198
View File
@@ -0,0 +1,198 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Social;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Rendering;
using PrivaPub.Federation.Signing;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class GroupTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
async Task<(GroupEntity Entity, PrivaPub.Federation.Actors.LocalActor Actor)> Group(GroupKind kind, PostingPolicy policy, string ownerId, params string[] remoteMembers)
{
var (privateKey, publicKey) = PrivaPub.Federation.Actors.Keys.NewKeyPair();
var group = new GroupEntity
{
UserName = $"{kind}{Guid.NewGuid():N}"[..20].ToLowerInvariant(),
Kind = kind,
PostingPolicy = policy,
PrivateKey = privateKey,
PublicKey = publicKey,
Members = new() { new GroupMember { AvatarId = ownerId, Role = GroupRole.Owner } }
};
group.Members.AddRange(remoteMembers.Select(m => new GroupMember { AvatarId = m, IsForeign = true }));
await DB.Default.SaveAsync(group);
return (group, _harness.Local.FromGroup(group));
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static JsonObject Create(RemoteActor author, IEnumerable<string> to, string audience = default)
{
var note = new JsonObject
{
["id"] = $"{Origin(author)}/notes/{Guid.NewGuid():N}",
["type"] = "Note",
["attributedTo"] = author.Id,
["content"] = "<p>to the group</p>",
["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray())
};
if (audience != default)
note["audience"] = audience;
return new JsonObject { ["id"] = $"{Origin(author)}/activities/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Id, ["object"] = note };
}
[Fact]
public async Task A_follower_posting_to_a_community_is_announced_activity_and_object()
{
var token = TestContext.Current.CancellationToken;
var (_, owner) = await _harness.Persona("owner");
var (_, community) = await Group(GroupKind.Community, PostingPolicy.Followers, owner.Id);
var lemmy = new RemoteActor(_harness.Peer, "lemmy");
var stranger = new RemoteActor(_harness.Peer, "stranger");
await _harness.FollowedBy(community, lemmy);
await _harness.Deliver(lemmy, "/human-centipede", Create(lemmy, new[] { community.Uri, Addressing.Public }, community.Uri));
await _harness.Deliver(stranger, "/human-centipede", Create(stranger, new[] { community.Uri, Addressing.Public }, community.Uri));
Assert.True(await DB.Default.Find<Post>().Match(p => p.GroupId == community.Id && p.ActorURI == lemmy.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Post>().Match(p => p.GroupId == community.Id && p.ActorURI == stranger.Id).ExecuteAnyAsync(token));
var announces = (await _harness.Outgoing(lemmy.SharedInbox)).Where(a => a["actor"]!.GetValue<string>() == community.Uri).ToList();
Assert.Equal(2, announces.Count);
Assert.Contains(announces, a => a["object"] is JsonObject inner && inner["type"]!.GetValue<string>() == "Create" && a["audience"]!.GetValue<string>() == community.Uri);
Assert.Contains(announces, a => a["object"] is JsonValue);
}
[Fact]
public async Task An_open_community_takes_posts_from_anyone()
{
var token = TestContext.Current.CancellationToken;
var (_, owner) = await _harness.Persona("owner");
var (_, community) = await Group(GroupKind.Community, PostingPolicy.Anyone, owner.Id);
var stranger = new RemoteActor(_harness.Peer, "stranger");
await _harness.Deliver(stranger, "/human-centipede", Create(stranger, new[] { community.Uri, Addressing.Public }));
Assert.True(await DB.Default.Find<Post>().Match(p => p.GroupId == community.Id && p.ActorURI == stranger.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task Mentioning_a_community_posts_into_it_as_a_titled_page()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, community) = await Group(GroupKind.Community, PostingPolicy.Anyone, alice.Id);
var follower = new RemoteActor(_harness.Peer, "follower");
await _harness.FollowedBy(community, follower);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = $"@{community.UserName} a new thread", PlainText = true }, token);
Assert.Equal(community.Id, outcome.Post.GroupId);
var note = ActivityPubRenderer.Note(outcome.Post, alice, community, default);
Assert.Equal("Page", note["type"]!.GetValue<string>());
Assert.Equal(community.Uri, note["audience"]!.GetValue<string>());
Assert.False(string.IsNullOrEmpty(note["name"]!.GetValue<string>()));
Assert.Contains(await _harness.Outgoing(follower.SharedInbox), a => a["type"]!.GetValue<string>() == "Announce");
}
[Fact]
public async Task A_circle_post_goes_only_to_members_and_only_members_may_read_it()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var member = new RemoteActor(_harness.Peer, "member");
var outsider = new RemoteActor(_harness.Peer, "outsider");
var (circleEntity, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id, member.Id);
await _harness.Remote.GetActor(member.Id, refresh: false, token);
var follower = new RemoteActor(_harness.Peer, "follower");
await _harness.FollowedBy(alice, follower);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
Assert.Equal(PostVisibility.Circle, outcome.Post.Visibility);
var create = Assert.Single(await _harness.Outgoing(member.Id + "/inbox"));
Assert.Equal(new[] { circle.Uri, circle.Flock }, create["object"]!["to"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.DoesNotContain(Addressing.Public, create.ToJsonString());
Assert.Empty(await _harness.Outgoing(follower.SharedInbox));
Assert.Empty((await _harness.Outgoing(member.SharedInbox)).Where(a => a["type"]!.GetValue<string>() == "Announce"));
var authorizer = new SignedFetchAuthorizer(_harness.Remote);
var path = new Uri(outcome.Post.ObjectURI).AbsolutePath;
var asMember = await authorizer.Requester(member.Get(Harness.Host, path), token);
var asOutsider = await authorizer.Requester(outsider.Get(Harness.Host, path), token);
Assert.True(SignedFetchAuthorizer.MayReadCircle(circleEntity, asMember));
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, asOutsider));
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, default));
Assert.True(circle.IsCircle);
Assert.False(circle.Discoverable);
}
[Fact]
public async Task Only_circle_members_can_post_into_a_circle()
{
var token = TestContext.Current.CancellationToken;
var (aliceRoot, alice) = await _harness.Persona("alice");
var member = new RemoteActor(_harness.Peer, "member");
var outsider = new RemoteActor(_harness.Peer, "outsider");
var (_, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id, member.Id);
await _harness.Deliver(member, "/human-centipede", Create(member, new[] { circle.Uri, circle.Flock }));
await _harness.Deliver(outsider, "/human-centipede", Create(outsider, new[] { circle.Uri, circle.Flock }));
var stored = await DB.Default.Find<Post>().Match(p => p.GroupId == circle.Id).ExecuteAsync(token);
Assert.Equal(member.Id, Assert.Single(stored).ActorURI);
Assert.Equal(PostVisibility.Circle, stored[0].Visibility);
Assert.True(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id && e.PostId == stored[0].ID).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_followed_remote_community_announcing_a_post_puts_it_in_home()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var lemmyCommunity = new RemoteActor(_harness.Peer, "cats", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = lemmyCommunity.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
var create = Create(poster, new[] { lemmyCommunity.Id, Addressing.Public }, lemmyCommunity.Id);
var noteId = create["object"]!["id"]!.GetValue<string>();
_harness.Peer.Serve(new Uri(noteId).AbsolutePath, create["object"]!.ToJsonString());
await _harness.Deliver(lemmyCommunity, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(lemmyCommunity)}/activities/announce/{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = lemmyCommunity.Id,
["to"] = new JsonArray(Addressing.Public), ["object"] = create
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteSingleAsync(token);
Assert.Equal(lemmyCommunity.Id, post.AudienceURI);
Assert.True(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id && e.PostId == post.ID).ExecuteAnyAsync(token));
}
}
}
@@ -0,0 +1,139 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class InboundRoutingTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task<Post> LocalPost(string text)
{
var (root, alice) = await _harness.Persona("alice");
await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = text }, TestContext.Current.CancellationToken);
return await DB.Default.Find<Post>().Match(p => p.GroupUserId == alice.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken);
}
[Fact]
public async Task A_downvote_counts_and_its_undo_takes_it_back()
{
var token = TestContext.Current.CancellationToken;
var post = await LocalPost("vote on me");
var lemmy = new RemoteActor(_harness.Peer, "voter");
var dislike = new JsonObject { ["id"] = NewId(lemmy, "activities/dislike"), ["type"] = "Dislike", ["actor"] = lemmy.Id, ["object"] = post.ObjectURI };
await _harness.Deliver(lemmy, "/human-centipede", dislike);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).DownvotesCount);
await _harness.Deliver(lemmy, "/human-centipede", new JsonObject
{
["id"] = NewId(lemmy, "activities/undo"), ["type"] = "Undo", ["actor"] = lemmy.Id, ["object"] = dislike.DeepClone()
});
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).DownvotesCount);
}
[Fact]
public async Task A_delete_that_arrives_first_keeps_the_post_from_coming_back()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var noteId = NewId(mallory, "notes");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities/delete"), ["type"] = "Delete", ["actor"] = mallory.Id,
["object"] = new JsonObject { ["id"] = noteId, ["type"] = "Tombstone" }
});
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities/create"), ["type"] = "Create", ["actor"] = mallory.Id,
["object"] = new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = mallory.Id, ["content"] = "<p>late</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(alice.Uri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" })
}
});
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_lemmy_private_message_arrives_as_a_direct_message()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var lemmy = new RemoteActor(_harness.Peer, "pm");
var messageId = NewId(lemmy, "private_message");
await _harness.Deliver(lemmy, "/human-centipede", new JsonObject
{
["id"] = NewId(lemmy, "activities/create"), ["type"] = "Create", ["actor"] = lemmy.Id, ["to"] = new JsonArray(alice.Uri),
["object"] = new JsonObject
{
["id"] = messageId, ["type"] = "ChatMessage", ["attributedTo"] = lemmy.Id, ["to"] = new JsonArray(alice.Uri),
["content"] = "<p>psst</p>", ["mediaType"] = "text/html", ["published"] = DateTime.UtcNow.ToString("O")
}
});
var message = await DB.Default.Find<Post>().Match(p => p.ObjectURI == messageId).ExecuteFirstAsync(token);
Assert.NotNull(message);
Assert.Equal(PostVisibility.Direct, message.Visibility);
Assert.NotNull(message.ConversationId);
}
[Fact]
public async Task An_unreachable_key_asks_the_sender_to_retry_instead_of_refusing()
{
var (_, alice) = await _harness.Persona("alice");
var offline = new RemoteActor(_harness.Peer, "offline", origin: "http://127.0.0.1:9");
var result = await _harness.Deliver(offline, "/human-centipede", new JsonObject
{
["id"] = NewId(offline, "follows"), ["type"] = "Follow", ["actor"] = offline.Id, ["object"] = alice.Uri
});
Assert.Equal(503, result.StatusCode);
Assert.Equal(300, result.RetryAfterSeconds);
}
[Fact]
public async Task A_join_of_a_local_post_is_answered_with_ignore()
{
var post = await LocalPost("not an event");
var mobilizon = new RemoteActor(_harness.Peer, "joiner");
await _harness.Deliver(mobilizon, "/human-centipede", new JsonObject
{
["id"] = NewId(mobilizon, "join"), ["type"] = "Join", ["actor"] = mobilizon.Id, ["object"] = post.ObjectURI
});
var answer = Assert.Single(await _harness.Outgoing(mobilizon.Id + "/inbox"));
Assert.Equal("Ignore", answer["type"]!.GetValue<string>());
}
}
}
+151 -25
View File
@@ -3,13 +3,22 @@ using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Domain.Timelines;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Moderation;
using PrivaPub.Domain.Social;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Models;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Models;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
@@ -27,7 +36,9 @@ namespace PrivaPub.Tests.Federation
Peer _peer;
LocalActorService _local;
InboxService _inbox;
InboxReceiver _receiver;
InboxProcessor _processor;
DomainBlocks _blocks;
public async ValueTask InitializeAsync()
{
@@ -36,7 +47,21 @@ namespace PrivaPub.Tests.Federation
var cache = new MemoryCache(new MemoryCacheOptions());
_local = new LocalActorService(new DbEntities(), new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = Base }));
var remote = new RemoteActorService(Peer.Http(cache), _local, cache, new DbEntities());
_inbox = new InboxService(new DbEntities(), _local, remote, new DeliveryService(new DbEntities()), NullLogger<InboxService>.Instance);
var queue = new JobQueue();
var delivery = new DeliveryService(new DbEntities(), queue);
var db = new DbEntities();
_blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
_receiver = new InboxReceiver(_local, remote, queue, _blocks, NullLogger<InboxReceiver>.Instance);
var remotePosts = new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue), new NoPreviews());
var quotes = new QuoteService(db, remote, remotePosts, _local, delivery, new OutboxPublisher(db, _local, delivery));
_processor = new InboxProcessor(remote, new IActivityHandler[]
{
new FollowHandler(db, _local, remote, delivery),
new UndoHandler(db, _local, new Reactions(db, delivery)),
new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), remotePosts, new GroupDistributor(delivery), new ObjectRecords(queue), new PollService(db, _local, delivery, queue), new NoPreviews(), quotes),
new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery), quotes),
new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue), quotes)
}, NullLogger<InboxProcessor>.Instance);
}
public async ValueTask DisposeAsync()
@@ -77,6 +102,17 @@ namespace PrivaPub.Tests.Federation
};
}
async Task<InboxResult> Deliver(RemoteActor sender, string path, JsonNode activity)
{
var token = TestContext.Current.CancellationToken;
var result = await _receiver.Receive(sender.Post(Host, path, activity), default, token);
var dedupe = "inbox|" + (activity is JsonObject ? activity["id"]?.GetValue<string>() : default);
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == dedupe).ExecuteFirstAsync(token);
if (job != default)
Assert.Equal(JobResult.Done, (await _processor.Handle(job, token)).Result);
return result;
}
static string Origin(string uri) => new Uri(uri).GetLeftPart(UriPartial.Authority);
[Fact]
@@ -88,15 +124,15 @@ namespace PrivaPub.Tests.Federation
var mallory = new RemoteActor(_peer, "mallory");
var context = $"{_peer.A}/contexts/{Guid.NewGuid():N}";
var first = await _inbox.Receive(bob.Post(Host, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri, context)), alice, token);
var injected = await _inbox.Receive(mallory.Post(Host, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, context)), alice, token);
var first = await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri, context));
var injected = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, context));
Assert.Equal(202, first.StatusCode);
Assert.Equal(202, injected.StatusCode);
var bobDm = await DB.Default.Find<DmPost>().Match(p => p.ActorURI == bob.Id).ExecuteSingleAsync(token);
var malloryDm = await DB.Default.Find<DmPost>().Match(p => p.ActorURI == mallory.Id).ExecuteSingleAsync(token);
Assert.NotEqual(bobDm.GroupId, malloryDm.GroupId);
var bobConversation = await DB.Default.Find<DmGroup>().OneAsync(bobDm.GroupId, token);
var bobDm = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id && p.Visibility == PostVisibility.Direct).ExecuteSingleAsync(token);
var malloryDm = await DB.Default.Find<Post>().Match(p => p.ActorURI == mallory.Id).ExecuteSingleAsync(token);
Assert.NotEqual(bobDm.ConversationId, malloryDm.ConversationId);
var bobConversation = await DB.Default.Find<DmGroup>().OneAsync(bobDm.ConversationId, token);
Assert.DoesNotContain(bobConversation.Members, m => m.AvatarId == mallory.Id);
}
@@ -107,12 +143,12 @@ namespace PrivaPub.Tests.Federation
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob");
await _inbox.Receive(bob.Post(Host, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri)), alice, token);
await _inbox.Receive(bob.Post(Host, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri)), alice, token);
await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri));
await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri));
var dms = await DB.Default.Find<DmPost>().Match(p => p.ActorURI == bob.Id).ExecuteAsync(token);
var dms = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id && p.Visibility == PostVisibility.Direct).ExecuteAsync(token);
Assert.Equal(2, dms.Count);
Assert.Single(dms.Select(d => d.GroupId).Distinct());
Assert.Single(dms.Select(d => d.ConversationId).Distinct());
}
[Fact]
@@ -124,7 +160,7 @@ namespace PrivaPub.Tests.Federation
var create = DirectCreate(mallory, alice.Uri);
create["id"] = $"{_peer.B}/activities/{Guid.NewGuid():N}";
var result = await _inbox.Receive(mallory.Post(Host, $"/peasants/{alice.UserName}/mouth", create), alice, token);
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", create);
Assert.Equal(400, result.StatusCode);
}
@@ -137,11 +173,10 @@ namespace PrivaPub.Tests.Federation
var mallory = new RemoteActor(_peer, "mallory");
var victim = new RemoteActor(_peer, "victim");
var result = await _inbox.Receive(mallory.Post(Host, $"/peasants/{alice.UserName}/mouth",
DirectCreate(mallory, alice.Uri, attributedTo: victim.Id)), alice, token);
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, attributedTo: victim.Id));
Assert.Equal(400, result.StatusCode);
Assert.False(await DB.Default.Find<DmPost>().Match(p => p.ActorURI == victim.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == victim.Id).ExecuteAnyAsync(token));
}
[Fact]
@@ -151,11 +186,97 @@ namespace PrivaPub.Tests.Federation
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
var result = await _inbox.Receive(mallory.Post(Host, $"/peasants/{alice.UserName}/mouth",
DirectCreate(mallory, alice.Uri, objectOrigin: _peer.B)), alice, token);
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, objectOrigin: _peer.B));
Assert.Equal(202, result.StatusCode);
Assert.False(await DB.Default.Find<DmPost>().Match(p => p.ActorURI == mallory.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == mallory.Id).ExecuteAnyAsync(token));
}
static JsonObject PublicCreate(RemoteActor author, string inReplyTo = default, params string[] cc)
{
var id = $"{Origin(author.Id)}/notes/{Guid.NewGuid():N}";
var note = new JsonObject
{
["id"] = id,
["type"] = "Note",
["attributedTo"] = author.Id,
["content"] = "<p>hello</p>",
["to"] = new JsonArray(Addressing.Public),
["cc"] = new JsonArray(cc.Prepend(author.Id + "/followers").Select(c => (JsonNode)c).ToArray())
};
if (inReplyTo != default)
note["inReplyTo"] = inReplyTo;
return new JsonObject
{
["id"] = $"{Origin(author.Id)}/activities/{Guid.NewGuid():N}",
["type"] = "Create",
["actor"] = author.Id,
["object"] = note
};
}
[Fact]
public async Task A_public_reply_to_a_local_post_is_kept_and_counted()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var parent = new Post { GroupUserId = alice.Id, AuthorAccountId = alice.Id, ActorURI = alice.Uri, Text = "hi" };
parent.ID = (string)parent.GenerateNewID();
parent.ObjectURI = alice.PostUri(parent.ID);
await DB.Default.SaveAsync(parent, token);
var bob = new RemoteActor(_peer, "bob");
await Deliver(bob, "/human-centipede", PublicCreate(bob, parent.ObjectURI));
var reply = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteSingleAsync(token);
Assert.Equal(PostVisibility.Public, reply.Visibility);
Assert.Equal(parent.ID, reply.AnsweringToPostId);
Assert.Equal(alice.Id, reply.InReplyToAccountId);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(parent.ID, token)).RepliesCount);
}
[Fact]
public async Task A_public_post_nobody_here_asked_for_is_dropped()
{
var token = TestContext.Current.CancellationToken;
var bob = new RemoteActor(_peer, "bob");
await Deliver(bob, "/human-centipede", PublicCreate(bob));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_mention_is_kept_and_linked_to_the_local_persona()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob");
var create = PublicCreate(bob, default, alice.Uri);
create["object"]!["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@" + alice.Handle });
await Deliver(bob, "/human-centipede", create);
var post = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteSingleAsync(token);
var mention = Assert.Single(post.Mentions);
Assert.True(mention.IsLocal);
Assert.Equal(alice.Id, mention.AccountId);
}
[Fact]
public async Task A_suspended_domain_is_dropped_before_its_key_is_fetched()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob", _peer.B);
_blocks.Load(new[] { new DomainBlock { Domain = "localhost", Severity = DomainBlockSeverity.Suspend } });
var before = _peer.Requests.Count;
var result = await Deliver(bob, $"/peasants/{alice.UserName}/mouth", DirectCreate(bob, alice.Uri));
Assert.Equal(202, result.StatusCode);
Assert.Equal(before, _peer.Requests.Count);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteAnyAsync(token));
}
[Fact]
@@ -167,7 +288,7 @@ namespace PrivaPub.Tests.Federation
var request = mallory.Post(Host, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri));
request.Headers["Signature"] = request.Headers["Signature"].ToString().Replace("signature=\"", "signature=\"AAAA");
Assert.Equal(401, (await _inbox.Receive(request, alice, token)).StatusCode);
Assert.Equal(401, (await _receiver.Receive(request, alice, token)).StatusCode);
}
[Fact]
@@ -178,11 +299,11 @@ namespace PrivaPub.Tests.Federation
var mallory = new RemoteActor(_peer, "mallory");
foreach (var junk in new JsonNode[] { new JsonArray(1, 2), JsonValue.Create("x"), new JsonObject { ["type"] = "Create" } })
Assert.Equal(400, (await _inbox.Receive(mallory.Post(Host, $"/peasants/{alice.UserName}/mouth", junk), alice, token)).StatusCode);
Assert.Equal(400, (await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", junk)).StatusCode);
}
[Fact]
public async Task A_circle_is_not_a_federated_actor()
public async Task A_circle_only_takes_follow_requests()
{
var token = TestContext.Current.CancellationToken;
var (privateKey, publicKey) = Keys.NewKeyPair();
@@ -198,8 +319,13 @@ namespace PrivaPub.Tests.Federation
["object"] = actor.Uri
};
Assert.False(actor.IsFederated);
Assert.Equal(404, (await _inbox.Receive(bob.Post(Host, "/human-centipede", follow), default, token)).StatusCode);
Assert.True(actor.IsCircle);
Assert.True(actor.ManuallyApprovesFollowers);
Assert.False(actor.Discoverable);
Assert.Equal(202, (await Deliver(bob, "/human-centipede", follow)).StatusCode);
var request = await DB.Default.Find<Follower>().Match(f => f.LocalActorId == circle.ID).ExecuteSingleAsync(token);
Assert.False(request.IsAccepted);
Assert.DoesNotContain(circle.Members, m => m.AvatarId == bob.Id);
}
}
}
@@ -0,0 +1,137 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.ClientModels.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class InteractionTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
async Task<(string Root, Post Post)> LocalPost(string visibility = default)
{
var (root, alice) = await _harness.Persona("alice");
var view = (ViewPost)(await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "like me", Visibility = visibility },
TestContext.Current.CancellationToken)).Data;
return (root, await DB.Default.Find<Post>().OneAsync(view.Id));
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
[Fact]
public async Task A_like_counts_once_notifies_and_can_be_undone()
{
var token = TestContext.Current.CancellationToken;
var (_, post) = await LocalPost();
var carol = new RemoteActor(_harness.Peer, "carol");
var like = new JsonObject { ["id"] = $"{Origin(carol)}/likes/{Guid.NewGuid():N}", ["type"] = "Like", ["actor"] = carol.Id, ["object"] = post.ObjectURI };
await _harness.Deliver(carol, "/human-centipede", like);
await _harness.Deliver(carol, "/human-centipede", new JsonObject { ["id"] = $"{Origin(carol)}/likes/{Guid.NewGuid():N}", ["type"] = "Like", ["actor"] = carol.Id, ["object"] = post.ObjectURI });
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
Assert.Equal(NotificationType.Favourite, (await DB.Default.Find<Notification>().Match(n => n.AvatarId == post.GroupUserId).ExecuteSingleAsync(token)).Type);
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/undos/{Guid.NewGuid():N}", ["type"] = "Undo", ["actor"] = carol.Id, ["object"] = like
});
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
}
[Fact]
public async Task A_stranger_cannot_like_a_followers_only_post()
{
var token = TestContext.Current.CancellationToken;
var (_, post) = await LocalPost("followersonly");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(mallory)}/likes/{Guid.NewGuid():N}", ["type"] = "Like", ["actor"] = mallory.Id, ["object"] = post.ObjectURI
});
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
}
[Fact]
public async Task A_boost_of_a_local_post_counts_notifies_and_can_be_undone()
{
var token = TestContext.Current.CancellationToken;
var (_, post) = await LocalPost();
var carol = new RemoteActor(_harness.Peer, "carol");
var announce = new JsonObject
{
["id"] = $"{Origin(carol)}/announces/{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = carol.Id, ["object"] = post.ObjectURI,
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(carol.Id + "/followers")
};
await _harness.Deliver(carol, "/human-centipede", announce);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).ReblogsCount);
Assert.Equal(NotificationType.Reblog, (await DB.Default.Find<Notification>().Match(n => n.AvatarId == post.GroupUserId).ExecuteSingleAsync(token)).Type);
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/undos/{Guid.NewGuid():N}", ["type"] = "Undo", ["actor"] = carol.Id, ["object"] = announce["id"]!.GetValue<string>()
});
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).ReblogsCount);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == announce["id"]!.GetValue<string>()).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_followed_account_boosting_a_remote_post_puts_it_in_home()
{
var token = TestContext.Current.CancellationToken;
var (bobRoot, bob) = await _harness.Persona("bob");
var carol = new RemoteActor(_harness.Peer, "carol");
var dave = new RemoteActor(_harness.Peer, "dave");
var stranger = new RemoteActor(_harness.Peer, "stranger");
await _harness.Follows.Follow(bobRoot, new FollowForm { AvatarId = bob.Id, Target = carol.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == bob.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
var noteId = $"{Origin(dave)}/notes/{Guid.NewGuid():N}";
_harness.Peer.Serve(new Uri(noteId).AbsolutePath, new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = dave.Id, ["content"] = "<p>worth sharing</p>", ["to"] = new JsonArray(Addressing.Public)
}.ToJsonString());
await _harness.Deliver(stranger, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(stranger)}/announces/{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = stranger.Id, ["object"] = noteId,
["to"] = new JsonArray(Addressing.Public)
});
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteAnyAsync(token));
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/announces/{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = carol.Id, ["object"] = noteId,
["to"] = new JsonArray(Addressing.Public)
});
var home = (List<ViewPost>)(await _harness.Timelines.Home(bobRoot, bob.Id, default, 20, token)).Data;
var boosted = Assert.Single(home);
Assert.Equal("<p>worth sharing</p>", boosted.ReblogOf.ContentHtml);
Assert.Equal(carol.Id, boosted.AuthorActorURI);
}
}
}
@@ -0,0 +1,137 @@
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class NoteParserTests
{
const string MastodonNote = """
{
"id": "https://m.example/users/alice/statuses/1",
"type": "Note",
"summary": "spoilers",
"inReplyTo": "https://privapub.test/peasants/bob/posts/abc",
"published": "2026-09-30T10:00:00Z",
"url": "https://m.example/@alice/1",
"attributedTo": "https://m.example/users/alice",
"to": ["https://www.w3.org/ns/activitystreams#Public"],
"cc": ["https://m.example/users/alice/followers", "https://privapub.test/peasants/bob"],
"sensitive": true,
"conversation": "tag:m.example,2026-09-30:objectId=1:objectType=Conversation",
"context": "https://m.example/contexts/1",
"content": "<p><span class=\"h-card\"><a href=\"https://privapub.test/peasants/bob\" class=\"u-url mention\">@<span>bob</span></a></span> hi <a href=\"https://m.example/tags/Fedi\" class=\"mention hashtag\" rel=\"tag\">#<span>Fedi</span></a><script>x</script></p>",
"contentMap": { "en": "<p>hi</p>" },
"attachment": [{
"type": "Document", "mediaType": "image/png", "url": "https://files.m.example/a.png",
"name": "a cat", "blurhash": "UBL_:rOpGG-oBUNG,qRj2so|=eE1w^n4S5NH", "focalPoint": [0.5, -2.0], "width": 640, "height": 480
}],
"tag": [
{ "type": "Mention", "href": "https://privapub.test/peasants/bob", "name": "@bob@privapub.test" },
{ "type": "Hashtag", "href": "https://m.example/tags/fedi", "name": "#Fedi" },
{ "type": "Emoji", "name": ":blob:", "icon": { "type": "Image", "url": "https://files.m.example/blob.png" } }
]
}
""";
[Fact]
public void Reads_a_mastodon_note()
{
var note = NoteParser.Parse(JsonNode.Parse(MastodonNote));
Assert.Equal("https://m.example/users/alice", note.AttributedTo);
Assert.Equal("spoilers", note.SpoilerText);
Assert.True(note.Sensitive);
Assert.Equal("en", note.Language);
Assert.Equal("https://m.example/@alice/1", note.Url);
Assert.Equal("https://m.example/contexts/1", note.Context);
Assert.DoesNotContain("<script>", note.ContentHtml);
Assert.Contains("u-url mention", note.ContentHtml);
Assert.Equal("https://privapub.test/peasants/bob", Assert.Single(note.Mentions).ActorURI);
Assert.Equal("fedi", Assert.Single(note.Tags));
var media = Assert.Single(note.Attachments);
Assert.Equal("a cat", media.Description);
Assert.Equal(new[] { 0.5f, -1f }, media.Focus);
Assert.Equal(new DateTime(2026, 9, 30, 10, 0, 0, DateTimeKind.Utc), note.Published);
Assert.Null(note.Updated);
}
[Fact]
public void Reads_a_lemmy_page_with_a_title()
{
var note = NoteParser.Parse(JsonNode.Parse("""
{
"id": "https://l.example/post/7", "type": "Page", "name": "A title",
"attributedTo": "https://l.example/u/carol", "audience": "https://l.example/c/cats",
"content": "<p>body</p>", "to": ["https://l.example/c/cats", "https://www.w3.org/ns/activitystreams#Public"],
"published": "2026-09-30T10:00:00.123456+00:00"
}
"""));
Assert.Equal("A title", note.Title);
Assert.Equal("https://l.example/c/cats", note.Audience);
Assert.False(note.Sensitive);
}
[Fact]
public void Falls_back_to_misskey_content_and_takes_the_person_from_a_peertube_attribution()
{
var note = NoteParser.Parse(JsonNode.Parse("""
{
"id": "https://k.example/notes/9", "type": "Note",
"attributedTo": [{ "type": "Group", "id": "https://k.example/c/chan" }, { "type": "Person", "id": "https://k.example/users/9" }],
"_misskey_content": "hello <b>there</b>\nnext", "_misskey_quote": "https://k.example/notes/8"
}
"""));
Assert.Equal("https://k.example/users/9", note.AttributedTo);
Assert.Equal("hello &lt;b&gt;there&lt;/b&gt;<br>next", note.ContentHtml);
Assert.Equal("https://k.example/notes/8", note.QuoteUri);
}
[Theory]
[InlineData("Article", false, false)]
[InlineData("Event", false, false)]
[InlineData("Video", false, false)]
[InlineData("Page", false, false)]
[InlineData("Note", false, true)]
[InlineData("Question", false, true)]
[InlineData("Article", true, true)]
public void A_summary_warns_only_on_notes_or_when_marked_sensitive(string type, bool sensitive, bool warns)
{
var note = NoteParser.Parse(JsonNode.Parse($$"""
{
"id": "https://w.example/?p=12", "type": "{{type}}", "name": "Twelve", "summary": "Saturday, 10:00 at the library",
"sensitive": {{(sensitive ? "true" : "false")}}, "attributedTo": "https://w.example/author", "content": "<p>body</p>"
}
"""));
Assert.Equal(warns, note.Sensitive);
Assert.Equal(warns ? "Saturday, 10:00 at the library" : null, note.SpoilerText);
Assert.Equal(warns ? null : "Saturday, 10:00 at the library", note.Excerpt);
}
[Theory]
[InlineData("""{ "id": "https://x.example/1", "type": "Person" }""")]
[InlineData("""{ "id": "not a uri", "type": "Note" }""")]
[InlineData("""[1, 2]""")]
public void Refuses_what_is_not_a_note(string json) =>
Assert.Null(NoteParser.Parse(JsonNode.Parse(json)));
}
public class AddressingTests
{
const string Followers = "https://m.example/users/alice/followers";
[Theory]
[InlineData(new[] { Addressing.Public }, new[] { Followers }, PostVisibility.Public)]
[InlineData(new[] { "as:Public" }, new string[0], PostVisibility.Public)]
[InlineData(new[] { Followers }, new[] { Addressing.Public }, PostVisibility.Unlisted)]
[InlineData(new[] { Followers }, new[] { "https://privapub.test/peasants/bob" }, PostVisibility.FollowersOnly)]
[InlineData(new[] { "https://privapub.test/peasants/bob" }, new string[0], PostVisibility.Direct)]
[InlineData(new[] { "https://m.example/users/mallory/followers" }, new string[0], PostVisibility.Direct)]
public void Classifies_like_mastodon(string[] to, string[] cc, PostVisibility expected) =>
Assert.Equal(expected, Addressing.Classify(to, cc, Followers));
}
}
@@ -0,0 +1,36 @@
using PrivaPub.Federation.Objects;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class ObjectRecordsTests
{
[Fact]
public void An_object_larger_than_the_cap_keeps_only_its_hash()
{
var big = new JsonObject { ["content"] = new string('x', ObjectRecords.MaxRawBytes + 1) };
var (text, hash, bytes, truncated) = ObjectRecords.Capture(big);
Assert.Null(text);
Assert.True(truncated);
Assert.True(bytes > ObjectRecords.MaxRawBytes);
Assert.Equal(64, hash.Length);
}
[Fact]
public void A_small_object_is_kept_whole_with_its_hash()
{
var small = new JsonObject { ["type"] = "Note", ["content"] = "hi" };
var (text, hash, bytes, truncated) = ObjectRecords.Capture(small);
Assert.Equal(small.ToJsonString(), text);
Assert.False(truncated);
Assert.Equal(text.Length, bytes);
Assert.Equal(64, hash.Length);
Assert.Equal(default, ObjectRecords.Capture(default));
}
}
}
@@ -0,0 +1,248 @@
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class ObjectShapesTests
{
static NoteDocument Parse(string json) => NoteParser.Parse(JsonNode.Parse(json));
[Fact]
public void Keeps_a_peertube_video_with_its_files_poster_captions_and_markdown_description()
{
var note = Parse("""
{
"id": "https://tube.example/videos/watch/1", "type": "Video", "name": "A *talk*", "duration": "PT6299S",
"mediaType": "text/markdown", "content": "**Bold** words", "sensitive": false, "views": 14771, "state": 1,
"isLiveBroadcast": false, "commentsPolicy": 2, "downloadEnabled": true,
"licence": { "identifier": "5", "name": "Attribution - Share Alike" }, "category": { "identifier": "2", "name": "Films" },
"attributedTo": [{ "type": "Person", "id": "https://tube.example/accounts/ann" }, { "type": "Group", "id": "https://tube.example/video-channels/films" }],
"icon": [
{ "type": "Image", "url": "https://tube.example/thumb-280.jpg", "mediaType": "image/jpeg", "width": 280, "height": 157 },
{ "type": "Image", "url": "https://tube.example/thumb-1920.jpg", "mediaType": "image/jpeg", "width": 1920, "height": 1080 },
{ "type": "Image", "url": "https://tube.example/thumb-850.jpg", "mediaType": "image/jpeg", "width": 850, "height": 480 }
],
"subtitleLanguage": [{ "identifier": "en", "url": [{ "mediaType": "text/vtt", "href": "https://tube.example/en.vtt" }] }],
"url": [
{ "type": "Link", "mediaType": "text/html", "href": "https://tube.example/w/abc" },
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/720.mp4", "height": 720, "width": 1280, "fps": 24, "size": 942586378,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }] },
{ "type": "Link", "mediaType": "application/x-mpegURL", "href": "https://tube.example/master.m3u8" },
{ "type": "Link", "mediaType": "application/x-bittorrent;x-scheme-handler/magnet", "href": "magnet:?xt=urn:btih:abc" }
]
}
""");
Assert.Equal("https://tube.example/w/abc", note.Url);
Assert.Equal("A *talk*", note.Title);
Assert.Contains("<strong>Bold</strong>", note.ContentHtml);
Assert.Equal("https://tube.example/thumb-850.jpg", note.CoverURL);
Assert.Equal(6299, note.Video.DurationSeconds);
var file = Assert.Single(note.Video.Variants);
Assert.Equal(new[] { "audio", "video" }, file.Streams);
Assert.Equal(942586378, file.Size);
Assert.Equal("https://tube.example/master.m3u8", note.Video.PlaylistURL);
Assert.Equal(2, note.Video.CommentsPolicy);
Assert.Equal("Attribution - Share Alike", note.Video.Licence);
Assert.Equal("https://tube.example/video-channels/films", note.Video.ChannelURI);
Assert.Equal("https://tube.example/en.vtt", Assert.Single(note.Video.Captions).Url);
Assert.Equal("https://tube.example/accounts/ann", note.AttributedTo);
}
[Fact]
public void Keeps_a_mobilizon_event_with_its_time_place_and_online_link()
{
var note = Parse("""
{
"id": "https://events.example/events/9", "type": "Event", "name": "Repair café",
"summary": "<p>Saturday at the library</p>", "content": "<p>Bring broken things.</p>",
"startTime": "2026-10-03T10:00:00+02:00", "endTime": "2026-10-03T13:00:00+02:00", "timezone": "Europe/Rome",
"joinMode": "free", "participantCount": 12, "maximumAttendeeCapacity": 40, "ical:status": "CONFIRMED",
"location": { "type": "Place", "name": "Biblioteca", "latitude": 45.07, "longitude": 7.68,
"address": { "type": "PostalAddress", "streetAddress": "Via Roma 1", "addressLocality": "Torino", "addressCountry": "IT" } },
"attachment": [{ "type": "Link", "name": "Website", "href": "https://meet.example/repair" }],
"attributedTo": "https://events.example/@club"
}
""");
Assert.Null(note.SpoilerText);
Assert.False(note.Sensitive);
Assert.Equal("Saturday at the library", note.Excerpt);
Assert.Equal(new DateTime(2026, 10, 3, 8, 0, 0, DateTimeKind.Utc), note.Event.Start);
Assert.Equal("Via Roma 1, Torino, IT", Assert.Single(note.Event.Places).Address);
Assert.Equal("https://meet.example/repair", note.Event.OnlineURL);
Assert.Equal(40, note.Event.Capacity);
Assert.Equal("CONFIRMED", note.Event.Status);
Assert.Equal("2026-10-03 10:00 – 13:00 (Europe/Rome) · Biblioteca", MastodonMapper.EventLine(note.Event));
}
[Fact]
public void Keeps_a_lemmy_link_post_as_a_link_with_its_thumbnail()
{
var note = Parse("""
{
"id": "https://lemmy.example/post/5", "type": "Page", "name": "A good read", "attributedTo": "https://lemmy.example/u/bo",
"attachment": [{ "type": "Link", "href": "https://news.example/story", "mediaType": "text/html" }],
"image": { "type": "Image", "url": "https://lemmy.example/pictrs/thumb.jpg" },
"language": { "identifier": "en", "name": "English" }
}
""");
Assert.Equal("https://news.example/story", note.Link.Href);
Assert.Equal("https://lemmy.example/pictrs/thumb.jpg", note.Link.ImageURL);
Assert.False(note.Link.FromPublisher);
Assert.Empty(note.Attachments);
}
[Fact]
public void Takes_a_publisher_supplied_preview_from_a_link_attachment()
{
var note = Parse("""
{
"id": "https://m.example/statuses/1", "type": "Note", "attributedTo": "https://m.example/users/a", "content": "<p>look</p>",
"attachment": [{ "type": "Link", "href": "https://news.example/a",
"preview": { "type": "Article", "name": "Headline &amp; more", "summary": "<p>What happened</p>", "image": { "type": "Image", "url": "https://news.example/a.jpg" } } }]
}
""");
Assert.Equal("Headline & more", note.Link.Title);
Assert.Equal("What happened", note.Link.Description);
Assert.Equal("https://news.example/a.jpg", note.Link.ImageURL);
Assert.True(note.Link.FromPublisher);
}
[Fact]
public void Reads_media_in_every_shape_and_infers_what_is_missing()
{
var note = Parse("""
{
"id": "https://bsky.brid.gy/convert/ap/at://did:plc:x/post/1", "type": "Note", "attributedTo": "https://bsky.brid.gy/ap/did:plc:x",
"content": "<p>pics</p>",
"@context": ["https://www.w3.org/ns/activitystreams", { "@language": "de" }],
"attachment": [
{ "type": "Image", "url": "https://cdn.example/a.jpeg", "summary": "alt from summary" },
{ "type": "Document", "url": [{ "type": "Link", "href": "https://cdn.example/clip", "mediaType": "video/mp4" }],
"icon": { "type": "Image", "url": "https://cdn.example/clip.jpg" }, "duration": "PT12.4S", "sensitive": true },
{ "type": "Image", "url": "https://cdn.example/nameless" }
]
}
""");
Assert.Equal("de", note.Language);
Assert.Equal("image/jpeg", note.Attachments[0].ContentType);
Assert.Equal("alt from summary", note.Attachments[0].Description);
Assert.Equal("https://cdn.example/clip", note.Attachments[1].RemoteURL);
Assert.Equal("video/mp4", note.Attachments[1].ContentType);
Assert.Equal("https://cdn.example/clip.jpg", note.Attachments[1].RemotePreviewURL);
Assert.Equal(12.4, note.Attachments[1].DurationSeconds);
Assert.True(note.Attachments[1].Sensitive);
Assert.Equal("image/*", note.Attachments[2].ContentType);
}
[Fact]
public void Keeps_a_funkwhale_track_and_a_loops_poster()
{
var track = Parse("""
{
"id": "https://music.example/federation/music/uploads/1", "type": "Audio", "name": "Song", "attributedTo": "https://music.example/@band",
"summary": "#rock #live", "duration": 241, "position": 3, "album": "https://music.example/albums/2", "license": "https://creativecommons.org/licenses/by/4.0/",
"url": [{ "type": "Link", "mediaType": "text/html", "href": "https://music.example/library/tracks/1" },
{ "type": "Link", "mediaType": "audio/mpeg", "href": "https://music.example/listen/1.mp3", "bitrate": 320000, "size": 9640000 }]
}
""");
var loop = Parse("""
{
"id": "https://loops.example/v/1", "type": "Note", "attributedTo": "https://loops.example/@dancer", "content": "",
"preview": { "type": "Image", "url": { "href": "https://loops.example/poster.jpg", "mediaType": "image/jpeg" } },
"attachment": [{ "type": "Document", "mediaType": "video/mp4", "url": "https://loops.example/v/1.mp4", "width": 720, "height": 1280 }]
}
""");
Assert.Equal("#rock #live", track.Excerpt);
Assert.Null(track.SpoilerText);
Assert.Equal(241, track.Audio.DurationSeconds);
Assert.Equal(3, track.Audio.Position);
Assert.Equal("https://music.example/listen/1.mp3", Assert.Single(track.Audio.Variants).Url);
Assert.Equal("https://loops.example/poster.jpg", loop.CoverURL);
}
[Fact]
public void Keeps_custom_emoji_and_drops_ones_that_could_not_render()
{
var note = Parse("""
{
"id": "https://k.example/notes/1", "type": "Note", "attributedTo": "https://k.example/users/1", "content": ":blobcat: hi :bad:",
"tag": [
{ "type": "Emoji", "name": ":blobcat:", "updated": "2026-01-01T00:00:00Z", "icon": { "type": "Image", "mediaType": "image/png", "url": "https://k.example/emoji/blobcat.png" } },
{ "type": "Emoji", "name": ":blobcat:", "icon": { "type": "Image", "url": "https://k.example/emoji/duplicate.png" } },
{ "type": "Emoji", "name": ":bad name:", "icon": { "type": "Image", "url": "https://k.example/emoji/x.png" } },
{ "type": "Emoji", "name": ":nourl:", "icon": { "type": "Image", "url": "data:image/png;base64,AAAA" } }
]
}
""");
var emoji = Assert.Single(note.Emojis);
Assert.Equal("blobcat", emoji.Shortcode);
Assert.Equal("https://k.example/emoji/blobcat.png", emoji.URL);
}
[Fact]
public void Reads_polls_from_mastodon_misskey_and_closed_ones()
{
var mastodon = Parse("""
{ "id": "https://m.example/q/1", "type": "Question", "attributedTo": "https://m.example/u", "content": "?", "endTime": "2026-10-02T00:00:00Z", "votersCount": 7,
"oneOf": [{ "type": "Note", "name": "a", "replies": { "type": "Collection", "totalItems": 4 } }, { "type": "Note", "name": "b", "replies": { "type": "Collection", "totalItems": 3 } }] }
""");
var misskey = Parse("""
{ "id": "https://k.example/q/1", "type": "Question", "attributedTo": "https://k.example/u", "_misskey_content": "?",
"anyOf": [{ "type": "Note", "name": "x", "_misskey_votes": 2 }, { "type": "Note", "name": "y", "_misskey_votes": 5 }] }
""");
var closed = Parse("""
{ "id": "https://g.example/q/1", "type": "Question", "attributedTo": "https://g.example/u", "content": "?", "closed": "2026-09-30T12:00:00Z",
"oneOf": [{ "type": "Note", "name": "p" }, { "type": "Note", "name": "q" }] }
""");
Assert.False(mastodon.Poll.Multiple);
Assert.Equal(new[] { 4, 3 }, mastodon.Poll.Options.Select(o => o.Votes));
Assert.Equal(7, mastodon.Poll.VotersCount);
Assert.True(misskey.Poll.Multiple);
Assert.Equal(new[] { 2, 5 }, misskey.Poll.Options.Select(o => o.Votes));
Assert.Null(misskey.Poll.VotersCount);
Assert.Equal(new DateTime(2026, 9, 30, 12, 0, 0, DateTimeKind.Utc), closed.Poll.ClosedAt);
}
[Fact]
public void Finds_playable_files_inside_an_hls_only_peertube_video_and_picks_one_with_sound()
{
var note = Parse("""
{
"id": "https://tube.example/videos/watch/2", "type": "Video", "name": "HLS only", "attributedTo": "https://tube.example/accounts/ann",
"url": [
{ "type": "Link", "mediaType": "text/html", "href": "https://tube.example/w/def" },
{ "type": "Link", "mediaType": "application/x-mpegURL", "href": "https://tube.example/master.m3u8", "tag": [
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/1080-fragmented.mp4", "height": 1080,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }] },
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/720-fragmented.mp4", "height": 720,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }] },
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/480-fragmented.mp4", "height": 480,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }] }
] }
]
}
""");
Assert.Equal(3, note.Video.Variants.Count);
Assert.Equal("https://tube.example/480-fragmented.mp4", MastodonMapper.BestVideo(note.Video.Variants).Url);
}
[Fact]
public void Normalises_hashtags_like_mastodon() =>
Assert.Equal("fedi".Normalize(System.Text.NormalizationForm.FormKC).ToLowerInvariant(), Assert.Single(Parse("""
{ "id": "https://x.example/1", "type": "Note", "attributedTo": "https://x.example/u", "content": "",
"tag": [{ "type": "Hashtag", "name": "#FEDI" }] }
""").Tags));
}
}
+138
View File
@@ -0,0 +1,138 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class PollTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task Follows(RemoteActor follower, string target) =>
await _harness.Deliver(follower, "/human-centipede", new JsonObject
{
["id"] = NewId(follower, "follows"), ["type"] = "Follow", ["actor"] = follower.Id, ["object"] = target
});
JsonObject Vote(RemoteActor voter, string question, string owner, string choice) => new()
{
["id"] = NewId(voter, "activities"), ["type"] = "Create", ["actor"] = voter.Id, ["to"] = new JsonArray(owner),
["object"] = new JsonObject
{
["id"] = NewId(voter, "votes"), ["type"] = "Note", ["name"] = choice, ["attributedTo"] = voter.Id,
["inReplyTo"] = question, ["to"] = new JsonArray(owner)
}
};
[Fact]
public async Task A_local_poll_goes_out_as_a_question_and_counts_each_remote_voter_once()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await Follows(mallory, alice.Uri);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "which?", Poll = new PollDraft(new[] { "tea", "coffee" }, 3600, false, false) }, token);
Assert.True(outcome.Ok);
var question = (await _harness.Outgoing(mallory.Id + "/inbox")).Single(a => a["type"]!.GetValue<string>() == "Create")["object"]!;
Assert.Equal("Question", question["type"]!.GetValue<string>());
Assert.Equal(new[] { "tea", "coffee" }, question["oneOf"]!.AsArray().Select(o => o!["name"]!.GetValue<string>()));
await _harness.Deliver(mallory, "/human-centipede", Vote(mallory, outcome.Post.ObjectURI, alice.Uri, "coffee"));
await _harness.Deliver(mallory, "/human-centipede", Vote(mallory, outcome.Post.ObjectURI, alice.Uri, "tea"));
var poll = (await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token)).Poll;
Assert.Equal(new[] { 0, 1 }, poll.Options.Select(o => o.Votes));
Assert.Equal(1, poll.VotersCount);
Assert.False(await DB.Default.Find<Post>().Match(p => p.InReplyToURI == outcome.Post.ObjectURI).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollRefresh && j.Payload == outcome.Post.ID).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollClose && j.Payload == outcome.Post.ID).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_persona_votes_on_a_remote_poll_and_a_count_refresh_is_not_an_edit()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var questionId = NewId(mallory, "questions");
JsonObject Question(int first, int second) => new()
{
["id"] = questionId, ["type"] = "Question", ["attributedTo"] = mallory.Id, ["content"] = "<p>pick</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(alice.Uri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" }),
["endTime"] = DateTime.UtcNow.AddDays(1).ToString("O"),
["anyOf"] = new JsonArray(
new JsonObject { ["type"] = "Note", ["name"] = "red", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = first } },
new JsonObject { ["type"] = "Note", ["name"] = "blue", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = second } })
};
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities"), ["type"] = "Create", ["actor"] = mallory.Id, ["object"] = Question(3, 1)
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == questionId).ExecuteFirstAsync(token);
Assert.True(post.Poll.Multiple);
Assert.True((await _harness.Polls.Vote(alice, post, new[] { 0, 1 }, token)).Ok);
Assert.False((await _harness.Polls.Vote(alice, post, new[] { 0 }, token)).Ok);
var votes = (await _harness.Outgoing(mallory.Id + "/inbox")).Where(a => a["type"]!.GetValue<string>() == "Create").Select(a => a["object"]!).ToList();
Assert.Equal(new[] { "red", "blue" }, votes.Select(v => v["name"]!.GetValue<string>()).Order().Reverse());
Assert.All(votes, v => Assert.Equal(questionId, v["inReplyTo"]!.GetValue<string>()));
Assert.All(votes, v => Assert.Null(v["published"]));
Assert.Equal(new[] { 0, 1 }, (await _harness.Polls.OwnVotes(post.ID, alice.Id, token)).Order());
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities"), ["type"] = "Update", ["actor"] = mallory.Id, ["object"] = Question(9, 5)
});
var refreshed = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Equal(new[] { 9, 5 }, refreshed.Poll.Options.Select(o => o.Votes));
Assert.Empty(refreshed.Revisions);
Assert.Null(refreshed.EditedAt);
}
[Fact]
public async Task Closing_a_poll_tells_its_voters_and_its_author()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "quick", Poll = new PollDraft(new[] { "yes", "no" }, 300, false, false) }, token);
Assert.True((await _harness.Polls.Vote(bob, outcome.Post, new[] { 0 }, token)).Ok);
Assert.False((await _harness.Polls.Vote(alice, outcome.Post, new[] { 0 }, token)).Ok);
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollClose && j.Payload == outcome.Post.ID).ExecuteFirstAsync(token);
await new PollCloseJob(_harness.Db, _harness.Local, _harness.Outbox).Handle(job, token);
Assert.NotNull((await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token)).Poll.ClosedAt);
var told = await DB.Default.Find<Notification>().Match(n => n.Type == NotificationType.Poll && n.PostId == outcome.Post.ID).ExecuteAsync(token);
Assert.Equal(new[] { alice.Id, bob.Id }.Order(), told.Select(n => n.AvatarId).Order());
Assert.False((await _harness.Polls.Vote(bob, await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token), new[] { 1 }, token)).Ok);
}
}
}
@@ -0,0 +1,72 @@
using MongoDB.Entities;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class ProvenanceTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task A_delivered_post_keeps_its_raw_object_how_it_came_and_every_edit()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var origin = new Uri(mallory.Id).GetLeftPart(UriPartial.Authority);
var note = new JsonObject
{
["id"] = $"{origin}/notes/{Guid.NewGuid():N}",
["type"] = "Note",
["attributedTo"] = mallory.Id,
["content"] = "<p>hi</p>",
["_misskey_quote"] = $"{origin}/notes/elsewhere",
["to"] = new JsonArray(Addressing.Public),
["cc"] = new JsonArray(alice.Uri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" })
};
var create = new JsonObject { ["id"] = $"{origin}/activities/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = mallory.Id, ["object"] = note.DeepClone() };
await _harness.Deliver(mallory, "/human-centipede", create);
var edited = note.DeepClone().AsObject();
edited["content"] = "<p>hi again</p>";
edited["updated"] = DateTime.UtcNow.ToString("O");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = $"{origin}/activities/{Guid.NewGuid():N}", ["type"] = "Update", ["actor"] = mallory.Id, ["object"] = edited
});
var record = await DB.Default.Find<ObjectRecord>().Match(r => r.ObjectURI == note["id"]!.GetValue<string>()).ExecuteFirstAsync(token);
Assert.NotNull(record);
Assert.Equal(ObjectPath.Delivered, record.Path);
Assert.False(record.Refetched);
Assert.Equal("shared", record.Inbox);
Assert.Equal(mallory.KeyId, record.KeyId);
Assert.Equal(create["id"]!.GetValue<string>(), record.ActivityId);
Assert.Contains("digest", record.SignedHeaders);
Assert.Equal("<p>hi</p>", JsonNode.Parse(record.Raw)!["content"]!.GetValue<string>());
Assert.Contains("legacy-quote", ObjectFeatures.Detect(JsonNode.Parse(record.Raw)!.AsObject()));
Assert.Equal("<p>hi again</p>", JsonNode.Parse(Assert.Single(record.Revisions).Raw)!["content"]!.GetValue<string>());
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.DescribeInstance && j.Payload == record.Host).ExecuteAnyAsync(token));
}
}
}
+221
View File
@@ -0,0 +1,221 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class QuoteTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task<Post> Delivered(RemoteActor author, string mentioned, Action<JsonObject> shape = default)
{
var note = new JsonObject
{
["id"] = NewId(author, "notes"), ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = "<p>hello</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(mentioned),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = mentioned, ["name"] = "@someone" })
};
shape?.Invoke(note);
await _harness.Deliver(author, "/human-centipede", new JsonObject { ["id"] = NewId(author, "activities"), ["type"] = "Create", ["actor"] = author.Id, ["object"] = note });
return await DB.Default.Find<Post>().Match(p => p.ObjectURI == note["id"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
}
string Stamp(RemoteActor quotedAuthor, string quoting, string quoted)
{
var path = $"/stamps/{Guid.NewGuid():N}";
var id = Origin(quotedAuthor) + path;
_harness.Peer.Serve(path, new JsonObject
{
["@context"] = "https://www.w3.org/ns/activitystreams", ["id"] = id, ["type"] = "QuoteAuthorization",
["attributedTo"] = quotedAuthor.Id, ["interactingObject"] = quoting, ["interactionTarget"] = quoted
}.ToJsonString());
return id;
}
[Fact]
public async Task A_misskey_quote_of_a_public_post_is_shown_and_counted()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var kitty = new RemoteActor(_harness.Peer, "kitty");
var original = await Delivered(ann, alice.Uri);
var quoting = await Delivered(kitty, alice.Uri, n =>
{
n["_misskey_quote"] = original.ObjectURI;
n["content"] = $"<p>look<span class=\"quote-inline\"><br>RE: <a href=\"{original.ObjectURI}\">{original.ObjectURI}</a></span></p>";
});
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.Equal(original.ID, quoting.QuotedPostId);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
var status = await new PrivaPub.Api.Mastodon.Mappers.MastodonMapper(_harness.Db, _harness.Local).Status(quoting, alice.Id, token);
Assert.Equal("accepted", status.Quote.State);
Assert.Equal(original.ID, status.Quote.QuotedStatus.Id);
Assert.DoesNotContain("RE:", status.Content);
}
[Fact]
public async Task A_stamped_quote_is_accepted_a_forged_one_is_not_and_revoking_takes_it_back()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var bob = new RemoteActor(_harness.Peer, "bob");
var original = await Delivered(ann, alice.Uri);
var quotingId = NewId(bob, "notes");
var stamp = Stamp(ann, quotingId, original.ObjectURI);
var forged = Stamp(ann, NewId(bob, "notes"), original.ObjectURI);
var quoting = await Delivered(bob, alice.Uri, n => { n["id"] = quotingId; n["quote"] = original.ObjectURI; n["quoteAuthorization"] = stamp; });
var cheat = await Delivered(bob, alice.Uri, n => { n["quote"] = original.ObjectURI; n["quoteAuthorization"] = forged; });
var waiting = await Delivered(bob, alice.Uri, n => n["quote"] = original.ObjectURI);
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.Equal(QuoteState.Unauthorized, cheat.QuoteState);
Assert.Equal(QuoteState.Pending, waiting.QuoteState);
await _harness.Deliver(ann, "/human-centipede", new JsonObject
{
["id"] = NewId(ann, "activities"), ["type"] = "Delete", ["actor"] = ann.Id, ["object"] = stamp
});
Assert.Equal(QuoteState.Revoked, (await DB.Default.Find<Post>().OneAsync(quoting.ID, token)).QuoteState);
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
}
[Fact]
public async Task A_persona_asks_before_quoting_and_is_stamped_when_accepted()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var original = await Delivered(ann, alice.Uri, n => n["interactionPolicy"] = new JsonObject
{
["canQuote"] = new JsonObject { ["automaticApproval"] = new JsonArray(Addressing.Public) }
});
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "this", QuotedStatusId = original.ID }, token);
Assert.Equal(QuoteState.Pending, outcome.Post.QuoteState);
var request = Assert.Single(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "QuoteRequest");
Assert.Equal(original.ObjectURI, request["object"]!.GetValue<string>());
Assert.Equal(original.ObjectURI, request["instrument"]!["quote"]!.GetValue<string>());
await _harness.Deliver(ann, "/human-centipede", new JsonObject
{
["id"] = NewId(ann, "activities"), ["type"] = "Accept", ["actor"] = ann.Id,
["object"] = request["id"]!.GetValue<string>(), ["result"] = Stamp(ann, outcome.Post.ObjectURI, original.ObjectURI)
});
var quoting = await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token);
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.NotNull(quoting.QuoteAuthorizationURI);
Assert.Contains(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Update"
&& a["object"]!["quoteAuthorization"]?.GetValue<string>() == quoting.QuoteAuthorizationURI);
}
[Fact]
public async Task A_legacy_post_is_quoted_without_asking_and_a_persona_that_allows_nobody_cannot_be_quoted()
{
var token = TestContext.Current.CancellationToken;
var (aliceRoot, alice) = await _harness.Persona("alice");
var (bobRoot, bob) = await _harness.Persona("bob");
var kitty = new RemoteActor(_harness.Peer, "kitty");
var legacy = await Delivered(kitty, alice.Uri);
bob.Settings.QuotePolicy = PrivaPub.Models.User.QuotePolicies.Nobody;
var bobs = (await _harness.Statuses.Publish(bob, new StatusDraft { Text = "mine" }, token)).Post;
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "nice", QuotedStatusId = legacy.ID }, token);
Assert.Equal(QuoteState.Accepted, outcome.Post.QuoteState);
Assert.False(outcome.Post.QuoteByConsent);
Assert.False((await _harness.Statuses.Publish(alice, new StatusDraft { Text = "and this", QuotedStatusId = bobs.ID }, token)).Ok);
Assert.True((await _harness.Statuses.Publish(bob, new StatusDraft { Text = "me again", QuotedStatusId = bobs.ID }, token)).Ok);
}
JsonObject Request(RemoteActor quoter, string quotedUri, string quotingId) => new()
{
["id"] = NewId(quoter, "activities"), ["type"] = "QuoteRequest", ["actor"] = quoter.Id, ["object"] = quotedUri,
["instrument"] = new JsonObject { ["id"] = quotingId, ["type"] = "Note", ["attributedTo"] = quoter.Id, ["quote"] = quotedUri, ["content"] = "<p>so true</p>" }
};
[Fact]
public async Task Anyone_may_quote_a_persona_by_default_and_the_licence_can_be_revoked()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject { ["id"] = NewId(mallory, "follows"), ["type"] = "Follow", ["actor"] = mallory.Id, ["object"] = alice.Uri });
var original = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "quote me" }, token)).Post;
var created = (await _harness.Outgoing(mallory.Id + "/inbox")).Single(a => a["type"]!.GetValue<string>() == "Create")["object"]!;
Assert.Equal(Addressing.Public, created["interactionPolicy"]!["canQuote"]!["automaticApproval"]![0]!.GetValue<string>());
var quotingId = NewId(mallory, "notes");
await _harness.Deliver(mallory, "/human-centipede", Request(mallory, original.ObjectURI, quotingId));
var accept = Assert.Single(await _harness.Outgoing(mallory.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Accept" && a["result"] != null);
var licence = accept["result"]!.GetValue<string>();
Assert.StartsWith(alice.Uri + "/parrot-licences/", licence);
var quoting = await Delivered(mallory, alice.Uri, n => { n["id"] = quotingId; n["quote"] = original.ObjectURI; n["quoteAuthorization"] = licence; });
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
Assert.True(await _harness.Quotes.RevokeLicence(original, quoting, token));
Assert.Equal(QuoteState.Revoked, (await DB.Default.Find<Post>().OneAsync(quoting.ID, token)).QuoteState);
Assert.Contains(await _harness.Outgoing(mallory.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Delete" && a["object"]!.GetValue<string>() == licence);
}
[Fact]
public async Task A_followers_only_persona_turns_a_stranger_down()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
alice.Settings.QuotePolicy = PrivaPub.Models.User.QuotePolicies.Followers;
var original = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "friends only quoting" }, token)).Post;
var stranger = new RemoteActor(_harness.Peer, "stranger");
await _harness.Deliver(stranger, "/human-centipede", Request(stranger, original.ObjectURI, NewId(stranger, "notes")));
Assert.Equal("Reject", Assert.Single(await _harness.Outgoing(stranger.Id + "/inbox"))["type"]!.GetValue<string>());
}
[Fact]
public async Task One_persona_quotes_another_with_a_licence()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var original = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "public thought" }, token)).Post;
var outcome = await _harness.Statuses.Publish(bob, new StatusDraft { Text = "agreed", QuotedStatusId = original.ID }, token);
Assert.True(outcome.Ok);
Assert.Equal(QuoteState.Accepted, outcome.Post.QuoteState);
Assert.StartsWith(alice.Uri + "/parrot-licences/", outcome.Post.QuoteAuthorizationURI);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
}
}
}
+119
View File
@@ -0,0 +1,119 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class ReactionNamesTests
{
[Theory]
[InlineData(":blobcat@k.example:", "blobcat")]
[InlineData("👍🏽", "👍🏽")]
[InlineData("❤️", "❤️")]
[InlineData("nope", null)]
[InlineData("👍👍", null)]
[InlineData(":unknown:", null)]
public void Normalises_what_peers_send(string content, string expected) =>
Assert.Equal(expected, Reactions.Normalise(content, new List<CustomEmoji> { new() { Shortcode = "blobcat", URL = "https://k.example/e.png" } })?.Name);
}
[Trait("Category", "Integration")]
public sealed class ReactionTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task<Post> LocalPost()
{
var (root, alice) = await _harness.Persona("alice");
await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "react to me" }, TestContext.Current.CancellationToken);
return await DB.Default.Find<Post>().Match(p => p.GroupUserId == alice.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken);
}
[Fact]
public async Task Misskey_and_pleroma_reactions_are_kept_apart_from_favourites()
{
var token = TestContext.Current.CancellationToken;
var post = await LocalPost();
var misskey = new RemoteActor(_harness.Peer, "kitty");
var pleroma = new RemoteActor(_harness.Peer, "pleroma");
await _harness.Deliver(misskey, "/human-centipede", new JsonObject
{
["id"] = NewId(misskey, "likes"), ["type"] = "Like", ["actor"] = misskey.Id, ["object"] = post.ObjectURI, ["content"] = ":blobcat:",
["_misskey_reaction"] = ":blobcat:",
["tag"] = new JsonArray(new JsonObject { ["type"] = "Emoji", ["name"] = ":blobcat:", ["icon"] = new JsonObject { ["type"] = "Image", ["url"] = $"{Origin(misskey)}/emoji/blobcat.png" } })
});
await _harness.Deliver(misskey, "/human-centipede", new JsonObject
{
["id"] = NewId(misskey, "likes"), ["type"] = "Like", ["actor"] = misskey.Id, ["object"] = post.ObjectURI, ["content"] = "❤"
});
var fire = new JsonObject { ["id"] = NewId(pleroma, "activities"), ["type"] = "EmojiReact", ["actor"] = pleroma.Id, ["object"] = post.ObjectURI, ["content"] = "🔥" };
await _harness.Deliver(pleroma, "/human-centipede", fire);
var kept = await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID).ExecuteAsync(token);
Assert.Equal(new HashSet<string> { "blobcat", "🔥" }, kept.Select(r => r.Emoji).ToHashSet());
Assert.Equal($"{Origin(misskey)}/emoji/blobcat.png", kept.Single(r => r.Emoji == "blobcat").EmojiURL);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
Assert.True(await DB.Default.Find<Notification>().Match(n => n.Type == NotificationType.Reaction && n.Emoji == "🔥").ExecuteAnyAsync(token));
await _harness.Deliver(pleroma, "/human-centipede", new JsonObject
{
["id"] = NewId(pleroma, "activities"), ["type"] = "Undo", ["actor"] = pleroma.Id, ["object"] = fire.DeepClone()
});
Assert.Equal("blobcat", Assert.Single(await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID).ExecuteAsync(token)).Emoji);
}
[Fact]
public async Task A_persona_reacts_to_a_remote_post_and_takes_it_back()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var noteId = NewId(mallory, "notes");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities"), ["type"] = "Create", ["actor"] = mallory.Id,
["object"] = new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = mallory.Id, ["content"] = "<p>hi</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(alice.Uri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" })
}
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(token);
Assert.True(await _harness.Reactions.React(alice, post, "🎉", token));
Assert.False(await _harness.Reactions.React(alice, post, "not an emoji", token));
Assert.True(await _harness.Reactions.Unreact(alice, post, "🎉", token));
var sent = await _harness.Outgoing(mallory.Id + "/inbox");
var react = Assert.Single(sent, a => a["type"]!.GetValue<string>() == "EmojiReact");
Assert.Equal("🎉", react["content"]!.GetValue<string>());
Assert.Equal(noteId, react["object"]!.GetValue<string>());
Assert.Equal(react["id"]!.GetValue<string>(), Assert.Single(sent, a => a["type"]!.GetValue<string>() == "Undo")["object"]!["id"]!.GetValue<string>());
}
}
}
@@ -32,6 +32,16 @@ namespace PrivaPub.Tests.Federation
await _peer.DisposeAsync();
}
[Fact]
public async Task A_stored_actor_remembers_the_extensions_its_document_uses()
{
var bob = new RemoteActor(_peer, "bob");
var stored = await _service.GetActor(bob.Id, refresh: true, TestContext.Current.CancellationToken);
Assert.Contains("key:rsa-2048", stored.Features);
}
static string Actor(string id, string keyId = default, string owner = default, string pem = default, string type = "Person") =>
new JsonObject
{
+103
View File
@@ -0,0 +1,103 @@
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class RendererTests
{
static readonly LocalActor Alice = new()
{
Id = "a1",
Kind = LocalActorKind.Person,
UserName = "alice",
Name = "Alice",
Summary = "I **like** <b>cats</b>",
PublicKeyPem = Keys.NewKeyPair().PublicKeyPem,
Published = new DateTime(2026, 9, 30, 17, 45, 12, DateTimeKind.Utc),
Fields = new Dictionary<string, string> { ["site"] = "https://alice.example", ["mood"] = "<i>ok</i>" },
BaseAddress = "https://privapub.test"
};
[Fact]
public void An_actor_looks_like_a_mastodon_actor_with_our_route_names()
{
var actor = ActivityPubRenderer.Actor(Alice);
Assert.Equal("https://privapub.test/peasants/alice", actor["id"]!.GetValue<string>());
Assert.Equal("https://privapub.test/@alice", actor["url"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/mouth", actor["inbox"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/anus", actor["outbox"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/groupies", actor["followers"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice/stalking", actor["following"]!.GetValue<string>());
Assert.Equal("https://privapub.test/human-centipede", actor["endpoints"]!["sharedInbox"]!.GetValue<string>());
Assert.Equal("2026-09-30T00:00:00Z", actor["published"]!.GetValue<string>());
Assert.Equal("alice@privapub.test", actor["webfinger"]!.GetValue<string>());
Assert.False(actor["indexable"]!.GetValue<bool>());
Assert.Equal("<p>I <strong>like</strong> &lt;b&gt;cats&lt;/b&gt;</p>", actor["summary"]!.GetValue<string>());
var fields = actor["attachment"]!.AsArray();
Assert.Contains("rel=\"nofollow noopener noreferrer me\"", fields[0]!["value"]!.GetValue<string>());
Assert.Equal("&lt;i&gt;ok&lt;/i&gt;", fields[1]!["value"]!.GetValue<string>());
Assert.Equal("https://privapub.test/peasants/alice#main-key", actor["publicKey"]!["id"]!.GetValue<string>());
Assert.Equal(actor["id"]!.GetValue<string>(), actor["publicKey"]!["owner"]!.GetValue<string>());
}
[Fact]
public void A_titled_post_with_a_warning_keeps_the_title_visible_on_mastodon()
{
var post = new Post
{
ID = "p1",
Title = "On <cats>",
ContentHtml = "<p>body</p>",
HasContentWarning = true,
Language = "en",
Mentions = new() { new PostMention { ActorURI = "https://m.example/users/bob", Handle = "@bob@m.example" } },
Tags = new() { "cats" },
CreationDate = new DateTime(2026, 9, 30, 17, 45, 12, 345, DateTimeKind.Utc)
};
var note = ActivityPubRenderer.Note(post, Alice, default, "https://m.example/users/bob/statuses/1");
Assert.Equal("https://privapub.test/peasants/alice/scribbles/p1", note["id"]!.GetValue<string>());
Assert.Equal("https://privapub.test/@alice/p1", note["url"]!.GetValue<string>());
Assert.Equal("<p><strong>On &lt;cats&gt;</strong></p><p>body</p>", note["content"]!.GetValue<string>());
Assert.Equal("On <cats>", note["summary"]!.GetValue<string>());
Assert.True(note["sensitive"]!.GetValue<bool>());
Assert.Equal("2026-09-30T17:45:12.345Z", note["published"]!.GetValue<string>());
Assert.Equal(new[] { ActivityPubRenderer.Public }, note["to"]!.AsArray().Select(n => n!.GetValue<string>()));
Assert.Equal(new[] { Alice.Followers, "https://m.example/users/bob" }, note["cc"]!.AsArray().Select(n => n!.GetValue<string>()));
var tags = note["tag"]!.AsArray();
Assert.Equal("@bob@m.example", tags[0]!["name"]!.GetValue<string>());
Assert.Equal("https://privapub.test/tags/cats", tags[1]!["href"]!.GetValue<string>());
Assert.Equal("https://m.example/users/bob/statuses/1", note["inReplyTo"]!.GetValue<string>());
Assert.NotNull(note["contentMap"]!["en"]);
}
[Fact]
public void A_warning_without_a_title_or_spoiler_gets_a_generic_summary()
{
var note = ActivityPubRenderer.Note(new Post { ID = "p2", ContentHtml = "<p>x</p>", HasContentWarning = true }, Alice, default, default);
Assert.Equal(ActivityPubRenderer.ContentWarning, note["summary"]!.GetValue<string>());
}
[Fact]
public void A_direct_note_is_addressed_to_its_recipients_only()
{
var post = new Post { ID = "p3", ContentHtml = "<p>psst</p>", Visibility = PostVisibility.Direct };
var note = ActivityPubRenderer.DirectNote(post, Alice, new[] { ("https://m.example/users/bob", "bob@m.example") },
Alice.ConversationUri("c1"));
Assert.Equal(new[] { "https://m.example/users/bob" }, note["to"]!.AsArray().Select(n => n!.GetValue<string>()));
Assert.Empty(note["cc"]!.AsArray());
Assert.Equal("https://privapub.test/peasants/alice/whispers/c1", note["context"]!.GetValue<string>());
Assert.StartsWith("<p><span class=\"h-card\"", note["content"]!.GetValue<string>());
Assert.Equal("Mention", note["tag"]![0]!["type"]!.GetValue<string>());
}
}
}
+146
View File
@@ -0,0 +1,146 @@
using Microsoft.AspNetCore.Mvc.Controllers;
using Microsoft.AspNetCore.Mvc.Infrastructure;
using Microsoft.AspNetCore.Mvc.RazorPages;
using Microsoft.Extensions.DependencyInjection;
using MongoDB.Entities;
using PrivaPub.Federation.Inbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class HostBootTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
[Fact]
public async Task The_host_shares_the_fixture_database()
{
Assert.True(await DB.Default.Find<AppConfiguration>().ExecuteAnyAsync(TestContext.Current.CancellationToken));
using var client = _host.Client();
Assert.Equal(HttpStatusCode.OK, (await client.GetAsync("/build.json", TestContext.Current.CancellationToken)).StatusCode);
}
[Fact]
public async Task The_harness_handles_the_same_activities_as_the_server()
{
await using var harness = await Harness.Start();
var registered = _host.Services.GetServices<IActivityHandler>().Select(h => h.GetType()).OrderBy(t => t.Name);
var harnessed = harness.Handlers.Select(h => h.GetType()).OrderBy(t => t.Name);
Assert.Equal(registered, harnessed);
}
[Fact]
public void Every_job_kind_has_exactly_one_handler()
{
var handlers = _host.Services.GetServices<IJobHandler>().GroupBy(h => h.Kind).ToDictionary(g => g.Key, g => g.Count());
foreach (var kind in Enum.GetValues<JobKind>())
Assert.True(handlers.GetValueOrDefault(kind) == 1, $"{kind} has {handlers.GetValueOrDefault(kind)} handlers");
}
[Fact]
public void Every_controller_and_page_model_can_be_made()
{
using var scope = _host.Services.CreateScope();
var controllers = _host.Get<IActionDescriptorCollectionProvider>().ActionDescriptors.Items
.OfType<ControllerActionDescriptor>()
.Select(d => d.ControllerTypeInfo.AsType())
.Distinct()
.ToList();
var pages = typeof(Program).Assembly.GetTypes().Where(t => t.IsSubclassOf(typeof(PageModel)) && !t.IsAbstract).ToList();
Assert.NotEmpty(controllers);
Assert.NotEmpty(pages);
foreach (var type in controllers.Concat(pages))
Assert.NotNull(ActivatorUtilities.CreateInstance(scope.ServiceProvider, type));
}
[Fact]
public void The_service_graph_is_valid()
{
var services = new ServiceCollection();
foreach (var descriptor in _host.Registered)
((ICollection<ServiceDescriptor>)services).Add(descriptor);
using var provider = services.BuildServiceProvider(new ServiceProviderOptions { ValidateOnBuild = true, ValidateScopes = true });
}
[Fact]
public async Task Swagger_is_not_served_outside_development()
{
using var client = _host.Client();
Assert.Equal(HttpStatusCode.NotFound, (await client.GetAsync("/swagger/index.html", TestContext.Current.CancellationToken)).StatusCode);
}
[Fact]
public async Task A_persona_gets_a_mastodon_token_through_the_real_code_flow()
{
var root = await _host.SignUp();
var persona = await _host.Persona(root, "boot");
var token = await _host.MastodonToken(persona);
using var client = _host.As(token);
var account = await client.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken);
Assert.Contains($"\"id\":\"{persona.Id}\"", account);
Assert.DoesNotContain(root.Id, account);
Assert.DoesNotContain(root.UserName, account);
}
[Fact]
public async Task A_signed_delivery_through_the_real_route_is_queued_and_stored()
{
var token = TestContext.Current.CancellationToken;
await using var peer = await Peer.Start();
var persona = await _host.Persona(await _host.SignUp(), "mouth");
var bob = new RemoteActor(peer, "bob");
var noteId = $"{peer.A}/notes/{Guid.NewGuid():N}";
var activity = new JsonObject
{
["id"] = noteId + "/activity",
["type"] = "Create",
["actor"] = bob.Id,
["to"] = new JsonArray($"{PrivaPubHost.Base}/peasants/{persona.UserName}"),
["object"] = new JsonObject
{
["id"] = noteId,
["type"] = "Note",
["attributedTo"] = bob.Id,
["to"] = new JsonArray($"{PrivaPubHost.Base}/peasants/{persona.UserName}"),
["content"] = "<p>hello through the front door</p>",
["published"] = DateTime.UtcNow.ToString("O")
}
};
using var client = _host.Client();
var response = await client.SendAsync(bob.SignedPost($"/peasants/{persona.UserName}/mouth", activity), token);
Assert.Equal(HttpStatusCode.Accepted, response.StatusCode);
Assert.Equal(1, await _host.RunInbox(noteId + "/activity", token));
var stored = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(token);
Assert.Equal(PostVisibility.Direct, stored.Visibility);
Assert.Contains("front door", stored.ContentHtml);
}
}
}
@@ -0,0 +1,127 @@
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Microsoft.AspNetCore.Routing.Patterns;
using Microsoft.Extensions.DependencyInjection;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class NeverFiveHundredTests : IAsyncLifetime
{
static readonly string[] Junk = { "x", "000000000000000000000000", "%2e%2e", new string('a', 5000), "5f0c1a2b3c4d5e6f70819203" };
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
public static TheoryData<string> Credentials() => new() { "anonymous", "persona", "junk" };
[Theory]
[MemberData(nameof(Credentials))]
public async Task No_route_answers_500_to_junk(string credentials)
{
var token = TestContext.Current.CancellationToken;
var bearer = credentials switch
{
"persona" => await _host.MastodonToken(await _host.Persona(await _host.SignUp(), "junk")),
"junk" => "not-a-token",
_ => default
};
var jwt = credentials == "persona" ? (await _host.SignUp()).Jwt : bearer;
var failures = new List<string>();
foreach (var (method, template) in Routes())
foreach (var value in Junk)
{
var path = Fill(template, value);
using var request = new HttpRequestMessage(new HttpMethod(method), path);
request.Headers.Add(PrivaPubHost.ClientHeader, $"10.9.{Random.Shared.Next(256)}.{Random.Shared.Next(1, 255)}");
var auth = path.StartsWith("/api/", StringComparison.Ordinal) || path.StartsWith("/oauth/", StringComparison.Ordinal) ? bearer : jwt;
if (auth != default)
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", auth);
if (method is "POST" or "PUT" or "PATCH")
request.Content = new StringContent("{}", Encoding.UTF8, "application/json");
using var client = _host.Client();
HttpResponseMessage response;
try
{
response = await client.SendAsync(request, token);
}
catch (Exception ex)
{
failures.Add($"{method} {Short(path)} threw {ex.GetType().Name}: {ex.Message}");
continue;
}
using (response)
{
if ((int)response.StatusCode >= 500)
failures.Add($"{method} {Short(path)} answered {(int)response.StatusCode}");
else if (path.StartsWith("/api/", StringComparison.Ordinal) && (int)response.StatusCode >= 400 && !await IsMastodonError(response))
failures.Add($"{method} {Short(path)} answered {(int)response.StatusCode} without a Mastodon error body");
}
}
Assert.True(failures.Count == 0, $"{failures.Count} problems as {credentials}:\n" + string.Join("\n", failures.Distinct().Take(80)));
}
IEnumerable<(string Method, string Template)> Routes()
{
var endpoints = _host.Services.GetRequiredService<EndpointDataSource>().Endpoints.OfType<RouteEndpoint>();
foreach (var endpoint in endpoints)
{
var methods = endpoint.Metadata.GetMetadata<IHttpMethodMetadata>()?.HttpMethods;
var template = "/" + endpoint.RoutePattern.RawText?.TrimStart('/');
foreach (var method in methods is { Count: > 0 } ? methods : new[] { "GET", "POST" })
yield return (method, template);
}
}
static string Fill(string template, string value)
{
var pattern = RoutePatternFactory.Parse(template);
var builder = new StringBuilder();
foreach (var segment in pattern.PathSegments)
{
builder.Append('/');
foreach (var part in segment.Parts)
builder.Append(part switch
{
RoutePatternLiteralPart literal => literal.Content,
RoutePatternParameterPart => value,
RoutePatternSeparatorPart separator => separator.Content,
_ => string.Empty
});
}
return builder.Length == 0 ? "/" : builder.ToString();
}
static async Task<bool> IsMastodonError(HttpResponseMessage response)
{
var body = await response.Content.ReadAsStringAsync();
try
{
return JsonNode.Parse(body) is JsonObject error && error["error"] is JsonValue;
}
catch (System.Text.Json.JsonException)
{
return false;
}
}
static string Short(string path) => path.Length > 120 ? path[..120] + "…" : path;
}
}
@@ -0,0 +1,95 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Infrastructure
{
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public sealed class DeliveryWorkerTests : IAsyncLifetime
{
static readonly string[] PeerHosts = { "localhost", "127.0.0.1" };
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
public async ValueTask DisposeAsync()
{
if (_peer == default)
return;
await _peer.DisposeAsync();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
[Fact]
public async Task A_dead_host_does_not_hold_up_deliveries_to_live_ones()
{
var token = TestContext.Current.CancellationToken;
var (privateKey, publicKey) = Keys.NewKeyPair();
var avatar = new Avatar { UserName = $"sender{Guid.NewGuid():N}"[..20], PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(avatar, token);
var local = new LocalActorService(new DbEntities(), new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" }));
var sender = local.FromAvatar(avatar);
var run = $"https://privapub.test/a/{Guid.NewGuid():N}/";
var queue = new JobQueue(j => j.DedupeKey.StartsWith(run));
var delivery = new DeliveryService(new DbEntities(), queue);
_peer.Answer("/dead/inbox", 503, TimeSpan.FromMilliseconds(300));
_peer.Answer("/live/inbox", 202);
var deadInboxes = Enumerable.Range(0, 60).Select(i => $"{_peer.B}/dead/inbox?{i}");
await delivery.Enqueue(sender, deadInboxes, new JsonObject { ["id"] = run + "dead", ["type"] = "Create" }, token);
await delivery.Enqueue(sender, new[] { $"{_peer.A}/live/inbox" }, new JsonObject { ["id"] = run + "live", ["type"] = "Create" }, token);
var handler = new DeliveryJobHandler(local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())), NullLogger<DeliveryJobHandler>.Instance);
using var worker = new JobWorker(queue, new IJobHandler[] { handler }, NullLogger<JobWorker>.Instance);
await worker.StartAsync(token);
try
{
await Until(() => Task.FromResult(_peer.Requests.Any(r => r.Path == "/live/inbox")), token);
var deadSoFar = _peer.Requests.Count(r => r.Path == "/dead/inbox");
var live = Assert.Single(_peer.Requests, r => r.Path == "/live/inbox");
Assert.Contains($"keyId=\"{sender.KeyId}\"", live.Signature);
Assert.True(deadSoFar < 30, $"{deadSoFar} dead deliveries were tried before the live one");
await Until(() => DB.Default.Find<Job>().Match(j => j.DedupeKey.StartsWith(run + "dead") && j.State == JobState.Pending && j.Attempts > 0).ExecuteAnyAsync(token), token);
}
finally
{
await worker.StopAsync(CancellationToken.None);
}
var retried = await DB.Default.Find<Job>().Match(j => j.DedupeKey.StartsWith(run + "dead") && j.Attempts > 0 && j.State == JobState.Pending).ExecuteFirstAsync(token);
Assert.Equal("503", retried.LastError?.Split(' ')[0]);
Assert.True(retried.RunAt > DateTime.UtcNow);
Assert.True((await DB.Default.Find<RemoteInstance>().Match(i => i.Host == "localhost").ExecuteFirstAsync(token))?.ConsecutiveFailures > 0);
}
static async Task Until(Func<Task<bool>> condition, CancellationToken token)
{
var deadline = DateTime.UtcNow.AddSeconds(30);
while (!await condition())
{
Assert.True(DateTime.UtcNow < deadline, "timed out");
await Task.Delay(50, token);
}
}
}
}
@@ -6,7 +6,9 @@ using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using PrivaPub.Federation.Moderation;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Federation;
namespace PrivaPub.Tests.Infrastructure
{
@@ -39,7 +41,7 @@ namespace PrivaPub.Tests.Infrastructure
.ConfigurePrimaryHttpMessageHandler(() => SafeHttpHandlerFactory.Create(options));
var provider = services.BuildServiceProvider();
return new FederationHttp(provider.GetRequiredService<IHttpClientFactory>(), new MemoryCache(new MemoryCacheOptions()),
new StaticOptionsMonitor(options), NullLogger<FederationHttp>.Instance);
new StaticOptionsMonitor(options), new StaticBlocks(), NullLogger<FederationHttp>.Instance);
}
[Fact]
@@ -91,6 +93,29 @@ namespace PrivaPub.Tests.Infrastructure
public void IsAllowed_takes_https_dns_names_only(string url, bool allowed) =>
Assert.Equal(allowed, Client(allowTestNetwork: false).IsAllowed(new Uri(url)));
[Fact]
public void IsAllowed_refuses_a_suspended_domain_and_its_subdomains()
{
var http = new FederationHttp(new ServiceCollection().AddHttpClient().BuildServiceProvider().GetRequiredService<IHttpClientFactory>(),
new MemoryCache(new MemoryCacheOptions()), new StaticOptionsMonitor(new FederationOptions()), new StaticBlocks("evil.example"),
NullLogger<FederationHttp>.Instance);
Assert.False(http.IsAllowed(new Uri("https://evil.example/users/x")));
Assert.False(http.IsAllowed(new Uri("https://cdn.evil.example/a.png")));
Assert.True(http.IsAllowed(new Uri("https://notevil.example/users/x")));
}
sealed class StaticBlocks : IDomainBlocks
{
readonly string[] _suspended;
public StaticBlocks(params string[] suspended) => _suspended = suspended;
public DomainBlock Find(string host) => _suspended.Any(s => host == s || host.EndsWith("." + s))
? new DomainBlock { Domain = host, Severity = DomainBlockSeverity.Suspend }
: default;
public bool IsSuspended(string host) => Find(host) != default;
public Task Reload(CancellationToken token) => Task.CompletedTask;
}
sealed class StaticOptionsMonitor : IOptionsMonitor<FederationOptions>
{
public StaticOptionsMonitor(FederationOptions value) => CurrentValue = value;
@@ -0,0 +1,67 @@
using Microsoft.Extensions.Caching.Memory;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
[Trait("Category", "Integration")]
public sealed class HostCircuitBreakerTests : IAsyncLifetime
{
readonly string _host = $"breaker{Guid.NewGuid():N}.example";
public ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
return ValueTask.CompletedTask;
}
public async ValueTask DisposeAsync()
{
if (MongoFixture.Enabled)
await DB.Default.DeleteAsync<RemoteInstance>(i => i.Host == _host);
}
[Fact]
public async Task A_host_is_quarantined_only_past_the_threshold_and_freed_by_a_success()
{
var token = TestContext.Current.CancellationToken;
var breaker = new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions()));
for (var i = 1; i < HostCircuitBreaker.Threshold; i++)
await breaker.Failed(_host, "503", token);
Assert.Null(await breaker.UnavailableUntil(_host, token));
await breaker.Failed(_host, "503", token);
var until = await breaker.UnavailableUntil(_host, token);
Assert.NotNull(until);
Assert.InRange(until.Value, DateTime.UtcNow.AddMinutes(55), DateTime.UtcNow.AddMinutes(65));
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == _host).ExecuteFirstAsync(token);
Assert.Equal(HostCircuitBreaker.Threshold, instance.ConsecutiveFailures);
Assert.Equal("503", instance.LastError);
await breaker.Succeeded(_host, token);
Assert.Null(await breaker.UnavailableUntil(_host, token));
instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == _host).ExecuteFirstAsync(token);
Assert.Equal(0, instance.ConsecutiveFailures);
Assert.NotNull(instance.LastSuccessAt);
}
[Fact]
public async Task A_healthy_host_is_written_at_most_once_an_hour()
{
var token = TestContext.Current.CancellationToken;
var breaker = new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions()));
await breaker.Succeeded(_host, token);
var first = (await DB.Default.Find<RemoteInstance>().Match(i => i.Host == _host).ExecuteFirstAsync(token)).LastSuccessAt;
await breaker.Succeeded(_host, token);
var second = (await DB.Default.Find<RemoteInstance>().Match(i => i.Host == _host).ExecuteFirstAsync(token)).LastSuccessAt;
Assert.Equal(first, second);
}
}
}
+3 -1
View File
@@ -12,6 +12,7 @@ using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public class IndexTests
{
@@ -20,6 +21,8 @@ namespace PrivaPub.Tests.Infrastructure
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
await DB.Default.Index<ForeignAvatar>().DropAllAsync(token);
await DB.Default.Index<Follower>().DropAllAsync(token);
var actorUri = $"https://r.example/users/{Guid.NewGuid():N}";
var localId = Guid.NewGuid().ToString("N")[..24];
await DB.Default.SaveAsync(new[]
@@ -48,7 +51,6 @@ namespace PrivaPub.Tests.Infrastructure
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
await Indexes.Create(token);
var local = new LocalActorService(new DbEntities(), new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" }));
var name = $"n{Guid.NewGuid():N}"[..20];
@@ -0,0 +1,94 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
public class BackoffTests
{
[Fact]
public void Grows_like_mastodons()
{
Assert.InRange(Backoff.After(1).TotalSeconds, 16, 16 + 9 * 2);
Assert.InRange(Backoff.After(10).TotalSeconds, 10015, 10015 + 9 * 11);
}
[Fact]
public void Quarantines_a_host_only_past_the_threshold()
{
Assert.Equal(TimeSpan.Zero, Backoff.HostQuarantine(9, 10));
Assert.Equal(TimeSpan.FromHours(1), Backoff.HostQuarantine(10, 10));
Assert.Equal(TimeSpan.FromHours(4), Backoff.HostQuarantine(12, 10));
Assert.Equal(TimeSpan.FromDays(7), Backoff.HostQuarantine(40, 10));
}
}
[Trait("Category", "Integration")]
public sealed class JobQueueTests : IAsyncLifetime
{
public ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
return ValueTask.CompletedTask;
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static Job NewJob(string host = default, string dedupe = default) => new()
{
Kind = (JobKind)99,
Host = host,
DedupeKey = dedupe,
Payload = "{}"
};
[Fact]
public async Task A_dedupe_key_is_queued_once()
{
var queue = new JobQueue();
var key = Guid.NewGuid().ToString("N");
var inserted = await queue.EnqueueMany(new[] { NewJob(dedupe: key), NewJob(dedupe: key) }, TestContext.Current.CancellationToken);
Assert.Equal(1, inserted);
}
[Fact]
public async Task A_busy_host_is_skipped_and_a_failed_job_backs_off()
{
var token = TestContext.Current.CancellationToken;
var queue = new JobQueue();
var kind = (JobKind)(100 + Random.Shared.Next(1000));
var busy = $"busy{Guid.NewGuid():N}.example";
var free = $"free{Guid.NewGuid():N}.example";
await queue.EnqueueMany(new[] { new Job { Kind = kind, Host = busy }, new Job { Kind = kind, Host = free } }, token);
var leased = await queue.Lease(kind, new[] { busy }, token);
Assert.Equal(free, leased.Host);
Assert.Equal(1, leased.Attempts);
await queue.Finish(leased, JobOutcome.Retry("boom"), maxAttempts: 3, token);
var after = await DB.Default.Find<Job>().OneAsync(leased.ID, token);
Assert.Equal(JobState.Pending, after.State);
Assert.True(after.RunAt > DateTime.UtcNow.AddSeconds(10));
leased.Attempts = 3;
await queue.Finish(leased, JobOutcome.Retry("boom"), maxAttempts: 3, token);
Assert.Equal(JobState.Dead, (await DB.Default.Find<Job>().OneAsync(leased.ID, token)).State);
}
[Fact]
public async Task An_expired_lease_returns_to_the_queue()
{
var token = TestContext.Current.CancellationToken;
var job = new Job { Kind = (JobKind)98, State = JobState.Running, LeasedUntil = DateTime.UtcNow.AddMinutes(-1) };
await DB.Default.SaveAsync(job, token);
await new JobQueue(j => j.ID == job.ID).Reap(token);
Assert.Equal(JobState.Pending, (await DB.Default.Find<Job>().OneAsync(job.ID, token)).State);
}
}
}
@@ -1,11 +1,13 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Data.Migrations;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public class MigrationTests
{
@@ -29,5 +31,53 @@ namespace PrivaPub.Tests.Infrastructure
Assert.Equal("<p><strong>bold</strong> &lt;b&gt;raw&lt;/b&gt;</p>", migratedLocal.ContentHtml);
Assert.Equal(ContentFormat.Markdown, migratedLocal.ContentFormat);
}
[Fact]
public async Task Direct_posts_move_into_posts_with_their_conversation()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var dm = new DmPost { GroupId = "conversation1", Text = "psst", ObjectURI = $"https://privapub.test/{Guid.NewGuid():N}", GroupUserId = "a1" };
await DB.Default.SaveAsync(dm, token);
await new _005_direct_posts_join_posts().UpgradeAsync();
var post = await DB.Default.Find<Post>().OneAsync(dm.ID, token);
Assert.Equal(PostVisibility.Direct, post.Visibility);
Assert.Equal("conversation1", post.ConversationId);
Assert.Equal("a1", post.AuthorAccountId);
}
[Fact]
public async Task Fetched_records_lose_the_signature_they_never_had_and_every_record_gets_its_extensions()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var fetched = new ObjectRecord
{
ObjectURI = $"https://r.example/{Guid.NewGuid():N}", Path = ObjectPath.Fetched, KeyId = "https://x.example/u#k", SignatureScheme = "draft-cavage",
Algorithm = "rsa-sha256", SignedHeaders = new() { "host" }, ActivityContext = "\"https://www.w3.org/ns/activitystreams\"",
Raw = "{\"type\":\"Note\",\"quoteUrl\":\"https://q.example/1\"}"
};
var delivered = new ObjectRecord
{
ObjectURI = $"https://r.example/{Guid.NewGuid():N}", Path = ObjectPath.Delivered, KeyId = "https://x.example/u#k", SignatureScheme = "draft-cavage",
Raw = "not json"
};
await DB.Default.SaveAsync(new[] { fetched, delivered }, token);
await new _008_fetched_records_lose_the_trigger_signature().UpgradeAsync();
await new _009_object_records_keep_their_extensions().UpgradeAsync();
var afterFetched = await DB.Default.Find<ObjectRecord>().OneAsync(fetched.ID, token);
var afterDelivered = await DB.Default.Find<ObjectRecord>().OneAsync(delivered.ID, token);
Assert.Null(afterFetched.KeyId);
Assert.Null(afterFetched.SignatureScheme);
Assert.Empty(afterFetched.SignedHeaders);
Assert.Null(afterFetched.ActivityContext);
Assert.Equal("https://x.example/u#k", afterDelivered.KeyId);
Assert.Contains("legacy-quote", afterFetched.Extensions);
Assert.Empty(afterDelivered.Extensions);
}
}
}
@@ -0,0 +1,70 @@
using MongoDB.Bson;
using PrivaPub.Infrastructure.Ids;
using PrivaPub.Models.User;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Infrastructure
{
public class PrivacyIdsTests
{
[Fact]
public void Persona_ids_carry_only_the_day_and_share_nothing_else()
{
var avatar = new Avatar();
var first = (string)avatar.GenerateNewID();
var second = (string)new Avatar { PublishedOn = avatar.PublishedOn }.GenerateNewID();
var group = new GroupEntity();
Assert.True(ObjectId.TryParse(first, out var a));
Assert.True(ObjectId.TryParse(second, out var b));
Assert.Equal(avatar.PublishedOn, a.CreationTime);
Assert.InRange(a.CreationTime, DateTime.UtcNow.Date.AddDays(-13), DateTime.UtcNow.Date);
Assert.Equal(a.CreationTime, b.CreationTime);
Assert.NotEqual(first[8..14], second[8..14]);
Assert.Equal(group.PublishedOn, ObjectId.Parse((string)group.GenerateNewID()).CreationTime);
}
[Fact]
public void A_remote_post_id_sorts_by_its_published_time()
{
var older = PrivacyIds.At(new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc));
var newer = PrivacyIds.At(new DateTime(2025, 1, 2, 0, 0, 0, DateTimeKind.Utc));
Assert.True(string.CompareOrdinal(older, newer) < 0);
Assert.Equal(new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc), ObjectId.Parse(older).CreationTime);
}
[Fact]
public void A_post_arriving_live_sorts_after_everything_already_stored()
{
var local = ObjectId.GenerateNewId().ToString();
var reply = PrivacyIds.Arrived(DateTime.UtcNow.AddSeconds(-30));
var skewed = PrivacyIds.Arrived(DateTime.UtcNow.AddYears(50));
Assert.True(string.CompareOrdinal(local, reply) < 0);
Assert.True(string.CompareOrdinal(reply, skewed) < 0);
Assert.True(ObjectId.Parse(skewed).CreationTime < DateTime.UtcNow.AddMinutes(1));
}
[Fact]
public void A_backfilled_post_keeps_its_published_time() =>
Assert.Equal(new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc), ObjectId.Parse(PrivacyIds.Arrived(new DateTime(2025, 1, 1, 0, 0, 0, DateTimeKind.Utc))).CreationTime);
[Fact]
public void A_persona_publishes_a_day_within_two_weeks_before_its_creation()
{
var created = new DateTime(2026, 10, 1, 15, 30, 0, DateTimeKind.Utc);
var days = Enumerable.Range(0, 200).Select(_ => PrivacyIds.PublishedDay(created)).ToList();
Assert.All(days, d => Assert.Equal(TimeSpan.Zero, d.TimeOfDay));
Assert.All(days, d => Assert.InRange(d, created.Date.AddDays(-13), created.Date));
Assert.True(days.Distinct().Count() > 1);
}
[Fact]
public void A_published_time_in_the_future_is_clamped() =>
Assert.True(ObjectId.Parse(PrivacyIds.At(DateTime.UtcNow.AddYears(50))).CreationTime < DateTime.UtcNow.AddDays(2));
}
}
+1
View File
@@ -9,6 +9,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.9" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.10.1" />
<PackageReference Include="xunit.v3" Version="3.2.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5" />
@@ -0,0 +1,111 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Statistics
{
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public sealed class DeliveryAttemptTests : IAsyncLifetime
{
static readonly string[] PeerHosts = { "localhost", "127.0.0.1" };
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
public async ValueTask DisposeAsync()
{
if (_harness == default)
return;
await _harness.DisposeAsync();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
async Task<(JobOutcome Outcome, InteractionEvent Event)> Attempt(string path, JsonObject activity, LocalActor signer = default)
{
var ledger = new MemoryLedger();
var handler = new DeliveryJobHandler(_harness.Local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())),
NullLogger<DeliveryJobHandler>.Instance, ledger);
signer ??= (await _harness.Persona("sender")).Avatar;
var id = activity["id"]!.GetValue<string>();
await _harness.Delivery.Enqueue(signer, new[] { _harness.Peer.A + path }, activity, TestContext.Current.CancellationToken);
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == $"{id}|{_harness.Peer.A + path}").ExecuteFirstAsync(TestContext.Current.CancellationToken);
job.Attempts = 1;
var outcome = await handler.Handle(job, TestContext.Current.CancellationToken);
return (outcome, Assert.Single(ledger.Of("out")));
}
static JsonObject Create(string audience) => new()
{
["id"] = $"https://privapub.test/a/{Guid.NewGuid():N}",
["type"] = "Create",
["to"] = new JsonArray(audience),
["object"] = new JsonObject { ["type"] = "Note", ["content"] = "hi" }
};
[Fact]
public async Task A_delivered_public_post_is_ok_with_its_shape_size_and_time()
{
_harness.Peer.Answer("/ok/inbox", 202);
var (outcome, e) = await Attempt("/ok/inbox", Create("https://www.w3.org/ns/activitystreams#Public"));
Assert.Equal(JobResult.Done, outcome.Result);
Assert.Equal(("ok", "Create", "Note", "public", "person"), (e.Outcome, e.Activity, e.Object, e.Audience, e.LocalKind));
Assert.Equal(202, e.Status);
Assert.Equal("127.0.0.1", e.Host);
Assert.Null(e.Reason);
Assert.True(e.Bytes > 0);
Assert.NotNull(e.LatencyMs);
Assert.Equal(1, e.Attempt);
}
[Fact]
public async Task Refusals_and_failures_carry_their_status_and_private_traffic_has_no_kind()
{
_harness.Peer.Answer("/gone/inbox", 410);
_harness.Peer.Answer("/busy/inbox", 429);
_harness.Peer.Answer("/broken/inbox", 500);
var (_, gone) = await Attempt("/gone/inbox", Create("https://elsewhere.example/users/x"));
var (_, busy) = await Attempt("/busy/inbox", Create("https://elsewhere.example/users/x"));
var (_, broken) = await Attempt("/broken/inbox", Create("https://elsewhere.example/users/x"));
Assert.Equal(("dead", "410", 410), (gone.Outcome, gone.Reason, gone.Status!.Value));
Assert.Equal(("retry", "429"), (busy.Outcome, busy.Reason));
Assert.Equal(("retry", "500"), (broken.Outcome, broken.Reason));
Assert.All(new[] { gone, busy, broken }, e => Assert.Equal(("private", (string)null), (e.Audience, e.LocalKind)));
}
[Fact]
public async Task A_signer_that_no_longer_exists_is_dead()
{
var ghost = new LocalActor { Id = "000000000000000000000000", Kind = LocalActorKind.Person, UserName = "ghost", BaseAddress = Harness.Base };
var (outcome, e) = await Attempt("/never/inbox", Create("https://www.w3.org/ns/activitystreams#Public"), ghost);
Assert.Equal(JobResult.Dead, outcome.Result);
Assert.Equal(("dead", "signer-gone"), (e.Outcome, e.Reason));
Assert.Null(e.Status);
}
}
}
@@ -0,0 +1,211 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Statistics
{
public class ArrivalVerdictTests
{
[Fact]
public void The_first_decision_wins_and_a_circle_has_no_kind()
{
var verdict = new ArrivalVerdict();
verdict.Drop("not-addressed");
verdict.Accept("stored");
verdict.About("Note", PostVisibility.Circle, DateTime.UtcNow.AddMinutes(-2), new LocalActor { Kind = LocalActorKind.Group, IsCircle = true });
Assert.Equal(("dropped", "not-addressed"), (verdict.Outcome, verdict.Reason));
Assert.Equal("private", verdict.Audience);
Assert.Null(verdict.LocalKind);
Assert.InRange(verdict.AgeSeconds!.Value, 110, 130);
}
[Theory]
[InlineData(PostVisibility.Public, "public")]
[InlineData(PostVisibility.Unlisted, "unlisted")]
[InlineData(PostVisibility.FollowersOnly, "private")]
[InlineData(PostVisibility.Direct, "private")]
[InlineData(PostVisibility.Circle, "private")]
[InlineData(PostVisibility.LocalGeo, "private")]
public void Every_visibility_but_public_and_unlisted_is_private(PostVisibility visibility, string audience) =>
Assert.Equal(audience, ArrivalVerdict.AudienceOf(visibility));
[Fact]
public void Outside_an_arrival_nothing_happens()
{
Arrival.Current = default;
Arrival.Drop("x");
Arrival.About("Note");
}
}
[Trait("Category", "Integration")]
public sealed class HandlerVerdictTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
InteractionEvent Processed(string activityType) =>
_harness.Ledger.Of("in").Last(e => e.Activity == activityType);
JsonObject Create(RemoteActor author, IEnumerable<string> to, JsonObject extra = default)
{
var origin = new Uri(author.Id).GetLeftPart(UriPartial.Authority);
var note = new JsonObject
{
["id"] = $"{origin}/notes/{Guid.NewGuid():N}",
["type"] = "Note",
["attributedTo"] = author.Id,
["content"] = "<p>hello</p>",
["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray())
};
foreach (var (key, value) in extra ?? new JsonObject())
note[key] = value?.DeepClone();
return new JsonObject { ["id"] = $"{origin}/activities/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Id, ["object"] = note };
}
[Fact]
public async Task A_direct_message_is_stored_private_and_without_a_local_kind()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
await _harness.Deliver(bob, "/human-centipede", Create(bob, new[] { alice.Uri }));
var e = Processed("Create");
Assert.Equal(("accepted", "stored", "private"), (e.Outcome, e.Reason, e.Audience));
Assert.Equal("Note", e.Object);
Assert.Null(e.LocalKind);
Assert.Equal("127.0.0.1", e.Host);
Assert.NotNull(e.Attempt);
Assert.NotNull(e.WaitMs);
Assert.Equal(bob.Id, _harness.Ledger.Events.Last(x => x.Event == e).ActorUri);
}
[Fact]
public async Task A_public_reply_to_a_persona_names_its_kind_and_its_features()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var parent = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "first" }, TestContext.Current.CancellationToken);
await _harness.Deliver(bob, "/human-centipede", Create(bob, new[] { Addressing.Public, alice.Uri },
new JsonObject { ["inReplyTo"] = parent.Post.ObjectURI, ["interactionPolicy"] = new JsonObject(), ["quoteUrl"] = "https://elsewhere.example/1" }));
var e = Processed("Create");
Assert.Equal(("accepted", "public", "person"), (e.Outcome, e.Audience, e.LocalKind));
Assert.Contains("interaction-policy", e.Features);
Assert.Contains("legacy-quote", e.Features);
}
[Fact]
public async Task Unwanted_things_are_dropped_with_a_reason()
{
var stranger = new RemoteActor(_harness.Peer, "stranger");
await _harness.Deliver(stranger, "/human-centipede", Create(stranger, new[] { Addressing.Public }));
var unaddressed = Processed("Create");
await _harness.Deliver(stranger, "/human-centipede", new JsonObject
{
["id"] = new Uri(stranger.Id).GetLeftPart(UriPartial.Authority) + "/likes/" + Guid.NewGuid().ToString("N"),
["type"] = "Like",
["actor"] = stranger.Id,
["object"] = $"{Harness.Base}/peasants/x/scribbles/{Guid.NewGuid():N}"
});
var like = Processed("Like");
await _harness.Deliver(stranger, "/human-centipede", new JsonObject
{
["id"] = new Uri(stranger.Id).GetLeftPart(UriPartial.Authority) + "/x/" + Guid.NewGuid().ToString("N"),
["type"] = "Bite",
["actor"] = stranger.Id,
["object"] = stranger.Id
});
var bite = Processed("Bite");
await _harness.Deliver(stranger, "/human-centipede", new JsonObject
{
["id"] = new Uri(stranger.Id).GetLeftPart(UriPartial.Authority) + "/undo/" + Guid.NewGuid().ToString("N"),
["type"] = "Undo",
["actor"] = stranger.Id,
["object"] = new JsonObject { ["id"] = stranger.Id + "/likes/never", ["type"] = "Like", ["actor"] = stranger.Id, ["object"] = "https://nowhere.example/1" }
});
var undo = Processed("Undo");
Assert.Equal(("dropped", "not-addressed", "public"), (unaddressed.Outcome, unaddressed.Reason, unaddressed.Audience));
Assert.Equal(("dropped", "unknown-object"), (like.Outcome, like.Reason));
Assert.Equal(("dropped", "unknown-type"), (bite.Outcome, bite.Reason));
Assert.Equal(("dropped", "unknown-object", "Like"), (undo.Outcome, undo.Reason, undo.Object));
}
[Fact]
public async Task A_stranger_posting_into_a_circle_never_reveals_the_circle()
{
var (_, owner) = await _harness.Persona("owner");
var (privateKey, publicKey) = Keys.NewKeyPair();
var entity = new GroupEntity
{
UserName = $"circle{Guid.NewGuid():N}"[..20],
Kind = GroupKind.Circle,
PostingPolicy = PostingPolicy.Followers,
PrivateKey = privateKey,
PublicKey = publicKey,
Members = new() { new GroupMember { AvatarId = owner.Id, Role = GroupRole.Owner } }
};
await DB.Default.SaveAsync(entity);
var circle = _harness.Local.FromGroup(entity);
var outsider = new RemoteActor(_harness.Peer, "outsider");
await _harness.Deliver(outsider, "/human-centipede", Create(outsider, new[] { circle.Uri, circle.Flock }));
var e = Processed("Create");
Assert.Equal(("dropped", "not-addressed", "private"), (e.Outcome, e.Reason, e.Audience));
Assert.Null(e.LocalKind);
Assert.DoesNotContain(_harness.Ledger.Events, x => System.Text.Json.JsonSerializer.Serialize(x.Event).Contains("circle", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public async Task A_follow_is_accepted_and_its_undo_is_recorded()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var follow = new JsonObject { ["id"] = bob.Id + "/follows/" + Guid.NewGuid().ToString("N"), ["type"] = "Follow", ["actor"] = bob.Id, ["object"] = alice.Uri };
await _harness.Deliver(bob, "/human-centipede", follow);
var followed = Processed("Follow");
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = bob.Id + "/undo/" + Guid.NewGuid().ToString("N"),
["type"] = "Undo",
["actor"] = bob.Id,
["object"] = follow.DeepClone()
});
var undone = Processed("Undo");
Assert.Equal(("accepted", "auto-accepted", "none"), (followed.Outcome, followed.Reason, followed.Audience));
Assert.Equal(("accepted", "undone", "Follow"), (undone.Outcome, undone.Reason, undone.Object));
}
}
}
@@ -0,0 +1,169 @@
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Signing;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Statistics
{
public class SignatureProblemCodeTests
{
[Theory]
[InlineData("unsupported signature algorithm 'ed25519'", "algorithm-unsupported")]
[InlineData("(request-target) is not signed", "headers-unsigned")]
[InlineData("host is not signed", "headers-unsigned")]
[InlineData("the digest is not signed", "headers-unsigned")]
[InlineData("neither date nor (created) is signed", "headers-unsigned")]
[InlineData("the digest does not match the body", "digest-mismatch")]
[InlineData("unreadable Date header", "header-unreadable")]
[InlineData("unreadable (created)", "header-unreadable")]
[InlineData("unreadable (expires)", "header-unreadable")]
[InlineData("the Date header is outside the allowed window", "date-skew")]
[InlineData("(created) is outside the allowed window", "date-skew")]
[InlineData("the signature has expired", "expired")]
[InlineData("something new", "signature-problem")]
public void Every_signature_problem_has_a_reason_code(string problem, string code) =>
Assert.Equal(code, HttpSignatures.ProblemCode(problem));
}
[Trait("Category", "Integration")]
public sealed class InboxAnswerTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
JsonObject Create(RemoteActor sender, string to)
{
var origin = new Uri(sender.Id).GetLeftPart(UriPartial.Authority);
var id = $"{origin}/notes/{Guid.NewGuid():N}";
return new JsonObject
{
["id"] = id + "/activity",
["type"] = "Create",
["actor"] = sender.Id,
["to"] = new JsonArray(to),
["object"] = new JsonObject { ["id"] = id, ["type"] = "Note", ["attributedTo"] = sender.Id, ["to"] = new JsonArray(to), ["content"] = "hi" }
};
}
(InboxResult Result, InteractionEvent Event, string ActorUri, bool Claimed) Last(InboxResult result)
{
var recorded = _harness.Ledger.Events.Last(e => e.Event.Channel == "recv");
return (result, recorded.Event, recorded.ActorUri, recorded.HostClaimed);
}
[Fact]
public async Task A_verified_delivery_is_queued_once_and_then_a_duplicate()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var activity = Create(bob, alice.Uri);
var first = Last(await _harness.Receiver.Receive(bob.Post(Harness.Host, "/human-centipede", activity), default, CancellationToken.None));
var second = Last(await _harness.Receiver.Receive(bob.Post(Harness.Host, "/human-centipede", activity), default, CancellationToken.None));
Assert.Equal(202, first.Event.Status);
Assert.Equal("queued", first.Event.Reason);
Assert.Equal("queued", first.Event.Outcome);
Assert.Equal("127.0.0.1", first.Event.Host);
Assert.Equal("Create", first.Event.Activity);
Assert.Equal("Note", first.Event.Object);
Assert.Equal("shared", first.Event.Inbox);
Assert.Equal("cavage:rsa-sha256", first.Event.Signature);
Assert.True(first.Event.Bytes > 0);
Assert.Equal(bob.Id, first.ActorUri);
Assert.False(first.Claimed);
Assert.Equal("duplicate", second.Event.Reason);
}
[Fact]
public async Task Refusals_carry_their_reason_and_an_unverified_host_is_only_claimed()
{
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var unsigned = mallory.Post(Harness.Host, "/human-centipede", Create(mallory, alice.Uri));
unsigned.Headers.Remove("Signature");
var noSignature = Last(await _harness.Receiver.Receive(unsigned, default, CancellationToken.None));
var tampered = mallory.Post(Harness.Host, "/human-centipede", Create(mallory, alice.Uri));
tampered.Headers["Digest"] = "SHA-256=AAAA";
var badDigest = Last(await _harness.Receiver.Receive(tampered, default, CancellationToken.None));
var forged = mallory.Post(Harness.Host, "/human-centipede", Create(mallory, alice.Uri));
forged.Headers["Signature"] = forged.Headers["Signature"].ToString().Replace("signature=\"", "signature=\"AAAA");
var badSignature = Last(await _harness.Receiver.Receive(forged, default, CancellationToken.None));
var junk = Last(await _harness.Deliver(mallory, "/human-centipede", new JsonArray(1, 2)));
Assert.Equal((401, "no-signature", "none"), (noSignature.Event.Status!.Value, noSignature.Event.Reason, noSignature.Event.Signature));
Assert.Equal((401, "digest-mismatch"), (badDigest.Event.Status!.Value, badDigest.Event.Reason));
Assert.Equal((401, "signature-invalid"), (badSignature.Event.Status!.Value, badSignature.Event.Reason));
Assert.Equal((400, "not-activity"), (junk.Event.Status!.Value, junk.Event.Reason));
foreach (var refused in new[] { noSignature, badDigest, badSignature })
{
Assert.Equal("refused", refused.Event.Outcome);
Assert.True(refused.Claimed);
Assert.Null(refused.ActorUri);
}
}
[Fact]
public async Task A_follow_of_nobody_is_a_404_and_a_self_delete_with_an_unknown_key_is_a_silent_202()
{
var stranger = new RemoteActor(_harness.Peer, "stranger");
var follow = new JsonObject
{
["id"] = stranger.Id + "/follows/" + Guid.NewGuid().ToString("N"),
["type"] = "Follow",
["actor"] = stranger.Id,
["object"] = $"{Harness.Base}/peasants/nobody{Guid.NewGuid():N}"
};
var notFound = Last(await _harness.Receiver.Receive(stranger.Post(Harness.Host, "/human-centipede", follow), default, CancellationToken.None));
var gone = new RemoteActor(_harness.Peer, "gone");
_harness.Peer.Answer(new Uri(gone.Id).AbsolutePath, 410);
var delete = new JsonObject { ["id"] = gone.Id + "#delete", ["type"] = "Delete", ["actor"] = gone.Id, ["object"] = gone.Id };
var deleted = Last(await _harness.Receiver.Receive(gone.Post(Harness.Host, "/human-centipede", delete), default, CancellationToken.None));
Assert.Equal((404, "unknown-recipient"), (notFound.Event.Status!.Value, notFound.Event.Reason));
Assert.Equal((202, "self-delete-unknown-key"), (deleted.Event.Status!.Value, deleted.Event.Reason));
Assert.True(deleted.Claimed);
}
[Fact]
public async Task A_suspended_server_is_recorded_under_its_own_name()
{
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
blocks.Load(new[] { new DomainBlock { Domain = "localhost", Severity = DomainBlockSeverity.Suspend } });
var ledger = new MemoryLedger();
var receiver = new InboxReceiver(_harness.Local, _harness.Remote, _harness.Queue, blocks, NullLogger<InboxReceiver>.Instance, ledger);
var (_, alice) = await _harness.Persona("alice");
var spammer = new RemoteActor(_harness.Peer, "spammer", _harness.Peer.B);
var result = await receiver.Receive(spammer.Post(Harness.Host, "/human-centipede", Create(spammer, alice.Uri)), default, CancellationToken.None);
Assert.Equal(202, result.StatusCode);
var (recorded, actorUri, claimed) = Assert.Single(ledger.Events);
Assert.Equal(("localhost", "suspended", "queued"), (recorded.Host, recorded.Reason, recorded.Outcome));
Assert.False(claimed);
Assert.Null(actorUri);
}
}
}
@@ -0,0 +1,105 @@
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Statistics
{
public class InteractionsTests
{
[Fact]
public void The_local_kind_is_kept_only_on_public_and_unlisted_traffic()
{
Assert.Equal("person", Interactions.Sanitize(new InteractionEvent { Audience = "public", LocalKind = "person" }).LocalKind);
Assert.Equal("group", Interactions.Sanitize(new InteractionEvent { Audience = "unlisted", LocalKind = "group" }).LocalKind);
Assert.Null(Interactions.Sanitize(new InteractionEvent { Audience = "private", LocalKind = "group" }).LocalKind);
Assert.Null(Interactions.Sanitize(new InteractionEvent { Audience = "none", LocalKind = "person" }).LocalKind);
Assert.Null(Interactions.Sanitize(new InteractionEvent { LocalKind = "person" }).LocalKind);
Assert.Null(Interactions.Sanitize(new InteractionEvent { Audience = "public", LocalKind = "circle" }).LocalKind);
}
[Fact]
public void Junk_becomes_other_and_hosts_are_normalised()
{
var e = Interactions.Sanitize(new InteractionEvent
{
Host = "Mastodon.Social.",
Channel = "sideways",
Activity = "https://www.w3.org/ns/activitystreams#Create",
Object = "Note",
Outcome = "Dropped!",
Reason = "not-addressed",
Audience = "everyone",
Inbox = "front-door",
Signature = "cavage:rsa-sha256",
Features = new() { "fep-044f", "Bad Feature", "fep-044f", "source:text/markdown" }
});
Assert.Equal("mastodon.social", e.Host);
Assert.Equal("other", e.Channel);
Assert.Equal("other", e.Activity);
Assert.Equal("Note", e.Object);
Assert.Equal("other", e.Outcome);
Assert.Equal("not-addressed", e.Reason);
Assert.Null(e.Audience);
Assert.Null(e.Inbox);
Assert.Equal("cavage:rsa-sha256", e.Signature);
Assert.Equal(new[] { "fep-044f", "other", "source:text/markdown" }, e.Features);
Assert.Equal("-", Interactions.Host("evil host/with path"));
Assert.Equal("-", Interactions.Host(default));
Assert.Equal("social.example", Interactions.HostOf("https://Social.Example:8443/users/x"));
}
[Fact]
public void Latencies_and_waits_fall_into_buckets()
{
Assert.Equal("le50ms", Interactions.Latency(0));
Assert.Equal("le250ms", Interactions.Latency(101));
Assert.Equal("inf", Interactions.Latency(31_000));
Assert.Equal("le1s", Interactions.Wait(900));
Assert.Equal("le60s", Interactions.Wait(59_000));
Assert.Equal("inf", Interactions.Wait(100_000_000));
}
[Fact]
public void An_actor_hash_is_stable_for_a_salt_and_unlinkable_across_salts()
{
var monday = new byte[32];
var tuesday = Enumerable.Repeat((byte)1, 32).ToArray();
const string actor = "https://social.example/users/alice";
var first = InteractionSalts.Hash(monday, actor);
Assert.Equal(first, InteractionSalts.Hash(monday, actor));
Assert.NotEqual(first, InteractionSalts.Hash(tuesday, actor));
Assert.NotEqual(first, InteractionSalts.Hash(monday, actor + "2"));
Assert.Equal(16, first.Length);
Assert.DoesNotContain("alice", first);
Assert.Null(InteractionSalts.Hash(default, actor));
}
[Fact]
public void A_full_ledger_drops_and_counts_but_never_blocks_or_throws()
{
var ledger = new InteractionLedger(new InteractionSalts(), new StaticOptions<StatisticsOptions>(new StatisticsOptions()), NullLogger<InteractionLedger>.Instance);
for (var i = 0; i < 10_005; i++)
ledger.Record(new InteractionEvent { Channel = Interactions.In });
ledger.Record(default);
Assert.Equal(5, ledger.Dropped);
}
[Fact]
public void A_disabled_ledger_records_nothing()
{
var ledger = new InteractionLedger(new InteractionSalts(), new StaticOptions<StatisticsOptions>(new StatisticsOptions { Enabled = false }), NullLogger<InteractionLedger>.Instance);
for (var i = 0; i < 10_005; i++)
ledger.Record(new InteractionEvent { Channel = Interactions.In });
Assert.Equal(0, ledger.Dropped);
}
}
}
@@ -0,0 +1,113 @@
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Statistics
{
[Trait("Category", "Integration")]
public sealed class LedgerOverHttpTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
JsonObject DirectNote(RemoteActor sender, Persona persona, string text)
{
var noteId = $"{_peer.A}/notes/{Guid.NewGuid():N}";
var to = new JsonArray($"{PrivaPubHost.Base}/peasants/{persona.UserName}");
return new JsonObject
{
["id"] = noteId + "/activity",
["type"] = "Create",
["actor"] = sender.Id,
["to"] = to.DeepClone(),
["object"] = new JsonObject { ["id"] = noteId, ["type"] = "Note", ["attributedTo"] = sender.Id, ["to"] = to.DeepClone(), ["content"] = text }
};
}
async Task<List<BsonDocument>> StoredSince(DateTime since, CancellationToken token)
{
await _host.Get<InteractionLedger>().Flush(token);
return await DB.Default.Database().GetCollection<BsonDocument>(nameof(InteractionEvent))
.Find(Builders<BsonDocument>.Filter.Gte(nameof(InteractionEvent.At), since) & Builders<BsonDocument>.Filter.In(nameof(InteractionEvent.Host), new[] { "127.0.0.1", Interactions.Unknown }))
.ToListAsync(token);
}
[Fact]
public async Task A_delivery_and_its_refusals_are_recorded_naming_only_the_server()
{
var token = TestContext.Current.CancellationToken;
var since = DateTime.UtcNow.AddSeconds(-1);
var persona = await _host.Persona(await _host.SignUp(), "ledger");
var bob = new RemoteActor(_peer, "bob");
using var client = _host.Client();
Assert.Equal(HttpStatusCode.Accepted, (await client.SendAsync(bob.SignedPost($"/peasants/{persona.UserName}/mouth", DirectNote(bob, persona, "psst")), token)).StatusCode);
var forged = bob.SignedPost("/human-centipede", DirectNote(bob, persona, "forged"));
forged.Headers.Remove("Signature");
forged.Headers.TryAddWithoutValidation("Signature", bob.SignedPost("/elsewhere", new JsonObject()).Headers.GetValues("Signature").Single());
Assert.Equal(HttpStatusCode.Unauthorized, (await client.SendAsync(forged, token)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await client.SendAsync(bob.SignedPost($"/peasants/nobody{Guid.NewGuid():N}"[..20] + "/mouth", DirectNote(bob, persona, "lost")), token)).StatusCode);
var stored = await StoredSince(since, token);
var reasons = stored.Select(e => e.GetValue(nameof(InteractionEvent.Reason), BsonNull.Value).ToString()).ToList();
Assert.Contains("queued", reasons);
Assert.Contains("signature-invalid", reasons);
Assert.Contains("unknown-recipient", reasons);
var everything = string.Join("\n", stored.Select(e => e.ToJson()));
Assert.DoesNotContain(persona.UserName, everything);
Assert.DoesNotContain(persona.Id, everything);
Assert.DoesNotContain(persona.Root.Id, everything);
Assert.DoesNotContain(bob.Name, everything);
Assert.DoesNotContain("/notes/", everything);
}
[Fact]
public async Task Client_and_served_traffic_is_counted_per_day_never_per_server()
{
var token = TestContext.Current.CancellationToken;
var persona = await _host.Persona(await _host.SignUp(), "served");
var reader = new RemoteActor(_peer, "reader");
var day = DateTime.UtcNow.Date;
await _host.Get<InteractionLedger>().Flush(token);
var before = await DB.Default.Find<ServerDay>().Match(d => d.Day == day).ExecuteFirstAsync(token) ?? new ServerDay();
using var client = _host.Client();
Assert.Equal(HttpStatusCode.OK, (await client.GetAsync("/api/v1/instance", token)).StatusCode);
using (var actorRequest = new HttpRequestMessage(HttpMethod.Get, $"/peasants/{persona.UserName}"))
{
actorRequest.Headers.Accept.ParseAdd("application/activity+json");
Assert.Equal(HttpStatusCode.OK, (await client.SendAsync(actorRequest, token)).StatusCode);
}
Assert.Equal(HttpStatusCode.OK, (await client.SendAsync(reader.SignedGet($"/peasants/{persona.UserName}/anus"), token)).StatusCode);
await _host.Get<InteractionLedger>().Flush(token);
var after = await DB.Default.Find<ServerDay>().Match(d => d.Day == day).ExecuteFirstAsync(token);
long Grew(Dictionary<string, long> now, Dictionary<string, long> then, string key) => now.GetValueOrDefault(key) - then.GetValueOrDefault(key);
Assert.True(Grew(after.Client, before.Client, "api/v1/instance:GET:2xx") >= 1);
Assert.True(Grew(after.Served, before.Served, "actor:200:unsigned") >= 1);
Assert.True(Grew(after.Served, before.Served, "outbox:200:signed") >= 1);
Assert.DoesNotContain(after.Served.Keys, k => k.Contains(persona.UserName));
}
}
}
@@ -0,0 +1,89 @@
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Statistics
{
[Trait("Category", "Integration")]
public sealed class LedgerStoreTests : IAsyncLifetime
{
readonly string _known = $"known{Guid.NewGuid():N}.example";
readonly string _stranger = $"stranger{Guid.NewGuid():N}.example";
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
await DB.Default.SaveAsync(new RemoteInstance { Host = _known });
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static InteractionLedger Ledger() =>
new(new InteractionSalts(), new StaticOptions<StatisticsOptions>(new StatisticsOptions()), NullLogger<InteractionLedger>.Instance);
[Fact]
public async Task Events_are_stored_with_a_hash_and_never_the_actor()
{
var token = TestContext.Current.CancellationToken;
var ledger = Ledger();
var actor = $"https://{_known}/users/someone{Guid.NewGuid():N}";
ledger.Record(new InteractionEvent { Host = _known, Channel = Interactions.In, Activity = "Create", Object = "Note", Outcome = Interactions.Accepted }, actor);
ledger.Record(new InteractionEvent { Host = _stranger, Channel = Interactions.Receive, Status = 401, Outcome = Interactions.Refused, Reason = "no-signature" }, hostClaimed: true);
ledger.Record(new InteractionEvent { Host = _known, Channel = Interactions.Receive, Status = 401, Outcome = Interactions.Refused, Reason = "signature-invalid" }, hostClaimed: true);
await ledger.Flush(token);
var stored = await DB.Default.Find<InteractionEvent>().Match(e => e.Host == _known).ExecuteAsync(token);
Assert.Equal(2, stored.Count);
var accepted = Assert.Single(stored, e => e.Channel == Interactions.In);
Assert.NotNull(accepted.ActorHash);
Assert.Equal("signature-invalid", Assert.Single(stored, e => e.Channel == Interactions.Receive).Reason);
Assert.False(await DB.Default.Find<InteractionEvent>().Match(e => e.Host == _stranger).ExecuteAnyAsync(token));
var raw = await DB.Default.Database().GetCollection<BsonDocument>(nameof(InteractionEvent)).Find(new BsonDocument("_id", ObjectId.Parse(accepted.ID))).FirstAsync(token);
Assert.DoesNotContain("someone", raw.ToJson());
Assert.DoesNotContain("/users/", raw.ToJson());
}
[Fact]
public async Task Two_processes_agree_on_a_days_salt_and_no_salt_is_made_for_the_past()
{
var token = TestContext.Current.CancellationToken;
var one = await new InteractionSalts().For(DateTime.UtcNow, token);
var two = await new InteractionSalts().For(DateTime.UtcNow, token);
Assert.Equal(Convert.ToBase64String(one), Convert.ToBase64String(two));
Assert.Null(await new InteractionSalts().For(new DateTime(2001, 1, 1, 0, 0, 0, DateTimeKind.Utc), token));
}
[Fact]
public async Task Counters_add_up_per_day()
{
var token = TestContext.Current.CancellationToken;
var ledger = Ledger();
ledger.Count(_known, "media:hit", 1000);
ledger.Count(_known, "media:hit", 500);
ledger.Count(_known, "media.miss");
ledger.CountServer(ServerSections.Client, $"test{_known}:GET:2xx", 120);
ledger.CountServer(ServerSections.Served, $"test{_known}:200:signed");
await ledger.Flush(token);
var day = await DB.Default.Find<InstanceDay>().Match(d => d.Host == _known && d.Day == DateTime.UtcNow.Date).ExecuteFirstAsync(token);
Assert.Equal(2, day.Reads["media:hit"]);
Assert.Equal(1500, day.Reads["media:hit:bytes"]);
Assert.Equal(1, day.Reads["media_miss"]);
var server = await DB.Default.Find<ServerDay>().Match(d => d.Day == DateTime.UtcNow.Date).ExecuteFirstAsync(token);
Assert.Equal(1, server.Client[$"test{_known.Replace('.', '_')}:GET:2xx"]);
Assert.Equal(1, server.Served[$"test{_known.Replace('.', '_')}:200:signed"]);
Assert.True(server.ClientLatency["le250ms"] >= 1);
}
}
}
@@ -0,0 +1,130 @@
using Microsoft.AspNetCore.Http;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Statistics
{
public class HttpScopeTests
{
[Fact]
public async Task Scopes_nest_and_restore_across_awaits()
{
Assert.Null(HttpScope.Purpose);
Assert.Equal("request", HttpScope.Trigger);
using (HttpScope.Triggered("deliver"))
{
using (HttpScope.Default("object"))
{
Assert.Equal("object", HttpScope.Purpose);
using (HttpScope.For("actor"))
{
await Task.Yield();
Assert.Equal("actor", HttpScope.Purpose);
using (HttpScope.Default("object"))
Assert.Equal("actor", HttpScope.Purpose);
}
Assert.Equal("object", HttpScope.Purpose);
}
Assert.Null(HttpScope.Purpose);
Assert.Equal("deliver", HttpScope.Trigger);
Assert.False(HttpScope.Crawling);
using (HttpScope.Crawl())
Assert.True(HttpScope.Crawling);
}
Assert.Equal("request", HttpScope.Trigger);
}
}
public class TrafficMeterTests
{
[Theory]
[InlineData("api/v1/accounts/{id}/follow", "api/v1/accounts")]
[InlineData("/api/v1/statuses", "api/v1/statuses")]
[InlineData("/oauth/token", "oauth/token")]
[InlineData("/clientapi/avatar/private/insert", "clientapi/avatar/private")]
[InlineData("api/{x}", "api")]
[InlineData(null, "unmatched")]
public void Client_routes_group_by_their_first_literal_segments(string template, string group) =>
Assert.Equal(group, TrafficMeter.Group(template));
[Theory]
[InlineData("peasants/{actor}", "/peasants/alice", "actor")]
[InlineData("peasants/{actor}/anus", "/peasants/alice/anus", "outbox")]
[InlineData("peasants/{actor}/flock", "/peasants/c/flock", "collection")]
[InlineData("peasants/{actor}/groupies", "/peasants/alice/groupies", "collection")]
[InlineData("peasants/{actor}/scribbles/{postId}", "/peasants/alice/scribbles/1", "object")]
[InlineData("peasants/{actor}/grunts/{activityId}", "/peasants/alice/grunts/1", "activity")]
[InlineData("peasants/{actor}/parrot-licences/{licenceId}", "/peasants/alice/parrot-licences/1", "licence")]
[InlineData("users/{actor}", "/users/alice", "actor")]
[InlineData(".well-known/webfinger", "/.well-known/webfinger", "webfinger")]
[InlineData("nodeinfo/2.1", "/nodeinfo/2.1", "nodeinfo")]
[InlineData("@{user}", "/@alice", null)]
public void Served_documents_are_counted_by_kind_never_by_name(string template, string path, string kind) =>
Assert.Equal(kind, TrafficMeter.Served(template, new PathString(path)));
}
public sealed class OutboundRequestTests : IAsyncLifetime
{
Peer _peer;
public async ValueTask InitializeAsync() => _peer = await Peer.Start();
public async ValueTask DisposeAsync() => await _peer.DisposeAsync();
[Fact]
public async Task A_federation_fetch_is_recorded_with_its_purpose_trigger_and_size()
{
var ledger = new MemoryLedger();
var http = Peer.Http(ledger: ledger);
_peer.Serve("/users/a", "{\"id\":\"{A}/users/a\",\"type\":\"Person\"}");
using (HttpScope.Triggered("processinbox"))
using (HttpScope.For("actor"))
Assert.NotNull(await http.GetJson($"{_peer.A}/users/a", "application/activity+json", default, TestContext.Current.CancellationToken));
var e = Assert.Single(ledger.Of("http"));
Assert.Equal(("127.0.0.1", "actor", "processinbox", "ok", 200), (e.Host, e.Purpose, e.Trigger, e.Outcome, e.Status!.Value));
Assert.True(e.Bytes > 10);
Assert.Equal(0, e.Redirects);
}
[Fact]
public async Task Refusals_are_recorded_and_a_second_try_is_remembered()
{
var ledger = new MemoryLedger();
var http = Peer.Http(ledger: ledger);
_peer.Answer("/gone", 410);
_peer.ServeText("/page", "<html></html>", "text/html");
using (HttpScope.Triggered("fetchancestors"))
{
await http.GetJson($"{_peer.A}/gone", "application/activity+json", default, TestContext.Current.CancellationToken);
await http.GetJson($"{_peer.A}/gone", "application/activity+json", default, TestContext.Current.CancellationToken);
await http.GetJson($"{_peer.A}/page", "application/activity+json", default, TestContext.Current.CancellationToken);
await http.GetJson("ftp://example.org/x", "application/activity+json", default, TestContext.Current.CancellationToken);
}
var events = ledger.Of("http");
Assert.Equal(new[] { ("refused", "410"), ("refused", "remembered"), ("refused", "content-type"), ("refused", "disallowed") },
events.Select(e => (e.Outcome, e.Reason)));
Assert.Equal("object", events[0].Purpose);
}
[Fact]
public async Task A_fetch_a_reader_caused_is_only_counted()
{
var ledger = new MemoryLedger();
var http = Peer.Http(ledger: ledger);
_peer.Serve("/users/b", "{\"id\":\"{A}/users/b\",\"type\":\"Person\"}");
using (HttpScope.For("webfinger"))
await http.GetJson($"{_peer.A}/users/b", "application/activity+json", default, TestContext.Current.CancellationToken);
Assert.Empty(ledger.Events);
Assert.Equal(1, ledger.Counts[("127.0.0.1", "http:webfinger:ok")]);
}
}
}
+7
View File
@@ -0,0 +1,7 @@
namespace PrivaPub.Tests.Support
{
[CollectionDefinition(nameof(Exclusive), DisableParallelization = true)]
public sealed class Exclusive
{
}
}
+166
View File
@@ -0,0 +1,166 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Localization;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Media;
using PrivaPub.Domain.Relationships;
using PrivaPub.Domain.Social;
using PrivaPub.Domain.Statuses;
using PrivaPub.Domain.Timelines;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using PrivaPub.Resources;
using PrivaPub.Services;
using PrivaPub.StaticServices;
using System.Text.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Support
{
public sealed class Harness : IAsyncDisposable
{
public const string Host = "privapub.test";
public const string Base = "https://" + Host;
readonly DateTime _started = Millisecond(DateTime.UtcNow);
Harness(Peer peer)
{
Peer = peer;
var cache = new MemoryCache(new MemoryCacheOptions());
Local = new LocalActorService(Db, new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = Base }));
Remote = new RemoteActorService(Peer.Http(cache), Local, cache, Db);
Delivery = new DeliveryService(Db, Queue);
Fanout = new Fanout(Db);
Groups = new GroupDistributor(Delivery);
Records = new ObjectRecords(Queue);
Polls = new PollService(Db, Local, Delivery, Queue);
Reactions = new Reactions(Db, Delivery);
RemotePosts = new RemotePosts(Db, Local, Remote, new NoBlocks(), Queue, Records, new NoPreviews());
Outbox = new OutboxPublisher(Db, Local, Delivery);
Quotes = new QuoteService(Db, Remote, RemotePosts, Local, Delivery, Outbox);
Receiver = new InboxReceiver(Local, Remote, Queue, new NoBlocks(), NullLogger<InboxReceiver>.Instance, Ledger);
Handlers = new IActivityHandler[]
{
new FollowHandler(Db, Local, Remote, Delivery),
new AcceptHandler(Db, Local, Quotes),
new RejectHandler(Db, Local, Quotes),
new UndoHandler(Db, Local, Reactions),
new LikeHandler(Db, Reactions),
new EmojiReactHandler(Db, Reactions),
new QuoteRequestHandler(Quotes),
new DislikeHandler(Db),
new JoinHandler(Db, Local, Delivery),
new AnnounceHandler(Db, Local, RemotePosts, Fanout, Remote, Records, Quotes),
new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records, Polls, new NoPreviews(), Quotes),
new DeleteHandler(Db, Local, Remote, Delivery, Groups, Quotes),
new UpdateHandler(Db, Local, Remote, Groups, Records, Quotes),
new FlagHandler(Db, Local)
};
Processor = new InboxProcessor(Remote, Handlers, NullLogger<InboxProcessor>.Instance, Ledger);
Follows = new FollowService(Db, Local, Remote, Delivery, new KeyLocalizer<GenericRes>(), NullLogger<FollowService>.Instance);
Content = new ContentRenderer(Local, Remote);
Media = new MediaService(new StaticOptions<MediaOptions>(new MediaOptions { Root = Path.Combine(Path.GetTempPath(), $"privapub-media-{Guid.NewGuid():N}") }),
Local, default, NullLogger<MediaService>.Instance);
Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups, Polls, new NoPreviews(), Quotes);
Posts = new PostsService(Db, Local, Statuses, new KeyLocalizer<GenericRes>(), NullLogger<PostsService>.Instance);
Timelines = new TimelineService(Db, new KeyLocalizer<GenericRes>());
Relationships = new RelationshipService(Db, Follows, Delivery);
Reports = new ReportService(Db, Local, Delivery);
}
public Peer Peer { get; }
public DbEntities Db { get; } = new();
public MemoryLedger Ledger { get; } = new();
public JobQueue Queue { get; } = new();
public LocalActorService Local { get; }
public RemoteActorService Remote { get; }
public DeliveryService Delivery { get; }
public InboxReceiver Receiver { get; }
public InboxProcessor Processor { get; }
public IReadOnlyList<IActivityHandler> Handlers { get; }
public FollowService Follows { get; }
public ContentRenderer Content { get; }
public OutboxPublisher Outbox { get; }
public PostsService Posts { get; }
public StatusService Statuses { get; }
public MediaService Media { get; }
public GroupDistributor Groups { get; }
public Fanout Fanout { get; }
public RemotePosts RemotePosts { get; }
public ObjectRecords Records { get; }
public PollService Polls { get; }
public Reactions Reactions { get; }
public QuoteService Quotes { get; }
public TimelineService Timelines { get; }
public RelationshipService Relationships { get; }
public ReportService Reports { get; }
public async Task FollowedBy(LocalActor local, RemoteActor follower) =>
await DB.Default.SaveAsync(new Follower
{
LocalActorId = local.Id,
LocalActorKind = local.Kind,
ActorURI = follower.Id,
InboxURL = follower.Id + "/inbox",
SharedInboxURL = follower.SharedInbox
});
public static async Task<Harness> Start() => new(await Peer.Start());
public async Task<(string RootId, LocalActor Avatar)> Persona(string name, string rootId = default)
{
rootId ??= Guid.NewGuid().ToString("N")[..24];
var (privateKey, publicKey) = Keys.NewKeyPair();
var avatar = new Avatar { UserName = $"{name}{Guid.NewGuid():N}"[..20], Name = name, PrivateKey = privateKey, PublicKey = publicKey };
await DB.Default.SaveAsync(avatar);
await DB.Default.SaveAsync(new RootToAvatar { RootId = rootId, AvatarId = avatar.ID });
return (rootId, Local.FromAvatar(avatar));
}
public async Task<InboxResult> Deliver(RemoteActor sender, string path, JsonNode activity)
{
var result = await Receiver.Receive(sender.Post(Host, path, activity), default, CancellationToken.None);
var dedupe = "inbox|" + (activity is JsonObject ? activity["id"]?.GetValue<string>() : default);
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == dedupe).ExecuteFirstAsync();
if (job != default)
Assert.Equal(JobResult.Done, (await Processor.Handle(job, CancellationToken.None)).Result);
return result;
}
public async Task<List<JsonObject>> Outgoing(string inbox) =>
(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.Deliver && j.CreatedAt >= _started).ExecuteAsync())
.Select(j => JsonSerializer.Deserialize<DeliveryPayload>(j.Payload))
.Where(p => p.Inbox == inbox)
.Select(p => JsonNode.Parse(p.Body)!.AsObject())
.ToList();
public async ValueTask DisposeAsync() => await Peer.DisposeAsync();
static DateTime Millisecond(DateTime time) => new(time.Ticks - time.Ticks % TimeSpan.TicksPerMillisecond, DateTimeKind.Utc);
}
public sealed class KeyLocalizer<T> : IStringLocalizer<T>
{
public LocalizedString this[string name] => new(name, name);
public LocalizedString this[string name, params object[] arguments] => new(name, string.Format(name, arguments));
public IEnumerable<LocalizedString> GetAllStrings(bool includeParentCultures) => Enumerable.Empty<LocalizedString>();
}
public sealed class NoPreviews : ILinkPreviews
{
public Task Wanted(PrivaPub.Models.Post.Post post, CancellationToken token) => Task.CompletedTask;
}
}
+117
View File
@@ -0,0 +1,117 @@
using PrivaPub.Infrastructure.Cli;
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json.Nodes;
using System.Text.RegularExpressions;
namespace PrivaPub.Tests.Support.Host
{
public sealed record Root(string Id, string UserName, string Password, string Jwt);
public sealed record Persona(string Id, string UserName, Root Root);
public static partial class Accounts
{
public const string Password = "Test-Pass-1!";
const string OutOfBand = "urn:ietf:wg:oauth:2.0:oob";
public static async Task<Root> SignUp(this PrivaPubHost host, string name = "root")
{
var userName = $"{name}{Guid.NewGuid():N}"[..24];
using var client = host.Client();
var response = await client.PostAsJsonAsync("/clientapi/user/signup", new { userName, password = Password });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var jwt = (await response.Content.ReadFromJsonAsync<JsonObject>())!;
return new Root(jwt["userId"]!.GetValue<string>(), userName, Password, jwt["token"]!.GetValue<string>());
}
public static async Task<Root> LogIn(this PrivaPubHost host, Root root)
{
using var client = host.Client();
var response = await client.PostAsJsonAsync("/clientapi/user/login", new { userName = root.UserName, password = root.Password });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var jwt = (await response.Content.ReadFromJsonAsync<JsonObject>())!;
return root with { Jwt = jwt["token"]!.GetValue<string>() };
}
public static async Task<Root> Admin(this PrivaPubHost host)
{
var root = await host.SignUp("admin");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }));
return await host.LogIn(root);
}
public static async Task<Persona> Persona(this PrivaPubHost host, Root root, string name = "persona")
{
var userName = $"{name}{Guid.NewGuid():N}"[..20];
using var client = host.As(root.Jwt);
var response = await client.PostAsJsonAsync("/clientapi/avatar/private/insert", new { userName, name, biography = "testing" });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var avatar = (await response.Content.ReadFromJsonAsync<JsonObject>())!;
return new Persona(avatar["id"]!.GetValue<string>(), userName, root);
}
public static async Task<string> MastodonToken(this PrivaPubHost host, Persona persona, string scopes = "read write follow")
{
using var client = host.Client(cookies: true);
var app = await Form(client, "/api/v1/apps", ("client_name", "privapub-tests"), ("redirect_uris", OutOfBand), ("scopes", scopes));
var clientId = app["client_id"]!.GetValue<string>();
var clientSecret = app["client_secret"]!.GetValue<string>();
var query = $"client_id={Uri.EscapeDataString(clientId)}&redirect_uri={Uri.EscapeDataString(OutOfBand)}&response_type=code&scope={Uri.EscapeDataString(scopes)}";
var code = await Authorize(client, persona, query);
var token = await Form(client, "/oauth/token", ("grant_type", "authorization_code"), ("code", code), ("client_id", clientId),
("client_secret", clientSecret), ("redirect_uri", OutOfBand));
return token["access_token"]!.GetValue<string>();
}
public static async Task<string> Authorize(HttpClient client, Persona persona, string query, string decision = "allow")
{
var returnUrl = "/oauth/authorize?" + query;
var login = await client.GetStringAsync("/oauth/login?returnUrl=" + Uri.EscapeDataString(returnUrl));
var antiforgery = AntiforgeryToken().Match(login).Groups[1].Value;
var signedIn = await client.PostAsync("/oauth/login", new FormUrlEncodedContent(new Dictionary<string, string>
{
["returnUrl"] = returnUrl,
["__RequestVerificationToken"] = antiforgery,
["userName"] = persona.Root.UserName,
["password"] = persona.Root.Password
}));
Assert.Equal(HttpStatusCode.Redirect, signedIn.StatusCode);
var choose = await client.GetStringAsync(returnUrl + "&signed_in=1");
var fields = HiddenInput().Matches(choose).Select(m => new KeyValuePair<string, string>(m.Groups[1].Value, WebUtility.HtmlDecode(m.Groups[2].Value))).ToList();
fields.Add(new("avatarId", persona.Id));
fields.Add(new("decision", decision));
var answer = await client.PostAsync("/oauth/authorize", new FormUrlEncodedContent(fields));
var page = await answer.Content.ReadAsStringAsync();
return Code().Match(page) is { Success: true } match ? match.Groups[1].Value : default;
}
public static HttpClient As(this PrivaPubHost host, string bearer)
{
var client = host.Client();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", bearer);
return client;
}
static async Task<JsonObject> Form(HttpClient client, string path, params (string Key, string Value)[] fields)
{
var response = await client.PostAsync(path, new FormUrlEncodedContent(fields.Select(f => new KeyValuePair<string, string>(f.Key, f.Value))));
var body = await response.Content.ReadAsStringAsync();
Assert.True(response.IsSuccessStatusCode, $"{path} answered {(int)response.StatusCode}: {body}");
return JsonNode.Parse(body)!.AsObject();
}
[GeneratedRegex("name=\"__RequestVerificationToken\" type=\"hidden\" value=\"([^\"]*)\"")]
private static partial Regex AntiforgeryToken();
[GeneratedRegex("<input type=\"hidden\" name=\"([^\"]*)\" value=\"([^\"]*)\"")]
private static partial Regex HiddenInput();
[GeneratedRegex("<code>([^<]*)</code>")]
private static partial Regex Code();
}
}
+64
View File
@@ -0,0 +1,64 @@
using Microsoft.Extensions.DependencyInjection;
using MongoDB.Entities;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using System.Linq.Expressions;
using System.Text.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Support.Host
{
public static class Jobs
{
public static async Task<int> Run(this PrivaPubHost host, Expression<Func<Job, bool>> which, CancellationToken token = default)
{
var handlers = host.Services.GetServices<IJobHandler>().ToDictionary(h => h.Kind);
var queue = host.Get<IJobQueue>();
var ran = 0;
for (var round = 0; round < 50; round++)
{
var pending = await DB.Default.Find<Job>().Match(which).Match(j => j.State == JobState.Pending).ExecuteAsync(token);
if (pending.Count == 0)
return ran;
foreach (var candidate in pending)
{
var job = await DB.Default.UpdateAndGet<Job>()
.Match(j => j.ID == candidate.ID && j.State == JobState.Pending)
.Modify(j => j.State, JobState.Running)
.Modify(j => j.LeasedUntil, DateTime.UtcNow + JobQueue.LeaseTime)
.Modify(b => b.Inc(j => j.Attempts, 1))
.ExecuteAsync(token);
if (job == default)
continue;
var handler = handlers[job.Kind];
JobOutcome outcome;
try
{
outcome = await handler.Handle(job, token);
}
catch (Exception ex)
{
outcome = JobOutcome.Dead(ex.GetType().Name + ": " + ex.Message);
}
await queue.Finish(job, outcome, handler.MaxAttempts, token);
ran++;
}
}
return ran;
}
public static Task<int> RunInbox(this PrivaPubHost host, string activityId, CancellationToken token = default) =>
host.Run(j => j.Kind == JobKind.ProcessInbox && j.DedupeKey == "inbox|" + activityId, token);
public static async Task<List<JsonObject>> Deliveries(string inbox, DateTime since, CancellationToken token = default) =>
(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.Deliver && j.CreatedAt >= since).ExecuteAsync(token))
.Select(j => JsonSerializer.Deserialize<DeliveryPayload>(j.Payload))
.Where(p => p.Inbox == inbox)
.Select(p => JsonNode.Parse(p.Body)!.AsObject())
.ToList();
}
}
+160
View File
@@ -0,0 +1,160 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.DataProtection;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using PrivaPub.Api.Mastodon.Auth;
using PrivaPub.Domain.Media;
using PrivaPub.Infrastructure.Jobs;
using System.Net;
namespace PrivaPub.Tests.Support.Host
{
public class PrivaPubHost : WebApplicationFactory<Program>
{
public const string Host = "privapub.test";
public const string Base = "https://" + Host;
public const string ClientHeader = "X-Test-Client";
static readonly SemaphoreSlim Boot = new(1, 1);
static readonly Type[] Unwanted = { typeof(JobWorker), typeof(MediaJanitor), typeof(OAuthPruner) };
static PrivaPubHost _shared;
readonly string _mediaRoot = Path.Combine(Path.GetTempPath(), $"privapub-tests-{Guid.NewGuid():N}");
public IReadOnlyList<ServiceDescriptor> Registered { get; private set; }
public static async Task<PrivaPubHost> Shared()
{
await Boot.WaitAsync();
try
{
if (_shared == default)
{
var host = new PrivaPubHost();
_ = host.Services;
_shared = host;
}
return _shared;
}
finally
{
Boot.Release();
}
}
protected virtual IEnumerable<KeyValuePair<string, string>> Settings() => new Dictionary<string, string>
{
["MongoSettings:ConnectionString"] = MongoFixture.Connection,
["MongoSettings:Database"] = MongoFixture.Database,
["MongoSettings:LogsDatabase"] = "logs",
["AppConfiguration:Version"] = "0.0.0-test",
["AppConfiguration:BackendBaseAddress"] = Base,
["AppConfiguration:MaxAllowedUploadFiles"] = "3",
["AppConfiguration:MaxAllowedFileSize"] = "2097152",
["AppConfiguration:SupportedLanguages:0"] = "en",
["AppConfiguration:SupportedLanguages:1"] = "it",
["AppConfiguration:HashingOptions:Iterations"] = "1000",
["AppConfiguration:Jwt:Key"] = "privapub-tests-only-signing-key-0123456789abcdef0123456789abcdef",
["AppConfiguration:Jwt:Issuer"] = Base,
["AppConfiguration:Jwt:Audience"] = Base,
["AppConfiguration:Jwt:HoursTimeout"] = "1",
["AppConfiguration:EmailConfiguration:SmtpServer"] = "127.0.0.1",
["AppConfiguration:EmailConfiguration:SmtpPort"] = "9",
["AppConfiguration:EmailConfiguration:UseSSL"] = "false",
["AppConfiguration:EmailConfiguration:SmtpUsername"] = "nobody@privapub.test",
["AppConfiguration:EmailConfiguration:SmtpPassword"] = "none",
["Federation:AllowPrivateNetworks"] = "true",
["Federation:AllowPlainHttp"] = "true",
["Federation:FetchLinkPreviews"] = "false",
["Media:Root"] = _mediaRoot,
["Logging:LogLevel:Default"] = "Warning",
["Serilog:MinimumLevel:Default"] = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_LOGS") == "1" ? "Information" : "Fatal"
};
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Testing");
foreach (var (key, value) in Settings())
builder.UseSetting(key, value);
builder.ConfigureTestServices(services =>
{
foreach (var hosted in services.Where(s => s.ServiceType == typeof(IHostedService) && Unwanted.Contains(s.ImplementationType)).ToList())
services.Remove(hosted);
services.AddDataProtection().UseEphemeralDataProtectionProvider();
services.AddSingleton<IStartupFilter, ClientAddressFilter>();
Registered = services.ToList();
});
}
protected override void ConfigureClient(HttpClient client)
{
client.BaseAddress = new Uri(Base + "/");
client.DefaultRequestHeaders.Add(ClientHeader, $"10.{Random.Shared.Next(256)}.{Random.Shared.Next(256)}.{Random.Shared.Next(1, 255)}");
}
public HttpClient Client(bool cookies = false) => CreateClient(new WebApplicationFactoryClientOptions
{
BaseAddress = new Uri(Base + "/"),
AllowAutoRedirect = false,
HandleCookies = cookies
});
public T Get<T>() where T : notnull => Services.GetRequiredService<T>();
protected override void Dispose(bool disposing)
{
base.Dispose(disposing);
if (disposing && Directory.Exists(_mediaRoot))
Directory.Delete(_mediaRoot, recursive: true);
}
sealed class ClientAddressFilter : IStartupFilter
{
public Action<IApplicationBuilder> Configure(Action<IApplicationBuilder> next) => app =>
{
app.Use(async (context, call) =>
{
if (IPAddress.TryParse(context.Request.Headers[ClientHeader].ToString(), out var address))
context.Connection.RemoteIpAddress = address;
await call(context);
});
next(app);
};
}
}
public sealed class SecureModeHost : PrivaPubHost
{
static readonly SemaphoreSlim Boot = new(1, 1);
static SecureModeHost _shared;
public static new async Task<SecureModeHost> Shared()
{
await PrivaPubHost.Shared();
await Boot.WaitAsync();
try
{
if (_shared == default)
{
var host = new SecureModeHost();
_ = host.Services;
_shared = host;
}
return _shared;
}
finally
{
Boot.Release();
}
}
protected override IEnumerable<KeyValuePair<string, string>> Settings() =>
base.Settings().Append(new KeyValuePair<string, string>("Federation:SecureMode", "true"));
}
}
+27
View File
@@ -0,0 +1,27 @@
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Statistics;
using System.Collections.Concurrent;
namespace PrivaPub.Tests.Support
{
public sealed class MemoryLedger : IInteractionLedger
{
public ConcurrentQueue<(InteractionEvent Event, string ActorUri, bool HostClaimed)> Events { get; } = new();
public ConcurrentDictionary<(string Host, string Key), long> Counts { get; } = new();
public ConcurrentDictionary<(string Section, string Key), long> ServerCounts { get; } = new();
public long Dropped => 0;
public void Record(InteractionEvent interaction, string actorUri = default, bool hostClaimed = false) =>
Events.Enqueue((Interactions.Sanitize(interaction), actorUri, hostClaimed));
public void Count(string host, string key, long bytes = 0) =>
Counts.AddOrUpdate((Interactions.Host(host), key), 1, (_, count) => count + 1);
public void CountServer(string section, string key, int? latencyMs = default) =>
ServerCounts.AddOrUpdate((section, key), 1, (_, count) => count + 1);
public IReadOnlyList<InteractionEvent> Of(string channel) => Events.Select(e => e.Event).Where(e => e.Channel == channel).ToList();
}
}
+41 -3
View File
@@ -13,15 +13,24 @@ namespace PrivaPub.Tests.Support
public sealed class MongoFixture : IAsyncLifetime
{
public const string Skip = "set PRIVAPUB_TEST_MONGOD=1 (and optionally PRIVAPUB_TEST_MONGO) to run the tests that need a mongod";
const int ProductionPort = 27022;
const int DefaultPort = 27017;
public static bool Enabled => Environment.GetEnvironmentVariable("PRIVAPUB_TEST_MONGOD") == "1";
static bool Required => Environment.GetEnvironmentVariable("PRIVAPUB_TEST_REQUIRE_MONGOD") == "1";
static bool InCi => !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("CI")) || !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITEA_ACTIONS"));
public string Database { get; } = $"PrivaPubTests_{Guid.NewGuid():N}";
public static string Connection { get; } = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_MONGO") ?? $"mongodb://127.0.0.1:{DefaultPort}";
public static string Database { get; } = $"PrivaPubTests_{Guid.NewGuid():N}";
public async ValueTask InitializeAsync()
{
if (Required && !Enabled)
throw new InvalidOperationException("PRIVAPUB_TEST_REQUIRE_MONGOD=1 but PRIVAPUB_TEST_MONGOD is not 1: the integration tests would be skipped");
if (!Enabled)
return;
var settings = MongoClientSettings.FromConnectionString(Connection);
Refuse(settings);
try
{
BsonSerializer.RegisterSerializer(new GuidSerializer(GuidRepresentation.Standard));
@@ -29,9 +38,11 @@ namespace PrivaPub.Tests.Support
catch (BsonSerializationException)
{
}
var connection = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_MONGO") ?? "mongodb://127.0.0.1:27017";
await DB.InitAsync(Database, MongoClientSettings.FromConnectionString(connection));
await DB.InitAsync(Database, settings);
await RefuseProductionData();
EntityMaps.Warm();
await DB.Default.MigrateAsync<Program>();
await Indexes.Create();
}
public async ValueTask DisposeAsync()
@@ -39,5 +50,32 @@ namespace PrivaPub.Tests.Support
if (Enabled)
await DB.Default.Database().Client.DropDatabaseAsync(Database);
}
static void Refuse(MongoClientSettings settings)
{
var ports = settings.Servers.Select(s => s.Port).ToList();
if (ports.Contains(ProductionPort))
throw new InvalidOperationException($"the tests never run against port {ProductionPort}: that is production's mongod");
if (InCi && (Environment.GetEnvironmentVariable("PRIVAPUB_TEST_MONGO") == default || ports.Contains(DefaultPort)))
throw new InvalidOperationException("in CI the tests run only against a throwaway mongod: use tools/ci/with-test-mongod.sh");
}
static async Task RefuseProductionData()
{
BsonDocument options;
try
{
options = await DB.Default.Database().Client.GetDatabase("admin").RunCommandAsync<BsonDocument>(new BsonDocument("getCmdLineOpts", 1));
}
catch (MongoCommandException)
{
return;
}
var dbPath = options.GetValue("parsed", new BsonDocument()).AsBsonDocument
.GetValue("storage", new BsonDocument()).AsBsonDocument
.GetValue("dbPath", BsonNull.Value);
if (dbPath.IsString && dbPath.AsString.StartsWith("/var/lib/privapub", StringComparison.Ordinal))
throw new InvalidOperationException($"the mongod at {Connection} keeps its data in {dbPath.AsString}: that is production's");
}
}
}
+43 -8
View File
@@ -6,7 +6,10 @@ using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using PrivaPub.Federation.Moderation;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Federation;
using System.Collections.Concurrent;
@@ -15,7 +18,9 @@ namespace PrivaPub.Tests.Support
public sealed class Peer : IAsyncDisposable
{
readonly WebApplication _app;
readonly ConcurrentDictionary<string, string> _documents = new();
readonly ConcurrentDictionary<string, (string Text, string ContentType)> _documents = new();
readonly ConcurrentDictionary<string, (int Status, TimeSpan Delay)> _answers = new();
readonly ConcurrentDictionary<string, (byte[] Bytes, string ContentType)> _files = new();
public int Port { get; }
public string A => $"http://127.0.0.1:{Port}";
@@ -36,24 +41,46 @@ namespace PrivaPub.Tests.Support
Peer peer = default;
app.Run(async context =>
{
peer.Requests.Enqueue(new(context.Request.Method, context.Request.Path, context.Request.Headers["Signature"].ToString()));
var body = context.Request.ContentLength > 0 || context.Request.Headers.TransferEncoding.Count > 0
? await new StreamReader(context.Request.Body).ReadToEndAsync()
: default;
peer.Requests.Enqueue(new(context.Request.Method, context.Request.Path, context.Request.Headers["Signature"].ToString(), body,
context.Request.Headers.ToDictionary(h => h.Key, h => h.Value.ToString(), StringComparer.OrdinalIgnoreCase)));
var key = context.Request.Path.Value;
if (peer._files.TryGetValue(key, out var file))
{
await Results.Bytes(file.Bytes, file.ContentType, enableRangeProcessing: true).ExecuteAsync(context);
return;
}
if (peer._answers.TryGetValue(key, out var answer))
{
if (answer.Delay > TimeSpan.Zero)
await Task.Delay(answer.Delay);
context.Response.StatusCode = answer.Status;
return;
}
if (!peer._documents.TryGetValue(key, out var document))
{
context.Response.StatusCode = StatusCodes.Status404NotFound;
return;
}
context.Response.ContentType = "application/activity+json";
await context.Response.WriteAsync(document.Replace("{A}", peer.A).Replace("{B}", peer.B));
context.Response.ContentType = document.ContentType;
await context.Response.WriteAsync(document.Text.Replace("{A}", peer.A).Replace("{B}", peer.B));
});
await app.StartAsync();
peer = new Peer(app, new Uri(app.Urls.First()).Port);
return peer;
}
public void Serve(string path, string json) => _documents[path] = json;
public void Serve(string path, string json) => _documents[path] = (json, "application/activity+json");
public static FederationHttp Http(IMemoryCache cache = default)
public void ServeText(string path, string text, string contentType) => _documents[path] = (text, contentType);
public void ServeFile(string path, byte[] bytes, string contentType) => _files[path] = (bytes, contentType);
public void Answer(string path, int status, TimeSpan delay = default) => _answers[path] = (status, delay);
public static FederationHttp Http(IMemoryCache cache = default, IDomainBlocks blocks = default, IInteractionLedger ledger = default)
{
var options = new FederationOptions { AllowPrivateNetworks = true, AllowPlainHttp = true };
var services = new ServiceCollection();
@@ -61,13 +88,21 @@ namespace PrivaPub.Tests.Support
.ConfigurePrimaryHttpMessageHandler(() => SafeHttpHandlerFactory.Create(options));
return new FederationHttp(services.BuildServiceProvider().GetRequiredService<IHttpClientFactory>(),
cache ?? new MemoryCache(new MemoryCacheOptions()), new StaticOptions<FederationOptions>(options),
NullLogger<FederationHttp>.Instance);
blocks ?? new NoBlocks(), NullLogger<FederationHttp>.Instance, ledger);
}
public async ValueTask DisposeAsync() => await _app.DisposeAsync();
}
public sealed record HttpRequestRecord(string Method, string Path, string Signature);
public sealed class NoBlocks : IDomainBlocks
{
public DomainBlock Find(string host) => default;
public bool IsSuspended(string host) => false;
public Task Reload(CancellationToken token) => Task.CompletedTask;
}
public sealed record HttpRequestRecord(string Method, string Path, string Signature, string Body = default,
IReadOnlyDictionary<string, string> Headers = default);
public sealed class StaticOptions<T> : IOptionsMonitor<T>
{
+50 -6
View File
@@ -14,21 +14,25 @@ namespace PrivaPub.Tests.Support
{
readonly RSA _key = RSA.Create(2048);
public RemoteActor(Peer peer, string name, string origin = default)
public RemoteActor(Peer peer, string name, string origin = default, string type = "Person")
{
Name = $"{name}{Guid.NewGuid():N}"[..20];
Id = $"{origin ?? peer.A}/users/{Name}";
Type = type;
peer.Serve($"/users/{Name}", Document().ToJsonString());
}
public string Type { get; }
public string Name { get; }
public string Id { get; }
public string KeyId => Id + "#main-key";
public string SharedInbox => Id.Split("/users/")[0] + "/inbox";
public JsonObject Document() => new()
{
["id"] = Id,
["type"] = "Person",
["type"] = Type,
["preferredUsername"] = Name,
["inbox"] = Id + "/inbox",
["followers"] = Id + "/followers",
@@ -40,14 +44,54 @@ namespace PrivaPub.Tests.Support
}
};
public HttpRequestMessage SignedGet(string path, string host = "privapub.test")
{
var date = DateTimeOffset.UtcNow.ToString("r", CultureInfo.InvariantCulture);
var request = new HttpRequestMessage(HttpMethod.Get, path);
request.Headers.TryAddWithoutValidation("Date", date);
request.Headers.TryAddWithoutValidation("Accept", "application/activity+json");
request.Headers.TryAddWithoutValidation("Signature", Signature($"(request-target): get {path}\nhost: {host}\ndate: {date}", "(request-target) host date"));
return request;
}
public HttpRequestMessage SignedPost(string path, JsonNode activity, string host = "privapub.test", string date = default)
{
var body = Encoding.UTF8.GetBytes(activity.ToJsonString());
date ??= DateTimeOffset.UtcNow.ToString("r", CultureInfo.InvariantCulture);
var digest = HttpSignatures.Digest(body);
var request = new HttpRequestMessage(HttpMethod.Post, path) { Content = new ByteArrayContent(body) };
request.Content.Headers.TryAddWithoutValidation("Content-Type", "application/activity+json");
request.Headers.TryAddWithoutValidation("Date", date);
request.Headers.TryAddWithoutValidation("Digest", digest);
request.Headers.TryAddWithoutValidation("Signature",
Signature($"(request-target): post {path}\nhost: {host}\ndate: {date}\ndigest: {digest}", "(request-target) host date digest"));
return request;
}
string Signature(string signingString, string headers)
{
var signature = Convert.ToBase64String(_key.SignData(Encoding.UTF8.GetBytes(signingString), HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
return $"keyId=\"{KeyId}\",algorithm=\"rsa-sha256\",headers=\"{headers}\",signature=\"{signature}\"";
}
public HttpRequest Get(string host, string path)
{
var date = DateTimeOffset.UtcNow.ToString("r", CultureInfo.InvariantCulture);
var context = new DefaultHttpContext();
context.Request.Method = "GET";
context.Request.Host = new HostString(host);
context.Request.Path = path;
context.Features.Get<IHttpRequestFeature>().RawTarget = path;
context.Request.Headers["Date"] = date;
context.Request.Headers["Signature"] = Signature($"(request-target): get {path}\nhost: {host}\ndate: {date}", "(request-target) host date");
return context.Request;
}
public HttpRequest Post(string host, string path, JsonNode activity)
{
var body = Encoding.UTF8.GetBytes(activity.ToJsonString());
var date = DateTimeOffset.UtcNow.ToString("r", CultureInfo.InvariantCulture);
var digest = HttpSignatures.Digest(body);
var signingString = $"(request-target): post {path}\nhost: {host}\ndate: {date}\ndigest: {digest}";
var signature = Convert.ToBase64String(_key.SignData(Encoding.UTF8.GetBytes(signingString), HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
var context = new DefaultHttpContext();
context.Request.Method = "POST";
context.Request.Host = new HostString(host);
@@ -56,7 +100,7 @@ namespace PrivaPub.Tests.Support
context.Request.Headers["Date"] = date;
context.Request.Headers["Digest"] = digest;
context.Request.Headers["Content-Type"] = "application/activity+json";
context.Request.Headers["Signature"] = $"keyId=\"{KeyId}\",algorithm=\"rsa-sha256\",headers=\"(request-target) host date digest\",signature=\"{signature}\"";
context.Request.Headers["Signature"] = Signature($"(request-target): post {path}\nhost: {host}\ndate: {date}\ndigest: {digest}", "(request-target) host date digest");
context.Request.Body = new MemoryStream(body);
context.Request.ContentLength = body.Length;
return context.Request;
@@ -0,0 +1,37 @@
namespace PrivaPub.Api.Mastodon.Auth
{
public static class MastodonScopes
{
public static readonly string[] All =
{
"read", "write", "follow", "push", "profile",
"read:accounts", "read:blocks", "read:bookmarks", "read:favourites", "read:filters", "read:follows", "read:lists",
"read:mutes", "read:notifications", "read:search", "read:statuses",
"write:accounts", "write:blocks", "write:bookmarks", "write:conversations", "write:favourites", "write:filters",
"write:follows", "write:lists", "write:media", "write:mutes", "write:notifications", "write:reports", "write:statuses"
};
public static IReadOnlyList<string> Parse(string scopes)
{
var parsed = (scopes ?? string.Empty).Split(new[] { ' ', '+', ',' }, StringSplitOptions.RemoveEmptyEntries)
.Where(s => All.Contains(s))
.Distinct()
.ToList();
return parsed.Count == 0 ? new[] { "read" } : parsed;
}
public static bool Grants(IEnumerable<string> granted, string required)
{
var scopes = granted as ICollection<string> ?? granted.ToList();
if (scopes.Contains(required))
return true;
var colon = required.IndexOf(':');
if (colon > 0 && scopes.Contains(required[..colon]))
return true;
return required.StartsWith("read:follows") || required.StartsWith("write:follows") || required.StartsWith("write:blocks")
|| required.StartsWith("read:blocks") || required.StartsWith("write:mutes") || required.StartsWith("read:mutes")
? scopes.Contains("follow")
: false;
}
}
}
+44
View File
@@ -0,0 +1,44 @@
using OpenIddict.Abstractions;
namespace PrivaPub.Api.Mastodon.Auth
{
public class OAuthPruner : BackgroundService
{
static readonly TimeSpan Interval = TimeSpan.FromHours(6);
static readonly TimeSpan Threshold = TimeSpan.FromDays(14);
readonly IServiceProvider _services;
readonly ILogger<OAuthPruner> _logger;
public OAuthPruner(IServiceProvider services, ILogger<OAuthPruner> logger)
{
_services = services;
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
await Task.Delay(Interval, stoppingToken);
using var scope = _services.CreateScope();
var threshold = DateTimeOffset.UtcNow - Threshold;
var tokens = await scope.ServiceProvider.GetRequiredService<IOpenIddictTokenManager>().PruneAsync(threshold, stoppingToken);
var authorizations = await scope.ServiceProvider.GetRequiredService<IOpenIddictAuthorizationManager>().PruneAsync(threshold, stoppingToken);
if (tokens + authorizations > 0)
_logger.LogInformation("{Service} removed {Tokens} tokens and {Authorizations} authorizations", nameof(OAuthPruner), tokens, authorizations);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "{Service} pass failed", nameof(OAuthPruner));
}
}
}
}
}
+112
View File
@@ -0,0 +1,112 @@
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Authentication.Cookies;
using Microsoft.Extensions.Options;
using Microsoft.IdentityModel.Tokens;
using MongoDB.Driver;
using MongoDB.Entities;
using OpenIddict.Abstractions;
using OpenIddict.Server;
using OpenIddict.Server.AspNetCore;
using PrivaPub.Models;
using System.Net;
using static OpenIddict.Server.OpenIddictServerEvents;
namespace PrivaPub.Api.Mastodon.Auth
{
public static class OAuthSetup
{
public const string LoginScheme = "PrivaPub.OAuth";
public const string OutOfBand = "urn:ietf:wg:oauth:2.0:oob";
public static IServiceCollection PrivaPubOAuth(this IServiceCollection services, IWebHostEnvironment environment)
{
services.AddSingleton<IMongoDatabase>(_ => DB.Default.Database());
services.AddHostedService<OAuthPruner>();
services.AddAuthentication()
.AddCookie(LoginScheme, options =>
{
options.Cookie.Name = "privapub.oauth";
options.Cookie.Path = "/oauth";
options.Cookie.HttpOnly = true;
options.Cookie.SameSite = SameSiteMode.Lax;
options.Cookie.SecurePolicy = CookieSecurePolicy.SameAsRequest;
options.ExpireTimeSpan = TimeSpan.FromMinutes(15);
options.SlidingExpiration = false;
options.LoginPath = "/oauth/login";
});
services.AddOpenIddict()
.AddCore(options => options.UseMongoDb())
.AddServer(options =>
{
options.SetAuthorizationEndpointUris("/oauth/authorize")
.SetTokenEndpointUris("/oauth/token")
.SetRevocationEndpointUris("/oauth/revoke")
.SetConfigurationEndpointUris("/.well-known/openid-configuration", "/.well-known/oauth-authorization-server");
options.AllowAuthorizationCodeFlow().AllowClientCredentialsFlow();
options.RegisterScopes(MastodonScopes.All);
options.UseReferenceAccessTokens();
options.SetAccessTokenLifetime(null);
options.SetAuthorizationCodeLifetime(TimeSpan.FromMinutes(10));
options.DisableAccessTokenEncryption();
var aspNetCore = options.UseAspNetCore()
.EnableAuthorizationEndpointPassthrough()
.EnableTokenEndpointPassthrough();
if (!environment.IsProduction())
aspNetCore.DisableTransportSecurityRequirement();
options.AddEventHandler<ApplyAuthorizationResponseContext>(builder => builder.UseInlineHandler(OutOfBandCode)
.SetOrder(OpenIddictServerAspNetCoreHandlers.Authentication.ProcessFormPostResponse.Descriptor.Order - 1_000)
.SetType(OpenIddictServerHandlerType.Custom));
options.AddEventHandler<ApplyTokenResponseContext>(builder => builder.UseInlineHandler(context =>
{
if (context.Response.AccessToken != default)
context.Response["created_at"] = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
return default;
}));
})
.AddValidation(options =>
{
options.UseLocalServer();
options.UseAspNetCore();
});
services.AddOptions<OpenIddictServerOptions>().Configure<IOptionsMonitor<AppConfiguration>>((options, app) =>
{
options.Issuer = new Uri(app.CurrentValue.BackendBaseAddress.TrimEnd('/') + "/");
options.SigningCredentials.Add(new SigningCredentials(OidcKey.Load(OidcKey.Signing), SecurityAlgorithms.RsaSha256));
options.EncryptionCredentials.Add(new EncryptingCredentials(OidcKey.Load(OidcKey.Encryption),
SecurityAlgorithms.RsaOAEP, SecurityAlgorithms.Aes256CbcHmacSha512));
});
return services;
}
static async ValueTask OutOfBandCode(ApplyAuthorizationResponseContext context)
{
if (context.RedirectUri != OutOfBand || context.Response.Code == default)
return;
var http = context.Transaction.GetHttpRequest()?.HttpContext;
if (http == default)
return;
var code = WebUtility.HtmlEncode(context.Response.Code);
http.Response.StatusCode = StatusCodes.Status200OK;
http.Response.ContentType = "text/html; charset=utf-8";
http.Response.Headers["Content-Security-Policy"] = "default-src 'none'; style-src 'unsafe-inline'";
http.Response.Headers["Cache-Control"] = "no-store";
await http.Response.WriteAsync($$"""
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
<title>Authorization code</title><style>body{font:16px/1.5 system-ui,sans-serif;max-width:480px;margin:48px auto;padding:0 16px}
code{display:block;font-size:1.1rem;padding:12px;border:1px solid #ccc;word-break:break-all}</style></head>
<body><h1>Authorization code</h1><p>Copy this code into the application:</p><code>{{code}}</code></body></html>
""");
context.HandleRequest();
}
}
}
+33
View File
@@ -0,0 +1,33 @@
using Microsoft.IdentityModel.Tokens;
using MongoDB.Entities;
using System.Security.Cryptography;
namespace PrivaPub.Api.Mastodon.Auth
{
public class OidcKey : Entity
{
public string Use { get; set; }
public string PrivateKeyPem { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public const string Signing = "signing";
public const string Encryption = "encryption";
public static RsaSecurityKey Load(string use)
{
var key = DB.Default.Find<OidcKey>().Match(k => k.Use == use).Sort(k => k.CreatedAt, Order.Ascending).ExecuteFirstAsync().GetAwaiter().GetResult();
if (key == default)
{
using var created = RSA.Create(2048);
key = new OidcKey { Use = use, PrivateKeyPem = created.ExportRSAPrivateKeyPem() };
DB.Default.SaveAsync(key).GetAwaiter().GetResult();
key = DB.Default.Find<OidcKey>().Match(k => k.Use == use).Sort(k => k.CreatedAt, Order.Ascending).ExecuteFirstAsync().GetAwaiter().GetResult();
}
var rsa = RSA.Create();
rsa.ImportFromPem(key.PrivateKeyPem);
return new RsaSecurityKey(rsa) { KeyId = key.ID };
}
}
}
@@ -0,0 +1,70 @@
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Mvc;
using Microsoft.IdentityModel.Tokens;
using OpenIddict.Abstractions;
using OpenIddict.Server.AspNetCore;
using PrivaPub.StaticServices;
using System.Security.Claims;
using static OpenIddict.Abstractions.OpenIddictConstants;
namespace PrivaPub.Api.Mastodon.Auth
{
[ApiController]
public class TokenController : ControllerBase
{
readonly DbEntities _dbEntities;
public TokenController(DbEntities dbEntities)
{
_dbEntities = dbEntities;
}
[HttpPost("/oauth/token"), IgnoreAntiforgeryToken, Produces("application/json")]
public async Task<IActionResult> Exchange(CancellationToken token)
{
var request = HttpContext.GetOpenIddictServerRequest();
if (request == default)
return BadRequest();
if (request.IsAuthorizationCodeGrantType())
{
var principal = (await HttpContext.AuthenticateAsync(OpenIddictServerAspNetCoreDefaults.AuthenticationScheme)).Principal;
if (principal == default || !await Usable(principal.GetClaim(Claims.Subject), token))
return Refuse(Errors.InvalidGrant, "The persona can no longer be used.");
return SignIn(principal, OpenIddictServerAspNetCoreDefaults.AuthenticationScheme);
}
if (request.IsClientCredentialsGrantType())
{
var identity = new ClaimsIdentity(TokenValidationParameters.DefaultAuthenticationType, Claims.Name, Claims.Role);
identity.SetClaim(Claims.Subject, "app:" + request.ClientId);
identity.SetScopes(MastodonScopes.Parse(request.Scope));
identity.SetDestinations(_ => new[] { Destinations.AccessToken });
return SignIn(new ClaimsPrincipal(identity), OpenIddictServerAspNetCoreDefaults.AuthenticationScheme);
}
return Refuse(Errors.UnsupportedGrantType, "The grant type is not supported.");
}
async Task<bool> Usable(string avatarId, CancellationToken token)
{
if (string.IsNullOrEmpty(avatarId))
return false;
var link = await _dbEntities.RootToAvatars.Match(r => r.AvatarId == avatarId).ExecuteFirstAsync(token);
var root = link == default ? default : await _dbEntities.RootUsers.MatchID(link.RootId).ExecuteFirstAsync(token);
var avatar = await _dbEntities.Avatars.MatchID(avatarId).ExecuteFirstAsync(token);
return root is { IsBanned: false, DeletedAt: null } && avatar is { DeletionAt: null };
}
ForbidResult Refuse(string error, string description) => Forbid(new AuthenticationProperties(new Dictionary<string, string>
{
[OpenIddictServerAspNetCoreConstants.Properties.Error] = error,
[OpenIddictServerAspNetCoreConstants.Properties.ErrorDescription] = description
}), OpenIddictServerAspNetCoreDefaults.AuthenticationScheme);
}
}
@@ -0,0 +1,401 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Media;
using PrivaPub.Domain.Privacy;
using PrivaPub.Domain.Relationships;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class AccountsController : MastodonController
{
readonly MastodonMapper _mapper;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IFollowService _follows;
readonly IOutboxPublisher _outbox;
readonly IRelationshipService _relationships;
public AccountsController(MastodonMapper mapper, DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors,
IFollowService follows, IOutboxPublisher outbox, IRelationshipService relationships)
{
_relationships = relationships;
_mapper = mapper;
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_follows = follows;
_outbox = outbox;
}
[HttpGet("/api/v1/accounts/verify_credentials"), Scope("read:accounts")]
public async Task<IActionResult> VerifyCredentials(CancellationToken token) => Json(await _mapper.Local(Me, withSource: true, token));
[HttpPatch("/api/v1/accounts/update_credentials"), Scope("write:accounts"), RequestSizeLimit(20 * 1024 * 1024),
RequestFormLimits(MultipartBodyLengthLimit = 20 * 1024 * 1024)]
public async Task<IActionResult> UpdateCredentials([FromServices] IMediaService media, CancellationToken token)
{
var avatar = await _dbEntities.Avatars.MatchID(Me.Id).ExecuteFirstAsync(token);
if (Request.HasFormContentType)
{
var form = await Request.ReadFormAsync(token);
if (form.Files["avatar"] is { } picture && await media.ProfileImage(picture, 400, 400, token) is { } pictureUrl)
avatar.PictureURL = pictureUrl;
if (form.Files["header"] is { } header && await media.ProfileImage(header, 1500, 500, token) is { } headerUrl)
avatar.ThumbnailURL = headerUrl;
}
if (Params.Has("display_name"))
avatar.Name = Params.Get("display_name")?.Trim();
if (Params.Has("note"))
avatar.Biography = Params.Get("note");
if (Params.Bool("locked") is { } locked)
avatar.Settings.IsLocked = locked;
if (Params.Bool("bot") is { } bot)
avatar.Settings.IsBot = bot;
if (Params.Bool("discoverable") is { } discoverable)
avatar.Settings.IsDiscoverable = discoverable;
if (Params.Bool("indexable") is { } indexable)
avatar.Settings.IsIndexable = indexable;
if (Params.Bool("hide_collections") is { } hide)
avatar.Settings.HideCollections = hide;
if (Params.Get("source[privacy]") is { } privacy && privacy is "public" or "unlisted" or "private")
avatar.Settings.DefaultVisibility = privacy;
if (Params.Bool("source[sensitive]") is { } sensitive)
avatar.Settings.DefaultSensitive = sensitive;
if (Params.Has("source[language]"))
avatar.Settings.DefaultLanguage = Params.Get("source[language]");
if (Params.Get("source[quote_policy]") is { } quotePolicy && QuotePolicies.IsKnown(quotePolicy))
avatar.Settings.QuotePolicy = quotePolicy;
var fields = new Dictionary<string, string>();
for (var i = 0; i < 4; i++)
{
var name = Params.Get($"fields_attributes[{i}][name]") ?? Params.Get($"fields_attributes[][name]");
if (!string.IsNullOrWhiteSpace(name))
fields[name.Trim()] = Params.Get($"fields_attributes[{i}][value]")?.Trim() ?? string.Empty;
}
if (Params.List("fields_attributes[][name]").Count > 0)
{
var names = Params.List("fields_attributes[][name]");
var values = Params.List("fields_attributes[][value]");
fields = names.Select((n, i) => (n, v: i < values.Count ? values[i] : string.Empty)).Where(f => !string.IsNullOrWhiteSpace(f.n))
.Take(4).ToDictionary(f => f.n.Trim(), f => f.v.Trim());
}
if (fields.Count > 0 || Params.Has("fields_attributes[0][name]"))
avatar.Fields = fields;
avatar.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(avatar, token);
var actor = _localActors.FromAvatar(avatar);
await _outbox.PublishProfile(actor, token);
return Json(await _mapper.Local(actor, withSource: true, token));
}
[HttpGet("/api/v1/accounts/lookup"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Lookup(CancellationToken token)
{
var acct = Params.Get("acct")?.Trim().TrimStart('@');
var parts = acct?.Split('@');
if (parts is not { Length: 1 or 2 } || string.IsNullOrEmpty(parts[0]))
return NotFoundError();
var localDomain = new Uri(_localActors.BaseAddress).Authority;
if (parts.Length == 1 || parts[1].Equals(localDomain, StringComparison.OrdinalIgnoreCase))
{
var local = await _localActors.FindByUserName(parts[0], token);
return local is { IsFederated: true, IsCircle: false, Kind: not LocalActorKind.Application } ? Json(await _mapper.Local(local, false, token)) : NotFoundError();
}
var userName = parts[0];
var domain = parts[1].ToLowerInvariant();
var foreign = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain).ExecuteFirstAsync(token);
return foreign == default ? NotFoundError() : Json(_mapper.Foreign(foreign));
}
[HttpGet("/api/v1/accounts/relationships"), Scope("read:follows")]
public async Task<IActionResult> Relationships(CancellationToken token)
{
var relationships = new List<Relationship>();
foreach (var id in Params.List("id").Distinct().Take(40))
relationships.Add(await Relationship(id, token));
return Json(relationships);
}
[HttpGet("/api/v1/accounts/search"), Scope("read:accounts")]
public async Task<IActionResult> Search([FromServices] AccountSearch search, CancellationToken token) =>
Json(await search.Find(Params.Get("q"), Params.Bool("resolve") == true && MyId != default, Limit(), token));
[HttpGet("/api/v1/accounts/{id}"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Get(string id, CancellationToken token)
{
var account = await _mapper.Account(id, token);
return account == default ? NotFoundError() : Json(account);
}
[HttpGet("/api/v1/accounts/{id}/statuses"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Statuses(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
if (Params.Bool("pinned") == true)
{
if (local == default)
return Json(Array.Empty<Status>());
var pinIds = (await DB.Default.Find<Pin>().Match(p => p.AvatarId == local.Id).Sort(p => p.ID, Order.Descending).ExecuteAsync(token)).Select(p => p.PostId).ToList();
var pinnedPosts = await _dbEntities.Posts.Match(p => pinIds.Contains(p.ID) && !p.DeletedAt.HasValue).Match(VisibilityPolicy.IsPublic).ExecuteAsync(token);
return Json(await _mapper.Statuses(pinIds.Select(id => pinnedPosts.FirstOrDefault(p => p.ID == id)).Where(p => p != default).ToList(), MyId, token));
}
var query = local != default
? _dbEntities.Posts.Match(p => p.GroupUserId == local.Id && !p.IsFederatedCopy && !p.DeletedAt.HasValue)
: _dbEntities.Posts.Match(p => p.ActorURI == remote.ActorURI && p.IsFederatedCopy && !p.DeletedAt.HasValue);
var viewerFollows = MyId != default && local != default
&& await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.TargetAccountId == local.Id && f.State == FollowState.Accepted).ExecuteAnyAsync(token);
if (MyId != id)
query.Match(viewerFollows
? p => p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted || p.Visibility == PostVisibility.FollowersOnly
: p => p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted);
else
query.Match(p => p.Visibility != PostVisibility.LocalGeo);
if (Params.Bool("exclude_replies") == true)
query.Match(p => p.InReplyToURI == null);
if (Params.Bool("exclude_reblogs") == true)
query.Match(p => p.ReblogOfPostId == null);
if (Params.Bool("only_media") == true)
query.Match(p => p.Media.Count > 0);
if (Params.Get("tagged") is { } tag)
query.Match(p => p.Tags.Contains(tag.ToLowerInvariant()));
var posts = await Page.From(Params, Limit()).Fetch(query, p => p.ID, token);
var statuses = await _mapper.Statuses(posts, MyId, token);
Link($"/api/v1/accounts/{id}/statuses", posts.LastOrDefault()?.ID, posts.FirstOrDefault()?.ID);
return Json(statuses);
}
[HttpGet("/api/v1/accounts/{id}/followers"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Followers(string id, CancellationToken token)
{
var (local, _) = await Find(id, token);
if (local == default || local.Id != MyId)
return Json(Array.Empty<Account>());
var followers = await _dbEntities.Followers.Match(f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted)
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
return Json(await AccountsFor(followers.Select(f => f.ActorURI), token));
}
[HttpGet("/api/v1/accounts/{id}/following"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Following(string id, CancellationToken token)
{
var (local, _) = await Find(id, token);
if (local == default || local.Id != MyId)
return Json(Array.Empty<Account>());
var following = await _dbEntities.Followings.Match(f => f.AvatarId == local.Id && f.State == FollowState.Accepted)
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
var accounts = await _mapper.Accounts(following.Select(f => f.TargetAccountId), token);
return Json(following.Select(f => accounts.GetValueOrDefault(f.TargetAccountId)).Where(a => a != default).ToList());
}
[HttpPost("/api/v1/accounts/{id}/follow"), Scope("write:follows")]
public async Task<IActionResult> Follow(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
var following = await _follows.FollowAs(Me, local?.Uri ?? remote.ActorURI, Params.Bool("reblogs") != false, token);
return following == default ? Error(StatusCodes.Status403Forbidden, "This action is not allowed") : Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/unfollow"), Scope("write:follows")]
public async Task<IActionResult> Unfollow(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
await _follows.UnfollowAs(Me, local?.Uri ?? remote.ActorURI, token);
return Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/remove_from_followers"), Scope("write:follows")]
public async Task<IActionResult> RemoveFromFollowers(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
var uri = local?.Uri ?? remote?.ActorURI;
if (uri == default)
return NotFoundError();
await DB.Default.DeleteAsync<Follower>(f => f.LocalActorId == Me.Id && f.ActorURI == uri);
if (local != default)
await DB.Default.DeleteAsync<Following>(f => f.AvatarId == local.Id && f.TargetActorURI == Me.Uri);
return Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/block"), Scope("write:blocks")]
public async Task<IActionResult> Block(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default || local?.Id == MyId)
return NotFoundError();
await _relationships.Block(Me, local?.Uri ?? remote.ActorURI, id, token);
return Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/unblock"), Scope("write:blocks")]
public async Task<IActionResult> Unblock(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
await _relationships.Unblock(Me, local?.Uri ?? remote.ActorURI, token);
return Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/mute"), Scope("write:mutes")]
public async Task<IActionResult> Mute(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default || local?.Id == MyId)
return NotFoundError();
var duration = Params.Int("duration") is { } seconds and > 0 ? TimeSpan.FromSeconds(seconds) : (TimeSpan?)null;
await _relationships.Mute(Me, local?.Uri ?? remote.ActorURI, id, Params.Bool("notifications") != false, duration, token);
return Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/unmute"), Scope("write:mutes")]
public async Task<IActionResult> Unmute(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
await _relationships.Unmute(Me, local?.Uri ?? remote.ActorURI, token);
return Json(await Relationship(id, token));
}
[HttpGet("/api/v1/blocks"), Scope("read:blocks")]
public async Task<IActionResult> Blocks(CancellationToken token)
{
var blocks = await Page.From(Params, Limit(40, 80)).Fetch(DB.Default.Find<Block>().Match(b => b.AvatarId == MyId), b => b.ID, token);
var accounts = await _mapper.Accounts(blocks.Select(b => b.TargetAccountId), token);
Link("/api/v1/blocks", blocks.LastOrDefault()?.ID, blocks.FirstOrDefault()?.ID);
return Json(blocks.Select(b => accounts.GetValueOrDefault(b.TargetAccountId)).Where(a => a != default).ToList());
}
[HttpGet("/api/v1/mutes"), Scope("read:mutes")]
public async Task<IActionResult> Mutes(CancellationToken token)
{
var mutes = await Page.From(Params, Limit(40, 80)).Fetch(DB.Default.Find<Mute>().Match(m => m.AvatarId == MyId), m => m.ID, token);
var accounts = await _mapper.Accounts(mutes.Select(m => m.TargetAccountId), token);
Link("/api/v1/mutes", mutes.LastOrDefault()?.ID, mutes.FirstOrDefault()?.ID);
return Json(mutes.Select(m => accounts.GetValueOrDefault(m.TargetAccountId)).Where(a => a != default).ToList());
}
[HttpGet("/api/v1/domain_blocks"), Scope("read:blocks")]
public async Task<IActionResult> DomainBlocks(CancellationToken token) =>
Json((await DB.Default.Find<AccountDomainBlock>().Match(b => b.AvatarId == MyId).Sort(b => b.Domain, Order.Ascending).ExecuteAsync(token)).Select(b => b.Domain).ToList());
[HttpPost("/api/v1/domain_blocks"), Scope("write:blocks")]
public async Task<IActionResult> BlockDomain(CancellationToken token)
{
await _relationships.BlockDomain(Me, Params.Get("domain"), token);
return Json(new { });
}
[HttpDelete("/api/v1/domain_blocks"), Scope("write:blocks")]
public async Task<IActionResult> UnblockDomain(CancellationToken token)
{
await _relationships.UnblockDomain(Me, Params.Get("domain"), token);
return Json(new { });
}
[HttpGet("/api/v1/follow_requests"), Scope("read:follows")]
public async Task<IActionResult> FollowRequests(CancellationToken token)
{
var requests = await _dbEntities.Followers.Match(f => f.LocalActorId == Me.Id && f.LocalActorKind == LocalActorKind.Person && !f.IsAccepted)
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
return Json(await AccountsFor(requests.Select(r => r.ActorURI), token));
}
[HttpPost("/api/v1/follow_requests/{id}/authorize"), Scope("write:follows")]
public async Task<IActionResult> Authorize(string id, CancellationToken token) =>
await _follows.Decide(Me, id, accept: true, token) ? Json(await Relationship(id, token)) : NotFoundError();
[HttpPost("/api/v1/follow_requests/{id}/reject"), Scope("write:follows")]
public async Task<IActionResult> Reject(string id, CancellationToken token) =>
await _follows.Decide(Me, id, accept: false, token) ? Json(await Relationship(id, token)) : NotFoundError();
[HttpGet("/api/v1/accounts/{id}/featured_tags"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult FeaturedTags(string id) => Json(Array.Empty<object>());
[HttpGet("/api/v1/accounts/{id}/lists"), Scope("read:lists")]
public IActionResult Lists(string id) => Json(Array.Empty<object>());
[HttpGet("/api/v1/accounts/familiar_followers"), Scope("read:follows")]
public IActionResult FamiliarFollowers() => Json(Params.List("id").Select(id => new { id, accounts = Array.Empty<Account>() }).ToList());
async Task<List<Account>> AccountsFor(IEnumerable<string> actorUris, CancellationToken token)
{
var accounts = new List<Account>();
foreach (var uri in actorUris)
{
var local = await _localActors.FindByUri(uri, token);
if (local != default)
{
accounts.Add(await _mapper.Local(local, false, token));
continue;
}
var foreign = await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == uri).ExecuteFirstAsync(token);
if (foreign != default)
accounts.Add(_mapper.Foreign(foreign));
}
return accounts;
}
async Task<(LocalActor Local, ForeignAvatar Remote)> Find(string id, CancellationToken token)
{
var avatar = await _dbEntities.Avatars.MatchID(id).ExecuteFirstAsync(token);
if (avatar is { DeletionAt: null })
return (_localActors.FromAvatar(avatar), default);
var group = await _dbEntities.Groups.MatchID(id).ExecuteFirstAsync(token);
if (group is { DeletionAt: null } && _localActors.FromGroup(group) is { IsFederated: true, IsCircle: false } community)
return (community, default);
return (default, await _dbEntities.ForeignAvatars.MatchID(id).ExecuteFirstAsync(token));
}
async Task<Relationship> Relationship(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
var uri = local?.Uri ?? remote?.ActorURI;
var following = uri == default ? default : await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.TargetActorURI == uri).ExecuteFirstAsync(token);
var followedBy = uri == default ? default : await _dbEntities.Followers.Match(f => f.LocalActorId == MyId && f.ActorURI == uri).ExecuteFirstAsync(token);
var blocking = uri != default && await DB.Default.Find<Block>().Match(b => b.AvatarId == MyId && b.TargetActorURI == uri).ExecuteAnyAsync(token);
var mute = uri == default ? default : await DB.Default.Find<Mute>().Match(m => m.AvatarId == MyId && m.TargetActorURI == uri).ExecuteFirstAsync(token);
var blockedBy = local != default && await DB.Default.Find<Block>().Match(b => b.AvatarId == local.Id && b.TargetActorURI == Me.Uri).ExecuteAnyAsync(token);
var domainBlocked = remote != default && await DB.Default.Find<AccountDomainBlock>().Match(b => b.AvatarId == MyId && b.Domain == remote.Domain).ExecuteAnyAsync(token);
return new Relationship
{
Id = id,
Blocking = blocking,
BlockedBy = blockedBy,
Muting = mute != default && (mute.ExpiresAt == default || mute.ExpiresAt > DateTime.UtcNow),
MutingNotifications = mute?.HideNotifications == true,
DomainBlocking = domainBlocked,
Following = following?.State == FollowState.Accepted,
Requested = following?.State == FollowState.Requested,
ShowingReblogs = following?.ShowReblogs ?? false,
FollowedBy = followedBy?.IsAccepted == true,
RequestedBy = followedBy is { IsAccepted: false }
};
}
}
}
@@ -0,0 +1,97 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using OpenIddict.Abstractions;
using PrivaPub.Api.Mastodon.Auth;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using System.Security.Cryptography;
using static OpenIddict.Abstractions.OpenIddictConstants;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class AppsController : MastodonController
{
readonly IOpenIddictApplicationManager _applications;
public AppsController(IOpenIddictApplicationManager applications)
{
_applications = applications;
}
[HttpPost("/api/v1/apps"), AllowAnonymous]
public async Task<IActionResult> Create(CancellationToken token)
{
var name = Params.Get("client_name")?.Trim();
if (string.IsNullOrEmpty(name) || name.Length > 200)
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Application name can't be blank");
var redirects = Params.List("redirect_uris").SelectMany(r => r.Split('\n', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)).Distinct().ToList();
if (redirects.Count == 0 || redirects.Any(r => !Uri.TryCreate(r, UriKind.Absolute, out var uri) || uri.Scheme is "javascript" or "data"))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Redirect URI must be an absolute URI.");
var scopes = MastodonScopes.Parse(string.Join(' ', Params.List("scopes")));
var descriptor = new OpenIddictApplicationDescriptor
{
ClientId = Secret(32),
ClientSecret = Secret(32),
ClientType = ClientTypes.Confidential,
ConsentType = ConsentTypes.Explicit,
DisplayName = name,
Permissions =
{
Permissions.Endpoints.Authorization,
Permissions.Endpoints.Token,
Permissions.Endpoints.Revocation,
Permissions.GrantTypes.AuthorizationCode,
Permissions.GrantTypes.ClientCredentials,
Permissions.ResponseTypes.Code
}
};
foreach (var scope in scopes)
descriptor.Permissions.Add(Permissions.Prefixes.Scope + scope);
foreach (var redirect in redirects)
descriptor.RedirectUris.Add(new Uri(redirect));
var website = Params.Get("website");
if (Uri.TryCreate(website, UriKind.Absolute, out var site) && site.Scheme is "https" or "http")
descriptor.Properties["website"] = System.Text.Json.JsonSerializer.SerializeToElement(website);
var application = await _applications.CreateAsync(descriptor, token);
return Json(new Application
{
Id = await _applications.GetIdAsync(application, token),
Name = name,
Website = website,
Scopes = scopes.ToList(),
RedirectUris = redirects,
RedirectUri = string.Join("\n", redirects),
ClientId = descriptor.ClientId,
ClientSecret = descriptor.ClientSecret
});
}
[HttpGet("/api/v1/apps/verify_credentials")]
public async Task<IActionResult> Verify(CancellationToken token)
{
var clientId = User.GetPresenters().FirstOrDefault() ?? User.GetClaim(Claims.ClientId);
var application = clientId == default ? default : await _applications.FindByClientIdAsync(clientId, token);
if (application == default)
return Error(StatusCodes.Status401Unauthorized, "The access token is invalid");
var properties = await _applications.GetPropertiesAsync(application, token);
return Json(new
{
id = await _applications.GetIdAsync(application, token),
name = await _applications.GetDisplayNameAsync(application, token),
website = properties.TryGetValue("website", out var website) ? website.GetString() : default,
scopes = User.GetScopes().ToList(),
redirect_uris = (await _applications.GetRedirectUrisAsync(application, token)).ToList(),
vapid_key = string.Empty
});
}
static string Secret(int bytes) => Convert.ToBase64String(RandomNumberGenerator.GetBytes(bytes)).TrimEnd('=').Replace('+', '-').Replace('/', '_');
}
}
@@ -0,0 +1,108 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.User;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class InstanceController : MastodonController
{
public const string Version = "4.2.0 (compatible; PrivaPub)";
const string Description = "A small ActivityPub server where one private login keeps several unlinkable public personas.";
readonly ILocalActorService _localActors;
public InstanceController(ILocalActorService localActors)
{
_localActors = localActors;
}
string Domain => new Uri(_localActors.BaseAddress).Authority;
static object Statuses => new { max_characters = 5000, max_media_attachments = 4, characters_reserved_per_url = 23 };
static object MediaAttachments => new
{
supported_mime_types = new[] { "image/jpeg", "image/png", "image/gif", "image/webp", "video/mp4", "video/webm", "audio/mpeg", "audio/ogg" },
image_size_limit = 16 * 1024 * 1024,
image_matrix_limit = 16_777_216,
video_size_limit = 99 * 1024 * 1024,
video_frame_rate_limit = 60,
video_matrix_limit = 2_304_000
};
static object Polls => new { max_options = 4, max_characters_per_option = 50, min_expiration = 300, max_expiration = 2_629_746 };
[HttpGet("/api/v1/instance"), AllowAnonymous]
public async Task<IActionResult> V1(CancellationToken token)
{
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token);
var statuses = await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token);
var domains = (await DB.Default.Find<ForeignAvatar, string>().Project(a => a.Domain).ExecuteAsync(token)).Distinct().Count();
return Json(new
{
uri = Domain,
title = "PrivaPub",
short_description = Description,
description = Description,
email = string.Empty,
version = Version,
urls = new { streaming_api = $"wss://{Domain}" },
stats = new { user_count = users, status_count = statuses, domain_count = domains },
thumbnail = $"{_localActors.BaseAddress}/media/missing-header.png",
languages = new[] { "en" },
registrations = false,
approval_required = false,
invites_enabled = false,
configuration = new { accounts = new { max_featured_tags = 0 }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls },
contact_account = default(object),
rules = Array.Empty<object>()
});
}
[HttpGet("/api/v2/instance"), AllowAnonymous]
public IActionResult V2() => Json(new
{
domain = Domain,
title = "PrivaPub",
version = Version,
api_versions = new { mastodon = 7 },
source_url = "https://git.thepra.dev/thepra/SocialPub",
description = Description,
usage = new { users = new { active_month = 0 } },
thumbnail = new { url = $"{_localActors.BaseAddress}/media/missing-header.png" },
languages = new[] { "en" },
configuration = new
{
urls = new { streaming = $"wss://{Domain}" },
accounts = new { max_featured_tags = 0, max_pinned_statuses = 0 },
statuses = Statuses,
media_attachments = MediaAttachments,
polls = Polls,
translation = new { enabled = false }
},
registrations = new { enabled = false, approval_required = false, message = default(string) },
contact = new { email = string.Empty, account = default(object) },
rules = Array.Empty<object>()
});
[HttpGet("/api/v1/instance/peers"), AllowAnonymous]
public IActionResult Peers() => Json(Array.Empty<string>());
[HttpGet("/api/v1/instance/activity"), AllowAnonymous]
public IActionResult Activity() => Json(Array.Empty<object>());
[HttpGet("/api/v1/instance/rules"), AllowAnonymous]
public IActionResult Rules() => Json(Array.Empty<object>());
[HttpGet("/api/v1/instance/extended_description"), AllowAnonymous]
public IActionResult ExtendedDescription() => Json(new { updated_at = MastodonJson.Day(DateTime.UtcNow), content = $"<p>{Description}</p>" });
}
}
@@ -0,0 +1,123 @@
using PrivaPub.Infrastructure.Statistics;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Media;
using PrivaPub.Models.Media;
using System.Globalization;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class MediaController : MastodonController
{
const long UploadLimit = 100L * 1024 * 1024;
readonly IMediaService _media;
readonly IMediaProxy _proxy;
readonly IInteractionLedger _ledger;
public MediaController(IMediaService media, IMediaProxy proxy, IInteractionLedger ledger = default)
{
_media = media;
_proxy = proxy;
_ledger = ledger;
}
[HttpPost("/api/v1/media"), HttpPost("/api/v2/media"), Scope("write:media"), RequestSizeLimit(UploadLimit),
RequestFormLimits(MultipartBodyLengthLimit = UploadLimit)]
public async Task<IActionResult> Upload(CancellationToken token)
{
if (!Request.HasFormContentType)
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: File can't be blank");
var form = await Request.ReadFormAsync(token);
var outcome = await _media.Upload(Me, form.Files["file"], form["description"], form["focus"], token);
return outcome.Ok ? Json(View(outcome.Attachment)) : Error(outcome.Status, outcome.Error);
}
[HttpGet("/api/v1/media/{id}"), Scope("write:media")]
public async Task<IActionResult> Get(string id, CancellationToken token)
{
var attachment = await DB.Default.Find<MediaAttachment>().Match(m => m.ID == id && m.OwnerAvatarId == MyId).ExecuteFirstAsync(token);
return attachment == default ? NotFoundError() : Json(View(attachment));
}
[HttpPut("/api/v1/media/{id}"), Scope("write:media")]
public async Task<IActionResult> Update(string id, CancellationToken token)
{
var attachment = await DB.Default.Find<MediaAttachment>().Match(m => m.ID == id && m.OwnerAvatarId == MyId).ExecuteFirstAsync(token);
if (attachment == default)
return NotFoundError();
if (Params.Has("description"))
attachment.Description = Params.Get("description")?.Trim() is { Length: > 0 } description ? description[..Math.Min(description.Length, 1500)] : default;
if (Params.Get("focus")?.Split(',') is [var x, var y]
&& float.TryParse(x, NumberStyles.Float, CultureInfo.InvariantCulture, out var fx) && float.TryParse(y, NumberStyles.Float, CultureInfo.InvariantCulture, out var fy))
attachment.Focus = new[] { Math.Clamp(fx, -1, 1), Math.Clamp(fy, -1, 1) };
await DB.Default.SaveAsync(attachment, token);
return Json(View(attachment));
}
[HttpGet("/media/proxy/{signature}/{encoded}"), AllowAnonymous, ApiExplorerSettings(IgnoreApi = true)]
public async Task<IActionResult> Proxy(string signature, string encoded, CancellationToken token)
{
var url = _proxy.Verified(signature, encoded);
if (url == default)
return NotFound();
Response.Headers["X-Content-Type-Options"] = "nosniff";
Response.Headers["Content-Security-Policy"] = "default-src 'none'; sandbox";
Response.Headers["Cache-Control"] = "public, max-age=604800";
if (_proxy.Cached(url) is { Path: not null } cached)
{
_ledger?.Count(Interactions.HostOf(url), "media:hit");
return PhysicalFile(cached.Path, cached.ContentType, enableRangeProcessing: true);
}
if (Request.Headers.Range.Count == 0)
{
var (path, contentType) = await _proxy.Fetch(signature, encoded, token);
if (path != default)
return PhysicalFile(path, contentType, enableRangeProcessing: true);
}
return await Stream(url, token);
}
async Task<IActionResult> Stream(string url, CancellationToken token)
{
var range = System.Net.Http.Headers.RangeHeaderValue.TryParse(Request.Headers.Range.ToString(), out var asked) ? asked : default;
using var upstream = await _proxy.Open(url, range, token);
if (upstream == default)
return NotFound();
Response.StatusCode = (int)upstream.StatusCode;
Response.ContentType = upstream.Content.Headers.ContentType?.ToString() ?? "application/octet-stream";
if (upstream.Content.Headers.ContentLength is { } length)
Response.ContentLength = length;
if (upstream.Content.Headers.ContentRange is { } contentRange)
Response.Headers.ContentRange = contentRange.ToString();
Response.Headers.AcceptRanges = "bytes";
await upstream.Content.CopyToAsync(Response.Body, token);
return new EmptyResult();
}
object View(MediaAttachment attachment) => new
{
id = attachment.ID,
type = attachment.Kind,
url = _media.Url(attachment.FilePath),
preview_url = _media.Url(attachment.PreviewPath ?? attachment.FilePath),
remote_url = default(string),
text_url = default(string),
meta = new
{
original = attachment.Width.HasValue && attachment.Height > 0
? new { width = attachment.Width, height = attachment.Height, size = $"{attachment.Width}x{attachment.Height}", aspect = (double)attachment.Width / attachment.Height.Value }
: default,
focus = attachment.Focus is { Length: 2 } ? new { x = attachment.Focus[0], y = attachment.Focus[1] } : default
},
description = attachment.Description,
blurhash = attachment.Blurhash
};
}
}
@@ -0,0 +1,54 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Post;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class PollsController : MastodonController
{
readonly DbEntities _dbEntities;
readonly MastodonMapper _mapper;
readonly IPollService _polls;
public PollsController(DbEntities dbEntities, MastodonMapper mapper, IPollService polls)
{
_dbEntities = dbEntities;
_mapper = mapper;
_polls = polls;
}
[HttpGet("/api/v1/polls/{id}"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
public async Task<IActionResult> Get(string id, CancellationToken token)
{
var post = await Visible(id, token);
return post?.Poll == default ? NotFoundError() : Json(_mapper.Poll(post, MyId, await _polls.OwnVotes(post.ID, MyId, token)));
}
[HttpPost("/api/v1/polls/{id}/votes"), Scope("write:statuses")]
public async Task<IActionResult> Vote(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post?.Poll == default)
return NotFoundError();
var choices = Params.List("choices").Select(c => int.TryParse(c, out var index) ? index : -1).ToList();
var outcome = await _polls.Vote(Me, post, choices, token);
if (!outcome.Ok)
return Error(outcome.Status, outcome.Error);
return Json(_mapper.Poll(outcome.Post, MyId, await _polls.OwnVotes(post.ID, MyId, token)));
}
async Task<PostEntity> Visible(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
return post != default && post.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(post, MyId, token) ? post : default;
}
}
}
@@ -0,0 +1,183 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class ProvenanceController : MastodonController
{
readonly DbEntities _dbEntities;
public ProvenanceController(DbEntities dbEntities) => _dbEntities = dbEntities;
[HttpGet("/api/privapub/v1/statuses/{id}/provenance"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
public async Task<IActionResult> Status(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
if (post?.ReblogOfPostId != default)
post = await _dbEntities.Posts.Match(p => p.ID == post.ReblogOfPostId && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
if (post == default || post.Visibility == PostVisibility.LocalGeo || !await VisibilityPolicy.CanSee(post, MyId, token))
return NotFoundError();
if (!post.IsFederatedCopy)
return Json(new Provenance { ObjectUri = post.ObjectURI, Url = post.Url, Path = "local" });
var record = await DB.Default.Find<ObjectRecord>().Match(r => r.PostId == post.ID).ExecuteFirstAsync(token)
?? await DB.Default.Find<ObjectRecord>().Match(r => r.ObjectURI == post.ObjectURI).ExecuteFirstAsync(token);
var host = record?.Host ?? (Uri.TryCreate(post.ObjectURI, UriKind.Absolute, out var uri) ? uri.Host.ToLowerInvariant() : default);
var instance = host == default ? default : await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteFirstAsync(token);
return Json(Describe(post, record, instance));
}
[HttpGet("/api/privapub/v1/instances/{host}"), Scope("read", requiresUser: false), AllowAnonymous]
public async Task<IActionResult> Instance(string host, CancellationToken token)
{
host = host?.Trim().ToLowerInvariant();
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteFirstAsync(token);
return instance == default ? NotFoundError() : Json(Describe(instance));
}
static Provenance Describe(PostEntity post, ObjectRecord record, RemoteInstance instance)
{
var raw = record?.Raw == default ? default : JsonNode.Parse(record.Raw) as JsonObject;
return new Provenance
{
ObjectUri = post.ObjectURI,
ObjectType = record?.ObjectType ?? post.ObjectType,
Url = post.Url,
Path = record == default ? "unrecorded" : record.Path == ObjectPath.Delivered ? "delivered" : "fetched",
Refetched = record?.Refetched == true,
Activity = record?.ActivityId == default ? default : new ProvenanceActivity
{
Id = record.ActivityId, Type = record.ActivityType, Actor = record.ActivityActorURI
},
Inbox = record?.Inbox,
FetchedBy = record?.Path == ObjectPath.Fetched ? "instance-actor" : default,
Signature = record?.KeyId == default || record.Path == ObjectPath.Fetched ? default : new ProvenanceSignature
{
Scheme = record.SignatureScheme, KeyId = record.KeyId, Algorithm = record.Algorithm, Headers = record.SignedHeaders
},
ReceivedAt = record == default ? default : MastodonJson.Time(record.ReceivedAt),
Published = record?.Published is { } published ? MastodonJson.Time(published) : default,
Updated = record?.Updated is { } updated ? MastodonJson.Time(updated) : default,
Extensions = record?.Extensions ?? ObjectFeatures.Detect(raw),
ContextNamespaces = record?.ContextNamespaces ?? ObjectFeatures.Namespaces(raw, record?.ActivityContext),
Raw = raw,
RawBytes = record?.RawBytes ?? 0,
RawHash = record?.RawHash,
RawTruncated = record?.RawTruncated == true,
Revisions = record?.Revisions.Select(r => new ProvenanceRevision
{
ActivityId = r.ActivityId,
Updated = r.Updated is { } at ? MastodonJson.Time(at) : default,
ReceivedAt = MastodonJson.Time(r.ReceivedAt),
RawHash = r.RawHash,
Raw = r.Raw == default ? default : JsonNode.Parse(r.Raw)
}).ToList() ?? new(),
Instance = instance == default ? default : Describe(instance)
};
}
static InstanceDescription Describe(RemoteInstance instance) => new()
{
Host = instance.Host,
Software = instance.Software,
Version = instance.SoftwareVersion,
NodeName = instance.NodeName,
Protocols = instance.Protocols ?? new(),
OpenRegistrations = instance.OpenRegistrations,
DescribedAt = instance.DescribedAt is { } described ? MastodonJson.Time(described) : default,
DescriptionError = instance.DescriptionError,
NodeInfo = instance.NodeInfo == default ? default : JsonNode.Parse(instance.NodeInfo),
Delivery = new InstanceDelivery
{
ConsecutiveFailures = instance.ConsecutiveFailures,
UnavailableUntil = instance.UnavailableUntil is { } until ? MastodonJson.Time(until) : default,
LastSuccessAt = instance.LastSuccessAt is { } success ? MastodonJson.Time(success) : default,
LastFailureAt = instance.LastFailureAt is { } failure ? MastodonJson.Time(failure) : default
}
};
public class Provenance
{
public string ObjectUri { get; set; }
public string ObjectType { get; set; }
public string Url { get; set; }
public string Path { get; set; }
public bool Refetched { get; set; }
public ProvenanceActivity Activity { get; set; }
public string Inbox { get; set; }
public string FetchedBy { get; set; }
public ProvenanceSignature Signature { get; set; }
public string ReceivedAt { get; set; }
public string Published { get; set; }
public string Updated { get; set; }
public List<string> Extensions { get; set; } = new();
public List<string> ContextNamespaces { get; set; } = new();
public JsonNode Raw { get; set; }
public int RawBytes { get; set; }
public string RawHash { get; set; }
public bool RawTruncated { get; set; }
public List<ProvenanceRevision> Revisions { get; set; } = new();
public InstanceDescription Instance { get; set; }
}
public class ProvenanceActivity
{
public string Id { get; set; }
public string Type { get; set; }
public string Actor { get; set; }
}
public class ProvenanceSignature
{
public string Scheme { get; set; }
public string KeyId { get; set; }
public string Algorithm { get; set; }
public List<string> Headers { get; set; } = new();
}
public class ProvenanceRevision
{
public string ActivityId { get; set; }
public string Updated { get; set; }
public string ReceivedAt { get; set; }
public string RawHash { get; set; }
public JsonNode Raw { get; set; }
}
public class InstanceDescription
{
public string Host { get; set; }
public string Software { get; set; }
public string Version { get; set; }
public string NodeName { get; set; }
public List<string> Protocols { get; set; } = new();
public bool? OpenRegistrations { get; set; }
public string DescribedAt { get; set; }
public string DescriptionError { get; set; }
public JsonNode NodeInfo { get; set; }
public InstanceDelivery Delivery { get; set; }
}
public class InstanceDelivery
{
public int ConsecutiveFailures { get; set; }
public string UnavailableUntil { get; set; }
public string LastSuccessAt { get; set; }
public string LastFailureAt { get; set; }
}
}
}
@@ -0,0 +1,76 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Domain.Social;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class ReactionsController : MastodonController
{
readonly DbEntities _dbEntities;
readonly MastodonMapper _mapper;
readonly IReactions _reactions;
public ReactionsController(DbEntities dbEntities, MastodonMapper mapper, IReactions reactions)
{
_dbEntities = dbEntities;
_mapper = mapper;
_reactions = reactions;
}
[HttpGet("/api/v1/pleroma/statuses/{id}/reactions"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
public async Task<IActionResult> List(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
var reactions = await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID).ExecuteAsync(token);
var accounts = await _mapper.Accounts(reactions.Select(r => r.AccountId), token);
return Json(reactions.GroupBy(r => r.Emoji).Select(group => new
{
name = group.Key,
count = group.Count(),
me = MyId != default && group.Any(r => r.AccountId == MyId),
url = _mapper.ProxiedUrl(group.First().EmojiURL),
accounts = group.Select(r => accounts.GetValueOrDefault(r.AccountId)).Where(a => a != default).ToList()
}).OrderByDescending(r => r.count).ToList());
}
[HttpPut("/api/v1/pleroma/statuses/{id}/reactions/{emoji}"), Scope("write:favourites")]
public async Task<IActionResult> React(string id, string emoji, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
if (!await _reactions.React(Me, post, Uri.UnescapeDataString(emoji), token))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: That emoji cannot be used here");
return Json(await _mapper.Status(post, MyId, token));
}
[HttpDelete("/api/v1/pleroma/statuses/{id}/reactions/{emoji}"), Scope("write:favourites")]
public async Task<IActionResult> Unreact(string id, string emoji, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
await _reactions.Unreact(Me, post, Uri.UnescapeDataString(emoji), token);
return Json(await _mapper.Status(post, MyId, token));
}
async Task<PostEntity> Visible(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token);
return post != default && post.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(post, MyId, token) ? post : default;
}
}
}
@@ -0,0 +1,42 @@
using Microsoft.AspNetCore.Mvc;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Relationships;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class ReportsController : MastodonController
{
readonly IReportService _reports;
readonly MastodonMapper _mapper;
public ReportsController(IReportService reports, MastodonMapper mapper)
{
_reports = reports;
_mapper = mapper;
}
[HttpPost("/api/v1/reports"), Scope("write:reports")]
public async Task<IActionResult> Create(CancellationToken token)
{
var report = await _reports.File(Me, Params.Get("account_id"), Params.List("status_ids"), Params.Get("comment"), Params.Get("category"),
Params.Bool("forward") == true, token);
if (report == default)
return NotFoundError();
return Json(new
{
id = report.ID,
action_taken = false,
action_taken_at = default(string),
category = report.Category,
comment = report.Comment ?? string.Empty,
forwarded = report.Forwarded,
created_at = MastodonJson.Time(report.CreatedAt),
status_ids = report.PostIds,
rule_ids = Array.Empty<string>(),
target_account = await _mapper.Account(report.TargetAccountId, token)
});
}
}
}
@@ -0,0 +1,137 @@
using Microsoft.AspNetCore.Mvc;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Outbox;
using PrivaPub.Domain.Social;
using PrivaPub.Models.Post;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class SearchController : MastodonController
{
readonly MastodonMapper _mapper;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
readonly IRemotePosts _remotePosts;
readonly AccountSearch _search;
public SearchController(MastodonMapper mapper, DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors,
IRemotePosts remotePosts, AccountSearch search)
{
_mapper = mapper;
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
_remotePosts = remotePosts;
_search = search;
}
[HttpGet("/api/v2/search"), Scope("read:search", requiresUser: false)]
public async Task<IActionResult> Search(CancellationToken token)
{
var q = Params.Get("q")?.Trim();
var type = Params.Get("type");
var resolve = Params.Bool("resolve") == true && MyId != default;
var results = new SearchResults();
if (string.IsNullOrEmpty(q))
return Json(results);
if (q.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
{
var post = await _dbEntities.Posts.Match(p => (p.ObjectURI == q || p.Url == q) && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
if (post == default && resolve && type is null or "statuses")
post = await _remotePosts.StoreContext(q, 0, token);
if (post != default && type is null or "statuses" && await VisibilityPolicy.CanSee(post, MyId, token))
results.Statuses.Add(await _mapper.Status(post, MyId, token));
if (results.Statuses.Count == 0 && type is null or "accounts")
{
var local = await _localActors.FindByUri(q, token);
var foreign = local == default ? (resolve ? await _remoteActors.GetActor(q, refresh: false, token) : default) : default;
if (local is { IsFederated: true, IsCircle: false })
results.Accounts.Add(await _mapper.Local(local, false, token));
else if (foreign != default)
results.Accounts.Add(_mapper.Foreign(foreign));
}
return Json(results);
}
if (type is null or "accounts")
{
results.Accounts = await _search.Find(q, resolve, Limit(), token);
}
if (type is null or "hashtags" && q.TrimStart('#') is { Length: > 0 } tag)
results.Hashtags.Add(new { name = tag.ToLowerInvariant(), url = $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag.ToLowerInvariant())}", history = Array.Empty<object>() });
return Json(results);
}
}
public class StubsController : MastodonController
{
[HttpGet("/api/v1/custom_emojis"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult CustomEmojis() => Json(Array.Empty<object>());
[HttpGet("/api/v1/filters"), Scope("read:filters")]
public IActionResult FiltersV1() => Json(Array.Empty<object>());
[HttpGet("/api/v2/filters"), Scope("read:filters")]
public IActionResult FiltersV2() => Json(Array.Empty<object>());
[HttpGet("/api/v1/lists"), Scope("read:lists")]
public IActionResult Lists() => Json(Array.Empty<object>());
[HttpGet("/api/v1/announcements"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Announcements() => Json(Array.Empty<object>());
[HttpGet("/api/v1/trends"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Trends() => Json(Array.Empty<object>());
[HttpGet("/api/v1/trends/tags"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult TrendTags() => Json(Array.Empty<object>());
[HttpGet("/api/v1/trends/statuses"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult TrendStatuses() => Json(Array.Empty<object>());
[HttpGet("/api/v1/trends/links"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult TrendLinks() => Json(Array.Empty<object>());
[HttpGet("/api/v1/suggestions"), Scope("read")]
public IActionResult SuggestionsV1() => Json(Array.Empty<object>());
[HttpGet("/api/v2/suggestions"), Scope("read")]
public IActionResult SuggestionsV2() => Json(Array.Empty<object>());
[HttpGet("/api/v1/followed_tags"), Scope("read:follows")]
public IActionResult FollowedTags() => Json(Array.Empty<object>());
[HttpGet("/api/v1/endorsements"), Scope("read:accounts")]
public IActionResult Endorsements() => Json(Array.Empty<object>());
[HttpGet("/api/v1/featured_tags"), Scope("read:accounts")]
public IActionResult FeaturedTags() => Json(Array.Empty<object>());
[HttpGet("/api/v1/scheduled_statuses"), Scope("read:statuses")]
public IActionResult ScheduledStatuses() => Json(Array.Empty<object>());
[HttpGet("/api/v1/preferences"), Scope("read:accounts")]
public IActionResult Preferences() => Json(new Dictionary<string, object>
{
["posting:default:visibility"] = Me.Settings.DefaultVisibility ?? "public",
["posting:default:sensitive"] = Me.Settings.DefaultSensitive,
["posting:default:language"] = Me.Settings.DefaultLanguage,
["reading:expand:media"] = "default",
["reading:expand:spoilers"] = false
});
[HttpGet("/api/v1/push/subscription"), Scope("push")]
public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found");
}
}
@@ -0,0 +1,380 @@
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Caching.Memory;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Federation.Rendering;
using PrivaPub.Federation.Outbox;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class StatusesController : MastodonController
{
const int MaxContext = 200;
public const int MaxPins = 5;
readonly IStatusService _statuses;
readonly MastodonMapper _mapper;
readonly DbEntities _dbEntities;
readonly IMemoryCache _cache;
readonly IQuoteService _quotes;
readonly IOutboxPublisher _outbox;
public StatusesController(IStatusService statuses, MastodonMapper mapper, DbEntities dbEntities, IMemoryCache cache, IQuoteService quotes,
IOutboxPublisher outbox)
{
_quotes = quotes;
_outbox = outbox;
_statuses = statuses;
_mapper = mapper;
_dbEntities = dbEntities;
_cache = cache;
}
[HttpPost("/api/v1/statuses"), Scope("write:statuses")]
public async Task<IActionResult> Create(CancellationToken token)
{
var idempotency = Request.Headers["Idempotency-Key"].ToString();
var cacheKey = string.IsNullOrEmpty(idempotency) ? default : $"idempotency:{MyId}:{idempotency}";
if (cacheKey != default && _cache.TryGetValue(cacheKey, out string earlier))
{
var existing = await _dbEntities.Posts.MatchID(earlier).ExecuteFirstAsync(token);
if (existing != default)
return Json(await _mapper.Status(existing, MyId, token));
}
var outcome = await _statuses.Publish(Me, new StatusDraft
{
Text = Params.Get("status"),
PlainText = true,
SpoilerText = Params.Get("spoiler_text"),
Sensitive = Params.Bool("sensitive") ?? Me.Settings.DefaultSensitive,
Visibility = Visibility(Params.Get("visibility") ?? Me.Settings.DefaultVisibility),
InReplyTo = Params.Get("in_reply_to_id"),
Language = Params.Get("language") ?? Me.Settings.DefaultLanguage,
MediaIds = Params.List("media_ids"),
QuotedStatusId = Params.Get("quoted_status_id"),
QuotePolicy = Params.Get("quote_approval_policy"),
Poll = Params.Has("poll[options]")
? new PollDraft(Params.List("poll[options]"), Params.Int("poll[expires_in]") ?? 0, Params.Bool("poll[multiple]") == true,
Params.Bool("poll[hide_totals]") == true)
: default
}, token);
if (!outcome.Ok)
return Error(outcome.Status, outcome.Error);
if (cacheKey != default)
_cache.Set(cacheKey, outcome.Post.ID, TimeSpan.FromHours(1));
return Json(await _mapper.Status(outcome.Post, MyId, token));
}
[HttpGet("/api/v1/statuses"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Many(CancellationToken token)
{
var ids = Params.List("id").Take(20).ToList();
var posts = await _dbEntities.Posts.Match(p => ids.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token);
var visible = new List<PostEntity>();
foreach (var post in posts)
if (await VisibilityPolicy.CanSee(post, MyId, token))
visible.Add(post);
return Json(await _mapper.Statuses(visible, MyId, token));
}
[HttpGet("/api/v1/statuses/{id}"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Get(string id, CancellationToken token)
{
var post = await Visible(id, token);
return post == default ? NotFoundError() : Json(await _mapper.Status(post, MyId, token));
}
[HttpPut("/api/v1/statuses/{id}"), Scope("write:statuses")]
public async Task<IActionResult> Edit(string id, CancellationToken token)
{
var outcome = await _statuses.Edit(Me, id, new StatusDraft
{
Text = Params.Get("status"),
PlainText = true,
SpoilerText = Params.Get("spoiler_text"),
Sensitive = Params.Bool("sensitive") ?? false,
Language = Params.Get("language"),
MediaIds = Params.Has("media_ids") ? Params.List("media_ids") : default
}, token);
return outcome.Ok ? Json(await _mapper.Status(outcome.Post, MyId, token)) : Error(outcome.Status, outcome.Error);
}
[HttpDelete("/api/v1/statuses/{id}"), Scope("write:statuses")]
public async Task<IActionResult> Delete(string id, CancellationToken token)
{
var before = await _dbEntities.Posts.Match(p => p.ID == id && p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
if (before == default)
return NotFoundError();
var status = await _mapper.Status(before, MyId, token);
var outcome = await _statuses.Remove(Me, id, token);
if (!outcome.Ok)
return Error(outcome.Status, outcome.Error);
if (status != default)
{
status.Text = before.Text;
status.SpoilerText = before.SpoilerText ?? string.Empty;
}
return Json(status);
}
[HttpGet("/api/v1/statuses/{id}/context"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Context(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
var ancestors = new List<PostEntity>();
var cursor = post;
while (!string.IsNullOrEmpty(cursor.AnsweringToPostId) && ancestors.Count < MaxContext)
{
cursor = await _dbEntities.Posts.MatchID(cursor.AnsweringToPostId).ExecuteFirstAsync(token);
if (cursor == default || cursor.DeletedAt.HasValue || !await VisibilityPolicy.CanSee(cursor, MyId, token))
break;
ancestors.Insert(0, cursor);
}
var descendants = new List<PostEntity>();
var frontier = new List<string> { post.ID };
while (frontier.Count > 0 && descendants.Count < MaxContext)
{
var parents = frontier;
var children = await _dbEntities.Posts.Match(p => parents.Contains(p.AnsweringToPostId) && !p.DeletedAt.HasValue && p.ReblogOfPostId == null)
.Sort(p => p.ID, Order.Ascending).Limit(MaxContext).ExecuteAsync(token);
frontier = new List<string>();
foreach (var child in children)
if (await VisibilityPolicy.CanSee(child, MyId, token))
{
descendants.Add(child);
frontier.Add(child.ID);
}
}
return Json(new Context
{
Ancestors = await _mapper.Statuses(ancestors, MyId, token),
Descendants = await _mapper.Statuses(descendants.OrderBy(d => d.ID, StringComparer.Ordinal).ToList(), MyId, token)
});
}
[HttpGet("/api/v1/statuses/{id}/history"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> History(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
var account = await _mapper.Account(MastodonMapper.AuthorOf(post), token);
var edits = post.Revisions.Select(r => new StatusEdit
{
Content = r.ContentHtml ?? string.Empty,
SpoilerText = r.SpoilerText ?? string.Empty,
Sensitive = r.HasContentWarning,
CreatedAt = MastodonJson.Time(r.EditedAt),
Account = account
}).ToList();
edits.Add(new StatusEdit
{
Content = MastodonMapper.Content(post),
SpoilerText = post.SpoilerText ?? string.Empty,
Sensitive = post.HasContentWarning,
CreatedAt = MastodonJson.Time(post.EditedAt ?? post.CreationDate),
Account = account
});
return Json(edits);
}
[HttpGet("/api/v1/statuses/{id}/source"), Scope("read:statuses")]
public async Task<IActionResult> Source(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
return post == default ? NotFoundError() : Json(new StatusSource { Id = post.ID, Text = post.Text ?? string.Empty, SpoilerText = post.SpoilerText ?? string.Empty });
}
[HttpPost("/api/v1/statuses/{id}/favourite"), Scope("write:favourites")]
public Task<IActionResult> Favourite(string id, CancellationToken token) => Toggle(_statuses.Favourite(Me, id, true, token), id, token);
[HttpPost("/api/v1/statuses/{id}/unfavourite"), Scope("write:favourites")]
public Task<IActionResult> Unfavourite(string id, CancellationToken token) => Toggle(_statuses.Favourite(Me, id, false, token), id, token);
[HttpPost("/api/v1/statuses/{id}/reblog"), Scope("write:statuses")]
public async Task<IActionResult> Reblog(string id, CancellationToken token)
{
var outcome = await _statuses.Reblog(Me, id, true, Visibility(Params.Get("visibility")), token);
return outcome.Ok ? Json(await _mapper.Status(outcome.Post, MyId, token)) : Error(outcome.Status, outcome.Error);
}
[HttpPost("/api/v1/statuses/{id}/unreblog"), Scope("write:statuses")]
public Task<IActionResult> Unreblog(string id, CancellationToken token) => Toggle(_statuses.Reblog(Me, id, false, PostVisibility.Public, token), id, token);
[HttpGet("/api/v1/statuses/{id}/favourited_by"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> FavouritedBy(string id, CancellationToken token)
{
if (await Visible(id, token) == default)
return NotFoundError();
var favourites = await _dbEntities.Favourites.Match(f => f.PostId == id).Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
var accounts = await _mapper.Accounts(favourites.Select(f => f.AccountId), token);
return Json(favourites.Select(f => accounts.GetValueOrDefault(f.AccountId)).Where(a => a != default).ToList());
}
[HttpGet("/api/v1/statuses/{id}/reblogged_by"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> RebloggedBy(string id, CancellationToken token)
{
if (await Visible(id, token) == default)
return NotFoundError();
var reblogs = await _dbEntities.Posts.Match(p => p.ReblogOfPostId == id && !p.DeletedAt.HasValue).Sort(p => p.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
var accounts = await _mapper.Accounts(reblogs.Select(MastodonMapper.AuthorOf), token);
return Json(reblogs.Select(r => accounts.GetValueOrDefault(MastodonMapper.AuthorOf(r))).Where(a => a != default).ToList());
}
[HttpGet("/api/v1/statuses/{id}/quotes"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Quotes(string id, CancellationToken token)
{
if (await Visible(id, token) == default)
return NotFoundError();
var query = _dbEntities.Posts.Match(p => p.QuotedPostId == id && p.QuoteState == QuoteState.Accepted && !p.DeletedAt.HasValue);
var quotes = await Page.From(Params, Limit()).Fetch(query, p => p.ID, token);
var visible = new List<PostEntity>();
foreach (var quote in quotes)
if (quote.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quote, MyId, token))
visible.Add(quote);
Link($"/api/v1/statuses/{id}/quotes", quotes.LastOrDefault()?.ID, quotes.FirstOrDefault()?.ID);
return Json(await _mapper.Statuses(visible, MyId, token));
}
[HttpPost("/api/v1/statuses/{id}/quotes/{quotingId}/revoke"), Scope("write:statuses")]
public async Task<IActionResult> RevokeQuote(string id, string quotingId, CancellationToken token)
{
var quoted = await _dbEntities.Posts.Match(p => p.ID == id && p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
var quoting = quoted == default ? default : await _dbEntities.Posts.Match(p => p.ID == quotingId && p.QuotedPostId == quoted.ID).ExecuteFirstAsync(token);
if (quoting == default || !await _quotes.RevokeLicence(quoted, quoting, token))
return NotFoundError();
return Json(await _mapper.Status(await _dbEntities.Posts.MatchID(quoting.ID).ExecuteFirstAsync(token), MyId, token));
}
[HttpPut("/api/v1/statuses/{id}/interaction_policy"), Scope("write:statuses")]
public async Task<IActionResult> InteractionPolicy(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
if (post == default)
return NotFoundError();
var policy = Params.Get("quote_approval_policy");
if (!QuotePolicies.IsKnown(policy))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Unknown quote approval policy");
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.LocalQuotePolicy, policy).ExecuteAsync(token);
post.LocalQuotePolicy = policy;
if (!post.IsLocalOnly)
await _outbox.Publish(Me, post, ActivityPubRenderer.UpdateOf(post, Me, $"policy-{DateTime.UtcNow.Ticks}"), token);
return Json(await _mapper.Status(post, MyId, token));
}
[HttpPost("/api/v1/statuses/{id}/bookmark"), Scope("write:bookmarks")]
public async Task<IActionResult> Bookmark(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
try
{
await DB.Default.SaveAsync(new Models.Social.Bookmark { AvatarId = MyId, PostId = post.ID }, token);
}
catch (MongoDB.Driver.MongoWriteException ex) when (ex.WriteError?.Category == MongoDB.Driver.ServerErrorCategory.DuplicateKey)
{
}
return Json(await _mapper.Status(post, MyId, token));
}
[HttpPost("/api/v1/statuses/{id}/unbookmark"), Scope("write:bookmarks")]
public async Task<IActionResult> Unbookmark(string id, CancellationToken token)
{
await DB.Default.DeleteAsync<Models.Social.Bookmark>(b => b.AvatarId == MyId && b.PostId == id);
return await Unchanged(id, token);
}
[HttpGet("/api/v1/bookmarks"), Scope("read:bookmarks")]
public async Task<IActionResult> Bookmarks(CancellationToken token)
{
var bookmarks = await Page.From(Params, Limit()).Fetch(DB.Default.Find<Models.Social.Bookmark>().Match(b => b.AvatarId == MyId), b => b.ID, token);
var ids = bookmarks.Select(b => b.PostId).ToList();
var posts = (await _dbEntities.Posts.Match(p => ids.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)).ToDictionary(p => p.ID);
var visible = new List<PostEntity>();
foreach (var postId in ids.Where(posts.ContainsKey))
if (await VisibilityPolicy.CanSee(posts[postId], MyId, token))
visible.Add(posts[postId]);
Link("/api/v1/bookmarks", bookmarks.LastOrDefault()?.ID, bookmarks.FirstOrDefault()?.ID);
return Json(await _mapper.Statuses(visible, MyId, token));
}
[HttpPost("/api/v1/statuses/{id}/mute"), Scope("write:mutes")]
public Task<IActionResult> Mute(string id, CancellationToken token) => Unchanged(id, token);
[HttpPost("/api/v1/statuses/{id}/unmute"), Scope("write:mutes")]
public Task<IActionResult> Unmute(string id, CancellationToken token) => Unchanged(id, token);
[HttpPost("/api/v1/statuses/{id}/pin"), Scope("write:accounts")]
public async Task<IActionResult> Pin(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.ReblogOfPostId == null)
.ExecuteFirstAsync(token);
if (post == default)
return NotFoundError();
if (post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Only public posts can be pinned");
if (await DB.Default.CountAsync<Models.Social.Pin>(p => p.AvatarId == MyId, token) >= MaxPins)
return Error(StatusCodes.Status422UnprocessableEntity, $"Validation failed: You can pin at most {MaxPins} posts");
try
{
await DB.Default.SaveAsync(new Models.Social.Pin { AvatarId = MyId, PostId = post.ID }, token);
}
catch (MongoDB.Driver.MongoWriteException ex) when (ex.WriteError?.Category == MongoDB.Driver.ServerErrorCategory.DuplicateKey)
{
}
return Json(await _mapper.Status(post, MyId, token));
}
[HttpPost("/api/v1/statuses/{id}/unpin"), Scope("write:accounts")]
public async Task<IActionResult> Unpin(string id, CancellationToken token)
{
await DB.Default.DeleteAsync<Models.Social.Pin>(p => p.AvatarId == MyId && p.PostId == id);
return await Unchanged(id, token);
}
async Task<IActionResult> Toggle(Task<StatusOutcome> action, string id, CancellationToken token)
{
var outcome = await action;
if (!outcome.Ok)
return Error(outcome.Status, outcome.Error);
var post = await _dbEntities.Posts.MatchID(id).ExecuteFirstAsync(token);
return Json(await _mapper.Status(post, MyId, token));
}
async Task<IActionResult> Unchanged(string id, CancellationToken token)
{
var post = await Visible(id, token);
return post == default ? NotFoundError() : Json(await _mapper.Status(post, MyId, token));
}
async Task<PostEntity> Visible(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
return post != default && post.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(post, MyId, token) ? post : default;
}
static PostVisibility Visibility(string value) => value?.ToLowerInvariant() switch
{
"unlisted" => PostVisibility.Unlisted,
"private" => PostVisibility.FollowersOnly,
"direct" => PostVisibility.Direct,
_ => PostVisibility.Public
};
}
}
@@ -0,0 +1,248 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
using NotificationEntity = PrivaPub.Models.Social.Notification;
using MarkerEntity = PrivaPub.Models.Social.Marker;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class TimelinesController : MastodonController
{
readonly MastodonMapper _mapper;
readonly DbEntities _dbEntities;
public TimelinesController(MastodonMapper mapper, DbEntities dbEntities)
{
_mapper = mapper;
_dbEntities = dbEntities;
}
[HttpGet("/api/v1/timelines/home"), Scope("read:statuses")]
public async Task<IActionResult> Home(CancellationToken token)
{
var entries = await Page.From(Params, Limit()).Fetch(_dbEntities.TimelineEntries.Match(e => e.AvatarId == MyId), e => e.PostId, token);
var ids = entries.Select(e => e.PostId).ToList();
var posts = (await _dbEntities.Posts.Match(p => ids.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)).ToDictionary(p => p.ID);
var statuses = await _mapper.Statuses(ids.Where(posts.ContainsKey).Select(id => posts[id]).ToList(), MyId, token);
Link("/api/v1/timelines/home", entries.LastOrDefault()?.PostId, entries.FirstOrDefault()?.PostId);
return Json(statuses);
}
[HttpGet("/api/v1/timelines/public"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Public(CancellationToken token)
{
var query = _dbEntities.Posts.Match(p => p.Visibility == PostVisibility.Public && !p.DeletedAt.HasValue && p.ReblogOfPostId == null);
if (Params.Bool("local") == true)
query.Match(p => !p.IsFederatedCopy);
if (Params.Bool("remote") == true)
query.Match(p => p.IsFederatedCopy);
if (Params.Bool("only_media") == true)
query.Match(p => p.Media.Count > 0);
return await Respond(query, "/api/v1/timelines/public", token);
}
[HttpGet("/api/v1/timelines/tag/{hashtag}"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Tag(string hashtag, CancellationToken token)
{
var tag = hashtag.TrimStart('#').ToLowerInvariant();
var query = _dbEntities.Posts.Match(p => p.Tags.Contains(tag) && p.Visibility == PostVisibility.Public && !p.DeletedAt.HasValue && p.ReblogOfPostId == null);
if (Params.Bool("local") == true)
query.Match(p => !p.IsFederatedCopy);
return await Respond(query, $"/api/v1/timelines/tag/{hashtag}", token);
}
[HttpGet("/api/v1/timelines/list/{id}"), Scope("read:lists")]
public IActionResult List(string id) => Json(Array.Empty<Status>());
[HttpGet("/api/v1/favourites"), Scope("read:favourites")]
public async Task<IActionResult> Favourites(CancellationToken token)
{
var favourites = await Page.From(Params, Limit()).Fetch(_dbEntities.Favourites.Match(f => f.AccountId == MyId), f => f.ID, token);
var ids = favourites.Select(f => f.PostId).ToList();
var posts = (await _dbEntities.Posts.Match(p => ids.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)).ToDictionary(p => p.ID);
var visible = new List<PostEntity>();
foreach (var id in ids.Where(posts.ContainsKey))
if (await VisibilityPolicy.CanSee(posts[id], MyId, token))
visible.Add(posts[id]);
Link("/api/v1/favourites", favourites.LastOrDefault()?.ID, favourites.FirstOrDefault()?.ID);
return Json(await _mapper.Statuses(visible, MyId, token));
}
[HttpGet("/api/v1/conversations"), Scope("read:statuses")]
public async Task<IActionResult> Conversations(CancellationToken token)
{
var groups = await _dbEntities.DmGroups.Match(g => !g.DeletionAt.HasValue && g.Members.Any(m => !m.IsForeign && m.AvatarId == MyId))
.Sort(g => g.UpdatedAt, Order.Descending).Limit(Limit()).ExecuteAsync(token);
var conversations = new List<Conversation>();
foreach (var group in groups)
{
var last = await _dbEntities.Posts.Match(p => p.ConversationId == group.ID && !p.DeletedAt.HasValue).Sort(p => p.ID, Order.Descending).ExecuteFirstAsync(token);
if (last == default)
continue;
var others = new List<string>();
foreach (var member in group.Members.Where(m => !(m.AvatarId == MyId && !m.IsForeign)))
others.Add(member.IsForeign
? (await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == member.AvatarId).ExecuteFirstAsync(token))?.ID
: member.AvatarId);
var accounts = await _mapper.Accounts(others, token);
conversations.Add(new Conversation
{
Id = group.ID,
Accounts = accounts.Values.ToList(),
LastStatus = await _mapper.Status(last, MyId, token)
});
}
return Json(conversations);
}
[HttpPost("/api/v1/conversations/{id}/read"), Scope("write:conversations")]
public async Task<IActionResult> ReadConversation(string id, CancellationToken token)
{
var group = await _dbEntities.DmGroups.Match(g => g.ID == id && g.Members.Any(m => !m.IsForeign && m.AvatarId == MyId)).ExecuteFirstAsync(token);
if (group == default)
return NotFoundError();
var last = await _dbEntities.Posts.Match(p => p.ConversationId == group.ID && !p.DeletedAt.HasValue).Sort(p => p.ID, Order.Descending).ExecuteFirstAsync(token);
return Json(new Conversation { Id = group.ID, LastStatus = last == default ? default : await _mapper.Status(last, MyId, token) });
}
[HttpGet("/api/v1/markers"), Scope("read:statuses")]
public async Task<IActionResult> Markers(CancellationToken token)
{
var wanted = Params.List("timeline");
var markers = await DB.Default.Find<MarkerEntity>().Match(m => m.AvatarId == MyId).ExecuteAsync(token);
return Json(markers.Where(m => wanted.Count == 0 || wanted.Contains(m.Timeline))
.ToDictionary(m => m.Timeline, m => new Entities.Marker { LastReadId = m.LastReadId, Version = m.Version, UpdatedAt = MastodonJson.Time(m.UpdatedAt) }));
}
[HttpPost("/api/v1/markers"), Scope("write:statuses")]
public async Task<IActionResult> SaveMarkers(CancellationToken token)
{
var result = new Dictionary<string, Entities.Marker>();
foreach (var timeline in new[] { "home", "notifications" })
{
var lastRead = Params.Get($"{timeline}[last_read_id]");
if (string.IsNullOrEmpty(lastRead))
continue;
var marker = await DB.Default.UpdateAndGet<MarkerEntity>()
.Match(m => m.AvatarId == MyId && m.Timeline == timeline)
.Modify(m => m.LastReadId, lastRead)
.Modify(m => m.UpdatedAt, DateTime.UtcNow)
.Modify(b => b.Inc(m => m.Version, 1))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
result[timeline] = new Entities.Marker { LastReadId = marker.LastReadId, Version = marker.Version, UpdatedAt = MastodonJson.Time(marker.UpdatedAt) };
}
return Json(result);
}
async Task<IActionResult> Respond(Find<PostEntity, PostEntity> query, string path, CancellationToken token)
{
var posts = await Page.From(Params, Limit()).Fetch(query, p => p.ID, token);
Link(path, posts.LastOrDefault()?.ID, posts.FirstOrDefault()?.ID);
return Json(await _mapper.Statuses(posts, MyId, token));
}
}
public class NotificationsController : MastodonController
{
static readonly Dictionary<NotificationType, string> Names = new()
{
[NotificationType.Mention] = "mention",
[NotificationType.Follow] = "follow",
[NotificationType.FollowRequest] = "follow_request",
[NotificationType.Favourite] = "favourite",
[NotificationType.Reblog] = "reblog",
[NotificationType.Update] = "update",
[NotificationType.Poll] = "poll",
[NotificationType.Reaction] = "pleroma:emoji_reaction",
[NotificationType.Quote] = "quote"
};
readonly MastodonMapper _mapper;
readonly DbEntities _dbEntities;
public NotificationsController(MastodonMapper mapper, DbEntities dbEntities)
{
_mapper = mapper;
_dbEntities = dbEntities;
}
[HttpGet("/api/v1/notifications"), Scope("read:notifications")]
public async Task<IActionResult> List(CancellationToken token)
{
var types = Params.List("types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList();
var excluded = Params.List("exclude_types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList();
var query = _dbEntities.Notifications.Match(n => n.AvatarId == MyId);
if (types.Count > 0)
query.Match(n => types.Contains(n.Type));
if (excluded.Count > 0)
query.Match(n => !excluded.Contains(n.Type));
if (Params.Get("account_id") is { } accountId)
query.Match(n => n.FromAccountId == accountId);
var notifications = await Page.From(Params, Limit(15, 30)).Fetch(query, n => n.ID, token);
Link("/api/v1/notifications", notifications.LastOrDefault()?.ID, notifications.FirstOrDefault()?.ID);
return Json(await Map(notifications, token));
}
[HttpGet("/api/v1/notifications/{id}"), Scope("read:notifications")]
public async Task<IActionResult> Get(string id, CancellationToken token)
{
var notification = await _dbEntities.Notifications.Match(n => n.ID == id && n.AvatarId == MyId).ExecuteFirstAsync(token);
return notification == default ? NotFoundError() : Json((await Map(new List<NotificationEntity> { notification }, token)).FirstOrDefault());
}
[HttpPost("/api/v1/notifications/clear"), Scope("write:notifications")]
public async Task<IActionResult> Clear(CancellationToken token)
{
await DB.Default.DeleteAsync<NotificationEntity>(n => n.AvatarId == MyId);
return Json(new { });
}
[HttpPost("/api/v1/notifications/{id}/dismiss"), Scope("write:notifications")]
public async Task<IActionResult> Dismiss(string id, CancellationToken token)
{
await DB.Default.DeleteAsync<NotificationEntity>(n => n.ID == id && n.AvatarId == MyId);
return Json(new { });
}
[HttpGet("/api/v1/notifications/unread_count"), Scope("read:notifications")]
public async Task<IActionResult> UnreadCount(CancellationToken token) =>
Json(new { count = await DB.Default.CountAsync<NotificationEntity>(n => n.AvatarId == MyId && !n.IsRead, token) });
async Task<List<Entities.Notification>> Map(List<NotificationEntity> notifications, CancellationToken token)
{
var accounts = await _mapper.Accounts(notifications.Select(n => n.FromAccountId), token);
var postIds = notifications.Where(n => n.PostId != default).Select(n => n.PostId).Distinct().ToList();
var posts = postIds.Count == 0
? new List<PostEntity>()
: await _dbEntities.Posts.Match(p => postIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token);
var statuses = (await _mapper.Statuses(posts, MyId, token)).ToDictionary(s => s.Id);
return notifications
.Where(n => accounts.ContainsKey(n.FromAccountId ?? string.Empty) && (n.PostId == default || statuses.ContainsKey(n.PostId)))
.Select(n => new Entities.Notification
{
Id = n.ID,
Type = Names[n.Type],
CreatedAt = MastodonJson.Time(n.CreatedAt),
GroupKey = $"ungrouped-{n.ID}",
Account = accounts[n.FromAccountId],
Status = n.PostId == default ? default : statuses[n.PostId],
Emoji = n.Emoji == default ? default : n.EmojiURL == default ? n.Emoji : $":{n.Emoji}:",
EmojiUrl = _mapper.ProxiedUrl(n.EmojiURL)
})
.ToList();
}
static NotificationType? Parse(string name) => Names.FirstOrDefault(n => n.Value == name) is { Value: not null } pair ? pair.Key : (NotificationType?)null;
}
}
+391
View File
@@ -0,0 +1,391 @@
namespace PrivaPub.Api.Mastodon.Entities
{
public class Account
{
public string Id { get; set; }
public string Username { get; set; }
public string Acct { get; set; }
public string DisplayName { get; set; }
public bool Locked { get; set; }
public bool Bot { get; set; }
public bool? Discoverable { get; set; }
public bool Indexable { get; set; }
public bool Group { get; set; }
public string CreatedAt { get; set; }
public string Note { get; set; }
public string Url { get; set; }
public string Uri { get; set; }
public string Avatar { get; set; }
public string AvatarStatic { get; set; }
public string Header { get; set; }
public string HeaderStatic { get; set; }
public int FollowersCount { get; set; }
public int FollowingCount { get; set; }
public int StatusesCount { get; set; }
public string LastStatusAt { get; set; }
public bool? HideCollections { get; set; }
public bool Noindex { get; set; } = true;
public List<CustomEmojiEntity> Emojis { get; set; } = new();
public List<object> Roles { get; set; } = new();
public bool? Memorial { get; set; }
public string AvatarDescription { get; set; } = string.Empty;
public string HeaderDescription { get; set; } = string.Empty;
public List<Field> Fields { get; set; } = new();
public Source Source { get; set; }
}
public class CustomEmojiEntity
{
public string Shortcode { get; set; }
public string Url { get; set; }
public string StaticUrl { get; set; }
public bool VisibleInPicker { get; set; }
public string Category { get; set; }
}
public class Field
{
public string Name { get; set; }
public string Value { get; set; }
public string VerifiedAt { get; set; }
}
public class Source
{
public string Privacy { get; set; } = "public";
public bool Sensitive { get; set; }
public string Language { get; set; }
public string Note { get; set; } = string.Empty;
public List<Field> Fields { get; set; } = new();
public int FollowRequestsCount { get; set; }
public string AttributionDomains { get; set; }
public string QuotePolicy { get; set; } = "public";
}
public class PrivaPubStatus
{
public string ObjectType { get; set; }
public string Title { get; set; }
public string Excerpt { get; set; }
public string Cover { get; set; }
public PrivaPubSource Source { get; set; }
public PrivaPubLink Link { get; set; }
public PrivaPubVideo Video { get; set; }
public PrivaPubAudio Audio { get; set; }
public PrivaPubEvent Event { get; set; }
public PrivaPubVotes Votes { get; set; } = new();
}
public class PrivaPubSource
{
public string Content { get; set; }
public string MediaType { get; set; }
}
public class PrivaPubLink
{
public string Url { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public string Image { get; set; }
public bool FromPublisher { get; set; }
}
public class PrivaPubVariant
{
public string Url { get; set; }
public string MediaType { get; set; }
public int? Width { get; set; }
public int? Height { get; set; }
public double? Fps { get; set; }
public long? Size { get; set; }
public List<string> Streams { get; set; } = new();
}
public class PrivaPubVideo
{
public double? Duration { get; set; }
public List<PrivaPubVariant> Variants { get; set; } = new();
public bool IsLive { get; set; }
public int? State { get; set; }
public long? Views { get; set; }
public int? CommentsPolicy { get; set; }
public bool? DownloadEnabled { get; set; }
public string Licence { get; set; }
public string Category { get; set; }
public string Channel { get; set; }
public List<PrivaPubCaption> Captions { get; set; } = new();
}
public class PrivaPubCaption
{
public string Language { get; set; }
public string Url { get; set; }
}
public class PrivaPubAudio
{
public double? Duration { get; set; }
public List<PrivaPubVariant> Variants { get; set; } = new();
public string Album { get; set; }
public int? Position { get; set; }
public string Licence { get; set; }
}
public class PrivaPubEvent
{
public string Start { get; set; }
public string End { get; set; }
public string TimeZone { get; set; }
public bool Floating { get; set; }
public List<PrivaPubPlace> Places { get; set; } = new();
public string OnlineUrl { get; set; }
public string JoinMode { get; set; }
public int? Participants { get; set; }
public int? Capacity { get; set; }
public string Status { get; set; }
public string Category { get; set; }
}
public class PrivaPubPlace
{
public string Name { get; set; }
public string Address { get; set; }
public double? Latitude { get; set; }
public double? Longitude { get; set; }
public string Url { get; set; }
}
public class PrivaPubVotes
{
public int Up { get; set; }
public int Down { get; set; }
}
public class QuoteEntity
{
public string State { get; set; }
public Status QuotedStatus { get; set; }
public string QuotedStatusId { get; set; }
}
public class QuoteApprovalEntity
{
public List<string> Automatic { get; set; } = new();
public List<string> Manual { get; set; } = new();
public string CurrentUser { get; set; } = "denied";
}
public class EmojiReactionEntity
{
public string Name { get; set; }
public int Count { get; set; }
public bool Me { get; set; }
public string Url { get; set; }
public string StaticUrl { get; set; }
}
public class PleromaStatus
{
public List<EmojiReactionEntity> EmojiReactions { get; set; } = new();
}
public class PollEntity
{
public string Id { get; set; }
public string ExpiresAt { get; set; }
public bool Expired { get; set; }
public bool Multiple { get; set; }
public int VotesCount { get; set; }
public int? VotersCount { get; set; }
public bool? Voted { get; set; }
public List<int> OwnVotes { get; set; }
public List<PollOptionEntity> Options { get; set; } = new();
public List<CustomEmojiEntity> Emojis { get; set; } = new();
}
public class PollOptionEntity
{
public string Title { get; set; }
public int? VotesCount { get; set; }
}
public class PreviewCard
{
public string Url { get; set; }
public string Title { get; set; } = string.Empty;
public string Description { get; set; } = string.Empty;
public string Type { get; set; } = "link";
public string AuthorName { get; set; } = string.Empty;
public string AuthorUrl { get; set; } = string.Empty;
public string ProviderName { get; set; } = string.Empty;
public string ProviderUrl { get; set; } = string.Empty;
public string Html { get; set; } = string.Empty;
public int Width { get; set; }
public int Height { get; set; }
public string Image { get; set; }
public string ImageDescription { get; set; } = string.Empty;
public string EmbedUrl { get; set; } = string.Empty;
public string Blurhash { get; set; }
public string Language { get; set; }
public string PublishedAt { get; set; }
public List<object> Authors { get; set; } = new();
}
public class Status
{
public string Id { get; set; }
public string Uri { get; set; }
public string Url { get; set; }
public Account Account { get; set; }
public string InReplyToId { get; set; }
public string InReplyToAccountId { get; set; }
public Status Reblog { get; set; }
public string Content { get; set; } = string.Empty;
public string CreatedAt { get; set; }
public string EditedAt { get; set; }
public List<CustomEmojiEntity> Emojis { get; set; } = new();
public int RepliesCount { get; set; }
public int ReblogsCount { get; set; }
public int FavouritesCount { get; set; }
public bool Reblogged { get; set; }
public bool Favourited { get; set; }
public bool Muted { get; set; }
public bool Bookmarked { get; set; }
public bool Pinned { get; set; }
public bool Sensitive { get; set; }
public string SpoilerText { get; set; } = string.Empty;
public string Visibility { get; set; }
public List<MediaAttachment> MediaAttachments { get; set; } = new();
public List<StatusMention> Mentions { get; set; } = new();
public List<StatusTag> Tags { get; set; } = new();
public PreviewCard Card { get; set; }
public PollEntity Poll { get; set; }
public Application Application { get; set; }
public string Language { get; set; }
public string Text { get; set; }
public List<object> Filtered { get; set; } = new();
public PrivaPubStatus Privapub { get; set; }
public List<EmojiReactionEntity> EmojiReactions { get; set; } = new();
public PleromaStatus Pleroma { get; set; }
public QuoteEntity Quote { get; set; }
public int QuotesCount { get; set; }
public QuoteApprovalEntity QuoteApproval { get; set; }
}
public class MediaAttachment
{
public string Id { get; set; }
public string Type { get; set; }
public string Url { get; set; }
public string PreviewUrl { get; set; }
public string RemoteUrl { get; set; }
public object Meta { get; set; }
public string Description { get; set; }
public string Blurhash { get; set; }
}
public class StatusMention
{
public string Id { get; set; }
public string Username { get; set; }
public string Url { get; set; }
public string Acct { get; set; }
}
public class StatusTag
{
public string Name { get; set; }
public string Url { get; set; }
}
public class Application
{
public string Id { get; set; }
public string Name { get; set; }
public string Website { get; set; }
public List<string> Scopes { get; set; } = new();
public List<string> RedirectUris { get; set; } = new();
public string RedirectUri { get; set; }
public string ClientId { get; set; }
public string ClientSecret { get; set; }
public string VapidKey { get; set; } = string.Empty;
}
public class Relationship
{
public string Id { get; set; }
public bool Following { get; set; }
public bool ShowingReblogs { get; set; }
public bool Notifying { get; set; }
public List<string> Languages { get; set; }
public bool FollowedBy { get; set; }
public bool Blocking { get; set; }
public bool BlockedBy { get; set; }
public bool Muting { get; set; }
public bool MutingNotifications { get; set; }
public bool Requested { get; set; }
public bool RequestedBy { get; set; }
public bool DomainBlocking { get; set; }
public bool Endorsed { get; set; }
public string Note { get; set; } = string.Empty;
}
public class Notification
{
public string Id { get; set; }
public string Type { get; set; }
public string CreatedAt { get; set; }
public string GroupKey { get; set; }
public Account Account { get; set; }
public Status Status { get; set; }
public string Emoji { get; set; }
public string EmojiUrl { get; set; }
}
public class Context
{
public List<Status> Ancestors { get; set; } = new();
public List<Status> Descendants { get; set; } = new();
}
public class StatusEdit
{
public string Content { get; set; }
public string SpoilerText { get; set; }
public bool Sensitive { get; set; }
public string CreatedAt { get; set; }
public Account Account { get; set; }
public List<MediaAttachment> MediaAttachments { get; set; } = new();
public List<CustomEmojiEntity> Emojis { get; set; } = new();
}
public class StatusSource
{
public string Id { get; set; }
public string Text { get; set; }
public string SpoilerText { get; set; }
}
public class SearchResults
{
public List<Account> Accounts { get; set; } = new();
public List<Status> Statuses { get; set; } = new();
public List<object> Hashtags { get; set; } = new();
}
public class Conversation
{
public string Id { get; set; }
public bool Unread { get; set; }
public List<Account> Accounts { get; set; } = new();
public Status LastStatus { get; set; }
}
public class Marker
{
public string LastReadId { get; set; }
public int Version { get; set; }
public string UpdatedAt { get; set; }
}
}
@@ -0,0 +1,105 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Filters;
using MongoDB.Entities;
using OpenIddict.Abstractions;
using OpenIddict.Validation.AspNetCore;
using PrivaPub.Api.Mastodon.Auth;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using static OpenIddict.Abstractions.OpenIddictConstants;
namespace PrivaPub.Api.Mastodon.Infrastructure
{
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
public sealed class ScopeAttribute : Attribute
{
public ScopeAttribute(string scope, bool requiresUser = true)
{
Scope = scope;
RequiresUser = requiresUser;
}
public string Scope { get; }
public bool RequiresUser { get; }
}
[ApiController, Authorize(AuthenticationSchemes = OpenIddictValidationAspNetCoreDefaults.AuthenticationScheme)]
public abstract class MastodonController : ControllerBase, IAsyncActionFilter
{
MastodonParams _params;
protected LocalActor Me { get; private set; }
protected string MyId => Me?.Id;
protected MastodonParams Params => _params;
[NonAction]
public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
_params = await MastodonParams.Read(Request, HttpContext.RequestAborted);
var scope = context.ActionDescriptor.EndpointMetadata.OfType<ScopeAttribute>().LastOrDefault();
if (User.Identity?.IsAuthenticated == true)
{
if (scope != default && !MastodonScopes.Grants(User.GetScopes(), scope.Scope))
{
context.Result = Error(StatusCodes.Status403Forbidden, "This action is outside the authorized scopes");
return;
}
Me = await Resolve(User.GetClaim(Claims.Subject), HttpContext.RequestAborted);
}
if (scope?.RequiresUser == true && Me == default)
{
context.Result = Error(StatusCodes.Status401Unauthorized, "This method requires an authenticated user");
return;
}
await next();
}
async Task<LocalActor> Resolve(string avatarId, CancellationToken token)
{
if (string.IsNullOrEmpty(avatarId))
return default;
var services = HttpContext.RequestServices;
var db = services.GetRequiredService<DbEntities>();
var link = await db.RootToAvatars.Match(r => r.AvatarId == avatarId).ExecuteFirstAsync(token);
if (link == default)
return default;
var root = await db.RootUsers.MatchID(link.RootId).ExecuteFirstAsync(token);
if (root is not { IsBanned: false, DeletedAt: null })
return default;
return await services.GetRequiredService<ILocalActorService>().FindById(LocalActorKind.Person, avatarId, token);
}
protected JsonResult Json(object value, int status = StatusCodes.Status200OK) =>
new(value, MastodonJson.Options) { StatusCode = status };
protected static JsonResult Error(int status, string message) =>
new(new { error = message }, MastodonJson.Options) { StatusCode = status };
protected JsonResult NotFoundError() => Error(StatusCodes.Status404NotFound, "Record not found");
protected void Link(string path, string maxId, string minId)
{
var links = new List<string>();
var baseUrl = $"{Request.Scheme}://{Request.Host}{path}";
var query = Request.Query.Where(q => q.Key is not ("max_id" or "min_id" or "since_id")).Select(q => $"{q.Key}={Uri.EscapeDataString(q.Value.ToString())}").ToList();
string With(string key, string value) => baseUrl + "?" + string.Join("&", query.Append($"{key}={value}"));
if (maxId != default)
links.Add($"<{With("max_id", maxId)}>; rel=\"next\"");
if (minId != default)
links.Add($"<{With("min_id", minId)}>; rel=\"prev\"");
if (links.Count > 0)
Response.Headers["Link"] = string.Join(", ", links);
}
protected int Limit(int fallback = 20, int max = 40) => Math.Clamp(Params.Int("limit") ?? fallback, 1, max);
}
}
@@ -0,0 +1,29 @@
using System.Text.Json;
namespace PrivaPub.Api.Mastodon.Infrastructure
{
public static class MastodonErrors
{
public static IApplicationBuilder UseMastodonErrorBodies(this IApplicationBuilder app) =>
app.UseWhen(context => context.Request.Path.StartsWithSegments("/api"), api => api.UseStatusCodePages(async pages =>
{
var response = pages.HttpContext.Response;
response.ContentType = "application/json; charset=utf-8";
await response.WriteAsync(JsonSerializer.Serialize(new { error = Message(response.StatusCode) }, MastodonJson.Options));
}));
public static string Message(int status) => status switch
{
StatusCodes.Status401Unauthorized => "The access token is invalid",
StatusCodes.Status403Forbidden => "This action is outside the authorized scopes",
StatusCodes.Status404NotFound => "Record not found",
StatusCodes.Status405MethodNotAllowed => "Method not allowed",
StatusCodes.Status413PayloadTooLarge => "Payload too large",
StatusCodes.Status415UnsupportedMediaType => "Unsupported media type",
StatusCodes.Status422UnprocessableEntity => "Validation failed",
StatusCodes.Status429TooManyRequests => "Too many requests",
>= 500 => "Something went wrong.",
_ => "Bad request"
};
}
}
@@ -0,0 +1,24 @@
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace PrivaPub.Api.Mastodon.Infrastructure
{
public static class MastodonJson
{
public static readonly JsonSerializerOptions Options = new()
{
PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower,
DictionaryKeyPolicy = default,
DefaultIgnoreCondition = JsonIgnoreCondition.Never,
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
Converters = { new JsonStringEnumConverter(JsonNamingPolicy.SnakeCaseLower) }
};
public static string Time(DateTime value) =>
DateTime.SpecifyKind(value, DateTimeKind.Utc).ToString("yyyy-MM-ddTHH:mm:ss.fffZ", System.Globalization.CultureInfo.InvariantCulture);
public static string Day(DateTime value) =>
DateTime.SpecifyKind(value, DateTimeKind.Utc).Date.ToString("yyyy-MM-ddT00:00:00.000Z", System.Globalization.CultureInfo.InvariantCulture);
}
}
@@ -0,0 +1,109 @@
using Microsoft.AspNetCore.Http.Features;
using System.Text.Json;
namespace PrivaPub.Api.Mastodon.Infrastructure
{
public sealed class MastodonParams
{
readonly Dictionary<string, List<string>> _values;
MastodonParams(Dictionary<string, List<string>> values)
{
_values = values;
}
public static async Task<MastodonParams> Read(HttpRequest request, CancellationToken token)
{
var values = new Dictionary<string, List<string>>(StringComparer.Ordinal);
foreach (var pair in request.Query)
foreach (var value in pair.Value)
Add(values, pair.Key, value);
if (request.HasFormContentType)
{
var form = await request.ReadFormAsync(token);
foreach (var pair in form)
foreach (var value in pair.Value)
Add(values, pair.Key, value);
}
else if (request.ContentType?.Contains("json", StringComparison.OrdinalIgnoreCase) == true && request.ContentLength != 0)
{
try
{
using var document = await JsonDocument.ParseAsync(request.Body, cancellationToken: token);
Flatten(values, default, document.RootElement);
}
catch (JsonException)
{
}
}
return new MastodonParams(values);
}
public static MastodonParams From(IEnumerable<KeyValuePair<string, string>> pairs)
{
var values = new Dictionary<string, List<string>>(StringComparer.Ordinal);
foreach (var pair in pairs)
Add(values, pair.Key, pair.Value);
return new MastodonParams(values);
}
public bool Has(string name) => _values.ContainsKey(name);
public string Get(string name) => _values.TryGetValue(name, out var list) && list.Count > 0 ? list[^1] : default;
public IReadOnlyList<string> List(string name) => _values.TryGetValue(name, out var list) ? list : (IReadOnlyList<string>)Array.Empty<string>();
public bool? Bool(string name) => Get(name)?.ToLowerInvariant() switch
{
"true" or "1" or "on" or "yes" => true,
"false" or "0" or "off" or "no" or "" => false,
_ => (bool?)null
};
public int? Int(string name) => int.TryParse(Get(name), out var value) ? value : (int?)null;
static void Add(Dictionary<string, List<string>> values, string key, string value)
{
key = key.EndsWith("[]", StringComparison.Ordinal) ? key[..^2] : key;
if (!values.TryGetValue(key, out var list))
values[key] = list = new List<string>();
list.Add(value);
}
static void Flatten(Dictionary<string, List<string>> values, string prefix, JsonElement element)
{
switch (element.ValueKind)
{
case JsonValueKind.Object:
foreach (var property in element.EnumerateObject())
Flatten(values, prefix == default ? property.Name : $"{prefix}[{property.Name}]", property.Value);
break;
case JsonValueKind.Array:
foreach (var item in element.EnumerateArray())
{
if (item.ValueKind is JsonValueKind.Object or JsonValueKind.Array)
Flatten(values, prefix + "[]", item);
else
Add(values, prefix, Scalar(item));
}
break;
case JsonValueKind.Null or JsonValueKind.Undefined:
break;
default:
if (prefix != default)
Add(values, prefix, Scalar(element));
break;
}
}
static string Scalar(JsonElement element) => element.ValueKind switch
{
JsonValueKind.String => element.GetString(),
JsonValueKind.True => "true",
JsonValueKind.False => "false",
_ => element.GetRawText()
};
}
}
@@ -0,0 +1,30 @@
using MongoDB.Driver;
using MongoDB.Entities;
using System.Linq.Expressions;
namespace PrivaPub.Api.Mastodon.Infrastructure
{
public sealed record Page(string MaxId, string SinceId, string MinId, int Limit)
{
public static Page From(MastodonParams parameters, int limit) =>
new(parameters.Get("max_id"), parameters.Get("since_id"), parameters.Get("min_id"), limit);
public async Task<List<T>> Fetch<T>(Find<T, T> query, Expression<Func<T, string>> id, CancellationToken token) where T : IEntity
{
var field = new ExpressionFieldDefinition<T, string>(id);
if (!string.IsNullOrEmpty(MaxId))
query.Match(Builders<T>.Filter.Lt(field, MaxId));
if (!string.IsNullOrEmpty(SinceId))
query.Match(Builders<T>.Filter.Gt(field, SinceId));
if (!string.IsNullOrEmpty(MinId))
query.Match(Builders<T>.Filter.Gt(field, MinId));
var ascending = !string.IsNullOrEmpty(MinId);
var sortField = new ExpressionFieldDefinition<T>(Expression.Lambda<Func<T, object>>(Expression.Convert(id.Body, typeof(object)), id.Parameters));
var items = await query.Sort(s => ascending ? s.Ascending(sortField) : s.Descending(sortField)).Limit(Limit).ExecuteAsync(token);
if (ascending)
items.Reverse();
return items;
}
}
}
@@ -0,0 +1,55 @@
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.StaticServices;
namespace PrivaPub.Api.Mastodon.Mappers
{
public class AccountSearch
{
readonly MastodonMapper _mapper;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IRemoteActorService _remoteActors;
public AccountSearch(MastodonMapper mapper, DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors)
{
_mapper = mapper;
_dbEntities = dbEntities;
_localActors = localActors;
_remoteActors = remoteActors;
}
public async Task<List<Account>> Find(string q, bool resolve, int limit, CancellationToken token)
{
q = q?.Trim().TrimStart('@');
if (string.IsNullOrEmpty(q))
return new List<Account>();
var results = new List<Account>();
var parts = q.Split('@');
var localDomain = new Uri(_localActors.BaseAddress).Authority;
if (parts.Length == 2 && !parts[1].Equals(localDomain, StringComparison.OrdinalIgnoreCase) && !string.IsNullOrEmpty(parts[0]))
{
var userName = parts[0];
var domain = parts[1].ToLowerInvariant();
var known = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain).ExecuteFirstAsync(token);
if (known == default && resolve)
{
var actorUri = await _remoteActors.ResolveHandle(q, token);
known = actorUri == default ? default : await _remoteActors.GetActor(actorUri, refresh: false, token);
}
if (known != default)
results.Add(_mapper.Foreign(known));
return results;
}
var prefix = parts[0].ToLowerInvariant();
foreach (var avatar in await _dbEntities.Avatars.Match(a => a.UserName.StartsWith(prefix) && !a.DeletionAt.HasValue && a.Settings.IsDiscoverable != false)
.Limit(limit).ExecuteAsync(token))
results.Add(await _mapper.Local(_localActors.FromAvatar(avatar), false, token));
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => f.UserName.StartsWith(parts[0]) && f.IsDiscoverable)
.Limit(Math.Max(0, limit - results.Count)).ExecuteAsync(token))
results.Add(_mapper.Foreign(foreign));
return results;
}
}
}
@@ -0,0 +1,591 @@
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Media;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Globalization;
using System.Net;
using GroupEntity = PrivaPub.Models.Group.Group;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Mappers
{
public class MastodonMapper
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IMediaProxy _proxy;
public MastodonMapper(DbEntities dbEntities, ILocalActorService localActors, IMediaProxy proxy = default)
{
_dbEntities = dbEntities;
_localActors = localActors;
_proxy = proxy;
}
string Proxied(string url) => _proxy == default ? url : _proxy.Wrap(url);
public string ProxiedUrl(string url) => Proxied(url);
string MissingAvatar => $"{_localActors.BaseAddress}/media/missing-avatar.png";
string MissingHeader => $"{_localActors.BaseAddress}/media/missing-header.png";
public async Task<Account> Local(LocalActor actor, bool withSource, CancellationToken token)
{
var followers = await DB.Default.CountAsync<Follower>(f => f.LocalActorId == actor.Id && f.LocalActorKind == actor.Kind && f.IsAccepted, token);
var following = actor.Kind == LocalActorKind.Person
? await DB.Default.CountAsync<Following>(f => f.AvatarId == actor.Id && f.State == FollowState.Accepted, token)
: 0;
var latest = await _dbEntities.Posts
.Match(p => (actor.Kind == LocalActorKind.Group ? p.GroupId == actor.Id : p.GroupUserId == actor.Id) && !p.IsFederatedCopy
&& !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct && p.ReblogOfPostId == null)
.Sort(p => p.ID, Order.Descending)
.Limit(1)
.ExecuteFirstAsync(token);
var statuses = latest == default
? 0
: await DB.Default.CountAsync<PostEntity>(p => (actor.Kind == LocalActorKind.Group ? p.GroupId == actor.Id : p.GroupUserId == actor.Id)
&& !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct && p.ReblogOfPostId == null, token);
var account = new Account
{
Id = actor.Id,
Username = actor.UserName,
Acct = actor.UserName,
DisplayName = actor.Name ?? actor.UserName,
Locked = actor.ManuallyApprovesFollowers,
Bot = actor.IsBot,
Discoverable = actor.Discoverable,
Indexable = actor.Indexable,
Noindex = !actor.Indexable,
HideCollections = actor.Settings.HideCollections,
Group = actor.Kind == LocalActorKind.Group,
CreatedAt = MastodonJson.Day(actor.Published),
Note = ActivityPubRenderer.Html(actor.Summary),
Url = actor.HtmlUrl,
Uri = actor.Uri,
Avatar = actor.PictureURL ?? MissingAvatar,
AvatarStatic = actor.PictureURL ?? MissingAvatar,
Header = actor.ThumbnailURL ?? MissingHeader,
HeaderStatic = actor.ThumbnailURL ?? MissingHeader,
FollowersCount = (int)followers,
FollowingCount = (int)following,
StatusesCount = (int)statuses,
LastStatusAt = latest == default ? default : latest.CreationDate.ToString("yyyy-MM-dd"),
Fields = actor.Fields.Select(f => new Field { Name = f.Key, Value = ActivityPubRenderer.FieldValue(f.Value) }).ToList()
};
if (withSource)
account.Source = new Source
{
Privacy = actor.Settings.DefaultVisibility ?? "public",
Sensitive = actor.Settings.DefaultSensitive,
Language = actor.Settings.DefaultLanguage,
QuotePolicy = actor.Settings.QuotePolicy ?? QuotePolicies.Public,
Note = actor.Summary ?? string.Empty,
Fields = actor.Fields.Select(f => new Field { Name = f.Key, Value = f.Value }).ToList(),
FollowRequestsCount = (int)await DB.Default.CountAsync<Follower>(f => f.LocalActorId == actor.Id && !f.IsAccepted, token)
};
return account;
}
public Account Foreign(ForeignAvatar foreign) => new()
{
Id = foreign.ID,
Username = foreign.UserName,
Acct = $"{foreign.UserName}@{foreign.Domain}",
DisplayName = foreign.Name ?? foreign.UserName ?? string.Empty,
Bot = foreign.AvatarType is AvatarType.Service or AvatarType.Application,
Group = foreign.AvatarType == AvatarType.Group,
Locked = foreign.IsLocked,
Discoverable = foreign.IsDiscoverable,
Indexable = foreign.IsIndexable,
Memorial = foreign.IsMemorial ? true : default(bool?),
CreatedAt = MastodonJson.Day(foreign.Published ?? foreign.CreatedAt),
Emojis = Emojis(foreign.Emojis),
Fields = foreign.Fields.Select(f => new Field { Name = f.Key, Value = f.Value }).ToList(),
AvatarDescription = foreign.PictureDescription ?? string.Empty,
HeaderDescription = foreign.HeaderDescription ?? string.Empty,
Note = foreign.Biography ?? string.Empty,
Url = foreign.Url ?? foreign.ActorURI,
Uri = foreign.ActorURI,
Avatar = Proxied(foreign.PictureURL) ?? MissingAvatar,
AvatarStatic = Proxied(foreign.PictureURL) ?? MissingAvatar,
Header = Proxied(foreign.ThumbnailURL) ?? MissingHeader,
HeaderStatic = Proxied(foreign.ThumbnailURL) ?? MissingHeader
};
public async Task<Account> Account(string id, CancellationToken token) =>
(await Accounts(new[] { id }, token)).GetValueOrDefault(id);
public async Task<Dictionary<string, Account>> Accounts(IEnumerable<string> ids, CancellationToken token)
{
var wanted = ids.Where(id => !string.IsNullOrEmpty(id)).Distinct().ToList();
var accounts = new Dictionary<string, Account>();
if (wanted.Count == 0)
return accounts;
foreach (var avatar in await _dbEntities.Avatars.Match(a => wanted.Contains(a.ID) && !a.DeletionAt.HasValue).ExecuteAsync(token))
accounts[avatar.ID] = await Local(_localActors.FromAvatar(avatar), false, token);
foreach (var group in await _dbEntities.Groups.Match(g => wanted.Contains(g.ID) && !g.DeletionAt.HasValue && g.Kind == GroupKind.Community).ExecuteAsync(token))
accounts[group.ID] = await Local(_localActors.FromGroup(group), false, token);
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => wanted.Contains(f.ID)).ExecuteAsync(token))
accounts[foreign.ID] = Foreign(foreign);
return accounts;
}
public async Task<Status> Status(PostEntity post, string viewerId, CancellationToken token) =>
(await Statuses(new[] { post }, viewerId, token)).FirstOrDefault();
public Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, CancellationToken token) =>
Statuses(posts, viewerId, nested: false, token);
async Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, bool nested, CancellationToken token)
{
var originalIds = posts.Where(p => p.ReblogOfPostId != default).Select(p => p.ReblogOfPostId).Distinct().ToList();
var originals = originalIds.Count == 0
? new Dictionary<string, PostEntity>()
: (await _dbEntities.Posts.Match(p => originalIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)).ToDictionary(p => p.ID);
var all = posts.Concat(originals.Values).ToList();
var mentionUris = all.SelectMany(p => p.Mentions).Where(m => !m.IsLocal).Select(m => m.ActorURI).Distinct().ToList();
var mentionAccounts = mentionUris.Count == 0
? new Dictionary<string, ForeignAvatar>()
: (await _dbEntities.ForeignAvatars.Match(f => mentionUris.Contains(f.ActorURI)).ExecuteAsync(token)).ToDictionary(f => f.ActorURI);
var accounts = await Accounts(all.Select(AuthorOf), token);
var ids = all.Select(p => p.ID).ToList();
var favourited = viewerId == default
? new HashSet<string>()
: (await _dbEntities.Favourites.Match(f => f.AccountId == viewerId && ids.Contains(f.PostId)).ExecuteAsync(token)).Select(f => f.PostId).ToHashSet();
var bookmarked = viewerId == default
? new HashSet<string>()
: (await DB.Default.Find<Bookmark>().Match(b => b.AvatarId == viewerId && ids.Contains(b.PostId)).ExecuteAsync(token)).Select(b => b.PostId).ToHashSet();
var pinned = (await DB.Default.Find<Pin>().Match(p => ids.Contains(p.PostId)).ExecuteAsync(token)).Select(p => p.PostId).ToHashSet();
var reactions = (await DB.Default.Find<Reaction>().Match(r => ids.Contains(r.PostId)).ExecuteAsync(token))
.GroupBy(r => r.PostId)
.ToDictionary(g => g.Key, g => g.GroupBy(r => r.Emoji)
.Select(e => new EmojiReactionEntity
{
Name = e.Key,
Count = e.Count(),
Me = viewerId != default && e.Any(r => r.AccountId == viewerId),
Url = Proxied(e.First().EmojiURL),
StaticUrl = Proxied(e.First().EmojiURL)
})
.OrderByDescending(e => e.Count)
.ToList());
var pollIds = all.Where(p => p.Poll != default).Select(p => p.ID).ToList();
var ownVotes = viewerId == default || pollIds.Count == 0
? new Dictionary<string, List<int>>()
: (await DB.Default.Find<PollVote>().Match(v => pollIds.Contains(v.PostId) && v.VoterAccountId == viewerId && v.IsLocalVoter).ExecuteAsync(token))
.GroupBy(v => v.PostId).ToDictionary(g => g.Key, g => g.Select(v => v.Choice).ToList());
var hidden = viewerId == default
? new HashSet<string>()
: await Domain.Relationships.Hidden.AuthorsHiddenFrom(viewerId, all.Select(p => p.ActorURI), forNotifications: false, token);
var reblogged = viewerId == default
? new HashSet<string>()
: (await _dbEntities.Posts.Match(p => p.AuthorAccountId == viewerId && ids.Contains(p.ReblogOfPostId) && !p.DeletedAt.HasValue).ExecuteAsync(token))
.Select(p => p.ReblogOfPostId).ToHashSet();
Status Map(PostEntity post)
{
if (!accounts.TryGetValue(AuthorOf(post), out var account))
return default;
var status = new Status
{
Id = post.ID,
Uri = post.ObjectURI,
Url = post.Url ?? post.ObjectURI,
Account = account,
InReplyToId = post.AnsweringToPostId,
InReplyToAccountId = post.InReplyToAccountId,
Content = Content(post),
CreatedAt = MastodonJson.Time(post.CreationDate),
EditedAt = post.EditedAt.HasValue ? MastodonJson.Time(post.EditedAt.Value) : default,
RepliesCount = post.RepliesCount,
ReblogsCount = post.ReblogsCount,
FavouritesCount = post.FavouritesCount,
Favourited = favourited.Contains(post.ID),
Reblogged = reblogged.Contains(post.ID),
Bookmarked = bookmarked.Contains(post.ID),
Pinned = pinned.Contains(post.ID),
Sensitive = post.HasContentWarning,
SpoilerText = post.SpoilerText ?? (post.HasContentWarning ? post.Title ?? ActivityPubRenderer.ContentWarning : string.Empty),
Visibility = Visibility(post.Visibility),
Language = post.Language,
MediaAttachments = post.Media.Count > 0 ? post.Media.Select(Media).ToList() : Playable(post),
Card = Card(post),
Poll = Poll(post, viewerId, ownVotes.GetValueOrDefault(post.ID)),
Emojis = Emojis(post.Emojis),
Privapub = Extension(post),
EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new(),
Pleroma = new PleromaStatus { EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new() },
Mentions = post.Mentions.Select(m => Mention(m, mentionAccounts)).Where(m => m != default).ToList(),
Tags = post.Tags.Select(t => new StatusTag { Name = t, Url = ActivityPubRenderer.TagUrl(_localActors.BaseAddress, t) }).ToList()
};
return status;
}
var localAuthors = all.Where(p => !p.IsFederatedCopy && p.GroupUserId != default).Select(p => p.GroupUserId).Distinct().ToList();
var followsAuthor = viewerId == default || localAuthors.Count == 0
? new HashSet<string>()
: await FollowedLocalAuthors(viewerId, localAuthors, token);
var quotedIds = nested ? new List<string>() : all.Where(p => p.QuoteState == QuoteState.Accepted && p.QuotedPostId != default).Select(p => p.QuotedPostId).Distinct().ToList();
var quotedPosts = new List<PostEntity>();
foreach (var quotedPost in quotedIds.Count == 0 ? new List<PostEntity>() : await _dbEntities.Posts.Match(p => quotedIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token))
if (quotedPost.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quotedPost, viewerId, token))
quotedPosts.Add(quotedPost);
var quotedStatuses = quotedPosts.Count == 0
? new Dictionary<string, Status>()
: (await Statuses(quotedPosts, viewerId, nested: true, token)).ToDictionary(q => q.Id);
void Quote(Status status, PostEntity post)
{
status.QuotesCount = post.QuotesCount;
status.QuoteApproval = post.IsFederatedCopy ? Approval(post, viewerId) : LocalApproval(post, viewerId, followsAuthor.Contains(post.GroupUserId));
if (post.QuoteState == QuoteState.None)
return;
var state = post.QuoteState.ToString().ToLowerInvariant();
var quoted = post.QuotedPostId != default && quotedStatuses.TryGetValue(post.QuotedPostId, out var shown) ? shown : default;
status.Quote = nested
? new QuoteEntity { State = state, QuotedStatusId = post.QuotedPostId }
: new QuoteEntity { State = state, QuotedStatus = post.QuoteState == QuoteState.Accepted ? quoted : default, QuotedStatusId = post.QuotedPostId };
if (!nested && post.QuoteState == QuoteState.Accepted && quoted != default)
status.Content = WithoutQuoteFallback(status.Content);
}
var mapped = new List<Status>();
foreach (var post in posts)
{
if (hidden.Contains(post.ActorURI) || post.ReblogOfPostId != default && originals.TryGetValue(post.ReblogOfPostId, out var hiddenOriginal)
&& hidden.Contains(hiddenOriginal.ActorURI))
continue;
var status = Map(post);
if (status == default)
continue;
Quote(status, post);
if (post.ReblogOfPostId != default)
{
if (!originals.TryGetValue(post.ReblogOfPostId, out var original) || Map(original) is not { } inner)
continue;
Quote(inner, original);
status.Reblog = inner;
status.Content = string.Empty;
status.Reblogged = reblogged.Contains(original.ID);
}
mapped.Add(status);
}
return mapped;
}
public static string AuthorOf(PostEntity post) => post.AuthorAccountId ?? post.GroupUserId;
static readonly AngleSharp.Html.Parser.HtmlParser Fragments = new();
public static string WithoutQuoteFallback(string html)
{
if (string.IsNullOrEmpty(html) || !html.Contains("quote-inline", StringComparison.Ordinal))
return html;
var document = Fragments.ParseDocument($"<body>{html}</body>");
foreach (var fallback in document.QuerySelectorAll(".quote-inline").ToList())
fallback.Remove();
return document.Body!.InnerHtml;
}
async Task<HashSet<string>> FollowedLocalAuthors(string viewerId, List<string> authorIds, CancellationToken token)
{
var uris = new Dictionary<string, string>();
foreach (var avatar in await _dbEntities.Avatars.Match(a => authorIds.Contains(a.ID)).ExecuteAsync(token))
uris[_localActors.FromAvatar(avatar).Uri] = avatar.ID;
var targets = uris.Keys.ToList();
return (await _dbEntities.Followings.Match(f => f.AvatarId == viewerId && targets.Contains(f.TargetActorURI) && f.State == FollowState.Accepted).ExecuteAsync(token))
.Select(f => uris[f.TargetActorURI]).ToHashSet();
}
static QuoteApprovalEntity LocalApproval(PostEntity post, string viewerId, bool viewerFollows)
{
var policy = ActivityPubRenderer.QuotableBy(post);
var everyone = policy switch { QuotePolicies.Public => new List<string> { "public" }, QuotePolicies.Followers => new List<string> { "followers" }, _ => new List<string>() };
var current = viewerId == default ? "unknown"
: viewerId == post.GroupUserId && post.Visibility is PostVisibility.Public or PostVisibility.Unlisted ? "automatic"
: policy == QuotePolicies.Public || policy == QuotePolicies.Followers && viewerFollows ? "automatic"
: "denied";
return new QuoteApprovalEntity { Automatic = everyone, CurrentUser = current };
}
static QuoteApprovalEntity Approval(PostEntity post, string viewerId)
{
if (!post.IsFederatedCopy || post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
return new QuoteApprovalEntity();
if (post.QuotePolicy is not { } policy)
return new QuoteApprovalEntity { Automatic = new List<string> { "public" }, CurrentUser = viewerId == default ? "unknown" : "automatic" };
static List<string> Named(IEnumerable<string> who) => who.Select(w => Addressing.IsPublic(w) ? "public" : w.EndsWith("/followers") ? "followers" : "unsupported_policy").Distinct().ToList();
return new QuoteApprovalEntity
{
Automatic = Named(policy.Automatic),
Manual = Named(policy.Manual),
CurrentUser = viewerId == default ? "unknown" : policy.Automatic.Any(Addressing.IsPublic) ? "automatic" : policy.Manual.Any(Addressing.IsPublic) ? "manual" : "denied"
};
}
public static string Content(PostEntity post)
{
var content = post.ContentHtml ?? ActivityPubRenderer.Html(post.Text);
if (post.IsFederatedCopy && string.IsNullOrWhiteSpace(content) && !string.IsNullOrEmpty(post.Excerpt))
content = $"<p>{WebUtility.HtmlEncode(post.Excerpt)}</p>";
if (post.Event is { } happening && EventLine(happening) is { } line)
content = $"<p>{WebUtility.HtmlEncode(line)}</p>{content}";
var titled = !string.IsNullOrEmpty(post.Title) && (!post.IsFederatedCopy || post.ObjectType is not (null or "Note" or "Question"));
return titled
? $"<p><strong>{WebUtility.HtmlEncode(post.Title)}</strong></p>{content}"
: content ?? string.Empty;
}
public static string EventLine(EventDetails happening)
{
if (happening.Start is not { } start)
return default;
var zone = Zone(happening.FloatingTime ? default : happening.TimeZone);
var local = zone == default ? start : TimeZoneInfo.ConvertTimeFromUtc(start, zone);
var when = local.ToString("yyyy-MM-dd HH:mm", CultureInfo.InvariantCulture);
if (happening.End is { } end && end > start)
{
var localEnd = zone == default ? end : TimeZoneInfo.ConvertTimeFromUtc(end, zone);
when += " – " + localEnd.ToString(localEnd.Date == local.Date ? "HH:mm" : "yyyy-MM-dd HH:mm", CultureInfo.InvariantCulture);
}
when += happening.FloatingTime ? " (local time)" : zone == default ? " UTC" : $" ({happening.TimeZone})";
var where = happening.Places.Select(p => p.Name ?? p.Address).FirstOrDefault(p => p != default)
?? (happening.OnlineURL != default ? "online" : default);
return where == default ? when : $"{when} · {where}";
}
static TimeZoneInfo Zone(string id)
{
if (string.IsNullOrWhiteSpace(id))
return default;
try
{
return TimeZoneInfo.FindSystemTimeZoneById(id);
}
catch (Exception ex) when (ex is TimeZoneNotFoundException or InvalidTimeZoneException)
{
return default;
}
}
public PollEntity Poll(PostEntity post, string viewerId, IReadOnlyCollection<int> ownVotes)
{
if (post.Poll is not { } poll)
return default;
var expired = PollService.Ended(poll);
var hidden = poll.HideTotals && !expired && post.GroupUserId != viewerId;
return new PollEntity
{
Id = post.ID,
ExpiresAt = poll.ExpiresAt is { } ends ? MastodonJson.Time(ends) : default,
Expired = expired,
Multiple = poll.Multiple,
VotesCount = poll.Options.Sum(o => o.Votes),
VotersCount = poll.Multiple ? poll.VotersCount : poll.VotersCount ?? poll.Options.Sum(o => o.Votes),
Voted = viewerId == default ? default(bool?) : ownVotes?.Count > 0 || post.GroupUserId == viewerId,
OwnVotes = viewerId == default ? default : ownVotes?.ToList() ?? new List<int>(),
Options = poll.Options.Select(o => new PollOptionEntity { Title = o.Title, VotesCount = hidden ? default(int?) : o.Votes }).ToList(),
Emojis = Emojis(post.Emojis.Where(e => poll.Options.Any(o => o.Title.Contains($":{e.Shortcode}:"))))
};
}
List<CustomEmojiEntity> Emojis(IEnumerable<CustomEmoji> emojis) =>
emojis?.Select(e => new CustomEmojiEntity { Shortcode = e.Shortcode, Url = Proxied(e.URL), StaticUrl = Proxied(e.URL) }).ToList() ?? new();
PrivaPubStatus Extension(PostEntity post) => new()
{
ObjectType = post.ObjectType,
Title = post.IsFederatedCopy ? post.Title : default,
Excerpt = post.Excerpt,
Cover = Proxied(post.CoverURL),
Source = post.Source == default ? default : new PrivaPubSource { Content = post.Source.Content, MediaType = post.Source.MediaType },
Link = post.Link == default ? default : new PrivaPubLink
{
Url = post.Link.Href, Title = post.Link.Title, Description = post.Link.Description, Image = Proxied(post.Link.ImageURL), FromPublisher = post.Link.FromPublisher
},
Video = post.Video == default ? default : new PrivaPubVideo
{
Duration = post.Video.DurationSeconds,
Variants = post.Video.Variants.Select(Variant).ToList(),
IsLive = post.Video.IsLive,
State = post.Video.State,
Views = post.Video.Views,
CommentsPolicy = post.Video.CommentsPolicy,
DownloadEnabled = post.Video.DownloadEnabled,
Licence = post.Video.Licence,
Category = post.Video.Category,
Channel = post.Video.ChannelURI,
Captions = post.Video.Captions.Select(c => new PrivaPubCaption { Language = c.Language, Url = Proxied(c.Url) }).ToList()
},
Audio = post.Audio == default ? default : new PrivaPubAudio
{
Duration = post.Audio.DurationSeconds,
Variants = post.Audio.Variants.Select(Variant).ToList(),
Album = post.Audio.Album,
Position = post.Audio.Position,
Licence = post.Audio.Licence
},
Event = post.Event == default ? default : new PrivaPubEvent
{
Start = post.Event.Start is { } start ? MastodonJson.Time(start) : default,
End = post.Event.End is { } end ? MastodonJson.Time(end) : default,
TimeZone = post.Event.TimeZone,
Floating = post.Event.FloatingTime,
Places = post.Event.Places.Select(p => new PrivaPubPlace { Name = p.Name, Address = p.Address, Latitude = p.Latitude, Longitude = p.Longitude, Url = p.Url }).ToList(),
OnlineUrl = post.Event.OnlineURL,
JoinMode = post.Event.JoinMode,
Participants = post.Event.ParticipantCount,
Capacity = post.Event.Capacity,
Status = post.Event.Status,
Category = post.Event.Category
},
Votes = new PrivaPubVotes { Up = post.FavouritesCount, Down = post.DownvotesCount }
};
PrivaPubVariant Variant(MediaVariant variant) => new()
{
Url = Proxied(variant.Url),
MediaType = variant.MediaType,
Width = variant.Width,
Height = variant.Height,
Fps = variant.Fps,
Size = variant.Size,
Streams = variant.Streams
};
List<MediaAttachment> Playable(PostEntity post)
{
var (variant, kind, duration) = post.Video is { } video ? (BestVideo(video.Variants), "video", video.DurationSeconds)
: post.Audio is { } audio ? (audio.Variants.FirstOrDefault(), "audio", audio.DurationSeconds)
: default;
if (variant == default)
return new List<MediaAttachment>();
return new List<MediaAttachment>
{
Media(new PostMedia
{
Id = new Guid(System.Security.Cryptography.MD5.HashData(System.Text.Encoding.UTF8.GetBytes(variant.Url))),
RemoteURL = variant.Url,
ContentType = variant.MediaType ?? kind + "/*",
RemotePreviewURL = post.CoverURL,
Width = variant.Width,
Height = variant.Height,
DurationSeconds = duration,
Description = post.Title
})
};
}
public static MediaVariant BestVideo(IReadOnlyCollection<MediaVariant> variants)
{
var playable = variants.Where(v => v.MediaType?.StartsWith("video/") == true && (v.Streams.Count == 0 || v.Streams.Contains("audio") && v.Streams.Contains("video"))).ToList();
return playable.Where(v => v.Height <= 720).OrderByDescending(v => v.Height).FirstOrDefault()
?? playable.OrderBy(v => v.Height ?? int.MaxValue).FirstOrDefault();
}
PreviewCard Card(PostEntity post)
{
if (post.Link is { } link)
return Card(link.Href, link.Title ?? post.Title, link.Description ?? (post.Media.Count == 0 ? post.Excerpt : default), link.ImageURL);
if (post.IsFederatedCopy && post.ObjectType is "Article" or "Event" && post.Url != default && post.Media.Count == 0
|| post.IsFederatedCopy && post.ObjectType is "Video" or "Audio" && post.Url != default && post.Media.Count == 0 && Playable(post).Count == 0)
return Card(post.Url, post.Title, post.Excerpt, post.CoverURL);
return default;
}
PreviewCard Card(string url, string title, string description, string image)
{
if (!Uri.TryCreate(url, UriKind.Absolute, out var uri))
return default;
return new PreviewCard
{
Url = url,
Title = title ?? uri.Host,
Description = description ?? string.Empty,
ProviderName = uri.Host,
ProviderUrl = uri.GetLeftPart(UriPartial.Authority),
Image = Proxied(image)
};
}
public static string Visibility(PostVisibility visibility) => visibility switch
{
PostVisibility.Unlisted => "unlisted",
PostVisibility.FollowersOnly or PostVisibility.Circle => "private",
PostVisibility.Direct => "direct",
_ => "public"
};
StatusMention Mention(PostMention mention, IReadOnlyDictionary<string, ForeignAvatar> foreign)
{
if (mention.IsLocal)
{
var handle = mention.Handle?.TrimStart('@').Split('@')[0];
return new StatusMention { Id = mention.AccountId, Username = handle, Acct = handle, Url = mention.ActorURI };
}
if (!foreign.TryGetValue(mention.ActorURI, out var account))
return default;
return new StatusMention { Id = account.ID, Username = account.UserName, Acct = $"{account.UserName}@{account.Domain}", Url = account.Url ?? account.ActorURI };
}
static Dictionary<string, object> Meta(PostMedia media)
{
var original = new Dictionary<string, object>();
if (media.Width > 0 && media.Height > 0)
{
original["width"] = media.Width;
original["height"] = media.Height;
original["size"] = $"{media.Width}x{media.Height}";
original["aspect"] = (double)media.Width.Value / media.Height.Value;
}
if (media.DurationSeconds is { } seconds)
original["duration"] = seconds;
var meta = new Dictionary<string, object>();
if (original.Count > 0)
meta["original"] = original;
if (media.DurationSeconds is { } length)
meta["length"] = TimeSpan.FromSeconds(length).ToString(@"h\:mm\:ss\.ff", CultureInfo.InvariantCulture);
if (media.Focus is { Length: 2 })
meta["focus"] = new { x = media.Focus[0], y = media.Focus[1] };
return meta;
}
MediaAttachment Media(PostMedia media) => new()
{
Id = media.AttachmentId ?? media.Id.ToString("N"),
Type = media.ContentType switch
{
{ } type when type.StartsWith("image/gif") => "gifv",
{ } type when type.StartsWith("image/") => "image",
{ } type when type.StartsWith("video/") => "video",
{ } type when type.StartsWith("audio/") => "audio",
_ => "unknown"
},
Url = media.URL ?? Proxied(media.RemoteURL),
PreviewUrl = media.PreviewURL ?? Proxied(media.RemotePreviewURL)
?? (media.ContentType?.StartsWith("image/") == true || media.ContentType == default ? media.URL ?? Proxied(media.RemoteURL) : default),
RemoteUrl = media.RemoteURL,
Description = media.Description,
Blurhash = media.Blurhash,
Meta = Meta(media)
};
}
}
@@ -0,0 +1,74 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Localization;
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.ClientModels.Admin;
using PrivaPub.Federation.Moderation;
using PrivaPub.Models.Federation;
using PrivaPub.Resources;
namespace PrivaPub.Controllers.ClientToServer
{
[ApiController,
Route("clientapi/admin/domainblocks"),
Authorize(Policy = Policies.IsAdmin)]
public class DomainBlockController : ControllerBase
{
readonly IDomainBlocks _domainBlocks;
readonly IStringLocalizer _localizer;
public DomainBlockController(IDomainBlocks domainBlocks, IStringLocalizer<GenericRes> localizer)
{
_domainBlocks = domainBlocks;
_localizer = localizer;
}
[HttpGet, Route("/clientapi/admin/domainblocks/list")]
public async Task<IActionResult> List(CancellationToken token) =>
Ok((await DB.Default.Find<DomainBlock>().Sort(b => b.Domain, Order.Ascending).ExecuteAsync(token)).Select(ToView).ToList());
[HttpPost, Route("/clientapi/admin/domainblocks/insert")]
public async Task<IActionResult> Insert(DomainBlockForm form, CancellationToken token)
{
var domain = DomainBlocks.Normalise(form.Domain);
if (!ModelState.IsValid || Uri.CheckHostName(domain) != UriHostNameType.Dns)
return BadRequest(new WebResult().Invalidate(_localizer["Invalid model."]));
var block = await DB.Default.UpdateAndGet<DomainBlock>()
.Match(b => b.Domain == domain)
.Modify(b => b.Domain, domain)
.Modify(b => b.Severity, form.Suspend ? DomainBlockSeverity.Suspend : DomainBlockSeverity.Silence)
.Modify(b => b.RejectMedia, form.RejectMedia)
.Modify(b => b.PublicComment, form.PublicComment)
.Modify(b => b.PrivateComment, form.PrivateComment)
.Modify(b => b.SetOnInsert(x => x.CreatedAt, DateTime.UtcNow))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
await _domainBlocks.Reload(token);
return Ok(ToView(block));
}
[HttpPost, Route("/clientapi/admin/domainblocks/delete")]
public async Task<IActionResult> Delete([FromQuery] string domain, CancellationToken token)
{
domain = DomainBlocks.Normalise(domain);
await DB.Default.DeleteAsync<DomainBlock>(b => b.Domain == domain);
await _domainBlocks.Reload(token);
return Ok();
}
static ViewDomainBlock ToView(DomainBlock block) => new()
{
Id = block.ID,
Domain = block.Domain,
Severity = block.Severity.ToString(),
RejectMedia = block.RejectMedia,
PublicComment = block.PublicComment,
PrivateComment = block.PrivateComment,
CreatedAt = block.CreatedAt
};
}
}
@@ -1,10 +1,47 @@
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.Extensions;
using PrivaPub.Models.Social;
namespace PrivaPub.Controllers.ClientToServer
{
[ApiController,
Route("clientapi/moderator")]
Route("clientapi/moderator"),
Authorize(Policy = Policies.IsModerator)]
public class ModeratorController : ControllerBase
{
[HttpGet, Route("/clientapi/moderator/reports")]
public async Task<IActionResult> Reports([FromQuery] bool resolved, CancellationToken token) =>
Ok((await DB.Default.Find<Report>().Match(r => r.IsResolved == resolved).Sort(r => r.ID, Order.Descending).Limit(100).ExecuteAsync(token))
.Select(r => new
{
r.ID,
Reporter = r.ReporterAvatarId != default ? "local" : r.ReporterActorURI,
r.TargetAccountId,
r.TargetActorURI,
r.PostIds,
r.ObjectURIs,
r.Category,
r.Comment,
r.Forwarded,
r.IsResolved,
r.CreatedAt,
r.ResolvedAt
}));
[HttpPost, Route("/clientapi/moderator/reports/{id}/resolve")]
public async Task<IActionResult> Resolve(string id, CancellationToken token)
{
var result = await DB.Default.Update<Report>().MatchID(id)
.Modify(r => r.IsResolved, true)
.Modify(r => r.ResolvedAt, DateTime.UtcNow)
.Modify(r => r.ResolvedBy, User.GetUserId())
.ExecuteAsync(token);
return result.MatchedCount == 0 ? NotFound() : Ok();
}
}
}
Loaded 100 of 240 files, more files were not shown because too many files have changed in this diff. Show more