Compare commits

...
168 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 4eda3da75d mongod's oplog size flag is --oplogSize on the command line
Build / Build (push) Successful in 8m26s
Deploy / privapub.thepra.dev (push) Successful in 8m53s
--oplogSizeMB is the configuration file's key (replication.oplogSizeMB); mongod refuses it as a flag, so setup.sh's
replica-set step left the unit restarting until the flag was fixed on Max.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 17:32:02 +02:00
thepraandClaude Opus 5.5 163f7e4e2b An import reads the archive's followers once, finds threads in linear time, and imports a located post once
A post's visibility no longer parses actor.json again for every outbox item; a reply's conversation is its thread's
first post's, found from what the first pass read instead of searching every planned post (quadratic on a large
archive); and a located post always records where it came from, so importing a persona's own archive twice made it
twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 12:51:58 +02:00
thepraandClaude Opus 5.5 44a5895b0c ROADMAP: the owner's decisions on media, backups and archives, and what the next deploy needs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 12:50:17 +02:00
thepraandClaude Opus 5.5 3d7d406834 A persona's archive: exported in Mastodon's layout, imported without telling anyone
A root exports one of its personas (a job) as Mastodon's account archive, so other servers' importers read it: the
actor with its public key only, its own posts and boosts with their media, likes, bookmarks and Mastodon's CSV files,
plus PrivaPub's filters, followed hashtags, notification policy, pins, scheduled and located posts; nothing of its root,
its siblings, its keys or anyone's token. A ticket link downloads it, for a week.

An archive (PrivaPub's or Mastodon's) uploaded in pieces is imported into a persona in a job, the parts the root picks,
with progress and a stop. SafeArchive refuses links, escaping paths, duplicates, bombs and oversized items, and reads the
outbox one item at a time. Imported posts are delivered to no one, put in no home and notify nobody, yet show on the
profile, outbox, hashtags and search; back home a post keeps its id, from another actor it gets one of its date and
ImportedFromURI, so importing twice changes nothing. Relationships go through the existing services; located, scheduled
and likes only when asked; followers never.

tools/pasture/scenarios/persona-archive.sh imports mastouser's real Mastodon archive (156 posts, 24 pictures) into a
persona Mastodon follows: Mastodon receives none of it, and a second import changes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 12:43:12 +02:00
thepraandClaude Opus 5.5 bdc8be4508 A restore on the live pasture: its own scenario, and two fixes it found
tools/pasture/scenarios/restore.sh backs the pasture up from the administrator's endpoint, then alice_restore deletes a
post and makes another, mastouser follows her, she blocks bob_restore and carol_restore is made; the restore, asked for
from the endpoint, keeps every protective act (19 checks). town.sh renew <peer> signs a peer's town accounts in again,
since a restore ends every session.

It found that a backup listed media files already missing when it was made, so verifying it failed and the restore was
refused: a manifest now lists only the files it holds (refusals are cut to five lines). And a local post made after the
backup now comes back as a deleted row, as a deletion leaves it, so it answers 410 and its id is never given again;
DeletedObject holds remote tombstones only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 12:24:23 +02:00
thepraandClaude Opus 5.5 9ad96f560d FEP-8fcf rests for two weeks after a restore
What a restore lost of either side is not the other server's to mend: for RestoreRecord.FollowersGrace (14 days)
deliveries carry no Collection-Synchronization header, and a follow only the remote remembers is adopted rather than
undone. Without a restore nothing changes; the interop sweep (gts, mastodon, misskey, sharkey, akkoma, pixelfed,
smithereen, followsync, pins, moves) passed, Smithereen's poll check on a second run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 12:16:35 +02:00
thepraandClaude Opus 5.5 bc5f2beaf7 The administrator's page backs up, downloads, uploads and restores
/clientapi/admin/backups (owner decision 2026-10-07, approving these endpoints in production): the list with what runs,
the restore waiting and the last restore's report; back up now; delete; a download for the password, through a ticket
good for ten minutes in the link's path, as one tar whose length is known first and which honours Range (BackupTar);
an upload in pieces of at most 32 MB, each at the offset already received or refused with it, so a broken upload
resumes, read into a backup only when it holds nothing but plain files under one backup's folder; and a restore for
the password and the host typed out. Every call checks the administrator against the database. nginx streams downloads
for an hour and takes upload pieces unbuffered, rate-limited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 12:01:03 +02:00
thepraandClaude Opus 5.5 fba57318fa A backup is restored at boot, and never undoes a protective act
PrivaPub admin restore <id> (and soon the administrator's page) checks the backup (same host, a format and newest
migration this build reads, every hash) and writes restore.json; the running service sees it within seconds and stops,
and the next start restores it in MaintenanceGate, before migrations, indexes and hosted services: a pre-restore backup
taken once, every collection dropped and imported raw with its indexes, the media the live directory lacks brought
back, then the protective merge from the pre-restore backup. Followers and follows are the live ones; blocks, mutes,
domain blocks, reserved names, tombstones, reports, filters and OAuth applications are the union; deletions win;
accounts made since become tombstones and local posts made since answer 410; every session ends.

Each attempt redoes everything; one refused before any change is abandoned and recorded, one failed midway exits 1 for
systemd to retry, and after three it exits 75, which the unit no longer restarts. Commands wait (exit 75) while a
restore is pending. RestoreRecord tells what happened (admin restore --status).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:55:17 +02:00
thepraandClaude Opus 5.5 12bb75809f The server backs itself up: every collection byte for byte, the media linked
A backup is a directory <stamp>-<kind> under Backups:Root (/var/lib/privapub/backups, 2770, files 0640), written as
.partial and renamed once whole: a manifest (host, build, newest migration, each collection's count, size, sha256 and
indexes, what was left out and why, the media list), each collection as gzipped canonical Extended JSON read raw, and
hard links to the files of untrashed media rows (copies where a link can't be made). On a replica set every collection
is read in one snapshot session. Never in a backup: the statistics salt, jobs, recovery codes, sessions, the
maintenance lock and the configuration's copy with its SMTP password.

One backup or restore at a time (MaintenanceLock, a heartbeat document), and the janitor purges nothing meanwhile.
BackupScheduler backs up nightly at 03:30 UTC (or at once after missing a night); rotation keeps 7 daily, 4 weekly,
3 pre-deploy and 3 pre-restore backups. CLI: admin backup [--kind] [--db-only], admin backups, admin backup verify;
these run before migrations, so the deploy's own pre-deploy backup, which replaces mongodump, is of the database as the
live build left it. EntityMaps.Warm runs once under a lock, since test hosts now boot side by side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:40:06 +02:00
thepraandClaude Opus 5.5 37b12c5fee Mongo is a one-member replica set
Owner decision 2026-10-07: production's mongod becomes a one-member replica set (rs0), so that a backup can read every
collection at one instant (a snapshot read session), which a standalone mongod can't. The unit runs mongod with
--replSet rs0 and a 990 MB oplog; setup.sh converts it once, idempotently (PrivaPub stopped, mongod restarted,
rs.initiate, the primary awaited, PrivaPub started); every connection string says directConnection=true, which works
against the standalone too, so this code can deploy before the conversion. The CI's throwaway mongod and the pasture's
are replica sets as well, so the snapshot path is what the tests exercise; MongoTopology tells which a mongod is, and
TopologyTests holds the test mongod to it. The suite passes on it (906).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:21:13 +02:00
thepraandClaude Opus 5.5 830385ea2b An empty media_ids takes an edited post's media away
A JSON body's empty array flattened to nothing, so `media_ids: []` on an edit read as "media not sent" and the
post kept every attachment, where Mastodon removes them. An empty array is now a parameter with no values: the edit
leaves the post without media, and they go to the trash. MediaLifecycleTests checks it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:07:17 +02:00
thepraandClaude Opus 5.5 dc63fa57b8 A login's storage is counted, and can have a quota
MediaAttachment.Size was stored and never summed: nobody, the administrator included, could tell what media took,
and nothing bounded it. Counted.Media and MediaOfRoot sum what is kept (not trashed). A login sees what its personas'
uploads and pictures take at /clientapi/user/storage (ViewStorage), with its quota when the server sets one;
Media:QuotaBytesPerRoot (0, no quota, by default) refuses an upload or a picture over it with 422, whichever persona
sends it; the statistics overview gains the media totals, the proxy cache and the trash (ViewMediaTotals). Tests: a
login's storage counts what its personas keep and forgets what is trashed; two uploads from two personas of one login
are refused together past the quota.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:03:25 +02:00
thepraandClaude Opus 5.5 5252b8f320 A posted picture is described again by editing its post
PUT /api/v1/media/:id on a picture already posted changed the upload but never the post, which keeps its own copy, so
the new description or focal point reached nobody. As on Mastodon, PUT now takes only media not posted yet (404
otherwise), and a status edit takes media_attributes[] (id, description, focus; from a JSON body or a form): the
uploads and the post's copies change, and the Update the edit federates carries them. An edit that sends no media
keeps them as they were described. No pasture scenario edits a description, so the sweep could not see this.
MediaLifecycleTests: PUT on a posted picture is refused, an edit describes it again and keeps its warning, flag and
language as sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:00:04 +02:00
thepraandClaude Opus 5.5 a4ced664cf The proxy's shared download is started once
ConcurrentDictionary.GetOrAdd may run its factory more than once when asked at the same time, and each run started a
download: five clients asking at once for a new file made three upstream requests in one run of
Clients_asking_at_once_share_one_download. The factory now makes a Lazy, of which only the one kept is ever started,
and a download that begins just after another cached the file uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 11:00:04 +02:00
thepraandClaude Opus 5.5 8fecf7de2d A persona's avatar and header can be removed
Mastodon's DELETE /api/v1/profile/avatar and /api/v1/profile/header: the picture goes to the trash (its file stops
being served at once), the profile shows the default again, followers are sent the updated profile, and the answer is
the account with its source. The profile Update is the one a persona that never had a picture sends, so nothing new
leaves PrivaPub. MediaLifecycleTests: the avatar, then the header, removed and no longer served.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:55:12 +02:00
thepraandClaude Opus 5.5 3ca29603ed The media proxy is bounded
Anyone could mint signed proxy URLs (a remote account changes its icon, an anonymous lookup returns the URL), and each
anonymous request held up to 40 MB in memory; the cache grew without bound between hourly trims; two clients asking
for the same new file downloaded it twice and wrote over each other in place, so a reader could get half a file with a
7-day cache header; a file over the limit was downloaded twice on every request; a failed fetch, a 404, was cached by
browsers for a week; cached media of a server suspended later were still served, and RejectMedia skipped avatars,
emoji, covers, video variants, link cards and remote edits; /clientapi/group/members returned remote pictures raw.

Now a download is shared by everyone asking at once, streamed into a .part file and renamed into place
(FederationHttp.DownloadMedia copies bounded, never into memory), at most eight at a time; a file too big to cache is
remembered for an hour and only streamed, a failure for five minutes; the cache's size is counted as it grows and
trimmed as soon as it passes the cap; a cached file is opened before it is answered; browsers may cache only a
success; nothing of a suspended server, or of one whose media are rejected, is proxied (everything remote a client
sees goes through the proxy, so that covers every kind), and blocking one purges its cache; the proxy has its own rate
limit per client address; group members' pictures are proxied; the proxy's key is loaded once, the oldest if two
were made. This changes what PrivaPub serves its clients, not what it sends to other servers.

Tests: clients asking at once share one download, a failure isn't cached by browsers, an over-limit file is fetched
three times for two requests instead of four, a blocked server's media are refused and its cache purged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:54:33 +02:00
thepraandClaude Opus 5.5 8e59145826 Audio and video are processed off the request
Every audio and video upload was probed and remuxed inside the request, whatever its length; ffmpeg would read any
protocol and probe any format; `-map 0` kept the data tracks iPhones add, which mp4 refuses; nothing was ever
transcoded, so HEVC or MPEG-4 Part 2 reached browsers that can't play them, and the advertised video_matrix_limit
and frame rate limit were never applied; the output was read whole into memory, the video was saved before its
poster could fail, and the poster's frame leaked in /tmp. FLAC uploads were served as 404.

Now an upload sent to /api/v2/media is stored as sent in media-incoming (beside the media root, never served) and
answered with 202 and no url, while a ProcessMedia job, one at a time, does the work; GET /api/v1/media/:id answers 206
until it is ready, or 422 with why, and media still processing can't be posted. v1 processes before answering.
ffmpeg reads only that file (protocol whitelist, format forced from the probe) and drops data and subtitle tracks. A
video browsers play as it is (H.264, VP8, VP9, AV1 within Media:MaxVideoPixels and MaxFrameRate) is remuxed, anything
else transcoded to H.264 that fits, as Mastodon does; longer than Media:MaxSeconds is refused. Outputs move into place
only once everything succeeded, every temporary file goes, durations are kept, FLAC is served as audio/flac, and the
unit gets PrivateTmp. The instance API advertises the limits that are now applied.

No pasture scenario uploads audio or video through PrivaPub, so the sweep could not see this. MastodonMediaTests: v2
answers 202 then the job makes it playable (and an unreadable file 422 once processed), media still processing can't
be posted, MPEG-4 Part 2 becomes H.264, a video over the limit is made smaller, FLAC is served.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:48:56 +02:00
thepraandClaude Opus 5.5 e4f9d0b61e An image is checked before it is decoded
An upload went straight to libvips: whatever loader recognised the bytes ran (an SVG sent as image/png was rasterised),
nothing bounded how many pixels it would decode to (a small PNG could decode to gigabytes, three times over), a GIF
was loaded frame by frame and never resized, and all of it ran inside the request with nothing limiting how many at
once. A GIF was typed gifv but stayed a .gif, which a gifv player can't play; its metadata was kept; colours lost
their ICC profile without being converted; HEIC was advertised but the bundled libvips can't decode it.

Now:
- only libvips' JPEG, PNG, GIF, WebP and HEIF loaders ever run on an upload (every other loader is blocked);
- the header alone says how big an image would decode, refused above Media:MaxPixels (40 MP) or MaxFrames;
- a still image is shrunk on load, turned by its orientation and brought into sRGB (thumbnail), then written without
  metadata, a profile picture the same way;
- an animated GIF becomes a looping silent H.264 mp4 typed gifv, as on Mastodon (PostMedia.Kind keeps it a gifv),
  and a remote GIF is an image;
- processing runs Media:Concurrency at a time, and uploads have their own rate limit per credential;
- HEIC and HEIF are no longer offered.

Tests: only the upload formats load, the header tells the size, an SVG posing as a PNG and an image too large are
refused before decoding, an animated GIF becomes a gifv and a still one an image, HEIC isn't advertised. The media
scenarios against the pasture (GoToSocial, Mastodon, Misskey, Akkoma, Pixelfed, Smithereen, Vernissage, Castopod,
PeerTube) pass: 329 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:48:44 +02:00
thepraandClaude Opus 5.5 65938bb2a0 Media are reserved for one post, atomically
An upload's owner checked that it was free, then attached it in a second step, so two posts asking for the same
upload at once could both get it; a post now, or an edit, could take media a scheduled post held, which then failed
when its time came (only logged) and was deleted; and a persona deleted or banned after scheduling a post still
published it.

Now a post claims its media in one conditional update before anything is written (its id is minted first), and a
post that loses the race is refused with 422, the media it took put back. Scheduling reserves media the same way.
Media held by a scheduled post are that post's alone, and its job publishes only for a persona still there whose root
is neither deleted nor banned; otherwise what it held is trashed. MastodonScheduledStatusesTests: a post now and an
edit can't take scheduled media, two racing posts never both get an upload, a gone persona's scheduled post never
publishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:34:16 +02:00
thepraandClaude Opus 5.5 bb680e8cb8 A file lives exactly as long as something holds it
Deleting a post, editing media out, replacing an avatar or a header, and removing a whole root deleted no file: every
one stayed on disk and publicly served from /media/files with a year-long immutable cache, its row orphaned. Now every
upload is a row, profile pictures too (Kind avatar or header, ProfileOfAvatarId), and each of those acts trashes what
it held, as does an upload never posted for a day and a dropped scheduled post. A trashed row is marked in one
conditional update (an upload attached meanwhile is left alone), its files move into media-trash, beside the media
root and outside what /media/files serves, and the janitor deletes them a day later. Nothing is deleted for looking
unused.

Along the way: a profile picture that isn't an image, or can't be read, answers 422 instead of being silently ignored
with a 200; a removed root's scheduled posts are dropped, so nothing of it publishes later; media rows get indexes
(they had none), and the janitor's first pass comes five minutes after boot instead of an hour.

`PrivaPub admin media audit [--fix]` compares the disk with the database. With --fix (as www-data) it gives the
pictures personas show today a row, and trashes media of deleted posts or personas, rows whose files are missing, and
files nothing holds: the leftovers of every deletion until now. MediaLifecycleTests covers each act, that the trash is
never served, and the audit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:31:14 +02:00
thepraandClaude Opus 5.5 52d201eee9 A focal point is two finite numbers
float.TryParse takes "NaN" and "Infinity", and Math.Clamp keeps a NaN, so `focus=NaN,NaN` on an upload or a media PUT
was stored as given. Once the media was posted, every status and timeline response holding the post, and the Note
and its delivery, failed to serialise, for every viewer. FocalPoint.Parse takes finite numbers only (anything else is
ignored, as an unreadable focus was); the mapper and the renderer leave out a focus that isn't sound, and migration
_016 removes the ones stored before, from the uploads and from the copy each post keeps. What PrivaPub sends is
unchanged for any focus that could be sent before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:23:09 +02:00
thepraandClaude Opus 5.5 402f9e0d75 A long job keeps its lease, and runs once
A lease lasted two minutes and was never renewed, so the reaper gave any longer job to a second worker while the first
still ran it, and both finished it. The worker now renews the lease every third of its length while the handler runs;
a lease found taken (reaped and leased again) cancels the handler. Each lease carries its own owner stamp, since every
worker of a process shares one name, and Finish only counts for the lease it was given. Media processing and persona
archives will run longer than two minutes. JobQueueTests: a job three times its lease runs once with the reaper finding
nothing, and a stolen lease stops its handler and drops its outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:20:24 +02:00
thepraandClaude Opus 5.5 8492f24064 The deploy's dump leaves out the statistics salt
The pre-deploy mongodump kept InteractionSalt, the day's salt that the owner decided must be destroyed at each rollup:
a dump kept seven deploys long let the day's actor hashes be reversed. It is excluded now, and the deploy refuses a
dump that still names it (mongorestore's dry run lists every collection). The dumps already on the box still hold
old salts; removing them waits for the owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 10:16:35 +02:00
thepraandClaude Opus 5.5 6f240828cf A located post reaches no home, its author's included
Build / Build (push) Successful in 7m38s
Deploy / privapub.thepra.dev (push) Successful in 7m48s
The fan-out put every post in its author's home, located ones too, while the Mastodon API looks past located posts
everywhere else. In the author's home a located post showed as public (the mapper has no word for it) and every action
on it answered 404: decePub's end-to-end runs, which write one from Milano for the globe, found it there. The fan-out
now gives a located post no home, no stream and no notification, and the home timeline passes over the entries left
from before. A located post is read through /clientapi/post/nearby and deleted through /clientapi/post/delete.
NearbyTests checks it has no TimelineEntry; the suite passes (881).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-07 02:14:16 +02:00
thepraandClaude Opus 5.5 5ecf2d9629 The report tests read only the deliveries they queued
Build / Build (push) Successful in 7m8s
ServiceReportTests looked at every delivery job of the last five minutes and read each body's actor. On the CI runner,
slower than the workstation, other tests' jobs were still in that window, one of them without an actor, and four cases
failed with a NullReferenceException. A test now sees only the jobs queued since it began, and checks the signer from
the job itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 19:41:22 +02:00
thepraandClaude Opus 5.5 f105d1f7ea PieFed and Mbin take reports from the reporter too
Build / Build (push) Failing after 7m51s
Deploy / privapub.thepra.dev (push) Successful in 7m58s
Both speak Lemmy's shapes and joined the servers that get reports in that shape only once the pasture showed each keeps
one with its reason. By their source, PieFed dropped the instance actor's report (it makes a user only of a Person or a
Service) and Mbin kept it without the persona's words (it reads the reason from `summary` alone). With them in
`ServiceReportTakers`, each keeps the reports of a thread and of a comment from "Reports from privapub.test" with
alice's words, and none names her (piefed.sh and mbin.sh, 65 checks with the full sweep's 876). A report of a PieFed
account alone, which PieFed would take from the reporter, is not sent yet; INTEROP says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 19:24:09 +02:00
thepraandClaude Opus 5.5 f66c280b0b Reports reach Lemmy's moderators, from an anonymous reporter
Lemmy takes a report only from a person or a service, about one post or comment, addressed to its community, and it
answered PrivaPub's Flag (the instance actor's, an Application, with no `to` and the account and posts as its object)
400. A report of a post or comment in a community on a server whose NodeInfo names Lemmy now leaves from
`privapub_reports`, a Service with its own key that names nobody: one Flag per post, `to` the community (its own
audience, else its thread's), with the persona's words, or the category, in `summary` and `content`, sent to the
community's inbox. This is the second exception to "a server's software is for display" (owner decision 2026-10-06,
`ReportService.ServiceReportTakers`). Every other server keeps the instance actor's report. An account alone is not
reported to Lemmy, which takes no such report, and `forwarded` now says whether anything left.

The reporter is read unsigned in SecureMode and answers WebFinger like the instance actor. Nobody follows or mentions
it, the Mastodon API has no account for it, and a migration reserves its name. Checked live: Lemmy 1.0 and 0.19 keep the
reports of a thread and of a comment, with alice's words, from "Reports from privapub.test", and none names her (69
checks). A sweep of every scenario with this and the next commit: 876 checks pass; Ghost's Network feed listed alice's
post too late once, and Ghost passes alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 19:23:17 +02:00
thepraandClaude Opus 5.5 10ff4b3d2a The crawler scenario brings mastodon.test's visit forward; the backlog after a full sweep
The crawler visits one server a minute, every one PrivaPub knows, and as the pasture grew mastodon.test's turn came
after the scenario stopped waiting; the scenario now makes it due at once. The backlog is regenerated after a sweep of
every peer: 2581 checks pass, none fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 16:57:50 +02:00
thepraandClaude Opus 5.5 bebb4d6370 Pasture: WordPress pinned with its updates off, Mbin's worker restarting, threads checking our own deliveries
A full sweep found three peers silent. WordPress had updated itself to 7.1.2 from WP-Cron, and the new CA bundle dropped
Caddy's root: it now runs the 7.1.2 image with automatic updates off. Mbin's messenger worker had died when the shared
Postgres restarted under it: it now consumes again whenever it stops. Friendica sat in a quarantine left by the old
breaker. The threads scenario checks that PrivaPub never sends carol's reply to Mastodon, since a relay Mastodon has just
left (the relay scenario's) may still pass it on. With these, every scenario passes but the crawler's, which needs it
switched on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 16:29:21 +02:00
thepraandClaude Opus 5.5 cb34ab88f7 A deleted post's id given to a new post is that new post
PrivaPub remembers a deleted remote post's id for 90 days so that a late Create cannot bring it back. Gancio numbers a
new event after the last one it keeps, so deleting its last event gives the next the same id, and that event was
dropped as deleted. A Create published after the deletion is now kept as the new post; the deleted one's own Create,
however late, still is not. Checked live: Gancio's scenario passes again (17 checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 16:29:21 +02:00
thepraandClaude Opus 5.5 af2c61292b A community announces back to its author's own server
Lemmy keeps its user's post in a community pending until the community announces it back. A community hosted here
announced only to its followers, so a post from a server where nobody follows the community stayed pending there; the
author's own server (its shared inbox) now gets the announces too. INTEROP records, confirmed live, why Lemmy refuses
our Flags (an Application reporter, no `to`, an array object). FEDERATION.md's custom emoji line is put back before the
replies paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 16:29:04 +02:00
thepraandClaude Opus 5.5 a3a66b6db2 Replies a post's author approves (FEP-5624), checked live with PeerTube
PeerTube puts canReply on a video whose comments wait for approval, and answers each comment with ApproveReply. A
persona's reply to such a post now waits (privapub.approval: pending), its Create going to the author alone; the
author's ApproveReply, signed by the author and naming the post answered, lets it out to its audience with
replyApproval, and RejectReply leaves it ours. A null canReply (PeerTube's open comments) says nothing; an empty one
refuses. The PeerTube scenario holds a comment for review and approves it through PeerTube's API (28 checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 16:16:34 +02:00
thepraandClaude Opus 5.5 5d5f1d6909 A persona's notification policy and its requests
Mastodon 4.3's policy was a stub that accepted everything. It is now kept per persona: notifications from accounts it
does not follow, accounts that do not follow it (or only for three days), accounts newer than 30 days, private mentions
it did not ask for and silenced accounts are accepted, filtered or dropped. Filtered ones stay out of every list and
count unless asked for, gathered in a request per account; letting a request in lets that account in for good,
dismissing it deletes them. Everything is accepted until the persona chooses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 15:48:33 +02:00
thepraandClaude Opus 5.5 9872b78677 Ktistec in the pasture
Ktistec 3.13.0 (peers/ktistec.sh): built from its tag (images/ktistec, its own Dockerfile pinned), set up through its
API. scenarios/ktistec.sh, 27 checks, twice in a row: follows, posts, replies, likes and boosts with their undos both
ways, its poll and alice's vote, FEP-044f quotes both ways, edits and deletions both ways, the unfollow, statistics. It
is driven through the part of the Mastodon API it speaks and its own outbox API, and read from a copy of its SQLite
database, since its API counts no likes or boosts. CLAUDE.md also asks that a change to what PrivaPub sends be run
against every peer before it is committed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 14:09:29 +02:00
thepraandClaude Opus 5.5 b478049303 Our @context stays JSON-LD 1.0, and a 500 no longer takes a host out
The proof term added with integrity proofs had a @graph container, which only JSON-LD 1.1 knows: Smithereen's reader
refused every document carrying our context with a 500. Those 500s, retried, then put Smithereen's host in quarantine
for an hour, and every delivery to it waited. Proofs are canonicalised as JSON (JCS), so the term loses nothing as a
plain id; a test keeps every term within 1.0. Only an unreachable host, a timeout or a gateway's 502, 503 or 504 now
counts against a host: a 500 is the server failing on that one activity, retried on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 13:02:02 +02:00
thepraandClaude Opus 5.5 b66f0acf4e The Mastodon scenario waits on Mastodon's database, and tells the quoted post by its id
Mastodon's relationships API keeps each pair a day, and a read racing the change awaited writes the old answer back
after the change cleared it: an unfollow that had landed showed as a follow until the block. Changes from PrivaPub are
now awaited in Mastodon's database. The post to quote was found by its words, which alice's quotes from earlier runs
hold too; it is found by its id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 12:07:49 +02:00
thepraandClaude Opus 5.5 679463eb3f Edit history from formerRepresentations
Pleroma and Akkoma send a post's earlier versions with it. A post PrivaPub first meets after its edits now has its
history, and an edit that carries them brings the versions missed in between; each is read as the post itself is. The
Akkoma scenario checks a post fetched after two edits by an account no persona follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 12:07:49 +02:00
thepraandClaude Opus 5.5 02928caac1 A remote handle on its own domain, once that domain's WebFinger agrees (FEP-2c59)
An account on a server whose handles are not its host's (Mastodon's LOCAL_DOMAIN apart from WEB_DOMAIN) showed as
user@host. The actor's `webfinger` names the handle; its domain is kept once WebFinger there points back to the actor,
and asked again when the name changes. A persona's blocked servers match a handle's domain as well as the actor's host,
as the lists Mastodon exports name handles' domains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 11:49:32 +02:00
thepraandClaude Opus 5.5 e708f1ad2e hs2019 signatures hashed with SHA-512, and SHA-512 digests
A draft-cavage hs2019 signature leaves the hash to the key: with an RSA key it is tried with SHA-256, as nearly everyone
signs, then with SHA-512, as some do; rsa-sha256 stays SHA-256 only. A Digest of SHA-512= is checked as SHA-256= is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 11:13:59 +02:00
thepraandClaude Opus 5.5 0310bbe0f9 Castopod in the pasture
Castopod 1.15.5 (peers/castopod.sh): the official image on the shared MySQL, its podcast @pod made and published through
its admin pages, episodes through its REST API, its fediverse queue sent by spark. scenarios/castopod.sh, 15 checks: the
follow, an episode reaching alice with its sound, her like, boost and comment landing there, the unfollow, statistics
(from NodeInfo2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 11:12:06 +02:00
thepraandClaude Opus 5.5 69ab8e50eb Podcast episodes play; servers with NodeInfo2 only are described
Castopod announces an episode with a Note that only links to the episode's page, which serves a PodcastEpisode. PrivaPub
reads PodcastEpisode (its words in description, its sound in audio, its cover in image), and a Note that is only a link
to its author's own episode takes the episode's sound, title, cover and words, so the post plays. A server that
publishes no NodeInfo is described from NodeInfo2 (/.well-known/x-nodeinfo2), as Castopod publishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 11:12:05 +02:00
thepraandClaude Opus 5.5 3df3c333bf Forte in the pasture
Forte 26.9.10, the ActivityPub-only fork of Streams, built from its tag (images/forte: composer on Hubzilla's PHP image)
with portable identities (did:key actors behind /.well-known/apgateway). scenarios/forte.sh, 21 checks: follows, posts,
comments, likes, edits and deletions both ways, a third channel's comment passed on by the thread's owner, the
unfollow, statistics. INTEROP.md records what it does its own way: follows of profile pages, edits only as
Add{Update} under the Create's id, collections that misname themselves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 10:58:45 +02:00
thepraandClaude Opus 5.5 99b5e2dbf3 A follow of a persona's profile page; an Update under its Create's id
Forte follows a persona by its profile page (/@name, WebFinger's alias) rather than its actor's id: Follow and
Undo{Follow} now find the persona by either (it was a 404). Forte also sends an edit only added to the thread's context
(FEP-171b), under the same activity id as the post's Create: the unwrapping now tells two activities that share an id
apart by what they carry, as the inbox does, so the Update is not taken for a copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 10:58:45 +02:00
thepraandClaude Opus 5.5 d06f684f0d A thread owner's Add of a reply to its context is the reply passed on
FEP-171b, received (Hubzilla, Streams, Forte): an Add whose target is a collection of the actor's own server, and whose
object is someone's Create, Update or Delete of their own object, is queued as that activity forwarded by the owner, so
it is believed on its FEP-8b32 proof or as its origin has it, and shares its job with the plain forward Hubzilla also
sends. Checked live against Hubzilla (unchanged, 20 checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 10:03:44 +02:00
thepraandClaude Opus 5.5 b39eadffeb Hubzilla in the pasture
Hubzilla 11.4.1 with its pubcrawl addon (peers/hubzilla.sh): the hlhd image on the shared MySQL, a cron sidecar, hzuser
and hzfriend made through Hubzilla's own PHP as public channels that speak ActivityPub. scenarios/hubzilla.sh, 20
checks: follows, posts, comments, likes, edits and deletions both ways, and a third channel's comment that the thread's
owner passes on, which PrivaPub takes on its FEP-8b32 proof. Known gap G-0010 (upstream): Hubzilla 11 keeps a follower
after its Undo{Follow}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 09:59:49 +02:00
thepraandClaude Opus 5.5 9d0b5fac3e Backlog from a fresh village on today's PrivaPub: 2577 pass, one peer gap
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 09:17:36 +02:00
thepraandClaude Opus 5.5 9ab87b2779 Personas prove what goes to relays; the server says what it reads
FEP-521a and FEP-8b32. Every persona has an Ed25519 key of its own (Avatar.SigningKey; migration 014 gives the earlier
ones theirs), named in its actor's assertionMethod as a Multikey, the terms defined in the actor's own context. A
persona's activity going to a relay carries an eddsa-jcs-2022 proof (JSON canonicalised by RFC 8785, Jcs), so what
Activity-Relay forwards reaches Mastodon, which verifies it with its own code. Nothing else carries one: Mitra takes a
proof over the HTTP signature and refuses one by a key it has not read, without reading the actor again. Received: an
actor's own Multikeys are kept, and a forwarded activity whose proof one of them verifies is taken as it came instead of
being read again from its origin.

Discovery: WebFinger for the server's origin links its instance actor (FEP-d556), NodeInfo links it as the application
actor (FEP-2677), and actors name RFC 9421 under implements (FEP-844e).

Checked live: relay 16 (Activity-Relay's forward of alice's post reaches Mastodon), Mitra, GoToSocial and Mastodon
unchanged (165 in all).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 09:01:21 +02:00
thepraandClaude Opus 5.5 c47b6e5533 Personas have walls their followers write on
Owner decision of 2026-10-06 (G-0009, FEP-400e). A persona's actor names its wall (…/graffiti, sm:wall) and, in
Smithereen's privacySettings, that its followers may write on it. A public post that is not a reply, sent with the wall
as its target by an account following the persona, is hosted: the persona is notified, it reaches the persona's and its
followers' homes, and the followers' servers and the author's are told with Add{Note}. The persona deletes it with
DELETE /api/v1/statuses/:id, which sends Remove{Note}; Smithereen deletes the post then. The wall lists the persona's
public posts that start a thread and what was written on it.

Elsewhere: an account's Add{Note} on its own wall shows the post to its followers here as on that wall (privapub.wall on
the status), and its Remove takes it away. An Add or Remove naming a collection of the account's own server PrivaPub
does not know has its document read again first, at most hourly: accounts kept before walls were read had none.

Checked live: Smithereen 40 checks (G-0009 closed); GoToSocial and Mastodon unchanged (121).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 08:17:10 +02:00
thepraandClaude Opus 5.5 e0682fa868 A sender's digest of its followers here is compared and mended
FEP-8fcf, received. When a delivery's Collection-Synchronization header digests the sender's followers on PrivaPub
otherwise than the personas following it, a job reads the list the header names (on the sender's origin, signed by the
instance actor): a follow the list leaves out ends, only when the list is the one the digest describes; a request it
lists is taken as accepted; a persona it lists that follows nothing there sends Undo{Follow}, as Mastodon does. Each
claiming delivery is compared once.

Checked live (scenarios/followsync.sh, now 15 checks): PrivaPub ends a follow Mastodon lost and undoes one only
Mastodon remembered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 07:46:46 +02:00
thepraandClaude Opus 5.5 bd18de35a6 Deliveries to followers carry a digest of them, per server
Owner decision of 2026-10-06 (FEP-8fcf). A persona's delivery addressed to its followers carries a signed
Collection-Synchronization header naming its followers, its roll-call (…/groupies/roll-call) and the digest of its
accepted followers on the receiving server only. The roll-call answers a signed request with the persona's followers
on the signer's server and nobody else's.

Mastodon gives every Undo{Follow} it sends after reading a roll-call the same id (…#follows//undo), so a second one
looked like a copy: an Undo of a Follow that comes again while the follow it ends exists again is now kept once per
follow.

Checked live (scenarios/followsync.sh): Mastodon drops a follow PrivaPub lost, and undoes one it lost itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 07:37:03 +02:00
thepraandClaude Opus 5.5 bf37223849 Public threads name their replies and context
Owner decision of 2026-10-06: a persona's public or unlisted post outside any group names its replies
(…/scribbles/{id}/replies) and its conversation's context (FEP-7888), the root's …/context that a reply inherits from
its parent, ours or another server's. Both list only the public and unlisted posts PrivaPub holds; followers-only,
circle, direct and local-only posts name neither and their collections answer 404. Checked live: opening alice's
thread on Mastodon finds carol's reply, which nobody there follows (scenarios/threads.sh).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 07:13:54 +02:00
thepraandClaude Opus 5.5 93e13e5563 Owner decisions of 2026-10-06; personas' public posts go to relays
The owner decided four gated questions, now in ROADMAP: personas publish a wall (G-0009), their public posts go to the
relays PrivaPub subscribes to, public threads publish their replies and context, and FEP-8fcf follower digests are sent.

The first is in: a persona's own public post outside any group, its edit and its deletion also go to the relays that
accepted us, as Mastodon sends them; nothing less public, and no boost. Checked live (scenarios/relay.sh, 15 checks):
the post reaches Activity-Relay, and Mastodon through aode-relay's announce. Mastodon drops what Activity-Relay forwards
without an LD signature or FEP-8b32 proof, which PrivaPub does not add yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 07:08:37 +02:00
thepraandClaude Opus 5.5 b45321f28d Following an account brings its earlier posts to its profile
Once a follow holds, OutboxBackfill reads the account's latest public posts from its outbox's first page (twenty at
most, once a day) and keeps them as any fetched post: its profile shows them at once instead of only what it posts from
then on. Homes still get only what arrives afterwards, as on Mastodon. Announces and other servers' objects in the
outbox are left out. Checked live against Mastodon (scenarios/pins.sh, now 9 checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 02:15:49 +02:00
thepraandClaude Opus 5.5 b218768904 Pasture: Vernissage 1.43.0
Vernissage joins the pasture on SQLite with its queues on the shared Redis; scenarios/vernissage.sh: follows both ways,
a photo with its alt text, likes, boosts and comments both ways, a deletion, the unfollow and statistics: 19 checks,
with no change to PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 02:11:53 +02:00
thepraandClaude Opus 5.5 53158b4cf4 One delivery a server; BookWyrm 0.9.3 in the pasture
An activity now goes once to each server, to its shared inbox when it has one, for the accounts a post names, answers
or quotes as for its followers, as Mastodon delivers (a circle post excepted: each member's copy names that member).
BookWyrm took the same post twice when one copy reached its shared inbox and another the named account's inbox at once.
SharedInboxTests checks a reply to a follower's post goes once.

BookWyrm joins the pasture (peers/bookwyrm.sh: its image on the shared Postgres and Redis, gunicorn and a Celery worker,
its user, book and statuses made in its Django shell). scenarios/bookwyrm.sh: follows both ways, a review, a comment and
a quotation reaching alice as BookWyrm's pure posts, her like, boost and reply landing there, her post naming bwuser and
bwuser's like and reply, a deletion, the unfollow and statistics: 20 checks. GoToSocial (64), Mastodon (57) and Misskey
(35) still pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 01:56:06 +02:00
thepraandClaude Opus 5.5 12b51a211a Pasture: Ghost 6.67 and its ActivityPub service
Ghost joins the pasture with its ActivityPub service (Fedify) on the shared MySQL, routed by Caddy as Ghost's own proxy
does. scenarios/ghost.sh: follows both ways, the publication's titled Articles with their edit and deletion, likes and
boosts both ways, Ghost's reply and note, alice's post in its Network feed, both unfollows and statistics: 22 checks,
with no change to PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 00:49:18 +02:00
thepraandClaude Opus 5.5 a584134858 Pasture: Owncast 0.3.0
Owncast joins the pasture with federation turned on through its admin API; scenarios/owncast.sh has alice follow the
stream, receive its admin's message, like and boost it (both show in Owncast's admin), and unfollow: 13 checks, with
no change to PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 00:32:46 +02:00
thepraandClaude Opus 5.5 0aea550c6c Pasture: WriteFreely 0.17.2
WriteFreely joins the pasture, built from its release with openssl beside it (it makes each blog's keys with the
command); scenarios/writefreely.sh has alice follow a blog and receive its post as a titled Article, its edit and its
deletion, then unfollow: 13 checks, with no change to PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 00:28:11 +02:00
thepraandClaude Opus 5.5 5721bfd796 Pasture: snac2 2.95
snac2 joins the pasture, built from its tag (images/snac2); scenarios/snac.sh drives its Mastodon API and reads its
own files, since its timelines lag behind what it has taken in: follows, posts, replies, likes, boosts, a poll, edits,
deletions, direct messages, the unfollow and statistics, 27 checks, with no change to PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-06 00:19:56 +02:00
thepraandClaude Opus 5.5 1c0dfeb9a4 Pasture: Mitra 5.9.1
Mitra joins the pasture (peers/mitra.sh) on the shared Postgres; scenarios/mitra.sh drives its Mastodon API through
follows, posts, replies, likes, reposts, a reaction, a poll, edits, deletions, direct messages, the unfollow and
statistics: 28 checks, with no change to PrivaPub. Mitra carries FEP-8b32 proofs made with Ed25519, which PrivaPub does
not verify yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 23:35:35 +02:00
thepraandClaude Opus 5.5 f6fc0c535c Relays: PrivaPub reads from the relays its configuration names
Federation:Relays names relays by their actor (or inbox) address; the instance actor follows Public at each, as Mastodon
subscribes, a minute after start and every six hours (asked again a day after no answer or a refusal, undone when a
relay is no longer named). What an accepted relay passes on comes to the federated timeline and nobody's home: a public
post it forwards (Activity-Relay), read again from its origin like any forwarded post, and a post it announces
(aode-relay), kept as its author's and never as the relay's boost. Nothing of a persona's is sent to a relay; sending
public posts there waits for the owner.

The pasture gains both relays (peers/relay.sh, peers/aoderelay.sh) and scenarios/relay.sh, 13 checks. The village
backlog is clean: 2574 checks pass, one known gap (Misskey's).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 23:11:59 +02:00
thepraandClaude Opus 5.5 1c7d1ece9c Pins across servers, both ways
An account elsewhere that pins or unpins one of its posts (Add or Remove on its `featured`) now shows those pins on its
profile here (`pinned=true`), in its order and its public posts only; a community's announce of a moderator's Add does
the same for the community. The `featured` collection itself is read with the account's counts, at most once a day, so
pins made before PrivaPub ever saw an account show too. Any other target (Smithereen's wall, a community's moderators)
is dropped. A persona's pin and unpin go to the post's audience as Add and Remove on /trophies, as Mastodon sends them.

Checked live against Mastodon (scenarios/pins.sh, 8 checks). The town's checker learnt three peer rules from the
village: Misskey and Sharkey keep a forwarded reply only with its author's LD signature (only Mastodon signs), they
count no renote by a bot, and Mastodon never sees a Lemmy vote on a post in a community. The village of 2026-10-05
checks clean, 2454 of 2454.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 22:48:57 +02:00
thepraandClaude Opus 5.5 5f150b7a59 Pasture: Smithereen 1.0.3, and failed server descriptions retried
Smithereen joins the pasture (tools/pasture/peers/smithereen.sh): its image on the shared MySQL, with imgproxy and a
file server behind Caddy, a JDK trust store with the pasture's CA, accounts from its signup form, and a password grant
for a local application. scenarios/smithereen.sh drives its VKontakte-like API: friends as mutual follows, wall posts,
comments, likes, reposts (quotes there), polls, edits, deletions, private messages, the unfollow and statistics, 30
checks. A post on someone else's wall never reaches PrivaPub, since Smithereen sends it only to servers whose actors
publish a wall: G-0009, waiting for the owner.

PrivaPub: a server description that failed (Smithereen serves no NodeInfo until it has a description) frees its week,
so the server's next arrival asks again instead of a week later.

The shared Postgres takes 400 connections: at 100 the village seed ran it dry (Sharkey's API answered 500, Misskey
dropped deliveries). The seeder records a follow that stands from an earlier seed when the step itself fails, so the
checker no longer expects that follower to see nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 22:03:35 +02:00
thepraandClaude Opus 5.5 7b0377c85f Moves carry follows over; first DMs to Lemmy 0.19 and Mbin go as ChatMessage
Two owner decisions of 2026-10-05, both recorded in ROADMAP:
- After a verified Move the personas following the old account follow the new one, in the same lists, and a mute or
  block of the old account carries over, as Mastodon does it.
- A direct message to one account on a server whose NodeInfo names Lemmy before 1.0 or Mbin goes as a ChatMessage,
  the one place PrivaPub decides by a server's software (invariant 17). G-0008 is closed.

Mbin addresses its private messages to the recipient's profile page, so a Create addressed to a persona's /@name now
reaches the persona. Checked live: moves 8/8, Lemmy 0.19 30/30, Mbin 26/26 with messages both ways. The software
theory runs alone, since every test's peer shares 127.0.0.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 20:57:43 +02:00
thepraandClaude Opus 5.5 f3e8cce2ed Pasture: PrivaPub's uploads live in a volume
They lived in the container, so a container made again (a run with other settings)
lost every picture the town had posted from PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 20:30:45 +02:00
thepraandClaude Opus 5.5 eeb873b816 Load: a flood of fake servers; unique-key lookups no longer scan
tools/pasture/flood/flood.cs answers as twenty fake servers (flood1..20.test)
and sends signed Creates, Likes and Follows at a set rate; load.sh measures
the answers, the queue's wait and processing times, its drain and a persona's
home timeline meanwhile (docs/LOAD.md has the method and the runs).

What the runs found:
- Every unique index was partial on $type: "string", which MongoDB never uses
  for an equality lookup, so every post by ObjectURI, actor by ActorURI,
  deleted object, domain block, remote instance and the rest was a
  collection scan (280 ms a post lookup at 30 000 posts). They are partial on
  $gt: "" now, which an equality on a string implies; MongoDB.Entities
  rebuilds them in place at the next start.
- Two inbox workers capped intake near 110 activities a second:
  Federation:InboxConcurrency and DeliveryConcurrency (default 8) set them.
- The indexes the plan listed as missing: a post's boosts and replies, a
  persona's boosts, who follows an actor, timeline rows by author, a post's
  likes and pins.

At 300 activities a second (200 let through, the rest 429 by the per-origin
limit) the queue wait went from 29 s to 6 ms at p50; with the limits lifted
PrivaPub processes about 900 a second, each in under 10 ms, and the home
timeline stays under 20 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 19:16:30 +02:00
thepraandClaude Opus 5.5 f1e743c331 An account that moves shows where it went
PrivaPub dropped Move as an unknown type and showed no `moved` on accounts.
Now a Move is believed as Mastodon believes it: the moving account sends it
about itself, and the new account, read again from its own server, names it
in alsoKnownAs (now kept on remote accounts). The old account then shows the
new one as `moved` in the Mastodon API. The personas following it keep
following it: following the new account on their behalf would tell another
server about them, so that waits for the owner.

Checked live against GoToSocial (scenarios/moves.sh: an alias, a move, 6
checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 18:39:46 +02:00
thepraandClaude Opus 5.5 1265611f9e A group's owner sees its members and requests, declines and removes
/clientapi/group had no way to see who is in a group or asks to join it,
so a circle's owner could not answer a request from elsewhere. Now:
- GET /clientapi/group/members: the members (local and remote, with their
  role) and the pending requests, for the group's owner and moderators
  only;
- POST /clientapi/group/reject: declines a request, telling the asker's
  server with a Reject of its Follow;
- POST /clientapi/group/remove: takes a member out (never the owner): a
  persona here stops following the group, one elsewhere gets a Reject of
  its Follow, as Mastodon removes a follower.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 18:24:29 +02:00
thepraandClaude Opus 5.5 f38ac73615 A direct message answers in the form its recipient writes in
Lemmy 0.19 (most of the threadiverse) and Mbin take a private message only
as a ChatMessage and refuse a direct Note. A direct message to one account
elsewhere that writes to us as ChatMessages now goes out as one: to it
alone, without a mention in its text, kept on the post (Post.AsChatMessage)
so the served copy and an edit match. No software name decides it.

Live against Lemmy 0.19: alice's answer to lemmyuser's private message and
another persona's message to lemmyuser arrive (29 checks). A first message
to an account that never wrote to anyone here is still a Note, which they
refuse; G-0008 keeps that open for the owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 18:13:51 +02:00
thepraandClaude Opus 5.5 a85ebd30c0 Pasture: Lemmy 0.19 joins, built to trust the pasture's CA
Most of the threadiverse runs Lemmy 0.19, whose release trusts only the
roots its rustls bundles. images/lemmy19 builds 0.19.20 from its tag with
reqwest's rustls-tls-native-roots added, and the pasture runs it as
lemmy19.test. scenarios/lemmy19.sh passes 27 checks with no change to
PrivaPub (communities, threads, comments, votes, its private message,
moderation) and one known gap: 0.19 takes private messages only as
ChatMessage and answers our direct Note 400, like Mbin, so G-0008 now
covers both and waits for the owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 17:32:22 +02:00
thepraandClaude Opus 5.5 bd5901d15e Town: the report shows replies passed on; the backlog after wave 2's first peers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 17:02:05 +02:00
thepraandClaude Opus 5.5 6b08da458a Pasture: NodeBB joins
NodeBB 4.16.1 runs in the pasture on the pasture's Mongo, set up by its
automated setup, with an API token written where NodeBB keeps them.
scenarios/nodebb.sh passes its 23 checks with no change to PrivaPub: a
category followed and its topic as a titled thread, replies both ways, a
follow of alice and her post there, votes both ways, an edit and a
deletion, a chat both ways, the unfollow and statistics.

NodeBB never federates a topic's lock, and its API follows an account
elsewhere only when named by its handle; both are in docs/INTEROP.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 16:58:08 +02:00
thepraandClaude Opus 5.5 26346cde30 Mbin joins the pasture; a magazine's own threads and locks are taken
Mbin 1.10.1 runs in the pasture (its image, a messenger worker, a RabbitMQ
of its own, its API limits raised), and peers/mbin_token.py gets mbuser's
token through the authorization-code flow. scenarios/mbin.sh: 24 checks and
one known gap, magazines both ways, titled threads, a Note to a magazine as
a microblog post, comments, favourites and upvotes both ways, a moderator's
lock, unlock and removal, the unfollow and statistics.

What it showed:
- Mbin sends a magazine's threads to its subscribers as the author's Create,
  the magazine as its audience, never announced. A post whose group is
  followed here and lives on the post's own server is now kept as if
  announced; the same from another server is not.
- A moderator's lock is a bare Lock (and Undo{Lock}): LockHandler takes it
  from the post's own server only.
- Mbin takes private messages only as ChatMessage and its actors say
  nothing about it; PrivaPub never decides by a server's software, so this
  stays open as G-0008 for the owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 16:41:50 +02:00
thepraandClaude Opus 5.5 47d6e22988 PieFed joins the pasture; the instance actor answers at the root
PieFed 1.7.17 (dockurr's image of the release) runs in the pasture with its
Celery worker on the shared Postgres and Redis, and scenarios/piefed.sh
checks it both ways: 29 checks, communities, titled threads, comments, votes
up and down, a community poll and a vote in it, private messages, a
moderator's lock, unlock and removal, the unfollow and statistics.

What it showed:
- PieFed sends a community's announces to the inbox of the Application at a
  peer's root (as Lemmy serves its site actor) and to /inbox otherwise.
  PrivaPub answered 404 at its root, so every announce went to an /inbox it
  does not have. The instance actor now answers at / for ActivityPub
  requests, unsigned under SecureMode as at its own address.
- PieFed keeps serving a thread its moderator removed, so the removal could
  never be checked against the post's origin. A community on the post's own
  server now speaks for it; one elsewhere still waits for the origin to say
  the post is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 15:44:13 +02:00
thepraandClaude Opus 5.5 0646de22bd Replies to a persona's posts reach its followers; personas join remote events
Two owner decisions of 2026-10-05, recorded in the roadmap.

Replies passed on ("the fediverse is broken without"): a public or unlisted
reply from another server to a persona's public, unlisted or followers-only
post goes on to the persona's followers as its author's server sent it, as
Mastodon forwards it, never to the replier's own server, never for a
local-only or group post; its edit and deletion follow. Only an activity its
own actor delivered is passed on (Arrival.Raw), so nothing forwarded is
forwarded again. The town checks it as relay.reply cells (specs/relay-five:
882 checks pass); Mastodon takes a passed-on activity only with an LD
signature, which GoToSocial and Akkoma don't add, and the checker knows it.

Events: a persona joins another server's event with a Join and leaves it with
a Leave, both to the organiser only, through
POST /api/privapub/v1/statuses/:id/join|leave; the organiser's Accept or
Reject is routed by our join id and shows as privapub.event.participation.
Events by invitation or taken on another site are refused before anything
is sent. Mobilizon's scenario joins and leaves an event (28 checks) and keeps
one for decePubClient's e2e.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 14:55:20 +02:00
thepraandClaude Opus 5.5 2b42377b3f Roadmap: wave 1 is done
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 12:02:46 +02:00
thepraandClaude Opus 5.5 8ae265c8f8 Pasture: Funkwhale joins, closing wave 1
Funkwhale 2.0.11: the API, a Celery worker with its beat and the front's nginx sharing one volume, on the shared Postgres
and Redis, trusting the pasture's CA through REQUESTS_CA_BUNDLE; fwuser and its OAuth token made by its own tools.
scenarios/funkwhale.sh passes its 16 checks: alice follows a channel, a track uploaded to it arrives as Audio with its
file and duration, its deletion reaches PrivaPub, the unfollow, statistics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 12:02:25 +02:00
thepraandClaude Opus 5.5 9f0b25e92b Funkwhale's answers, deletions and NodeInfo are understood
Three shapes Funkwhale 2.0 sends, each of which lost something:
- its Accept is named after the Follow it answers, on our origin (`…#follows/<uuid>/accept`), and was refused as off its
  actor's origin, so no follow of a channel completed: an Accept or Reject whose id extends the id of the activity it
  answers, on our origin, is now taken;
- a channel deletes its uploads in one Delete without an id, their ids in a list as the object's id: each is deleted;
- its NodeInfo discovery names the document under its swagger schema's URL: a link whose path names NodeInfo is taken
  when no rel is known.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 12:02:25 +02:00
thepraandClaude Opus 5.5 3866558d1d Roadmap: wave 1 and GoToSocial's interaction policies
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 11:46:16 +02:00
thepraandClaude Opus 5.5 fb90558dec Town: client cells from their latest run; the backlog after a fresh village
decePub's e2e runs append to out/client.jsonl. The report kept the newest row per test and account, and the backlog
every row, so a failure from a run that picked other accounts stayed in both after later runs passed. Each client
cell now counts as its latest run left it (check.latest_client: the rows within an hour of its newest).

The backlog comes from a fresh village on today's build: 2556 checks pass and 4 fail, three of them peer-to-peer
counts, and one GoToSocial losing a status from its cache (an Update handled as a Create; a restart heals it),
recorded in INTEROP.md rather than as a gap, which would hide our own edits failing there. The community vote and the
Lemmy thread that failed before were data from older builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 11:45:57 +02:00
thepraandClaude Opus 5.5 5860b71223 GoToSocial's interaction policies are honoured
A remote post's canReply, canLike and canAnnounce (with the older always and approvalRequired) are kept beside
canQuote and judged for each persona: let in at once when the rule names the public, the persona, the author's
followers while it follows the author, or the accounts the author follows while the author follows it; asked first
when only the manual list names it; refused (422) otherwise. Asked first, a ReplyRequest, LikeRequest or
AnnounceRequest with the interaction as its instrument goes to the author alone, and the interaction waits
(privapub.approval: pending). The author's Accept brings an authorization, verified on the author's origin as naming the
interaction and the post; the reply then goes out with replyAuthorization, the boost with announceAuthorization, the
like with likeAuthorization. A Reject leaves the reply ours alone and takes a like or a boost back. As a third party, a
reply a policy does not let in at once is kept only with an authorization that verifies. Clients see the rules as
GoToSocial's interaction_policy.

Checked live against GoToSocial 0.22.1: the scenario's nine new checks pass (64 in all), a reply and a like approved
through its interaction requests and a boost refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 11:43:58 +02:00
thepraandClaude Opus 5.5 c65a44ba88 Pasture: Gancio joins
Gancio 1.28.2 on its own sqlite, its config.json written before the first start (without it Gancio waits in its setup
wizard), trusting Caddy's CA through NODE_EXTRA_CA_CERTS; fediverse replies are kept as event resources. Its one
Application actor publishes the agenda, and scenarios/gancio.sh passes its 17 checks with no change to PrivaPub: the
follow, an event with its start, end and place, a reply kept as a resource, the edit and the deletion, the unfollow,
statistics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 11:18:50 +02:00
thepraandClaude Opus 5.5 761cfc2cce Pasture: Mobilizon joins
Mobilizon 5.2.4 on a PostGIS of its own (it needs the extension, which the shared Postgres has not got), trusting
Caddy's CA through the bundle mounted over certifi's and castore's files (hackney trusts only those), with geocoding
pointed at a closed local port. scenarios/mobilizon.sh passes its 23 checks: alice follows a group; the event its
organiser makes arrives as an Event with its start, end and located place; comments both ways; the organiser's edit,
closing the comments (PrivaPub then refuses a reply) and deletes of a comment and the event; a group post with its
title; the unfollow; statistics. An event made through Mobilizon's API without options has its comments closed, so
the scenario opens them. No RSVP yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 10:54:32 +02:00
thepraandClaude Opus 5.5 2d293a6148 An organiser's edits to a group's event follow its server
Mobilizon's organiser sends the Create, Update and Delete of an event attributed to the group, which announces the
Event itself. PrivaPub refused the organiser's activities as misattributed (400) and kept the event through the
group's Announce, so an edit was lost and a deletion left the event in place. An object attributed to another account
of the actor's own server is now that server's to vouch for: created or edited as the server has it, under the account
it is attributed to, and deleted once the server answers 404 or 410. Attributed to an account elsewhere, it is still
refused. Checked against Mobilizon 5.2.4 in the pasture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 10:54:32 +02:00
thepraandClaude Opus 5.5 7f6837ccb1 NodeInfo is read as Mobilizon serves it
Mobilizon sends its NodeInfo as `application/json; profile=http://…#` with the URL unquoted, which .NET cannot parse,
so the document was refused for its content type and the server never described; and it names its software
"Mobilizon" where NodeInfo wants lower case. The media type is now read from the raw header when the parsed one is
missing, and software names are lowercased, so one software is counted under one name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 10:45:18 +02:00
thepraandClaude Opus 5.5 206506f5e9 A follow request sent again carries a new id
Lemmy keeps the ids of the activities it received and never answers one again, so resending the same Follow could not
heal a follow whose Accept it lost: the village's community follow stayed pending through two resends. Each resend is
now the same follow under its own id (<follow id>-again-<n>), and an Accept naming any of them answers the follow;
the Undo still embeds the follow, so servers match it by its actor and object. Sent this way, the stuck follow was
accepted at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 10:45:18 +02:00
thepraandClaude Opus 5.5 a5d493a8c9 Pasture: Friendica joins, in the scenarios and the town
Friendica 2026.05 on the shared MySQL and Redis, with its worker daemon as a sidecar. friendica_settle repairs what
its install leaves: the system user has no name, so the system account that signs its fetches is never made; the web
container cannot see the sidecar's daemon, so a queued job waits for the five-minute cron unless the daemon is
declared running; its log needs a file and debugging on. Accounts are saved through its API with locked=0 (a number:
"true" reads as 0, unlocked), which makes them soapbox pages that take followers without following back, and each
one's outbox is read once: a Follow that reaches an account Friendica has not cached makes it fetch the account from
itself, signed, and checking that signature recursed for five minutes, holding PrivaPub's first follow past its timeout.

scenarios/friendica.sh passes its 25 checks from a clean install: follows and unfollows, posts (a titled one with its
title), comments, likes, Friendica's dislike as a downvote, boosts, edits (through its web editor: its Mastodon API
never federates one) and deletes, both ways.

The town gets a Friendica driver (HTTP Basic, its MySQL read through mysql_json, edits in the web editor, no bookmark
on a reply) and specs/friendica-pair.json, which passes its 275 checks. On the way:
- the checker knows Friendica's thread model: a non-public reply reaches an account only under posts it holds, and a
  Friendica account's non-public reply in a thread another server owns reaches nobody else there;
- the seeder answers a follow request the target still holds whatever the follower's server says: Friendica reports a
  follow of someone already following its account as made at once (and shows that persona's followers-only posts
  while a locked persona still holds the request);
- the selftest skips polls where a platform has none; the shared MySQL helpers move to peers/shared.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 10:14:02 +02:00
thepraandClaude Opus 5.5 e485f7bd47 An id reused for another activity is not a copy
Friendica's activity ids are uniqid(): a short prefix and the microsecond. Two of its processes answering two follows at
once gave both Accepts one id, and PrivaPub, queueing each inbox activity once per id, dropped the second as a copy:
that follow stayed pending on our side while Friendica counted the persona as a follower (seen in the town's Friendica
pair). An id that comes back carrying another type, actor or object is now queued apart; a true copy is still dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 09:52:02 +02:00
thepraandClaude Opus 5.5 7eb7c017a5 Forwarded activities are believed as far as their origin vouches
A thread's server passes on what happens in it, signed with its own key: Mastodon forwards the replies to its
accounts' posts and their deletions, Friendica every activity in its threads. PrivaPub answered them 401, which also
tells a sender its signature failed. Now they get 202 and nothing in them is believed: a forwarded Create or Update is
taken as its object reads at the actor's origin, a Delete of a public or unlisted copy once that origin answers 404 or
410 (RemoteActorService.IsGone; FederationHttp remembers the status of a refusal), anything else is let go, and our own
activities coming back are ignored. A forwarded copy has its own dedupe key, so one that failed never hides the
author's own delivery.

A reply in the thread of someone followed here is kept, as Mastodon keeps them. Mastodon delivers a reply to the
followers of the account it answers; PrivaPub dropped those as unaddressed, which the pasture showed: the outsider's
reply its Mastodon scenario said was never delivered had been, and was thrown away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 09:34:08 +02:00
thepraandClaude Opus 5.5 01808fa644 Follow requests nobody answers are sent again
A Follow was resent only when the persona followed again. Lemmy 1.0
sends nothing it queued for a server before it started sending there, so
the Accept of a community follow made on first contact was lost for good:
the village's persona stayed "requested" a day while Lemmy listed her as a
follower, and every post the community announced was refused as not
followed. A request still unanswered is now sent again, the same activity,
after 15 minutes, an hour, 6 hours, a day, two and four days
(FollowResender, every 15 minutes); a server that holds the follow answers
the copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 07:33:59 +02:00
thepraandClaude Opus 5.5 7e37f56db6 A community's vote on our reply counts, followed or not
Lemmy sends a vote to the community alone, which relays it to the post's
server. A persona's reply in a thread of a community nobody here follows
lost its Lemmy upvotes: the relay was dropped as "not followed". Such a
relay is now taken when the vote (or its undo) is on one of our posts in a
thread rooted in that community; from any other unfollowed group it is
still dropped. For that, a post keeps the community its `audience` names
(FEP-1b12) however it arrived, fetched for a thread as well as announced.

Found by the town's village (p191: 1 like counted of 2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 07:29:12 +02:00
thepraandClaude Opus 5.5 8686275b69 The statistics name RFC 9421 signatures
A delivery signed with RFC 9421 was counted under "other": its algorithm
(rsa-v1_5-sha256) has an underscore, which a signature name could not
hold. It is now counted as rfc9421:rsa-v1_5-sha256, as WordPress's are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 07:04:35 +02:00
thepraandClaude Opus 5.5 01e5aa4ddd Pasture: Pixelfed and WordPress join
Pixelfed 0.14.4 (FrankenPHP with Horizon and the scheduler, on the shared
Postgres and Redis) and WordPress 6 with ActivityPub 9.3.1 (on a new shared
MySQL, WP-Cron run by a sidecar) are peers now, each with its scenario:
Pixelfed 26 checks (photos both ways with alt text, its place arriving as
Rome with its coordinates, comments, likes, a boost, edits and deletes),
WordPress 17 (an Article with its title, our reply, like and boost kept as
comments of their kinds, its edit and removal). Pixelfed has a town driver
and a pair spec: 145 checks pass, 2 expected (G-0007).

What they showed: Pixelfed names our posts by their page (fixed in
26dac40); on PostgreSQL its migration making caption nullable never runs,
so every remote boost failed (the pasture applies it); it files a DM it
fetches as followers-only (G-0007, upstream); Passport refuses a token
whose user id equals its client's id. WordPress signs with RFC 9421
first, which PrivaPub now verifies (c5a69d2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 07:01:38 +02:00
thepraandClaude Opus 5.5 c5a69d240a RFC 9421 signatures are verified, not refused
WordPress's ActivityPub plugin (and Ghost and Fedify) sign with RFC 9421
first and fall back to draft-cavage only after a refusal, so each first
delivery cost two requests and a 401 in our statistics. Now a request
carrying Signature-Input is verified as an HTTP message signature: its
covered components (the method and our own public target, the body's
Content-Digest), its created and expires, with the actor's RSA key under
PKCS#1 v1.5 or PSS. Deliveries and signed fetches both take it; the
ledger names the scheme (rfc9421:rsa-v1_5-sha256). What PrivaPub sends
stays draft-cavage, which every server reads. Ed25519 waits for FEP-521a
keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 06:48:22 +02:00
thepraandClaude Opus 5.5 26dac40720 A post named by its page is found as by its id
Pixelfed names one of our posts by the address of its page
(/@name/<post id>, the post's url) in its Like, Announce and their Undo,
so its likes were dropped as unknown objects. Before an activity is
handled, such a reference to a post of ours, as its object or the object
of the activity it undoes, is replaced by the post's id. Found by the
pasture's new Pixelfed peer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 06:40:46 +02:00
thepraandClaude Opus 5.5 34c0f696af A community's moderators lock threads and ban members
Lemmy's moderation reached PrivaPub only as removals. Now a remote
community's lock and ban, relayed in its Announce, apply too:

- a lock (Announce{Lock}, or commentsEnabled false on the post) refuses
  replies to the thread, ours included, until Undo{Lock}; statuses say so
  in privapub.locked;
- a ban of a persona (Announce{Block} with the community as target, or the
  moderator's own Block sent straight to us, which is the community's ban
  and never the moderator's block of the persona) shows as blocked_by on
  the community and refuses the persona's posts and replies there until
  the Undo.

The Lemmy scenario's removal was an expected failure only because it gave
up before Lemmy's 30-second batch; it now waits, and checks the lock and
the ban live (Lemmy refuses a lock or an unban without a reason): 29
checks, none expected to fail. G-0003 and G-0006 are closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 06:10:33 +02:00
thepraandClaude Opus 5.5 4a9a3d3235 A handle finds its account, not posts and a made-up hashtag
Since status search by words, searching "@name@host" also answered posts
sharing the handle's words and a hashtag "#@name@host" made of the query,
so a client that opens the profile when the account is the only result
(decePub's search) stayed on the list. A handle now searches accounts only,
and a hashtag result is offered only when the query is one (letters, marks,
digits and underscores, with a letter).

Found by decePub's end-to-end tests: every profile follow test failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 06:10:25 +02:00
thepraandClaude Opus 5.5 17922f4dfd Conversations page, and know what is unread
/api/v1/conversations answered one page, never unread, its read endpoint
did nothing, and DELETE was missing; each conversation cost a query per
member. Now each conversation keeps its newest post (DmGroup.LastPostId,
set as posts arrive, learnt once by migration _013) and pages by it as
Mastodon does, and each persona's ConversationState holds what it read and
what it took off its list:

- unread when someone else wrote last, after what the persona read;
- read marks it so, and writing in a conversation reads it;
- DELETE takes it off the list until a newer message brings it back.

The list reads its states, newest posts, members and accounts in a few
queries per page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 05:37:28 +02:00
thepraandClaude Opus 5.5 ea5e607779 Reports and statistics answer in ClientModels' shapes
The moderators' reports and the admin statistics (overview, hosts,
crawler) were anonymous objects, so a client could read them only as loose
JSON. They are now ViewReport, ViewStatisticsOverview, ViewHostsPage and
ViewCrawler in PrivaPub.ClientModels, with the same JSON as before, for
decePub's Administration page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 05:32:32 +02:00
thepraandClaude Opus 5.5 b146e182c6 Pasture: scenarios run again on the same pasture, and Mastodon's threads
GoToSocial's and Mastodon's scenarios passed once on a fresh pasture only:
their locked follows ended followed, so a second run found no request to
reject, and Mastodon's ended with mastouser blocking alice_masto, so every
follow after it was rejected. Each now unfollows first, and Mastodon's
undoes its block (checking that its Undo{Block} reaches our blocked_by) and
its lock; a run cut short is undone at the start.

Mastodon's scenario also checks the thread backfill live: a reply by an
account nobody here follows is never delivered, and joins the thread once
alice_masto opens it, read from Mastodon's FEP-7888 context.

interop.sh keeps the results of the peers it does not run, so the report
merges a partial rerun. All seven scenarios: 276 pass, 0 fail, 1 expected
(Lemmy moderation, P7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 05:32:01 +02:00
thepraandClaude Opus 5.5 0695c08ac3 A remote thread brings the replies that never reached us
The context of a remote post showed only what PrivaPub happened to hold:
replies from servers nobody here follows were never seen, and only the
ancestors were ever fetched. Now a persona opening a public remote thread
queues FetchReplies for the post and its root, at most once an hour each.

The job reads the thread's own collection first (FEP-7888 `context`, which
Mastodon 4.5+ serves with every reply at any depth; posts or, as FEP-f228
allows, the activities that made them), and otherwise the post's `replies`
(PeerTube's `comments`) and the replies' own, two levels down. At most 5
pages and 100 posts a job, signed by the instance actor, never a persona;
each post is fetched from its own origin and stored through StoreContext,
so only public and unlisted ones are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 04:54:38 +02:00
thepraandClaude Opus 5.5 bbeeda7268 A vote turned the other way replaces the first
Lemmy turns an upvote into a downvote with a Dislike alone (and back with a
Like), so the like stayed counted next to the downvote. Now an account has
one vote on a post: a Dislike takes its like away, a Like its downvote.

The Lemmy scenario's relayed votes were never failing for that reason only:
Lemmy 1.0 sends what it queued every 30 seconds, and the check gave up after
30. It waits a minute now, and both votes are plain checks: 22 pass, the
moderator's removal stays an expected failure (P7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 04:35:23 +02:00
thepraandClaude Opus 5.5 325f7605f2 A notifications marker reads what it passes
Saving the notifications marker (/api/v1/markers) moved the marker but left
every notification unread, so a Mastodon client's unread count never fell.
The marker now marks read every notification up to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 04:18:46 +02:00
thepraandClaude Opus 5.5 1d12dff033 A notification told alone names its group
v1 notifications, and those a stream tells, carried an 'ungrouped' key, so
a client reading grouped notifications could not fold a new like into the
group of likes it already shows. Each now names the group it belongs to by
default (likes and boosts of one post, follows within an hour).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 04:17:27 +02:00
thepraandClaude Opus 5.5 aecbf620a8 A community's comments reach its microblog members
A comment in a community was announced only as Announce{Create}, which
Akkoma, Mastodon and Misskey drop: their members never saw it (the village
found Akkoma's missing). A comment is now also announced as itself, as a
new post already was; Lemmy answers that form 400, harmlessly. Nothing is
decided by the follower's software.

The town's checker expects a followers-only quote to stay with the
followers, and G-0003 covers only Lemmy-hosted communities (their relayed
moderation), since relayed likes count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 04:12:53 +02:00
thepraandClaude Opus 5.5 9268968fd4 Streaming: what happens, told as it happens
Mastodon's streaming API: /api/v1/streaming as a WebSocket (streams
subscribed in the URL or by message) and /api/v1/streaming/{stream} as
server-sent events, with health and the URL advertised. The user stream
tells posts reaching the persona's home (not those an exclusive list keeps
apart, which its list stream tells), notifications, edits and deletions;
public, hashtag and list streams tell what belongs in them. An in-process
hub carries ids only; each connection maps a post or a notification for its
own persona as it sends it, so nothing it may not see, or whose author it
blocked or muted, goes out. A deletion reaches only the streams that showed
the post. The token comes as access_token, header or WebSocket protocol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 04:04:08 +02:00
thepraandClaude Opus 5.5 b274ce1268 Town: one report for several runs
A village and the pair runs (Hollo, Iceshrimp.NET, Pleroma) are checked
apart; town.sh report takes them together and adds their cells up, so one
matrix shows every server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:54:29 +02:00
thepraandClaude Opus 5.5 40217f3261 Pasture: PeerTube joins, and its videos play through us
PeerTube 8.3.1 runs on the shared Postgres and Redis, transcoding off. Its
scenario passes 24 checks: a persona follows a channel; a new video comes as
the channel's Announce and reaches the persona's home as a boost, playable
through the media proxy with byte ranges; comments both ways thread; a like
and its undo count; renaming and deletion arrive; the unfollow; statistics.

The town's seeder also takes reruns on the same accounts in its stride: a
Lemmy community already made is found, a circle member already approved
asks nothing again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:52:29 +02:00
thepraandClaude Opus 5.5 d61eac0a01 ROADMAP: P9's server side is done
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:35:24 +02:00
thepraandClaude Opus 5.5 b2570eab38 Statistics counters survive two flushes at once
Two ledger flushes that upsert the same new day (or touch the same new
instance) at once made one of them fail on the unique key, losing its
counts; MongoDB retries that only for single-document updates. The loser
now tries again and adds to the document the winner made. Seen as a rare
failure of LedgerOverHttpTests in full runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:35:10 +02:00
thepraandClaude Opus 5.5 71ca134599 The directory's pages are cut the same way every time
Many personas share a published day, and a page cut through such a day could
take a different few of them each time, skipping some between two pages: the
cut now takes the whole day at its edge. The test reads the directory page by
page until it finds its persona and checks each page's order, since other
tests make personas between two pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:35:10 +02:00
thepraandClaude Opus 5.5 d4226a289c Posts found by their words
/api/v2/search now searches statuses by words for a signed-in persona, in
Mastodon's scope: what it wrote, boosted, favourited, bookmarked or was
named in, and public posts of authors who let themselves be indexed
(indexable, off by default). Newest first, only what the persona may see,
through a text index over the posts' words in every language alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:16:40 +02:00
thepraandClaude Opus 5.5 83d4148de1 Trends and the directory, from what PrivaPub holds
trends/tags, trends/statuses and trends/links replace the stubs. Hashtags
and links rise with the number of different authors using them today
against the week before, two at least; posts with their favourites, boosts
and replies, halving every twelve hours. Only public posts PrivaPub already
holds count, nothing behind a content warning, and a post trends only if
its author is discoverable. Computed at most every ten minutes.

The directory lists discoverable accounts by their latest public post, or
by the day they say they joined: a persona's published day, never its
creation, with ties broken by a hash of the name, so personas made together
are not told apart by their order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:14:56 +02:00
thepraandClaude Opus 5.5 5f72435bfb Town: Misskey's renote count outlives an undone boost (G-0005)
decePub's e2e boost on Misskey undoes its boost and finds the count still
up: Misskey deletes the renote but its NoteDeleteService lowers only
repliesCount. Recorded as a peer gap, so the cell is a known failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:10:09 +02:00
thepraandClaude Opus 5.5 7bfd7ce285 Followed hashtags, and a tag's week of use
tags/:name shows the tag's last seven days in public posts PrivaPub holds
(uses and authors, as Mastodon gives them) and whether the persona follows
it; follow, unfollow and followed_tags replace the stubs. A public post that
is neither a boost nor a reply comes, as it arrives, to the homes of those
following one of its tags. Nothing is fetched for a followed tag and no
other server hears of it. Posts are indexed by tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 03:04:27 +02:00
thepraandClaude Opus 5.5 864dae4469 A boost reaches the author's server once
A boost and its undo went to the author's personal inbox and to the shared
inbox of its server's followers: two copies at once, which Misskey counted
twice as it processed them and undid once (seen in decePub's e2e boost on
Misskey). The author's server now gets them through its shared inbox, as
Mastodon sends them; a like still goes to the author's own inbox.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 02:57:33 +02:00
thepraandClaude Opus 5.5 cafd2e6db3 Scheduled posts: written now, published at their time
POST /api/v1/statuses with scheduled_at no longer posts at once: the post is
kept as asked (at least five minutes ahead; 300 waiting, 25 a day, as
Mastodon allows), its media kept from the janitor, and a PublishScheduled job
publishes it at its time as the persona. scheduled_statuses lists, moves and
drops them; a moved post's old job finds it not due. Idempotency-Key holds
for scheduling too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 02:55:17 +02:00
thepraandClaude Opus 5.5 73c362de96 Town: Pleroma joins, and a rerun finds what it made
Pleroma 2.10.2 runs in the pasture from its OTP release (libvips, the
dedupe question, and hackney pointed at the system CA bundle); its driver is
Akkoma's on its own container and database. The pair passes 241 cells, after
its first run showed Pleroma dropping Follows it could not verify yet.

The PrivaPub driver finds the group an earlier run made instead of failing
on the name; the seeder posts a quote or reply of a post from its own round
after that post.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 02:53:58 +02:00
thepraandClaude Opus 5.5 5b3e456e09 Filters: words and posts a persona would rather not see
Mastodon's v2 filters replace the empty stubs: a filter's title, contexts,
action (warn, hide, blur) and expiry, its keywords (whole words or not,
taken as JSON objects, listed or numbered form fields, with id and _destroy
on update) and its statuses, each with their own endpoints; the v1 API is
the same filters seen keyword by keyword. Every status a persona reads
carries the filters it matches in `filtered` (a boost as what it boosts),
matched as Mastodon matches: warning, title, text, poll options and media
descriptions. Clients apply context and action. Filters never federate, and
go with a deleted persona.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 02:48:49 +02:00
thepraandClaude Opus 5.5 b710493701 A boost or a favourite given again is sent again
Every boost of a post was announce-{post}, and every favourite like-{post}:
after an Undo, giving it again made the same activity id, which the
delivery queue had already delivered and dropped (and a server that
remembers ids would ignore). Found by decePub's e2e actions against the
pasture, which unboost and boost again. Now each boost and favourite is an
activity of its own, as Mastodon's are, and an Undo names the one it ends;
a concurrent second boost is removed after saving. An edit's Update is
named to the millisecond, a quote approval's to the tick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 02:48:35 +02:00
thepraandClaude Opus 5.5 ce473f2741 Follow requests asked again, and followed accounts found by name
A server can take a Follow with 202 and drop it afterwards, as Pleroma does
while it cannot fetch our actor; the request then stayed pending for good.
Following again now sends an unanswered request once more, the same
activity, at most once an hour (a delivery's `again` key).

accounts/search takes following=true: only accounts the persona follows,
by the start of their name, display name or server, never resolved; a
client fills a list with it. "already take" becomes "already taken".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 02:40:40 +02:00
thepraandClaude Opus 5.5 02a05d7a9a Town: Iceshrimp.NET joins, and its mentions go nowhere new
Iceshrimp.NET 2026.1.2-beta runs in the pasture with AuthorizedFetch on;
its driver registers through the native API and takes Mastodon tokens from
its OAuth form. The pair passes 225 cells. A new Iceshrimp note reaches its
author's followers only, never an account it mentions or answers, until it
is edited (G-0004, peer): delivery cells now say when a server holds a post
only because it was addressed.

The checker credits a seeder's fetch to the object fetched, not to the reply
that needed it; the seeder skips the wait for an accept whose follow was
already accepted; the PrivaPub driver signs its roots in again when a run
reuses saved sessions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 01:54:36 +02:00
thepraandClaude Opus 5.5 0614bdcf18 Lists: a persona's followed accounts, read apart
Mastodon's lists replace the empty stubs: CRUD, members (only accounts the
persona follows; a follow that ends takes its memberships with it),
accounts/:id/lists, and timelines/list/:id from the persona's home entries
with the replies policy (followed, list, none; self-replies and replies to
the persona always). An exclusive list's members stay out of home. Lists
never federate, and go with a deleted persona.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 01:54:30 +02:00
thepraandClaude Opus 5.5 4b6d261739 Town: decePub's cells in the report, a Hollo pair, kinder waits
The report merges decePub's end-to-end cells (out/client.jsonl, the
latest result of each test) into the matrix under the observer
"decepub", and names its run without the workstation's paths. A second
spec, hollo-pair, seeds PrivaPub and Hollo alone (258 of 261 checks pass;
the three left found the emoji variation selector bug fixed in
PrivaPub), and Hollo accounts may join circles. The seeder waits up to
90 s for a follow request, two of a sender's retries after a refused
first delivery (Hollo answers its first concurrent deliveries 500 while
it creates its tables). town.sh drive prints its JSON unescaped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 01:24:17 +02:00
thepraandClaude Opus 5.5 f70ec59359 One emoji, one reaction: variation selectors made canonical
A heart reaction arrived as ❤ from some servers and as ❤️ (with the
emoji variation selector) from others, and a persona's own reaction kept
whatever it was given: the same emoji was two reactions, counted apart.
The selector is now dropped and put back only on a symbol that shows as
text without it (U+2000 to U+2BFF), so every heart is ❤️ and every 🔥
is 🔥. Found by the town's Hollo pair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 00:58:48 +02:00
thepraandClaude Opus 5.5 ed08f80f05 Votes a community relays count
Lemmy, PieFed and Mbin relay their members' votes, and the undoing of
them, inside the community's Announce; PrivaPub dropped them all as
unsupported (G-0003, the Lemmy scenario's expected failures). A relayed
Like, Dislike or Undo from a community a persona follows is now handled
as if its actor had sent it. The community vouches for what accounts on
its own server do and for what is done to its own posts, as Lemmy trusts
it (refetching every vote would not scale); a vote from elsewhere on
anything else is believed only once fetched from its own origin.
Moderation relayed the same way is still open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 00:55:04 +02:00
thepraandClaude Opus 5.5 2873344690 The town: a fake community across the pasture, checked for coherence
tools/pasture/town/ (run through tools/pasture/town.sh) seeds a fake
community across every running peer and checks that all of them, and
PrivaPub, agree on what happened:

- drivers per platform on four dialect bases (Mastodon API, Misskey API,
  Lemmy API, PrivaPub with /clientapi), each with a selftest against
  its own server; what a server holds is read from its database, never
  by making it fetch;
- a deterministic generator (specs/village.json: 23 accounts on seven
  servers, roots with several personas, circles and communities, a
  cross-server follow graph, posts of every kind and visibility, reply
  rounds, likes, boosts, reactions, votes, edits, deletes, blocks,
  mutes and a report) and a seeder that keeps a ledger of what happened;
- a sweep that expects delivery and confinement per server, what each
  account sees, counts, threads, edits, deletes, follows and privacy
  rows (sibling keys, published days, canary root credentials in every
  peer's database, located posts that never leave), with what the peers
  do on purpose modelled (Misskey drops orphan replies, Lemmy keeps only
  community content, edits go to the post's own audience);
- known gaps (gaps.json) turn failures into xfail and passes into xpass;
  a self-contained report.html, and docs/INTEROP-BACKLOG.md.

The pasture moves to a public-looking subnet (peers with no private
address switch can join), takes PASTURE_PORT when 6971 is in use, adds
peers to a running pasture (run.sh add, Caddy recreated with its CA
kept), removes its volumes on down, writes every scenario check to
out/scenarios.jsonl, serves decePub as decepub.test for its e2e tests,
lifts GoToSocial's and Lemmy's own rate limits, trusts Caddy in
Mastodon (TRUSTED_PROXY_IP) and gains Hollo (Fedify), whose one login
owning several accounts is the nearest peer to PrivaPub's personas.

The first village found the four PrivaPub bugs fixed in the commits
before this one; the second run, on the fixed server, passes 2319 checks
with 11 failures left, all between peers or from Lemmy's send worker,
which the seeder now warms up first. ROADMAP records the owner's
decisions of 2026-10-04 (the town, and P9 back from the cut list).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 00:32:54 +02:00
thepraandClaude Opus 5.5 d6131af289 A public reply names the author it answers in cc
A reply whose text had lost the @name of the author it answers (decePub
prefills it, a reader may delete it) was addressed to nobody on the
author's server. PrivaPub delivered it to the author's inbox anyway, but
GoToSocial keeps only what is addressed to someone there, so the reply
never appeared under the post. A public or unlisted reply to a remote
post now names its parent's author in cc, as Pleroma does; the reply
already says whom it answers, so nothing new is revealed. The parent
author's actor id is kept on the reply (InReplyToActorURI).

Found by decePub's end-to-end tests on the town.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 00:22:48 +02:00
thepraandClaude Opus 5.5 6b55b5c35c A status asked for by id is shown, and remote warnings are their own
GET /api/v1/statuses/:id applied the viewer's own blocks and mutes and
answered 200 with null for a post of an author the viewer had blocked;
decePub's thread page then showed nothing. As in Mastodon, a status asked
for by id (or acted on) is shown whatever the viewer blocked or muted,
and only lists leave such posts out; when nothing can be shown the
answer is 404, never null, and search leaves it out.

A remote post marked sensitive with an empty summary (Akkoma's sensitive
media, Lemmy's NSFW) got an invented "Content warning" that hid its
words. A remote post now keeps its own summary: sensitive without one
hides the media only. A local post warned without words still gets the
default warning.

Both found by decePub's end-to-end tests on the town.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 00:09:51 +02:00
thepraandClaude Opus 5.5 d405269526 A remote account's Block of a persona is enforced
Inbound Block was dropped as an unknown type (the pasture's last
Mastodon expected failure, G-0002). Now, as Mastodon does it: the
follows between the blocker and the persona end here too (nothing is
sent back; the blocker already ended its side), the blocker's posts and
notifications are hidden from the persona and kept out of its home, the
persona's posts are no longer addressed to the blocker by mention or
reply, and the relationship says blocked_by. Undo{Block} lifts it. The
block is kept in BlockedBy, unique per persona and blocker.

The Mastodon scenario checks blocked_by instead of expecting a failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 23:49:15 +02:00
thepraandClaude Opus 5.5 08acdc0503 Personas a remote post is addressed to see it, named or not
A remote post decided who here may see it by its Mention tags alone. A
followers-only post addressed in to or cc to a persona without naming
it (Akkoma's to[], or a GoToSocial edit that took the @name out while
the post stayed addressed to the persona) was hidden from that persona.
Such personas are now kept as silent mentions, as Mastodon does: they
see the post and it reaches their home, and no list shows them as
mentioned. An edit never narrows who a post was for.

The GoToSocial note that showed it is the first captured fixture
(Fixtures/gotosocial), and parses with its lone tag, to and cc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 23:42:23 +02:00
thepraandClaude Opus 5.5 b9286c2c6e Deliveries refused with 409 or 422 get three tries
Mastodon answers 422 when two first contacts from one actor race to
create its account (ActiveRecord::RecordInvalid on the unique uri), and
409 while another worker holds its lock. A persona that followed two
Mastodon accounts at once had one Follow refused that way; the job died
on its first attempt and the persona waited on "requested" forever.
Both answers are now retried twice on the usual backoff before they
count as refusals.

Found by the town (a village of 23 accounts on seven servers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 23:42:23 +02:00
thepraandClaude Opus 5.5 128ff89426 Followers and circle members may like, react, vote and downvote
LikeHandler.MaySee let a remote account interact with a followers-only
post only when the post named it in to, cc or its mentions. Our own
followers-only posts are addressed to the followers collection, never to
each follower, and circle posts were not considered at all, so every
like, reaction, downvote and poll vote from a follower on a
followers-only post, and from a member on a circle post, was dropped as
"not-visible". Likes, dislikes, reactions and poll votes now ask
VisibilityPolicy.RemoteCanSee: anyone for public and unlisted posts, an
addressed account for a DM, an accepted follower or an addressed account
for followers-only, a foreign member for a circle post, and never a
server's instance actor (SignedFetchAuthorizer's alias is for refetches
only).

Found by planning the town's multi-server interaction checks (G-0001).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 23:05:22 +02:00
thepraandClaude Opus 5.5 436f7da464 CDNs found by themselves, and servers followed through time
Build / Build (push) Successful in 5m11s
Deploy / privapub.thepra.dev (push) Successful in 5m48s
PrivaPub now finds CDNs three ways, best first: the address ranges the
CDNs publish (Cloudflare, Fastly, Amazon CloudFront, Bunny, Gcore,
Imperva), downloaded daily by CdnUpdater and kept in CdnRangeSet; the
CDN's fingerprint in the responses it already gets from a server
(EdgeHintsHandler on the federation client); and the networks that carry
only a CDN. The fixed ASN list is gone; ASNs shared with plain hosting
(AWS, DataPacket) no longer hide a server. A server's Geo records the
CDN, its domain and how it was found, and weekly snapshots now keep the
city and coordinates too.

Servers through time (ServerPlaces): /instances/:host/history lists a
server's weekly snapshots, a CDN-fronted server's geo names the CDN's
domain and where the server was before it (before_cdn), and
/api/privapub/v1/cdns and /cdns/:domain group servers by CDN with week
by week who joined and who left. Owner decisions recorded in ROADMAP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 11:33:39 +02:00
thepraandClaude Opus 5.5 5d7edc4a4c ROADMAP: no size or bucket restrictions on the public statistics
The plan for the public /stargazing statistics showed per-server
activity only as order-of-magnitude buckets and only for servers
reporting at least 10 users. The owner never intended either
restriction; both are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 11:20:44 +02:00
thepraandClaude Opus 5.5 eb7552e8f3 Server locations: no user threshold
The public projection of a server's place showed only the country for
servers reporting fewer than 10 users. The owner never decided that
threshold: every located server now shows its city, coordinates (0.1°)
and network, and a CDN-fronted one still shows only its CDN, since the
address reached is the CDN's edge. Statistics:PublicCityMinUsers is
gone; the ROADMAP decision on server locations, CLAUDE.md and the
/stargazing explainer are corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 11:12:02 +02:00
thepraandClaude Opus 5.5 31d1d21d1e Pixelfed places: a remote post's own place, shown and never federated again
A Note's `location` that is a Place with coordinates (Pixelfed sends
them as strings) is kept in the new Post.Place and shown as
Status.privapub.place {name, latitude, longitude, country}; an edit
replaces it. An Event's location stays its own `places`, and nothing
renders a place back out. Closes the INTEROP P2 Pixelfed location gap
and the ROADMAP long-tail item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 10:07:06 +02:00
thepraandClaude Opus 5.5 d9fb5c582f Where a server is, on the public instance API
/api/privapub/v1/instances/:host gains `geo`, the public projection of a
server's place already decided for public server locations
(PublicGeo.Project): city, coordinates and network for servers reporting
at least 10 users and not behind a CDN, the country otherwise, the CDN's
name for a CDN-fronted one, with DB-IP's attribution. `?host[]=`
answers up to 40 servers at once, and this server describes itself:
its host's address located once a day (SelfLocation), or
Statistics:Geo:Self when the owner sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 10:03:59 +02:00
thepraandClaude Opus 5.5 2cd75176a4 One sign-in for decePubClient: the root JWT exchanged for a persona's token
The first-party client signs in on /clientapi and exchanges that JWT on
/oauth/token (RFC 8693, subject_token_type jwt, avatar_id) for one
persona's Mastodon token. Only the seeded public application `decepub`
holds the grant; RootJwtSubjectToken validates the JWT through RootJwt,
which JwtBearer now shares (signature, lifetime, ban, deletion, session
stamp). The issued token names the avatar, never the root.

Owner decision recorded in ROADMAP; it supersedes "moving decePubClient
onto the Mastodon API is out of scope".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-04 10:03:46 +02:00
thepraandClaude Opus 5.5 c7e8760ffe docs: v1.20.0 deployed and verified; everything on in production
Build / Build (push) Successful in 4m34s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 04:27:40 +02:00
thepraandClaude Opus 5.5 ad8d353f9a docs: FEDERATION.md tested against six peers with signed fetches, the shipped FEPs; v1.19.1 and v1.20.0 in ROADMAP
Build / Build (push) Successful in 5m14s
Deploy / privapub.thepra.dev (push) Successful in 5m25s
- FEDERATION.md:
  - It said only GoToSocial was tested live. It now lists all six pasture peers, run with signed fetches required as
    production runs, and what is checked both ways.
  - FEP-044f, FEP-9967, FEP-c0e0 and FEP-5feb moved from planned to supported.
  - indexable and discoverable are described as the settings they are.
- ROADMAP: v1.19.1 deployed and verified, v1.20.0 (phase 4).
- Pasture, Akkoma: its like and boost count gets two minutes. In the final clean pass, all six peers with SecureMode on
  gave 245 passed, 1 failed, 4 expected. The failure was this count, which Akkoma's job queue was slow to update
  under the load of six peers; two reruns gave 44/44.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 04:16:16 +02:00
thepraandClaude Opus 5.5 e2f61ede56 Everything on, phase 4b: ids that resolve, a hashtag page, grouped notifications, remote accounts' real counts
- A community's announces resolve, as FEDERATION.md and ROADMAP always said Announce ids do. GroupDistributor keeps
  each one it sends (GroupAnnouncement, unique by id). /grunts serves it while the post it carries is shown and 410
  after, and also serves the announce-{postId} ids the group outbox lists, which now keep a stable `published`
  instead of the time of the fetch.
- A persona's boost points at the boosted post: its `url` in the Mastodon API is the boosted post's page, and a browser
  following the boost's id is redirected there instead of getting JSON.
- /tags/{tag}, where every Hashtag link we send points, is now a public page of this server's public posts with that
  tag, with the same strict CSP and noindex as the profile pages.
- Grouped notifications (/api/v2/notifications, its unread count, a group, its accounts and dismiss). We advertise
  api_versions.mastodon = 7 so clients show quotes, and clients that trust it call these; they answered 404. Likes and
  boosts of one post group together, as do follows within an hour. The version string stays 4.2.0 until streaming
  and Web Push exist.
- A remote account's follower, following and post counts are what its server publishes. AccountCountsJob reads its
  collections' totalItems from its own origin, signed by the instance actor, at most daily and only after the account
  was fetched, never when someone looks. They used to be 0.
- The other ids that do not resolve are documented as such: Update, Delete, EmojiReact, QuoteRequest and its answers,
  Flag, Ignore and poll votes.

676 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:56:18 +02:00
thepraandClaude Opus 5.5 8f25bf056d Everything on, phase 4a: one answer everywhere for counts, search, collections and the instance API
Owner decision 2026-10-04: fix the mismatches and every other mismatch of the same kind.

- One counting rule (Domain/Privacy/Counted), Mastodon's. It is used for a persona's statuses_count, its outbox
  totalItems, NodeInfo localPosts and the instance status_count, which used to count four different things. It
  counts every post that is neither deleted nor a DM, boosts included, and circle and located posts too (owner
  decision). A group's count includes its remote members' posts.
- Users. Personas of banned or deleted roots no longer count, and are not found in search. NodeInfo now gives
  activeMonth and activeHalfyear, and the v2 instance gives active_month instead of a constant 0.
- replies_count counts only public and unlisted replies, so it no longer tells anyone that a private reply exists.
  Migration _012 recounts it.
- A remote account that deletes itself takes everything out of every count (GoneActors): its likes, downvotes,
  reactions and poll votes go and their counters come back, as do its boosts', replies' and quotes' counts, and its
  notifications. Lookups, account lists, search and favourited_by no longer show it. Migration _012 applies this to
  accounts already gone.
- Deleting a post also deletes its pins and the local boosts of it.
- /stalking gives the same total as following_count. Members are still never listed, and hide_collections is now
  always true, since the setting never did anything.
- Joining a community by invitation is following it, so /flock and /groupies agree; leaving unfollows.
- Search. Anyone may search, as on Mastodon; resolve and offset need a sign-in, offset pages, and deleted accounts
  are never found.
- notifications/unread_count counts what the list shows, and the owner's follower and following lists page with
  Link.
- The instance API advertises what is enforced:
  - max_characters, now enforced with a 422;
  - max_pinned_statuses = MaxPins;
  - the media types and limits MediaService and MediaOptions accept;
  - PollService's limits;
  - the configured languages;
  - no streaming URL until streaming exists.

  domain_count counts the servers we have exchanged with; which ones stays unpublished (peers is empty).
- Routes Mastodon answers now answer instead of 404:
  - directory, tags/{name}, timelines/link and identity_proofs;
  - instance/languages, translation_languages, domain_blocks and privacy_policy;
  - the v1 and v2 notification policy, and notification requests.

Also, from phase 3: a recovered password ends /clientapi sessions through a per-root SessionStamp claim instead of
comparing the JWT's whole-second nbf with the change time. That comparison let a token issued in the same second
survive, which made a test flaky.

671 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:48:40 +02:00
thepraandClaude Opus 5.5 123ba39520 A post whose author cannot be shown is left out of a list, never a 500
Build / Build (push) Successful in 4m19s
Deploy / privapub.thepra.dev (push) Successful in 4m50s
The v1.19.0 deploy's tests failed where build.yml's passed. A test stored a public remote post with no author id, and
when the public timeline test ran after it, MastodonMapper.Statuses looked that null up in its accounts and answered
500. The mapper now skips such a post, and the test stores a real author.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:46:48 +02:00
thepraandClaude Opus 5.5 f6dbf71964 Everything on, phase 2 completed: SecureMode on in production, checked by the deploy
Build / Build (push) Successful in 5m5s
Deploy / privapub.thepra.dev (push) Failing after 4m39s
Owner decision 2026-10-04: SecureMode on once the pasture passes with it.

- Both clean pasture passes were run over all six peers:
  - normally: 246 passed, 0 failed;
  - with Federation__SecureMode=true: every federation check passed. The only failures were four checks expecting an
    unsigned GET to get 404 or 410 where SecureMode answers 401. Those checks now go through `unserved` and
    `gone_unsigned` (lib/interop.sh), which expect 401 when SecureMode is on.
- Circle posts now reach their member on GoToSocial and Mastodon, and survive Mastodon's signed refetch, as does a
  followers-only post. The GoToSocial expected failure is gone.
- appsettings.Production.json turns SecureMode on.
- The deploy now checks that an unsigned GET of @thepra answers 401 and that a browser is redirected. It reads
  @thepra's discoverability through the Mastodon API, since the actor is no longer readable unsigned.
- docs/INTEROP.md (Mastodon, GoToSocial), CLAUDE.md and ROADMAP updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:34:48 +02:00
thepraandClaude Opus 5.5 8c2eba6cbb Everything on, phase 3: sign-in and recovery tell nothing, recovered passwords end sessions, deleted roots are gone everywhere
Owner decision 2026-10-04: fix the account privacy findings.

- Sign-in. Every failure answers "That username and password do not match." after the same work: an unknown login
  is hashed against a decoy, and the comparison is constant-time. "Banned" is told only to someone who gave the right
  password. This covers /clientapi/user/login, /invitation/login and /oauth/login.
- Recovery.
  - Every request answers the same sentence and queues a SendRecovery job, whether or not the account exists or has an
    email. The lookup, the code and SMTP move to RecoveryJob, so neither the answer nor its timing says anything.
  - Codes are kept only as a SHA-256 hash, for one hour. Migration _011 drops the plaintext ones, which never expired.
  - A recovered password ends every session of the root. RootSessions sets CredentialsChangedAt, which JwtEvents
    checks against the JWT's issue time, now stamped as nbf, and revokes each persona's OAuth tokens and authorizations.
- Deleting a root (RootRemoval: the admin route, or the restored self-delete at /clientapi/user/delete, which asks for
  the password).
  - Its sessions end.
  - Each persona and each group it owns sends Delete{Actor} to its followers, its members and the accounts it follows.
  - The personas' posts are emptied.
  - /peasants/{name} answers 410 with a Tombstone (formerType Person or Group), as do its inbox and WebFinger, through
    LocalActorService.Gone. The names stay reserved.
  - The root keeps only a unique `deleted-{id}` name; the second deletion on an instance used to collide on
    "Deleted user".

Also, from phase 2's pasture: GoToSocial files a circle post like a DM and shows it only to accounts it mentions. Each
member's copy, and a member's refetch, now also mentions that member silently. The GoToSocial scenario checks circle
posts in conversations, like DMs, and they pass there now, as on Mastodon.

657 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:16:59 +02:00
thepraandClaude Opus 5.5 fcd35f5043 Everything on, phase 2: circle posts for everyone, private posts on signed refetch, browsers past SecureMode
Circles (owner decision 2026-10-04: fix them for compatibility):
- Mastodon 4.7 and GoToSocial drop a post that names none of their accounts, and a circle post named only the circle
  and its /flock. OutboxPublisher.Publish now sends each member a copy that also names that member in `cc`, on the
  activity and on the object, and names no other member. The Create, every Update (edit, poll, quote approval, policy,
  through the new PublishUpdate) and the Delete (StatusService.Remove now uses Publish) all go that way.
- UpdateOf renders with the post's group, so an Update keeps a circle post's `audience` and a community post's `Page`
  and title.
- A reply to a circle post stays in the circle, whichever client wrote it.
- A circle post can no longer quote a post that needs permission: asking would show the circle post to its author.

Posts that are not public, on refetch (SignedFetchAuthorizer.MayRead):
- Followers-only, direct and circle posts are served to a signed request from someone they were for, or from the
  instance actor of a server where one of them lives. That is a follower or an addressed account, an addressed
  account, or a member. Everyone else still gets 404.
- Once deleted they answer those readers 410. Mastodon deletes its copy when a refetch answers 404.
- A circle refetch names the requesting member, or the members on the requesting server, as the delivered copy did.
- /grunts/create-{id} serves the same.
- /peasants/{name}/whispers/{id}, a DM's `context`, was never routed. It is now the conversation's posts, for its
  participants only.

SecureMode lets browsers through to the redirect to the public page, instead of answering them 401.

653 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:01:14 +02:00
thepraandClaude Opus 5.5 5f56681c01 Everything on, phase 1: geolocation fetches itself, the deploy signs in as @thepra, the crawler is on, sign-up by invitation
Build / Build (push) Successful in 5m1s
Deploy / privapub.thepra.dev (push) Successful in 5m39s
Owner decisions (2026-10-04, recorded in docs/ROADMAP.md): production runs everything that is built, and nothing waits
on a person running a command.

- Geolocation updates itself. GeoUpdater, a hosted service, checks daily whether each DB-IP Lite database was built this
  month. If not, it fetches this month's, or last month's early in the month. It installs a file only once it opens as
  the right kind of database, then swaps it in atomically, and the locator reloads at once. Lookups now run under the
  lock, so a reload can no longer dispose a reader mid-lookup. The systemd timer, its script and their setup.sh lines
  are gone: the root step they needed never happened, and none is needed now. /stargazing names the database in use.
- The admin CLI runs after the app is built, with every service and nothing started.
  - `create-root <login> [--admin]` takes the password on stdin; it is how the first login is made while sign-up is
    closed.
  - `smoke <persona>` keeps the root `deploy-smoke` and an undiscoverable persona, and gives the root a new password
    on every run.
- The deploy signs in as @thepra. It runs the CLI, gets a token through the real OAuth flow (tools/smoke/oauth.sh,
  moved out of the pasture's privapub_token, which now uses it), checks the signed-in API and that @thepra is
  undiscoverable, then revokes the token. PRIVAPUB_SMOKE_TOKEN is gone.
- The deploy also fails when:
  - NodeInfo and the instance API disagree about registrations;
  - /stargazing does not say the crawler is on;
  - the geolocation databases are missing or more than 40 days old.
- The crawler is on in production, seeded with ten large servers of different kinds. FEDERATION.md now describes it
  and how to opt out.
- One registrations switch (Registrations:Mode, default Invitations; Open in tests and the pasture). It is read by
  open sign-up (403 when closed), NodeInfo `openRegistrations`, and v1 and v2 of the instance API, so they can no longer
  disagree. Before, NodeInfo said open and the instance API said closed. Group invitations always work, so
  invites_enabled is true.
- A persona edit through /clientapi no longer resets what the Mastodon API set (discoverable, locked, quote policy…):
  the theme is merged into the settings instead of replacing them.

650 tests pass. The deploy's smoke step was rehearsed against the pasture's PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 02:37:38 +02:00
thepraandClaude Opus 5.5 cd5eb25948 docs: v1.17.1 deployed and verified; what the T/M track still needs from the owner
Build / Build (push) Successful in 4m8s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 15:35:14 +02:00
thepraandClaude Opus 5.5 ab526ed291 T12, T13: Sharkey and Akkoma in the pasture, and two bugs Akkoma found
Build / Build (push) Successful in 4m51s
Deploy / privapub.thepra.dev (push) Successful in 5m2s
Sharkey 2025.4.7 is the Misskey peer under another name, image, database, Redis db and
home. Its scenario is Misskey's plus edits both ways and the FEP-e232 quote tag: 40 checks
pass.

Akkoma 3.20.1 publishes no image, so tools/pasture/images/akkoma installs its OTP
release, pinned by checksum, and appends Caddy's CA to the CA bundles the release ships.
Its scenario has 44 checks, which pass three runs in a row:
- follows, posts, CW, followers-only posts and the published time;
- replies, likes, boosts and their undos;
- EmojiReact both ways and withdrawn;
- DMs, polls, quotes, media, edits with history and deletes, both ways;
- unfollow, block, unblock and statistics.

The two bugs, both fixed:
- Followers-only posts arrived as DMs on Pleroma and Akkoma. They call a post private
  only if an address in `to` contains "/followers" or its cc is not empty. Ours is
  /groupies, and a post mentioning nobody had an empty cc. The followers collection is now
  named in cc as well, which tells nobody anything new.
- Akkoma's open polls showed as ended and refused votes. Akkoma carries an open poll's
  end in `closed` and sends no `endTime`. A `closed` in the future is now read as the end.

Neither of these is a PrivaPub bug:
- Akkoma's Linkify never takes @user@host.test for a mention, so its DM addresses alice
  with to[].
- Its API never reports a remote blocker as blocked_by, so the block is read from its
  database.

Also in this commit:
- The rate limits are configuration (RateLimits: AccountsPerMinute, InboxBurst,
  InboxPerTenSeconds), with the old values as defaults. The pasture raises the accounts
  limit, which back-to-back runs from one address had hit.
- A new Lemmy never sends what it queued for a server before it started that server's
  send worker, so the scenario waits for the worker before its first follow.

All six peers in one clean pass: GoToSocial 54 (+1 expected), Mastodon 49 (+2), Misskey
35, Sharkey 40, Akkoma 44, and Lemmy 20 (+3) once the worker wait was added. 642 tests
pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 15:24:08 +02:00
thepraandClaude Opus 5.5 aa7e43a61e T14: Lemmy 1.0 in the pasture
peers/lemmy.sh runs the Lemmy 1.0.0-beta.2 backend on the shared Postgres. It trusts
Caddy's CA through SSL_CERT_FILE and reaches the pasture through
DANGER_FEDERATION_ALLOW_LOCAL_IP. Lemmy 0.19 cannot join: its rustls trusts only its
bundled roots. LEMMY_LOG sets RUST_LOG.

scenarios/lemmy.sh drives Lemmy through its v4 API. 20 checks pass, three runs in a row:
- communities both ways;
- a titled thread each way, and alice's mention of a Lemmy community becoming a thread
  there;
- the Lemmy community's announce reaching alice's home;
- comments both ways and alice's like as an upvote;
- private messages both ways;
- statistics.

Two expected failures: votes, which Lemmy sends only through the community as
Announce{Like|Dislike}, and a moderator's removal. Both belong to P7.

What it showed, in docs/INTEROP.md:
- Lemmy 1.0 keeps its user's thread in a remote community pending until the community
  announces it back. It clears the flag before answering that echo 400 ("Object is not
  remote"). Leaving the author's server out of the Announce, as tried here, left every
  such thread pending, so GroupDistributor now says why the echo stays.
- Every bare Announce{object} is answered 400, as Lemmy answers its own compatibility
  Announce(Page).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 14:24:15 +02:00
thepraandClaude Opus 5.5 e29da1b47a T12: Misskey 2026.10 in the pasture
peers/misskey.sh runs Misskey on the shared Postgres and Redis. Its config is generated with
the pasture's private networks allowed and a setup password, it trusts Caddy's CA through
NODE_EXTRA_CA_CERTS, and the first account it makes is the scenario's user. A new Misskey
federates with nobody, so the setup also turns federation on.

scenarios/misskey.sh adds 35 checks, all passing, as listed in docs/INTEROP.md. They cover
posts, CW, MFM source, replies, unicode reactions both ways and their withdrawal, likes as
reactions, legacy quotes both ways, polls, specified notes, media, deletes, unfollow,
block and statistics. MISSKEY_NAME and MISSKEY_IMAGE are there so Sharkey can reuse the peer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 13:29:17 +02:00
thepraandClaude Opus 5.5 d4e9acdb79 T10: what GoToSocial had not yet been asked, and quick edits that were lost
The GoToSocial scenario gains 17 checks:
- alice's reply threads under gtsuser's post;
- gtsuser's edit arrives with edited_at and two history entries;
- unlike and unboost both ways;
- images with alt text both ways, theirs through our proxy;
- gtsuser follows a PrivaPub community and its announce brings the post;
- gtsuser's request to join a circle waits for the owner and is approved, and the
  circle post is never served unsigned;
- a locked persona holds gtsuser's follow, rejects it, then authorizes it;
- the deploy's Mastodon smoke check passes, signed in.

54 checks passed, three runs in a row. The circle post reaching gtsuser is an expected
failure: GoToSocial keeps no post addressed only to a collection it does not know.

It found a bug. An edit made within the second the post was published carries GoToSocial's
whole-second updated == published, and IsEdit wanted strictly newer, so the edit was taken
as a refresh and lost. A first edit now also counts when it is no older and the text,
warning or title actually changed. A bare refresh still never makes a revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 12:54:01 +02:00
thepraandClaude Opus 5.5 5f31dbd521 docs: what v1.17.0 ships and what the T/M track still has open
Build / Build (push) Successful in 4m49s
Deploy / privapub.thepra.dev (push) Successful in 5m8s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 12:30:18 +02:00
thepraandClaude Opus 5.5 e0f2eb7da7 T11: Mastodon 4.7.3 in the pasture
peers/mastodon.sh runs Mastodon's web and sidekiq containers on a shared Postgres and Redis
(peers/shared.sh). They trust Caddy's CA through SSL_CERT_FILE and reach private addresses
through ALLOWED_PRIVATE_ADDRESSES. Its users and tokens come from tootctl and rails runner.

scenarios/mastodon.sh adds 49 checks, as listed in docs/INTEROP.md: follows, posts,
replies, likes and boosts with undos, DMs, polls, FEP-044f quotes both ways, media through
the proxy, edits, deletes, locked follows, a circle request, reports, blocks and statistics.
Two are expected failures:
- inbound Block (P7);
- circle posts. Mastodon 4.7 loses the recipient of deliveries to its numeric
  /ap/users/<id>/inbox and then drops a post that names no local account. The fix on our
  side changes what a circle reveals, so it waits for the owner.

GoToSocial and Mastodon together: 86 passed, 0 failed.

The pasture now copies Caddy's root certificate reliably, readable by the peers, and
rebuilds the bundle each time. The CA directory is mounted shared (:z), because a private
:Z label locks out every container but the last. pfetch reaches PrivaPub's own https URIs
through Caddy. PRIVAPUB_ENV passes settings to PrivaPub, which scenarios/crawler.sh uses to
check the opt-in crawler against Mastodon: it visits, describes and reads the peers list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 12:30:05 +02:00
thepraandClaude Opus 5.5 d0a13d9cc2 M11: the planner accepts reserved test names for seeds on a test network, as the crawler does for peers
Found by the pasture's crawler scenario: a mastodon.test seed was dropped before it was ever planned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 12:30:05 +02:00
thepraandClaude Opus 5.5 4f5df743ae M11: the opt-in crawler, PrivaPub-Stargazer, and /stargazing
Off by default (Statistics:Crawler:Enabled), as the owner decided. When it is on:
- CrawlPlan runs hourly, from StatisticsSchedule. It inserts the configured seeds and
  queues up to HostsPerHour servers whose last visit is older than RevisitDays, are not
  paused by the breaker and are not domain-blocked, spread across the hour.
- CrawlInstance visits one server at a time as
  "PrivaPub-Stargazer/<ref> (+<base>/stargazing)":
  - it reads robots.txt (RFC 9309: its own group first, then PrivaPub, then *; longest
    rule wins; a 4xx allows everything; a 5xx or no answer keeps it out);
  - it describes servers that only crawling ever found, through
    InstanceDescriber.Describe with robots.txt as the path filter;
  - it reads /api/v1/instance/peers through the new GetStringArray, which keeps what it
    read from the first 1 MB instead of refusing a large list;
  - it adds the names a server could ever be reached at as "crawled": DNS only,
    punycode, no addresses, ports or hidden services, and the reserved test names only on
    a test network. Never more than MaxNewHostsPerCrawl per visit or MaxHosts in all, and
    never over a touched server.
- It reads nothing but robots.txt, NodeInfo, the instance API and the peers list.
  IFederationHttp.GetText serves robots.txt, and HttpScope.Crawl carries the
  User-Agent.
- /stargazing explains all this and how to keep the crawler out, says whether it is on,
  and credits DB-IP. GET /clientapi/admin/statistics/crawler shows the frontier.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 12:19:29 +02:00
thepraandClaude Opus 5.5 c8a305ae92 M8: every server we touch is described, located and snapshotted weekly
- Touches: the ledger marks a server as touched when it sends us a verified activity, when
  we exchange activities with it, or when we read its actors, keys, objects or WebFinger.
  It upserts RemoteInstance.Seen, FirstSeenAt and LastSeenAt at most hourly per server, and
  queues one DescribeInstance a week with the same dedupe key ObjectRecords uses. Suspended
  servers and pages behind link previews are never described. Migration _010 marks the
  servers already known as touched, with their dates.
- InstanceDescriber.Describe(host, crawled, allowed) reads:
  - NodeInfo 2.2/2.1/2.0, now with its published user counts, posts, comments,
    description, languages and schema version;
  - for software with a Mastodon API, /api/v2/instance falling back to v1: title,
    languages, registration mode, character limit, API version, source URL.

  It never keeps a contact as a field; the raw document is kept for the admin only. It
  locates the server from the address our connection reached (DB-IP Lite city and ASN, the
  CDN named when fronted) and writes a RemoteInstanceSnapshot per ISO week, unreachable
  weeks included. A crawled server is upserted as crawled only on insert, so it never
  downgrades a touched one, and robots.txt can deny any path.
- PublicGeo.Project is the only public form of a location: a CDN-fronted server shows its
  CDN only, a server reporting at least ten users shows its city, coordinates and network,
  any other only its country.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 12:07:57 +02:00
thepraandClaude Opus 5.5 fc5bb9511f T6: the Mastodon API over HTTP
88 integration tests drive the Mastodon client API through the whole server
(PrivaPubHost), with remote actors on an in-process Peer and deliveries read
from the job queue. Helpers live in Support/Host/MastodonHelpers.cs.

Coverage:
- Accounts: verify_credentials (no root id or login name); update_credentials
  with indexed and array fields_attributes (form and JSON), source[*],
  quote_policy, locked/bot, avatar and header uploads resized and stripped of
  EXIF and XMP; lookup (local, @domain, remote; a circle, the instance actor
  and a circle's id answer 404); search with and without resolve (only a
  signed-in persona resolves, the Peer is untouched otherwise), by post and
  actor address, hashtags, undiscoverable personas; account statuses with
  pinned, exclude_replies, exclude_reblogs, only_media, tagged and Link paging
  both ways; followers-only posts for followers (local and remote authors);
  community accounts; followers/following only to their owner; follow (open,
  locked, remote Follow delivery), unfollow and Undo; follow requests from
  local and remote followers answered with the original Follow;
  remove_from_followers; blocks with Reject and Block/Undo deliveries; mutes
  with duration and the notifications choice, never federated; domain blocks;
  relationships with junk ids; a banned login's tokens; reports forwarded as a
  Flag from the instance actor only; account stub routes.
- Statuses: each visibility's to/cc as delivered; CW as summary; replies to
  local and remote posts (mention, inReplyTo, the author's inbox); polls and
  votes (local, and remote votes only to the author without published); media
  attached only by its owner; quotes, the quotes list and revocation; edit
  history, source and the Update delivery; delete for redraft, the 410
  Tombstone and the Delete delivery; favourite/reblog counts with Like,
  Announce and their Undos; favourited_by, reblogged_by; bookmarks; pins;
  interaction_policy matching canQuote in the note and in the Update;
  strangers get 404 for followers-only and direct posts; a located post is
  unreachable by id for anyone else on every route; statuses?id[];
  Idempotency-Key; scopes; deleting a reblog.
- Timelines: home paging with max_id, since_id and min_id; public local and
  remote; tag (anonymous); list stub; favourites; conversations and read;
  markers; notifications with types[], exclude_types[], account_id, paging,
  get, dismiss, clear and unread_count.
- Instance: v1 and v2 (4.2.0 (compatible; PrivaPub)), peers, activity, rules,
  extended_description, apps and every stub route.
- Media: v1 and v2 uploads, owner-only GET and PUT, 422 for unsupported or
  unreadable files, video and audio made with ffmpeg lavfi sources and checked
  with ffprobe; every remote media address goes through the proxy; the proxy
  refuses unsigned URLs, streams ranges as 206 without caching, caches whole
  downloads and serves them with ranges, and streams anything over
  Media:MaxProxiedBytes (a SmallProxyHost) without caching.
- Provenance of local, delivered (signature) and fetched (instance actor,
  no signature, the trigger as activity) posts, visibility of provenance,
  instance descriptions; reading any of them makes no outbound request.
  Pleroma reactions with EmojiReact and Undo deliveries, and local reaction
  notifications.

Bugs fixed:
- remove_from_followers deleted the Follower row but never told a remote
  follower. It now sends Reject{Follow} with the stored Follow id, through
  RelationshipService.RemoveFollower, which Block now shares.
- VisibilityPolicy.CanSee refused followers-only posts to accepted followers,
  so a post in their home timeline answered 404 to GET, context, favourite and
  reply. Followers of the author (local or remote) may now see them.
- Account statuses of a remote account hid followers-only posts from
  personas that follow it.
- exclude_replies dropped the author's own threads; like Mastodon it now
  drops only replies to other accounts.
- A community account's statuses were always empty: they are now the posts
  addressed to the community.
- Pinning someone else's visible post answered 404; it answers 422 like
  Mastodon.
- GET /api/v1/notifications/:id answered 200 with null when the notification's
  post was gone; it answers 404.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:58:31 +02:00
thepraandClaude Opus 5.5 31d614efd4 tests: the federation surface's group helper is FederatedGroup, so it no longer clashes with the client API's
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:54:57 +02:00
thepraandClaude Opus 5.5 2645dea26f T8: inbox gaps, jobs, migrations and pages; a deleted remote account's posts are hidden
Owner decision (2026-10-03, "A remote account deletes itself"): its posts are kept but
hidden everywhere.
- Post.AuthorGone (additive bool). DeleteHandler's actor-delete branch sets it on every
  post whose ActorURI is the actor (one update-many), besides dropping its follows and
  timeline rows as before. RemotePosts.Build sets it on a post stored later for an
  account already marked Deleted.
- One rule in VisibilityPolicy: IsShown (not deleted, author not gone), IsPublic and
  CanSee exclude AuthorGone, plus Shown(post) for loaded posts.
- Lookups by id answer 404 through CanSee (statuses/:id and every sub-route, context,
  bookmarks, favourites, polls, reactions, search); provenance, account statuses,
  home/public/tag timelines, notifications, conversations, reblogged_by, the clientapi
  home and post/DM lists, a community's outbox and our Announces filter on IsShown or
  IsPublic; the Mastodon mapper never renders a hidden post or a boost of one.

Tests (30 new):
- AuthorGoneTests: the rule, the handler (posts kept, boosts included, follows and rows
  gone), a post fetched after the delete, and 20 Mastodon/ActivityPub lookups over HTTP
  seen before and hidden after.
- InboxGapTests: actor Update refresh (name, sanitised summary, key rotation in place
  and to a new key id) even with an older `updated`; Undo{Follow} by activity id and by
  object; Reject of our QuoteRequest (and a stranger's ignored); group-wrapped
  Announce{Like} and Announce{Undo{Like}}; a locked persona's pending follow,
  FollowRequest notification, and Decide accepting and rejecting with the original Follow.
- JobHandlerTests: AncestorsJobHandler up to its depth limit; PollRefreshJob and
  PollCloseJob (local and remote polls); InstanceDescriber from a peer's NodeInfo and
  the weekly dedupe through ObjectRecords; LinkPreviews for public posts only;
  DeliveryJobHandler outcomes (2xx, 404/410, 429/503 with Retry-After in seconds and as
  a date, 5xx) and a signature and Digest the peer can verify; MediaJanitor.Sweep;
  OAuthPruner.Prune.
- MigrationTests: _003, _004, _006 and _007 on seeded rows.
- PublicPagesTests: /@user and /@user/{id} (visibility, junk ids, exact CSP,
  Referrer-Policy and nosniff), circle 404, community page, the instance actor,
  ActivityPub redirects, and markup escaped in posts, titles and bios.

Production changes besides the rule:
- LinkPreviews.Handle re-checks that a post is still shown and public (the rule
  Wanted applies) before fetching anything; before, only enqueueing checked it.
- The legacy /clientapi post and DM lists no longer return soft-deleted posts.
- MediaJanitor.Sweep and OAuthPruner.Prune are the loop bodies, now public and tested.
- InstanceDescriber.Address: a protected virtual identity seam so a test can point
  the https NodeInfo addresses at a plain-http peer; production behaviour unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:53:23 +02:00
thepraandClaude Opus 5.5 2cfea7b60c T7: the federation surface over HTTP
Tests through the real routes of PeasantsController, WellKnownController and
UsersController, on the whole server under test (34 new tests):

- FederationGetTests: the actor document (activity+json, SPKI key at
  #main-key owned by the actor, sharedInbox, published = PublishedOn's day,
  no creation date, no root); ld+json; browsers sent to /@name; Vary: Accept;
  /users 301; the outbox's totalItems and ?page=true&max_id paging across
  the 20-item boundary, boosts as Announces, and no followers-only, direct,
  located, federated-copy or deleted post; /groupies and /stalking naming
  nobody; /trophies (public pins, newest first) and /tattoos; /scribbles
  (public and unlisted 200, browsers redirected, followers-only, direct and
  located 404, deleted 410 Tombstone); a circle post only for a signed member
  or its instance actor; a circle's /groupies, /flock and /wardens only for
  members; /grunts create- and announce- ids; /parrot-licences 200, revoked
  410, wrong author 404; secure mode's 401 for every unsigned GET but the
  instance actor's.
- WellKnownTests: WebFinger by acct:, @-prefixed, bare, upper-case and actor
  URI; other domains, unknown names, a root's login name and no resource;
  the instance actor, a community, a circle (answered: current behaviour);
  NodeInfo links, 2.0 and 2.1 naming no root, unknown versions 404; usage
  counting only public, unlisted, non-boost local posts (Exclusive).
- InboxRouteTests: all three inboxes accept a signed delivery and refuse
  junk (400), unsigned (401), a bad Digest, a two-hour-old Date, a signature
  for another host and a swapped body (401); an unknown persona's /mouth is
  404; ld+json with the ActivityStreams profile is accepted; a signer whose
  actor answers 503 gets 503 with Retry-After; the 301st unsigned POST from
  one address is 429 while a signed server from it is not.
- PersonaSeparationHttpTests: with a sibling persona and its community on
  the same login, every GET under /api (filled with the persona's ids) plus
  search, lookup and relationships, and a crawl of everything federation
  publishes about the persona (actor, outbox pages, collections, scribbles,
  grunts, WebFinger, NodeInfo, /@ pages), never name the sibling, its
  community or the login.

Fixed:
- A circle's /groupies told anyone how many followers (members) it has,
  while its /flock and /wardens were already for members only; it now
  answers 404 to anyone but a signed member or a member's instance actor.
- WebFinger answered 404 to a bare user@domain or @user@domain resource,
  which Mastodon, GoToSocial and Pleroma all accept; it now treats them as
  acct: (noted in docs/INTEROP.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:53:23 +02:00
thepraandClaude Opus 5.5 c5e4934ba6 T5: OAuth and the client API over HTTP
96 integration tests through PrivaPubHost, the real pipeline end to end:
- OAuth: the token's subject is the persona, and neither the token response,
  verify_credentials nor the stored token entries name the root. A wrong password shows
  an error and sets no login cookie; a login without the antiforgery token is a 400; the
  return address never leaves the site; deny answers access_denied with no code; another
  root's persona re-renders the choice with no code; a banned root is sent back to the
  login and a code issued before the ban buys no token; force_login asks again; a code
  works once and its reuse revokes the token it bought; password and refresh_token
  grants are refused; a client_credentials token gets 401 on user routes; a read-only
  token gets 403 with a Mastodon error on POST /api/v1/statuses; follow covers
  read:follows; revoke works; the login and authorize pages send their CSP and no-store;
  the 11th /oauth/login from one address in a minute is a 429.
- Accounts: sign-up, duplicates in any case, invalid models answer 400 with a message,
  login and logout, recovery email, settings, password change, invitation sign-up and
  login (refusing a persona named after the login), recovery without an email, through
  an unreachable mail server, with a wrong and with a valid code, token refresh, the 11th
  sign-up from one address, expired, garbage and foreign-key JWTs.
- Personas: a rootId in the body is ignored, the username regex and reserved names hold,
  personas and groups share ReservedName, an update delivers Update{Person} to followers,
  PublishedOn and the id's day fall within two weeks before creation, the list holds only
  one's own personas, another root's persona cannot be updated.
- Groups: communities and circles are created, joining takes the code and the password,
  members leave and owners cannot, a remote follow request becomes a member only on
  approval, a circle never shows in lookup, account by id, v2 search or /@name, and its
  /flock and /wardens answer 404 unsigned and to non-members, 200 to a member's signed GET.
- Moderation (Exclusive, it suspends localhost): ban, unban and remove; non-admins get
  403; reports are listed without the reporter and resolved; domain blocks are inserted,
  listed and deleted, bad domains refused, and a suspended server's delivery is answered
  202 and kept nowhere; the data endpoints.
- AdminCommands: exit codes 0, 1 and 2 and the resulting policies.

Fixed:
- A banned or removed root kept using /clientapi with its JWT until it expired: only /api
  re-checked the root. JwtEvents.TokenValidated now loads the root and fails the request
  when it is banned or deleted, and takes the policy claims from the database, so a
  demoted admin loses admin at once (and a promoted one gains it).
- The 401 and 403 bodies JwtEvents writes were PascalCase while every other /clientapi
  answer is camelCase; they now use the web defaults.
- /clientapi/user/sniff/again (token refresh) answered an empty 200; it now answers a
  fresh JwtUser, like login.
- Password recovery answered SMTP reply codes as HTTP statuses (421, 454, 554, and 550 for
  an invalid address); a mail server failure is now 503 and an invalid address 400.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:53:23 +02:00
thepraandClaude Opus 5.5 c301f0c498 T9: the pasture as plugins
tools/pasture/run.sh up [peer...] | down | logs | ps and interop.sh [peer...] are now built
from parts:
- lib/pasture.sh: network, Caddy with its CA in a volume and copied to .ca/root.crt, Mongo,
  PrivaPub;
- peers/<name>.sh: each peer's <name>_up;
- lib/interop.sh: ok, ko, and xf for checks expected to fail until a later phase;
  privapub_token <persona>, which gives each peer its own persona under one root; and
  stats_check;
- scenarios/<name>.sh: each peer's checks.

GoToSocial is pinned at 0.22.1, the version the 33 checks were proven on. Its scenario
gains four statistics checks:
- the admin statistics describe gts.test as gotosocial;
- they count inbound and outbound traffic;
- they name no account.

37/37 passed three runs in a row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:42:03 +02:00
thepraandClaude Opus 5.5 6ef7e2891a M10: the admin statistics API
Under /clientapi/admin/statistics, admin only (the root JWT's IsAdmin policy, like the domain
blocks; /api tokens are persona tokens):
- GET overview?days: totals per channel, inbound and outbound outcomes, the delivery
  success rate, delivery latency p50/p95 from the buckets, active and known servers,
  distinct accounts, and the ledger's written/dropped/failed counts;
- GET hosts?days&sort=volume|failures|latency|host&software&page&limit: per server
  traffic, refusals, failures, latency, accounts, software and delivery health;
- GET hosts/{host}?days: the server's description, its daily series and its last 100
  events;
- GET events?host&channel&outcome&reason&before&limit, and GET server?days for the
  ServerDay rows;
- POST rollups/{day} refolds a past day; POST hosts/{host}/describe describes a server
  again.

Days not yet rolled up, today included, are folded live from the events, so the numbers are
current. Answers that may carry locations carry the DB-IP attribution.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:35:40 +02:00
thepraandClaude Opus 5.5 cee85309b8 M9 (first part): the means to locate a server
- IConnectedAddresses remembers which address each host's last connection reached, set in
  SafeHttpHandlerFactory's connect callback. That is once per pooled connection, with no
  second DNS lookup, and the address is never stored.
- IGeoLocator / DbIpLocator reads the offline DB-IP Lite city and ASN databases (MaxMind
  .mmdb, via MaxMind.Db). It maps memory, swaps to new files within ten minutes, rounds
  coordinates to one decimal, never looks up a private address, and answers nothing
  when the files are missing. CdnNetworks names the CDNs whose edge addresses say nothing
  about where a server is.
- deploy/max/geo-update.sh fetches this or last month's databases, checks them and swaps
  them in atomically. The privapub-geo timer runs it monthly as www-data, and setup.sh
  installs the directory, the script, the units and a first download.

Describing servers will use these in M8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:33:30 +02:00
thepraandClaude Opus 5.5 fc15f6356d M7: daily rollups
A RollupDay job folds each finished day's InteractionEvents into one InstanceDay per server:
- Counters for the admin, keyed channel:activity:object:outcome:reason, plus signature
  schemes, audiences, local kinds and features;
- PublicCounters, everything a public page may ever read:
  - inbound and outbound activities from an allowlist, on public, unlisted or unaddressed
    traffic only, with the outcome collapsed (accepted or dropped, delivered or failed);
  - no reasons, no Flag or Block;
  - features of public objects;
  - health:ok or health:failed from reachability (a 4xx means the server answered);
- latency, wait and byte histograms, and the number of distinct accounts from that day's
  hashes.

Re-running a day replaces it and keeps the live Reads counters. The day's salt is then
deleted, so its hashes can never be recomputed, and the next day is queued.
StatisticsSchedule plans today's rollup every hour and catches up any of the last seven
days that have events but no rollup. Waits round up into their bucket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:30:19 +02:00
thepraandClaude Opus 5.5 bc8ce63707 docs: v1.15.1 and v1.16.0 shipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:25:10 +02:00
474 changed files with 48765 additions and 1437 deletions

No files matched your search

+58 -11
View File
@@ -12,8 +12,7 @@ env:
BACKUPS: /var/backups/privapub.thepra.dev
LOCAL_URL: http://127.0.0.1:6970
PUBLIC_URL: https://privapub.thepra.dev
MONGO_URI: mongodb://127.0.0.1:27022
MONGO_DB: PrivaPub
SERVER_BACKUPS: /var/lib/privapub/backups
jobs:
site:
@@ -57,12 +56,24 @@ jobs:
echo "SNAPSHOT=$BACKUPS/site-$STAMP" >> "$GITHUB_ENV"
ls -1dt "$BACKUPS"/site-* 2>/dev/null | tail -n +4 | xargs -r rm -rf || true
- name: Dump the database
# The server's own backup (PrivaPub admin backup, owner decision 2026-10-07) of the database as the live build left it:
# the new build's command runs before its migrations. The media are listed, not linked (the runner may not link
# www-data's files; the nightly backups hold them). Never the statistics salt, which must not outlive its day (owner
# rule), nor the configuration's copy with its secrets. No deploy while a restore waits for its boot.
- name: Back up the database
run: |
DUMP="$BACKUPS/mongo-$(date +%Y%m%d-%H%M%S).archive.gz"
mongodump --quiet --uri "$MONGO_URI" --db "$MONGO_DB" --gzip --archive="$DUMP"
echo "::notice::database dumped to $DUMP ($(du -h "$DUMP" | cut -f1))"
ls -1t "$BACKUPS"/mongo-*.archive.gz 2>/dev/null | tail -n +8 | xargs -r rm -f || true
[ ! -e "$SERVER_BACKUPS/restore.json" ] || { echo "::error::a restore is pending or running ($SERVER_BACKUPS/restore.json): deploy after it"; exit 1; }
out=$(cd "$GITHUB_WORKSPACE/publish" && ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin backup --kind pre-deploy --db-only)
echo "$out"
id=$(echo "$out" | grep -oE '^[0-9]{8}-[0-9]{6}-pre-deploy' | tail -1)
[ -d "$SERVER_BACKUPS/$id" ] || { echo "::error::the backup was not made"; exit 1; }
(cd "$GITHUB_WORKSPACE/publish" && ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin backup verify "$id")
if [ -e "$SERVER_BACKUPS/$id/db/InteractionSalt.jsonl.gz" ]; then
echo "::error::the backup holds the statistics salt"
exit 1
fi
echo "BACKUP_ID=$id" >> "$GITHUB_ENV"
echo "::notice::database backed up as $id"
- name: Stop, sync, start
run: |
@@ -90,13 +101,16 @@ jobs:
fi
- name: Verify what is being served
env:
SMOKE_TOKEN: ${{ secrets.PRIVAPUB_SMOKE_TOKEN }}
run: |
served=$(curl -fsS "$PUBLIC_URL/build.json" | python3 -c "import json,sys; print(json.load(sys.stdin).get('commit',''))")
[ "$served" = "$BUILD_COMMIT" ] || { echo "::error::served build is '$served', expected '$BUILD_COMMIT'"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/privapub")
[ "$code" = "200" ] || { echo "::error::the instance actor answered $code"; exit 1; }
# SecureMode (owner decision 2026-10-04): an unsigned reader gets 401 from a persona, a browser the public page
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/thepra")
[ "$code" = "401" ] || { echo "::error::an unsigned GET of a persona answered $code, not 401: SecureMode is off"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: text/html' "$PUBLIC_URL/peasants/thepra")
[ "$code" = "302" ] || { echo "::error::a browser asking for a persona got $code, not a redirect"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/.well-known/nodeinfo")
[ "$code" = "200" ] || { echo "::error::nodeinfo answered $code"; exit 1; }
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/swagger/index.html")
@@ -106,6 +120,39 @@ jobs:
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/activity+json' \
--data '{"type":"Follow","actor":"https://example.org/users/x","object":"'"$PUBLIC_URL"'/peasants/privapub"}' "$PUBLIC_URL/human-centipede")
[ "$code" = "401" ] || { echo "::error::an unsigned inbox POST answered $code"; exit 1; }
tools/smoke/mastodon-api.sh "$PUBLIC_URL" "$SMOKE_TOKEN"
[ -n "$SMOKE_TOKEN" ] || echo "::warning::PRIVAPUB_SMOKE_TOKEN is not set, so the signed-in API was not checked"
tools/smoke/mastodon-api.sh "$PUBLIC_URL"
open=$(curl -fsS "$PUBLIC_URL/nodeinfo/2.1" | python3 -c "import json,sys; print(json.load(sys.stdin)['openRegistrations'])")
enabled=$(curl -fsS "$PUBLIC_URL/api/v2/instance" | python3 -c "import json,sys; print(json.load(sys.stdin)['registrations']['enabled'])")
[ "$open" = "$enabled" ] || { echo "::error::NodeInfo says registrations are $open, the instance API $enabled"; exit 1; }
curl -fsS "$PUBLIC_URL/stargazing" | grep -q 'The crawler is <strong>on</strong>' \
|| { echo "::error::/stargazing does not say the crawler is on"; exit 1; }
echo "::notice::serving $served"
# @thepra is the deploy's own persona (owner decision, 2026-10-04): the server's CLI makes or keeps it, undiscoverable,
# and gives its root a new password on every deploy, so no secret is stored and nobody has to create anything.
- name: Sign in as @thepra and check the signed-in API
run: |
creds=$(cd "$WEB_ROOT" && ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin smoke thepra | grep -E '^deploy-smoke [^ ]+$' | tail -1)
[ -n "$creds" ] || { echo "::error::the smoke persona could not be prepared"; exit 1; }
read -r login password <<< "$creds"
echo "::add-mask::$password"
read -r token cid cs <<< "$(tools/smoke/oauth.sh "$PUBLIC_URL" "$login" "$password" thepra read)"
echo "::add-mask::$token"
tools/smoke/mastodon-api.sh "$PUBLIC_URL" "$token"
discoverable=$(curl -fsS -H "Authorization: Bearer $token" "$PUBLIC_URL/api/v1/accounts/verify_credentials" | python3 -c "import json,sys; print(json.load(sys.stdin).get('discoverable'))")
[ "$discoverable" = "False" ] || { echo "::error::@thepra is discoverable"; exit 1; }
curl -fsS -o /dev/null -X POST "$PUBLIC_URL/oauth/revoke" --data-urlencode "token=$token" \
--data-urlencode "client_id=$cid" --data-urlencode "client_secret=$cs"
echo "::notice::signed in as @thepra"
# The server fetches DB-IP Lite itself (GeoUpdater) about 30 s after it starts and then daily; the deploy only checks.
- name: Geolocation databases are current
run: |
for kind in city asn; do
db="/var/lib/privapub/geo/dbip-$kind-lite.mmdb"
for i in $(seq 1 60); do [ -s "$db" ] && break; sleep 10; done
[ -s "$db" ] || { echo "::error::$db is missing: the server has not fetched DB-IP Lite"; exit 1; }
days=$(( ($(date +%s) - $(stat -c %Y "$db")) / 86400 ))
[ "$days" -le 40 ] || { echo "::error::$db was installed $days days ago"; exit 1; }
done
curl -fsS "$PUBLIC_URL/stargazing" | grep -o 'DB-IP Lite [0-9-]*' | head -1 | sed 's/^/::notice::locating with /'
+9
View File
@@ -257,6 +257,7 @@ Generated_Code/
# because we have git ;-)
_UpgradeReport_Files/
Backup*/
!PrivaPub/Infrastructure/Backup/
UpgradeLog*.XML
UpgradeLog*.htm
ServiceFabricBackup/
@@ -401,4 +402,12 @@ FodyWeavers.xsd
PrivaPub/media-store/
PrivaPub/media-store-proxy/
PrivaPub/media-store-trash/
PrivaPub/media-store-incoming/
PrivaPub/media-store-backups/
tools/pasture/.publish/
tools/pasture/.flood/
.claude/worktrees/
tools/pasture/.ca/
tools/pasture/.state/
tools/pasture/out/
+582 -60
View File
@@ -1,4 +1,4 @@
# CLAUDE.md
# CLAUDE.md
Guidance for working in this repository. `docs/ROADMAP.md` holds the owner's decisions and the phased plan to full
ActivityPub interop; read it before changing anything federation-, privacy- or API-shaped. `docs/INTEROP.md` is the
@@ -21,6 +21,9 @@ Its defining idea: **one private login owns several public personas.**
becoming two kinds: a public **community** (FEP-1b12, Lemmy-compatible) and a private, invitation-only **circle**.
- **`DmGroup` is a direct-message conversation.**
- **`privapub` is the instance actor** (type Application). It signs fetches no persona should be tied to.
- **`privapub_reports` is the reporter** (type Service, `LocalActorKind.Reporter`). It carries reports to Lemmy, which
takes none from an Application, and to PieFed and Mbin, and names nobody. Both are server actors
(`LocalActor.IsServerActor`): never followed, mentioned or shown as accounts.
Privacy features in the model:
- location-ranged posts (`Post.Location`, `RangeKm`), to become local-only and never federated;
@@ -29,8 +32,8 @@ Privacy features in the model:
The intended client is the Mastodon client API (Tusky, Elk, Phanpy, Ivory), where each avatar logs in as its own
account. The private `/clientapi` covers what Mastodon can't express. `thepra/decePubClient`
(https://decepub.thepra.dev) is the owner's own Pleroma-FE-like PWA. It still shows mock data and is not wired to the
server.
(https://decepub.thepra.dev) is the owner's own Pleroma-FE-like PWA. It signs in once on `/clientapi` and exchanges that
JWT for each persona's Mastodon token (`PersonaExchange`), then uses both APIs.
## Route names are deliberate
@@ -83,9 +86,14 @@ PrivaPub/ ASP.NET Core Web API, net10.0
(authoritative fetch, key verification, WebFinger), ActorDocument (parser)
Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer
Moderation/ DomainBlocks (suspend / silence / reject media)
Signing/ HttpSignatures (draft-cavage sign/verify)
Signing/ HttpSignatures (draft-cavage sign/verify), MessageSignatures (RFC 9421 verify),
RequestSignature (whichever a request carries), IntegrityProofs (FEP-8b32
eddsa-jcs-2022 by a persona's Ed25519 key, FEP-521a Multikeys), Jcs (RFC 8785)
Inbox/ InboxReceiver (verify, queue, 202) → InboxProcessor (job) → Handlers/{Follow,Accept,Reject,
Undo,Create,Update,Delete,Like,Announce}; RemotePosts (build, fetch parents, FetchAncestors)
Undo,Create,Update,Delete,Like,Announce}; RemotePosts (build, fetch parents, FetchAncestors);
RemoteReplies (FetchReplies: a thread's `context`, else `replies` two levels down);
Forwarded (what a thread's server passes on, believed as far as the origin vouches);
ReplyRelay (third-party replies to a persona's posts passed on to its followers)
Outbox/ OutboxPublisher (who a post goes to), DeliveryService (queues jobs) + DeliveryJobHandler
Rendering/ ActivityPubRenderer (Mastodon @context, actors, notes, collections)
Domain/
@@ -95,7 +103,9 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Privacy/ VisibilityPolicy (IsPublic expression, CanSee)
Relationships/ RelationshipService (blocks, mutes, account domain blocks; Hidden), ReportService
Media/ MediaService (libvips, ffmpeg remux, blurhash), MediaProxy, MediaJanitor
Domain/Statuses/ StatusService: publish, edit, remove, favourite, reblog, for a persona (both client APIs use it)
Domain/Statuses/ StatusService: publish, edit, remove, favourite, reblog, for a persona (both client APIs use it);
InteractionApprovals: GoToSocial's canReply/canLike/canAnnounce, judged, asked and answered;
Participations: a persona's Join and Leave of a remote event, and the organiser's answer
Api/Mastodon/
Auth/ OpenIddict setup (keys in Mongo), MastodonScopes, TokenController, OAuthPruner
Infrastructure/ MastodonController (avatar context, scopes, errors, Link), MastodonParams, MastodonJson, Page
@@ -136,29 +146,50 @@ curl-able. Outbound fetches only go to https DNS names resolving to public addre
`Federation__AllowPrivateNetworks=true`, `Federation__AllowPlainHttp=true` and `Federation__AcceptAnyCertificate=true`,
which startup refuses in Production.
Promoting an admin on the box (signing up as "admin" grants nothing):
The admin CLI runs with every service built but nothing started (no Kestrel, no hosted services), so it can run next to
the live service. Sign-up is closed in production (invitations only), so the first login is made here; promoting is how
an admin is made (signing up as "admin" grants nothing):
```bash
cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin promote <root>
cd /var/www/privapub.thepra.dev
echo '<password>' | ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin create-root <login> --admin # password on stdin
ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin promote|demote <root>
ASPNETCORE_ENVIRONMENT=Production ./PrivaPub admin smoke <persona> # the deploy's: prints "deploy-smoke <new password>"
```
The deploy runs them as `build-runner`, which owns the published files, reads `appsettings.Production.json` through
group www-data and reaches the private mongod; `sudo -u www-data` works too.
## Federation invariants
1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only
to public addresses; redirects are followed by hand (three at most, each re-checked); bodies are capped at 1 MB;
only JSON media types are read; a refused URL is not asked again for five minutes. Never create another
`HttpClient` for federation.
`HttpClient` for federation. The only other outbound traffic is SMTP and `GeoUpdater`'s monthly DB-IP Lite download
(its own `geo` client, a fixed HTTPS host, size-capped, the file checked before it is swapped in).
2. **Every fetch is signed by the instance actor** (`privapub`), never by a persona; deliveries are signed by the acting
avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies).
avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies). Inbound,
an RFC 9421 signature (`Signature-Input`) is verified too, its target against our public address. A persona's own
activity going to a relay also carries a FEP-8b32 proof by its Ed25519 key (`Avatar.SigningKey`, one per persona,
never shared), and nothing else does: Mitra refuses a proof by a key it has not read. A forwarded activity is taken
as it came only when its proof verifies with a Multikey of its actor's.
3. **A remote document is believed only from its own address.** `RemoteActorService.FetchObject` requires the
document's `id` to be the URL it was served from (a same-origin alias is followed once). A key is accepted only if
the actor lists it, its `owner` is the actor and it shares the actor's origin.
4. **Inbound inboxes verify everything before acting:**
- the signature covers `(request-target)`, `host`, `digest` and `date` or `(created)`;
- the Digest matches the body and the date is at most an hour old and fifteen minutes ahead;
- the signature verifies against the key owner's key, and the activity's `actor` is the key owner;
- the signature verifies against the key owner's key, and the activity's `actor` is the key owner, or else it was
forwarded (`Forwarded`: a thread's server passing on what happens in it, as Mastodon and Friendica do). A forwarded
activity proves nothing about its author: answered 202, a Create or Update is taken as its object reads at the
actor's origin now, a Delete of a public or unlisted copy once that origin answers 404 or 410
(`RemoteActorService.IsGone`), anything else let go. Forwarded copies have their own dedupe key, so a copy that
failed never hides the author's own delivery;
- an activity is queued once per id, but an id that comes back carrying another type, actor or object is a second
activity, not a copy (Friendica's ids are `uniqid()`, which two of its processes can share), and is queued apart;
- the activity's `id`, and any object it creates, updates or deletes, is on the actor's origin; a cross-origin
object is refetched from its own origin.
object is refetched from its own origin. One attributed to another account of the actor's server (Mobilizon's
organiser and the group's event) is taken as that server has it, and deleted once it answers 404 or 410.
5. **Status codes:**
- bad or missing signature: **401**;
- malformed or forged body: **400**;
@@ -173,27 +204,71 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
`ContentFormat` says what `Text` holds. Remote names are plain text.
8. **Circles federate to members only.** A circle is an undiscoverable Group actor that takes follow requests (the owner
approves); its posts are addressed to the circle and its `/flock`, delivered to members' personal inboxes, never
announced, and served only to a signed request from a member or a member's instance actor
(`SignedFetchAuthorizer`), 404 otherwise. Circles never appear in search, lookups, mentions or profile pages.
announced. Each member's copy also names that member in `cc` (`OutboxPublisher.Naming`, owner decision
2026-10-04), because Mastodon and GoToSocial keep a post only when it names one of their accounts; Create, every
Update and the Delete all go through `OutboxPublisher.Publish` for that. A reply to a circle post stays in the circle,
and a circle post never asks a non-member for a quote. Circles never appear in search, lookups, mentions or profile
pages.
**A post that is not public is served only to a signed request from someone it was for** (`SignedFetchAuthorizer.MayRead`):
a follower or an addressed account for followers-only, an addressed account for a DM, a member for a circle, or the
instance actor of a server where one of them lives; 404 to anyone else, 410 to them once it is deleted. A circle
refetch names the requesting member, or the members on the requesting server. A DM's `context`
(`/peasants/{name}/whispers/{id}`) lists the conversation's posts for its participants. Mastodon deletes its copy
when a refetch answers 404, which is why this matters.
**Communities** are FEP-1b12 groups: `GroupDistributor` announces the whole activity (plus the object for new posts,
for Mastodon), top-level posts are `Page`s with a `name`, posting follows `Group.PostingPolicy`.
Located posts (`LocalGeo`) are the only local-only posts.
Located posts (`LocalGeo`) are the only local-only posts. The Mastodon API looks past them everywhere, homes
included (the fan-out gives them no `TimelineEntry`, not even their author's): they are read through
`/clientapi/post/nearby` and deleted through `/clientapi/post/delete`.
9. **A DM joins a conversation only by `DmGroup.ParticipantsKey`**, the exact set of its participants; a remote
`context` decides nothing. DMs are `Post`s with `Visibility = Direct` and a `ConversationId` (`DmPost` is legacy).
`DmGroup.LastPostId` pages `/api/v1/conversations`, and `ConversationState` keeps each persona's read and removed
marks (`Domain/Statuses/ConversationStates.cs`; writing in a conversation reads it).
10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`):
leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`).
11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`.
12. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`). A lease
(2 minutes, stamped with its own owner) is renewed every third of its length while the handler runs, so a long job
runs once; a lease found taken cancels the handler, and `Finish` only counts for the lease it was given.
11. **Every number about posts and users goes through `Domain/Privacy/Counted`** (owner decision 2026-10-04: one
answer everywhere): a persona's `statuses_count`, its outbox `totalItems`, NodeInfo `localPosts` and the instance
`status_count` all count Mastodon's way (not deleted, not a DM, boosts included, circle and located posts too);
users exclude personas of banned or deleted roots; `replies_count` counts only public and unlisted replies; a
remote account that deletes itself takes its likes, votes, reactions, boosts and replies out of every count
(`GoneActors`). Follower and following counts are public, their members never are (`hide_collections` is always
true).
12. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`, and
any other read of stored posts filters by `IsShown`. All three hide deleted posts and the posts of a remote account
that deleted itself (`Post.AuthorGone`: kept, hidden everywhere, owner decision).
13. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is
fetched as context. Followers-only is detected by the author's stored `followers` URL.
13. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and
14. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and
every delete removes its `TimelineEntry` rows. Local deletes are soft (content cleared, 410 Tombstone).
15. **What PrivaPub passes on is exactly what it received** (owner decision 2026-10-05). A public or unlisted reply from
another server to a persona's public, unlisted or followers-only post, and its Update and Delete, go to the
persona's followers as the activity arrived (`Arrival.Raw`, `Federation/Inbox/ReplyRelay.cs`), signed by the
persona for the transport only: never to the replier's own server, never for a local-only or group post. Only an
activity its own actor delivered carries `Raw`, so a forwarded one is never passed on again.
16. **An `Accept` or `Reject` is routed by what it answers:** our Follow (`FollowService`, any `-again-N` resend), an
interaction request (`InteractionApprovals`, with the author's authorization read back from its origin) or a
persona's `Join` (`Participations`, by its `/grunts/join-<id>` id), each only from the origin of what it answers.
17. **A server's software is for display, with two exceptions** (owner decisions 2026-10-05 and 2026-10-06): a direct
message to one account on a server whose NodeInfo names Lemmy before 1.0 or Mbin goes as a `ChatMessage`
(`StatusService.TakesOnlyChatMessages`), as one to an account that writes to us that way does; and a report of a
post in a community on a server whose NodeInfo is in `ReportService.ServiceReportTakers` leaves in Lemmy's shape,
from the reporter, one `Flag` per post. A server joins that set only once the pasture shows it keeps such a report
with its reason. Nothing else may branch on `RemoteInstance.Software`.
## Mastodon client API invariants
1. **A token is one persona.** Its subject is the avatar id; the root id lives only in the fifteen-minute `/oauth`
cookie used while choosing the persona, and never in a token, an authorization or a response.
- **The first-party client exchanges the root JWT for one persona's token** (RFC 8693 on `/oauth/token`,
`subject_token_type` `urn:ietf:params:oauth:token-type:jwt`, `avatar_id`; `Api/Mastodon/Auth/PersonaExchange.cs`).
Only the seeded public application (`ClientApi:FirstPartyClientId`, default `decepub`) holds the grant.
`RootJwtSubjectToken` validates the JWT through `RootJwt`, the same rules as JwtBearer (signature, lifetime, ban,
deletion, session stamp). The issued token is the same kind as the authorization code flow's, so revocation and
`RootSessions` cover it.
2. `/api/*` authenticates with OpenIddict validation, everything else with the old JWT (`PrivaPub` policy scheme).
Every `/api` request re-checks that the persona's root is neither banned nor deleted (`MastodonController`).
3. Read parameters through `Params` (query, form and JSON merged Rails-style), never MVC binding. A value type read
@@ -203,9 +278,17 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
error that leaves without a body (a challenge, a 404 from routing, a 429) gets Mastodon's `{"error": ...}` from
`UseMastodonErrorBodies`. `NeverFiveHundredTests` walks every route with junk ids, anonymously, as a persona and
with a junk token.
6. Advertise `4.2.0 (compatible; PrivaPub)` until grouped notifications exist.
6. Advertise `4.2.0 (compatible; PrivaPub)` until streaming and Web Push exist too; `api_versions.mastodon = 7` stays,
because clients show quotes from it, so everything a 4.3+ client calls because of it must answer: grouped
notifications (`/api/v2/notifications`, its unread count, groups, accounts and dismiss) and the notification policy.
The policy is real (`Domain/Social/NotificationPolicies.cs`, applied in `Notifications.Add`): accepted by default,
it filters (into a `NotificationRequest` per account, out of every list and count unless `include_filtered`) or drops
what strangers, accounts that do not follow the persona (or only for three days), accounts newer than 30 days,
unasked private mentions and silenced accounts send; accepting a request lets that account in for good
(`NotificationPermission`).
7. **What a Mastodon `Status` cannot say goes in `Status.privapub`** (`PrivaPubStatus`): object type, title, excerpt,
cover, the author's source, link, video, audio and event details, and up/down votes. Every media URL in it goes
cover, the author's source, link, video, audio and event details, a remote post's own place (Pixelfed's `location`,
never federated again), and up/down votes. Every media URL in it goes
through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following
one is the reader's choice.
@@ -213,18 +296,72 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
1. **No upload keeps its metadata.** Images are re-encoded by libvips with `keep=none`; audio and video are remuxed with
`-map_metadata -1`. `MediaProcessingTests` checks EXIF and XMP are gone.
2. Files live under `Media:Root` (`/var/lib/privapub/media`), never in the published directory; the proxy cache is the
sibling `media-proxy`, which `/media/files` does not serve.
3. **A client never contacts a remote server for media:** every remote URL the API returns goes through
2. **An image is checked before it is decoded.**
- **Loaders:** only libvips' JPEG, PNG, GIF, WebP and HEIF loaders ever run on an upload (`MediaService`'s static
constructor blocks every other loader), so an SVG, PDF or TIFF claiming another type never loads.
- **Size:** the header alone tells width × height (× frames for a GIF), refused above `Media:MaxPixels` (40 MP) or
`Media:MaxFrames`.
- **Then:** the image is shrunk on load to `MaxImageSide`, turned by its orientation, and its colours brought into
sRGB (`thumbnail`).
- **GIFs:** an animated GIF becomes a looping silent H.264 mp4 typed `gifv`, as on Mastodon (ffmpeg with libx264;
`PostMedia.Kind` says so), and a still GIF is an image.
- **Types:** HEIC and HEIF are not accepted, since the bundled libvips has no HEVC decoder.
- **Load:** processing runs `Media:Concurrency` at a time, and uploads have their own rate limit (`uploads`, per
credential).
3. **Audio and video are processed off the request when the client allows it.**
- **v2:** `POST /api/v2/media` stores them as sent in `media-incoming` (beside the root, never served) and answers
202 with no url, while a `ProcessMedia` job (one at a time, its lease renewed) does the work. Until then
`GET /api/v1/media/:id` answers 206, and 422 with the reason if it failed. Media still processing can't be posted.
- **v1:** processes before answering.
- **ffmpeg** reads only the stored file: `-protocol_whitelist file`, and the input format is forced from the probe.
Data and subtitle tracks are dropped (`-dn -sn`, which iPhone MOVs need).
- **Remux or transcode:** a video browsers play as it is (H.264, VP8, VP9 or AV1, within `Media:MaxVideoPixels` and
`MaxFrameRate`) is remuxed; anything else is transcoded to H.264 that fits, as Mastodon does.
- **Limits:** longer than `Media:MaxSeconds` is refused.
- **Saving:** nothing is saved until everything succeeded; outputs move into place rather than being read into
memory, and every temporary file goes. FLAC is served as `audio/flac`, and the unit runs with `PrivateTmp`.
4. Files live under `Media:Root` (`/var/lib/privapub/media`), never in the published directory; the proxy cache is the
sibling `media-proxy` and the trash the sibling `media-trash`, neither of which `/media/files` serves.
5. **A file lives exactly as long as something holds it.** Every upload is a `MediaAttachment` row, profile pictures
too (`Kind` avatar or header, `ProfileOfAvatarId`). Deleting a post, an edit leaving media out, a replaced picture,
a dropped scheduled post, a removed root, and an upload never posted for a day each trash theirs
(`IMediaService.Trash`):
- the row gets `TrashedAt` in one conditional update, so a row attached meanwhile is left alone;
- its files move into `media-trash` at once, so `/media/files` stops serving them;
- `MediaJanitor` deletes them a day later (`TrashGrace`).
What a login's media take (every persona's, trashed ones no longer) is `Counted.MediaOfRoot`, shown to the login at
`/clientapi/user/storage`; `Media:QuotaBytesPerRoot` (0, none, by default) refuses an upload over it with 422; the
administrator's statistics overview has the totals, the proxy cache and the trash.
Nothing is deleted because it looks unused. `PrivaPub admin media audit [--fix]` compares disk and database: with
`--fix` (as www-data) it gives pictures shown from before their rows a row, and trashes media of deleted posts or
personas, rows whose files are missing, and files nothing holds.
6. **A client never contacts a remote server for media:** every remote URL the API returns goes through
`IMediaProxy.Wrap`, an HMAC-signed `/media/proxy/` URL fetched by `IFederationHttp.GetMedia`.
4. **The proxy serves three ways:**
- **Cached:** a file already cached is served from disk, ranges included.
- **Downloaded:** a request without a `Range` is downloaded whole, up to `Media:MaxProxiedBytes`, then cached.
7. **The proxy serves three ways:**
- **Cached:** a file already cached is served from disk, ranges included. It is opened before the answer, so a trim
that deletes it meanwhile breaks nothing.
- **Downloaded:** a request without a `Range` is downloaded whole, up to `Media:MaxProxiedBytes`, then cached:
- once for everyone asking for it at the same time;
- streamed into a `.part` file beside its place and renamed there, never held in memory;
- with its host in its `.type` sidecar, so that blocking a server can purge it.
- **Streamed:** a ranged request, or anything too big to cache, is streamed from the origin with the range passed on,
and never cached. That is how remote video plays.
and never cached. That is how remote video plays. A file found too big is remembered for an hour, so it isn't
fetched twice; one that failed is remembered for five minutes.
Also:
- The cache's size is counted as it grows and trimmed as soon as it passes `ProxyCacheBytes`; the janitor
recounts hourly.
- Browsers may cache only a success.
- Nothing of a suspended server, or of one whose media are rejected, is proxied (avatars, emoji, covers and link
cards included, since they all go through it), and blocking one purges its cache.
- The `proxy` rate limit counts per client address.
nginx has a `/media/proxy/` location with `proxy_buffering off` and a 600 s read timeout for those streams.
5. **Remote video and audio become one playable attachment** in the Mastodon API (`MastodonMapper.Playable`): the best MP4
8. **A focal point is two finite numbers** within -1..1 (`FocalPoint.Parse`); anything else is ignored. A stored NaN made
every status, timeline and Note holding its post fail to serialise; migration `_016` removed the ones stored before.
9. **Remote video and audio become one playable attachment** in the Mastodon API (`MastodonMapper.Playable`): the best MP4
up to 720p that carries both sound and picture, including PeerTube's fragmented files inside an HLS entry. HLS
playlists themselves are not rewritten.
@@ -232,6 +369,16 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
- **No root id in federation output, NodeInfo or logs, no IP next to an identity in logs, and no `ex.Message` to a
client** ("Something went wrong." instead).
- **Sign-in and recovery never tell which logins or addresses exist** (owner decision 2026-10-04): every failed sign-in
gets "That username and password do not match." after the same hashing (`RootUsersService.Decoy`, a constant-time
comparison), and only a correct password learns of a ban. Every recovery request answers the same sentence and
queues a `SendRecovery` job, found or not; `RecoveryJob` sends the email and keeps only a SHA-256 of the code, for an
hour. A recovered password ends every session of the root (`RootSessions`: a new `SessionStamp`, which every `/clientapi`
JWT carries, and OpenIddict revocation for each persona).
- **Deleting a root deletes everything public it had** (`RootRemoval`, from the admin route or `/clientapi/user/delete`
with the password): its sessions end, each persona and each group it owns sends `Delete{Actor}` to followers, members
and the accounts it follows, the personas' posts are emptied, and `/peasants/{name}`, its inbox and WebFinger answer
410 (`LocalActorService.Gone`) while the names stay reserved. The root keeps only a `deleted-{id}` name.
- **Never derive a public name from the root username.** Invitation sign-up takes `AvatarUserName` and refuses one
equal to the login.
- **One username space:** personas, groups and the instance reserve their name in `ReservedName` (unique index) before
@@ -243,7 +390,8 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
see ids. Never emit `CreatedAt`.
- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows. Nothing may relate sibling avatars.
- **Blocks federate** (owner decision, 2026-10-01): a block is sent as `Block` from the blocking avatar, an unblock as
`Undo{Block}`. Reports still leave as `Flag` from the instance actor, never from the reporting avatar.
`Undo{Block}`. Reports still leave as `Flag` from the server's own actors (the instance actor, or the reporter for
communities on Lemmy, PieFed and Mbin), never from the reporting avatar.
- **What PrivaPub reveals is the owner's call.** Previews, blocks, website authorship, views, bridging and reactions were
decided in `docs/ROADMAP.md` ("Owner decisions on what PrivaPub reveals"). Anything new that tells another server
something about an avatar gets the same treatment: ask, then record it there.
@@ -274,18 +422,32 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
revision. Delivery details reach them through `Arrival.Current`, which `InboxProcessor` sets for the handler's
duration. A fetched record has no signature, key or `@context` of the activity that caused the fetch (its activity
fields name that trigger), and every record stores its `Extensions` and `ContextNamespaces` for statistics. The raw form lives outside `Post` so timelines never load it. Read through
`/api/privapub/v1/statuses/:id/provenance` and `/api/privapub/v1/instances/:host`.
`/api/privapub/v1/statuses/:id/provenance` and `/api/privapub/v1/instances/:host` (or `?host[]=`, up to 40; this
server answers too).
- **A server's place leaves only through `PublicGeo.Project`:** the city, coordinates (0.1°) and network it was located
to, whatever its size (there is no user threshold), and only the CDN's name for a CDN-fronted one. This server's own
place is its host's address located once a day (`SelfLocation`), or `Statistics:Geo:Self` when the owner sets it.
- **CDNs are found three ways, best first** (`Infrastructure/Geo/CdnCatalog.cs`): the ranges a CDN publishes
(`CdnUpdater` downloads them daily into `CdnRangeSet`, keeping the old set when a download or parse fails), the CDN's
fingerprint in the responses PrivaPub already gets (`EdgeHintsHandler` on the federation client, memory only), and
a network that carries nothing but a CDN. Never list an ASN shared with plain hosting (AWS, Google, Microsoft,
DataPacket). A server's `Geo` records the CDN, its domain and how it was found.
- **Servers through time** (`Domain/Statistics/ServerPlaces.cs`, from the weekly `RemoteInstanceSnapshot`s):
`/api/privapub/v1/instances/:host/history`, a CDN-fronted server's `geo.before_cdn`, and the CDNs grouped by their
own domain (`/api/privapub/v1/cdns`, `/cdns/:domain` with weekly joined/left).
- **Remote objects are parsed for every shape in `ObjectShapes`:** `url`/`icon`/`image` as a value, an object or an
array; Markdown `content`; missing `mediaType`s inferred; thumbnails from any of their five places; and the typed
`Link`, `Video`, `Audio` and `Event` details. A Mastodon API card is built from those, never by fetching the linked page
(that fetch is the owner's decision 1, for P6).
- **An `Update` is an edit only when its `updated` is newer than ours** (`RemoteEdits.IsEdit`; plain and community-wrapped
- **An `Update` is an edit only when its `updated` is newer than ours**, or for a first edit no older and with the text
actually changed, since GoToSocial's whole-second timestamps make a quick edit carry `updated == published` (`RemoteEdits.IsEdit`; plain and community-wrapped
Updates both go through `RemoteEdits.Apply`, and deletes through `RemoteDeletes.Remove`). Otherwise it refreshes
the poll, video, audio and event details and nothing else, and leaves no revision: Mastodon and Misskey refresh poll
counts with bare Updates.
- **A poll vote is a `Note` with a `name`, an `inReplyTo` that is a poll we hold, and no content.** `CreateHandler`
hands it to `PollService.Receive` before anything else, so it never becomes a reply. Our votes on other servers' polls
go only to the poll's author, without `published` (PieFed counts a vote only then).
go only to the poll's author, without `published` (PieFed counts a vote only then). A `closed` in the future is the
poll's end, not its closing: Akkoma sends an open poll's end only there (`ObjectShapes`).
- **Link previews follow owner decision 1** (`Domain/Content/LinkPreviews.cs`): only public posts, queued on arrival with
0–60 s of jitter, one cached `LinkPreview` per address for the whole server, never fetched when someone reads.
`LinkPreviews.Wanted` is called where posts are saved (`CreateHandler`, `StoreContext`, `StatusService.Publish`).
@@ -302,7 +464,8 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
same rule writes `canQuote`, answers `QuoteRequest`s (`QuoteService.ReceiveRequest`) and fills `quote_approval`, so they
cannot disagree. A persona quoting another persona gets a parrot-licence too, so other servers see an approved quote.
- **A server is described on arrival, never on read.** The first record from a host enqueues `DescribeInstance` (its
NodeInfo, at most once a week, into `RemoteInstance`), so opening the details view tells nobody anything.
NodeInfo, at most once a week, into `RemoteInstance`), so opening the details view tells nobody anything. A
description that failed frees its week, so the server's next arrival (an hour on at the soonest) asks again.
- **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post
published within the last hour (or in the future) a fresh `ObjectId`, which sorts after every post already stored,
and only backfill keeps a `published`-derived id. With `published` ids a reply arriving in the same second could sort
@@ -317,7 +480,89 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
`_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a
dedupe migration before it.
- Production runs its **own mongod** on 127.0.0.1:27022 (unit `privapub-mongod`, no auth, data in
`/var/lib/privapub/mongo`). The box's shared mongod needs credentials nobody here has.
`/var/lib/privapub/mongo`). The box's shared mongod needs credentials nobody here has. It is a one-member replica set
(`--replSet rs0`, a 990 MB oplog; owner decision 2026-10-07), so a backup reads every collection at one instant;
`setup.sh` converts it once (PrivaPub stopped, mongod restarted, `rs.initiate`), and every connection string says
`directConnection=true`, which also works against a standalone. The pasture's mongo is one too.
- **Backups** (`Infrastructure/Backup/`; owner decisions 2026-10-07: the whole server, plain files protected by their
permissions, run from the CLI, nightly and from the administrator's page). Each backup is a directory
`<yyyyMMdd-HHmmss>-<kind>` under `Backups:Root` (`/var/lib/privapub/backups`, www-data 2770, files 0640), written as
`.partial` and renamed once whole:
- `manifest.json` (`ArchiveManifest`): host, build, newest migration, whether it was read at one instant, each
collection's count, size, sha256 and indexes, what was left out and why, and the media list;
- `db/<collection>.jsonl.gz`: each document as one line of canonical Extended JSON, read raw, so every BSON type comes
back byte for byte; on a replica set every collection is read in one snapshot session
(`minSnapshotHistoryWindowInSeconds` is raised to an hour only while it reads);
- `media/`: hard links to the files of untrashed `MediaAttachment` rows (no disk spent; a deleted file stays until the
backup rotates out), copies where a link can't be made. `--db-only` lists them instead.
**Never in a backup** (`ServerBackup.Excluded`): `InteractionSalt` (owner rule), `Job` and `Delivery` (work in flight),
`EmailRecovery`, `openiddict.tokens` and `.authorizations` (sessions), `MaintenanceLock`, and `AppConfiguration`
(its SMTP password; it is made again from appsettings at boot). One backup or restore runs at a time
(`MaintenanceLock`, a heartbeat document; a holder silent for 2 minutes is taken over), and the media janitor purges
nothing meanwhile. `BackupScheduler` backs up at `Backups:NightlyAt` (03:30 UTC), or at once when it missed the night;
rotation keeps 7 daily and 4 weekly nightly backups, 3 pre-deploy, 3 pre-restore, and manual and uploaded ones until
deleted. CLI: `PrivaPub admin backup [--kind manual|pre-deploy] [--db-only]`, `admin backups`, `admin backup verify
<id>`; these run before migrations, so the deploy's backup is of the database as the live build left it.
- **Restores** (`ServerRestore`, `ProtectiveMerge`; owner decision 2026-10-07). Asking (`PrivaPub admin restore <id>`, or
the administrator's page) checks the backup (same host, a format and newest migration this build reads, every hash)
and writes `restore.json` in the backups' root; the running service sees it within seconds (`RestoreWatcher`) and
stops, and systemd starts it again. `MaintenanceGate` (in `Program`, right after the build, before migrations, indexes
and hosted services) carries it out:
1. a pre-restore backup P, taken once (a retry reuses it, never a half-restored database);
2. every collection the backup holds dropped and imported raw with its indexes, every other one dropped, except what a
backup never holds, which stays as it is;
3. media files the live directory lacks linked back from the backup (or moved from the trash); a restore deletes no file;
4. **the protective merge from P: a restore never undoes a protective act.** Followers and follows are P's; blocks,
mutes, domain blocks, being blocked, reserved names, `DeletedObject`, reports, filters and OAuth applications are
the union, P's row winning; roots, personas, groups, posts and remote accounts deleted since are deleted again, with
what their deletion takes away; P's moderation of a persona and a root's password, e-mail, ban and policies stay;
roots, personas and groups made since become tombstones (deleted, names kept), local posts made since answer 410,
media made or trashed since go to the trash;
5. every session ends (a new `SessionStamp` for each root, persona tokens deleted), circuits close, and a
`RestoreRecord` (never in a backup) tells what happened (`admin restore --status`).
Each attempt redoes everything. Refused before anything changed, it is abandoned and recorded, and the server boots as
it was; failed midway, the process exits 1 and the next start tries again; after 3 failures it exits 75, which the
unit's `RestartPreventExitStatus` leaves down for someone to look. While `restore.json` exists, commands exit 75 (but
`admin restore --status`) and the deploy refuses to run. For `RestoreRecord.FollowersGrace` (14 days) after a restore,
FEP-8fcf rests: no `Collection-Synchronization` header goes out, and a follow only the remote remembers is adopted,
not undone.
- **The administrator's page** (`BackupController`, `/clientapi/admin/backups`; owner decision 2026-10-07, which approved
these endpoints in production): the list (with what runs, the restore waiting, the last restore's report, logins
deleted since each backup), back up now (202; the page polls), delete, and:
- **download** for the password: a ticket good for ten minutes in the link's path (`/download/{ticket}`, anonymous, so a
browser saves it to disk), the backup as one tar (`BackupTar`: its exact length known first, written from the
files, `Range` honoured);
- **upload** in pieces of at most 32 MB (`TransferStore`): each `PUT ?offset=` must start where the upload stands, else
409 with what was received, so a broken upload resumes; `finish` reads it into a backup (`kind` uploaded), refusing
anything but plain files under one backup's folder, a path leaving it, or more than the disk holds;
- **restore** for the password and the server's host typed out.
Every call checks the administrator against the database, not only the token. nginx streams downloads for an hour and
takes upload pieces unbuffered (`deploy/nginx`, applied by `setup.sh`).
- **A persona's archive** (`Domain/Portability/`, `PersonaArchiveController` at `/clientapi/persona/{avatarId}/archive`;
owner decision 2026-10-07), for the persona's own root only, never a banned one. State per persona in `PersonaArchive`
(never in a backup), files in `<backups>/.personas/<avatarId>/`, a week.
- **Export** (`ExportArchiveJob`): Mastodon's account-archive layout, so other servers' importers read it: `actor.json`
(its public key only), `outbox.json` (its own posts as Creates, addressed as delivered, but circles' and located
posts; its boosts as Announces), `media_attachments/files/` (what the attachments' urls name), avatar and header,
`likes.json`, `bookmarks.json` and Mastodon's CSV files; then `privapub/`: filters, followed hashtags, the
notification policy, pins, scheduled and located posts, `archive.json`. Nothing of its root, its siblings, its keys,
anyone's token. Downloaded through a ten-minute ticket in the link's path.
- **Import** (`ImportArchiveJob`, the parts the root picks, with progress, stoppable): uploaded in pieces like a backup,
read through `SafeArchive`, which refuses links, paths leaving it, names twice, too many or too large files, a file
compressed over 100:1, and streams the outbox an item at a time (none over 1 MB). Only the archive actor's own
Creates become posts; boosts, direct and circle posts are counted. **Imported posts are delivered to no one**, put in
no home and notify nobody (an exception next to located posts), yet show on the profile, the outbox, hashtags and
search. Back home (same actor) a post keeps its id and address, and one already here, even deleted, or tombstoned,
stays as it is; from another actor it gets an id of its date, its address here and `ImportedFromURI` (unique per
persona), so importing twice changes nothing. Replies and self-quotes inside the archive point at their copies; polls
come closed without votes; mentions stay links; HTML is sanitized; media go through `MediaService` and the quota.
The other parts go through the existing services: follows asked again (never on a blocked server), blocks, mutes,
blocked servers, lists (members only if followed), bookmarks (by signed fetch), filters, followed hashtags, the
notification policy, pins (no `Add` delivered), the profile (announced as Settings would). Located and scheduled
posts (PrivaPub's archives) and likes (each tells its author) only when asked. Followers are never imported.
## Code style
@@ -342,6 +587,8 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
`PrivaPub.Tests` (xUnit v3). Unit tests need nothing; tests marked `Category=Integration` need a mongod and skip
without `PRIVAPUB_TEST_MONGOD=1`. CI (`build.yml` and `deploy.yml`) runs all of them through
`tools/ci/with-test-mongod.sh`, which starts a throwaway mongod on a random localhost port and deletes it afterwards.
Like production's, it is a one-member replica set (`rs0`, reached with `directConnection=true`), so backups' snapshot
reads are what the tests exercise (`TopologyTests`).
The box's own mongods are production, so `MongoFixture` refuses port 27022, a data directory under `/var/lib/privapub`,
and in CI anything but the wrapper's mongod. With `PRIVAPUB_TEST_REQUIRE_MONGOD=1`, which the wrapper sets, a missing
mongod fails the run instead of silently skipping half the tests.
@@ -372,33 +619,299 @@ Beyond the tests, verify by building, running locally, and exercising:
Interop is checked against real servers, starting with the workstation's own pasture:
```bash
DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up # podman: PrivaPub + the latest GoToSocial + Mongo, behind Caddy
tools/pasture/interop.sh # 33 checks, each side driven through its own Mastodon API
tools/pasture/run.sh down
DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up [gts mastodon ...] # podman: PrivaPub + the named peers (default gts) + Mongo, behind Caddy
tools/pasture/interop.sh [gts mastodon ...] # each peer's scenario, then what PrivaPub's statistics saw of it
tools/pasture/run.sh down # removes every pasture container and volume
```
- **Two sites on one podman network, one Caddy in front.** `privapub.test` and `gts.test` are network aliases of the
Caddy container, which serves both with its internal CA (`tls internal`). Both servers are told to accept any
certificate: PrivaPub through `appsettings.Pasture.json`, GoToSocial through `GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`.
GoToSocial WebFingers and fetches over https only, so plain http between them is not an option.
- From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. GoToSocial is reached as `https://gts.test:6443`
with `curl -k --resolve gts.test:6443:127.0.0.1`, because its sign-in cookie is bound to the host name.
- **GoToSocial's cached home timeline can stop taking new posts after its first read**, its owner's own included, while
a `min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false`
- **A change to what PrivaPub sends** (the `@context`, an actor's properties, an activity's shape, a delivery's headers)
is run against every peer's scenario before it is committed, not only the peers it was written for. One reader
stricter than the rest fails on it alone: the 2026-10-06 `proof` term passed every peer but Smithereen, whose JSON-LD
1.0 reader refused every document carrying it.
- **Layout:** `lib/pasture.sh` (network, Caddy, Mongo, PrivaPub), `peers/<name>.sh` (`<name>_up`, plus `peers/shared.sh`
for the Postgres and Redis several peers share), `lib/interop.sh` (`ok`, `ko`, `xf` for a check expected to fail until
a later phase, `privapub_token <persona>`, `stats_check <host> <software>`), `scenarios/<name>.sh`. Each peer talks to
its own PrivaPub persona under one root, so OAuth's persona choice is exercised too. Images are pinned.
- Caddy's CA lives in the `pasture-caddy-data` volume and is copied to `tools/pasture/.ca/root.crt` (and `bundle.pem`
with the system roots) for peers that must trust it instead of skipping verification.
- **All sites on one podman network, one Caddy in front.** `privapub.test`, `gts.test`, `mastodon.test` and the other
peers are network aliases of the Caddy container, which serves them all with its internal CA (`tls internal`).
PrivaPub accepts any certificate (`appsettings.Pasture.json`) and GoToSocial is told to skip verification
(`GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`); Mastodon trusts the copied CA through `SSL_CERT_FILE`. Peers fetch over
https only, so plain http between them is not an option.
- From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. A peer is reached as `https://<name>.test:6443` with
`curl -k --resolve <name>.test:6443:127.0.0.1`, because sign-in cookies are bound to the host name.
- **GoToSocial (0.22.1):**
- Its cached home timeline can stop taking new posts after its first read, its owner's own included, while a
`min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false`
(`on_gts`), which answers from GoToSocial's database and never fetches from us. A home-timeline check there proves
nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a delete
is checked as a 404 on `/api/v1/statuses/{id}`.
- **GoToSocial creates its accounts locked**, so `interop.sh` approves alice's request through
`/api/v1/follow_requests`. That also checks our pending (`requested`) state and the manual Accept.
- The scenario covers:
- discovery and follows both ways;
- posts and CW;
- a reply and its notification;
- likes and boosts both ways;
- DMs both ways, and the DM staying off public timelines;
- edit, delete both ways, and unfollow.
nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a
delete is checked as a 404 on `/api/v1/statuses/{id}`.
- It creates its accounts locked, so the scenario approves alice's request through `/api/v1/follow_requests`, which
also checks our pending (`requested`) state and the manual Accept.
- 64 checks: discovery and follows both ways; posts and CW; a reply and its notification; likes and boosts both
ways; DMs both ways and off public timelines; polls both ways; quote policy; link cards; edits and deletes both
ways; communities and circles; locked personas; interaction policies (a reply and a like asked for and approved
through `/api/v1/interaction_requests`, a boost refused); unfollow; block and unblock; statistics.
- It files a circle post as a direct message and shows it only to the accounts it mentions, so like a DM it is
checked in the member's `/api/v1/conversations`, never by URI.
- **Mastodon (4.7.3):** web and sidekiq on the shared Postgres and Redis, `ALLOWED_PRIVATE_ADDRESSES` for the network.
Its token comes from `rails runner` (no password grant). Without Elasticsearch its status search finds nothing, so
deliveries are checked through `/api/v1/accounts/:id/statuses` of the sender as Mastodon knows them, or
`Status.exists?` through `rails runner`. Its actors are numbered (`/ap/users/<id>`), so look URIs up rather than
build them. Circle posts and a followers-only post survive its signed refetch (`ActivityPub::FetchRemoteStatusService`
through `rails runner`). Its block of a persona, and the unblock that ends the run, are checked through our
`blocked_by`; the run starts with that unblock too, since Mastodon rejects every follow from an account it blocks.
- **Misskey (2026.10.0):** one container on the shared Postgres and Redis. A new Misskey federates with nobody
(`federation: none`) until `admin/update-meta` says `all`, which `misskey_up` does. Its API is `POST /api/<endpoint>`
with the token as `i` (`mk` in the scenario); `users/relation` answers a list, `users/notes` leaves replies out unless
asked, and a user has one reaction per note. Never verify a delivery with `ap/show`: it fetches. 35 checks.
- **Sharkey (2025.4.7):** `peers/sharkey.sh` is the Misskey peer under another name, image, database, Redis db and
home (`/sharkey/.config`); `scenarios/sharkey.sh` runs Misskey's scenario with `MISSKEY_NAME=sharkey`, then follows
again (Misskey's ends unfollowed and blocked) and checks edits both ways and the FEP-e232 quote tag. 40 checks.
- **Akkoma (3.20.1):** no official image, so `images/akkoma` installs the OTP release (pinned by checksum; the "stable"
zip moves, and a moved one fails the build) and `akkoma_up` builds it once. Its HTTP clients read the CA bundles
shipped in the release (CAStore, certifi), so the entrypoint appends Caddy's CA there. `pleroma_ctl` passes its
arguments on unquoted: no value may contain a space. Its Linkify never takes `@user@host.test` for a mention, so the
scenario addresses alice with Pleroma's `to[]`; its API never reports a remote blocker as `blocked_by`, so the block is
read from its `user_relationships`. `akkoma_token <user>` signs one of its users in. 45 checks.
- **Lemmy (1.0.0-beta.2):** the backend alone on the shared Postgres, its admin made by `setup` in the generated
`config.hjson`. It trusts Caddy's CA through `SSL_CERT_FILE` and reaches the network through
`DANGER_FEDERATION_ALLOW_LOCAL_IP=1`; 0.19 cannot join (its rustls trusts only its bundled roots). Its API is
`/api/v4/<path>` with a bearer token (`lm` in the scenario), and `sort` values are lowercase. It logs no refused
activity at `warn` (`LEMMY_LOG` sets `RUST_LOG`); the reason is in the 400's body. It answers our community's echo of
its own activity and every bare `Announce{object}` 400 by design, and the echo is still needed (see
`docs/INTEROP.md`, Lemmy). A new Lemmy never sends what it queued for a server before its send worker for that
server started, so the scenario waits for that worker (`lm_worker`) before its first follow, and it sends what it
queued every 30 seconds, so a vote or a moderator's act takes up to a minute. Its lock, unlock and unban need a
`reason`, or it refuses them and federates nothing. 29 checks, among them a moderator's lock, ban and removal.
- **PieFed (1.7.17):** dockurr's image of the release, its web app (with PieFed's own cron, `CRON=true`, for its send
queue) and a Celery worker on the shared Postgres and Redis (dbs 10 and 11), sharing the `pasture-piefed-media`
volume. httpx trusts only certifi's bundle, so the pasture's is mounted over it. `flask init-db` reads its admin
(pfuser) from stdin and drops every table it finds, so it runs once, after the web app's migrations ("Starting
Gunicorn"). Its API is Lemmy's v3 under `/api/alpha` with a JWT (`pf` in the scenario); `resolve_object` answers a
view (`community.community.id`). It sends a community's announces to the Application at a peer's root, assuming
`/inbox` when there is none, and keeps serving a thread its moderator removed. `scenarios/piefed.sh`, 29 checks:
communities both ways, threads with titles, comments, votes up and down both ways, a community poll and alice's vote,
private messages both ways, a moderator's lock, unlock and removal, the unfollow, statistics.
- **Mbin (1.10.1):** its own image (FrankenPHP serving plain HTTP behind Caddy, `SERVER_NAME=:80`) and a messenger
worker (`mbin_worker`, consuming again whenever it stops: once, the shared Postgres restarted under it and Mbin sent
nothing for a day), on the shared Postgres and Redis (db 12) with a RabbitMQ of its own (its transports carry AMQP options; it
runs as its own user on its own volume, or it cannot read the `.erlang.cookie` it wrote as root). The pasture's bundle
is mounted over the system one; its API's rate limits (two threads every six minutes) are raised by a copy of its
`rate_limiter.yaml`. Its admin mbuser is made by its console; `peers/mbin_token.py` gets mbuser's OAuth token through
the authorization-code flow (login form, consent), since a client-credentials client acts as a bot that may not vote.
Mbin names what it makes during a request after the request's host, so every call says `Host: mbin.test`, never
the workstation's port. Its access token lasts an hour, and the scenario gets a new one when it has expired. Its API
never starts a conversation with an account elsewhere, so mbuser answers in the thread alice began; Mbin addresses
that answer to alice's profile page. `scenarios/mbin.sh`, 26 checks: magazines both ways, threads with titles, a Note
to a magazine as a microblog post, comments both ways, favourites and upvotes both ways, private messages both ways,
a moderator's lock, unlock and removal, the unfollow, statistics.
- **NodeBB (4.16.1):** the official image on the pasture's Mongo (database `nodebb`), set up once by its automated
setup (`SETUP` with `NODEBB_*` variables, admin nbuser) into the `pasture-nodebb-config` volume, trusting the CA
through `NODE_EXTRA_CA_CERTS`; its image runs `npm install` at every start. Its API (`/api/v3`) takes a bearer token
that `nodebb_settle` writes where NodeBB keeps its tokens. A remote account's uid there is its actor's URL, but its
API follows one only when named by its handle; a chat room is made, then written in. `scenarios/nodebb.sh`, 23
checks: a category followed and its topic as a titled thread, replies both ways, nbuser following alice, votes both
ways, an edit and a deletion, a chat both ways, the unfollow, statistics.
- **Lemmy 0.19 (0.19.20), as `lemmy19.test`:** what most of the threadiverse runs. Its release trusts only the roots its
rustls bundles, so `images/lemmy19` builds the tag with reqwest's `rustls-tls-native-roots` added (about ten minutes
the first time) and the pasture's bundle is mounted as the system's. Its config names the database as `uri` (1.0:
`connection`) and its API is v3 (`/api/v3`, `sort=New`, `resolve_object` answering views). It takes private messages
only as `ChatMessage`, which PrivaPub sends it, a first message too (invariant 17). `scenarios/lemmy19.sh`, 30
checks.
- **Threads (`scenarios/threads.sh`, needs mastodon):** mastouser follows alice, carol answers alice's public post,
and opening the thread on Mastodon finds carol's reply through alice's `replies` (the scenario makes Mastodon's copy
look ten minutes old: Mastodon reads a thread's replies only five minutes after it first holds the post). 5 checks.
- **Followers synchronisation (`scenarios/followsync.sh`, needs mastodon):** sent: PrivaPub forgets mastouser's follow
of alice (mongo), alice's followers-only post naming mastouser makes Mastodon read her roll-call and drop it; then
Mastodon forgets a follow (rails) and alice's next followers-only post makes it send the `Undo` PrivaPub applies.
Received: alice and bob follow mastouser; Mastodon forgets alice's follow and mastouser's followers-only post makes
PrivaPub end it, then PrivaPub forgets it and the next makes PrivaPub undo it there. Follows are checked in Mastodon's
database: its API caches relationships for a day. 15 checks.
- **Pins (`scenarios/pins.sh`, needs mastodon):** a Mastodon account pins and unpins while alice follows it, alice pins
and unpins while it follows her, a fresh account's earlier pin shows once PrivaPub resolves it, and following it brings
its earlier posts (its outbox). 9 checks.
- **Vernissage (1.43.0):** its image on SQLite in the `pasture-vernissage` volume, its queues on the shared Redis's
database 0 (it federates nothing without them; every other database is taken), its seeded admin (admin / admin)
logged in by the scenario. `scenarios/vernissage.sh`, 19 checks.
- **BookWyrm (0.9.3):** its image, migrated, on the shared Postgres (database `bookwyrm`) and Redis (databases 14
and 15), gunicorn and one Celery worker (federation runs there). It has no client API: bwuser (named
`bwuser@bookwyrm.test`, as its signup names users), a book and every status come from its Django shell
(`bookwyrm_shell`, `-v 0`). `scenarios/bookwyrm.sh`, 20 checks.
- **Ghost (6.67) with its ActivityPub service (1.2.14):** Ghost and the service on the shared MySQL (databases `ghost`
and `activitypub`, the service's migrations run first), routed by Caddy as Ghost's own proxy does. The owner comes
from Ghost's setup API (a password that repeats the site's name is refused), staff device verification is off, and
Ghost starts again once so it sets up its publishing webhooks. Calls to the service say `Host: ghost.test`, or it
cannot find the site's keys. `scenarios/ghost.sh`, 22 checks.
- **Owncast (0.3.0):** its image, its data in the `pasture-owncast` volume, federation turned on through its admin
API (`owncast_admin`, its default admin / abc123) with the server's address and the actor's name.
`scenarios/owncast.sh`, 13 checks.
- **WriteFreely (0.17.2):** built from its release tarball with `openssl` beside it (`images/writefreely`, it makes
each blog's keys with the command), on SQLite in the `pasture-writefreely` volume, which the scenario reads from the
host. wfuser and its blog come from its CLI; its API takes a login. `scenarios/writefreely.sh`, 13 checks.
- **snac2 (2.95):** built from its tag (`images/snac2`, the project publishes no image), its data in the
`pasture-snac` volume from `snac init` and `snac adduser`; the password comes from `snac resetpwd` and the token from
its login form and a code grant. Its API lags behind what it has taken in, so `scenarios/snac.sh` reads its files
(`snac_stored`, `snac_in_timeline`). 27 checks.
- **Mitra (5.9.1):** its image on the shared Postgres (database `mitra`), as `mitra.test`, with the pasture's bundle as
the system's roots; mitrauser comes from its CLI and its token from a password grant. Its search resolves an account
elsewhere only without `type`. `scenarios/mitra.sh`, 28 checks.
- **Relays (`peers/relay.sh` Activity-Relay 2.0.9 as `relay.test`, on the shared Redis's database 13;
`peers/aoderelay.sh` aode-relay 0.3.129 as `aoderelay.test`):** `appsettings.Pasture.json` names both in
`Federation:Relays`, so PrivaPub subscribes a minute after it starts. `scenarios/relay.sh` has Mastodon subscribe to
each in turn, checks that a post of an account nobody here follows reaches the federated timeline (forwarded by one,
announced by the other), that alice's public post goes to the relays (and through both to Mastodon: Activity-Relay's
forward on its FEP-8b32 proof, after the scenario makes Mastodon read alice's keys anew) and nothing less public, and
has Mastodon leave again, so the town sees no relayed posts. 16 checks.
- **Castopod (1.15.5, `peers/castopod.sh`):** the official image on the shared MySQL (database `castopod`, file cache,
`CP_DISABLE_HTTPS`), its database and superadmin (admin@castopod.test) from `spark`, its REST API switched on in the
`.env` the image rewrites at each start, `spark fediverse:broadcast` looping in the container. The podcast `@pod` is
made and published through its admin pages (`cp_web`, CSRF on each form). `scenarios/castopod.sh`, 15 checks; the
episode's sound is made with ffmpeg on the workstation.
- **Ktistec (3.13.0, `peers/ktistec.sh`):** built from its tag by `images/ktistec` (its own Dockerfile, pinned: a static
Crystal server, its assets with node), SQLite in the `pasture-ktistec` volume, the pasture bundle mounted as
`/etc/ssl/cert.pem`. The server's name and ktuser come from its setup API (every field required, `summary` too);
`ktistec_token` signs in on `/sessions`. `scenarios/ktistec.sh`, 27 checks: its Mastodon API where it has the route,
its outbox API (`kn`) for follows, polls (`poll-duration` is seconds from now), edits, deletions and quotes, and a
copy of its SQLite database (`kt_sql`) for what it holds.
- **Forte (26.9.10, `peers/forte.sh`):** built from its tag by `images/forte` (composer on Hubzilla's PHP image, an nginx
routing through `index.php?req=`), on the shared MySQL (database `forte`), its `.htconfig.php` written to
`.state/forte`, a cron sidecar (`src/Daemon/Run.php Cron`). The pasture bundle is mounted over `library/cacert.pem`,
the bundle its curl uses. ftuser (administrator) and ftfriend are made through its PHP (`forte_eval`: `Account::create`,
`Channel::create` with the role `social` and `autoperms`, or every connection waits). Connecting from the command line
runs the notifier itself (`ft_connect`). `scenarios/forte.sh`.
- **Hubzilla (11.4.1, `peers/hubzilla.sh`):** the hlhd image (php-fpm and nginx on 8080) on the shared MySQL
(database `hubzilla`, its schema loaded from the image), `.htconfig.php` written to `.state/hubzilla` and mounted, a
cron sidecar running `Zotlabs/Daemon/Master.php Cron` each minute. Its addons `pubcrawl` (ActivityPub) and `statistics`
(NodeInfo) are installed with `util/addons`; hzuser (the hub's administrator) and hzfriend are made through its own PHP
(`hz_eval`), as accounts written the way its registration queue would and channels with the role `public` (the older
role names grant connections nothing), the ActivityPub app installed as each channel. Its API (`api/z/1.0`) posts and
edits with HTTP Basic (email and password); liking and dropping go through its web pages signed in (`hz_web`).
`scenarios/hubzilla.sh`, 20 checks and one known gap (G-0010, its unfollow).
- **Smithereen (1.0.3):** its image on the shared MySQL (database `smithereen`, its schema from the image's commit),
with imgproxy and a file server behind Caddy as `smithereen.test` (`/i` and `/s`), trusting the CA through a JDK
store with it added (`JAVA_TOOL_OPTIONS`). MySQL takes its stored functions only with
`log_bin_trust_function_creators`, and a first start that failed there leaves the database without triggers (drop it
and start again). Accounts come from its signup form (opened, with a short description, which its NodeInfo needs) and
are renamed in its database; its API is VKontakte's (`/api/method/<name>?v=1.0`), with a password grant for a local
application the peer script inserts. `scenarios/smithereen.sh`, 40 checks, walls both ways (alice's profile is
told again first, so Smithereen reads her `sm:wall`; `servers.features & 1` says it took PrivaPub for a server with
walls). Its scenario texts avoid apostrophes: `p_home_has` puts them inside a Python string.
- **Load (`load.sh`, needs the `flood` peer):** `flood` (`flood/flood.cs`, published once into `.flood`) answers as
twenty fake servers and sends signed activities at a set rate; `load.sh --rate=N --seconds=N` measures the answers,
the queue's wait and processing times, its drain, and a persona's home timeline meanwhile, and keeps each run in
`out/load/`. `docs/LOAD.md` has the method and the runs.
- **Account moves (`scenarios/moves.sh`, needs gts):** two fresh GoToSocial accounts made by its admin CLI; alice follows
the old one, the new one names it as an alias (`/api/v1/accounts/alias`), the old one moves (`/api/v1/accounts/move`),
and PrivaPub shows it `moved` and moves alice's follow to the new one, which (locked by GoToSocial) approves it.
8 checks.
- **Hollo (0.9.19):** Fedify's microblog server on the shared Postgres, set up through its web form (which
checks `Origin` against `Host`, so the request names `hollo.test` without the port). It needs a 44-character
`SECRET_KEY`, a media directory and a `themeColor`; statuses and votes go as JSON. Town only, no scenario.
- **PeerTube (8.3.1):** the official image on the shared Postgres and Redis (db 4), configured through `PEERTUBE_*`
variables, trusting Caddy's CA through `NODE_EXTRA_CA_CERTS`; `peertube_settle` turns transcoding off (a test video
is served as uploaded) and keeps root's token in `.state/peertube.token`. It wants a bare `Host: peertube.test`.
Its list of comments held for review (`/api/v1/users/me/videos/comments?isHeldForReview=true`) answers 500 in 8.3.1;
the video's owner sees them in the video's threads (`heldForReview`). `scenarios/peertube.sh`, 28 checks.
- **Pleroma (2.10.2):** `images/pleroma` installs the OTP release, pinned by checksum, as Akkoma's does; it also needs
libvips, and `instance gen` asks about deduplicating uploads. Its federation runs on hackney, which trusts only
certifi's compiled-in roots, so the entrypoint points `:pleroma, :http, adapter` at the system CA bundle. It answers
an inbox POST at once and verifies in a worker (`oban_jobs` shows what it refused and why). Town only, no scenario.
- **Iceshrimp.NET (2026.1.2-beta):** on the shared Postgres with AuthorizedFetch on and open registrations, trusting
Caddy's CA through `SSL_CERT_FILE`. Accounts come from its own `/api/iceshrimp/auth/register` (its login cuts the
connection short for a name it does not know), Mastodon API tokens from its OAuth form, which prints the out-of-band
code in the page. Its `jobs` table shows what it queued for whom. Town only, no scenario.
- **Pixelfed (0.14.4):** serversideup's FrankenPHP image on the shared Postgres (database pixelfed) and Redis (dbs 5 and
6), with Horizon and the scheduler as sidecars of the same image sharing its storage volume, and Caddy's CA mounted
as its system bundle. `pixelfed_settle` makes `caption` nullable (its PostgreSQL migration never runs, so every remote
boost failed), creates Passport's keys and restarts the web server so FrankenPHP reads them, numbers the OAuth
clients from a million (Passport refuses a token whose user id equals its client's id), and imports its cities.
Tokens are personal access tokens made through tinker (`App\Models\User`). Every top-level post needs a picture.
`scenarios/pixelfed.sh`, 26 checks; the town's driver and `specs/pixelfed-pair.json`.
- **WordPress (7.1.2, ActivityPub plugin 9.3.1):** the official image on a shared MySQL 8.4 (`shared_mysql_up`, ready
only when it answers over TCP: its first start runs a server without networking), installed and configured by wp-cli,
its CA bundle (`wp-includes/certificates/ca-bundle.crt`) given Caddy's root, and WP-Cron run every five seconds by a
sidecar (`DISABLE_WP_CRON`), since the plugin federates from it. Its automatic updates are off
(`AUTOMATIC_UPDATER_DISABLED`): the sidecar ran one (6 to 7.1.2) whose new CA bundle dropped Caddy's root, and
WordPress reached nobody after it. Authors are actors; the REST API takes application passwords.
`scenarios/wordpress.sh`, 17 checks.
- **Friendica (2026.05):** the official image on the shared MySQL (database friendica) and Redis (db 7, for its cache,
locks and sessions), installed by its autoinstall, with the image's worker daemon (`cron.sh`) as a sidecar sharing
its files. `friendica_settle` repairs what the install leaves: it names the system user (uid 0), or Friendica never
makes the system account that signs its fetches; it declares the daemon running (`worker_daemon_mode` in
`key-value`), or the web container, which cannot see the sidecar's pid, never wakes it and everything waits for its
five-minute cron; it makes the log file and turns logging on. `friendica_user` saves each account through the API
with `locked=0` (a number: "true" reads as 0), which makes it a "soapbox" page that takes followers without asking and
follows nobody back, and reads its outbox once: a Follow that reaches a user Friendica has not cached yet makes it
fetch that user from itself, signed, and checking the signature recurses for about five minutes. Its API takes HTTP
Basic (`nick:password`); an edit through it never federates, so the scenario edits in the web editor.
`scenarios/friendica.sh`, 25 checks; the town's driver and `specs/friendica-pair.json`.
- **Mobilizon (5.2.4):** kaihuri's image of the release on a PostGIS of its own (`shared_postgis_up`: it needs the
extension), with the pasture's bundle mounted over certifi's and castore's CA files (hackney trusts only those) and
the system store, and geocoding pointed at a closed local port. `mobilizon_ctl users.new` makes mzuser (it splits
its arguments on spaces); the API is GraphQL (`/api`), signed in by the `login` mutation. An event made through it
without options has its comments closed. `scenarios/mobilizon.sh`, 27 checks: a group and its events (dates, place),
alice joining an event (Mobilizon's participant row and its Accept) and leaving it, comments both ways, the
organiser's edit, closed comments and deletes, a group post, the unfollow.
- **Gancio (1.28.2):** cisti's image on sqlite in the `pasture-gancio-data` volume, whose `config.json` is written before
the first start (without it Gancio waits in its setup wizard), trusting Caddy's CA through `NODE_EXTRA_CA_CERTS`.
`gancio users create` makes gcadmin; `gancio settings set enable_resources true` keeps fediverse replies. One
Application actor, `relay`, publishes the agenda. Its API takes a token from an OAuth password grant
(`/oauth/login`, client_id `self`); an event is a multipart POST (`--form-string` for text starting with `<`).
`scenarios/gancio.sh`, 17 checks.
- **Funkwhale (2.0.11):** the API (gunicorn), a Celery worker with its beat and the front's nginx, sharing the
`pasture-funkwhale-data` volume, on the shared Postgres (database funkwhale) and Redis (dbs 8 and 9); Python's requests
trusts the pasture's bundle through `REQUESTS_CA_BUNDLE`. `funkwhale-manage fw users create` makes fwuser and an OAuth
access token is made in its Django shell. Its API is `/api/v1` and `/api/v2` (uploads); a track is a fresh tone made
by ffmpeg in its container, since it skips a file it already has. `scenarios/funkwhale.sh`, 16 checks.
- **SecureMode:** `PRIVAPUB_ENV="Federation__SecureMode=true" run.sh up …`, as production runs. A check of what an
unsigned reader sees uses `unserved` and `gone_unsigned` (`lib/interop.sh`), which expect 401 when SecureMode is on and
404 or 410 when it is off.
- **Crawler:** `PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test"
run.sh up mastodon`, then `interop.sh crawler`. `PRIVAPUB_ENV` passes any setting to the PrivaPub container.
- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into
`tools/pasture/.publish` and `podman restart pasture-privapub`; that is how a migration is tried on dirty data.
- `run.sh add <peer>` starts more peers next to a running pasture; `run.sh down` removes every pasture container with
its volumes (`--purge` drops Caddy's CA too); `run.sh stats` shows each container's memory.
- **The network looks public** (`PASTURE_SUBNET`, default `11.42.0.0/24`), so peers that refuse private addresses with no
switch (Pixelfed, Mbin, WordPress, Discourse, Fedify) federate. Mastodon then needs `TRUSTED_PROXY_IP` for Caddy, or it
answers every request with an IP spoofing error.
- **`PASTURE_PORT`** moves PrivaPub's workstation port off 6971 when something else holds it; `lib/interop.sh` follows.
- Every `ok`/`ko`/`xf` is also a line of `out/scenarios.jsonl`, which the town's report shows.
#### The town (`tools/pasture/town.sh`, `tools/pasture/town/`)
A fake community seeded across every running peer, then checked for coherence: what each server holds (read from its
database, never by making it fetch), what each account sees through its API, counts, threads, edits, deletes, follows
and the privacy rows. Python 3 standard library only, tabs.
```bash
tools/pasture/town.sh selftest [peer...] # each driver against its own server; run after changing a driver or a pin
tools/pasture/town.sh seed village # specs/village.json: 23 accounts on seven servers, about 11 minutes
tools/pasture/town.sh check village # out/<run>/results.json; --deadline=0 re-sweeps an old run at once
tools/pasture/town.sh report village # out/<run>/report.html, publishable (fake data, no tokens)
tools/pasture/town.sh report village hollo-pair iceshrimp-pair pleroma-pair pixelfed-pair friendica-pair # one matrix
tools/pasture/town.sh backlog --write # docs/INTEROP-BACKLOG.md
```
- **Drivers** (`dialects/`): one class per platform on a dialect base (`mastodon_api`, `misskey_api`, `lemmy_api`,
`privapub`). `stored()` reads the database (shared Postgres through `psql`, the shared MySQL through `mysql_json`,
GoToSocial's sqlite copied out, Mongo);
`seen()` asks the API as one account. A driver that cannot do something raises `Unsupported`, and the planner never
asks for it (`gen.CAPS`).
- **The plan** (`gen.py`) is a pure function of the spec and its seed; each run keeps its own `plan.json`, `ledger.jsonl`
(what happened, one fact per line) and `results.json`. `seed --resume` redoes only failed steps.
- **Cells** are `feature|origin|observer|direction` (`in`, `out`, `via`, `control`, `local`). A known gap is a
`gaps.json` entry: matching failures are `xfail`, matching passes `xpass` (close the gap).
- **What the peers do on purpose**, so the checker expects it: Misskey and Sharkey drop a reply whose parent they cannot
fetch; Lemmy keeps only community content and private messages; servers send edits and deletes to the post's own
audience only; a Misskey heart reaction arrives as a Like.
- Peers' own limits are lifted for the town: GoToSocial `GTS_ADVANCED_RATE_LIMIT_REQUESTS=0`, Lemmy's
`local_site_rate_limit`. Mastodon reserves usernames that contain `mastodon`.
- decePubClient's e2e tests (`decePubClient/tests/e2e/run.sh`) publish the client with the Pasture environment, serve
it as `https://decepub.test` (`peers/decepub.sh`) and write their cells to `out/client.jsonl`.
After the pasture, verify.funfedi.dev and the owner's GoToSocial at social.arasaka.software. **Ask before acting from
the owner's GoToSocial account.**
@@ -406,13 +919,22 @@ the owner's GoToSocial account.**
## Deploy
- **CI/CD:** push to `master` runs `build.yml` (build + tests) on the instance-wide `build` runner. A `v*` tag runs
`deploy.yml`: tests, self-contained linux-x64 publish, snapshot and `mongodump` to `/var/backups/privapub.thepra.dev`,
`deploy.yml`: tests, self-contained linux-x64 publish, a snapshot of the site to `/var/backups/privapub.thepra.dev`,
the server's own pre-deploy backup (`PrivaPub admin backup --kind pre-deploy --db-only`, run from the new build before
its migrations, verified, and checked to hold no statistics salt; no deploy while `restore.json` waits),
stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo,
Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration, client credentials,
discovery, instance, public timeline; pass a persona token as a second argument to check the signed-in side).
discovery, instance, public timeline). Then it checks that what production should be running is running:
- NodeInfo and the instance API agree on registrations (closed, invitations only);
- `/stargazing` says the crawler is on;
- **@thepra signs in**: `PrivaPub admin smoke thepra` gives the root `deploy-smoke` a new password, `tools/smoke/oauth.sh`
runs the real OAuth flow (the pasture's `privapub_token` uses the same script), the signed-in API is checked, the
persona must be undiscoverable, and the token is revoked;
- the DB-IP Lite databases, which the server fetches itself (`GeoUpdater`), exist and are at most 40 days old.
- **The box:** Max (`nuvola.xyz`). Unit `privapub` runs as www-data from `/var/www/privapub.thepra.dev` with
`ASPNETCORE_ENVIRONMENT=Production`.
- **One-time root setup:** `deploy/max/setup.sh`, run through `../arasaka.software/tools/max/run.sh`.
- **One-time root setup:** `deploy/max/setup.sh`, run through `../arasaka.software/tools/max/run.sh` (directories, the
runner's sudoers line, the units, nginx and the certificate). Nothing that runs later needs it again.
- **Config:** `appsettings.Production.json` is committed and deployed, **secrets included, by the owner's convention**
(the same as arasaka.software). A hand edit on the box is lost at the next deploy.
- **Secrets drift:** `AppConfigurationService` copies `AppConfiguration` into Mongo on first boot and reads the stored
+286 -37
View File
@@ -8,33 +8,100 @@ PrivaPub is an ActivityPub server written in C#. This document follows
- [ActivityPub](https://www.w3.org/TR/activitypub/) (server-to-server)
- [WebFinger](https://webfinger.net/)
- [HTTP Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures), `rsa-sha256` / `hs2019` with RSA keys
(`hs2019` hashed with SHA-256, or SHA-512 as some sign it; a `SHA-256=` or `SHA-512=` digest)
- [HTTP Message Signatures](https://www.rfc-editor.org/rfc/rfc9421) (RFC 9421) and Content-Digest (RFC 9530), verified on
deliveries and signed fetches: `rsa-v1_5-sha256` and `rsa-pss-sha512` with RSA keys
- [NodeInfo](https://nodeinfo.diaspora.software/) 2.0 and 2.1
- Object integrity proofs ([FEP-8b32](https://codeberg.org/fediverse/fep/src/branch/main/fep/8b32/fep-8b32.md),
`eddsa-jcs-2022`, JSON canonicalised by [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785)) by Ed25519 keys published
as Multikeys ([FEP-521a](https://codeberg.org/fediverse/fep/src/branch/main/fep/521a/fep-521a.md)), on what personas
send to relays and verified on what is forwarded
## Tested against
- **GoToSocial 0.22.1, end to end.** It runs in a private network on the workstation (`tools/pasture/`) and is driven
through its own client API. Checked both ways:
- follows, including to a locked account;
- public posts with a content warning;
- replies with notifications;
- likes and boosts;
- direct messages;
- edits and deletes;
- unfollow.
- **Mastodon, Misskey, Lemmy and PeerTube, by unit tests only.** The tests feed the parser documents written in each
server's shape. No live exchange with any of them has run yet.
End to end, in a private network on the workstation (`tools/pasture/`), each peer driven through its own client API
and every run starting clean, with signed fetches required (as privapub.thepra.dev runs):
- **GoToSocial 0.22.1**
- **Mastodon 4.7.3**
- **Misskey 2026.10**
- **Sharkey 2025.4.7**
- **Akkoma 3.20.1**
- **Lemmy 1.0.0-beta.2**, and **Lemmy 0.19.20** (built with the platform's roots, so it trusts the pasture's CA)
- **PeerTube 8.3.1**
- **Pixelfed 0.14.4**
- **WordPress 7.1.2 with ActivityPub 9.3.1**
- **Friendica 2026.05**
- **Mobilizon 5.2.4**
- **Gancio 1.28.2**
- **Funkwhale 2.0.11**
- **PieFed 1.7.17**
- **Mbin 1.10.1**
- **NodeBB 4.16.1**
- **Smithereen 1.0.3**
- **Mitra 5.9.1**
- **snac2 2.95**
- **WriteFreely 0.17.2**
- **Owncast 0.3.0**
- **Ghost 6.67** with its ActivityPub service 1.2.14
- **BookWyrm 0.9.3**
- **Vernissage 1.43.0**
- **Hubzilla 11.4.1** with its pubcrawl addon
- **Forte 26.9.10** (portable identities: actors served through `/.well-known/apgateway/did:key:…`)
- **Castopod 1.15.5** (podcasts as actors, their episodes with the sound)
- **Ktistec 3.13.0**
- **Activity-Relay 2.0.9** and **aode-relay 0.3.129**, as relays PrivaPub reads from
- in the town only (a seeded community checked server by server): **Hollo 0.9.19**, **Iceshrimp.NET 2026.1.2-beta**,
**Pleroma 2.10.2**
Checked both ways, where the peer has the feature:
- follows, locked accounts included;
- every visibility, content warnings, replies and their notifications;
- likes, boosts and emoji reactions with their undos;
- direct messages;
- polls;
- quotes;
- images with alt text;
- edits with history and deletes;
- communities and circles;
- blocks and unfollows.
`docs/INTEROP.md` has each peer's evidence and what is still expected to fail. The platforms not yet in the pasture are
covered by unit tests written in their documents' shape.
## Supported FEPs
- [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md)
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
- [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md)
- [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md):
every actor names its handle, and a peer's `webfinger` on another domain than its actor's host (Mastodon's
`LOCAL_DOMAIN`) is the handle shown once that domain's WebFinger points back to the actor; asked again on a rename
- [FEP-d556: Server-Level Actor Discovery Using WebFinger](https://codeberg.org/fediverse/fep/src/branch/main/fep/d556/fep-d556.md)
(WebFinger for the server's origin links its instance actor as `…#Service`) and
[FEP-2677: Identifying the Application Actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2677/fep-2677.md)
(`/.well-known/nodeinfo` links it as `…#Application`)
- [FEP-844e: Capability discovery](https://codeberg.org/fediverse/fep/src/branch/main/fep/844e/fep-844e.md): the instance
actor's `implements`, and every other actor's `generator`, name RFC 9421 (verified with RSA keys)
- [FEP-1b12: Group federation](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md) (communities; see "Groups")
- [FEP-044f: Consent-respecting quote posts](https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md)
(`QuoteAuthorization` at `/parrot-licences/{id}`; checked with Mastodon both ways)
- [FEP-9967: Polls](https://codeberg.org/fediverse/fep/src/branch/main/fep/9967/fep-9967.md)
- [FEP-c0e0: Emoji reactions](https://codeberg.org/fediverse/fep/src/branch/main/fep/c0e0/fep-c0e0.md) (`EmojiReact`, and Misskey's
`Like` with content)
- [FEP-5feb: Search indexing consent](https://codeberg.org/fediverse/fep/src/branch/main/fep/5feb/fep-5feb.md) (`indexable`)
- [FEP-521a: Representing actor's public keys](https://codeberg.org/fediverse/fep/src/branch/main/fep/521a/fep-521a.md) and
[FEP-8b32: Object Integrity Proofs](https://codeberg.org/fediverse/fep/src/branch/main/fep/8b32/fep-8b32.md) (see
"Integrity proofs")
- [FEP-171b: Conversation Containers](https://codeberg.org/fediverse/fep/src/branch/main/fep/171b/fep-171b.md), received:
a thread owner's `Add` of someone's `Create`, `Update` or `Delete` to the thread's context is taken as that activity
passed on by the owner (on its FEP-8b32 proof, or as its origin has it)
- [FEP-8fcf: Followers collection synchronization across servers](https://codeberg.org/fediverse/fep/src/branch/main/fep/8fcf/fep-8fcf.md)
(sent and honoured; see "Followers synchronisation")
- [FEP-5624: Per-object reply control policies](https://codeberg.org/fediverse/fep/src/branch/main/fep/5624/fep-5624.md):
read (see "Replies its author approves")
Planned (see `docs/ROADMAP.md`, phases P5 to P8, and the per-platform notes in `docs/INTEROP.md`): FEP-044f (quotes),
FEP-9967 (polls), FEP-c0e0 (emoji reactions), FEP-9098 (custom emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move),
FEP-8fcf (followers synchronisation), FEP-5feb (`indexable`), FEP-8967 (link attachments), FEP-521a and FEP-8b32 (keys
and integrity proofs), FEP-ae0c (relays).
Planned (see `docs/ROADMAP.md`, phases P7 and P8, and the per-platform notes in `docs/INTEROP.md`): FEP-9098 (custom
emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), FEP-8967 (link
attachments), FEP-ae0c (relays).
## Actors
@@ -52,15 +119,23 @@ actor document, a collection, NodeInfo or a delivery relates two avatars of the
| Activities | `/peasants/{name}/grunts/{id}` |
| Quote permissions (`QuoteAuthorization`) | `/peasants/{name}/parrot-licences/{id}` |
| Direct-message context | `/peasants/{name}/whispers/{id}` |
| A persona's wall (FEP-400e) | `/peasants/{name}/graffiti` |
| Followers on the asking server (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` |
| Profile and post pages | `/@{name}`, `/@{name}/{id}` |
The names are the project's own and are stable; resolve actors through WebFinger, not by guessing a path.
- The key is `{actor}#main-key`, RSA 2048, served as SPKI PEM with `owner` set to the actor.
- `published` on an actor is a whole day, chosen at random up to two weeks before the account was made, so two
personas made on the same day do not share a date. `indexable` is `false`.
personas made on the same day do not share a date. `indexable` is `false` unless the persona turns it on, and
`discoverable` is `true` unless it turns that off (the deploy's own persona, @thepra, is undiscoverable).
- The instance actor is `/peasants/privapub` (type `Application`). It signs every fetch PrivaPub makes, so no avatar's key
is used to read another server's content.
is used to read another server's content. It also answers at the server's root (`/`) for a request that asks for
ActivityPub, as Lemmy's site actor does: PieFed sends a community's announces to the inbox of the Application at a
peer's root, and to `/inbox` when there is none.
- The reporter is `/peasants/privapub_reports` (type `Service`, "Reports from <host>"), one for the whole server with
its own key. It sends the reports Lemmy, PieFed and Mbin take in Lemmy's shape (see **Reports** below) and does
nothing else: nobody follows or mentions it, the Mastodon API has no account for it, and it has no page.
## Groups
@@ -68,16 +143,31 @@ A group is either a **community** or a **circle**.
- A **community** follows [FEP-1b12](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md). Posts
addressed to it (in `to`, `cc` or `audience`) are accepted according to its posting policy (followers, anyone, or
moderators only) and the group `Announce`s the whole activity, with `audience` set, to its followers. A new post is
moderators only) and the group `Announce`s the whole activity, with `audience` set, to its followers and to the
author's own server, whether anyone there follows it or not (Lemmy keeps its user's post pending until then). A new post is
also announced as an object so Mastodon shows it. Updates and deletes of community content are announced too. A
top-level post is a `Page` with a `name`. Members are counted at `/flock`; moderators are listed at `/wardens`, which
the actor's `attributedTo` points to, with `postingRestrictedToMods` as Lemmy expects. A mention of a community posts into it.
- A **circle** is private. Its actor is not discoverable and every follow is a request. Its posts are addressed to the
circle and its members collection, delivered to each member's own inbox and never announced. They are served only
to a signed request from a member, or from the instance actor of a member's server; anyone else gets 404. A
Mastodon member's replies reach only the people they mention.
circle and its members collection, delivered to each member's own inbox and never announced. Each member's copy also
names that member in `cc`, so servers that keep only posts naming one of their accounts (Mastodon, GoToSocial) keep
it; it names no other member. The posts are served only to a signed request from a member, or from the instance actor
of a member's server, and that copy names the member (or the members on that server); anyone else gets 404. A
Mastodon member's replies reach only the people they mention. Its owner lets a request in with an `Accept` of the
Follow, and declines one or takes a member from elsewhere out with a `Reject` of it, as Mastodon removes a follower.
- Announces from **remote** groups (Lemmy communities) are followed through to the activity: the object is fetched
from its own origin, never taken from the announce.
from its own origin, never taken from the announce. They are taken from a group someone here follows, and also when
they relay a vote on one of our posts in a thread of that group (Lemmy sends a vote to the community alone). A post
keeps the group its `audience` names, however it arrived.
- A post in a remote group followed here that the group's own server delivers itself, as its author's `Create` with the
group as its `audience` (Mbin sends a magazine's threads that way, with no announce), is kept as if announced: the
group's server speaks for posts in it. The same from another server is not.
- A remote community's **moderation** reaches the posts it holds. A removal (`Announce{Delete}`) is believed at once
from a community on the post's own server, which speaks for it (PieFed keeps serving a thread its moderator removed),
and from a community elsewhere once the post's origin answers it gone. A lock sent as it is (`Lock`, Mbin) is taken
from the post's own server only. A lock (`Announce{Lock}`, or `commentsEnabled: false` on the post) refuses replies,
ours included, until `Undo{Lock}`. A ban of a persona (`Announce{Block}` with the community as `target`) shows as
`blocked_by` on the community and refuses the persona's posts and replies there until the `Undo`.
## Activities
@@ -86,28 +176,87 @@ Received:
| Activity | Effect |
|---|---|
| `Follow` | follows an avatar or community; `Accept` is sent unless the community approves members by hand |
| `Accept{Follow}`, `Reject{Follow}` | completes or ends a follow an avatar requested |
| `Accept{Follow}`, `Reject{Follow}` | completes or ends a follow an avatar requested; once a follow holds, the account's latest public posts are read from its outbox's first page (twenty at most, once a day), so its profile is not empty; homes get only what arrives from then on |
| `Accept{Join}`, `Reject{Join}` | from the event's server: a persona's participation in the event is accepted or refused |
| `Undo{Follow, Like, Announce}` | reverses it |
| `Create{Note, Article, Page, Question, Video, Audio, Event, ChatMessage, …}` | stored when a local avatar follows the author, is addressed or mentioned, when it replies to a local post, or when it is addressed to a community the author follows; a public parent is fetched to complete the thread |
| `Create{Note, Article, Page, Question, Video, Audio, Event, ChatMessage, …}` | stored when a local avatar follows the author, is addressed or mentioned, when it replies to a local post, or when it is addressed to a community the author follows (a persona is addressed by its actor id or by its profile page, as Mbin addresses its private messages); a public parent is fetched to complete the thread |
| `Update{Note}` | replaces the content; the previous version is kept |
| `Update{Person}` | refetches the actor |
| `Like` | counted and notified, on posts the liker could see |
| `Dislike` | counted as a downvote (Lemmy, PieFed, Mbin, Friendica); `Undo` takes it back |
| `EmojiReact`, `Like` with an emoji `content` | an emoji reaction (FEP-c0e0; Pleroma, Akkoma, Iceshrimp.NET, Misskey, Sharkey), Unicode or a custom emoji from its `tag`; a `Like` whose content is ❤ stays a favourite; `Undo` takes it back |
| `Join` | answered with `Ignore`: PrivaPub hosts no events yet (FEP-8a8e) |
| `Join` | answered with `Ignore`: PrivaPub hosts no events of its own yet (FEP-8a8e) |
| `Announce` | counted and notified for local posts; shown to followers of the announcer, with the original refetched from its origin |
| `Delete` | deletes the object, or the actor and its follows; a deleted object id is remembered for 90 days, so a late `Create` cannot bring it back |
| `Delete` | deletes the object, or the actor and its follows; a deleted object id is remembered for 90 days, so a late `Create` cannot bring it back, though a new post its server publishes under that id after the deletion (Gancio numbers events from the last one kept) is kept |
| `Flag` | becomes a report for this server's moderators |
| `Move` | an account moving: believed as Mastodon believes it, when the account sends it about itself and the new account, read again from its server, names it in `alsoKnownAs`. The old account then shows where it went (`moved`), and, as Mastodon does it (owner decision 2026-10-05), the personas following it follow the new one instead (a Follow to its server, an Undo to the old), in the same lists; a mute or a block of the old account carries over |
| `Add`/`Remove` on the actor's `featured` | the account pins or unpins one of its own posts (fetched from its server when not held); inside a community's announce, the community features a post made in it. Its profile shows them first (`pinned=true`). The collection itself is read with the account's counts, at most once a day. On the account's wall (`sm:wall`), see "Walls". Any other target (a community's moderators) is dropped; one on the account's own server that PrivaPub does not know has its document read again first, at most hourly |
| `Block` of a persona | the follows between them end, the blocker's posts and notifications are hidden from the persona, nothing of the persona's is addressed to the blocker, and the relationship says `blocked_by`; `Undo{Block}` lifts it |
Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Create{Question}` and poll votes, `EmojiReact` and its `Undo`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`,
`Reject{Follow}`, `Like`, `Announce` and their `Undo`, `Flag`. A deleted post answers 410 with a `Tombstone`.
`Reject{Follow}`, `Like`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event, and the replies to a
persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`.
- **Attachments** are `Document`s with `mediaType`, `name` (alt text), `blurhash`, `focalPoint`, `width` and `height`.
Uploaded files have all metadata removed.
- **Pinned posts** are the actor's `featured` collection (`/trophies`); `featuredTags` is `/tattoos`.
- **Threads** (owner decision 2026-10-06). A persona's public or unlisted post outside any group names its `replies`
(`…/scribbles/{id}/replies`) and its conversation, `context` (FEP-7888): the root post's `…/scribbles/{id}/context`,
which a reply inherits from its parent, ours or another server's. Both list only the public and unlisted posts
PrivaPub holds, the context every one under the root (at most 500); followers-only, circle, direct and local-only posts
name neither, and their collections answer 404. Mastodon completes a thread from them.
- **Followers synchronisation** (FEP-8fcf, owner decision 2026-10-06). A persona's delivery addressed to its followers
carries a signed `Collection-Synchronization` header: its `followers`, its roll-call (`…/groupies/roll-call`) and the
digest (XOR of each id's SHA-256) of its accepted followers on the receiving server only. The roll-call answers a
signed request with the persona's followers on the signer's server and nobody else's; unsigned, 401. A server whose
view differs mends itself from it, Mastodon both ways (checked live). Mastodon's `Undo{Follow}` for a follow it never
knew of has one id per account (`…#follows//undo`); when it comes again after a new follow, it ends that follow too.
Honoured the other way: when a delivery's header digests the sender's followers here otherwise than the personas
following it, PrivaPub reads the list it names (same origin, signed by the instance actor). A follow the list leaves out
ends, but only when the list is the one the digest describes; a request it lists is taken as accepted; a persona it
lists that follows nothing there sends `Undo{Follow}`, as Mastodon does.
- **Walls** (FEP-400e, Smithereen's; owner decision 2026-10-06). A persona's actor names its `wall`
(`…/graffiti`, `sm:wall`) and Smithereen's `privacySettings`: its followers may write on it (`wallPosting`), everyone
may read it (`wallPostVisibility`). A public post that is not a reply, sent with the wall as its `target` by an account
following the persona, is hosted: the persona is notified (as a mention), it reaches the persona's home and its
followers' here, and the persona's followers' servers and the author's are told with `Add{Note}` (the wall as the
target). Anything else written there is dropped. The persona may delete it (`DELETE /api/v1/statuses/:id`), which
sends `Remove{Note}` with the wall as a plain id; Smithereen deletes the post then, as it does when an owner deletes a
post on its wall. The wall lists the persona's public posts that start a thread and what was written on it, newest
first. An account elsewhere that tells its followers here, with `Add{Note}` on its own wall, of a post written there
has it shown to them as on its wall (`privapub.wall` on the status); a `Remove` there takes it away.
- **Pinned posts** are the actor's `featured` collection (`/trophies`); `featuredTags` is `/tattoos`. A pin or an unpin
is told to the post's audience as `Add` or `Remove` on `featured`, as Mastodon tells it.
- **Blocks are sent.** A blocked remote account receives `Block` from the blocking account (and `Reject{Follow}` if it
followed); an unblock sends `Undo{Block}`.
- **Reports** are sent as `Flag` by the instance actor, never by the reporting account.
- **Reports** are sent as `Flag`, never by the reporting account and never naming it.
- To the reported account's server: from the instance actor, with the account and the posts as `object` and the
persona's words in `content`.
- To a community on a server whose NodeInfo names Lemmy, PieFed or Mbin (owner decision 2026-10-06, the second place
PrivaPub decides by a server's software; PieFed and Mbin once the pasture showed each keeps such a report with its
reason): one `Flag` per reported post or comment that was made in that community (its own
`audience`, else its thread's), from the reporter, `to` the community, the post alone as `object`, the words (or,
with none, the category) in `summary` and `content`, sent to the community's inbox. Lemmy refuses a Flag from an
`Application`, with no `to` or with no reason; PieFed drops one from an `Application`; Mbin reads the reason from
`summary` alone. Such a server gets no instance actor's Flag: an account alone, or a post outside its communities,
is not reported there, since Lemmy takes neither (PieFed would take an account alone from the reporter; that is not
sent yet).
- **Direct messages** go out as a `Note` addressed to their recipients, except a message to one account elsewhere that
writes to us as `ChatMessage`s (Pleroma's type), or whose server takes nothing else: Lemmy before 1.0 and Mbin, as
their NodeInfo names them (owner decision 2026-10-05, the first place PrivaPub decides by a server's software). That
message goes out as a `ChatMessage`, to the account alone, without a mention in its text.
- **Replies are passed on** (inbox forwarding; owner decision 2026-10-05). A public or unlisted reply from another server
to a persona's public, unlisted or followers-only post goes on to the persona's followers as its author's server sent
and signed it, the way Mastodon forwards it: to every follower's server but the replier's own. Its `Update` and
`Delete` follow the same way. Nothing is passed on for a local-only post, a group's post or a reply that is not public
or unlisted. A server that receives it checks it as it checks any forwarded activity: Mastodon, Misskey and Sharkey
keep it only when its author LD-signed it (Mastodon does; Misskey 2026.10 and Akkoma do not), others read the reply
from its origin.
- **Events** (owner decision 2026-10-05). A persona joins another server's event with a `Join` and leaves it with a
`Leave`, both sent to the event's organiser only (`…/grunts/join-<id>`, `…/grunts/leave-<id>`). The organiser's
server answers `Accept` (Mobilizon at once for an open event) or `Reject`, which the persona sees as
`privapub.event.participation`: `pending`, `accepted` or `rejected`. An event that takes participants by invitation
or on another site (Mobilizon's `joinMode` `invite` or `external`) is refused before anything is sent (422). Who
takes part is public on the event's server.
A `Create`'s `Note` carries Mastodon's `content`, `contentMap`, `summary` and `sensitive`, plus `Mention` and `Hashtag`
tags.
@@ -117,6 +266,8 @@ tags.
content, one per choice, counted once per voter. Counts are refreshed with an `Update{Question}` at most every three
minutes. Our own votes on other servers' polls are sent the same way to the poll's author only, without `published`.
An incoming `Update` without a newer `updated` only refreshes counts and details; it is never recorded as an edit.
A post's earlier versions in `formerRepresentations` (Pleroma, Akkoma) are its edit history, read as the post itself is:
a post first met after its edits has them, and an edit that carries them brings the versions missed in between.
**Quotes** (FEP-044f, with the older keys):
- **Received.** A quote is read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 `Link` tag.
@@ -139,6 +290,28 @@ An incoming `Update` without a newer `updated` only refreshes counts and details
`Delete{stamp}`.
- Followers-only posts and DMs can never be quoted.
**Interaction policies** (GoToSocial's `canReply`, `canLike`, `canAnnounce`, with the older `always` and
`approvalRequired`):
- **Read** on every remote post; one left out means anyone, automatically. A persona is let in at once when the rule
names the public, the persona itself, the author's followers while it follows the author, or the accounts the author
follows while the author follows it.
- **Shut out:** the client API answers 422 to a reply, favourite or boost the rule does not allow at all.
- **Asked first:** a `ReplyRequest`, `LikeRequest` or `AnnounceRequest` (the interaction as its `instrument`) goes to the
author alone, and the interaction waits (`privapub.approval: pending`). The author's `Accept` brings an authorization
(`result`), which must be on the author's origin, attributed to them and naming the interaction and the post; only then
does the reply go out with `replyAuthorization`, the boost with `announceAuthorization` and the like with
`likeAuthorization`. A `Reject` leaves it ours alone (`rejected`), and takes a like or a boost back.
- **As a third party,** a reply that a post's policy does not let in at once is kept only with an authorization that
verifies the same way. A rule naming a collection we cannot list (followers, following) lets the reply in.
- **Shown** to clients as GoToSocial's `interaction_policy` (`can_favourite`, `can_reply`, `can_reblog`).
- Our own posts state only `canQuote`: anyone may reply to, like and boost them.
**Replies its author approves** (FEP-5624's `canReply`, as PeerTube sets it on a video whose comments are moderated; a
`null` one says nothing): a persona it names, or that the post mentions, may reply, and the reply waits
(`privapub.approval: pending`), its `Create` going to the author alone. The author's `ApproveReply`, signed by the author
and naming the post answered, lets it out to its audience with `replyApproval`; a `RejectReply` leaves it ours alone
(`rejected`). An empty `canReply` refuses every reply (422).
**Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object.
**Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`,
`memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
@@ -162,6 +335,35 @@ The page's OpenGraph and Twitter tags give the title, description and image; the
days and shared by every account on the server. Images are served to clients only through PrivaPub's media proxy.
`Federation:FetchLinkPreviews=false` turns page fetching off.
## Relays
PrivaPub reads from the relays `Federation:Relays` names (by their actor's address, or their inbox's), none by default.
Its instance actor follows `Public` at each, as Mastodon subscribes (`Federation/Relays/Relays.cs`, a minute after start
and every six hours: an unanswered or refused subscription is asked again a day later, and a relay no longer named gets
the `Undo`). What an accepted relay passes on comes to the federated timeline, never to anyone's home: a public post it
forwards as its author sent it (Activity-Relay), read again from its origin like any forwarded post, and a post it
announces (aode-relay), kept as its author's and never as the relay's boost. Nothing else is taken from a relay. A
persona's public post outside any group, its edit and its deletion also go to the relays that accepted us (owner decision
2026-10-06), as Mastodon sends them; nothing less public, and no boost. Mastodon takes a post Activity-Relay forwards
only with an LD signature or an FEP-8b32 proof: PrivaPub's carry a proof, so they reach Mastodon through both kinds of
relay (checked live).
## Server descriptions and the crawler
PrivaPub keeps statistics about servers, never about accounts (see `/stargazing` on the server).
- **Describing a server:** a server it exchanges activities with is described at most once a week, from:
- `/.well-known/nodeinfo` and the NodeInfo document it links;
- `/api/v2/instance`, falling back to `/api/v1/instance`.
These requests are unsigned, because they are not ActivityPub documents.
- **The crawler** is on at privapub.thepra.dev. It identifies as
`PrivaPub-Stargazer/<version> (+https://privapub.thepra.dev/stargazing)`.
- **What it reads:** `/robots.txt`, then the documents above and `/api/v1/instance/peers`, and nothing else: no
accounts, posts or directories.
- **How often:** one server a minute, each at most weekly.
- **Opting out:** in robots.txt, the crawler obeys the group `PrivaPub-Stargazer`, then `PrivaPub`, then `*`. A
robots.txt that answers with a server error or times out also keeps it out, as do this server's domain blocks.
## Local-only posts
Posts with a location (shown to nearby users of this server) never leave the server, in any form.
@@ -174,21 +376,68 @@ Posts with a location (shown to nearby users of this server) never leave the ser
- **Origins.** An actor document is accepted only from the address it names as its `id`. A key only if its actor lists
it with `owner` set to the actor and on the actor's origin. An activity's `id`, and any object it creates, updates or
deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin.
- **Another account of the same server.** An object attributed to another account of the actor's own server (Mobilizon's
organiser creates, edits and deletes the group's event) is that server's to vouch for: a creation or an edit is
taken as the server has the object, a deletion once it answers 404 or 410. Attributed to an account elsewhere, it is
refused (400).
- **Forwarded activities** (ActivityPub's inbox forwarding: a thread's server passing on the replies in it, signed with
its own key) are answered 202 and believed only as far as their origin vouches for them: a created or edited post
is read again from its author's server, a deletion of a public or unlisted post is applied once that server answers
404 or 410, and anything else is dropped. LD signatures are not verified. A reply forwarded this way is kept when
someone here follows the author of the post it answers, as Mastodon does.
- **Follow requests** still unanswered are sent again after 15 minutes, an hour, 6 hours, a day, two and four days: a
server can take a Follow and lose its answer (Lemmy 1.0 sends nothing it queued for a server before it started
sending there), and one that holds the follow answers it again. Each is the same follow under a new id
(`…/follow-<id>-again-<n>`), since Lemmy ignores an id it has seen; an answer naming any of them counts.
- **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three
redirects and read at most 1 MB.
- **Threads.** A reply's missing parents are fetched, up to 10 levels. When someone here opens a public remote thread,
its replies are read from their servers, at most once an hour: the thread's `context` collection (FEP-7888) if it has
one, else its `replies` (PeerTube's `comments`) and theirs, two levels down; 5 pages and 100 posts at most. Only
public and unlisted replies are kept, each fetched from its own origin.
- **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like
Mastodon's authorized fetch; the server's own actors (the instance actor `/peasants/privapub` and the reporter
`/peasants/privapub_reports`) are the exception, since their keys are needed first.
A browser asking for HTML is redirected to the public page instead.
- **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were
for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted
they answer those same readers 410. A direct message's `context`, `/peasants/{name}/whispers/{id}`, is an
`OrderedCollection` of the conversation's posts for its participants.
- **HTML.** Received HTML is sanitised to Mastodon's allowlist.
- **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered
5xx, the inbox answers 503 with `Retry-After: 300` rather than 401.
- **Activity ids.** `Create` and `Announce` ids dereference. `Follow`, `Like`, `Block` and the `Accept`, `Reject` and
`Undo` that answer them do not, because serving them would reveal who follows, likes and blocks whom; they are always
sent with their object embedded.
- **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
starting at an hour and growing to a week.
- **Activity ids.** `Create` and `Announce` ids dereference, a community's announces included, until the post they
carry is deleted (then 410); a persona's boost id sends a browser to the boosted post. `Follow`, `Like`, `Block` and
the `Accept`, `Reject` and `Undo` that answer them do not, because serving them would reveal who follows, likes and
blocks whom. Neither do `Update`, `Delete`, `EmojiReact`, `QuoteRequest` and its answers, `Flag`, `Ignore` or poll
votes. All of them are always sent with their object embedded. Every boost and every favourite is an activity of its
own (`announce-{boost}`, `like-{favourite}`), so one given again after its `Undo` is new to the server it reaches.
- **Hashtags.** A post's `Hashtag` links go to `/tags/{tag}`, a public page of this server's public posts with that tag.
- **Delivery.** An activity goes once to each server, to its shared inbox when the server has one, whoever there it
follows, names or answers (a circle post excepted: each member's copy names that member). Failed deliveries are
retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
starting at an hour and growing to a week: unreachable, timing out, or a gateway answering 502, 503 or 504. A 500 is
the server failing on that one activity, retried on its own while its other deliveries go on. A server can also take a Follow (202) and drop it afterwards, as Pleroma
does when it cannot yet fetch our actor, so a follow request still unanswered is sent again, same activity, when the
persona follows once more, at most once an hour.
## Known limitations
- Our own posts carry no custom emoji: emoji are received and shown, not offered.
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
- Collections expose counts, not members.
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
- Collections expose counts, not members: `/groupies` and `/stalking` give the same totals as the Mastodon API's
follower and following counts, and the outbox's `totalItems` is the persona's post count, though only public posts
are listed. `/api/v1/instance/peers` is empty: which servers this one talks to is not published.
- HTTP signatures are verified with RSA keys only, under draft-cavage or RFC 9421. What PrivaPub sends is signed with
draft-cavage only, which every server reads.
- **Integrity proofs** (FEP-8b32, FEP-521a). Every persona has an Ed25519 key of its own (never shared between personas),
named in its actor's `assertionMethod` as a `Multikey` (`…#ed25519-key`, `publicKeyMultibase`); the terms are defined
in the actor's own context, so no context document has to be fetched. A persona's activity going to a relay carries a
`DataIntegrityProof` (`eddsa-jcs-2022`, `proofPurpose` `assertionMethod`) over the activity as delivered; Mastodon
4.7 verifies it (checked against its own verifier), so what the relay forwards reaches it. Nothing else carries one:
Mitra takes a proof over the HTTP signature, and refuses one by a key it has not read yet without reading the actor
again, which every server that knew a persona before it had its key would do. What a persona passes on of others'
carries theirs or none. Received: an actor's own Multikeys (at most five, under its id, controlled by it) are kept;
a forwarded activity whose proof one of them verifies is taken as it came, instead of being read again from its origin
(an unknown key of the actor's has the actor read again first).
+103
View File
@@ -0,0 +1,103 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.Admin
{
// the server's backups as the administrator's page shows them, with what is running and the last restore
public class ViewBackups
{
public List<ViewBackup> Backups { get; set; } = [];
public string Running { get; set; }//"backup" or "restore" while one runs
public ViewRestoreWaiting Waiting { get; set; }
public ViewRestore LastRestore { get; set; }
public string Host { get; set; }//what a restore asks to be typed
public long FreeBytes { get; set; }
}
public class ViewBackup
{
public string Id { get; set; }
public string Kind { get; set; }//nightly, manual, pre-deploy, pre-restore, uploaded
public DateTime CreatedAt { get; set; }
public long Bytes { get; set; }//the database's files
public long MediaBytes { get; set; }
public int MediaFiles { get; set; }
public int MediaMissing { get; set; }
public bool DbOnly { get; set; }
public bool Consistent { get; set; }
public int Collections { get; set; }
public long Documents { get; set; }
public string AppCommit { get; set; }
public int MigrationNumber { get; set; }
public int RootsDeletedSince { get; set; }//logins deleted after it, which a restore deletes again
public string NotRestorable { get; set; }//why not, from its manifest; hashes are checked when asked
}
public class ViewRestoreWaiting
{
public string Backup { get; set; }
public string State { get; set; }
public int Attempts { get; set; }
public string RequestedBy { get; set; }
public DateTime RequestedAt { get; set; }
public string Error { get; set; }
}
public class ViewRestore
{
public string Backup { get; set; }
public DateTime BackupCreatedAt { get; set; }
public string PreRestore { get; set; }
public string RequestedBy { get; set; }
public DateTime RestoredAt { get; set; }
public bool Abandoned { get; set; }
public string Error { get; set; }
public long Documents { get; set; }
public int Collections { get; set; }
public int MediaRestored { get; set; }
public int MediaMissing { get; set; }
public int RootsDeleted { get; set; }
public int PersonasDeleted { get; set; }
public int GroupsDeleted { get; set; }
public int PostsDeleted { get; set; }
public List<string> RootsTombstoned { get; set; } = [];
public List<string> PersonasTombstoned { get; set; } = [];
public List<string> GroupsTombstoned { get; set; } = [];
public int PostsGone { get; set; }
public int MediaTrashed { get; set; }
public int ProtectiveKept { get; set; }
public int SessionsEnded { get; set; }
}
// a backup downloaded or restored: the administrator's password again, and for a restore the server's host typed out
public class BackupPasswordForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(200, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Password { get; set; }
}
public class RestoreBackupForm : BackupPasswordForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(253, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Host { get; set; }
}
// a download link good for a while, for one backup
public class ViewBackupTicket
{
public string Url { get; set; }
public DateTime ExpiresAt { get; set; }
public long Bytes { get; set; }
}
// a backup being uploaded in pieces: how much has arrived
public class ViewBackupUpload
{
public string Id { get; set; }
public long Received { get; set; }
public long ChunkBytes { get; set; }//the largest piece accepted
}
}
+20
View File
@@ -0,0 +1,20 @@
namespace PrivaPub.ClientModels.Admin
{
// a report as moderators read it: who sent it ("local" for a persona here, never which one; or the remote actor), whom
// and which posts it is about, and whether it was resolved
public class ViewReport
{
public string ID { get; set; }
public string Reporter { get; set; }
public string TargetAccountId { get; set; }
public string TargetActorURI { get; set; }
public List<string> PostIds { get; set; } = new();
public List<string> ObjectURIs { get; set; } = new();
public string Category { get; set; }
public string Comment { get; set; }
public bool Forwarded { get; set; }
public bool IsResolved { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime? ResolvedAt { get; set; }
}
}
@@ -0,0 +1,99 @@
namespace PrivaPub.ClientModels.Admin
{
// /clientapi/admin/statistics/overview: the last days' federation at a glance; counters keyed as the ledger keys them
public class ViewStatisticsOverview
{
public int Days { get; set; }
public Dictionary<string, long> Channels { get; set; } = new();
public Dictionary<string, long> Inbound { get; set; } = new();
public Dictionary<string, long> Outbound { get; set; } = new();
public double? DeliverySuccess { get; set; }
public ViewLatency DeliveryLatencyMs { get; set; } = new();
public ViewHostCounts Hosts { get; set; } = new();
public int Accounts { get; set; }
public ViewLedgerCounts Ledger { get; set; } = new();
public ViewMediaTotals Media { get; set; } = new();
}
// what media take on the server: the uploads and pictures kept, the remote media cache, the trash waiting its grace
public class ViewMediaTotals
{
public long MediaBytes { get; set; }
public long MediaFiles { get; set; }
public long ProxyCacheBytes { get; set; }
public long TrashBytes { get; set; }
}
public class ViewLatency
{
public int? P50 { get; set; }
public int? P95 { get; set; }
}
public class ViewHostCounts
{
public int Active { get; set; }
public long Known { get; set; }
}
public class ViewLedgerCounts
{
public long Written { get; set; }
public long Dropped { get; set; }
public long Failed { get; set; }
}
// /clientapi/admin/statistics/hosts: one page of the servers we exchanged with
public class ViewHostsPage
{
public int Total { get; set; }
public int Page { get; set; }
public List<ViewHostRow> Hosts { get; set; } = new();
public string Attribution { get; set; }
}
public class ViewHostRow
{
public string Host { get; set; }
public string Software { get; set; }
public string Version { get; set; }
public long In { get; set; }
public long Out { get; set; }
public long Refused { get; set; }
public long Failures { get; set; }
public int? LatencyP50 { get; set; }
public int Accounts { get; set; }
public DateTime? DescribedAt { get; set; }
public ViewHostDelivery Delivery { get; set; }
}
public class ViewHostDelivery
{
public int ConsecutiveFailures { get; set; }
public DateTime? UnavailableUntil { get; set; }
public DateTime? LastSuccessAt { get; set; }
public DateTime? LastFailureAt { get; set; }
}
// /clientapi/admin/statistics/crawler: how far the crawler got
public class ViewCrawler
{
public bool Enabled { get; set; }
public long Touched { get; set; }
public long Crawled { get; set; }
public long Due { get; set; }
public long RobotsDisallowed { get; set; }
public List<ViewCrawledHost> Recent { get; set; } = new();
}
public class ViewCrawledHost
{
public string Host { get; set; }
public string Seen { get; set; }
public DateTime? LastCrawledAt { get; set; }
public int? PeersCount { get; set; }
public bool? RobotsDisallowed { get; set; }
public string CrawlError { get; set; }
public string Software { get; set; }
}
}
@@ -0,0 +1,15 @@
namespace PrivaPub.ClientModels.Group
{
// someone in a group, or asking to join it, as its owner and moderators see them
public class ViewGroupMember
{
public string ActorUri { get; set; }
public string Handle { get; set; }
public string Name { get; set; }
public string PictureUrl { get; set; }
public bool IsLocal { get; set; }
public bool IsPending { get; set; }
public string Role { get; set; }//owner, moderator, member
public DateTime Since { get; set; }
}
}
@@ -0,0 +1,14 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.User
{
// Deleting one's own login, and with it every persona and group it owns, asks for the password again.
public class RemoveSelfForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
Display(Name = "Password", ResourceType = typeof(FieldsNameResource))]
public string Password { get; set; }
}
}
@@ -0,0 +1,48 @@
namespace PrivaPub.ClientModels.User
{
// A persona's archive (owner decision 2026-10-07): made for it to take away, in Mastodon's account-archive layout, and
// restored into it. States: none, queued, running, ready (export) or done (import), failed, stopped.
public class ViewPersonaArchive
{
public string ExportState { get; set; }
public DateTime? ExportAskedAt { get; set; }
public DateTime? ExportReadyAt { get; set; }
public DateTime? ExportExpiresAt { get; set; }
public long ExportBytes { get; set; }
public string ExportError { get; set; }
public string ImportState { get; set; }
public DateTime? ImportAskedAt { get; set; }
public DateTime? ImportEndedAt { get; set; }
public List<string> ImportParts { get; set; } = [];
public long ImportDone { get; set; }
public long ImportTotal { get; set; }
public Dictionary<string, int> ImportCounts { get; set; } = [];//what was imported and what skipped, by why
public string ImportError { get; set; }
public string ImportFrom { get; set; }//the archive's actor
public bool ImportSameActor { get; set; }
}
// the parts of an archive to import: posts (with their media), profile, follows, blocks, mutes, domainblocks, lists,
// bookmarks, filters, tags, notificationpolicy, pins; and only when asked, located, scheduled, likes (which tell their
// authors)
public class ImportPersonaArchiveForm
{
public string UploadId { get; set; }
public List<string> Parts { get; set; } = [];
}
public class ViewArchiveTicket
{
public string Url { get; set; }
public DateTime ExpiresAt { get; set; }
public long Bytes { get; set; }
}
public class ViewArchiveUpload
{
public string Id { get; set; }
public long Received { get; set; }
public long ChunkBytes { get; set; }
}
}
+11
View File
@@ -0,0 +1,11 @@
namespace PrivaPub.ClientModels.User
{
// /clientapi/user/storage: what the login's media take (every persona's uploads and pictures), and its quota when the
// server sets one
public class ViewStorage
{
public long MediaBytes { get; set; }
public long MediaFiles { get; set; }
public long? QuotaBytes { get; set; }
}
}
@@ -60,6 +60,7 @@ namespace PrivaPub.Tests.Domain
{
public string BaseAddress => Base;
public Task<GoneActor> Gone(string userName, CancellationToken token) => Task.FromResult<GoneActor>(default);
public Task<LocalActor> FindByUserName(string userName, CancellationToken token) =>
Task.FromResult(userName.Equals("alice", StringComparison.OrdinalIgnoreCase)
? new LocalActor { Id = "a1", UserName = "alice", BaseAddress = Base, Kind = LocalActorKind.Person }
@@ -67,7 +68,9 @@ namespace PrivaPub.Tests.Domain
public Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> FindByUri(string actorUri, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> FindByAddress(string address, CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> GetInstanceActor(CancellationToken token) => throw new NotSupportedException();
public Task<LocalActor> GetReporterActor(CancellationToken token) => throw new NotSupportedException();
public Task<bool> IsUserNameTaken(string userName, CancellationToken token) => throw new NotSupportedException();
public Task<bool> TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token) => throw new NotSupportedException();
public LocalActor FromAvatar(Avatar avatar) => throw new NotSupportedException();
+60 -2
View File
@@ -1,3 +1,4 @@
using PrivaPub.Models.Federation;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Memory;
@@ -46,7 +47,7 @@ namespace PrivaPub.Tests.Domain
var (_, alice) = await _harness.Persona("alice");
var (_, mallory) = await _harness.Persona("mallory");
var upload = await _harness.Media.Upload(alice, Upload(Png(300, 200), "image/png"), "a blue square", "0.25,-0.5", token);
var upload = await _harness.Media.Upload(alice, Upload(Png(300, 200), "image/png"), "a blue square", "0.25,-0.5", false, token);
Assert.True(upload.Ok);
Assert.True(File.Exists(Path.Combine(_harness.Media.Root, upload.Attachment.FilePath)));
@@ -73,7 +74,7 @@ namespace PrivaPub.Tests.Domain
{
var (_, alice) = await _harness.Persona("alice");
var outcome = await _harness.Media.Upload(alice, Upload(System.Text.Encoding.UTF8.GetBytes("<svg/>"), "image/svg+xml"), default, default, TestContext.Current.CancellationToken);
var outcome = await _harness.Media.Upload(alice, Upload(System.Text.Encoding.UTF8.GetBytes("<svg/>"), "image/svg+xml"), default, default, false, TestContext.Current.CancellationToken);
Assert.False(outcome.Ok);
Assert.Equal(422, outcome.Status);
@@ -98,5 +99,62 @@ namespace PrivaPub.Tests.Domain
Assert.StartsWith(_harness.Media.ProxyRoot, file);
Assert.Null(tampered);
}
// a login over its quota uploads nothing more, whichever persona tries
[Fact]
public async Task A_login_over_its_quota_uploads_nothing_more()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var (_, sibling) = await _harness.Persona("sibling", root);
var quota = new MediaService(new StaticOptions<MediaOptions>(new MediaOptions { Root = _harness.Media.Root, QuotaBytesPerRoot = 3000 }),
_harness.Local, default, Microsoft.Extensions.Logging.Abstractions.NullLogger<MediaService>.Instance);
// noise: its PNG is about as big as its pixels, so two of them are over the quota and one is not
static byte[] Noise()
{
var pixels = new byte[30 * 20 * 3];
Random.Shared.NextBytes(pixels);
using var image = NetVips.Image.NewFromMemory(pixels, 30, 20, 3, NetVips.Enums.BandFormat.Uchar);
return image.WriteToBuffer(".png");
}
Assert.True((await quota.Upload(alice, Upload(Noise(), "image/png"), default, default, false, token)).Ok);
var full = await quota.Upload(sibling, Upload(Noise(), "image/png"), default, default, false, token);
Assert.False(full.Ok);
Assert.Contains("storage is full", full.Error);
}
// nothing of a suspended server, or of one whose media are rejected, is proxied; blocking one purges what was cached
[Fact]
public async Task A_blocked_servers_media_are_not_proxied_and_their_cache_goes()
{
var token = TestContext.Current.CancellationToken;
var path = $"/files/{Guid.NewGuid():N}.png";
_harness.Peer.ServeFile(path, Png(10, 10), "image/png");
var remote = _harness.Peer.A + path;
var host = new Uri(remote).Host;
var blocks = new Blocks();
var proxy = new MediaProxy(_harness.Local, Peer.Http(), _harness.Media, new StaticOptions<MediaOptions>(new MediaOptions()), blocks);
Assert.Equal(ProxyOutcome.Cached, (await proxy.Download(remote, token)).Outcome);
blocks.Blocked[host] = new DomainBlock { Domain = host, Severity = DomainBlockSeverity.Silence, RejectMedia = true };
Assert.True(proxy.Refuses(remote));
Assert.True(proxy.Purge(host) >= 1);
Assert.Equal(default, proxy.Cached(remote));
blocks.Blocked[host] = new DomainBlock { Domain = host, Severity = DomainBlockSeverity.Silence };
Assert.False(proxy.Refuses(remote));
blocks.Blocked[host] = new DomainBlock { Domain = host, Severity = DomainBlockSeverity.Suspend };
Assert.True(proxy.Refuses(remote));
}
sealed class Blocks : PrivaPub.Federation.Moderation.IDomainBlocks
{
public Dictionary<string, DomainBlock> Blocked { get; } = new();
public DomainBlock Find(string host) => Blocked.GetValueOrDefault(host);
public bool IsSuspended(string host) => Find(host)?.Severity == DomainBlockSeverity.Suspend;
public Task Reload(CancellationToken token) => Task.CompletedTask;
}
}
}
+34 -3
View File
@@ -19,6 +19,37 @@ namespace PrivaPub.Tests.Domain
return tagged.WriteToBuffer(".jpg");
}
// SVG, PDF and the other formats libvips could read never load from an upload, whatever it claims to be
[Fact]
public void Only_the_upload_formats_are_ever_loaded()
{
var svg = "<svg xmlns='http://www.w3.org/2000/svg' width='20000' height='20000'><rect width='10' height='10'/></svg>"u8.ToArray();
Assert.Throws<VipsException>(() => MediaService.Inspect(svg));
Assert.Throws<VipsException>(() => MediaService.Inspect("%PDF-1.4\n1 0 obj<<>>endobj\ntrailer<<>>\n%%EOF"u8.ToArray()));
using var colour = (Image.Black(8, 8, bands: 3) + 100).Cast(Enums.BandFormat.Uchar);
foreach (var format in new[] { ".jpg", ".png", ".gif", ".webp" })
Assert.Equal(8, MediaService.Inspect(colour.WriteToBuffer(format)).Width);
Assert.Throws<VipsException>(() => MediaService.Inspect(colour.WriteToBuffer(".tif")));
}
// the header tells how big an image would decode before anything is decoded
[Fact]
public void The_header_says_how_big_an_image_would_decode()
{
using var huge = Image.Black(8000, 6000);
var header = MediaService.Inspect(huge.WriteToBuffer(".png"));
Assert.Equal(48_000_000, header.Pixels);
Assert.False(header.Animated);
using var frame = (Image.Black(40, 30, bands: 3) + 60).Cast(Enums.BandFormat.Uchar);
using var frames = Image.Arrayjoin(new[] { frame, frame + 80, frame + 160 }, across: 1).Cast(Enums.BandFormat.Uchar);
using var paged = frames.Mutate(m => m.Set(GValue.GIntType, "page-height", 30));
var gif = MediaService.Inspect(paged.WriteToBuffer(".gif"));
Assert.True(gif.Animated);
Assert.Equal(3, gif.Pages);
Assert.Equal(40 * 30 * 3, gif.Pixels);
}
[Fact]
public void Uploaded_images_lose_every_kind_of_metadata()
{
@@ -26,7 +57,7 @@ namespace PrivaPub.Tests.Domain
using (var original = Image.NewFromBuffer(input))
Assert.Contains("exif-data", original.GetFields());
var processed = MediaService.ProcessImage(input, 4096, 640, animated: false);
var processed = MediaService.ProcessImage(input, 4096, 640);
using var output = Image.NewFromBuffer(processed.Bytes);
var fields = output.GetFields();
@@ -44,7 +75,7 @@ namespace PrivaPub.Tests.Domain
[Fact]
public void Large_images_are_capped()
{
var processed = MediaService.ProcessImage(JpegWithMetadata(5000, 2500), 4096, 640, animated: false);
var processed = MediaService.ProcessImage(JpegWithMetadata(5000, 2500), 4096, 640);
Assert.Equal((4096, 2048), (processed.Width, processed.Height));
}
@@ -52,7 +83,7 @@ namespace PrivaPub.Tests.Domain
[Fact]
public void A_blurhash_is_well_formed()
{
var processed = MediaService.ProcessImage(JpegWithMetadata(64, 64), 4096, 640, animated: false);
var processed = MediaService.ProcessImage(JpegWithMetadata(64, 64), 4096, 640);
Assert.Equal(28, processed.Blurhash.Length);
Assert.Equal('L', processed.Blurhash[0]);
+2
View File
@@ -59,6 +59,8 @@ namespace PrivaPub.Tests.Domain
.Select(j => JsonSerializer.Deserialize<DeliveryPayload>(j.Payload))
.Where(p => p.Body.Contains(post.ID));
Assert.Empty(outgoing);
// nor reaches a home, its author's included: the Mastodon API looks past it, so there it could not be acted on
Assert.False(await DB.Default.Find<PrivaPub.Models.Social.TimelineEntry>().Match(e => e.PostId == post.ID).ExecuteAnyAsync(token));
}
[Fact]
+2
View File
@@ -65,6 +65,8 @@ namespace PrivaPub.Tests.Domain
var sent = await _harness.Outgoing(follower.SharedInbox);
Assert.Equal(new[] { "Create", "Update", "Delete" }, sent.Select(a => a["type"]!.GetValue<string>()));
Assert.DoesNotContain(sent[0]["to"]!.AsArray(), n => n!.GetValue<string>().EndsWith("#Public"));
// Pleroma reads an empty cc without a "/followers" address as a direct message
Assert.Contains(sent[0]["object"]!["cc"]!.AsArray(), n => n!.GetValue<string>() == alice.Followers);
Assert.Equal("cw", sent[1]["object"]!["summary"]!.GetValue<string>());
Assert.NotNull(sent[1]["object"]!["updated"]);
Assert.Equal(sent[0]["to"]!.ToJsonString(), sent[2]["to"]!.ToJsonString());
@@ -0,0 +1,277 @@
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Federation
{
public class AuthorGoneRuleTests
{
[Fact]
public async Task A_post_whose_author_is_gone_is_neither_shown_nor_public_nor_seen_by_anyone()
{
var gone = new Post { Visibility = PostVisibility.Public, AuthorGone = true, GroupUserId = "alice" };
Assert.False(VisibilityPolicy.Shown(gone));
Assert.False(VisibilityPolicy.IsPublic.Compile()(gone));
Assert.False(await VisibilityPolicy.CanSee(gone, default, TestContext.Current.CancellationToken));
Assert.False(await VisibilityPolicy.CanSee(gone, "alice", TestContext.Current.CancellationToken));
Assert.True(VisibilityPolicy.Shown(new Post()));
Assert.False(VisibilityPolicy.Shown(new Post { DeletedAt = DateTime.UtcNow }));
Assert.False(VisibilityPolicy.Shown(default));
}
}
[Trait("Category", "Integration")]
public sealed class AuthorGoneTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
async Task<Post> Delivered(RemoteActor author, JsonObject note)
{
await _harness.Deliver(author, "/human-centipede", Create(author, note));
return await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(note)).ExecuteFirstAsync(TestContext.Current.CancellationToken);
}
[Fact]
public async Task Deleting_itself_keeps_every_post_of_the_account_but_hides_them_and_drops_its_follows_and_rows()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var ghost = new RemoteActor(_harness.Peer, "ghost");
var friend = new RemoteActor(_harness.Peer, "friend");
await Follows(alice.Id, ghost);
await Follows(alice.Id, friend);
await _harness.FollowedBy(alice, ghost);
var post = await Delivered(ghost, PublicNote(ghost, "<p>soon gone</p>"));
var friends = await Delivered(friend, PublicNote(friend, "<p>still here</p>"));
await _harness.Deliver(ghost, "/human-centipede", new JsonObject
{
["id"] = NewId(ghost, "announces"), ["type"] = "Announce", ["actor"] = ghost.Id, ["object"] = friends.ObjectURI,
["to"] = new JsonArray(Addressing.Public)
});
var ghostAccount = await DB.Default.Find<ForeignAvatar>().Match(f => f.ActorURI == ghost.Id).ExecuteSingleAsync(token);
Assert.True(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id && e.AuthorAccountId == ghostAccount.ID).ExecuteAnyAsync(token));
Assert.Equal(202, (await _harness.Deliver(ghost, "/human-centipede", Activity(ghost, "Delete", JsonValue.Create(ghost.Id)!))).StatusCode);
var kept = await DB.Default.Find<Post>().Match(p => p.ActorURI == ghost.Id).ExecuteAsync(token);
Assert.Equal(2, kept.Count);
Assert.All(kept, p => Assert.True(p.AuthorGone));
Assert.Contains(kept, p => p.ID == post.ID && p.ContentHtml == "<p>soon gone</p>");
Assert.Contains(kept, p => p.ReblogOfPostId == friends.ID);
Assert.False((await DB.Default.Find<Post>().OneAsync(friends.ID, token)).AuthorGone);
Assert.All(kept, p => Assert.False(VisibilityPolicy.Shown(p)));
foreach (var hidden in kept)
Assert.False(await VisibilityPolicy.CanSee(hidden, alice.Id, token));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == ghost.Id).Match(VisibilityPolicy.IsPublic).ExecuteAnyAsync(token));
Assert.Equal(AvatarAccountState.Deleted, (await DB.Default.Find<ForeignAvatar>().OneAsync(ghostAccount.ID, token)).AccountState);
Assert.False(await DB.Default.Find<Follower>().Match(f => f.ActorURI == ghost.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Following>().Match(f => f.TargetActorURI == ghost.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<TimelineEntry>().Match(e => e.AuthorAccountId == ghostAccount.ID).ExecuteAnyAsync(token));
var mapper = new MastodonMapper(_harness.Db, _harness.Local);
Assert.Empty(await mapper.Statuses(kept, alice.Id, token));
var home = (List<PrivaPub.ClientModels.Post.ViewPost>)(await _harness.Timelines.Home(root, alice.Id, default, 40, token)).Data;
Assert.DoesNotContain(home, v => v.AuthorActorURI == ghost.Id);
Assert.Contains(home, v => v.Id == friends.ID);
}
[Fact]
public async Task A_post_by_an_account_already_gone_is_hidden_from_the_moment_it_is_stored()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ghost = new RemoteActor(_harness.Peer, "ghost");
await _harness.FollowedBy(alice, ghost);
await _harness.Deliver(ghost, "/human-centipede", Activity(ghost, "Delete", JsonValue.Create(ghost.Id)!));
var late = PublicNote(ghost, "<p>from beyond</p>");
_harness.Peer.Serve(new Uri(IdOf(late)).AbsolutePath, late.ToJsonString());
var stored = await _harness.RemotePosts.StoreContext(IdOf(late), 0, token);
Assert.NotNull(stored);
Assert.True(stored.AuthorGone);
Assert.False(await VisibilityPolicy.CanSee(stored, alice.Id, token));
}
}
[Trait("Category", "Integration")]
public sealed class AuthorGoneOverHttpTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static async Task<(HttpStatusCode Status, JsonNode Body)> Get(HttpClient client, string path)
{
var response = await client.GetAsync(path, TestContext.Current.CancellationToken);
var text = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
return (response.StatusCode, string.IsNullOrEmpty(text) || response.Content.Headers.ContentType?.MediaType?.Contains("json") != true ? default : JsonNode.Parse(text));
}
static async Task<List<string>> Ids(HttpClient client, string path)
{
var (status, body) = await Get(client, path);
Assert.Equal(HttpStatusCode.OK, status);
return body!.AsArray().Select(s => IdOf(s!)).ToList();
}
static async Task<bool> Found(HttpClient client, string path) => (await Get(client, path)).Status == HttpStatusCode.OK;
sealed record Seen(bool Status, bool Context, bool FavouritedBy, bool RebloggedBy, bool History, bool Many, bool Boost, bool InAncestors,
bool BoostersListed, bool OnProfile, bool OnTag, bool OnPublic, bool OnHome, bool Notified, bool InConversation, bool Searched,
bool Favourited, bool Bookmarked, bool Provenance, bool InCommunityOutbox);
[Fact]
public async Task Every_lookup_treats_a_post_whose_author_deleted_itself_as_not_found()
{
var token = TestContext.Current.CancellationToken;
var persona = await _host.Persona(await _host.SignUp(), "reader");
var bearer = await _host.MastodonToken(persona);
using var client = _host.As(bearer);
var (privateKey, publicKey) = PrivaPub.Federation.Actors.Keys.NewKeyPair();
var community = new GroupEntity
{
UserName = $"commons{Guid.NewGuid():N}"[..20], Kind = GroupKind.Community, PostingPolicy = PostingPolicy.Anyone,
PrivateKey = privateKey, PublicKey = publicKey, Members = new() { new GroupMember { AvatarId = persona.Id, Role = GroupRole.Owner } }
};
await DB.Default.SaveAsync(community, token);
var personaUri = $"{PrivaPubHost.Base}/peasants/{persona.UserName}";
var communityUri = $"{PrivaPubHost.Base}/peasants/{community.UserName}";
var ghost = new RemoteActor(_peer, "ghost");
var friend = new RemoteActor(_peer, "friend");
await Follows(persona.Id, ghost);
await Follows(persona.Id, friend);
var tag = $"gone{Guid.NewGuid():N}"[..16];
var note = PublicNote(ghost, $"<p>hello #{tag}</p>", personaUri, communityUri);
note["tag"] = new JsonArray(
new JsonObject { ["type"] = "Mention", ["href"] = personaUri, ["name"] = "@reader" },
new JsonObject { ["type"] = "Hashtag", ["name"] = "#" + tag, ["href"] = $"{Origin(ghost)}/tags/{tag}" });
Assert.Equal(HttpStatusCode.Accepted, await DeliverSigned(_host, ghost, Create(ghost, note)));
var friendsNote = PublicNote(friend, "<p>a friend's post</p>");
await DeliverSigned(_host, friend, Create(friend, friendsNote));
var reply = PublicNote(friend, "<p>a reply</p>");
reply["inReplyTo"] = IdOf(note);
await DeliverSigned(_host, friend, Create(friend, reply));
await DeliverSigned(_host, friend, new JsonObject
{
["id"] = NewId(friend, "announces"), ["type"] = "Announce", ["actor"] = friend.Id, ["object"] = IdOf(note), ["to"] = new JsonArray(Addressing.Public)
});
var ghostBoostId = NewId(ghost, "announces");
await DeliverSigned(_host, ghost, new JsonObject
{
["id"] = ghostBoostId, ["type"] = "Announce", ["actor"] = ghost.Id, ["object"] = IdOf(friendsNote), ["to"] = new JsonArray(Addressing.Public)
});
var dm = new JsonObject
{
["id"] = NewId(ghost, "notes"), ["type"] = "Note", ["attributedTo"] = ghost.Id, ["content"] = "<p>psst</p>", ["to"] = new JsonArray(personaUri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = personaUri, ["name"] = "@reader" })
};
await DeliverSigned(_host, ghost, Create(ghost, dm));
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(note)).ExecuteSingleAsync(token);
var friends = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(friendsNote)).ExecuteSingleAsync(token);
var replyPost = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(reply)).ExecuteSingleAsync(token);
var ghostBoost = await DB.Default.Find<Post>().Match(p => p.ObjectURI == ghostBoostId).ExecuteSingleAsync(token);
var dmPost = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(dm)).ExecuteSingleAsync(token);
var ghostAccount = await DB.Default.Find<ForeignAvatar>().Match(f => f.ActorURI == ghost.Id).ExecuteSingleAsync(token);
Assert.Equal(community.ID, post.GroupId);
//search looks up https addresses only, which the peer does not have
var searchable = new Post
{
ObjectURI = $"https://ghost{Guid.NewGuid():N}.example/notes/1", ActorURI = ghost.Id, AuthorAccountId = ghostAccount.ID, IsFederatedCopy = true,
Visibility = PostVisibility.Public, ContentHtml = "<p>found by its address</p>"
};
await DB.Default.SaveAsync(searchable, token);
Assert.Equal(HttpStatusCode.OK, (await client.PostAsync($"/api/v1/statuses/{post.ID}/favourite", default, token)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await client.PostAsync($"/api/v1/statuses/{post.ID}/bookmark", default, token)).StatusCode);
async Task<Seen> Look()
{
var home = (await Get(client, "/api/v1/timelines/home?limit=40")).Body!.AsArray();
var notifications = (await Get(client, "/api/v1/notifications?limit=30")).Body!.AsArray();
var conversations = (await Get(client, "/api/v1/conversations")).Body!.AsArray();
var search = (await Get(client, $"/api/v2/search?type=statuses&q={Uri.EscapeDataString(searchable.ObjectURI)}")).Body!;
using var anonymous = _host.Client();
using var outboxRequest = new HttpRequestMessage(HttpMethod.Get, $"/peasants/{community.UserName}/anus?page=true");
outboxRequest.Headers.Accept.ParseAdd("application/activity+json");
var outbox = JsonNode.Parse(await (await anonymous.SendAsync(outboxRequest, token)).Content.ReadAsStringAsync(token))!;
return new Seen(
await Found(client, $"/api/v1/statuses/{post.ID}"),
await Found(client, $"/api/v1/statuses/{post.ID}/context"),
await Found(client, $"/api/v1/statuses/{post.ID}/favourited_by"),
await Found(client, $"/api/v1/statuses/{post.ID}/reblogged_by"),
await Found(client, $"/api/v1/statuses/{post.ID}/history"),
(await Ids(client, $"/api/v1/statuses?id[]={post.ID}")).Contains(post.ID),
await Found(client, $"/api/v1/statuses/{ghostBoost.ID}"),
(await Get(client, $"/api/v1/statuses/{replyPost.ID}/context")).Body!["ancestors"]!.AsArray().Any(a => IdOf(a!) == post.ID),
(await Ids(client, $"/api/v1/statuses/{friends.ID}/reblogged_by")).Contains(ghostAccount.ID),
(await Ids(client, $"/api/v1/accounts/{ghostAccount.ID}/statuses")).Count > 0,
(await Ids(client, $"/api/v1/timelines/tag/{tag}")).Contains(post.ID),
(await Ids(client, $"/api/v1/timelines/public?remote=true&limit=1&max_id={IdAbove(post.ID)}")).Contains(post.ID),
home.Any(s => IdOf(s!) == post.ID || IdOf(s!) == ghostBoost.ID || s!["reblog"] is JsonObject inner && IdOf(inner) == post.ID),
notifications.Any(n => n!["status"] is JsonObject status && IdOf(status) == post.ID),
conversations.Any(c => c!["last_status"] is JsonObject last && IdOf(last) == dmPost.ID),
search["statuses"]!.AsArray().Count > 0,
(await Ids(client, "/api/v1/favourites")).Contains(post.ID),
(await Ids(client, "/api/v1/bookmarks")).Contains(post.ID),
await Found(client, $"/api/privapub/v1/statuses/{post.ID}/provenance"),
outbox["orderedItems"]!.AsArray().Any(a => a!["object"]?.GetValue<string>() == post.ObjectURI));
}
var before = await Look();
Assert.Equal(HttpStatusCode.Accepted, await DeliverSigned(_host, ghost, Activity(ghost, "Delete", JsonValue.Create(ghost.Id)!)));
var after = await Look();
var everything = new Seen(true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true);
var nothing = new Seen(false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false);
Assert.Equal(everything, before);
Assert.Equal(nothing, after);
Assert.True(await DB.Default.Find<Post>().Match(p => p.ID == post.ID && p.AuthorGone && p.DeletedAt == null).ExecuteAnyAsync(token));
Assert.True(await Found(client, $"/api/v1/statuses/{friends.ID}"));
Assert.True(await Found(client, $"/api/v1/statuses/{replyPost.ID}"));
}
}
}
@@ -0,0 +1,78 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// a follow that holds reads the account's latest public posts from its outbox, so its profile is not empty
[Trait("Category", "Integration")]
public sealed class BackfillTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task An_accepted_follow_reads_the_accounts_latest_public_posts_from_its_outbox()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var mallory = new RemoteActor(_harness.Peer, "mallory", _harness.Peer.B);
var origin = new Uri(bob.Id).GetLeftPart(UriPartial.Authority);
JsonObject Note(string id, string text, RemoteActor author) => new()
{
["id"] = id, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"<p>{text}</p>",
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"), ["published"] = DateTime.UtcNow.ToString("O")
};
var older = $"{origin}/notes/{Guid.NewGuid():N}";
var newer = $"{origin}/notes/{Guid.NewGuid():N}";
var notBobs = $"{new Uri(mallory.Id).GetLeftPart(UriPartial.Authority)}/notes/{Guid.NewGuid():N}";
_harness.Peer.Serve(new Uri(older).AbsolutePath, Note(older, "an older post", bob).ToJsonString());
_harness.Peer.Serve(new Uri(newer).AbsolutePath, Note(newer, "a newer post", bob).ToJsonString());
_harness.Peer.Serve(new Uri(bob.Id).AbsolutePath + "/outbox", new JsonObject
{
["id"] = bob.Id + "/outbox", ["type"] = "OrderedCollection", ["totalItems"] = 4,
["orderedItems"] = new JsonArray(
new JsonObject { ["type"] = "Create", ["object"] = Note(newer, "a newer post", bob) },
new JsonObject { ["type"] = "Announce", ["object"] = notBobs },
new JsonObject { ["type"] = "Create", ["object"] = notBobs },
new JsonObject { ["type"] = "Create", ["object"] = older })
}.ToJsonString());
var account = await _harness.Remote.GetActor(bob.Id, refresh: false, token);
var followId = alice.ActivityUri($"follow-{Guid.NewGuid():N}");
await DB.Default.SaveAsync(new Following
{
AvatarId = alice.Id, TargetActorURI = bob.Id, TargetAccountId = account.ID, State = FollowState.Requested, FollowActivityURI = followId
}, token);
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = $"{bob.Id}#accept-{Guid.NewGuid():N}", ["type"] = "Accept", ["actor"] = bob.Id,
["object"] = new JsonObject { ["id"] = followId, ["type"] = "Follow", ["actor"] = alice.Uri, ["object"] = bob.Id }
});
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.BackfillOutbox && j.Payload == bob.Id).ExecuteSingleAsync(token);
await new OutboxBackfill(_harness.Db, _harness.Remote, _harness.RemotePosts).Handle(job, token);
var held = (await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteAsync(token)).Select(p => p.ObjectURI).ToHashSet();
Assert.Equal(new HashSet<string> { older, newer }, held);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == notBobs).ExecuteAnyAsync(token));
}
}
}
@@ -0,0 +1,75 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// alone in its collection: every test's peer is 127.0.0.1, and the server row it writes decides for whoever writes there
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class ChatMessageSoftwareTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string NewId(RemoteActor actor, string kind) => $"{new Uri(actor.Id).GetLeftPart(UriPartial.Authority)}/{kind}/{Guid.NewGuid():N}";
// the one decision taken by a server's software (owner decision 2026-10-05): Lemmy before 1.0 and Mbin get a
// ChatMessage even from a persona answering a Note; Lemmy 1.0 a Note
[Theory]
[InlineData("lemmy", "0.19.20", "ChatMessage")]
[InlineData("mbin", "1.10.1", "ChatMessage")]
[InlineData("lemmy", "1.0.0-beta.2", "Note")]
public async Task A_direct_message_to_lemmy_before_1_0_or_mbin_is_a_chat_message(string software, string version, string sent)
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var sender = new RemoteActor(_harness.Peer, "pm");
var host = new Uri(sender.Id).Host;
await DB.Default.DeleteAsync<PrivaPub.Models.Jobs.RemoteInstance>(i => i.Host == host);
await DB.Default.SaveAsync(new PrivaPub.Models.Jobs.RemoteInstance { Host = host, Software = software, SoftwareVersion = version }, token);
try
{
var id = NewId(sender, "messages");
await _harness.Deliver(sender, "/human-centipede", new JsonObject
{
["id"] = NewId(sender, "activities/create"), ["type"] = "Create", ["actor"] = sender.Id, ["to"] = new JsonArray(alice.Uri),
["object"] = new JsonObject
{
["id"] = id, ["type"] = "Note", ["attributedTo"] = sender.Id, ["to"] = new JsonArray(alice.Uri),
["content"] = "<p>psst</p>", ["published"] = DateTime.UtcNow.ToString("O")
}
});
var message = await DB.Default.Find<Post>().Match(p => p.ObjectURI == id).ExecuteSingleAsync(token);
var answer = await _harness.Statuses.Publish(alice, new PrivaPub.Domain.Statuses.StatusDraft
{
Text = "psst back", Visibility = PostVisibility.Direct, ConversationId = message.ConversationId
}, token);
Assert.True(answer.Ok, answer.Error);
var create = (await _harness.Outgoing(sender.Id + "/inbox")).Last(a => a["type"]!.GetValue<string>() == "Create");
Assert.Equal(sent, create["object"]!["type"]!.GetValue<string>());
}
finally
{
await DB.Default.DeleteAsync<PrivaPub.Models.Jobs.RemoteInstance>(i => i.Host == host);
}
}
}
}
@@ -30,11 +30,12 @@ namespace PrivaPub.Tests.Federation
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
async Task<(RemoteActor Community, RemoteActor Poster, JsonObject Note, Post Post)> Announced()
// a post in a community the persona follows; the community on the poster's server unless another origin is given
async Task<(RemoteActor Community, RemoteActor Poster, JsonObject Note, Post Post)> Announced(string communityOrigin = default)
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "cats", type: "Group");
var community = new RemoteActor(_harness.Peer, "cats", communityOrigin, type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = community.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
@@ -109,6 +110,97 @@ namespace PrivaPub.Tests.Federation
Assert.True(await DB.Default.Find<DeletedObject>().Match(d => d.ObjectURI == post.ObjectURI).ExecuteAnyAsync(token));
}
// PieFed keeps serving a thread its moderator removed: a community on the post's own server speaks for it
[Fact]
public async Task A_removal_a_community_on_the_posts_server_announces_is_taken_though_the_post_is_still_served()
{
var token = TestContext.Current.CancellationToken;
var (community, poster, note, post) = await Announced();
await Announce(community, new JsonObject
{
["id"] = $"{Origin(poster)}/delete/{Guid.NewGuid():N}", ["type"] = "Delete", ["actor"] = poster.Id,
["object"] = note["id"]!.GetValue<string>(), ["summary"] = "off topic"
});
Assert.False(await DB.Default.Find<Post>().Match(p => p.ID == post.ID).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_removal_a_community_elsewhere_announces_waits_for_the_posts_server_to_say_it_is_gone()
{
var token = TestContext.Current.CancellationToken;
var (community, poster, note, post) = await Announced(_harness.Peer.B);
var delete = new JsonObject
{
["id"] = $"{Origin(community)}/delete/{Guid.NewGuid():N}", ["type"] = "Delete", ["actor"] = community.Id,
["object"] = note["id"]!.GetValue<string>()
};
await Announce(community, (JsonObject)delete.DeepClone());
var kept = await DB.Default.Find<Post>().Match(p => p.ID == post.ID).ExecuteAnyAsync(token);
_harness.Peer.Serve(new Uri(note["id"]!.GetValue<string>()).AbsolutePath, new JsonObject { ["id"] = note["id"]!.GetValue<string>(), ["type"] = "Tombstone" }.ToJsonString());
delete["id"] = $"{Origin(community)}/delete/{Guid.NewGuid():N}";
await Announce(community, delete);
Assert.True(kept);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ID == post.ID).ExecuteAnyAsync(token));
}
// Mbin sends a magazine's thread to its subscribers as the author's Create, and a moderator's lock as it is, with no
// announce: the magazine's server speaks for them, another server does not
[Fact]
public async Task A_thread_and_its_lock_that_a_followed_groups_own_server_sends_without_announcing_them_are_taken()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var magazine = new RemoteActor(_harness.Peer, "books", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
var stranger = new RemoteActor(_harness.Peer, "stranger", _harness.Peer.B);
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = magazine.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
JsonObject Page(RemoteActor author) => new()
{
["id"] = $"{Origin(author)}/m/books/t/{Guid.NewGuid():N}", ["type"] = "Page", ["name"] = "a thread",
["attributedTo"] = author.Id, ["content"] = "<p>in the magazine</p>",
["to"] = new JsonArray(magazine.Id, Addressing.Public), ["cc"] = new JsonArray(author.Id + "/followers"),
["audience"] = magazine.Id, ["published"] = DateTime.UtcNow.ToString("O")
};
JsonObject Create(RemoteActor author, JsonObject page) => new()
{
["id"] = $"{Origin(author)}/f/object/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Id,
["to"] = page["to"]!.DeepClone(), ["cc"] = page["cc"]!.DeepClone(), ["object"] = page
};
JsonObject Lock(RemoteActor moderator, string uri) => new()
{
["id"] = $"{Origin(moderator)}/f/object/{Guid.NewGuid():N}", ["type"] = "Lock", ["actor"] = moderator.Id, ["object"] = uri
};
var page = Page(poster);
var elsewhere = Page(stranger);
var uri = page["id"]!.GetValue<string>();
await _harness.Deliver(poster, "/human-centipede", Create(poster, page));
await _harness.Deliver(stranger, "/human-centipede", Create(stranger, elsewhere));
var kept = await DB.Default.Find<Post>().Match(p => p.ObjectURI == uri).ExecuteSingleAsync(token);
await _harness.Deliver(stranger, "/human-centipede", Lock(stranger, uri));
var lockedByStranger = (await DB.Default.Find<Post>().OneAsync(kept.ID, token)).LockedAt;
var moderator = new RemoteActor(_harness.Peer, "moderator");
var locking = Lock(moderator, uri);
await _harness.Deliver(moderator, "/human-centipede", locking);
var locked = (await DB.Default.Find<Post>().OneAsync(kept.ID, token)).LockedAt;
await _harness.Deliver(moderator, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(moderator)}/f/object/{Guid.NewGuid():N}", ["type"] = "Undo", ["actor"] = moderator.Id, ["object"] = locking.DeepClone()
});
Assert.Equal(magazine.Id, kept.AudienceURI);
Assert.True(await DB.Default.Find<TimelineEntry>().Match(e => e.PostId == kept.ID && e.AvatarId == alice.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == elsewhere["id"]!.GetValue<string>()).ExecuteAnyAsync(token));
Assert.Null(lockedByStranger);
Assert.NotNull(locked);
Assert.Null((await DB.Default.Find<Post>().OneAsync(kept.ID, token)).LockedAt);
}
[Fact]
public async Task A_fetched_object_does_not_wear_the_signature_of_the_activity_that_caused_the_fetch()
{
+32
View File
@@ -0,0 +1,32 @@
using PrivaPub.Federation.Rendering;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// The @context on everything we send. Readers that expand JSON-LD 1.0 (Smithereen) refuse a whole document over one
// term only 1.1 knows, so every term stays within 1.0
public class ContextTests
{
static readonly string[] Containers10 = ["@list", "@set", "@index", "@language"];
static readonly string[] Keywords11 = ["@version", "@protected", "@propagate", "@import", "@nest", "@prefix", "@context", "@direction", "@json"];
[Fact]
public void Every_term_is_json_ld_1_0()
{
var terms = ActivityPubRenderer.Context().OfType<JsonObject>().Single();
foreach (var (name, definition) in terms)
{
if (definition is not JsonObject term)
continue;
if (term["@container"] is JsonValue container)
Assert.True(Containers10.Contains(container.GetValue<string>()), $"{name}: @container {container} is JSON-LD 1.1");
Assert.True(term["@container"] is null or JsonValue, $"{name}: an array of containers is JSON-LD 1.1");
Assert.DoesNotContain(term.Select(p => p.Key), Keywords11.Contains);
Assert.NotEqual("@json", term["@type"]?.GetValue<string>());
}
Assert.DoesNotContain(terms.Select(p => p.Key), Keywords11.Contains);
}
}
}
@@ -0,0 +1,142 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Federation
{
// FEP-171b: a thread's owner adds a reply someone else wrote to the thread's context (Hubzilla's Add{Create}), and the
// reply is taken as passed on by the owner, on its own proof; an Add aimed at a collection elsewhere is not a thread's
[Trait("Category", "Integration")]
public sealed class ConversationContainerTests : IAsyncLifetime
{
const string Public = "https://www.w3.org/ns/activitystreams#Public";
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static JsonObject Create(RemoteActor author, string text, string context, string inReplyTo = default)
{
var noteId = $"{Origin(author)}/item/{Guid.NewGuid():N}";
var note = new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"<p>{text}</p>", ["context"] = context,
["to"] = new JsonArray(Public), ["published"] = DateTime.UtcNow.ToString("O")
};
if (inReplyTo != default)
note["inReplyTo"] = inReplyTo;
return new JsonObject
{
["id"] = $"{Origin(author)}/activity/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Id, ["to"] = new JsonArray(Public),
["object"] = note
};
}
async Task<bool> Held(string objectUri) => await DB.Default.Find<PostEntity>().Match(p => p.ObjectURI == objectUri).ExecuteAnyAsync(Token);
async Task RunQueued(string innerId)
{
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == "inbox|forwarded|" + innerId && j.State == JobState.Pending).ExecuteFirstAsync(Token);
Assert.NotNull(job);
Assert.Equal(JobResult.Done, (await _harness.Processor.Handle(job, Token)).Result);
}
[Fact]
public async Task A_reply_the_threads_owner_adds_to_its_context_is_taken_on_its_proof()
{
var (_, alice) = await _harness.Persona("alice");
var owner = new RemoteActor(_harness.Peer, "owner", ed25519: true);
var commenter = new RemoteActor(_harness.Peer, "commenter", ed25519: true);
await DB.Default.SaveAsync(new Following { AvatarId = alice.Id, TargetActorURI = owner.Id, TargetInboxURL = owner.Id + "/inbox", State = FollowState.Accepted }, Token);
var context = $"{Origin(owner)}/conversation/{Guid.NewGuid():N}";
var root = Create(owner, "a thread", context);
await _harness.Deliver(owner, "/human-centipede", root);
var rootId = root["object"]!["id"]!.GetValue<string>();
Assert.True(await Held(rootId));
var reply = commenter.Prove(Create(commenter, "a reply in it", context, rootId));
var replyId = reply["object"]!["id"]!.GetValue<string>();
await _harness.Deliver(owner, "/human-centipede", owner.Prove(new JsonObject
{
["id"] = $"{Origin(owner)}/activity/{Guid.NewGuid():N}", ["type"] = "Add", ["actor"] = owner.Id, ["object"] = reply,
["target"] = new JsonObject { ["id"] = context, ["type"] = "Collection", ["attributedTo"] = owner.Id }, ["to"] = new JsonArray(Public)
}));
await RunQueued(reply["id"]!.GetValue<string>());
var stored = await DB.Default.Find<PostEntity>().Match(p => p.ObjectURI == replyId).ExecuteFirstAsync(Token);
Assert.NotNull(stored);
Assert.Equal(commenter.Id, stored.ActorURI);
Assert.Equal((await DB.Default.Find<PostEntity>().Match(p => p.ObjectURI == rootId).ExecuteFirstAsync(Token)).ID, stored.AnsweringToPostId);
}
// Forte gives a post's Update the id of its Create, and sends it only added to the thread's context
[Fact]
public async Task An_update_added_under_its_creates_id_is_taken_as_the_edit()
{
var (_, alice) = await _harness.Persona("alice");
var owner = new RemoteActor(_harness.Peer, "owner", ed25519: true);
await DB.Default.SaveAsync(new Following { AvatarId = alice.Id, TargetActorURI = owner.Id, TargetInboxURL = owner.Id + "/inbox", State = FollowState.Accepted }, Token);
var context = $"{Origin(owner)}/conversation/{Guid.NewGuid():N}";
var create = owner.Prove(Create(owner, "as first written", context));
JsonObject Added(JsonObject inner) => owner.Prove(new JsonObject
{
["id"] = $"{Origin(owner)}/activity/{Guid.NewGuid():N}", ["type"] = "Add", ["actor"] = owner.Id, ["object"] = inner,
["target"] = new JsonObject { ["id"] = context, ["type"] = "Collection", ["attributedTo"] = owner.Id }
});
await _harness.Deliver(owner, "/human-centipede", Added(create));
await RunQueued(create["id"]!.GetValue<string>());
var update = create.DeepClone().AsObject();
update.Remove("proof");
update["type"] = "Update";
update["object"]!["content"] = "<p>as edited</p>";
update["object"]!["updated"] = DateTime.UtcNow.AddSeconds(1).ToString("O");
owner.Prove(update);
await _harness.Deliver(owner, "/human-centipede", Added(update));
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey.StartsWith("inbox|forwarded|" + update["id"]!.GetValue<string>() + "|") && j.State == JobState.Pending)
.ExecuteFirstAsync(Token);
Assert.NotNull(job);
Assert.Equal(JobResult.Done, (await _harness.Processor.Handle(job, Token)).Result);
var noteId = create["object"]!["id"]!.GetValue<string>();
Assert.Contains("as edited", (await DB.Default.Find<PostEntity>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(Token)).ContentHtml);
}
[Fact]
public async Task An_add_to_a_collection_on_another_server_is_no_threads()
{
var owner = new RemoteActor(_harness.Peer, "owner", ed25519: true);
var commenter = new RemoteActor(_harness.Peer, "commenter", _harness.Peer.B, ed25519: true);
var reply = commenter.Prove(Create(commenter, "elsewhere", $"{Origin(commenter)}/conversation/1"));
await _harness.Deliver(owner, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(owner)}/activity/{Guid.NewGuid():N}", ["type"] = "Add", ["actor"] = owner.Id, ["object"] = reply,
["target"] = $"{Origin(commenter)}/conversation/1"
});
Assert.False(await DB.Default.Find<Job>().Match(j => j.DedupeKey == "inbox|forwarded|" + reply["id"]!.GetValue<string>()).ExecuteAnyAsync(Token));
}
}
}
+143
View File
@@ -0,0 +1,143 @@
using MongoDB.Entities;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// pins across servers: an account elsewhere features its own posts (Add and Remove on its `featured` collection, and
// the collection itself), a community features posts made in it, and a persona's pin is told to its followers
[Trait("Category", "Integration")]
public sealed class FeaturedTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
// a public note of the actor's, served where its id points
string Note(RemoteActor author, string text, string audience = default)
{
var path = $"/notes/{Guid.NewGuid():N}";
var note = new JsonObject
{
["id"] = Origin(author) + path, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"<p>{text}</p>",
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"), ["published"] = DateTime.UtcNow.ToString("O")
};
if (audience != default)
note["audience"] = audience;
_harness.Peer.Serve(path, note.ToJsonString());
return Origin(author) + path;
}
JsonObject Featuring(RemoteActor actor, string type, string objectUri, string target = default) => new()
{
["id"] = $"{actor.Id}#{type}-{Guid.NewGuid():N}", ["type"] = type, ["actor"] = actor.Id, ["object"] = objectUri,
["target"] = target ?? actor.Id + "/featured"
};
Task<List<RemoteFeatured>> FeaturedBy(RemoteActor actor) =>
DB.Default.Find<RemoteFeatured>().Match(f => f.ActorURI == actor.Id).Sort(f => f.Position, Order.Ascending).ExecuteAsync(TestContext.Current.CancellationToken);
[Fact]
public async Task An_account_features_its_own_post_and_takes_it_back_but_nothing_else()
{
var bob = new RemoteActor(_harness.Peer, "bob");
var carol = new RemoteActor(_harness.Peer, "carol", _harness.Peer.B);
var mine = Note(bob, "pinned");
var theirs = Note(carol, "not bob's");
var older = Note(bob, "pinned before");
await _harness.Deliver(bob, "/human-centipede", Featuring(bob, "Add", older));
await _harness.Deliver(bob, "/human-centipede", Featuring(bob, "Add", mine));
await _harness.Deliver(bob, "/human-centipede", Featuring(bob, "Add", theirs));
// Smithereen's wall is a collection of the actor's too, but no pin
await _harness.Deliver(bob, "/human-centipede", Featuring(bob, "Add", Note(bob, "on the wall"), bob.Id + "/wall"));
Assert.Equal(new[] { mine, older }, (await FeaturedBy(bob)).Select(f => f.ObjectURI));
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == mine).ExecuteSingleAsync(TestContext.Current.CancellationToken);
Assert.Equal(post.ID, (await FeaturedBy(bob))[0].PostId);
await _harness.Deliver(bob, "/human-centipede", Featuring(bob, "Remove", mine));
Assert.Equal(new[] { older }, (await FeaturedBy(bob)).Select(f => f.ObjectURI));
}
[Fact]
public async Task The_featured_collection_is_read_in_its_order_from_the_accounts_own_server()
{
var token = TestContext.Current.CancellationToken;
var bob = new RemoteActor(_harness.Peer, "bob");
var first = Note(bob, "first pin");
var second = Note(bob, "second pin");
_harness.Peer.Serve(new Uri(bob.Id).AbsolutePath + "/featured", new JsonObject
{
["id"] = bob.Id + "/featured", ["type"] = "OrderedCollection", ["totalItems"] = 2,
["orderedItems"] = new JsonArray(new JsonObject { ["id"] = first, ["type"] = "Note" }, second)
}.ToJsonString());
var account = await _harness.Remote.GetActor(bob.Id, refresh: false, token);
await _harness.Featured.Sync(account, token);
Assert.Equal(new[] { first, second }, (await FeaturedBy(bob)).Select(f => f.ObjectURI));
}
[Fact]
public async Task A_personas_pin_and_unpin_reach_its_followers_as_add_and_remove()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var follower = new RemoteActor(_harness.Peer, "fan");
await _harness.FollowedBy(alice, follower);
await _harness.Posts.InsertPost(root, new PrivaPub.ClientModels.Post.InsertPostForm { AvatarId = alice.Id, Text = "worth a pin" }, token);
var post = await DB.Default.Find<Post>().Match(p => p.GroupUserId == alice.Id).ExecuteFirstAsync(token);
await _harness.Outbox.PublishFeatured(alice.Id, post, featured: true, token);
await _harness.Outbox.PublishFeatured(alice.Id, post, featured: false, token);
var sent = (await _harness.Outgoing(follower.SharedInbox)).Concat(await _harness.Outgoing(follower.Id + "/inbox"))
.Where(a => a["type"]!.GetValue<string>() is "Add" or "Remove").ToList();
Assert.Equal(new[] { "Add", "Remove" }, sent.Select(a => a["type"]!.GetValue<string>()));
Assert.All(sent, a => Assert.Equal((alice.PostUri(post.ID), alice.Featured), (a["object"]!.GetValue<string>(), a["target"]!.GetValue<string>())));
}
[Fact]
public async Task A_community_features_a_post_made_in_it()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "books", type: "Group");
var member = new RemoteActor(_harness.Peer, "reader", _harness.Peer.B);
var account = await _harness.Remote.GetActor(community.Id, refresh: false, token);
await DB.Default.SaveAsync(new Following { AvatarId = alice.Id, TargetActorURI = community.Id, TargetAccountId = account.ID, State = FollowState.Accepted }, token);
var post = Note(member, "a thread in the community", community.Id);
await _harness.Deliver(community, "/human-centipede", new JsonObject
{
["id"] = $"{community.Id}#announce-{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = community.Id,
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"),
["object"] = new JsonObject
{
["id"] = $"{Origin(member)}/activities/{Guid.NewGuid():N}", ["type"] = "Add", ["actor"] = member.Id, ["object"] = post,
["target"] = community.Id + "/featured"
}
});
Assert.Equal(new[] { post }, (await FeaturedBy(community)).Select(f => f.ObjectURI));
}
}
}
@@ -0,0 +1,282 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// FEP-8fcf (owner decision 2026-10-06): a delivery addressed to a persona's followers tells the receiving server, in a signed
// header, a digest of the persona's followers there; the roll-call it names lists them, to that server only
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class FollowersSynchronizationTests : IAsyncLifetime
{
static readonly string[] PeerHosts = { "localhost", "127.0.0.1" };
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
[Fact]
public void The_digest_is_the_xor_of_each_ids_sha256_whatever_the_order()
{
Assert.Equal(new string('0', 64), FollowersSynchronization.Digest([]));
Assert.Equal(FollowersSynchronization.Digest(["https://a.example/users/one", "https://a.example/users/two"]),
FollowersSynchronization.Digest(["https://a.example/users/two", "https://a.example/users/one"]));
Assert.Equal(Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes("https://a.example/users/one"))),
FollowersSynchronization.Digest(["https://a.example/users/one"]));
}
async Task<HttpRequestRecord> Delivered(LocalActor signer, string inbox, JsonObject activity)
{
await _harness.Delivery.Enqueue(signer, new[] { inbox }, activity, Token);
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == $"{activity["id"]!.GetValue<string>()}|{inbox}").ExecuteSingleAsync(Token);
var handler = new DeliveryJobHandler(_harness.Local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())),
NullLogger<DeliveryJobHandler>.Instance);
Assert.Equal(JobResult.Done, (await handler.Handle(job, Token)).Result);
return Assert.Single(_harness.Peer.Requests, r => r.Body?.Contains(activity["id"]!.GetValue<string>()) == true);
}
static JsonObject Create(LocalActor author, params string[] to) => new()
{
["id"] = $"{author.Uri}/grunts/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Uri, ["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray()),
["object"] = new JsonObject { ["type"] = "Note", ["content"] = "hi", ["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray()) }
};
[Fact]
public async Task A_delivery_to_followers_carries_a_signed_digest_of_the_followers_on_that_server_only()
{
var (_, alice) = await _harness.Persona("alice");
var one = new RemoteActor(_harness.Peer, "one");
var two = new RemoteActor(_harness.Peer, "two");
var far = new RemoteActor(_harness.Peer, "far", _harness.Peer.B);
await _harness.FollowedBy(alice, one);
await _harness.FollowedBy(alice, two);
await _harness.FollowedBy(alice, far);
await DB.Default.SaveAsync(new Follower
{
LocalActorId = alice.Id, LocalActorKind = alice.Kind, ActorURI = _harness.Peer.A + "/users/asking", InboxURL = _harness.Peer.A + "/users/asking/inbox", IsAccepted = false
}, Token);
_harness.Peer.Answer("/inbox", 202);
var received = await Delivered(alice, _harness.Peer.A + "/inbox", Create(alice, alice.Followers));
Assert.Equal(alice.Followers, Value(received, "collectionId"));
Assert.Equal(alice.Followers + "/roll-call", Value(received, "url"));
Assert.Equal(FollowersSynchronization.Digest([one.Id, two.Id]), Value(received, "digest"));
var signature = HttpSignatures.Parse(received.Signature);
Assert.Equal(new[] { "(request-target)", "host", "date", "digest", "collection-synchronization" }, signature.Headers);
var signingString = $"(request-target): post /inbox\nhost: {received.Headers["Host"]}\ndate: {received.Headers["Date"]}\ndigest: {received.Headers["Digest"]}\n"
+ $"collection-synchronization: {received.Headers[FollowersSynchronization.Header]}";
using var key = RSA.Create();
key.ImportFromPem(alice.PublicKeyPem);
Assert.True(key.VerifyData(Encoding.UTF8.GetBytes(signingString), signature.Signature, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
}
static string Value(HttpRequestRecord received, string name)
{
var values = received.Headers[FollowersSynchronization.Header].Split(',', StringSplitOptions.TrimEntries)
.Select(p => p.Split('=', 2)).ToDictionary(p => p[0], p => p[1].Trim('"'));
return values[name];
}
[Fact]
public async Task A_delivery_that_is_not_for_followers_carries_none()
{
var (_, alice) = await _harness.Persona("alice");
var recipient = new RemoteActor(_harness.Peer, "recipient");
await _harness.FollowedBy(alice, recipient);
_harness.Peer.Answer("/inbox", 202);
var received = await Delivered(alice, _harness.Peer.A + "/inbox", Create(alice, recipient.Id));
Assert.False(received.Headers.ContainsKey(FollowersSynchronization.Header));
Assert.DoesNotContain("collection-synchronization", HttpSignatures.Parse(received.Signature).Headers);
}
// Mastodon undoes a follow it never knew of with the same id each time ({actor}#follows//undo): when it comes again,
// after a new follow, it ends that one too; a copy of it, with nothing to end, stays a copy
[Fact]
public async Task An_undo_of_a_follow_sent_again_after_a_new_follow_ends_that_one_too()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var undo = new JsonObject
{
["id"] = bob.Id + "#follows//undo", ["type"] = "Undo", ["actor"] = bob.Id,
["object"] = new JsonObject { ["id"] = bob.Id + "#follows/", ["type"] = "Follow", ["actor"] = bob.Id, ["object"] = alice.Uri }
};
Task<bool> Follows() => DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == bob.Id).ExecuteAnyAsync(Token);
await _harness.FollowedBy(alice, bob);
await _harness.Deliver(bob, "/human-centipede", undo.DeepClone());
Assert.False(await Follows());
await _harness.FollowedBy(alice, bob);
await _harness.Deliver(bob, "/human-centipede", undo.DeepClone());
Assert.False(await Follows());
Assert.Equal("duplicate", (await _harness.Deliver(bob, "/human-centipede", undo.DeepClone())).Reason);
}
// received: bob's delivery claims a digest of his followers here, and lists them at his own synchronisation URL
async Task Claimed(RemoteActor bob, string digest, params string[] listed)
{
var path = new Uri(bob.Id).AbsolutePath + "/followers/sync";
_harness.Peer.Serve(path, new JsonObject
{
["id"] = bob.Id + "/followers/sync", ["type"] = "OrderedCollection",
["orderedItems"] = new JsonArray(listed.Select(l => (JsonNode)l).ToArray())
}.ToJsonString());
var note = $"{bob.Id}/notes/{Guid.NewGuid():N}";
var request = bob.Post(Harness.Host, "/human-centipede", new JsonObject
{
["id"] = note + "/activity", ["type"] = "Create", ["actor"] = bob.Id, ["to"] = new JsonArray(bob.Id + "/followers"),
["object"] = new JsonObject { ["id"] = note, ["type"] = "Note", ["attributedTo"] = bob.Id, ["to"] = new JsonArray(bob.Id + "/followers"), ["content"] = "hi" }
});
request.Headers[FollowersSynchronization.Header] = $"collectionId=\"{bob.Id}/followers\", url=\"{bob.Id}/followers/sync\", digest=\"{digest}\"";
await _harness.Receiver.Receive(request, default, Token);
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.SynchronizeFollowing && j.State == JobState.Pending && j.Payload.Contains(bob.Id))
.ExecuteSingleAsync(Token);
var handler = new FollowingSynchronization(_harness.Db, _harness.Local, _harness.Remote, _harness.Follows, _harness.Delivery);
Assert.Equal(JobResult.Done, (await handler.Handle(job, Token)).Result);
}
async Task<Following> Follows(LocalActor persona, RemoteActor bob, FollowState state)
{
var following = new Following
{
AvatarId = persona.Id, TargetActorURI = bob.Id, TargetInboxURL = bob.Id + "/inbox", State = state, FollowActivityURI = persona.ActivityUri($"follow-{Guid.NewGuid():N}")
};
await DB.Default.SaveAsync(following, Token);
return following;
}
Task<Following> Following(LocalActor persona, RemoteActor bob) =>
DB.Default.Find<Following>().Match(f => f.AvatarId == persona.Id && f.TargetActorURI == bob.Id).ExecuteFirstAsync(Token);
[Fact]
public async Task A_claim_that_differs_is_mended_from_the_accounts_own_list()
{
var (_, alice) = await _harness.Persona("alice");
var (_, carol) = await _harness.Persona("carol");
var (_, dave) = await _harness.Persona("dave");
var bob = new RemoteActor(_harness.Peer, "bob");
await Follows(alice, bob, FollowState.Accepted);
await Follows(carol, bob, FollowState.Requested);
await Claimed(bob, FollowersSynchronization.Digest([carol.Uri, dave.Uri]), carol.Uri, dave.Uri);
Assert.Null(await Following(alice, bob));
Assert.Equal(FollowState.Accepted, (await Following(carol, bob)).State);
var undos = (await _harness.Outgoing(bob.Id + "/inbox")).Where(a => a["type"]!.GetValue<string>() == "Undo").ToList();
Assert.Equal(new[] { alice.Uri, dave.Uri }.Order(), undos.Select(u => u["actor"]!.GetValue<string>()).Order());
var daves = Assert.Single(undos, u => u["actor"]!.GetValue<string>() == dave.Uri);
Assert.Equal(("Follow", bob.Id), (daves["object"]!["type"]!.GetValue<string>(), daves["object"]!["object"]!.GetValue<string>()));
}
[Fact]
public async Task A_list_the_claim_does_not_describe_ends_no_follow()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
await Follows(alice, bob, FollowState.Accepted);
await Claimed(bob, FollowersSynchronization.Digest(["https://privapub.test/peasants/someone"]));
Assert.Equal(FollowState.Accepted, (await Following(alice, bob)).State);
}
[Fact]
public async Task A_claim_that_agrees_reads_nothing()
{
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
await Follows(alice, bob, FollowState.Accepted);
await Claimed(bob, FollowersSynchronization.Digest([alice.Uri]));
Assert.DoesNotContain(_harness.Peer.Requests, r => r.Path.EndsWith("/followers/sync"));
Assert.Equal(FollowState.Accepted, (await Following(alice, bob)).State);
}
}
[Trait("Category", "Integration")]
public sealed class RollCallTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
static string[] Items(JsonObject collection) => collection["orderedItems"]!.AsArray().Select(i => i!.GetValue<string>()).ToArray();
[Fact]
public async Task The_roll_call_lists_the_followers_on_the_signing_server_and_asks_for_a_signature()
{
var token = TestContext.Current.CancellationToken;
var persona = await _host.Persona(await _host.SignUp(), "rollcall");
var near = new RemoteActor(_peer, "near");
var server = new RemoteActor(_peer, "instance", type: "Application");
var far = new RemoteActor(_peer, "far", _peer.B);
foreach (var follower in new[] { near, far })
await DB.Default.SaveAsync(new Follower
{
LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = follower.Id, InboxURL = follower.Id + "/inbox"
}, token);
var path = $"/peasants/{persona.UserName}/groupies/roll-call";
var asServer = await _client.Fetch(server.SignedGet(path));
var asFar = await _client.Fetch(far.SignedGet(path));
Assert.Equal(HttpStatusCode.OK, asServer.Status);
Assert.Equal(persona.ActorUri() + "/groupies/roll-call", asServer.Json["id"]!.GetValue<string>());
Assert.Equal(new[] { near.Id }, Items(asServer.Json));
Assert.Equal(new[] { far.Id }, Items(asFar.Json));
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Fetch(path)).Status);
}
}
}
@@ -0,0 +1,97 @@
using MongoDB.Entities;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// Pleroma's and Akkoma's edited posts carry their earlier versions (formerRepresentations): a post first met after its
// edits has its history, and an edit tells the ones missed in between
[Trait("Category", "Integration")]
public sealed class FormerRepresentationsTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static readonly DateTime Start = DateTime.UtcNow.AddHours(-1);
static JsonObject Version(string text, int minutes) => new()
{
["type"] = "Note", ["content"] = $"<p>{text}</p>", ["published"] = Start.ToString("O"),
["updated"] = minutes == 0 ? default : Start.AddMinutes(minutes).ToString("O")
};
async Task<(RemoteActor Carol, JsonObject Note)> Followed(string text, int minutes, params JsonObject[] former)
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
await DB.Default.SaveAsync(new Following { AvatarId = alice.Id, TargetActorURI = carol.Id, State = FollowState.Accepted }, token);
var note = Version(text, minutes);
note["id"] = $"{Origin(carol)}/objects/{Guid.NewGuid():N}";
note["attributedTo"] = carol.Id;
note["to"] = new JsonArray(Addressing.Public);
note["cc"] = new JsonArray($"{carol.Id}/followers");
if (former.Length > 0)
note["formerRepresentations"] = new JsonObject { ["type"] = "OrderedCollection", ["totalItems"] = former.Length, ["orderedItems"] = new JsonArray(former) };
return (carol, note);
}
Task Deliver(RemoteActor carol, string type, JsonObject note) => _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/activities/{Guid.NewGuid():N}", ["type"] = type, ["actor"] = carol.Id,
["to"] = new JsonArray(Addressing.Public), ["object"] = note.DeepClone()
});
Task<Post> Stored(JsonObject note) =>
DB.Default.Find<Post>().Match(p => p.ObjectURI == note["id"]!.GetValue<string>()).ExecuteSingleAsync(TestContext.Current.CancellationToken);
[Fact]
public async Task A_post_first_met_after_its_edits_has_its_history()
{
var (carol, note) = await Followed("third", 40, Version("second", 20), Version("first", 0));
await Deliver(carol, "Create", note);
var post = await Stored(note);
Assert.Equal(["<p>first</p>", "<p>second</p>"], post.Revisions.Select(r => r.ContentHtml));
Assert.Equal("<p>third</p>", post.ContentHtml);
}
[Fact]
public async Task An_edit_tells_the_versions_missed_in_between()
{
var (carol, note) = await Followed("first", 0);
await Deliver(carol, "Create", note);
var edited = (JsonObject)note.DeepClone();
edited["content"] = "<p>third</p>";
edited["updated"] = Start.AddMinutes(40).ToString("O");
edited["formerRepresentations"] = new JsonObject
{
["type"] = "OrderedCollection", ["totalItems"] = 2, ["orderedItems"] = new JsonArray(Version("second", 20), Version("first", 0))
};
await Deliver(carol, "Update", edited);
var post = await Stored(note);
Assert.Equal(["<p>first</p>", "<p>second</p>"], post.Revisions.Select(r => r.ContentHtml));
Assert.Equal("<p>third</p>", post.ContentHtml);
}
}
}
+167
View File
@@ -0,0 +1,167 @@
using MongoDB.Entities;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
// Inbox forwarding: a thread's server passes on the activities in it, signed with its own key (Mastodon the replies to
// its accounts' posts and their deletions, Friendica everything in its threads). They used to be refused with 401.
[Trait("Category", "Integration")]
public sealed class ForwardedTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
PrivaPub.Models.Statistics.InteractionEvent Processed(string activity) => _harness.Ledger.Of("in").Last(e => e.Activity == activity);
PrivaPub.Models.Statistics.InteractionEvent Received(string activity) => _harness.Ledger.Of("recv").Last(e => e.Activity == activity);
Task<Post> Stored(string objectUri) =>
DB.Default.Find<Post>().Match(p => p.ObjectURI == objectUri).ExecuteFirstAsync(TestContext.Current.CancellationToken);
// alice follows the thread's author (on its own server), whose post she holds; bob, elsewhere, replies to it
async Task<(RemoteActor Owner, RemoteActor Bob, JsonObject Reply)> Thread()
{
var (_, alice) = await _harness.Persona("alice");
var owner = new RemoteActor(_harness.Peer, "owner", _harness.Peer.B);
var bob = new RemoteActor(_harness.Peer, "bob");
await Follows(alice.Id, owner);
var post = PublicNote(owner, "<p>the thread</p>");
await _harness.Deliver(owner, "/human-centipede", Create(owner, post));
Assert.NotNull(await Stored(IdOf(post)));
var reply = PublicNote(bob, "<p>bob's reply as he wrote it</p>", owner.Id);
reply["inReplyTo"] = IdOf(post);
_harness.Peer.Serve(new Uri(IdOf(reply)).AbsolutePath, reply.ToJsonString());
return (owner, bob, reply);
}
[Fact]
public async Task A_reply_the_threads_server_forwards_is_kept_as_its_author_wrote_it()
{
var (owner, bob, reply) = await Thread();
var claimed = (JsonObject)reply.DeepClone();
claimed["content"] = "<p>what the forwarder says bob wrote</p>";
var result = await _harness.Deliver(owner, "/human-centipede", Create(bob, claimed));
Assert.Equal((202, "forwarded"), (result.StatusCode, result.Reason));
var stored = await Stored(IdOf(reply));
Assert.NotNull(stored);
Assert.Equal(bob.Id, stored.ActorURI);
Assert.Contains("as he wrote it", stored.ContentHtml);
Assert.Equal(("accepted", "stored"), (Processed("Create").Outcome, Processed("Create").Reason));
Assert.True((await DB.Default.Find<PrivaPub.Models.Federation.ObjectRecord>().Match(r => r.ObjectURI == IdOf(reply)).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Refetched);
}
[Fact]
public async Task A_forwarded_edit_applies_the_post_as_its_origin_has_it_now()
{
var (owner, bob, reply) = await Thread();
await _harness.Deliver(owner, "/human-centipede", Create(bob, reply));
var edited = (JsonObject)reply.DeepClone();
edited["content"] = "<p>bob's reply, edited</p>";
edited["updated"] = DateTime.UtcNow.AddMinutes(1).ToString("O");
_harness.Peer.Serve(new Uri(IdOf(reply)).AbsolutePath, edited.ToJsonString());
var claimed = (JsonObject)edited.DeepClone();
claimed["content"] = "<p>an edit bob never made</p>";
await _harness.Deliver(owner, "/human-centipede", Activity(bob, "Update", claimed));
Assert.Contains("bob's reply, edited", (await Stored(IdOf(reply))).ContentHtml);
}
[Fact]
public async Task A_forwarded_deletion_waits_for_the_origin_to_say_the_post_is_gone()
{
var (owner, bob, reply) = await Thread();
await _harness.Deliver(owner, "/human-centipede", Create(bob, reply));
await _harness.Deliver(owner, "/human-centipede", Activity(bob, "Delete", JsonValue.Create(IdOf(reply))!));
Assert.Equal(("dropped", "forwarded-not-gone"), (Processed("Delete").Outcome, Processed("Delete").Reason));
Assert.NotNull(await Stored(IdOf(reply)));
_harness.Peer.Answer(new Uri(IdOf(reply)).AbsolutePath, 410);
await _harness.Deliver(owner, "/human-centipede", Activity(bob, "Delete", JsonValue.Create(IdOf(reply))!));
Assert.Null(await Stored(IdOf(reply)));
}
// the origin answers 404 for a followers-only post to our instance actor too: only its author's server says it is gone
[Fact]
public async Task A_forwarded_deletion_of_a_followers_only_post_is_left_to_its_author()
{
var (owner, bob, reply) = await Thread();
await _harness.Deliver(owner, "/human-centipede", Create(bob, reply));
await DB.Default.Update<Post>().Match(p => p.ObjectURI == IdOf(reply)).Modify(p => p.Visibility, PostVisibility.FollowersOnly).ExecuteAsync(TestContext.Current.CancellationToken);
_harness.Peer.Answer(new Uri(IdOf(reply)).AbsolutePath, 404);
await _harness.Deliver(owner, "/human-centipede", Activity(bob, "Delete", JsonValue.Create(IdOf(reply))!));
Assert.Equal(("dropped", "forwarded-private"), (Processed("Delete").Outcome, Processed("Delete").Reason));
Assert.NotNull(await Stored(IdOf(reply)));
}
// a forwarded copy is kept apart from the author's own delivery, which carries what the copy could not
[Fact]
public async Task The_authors_own_delivery_is_still_taken_after_a_forwarded_copy_that_failed()
{
var (owner, bob, reply) = await Thread();
_harness.Peer.Answer(new Uri(IdOf(reply)).AbsolutePath, 404);
var create = Create(bob, reply);
await _harness.Deliver(owner, "/human-centipede", create);
Assert.Null(await Stored(IdOf(reply)));
var direct = await _harness.Deliver(bob, "/human-centipede", create);
Assert.Equal("queued", direct.Reason);
Assert.NotNull(await Stored(IdOf(reply)));
}
// what cannot be checked against its origin (a vote, a follow, someone else's post) is let go, answered 202: a 401
// tells the forwarder its signature failed
[Fact]
public async Task What_a_forwarder_cannot_vouch_for_is_let_go_without_an_error()
{
var (owner, bob, reply) = await Thread();
var like = new JsonObject { ["id"] = NewId(bob, "likes"), ["type"] = "Like", ["actor"] = bob.Id, ["object"] = IdOf(reply) };
var result = await _harness.Deliver(owner, "/human-centipede", like);
Assert.Equal((202, "forwarded-ignored"), (result.StatusCode, result.Reason));
Assert.Equal("dropped", Received("Like").Outcome);
var onTheForwardersServer = PublicNote(owner, "<p>a post of the forwarder's</p>");
onTheForwardersServer["attributedTo"] = bob.Id;
result = await _harness.Deliver(owner, "/human-centipede", Create(bob, onTheForwardersServer));
Assert.Equal((202, "forwarded-ignored"), (result.StatusCode, result.Reason));
Assert.Null(await Stored(IdOf(onTheForwardersServer)));
// our own, coming back from a thread on another server
var (_, carol) = await _harness.Persona("carol");
var ours = new JsonObject
{
["id"] = carol.Uri + "/grunts/create-" + Guid.NewGuid().ToString("N"),
["type"] = "Create",
["actor"] = carol.Uri,
["object"] = new JsonObject { ["id"] = carol.Uri + "/scribbles/" + Guid.NewGuid().ToString("N"), ["type"] = "Note", ["attributedTo"] = carol.Uri }
};
result = await _harness.Deliver(owner, "/human-centipede", ours);
Assert.Equal((202, "forwarded-ignored"), (result.StatusCode, result.Reason));
}
}
}
+104 -1
View File
@@ -9,6 +9,7 @@ using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
@@ -89,6 +90,28 @@ namespace PrivaPub.Tests.Federation
Assert.Contains(announces, a => a["object"] is JsonValue);
}
[Fact]
public async Task A_comment_in_a_community_is_announced_activity_and_object_too()
{
var (_, owner) = await _harness.Persona("owner");
var (_, community) = await Group(GroupKind.Community, PostingPolicy.Followers, owner.Id);
var lemmy = new RemoteActor(_harness.Peer, "lemmy");
var akkoma = new RemoteActor(_harness.Peer, "akkoma");
await _harness.FollowedBy(community, lemmy);
await _harness.FollowedBy(community, akkoma);
var thread = Create(lemmy, new[] { community.Uri, Addressing.Public }, community.Uri);
await _harness.Deliver(lemmy, "/human-centipede", thread);
var comment = Create(lemmy, new[] { community.Uri, Addressing.Public }, community.Uri);
comment["object"]!["inReplyTo"] = thread["object"]!["id"]!.GetValue<string>();
await _harness.Deliver(lemmy, "/human-centipede", comment);
var commentId = comment["object"]!["id"]!.GetValue<string>();
var announces = (await _harness.Outgoing(akkoma.SharedInbox)).Where(a => a["actor"]!.GetValue<string>() == community.Uri).ToList();
Assert.Contains(announces, a => a["object"] is JsonObject inner && inner["object"]?["id"]?.GetValue<string>() == commentId);
Assert.Contains(announces, a => a["object"] is JsonValue bare && bare.GetValue<string>() == commentId);
}
[Fact]
public async Task An_open_community_takes_posts_from_anyone()
{
@@ -102,6 +125,21 @@ namespace PrivaPub.Tests.Federation
Assert.True(await DB.Default.Find<Post>().Match(p => p.GroupId == community.Id && p.ActorURI == stranger.Id).ExecuteAnyAsync(token));
}
// Lemmy keeps its user's post pending until the community announces it back, whether anyone there follows the
// community or not
[Fact]
public async Task A_post_into_a_community_is_announced_back_to_its_authors_server_though_nobody_there_follows_it()
{
var (_, owner) = await _harness.Persona("owner");
var (_, community) = await Group(GroupKind.Community, PostingPolicy.Anyone, owner.Id);
var stranger = new RemoteActor(_harness.Peer, "stranger", sharedInbox: true);
await _harness.Deliver(stranger, "/human-centipede", Create(stranger, new[] { community.Uri, Addressing.Public }, community.Uri));
var announces = (await _harness.Outgoing(stranger.SharedInbox)).Where(a => a["actor"]!.GetValue<string>() == community.Uri).ToList();
Assert.Contains(announces, a => a["object"] is JsonObject inner && inner["type"]!.GetValue<string>() == "Create");
}
[Fact]
public async Task Mentioning_a_community_posts_into_it_as_a_titled_page()
{
@@ -138,21 +176,86 @@ namespace PrivaPub.Tests.Federation
Assert.Equal(PostVisibility.Circle, outcome.Post.Visibility);
var create = Assert.Single(await _harness.Outgoing(member.Id + "/inbox"));
Assert.Equal(new[] { circle.Uri, circle.Flock }, create["object"]!["to"]!.AsArray().Select(t => t!.GetValue<string>()));
// the member's copy names the member, so Mastodon and GoToSocial keep it, and names nobody else
Assert.Equal(new[] { member.Id }, create["cc"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.Equal(new[] { member.Id }, create["object"]!["cc"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.Contains(create["object"]!["tag"]!.AsArray(), t => t!["type"]!.GetValue<string>() == "Mention" && t["href"]!.GetValue<string>() == member.Id);
Assert.DoesNotContain(Addressing.Public, create.ToJsonString());
Assert.Empty(await _harness.Outgoing(follower.SharedInbox));
Assert.Empty((await _harness.Outgoing(member.SharedInbox)).Where(a => a["type"]!.GetValue<string>() == "Announce"));
var authorizer = new SignedFetchAuthorizer(_harness.Remote);
var authorizer = new SignedFetchAuthorizer(_harness.Remote, new DbEntities());
var path = new Uri(outcome.Post.ObjectURI).AbsolutePath;
var asMember = await authorizer.Requester(member.Get(Harness.Host, path), token);
var asOutsider = await authorizer.Requester(outsider.Get(Harness.Host, path), token);
Assert.True(SignedFetchAuthorizer.MayReadCircle(circleEntity, asMember));
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, asOutsider));
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, default));
Assert.True(await authorizer.MayRead(outcome.Post, asMember, token));
Assert.False(await authorizer.MayRead(outcome.Post, asOutsider, token));
Assert.Equal(new[] { member.Id }, SignedFetchAuthorizer.CircleReaders(circleEntity, asMember));
Assert.True(circle.IsCircle);
Assert.False(circle.Discoverable);
}
[Fact]
public async Task A_circle_posts_edit_and_delete_reach_each_member_naming_only_that_member()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var first = new RemoteActor(_harness.Peer, "first");
var second = new RemoteActor(_harness.Peer, "second", _harness.Peer.B);
var (_, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id, first.Id, second.Id);
await _harness.Remote.GetActor(first.Id, refresh: false, token);
await _harness.Remote.GetActor(second.Id, refresh: false, token);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
await _harness.Statuses.Edit(alice, outcome.Post.ID, new StatusDraft { Text = "just us, edited" }, token);
await _harness.Statuses.Remove(alice, outcome.Post.ID, token);
foreach (var (member, other) in new[] { (first, second), (second, first) })
{
var sent = await _harness.Outgoing(member.Id + "/inbox");
Assert.Equal(new[] { "Create", "Update", "Delete" }, sent.Select(a => a["type"]!.GetValue<string>()));
Assert.All(sent, a => Assert.Contains(member.Id, a["cc"]!.AsArray().Select(c => c!.GetValue<string>())));
Assert.All(sent, a => Assert.DoesNotContain(other.Id, a.ToJsonString()));
}
}
[Fact]
public async Task A_reply_to_a_circle_post_stays_in_it_and_a_circle_post_asks_nobody_outside_for_a_quote()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var (entity, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id);
entity.Members.Add(new GroupMember { AvatarId = bob.Id });
await DB.Default.SaveAsync(entity, token);
var outsider = new RemoteActor(_harness.Peer, "asked");
var outsiderAccount = await _harness.Remote.GetActor(outsider.Id, refresh: false, token);
var asksFirst = new Post
{
ObjectURI = $"{Origin(outsider)}/notes/{Guid.NewGuid():N}",
ActorURI = outsider.Id,
GroupUserId = outsiderAccount.ID,
AuthorAccountId = outsiderAccount.ID,
IsFederatedCopy = true,
Visibility = PostVisibility.Public,
ContentHtml = "<p>ask me first</p>",
QuotePolicy = new InteractionRule { Manual = new() { Addressing.Public } }
};
await DB.Default.SaveAsync(asksFirst, token);
var root = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
var reply = await _harness.Statuses.Publish(bob, new StatusDraft { Text = "still just us", InReplyTo = root.Post.ID }, token);
var quote = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "look", GroupId = circle.Id, QuotedStatusId = asksFirst.ID }, token);
Assert.Equal(PostVisibility.Circle, reply.Post.Visibility);
Assert.Equal(circle.Id, reply.Post.GroupId);
Assert.Equal(422, quote.Status);
Assert.Empty(await _harness.Outgoing(outsider.Id + "/inbox"));
}
[Fact]
public async Task Only_circle_members_can_post_into_a_circle()
{
@@ -101,6 +101,33 @@ namespace PrivaPub.Tests.Federation
Assert.Equal("the signature has expired",
Check(MastodonRequest(extra: $",expires=\"{Now.AddMinutes(-20).ToUnixTimeSeconds()}\"")));
// hs2019 leaves the hash to the key: some sign RSA with SHA-512, which rsa-sha256 does not allow; and a SHA-512 digest
[Fact]
public void Accepts_hs2019_hashed_with_sha512_and_a_sha512_digest()
{
var context = new DefaultHttpContext();
context.Request.Method = "POST";
context.Request.Host = new HostString(Host);
context.Request.Path = "/peasants/bob/mouth";
context.Features.Get<IHttpRequestFeature>().RawTarget = "/peasants/bob/mouth";
var date = Now.ToString("r", CultureInfo.InvariantCulture);
var digest = "SHA-512=" + Convert.ToBase64String(SHA512.HashData(Body));
context.Request.Headers["Date"] = date;
context.Request.Headers["Digest"] = digest;
var signingString = $"(request-target): post /peasants/bob/mouth\nhost: {Host}\ndate: {date}\ndigest: {digest}";
var signature = Convert.ToBase64String(_key.SignData(Encoding.UTF8.GetBytes(signingString), HashAlgorithmName.SHA512, RSASignaturePadding.Pkcs1));
string Signed(string algorithm) => $"keyId=\"{KeyId}\",algorithm=\"{algorithm}\",headers=\"(request-target) host date digest\",signature=\"{signature}\"";
context.Request.Headers["Signature"] = Signed("hs2019");
var parameters = HttpSignatures.Parse(context.Request.Headers["Signature"].ToString());
Assert.Null(HttpSignatures.CheckRequest(context.Request, parameters, Body, Now));
Assert.True(HttpSignatures.Verify(PublicKey, HttpSignatures.SigningString(context.Request, parameters), parameters.Signature, parameters.Algorithm));
context.Request.Headers["Signature"] = Signed("rsa-sha256");
parameters = HttpSignatures.Parse(context.Request.Headers["Signature"].ToString());
Assert.False(HttpSignatures.Verify(PublicKey, HttpSignatures.SigningString(context.Request, parameters), parameters.Signature, parameters.Algorithm));
}
[Fact]
public void Signs_with_the_raw_request_target()
{
@@ -0,0 +1,77 @@
using MongoDB.Entities;
using PrivaPub.Domain.Relationships;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// A remote account's Block of a persona, as Mastodon sends it (the pasture's last Mastodon expected failure, G-0002)
[Trait("Category", "Integration")]
public sealed class InboundBlockTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
[Fact]
public async Task A_block_ends_the_follows_both_ways_hides_the_blocker_and_its_undo_lifts_it()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
await _harness.FollowedBy(alice, carol);
await DB.Default.SaveAsync(new Following { AvatarId = alice.Id, TargetActorURI = carol.Id, State = FollowState.Accepted }, token);
var block = new JsonObject
{
["id"] = $"{Origin(carol)}/blocks/{Guid.NewGuid():N}", ["type"] = "Block", ["actor"] = carol.Id, ["object"] = alice.Uri
};
await _harness.Deliver(carol, $"/peasants/{alice.UserName}/mouth", block);
Assert.True(await DB.Default.Find<BlockedBy>().Match(b => b.AvatarId == alice.Id && b.ActorURI == carol.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == carol.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id && f.TargetActorURI == carol.Id).ExecuteAnyAsync(token));
Assert.Contains(carol.Id, await Hidden.AuthorsHiddenFrom(alice.Id, new[] { carol.Id }, forNotifications: false, token));
Assert.Contains(alice.Id, await Hidden.RecipientsHiding(new[] { alice.Id }, carol.Id, token));
await _harness.Deliver(carol, $"/peasants/{alice.UserName}/mouth", new JsonObject
{
["id"] = $"{Origin(carol)}/undos/{Guid.NewGuid():N}", ["type"] = "Undo", ["actor"] = carol.Id, ["object"] = block
});
Assert.False(await DB.Default.Find<BlockedBy>().Match(b => b.AvatarId == alice.Id && b.ActorURI == carol.Id).ExecuteAnyAsync(token));
Assert.Empty(await Hidden.AuthorsHiddenFrom(alice.Id, new[] { carol.Id }, forNotifications: false, token));
}
[Fact]
public async Task A_block_of_someone_who_is_not_ours_is_dropped()
{
var token = TestContext.Current.CancellationToken;
var carol = new RemoteActor(_harness.Peer, "carol");
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/blocks/{Guid.NewGuid():N}", ["type"] = "Block", ["actor"] = carol.Id,
["object"] = $"{Origin(carol)}/users/somebody-else"
});
Assert.False(await DB.Default.Find<BlockedBy>().Match(b => b.ActorURI == carol.Id).ExecuteAnyAsync(token));
}
}
}
@@ -2,6 +2,7 @@ using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
@@ -106,6 +107,75 @@ namespace PrivaPub.Tests.Federation
Assert.NotNull(message.ConversationId);
}
// Mbin addresses its private messages to the recipient's profile page (its `url`), not to the actor
[Fact]
public async Task A_chat_message_addressed_to_a_personas_profile_page_reaches_the_persona()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mbin = new RemoteActor(_harness.Peer, "mbuser");
var messageId = NewId(mbin, "f/object");
await _harness.Deliver(mbin, "/human-centipede", new JsonObject
{
["id"] = NewId(mbin, "f/object"), ["type"] = "Create", ["actor"] = mbin.Id, ["to"] = new JsonArray(alice.HtmlUrl), ["cc"] = new JsonArray(),
["object"] = new JsonObject
{
["id"] = messageId, ["type"] = "ChatMessage", ["attributedTo"] = mbin.Id, ["to"] = new JsonArray(alice.HtmlUrl), ["cc"] = new JsonArray(),
["content"] = "<p>an answer</p>", ["mediaType"] = "text/html", ["published"] = DateTime.UtcNow.ToString("O")
}
});
var message = await DB.Default.Find<Post>().Match(p => p.ObjectURI == messageId).ExecuteFirstAsync(token);
Assert.NotNull(message);
Assert.Equal(PostVisibility.Direct, message.Visibility);
var conversation = await DB.Default.Find<DmGroup>().OneAsync(message.ConversationId, token);
Assert.Equal(new[] { alice.Id, mbin.Id }.Order(StringComparer.Ordinal), conversation.Members.Select(m => m.AvatarId).Order(StringComparer.Ordinal));
}
// Lemmy 0.19 and Mbin take a private message only as a ChatMessage: someone who writes to us that way is answered in
// kind, anyone else with a Note
[Fact]
public async Task A_direct_message_answers_in_the_form_its_recipient_writes_in()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var lemmy = new RemoteActor(_harness.Peer, "pm");
var mastodon = new RemoteActor(_harness.Peer, "dm");
async Task<Post> Received(RemoteActor sender, string type)
{
var id = NewId(sender, "messages");
await _harness.Deliver(sender, "/human-centipede", new JsonObject
{
["id"] = NewId(sender, "activities/create"), ["type"] = "Create", ["actor"] = sender.Id, ["to"] = new JsonArray(alice.Uri),
["object"] = new JsonObject
{
["id"] = id, ["type"] = type, ["attributedTo"] = sender.Id, ["to"] = new JsonArray(alice.Uri),
["content"] = "<p>psst</p>", ["published"] = DateTime.UtcNow.ToString("O")
}
});
return await DB.Default.Find<Post>().Match(p => p.ObjectURI == id).ExecuteSingleAsync(token);
}
async Task<JsonObject> Answer(RemoteActor to, Post message)
{
var answer = await _harness.Statuses.Publish(alice, new PrivaPub.Domain.Statuses.StatusDraft
{
Text = "psst back", Visibility = PostVisibility.Direct, ConversationId = message.ConversationId
}, token);
Assert.True(answer.Ok, answer.Error);
return (JsonObject)(await _harness.Outgoing(to.Id + "/inbox")).Last(a => a["type"]!.GetValue<string>() == "Create")["object"]!;
}
var chat = await Answer(lemmy, await Received(lemmy, "ChatMessage"));
var note = await Answer(mastodon, await Received(mastodon, "Note"));
Assert.Equal("ChatMessage", chat["type"]!.GetValue<string>());
Assert.Equal(lemmy.Id, Assert.Single(chat["to"]!.AsArray())!.GetValue<string>());
Assert.Null(chat["cc"]);
Assert.DoesNotContain("h-card", chat["content"]!.GetValue<string>());
Assert.Equal("Note", note["type"]!.GetValue<string>());
}
[Fact]
public async Task An_unreachable_key_asks_the_sender_to_retry_instead_of_refusing()
{
@@ -121,6 +191,68 @@ namespace PrivaPub.Tests.Federation
Assert.Equal(300, result.RetryAfterSeconds);
}
// Forte follows a persona by its profile page, as WebFinger names it, rather than by its actor's id; so does its Undo
[Fact]
public async Task A_follow_of_a_personas_profile_page_follows_the_persona()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var forte = new RemoteActor(_harness.Peer, "forte");
var follow = new JsonObject { ["id"] = NewId(forte, "follow"), ["type"] = "Follow", ["actor"] = forte.Id, ["object"] = alice.HtmlUrl };
Assert.Equal(202, (await _harness.Deliver(forte, "/human-centipede", follow)).StatusCode);
Assert.True(await DB.Default.Find<PrivaPub.Models.Federation.Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == forte.Id && f.IsAccepted).ExecuteAnyAsync(token));
Assert.Contains(await _harness.Outgoing(forte.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Accept");
await _harness.Deliver(forte, "/human-centipede", new JsonObject
{
["id"] = NewId(forte, "undo"), ["type"] = "Undo", ["actor"] = forte.Id,
["object"] = new JsonObject { ["type"] = "Follow", ["actor"] = forte.Id, ["object"] = alice.HtmlUrl }
});
Assert.False(await DB.Default.Find<PrivaPub.Models.Federation.Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == forte.Id).ExecuteAnyAsync(token));
}
// Castopod announces an episode with a note that only links to it: the post takes the episode's sound, title and cover
[Fact]
public async Task A_note_that_only_links_its_authors_episode_carries_the_episodes_sound()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var podcast = new RemoteActor(_harness.Peer, "pod");
await DB.Default.SaveAsync(new PrivaPub.Models.Social.Following
{
AvatarId = alice.Id, TargetActorURI = podcast.Id, TargetInboxURL = podcast.Id + "/inbox", State = PrivaPub.Models.Social.FollowState.Accepted
}, token);
var origin = new Uri(podcast.Id).GetLeftPart(UriPartial.Authority);
var episodePath = $"/episodes/{Guid.NewGuid():N}";
_harness.Peer.Serve(episodePath, new JsonObject
{
["id"] = origin + episodePath, ["type"] = "PodcastEpisode", ["attributedTo"] = podcast.Id, ["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"),
["description"] = new JsonObject { ["type"] = "Note", ["content"] = "the meadow", ["mediaType"] = "text/markdown" },
["image"] = new JsonObject { ["type"] = "Image", ["url"] = origin + "/cover.png" },
["audio"] = new JsonObject
{
["id"] = origin + "/one.mp3", ["type"] = "Audio", ["name"] = "Episode one", ["duration"] = 4.0,
["url"] = new JsonObject { ["href"] = origin + "/one.mp3", ["type"] = "Link", ["mediaType"] = "audio/mpeg" }
}
}.ToJsonString());
var noteId = NewId(podcast, "posts");
await _harness.Deliver(podcast, "/human-centipede", new JsonObject
{
["id"] = noteId + "/activity", ["type"] = "Create", ["actor"] = podcast.Id, ["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"),
["object"] = new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = podcast.Id, ["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"),
["content"] = $"<a href=\"{origin + episodePath}\">Episode one</a><br/>", ["published"] = DateTime.UtcNow.ToString("O")
}
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteSingleAsync(token);
Assert.Equal(origin + "/one.mp3", Assert.Single(post.Media).RemoteURL);
Assert.Equal(("Episode one", origin + "/cover.png", 4.0), (post.Title, post.CoverURL, post.Audio.DurationSeconds));
}
[Fact]
public async Task A_join_of_a_local_post_is_answered_with_ignore()
{
+569
View File
@@ -0,0 +1,569 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Social;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using System.Security.Cryptography;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class InboxGapTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static JsonObject Follow(RemoteActor follower, string target) =>
new() { ["id"] = NewId(follower, "follows"), ["type"] = "Follow", ["actor"] = follower.Id, ["object"] = target };
Task<ForeignAvatar> Stored(RemoteActor actor) =>
DB.Default.Find<ForeignAvatar>().Match(f => f.ActorURI == actor.Id).ExecuteSingleAsync(TestContext.Current.CancellationToken);
PrivaPub.Models.Statistics.InteractionEvent Processed(string activity) => _harness.Ledger.Of("in").Last(e => e.Activity == activity);
// a follow request nobody answers is sent again after 15 minutes, an hour, 6 hours... (Lemmy 1.0 lost the Accept of a
// follow it took before it started sending to us, and the community's posts were refused as not followed)
[Fact]
public async Task A_follow_request_nobody_answers_is_sent_again_with_a_growing_pause_until_it_is()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var quiet = new RemoteActor(_harness.Peer, "quiet");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = quiet.Id }, token);
var following = await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id && f.TargetActorURI == quiet.Id).ExecuteSingleAsync(token);
async Task<int> Follows() => (await _harness.Outgoing(quiet.Id + "/inbox")).Count(a => a["type"]!.GetValue<string>() == "Follow");
var made = following.CreatedAt;
Assert.Equal(1, await Follows());
// (the store is shared with other tests' requests: what this one got is counted on its own inbox)
async Task<int> After(TimeSpan wait)
{
var before = await Follows();
await _harness.Follows.ResendPending(made + wait, token);
return await Follows() - before;
}
Assert.Equal(0, await After(TimeSpan.FromMinutes(10)));
Assert.Equal(1, await After(TimeSpan.FromMinutes(16)));
Assert.Equal(0, await After(TimeSpan.FromMinutes(30)));
Assert.Equal(1, await After(TimeSpan.FromMinutes(77)));
Assert.Equal(3, await Follows());
// each one again under its own id (Lemmy never answers an id it has seen), and an answer naming it counts
var sentIds = (await _harness.Outgoing(quiet.Id + "/inbox")).Select(a => a["id"]!.GetValue<string>()).ToList();
Assert.Equal(new[] { following.FollowActivityURI, following.FollowActivityURI + "-again-1", following.FollowActivityURI + "-again-2" }, sentIds);
// a request older than the count is sent again too; an answered one never
await DB.Default.Update<Following>().MatchID(following.ID).Modify(b => b.Unset(f => f.Resent)).Modify(b => b.Unset(f => f.ResentAt)).ExecuteAsync(token);
Assert.Equal(1, await After(TimeSpan.FromDays(1)));
await _harness.Deliver(quiet, "/human-centipede", new JsonObject
{
["id"] = NewId(quiet, "accepts"), ["type"] = "Accept", ["actor"] = quiet.Id, ["object"] = following.FollowActivityURI + "-again-2"
});
Assert.Equal(FollowState.Accepted, (await DB.Default.Find<Following>().OneAsync(following.ID, token)).State);
Assert.Equal(0, await After(TimeSpan.FromDays(30)));
}
// Pixelfed names our post by its page (/@name/<id>, the post's url) in its Like, Announce and their Undo: the post is
// found the same as by its id
// Friendica's activity ids are uniqid(), a prefix and the microsecond: two of its processes answering two follows at
// once gave both Accepts one id, and the second was dropped as a copy of the first, leaving that follow pending
[Fact]
public async Task Two_activities_a_server_gave_one_id_are_both_taken_and_a_true_copy_once()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var dario = new RemoteActor(_harness.Peer, "dario");
var jun = new RemoteActor(_harness.Peer, "jun");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = dario.Id }, token);
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = jun.Id }, token);
Task<Following> Of(RemoteActor target) =>
DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id && f.TargetActorURI == target.Id).ExecuteSingleAsync(token);
async Task<JsonObject> Accept(RemoteActor by, string id) => new()
{
["id"] = id,
["type"] = "Accept",
["actor"] = by.Id,
["object"] = new JsonObject { ["id"] = (await Of(by)).FollowActivityURI, ["type"] = "Follow", ["actor"] = alice.Uri, ["object"] = by.Id }
};
var shared = NewId(dario, "activity");
var first = await _harness.Deliver(dario, "/human-centipede", await Accept(dario, shared));
var second = await _harness.Deliver(jun, "/human-centipede", await Accept(jun, shared));
var copy = await _harness.Deliver(jun, "/human-centipede", await Accept(jun, shared));
Assert.Equal(("queued", "queued", "duplicate"), (first.Reason, second.Reason, copy.Reason));
Assert.Equal(FollowState.Accepted, (await Of(dario)).State);
Assert.Equal(FollowState.Accepted, (await Of(jun)).State);
}
// Mobilizon: the organiser creates, edits and deletes an event attributed to the group, which announces it. They used
// to be refused as misattributed (400), so the edit was lost and the deletion left the event in place
[Fact]
public async Task An_event_its_organiser_edits_and_deletes_for_a_group_on_the_same_server_follows_that_server()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var group = new RemoteActor(_harness.Peer, "group", type: "Group");
var organiser = new RemoteActor(_harness.Peer, "organiser");
await Follows(alice.Id, group);
var path = $"/events/{Guid.NewGuid():N}";
JsonObject Event(string name) => new()
{
["id"] = _harness.Peer.A + path, ["type"] = "Event", ["name"] = name, ["content"] = "<p>bring bread</p>",
["attributedTo"] = group.Id, ["actor"] = organiser.Id, ["startTime"] = "2026-10-08T18:00:00Z",
["to"] = new JsonArray(PrivaPub.Federation.Objects.Addressing.Public), ["cc"] = new JsonArray(group.Id + "/followers")
};
_harness.Peer.Serve(path, Event("A picnic").ToJsonString());
Task<Post> Stored() => DB.Default.Find<Post>().Match(p => p.ObjectURI == _harness.Peer.A + path).ExecuteFirstAsync(token);
var created = await _harness.Deliver(organiser, "/human-centipede", Create(organiser, Event("A picnic")));
Assert.Equal(202, created.StatusCode);
Assert.Equal(group.Id, (await Stored()).ActorURI);
var moved = Event("A picnic, moved indoors");
moved["updated"] = DateTime.UtcNow.AddMinutes(1).ToString("O");
_harness.Peer.Serve(path, moved.ToJsonString());
await _harness.Deliver(organiser, "/human-centipede", Activity(organiser, "Update", Event("what the activity claims")));
Assert.Equal("A picnic, moved indoors", (await Stored()).Title);
await _harness.Deliver(organiser, "/human-centipede", Activity(organiser, "Delete", JsonValue.Create(_harness.Peer.A + path)!));
Assert.NotNull(await Stored());
_harness.Peer.Answer(path, 410);
await _harness.Deliver(organiser, "/human-centipede", Activity(organiser, "Delete", JsonValue.Create(_harness.Peer.A + path)!));
Assert.Null(await Stored());
// attributed to someone of another server, it is still refused
var elsewhere = new RemoteActor(_harness.Peer, "elsewhere", _harness.Peer.B);
var claimed = Event("Someone else's");
claimed["id"] = _harness.Peer.A + $"/events/{Guid.NewGuid():N}";
claimed["attributedTo"] = elsewhere.Id;
Assert.Equal(400, (await _harness.Deliver(organiser, "/human-centipede", Create(organiser, claimed))).StatusCode);
}
// Funkwhale names its Accept after the Follow it answers (`<our Follow's id>/accept`), on our origin, not its own
[Fact]
public async Task An_answer_named_after_our_follow_is_taken_and_one_naming_someone_elses_is_not()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var channel = new RemoteActor(_harness.Peer, "channel");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = channel.Id }, token);
var following = await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id && f.TargetActorURI == channel.Id).ExecuteSingleAsync(token);
var elsewhere = await _harness.Deliver(channel, "/human-centipede", new JsonObject
{
["id"] = "https://privapub.test/peasants/somebody/accept", ["type"] = "Accept", ["actor"] = channel.Id, ["object"] = following.FollowActivityURI
});
Assert.Equal(400, elsewhere.StatusCode);
var named = await _harness.Deliver(channel, "/human-centipede", new JsonObject
{
["id"] = following.FollowActivityURI + "/accept", ["type"] = "Accept", ["actor"] = channel.Id,
["object"] = new JsonObject { ["id"] = following.FollowActivityURI, ["type"] = "Follow", ["actor"] = alice.Uri, ["object"] = channel.Id }
});
Assert.Equal(202, named.StatusCode);
Assert.Equal(FollowState.Accepted, (await DB.Default.Find<Following>().OneAsync(following.ID, token)).State);
}
// Funkwhale's channel deletes its uploads in one Delete, their ids in a list as the object's id (and the Delete has
// no id of its own)
[Fact]
public async Task A_delete_naming_several_objects_removes_each()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var channel = new RemoteActor(_harness.Peer, "channel");
await Follows(alice.Id, channel);
var first = PublicNote(channel, "<p>a first track</p>");
var second = PublicNote(channel, "<p>a second track</p>");
await _harness.Deliver(channel, "/human-centipede", Create(channel, first));
await _harness.Deliver(channel, "/human-centipede", Create(channel, second));
Task<bool> Held(JsonObject note) => DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(note)).ExecuteAnyAsync(token);
Assert.True(await Held(first) && await Held(second));
var result = await _harness.Receiver.Receive(channel.Post(Harness.Host, "/human-centipede", new JsonObject
{
["type"] = "Delete", ["actor"] = channel.Id,
["object"] = new JsonObject { ["id"] = new JsonArray(IdOf(first), IdOf(second)), ["type"] = "Audio" }
}), default, token);
Assert.Equal(202, result.StatusCode);
var listed = IdOf(first);
var job = await DB.Default.Find<PrivaPub.Models.Jobs.Job>().Match(j => j.Kind == PrivaPub.Models.Jobs.JobKind.ProcessInbox
&& j.State == PrivaPub.Models.Jobs.JobState.Pending && j.Payload.Contains(listed)).Sort(j => j.CreatedAt, MongoDB.Entities.Order.Descending)
.ExecuteFirstAsync(token);
await _harness.Processor.Handle(job, token);
Assert.False(await Held(first));
Assert.False(await Held(second));
}
[Fact]
public async Task A_like_naming_a_post_by_its_page_counts_and_its_undo_too()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var fan = new RemoteActor(_harness.Peer, "fan");
var post = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "a picture" }, token)).Post;
var page = alice.PostHtmlUrl(post.ID);
Assert.NotEqual(post.ObjectURI, page);
var like = new JsonObject { ["id"] = NewId(fan, "likes"), ["type"] = "Like", ["actor"] = fan.Id, ["object"] = page };
await _harness.Deliver(fan, "/human-centipede", like);
Assert.Equal("accepted", Processed("Like").Outcome);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
await _harness.Deliver(fan, "/human-centipede", Activity(fan, "Undo", like));
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
// a page that is no post of ours stays as it is
var elsewhere = new JsonObject { ["id"] = NewId(fan, "likes"), ["type"] = "Like", ["actor"] = fan.Id, ["object"] = alice.PostHtmlUrl("000000000000000000000000") };
await _harness.Deliver(fan, "/human-centipede", elsewhere);
Assert.Equal(("dropped", "unknown-object"), (Processed("Like").Outcome, Processed("Like").Reason));
}
[Fact]
public async Task An_actor_update_refreshes_the_account_from_its_origin_even_when_its_updated_is_older()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
await _harness.Deliver(bob, "/human-centipede", Follow(bob, alice.Uri));
var before = await Stored(bob);
using var rotated = RSA.Create(2048);
var document = bob.Document();
document["name"] = "Bob Renamed";
document["summary"] = "<p>a new bio<script>alert(1)</script></p>";
document["publicKey"]!["publicKeyPem"] = rotated.ExportSubjectPublicKeyInfoPem();
_harness.Peer.Serve(new Uri(bob.Id).AbsolutePath, document.ToJsonString());
var embedded = (JsonObject)document.DeepClone();
embedded["name"] = "What the activity claims";
embedded["updated"] = "2001-01-01T00:00:00Z";
var result = await _harness.Deliver(bob, "/human-centipede", Activity(bob, "Update", embedded));
var after = await Stored(bob);
Assert.Equal(202, result.StatusCode);
Assert.Null(before.Name);
Assert.Equal(before.ID, after.ID);
Assert.Equal("Bob Renamed", after.Name);
Assert.Equal("<p>a new bio</p>", after.Biography);
Assert.Equal(bob.KeyId, after.PublicKeyId);
Assert.NotEqual(before.PublicKey, after.PublicKey);
Assert.Equal(rotated.ExportSubjectPublicKeyInfoPem(), after.PublicKey);
Assert.Equal(("accepted", "actor-refresh"), (Processed("Update").Outcome, Processed("Update").Reason));
}
[Fact]
public async Task A_key_moved_to_a_new_id_replaces_the_old_one()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
await _harness.Deliver(bob, "/human-centipede", Follow(bob, alice.Uri));
using var rotated = RSA.Create(2048);
var document = bob.Document();
document["publicKey"] = new JsonObject { ["id"] = bob.Id + "#key-2", ["owner"] = bob.Id, ["publicKeyPem"] = rotated.ExportSubjectPublicKeyInfoPem() };
_harness.Peer.Serve(new Uri(bob.Id).AbsolutePath, document.ToJsonString());
await _harness.Deliver(bob, "/human-centipede", Activity(bob, "Update", JsonValue.Create(bob.Id)!));
var after = await Stored(bob);
Assert.Equal(bob.Id + "#key-2", after.PublicKeyId);
Assert.Equal(rotated.ExportSubjectPublicKeyInfoPem(), after.PublicKey);
}
[Fact]
public async Task Undoing_a_follow_removes_the_follower_by_the_activity_id_or_by_its_object()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var carol = new RemoteActor(_harness.Peer, "carol");
var bobFollows = Follow(bob, alice.Uri);
await _harness.Deliver(bob, "/human-centipede", bobFollows);
await _harness.Deliver(carol, "/human-centipede", Follow(carol, alice.Uri));
Assert.Equal(2, await DB.Default.CountAsync<Follower>(f => f.LocalActorId == alice.Id, token));
await _harness.Deliver(bob, "/human-centipede", Activity(bob, "Undo", JsonValue.Create(IdOf(bobFollows))!));
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == bob.Id).ExecuteAnyAsync(token));
Assert.Equal(("accepted", "undone"), (Processed("Undo").Outcome, Processed("Undo").Reason));
var forgotten = Follow(carol, alice.Uri);
forgotten["id"] = NewId(carol, "follows");
await _harness.Deliver(carol, "/human-centipede", Activity(carol, "Undo", forgotten));
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id).ExecuteAnyAsync(token));
await _harness.Deliver(carol, "/human-centipede", Activity(carol, "Undo", forgotten));
Assert.Equal(("dropped", "unknown-object"), (Processed("Undo").Outcome, Processed("Undo").Reason));
}
[Fact]
public async Task A_rejected_quote_request_marks_the_quote_rejected_and_only_the_quoted_author_can_reject_it()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var note = PublicNote(ann, "<p>ask me first</p>", alice.Uri);
note["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" });
note["interactionPolicy"] = new JsonObject { ["canQuote"] = new JsonObject { ["manualApproval"] = new JsonArray(Addressing.Public) } };
await _harness.Deliver(ann, "/human-centipede", Create(ann, note));
var original = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(note)).ExecuteSingleAsync(token);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "may I?", QuotedStatusId = original.ID }, token);
Assert.Equal(QuoteState.Pending, outcome.Post.QuoteState);
var request = Assert.Single(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "QuoteRequest");
await _harness.Deliver(mallory, "/human-centipede", Activity(mallory, "Reject", JsonValue.Create(IdOf(request))!));
Assert.Equal(QuoteState.Pending, (await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token)).QuoteState);
await _harness.Deliver(ann, "/human-centipede", Activity(ann, "Reject", request));
var quoting = await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token);
Assert.Equal(QuoteState.Rejected, quoting.QuoteState);
Assert.Null(quoting.QuoteAuthorizationURI);
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
Assert.Equal(("accepted", "quote-answer"), (Processed("Reject").Outcome, Processed("Reject").Reason));
}
[Fact]
public async Task A_vote_a_followed_community_relays_counts_and_its_undo_takes_it_back()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "cats", type: "Group");
var stranger = new RemoteActor(_harness.Peer, "dogs", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
var voter = new RemoteActor(_harness.Peer, "voter");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = community.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
var page = PublicNote(poster, "<p>a post</p>", community.Id);
page["type"] = "Page";
page["name"] = "a title";
page["audience"] = community.Id;
_harness.Peer.Serve(new Uri(IdOf(page)).AbsolutePath, page.ToJsonString());
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Create(poster, page)));
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(page)).ExecuteSingleAsync(token);
var like = new JsonObject { ["id"] = NewId(voter, "likes"), ["type"] = "Like", ["actor"] = voter.Id, ["object"] = post.ObjectURI, ["audience"] = community.Id };
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", like));
var liked = Processed("Announce");
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Activity(voter, "Undo", like)));
var unliked = Processed("Announce");
await _harness.Deliver(stranger, "/human-centipede", Activity(stranger, "Announce", like));
var unfollowed = Processed("Announce");
//Lemmy relays its members' votes inside the community's Announce (G-0003): a voter on the community's own server
//is vouched for by it, and the vote is handled as if the voter had sent it; a community nobody follows is ignored
Assert.Equal("accepted", liked.Outcome);
Assert.Equal("accepted", unliked.Outcome);
Assert.Equal(("dropped", "not-followed"), (unfollowed.Outcome, unfollowed.Reason));
var after = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Equal(0, after.FavouritesCount);
Assert.False(await DB.Default.Find<Favourite>().Match(f => f.PostId == post.ID).ExecuteAnyAsync(token));
}
// Lemmy sends a vote to the community alone: a vote on our reply in a thread of a community nobody here follows still
// reaches us through it, and only through that community (found by the town's village)
[Fact]
public async Task A_vote_on_our_reply_in_an_unfollowed_communitys_thread_counts_through_that_community_only()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "cats", type: "Group");
var other = new RemoteActor(_harness.Peer, "dogs", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
var voter = new RemoteActor(_harness.Peer, "voter");
var page = PublicNote(poster, "<p>a thread</p>", community.Id);
page["type"] = "Page";
page["name"] = "a thread";
page["audience"] = community.Id;
_harness.Peer.Serve(new Uri(IdOf(page)).AbsolutePath, page.ToJsonString());
var root = await _harness.RemotePosts.StoreContext(IdOf(page), 0, token);
Assert.Equal(community.Id, root.AudienceURI);
var reply = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "my answer", InReplyTo = root.ID }, token)).Post;
var like = new JsonObject { ["id"] = NewId(voter, "likes"), ["type"] = "Like", ["actor"] = voter.Id, ["object"] = reply.ObjectURI };
await _harness.Deliver(other, "/human-centipede", Activity(other, "Announce", like));
Assert.Equal(("dropped", "not-followed"), (Processed("Announce").Outcome, Processed("Announce").Reason));
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", like));
Assert.Equal("accepted", Processed("Announce").Outcome);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(reply.ID, token)).FavouritesCount);
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Activity(voter, "Undo", like)));
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(reply.ID, token)).FavouritesCount);
}
// Lemmy's moderators lock a community's post and ban members, and the community relays both inside its Announce
// (G-0006): a locked post takes no reply, a banned persona posts nothing there, until the Undo
[Fact]
public async Task A_community_locks_a_post_and_bans_a_persona_until_it_undoes_either()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "cats", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
var moderator = new RemoteActor(_harness.Peer, "moderator");
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = community.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
JsonObject Page(string text, bool commentsEnabled = true)
{
var page = PublicNote(poster, $"<p>{text}</p>", community.Id);
page["type"] = "Page";
page["name"] = text;
page["audience"] = community.Id;
page["commentsEnabled"] = commentsEnabled;
_harness.Peer.Serve(new Uri(IdOf(page)).AbsolutePath, page.ToJsonString());
return page;
}
async Task<Post> Announced(JsonObject page)
{
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Create(poster, page)));
return await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(page)).ExecuteSingleAsync(token);
}
Task<StatusOutcome> Reply(Post post) => _harness.Statuses.Publish(alice, new StatusDraft { Text = "a reply", InReplyTo = post.ID }, token);
var post = await Announced(Page("a thread"));
var lockIt = new JsonObject { ["id"] = NewId(moderator, "locks"), ["type"] = "Lock", ["actor"] = moderator.Id, ["object"] = post.ObjectURI };
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", lockIt));
Assert.Equal(("accepted", "locked"), (Processed("Announce").Outcome, Processed("Announce").Reason));
var refused = await Reply(post);
Assert.Equal((422, "Validation failed: This thread is locked"), (refused.Status, refused.Error));
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Activity(moderator, "Undo", lockIt)));
Assert.Null((await DB.Default.Find<Post>().OneAsync(post.ID, token)).LockedAt);
Assert.True((await Reply(post)).Ok);
// a post its community serves locked arrives locked
Assert.NotNull((await Announced(Page("born locked", commentsEnabled: false))).LockedAt);
var ban = new JsonObject { ["id"] = NewId(moderator, "bans"), ["type"] = "Block", ["actor"] = moderator.Id, ["object"] = alice.Uri, ["target"] = community.Id };
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", ban));
Assert.Equal(("accepted", "banned"), (Processed("Announce").Outcome, Processed("Announce").Reason));
var banned = await Reply(post);
Assert.Equal((422, "Validation failed: This community banned you"), (banned.Status, banned.Error));
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Activity(moderator, "Undo", ban)));
Assert.Equal(("accepted", "unbanned"), (Processed("Announce").Outcome, Processed("Announce").Reason));
Assert.True((await Reply(post)).Ok);
// Lemmy also sends the ban straight to the persona's server, as its moderator's Block: still the community's ban,
// never the moderator's own block of the persona
var direct = new JsonObject { ["id"] = NewId(moderator, "bans"), ["type"] = "Block", ["actor"] = moderator.Id, ["object"] = alice.Uri, ["target"] = community.Id };
await _harness.Deliver(moderator, "/human-centipede", direct);
Assert.Equal(("accepted", "banned"), (Processed("Block").Outcome, Processed("Block").Reason));
Assert.False(await DB.Default.Find<BlockedBy>().Match(b => b.AvatarId == alice.Id && b.ActorURI == moderator.Id).ExecuteAnyAsync(token));
Assert.Equal(422, (await Reply(post)).Status);
await _harness.Deliver(moderator, "/human-centipede", Activity(moderator, "Undo", direct));
Assert.True((await Reply(post)).Ok);
var elsewhere = new JsonObject { ["id"] = NewId(moderator, "bans"), ["type"] = "Block", ["actor"] = moderator.Id, ["object"] = poster.Id, ["target"] = community.Id };
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", elsewhere));
Assert.Equal(("dropped", "not-ours"), (Processed("Announce").Outcome, Processed("Announce").Reason));
}
// a voter on another server than the community is believed for the community's own posts, as Lemmy trusts it; for
// anything else only once its vote is fetched from its own origin
[Fact]
public async Task A_relayed_vote_from_another_server_counts_on_the_communitys_posts_and_elsewhere_only_once_fetched()
{
var token = TestContext.Current.CancellationToken;
var (root, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "cats", type: "Group");
var poster = new RemoteActor(_harness.Peer, "poster");
var voter = new RemoteActor(_harness.Peer, "voter", origin: _harness.Peer.B);
var forger = new RemoteActor(_harness.Peer, "forger", origin: _harness.Peer.B);
await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = community.Id }, token);
await DB.Default.Update<Following>().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token);
var page = PublicNote(poster, "<p>a post</p>", community.Id);
page["type"] = "Page";
page["name"] = "a title";
_harness.Peer.Serve(new Uri(IdOf(page)).AbsolutePath, page.ToJsonString());
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", Create(poster, page)));
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(page)).ExecuteSingleAsync(token);
var other = await OwnPost(alice, token);
var like = new JsonObject { ["id"] = NewId(voter, "likes"), ["type"] = "Like", ["actor"] = voter.Id, ["object"] = post.ObjectURI };
var forged = new JsonObject { ["id"] = NewId(forger, "likes"), ["type"] = "Like", ["actor"] = forger.Id, ["object"] = other.ObjectURI };
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", like));
await _harness.Deliver(community, "/human-centipede", Activity(community, "Announce", forged));
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
Assert.True(await DB.Default.Find<Favourite>().Match(f => f.PostId == post.ID && f.ActorURI == voter.Id).ExecuteAnyAsync(token));
Assert.Equal(("dropped", "fetch-failed"), (Processed("Announce").Outcome, Processed("Announce").Reason));
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(other.ID, token)).FavouritesCount);
}
// a public post of a persona's own, outside any community
static async Task<Post> OwnPost(PrivaPub.Federation.Actors.LocalActor author, CancellationToken token)
{
var post = new Post { GroupUserId = author.Id, AuthorAccountId = author.Id, Text = "not the community's" };
post.ID = (string)post.GenerateNewID();
post.ObjectURI = author.PostUri(post.ID);
await DB.Default.SaveAsync(post, token);
return post;
}
[Fact]
public async Task A_locked_persona_holds_a_follow_until_it_decides_and_answers_with_the_original_follow()
{
var token = TestContext.Current.CancellationToken;
var (_, open) = await _harness.Persona("alice");
await DB.Default.Update<Avatar>().MatchID(open.Id).Modify(a => a.Settings.IsLocked, true).ExecuteAsync(token);
var alice = await _harness.Local.FindById(LocalActorKind.Person, open.Id, token);
var bob = new RemoteActor(_harness.Peer, "bob");
var carol = new RemoteActor(_harness.Peer, "carol");
var bobFollows = Follow(bob, alice.Uri);
var carolFollows = Follow(carol, alice.Uri);
Assert.True(alice.ManuallyApprovesFollowers);
await _harness.Deliver(bob, "/human-centipede", bobFollows);
await _harness.Deliver(carol, "/human-centipede", carolFollows);
var pending = await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id).ExecuteAsync(token);
Assert.Equal(2, pending.Count);
Assert.All(pending, f => Assert.False(f.IsAccepted));
Assert.Equal(("accepted", "pending"), (Processed("Follow").Outcome, Processed("Follow").Reason));
var requests = await DB.Default.Find<Notification>().Match(n => n.AvatarId == alice.Id).ExecuteAsync(token);
Assert.Equal(2, requests.Count);
Assert.All(requests, n => Assert.Equal(NotificationType.FollowRequest, n.Type));
Assert.Empty(await _harness.Outgoing(bob.Id + "/inbox"));
Assert.Empty(await _harness.Delivery.FollowerInboxes(alice, token));
var bobAccount = await Stored(bob);
var carolAccount = await Stored(carol);
Assert.True(await _harness.Follows.Decide(alice, bobAccount.ID, accept: true, token));
Assert.True(await _harness.Follows.Decide(alice, carolAccount.ID, accept: false, token));
Assert.False(await _harness.Follows.Decide(alice, carolAccount.ID, accept: true, token));
var accept = Assert.Single(await _harness.Outgoing(bob.Id + "/inbox"));
Assert.Equal("Accept", accept["type"]!.GetValue<string>());
Assert.Equal(alice.Uri, accept["actor"]!.GetValue<string>());
Assert.Equal(IdOf(bobFollows), IdOf(accept["object"]!));
Assert.Equal(bob.Id, accept["object"]!["actor"]!.GetValue<string>());
var reject = Assert.Single(await _harness.Outgoing(carol.Id + "/inbox"));
Assert.Equal("Reject", reject["type"]!.GetValue<string>());
Assert.Equal(IdOf(carolFollows), IdOf(reject["object"]!));
Assert.True((await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == bob.Id).ExecuteSingleAsync(token)).IsAccepted);
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == alice.Id && f.ActorURI == carol.Id).ExecuteAnyAsync(token));
Assert.Contains(await DB.Default.Find<Notification>().Match(n => n.AvatarId == alice.Id && n.Type == NotificationType.Follow).ExecuteAsync(token),
n => n.FromAccountId == bobAccount.ID);
Assert.Equal(new[] { bob.Id + "/inbox" }, await _harness.Delivery.FollowerInboxes(alice, token));
}
}
}
@@ -171,12 +171,19 @@ namespace PrivaPub.Tests.Federation
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var mallory = new RemoteActor(_peer, "mallory");
var victim = new RemoteActor(_peer, "victim");
var victim = new RemoteActor(_peer, "victim", _peer.B);
var result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, attributedTo: victim.Id));
Assert.Equal(400, result.StatusCode);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == victim.Id).ExecuteAnyAsync(token));
// a colleague on mallory's own server is that server's to vouch for: believed only as the server has it, and
// it has no such note
var colleague = new RemoteActor(_peer, "colleague");
result = await Deliver(mallory, $"/peasants/{alice.UserName}/mouth", DirectCreate(mallory, alice.Uri, attributedTo: colleague.Id));
Assert.Equal(202, result.StatusCode);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ActorURI == colleague.Id).ExecuteAnyAsync(token));
}
[Fact]
@@ -263,6 +270,28 @@ namespace PrivaPub.Tests.Federation
Assert.Equal(alice.Id, mention.AccountId);
}
// GoToSocial kept a persona in cc after an edit took the @name out, and Akkoma addresses with to[] alone: the persona it
// is addressed to sees a followers-only post as a silent mention, which no list shows as a mention (found by the town)
[Fact]
public async Task A_followers_only_post_addressed_to_a_persona_without_naming_it_is_a_silent_mention()
{
var token = TestContext.Current.CancellationToken;
var alice = await LocalAvatar("alice");
var bob = new RemoteActor(_peer, "bob");
var create = PublicCreate(bob, default, alice.Uri);
create["object"]!["to"] = bob.Id + "/followers";
create["object"]!["cc"] = alice.Uri;
await Deliver(bob, "/human-centipede", create);
var post = await DB.Default.Find<Post>().Match(p => p.ActorURI == bob.Id).ExecuteSingleAsync(token);
Assert.Equal(PostVisibility.FollowersOnly, post.Visibility);
var mention = Assert.Single(post.Mentions);
Assert.True(mention.Silent);
Assert.Equal(alice.Id, mention.AccountId);
Assert.True(await PrivaPub.Domain.Privacy.VisibilityPolicy.CanSee(post, alice.Id, token));
}
[Fact]
public async Task A_suspended_domain_is_dropped_before_its_key_is_fetched()
{
@@ -0,0 +1,175 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Federation.Signing;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// FEP-8b32 proofs (eddsa-jcs-2022) by a persona's Ed25519 key, which its actor names as a FEP-521a Multikey
public sealed class IntegrityProofTests
{
[Fact]
public void Canonical_json_is_rfc_8785s()
{
// RFC 8785, section 3.2.2
var input = JsonNode.Parse("{\"numbers\":[333333333.33333329,1E30,4.50,2e-3,0.000000000000000000000000001],"
+ "\"string\":\"\\u20ac$\\u000F\\u000aA'\\u0042\\u0022\\u005c\\\\\\\"\\/\",\"literals\":[null,true,false]}");
Assert.Equal("{\"literals\":[null,true,false],\"numbers\":[333333333.3333333,1e+30,4.5,0.002,1e-27],\"string\":\"€$\\u000f\\nA'B\\\"\\\\\\\\\\\"/\"}",
Jcs.Serialize(input));
}
[Fact]
public void Base58_and_multikeys_read_back_what_they_write()
{
Assert.Equal("2NEpo7TZRRrLZSi2U", IntegrityProofs.Base58(Encoding.ASCII.GetBytes("Hello World!")));
Assert.Equal("112", IntegrityProofs.Base58(new byte[] { 0, 0, 1 }));
Assert.Equal(new byte[] { 0, 0, 1 }, IntegrityProofs.FromBase58("112"));
var publicKey = IntegrityProofs.PublicKey(IntegrityProofs.NewSeed());
var multikey = IntegrityProofs.Multikey(publicKey);
Assert.StartsWith("z6Mk", multikey);
Assert.Equal(publicKey, IntegrityProofs.FromMultikey(multikey));
}
[Fact]
public void A_proof_verifies_with_its_key_and_with_nothing_changed()
{
var seed = IntegrityProofs.NewSeed();
var activity = new JsonObject
{
["@context"] = ActivityPubRenderer.Context(), ["id"] = "https://privapub.test/peasants/alice/grunts/1", ["type"] = "Create",
["actor"] = "https://privapub.test/peasants/alice", ["object"] = new JsonObject { ["type"] = "Note", ["content"] = "<p>ciao, è così</p>", ["width"] = 4.5 }
};
activity["proof"] = IntegrityProofs.Create(activity, "https://privapub.test/peasants/alice#ed25519-key", seed, DateTime.UtcNow);
var delivered = JsonNode.Parse(activity.ToJsonString())!.AsObject();
Assert.True(IntegrityProofs.Verify(delivered, IntegrityProofs.PublicKey(seed)));
Assert.Equal("assertionMethod", delivered["proof"]!["proofPurpose"]!.GetValue<string>());
var changed = delivered.DeepClone().AsObject();
changed["object"]!["content"] = "<p>something else</p>";
Assert.False(IntegrityProofs.Verify(changed, IntegrityProofs.PublicKey(seed)));
Assert.False(IntegrityProofs.Verify(delivered, IntegrityProofs.PublicKey(IntegrityProofs.NewSeed())));
}
[Fact]
public void A_persona_with_a_key_names_it_as_a_multikey()
{
var seed = IntegrityProofs.NewSeed();
var actor = ActivityPubRenderer.Actor(new LocalActor { Id = "a", UserName = "alice", BaseAddress = Harness.Base, Kind = LocalActorKind.Person, SigningKey = seed });
var keyless = ActivityPubRenderer.Actor(new LocalActor { Id = "b", UserName = "bob", BaseAddress = Harness.Base, Kind = LocalActorKind.Person });
var key = Assert.Single(actor["assertionMethod"]!.AsArray())!;
Assert.Equal(($"{Harness.Base}/peasants/alice#ed25519-key", "Multikey", $"{Harness.Base}/peasants/alice"),
(key["id"]!.GetValue<string>(), key["type"]!.GetValue<string>(), key["controller"]!.GetValue<string>()));
Assert.Equal(IntegrityProofs.PublicKey(seed), IntegrityProofs.FromMultikey(key["publicKeyMultibase"]!.GetValue<string>()));
Assert.Null(keyless["assertionMethod"]);
}
}
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class DeliveredProofTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
// what goes to a relay carries the persona's proof, to be forwarded on its strength; what goes to a server directly
// does not (Mitra refuses a proof by a key it has not read yet); another's activity passed on never gets ours
[Fact]
public async Task A_personas_activity_goes_to_a_relay_with_its_proof_and_elsewhere_without()
{
var token = TestContext.Current.CancellationToken;
var (_, persona) = await _harness.Persona("alice");
var seed = IntegrityProofs.NewSeed();
await DB.Default.Update<Avatar>().MatchID(persona.Id).Modify(a => a.SigningKey, seed).ExecuteAsync(token);
var alice = _harness.Local.FromAvatar(await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteSingleAsync(token));
var relayInbox = $"{_harness.Peer.A}/relay-{Guid.NewGuid():N}/inbox";
var serverInbox = _harness.Peer.A + "/proofs/inbox";
await DB.Default.SaveAsync(new RelaySubscription
{
Configured = relayInbox, ActorURI = relayInbox.Replace("/inbox", "/actor"), InboxURL = relayInbox, State = RelayState.Accepted
}, token);
var own = new JsonObject { ["id"] = alice.ActivityUri(Guid.NewGuid().ToString("N")), ["type"] = "Create", ["actor"] = alice.Uri, ["object"] = new JsonObject { ["type"] = "Note" } };
var passedOn = new JsonObject { ["id"] = $"https://elsewhere.example/activities/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = "https://elsewhere.example/users/bob" };
try
{
await _harness.Delivery.Enqueue(alice, new[] { relayInbox, serverInbox }, own, token);
await _harness.Delivery.Enqueue(alice, new[] { relayInbox }, passedOn, token);
var toRelay = await _harness.Outgoing(relayInbox);
var signed = Assert.Single(toRelay, a => a["actor"]!.GetValue<string>() == alice.Uri);
Assert.Equal(alice.AssertionKeyId, signed["proof"]!["verificationMethod"]!.GetValue<string>());
Assert.True(IntegrityProofs.Verify(signed, IntegrityProofs.PublicKey(seed)));
Assert.Null(Assert.Single(toRelay, a => a["actor"]!.GetValue<string>() != alice.Uri)["proof"]);
Assert.Null(Assert.Single(await _harness.Outgoing(serverInbox))["proof"]);
Assert.Null(own["proof"]);
}
finally
{
await DB.Default.DeleteAsync<RelaySubscription>(s => s.InboxURL == relayInbox);
}
}
// a post forwarded by another server, which its origin does not serve: taken on its author's proof, never without
[Fact]
public async Task A_forwarded_post_with_its_authors_proof_is_taken_as_it_came()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var author = new RemoteActor(_harness.Peer, "author", ed25519: true);
var forwarder = new RemoteActor(_harness.Peer, "forwarder", _harness.Peer.B);
await DB.Default.SaveAsync(new PrivaPub.Models.Social.Following
{
AvatarId = alice.Id, TargetActorURI = author.Id, TargetInboxURL = author.Id + "/inbox", State = PrivaPub.Models.Social.FollowState.Accepted
}, token);
JsonObject Create(string text)
{
var noteId = $"{new Uri(author.Id).GetLeftPart(UriPartial.Authority)}/notes/{Guid.NewGuid():N}";
return new JsonObject
{
["@context"] = "https://www.w3.org/ns/activitystreams", ["id"] = noteId + "/activity", ["type"] = "Create", ["actor"] = author.Id,
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"),
["object"] = new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"<p>{text}</p>",
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"), ["published"] = DateTime.UtcNow.ToString("O")
}
};
}
var proven = author.Prove(Create("proven"));
var tampered = author.Prove(Create("as written"));
tampered["object"]!["content"] = "<p>changed on the way</p>";
var bare = Create("unproven");
await _harness.Deliver(forwarder, "/human-centipede", proven);
await _harness.Deliver(forwarder, "/human-centipede", tampered);
await _harness.Deliver(forwarder, "/human-centipede", bare);
Task<bool> Held(JsonObject create) => DB.Default.Find<PrivaPub.Models.Post.Post>().Match(p => p.ObjectURI == create["object"]!["id"]!.GetValue<string>()).ExecuteAnyAsync(token);
Assert.True(await Held(proven));
Assert.False(await Held(tampered));
Assert.False(await Held(bare));
}
}
}
@@ -0,0 +1,284 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
// GoToSocial's interaction policies: who may reply to, like and boost a post at once, who must ask its author first, and
// who may not
[Trait("Category", "Integration")]
public sealed class InteractionPolicyTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static JsonObject Rule(params string[] automatic) => Rule(automatic, Array.Empty<string>());
static JsonObject Rule(string[] automatic, string[] manual) => new()
{
["automaticApproval"] = new JsonArray(automatic.Select(a => (JsonNode)a).ToArray()),
["manualApproval"] = new JsonArray(manual.Select(m => (JsonNode)m).ToArray())
};
// bob's public post, which alice follows bob to see, under the policy given
async Task<(PrivaPub.Federation.Actors.LocalActor Alice, RemoteActor Bob, Post Post, RemoteActor Fan)> Posted(JsonObject policy)
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var fan = new RemoteActor(_harness.Peer, "fan");
await Follows(alice.Id, bob);
await _harness.FollowedBy(alice, fan);
var note = PublicNote(bob, "<p>a post with rules</p>");
note["interactionPolicy"] = policy;
await _harness.Deliver(bob, "/human-centipede", Create(bob, note));
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(note)).ExecuteSingleAsync(token);
return (alice, bob, post, fan);
}
Task<List<JsonObject>> To(RemoteActor actor) => _harness.Outgoing(actor.Id + "/inbox");
Task<List<JsonObject>> ToShared(RemoteActor actor) => _harness.Outgoing(actor.SharedInbox);
JsonObject Authorization(RemoteActor author, string type, string interaction, Post target)
{
var path = $"/authorizations/{Guid.NewGuid():N}";
var stamp = new JsonObject
{
["id"] = Origin(author) + path, ["type"] = type, ["attributedTo"] = author.Id,
["interactingObject"] = interaction, ["interactionTarget"] = target.ObjectURI
};
_harness.Peer.Serve(path, stamp.ToJsonString());
return stamp;
}
[Fact]
public async Task A_reply_the_policy_lets_in_at_once_goes_out_as_always_and_one_it_shuts_out_is_refused()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, _) = await Posted(new JsonObject { ["canReply"] = Rule("https://www.w3.org/ns/activitystreams#Public") });
var reply = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "an open reply", InReplyTo = post.ID }, token);
Assert.Equal(ApprovalState.None, reply.Post.Approval);
Assert.Contains(await To(bob), a => a["type"]!.GetValue<string>() == "Create");
var (carol, dave, shut, _) = await Posted(new JsonObject { ["canReply"] = Rule(new string[0], new string[0]) });
var refused = await _harness.Statuses.Publish(carol, new StatusDraft { Text = "a shut-out reply", InReplyTo = shut.ID }, token);
Assert.Equal(422, refused.Status);
}
[Fact]
public async Task A_reply_its_author_must_approve_is_asked_for_and_goes_out_with_the_authorization_once_given()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, fan) = await Posted(new JsonObject
{
["canReply"] = Rule(new[] { "https://dummy.invalid/nobody" }, new[] { "https://www.w3.org/ns/activitystreams#Public" })
});
var reply = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "may I?", InReplyTo = post.ID }, token)).Post;
Assert.Equal(ApprovalState.Pending, reply.Approval);
var request = Assert.Single(await To(bob), a => a["type"]!.GetValue<string>() == "ReplyRequest");
Assert.Equal(post.ObjectURI, request["object"]!.GetValue<string>());
Assert.Equal(reply.ObjectURI, request["instrument"]!["id"]!.GetValue<string>());
Assert.DoesNotContain(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Create");
var stamp = Authorization(bob, "ReplyAuthorization", reply.ObjectURI, post);
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = NewId(bob, "accepts"), ["type"] = "Accept", ["actor"] = bob.Id,
["object"] = new JsonObject { ["type"] = "ReplyRequest", ["id"] = request["id"]!.GetValue<string>() },
["result"] = IdOf(stamp)
});
var after = await DB.Default.Find<Post>().OneAsync(reply.ID, token);
Assert.Equal((ApprovalState.Accepted, IdOf(stamp)), (after.Approval, after.ApprovalURI));
var create = Assert.Single(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Create");
Assert.Equal(IdOf(stamp), create["object"]!["replyAuthorization"]!.GetValue<string>());
}
[Fact]
public async Task A_rejected_reply_stays_ours_alone_and_a_forged_authorization_changes_nothing()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, fan) = await Posted(new JsonObject
{
["canReply"] = Rule(new[] { "https://dummy.invalid/nobody" }, new[] { "https://www.w3.org/ns/activitystreams#Public" })
});
var reply = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "may I?", InReplyTo = post.ID }, token)).Post;
var request = Assert.Single(await To(bob), a => a["type"]!.GetValue<string>() == "ReplyRequest");
// an authorization naming another post is no authorization
var forged = Authorization(bob, "ReplyAuthorization", reply.ObjectURI, new Post { ObjectURI = "https://elsewhere.invalid/post" });
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = NewId(bob, "accepts"), ["type"] = "Accept", ["actor"] = bob.Id, ["object"] = request["id"]!.GetValue<string>(), ["result"] = IdOf(forged)
});
Assert.Equal(ApprovalState.Pending, (await DB.Default.Find<Post>().OneAsync(reply.ID, token)).Approval);
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = NewId(bob, "rejects"), ["type"] = "Reject", ["actor"] = bob.Id, ["object"] = request["id"]!.GetValue<string>()
});
Assert.Equal(ApprovalState.Rejected, (await DB.Default.Find<Post>().OneAsync(reply.ID, token)).Approval);
Assert.DoesNotContain(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Create");
}
[Fact]
public async Task A_like_and_a_boost_its_author_must_approve_are_asked_for_and_a_shut_out_boost_is_refused()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, fan) = await Posted(new JsonObject
{
["canLike"] = Rule(new[] { "https://dummy.invalid/nobody" }, new[] { "https://www.w3.org/ns/activitystreams#Public" }),
["canAnnounce"] = Rule(new[] { "https://dummy.invalid/nobody" }, new[] { "https://www.w3.org/ns/activitystreams#Public" })
});
await _harness.Statuses.Favourite(alice, post.ID, true, token);
var like = await DB.Default.Find<Favourite>().Match(f => f.AccountId == alice.Id && f.PostId == post.ID).ExecuteSingleAsync(token);
Assert.Equal(ApprovalState.Pending, like.Approval);
var likeRequest = Assert.Single(await To(bob), a => a["type"]!.GetValue<string>() == "LikeRequest");
Assert.DoesNotContain(await To(bob), a => a["type"]!.GetValue<string>() == "Like");
var stamp = Authorization(bob, "LikeAuthorization", like.ActivityURI, post);
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = NewId(bob, "accepts"), ["type"] = "Accept", ["actor"] = bob.Id, ["object"] = likeRequest["id"]!.GetValue<string>(), ["result"] = IdOf(stamp)
});
Assert.Equal(ApprovalState.Accepted, (await DB.Default.Find<Favourite>().OneAsync(like.ID, token)).Approval);
Assert.Equal(IdOf(stamp), Assert.Single(await To(bob), a => a["type"]!.GetValue<string>() == "Like")["likeAuthorization"]!.GetValue<string>());
var boost = (await _harness.Statuses.Reblog(alice, post.ID, true, PostVisibility.Public, token)).Post;
Assert.Equal(ApprovalState.Pending, boost.Approval);
var announceRequest = Assert.Single(await _harness.Outgoing(bob.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "AnnounceRequest");
Assert.DoesNotContain(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Announce");
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = NewId(bob, "rejects"), ["type"] = "Reject", ["actor"] = bob.Id, ["object"] = announceRequest["id"]!.GetValue<string>()
});
Assert.Equal(ApprovalState.Rejected, (await DB.Default.Find<Post>().OneAsync(boost.ID, token)).Approval);
var (carol, _, shut, _) = await Posted(new JsonObject { ["canAnnounce"] = Rule(new string[0], new string[0]) });
Assert.Equal(422, (await _harness.Statuses.Reblog(carol, shut.ID, true, PostVisibility.Public, token)).Status);
}
// as a third party: a reply bob's policy does not let in at once is shown only with bob's authorization
[Fact]
public async Task A_reply_to_a_post_that_needs_its_authors_approval_is_kept_only_with_that_approval()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, _) = await Posted(new JsonObject
{
["canReply"] = Rule(new[] { "https://dummy.invalid/nobody" }, new[] { "https://www.w3.org/ns/activitystreams#Public" })
});
var carol = new RemoteActor(_harness.Peer, "carol");
await Follows(alice.Id, carol);
var unasked = PublicNote(carol, "<p>barging in</p>");
unasked["inReplyTo"] = post.ObjectURI;
await _harness.Deliver(carol, "/human-centipede", Create(carol, unasked));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(unasked)).ExecuteAnyAsync(token));
var asked = PublicNote(carol, "<p>with leave</p>");
asked["inReplyTo"] = post.ObjectURI;
asked["replyAuthorization"] = IdOf(Authorization(bob, "ReplyAuthorization", IdOf(asked), post));
await _harness.Deliver(carol, "/human-centipede", Create(carol, asked));
Assert.True(await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(asked)).ExecuteAnyAsync(token));
}
// bob's public post whose replies wait for his approval (FEP-5624's canReply, as PeerTube sets it on a video whose
// comments are moderated), which alice follows bob to see
async Task<(PrivaPub.Federation.Actors.LocalActor Alice, RemoteActor Bob, Post Post, RemoteActor Fan)> Moderated(JsonNode canReply)
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var fan = new RemoteActor(_harness.Peer, "fan");
await Follows(alice.Id, bob);
await _harness.FollowedBy(alice, fan);
var note = PublicNote(bob, "<p>a moderated post</p>");
note["canReply"] = canReply;
await _harness.Deliver(bob, "/human-centipede", Create(bob, note));
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(note)).ExecuteSingleAsync(token);
return (alice, bob, post, fan);
}
Task Answer(RemoteActor from, string type, Post reply, string inReplyTo) => _harness.Deliver(from, "/human-centipede", new JsonObject
{
["id"] = NewId(from, "approvals"), ["type"] = type, ["actor"] = from.Id, ["object"] = reply.ObjectURI, ["inReplyTo"] = inReplyTo
});
[Fact]
public async Task A_reply_its_author_approves_goes_to_that_author_alone_then_out_with_the_approval()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, fan) = await Moderated("https://www.w3.org/ns/activitystreams#Public");
var reply = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "a comment to approve", InReplyTo = post.ID }, token)).Post;
Assert.Equal(ApprovalState.Pending, reply.Approval);
var asked = Assert.Single(await To(bob), a => a["type"]!.GetValue<string>() == "Create");
Assert.Equal(reply.ObjectURI, asked["object"]!["id"]!.GetValue<string>());
Assert.DoesNotContain(await To(bob), a => a["type"]!.GetValue<string>() == "ReplyRequest");
Assert.DoesNotContain(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Create");
var approval = NewId(bob, "approve-reply");
await _harness.Deliver(bob, "/human-centipede", new JsonObject
{
["id"] = approval, ["type"] = "ApproveReply", ["actor"] = bob.Id, ["object"] = reply.ObjectURI, ["inReplyTo"] = post.ObjectURI
});
var after = await DB.Default.Find<Post>().OneAsync(reply.ID, token);
Assert.Equal((ApprovalState.Accepted, approval), (after.Approval, after.ReplyApprovalURI));
var create = Assert.Single(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Create");
Assert.Equal(approval, create["object"]!["replyApproval"]!.GetValue<string>());
Assert.Null(create["object"]!["replyAuthorization"]);
}
[Fact]
public async Task A_refused_reply_stays_ours_alone_and_an_approval_from_elsewhere_changes_nothing()
{
var token = TestContext.Current.CancellationToken;
var (alice, bob, post, fan) = await Moderated("https://www.w3.org/ns/activitystreams#Public");
var reply = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "a comment to refuse", InReplyTo = post.ID }, token)).Post;
// an approval by someone else than the post's author, or naming another post, is no approval
await Answer(new RemoteActor(_harness.Peer, "mallory"), "ApproveReply", reply, post.ObjectURI);
await Answer(bob, "ApproveReply", reply, "https://elsewhere.invalid/post");
Assert.Equal(ApprovalState.Pending, (await DB.Default.Find<Post>().OneAsync(reply.ID, token)).Approval);
await Answer(bob, "RejectReply", reply, post.ObjectURI);
Assert.Equal(ApprovalState.Rejected, (await DB.Default.Find<Post>().OneAsync(reply.ID, token)).Approval);
Assert.DoesNotContain(await ToShared(fan), a => a["type"]!.GetValue<string>() == "Create");
}
[Fact]
public async Task Only_those_its_canReply_names_or_it_mentions_may_reply()
{
var (alice, _, post, _) = await Moderated(new JsonArray());
var refused = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "nobody asked me", InReplyTo = post.ID },
TestContext.Current.CancellationToken);
Assert.Equal(422, refused.Status);
}
}
}
@@ -3,8 +3,10 @@ using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.ClientModels.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Group;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
@@ -74,6 +76,92 @@ namespace PrivaPub.Tests.Federation
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
}
// Our followers-only posts are addressed to the followers collection, never to each follower, so a follower is found
// among our followers, not in to/cc. The town found every follower's like, reaction and downvote dropped (G-0001).
[Fact]
public async Task A_follower_likes_reacts_to_and_downvotes_a_followers_only_post()
{
var token = TestContext.Current.CancellationToken;
var (_, post) = await LocalPost("followersonly");
var carol = new RemoteActor(_harness.Peer, "carol");
await _harness.FollowedBy(_harness.Local.FromAvatar(await DB.Default.Find<Avatar>().OneAsync(post.GroupUserId, token)), carol);
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/likes/{Guid.NewGuid():N}", ["type"] = "Like", ["actor"] = carol.Id, ["object"] = post.ObjectURI
});
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/reactions/{Guid.NewGuid():N}", ["type"] = "EmojiReact", ["actor"] = carol.Id, ["object"] = post.ObjectURI,
["content"] = "🔥"
});
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(carol)}/dislikes/{Guid.NewGuid():N}", ["type"] = "Dislike", ["actor"] = carol.Id, ["object"] = post.ObjectURI
});
// the downvote counts, and takes the place of carol's like (one vote each, as Lemmy votes)
var stored = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Equal(0, stored.FavouritesCount);
Assert.Equal(1, stored.DownvotesCount);
Assert.True(await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.ActorURI == carol.Id && r.Emoji == "🔥").ExecuteAnyAsync(token));
}
[Fact]
public async Task A_vote_turned_the_other_way_replaces_the_first_as_lemmy_sends_it()
{
var token = TestContext.Current.CancellationToken;
var (_, post) = await LocalPost();
var voter = new RemoteActor(_harness.Peer, "voter");
var other = new RemoteActor(_harness.Peer, "other");
JsonObject Vote(RemoteActor actor, string type) => new()
{
["id"] = $"{Origin(actor)}/{type.ToLowerInvariant()}s/{Guid.NewGuid():N}", ["type"] = type, ["actor"] = actor.Id, ["object"] = post.ObjectURI
};
await _harness.Deliver(other, "/human-centipede", Vote(other, "Like"));
await _harness.Deliver(voter, "/human-centipede", Vote(voter, "Dislike"));
await _harness.Deliver(voter, "/human-centipede", Vote(voter, "Like"));
var up = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Equal((2, 0), (up.FavouritesCount, up.DownvotesCount));
await _harness.Deliver(voter, "/human-centipede", Vote(voter, "Dislike"));
var down = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Equal((1, 1), (down.FavouritesCount, down.DownvotesCount));
Assert.False(await DB.Default.Find<Favourite>().Match(f => f.PostId == post.ID && f.ActorURI == voter.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_circle_member_likes_a_circle_post_and_an_outsider_cannot()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var circle = new Group
{
UserName = $"circle{Guid.NewGuid():N}"[..20], Name = "a circle", Kind = GroupKind.Circle, OwnerAvatarId = alice.Id,
Members = { new() { AvatarId = alice.Id }, new() { AvatarId = carol.Id, IsForeign = true } }
};
await DB.Default.SaveAsync(circle, token);
var post = new Post
{
GroupUserId = alice.Id, AuthorAccountId = alice.Id, GroupId = circle.ID, Visibility = PostVisibility.Circle, Text = "only us",
ObjectURI = $"{alice.Uri}/scribbles/{Guid.NewGuid():N}"
};
await DB.Default.SaveAsync(post, token);
foreach (var who in new[] { carol, mallory })
await _harness.Deliver(who, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(who)}/likes/{Guid.NewGuid():N}", ["type"] = "Like", ["actor"] = who.Id, ["object"] = post.ObjectURI
});
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
Assert.True(await DB.Default.Find<Favourite>().Match(f => f.PostId == post.ID && f.ActorURI == carol.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_boost_of_a_local_post_counts_notifies_and_can_be_undone()
{
@@ -0,0 +1,714 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using OpenIddict.Abstractions;
using PrivaPub.Api.Mastodon.Auth;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Media;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Media;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class JobHandlerTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
sealed class PeerDescriber : InstanceDescriber
{
readonly string _peer;
public PeerDescriber(IFederationHttp http, string peer) : base(http) => _peer = peer;
protected override string Address(string url) => _peer + new Uri(url).PathAndQuery;
}
[Fact]
public async Task Ancestors_are_fetched_one_job_at_a_time_and_stop_at_the_depth_limit()
{
var token = TestContext.Current.CancellationToken;
var poster = new RemoteActor(_harness.Peer, "poster");
var chain = new List<JsonObject>();
for (var i = 0; i < RemotePosts.MaxDepth + 3; i++)
{
var note = PublicNote(poster, $"<p>number {i}</p>");
if (i > 0)
note["inReplyTo"] = IdOf(chain[^1]);
_harness.Peer.Serve(new Uri(IdOf(note)).AbsolutePath, note.ToJsonString());
chain.Add(note);
}
var leaf = new Post
{
ObjectURI = NewId(poster, "notes"), ActorURI = poster.Id, IsFederatedCopy = true, Visibility = PostVisibility.Public,
InReplyToURI = IdOf(chain[^1]), ContentHtml = "<p>the leaf</p>"
};
await DB.Default.SaveAsync(leaf, token);
await _harness.Queue.Enqueue(JobKind.FetchAncestors, JsonSerializer.Serialize(new AncestorsPayload(leaf.ID, 1)), "127.0.0.1", "ancestors|" + leaf.ID, token);
var handler = new AncestorsJobHandler(_harness.Db, _harness.RemotePosts);
var keys = new List<string> { "ancestors|" + leaf.ID };
var ran = 0;
while (await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.FetchAncestors && j.State == JobState.Pending && keys.Contains(j.DedupeKey))
.ExecuteFirstAsync(token) is { } job)
{
Assert.Equal(JobResult.Done, (await handler.Handle(job, token)).Result);
await DB.Default.Update<Job>().MatchID(job.ID).Modify(j => j.State, JobState.Done).ExecuteAsync(token);
ran++;
var uris = chain.Select(IdOf).ToList();
keys = (await DB.Default.Find<Post>().Match(p => uris.Contains(p.ObjectURI)).ExecuteAsync(token)).Select(p => "ancestors|" + p.ID).Append("ancestors|" + leaf.ID).ToList();
Assert.True(ran <= RemotePosts.MaxDepth, "the chain did not stop");
}
var all = chain.Select(IdOf).ToList();
var stored = (await DB.Default.Find<Post>().Match(p => all.Contains(p.ObjectURI)).ExecuteAsync(token)).ToDictionary(p => p.ObjectURI);
Assert.Equal(RemotePosts.MaxDepth, ran);
Assert.Equal(all.Skip(3), stored.Keys.OrderBy(all.IndexOf));
Assert.Equal(stored[all[^1]].ID, (await DB.Default.Find<Post>().OneAsync(leaf.ID, token)).AnsweringToPostId);
for (var i = 4; i < all.Count; i++)
Assert.Equal(stored[all[i - 1]].ID, stored[all[i]].AnsweringToPostId);
Assert.Null(stored[all[3]].AnsweringToPostId);
Assert.Equal(all[2], stored[all[3]].InReplyToURI);
Assert.DoesNotContain(_harness.Peer.Requests, r => r.Path == new Uri(all[2]).AbsolutePath);
}
JsonObject Served(JsonObject document)
{
_harness.Peer.Serve(new Uri(IdOf(document)).AbsolutePath, document.ToJsonString());
return document;
}
static JsonObject Reply(RemoteActor author, string text, JsonObject parent)
{
var note = PublicNote(author, $"<p>{text}</p>");
note["inReplyTo"] = IdOf(parent);
return note;
}
static async Task<Post> Held(JsonObject note)
{
var post = new Post
{
ObjectURI = IdOf(note), ActorURI = note["attributedTo"]!.GetValue<string>(), IsFederatedCopy = true, Visibility = PostVisibility.Public,
ContentHtml = note["content"]!.GetValue<string>()
};
await DB.Default.SaveAsync(post, TestContext.Current.CancellationToken);
return post;
}
// runs the FetchReplies job for the post, and those it queues for this peer's posts, until none is left; how many ran
async Task<int> RunReplies(Post first)
{
var token = TestContext.Current.CancellationToken;
var handler = new RepliesJobHandler(_harness.Db, _harness.Remote, _harness.RemotePosts, _harness.Queue);
var origin = new Uri(first.ObjectURI).GetLeftPart(UriPartial.Authority);
await _harness.Queue.EnqueueMany(new[] { RepliesJobHandler.For(first, 1) }, token);
var ran = 0;
while (true)
{
var mine = new List<Job>();
foreach (var job in await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.FetchReplies && j.State == JobState.Pending).ExecuteAsync(token))
{
var post = await DB.Default.Find<Post>().OneAsync(JsonSerializer.Deserialize<RepliesPayload>(job.Payload)!.PostId, token);
if (post?.ObjectURI.StartsWith(origin) == true)
mine.Add(job);
}
if (mine.Count == 0)
return ran;
foreach (var job in mine)
{
Assert.Equal(JobResult.Done, (await handler.Handle(job, token)).Result);
await DB.Default.Update<Job>().MatchID(job.ID).Modify(j => j.State, JobState.Done).ExecuteAsync(token);
ran++;
}
}
}
static async Task<Dictionary<string, Post>> Stored(params JsonObject[] notes)
{
var uris = notes.Select(IdOf).ToList();
return (await DB.Default.Find<Post>().Match(p => uris.Contains(p.ObjectURI)).ExecuteAsync(TestContext.Current.CancellationToken))
.ToDictionary(p => p.ObjectURI);
}
[Fact]
public async Task A_threads_replies_are_read_from_their_servers_two_levels_down_and_only_public_ones_kept()
{
var poster = new RemoteActor(_harness.Peer, "poster");
var answerer = new RemoteActor(_harness.Peer, "answerer");
var root = PublicNote(poster, "<p>the root</p>");
// what a post carries is as old as the post: the collection is read again by its id
root["replies"] = new JsonObject
{
["id"] = IdOf(root) + "/replies", ["type"] = "Collection", ["first"] = new JsonObject { ["type"] = "CollectionPage", ["items"] = new JsonArray() }
};
Served(root);
var first = Reply(answerer, "first answer", root);
var second = Served(Reply(poster, "second answer", root));
var hidden = Reply(answerer, "for followers", root);
hidden["to"] = new JsonArray(answerer.Id + "/followers");
hidden["cc"] = new JsonArray();
Served(hidden);
var nested = Reply(poster, "nested", first);
var deeper = Served(Reply(answerer, "too deep", nested));
first["replies"] = IdOf(first) + "/replies";
Served(first);
nested["replies"] = IdOf(nested) + "/replies";
Served(nested);
Served(new JsonObject { ["id"] = IdOf(root) + "/replies", ["type"] = "Collection", ["first"] = IdOf(root) + "/replies/1" });
Served(new JsonObject
{
["id"] = IdOf(root) + "/replies/1", ["type"] = "CollectionPage", ["next"] = IdOf(root) + "/replies/2",
["items"] = new JsonArray(first.DeepClone(), IdOf(second))
});
Served(new JsonObject { ["id"] = IdOf(root) + "/replies/2", ["type"] = "CollectionPage", ["items"] = new JsonArray(IdOf(hidden)) });
Served(new JsonObject { ["id"] = IdOf(first) + "/replies", ["type"] = "OrderedCollection", ["orderedItems"] = new JsonArray(IdOf(nested)) });
Served(new JsonObject { ["id"] = IdOf(nested) + "/replies", ["type"] = "OrderedCollection", ["orderedItems"] = new JsonArray(IdOf(deeper)) });
var held = await Held(root);
Assert.Equal(3, await RunReplies(held));
var stored = await Stored(first, second, hidden, nested, deeper);
Assert.Equal(new[] { IdOf(first), IdOf(second), IdOf(nested) }.Order(), stored.Keys.Order());
Assert.Equal(held.ID, stored[IdOf(first)].AnsweringToPostId);
Assert.Equal(held.ID, stored[IdOf(second)].AnsweringToPostId);
Assert.Equal(stored[IdOf(first)].ID, stored[IdOf(nested)].AnsweringToPostId);
Assert.DoesNotContain(_harness.Peer.Requests, r => r.Path == new Uri(IdOf(nested) + "/replies").AbsolutePath);
Assert.All(_harness.Peer.Requests.Where(r => r.Path.EndsWith("/replies")), r => Assert.False(string.IsNullOrEmpty(r.Signature)));
}
[Fact]
public async Task A_threads_context_collection_brings_every_reply_at_once()
{
var poster = new RemoteActor(_harness.Peer, "poster");
var answerer = new RemoteActor(_harness.Peer, "answerer");
var root = PublicNote(poster, "<p>a thread</p>");
var context = NewId(poster, "contexts");
root["context"] = context;
root["replies"] = NewId(poster, "never");
Served(root);
var answer = Served(Reply(answerer, "an answer", root));
var deep = Served(Reply(poster, "an answer to it", answer));
// FEP-f228 lets a context list the activities that made the thread
Served(new JsonObject
{
["id"] = context, ["type"] = "OrderedCollection",
["first"] = new JsonObject { ["type"] = "OrderedCollectionPage", ["orderedItems"] = new JsonArray(IdOf(root), Create(answerer, answer), IdOf(deep)) }
});
var held = await Held(root);
Assert.Equal(1, await RunReplies(held));
var stored = await Stored(answer, deep);
Assert.Equal(held.ID, stored[IdOf(answer)].AnsweringToPostId);
Assert.Equal(stored[IdOf(answer)].ID, stored[IdOf(deep)].AnsweringToPostId);
Assert.DoesNotContain(_harness.Peer.Requests, r => r.Path == new Uri(root["replies"]!.GetValue<string>()).AbsolutePath);
}
async Task<(LocalActor Alice, RemoteActor Mallory, Post Poll)> LocalPoll()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "follows"), ["type"] = "Follow", ["actor"] = mallory.Id, ["object"] = alice.Uri
});
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "which?", Poll = new PollDraft(new[] { "tea", "coffee" }, 3600, false, false) }, token);
Assert.True(outcome.Ok);
return (alice, mallory, outcome.Post);
}
async Task<List<JsonObject>> Updates(RemoteActor to) =>
(await _harness.Outgoing(to.Id + "/inbox")).Where(a => a["type"]!.GetValue<string>() == "Update").ToList();
static int[] Counts(JsonNode question) =>
question["oneOf"]!.AsArray().Select(o => o!["replies"]!["totalItems"]!.GetValue<int>()).ToArray();
[Fact]
public async Task A_poll_refresh_sends_the_new_counts_to_followers_as_an_update()
{
var token = TestContext.Current.CancellationToken;
var (_, mallory, poll) = await LocalPoll();
var (_, bob) = await _harness.Persona("bob");
Assert.True((await _harness.Polls.Vote(bob, poll, new[] { 1 }, token)).Ok);
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollRefresh && j.Payload == poll.ID).ExecuteSingleAsync(token);
Assert.True(job.RunAt > DateTime.UtcNow);
Assert.Empty(await Updates(mallory));
Assert.Equal(JobResult.Done, (await new PollRefreshJob(_harness.Db, _harness.Local, _harness.Outbox).Handle(job, token)).Result);
var question = Assert.Single(await Updates(mallory))["object"]!;
Assert.Equal("Question", question["type"]!.GetValue<string>());
Assert.Equal(new[] { 0, 1 }, Counts(question));
Assert.Null(question["closed"]);
Assert.Null((await DB.Default.Find<Post>().OneAsync(poll.ID, token)).Poll.ClosedAt);
}
[Fact]
public async Task Closing_a_local_poll_tells_its_voters_and_sends_the_closed_question()
{
var token = TestContext.Current.CancellationToken;
var (alice, mallory, poll) = await LocalPoll();
var (_, bob) = await _harness.Persona("bob");
Assert.True((await _harness.Polls.Vote(bob, poll, new[] { 0 }, token)).Ok);
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollClose && j.Payload == poll.ID).ExecuteSingleAsync(token);
Assert.Equal(JobResult.Done, (await new PollCloseJob(_harness.Db, _harness.Local, _harness.Outbox).Handle(job, token)).Result);
var closed = (await DB.Default.Find<Post>().OneAsync(poll.ID, token)).Poll.ClosedAt;
Assert.NotNull(closed);
var question = Assert.Single(await Updates(mallory))["object"]!;
Assert.NotNull(question["closed"]);
Assert.Equal(new[] { 1, 0 }, Counts(question));
var told = await DB.Default.Find<Notification>().Match(n => n.Type == NotificationType.Poll && n.PostId == poll.ID).ExecuteAsync(token);
Assert.Equal(new[] { alice.Id, bob.Id }.Order(), told.Select(n => n.AvatarId).Order());
}
[Fact]
public async Task Closing_a_remote_poll_tells_the_local_voters_and_sends_nothing()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var question = PublicNote(mallory, "<p>pick</p>", alice.Uri);
question["type"] = "Question";
question["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" });
question["endTime"] = DateTime.UtcNow.AddMinutes(10).ToString("O");
question["oneOf"] = new JsonArray(
new JsonObject { ["type"] = "Note", ["name"] = "red", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = 0 } },
new JsonObject { ["type"] = "Note", ["name"] = "blue", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = 0 } });
await _harness.Deliver(mallory, "/human-centipede", Create(mallory, question));
var poll = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(question)).ExecuteSingleAsync(token);
Assert.True((await _harness.Polls.Vote(alice, poll, new[] { 1 }, token)).Ok);
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollClose && j.Payload == poll.ID).ExecuteSingleAsync(token);
Assert.Equal(JobResult.Done, (await new PollCloseJob(_harness.Db, _harness.Local, _harness.Outbox).Handle(job, token)).Result);
var told = Assert.Single(await DB.Default.Find<Notification>().Match(n => n.Type == NotificationType.Poll && n.PostId == poll.ID).ExecuteAsync(token));
Assert.Equal(alice.Id, told.AvatarId);
Assert.Equal(poll.AuthorAccountId, told.FromAccountId);
Assert.Empty(await Updates(mallory));
Assert.Null((await DB.Default.Find<Post>().OneAsync(poll.ID, token)).Poll.ClosedAt);
}
[Fact]
public async Task A_server_is_described_from_its_nodeinfo_and_at_most_once_a_week()
{
var token = TestContext.Current.CancellationToken;
var host = $"describe{Guid.NewGuid():N}.example";
var peer = _harness.Peer;
peer.ServeText("/.well-known/nodeinfo", new JsonObject
{
["links"] = new JsonArray(
new JsonObject { ["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.0", ["href"] = $"https://{host}/nodeinfo/2.0" },
new JsonObject { ["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.1", ["href"] = $"https://{host}/nodeinfo/2.1" })
}.ToJsonString(), "application/json");
peer.ServeText("/nodeinfo/2.1", new JsonObject
{
["version"] = "2.1",
["software"] = new JsonObject { ["name"] = "gotosocial", ["version"] = "0.20.1" },
["protocols"] = new JsonArray("activitypub", 42),
["openRegistrations"] = false,
["usage"] = new JsonObject { ["users"] = new JsonObject { ["total"] = 3 } },
["metadata"] = new JsonObject { ["nodeName"] = "A small place" }
}.ToJsonString(), "application/json");
var describer = new PeerDescriber(Peer.Http(), peer.A);
async Task Record()
{
var note = NoteParser.Parse(new JsonObject
{
["id"] = $"https://{host}/notes/{Guid.NewGuid():N}", ["type"] = "Note", ["attributedTo"] = $"https://{host}/users/x",
["content"] = "<p>hi</p>", ["to"] = new JsonArray(Addressing.Public)
});
var post = new Post { ObjectURI = note.Id, IsFederatedCopy = true };
await DB.Default.SaveAsync(post, token);
await _harness.Records.Record(note, post, ObjectPath.Fetched, refetched: false, token);
}
Task<List<Job>> Jobs() => DB.Default.Find<Job>().Match(j => j.Kind == JobKind.DescribeInstance && j.Payload == host).ExecuteAsync(token);
await Record();
await Record();
var job = Assert.Single(await Jobs());
Assert.StartsWith($"describe|{host}|", job.DedupeKey);
Assert.Equal(JobResult.Done, (await describer.Handle(job, token)).Result);
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteSingleAsync(token);
Assert.Equal(("gotosocial", "0.20.1", "A small place"), (instance.Software, instance.SoftwareVersion, instance.NodeName));
Assert.Equal(new[] { "activitypub" }, instance.Protocols);
Assert.False(instance.OpenRegistrations);
Assert.Equal(3, JsonNode.Parse(instance.NodeInfo)!["usage"]!["users"]!["total"]!.GetValue<int>());
Assert.Null(instance.DescriptionError);
Assert.InRange(instance.DescribedAt!.Value, DateTime.UtcNow.AddMinutes(-1), DateTime.UtcNow.AddSeconds(1));
Assert.DoesNotContain(peer.Requests, r => r.Path == "/nodeinfo/2.0");
await DB.Default.DeleteAsync<Job>(job.ID);
await Record();
Assert.Empty(await Jobs());
await DB.Default.Update<RemoteInstance>().MatchID(instance.ID).Modify(i => i.DescribedAt, DateTime.UtcNow.AddDays(-8)).ExecuteAsync(token);
await Record();
Assert.Single(await Jobs());
}
[Fact]
public async Task A_server_without_usable_nodeinfo_is_described_as_such()
{
var token = TestContext.Current.CancellationToken;
var plain = $"plain{Guid.NewGuid():N}.example";
var describer = new PeerDescriber(Peer.Http(), _harness.Peer.A + "/" + plain);
_harness.Peer.ServeText($"/{plain}/.well-known/nodeinfo", new JsonObject
{
["links"] = new JsonArray(new JsonObject { ["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.1", ["href"] = $"http://{plain}/nodeinfo/2.1" })
}.ToJsonString(), "application/json");
var silent = $"silent{Guid.NewGuid():N}.example";
await describer.Handle(new Job { Kind = JobKind.DescribeInstance, Payload = plain }, token);
await new PeerDescriber(Peer.Http(), _harness.Peer.A + "/" + silent).Handle(new Job { Kind = JobKind.DescribeInstance, Payload = silent }, token);
Assert.Equal("no NodeInfo", (await DB.Default.Find<RemoteInstance>().Match(i => i.Host == plain).ExecuteSingleAsync(token)).DescriptionError);
Assert.Equal("no NodeInfo", (await DB.Default.Find<RemoteInstance>().Match(i => i.Host == silent).ExecuteSingleAsync(token)).DescriptionError);
}
LinkPreviews Previews() => new(_harness.Db, _harness.Local, Peer.Http(), _harness.Queue,
new StaticOptions<FederationOptions>(new FederationOptions { AllowPrivateNetworks = true, AllowPlainHttp = true }));
async Task<Post> Linking(PostVisibility visibility, string path)
{
var post = new Post
{
ObjectURI = $"https://elsewhere.example/notes/{Guid.NewGuid():N}", IsFederatedCopy = true, Visibility = visibility,
ContentHtml = $"<p>read <a href=\"{_harness.Peer.A}{path}\">this</a></p>"
};
await DB.Default.SaveAsync(post, TestContext.Current.CancellationToken);
_harness.Peer.ServeText(path, """
<html><head><title>Fallback</title>
<meta property="og:title" content="A headline">
<meta property="og:description" content="What it says">
<meta property="og:image" content="/cover.jpg">
</head><body></body></html>
""", "text/html; charset=utf-8");
return post;
}
[Fact]
public async Task A_link_preview_is_built_from_open_graph_for_a_public_post_only()
{
var token = TestContext.Current.CancellationToken;
var publicPath = $"/articles/{Guid.NewGuid():N}";
var privatePath = $"/articles/{Guid.NewGuid():N}";
var open = await Linking(PostVisibility.Public, publicPath);
var followersOnly = await Linking(PostVisibility.FollowersOnly, privatePath);
Assert.Equal(JobResult.Done, (await Previews().Handle(new Job { Kind = JobKind.FetchPreview, Payload = open.ID }, token)).Result);
Assert.Equal(JobResult.Done, (await Previews().Handle(new Job { Kind = JobKind.FetchPreview, Payload = followersOnly.ID }, token)).Result);
var card = (await DB.Default.Find<Post>().OneAsync(open.ID, token)).Link;
Assert.Equal(_harness.Peer.A + publicPath, card.Href);
Assert.Equal(("A headline", "What it says", _harness.Peer.A + "/cover.jpg"), (card.Title, card.Description, card.ImageURL));
Assert.True(await DB.Default.Find<LinkPreview>().Match(p => p.Url == _harness.Peer.A + publicPath).ExecuteAnyAsync(token));
Assert.Null((await DB.Default.Find<Post>().OneAsync(followersOnly.ID, token)).Link);
Assert.DoesNotContain(_harness.Peer.Requests, r => r.Path == privatePath);
Assert.False(await DB.Default.Find<LinkPreview>().Match(p => p.Url == _harness.Peer.A + privatePath).ExecuteAnyAsync(token));
}
[Fact]
public async Task The_janitor_removes_stale_unattached_uploads_and_trims_the_proxy_cache_oldest_first()
{
var token = TestContext.Current.CancellationToken;
var root = Path.Combine(Path.GetTempPath(), $"privapub-janitor-{Guid.NewGuid():N}");
var options = new StaticOptions<MediaOptions>(new MediaOptions { Root = root, ProxyCacheBytes = 800 });
var media = new MediaService(options, _harness.Local, default, NullLogger<MediaService>.Instance);
var seeded = new List<string>();
try
{
MediaAttachment Upload(string name, DateTime created, string postId = default)
{
Directory.CreateDirectory(Path.Combine(root, "files"));
File.WriteAllBytes(Path.Combine(root, "files", name), new byte[] { 1, 2, 3 });
File.WriteAllBytes(Path.Combine(root, "files", "small-" + name), new byte[] { 1 });
return new MediaAttachment { FilePath = Path.Combine("files", name), PreviewPath = Path.Combine("files", "small-" + name), CreatedAt = created, PostId = postId };
}
var stale = Upload("stale.webp", DateTime.UtcNow.AddDays(-2));
var fresh = Upload("fresh.webp", DateTime.UtcNow.AddHours(-1));
var attached = Upload("attached.webp", DateTime.UtcNow.AddDays(-3), "000000000000000000000001");
await DB.Default.SaveAsync(new[] { stale, fresh, attached }, token);
seeded.AddRange(new[] { stale.ID, fresh.ID, attached.ID });
Directory.CreateDirectory(media.ProxyRoot);
FileInfo Cached(string name, int bytes, int minutesAgo)
{
var path = Path.Combine(media.ProxyRoot, name);
File.WriteAllBytes(path, new byte[bytes]);
File.WriteAllText(path + ".type", "image/png");
File.SetLastWriteTimeUtc(path, DateTime.UtcNow.AddMinutes(-minutesAgo));
return new FileInfo(path);
}
var oldest = Cached("a", 600, 30);
var older = Cached("b", 400, 20);
var newest = Cached("c", 300, 10);
await new MediaJanitor(media, options, NullLogger<MediaJanitor>.Instance).Sweep(token);
// never posted for a day: trashed, its files out of what /media/files serves at once
Assert.NotNull((await DB.Default.Find<MediaAttachment>().OneAsync(stale.ID, token)).TrashedAt);
Assert.False(File.Exists(Path.Combine(root, stale.FilePath)));
Assert.False(File.Exists(Path.Combine(root, stale.PreviewPath)));
Assert.True(File.Exists(Path.Combine(media.TrashRoot, stale.FilePath)));
// and once its grace has passed, deleted with its row
await DB.Default.Update<MediaAttachment>().MatchID(stale.ID).Modify(m => m.TrashedAt, DateTime.UtcNow - MediaJanitor.TrashGrace - TimeSpan.FromMinutes(1)).ExecuteAsync(token);
await new MediaJanitor(media, options, NullLogger<MediaJanitor>.Instance).Sweep(token);
Assert.False(await DB.Default.Find<MediaAttachment>().Match(m => m.ID == stale.ID).ExecuteAnyAsync(token));
Assert.False(File.Exists(Path.Combine(media.TrashRoot, stale.FilePath)));
Assert.False(File.Exists(Path.Combine(media.TrashRoot, stale.PreviewPath)));
Assert.True(await DB.Default.Find<MediaAttachment>().Match(m => m.ID == fresh.ID).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<MediaAttachment>().Match(m => m.ID == attached.ID).ExecuteAnyAsync(token));
Assert.True(File.Exists(Path.Combine(root, fresh.FilePath)));
Assert.False(File.Exists(oldest.FullName));
Assert.False(File.Exists(oldest.FullName + ".type"));
Assert.True(File.Exists(older.FullName));
Assert.True(File.Exists(older.FullName + ".type"));
Assert.True(File.Exists(newest.FullName));
}
finally
{
await DB.Default.DeleteAsync<MediaAttachment>(m => seeded.Contains(m.ID));
foreach (var directory in new[] { root, media.ProxyRoot }.Where(Directory.Exists))
Directory.Delete(directory, recursive: true);
}
}
}
[Trait("Category", "Integration")]
public sealed class OAuthPrunerTests
{
[Fact]
public async Task Old_tokens_and_authorizations_that_are_no_longer_valid_are_pruned_and_the_rest_kept()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var host = await PrivaPubHost.Shared();
var now = DateTimeOffset.UtcNow;
using var scope = host.Services.CreateScope();
var tokens = scope.ServiceProvider.GetRequiredService<IOpenIddictTokenManager>();
var authorizations = scope.ServiceProvider.GetRequiredService<IOpenIddictAuthorizationManager>();
var subject = $"pruned{Guid.NewGuid():N}";
async Task<string> Token(DateTimeOffset created, DateTimeOffset expires, string status) =>
await tokens.GetIdAsync(await tokens.CreateAsync(new OpenIddictTokenDescriptor
{
CreationDate = created, ExpirationDate = expires, Status = status, Subject = subject, Type = OpenIddictConstants.TokenTypeHints.AccessToken
}, token), token);
async Task<string> Authorization(DateTimeOffset created, string status) =>
await authorizations.GetIdAsync(await authorizations.CreateAsync(new OpenIddictAuthorizationDescriptor
{
CreationDate = created, Status = status, Subject = subject, Type = OpenIddictConstants.AuthorizationTypes.Permanent
}, token), token);
var expired = await Token(now.AddDays(-30), now.AddDays(-29), OpenIddictConstants.Statuses.Valid);
var revoked = await Token(now.AddDays(-30), now.AddDays(1), OpenIddictConstants.Statuses.Revoked);
var current = await Token(now.AddMinutes(-5), now.AddHours(1), OpenIddictConstants.Statuses.Valid);
var recentlyRevoked = await Token(now.AddDays(-1), now.AddDays(-1).AddMinutes(1), OpenIddictConstants.Statuses.Revoked);
var oldRevoked = await Authorization(now.AddDays(-30), OpenIddictConstants.Statuses.Revoked);
var oldValid = await Authorization(now.AddDays(-30), OpenIddictConstants.Statuses.Valid);
var (prunedTokens, prunedAuthorizations) = await new OAuthPruner(host.Services, NullLogger<OAuthPruner>.Instance).Prune(now.AddDays(-14), token);
Assert.True(prunedTokens >= 2, $"{prunedTokens} tokens pruned");
Assert.True(prunedAuthorizations >= 1, $"{prunedAuthorizations} authorizations pruned");
//a fresh scope: the managers cache what they created
using var after = host.Services.CreateScope();
tokens = after.ServiceProvider.GetRequiredService<IOpenIddictTokenManager>();
authorizations = after.ServiceProvider.GetRequiredService<IOpenIddictAuthorizationManager>();
Assert.Null(await tokens.FindByIdAsync(expired, token));
Assert.Null(await tokens.FindByIdAsync(revoked, token));
Assert.NotNull(await tokens.FindByIdAsync(current, token));
Assert.NotNull(await tokens.FindByIdAsync(recentlyRevoked, token));
Assert.Null(await authorizations.FindByIdAsync(oldRevoked, token));
Assert.NotNull(await authorizations.FindByIdAsync(oldValid, token));
}
}
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public sealed class DeliveryOutcomeTests : IAsyncLifetime
{
static readonly string[] PeerHosts = { "localhost", "127.0.0.1" };
Harness _harness;
WebApplication _hinting;
string _hintingBase;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
var builder = WebApplication.CreateSlimBuilder();
builder.WebHost.UseUrls("http://127.0.0.1:0");
_hinting = builder.Build();
//answers /{status}/{retry-after}: the peer cannot send a Retry-After
_hinting.Run(context =>
{
var parts = context.Request.Path.Value!.Trim('/').Split('/');
context.Response.StatusCode = int.Parse(parts[0]);
if (parts.Length > 1)
context.Response.Headers.RetryAfter = parts[1] == "date"
? DateTimeOffset.UtcNow.AddMinutes(10).ToString("r")
: parts[1];
return Task.CompletedTask;
});
await _hinting.StartAsync();
_hintingBase = _hinting.Urls.First();
}
public async ValueTask DisposeAsync()
{
if (_hinting != default)
await _hinting.DisposeAsync();
if (_harness == default)
return;
await _harness.DisposeAsync();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
async Task<JobOutcome> Attempt(LocalActor signer, string inbox, int attempts = 1)
{
var token = TestContext.Current.CancellationToken;
var id = $"{Harness.Base}/a/{Guid.NewGuid():N}";
await _harness.Delivery.Enqueue(signer, new[] { inbox }, new JsonObject
{
["id"] = id, ["type"] = "Create", ["actor"] = signer.Uri, ["to"] = new JsonArray(Addressing.Public),
["object"] = new JsonObject { ["type"] = "Note", ["content"] = "hi" }
}, token);
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == $"{id}|{inbox}").ExecuteSingleAsync(token);
job.Attempts = attempts;
var handler = new DeliveryJobHandler(_harness.Local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())),
NullLogger<DeliveryJobHandler>.Instance);
return await handler.Handle(job, token);
}
[Fact]
public async Task A_delivery_carries_a_valid_digest_and_a_signature_by_the_signer()
{
var (_, alice) = await _harness.Persona("alice");
_harness.Peer.Answer("/ok/inbox", 202);
var outcome = await Attempt(alice, _harness.Peer.A + "/ok/inbox");
Assert.Equal(JobResult.Done, outcome.Result);
var received = Assert.Single(_harness.Peer.Requests, r => r.Path == "/ok/inbox");
Assert.Equal("POST", received.Method);
Assert.Equal(alice.Uri, JsonNode.Parse(received.Body)!["actor"]!.GetValue<string>());
Assert.Equal(HttpSignatures.Digest(Encoding.UTF8.GetBytes(received.Body)), received.Headers["Digest"]);
var signature = HttpSignatures.Parse(received.Signature);
Assert.Equal(alice.KeyId, signature.KeyId);
Assert.Equal(new[] { "(request-target)", "host", "date", "digest" }, signature.Headers);
var signingString = $"(request-target): post /ok/inbox\nhost: {received.Headers["Host"]}\ndate: {received.Headers["Date"]}\ndigest: {received.Headers["Digest"]}";
using var key = RSA.Create();
key.ImportFromPem(alice.PublicKeyPem);
Assert.True(key.VerifyData(Encoding.UTF8.GetBytes(signingString), signature.Signature, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
}
[Fact]
public async Task Refusals_are_dead_hints_defer_and_failures_retry()
{
var (_, alice) = await _harness.Persona("alice");
_harness.Peer.Answer("/gone/inbox", 410);
_harness.Peer.Answer("/missing/inbox", 404);
_harness.Peer.Answer("/broken/inbox", 500);
var gone = await Attempt(alice, _harness.Peer.A + "/gone/inbox");
var missing = await Attempt(alice, _harness.Peer.A + "/missing/inbox");
var busy = await Attempt(alice, _hintingBase + "/429/120");
var down = await Attempt(alice, _hintingBase + "/503/date");
var unhinted = await Attempt(alice, _hintingBase + "/503");
var broken = await Attempt(alice, _harness.Peer.A + "/broken/inbox");
Assert.Equal(JobResult.Dead, gone.Result);
Assert.StartsWith("410", gone.Error);
Assert.Equal(JobResult.Dead, missing.Result);
Assert.Equal(JobResult.Defer, busy.Result);
Assert.InRange(busy.RetryAt!.Value, DateTime.UtcNow.AddSeconds(100), DateTime.UtcNow.AddSeconds(125));
Assert.Equal(JobResult.Defer, down.Result);
Assert.InRange(down.RetryAt!.Value, DateTime.UtcNow.AddMinutes(9), DateTime.UtcNow.AddMinutes(11));
Assert.Equal(JobResult.Retry, unhinted.Result);
Assert.Equal(JobResult.Retry, broken.Result);
Assert.StartsWith("500", broken.Error);
}
// a server that fails on one activity is up: its 500s are that delivery's to retry, and only a gateway's 502, 503 or
// 504 counts towards taking the host out (Smithereen's 500s on documents it could not read once held every delivery
// to it back for an hour)
[Fact]
public async Task Only_a_gateways_failures_take_the_host_out()
{
var (_, alice) = await _harness.Persona("alice");
_harness.Peer.Answer("/broken/inbox", 500);
_harness.Peer.Answer("/gateway/inbox", 502);
for (var i = 0; i < HostCircuitBreaker.Threshold; i++)
await Attempt(alice, _harness.Peer.A + "/broken/inbox");
var stillTried = await Attempt(alice, _harness.Peer.A + "/broken/inbox");
for (var i = 0; i < HostCircuitBreaker.Threshold; i++)
await Attempt(alice, _harness.Peer.A + "/gateway/inbox");
var heldBack = await Attempt(alice, _harness.Peer.A + "/broken/inbox");
Assert.Equal((JobResult.Retry, "500"), (stillTried.Result, stillTried.Error[..3]));
Assert.Equal((JobResult.Defer, "the host is unavailable"), (heldBack.Result, heldBack.Error));
}
// Mastodon's 422 for two first contacts racing to create one account, and its 409 for a held lock, are retried a
// few times; the third refusal is final like any other
[Fact]
public async Task Conflicts_and_unprocessable_answers_are_retried_twice_then_dead()
{
var (_, alice) = await _harness.Persona("alice");
_harness.Peer.Answer("/racing/inbox", 422);
_harness.Peer.Answer("/locked/inbox", 409);
Assert.Equal(JobResult.Retry, (await Attempt(alice, _harness.Peer.A + "/racing/inbox")).Result);
Assert.Equal(JobResult.Retry, (await Attempt(alice, _harness.Peer.A + "/locked/inbox", attempts: 2)).Result);
var final = await Attempt(alice, _harness.Peer.A + "/racing/inbox", attempts: 3);
Assert.Equal(JobResult.Dead, final.Result);
Assert.StartsWith("422", final.Error);
}
}
}
+106
View File
@@ -0,0 +1,106 @@
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
// An account moving to another (Mastodon's Move), believed as Mastodon believes it: sent by the moving account about
// itself, and the new account naming it among its alsoKnownAs
[Trait("Category", "Integration")]
public sealed class MoveTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
// the new account, serving a document that names the old one (or not)
RemoteActor Target(string name, RemoteActor aliasOf)
{
var target = new RemoteActor(_harness.Peer, name);
var document = target.Document();
if (aliasOf != default)
document["alsoKnownAs"] = new JsonArray(aliasOf.Id);
_harness.Peer.Serve(new Uri(target.Id).AbsolutePath, document.ToJsonString());
return target;
}
Task Move(RemoteActor sender, RemoteActor moved, RemoteActor target) =>
_harness.Deliver(sender, "/human-centipede", new JsonObject
{
["id"] = NewId(sender, "moves"), ["type"] = "Move", ["actor"] = sender.Id, ["object"] = moved.Id, ["target"] = target.Id
});
static Task<ForeignAvatar> Stored(RemoteActor actor) =>
DB.Default.Find<ForeignAvatar>().Match(a => a.ActorURI == actor.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken);
// as Mastodon does it (owner decision 2026-10-05): the follow moves to the new account, in the same lists, and a mute
// or a block of the old account carries over
[Fact]
public async Task A_move_the_new_account_confirms_moves_the_personas_follows_lists_mutes_and_blocks()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var old = new RemoteActor(_harness.Peer, "old");
await _harness.Remote.GetActor(old.Id, refresh: true, token);
await Follows(alice.Id, old);
var oldAccount = await Stored(old);
var list = new PersonaList { AvatarId = alice.Id, Title = "friends" };
await DB.Default.SaveAsync(list, token);
await DB.Default.SaveAsync(new PersonaListMember { ListId = list.ID, AvatarId = alice.Id, AccountId = oldAccount.ID }, token);
await _harness.Relationships.Mute(alice, old.Id, oldAccount.ID, hideNotifications: true, duration: default, token);
await _harness.Relationships.Block(bob, old.Id, oldAccount.ID, token);
var target = Target("new", aliasOf: old);
await Move(old, old, target);
var stored = await Stored(old);
Assert.Equal(target.Id, stored.MovedToURL);
var account = await new MastodonMapper(_harness.Db, _harness.Local).Account(stored.ID, token);
Assert.Equal(target.Name, account.Moved?.Username);
Assert.Contains(_harness.Ledger.Of("in"), e => e.Activity == "Move" && e.Reason == "moved");
var newAccount = await Stored(target);
Assert.True(await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id && f.TargetActorURI == target.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Following>().Match(f => f.AvatarId == alice.Id && f.TargetActorURI == old.Id).ExecuteAnyAsync(token));
Assert.Contains(await _harness.Outgoing(target.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Follow");
Assert.Contains(await _harness.Outgoing(old.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Undo");
Assert.True(await DB.Default.Find<PersonaListMember>().Match(m => m.ListId == list.ID && m.AccountId == newAccount.ID).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Mute>().Match(m => m.AvatarId == alice.Id && m.TargetActorURI == target.Id).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Block>().Match(b => b.AvatarId == bob.Id && b.TargetActorURI == target.Id).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_move_the_new_account_does_not_confirm_or_someone_else_sends_changes_nothing()
{
var (_, alice) = await _harness.Persona("alice");
var old = new RemoteActor(_harness.Peer, "old");
var stranger = new RemoteActor(_harness.Peer, "stranger");
await Follows(alice.Id, old);
var unconfirmed = Target("new", aliasOf: default);
var hijack = Target("hijack", aliasOf: stranger);
await Move(old, old, unconfirmed);
await Move(stranger, old, hijack);
Assert.Null((await Stored(old)).MovedToURL);
Assert.Equal(new[] { "not-aliased", "not-self" }, _harness.Ledger.Of("in").Where(e => e.Activity == "Move").Select(e => e.Reason));
}
}
}
@@ -35,6 +35,26 @@ namespace PrivaPub.Tests.Federation
}
""";
// Castopod's episode: its words in description, its sound in audio, the sound's name for a title
[Fact]
public void A_podcast_episode_is_read_with_its_words_sound_and_cover()
{
var episode = NoteParser.Parse(JsonNode.Parse("""
{"id":"https://castopod.test/@pod/episodes/one","type":"PodcastEpisode","published":"2026-10-06T09:08:17+00:00",
"to":["https://www.w3.org/ns/activitystreams#Public"],"attributedTo":"https://castopod.test/@pod",
"description":{"type":"Note","mediaType":"text/markdown","content":"Listen to the **meadow**","contentMap":{"en":"<p>Listen to the meadow</p>"}},
"image":{"type":"Image","mediaType":"image/png","url":"https://castopod.test/media/cover.png"},
"audio":{"id":"https://castopod.test/audio/one.mp3","type":"Audio","name":"Episode one","size":16791,"duration":4.02,
"url":{"href":"https://castopod.test/audio/one.mp3","type":"Link","mediaType":"audio/mpeg"}}}
"""));
Assert.Equal(("PodcastEpisode", "Episode one", "https://castopod.test/media/cover.png"), (episode.Type, episode.Title, episode.CoverURL));
Assert.Contains("<strong>meadow</strong>", episode.ContentHtml);
var sound = Assert.Single(episode.Attachments);
Assert.Equal(("https://castopod.test/audio/one.mp3", "audio/mpeg"), (sound.RemoteURL, sound.ContentType));
Assert.Equal(4.02, episode.Audio.DurationSeconds);
}
[Fact]
public void Reads_a_mastodon_note()
{
@@ -57,6 +77,69 @@ namespace PrivaPub.Tests.Federation
Assert.Null(note.Updated);
}
// GoToSocial writes a lone tag, and lone to and cc, as plain values instead of arrays (captured by the town)
[Fact]
public void Reads_a_gotosocial_reply_whose_only_tag_and_audience_are_single_values()
{
var note = NoteParser.Parse(JsonNode.Parse(File.ReadAllText(Path.Combine(AppContext.BaseDirectory, "Fixtures", "gotosocial",
"followers-only-reply-one-tag.json"))));
var mention = Assert.Single(note.Mentions);
Assert.Equal("https://privapub.test/peasants/flor_pp98", mention.ActorURI);
Assert.Equal(["https://gts.test/users/ines_gt59/followers"], note.To);
Assert.Equal(["https://privapub.test/peasants/flor_pp98"], note.Cc);
}
// PeerTube's video whose comments wait for approval names who may comment (FEP-5624), apart from GoToSocial's policy
[Fact]
public void Reads_who_may_reply_pending_approval()
{
var video = NoteParser.Parse(JsonNode.Parse("""
{"id":"https://peertube.test/videos/watch/1","type":"Video","name":"a video","attributedTo":["https://peertube.test/accounts/pt"],
"canReply":"https://www.w3.org/ns/activitystreams#Public","commentsEnabled":true}
"""));
var open = NoteParser.Parse(JsonNode.Parse("""
{"id":"https://peertube.test/videos/watch/2","type":"Video","name":"an open video","attributedTo":["https://peertube.test/accounts/pt"],
"canReply":null,"commentsPolicy":1}
"""));
var note = NoteParser.Parse(JsonNode.Parse(MastodonNote));
Assert.Equal(["https://www.w3.org/ns/activitystreams#Public"], video.ReplyApprovals.Manual);
Assert.Null(video.ReplyPolicy);
Assert.Null(open.ReplyApprovals);
Assert.Null(note.ReplyApprovals);
}
// Pleroma's edited post carries its earlier versions (formerRepresentations, newest first)
[Fact]
public void Reads_the_earlier_versions_of_a_pleroma_edit()
{
var note = NoteParser.Parse(JsonNode.Parse(File.ReadAllText(Path.Combine(AppContext.BaseDirectory, "Fixtures", "pleroma", "edited.json"))));
var before = Assert.Single(note.FormerRepresentations);
Assert.StartsWith("tomorrow library garden bridge", before.ContentHtml);
Assert.Equal(new DateTime(2026, 10, 5, 0, 49, 58, DateTimeKind.Utc).AddTicks(8_600_710), before.EditedAt);
Assert.Null(before.SpoilerText);
Assert.EndsWith("(edited)", note.ContentHtml);
}
[Fact]
public void Earlier_versions_come_oldest_first_each_read_as_the_post_is()
{
var note = NoteParser.Parse(JsonNode.Parse("""
{"id":"https://pl.example/objects/1","type":"Note","attributedTo":"https://pl.example/users/carol","content":"<p>third</p>",
"published":"2026-10-06T08:00:00Z","updated":"2026-10-06T09:00:00Z",
"formerRepresentations":{"type":"OrderedCollection","totalItems":2,"orderedItems":[
{"type":"Note","content":"<p>second</p>","summary":"careful","published":"2026-10-06T08:00:00Z","updated":"2026-10-06T08:30:00Z"},
{"type":"Note","content":"<p>first<script>x</script></p>","summary":"","published":"2026-10-06T08:00:00Z"}]}}
"""));
Assert.Equal(["<p>first</p>", "<p>second</p>"], note.FormerRepresentations.Select(r => r.ContentHtml));
Assert.Equal([new DateTime(2026, 10, 6, 8, 0, 0, DateTimeKind.Utc), new DateTime(2026, 10, 6, 8, 30, 0, DateTimeKind.Utc)],
note.FormerRepresentations.Select(r => r.EditedAt));
Assert.Equal([(default(string), false), ("careful", true)], note.FormerRepresentations.Select(r => (r.SpoilerText, r.HasContentWarning)));
}
[Fact]
public void Reads_a_lemmy_page_with_a_title()
{
+53 -2
View File
@@ -1,4 +1,4 @@
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
@@ -79,6 +79,51 @@ namespace PrivaPub.Tests.Federation
Assert.Equal("2026-10-03 10:00 – 13:00 (Europe/Rome) · Biblioteca", MastodonMapper.EventLine(note.Event));
}
[Fact]
public void Keeps_a_pixelfed_place_whose_coordinates_are_strings()
{
var note = Parse("""
{
"id": "https://pixel.example/p/ann/1", "type": "Note", "attributedTo": "https://pixel.example/users/ann", "content": "<p>view</p>",
"location": { "type": "Place", "name": "Tokyo", "longitude": "139.6917", "latitude": "35.6895", "country": "Japan" },
"commentsEnabled": true
}
""");
Assert.Equal(("Tokyo", 35.6895, 139.6917, "Japan"), (note.Place.Name, note.Place.Latitude, note.Place.Longitude, note.Place.Country));
Assert.Null(note.Event);
}
[Theory]
[InlineData("null")]
[InlineData("""{ "type": "Place", "name": "Nowhere" }""")]
[InlineData("""{ "type": "Place", "name": "Off the map", "latitude": 123, "longitude": 9 }""")]
[InlineData("""{ "type": "Place", "latitude": "north", "longitude": "9.19" }""")]
[InlineData("""{ "type": "VirtualLocation", "url": "https://meet.example/a", "latitude": 45, "longitude": 9 }""")]
public void Has_no_place_without_a_place_with_coordinates(string location)
{
var note = Parse($$"""
{ "id": "https://pixel.example/p/ann/2", "type": "Note", "attributedTo": "https://pixel.example/users/ann", "content": "<p>x</p>", "location": {{location}} }
""");
Assert.Null(note.Place);
}
[Fact]
public void An_events_location_stays_the_events_own()
{
var note = Parse("""
{
"id": "https://events.example/events/10", "type": "Event", "name": "Picnic", "startTime": "2026-10-03T10:00:00+02:00",
"location": { "type": "Place", "name": "Park", "latitude": 45.07, "longitude": 7.68 },
"attributedTo": "https://events.example/@club"
}
""");
Assert.Null(note.Place);
Assert.Equal("Park", Assert.Single(note.Event.Places).Name);
}
[Fact]
public void Keeps_a_lemmy_link_post_as_a_link_with_its_thumbnail()
{
@@ -190,7 +235,7 @@ namespace PrivaPub.Tests.Federation
}
[Fact]
public void Reads_polls_from_mastodon_misskey_and_closed_ones()
public void Reads_polls_from_mastodon_misskey_akkoma_and_closed_ones()
{
var mastodon = Parse("""
{ "id": "https://m.example/q/1", "type": "Question", "attributedTo": "https://m.example/u", "content": "?", "endTime": "2026-10-02T00:00:00Z", "votersCount": 7,
@@ -204,6 +249,10 @@ namespace PrivaPub.Tests.Federation
{ "id": "https://g.example/q/1", "type": "Question", "attributedTo": "https://g.example/u", "content": "?", "closed": "2026-09-30T12:00:00Z",
"oneOf": [{ "type": "Note", "name": "p" }, { "type": "Note", "name": "q" }] }
""");
var akkoma = Parse("""
{ "id": "https://a.example/objects/1", "type": "Question", "attributedTo": "https://a.example/users/u", "content": "?", "closed": "2099-01-01T00:00:00.000000Z",
"oneOf": [{ "type": "Note", "name": "p" }, { "type": "Note", "name": "q" }] }
""");
Assert.False(mastodon.Poll.Multiple);
Assert.Equal(new[] { 4, 3 }, mastodon.Poll.Options.Select(o => o.Votes));
@@ -212,6 +261,8 @@ namespace PrivaPub.Tests.Federation
Assert.Equal(new[] { 2, 5 }, misskey.Poll.Options.Select(o => o.Votes));
Assert.Null(misskey.Poll.VotersCount);
Assert.Equal(new DateTime(2026, 9, 30, 12, 0, 0, DateTimeKind.Utc), closed.Poll.ClosedAt);
Assert.Null(akkoma.Poll.ClosedAt);
Assert.Equal(new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc), akkoma.Poll.ExpiresAt);
}
[Fact]
@@ -0,0 +1,100 @@
using MongoDB.Entities;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
// a persona joins a remote event (owner decision 2026-10-05): Join, answered Accept or Reject; Leave
[Trait("Category", "Integration")]
public sealed class ParticipationTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
async Task<(PrivaPub.Federation.Actors.LocalActor Alice, RemoteActor Group, Post Event)> Announced(string joinMode = "free")
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var group = new RemoteActor(_harness.Peer, "group", type: "Group");
await Follows(alice.Id, group);
var happening = new JsonObject
{
["id"] = NewId(group, "events"), ["type"] = "Event", ["name"] = "A picnic", ["content"] = "<p>bring bread</p>",
["attributedTo"] = group.Id, ["startTime"] = DateTime.UtcNow.AddDays(3).ToString("O"), ["joinMode"] = joinMode,
["to"] = new JsonArray(PrivaPub.Federation.Objects.Addressing.Public), ["cc"] = new JsonArray(group.Id + "/followers")
};
await _harness.Deliver(group, "/human-centipede", Create(group, happening));
return (alice, group, await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(happening)).ExecuteSingleAsync(token));
}
Task<Participation> Of(PrivaPub.Federation.Actors.LocalActor alice, Post happening) =>
DB.Default.Find<Participation>().Match(p => p.AvatarId == alice.Id && p.PostId == happening.ID).ExecuteFirstAsync(TestContext.Current.CancellationToken);
[Fact]
public async Task A_persona_joins_an_event_its_server_accepts_and_leaves_it()
{
var token = TestContext.Current.CancellationToken;
var (alice, group, happening) = await Announced();
Assert.True((await _harness.Participations.Join(alice, happening.ID, token)).Ok);
var join = Assert.Single(await _harness.Outgoing(group.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Join");
Assert.Equal((alice.Uri, happening.ObjectURI), (join["actor"]!.GetValue<string>(), join["object"]!.GetValue<string>()));
Assert.Equal(ParticipationState.Pending, (await Of(alice, happening)).State);
await _harness.Deliver(group, "/human-centipede", new JsonObject
{
["id"] = NewId(group, "accepts"), ["type"] = "Accept", ["actor"] = group.Id, ["object"] = join["id"]!.GetValue<string>()
});
Assert.Equal(ParticipationState.Accepted, (await Of(alice, happening)).State);
Assert.True((await _harness.Participations.Leave(alice, happening.ID, token)).Ok);
Assert.Null(await Of(alice, happening));
var leave = Assert.Single(await _harness.Outgoing(group.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Leave");
Assert.Equal(happening.ObjectURI, leave["object"]!.GetValue<string>());
}
[Fact]
public async Task A_refusal_counts_an_answer_from_another_server_does_not_and_an_invitation_only_event_is_not_joined()
{
var token = TestContext.Current.CancellationToken;
var (alice, group, happening) = await Announced();
await _harness.Participations.Join(alice, happening.ID, token);
var join = Assert.Single(await _harness.Outgoing(group.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Join");
var stranger = new RemoteActor(_harness.Peer, "stranger", _harness.Peer.B);
await _harness.Deliver(stranger, "/human-centipede", new JsonObject
{
["id"] = NewId(stranger, "accepts"), ["type"] = "Accept", ["actor"] = stranger.Id, ["object"] = join["id"]!.GetValue<string>()
});
Assert.Equal(ParticipationState.Pending, (await Of(alice, happening)).State);
await _harness.Deliver(group, "/human-centipede", new JsonObject
{
["id"] = NewId(group, "rejects"), ["type"] = "Reject", ["actor"] = group.Id,
["object"] = new JsonObject { ["id"] = join["id"]!.GetValue<string>(), ["type"] = "Join", ["actor"] = alice.Uri, ["object"] = happening.ObjectURI }
});
Assert.Equal(ParticipationState.Rejected, (await Of(alice, happening)).State);
var (carol, _, invitationOnly) = await Announced("invite");
Assert.Equal(422, (await _harness.Participations.Join(carol, invitationOnly.ID, token)).Status);
}
}
}
@@ -17,6 +17,9 @@ namespace PrivaPub.Tests.Federation
[InlineData(":blobcat@k.example:", "blobcat")]
[InlineData("👍🏽", "👍🏽")]
[InlineData("❤️", "❤️")]
[InlineData("❤", "❤️")]
[InlineData("✌", "✌️")]
[InlineData("🔥\uFE0F", "🔥")]
[InlineData("nope", null)]
[InlineData("👍👍", null)]
[InlineData(":unknown:", null)]
+143
View File
@@ -0,0 +1,143 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// relays the configuration names: the instance actor follows Public there, and the public posts the relay passes on,
// forwarded or announced, come to the federated timeline as their authors' posts; nothing else is taken from it
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class RelayTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
{
_harness.RelayOptions.Relays.Clear();
await _harness.Relays.Reconcile(TestContext.Current.CancellationToken);
await _harness.DisposeAsync();
}
}
static CancellationToken Token => TestContext.Current.CancellationToken;
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
string Note(RemoteActor author, string text, string to = "https://www.w3.org/ns/activitystreams#Public")
{
var path = $"/notes/{Guid.NewGuid():N}";
_harness.Peer.Serve(path, new JsonObject
{
["id"] = Origin(author) + path, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"<p>{text}</p>",
["to"] = new JsonArray(to), ["published"] = DateTime.UtcNow.ToString("O")
}.ToJsonString());
return Origin(author) + path;
}
JsonObject Create(RemoteActor author, string noteId) => new()
{
["id"] = noteId + "/activity", ["type"] = "Create", ["actor"] = author.Id,
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"),
["object"] = new JsonObject { ["id"] = noteId, ["type"] = "Note", ["attributedTo"] = author.Id }
};
// a relay the instance actor subscribes to, which accepts
async Task<RemoteActor> Subscribed()
{
var relay = new RemoteActor(_harness.Peer, "relay", type: "Service");
_harness.RelayOptions.Relays.Add(relay.Id);
await _harness.Relays.Reconcile(Token);
var follow = Assert.Single(await _harness.Outgoing(relay.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Follow");
await _harness.Deliver(relay, "/human-centipede", new JsonObject
{
["id"] = $"{relay.Id}#accept-{Guid.NewGuid():N}", ["type"] = "Accept", ["actor"] = relay.Id, ["object"] = follow.DeepClone()
});
return relay;
}
[Fact]
public async Task The_instance_actor_follows_public_at_a_configured_relay_and_takes_its_answer()
{
var relay = await Subscribed();
var follow = Assert.Single(await _harness.Outgoing(relay.Id + "/inbox"));
Assert.Equal(("Follow", "https://www.w3.org/ns/activitystreams#Public"), (follow["type"]!.GetValue<string>(), follow["object"]!.GetValue<string>()));
Assert.Equal((await _harness.Local.GetInstanceActor(Token)).Uri, follow["actor"]!.GetValue<string>());
Assert.Equal(RelayState.Accepted, (await DB.Default.Find<RelaySubscription>().Match(s => s.ActorURI == relay.Id).ExecuteSingleAsync(Token)).State);
Assert.True(await _harness.Relays.Passes(relay.Id, Token));
_harness.RelayOptions.Relays.Clear();
await _harness.Relays.Reconcile(Token);
Assert.Contains(await _harness.Outgoing(relay.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Undo");
Assert.False(await DB.Default.Find<RelaySubscription>().Match(s => s.ActorURI == relay.Id).ExecuteAnyAsync(Token));
}
[Fact]
public async Task A_public_post_a_subscribed_relay_forwards_is_kept_and_one_from_anyone_else_is_not()
{
var relay = await Subscribed();
var stranger = new RemoteActor(_harness.Peer, "stranger", type: "Service");
var author = new RemoteActor(_harness.Peer, "author", _harness.Peer.B);
var relayed = Note(author, "through the relay");
var forwarded = Note(author, "through a stranger");
var followersOnly = Note(author, "for followers", author.Id + "/followers");
await _harness.Deliver(relay, "/human-centipede", Create(author, relayed));
await _harness.Deliver(stranger, "/human-centipede", Create(author, forwarded));
await _harness.Deliver(relay, "/human-centipede", Create(author, followersOnly));
var kept = await DB.Default.Find<Post>().Match(p => p.ActorURI == author.Id).ExecuteAsync(Token);
Assert.Equal(new[] { relayed }, kept.Select(p => p.ObjectURI));
Assert.Equal(PostVisibility.Public, kept[0].Visibility);
}
// owner decision 2026-10-06: a persona's public post, its edit and its deletion go to the relays too; nothing less
// public does, nor a boost
[Fact]
public async Task A_personas_public_post_goes_to_the_relays_and_nothing_less_public()
{
var relay = await Subscribed();
var (_, alice) = await _harness.Persona("alice");
var token = Token;
var open = await _harness.Statuses.Publish(alice, new PrivaPub.Domain.Statuses.StatusDraft { Text = "for everyone", Visibility = PostVisibility.Public }, token);
await _harness.Statuses.Publish(alice, new PrivaPub.Domain.Statuses.StatusDraft { Text = "quietly", Visibility = PostVisibility.Unlisted }, token);
await _harness.Statuses.Publish(alice, new PrivaPub.Domain.Statuses.StatusDraft { Text = "for followers", Visibility = PostVisibility.FollowersOnly }, token);
var sent = (await _harness.Outgoing(relay.Id + "/inbox")).Where(a => a["type"]!.GetValue<string>() == "Create").ToList();
var create = Assert.Single(sent);
Assert.Equal(alice.PostUri(open.Post.ID), create["object"]!["id"]!.GetValue<string>());
}
[Fact]
public async Task A_post_a_relay_announces_is_its_authors_never_the_relays_boost()
{
var relay = await Subscribed();
var author = new RemoteActor(_harness.Peer, "author", _harness.Peer.B);
var noteId = Note(author, "announced by the relay");
await _harness.Deliver(relay, "/human-centipede", new JsonObject
{
["id"] = $"{relay.Id}/activities/{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = relay.Id,
["to"] = new JsonArray(relay.Id + "/followers"), ["object"] = noteId
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteSingleAsync(Token);
Assert.Equal(author.Id, post.ActorURI);
Assert.False(await DB.Default.Find<Post>().Match(p => p.ReblogOfPostId == post.ID).ExecuteAnyAsync(Token));
}
}
}
@@ -0,0 +1,38 @@
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
namespace PrivaPub.Tests.Federation
{
public class RemoteEditsTests
{
static readonly DateTime Published = new(2026, 10, 3, 10, 40, 1, DateTimeKind.Utc);
static Post Stored(DateTime? editedAt = default) => new() { CreationDate = Published, EditedAt = editedAt, ContentHtml = "<p>before</p>" };
[Fact]
public void A_newer_update_is_an_edit_and_an_update_without_a_date_is_a_refresh()
{
Assert.True(RemoteEdits.IsEdit(new NoteDocument { Updated = Published.AddSeconds(1), ContentHtml = "<p>before</p>" }, Stored()));
Assert.False(RemoteEdits.IsEdit(new NoteDocument { ContentHtml = "<p>after</p>" }, Stored()));
Assert.False(RemoteEdits.IsEdit(new NoteDocument { Updated = Published.AddSeconds(-1), ContentHtml = "<p>after</p>" }, Stored()));
}
[Fact]
public void An_edit_in_the_second_the_post_was_published_counts_when_the_text_changed()
{
Assert.True(RemoteEdits.IsEdit(new NoteDocument { Updated = Published, ContentHtml = "<p>after</p>" }, Stored()));
Assert.True(RemoteEdits.IsEdit(new NoteDocument { Updated = Published, ContentHtml = "<p>before</p>", SpoilerText = "now with a warning" }, Stored()));
Assert.False(RemoteEdits.IsEdit(new NoteDocument { Updated = Published, ContentHtml = "<p>before</p>" }, Stored()));
}
[Fact]
public void After_a_first_edit_only_a_newer_one_counts()
{
var edited = Published.AddMinutes(1);
Assert.False(RemoteEdits.IsEdit(new NoteDocument { Updated = edited, ContentHtml = "<p>again</p>" }, Stored(edited)));
Assert.True(RemoteEdits.IsEdit(new NoteDocument { Updated = edited.AddSeconds(1), ContentHtml = "<p>again</p>" }, Stored(edited)));
}
}
}
@@ -0,0 +1,117 @@
using Microsoft.Extensions.Caching.Memory;
using MongoDB.Entities;
using PrivaPub.Domain.Relationships;
using PrivaPub.Federation.Actors;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// an account's handle: user@host, or the domain its `webfinger` names (FEP-2c59; Mastodon's LOCAL_DOMAIN apart from
// its WEB_DOMAIN) once that domain's WebFinger says the handle is this account
[Trait("Category", "Integration")]
public sealed class RemoteHandleTests : IAsyncLifetime
{
Harness _harness;
RemoteActorService _remote;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
_remote = new RemoteActorService(Peer.Http(), _harness.Local, new MemoryCache(new MemoryCacheOptions()), _harness.Db,
new StaticOptions<FederationOptions>(new FederationOptions { AllowPrivateNetworks = true, AllowPlainHttp = true }));
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
// the actor at the peer's first address, its handle at the second
RemoteActor Actor(string name) => Serve(new RemoteActor(_harness.Peer, name));
RemoteActor Serve(RemoteActor actor, string preferredUsername = default)
{
var document = actor.Document();
if (preferredUsername != default)
document["preferredUsername"] = preferredUsername;
document["webfinger"] = $"acct:{preferredUsername ?? actor.Name}@{new Uri(_harness.Peer.B).Authority}";
_harness.Peer.Serve(new Uri(actor.Id).AbsolutePath, document.ToJsonString());
return actor;
}
void WebFinger(string self) => _harness.Peer.ServeText("/.well-known/webfinger", new JsonObject
{
["links"] = new JsonArray(new JsonObject { ["rel"] = "self", ["type"] = "application/activity+json", ["href"] = self })
}.ToJsonString(), "application/jrd+json");
[Fact]
public async Task A_handle_on_another_domain_is_kept_once_its_webfinger_agrees()
{
var actor = Actor("split");
WebFinger(actor.Id);
var stored = await _remote.GetActor(actor.Id, refresh: true, TestContext.Current.CancellationToken);
Assert.Equal(new Uri(_harness.Peer.B).Authority, stored.Domain);
}
[Fact]
public async Task A_handle_its_webfinger_does_not_confirm_stays_on_the_actors_host()
{
var actor = Actor("claims");
WebFinger("http://localhost/users/someone-else");
var stored = await _remote.GetActor(actor.Id, refresh: true, TestContext.Current.CancellationToken);
Assert.Equal(new Uri(actor.Id).Authority, stored.Domain);
}
[Fact]
public async Task A_renamed_account_is_asked_again()
{
var actor = Actor("renames");
WebFinger(actor.Id);
await _remote.GetActor(actor.Id, refresh: true, TestContext.Current.CancellationToken);
Serve(actor, "renamed");
WebFinger("http://localhost/users/someone-else");
// (another cache: the first fetch holds the actor back for a while)
var later = new RemoteActorService(Peer.Http(), _harness.Local, new MemoryCache(new MemoryCacheOptions()), _harness.Db,
new StaticOptions<FederationOptions>(new FederationOptions { AllowPrivateNetworks = true, AllowPlainHttp = true }));
var stored = await later.GetActor(actor.Id, refresh: true, TestContext.Current.CancellationToken);
Assert.Equal("renamed", stored.UserName);
Assert.Equal(new Uri(actor.Id).Authority, stored.Domain);
}
[Fact]
public async Task A_persona_blocking_the_handles_server_hides_the_account()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var split = Actor("split");
WebFinger(split.Id);
await _remote.GetActor(split.Id, refresh: true, token);
var claims = Actor("claims");
WebFinger("http://localhost/users/someone-else");
await new RemoteActorService(Peer.Http(), _harness.Local, new MemoryCache(new MemoryCacheOptions()), _harness.Db,
new StaticOptions<FederationOptions>(new FederationOptions { AllowPrivateNetworks = true, AllowPlainHttp = true }))
.GetActor(claims.Id, refresh: true, token);
await DB.Default.SaveAsync(new AccountDomainBlock { AvatarId = alice.Id, Domain = new Uri(_harness.Peer.B).Host }, token);
Assert.Equal([split.Id], await Hidden.AuthorsHiddenFrom(alice.Id, new[] { split.Id, claims.Id }, forNotifications: false, token));
Assert.Contains(alice.Id, await Hidden.RecipientsHiding(new[] { alice.Id }, split.Id, token));
Assert.Empty(await Hidden.RecipientsHiding(new[] { alice.Id }, claims.Id, token));
}
}
}
@@ -0,0 +1,114 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
// Inbox forwarding of replies (owner decision 2026-10-05): a reply to a persona's post reaches the persona's followers,
// who are not among those its author's server sends it to
[Trait("Category", "Integration")]
public sealed class ReplyRelayTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
// alice's post; bob replies from his server, fan follows alice from another, pal from bob's
async Task<(PrivaPub.Federation.Actors.LocalActor Alice, Post Post, RemoteActor Bob, RemoteActor Fan, RemoteActor Pal)> Thread()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob");
var fan = new RemoteActor(_harness.Peer, "fan", _harness.Peer.B);
var pal = new RemoteActor(_harness.Peer, "pal");
await _harness.FollowedBy(alice, fan);
await _harness.FollowedBy(alice, pal);
var post = (await _harness.Statuses.Publish(alice, new StatusDraft { Text = "what do you think?" }, token)).Post;
return (alice, post, bob, fan, pal);
}
JsonObject Reply(RemoteActor bob, Post post, string visibility = "public")
{
var reply = PublicNote(bob, "<p>I think so</p>", post.ActorURI);
reply["inReplyTo"] = post.ObjectURI;
if (visibility == "followers")
{
reply["to"] = new JsonArray(bob.Id + "/followers");
reply["cc"] = new JsonArray(post.ActorURI);
}
return reply;
}
Task<List<JsonObject>> At(RemoteActor follower) => _harness.Outgoing(follower.SharedInbox);
[Fact]
public async Task A_public_reply_to_a_personas_post_reaches_its_followers_as_it_came_and_not_its_authors_server()
{
var (_, post, bob, fan, pal) = await Thread();
var create = Create(bob, Reply(bob, post));
create["signature"] = new JsonObject { ["type"] = "RsaSignature2017", ["signatureValue"] = "kept as it came" };
await _harness.Deliver(bob, "/human-centipede", create);
var relayed = Assert.Single(await At(fan), a => a["id"]!.GetValue<string>() == IdOf(create));
Assert.Equal(bob.Id, relayed["actor"]!.GetValue<string>());
Assert.Equal("kept as it came", relayed["signature"]!["signatureValue"]!.GetValue<string>());
Assert.DoesNotContain(await At(pal), a => a["id"]!.GetValue<string>() == IdOf(create));
}
[Fact]
public async Task Its_edit_and_deletion_follow_it_and_a_followers_only_reply_stays_where_it_was_sent()
{
var (_, post, bob, fan, _) = await Thread();
var reply = Reply(bob, post);
await _harness.Deliver(bob, "/human-centipede", Create(bob, reply));
var edited = (JsonObject)reply.DeepClone();
edited["content"] = "<p>I think so, on second thought</p>";
edited["updated"] = DateTime.UtcNow.AddMinutes(1).ToString("O");
var update = Activity(bob, "Update", edited);
await _harness.Deliver(bob, "/human-centipede", update);
var delete = Activity(bob, "Delete", JsonValue.Create(IdOf(reply))!);
await _harness.Deliver(bob, "/human-centipede", delete);
var relayed = (await At(fan)).Select(a => a["id"]!.GetValue<string>()).ToList();
Assert.Contains(IdOf(update), relayed);
Assert.Contains(IdOf(delete), relayed);
var quiet = Create(bob, Reply(bob, post, "followers"));
await _harness.Deliver(bob, "/human-centipede", quiet);
Assert.DoesNotContain(await At(fan), a => a["id"]!.GetValue<string>() == IdOf(quiet));
}
[Fact]
public async Task A_reply_that_reached_us_forwarded_is_not_passed_on_again()
{
var (_, post, bob, fan, _) = await Thread();
var other = new RemoteActor(_harness.Peer, "other", _harness.Peer.B);
var reply = Reply(bob, post);
_harness.Peer.Serve(new Uri(IdOf(reply)).AbsolutePath, reply.ToJsonString());
var create = Create(bob, reply);
var result = await _harness.Deliver(other, "/human-centipede", create);
Assert.Equal("forwarded", result.Reason);
Assert.True(await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(reply)).ExecuteAnyAsync(TestContext.Current.CancellationToken));
Assert.DoesNotContain(await At(fan), a => a["id"]!.GetValue<string>() == IdOf(create));
}
}
}
@@ -0,0 +1,64 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.FederatedSeeds;
namespace PrivaPub.Tests.Federation
{
// A deleted remote post is never brought back by its own Create, however late it comes; an id its server gives a new
// post afterwards (Gancio numbers an event after the last one it keeps) is that new post
[Trait("Category", "Integration")]
public sealed class ReusedIdTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task A_deleted_posts_create_stays_dropped_and_a_new_post_under_its_id_is_kept()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var carol = new RemoteActor(_harness.Peer, "carol");
await Follows(alice.Id, carol);
var first = PublicNote(carol, "<p>the first event</p>");
first["published"] = DateTime.UtcNow.AddMinutes(-10).ToString("O");
await _harness.Deliver(carol, "/human-centipede", Create(carol, first));
await _harness.Deliver(carol, "/human-centipede", new JsonObject
{
["id"] = NewId(carol, "deletes"), ["type"] = "Delete", ["actor"] = carol.Id,
["object"] = new JsonObject { ["id"] = IdOf(first), ["type"] = "Tombstone" }
});
Assert.True(await DB.Default.Find<DeletedObject>().Match(d => d.ObjectURI == IdOf(first)).ExecuteAnyAsync(token));
// the first Create again, late
await _harness.Deliver(carol, "/human-centipede", Create(carol, (JsonObject)first.DeepClone()));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(first) && !p.DeletedAt.HasValue).ExecuteAnyAsync(token));
var second = PublicNote(carol, "<p>a new event, numbered as the first</p>");
second["id"] = IdOf(first);
second["published"] = DateTime.UtcNow.ToString("O");
await _harness.Deliver(carol, "/human-centipede", Create(carol, second));
var kept = await DB.Default.Find<Post>().Match(p => p.ObjectURI == IdOf(first) && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
Assert.Contains("a new event", kept?.ContentHtml);
Assert.False(await DB.Default.Find<DeletedObject>().Match(d => d.ObjectURI == IdOf(first)).ExecuteAnyAsync(token));
}
}
}
@@ -0,0 +1,252 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Signing;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// Lemmy takes a report only from a person or a service, about one post or comment, addressed to its community (owner
// decision 2026-10-06): a report about a post in a community on a Lemmy leaves from the server's reporter, one per post.
// Alone in its collection: the server rows it writes for the peer's hosts decide for whoever reports there.
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class ServiceReportTests : IAsyncLifetime
{
static readonly string[] PeerHosts = { "localhost", "127.0.0.1" };
// (a test sees only the deliveries queued since it began: the others' share the database)
readonly DateTime _started = DateTime.UtcNow;
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
}
public async ValueTask DisposeAsync()
{
if (_harness == default)
return;
await DB.Default.DeleteAsync<RemoteInstance>(i => PeerHosts.Contains(i.Host));
await _harness.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
// what NodeInfo said the server at that host runs
static async Task Runs(string host, string software)
{
await DB.Default.DeleteAsync<RemoteInstance>(i => i.Host == host);
await DB.Default.SaveAsync(new RemoteInstance { Host = host, Software = software, SoftwareVersion = "1.0.0" }, Token);
}
async Task<ForeignAvatar> Known(RemoteActor actor) => await _harness.Remote.GetActor(actor.Id, refresh: false, Token);
static async Task<Post> Held(RemoteActor author, string community = default, Post parent = default)
{
var post = new Post
{
ObjectURI = $"{Origin(author)}/post/{Guid.NewGuid():N}",
ActorURI = author.Id,
AudienceURI = community,
AnsweringToPostId = parent?.ID,
ContentHtml = "<p>reported</p>"
};
await DB.Default.SaveAsync(post, Token);
return post;
}
async Task<List<DeliveryPayload>> Queued() =>
(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.Deliver && j.CreatedAt >= _started).ExecuteAsync(Token))
.Select(j => JsonSerializer.Deserialize<DeliveryPayload>(j.Payload))
.ToList();
// a community on a Lemmy, or on a server that takes reports as Lemmy does (the peer as localhost), and one of its posters there
async Task<(RemoteActor Community, RemoteActor Poster)> OnLemmy(string software = "lemmy")
{
var community = new RemoteActor(_harness.Peer, "cats", _harness.Peer.B, type: "Group", sharedInbox: true);
var poster = new RemoteActor(_harness.Peer, "poster", _harness.Peer.B);
await Known(community);
await Known(poster);
await Runs("localhost", software);
return (community, poster);
}
// PieFed and Mbin take it too, proven live (2026-10-06); NodeInfo names Lemmy as "lemmy" and kbin's heir as "mbin"
[Theory]
[InlineData("lemmy")]
[InlineData("Lemmy")]
[InlineData("piefed")]
[InlineData("mbin")]
public async Task A_post_in_a_lemmy_community_is_reported_to_its_community_by_the_reporter_and_nowhere_else(string software)
{
var (_, alice) = await _harness.Persona("alice");
var (community, poster) = await OnLemmy(software);
var post = await Held(poster, community.Id);
var reporter = await _harness.Local.GetReporterActor(Token);
var report = await _harness.Reports.File(alice, (await Known(poster)).ID, new[] { post.ID }, " a slur in the title ", "violation", forward: true, Token);
Assert.True(report.Forwarded);
var flag = Assert.Single(await _harness.Outgoing(community.SharedInbox));
Assert.Equal("Flag", flag["type"]!.GetValue<string>());
Assert.Equal(reporter.Uri, flag["actor"]!.GetValue<string>());
Assert.EndsWith("/peasants/privapub_reports", reporter.Uri);
Assert.StartsWith(reporter.Uri + "/", flag["id"]!.GetValue<string>());
Assert.Equal(new[] { community.Id }, flag["to"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.Equal(community.Id, flag["audience"]!.GetValue<string>());
Assert.Equal(post.ObjectURI, flag["object"]!.GetValue<string>());
Assert.Equal("a slur in the title", flag["summary"]!.GetValue<string>());
Assert.Equal("a slur in the title", flag["content"]!.GetValue<string>());
Assert.Empty(await _harness.Outgoing(poster.Id + "/inbox"));
Assert.DoesNotContain(await Queued(), q => q.Inbox != community.SharedInbox && q.Body.Contains(post.ObjectURI));
Assert.DoesNotContain(alice.UserName, flag.ToJsonString());
Assert.DoesNotContain(alice.Id, flag.ToJsonString());
var queued = Assert.Single(await Queued(), q => q.Inbox == community.SharedInbox);
Assert.Equal(LocalActorKind.Reporter, queued.SignerKind);
Assert.Equal(reporter.Id, queued.SignerId);
}
[Fact]
public async Task The_reporter_signs_its_flag_with_its_own_key()
{
var (_, alice) = await _harness.Persona("alice");
var (community, poster) = await OnLemmy();
var post = await Held(poster, community.Id);
var reporter = await _harness.Local.GetReporterActor(Token);
var instance = await _harness.Local.GetInstanceActor(Token);
_harness.Peer.Answer("/inbox", 202);
await _harness.Reports.File(alice, (await Known(poster)).ID, new[] { post.ID }, default, "spam", forward: true, Token);
var job = await DB.Default.Find<Job>()
.Match(j => j.Kind == JobKind.Deliver && j.DedupeKey.StartsWith(reporter.ActivityUri($"flag-")))
.Sort(j => j.CreatedAt, Order.Descending).ExecuteFirstAsync(Token);
var handler = new DeliveryJobHandler(_harness.Local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())),
NullLogger<DeliveryJobHandler>.Instance);
var outcome = await handler.Handle(job, Token);
Assert.Equal(JobResult.Done, outcome.Result);
var received = Assert.Single(_harness.Peer.Requests, r => r.Path == "/inbox" && r.Method == "POST");
// no words: Lemmy takes no report without a reason, so the category is it
Assert.Equal("spam", JsonNode.Parse(received.Body)!["summary"]!.GetValue<string>());
var signature = HttpSignatures.Parse(received.Signature);
Assert.Equal(reporter.KeyId, signature.KeyId);
Assert.NotEqual(instance.PublicKeyPem, reporter.PublicKeyPem);
var signingString = $"(request-target): post /inbox\nhost: {received.Headers["Host"]}\ndate: {received.Headers["Date"]}\ndigest: {received.Headers["Digest"]}";
using var key = RSA.Create();
key.ImportFromPem(reporter.PublicKeyPem);
Assert.True(key.VerifyData(Encoding.UTF8.GetBytes(signingString), signature.Signature, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
}
[Fact]
public async Task A_comment_finds_its_community_through_its_thread_and_two_posts_are_two_flags()
{
var (_, alice) = await _harness.Persona("alice");
var (community, poster) = await OnLemmy();
var thread = await Held(poster, community.Id);
var comment = await Held(poster, parent: thread);
var answer = await Held(poster, parent: comment);
var report = await _harness.Reports.File(alice, (await Known(poster)).ID, new[] { comment.ID, answer.ID }, "harassment", "other", forward: true, Token);
Assert.True(report.Forwarded);
var flags = await _harness.Outgoing(community.SharedInbox);
Assert.Equal(2, flags.Count);
Assert.Equal(new[] { comment.ObjectURI, answer.ObjectURI }.Order(), flags.Select(f => f["object"]!.GetValue<string>()).Order());
Assert.All(flags, f => Assert.Equal(community.Id, f["to"]![0]!.GetValue<string>()));
Assert.Equal(2, flags.Select(f => f["id"]!.GetValue<string>()).Distinct().Count());
}
[Fact]
public async Task An_account_alone_on_a_lemmy_is_reported_to_nobody()
{
var (_, alice) = await _harness.Persona("alice");
var (community, poster) = await OnLemmy();
var report = await _harness.Reports.File(alice, (await Known(poster)).ID, Array.Empty<string>(), "a spammer", "spam", forward: true, Token);
Assert.False(report.Forwarded);
Assert.False((await DB.Default.Find<PrivaPub.Models.Social.Report>().OneAsync(report.ID, Token)).Forwarded);
Assert.Empty(await _harness.Outgoing(community.SharedInbox));
Assert.Empty(await _harness.Outgoing(poster.Id + "/inbox"));
}
[Fact]
public async Task Any_other_server_still_gets_the_instance_flag_and_a_lemmy_community_its_own()
{
var (_, alice) = await _harness.Persona("alice");
var (community, _) = await OnLemmy();
// an account on a Mastodon (the peer as 127.0.0.1) that wrote in the community on the Lemmy, and elsewhere
var mastodonian = new RemoteActor(_harness.Peer, "masto");
await Runs("127.0.0.1", "mastodon");
var inCommunity = await Held(mastodonian, community.Id);
var elsewhere = await Held(mastodonian);
var instance = await _harness.Local.GetInstanceActor(Token);
var report = await _harness.Reports.File(alice, (await Known(mastodonian)).ID, new[] { inCommunity.ID, elsewhere.ID }, "spam", "spam", forward: true, Token);
Assert.True(report.Forwarded);
var toCommunity = Assert.Single(await _harness.Outgoing(community.SharedInbox));
Assert.Equal(inCommunity.ObjectURI, toCommunity["object"]!.GetValue<string>());
var toAuthor = Assert.Single(await _harness.Outgoing(mastodonian.Id + "/inbox"));
Assert.Equal(instance.Uri, toAuthor["actor"]!.GetValue<string>());
Assert.Equal(new[] { mastodonian.Id, inCommunity.ObjectURI, elsewhere.ObjectURI }.Order(),
toAuthor["object"]!.AsArray().Select(o => o!.GetValue<string>()).Order());
Assert.Null(toAuthor["to"]);
}
[Fact]
public async Task A_community_on_a_server_that_takes_no_service_reports_gets_nothing_new()
{
var (_, alice) = await _harness.Persona("alice");
var community = new RemoteActor(_harness.Peer, "forum", _harness.Peer.B, type: "Group", sharedInbox: true);
var poster = new RemoteActor(_harness.Peer, "poster", _harness.Peer.B);
await Known(community);
var foreign = await Known(poster);
await Runs("localhost", "friendica");
var post = await Held(poster, community.Id);
var report = await _harness.Reports.File(alice, foreign.ID, new[] { post.ID }, "spam", "spam", forward: true, Token);
Assert.True(report.Forwarded);
Assert.Empty(await _harness.Outgoing(community.SharedInbox));
var flag = Assert.Single(await _harness.Outgoing(poster.Id + "/inbox"));
Assert.EndsWith("/peasants/privapub", flag["actor"]!.GetValue<string>());
}
[Fact]
public async Task Nobody_follows_the_reporter()
{
var follower = new RemoteActor(_harness.Peer, "follower");
var reporter = await _harness.Local.GetReporterActor(Token);
var result = await _harness.Deliver(follower, "/human-centipede", new JsonObject
{
["id"] = $"{Origin(follower)}/follow/{Guid.NewGuid():N}", ["type"] = "Follow", ["actor"] = follower.Id, ["object"] = reporter.Uri
});
Assert.Equal(404, result.StatusCode);
Assert.False(await DB.Default.Find<Follower>().Match(f => f.ActorURI == follower.Id).ExecuteAnyAsync(Token));
}
}
}
@@ -0,0 +1,50 @@
using PrivaPub.Domain.Statuses;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
// one delivery a server: a reply to a follower's post goes to its server's shared inbox once, not also to the
// follower's own inbox (BookWyrm kept two copies of a post that reached both at once)
[Trait("Category", "Integration")]
public sealed class SharedInboxTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
[Fact]
public async Task A_reply_to_a_followers_post_reaches_its_server_once_through_the_shared_inbox()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var bob = new RemoteActor(_harness.Peer, "bob", sharedInbox: true);
await _harness.FollowedBy(alice, bob);
var path = $"/notes/{Guid.NewGuid():N}";
var noteId = new Uri(bob.Id).GetLeftPart(UriPartial.Authority) + path;
_harness.Peer.Serve(path, new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = bob.Id, ["content"] = "<p>a question</p>",
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"), ["published"] = DateTime.UtcNow.ToString("O")
}.ToJsonString());
var parent = await _harness.RemotePosts.StoreContext(noteId, 0, token);
var reply = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "an answer", InReplyTo = parent.ID }, token);
Assert.True(reply.Ok, reply.Error);
Assert.Single(await _harness.Outgoing(bob.SharedInbox), a => a["type"]!.GetValue<string>() == "Create");
Assert.Empty(await _harness.Outgoing(bob.Id + "/inbox"));
}
}
}
+213
View File
@@ -0,0 +1,213 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.Host.FederationHelpers;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Federation
{
// walls (FEP-400e, Smithereen's; owner decision 2026-10-06): a persona's followers write on its wall, PrivaPub hosts what
// they write and tells the persona's followers with Add{Note}; the persona takes it off with Remove; an account elsewhere
// that tells of a post on its own wall has it shown to its followers here
[Trait("Category", "Integration")]
public sealed class WallTests : IAsyncLifetime
{
const string Public = "https://www.w3.org/ns/activitystreams#Public";
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
// a public note by author, written on the wall of owner (whose actor and wall are given), served at its id
string WallNote(RemoteActor author, string ownerUri, string wall, string text, string inReplyTo = default, string to = Public)
{
var path = $"/notes/{Guid.NewGuid():N}";
var origin = new Uri(author.Id).GetLeftPart(UriPartial.Authority);
var note = new JsonObject
{
["id"] = origin + path, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"<p>{text}</p>",
["to"] = new JsonArray(to, ownerUri), ["published"] = DateTime.UtcNow.ToString("O"),
["target"] = new JsonObject { ["type"] = "Collection", ["id"] = wall, ["attributedTo"] = ownerUri }
};
if (inReplyTo != default)
note["inReplyTo"] = inReplyTo;
_harness.Peer.Serve(path, note.ToJsonString());
_notes[origin + path] = note;
return origin + path;
}
readonly Dictionary<string, JsonObject> _notes = new();
// as Smithereen sends it: the whole note, to the wall's owner
JsonObject Create(RemoteActor author, string noteId) => new()
{
["id"] = noteId + "/activityCreate", ["type"] = "Create", ["actor"] = author.Id, ["to"] = _notes[noteId]["to"]!.DeepClone(),
["object"] = _notes[noteId].DeepClone()
};
Task<PostEntity> Post(string objectUri) => DB.Default.Find<PostEntity>().Match(p => p.ObjectURI == objectUri).ExecuteFirstAsync(Token);
[Fact]
public void A_persona_names_its_wall_and_lets_its_followers_write_on_it()
{
var actor = ActivityPubRenderer.Actor(new LocalActor { Id = "a", UserName = "alice", BaseAddress = Harness.Base, Kind = PrivaPub.Models.Federation.LocalActorKind.Person });
var circle = ActivityPubRenderer.Actor(new LocalActor { Id = "c", UserName = "circle", BaseAddress = Harness.Base, Kind = PrivaPub.Models.Federation.LocalActorKind.Group, IsCircle = true });
Assert.Equal($"{Harness.Base}/peasants/alice/graffiti", actor["wall"]!.GetValue<string>());
Assert.Equal($"{Harness.Base}/peasants/alice/groupies", actor["privacySettings"]!["wallPosting"]!["allowedTo"]![0]!.GetValue<string>());
Assert.Equal(Public, actor["privacySettings"]!["wallPostVisibility"]!["allowedTo"]![0]!.GetValue<string>());
Assert.Contains(actor["@context"]!.AsArray(), c => c is JsonObject terms && terms["wall"]?["@id"]?.GetValue<string>() == "sm:wall");
Assert.Null(circle["wall"]);
}
[Fact]
public async Task A_followers_post_on_the_wall_is_hosted_shown_and_told_to_the_personas_followers()
{
var (_, alice) = await _harness.Persona("alice");
var smuser = new RemoteActor(_harness.Peer, "smuser", wall: true);
await _harness.FollowedBy(alice, smuser);
var noteId = WallNote(smuser, alice.Uri, alice.Wall, "on alice's wall");
await _harness.Deliver(smuser, "/human-centipede", Create(smuser, noteId));
var post = await Post(noteId);
Assert.NotNull(post);
Assert.Equal((alice.Wall, alice.Uri, alice.Id), (post.WallURI, post.WallOwnerURI, post.WallOwnerAccountId));
Assert.True(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id && e.PostId == post.ID).ExecuteAnyAsync(Token));
Assert.True(await DB.Default.Find<Notification>().Match(n => n.AvatarId == alice.Id && n.PostId == post.ID && n.Type == NotificationType.Mention).ExecuteAnyAsync(Token));
var add = Assert.Single(await _harness.Outgoing(_harness.Peer.A + "/inbox"), a => a["type"]!.GetValue<string>() == "Add");
Assert.Equal((alice.Uri, noteId, alice.Wall), (add["actor"]!.GetValue<string>(), add["object"]!.GetValue<string>(), add["target"]!["id"]!.GetValue<string>()));
Assert.Contains(await _harness.Outgoing(smuser.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Add");
Assert.True(await _harness.Walls.Remove(alice, post, Token));
Assert.Null(await Post(noteId));
var remove = Assert.Single(await _harness.Outgoing(_harness.Peer.A + "/inbox"), a => a["type"]!.GetValue<string>() == "Remove");
Assert.Equal((noteId, alice.Wall), (remove["object"]!.GetValue<string>(), remove["target"]!.GetValue<string>()));
}
[Fact]
public async Task A_wall_post_from_a_stranger_a_reply_or_a_quiet_one_is_refused()
{
var (_, alice) = await _harness.Persona("alice");
var stranger = new RemoteActor(_harness.Peer, "stranger", wall: true);
var follower = new RemoteActor(_harness.Peer, "follower", wall: true);
await _harness.FollowedBy(alice, follower);
var fromStranger = WallNote(stranger, alice.Uri, alice.Wall, "from a stranger");
var reply = WallNote(follower, alice.Uri, alice.Wall, "a reply", inReplyTo: "https://elsewhere.example/notes/1");
var quiet = WallNote(follower, alice.Uri, alice.Wall, "quietly", to: follower.Id + "/followers");
await _harness.Deliver(stranger, "/human-centipede", Create(stranger, fromStranger));
await _harness.Deliver(follower, "/human-centipede", Create(follower, reply));
await _harness.Deliver(follower, "/human-centipede", Create(follower, quiet));
Assert.Null(await Post(fromStranger));
Assert.Null(await Post(reply));
Assert.Null(await Post(quiet));
Assert.DoesNotContain(await _harness.Outgoing(_harness.Peer.A + "/inbox"), a => a["type"]!.GetValue<string>() == "Add");
}
[Fact]
public async Task A_post_on_a_followed_accounts_wall_elsewhere_reaches_its_followers_here_until_it_is_taken_off()
{
var (_, alice) = await _harness.Persona("alice");
var owner = new RemoteActor(_harness.Peer, "owner", wall: true);
var writer = new RemoteActor(_harness.Peer, "writer");
await DB.Default.SaveAsync(new Following { AvatarId = alice.Id, TargetActorURI = owner.Id, TargetInboxURL = owner.Id + "/inbox", State = FollowState.Accepted }, Token);
var noteId = WallNote(writer, owner.Id, owner.Wall, "on the owner's wall");
// (kept before PrivaPub read walls: its document is read again when it names one)
await DB.Default.SaveAsync(new PrivaPub.Models.User.ForeignAvatar
{
ActorURI = owner.Id, UserName = owner.Name, PublicKeyId = owner.KeyId, PublicKey = owner.PublicKeyPem, InboxURL = owner.Id + "/inbox",
FollowersURL = owner.Id + "/followers", UpdatedAt = DateTime.UtcNow.AddDays(-2)
}, Token);
await _harness.Deliver(owner, "/human-centipede", new JsonObject
{
["id"] = $"{owner.Id}/posts/{Guid.NewGuid():N}/activityAdd", ["type"] = "Add", ["actor"] = owner.Id, ["object"] = noteId,
["target"] = new JsonObject { ["type"] = "Collection", ["id"] = owner.Wall, ["attributedTo"] = owner.Id },
["to"] = new JsonArray(Public, owner.Id + "/followers")
});
var post = await Post(noteId);
Assert.NotNull(post);
Assert.Equal((writer.Id, owner.Id), (post.ActorURI, post.WallOwnerURI));
Assert.True(await DB.Default.Find<TimelineEntry>().Match(e => e.AvatarId == alice.Id && e.PostId == post.ID).ExecuteAnyAsync(Token));
Assert.False(await DB.Default.Find<RemoteFeatured>().Match(f => f.ObjectURI == noteId).ExecuteAnyAsync(Token));
await _harness.Deliver(owner, "/human-centipede", new JsonObject
{
["id"] = $"{owner.Id}/posts/{Guid.NewGuid():N}/activityRemove", ["type"] = "Remove", ["actor"] = owner.Id, ["object"] = noteId, ["target"] = owner.Wall
});
Assert.Null(await Post(noteId));
}
}
[Trait("Category", "Integration")]
public sealed class WallCollectionTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
}
public ValueTask DisposeAsync()
{
_client?.Dispose();
return ValueTask.CompletedTask;
}
[Fact]
public async Task The_wall_lists_the_personas_public_posts_and_what_others_wrote_on_it()
{
var token = TestContext.Current.CancellationToken;
var persona = await _host.Persona(await _host.SignUp(), "wall");
var open = await _host.Publish(persona, "for everyone", PostVisibility.Public);
var quiet = await _host.Publish(persona, "for followers", PostVisibility.FollowersOnly);
var wall = persona.ActorUri() + "/graffiti";
var written = new PostEntity
{
ObjectURI = $"https://elsewhere.example/notes/{Guid.NewGuid():N}", ActorURI = "https://elsewhere.example/users/writer", IsFederatedCopy = true,
Visibility = PostVisibility.Public, WallURI = wall, WallOwnerURI = persona.ActorUri(), WallOwnerAccountId = persona.Id
};
await DB.Default.SaveAsync(written, token);
var collection = await _client.Fetch($"/peasants/{persona.UserName}/graffiti");
var page = await _client.Fetch($"/peasants/{persona.UserName}/graffiti?page=true");
Assert.Equal(HttpStatusCode.OK, collection.Status);
Assert.Equal((wall, 2), (collection.Json["id"]!.GetValue<string>(), collection.Json["totalItems"]!.GetValue<int>()));
var items = page.Json["orderedItems"]!.AsArray().Select(i => i!.GetValue<string>()).ToList();
Assert.Equal(new[] { written.ObjectURI, persona.ActorUri() + "/scribbles/" + open.ID }, items);
Assert.DoesNotContain(items, i => i.EndsWith(quiet.ID));
}
}
}
@@ -0,0 +1 @@
{"attributedTo":"https://gts.test/users/ines_gt59","cc":"https://privapub.test/peasants/flor_pp98","content":"\u003Cp\u003E\u003Cspan class=\u0022h-card\u0022\u003E\u003Ca href=\u0022https://privapub.test/@flor_pp98\u0022 class=\u0022u-url mention\u0022 rel=\u0022nofollow noreferrer noopener\u0022 target=\u0022_blank\u0022\u003E@\u003Cspan\u003Eflor_pp98\u003C/span\u003E\u003C/a\u003E\u003C/span\u003E bicycle the window bicycle tomorrow festival (p111)\u003C/p\u003E","contentMap":{"en":"\u003Cp\u003E\u003Cspan class=\u0022h-card\u0022\u003E\u003Ca href=\u0022https://privapub.test/@flor_pp98\u0022 class=\u0022u-url mention\u0022 rel=\u0022nofollow noreferrer noopener\u0022 target=\u0022_blank\u0022\u003E@\u003Cspan\u003Eflor_pp98\u003C/span\u003E\u003C/a\u003E\u003C/span\u003E bicycle the window bicycle tomorrow festival (p111)\u003C/p\u003E"},"id":"https://gts.test/users/ines_gt59/statuses/01M44BKDVTWFAMAQ3YSBM4QETY","inReplyTo":"https://privapub.test/peasants/flor_pp98/scribbles/6ac2beaf092dc0af8ce6035e","interactionPolicy":{"canAnnounce":{"automaticApproval":["https://gts.test/users/ines_gt59"]},"canLike":{"automaticApproval":["https://gts.test/users/ines_gt59","https://gts.test/users/ines_gt59/followers","https://privapub.test/peasants/flor_pp98"]},"canQuote":{"automaticApproval":["https://gts.test/users/ines_gt59"]},"canReply":{"automaticApproval":["https://gts.test/users/ines_gt59","https://gts.test/users/ines_gt59/followers","https://privapub.test/peasants/flor_pp98"]}},"published":"2026-10-04T21:02:31Z","replies":{"first":{"id":"https://gts.test/users/ines_gt59/statuses/01M44BKDVTWFAMAQ3YSBM4QETY/replies?page=true","next":"https://gts.test/users/ines_gt59/statuses/01M44BKDVTWFAMAQ3YSBM4QETY/replies?page=true\u0026only_other_accounts=false","partOf":"https://gts.test/users/ines_gt59/statuses/01M44BKDVTWFAMAQ3YSBM4QETY/replies","type":"CollectionPage"},"id":"https://gts.test/users/ines_gt59/statuses/01M44BKDVTWFAMAQ3YSBM4QETY/replies","type":"Collection"},"tag":{"href":"https://privapub.test/peasants/flor_pp98","name":"@flor_pp98@privapub.test","type":"Mention"},"to":"https://gts.test/users/ines_gt59/followers","type":"Note","url":"https://gts.test/@ines_gt59/statuses/01M44BKDVTWFAMAQ3YSBM4QETY"}
@@ -0,0 +1,71 @@
{
"actor": "https://pleroma.test/users/flor_pl97",
"attachment": [],
"attributedTo": "https://pleroma.test/users/flor_pl97",
"cc": [
"https://pleroma.test/users/flor_pl97/followers"
],
"content": "mountain late window map bright postcard (p10) (edited)",
"context": "https://pleroma.test/contexts/891653c5-91ba-43f3-b3ef-b066caa3fa4d",
"conversation": "https://pleroma.test/contexts/891653c5-91ba-43f3-b3ef-b066caa3fa4d",
"formerRepresentations": {
"orderedItems": [
{
"actor": "https://pleroma.test/users/flor_pl97",
"attachment": [],
"attributedTo": "https://pleroma.test/users/flor_pl97",
"cc": [
"https://pleroma.test/users/flor_pl97/followers"
],
"content": "tomorrow library garden bridge the festival bicycle together garden early quiet morning library the postcard tomorrow mountain the small rain postcard warm garden market map mountain map (p10)",
"contentMap": {
"en": "tomorrow library garden bridge the festival bicycle together garden early quiet morning library the postcard tomorrow mountain the small rain postcard warm garden market map mountain map (p10)"
},
"context": "https://pleroma.test/contexts/891653c5-91ba-43f3-b3ef-b066caa3fa4d",
"conversation": "https://pleroma.test/contexts/891653c5-91ba-43f3-b3ef-b066caa3fa4d",
"published": "2026-10-05T00:49:58.860071Z",
"repliesCount": 2,
"source": {
"content": "tomorrow library garden bridge the festival bicycle together garden early quiet morning library the postcard tomorrow mountain the small rain postcard warm garden market map mountain map (p10)",
"mediaType": "text/plain"
},
"summary": "",
"tag": [],
"to": [
"https://www.w3.org/ns/activitystreams#Public"
],
"type": "Note"
}
],
"totalItems": 1,
"type": "OrderedCollection"
},
"id": "https://pleroma.test/objects/6101c4fd-ed46-461c-a3fe-9742852f47da",
"published": "2026-10-05T00:49:58.860071Z",
"replies": {
"first": {
"id": "https://pleroma.test/objects/6101c4fd-ed46-461c-a3fe-9742852f47da/replies?page=true",
"orderedItems": [
"https://pleroma.test/objects/b7b7c066-51a8-46d1-8919-07b87f158887",
"https://privapub.test/peasants/vera_pp40/scribbles/6ac2f4520c61eaf3ab241d35"
],
"partOf": "https://pleroma.test/objects/6101c4fd-ed46-461c-a3fe-9742852f47da/replies",
"type": "OrderedCollectionPage"
},
"id": "https://pleroma.test/objects/6101c4fd-ed46-461c-a3fe-9742852f47da/replies",
"totalItems": 2,
"type": "OrderedCollection"
},
"repliesCount": 2,
"source": {
"content": "mountain late window map bright postcard (p10) (edited)",
"mediaType": "text/plain"
},
"summary": "",
"tag": [],
"to": [
"https://www.w3.org/ns/activitystreams#Public"
],
"type": "Note",
"updated": "2026-10-05T00:52:40.105051Z"
}
+200
View File
@@ -0,0 +1,200 @@
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.Infrastructure.Backup;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Formats.Tar;
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// The administrator's page backs up, downloads, uploads and restores (owner decision 2026-10-07): only an
// administrator the database still says is one; a download and a restore ask for the password again, a restore for
// the host typed out; an upload resumes where it stopped, and one that would write outside its backup is refused.
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class BackupEndpointTests : IAsyncLifetime
{
const string Base = "/clientapi/admin/backups";
PrivaPubHost _host;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync()
{
RestoreMarker.Clear(_host?.Get<Backups>().Root ?? Path.GetTempPath());
return ValueTask.CompletedTask;
}
async Task<string> BackedUp(HttpClient admin)
{
var before = _host.Get<Backups>().List().Select(b => b.Id).ToHashSet();
Assert.Equal(HttpStatusCode.Accepted, (await admin.PostAsync(Base, null, Token)).StatusCode);
var made = default(string);
Assert.True(await Wait(async () =>
{
var list = (await admin.GetFromJsonAsync<JsonObject>(Base, Token))!;
made = list["backups"]!.AsArray().Select(b => b!["id"]!.GetValue<string>()).FirstOrDefault(id => !before.Contains(id));
return made != default && list["running"] is null;
}), "the backup was never made");
return made;
}
static async Task<bool> Wait(Func<Task<bool>> done)
{
for (var i = 0; i < 120; i++)
{
if (await done())
return true;
await Task.Delay(250, Token);
}
return false;
}
[Fact]
public async Task Only_an_administrator_the_database_still_knows()
{
var root = await _host.SignUp("plain");
using (var plain = _host.As(root.Jwt))
Assert.Equal(HttpStatusCode.Forbidden, (await plain.GetAsync(Base, Token)).StatusCode);
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
Assert.Equal(HttpStatusCode.OK, (await client.GetAsync(Base, Token)).StatusCode);
await DB.Default.Update<RootUser>().MatchID(admin.Id).Modify(u => u.Policies, [Policies.IsUser]).ExecuteAsync(Token);
Assert.Equal(HttpStatusCode.Forbidden, (await client.GetAsync(Base, Token)).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await client.PostAsync(Base, null, Token)).StatusCode);
}
[Fact]
public async Task A_backup_is_made_downloaded_whole_or_in_part_and_uploaded_again()
{
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
var id = await BackedUp(client);
var listed = (await client.GetFromJsonAsync<JsonObject>(Base, Token))!;
Assert.Equal("privapub.test", listed["host"]!.GetValue<string>());
Assert.Equal("manual", listed["backups"]!.AsArray().Single(b => b!["id"]!.GetValue<string>() == id)!["kind"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await client.PostAsJsonAsync($"{Base}/{id}/ticket", new { password = "wrong" }, Token)).StatusCode);
var ticket = (await (await client.PostAsJsonAsync($"{Base}/{id}/ticket", new { password = admin.Password }, Token)).Content.ReadFromJsonAsync<JsonObject>(Token))!;
var url = ticket["url"]!.GetValue<string>();
// a plain link: no token needed, the ticket is the credential
using var anonymous = _host.Client();
var whole = await anonymous.GetAsync(url, Token);
Assert.Equal(HttpStatusCode.OK, whole.StatusCode);
var bytes = await whole.Content.ReadAsByteArrayAsync(Token);
Assert.Equal(ticket["bytes"]!.GetValue<long>(), bytes.Length);
Assert.Equal(bytes.Length, whole.Content.Headers.ContentLength);
var names = new List<string>();
await using (var reader = new TarReader(new MemoryStream(bytes)))
while (await reader.GetNextEntryAsync(cancellationToken: Token) is { } entry)
names.Add(entry.Name);
Assert.Contains($"{id}/manifest.json", names);
Assert.Contains($"{id}/db/Avatar.jsonl.gz", names);
var part = new HttpRequestMessage(HttpMethod.Get, url);
part.Headers.Range = new RangeHeaderValue(700, 2747);
var partial = await anonymous.SendAsync(part, Token);
Assert.Equal(HttpStatusCode.PartialContent, partial.StatusCode);
Assert.Equal($"bytes 700-2747/{bytes.Length}", partial.Content.Headers.ContentRange!.ToString());
Assert.Equal(bytes.AsSpan(700, 2048).ToArray(), await partial.Content.ReadAsByteArrayAsync(Token));
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.GetAsync($"{Base}/download/{new string('0', 64)}", Token)).StatusCode);
// gone, then uploaded again in three pieces, one sent at the wrong place and refused with where to go on
Assert.Equal(HttpStatusCode.OK, (await client.DeleteAsync($"{Base}/{id}", Token)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await client.DeleteAsync($"{Base}/{id}", Token)).StatusCode);
var upload = (await (await client.PostAsync($"{Base}/uploads", null, Token)).Content.ReadFromJsonAsync<JsonObject>(Token))!["id"]!.GetValue<string>();
var third = bytes.Length / 3;
Assert.Equal(HttpStatusCode.OK, (await Piece(client, upload, 0, bytes[..third])).StatusCode);
var gap = await Piece(client, upload, third + 10, bytes[(third + 10)..]);
Assert.Equal(HttpStatusCode.Conflict, gap.StatusCode);
Assert.Equal(third, (await gap.Content.ReadFromJsonAsync<JsonObject>(Token))!["received"]!.GetValue<long>());
Assert.Equal(HttpStatusCode.OK, (await Piece(client, upload, third, bytes[third..(2 * third)])).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await Piece(client, upload, 2 * third, bytes[(2 * third)..])).StatusCode);
var finished = await client.PostAsync($"{Base}/uploads/{upload}/finish", null, Token);
Assert.Equal(HttpStatusCode.OK, finished.StatusCode);
Assert.Equal("uploaded", (await finished.Content.ReadFromJsonAsync<JsonObject>(Token))!["kind"]!.GetValue<string>());
Assert.Empty(await _host.Get<Backups>().Verify(id, Token));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await client.PostAsync($"{Base}/uploads/{upload}/finish", null, Token)).StatusCode);
_host.Get<Backups>().Delete(id);
}
Task<HttpResponseMessage> Piece(HttpClient client, string upload, long offset, byte[] bytes) =>
client.PutAsync($"{Base}/uploads/{upload}?offset={offset}", new ByteArrayContent(bytes), Token);
[Theory]
[InlineData("escape")]
[InlineData("link")]
[InlineData("elsewhere")]
[InlineData("junk")]
public async Task An_upload_that_is_not_a_plain_backup_is_refused_and_leaves_nothing(string how)
{
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
const string id = "20260101-000000-manual";
var tar = new MemoryStream();
if (how == "junk")
tar.Write("not a tar at all, not even close"u8);
else
await using (var writer = new TarWriter(tar, TarEntryFormat.Pax, leaveOpen: true))
{
await writer.WriteEntryAsync(new PaxTarEntry(TarEntryType.RegularFile, $"{id}/manifest.json") { DataStream = new MemoryStream("{}"u8.ToArray()) }, Token);
await writer.WriteEntryAsync(how switch
{
"escape" => new PaxTarEntry(TarEntryType.RegularFile, $"{id}/../../evil") { DataStream = new MemoryStream("x"u8.ToArray()) },
"link" => new PaxTarEntry(TarEntryType.SymbolicLink, $"{id}/media/x.jpg") { LinkName = "/etc/passwd" },
_ => new PaxTarEntry(TarEntryType.RegularFile, "20260101-000000-nightly/db/Post.jsonl.gz") { DataStream = new MemoryStream("x"u8.ToArray()) }
}, Token);
}
var upload = (await (await client.PostAsync($"{Base}/uploads", null, Token)).Content.ReadFromJsonAsync<JsonObject>(Token))!["id"]!.GetValue<string>();
Assert.Equal(HttpStatusCode.OK, (await Piece(client, upload, 0, tar.ToArray())).StatusCode);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await client.PostAsync($"{Base}/uploads/{upload}/finish", null, Token)).StatusCode);
var root = _host.Get<Backups>().Root;
Assert.False(Directory.Exists(Path.Combine(root, id)));
Assert.False(Directory.Exists(Path.Combine(root, id + ServerBackup.PartialSuffix)));
Assert.False(File.Exists(Path.Combine(root, "..", "evil")));
}
// asked for: written for the next start (the test host has no watcher to stop it, and the marker is taken away)
[Fact]
public async Task A_restore_asks_for_the_password_and_the_host_typed_out()
{
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
var id = await BackedUp(client);
var root = _host.Get<Backups>().Root;
try
{
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await client.PostAsJsonAsync($"{Base}/{id}/restore", new { password = "wrong", host = "privapub.test" }, Token)).StatusCode);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await client.PostAsJsonAsync($"{Base}/{id}/restore", new { password = admin.Password, host = "elsewhere.test" }, Token)).StatusCode);
Assert.Null(RestoreMarker.Read(root));
Assert.Equal(HttpStatusCode.Accepted, (await client.PostAsJsonAsync($"{Base}/{id}/restore", new { password = admin.Password, host = "PrivaPub.test" }, Token)).StatusCode);
var marker = RestoreMarker.Read(root);
Assert.Equal((id, "pending", admin.UserName), (marker.Backup, marker.State, marker.RequestedBy));
Assert.Equal(id, (await client.GetFromJsonAsync<JsonObject>(Base, Token))!["waiting"]!["backup"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.Conflict, (await client.DeleteAsync($"{Base}/{id}", Token)).StatusCode);
}
finally
{
RestoreMarker.Clear(root);
_host.Get<Backups>().Delete(id);
}
}
}
}
@@ -0,0 +1,492 @@
using Microsoft.Extensions.Configuration;
using Microsoft.IdentityModel.Tokens;
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.Models.Group;
using PrivaPub.Models.User;
using PrivaPub.Services;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.IdentityModel.Tokens.Jwt;
using System.Net;
using System.Security.Claims;
using System.Text;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class ClientApiAccountsTests : IAsyncLifetime
{
const string NewPassword = "Other-Pass-2!";
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static string Name(string prefix) => $"{prefix}{Guid.NewGuid():N}"[..20];
async Task<HttpResponseMessage> LogIn(string userName, string password)
{
using var client = _host.Client();
return await client.PostJson("/clientapi/user/login", new { userName, password });
}
async Task<HttpStatusCode> Settings(string jwt)
{
using var client = _host.As(jwt);
return (await client.GetAsync("/clientapi/user/settings", TestContext.Current.CancellationToken)).StatusCode;
}
static async Task<string> Message(HttpResponseMessage response)
{
var body = await response.JsonBody();
return body["errorMessage"]?.GetValue<string>() ?? body["title"]?.GetValue<string>();
}
string Jwt(string rootId, string userName, DateTime expires, string key = default)
{
var configuration = _host.Get<IConfiguration>();
var signing = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(key ?? configuration["AppConfiguration:Jwt:Key"]!));
var token = new JwtSecurityToken(configuration["AppConfiguration:Jwt:Issuer"], configuration["AppConfiguration:Jwt:Audience"],
new[] { new Claim(ClaimTypes.UserData, rootId), new Claim(ClaimTypes.Name, userName), new Claim(Policies.IsUser, "true") },
notBefore: expires.AddHours(-1), expires: expires, signingCredentials: new SigningCredentials(signing, SecurityAlgorithms.HmacSha512));
return new JwtSecurityTokenHandler().WriteToken(token);
}
[Fact]
public async Task Signing_up_answers_a_token_for_the_new_root()
{
var userName = Name("Signup");
using var client = _host.Client();
var response = await client.PostJson("/clientapi/user/signup", new { userName, password = Accounts.Password });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var jwt = await response.JsonBody();
Assert.Equal(userName.ToLowerInvariant(), jwt["username"]!.GetValue<string>());
Assert.Equal(new[] { Policies.IsUser }, jwt["policies"]!.AsArray().Select(p => p!.GetValue<string>()));
Assert.Equal(HttpStatusCode.OK, await Settings(jwt["token"]!.GetValue<string>()));
var stored = await DB.Default.Find<RootUser>().MatchID(jwt["userId"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
Assert.NotEqual(Accounts.Password, stored.HashedPassword);
}
[Fact]
public async Task A_taken_username_is_refused_whatever_its_case()
{
var root = await _host.SignUp("taken");
using var client = _host.Client();
var again = await client.PostJson("/clientapi/user/signup", new { userName = root.UserName, password = Accounts.Password });
var shouted = await client.PostJson("/clientapi/user/signup", new { userName = root.UserName.ToUpperInvariant(), password = Accounts.Password });
Assert.Equal(HttpStatusCode.BadRequest, again.StatusCode);
Assert.Contains("already taken", await Message(again));
Assert.Equal(HttpStatusCode.BadRequest, shouted.StatusCode);
Assert.Equal(1, await DB.Default.CountAsync<RootUser>(u => u.UserName == root.UserName, TestContext.Current.CancellationToken));
}
public static TheoryData<string> InvalidSignUps() => new()
{
"{}",
"{\"userName\":\"ab\",\"password\":\"Test-Pass-1!\"}",
"{\"userName\":\"has space\",\"password\":\"Test-Pass-1!\"}",
"{\"userName\":\"validname\",\"password\":\"short\"}",
"{\"userName\":\"validname\",\"password\":\"nouppercase1\"}",
"{\"userName\":\"validname\",\"password\":\"Has Space1\"}",
"{\"userName\":\"validname\",\"password\":\"Test-Pass-1!\",\"lightThemeIndexColour\":400}",
"{\"userName\":\"" + new string('a', 40) + "\",\"password\":\"Test-Pass-1!\"}",
"not json"
};
[Theory]
[MemberData(nameof(InvalidSignUps))]
public async Task An_invalid_sign_up_answers_400_with_a_message(string body)
{
using var client = _host.Client();
var response = await client.PostAsync("/clientapi/user/signup", new StringContent(body, Encoding.UTF8, "application/json"), TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.False(string.IsNullOrEmpty(await Message(response)));
}
[Fact]
public async Task Logging_in_and_out()
{
var root = await _host.SignUp("login");
var ok = await LogIn(root.UserName, root.Password);
var wrong = await LogIn(root.UserName, "Wrong-Pass-1!");
var unknown = await LogIn(Name("nobody"), root.Password);
using var signedIn = _host.As((await ok.JsonBody())["token"]!.GetValue<string>());
var logout = await signedIn.GetAsync("/clientapi/user/logout", TestContext.Current.CancellationToken);
using var anonymous = _host.Client();
var anonymousLogout = await anonymous.GetAsync("/clientapi/user/logout", TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, ok.StatusCode);
Assert.Equal(root.Id, (await ok.JsonBody())["userId"]!.GetValue<string>());
// a wrong password and an unknown login get the same answer, so sign-in tells nobody which logins exist
Assert.Equal(HttpStatusCode.BadRequest, wrong.StatusCode);
Assert.Equal(RootUsersService.NoMatch, await Message(wrong));
Assert.Equal(HttpStatusCode.BadRequest, unknown.StatusCode);
Assert.Equal(await Message(wrong), await Message(unknown));
Assert.Equal(HttpStatusCode.OK, logout.StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, anonymousLogout.StatusCode);
}
[Fact]
public async Task Update_sets_the_recovery_email_and_refuses_one_in_use()
{
var root = await _host.SignUp("email");
var other = await _host.SignUp("email");
var email = $"{Guid.NewGuid():N}@example.test";
using var client = _host.As(root.Jwt);
using var otherClient = _host.As(other.Jwt);
var set = await client.PostJson("/clientapi/user/update", new { email });
var taken = await otherClient.PostJson("/clientapi/user/update", new { email });
var invalid = await otherClient.PostJson("/clientapi/user/update", new { email = "not an email" });
Assert.Equal(HttpStatusCode.OK, set.StatusCode);
Assert.Equal(email, (await DB.Default.Find<RootUser>().MatchID(root.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Email);
Assert.Equal(HttpStatusCode.BadRequest, taken.StatusCode);
Assert.Contains("already taken", await Message(taken));
Assert.Equal(HttpStatusCode.BadRequest, invalid.StatusCode);
Assert.Null((await DB.Default.Find<RootUser>().MatchID(other.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Email);
}
[Fact]
public async Task Settings_round_trip()
{
var root = await _host.SignUp("settings");
using var client = _host.As(root.Jwt);
var updated = await client.PostJson("/clientapi/user/update/settings", new
{
languageCode = "it",
lightThemeIndexColour = 100,
darkThemeIndexColour = 200,
iconsThemeIndexColour = 50,
themeIsDarkMode = true,
themeIsDarkGray = true
});
var settings = await (await client.GetAsync("/clientapi/user/settings", TestContext.Current.CancellationToken)).JsonBody();
var unsupported = await client.PostJson("/clientapi/user/update/settings", new { languageCode = "xx" });
var outOfRange = await client.PostJson("/clientapi/user/update/settings", new { languageCode = "en", lightThemeIndexColour = 360 });
Assert.Equal(HttpStatusCode.OK, updated.StatusCode);
Assert.Equal("it", settings["languageCode"]!.GetValue<string>());
Assert.Equal(100, settings["lightThemeIndexColour"]!.GetValue<int>());
Assert.Equal(200, settings["darkThemeIndexColour"]!.GetValue<int>());
Assert.Equal(50, settings["iconsThemeIndexColour"]!.GetValue<int>());
Assert.True(settings["themeIsDarkMode"]!.GetValue<bool>());
Assert.True(settings["themeIsDarkGray"]!.GetValue<bool>());
Assert.Equal(HttpStatusCode.BadRequest, unsupported.StatusCode);
Assert.Contains("unsupported", await Message(unsupported));
Assert.Equal(HttpStatusCode.BadRequest, outOfRange.StatusCode);
Assert.Equal("it", (await DB.Default.Find<RootUser>().MatchID(root.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Settings.LanguageCode);
}
[Fact]
public async Task Changing_the_password_needs_the_old_one()
{
var root = await _host.SignUp("password");
using var client = _host.As(root.Jwt);
var wrongOld = await client.PostJson("/clientapi/user/update/password", new { oldPassword = "Wrong-Pass-1!", newPassword = NewPassword, repeatedPassword = NewPassword });
var mismatch = await client.PostJson("/clientapi/user/update/password", new { oldPassword = root.Password, newPassword = NewPassword, repeatedPassword = "Something-3!" });
var weak = await client.PostJson("/clientapi/user/update/password", new { oldPassword = root.Password, newPassword = "weak", repeatedPassword = "weak" });
var changed = await client.PostJson("/clientapi/user/update/password", new { oldPassword = root.Password, newPassword = NewPassword, repeatedPassword = NewPassword });
Assert.Equal(HttpStatusCode.BadRequest, wrongOld.StatusCode);
Assert.Equal("Wrong password.", await Message(wrongOld));
Assert.Equal(HttpStatusCode.BadRequest, mismatch.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, weak.StatusCode);
Assert.Equal(HttpStatusCode.OK, changed.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await LogIn(root.UserName, root.Password)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, NewPassword)).StatusCode);
}
[Fact]
public async Task An_invitation_signs_up_a_root_with_a_persona_in_the_group()
{
var owner = await _host.Persona(await _host.SignUp("owner"), "owner");
var group = await _host.Group(owner, community: false, password: "door-Pass-1");
var invitationCode = group["invitationCode"]!.GetValue<string>();
var userName = Name("invited");
var avatarUserName = Name("guest");
using var client = _host.Client();
var wrongPassword = await client.PostJson("/clientapi/user/invitation/signup", new { userName, password = Accounts.Password, invitationCode, invitationPassword = "nope", avatarUserName });
var wrongCode = await client.PostJson("/clientapi/user/invitation/signup", new { userName, password = Accounts.Password, invitationCode = $"{Guid.NewGuid():N}{Guid.NewGuid():N}", invitationPassword = "door-Pass-1", avatarUserName });
var response = await client.PostJson("/clientapi/user/invitation/signup", new { userName, password = Accounts.Password, invitationCode, invitationPassword = "door-Pass-1", avatarUserName, avatarName = "Guest" });
Assert.Equal(HttpStatusCode.NotAcceptable, wrongPassword.StatusCode);
Assert.Equal(HttpStatusCode.NotFound, wrongCode.StatusCode);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var jwt = await response.JsonBody();
using var invited = _host.As(jwt["token"]!.GetValue<string>());
var avatars = await (await invited.GetAsync("/clientapi/avatar/private/list", TestContext.Current.CancellationToken)).JsonItems();
var avatar = Assert.Single(avatars)!;
Assert.Equal(avatarUserName, avatar["userName"]!.GetValue<string>());
Assert.Equal("Guest", avatar["name"]!.GetValue<string>());
var stored = await DB.Default.Find<GroupEntity>().MatchID(group["id"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
Assert.Contains(stored.Members, m => !m.IsForeign && m.AvatarId == avatar["id"]!.GetValue<string>() && m.Role == GroupRole.Member);
}
[Fact]
public async Task An_invitation_never_names_the_persona_after_the_login()
{
var owner = await _host.Persona(await _host.SignUp("owner"), "owner");
var invitationCode = (await _host.Group(owner, community: true))["invitationCode"]!.GetValue<string>();
var userName = Name("same");
using var client = _host.Client();
var same = await client.PostJson("/clientapi/user/invitation/signup", new { userName, password = Accounts.Password, invitationCode, avatarUserName = userName });
var shouted = await client.PostJson("/clientapi/user/invitation/signup", new { userName = userName.ToUpperInvariant(), password = Accounts.Password, invitationCode, avatarUserName = userName });
var missing = await client.PostJson("/clientapi/user/invitation/signup", new { userName, password = Accounts.Password, invitationCode });
Assert.Equal(HttpStatusCode.BadRequest, same.StatusCode);
Assert.Equal("Your persona's username must differ from your login.", await Message(same));
Assert.Equal(HttpStatusCode.BadRequest, shouted.StatusCode);
Assert.Equal("Your persona's username must differ from your login.", await Message(shouted));
Assert.Equal(HttpStatusCode.BadRequest, missing.StatusCode);
Assert.False(await DB.Default.Find<RootUser>().Match(u => u.UserName == userName).ExecuteAnyAsync(TestContext.Current.CancellationToken));
Assert.False(await DB.Default.Find<Avatar>().Match(a => a.UserName == userName).ExecuteAnyAsync(TestContext.Current.CancellationToken));
}
[Fact]
public async Task An_existing_root_joins_through_an_invitation_with_a_new_persona()
{
var owner = await _host.Persona(await _host.SignUp("owner"), "owner");
var group = await _host.Group(owner, community: true);
var root = await _host.SignUp("joiner");
var avatarUserName = Name("joined");
using var client = _host.Client();
var response = await client.PostJson("/clientapi/user/invitation/login", new
{
userName = root.UserName,
password = root.Password,
invitationCode = group["invitationCode"]!.GetValue<string>(),
avatarUserName
});
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
using var joined = _host.As(root.Jwt);
var avatar = Assert.Single(await (await joined.GetAsync("/clientapi/avatar/private/list", TestContext.Current.CancellationToken)).JsonItems())!;
Assert.Equal(avatarUserName, avatar["userName"]!.GetValue<string>());
var groups = await (await joined.GetAsync("/clientapi/group/list?avatarId=" + avatar["id"]!.GetValue<string>(), TestContext.Current.CancellationToken)).JsonItems();
Assert.Equal(group["id"]!.GetValue<string>(), Assert.Single(groups)!["id"]!.GetValue<string>());
}
[Fact]
public async Task Recovery_answers_the_same_whoever_asks_and_queues_the_work()
{
var token = TestContext.Current.CancellationToken;
var since = DateTime.UtcNow.AddSeconds(-1);
var withEmail = await _host.SignUp("withemail");
var withoutEmail = await _host.SignUp("noemail");
var email = $"{Guid.NewGuid():N}@example.test";
using (var signedIn = _host.As(withEmail.Jwt))
Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode);
using var client = _host.Client();
var answers = new[]
{
await client.PostJson("/clientapi/user/recover/password", new { userName = withEmail.UserName }),
await client.PostJson("/clientapi/user/recover/password", new { email }),
await client.PostJson("/clientapi/user/recover/password", new { userName = withoutEmail.UserName }),
await client.PostJson("/clientapi/user/recover/password", new { userName = Name("nobody") }),
await client.PostJson("/clientapi/user/recover/password", new { email = $"{Guid.NewGuid():N}@example.test" })
};
var bodies = new List<string>();
foreach (var answer in answers)
{
Assert.Equal(HttpStatusCode.OK, answer.StatusCode);
bodies.Add(await answer.Content.ReadAsStringAsync(token));
}
Assert.Single(bodies.Distinct());
Assert.Contains("recovery link is on its way", bodies[0]);
var queued = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.SendRecovery && j.CreatedAt >= since).ExecuteAsync(token);
Assert.True(queued.Count >= answers.Length);
Assert.Contains(queued, j => j.Payload.Contains(withEmail.Id));
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostJson("/clientapi/user/recover/password", new { })).StatusCode);
}
[Fact]
public async Task The_recovery_job_keeps_only_a_hash_and_an_unreachable_mail_server_says_nothing_to_anyone()
{
var token = TestContext.Current.CancellationToken;
var root = await _host.SignUp("smtp");
var email = $"{Guid.NewGuid():N}@example.test";
using (var signedIn = _host.As(root.Jwt))
Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode);
using var client = _host.Client();
var answer = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName });
await _host.Run(j => j.Kind == JobKind.SendRecovery && j.Payload.Contains(root.Id), token);
Assert.Equal(HttpStatusCode.OK, answer.StatusCode);
var recovery = await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteFirstAsync(token);
Assert.NotNull(recovery);
Assert.Null(recovery.RecoveryCode);
Assert.Equal(64, recovery.CodeHash.Length);
Assert.InRange(recovery.ExpiresAt, DateTime.UtcNow.AddMinutes(50), DateTime.UtcNow.AddMinutes(61));
}
[Fact]
public async Task A_wrong_recovery_code_changes_nothing()
{
using var client = _host.Client();
var code = Guid.NewGuid().ToString("N");
var valid = await client.PostAsync("/clientapi/user/recover/valid", new StringContent($"\"{code}\"", Encoding.UTF8, "application/json"), TestContext.Current.CancellationToken);
var change = await client.PostJson("/clientapi/user/recover/update/password", new { newPassword = NewPassword, repeatedPassword = NewPassword, recoveryCode = code });
Assert.Equal(HttpStatusCode.OK, valid.StatusCode);
Assert.Equal("false", await valid.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
Assert.Equal(HttpStatusCode.NotFound, change.StatusCode);
Assert.Equal("Invalid recovery code.", await Message(change));
}
[Fact]
public async Task A_valid_recovery_code_resets_the_password_once()
{
var root = await _host.SignUp("recover");
var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N");
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, CodeHash = RecoveryJob.Hash(code), ExpiresAt = DateTime.UtcNow.AddHours(1) },
TestContext.Current.CancellationToken);
var persona = await _host.Persona(root, "recovered");
var apiToken = await _host.MastodonToken(persona);
using var client = _host.Client();
var valid = await client.PostAsync("/clientapi/user/recover/valid", new StringContent($"\"{code}\"", Encoding.UTF8, "application/json"), TestContext.Current.CancellationToken);
var change = await client.PostJson("/clientapi/user/recover/update/password", new { newPassword = NewPassword, repeatedPassword = NewPassword, recoveryCode = code });
var again = await client.PostJson("/clientapi/user/recover/update/password", new { newPassword = "Third-Pass-3!", repeatedPassword = "Third-Pass-3!", recoveryCode = code });
Assert.Equal("true", await valid.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
Assert.True(change.IsSuccessStatusCode);
Assert.Equal(HttpStatusCode.NotFound, again.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await LogIn(root.UserName, root.Password)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, NewPassword)).StatusCode);
Assert.False(await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken));
// whoever held the old sessions may be why the password was recovered: they end
using (var oldSession = _host.As(root.Jwt))
Assert.Equal(HttpStatusCode.Unauthorized, (await oldSession.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken)).StatusCode);
using (var oldApp = _host.Client())
{
oldApp.DefaultRequestHeaders.Authorization = new("Bearer", apiToken);
Assert.Equal(HttpStatusCode.Unauthorized, (await oldApp.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode);
}
}
[Fact]
public async Task An_expired_recovery_code_changes_nothing()
{
var root = await _host.SignUp("expired");
var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N");
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, CodeHash = RecoveryJob.Hash(code), ExpiresAt = DateTime.UtcNow.AddMinutes(-1) },
TestContext.Current.CancellationToken);
using var client = _host.Client();
var change = await client.PostJson("/clientapi/user/recover/update/password", new { newPassword = NewPassword, repeatedPassword = NewPassword, recoveryCode = code });
Assert.Equal(HttpStatusCode.NotFound, change.StatusCode);
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, root.Password)).StatusCode);
}
[Fact]
public async Task Sniffing_again_answers_a_fresh_token()
{
var root = await _host.SignUp("sniff");
using var client = _host.As(root.Jwt);
var response = await client.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken);
using var anonymous = _host.Client();
var refused = await anonymous.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var jwt = await response.JsonBody();
Assert.Equal(root.Id, jwt["userId"]!.GetValue<string>());
Assert.Equal(root.UserName, jwt["username"]!.GetValue<string>());
Assert.True(jwt["expiration"]!.GetValue<long>() > DateTime.UtcNow.Ticks);
Assert.Equal(HttpStatusCode.OK, await Settings(jwt["token"]!.GetValue<string>()));
Assert.Equal(HttpStatusCode.Unauthorized, refused.StatusCode);
}
[Fact]
public async Task The_eleventh_sign_up_from_one_address_in_a_minute_is_refused()
{
using var client = _host.ClientAt("198.51.100.21");
for (var attempt = 1; attempt <= 10; attempt++)
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostJson("/clientapi/user/signup", new { })).StatusCode);
var eleventh = await client.PostJson("/clientapi/user/signup", new { userName = Name("limited"), password = Accounts.Password });
Assert.Equal(HttpStatusCode.TooManyRequests, eleventh.StatusCode);
}
[Fact]
public async Task A_bad_token_answers_401_with_a_json_body()
{
var root = await _host.SignUp("badjwt");
var tokens = new[]
{
"not-a-jwt",
Jwt(root.Id, root.UserName, DateTime.UtcNow.AddHours(-1)),
Jwt(root.Id, root.UserName, DateTime.UtcNow.AddHours(1), key: "another-key-entirely-0123456789abcdef0123456789abcdef0123456789abcdef")
};
foreach (var token in tokens)
{
using var client = _host.As(token);
var response = await client.GetAsync("/clientapi/user/settings", TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode);
Assert.Equal("application/json", response.Content.Headers.ContentType!.MediaType);
var body = await response.JsonBody();
Assert.Equal(401, body["statusCode"]!.GetValue<int>());
Assert.False(body["isValid"]!.GetValue<bool>());
Assert.Contains("invalid_token", response.Headers.WwwAuthenticate.ToString());
}
Assert.Equal(HttpStatusCode.OK, await Settings(Jwt(root.Id, root.UserName, DateTime.UtcNow.AddHours(1))));
}
[Fact]
public async Task A_banned_or_removed_root_loses_the_client_api_at_once()
{
var banned = await _host.SignUp("banned");
var removed = await _host.SignUp("removed");
Assert.Equal(HttpStatusCode.OK, await Settings(banned.Jwt));
Assert.Equal(HttpStatusCode.OK, await Settings(removed.Jwt));
await ClientApi.Ban(banned.Id);
await DB.Default.Update<RootUser>().MatchID(removed.Id).Modify(u => u.DeletedAt, DateTime.UtcNow).ExecuteAsync(TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.Unauthorized, await Settings(banned.Jwt));
Assert.Equal(HttpStatusCode.Unauthorized, await Settings(removed.Jwt));
using (var client = _host.As(banned.Jwt))
{
Assert.Equal(HttpStatusCode.Unauthorized, (await client.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken)).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await client.PostJson("/clientapi/avatar/private/insert", new { userName = Name("late"), name = "late", biography = "testing" })).StatusCode);
}
await ClientApi.Ban(banned.Id, banned: false);
Assert.Equal(HttpStatusCode.OK, await Settings(banned.Jwt));
}
}
}
+302
View File
@@ -0,0 +1,302 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class ClientApiGroupsTests : IAsyncLifetime
{
const string DoorPassword = "door-Pass-1";
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
async Task<Persona> NewPersona(string name) => await _host.Persona(await _host.SignUp(name), name);
static Task<GroupEntity> Stored(JsonObject group) =>
DB.Default.Find<GroupEntity>().MatchID(group["id"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
async Task<List<string>> Groups(Persona persona)
{
using var client = _host.As(persona.Root.Jwt);
var response = await client.GetAsync("/clientapi/group/list?avatarId=" + persona.Id, TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
return (await response.JsonItems()).Select(g => g!["id"]!.GetValue<string>()).ToList();
}
async Task<HttpResponseMessage> Approve(Persona manager, JsonObject group, string actorUri)
{
using var client = _host.As(manager.Root.Jwt);
return await client.PostJson("/clientapi/group/approve", new { avatarId = manager.Id, groupId = group["id"]!.GetValue<string>(), memberActorURI = actorUri });
}
async Task<(Persona Owner, JsonObject Circle, RemoteActor Member)> CircleWithRemoteMember()
{
var owner = await NewPersona("circleowner");
var circle = await _host.Group(owner, community: false, name: "circle");
var member = new RemoteActor(_peer, "member");
await _host.Follow(member, _peer.A, circle["userName"]!.GetValue<string>());
Assert.True((await Approve(owner, circle, member.Id)).IsSuccessStatusCode);
return (owner, circle, member);
}
async Task<HttpStatusCode> Get(string path, RemoteActor signer = default)
{
using var client = _host.Client();
using var request = signer == default ? new HttpRequestMessage(HttpMethod.Get, path) : signer.SignedGet(path);
request.Headers.Accept.ParseAdd("application/activity+json");
return (await client.SendAsync(request, TestContext.Current.CancellationToken)).StatusCode;
}
[Fact]
public async Task A_community_and_a_circle_are_created()
{
var owner = await NewPersona("founder");
var community = await _host.Group(owner, community: true, name: "community");
var circle = await _host.Group(owner, community: false, password: DoorPassword, name: "circle");
Assert.True(community["isCommunity"]!.GetValue<bool>());
Assert.False(circle["isCommunity"]!.GetValue<bool>());
foreach (var group in new[] { community, circle })
{
Assert.True(group["isOwner"]!.GetValue<bool>());
Assert.Equal(64, group["invitationCode"]!.GetValue<string>().Length);
Assert.Equal($"{PrivaPubHost.Base}/peasants/{group["userName"]!.GetValue<string>()}", group["url"]!.GetValue<string>());
Assert.Equal(1, group["membersCount"]!.GetValue<int>());
}
Assert.True(circle["isPasswordRequired"]!.GetValue<bool>());
Assert.False(community["isPasswordRequired"]!.GetValue<bool>());
Assert.Equal(GroupKind.Community, (await Stored(community)).Kind);
var storedCircle = await Stored(circle);
Assert.Equal(GroupKind.Circle, storedCircle.Kind);
Assert.NotEqual(DoorPassword, storedCircle.HashedInvitationPassword);
Assert.Contains(storedCircle.Members, m => m.AvatarId == owner.Id && m.Role == GroupRole.Owner);
Assert.Equal(new[] { community["id"]!.GetValue<string>(), circle["id"]!.GetValue<string>() }.Order(), (await Groups(owner)).Order());
}
[Fact]
public async Task A_group_needs_a_persona_of_ones_own()
{
var owner = await NewPersona("founder");
var stranger = await _host.SignUp("stranger");
using var client = _host.As(stranger.Jwt);
var insert = await client.PostJson("/clientapi/group/insert", new { avatarId = owner.Id, userName = $"stolen{Guid.NewGuid():N}"[..20], name = "stolen", isCommunity = true });
var list = await client.GetAsync("/clientapi/group/list?avatarId=" + owner.Id, TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.NotFound, insert.StatusCode);
Assert.Equal(HttpStatusCode.NotFound, list.StatusCode);
}
[Fact]
public async Task Joining_takes_the_code_and_the_password()
{
var owner = await NewPersona("host");
var circle = await _host.Group(owner, community: false, password: DoorPassword);
var joiner = await NewPersona("guest");
var invitationCode = circle["invitationCode"]!.GetValue<string>();
using var client = _host.As(joiner.Root.Jwt);
var noPassword = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode });
var wrongPassword = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode, invitationPassword = "wrong" });
var wrongCode = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode = $"{Guid.NewGuid():N}{Guid.NewGuid():N}", invitationPassword = DoorPassword });
var joined = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode, invitationPassword = DoorPassword });
var twice = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode, invitationPassword = DoorPassword });
Assert.Equal(HttpStatusCode.NotAcceptable, noPassword.StatusCode);
Assert.Equal(HttpStatusCode.NotAcceptable, wrongPassword.StatusCode);
Assert.Equal(HttpStatusCode.NotFound, wrongCode.StatusCode);
Assert.Equal(HttpStatusCode.OK, joined.StatusCode);
var view = await joined.JsonBody();
Assert.Equal(2, view["membersCount"]!.GetValue<int>());
Assert.False(view["isOwner"]!.GetValue<bool>());
Assert.Null(view["invitationCode"]);
Assert.Equal(HttpStatusCode.OK, twice.StatusCode);
Assert.Equal(2, (await Stored(circle)).Members.Count);
Assert.Equal(new[] { circle["id"]!.GetValue<string>() }, await Groups(joiner));
}
[Fact]
public async Task Joining_needs_a_persona_of_ones_own()
{
var owner = await NewPersona("host");
var community = await _host.Group(owner, community: true);
var victim = await NewPersona("victim");
var stranger = await _host.SignUp("stranger");
using var client = _host.As(stranger.Jwt);
var response = await client.PostJson("/clientapi/group/join", new { avatarId = victim.Id, invitationCode = community["invitationCode"]!.GetValue<string>() });
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
Assert.DoesNotContain((await Stored(community)).Members, m => m.AvatarId == victim.Id);
}
[Fact]
public async Task Members_leave_and_owners_cannot()
{
var owner = await NewPersona("host");
var community = await _host.Group(owner, community: true);
var member = await NewPersona("leaver");
var groupId = community["id"]!.GetValue<string>();
using (var joining = _host.As(member.Root.Jwt))
Assert.Equal(HttpStatusCode.OK, (await joining.PostJson("/clientapi/group/join", new { avatarId = member.Id, invitationCode = community["invitationCode"]!.GetValue<string>() })).StatusCode);
using var memberClient = _host.As(member.Root.Jwt);
using var ownerClient = _host.As(owner.Root.Jwt);
var left = await memberClient.PostJson("/clientapi/group/leave", new { avatarId = member.Id, groupId });
var ownerLeft = await ownerClient.PostJson("/clientapi/group/leave", new { avatarId = owner.Id, groupId });
Assert.True(left.IsSuccessStatusCode);
Assert.Empty(await Groups(member));
Assert.Equal(HttpStatusCode.BadRequest, ownerLeft.StatusCode);
Assert.Equal(new[] { owner.Id }, (await Stored(community)).Members.Select(m => m.AvatarId));
}
[Fact]
public async Task A_circle_takes_a_remote_member_only_once_the_owner_approves()
{
var owner = await NewPersona("approver");
var circle = await _host.Group(owner, community: false);
var member = new RemoteActor(_peer, "asker");
var since = DateTime.UtcNow.AddSeconds(-1);
await _host.Follow(member, _peer.A, circle["userName"]!.GetValue<string>());
var request = await DB.Default.Find<Follower>().Match(f => f.LocalActorId == circle["id"]!.GetValue<string>() && f.ActorURI == member.Id).ExecuteSingleAsync(TestContext.Current.CancellationToken);
Assert.False(request.IsAccepted);
Assert.DoesNotContain((await Stored(circle)).Members, m => m.AvatarId == member.Id);
Assert.Empty(await Jobs.Deliveries(member.Id + "/inbox", since, TestContext.Current.CancellationToken));
var outsider = await NewPersona("outsider");
Assert.Equal(HttpStatusCode.NotFound, (await Approve(outsider, circle, member.Id)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await Approve(owner, circle, $"{_peer.A}/users/nobody")).StatusCode);
Assert.True((await Approve(owner, circle, member.Id)).IsSuccessStatusCode);
Assert.Contains((await Stored(circle)).Members, m => m.IsForeign && m.AvatarId == member.Id);
Assert.True((await DB.Default.Find<Follower>().MatchID(request.ID).ExecuteFirstAsync(TestContext.Current.CancellationToken)).IsAccepted);
var accept = Assert.Single(await Jobs.Deliveries(member.Id + "/inbox", since, TestContext.Current.CancellationToken));
Assert.Equal("Accept", accept["type"]!.GetValue<string>());
Assert.Equal(circle["url"]!.GetValue<string>(), accept["actor"]!.GetValue<string>());
}
// its owner sees who is in the circle and who asks to join, declines one request and takes members out; nobody else
// sees them
[Fact]
public async Task An_owner_sees_the_members_and_requests_declines_one_and_removes_members()
{
var token = TestContext.Current.CancellationToken;
var (owner, circle, member) = await CircleWithRemoteMember();
var groupId = circle["id"]!.GetValue<string>();
var joiner = await NewPersona("joiner");
using (var joining = _host.As(joiner.Root.Jwt))
Assert.True((await joining.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode = circle["invitationCode"]!.GetValue<string>() })).IsSuccessStatusCode);
var asker = new RemoteActor(_peer, "asker");
await _host.Follow(asker, _peer.A, circle["userName"]!.GetValue<string>());
var since = DateTime.UtcNow.AddSeconds(-1);
using var client = _host.As(owner.Root.Jwt);
async Task<List<JsonNode>> Members(HttpClient as_, string avatarId)
{
var response = await as_.GetAsync($"/clientapi/group/members?avatarId={avatarId}&groupId={groupId}", token);
return response.IsSuccessStatusCode ? (await response.JsonItems()).ToList() : null;
}
var listed = await Members(client, owner.Id);
using var joinerClient = _host.As(joiner.Root.Jwt);
var forJoiner = await Members(joinerClient, joiner.Id);
var declined = await client.PostJson("/clientapi/group/reject", new { avatarId = owner.Id, groupId, memberActorURI = asker.Id });
var removedRemote = await client.PostJson("/clientapi/group/remove", new { avatarId = owner.Id, groupId, memberActorURI = member.Id });
var removedLocal = await client.PostJson("/clientapi/group/remove", new { avatarId = owner.Id, groupId, memberActorURI = joiner.ActorUri() });
var removedOwner = await client.PostJson("/clientapi/group/remove", new { avatarId = owner.Id, groupId, memberActorURI = owner.ActorUri() });
Assert.Null(forJoiner);
Assert.Equal(4, listed.Count);
Assert.Contains(listed, m => m!["actorUri"]!.GetValue<string>() == owner.ActorUri() && m["role"]!.GetValue<string>() == "owner" && m["isLocal"]!.GetValue<bool>());
Assert.Contains(listed, m => m!["actorUri"]!.GetValue<string>() == joiner.ActorUri() && !m["isPending"]!.GetValue<bool>());
Assert.Contains(listed, m => m!["actorUri"]!.GetValue<string>() == member.Id && !m["isPending"]!.GetValue<bool>() && !m["isLocal"]!.GetValue<bool>());
Assert.Contains(listed, m => m!["actorUri"]!.GetValue<string>() == asker.Id && m["isPending"]!.GetValue<bool>());
Assert.True(declined.IsSuccessStatusCode);
Assert.True(removedRemote.IsSuccessStatusCode);
Assert.True(removedLocal.IsSuccessStatusCode);
Assert.Equal(HttpStatusCode.BadRequest, removedOwner.StatusCode);
Assert.Equal(new[] { owner.Id }, (await Stored(circle)).Members.Select(m => m.AvatarId));
Assert.False(await DB.Default.Find<Follower>().Match(f => f.LocalActorId == groupId && f.ActorURI != owner.ActorUri()).ExecuteAnyAsync(token));
Assert.Contains(await Jobs.Deliveries(asker.Id + "/inbox", since, token), d => d["type"]!.GetValue<string>() == "Reject");
Assert.Contains(await Jobs.Deliveries(member.Id + "/inbox", since, token), d => d["type"]!.GetValue<string>() == "Reject");
}
[Fact]
public async Task A_circle_is_never_found()
{
var owner = await NewPersona("hidden");
var circle = await _host.Group(owner, community: false, name: "circle");
var community = await _host.Group(owner, community: true, name: "community");
var circleName = circle["userName"]!.GetValue<string>();
var communityName = community["userName"]!.GetValue<string>();
using var anonymous = _host.Client();
using var searcher = _host.As(await _host.MastodonToken(await NewPersona("searcher")));
foreach (var acct in new[] { circleName, $"{circleName}@{PrivaPubHost.Host}", $"@{circleName}@{PrivaPubHost.Host}" })
{
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.GetAsync("/api/v1/accounts/lookup?acct=" + Uri.EscapeDataString(acct), TestContext.Current.CancellationToken)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await searcher.GetAsync("/api/v1/accounts/lookup?acct=" + Uri.EscapeDataString(acct), TestContext.Current.CancellationToken)).StatusCode);
}
Assert.Equal(HttpStatusCode.OK, (await anonymous.GetAsync("/api/v1/accounts/lookup?acct=" + communityName, TestContext.Current.CancellationToken)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await searcher.GetAsync("/api/v1/accounts/" + circle["id"]!.GetValue<string>(), TestContext.Current.CancellationToken)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await searcher.GetAsync("/api/v1/accounts/" + community["id"]!.GetValue<string>(), TestContext.Current.CancellationToken)).StatusCode);
foreach (var q in new[] { circleName, $"@{circleName}@{PrivaPubHost.Host}", circle["url"]!.GetValue<string>() })
{
var found = await searcher.GetStringAsync($"/api/v2/search?resolve=true&q={Uri.EscapeDataString(q)}", TestContext.Current.CancellationToken);
Assert.Empty(JsonNode.Parse(found)!["accounts"]!.AsArray());
Assert.Empty(JsonNode.Parse(found)!["statuses"]!.AsArray());
}
var communityFound = await searcher.GetStringAsync($"/api/v2/search?resolve=true&q={Uri.EscapeDataString(community["url"]!.GetValue<string>())}", TestContext.Current.CancellationToken);
Assert.Single(JsonNode.Parse(communityFound)!["accounts"]!.AsArray());
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.GetAsync($"/@{circleName}", TestContext.Current.CancellationToken)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await anonymous.GetAsync($"/@{communityName}", TestContext.Current.CancellationToken)).StatusCode);
}
[Fact]
public async Task A_circles_members_and_wardens_are_shown_only_to_members()
{
var (_, circle, member) = await CircleWithRemoteMember();
var outsider = new RemoteActor(_peer, "outsider");
var community = await _host.Group(await NewPersona("open"), community: true);
var circleName = circle["userName"]!.GetValue<string>();
var communityName = community["userName"]!.GetValue<string>();
foreach (var collection in new[] { "flock", "wardens" })
{
var path = $"/peasants/{circleName}/{collection}";
Assert.Equal(HttpStatusCode.NotFound, await Get(path));
Assert.Equal(HttpStatusCode.NotFound, await Get(path, outsider));
Assert.Equal(HttpStatusCode.OK, await Get(path, member));
Assert.Equal(HttpStatusCode.OK, await Get($"/peasants/{communityName}/{collection}"));
}
}
}
}
@@ -0,0 +1,308 @@
using MongoDB.Entities;
using PrivaPub.Federation.Moderation;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Net.Http.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class ClientApiModerationTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
async Task<HttpStatusCode> Settings(string jwt)
{
using var client = _host.As(jwt);
return (await client.GetAsync("/clientapi/user/settings", TestContext.Current.CancellationToken)).StatusCode;
}
async Task<HttpStatusCode> Credentials(string token)
{
using var client = _host.As(token);
return (await client.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode;
}
async Task<HttpResponseMessage> LogIn(Root root)
{
using var client = _host.Client();
return await client.PostJson("/clientapi/user/login", new { userName = root.UserName, password = root.Password });
}
static Task<HttpResponseMessage> Remove(HttpClient client, params string[] ids) =>
client.SendAsync(new HttpRequestMessage(HttpMethod.Delete, "/clientapi/admin/remove/users") { Content = JsonContent.Create(new { userIdList = ids }) });
static Task<RootUser> Stored(string id) => DB.Default.Find<RootUser>().MatchID(id).ExecuteFirstAsync(TestContext.Current.CancellationToken);
static JsonObject DirectNote(RemoteActor sender, string origin, Persona persona)
{
var noteId = $"{origin}/notes/{Guid.NewGuid():N}";
var to = new JsonArray($"{PrivaPubHost.Base}/peasants/{persona.UserName}");
return new JsonObject
{
["id"] = noteId + "/activity",
["type"] = "Create",
["actor"] = sender.Id,
["to"] = to.DeepClone(),
["object"] = new JsonObject { ["id"] = noteId, ["type"] = "Note", ["attributedTo"] = sender.Id, ["to"] = to.DeepClone(), ["content"] = "<p>hello</p>" }
};
}
[Fact]
public async Task An_admin_bans_and_unbans_a_root()
{
var admin = await _host.Admin();
var victim = await _host.Persona(await _host.SignUp("victim"), "victim");
var token = await _host.MastodonToken(victim);
using var client = _host.As(admin.Jwt);
var banned = await client.PostJson("/clientapi/admin/ban/users", new { userIdList = new[] { victim.Root.Id } });
Assert.Equal(HttpStatusCode.OK, banned.StatusCode);
Assert.True((await Stored(victim.Root.Id)).IsBanned);
Assert.Equal(HttpStatusCode.Unauthorized, await Settings(victim.Root.Jwt));
Assert.Equal(HttpStatusCode.Unauthorized, await Credentials(token));
var login = await LogIn(victim.Root);
Assert.Equal(HttpStatusCode.BadRequest, login.StatusCode);
Assert.Contains("banned", (await login.JsonBody())["errorMessage"]!.GetValue<string>());
var unbanned = await client.PostJson("/clientapi/admin/unban/users", new { userIdList = new[] { victim.Root.Id } });
Assert.Equal(HttpStatusCode.OK, unbanned.StatusCode);
Assert.Equal(HttpStatusCode.OK, await Settings(victim.Root.Jwt));
Assert.Equal(HttpStatusCode.OK, await Credentials(token));
Assert.Equal(HttpStatusCode.OK, (await LogIn(victim.Root)).StatusCode);
}
[Fact]
public async Task An_admin_cannot_ban_or_remove_itself()
{
var admin = await _host.Admin();
var victim = await _host.SignUp("victim");
using var client = _host.As(admin.Jwt);
Assert.Equal(HttpStatusCode.OK, (await client.PostJson("/clientapi/admin/ban/users", new { userIdList = new[] { admin.Id, victim.Id } })).StatusCode);
var removed = await Remove(client, admin.Id);
Assert.False((await Stored(admin.Id)).IsBanned);
Assert.True((await Stored(victim.Id)).IsBanned);
Assert.Equal(HttpStatusCode.BadRequest, removed.StatusCode);
Assert.Null((await Stored(admin.Id)).DeletedAt);
Assert.Equal(HttpStatusCode.OK, await Settings(admin.Jwt));
}
[Fact]
public async Task An_admin_removes_a_root()
{
var admin = await _host.Admin();
var victim = await _host.Persona(await _host.SignUp("removed"), "removed");
var token = await _host.MastodonToken(victim);
using var client = _host.As(admin.Jwt);
var removed = await Remove(client, victim.Root.Id);
var again = await Remove(client, victim.Root.Id);
Assert.Equal(HttpStatusCode.OK, removed.StatusCode);
var stored = await Stored(victim.Root.Id);
Assert.NotNull(stored.DeletedAt);
Assert.Null(stored.HashedPassword);
Assert.NotEqual(victim.Root.UserName, stored.UserName);
Assert.Equal(HttpStatusCode.BadRequest, again.StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, await Settings(victim.Root.Jwt));
Assert.Equal(HttpStatusCode.Unauthorized, await Credentials(token));
Assert.Equal(HttpStatusCode.BadRequest, (await LogIn(victim.Root)).StatusCode);
}
[Fact]
public async Task Only_admins_moderate()
{
var root = await _host.SignUp("plain");
var target = await _host.SignUp("target");
using var client = _host.As(root.Jwt);
using var anonymous = _host.Client();
var forbidden = new List<HttpResponseMessage>
{
await client.PostJson("/clientapi/admin/ban/users", new { userIdList = new[] { target.Id } }),
await client.PostJson("/clientapi/admin/unban/users", new { userIdList = new[] { target.Id } }),
await Remove(client, target.Id),
await client.GetAsync("/clientapi/admin/domainblocks/list", TestContext.Current.CancellationToken),
await client.PostJson("/clientapi/admin/domainblocks/insert", new { domain = "forbidden.example" }),
await client.PostAsync("/clientapi/admin/domainblocks/delete?domain=forbidden.example", default, TestContext.Current.CancellationToken),
await client.GetAsync("/clientapi/moderator/reports", TestContext.Current.CancellationToken),
await client.PostAsync($"/clientapi/moderator/reports/{target.Id}/resolve", default, TestContext.Current.CancellationToken)
};
var unauthorized = await anonymous.PostJson("/clientapi/admin/ban/users", new { userIdList = new[] { target.Id } });
foreach (var response in forbidden)
{
Assert.Equal(HttpStatusCode.Forbidden, response.StatusCode);
var body = await response.JsonBody();
Assert.Equal(403, body["statusCode"]!.GetValue<int>());
Assert.Equal("Forbidden.", body["errorMessage"]!.GetValue<string>());
}
Assert.Equal(HttpStatusCode.Unauthorized, unauthorized.StatusCode);
var stored = await Stored(target.Id);
Assert.False(stored.IsBanned);
Assert.Null(stored.DeletedAt);
Assert.False(await DB.Default.Find<DomainBlock>().Match(b => b.Domain == "forbidden.example").ExecuteAnyAsync(TestContext.Current.CancellationToken));
}
[Fact]
public async Task A_moderator_lists_and_resolves_reports()
{
var admin = await _host.Admin();
var reporter = await _host.Persona(await _host.SignUp("reporter"), "reporter");
var target = await _host.Persona(await _host.SignUp("reported"), "reported");
var comment = $"spam {Guid.NewGuid():N}";
using (var api = _host.As(await _host.MastodonToken(reporter)))
Assert.Equal(HttpStatusCode.OK, (await api.Form("/api/v1/reports", ("account_id", target.Id), ("comment", comment), ("category", "spam"))).StatusCode);
var reportId = (await DB.Default.Find<Models.Social.Report>().Match(r => r.Comment == comment).ExecuteSingleAsync(TestContext.Current.CancellationToken)).ID;
using var client = _host.As(admin.Jwt);
var open = await client.GetStringAsync("/clientapi/moderator/reports?resolved=false", TestContext.Current.CancellationToken);
var resolved = await client.PostAsync($"/clientapi/moderator/reports/{reportId}/resolve", default, TestContext.Current.CancellationToken);
var stillOpen = await client.GetStringAsync("/clientapi/moderator/reports?resolved=false", TestContext.Current.CancellationToken);
var closed = await client.GetStringAsync("/clientapi/moderator/reports?resolved=true", TestContext.Current.CancellationToken);
var unknown = await client.PostAsync($"/clientapi/moderator/reports/{MongoDB.Bson.ObjectId.GenerateNewId()}/resolve", default, TestContext.Current.CancellationToken);
var junk = await client.PostAsync("/clientapi/moderator/reports/x/resolve", default, TestContext.Current.CancellationToken);
var listed = JsonNode.Parse(open)!.AsArray().Single(r => r!["id"]!.GetValue<string>() == reportId)!;
Assert.Equal("local", listed["reporter"]!.GetValue<string>());
Assert.Equal(target.Id, listed["targetAccountId"]!.GetValue<string>());
Assert.Equal("spam", listed["category"]!.GetValue<string>());
Assert.DoesNotContain(reporter.Id, open);
Assert.DoesNotContain(reporter.UserName, open);
Assert.DoesNotContain(reporter.Root.Id, open);
Assert.Equal(HttpStatusCode.OK, resolved.StatusCode);
Assert.DoesNotContain(reportId, stillOpen);
Assert.Contains(reportId, closed);
var stored = await DB.Default.Find<Models.Social.Report>().MatchID(reportId).ExecuteFirstAsync(TestContext.Current.CancellationToken);
Assert.Equal(admin.Id, stored.ResolvedBy);
Assert.NotNull(stored.ResolvedAt);
Assert.Equal(HttpStatusCode.NotFound, unknown.StatusCode);
Assert.Equal(HttpStatusCode.NotFound, junk.StatusCode);
}
[Fact]
public async Task Domain_blocks_are_inserted_listed_and_deleted()
{
var admin = await _host.Admin();
var domain = $"blocked-{Guid.NewGuid():N}.example";
using var client = _host.As(admin.Jwt);
try
{
var inserted = await client.PostJson("/clientapi/admin/domainblocks/insert", new { domain = $" {domain.ToUpperInvariant()}. ", suspend = false, rejectMedia = true, publicComment = "noisy" });
var listed = await (await client.GetAsync("/clientapi/admin/domainblocks/list", TestContext.Current.CancellationToken)).JsonItems();
var suspended = await client.PostJson("/clientapi/admin/domainblocks/insert", new { domain, suspend = true });
var deleted = await client.PostAsync($"/clientapi/admin/domainblocks/delete?domain={domain}", default, TestContext.Current.CancellationToken);
var after = await (await client.GetAsync("/clientapi/admin/domainblocks/list", TestContext.Current.CancellationToken)).JsonItems();
Assert.Equal(HttpStatusCode.OK, inserted.StatusCode);
var view = await inserted.JsonBody();
Assert.Equal(domain, view["domain"]!.GetValue<string>());
Assert.Equal("Silence", view["severity"]!.GetValue<string>());
Assert.True(view["rejectMedia"]!.GetValue<bool>());
Assert.Contains(listed, b => b!["domain"]!.GetValue<string>() == domain);
Assert.Equal(HttpStatusCode.OK, suspended.StatusCode);
Assert.Equal("Suspend", (await suspended.JsonBody())["severity"]!.GetValue<string>());
Assert.Equal(view["id"]!.GetValue<string>(), (await suspended.JsonBody())["id"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.OK, deleted.StatusCode);
Assert.DoesNotContain(after, b => b!["domain"]!.GetValue<string>() == domain);
Assert.Null(_host.Get<IDomainBlocks>().Find(domain));
}
finally
{
await DB.Default.DeleteAsync<DomainBlock>(b => b.Domain == domain);
await _host.Get<IDomainBlocks>().Reload(CancellationToken.None);
}
}
[Theory]
[InlineData("")]
[InlineData("ab")]
[InlineData("not a domain")]
[InlineData("127.0.0.1")]
[InlineData("https://bad.example/")]
[InlineData("bad..example")]
public async Task A_bad_domain_is_refused(string domain)
{
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
var response = await client.PostJson("/clientapi/admin/domainblocks/insert", new { domain });
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
var normalised = DomainBlocks.Normalise(domain);
Assert.False(await DB.Default.Find<DomainBlock>().Match(b => b.Domain == normalised).ExecuteAnyAsync(TestContext.Current.CancellationToken));
}
[Fact]
public async Task A_suspended_server_is_answered_202_and_nothing_is_kept()
{
var token = TestContext.Current.CancellationToken;
var admin = await _host.Admin();
var persona = await _host.Persona(await _host.SignUp("suspend"), "suspend");
await using var peer = await Peer.Start();
var sender = new RemoteActor(peer, "suspended", origin: peer.B);
var blocked = DirectNote(sender, peer.B, persona);
var welcome = DirectNote(sender, peer.B, persona);
using var adminClient = _host.As(admin.Jwt);
using var client = _host.Client();
try
{
Assert.Equal(HttpStatusCode.OK, (await adminClient.PostJson("/clientapi/admin/domainblocks/insert", new { domain = "localhost", suspend = true })).StatusCode);
var answer = await client.SendAsync(sender.SignedPost($"/peasants/{persona.UserName}/mouth", blocked), token);
Assert.Equal(HttpStatusCode.Accepted, answer.StatusCode);
Assert.False(await DB.Default.Find<Job>().Match(j => j.DedupeKey == "inbox|" + blocked["id"]!.GetValue<string>()).ExecuteAnyAsync(token));
Assert.Equal(0, await _host.RunInbox(blocked["id"]!.GetValue<string>(), token));
Assert.False(await DB.Default.Find<Post>().Match(p => p.ObjectURI == blocked["object"]!["id"]!.GetValue<string>()).ExecuteAnyAsync(token));
Assert.Empty(peer.Requests);
Assert.Equal(HttpStatusCode.OK, (await adminClient.PostAsync("/clientapi/admin/domainblocks/delete?domain=localhost", default, token)).StatusCode);
Assert.Equal(HttpStatusCode.Accepted, (await client.SendAsync(sender.SignedPost($"/peasants/{persona.UserName}/mouth", welcome), token)).StatusCode);
Assert.Equal(1, await _host.RunInbox(welcome["id"]!.GetValue<string>(), token));
Assert.True(await DB.Default.Find<Post>().Match(p => p.ObjectURI == welcome["object"]!["id"]!.GetValue<string>()).ExecuteAnyAsync(token));
}
finally
{
await DB.Default.DeleteAsync<DomainBlock>(b => b.Domain == "localhost");
await _host.Get<IDomainBlocks>().Reload(CancellationToken.None);
}
}
[Fact]
public async Task The_data_endpoints_answer_anyone()
{
using var client = _host.Client();
var ping = await client.GetAsync("/clientapi/data/ping", TestContext.Current.CancellationToken);
var version = await client.GetAsync("/clientapi/data/current-version", TestContext.Current.CancellationToken);
var languages = await client.GetAsync("/clientapi/data/languages", TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.NoContent, ping.StatusCode);
Assert.Equal(HttpStatusCode.OK, version.StatusCode);
Assert.False(string.IsNullOrWhiteSpace(await version.Content.ReadAsStringAsync(TestContext.Current.CancellationToken)));
Assert.Equal(HttpStatusCode.OK, languages.StatusCode);
var codes = (await languages.JsonItems()).Select(l => l!["international2Code"]!.GetValue<string>()).Order().ToList();
Assert.Equal(new[] { "en", "it" }, codes);
}
}
}
@@ -0,0 +1,232 @@
using MongoDB.Bson;
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Globalization;
using System.Net;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class ClientApiPersonasTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static string Name(string prefix) => $"{prefix}{Guid.NewGuid():N}"[..20];
async Task<HttpResponseMessage> Insert(Root root, object body)
{
using var client = _host.As(root.Jwt);
return await client.PostJson("/clientapi/avatar/private/insert", body);
}
async Task<List<string>> Listed(Root root)
{
using var client = _host.As(root.Jwt);
return (await (await client.GetAsync("/clientapi/avatar/private/list", TestContext.Current.CancellationToken)).JsonItems())
.Select(a => a!["id"]!.GetValue<string>()).ToList();
}
static DateTime IdDay(string id) => ObjectId.Parse(id).CreationTime;
static void WithinTwoWeeksBefore(DateTime created, DateTime published, string id)
{
Assert.Equal(published.Date, published);
Assert.InRange(published, created.Date.AddDays(-13), DateTime.UtcNow.Date);
Assert.Equal(published, IdDay(id));
}
[Fact]
public async Task A_root_id_in_the_body_is_ignored()
{
var root = await _host.SignUp("owner");
var other = await _host.SignUp("victim");
var userName = Name("planted");
var response = await Insert(root, new { userName, name = "planted", biography = "testing", rootId = other.Id });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var id = (await response.JsonBody())["id"]!.GetValue<string>();
var link = await DB.Default.Find<RootToAvatar>().Match(r => r.AvatarId == id).ExecuteSingleAsync(TestContext.Current.CancellationToken);
Assert.Equal(root.Id, link.RootId);
Assert.Contains(id, await Listed(root));
Assert.DoesNotContain(id, await Listed(other));
Assert.DoesNotContain(root.Id, await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
}
[Theory]
[InlineData("Upper")]
[InlineData("has-dash")]
[InlineData("dot.ted")]
[InlineData("spa ce")]
[InlineData("ünïcode")]
[InlineData("at@sign")]
[InlineData("")]
public async Task Usernames_outside_the_regex_are_refused(string userName)
{
var root = await _host.SignUp("regex");
var response = await Insert(root, new { userName, name = "regex", biography = "testing" });
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.Contains((await response.JsonBody())["errors"]!.AsObject(), e => e.Key.Equals("userName", StringComparison.OrdinalIgnoreCase));
Assert.Empty(await Listed(root));
if (userName.Length > 0)
Assert.False(await DB.Default.Find<ReservedName>().Match(r => r.Name == userName || r.Name == userName.ToLowerInvariant()).ExecuteAnyAsync(TestContext.Current.CancellationToken));
}
[Theory]
[InlineData("admin")]
[InlineData("privapub")]
[InlineData("root")]
[InlineData("moderator")]
[InlineData("abuse")]
public async Task Reserved_names_are_refused(string userName)
{
var root = await _host.SignUp("reserved");
var response = await Insert(root, new { userName, name = userName, biography = "testing" });
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.Contains("already taken", (await response.JsonBody())["errorMessage"]!.GetValue<string>());
Assert.Empty(await Listed(root));
}
[Fact]
public async Task Personas_and_groups_share_one_name_space()
{
var root = await _host.SignUp("names");
var persona = await _host.Persona(root, "names");
var group = await _host.Group(persona, community: true);
var groupName = group["userName"]!.GetValue<string>();
using var client = _host.As(root.Jwt);
var personaOverGroup = await Insert(root, new { userName = groupName, name = "copy", biography = "testing" });
var groupOverPersona = await client.PostJson("/clientapi/group/insert", new { avatarId = persona.Id, userName = persona.UserName, name = "copy", isCommunity = true });
var personaOverPersona = await Insert(await _host.SignUp("names"), new { userName = persona.UserName, name = "copy", biography = "testing" });
Assert.Equal(HttpStatusCode.BadRequest, personaOverGroup.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, groupOverPersona.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, personaOverPersona.StatusCode);
var reserved = await DB.Default.Find<ReservedName>().Match(r => r.Name == groupName || r.Name == persona.UserName).ExecuteAsync(TestContext.Current.CancellationToken);
Assert.Equal(2, reserved.Count);
Assert.Contains(reserved, r => r.Name == persona.UserName && r.OwnerKind == LocalActorKind.Person && r.OwnerId == persona.Id);
Assert.Contains(reserved, r => r.Name == groupName && r.OwnerKind == LocalActorKind.Group && r.OwnerId == group["id"]!.GetValue<string>());
Assert.Equal(1, await DB.Default.CountAsync<Avatar>(a => a.UserName == persona.UserName, TestContext.Current.CancellationToken));
Assert.Equal(1, await DB.Default.CountAsync<GroupEntity>(g => g.UserName == groupName, TestContext.Current.CancellationToken));
}
[Fact]
public async Task An_update_is_delivered_to_followers()
{
await using var peer = await Peer.Start();
var persona = await _host.Persona(await _host.SignUp("update"), "update");
var follower = new RemoteActor(peer, "fan");
await _host.Follow(follower, peer.A, persona.UserName);
var since = DateTime.UtcNow.AddSeconds(-1);
using var client = _host.As(persona.Root.Jwt);
var response = await client.PostJson("/clientapi/avatar/private/update", new { avatarId = persona.Id, name = "Renamed", biography = "new biography" });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var update = Assert.Single(await Jobs.Deliveries(follower.Id + "/inbox", since, TestContext.Current.CancellationToken), d => d["type"]!.GetValue<string>() == "Update");
Assert.Equal($"{PrivaPubHost.Base}/peasants/{persona.UserName}", update["actor"]!.GetValue<string>());
var person = update["object"]!.AsObject();
Assert.Equal("Person", person["type"]!.GetValue<string>());
Assert.Equal($"{PrivaPubHost.Base}/peasants/{persona.UserName}", person["id"]!.GetValue<string>());
Assert.Equal("Renamed", person["name"]!.GetValue<string>());
Assert.DoesNotContain(persona.Root.Id, update.ToJsonString());
Assert.DoesNotContain(persona.Root.UserName, update.ToJsonString());
}
[Fact]
public async Task A_theme_change_here_keeps_what_the_mastodon_api_set()
{
var persona = await _host.Persona(await _host.SignUp("keep"), "keep");
await DB.Default.Update<Avatar>().MatchID(persona.Id)
.Modify(a => a.Settings.IsDiscoverable, false)
.Modify(a => a.Settings.IsLocked, true)
.Modify(a => a.Settings.QuotePolicy, QuotePolicies.Nobody)
.ExecuteAsync(TestContext.Current.CancellationToken);
using var client = _host.As(persona.Root.Jwt);
var response = await client.PostJson("/clientapi/avatar/private/update", new
{
avatarId = persona.Id,
name = "Kept",
biography = "same settings",
settings = new { isDefault = false, darkThemeIndexColour = 100, themeIsDarkMode = true }
});
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var settings = (await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Settings;
Assert.True(settings.ThemeIsDarkMode);
Assert.Equal(100, settings.DarkThemeIndexColour);
Assert.False(settings.IsDiscoverable);
Assert.True(settings.IsLocked);
Assert.Equal(QuotePolicies.Nobody, settings.QuotePolicy);
}
[Fact]
public async Task Updating_another_roots_persona_is_refused()
{
var persona = await _host.Persona(await _host.SignUp("mine"), "mine");
var intruder = await _host.SignUp("intruder");
using var client = _host.As(intruder.Jwt);
var response = await client.PostJson("/clientapi/avatar/private/update", new { avatarId = persona.Id, name = "Hijacked", biography = "testing" });
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
Assert.NotEqual("Hijacked", (await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Name);
}
[Fact]
public async Task Published_days_and_ids_fall_within_two_weeks_before_creation()
{
var created = DateTime.UtcNow;
var persona = await _host.Persona(await _host.SignUp("published"), "published");
var group = await _host.Group(persona, community: true);
using var client = _host.Client();
using var request = new HttpRequestMessage(HttpMethod.Get, $"/peasants/{persona.UserName}");
request.Headers.Accept.ParseAdd("application/activity+json");
var avatar = await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken);
var stored = await DB.Default.Find<GroupEntity>().MatchID(group["id"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
var actor = JsonNode.Parse(await (await client.SendAsync(request, TestContext.Current.CancellationToken)).Content.ReadAsStringAsync(TestContext.Current.CancellationToken))!;
WithinTwoWeeksBefore(created, avatar.PublishedOn, avatar.ID);
WithinTwoWeeksBefore(created, stored.PublishedOn, stored.ID);
var published = DateTime.Parse(actor["published"]!.GetValue<string>(), CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal);
Assert.Equal(avatar.PublishedOn, published);
Assert.DoesNotContain(avatar.CreatedAt.ToString("yyyy-MM-ddTHH:mm:ss", CultureInfo.InvariantCulture), actor.ToJsonString());
}
[Fact]
public async Task The_list_holds_only_the_roots_own_personas()
{
var root = await _host.SignUp("list");
var other = await _host.SignUp("list");
var first = await _host.Persona(root, "first");
var second = await _host.Persona(root, "second");
var theirs = await _host.Persona(other, "theirs");
Assert.Equal(new[] { first.Id, second.Id }.Order(), (await Listed(root)).Order());
Assert.Equal(new[] { theirs.Id }, await Listed(other));
}
}
}
+579
View File
@@ -0,0 +1,579 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Security.Cryptography;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.Host.FederationHelpers;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class FederationGetTests : IAsyncLifetime
{
const string Base = PrivaPubHost.Base;
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
[Fact]
public async Task The_actor_document_is_activity_json_with_an_spki_key_on_the_actor()
{
var persona = await _host.Persona(await _host.SignUp(), "actor");
var avatar = await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteSingleAsync(TestContext.Current.CancellationToken);
var fetched = await _client.Fetch($"/peasants/{persona.UserName}");
Assert.Equal(HttpStatusCode.OK, fetched.Status);
Assert.Equal(ActivityJson, fetched.MediaType);
Assert.Contains("Accept", fetched.Response.Headers.Vary);
var actor = fetched.Json;
var id = persona.ActorUri();
Assert.Equal(id, actor["id"]!.GetValue<string>());
Assert.Equal("Person", actor["type"]!.GetValue<string>());
Assert.Equal(persona.UserName, actor["preferredUsername"]!.GetValue<string>());
Assert.Equal(id + "/mouth", actor["inbox"]!.GetValue<string>());
Assert.Equal(id + "/anus", actor["outbox"]!.GetValue<string>());
Assert.Equal(id + "/groupies", actor["followers"]!.GetValue<string>());
Assert.Equal(id + "/stalking", actor["following"]!.GetValue<string>());
Assert.Equal(id + "/trophies", actor["featured"]!.GetValue<string>());
Assert.Equal(id + "/tattoos", actor["featuredTags"]!.GetValue<string>());
Assert.Equal($"{Base}/@{persona.UserName}", actor["url"]!.GetValue<string>());
Assert.Equal(Base + "/human-centipede", actor["endpoints"]!["sharedInbox"]!.GetValue<string>());
var key = actor["publicKey"]!;
Assert.Equal(id + "#main-key", key["id"]!.GetValue<string>());
Assert.Equal(id, key["owner"]!.GetValue<string>());
var pem = key["publicKeyPem"]!.GetValue<string>();
Assert.StartsWith("-----BEGIN PUBLIC KEY-----", pem);
using var rsa = RSA.Create();
rsa.ImportFromPem(pem);
Assert.True(rsa.KeySize >= 2048);
Assert.Equal(ActivityPubRenderer.Day(avatar.PublishedOn), actor["published"]!.GetValue<string>());
Assert.EndsWith("T00:00:00Z", actor["published"]!.GetValue<string>());
Assert.DoesNotContain(Keys(actor), k => k.Contains("created", StringComparison.OrdinalIgnoreCase));
Assert.DoesNotContain(ActivityPubRenderer.Timestamp(avatar.CreatedAt)[..16], fetched.Text);
Assert.DoesNotContain(persona.Root.Id, fetched.Text);
Assert.DoesNotContain(persona.Root.UserName, fetched.Text);
}
[Fact]
public async Task The_actor_answers_ld_json_and_sends_browsers_to_the_profile_page()
{
var persona = await _host.Persona(await _host.SignUp(), "browser");
var ld = await _client.Fetch($"/peasants/{persona.UserName}", LdJson);
var browser = await _client.Fetch($"/peasants/{persona.UserName}", Browser);
var instance = await _client.Fetch("/peasants/privapub", Browser);
Assert.Equal(HttpStatusCode.OK, ld.Status);
Assert.Equal(persona.ActorUri(), ld.Json["id"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
Assert.Equal($"{Base}/@{persona.UserName}", browser.Location);
Assert.Contains("Accept", browser.Response.Headers.Vary);
Assert.Equal(HttpStatusCode.OK, instance.Status);
Assert.Equal("Application", instance.Json["type"]!.GetValue<string>());
}
// PieFed reads the inbox it sends a community's announces to from the Application at a peer's root
[Fact]
public async Task The_instance_actor_answers_at_the_root_for_activitypub_only()
{
var root = await _client.Fetch("/");
var browser = await _client.Fetch("/", Browser);
Assert.Equal(HttpStatusCode.OK, root.Status);
Assert.Equal("Application", root.Json["type"]!.GetValue<string>());
Assert.Equal($"{Base}/peasants/privapub", root.Json["id"]!.GetValue<string>());
Assert.Equal($"{Base}/peasants/privapub/mouth", root.Json["inbox"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.NotFound, browser.Status);
}
// Lemmy takes a report only from a Person, a Service or an Organization, whose document names its preferredUsername,
// inbox, outbox and key; a browser gets the document too, as from the instance actor: the reporter has no page
[Fact]
public async Task The_reporter_is_a_service_with_an_inbox_an_outbox_and_its_own_key()
{
var fetched = await _client.Fetch("/peasants/privapub_reports");
var browser = await _client.Fetch("/peasants/privapub_reports", Browser);
var outbox = await _client.Fetch("/peasants/privapub_reports/anus");
var instance = await _client.Fetch("/peasants/privapub");
Assert.Equal(HttpStatusCode.OK, fetched.Status);
var actor = fetched.Json;
var id = $"{Base}/peasants/privapub_reports";
Assert.Equal(id, actor["id"]!.GetValue<string>());
Assert.Equal("Service", actor["type"]!.GetValue<string>());
Assert.Equal("privapub_reports", actor["preferredUsername"]!.GetValue<string>());
Assert.Equal(id + "/mouth", actor["inbox"]!.GetValue<string>());
Assert.Equal(id + "/anus", actor["outbox"]!.GetValue<string>());
Assert.Equal(id, actor["url"]!.GetValue<string>());
Assert.Equal(id + "#main-key", actor["publicKey"]!["id"]!.GetValue<string>());
Assert.Equal(id, actor["publicKey"]!["owner"]!.GetValue<string>());
Assert.NotEqual(instance.Json["publicKey"]!["publicKeyPem"]!.GetValue<string>(), actor["publicKey"]!["publicKeyPem"]!.GetValue<string>());
Assert.False(actor["discoverable"]!.GetValue<bool>());
Assert.Null(actor["wall"]);
Assert.Null(actor["implements"]);
Assert.Equal(HttpStatusCode.OK, browser.Status);
Assert.Equal("Service", browser.Json["type"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.OK, outbox.Status);
Assert.Equal(0, outbox.Json["totalItems"]!.GetValue<int>());
}
[Fact]
public async Task Users_moves_permanently_to_peasants_and_unknown_actors_are_404()
{
var persona = await _host.Persona(await _host.SignUp(), "users");
var moved = await _client.Fetch($"/users/{persona.UserName}");
var unknown = await _client.Fetch($"/peasants/nobody{Guid.NewGuid():N}"[..28]);
Assert.Equal(HttpStatusCode.MovedPermanently, moved.Status);
Assert.Equal($"/peasants/{persona.UserName}", moved.Location);
Assert.Equal(HttpStatusCode.NotFound, unknown.Status);
}
[Fact]
public async Task The_outbox_pages_public_posts_and_boosts_and_nothing_else()
{
var persona = await _host.Persona(await _host.SignUp(), "outbox");
var friend = await _host.Persona(await _host.SignUp(), "friend");
var boosted = await _host.Publish(friend, "worth a boost");
var shown = new List<Post>();
for (var i = 0; i < 20; i++)
shown.Add(await _host.Publish(persona, $"public number {i}"));
shown.Add(await _host.Publish(persona, "quietly public", PostVisibility.Unlisted));
var hidden = new List<Post>
{
await _host.Publish(persona, "for followers", PostVisibility.FollowersOnly),
await _host.Publish(persona, $"@{friend.UserName} just you", PostVisibility.Direct),
await _host.PublishLocated(persona, "only around here"),
await _host.FederatedCopy(persona)
};
var deleted = await _host.Publish(persona, "regretted");
await _host.Remove(persona, deleted);
hidden.Add(deleted);
var reblog = await _host.Reblog(persona, boosted);
Assert.Equal(PostVisibility.LocalGeo, hidden[2].Visibility);
var outbox = await _client.Fetch($"/peasants/{persona.UserName}/anus");
Assert.Equal(HttpStatusCode.OK, outbox.Status);
Assert.Equal(ActivityJson, outbox.MediaType);
Assert.Equal("OrderedCollection", outbox.Json["type"]!.GetValue<string>());
Assert.Equal(persona.ActorUri() + "/anus", outbox.Json["id"]!.GetValue<string>());
// statuses_count, as Mastodon counts it: everything but the DM, the deleted post and the copy of a remote post,
// the followers-only and located posts and the boost included; only the public posts are listed
Assert.Equal(24, outbox.Json["totalItems"]!.GetValue<int>());
Assert.Null(outbox.Json["orderedItems"]);
var first = outbox.Json["first"]!.GetValue<string>();
Assert.Equal(persona.ActorUri() + "/anus?page=true", first);
var page1 = await _client.Fetch(PathOf(first));
Assert.Equal(HttpStatusCode.OK, page1.Status);
Assert.Equal("OrderedCollectionPage", page1.Json["type"]!.GetValue<string>());
Assert.Equal(persona.ActorUri() + "/anus", page1.Json["partOf"]!.GetValue<string>());
var items1 = page1.Json["orderedItems"]!.AsArray().Select(i => i!.AsObject()).ToList();
Assert.Equal(20, items1.Count);
var next = page1.Json["next"]!.GetValue<string>();
Assert.StartsWith(persona.ActorUri() + "/anus?page=true&max_id=", next);
var page2 = await _client.Fetch(PathOf(next));
Assert.Equal(HttpStatusCode.OK, page2.Status);
Assert.Equal(next, page2.Json["id"]!.GetValue<string>());
var items2 = page2.Json["orderedItems"]!.AsArray().Select(i => i!.AsObject()).ToList();
Assert.Equal(2, items2.Count);
Assert.Null(page2.Json["next"]);
var items = items1.Concat(items2).ToList();
Assert.Equal(22, items.Select(i => i["id"]!.GetValue<string>()).Distinct().Count());
var announce = items[0];
Assert.Equal("Announce", announce["type"]!.GetValue<string>());
Assert.Equal(reblog.ObjectURI, announce["id"]!.GetValue<string>());
Assert.Equal($"{persona.ActorUri()}/grunts/announce-{reblog.ID}", announce["id"]!.GetValue<string>());
Assert.Equal(boosted.ObjectURI, announce["object"]!.GetValue<string>());
Assert.Equal(persona.ActorUri(), announce["actor"]!.GetValue<string>());
var creates = items.Skip(1).ToList();
Assert.All(creates, c =>
{
Assert.Equal("Create", c["type"]!.GetValue<string>());
Assert.Equal(persona.ActorUri(), c["actor"]!.GetValue<string>());
Assert.Equal(persona.ActorUri(), c["object"]!["attributedTo"]!.GetValue<string>());
});
Assert.Equal(shown.Select(p => p.ObjectURI).Reverse(), creates.Select(c => c["object"]!["id"]!.GetValue<string>()));
var everything = page1.Text + page2.Text;
foreach (var post in hidden)
Assert.DoesNotContain(post.ID, everything);
var junk = await _client.Fetch($"/peasants/{persona.UserName}/anus?page=true&max_id=not-an-id");
Assert.True((int)junk.Status < 500, $"a junk max_id answered {(int)junk.Status}");
}
[Fact]
public async Task Followers_and_following_give_no_names_away()
{
var token = TestContext.Current.CancellationToken;
var persona = await _host.Persona(await _host.SignUp(), "groupies");
var friend = await _host.Persona(await _host.SignUp(), "stalked");
var accepted = $"https://remote.example/users/fan{Guid.NewGuid():N}";
var pending = $"https://remote.example/users/hopeful{Guid.NewGuid():N}";
await DB.Default.SaveAsync(new[]
{
new Follower { LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = accepted, InboxURL = accepted + "/inbox" },
new Follower { LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = pending, InboxURL = pending + "/inbox", IsAccepted = false }
}, token);
await DB.Default.SaveAsync(new Following
{
AvatarId = persona.Id,
TargetAccountId = friend.Id,
TargetActorURI = friend.ActorUri(),
TargetIsLocal = true,
State = FollowState.Accepted
}, token);
var followers = await _client.Fetch($"/peasants/{persona.UserName}/groupies");
var following = await _client.Fetch($"/peasants/{persona.UserName}/stalking");
Assert.Equal(HttpStatusCode.OK, followers.Status);
Assert.Equal("OrderedCollection", followers.Json["type"]!.GetValue<string>());
Assert.Equal(persona.ActorUri() + "/groupies", followers.Json["id"]!.GetValue<string>());
Assert.Equal(1, followers.Json["totalItems"]!.GetValue<int>());
Assert.DoesNotContain(accepted, followers.Text);
Assert.DoesNotContain(pending, followers.Text);
Assert.Equal(HttpStatusCode.OK, following.Status);
Assert.Equal(persona.ActorUri() + "/stalking", following.Json["id"]!.GetValue<string>());
Assert.Equal(1, following.Json["totalItems"]!.GetValue<int>());//counted, as following_count says, never listed
Assert.DoesNotContain(friend.UserName, following.Text);
}
[Fact]
public async Task Trophies_are_the_public_pins_newest_first_and_tattoos_is_empty()
{
var token = TestContext.Current.CancellationToken;
var persona = await _host.Persona(await _host.SignUp(), "trophies");
var older = await _host.Publish(persona, "pinned first");
var newer = await _host.Publish(persona, "pinned second");
var followersOnly = await _host.Publish(persona, "pinned but private", PostVisibility.FollowersOnly);
var removed = await _host.Publish(persona, "pinned then deleted");
foreach (var post in new[] { older, followersOnly, removed, newer })
await DB.Default.SaveAsync(new Pin { AvatarId = persona.Id, PostId = post.ID }, token);
await _host.Remove(persona, removed);
var featured = await _client.Fetch($"/peasants/{persona.UserName}/trophies");
var tags = await _client.Fetch($"/peasants/{persona.UserName}/tattoos");
Assert.Equal(HttpStatusCode.OK, featured.Status);
Assert.Equal(persona.ActorUri() + "/trophies", featured.Json["id"]!.GetValue<string>());
Assert.Equal(2, featured.Json["totalItems"]!.GetValue<int>());
var notes = featured.Json["orderedItems"]!.AsArray();
Assert.Equal(new[] { newer.ObjectURI, older.ObjectURI }, notes.Select(n => n!["id"]!.GetValue<string>()));
Assert.All(notes, n => Assert.Equal("Note", n!["type"]!.GetValue<string>()));
Assert.DoesNotContain(followersOnly.ID, featured.Text);
Assert.DoesNotContain(removed.ID, featured.Text);
Assert.Equal(HttpStatusCode.OK, tags.Status);
Assert.Equal(persona.ActorUri() + "/tattoos", tags.Json["id"]!.GetValue<string>());
Assert.Equal(0, tags.Json["totalItems"]!.GetValue<int>());
Assert.Empty(tags.Json["orderedItems"]!.AsArray());
}
[Fact]
public async Task A_scribble_is_served_while_public_and_a_tombstone_once_deleted()
{
var persona = await _host.Persona(await _host.SignUp(), "scribble");
var other = await _host.Persona(await _host.SignUp(), "other");
var shown = await _host.Publish(persona, "hello fediverse");
var unlisted = await _host.Publish(persona, "hello quietly", PostVisibility.Unlisted);
var followersOnly = await _host.Publish(persona, "hello followers", PostVisibility.FollowersOnly);
var direct = await _host.Publish(persona, $"@{other.UserName} hello you", PostVisibility.Direct);
var located = await _host.PublishLocated(persona, "hello neighbours");
var deleted = await _host.Publish(persona, "goodbye");
await _host.Remove(persona, deleted);
string Path(Post post) => $"/peasants/{persona.UserName}/scribbles/{post.ID}";
var note = await _client.Fetch(Path(shown));
Assert.Equal(HttpStatusCode.OK, note.Status);
Assert.Equal(ActivityJson, note.MediaType);
Assert.Equal(shown.ObjectURI, note.Json["id"]!.GetValue<string>());
Assert.Equal("Note", note.Json["type"]!.GetValue<string>());
Assert.Equal(persona.ActorUri(), note.Json["attributedTo"]!.GetValue<string>());
Assert.NotNull(note.Json["@context"]);
Assert.Contains(ActivityPubRenderer.Public, Strings(note.Json["to"]));
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(Path(unlisted))).Status);
var browser = await _client.Fetch(Path(shown), Browser);
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
Assert.Equal($"{Base}/@{persona.UserName}/{shown.ID}", browser.Location);
foreach (var post in new[] { followersOnly, direct, located })
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(Path(post))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch($"/peasants/{other.UserName}/scribbles/{shown.ID}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch($"/peasants/{persona.UserName}/scribbles/{Guid.NewGuid():N}"[..48])).Status);
var tombstone = await _client.Fetch(Path(deleted));
Assert.Equal(HttpStatusCode.Gone, tombstone.Status);
Assert.Equal(ActivityJson, tombstone.MediaType);
Assert.Equal("Tombstone", tombstone.Json["type"]!.GetValue<string>());
Assert.Equal(deleted.ObjectURI, tombstone.Json["id"]!.GetValue<string>());
Assert.Equal("Note", tombstone.Json["formerType"]!.GetValue<string>());
Assert.NotNull(tombstone.Json["deleted"]);
Assert.DoesNotContain("goodbye", tombstone.Text);
}
[Fact]
public async Task A_circle_post_is_served_only_to_a_member_or_its_instance_actor()
{
var owner = await _host.Persona(await _host.SignUp(), "circleowner");
var circle = await _host.FederatedGroup(owner, community: false);
var member = new RemoteActor(_peer, "member");
var memberServer = new RemoteActor(_peer, "instance", type: "Application");
var neighbour = new RemoteActor(_peer, "neighbour");
var stranger = new RemoteActor(_peer, "stranger", _peer.B);
var strangerServer = new RemoteActor(_peer, "instance", _peer.B, "Application");
await circle.AddForeignMember(member.Id);
var post = await _host.Publish(owner, new StatusDraft { Text = "just us", PlainText = true, GroupId = circle.Id });
Assert.Equal(PostVisibility.Circle, post.Visibility);
var path = $"/peasants/{owner.UserName}/scribbles/{post.ID}";
var asMember = await _client.Fetch(member.SignedGet(path));
var asMemberServer = await _client.Fetch(memberServer.SignedGet(path));
Assert.Equal(HttpStatusCode.OK, asMember.Status);
Assert.Equal(post.ObjectURI, asMember.Json["id"]!.GetValue<string>());
Assert.Equal(new[] { circle.Uri, circle.Uri + "/flock" }, Strings(asMember.Json["to"]));
Assert.DoesNotContain(ActivityPubRenderer.Public, asMember.Text);
Assert.Equal(HttpStatusCode.OK, asMemberServer.Status);
// a refetch names the member, as the member's copy did; the instance actor sees the members on its server only
Assert.Equal(new[] { member.Id }, Strings(asMember.Json["cc"]));
Assert.Equal(new[] { member.Id }, Strings(asMemberServer.Json["cc"]));
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path)).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path, Browser)).Status);
foreach (var outsider in new[] { neighbour, stranger, strangerServer })
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(outsider.SignedGet(path))).Status);
var create = $"/peasants/{owner.UserName}/grunts/create-{post.ID}";
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(create)).Status);
var createAsMember = await _client.Fetch(member.SignedGet(create));
Assert.Equal(HttpStatusCode.OK, createAsMember.Status);
Assert.Equal(post.ObjectURI, createAsMember.Json["object"]!["id"]!.GetValue<string>());
Assert.Equal(new[] { member.Id }, Strings(createAsMember.Json["object"]!["cc"]));
}
[Fact]
public async Task Followers_only_and_direct_posts_are_served_to_signed_fetches_from_those_they_were_for()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "private");
var follower = new RemoteActor(_peer, "follower");
var followerServer = new RemoteActor(_peer, "instance", type: "Application");
var recipient = new RemoteActor(_peer, "recipient");
var stranger = new RemoteActor(_peer, "stranger", _peer.B);
_peer.WebFinger(recipient);
await DB.Default.SaveAsync(new Follower
{
LocalActorId = owner.Id,
LocalActorKind = LocalActorKind.Person,
ActorURI = follower.Id,
InboxURL = follower.Id + "/inbox"
}, token);
var quiet = await _host.Publish(owner, "for followers", PostVisibility.FollowersOnly);
var direct = await _host.Publish(owner, new StatusDraft { Text = $"@{recipient.Handle()} just you", PlainText = true, Visibility = PostVisibility.Direct });
var quietPath = $"/peasants/{owner.UserName}/scribbles/{quiet.ID}";
var directPath = $"/peasants/{owner.UserName}/scribbles/{direct.ID}";
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(follower.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(followerServer.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status);
var asRecipient = await _client.Fetch(recipient.SignedGet(directPath));
Assert.Equal(HttpStatusCode.OK, asRecipient.Status);
Assert.Contains(recipient.Id, Strings(asRecipient.Json["to"]));
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(follower.SignedGet(directPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(directPath)).Status);
// the DM's context is the conversation, for its participants only
var context = PathOf(asRecipient.Json["context"]!.GetValue<string>());
var conversation = await _client.Fetch(recipient.SignedGet(context));
Assert.Equal(HttpStatusCode.OK, conversation.Status);
Assert.Equal(new[] { direct.ObjectURI }, Strings(conversation.Json["orderedItems"]));
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(context))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(context)).Status);
// once deleted, those it was for learn it is gone; everyone else still learns nothing
await _host.Remove(owner, quiet);
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(follower.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status);
}
[Fact]
public async Task A_circles_followers_members_and_wardens_are_shown_only_to_its_members()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "circlelists");
var circle = await _host.FederatedGroup(owner, community: false);
var community = await _host.FederatedGroup(owner, community: true);
var member = new RemoteActor(_peer, "member");
var stranger = new RemoteActor(_peer, "stranger", _peer.B);
await circle.AddForeignMember(member.Id);
foreach (var group in new[] { circle, community })
await DB.Default.SaveAsync(new Follower
{
LocalActorId = group.Id,
LocalActorKind = LocalActorKind.Group,
ActorURI = member.Id,
InboxURL = member.Id + "/inbox"
}, token);
foreach (var collection in new[] { "groupies", "flock", "wardens" })
{
var path = $"/peasants/{circle.UserName}/{collection}";
Assert.True((await _client.Fetch(path)).Status == HttpStatusCode.NotFound, $"the circle's {collection} answered an unsigned GET");
Assert.True((await _client.Fetch(stranger.SignedGet(path))).Status == HttpStatusCode.NotFound, $"the circle's {collection} answered a stranger");
var asMember = await _client.Fetch(member.SignedGet(path));
Assert.True(asMember.Status == HttpStatusCode.OK, $"the circle's {collection} refused a member");
Assert.Equal(circle.Uri + "/" + collection, asMember.Json["id"]!.GetValue<string>());
var open = await _client.Fetch($"/peasants/{community.UserName}/{collection}");
Assert.True(open.Status == HttpStatusCode.OK, $"the community's {collection} answered {(int)open.Status}");
}
Assert.Equal(1, (await _client.Fetch(member.SignedGet($"/peasants/{circle.UserName}/groupies"))).Json["totalItems"]!.GetValue<int>());
Assert.Equal(2, (await _client.Fetch(member.SignedGet($"/peasants/{circle.UserName}/flock"))).Json["totalItems"]!.GetValue<int>());
var wardens = await _client.Fetch($"/peasants/{community.UserName}/wardens");
Assert.Equal(new[] { owner.ActorUri() }, Strings(wardens.Json["orderedItems"]));
Assert.Equal(community.Uri + "/wardens", (await _client.Fetch($"/peasants/{community.UserName}")).Json["attributedTo"]!.GetValue<string>());
Assert.Null((await _client.Fetch($"/peasants/{circle.UserName}")).Json["attributedTo"]);
}
[Fact]
public async Task Grunts_resolve_creates_and_boosts()
{
var persona = await _host.Persona(await _host.SignUp(), "grunts");
var friend = await _host.Persona(await _host.SignUp(), "boosted");
var post = await _host.Publish(persona, "a grunt");
var followersOnly = await _host.Publish(persona, "a private grunt", PostVisibility.FollowersOnly);
var original = await _host.Publish(friend, "boost me");
var reblog = await _host.Reblog(persona, original);
string Path(string id) => $"/peasants/{persona.UserName}/grunts/{id}";
var create = await _client.Fetch(Path($"create-{post.ID}"));
var announce = await _client.Fetch(Path($"announce-{reblog.ID}"));
Assert.Equal(HttpStatusCode.OK, create.Status);
Assert.Equal(ActivityJson, create.MediaType);
Assert.Equal("Create", create.Json["type"]!.GetValue<string>());
Assert.Equal(Base + Path($"create-{post.ID}"), create.Json["id"]!.GetValue<string>());
Assert.Equal(post.ObjectURI, create.Json["object"]!["id"]!.GetValue<string>());
Assert.Equal(post.ActivityURI, create.Json["id"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.OK, announce.Status);
Assert.Equal("Announce", announce.Json["type"]!.GetValue<string>());
Assert.Equal(reblog.ObjectURI, announce.Json["id"]!.GetValue<string>());
Assert.Equal(original.ObjectURI, announce.Json["object"]!.GetValue<string>());
foreach (var unknown in new[] { $"create-{followersOnly.ID}", $"create-{Guid.NewGuid():N}"[..31], $"announce-{post.ID}", $"like-{post.ID}", "x" })
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(Path(unknown))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(Path($"announce-{original.ID}"))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch($"/peasants/{friend.UserName}/grunts/announce-{reblog.ID}")).Status);
}
[Fact]
public async Task A_parrot_licence_is_served_by_its_author_until_revoked()
{
var token = TestContext.Current.CancellationToken;
var quoted = await _host.Persona(await _host.SignUp(), "quoted");
var quoter = await _host.Persona(await _host.SignUp(), "quoter");
var original = await _host.Publish(quoted, "quote me");
var quoting = await _host.Publish(quoter, new StatusDraft { Text = "well said", PlainText = true, QuotedStatusId = original.ID });
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
var licenceUri = quoting.QuoteAuthorizationURI;
Assert.StartsWith(quoted.ActorUri() + "/parrot-licences/", licenceUri);
var licenceId = licenceUri.Split('/')[^1];
var licence = await _client.Fetch(PathOf(licenceUri));
var wrongAuthor = await _client.Fetch($"/peasants/{quoter.UserName}/parrot-licences/{licenceId}");
var unknown = await _client.Fetch($"/peasants/{quoted.UserName}/parrot-licences/{Guid.NewGuid():N}"[..48]);
Assert.Equal(HttpStatusCode.OK, licence.Status);
Assert.Equal(ActivityJson, licence.MediaType);
Assert.Equal("QuoteAuthorization", licence.Json["type"]!.GetValue<string>());
Assert.Equal(licenceUri, licence.Json["id"]!.GetValue<string>());
Assert.Equal(quoted.ActorUri(), licence.Json["attributedTo"]!.GetValue<string>());
Assert.Equal(quoting.ObjectURI, licence.Json["interactingObject"]!.GetValue<string>());
Assert.Equal(original.ObjectURI, licence.Json["interactionTarget"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.NotFound, wrongAuthor.Status);
Assert.Equal(HttpStatusCode.NotFound, unknown.Status);
Assert.True(await _host.Get<IQuoteService>().RevokeLicence(original, quoting, token));
var revoked = await _client.Fetch(PathOf(licenceUri));
Assert.Equal(HttpStatusCode.Gone, revoked.Status);
Assert.Equal(ActivityJson, revoked.MediaType);
Assert.Equal("Tombstone", revoked.Json["type"]!.GetValue<string>());
Assert.Equal("QuoteAuthorization", revoked.Json["formerType"]!.GetValue<string>());
Assert.Equal(licenceUri, revoked.Json["id"]!.GetValue<string>());
}
[Fact]
public async Task Secure_mode_refuses_unsigned_gets_except_for_the_servers_own_actors()
{
var secure = await SecureModeHost.Shared();
using var client = secure.Client();
var persona = await _host.Persona(await _host.SignUp(), "secure");
var post = await _host.Publish(persona, "only for the signed");
var reader = new RemoteActor(_peer, "reader");
var paths = new[]
{
$"/peasants/{persona.UserName}",
$"/peasants/{persona.UserName}/anus",
$"/peasants/{persona.UserName}/anus?page=true",
$"/peasants/{persona.UserName}/groupies",
$"/peasants/{persona.UserName}/stalking",
$"/peasants/{persona.UserName}/trophies",
$"/peasants/{persona.UserName}/tattoos",
$"/peasants/{persona.UserName}/scribbles/{post.ID}",
$"/peasants/{persona.UserName}/grunts/create-{post.ID}",
$"/peasants/nobody{Guid.NewGuid():N}"[..28]
};
foreach (var path in paths)
Assert.True((await client.Fetch(path)).Status == HttpStatusCode.Unauthorized, $"{path} answered an unsigned GET");
var instance = await client.Fetch("/peasants/privapub");
Assert.Equal(HttpStatusCode.OK, instance.Status);
Assert.Equal("Application", instance.Json["type"]!.GetValue<string>());
var reporter = await client.Fetch("/peasants/privapub_reports");
Assert.Equal(HttpStatusCode.OK, reporter.Status);
Assert.Equal("Service", reporter.Json["type"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.OK, (await client.Fetch("/")).Status);
foreach (var path in paths[..^1])
Assert.True((await client.Fetch(reader.SignedGet(path))).Status == HttpStatusCode.OK, $"{path} refused a signed GET");
// a browser is not asked for a signature: it is only sent to the public pages
var browser = await client.Fetch($"/peasants/{persona.UserName}", Browser);
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
Assert.Equal(HttpStatusCode.Redirect, (await client.Fetch($"/peasants/{persona.UserName}/scribbles/{post.ID}", Browser)).Status);
}
}
}
+205
View File
@@ -0,0 +1,205 @@
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Globalization;
using System.Net;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class InboxRouteTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
Persona _persona;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
_persona = await _host.Persona(await _host.SignUp(), "inbox");
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
public static TheoryData<string> Inboxes() => new() { "personal", "personal-shared", "shared" };
string Route(string inbox) => inbox switch
{
"personal" => $"/peasants/{_persona.UserName}/mouth",
"personal-shared" => $"/peasants/{_persona.UserName}/human-centipede",
_ => "/human-centipede"
};
JsonObject Direct(RemoteActor sender)
{
var noteId = $"{new Uri(sender.Id).GetLeftPart(UriPartial.Authority)}/notes/{Guid.NewGuid():N}";
var to = new JsonArray($"{PrivaPubHost.Base}/peasants/{_persona.UserName}");
return new JsonObject
{
["id"] = noteId + "/activity",
["type"] = "Create",
["actor"] = sender.Id,
["to"] = to.DeepClone(),
["object"] = new JsonObject
{
["id"] = noteId,
["type"] = "Note",
["attributedTo"] = sender.Id,
["to"] = to.DeepClone(),
["content"] = "<p>knock knock</p>",
["published"] = DateTime.UtcNow.ToString("O")
}
};
}
static string Id(JsonObject activity) => activity["id"]!.GetValue<string>();
async Task<bool> Queued(JsonObject activity) =>
await DB.Default.Find<Job>().Match(j => j.DedupeKey == "inbox|" + Id(activity)).ExecuteAnyAsync(TestContext.Current.CancellationToken);
async Task<HttpResponseMessage> Send(HttpRequestMessage request) => await _client.SendAsync(request, TestContext.Current.CancellationToken);
[Theory]
[MemberData(nameof(Inboxes))]
public async Task A_signed_delivery_is_accepted_and_queued(string inbox)
{
var sender = new RemoteActor(_peer, "sender");
var activity = Direct(sender);
var response = await Send(sender.SignedPost(Route(inbox), activity));
Assert.Equal(HttpStatusCode.Accepted, response.StatusCode);
Assert.Equal(1, await _host.RunInbox(Id(activity), TestContext.Current.CancellationToken));
}
[Theory]
[MemberData(nameof(Inboxes))]
public async Task Junk_is_400_and_an_unsigned_activity_401(string inbox)
{
var sender = new RemoteActor(_peer, "unsigned");
foreach (var junk in new[] { "not json", "[1,2,3]", "\"Create\"", "{}", "{\"type\":\"Create\"}" })
{
using var request = new HttpRequestMessage(HttpMethod.Post, Route(inbox)) { Content = new StringContent(junk, Encoding.UTF8, "application/activity+json") };
Assert.Equal(HttpStatusCode.BadRequest, (await Send(request)).StatusCode);
}
var activity = Direct(sender);
using var unsigned = new HttpRequestMessage(HttpMethod.Post, Route(inbox))
{
Content = new StringContent(activity.ToJsonString(), Encoding.UTF8, "application/activity+json")
};
Assert.Equal(HttpStatusCode.Unauthorized, (await Send(unsigned)).StatusCode);
Assert.False(await Queued(activity));
}
[Fact]
public async Task An_unknown_persona_has_no_mouth()
{
var sender = new RemoteActor(_peer, "lost");
var activity = Direct(sender);
var response = await Send(sender.SignedPost($"/peasants/nobody{Guid.NewGuid():N}"[..28] + "/mouth", activity));
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
Assert.False(await Queued(activity));
}
[Theory]
[MemberData(nameof(Inboxes))]
public async Task A_tampered_stale_or_misdirected_signature_is_401(string inbox)
{
var sender = new RemoteActor(_peer, "forger");
var path = Route(inbox);
var tampered = Direct(sender);
var badDigest = sender.SignedPost(path, tampered);
badDigest.Headers.Remove("Digest");
badDigest.Headers.TryAddWithoutValidation("Digest", "SHA-256=" + Convert.ToBase64String(new byte[32]));
var stale = Direct(sender);
var twoHoursAgo = DateTimeOffset.UtcNow.AddHours(-2).ToString("r", CultureInfo.InvariantCulture);
var elsewhere = Direct(sender);
var otherBody = Direct(sender);
var swapped = sender.SignedPost(path, Direct(sender));
swapped.Content = new StringContent(otherBody.ToJsonString(), Encoding.UTF8, "application/activity+json");
Assert.Equal(HttpStatusCode.Unauthorized, (await Send(badDigest)).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await Send(sender.SignedPost(path, stale, date: twoHoursAgo))).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await Send(sender.SignedPost(path, elsewhere, host: "elsewhere.example"))).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await Send(swapped)).StatusCode);
foreach (var activity in new[] { tampered, stale, elsewhere, otherBody })
Assert.False(await Queued(activity));
}
[Fact]
public async Task Ld_json_with_the_activitystreams_profile_is_accepted()
{
var sender = new RemoteActor(_peer, "ldjson");
var activity = Direct(sender);
var request = sender.SignedPost($"/peasants/{_persona.UserName}/mouth", activity);
request.Content!.Headers.Remove("Content-Type");
request.Content.Headers.TryAddWithoutValidation("Content-Type", "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"");
var response = await Send(request);
Assert.Equal(HttpStatusCode.Accepted, response.StatusCode);
Assert.True(await Queued(activity));
}
[Fact]
public async Task An_unreachable_signing_key_asks_the_sender_to_retry()
{
var sender = new RemoteActor(_peer, "flaky");
_peer.Answer(new Uri(sender.Id).AbsolutePath, 503);
var activity = Direct(sender);
var response = await Send(sender.SignedPost($"/peasants/{_persona.UserName}/mouth", activity));
Assert.Equal(HttpStatusCode.ServiceUnavailable, response.StatusCode);
Assert.True(response.Headers.RetryAfter?.Delta is { TotalSeconds: > 0 }, "no Retry-After");
Assert.False(await Queued(activity));
}
[Fact]
public async Task Unsigned_posts_from_one_address_are_limited()
{
var token = TestContext.Current.CancellationToken;
using var client = _host.Client();
client.DefaultRequestHeaders.Remove(PrivaPubHost.ClientHeader);
client.DefaultRequestHeaders.Add(PrivaPubHost.ClientHeader, $"fd7e:{Random.Shared.Next(0x10000):x}:{Random.Shared.Next(0x10000):x}::1");
var body = Direct(new RemoteActor(_peer, "flood")).ToJsonString();
async Task<HttpStatusCode> Post()
{
using var content = new StringContent(body, Encoding.UTF8, "application/activity+json");
using var response = await client.PostAsync("/human-centipede", content, token);
return response.StatusCode;
}
// The bucket holds 300 and refills 50 every ten seconds on its own timer, so a refill can land at any moment:
// the first refusal comes after at least 300 posts and within a few refills.
var answers = new List<HttpStatusCode>();
while (answers.Count < 600 && (answers.Count == 0 || answers[^1] != HttpStatusCode.TooManyRequests))
answers.Add(await Post());
Assert.Equal(HttpStatusCode.TooManyRequests, answers[^1]);
Assert.True(answers.Count > 300, $"limited after only {answers.Count - 1} posts");
Assert.All(answers.SkipLast(1), status => Assert.Equal(HttpStatusCode.Unauthorized, status));
var polite = new RemoteActor(_peer, "polite");
using var signed = await client.SendAsync(polite.SignedPost("/human-centipede", Direct(polite)), token);
Assert.Equal(HttpStatusCode.Accepted, signed.StatusCode);
}
}
}
@@ -0,0 +1,291 @@
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// Where a server is, on the public instance API: where it was located (city, coordinates and network, whatever its size),
// only the CDN's name for a CDN-fronted one, the batch form, and this server itself.
[Trait("Category", "Integration")]
public sealed class InstanceLocationTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static CancellationToken Token => TestContext.Current.CancellationToken;
static async Task<string> Instance(int users, InstanceGeo geo, string name = "located")
{
var host = $"{name}{Guid.NewGuid():N}.example";
await DB.Default.SaveAsync(new RemoteInstance { Host = host, Software = "mastodon", UsersTotal = users, Geo = geo }, Token);
return host;
}
static InstanceGeo Milan(string cdn = default) => new()
{
Country = "IT",
City = "Milan",
Latitude = 45.5,
Longitude = 9.2,
Asn = 12345,
AsnOrg = "Example Hosting",
Cdn = cdn,
Ipv6 = true,
Source = "DB-IP Lite 2026-10",
LocatedAt = DateTime.UtcNow.AddDays(-2)
};
async Task<JsonNode> Geo(string host) => (await _host.Client().Get($"/api/privapub/v1/instances/{host}")).Ok().Body["geo"];
[Fact]
public async Task A_located_server_shows_its_city_coordinates_and_network()
{
var geo = await Geo(await Instance(50, Milan()));
Assert.Equal("city", geo.Text("precision"));
Assert.Equal(("IT", "Milan", "Example Hosting"), (geo.Text("country"), geo.Text("city"), geo.Text("network")));
Assert.Equal(45.5, geo["latitude"]!.GetValue<double>());
Assert.Equal(9.2, geo["longitude"]!.GetValue<double>());
Assert.Equal("DB-IP Lite 2026-10", geo.Text("source"));
Assert.NotNull(geo.Text("located_at"));
Assert.Null(geo["asn"]);
Assert.Null(geo["ipv6"]);
}
[Fact]
public async Task A_one_user_server_shows_its_city_too()
{
var geo = await Geo(await Instance(1, Milan()));
Assert.Equal("city", geo.Text("precision"));
Assert.Equal(("IT", "Milan", "Example Hosting"), (geo.Text("country"), geo.Text("city"), geo.Text("network")));
Assert.Equal(45.5, geo["latitude"]!.GetValue<double>());
}
[Fact]
public async Task A_server_located_only_to_its_country_shows_the_country()
{
var geo = await Geo(await Instance(50, new InstanceGeo { Country = "IT", Source = "DB-IP Lite 2026-10", LocatedAt = DateTime.UtcNow }));
Assert.Equal("country", geo.Text("precision"));
Assert.Equal("IT", geo.Text("country"));
Assert.Null(geo["city"]);
Assert.Null(geo["latitude"]);
}
[Fact]
public async Task A_cdn_fronted_server_shows_only_its_cdn()
{
var geo = await Geo(await Instance(5000, Milan(cdn: "Cloudflare")));
Assert.Equal("cdn", geo.Text("precision"));
Assert.Equal("Cloudflare", geo.Text("cdn"));
Assert.Null(geo["country"]);
Assert.Null(geo["latitude"]);
Assert.Null(geo["network"]);
}
[Fact]
public async Task An_unlocated_server_has_no_place()
{
Assert.Null(await Geo(await Instance(50, default)));
}
[Fact]
public async Task The_provenance_of_a_post_carries_the_same_projection()
{
var host = await Instance(3, Milan());
var described = (await _host.Client().Get($"/api/privapub/v1/instances/{host}")).Ok().Body;
Assert.Equal("city", described["geo"].Text("precision"));
Assert.Equal(described["geo"]!.ToJsonString(), (await Geo(host))!.ToJsonString());
}
[Fact]
public async Task Several_servers_answer_at_once_in_the_order_asked_without_the_unknown_ones()
{
var big = await Instance(50, Milan(), "big");
var small = await Instance(3, Milan(), "small");
var unknown = $"unknown{Guid.NewGuid():N}.example";
var answer = (await _host.Client().Get($"/api/privapub/v1/instances?host[]={small}&host[]={unknown}&host[]={big.ToUpperInvariant()}&host[]={small}")).Ok();
var hosts = answer.Body!.AsArray().Select(i => i.Text("host")).ToList();
Assert.Equal(new[] { small, big }, hosts);
Assert.Equal("city", answer.Body![0]["geo"].Text("precision"));
Assert.Equal("city", answer.Body![1]["geo"].Text("precision"));
Assert.Empty((await _host.Client().Get("/api/privapub/v1/instances")).Ok().Body!.AsArray());
}
[Fact]
public async Task At_most_forty_servers_answer_at_once()
{
var hosts = new List<string>();
for (var i = 0; i < 42; i++)
hosts.Add(await Instance(1, default, "many"));
var answer = (await _host.Client().Get("/api/privapub/v1/instances?" + string.Join("&", hosts.Select(h => "host[]=" + h)))).Ok();
Assert.Equal(hosts.Take(40), answer.Body!.AsArray().Select(i => i.Text("host")));
}
[Fact]
public async Task This_server_describes_itself()
{
var self = (await _host.Client().Get($"/api/privapub/v1/instances/{PrivaPubHost.Host}")).Ok().Body;
Assert.Equal(PrivaPubHost.Host, self.Text("host"));
Assert.Equal("privapub", self.Text("software"));
Assert.Equal("activitypub", Assert.Single(self["protocols"]!.AsArray())!.GetValue<string>());
Assert.Null(self["geo"]);
var batch = (await _host.Client().Get($"/api/privapub/v1/instances?host[]={PrivaPubHost.Host}")).Ok().Body!.AsArray();
Assert.Equal("privapub", Assert.Single(batch).Text("software"));
}
[Fact]
public async Task The_owner_may_place_this_server()
{
var placed = await SelfPlacedHost.Shared();
var geo = (await placed.Client().Get($"/api/privapub/v1/instances/{PrivaPubHost.Host}")).Ok().Body["geo"];
Assert.Equal("city", geo.Text("precision"));
Assert.Equal(("IT", "Milan", "configured"), (geo.Text("country"), geo.Text("city"), geo.Text("source")));
Assert.Equal(45.5, geo["latitude"]!.GetValue<double>());
Assert.Equal(9.2, geo["longitude"]!.GetValue<double>());
}
static string WeeksAgo(int weeks) => PrivaPub.Federation.Objects.InstanceDescriber.Week(DateTime.UtcNow.AddDays(-7 * weeks));
static Task Snapshot(string host, int weeksAgo, string cdn = default, string cdnDomain = default, string city = default, double? latitude = default,
double? longitude = default, string country = default) =>
DB.Default.SaveAsync(new RemoteInstanceSnapshot
{
Host = host, Week = WeeksAgo(weeksAgo), TakenAt = DateTime.UtcNow.AddDays(-7 * weeksAgo), Reachable = true, Software = "mastodon", UsersTotal = 3,
Cdn = cdn, CdnDomain = cdnDomain, CdnSource = cdn == default ? default : "ranges", Country = country, City = city, Latitude = latitude, Longitude = longitude
}, Token);
[Fact]
public async Task A_cdn_fronted_server_names_its_cdn_and_where_it_was_before_it()
{
var host = await Instance(3, new InstanceGeo
{
Country = "US", City = "San Francisco", Latitude = 37.8, Longitude = -122.4, Cdn = "Cloudflare", CdnDomain = "cloudflare.com", CdnSource = "ranges",
Source = "DB-IP Lite 2026-10", LocatedAt = DateTime.UtcNow
}, "fronted");
await Snapshot(host, 3, city: "Berlin", latitude: 52.5, longitude: 13.4, country: "DE");
await Snapshot(host, 2, city: "Hamburg", latitude: 53.6, longitude: 10.0, country: "DE");
await Snapshot(host, 1, cdn: "Cloudflare", cdnDomain: "cloudflare.com", city: "San Francisco", latitude: 37.8, longitude: -122.4, country: "US");
var geo = await Geo(host);
Assert.Equal(("cdn", "Cloudflare", "cloudflare.com", "ranges"), (geo.Text("precision"), geo.Text("cdn"), geo.Text("cdn_domain"), geo.Text("cdn_source")));
Assert.Null(geo["latitude"]);
Assert.Equal((WeeksAgo(2), "Hamburg", "DE"), (geo["before_cdn"].Text("week"), geo["before_cdn"].Text("city"), geo["before_cdn"].Text("country")));
Assert.Equal(53.6, geo["before_cdn"]!["latitude"]!.GetValue<double>());
}
[Fact]
public async Task A_server_history_is_weekly_newest_first_and_a_cdn_hides_its_place()
{
var host = await Instance(3, Milan(), "historic");
await Snapshot(host, 2, city: "Milan", latitude: 45.5, longitude: 9.2, country: "IT");
await Snapshot(host, 1, cdn: "Fastly", cdnDomain: "fastly.com", city: "Frankfurt", latitude: 50.1, longitude: 8.7, country: "DE");
var weeks = (await _host.Client().Get($"/api/privapub/v1/instances/{host}/history")).Ok().Body!.AsArray();
Assert.Equal(new[] { WeeksAgo(1), WeeksAgo(2) }, weeks.Select(w => w.Text("week")));
Assert.Equal(("cdn", "fastly.com"), (weeks[0]!["geo"].Text("precision"), weeks[0]!["geo"].Text("cdn_domain")));
Assert.Null(weeks[0]!["geo"]!["city"]);
Assert.Equal(("city", "Milan"), (weeks[1]!["geo"].Text("precision"), weeks[1]!["geo"].Text("city")));
Assert.Equal(3, weeks[1]!["users_total"]!.GetValue<long>());
Assert.Single((await _host.Client().Get($"/api/privapub/v1/instances/{host}/history?weeks=1")).Ok().Body!.AsArray());
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/instances/unknown{Guid.NewGuid():N}.example/history")).Status);
}
[Fact]
public async Task Servers_are_grouped_by_their_cdn_week_by_week()
{
var cdn = $"Edge{Guid.NewGuid():N}"[..20];
var key = cdn.ToLowerInvariant();
var stayed = await Instance(3, new InstanceGeo { Country = "FR", Cdn = cdn, CdnSource = "headers" }, "stayed");
var moved = await Instance(3, Milan(), "moved");
await Snapshot(moved, 3, cdn: cdn);
await Snapshot(moved, 2, cdn: cdn);
await Snapshot(stayed, 2, cdn: cdn);
await Snapshot(moved, 1, city: "Milan", latitude: 45.5, longitude: 9.2, country: "IT");
await Snapshot(stayed, 1, cdn: cdn);
var list = (await _host.Client().Get("/api/privapub/v1/cdns")).Ok().Body!.AsArray();
var group = list.Single(g => g.Text("key") == key);
Assert.Equal((cdn, 1), (group.Text("name"), group["servers"]!.GetValue<int>()));
var cloudflare = list.Single(g => g.Text("key") == "cloudflare.com");
Assert.Equal(new[] { "ranges", "headers", "asn" }, cloudflare["detected_by"]!.AsArray().Select(v => v!.GetValue<string>()));
var detail = (await _host.Client().Get($"/api/privapub/v1/cdns/{key}")).Ok().Body;
Assert.Equal(new[] { stayed }, detail["hosts"]!.AsArray().Select(h => h!.GetValue<string>()));
var weeks = detail["weeks"]!.AsArray();
Assert.Equal(new[] { WeeksAgo(1), WeeksAgo(2), WeeksAgo(3) }, weeks.Select(w => w.Text("week")));
Assert.Equal(new[] { 1, 2, 1 }, weeks.Select(w => w!["servers"]!.GetValue<int>()));
Assert.Equal(new[] { moved }, weeks[0]!["left"]!.AsArray().Select(h => h!.GetValue<string>()));
Assert.Equal(new[] { stayed }, weeks[1]!["joined"]!.AsArray().Select(h => h!.GetValue<string>()));
Assert.Equal(new[] { moved }, weeks[2]!["joined"]!.AsArray().Select(h => h!.GetValue<string>()));
Assert.Equal("Cloudflare", (await _host.Client().Get("/api/privapub/v1/cdns/cloudflare.com")).Ok().Body.Text("name"));
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/cdns/none{Guid.NewGuid():N}")).Status);
}
[Fact]
public async Task Junk_hosts_are_not_an_error()
{
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/instances/{new string('a', 300)}")).Status);
Assert.Empty((await _host.Client().Get($"/api/privapub/v1/instances?host[]=&host[]={new string('a', 300)}")).Ok().Body!.AsArray());
}
}
public sealed class SelfPlacedHost : PrivaPubHost
{
static readonly SemaphoreSlim Boot = new(1, 1);
static SelfPlacedHost _shared;
public static new async Task<SelfPlacedHost> Shared()
{
await PrivaPubHost.Shared();
await Boot.WaitAsync();
try
{
if (_shared == default)
{
var host = new SelfPlacedHost();
_ = host.Services;
_shared = host;
}
return _shared;
}
finally
{
Boot.Release();
}
}
protected override IEnumerable<KeyValuePair<string, string>> Settings() => base.Settings().Concat(new Dictionary<string, string>
{
["Statistics:Geo:Self:Latitude"] = "45.4642",
["Statistics:Geo:Self:Longitude"] = "9.19",
["Statistics:Geo:Self:City"] = "Milan",
["Statistics:Geo:Self:Country"] = "IT"
});
}
}
@@ -0,0 +1,672 @@
using MongoDB.Entities;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Net.Http.Json;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class MastodonAccountsTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
async Task<(RemoteActor Actor, string Id)> Remote(string name = "bob")
{
var actor = new RemoteActor(_peer, name);
var known = await _host.Known(actor);
return (actor, known.ID);
}
[Fact]
public async Task Verify_credentials_shows_the_persona_and_its_source_and_nothing_of_the_login()
{
var alice = await _host.Mastodon("alice");
var me = (await alice.Client.Get("/api/v1/accounts/verify_credentials")).Ok();
Assert.Equal(alice.Id, me.Body.Text("id"));
Assert.Equal(alice.UserName, me.Body.Text("username"));
Assert.Equal(alice.UserName, me.Body.Text("acct"));
Assert.Equal(alice.Uri, me.Body.Text("uri"));
Assert.Equal("public", me.Body["source"].Text("privacy"));
Assert.Equal("public", me.Body["source"].Text("quote_policy"));
Assert.Equal(0, me.Body["source"].Number("follow_requests_count"));
Assert.DoesNotContain(alice.Persona.Root.Id, me.Text);
Assert.DoesNotContain(alice.Persona.Root.UserName, me.Text);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get("/api/v1/accounts/verify_credentials")).Status);
}
[Fact]
public async Task Update_credentials_takes_fields_in_both_shapes_the_source_and_the_quote_policy()
{
var alice = await _host.Mastodon("alice");
var updated = (await alice.Client.Patch("/api/v1/accounts/update_credentials",
("display_name", " Alice Liddell "), ("note", "down the hole"), ("locked", "true"), ("bot", "true"),
("fields_attributes[0][name]", "Site"), ("fields_attributes[0][value]", "https://example.com"),
("fields_attributes[1][name]", "Pronouns"), ("fields_attributes[1][value]", "she/her"),
("source[privacy]", "private"), ("source[sensitive]", "true"), ("source[language]", "it"), ("source[quote_policy]", "followers"))).Ok();
Assert.Equal("Alice Liddell", updated.Body.Text("display_name"));
Assert.True(updated.Body.Flag("locked"));
Assert.True(updated.Body.Flag("bot"));
Assert.Equal(new[] { "Site", "Pronouns" }, updated.Body["fields"]!.AsArray().Select(f => f.Text("name")));
Assert.Equal("down the hole", updated.Body["source"].Text("note"));
Assert.Equal("private", updated.Body["source"].Text("privacy"));
Assert.True(updated.Body["source"].Flag("sensitive"));
Assert.Equal("it", updated.Body["source"].Text("language"));
Assert.Equal("followers", updated.Body["source"].Text("quote_policy"));
var arrayShape = (await alice.Client.Json(HttpMethod.Patch, "/api/v1/accounts/update_credentials", new JsonObject
{
["fields_attributes"] = new JsonArray(new JsonObject { ["name"] = "Garden", ["value"] = "roses" }, new JsonObject { ["name"] = "Cat", ["value"] = "Dinah" }),
["source"] = new JsonObject { ["quote_policy"] = "everyone" }
})).Ok();
Assert.Equal(new[] { ("Garden", "roses"), ("Cat", "Dinah") },
arrayShape.Body["source"]!["fields"]!.AsArray().Select(f => (f.Text("name"), f.Text("value"))));
Assert.Equal("followers", arrayShape.Body["source"].Text("quote_policy"));
var formArray = (await alice.Client.Patch("/api/v1/accounts/update_credentials",
("fields_attributes[][name]", "One"), ("fields_attributes[][value]", "1"), ("fields_attributes[][name]", "Two"), ("fields_attributes[][value]", "2"))).Ok();
Assert.Equal(new[] { ("One", "1"), ("Two", "2") }, formArray.Body["source"]!["fields"]!.AsArray().Select(f => (f.Text("name"), f.Text("value"))));
var again = (await alice.Client.Get("/api/v1/accounts/verify_credentials")).Ok();
Assert.Equal(2, again.Body["fields"]!.AsArray().Count);
Assert.Equal("private", again.Body["source"].Text("privacy"));
var status = await alice.Status("with the default visibility");
Assert.Equal("private", status.Text("visibility"));
Assert.True(status.Flag("sensitive"));
Assert.Equal("it", status.Text("language"));
}
[Fact]
public async Task Avatar_and_header_uploads_are_resized_and_lose_their_metadata()
{
var alice = await _host.Mastodon("alice");
var picture = MastodonHelpers.JpegWithMetadata(900, 700);
using (var original = NetVips.Image.NewFromBuffer(picture))
Assert.Contains("exif-data", original.GetFields());
var form = MastodonHelpers.Multipart(("avatar", picture, "image/jpeg", "me.jpg"), ("header", MastodonHelpers.JpegWithMetadata(2000, 900), "image/jpeg", "header.jpg"));
form.Add(new StringContent("Alice"), "display_name");
var updated = (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Patch, "/api/v1/accounts/update_credentials") { Content = form })).Ok();
Assert.Equal("Alice", updated.Body.Text("display_name"));
foreach (var (field, maxWidth, maxHeight) in new[] { ("avatar", 400, 400), ("header", 1500, 500) })
{
var url = updated.Body.Text(field);
Assert.StartsWith($"{PrivaPubHost.Base}/media/files/", url);
var bytes = await _host.Client().GetByteArrayAsync(url, Token);
MastodonHelpers.AssertNoMetadata(bytes);
using var stored = NetVips.Image.NewFromBuffer(bytes);
Assert.True(stored.Width <= maxWidth && stored.Height <= maxHeight, $"{field} is {stored.Width}x{stored.Height}");
}
}
[Fact]
public async Task Lookup_finds_local_and_remote_accounts_but_never_a_circle_or_the_servers_own_actors()
{
var alice = await _host.Mastodon("alice");
var (bob, bobId) = await Remote();
var circleName = $"circle{Guid.NewGuid():N}"[..20];
string circleId;
using (var clientApi = _host.As(alice.Persona.Root.Jwt))
{
var circle = await clientApi.PostAsJsonAsync("/clientapi/group/insert", new { avatarId = alice.Id, userName = circleName, name = "inner", isCommunity = false }, Token);
Assert.Equal(HttpStatusCode.OK, circle.StatusCode);
circleId = (await circle.Content.ReadFromJsonAsync<JsonObject>(Token))!["id"]!.GetValue<string>();
}
var anonymous = _host.Client();
Assert.Equal(alice.Id, (await anonymous.Get($"/api/v1/accounts/lookup?acct={alice.UserName}")).Ok().Body.Text("id"));
Assert.Equal(alice.Id, (await anonymous.Get($"/api/v1/accounts/lookup?acct=@{alice.UserName}@{PrivaPubHost.Host}")).Ok().Body.Text("id"));
var remote = (await anonymous.Get($"/api/v1/accounts/lookup?acct={Uri.EscapeDataString(bob.Handle())}")).Ok();
Assert.Equal(bobId, remote.Body.Text("id"));
Assert.Equal(bob.Handle(), remote.Body.Text("acct"));
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get($"/api/v1/accounts/lookup?acct={circleName}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/accounts/lookup?acct={circleName}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get("/api/v1/accounts/lookup?acct=privapub")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get("/api/v1/accounts/lookup?acct=privapub_reports")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get($"/api/v1/accounts/lookup?acct=nobody{Guid.NewGuid():N}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get("/api/v1/accounts/lookup?acct=a@b@c")).Status);
Assert.Equal(alice.UserName, (await anonymous.Get($"/api/v1/accounts/{alice.Id}")).Ok().Body.Text("username"));
Assert.Equal(bob.Handle(), (await anonymous.Get($"/api/v1/accounts/{bobId}")).Ok().Body.Text("acct"));
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get("/api/v1/accounts/000000000000000000000000")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/accounts/{circleId}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/accounts/{circleId}/statuses")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post($"/api/v1/accounts/{circleId}/follow")).Status);
}
[Fact]
public async Task Search_resolves_a_remote_handle_only_when_asked_by_a_signed_in_persona()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
_peer.WebFinger(bob);
var handle = Uri.EscapeDataString(bob.Handle());
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/search?q={handle}")).Ok().Array);
Assert.Empty((await alice.Client.Get($"/api/v2/search?q={handle}")).Ok().Body["accounts"]!.AsArray());
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get($"/api/v1/accounts/search?q={handle}&resolve=true")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get($"/api/v2/search?q={handle}&resolve=true")).Status);
Assert.Empty(_peer.Requests);
var resolved = (await alice.Client.Get($"/api/v1/accounts/search?q={handle}&resolve=true")).Ok();
var found = Assert.Single(resolved.Array);
Assert.Equal(bob.Handle(), found.Text("acct"));
Assert.Contains(_peer.Requests, r => r.Path == "/.well-known/webfinger");
Assert.Equal(found.Text("id"), Assert.Single((await alice.Client.Get($"/api/v2/search?q={handle}&type=accounts")).Ok().Body["accounts"]!.AsArray()).Text("id"));
Assert.Contains(alice.Id, (await alice.Client.Get($"/api/v1/accounts/search?q={alice.UserName}")).Ok().Ids);
}
// an account elsewhere: what it features, in its order, its public posts only
[Fact]
public async Task A_remote_accounts_pinned_statuses_are_what_it_features()
{
var alice = await _host.Mastodon("alice");
var (bob, bobId) = await Remote();
Post Held(string text, PostVisibility visibility) => new()
{
ObjectURI = $"{bob.Id}/statuses/{Guid.NewGuid():N}", ActorURI = bob.Id, AuthorAccountId = bobId, IsFederatedCopy = true,
Visibility = visibility, ContentHtml = $"<p>{text}</p>", CreationDate = DateTime.UtcNow, UpdateDate = DateTime.UtcNow
};
var first = Held("first pin", PostVisibility.Public);
var second = Held("second pin", PostVisibility.Unlisted);
var hidden = Held("for followers", PostVisibility.FollowersOnly);
await DB.Default.SaveAsync(new[] { first, second, hidden }, Token);
await DB.Default.SaveAsync(new[]
{
new RemoteFeatured { ActorURI = bob.Id, ObjectURI = second.ObjectURI, PostId = second.ID, Position = 1 },
new RemoteFeatured { ActorURI = bob.Id, ObjectURI = hidden.ObjectURI, PostId = hidden.ID, Position = 2 },
new RemoteFeatured { ActorURI = bob.Id, ObjectURI = first.ObjectURI, PostId = first.ID, Position = 0 }
}, Token);
var pinned = (await alice.Client.Get($"/api/v1/accounts/{bobId}/statuses?pinned=true")).Ok();
Assert.Equal(new[] { first.ID, second.ID }, pinned.Ids);
}
[Fact]
public async Task Account_statuses_filter_pins_replies_reblogs_media_and_tags_and_page_both_ways()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var tag = $"tag{Guid.NewGuid():N}"[..16];
var tagged = await alice.Status($"first #{tag}");
var thread = await alice.Status("and a thread", ("in_reply_to_id", tagged.Text("id")));
var upload = (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media")
{
Content = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(64, 48), "image/jpeg", "a.jpg"))
})).Ok();
var withMedia = await alice.Status("a picture", ("media_ids[]", upload.Body.Text("id")));
var carolPost = await carol.Status("boost me");
var answer = await alice.Status("an answer", ("in_reply_to_id", carolPost.Text("id")));
var reblog = (await alice.Client.Post($"/api/v1/statuses/{carolPost.Text("id")}/reblog")).Ok().Body;
var pinned = await alice.Status("pinned");
(await alice.Client.Post($"/api/v1/statuses/{pinned.Text("id")}/pin")).Ok();
var path = $"/api/v1/accounts/{alice.Id}/statuses";
string[] Ids(params JsonObject[] statuses) => statuses.Select(s => s.Text("id")).ToArray();
Assert.Equal(Ids(pinned), (await carol.Client.Get(path + "?pinned=true")).Ok().Ids);
Assert.True((await carol.Client.Get(path + "?pinned=true")).Ok().Array[0].Flag("pinned"));
Assert.Equal(Ids(pinned, reblog.AsObject(), answer, withMedia, thread, tagged), (await carol.Client.Get(path)).Ok().Ids);
Assert.Equal(Ids(pinned, reblog.AsObject(), withMedia, thread, tagged), (await carol.Client.Get(path + "?exclude_replies=true")).Ok().Ids);
Assert.DoesNotContain(reblog.Text("id"), (await carol.Client.Get(path + "?exclude_reblogs=true")).Ok().Ids);
Assert.Equal(Ids(withMedia), (await carol.Client.Get(path + "?only_media=true")).Ok().Ids);
Assert.Equal(Ids(tagged), (await _host.Client().Get(path + $"?tagged={tag.ToUpperInvariant()}")).Ok().Ids);
var first = (await carol.Client.Get(path + "?limit=2")).Ok();
Assert.Equal(Ids(pinned, reblog.AsObject()), first.Ids);
var next = (await carol.Client.Get(first.Link("next"))).Ok();
Assert.Equal(Ids(answer, withMedia), next.Ids);
Assert.Contains("limit=2", next.Link("prev"));
Assert.Equal(first.Ids, (await carol.Client.Get(next.Link("prev"))).Ok().Ids);
var last = (await carol.Client.Get(next.Link("next"))).Ok();
Assert.Equal(Ids(thread, tagged), last.Ids);
Assert.Empty((await carol.Client.Get(last.Link("next"))).Ok().Array);
}
[Fact]
public async Task A_community_account_lists_the_posts_addressed_to_it()
{
var alice = await _host.Mastodon("alice");
var reader = await _host.Mastodon("reader");
var name = $"community{Guid.NewGuid():N}"[..20];
string communityId;
using (var clientApi = _host.As(alice.Persona.Root.Jwt))
{
var created = await clientApi.PostAsJsonAsync("/clientapi/group/insert", new { avatarId = alice.Id, userName = name, name = "Gardeners", isCommunity = true }, Token);
Assert.Equal(HttpStatusCode.OK, created.StatusCode);
communityId = (await created.Content.ReadFromJsonAsync<JsonObject>(Token))!["id"]!.GetValue<string>();
}
var addressed = await alice.Status($"@{name} first meeting on Sunday");
await alice.Status("not for the garden");
var community = (await reader.Client.Get($"/api/v1/accounts/lookup?acct={name}")).Ok();
Assert.Equal(communityId, community.Body.Text("id"));
Assert.True(community.Body.Flag("group"));
Assert.Equal(new[] { addressed.Text("id") }, (await reader.Client.Get($"/api/v1/accounts/{communityId}/statuses")).Ok().Ids);
Assert.Equal(1, (await _host.Client().Get($"/api/v1/accounts/{communityId}")).Ok().Body.Number("statuses_count"));
}
[Fact]
public async Task A_follower_sees_followers_only_posts_and_a_stranger_does_not()
{
var alice = await _host.Mastodon("alice");
var follower = await _host.Mastodon("follower");
var stranger = await _host.Mastodon("stranger");
(await follower.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
var open = await alice.Status("for everyone");
var quiet = await alice.Status("for followers", ("visibility", "private"));
var direct = await alice.Status($"@{follower.UserName} for you", ("visibility", "direct"));
var path = $"/api/v1/accounts/{alice.Id}/statuses";
Assert.Equal(new[] { quiet.Text("id"), open.Text("id") }, (await follower.Client.Get(path)).Ok().Ids);
Assert.Equal(new[] { open.Text("id") }, (await stranger.Client.Get(path)).Ok().Ids);
Assert.Equal(new[] { open.Text("id") }, (await _host.Client().Get(path)).Ok().Ids);
Assert.Equal(new[] { direct.Text("id"), quiet.Text("id"), open.Text("id") }, (await alice.Client.Get(path)).Ok().Ids);
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Get($"/api/v1/statuses/{quiet.Text("id")}")).Status);
Assert.Equal(HttpStatusCode.OK, (await follower.Client.Get($"/api/v1/statuses/{quiet.Text("id")}")).Status);
}
[Fact]
public async Task Followers_and_following_are_listed_only_to_their_owner()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
(await bob.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
Assert.Equal(new[] { bob.Id }, (await alice.Client.Get($"/api/v1/accounts/{alice.Id}/followers")).Ok().Ids);
Assert.Empty((await bob.Client.Get($"/api/v1/accounts/{alice.Id}/followers")).Ok().Array);
Assert.Empty((await _host.Client().Get($"/api/v1/accounts/{alice.Id}/followers")).Ok().Array);
Assert.Equal(new[] { alice.Id }, (await bob.Client.Get($"/api/v1/accounts/{bob.Id}/following")).Ok().Ids);
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{bob.Id}/following")).Ok().Array);
var account = (await bob.Client.Get($"/api/v1/accounts/{alice.Id}")).Ok();
Assert.Equal(1, account.Body.Number("followers_count"));
}
[Fact]
public async Task Following_is_immediate_for_an_open_persona_requested_for_a_locked_one_and_queued_for_a_remote_one()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var locked = await _host.Mastodon("locked");
var follower = await _host.Mastodon("follower");
(await locked.Client.Patch("/api/v1/accounts/update_credentials", ("locked", "true"))).Ok();
var (bob, bobId) = await Remote();
var open = (await follower.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
Assert.True(open.Body.Flag("following"));
Assert.False(open.Body.Flag("requested"));
Assert.True(open.Body.Flag("showing_reblogs"));
var notified = (await alice.Client.Get("/api/v1/notifications?types[]=follow")).Ok();
Assert.Equal(follower.Id, Assert.Single(notified.Array)!["account"].Text("id"));
var asked = (await follower.Client.Post($"/api/v1/accounts/{locked.Id}/follow", ("reblogs", "false"))).Ok();
Assert.False(asked.Body.Flag("following"));
Assert.True(asked.Body.Flag("requested"));
Assert.Equal(new[] { follower.Id }, (await locked.Client.Get("/api/v1/follow_requests")).Ok().Ids);
Assert.Equal(1, (await locked.Client.Get("/api/v1/accounts/verify_credentials")).Ok().Body["source"].Number("follow_requests_count"));
Assert.True((await locked.Client.Post($"/api/v1/follow_requests/{follower.Id}/authorize")).Ok().Body.Flag("followed_by"));
var accepted = Assert.Single((await follower.Client.Get($"/api/v1/accounts/relationships?id[]={locked.Id}")).Ok().Array);
Assert.True(accepted.Flag("following"));
Assert.False(accepted.Flag("showing_reblogs"));
Assert.Empty((await locked.Client.Get("/api/v1/follow_requests")).Ok().Array);
var remote = (await follower.Client.Post($"/api/v1/accounts/{bobId}/follow")).Ok();
Assert.True(remote.Body.Flag("requested"));
var follow = Assert.Single(await bob.Delivered(since), d => d.Type() == "Follow");
Assert.Equal(follower.Uri, follow.Text("actor"));
Assert.Equal(bob.Id, follow.Text("object"));
Assert.False((await follower.Client.Post($"/api/v1/accounts/{alice.Id}/unfollow")).Ok().Body.Flag("following"));
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/followers")).Ok().Array);
Assert.False((await follower.Client.Post($"/api/v1/accounts/{bobId}/unfollow")).Ok().Body.Flag("requested"));
var undo = Assert.Single(await bob.Delivered(since), d => d.Type() == "Undo");
Assert.Equal(follow.Text("id"), undo["object"].Text("id"));
Assert.Equal(HttpStatusCode.NotFound, (await follower.Client.Post("/api/v1/accounts/000000000000000000000000/follow")).Status);
Assert.Equal(HttpStatusCode.Forbidden, (await follower.Client.Post($"/api/v1/accounts/{follower.Id}/follow")).Status);
}
[Fact]
public async Task Remote_follow_requests_are_answered_with_the_original_follow()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var locked = await _host.Mastodon("locked");
(await locked.Client.Patch("/api/v1/accounts/update_credentials", ("locked", "true"))).Ok();
var bob = new RemoteActor(_peer, "bob");
var carol = new RemoteActor(_peer, "carol");
var bobFollow = await _host.FollowedBy(locked, bob);
var carolFollow = await _host.FollowedBy(locked, carol);
var bobId = (await _host.Known(bob)).ID;
var carolId = (await _host.Known(carol)).ID;
var requests = (await locked.Client.Get("/api/v1/follow_requests")).Ok();
Assert.Equal(new[] { carolId, bobId }, requests.Ids);
Assert.True(Assert.Single((await locked.Client.Get($"/api/v1/accounts/relationships?id[]={bobId}")).Ok().Array).Flag("requested_by"));
Assert.Empty(await bob.Delivered(since));
Assert.True((await locked.Client.Post($"/api/v1/follow_requests/{bobId}/authorize")).Ok().Body.Flag("followed_by"));
Assert.False((await locked.Client.Post($"/api/v1/follow_requests/{carolId}/reject")).Ok().Body.Flag("requested_by"));
var accept = Assert.Single(await bob.Delivered(since));
Assert.Equal("Accept", accept.Type());
Assert.Equal(locked.Uri, accept.Text("actor"));
Assert.Equal(bobFollow, accept["object"].Text("id"));
var reject = Assert.Single(await carol.Delivered(since));
Assert.Equal("Reject", reject.Type());
Assert.Equal(carolFollow, reject["object"].Text("id"));
Assert.Equal(carol.Id, reject["object"].Text("actor"));
Assert.Empty((await locked.Client.Get("/api/v1/follow_requests")).Ok().Array);
Assert.Equal(new[] { bobId }, (await locked.Client.Get($"/api/v1/accounts/{locked.Id}/followers")).Ok().Ids);
Assert.Equal(HttpStatusCode.NotFound, (await locked.Client.Post($"/api/v1/follow_requests/{carolId}/authorize")).Status);
var local = await _host.Mastodon("local");
(await local.Client.Post($"/api/v1/accounts/{locked.Id}/follow")).Ok();
(await locked.Client.Post($"/api/v1/follow_requests/{local.Id}/reject")).Ok();
var refused = Assert.Single((await local.Client.Get($"/api/v1/accounts/relationships?id[]={locked.Id}")).Ok().Array);
Assert.False(refused.Flag("requested"));
Assert.False(refused.Flag("following"));
}
[Fact]
public async Task Removing_a_follower_ends_the_follow_on_both_sides_and_tells_a_remote_follower()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var local = await _host.Mastodon("local");
var bob = new RemoteActor(_peer, "bob");
var follow = await _host.FollowedBy(alice, bob);
var bobId = (await _host.Known(bob)).ID;
(await local.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
Assert.True(Assert.Single((await alice.Client.Get($"/api/v1/accounts/relationships?id[]={bobId}")).Ok().Array).Flag("followed_by"));
var removed = (await alice.Client.Post($"/api/v1/accounts/{bobId}/remove_from_followers")).Ok();
Assert.False(removed.Body.Flag("followed_by"));
var reject = Assert.Single(await bob.Delivered(since), d => d.Type() == "Reject");
Assert.Equal(alice.Uri, reject.Text("actor"));
Assert.Equal(follow, reject["object"].Text("id"));
Assert.Equal("Follow", reject["object"].Text("type"));
Assert.Equal(bob.Id, reject["object"].Text("actor"));
Assert.Equal(alice.Uri, reject["object"].Text("object"));
Assert.False((await alice.Client.Post($"/api/v1/accounts/{local.Id}/remove_from_followers")).Ok().Body.Flag("followed_by"));
Assert.False(Assert.Single((await local.Client.Get($"/api/v1/accounts/relationships?id[]={alice.Id}")).Ok().Array).Flag("following"));
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/followers")).Ok().Array);
Assert.Empty((await local.Client.Get($"/api/v1/accounts/{local.Id}/following")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post("/api/v1/accounts/000000000000000000000000/remove_from_followers")).Status);
}
[Fact]
public async Task Blocks_federate_and_reject_a_remote_follower()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var local = await _host.Mastodon("local");
var bob = new RemoteActor(_peer, "bob");
var follow = await _host.FollowedBy(alice, bob);
var bobId = (await _host.Known(bob)).ID;
(await local.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
var blocked = (await alice.Client.Post($"/api/v1/accounts/{bobId}/block")).Ok();
Assert.True(blocked.Body.Flag("blocking"));
Assert.False(blocked.Body.Flag("followed_by"));
var sent = await bob.Delivered(since);
var reject = Assert.Single(sent, d => d.Type() == "Reject");
Assert.Equal((follow, "Follow", bob.Id), (reject["object"].Text("id"), reject["object"].Text("type"), reject["object"].Text("actor")));
var block = Assert.Single(sent, d => d.Type() == "Block");
Assert.Equal((alice.Uri, bob.Id), (block.Text("actor"), block.Text("object")));
Assert.True((await alice.Client.Post($"/api/v1/accounts/{local.Id}/block")).Ok().Body.Flag("blocking"));
Assert.Equal(new[] { local.Id, bobId }, (await alice.Client.Get("/api/v1/blocks")).Ok().Ids);
var blockedBy = Assert.Single((await local.Client.Get($"/api/v1/accounts/relationships?id[]={alice.Id}")).Ok().Array);
Assert.True(blockedBy.Flag("blocked_by"));
Assert.False(blockedBy.Flag("following"));
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/followers")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post($"/api/v1/accounts/{alice.Id}/block")).Status);
Assert.False((await alice.Client.Post($"/api/v1/accounts/{bobId}/unblock")).Ok().Body.Flag("blocking"));
var undo = Assert.Single(await bob.Delivered(since), d => d.Type() == "Undo");
Assert.Equal(block.Text("id"), undo["object"].Text("id"));
(await alice.Client.Post($"/api/v1/accounts/{local.Id}/unblock")).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/blocks")).Ok().Array);
Assert.Equal(new[] { "Accept", "Block", "Reject", "Undo" }, (await bob.Delivered(since)).Select(d => d.Type()).Order());
}
[Fact]
public async Task Mutes_keep_their_duration_and_notification_choice_and_never_federate()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var chatty = await _host.Mastodon("chatty");
var muffled = await _host.Mastodon("muffled");
var (bob, bobId) = await Remote();
(await alice.Client.Post($"/api/v1/accounts/{chatty.Id}/follow")).Ok();
var post = (await alice.Status("like me")).Text("id");
var muted = (await alice.Client.Post($"/api/v1/accounts/{chatty.Id}/mute", ("duration", "3600"), ("notifications", "false"))).Ok();
Assert.True(muted.Body.Flag("muting"));
Assert.False(muted.Body.Flag("muting_notifications"));
var mute = await DB.Default.Find<Mute>().Match(m => m.AvatarId == alice.Id && m.TargetAccountId == chatty.Id).ExecuteFirstAsync(Token);
Assert.InRange(mute.ExpiresAt!.Value, DateTime.UtcNow.AddMinutes(59), DateTime.UtcNow.AddMinutes(61));
var forever = (await alice.Client.Post($"/api/v1/accounts/{muffled.Id}/mute")).Ok();
Assert.True(forever.Body.Flag("muting_notifications"));
Assert.Null((await DB.Default.Find<Mute>().Match(m => m.AvatarId == alice.Id && m.TargetAccountId == muffled.Id).ExecuteFirstAsync(Token)).ExpiresAt);
(await alice.Client.Post($"/api/v1/accounts/{bobId}/mute")).Ok();
Assert.Equal(new[] { bobId, muffled.Id, chatty.Id }, (await alice.Client.Get("/api/v1/mutes")).Ok().Ids);
var hidden = (await chatty.Status("you will not see this")).Text("id");
(await chatty.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
(await muffled.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
Assert.DoesNotContain(hidden, (await alice.Client.Get("/api/v1/timelines/home")).Ok().Ids);
Assert.Equal(new[] { chatty.Id }, (await alice.Client.Get("/api/v1/notifications?types[]=favourite")).Ok().Array.Select(n => n!["account"].Text("id")));
Assert.Empty(await bob.Delivered(since));
Assert.False((await alice.Client.Post($"/api/v1/accounts/{chatty.Id}/unmute")).Ok().Body.Flag("muting"));
(await alice.Client.Post($"/api/v1/accounts/{muffled.Id}/unmute")).Ok();
(await alice.Client.Post($"/api/v1/accounts/{bobId}/unmute")).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/mutes")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post($"/api/v1/accounts/{alice.Id}/mute")).Status);
}
[Fact]
public async Task Domain_blocks_are_listed_added_and_removed()
{
var alice = await _host.Mastodon("alice");
var domain = $"blocked{Guid.NewGuid():N}.example";
Assert.Equal("{}", (await alice.Client.Post("/api/v1/domain_blocks", ("domain", domain.ToUpperInvariant()))).Ok().Text);
(await alice.Client.Post("/api/v1/domain_blocks", ("domain", "not a domain"))).Ok();
Assert.Equal(new[] { domain }, (await alice.Client.Get("/api/v1/domain_blocks")).Ok().Array.Select(d => d!.GetValue<string>()));
(await alice.Client.Delete("/api/v1/domain_blocks", ("domain", domain))).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/domain_blocks")).Ok().Array);
}
[Fact]
public async Task Relationships_answer_every_asked_id_even_a_junk_one()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var answer = (await alice.Client.Get($"/api/v1/accounts/relationships?id[]=junk&id[]={bob.Id}&id[]=junk")).Ok();
Assert.Equal(new[] { "junk", bob.Id }, answer.Ids);
Assert.All(answer.Array, r => Assert.False(r.Flag("following") || r.Flag("blocking") || r.Flag("muting") || r.Flag("followed_by")));
}
[Fact]
public async Task Following_again_sends_an_unanswered_request_once_more_and_no_more_than_hourly()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var (bob, bobId) = await Remote();
Assert.True((await alice.Client.Post($"/api/v1/accounts/{bobId}/follow")).Ok().Body.Flag("requested"));
var first = Assert.Single(await bob.Delivered(since), d => d.Type() == "Follow");
Assert.True((await alice.Client.Post($"/api/v1/accounts/{bobId}/follow")).Ok().Body.Flag("requested"));
(await alice.Client.Post($"/api/v1/accounts/{bobId}/follow")).Ok();
var follows = (await bob.Delivered(since)).Where(d => d.Type() == "Follow").ToList();
Assert.Equal(2, follows.Count);
Assert.All(follows, f => Assert.Equal(first.Text("id"), f.Text("id")));
}
[Fact]
public async Task A_remote_authors_followers_only_posts_show_only_to_the_personas_that_follow_them()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var (bob, bobId) = await Remote();
(await alice.Client.Post($"/api/v1/accounts/{bobId}/follow")).Ok();
var follow = Assert.Single(await bob.Delivered(since), d => d.Type() == "Follow");
await _host.Deliver(bob, alice.Mouth, new JsonObject
{
["id"] = $"{bob.Origin()}/accepts/{Guid.NewGuid():N}",
["type"] = "Accept",
["actor"] = bob.Id,
["object"] = follow.DeepClone()
});
Assert.True(Assert.Single((await alice.Client.Get($"/api/v1/accounts/relationships?id[]={bobId}")).Ok().Array).Flag("following"));
var create = bob.Create("<p>for my followers</p>", new[] { bob.Id + "/followers" });
await _host.Deliver(bob, alice.Mouth, create);
var noteId = create["object"].Text("id");
var stored = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(Token);
Assert.Equal(PostVisibility.FollowersOnly, stored.Visibility);
var path = $"/api/v1/accounts/{bobId}/statuses";
Assert.Equal(new[] { stored.ID }, (await alice.Client.Get(path)).Ok().Ids);
Assert.Empty((await carol.Client.Get(path)).Ok().Array);
Assert.Empty((await _host.Client().Get(path)).Ok().Array);
Assert.Equal("private", (await alice.Client.Get($"/api/v1/statuses/{stored.ID}")).Ok().Body.Text("visibility"));
Assert.Equal(HttpStatusCode.NotFound, (await carol.Client.Get($"/api/v1/statuses/{stored.ID}")).Status);
Assert.Contains(stored.ID, (await alice.Client.Get("/api/v1/timelines/home")).Ok().Ids);
Assert.True((await alice.Client.Post($"/api/v1/statuses/{stored.ID}/favourite")).Ok().Body.Flag("favourited"));
}
[Fact]
public async Task Search_by_address_finds_local_posts_and_personas_but_never_a_circle_or_a_hidden_post()
{
var alice = await _host.Mastodon("alice");
var stranger = await _host.Mastodon("stranger");
var open = await alice.Status("findable");
var quiet = await alice.Status("not for strangers", ("visibility", "private"));
var circleName = $"circle{Guid.NewGuid():N}"[..20];
using (var clientApi = _host.As(alice.Persona.Root.Jwt))
Assert.Equal(HttpStatusCode.OK, (await clientApi.PostAsJsonAsync("/clientapi/group/insert",
new { avatarId = alice.Id, userName = circleName, name = "inner", isCommunity = false }, Token)).StatusCode);
var byPost = (await stranger.Client.Get($"/api/v2/search?q={Uri.EscapeDataString(open.Text("uri"))}")).Ok();
Assert.Equal(open.Text("id"), Assert.Single(byPost.Body["statuses"]!.AsArray()).Text("id"));
Assert.Equal(open.Text("id"), Assert.Single((await stranger.Client.Get($"/api/v2/search?q={Uri.EscapeDataString(open.Text("url"))}")).Ok().Body["statuses"]!.AsArray()).Text("id"));
var hidden = (await stranger.Client.Get($"/api/v2/search?q={Uri.EscapeDataString(quiet.Text("uri"))}&resolve=true")).Ok();
Assert.Empty(hidden.Body["statuses"]!.AsArray());
Assert.Equal(alice.Id, Assert.Single((await stranger.Client.Get($"/api/v2/search?q={Uri.EscapeDataString(alice.Uri)}")).Ok().Body["accounts"]!.AsArray()).Text("id"));
Assert.Empty((await stranger.Client.Get($"/api/v2/search?q={Uri.EscapeDataString($"{PrivaPubHost.Base}/peasants/{circleName}")}&resolve=true")).Ok().Body["accounts"]!.AsArray());
Assert.Empty((await stranger.Client.Get($"/api/v1/accounts/search?q={circleName}")).Ok().Array);
var tag = (await stranger.Client.Get("/api/v2/search?q=%23Cats&type=hashtags")).Ok();
Assert.Equal("cats", Assert.Single(tag.Body["hashtags"]!.AsArray()).Text("name"));
Assert.Empty(tag.Body["accounts"]!.AsArray());
(await alice.Client.Patch("/api/v1/accounts/update_credentials", ("discoverable", "false"))).Ok();
Assert.Empty((await stranger.Client.Get($"/api/v1/accounts/search?q={alice.UserName}")).Ok().Array);
Assert.Equal(alice.Id, (await stranger.Client.Get($"/api/v1/accounts/lookup?acct={alice.UserName}")).Ok().Body.Text("id"));
}
[Fact]
public async Task A_banned_login_loses_every_persona_token()
{
var alice = await _host.Mastodon("alice");
Assert.Equal(HttpStatusCode.OK, (await alice.Client.Get("/api/v1/accounts/verify_credentials")).Status);
await DB.Default.Update<RootUser>().MatchID(alice.Persona.Root.Id).Modify(r => r.IsBanned, true).ExecuteAsync(Token);
Assert.Equal(HttpStatusCode.Unauthorized, (await alice.Client.Get("/api/v1/accounts/verify_credentials")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await alice.Client.Post("/api/v1/statuses", ("status", "still here?"))).Status);
Assert.Equal(alice.Id, (await _host.Client().Get($"/api/v1/accounts/{alice.Id}")).Ok().Body.Text("id"));
}
[Fact]
public async Task A_forwarded_report_leaves_as_a_flag_from_the_instance_actor_never_from_the_reporter()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var local = await _host.Mastodon("local");
var bob = new RemoteActor(_peer, "bob");
var post = await _host.PublicPostFrom(bob, alice, "<p>something nasty</p>");
var bobId = (await _host.Known(bob)).ID;
var unrelated = await alice.Status("mine");
var report = (await alice.Client.Post("/api/v1/reports", ("account_id", bobId), ("status_ids[]", post.ID), ("status_ids[]", unrelated.Text("id")),
("comment", "please look"), ("category", "spam"), ("forward", "true"))).Ok();
Assert.Equal(bobId, report.Body["target_account"].Text("id"));
Assert.Equal(new[] { post.ID }, report.Body["status_ids"]!.AsArray().Select(i => i!.GetValue<string>()));
Assert.True(report.Body.Flag("forwarded"));
Assert.Equal("spam", report.Body.Text("category"));
var flag = Assert.Single(await bob.Delivered(since));
Assert.Equal("Flag", flag.Type());
Assert.Equal($"{PrivaPubHost.Base}/peasants/privapub", flag.Text("actor"));
Assert.Equal(new[] { bob.Id, post.ObjectURI }, flag["object"]!.AsArray().Select(o => o!.GetValue<string>()));
Assert.DoesNotContain(alice.UserName, flag.ToJsonString());
Assert.DoesNotContain(alice.Id, flag.ToJsonString());
var quiet = (await alice.Client.Post("/api/v1/reports", ("account_id", bobId), ("comment", "just noting"))).Ok();
Assert.False(quiet.Body.Flag("forwarded"));
Assert.False((await alice.Client.Post("/api/v1/reports", ("account_id", local.Id), ("forward", "true"))).Ok().Body.Flag("forwarded"));
Assert.Single(await bob.Delivered(since));
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post("/api/v1/reports", ("account_id", "000000000000000000000000"))).Status);
}
[Fact]
public async Task Statuses_count_still_counts_located_posts_pending_an_owner_decision()
{
var alice = await _host.Mastodon("alice");
await alice.Status("visible");
await _host.Located(alice, "by the river");
var account = (await _host.Client().Get($"/api/v1/accounts/{alice.Id}")).Ok();
Assert.Equal(2, account.Body.Number("statuses_count"));
Assert.Single((await _host.Client().Get($"/api/v1/accounts/{alice.Id}/statuses")).Ok().Array);
}
[Fact]
public async Task Account_stub_routes_answer_empty_lists()
{
var alice = await _host.Mastodon("alice");
Assert.Empty((await _host.Client().Get($"/api/v1/accounts/{alice.Id}/featured_tags")).Ok().Array);
var familiar = (await alice.Client.Get($"/api/v1/accounts/familiar_followers?id[]={alice.Id}")).Ok();
Assert.Equal(alice.Id, Assert.Single(familiar.Array).Text("id"));
Assert.Empty(familiar.Array[0]!["accounts"]!.AsArray());
}
}
}
+169
View File
@@ -0,0 +1,169 @@
using MongoDB.Entities;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// Mastodon's filters: a persona's v2 filters with keywords and statuses, the v1 API over their keywords, and the
// `filtered` each status carries for the filters it matches
[Trait("Category", "Integration")]
public sealed class MastodonFiltersTests : IAsyncLifetime
{
PrivaPubHost _host;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static Task<ApiAnswer> Create(Mastodon owner, string title, string action, params (string Keyword, bool WholeWord)[] keywords) =>
owner.Client.Json(HttpMethod.Post, "/api/v2/filters", new JsonObject
{
["title"] = title,
["context"] = new JsonArray("home", "public", "thread"),
["filter_action"] = action,
["keywords_attributes"] = new JsonArray(keywords.Select(k => (JsonNode)new JsonObject { ["keyword"] = k.Keyword, ["whole_word"] = k.WholeWord }).ToArray())
});
[Fact]
public async Task A_filter_is_made_changed_and_deleted_by_its_owner_only()
{
var alice = await _host.Mastodon("alice");
var mallory = await _host.Mastodon("mallory");
var made = (await Create(alice, "Spoilers", "hide", ("spoiler", true), ("leak", false))).Ok().Object;
var id = made.Text("id");
Assert.Equal("hide", made.Text("filter_action"));
Assert.Equal(new[] { "home", "public", "thread" }, made["context"]!.AsArray().Select(c => c!.GetValue<string>()));
var keywords = made["keywords"]!.AsArray();
Assert.Equal(new[] { "spoiler", "leak" }, keywords.Select(k => k.Text("keyword")));
Assert.Equal(new[] { true, false }, keywords.Select(k => k!.Flag("whole_word")));
Assert.Null(made["expires_at"]);
var changed = (await alice.Client.Json(HttpMethod.Put, $"/api/v2/filters/{id}", new JsonObject
{
["title"] = "Leaks",
["filter_action"] = "warn",
["expires_in"] = 3600,
["keywords_attributes"] = new JsonArray(
new JsonObject { ["id"] = keywords[0].Text("id"), ["_destroy"] = true },
new JsonObject { ["keyword"] = "ending" })
})).Ok().Object;
Assert.Equal("Leaks", changed.Text("title"));
Assert.Equal("warn", changed.Text("filter_action"));
Assert.NotNull(changed.Text("expires_at"));
Assert.Equal(new[] { "leak", "ending" }, changed["keywords"]!.AsArray().Select(k => k.Text("keyword")));
var added = (await alice.Client.Post($"/api/v2/filters/{id}/keywords", ("keyword", "finale"), ("whole_word", "false"))).Ok().Object;
Assert.False(added.Flag("whole_word"));
var keywordId = added.Text("id");
Assert.Equal("finale", (await alice.Client.Get($"/api/v2/filters/keywords/{keywordId}")).Ok().Object.Text("keyword"));
Assert.Equal("last scene", (await alice.Client.Put($"/api/v2/filters/keywords/{keywordId}", ("keyword", "last scene"))).Ok().Object.Text("keyword"));
(await alice.Client.Delete($"/api/v2/filters/keywords/{keywordId}")).Ok();
Assert.Equal(2, (await alice.Client.Get($"/api/v2/filters/{id}/keywords")).Ok().Array.Count);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await Create(alice, " ", "warn", ("x", true))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await Create(alice, "x", "explode", ("x", true))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v2/filters", ("title", "x"), ("context[]", "everywhere"))).Status);
Assert.Empty((await mallory.Client.Get("/api/v2/filters")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Get($"/api/v2/filters/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Delete($"/api/v2/filters/keywords/{changed["keywords"]![0].Text("id")}")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get("/api/v2/filters")).Status);
(await alice.Client.Delete($"/api/v2/filters/{id}")).Ok();
Assert.Empty((await alice.Client.Get("/api/v2/filters")).Ok().Array);
}
[Fact]
public async Task Statuses_carry_the_filters_they_match_by_word_and_by_id()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
(await bob.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
var filterId = (await Create(alice, "Spoilers", "warn", ("spoiler", true), ("#tv", true))).Ok().Object.Text("id");
var other = (await Create(bob, "Not mine", "hide", ("spoiler", false))).Ok().Object.Text("id");
var whole = (await alice.Status("a big SPOILER ahead")).Text("id");
var partial = (await alice.Status("spoilers galore")).Text("id");
var tagged = (await alice.Status("watching #TV tonight")).Text("id");
var plain = (await alice.Status("nothing to see")).Text("id");
var warned = (await alice.Status("calm text", ("spoiler_text", "spoiler inside"))).Text("id");
var byId = (await alice.Client.Post($"/api/v2/filters/{filterId}/statuses", ("status_id", plain))).Ok().Object;
Assert.Equal(plain, byId.Text("status_id"));
var boost = (await bob.Client.Post($"/api/v1/statuses/{whole}/reblog")).Ok().Object.Text("id");
var home = (await alice.Client.Get("/api/v1/timelines/home?limit=40")).Ok().Array.ToDictionary(s => s.Text("id"), s => s!["filtered"]!.AsArray());
var match = Assert.Single(home[whole]);
Assert.Equal(filterId, match!["filter"].Text("id"));
Assert.Equal("warn", match["filter"].Text("filter_action"));
Assert.Equal(new[] { "spoiler" }, match["keyword_matches"]!.AsArray().Select(k => k!.GetValue<string>()));
Assert.Empty(home[partial]);
Assert.Equal(new[] { "#tv" }, Assert.Single(home[tagged])!["keyword_matches"]!.AsArray().Select(k => k!.GetValue<string>()));
Assert.Equal(new[] { byId.Text("id") }, Assert.Single(home[plain])!["status_matches"]!.AsArray().Select(k => k!.GetValue<string>()));
Assert.Single(home[warned]);
Assert.Single((await alice.Client.Get($"/api/v1/statuses/{boost}")).Ok().Body["filtered"]!.AsArray());
var asBob = (await bob.Client.Get($"/api/v1/statuses/{partial}")).Ok().Body["filtered"]!.AsArray();
Assert.Equal(other, Assert.Single(asBob)!["filter"].Text("id"));
Assert.Empty((await _host.Client().Get($"/api/v1/statuses/{whole}")).Ok().Body["filtered"]!.AsArray());
// an expired filter matches nothing
await DB.Default.Update<PersonaFilter>().MatchID(filterId).Modify(f => f.ExpiresAt, DateTime.UtcNow.AddMinutes(-1)).ExecuteAsync(Token);
Assert.Empty((await alice.Client.Get($"/api/v1/statuses/{whole}")).Ok().Body["filtered"]!.AsArray());
}
[Fact]
public async Task Keywords_come_as_form_fields_listed_or_numbered()
{
var alice = await _host.Mastodon("alice");
var listed = (await alice.Client.Post("/api/v2/filters", ("title", "Listed"), ("context[]", "home"),
("keywords_attributes[][keyword]", "one"), ("keywords_attributes[][whole_word]", "false"),
("keywords_attributes[][keyword]", "two"), ("keywords_attributes[][whole_word]", "true"))).Ok().Object;
Assert.Equal(new[] { ("one", false), ("two", true) },
listed["keywords"]!.AsArray().Select(k => (k.Text("keyword"), k!.Flag("whole_word"))));
var numbered = (await alice.Client.Post("/api/v2/filters", ("title", "Numbered"), ("context[]", "public"),
("keywords_attributes[1][keyword]", "second"), ("keywords_attributes[0][keyword]", "first"),
("keywords_attributes[0][whole_word]", "false"))).Ok().Object;
Assert.Equal(new[] { ("first", false), ("second", true) },
numbered["keywords"]!.AsArray().Select(k => (k.Text("keyword"), k!.Flag("whole_word"))));
}
[Fact]
public async Task A_v1_filter_is_one_keyword_of_a_v2_filter()
{
var alice = await _host.Mastodon("alice");
var made = (await alice.Client.Post("/api/v1/filters", ("phrase", "election"), ("context[]", "home"), ("context[]", "notifications"),
("irreversible", "true"), ("whole_word", "false"))).Ok().Object;
var id = made.Text("id");
Assert.Equal("election", made.Text("phrase"));
Assert.True(made.Flag("irreversible"));
Assert.False(made.Flag("whole_word"));
var v2 = Assert.Single((await alice.Client.Get("/api/v2/filters")).Ok().Array)!;
Assert.Equal("election", v2.Text("title"));
Assert.Equal("hide", v2.Text("filter_action"));
Assert.Equal(id, Assert.Single(v2["keywords"]!.AsArray()).Text("id"));
var changed = (await alice.Client.Put($"/api/v1/filters/{id}", ("phrase", "elections"), ("irreversible", "false"))).Ok().Object;
Assert.Equal("elections", changed.Text("phrase"));
Assert.False(changed.Flag("irreversible"));
Assert.Equal(new[] { id }, (await alice.Client.Get("/api/v1/filters")).Ok().Ids);
Assert.Equal("elections", (await alice.Client.Get($"/api/v1/filters/{id}")).Ok().Object.Text("phrase"));
(await alice.Client.Delete($"/api/v1/filters/{id}")).Ok();
Assert.Empty((await alice.Client.Get("/api/v2/filters")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/filters/{id}")).Status);
}
}
}
@@ -0,0 +1,131 @@
using PrivaPub.Api.Mastodon.Controllers;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class MastodonInstanceTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
[Fact]
public async Task Both_instance_documents_advertise_mastodon_4_2_and_the_domain()
{
var anonymous = _host.Client();
var v1 = (await anonymous.Get("/api/v1/instance")).Ok();
Assert.Equal("4.2.0 (compatible; PrivaPub)", v1.Body.Text("version"));
Assert.Equal(InstanceController.Version, v1.Body.Text("version"));
Assert.Equal(PrivaPubHost.Host, v1.Body.Text("uri"));
Assert.True(v1.Body.Flag("registrations"));
Assert.True(v1.Body.Flag("invites_enabled"));
Assert.True(v1.Body["stats"].Number("user_count") >= 0);
Assert.Equal(5000, v1.Body["configuration"]!["statuses"].Number("max_characters"));
Assert.Equal(4, v1.Body["configuration"]!["polls"].Number("max_options"));
Assert.Contains("image/jpeg", v1.Body["configuration"]!["media_attachments"]!["supported_mime_types"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.Empty(v1.Body["rules"]!.AsArray());
Assert.Null(v1.Body["contact_account"]);
Assert.True(v1.Body.AsObject().ContainsKey("contact_account"));
var v2 = (await anonymous.Get("/api/v2/instance")).Ok();
Assert.Equal(InstanceController.Version, v2.Body.Text("version"));
Assert.Equal(PrivaPubHost.Host, v2.Body.Text("domain"));
Assert.Equal($"wss://{PrivaPubHost.Host}", v2.Body["configuration"]!["urls"]!.Text("streaming"));
Assert.Equal($"wss://{PrivaPubHost.Host}", v1.Body["urls"]!.Text("streaming_api"));
Assert.Equal(PrivaPub.Api.Mastodon.Controllers.StatusesController.MaxPins, v2.Body["configuration"]!["accounts"].Number("max_pinned_statuses"));
Assert.Contains("image/avif", v2.Body["configuration"]!["media_attachments"]!["supported_mime_types"]!.AsArray().Select(t => t!.GetValue<string>()));
// what the bundled libvips can't decode is not offered
Assert.DoesNotContain("image/heic", v2.Body["configuration"]!["media_attachments"]!["supported_mime_types"]!.AsArray().Select(t => t!.GetValue<string>()));
Assert.True(v2.Body["registrations"].Flag("enabled"));
Assert.True((await anonymous.Get("/nodeinfo/2.1")).Ok().Body.Flag("openRegistrations"));
Assert.False(v2.Body["configuration"]!["translation"].Flag("enabled"));
Assert.Equal(7, v2.Body["api_versions"].Number("mastodon"));
var signedIn = await _host.Mastodon("reader");
Assert.Equal(InstanceController.Version, (await signedIn.Client.Get("/api/v1/instance")).Ok().Body.Text("version"));
}
[Fact]
public async Task The_instance_side_routes_answer_without_a_login()
{
var anonymous = _host.Client();
Assert.Empty((await anonymous.Get("/api/v1/instance/peers")).Ok().Array);
Assert.Empty((await anonymous.Get("/api/v1/instance/activity")).Ok().Array);
Assert.Empty((await anonymous.Get("/api/v1/instance/rules")).Ok().Array);
var extended = (await anonymous.Get("/api/v1/instance/extended_description")).Ok();
Assert.StartsWith("<p>", extended.Body.Text("content"));
Assert.EndsWith("T00:00:00.000Z", extended.Body.Text("updated_at"));
}
[Theory]
[InlineData("/api/v1/custom_emojis")]
[InlineData("/api/v1/announcements")]
public async Task Public_stub_routes_answer_empty_lists_to_anyone(string path)
{
Assert.Empty((await _host.Client().Get(path)).Ok().Array);
}
[Theory]
[InlineData("/api/v1/suggestions")]
[InlineData("/api/v2/suggestions")]
[InlineData("/api/v1/endorsements")]
[InlineData("/api/v1/featured_tags")]
public async Task Signed_in_stub_routes_answer_empty_lists(string path)
{
var reader = await _host.Mastodon("reader");
Assert.Empty((await reader.Client.Get(path)).Ok().Array);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get(path)).Status);
}
[Fact]
public async Task Apps_are_registered_validated_and_verified()
{
var anonymous = _host.Client();
var app = (await anonymous.Post("/api/v1/apps", ("client_name", "tusky-ish"), ("redirect_uris", "urn:ietf:wg:oauth:2.0:oob"), ("scopes", "read write"),
("website", "https://example.com"))).Ok();
Assert.Equal("tusky-ish", app.Body.Text("name"));
Assert.Equal(new[] { "read", "write" }, app.Body["scopes"]!.AsArray().Select(s => s!.GetValue<string>()));
Assert.False(string.IsNullOrEmpty(app.Body.Text("client_id")));
Assert.False(string.IsNullOrEmpty(app.Body.Text("client_secret")));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await anonymous.Post("/api/v1/apps", ("redirect_uris", "urn:ietf:wg:oauth:2.0:oob"))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await anonymous.Post("/api/v1/apps", ("client_name", "x"), ("redirect_uris", "javascript:alert(1)"))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await anonymous.Post("/api/v1/apps", ("client_name", "x"))).Status);
var reader = await _host.Mastodon("reader");
var verified = (await reader.Client.Get("/api/v1/apps/verify_credentials")).Ok();
Assert.Equal("privapub-tests", verified.Body.Text("name"));
Assert.Equal(HttpStatusCode.Unauthorized, (await anonymous.Get("/api/v1/apps/verify_credentials")).Status);
}
[Fact]
public async Task Preferences_follow_the_persona_and_push_has_no_subscription()
{
var reader = await _host.Mastodon("reader", scopes: "read write follow push");
(await reader.Client.Patch("/api/v1/accounts/update_credentials", ("source[privacy]", "unlisted"), ("source[sensitive]", "true"))).Ok();
var preferences = (await reader.Client.Get("/api/v1/preferences")).Ok();
Assert.Equal("unlisted", preferences.Body.Text("posting:default:visibility"));
Assert.True(preferences.Body.Flag("posting:default:sensitive"));
Assert.False(preferences.Body.Flag("reading:expand:spoilers"));
var push = await reader.Client.Get("/api/v1/push/subscription");
Assert.Equal(HttpStatusCode.NotFound, push.Status);
Assert.Equal("Record not found", push.Body.Text("error"));
}
}
}
+179
View File
@@ -0,0 +1,179 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
// Mastodon's lists: a persona's own groups of accounts it follows, read as timelines, never federated
[Trait("Category", "Integration")]
public sealed class MastodonListsTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static async Task Follow(Mastodon follower, Mastodon target) =>
(await follower.Client.Post($"/api/v1/accounts/{target.Id}/follow")).Ok();
static async Task<string> List(Mastodon owner, string title, params (string Key, string Value)[] more) =>
(await owner.Client.Post("/api/v1/lists", more.Prepend(("title", title)).ToArray())).Ok().Object.Text("id");
[Fact]
public async Task A_list_is_created_renamed_and_deleted_by_its_owner_only()
{
var alice = await _host.Mastodon("alice");
var mallory = await _host.Mastodon("mallory");
var created = (await alice.Client.Post("/api/v1/lists", ("title", "Friends"), ("replies_policy", "followed"), ("exclusive", "true"))).Ok().Object;
var id = created.Text("id");
Assert.Equal("Friends", created.Text("title"));
Assert.Equal("followed", created.Text("replies_policy"));
Assert.True(created.Flag("exclusive"));
var other = await List(alice, "Art");
Assert.Equal(new[] { other, id }, (await alice.Client.Get("/api/v1/lists")).Ok().Ids);
var renamed = (await alice.Client.Put($"/api/v1/lists/{id}", ("title", "Close friends"), ("exclusive", "false"))).Ok().Object;
Assert.Equal("Close friends", renamed.Text("title"));
Assert.Equal("followed", renamed.Text("replies_policy"));
Assert.False(renamed.Flag("exclusive"));
Assert.Equal("Close friends", (await alice.Client.Get($"/api/v1/lists/{id}")).Ok().Object.Text("title"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/lists", ("title", " "))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/lists", ("title", "x"), ("replies_policy", "everyone"))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Put($"/api/v1/lists/{id}", ("title", ""))).Status);
Assert.Empty((await mallory.Client.Get("/api/v1/lists")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Get($"/api/v1/lists/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Put($"/api/v1/lists/{id}", ("title", "mine"))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Delete($"/api/v1/lists/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Get($"/api/v1/timelines/list/{id}")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get("/api/v1/lists")).Status);
(await alice.Client.Delete($"/api/v1/lists/{id}")).Ok();
Assert.Equal(new[] { other }, (await alice.Client.Get("/api/v1/lists")).Ok().Ids);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/lists/{id}")).Status);
}
[Fact]
public async Task Only_followed_accounts_join_a_list_and_its_timeline_shows_their_posts()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var carol = await _host.Mastodon("carol");
var dave = await _host.Mastodon("dave");
await Follow(alice, bob);
await Follow(alice, dave);
var list = await List(alice, "Bob");
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post($"/api/v1/lists/{list}/accounts", ("account_ids[]", carol.Id))).Status);
(await alice.Client.Post($"/api/v1/lists/{list}/accounts", ("account_ids[]", bob.Id))).Ok();
(await alice.Client.Post($"/api/v1/lists/{list}/accounts", ("account_ids[]", bob.Id))).Ok();
Assert.Equal(new[] { bob.Id }, (await alice.Client.Get($"/api/v1/lists/{list}/accounts")).Ok().Ids);
Assert.Equal(new[] { list }, (await alice.Client.Get($"/api/v1/accounts/{bob.Id}/lists")).Ok().Ids);
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{dave.Id}/lists")).Ok().Array);
var fromBob = (await bob.Status("from bob")).Text("id");
await dave.Status("from dave");
await carol.Status("from carol");
Assert.Equal(new[] { fromBob }, (await alice.Client.Get($"/api/v1/timelines/list/{list}")).Ok().Ids);
(await alice.Client.Delete($"/api/v1/lists/{list}/accounts", ("account_ids[]", bob.Id))).Ok();
Assert.Empty((await alice.Client.Get($"/api/v1/lists/{list}/accounts")).Ok().Array);
Assert.Empty((await alice.Client.Get($"/api/v1/timelines/list/{list}")).Ok().Array);
}
[Theory]
[InlineData("list", true, false, false)]
[InlineData("followed", true, true, false)]
[InlineData("none", true, false, false)]
public async Task A_lists_replies_follow_its_policy(string policy, bool toSelf, bool toFollowed, bool toStranger)
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var dave = await _host.Mastodon("dave");
var carol = await _host.Mastodon("carol");
await Follow(alice, bob);
await Follow(alice, dave);
var list = await List(alice, "Bob", ("replies_policy", policy));
(await alice.Client.Post($"/api/v1/lists/{list}/accounts", ("account_ids[]", bob.Id))).Ok();
var root = (await bob.Status("a thread")).Text("id");
var mine = (await alice.Status("asking bob")).Text("id");
var ofDave = (await dave.Status("dave's")).Text("id");
var ofCarol = (await carol.Status("carol's")).Text("id");
var self = (await bob.Status("and more", ("in_reply_to_id", root))).Text("id");
var toAlice = (await bob.Status($"@{alice.UserName} answered", ("in_reply_to_id", mine))).Text("id");
var toDave = (await bob.Status($"@{dave.UserName} agreed", ("in_reply_to_id", ofDave))).Text("id");
var toCarol = (await bob.Status($"@{carol.UserName} hm", ("in_reply_to_id", ofCarol))).Text("id");
var shown = (await alice.Client.Get($"/api/v1/timelines/list/{list}")).Ok().Ids.ToList();
Assert.Contains(root, shown);
Assert.Equal(toSelf, shown.Contains(self));
Assert.Contains(toAlice, shown);
Assert.Equal(toFollowed, shown.Contains(toDave));
Assert.Equal(toStranger, shown.Contains(toCarol));
}
[Fact]
public async Task An_exclusive_lists_members_are_read_there_and_not_at_home()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var dave = await _host.Mastodon("dave");
await Follow(alice, bob);
await Follow(alice, dave);
var list = await List(alice, "Apart", ("exclusive", "true"));
(await alice.Client.Post($"/api/v1/lists/{list}/accounts", ("account_ids[]", bob.Id))).Ok();
var fromBob = (await bob.Status("from bob")).Text("id");
var fromDave = (await dave.Status("from dave")).Text("id");
var home = (await alice.Client.Get("/api/v1/timelines/home")).Ok().Ids.ToList();
Assert.Contains(fromDave, home);
Assert.DoesNotContain(fromBob, home);
Assert.Equal(new[] { fromBob }, (await alice.Client.Get($"/api/v1/timelines/list/{list}")).Ok().Ids);
(await alice.Client.Put($"/api/v1/lists/{list}", ("exclusive", "false"))).Ok();
Assert.Contains(fromBob, (await alice.Client.Get("/api/v1/timelines/home")).Ok().Ids);
}
[Fact]
public async Task Account_search_with_following_finds_only_followed_accounts()
{
var alice = await _host.Mastodon("alice");
var prefix = $"ls{Guid.NewGuid():N}"[..10];
var followed = await _host.Mastodon(prefix + "a");
var stranger = await _host.Mastodon(prefix + "b");
await Follow(alice, followed);
var found = (await alice.Client.Get($"/api/v1/accounts/search?q={prefix}&following=true")).Ok().Ids.ToList();
Assert.Equal(new[] { followed.Id }, found);
Assert.Contains(stranger.Id, (await alice.Client.Get($"/api/v1/accounts/search?q={prefix}")).Ok().Ids);
Assert.Empty((await stranger.Client.Get($"/api/v1/accounts/search?q={prefix}&following=true")).Ok().Array);
}
[Fact]
public async Task An_unfollowed_account_leaves_the_personas_lists_and_does_not_come_back_with_a_new_follow()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
await Follow(alice, bob);
var list = await List(alice, "Bob", ("exclusive", "true"));
(await alice.Client.Post($"/api/v1/lists/{list}/accounts", ("account_ids[]", bob.Id))).Ok();
(await alice.Client.Post($"/api/v1/accounts/{bob.Id}/unfollow")).Ok();
Assert.Empty((await alice.Client.Get($"/api/v1/lists/{list}/accounts")).Ok().Array);
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{bob.Id}/lists")).Ok().Array);
await Follow(alice, bob);
var fromBob = (await bob.Status("back again")).Text("id");
Assert.Contains(fromBob, (await alice.Client.Get("/api/v1/timelines/home")).Ok().Ids);
Assert.Empty((await alice.Client.Get($"/api/v1/lists/{list}/accounts")).Ok().Array);
}
}
}
+504
View File
@@ -0,0 +1,504 @@
using PrivaPub.Models.Jobs;
using PrivaPub.Domain.Media;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Diagnostics;
using System.Net;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class MastodonMediaTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
// what GET /api/v1/media/:id answers once the job processing an upload sent to v2 has run
async Task<ApiAnswer> Processed(Mastodon account, ApiAnswer accepted)
{
var id = accepted.Body.Text("id");
await _host.Run(j => j.Kind == JobKind.ProcessMedia && j.Payload == id, Token);
return await account.Client.Get($"/api/v1/media/{id}");
}
static Task<ApiAnswer> Upload(Mastodon account, string path, byte[] bytes, string contentType, string fileName, params (string Key, string Value)[] fields)
{
var form = MastodonHelpers.Multipart(("file", bytes, contentType, fileName));
foreach (var (key, value) in fields)
form.Add(new StringContent(value), key);
return account.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, path) { Content = form });
}
static async Task<(int ExitCode, string Output)> Run(string program, params string[] arguments)
{
var start = new ProcessStartInfo(program) { RedirectStandardOutput = true, RedirectStandardError = true };
foreach (var argument in arguments)
start.ArgumentList.Add(argument);
try
{
using var process = Process.Start(start)!;
var output = process.StandardOutput.ReadToEndAsync(Token);
var errors = process.StandardError.ReadToEndAsync(Token);
await process.WaitForExitAsync(Token);
return (process.ExitCode, await output + await errors);
}
catch (System.ComponentModel.Win32Exception)
{
return (-1, $"{program} is not installed");
}
}
static async Task<byte[]> Made(string extension, params string[] arguments)
{
if ((await Run("ffmpeg", "-version")).ExitCode != 0 || (await Run("ffprobe", "-version")).ExitCode != 0)
Assert.Skip("ffmpeg and ffprobe are needed to make and inspect audio and video");
var path = Path.Combine(Path.GetTempPath(), $"privapub-av-{Guid.NewGuid():N}.{extension}");
try
{
var (exitCode, output) = await Run("ffmpeg", arguments.Append(path).Prepend("-nostdin").Prepend("-y").ToArray());
Assert.True(exitCode == 0, output);
return await File.ReadAllBytesAsync(path, Token);
}
finally
{
File.Delete(path);
}
}
async Task<string> Probe(string url)
{
var path = Path.Combine(Path.GetTempPath(), $"privapub-probe-{Guid.NewGuid():N}");
try
{
await File.WriteAllBytesAsync(path, await _host.Client().GetByteArrayAsync(url, Token), Token);
var (exitCode, output) = await Run("ffprobe", "-v", "quiet", "-print_format", "json", "-show_format", "-show_streams", path);
Assert.Equal(0, exitCode);
return output;
}
finally
{
File.Delete(path);
}
}
[Theory]
[InlineData("/api/v1/media")]
[InlineData("/api/v2/media")]
public async Task Images_upload_with_their_description_and_focus_and_without_their_metadata(string route)
{
var alice = await _host.Mastodon("alice");
var uploaded = (await Upload(alice, route, MastodonHelpers.JpegWithMetadata(640, 480), "image/jpeg", "holiday.jpg",
("description", "a blue square"), ("focus", "0.5,-0.25"))).Ok();
Assert.Equal("image", uploaded.Body.Text("type"));
Assert.Equal("a blue square", uploaded.Body.Text("description"));
Assert.Equal(640, uploaded.Body["meta"]!["original"].Number("width"));
Assert.Equal(480, uploaded.Body["meta"]!["original"].Number("height"));
Assert.Equal(0.5, uploaded.Body["meta"]!["focus"]!["x"]!.GetValue<double>());
Assert.Equal(-0.25, uploaded.Body["meta"]!["focus"]!["y"]!.GetValue<double>());
Assert.False(string.IsNullOrEmpty(uploaded.Body.Text("blurhash")));
Assert.Null(uploaded.Body.Text("remote_url"));
foreach (var field in new[] { "url", "preview_url" })
{
Assert.StartsWith($"{PrivaPubHost.Base}/media/files/", uploaded.Body.Text(field));
MastodonHelpers.AssertNoMetadata(await _host.Client().GetByteArrayAsync(uploaded.Body.Text(field), Token));
}
}
[Fact]
public async Task Media_is_read_and_described_only_by_its_owner()
{
var alice = await _host.Mastodon("alice");
var mallory = await _host.Mastodon("mallory");
var id = (await Upload(alice, "/api/v2/media", MastodonHelpers.JpegWithMetadata(32, 32), "image/jpeg", "a.jpg")).Ok().Body.Text("id");
Assert.Equal(id, (await alice.Client.Get($"/api/v1/media/{id}")).Ok().Body.Text("id"));
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Get($"/api/v1/media/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Put($"/api/v1/media/{id}", ("description", "mine now"))).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get($"/api/v1/media/{id}")).Status);
var described = (await alice.Client.Put($"/api/v1/media/{id}", ("description", " a tiny square "), ("focus", "2,-3"))).Ok();
Assert.Equal("a tiny square", described.Body.Text("description"));
Assert.Equal((1.0, -1.0), (described.Body["meta"]!["focus"]!["x"]!.GetValue<double>(), described.Body["meta"]!["focus"]!["y"]!.GetValue<double>()));
Assert.Equal("a tiny square", (await alice.Client.Get($"/api/v1/media/{id}")).Ok().Body.Text("description"));
Assert.Null((await alice.Client.Put($"/api/v1/media/{id}", ("description", ""))).Ok().Body.Text("description"));
}
// a focal point is two finite numbers: NaN or Infinity is ignored, never stored (it broke every timeline holding the post)
[Fact]
public async Task A_focal_point_that_is_not_a_number_is_ignored()
{
var alice = await _host.Mastodon("alice");
var uploaded = (await Upload(alice, "/api/v2/media", MastodonHelpers.JpegWithMetadata(32, 32), "image/jpeg", "a.jpg", ("focus", "NaN,NaN"))).Ok();
Assert.Null(uploaded.Body["meta"]?["focus"]);
var id = uploaded.Body.Text("id");
(await alice.Client.Put($"/api/v1/media/{id}", ("focus", "0.5,0.25"))).Ok();
foreach (var unsound in new[] { "NaN,0", "Infinity,1", "0,-Infinity" })
{
var focus = (await alice.Client.Put($"/api/v1/media/{id}", ("focus", unsound))).Ok().Body["meta"]!["focus"]!;
Assert.Equal((0.5, 0.25), (focus["x"]!.GetValue<double>(), focus["y"]!.GetValue<double>()));
}
var status = (await alice.Client.Post("/api/v1/statuses", ("status", "focused"), ("media_ids[]", id))).Ok();
Assert.Equal(0.5, status.Body["media_attachments"]![0]!["meta"]!["focus"]!["x"]!.GetValue<double>());
}
[Fact]
public async Task Unsupported_unreadable_and_missing_files_are_refused_with_422()
{
var alice = await _host.Mastodon("alice");
var text = await Upload(alice, "/api/v2/media", Encoding.UTF8.GetBytes("hello"), "text/plain", "a.txt");
Assert.Equal(HttpStatusCode.UnprocessableEntity, text.Status);
Assert.Contains("not supported", text.Body.Text("error"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await Upload(alice, "/api/v2/media", Encoding.UTF8.GetBytes("not a jpeg"), "image/jpeg", "a.jpg")).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await Upload(alice, "/api/v1/media", Encoding.UTF8.GetBytes("not a video"), "video/mp4", "a.mp4")).Status);
// sent to v2, it is taken, then refused once processed
var later = await Upload(alice, "/api/v2/media", Encoding.UTF8.GetBytes("not a video"), "video/mp4", "a.mp4");
Assert.Equal(HttpStatusCode.Accepted, later.Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await Processed(alice, later)).Status);
var empty = new MultipartFormDataContent { { new StringContent("no file"), "description" } };
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media") { Content = empty })).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/media")).Status);
}
// refused before anything is decoded: an SVG claiming to be a PNG, and an image that would decode to too many pixels
[Fact]
public async Task An_image_that_is_not_one_or_too_large_is_refused_before_decoding()
{
var alice = await _host.Mastodon("alice");
var svg = "<svg xmlns='http://www.w3.org/2000/svg' width='30000' height='30000'><rect width='1' height='1'/></svg>"u8.ToArray();
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await Upload(alice, "/api/v2/media", svg, "image/png", "a.png")).Status);
using var huge = NetVips.Image.Black(8000, 6000);
var tooLarge = await Upload(alice, "/api/v2/media", huge.WriteToBuffer(".png"), "image/png", "huge.png");
Assert.Equal(HttpStatusCode.UnprocessableEntity, tooLarge.Status);
Assert.Contains("too large", tooLarge.Body.Text("error"));
}
// an animated GIF becomes what Mastodon makes of one, a looping mp4 typed gifv; a still GIF stays an image
[Fact]
public async Task An_animated_gif_becomes_a_gifv_and_a_still_one_an_image()
{
var alice = await _host.Mastodon("alice");
using var frame = (NetVips.Image.Black(64, 48, bands: 3) + 60).Cast(NetVips.Enums.BandFormat.Uchar);
using var frames = NetVips.Image.Arrayjoin(new[] { frame, frame + 80, frame + 160 }, across: 1).Cast(NetVips.Enums.BandFormat.Uchar);
using var animated = frames.Mutate(m => m.Set(NetVips.GValue.GIntType, "page-height", 48));
var gifv = (await Upload(alice, "/api/v2/media", animated.WriteToBuffer(".gif"), "image/gif", "dance.gif")).Ok();
Assert.Equal("gifv", gifv.Body.Text("type"));
Assert.EndsWith(".mp4", gifv.Body.Text("url"));
Assert.Equal(64, gifv.Body["meta"]!["original"]!["width"]!.GetValue<int>());
var status = (await alice.Client.Post("/api/v1/statuses", ("status", "dancing"), ("media_ids[]", gifv.Body.Text("id")))).Ok();
Assert.Equal("gifv", status.Body["media_attachments"]![0]!.Text("type"));
var still = (await Upload(alice, "/api/v2/media", frame.WriteToBuffer(".gif"), "image/gif", "still.gif")).Ok();
Assert.Equal("image", still.Body.Text("type"));
}
[Fact]
public async Task Video_is_remuxed_without_its_metadata()
{
var alice = await _host.Mastodon("alice");
var video = await Made("mp4", "-f", "lavfi", "-i", "testsrc=duration=1:size=320x240:rate=10", "-f", "lavfi", "-i", "sine=frequency=440:duration=1",
"-metadata", "title=secret title", "-metadata", "comment=filmed at home", "-metadata:s:v:0", "handler_name=hidden handler",
"-c:v", "mpeg4", "-c:a", "aac", "-shortest");
Assert.Contains("secret title", Encoding.Latin1.GetString(video));
// v2 answers before it is processed: 202, no url yet, and it can't be posted until it is
var accepted = await Upload(alice, "/api/v2/media", video, "video/mp4", "clip.mp4");
Assert.Equal(HttpStatusCode.Accepted, accepted.Status);
Assert.Null(accepted.Body["url"]);
Assert.Equal(HttpStatusCode.PartialContent, (await alice.Client.Get($"/api/v1/media/{accepted.Body.Text("id")}")).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", "too soon"), ("media_ids[]", accepted.Body.Text("id")))).Status);
var uploaded = (await Processed(alice, accepted)).Ok();
Assert.Equal("video", uploaded.Body.Text("type"));
Assert.Equal(320, uploaded.Body["meta"]!["original"].Number("width"));
Assert.EndsWith(".mp4", uploaded.Body.Text("url"));
Assert.EndsWith(".jpg", uploaded.Body.Text("preview_url"));
var probe = await Probe(uploaded.Body.Text("url"));
Assert.Contains("\"codec_type\": \"video\"", probe);
// MPEG-4 Part 2, which browsers don't play, made H.264
Assert.Contains("\"codec_name\": \"h264\"", probe);
Assert.DoesNotContain("secret title", probe);
Assert.DoesNotContain("filmed at home", probe);
Assert.DoesNotContain("hidden handler", probe);
}
[Fact]
public async Task Audio_is_remuxed_without_its_metadata()
{
var alice = await _host.Mastodon("alice");
var audio = await Made("m4a", "-f", "lavfi", "-i", "sine=frequency=330:duration=1", "-metadata", "title=secret song", "-metadata", "artist=Alice Smith",
"-c:a", "aac");
// v1 waits for it
var uploaded = (await Upload(alice, "/api/v1/media", audio, "audio/mp4", "song.m4a")).Ok();
Assert.Equal("audio", uploaded.Body.Text("type"));
var probe = await Probe(uploaded.Body.Text("url"));
Assert.Contains("\"codec_type\": \"audio\"", probe);
Assert.DoesNotContain("secret song", probe);
Assert.DoesNotContain("Alice Smith", probe);
}
// FLAC is served as what it is (ASP.NET's map has no entry for it, so it was a 404)
[Fact]
public async Task Flac_is_served()
{
var alice = await _host.Mastodon("alice");
var flac = await Made("flac", "-f", "lavfi", "-i", "sine=frequency=500:duration=1", "-c:a", "flac");
var uploaded = (await Upload(alice, "/api/v1/media", flac, "audio/flac", "song.flac")).Ok();
var served = await _host.Client().GetAsync(new Uri(uploaded.Body.Text("url")).PathAndQuery, Token);
Assert.Equal(HttpStatusCode.OK, served.StatusCode);
Assert.Equal("audio/flac", served.Content.Headers.ContentType?.MediaType);
}
// a video larger than video_matrix_limit is made smaller, its shape kept
[Fact]
public async Task A_video_larger_than_the_limit_is_made_smaller()
{
var alice = await _host.Mastodon("alice");
var video = await Made("mp4", "-f", "lavfi", "-i", "testsrc=duration=1:size=2000x1500:rate=5", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p");
var uploaded = (await Upload(alice, "/api/v1/media", video, "video/mp4", "big.mp4")).Ok();
var width = uploaded.Body["meta"]!["original"].Number("width");
var height = uploaded.Body["meta"]!["original"].Number("height");
Assert.True((long)width * height <= 2_304_000, $"{width}x{height}");
Assert.InRange((double)width / height, 1.3, 1.37);
}
static byte[] Bytes(int length)
{
var bytes = new byte[length];
Random.Shared.NextBytes(bytes);
return bytes;
}
string Served(byte[] bytes, string contentType = "video/mp4")
{
var path = $"/media/{Guid.NewGuid():N}.bin";
_peer.ServeFile(path, bytes, contentType);
return _peer.A + path;
}
static HttpRequestMessage Ranged(string url, long from, long to)
{
var request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Range = new RangeHeaderValue(from, to);
return request;
}
[Fact]
public async Task Every_remote_media_address_the_api_returns_goes_through_the_proxy()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var document = bob.Document();
document["icon"] = new JsonObject { ["type"] = "Image", ["url"] = _peer.A + "/avatar.png" };
document["image"] = new JsonObject { ["type"] = "Image", ["url"] = _peer.A + "/header.png" };
_peer.Serve($"/users/{bob.Name}", document.ToJsonString());
var bobId = (await _host.Known(bob)).ID;
var post = await _host.PublicPostFrom(bob, alice, "<p>look :blob:</p>", note =>
{
note["attachment"] = new JsonArray(new JsonObject
{
["type"] = "Document", ["mediaType"] = "image/png", ["url"] = _peer.A + "/picture.png", ["name"] = "a picture"
});
note["tag"]!.AsArray().Add(new JsonObject
{
["type"] = "Emoji", ["name"] = ":blob:", ["icon"] = new JsonObject { ["type"] = "Image", ["url"] = _peer.A + "/blob.png" }
});
});
var proxied = $"{PrivaPubHost.Base}/media/proxy/";
var account = (await alice.Client.Get($"/api/v1/accounts/{bobId}")).Ok().Body;
var status = (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body;
foreach (var field in new[] { "avatar", "avatar_static", "header", "header_static" })
Assert.StartsWith(proxied, account.Text(field));
Assert.StartsWith(proxied, status["account"].Text("avatar"));
var attachment = Assert.Single(status["media_attachments"]!.AsArray());
Assert.StartsWith(proxied, attachment.Text("url"));
Assert.StartsWith(proxied, attachment.Text("preview_url"));
Assert.Equal(_peer.A + "/picture.png", attachment.Text("remote_url"));
Assert.Equal("a picture", attachment.Text("description"));
Assert.StartsWith(proxied, Assert.Single(status["emojis"]!.AsArray()).Text("url"));
var everything = account.ToJsonString() + status.ToJsonString();
foreach (var file in new[] { "/avatar.png", "/header.png", "/blob.png" })
Assert.DoesNotContain(_peer.A + file, everything);
}
[Fact]
public async Task The_proxy_refuses_a_url_it_did_not_sign()
{
var proxy = _host.Get<IMediaProxy>();
var remote = Served(Bytes(100));
var wrapped = proxy.Wrap(remote);
var parts = new Uri(wrapped).AbsolutePath.Split('/');
var other = new Uri(proxy.Wrap(Served(Bytes(100)))).AbsolutePath.Split('/');
using var client = _host.Client();
Assert.StartsWith($"{PrivaPubHost.Base}/media/proxy/", wrapped);
Assert.Equal(HttpStatusCode.NotFound, (await client.GetAsync($"/media/proxy/AAAAAAAAAAAAAAAAAAAAAA/{parts[^1]}", Token)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await client.GetAsync($"/media/proxy/{parts[^2]}/{other[^1]}", Token)).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await client.GetAsync($"/media/proxy/{parts[^2]}/!!!", Token)).StatusCode);
Assert.Empty(_peer.Requests);
Assert.Equal($"{PrivaPubHost.Base}/media/files/a.jpg", proxy.Wrap($"{PrivaPubHost.Base}/media/files/a.jpg"));
}
[Fact]
public async Task A_ranged_request_is_streamed_as_206_and_never_cached()
{
var proxy = _host.Get<IMediaProxy>();
var bytes = Bytes(10_000);
var remote = Served(bytes);
using var client = _host.Client();
using var response = await client.SendAsync(Ranged(proxy.Wrap(remote), 100, 199), Token);
Assert.Equal(HttpStatusCode.PartialContent, response.StatusCode);
Assert.Equal("bytes 100-199/10000", response.Content.Headers.ContentRange!.ToString());
Assert.Equal(bytes[100..200], await response.Content.ReadAsByteArrayAsync(Token));
Assert.Equal("nosniff", response.Headers.GetValues("X-Content-Type-Options").Single());
Assert.Equal(default, proxy.Cached(remote));
using var again = await client.SendAsync(Ranged(proxy.Wrap(remote), 0, 9), Token);
Assert.Equal(bytes[..10], await again.Content.ReadAsByteArrayAsync(Token));
Assert.Equal(2, _peer.Requests.Count);
Assert.All(_peer.Requests, r => Assert.StartsWith("bytes=", r.Headers["Range"]));
}
[Fact]
public async Task A_whole_download_is_cached_and_then_served_with_ranges()
{
var proxy = _host.Get<IMediaProxy>();
var bytes = Bytes(5_000);
var remote = Served(bytes, "image/png");
using var client = _host.Client();
using var whole = await client.GetAsync(proxy.Wrap(remote), Token);
Assert.Equal(HttpStatusCode.OK, whole.StatusCode);
Assert.Equal("image/png", whole.Content.Headers.ContentType!.MediaType);
Assert.Equal(bytes, await whole.Content.ReadAsByteArrayAsync(Token));
Assert.NotNull(proxy.Cached(remote).Path);
using var part = await client.SendAsync(Ranged(proxy.Wrap(remote), 10, 19), Token);
Assert.Equal(HttpStatusCode.PartialContent, part.StatusCode);
Assert.Equal(bytes[10..20], await part.Content.ReadAsByteArrayAsync(Token));
using var cached = await client.GetAsync(proxy.Wrap(remote), Token);
Assert.Equal(bytes, await cached.Content.ReadAsByteArrayAsync(Token));
Assert.Single(_peer.Requests);
}
// clients asking at once for a file not cached yet share one download
[Fact]
public async Task Clients_asking_at_once_share_one_download()
{
var proxy = _host.Get<IMediaProxy>();
var bytes = Bytes(200_000);
var remote = Served(bytes, "image/png");
var path = new Uri(remote).AbsolutePath;
using var client = _host.Client();
var answers = await Task.WhenAll(Enumerable.Range(0, 5).Select(_ => client.GetAsync(proxy.Wrap(remote), Token)));
foreach (var answer in answers)
Assert.Equal(bytes, await answer.Content.ReadAsByteArrayAsync(Token));
Assert.Single(_peer.Requests, r => r.Path == path);
}
// a remote file that can't be had is a 404 no browser keeps
[Fact]
public async Task A_remote_file_that_fails_is_not_cached_by_browsers()
{
var proxy = _host.Get<IMediaProxy>();
using var client = _host.Client();
using var missing = await client.GetAsync(proxy.Wrap($"{_peer.A}/media/{Guid.NewGuid():N}.png"), Token);
Assert.Equal(HttpStatusCode.NotFound, missing.StatusCode);
Assert.Null(missing.Headers.CacheControl);
}
[Fact]
public async Task Anything_over_the_proxy_limit_is_streamed_and_never_cached()
{
var host = await SmallProxyHost.Shared();
var proxy = host.Get<IMediaProxy>();
var bytes = Bytes(SmallProxyHost.Limit * 4);
var remote = Served(bytes);
using var client = host.Client();
using var response = await client.GetAsync(proxy.Wrap(remote), Token);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.Equal(bytes.Length, response.Content.Headers.ContentLength);
Assert.Equal(bytes, await response.Content.ReadAsByteArrayAsync(Token));
Assert.Equal(default, proxy.Cached(remote));
using var again = await client.GetAsync(proxy.Wrap(remote), Token);
Assert.Equal(bytes, await again.Content.ReadAsByteArrayAsync(Token));
// the first time, an attempt to cache it and the stream; then it is known to be too big, and only streamed
Assert.Equal(3, _peer.Requests.Count);
var small = Served(Bytes(SmallProxyHost.Limit / 2));
using (var fits = await client.GetAsync(proxy.Wrap(small), Token))
Assert.Equal(HttpStatusCode.OK, fits.StatusCode);
Assert.NotNull(proxy.Cached(small).Path);
}
}
public sealed class SmallProxyHost : PrivaPubHost
{
public const int Limit = 16 * 1024;
static readonly SemaphoreSlim Boot = new(1, 1);
static SmallProxyHost _shared;
public static new async Task<SmallProxyHost> Shared()
{
await PrivaPubHost.Shared();
await Boot.WaitAsync();
try
{
if (_shared == default)
{
var host = new SmallProxyHost();
_ = host.Services;
_shared = host;
}
return _shared;
}
finally
{
Boot.Release();
}
}
protected override IEnumerable<KeyValuePair<string, string>> Settings() =>
base.Settings().Append(new KeyValuePair<string, string>("Media:MaxProxiedBytes", Limit.ToString(System.Globalization.CultureInfo.InvariantCulture)));
}
}
@@ -0,0 +1,158 @@
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Media;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
// Mastodon's scheduled statuses: a post given a scheduled_at waits as asked, can be moved or dropped, and is
// published by its job when its time comes
[Trait("Category", "Integration")]
public sealed class MastodonScheduledStatusesTests : IAsyncLifetime
{
PrivaPubHost _host;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static string In(TimeSpan from) => DateTime.UtcNow.Add(from).ToString("o");
[Fact]
public async Task A_scheduled_post_waits_is_moved_and_is_dropped_by_its_owner_only()
{
var alice = await _host.Mastodon("alice");
var mallory = await _host.Mastodon("mallory");
var scheduled = (await alice.Client.Post("/api/v1/statuses", ("status", "later, friends"), ("visibility", "unlisted"),
("spoiler_text", "soon"), ("scheduled_at", In(TimeSpan.FromMinutes(30))))).Ok().Object;
var id = scheduled.Text("id");
Assert.Null(scheduled["uri"]);
Assert.Equal("later, friends", scheduled["params"].Text("text"));
Assert.Equal("unlisted", scheduled["params"].Text("visibility"));
Assert.Equal("soon", scheduled["params"].Text("spoiler_text"));
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/statuses")).Ok().Array);
Assert.Equal(new[] { id }, (await alice.Client.Get("/api/v1/scheduled_statuses")).Ok().Ids);
var later = DateTime.UtcNow.AddHours(2);
var moved = (await alice.Client.Put($"/api/v1/scheduled_statuses/{id}", ("scheduled_at", later.ToString("o")))).Ok().Object;
Assert.Equal(later.ToString("yyyy-MM-ddTHH:mm"), moved.Text("scheduled_at")[..16]);
Assert.Equal(HttpStatusCode.UnprocessableEntity,
(await alice.Client.Put($"/api/v1/scheduled_statuses/{id}", ("scheduled_at", In(TimeSpan.FromMinutes(2))))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity,
(await alice.Client.Post("/api/v1/statuses", ("status", "too soon"), ("scheduled_at", In(TimeSpan.FromMinutes(1))))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity,
(await alice.Client.Post("/api/v1/statuses", ("status", " "), ("scheduled_at", In(TimeSpan.FromHours(1))))).Status);
Assert.Empty((await mallory.Client.Get("/api/v1/scheduled_statuses")).Ok().Array);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Get($"/api/v1/scheduled_statuses/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await mallory.Client.Delete($"/api/v1/scheduled_statuses/{id}")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get("/api/v1/scheduled_statuses")).Status);
(await alice.Client.Delete($"/api/v1/scheduled_statuses/{id}")).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/scheduled_statuses")).Ok().Array);
await DB.Default.Update<Job>().Match(j => j.Kind == JobKind.PublishScheduled && j.Payload == id)
.Modify(j => j.RunAt, DateTime.UtcNow).ExecuteAsync(Token);
await _host.Run(j => j.Kind == JobKind.PublishScheduled && j.Payload == id, Token);
Assert.Empty((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/statuses")).Ok().Array);
}
[Fact]
public async Task A_scheduled_post_is_published_with_its_media_when_its_time_comes()
{
var alice = await _host.Mastodon("alice");
var upload = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(16, 16), "image/jpeg", "a.jpg"));
var mediaId = (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media") { Content = upload })).Ok().Body.Text("id");
var id = (await alice.Client.Post("/api/v1/statuses", ("status", "on time"), ("media_ids[]", mediaId), ("poll[options][]", "yes"),
("poll[options][]", "no"), ("poll[expires_in]", "3600"), ("scheduled_at", In(TimeSpan.FromMinutes(10))))).Ok().Object.Text("id");
Assert.Equal(id, (await DB.Default.Find<MediaAttachment>().MatchID(mediaId).ExecuteFirstAsync(Token)).ScheduledStatusId);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", "same media"), ("media_ids[]", mediaId),
("scheduled_at", In(TimeSpan.FromMinutes(20))))).Status);
// not due yet: its job leaves it waiting
Assert.Equal(1, await _host.Run(j => j.Kind == JobKind.PublishScheduled && j.Payload == id, Token));
Assert.Single((await alice.Client.Get("/api/v1/scheduled_statuses")).Ok().Array);
// its time comes (moved into the past, as the clock would have it)
await DB.Default.Update<ScheduledStatus>().MatchID(id).Modify(s => s.ScheduledAt, DateTime.UtcNow.AddSeconds(-1)).ExecuteAsync(Token);
await DB.Default.Update<Job>().Match(j => j.Kind == JobKind.PublishScheduled && j.Payload == id).Modify(j => j.State, JobState.Pending).ExecuteAsync(Token);
await _host.Run(j => j.Kind == JobKind.PublishScheduled && j.Payload == id, Token);
Assert.Empty((await alice.Client.Get("/api/v1/scheduled_statuses")).Ok().Array);
var published = Assert.Single((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/statuses")).Ok().Array)!;
Assert.Contains("on time", published.Text("content"));
Assert.Equal(mediaId, Assert.Single(published["media_attachments"]!.AsArray()).Text("id"));
Assert.Equal(2, published["poll"]!["options"]!.AsArray().Count);
var media = await DB.Default.Find<MediaAttachment>().MatchID(mediaId).ExecuteFirstAsync(Token);
Assert.Equal(published.Text("id"), media.PostId);
Assert.Null(media.ScheduledStatusId);
}
async Task<string> Picture(Mastodon account)
{
var upload = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(16, 16), "image/jpeg", "a.jpg"));
return (await account.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media") { Content = upload })).Ok().Body.Text("id");
}
// a scheduled post's media are its alone: a post now, or an edit, can't take them
[Fact]
public async Task A_scheduled_posts_media_cannot_be_taken_by_a_post_now()
{
var alice = await _host.Mastodon("alice");
var mediaId = await Picture(alice);
(await alice.Client.Post("/api/v1/statuses", ("status", "later"), ("media_ids[]", mediaId), ("scheduled_at", In(TimeSpan.FromMinutes(10))))).Ok();
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", "now"), ("media_ids[]", mediaId))).Status);
var plain = (await alice.Client.Post("/api/v1/statuses", ("status", "plain"))).Ok().Body.Text("id");
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Put($"/api/v1/statuses/{plain}", ("status", "edited"), ("media_ids[]", mediaId))).Status);
var media = await DB.Default.Find<MediaAttachment>().OneAsync(mediaId, Token);
Assert.Null(media.PostId);
Assert.NotNull(media.ScheduledStatusId);
}
// two posts asking for the same upload at once: exactly one gets it
[Fact]
public async Task Two_posts_racing_for_one_upload_never_both_get_it()
{
var alice = await _host.Mastodon("alice");
for (var round = 0; round < 5; round++)
{
var mediaId = await Picture(alice);
var answers = await Task.WhenAll(Enumerable.Range(0, 2).Select(i =>
alice.Client.Post("/api/v1/statuses", ("status", $"race {round} {i}"), ("media_ids[]", mediaId))));
Assert.Single(answers, a => a.Status == HttpStatusCode.OK);
Assert.Single(answers, a => a.Status == HttpStatusCode.UnprocessableEntity);
}
}
// a persona deleted since it scheduled a post publishes nothing, and what the post held is trashed
[Fact]
public async Task A_gone_personas_scheduled_post_never_publishes()
{
var gone = await _host.Mastodon($"gone{Guid.NewGuid():N}"[..12]);
var mediaId = await Picture(gone);
var id = (await gone.Client.Post("/api/v1/statuses", ("status", "from beyond"), ("media_ids[]", mediaId), ("scheduled_at", In(TimeSpan.FromMinutes(10))))).Ok().Object.Text("id");
await DB.Default.Update<Models.User.Avatar>().MatchID(gone.Persona.Id).Modify(a => a.DeletionAt, DateTime.UtcNow).ExecuteAsync(Token);
await DB.Default.Update<ScheduledStatus>().MatchID(id).Modify(s => s.ScheduledAt, DateTime.UtcNow.AddSeconds(-1)).ExecuteAsync(Token);
await DB.Default.Update<Job>().Match(j => j.Kind == JobKind.PublishScheduled && j.Payload == id).Modify(j => j.RunAt, DateTime.UtcNow).ExecuteAsync(Token);
await _host.Run(j => j.Kind == JobKind.PublishScheduled && j.Payload == id, Token);
Assert.False(await DB.Default.Find<Models.Post.Post>().Match(p => p.GroupUserId == gone.Persona.Id && p.Text == "from beyond").ExecuteAnyAsync(Token));
Assert.False(await DB.Default.Find<ScheduledStatus>().Match(s => s.ID == id).ExecuteAnyAsync(Token));
Assert.NotNull((await DB.Default.Find<MediaAttachment>().OneAsync(mediaId, Token)).TrashedAt);
}
}
}
@@ -0,0 +1,687 @@
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class MastodonStatusesTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static IEnumerable<string> Strings(JsonNode node) => node!.AsArray().Select(n => n!.GetValue<string>());
static async Task<JsonObject> Image(Mastodon account) =>
(await account.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v1/media")
{
Content = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(80, 60), "image/jpeg", "a.jpg"))
})).Ok().Object;
[Fact]
public async Task Each_visibility_is_addressed_the_way_mastodon_addresses_it()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var carol = new RemoteActor(_peer, "carol");
await _host.FollowedBy(alice, bob);
_peer.WebFinger(carol);
var followers = alice.Uri + "/groupies";
var open = await alice.Status("for everyone");
var unlisted = await alice.Status("quietly public", ("visibility", "unlisted"));
var quiet = await alice.Status("for followers", ("visibility", "private"));
var direct = await alice.Status($"@{carol.Handle()} just you", ("visibility", "direct"));
Assert.Equal(new[] { "public", "unlisted", "private", "direct" }, new[] { open, unlisted, quiet, direct }.Select(s => s.Text("visibility")));
var creates = (await bob.Delivered(since)).Where(d => d.Type() == "Create").ToDictionary(d => d["object"].Text("id"));
Assert.Equal(3, creates.Count);
var publicNote = creates[open.Text("uri")]["object"];
Assert.Equal(new[] { MastodonHelpers.Public }, Strings(publicNote["to"]));
Assert.Equal(new[] { followers }, Strings(publicNote["cc"]));
var unlistedNote = creates[unlisted.Text("uri")]["object"];
Assert.Equal(new[] { followers }, Strings(unlistedNote["to"]));
Assert.Equal(new[] { MastodonHelpers.Public }, Strings(unlistedNote["cc"]));
var quietNote = creates[quiet.Text("uri")]["object"];
Assert.Equal(new[] { followers }, Strings(quietNote["to"]));
// named again in cc: Pleroma and Akkoma read an empty cc without a "/followers" address as a direct message
Assert.Equal(new[] { followers }, Strings(quietNote["cc"]));
var whisper = Assert.Single(await carol.Delivered(since));
Assert.Equal(direct.Text("uri"), whisper["object"].Text("id"));
Assert.Equal(new[] { carol.Id }, Strings(whisper["object"]!["to"]));
Assert.Empty(whisper["object"]!["cc"]!.AsArray());
Assert.Equal(carol.Handle(), Assert.Single(direct["mentions"]!.AsArray()).Text("acct"));
}
[Fact]
public async Task A_content_warning_is_kept_as_spoiler_text_and_federates_as_a_summary()
{
var alice = await _host.Mastodon("alice");
var warned = await alice.Status("the butler did it", ("spoiler_text", "spoilers"));
Assert.True(warned.Flag("sensitive"));
Assert.Equal("spoilers", warned.Text("spoiler_text"));
var note = (await _host.ActivityPub($"/peasants/{alice.UserName}/scribbles/{warned.Text("id")}")).Ok().Body;
Assert.Equal("spoilers", note.Text("summary"));
Assert.True(note.Flag("sensitive"));
Assert.Contains("the butler did it", note.Text("content"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", " "))).Status);
}
[Fact]
public async Task A_reply_to_a_local_post_threads_counts_and_shows_in_the_context()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var root = await alice.Status("what do you think?");
var reply = await bob.Status("I agree", ("in_reply_to_id", root.Text("id")));
var nested = await alice.Status("thanks", ("in_reply_to_id", reply.Text("id")));
Assert.Equal(root.Text("id"), reply.Text("in_reply_to_id"));
Assert.Equal(alice.Id, reply.Text("in_reply_to_account_id"));
Assert.Equal(1, (await bob.Client.Get($"/api/v1/statuses/{root.Text("id")}")).Ok().Body.Number("replies_count"));
var context = (await _host.Client().Get($"/api/v1/statuses/{reply.Text("id")}/context")).Ok();
Assert.Equal(new[] { root.Text("id") }, context.Body["ancestors"]!.AsArray().Select(s => s.Text("id")));
Assert.Equal(new[] { nested.Text("id") }, context.Body["descendants"]!.AsArray().Select(s => s.Text("id")));
var whole = (await alice.Client.Get($"/api/v1/statuses/{root.Text("id")}/context")).Ok();
Assert.Equal(new[] { reply.Text("id"), nested.Text("id") }, whole.Body["descendants"]!.AsArray().Select(s => s.Text("id")));
Assert.Equal(HttpStatusCode.NotFound, (await bob.Client.Post("/api/v1/statuses", ("status", "lost"), ("in_reply_to_id", "000000000000000000000000"))).Status);
}
[Fact]
public async Task A_reply_to_a_remote_post_mentions_its_author_and_reaches_its_inbox()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
_peer.WebFinger(bob);
var parent = await _host.PublicPostFrom(bob, alice);
var bobId = (await _host.Known(bob)).ID;
var reply = await alice.Status($"@{bob.Handle()} well said", ("in_reply_to_id", parent.ID));
Assert.Equal(parent.ID, reply.Text("in_reply_to_id"));
Assert.Equal(bobId, reply.Text("in_reply_to_account_id"));
var mention = Assert.Single(reply["mentions"]!.AsArray());
Assert.Equal((bobId, bob.Handle()), (mention.Text("id"), mention.Text("acct")));
var create = Assert.Single(await bob.Delivered(since), d => d.Type() == "Create");
var note = create["object"]!;
Assert.Equal(parent.ObjectURI, note.Text("inReplyTo"));
Assert.Contains(bob.Id, Strings(note["cc"]));
Assert.Contains(note["tag"]!.AsArray(), t => t.Text("type") == "Mention" && t.Text("href") == bob.Id);
Assert.Equal(1, (await alice.Client.Get($"/api/v1/statuses/{parent.ID}")).Ok().Body.Number("replies_count"));
}
// a remote thread opened by a persona has its replies read from its server (here its FEP-7888 context), at most hourly
[Fact]
public async Task Opening_a_remote_thread_reads_the_replies_that_never_reached_us()
{
var token = TestContext.Current.CancellationToken;
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var carol = new RemoteActor(_peer, "carol");
var context = $"{bob.Origin()}/contexts/{Guid.NewGuid():N}";
var post = await _host.PublicPostFrom(bob, alice, shape: note => note["context"] = context);
var answer = carol.Create("<p>said elsewhere</p>", new[] { MastodonHelpers.Public }, new[] { carol.Id + "/followers" },
note => note["inReplyTo"] = post.ObjectURI)["object"]!;
var answerId = answer["id"]!.GetValue<string>();
_peer.Serve(new Uri(answerId).AbsolutePath, answer.ToJsonString());
_peer.Serve(new Uri(context).AbsolutePath, new JsonObject
{
["id"] = context, ["type"] = "OrderedCollection", ["orderedItems"] = new JsonArray(post.ObjectURI, answerId)
}.ToJsonString());
async Task<int> Queued() => (await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.FetchReplies && j.Payload.Contains(post.ID)).ExecuteAsync(token)).Count;
(await _host.Client().Get($"/api/v1/statuses/{post.ID}/context")).Ok();
Assert.Equal(0, await Queued());
Assert.Empty((await alice.Client.Get($"/api/v1/statuses/{post.ID}/context")).Ok().Body["descendants"]!.AsArray());
Assert.Equal(1, await _host.Run(j => j.Kind == JobKind.FetchReplies && j.Payload.Contains(post.ID), token));
var descendants = (await alice.Client.Get($"/api/v1/statuses/{post.ID}/context")).Ok().Body["descendants"]!.AsArray();
Assert.Equal(answerId, Assert.Single(descendants).Text("uri"));
Assert.Equal(1, await Queued());
}
// Akkoma marks a post with sensitive media "sensitive" and leaves its summary empty: the media is hidden, the words are
// not, and no warning is made up for them (found by decePub's e2e tests on the town)
[Fact]
public async Task A_remote_post_sensitive_without_a_summary_hides_its_media_and_not_its_words()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
_peer.WebFinger(bob);
var post = await _host.PublicPostFrom(bob, alice, shape: note => { note["sensitive"] = true; note["summary"] = ""; });
var status = (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body;
Assert.True(status["sensitive"]!.GetValue<bool>());
Assert.Equal("", status.Text("spoiler_text"));
}
// a status asked for by id is shown even when the viewer blocked or muted its author, as Mastodon shows it: only lists
// leave such posts out (it once answered 200 with null, and decePub's thread page showed nothing)
[Fact]
public async Task A_status_of_an_author_the_viewer_blocked_is_still_shown_by_its_id()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
_peer.WebFinger(bob);
var post = await _host.PublicPostFrom(bob, alice);
var bobId = (await _host.Known(bob)).ID;
(await alice.Client.Post($"/api/v1/accounts/{bobId}/block")).Ok();
var status = (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body;
Assert.Equal(post.ID, status.Text("id"));
}
// a reader who deletes the @name decePub prefills still answers its author: GoToSocial dropped such a reply, addressed
// to nobody there (found by decePub's e2e tests on the town)
[Fact]
public async Task A_reply_without_the_authors_name_still_names_the_author_in_cc()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
_peer.WebFinger(bob);
var parent = await _host.PublicPostFrom(bob, alice);
var reply = await alice.Status("no name in this one", ("in_reply_to_id", parent.ID));
Assert.Empty(reply["mentions"]!.AsArray());
var create = Assert.Single(await bob.Delivered(since), d => d.Type() == "Create");
Assert.Contains(bob.Id, Strings(create["object"]!["cc"]));
}
[Fact]
public async Task A_poll_is_created_validated_and_federated_as_a_question()
{
var alice = await _host.Mastodon("alice");
var status = await alice.Status("tea or coffee?", ("poll[options][]", "tea"), ("poll[options][]", "coffee"), ("poll[expires_in]", "600"),
("poll[multiple]", "true"));
var poll = status["poll"]!;
Assert.Equal(new[] { "tea", "coffee" }, poll["options"]!.AsArray().Select(o => o.Text("title")));
Assert.True(poll.Flag("multiple"));
Assert.False(poll.Flag("expired"));
Assert.NotNull(poll.Text("expires_at"));
Assert.Equal(status.Text("id"), (await _host.Client().Get($"/api/v1/polls/{status.Text("id")}")).Ok().Body.Text("id"));
var question = (await _host.ActivityPub($"/peasants/{alice.UserName}/scribbles/{status.Text("id")}")).Ok().Body;
Assert.Equal("Question", question.Text("type"));
Assert.Equal(2, question["anyOf"]!.AsArray().Count);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", "one?"), ("poll[options][]", "only"),
("poll[expires_in]", "600"))).Status);
}
[Fact]
public async Task Media_is_attached_only_by_its_owner()
{
var alice = await _host.Mastodon("alice");
var mallory = await _host.Mastodon("mallory");
var upload = await Image(alice);
var status = await alice.Status("look", ("media_ids[]", upload.Text("id")));
var attachment = Assert.Single(status["media_attachments"]!.AsArray());
Assert.Equal(upload.Text("id"), attachment.Text("id"));
Assert.Equal("image", attachment.Text("type"));
var note = (await _host.ActivityPub($"/peasants/{alice.UserName}/scribbles/{status.Text("id")}")).Ok().Body;
Assert.Equal(upload.Text("url"), Assert.Single(note["attachment"]!.AsArray()).Text("url"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await mallory.Client.Post("/api/v1/statuses", ("status", "mine"), ("media_ids[]", (await Image(alice)).Text("id")))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", "again"), ("media_ids[]", upload.Text("id")))).Status);
}
[Fact]
public async Task A_quote_of_a_local_post_is_accepted_listed_and_revocable()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var original = await carol.Status("quote me");
var closed = await carol.Status("do not quote me", ("quote_approval_policy", "nobody"));
var quote = await alice.Status("so true", ("quoted_status_id", original.Text("id")));
Assert.Equal("accepted", quote["quote"].Text("state"));
Assert.Equal(original.Text("id"), quote["quote"]!["quoted_status"].Text("id"));
Assert.Equal(1, (await carol.Client.Get($"/api/v1/statuses/{original.Text("id")}")).Ok().Body.Number("quotes_count"));
Assert.Equal(new[] { quote.Text("id") }, (await _host.Client().Get($"/api/v1/statuses/{original.Text("id")}/quotes")).Ok().Ids);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post("/api/v1/statuses", ("status", "anyway"), ("quoted_status_id", closed.Text("id")))).Status);
Assert.Empty(closed["quote_approval"]!["automatic"]!.AsArray());
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post($"/api/v1/statuses/{original.Text("id")}/quotes/{quote.Text("id")}/revoke")).Status);
var revoked = (await carol.Client.Post($"/api/v1/statuses/{original.Text("id")}/quotes/{quote.Text("id")}/revoke")).Ok();
Assert.Equal(quote.Text("id"), revoked.Body.Text("id"));
Assert.Equal("revoked", revoked.Body["quote"].Text("state"));
Assert.Empty((await carol.Client.Get($"/api/v1/statuses/{original.Text("id")}/quotes")).Ok().Array);
Assert.Equal(0, (await carol.Client.Get($"/api/v1/statuses/{original.Text("id")}")).Ok().Body.Number("quotes_count"));
}
[Fact]
public async Task An_edit_keeps_its_history_shows_its_source_and_federates_an_update()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var other = await _host.Mastodon("other");
var bob = new RemoteActor(_peer, "bob");
await _host.FollowedBy(alice, bob);
var status = await alice.Status("first version", ("spoiler_text", "cw"));
var id = status.Text("id");
var edited = (await alice.Client.Put($"/api/v1/statuses/{id}", ("status", "second version"))).Ok();
Assert.NotNull(edited.Body.Text("edited_at"));
Assert.Contains("second version", edited.Body.Text("content"));
Assert.Contains("second version", (await other.Client.Get($"/api/v1/statuses/{id}")).Ok().Body.Text("content"));
var history = (await _host.Client().Get($"/api/v1/statuses/{id}/history")).Ok().Array;
Assert.Equal(2, history.Count);
Assert.Contains("first version", history[0].Text("content"));
Assert.Equal("cw", history[0].Text("spoiler_text"));
Assert.Contains("second version", history[1].Text("content"));
Assert.Equal(alice.Id, history[1]!["account"].Text("id"));
var source = (await alice.Client.Get($"/api/v1/statuses/{id}/source")).Ok();
Assert.Equal(("second version", ""), (source.Body.Text("text"), source.Body.Text("spoiler_text")));
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Get($"/api/v1/statuses/{id}/source")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Put($"/api/v1/statuses/{id}", ("status", "hijacked"))).Status);
var update = Assert.Single(await bob.Delivered(since), d => d.Type() == "Update");
Assert.Equal(status.Text("uri"), update["object"].Text("id"));
Assert.Contains("second version", update["object"].Text("content"));
}
[Fact]
public async Task Delete_returns_the_text_for_a_redraft_and_leaves_a_tombstone()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var other = await _host.Mastodon("other");
var bob = new RemoteActor(_peer, "bob");
await _host.FollowedBy(alice, bob);
var status = await alice.Status("regrettable", ("spoiler_text", "hot take"));
var id = status.Text("id");
var scribble = $"/peasants/{alice.UserName}/scribbles/{id}";
Assert.Equal(HttpStatusCode.OK, (await _host.ActivityPub(scribble)).Status);
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Delete($"/api/v1/statuses/{id}")).Status);
var deleted = (await alice.Client.Delete($"/api/v1/statuses/{id}")).Ok();
Assert.Equal(id, deleted.Body.Text("id"));
Assert.Equal("regrettable", deleted.Body.Text("text"));
Assert.Equal("hot take", deleted.Body.Text("spoiler_text"));
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/statuses/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Delete($"/api/v1/statuses/{id}")).Status);
var tombstone = await _host.ActivityPub(scribble);
Assert.Equal(HttpStatusCode.Gone, tombstone.Status);
Assert.Equal("Tombstone", tombstone.Body.Text("type"));
Assert.Equal(status.Text("uri"), tombstone.Body.Text("id"));
var delete = Assert.Single(await bob.Delivered(since), d => d.Type() == "Delete");
Assert.Equal(status.Text("uri"), delete["object"].Text("id"));
}
[Fact]
public async Task Favourites_and_reblogs_count_and_federate_with_their_undos()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var bob = new RemoteActor(_peer, "bob");
var remote = await _host.PublicPostFrom(bob, alice);
var path = $"/api/v1/statuses/{remote.ID}";
var liked = (await alice.Client.Post(path + "/favourite")).Ok();
Assert.True(liked.Body.Flag("favourited"));
Assert.Equal(1, liked.Body.Number("favourites_count"));
Assert.Equal(new[] { alice.Id }, (await _host.Client().Get(path + "/favourited_by")).Ok().Ids);
Assert.Equal(1, (await alice.Client.Post(path + "/favourite")).Ok().Body.Number("favourites_count"));
var unliked = (await alice.Client.Post(path + "/unfavourite")).Ok();
Assert.False(unliked.Body.Flag("favourited"));
Assert.Equal(0, unliked.Body.Number("favourites_count"));
var reblog = (await alice.Client.Post(path + "/reblog")).Ok();
Assert.Equal(remote.ID, reblog.Body["reblog"].Text("id"));
Assert.True(reblog.Body["reblog"].Flag("reblogged"));
Assert.Equal(1, reblog.Body["reblog"].Number("reblogs_count"));
Assert.Equal(new[] { alice.Id }, (await carol.Client.Get(path + "/reblogged_by")).Ok().Ids);
var unreblogged = (await alice.Client.Post(path + "/unreblog")).Ok();
Assert.False(unreblogged.Body.Flag("reblogged"));
Assert.Equal(0, unreblogged.Body.Number("reblogs_count"));
Assert.Empty((await carol.Client.Get(path + "/reblogged_by")).Ok().Array);
var sent = await bob.Delivered(since);
var like = Assert.Single(sent, d => d.Type() == "Like");
Assert.Equal((alice.Uri, remote.ObjectURI), (like.Text("actor"), like.Text("object")));
var announce = Assert.Single(sent, d => d.Type() == "Announce");
Assert.Equal(remote.ObjectURI, announce.Text("object"));
var undos = sent.Where(d => d.Type() == "Undo").ToList();
Assert.Equal(new[] { "Announce", "Like" }, undos.Select(u => u["object"].Text("type")).Order());
Assert.Equal(like.Text("id"), undos.Single(u => u["object"].Text("type") == "Like")["object"].Text("id"));
var local = await carol.Status("a local one");
(await alice.Client.Post($"/api/v1/statuses/{local.Text("id")}/favourite")).Ok();
var notification = Assert.Single((await carol.Client.Get("/api/v1/notifications?types[]=favourite")).Ok().Array);
Assert.Equal((alice.Id, local.Text("id")), (notification["account"].Text("id"), notification["status"].Text("id")));
var quiet = await carol.Status("followers only", ("visibility", "private"));
(await alice.Client.Post($"/api/v1/accounts/{carol.Id}/follow")).Ok();
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post($"/api/v1/statuses/{quiet.Text("id")}/reblog")).Status);
Assert.True((await alice.Client.Post($"/api/v1/statuses/{quiet.Text("id")}/favourite")).Ok().Body.Flag("favourited"));
}
[Fact]
public async Task A_favourite_or_a_boost_given_again_after_its_undo_is_a_new_activity_that_is_delivered()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var remote = await _host.PublicPostFrom(bob, alice);
var path = $"/api/v1/statuses/{remote.ID}";
foreach (var (on, off) in new[] { ("favourite", "unfavourite"), ("reblog", "unreblog") })
{
(await alice.Client.Post($"{path}/{on}")).Ok();
(await alice.Client.Post($"{path}/{off}")).Ok();
(await alice.Client.Post($"{path}/{on}")).Ok();
}
var sent = await bob.Delivered(since);
foreach (var type in new[] { "Like", "Announce" })
{
var given = sent.Where(d => d.Type() == type).ToList();
Assert.Equal(2, given.Count);
Assert.NotEqual(given[0].Text("id"), given[1].Text("id"));
var undo = Assert.Single(sent, d => d.Type() == "Undo" && d["object"].Text("type") == type);
Assert.Equal(given[0].Text("id"), undo["object"].Text("id"));
}
var status = (await alice.Client.Get(path)).Ok().Body;
Assert.True(status.Flag("favourited"));
Assert.True(status.Flag("reblogged"));
Assert.Equal(1, status.Number("reblogs_count"));
}
[Fact]
public async Task A_boost_reaches_the_authors_server_once_through_its_shared_inbox_and_a_like_its_own_inbox()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var remote = await _host.PublicPostFrom(bob, alice);
await DB.Default.Update<ForeignAvatar>().Match(f => f.ActorURI == bob.Id).Modify(f => f.SharedInboxURL, bob.SharedInbox)
.ExecuteAsync(TestContext.Current.CancellationToken);
(await alice.Client.Post($"/api/v1/statuses/{remote.ID}/reblog")).Ok();
(await alice.Client.Post($"/api/v1/statuses/{remote.ID}/unreblog")).Ok();
(await alice.Client.Post($"/api/v1/statuses/{remote.ID}/favourite")).Ok();
var shared = await Jobs.Deliveries(bob.SharedInbox, since, TestContext.Current.CancellationToken);
Assert.Single(shared, d => d.Type() == "Announce");
Assert.Single(shared, d => d.Type() == "Undo");
var own = await bob.Delivered(since);
Assert.DoesNotContain(own, d => d.Type() is "Announce" or "Undo");
Assert.Single(own, d => d.Type() == "Like");
}
[Fact]
public async Task Bookmarks_are_kept_listed_and_removed()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var first = await carol.Status("worth keeping");
var second = await carol.Status("this too");
Assert.True((await alice.Client.Post($"/api/v1/statuses/{first.Text("id")}/bookmark")).Ok().Body.Flag("bookmarked"));
(await alice.Client.Post($"/api/v1/statuses/{second.Text("id")}/bookmark")).Ok();
(await alice.Client.Post($"/api/v1/statuses/{second.Text("id")}/bookmark")).Ok();
Assert.Equal(new[] { second.Text("id"), first.Text("id") }, (await alice.Client.Get("/api/v1/bookmarks")).Ok().Ids);
Assert.Empty((await carol.Client.Get("/api/v1/bookmarks")).Ok().Array);
var page = (await alice.Client.Get("/api/v1/bookmarks?limit=1")).Ok();
Assert.Equal(new[] { second.Text("id") }, page.Ids);
Assert.Equal(new[] { first.Text("id") }, (await alice.Client.Get(page.Link("next"))).Ok().Ids);
Assert.False((await alice.Client.Post($"/api/v1/statuses/{first.Text("id")}/unbookmark")).Ok().Body.Flag("bookmarked"));
Assert.Equal(new[] { second.Text("id") }, (await alice.Client.Get("/api/v1/bookmarks")).Ok().Ids);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Post("/api/v1/statuses/000000000000000000000000/bookmark")).Status);
}
[Fact]
public async Task Pins_take_only_the_owners_public_posts()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var mine = await alice.Status("pin me");
var quiet = await alice.Status("not this", ("visibility", "private"));
var theirs = await carol.Status("not yours");
Assert.True((await alice.Client.Post($"/api/v1/statuses/{mine.Text("id")}/pin")).Ok().Body.Flag("pinned"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post($"/api/v1/statuses/{theirs.Text("id")}/pin")).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post($"/api/v1/statuses/{quiet.Text("id")}/pin")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await carol.Client.Post($"/api/v1/statuses/{quiet.Text("id")}/pin")).Status);
var featured = (await _host.ActivityPub($"/peasants/{alice.UserName}/trophies")).Ok().Body;
Assert.Equal(mine.Text("uri"), Assert.Single(featured["orderedItems"]!.AsArray()).Text("id"));
for (var i = 0; i < 4; i++)
(await alice.Client.Post($"/api/v1/statuses/{(await alice.Status($"more {i}")).Text("id")}/pin")).Ok();
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post($"/api/v1/statuses/{(await alice.Status("one too many")).Text("id")}/pin")).Status);
Assert.False((await alice.Client.Post($"/api/v1/statuses/{mine.Text("id")}/unpin")).Ok().Body.Flag("pinned"));
Assert.Equal(4, (await alice.Client.Get($"/api/v1/accounts/{alice.Id}/statuses?pinned=true")).Ok().Array.Count);
}
[Fact]
public async Task The_interaction_policy_sets_who_may_quote_and_matches_the_activitypub_note()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
await _host.FollowedBy(alice, bob);
var open = await alice.Status("quotable");
var quiet = await alice.Status("followers only", ("visibility", "private"));
var scribble = $"/peasants/{alice.UserName}/scribbles/{open.Text("id")}";
string CanQuote(JsonNode note) => Assert.Single(note["interactionPolicy"]!["canQuote"]!["automaticApproval"]!.AsArray())!.GetValue<string>();
Assert.Equal(new[] { "public" }, Strings(open["quote_approval"]!["automatic"]));
Assert.Equal(MastodonHelpers.Public, CanQuote((await _host.ActivityPub(scribble)).Ok().Body));
var followers = (await alice.Client.Put($"/api/v1/statuses/{open.Text("id")}/interaction_policy", ("quote_approval_policy", "followers"))).Ok();
Assert.Equal(new[] { "followers" }, Strings(followers.Body["quote_approval"]!["automatic"]));
Assert.Equal(alice.Uri + "/groupies", CanQuote((await _host.ActivityPub(scribble)).Ok().Body));
var nobody = (await alice.Client.Put($"/api/v1/statuses/{open.Text("id")}/interaction_policy", ("quote_approval_policy", "nobody"))).Ok();
Assert.Empty(nobody.Body["quote_approval"]!["automatic"]!.AsArray());
Assert.Equal(alice.Uri, CanQuote((await _host.ActivityPub(scribble)).Ok().Body));
var forced = (await alice.Client.Put($"/api/v1/statuses/{quiet.Text("id")}/interaction_policy", ("quote_approval_policy", "public"))).Ok();
Assert.Empty(forced.Body["quote_approval"]!["automatic"]!.AsArray());
var updates = (await bob.Delivered(since)).Where(d => d.Type() == "Update").ToList();
Assert.Equal(3, updates.Count);
Assert.Equal(alice.Uri, CanQuote(updates.Single(u => u["object"].Text("id") == quiet.Text("uri"))["object"]));
Assert.Equal(HttpStatusCode.UnprocessableEntity,
(await alice.Client.Put($"/api/v1/statuses/{open.Text("id")}/interaction_policy", ("quote_approval_policy", "everyone"))).Status);
Assert.Equal(HttpStatusCode.NotFound,
(await (await _host.Mastodon("other")).Client.Put($"/api/v1/statuses/{open.Text("id")}/interaction_policy", ("quote_approval_policy", "public"))).Status);
}
[Fact]
public async Task A_stranger_gets_404_for_followers_only_and_direct_posts()
{
var alice = await _host.Mastodon("alice");
var follower = await _host.Mastodon("follower");
var friend = await _host.Mastodon("friend");
var stranger = await _host.Mastodon("stranger");
(await follower.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
var quiet = await alice.Status("followers only", ("visibility", "private"));
var direct = await alice.Status($"@{friend.UserName} only you", ("visibility", "direct"));
var anonymous = _host.Client();
foreach (var id in new[] { quiet.Text("id"), direct.Text("id") })
{
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Get($"/api/v1/statuses/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Get($"/api/v1/statuses/{id}/context")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Post($"/api/v1/statuses/{id}/favourite")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Post($"/api/v1/statuses/{id}/bookmark")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Post("/api/v1/statuses", ("status", "re"), ("in_reply_to_id", id))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get($"/api/v1/statuses/{id}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await anonymous.Get($"/api/v1/statuses/{id}/favourited_by")).Status);
}
Assert.Empty((await stranger.Client.Get($"/api/v1/statuses?id[]={quiet.Text("id")}&id[]={direct.Text("id")}")).Ok().Array);
Assert.Equal(new[] { quiet.Text("id") }, (await follower.Client.Get($"/api/v1/statuses?id[]={quiet.Text("id")}&id[]={direct.Text("id")}")).Ok().Ids);
Assert.Equal(new[] { direct.Text("id") }, (await friend.Client.Get($"/api/v1/statuses?id[]={quiet.Text("id")}&id[]={direct.Text("id")}")).Ok().Ids);
Assert.Equal(HttpStatusCode.OK, (await follower.Client.Get($"/api/v1/statuses/{quiet.Text("id")}/context")).Status);
Assert.Equal(quiet.Text("id"), (await follower.Client.Post("/api/v1/statuses", ("status", "re"), ("in_reply_to_id", quiet.Text("id")))).Ok().Body.Text("in_reply_to_id"));
Assert.Equal(HttpStatusCode.NotFound, (await _host.ActivityPub($"/peasants/{alice.UserName}/scribbles/{quiet.Text("id")}")).Status);
}
[Fact]
public async Task A_located_post_is_never_reachable_by_id_by_anyone_else()
{
var alice = await _host.Mastodon("alice");
var other = await _host.Mastodon("other");
var id = await _host.Located(alice, "the bench by the river");
var anonymous = _host.Client();
foreach (var client in new[] { other.Client, anonymous })
{
foreach (var path in new[] { "", "/context", "/history", "/favourited_by", "/reblogged_by", "/quotes" })
Assert.Equal(HttpStatusCode.NotFound, (await client.Get($"/api/v1/statuses/{id}{path}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await client.Get($"/api/privapub/v1/statuses/{id}/provenance")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await client.Get($"/api/v1/pleroma/statuses/{id}/reactions")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await client.Get($"/api/v1/polls/{id}")).Status);
Assert.Empty((await client.Get($"/api/v1/statuses?id[]={id}")).Ok().Array);
Assert.DoesNotContain(id, (await client.Get($"/api/v1/accounts/{alice.Id}/statuses")).Ok().Ids);
}
foreach (var action in new[] { "favourite", "reblog", "bookmark", "pin" })
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Post($"/api/v1/statuses/{id}/{action}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Put($"/api/v1/pleroma/statuses/{id}/reactions/%F0%9F%91%8D")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Post("/api/v1/statuses", ("status", "re"), ("in_reply_to_id", id))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Post("/api/v1/statuses", ("status", "qt"), ("quoted_status_id", id))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _host.ActivityPub($"/peasants/{alice.UserName}/scribbles/{id}")).Status);
}
[Fact]
public async Task Creating_honours_the_idempotency_key_the_scopes_and_needs_a_recipient_for_a_direct_message()
{
var alice = await _host.Mastodon("alice");
var reader = await _host.Mastodon("reader", scopes: "read");
var key = Guid.NewGuid().ToString("N");
Task<ApiAnswer> Keyed(string text)
{
var request = new HttpRequestMessage(HttpMethod.Post, "/api/v1/statuses") { Content = new FormUrlEncodedContent(new Dictionary<string, string> { ["status"] = text }) };
request.Headers.Add("Idempotency-Key", key);
return alice.Client.Exchange(request);
}
var first = (await Keyed("once")).Ok();
var second = (await Keyed("once again")).Ok();
Assert.Equal(first.Body.Text("id"), second.Body.Text("id"));
Assert.Single((await alice.Client.Get($"/api/v1/accounts/{alice.Id}/statuses")).Ok().Array);
var noRecipient = await alice.Client.Post("/api/v1/statuses", ("status", "to nobody"), ("visibility", "direct"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, noRecipient.Status);
Assert.Contains("recipient", noRecipient.Body.Text("error"));
Assert.Equal(HttpStatusCode.Forbidden, (await reader.Client.Post("/api/v1/statuses", ("status", "read only"))).Status);
Assert.Equal(HttpStatusCode.Forbidden, (await reader.Client.Post($"/api/v1/statuses/{first.Body.Text("id")}/favourite")).Status);
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get($"/api/v1/statuses/{first.Body.Text("id")}")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Post("/api/v1/statuses", ("status", "anonymous"))).Status);
}
[Fact]
public async Task Local_polls_take_one_vote_per_persona_and_never_from_their_author()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var poll = await alice.Status("pick one", ("poll[options][]", "red"), ("poll[options][]", "blue"), ("poll[expires_in]", "3600"));
var path = $"/api/v1/polls/{poll.Text("id")}";
var voted = (await carol.Client.Post(path + "/votes", ("choices[]", "1"))).Ok();
Assert.True(voted.Body.Flag("voted"));
Assert.Equal(new[] { 1 }, voted.Body["own_votes"]!.AsArray().Select(v => v!.GetValue<int>()));
Assert.Equal(1, voted.Body.Number("votes_count"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await carol.Client.Post(path + "/votes", ("choices[]", "0"))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Post(path + "/votes", ("choices[]", "0"))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await (await _host.Mastodon("dave")).Client.Post(path + "/votes", ("choices[]", "0"), ("choices[]", "1"))).Status);
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await (await _host.Mastodon("erin")).Client.Post(path + "/votes", ("choices[]", "7"))).Status);
var anonymous = (await _host.Client().Get(path)).Ok();
Assert.Null(anonymous.Body["voted"]);
Assert.Equal(new[] { 0, 1 }, anonymous.Body["options"]!.AsArray().Select(o => o.Number("votes_count")));
}
[Fact]
public async Task A_vote_on_a_remote_poll_goes_only_to_its_author_without_a_published_date()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var carol = new RemoteActor(_peer, "carol");
await _host.FollowedBy(alice, carol);
var question = await _host.PublicPostFrom(bob, alice, "<p>which?</p>", note =>
{
note["type"] = "Question";
note["oneOf"] = new JsonArray(
new JsonObject { ["type"] = "Note", ["name"] = "this", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = 0 } },
new JsonObject { ["type"] = "Note", ["name"] = "that", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = 0 } });
note["endTime"] = DateTime.UtcNow.AddDays(1).ToString("O");
});
var voted = (await alice.Client.Post($"/api/v1/polls/{question.ID}/votes", ("choices[]", "1"))).Ok();
Assert.True(voted.Body.Flag("voted"));
var vote = Assert.Single(await bob.Delivered(since), d => d.Type() == "Create");
Assert.Equal("that", vote["object"].Text("name"));
Assert.Equal(question.ObjectURI, vote["object"].Text("inReplyTo"));
Assert.Null(vote["object"]!["published"]);
Assert.Equal(new[] { bob.Id }, Strings(vote["object"]!["to"]));
Assert.DoesNotContain(await carol.Delivered(since), d => d.Type() == "Create");
}
[Fact]
public async Task Deleting_a_reblog_undoes_it()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var remote = await _host.PublicPostFrom(bob, alice);
var reblog = (await alice.Client.Post($"/api/v1/statuses/{remote.ID}/reblog")).Ok().Body;
(await alice.Client.Delete($"/api/v1/statuses/{reblog.Text("id")}")).Ok();
Assert.False((await alice.Client.Get($"/api/v1/statuses/{remote.ID}")).Ok().Body.Flag("reblogged"));
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/statuses/{reblog.Text("id")}")).Status);
var undo = Assert.Single(await bob.Delivered(since), d => d.Type() == "Undo");
Assert.Equal("Announce", undo["object"].Text("type"));
}
[Fact]
public async Task Many_statuses_are_fetched_by_id_in_one_call()
{
var alice = await _host.Mastodon("alice");
var first = await alice.Status("one");
var second = await alice.Status("two");
var answer = (await _host.Client().Get($"/api/v1/statuses?id[]={first.Text("id")}&id[]={second.Text("id")}&id[]=junk")).Ok();
Assert.Equal(new[] { first.Text("id"), second.Text("id") }, answer.Ids.Order());
Assert.Empty((await _host.Client().Get("/api/v1/statuses")).Ok().Array);
}
}
}
@@ -0,0 +1,165 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// Mastodon's streaming API: a WebSocket whose streams are subscribed by message or in the URL, and server-sent events
// per stream; each event mapped for its persona, never what it may not see
[Trait("Category", "Integration")]
public sealed class MastodonStreamingTests : IAsyncLifetime
{
PrivaPubHost _host;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
async Task<WebSocket> Connect(Mastodon account, string query = "")
{
var client = _host.Server.CreateWebSocketClient();
var uri = new Uri($"ws://{PrivaPubHost.Host}/api/v1/streaming?access_token={account.Token}{query}");
var socket = await client.ConnectAsync(uri, Token);
// the subscription is made as the socket opens: give the server a moment to hear it
await Task.Delay(200, Token);
return socket;
}
static async Task Say(WebSocket socket, JsonObject message) =>
await socket.SendAsync(Encoding.UTF8.GetBytes(message.ToJsonString()), WebSocketMessageType.Text, true, Token);
// the next message for which `wanted` holds, within a few seconds, or null
static async Task<JsonObject> Next(WebSocket socket, Func<JsonObject, bool> wanted, int seconds = 5)
{
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(Token);
timeout.CancelAfter(TimeSpan.FromSeconds(seconds));
var buffer = new byte[65536];
try
{
while (true)
{
var text = new StringBuilder();
WebSocketReceiveResult received;
do
{
received = await socket.ReceiveAsync(buffer, timeout.Token);
text.Append(Encoding.UTF8.GetString(buffer, 0, received.Count));
}
while (!received.EndOfMessage);
if (JsonNode.Parse(text.ToString()) is JsonObject message && wanted(message))
return message;
}
}
catch (OperationCanceledException)
{
return null;
}
}
static string Id(JsonObject message) => JsonNode.Parse(message["payload"]!.GetValue<string>())!["id"]!.GetValue<string>();
[Fact]
public async Task Health_is_answered_and_a_stream_needs_a_persona()
{
var health = await _host.Client().GetAsync("/api/v1/streaming/health", Token);
Assert.Equal("OK", await health.Content.ReadAsStringAsync(Token));
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().GetAsync("/api/v1/streaming/public", Token)).StatusCode);
var instance = (await _host.Client().Get("/api/v2/instance")).Ok().Body;
Assert.StartsWith("ws", instance["configuration"]!["urls"]!["streaming"]!.GetValue<string>());
}
[Fact]
public async Task The_user_stream_tells_home_posts_notifications_edits_and_deletions()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var carol = await _host.Mastodon("carol");
var stranger = await _host.Mastodon("stranger");
(await alice.Client.Post($"/api/v1/accounts/{bob.Id}/follow")).Ok();
using var socket = await Connect(alice, "&stream=user");
using var elsewhere = await Connect(stranger, "&stream=user");
var post = (await bob.Status("streamed to alice")).Text("id");
var update = await Next(socket, m => m.Text("event") == "update");
Assert.NotNull(update);
Assert.Equal(new[] { "user" }, update!["stream"]!.AsArray().Select(s => s!.GetValue<string>()));
Assert.Equal(post, Id(update));
await carol.Status($"@{alice.UserName} hello there", ("visibility", "direct"));
var notification = await Next(socket, m => m.Text("event") == "notification");
Assert.Equal("mention", JsonNode.Parse(notification!["payload"]!.GetValue<string>())!.Text("type"));
(await bob.Client.Put($"/api/v1/statuses/{post}", ("status", "streamed, then edited"))).Ok();
var edited = await Next(socket, m => m.Text("event") == "status.update");
Assert.Contains("then edited", JsonNode.Parse(edited!["payload"]!.GetValue<string>())!.Text("content"));
(await bob.Client.Delete($"/api/v1/statuses/{post}")).Ok();
Assert.NotNull(await Next(socket, m => m.Text("event") == "delete" && m["payload"]!.GetValue<string>() == post));
// a persona whose home never held it hears nothing of it, not even its id
Assert.Null(await Next(elsewhere, m => m.Text("event") is "update" or "delete" && m["payload"]!.GetValue<string>().Contains(post), seconds: 2));
}
[Fact]
public async Task A_hashtag_stream_subscribed_by_message_tells_public_posts_only_and_never_a_blocked_authors()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var dave = await _host.Mastodon("dave");
(await alice.Client.Post($"/api/v1/accounts/{dave.Id}/block")).Ok();
var tag = $"st{Guid.NewGuid():N}"[..12];
using var socket = await Connect(alice);
await Say(socket, new JsonObject { ["type"] = "subscribe", ["stream"] = "hashtag", ["tag"] = tag });
await Task.Delay(200, Token);
await dave.Status($"blocked #{tag}");
await bob.Status($"quiet #{tag}", ("visibility", "unlisted"));
var open = (await bob.Status($"open #{tag}")).Text("id");
var told = await Next(socket, m => m.Text("event") == "update");
Assert.Equal(open, Id(told!));
Assert.Equal(new[] { "hashtag", tag }, told["stream"]!.AsArray().Select(s => s!.GetValue<string>()));
Assert.Null(await Next(socket, m => m.Text("event") == "update", seconds: 2));
await Say(socket, new JsonObject { ["type"] = "unsubscribe", ["stream"] = "hashtag", ["tag"] = tag });
await Task.Delay(200, Token);
await bob.Status($"again #{tag}");
Assert.Null(await Next(socket, m => m.Text("event") == "update", seconds: 2));
await Say(socket, new JsonObject { ["type"] = "subscribe", ["stream"] = "list", ["list"] = "000000000000000000000000" });
Assert.Equal("Unknown list", (await Next(socket, m => m["error"] != null))!.Text("error"));
}
[Fact]
public async Task Server_sent_events_tell_a_personas_notifications()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
using var request = new HttpRequestMessage(HttpMethod.Get, "/api/v1/streaming/user/notification");
request.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", alice.Token);
using var response = await _host.Client().SendAsync(request, HttpCompletionOption.ResponseHeadersRead, Token);
Assert.Equal("text/event-stream", response.Content.Headers.ContentType!.MediaType);
using var reader = new StreamReader(await response.Content.ReadAsStreamAsync(Token));
Assert.Equal(":)", await reader.ReadLineAsync(Token));
(await bob.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(Token);
timeout.CancelAfter(TimeSpan.FromSeconds(5));
string line;
while ((line = await reader.ReadLineAsync(timeout.Token)) != null && line != "event: notification")
{
}
Assert.Equal("event: notification", line);
var data = await reader.ReadLineAsync(timeout.Token);
Assert.Equal("follow", JsonNode.Parse(data!["data: ".Length..])!.Text("type"));
}
}
}
+89
View File
@@ -0,0 +1,89 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// Hashtags: a tag's week of use in public posts, and followed tags, whose public posts come to the persona's home
[Trait("Category", "Integration")]
public sealed class MastodonTagsTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static string NewTag() => $"t{Guid.NewGuid():N}"[..12];
[Fact]
public async Task A_tag_shows_its_week_of_public_use_and_whether_the_persona_follows_it()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var tag = NewTag();
await alice.Status($"about #{tag}");
await alice.Status($"more about #{tag.ToUpperInvariant()}");
await bob.Status($"also #{tag}");
await bob.Status($"quietly #{tag}", ("visibility", "unlisted"));
var shown = (await _host.Client().Get($"/api/v1/tags/{tag}")).Ok().Object;
Assert.Equal(tag, shown.Text("name"));
Assert.False(shown.Flag("following"));
var history = shown["history"]!.AsArray();
Assert.Equal(7, history.Count);
Assert.Equal(("3", "2"), (history[0].Text("uses"), history[0].Text("accounts")));
Assert.Equal(new DateTimeOffset(DateTime.UtcNow.Date, TimeSpan.Zero).ToUnixTimeSeconds().ToString(), history[0].Text("day"));
Assert.True((await alice.Client.Post($"/api/v1/tags/{tag}/follow")).Ok().Body.Flag("following"));
Assert.True((await alice.Client.Get($"/api/v1/tags/%23{tag.ToUpperInvariant()}")).Ok().Body.Flag("following"));
Assert.Equal(new[] { tag }, (await alice.Client.Get("/api/v1/followed_tags")).Ok().Array.Select(t => t.Text("name")));
Assert.False((await bob.Client.Get($"/api/v1/tags/{tag}")).Ok().Body.Flag("following"));
Assert.False((await alice.Client.Post($"/api/v1/tags/{tag}/unfollow")).Ok().Body.Flag("following"));
Assert.Empty((await alice.Client.Get("/api/v1/followed_tags")).Ok().Array);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Post($"/api/v1/tags/{tag}/follow")).Status);
}
[Fact]
public async Task A_followed_tags_public_posts_come_home_from_here_and_from_other_servers()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var carol = await _host.Mastodon("carol");
var remote = new RemoteActor(_peer, "dora");
var tag = NewTag();
(await alice.Client.Post($"/api/v1/tags/{tag}/follow")).Ok();
var open = (await bob.Status($"news about #{tag}")).Text("id");
var quiet = (await bob.Status($"quiet news #{tag}", ("visibility", "unlisted"))).Text("id");
var answer = (await bob.Status($"and #{tag} again", ("in_reply_to_id", open))).Text("id");
var untagged = (await bob.Status("nothing tagged")).Text("id");
var far = await _host.PublicPostFrom(remote, carol, $"<p>far away #{tag}</p>", note => note["tag"]!.AsArray().Add(new JsonObject
{
["type"] = "Hashtag",
["name"] = "#" + tag,
["href"] = $"{remote.Origin()}/tags/{tag}"
}));
var home = (await alice.Client.Get("/api/v1/timelines/home?limit=40")).Ok().Ids.ToList();
Assert.Contains(open, home);
Assert.Contains(far.ID, home);
Assert.DoesNotContain(quiet, home);
Assert.DoesNotContain(answer, home);
Assert.DoesNotContain(untagged, home);
Assert.DoesNotContain(open, (await carol.Client.Get("/api/v1/timelines/home?limit=40")).Ok().Ids);
}
}
}
@@ -0,0 +1,265 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class MastodonTimelinesTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
[Fact]
public async Task The_home_timeline_pages_with_max_since_and_min_id()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var posts = new List<string>();
for (var i = 1; i <= 4; i++)
posts.Add((await alice.Status($"post {i}")).Text("id"));
(await bob.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
var afterFollow = (await alice.Status("post 5")).Text("id");
var (p1, p2, p3, p4) = (posts[0], posts[1], posts[2], posts[3]);
var first = (await alice.Client.Get("/api/v1/timelines/home?limit=2")).Ok();
Assert.Equal(new[] { afterFollow, p4 }, first.Ids);
Assert.Equal(new[] { p3, p2 }, (await alice.Client.Get(first.Link("next"))).Ok().Ids);
Assert.Empty((await alice.Client.Get(first.Link("prev"))).Ok().Array);
Assert.Equal(new[] { p2, p1 }, (await alice.Client.Get($"/api/v1/timelines/home?max_id={p3}&limit=2")).Ok().Ids);
Assert.Equal(new[] { afterFollow, p4 }, (await alice.Client.Get($"/api/v1/timelines/home?since_id={p1}&limit=2")).Ok().Ids);
Assert.Equal(new[] { p3, p2 }, (await alice.Client.Get($"/api/v1/timelines/home?min_id={p1}&limit=2")).Ok().Ids);
Assert.Equal(new[] { p3 }, (await alice.Client.Get($"/api/v1/timelines/home?min_id={p2}&max_id={p4}")).Ok().Ids);
Assert.Equal(new[] { afterFollow }, (await bob.Client.Get("/api/v1/timelines/home")).Ok().Ids);
Assert.Equal(HttpStatusCode.Unauthorized, (await _host.Client().Get("/api/v1/timelines/home")).Status);
}
static async Task<List<string>> Window(HttpClient client, string filter, string since, string until)
{
var ids = new List<string>();
for (var page = 0; page < 50; page++)
{
var batch = (await client.Get($"/api/v1/timelines/public?{filter}&since_id={since}&max_id={until}&limit=40")).Ok().Ids.ToList();
if (batch.Count == 0)
break;
ids.AddRange(batch);
until = batch[^1];
}
return ids;
}
[Fact]
public async Task The_public_timeline_splits_local_from_remote_and_leaves_out_unlisted_posts()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var since = MastodonHelpers.IdBefore(DateTime.UtcNow);
var remote = await _host.PublicPostFrom(bob, alice);
var open = (await alice.Status("hello world")).Text("id");
var unlisted = (await alice.Status("hello quietly", ("visibility", "unlisted"))).Text("id");
var until = MastodonHelpers.IdBefore(DateTime.UtcNow.AddSeconds(1));
var anonymous = _host.Client();
var everything = await Window(anonymous, "local=false", since, until);
Assert.Contains(open, everything);
Assert.Contains(remote.ID, everything);
Assert.DoesNotContain(unlisted, everything);
var local = await Window(anonymous, "local=true", since, until);
Assert.Contains(open, local);
Assert.DoesNotContain(remote.ID, local);
var federated = await Window(alice.Client, "remote=true", since, until);
Assert.Contains(remote.ID, federated);
Assert.DoesNotContain(open, federated);
}
[Fact]
public async Task The_tag_timeline_is_open_to_anonymous_readers_and_shows_only_public_posts()
{
var alice = await _host.Mastodon("alice");
var tag = $"t{Guid.NewGuid():N}"[..16];
var open = (await alice.Status($"about #{tag}")).Text("id");
await alice.Status($"quietly about #{tag}", ("visibility", "unlisted"));
await alice.Status($"privately about #{tag}", ("visibility", "private"));
var later = (await alice.Status($"more about #{tag.ToUpperInvariant()}")).Text("id");
var tagged = (await _host.Client().Get($"/api/v1/timelines/tag/{tag}")).Ok();
Assert.Equal(new[] { later, open }, tagged.Ids);
Assert.Equal(new[] { later }, (await _host.Client().Get($"/api/v1/timelines/tag/{tag}?local=true&limit=1")).Ok().Ids);
Assert.Equal(new[] { later, open }, (await alice.Client.Get($"/api/v1/timelines/tag/%23{tag}")).Ok().Ids);
}
[Fact]
public async Task Favourites_list_what_the_persona_liked_newest_first()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var first = (await carol.Status("one")).Text("id");
var second = (await carol.Status("two")).Text("id");
(await alice.Client.Post($"/api/v1/statuses/{first}/favourite")).Ok();
(await alice.Client.Post($"/api/v1/statuses/{second}/favourite")).Ok();
Assert.Equal(new[] { second, first }, (await alice.Client.Get("/api/v1/favourites")).Ok().Ids);
var page = (await alice.Client.Get("/api/v1/favourites?limit=1")).Ok();
Assert.Equal(new[] { second }, page.Ids);
Assert.Equal(new[] { first }, (await alice.Client.Get(page.Link("next"))).Ok().Ids);
Assert.Empty((await carol.Client.Get("/api/v1/favourites")).Ok().Array);
}
[Fact]
public async Task Conversations_list_direct_messages_to_their_participants_only()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var stranger = await _host.Mastodon("stranger");
(await alice.Status($"@{bob.UserName} psst", ("visibility", "direct"))).Text("id");
var answer = (await bob.Status($"@{alice.UserName} what?", ("visibility", "direct"))).Text("id");
var mine = Assert.Single((await alice.Client.Get("/api/v1/conversations")).Ok().Array);
Assert.Equal(new[] { bob.Id }, mine!["accounts"]!.AsArray().Select(a => a.Text("id")));
Assert.Equal(answer, mine["last_status"].Text("id"));
var theirs = Assert.Single((await bob.Client.Get("/api/v1/conversations")).Ok().Array);
Assert.Equal(mine.Text("id"), theirs.Text("id"));
Assert.Equal(new[] { alice.Id }, theirs!["accounts"]!.AsArray().Select(a => a.Text("id")));
Assert.Empty((await stranger.Client.Get("/api/v1/conversations")).Ok().Array);
var read = (await bob.Client.Post($"/api/v1/conversations/{mine.Text("id")}/read")).Ok();
Assert.Equal(mine.Text("id"), read.Body.Text("id"));
Assert.Equal(answer, read.Body["last_status"].Text("id"));
Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Post($"/api/v1/conversations/{mine.Text("id")}/read")).Status);
Assert.DoesNotContain(answer, (await stranger.Client.Get($"/api/v1/accounts/{bob.Id}/statuses")).Ok().Ids);
}
[Fact]
public async Task Conversations_page_by_their_newest_message_and_know_what_is_unread_read_and_removed()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var carol = await _host.Mastodon("carol");
await bob.Status($"@{alice.UserName} first", ("visibility", "direct"));
var fromCarol = (await carol.Status($"@{alice.UserName} second", ("visibility", "direct"))).Text("id");
// writing in a conversation reads it
await alice.Status($"@{bob.UserName} third", ("visibility", "direct"));
async Task<JsonArray> List() => (await alice.Client.Get("/api/v1/conversations")).Ok().Array;
var all = await List();
Assert.Equal(2, all.Count);
var (withBob, withCarol) = (all[0].Text("id"), all[1].Text("id"));
Assert.Equal(new[] { bob.Id }, all[0]!["accounts"]!.AsArray().Select(a => a.Text("id")));
Assert.False(all[0].Flag("unread"));
Assert.True(all[1].Flag("unread"));
Assert.Equal(fromCarol, all[1]!["last_status"].Text("id"));
var page = (await alice.Client.Get("/api/v1/conversations?limit=1")).Ok();
Assert.Equal(withBob, Assert.Single(page.Array).Text("id"));
Assert.Equal(withCarol, Assert.Single((await alice.Client.Get(page.Link("next"))).Ok().Array).Text("id"));
Assert.False((await alice.Client.Post($"/api/v1/conversations/{withCarol}/read")).Ok().Body.Flag("unread"));
Assert.False((await List()).Single(c => c.Text("id") == withCarol).Flag("unread"));
// off the list until a newer message, which brings it back unread; carol's own list never changed
(await alice.Client.Delete($"/api/v1/conversations/{withCarol}")).Ok();
Assert.Equal(new[] { withBob }, (await List()).Select(c => c.Text("id")));
Assert.Equal(withCarol, Assert.Single((await carol.Client.Get("/api/v1/conversations")).Ok().Array).Text("id"));
var again = (await carol.Status($"@{alice.UserName} are you there?", ("visibility", "direct"))).Text("id");
var back = (await List())[0];
Assert.Equal((withCarol, again, true), (back.Text("id"), back!["last_status"].Text("id"), back.Flag("unread")));
Assert.Equal(HttpStatusCode.NotFound, (await bob.Client.Delete($"/api/v1/conversations/{withCarol}")).Status);
}
[Fact]
public async Task Markers_are_saved_versioned_and_read_back_per_timeline()
{
var alice = await _host.Mastodon("alice");
var saved = (await alice.Client.Post("/api/v1/markers", ("home[last_read_id]", "100"))).Ok();
Assert.Equal("100", saved.Body["home"].Text("last_read_id"));
Assert.Equal(1, saved.Body["home"].Number("version"));
Assert.Null(saved.Body["notifications"]);
var again = (await alice.Client.Json(HttpMethod.Post, "/api/v1/markers", new JsonObject
{
["home"] = new JsonObject { ["last_read_id"] = "200" },
["notifications"] = new JsonObject { ["last_read_id"] = "7" }
})).Ok();
Assert.Equal(("200", 2), (again.Body["home"].Text("last_read_id"), again.Body["home"].Number("version")));
Assert.Equal(("7", 1), (again.Body["notifications"].Text("last_read_id"), again.Body["notifications"].Number("version")));
var home = (await alice.Client.Get("/api/v1/markers?timeline[]=home")).Ok().Object;
Assert.Equal(new[] { "home" }, home.Select(p => p.Key));
Assert.Equal("200", home["home"].Text("last_read_id"));
Assert.Equal(new[] { "home", "notifications" }, (await alice.Client.Get("/api/v1/markers")).Ok().Object.Select(p => p.Key).Order());
Assert.Empty((await (await _host.Mastodon("other")).Client.Get("/api/v1/markers")).Ok().Object);
}
[Fact]
public async Task A_notification_about_a_deleted_post_is_gone_not_null()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var post = (await alice.Status("short lived")).Text("id");
(await carol.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
var id = Assert.Single((await alice.Client.Get("/api/v1/notifications")).Ok().Array).Text("id");
(await alice.Client.Delete($"/api/v1/statuses/{post}")).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/notifications")).Ok().Array);
var gone = await alice.Client.Get($"/api/v1/notifications/{id}");
Assert.Equal(HttpStatusCode.NotFound, gone.Status);
Assert.Equal("Record not found", gone.Body.Text("error"));
}
[Fact]
public async Task Notifications_filter_page_dismiss_and_clear()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var post = (await alice.Status("notice me")).Text("id");
(await carol.Client.Post($"/api/v1/accounts/{alice.Id}/follow")).Ok();
(await carol.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
var mention = (await carol.Status($"hey @{alice.UserName}")).Text("id");
(await carol.Client.Post($"/api/v1/statuses/{post}/reblog")).Ok();
var all = (await alice.Client.Get("/api/v1/notifications")).Ok();
Assert.Equal(new[] { "reblog", "mention", "favourite", "follow" }, all.Array.Select(n => n.Text("type")));
Assert.All(all.Array, n => Assert.Equal(carol.Id, n!["account"].Text("id")));
Assert.Equal(mention, all.Array[1]!["status"].Text("id"));
Assert.Equal(new[] { "mention" }, (await alice.Client.Get("/api/v1/notifications?types[]=mention")).Ok().Array.Select(n => n.Text("type")));
Assert.Equal(new[] { "mention", "favourite" },
(await alice.Client.Get("/api/v1/notifications?exclude_types[]=follow&exclude_types[]=reblog")).Ok().Array.Select(n => n.Text("type")));
Assert.Equal(4, (await alice.Client.Get($"/api/v1/notifications?account_id={carol.Id}")).Ok().Array.Count);
var page = (await alice.Client.Get("/api/v1/notifications?limit=3")).Ok();
Assert.Equal(new[] { "follow" }, (await alice.Client.Get(page.Link("next"))).Ok().Array.Select(n => n.Text("type")));
Assert.Equal(4, (await alice.Client.Get("/api/v1/notifications/unread_count")).Ok().Body.Number("count"));
var followId = all.Array[3].Text("id");
Assert.Equal("follow", (await alice.Client.Get($"/api/v1/notifications/{followId}")).Ok().Body.Text("type"));
Assert.Equal(HttpStatusCode.NotFound, (await carol.Client.Get($"/api/v1/notifications/{followId}")).Status);
(await carol.Client.Post($"/api/v1/notifications/{followId}/dismiss")).Ok();
Assert.Equal(HttpStatusCode.OK, (await alice.Client.Get($"/api/v1/notifications/{followId}")).Status);
(await alice.Client.Post($"/api/v1/notifications/{followId}/dismiss")).Ok();
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/notifications/{followId}")).Status);
Assert.Equal(3, (await alice.Client.Get("/api/v1/notifications")).Ok().Array.Count);
(await alice.Client.Post("/api/v1/notifications/clear")).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/notifications")).Ok().Array);
Assert.Equal(0, (await alice.Client.Get("/api/v1/notifications/unread_count")).Ok().Body.Number("count"));
}
}
}
+133
View File
@@ -0,0 +1,133 @@
using MongoDB.Entities;
using PrivaPub.Domain.Social;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
namespace PrivaPub.Tests.Http
{
// Trends from what PrivaPub holds (hashtags and links by how many authors use them, posts by their interactions,
// from discoverable authors only) and the directory of accounts that let themselves be discovered
[Trait("Category", "Integration")]
public sealed class MastodonTrendsTests : IAsyncLifetime
{
PrivaPubHost _host;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
void Fresh() => _host.Get<Trends>().Forget();
static string NewTag() => $"tr{Guid.NewGuid():N}"[..12];
[Fact]
public async Task A_hashtag_trends_once_two_authors_use_it_today_and_never_from_one_or_behind_a_warning()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var carol = await _host.Mastodon("carol");
var shared = NewTag();
var lonely = NewTag();
var warned = NewTag();
await alice.Status($"#{shared} is good");
await bob.Status($"#{shared} indeed");
for (var i = 0; i < 4; i++)
await carol.Status($"#{lonely} again {i}");
await alice.Status($"#{warned}", ("spoiler_text", "careful"));
await bob.Status($"#{warned}", ("spoiler_text", "careful"));
Fresh();
var trends = (await _host.Client().Get("/api/v1/trends/tags?limit=20")).Ok().Array;
var trend = Assert.Single(trends, t => t.Text("name") == shared)!;
Assert.Equal(("2", "2"), (trend["history"]![0].Text("uses"), trend["history"]![0].Text("accounts")));
Assert.DoesNotContain(trends, t => t.Text("name") == lonely);
Assert.DoesNotContain(trends, t => t.Text("name") == warned);
Assert.Contains((await alice.Client.Get("/api/v1/trends?limit=20")).Ok().Array, t => t.Text("name") == shared);
}
[Fact]
public async Task A_post_trends_by_its_interactions_when_its_author_is_discoverable()
{
var alice = await _host.Mastodon("alice");
var hidden = await _host.Mastodon("hidden");
var bob = await _host.Mastodon("bob");
(await hidden.Client.Patch("/api/v1/accounts/update_credentials", ("discoverable", "false"))).Ok();
var liked = (await alice.Status("worth a like")).Text("id");
var quiet = (await hidden.Status("worth a like too")).Text("id");
var answer = (await alice.Status("a reply", ("in_reply_to_id", liked))).Text("id");
foreach (var id in new[] { liked, quiet, answer })
{
(await bob.Client.Post($"/api/v1/statuses/{id}/favourite")).Ok();
(await bob.Client.Post($"/api/v1/statuses/{id}/reblog")).Ok();
}
Fresh();
var trending = (await _host.Client().Get("/api/v1/trends/statuses?limit=40")).Ok().Ids.ToList();
Assert.Contains(liked, trending);
Assert.DoesNotContain(quiet, trending);
Assert.DoesNotContain(answer, trending);
}
[Fact]
public async Task A_link_trends_once_two_authors_share_it()
{
var alice = await _host.Mastodon("alice");
var bob = await _host.Mastodon("bob");
var url = $"https://news.example/{Guid.NewGuid():N}";
foreach (var account in new[] { alice, bob })
{
var id = (await account.Status($"read {url}")).Text("id");
await DB.Default.Update<Post>().MatchID(id).Modify(p => p.Link, new PostLink { Href = url, Title = "The news" }).ExecuteAsync(Token);
}
Fresh();
var link = Assert.Single((await _host.Client().Get("/api/v1/trends/links?limit=20")).Ok().Array, l => l.Text("url") == url)!;
Assert.Equal("The news", link.Text("title"));
Assert.Equal("2", link["history"]![0].Text("accounts"));
}
[Fact]
public async Task The_directory_lists_discoverable_accounts_by_activity_or_by_joining_and_never_by_creation()
{
var alice = await _host.Mastodon("alice");
var hidden = await _host.Mastodon("hidden");
(await hidden.Client.Patch("/api/v1/accounts/update_credentials", ("discoverable", "false"))).Ok();
await hidden.Status("not listed");
await alice.Status("listed");
// by their latest public post, newest first: other tests post too, so the directory is read until alice
var active = new List<string>();
for (var offset = 0; offset < 4000 && !active.Contains(alice.Id); offset += 80)
{
var page = (await _host.Client().Get($"/api/v1/directory?order=active&local=true&limit=80&offset={offset}")).Ok().Ids.ToList();
active.AddRange(page);
if (page.Count < 80)
break;
}
Assert.Contains(alice.Id, active);
Assert.DoesNotContain(hidden.Id, active);
// by the day each says it joined, newest first: a persona's is its published day, set some days back
// (each page in order: other tests make personas between two pages)
var joined = new List<System.Text.Json.Nodes.JsonNode>();
for (var offset = 0; offset < 4000; offset += 80)
{
var page = (await _host.Client().Get($"/api/v1/directory?order=new&local=true&limit=80&offset={offset}")).Ok().Array;
var days = page.Select(a => a.Text("created_at")[..10]).ToList();
Assert.Equal(days.OrderByDescending(d => d, StringComparer.Ordinal), days);
joined.AddRange(page!);
if (page.Count < 80 || joined.Any(a => a.Text("id") == alice.Id))
break;
}
Assert.Contains(joined, a => a.Text("id") == alice.Id);
Assert.DoesNotContain(joined, a => a.Text("id") == hidden.Id);
}
}
}
@@ -0,0 +1,68 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
namespace PrivaPub.Tests.Http
{
// Status search by words, as Mastodon scopes it: the persona's own posts and those it boosted, favourited, bookmarked
// or was named in, and public posts of authors who let themselves be indexed
[Trait("Category", "Integration")]
public sealed class MastodonWordSearchTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
[Fact]
public async Task Words_find_what_the_persona_wrote_or_touched_and_what_indexable_authors_made_public()
{
var reader = await _host.Mastodon("reader");
var open = await _host.Mastodon("open");
var shy = await _host.Mastodon("shy");
(await open.Client.Patch("/api/v1/accounts/update_credentials", ("indexable", "true"))).Ok();
var word = $"w{Guid.NewGuid():N}"[..12];
var own = (await reader.Status($"my {word}")).Text("id");
var indexed = (await open.Status($"open {word}")).Text("id");
var quietIndexed = (await open.Status($"followers {word}", ("visibility", "private"))).Text("id");
var unindexed = (await shy.Status($"shy {word}")).Text("id");
var liked = (await shy.Status($"liked {word}")).Text("id");
var named = (await shy.Status($"@{reader.UserName} about {word}")).Text("id");
var warned = (await shy.Status("calm", ("spoiler_text", $"about {word}"))).Text("id");
(await reader.Client.Post($"/api/v1/statuses/{liked}/favourite")).Ok();
(await reader.Client.Post($"/api/v1/statuses/{warned}/bookmark")).Ok();
var found = (await reader.Client.Get($"/api/v2/search?q={word}&type=statuses&limit=40")).Ok().Body["statuses"]!.AsArray()
.Select(s => s.Text("id")).ToList();
Assert.Equal(new[] { warned, named, liked, indexed, own }, found);
Assert.DoesNotContain(unindexed, found);
Assert.DoesNotContain(quietIndexed, found);
Assert.Empty((await _host.Client().Get($"/api/v2/search?q={word}&type=statuses")).Ok().Body["statuses"]!.AsArray());
Assert.Equal(new[] { named },
(await reader.Client.Get($"/api/v2/search?q={word}&type=statuses&limit=1&offset=1")).Ok().Body["statuses"]!.AsArray().Select(s => s.Text("id")));
}
// a handle asks for its account: no posts sharing its words, and no hashtag made of it
[Fact]
public async Task A_handle_finds_its_account_and_neither_posts_nor_a_hashtag()
{
var reader = await _host.Mastodon("reader");
var named = await _host.Mastodon("named");
await reader.Status($"talking of {named.UserName} here");
foreach (var q in new[] { $"@{named.UserName}", $"@{named.UserName}@{PrivaPubHost.Host}", $"{named.UserName}@{PrivaPubHost.Host}" })
{
var body = (await reader.Client.Get($"/api/v2/search?q={Uri.EscapeDataString(q)}")).Ok().Body;
Assert.Contains(body["accounts"]!.AsArray(), a => a.Text("id") == named.Id);
Assert.Empty(body["statuses"]!.AsArray());
Assert.Empty(body["hashtags"]!.AsArray());
}
Assert.Equal("cats", Assert.Single((await reader.Client.Get("/api/v2/search?q=%23Cats")).Ok().Body["hashtags"]!.AsArray()).Text("name"));
}
}
}
+234
View File
@@ -0,0 +1,234 @@
using MongoDB.Entities;
using PrivaPub.Domain.Media;
using PrivaPub.Models.Media;
using PrivaPub.Services;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
// A file lives exactly as long as something holds it: a deleted post, an edit leaving media out, a replaced picture and
// a removed root each trash theirs, which stops /media/files serving them at once; the janitor deletes them later.
[Trait("Category", "Integration")]
public sealed class MediaLifecycleTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static CancellationToken Token => TestContext.Current.CancellationToken;
async Task<string> Upload(Mastodon account, string name)
{
var form = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(64, 48), "image/jpeg", name));
return (await account.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media") { Content = form })).Ok().Body.Text("id");
}
async Task<HttpStatusCode> Served(string url) =>
(await _host.Client().GetAsync(new Uri(url).PathAndQuery, Token)).StatusCode;
[Fact]
public async Task A_deleted_posts_media_stop_being_served()
{
var alice = await _host.Mastodon("alice");
var id = await Upload(alice, "a.jpg");
var status = (await alice.Client.Post("/api/v1/statuses", ("status", "with a picture"), ("media_ids[]", id))).Ok();
var url = status.Body["media_attachments"]![0]!["url"]!.GetValue<string>();
var preview = status.Body["media_attachments"]![0]!["preview_url"]!.GetValue<string>();
Assert.Equal(HttpStatusCode.OK, await Served(url));
(await alice.Client.Delete($"/api/v1/statuses/{status.Body.Text("id")}")).Ok();
Assert.Equal(HttpStatusCode.NotFound, await Served(url));
Assert.Equal(HttpStatusCode.NotFound, await Served(preview));
Assert.Equal("post deleted", (await DB.Default.Find<MediaAttachment>().OneAsync(id, Token)).TrashReason);
// the file waits in the trash, beside what /media/files serves
var relative = url[(url.IndexOf("/media/files/", StringComparison.Ordinal) + "/media/files/".Length)..];
Assert.True(File.Exists(Path.Combine(_host.Get<IMediaService>().TrashRoot, relative)));
Assert.Equal(HttpStatusCode.NotFound, await Served(url.Replace("/media/files/", "/media/files/.trash/")));
}
[Fact]
public async Task An_edit_that_leaves_media_out_trashes_them()
{
var alice = await _host.Mastodon("alice");
var kept = await Upload(alice, "kept.jpg");
var dropped = await Upload(alice, "dropped.jpg");
var status = (await alice.Client.Post("/api/v1/statuses", ("status", "two pictures"), ("media_ids[]", kept), ("media_ids[]", dropped))).Ok();
var urls = status.Body["media_attachments"]!.AsArray().ToDictionary(m => m!["id"]!.GetValue<string>(), m => m!["url"]!.GetValue<string>());
(await alice.Client.Put($"/api/v1/statuses/{status.Body.Text("id")}", ("status", "one picture"), ("media_ids[]", kept))).Ok();
Assert.Equal(HttpStatusCode.OK, await Served(urls[kept]));
Assert.Equal(HttpStatusCode.NotFound, await Served(urls[dropped]));
Assert.Null((await DB.Default.Find<MediaAttachment>().OneAsync(kept, Token)).TrashedAt);
Assert.Equal("edited out", (await DB.Default.Find<MediaAttachment>().OneAsync(dropped, Token)).TrashReason);
}
// a posted picture's description and focus change by editing its post (media_attributes), not by PUT, as on Mastodon
[Fact]
public async Task An_edit_describes_its_media_again_and_keeps_what_it_is_not_sent()
{
var alice = await _host.Mastodon("alice");
var id = await Upload(alice, "a.jpg");
var status = (await alice.Client.Post("/api/v1/statuses", ("status", "described later"), ("media_ids[]", id),
("spoiler_text", "a warning"), ("sensitive", "true"), ("language", "it"))).Ok();
var statusId = status.Body.Text("id");
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Put($"/api/v1/media/{id}", ("description", "too late"))).Status);
var edited = (await alice.Client.Json(HttpMethod.Put, $"/api/v1/statuses/{statusId}", new System.Text.Json.Nodes.JsonObject
{
["status"] = "described now",
["spoiler_text"] = "a warning",
["sensitive"] = true,
["media_ids"] = new System.Text.Json.Nodes.JsonArray(id),
["media_attributes"] = new System.Text.Json.Nodes.JsonArray(new System.Text.Json.Nodes.JsonObject { ["id"] = id, ["description"] = "a red square", ["focus"] = "0.5,0.5" })
})).Ok();
Assert.Equal("a red square", edited.Body["media_attachments"]![0]!.Text("description"));
Assert.Equal(0.5, edited.Body["media_attachments"]![0]!["meta"]!["focus"]!["x"]!.GetValue<double>());
Assert.Equal("a warning", edited.Body.Text("spoiler_text"));
Assert.True(edited.Body["sensitive"]!.GetValue<bool>());
Assert.Equal("it", edited.Body.Text("language"));
Assert.Equal("a red square", (await DB.Default.Find<MediaAttachment>().OneAsync(id, Token)).Description);
// an edit that doesn't send media keeps them, described as they were; an empty media_ids takes them away
var kept = (await alice.Client.Put($"/api/v1/statuses/{statusId}", ("status", "media kept"))).Ok();
Assert.Equal("a red square", kept.Body["media_attachments"]![0]!.Text("description"));
var bare = (await alice.Client.Json(HttpMethod.Put, $"/api/v1/statuses/{statusId}", new System.Text.Json.Nodes.JsonObject
{
["status"] = "no picture now",
["media_ids"] = new System.Text.Json.Nodes.JsonArray()
})).Ok();
Assert.Empty(bare.Body["media_attachments"]!.AsArray());
Assert.Equal("edited out", (await DB.Default.Find<MediaAttachment>().OneAsync(id, Token)).TrashReason);
}
[Fact]
public async Task A_replaced_avatar_is_trashed_and_a_picture_that_is_not_one_is_refused()
{
var alice = await _host.Mastodon("alice");
async Task<ApiAnswer> Picture(byte[] bytes, string type, string name) =>
await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Patch, "/api/v1/accounts/update_credentials")
{
Content = MastodonHelpers.Multipart(("avatar", bytes, type, name))
});
var first = (await Picture(MastodonHelpers.JpegWithMetadata(300, 300), "image/jpeg", "one.jpg")).Ok().Body.Text("avatar");
var second = (await Picture(MastodonHelpers.JpegWithMetadata(320, 300), "image/jpeg", "two.jpg")).Ok().Body.Text("avatar");
Assert.NotEqual(first, second);
Assert.Equal(HttpStatusCode.NotFound, await Served(first));
Assert.Equal(HttpStatusCode.OK, await Served(second));
var row = await DB.Default.Find<MediaAttachment>().Match(m => m.ProfileOfAvatarId == alice.Persona.Id && m.Kind == "avatar" && m.TrashedAt == null).ExecuteSingleAsync(Token);
Assert.EndsWith(row.FilePath, second);
// not an image: 422, and the avatar stays as it was
var refused = await Picture("<svg xmlns='http://www.w3.org/2000/svg'/>"u8.ToArray(), "image/svg+xml", "x.svg");
Assert.Equal(HttpStatusCode.UnprocessableEntity, refused.Status);
Assert.Equal(second, (await alice.Client.Get("/api/v1/accounts/verify_credentials")).Ok().Body.Text("avatar"));
}
// Mastodon's DELETE /api/v1/profile/avatar and /header: the picture goes, the default shows again
[Fact]
public async Task A_picture_is_removed_and_the_default_shows_again()
{
var alice = await _host.Mastodon($"pic{Guid.NewGuid():N}"[..12]);
var form = MastodonHelpers.Multipart(("avatar", MastodonHelpers.JpegWithMetadata(200, 200), "image/jpeg", "me.jpg"),
("header", MastodonHelpers.JpegWithMetadata(900, 300), "image/jpeg", "top.jpg"));
var set = (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Patch, "/api/v1/accounts/update_credentials") { Content = form })).Ok();
var avatar = set.Body.Text("avatar");
var header = set.Body.Text("header");
var removed = (await alice.Client.Delete("/api/v1/profile/avatar")).Ok();
Assert.Contains("missing", removed.Body.Text("avatar"));
Assert.Equal(header, removed.Body.Text("header"));
Assert.Equal(HttpStatusCode.NotFound, await Served(avatar));
Assert.Equal(HttpStatusCode.OK, await Served(header));
Assert.Contains("missing", (await alice.Client.Delete("/api/v1/profile/header")).Ok().Body.Text("header"));
Assert.Equal(HttpStatusCode.NotFound, await Served(header));
Assert.False(await DB.Default.Find<MediaAttachment>().Match(m => m.ProfileOfAvatarId == alice.Persona.Id && m.TrashedAt == null).ExecuteAnyAsync(Token));
}
[Fact]
public async Task A_removed_roots_media_and_pictures_are_trashed_and_its_scheduled_posts_dropped()
{
var gone = await _host.Mastodon($"gone{Guid.NewGuid():N}"[..12]);
var posted = await Upload(gone, "posted.jpg");
var url = (await gone.Client.Post("/api/v1/statuses", ("status", "soon gone"), ("media_ids[]", posted))).Ok()
.Body["media_attachments"]![0]!["url"]!.GetValue<string>();
var held = await Upload(gone, "held.jpg");
(await gone.Client.Post("/api/v1/statuses", ("status", "later"), ("media_ids[]", held),
("scheduled_at", DateTime.UtcNow.AddHours(2).ToString("O")))).Ok();
var avatar = (await gone.Client.Exchange(new HttpRequestMessage(HttpMethod.Patch, "/api/v1/accounts/update_credentials")
{
Content = MastodonHelpers.Multipart(("avatar", MastodonHelpers.JpegWithMetadata(200, 200), "image/jpeg", "me.jpg"))
})).Ok().Body.Text("avatar");
Assert.True(await _host.Get<IRootRemoval>().Remove(gone.Persona.Root.Id, Token));
Assert.Equal(HttpStatusCode.NotFound, await Served(url));
Assert.Equal(HttpStatusCode.NotFound, await Served(avatar));
Assert.False(await DB.Default.Find<MediaAttachment>().Match(m => m.OwnerAvatarId == gone.Persona.Id && m.TrashedAt == null).ExecuteAnyAsync(Token));
Assert.False(await DB.Default.Find<Models.Social.ScheduledStatus>().Match(s => s.AvatarId == gone.Persona.Id).ExecuteAnyAsync(Token));
}
// what a login's media take: every persona's uploads and pictures, trashed ones no longer
[Fact]
public async Task A_logins_storage_counts_what_its_personas_keep()
{
var alice = await _host.Mastodon($"store{Guid.NewGuid():N}"[..12]);
using var root = _host.As(alice.Persona.Root.Jwt);
async Task<System.Text.Json.Nodes.JsonObject> Storage() => await (await root.GetAsync("/clientapi/user/storage", Token)).JsonBody();
Assert.Equal(0, Storage().Result["mediaBytes"]!.GetValue<long>());
var id = await Upload(alice, "a.jpg");
var stored = await Storage();
Assert.True(stored["mediaBytes"]!.GetValue<long>() > 0);
Assert.Equal(1, stored["mediaFiles"]!.GetValue<long>());
Assert.Null(stored["quotaBytes"]);
await _host.Get<IMediaService>().Trash(m => m.ID == id, "test", Token);
Assert.Equal(0, (await Storage())["mediaFiles"]!.GetValue<long>());
}
[Fact]
public async Task The_audit_adopts_todays_pictures_and_trashes_what_nothing_holds()
{
var media = _host.Get<IMediaService>();
var bob = await _host.Mastodon($"bob{Guid.NewGuid():N}"[..12]);
// a picture shown from before pictures had rows, and a leftover nothing holds
var picture = $"2020/01/{Guid.NewGuid():N}.jpg";
var leftover = $"2020/01/{Guid.NewGuid():N}.jpg";
foreach (var relative in new[] { picture, leftover })
{
Directory.CreateDirectory(Path.GetDirectoryName(Path.Combine(media.Root, relative))!);
await File.WriteAllBytesAsync(Path.Combine(media.Root, relative), new byte[] { 1, 2, 3 }, Token);
}
await DB.Default.Update<Models.User.Avatar>().MatchID(bob.Persona.Id).Modify(a => a.PictureURL, media.Url(picture)).ExecuteAsync(Token);
var looked = await MediaAudit.Run(media, fix: false, Token);
Assert.True(looked.Adopted >= 1);
Assert.True(looked.Unheld >= 1);
Assert.True(File.Exists(Path.Combine(media.Root, leftover)));
await MediaAudit.Run(media, fix: true, Token);
Assert.True(await DB.Default.Find<MediaAttachment>().Match(m => m.ProfileOfAvatarId == bob.Persona.Id && m.FilePath == picture && m.TrashedAt == null).ExecuteAnyAsync(Token));
Assert.Equal(HttpStatusCode.OK, await Served(media.Url(picture)));
Assert.Equal(HttpStatusCode.NotFound, await Served(media.Url(leftover)));
Assert.True(File.Exists(Path.Combine(media.TrashRoot, leftover)));
Assert.True(await DB.Default.Find<MediaAttachment>().Match(m => m.FilePath == leftover && m.TrashedAt != null).ExecuteAnyAsync(Token));
}
}
}
@@ -0,0 +1,103 @@
using MongoDB.Entities;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Features;
using PrivaPub.Federation.Signing;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// RFC 9421 message signatures, as WordPress's ActivityPub plugin, Ghost and Fedify send them first: a delivery so signed
// is taken in one request, and refused when its body or its target is not what was signed
[Trait("Category", "Integration")]
public sealed class MessageSignatureTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
[Fact]
public async Task A_delivery_signed_as_rfc9421_is_taken_and_one_whose_body_or_target_differs_is_refused()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var create = bob.Create("<p>signed the new way</p>", new[] { alice.Uri });
var noteId = create["object"]!["id"]!.GetValue<string>();
using var client = _host.Client();
var tampered = bob.MessageSignedPost(alice.Mouth, create);
tampered.Content = new ByteArrayContent(Encoding.UTF8.GetBytes(create.ToJsonString().Replace("new way", "other way")));
tampered.Content.Headers.TryAddWithoutValidation("Content-Type", "application/activity+json");
tampered.Content.Headers.TryAddWithoutValidation("Content-Digest", bob.MessageSignedPost(alice.Mouth, create).Content!.Headers.GetValues("Content-Digest").First());
Assert.Equal(HttpStatusCode.Unauthorized, (await client.SendAsync(tampered, Token)).StatusCode);
var elsewhere = bob.MessageSignedPost(alice.Mouth, create, signedFor: "/human-centipede");
Assert.Equal(HttpStatusCode.Unauthorized, (await client.SendAsync(elsewhere, Token)).StatusCode);
Assert.Equal(HttpStatusCode.Accepted, (await client.SendAsync(bob.MessageSignedPost(alice.Mouth, create), Token)).StatusCode);
Assert.Equal(1, await _host.RunInbox(Id(create), Token));
Assert.Contains("signed the new way", (await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteSingleAsync(Token)).ContentHtml);
}
static string Id(JsonObject activity) => activity["id"]!.GetValue<string>();
// a signed fetch (a GET, no body) verifies the same way, and so does RSA-PSS with SHA-512
[Fact]
public void A_signed_fetch_and_an_rsa_pss_signature_verify()
{
using var key = RSA.Create(2048);
var message = new HttpRequestMessage(HttpMethod.Get, "https://privapub.test/peasants/alice/scribbles/1");
MessageSignatures.Sign(message, "https://peer.example/users/bob#main-key", key.ExportPkcs8PrivateKeyPem(), body: null);
var request = Request(message);
var signature = RequestSignature.Of(request);
Assert.Equal("rfc9421:rsa-v1_5-sha256", signature.Scheme);
Assert.Null(signature.Problem(request, body: null));
Assert.True(signature.VerifiedBy(key.ExportSubjectPublicKeyInfoPem(), signature.Signed(request, "https://privapub.test")));
Assert.False(signature.VerifiedBy(key.ExportSubjectPublicKeyInfoPem(), signature.Signed(request, "https://elsewhere.example")));
var created = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
var parameters = $"(\"@method\" \"@target-uri\");created={created};keyid=\"bob\";alg=\"rsa-pss-sha512\"";
var signatureBase = $"\"@method\": GET\n\"@target-uri\": https://privapub.test/peasants/alice/scribbles/1\n\"@signature-params\": {parameters}";
var signed = key.SignData(Encoding.UTF8.GetBytes(signatureBase), HashAlgorithmName.SHA512, RSASignaturePadding.Pss);
var pss = new HttpRequestMessage(HttpMethod.Get, "https://privapub.test/peasants/alice/scribbles/1");
pss.Headers.TryAddWithoutValidation("Signature-Input", $"pss={parameters}");
pss.Headers.TryAddWithoutValidation("Signature", $"pss=:{Convert.ToBase64String(signed)}:");
var pssRequest = Request(pss);
var pssSignature = RequestSignature.Of(pssRequest);
Assert.True(pssSignature.VerifiedBy(key.ExportSubjectPublicKeyInfoPem(), pssSignature.Signed(pssRequest, "https://privapub.test")));
}
static HttpRequest Request(HttpRequestMessage message)
{
var context = new DefaultHttpContext();
context.Request.Method = message.Method.Method;
context.Request.Host = new HostString(message.RequestUri!.Host);
context.Request.Path = message.RequestUri.AbsolutePath;
context.Features.Get<IHttpRequestFeature>()!.RawTarget = message.RequestUri.PathAndQuery;
foreach (var (name, values) in message.Headers)
context.Request.Headers[name] = values.ToArray();
return context.Request;
}
}
}
@@ -0,0 +1,119 @@
using MongoDB.Entities;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
// Mastodon's notification policy and requests (4.3): accepted by default; a policy filters what strangers, new accounts
// or unasked private mentions send into a request per account, which the persona lets in (and that account from then
// on) or dismisses; or drops it
[Trait("Category", "Integration")]
public sealed class NotificationPolicyTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static async Task<IEnumerable<string>> Notifiers(Mastodon persona, string query = "") =>
(await persona.Client.Get("/api/v1/notifications" + query)).Ok().Array.Select(n => n!["account"].Text("id"));
[Fact]
public async Task Everything_is_accepted_until_the_persona_chooses()
{
var alice = await _host.Mastodon("alice");
var stranger = await _host.Mastodon("stranger");
var post = (await alice.Status("like me")).Text("id");
var policy = (await alice.Client.Get("/api/v2/notifications/policy")).Ok().Body;
(await stranger.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
Assert.Equal(["accept"], new[] { "for_not_following", "for_not_followers", "for_new_accounts", "for_private_mentions", "for_limited_accounts" }
.Select(key => policy.Text(key)).Distinct());
Assert.Contains(stranger.Id, await Notifiers(alice));
Assert.Empty((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array);
}
[Fact]
public async Task A_strangers_notifications_wait_in_a_request_which_lets_them_and_the_stranger_in()
{
var alice = await _host.Mastodon("alice");
var friend = await _host.Mastodon("friend");
var stranger = await _host.Mastodon("stranger");
(await alice.Client.Post($"/api/v1/accounts/{friend.Id}/follow")).Ok();
var post = (await alice.Status("like me")).Text("id");
var policy = (await alice.Client.Patch("/api/v2/notifications/policy", ("for_not_following", "filter"))).Ok().Body;
Assert.Equal("filter", policy.Text("for_not_following"));
(await friend.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
(await stranger.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
Assert.Equal([friend.Id], await Notifiers(alice));
Assert.Equal(1, (await alice.Client.Get("/api/v1/notifications/unread_count")).Ok().Body["count"]!.GetValue<int>());
Assert.DoesNotContain(stranger.Id, (await alice.Client.Get("/api/v2/notifications")).Ok().Body["accounts"]!.AsArray().Select(a => a.Text("id")));
var request = Assert.Single((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array)!;
Assert.Equal((stranger.Id, "1", post), (request["account"].Text("id"), request.Text("notifications_count"), request["last_status"].Text("id")));
var summary = (await alice.Client.Get("/api/v2/notifications/policy")).Ok().Body["summary"]!;
Assert.Equal((1, 1), (summary["pending_requests_count"]!.GetValue<int>(), summary["pending_notifications_count"]!.GetValue<int>()));
Assert.Equal([stranger.Id], await Notifiers(alice, $"?account_id={stranger.Id}&include_filtered=true"));
(await alice.Client.Post($"/api/v1/notifications/requests/{request.Text("id")}/accept")).Ok();
Assert.Contains(stranger.Id, await Notifiers(alice));
Assert.Empty((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array);
var again = (await alice.Status("like me again")).Text("id");
(await stranger.Client.Post($"/api/v1/statuses/{again}/favourite")).Ok();
Assert.Equal(2, (await Notifiers(alice, "?types[]=favourite")).Count(id => id == stranger.Id));
}
[Fact]
public async Task A_dismissed_request_takes_its_notifications_with_it_and_a_dropped_one_is_never_kept()
{
var alice = await _host.Mastodon("alice");
var stranger = await _host.Mastodon("stranger");
var newcomer = await _host.Mastodon("newcomer");
var post = (await alice.Status("like me")).Text("id");
// (v1 speaks of filtering only)
Assert.True((await alice.Client.Put("/api/v1/notifications/policy", ("filter_not_following", "true"))).Ok().Body.Flag("filter_not_following"));
(await stranger.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
var request = Assert.Single((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array)!;
(await alice.Client.Post($"/api/v1/notifications/requests/{request.Text("id")}/dismiss")).Ok();
Assert.Empty((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array);
Assert.Empty(await Notifiers(alice, $"?account_id={stranger.Id}&include_filtered=true"));
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Get($"/api/v1/notifications/requests/{request.Text("id")}")).Status);
(await alice.Client.Patch("/api/v2/notifications/policy", ("for_new_accounts", "drop"))).Ok();
(await newcomer.Client.Post($"/api/v1/statuses/{post}/favourite")).Ok();
Assert.Empty(await Notifiers(alice, $"?account_id={newcomer.Id}&include_filtered=true"));
Assert.Empty((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array);
}
[Fact]
public async Task An_unasked_private_mention_is_filtered_and_one_answering_the_persona_is_not()
{
var alice = await _host.Mastodon("alice");
var stranger = await _host.Mastodon("stranger");
(await alice.Client.Patch("/api/v2/notifications/policy", ("for_private_mentions", "filter"))).Ok();
var asked = (await alice.Status($"@{stranger.UserName} write to me", ("visibility", "direct"))).Text("id");
await stranger.Status($"@{alice.UserName} hello out of nowhere", ("visibility", "direct"));
await stranger.Status($"@{alice.UserName} as you asked", ("visibility", "direct"), ("in_reply_to_id", asked));
var mentions = (await alice.Client.Get("/api/v1/notifications?types[]=mention")).Ok().Array.Select(n => n!["status"].Text("content")).ToList();
Assert.Contains(mentions, c => c.Contains("as you asked"));
Assert.DoesNotContain(mentions, c => c.Contains("out of nowhere"));
Assert.Single((await alice.Client.Get("/api/v1/notifications/requests")).Ok().Array);
}
}
}
+346
View File
@@ -0,0 +1,346 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class OAuthTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
async Task<Persona> NewPersona(string name = "oauth") => await _host.Persona(await _host.SignUp(), name);
static async Task<bool> SignedIn(HttpClient client, OAuthApp app)
{
var response = await client.GetAsync(app.ReturnUrl(), TestContext.Current.CancellationToken);
return response.StatusCode == HttpStatusCode.OK;
}
static async Task AssertMastodonError(HttpResponseMessage response, HttpStatusCode status)
{
Assert.Equal(status, response.StatusCode);
Assert.False(string.IsNullOrEmpty((await response.JsonBody())["error"]?.GetValue<string>()));
}
[Fact]
public async Task The_token_names_the_persona_and_never_the_root()
{
var persona = await NewPersona();
var root = persona.Root;
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
var exchanged = await client.Exchange(app, await client.Code(app, persona));
var body = await exchanged.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, exchanged.StatusCode);
Assert.DoesNotContain(root.Id, body);
Assert.DoesNotContain(root.UserName, body);
var token = JsonNode.Parse(body)!["access_token"]!.GetValue<string>();
using var api = _host.As(token);
var account = await api.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken);
Assert.Contains($"\"id\":\"{persona.Id}\"", account);
Assert.DoesNotContain(root.Id, account);
Assert.DoesNotContain(root.UserName, account);
var stored = await ClientApi.StoredTokens(app.ClientId);
Assert.NotEmpty(stored);
foreach (var entry in stored)
{
Assert.Equal(persona.Id, entry["subject"].AsString);
var raw = entry.ToString();
var payload = entry.Contains("payload") && entry["payload"].IsString ? ClientApi.JwtPayload(entry["payload"].AsString) : string.Empty;
Assert.DoesNotContain(root.Id, raw + payload);
Assert.DoesNotContain(root.UserName, raw + payload);
}
var access = stored.Single(t => ClientApi.IsAccessToken(t));
Assert.Equal(persona.Id, JsonNode.Parse(ClientApi.JwtPayload(access["payload"].AsString))!["sub"]!.GetValue<string>());
}
[Fact]
public async Task A_wrong_password_shows_an_error_and_signs_nobody_in()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
var response = await client.SignIn(persona.Root, app.ReturnUrl(), password: "Wrong-Pass-1!");
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.Contains("That username and password do not match.", await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
Assert.DoesNotContain(response.Headers.TryGetValues("Set-Cookie", out var cookies) ? cookies : Enumerable.Empty<string>(),
c => c.StartsWith("privapub.oauth=", StringComparison.Ordinal));
var authorize = await client.GetAsync(app.ReturnUrl(), TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.Redirect, authorize.StatusCode);
Assert.StartsWith("/oauth/login?", authorize.Headers.Location!.OriginalString);
}
[Fact]
public async Task A_login_without_the_antiforgery_token_is_refused()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
await client.GetStringAsync("/oauth/login", TestContext.Current.CancellationToken);
var response = await client.Form("/oauth/login", ("returnUrl", "/oauth/authorize?x=1"), ("userName", persona.Root.UserName), ("password", persona.Root.Password));
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.False(response.Headers.TryGetValues("Set-Cookie", out var cookies) && cookies.Any(c => c.StartsWith("privapub.oauth=", StringComparison.Ordinal)));
}
[Theory]
[InlineData("https://evil.example/oauth/authorize?client_id=x")]
[InlineData("//evil.example/oauth/authorize?client_id=x")]
[InlineData("/\\evil.example/oauth/authorize?client_id=x")]
[InlineData("/somewhere/else")]
public async Task The_return_address_never_leaves_the_site(string returnUrl)
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var response = await client.SignIn(persona.Root, returnUrl);
Assert.Equal(HttpStatusCode.Redirect, response.StatusCode);
Assert.Equal("/oauth/authorize?signed_in=1", response.Headers.Location!.OriginalString);
}
[Fact]
public async Task Denying_answers_access_denied_and_no_code()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
Assert.Equal(HttpStatusCode.Redirect, (await client.SignIn(persona.Root, app.ReturnUrl())).StatusCode);
var response = await client.Choose(app, persona.Id, decision: "deny");
var page = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
Assert.Null(ClientApi.CodeIn(page));
Assert.Equal(HttpStatusCode.Redirect, response.StatusCode);
var location = response.Headers.Location!.OriginalString;
Assert.StartsWith(ClientApi.OutOfBand + "?error=access_denied", location);
Assert.DoesNotContain("code=", location);
Assert.DoesNotContain(await ClientApi.StoredTokens(app.ClientId), ClientApi.IsAccessToken);
}
[Fact]
public async Task Choosing_another_roots_persona_shows_the_choice_again_without_a_code()
{
var mine = await NewPersona();
var theirs = await NewPersona("theirs");
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
Assert.Equal(HttpStatusCode.Redirect, (await client.SignIn(mine.Root, app.ReturnUrl())).StatusCode);
var response = await client.Choose(app, theirs.Id);
var page = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.Null(ClientApi.CodeIn(page));
Assert.Contains(mine.Id, page);
Assert.DoesNotContain(theirs.Id, page);
Assert.DoesNotContain(theirs.UserName, page);
Assert.DoesNotContain(await ClientApi.StoredTokens(app.ClientId), ClientApi.IsAccessToken);
}
[Fact]
public async Task A_banned_root_is_sent_back_to_the_login()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
Assert.Equal(HttpStatusCode.Redirect, (await client.SignIn(persona.Root, app.ReturnUrl())).StatusCode);
var choice = await client.Choice(app);
await ClientApi.Ban(persona.Root.Id);
var authorize = await client.GetAsync(app.ReturnUrl(), TestContext.Current.CancellationToken);
var chosen = await client.Decide(choice, persona.Id);
var again = await client.SignIn(persona.Root, app.ReturnUrl());
Assert.Equal(HttpStatusCode.Redirect, authorize.StatusCode);
Assert.StartsWith("/oauth/login?returnUrl=", authorize.Headers.Location!.OriginalString);
Assert.Equal(HttpStatusCode.Redirect, chosen.StatusCode);
Assert.StartsWith("/oauth/login?returnUrl=", chosen.Headers.Location!.OriginalString);
Assert.Equal(HttpStatusCode.OK, again.StatusCode);
Assert.Contains("That username and password do not match.", await again.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
}
[Fact]
public async Task A_code_issued_before_a_ban_buys_no_token()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
var code = await client.Code(app, persona);
await ClientApi.Ban(persona.Root.Id);
var response = await client.Exchange(app, code);
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.Equal("invalid_grant", (await response.JsonBody())["error"]!.GetValue<string>());
}
[Fact]
public async Task Force_login_asks_for_the_password_again()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
Assert.Equal(HttpStatusCode.Redirect, (await client.SignIn(persona.Root, app.ReturnUrl())).StatusCode);
Assert.True(await SignedIn(client, app));
var forced = await client.GetAsync(app.ReturnUrl("force_login=true"), TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.Redirect, forced.StatusCode);
var location = forced.Headers.Location!.OriginalString;
Assert.StartsWith("/oauth/login?returnUrl=", location);
var returnUrl = Uri.UnescapeDataString(location["/oauth/login?returnUrl=".Length..]);
Assert.Contains("force_login=true", returnUrl);
Assert.DoesNotContain("signed_in", returnUrl);
var signedIn = await client.SignIn(persona.Root, returnUrl);
Assert.Equal(HttpStatusCode.Redirect, signedIn.StatusCode);
Assert.EndsWith("&signed_in=1", signedIn.Headers.Location!.OriginalString);
var page = await client.GetAsync(signedIn.Headers.Location!.OriginalString, TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, page.StatusCode);
Assert.Contains(persona.Id, await page.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
}
[Fact]
public async Task An_authorization_code_works_once()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
var code = await client.Code(app, persona);
var first = await client.Exchange(app, code);
var second = await client.Exchange(app, code);
Assert.Equal(HttpStatusCode.OK, first.StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, second.StatusCode);
Assert.Equal("invalid_grant", (await second.JsonBody())["error"]!.GetValue<string>());
using var api = _host.As((await first.JsonBody())["access_token"]!.GetValue<string>());
await AssertMastodonError(await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken), HttpStatusCode.Unauthorized);
}
[Theory]
[InlineData("password")]
[InlineData("refresh_token")]
public async Task Grants_other_than_code_and_client_credentials_are_refused(string grant)
{
var persona = await NewPersona();
using var client = _host.Client();
var app = await client.RegisterApp();
var response = await client.Form("/oauth/token", ("grant_type", grant), ("client_id", app.ClientId), ("client_secret", app.ClientSecret),
("username", persona.Root.UserName), ("password", persona.Root.Password), ("refresh_token", "anything"), ("scope", "read"));
var body = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.DoesNotContain("access_token", body);
Assert.Equal("unsupported_grant_type", JsonNode.Parse(body)!["error"]!.GetValue<string>());
}
[Fact]
public async Task A_client_credentials_token_is_no_persona()
{
using var client = _host.Client();
var app = await client.RegisterApp("read write");
var response = await client.Form("/oauth/token", ("grant_type", "client_credentials"), ("client_id", app.ClientId), ("client_secret", app.ClientSecret), ("scope", "read"));
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var token = (await response.JsonBody())["access_token"]!.GetValue<string>();
using var api = _host.As(token);
Assert.Equal(HttpStatusCode.OK, (await api.GetAsync("/api/v1/apps/verify_credentials", TestContext.Current.CancellationToken)).StatusCode);
await AssertMastodonError(await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken), HttpStatusCode.Unauthorized);
await AssertMastodonError(await api.GetAsync("/api/v1/notifications", TestContext.Current.CancellationToken), HttpStatusCode.Unauthorized);
}
[Fact]
public async Task A_read_only_token_cannot_post()
{
var token = await _host.MastodonToken(await NewPersona(), "read");
using var api = _host.As(token);
var response = await api.Form("/api/v1/statuses", ("status", "nope"));
await AssertMastodonError(response, HttpStatusCode.Forbidden);
Assert.Equal(HttpStatusCode.OK, (await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode);
}
[Fact]
public async Task The_follow_scope_covers_reading_follows()
{
var token = await _host.MastodonToken(await NewPersona(), "follow");
using var api = _host.As(token);
Assert.Equal(HttpStatusCode.OK, (await api.GetAsync("/api/v1/follow_requests", TestContext.Current.CancellationToken)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await api.GetAsync("/api/v1/blocks", TestContext.Current.CancellationToken)).StatusCode);
await AssertMastodonError(await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken), HttpStatusCode.Forbidden);
}
[Fact]
public async Task A_revoked_token_stops_working()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
var token = await client.Token(app, persona);
using var api = _host.As(token);
Assert.Equal(HttpStatusCode.OK, (await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode);
var revoked = await client.Form("/oauth/revoke", ("token", token), ("client_id", app.ClientId), ("client_secret", app.ClientSecret));
Assert.Equal(HttpStatusCode.OK, revoked.StatusCode);
await AssertMastodonError(await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken), HttpStatusCode.Unauthorized);
}
[Fact]
public async Task The_login_and_authorize_pages_are_hardened()
{
var persona = await NewPersona();
using var client = _host.Client(cookies: true);
var app = await client.RegisterApp();
var login = await client.GetAsync("/oauth/login?returnUrl=" + Uri.EscapeDataString(app.ReturnUrl()), TestContext.Current.CancellationToken);
var failed = await client.SignIn(persona.Root, app.ReturnUrl(), password: "Wrong-Pass-1!");
Assert.Equal(HttpStatusCode.Redirect, (await client.SignIn(persona.Root, app.ReturnUrl())).StatusCode);
var authorize = await client.GetAsync(app.ReturnUrl(), TestContext.Current.CancellationToken);
foreach (var response in new[] { login, failed, authorize })
{
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var csp = string.Join(";", response.Headers.GetValues("Content-Security-Policy"));
Assert.Contains("default-src 'none'", csp);
Assert.Contains("frame-ancestors 'none'", csp);
Assert.Contains("no-store", response.Headers.CacheControl!.ToString());
}
}
[Fact]
public async Task The_eleventh_login_from_one_address_in_a_minute_is_refused()
{
using var client = _host.ClientAt("198.51.100.12");
for (var attempt = 1; attempt <= 10; attempt++)
Assert.Equal(HttpStatusCode.OK, (await client.GetAsync("/oauth/login", TestContext.Current.CancellationToken)).StatusCode);
var eleventh = await client.GetAsync("/oauth/login", TestContext.Current.CancellationToken);
using var elsewhere = _host.ClientAt("198.51.100.13");
var other = await elsewhere.GetAsync("/oauth/login", TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.TooManyRequests, eleventh.StatusCode);
Assert.Equal(HttpStatusCode.OK, other.StatusCode);
}
}
}
+270
View File
@@ -0,0 +1,270 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.Host.FederationHelpers;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Http
{
// Two views of the same fact give one answer (owner decision 2026-10-04: fix the mismatches).
[Trait("Category", "Integration")]
public sealed class OneAnswerTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
[Fact]
public async Task Anyone_may_search_and_only_a_signed_in_reader_resolves_or_pages()
{
var reader = await _host.Mastodon("searcher");
Assert.Equal(HttpStatusCode.OK, (await _client.Get("/api/v2/search?q=somebody")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Get("/api/v2/search?q=somebody&resolve=true")).Status);
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Get("/api/v2/search?q=somebody&offset=5")).Status);
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get("/api/v2/search?q=somebody&offset=5")).Status);
}
[Fact]
public async Task A_remote_account_that_deletes_itself_leaves_nothing_in_the_counts()
{
var token = TestContext.Current.CancellationToken;
var author = await _host.Mastodon("liked");
var post = await author.Status("like me");
var fan = new RemoteActor(_peer, "fickle");
var like = new JsonObject
{
["id"] = $"{fan.Id}#likes/{Guid.NewGuid():N}",
["type"] = "Like",
["actor"] = fan.Id,
["object"] = post.Text("uri")
};
Assert.Equal(HttpStatusCode.Accepted, (await _client.SendAsync(fan.SignedPost("/human-centipede", like), token)).StatusCode);
await _host.RunInbox(like["id"]!.GetValue<string>(), token);
Assert.Equal(1, (await author.Client.Get($"/api/v1/statuses/{post.Text("id")}")).Ok().Body.Number("favourites_count"));
var delete = new JsonObject
{
["id"] = $"{fan.Id}#delete",
["type"] = "Delete",
["actor"] = fan.Id,
["object"] = fan.Id,
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public")
};
Assert.Equal(HttpStatusCode.Accepted, (await _client.SendAsync(fan.SignedPost("/human-centipede", delete), token)).StatusCode);
await _host.RunInbox(delete["id"]!.GetValue<string>(), token);
Assert.Equal(0, (await author.Client.Get($"/api/v1/statuses/{post.Text("id")}")).Ok().Body.Number("favourites_count"));
Assert.Empty((await author.Client.Get($"/api/v1/statuses/{post.Text("id")}/favourited_by")).Ok().Array);
Assert.False(await DB.Default.Find<Favourite>().Match(f => f.ActorURI == fan.Id).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Notification>().Match(n => n.FromActorURI == fan.Id).ExecuteAnyAsync(token));
Assert.Equal(0, (await author.Client.Get("/api/v1/notifications/unread_count")).Ok().Body.Number("count"));
}
[Fact]
public async Task Only_public_and_unlisted_replies_are_counted()
{
var author = await _host.Mastodon("threadstart");
var replier = await _host.Mastodon("replier");
var root = await author.Status("say something");
var id = root.Text("id");
await replier.Status("in public", ("in_reply_to_id", id));
await replier.Status("quietly", ("in_reply_to_id", id), ("visibility", "unlisted"));
await replier.Status("for my followers", ("in_reply_to_id", id), ("visibility", "private"));
await replier.Status($"@{author.UserName} just you", ("in_reply_to_id", id), ("visibility", "direct"));
Assert.Equal(2, (await author.Client.Get($"/api/v1/statuses/{id}")).Ok().Body.Number("replies_count"));
}
[Fact]
public async Task A_status_longer_than_advertised_is_refused()
{
var author = await _host.Mastodon("wordy");
var limit = (await _client.Get("/api/v2/instance")).Ok().Body["configuration"]!["statuses"].Number("max_characters");
var refused = await author.Client.Post("/api/v1/statuses", ("status", new string('a', limit + 1)));
var accepted = await author.Client.Post("/api/v1/statuses", ("status", new string('a', limit)));
Assert.Equal(HttpStatusCode.UnprocessableEntity, refused.Status);
Assert.Equal(HttpStatusCode.OK, accepted.Status);
}
[Fact]
public async Task Joining_a_community_by_invitation_is_following_it()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "commowner");
var joinerRoot = await _host.SignUp("joiner");
var joiner = await _host.Persona(joinerRoot, "joiner");
var group = await _host.Group(owner, community: true);
var groupName = group["userName"]!.GetValue<string>();
using var client = _host.As(joinerRoot.Jwt);
var joined = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode = group["invitationCode"]!.GetValue<string>() });
Assert.Equal(HttpStatusCode.OK, joined.StatusCode);
Assert.Equal(1, (await _client.Fetch($"/peasants/{groupName}/groupies")).Json["totalItems"]!.GetValue<int>());
Assert.True(await DB.Default.Find<Following>().Match(f => f.AvatarId == joiner.Id && f.State == FollowState.Accepted).ExecuteAnyAsync(token));
}
public static TheoryData<string> AnsweredRoutes() => new()
{
"/api/v1/directory",
"/api/v1/tags/cats",
"/api/v1/instance/languages",
"/api/v1/instance/translation_languages",
"/api/v1/instance/domain_blocks",
"/api/v1/instance/privacy_policy",
"/api/v1/timelines/link?url=https%3A%2F%2Fexample.org%2F",
"/api/v1/accounts/000000000000000000000000/identity_proofs"
};
[Theory]
[MemberData(nameof(AnsweredRoutes))]
public async Task Mastodon_routes_we_have_nothing_behind_answer_instead_of_404(string path)
{
Assert.Equal(HttpStatusCode.OK, (await _client.Get(path)).Status);
}
[Fact]
public async Task Likes_of_one_post_come_as_one_group_of_notifications()
{
var author = await _host.Mastodon("grouped");
var first = await _host.Mastodon("fanone");
var second = await _host.Mastodon("fantwo");
var post = await author.Status("like me twice");
var id = post.Text("id");
await first.Client.Post($"/api/v1/statuses/{id}/favourite");
await second.Client.Post($"/api/v1/statuses/{id}/favourite");
var list = (await author.Client.Get("/api/v2/notifications")).Ok().Body;
var group = Assert.Single(list["notification_groups"]!.AsArray(), g => g!.Text("type") == "favourite")!;
var key = group.Text("group_key");
Assert.Equal($"grouped-favourite-{id}", key);
// told alone (v1, a stream), each notification names the group it belongs to
Assert.All((await author.Client.Get("/api/v1/notifications?types[]=favourite")).Ok().Array, n => Assert.Equal(key, n.Text("group_key")));
Assert.Equal(2, group.Number("notifications_count"));
Assert.Equal(2, group["sample_account_ids"]!.AsArray().Count);
Assert.Equal(id, group.Text("status_id"));
Assert.Contains(list["statuses"]!.AsArray(), s => s!.Text("id") == id);
Assert.True((await author.Client.Get("/api/v2/notifications/unread_count")).Ok().Body.Number("count") >= 1);
Assert.Equal(key, Assert.Single((await author.Client.Get($"/api/v2/notifications/{key}")).Ok().Body["notification_groups"]!.AsArray())!.Text("group_key"));
Assert.Equal(2, (await author.Client.Get($"/api/v2/notifications/{key}/accounts")).Ok().Array.Count);
// a notifications marker past them reads them
var newest = (await author.Client.Get("/api/v1/notifications")).Ok().Ids.First();
(await author.Client.Post("/api/v1/markers", ("notifications[last_read_id]", newest))).Ok();
Assert.Equal(0, (await author.Client.Get("/api/v2/notifications/unread_count")).Ok().Body.Number("count"));
Assert.Equal(0, (await author.Client.Get("/api/v1/notifications/unread_count")).Ok().Body.Number("count"));
Assert.Equal(HttpStatusCode.OK, (await author.Client.Post($"/api/v2/notifications/{key}/dismiss")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await author.Client.Get($"/api/v2/notifications/{key}")).Status);
}
[Fact]
public async Task A_hashtag_link_opens_a_page_of_public_posts_with_that_tag()
{
var author = await _host.Mastodon("tagger");
var tag = $"tag{Guid.NewGuid():N}"[..16];
await author.Status($"about #{tag}");
await author.Status($"quietly about #{tag}", ("visibility", "private"));
var page = await _client.Fetch($"/tags/{tag}", Browser);
Assert.Equal(HttpStatusCode.OK, page.Status);
Assert.Contains("about", page.Text);
Assert.DoesNotContain("quietly", page.Text);
Assert.Contains("default-src 'none'", page.Response.Headers.GetValues("Content-Security-Policy").Single());
}
[Fact]
public async Task A_communitys_announces_resolve_while_the_post_is_shown()
{
var token = TestContext.Current.CancellationToken;
var owner = await _host.Persona(await _host.SignUp(), "announcer");
var community = await _host.FederatedGroup(owner, community: true);
var post = await _host.Publish(owner, new PrivaPub.Domain.Statuses.StatusDraft { Text = "into the community", PlainText = true, GroupId = community.Id });
var kept = await DB.Default.Find<GroupAnnouncement>().Match(a => a.GroupId == community.Id && a.ObjectURI == post.ObjectURI).ExecuteAsync(token);
Assert.Equal(2, kept.Count);//the activity, and the object for Mastodon
foreach (var announcement in kept)
{
var fetched = await _client.Fetch(PathOf(announcement.ActivityURI));
Assert.Equal(HttpStatusCode.OK, fetched.Status);
Assert.Equal("Announce", fetched.Json["type"]!.GetValue<string>());
Assert.Equal(announcement.ActivityURI, fetched.Json["id"]!.GetValue<string>());
}
var listed = await _client.Fetch($"/peasants/{community.UserName}/grunts/announce-{post.ID}");
Assert.Equal(HttpStatusCode.OK, listed.Status);
await _host.Remove(owner, post);
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(PathOf(kept[0].ActivityURI))).Status);
}
[Fact]
public async Task A_boost_points_at_the_boosted_posts_page()
{
var author = await _host.Mastodon("boosted");
var booster = await _host.Mastodon("booster");
var post = await author.Status("boost me");
var boost = (await booster.Client.Post($"/api/v1/statuses/{post.Text("id")}/reblog")).Ok().Body;
Assert.Equal(post.Text("url"), boost.Text("url"));
var browser = await _client.Fetch(PathOf(boost.Text("uri")), Browser);
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
}
[Fact]
public async Task A_remote_accounts_counts_are_what_its_server_publishes()
{
var token = TestContext.Current.CancellationToken;
var reader = await _host.Mastodon("counter");
var popular = new RemoteActor(_peer, "popular");
_peer.Serve(new Uri(popular.Id).AbsolutePath + "/followers",
$$"""{"@context":"https://www.w3.org/ns/activitystreams","id":"{{popular.Id}}/followers","type":"OrderedCollection","totalItems":42}""");
var remote = _host.Get<PrivaPub.Federation.Actors.IRemoteActorService>();
var account = await remote.GetActor(popular.Id, refresh: true, token);
await _host.Run(j => j.Kind == PrivaPub.Models.Jobs.JobKind.CountAccount && j.Payload == popular.Id, token);
var shown = (await reader.Client.Get($"/api/v1/accounts/{account.ID}")).Ok().Body;
Assert.Equal(42, shown.Number("followers_count"));
Assert.Equal(0, shown.Number("following_count"));
}
[Fact]
public async Task Notification_policy_and_requests_answer_a_signed_in_reader()
{
var reader = await _host.Mastodon("policy");
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get("/api/v1/notifications/policy")).Status);
Assert.Equal("accept", (await reader.Client.Get("/api/v2/notifications/policy")).Ok().Body.Text("for_not_following"));
Assert.Empty((await reader.Client.Get("/api/v1/notifications/requests")).Ok().Array);
}
}
}
+346
View File
@@ -0,0 +1,346 @@
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Media;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.IO.Compression;
using System.Net;
using System.Net.Http.Json;
using System.Text;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Http
{
// A persona's archive (owner decision 2026-10-07): exported in Mastodon's layout with nothing of its root, its siblings
// or its keys; imported back home without bringing anything back, or into another persona as posts nobody is told of,
// once; Mastodon's own archives read too; and hostile ones refused.
[Trait("Category", "Integration")]
public sealed class PersonaArchiveTests : IAsyncLifetime
{
PrivaPubHost _host;
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static string Base(Persona persona) => $"/clientapi/persona/{persona.Id}/archive";
async Task<JsonObject> Status(Persona persona)
{
using var client = _host.As(persona.Root.Jwt);
return (await client.GetFromJsonAsync<JsonObject>(Base(persona), Token))!;
}
async Task<byte[]> Exported(Persona persona)
{
using var client = _host.As(persona.Root.Jwt);
Assert.Equal(HttpStatusCode.Accepted, (await client.PostAsync(Base(persona), null, Token)).StatusCode);
await _host.Run(j => j.Kind == JobKind.ExportArchive && j.Payload == persona.Id, Token);
Assert.Equal("ready", (await Status(persona))["exportState"]!.GetValue<string>());
var ticket = (await (await client.PostAsync(Base(persona) + "/ticket", null, Token)).Content.ReadFromJsonAsync<JsonObject>(Token))!;
using var anonymous = _host.Client();
var response = await anonymous.GetAsync(ticket["url"]!.GetValue<string>(), Token);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
return await response.Content.ReadAsByteArrayAsync(Token);
}
// uploaded in two pieces, then imported with the parts asked for, the job run: the import's state
async Task<(HttpStatusCode Status, JsonObject State)> Imported(Persona persona, byte[] archive, params string[] parts)
{
using var client = _host.As(persona.Root.Jwt);
var upload = (await (await client.PostAsync("/clientapi/persona/archive/uploads", null, Token)).Content.ReadFromJsonAsync<JsonObject>(Token))!["id"]!.GetValue<string>();
var half = archive.Length / 2;
Assert.Equal(HttpStatusCode.OK, (await client.PutAsync($"/clientapi/persona/archive/uploads/{upload}?offset=0", new ByteArrayContent(archive[..half]), Token)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await client.PutAsync($"/clientapi/persona/archive/uploads/{upload}?offset={half}", new ByteArrayContent(archive[half..]), Token)).StatusCode);
var asked = await client.PostAsJsonAsync(Base(persona) + "/import", new { uploadId = upload, parts }, Token);
if (asked.StatusCode != HttpStatusCode.Accepted)
return (asked.StatusCode, default);
await _host.Run(j => j.Kind == JobKind.ImportArchive && j.Payload == persona.Id, Token);
return (asked.StatusCode, await Status(persona));
}
static int Count(JsonObject state, string what) => state["importCounts"]?[what]?.GetValue<int>() ?? 0;
static Dictionary<string, string> Texts(byte[] zip)
{
using var archive = new ZipArchive(new MemoryStream(zip));
return archive.Entries.Where(e => e.Name.EndsWith(".json") || e.Name.EndsWith(".csv"))
.ToDictionary(e => e.FullName, e => new StreamReader(e.Open()).ReadToEnd());
}
[Fact]
public async Task An_archive_holds_the_persona_and_nothing_of_its_root_its_siblings_or_its_keys()
{
var alice = await _host.Mastodon("alice");
var sibling = await _host.Persona(alice.Persona.Root, "sibling");
var bob = await _host.Mastodon("bob");
var picture = (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media")
{
Content = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(64, 48), "image/jpeg", "dot.jpg"))
})).Ok().Body.Text("id");
var first = await alice.Status("a public post with a picture #archives", ("media_ids[]", picture));
await alice.Status("for my followers only", ("visibility", "private"));
await alice.Status("a reply to myself", ("in_reply_to_id", first.Text("id")));
await alice.Client.Post($"/api/v1/statuses/{first.Text("id")}/bookmark");
await alice.Client.Post($"/api/v1/statuses/{first.Text("id")}/pin");
await alice.Client.Post($"/api/v1/accounts/{bob.Persona.Id}/block");
await alice.Client.Post("/api/v1/domain_blocks", ("domain", "spam.example"));
await alice.Client.Post("/api/v1/tags/archives/follow");
var zip = await Exported(alice.Persona);
var texts = Texts(zip);
foreach (var file in new[] { "actor.json", "outbox.json", "likes.json", "bookmarks.json", "following_accounts.csv", "blocked_accounts.csv",
"muted_accounts.csv", "blocked_domains.csv", "lists.csv", "bookmarks.csv", "privapub/archive.json", "privapub/pins.json",
"privapub/followed_tags.json", "privapub/filters.json", "privapub/notification_policy.json" })
Assert.True(texts.ContainsKey(file), $"{file} is missing");
using (var archive = new ZipArchive(new MemoryStream(zip)))
Assert.Contains(archive.Entries, e => e.FullName.StartsWith("media_attachments/files/", StringComparison.Ordinal));
var outbox = JsonNode.Parse(texts["outbox.json"])!["orderedItems"]!.AsArray();
Assert.Equal(3, outbox.Count);
Assert.Contains(outbox, i => i!["object"]!["attachment"]?[0]?["url"]?.GetValue<string>().StartsWith("/media_attachments/files/") == true);
Assert.Contains(bob.Persona.UserName, texts["blocked_accounts.csv"]);
Assert.Contains("spam.example", texts["blocked_domains.csv"]);
Assert.Contains(first.Text("uri"), texts["privapub/pins.json"]);
var avatar = await DB.Default.Find<Avatar>().MatchID(alice.Persona.Id).ExecuteFirstAsync(Token);
var root = await DB.Default.Find<RootUser>().MatchID(alice.Persona.Root.Id).ExecuteFirstAsync(Token);
var all = string.Concat(texts.Values);
Assert.DoesNotContain(avatar.PrivateKey.Split('\n')[1], all);
Assert.DoesNotContain(avatar.SigningKey, all);
Assert.DoesNotContain(root.UserName, all);
Assert.DoesNotContain(root.HashedPassword, all);
Assert.DoesNotContain(sibling.UserName, all);
Assert.DoesNotContain(alice.Token, all);
}
[Fact]
public async Task Back_home_an_archive_brings_nothing_back()
{
var alice = await _host.Mastodon("alice");
await alice.Status("kept");
var regretted = await alice.Status("regretted");
var zip = await Exported(alice.Persona);
await alice.Client.Delete($"/api/v1/statuses/{regretted.Text("id")}");
var before = await DB.Default.CountAsync<PostEntity>(p => p.GroupUserId == alice.Persona.Id, Token);
var (status, state) = await Imported(alice.Persona, zip, "posts", "pins");
Assert.Equal(HttpStatusCode.Accepted, status);
Assert.Equal("done", state["importState"]!.GetValue<string>());
Assert.True(state["importSameActor"]!.GetValue<bool>());
Assert.Equal((0, 1, 1), (Count(state, "posts"), Count(state, "posts already here"), Count(state, "posts deleted here")));
Assert.Equal(before, await DB.Default.CountAsync<PostEntity>(p => p.GroupUserId == alice.Persona.Id, Token));
Assert.NotNull((await DB.Default.Find<PostEntity>().MatchID(regretted.Text("id")).ExecuteFirstAsync(Token)).DeletedAt);
}
[Fact]
public async Task Into_another_persona_posts_arrive_once_and_nobody_is_told()
{
var alice = await _host.Mastodon("alice");
var follower = await _host.Mastodon("follower");
var picture = (await alice.Client.Exchange(new HttpRequestMessage(HttpMethod.Post, "/api/v2/media")
{
Content = MastodonHelpers.Multipart(("file", MastodonHelpers.JpegWithMetadata(64, 48), "image/jpeg", "dot.jpg"))
})).Ok().Body.Text("id");
var first = await alice.Status("the first words #moving", ("media_ids[]", picture));
await alice.Status("and a reply to them", ("in_reply_to_id", first.Text("id")));
await _host.Located(alice, "here and nowhere else");
var zip = await Exported(alice.Persona);
var carol = await _host.Mastodon("carol");
await follower.Client.Post($"/api/v1/accounts/{carol.Persona.Id}/follow");
var actor = $"/peasants/{carol.Persona.UserName}";
var jobs = await DB.Default.CountAsync<Job>(j => j.Kind == JobKind.Deliver && j.Payload.Contains(actor), Token);
var (_, state) = await Imported(carol.Persona, zip, "posts", "pins", "tags", "located");
Assert.Equal("done", state["importState"]!.GetValue<string>());
Assert.False(state["importSameActor"]!.GetValue<bool>());
Assert.Equal(2, Count(state, "posts"));
Assert.Equal(1, Count(state, "media"));
Assert.Equal(1, Count(state, "located posts"));
var copies = await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == carol.Persona.Id && p.Visibility != Models.Post.PostVisibility.LocalGeo).ExecuteAsync(Token);
Assert.Equal(2, copies.Count);
var copy = copies.Single(p => p.ImportedFromURI == first.Text("uri"));
Assert.StartsWith($"{PrivaPubHost.Base}/peasants/{carol.Persona.UserName}/scribbles/", copy.ObjectURI);
Assert.Equal(["moving"], copy.Tags);
Assert.Single(copy.Media);
Assert.Equal(carol.Persona.Id, (await DB.Default.Find<MediaAttachment>().MatchID(copy.Media[0].AttachmentId).ExecuteFirstAsync(Token)).OwnerAvatarId);
var reply = copies.Single(p => p.ID != copy.ID);
Assert.Equal(copy.ID, reply.AnsweringToPostId);
Assert.Equal(copy.ObjectURI + "/context", reply.ContextURI);
// nobody told: no delivery, no home, no notification; on the profile all the same
Assert.Equal(jobs, await DB.Default.CountAsync<Job>(j => j.Kind == JobKind.Deliver && j.Payload.Contains(actor), Token));
var ids = copies.Select(p => p.ID).ToList();
Assert.False(await DB.Default.Find<TimelineEntry>().Match(e => ids.Contains(e.PostId)).ExecuteAnyAsync(Token));
Assert.Contains((await carol.Client.Get($"/api/v1/accounts/{carol.Persona.Id}/statuses")).Ok().Body!.AsArray(), s => s!.Text("id") == copy.ID);
// twice changes nothing
var (_, again) = await Imported(carol.Persona, zip, "posts", "located");
Assert.Equal((0, 2, 1), (Count(again, "posts"), Count(again, "posts already here"), Count(again, "located posts already here")));
Assert.Equal(3, await DB.Default.CountAsync<PostEntity>(p => p.GroupUserId == carol.Persona.Id, Token));
}
// Mastodon's own archive: its posts, not its boosts nor its direct messages
[Fact]
public async Task A_mastodon_archive_is_read()
{
var dave = await _host.Mastodon("dave");
const string actor = "https://mastodon.example/users/dave";
const string Public = "https://www.w3.org/ns/activitystreams#Public";
var outbox = new JsonObject
{
["@context"] = "https://www.w3.org/ns/activitystreams",
["id"] = "outbox.json",
["type"] = "OrderedCollection",
["orderedItems"] = new JsonArray(
Create(actor, "1", "<p>hello from <b>Mastodon</b><script>alert(1)</script></p>", [Public], [actor + "/followers"], "2023-04-01T10:00:00Z"),
Create(actor, "2", "<p>only for one</p>", ["https://elsewhere.example/users/x"], [], "2023-04-02T10:00:00Z"),
new JsonObject { ["type"] = "Announce", ["actor"] = actor, ["object"] = "https://elsewhere.example/notes/9" })
};
var zip = Zip(("actor.json", new JsonObject { ["id"] = actor, ["type"] = "Person", ["followers"] = actor + "/followers" }.ToJsonString()),
("outbox.json", outbox.ToJsonString()));
var (_, state) = await Imported(dave.Persona, zip, "posts");
Assert.Equal("done", state["importState"]!.GetValue<string>());
Assert.Equal((1, 1, 1), (Count(state, "posts"), Count(state, "boosts skipped"), Count(state, "direct or circle posts skipped")));
var post = await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == dave.Persona.Id).ExecuteFirstAsync(Token);
Assert.Equal(actor + "/statuses/1", post.ImportedFromURI);
Assert.Equal(new DateTime(2023, 4, 1, 10, 0, 0, DateTimeKind.Utc), post.CreationDate);
Assert.DoesNotContain("<script", post.ContentHtml);
Assert.Contains("Mastodon", post.ContentHtml);
}
static JsonObject Create(string actor, string id, string content, string[] to, string[] cc, string published) => new()
{
["type"] = "Create",
["actor"] = actor,
["object"] = new JsonObject
{
["id"] = $"{actor}/statuses/{id}",
["type"] = "Note",
["attributedTo"] = actor,
["content"] = content,
["published"] = published,
["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray()),
["cc"] = new JsonArray(cc.Select(c => (JsonNode)c).ToArray())
}
};
static byte[] Zip(params (string Name, string Text)[] files)
{
var stream = new MemoryStream();
using (var archive = new ZipArchive(stream, ZipArchiveMode.Create, leaveOpen: true))
foreach (var (name, text) in files)
{
using var entry = archive.CreateEntry(name).Open();
entry.Write(Encoding.UTF8.GetBytes(text));
}
return stream.ToArray();
}
[Theory]
[InlineData("escape")]
[InlineData("twice")]
[InlineData("bomb")]
[InlineData("link")]
[InlineData("junk")]
public async Task A_hostile_archive_is_refused(string how)
{
var erin = await _host.Mastodon("erin");
var actorJson = new JsonObject { ["id"] = "https://elsewhere.example/users/erin" }.ToJsonString();
var zip = how switch
{
"escape" => Zip(("actor.json", actorJson), ("../escape.txt", "x")),
"twice" => Zip(("actor.json", actorJson), ("outbox.json", "{}"), ("outbox.json", "{}")),
"bomb" => Zip(("actor.json", actorJson), ("outbox.json", new string(' ', 30 * 1024 * 1024))),
"link" => Link(actorJson),
_ => "not a zip at all"u8.ToArray()
};
var (status, _) = await Imported(erin.Persona, zip, "posts");
Assert.Equal(HttpStatusCode.UnprocessableEntity, status);
Assert.False(await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == erin.Persona.Id).ExecuteAnyAsync(Token));
}
// a symbolic link, as zip on Unix stores one
static byte[] Link(string actorJson)
{
var stream = new MemoryStream();
using (var archive = new ZipArchive(stream, ZipArchiveMode.Create, leaveOpen: true))
{
using (var actor = archive.CreateEntry("actor.json").Open())
actor.Write(Encoding.UTF8.GetBytes(actorJson));
var link = archive.CreateEntry("media_attachments/files/passwd");
link.ExternalAttributes = unchecked((int)(0xA1FFu << 16));
using var target = link.Open();
target.Write("/etc/passwd"u8);
}
return stream.ToArray();
}
[Fact]
public async Task An_outbox_item_too_large_stops_the_import()
{
var frank = await _host.Mastodon("frank");
const string actor = "https://elsewhere.example/users/frank";
// (random, so it is no compression bomb: that is refused before the import starts)
var words = Convert.ToBase64String(System.Security.Cryptography.RandomNumberGenerator.GetBytes(1536 * 1024));
var huge = Create(actor, "1", words, ["https://www.w3.org/ns/activitystreams#Public"], [], "2024-01-01T00:00:00Z");
var zip = Zip(("actor.json", new JsonObject { ["id"] = actor }.ToJsonString()),
("outbox.json", new JsonObject { ["orderedItems"] = new JsonArray(huge) }.ToJsonString()));
var (_, state) = await Imported(frank.Persona, zip, "posts");
Assert.Equal("failed", state["importState"]!.GetValue<string>());
Assert.Contains("larger than", state["importError"]!.GetValue<string>());
}
[Fact]
public async Task Only_the_personas_own_root_and_never_a_banned_one()
{
var grace = await _host.Mastodon("grace");
var stranger = await _host.SignUp("stranger");
using (var other = _host.As(stranger.Jwt))
Assert.Equal(HttpStatusCode.NotFound, (await other.GetAsync(Base(grace.Persona), Token)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await _host.As(grace.Persona.Root.Jwt).GetAsync(Base(grace.Persona), Token)).StatusCode);
await DB.Default.Update<RootUser>().MatchID(grace.Persona.Root.Id).Modify(u => u.IsBanned, true).ExecuteAsync(Token);
Assert.NotEqual(HttpStatusCode.OK, (await _host.As(grace.Persona.Root.Jwt).GetAsync(Base(grace.Persona), Token)).StatusCode);
}
[Fact]
public async Task An_import_asked_to_stop_stops()
{
var henry = await _host.Mastodon("henry");
const string actor = "https://elsewhere.example/users/henry";
var zip = Zip(("actor.json", new JsonObject { ["id"] = actor }.ToJsonString()),
("outbox.json", new JsonObject { ["orderedItems"] = new JsonArray(Create(actor, "1", "one", ["https://www.w3.org/ns/activitystreams#Public"], [], "2024-01-01T00:00:00Z")) }.ToJsonString()));
using var client = _host.As(henry.Persona.Root.Jwt);
var upload = (await (await client.PostAsync("/clientapi/persona/archive/uploads", null, Token)).Content.ReadFromJsonAsync<JsonObject>(Token))!["id"]!.GetValue<string>();
await client.PutAsync($"/clientapi/persona/archive/uploads/{upload}?offset=0", new ByteArrayContent(zip), Token);
Assert.Equal(HttpStatusCode.Accepted, (await client.PostAsJsonAsync(Base(henry.Persona) + "/import", new { uploadId = upload, parts = new[] { "posts" } }, Token)).StatusCode);
Assert.Equal(HttpStatusCode.Conflict, (await client.PostAsJsonAsync(Base(henry.Persona) + "/import", new { uploadId = upload, parts = new[] { "posts" } }, Token)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await client.PostAsync(Base(henry.Persona) + "/import/stop", null, Token)).StatusCode);
await _host.Run(j => j.Kind == JobKind.ImportArchive && j.Payload == henry.Persona.Id, Token);
Assert.Equal("stopped", (await Status(henry.Persona))["importState"]!.GetValue<string>());
Assert.False(await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == henry.Persona.Id).ExecuteAnyAsync(Token));
}
}
}
+218
View File
@@ -0,0 +1,218 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.IdentityModel.JsonWebTokens;
using Microsoft.IdentityModel.Tokens;
using PrivaPub.Api.Mastodon.Auth;
using PrivaPub.Services;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Security.Claims;
using System.Text;
namespace PrivaPub.Tests.Http
{
// The first-party client's one sign-in: the /clientapi JWT exchanged for one persona's Mastodon token (PersonaExchange).
[Trait("Category", "Integration")]
public sealed class PersonaExchangeTests : IAsyncLifetime
{
const string Grant = "urn:ietf:params:oauth:grant-type:token-exchange";
const string FirstParty = "decepub";
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
async Task<HttpResponseMessage> Exchange(string jwt, string avatarId, string clientId = FirstParty, string clientSecret = default,
string subjectTokenType = PersonaExchange.SubjectTokenType)
{
var fields = new List<(string, string)>
{
("grant_type", Grant),
("client_id", clientId),
("subject_token", jwt),
("subject_token_type", subjectTokenType),
(PersonaExchange.AvatarParameter, avatarId),
("scope", "read write follow")
};
if (clientSecret != default)
fields.Add(("client_secret", clientSecret));
using var client = _host.Client();
return await client.Form("/oauth/token", fields.Where(f => f.Item2 != default).ToArray());
}
async Task<string> Token(Persona persona)
{
var response = await Exchange(persona.Root.Jwt, persona.Id);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
return (await response.JsonBody())["access_token"]!.GetValue<string>();
}
static async Task AssertRefused(HttpResponseMessage response, string error = "invalid_grant")
{
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.Equal(error, (await response.JsonBody())["error"]?.GetValue<string>());
}
async Task<HttpStatusCode> Me(string token)
{
using var api = _host.As(token);
return (await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode;
}
static string Jwt(string rootId, string key, DateTime expires) => new JsonWebTokenHandler().CreateToken(new SecurityTokenDescriptor
{
Issuer = PrivaPubHost.Base,
Audience = PrivaPubHost.Base,
Subject = new ClaimsIdentity(new[] { new Claim(ClaimTypes.UserData, rootId) }),
NotBefore = expires.AddHours(-2),
IssuedAt = expires.AddHours(-2),
Expires = expires,
SigningCredentials = new(new SymmetricSecurityKey(Encoding.UTF8.GetBytes(key)), SecurityAlgorithms.HmacSha512)
});
[Fact]
public async Task The_root_jwt_becomes_one_personas_token_that_never_names_the_root()
{
var persona = await _host.Persona(await _host.SignUp(), "exchange");
var root = persona.Root;
var response = await Exchange(root.Jwt, persona.Id);
var body = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.DoesNotContain(root.Id, body);
Assert.DoesNotContain(root.UserName, body);
var token = (await response.JsonBody())["access_token"]!.GetValue<string>();
using var api = _host.As(token);
var account = await api.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken);
Assert.Contains($"\"id\":\"{persona.Id}\"", account);
Assert.DoesNotContain(root.Id, account);
foreach (var entry in await ClientApi.StoredTokens(FirstParty))
{
var payload = entry.Contains("payload") && entry["payload"].IsString ? ClientApi.JwtPayload(entry["payload"].AsString) : string.Empty;
Assert.DoesNotContain(root.Id, entry.ToString() + payload);
Assert.DoesNotContain(root.UserName, entry.ToString() + payload);
}
}
[Fact]
public async Task Each_persona_of_the_root_gets_its_own_token()
{
var root = await _host.SignUp();
var first = await _host.Persona(root, "first");
var second = await _host.Persona(root, "second");
using var firstApi = _host.As(await Token(first));
using var secondApi = _host.As(await Token(second));
Assert.Contains($"\"id\":\"{first.Id}\"", await firstApi.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken));
Assert.Contains($"\"id\":\"{second.Id}\"", await secondApi.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken));
}
[Fact]
public async Task Another_roots_persona_is_refused()
{
var mine = await _host.SignUp();
var theirs = await _host.Persona(await _host.SignUp(), "theirs");
await AssertRefused(await Exchange(mine.Jwt, theirs.Id));
await AssertRefused(await Exchange(mine.Jwt, "000000000000000000000000"));
await AssertRefused(await Exchange(mine.Jwt, "not an id"));
}
[Fact]
public async Task A_token_that_is_not_a_valid_root_jwt_is_refused()
{
var persona = await _host.Persona(await _host.SignUp(), "forged");
var key = _host.Services.GetRequiredService<IConfiguration>()["AppConfiguration:Jwt:Key"]!;
await AssertRefused(await Exchange("garbage", persona.Id));
await AssertRefused(await Exchange(Jwt(persona.Root.Id, key, DateTime.UtcNow.AddMinutes(-10)), persona.Id));
await AssertRefused(await Exchange(Jwt(persona.Root.Id, new string('k', 64), DateTime.UtcNow.AddHours(1)), persona.Id));
}
[Fact]
public async Task Ended_sessions_refuse_the_jwt_and_revoke_the_exchanged_tokens()
{
var persona = await _host.Persona(await _host.SignUp(), "ended");
var token = await Token(persona);
Assert.Equal(HttpStatusCode.OK, await Me(token));
using (var scope = _host.Services.CreateScope())
await scope.ServiceProvider.GetRequiredService<IRootSessions>().Revoke(persona.Root.Id, TestContext.Current.CancellationToken);
await AssertRefused(await Exchange(persona.Root.Jwt, persona.Id));
Assert.Equal(HttpStatusCode.Unauthorized, await Me(token));
}
[Fact]
public async Task A_banned_root_is_refused()
{
var persona = await _host.Persona(await _host.SignUp(), "banned");
await ClientApi.Ban(persona.Root.Id);
try
{
await AssertRefused(await Exchange(persona.Root.Jwt, persona.Id));
}
finally
{
await ClientApi.Ban(persona.Root.Id, banned: false);
}
}
[Fact]
public async Task A_revoked_token_no_longer_works()
{
var persona = await _host.Persona(await _host.SignUp(), "revoked");
var token = await Token(persona);
using var client = _host.Client();
var revoked = await client.Form("/oauth/revoke", ("token", token), ("client_id", FirstParty));
Assert.Equal(HttpStatusCode.OK, revoked.StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, await Me(token));
}
[Fact]
public async Task Only_the_first_party_client_may_exchange()
{
var persona = await _host.Persona(await _host.SignUp(), "thirdparty");
using var client = _host.Client();
var app = await client.RegisterApp();
var response = await Exchange(persona.Root.Jwt, persona.Id, app.ClientId, app.ClientSecret);
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
Assert.Equal("unauthorized_client", (await response.JsonBody())["error"]?.GetValue<string>());
}
[Fact]
public async Task A_mastodon_token_is_not_a_subject_token()
{
var persona = await _host.Persona(await _host.SignUp(), "mastodon");
var token = await Token(persona);
await AssertRefused(await Exchange(token, persona.Id, subjectTokenType: "urn:ietf:params:oauth:token-type:access_token"), "invalid_request");
await AssertRefused(await Exchange(token, persona.Id));
}
[Fact]
public async Task Missing_parameters_are_a_client_error()
{
var persona = await _host.Persona(await _host.SignUp(), "missing");
await AssertRefused(await Exchange(persona.Root.Jwt, default));
await AssertRefused(await Exchange(default, persona.Id), "invalid_request");
await AssertRefused(await Exchange(persona.Root.Jwt, persona.Id, subjectTokenType: default), "invalid_request");
}
}
}
@@ -0,0 +1,261 @@
using Microsoft.AspNetCore.Routing;
using Microsoft.AspNetCore.Routing.Patterns;
using Microsoft.Extensions.DependencyInjection;
using MongoDB.Entities;
using PrivaPub.ClientModels.Social;
using PrivaPub.Domain.Social;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text;
using System.Text.Json.Nodes;
using System.Text.RegularExpressions;
using static PrivaPub.Tests.Support.Host.FederationHelpers;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed partial class PersonaSeparationHttpTests : IAsyncLifetime
{
const string Base = PrivaPubHost.Base;
const int MaxDocuments = 300;
PrivaPubHost _host;
Root _root;
Persona _alpha;
Persona _bravo;
Persona _charlie;
LocalGroup _community;
string _tag;
Post _shown;
Post _poll;
List<Post> _alphaPosts;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
_host = await PrivaPubHost.Shared();
_root = await _host.SignUp("sep");
_alpha = await _host.Persona(_root, "sepalpha");
_bravo = await _host.Persona(_root, "sepbravo");
_charlie = await _host.Persona(await _host.SignUp(), "sepcharlie");
_community = await _host.FederatedGroup(_bravo, community: true, "sepcommunity");
_tag = $"septag{Guid.NewGuid():N}"[..14];
await _host.Publish(_bravo, "the sibling's own words", PostVisibility.FollowersOnly);
var follows = _host.Get<IFollowService>();
Assert.True((await follows.Follow(_root.Id, new FollowForm { AvatarId = _bravo.Id, Target = _charlie.UserName }, token)).IsValid);
_shown = await _host.Publish(_alpha, $"hello from alpha #{_tag}");
_poll = await _host.Publish(_alpha, new StatusDraft
{
Text = "which one?",
PlainText = true,
Poll = new PollDraft(new[] { "this", "that" }, 3600, false, false)
});
var charlieSays = await _host.Publish(_charlie, "worth sharing");
_alphaPosts = new List<Post>
{
_shown,
_poll,
await _host.Publish(_alpha, "quietly", PostVisibility.Unlisted),
await _host.Publish(_alpha, "for my followers", PostVisibility.FollowersOnly),
await _host.Publish(_alpha, $"@{_charlie.UserName} between us", PostVisibility.Direct),
await _host.PublishLocated(_alpha, "around here"),
await _host.Publish(_alpha, new StatusDraft { Text = "quoting charlie", PlainText = true, QuotedStatusId = charlieSays.ID }),
await _host.Reblog(_alpha, charlieSays)
};
await DB.Default.SaveAsync(new Pin { AvatarId = _alpha.Id, PostId = _shown.ID }, token);
Assert.True((await follows.Follow(_root.Id, new FollowForm { AvatarId = _alpha.Id, Target = _charlie.UserName }, token)).IsValid);
Assert.True((await follows.Follow(_charlie.Root.Id, new FollowForm { AvatarId = _charlie.Id, Target = _alpha.UserName }, token)).IsValid);
Assert.True((await _host.Get<IStatusService>().Favourite(await _host.Actor(_charlie), _shown.ID, true, token)).Ok);
await _host.Publish(_charlie, new StatusDraft { Text = $"@{_alpha.UserName} nice", PlainText = true, InReplyTo = _shown.ID });
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
string[] Forbidden() => new[] { _bravo.Id, _bravo.UserName, _root.Id, _root.UserName, _community.Id, _community.UserName };
[Fact]
public async Task Nothing_the_api_tells_a_persona_names_its_sibling_or_its_login()
{
var bearer = await _host.MastodonToken(_alpha);
using var client = _host.As(bearer);
var notifications = await client.Fetch("/api/v1/notifications", "application/json");
Assert.Equal(HttpStatusCode.OK, notifications.Status);
var notificationIds = JsonNode.Parse(notifications.Text)!.AsArray().Select(n => n!["id"]!.GetValue<string>()).ToList();
Assert.NotEmpty(notificationIds);
var paths = Routes().Select(Fill).Concat(new[]
{
$"/api/v2/search?q={_alpha.UserName}",
$"/api/v2/search?q={Uri.EscapeDataString($"@{_alpha.UserName}@{PrivaPubHost.Host}")}",
$"/api/v2/search?q={Uri.EscapeDataString(_alpha.ActorUri())}",
$"/api/v2/search?q={Uri.EscapeDataString(_shown.ObjectURI)}",
$"/api/v2/search?q=%23{_tag}",
$"/api/v1/accounts/search?q={_alpha.UserName}",
$"/api/v1/accounts/lookup?acct={_alpha.UserName}",
$"/api/v1/accounts/relationships?id[]={_alpha.Id}&id[]={_charlie.Id}",
$"/api/v1/accounts/familiar_followers?id[]={_charlie.Id}",
$"/api/v1/accounts/{_alpha.Id}/statuses?pinned=true",
$"/api/v1/accounts/{_alpha.Id}/statuses?exclude_replies=true&limit=40",
$"/api/v1/statuses?{string.Join("&", _alphaPosts.Select(p => "id[]=" + p.ID))}",
$"/api/v1/statuses/{_poll.ID}",
$"/api/v1/polls/{_poll.ID}",
$"/api/v1/notifications/{notificationIds[0]}",
"/api/v1/timelines/home?limit=40",
"/api/v1/timelines/public?local=true&limit=40"
}).Concat(_alphaPosts.Select(p => $"/api/v1/statuses/{p.ID}/context"))
.Distinct()
.ToList();
var answered = 0;
var failures = new List<string>();
foreach (var path in paths)
{
var fetched = await client.Fetch(path, "application/json");
if ((int)fetched.Status >= 500)
failures.Add($"{path} answered {(int)fetched.Status}");
if (fetched.Status == HttpStatusCode.OK)
answered++;
foreach (var secret in Forbidden())
if (fetched.Text.Contains(secret, StringComparison.OrdinalIgnoreCase) || fetched.Response.Headers.ToString().Contains(secret, StringComparison.OrdinalIgnoreCase))
failures.Add($"{path} names {secret}: {Short(fetched.Text)}");
}
Assert.True(failures.Count == 0, string.Join("\n", failures));
Assert.True(answered >= 40, $"only {answered} of {paths.Count} answers were 200");
var me = await client.Fetch("/api/v1/accounts/verify_credentials", "application/json");
Assert.Contains(_alpha.Id, me.Text);
Assert.Contains(_shown.ID, (await client.Fetch("/api/v1/timelines/home?limit=40", "application/json")).Text);
}
[Fact]
public async Task Nothing_published_about_a_persona_names_its_sibling_or_its_login()
{
using var client = _host.Client();
var seeds = new[]
{
$"/peasants/{_alpha.UserName}",
$"/users/{_alpha.UserName}",
$"/peasants/{_alpha.UserName}/anus?page=true",
$"/peasants/{_alpha.UserName}/scribbles/{_shown.ID}",
$"/peasants/{_alpha.UserName}/grunts/create-{_shown.ID}",
$"/.well-known/webfinger?resource=acct:{_alpha.UserName}@{PrivaPubHost.Host}",
"/.well-known/nodeinfo",
$"/@{_alpha.UserName}",
$"/@{_alpha.UserName}/{_shown.ID}"
};
var queue = new Queue<string>(seeds);
var seen = new HashSet<string>(seeds);
var failures = new List<string>();
var fetchedOk = 0;
while (queue.Count > 0 && seen.Count <= MaxDocuments)
{
var path = queue.Dequeue();
var html = path.StartsWith("/@", StringComparison.Ordinal);
var fetched = await client.Fetch(path, html ? Browser : path.StartsWith("/peasants/", StringComparison.Ordinal) ? ActivityJson : "application/json");
if ((int)fetched.Status >= 500)
failures.Add($"{path} answered {(int)fetched.Status}");
if (fetched.Status == HttpStatusCode.OK)
fetchedOk++;
foreach (var secret in Forbidden())
if (fetched.Text.Contains(secret, StringComparison.OrdinalIgnoreCase))
failures.Add($"{path} names {secret}: {Short(fetched.Text)}");
var links = Links(fetched.Text).ToList();
if (fetched.Location != default)
links.Add(fetched.Location.StartsWith('/') ? Base + fetched.Location : fetched.Location);
foreach (var link in links)
{
if (!link.StartsWith(Base + "/", StringComparison.Ordinal))
continue;
var next = PathOf(link.Split('#')[0]);
if (Followed(next) && seen.Add(next))
queue.Enqueue(next);
}
}
Assert.True(failures.Count == 0, string.Join("\n", failures));
Assert.Contains($"/peasants/{_alpha.UserName}/groupies", seen);
Assert.Contains($"/peasants/{_alpha.UserName}/trophies", seen);
Assert.Contains("/nodeinfo/2.1", seen);
Assert.True(fetchedOk >= 15, $"only {fetchedOk} of {seen.Count} documents were served");
}
bool Followed(string path) =>
path.StartsWith($"/peasants/{_alpha.UserName}", StringComparison.Ordinal)
|| path.StartsWith($"/@{_alpha.UserName}", StringComparison.Ordinal)
|| path.StartsWith("/.well-known/", StringComparison.Ordinal)
|| path.StartsWith("/nodeinfo/", StringComparison.Ordinal);
static IEnumerable<string> Links(string text)
{
JsonNode json = default;
try
{
json = JsonNode.Parse(text);
}
catch (System.Text.Json.JsonException)
{
}
if (json != default)
return Strings(json);
return Href().Matches(text).Select(m => WebUtility.HtmlDecode(m.Groups[1].Value));
}
IEnumerable<string> Routes()
{
var endpoints = _host.Services.GetRequiredService<EndpointDataSource>().Endpoints.OfType<RouteEndpoint>();
foreach (var endpoint in endpoints)
{
var methods = endpoint.Metadata.GetMetadata<IHttpMethodMetadata>()?.HttpMethods;
var template = "/" + endpoint.RoutePattern.RawText?.TrimStart('/');
if (template.StartsWith("/api/", StringComparison.Ordinal) && methods is { } verbs && verbs.Contains("GET"))
yield return template;
}
}
string Fill(string template)
{
var builder = new StringBuilder();
foreach (var segment in RoutePatternFactory.Parse(template).PathSegments)
{
builder.Append('/');
foreach (var part in segment.Parts)
builder.Append(part switch
{
RoutePatternLiteralPart literal => literal.Content,
RoutePatternSeparatorPart separator => separator.Content,
RoutePatternParameterPart parameter => Value(template, parameter.Name),
_ => string.Empty
});
}
return builder.ToString();
}
string Value(string template, string parameter) => parameter switch
{
"hashtag" => _tag,
"host" => PrivaPubHost.Host,
_ when template.StartsWith("/api/v1/statuses/", StringComparison.Ordinal)
|| template.Contains("/statuses/{id}", StringComparison.Ordinal) => _shown.ID,
_ when template.StartsWith("/api/v1/polls/", StringComparison.Ordinal) => _poll.ID,
_ => _alpha.Id
};
static string Short(string text) => text.Length > 400 ? text[..400] + "…" : text;
[GeneratedRegex("(?:href|src)=\"([^\"]+)\"")]
private static partial Regex Href();
}
}
+106
View File
@@ -0,0 +1,106 @@
using MongoDB.Entities;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// A Pixelfed post's own place: stored from `location`, shown as `privapub.place`, changed by an edit.
[Trait("Category", "Integration")]
public sealed class PostPlaceTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
static JsonObject Location(string name, string latitude, string longitude) => new()
{
["type"] = "Place", ["name"] = name, ["latitude"] = latitude, ["longitude"] = longitude, ["country"] = "Italy"
};
[Fact]
public async Task A_pixelfed_place_is_shown_and_follows_the_authors_edits()
{
var alice = await _host.Mastodon("alice");
var ann = new RemoteActor(_peer, "ann");
var post = await _host.PublicPostFrom(ann, alice, "<p>the duomo</p>", note => note["location"] = Location("Milan", "45.4642", "9.19"));
var place = (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"];
Assert.Equal(("Milan", "Italy"), (place.Text("name"), place.Text("country")));
Assert.Equal(45.4642, place["latitude"]!.GetValue<double>());
Assert.Equal(9.19, place["longitude"]!.GetValue<double>());
await Edit(ann, alice, post, note => note["location"] = Location("Turin", "45.0703", "7.6869"));
Assert.Equal("Turin", (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"].Text("name"));
await Edit(ann, alice, post, _ => { });
Assert.Null((await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"]);
Assert.Null((await DB.Default.Find<Post>().MatchID(post.ID).ExecuteFirstAsync(Token)).Place);
}
[Fact]
public async Task A_post_without_a_place_has_none()
{
var alice = await _host.Mastodon("alice");
var post = await _host.PublicPostFrom(new RemoteActor(_peer, "plain"), alice);
Assert.Null((await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"]);
}
[Fact]
public async Task A_local_post_never_carries_a_location()
{
var alice = await _host.Mastodon("alice");
var status = await alice.Status("made here");
using var client = _host.Client();
client.DefaultRequestHeaders.Accept.ParseAdd("application/activity+json");
var note = JsonNode.Parse(await client.GetStringAsync(status.Text("uri"), Token))!;
Assert.Null(note["location"]);
}
async Task Edit(RemoteActor author, Mastodon mentioned, Post post, Action<JsonObject> shape)
{
var note = new JsonObject
{
["id"] = post.ObjectURI,
["type"] = "Note",
["attributedTo"] = author.Id,
["to"] = new JsonArray(MastodonHelpers.Public),
["cc"] = new JsonArray(mentioned.Uri),
["content"] = "<p>the duomo, again</p>",
["published"] = post.CreationDate.ToString("O"),
["updated"] = DateTime.UtcNow.ToString("O"),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = mentioned.Uri, ["name"] = "@" + mentioned.UserName })
};
shape(note);
await _host.Deliver(author, mentioned.Mouth, new JsonObject
{
["id"] = $"{post.ObjectURI}/updates/{Guid.NewGuid():N}",
["type"] = "Update",
["actor"] = author.Id,
["to"] = new JsonArray(MastodonHelpers.Public),
["object"] = note
});
}
}
}
@@ -0,0 +1,224 @@
using MongoDB.Entities;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class ProvenanceAndReactionsTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
static CancellationToken Token => TestContext.Current.CancellationToken;
static string Provenance(string id) => $"/api/privapub/v1/statuses/{id}/provenance";
[Fact]
public async Task A_local_post_has_a_local_provenance()
{
var alice = await _host.Mastodon("alice");
var status = await alice.Status("made here");
var provenance = (await _host.Client().Get(Provenance(status.Text("id")))).Ok();
Assert.Equal("local", provenance.Body.Text("path"));
Assert.Equal(status.Text("uri"), provenance.Body.Text("object_uri"));
Assert.Equal(status.Text("url"), provenance.Body.Text("url"));
Assert.Null(provenance.Body["signature"]);
Assert.Null(provenance.Body["raw"]);
}
[Fact]
public async Task A_delivered_post_shows_its_signature_and_raw_form_and_reading_it_asks_nobody()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var create = bob.Create("<p>signed and sealed</p>", new[] { MastodonHelpers.Public }, new[] { alice.Uri },
note => note["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@" + alice.UserName }));
await _host.Deliver(bob, alice.Mouth, create);
var noteId = create["object"].Text("id");
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(Token);
var asked = _peer.Requests.Count;
var provenance = (await alice.Client.Get(Provenance(post.ID))).Ok().Body;
var anonymous = (await _host.Client().Get(Provenance(post.ID))).Ok().Body;
Assert.Equal("delivered", provenance.Text("path"));
Assert.Equal(noteId, provenance.Text("object_uri"));
Assert.Equal("Note", provenance.Text("object_type"));
Assert.Null(provenance["fetched_by"]);
Assert.Equal((create.Text("id"), "Create", bob.Id), (provenance["activity"].Text("id"), provenance["activity"].Text("type"), provenance["activity"].Text("actor")));
Assert.Equal(bob.KeyId, provenance["signature"].Text("key_id"));
Assert.Equal("rsa-sha256", provenance["signature"].Text("algorithm"));
Assert.Contains("digest", provenance["signature"]!["headers"]!.AsArray().Select(h => h!.GetValue<string>()));
Assert.Equal(noteId, provenance["raw"].Text("id"));
Assert.False(string.IsNullOrEmpty(provenance.Text("raw_hash")));
Assert.True(provenance.Number("raw_bytes") > 0);
Assert.NotNull(provenance.Text("received_at"));
Assert.Equal(provenance.ToJsonString(), anonymous.ToJsonString());
Assert.Equal(asked, _peer.Requests.Count);
}
[Fact]
public async Task A_fetched_parent_shows_no_signature_and_was_fetched_by_the_instance_actor()
{
var alice = await _host.Mastodon("alice");
var bob = new RemoteActor(_peer, "bob");
var parentPath = $"/notes/{Guid.NewGuid():N}";
var parentId = _peer.A + parentPath;
_peer.Serve(parentPath, new JsonObject
{
["id"] = parentId,
["type"] = "Note",
["attributedTo"] = bob.Id,
["to"] = new JsonArray(MastodonHelpers.Public),
["content"] = "<p>the start of it</p>",
["published"] = DateTime.UtcNow.AddMinutes(-5).ToString("O")
}.ToJsonString());
var reply = bob.Create("<p>and a reply</p>", new[] { MastodonHelpers.Public }, new[] { alice.Uri }, note =>
{
note["inReplyTo"] = parentId;
note["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@" + alice.UserName });
});
await _host.Deliver(bob, alice.Mouth, reply);
var parent = await DB.Default.Find<Post>().Match(p => p.ObjectURI == parentId).ExecuteFirstAsync(Token);
Assert.NotNull(parent);
var asked = _peer.Requests.Count;
var provenance = (await alice.Client.Get(Provenance(parent.ID))).Ok().Body;
Assert.Equal("fetched", provenance.Text("path"));
Assert.Equal("instance-actor", provenance.Text("fetched_by"));
Assert.Null(provenance["signature"]);
Assert.Equal((reply.Text("id"), "Create"), (provenance["activity"].Text("id"), provenance["activity"].Text("type")));
Assert.Null(provenance["inbox"]);
Assert.Equal(parentId, provenance["raw"].Text("id"));
Assert.Contains(_peer.Requests, r => r.Path == parentPath && r.Signature.Contains("/peasants/privapub#main-key"));
Assert.Equal(asked, _peer.Requests.Count);
}
[Fact]
public async Task Provenance_follows_who_may_see_the_post()
{
var alice = await _host.Mastodon("alice");
var other = await _host.Mastodon("other");
var bob = new RemoteActor(_peer, "bob");
var direct = bob.Create("<p>between us</p>", new[] { alice.Uri }, shape: note =>
note["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@" + alice.UserName }));
await _host.Deliver(bob, alice.Mouth, direct);
var noteId = direct["object"].Text("id");
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(Token);
Assert.Equal("delivered", (await alice.Client.Get(Provenance(post.ID))).Ok().Body.Text("path"));
Assert.Equal(HttpStatusCode.NotFound, (await other.Client.Get(Provenance(post.ID))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get(Provenance(post.ID))).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get(Provenance("000000000000000000000000"))).Status);
}
[Fact]
public async Task An_instance_is_described_from_what_was_stored_and_reading_it_asks_nobody()
{
var host = $"described{Guid.NewGuid():N}.example";
await DB.Default.SaveAsync(new RemoteInstance
{
Host = host,
Software = "gotosocial",
SoftwareVersion = "0.20.0",
NodeName = "a test pasture",
Protocols = new List<string> { "activitypub" },
OpenRegistrations = false,
DescribedAt = DateTime.UtcNow.AddDays(-1),
NodeInfo = "{\"software\":{\"name\":\"gotosocial\",\"version\":\"0.20.0\"}}",
ConsecutiveFailures = 2,
LastFailureAt = DateTime.UtcNow.AddHours(-1)
}, Token);
var alice = await _host.Mastodon("alice");
var described = (await alice.Client.Get($"/api/privapub/v1/instances/{host.ToUpperInvariant()}")).Ok().Body;
Assert.Equal(host, described.Text("host"));
Assert.Equal(("gotosocial", "0.20.0", "a test pasture"), (described.Text("software"), described.Text("version"), described.Text("node_name")));
Assert.Equal("activitypub", Assert.Single(described["protocols"]!.AsArray())!.GetValue<string>());
Assert.False(described.Flag("open_registrations"));
Assert.Equal("gotosocial", described["node_info"]!["software"].Text("name"));
Assert.Equal(2, described["delivery"].Number("consecutive_failures"));
Assert.NotNull(described["delivery"].Text("last_failure_at"));
Assert.Null(described["delivery"]["last_success_at"]);
Assert.Equal(described.ToJsonString(), (await _host.Client().Get($"/api/privapub/v1/instances/{host}")).Ok().Body.ToJsonString());
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/instances/unknown{Guid.NewGuid():N}.example")).Status);
Assert.Empty(_peer.Requests);
}
[Fact]
public async Task Reacting_to_a_remote_post_federates_an_emoji_react_and_its_undo()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var bob = new RemoteActor(_peer, "bob");
var post = await _host.PublicPostFrom(bob, alice);
var path = $"/api/v1/pleroma/statuses/{post.ID}/reactions";
var reacted = (await alice.Client.Put(path + "/%F0%9F%91%8D")).Ok();
var reaction = Assert.Single(reacted.Body["emoji_reactions"]!.AsArray());
Assert.Equal(("👍", 1, true), (reaction.Text("name"), reaction.Number("count"), reaction.Flag("me")));
Assert.Equal("👍", Assert.Single(reacted.Body["pleroma"]!["emoji_reactions"]!.AsArray()).Text("name"));
(await carol.Client.Put(path + "/%F0%9F%91%8D")).Ok();
var listed = Assert.Single((await _host.Client().Get(path)).Ok().Array);
Assert.Equal(("👍", 2, false), (listed.Text("name"), listed.Number("count"), listed.Flag("me")));
Assert.Equal(new[] { alice.Id, carol.Id }.Order(), listed!["accounts"]!.AsArray().Select(a => a.Text("id")).Order());
Assert.True(Assert.Single((await alice.Client.Get(path)).Ok().Array).Flag("me"));
var react = Assert.Single(await bob.Delivered(since), d => d.Type() == "EmojiReact" && d.Text("actor") == alice.Uri);
Assert.Equal(("👍", post.ObjectURI), (react.Text("content"), react.Text("object")));
var withdrawn = (await alice.Client.Delete(path + "/%F0%9F%91%8D")).Ok();
Assert.Equal(1, Assert.Single(withdrawn.Body["emoji_reactions"]!.AsArray()).Number("count"));
var undo = Assert.Single(await bob.Delivered(since), d => d.Type() == "Undo" && d.Text("actor") == alice.Uri);
Assert.Equal(react.Text("id"), undo["object"].Text("id"));
Assert.Equal("EmojiReact", undo["object"].Text("type"));
Assert.Equal(HttpStatusCode.UnprocessableEntity, (await alice.Client.Put(path + "/abc")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await alice.Client.Put("/api/v1/pleroma/statuses/000000000000000000000000/reactions/%F0%9F%91%8D")).Status);
}
[Fact]
public async Task Reacting_to_a_local_post_notifies_its_author_and_federates_nothing()
{
var since = DateTime.UtcNow.AddSeconds(-1);
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var bob = new RemoteActor(_peer, "bob");
await _host.FollowedBy(carol, bob);
var post = await carol.Status("react to me");
(await alice.Client.Put($"/api/v1/pleroma/statuses/{post.Text("id")}/reactions/%F0%9F%8E%89")).Ok();
var notification = Assert.Single((await carol.Client.Get("/api/v1/notifications?types[]=pleroma:emoji_reaction")).Ok().Array);
Assert.Equal(("🎉", alice.Id, post.Text("id")), (notification.Text("emoji"), notification["account"].Text("id"), notification["status"].Text("id")));
Assert.DoesNotContain(await bob.Delivered(since), d => d.Type() is "EmojiReact" or "Like");
(await alice.Client.Delete($"/api/v1/pleroma/statuses/{post.Text("id")}/reactions/%F0%9F%8E%89")).Ok();
Assert.Empty((await carol.Client.Get($"/api/v1/pleroma/statuses/{post.Text("id")}/reactions")).Ok().Array);
}
}
}
+50
View File
@@ -0,0 +1,50 @@
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using PrivaPub.Infrastructure;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class RegistrationsTests : IAsyncLifetime
{
WebApplicationFactory<Program> _closed;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var shared = await PrivaPubHost.Shared();
_closed = shared.WithWebHostBuilder(builder => builder.UseSetting("Registrations:Mode", nameof(RegistrationMode.Invitations)));
}
public async ValueTask DisposeAsync()
{
if (_closed != default)
await _closed.DisposeAsync();
}
[Fact]
public async Task Invitations_only_refuses_sign_up_and_every_document_says_closed()
{
using var client = _closed.CreateClient(new WebApplicationFactoryClientOptions { BaseAddress = new Uri(PrivaPubHost.Base + "/"), AllowAutoRedirect = false });
client.DefaultRequestHeaders.Add(PrivaPubHost.ClientHeader, $"10.9.{Random.Shared.Next(256)}.{Random.Shared.Next(1, 255)}");
var signUp = await client.PostJson("/clientapi/user/signup", new { userName = $"closed{Guid.NewGuid():N}"[..20], password = "Closed-Door-Pass-1" });
var nodeInfo = (await client.Get("/nodeinfo/2.1")).Ok().Body;
var v1 = (await client.Get("/api/v1/instance")).Ok().Body;
var v2 = (await client.Get("/api/v2/instance")).Ok().Body;
Assert.Equal(HttpStatusCode.Forbidden, signUp.StatusCode);
Assert.Contains("invitation", await signUp.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
Assert.False(nodeInfo.Flag("openRegistrations"));
Assert.False(v1.Flag("registrations"));
Assert.True(v1.Flag("invites_enabled"));
Assert.False(v2["registrations"].Flag("enabled"));
Assert.Equal(RegistrationOptions.ClosedMessage, v2["registrations"].Text("message"));
}
}
}
+118
View File
@@ -0,0 +1,118 @@
using MongoDB.Entities;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Net.Http.Json;
using GroupEntity = PrivaPub.Models.Group.Group;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Tests.Http
{
// Deleting a root deletes its personas and their groups everywhere (owner decision 2026-10-04).
[Trait("Category", "Integration")]
public sealed class RootRemovalTests : IAsyncLifetime
{
PrivaPubHost _host;
HttpClient _client;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
_client?.Dispose();
if (_peer != default)
await _peer.DisposeAsync();
}
async Task<HttpResponseMessage> AdminRemoves(params string[] rootIds)
{
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
using var request = new HttpRequestMessage(HttpMethod.Delete, "/clientapi/admin/remove/users") { Content = JsonContent.Create(new { userIdList = rootIds }) };
return await client.SendAsync(request, TestContext.Current.CancellationToken);
}
[Fact]
public async Task A_removed_roots_personas_and_groups_are_deleted_everywhere_and_say_so()
{
var token = TestContext.Current.CancellationToken;
var root = await _host.SignUp("leaving");
var persona = await _host.Persona(root, "leaving");
var follower = new RemoteActor(_peer, "fan");
var member = new RemoteActor(_peer, "member");
await DB.Default.SaveAsync(new Follower { LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = follower.Id, InboxURL = follower.Id + "/inbox" }, token);
var group = await _host.Group(persona, community: true);
var groupId = group["id"]!.GetValue<string>();
await DB.Default.SaveAsync(new Follower { LocalActorId = groupId, LocalActorKind = LocalActorKind.Group, ActorURI = member.Id, InboxURL = member.Id + "/inbox" }, token);
var post = await _host.Publish(persona, "goodbye soon");
var apiToken = await _host.MastodonToken(persona);
var since = DateTime.UtcNow.AddSeconds(-1);
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(root.Id)).StatusCode);
var toFollower = Assert.Single(await Jobs.Deliveries(follower.Id + "/inbox", since, token), d => d["type"]!.GetValue<string>() == "Delete");
Assert.Equal(persona.ActorUri(), toFollower["object"]!.GetValue<string>());
Assert.Equal(persona.ActorUri(), toFollower["actor"]!.GetValue<string>());
var groupName = group["userName"]!.GetValue<string>();
Assert.Contains(await Jobs.Deliveries(member.Id + "/inbox", since, token),
d => d["type"]!.GetValue<string>() == "Delete" && d["object"]!.GetValue<string>() == $"{PrivaPubHost.Base}/peasants/{groupName}");
var gone = await _client.Fetch($"/peasants/{persona.UserName}");
Assert.Equal(HttpStatusCode.Gone, gone.Status);
Assert.Equal("Person", gone.Json["formerType"]!.GetValue<string>());
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch($"/peasants/{groupName}")).Status);
Assert.Equal(HttpStatusCode.Gone, (await _client.GetAsync($"/.well-known/webfinger?resource=acct:{persona.UserName}@{PrivaPubHost.Host}", token)).StatusCode);
var stored = await DB.Default.Find<PostEntity>().MatchID(post.ID).ExecuteFirstAsync(token);
Assert.NotNull(stored.DeletedAt);
Assert.Null(stored.ContentHtml);
Assert.NotNull((await DB.Default.Find<GroupEntity>().MatchID(groupId).ExecuteFirstAsync(token)).DeletionAt);
var removed = await DB.Default.Find<RootUser>().MatchID(root.Id).ExecuteFirstAsync(token);
Assert.Equal($"deleted-{root.Id}", removed.UserName);
Assert.Null(removed.HashedPassword);
using var app = _host.Client();
app.DefaultRequestHeaders.Authorization = new("Bearer", apiToken);
Assert.Equal(HttpStatusCode.Unauthorized, (await app.GetAsync("/api/v1/accounts/verify_credentials", token)).StatusCode);
}
[Fact]
public async Task A_second_removal_does_not_collide_with_the_first()
{
var first = await _host.SignUp("first");
var second = await _host.SignUp("second");
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(first.Id)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(second.Id)).StatusCode);
Assert.NotNull((await DB.Default.Find<RootUser>().MatchID(second.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).DeletedAt);
}
[Fact]
public async Task A_root_deletes_itself_only_with_its_password()
{
var token = TestContext.Current.CancellationToken;
var root = await _host.SignUp("self");
var persona = await _host.Persona(root, "self");
using var client = _host.As(root.Jwt);
var refused = await client.PostJson("/clientapi/user/delete", new { password = "Not-The-Password-1" });
Assert.Equal(HttpStatusCode.Forbidden, refused.StatusCode);
Assert.Null((await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(token)).DeletionAt);
var accepted = await client.PostJson("/clientapi/user/delete", new { password = root.Password });
Assert.Equal(HttpStatusCode.OK, accepted.StatusCode);
Assert.NotNull((await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(token)).DeletionAt);
Assert.Equal(HttpStatusCode.Unauthorized, (await client.GetAsync("/clientapi/user/sniff/again", token)).StatusCode);
}
}
}
@@ -0,0 +1,63 @@
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Http
{
// a public thread names its replies and its conversation (owner decision 2026-10-06), and they list only what is public
// or unlisted; a followers-only post names neither
[Trait("Category", "Integration")]
public sealed class ThreadCollectionsTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static string PathOf(JsonObject status) => new Uri(status.Text("uri")).AbsolutePath;
static string[] Items(JsonNode collection) => collection["orderedItems"]!.AsArray().Select(i => i!.GetValue<string>()).ToArray();
[Fact]
public async Task A_public_post_names_its_replies_and_context_which_list_only_public_answers()
{
var alice = await _host.Mastodon("alice");
var carol = await _host.Mastodon("carol");
var root = await alice.Status("a public question");
var answer = await carol.Status("a public answer", ("in_reply_to_id", root.Text("id")));
var hidden = await carol.Status("for my followers", ("in_reply_to_id", root.Text("id")), ("visibility", "private"));
var deeper = await alice.Status("and a deeper one", ("in_reply_to_id", answer.Text("id")), ("visibility", "unlisted"));
var note = (await _host.ActivityPub(PathOf(root))).Ok().Object;
Assert.Equal(root.Text("uri") + "/replies", note.Text("replies"));
Assert.Equal(root.Text("uri") + "/context", note.Text("context"));
var answerNote = (await _host.ActivityPub(PathOf(deeper))).Ok().Object;
Assert.Equal(root.Text("uri") + "/context", answerNote.Text("context"));
var replies = (await _host.ActivityPub(PathOf(root) + "/replies")).Ok().Object;
Assert.Equal(new[] { answer.Text("uri") }, Items(replies));
var context = (await _host.ActivityPub(PathOf(root) + "/context")).Ok().Object;
Assert.Equal(new[] { root.Text("uri"), answer.Text("uri"), deeper.Text("uri") }, Items(context));
Assert.Equal(alice.Uri, context.Text("attributedTo"));
Assert.DoesNotContain(hidden.Text("uri"), Items(context));
}
[Fact]
public async Task A_followers_only_post_names_neither_and_serves_neither()
{
var alice = await _host.Mastodon("alice");
var quiet = await alice.Status("for my followers", ("visibility", "private"));
Assert.Equal(HttpStatusCode.NotFound, (await _host.ActivityPub(PathOf(quiet) + "/replies")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await _host.ActivityPub(PathOf(quiet) + "/context")).Status);
}
}
}
+237
View File
@@ -0,0 +1,237 @@
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
using static PrivaPub.Tests.Support.Host.FederationHelpers;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class WellKnownTests : IAsyncLifetime
{
const string Base = PrivaPubHost.Base;
const string Domain = PrivaPubHost.Host;
PrivaPubHost _host;
HttpClient _client;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_client = _host.Client();
}
public ValueTask DisposeAsync()
{
_client?.Dispose();
return ValueTask.CompletedTask;
}
Task<Fetched> WebFinger(string resource) =>
_client.Fetch("/.well-known/webfinger" + (resource == default ? string.Empty : "?resource=" + Uri.EscapeDataString(resource)), "application/jrd+json");
static string Link(JsonObject document, string rel) =>
document["links"]!.AsArray().Single(l => l!["rel"]!.GetValue<string>() == rel)!["href"]!.GetValue<string>();
// FEP-d556, FEP-2677 and FEP-844e: the server's own actor is found from its origin and from NodeInfo, and says what
// the server reads that its documents do not show
[Fact]
public async Task The_server_actor_is_found_from_the_origin_and_nodeinfo_and_tells_what_it_implements()
{
var instance = $"{Base}/peasants/privapub";
foreach (var resource in new[] { Base, Base + "/" })
{
var found = await WebFinger(resource);
Assert.Equal(HttpStatusCode.OK, found.Status);
Assert.Equal(resource, found.Json["subject"]!.GetValue<string>());
Assert.Equal(instance, Link(found.Json, "https://www.w3.org/ns/activitystreams#Service"));
}
var nodeinfo = await _client.Fetch("/.well-known/nodeinfo", "application/json");
Assert.Equal(instance, Link(nodeinfo.Json, "https://www.w3.org/ns/activitystreams#Application"));
var actor = (await _client.Fetch("/peasants/privapub")).Json;
Assert.Contains(actor["implements"]!.AsArray(), i => i!["href"]!.GetValue<string>() == "https://datatracker.ietf.org/doc/html/rfc9421");
var persona = await _host.Persona(await _host.SignUp(), "generated");
var personaActor = (await _client.Fetch($"/peasants/{persona.UserName}")).Json;
Assert.Equal("Application", personaActor["generator"]!["type"]!.GetValue<string>());
Assert.Contains(personaActor["generator"]!["implements"]!.AsArray(), i => i!["href"]!.GetValue<string>() == "https://datatracker.ietf.org/doc/html/rfc9421");
Assert.StartsWith("z6Mk", personaActor["assertionMethod"]![0]!["publicKeyMultibase"]!.GetValue<string>());
}
[Fact]
public async Task WebFinger_finds_a_persona_by_acct_and_by_actor_uri()
{
var persona = await _host.Persona(await _host.SignUp(), "finger");
var resources = new[]
{
$"acct:{persona.UserName}@{Domain}",
$"acct:@{persona.UserName}@{Domain}",
$"acct:{persona.UserName.ToUpperInvariant()}@{Domain.ToUpperInvariant()}",
$"{persona.UserName}@{Domain}",
$"@{persona.UserName}@{Domain}",
persona.ActorUri()
};
foreach (var resource in resources)
{
var found = await WebFinger(resource);
Assert.True(found.Status == HttpStatusCode.OK, $"{resource} answered {(int)found.Status}");
Assert.Equal("application/jrd+json", found.MediaType);
var document = found.Json;
Assert.Equal($"acct:{persona.UserName}@{Domain}", document["subject"]!.GetValue<string>());
Assert.Equal(new[] { $"{Base}/@{persona.UserName}", persona.ActorUri() }, Strings(document["aliases"]));
Assert.Equal(persona.ActorUri(), Link(document, "self"));
Assert.Equal("application/activity+json", document["links"]!.AsArray().Single(l => l!["rel"]!.GetValue<string>() == "self")!["type"]!.GetValue<string>());
Assert.Equal($"{Base}/@{persona.UserName}", Link(document, "http://webfinger.net/rel/profile-page"));
Assert.DoesNotContain(persona.Root.Id, found.Text);
Assert.DoesNotContain(persona.Root.UserName, found.Text);
}
}
[Fact]
public async Task WebFinger_refuses_other_domains_unknown_names_and_a_missing_resource()
{
var persona = await _host.Persona(await _host.SignUp(), "nofinger");
Assert.Equal(HttpStatusCode.NotFound, (await WebFinger($"acct:{persona.UserName}@elsewhere.example")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await WebFinger($"acct:{persona.UserName}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await WebFinger($"acct:nobody{Guid.NewGuid():N}"[..20] + "@" + Domain)).Status);
Assert.Equal(HttpStatusCode.NotFound, (await WebFinger($"https://elsewhere.example/peasants/{persona.UserName}")).Status);
Assert.Equal(HttpStatusCode.NotFound, (await WebFinger($"acct:{persona.Root.UserName}@{Domain}")).Status);
Assert.Equal(HttpStatusCode.BadRequest, (await WebFinger(default)).Status);
Assert.Equal(HttpStatusCode.BadRequest, (await WebFinger(string.Empty)).Status);
}
[Fact]
public async Task WebFinger_finds_the_servers_own_actors_and_a_community()
{
var owner = await _host.Persona(await _host.SignUp(), "fingerowner");
var community = await _host.FederatedGroup(owner, community: true);
var instance = await WebFinger($"acct:privapub@{Domain}");
var group = await WebFinger($"acct:{community.UserName}@{Domain}");
Assert.Equal(HttpStatusCode.OK, instance.Status);
Assert.Equal($"acct:privapub@{Domain}", instance.Json["subject"]!.GetValue<string>());
Assert.Equal($"{Base}/peasants/privapub", Link(instance.Json, "self"));
Assert.Single(instance.Json["links"]!.AsArray());
var reporter = await WebFinger($"acct:privapub_reports@{Domain}");
Assert.Equal(HttpStatusCode.OK, reporter.Status);
Assert.Equal($"{Base}/peasants/privapub_reports", Link(reporter.Json, "self"));
Assert.Single(reporter.Json["links"]!.AsArray());
Assert.Equal(HttpStatusCode.OK, group.Status);
Assert.Equal($"acct:{community.UserName}@{Domain}", group.Json["subject"]!.GetValue<string>());
Assert.Equal(community.Uri, Link(group.Json, "self"));
Assert.DoesNotContain(owner.UserName, group.Text);
Assert.DoesNotContain(owner.Id, group.Text);
}
// Current behaviour, for the owner to decide: WebFinger answers for a circle, because Mastodon resolves a handle
// through WebFinger before it can send the Follow a circle takes members through, while /api/v1/accounts/lookup,
// search and the profile page do not show circles at all.
[Fact]
public async Task WebFinger_answers_for_a_circle()
{
var owner = await _host.Persona(await _host.SignUp(), "circlefinger");
var circle = await _host.FederatedGroup(owner, community: false);
var found = await WebFinger($"acct:{circle.UserName}@{Domain}");
Assert.Equal(HttpStatusCode.OK, found.Status);
Assert.Equal(circle.Uri, Link(found.Json, "self"));
Assert.DoesNotContain(owner.UserName, found.Text);
}
[Fact]
public async Task NodeInfo_is_linked_and_served_in_both_versions_without_naming_anyone()
{
var root = await _host.SignUp();
var persona = await _host.Persona(root, "nodeinfo");
await _host.Publish(persona, "counted");
var links = await _client.Fetch("/.well-known/nodeinfo", "application/json");
Assert.Equal(HttpStatusCode.OK, links.Status);
Assert.Equal("application/json", links.MediaType);
var hrefs = links.Json["links"]!.AsArray().ToDictionary(l => l!["rel"]!.GetValue<string>(), l => l!["href"]!.GetValue<string>());
Assert.Equal($"{Base}/nodeinfo/2.1", hrefs["http://nodeinfo.diaspora.software/ns/schema/2.1"]);
Assert.Equal($"{Base}/nodeinfo/2.0", hrefs["http://nodeinfo.diaspora.software/ns/schema/2.0"]);
foreach (var version in new[] { "2.0", "2.1" })
{
var info = await _client.Fetch(PathOf(hrefs[$"http://nodeinfo.diaspora.software/ns/schema/{version}"]), "application/json");
Assert.Equal(HttpStatusCode.OK, info.Status);
Assert.Equal("application/json", info.MediaType);
Assert.Contains($"profile=\"http://nodeinfo.diaspora.software/ns/schema/{version}#\"", info.Response.Content.Headers.ContentType!.ToString());
var document = info.Json;
Assert.Equal(version, document["version"]!.GetValue<string>());
Assert.Equal("privapub", document["software"]!["name"]!.GetValue<string>());
Assert.Equal(version == "2.1", document["software"]!["repository"] != default);
Assert.Equal(new[] { "activitypub" }, Strings(document["protocols"]));
Assert.True(document["usage"]!["users"]!["total"]!.GetValue<long>() >= 1);
Assert.True(document["usage"]!["localPosts"]!.GetValue<long>() >= 1);
Assert.DoesNotContain(root.Id, info.Text);
Assert.DoesNotContain(root.UserName, info.Text);
Assert.DoesNotContain(persona.UserName, info.Text);
}
}
[Fact]
public async Task Unknown_nodeinfo_versions_are_404()
{
foreach (var version in new[] { "1.0", "2.2", "3.0", "2.0.json", "2" })
Assert.True((await _client.Fetch($"/nodeinfo/{version}", "application/json")).Status == HttpStatusCode.NotFound, $"/nodeinfo/{version} was served");
}
}
[Trait("Category", "Integration")]
[Collection(nameof(Exclusive))]
public sealed class NodeInfoUsageTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
async Task<long> LocalPosts()
{
using var client = _host.Client();
return (await client.Fetch("/nodeinfo/2.1", "application/json")).Json["usage"]!["localPosts"]!.GetValue<long>();
}
// localPosts is Mastodon's: the sum of statuses_count, every local post but DMs and deleted ones, boosts included,
// and circle and located posts too (owner decision 2026-10-04)
[Fact]
public async Task Local_posts_count_as_mastodon_counts_statuses()
{
var persona = await _host.Persona(await _host.SignUp(), "usage");
var friend = await _host.Persona(await _host.SignUp(), "usagefriend");
var start = await LocalPosts();
var shown = await _host.Publish(persona, "counted");
await _host.Publish(persona, "counted too", PostVisibility.Unlisted);
Assert.Equal(start + 2, await LocalPosts());
await _host.Publish(persona, "for followers", PostVisibility.FollowersOnly);
await _host.Publish(persona, $"@{friend.UserName} between us", PostVisibility.Direct);
await _host.PublishLocated(persona, "around here");
await _host.FederatedCopy(persona);
await _host.Reblog(friend, shown);
var circle = await _host.FederatedGroup(persona, community: false);
await _host.Publish(persona, new StatusDraft { Text = "circle only", PlainText = true, GroupId = circle.Id });
Assert.Equal(start + 6, await LocalPosts());//followers-only, located, the boost and the circle post; not the DM nor the copy
await _host.Remove(persona, shown);
Assert.Equal(start + 4, await LocalPosts());//the post and the boost of it
}
}
}
@@ -0,0 +1,147 @@
using Microsoft.Extensions.DependencyInjection;
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.Infrastructure.Cli;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Infrastructure
{
[Trait("Category", "Integration")]
public sealed class AdminCommandsTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
static async Task<List<string>> StoredPolicies(Root root) =>
(await DB.Default.Find<RootUser>().MatchID(root.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Policies.Order().ToList();
async Task<HttpStatusCode> AdminRoute(Root root)
{
using var client = _host.As(root.Jwt);
return (await client.GetAsync("/clientapi/admin/domainblocks/list", TestContext.Current.CancellationToken)).StatusCode;
}
[Fact]
public async Task Promote_makes_an_admin_and_a_moderator()
{
var root = await _host.SignUp("promote");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName.ToUpperInvariant() }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(new[] { Policies.IsAdmin, Policies.IsModerator, Policies.IsUser }.Order(), await StoredPolicies(root));
}
[Fact]
public async Task Demote_leaves_a_plain_user()
{
var root = await _host.SignUp("demote");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(new[] { Policies.IsUser }, await StoredPolicies(root));
}
[Fact]
public async Task An_unknown_root_exits_1()
{
Assert.Equal(1, await AdminCommands.Run(new[] { "promote", $"nobody{Guid.NewGuid():N}" }, _host.Services));
Assert.Equal(1, await AdminCommands.Run(new[] { "demote", $"nobody{Guid.NewGuid():N}" }, _host.Services));
}
public static TheoryData<string[]> Misuses() => new()
{
Array.Empty<string>(),
new[] { "promote" },
new[] { "promote", "a", "b" },
new[] { "delete", "a" },
new[] { "PROMOTE", "a" }
};
[Theory]
[MemberData(nameof(Misuses))]
public async Task Misuse_exits_2(string[] args)
{
Assert.Equal(2, await AdminCommands.Run(args, _host.Services));
}
[Fact]
public async Task A_token_follows_its_roots_policies_at_once()
{
var root = await _host.SignUp("policies");
Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(HttpStatusCode.OK, await AdminRoute(root));
using (var client = _host.As(root.Jwt))
{
var refreshed = await (await client.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken)).JsonBody();
Assert.Contains(Policies.IsAdmin, refreshed["policies"]!.AsArray().Select(p => p!.GetValue<string>()));
}
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root));
}
async Task<HttpStatusCode> LogIn(string login, string password)
{
using var client = _host.Client();
return (await client.PostJson("/clientapi/user/login", new { userName = login, password })).StatusCode;
}
[Fact]
public async Task Create_root_makes_a_login_from_a_password_on_standard_input()
{
using var scope = _host.Services.CreateScope();
var login = $"made{Guid.NewGuid():N}"[..20];
var output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(new[] { "create-root", login, "--admin" }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-1\n"), output));
Assert.Equal(1, await AdminCommands.Run(new[] { "create-root", login }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-2\n"), new StringWriter()));
Assert.Equal(1, await AdminCommands.Run(new[] { "create-root", $"weak{Guid.NewGuid():N}"[..20] }, scope.ServiceProvider, new StringReader("short\n"), new StringWriter()));
Assert.Equal(2, await AdminCommands.Run(new[] { "create-root", login, "--root" }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-3\n"), new StringWriter()));
Assert.Equal(HttpStatusCode.OK, await LogIn(login, "Typed-On-Stdin-1"));
var stored = await DB.Default.Find<RootUser>().Match(u => u.UserName == login).ExecuteFirstAsync(TestContext.Current.CancellationToken);
Assert.Contains(Policies.IsAdmin, stored.Policies);
}
[Fact]
public async Task Smoke_keeps_one_undiscoverable_persona_and_a_new_password_each_run()
{
using var scope = _host.Services.CreateScope();
var persona = $"smoke{Guid.NewGuid():N}"[..20];
var first = new StringWriter();
var second = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(new[] { "smoke", persona }, scope.ServiceProvider, output: first));
Assert.Equal(0, await AdminCommands.Run(new[] { "smoke", persona }, scope.ServiceProvider, output: second));
var (login, oldPassword) = (first.ToString().Trim().Split(' ')[0], first.ToString().Trim().Split(' ')[1]);
var newPassword = second.ToString().Trim().Split(' ')[1];
Assert.Equal(AdminCommands.SmokeLogin, login);
Assert.NotEqual(oldPassword, newPassword);
Assert.Equal(HttpStatusCode.OK, await LogIn(login, newPassword));
Assert.NotEqual(HttpStatusCode.OK, await LogIn(login, oldPassword));
var avatars = await DB.Default.Find<Avatar>().Match(a => a.UserName == persona).ExecuteAsync(TestContext.Current.CancellationToken);
var avatar = Assert.Single(avatars);
Assert.False(avatar.Settings.IsDiscoverable);
Assert.False(avatar.Settings.IsIndexable);
}
}
}
@@ -0,0 +1,287 @@
using MongoDB.Bson;
using MongoDB.Bson.IO;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Backup;
using PrivaPub.Infrastructure.Cli;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.IO.Compression;
namespace PrivaPub.Tests.Infrastructure
{
// The server's backup: every collection as it was, byte for byte, read at one instant on a replica set, without what
// belongs to the moment (the statistics salt above all), with the media rows' files linked, checkable and rotated.
// Each test backs up a database of its own; alone, since the maintenance lock is the server's one.
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class BackupTests : IAsyncLifetime
{
readonly string _scratch = Path.Combine(Path.GetTempPath(), $"privapub-backup-tests-{Guid.NewGuid():N}");
IMongoDatabase _database;
string Backups => Path.Combine(_scratch, "backups");
string Media => Path.Combine(_scratch, "media");
string Trash => Path.Combine(_scratch, "media-trash");
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
await PrivaPubHost.Shared();
_database = DB.Default.Database().Client.GetDatabase($"PrivaPubBackup_{Guid.NewGuid():N}");
Directory.CreateDirectory(Media);
Directory.CreateDirectory(Trash);
}
public async ValueTask DisposeAsync()
{
if (_database != default)
await _database.Client.DropDatabaseAsync(_database.DatabaseNamespace.DatabaseName);
if (Directory.Exists(_scratch))
Directory.Delete(_scratch, recursive: true);
}
BackupContext Context(BackupOptions options = default) => new(_database, Backups, Media, Trash, "https://privapub.test", options ?? new BackupOptions());
static CancellationToken Token => TestContext.Current.CancellationToken;
static List<BsonDocument> Read(string file)
{
using var gzip = new GZipStream(File.OpenRead(file), CompressionMode.Decompress);
using var reader = new StreamReader(gzip);
var documents = new List<BsonDocument>();
while (reader.ReadLine() is { } line)
documents.Add(BsonDocument.Parse(line));
return documents;
}
async Task<List<byte[]>> Raw(string collection) =>
(await (await _database.GetCollection<RawBsonDocument>(collection).FindAsync(FilterDefinition<RawBsonDocument>.Empty, cancellationToken: Token)).ToListAsync(Token))
.Select(d =>
{
var bytes = new byte[d.Slice.Length];
d.Slice.GetBytes(0, bytes, 0, bytes.Length);
return bytes;
})
.ToList();
[Fact]
public async Task Every_document_comes_back_byte_for_byte_and_the_salt_never_leaves()
{
var odd = new BsonDocument
{
{ "_id", ObjectId.GenerateNewId() },
{ "Guid", new BsonBinaryData(Guid.NewGuid(), GuidRepresentation.Standard) },
{ "OldGuid", new BsonBinaryData(new byte[16], BsonBinarySubType.UuidLegacy) },
{ "Money", new BsonDecimal128(Decimal128.Parse("12345.678900")) },
{ "Long", new BsonInt64(long.MaxValue) },
{ "Int", 7 },
{ "Double", -0.0 },
{ "NaN", double.NaN },
{ "At", new BsonDateTime(new DateTime(2026, 10, 7, 3, 30, 0, 123, DateTimeKind.Utc)) },
{ "Stamp", new BsonTimestamp(1, 2) },
{ "Null", BsonNull.Value },
{ "Regex", new BsonRegularExpression("^a.b$", "i") },
{ "Nested", new BsonDocument { { "z", 1 }, { "a", new BsonArray { 1, "two", new BsonDocument("three", 3) } } } },
{ "Unicode", "città 🌍 \u0000 end" }
};
// an ObjectRecord as big as a remote's long post gets
var big = new BsonDocument { { "_id", ObjectId.GenerateNewId() }, { "Json", new string('x', 10 * 1024 * 1024) } };
await _database.GetCollection<BsonDocument>("Odd").InsertOneAsync(odd, cancellationToken: Token);
await _database.GetCollection<BsonDocument>("ObjectRecord").InsertOneAsync(big, cancellationToken: Token);
await _database.GetCollection<BsonDocument>("Odd").Indexes.CreateOneAsync(
new CreateIndexModel<BsonDocument>(Builders<BsonDocument>.IndexKeys.Ascending("At"), new CreateIndexOptions { Name = "at", Unique = true }), cancellationToken: Token);
await _database.GetCollection<BsonDocument>("InteractionSalt").InsertOneAsync(new BsonDocument("Salt", "never in a backup"), cancellationToken: Token);
await _database.GetCollection<BsonDocument>("Job").InsertOneAsync(new BsonDocument("Kind", "Deliver"), cancellationToken: Token);
await _database.GetCollection<BsonDocument>("_migration_history_").InsertManyAsync([
new BsonDocument { { "Number", 15 }, { "Name", "older" } }, new BsonDocument { { "Number", 16 }, { "Name", "focal points are finite" } }], cancellationToken: Token);
var (backup, error) = await ServerBackup.Create(Context(), "manual", dbOnly: false, Token);
Assert.Null(error);
var directory = Path.Combine(Backups, backup.Id);
Assert.False(Directory.Exists(directory + ServerBackup.PartialSuffix));
Assert.Equal(await Raw("Odd"), Read(Path.Combine(directory, "db", "Odd.jsonl.gz")).Select(d => d.ToBson()).ToList());
Assert.Equal(await Raw("ObjectRecord"), Read(Path.Combine(directory, "db", "ObjectRecord.jsonl.gz")).Select(d => d.ToBson()).ToList());
Assert.False(File.Exists(Path.Combine(directory, "db", "InteractionSalt.jsonl.gz")));
Assert.False(File.Exists(Path.Combine(directory, "db", "Job.jsonl.gz")));
foreach (var file in Directory.EnumerateFiles(directory, "*", SearchOption.AllDirectories))
Assert.DoesNotContain("never in a backup", await ReadAll(file));
Assert.Contains(backup.Manifest.Excluded, e => e.Name == "InteractionSalt");
var odds = backup.Manifest.Collections.Single(c => c.Name == "Odd");
Assert.Equal(1, odds.Count);
Assert.Contains(odds.Indexes, i => BsonDocument.Parse(i)["name"] == "at" && BsonDocument.Parse(i)["unique"].ToBoolean());
Assert.Equal((16, "focal points are finite"), (backup.Manifest.MigrationNumber, backup.Manifest.NewestMigration));
Assert.Equal("https://privapub.test", backup.Manifest.Host);
Assert.Equal(MongoFixture.Connection.Contains("directConnection=true"), backup.Manifest.Consistent);
Assert.Empty(await ServerBackup.Verify(Backups, backup.Id, Token));
if (!OperatingSystem.IsWindows())
{
Assert.Equal(UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.GroupRead, File.GetUnixFileMode(Path.Combine(directory, "db", "Odd.jsonl.gz")));
Assert.False(File.GetUnixFileMode(directory).HasFlag(UnixFileMode.OtherRead));
}
}
static async Task<string> ReadAll(string file)
{
if (!file.EndsWith(".gz", StringComparison.Ordinal))
return await File.ReadAllTextAsync(file, Token);
await using var gzip = new GZipStream(File.OpenRead(file), CompressionMode.Decompress);
using var reader = new StreamReader(gzip);
return await reader.ReadToEndAsync(Token);
}
[Fact]
public async Task Media_rows_files_are_linked_from_the_live_directory_or_the_trash()
{
Directory.CreateDirectory(Path.Combine(Media, "2026", "10"));
await File.WriteAllTextAsync(Path.Combine(Media, "2026", "10", "live.jpg"), "live", Token);
await File.WriteAllTextAsync(Path.Combine(Media, "2026", "10", "live-preview.jpg"), "preview", Token);
Directory.CreateDirectory(Path.Combine(Trash, "2026", "10"));
await File.WriteAllTextAsync(Path.Combine(Trash, "2026", "10", "moving.jpg"), "moving", Token);
await File.WriteAllTextAsync(Path.Combine(Media, "2026", "10", "trashed.jpg"), "trashed", Token);
await _database.GetCollection<BsonDocument>("MediaAttachment").InsertManyAsync([
new BsonDocument { { "FilePath", "2026/10/live.jpg" }, { "PreviewPath", "2026/10/live-preview.jpg" }, { "TrashedAt", BsonNull.Value } },
new BsonDocument { { "FilePath", "2026/10/moving.jpg" } },//a file the janitor moved while its row was being trashed
new BsonDocument { { "FilePath", "2026/10/gone.jpg" } },
new BsonDocument { { "FilePath", "2026/10/trashed.jpg" }, { "TrashedAt", DateTime.UtcNow } },
new BsonDocument { { "FilePath", "../../etc/passwd" } }], cancellationToken: Token);
var (backup, _) = await ServerBackup.Create(Context(), "manual", dbOnly: false, Token);
var media = Path.Combine(Backups, backup.Id, "media");
Assert.Equal("live", await File.ReadAllTextAsync(Path.Combine(media, "2026", "10", "live.jpg"), Token));
Assert.Equal("preview", await File.ReadAllTextAsync(Path.Combine(media, "2026", "10", "live-preview.jpg"), Token));
Assert.Equal("moving", await File.ReadAllTextAsync(Path.Combine(media, "2026", "10", "moving.jpg"), Token));
Assert.False(File.Exists(Path.Combine(media, "2026", "10", "trashed.jpg")));
Assert.Equal(["2026/10/live-preview.jpg", "2026/10/live.jpg", "2026/10/moving.jpg"], backup.Manifest.Media.List);
Assert.Equal((3, 1), (backup.Manifest.Media.Files, backup.Manifest.Media.Missing));
// a link, not a copy: the live file deleted, the backup's stays
File.Delete(Path.Combine(Media, "2026", "10", "live.jpg"));
Assert.Equal("live", await File.ReadAllTextAsync(Path.Combine(media, "2026", "10", "live.jpg"), Token));
// db-only lists them and links none
var (listed, _) = await ServerBackup.Create(Context(), "pre-deploy", dbOnly: true, Token);
Assert.False(Directory.Exists(Path.Combine(Backups, listed.Id, "media")));
Assert.Equal(["2026/10/live-preview.jpg", "2026/10/moving.jpg"], listed.Manifest.Media.List);
Assert.Empty(await ServerBackup.Verify(Backups, listed.Id, Token));
Assert.Empty(await ServerBackup.Verify(Backups, backup.Id, Token));
}
[Fact]
public async Task Verify_finds_an_altered_or_missing_file()
{
await _database.GetCollection<BsonDocument>("Post").InsertOneAsync(new BsonDocument("Content", "hello"), cancellationToken: Token);
await _database.GetCollection<BsonDocument>("Avatar").InsertOneAsync(new BsonDocument("UserName", "someone"), cancellationToken: Token);
var (backup, _) = await ServerBackup.Create(Context(), "manual", dbOnly: false, Token);
var db = Path.Combine(Backups, backup.Id, "db");
await File.AppendAllTextAsync(Path.Combine(db, "Post.jsonl.gz"), "tampered", Token);
File.Delete(Path.Combine(db, "Avatar.jsonl.gz"));
Assert.Equal(["Avatar: missing", "Post: altered"], (await ServerBackup.Verify(Backups, backup.Id, Token)).Order());
Assert.Equal(["no such backup, or no manifest"], await ServerBackup.Verify(Backups, "../" + backup.Id, Token));
}
[Fact]
public async Task One_backup_at_a_time()
{
await using (var held = await MaintenanceLock.Take("restore", Token))
{
Assert.NotNull(held);
var (backup, error) = await ServerBackup.Create(Context(), "manual", dbOnly: true, Token);
Assert.Null(backup);
Assert.Contains("already running", error);
Assert.Equal("restore", (await MaintenanceLock.Current(Token)).What);
}
Assert.Null(await MaintenanceLock.Current(Token));
Assert.NotNull((await ServerBackup.Create(Context(), "manual", dbOnly: true, Token)).Backup);
}
[Fact]
public async Task A_holder_that_died_is_taken_over()
{
await using var dead = await MaintenanceLock.Take("backup", Token);
await DB.Default.Update<MaintenanceLock>().Match(l => l.ID == MaintenanceLock.Name)
.Modify(l => l.Heartbeat, DateTime.UtcNow.AddMinutes(-3)).ExecuteAsync(Token);
Assert.Null(await MaintenanceLock.Current(Token));
await using var alive = await MaintenanceLock.Take("restore", Token);
Assert.NotNull(alive);
Assert.Equal("restore", (await MaintenanceLock.Current(Token)).What);
}
// the newest 7 nightly, the newest of each of the 4 weeks before, the newest 3 pre-deploy; manual ones kept
[Fact]
public void Rotation_keeps_days_weeks_and_the_last_deploys()
{
var today = new DateTime(2026, 10, 7, 3, 30, 0, DateTimeKind.Utc);
for (var day = 0; day < 60; day++)
Fake("nightly", today.AddDays(-day));
for (var deploy = 0; deploy < 5; deploy++)
Fake("pre-deploy", today.AddDays(-deploy).AddHours(5));
Fake("manual", today.AddYears(-1));
Directory.CreateDirectory(Path.Combine(Backups, "20260901-000000-nightly" + ServerBackup.PartialSuffix));
Directory.SetLastWriteTimeUtc(Path.Combine(Backups, "20260901-000000-nightly" + ServerBackup.PartialSuffix), today.AddDays(-30));
ServerBackup.Retain(Backups, new BackupOptions());
var kept = ServerBackup.List(Backups);
var nightly = kept.Where(b => b.Kind == "nightly").Select(b => b.CreatedAt).ToList();
Assert.Equal(11, nightly.Count);
Assert.Equal(Enumerable.Range(0, 7).Select(d => today.AddDays(-d)), nightly.Take(7));
Assert.Equal(4, nightly.Skip(7).Select(d => System.Globalization.ISOWeek.GetWeekOfYear(d)).Distinct().Count());
Assert.Equal(3, kept.Count(b => b.Kind == "pre-deploy"));
Assert.Single(kept, b => b.Kind == "manual");
Assert.Empty(Directory.EnumerateDirectories(Backups, "*" + ServerBackup.PartialSuffix));
}
void Fake(string kind, DateTime at)
{
var directory = Path.Combine(Backups, $"{at:yyyyMMdd-HHmmss}-{kind}");
Directory.CreateDirectory(Path.Combine(directory, "db"));
new ArchiveManifest { Kind = kind, CreatedAt = at }.Write(directory);
}
[Theory]
[InlineData("2026-10-07T03:29:00", "2026-10-06T03:31:00", false)]//yesterday's is the last one due
[InlineData("2026-10-07T03:31:00", "2026-10-06T03:31:00", true)]
[InlineData("2026-10-07T03:31:00", "2026-10-07T03:30:30", false)]
[InlineData("2026-10-07T01:00:00", "2026-10-05T03:31:00", true)]//missed last night: at once
public void A_nightly_backup_is_due_once_a_night(string now, string last, bool due) =>
Assert.Equal(due, BackupScheduler.IsDue(DateTime.Parse(now, null, System.Globalization.DateTimeStyles.AdjustToUniversal),
new TimeOnly(3, 30), [DateTime.Parse(last, null, System.Globalization.DateTimeStyles.AdjustToUniversal)]));
// the deploy's: the server's own database, through the configuration, without media links
[Fact]
public async Task The_deploy_backs_up_from_the_command_line()
{
var host = await PrivaPubHost.Shared();
var output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(["backup", "--kind", "pre-deploy", "--db-only"], host.Services, output: output));
var id = output.ToString().Split(':')[0];
Assert.EndsWith("-pre-deploy", id);
Assert.Equal(2, await AdminCommands.Run(["backup", "--kind", "nightly"], host.Services, output: TextWriter.Null));
output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(["backups"], host.Services, output: output));
Assert.StartsWith(id + "\tpre-deploy", output.ToString());
output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(["backup", "verify", id], host.Services, output: output));
Assert.Contains("whole", output.ToString());
var backups = host.Get<Backups>();
var manifest = backups.Find(id).Manifest;
Assert.Contains(manifest.Collections, c => c.Name == "Avatar");
Assert.DoesNotContain(manifest.Collections, c => c.Name is "InteractionSalt" or "Job" or "MaintenanceLock" or "openiddict.tokens" or "AppConfiguration");
Assert.Equal(ServerBackup.CodeMigration(), manifest.MigrationNumber);
Assert.True(backups.Delete(id));
}
}
}
@@ -0,0 +1,111 @@
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Jobs;
using System.Net;
namespace PrivaPub.Tests.Infrastructure
{
public class CdnCatalogTests
{
static HttpResponseMessage Response(params (string Name, string Value)[] headers)
{
var response = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent("{}") };
foreach (var (name, value) in headers)
response.Headers.TryAddWithoutValidation(name, value);
return response;
}
[Theory]
[InlineData("cf-ray", "8c1f0e2b4d5a1234-FRA", "cloudflare.com")]
[InlineData("server", "cloudflare", "cloudflare.com")]
[InlineData("x-amz-cf-id", "abc==", "cloudfront.net")]
[InlineData("via", "1.1 5a1b.cloudfront.net (CloudFront)", "cloudfront.net")]
[InlineData("x-served-by", "cache-fra-eddf8230045-FRA", "fastly.com")]
[InlineData("server", "BunnyCDN-DE1-1078", "bunny.net")]
[InlineData("akamai-grn", "0.1234", "akamai.com")]
[InlineData("x-azure-ref", "20261004T101010Z-abc", "azure.microsoft.com")]
[InlineData("x-vercel-id", "fra1::abc", "vercel.com")]
[InlineData("x-iinfo", "1-2-3", "imperva.com")]
public void A_cdn_is_recognised_by_its_fingerprint(string header, string value, string domain)
{
Assert.Equal(domain, CdnCatalog.FromResponse(Response((header, value)))?.Domain);
}
[Fact]
public void A_plain_server_has_no_cdn_and_an_unknown_cdn_names_itself()
{
Assert.Null(CdnCatalog.FromResponse(Response(("server", "nginx"), ("via", "1.1 varnish"))));
var named = CdnCatalog.FromResponse(Response(("x-cdn", "EdgeOne")));
Assert.Equal(("EdgeOne", null, "edgeone"), (named.Name, named.Domain, named.Key));
Assert.Null(CdnCatalog.FromResponse(Response(("x-cdn", "<script>"))));
}
[Fact]
public void Only_networks_that_carry_nothing_but_a_cdn_name_it()
{
Assert.Equal("Cloudflare", CdnCatalog.OfAsn(13335)?.Name);
Assert.Equal("Akamai", CdnCatalog.OfAsn(20940)?.Name);
Assert.Null(CdnCatalog.OfAsn(16509));//AWS: CloudFront only by its ranges or its headers
Assert.Null(CdnCatalog.OfAsn(60068));//DataPacket also rents servers
}
[Fact]
public void Every_published_list_format_is_read()
{
Assert.Equal(new[] { "173.245.48.0/20", "2400:cb00::/32" },
CdnCatalog.Parse(CdnListFormat.Lines, "173.245.48.0/20\n# comment\n\n2400:cb00::/32\n").Select(n => n.ToString()));
Assert.Equal(new[] { "89.187.188.227/32", "2400:52e0:1500::714:1/128" },
CdnCatalog.Parse(CdnListFormat.StringArray, """["89.187.188.227","2400:52e0:1500::714:1"]""").Select(n => n.ToString()));
Assert.Equal(new[] { "23.235.32.0/20", "2a04:4e40::/32" },
CdnCatalog.Parse(CdnListFormat.AddressesJson, """{"addresses":["23.235.32.0/20"],"ipv6_addresses":["2a04:4e40::/32"]}""").Select(n => n.ToString()));
Assert.Equal(new[] { "92.223.124.12/32", "2a03:90c0::/32" },
CdnCatalog.Parse(CdnListFormat.AddressesJson, """{"addresses":["92.223.124.12/32"],"addresses_v6":["2a03:90c0::/32"]}""").Select(n => n.ToString()));
Assert.Equal(new[] { "199.83.128.0/21", "2a02:e980::/29" },
CdnCatalog.Parse(CdnListFormat.ImpervaJson, """{"ipRanges":["199.83.128.0/21"],"ipv6Ranges":["2a02:e980::/29"],"res":0}""").Select(n => n.ToString()));
Assert.Equal(new[] { "13.32.0.0/15", "2600:9000::/28" },
CdnCatalog.Parse(CdnListFormat.AwsCloudFront, """
{"prefixes":[{"ip_prefix":"3.4.12.4/32","service":"AMAZON"},{"ip_prefix":"13.32.0.0/15","service":"CLOUDFRONT"}],
"ipv6_prefixes":[{"ipv6_prefix":"2600:9000::/28","service":"CLOUDFRONT"},{"ipv6_prefix":"2406:daba::/40","service":"EC2"}]}
""").Select(n => n.ToString()));
Assert.Throws<FormatException>(() => CdnCatalog.Parse(CdnListFormat.Lines, "173.245.48.0/20\nnot an address"));
}
[Fact]
public void The_published_ranges_name_the_cdn_of_an_address()
{
var ranges = new CdnRanges();
ranges.Replace(new CdnRangeSet { Key = "cloudflare.com", Name = "Cloudflare", Ranges = new() { "104.16.0.0/13", "2606:4700::/32" }, FetchedAt = DateTime.UtcNow });
Assert.Equal("Cloudflare", ranges.Of(IPAddress.Parse("104.18.2.3"))?.Name);
Assert.Equal("Cloudflare", ranges.Of(IPAddress.Parse("::ffff:104.18.2.3"))?.Name);
Assert.Equal("Cloudflare", ranges.Of(IPAddress.Parse("2606:4700::1"))?.Name);
Assert.Null(ranges.Of(IPAddress.Parse("203.0.113.7")));
Assert.Equal(2, ranges.Status["cloudflare.com"].Ranges);
}
[Fact]
public async Task Responses_leave_a_cdn_hint_for_their_host_until_one_comes_without()
{
var hints = new EdgeHints();
var answers = new Queue<HttpResponseMessage>(new[] { Response(("cf-ray", "1-FRA")), Response(("server", "nginx")) });
using var client = new HttpClient(new EdgeHintsHandler(hints) { InnerHandler = new Answering(answers) });
await client.GetAsync("https://social.example/.well-known/nodeinfo", TestContext.Current.CancellationToken);
Assert.Equal("Cloudflare", hints.Of("social.example")?.Name);
await client.GetAsync("https://social.example/nodeinfo/2.1", TestContext.Current.CancellationToken);
Assert.Null(hints.Of("social.example"));
}
sealed class Answering : HttpMessageHandler
{
readonly Queue<HttpResponseMessage> _answers;
public Answering(Queue<HttpResponseMessage> answers) => _answers = answers;
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) =>
Task.FromResult(_answers.Dequeue());
}
}
}
@@ -0,0 +1,105 @@
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using System.Net;
namespace PrivaPub.Tests.Infrastructure
{
[Trait("Category", "Integration")]
public sealed class CdnUpdaterTests : IAsyncLifetime
{
Peer _peer;
CdnRanges _ranges;
CdnUpdater _updater;
string _key;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
_ranges = new CdnRanges();
_updater = new CdnUpdater(new Clients(), new StaticOptions<StatisticsOptions>(new StatisticsOptions()), _ranges, NullLogger<CdnUpdater>.Instance);
_key = $"edge{Guid.NewGuid():N}.example";
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
sealed class Clients : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new();
}
CdnProvider Provider() => new("Test Edge", _key, Array.Empty<int>(), new[]
{
new CdnList($"{_peer.A}/edge/v4", CdnListFormat.Lines),
new CdnList($"{_peer.A}/edge/v6", CdnListFormat.StringArray)
}, _ => false);
static CancellationToken Token => TestContext.Current.CancellationToken;
[Fact]
public async Task The_published_ranges_are_downloaded_stored_and_used()
{
_peer.ServeText("/edge/v4", "198.51.100.0/24\n", "text/plain");
_peer.ServeText("/edge/v6", """["2001:db8::/32","192.0.2.9"]""", "application/json");
Assert.Equal(1, await _updater.Update(new[] { Provider() }, Token));
var stored = await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == _key).ExecuteSingleAsync(Token);
Assert.Equal(new[] { "198.51.100.0/24", "2001:db8::/32", "192.0.2.9/32" }, stored.Ranges);
Assert.Equal("Test Edge", _ranges.Of(IPAddress.Parse("198.51.100.20"))?.Name);
Assert.Equal("Test Edge", _ranges.Of(IPAddress.Parse("192.0.2.9"))?.Name);
var restarted = new CdnRanges();
await restarted.Load(Token);
Assert.Equal("Test Edge", restarted.Of(IPAddress.Parse("2001:db8::7"))?.Name);
}
[Theory]
[InlineData("not a network")]
[InlineData("")]
public async Task A_broken_or_empty_list_keeps_the_stored_ranges(string broken)
{
_peer.ServeText("/edge/v4", "198.51.100.0/24", "text/plain");
_peer.ServeText("/edge/v6", "[]", "application/json");
Assert.Equal(1, await _updater.Update(new[] { Provider() }, Token));
_peer.ServeText("/edge/v4", broken, "text/plain");
Assert.Equal(0, await _updater.Update(new[] { Provider() }, Token));
Assert.Equal(new[] { "198.51.100.0/24" }, (await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == _key).ExecuteSingleAsync(Token)).Ranges);
Assert.Equal("Test Edge", _ranges.Of(IPAddress.Parse("198.51.100.1"))?.Name);
}
// The real lists, from the CDNs themselves: set PRIVAPUB_TEST_CDN_LIVE=1 to check that every format still parses.
[Fact]
public async Task Every_cdns_real_list_still_parses()
{
Assert.SkipUnless(Environment.GetEnvironmentVariable("PRIVAPUB_TEST_CDN_LIVE") == "1", "set PRIVAPUB_TEST_CDN_LIVE=1 to download the real lists");
var listed = CdnCatalog.All.Where(p => p.Lists.Length > 0).ToList();
Assert.Equal(listed.Count, await _updater.Update(listed, Token));
Assert.Equal("Cloudflare", _ranges.Of(IPAddress.Parse("104.16.0.1"))?.Name);
foreach (var provider in listed)
Assert.True(_ranges.Status[provider.Key].Ranges > 0, provider.Name);
}
[Fact]
public async Task An_unreachable_list_changes_nothing()
{
Assert.Equal(0, await _updater.Update(new[] { Provider() }, Token));
Assert.False(await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == _key).ExecuteAnyAsync(Token));
}
}
}
@@ -27,6 +27,12 @@ namespace PrivaPub.Tests.Infrastructure
_peer.MapGet("/big", () => Results.Text("{\"a\":\"" + new string('a', FederationHttp.MaxResponseBytes) + "\"}", "application/activity+json"));
_peer.MapGet("/hop/{n:int}", (int n) => Results.Redirect(n == 0 ? "/actor" : $"/hop/{n - 1}"));
_peer.MapGet("/gone", () => Results.StatusCode(410));
// Mobilizon's NodeInfo: a profile URL left unquoted, which .NET cannot parse as a media type
_peer.MapGet("/nodeinfo", (HttpContext context) =>
{
context.Response.Headers.ContentType = "application/json; profile=http://nodeinfo.diaspora.software/ns/schema/2.1#; charset=utf-8";
return context.Response.WriteAsync("{\"software\":{\"name\":\"Mobilizon\"}}");
});
await _peer.StartAsync();
_base = _peer.Urls.First();
}
@@ -64,6 +70,13 @@ namespace PrivaPub.Tests.Infrastructure
public async Task Refuses_a_fourth_redirect() =>
Assert.Null(await Client().GetJson($"{_base}/hop/3", "application/activity+json", default, TestContext.Current.CancellationToken));
[Fact]
public async Task Reads_json_under_a_content_type_dotnet_cannot_parse()
{
using var fetched = await Client().GetJson($"{_base}/nodeinfo", "application/json", default, TestContext.Current.CancellationToken);
Assert.Equal("Mobilizon", fetched.Root.GetProperty("software").GetProperty("name").GetString());
}
[Fact]
public async Task Refuses_html() =>
Assert.Null(await Client().GetJson($"{_base}/html", "application/activity+json", default, TestContext.Current.CancellationToken));
@@ -0,0 +1,69 @@
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Tests.Support;
using System.Net;
namespace PrivaPub.Tests.Infrastructure
{
public class GeoLocatorTests
{
static DbIpLocator Locator(string directory) =>
new(new StaticOptions<StatisticsOptions>(new StatisticsOptions { GeoDirectory = directory }), NullLogger<DbIpLocator>.Instance);
[Fact]
public void Without_databases_nothing_is_located()
{
using var locator = Locator(Path.Combine(Path.GetTempPath(), $"no-geo-{Guid.NewGuid():N}"));
Assert.Null(locator.Locate(IPAddress.Parse("1.1.1.1")));
Assert.Null(locator.Source);
}
[Fact]
public void A_private_address_is_never_looked_up()
{
using var locator = Locator(Path.GetTempPath());
Assert.Null(locator.Locate(IPAddress.Parse("10.1.2.3")));
Assert.Null(locator.Locate(IPAddress.Loopback));
Assert.Null(locator.Locate(default));
}
[Fact]
public void The_real_databases_give_a_country_a_city_and_a_network()
{
var directory = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_GEO");
Assert.SkipWhen(string.IsNullOrEmpty(directory), "set PRIVAPUB_TEST_GEO to a directory holding the DB-IP Lite city and ASN files");
using var locator = Locator(directory);
var fix = locator.Locate(IPAddress.Parse("1.1.1.1"));
Assert.NotNull(fix);
Assert.Equal(2, fix.Country.Length);
Assert.Equal(13335, fix.Asn);
Assert.Equal("Cloudflare", CdnCatalog.OfAsn(fix.Asn)?.Name);
Assert.NotNull(fix.AsnOrg);
Assert.NotNull(fix.Latitude);
Assert.Equal(Math.Round(fix.Latitude.Value, 1), fix.Latitude);
Assert.StartsWith("DB-IP Lite ", locator.Source);
}
[Fact]
public void A_connection_remembers_its_address_as_ipv4()
{
var connected = new ConnectedAddresses();
connected.Remember("Social.Example", IPAddress.Parse("::ffff:203.0.113.7"));
Assert.Equal(IPAddress.Parse("203.0.113.7"), connected.Of("social.example"));
Assert.Null(connected.Of("other.example"));
Assert.Equal("Fastly", CdnCatalog.OfAsn(54113)?.Name);
Assert.Null(CdnCatalog.OfAsn(64496));
Assert.Null(CdnCatalog.OfAsn(default));
}
}
}
@@ -0,0 +1,125 @@
using MaxMind.Db;
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Tests.Support;
using System.IO.Compression;
namespace PrivaPub.Tests.Infrastructure
{
public sealed class GeoUpdaterTests : IAsyncLifetime
{
static readonly DateTime Now = new(2026, 10, 4, 12, 0, 0, DateTimeKind.Utc);
static readonly DateTime LastMonth = new(2026, 9, 2, 0, 0, 0, DateTimeKind.Utc);
readonly string _directory = Path.Combine(Path.GetTempPath(), $"privapub-geo-{Guid.NewGuid():N}");
Peer _peer;
GeoUpdater _updater;
DbIpLocator _locator;
public async ValueTask InitializeAsync()
{
_peer = await Peer.Start();
var options = new StaticOptions<StatisticsOptions>(new StatisticsOptions
{
GeoDirectory = _directory,
Geo = new GeoOptions { DownloadBase = $"{_peer.A}/free" }
});
_locator = new DbIpLocator(options, NullLogger<DbIpLocator>.Instance);
_updater = new GeoUpdater(new Clients(), options, _locator, NullLogger<GeoUpdater>.Instance);
}
public async ValueTask DisposeAsync()
{
_locator?.Dispose();
await _peer.DisposeAsync();
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
sealed class Clients : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new();
}
static byte[] Gzip(byte[] bytes)
{
using var output = new MemoryStream();
using (var gzip = new GZipStream(output, CompressionLevel.Fastest))
gzip.Write(bytes);
return output.ToArray();
}
void Publish(string kind, DateTime month, byte[] database) =>
_peer.ServeFile($"/free/dbip-{kind}-lite-{month:yyyy-MM}.mmdb.gz", Gzip(database), "application/gzip");
static byte[] City(DateTime built) => MiniMmdb.Build("DBIP-City-Lite", built);
static byte[] Asn(DateTime built) => MiniMmdb.Build("DBIP-ASN-Lite (compat=GeoLite2-ASN)", built);
DateTime Built(string kind)
{
using var reader = new Reader(Path.Combine(_directory, GeoUpdater.FileOf(kind)));
return reader.Metadata.BuildDate;
}
[Fact]
public async Task Missing_databases_are_fetched_checked_and_read_at_once()
{
Publish("city", Now, City(Now));
Publish("asn", Now, Asn(Now));
Assert.Equal(2, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(Now.Date, Built("city").Date);
Assert.Equal(Now.Date, Built("asn").Date);
Assert.Equal("DB-IP Lite 2026-10", _locator.Source);
Assert.Empty(Directory.GetFiles(_directory).Where(f => f.EndsWith(".part") || f.EndsWith(".new")));
}
[Fact]
public async Task Early_in_the_month_last_months_database_is_used_and_not_fetched_again()
{
Publish("city", LastMonth, City(LastMonth));
Publish("asn", LastMonth, Asn(LastMonth));
Assert.Equal(2, await _updater.Update(Now, TestContext.Current.CancellationToken));
var asked = _peer.Requests.Count;
Assert.Equal(0, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(LastMonth.Date, Built("city").Date);
Assert.DoesNotContain(_peer.Requests.Skip(asked), r => r.Path.Contains("2026-09"));
}
[Fact]
public async Task A_current_database_asks_nothing()
{
Publish("city", Now, City(Now));
Publish("asn", Now, Asn(Now));
await _updater.Update(Now, TestContext.Current.CancellationToken);
var asked = _peer.Requests.Count;
Assert.Equal(0, await _updater.Update(Now.AddDays(1), TestContext.Current.CancellationToken));
Assert.Equal(asked, _peer.Requests.Count);
}
[Fact]
public async Task A_broken_or_wrong_database_is_refused_and_the_old_one_kept()
{
Publish("city", LastMonth, City(LastMonth));
Publish("asn", LastMonth, Asn(LastMonth));
await _updater.Update(LastMonth.AddDays(1), TestContext.Current.CancellationToken);
Publish("city", Now, "not a database at all"u8.ToArray());
Publish("asn", Now, City(Now));
Assert.Equal(0, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(LastMonth.Date, Built("city").Date);
Assert.Equal(LastMonth.Date, Built("asn").Date);
Assert.Empty(Directory.GetFiles(_directory).Where(f => f.EndsWith(".part") || f.EndsWith(".new")));
}
}
}
@@ -59,6 +59,8 @@ namespace PrivaPub.Tests.Infrastructure
Assert.True(await local.IsUserNameTaken(name, token));
Assert.False(await local.TryReserveUserName("admin", LocalActorKind.Person, "c", token));
Assert.False(await local.TryReserveUserName(LocalActorService.InstanceUserName, LocalActorKind.Person, "c", token));
Assert.False(await local.TryReserveUserName(LocalActorService.ReporterUserName, LocalActorKind.Group, "c", token));
Assert.True(await local.IsUserNameTaken("PrivaPub_Reports", token));
}
}
}
@@ -1,3 +1,5 @@
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Jobs;
@@ -74,6 +76,8 @@ namespace PrivaPub.Tests.Infrastructure
Assert.Equal(JobState.Pending, after.State);
Assert.True(after.RunAt > DateTime.UtcNow.AddSeconds(10));
// leased again for its third attempt (a finish only counts for the lease it was given)
await DB.Default.Update<Job>().MatchID(leased.ID).Modify(j => j.State, JobState.Running).Modify(j => j.LeaseOwner, leased.LeaseOwner).ExecuteAsync(token);
leased.Attempts = 3;
await queue.Finish(leased, JobOutcome.Retry("boom"), maxAttempts: 3, token);
Assert.Equal(JobState.Dead, (await DB.Default.Find<Job>().OneAsync(leased.ID, token)).State);
@@ -90,5 +94,100 @@ namespace PrivaPub.Tests.Infrastructure
Assert.Equal(JobState.Pending, (await DB.Default.Find<Job>().OneAsync(job.ID, token)).State);
}
// a handler running three times its lease keeps it: the reaper finds nothing to take back and it runs once
[Fact]
public async Task A_long_job_keeps_its_lease_and_runs_once()
{
var token = TestContext.Current.CancellationToken;
var kind = (JobKind)(2000 + Random.Shared.Next(100000));
var queue = new JobQueue(TimeSpan.FromSeconds(1.5), j => j.Kind == kind);
var handler = new SlowHandler(kind, TimeSpan.FromSeconds(4.5));
await queue.Enqueue(kind, "{}", default, default, token);
using var worker = new JobWorker(queue, new IJobHandler[] { handler }, NullLogger<JobWorker>.Instance);
await worker.StartAsync(token);
try
{
var deadline = DateTime.UtcNow.AddSeconds(20);
while (!await DB.Default.Find<Job>().Match(j => j.Kind == kind && j.State == JobState.Done).ExecuteAnyAsync(token))
{
Assert.True(DateTime.UtcNow < deadline, "the job never finished");
Assert.Equal(0, await queue.Reap(token));
await Task.Delay(250, token);
}
}
finally
{
await worker.StopAsync(CancellationToken.None);
}
Assert.Equal(1, handler.Started);
}
// a lease taken by another worker (reaped, then leased again) stops the handler, and its outcome is not recorded
[Fact]
public async Task A_lost_lease_stops_the_handler_and_drops_its_outcome()
{
var token = TestContext.Current.CancellationToken;
var kind = (JobKind)(2000 + Random.Shared.Next(100000));
var queue = new JobQueue(TimeSpan.FromSeconds(1.5), j => j.Kind == kind);
var handler = new SlowHandler(kind, TimeSpan.FromSeconds(30));
await queue.Enqueue(kind, "{}", default, default, token);
using var worker = new JobWorker(queue, new IJobHandler[] { handler }, NullLogger<JobWorker>.Instance);
await worker.StartAsync(token);
try
{
var deadline = DateTime.UtcNow.AddSeconds(20);
while (handler.Started == 0)
{
Assert.True(DateTime.UtcNow < deadline, "the job never started");
await Task.Delay(100, token);
}
await DB.Default.Update<Job>().Match(j => j.Kind == kind).Modify(j => j.LeaseOwner, "another worker").ExecuteAsync(token);
while (!handler.Cancelled)
{
Assert.True(DateTime.UtcNow < deadline, "the handler was never stopped");
await Task.Delay(100, token);
}
await Task.Delay(300, token);
}
finally
{
await worker.StopAsync(CancellationToken.None);
}
var job = await DB.Default.Find<Job>().Match(j => j.Kind == kind).ExecuteFirstAsync(token);
Assert.Equal(JobState.Running, job.State);
Assert.Equal("another worker", job.LeaseOwner);
}
sealed class SlowHandler(JobKind kind, TimeSpan takes) : IJobHandler
{
int _started;
public int Started => _started;
public bool Cancelled { get; private set; }
public JobKind Kind => kind;
public int Concurrency => 1;
public int MaxAttempts => 3;
public int PerHostLimit => 1;
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
Interlocked.Increment(ref _started);
try
{
await Task.Delay(takes, token);
}
catch (OperationCanceledException)
{
Cancelled = true;
throw;
}
return JobOutcome.Done;
}
}
}
}
@@ -1,16 +1,53 @@
using PrivaPub.Models.Media;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Data.Migrations;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.User;
using PrivaPub.Tests.Support;
using System.Text.Json;
using System.Text.Json.Nodes;
using GroupEntity = PrivaPub.Models.Group.Group;
namespace PrivaPub.Tests.Infrastructure
{
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public class MigrationTests
{
[Fact]
public async Task A_focal_point_that_is_not_a_number_is_removed()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var broken = new MediaAttachment { OwnerAvatarId = "a1", Focus = [float.NaN, 0f] };
var sound = new MediaAttachment { OwnerAvatarId = "a1", Focus = [0.5f, -0.5f] };
var post = new Post
{
ObjectURI = $"https://privapub.test/{Guid.NewGuid():N}",
Media = [new PostMedia { URL = "https://privapub.test/media/files/a.jpg", Focus = [float.PositiveInfinity, 0f] }, new PostMedia { URL = "https://privapub.test/media/files/b.jpg", Focus = [0.1f, 0.2f] }]
};
await DB.Default.SaveAsync(broken, token);
await DB.Default.SaveAsync(sound, token);
await DB.Default.SaveAsync(post, token);
await new _016_focal_points_are_finite().UpgradeAsync();
Assert.Null((await DB.Default.Find<MediaAttachment>().OneAsync(broken.ID, token)).Focus);
Assert.Equal(new[] { 0.5f, -0.5f }, (await DB.Default.Find<MediaAttachment>().OneAsync(sound.ID, token)).Focus);
var migrated = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Null(migrated.Media[0].Focus);
Assert.Equal(new[] { 0.1f, 0.2f }, migrated.Media[1].Focus);
}
[Fact]
public async Task Stored_remote_content_is_sanitized_and_local_content_rendered()
{
@@ -79,5 +116,135 @@ namespace PrivaPub.Tests.Infrastructure
Assert.Contains("legacy-quote", afterFetched.Extensions);
Assert.Empty(afterDelivered.Extensions);
}
[Fact]
public async Task Conversations_get_their_participants_key_groups_become_circles_and_their_posts_stay_home()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var members = new List<GroupMember> { new() { AvatarId = "a1" }, new() { AvatarId = "https://r.example/u/b", IsForeign = true } };
var conversation = new DmGroup { Members = members };
var group = new GroupEntity { UserName = $"old{Guid.NewGuid():N}"[..20], Kind = GroupKind.Community };
await DB.Default.SaveAsync(conversation, token);
await DB.Default.SaveAsync(group, token);
var localInGroup = new Post { GroupId = group.ID, Text = "inside", ObjectURI = $"https://privapub.test/{Guid.NewGuid():N}" };
var remoteInGroup = new Post { GroupId = group.ID, IsFederatedCopy = true, Text = "visiting", ObjectURI = $"https://r.example/{Guid.NewGuid():N}" };
var ungrouped = new Post { Text = "outside", ObjectURI = $"https://privapub.test/{Guid.NewGuid():N}" };
await DB.Default.SaveAsync(new[] { localInGroup, remoteInGroup, ungrouped }, token);
//the migration turns every group into a circle and keeps every local group post home: put the others back afterwards
var communities = (await DB.Default.Find<GroupEntity>().Match(g => g.Kind != GroupKind.Circle && g.ID != group.ID).ExecuteAsync(token)).Select(g => g.ID).ToList();
var federatedGroupPosts = (await DB.Default.Find<Post>().Match(p => p.GroupId != null && !p.IsFederatedCopy && !p.IsLocalOnly && p.ID != localInGroup.ID)
.ExecuteAsync(token)).Select(p => p.ID).ToList();
try
{
await new _003_conversation_keys_and_circles().UpgradeAsync();
Assert.Equal(DmGroup.KeyOf(members), (await DB.Default.Find<DmGroup>().OneAsync(conversation.ID, token)).ParticipantsKey);
Assert.Equal(GroupKind.Circle, (await DB.Default.Find<GroupEntity>().OneAsync(group.ID, token)).Kind);
Assert.True((await DB.Default.Find<Post>().OneAsync(localInGroup.ID, token)).IsLocalOnly);
Assert.False((await DB.Default.Find<Post>().OneAsync(remoteInGroup.ID, token)).IsLocalOnly);
Assert.False((await DB.Default.Find<Post>().OneAsync(ungrouped.ID, token)).IsLocalOnly);
}
finally
{
if (communities.Count > 0)
await DB.Default.Update<GroupEntity>().Match(g => communities.Contains(g.ID)).Modify(g => g.Kind, GroupKind.Community).ExecuteAsync(token);
if (federatedGroupPosts.Count > 0)
await DB.Default.Update<Post>().Match(p => federatedGroupPosts.Contains(p.ID)).Modify(p => p.IsLocalOnly, false).ExecuteAsync(token);
}
}
[Fact]
public async Task Pending_deliveries_move_to_the_job_queue_once_and_the_rest_are_left_alone()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var host = $"r{Guid.NewGuid():N}.example";
var activityId = $"https://privapub.test/a/{Guid.NewGuid():N}";
var body = new JsonObject { ["id"] = activityId, ["type"] = "Create" }.ToJsonString();
var nextAttempt = DateTime.UtcNow.AddMinutes(7);
Delivery Legacy(string inbox, DateTime? deliveredAt = default, DateTime? abandonedAt = default) => new()
{
SignerId = "000000000000000000000001", SignerKind = LocalActorKind.Person, InboxURL = inbox, Body = body, Attempts = 3,
NextAttemptAt = nextAttempt, DeliveredAt = deliveredAt, AbandonedAt = abandonedAt
};
var pending = Legacy($"https://{host.ToUpperInvariant()}/inbox");
var twice = Legacy($"https://{host}/shared");
var already = Legacy($"https://{host}/shared");
var broken = Legacy("not an address");
var delivered = Legacy($"https://{host}/done", deliveredAt: DateTime.UtcNow.AddDays(-1));
var abandoned = Legacy($"https://{host}/given-up", abandonedAt: DateTime.UtcNow.AddDays(-1));
await DB.Default.SaveAsync(new[] { pending, twice, already, broken, delivered, abandoned }, token);
await new _004_pending_deliveries_become_jobs().UpgradeAsync();
var jobs = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.Deliver && j.DedupeKey.StartsWith(activityId + "|")).ExecuteAsync(token);
Assert.Equal(2, jobs.Count);
var job = Assert.Single(jobs, j => j.DedupeKey == $"{activityId}|{pending.InboxURL}");
Assert.Equal(host, job.Host);
Assert.Equal(3, job.Attempts);
Assert.InRange(job.RunAt, nextAttempt.AddSeconds(-1), nextAttempt.AddSeconds(1));
var payload = JsonSerializer.Deserialize<DeliveryPayload>(job.Payload);
Assert.Equal((pending.SignerId, LocalActorKind.Person, pending.InboxURL, body), (payload.SignerId, payload.SignerKind, payload.Inbox, payload.Body));
Assert.Contains(jobs, j => j.DedupeKey == $"{activityId}|{twice.InboxURL}");
foreach (var moved in new[] { pending, twice, already, broken })
{
var after = await DB.Default.Find<Delivery>().OneAsync(moved.ID, token);
Assert.NotNull(after.AbandonedAt);
Assert.Equal("moved to the job queue", after.LastError);
}
Assert.Null((await DB.Default.Find<Delivery>().OneAsync(delivered.ID, token)).AbandonedAt);
Assert.Null((await DB.Default.Find<Delivery>().OneAsync(abandoned.ID, token)).LastError);
}
[Fact]
public async Task Year_one_edit_dates_become_no_edit_and_year_one_revisions_take_the_creation_date()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var created = new DateTime(2026, 9, 1, 12, 0, 0, DateTimeKind.Utc);
var edited = new DateTime(2026, 9, 2, 12, 0, 0, DateTimeKind.Utc);
var broken = new Post
{
IsFederatedCopy = true, ObjectURI = $"https://r.example/{Guid.NewGuid():N}", CreationDate = created, EditedAt = DateTime.MinValue, UpdateDate = DateTime.MinValue,
Revisions = new() { new PostRevision { ContentHtml = "<p>first</p>", EditedAt = DateTime.MinValue }, new PostRevision { ContentHtml = "<p>second</p>", EditedAt = edited } }
};
var fine = new Post { IsFederatedCopy = true, ObjectURI = $"https://r.example/{Guid.NewGuid():N}", CreationDate = created, EditedAt = edited, UpdateDate = edited };
await DB.Default.SaveAsync(new[] { broken, fine }, token);
await new _006_unedited_remote_posts_lose_year_one().UpgradeAsync();
var repaired = await DB.Default.Find<Post>().OneAsync(broken.ID, token);
Assert.Null(repaired.EditedAt);
Assert.Equal(created, repaired.UpdateDate);
Assert.Equal(new[] { created, edited }, repaired.Revisions.Select(r => r.EditedAt));
var untouched = await DB.Default.Find<Post>().OneAsync(fine.ID, token);
Assert.Equal((edited, edited), (untouched.EditedAt, untouched.UpdateDate));
}
[Fact]
public async Task Personas_and_groups_without_a_published_day_get_one_within_two_weeks_before_creation()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var created = new DateTime(2026, 6, 15, 18, 30, 0, DateTimeKind.Utc);
var avatar = new Avatar { UserName = $"aged{Guid.NewGuid():N}"[..20], CreatedAt = created };
var group = new GroupEntity { UserName = $"aged{Guid.NewGuid():N}"[..20], CreationDate = created };
var dated = new Avatar { UserName = $"dated{Guid.NewGuid():N}"[..20], CreatedAt = created, PublishedOn = new DateTime(2026, 6, 10, 0, 0, 0, DateTimeKind.Utc) };
await DB.Default.SaveAsync(new[] { avatar, dated }, token);
await DB.Default.SaveAsync(group, token);
await DB.Default.Update<Avatar>().MatchID(avatar.ID).Modify(b => b.Unset(a => a.PublishedOn)).ExecuteAsync(token);
await DB.Default.Update<GroupEntity>().MatchID(group.ID).Modify(b => b.Unset(g => g.PublishedOn)).ExecuteAsync(token);
Assert.False(await DB.Default.Find<Avatar>().Match(new BsonDocument("_id", ObjectId.Parse(avatar.ID)).Add(nameof(Avatar.PublishedOn), new BsonDocument("$exists", true))).ExecuteAnyAsync(token));
await new _007_personas_publish_a_spread_day().UpgradeAsync();
foreach (var published in new[] { (await DB.Default.Find<Avatar>().OneAsync(avatar.ID, token)).PublishedOn, (await DB.Default.Find<GroupEntity>().OneAsync(group.ID, token)).PublishedOn })
{
Assert.Equal(TimeSpan.Zero, published.TimeOfDay);
Assert.InRange(published, created.Date.AddDays(-13), created.Date);
}
Assert.Equal(new DateTime(2026, 6, 10, 0, 0, 0, DateTimeKind.Utc), (await DB.Default.Find<Avatar>().OneAsync(dated.ID, token)).PublishedOn);
}
}
}
@@ -0,0 +1,316 @@
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Backup;
using PrivaPub.Infrastructure.Cli;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
namespace PrivaPub.Tests.Infrastructure
{
// A backup restored at boot: what was made since is lost, what protects is not (owner decision 2026-10-07). A server
// in a database of its own is backed up, changed, and restored; alone, since the maintenance lock is the server's one.
[Trait("Category", "Integration")]
[Xunit.Collection(nameof(Exclusive))]
public sealed class RestoreTests : IAsyncLifetime
{
const string Host = "https://privapub.test";
readonly string _scratch = Path.Combine(Path.GetTempPath(), $"privapub-restore-tests-{Guid.NewGuid():N}");
IMongoDatabase _database;
string Backups => Path.Combine(_scratch, "backups");
string Media => Path.Combine(_scratch, "media");
string Trash => Path.Combine(_scratch, "media-trash");
static CancellationToken Token => TestContext.Current.CancellationToken;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
await PrivaPubHost.Shared();
_database = DB.Default.Database().Client.GetDatabase($"PrivaPubRestore_{Guid.NewGuid():N}");
Directory.CreateDirectory(Media);
Directory.CreateDirectory(Trash);
}
public async ValueTask DisposeAsync()
{
if (_database != default)
await _database.Client.DropDatabaseAsync(_database.DatabaseNamespace.DatabaseName);
if (Directory.Exists(_scratch))
Directory.Delete(_scratch, recursive: true);
}
BackupContext Context(string host = Host) => new(_database, Backups, Media, Trash, host, new BackupOptions());
IMongoCollection<BsonDocument> C(string name) => _database.GetCollection<BsonDocument>(name);
async Task<BsonDocument> One(string collection, BsonDocument filter) =>
await (await C(collection).FindAsync(filter, cancellationToken: Token)).FirstOrDefaultAsync(Token);
async Task<List<BsonDocument>> All(string collection) =>
await (await C(collection).FindAsync(FilterDefinition<BsonDocument>.Empty, cancellationToken: Token)).ToListAsync(Token);
static BsonDocument Id(ObjectId id) => new("_id", id);
static Task Set(IMongoCollection<BsonDocument> collection, ObjectId id, BsonDocument fields) =>
collection.UpdateOneAsync(Id(id), new BsonDocument("$set", fields), cancellationToken: Token);
// the ids of the town this test builds
readonly ObjectId _alice = ObjectId.GenerateNewId(), _alicePersona = ObjectId.GenerateNewId();
readonly ObjectId _deletedPost = ObjectId.GenerateNewId(), _editedPost = ObjectId.GenerateNewId(), _laterPost = ObjectId.GenerateNewId();
readonly ObjectId _deletedMedia = ObjectId.GenerateNewId(), _laterMedia = ObjectId.GenerateNewId();
readonly ObjectId _bob = ObjectId.GenerateNewId(), _bobPersona = ObjectId.GenerateNewId();
static string Uri(ObjectId post) => $"{Host}/peasants/alice/posts/{post}";
async Task<BackupInfo> TownBackedUp()
{
await File.WriteAllTextAsync(Path.Combine(Media, "deleted.jpg"), "deleted", Token);
await C("RootUser").InsertOneAsync(new BsonDocument { { "_id", _alice }, { "UserName", "alice" }, { "HashedPassword", "old" }, { "SessionStamp", "s1" }, { "Policies", new BsonArray { "IsUser" } }, { "DeletedAt", BsonNull.Value } }, cancellationToken: Token);
await C("Avatar").InsertOneAsync(new BsonDocument { { "_id", _alicePersona }, { "UserName", "alice" }, { "SuspendedAt", BsonNull.Value }, { "DeletionAt", BsonNull.Value } }, cancellationToken: Token);
await C("RootToAvatar").InsertOneAsync(new BsonDocument { { "RootId", _alice.ToString() }, { "AvatarId", _alicePersona.ToString() } }, cancellationToken: Token);
await C("ReservedName").InsertOneAsync(new BsonDocument("Name", "alice"), cancellationToken: Token);
await C("Post").InsertManyAsync([
new BsonDocument { { "_id", _deletedPost }, { "GroupUserId", _alicePersona.ToString() }, { "IsFederatedCopy", false }, { "ObjectURI", Uri(_deletedPost) }, { "Text", "regret" },
{ "Media", new BsonArray { new BsonDocument("AttachmentId", _deletedMedia.ToString()) } }, { "DeletedAt", BsonNull.Value } },
new BsonDocument { { "_id", _editedPost }, { "GroupUserId", _alicePersona.ToString() }, { "IsFederatedCopy", false }, { "ObjectURI", Uri(_editedPost) }, { "Text", "first words" }, { "DeletedAt", BsonNull.Value } }
], cancellationToken: Token);
await C("TimelineEntry").InsertOneAsync(new BsonDocument { { "PostId", _deletedPost.ToString() }, { "ReblogOfPostId", BsonNull.Value } }, cancellationToken: Token);
await C("MediaAttachment").InsertOneAsync(new BsonDocument { { "_id", _deletedMedia }, { "PostId", _deletedPost.ToString() }, { "FilePath", "deleted.jpg" }, { "TrashedAt", BsonNull.Value } }, cancellationToken: Token);
await C("Follower").InsertOneAsync(new BsonDocument { { "LocalActorId", _alicePersona.ToString() }, { "ActorURI", "https://elsewhere.example/users/left" } }, cancellationToken: Token);
await C("RemoteInstance").InsertOneAsync(new BsonDocument { { "Host", "down.example" }, { "ConsecutiveFailures", 9 }, { "UnavailableUntil", DateTime.UtcNow.AddDays(1) } }, cancellationToken: Token);
await C("_migration_history_").InsertOneAsync(new BsonDocument { { "Number", 16 }, { "Name", "focal points are finite" } }, cancellationToken: Token);
await C("InteractionSalt").InsertOneAsync(new BsonDocument("Salt", "the backup's day"), cancellationToken: Token);
var (backup, error) = await ServerBackup.Create(Context(), "manual", dbOnly: false, Token);
Assert.Null(error);
return backup;
}
// what happens after the backup: protective acts, and things merely made or changed
async Task LifeGoesOn()
{
// alice regrets a post: deleted, its media trashed and moved out of what is served
await Set(C("Post"), _deletedPost, new BsonDocument { { "DeletedAt", DateTime.UtcNow }, { "Text", BsonNull.Value }, { "Media", new BsonArray() } });
await Set(C("MediaAttachment"), _deletedMedia, new BsonDocument { { "TrashedAt", DateTime.UtcNow }, { "TrashReason", "deleted" } });
File.Move(Path.Combine(Media, "deleted.jpg"), Path.Combine(Trash, "deleted.jpg"));
await C("TimelineEntry").DeleteManyAsync(new BsonDocument("PostId", _deletedPost.ToString()), Token);
// an edit, a new post with a picture: not protective
await Set(C("Post"), _editedPost, new BsonDocument("Text", "second words"));
await File.WriteAllTextAsync(Path.Combine(Media, "later.jpg"), "later", Token);
await C("Post").InsertOneAsync(new BsonDocument { { "_id", _laterPost }, { "GroupUserId", _alicePersona.ToString() }, { "IsFederatedCopy", false }, { "ObjectURI", Uri(_laterPost) }, { "Text", "later" }, { "DeletedAt", BsonNull.Value } }, cancellationToken: Token);
await C("MediaAttachment").InsertOneAsync(new BsonDocument { { "_id", _laterMedia }, { "PostId", _laterPost.ToString() }, { "FilePath", "later.jpg" }, { "TrashedAt", BsonNull.Value } }, cancellationToken: Token);
// a follower leaves, another comes
await C("Follower").DeleteManyAsync(FilterDefinition<BsonDocument>.Empty, Token);
await C("Follower").InsertOneAsync(new BsonDocument { { "LocalActorId", _alicePersona.ToString() }, { "ActorURI", "https://elsewhere.example/users/came" } }, cancellationToken: Token);
// alice blocks someone, the admin blocks a server, someone is reported, alice's persona is suspended, she changes
// her password
await C("Block").InsertOneAsync(new BsonDocument { { "AvatarId", _alicePersona.ToString() }, { "TargetActorURI", "https://elsewhere.example/users/troll" } }, cancellationToken: Token);
await C("DomainBlock").InsertOneAsync(new BsonDocument { { "Domain", "evil.example" }, { "Severity", "Suspend" } }, cancellationToken: Token);
await C("Report").InsertOneAsync(new BsonDocument("Comment", "harassment"), cancellationToken: Token);
await Set(C("Avatar"), _alicePersona, new BsonDocument("SuspendedAt", DateTime.UtcNow));
await Set(C("RootUser"), _alice, new BsonDocument("HashedPassword", "new"));
// bob arrives
await C("RootUser").InsertOneAsync(new BsonDocument { { "_id", _bob }, { "UserName", "bob" }, { "HashedPassword", "bob's" }, { "Policies", new BsonArray { "IsUser" } }, { "DeletedAt", BsonNull.Value } }, cancellationToken: Token);
await C("Avatar").InsertOneAsync(new BsonDocument { { "_id", _bobPersona }, { "UserName", "bob" }, { "DeletionAt", BsonNull.Value } }, cancellationToken: Token);
await C("RootToAvatar").InsertOneAsync(new BsonDocument { { "RootId", _bob.ToString() }, { "AvatarId", _bobPersona.ToString() } }, cancellationToken: Token);
await C("ReservedName").InsertOneAsync(new BsonDocument("Name", "bob"), cancellationToken: Token);
// sessions, a list made, today's salt, work queued
await C("openiddict.tokens").InsertOneAsync(new BsonDocument("subject", "alice"), cancellationToken: Token);
await C("PersonaList").InsertOneAsync(new BsonDocument { { "AvatarId", _alicePersona.ToString() }, { "Title", "friends" } }, cancellationToken: Token);
await C("InteractionSalt").DeleteManyAsync(FilterDefinition<BsonDocument>.Empty, Token);
await C("InteractionSalt").InsertOneAsync(new BsonDocument("Salt", "today's"), cancellationToken: Token);
await C("Job").InsertOneAsync(new BsonDocument("Kind", "Deliver"), cancellationToken: Token);
}
async Task AssertRestoredAndProtected(BackupInfo backup)
{
// lost: what was merely made or changed since
Assert.Equal("first words", (await One("Post", Id(_editedPost)))["Text"].AsString);
var later = await One("Post", Id(_laterPost));
Assert.False(later["DeletedAt"].IsBsonNull);
Assert.True(later["Text"].IsBsonNull);
Assert.Empty(await All("PersonaList"));
Assert.Equal(16, (await One("_migration_history_", new BsonDocument()))["Number"].ToInt32());
// kept: every protective act
var deleted = await One("Post", Id(_deletedPost));
Assert.False(deleted["DeletedAt"].IsBsonNull);
Assert.True(deleted["Text"].IsBsonNull);
Assert.Null(await One("TimelineEntry", new BsonDocument("PostId", _deletedPost.ToString())));
Assert.False((await One("MediaAttachment", Id(_deletedMedia)))["TrashedAt"].IsBsonNull);
Assert.Equal("restore: made after the backup", (await One("MediaAttachment", Id(_laterMedia)))["TrashReason"].AsString);
Assert.Equal(["https://elsewhere.example/users/came"], (await All("Follower")).Select(f => f["ActorURI"].AsString));
Assert.NotNull(await One("Block", new BsonDocument("TargetActorURI", "https://elsewhere.example/users/troll")));
Assert.NotNull(await One("DomainBlock", new BsonDocument("Domain", "evil.example")));
Assert.NotNull(await One("Report", new BsonDocument("Comment", "harassment")));
Assert.False((await One("Avatar", Id(_alicePersona)))["SuspendedAt"].IsBsonNull);
var alice = await One("RootUser", Id(_alice));
Assert.Equal("new", alice["HashedPassword"].AsString);
Assert.NotEqual("s1", alice["SessionStamp"].AsString);
// bob, made since: deleted, his name kept
var bob = await One("RootUser", Id(_bob));
Assert.Equal($"deleted-{_bob}", bob["UserName"].AsString);
Assert.True(bob["HashedPassword"].IsBsonNull);
Assert.False(bob["DeletedAt"].IsBsonNull);
Assert.False((await One("Avatar", Id(_bobPersona)))["DeletionAt"].IsBsonNull);
Assert.NotNull(await One("ReservedName", new BsonDocument("Name", "bob")));
// what a backup never holds is as it was; sessions end; circuits close
Assert.Equal("today's", (await One("InteractionSalt", new BsonDocument()))["Salt"].AsString);
Assert.Single(await All("Job"));
Assert.Empty(await All("openiddict.tokens"));
Assert.Equal(0, (await One("RemoteInstance", new BsonDocument("Host", "down.example")))["ConsecutiveFailures"].ToInt32());
// the picture of the deleted post came back from the backup, to be trashed again by the janitor
Assert.True(File.Exists(Path.Combine(Media, "deleted.jpg")));
Assert.Null(RestoreMarker.Read(Backups));
var record = await (await _database.GetCollection<RestoreRecord>(nameof(RestoreRecord)).FindAsync(r => r.Backup == backup.Id, cancellationToken: Token)).FirstAsync(Token);
Assert.Equal(["bob"], record.Report.RootsTombstoned);
Assert.Equal(["bob"], record.Report.PersonasTombstoned);
Assert.Equal(1, record.Report.PostsGone);
Assert.Equal(1, record.Report.PostsDeleted);
}
[Fact]
public async Task A_restore_loses_what_was_made_since_but_never_a_protective_act()
{
var backup = await TownBackedUp();
await LifeGoesOn();
Assert.Null(await ServerRestore.Request(Context(), backup.Id, "admin", Token));
Assert.Contains("already pending", await ServerRestore.Request(Context(), backup.Id, "admin", Token));
Assert.Equal(RestoreOutcome.Restored, await ServerRestore.ApplyPending(Context(), NullLogger.Instance, Token));
await AssertRestoredAndProtected(backup);
var before = ServerBackup.List(Backups).Single(b => b.Kind == "pre-restore");
Assert.Equal(before.Id, (await (await _database.GetCollection<RestoreRecord>(nameof(RestoreRecord)).FindAsync(FilterDefinition<RestoreRecord>.Empty, cancellationToken: Token)).FirstAsync(Token)).PreRestore);
}
// an attempt that died midway: the next one starts again from the backup, with the same pre-restore backup, never
// one of the half-restored database
[Fact]
public async Task A_restore_that_died_midway_converges_on_the_next_boot()
{
var backup = await TownBackedUp();
await LifeGoesOn();
Assert.Null(await ServerRestore.Request(Context(), backup.Id, "admin", Token));
Assert.Equal(RestoreOutcome.Restored, await ServerRestore.ApplyPending(Context(), NullLogger.Instance, Token));
var before = ServerBackup.List(Backups).Single(b => b.Kind == "pre-restore");
// as if the boot died after importing a collection, with the marker saying so
await C("Post").DeleteManyAsync(FilterDefinition<BsonDocument>.Empty, Token);
await C("Block").DeleteManyAsync(FilterDefinition<BsonDocument>.Empty, Token);
new RestoreMarker { Backup = backup.Id, PreRestore = before.Id, State = "running", Attempts = 1, RequestedBy = "admin" }.Write(Backups);
Assert.Equal(RestoreOutcome.Restored, await ServerRestore.ApplyPending(Context(), NullLogger.Instance, Token));
Assert.Single(ServerBackup.List(Backups), b => b.Kind == "pre-restore");
await AssertRestoredAndProtected(backup);
}
[Fact]
public async Task A_backup_from_elsewhere_from_a_newer_build_or_altered_is_refused()
{
var backup = await TownBackedUp();
Assert.Contains("not https://other.test", await ServerRestore.Request(Context("https://other.test"), backup.Id, "admin", Token));
Assert.Contains("no such backup", await ServerRestore.Request(Context(), "../" + backup.Id, "admin", Token));
var manifest = ArchiveManifest.Read(Path.Combine(Backups, backup.Id));
manifest.MigrationNumber = ServerBackup.CodeMigration() + 1;
manifest.Write(Path.Combine(Backups, backup.Id));
Assert.Contains("newer build", await ServerRestore.Request(Context(), backup.Id, "admin", Token));
manifest.MigrationNumber = 16;
manifest.Write(Path.Combine(Backups, backup.Id));
await File.AppendAllTextAsync(Path.Combine(Backups, backup.Id, "db", "Post.jsonl.gz"), "x", Token);
Assert.Contains("Post: altered", await ServerRestore.Request(Context(), backup.Id, "admin", Token));
Assert.Null(RestoreMarker.Read(Backups));
}
// refused at boot before anything changed (the backup went meanwhile): recorded, and the server boots as it was
[Fact]
public async Task A_restore_that_cannot_start_is_abandoned_and_changes_nothing()
{
var backup = await TownBackedUp();
await LifeGoesOn();
Assert.Null(await ServerRestore.Request(Context(), backup.Id, "admin", Token));
ServerBackup.Delete(Backups, backup.Id);
Assert.Equal(RestoreOutcome.Abandoned, await ServerRestore.ApplyPending(Context(), NullLogger.Instance, Token));
Assert.Equal("second words", (await One("Post", Id(_editedPost)))["Text"].AsString);
Assert.Null(RestoreMarker.Read(Backups));
Assert.DoesNotContain(ServerBackup.List(Backups), b => b.Kind == "pre-restore");
var record = await (await _database.GetCollection<RestoreRecord>(nameof(RestoreRecord)).FindAsync(FilterDefinition<RestoreRecord>.Empty, cancellationToken: Token)).FirstAsync(Token);
Assert.True(record.Abandoned);
Assert.Contains("no such backup", record.Error);
}
[Fact]
public async Task After_three_failures_it_waits_for_someone_to_look()
{
var backup = await TownBackedUp();
await LifeGoesOn();
new RestoreMarker { Backup = backup.Id, PreRestore = "gone", State = "running", Attempts = RestoreMarker.MaxAttempts, Error = "disk full" }.Write(Backups);
Assert.Equal(RestoreOutcome.GaveUp, await ServerRestore.ApplyPending(Context(), NullLogger.Instance, Token));
Assert.Equal("second words", (await One("Post", Id(_editedPost)))["Text"].AsString);
Assert.NotNull(RestoreMarker.Read(Backups));
await File.WriteAllTextAsync(RestoreMarker.PathIn(Backups), "{ not json", Token);
Assert.Equal(RestoreOutcome.GaveUp, await ServerRestore.ApplyPending(Context(), NullLogger.Instance, Token));
}
// a command waits while a restore does; asking how it goes doesn't
[Fact]
public async Task Commands_wait_for_a_restore_asked_for()
{
var host = await PrivaPubHost.Shared();
var backups = host.Get<Backups>();
new RestoreMarker { Backup = "20260101-000000-manual", RequestedBy = "cli" }.Write(backups.Root);
try
{
Assert.Equal(MaintenanceGate.TryLater, await MaintenanceGate.Enter(host.Services, ["promote", "someone"], Token));
Assert.Null(await MaintenanceGate.Enter(host.Services, ["restore", "--status"], Token));
var output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(["restore", "--status"], host.Services, output: output));
Assert.Contains("waiting: 20260101-000000-manual, pending", output.ToString());
Assert.Equal(1, await AdminCommands.Run(["restore", "20260101-000000-manual"], host.Services, output: TextWriter.Null));
}
finally
{
RestoreMarker.Clear(backups.Root);
}
}
// FEP-8fcf: for two weeks after a restore, no digests, and no follow undone for them
[Fact]
public async Task Followers_digests_rest_for_two_weeks_after_a_restore()
{
RestoreRecord.Forget();
Assert.False(await RestoreRecord.InFollowersGrace(Token));
var record = new RestoreRecord { Backup = "test", RestoredAt = DateTime.UtcNow.AddDays(-13) };
await DB.Default.SaveAsync(record, Token);
try
{
RestoreRecord.Forget();
Assert.True(await RestoreRecord.InFollowersGrace(Token));
await DB.Default.Update<RestoreRecord>().MatchID(record.ID).Modify(r => r.RestoredAt, DateTime.UtcNow.AddDays(-15)).ExecuteAsync(Token);
RestoreRecord.Forget();
Assert.False(await RestoreRecord.InFollowersGrace(Token));
}
finally
{
await DB.Default.DeleteAsync<RestoreRecord>(record.ID);
RestoreRecord.Forget();
}
}
}
}
@@ -0,0 +1,21 @@
using MongoDB.Entities;
using PrivaPub.Infrastructure.Data;
using PrivaPub.Tests.Support;
namespace PrivaPub.Tests.Infrastructure
{
[Trait("Category", "Integration")]
public sealed class TopologyTests
{
// tools/ci/with-test-mongod.sh starts a one-member replica set, as production's mongod is: the backup's snapshot reads
// are what the tests exercise
[Fact]
public async Task The_test_mongod_is_a_replica_set_like_productions()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
Assert.SkipUnless(MongoFixture.Connection.Contains("directConnection=true"), "a mongod not started by tools/ci/with-test-mongod.sh");
Assert.True(await MongoTopology.IsReplicaSet(DB.Default.Database(), TestContext.Current.CancellationToken));
}
}
}
@@ -0,0 +1,126 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statistics;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Statistics
{
public class StatisticsQueriesTests
{
[Fact]
public void Percentiles_come_from_the_latency_buckets()
{
var latency = new Dictionary<string, long> { ["out:le50ms"] = 50, ["out:le250ms"] = 45, ["out:le5000ms"] = 4, ["out:inf"] = 1, ["in:le50ms"] = 9 };
Assert.Equal(50, StatisticsQueries.Percentile(latency, "out", 0.5));
Assert.Equal(250, StatisticsQueries.Percentile(latency, "out", 0.95));
Assert.Equal(5000, StatisticsQueries.Percentile(latency, "out", 0.99));
Assert.Equal(30001, StatisticsQueries.Percentile(latency, "out", 1));
Assert.Null(StatisticsQueries.Percentile(latency, "http:actor", 0.5));
}
[Fact]
public void Outcomes_add_up_per_channel()
{
var counters = new Dictionary<string, long>
{
["out:Create:Note:ok"] = 3, ["out:Create:Note:retry:503"] = 2, ["out:Like:-:ok"] = 1, ["in:Create:Note:accepted:stored"] = 4, ["recv:Create:202:queued"] = 9
};
Assert.Equal(new Dictionary<string, long> { ["ok"] = 4, ["retry"] = 2 }, StatisticsQueries.Outcomes(counters, "out"));
Assert.Equal(new Dictionary<string, long> { ["accepted"] = 4 }, StatisticsQueries.Outcomes(counters, "in"));
}
}
[Trait("Category", "Integration")]
public sealed class AdminStatisticsTests : IAsyncLifetime
{
PrivaPubHost _host;
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
[Fact]
public async Task Only_the_admin_reads_the_statistics()
{
var token = TestContext.Current.CancellationToken;
var plain = await _host.SignUp();
using var anonymous = _host.Client();
using var user = _host.As(plain.Jwt);
Assert.Equal(HttpStatusCode.Unauthorized, (await anonymous.GetAsync("/clientapi/admin/statistics/overview", token)).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await user.GetAsync("/clientapi/admin/statistics/overview", token)).StatusCode);
Assert.Equal(HttpStatusCode.Forbidden, (await user.PostAsync("/clientapi/admin/statistics/rollups/2001-01-01", default, token)).StatusCode);
}
[Fact]
public async Task The_admin_sees_a_server_its_events_and_the_overview()
{
var token = TestContext.Current.CancellationToken;
var admin = await _host.Admin();
var persona = await _host.Persona(await _host.SignUp(), "stats");
var bob = new RemoteActor(_peer, "bob");
var noteId = $"{_peer.A}/notes/{Guid.NewGuid():N}";
var to = new JsonArray($"{PrivaPubHost.Base}/peasants/{persona.UserName}");
using (var client = _host.Client())
Assert.Equal(HttpStatusCode.Accepted, (await client.SendAsync(bob.SignedPost($"/peasants/{persona.UserName}/mouth", new JsonObject
{
["id"] = noteId + "/activity", ["type"] = "Create", ["actor"] = bob.Id, ["to"] = to.DeepClone(),
["object"] = new JsonObject { ["id"] = noteId, ["type"] = "Note", ["attributedTo"] = bob.Id, ["to"] = to.DeepClone(), ["content"] = "hi" }
}), token)).StatusCode);
await _host.RunInbox(noteId + "/activity", token);
await _host.Get<InteractionLedger>().Flush(token);
using var adminClient = _host.As(admin.Jwt);
var host = JsonNode.Parse(await adminClient.GetStringAsync("/clientapi/admin/statistics/hosts/127.0.0.1?days=1", token))!;
var overview = JsonNode.Parse(await adminClient.GetStringAsync("/clientapi/admin/statistics/overview?days=1", token))!;
var hosts = JsonNode.Parse(await adminClient.GetStringAsync("/clientapi/admin/statistics/hosts?days=1&sort=host", token))!;
var events = JsonNode.Parse(await adminClient.GetStringAsync("/clientapi/admin/statistics/events?host=127.0.0.1&channel=in&limit=5", token))!;
var today = host["days"]!.AsArray().Last()!;
Assert.True(today["live"]!.GetValue<bool>());
Assert.True(today["counters"]!["in:Create:Note:accepted:stored"]!.GetValue<long>() >= 1);
Assert.Contains("DB-IP", host["attribution"]!.GetValue<string>());
Assert.True(overview["channels"]!["in"]!.GetValue<long>() >= 1);
Assert.Contains(hosts["hosts"]!.AsArray(), h => h!["host"]!.GetValue<string>() == "127.0.0.1");
Assert.NotEmpty(events.AsArray());
Assert.DoesNotContain(persona.UserName, host.ToJsonString());
Assert.DoesNotContain(bob.Id, host.ToJsonString());
Assert.Equal(HttpStatusCode.NotFound, (await adminClient.GetAsync($"/clientapi/admin/statistics/hosts/never{Guid.NewGuid():N}.example", token)).StatusCode);
}
[Fact]
public async Task The_admin_can_refold_a_past_day_and_describe_a_server_again()
{
var token = TestContext.Current.CancellationToken;
var admin = await _host.Admin();
using var client = _host.As(admin.Jwt);
var describe = $"described{Guid.NewGuid():N}.example";
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostAsync($"/clientapi/admin/statistics/rollups/{DateTime.UtcNow:yyyy-MM-dd}", default, token)).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostAsync("/clientapi/admin/statistics/rollups/yesterday", default, token)).StatusCode);
Assert.Equal(HttpStatusCode.Accepted, (await client.PostAsync("/clientapi/admin/statistics/rollups/2001-01-05", default, token)).StatusCode);
Assert.Equal(HttpStatusCode.Accepted, (await client.PostAsync($"/clientapi/admin/statistics/hosts/{describe}/describe", default, token)).StatusCode);
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.RollupDay && j.Payload == "2001-01-05" && j.DedupeKey.StartsWith("rollup|2001-01-05|rerun|")).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.DescribeInstance && j.Payload == describe).ExecuteAnyAsync(token));
}
}
}
+276
View File
@@ -0,0 +1,276 @@
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Federation.Crawler;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Objects;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
using System.Text;
namespace PrivaPub.Tests.Statistics
{
public class RobotsTxtTests
{
const string Robots = """
# comments are ignored
User-agent: *
Disallow: /private
Allow: /private/ok
User-agent: PrivaPub-Stargazer
User-agent: other
Disallow: /api/
Allow: /api/v1/instance$
User-agent: greedy
Disallow: /*.json$
""";
[Fact]
public void The_crawlers_own_group_wins_and_the_longest_rule_decides()
{
var mine = RobotsTxt.Parse(Robots, "PrivaPub-Stargazer");
var anyone = RobotsTxt.Parse(Robots, "SomethingElse");
var greedy = RobotsTxt.Parse(Robots, "greedy");
Assert.False(mine.Allows("/api/v1/instance/peers"));
Assert.True(mine.Allows("/api/v1/instance"));
Assert.True(mine.Allows("/private"));
Assert.True(mine.Allows("/.well-known/nodeinfo"));
Assert.False(anyone.Allows("/private/nope"));
Assert.True(anyone.Allows("/private/ok/yes"));
Assert.True(anyone.Allows("/api/v1/instance/peers"));
Assert.False(greedy.Allows("/nodeinfo/2.1.json"));
Assert.True(greedy.Allows("/nodeinfo/2.1.json?x=1"));
}
[Fact]
public void An_error_keeps_the_crawler_out_and_a_missing_file_lets_it_in()
{
Assert.True(RobotsTxt.From(404, default, Stargazer.Token).Allows("/anything"));
Assert.False(RobotsTxt.From(503, default, Stargazer.Token).Allows("/anything"));
Assert.False(RobotsTxt.From(0, default, Stargazer.Token).Allows("/anything"));
Assert.False(RobotsTxt.From(200, "User-agent: *\nDisallow: /", Stargazer.Token).Allows("/robots.txt"));
Assert.True(RobotsTxt.From(200, "User-agent: *\nDisallow:", Stargazer.Token).Allows("/x"));
}
[Fact]
public void Peer_lists_yield_only_names_we_could_ever_reach()
{
var names = new[] { "Social.Example.org.", "10.0.0.1", "host.example.org:8443", "x.onion", "printer.local", "bücher.example.org", "a.test", "noDot", "social.example.org" };
Assert.Equal(new[] { "social.example.org", "xn--bcher-kva.example.org" }, PeerHosts.Filter(names, _ => true));
Assert.Contains("a.test", PeerHosts.Filter(names, _ => true, testNetwork: true));
Assert.Empty(PeerHosts.Filter(names, _ => false));
}
[Fact]
public void A_truncated_peers_list_still_gives_what_was_read()
{
var items = new List<string>();
FederationHttp.ReadStrings(Encoding.UTF8.GetBytes("[\"a.example\", 3, {\"x\":1}, \"b.example\", \"c.exa"), 10, items);
var capped = new List<string>();
FederationHttp.ReadStrings(Encoding.UTF8.GetBytes("[\"a\",\"b\",\"c\"]"), 2, capped);
Assert.Equal(new[] { "a.example", "b.example" }, items);
Assert.Equal(new[] { "a", "b" }, capped);
}
}
[Trait("Category", "Integration")]
public sealed class CrawlerTests : IAsyncLifetime
{
Peer _peer;
readonly string _run = Guid.NewGuid().ToString("N")[..10];
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
sealed class PeerDescriber : InstanceDescriber
{
readonly string _peer;
public PeerDescriber(IFederationHttp http, string peer) : base(http) => _peer = peer;
protected override string Address(string url) => $"{_peer}/{new Uri(url).Host}{new Uri(url).PathAndQuery}";
}
sealed class PeerCrawler : InstanceCrawler
{
readonly string _peer;
public PeerCrawler(IFederationHttp http, InstanceDescriber describer, IDomainBlocks blocks, CrawlerOptions crawler, string peer, MemoryLedger ledger)
: base(http, describer, blocks, new StaticOptions<StatisticsOptions>(new StatisticsOptions { Crawler = crawler }),
new StaticOptions<AppConfiguration>(new AppConfiguration { BackendBaseAddress = "https://privapub.test" }),
new StaticOptions<FederationOptions>(new FederationOptions { AllowPrivateNetworks = true }), ledger) => _peer = peer;
protected override string Address(string url) => $"{_peer}/{new Uri(url).Host}{new Uri(url).PathAndQuery}";
}
(PeerCrawler Crawler, MemoryLedger Ledger) Crawler(CrawlerOptions options, IDomainBlocks blocks = default)
{
var http = Peer.Http();
var ledger = new MemoryLedger();
return (new PeerCrawler(http, new PeerDescriber(http, _peer.A), blocks ?? new NoBlocks(), options, _peer.A, ledger), ledger);
}
string Host(string name) => $"{name}{_run}.example.org";
void Serve(string host, string robots = default, int robotsStatus = 200, params string[] peers)
{
if (robots != default)
_peer.ServeText($"/{host}/robots.txt", robots, "text/plain");
else
_peer.Answer($"/{host}/robots.txt", robotsStatus);
_peer.Serve($"/{host}/.well-known/nodeinfo", $$"""{"links":[{"rel":"http://nodeinfo.diaspora.software/ns/schema/2.0","href":"https://{{host}}/nodeinfo/2.0"}]}""");
_peer.Serve($"/{host}/nodeinfo/2.0", """{"software":{"name":"lemmy","version":"0.19.9"},"usage":{"users":{"total":3}}}""");
_peer.Serve($"/{host}/api/v1/instance/peers", "[" + string.Join(",", peers.Select(p => $"\"{p}\"")) + "]");
}
[Fact]
public async Task A_crawl_describes_a_crawled_server_and_learns_its_peers_politely()
{
var token = TestContext.Current.CancellationToken;
var host = Host("seed");
var blocked = Host("blocked");
Serve(host, "User-agent: *\nDisallow: /private", 200, Host("new1"), Host("new2"), blocked, "10.0.0.1", Host("new1"));
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
blocks.Load(new[] { new DomainBlock { Domain = blocked, Severity = DomainBlockSeverity.Silence } });
await DB.Default.SaveAsync(new RemoteInstance { Host = host, Seen = "crawled" }, token);
var (crawler, ledger) = Crawler(new CrawlerOptions { Enabled = true }, blocks);
await crawler.Handle(new Job { Kind = JobKind.CrawlInstance, Payload = host }, token);
var seed = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteSingleAsync(token);
Assert.Equal(("lemmy", 5, false), (seed.Software, seed.PeersCount!.Value, seed.RobotsDisallowed!.Value));
Assert.NotNull(seed.LastCrawledAt);
var found = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == Host("new1") || i.Host == Host("new2") || i.Host == blocked).ExecuteAsync(token);
Assert.Equal(new[] { Host("new1"), Host("new2") }, found.Select(i => i.Host).OrderBy(h => h));
Assert.All(found, i => Assert.Equal("crawled", i.Seen));
var visited = _peer.Requests.Where(r => r.Path.StartsWith("/" + host + "/")).ToList();
Assert.All(visited, r => Assert.Contains(r.Path[(host.Length + 1)..], new[] { "/robots.txt", "/.well-known/nodeinfo", "/nodeinfo/2.0", "/api/v1/instance/peers" }));
Assert.All(visited, r => Assert.StartsWith("PrivaPub-Stargazer/", r.Headers["User-Agent"]));
Assert.All(visited, r => Assert.Contains("/stargazing)", r.Headers["User-Agent"]));
Assert.Equal(("crawl", "ok"), (Assert.Single(ledger.Events).Event.Channel, ledger.Events.Single().Event.Outcome));
}
[Fact]
public async Task Robots_txt_and_its_absence_are_obeyed_and_a_touched_server_is_not_described_again()
{
var token = TestContext.Current.CancellationToken;
var shy = Host("shy");
var broken = Host("broken");
var touched = Host("touched");
Serve(shy, "User-agent: PrivaPub-Stargazer\nDisallow: /api/", 200, Host("hidden"));
Serve(broken, default, 503, Host("never"));
Serve(touched, "", 200);
await DB.Default.SaveAsync(new[] { new RemoteInstance { Host = shy, Seen = "crawled" }, new RemoteInstance { Host = touched, Seen = "touched" } }, token);
var (crawler, ledger) = Crawler(new CrawlerOptions { Enabled = true });
await crawler.Handle(new Job { Payload = shy }, token);
await crawler.Handle(new Job { Payload = broken }, token);
await crawler.Handle(new Job { Payload = touched }, token);
Assert.DoesNotContain(_peer.Requests, r => r.Path == $"/{shy}/api/v1/instance/peers");
Assert.Contains(_peer.Requests, r => r.Path == $"/{shy}/nodeinfo/2.0");
Assert.Equal(new[] { $"/{broken}/robots.txt" }, _peer.Requests.Where(r => r.Path.StartsWith($"/{broken}/")).Select(r => r.Path));
Assert.True((await DB.Default.Find<RemoteInstance>().Match(i => i.Host == broken).ExecuteSingleAsync(token)).RobotsDisallowed);
Assert.DoesNotContain(_peer.Requests, r => r.Path.StartsWith($"/{touched}/.well-known/"));
Assert.Contains(ledger.Events, e => e.Event.Host == broken && e.Event.Reason == "robots-disallowed");
Assert.False(await DB.Default.Find<RemoteInstance>().Match(i => i.Host == Host("hidden") || i.Host == Host("never")).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_disabled_crawler_does_nothing_and_the_caps_hold()
{
var token = TestContext.Current.CancellationToken;
var host = Host("capped");
Serve(host, "", 200, Enumerable.Range(0, 20).Select(i => Host($"many{i}-")).ToArray());
await DB.Default.SaveAsync(new RemoteInstance { Host = host, Seen = "touched" }, token);
await Crawler(new CrawlerOptions { Enabled = false }).Crawler.Handle(new Job { Payload = host }, token);
Assert.DoesNotContain(_peer.Requests, r => r.Path.StartsWith($"/{host}/"));
await Crawler(new CrawlerOptions { Enabled = true, MaxNewHostsPerCrawl = 3 }).Crawler.Handle(new Job { Payload = host }, token);
Assert.Equal(3, await DB.Default.CountAsync<RemoteInstance>(i => i.Host.StartsWith("many") && i.Host.EndsWith($"{_run}.example.org"), cancellation: token));
}
[Fact]
public async Task The_planner_spreads_due_servers_across_the_hour_and_skips_fresh_and_blocked_ones()
{
var token = TestContext.Current.CancellationToken;
var due = Host("due");
var fresh = Host("fresh");
var blocked = Host("plannedblock");
await DB.Default.SaveAsync(new[]
{
new RemoteInstance { Host = due, Seen = "crawled", LastCrawledAt = new DateTime(2001, 1, 1, 0, 0, 0, DateTimeKind.Utc) },
new RemoteInstance { Host = fresh, Seen = "crawled", LastCrawledAt = DateTime.UtcNow },
new RemoteInstance { Host = blocked, Seen = "crawled", LastCrawledAt = new DateTime(2001, 1, 1, 0, 0, 0, DateTimeKind.Utc) }
}, token);
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
blocks.Load(new[] { new DomainBlock { Domain = blocked, Severity = DomainBlockSeverity.Suspend } });
var options = new StatisticsOptions { Crawler = new CrawlerOptions { Enabled = true, HostsPerHour = 600, Seeds = new() { Host("seedling"), "10.1.1.1" } } };
await new CrawlPlanner(new JobQueue(j => j.DedupeKey == "never"), blocks, new StaticOptions<StatisticsOptions>(options)).Handle(new Job(), token);
await new CrawlPlanner(new JobQueue(j => j.DedupeKey == "never"), blocks, new StaticOptions<StatisticsOptions>(new StatisticsOptions())).Handle(new Job(), token);
var planned = (await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.CrawlInstance && j.Payload.EndsWith($"{_run}.example.org")).ExecuteAsync(token)).Select(j => j.Payload).ToList();
Assert.Contains(due, planned);
Assert.Contains(Host("seedling"), planned);
Assert.DoesNotContain(fresh, planned);
Assert.DoesNotContain(blocked, planned);
Assert.Equal("crawled", (await DB.Default.Find<RemoteInstance>().Match(i => i.Host == Host("seedling")).ExecuteSingleAsync(token)).Seen);
}
}
[Trait("Category", "Integration")]
public sealed class StargazingPageTests : IAsyncLifetime
{
PrivaPubHost _host;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_host = await PrivaPubHost.Shared();
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
[Fact]
public async Task The_page_names_the_crawler_says_it_is_off_and_how_to_keep_it_out()
{
using var client = _host.Client();
using var response = await client.GetAsync("/stargazing", TestContext.Current.CancellationToken);
var page = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.Contains("PrivaPub-Stargazer/", page);
Assert.Contains("https://privapub.test/stargazing", page);
Assert.Contains("<strong>off</strong>", page);
Assert.Contains("Disallow: /", page);
Assert.Contains("DB-IP", page);
Assert.StartsWith("default-src 'none'", response.Headers.GetValues("Content-Security-Policy").Single());
}
}
}
+332
View File
@@ -0,0 +1,332 @@
using Microsoft.Extensions.Logging.Abstractions;
using MongoDB.Entities;
using PrivaPub.Domain.Statistics;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Objects;
using PrivaPub.Infrastructure.Data.Migrations;
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Statistics;
using PrivaPub.Tests.Support;
using System.Net;
using System.Text.Json;
namespace PrivaPub.Tests.Statistics
{
public class InstanceDocumentsTests
{
static JsonElement Json(string text) => JsonDocument.Parse(text).RootElement;
[Fact]
public void NodeInfo_gives_software_usage_and_languages_whatever_their_shape()
{
var facts = InstanceDocuments.NodeInfo(Json("""
{"software":{"name":"mastodon","version":"4.7.3"},"protocols":["activitypub"],"openRegistrations":false,
"usage":{"users":{"total":"1234","activeMonth":56.0,"activeHalfyear":-3},"localPosts":99999999999,"localComments":7},
"metadata":{"nodeName":"Example","nodeDescription":"<p>A <b>small</b>\n place</p>","languages":["en","DE","klingon!"]}}
"""), "2.1");
Assert.Equal(("mastodon", "4.7.3", "Example", "2.1"), (facts.Software, facts.SoftwareVersion, facts.NodeName, facts.Version));
Assert.Equal("A small place", facts.NodeDescription);
Assert.Equal(new[] { "activitypub" }, facts.Protocols);
Assert.False(facts.OpenRegistrations);
Assert.Equal(1234, facts.UsersTotal);
Assert.Equal(56, facts.UsersActiveMonth);
Assert.Equal(0, facts.UsersActiveHalfyear);
Assert.Equal(1_000_000_000, facts.LocalPosts);
Assert.Equal(7, facts.LocalComments);
Assert.Equal(new[] { "en", "de" }, facts.Languages);
Assert.Null(InstanceDocuments.NodeInfo(Json("{}"), "2.0").UsersTotal);
// NodeInfo wants the name in lower case, which Mobilizon does not keep
Assert.Equal("mobilizon", InstanceDocuments.NodeInfo(Json("""{"software":{"name":"Mobilizon"}}"""), "2.1").Software);
}
// Castopod publishes NodeInfo2 only
[Fact]
public void NodeInfo2_gives_the_server_and_its_usage()
{
var facts = InstanceDocuments.NodeInfo2(Json("""
{"version":"1.0","server":{"baseUrl":"https://castopod.test/","name":"Castopod","software":"Castopod","version":"1.15.5"},
"protocols":["activitypub"],"openRegistrations":false,"usage":{"users":{"total":1,"activeMonth":1,"activeHalfyear":1},"localPosts":3}}
"""));
Assert.Equal(("castopod", "1.15.5", "Castopod", "nodeinfo2-1.0"), (facts.Software, facts.SoftwareVersion, facts.NodeName, facts.Version));
Assert.Equal(new[] { "activitypub" }, facts.Protocols);
Assert.Equal((1L, 3L), (facts.UsersTotal.Value, facts.LocalPosts.Value));
Assert.Null(InstanceDocuments.NodeInfo2(Json("""{"version":"1.0","server":{}}""")));
}
[Fact]
public void The_newest_https_nodeinfo_link_is_chosen()
{
var links = Json("""
{"links":[{"rel":"http://nodeinfo.diaspora.software/ns/schema/2.0","href":"https://a.example/2.0"},
{"rel":"http://nodeinfo.diaspora.software/ns/schema/2.1","href":"http://a.example/2.1"},
{"rel":"http://nodeinfo.diaspora.software/ns/schema/2.2","href":"https://a.example/2.2"}]}
""");
Assert.Equal(("https://a.example/2.2", "2.2"), InstanceDocuments.NodeInfoLink(links));
Assert.Equal(default, InstanceDocuments.NodeInfoLink(Json("[]")));
// Funkwhale 2.0 links its NodeInfo under its swagger schema's URL
Assert.Equal(("https://f.example/api/v2/instance/nodeinfo/2.1", "2.1"), InstanceDocuments.NodeInfoLink(Json("""
{"links":[{"rel":"https://docs.funkwhale.audio/swagger/schema.yml","href":"https://f.example/api/v2/instance/nodeinfo/2.1"}]}
""")));
}
[Fact]
public void The_instance_api_gives_registration_mode_limits_and_never_a_contact()
{
var v2 = InstanceDocuments.InstanceApi(Json("""
{"title":"Example","languages":["en"],"registrations":{"enabled":true,"approval_required":true},
"configuration":{"statuses":{"max_characters":500}},"api_versions":{"mastodon":7},"source_url":"https://github.com/mastodon/mastodon",
"contact":{"email":"admin@a.example","account":{"username":"admin"}}}
"""));
var v1 = InstanceDocuments.InstanceApi(Json("""{"title":"Old","registrations":false,"max_toot_chars":"5000"}"""));
Assert.Equal(("Example", "approval", 500, 7, "https://github.com/mastodon/mastodon"), (v2.Title, v2.RegistrationMode, v2.MaxPostChars!.Value, v2.ApiVersion!.Value, v2.SourceUrl));
Assert.Equal(new[] { "en" }, v2.Languages);
Assert.DoesNotContain("admin", System.Text.Json.JsonSerializer.Serialize(v2));
Assert.Equal(("closed", 5000), (v1.RegistrationMode, v1.MaxPostChars!.Value));
Assert.True(InstanceDocuments.SpeaksMastodonApi("GoToSocial"));
Assert.False(InstanceDocuments.SpeaksMastodonApi("lemmy"));
}
[Fact]
public void The_public_location_is_where_the_server_was_located_unless_a_cdn_hides_it()
{
var geo = new InstanceGeo { Country = "DE", City = "Berlin", Latitude = 52.5, Longitude = 13.4, AsnOrg = "Hetzner", Asn = 24940 };
Assert.Equal(new PublicLocation("DE", "Berlin", 52.5, 13.4, "Hetzner", null), PublicGeo.Project(geo));
Assert.Equal(new PublicLocation("DE", null, null, null, null, null), PublicGeo.Project(new InstanceGeo { Country = "DE" }));
Assert.Equal(new PublicLocation(null, null, null, null, null, "Cloudflare"),
PublicGeo.Project(new InstanceGeo { Country = "US", City = "SF", Latitude = 37.8, Longitude = -122.4, Cdn = "Cloudflare" }));
Assert.Null(PublicGeo.Project(new InstanceGeo()));
Assert.Null(PublicGeo.Project(null));
}
}
[Trait("Category", "Integration")]
public sealed class DescribeTests : IAsyncLifetime
{
Peer _peer;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_peer = await Peer.Start();
}
public async ValueTask DisposeAsync()
{
if (_peer != default)
await _peer.DisposeAsync();
}
sealed class PeerDescriber : InstanceDescriber
{
readonly string _peer;
public PeerDescriber(IFederationHttp http, IGeoLocator geo, IConnectedAddresses addresses, string peer, ICdnRanges ranges = default,
IEdgeHints edges = default) : base(http, geo, addresses, ranges, edges) => _peer = peer;
protected override string Address(string url) => _peer + new Uri(url).PathAndQuery;
}
sealed class FixedGeo : IGeoLocator
{
public GeoFix Locate(IPAddress address) => new("NL", "Amsterdam", 52.4, 4.9, 60781, "LeaseWeb");
public string Source => "test";
}
void ServeServer(string prefix, string software, bool v2 = true)
{
_peer.Serve($"{prefix}/.well-known/nodeinfo", $$"""{"links":[{"rel":"http://nodeinfo.diaspora.software/ns/schema/2.1","href":"https://ignored.example/nodeinfo/2.1"}]}""");
_peer.Serve($"{prefix}/nodeinfo/2.1", $$$"""{"software":{"name":"{{{software}}}","version":"1.0"},"usage":{"users":{"total":42,"activeMonth":7}},"openRegistrations":true}""");
if (v2)
_peer.Serve($"{prefix}/api/v2/instance", """{"title":"A test server","languages":["it"],"registrations":{"enabled":true,"approval_required":false}}""");
_peer.Serve($"{prefix}/api/v1/instance", """{"title":"An old test server","registrations":true}""");
}
[Fact]
public async Task A_server_is_described_located_and_snapshotted_weekly()
{
var token = TestContext.Current.CancellationToken;
var host = $"described{Guid.NewGuid():N}.example";
ServeServer("/" + host, "mastodon");
var addresses = new ConnectedAddresses();
var describer = new PeerDescriber(Peer.Http(connected: addresses), new FixedGeo(), addresses, _peer.A + "/" + host);
Assert.True(await describer.Describe(host, crawled: false, default, token));
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteSingleAsync(token);
Assert.Equal(("mastodon", 42L, 7L, "2.1"), (instance.Software, instance.UsersTotal!.Value, instance.UsersActiveMonth!.Value, instance.NodeInfoVersion));
Assert.Equal(("A test server", "open", "mastodon-v2"), (instance.Title, instance.RegistrationMode, instance.InstanceApi));
Assert.Equal(new[] { "it" }, instance.Languages);
Assert.Equal(("NL", "Amsterdam", "LeaseWeb", "test"), (instance.Geo.Country, instance.Geo.City, instance.Geo.AsnOrg, instance.Geo.Source));
Assert.False(instance.Geo.Ipv6);
var snapshot = await DB.Default.Find<RemoteInstanceSnapshot>().Match(s => s.Host == host).ExecuteSingleAsync(token);
Assert.True(snapshot.Reachable);
Assert.Equal((42L, "NL"), (snapshot.UsersTotal!.Value, snapshot.Country));
Assert.Equal(InstanceDescriber.Week(DateTime.UtcNow), snapshot.Week);
}
[Fact]
public async Task A_cdn_is_found_by_its_published_ranges_first_then_by_its_headers()
{
var token = TestContext.Current.CancellationToken;
var listed = $"listed{Guid.NewGuid():N}.example";
var hinted = $"hinted{Guid.NewGuid():N}.example";
ServeServer("/" + listed, "mastodon");
ServeServer("/" + hinted, "mastodon");
var addresses = new ConnectedAddresses();
var ranges = new CdnRanges();
ranges.Replace(new CdnRangeSet { Key = "cloudflare.com", Name = "Cloudflare", Ranges = new() { "127.0.0.0/8" }, FetchedAt = DateTime.UtcNow });
var edges = new EdgeHints();
edges.Remember("127.0.0.1", CdnCatalog.ByKey("fastly.com"));
await new PeerDescriber(Peer.Http(connected: addresses), new FixedGeo(), addresses, _peer.A + "/" + listed, ranges, edges).Describe(listed, crawled: false, default, token);
await new PeerDescriber(Peer.Http(connected: addresses), new FixedGeo(), addresses, _peer.A + "/" + hinted, new CdnRanges(), edges).Describe(hinted, crawled: false, default, token);
var listedOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == listed).ExecuteSingleAsync(token);
Assert.Equal(("Cloudflare", "cloudflare.com", "ranges"), (listedOne.Geo.Cdn, listedOne.Geo.CdnDomain, listedOne.Geo.CdnSource));
var hintedOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == hinted).ExecuteSingleAsync(token);
Assert.Equal(("Fastly", "fastly.com", "headers"), (hintedOne.Geo.Cdn, hintedOne.Geo.CdnDomain, hintedOne.Geo.CdnSource));
var snapshot = await DB.Default.Find<RemoteInstanceSnapshot>().Match(s => s.Host == listed).ExecuteSingleAsync(token);
Assert.Equal(("Cloudflare", "cloudflare.com", "ranges", "Amsterdam", 52.4), (snapshot.Cdn, snapshot.CdnDomain, snapshot.CdnSource, snapshot.City, snapshot.Latitude!.Value));
}
[Fact]
public async Task Without_api_v2_the_v1_api_is_read_and_an_unreachable_week_is_recorded_too()
{
var token = TestContext.Current.CancellationToken;
var old = $"old{Guid.NewGuid():N}.example";
var gone = $"gone{Guid.NewGuid():N}.example";
ServeServer("/" + old, "pleroma", v2: false);
await new PeerDescriber(Peer.Http(), default, default, _peer.A + "/" + old).Describe(old, crawled: false, default, token);
Assert.False(await new PeerDescriber(Peer.Http(), default, default, _peer.A + "/" + gone).Describe(gone, crawled: false, default, token));
var oldOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == old).ExecuteSingleAsync(token);
Assert.Equal(("mastodon-v1", "An old test server", "open"), (oldOne.InstanceApi, oldOne.Title, oldOne.RegistrationMode));
Assert.Null(oldOne.Geo);
var goneOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == gone).ExecuteSingleAsync(token);
Assert.Equal("no NodeInfo", goneOne.DescriptionError);
Assert.False((await DB.Default.Find<RemoteInstanceSnapshot>().Match(s => s.Host == gone).ExecuteSingleAsync(token)).Reachable);
}
// a server whose NodeInfo failed is asked again on its next arrival, not a week later
[Fact]
public async Task A_failed_description_frees_its_weeks_key_and_a_good_one_keeps_it()
{
var token = TestContext.Current.CancellationToken;
var broken = $"broken{Guid.NewGuid():N}.example";
var fine = $"fine{Guid.NewGuid():N}.example";
ServeServer("/" + fine, "smithereen", v2: false);
var queue = new JobQueue();
async Task Run(string host)
{
var key = InstanceDescriber.DedupeKey(host, DateTime.UtcNow);
Assert.True(await queue.Enqueue(JobKind.DescribeInstance, host, host, key, token));
var job = await DB.Default.Find<Job>().Match(j => j.DedupeKey == key).ExecuteSingleAsync(token);
await new PeerDescriber(Peer.Http(), default, default, _peer.A + "/" + host).Handle(job, token);
}
await Run(broken);
await Run(fine);
Assert.True(await queue.Enqueue(JobKind.DescribeInstance, broken, broken, InstanceDescriber.DedupeKey(broken, DateTime.UtcNow), token));
Assert.False(await queue.Enqueue(JobKind.DescribeInstance, fine, fine, InstanceDescriber.DedupeKey(fine, DateTime.UtcNow), token));
}
[Fact]
public async Task A_crawled_server_never_downgrades_a_touched_one_and_robots_are_obeyed()
{
var token = TestContext.Current.CancellationToken;
var touched = $"touched{Guid.NewGuid():N}.example";
var shy = $"shy{Guid.NewGuid():N}.example";
ServeServer("/" + touched, "gotosocial");
ServeServer("/" + shy, "mastodon");
await DB.Default.SaveAsync(new RemoteInstance { Host = touched, Seen = "touched" }, token);
await new PeerDescriber(Peer.Http(), default, default, _peer.A + "/" + touched).Describe(touched, crawled: true, default, token);
await new PeerDescriber(Peer.Http(), default, default, _peer.A + "/" + shy).Describe(shy, crawled: true, path => !path.StartsWith("/api/"), token);
Assert.Equal("touched", (await DB.Default.Find<RemoteInstance>().Match(i => i.Host == touched).ExecuteSingleAsync(token)).Seen);
var shyOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == shy).ExecuteSingleAsync(token);
Assert.Equal(("crawled", "mastodon"), (shyOne.Seen, shyOne.Software));
Assert.Null(shyOne.InstanceApi);
Assert.DoesNotContain(_peer.Requests, r => r.Path.StartsWith("/" + shy + "/api/", StringComparison.Ordinal));
Assert.Contains(_peer.Requests, r => r.Path == "/" + shy + "/nodeinfo/2.1");
}
}
[Trait("Category", "Integration")]
public sealed class TouchTests : IAsyncLifetime
{
public ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
return ValueTask.CompletedTask;
}
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
[Fact]
public async Task A_server_we_exchange_with_is_seen_and_described_once_a_week_unless_suspended()
{
var token = TestContext.Current.CancellationToken;
var friend = $"friend{Guid.NewGuid():N}.example";
var spammer = $"spam{Guid.NewGuid():N}.example";
var linked = $"linked{Guid.NewGuid():N}.example";
var looked = $"looked{Guid.NewGuid():N}.example";
var blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
blocks.Load(new[] { new DomainBlock { Domain = spammer, Severity = DomainBlockSeverity.Suspend } });
var ledger = new InteractionLedger(new InteractionSalts(), new StaticOptions<StatisticsOptions>(new StatisticsOptions()), NullLogger<InteractionLedger>.Instance,
new JobQueue(j => j.DedupeKey == "never"), blocks);
ledger.Record(new InteractionEvent { Host = friend, Channel = "in", Activity = "Create", Outcome = "accepted" }, "https://" + friend + "/users/a");
ledger.Record(new InteractionEvent { Host = friend, Channel = "out", Activity = "Like", Outcome = "ok" });
ledger.Record(new InteractionEvent { Host = spammer, Channel = "out", Activity = "Reject", Outcome = "ok" });
ledger.Record(new InteractionEvent { Host = linked, Channel = "preview", Outcome = "ok" });
ledger.Count(looked, "http:actor:ok");
await ledger.Flush(token);
var seen = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == friend).ExecuteSingleAsync(token);
Assert.Equal("touched", seen.Seen);
Assert.NotNull(seen.FirstSeenAt);
Assert.Single(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.DescribeInstance && j.Payload == friend).ExecuteAsync(token));
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.DescribeInstance && j.Payload == looked).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.DescribeInstance && j.Payload == spammer).ExecuteAnyAsync(token));
Assert.False(await DB.Default.Find<RemoteInstance>().Match(i => i.Host == linked).ExecuteAnyAsync(token));
}
}
[Xunit.Collection(nameof(Exclusive))]
[Trait("Category", "Integration")]
public sealed class TouchedMigrationTests
{
[Fact]
public async Task Known_servers_become_touched_with_their_dates()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var token = TestContext.Current.CancellationToken;
var host = $"legacy{Guid.NewGuid():N}.example";
var success = DateTime.UtcNow.AddDays(-2);
await DB.Default.SaveAsync(new RemoteInstance { Host = host, LastSuccessAt = success, DescribedAt = success.AddDays(-1) }, token);
await new _010_remote_instances_were_touched().UpgradeAsync();
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteSingleAsync(token);
Assert.Equal("touched", instance.Seen);
Assert.NotNull(instance.FirstSeenAt);
Assert.InRange(instance.LastSeenAt!.Value, success.AddSeconds(-1), success.AddSeconds(1));
}
}
}
Loaded 100 of 474 files, more files were not shown because too many files have changed in this diff. Show more