Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
436f7da464 | ||
|
|
5d7edc4a4c | ||
|
|
eb7552e8f3 | ||
|
|
31d1d21d1e | ||
|
|
d9fb5c582f | ||
|
|
2cd75176a4 | ||
|
|
c7e8760ffe | ||
|
|
ad8d353f9a | ||
|
|
e2f61ede56 | ||
|
|
8f25bf056d | ||
|
|
123ba39520 | ||
|
|
f6dbf71964 | ||
|
|
8c2eba6cbb | ||
|
|
fcd35f5043 |
No files matched your search
@@ -95,6 +95,11 @@ jobs:
|
||||
[ "$served" = "$BUILD_COMMIT" ] || { echo "::error::served build is '$served', expected '$BUILD_COMMIT'"; exit 1; }
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/privapub")
|
||||
[ "$code" = "200" ] || { echo "::error::the instance actor answered $code"; exit 1; }
|
||||
# SecureMode (owner decision 2026-10-04): an unsigned reader gets 401 from a persona, a browser the public page
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/thepra")
|
||||
[ "$code" = "401" ] || { echo "::error::an unsigned GET of a persona answered $code, not 401: SecureMode is off"; exit 1; }
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' -H 'Accept: text/html' "$PUBLIC_URL/peasants/thepra")
|
||||
[ "$code" = "302" ] || { echo "::error::a browser asking for a persona got $code, not a redirect"; exit 1; }
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/.well-known/nodeinfo")
|
||||
[ "$code" = "200" ] || { echo "::error::nodeinfo answered $code"; exit 1; }
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' "$PUBLIC_URL/swagger/index.html")
|
||||
@@ -123,7 +128,7 @@ jobs:
|
||||
read -r token cid cs <<< "$(tools/smoke/oauth.sh "$PUBLIC_URL" "$login" "$password" thepra read)"
|
||||
echo "::add-mask::$token"
|
||||
tools/smoke/mastodon-api.sh "$PUBLIC_URL" "$token"
|
||||
discoverable=$(curl -fsS -H 'Accept: application/activity+json' "$PUBLIC_URL/peasants/thepra" | python3 -c "import json,sys; print(json.load(sys.stdin).get('discoverable'))")
|
||||
discoverable=$(curl -fsS -H "Authorization: Bearer $token" "$PUBLIC_URL/api/v1/accounts/verify_credentials" | python3 -c "import json,sys; print(json.load(sys.stdin).get('discoverable'))")
|
||||
[ "$discoverable" = "False" ] || { echo "::error::@thepra is discoverable"; exit 1; }
|
||||
curl -fsS -o /dev/null -X POST "$PUBLIC_URL/oauth/revoke" --data-urlencode "token=$token" \
|
||||
--data-urlencode "client_id=$cid" --data-urlencode "client_secret=$cs"
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# CLAUDE.md
|
||||
# CLAUDE.md
|
||||
|
||||
Guidance for working in this repository. `docs/ROADMAP.md` holds the owner's decisions and the phased plan to full
|
||||
ActivityPub interop; read it before changing anything federation-, privacy- or API-shaped. `docs/INTEROP.md` is the
|
||||
@@ -29,8 +29,8 @@ Privacy features in the model:
|
||||
|
||||
The intended client is the Mastodon client API (Tusky, Elk, Phanpy, Ivory), where each avatar logs in as its own
|
||||
account. The private `/clientapi` covers what Mastodon can't express. `thepra/decePubClient`
|
||||
(https://decepub.thepra.dev) is the owner's own Pleroma-FE-like PWA. It still shows mock data and is not wired to the
|
||||
server.
|
||||
(https://decepub.thepra.dev) is the owner's own Pleroma-FE-like PWA. It signs in once on `/clientapi` and exchanges that
|
||||
JWT for each persona's Mastodon token (`PersonaExchange`), then uses both APIs.
|
||||
|
||||
## Route names are deliberate
|
||||
|
||||
@@ -182,8 +182,17 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
||||
`ContentFormat` says what `Text` holds. Remote names are plain text.
|
||||
8. **Circles federate to members only.** A circle is an undiscoverable Group actor that takes follow requests (the owner
|
||||
approves); its posts are addressed to the circle and its `/flock`, delivered to members' personal inboxes, never
|
||||
announced, and served only to a signed request from a member or a member's instance actor
|
||||
(`SignedFetchAuthorizer`), 404 otherwise. Circles never appear in search, lookups, mentions or profile pages.
|
||||
announced. Each member's copy also names that member in `cc` (`OutboxPublisher.Naming`, owner decision
|
||||
2026-10-04), because Mastodon and GoToSocial keep a post only when it names one of their accounts; Create, every
|
||||
Update and the Delete all go through `OutboxPublisher.Publish` for that. A reply to a circle post stays in the circle,
|
||||
and a circle post never asks a non-member for a quote. Circles never appear in search, lookups, mentions or profile
|
||||
pages.
|
||||
**A post that is not public is served only to a signed request from someone it was for** (`SignedFetchAuthorizer.MayRead`):
|
||||
a follower or an addressed account for followers-only, an addressed account for a DM, a member for a circle, or the
|
||||
instance actor of a server where one of them lives; 404 to anyone else, 410 to them once it is deleted. A circle
|
||||
refetch names the requesting member, or the members on the requesting server. A DM's `context`
|
||||
(`/peasants/{name}/whispers/{id}`) lists the conversation's posts for its participants. Mastodon deletes its copy
|
||||
when a refetch answers 404, which is why this matters.
|
||||
**Communities** are FEP-1b12 groups: `GroupDistributor` announces the whole activity (plus the object for new posts,
|
||||
for Mastodon), top-level posts are `Page`s with a `name`, posting follows `Group.PostingPolicy`.
|
||||
Located posts (`LocalGeo`) are the only local-only posts.
|
||||
@@ -192,19 +201,32 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
||||
10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`):
|
||||
leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers
|
||||
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`).
|
||||
11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`, and
|
||||
11. **Every number about posts and users goes through `Domain/Privacy/Counted`** (owner decision 2026-10-04: one
|
||||
answer everywhere): a persona's `statuses_count`, its outbox `totalItems`, NodeInfo `localPosts` and the instance
|
||||
`status_count` all count Mastodon's way (not deleted, not a DM, boosts included, circle and located posts too);
|
||||
users exclude personas of banned or deleted roots; `replies_count` counts only public and unlisted replies; a
|
||||
remote account that deletes itself takes its likes, votes, reactions, boosts and replies out of every count
|
||||
(`GoneActors`). Follower and following counts are public, their members never are (`hide_collections` is always
|
||||
true).
|
||||
12. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`, and
|
||||
any other read of stored posts filters by `IsShown`. All three hide deleted posts and the posts of a remote account
|
||||
that deleted itself (`Post.AuthorGone`: kept, hidden everywhere, owner decision).
|
||||
12. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
|
||||
13. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
|
||||
mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is
|
||||
fetched as context. Followers-only is detected by the author's stored `followers` URL.
|
||||
13. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and
|
||||
14. **Home timelines are written, not computed:** every stored or created post goes through `Fanout.Distribute`, and
|
||||
every delete removes its `TimelineEntry` rows. Local deletes are soft (content cleared, 410 Tombstone).
|
||||
|
||||
## Mastodon client API invariants
|
||||
|
||||
1. **A token is one persona.** Its subject is the avatar id; the root id lives only in the fifteen-minute `/oauth`
|
||||
cookie used while choosing the persona, and never in a token, an authorization or a response.
|
||||
- **The first-party client exchanges the root JWT for one persona's token** (RFC 8693 on `/oauth/token`,
|
||||
`subject_token_type` `urn:ietf:params:oauth:token-type:jwt`, `avatar_id`; `Api/Mastodon/Auth/PersonaExchange.cs`).
|
||||
Only the seeded public application (`ClientApi:FirstPartyClientId`, default `decepub`) holds the grant.
|
||||
`RootJwtSubjectToken` validates the JWT through `RootJwt`, the same rules as JwtBearer (signature, lifetime, ban,
|
||||
deletion, session stamp). The issued token is the same kind as the authorization code flow's, so revocation and
|
||||
`RootSessions` cover it.
|
||||
2. `/api/*` authenticates with OpenIddict validation, everything else with the old JWT (`PrivaPub` policy scheme).
|
||||
Every `/api` request re-checks that the persona's root is neither banned nor deleted (`MastodonController`).
|
||||
3. Read parameters through `Params` (query, form and JSON merged Rails-style), never MVC binding. A value type read
|
||||
@@ -214,9 +236,12 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
||||
error that leaves without a body (a challenge, a 404 from routing, a 429) gets Mastodon's `{"error": ...}` from
|
||||
`UseMastodonErrorBodies`. `NeverFiveHundredTests` walks every route with junk ids, anonymously, as a persona and
|
||||
with a junk token.
|
||||
6. Advertise `4.2.0 (compatible; PrivaPub)` until grouped notifications exist.
|
||||
6. Advertise `4.2.0 (compatible; PrivaPub)` until streaming and Web Push exist too; `api_versions.mastodon = 7` stays,
|
||||
because clients show quotes from it, so everything a 4.3+ client calls because of it must answer: grouped
|
||||
notifications (`/api/v2/notifications`, its unread count, groups, accounts and dismiss) and the notification policy.
|
||||
7. **What a Mastodon `Status` cannot say goes in `Status.privapub`** (`PrivaPubStatus`): object type, title, excerpt,
|
||||
cover, the author's source, link, video, audio and event details, and up/down votes. Every media URL in it goes
|
||||
cover, the author's source, link, video, audio and event details, a remote post's own place (Pixelfed's `location`,
|
||||
never federated again), and up/down votes. Every media URL in it goes
|
||||
through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following
|
||||
one is the reader's choice.
|
||||
|
||||
@@ -243,6 +268,16 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
||||
|
||||
- **No root id in federation output, NodeInfo or logs, no IP next to an identity in logs, and no `ex.Message` to a
|
||||
client** ("Something went wrong." instead).
|
||||
- **Sign-in and recovery never tell which logins or addresses exist** (owner decision 2026-10-04): every failed sign-in
|
||||
gets "That username and password do not match." after the same hashing (`RootUsersService.Decoy`, a constant-time
|
||||
comparison), and only a correct password learns of a ban. Every recovery request answers the same sentence and
|
||||
queues a `SendRecovery` job, found or not; `RecoveryJob` sends the email and keeps only a SHA-256 of the code, for an
|
||||
hour. A recovered password ends every session of the root (`RootSessions`: a new `SessionStamp`, which every `/clientapi`
|
||||
JWT carries, and OpenIddict revocation for each persona).
|
||||
- **Deleting a root deletes everything public it had** (`RootRemoval`, from the admin route or `/clientapi/user/delete`
|
||||
with the password): its sessions end, each persona and each group it owns sends `Delete{Actor}` to followers, members
|
||||
and the accounts it follows, the personas' posts are emptied, and `/peasants/{name}`, its inbox and WebFinger answer
|
||||
410 (`LocalActorService.Gone`) while the names stay reserved. The root keeps only a `deleted-{id}` name.
|
||||
- **Never derive a public name from the root username.** Invitation sign-up takes `AvatarUserName` and refuses one
|
||||
equal to the login.
|
||||
- **One username space:** personas, groups and the instance reserve their name in `ReservedName` (unique index) before
|
||||
@@ -285,7 +320,19 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
||||
revision. Delivery details reach them through `Arrival.Current`, which `InboxProcessor` sets for the handler's
|
||||
duration. A fetched record has no signature, key or `@context` of the activity that caused the fetch (its activity
|
||||
fields name that trigger), and every record stores its `Extensions` and `ContextNamespaces` for statistics. The raw form lives outside `Post` so timelines never load it. Read through
|
||||
`/api/privapub/v1/statuses/:id/provenance` and `/api/privapub/v1/instances/:host`.
|
||||
`/api/privapub/v1/statuses/:id/provenance` and `/api/privapub/v1/instances/:host` (or `?host[]=`, up to 40; this
|
||||
server answers too).
|
||||
- **A server's place leaves only through `PublicGeo.Project`:** the city, coordinates (0.1°) and network it was located
|
||||
to, whatever its size (there is no user threshold), and only the CDN's name for a CDN-fronted one. This server's own
|
||||
place is its host's address located once a day (`SelfLocation`), or `Statistics:Geo:Self` when the owner sets it.
|
||||
- **CDNs are found three ways, best first** (`Infrastructure/Geo/CdnCatalog.cs`): the ranges a CDN publishes
|
||||
(`CdnUpdater` downloads them daily into `CdnRangeSet`, keeping the old set when a download or parse fails), the CDN's
|
||||
fingerprint in the responses PrivaPub already gets (`EdgeHintsHandler` on the federation client, memory only), and
|
||||
a network that carries nothing but a CDN. Never list an ASN shared with plain hosting (AWS, Google, Microsoft,
|
||||
DataPacket). A server's `Geo` records the CDN, its domain and how it was found.
|
||||
- **Servers through time** (`Domain/Statistics/ServerPlaces.cs`, from the weekly `RemoteInstanceSnapshot`s):
|
||||
`/api/privapub/v1/instances/:host/history`, a CDN-fronted server's `geo.before_cdn`, and the CDNs grouped by their
|
||||
own domain (`/api/privapub/v1/cdns`, `/cdns/:domain` with weekly joined/left).
|
||||
- **Remote objects are parsed for every shape in `ObjectShapes`:** `url`/`icon`/`image` as a value, an object or an
|
||||
array; Markdown `content`; missing `mediaType`s inferred; thumbnails from any of their five places; and the typed
|
||||
`Link`, `Video`, `Audio` and `Event` details. A Mastodon API card is built from those, never by fetching the linked page
|
||||
@@ -412,14 +459,17 @@ tools/pasture/run.sh down # removes e
|
||||
delete is checked as a 404 on `/api/v1/statuses/{id}`.
|
||||
- It creates its accounts locked, so the scenario approves alice's request through `/api/v1/follow_requests`, which
|
||||
also checks our pending (`requested`) state and the manual Accept.
|
||||
- 37 checks: discovery and follows both ways; posts and CW; a reply and its notification; likes and boosts both
|
||||
- 55 checks: discovery and follows both ways; posts and CW; a reply and its notification; likes and boosts both
|
||||
ways; DMs both ways and off public timelines; polls both ways; quote policy; link cards; edits and deletes both
|
||||
ways; unfollow; block and unblock; statistics.
|
||||
ways; communities and circles; locked personas; unfollow; block and unblock; statistics.
|
||||
- It files a circle post as a direct message and shows it only to the accounts it mentions, so like a DM it is
|
||||
checked in the member's `/api/v1/conversations`, never by URI.
|
||||
- **Mastodon (4.7.3):** web and sidekiq on the shared Postgres and Redis, `ALLOWED_PRIVATE_ADDRESSES` for the network.
|
||||
Its token comes from `rails runner` (no password grant). Without Elasticsearch its status search finds nothing, so
|
||||
deliveries are checked through `/api/v1/accounts/:id/statuses` of the sender as Mastodon knows them, or
|
||||
`Status.exists?` through `rails runner`. Its actors are numbered (`/ap/users/<id>`), so look URIs up rather than
|
||||
build them. 49 checks; circle posts are an expected failure (see `docs/INTEROP.md`, Mastodon).
|
||||
build them. Circle posts and a followers-only post survive its signed refetch (`ActivityPub::FetchRemoteStatusService`
|
||||
through `rails runner`). Inbound Block is the one expected failure until P7.
|
||||
- **Misskey (2026.10.0):** one container on the shared Postgres and Redis. A new Misskey federates with nobody
|
||||
(`federation: none`) until `admin/update-meta` says `all`, which `misskey_up` does. Its API is `POST /api/<endpoint>`
|
||||
with the token as `i` (`mk` in the scenario); `users/relation` answers a list, `users/notes` leaves replies out unless
|
||||
@@ -441,6 +491,9 @@ tools/pasture/run.sh down # removes e
|
||||
its own activity and every bare `Announce{object}` 400 by design, and the echo is still needed (see
|
||||
`docs/INTEROP.md`, Lemmy). A new Lemmy never sends what it queued for a server before its send worker for that
|
||||
server started, so the scenario waits for that worker (`lm_worker`) before its first follow. 20 checks; relayed votes and a moderator's removal are expected failures (P7).
|
||||
- **SecureMode:** `PRIVAPUB_ENV="Federation__SecureMode=true" run.sh up …`, as production runs. A check of what an
|
||||
unsigned reader sees uses `unserved` and `gone_unsigned` (`lib/interop.sh`), which expect 401 when SecureMode is on and
|
||||
404 or 410 when it is off.
|
||||
- **Crawler:** `PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test"
|
||||
run.sh up mastodon`, then `interop.sh crawler`. `PRIVAPUB_ENV` passes any setting to the PrivaPub container.
|
||||
- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into
|
||||
|
||||
+54
-22
@@ -12,17 +12,29 @@ PrivaPub is an ActivityPub server written in C#. This document follows
|
||||
|
||||
## Tested against
|
||||
|
||||
- **GoToSocial 0.22.1, end to end.** It runs in a private network on the workstation (`tools/pasture/`) and is driven
|
||||
through its own client API. Checked both ways:
|
||||
- follows, including to a locked account;
|
||||
- public posts with a content warning;
|
||||
- replies with notifications;
|
||||
- likes and boosts;
|
||||
- direct messages;
|
||||
- edits and deletes;
|
||||
- unfollow.
|
||||
- **Mastodon, Misskey, Lemmy and PeerTube, by unit tests only.** The tests feed the parser documents written in each
|
||||
server's shape. No live exchange with any of them has run yet.
|
||||
End to end, in a private network on the workstation (`tools/pasture/`), each peer driven through its own client API
|
||||
and every run starting clean, with signed fetches required (as privapub.thepra.dev runs):
|
||||
- **GoToSocial 0.22.1**
|
||||
- **Mastodon 4.7.3**
|
||||
- **Misskey 2026.10**
|
||||
- **Sharkey 2025.4.7**
|
||||
- **Akkoma 3.20.1**
|
||||
- **Lemmy 1.0.0-beta.2**
|
||||
|
||||
Checked both ways, where the peer has the feature:
|
||||
- follows, locked accounts included;
|
||||
- every visibility, content warnings, replies and their notifications;
|
||||
- likes, boosts and emoji reactions with their undos;
|
||||
- direct messages;
|
||||
- polls;
|
||||
- quotes;
|
||||
- images with alt text;
|
||||
- edits with history and deletes;
|
||||
- communities and circles;
|
||||
- blocks and unfollows.
|
||||
|
||||
`docs/INTEROP.md` has each peer's evidence and what is still expected to fail. PeerTube, PieFed, Mbin and the others are
|
||||
covered by unit tests written in their documents' shape.
|
||||
|
||||
## Supported FEPs
|
||||
|
||||
@@ -30,11 +42,16 @@ PrivaPub is an ActivityPub server written in C#. This document follows
|
||||
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
|
||||
- [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md)
|
||||
- [FEP-1b12: Group federation](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md) (communities; see "Groups")
|
||||
- [FEP-044f: Consent-respecting quote posts](https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md)
|
||||
(`QuoteAuthorization` at `/parrot-licences/{id}`; checked with Mastodon both ways)
|
||||
- [FEP-9967: Polls](https://codeberg.org/fediverse/fep/src/branch/main/fep/9967/fep-9967.md)
|
||||
- [FEP-c0e0: Emoji reactions](https://codeberg.org/fediverse/fep/src/branch/main/fep/c0e0/fep-c0e0.md) (`EmojiReact`, and Misskey's
|
||||
`Like` with content)
|
||||
- [FEP-5feb: Search indexing consent](https://codeberg.org/fediverse/fep/src/branch/main/fep/5feb/fep-5feb.md) (`indexable`)
|
||||
|
||||
Planned (see `docs/ROADMAP.md`, phases P5 to P8, and the per-platform notes in `docs/INTEROP.md`): FEP-044f (quotes),
|
||||
FEP-9967 (polls), FEP-c0e0 (emoji reactions), FEP-9098 (custom emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move),
|
||||
FEP-8fcf (followers synchronisation), FEP-5feb (`indexable`), FEP-8967 (link attachments), FEP-521a and FEP-8b32 (keys
|
||||
and integrity proofs), FEP-ae0c (relays).
|
||||
Planned (see `docs/ROADMAP.md`, phases P7 and P8, and the per-platform notes in `docs/INTEROP.md`): FEP-9098 (custom
|
||||
emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), FEP-8fcf (followers synchronisation), FEP-8967 (link
|
||||
attachments), FEP-521a and FEP-8b32 (keys and integrity proofs), FEP-ae0c (relays).
|
||||
|
||||
## Actors
|
||||
|
||||
@@ -58,7 +75,8 @@ The names are the project's own and are stable; resolve actors through WebFinger
|
||||
|
||||
- The key is `{actor}#main-key`, RSA 2048, served as SPKI PEM with `owner` set to the actor.
|
||||
- `published` on an actor is a whole day, chosen at random up to two weeks before the account was made, so two
|
||||
personas made on the same day do not share a date. `indexable` is `false`.
|
||||
personas made on the same day do not share a date. `indexable` is `false` unless the persona turns it on, and
|
||||
`discoverable` is `true` unless it turns that off (the deploy's own persona, @thepra, is undiscoverable).
|
||||
- The instance actor is `/peasants/privapub` (type `Application`). It signs every fetch PrivaPub makes, so no avatar's key
|
||||
is used to read another server's content.
|
||||
|
||||
@@ -73,8 +91,10 @@ A group is either a **community** or a **circle**.
|
||||
top-level post is a `Page` with a `name`. Members are counted at `/flock`; moderators are listed at `/wardens`, which
|
||||
the actor's `attributedTo` points to, with `postingRestrictedToMods` as Lemmy expects. A mention of a community posts into it.
|
||||
- A **circle** is private. Its actor is not discoverable and every follow is a request. Its posts are addressed to the
|
||||
circle and its members collection, delivered to each member's own inbox and never announced. They are served only
|
||||
to a signed request from a member, or from the instance actor of a member's server; anyone else gets 404. A
|
||||
circle and its members collection, delivered to each member's own inbox and never announced. Each member's copy also
|
||||
names that member in `cc`, so servers that keep only posts naming one of their accounts (Mastodon, GoToSocial) keep
|
||||
it; it names no other member. The posts are served only to a signed request from a member, or from the instance actor
|
||||
of a member's server, and that copy names the member (or the members on that server); anyone else gets 404. A
|
||||
Mastodon member's replies reach only the people they mention.
|
||||
- Announces from **remote** groups (Lemmy communities) are followed through to the activity: the object is fetched
|
||||
from its own origin, never taken from the announce.
|
||||
@@ -192,12 +212,22 @@ Posts with a location (shown to nearby users of this server) never leave the ser
|
||||
deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin.
|
||||
- **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three
|
||||
redirects and read at most 1 MB.
|
||||
- **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like
|
||||
Mastodon's authorized fetch; the instance actor `/peasants/privapub` is the exception, since its key is needed first.
|
||||
A browser asking for HTML is redirected to the public page instead.
|
||||
- **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were
|
||||
for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted
|
||||
they answer those same readers 410. A direct message's `context`, `/peasants/{name}/whispers/{id}`, is an
|
||||
`OrderedCollection` of the conversation's posts for its participants.
|
||||
- **HTML.** Received HTML is sanitised to Mastodon's allowlist.
|
||||
- **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered
|
||||
5xx, the inbox answers 503 with `Retry-After: 300` rather than 401.
|
||||
- **Activity ids.** `Create` and `Announce` ids dereference. `Follow`, `Like`, `Block` and the `Accept`, `Reject` and
|
||||
`Undo` that answer them do not, because serving them would reveal who follows, likes and blocks whom; they are always
|
||||
sent with their object embedded.
|
||||
- **Activity ids.** `Create` and `Announce` ids dereference, a community's announces included, until the post they
|
||||
carry is deleted (then 410); a persona's boost id sends a browser to the boosted post. `Follow`, `Like`, `Block` and
|
||||
the `Accept`, `Reject` and `Undo` that answer them do not, because serving them would reveal who follows, likes and
|
||||
blocks whom. Neither do `Update`, `Delete`, `EmojiReact`, `QuoteRequest` and its answers, `Flag`, `Ignore` or poll
|
||||
votes. All of them are always sent with their object embedded.
|
||||
- **Hashtags.** A post's `Hashtag` links go to `/tags/{tag}`, a public page of this server's public posts with that tag.
|
||||
- **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
|
||||
starting at an hour and growing to a week.
|
||||
|
||||
@@ -206,5 +236,7 @@ Posts with a location (shown to nearby users of this server) never leave the ser
|
||||
- Our own posts carry no custom emoji: emoji are received and shown, not offered.
|
||||
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
|
||||
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
|
||||
- Collections expose counts, not members.
|
||||
- Collections expose counts, not members: `/groupies` and `/stalking` give the same totals as the Mastodon API's
|
||||
follower and following counts, and the outbox's `totalItems` is the persona's post count, though only public posts
|
||||
are listed. `/api/v1/instance/peers` is empty: which servers this one talks to is not published.
|
||||
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
|
||||
@@ -0,0 +1,14 @@
|
||||
using PrivaPub.ClientModels.Resources;
|
||||
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
|
||||
namespace PrivaPub.ClientModels.User
|
||||
{
|
||||
// Deleting one's own login, and with it every persona and group it owns, asks for the password again.
|
||||
public class RemoveSelfForm
|
||||
{
|
||||
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
|
||||
Display(Name = "Password", ResourceType = typeof(FieldsNameResource))]
|
||||
public string Password { get; set; }
|
||||
}
|
||||
}
|
||||
@@ -60,6 +60,7 @@ namespace PrivaPub.Tests.Domain
|
||||
{
|
||||
public string BaseAddress => Base;
|
||||
|
||||
public Task<GoneActor> Gone(string userName, CancellationToken token) => Task.FromResult<GoneActor>(default);
|
||||
public Task<LocalActor> FindByUserName(string userName, CancellationToken token) =>
|
||||
Task.FromResult(userName.Equals("alice", StringComparison.OrdinalIgnoreCase)
|
||||
? new LocalActor { Id = "a1", UserName = "alice", BaseAddress = Base, Kind = LocalActorKind.Person }
|
||||
|
||||
@@ -9,6 +9,7 @@ using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Group;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.Social;
|
||||
using PrivaPub.StaticServices;
|
||||
using PrivaPub.Tests.Support;
|
||||
|
||||
using System.Text.Json.Nodes;
|
||||
@@ -138,21 +139,86 @@ namespace PrivaPub.Tests.Federation
|
||||
Assert.Equal(PostVisibility.Circle, outcome.Post.Visibility);
|
||||
var create = Assert.Single(await _harness.Outgoing(member.Id + "/inbox"));
|
||||
Assert.Equal(new[] { circle.Uri, circle.Flock }, create["object"]!["to"]!.AsArray().Select(t => t!.GetValue<string>()));
|
||||
// the member's copy names the member, so Mastodon and GoToSocial keep it, and names nobody else
|
||||
Assert.Equal(new[] { member.Id }, create["cc"]!.AsArray().Select(t => t!.GetValue<string>()));
|
||||
Assert.Equal(new[] { member.Id }, create["object"]!["cc"]!.AsArray().Select(t => t!.GetValue<string>()));
|
||||
Assert.Contains(create["object"]!["tag"]!.AsArray(), t => t!["type"]!.GetValue<string>() == "Mention" && t["href"]!.GetValue<string>() == member.Id);
|
||||
Assert.DoesNotContain(Addressing.Public, create.ToJsonString());
|
||||
Assert.Empty(await _harness.Outgoing(follower.SharedInbox));
|
||||
Assert.Empty((await _harness.Outgoing(member.SharedInbox)).Where(a => a["type"]!.GetValue<string>() == "Announce"));
|
||||
|
||||
var authorizer = new SignedFetchAuthorizer(_harness.Remote);
|
||||
var authorizer = new SignedFetchAuthorizer(_harness.Remote, new DbEntities());
|
||||
var path = new Uri(outcome.Post.ObjectURI).AbsolutePath;
|
||||
var asMember = await authorizer.Requester(member.Get(Harness.Host, path), token);
|
||||
var asOutsider = await authorizer.Requester(outsider.Get(Harness.Host, path), token);
|
||||
Assert.True(SignedFetchAuthorizer.MayReadCircle(circleEntity, asMember));
|
||||
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, asOutsider));
|
||||
Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, default));
|
||||
Assert.True(await authorizer.MayRead(outcome.Post, asMember, token));
|
||||
Assert.False(await authorizer.MayRead(outcome.Post, asOutsider, token));
|
||||
Assert.Equal(new[] { member.Id }, SignedFetchAuthorizer.CircleReaders(circleEntity, asMember));
|
||||
Assert.True(circle.IsCircle);
|
||||
Assert.False(circle.Discoverable);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_circle_posts_edit_and_delete_reach_each_member_naming_only_that_member()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var (_, alice) = await _harness.Persona("alice");
|
||||
var first = new RemoteActor(_harness.Peer, "first");
|
||||
var second = new RemoteActor(_harness.Peer, "second", _harness.Peer.B);
|
||||
var (_, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id, first.Id, second.Id);
|
||||
await _harness.Remote.GetActor(first.Id, refresh: false, token);
|
||||
await _harness.Remote.GetActor(second.Id, refresh: false, token);
|
||||
|
||||
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
|
||||
await _harness.Statuses.Edit(alice, outcome.Post.ID, new StatusDraft { Text = "just us, edited" }, token);
|
||||
await _harness.Statuses.Remove(alice, outcome.Post.ID, token);
|
||||
|
||||
foreach (var (member, other) in new[] { (first, second), (second, first) })
|
||||
{
|
||||
var sent = await _harness.Outgoing(member.Id + "/inbox");
|
||||
Assert.Equal(new[] { "Create", "Update", "Delete" }, sent.Select(a => a["type"]!.GetValue<string>()));
|
||||
Assert.All(sent, a => Assert.Contains(member.Id, a["cc"]!.AsArray().Select(c => c!.GetValue<string>())));
|
||||
Assert.All(sent, a => Assert.DoesNotContain(other.Id, a.ToJsonString()));
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_reply_to_a_circle_post_stays_in_it_and_a_circle_post_asks_nobody_outside_for_a_quote()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var (_, alice) = await _harness.Persona("alice");
|
||||
var (_, bob) = await _harness.Persona("bob");
|
||||
var (entity, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id);
|
||||
entity.Members.Add(new GroupMember { AvatarId = bob.Id });
|
||||
await DB.Default.SaveAsync(entity, token);
|
||||
var outsider = new RemoteActor(_harness.Peer, "asked");
|
||||
var outsiderAccount = await _harness.Remote.GetActor(outsider.Id, refresh: false, token);
|
||||
var asksFirst = new Post
|
||||
{
|
||||
ObjectURI = $"{Origin(outsider)}/notes/{Guid.NewGuid():N}",
|
||||
ActorURI = outsider.Id,
|
||||
GroupUserId = outsiderAccount.ID,
|
||||
AuthorAccountId = outsiderAccount.ID,
|
||||
IsFederatedCopy = true,
|
||||
Visibility = PostVisibility.Public,
|
||||
ContentHtml = "<p>ask me first</p>",
|
||||
QuotePolicy = new InteractionRule { Manual = new() { Addressing.Public } }
|
||||
};
|
||||
await DB.Default.SaveAsync(asksFirst, token);
|
||||
|
||||
var root = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token);
|
||||
var reply = await _harness.Statuses.Publish(bob, new StatusDraft { Text = "still just us", InReplyTo = root.Post.ID }, token);
|
||||
var quote = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "look", GroupId = circle.Id, QuotedStatusId = asksFirst.ID }, token);
|
||||
|
||||
Assert.Equal(PostVisibility.Circle, reply.Post.Visibility);
|
||||
Assert.Equal(circle.Id, reply.Post.GroupId);
|
||||
Assert.Equal(422, quote.Status);
|
||||
Assert.Empty(await _harness.Outgoing(outsider.Id + "/inbox"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Only_circle_members_can_post_into_a_circle()
|
||||
{
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using PrivaPub.Api.Mastodon.Mappers;
|
||||
using PrivaPub.Api.Mastodon.Mappers;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Models.Post;
|
||||
|
||||
@@ -79,6 +79,51 @@ namespace PrivaPub.Tests.Federation
|
||||
Assert.Equal("2026-10-03 10:00 – 13:00 (Europe/Rome) · Biblioteca", MastodonMapper.EventLine(note.Event));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Keeps_a_pixelfed_place_whose_coordinates_are_strings()
|
||||
{
|
||||
var note = Parse("""
|
||||
{
|
||||
"id": "https://pixel.example/p/ann/1", "type": "Note", "attributedTo": "https://pixel.example/users/ann", "content": "<p>view</p>",
|
||||
"location": { "type": "Place", "name": "Tokyo", "longitude": "139.6917", "latitude": "35.6895", "country": "Japan" },
|
||||
"commentsEnabled": true
|
||||
}
|
||||
""");
|
||||
|
||||
Assert.Equal(("Tokyo", 35.6895, 139.6917, "Japan"), (note.Place.Name, note.Place.Latitude, note.Place.Longitude, note.Place.Country));
|
||||
Assert.Null(note.Event);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("null")]
|
||||
[InlineData("""{ "type": "Place", "name": "Nowhere" }""")]
|
||||
[InlineData("""{ "type": "Place", "name": "Off the map", "latitude": 123, "longitude": 9 }""")]
|
||||
[InlineData("""{ "type": "Place", "latitude": "north", "longitude": "9.19" }""")]
|
||||
[InlineData("""{ "type": "VirtualLocation", "url": "https://meet.example/a", "latitude": 45, "longitude": 9 }""")]
|
||||
public void Has_no_place_without_a_place_with_coordinates(string location)
|
||||
{
|
||||
var note = Parse($$"""
|
||||
{ "id": "https://pixel.example/p/ann/2", "type": "Note", "attributedTo": "https://pixel.example/users/ann", "content": "<p>x</p>", "location": {{location}} }
|
||||
""");
|
||||
|
||||
Assert.Null(note.Place);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void An_events_location_stays_the_events_own()
|
||||
{
|
||||
var note = Parse("""
|
||||
{
|
||||
"id": "https://events.example/events/10", "type": "Event", "name": "Picnic", "startTime": "2026-10-03T10:00:00+02:00",
|
||||
"location": { "type": "Place", "name": "Park", "latitude": 45.07, "longitude": 7.68 },
|
||||
"attributedTo": "https://events.example/@club"
|
||||
}
|
||||
""");
|
||||
|
||||
Assert.Null(note.Place);
|
||||
Assert.Equal("Park", Assert.Single(note.Event.Places).Name);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Keeps_a_lemmy_link_post_as_a_link_with_its_thumbnail()
|
||||
{
|
||||
|
||||
@@ -6,6 +6,8 @@ using MongoDB.Entities;
|
||||
using PrivaPub.ClientModels;
|
||||
using PrivaPub.Models.Group;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.Services;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Tests.Support;
|
||||
using PrivaPub.Tests.Support.Host;
|
||||
|
||||
@@ -136,9 +138,11 @@ namespace PrivaPub.Tests.Http
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, ok.StatusCode);
|
||||
Assert.Equal(root.Id, (await ok.JsonBody())["userId"]!.GetValue<string>());
|
||||
// a wrong password and an unknown login get the same answer, so sign-in tells nobody which logins exist
|
||||
Assert.Equal(HttpStatusCode.BadRequest, wrong.StatusCode);
|
||||
Assert.Equal("Wrong password.", await Message(wrong));
|
||||
Assert.Equal(RootUsersService.NoMatch, await Message(wrong));
|
||||
Assert.Equal(HttpStatusCode.BadRequest, unknown.StatusCode);
|
||||
Assert.Equal(await Message(wrong), await Message(unknown));
|
||||
Assert.Equal(HttpStatusCode.OK, logout.StatusCode);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, anonymousLogout.StatusCode);
|
||||
}
|
||||
@@ -290,42 +294,59 @@ namespace PrivaPub.Tests.Http
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Recovery_without_an_email_is_refused()
|
||||
public async Task Recovery_answers_the_same_whoever_asks_and_queues_the_work()
|
||||
{
|
||||
var root = await _host.SignUp("noemail");
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var since = DateTime.UtcNow.AddSeconds(-1);
|
||||
var withEmail = await _host.SignUp("withemail");
|
||||
var withoutEmail = await _host.SignUp("noemail");
|
||||
var email = $"{Guid.NewGuid():N}@example.test";
|
||||
using (var signedIn = _host.As(withEmail.Jwt))
|
||||
Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode);
|
||||
using var client = _host.Client();
|
||||
|
||||
var response = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName });
|
||||
var unknown = await client.PostJson("/clientapi/user/recover/password", new { userName = Name("nobody") });
|
||||
var neither = await client.PostJson("/clientapi/user/recover/password", new { });
|
||||
var answers = new[]
|
||||
{
|
||||
await client.PostJson("/clientapi/user/recover/password", new { userName = withEmail.UserName }),
|
||||
await client.PostJson("/clientapi/user/recover/password", new { email }),
|
||||
await client.PostJson("/clientapi/user/recover/password", new { userName = withoutEmail.UserName }),
|
||||
await client.PostJson("/clientapi/user/recover/password", new { userName = Name("nobody") }),
|
||||
await client.PostJson("/clientapi/user/recover/password", new { email = $"{Guid.NewGuid():N}@example.test" })
|
||||
};
|
||||
var bodies = new List<string>();
|
||||
foreach (var answer in answers)
|
||||
{
|
||||
Assert.Equal(HttpStatusCode.OK, answer.StatusCode);
|
||||
bodies.Add(await answer.Content.ReadAsStringAsync(token));
|
||||
}
|
||||
|
||||
Assert.Equal(HttpStatusCode.Locked, response.StatusCode);
|
||||
Assert.Contains("doesn't have an email", await Message(response));
|
||||
Assert.False(await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken));
|
||||
Assert.Equal(HttpStatusCode.NotFound, unknown.StatusCode);
|
||||
Assert.Equal(HttpStatusCode.BadRequest, neither.StatusCode);
|
||||
Assert.Single(bodies.Distinct());
|
||||
Assert.Contains("recovery link is on its way", bodies[0]);
|
||||
var queued = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.SendRecovery && j.CreatedAt >= since).ExecuteAsync(token);
|
||||
Assert.True(queued.Count >= answers.Length);
|
||||
Assert.Contains(queued, j => j.Payload.Contains(withEmail.Id));
|
||||
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostJson("/clientapi/user/recover/password", new { })).StatusCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Recovery_through_an_unreachable_mail_server_says_so_and_nothing_more()
|
||||
public async Task The_recovery_job_keeps_only_a_hash_and_an_unreachable_mail_server_says_nothing_to_anyone()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var root = await _host.SignUp("smtp");
|
||||
var email = $"{Guid.NewGuid():N}@example.test";
|
||||
using (var signedIn = _host.As(root.Jwt))
|
||||
Assert.Equal(HttpStatusCode.OK, (await signedIn.PostJson("/clientapi/user/update", new { email })).StatusCode);
|
||||
using var client = _host.Client();
|
||||
|
||||
var byName = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName });
|
||||
var byEmail = await client.PostJson("/clientapi/user/recover/password", new { email });
|
||||
var answer = await client.PostJson("/clientapi/user/recover/password", new { userName = root.UserName });
|
||||
await _host.Run(j => j.Kind == JobKind.SendRecovery && j.Payload.Contains(root.Id), token);
|
||||
|
||||
foreach (var response in new[] { byName, byEmail })
|
||||
{
|
||||
var body = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
|
||||
Assert.Equal(HttpStatusCode.ServiceUnavailable, response.StatusCode);
|
||||
Assert.Equal("Failed to send email.", JsonNode.Parse(body)!["errorMessage"]!.GetValue<string>());
|
||||
foreach (var leak in new[] { "xception", "efused", "ocket", "127.0.0.1", "smtp", "SMTP", " at " })
|
||||
Assert.DoesNotContain(leak, body);
|
||||
}
|
||||
Assert.Equal(HttpStatusCode.OK, answer.StatusCode);
|
||||
var recovery = await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteFirstAsync(token);
|
||||
Assert.NotNull(recovery);
|
||||
Assert.Null(recovery.RecoveryCode);
|
||||
Assert.Equal(64, recovery.CodeHash.Length);
|
||||
Assert.InRange(recovery.ExpiresAt, DateTime.UtcNow.AddMinutes(50), DateTime.UtcNow.AddMinutes(61));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -348,7 +369,10 @@ namespace PrivaPub.Tests.Http
|
||||
{
|
||||
var root = await _host.SignUp("recover");
|
||||
var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N");
|
||||
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, RecoveryCode = code }, TestContext.Current.CancellationToken);
|
||||
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, CodeHash = RecoveryJob.Hash(code), ExpiresAt = DateTime.UtcNow.AddHours(1) },
|
||||
TestContext.Current.CancellationToken);
|
||||
var persona = await _host.Persona(root, "recovered");
|
||||
var apiToken = await _host.MastodonToken(persona);
|
||||
using var client = _host.Client();
|
||||
|
||||
var valid = await client.PostAsync("/clientapi/user/recover/valid", new StringContent($"\"{code}\"", Encoding.UTF8, "application/json"), TestContext.Current.CancellationToken);
|
||||
@@ -361,6 +385,29 @@ namespace PrivaPub.Tests.Http
|
||||
Assert.Equal(HttpStatusCode.BadRequest, (await LogIn(root.UserName, root.Password)).StatusCode);
|
||||
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, NewPassword)).StatusCode);
|
||||
Assert.False(await DB.Default.Find<EmailRecovery>().Match(r => r.RootUserId == root.Id).ExecuteAnyAsync(TestContext.Current.CancellationToken));
|
||||
// whoever held the old sessions may be why the password was recovered: they end
|
||||
using (var oldSession = _host.As(root.Jwt))
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, (await oldSession.GetAsync("/clientapi/user/sniff/again", TestContext.Current.CancellationToken)).StatusCode);
|
||||
using (var oldApp = _host.Client())
|
||||
{
|
||||
oldApp.DefaultRequestHeaders.Authorization = new("Bearer", apiToken);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, (await oldApp.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_expired_recovery_code_changes_nothing()
|
||||
{
|
||||
var root = await _host.SignUp("expired");
|
||||
var code = Guid.NewGuid().ToString("N") + Guid.NewGuid().ToString("N");
|
||||
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = root.Id, CodeHash = RecoveryJob.Hash(code), ExpiresAt = DateTime.UtcNow.AddMinutes(-1) },
|
||||
TestContext.Current.CancellationToken);
|
||||
using var client = _host.Client();
|
||||
|
||||
var change = await client.PostJson("/clientapi/user/recover/update/password", new { newPassword = NewPassword, repeatedPassword = NewPassword, recoveryCode = code });
|
||||
|
||||
Assert.Equal(HttpStatusCode.NotFound, change.StatusCode);
|
||||
Assert.Equal(HttpStatusCode.OK, (await LogIn(root.UserName, root.Password)).StatusCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
@@ -142,7 +142,9 @@ namespace PrivaPub.Tests.Http
|
||||
Assert.Equal(ActivityJson, outbox.MediaType);
|
||||
Assert.Equal("OrderedCollection", outbox.Json["type"]!.GetValue<string>());
|
||||
Assert.Equal(persona.ActorUri() + "/anus", outbox.Json["id"]!.GetValue<string>());
|
||||
Assert.Equal(22, outbox.Json["totalItems"]!.GetValue<int>());
|
||||
// statuses_count, as Mastodon counts it: everything but the DM, the deleted post and the copy of a remote post,
|
||||
// the followers-only and located posts and the boost included; only the public posts are listed
|
||||
Assert.Equal(24, outbox.Json["totalItems"]!.GetValue<int>());
|
||||
Assert.Null(outbox.Json["orderedItems"]);
|
||||
var first = outbox.Json["first"]!.GetValue<string>();
|
||||
Assert.Equal(persona.ActorUri() + "/anus?page=true", first);
|
||||
@@ -221,7 +223,7 @@ namespace PrivaPub.Tests.Http
|
||||
Assert.DoesNotContain(pending, followers.Text);
|
||||
Assert.Equal(HttpStatusCode.OK, following.Status);
|
||||
Assert.Equal(persona.ActorUri() + "/stalking", following.Json["id"]!.GetValue<string>());
|
||||
Assert.Equal(0, following.Json["totalItems"]!.GetValue<int>());
|
||||
Assert.Equal(1, following.Json["totalItems"]!.GetValue<int>());//counted, as following_count says, never listed
|
||||
Assert.DoesNotContain(friend.UserName, following.Text);
|
||||
}
|
||||
|
||||
@@ -322,11 +324,66 @@ namespace PrivaPub.Tests.Http
|
||||
Assert.Equal(new[] { circle.Uri, circle.Uri + "/flock" }, Strings(asMember.Json["to"]));
|
||||
Assert.DoesNotContain(ActivityPubRenderer.Public, asMember.Text);
|
||||
Assert.Equal(HttpStatusCode.OK, asMemberServer.Status);
|
||||
// a refetch names the member, as the member's copy did; the instance actor sees the members on its server only
|
||||
Assert.Equal(new[] { member.Id }, Strings(asMember.Json["cc"]));
|
||||
Assert.Equal(new[] { member.Id }, Strings(asMemberServer.Json["cc"]));
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path)).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path, Browser)).Status);
|
||||
foreach (var outsider in new[] { neighbour, stranger, strangerServer })
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(outsider.SignedGet(path))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch($"/peasants/{owner.UserName}/grunts/create-{post.ID}")).Status);
|
||||
var create = $"/peasants/{owner.UserName}/grunts/create-{post.ID}";
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(create)).Status);
|
||||
var createAsMember = await _client.Fetch(member.SignedGet(create));
|
||||
Assert.Equal(HttpStatusCode.OK, createAsMember.Status);
|
||||
Assert.Equal(post.ObjectURI, createAsMember.Json["object"]!["id"]!.GetValue<string>());
|
||||
Assert.Equal(new[] { member.Id }, Strings(createAsMember.Json["object"]!["cc"]));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Followers_only_and_direct_posts_are_served_to_signed_fetches_from_those_they_were_for()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var owner = await _host.Persona(await _host.SignUp(), "private");
|
||||
var follower = new RemoteActor(_peer, "follower");
|
||||
var followerServer = new RemoteActor(_peer, "instance", type: "Application");
|
||||
var recipient = new RemoteActor(_peer, "recipient");
|
||||
var stranger = new RemoteActor(_peer, "stranger", _peer.B);
|
||||
_peer.WebFinger(recipient);
|
||||
await DB.Default.SaveAsync(new Follower
|
||||
{
|
||||
LocalActorId = owner.Id,
|
||||
LocalActorKind = LocalActorKind.Person,
|
||||
ActorURI = follower.Id,
|
||||
InboxURL = follower.Id + "/inbox"
|
||||
}, token);
|
||||
var quiet = await _host.Publish(owner, "for followers", PostVisibility.FollowersOnly);
|
||||
var direct = await _host.Publish(owner, new StatusDraft { Text = $"@{recipient.Handle()} just you", PlainText = true, Visibility = PostVisibility.Direct });
|
||||
var quietPath = $"/peasants/{owner.UserName}/scribbles/{quiet.ID}";
|
||||
var directPath = $"/peasants/{owner.UserName}/scribbles/{direct.ID}";
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(follower.SignedGet(quietPath))).Status);
|
||||
Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(followerServer.SignedGet(quietPath))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status);
|
||||
var asRecipient = await _client.Fetch(recipient.SignedGet(directPath));
|
||||
Assert.Equal(HttpStatusCode.OK, asRecipient.Status);
|
||||
Assert.Contains(recipient.Id, Strings(asRecipient.Json["to"]));
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(follower.SignedGet(directPath))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(directPath)).Status);
|
||||
|
||||
// the DM's context is the conversation, for its participants only
|
||||
var context = PathOf(asRecipient.Json["context"]!.GetValue<string>());
|
||||
var conversation = await _client.Fetch(recipient.SignedGet(context));
|
||||
Assert.Equal(HttpStatusCode.OK, conversation.Status);
|
||||
Assert.Equal(new[] { direct.ObjectURI }, Strings(conversation.Json["orderedItems"]));
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(context))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(context)).Status);
|
||||
|
||||
// once deleted, those it was for learn it is gone; everyone else still learns nothing
|
||||
await _host.Remove(owner, quiet);
|
||||
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(follower.SignedGet(quietPath))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -463,6 +520,10 @@ namespace PrivaPub.Tests.Http
|
||||
Assert.Equal("Application", instance.Json["type"]!.GetValue<string>());
|
||||
foreach (var path in paths[..^1])
|
||||
Assert.True((await client.Fetch(reader.SignedGet(path))).Status == HttpStatusCode.OK, $"{path} refused a signed GET");
|
||||
// a browser is not asked for a signature: it is only sent to the public pages
|
||||
var browser = await client.Fetch($"/peasants/{persona.UserName}", Browser);
|
||||
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
|
||||
Assert.Equal(HttpStatusCode.Redirect, (await client.Fetch($"/peasants/{persona.UserName}/scribbles/{post.ID}", Browser)).Status);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Tests.Support;
|
||||
using PrivaPub.Tests.Support.Host;
|
||||
|
||||
using System.Net;
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
namespace PrivaPub.Tests.Http
|
||||
{
|
||||
// Where a server is, on the public instance API: where it was located (city, coordinates and network, whatever its size),
|
||||
// only the CDN's name for a CDN-fronted one, the batch form, and this server itself.
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class InstanceLocationTests : IAsyncLifetime
|
||||
{
|
||||
PrivaPubHost _host;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_host = await PrivaPubHost.Shared();
|
||||
}
|
||||
|
||||
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
|
||||
|
||||
static CancellationToken Token => TestContext.Current.CancellationToken;
|
||||
|
||||
static async Task<string> Instance(int users, InstanceGeo geo, string name = "located")
|
||||
{
|
||||
var host = $"{name}{Guid.NewGuid():N}.example";
|
||||
await DB.Default.SaveAsync(new RemoteInstance { Host = host, Software = "mastodon", UsersTotal = users, Geo = geo }, Token);
|
||||
return host;
|
||||
}
|
||||
|
||||
static InstanceGeo Milan(string cdn = default) => new()
|
||||
{
|
||||
Country = "IT",
|
||||
City = "Milan",
|
||||
Latitude = 45.5,
|
||||
Longitude = 9.2,
|
||||
Asn = 12345,
|
||||
AsnOrg = "Example Hosting",
|
||||
Cdn = cdn,
|
||||
Ipv6 = true,
|
||||
Source = "DB-IP Lite 2026-10",
|
||||
LocatedAt = DateTime.UtcNow.AddDays(-2)
|
||||
};
|
||||
|
||||
async Task<JsonNode> Geo(string host) => (await _host.Client().Get($"/api/privapub/v1/instances/{host}")).Ok().Body["geo"];
|
||||
|
||||
[Fact]
|
||||
public async Task A_located_server_shows_its_city_coordinates_and_network()
|
||||
{
|
||||
var geo = await Geo(await Instance(50, Milan()));
|
||||
|
||||
Assert.Equal("city", geo.Text("precision"));
|
||||
Assert.Equal(("IT", "Milan", "Example Hosting"), (geo.Text("country"), geo.Text("city"), geo.Text("network")));
|
||||
Assert.Equal(45.5, geo["latitude"]!.GetValue<double>());
|
||||
Assert.Equal(9.2, geo["longitude"]!.GetValue<double>());
|
||||
Assert.Equal("DB-IP Lite 2026-10", geo.Text("source"));
|
||||
Assert.NotNull(geo.Text("located_at"));
|
||||
Assert.Null(geo["asn"]);
|
||||
Assert.Null(geo["ipv6"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_one_user_server_shows_its_city_too()
|
||||
{
|
||||
var geo = await Geo(await Instance(1, Milan()));
|
||||
|
||||
Assert.Equal("city", geo.Text("precision"));
|
||||
Assert.Equal(("IT", "Milan", "Example Hosting"), (geo.Text("country"), geo.Text("city"), geo.Text("network")));
|
||||
Assert.Equal(45.5, geo["latitude"]!.GetValue<double>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_server_located_only_to_its_country_shows_the_country()
|
||||
{
|
||||
var geo = await Geo(await Instance(50, new InstanceGeo { Country = "IT", Source = "DB-IP Lite 2026-10", LocatedAt = DateTime.UtcNow }));
|
||||
|
||||
Assert.Equal("country", geo.Text("precision"));
|
||||
Assert.Equal("IT", geo.Text("country"));
|
||||
Assert.Null(geo["city"]);
|
||||
Assert.Null(geo["latitude"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_cdn_fronted_server_shows_only_its_cdn()
|
||||
{
|
||||
var geo = await Geo(await Instance(5000, Milan(cdn: "Cloudflare")));
|
||||
|
||||
Assert.Equal("cdn", geo.Text("precision"));
|
||||
Assert.Equal("Cloudflare", geo.Text("cdn"));
|
||||
Assert.Null(geo["country"]);
|
||||
Assert.Null(geo["latitude"]);
|
||||
Assert.Null(geo["network"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_unlocated_server_has_no_place()
|
||||
{
|
||||
Assert.Null(await Geo(await Instance(50, default)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task The_provenance_of_a_post_carries_the_same_projection()
|
||||
{
|
||||
var host = await Instance(3, Milan());
|
||||
var described = (await _host.Client().Get($"/api/privapub/v1/instances/{host}")).Ok().Body;
|
||||
|
||||
Assert.Equal("city", described["geo"].Text("precision"));
|
||||
Assert.Equal(described["geo"]!.ToJsonString(), (await Geo(host))!.ToJsonString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Several_servers_answer_at_once_in_the_order_asked_without_the_unknown_ones()
|
||||
{
|
||||
var big = await Instance(50, Milan(), "big");
|
||||
var small = await Instance(3, Milan(), "small");
|
||||
var unknown = $"unknown{Guid.NewGuid():N}.example";
|
||||
|
||||
var answer = (await _host.Client().Get($"/api/privapub/v1/instances?host[]={small}&host[]={unknown}&host[]={big.ToUpperInvariant()}&host[]={small}")).Ok();
|
||||
var hosts = answer.Body!.AsArray().Select(i => i.Text("host")).ToList();
|
||||
|
||||
Assert.Equal(new[] { small, big }, hosts);
|
||||
Assert.Equal("city", answer.Body![0]["geo"].Text("precision"));
|
||||
Assert.Equal("city", answer.Body![1]["geo"].Text("precision"));
|
||||
Assert.Empty((await _host.Client().Get("/api/privapub/v1/instances")).Ok().Body!.AsArray());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task At_most_forty_servers_answer_at_once()
|
||||
{
|
||||
var hosts = new List<string>();
|
||||
for (var i = 0; i < 42; i++)
|
||||
hosts.Add(await Instance(1, default, "many"));
|
||||
|
||||
var answer = (await _host.Client().Get("/api/privapub/v1/instances?" + string.Join("&", hosts.Select(h => "host[]=" + h)))).Ok();
|
||||
|
||||
Assert.Equal(hosts.Take(40), answer.Body!.AsArray().Select(i => i.Text("host")));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task This_server_describes_itself()
|
||||
{
|
||||
var self = (await _host.Client().Get($"/api/privapub/v1/instances/{PrivaPubHost.Host}")).Ok().Body;
|
||||
|
||||
Assert.Equal(PrivaPubHost.Host, self.Text("host"));
|
||||
Assert.Equal("privapub", self.Text("software"));
|
||||
Assert.Equal("activitypub", Assert.Single(self["protocols"]!.AsArray())!.GetValue<string>());
|
||||
Assert.Null(self["geo"]);
|
||||
var batch = (await _host.Client().Get($"/api/privapub/v1/instances?host[]={PrivaPubHost.Host}")).Ok().Body!.AsArray();
|
||||
Assert.Equal("privapub", Assert.Single(batch).Text("software"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task The_owner_may_place_this_server()
|
||||
{
|
||||
var placed = await SelfPlacedHost.Shared();
|
||||
|
||||
var geo = (await placed.Client().Get($"/api/privapub/v1/instances/{PrivaPubHost.Host}")).Ok().Body["geo"];
|
||||
|
||||
Assert.Equal("city", geo.Text("precision"));
|
||||
Assert.Equal(("IT", "Milan", "configured"), (geo.Text("country"), geo.Text("city"), geo.Text("source")));
|
||||
Assert.Equal(45.5, geo["latitude"]!.GetValue<double>());
|
||||
Assert.Equal(9.2, geo["longitude"]!.GetValue<double>());
|
||||
}
|
||||
|
||||
static string WeeksAgo(int weeks) => PrivaPub.Federation.Objects.InstanceDescriber.Week(DateTime.UtcNow.AddDays(-7 * weeks));
|
||||
|
||||
static Task Snapshot(string host, int weeksAgo, string cdn = default, string cdnDomain = default, string city = default, double? latitude = default,
|
||||
double? longitude = default, string country = default) =>
|
||||
DB.Default.SaveAsync(new RemoteInstanceSnapshot
|
||||
{
|
||||
Host = host, Week = WeeksAgo(weeksAgo), TakenAt = DateTime.UtcNow.AddDays(-7 * weeksAgo), Reachable = true, Software = "mastodon", UsersTotal = 3,
|
||||
Cdn = cdn, CdnDomain = cdnDomain, CdnSource = cdn == default ? default : "ranges", Country = country, City = city, Latitude = latitude, Longitude = longitude
|
||||
}, Token);
|
||||
|
||||
[Fact]
|
||||
public async Task A_cdn_fronted_server_names_its_cdn_and_where_it_was_before_it()
|
||||
{
|
||||
var host = await Instance(3, new InstanceGeo
|
||||
{
|
||||
Country = "US", City = "San Francisco", Latitude = 37.8, Longitude = -122.4, Cdn = "Cloudflare", CdnDomain = "cloudflare.com", CdnSource = "ranges",
|
||||
Source = "DB-IP Lite 2026-10", LocatedAt = DateTime.UtcNow
|
||||
}, "fronted");
|
||||
await Snapshot(host, 3, city: "Berlin", latitude: 52.5, longitude: 13.4, country: "DE");
|
||||
await Snapshot(host, 2, city: "Hamburg", latitude: 53.6, longitude: 10.0, country: "DE");
|
||||
await Snapshot(host, 1, cdn: "Cloudflare", cdnDomain: "cloudflare.com", city: "San Francisco", latitude: 37.8, longitude: -122.4, country: "US");
|
||||
|
||||
var geo = await Geo(host);
|
||||
|
||||
Assert.Equal(("cdn", "Cloudflare", "cloudflare.com", "ranges"), (geo.Text("precision"), geo.Text("cdn"), geo.Text("cdn_domain"), geo.Text("cdn_source")));
|
||||
Assert.Null(geo["latitude"]);
|
||||
Assert.Equal((WeeksAgo(2), "Hamburg", "DE"), (geo["before_cdn"].Text("week"), geo["before_cdn"].Text("city"), geo["before_cdn"].Text("country")));
|
||||
Assert.Equal(53.6, geo["before_cdn"]!["latitude"]!.GetValue<double>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_server_history_is_weekly_newest_first_and_a_cdn_hides_its_place()
|
||||
{
|
||||
var host = await Instance(3, Milan(), "historic");
|
||||
await Snapshot(host, 2, city: "Milan", latitude: 45.5, longitude: 9.2, country: "IT");
|
||||
await Snapshot(host, 1, cdn: "Fastly", cdnDomain: "fastly.com", city: "Frankfurt", latitude: 50.1, longitude: 8.7, country: "DE");
|
||||
|
||||
var weeks = (await _host.Client().Get($"/api/privapub/v1/instances/{host}/history")).Ok().Body!.AsArray();
|
||||
|
||||
Assert.Equal(new[] { WeeksAgo(1), WeeksAgo(2) }, weeks.Select(w => w.Text("week")));
|
||||
Assert.Equal(("cdn", "fastly.com"), (weeks[0]!["geo"].Text("precision"), weeks[0]!["geo"].Text("cdn_domain")));
|
||||
Assert.Null(weeks[0]!["geo"]!["city"]);
|
||||
Assert.Equal(("city", "Milan"), (weeks[1]!["geo"].Text("precision"), weeks[1]!["geo"].Text("city")));
|
||||
Assert.Equal(3, weeks[1]!["users_total"]!.GetValue<long>());
|
||||
Assert.Single((await _host.Client().Get($"/api/privapub/v1/instances/{host}/history?weeks=1")).Ok().Body!.AsArray());
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/instances/unknown{Guid.NewGuid():N}.example/history")).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Servers_are_grouped_by_their_cdn_week_by_week()
|
||||
{
|
||||
var cdn = $"Edge{Guid.NewGuid():N}"[..20];
|
||||
var key = cdn.ToLowerInvariant();
|
||||
var stayed = await Instance(3, new InstanceGeo { Country = "FR", Cdn = cdn, CdnSource = "headers" }, "stayed");
|
||||
var moved = await Instance(3, Milan(), "moved");
|
||||
await Snapshot(moved, 3, cdn: cdn);
|
||||
await Snapshot(moved, 2, cdn: cdn);
|
||||
await Snapshot(stayed, 2, cdn: cdn);
|
||||
await Snapshot(moved, 1, city: "Milan", latitude: 45.5, longitude: 9.2, country: "IT");
|
||||
await Snapshot(stayed, 1, cdn: cdn);
|
||||
|
||||
var list = (await _host.Client().Get("/api/privapub/v1/cdns")).Ok().Body!.AsArray();
|
||||
var group = list.Single(g => g.Text("key") == key);
|
||||
Assert.Equal((cdn, 1), (group.Text("name"), group["servers"]!.GetValue<int>()));
|
||||
var cloudflare = list.Single(g => g.Text("key") == "cloudflare.com");
|
||||
Assert.Equal(new[] { "ranges", "headers", "asn" }, cloudflare["detected_by"]!.AsArray().Select(v => v!.GetValue<string>()));
|
||||
|
||||
var detail = (await _host.Client().Get($"/api/privapub/v1/cdns/{key}")).Ok().Body;
|
||||
Assert.Equal(new[] { stayed }, detail["hosts"]!.AsArray().Select(h => h!.GetValue<string>()));
|
||||
var weeks = detail["weeks"]!.AsArray();
|
||||
Assert.Equal(new[] { WeeksAgo(1), WeeksAgo(2), WeeksAgo(3) }, weeks.Select(w => w.Text("week")));
|
||||
Assert.Equal(new[] { 1, 2, 1 }, weeks.Select(w => w!["servers"]!.GetValue<int>()));
|
||||
Assert.Equal(new[] { moved }, weeks[0]!["left"]!.AsArray().Select(h => h!.GetValue<string>()));
|
||||
Assert.Equal(new[] { stayed }, weeks[1]!["joined"]!.AsArray().Select(h => h!.GetValue<string>()));
|
||||
Assert.Equal(new[] { moved }, weeks[2]!["joined"]!.AsArray().Select(h => h!.GetValue<string>()));
|
||||
|
||||
Assert.Equal("Cloudflare", (await _host.Client().Get("/api/privapub/v1/cdns/cloudflare.com")).Ok().Body.Text("name"));
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/cdns/none{Guid.NewGuid():N}")).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Junk_hosts_are_not_an_error()
|
||||
{
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await _host.Client().Get($"/api/privapub/v1/instances/{new string('a', 300)}")).Status);
|
||||
Assert.Empty((await _host.Client().Get($"/api/privapub/v1/instances?host[]=&host[]={new string('a', 300)}")).Ok().Body!.AsArray());
|
||||
}
|
||||
}
|
||||
|
||||
public sealed class SelfPlacedHost : PrivaPubHost
|
||||
{
|
||||
static readonly SemaphoreSlim Boot = new(1, 1);
|
||||
static SelfPlacedHost _shared;
|
||||
|
||||
public static new async Task<SelfPlacedHost> Shared()
|
||||
{
|
||||
await PrivaPubHost.Shared();
|
||||
await Boot.WaitAsync();
|
||||
try
|
||||
{
|
||||
if (_shared == default)
|
||||
{
|
||||
var host = new SelfPlacedHost();
|
||||
_ = host.Services;
|
||||
_shared = host;
|
||||
}
|
||||
return _shared;
|
||||
}
|
||||
finally
|
||||
{
|
||||
Boot.Release();
|
||||
}
|
||||
}
|
||||
|
||||
protected override IEnumerable<KeyValuePair<string, string>> Settings() => base.Settings().Concat(new Dictionary<string, string>
|
||||
{
|
||||
["Statistics:Geo:Self:Latitude"] = "45.4642",
|
||||
["Statistics:Geo:Self:Longitude"] = "9.19",
|
||||
["Statistics:Geo:Self:City"] = "Milan",
|
||||
["Statistics:Geo:Self:Country"] = "IT"
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -41,7 +41,9 @@ namespace PrivaPub.Tests.Http
|
||||
var v2 = (await anonymous.Get("/api/v2/instance")).Ok();
|
||||
Assert.Equal(InstanceController.Version, v2.Body.Text("version"));
|
||||
Assert.Equal(PrivaPubHost.Host, v2.Body.Text("domain"));
|
||||
Assert.Equal($"wss://{PrivaPubHost.Host}", v2.Body["configuration"]!["urls"].Text("streaming"));
|
||||
Assert.Null(v2.Body["configuration"]!["urls"]!["streaming"]);//nothing streams yet, so nothing is advertised
|
||||
Assert.Equal(PrivaPub.Api.Mastodon.Controllers.StatusesController.MaxPins, v2.Body["configuration"]!["accounts"].Number("max_pinned_statuses"));
|
||||
Assert.Contains("image/avif", v2.Body["configuration"]!["media_attachments"]!["supported_mime_types"]!.AsArray().Select(t => t!.GetValue<string>()));
|
||||
Assert.True(v2.Body["registrations"].Flag("enabled"));
|
||||
Assert.True((await anonymous.Get("/nodeinfo/2.1")).Ok().Body.Flag("openRegistrations"));
|
||||
Assert.False(v2.Body["configuration"]!["translation"].Flag("enabled"));
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.Social;
|
||||
using PrivaPub.Tests.Support;
|
||||
using PrivaPub.Tests.Support.Host;
|
||||
|
||||
using System.Net;
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
using static PrivaPub.Tests.Support.Host.FederationHelpers;
|
||||
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Tests.Http
|
||||
{
|
||||
// Two views of the same fact give one answer (owner decision 2026-10-04: fix the mismatches).
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class OneAnswerTests : IAsyncLifetime
|
||||
{
|
||||
PrivaPubHost _host;
|
||||
HttpClient _client;
|
||||
Peer _peer;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_host = await PrivaPubHost.Shared();
|
||||
_client = _host.Client();
|
||||
_peer = await Peer.Start();
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
_client?.Dispose();
|
||||
if (_peer != default)
|
||||
await _peer.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Anyone_may_search_and_only_a_signed_in_reader_resolves_or_pages()
|
||||
{
|
||||
var reader = await _host.Mastodon("searcher");
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, (await _client.Get("/api/v2/search?q=somebody")).Status);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Get("/api/v2/search?q=somebody&resolve=true")).Status);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Get("/api/v2/search?q=somebody&offset=5")).Status);
|
||||
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get("/api/v2/search?q=somebody&offset=5")).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_remote_account_that_deletes_itself_leaves_nothing_in_the_counts()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var author = await _host.Mastodon("liked");
|
||||
var post = await author.Status("like me");
|
||||
var fan = new RemoteActor(_peer, "fickle");
|
||||
var like = new JsonObject
|
||||
{
|
||||
["id"] = $"{fan.Id}#likes/{Guid.NewGuid():N}",
|
||||
["type"] = "Like",
|
||||
["actor"] = fan.Id,
|
||||
["object"] = post.Text("uri")
|
||||
};
|
||||
Assert.Equal(HttpStatusCode.Accepted, (await _client.SendAsync(fan.SignedPost("/human-centipede", like), token)).StatusCode);
|
||||
await _host.RunInbox(like["id"]!.GetValue<string>(), token);
|
||||
Assert.Equal(1, (await author.Client.Get($"/api/v1/statuses/{post.Text("id")}")).Ok().Body.Number("favourites_count"));
|
||||
|
||||
var delete = new JsonObject
|
||||
{
|
||||
["id"] = $"{fan.Id}#delete",
|
||||
["type"] = "Delete",
|
||||
["actor"] = fan.Id,
|
||||
["object"] = fan.Id,
|
||||
["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public")
|
||||
};
|
||||
Assert.Equal(HttpStatusCode.Accepted, (await _client.SendAsync(fan.SignedPost("/human-centipede", delete), token)).StatusCode);
|
||||
await _host.RunInbox(delete["id"]!.GetValue<string>(), token);
|
||||
|
||||
Assert.Equal(0, (await author.Client.Get($"/api/v1/statuses/{post.Text("id")}")).Ok().Body.Number("favourites_count"));
|
||||
Assert.Empty((await author.Client.Get($"/api/v1/statuses/{post.Text("id")}/favourited_by")).Ok().Array);
|
||||
Assert.False(await DB.Default.Find<Favourite>().Match(f => f.ActorURI == fan.Id).ExecuteAnyAsync(token));
|
||||
Assert.False(await DB.Default.Find<Notification>().Match(n => n.FromActorURI == fan.Id).ExecuteAnyAsync(token));
|
||||
Assert.Equal(0, (await author.Client.Get("/api/v1/notifications/unread_count")).Ok().Body.Number("count"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Only_public_and_unlisted_replies_are_counted()
|
||||
{
|
||||
var author = await _host.Mastodon("threadstart");
|
||||
var replier = await _host.Mastodon("replier");
|
||||
var root = await author.Status("say something");
|
||||
var id = root.Text("id");
|
||||
|
||||
await replier.Status("in public", ("in_reply_to_id", id));
|
||||
await replier.Status("quietly", ("in_reply_to_id", id), ("visibility", "unlisted"));
|
||||
await replier.Status("for my followers", ("in_reply_to_id", id), ("visibility", "private"));
|
||||
await replier.Status($"@{author.UserName} just you", ("in_reply_to_id", id), ("visibility", "direct"));
|
||||
|
||||
Assert.Equal(2, (await author.Client.Get($"/api/v1/statuses/{id}")).Ok().Body.Number("replies_count"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_status_longer_than_advertised_is_refused()
|
||||
{
|
||||
var author = await _host.Mastodon("wordy");
|
||||
var limit = (await _client.Get("/api/v2/instance")).Ok().Body["configuration"]!["statuses"].Number("max_characters");
|
||||
|
||||
var refused = await author.Client.Post("/api/v1/statuses", ("status", new string('a', limit + 1)));
|
||||
var accepted = await author.Client.Post("/api/v1/statuses", ("status", new string('a', limit)));
|
||||
|
||||
Assert.Equal(HttpStatusCode.UnprocessableEntity, refused.Status);
|
||||
Assert.Equal(HttpStatusCode.OK, accepted.Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Joining_a_community_by_invitation_is_following_it()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var owner = await _host.Persona(await _host.SignUp(), "commowner");
|
||||
var joinerRoot = await _host.SignUp("joiner");
|
||||
var joiner = await _host.Persona(joinerRoot, "joiner");
|
||||
var group = await _host.Group(owner, community: true);
|
||||
var groupName = group["userName"]!.GetValue<string>();
|
||||
using var client = _host.As(joinerRoot.Jwt);
|
||||
|
||||
var joined = await client.PostJson("/clientapi/group/join", new { avatarId = joiner.Id, invitationCode = group["invitationCode"]!.GetValue<string>() });
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, joined.StatusCode);
|
||||
Assert.Equal(1, (await _client.Fetch($"/peasants/{groupName}/groupies")).Json["totalItems"]!.GetValue<int>());
|
||||
Assert.True(await DB.Default.Find<Following>().Match(f => f.AvatarId == joiner.Id && f.State == FollowState.Accepted).ExecuteAnyAsync(token));
|
||||
}
|
||||
|
||||
public static TheoryData<string> AnsweredRoutes() => new()
|
||||
{
|
||||
"/api/v1/directory",
|
||||
"/api/v1/tags/cats",
|
||||
"/api/v1/instance/languages",
|
||||
"/api/v1/instance/translation_languages",
|
||||
"/api/v1/instance/domain_blocks",
|
||||
"/api/v1/instance/privacy_policy",
|
||||
"/api/v1/timelines/link?url=https%3A%2F%2Fexample.org%2F",
|
||||
"/api/v1/accounts/000000000000000000000000/identity_proofs"
|
||||
};
|
||||
|
||||
[Theory]
|
||||
[MemberData(nameof(AnsweredRoutes))]
|
||||
public async Task Mastodon_routes_we_have_nothing_behind_answer_instead_of_404(string path)
|
||||
{
|
||||
Assert.Equal(HttpStatusCode.OK, (await _client.Get(path)).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Likes_of_one_post_come_as_one_group_of_notifications()
|
||||
{
|
||||
var author = await _host.Mastodon("grouped");
|
||||
var first = await _host.Mastodon("fanone");
|
||||
var second = await _host.Mastodon("fantwo");
|
||||
var post = await author.Status("like me twice");
|
||||
var id = post.Text("id");
|
||||
await first.Client.Post($"/api/v1/statuses/{id}/favourite");
|
||||
await second.Client.Post($"/api/v1/statuses/{id}/favourite");
|
||||
|
||||
var list = (await author.Client.Get("/api/v2/notifications")).Ok().Body;
|
||||
var group = Assert.Single(list["notification_groups"]!.AsArray(), g => g!.Text("type") == "favourite")!;
|
||||
var key = group.Text("group_key");
|
||||
|
||||
Assert.Equal($"grouped-favourite-{id}", key);
|
||||
Assert.Equal(2, group.Number("notifications_count"));
|
||||
Assert.Equal(2, group["sample_account_ids"]!.AsArray().Count);
|
||||
Assert.Equal(id, group.Text("status_id"));
|
||||
Assert.Contains(list["statuses"]!.AsArray(), s => s!.Text("id") == id);
|
||||
Assert.True((await author.Client.Get("/api/v2/notifications/unread_count")).Ok().Body.Number("count") >= 1);
|
||||
Assert.Equal(key, Assert.Single((await author.Client.Get($"/api/v2/notifications/{key}")).Ok().Body["notification_groups"]!.AsArray())!.Text("group_key"));
|
||||
Assert.Equal(2, (await author.Client.Get($"/api/v2/notifications/{key}/accounts")).Ok().Array.Count);
|
||||
Assert.Equal(HttpStatusCode.OK, (await author.Client.Post($"/api/v2/notifications/{key}/dismiss")).Status);
|
||||
Assert.Equal(HttpStatusCode.NotFound, (await author.Client.Get($"/api/v2/notifications/{key}")).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_hashtag_link_opens_a_page_of_public_posts_with_that_tag()
|
||||
{
|
||||
var author = await _host.Mastodon("tagger");
|
||||
var tag = $"tag{Guid.NewGuid():N}"[..16];
|
||||
await author.Status($"about #{tag}");
|
||||
await author.Status($"quietly about #{tag}", ("visibility", "private"));
|
||||
|
||||
var page = await _client.Fetch($"/tags/{tag}", Browser);
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, page.Status);
|
||||
Assert.Contains("about", page.Text);
|
||||
Assert.DoesNotContain("quietly", page.Text);
|
||||
Assert.Contains("default-src 'none'", page.Response.Headers.GetValues("Content-Security-Policy").Single());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_communitys_announces_resolve_while_the_post_is_shown()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var owner = await _host.Persona(await _host.SignUp(), "announcer");
|
||||
var community = await _host.FederatedGroup(owner, community: true);
|
||||
var post = await _host.Publish(owner, new PrivaPub.Domain.Statuses.StatusDraft { Text = "into the community", PlainText = true, GroupId = community.Id });
|
||||
var kept = await DB.Default.Find<GroupAnnouncement>().Match(a => a.GroupId == community.Id && a.ObjectURI == post.ObjectURI).ExecuteAsync(token);
|
||||
Assert.Equal(2, kept.Count);//the activity, and the object for Mastodon
|
||||
|
||||
foreach (var announcement in kept)
|
||||
{
|
||||
var fetched = await _client.Fetch(PathOf(announcement.ActivityURI));
|
||||
Assert.Equal(HttpStatusCode.OK, fetched.Status);
|
||||
Assert.Equal("Announce", fetched.Json["type"]!.GetValue<string>());
|
||||
Assert.Equal(announcement.ActivityURI, fetched.Json["id"]!.GetValue<string>());
|
||||
}
|
||||
var listed = await _client.Fetch($"/peasants/{community.UserName}/grunts/announce-{post.ID}");
|
||||
Assert.Equal(HttpStatusCode.OK, listed.Status);
|
||||
|
||||
await _host.Remove(owner, post);
|
||||
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(PathOf(kept[0].ActivityURI))).Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_boost_points_at_the_boosted_posts_page()
|
||||
{
|
||||
var author = await _host.Mastodon("boosted");
|
||||
var booster = await _host.Mastodon("booster");
|
||||
var post = await author.Status("boost me");
|
||||
|
||||
var boost = (await booster.Client.Post($"/api/v1/statuses/{post.Text("id")}/reblog")).Ok().Body;
|
||||
|
||||
Assert.Equal(post.Text("url"), boost.Text("url"));
|
||||
var browser = await _client.Fetch(PathOf(boost.Text("uri")), Browser);
|
||||
Assert.Equal(HttpStatusCode.Redirect, browser.Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_remote_accounts_counts_are_what_its_server_publishes()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var reader = await _host.Mastodon("counter");
|
||||
var popular = new RemoteActor(_peer, "popular");
|
||||
_peer.Serve(new Uri(popular.Id).AbsolutePath + "/followers",
|
||||
$$"""{"@context":"https://www.w3.org/ns/activitystreams","id":"{{popular.Id}}/followers","type":"OrderedCollection","totalItems":42}""");
|
||||
var remote = _host.Get<PrivaPub.Federation.Actors.IRemoteActorService>();
|
||||
var account = await remote.GetActor(popular.Id, refresh: true, token);
|
||||
|
||||
await _host.Run(j => j.Kind == PrivaPub.Models.Jobs.JobKind.CountAccount && j.Payload == popular.Id, token);
|
||||
|
||||
var shown = (await reader.Client.Get($"/api/v1/accounts/{account.ID}")).Ok().Body;
|
||||
Assert.Equal(42, shown.Number("followers_count"));
|
||||
Assert.Equal(0, shown.Number("following_count"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Notification_policy_and_requests_answer_a_signed_in_reader()
|
||||
{
|
||||
var reader = await _host.Mastodon("policy");
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, (await reader.Client.Get("/api/v1/notifications/policy")).Status);
|
||||
Assert.Equal("accept", (await reader.Client.Get("/api/v2/notifications/policy")).Ok().Body.Text("for_not_following"));
|
||||
Assert.Empty((await reader.Client.Get("/api/v1/notifications/requests")).Ok().Array);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,218 @@
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.IdentityModel.JsonWebTokens;
|
||||
using Microsoft.IdentityModel.Tokens;
|
||||
|
||||
using PrivaPub.Api.Mastodon.Auth;
|
||||
using PrivaPub.Services;
|
||||
using PrivaPub.Tests.Support;
|
||||
using PrivaPub.Tests.Support.Host;
|
||||
|
||||
using System.Net;
|
||||
using System.Security.Claims;
|
||||
using System.Text;
|
||||
|
||||
namespace PrivaPub.Tests.Http
|
||||
{
|
||||
// The first-party client's one sign-in: the /clientapi JWT exchanged for one persona's Mastodon token (PersonaExchange).
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class PersonaExchangeTests : IAsyncLifetime
|
||||
{
|
||||
const string Grant = "urn:ietf:params:oauth:grant-type:token-exchange";
|
||||
const string FirstParty = "decepub";
|
||||
|
||||
PrivaPubHost _host;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_host = await PrivaPubHost.Shared();
|
||||
}
|
||||
|
||||
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
|
||||
|
||||
async Task<HttpResponseMessage> Exchange(string jwt, string avatarId, string clientId = FirstParty, string clientSecret = default,
|
||||
string subjectTokenType = PersonaExchange.SubjectTokenType)
|
||||
{
|
||||
var fields = new List<(string, string)>
|
||||
{
|
||||
("grant_type", Grant),
|
||||
("client_id", clientId),
|
||||
("subject_token", jwt),
|
||||
("subject_token_type", subjectTokenType),
|
||||
(PersonaExchange.AvatarParameter, avatarId),
|
||||
("scope", "read write follow")
|
||||
};
|
||||
if (clientSecret != default)
|
||||
fields.Add(("client_secret", clientSecret));
|
||||
using var client = _host.Client();
|
||||
return await client.Form("/oauth/token", fields.Where(f => f.Item2 != default).ToArray());
|
||||
}
|
||||
|
||||
async Task<string> Token(Persona persona)
|
||||
{
|
||||
var response = await Exchange(persona.Root.Jwt, persona.Id);
|
||||
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
|
||||
return (await response.JsonBody())["access_token"]!.GetValue<string>();
|
||||
}
|
||||
|
||||
static async Task AssertRefused(HttpResponseMessage response, string error = "invalid_grant")
|
||||
{
|
||||
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
|
||||
Assert.Equal(error, (await response.JsonBody())["error"]?.GetValue<string>());
|
||||
}
|
||||
|
||||
async Task<HttpStatusCode> Me(string token)
|
||||
{
|
||||
using var api = _host.As(token);
|
||||
return (await api.GetAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken)).StatusCode;
|
||||
}
|
||||
|
||||
static string Jwt(string rootId, string key, DateTime expires) => new JsonWebTokenHandler().CreateToken(new SecurityTokenDescriptor
|
||||
{
|
||||
Issuer = PrivaPubHost.Base,
|
||||
Audience = PrivaPubHost.Base,
|
||||
Subject = new ClaimsIdentity(new[] { new Claim(ClaimTypes.UserData, rootId) }),
|
||||
NotBefore = expires.AddHours(-2),
|
||||
IssuedAt = expires.AddHours(-2),
|
||||
Expires = expires,
|
||||
SigningCredentials = new(new SymmetricSecurityKey(Encoding.UTF8.GetBytes(key)), SecurityAlgorithms.HmacSha512)
|
||||
});
|
||||
|
||||
[Fact]
|
||||
public async Task The_root_jwt_becomes_one_personas_token_that_never_names_the_root()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "exchange");
|
||||
var root = persona.Root;
|
||||
|
||||
var response = await Exchange(root.Jwt, persona.Id);
|
||||
var body = await response.Content.ReadAsStringAsync(TestContext.Current.CancellationToken);
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
|
||||
Assert.DoesNotContain(root.Id, body);
|
||||
Assert.DoesNotContain(root.UserName, body);
|
||||
var token = (await response.JsonBody())["access_token"]!.GetValue<string>();
|
||||
using var api = _host.As(token);
|
||||
var account = await api.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken);
|
||||
Assert.Contains($"\"id\":\"{persona.Id}\"", account);
|
||||
Assert.DoesNotContain(root.Id, account);
|
||||
foreach (var entry in await ClientApi.StoredTokens(FirstParty))
|
||||
{
|
||||
var payload = entry.Contains("payload") && entry["payload"].IsString ? ClientApi.JwtPayload(entry["payload"].AsString) : string.Empty;
|
||||
Assert.DoesNotContain(root.Id, entry.ToString() + payload);
|
||||
Assert.DoesNotContain(root.UserName, entry.ToString() + payload);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Each_persona_of_the_root_gets_its_own_token()
|
||||
{
|
||||
var root = await _host.SignUp();
|
||||
var first = await _host.Persona(root, "first");
|
||||
var second = await _host.Persona(root, "second");
|
||||
|
||||
using var firstApi = _host.As(await Token(first));
|
||||
using var secondApi = _host.As(await Token(second));
|
||||
|
||||
Assert.Contains($"\"id\":\"{first.Id}\"", await firstApi.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken));
|
||||
Assert.Contains($"\"id\":\"{second.Id}\"", await secondApi.GetStringAsync("/api/v1/accounts/verify_credentials", TestContext.Current.CancellationToken));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Another_roots_persona_is_refused()
|
||||
{
|
||||
var mine = await _host.SignUp();
|
||||
var theirs = await _host.Persona(await _host.SignUp(), "theirs");
|
||||
|
||||
await AssertRefused(await Exchange(mine.Jwt, theirs.Id));
|
||||
await AssertRefused(await Exchange(mine.Jwt, "000000000000000000000000"));
|
||||
await AssertRefused(await Exchange(mine.Jwt, "not an id"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_token_that_is_not_a_valid_root_jwt_is_refused()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "forged");
|
||||
var key = _host.Services.GetRequiredService<IConfiguration>()["AppConfiguration:Jwt:Key"]!;
|
||||
|
||||
await AssertRefused(await Exchange("garbage", persona.Id));
|
||||
await AssertRefused(await Exchange(Jwt(persona.Root.Id, key, DateTime.UtcNow.AddMinutes(-10)), persona.Id));
|
||||
await AssertRefused(await Exchange(Jwt(persona.Root.Id, new string('k', 64), DateTime.UtcNow.AddHours(1)), persona.Id));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Ended_sessions_refuse_the_jwt_and_revoke_the_exchanged_tokens()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "ended");
|
||||
var token = await Token(persona);
|
||||
Assert.Equal(HttpStatusCode.OK, await Me(token));
|
||||
|
||||
using (var scope = _host.Services.CreateScope())
|
||||
await scope.ServiceProvider.GetRequiredService<IRootSessions>().Revoke(persona.Root.Id, TestContext.Current.CancellationToken);
|
||||
|
||||
await AssertRefused(await Exchange(persona.Root.Jwt, persona.Id));
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, await Me(token));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_banned_root_is_refused()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "banned");
|
||||
await ClientApi.Ban(persona.Root.Id);
|
||||
try
|
||||
{
|
||||
await AssertRefused(await Exchange(persona.Root.Jwt, persona.Id));
|
||||
}
|
||||
finally
|
||||
{
|
||||
await ClientApi.Ban(persona.Root.Id, banned: false);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_revoked_token_no_longer_works()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "revoked");
|
||||
var token = await Token(persona);
|
||||
|
||||
using var client = _host.Client();
|
||||
var revoked = await client.Form("/oauth/revoke", ("token", token), ("client_id", FirstParty));
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, revoked.StatusCode);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, await Me(token));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Only_the_first_party_client_may_exchange()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "thirdparty");
|
||||
using var client = _host.Client();
|
||||
var app = await client.RegisterApp();
|
||||
|
||||
var response = await Exchange(persona.Root.Jwt, persona.Id, app.ClientId, app.ClientSecret);
|
||||
|
||||
Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
|
||||
Assert.Equal("unauthorized_client", (await response.JsonBody())["error"]?.GetValue<string>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_mastodon_token_is_not_a_subject_token()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "mastodon");
|
||||
var token = await Token(persona);
|
||||
|
||||
await AssertRefused(await Exchange(token, persona.Id, subjectTokenType: "urn:ietf:params:oauth:token-type:access_token"), "invalid_request");
|
||||
await AssertRefused(await Exchange(token, persona.Id));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Missing_parameters_are_a_client_error()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "missing");
|
||||
|
||||
await AssertRefused(await Exchange(persona.Root.Jwt, default));
|
||||
await AssertRefused(await Exchange(default, persona.Id), "invalid_request");
|
||||
await AssertRefused(await Exchange(persona.Root.Jwt, persona.Id, subjectTokenType: default), "invalid_request");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,106 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Tests.Support;
|
||||
using PrivaPub.Tests.Support.Host;
|
||||
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
namespace PrivaPub.Tests.Http
|
||||
{
|
||||
// A Pixelfed post's own place: stored from `location`, shown as `privapub.place`, changed by an edit.
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class PostPlaceTests : IAsyncLifetime
|
||||
{
|
||||
PrivaPubHost _host;
|
||||
Peer _peer;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_host = await PrivaPubHost.Shared();
|
||||
_peer = await Peer.Start();
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_peer != default)
|
||||
await _peer.DisposeAsync();
|
||||
}
|
||||
|
||||
static CancellationToken Token => TestContext.Current.CancellationToken;
|
||||
|
||||
static JsonObject Location(string name, string latitude, string longitude) => new()
|
||||
{
|
||||
["type"] = "Place", ["name"] = name, ["latitude"] = latitude, ["longitude"] = longitude, ["country"] = "Italy"
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public async Task A_pixelfed_place_is_shown_and_follows_the_authors_edits()
|
||||
{
|
||||
var alice = await _host.Mastodon("alice");
|
||||
var ann = new RemoteActor(_peer, "ann");
|
||||
var post = await _host.PublicPostFrom(ann, alice, "<p>the duomo</p>", note => note["location"] = Location("Milan", "45.4642", "9.19"));
|
||||
|
||||
var place = (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"];
|
||||
|
||||
Assert.Equal(("Milan", "Italy"), (place.Text("name"), place.Text("country")));
|
||||
Assert.Equal(45.4642, place["latitude"]!.GetValue<double>());
|
||||
Assert.Equal(9.19, place["longitude"]!.GetValue<double>());
|
||||
|
||||
await Edit(ann, alice, post, note => note["location"] = Location("Turin", "45.0703", "7.6869"));
|
||||
Assert.Equal("Turin", (await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"].Text("name"));
|
||||
|
||||
await Edit(ann, alice, post, _ => { });
|
||||
Assert.Null((await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"]);
|
||||
Assert.Null((await DB.Default.Find<Post>().MatchID(post.ID).ExecuteFirstAsync(Token)).Place);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_post_without_a_place_has_none()
|
||||
{
|
||||
var alice = await _host.Mastodon("alice");
|
||||
var post = await _host.PublicPostFrom(new RemoteActor(_peer, "plain"), alice);
|
||||
|
||||
Assert.Null((await alice.Client.Get($"/api/v1/statuses/{post.ID}")).Ok().Body["privapub"]!["place"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_local_post_never_carries_a_location()
|
||||
{
|
||||
var alice = await _host.Mastodon("alice");
|
||||
var status = await alice.Status("made here");
|
||||
|
||||
using var client = _host.Client();
|
||||
client.DefaultRequestHeaders.Accept.ParseAdd("application/activity+json");
|
||||
var note = JsonNode.Parse(await client.GetStringAsync(status.Text("uri"), Token))!;
|
||||
|
||||
Assert.Null(note["location"]);
|
||||
}
|
||||
|
||||
async Task Edit(RemoteActor author, Mastodon mentioned, Post post, Action<JsonObject> shape)
|
||||
{
|
||||
var note = new JsonObject
|
||||
{
|
||||
["id"] = post.ObjectURI,
|
||||
["type"] = "Note",
|
||||
["attributedTo"] = author.Id,
|
||||
["to"] = new JsonArray(MastodonHelpers.Public),
|
||||
["cc"] = new JsonArray(mentioned.Uri),
|
||||
["content"] = "<p>the duomo, again</p>",
|
||||
["published"] = post.CreationDate.ToString("O"),
|
||||
["updated"] = DateTime.UtcNow.ToString("O"),
|
||||
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = mentioned.Uri, ["name"] = "@" + mentioned.UserName })
|
||||
};
|
||||
shape(note);
|
||||
await _host.Deliver(author, mentioned.Mouth, new JsonObject
|
||||
{
|
||||
["id"] = $"{post.ObjectURI}/updates/{Guid.NewGuid():N}",
|
||||
["type"] = "Update",
|
||||
["actor"] = author.Id,
|
||||
["to"] = new JsonArray(MastodonHelpers.Public),
|
||||
["object"] = note
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.Tests.Support;
|
||||
using PrivaPub.Tests.Support.Host;
|
||||
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
|
||||
using GroupEntity = PrivaPub.Models.Group.Group;
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Tests.Http
|
||||
{
|
||||
// Deleting a root deletes its personas and their groups everywhere (owner decision 2026-10-04).
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class RootRemovalTests : IAsyncLifetime
|
||||
{
|
||||
PrivaPubHost _host;
|
||||
HttpClient _client;
|
||||
Peer _peer;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_host = await PrivaPubHost.Shared();
|
||||
_client = _host.Client();
|
||||
_peer = await Peer.Start();
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
_client?.Dispose();
|
||||
if (_peer != default)
|
||||
await _peer.DisposeAsync();
|
||||
}
|
||||
|
||||
async Task<HttpResponseMessage> AdminRemoves(params string[] rootIds)
|
||||
{
|
||||
var admin = await _host.Admin();
|
||||
using var client = _host.As(admin.Jwt);
|
||||
using var request = new HttpRequestMessage(HttpMethod.Delete, "/clientapi/admin/remove/users") { Content = JsonContent.Create(new { userIdList = rootIds }) };
|
||||
return await client.SendAsync(request, TestContext.Current.CancellationToken);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_removed_roots_personas_and_groups_are_deleted_everywhere_and_say_so()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var root = await _host.SignUp("leaving");
|
||||
var persona = await _host.Persona(root, "leaving");
|
||||
var follower = new RemoteActor(_peer, "fan");
|
||||
var member = new RemoteActor(_peer, "member");
|
||||
await DB.Default.SaveAsync(new Follower { LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = follower.Id, InboxURL = follower.Id + "/inbox" }, token);
|
||||
var group = await _host.Group(persona, community: true);
|
||||
var groupId = group["id"]!.GetValue<string>();
|
||||
await DB.Default.SaveAsync(new Follower { LocalActorId = groupId, LocalActorKind = LocalActorKind.Group, ActorURI = member.Id, InboxURL = member.Id + "/inbox" }, token);
|
||||
var post = await _host.Publish(persona, "goodbye soon");
|
||||
var apiToken = await _host.MastodonToken(persona);
|
||||
var since = DateTime.UtcNow.AddSeconds(-1);
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(root.Id)).StatusCode);
|
||||
|
||||
var toFollower = Assert.Single(await Jobs.Deliveries(follower.Id + "/inbox", since, token), d => d["type"]!.GetValue<string>() == "Delete");
|
||||
Assert.Equal(persona.ActorUri(), toFollower["object"]!.GetValue<string>());
|
||||
Assert.Equal(persona.ActorUri(), toFollower["actor"]!.GetValue<string>());
|
||||
var groupName = group["userName"]!.GetValue<string>();
|
||||
Assert.Contains(await Jobs.Deliveries(member.Id + "/inbox", since, token),
|
||||
d => d["type"]!.GetValue<string>() == "Delete" && d["object"]!.GetValue<string>() == $"{PrivaPubHost.Base}/peasants/{groupName}");
|
||||
var gone = await _client.Fetch($"/peasants/{persona.UserName}");
|
||||
Assert.Equal(HttpStatusCode.Gone, gone.Status);
|
||||
Assert.Equal("Person", gone.Json["formerType"]!.GetValue<string>());
|
||||
Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch($"/peasants/{groupName}")).Status);
|
||||
Assert.Equal(HttpStatusCode.Gone, (await _client.GetAsync($"/.well-known/webfinger?resource=acct:{persona.UserName}@{PrivaPubHost.Host}", token)).StatusCode);
|
||||
var stored = await DB.Default.Find<PostEntity>().MatchID(post.ID).ExecuteFirstAsync(token);
|
||||
Assert.NotNull(stored.DeletedAt);
|
||||
Assert.Null(stored.ContentHtml);
|
||||
Assert.NotNull((await DB.Default.Find<GroupEntity>().MatchID(groupId).ExecuteFirstAsync(token)).DeletionAt);
|
||||
var removed = await DB.Default.Find<RootUser>().MatchID(root.Id).ExecuteFirstAsync(token);
|
||||
Assert.Equal($"deleted-{root.Id}", removed.UserName);
|
||||
Assert.Null(removed.HashedPassword);
|
||||
using var app = _host.Client();
|
||||
app.DefaultRequestHeaders.Authorization = new("Bearer", apiToken);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, (await app.GetAsync("/api/v1/accounts/verify_credentials", token)).StatusCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_second_removal_does_not_collide_with_the_first()
|
||||
{
|
||||
var first = await _host.SignUp("first");
|
||||
var second = await _host.SignUp("second");
|
||||
|
||||
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(first.Id)).StatusCode);
|
||||
Assert.Equal(HttpStatusCode.OK, (await AdminRemoves(second.Id)).StatusCode);
|
||||
|
||||
Assert.NotNull((await DB.Default.Find<RootUser>().MatchID(second.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).DeletedAt);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_root_deletes_itself_only_with_its_password()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var root = await _host.SignUp("self");
|
||||
var persona = await _host.Persona(root, "self");
|
||||
using var client = _host.As(root.Jwt);
|
||||
|
||||
var refused = await client.PostJson("/clientapi/user/delete", new { password = "Not-The-Password-1" });
|
||||
Assert.Equal(HttpStatusCode.Forbidden, refused.StatusCode);
|
||||
Assert.Null((await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(token)).DeletionAt);
|
||||
|
||||
var accepted = await client.PostJson("/clientapi/user/delete", new { password = root.Password });
|
||||
Assert.Equal(HttpStatusCode.OK, accepted.StatusCode);
|
||||
Assert.NotNull((await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(token)).DeletionAt);
|
||||
Assert.Equal(HttpStatusCode.Unauthorized, (await client.GetAsync("/clientapi/user/sniff/again", token)).StatusCode);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -179,8 +179,10 @@ namespace PrivaPub.Tests.Http
|
||||
return (await client.Fetch("/nodeinfo/2.1", "application/json")).Json["usage"]!["localPosts"]!.GetValue<long>();
|
||||
}
|
||||
|
||||
// localPosts is Mastodon's: the sum of statuses_count, every local post but DMs and deleted ones, boosts included,
|
||||
// and circle and located posts too (owner decision 2026-10-04)
|
||||
[Fact]
|
||||
public async Task Local_posts_count_only_what_anyone_may_see()
|
||||
public async Task Local_posts_count_as_mastodon_counts_statuses()
|
||||
{
|
||||
var persona = await _host.Persona(await _host.SignUp(), "usage");
|
||||
var friend = await _host.Persona(await _host.SignUp(), "usagefriend");
|
||||
@@ -197,10 +199,10 @@ namespace PrivaPub.Tests.Http
|
||||
await _host.Reblog(friend, shown);
|
||||
var circle = await _host.FederatedGroup(persona, community: false);
|
||||
await _host.Publish(persona, new StatusDraft { Text = "circle only", PlainText = true, GroupId = circle.Id });
|
||||
Assert.Equal(start + 2, await LocalPosts());
|
||||
Assert.Equal(start + 6, await LocalPosts());//followers-only, located, the boost and the circle post; not the DM nor the copy
|
||||
|
||||
await _host.Remove(persona, shown);
|
||||
Assert.Equal(start + 1, await LocalPosts());
|
||||
Assert.Equal(start + 4, await LocalPosts());//the post and the boost of it
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
using PrivaPub.Models.Jobs;
|
||||
|
||||
using System.Net;
|
||||
|
||||
namespace PrivaPub.Tests.Infrastructure
|
||||
{
|
||||
public class CdnCatalogTests
|
||||
{
|
||||
static HttpResponseMessage Response(params (string Name, string Value)[] headers)
|
||||
{
|
||||
var response = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent("{}") };
|
||||
foreach (var (name, value) in headers)
|
||||
response.Headers.TryAddWithoutValidation(name, value);
|
||||
return response;
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("cf-ray", "8c1f0e2b4d5a1234-FRA", "cloudflare.com")]
|
||||
[InlineData("server", "cloudflare", "cloudflare.com")]
|
||||
[InlineData("x-amz-cf-id", "abc==", "cloudfront.net")]
|
||||
[InlineData("via", "1.1 5a1b.cloudfront.net (CloudFront)", "cloudfront.net")]
|
||||
[InlineData("x-served-by", "cache-fra-eddf8230045-FRA", "fastly.com")]
|
||||
[InlineData("server", "BunnyCDN-DE1-1078", "bunny.net")]
|
||||
[InlineData("akamai-grn", "0.1234", "akamai.com")]
|
||||
[InlineData("x-azure-ref", "20261004T101010Z-abc", "azure.microsoft.com")]
|
||||
[InlineData("x-vercel-id", "fra1::abc", "vercel.com")]
|
||||
[InlineData("x-iinfo", "1-2-3", "imperva.com")]
|
||||
public void A_cdn_is_recognised_by_its_fingerprint(string header, string value, string domain)
|
||||
{
|
||||
Assert.Equal(domain, CdnCatalog.FromResponse(Response((header, value)))?.Domain);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_plain_server_has_no_cdn_and_an_unknown_cdn_names_itself()
|
||||
{
|
||||
Assert.Null(CdnCatalog.FromResponse(Response(("server", "nginx"), ("via", "1.1 varnish"))));
|
||||
var named = CdnCatalog.FromResponse(Response(("x-cdn", "EdgeOne")));
|
||||
Assert.Equal(("EdgeOne", null, "edgeone"), (named.Name, named.Domain, named.Key));
|
||||
Assert.Null(CdnCatalog.FromResponse(Response(("x-cdn", "<script>"))));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Only_networks_that_carry_nothing_but_a_cdn_name_it()
|
||||
{
|
||||
Assert.Equal("Cloudflare", CdnCatalog.OfAsn(13335)?.Name);
|
||||
Assert.Equal("Akamai", CdnCatalog.OfAsn(20940)?.Name);
|
||||
Assert.Null(CdnCatalog.OfAsn(16509));//AWS: CloudFront only by its ranges or its headers
|
||||
Assert.Null(CdnCatalog.OfAsn(60068));//DataPacket also rents servers
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Every_published_list_format_is_read()
|
||||
{
|
||||
Assert.Equal(new[] { "173.245.48.0/20", "2400:cb00::/32" },
|
||||
CdnCatalog.Parse(CdnListFormat.Lines, "173.245.48.0/20\n# comment\n\n2400:cb00::/32\n").Select(n => n.ToString()));
|
||||
Assert.Equal(new[] { "89.187.188.227/32", "2400:52e0:1500::714:1/128" },
|
||||
CdnCatalog.Parse(CdnListFormat.StringArray, """["89.187.188.227","2400:52e0:1500::714:1"]""").Select(n => n.ToString()));
|
||||
Assert.Equal(new[] { "23.235.32.0/20", "2a04:4e40::/32" },
|
||||
CdnCatalog.Parse(CdnListFormat.AddressesJson, """{"addresses":["23.235.32.0/20"],"ipv6_addresses":["2a04:4e40::/32"]}""").Select(n => n.ToString()));
|
||||
Assert.Equal(new[] { "92.223.124.12/32", "2a03:90c0::/32" },
|
||||
CdnCatalog.Parse(CdnListFormat.AddressesJson, """{"addresses":["92.223.124.12/32"],"addresses_v6":["2a03:90c0::/32"]}""").Select(n => n.ToString()));
|
||||
Assert.Equal(new[] { "199.83.128.0/21", "2a02:e980::/29" },
|
||||
CdnCatalog.Parse(CdnListFormat.ImpervaJson, """{"ipRanges":["199.83.128.0/21"],"ipv6Ranges":["2a02:e980::/29"],"res":0}""").Select(n => n.ToString()));
|
||||
Assert.Equal(new[] { "13.32.0.0/15", "2600:9000::/28" },
|
||||
CdnCatalog.Parse(CdnListFormat.AwsCloudFront, """
|
||||
{"prefixes":[{"ip_prefix":"3.4.12.4/32","service":"AMAZON"},{"ip_prefix":"13.32.0.0/15","service":"CLOUDFRONT"}],
|
||||
"ipv6_prefixes":[{"ipv6_prefix":"2600:9000::/28","service":"CLOUDFRONT"},{"ipv6_prefix":"2406:daba::/40","service":"EC2"}]}
|
||||
""").Select(n => n.ToString()));
|
||||
Assert.Throws<FormatException>(() => CdnCatalog.Parse(CdnListFormat.Lines, "173.245.48.0/20\nnot an address"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void The_published_ranges_name_the_cdn_of_an_address()
|
||||
{
|
||||
var ranges = new CdnRanges();
|
||||
ranges.Replace(new CdnRangeSet { Key = "cloudflare.com", Name = "Cloudflare", Ranges = new() { "104.16.0.0/13", "2606:4700::/32" }, FetchedAt = DateTime.UtcNow });
|
||||
|
||||
Assert.Equal("Cloudflare", ranges.Of(IPAddress.Parse("104.18.2.3"))?.Name);
|
||||
Assert.Equal("Cloudflare", ranges.Of(IPAddress.Parse("::ffff:104.18.2.3"))?.Name);
|
||||
Assert.Equal("Cloudflare", ranges.Of(IPAddress.Parse("2606:4700::1"))?.Name);
|
||||
Assert.Null(ranges.Of(IPAddress.Parse("203.0.113.7")));
|
||||
Assert.Equal(2, ranges.Status["cloudflare.com"].Ranges);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Responses_leave_a_cdn_hint_for_their_host_until_one_comes_without()
|
||||
{
|
||||
var hints = new EdgeHints();
|
||||
var answers = new Queue<HttpResponseMessage>(new[] { Response(("cf-ray", "1-FRA")), Response(("server", "nginx")) });
|
||||
using var client = new HttpClient(new EdgeHintsHandler(hints) { InnerHandler = new Answering(answers) });
|
||||
|
||||
await client.GetAsync("https://social.example/.well-known/nodeinfo", TestContext.Current.CancellationToken);
|
||||
Assert.Equal("Cloudflare", hints.Of("social.example")?.Name);
|
||||
|
||||
await client.GetAsync("https://social.example/nodeinfo/2.1", TestContext.Current.CancellationToken);
|
||||
Assert.Null(hints.Of("social.example"));
|
||||
}
|
||||
|
||||
sealed class Answering : HttpMessageHandler
|
||||
{
|
||||
readonly Queue<HttpResponseMessage> _answers;
|
||||
|
||||
public Answering(Queue<HttpResponseMessage> answers) => _answers = answers;
|
||||
|
||||
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) =>
|
||||
Task.FromResult(_answers.Dequeue());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Infrastructure.Statistics;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Tests.Support;
|
||||
|
||||
using System.Net;
|
||||
|
||||
namespace PrivaPub.Tests.Infrastructure
|
||||
{
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class CdnUpdaterTests : IAsyncLifetime
|
||||
{
|
||||
Peer _peer;
|
||||
CdnRanges _ranges;
|
||||
CdnUpdater _updater;
|
||||
string _key;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_peer = await Peer.Start();
|
||||
_ranges = new CdnRanges();
|
||||
_updater = new CdnUpdater(new Clients(), new StaticOptions<StatisticsOptions>(new StatisticsOptions()), _ranges, NullLogger<CdnUpdater>.Instance);
|
||||
_key = $"edge{Guid.NewGuid():N}.example";
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_peer != default)
|
||||
await _peer.DisposeAsync();
|
||||
}
|
||||
|
||||
sealed class Clients : IHttpClientFactory
|
||||
{
|
||||
public HttpClient CreateClient(string name) => new();
|
||||
}
|
||||
|
||||
CdnProvider Provider() => new("Test Edge", _key, Array.Empty<int>(), new[]
|
||||
{
|
||||
new CdnList($"{_peer.A}/edge/v4", CdnListFormat.Lines),
|
||||
new CdnList($"{_peer.A}/edge/v6", CdnListFormat.StringArray)
|
||||
}, _ => false);
|
||||
|
||||
static CancellationToken Token => TestContext.Current.CancellationToken;
|
||||
|
||||
[Fact]
|
||||
public async Task The_published_ranges_are_downloaded_stored_and_used()
|
||||
{
|
||||
_peer.ServeText("/edge/v4", "198.51.100.0/24\n", "text/plain");
|
||||
_peer.ServeText("/edge/v6", """["2001:db8::/32","192.0.2.9"]""", "application/json");
|
||||
|
||||
Assert.Equal(1, await _updater.Update(new[] { Provider() }, Token));
|
||||
|
||||
var stored = await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == _key).ExecuteSingleAsync(Token);
|
||||
Assert.Equal(new[] { "198.51.100.0/24", "2001:db8::/32", "192.0.2.9/32" }, stored.Ranges);
|
||||
Assert.Equal("Test Edge", _ranges.Of(IPAddress.Parse("198.51.100.20"))?.Name);
|
||||
Assert.Equal("Test Edge", _ranges.Of(IPAddress.Parse("192.0.2.9"))?.Name);
|
||||
|
||||
var restarted = new CdnRanges();
|
||||
await restarted.Load(Token);
|
||||
Assert.Equal("Test Edge", restarted.Of(IPAddress.Parse("2001:db8::7"))?.Name);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("not a network")]
|
||||
[InlineData("")]
|
||||
public async Task A_broken_or_empty_list_keeps_the_stored_ranges(string broken)
|
||||
{
|
||||
_peer.ServeText("/edge/v4", "198.51.100.0/24", "text/plain");
|
||||
_peer.ServeText("/edge/v6", "[]", "application/json");
|
||||
Assert.Equal(1, await _updater.Update(new[] { Provider() }, Token));
|
||||
|
||||
_peer.ServeText("/edge/v4", broken, "text/plain");
|
||||
Assert.Equal(0, await _updater.Update(new[] { Provider() }, Token));
|
||||
|
||||
Assert.Equal(new[] { "198.51.100.0/24" }, (await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == _key).ExecuteSingleAsync(Token)).Ranges);
|
||||
Assert.Equal("Test Edge", _ranges.Of(IPAddress.Parse("198.51.100.1"))?.Name);
|
||||
}
|
||||
|
||||
// The real lists, from the CDNs themselves: set PRIVAPUB_TEST_CDN_LIVE=1 to check that every format still parses.
|
||||
[Fact]
|
||||
public async Task Every_cdns_real_list_still_parses()
|
||||
{
|
||||
Assert.SkipUnless(Environment.GetEnvironmentVariable("PRIVAPUB_TEST_CDN_LIVE") == "1", "set PRIVAPUB_TEST_CDN_LIVE=1 to download the real lists");
|
||||
var listed = CdnCatalog.All.Where(p => p.Lists.Length > 0).ToList();
|
||||
|
||||
Assert.Equal(listed.Count, await _updater.Update(listed, Token));
|
||||
|
||||
Assert.Equal("Cloudflare", _ranges.Of(IPAddress.Parse("104.16.0.1"))?.Name);
|
||||
foreach (var provider in listed)
|
||||
Assert.True(_ranges.Status[provider.Key].Ranges > 0, provider.Name);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_unreachable_list_changes_nothing()
|
||||
{
|
||||
Assert.Equal(0, await _updater.Update(new[] { Provider() }, Token));
|
||||
Assert.False(await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == _key).ExecuteAnyAsync(Token));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
@@ -45,7 +45,7 @@ namespace PrivaPub.Tests.Infrastructure
|
||||
Assert.NotNull(fix);
|
||||
Assert.Equal(2, fix.Country.Length);
|
||||
Assert.Equal(13335, fix.Asn);
|
||||
Assert.Equal("Cloudflare", CdnNetworks.Of(fix.Asn));
|
||||
Assert.Equal("Cloudflare", CdnCatalog.OfAsn(fix.Asn)?.Name);
|
||||
Assert.NotNull(fix.AsnOrg);
|
||||
Assert.NotNull(fix.Latitude);
|
||||
Assert.Equal(Math.Round(fix.Latitude.Value, 1), fix.Latitude);
|
||||
@@ -61,9 +61,9 @@ namespace PrivaPub.Tests.Infrastructure
|
||||
|
||||
Assert.Equal(IPAddress.Parse("203.0.113.7"), connected.Of("social.example"));
|
||||
Assert.Null(connected.Of("other.example"));
|
||||
Assert.Equal("Fastly", CdnNetworks.Of(54113));
|
||||
Assert.Null(CdnNetworks.Of(64496));
|
||||
Assert.Null(CdnNetworks.Of(default));
|
||||
Assert.Equal("Fastly", CdnCatalog.OfAsn(54113)?.Name);
|
||||
Assert.Null(CdnCatalog.OfAsn(64496));
|
||||
Assert.Null(CdnCatalog.OfAsn(default));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
@@ -78,16 +78,16 @@ namespace PrivaPub.Tests.Statistics
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void The_public_location_depends_on_size_and_on_a_cdn()
|
||||
public void The_public_location_is_where_the_server_was_located_unless_a_cdn_hides_it()
|
||||
{
|
||||
var geo = new InstanceGeo { Country = "DE", City = "Berlin", Latitude = 52.5, Longitude = 13.4, AsnOrg = "Hetzner", Asn = 24940 };
|
||||
|
||||
Assert.Equal(new PublicLocation("DE", "Berlin", 52.5, 13.4, "Hetzner", null), PublicGeo.Project(new RemoteInstance { Geo = geo, UsersTotal = 10 }, 10));
|
||||
Assert.Equal(new PublicLocation("DE", null, null, null, null, null), PublicGeo.Project(new RemoteInstance { Geo = geo, UsersTotal = 1 }, 10));
|
||||
Assert.Equal(new PublicLocation("DE", null, null, null, null, null), PublicGeo.Project(new RemoteInstance { Geo = geo }, 10));
|
||||
Assert.Equal(new PublicLocation("DE", "Berlin", 52.5, 13.4, "Hetzner", null), PublicGeo.Project(geo));
|
||||
Assert.Equal(new PublicLocation("DE", null, null, null, null, null), PublicGeo.Project(new InstanceGeo { Country = "DE" }));
|
||||
Assert.Equal(new PublicLocation(null, null, null, null, null, "Cloudflare"),
|
||||
PublicGeo.Project(new RemoteInstance { Geo = new InstanceGeo { Country = "US", City = "SF", Cdn = "Cloudflare" }, UsersTotal = 5000 }, 10));
|
||||
Assert.Null(PublicGeo.Project(new RemoteInstance(), 10));
|
||||
PublicGeo.Project(new InstanceGeo { Country = "US", City = "SF", Latitude = 37.8, Longitude = -122.4, Cdn = "Cloudflare" }));
|
||||
Assert.Null(PublicGeo.Project(new InstanceGeo()));
|
||||
Assert.Null(PublicGeo.Project(null));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -112,7 +112,8 @@ namespace PrivaPub.Tests.Statistics
|
||||
{
|
||||
readonly string _peer;
|
||||
|
||||
public PeerDescriber(IFederationHttp http, IGeoLocator geo, IConnectedAddresses addresses, string peer) : base(http, geo, addresses) => _peer = peer;
|
||||
public PeerDescriber(IFederationHttp http, IGeoLocator geo, IConnectedAddresses addresses, string peer, ICdnRanges ranges = default,
|
||||
IEdgeHints edges = default) : base(http, geo, addresses, ranges, edges) => _peer = peer;
|
||||
|
||||
protected override string Address(string url) => _peer + new Uri(url).PathAndQuery;
|
||||
}
|
||||
@@ -155,6 +156,31 @@ namespace PrivaPub.Tests.Statistics
|
||||
Assert.Equal(InstanceDescriber.Week(DateTime.UtcNow), snapshot.Week);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_cdn_is_found_by_its_published_ranges_first_then_by_its_headers()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var listed = $"listed{Guid.NewGuid():N}.example";
|
||||
var hinted = $"hinted{Guid.NewGuid():N}.example";
|
||||
ServeServer("/" + listed, "mastodon");
|
||||
ServeServer("/" + hinted, "mastodon");
|
||||
var addresses = new ConnectedAddresses();
|
||||
var ranges = new CdnRanges();
|
||||
ranges.Replace(new CdnRangeSet { Key = "cloudflare.com", Name = "Cloudflare", Ranges = new() { "127.0.0.0/8" }, FetchedAt = DateTime.UtcNow });
|
||||
var edges = new EdgeHints();
|
||||
edges.Remember("127.0.0.1", CdnCatalog.ByKey("fastly.com"));
|
||||
|
||||
await new PeerDescriber(Peer.Http(connected: addresses), new FixedGeo(), addresses, _peer.A + "/" + listed, ranges, edges).Describe(listed, crawled: false, default, token);
|
||||
await new PeerDescriber(Peer.Http(connected: addresses), new FixedGeo(), addresses, _peer.A + "/" + hinted, new CdnRanges(), edges).Describe(hinted, crawled: false, default, token);
|
||||
|
||||
var listedOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == listed).ExecuteSingleAsync(token);
|
||||
Assert.Equal(("Cloudflare", "cloudflare.com", "ranges"), (listedOne.Geo.Cdn, listedOne.Geo.CdnDomain, listedOne.Geo.CdnSource));
|
||||
var hintedOne = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == hinted).ExecuteSingleAsync(token);
|
||||
Assert.Equal(("Fastly", "fastly.com", "headers"), (hintedOne.Geo.Cdn, hintedOne.Geo.CdnDomain, hintedOne.Geo.CdnSource));
|
||||
var snapshot = await DB.Default.Find<RemoteInstanceSnapshot>().Match(s => s.Host == listed).ExecuteSingleAsync(token);
|
||||
Assert.Equal(("Cloudflare", "cloudflare.com", "ranges", "Amsterdam", 52.4), (snapshot.Cdn, snapshot.CdnDomain, snapshot.CdnSource, snapshot.City, snapshot.Latitude!.Value));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Without_api_v2_the_v1_api_is_read_and_an_unreachable_week_is_recorded_too()
|
||||
{
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.DataProtection;
|
||||
using Microsoft.AspNetCore.Hosting;
|
||||
using Microsoft.AspNetCore.Http;
|
||||
@@ -75,6 +75,7 @@ namespace PrivaPub.Tests.Support.Host
|
||||
["Federation:FetchLinkPreviews"] = "false",
|
||||
["Registrations:Mode"] = "Open",
|
||||
["Statistics:Geo:AutoUpdate"] = "false",
|
||||
["Statistics:Cdn:AutoUpdate"] = "false",
|
||||
["Media:Root"] = _mediaRoot,
|
||||
["Logging:LogLevel:Default"] = "Warning",
|
||||
["Serilog:MinimumLevel:Default"] = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_LOGS") == "1" ? "Information" : "Fatal"
|
||||
|
||||
@@ -49,7 +49,14 @@ namespace PrivaPub.Api.Mastodon.Auth
|
||||
.SetTokenEndpointUris("/oauth/token")
|
||||
.SetRevocationEndpointUris("/oauth/revoke")
|
||||
.SetConfigurationEndpointUris("/.well-known/openid-configuration", "/.well-known/oauth-authorization-server");
|
||||
options.AllowAuthorizationCodeFlow().AllowClientCredentialsFlow();
|
||||
options.AllowAuthorizationCodeFlow().AllowClientCredentialsFlow().AllowTokenExchangeFlow();
|
||||
//the only subject token accepted is the root JWT, for PersonaExchange; OpenIddict's own tokens are not exchanged
|
||||
options.Configure(server =>
|
||||
{
|
||||
server.SubjectTokenTypes.Clear();
|
||||
server.SubjectTokenTypes.Add(PersonaExchange.SubjectTokenType);
|
||||
});
|
||||
options.AddEventHandler(RootJwtSubjectToken.Descriptor);
|
||||
options.RegisterScopes(MastodonScopes.All);
|
||||
options.UseReferenceAccessTokens();
|
||||
options.SetAccessTokenLifetime(null);
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
using Microsoft.IdentityModel.Tokens;
|
||||
|
||||
using OpenIddict.Abstractions;
|
||||
using OpenIddict.Server;
|
||||
|
||||
using PrivaPub.Services;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Security.Claims;
|
||||
|
||||
using static OpenIddict.Abstractions.OpenIddictConstants;
|
||||
using static OpenIddict.Server.OpenIddictServerEvents;
|
||||
|
||||
namespace PrivaPub.Api.Mastodon.Auth
|
||||
{
|
||||
// One sign-in for the first-party client (decePubClient): it signs in on /clientapi, then exchanges that root JWT for one
|
||||
// persona's Mastodon token (RFC 8693, grant_type urn:ietf:params:oauth:grant-type:token-exchange, subject_token_type
|
||||
// urn:ietf:params:oauth:token-type:jwt, avatar_id). Only the seeded first-party application holds the grant permission,
|
||||
// and the issued token is the same kind the authorization code flow issues: subject the avatar, never the root.
|
||||
public static class PersonaExchange
|
||||
{
|
||||
public const string SubjectTokenType = "urn:ietf:params:oauth:token-type:jwt";
|
||||
public const string AvatarParameter = "avatar_id";
|
||||
//the root the subject token belongs to; it lives on the subject token's principal only, which is never stored or issued
|
||||
public const string RootClaim = "privapub_root";
|
||||
|
||||
public static string FirstPartyClientId(IConfiguration configuration) =>
|
||||
configuration["ClientApi:FirstPartyClientId"] is { Length: > 0 } clientId ? clientId : "decepub";
|
||||
|
||||
//the first-party application: public (a browser keeps no secret), token exchange only, no redirect
|
||||
public static async Task EnsureFirstPartyClient(IServiceProvider services, CancellationToken token)
|
||||
{
|
||||
using var scope = services.CreateScope();
|
||||
var applications = scope.ServiceProvider.GetRequiredService<IOpenIddictApplicationManager>();
|
||||
var clientId = FirstPartyClientId(scope.ServiceProvider.GetRequiredService<IConfiguration>());
|
||||
var descriptor = new OpenIddictApplicationDescriptor
|
||||
{
|
||||
ClientId = clientId,
|
||||
ClientType = ClientTypes.Public,
|
||||
ConsentType = ConsentTypes.Implicit,
|
||||
DisplayName = "decePubClient",
|
||||
Permissions =
|
||||
{
|
||||
Permissions.Endpoints.Token,
|
||||
Permissions.Endpoints.Revocation,
|
||||
Permissions.GrantTypes.TokenExchange
|
||||
}
|
||||
};
|
||||
foreach (var scopeName in new[] { "read", "write", "follow" })
|
||||
descriptor.Permissions.Add(Permissions.Prefixes.Scope + scopeName);
|
||||
|
||||
var existing = await applications.FindByClientIdAsync(clientId, token);
|
||||
if (existing == default)
|
||||
{
|
||||
await applications.CreateAsync(descriptor, token);
|
||||
return;
|
||||
}
|
||||
await applications.PopulateAsync(existing, descriptor, token);
|
||||
await applications.UpdateAsync(existing, token);
|
||||
}
|
||||
}
|
||||
|
||||
// Validates a root JWT given as a token exchange subject token, before OpenIddict tries it as one of its own tokens.
|
||||
// The principal it builds names the root under PersonaExchange.RootClaim for TokenController, which signs in a fresh
|
||||
// identity for the avatar.
|
||||
public sealed class RootJwtSubjectToken : IOpenIddictServerHandler<ValidateTokenContext>
|
||||
{
|
||||
public static OpenIddictServerHandlerDescriptor Descriptor { get; } = OpenIddictServerHandlerDescriptor.CreateBuilder<ValidateTokenContext>()
|
||||
.UseScopedHandler<RootJwtSubjectToken>()
|
||||
.SetOrder(OpenIddictServerHandlers.Protection.ValidateIdentityModelToken.Descriptor.Order - 500)
|
||||
.SetType(OpenIddictServerHandlerType.Custom)
|
||||
.Build();
|
||||
|
||||
readonly IConfiguration _configuration;
|
||||
readonly DbEntities _dbEntities;
|
||||
|
||||
public RootJwtSubjectToken(IConfiguration configuration, DbEntities dbEntities)
|
||||
{
|
||||
_configuration = configuration;
|
||||
_dbEntities = dbEntities;
|
||||
}
|
||||
|
||||
public async ValueTask HandleAsync(ValidateTokenContext context)
|
||||
{
|
||||
if (context.ValidTokenTypes.Count != 1 || !context.ValidTokenTypes.Contains(PersonaExchange.SubjectTokenType))
|
||||
return;
|
||||
|
||||
var root = await RootJwt.Root(context.Token, _configuration, _dbEntities, context.Transaction.CancellationToken);
|
||||
if (root == default)
|
||||
{
|
||||
context.Reject(Errors.InvalidGrant, "The session token is not valid.");
|
||||
return;
|
||||
}
|
||||
|
||||
var identity = new ClaimsIdentity(TokenValidationParameters.DefaultAuthenticationType, Claims.Name, Claims.Role);
|
||||
identity.SetClaim(PersonaExchange.RootClaim, root.ID);
|
||||
var principal = new ClaimsPrincipal(identity)
|
||||
.SetTokenType(PersonaExchange.SubjectTokenType)
|
||||
.SetCreationDate(DateTimeOffset.UtcNow);
|
||||
if (context.ValidPresenters.Count > 0)
|
||||
principal.SetPresenters(context.ValidPresenters);
|
||||
context.Principal = principal;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using Microsoft.AspNetCore;
|
||||
using Microsoft.AspNetCore;
|
||||
using Microsoft.AspNetCore.Authentication;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
using Microsoft.IdentityModel.Tokens;
|
||||
@@ -48,9 +48,33 @@ namespace PrivaPub.Api.Mastodon.Auth
|
||||
return SignIn(new ClaimsPrincipal(identity), OpenIddictServerAspNetCoreDefaults.AuthenticationScheme);
|
||||
}
|
||||
|
||||
if (request.IsTokenExchangeGrantType())
|
||||
return await Persona(request, token);
|
||||
|
||||
return Refuse(Errors.UnsupportedGrantType, "The grant type is not supported.");
|
||||
}
|
||||
|
||||
//PersonaExchange: the root JWT, validated by RootJwtSubjectToken, for a token of one of the root's personas
|
||||
async Task<IActionResult> Persona(OpenIddictRequest request, CancellationToken token)
|
||||
{
|
||||
var subject = (await HttpContext.AuthenticateAsync(OpenIddictServerAspNetCoreDefaults.AuthenticationScheme)).Principal;
|
||||
var rootId = subject?.GetClaim(PersonaExchange.RootClaim);
|
||||
var avatarId = (string)request[PersonaExchange.AvatarParameter];
|
||||
if (string.IsNullOrEmpty(rootId) || string.IsNullOrEmpty(avatarId))
|
||||
return Refuse(Errors.InvalidGrant, "The persona can no longer be used.");
|
||||
var owns = await _dbEntities.RootToAvatars.Match(r => r.RootId == rootId && r.AvatarId == avatarId).ExecuteAnyAsync(token);
|
||||
var avatar = owns ? await _dbEntities.Avatars.MatchID(avatarId).ExecuteFirstAsync(token) : default;
|
||||
if (avatar == default || !await Usable(avatarId, token))
|
||||
return Refuse(Errors.InvalidGrant, "The persona can no longer be used.");
|
||||
|
||||
var identity = new ClaimsIdentity(TokenValidationParameters.DefaultAuthenticationType, Claims.Name, Claims.Role);
|
||||
identity.SetClaim(Claims.Subject, avatar.ID);
|
||||
identity.SetClaim(Claims.Name, avatar.UserName);
|
||||
identity.SetScopes(MastodonScopes.Parse(request.Scope).Intersect(new[] { "read", "write", "follow" }));
|
||||
identity.SetDestinations(_ => new[] { Destinations.AccessToken });
|
||||
return SignIn(new ClaimsPrincipal(identity), OpenIddictServerAspNetCoreDefaults.AuthenticationScheme);
|
||||
}
|
||||
|
||||
async Task<bool> Usable(string avatarId, CancellationToken token)
|
||||
{
|
||||
if (string.IsNullOrEmpty(avatarId))
|
||||
|
||||
@@ -71,8 +71,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
avatar.Settings.IsDiscoverable = discoverable;
|
||||
if (Params.Bool("indexable") is { } indexable)
|
||||
avatar.Settings.IsIndexable = indexable;
|
||||
if (Params.Bool("hide_collections") is { } hide)
|
||||
avatar.Settings.HideCollections = hide;
|
||||
// hide_collections stays true: who follows whom is never listed here, only counted (FEDERATION.md)
|
||||
if (Params.Get("source[privacy]") is { } privacy && privacy is "public" or "unlisted" or "private")
|
||||
avatar.Settings.DefaultVisibility = privacy;
|
||||
if (Params.Bool("source[sensitive]") is { } sensitive)
|
||||
@@ -120,7 +119,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
}
|
||||
var userName = parts[0];
|
||||
var domain = parts[1].ToLowerInvariant();
|
||||
var foreign = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain).ExecuteFirstAsync(token);
|
||||
var foreign = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain && !f.DeletionAt.HasValue).ExecuteFirstAsync(token);
|
||||
return foreign == default ? NotFoundError() : Json(_mapper.Foreign(foreign));
|
||||
}
|
||||
|
||||
@@ -197,8 +196,9 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
var (local, _) = await Find(id, token);
|
||||
if (local == default || local.Id != MyId)
|
||||
return Json(Array.Empty<Account>());
|
||||
var followers = await _dbEntities.Followers.Match(f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted)
|
||||
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
|
||||
var followers = await Page.From(Params, Limit(40, 80))
|
||||
.Fetch(_dbEntities.Followers.Match(f => f.LocalActorId == local.Id && f.LocalActorKind == local.Kind && f.IsAccepted), f => f.ID, token);
|
||||
Link($"/api/v1/accounts/{id}/followers", followers.LastOrDefault()?.ID, followers.FirstOrDefault()?.ID);
|
||||
return Json(await AccountsFor(followers.Select(f => f.ActorURI), token));
|
||||
}
|
||||
|
||||
@@ -208,8 +208,9 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
var (local, _) = await Find(id, token);
|
||||
if (local == default || local.Id != MyId)
|
||||
return Json(Array.Empty<Account>());
|
||||
var following = await _dbEntities.Followings.Match(f => f.AvatarId == local.Id && f.State == FollowState.Accepted)
|
||||
.Sort(f => f.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
|
||||
var following = await Page.From(Params, Limit(40, 80))
|
||||
.Fetch(_dbEntities.Followings.Match(f => f.AvatarId == local.Id && f.State == FollowState.Accepted), f => f.ID, token);
|
||||
Link($"/api/v1/accounts/{id}/following", following.LastOrDefault()?.ID, following.FirstOrDefault()?.ID);
|
||||
var accounts = await _mapper.Accounts(following.Select(f => f.TargetAccountId), token);
|
||||
return Json(following.Select(f => accounts.GetValueOrDefault(f.TargetAccountId)).Where(a => a != default).ToList());
|
||||
}
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
using Microsoft.AspNetCore.Authorization;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
|
||||
using PrivaPub.Api.Mastodon.Infrastructure;
|
||||
using PrivaPub.Domain.Statistics;
|
||||
|
||||
namespace PrivaPub.Api.Mastodon.Controllers
|
||||
{
|
||||
// The CDNs in front of the servers this server knows, each named by its own domain (owner decision 2026-10-04): how many
|
||||
// servers are behind it, how it is recognised, and week by week who joined and who left.
|
||||
public class CdnsController : MastodonController
|
||||
{
|
||||
const int MaxWeeks = 260;
|
||||
|
||||
readonly IServerPlaces _places;
|
||||
|
||||
public CdnsController(IServerPlaces places) => _places = places;
|
||||
|
||||
[HttpGet("/api/privapub/v1/cdns"), Scope("read", requiresUser: false), AllowAnonymous]
|
||||
public async Task<IActionResult> List(CancellationToken token) => Json((await _places.Cdns(token)).Select(Summary).ToList());
|
||||
|
||||
[HttpGet("/api/privapub/v1/cdns/{key}"), Scope("read", requiresUser: false), AllowAnonymous]
|
||||
public async Task<IActionResult> Detail(string key, CancellationToken token)
|
||||
{
|
||||
var history = await _places.Cdn(key, Math.Clamp(Params.Int("weeks") ?? 52, 1, MaxWeeks), token);
|
||||
if (history == default)
|
||||
return NotFoundError();
|
||||
var detail = new CdnDetail
|
||||
{
|
||||
Hosts = history.Hosts,
|
||||
HostsTruncated = history.Group.Servers > history.Hosts.Count,
|
||||
Weeks = history.Weeks.Select(w => new CdnWeekEntry { Week = w.Week, Servers = w.Servers, Joined = w.Joined, Left = w.Left }).ToList()
|
||||
};
|
||||
Fill(detail, history.Group);
|
||||
return Json(detail);
|
||||
}
|
||||
|
||||
static CdnSummary Summary(CdnGroup group)
|
||||
{
|
||||
var summary = new CdnSummary();
|
||||
Fill(summary, group);
|
||||
return summary;
|
||||
}
|
||||
|
||||
static void Fill(CdnSummary summary, CdnGroup group)
|
||||
{
|
||||
summary.Key = group.Key;
|
||||
summary.Name = group.Name;
|
||||
summary.Domain = group.Domain;
|
||||
summary.Servers = group.Servers;
|
||||
summary.Ranges = group.Ranges;
|
||||
summary.RangesUpdatedAt = group.RangesUpdatedAt is { } at ? MastodonJson.Time(at) : default;
|
||||
summary.DetectedBy = group.DetectedBy;
|
||||
}
|
||||
|
||||
public class CdnSummary
|
||||
{
|
||||
public string Key { get; set; }
|
||||
public string Name { get; set; }
|
||||
public string Domain { get; set; }
|
||||
public int Servers { get; set; }
|
||||
public int Ranges { get; set; }
|
||||
public string RangesUpdatedAt { get; set; }
|
||||
public List<string> DetectedBy { get; set; } = new();
|
||||
}
|
||||
|
||||
public class CdnDetail : CdnSummary
|
||||
{
|
||||
public List<string> Hosts { get; set; } = new();
|
||||
public bool HostsTruncated { get; set; }
|
||||
public List<CdnWeekEntry> Weeks { get; set; } = new();
|
||||
}
|
||||
|
||||
public class CdnWeekEntry
|
||||
{
|
||||
public string Week { get; set; }
|
||||
public int Servers { get; set; }
|
||||
public List<string> Joined { get; set; } = new();
|
||||
public List<string> Left { get; set; } = new();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -30,28 +30,50 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
|
||||
bool Open => _registrations.CurrentValue.IsOpen;
|
||||
|
||||
// the languages this server is set up for (AppConfiguration:SupportedLanguages), "en" when none are named
|
||||
public static IReadOnlyList<string> LanguageCodes(IServiceProvider services)
|
||||
{
|
||||
var configured = services.GetRequiredService<Microsoft.Extensions.Options.IOptionsMonitor<Models.AppConfiguration>>().CurrentValue.SupportedLanguages;
|
||||
var codes = (configured ?? new List<string>()).Where(c => !string.IsNullOrWhiteSpace(c)).Select(c => c.Split('-')[0].ToLowerInvariant()).Distinct().ToList();
|
||||
return codes.Count == 0 ? new[] { "en" } : codes;
|
||||
}
|
||||
|
||||
string Domain => new Uri(_localActors.BaseAddress).Authority;
|
||||
|
||||
static object Statuses => new { max_characters = 5000, max_media_attachments = 4, characters_reserved_per_url = 23 };
|
||||
// every limit is what the server enforces, read from where it is enforced
|
||||
static object Statuses => new { max_characters = PrivaPub.Domain.Statuses.StatusService.MaxCharacters, max_media_attachments = 4, characters_reserved_per_url = 23 };
|
||||
|
||||
static object MediaAttachments => new
|
||||
object MediaAttachments
|
||||
{
|
||||
supported_mime_types = new[] { "image/jpeg", "image/png", "image/gif", "image/webp", "video/mp4", "video/webm", "audio/mpeg", "audio/ogg" },
|
||||
image_size_limit = 16 * 1024 * 1024,
|
||||
image_matrix_limit = 16_777_216,
|
||||
video_size_limit = 99 * 1024 * 1024,
|
||||
video_frame_rate_limit = 60,
|
||||
video_matrix_limit = 2_304_000
|
||||
};
|
||||
get
|
||||
{
|
||||
var media = HttpContext.RequestServices.GetRequiredService<IOptionsMonitor<PrivaPub.Domain.Media.MediaOptions>>().CurrentValue;
|
||||
return new
|
||||
{
|
||||
supported_mime_types = PrivaPub.Domain.Media.MediaService.SupportedTypes,
|
||||
image_size_limit = media.MaxImageBytes,
|
||||
image_matrix_limit = media.MaxImageSide * media.MaxImageSide,
|
||||
video_size_limit = media.MaxVideoBytes,
|
||||
video_frame_rate_limit = 60,
|
||||
video_matrix_limit = 2_304_000
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
static object Polls => new { max_options = 4, max_characters_per_option = 50, min_expiration = 300, max_expiration = 2_629_746 };
|
||||
static object Polls => new
|
||||
{
|
||||
max_options = PrivaPub.Domain.Statuses.PollService.MaxOptions,
|
||||
max_characters_per_option = PrivaPub.Domain.Statuses.PollService.MaxOptionLength,
|
||||
min_expiration = PrivaPub.Domain.Statuses.PollService.MinExpiration,
|
||||
max_expiration = PrivaPub.Domain.Statuses.PollService.MaxExpiration
|
||||
};
|
||||
|
||||
[HttpGet("/api/v1/instance"), AllowAnonymous]
|
||||
public async Task<IActionResult> V1(CancellationToken token)
|
||||
{
|
||||
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token);
|
||||
var statuses = await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token);
|
||||
var domains = (await DB.Default.Find<ForeignAvatar, string>().Project(a => a.Domain).ExecuteAsync(token)).Distinct().Count();
|
||||
var users = await Counted.Users(token);
|
||||
var statuses = await Counted.LocalPosts(token);
|
||||
var domains = await KnownServers(token);
|
||||
return Json(new
|
||||
{
|
||||
uri = Domain,
|
||||
@@ -60,10 +82,10 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
description = Description,
|
||||
email = string.Empty,
|
||||
version = Version,
|
||||
urls = new { streaming_api = $"wss://{Domain}" },
|
||||
urls = new { },//no streaming API yet, so none is advertised
|
||||
stats = new { user_count = users, status_count = statuses, domain_count = domains },
|
||||
thumbnail = $"{_localActors.BaseAddress}/media/missing-header.png",
|
||||
languages = new[] { "en" },
|
||||
languages = LanguageCodes(HttpContext.RequestServices),
|
||||
registrations = Open,
|
||||
approval_required = false,
|
||||
invites_enabled = true,//a group invitation always creates an account
|
||||
@@ -73,8 +95,14 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
});
|
||||
}
|
||||
|
||||
// the servers we have exchanged activities with, as the statistics count them. Which ones is not published
|
||||
// (/instance/peers stays empty, as on a Mastodon with its peers API off): on a small server it would say who its
|
||||
// personas talk to.
|
||||
static async Task<long> KnownServers(CancellationToken token) =>
|
||||
await DB.Default.CountAsync<Models.Jobs.RemoteInstance>(i => i.Seen == "touched", token);
|
||||
|
||||
[HttpGet("/api/v2/instance"), AllowAnonymous]
|
||||
public IActionResult V2() => Json(new
|
||||
public async Task<IActionResult> V2(CancellationToken token) => Json(new
|
||||
{
|
||||
domain = Domain,
|
||||
title = "PrivaPub",
|
||||
@@ -82,13 +110,13 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
api_versions = new { mastodon = 7 },
|
||||
source_url = "https://git.thepra.dev/thepra/SocialPub",
|
||||
description = Description,
|
||||
usage = new { users = new { active_month = 0 } },
|
||||
usage = new { users = new { active_month = await Counted.ActiveUsers(30, token) } },
|
||||
thumbnail = new { url = $"{_localActors.BaseAddress}/media/missing-header.png" },
|
||||
languages = new[] { "en" },
|
||||
languages = LanguageCodes(HttpContext.RequestServices),
|
||||
configuration = new
|
||||
{
|
||||
urls = new { streaming = $"wss://{Domain}" },
|
||||
accounts = new { max_featured_tags = 0, max_pinned_statuses = 0 },
|
||||
urls = new { },//no streaming API yet, so none is advertised
|
||||
accounts = new { max_featured_tags = 0, max_pinned_statuses = StatusesController.MaxPins },
|
||||
statuses = Statuses,
|
||||
media_attachments = MediaAttachments,
|
||||
polls = Polls,
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
using Microsoft.AspNetCore.Authorization;
|
||||
using Microsoft.AspNetCore.Authorization;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Api.Mastodon.Infrastructure;
|
||||
using PrivaPub.Domain.Privacy;
|
||||
using PrivaPub.Domain.Statistics;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Infrastructure;
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Models.Post;
|
||||
@@ -19,9 +24,25 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
{
|
||||
public class ProvenanceController : MastodonController
|
||||
{
|
||||
readonly DbEntities _dbEntities;
|
||||
const int MaxHosts = 40;
|
||||
|
||||
public ProvenanceController(DbEntities dbEntities) => _dbEntities = dbEntities;
|
||||
readonly DbEntities _dbEntities;
|
||||
const int MaxWeeks = 260;
|
||||
|
||||
readonly ISelfLocation _self;
|
||||
readonly IServerPlaces _places;
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly IOptionsMonitor<RegistrationOptions> _registrations;
|
||||
|
||||
public ProvenanceController(DbEntities dbEntities, ISelfLocation self, IServerPlaces places, ILocalActorService localActors,
|
||||
IOptionsMonitor<RegistrationOptions> registrations)
|
||||
{
|
||||
_dbEntities = dbEntities;
|
||||
_self = self;
|
||||
_places = places;
|
||||
_localActors = localActors;
|
||||
_registrations = registrations;
|
||||
}
|
||||
|
||||
[HttpGet("/api/privapub/v1/statuses/{id}/provenance"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
|
||||
public async Task<IActionResult> Status(string id, CancellationToken token)
|
||||
@@ -38,18 +59,104 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
?? await DB.Default.Find<ObjectRecord>().Match(r => r.ObjectURI == post.ObjectURI).ExecuteFirstAsync(token);
|
||||
var host = record?.Host ?? (Uri.TryCreate(post.ObjectURI, UriKind.Absolute, out var uri) ? uri.Host.ToLowerInvariant() : default);
|
||||
var instance = host == default ? default : await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteFirstAsync(token);
|
||||
return Json(Describe(post, record, instance));
|
||||
var beforeCdn = instance?.Geo?.Cdn == default ? default : (await _places.BeforeCdn(new[] { instance.Host }, token)).GetValueOrDefault(instance.Host);
|
||||
return Json(Describe(post, record, instance, beforeCdn));
|
||||
}
|
||||
|
||||
[HttpGet("/api/privapub/v1/instances/{host}"), Scope("read", requiresUser: false), AllowAnonymous]
|
||||
public async Task<IActionResult> Instance(string host, CancellationToken token)
|
||||
{
|
||||
host = host?.Trim().ToLowerInvariant();
|
||||
var instance = await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteFirstAsync(token);
|
||||
return instance == default ? NotFoundError() : Json(Describe(instance));
|
||||
var described = await Describe(new[] { host }, token);
|
||||
return described.Count == 0 ? NotFoundError() : Json(described[0]);
|
||||
}
|
||||
|
||||
static Provenance Describe(PostEntity post, ObjectRecord record, RemoteInstance instance)
|
||||
//up to MaxHosts at once (host[]=...), in the order asked; hosts this server does not know are left out
|
||||
[HttpGet("/api/privapub/v1/instances"), Scope("read", requiresUser: false), AllowAnonymous]
|
||||
public async Task<IActionResult> Instances(CancellationToken token) => Json(await Describe(Params.List("host"), token));
|
||||
|
||||
//the server's weekly snapshots, newest first (weeks=, at most MaxWeeks): its place as shown publicly, its CDN, its size
|
||||
[HttpGet("/api/privapub/v1/instances/{host}/history"), Scope("read", requiresUser: false), AllowAnonymous]
|
||||
public async Task<IActionResult> History(string host, CancellationToken token)
|
||||
{
|
||||
host = host?.Trim().ToLowerInvariant();
|
||||
if (string.IsNullOrEmpty(host) || host.Length > 253 || !await DB.Default.Find<RemoteInstance>().Match(i => i.Host == host).ExecuteAnyAsync(token))
|
||||
return NotFoundError();
|
||||
var weeks = Math.Clamp(Params.Int("weeks") ?? 52, 1, MaxWeeks);
|
||||
return Json((await _places.History(host, weeks, token)).Select(s => new InstanceWeek
|
||||
{
|
||||
Week = s.Week,
|
||||
TakenAt = MastodonJson.Time(s.TakenAt),
|
||||
Reachable = s.Reachable,
|
||||
Software = s.Software,
|
||||
Version = s.SoftwareVersion,
|
||||
UsersTotal = s.UsersTotal,
|
||||
UsersActiveMonth = s.UsersActiveMonth,
|
||||
LocalPosts = s.LocalPosts,
|
||||
OpenRegistrations = s.OpenRegistrations,
|
||||
Geo = Located(PublicGeo.Project(new InstanceGeo
|
||||
{
|
||||
Country = s.Country, City = s.City, Latitude = s.Latitude, Longitude = s.Longitude, Asn = s.Asn,
|
||||
Cdn = s.Cdn, CdnDomain = s.CdnDomain, CdnSource = s.CdnSource, Ipv6 = s.Ipv6
|
||||
}), new InstanceGeo { Cdn = s.Cdn, CdnDomain = s.CdnDomain ?? CdnCatalog.ByKey(s.Cdn)?.Key, CdnSource = s.CdnSource })
|
||||
}).ToList());
|
||||
}
|
||||
|
||||
async Task<List<InstanceDescription>> Describe(IEnumerable<string> hosts, CancellationToken token)
|
||||
{
|
||||
var wanted = hosts.Select(h => h?.Trim().ToLowerInvariant()).Where(h => !string.IsNullOrEmpty(h) && h.Length <= 253)
|
||||
.Distinct().Take(MaxHosts).ToList();
|
||||
var selfHost = Uri.TryCreate(_localActors.BaseAddress, UriKind.Absolute, out var self) ? self.Host.ToLowerInvariant() : default;
|
||||
var remote = await DB.Default.Find<RemoteInstance>().Match(i => wanted.Contains(i.Host)).ExecuteAsync(token);
|
||||
var beforeCdn = await _places.BeforeCdn(remote.Where(i => i.Geo?.Cdn != default).Select(i => i.Host), token);
|
||||
var described = new List<InstanceDescription>();
|
||||
foreach (var host in wanted)
|
||||
{
|
||||
if (host == selfHost)
|
||||
described.Add(await DescribeSelf(host, token));
|
||||
else if (remote.FirstOrDefault(i => i.Host == host) is { } instance)
|
||||
described.Add(Describe(instance, beforeCdn.GetValueOrDefault(host)));
|
||||
}
|
||||
return described;
|
||||
}
|
||||
|
||||
async Task<InstanceDescription> DescribeSelf(string host, CancellationToken token)
|
||||
{
|
||||
var geo = await _self.Get(token);
|
||||
return new InstanceDescription
|
||||
{
|
||||
Host = host,
|
||||
Software = "privapub",
|
||||
Version = BuildInfo.Ref,
|
||||
NodeName = "PrivaPub",
|
||||
Protocols = new() { "activitypub" },
|
||||
OpenRegistrations = _registrations.CurrentValue.IsOpen,
|
||||
DescribedAt = MastodonJson.Time(DateTime.UtcNow),
|
||||
Geo = Located(PublicGeo.Project(geo), geo)
|
||||
};
|
||||
}
|
||||
|
||||
//the public projection of a server's place (PublicGeo); for a CDN-fronted one, the CDN's domain, how it was found and
|
||||
//where the server was before it
|
||||
static InstanceLocation Located(PublicLocation location, InstanceGeo geo, PlaceBeforeCdn beforeCdn = default) => location == default ? default : new InstanceLocation
|
||||
{
|
||||
Precision = location.Cdn != default ? "cdn" : location.Latitude != default ? "city" : "country",
|
||||
Country = location.Country,
|
||||
City = location.City,
|
||||
Latitude = location.Latitude,
|
||||
Longitude = location.Longitude,
|
||||
Network = location.Network,
|
||||
Cdn = location.Cdn,
|
||||
CdnDomain = location.Cdn != default ? ServerPlaces.KeyOf(geo) : default,
|
||||
CdnSource = location.Cdn != default ? geo?.CdnSource : default,
|
||||
BeforeCdn = location.Cdn == default || beforeCdn == default ? default : new InstancePlace
|
||||
{
|
||||
Week = beforeCdn.Week, Country = beforeCdn.Country, City = beforeCdn.City, Latitude = beforeCdn.Latitude, Longitude = beforeCdn.Longitude
|
||||
},
|
||||
Source = geo?.Source,
|
||||
LocatedAt = geo?.LocatedAt is { } at ? MastodonJson.Time(at) : default
|
||||
};
|
||||
|
||||
Provenance Describe(PostEntity post, ObjectRecord record, RemoteInstance instance, PlaceBeforeCdn beforeCdn)
|
||||
{
|
||||
var raw = record?.Raw == default ? default : JsonNode.Parse(record.Raw) as JsonObject;
|
||||
return new Provenance
|
||||
@@ -86,11 +193,11 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
RawHash = r.RawHash,
|
||||
Raw = r.Raw == default ? default : JsonNode.Parse(r.Raw)
|
||||
}).ToList() ?? new(),
|
||||
Instance = instance == default ? default : Describe(instance)
|
||||
Instance = instance == default ? default : Describe(instance, beforeCdn)
|
||||
};
|
||||
}
|
||||
|
||||
static InstanceDescription Describe(RemoteInstance instance) => new()
|
||||
static InstanceDescription Describe(RemoteInstance instance, PlaceBeforeCdn beforeCdn) => new()
|
||||
{
|
||||
Host = instance.Host,
|
||||
Software = instance.Software,
|
||||
@@ -107,7 +214,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
UnavailableUntil = instance.UnavailableUntil is { } until ? MastodonJson.Time(until) : default,
|
||||
LastSuccessAt = instance.LastSuccessAt is { } success ? MastodonJson.Time(success) : default,
|
||||
LastFailureAt = instance.LastFailureAt is { } failure ? MastodonJson.Time(failure) : default
|
||||
}
|
||||
},
|
||||
Geo = Located(PublicGeo.Project(instance.Geo), instance.Geo, beforeCdn)
|
||||
};
|
||||
|
||||
public class Provenance
|
||||
@@ -170,6 +278,48 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
public string DescriptionError { get; set; }
|
||||
public JsonNode NodeInfo { get; set; }
|
||||
public InstanceDelivery Delivery { get; set; }
|
||||
public InstanceLocation Geo { get; set; }
|
||||
}
|
||||
|
||||
//precision: "city" (coordinates to 0.1°, city and network), "country" (located only to a country) or "cdn" (the
|
||||
//CDN's name only)
|
||||
public class InstanceLocation
|
||||
{
|
||||
public string Precision { get; set; }
|
||||
public string Country { get; set; }
|
||||
public string City { get; set; }
|
||||
public double? Latitude { get; set; }
|
||||
public double? Longitude { get; set; }
|
||||
public string Network { get; set; }
|
||||
public string Cdn { get; set; }
|
||||
public string CdnDomain { get; set; }
|
||||
public string CdnSource { get; set; }//"ranges", "headers" or "asn"
|
||||
public InstancePlace BeforeCdn { get; set; }//where a CDN-fronted server was located before the CDN, by its snapshots
|
||||
public string Source { get; set; }
|
||||
public string LocatedAt { get; set; }
|
||||
}
|
||||
|
||||
public class InstancePlace
|
||||
{
|
||||
public string Week { get; set; }
|
||||
public string Country { get; set; }
|
||||
public string City { get; set; }
|
||||
public double? Latitude { get; set; }
|
||||
public double? Longitude { get; set; }
|
||||
}
|
||||
|
||||
public class InstanceWeek
|
||||
{
|
||||
public string Week { get; set; }
|
||||
public string TakenAt { get; set; }
|
||||
public bool Reachable { get; set; }
|
||||
public string Software { get; set; }
|
||||
public string Version { get; set; }
|
||||
public long? UsersTotal { get; set; }
|
||||
public long? UsersActiveMonth { get; set; }
|
||||
public long? LocalPosts { get; set; }
|
||||
public bool? OpenRegistrations { get; set; }
|
||||
public InstanceLocation Geo { get; set; }
|
||||
}
|
||||
|
||||
public class InstanceDelivery
|
||||
|
||||
@@ -35,11 +35,15 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
_search = search;
|
||||
}
|
||||
|
||||
[HttpGet("/api/v2/search"), Scope("read:search", requiresUser: false)]
|
||||
// Anyone may search, as on Mastodon; only resolving (which fetches from other servers) and paging need a sign-in.
|
||||
[HttpGet("/api/v2/search"), Scope("read:search", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public async Task<IActionResult> Search(CancellationToken token)
|
||||
{
|
||||
var q = Params.Get("q")?.Trim();
|
||||
var type = Params.Get("type");
|
||||
var offset = Params.Int("offset") ?? 0;
|
||||
if (MyId == default && (Params.Bool("resolve") == true || offset > 0))
|
||||
return Error(StatusCodes.Status401Unauthorized, "Search queries that resolve uris or use offset require the read:search scope");
|
||||
var resolve = Params.Bool("resolve") == true && MyId != default;
|
||||
var results = new SearchResults();
|
||||
if (string.IsNullOrEmpty(q))
|
||||
@@ -66,7 +70,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
|
||||
if (type is null or "accounts")
|
||||
{
|
||||
results.Accounts = await _search.Find(q, resolve, Limit(), token);
|
||||
results.Accounts = await _search.Find(q, resolve, Limit(), Math.Max(0, offset), token);
|
||||
}
|
||||
if (type is null or "hashtags" && q.TrimStart('#') is { Length: > 0 } tag)
|
||||
results.Hashtags.Add(new { name = tag.ToLowerInvariant(), url = $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag.ToLowerInvariant())}", history = Array.Empty<object>() });
|
||||
@@ -133,5 +137,67 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
|
||||
[HttpGet("/api/v1/push/subscription"), Scope("push")]
|
||||
public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found");
|
||||
|
||||
// Routes Mastodon answers that we have nothing behind: an empty or default answer lets clients degrade, where a
|
||||
// 404 breaks some of them (Mastodon invariant 5)
|
||||
[HttpGet("/api/v1/directory"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult Directory() => Json(Array.Empty<object>());
|
||||
|
||||
[HttpGet("/api/v1/timelines/link"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult LinkTimeline() => Json(Array.Empty<object>());
|
||||
|
||||
[HttpGet("/api/v1/accounts/{id}/identity_proofs"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult IdentityProofs(string id) => Json(Array.Empty<object>());
|
||||
|
||||
[HttpGet("/api/v1/tags/{name}"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult Tag(string name)
|
||||
{
|
||||
var tag = name?.TrimStart('#').ToLowerInvariant() ?? string.Empty;
|
||||
var baseAddress = HttpContext.RequestServices.GetRequiredService<Federation.Actors.ILocalActorService>().BaseAddress;
|
||||
return Json(new { name = tag, url = $"{baseAddress}/tags/{Uri.EscapeDataString(tag)}", history = Array.Empty<object>(), following = false });
|
||||
}
|
||||
|
||||
[HttpGet("/api/v1/instance/languages"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult Languages() => Json(InstanceController.LanguageCodes(HttpContext.RequestServices)
|
||||
.Select(code => new { code, name = System.Globalization.CultureInfo.GetCultureInfo(code).EnglishName }));
|
||||
|
||||
[HttpGet("/api/v1/instance/translation_languages"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult TranslationLanguages() => Json(new { });
|
||||
|
||||
[HttpGet("/api/v1/instance/domain_blocks"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult InstanceDomainBlocks() => Json(Array.Empty<object>());
|
||||
|
||||
[HttpGet("/api/v1/instance/privacy_policy"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public IActionResult PrivacyPolicy() => Json(new
|
||||
{
|
||||
updated_at = "2026-10-04T00:00:00.000Z",
|
||||
content = "<p>One private login owns several public personas, and nobody but this server's admin can tell they belong "
|
||||
+ "together. What a persona posts is shared as its visibility says; a located post never leaves this server. "
|
||||
+ "Uploads lose their metadata. Statistics name servers, never people: see /stargazing.</p>"
|
||||
});
|
||||
|
||||
[HttpGet("/api/v1/notifications/policy"), Scope("read:notifications")]
|
||||
public IActionResult NotificationPolicyV1() => Json(new
|
||||
{
|
||||
filter_not_following = false,
|
||||
filter_not_followers = false,
|
||||
filter_new_accounts = false,
|
||||
filter_private_mentions = false,
|
||||
summary = new { pending_requests_count = 0, pending_notifications_count = 0 }
|
||||
});
|
||||
|
||||
[HttpGet("/api/v2/notifications/policy"), Scope("read:notifications")]
|
||||
public IActionResult NotificationPolicyV2() => Json(new
|
||||
{
|
||||
for_not_following = "accept",
|
||||
for_not_followers = "accept",
|
||||
for_new_accounts = "accept",
|
||||
for_private_mentions = "accept",
|
||||
for_limited_accounts = "accept",
|
||||
summary = new { pending_requests_count = 0, pending_notifications_count = 0 }
|
||||
});
|
||||
|
||||
[HttpGet("/api/v1/notifications/requests"), Scope("read:notifications")]
|
||||
public IActionResult NotificationRequests() => Json(Array.Empty<object>());
|
||||
}
|
||||
}
|
||||
@@ -273,7 +273,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.LocalQuotePolicy, policy).ExecuteAsync(token);
|
||||
post.LocalQuotePolicy = policy;
|
||||
if (!post.IsLocalOnly)
|
||||
await _outbox.Publish(Me, post, ActivityPubRenderer.UpdateOf(post, Me, $"policy-{DateTime.UtcNow.Ticks}"), token);
|
||||
await _outbox.PublishUpdate(Me, post, $"policy-{DateTime.UtcNow.Ticks}", token);
|
||||
return Json(await _mapper.Status(post, MyId, token));
|
||||
}
|
||||
|
||||
|
||||
@@ -177,8 +177,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
_dbEntities = dbEntities;
|
||||
}
|
||||
|
||||
[HttpGet("/api/v1/notifications"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> List(CancellationToken token)
|
||||
Find<NotificationEntity, NotificationEntity> Filtered()
|
||||
{
|
||||
var types = Params.List("types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList();
|
||||
var excluded = Params.List("exclude_types").Select(Parse).Where(t => t.HasValue).Select(t => t.Value).ToList();
|
||||
@@ -189,11 +188,132 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
query.Match(n => !excluded.Contains(n.Type));
|
||||
if (Params.Get("account_id") is { } accountId)
|
||||
query.Match(n => n.FromAccountId == accountId);
|
||||
var notifications = await Page.From(Params, Limit(15, 30)).Fetch(query, n => n.ID, token);
|
||||
return query;
|
||||
}
|
||||
|
||||
[HttpGet("/api/v1/notifications"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> List(CancellationToken token)
|
||||
{
|
||||
var notifications = await Page.From(Params, Limit(15, 30)).Fetch(Filtered(), n => n.ID, token);
|
||||
Link("/api/v1/notifications", notifications.LastOrDefault()?.ID, notifications.FirstOrDefault()?.ID);
|
||||
return Json(await Map(notifications, token));
|
||||
}
|
||||
|
||||
// Grouped notifications (Mastodon 4.3): api_versions.mastodon says we speak that API, so clients call it. Likes and
|
||||
// boosts of one post group together, as do follows within an hour; everything else stands alone.
|
||||
static readonly HashSet<string> DefaultGrouped = new() { "favourite", "reblog", "follow" };
|
||||
|
||||
HashSet<string> Grouped() => Params.List("grouped_types") is { Count: > 0 } chosen ? chosen.ToHashSet() : DefaultGrouped;
|
||||
|
||||
static string GroupKey(NotificationEntity n, ISet<string> grouped) =>
|
||||
!grouped.Contains(Names[n.Type]) ? $"ungrouped-{n.ID}"
|
||||
: n.Type == NotificationType.Follow ? $"grouped-follow-{n.CreatedAt:yyyyMMddHH}"
|
||||
: string.IsNullOrEmpty(n.PostId) ? $"ungrouped-{n.ID}"
|
||||
: $"grouped-{Names[n.Type]}-{n.PostId}";
|
||||
|
||||
// the notifications a group key stands for
|
||||
System.Linq.Expressions.Expression<Func<NotificationEntity, bool>> InGroup(string key)
|
||||
{
|
||||
var me = MyId;
|
||||
if (key.StartsWith("ungrouped-", StringComparison.Ordinal))
|
||||
{
|
||||
var id = key["ungrouped-".Length..];
|
||||
return n => n.AvatarId == me && n.ID == id;
|
||||
}
|
||||
if (key.StartsWith("grouped-follow-", StringComparison.Ordinal)
|
||||
&& DateTime.TryParseExact(key["grouped-follow-".Length..], "yyyyMMddHH", System.Globalization.CultureInfo.InvariantCulture,
|
||||
System.Globalization.DateTimeStyles.AdjustToUniversal | System.Globalization.DateTimeStyles.AssumeUniversal, out var hour))
|
||||
{
|
||||
var end = hour.AddHours(1);
|
||||
return n => n.AvatarId == me && n.Type == NotificationType.Follow && n.CreatedAt >= hour && n.CreatedAt < end;
|
||||
}
|
||||
if (key.Split('-') is ["grouped", var name, var postId] && Parse(name) is { } type)
|
||||
return n => n.AvatarId == me && n.Type == type && n.PostId == postId;
|
||||
return default;
|
||||
}
|
||||
|
||||
async Task<List<NotificationEntity>> Members(string key, CancellationToken token) =>
|
||||
InGroup(key) is { } filter
|
||||
? await _dbEntities.Notifications.Match(filter).Sort(n => n.ID, Order.Descending).Limit(80).ExecuteAsync(token)
|
||||
: new List<NotificationEntity>();
|
||||
|
||||
async Task<object> Groups(List<NotificationEntity> page, ISet<string> grouped, CancellationToken token)
|
||||
{
|
||||
var mapped = (await Map(page, token)).ToDictionary(n => n.Id);
|
||||
var shown = page.Where(n => mapped.ContainsKey(n.ID)).ToList();
|
||||
var groups = new List<object>();
|
||||
foreach (var group in shown.GroupBy(n => GroupKey(n, grouped)))
|
||||
{
|
||||
var newest = mapped[group.First().ID];
|
||||
var total = group.Key.StartsWith("ungrouped-", StringComparison.Ordinal) ? 1 : await DB.Default.CountAsync(InGroup(group.Key), token);
|
||||
groups.Add(new
|
||||
{
|
||||
group_key = group.Key,
|
||||
notifications_count = Math.Max(total, group.Count()),
|
||||
type = newest.Type,
|
||||
most_recent_notification_id = newest.Id,
|
||||
page_min_id = group.Last().ID,
|
||||
page_max_id = group.First().ID,
|
||||
latest_page_notification_at = newest.CreatedAt,
|
||||
sample_account_ids = group.Select(n => mapped[n.ID].Account.Id).Distinct().Take(8).ToList(),
|
||||
status_id = newest.Status?.Id,
|
||||
emoji = newest.Emoji,
|
||||
emoji_url = newest.EmojiUrl
|
||||
});
|
||||
}
|
||||
return new
|
||||
{
|
||||
accounts = mapped.Values.Select(n => n.Account).DistinctBy(a => a.Id).ToList(),
|
||||
statuses = mapped.Values.Where(n => n.Status != default).Select(n => n.Status).DistinctBy(s => s.Id).ToList(),
|
||||
notification_groups = groups
|
||||
};
|
||||
}
|
||||
|
||||
[HttpGet("/api/v2/notifications"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> GroupedList(CancellationToken token)
|
||||
{
|
||||
var notifications = await Page.From(Params, Limit(40, 80)).Fetch(Filtered(), n => n.ID, token);
|
||||
Link("/api/v2/notifications", notifications.LastOrDefault()?.ID, notifications.FirstOrDefault()?.ID);
|
||||
return Json(await Groups(notifications, Grouped(), token));
|
||||
}
|
||||
|
||||
[HttpGet("/api/v2/notifications/unread_count"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> GroupedUnreadCount(CancellationToken token)
|
||||
{
|
||||
var unread = await Filtered().Match(n => !n.IsRead).Sort(n => n.ID, Order.Descending)
|
||||
.Limit(Math.Clamp(Params.Int("limit") ?? 100, 1, 1000)).ExecuteAsync(token);
|
||||
var shown = (await Map(unread, token)).Select(n => n.Id).ToHashSet();
|
||||
var grouped = Grouped();
|
||||
return Json(new { count = unread.Where(n => shown.Contains(n.ID)).Select(n => GroupKey(n, grouped)).Distinct().Count() });
|
||||
}
|
||||
|
||||
[HttpGet("/api/v2/notifications/{groupKey}"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> Group(string groupKey, CancellationToken token)
|
||||
{
|
||||
var members = await Members(groupKey, token);
|
||||
if (members.Count == 0)
|
||||
return NotFoundError();
|
||||
return Json(await Groups(members, new HashSet<string>(DefaultGrouped.Append(Names[members[0].Type])), token));
|
||||
}
|
||||
|
||||
[HttpGet("/api/v2/notifications/{groupKey}/accounts"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> GroupAccounts(string groupKey, CancellationToken token)
|
||||
{
|
||||
var members = await Members(groupKey, token);
|
||||
if (members.Count == 0)
|
||||
return NotFoundError();
|
||||
return Json((await Map(members, token)).Select(n => n.Account).DistinctBy(a => a.Id).ToList());
|
||||
}
|
||||
|
||||
[HttpPost("/api/v2/notifications/{groupKey}/dismiss"), Scope("write:notifications")]
|
||||
public async Task<IActionResult> DismissGroup(string groupKey, CancellationToken token)
|
||||
{
|
||||
if (InGroup(groupKey) is not { } filter)
|
||||
return NotFoundError();
|
||||
await DB.Default.DeleteAsync(filter);
|
||||
return Json(new { });
|
||||
}
|
||||
|
||||
[HttpGet("/api/v1/notifications/{id}"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> Get(string id, CancellationToken token)
|
||||
{
|
||||
@@ -216,9 +336,15 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
return Json(new { });
|
||||
}
|
||||
|
||||
// counts what the list would show (Map drops notifications from gone accounts and about hidden posts), at most
|
||||
// `limit` of them, as Mastodon does
|
||||
[HttpGet("/api/v1/notifications/unread_count"), Scope("read:notifications")]
|
||||
public async Task<IActionResult> UnreadCount(CancellationToken token) =>
|
||||
Json(new { count = await DB.Default.CountAsync<NotificationEntity>(n => n.AvatarId == MyId && !n.IsRead, token) });
|
||||
public async Task<IActionResult> UnreadCount(CancellationToken token)
|
||||
{
|
||||
var unread = await DB.Default.Find<NotificationEntity>().Match(n => n.AvatarId == MyId && !n.IsRead)
|
||||
.Sort(n => n.ID, Order.Descending).Limit(Math.Clamp(Params.Int("limit") ?? 100, 1, 1000)).ExecuteAsync(token);
|
||||
return Json(new { count = (await Map(unread, token)).Count });
|
||||
}
|
||||
|
||||
async Task<List<Entities.Notification>> Map(List<NotificationEntity> notifications, CancellationToken token)
|
||||
{
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
namespace PrivaPub.Api.Mastodon.Entities
|
||||
namespace PrivaPub.Api.Mastodon.Entities
|
||||
{
|
||||
public class Account
|
||||
{
|
||||
@@ -73,6 +73,7 @@ namespace PrivaPub.Api.Mastodon.Entities
|
||||
public PrivaPubVideo Video { get; set; }
|
||||
public PrivaPubAudio Audio { get; set; }
|
||||
public PrivaPubEvent Event { get; set; }
|
||||
public PrivaPubPlace Place { get; set; }//a remote post's own place (Pixelfed), shown only
|
||||
public PrivaPubVotes Votes { get; set; } = new();
|
||||
}
|
||||
|
||||
@@ -151,6 +152,7 @@ namespace PrivaPub.Api.Mastodon.Entities
|
||||
{
|
||||
public string Name { get; set; }
|
||||
public string Address { get; set; }
|
||||
public string Country { get; set; }//a post's own place only
|
||||
public double? Latitude { get; set; }
|
||||
public double? Longitude { get; set; }
|
||||
public string Url { get; set; }
|
||||
|
||||
@@ -19,7 +19,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
_remoteActors = remoteActors;
|
||||
}
|
||||
|
||||
public async Task<List<Account>> Find(string q, bool resolve, int limit, CancellationToken token)
|
||||
public async Task<List<Account>> Find(string q, bool resolve, int limit, CancellationToken token) => await Find(q, resolve, limit, 0, token);
|
||||
|
||||
// gone accounts (remote ones that deleted themselves, personas of banned or deleted roots) are never found
|
||||
public async Task<List<Account>> Find(string q, bool resolve, int limit, int offset, CancellationToken token)
|
||||
{
|
||||
q = q?.Trim().TrimStart('@');
|
||||
if (string.IsNullOrEmpty(q))
|
||||
@@ -31,24 +34,31 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
{
|
||||
var userName = parts[0];
|
||||
var domain = parts[1].ToLowerInvariant();
|
||||
var known = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain).ExecuteFirstAsync(token);
|
||||
var known = await _dbEntities.ForeignAvatars.Match(f => f.UserName == userName && f.Domain == domain && !f.DeletionAt.HasValue).ExecuteFirstAsync(token);
|
||||
if (known == default && resolve)
|
||||
{
|
||||
var actorUri = await _remoteActors.ResolveHandle(q, token);
|
||||
known = actorUri == default ? default : await _remoteActors.GetActor(actorUri, refresh: false, token);
|
||||
}
|
||||
if (known != default)
|
||||
if (known is { DeletionAt: null } && offset == 0)
|
||||
results.Add(_mapper.Foreign(known));
|
||||
return results;
|
||||
}
|
||||
|
||||
var prefix = parts[0].ToLowerInvariant();
|
||||
foreach (var avatar in await _dbEntities.Avatars.Match(a => a.UserName.StartsWith(prefix) && !a.DeletionAt.HasValue && a.Settings.IsDiscoverable != false)
|
||||
.Limit(limit).ExecuteAsync(token))
|
||||
var barred = await Domain.Privacy.Counted.BarredPersonas(token);
|
||||
var locals = await _dbEntities.Avatars
|
||||
.Match(a => a.UserName.StartsWith(prefix) && !a.DeletionAt.HasValue && a.Settings.IsDiscoverable != false && !barred.Contains(a.ID))
|
||||
.Sort(a => a.UserName, MongoDB.Entities.Order.Ascending)
|
||||
.Limit(offset + limit).ExecuteAsync(token);
|
||||
foreach (var avatar in locals.Skip(offset))
|
||||
results.Add(await _mapper.Local(_localActors.FromAvatar(avatar), false, token));
|
||||
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => f.UserName.StartsWith(parts[0]) && f.IsDiscoverable)
|
||||
.Limit(Math.Max(0, limit - results.Count)).ExecuteAsync(token))
|
||||
results.Add(_mapper.Foreign(foreign));
|
||||
var skipRemote = Math.Max(0, offset - locals.Count);
|
||||
if (results.Count < limit)
|
||||
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => f.UserName.StartsWith(parts[0]) && f.IsDiscoverable && !f.DeletionAt.HasValue)
|
||||
.Sort(f => f.UserName, MongoDB.Entities.Order.Ascending)
|
||||
.Skip(skipRemote).Limit(limit - results.Count).ExecuteAsync(token))
|
||||
results.Add(_mapper.Foreign(foreign));
|
||||
return results;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Entities;
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Api.Mastodon.Entities;
|
||||
using PrivaPub.Api.Mastodon.Infrastructure;
|
||||
@@ -50,15 +50,11 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
? await DB.Default.CountAsync<Following>(f => f.AvatarId == actor.Id && f.State == FollowState.Accepted, token)
|
||||
: 0;
|
||||
var latest = await _dbEntities.Posts
|
||||
.Match(p => (actor.Kind == LocalActorKind.Group ? p.GroupId == actor.Id : p.GroupUserId == actor.Id) && !p.IsFederatedCopy
|
||||
&& !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct && p.ReblogOfPostId == null)
|
||||
.Match(Counted.By(actor))
|
||||
.Sort(p => p.ID, Order.Descending)
|
||||
.Limit(1)
|
||||
.ExecuteFirstAsync(token);
|
||||
var statuses = latest == default
|
||||
? 0
|
||||
: await DB.Default.CountAsync<PostEntity>(p => (actor.Kind == LocalActorKind.Group ? p.GroupId == actor.Id : p.GroupUserId == actor.Id)
|
||||
&& !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct && p.ReblogOfPostId == null, token);
|
||||
var statuses = latest == default ? 0 : await Counted.PostsBy(actor, token);
|
||||
|
||||
var account = new Account
|
||||
{
|
||||
@@ -71,7 +67,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
Discoverable = actor.Discoverable,
|
||||
Indexable = actor.Indexable,
|
||||
Noindex = !actor.Indexable,
|
||||
HideCollections = actor.Settings.HideCollections,
|
||||
HideCollections = true,//follower and following lists are never shown to others, only their counts
|
||||
Group = actor.Kind == LocalActorKind.Group,
|
||||
CreatedAt = MastodonJson.Day(actor.Published),
|
||||
Note = ActivityPubRenderer.Html(actor.Summary),
|
||||
@@ -124,7 +120,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
Avatar = Proxied(foreign.PictureURL) ?? MissingAvatar,
|
||||
AvatarStatic = Proxied(foreign.PictureURL) ?? MissingAvatar,
|
||||
Header = Proxied(foreign.ThumbnailURL) ?? MissingHeader,
|
||||
HeaderStatic = Proxied(foreign.ThumbnailURL) ?? MissingHeader
|
||||
HeaderStatic = Proxied(foreign.ThumbnailURL) ?? MissingHeader,
|
||||
FollowersCount = foreign.FollowersCount ?? 0,//as its server publishes them (AccountCountsJob); 0 while unknown
|
||||
FollowingCount = foreign.FollowingCount ?? 0,
|
||||
StatusesCount = foreign.StatusesCount ?? 0
|
||||
};
|
||||
|
||||
public async Task<Account> Account(string id, CancellationToken token) =>
|
||||
@@ -141,7 +140,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
accounts[avatar.ID] = await Local(_localActors.FromAvatar(avatar), false, token);
|
||||
foreach (var group in await _dbEntities.Groups.Match(g => wanted.Contains(g.ID) && !g.DeletionAt.HasValue && g.Kind == GroupKind.Community).ExecuteAsync(token))
|
||||
accounts[group.ID] = await Local(_localActors.FromGroup(group), false, token);
|
||||
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => wanted.Contains(f.ID)).ExecuteAsync(token))
|
||||
foreach (var foreign in await _dbEntities.ForeignAvatars.Match(f => wanted.Contains(f.ID) && !f.DeletionAt.HasValue).ExecuteAsync(token))
|
||||
accounts[foreign.ID] = Foreign(foreign);
|
||||
return accounts;
|
||||
}
|
||||
@@ -202,8 +201,8 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
|
||||
Status Map(PostEntity post)
|
||||
{
|
||||
if (!accounts.TryGetValue(AuthorOf(post), out var account))
|
||||
return default;
|
||||
if (AuthorOf(post) is not { } author || !accounts.TryGetValue(author, out var account))
|
||||
return default;//a post whose author we cannot show is left out, never a 500
|
||||
var status = new Status
|
||||
{
|
||||
Id = post.ID,
|
||||
@@ -283,6 +282,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
continue;
|
||||
Quote(inner, original);
|
||||
status.Reblog = inner;
|
||||
status.Url = inner.Url;//a boost has no page of its own; /grunts would answer JSON to a browser
|
||||
status.Content = string.Empty;
|
||||
status.Reblogged = reblogged.Contains(original.ID);
|
||||
}
|
||||
@@ -457,6 +457,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
Status = post.Event.Status,
|
||||
Category = post.Event.Category
|
||||
},
|
||||
Place = post.Place == default ? default : new PrivaPubPlace
|
||||
{
|
||||
Name = post.Place.Name, Latitude = post.Place.Latitude, Longitude = post.Place.Longitude, Country = post.Place.Country
|
||||
},
|
||||
Votes = new PrivaPubVotes { Up = post.FavouritesCount, Down = post.DownvotesCount }
|
||||
};
|
||||
|
||||
|
||||
@@ -361,7 +361,7 @@ namespace PrivaPub.Controllers.ClientToServer
|
||||
if (!result.IsValid)
|
||||
return StatusCode(result.StatusCode, result);
|
||||
|
||||
return Ok(result);
|
||||
return Ok(result.Data);//the same sentence for everyone
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
@@ -414,26 +414,23 @@ namespace PrivaPub.Controllers.ClientToServer
|
||||
}
|
||||
}
|
||||
|
||||
//[HttpDelete, Route("delete"), Authorize(Policy = Policies.IsUser)]
|
||||
//public async Task<IActionResult> RemoveSelf()
|
||||
//{
|
||||
// var result = new WebResult();
|
||||
// try
|
||||
// {
|
||||
// //result = await UsersService.RemoveUserAsync(User.GetUserId());
|
||||
// if (!result.IsValid)
|
||||
// return StatusCode(result.StatusCode, result);
|
||||
|
||||
// await HttpContext.SignOutAsync();
|
||||
|
||||
// return Ok();
|
||||
// }
|
||||
// catch (Exception ex)
|
||||
// {
|
||||
// Logger.LogError(ex, $"{nameof(User)}.{nameof(RemoveSelf)}()");
|
||||
// return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
|
||||
// }
|
||||
//}
|
||||
[HttpPost, Route("/clientapi/user/delete"), EnableRateLimiting(RateLimiting.Accounts), Authorize(Policy = Policies.IsUser)]
|
||||
public async Task<IActionResult> RemoveSelf(RemoveSelfForm form)
|
||||
{
|
||||
var result = new WebResult();
|
||||
if (!ModelState.IsValid)
|
||||
return BadRequest(result.Invalidate(Localizer["Invalid model."]));
|
||||
try
|
||||
{
|
||||
result = await UsersService.RemoveSelfAsync(User.GetUserId(), form.Password);
|
||||
return result.IsValid ? Ok() : StatusCode(result.StatusCode, result);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogError(ex, $"{nameof(User)}.{nameof(RemoveSelf)}()");
|
||||
return BadRequest(result.Invalidate(Localizer["Something went wrong."]));
|
||||
}
|
||||
}
|
||||
|
||||
#endregion User endpoints
|
||||
|
||||
|
||||
@@ -42,6 +42,9 @@ namespace PrivaPub.Domain.Media
|
||||
["audio/mp4"] = "m4a", ["audio/x-m4a"] = "m4a", ["audio/flac"] = "flac", ["audio/webm"] = "webm"
|
||||
};
|
||||
|
||||
// what an upload may be, as the instance API advertises it
|
||||
public static IReadOnlyList<string> SupportedTypes => ImageTypes.Concat(AvTypes.Keys).ToList();
|
||||
|
||||
readonly IOptionsMonitor<MediaOptions> _options;
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly IWebHostEnvironment _environment;
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.User;
|
||||
|
||||
using System.Linq.Expressions;
|
||||
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Domain.Privacy
|
||||
{
|
||||
// What the numbers count, in one place, so a persona's statuses_count, its outbox's totalItems, NodeInfo and the
|
||||
// instance API agree (they used to count four different things).
|
||||
public static class Counted
|
||||
{
|
||||
// Mastodon's statuses_count: every post that is neither deleted nor a direct message, reblogs included. Circle and
|
||||
// located posts count too (owner decision 2026-10-04: "a good balance for the fediverse to work").
|
||||
public static readonly Expression<Func<PostEntity, bool>> Post = p => !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct;
|
||||
|
||||
// a persona's own posts, or every post in a group, its remote members' included
|
||||
public static Expression<Func<PostEntity, bool>> By(LocalActor actor)
|
||||
{
|
||||
var id = actor.Id;
|
||||
return actor.Kind == LocalActorKind.Group
|
||||
? p => p.GroupId == id && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct
|
||||
: p => p.GroupUserId == id && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility != PostVisibility.Direct;
|
||||
}
|
||||
|
||||
// replies_count counts only public and unlisted replies, as Mastodon does: a count must not tell anyone that a
|
||||
// followers-only, direct or circle reply exists
|
||||
public static bool Reply(PostEntity reply) => reply.Visibility is PostVisibility.Public or PostVisibility.Unlisted;
|
||||
|
||||
public static async Task<long> PostsBy(LocalActor actor, CancellationToken token) =>
|
||||
await DB.Default.CountAsync(By(actor), token);
|
||||
|
||||
// Personas that do not count as this server's users, and are not found in search: those of a banned or deleted root.
|
||||
public static async Task<List<string>> BarredPersonas(CancellationToken token)
|
||||
{
|
||||
var roots = (await DB.Default.Find<RootUser>().Match(u => u.IsBanned || u.DeletedAt != null).Project(u => u.Include(x => x.ID)).ExecuteAsync(token))
|
||||
.Select(u => u.ID).ToList();
|
||||
return roots.Count == 0
|
||||
? new List<string>()
|
||||
: (await DB.Default.Find<RootToAvatar>().Match(ra => roots.Contains(ra.RootId)).ExecuteAsync(token)).Select(ra => ra.AvatarId).ToList();
|
||||
}
|
||||
|
||||
public static async Task<long> Users(CancellationToken token)
|
||||
{
|
||||
var barred = await BarredPersonas(token);
|
||||
return await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue && !barred.Contains(a.ID), token);
|
||||
}
|
||||
|
||||
// users who posted within the last days, as NodeInfo's activeMonth and activeHalfyear and the instance API mean it
|
||||
public static async Task<long> ActiveUsers(int days, CancellationToken token)
|
||||
{
|
||||
var barred = await BarredPersonas(token);
|
||||
var since = DateTime.UtcNow.AddDays(-days);
|
||||
var authors = await DB.Default.Find<PostEntity, string>()
|
||||
.Match(p => !p.IsFederatedCopy && p.CreationDate >= since && !barred.Contains(p.GroupUserId))
|
||||
.Project(p => p.GroupUserId)
|
||||
.ExecuteAsync(token);
|
||||
var distinct = authors.Where(a => !string.IsNullOrEmpty(a)).Distinct().ToList();
|
||||
return distinct.Count == 0 ? 0 : await DB.Default.CountAsync<Avatar>(a => distinct.Contains(a.ID) && !a.DeletionAt.HasValue, token);
|
||||
}
|
||||
|
||||
public static async Task<long> LocalPosts(CancellationToken token)
|
||||
{
|
||||
var barred = await BarredPersonas(token);
|
||||
return await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy && !barred.Contains(p.GroupUserId)) & f.Where(Post), token);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,18 +6,17 @@ namespace PrivaPub.Domain.Statistics
|
||||
|
||||
public static class PublicGeo
|
||||
{
|
||||
//owner decision: a CDN-fronted server shows only its CDN (the address is an edge), a server reporting at least
|
||||
//minUsers users its city, coordinates and network, any other only its country
|
||||
public static PublicLocation Project(RemoteInstance instance, int minUsers)
|
||||
//a server's place as it is shown publicly: where it was located (country, city, coordinates to 0.1°, network),
|
||||
//except that a CDN-fronted server shows only its CDN, because the address we reached is the CDN's edge
|
||||
public static PublicLocation Project(InstanceGeo geo)
|
||||
{
|
||||
var geo = instance?.Geo;
|
||||
if (geo == default)
|
||||
return default;
|
||||
if (geo.Cdn != default)
|
||||
return new PublicLocation(default, default, default, default, default, geo.Cdn);
|
||||
if (instance.UsersTotal >= minUsers)
|
||||
return new PublicLocation(geo.Country, geo.City, geo.Latitude, geo.Longitude, geo.AsnOrg, default);
|
||||
return geo.Country == default ? default : new PublicLocation(geo.Country, default, default, default, default, default);
|
||||
if (geo.Country == default && geo.Latitude == default)
|
||||
return default;
|
||||
return new PublicLocation(geo.Country, geo.City, geo.Latitude, geo.Longitude, geo.AsnOrg, default);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Infrastructure.Statistics;
|
||||
using PrivaPub.Models;
|
||||
using PrivaPub.Models.Jobs;
|
||||
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
|
||||
namespace PrivaPub.Domain.Statistics
|
||||
{
|
||||
public interface ISelfLocation
|
||||
{
|
||||
//where this server is, or default when it cannot tell; Source is "configured" when the owner set it (Statistics:Geo:Self)
|
||||
Task<InstanceGeo> Get(CancellationToken token);
|
||||
}
|
||||
|
||||
// This server's own place, for the instance API (the client's globe draws every server from it). The owner may set it,
|
||||
// for a server behind a proxy or a CDN; otherwise the server's own host name is resolved and located like any other
|
||||
// server's, once a day. Never the address of a request.
|
||||
public class SelfLocation : ISelfLocation
|
||||
{
|
||||
static readonly TimeSpan Lifetime = TimeSpan.FromDays(1);
|
||||
|
||||
readonly IGeoLocator _geo;
|
||||
readonly ICdnRanges _cdnRanges;
|
||||
readonly IOptionsMonitor<StatisticsOptions> _options;
|
||||
readonly IOptionsMonitor<AppConfiguration> _app;
|
||||
readonly ILogger<SelfLocation> _logger;
|
||||
readonly SemaphoreSlim _gate = new(1, 1);
|
||||
|
||||
InstanceGeo _located;
|
||||
DateTime _locatedAt = DateTime.MinValue;
|
||||
|
||||
public SelfLocation(IGeoLocator geo, IOptionsMonitor<StatisticsOptions> options, IOptionsMonitor<AppConfiguration> app, ILogger<SelfLocation> logger,
|
||||
ICdnRanges cdnRanges = default)
|
||||
{
|
||||
_geo = geo;
|
||||
_cdnRanges = cdnRanges;
|
||||
_options = options;
|
||||
_app = app;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public async Task<InstanceGeo> Get(CancellationToken token)
|
||||
{
|
||||
if (_options.CurrentValue.Geo.Self is { Latitude: { } latitude, Longitude: { } longitude } self)
|
||||
return new InstanceGeo
|
||||
{
|
||||
Country = self.Country,
|
||||
City = self.City,
|
||||
Latitude = Math.Round(latitude, 1),
|
||||
Longitude = Math.Round(longitude, 1),
|
||||
Source = "configured"
|
||||
};
|
||||
|
||||
if (DateTime.UtcNow - _locatedAt < Lifetime)
|
||||
return _located;
|
||||
await _gate.WaitAsync(token);
|
||||
try
|
||||
{
|
||||
if (DateTime.UtcNow - _locatedAt >= Lifetime)
|
||||
{
|
||||
_located = await Locate(token);
|
||||
_locatedAt = DateTime.UtcNow;
|
||||
}
|
||||
return _located;
|
||||
}
|
||||
finally
|
||||
{
|
||||
_gate.Release();
|
||||
}
|
||||
}
|
||||
|
||||
async Task<InstanceGeo> Locate(CancellationToken token)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (!Uri.TryCreate(_app.CurrentValue.BackendBaseAddress, UriKind.Absolute, out var self))
|
||||
return default;
|
||||
var addresses = await Dns.GetHostAddressesAsync(self.Host, token);
|
||||
var address = addresses.FirstOrDefault(a => a.AddressFamily == AddressFamily.InterNetwork) ?? addresses.FirstOrDefault();
|
||||
var fix = address == default ? default : _geo.Locate(address);
|
||||
var (cdn, cdnSource) = Federation.Objects.InstanceDescriber.Cdn(address, self.Host, fix?.Asn, _cdnRanges, default);
|
||||
return fix == default ? default : new InstanceGeo
|
||||
{
|
||||
Country = fix.Country,
|
||||
City = fix.City,
|
||||
Latitude = fix.Latitude,
|
||||
Longitude = fix.Longitude,
|
||||
Asn = fix.Asn,
|
||||
AsnOrg = fix.AsnOrg,
|
||||
Cdn = cdn?.Name,
|
||||
CdnDomain = cdn?.Domain,
|
||||
CdnSource = cdnSource,
|
||||
Ipv6 = address.AddressFamily == AddressFamily.InterNetworkV6,
|
||||
Source = _geo.Source,
|
||||
LocatedAt = DateTime.UtcNow
|
||||
};
|
||||
}
|
||||
catch (Exception ex) when (ex is SocketException or ArgumentException)
|
||||
{
|
||||
_logger.LogInformation("{Service}: this server's host does not resolve", nameof(SelfLocation));
|
||||
return default;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Models.Jobs;
|
||||
|
||||
namespace PrivaPub.Domain.Statistics
|
||||
{
|
||||
// Where a server was before it went behind a CDN, as its weekly snapshots last saw it.
|
||||
public sealed record PlaceBeforeCdn(string Week, string Country, string City, double? Latitude, double? Longitude);
|
||||
|
||||
// A CDN as a group of servers: those behind it now, and week by week how many were behind it, who joined and who left.
|
||||
public sealed record CdnGroup(string Key, string Name, string Domain, int Servers, int Ranges, DateTime? RangesUpdatedAt, List<string> DetectedBy);
|
||||
|
||||
public sealed record CdnWeek(string Week, int Servers, List<string> Joined, List<string> Left);
|
||||
|
||||
public sealed record CdnHistory(CdnGroup Group, List<string> Hosts, List<CdnWeek> Weeks);
|
||||
|
||||
public interface IServerPlaces
|
||||
{
|
||||
Task<Dictionary<string, PlaceBeforeCdn>> BeforeCdn(IEnumerable<string> hosts, CancellationToken token);
|
||||
Task<List<RemoteInstanceSnapshot>> History(string host, int weeks, CancellationToken token);
|
||||
Task<List<CdnGroup>> Cdns(CancellationToken token);
|
||||
Task<CdnHistory> Cdn(string key, int weeks, CancellationToken token);
|
||||
}
|
||||
|
||||
// Servers through time (owner decision 2026-10-04): each server's weekly snapshots (place, CDN, size), and the servers
|
||||
// grouped by the CDN in front of them, named by the CDN's own domain. Read from the weekly snapshots InstanceDescriber
|
||||
// writes; the CDN pages are kept for half an hour, because a large CDN reads many snapshots.
|
||||
public class ServerPlaces : IServerPlaces
|
||||
{
|
||||
public const int MaxHosts = 500;
|
||||
const int MaxChanges = 50;
|
||||
static readonly TimeSpan CacheFor = TimeSpan.FromMinutes(30);
|
||||
|
||||
readonly ICdnRanges _ranges;
|
||||
readonly IMemoryCache _cache;
|
||||
|
||||
public ServerPlaces(ICdnRanges ranges, IMemoryCache cache)
|
||||
{
|
||||
_ranges = ranges;
|
||||
_cache = cache;
|
||||
}
|
||||
|
||||
// the CDN key a server's place belongs to: the CDN's domain, or its name for a CDN that has none
|
||||
public static string KeyOf(InstanceGeo geo) =>
|
||||
geo?.Cdn == default ? default : geo.CdnDomain ?? CdnCatalog.ByKey(geo.Cdn)?.Key ?? geo.Cdn.ToLowerInvariant();
|
||||
|
||||
static string KeyOf(RemoteInstanceSnapshot snapshot) =>
|
||||
snapshot.Cdn == default ? default : snapshot.CdnDomain ?? CdnCatalog.ByKey(snapshot.Cdn)?.Key ?? snapshot.Cdn.ToLowerInvariant();
|
||||
|
||||
public async Task<Dictionary<string, PlaceBeforeCdn>> BeforeCdn(IEnumerable<string> hosts, CancellationToken token)
|
||||
{
|
||||
var wanted = hosts.Where(h => h != default).Distinct().ToList();
|
||||
if (wanted.Count == 0)
|
||||
return new();
|
||||
var located = await DB.Default.Find<RemoteInstanceSnapshot>()
|
||||
.Match(s => wanted.Contains(s.Host) && s.Cdn == null && (s.Country != null || s.Latitude != null))
|
||||
.Sort(s => s.Week, Order.Descending)
|
||||
.ExecuteAsync(token);
|
||||
return located.GroupBy(s => s.Host)
|
||||
.Select(g => g.First())
|
||||
.ToDictionary(s => s.Host, s => new PlaceBeforeCdn(s.Week, s.Country, s.City, s.Latitude, s.Longitude));
|
||||
}
|
||||
|
||||
public async Task<List<RemoteInstanceSnapshot>> History(string host, int weeks, CancellationToken token) =>
|
||||
await DB.Default.Find<RemoteInstanceSnapshot>()
|
||||
.Match(s => s.Host == host)
|
||||
.Sort(s => s.Week, Order.Descending)
|
||||
.Limit(weeks)
|
||||
.ExecuteAsync(token);
|
||||
|
||||
public async Task<List<CdnGroup>> Cdns(CancellationToken token)
|
||||
{
|
||||
var fronted = await Fronted(token);
|
||||
var status = _ranges.Status;
|
||||
var keys = CdnCatalog.All.Select(p => p.Key).Concat(fronted.Select(i => KeyOf(i.Geo))).Distinct(StringComparer.OrdinalIgnoreCase);
|
||||
return keys.Select(key => Group(key, fronted, status)).OrderByDescending(g => g.Servers).ThenBy(g => g.Name).ToList();
|
||||
}
|
||||
|
||||
public async Task<CdnHistory> Cdn(string key, int weeks, CancellationToken token)
|
||||
{
|
||||
key = key?.Trim().ToLowerInvariant();
|
||||
if (string.IsNullOrEmpty(key) || key.Length > 100)
|
||||
return default;
|
||||
return await _cache.GetOrCreateAsync($"cdn-history|{key}|{weeks}", async entry =>
|
||||
{
|
||||
entry.AbsoluteExpirationRelativeToNow = CacheFor;
|
||||
var fronted = await Fronted(token);
|
||||
var provider = CdnCatalog.ByKey(key);
|
||||
if (provider == default && fronted.All(i => KeyOf(i.Geo) != key))
|
||||
return default;
|
||||
var group = Group(provider?.Key ?? key, fronted, _ranges.Status);
|
||||
var hosts = fronted.Where(i => KeyOf(i.Geo) == group.Key).Select(i => i.Host).OrderBy(h => h).Take(MaxHosts).ToList();
|
||||
return new CdnHistory(group, hosts, await Weeks(group, weeks, token));
|
||||
});
|
||||
}
|
||||
|
||||
async Task<List<CdnWeek>> Weeks(CdnGroup group, int weeks, CancellationToken token)
|
||||
{
|
||||
var since = InstanceDescriber.Week(DateTime.UtcNow.AddDays(-7 * weeks));
|
||||
var everBehind = (await DB.Default.Find<RemoteInstanceSnapshot>()
|
||||
.Match(f => f.Gte(s => s.Week, since) & (f.Eq(s => s.CdnDomain, group.Key) | f.Eq(s => s.Cdn, group.Name)))
|
||||
.Project(s => new RemoteInstanceSnapshot { Host = s.Host })
|
||||
.ExecuteAsync(token))
|
||||
.Select(s => s.Host).Distinct().ToList();
|
||||
if (everBehind.Count == 0)
|
||||
return new();
|
||||
|
||||
var snapshots = await DB.Default.Find<RemoteInstanceSnapshot>()
|
||||
.Match(f => f.In(s => s.Host, everBehind) & f.Gte(s => s.Week, since))
|
||||
.Project(s => new RemoteInstanceSnapshot { Host = s.Host, Week = s.Week, Cdn = s.Cdn, CdnDomain = s.CdnDomain })
|
||||
.ExecuteAsync(token);
|
||||
|
||||
// a server is behind the CDN from the week a snapshot shows it there until one shows it elsewhere
|
||||
var behind = new Dictionary<string, bool>();
|
||||
var history = new List<CdnWeek>();
|
||||
foreach (var week in snapshots.GroupBy(s => s.Week).OrderBy(w => w.Key, StringComparer.Ordinal))
|
||||
{
|
||||
var joined = new List<string>();
|
||||
var left = new List<string>();
|
||||
foreach (var snapshot in week)
|
||||
{
|
||||
var now = KeyOf(snapshot) == group.Key;
|
||||
var before = behind.GetValueOrDefault(snapshot.Host);
|
||||
if (now && !before)
|
||||
joined.Add(snapshot.Host);
|
||||
else if (!now && before)
|
||||
left.Add(snapshot.Host);
|
||||
behind[snapshot.Host] = now;
|
||||
}
|
||||
history.Add(new CdnWeek(week.Key, behind.Count(b => b.Value), joined.OrderBy(h => h).Take(MaxChanges).ToList(),
|
||||
left.OrderBy(h => h).Take(MaxChanges).ToList()));
|
||||
}
|
||||
history.Reverse();
|
||||
return history;
|
||||
}
|
||||
|
||||
static async Task<List<RemoteInstance>> Fronted(CancellationToken token) =>
|
||||
await DB.Default.Find<RemoteInstance>()
|
||||
.Match(i => i.Geo.Cdn != null)
|
||||
.Project(i => new RemoteInstance { Host = i.Host, Geo = i.Geo })
|
||||
.ExecuteAsync(token);
|
||||
|
||||
static CdnGroup Group(string key, List<RemoteInstance> fronted, IReadOnlyDictionary<string, (int Ranges, DateTime FetchedAt)> status)
|
||||
{
|
||||
var provider = CdnCatalog.ByKey(key);
|
||||
var servers = fronted.Where(i => KeyOf(i.Geo) == key).ToList();
|
||||
var detectedBy = new List<string>();
|
||||
if (provider?.Lists.Length > 0)
|
||||
detectedBy.Add("ranges");
|
||||
if (provider != default)
|
||||
detectedBy.Add("headers");
|
||||
if (provider?.Asns.Length > 0)
|
||||
detectedBy.Add("asn");
|
||||
var ranges = status.TryGetValue(key, out var known) ? known : default;
|
||||
return new CdnGroup(key, provider?.Name ?? servers.Select(i => i.Geo.Cdn).FirstOrDefault() ?? key, provider?.Domain, servers.Count, ranges.Ranges,
|
||||
ranges.Ranges > 0 ? ranges.FetchedAt : null, detectedBy);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -233,7 +233,7 @@ namespace PrivaPub.Domain.Statuses
|
||||
return JobOutcome.Done;
|
||||
await Closing(post, author, token);
|
||||
if (!post.IsLocalOnly)
|
||||
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, $"poll-{DateTime.UtcNow.Ticks}"), token);
|
||||
await _outbox.PublishUpdate(author, post, $"poll-{DateTime.UtcNow.Ticks}", token);
|
||||
return JobOutcome.Done;
|
||||
}
|
||||
|
||||
|
||||
@@ -239,7 +239,7 @@ namespace PrivaPub.Domain.Statuses
|
||||
.ExecuteAsync(token);
|
||||
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
|
||||
if (!post.IsLocalOnly)
|
||||
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, "quote-approved"), token);
|
||||
await _outbox.PublishUpdate(author, post, "quote-approved", token);
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@@ -95,8 +95,17 @@ namespace PrivaPub.Domain.Statuses
|
||||
_fanout = fanout;
|
||||
}
|
||||
|
||||
public const int MaxCharacters = 5000;//what the instance API advertises as max_characters, and enforced here
|
||||
|
||||
static StatusOutcome TooLong(StatusDraft draft) =>
|
||||
(draft.Text?.Length ?? 0) > MaxCharacters
|
||||
? StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, $"Validation failed: Text character limit of {MaxCharacters} exceeded")
|
||||
: default;
|
||||
|
||||
public async Task<StatusOutcome> Publish(LocalActor author, StatusDraft draft, CancellationToken token)
|
||||
{
|
||||
if (TooLong(draft) is { } tooLong)
|
||||
return tooLong;
|
||||
var media = await Media(author, draft.MediaIds, default, token);
|
||||
if (media == default)
|
||||
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid");
|
||||
@@ -119,6 +128,11 @@ namespace PrivaPub.Domain.Statuses
|
||||
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
|
||||
if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token))
|
||||
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
|
||||
// a reply in a circle stays in the circle, whichever client wrote it: Mastodon clients know nothing of groups
|
||||
if (group == default && parent is { Visibility: PostVisibility.Circle } && !string.IsNullOrEmpty(parent.GroupId)
|
||||
&& await _dbEntities.Groups.MatchID(parent.GroupId).ExecuteFirstAsync(token) is { DeletionAt: null } parentCircle
|
||||
&& await MayPost(parentCircle, author, token))
|
||||
group = _localActors.FromGroup(parentCircle);
|
||||
|
||||
PostEntity quoted = default;
|
||||
var quotePermission = QuotePermission.Denied;
|
||||
@@ -161,6 +175,9 @@ namespace PrivaPub.Domain.Statuses
|
||||
var visibility = located ? PostVisibility.LocalGeo
|
||||
: group is { IsCircle: true } ? PostVisibility.Circle
|
||||
: draft.Visibility is PostVisibility.Circle or PostVisibility.LocalGeo ? PostVisibility.Public : draft.Visibility;
|
||||
// asking a quoted post's author for permission would show them the circle post
|
||||
if (visibility == PostVisibility.Circle && quoted != default && quotePermission != QuotePermission.Granted)
|
||||
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: A circle post can only quote a post that needs no permission");
|
||||
var post = new PostEntity
|
||||
{
|
||||
GroupUserId = author.Id,
|
||||
@@ -231,7 +248,7 @@ namespace PrivaPub.Domain.Statuses
|
||||
|
||||
await DB.Default.SaveAsync(post, token);
|
||||
await Attach(media, post.ID, token);
|
||||
if (parent != default)
|
||||
if (parent != default && Counted.Reply(post))
|
||||
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
|
||||
await _fanout.Distribute(post, token);
|
||||
await _polls.Scheduled(post, token);
|
||||
@@ -252,6 +269,8 @@ namespace PrivaPub.Domain.Statuses
|
||||
var post = await Own(author, postId, token);
|
||||
if (post == default)
|
||||
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
|
||||
if (TooLong(draft) is { } tooLong)
|
||||
return tooLong;
|
||||
var media = draft.MediaIds == default ? default : await Media(author, draft.MediaIds, post.ID, token);
|
||||
if (draft.MediaIds != default && media == default)
|
||||
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid");
|
||||
@@ -334,13 +353,17 @@ namespace PrivaPub.Domain.Statuses
|
||||
.Modify(p => p.Revisions, new List<PostRevision>())
|
||||
.ExecuteAsync(token);
|
||||
await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID);
|
||||
if (!string.IsNullOrEmpty(post.AnsweringToPostId))
|
||||
// boosts of it end with it, as on Mastodon, so no count keeps them
|
||||
await DB.Default.Update<PostEntity>().Match(p => p.ReblogOfPostId == post.ID && !p.DeletedAt.HasValue)
|
||||
.Modify(p => p.DeletedAt, DateTime.UtcNow).ExecuteAsync(token);
|
||||
if (!string.IsNullOrEmpty(post.AnsweringToPostId) && Counted.Reply(post))
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token);
|
||||
await DB.Default.DeleteAsync<Models.Social.Pin>(p => p.PostId == post.ID);//a deleted post no longer takes a pin
|
||||
if (audience.Count > 0)
|
||||
{
|
||||
var delete = ActivityPubRenderer.Delete(author, post.ObjectURI, $"delete-{post.ID}",
|
||||
new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()), new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()));
|
||||
await _delivery.Enqueue(author, audience, delete, token);
|
||||
await _outbox.Publish(author, post, delete, token);//the post in hand still has its group and mentions
|
||||
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
|
||||
if (group is { IsCircle: false })
|
||||
await _groups.Announce(group, delete, post.ObjectURI, isNewPost: false, token);
|
||||
|
||||
@@ -1,10 +1,7 @@
|
||||
using Microsoft.AspNetCore.Authentication;
|
||||
using Microsoft.IdentityModel.Tokens;
|
||||
|
||||
using PrivaPub.Services;
|
||||
|
||||
using System.Text;
|
||||
|
||||
namespace PrivaPub.Extensions
|
||||
{
|
||||
public static class AddAuthExtension
|
||||
@@ -15,16 +12,7 @@ namespace PrivaPub.Extensions
|
||||
#if DEBUG
|
||||
options.RequireHttpsMetadata = false;
|
||||
#endif
|
||||
options.TokenValidationParameters = new()
|
||||
{
|
||||
ValidateIssuer = true,
|
||||
ValidateAudience = true,
|
||||
ValidateLifetime = true,
|
||||
ValidateIssuerSigningKey = true,
|
||||
ValidIssuer = configuration["AppConfiguration:Jwt:Issuer"],
|
||||
ValidAudience = configuration["AppConfiguration:Jwt:Audience"],
|
||||
IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(configuration["AppConfiguration:Jwt:Key"]))
|
||||
};
|
||||
options.TokenValidationParameters = RootJwt.Parameters(configuration);
|
||||
options.Events = new JwtEvents();
|
||||
});
|
||||
return builder;
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
using PrivaPub.Infrastructure.Jobs;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
namespace PrivaPub.Federation.Actors
|
||||
{
|
||||
// A remote account's follower, following and post counts, as its own server publishes them (the totalItems of its
|
||||
// followers, following and outbox collections). Read when the account is fetched, at most once a day, never when
|
||||
// someone looks at it; a hidden or missing collection leaves its count unknown. Mastodon and GoToSocial do the same.
|
||||
public class AccountCountsJob : IJobHandler
|
||||
{
|
||||
readonly IRemoteActorService _remoteActors;
|
||||
readonly DbEntities _dbEntities;
|
||||
|
||||
public AccountCountsJob(IRemoteActorService remoteActors, DbEntities dbEntities)
|
||||
{
|
||||
_remoteActors = remoteActors;
|
||||
_dbEntities = dbEntities;
|
||||
}
|
||||
|
||||
public JobKind Kind => JobKind.CountAccount;
|
||||
public int Concurrency => 2;
|
||||
public int MaxAttempts => 2;
|
||||
public int PerHostLimit => 1;
|
||||
|
||||
public static string DedupeKey(string actorUri, DateTime day) => $"count|{actorUri}|{day:yyyyMMdd}";
|
||||
|
||||
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
|
||||
{
|
||||
var actor = await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == job.Payload && !a.DeletionAt.HasValue).ExecuteFirstAsync(token);
|
||||
if (actor == default)
|
||||
return JobOutcome.Done;
|
||||
await DB.Default.Update<ForeignAvatar>().MatchID(actor.ID)
|
||||
.Modify(a => a.FollowersCount, await Total(actor, actor.FollowersURL, token))
|
||||
.Modify(a => a.FollowingCount, await Total(actor, actor.FollowingURL, token))
|
||||
.Modify(a => a.StatusesCount, await Total(actor, actor.OutboxURL, token))
|
||||
.Modify(a => a.CountedAt, DateTime.UtcNow)
|
||||
.ExecuteAsync(token);
|
||||
return JobOutcome.Done;
|
||||
}
|
||||
|
||||
// only from the actor's own server, as everything we believe about it
|
||||
async Task<int?> Total(ForeignAvatar actor, string collection, CancellationToken token)
|
||||
{
|
||||
if (string.IsNullOrEmpty(collection) || !Origin.Same(collection, actor.ActorURI))
|
||||
return default;
|
||||
using var scope = HttpScope.For("collection");
|
||||
using var fetched = await _remoteActors.FetchObject(collection, token);
|
||||
return fetched != default && fetched.Root.ValueKind == System.Text.Json.JsonValueKind.Object
|
||||
&& fetched.Root.TryGetProperty("totalItems", out var total) && total.TryGetInt32(out var count) && count >= 0
|
||||
? count
|
||||
: default;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -62,6 +62,7 @@ namespace PrivaPub.Federation.Actors
|
||||
{
|
||||
string BaseAddress { get; }
|
||||
Task<LocalActor> FindByUserName(string userName, CancellationToken token);
|
||||
Task<GoneActor> Gone(string userName, CancellationToken token);
|
||||
Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token);
|
||||
Task<LocalActor> FindByUri(string actorUri, CancellationToken token);
|
||||
Task<LocalActor> GetInstanceActor(CancellationToken token);
|
||||
@@ -71,6 +72,9 @@ namespace PrivaPub.Federation.Actors
|
||||
LocalActor FromGroup(GroupEntity group);
|
||||
}
|
||||
|
||||
// a persona or group that was deleted: its name stays reserved, and its documents answer 410 with this
|
||||
public sealed record GoneActor(string Uri, string FormerType, DateTime DeletedAt);
|
||||
|
||||
public class LocalActorService : ILocalActorService
|
||||
{
|
||||
public const string InstanceUserName = "privapub";
|
||||
@@ -113,6 +117,18 @@ namespace PrivaPub.Federation.Actors
|
||||
return group == default ? default : FromGroup(group);
|
||||
}
|
||||
|
||||
public async Task<GoneActor> Gone(string userName, CancellationToken token)
|
||||
{
|
||||
if (string.IsNullOrEmpty(userName))
|
||||
return default;
|
||||
userName = userName.ToLowerInvariant();
|
||||
var avatar = await _dbEntities.Avatars.Match(a => a.UserName == userName && a.DeletionAt.HasValue).ExecuteFirstAsync(token);
|
||||
if (avatar != default)
|
||||
return new GoneActor($"{BaseAddress}/peasants/{avatar.UserName}", "Person", avatar.DeletionAt.Value);
|
||||
var group = await _dbEntities.Groups.Match(g => g.UserName == userName && g.DeletionAt.HasValue).ExecuteFirstAsync(token);
|
||||
return group == default ? default : new GoneActor($"{BaseAddress}/peasants/{group.UserName}", "Group", group.DeletionAt.Value);
|
||||
}
|
||||
|
||||
public async Task<LocalActor> FindById(LocalActorKind kind, string id, CancellationToken token)
|
||||
{
|
||||
switch (kind)
|
||||
|
||||
@@ -36,9 +36,12 @@ namespace PrivaPub.Federation.Actors
|
||||
readonly DbEntities _dbEntities;
|
||||
readonly IOptionsMonitor<FederationOptions> _options;
|
||||
|
||||
readonly Infrastructure.Jobs.IJobQueue _jobs;
|
||||
|
||||
public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities,
|
||||
IOptionsMonitor<FederationOptions> options = default)
|
||||
IOptionsMonitor<FederationOptions> options = default, Infrastructure.Jobs.IJobQueue jobs = default)
|
||||
{
|
||||
_jobs = jobs;
|
||||
_http = http;
|
||||
_localActors = localActors;
|
||||
_cache = cache;
|
||||
@@ -89,7 +92,16 @@ namespace PrivaPub.Federation.Actors
|
||||
return cached;
|
||||
|
||||
var key = cached == default ? default : actor.Key(cached.PublicKeyId);
|
||||
return await Upsert(actor, key ?? actor.Keys.FirstOrDefault(), token);
|
||||
return await Counted(await Upsert(actor, key ?? actor.Keys.FirstOrDefault(), token), token);
|
||||
}
|
||||
|
||||
// its counts are read once a day, after it was fetched (AccountCountsJob)
|
||||
async Task<ForeignAvatar> Counted(ForeignAvatar stored, CancellationToken token)
|
||||
{
|
||||
if (_jobs != default && stored is { DeletionAt: null } && Uri.TryCreate(stored.ActorURI, UriKind.Absolute, out var uri))
|
||||
await _jobs.Enqueue(Models.Jobs.JobKind.CountAccount, stored.ActorURI, uri.Host.ToLowerInvariant(),
|
||||
AccountCountsJob.DedupeKey(stored.ActorURI, DateTime.UtcNow), token);
|
||||
return stored;
|
||||
}
|
||||
|
||||
public async Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token)
|
||||
@@ -121,7 +133,7 @@ namespace PrivaPub.Federation.Actors
|
||||
var key = actor?.Key(keyId);
|
||||
if (key == default)
|
||||
return default;
|
||||
return await Upsert(actor, key, token);
|
||||
return await Counted(await Upsert(actor, key, token), token);
|
||||
}
|
||||
|
||||
public bool KeyTemporarilyUnavailable(string keyId) => Origin.Of(keyId) != default && _http.FailedTemporarily(StripFragment(keyId));
|
||||
|
||||
@@ -17,6 +17,7 @@ using PrivaPub.Domain.Privacy;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Rendering;
|
||||
using PrivaPub.Federation.Inbox;
|
||||
using PrivaPub.Federation.Outbox;
|
||||
using PrivaPub.Federation.Signing;
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
using PrivaPub.Models.Group;
|
||||
@@ -52,6 +53,20 @@ namespace PrivaPub.Federation.Controllers
|
||||
public async Task<IActionResult> GetActor(string actor, CancellationToken token)
|
||||
{
|
||||
var local = await _localActors.FindByUserName(actor, token);
|
||||
if (local == default && await _localActors.Gone(actor, token) is { } gone)
|
||||
return new ContentResult
|
||||
{
|
||||
Content = new JsonObject
|
||||
{
|
||||
["@context"] = ActivityPubRenderer.ActivityStreams,
|
||||
["id"] = gone.Uri,
|
||||
["type"] = "Tombstone",
|
||||
["formerType"] = gone.FormerType,
|
||||
["deleted"] = ActivityPubRenderer.Timestamp(gone.DeletedAt)
|
||||
}.ToJsonString(),
|
||||
ContentType = ActivityContentType,
|
||||
StatusCode = StatusCodes.Status410Gone
|
||||
};
|
||||
if (local is not { IsFederated: true })
|
||||
return NotFound();
|
||||
if (WantsHtml() && local.Kind != LocalActorKind.Application)
|
||||
@@ -69,9 +84,8 @@ namespace PrivaPub.Federation.Controllers
|
||||
|
||||
if (!page)
|
||||
{
|
||||
var total = local.Kind == LocalActorKind.Group
|
||||
? await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupId == local.Id) & f.Where(VisibilityPolicy.IsPublic), token)
|
||||
: await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupUserId == local.Id && !p.IsFederatedCopy) & f.Where(VisibilityPolicy.IsPublic), token);
|
||||
// as Mastodon's: statuses_count, though only public posts are listed
|
||||
var total = await Counted.PostsBy(local, token);
|
||||
return Activity(ActivityPubRenderer.OrderedCollection(local.Outbox, (int)total, default, $"{local.Outbox}?page=true"));
|
||||
}
|
||||
|
||||
@@ -87,7 +101,7 @@ namespace PrivaPub.Federation.Controllers
|
||||
{
|
||||
if (local.Kind == LocalActorKind.Group)
|
||||
{
|
||||
items.Add(ActivityPubRenderer.Announce(local, post.ObjectURI, $"announce-{post.ID}"));
|
||||
items.Add(ActivityPubRenderer.Announce(local, post.ObjectURI, $"announce-{post.ID}", post.CreationDate));
|
||||
continue;
|
||||
}
|
||||
if (post.ReblogOfPostId != default)
|
||||
@@ -122,7 +136,13 @@ namespace PrivaPub.Federation.Controllers
|
||||
public async Task<IActionResult> Following(string actor, CancellationToken token)
|
||||
{
|
||||
var local = await _localActors.FindByUserName(actor, token);
|
||||
return local is not { IsFederated: true } ? NotFound() : Activity(ActivityPubRenderer.OrderedCollection(local.Following, 0, default));
|
||||
if (local is not { IsFederated: true })
|
||||
return NotFound();
|
||||
// the count, as following_count says it; who is followed is never listed (FEDERATION.md)
|
||||
var following = local.Kind == LocalActorKind.Person
|
||||
? await DB.Default.CountAsync<Following>(f => f.AvatarId == local.Id && f.State == FollowState.Accepted, token)
|
||||
: 0;
|
||||
return Activity(ActivityPubRenderer.OrderedCollection(local.Following, (int)following, default));
|
||||
}
|
||||
|
||||
[HttpGet, Route("{actor}/flock")]
|
||||
@@ -185,11 +205,13 @@ namespace PrivaPub.Federation.Controllers
|
||||
public async Task<IActionResult> Post(string actor, string postId, CancellationToken token)
|
||||
{
|
||||
var (local, post) = await PublicPost(actor, postId, token);
|
||||
if (post == default && await CirclePost(actor, postId, token) is { } circlePost)
|
||||
if (post == default && await SignedPost(actor, postId, token) is { } signed)
|
||||
{
|
||||
var circleNote = ActivityPubRenderer.Note(circlePost.Post, circlePost.Author, circlePost.Circle, circlePost.Post.InReplyToURI);
|
||||
circleNote["@context"] = ActivityPubRenderer.Context();
|
||||
return Activity(circleNote);
|
||||
if (signed.Post.DeletedAt.HasValue)
|
||||
return TombstoneOf(signed.Author, signed.Post);
|
||||
var signedNote = (JsonObject)signed.Note.DeepClone();
|
||||
signedNote["@context"] = ActivityPubRenderer.Context();
|
||||
return Activity(signedNote);
|
||||
}
|
||||
if (post == default)
|
||||
return await Tombstone(actor, postId, token) ?? NotFound();
|
||||
@@ -236,20 +258,55 @@ namespace PrivaPub.Federation.Controllers
|
||||
{
|
||||
var booster = await _localActors.FindByUserName(actor, token);
|
||||
var uri = booster?.ActivityUri(activityId);
|
||||
if (booster is { Kind: LocalActorKind.Group, IsCircle: false })
|
||||
return await GroupAnnounce(booster, activityId, uri, token);
|
||||
var reblog = uri == default ? default : await _dbEntities.Posts.Match(p => p.ObjectURI == uri && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
|
||||
var rendered = reblog == default ? default : await AnnounceFor(reblog, token);
|
||||
if (rendered != default && WantsHtml() && await _dbEntities.Posts.MatchID(reblog.ReblogOfPostId).ExecuteFirstAsync(token) is { } boosted)
|
||||
return Redirect(boosted.Url ?? boosted.ObjectURI);//a boost has no page of its own: the boosted post's
|
||||
return rendered == default ? NotFound() : Activity(rendered);
|
||||
}
|
||||
if (!activityId.StartsWith("create-", StringComparison.Ordinal))
|
||||
return NotFound();
|
||||
var (local, post) = await PublicPost(actor, activityId["create-".Length..], token);
|
||||
return post == default ? NotFound() : Activity(await CreateFor(post, local, token));
|
||||
if (post != default)
|
||||
return Activity(await CreateFor(post, local, token));
|
||||
return await SignedPost(actor, activityId["create-".Length..], token) is { Post.DeletedAt: null } signed
|
||||
? Activity(ActivityPubRenderer.Create(signed.Author, (JsonObject)signed.Note.DeepClone(), activityId))
|
||||
: NotFound();
|
||||
}
|
||||
|
||||
// A DM's `context`: the conversation's posts, for its participants (or their servers' instance actors) only.
|
||||
[HttpGet, Route("{actor}/whispers/{conversationId}")]
|
||||
public async Task<IActionResult> Whispers(string actor, string conversationId, CancellationToken token)
|
||||
{
|
||||
var local = await _localActors.FindByUserName(actor, token);
|
||||
if (local is not { IsFederated: true, Kind: LocalActorKind.Person })
|
||||
return NotFound();
|
||||
var uri = local.ConversationUri(conversationId);
|
||||
var conversation = await _dbEntities.DmGroups.Match(g => g.ID == conversationId && g.ConversationURI == uri && !g.DeletionAt.HasValue).ExecuteFirstAsync(token);
|
||||
if (conversation == default || !SignedFetchAuthorizer.MayReadConversation(conversation, await _fetches.Requester(Request, token)))
|
||||
return NotFound();
|
||||
var posts = await _dbEntities.Posts
|
||||
.Match(p => p.ConversationId == conversationId && p.Visibility == PostVisibility.Direct && !p.DeletedAt.HasValue)
|
||||
.Sort(p => p.CreationDate, Order.Ascending)
|
||||
.ExecuteAsync(token);
|
||||
return Activity(new JsonObject
|
||||
{
|
||||
["@context"] = ActivityPubRenderer.ActivityStreams,
|
||||
["id"] = uri,
|
||||
["type"] = "OrderedCollection",
|
||||
["totalItems"] = posts.Count,
|
||||
["orderedItems"] = new JsonArray(posts.Select(p => (JsonNode)p.ObjectURI).ToArray())
|
||||
});
|
||||
}
|
||||
|
||||
[HttpPost, Route("{actor}/mouth"), EnableRateLimiting(RateLimiting.Inbox)]
|
||||
public async Task<IActionResult> Inbox(string actor, CancellationToken token)
|
||||
{
|
||||
var local = await _localActors.FindByUserName(actor, token);
|
||||
if (local == default && await _localActors.Gone(actor, token) != default)
|
||||
return StatusCode(StatusCodes.Status410Gone);
|
||||
if (local is not { IsFederated: true })
|
||||
return Answer(_inbox.NoSuchRecipient(Request));
|
||||
return Answer(await _inbox.Receive(Request, local, token));
|
||||
@@ -275,18 +332,37 @@ namespace PrivaPub.Federation.Controllers
|
||||
return (local, post);
|
||||
}
|
||||
|
||||
async Task<(LocalActor Author, LocalActor Circle, PostEntity Post)?> CirclePost(string actor, string postId, CancellationToken token)
|
||||
// A followers-only, direct or circle post (deleted ones included), for a signed request from someone it was for
|
||||
// (SignedFetchAuthorizer.MayRead). It is rendered as it was delivered: a circle post also names, in cc, the requesting
|
||||
// member, or the members on the requesting instance actor's server.
|
||||
async Task<(LocalActor Author, PostEntity Post, JsonObject Note)?> SignedPost(string actor, string postId, CancellationToken token)
|
||||
{
|
||||
var local = await _localActors.FindByUserName(actor, token);
|
||||
if (local is not { Kind: LocalActorKind.Person })
|
||||
if (local is not { IsFederated: true, Kind: LocalActorKind.Person })
|
||||
return default;
|
||||
var post = await _dbEntities.Posts
|
||||
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility == PostVisibility.Circle)
|
||||
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.ReblogOfPostId == null
|
||||
&& (p.Visibility == PostVisibility.FollowersOnly || p.Visibility == PostVisibility.Direct || p.Visibility == PostVisibility.Circle))
|
||||
.ExecuteFirstAsync(token);
|
||||
var circle = post == default ? default : await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token);
|
||||
if (circle == default || !SignedFetchAuthorizer.MayReadCircle(circle, await _fetches.Requester(Request, token)))
|
||||
if (post == default)
|
||||
return default;
|
||||
return (local, _localActors.FromGroup(circle), post);
|
||||
var requester = await _fetches.Requester(Request, token);
|
||||
if (!await _fetches.MayRead(post, requester, token))
|
||||
return default;
|
||||
if (post.Visibility == PostVisibility.Circle)
|
||||
{
|
||||
var circle = await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token);
|
||||
var note = ActivityPubRenderer.Note(post, local, _localActors.FromGroup(circle), post.InReplyToURI);
|
||||
var readers = SignedFetchAuthorizer.CircleReaders(circle, requester);
|
||||
var named = readers.Count == 0 ? new List<Models.User.ForeignAvatar>() : await _dbEntities.ForeignAvatars.Match(a => readers.Contains(a.ActorURI)).ExecuteAsync(token);
|
||||
return (local, post, OutboxPublisher.Naming(note, named));
|
||||
}
|
||||
var rendered = post.Visibility == PostVisibility.Direct
|
||||
? ActivityPubRenderer.DirectNote(post, local, Array.Empty<(string, string)>(), post.ContextURI)
|
||||
: ActivityPubRenderer.Note(post, local, default, post.InReplyToURI);
|
||||
rendered["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
|
||||
rendered["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
|
||||
return (local, post, rendered);
|
||||
}
|
||||
|
||||
async Task<IActionResult> Tombstone(string actor, string postId, CancellationToken token)
|
||||
@@ -298,8 +374,11 @@ namespace PrivaPub.Federation.Controllers
|
||||
.Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.DeletedAt.HasValue
|
||||
&& (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted))
|
||||
.ExecuteFirstAsync(token);
|
||||
if (deleted == default)
|
||||
return default;
|
||||
return deleted == default ? default : TombstoneOf(local, deleted);
|
||||
}
|
||||
|
||||
ContentResult TombstoneOf(LocalActor local, PostEntity deleted)
|
||||
{
|
||||
var tombstone = new JsonObject
|
||||
{
|
||||
["@context"] = ActivityPubRenderer.ActivityStreams,
|
||||
@@ -311,6 +390,24 @@ namespace PrivaPub.Federation.Controllers
|
||||
return new ContentResult { Content = tombstone.ToJsonString(), ContentType = ActivityContentType, StatusCode = StatusCodes.Status410Gone };
|
||||
}
|
||||
|
||||
// A community's announce: one GroupDistributor sent (kept as sent), or one its outbox lists (announce-{postId}),
|
||||
// while the post it is about is still shown; 410 once that post is gone.
|
||||
async Task<IActionResult> GroupAnnounce(LocalActor group, string activityId, string uri, CancellationToken token)
|
||||
{
|
||||
if (await DB.Default.Find<GroupAnnouncement>().Match(a => a.ActivityURI == uri && a.GroupId == group.Id).ExecuteFirstAsync(token) is { } kept)
|
||||
{
|
||||
var about = await _dbEntities.Posts.Match(p => p.ObjectURI == kept.ObjectURI).ExecuteFirstAsync(token);
|
||||
if (!VisibilityPolicy.Shown(about) || about.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
|
||||
return StatusCode(StatusCodes.Status410Gone);
|
||||
var body = JsonNode.Parse(kept.Body)!.AsObject();
|
||||
body["@context"] = ActivityPubRenderer.Context();
|
||||
return Activity(body);
|
||||
}
|
||||
var postId = activityId["announce-".Length..];
|
||||
var post = await _dbEntities.Posts.Match(p => p.ID == postId && p.GroupId == group.Id).Match(VisibilityPolicy.IsPublic).ExecuteFirstAsync(token);
|
||||
return post == default ? NotFound() : Activity(ActivityPubRenderer.Announce(group, post.ObjectURI, activityId, post.CreationDate));
|
||||
}
|
||||
|
||||
async Task<JsonObject> AnnounceFor(PostEntity reblog, CancellationToken token)
|
||||
{
|
||||
var original = await _dbEntities.Posts.MatchID(reblog.ReblogOfPostId).ExecuteFirstAsync(token);
|
||||
@@ -349,7 +446,9 @@ namespace PrivaPub.Federation.Controllers
|
||||
{
|
||||
if (HttpMethods.IsGet(Request.Method))
|
||||
Response.Headers.Vary = "Accept";
|
||||
if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method)
|
||||
// SecureMode asks every reader of ActivityPub documents for a signature, except for the instance actor, whose key
|
||||
// peers need first, and except for browsers, which only get redirected to the public pages
|
||||
if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method) && !WantsHtml()
|
||||
&& !string.Equals(context.RouteData.Values["actor"] as string, LocalActorService.InstanceUserName, StringComparison.OrdinalIgnoreCase)
|
||||
&& await _fetches.Requester(Request, HttpContext.RequestAborted) == default)
|
||||
{
|
||||
|
||||
@@ -49,6 +49,8 @@ namespace PrivaPub.Federation.Controllers
|
||||
if (parts.Length != 2 || !parts[1].Equals(domain, StringComparison.OrdinalIgnoreCase))
|
||||
return NotFound();
|
||||
actor = await _localActors.FindByUserName(parts[0], token);
|
||||
if (actor == default && await _localActors.Gone(parts[0], token) != default)
|
||||
return StatusCode(StatusCodes.Status410Gone);
|
||||
}
|
||||
else
|
||||
actor = await _localActors.FindByUri(resource, token);
|
||||
@@ -92,8 +94,8 @@ namespace PrivaPub.Federation.Controllers
|
||||
[HttpGet, Route("/nodeinfo/{version:regex(^2\\.[[01]]$)}")]
|
||||
public async Task<IActionResult> NodeInfo(string version, CancellationToken token)
|
||||
{
|
||||
var users = await DB.Default.CountAsync<Avatar>(a => !a.DeletionAt.HasValue, token);
|
||||
var posts = await DB.Default.CountAsync<PostEntity>(f => f.Where(p => !p.IsFederatedCopy && p.ReblogOfPostId == null) & f.Where(VisibilityPolicy.IsPublic), token);
|
||||
var users = await Counted.Users(token);
|
||||
var posts = await Counted.LocalPosts(token);
|
||||
var software = new JsonObject { ["name"] = "privapub", ["version"] = BuildInfo.Ref };
|
||||
if (version == "2.1")
|
||||
{
|
||||
@@ -109,7 +111,12 @@ namespace PrivaPub.Federation.Controllers
|
||||
["openRegistrations"] = _registrations.CurrentValue.IsOpen,
|
||||
["usage"] = new JsonObject
|
||||
{
|
||||
["users"] = new JsonObject { ["total"] = users },
|
||||
["users"] = new JsonObject
|
||||
{
|
||||
["total"] = users,
|
||||
["activeMonth"] = await Counted.ActiveUsers(30, token),
|
||||
["activeHalfyear"] = await Counted.ActiveUsers(180, token)
|
||||
},
|
||||
["localPosts"] = posts
|
||||
},
|
||||
["metadata"] = new JsonObject
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Domain.Privacy;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.Social;
|
||||
using PrivaPub.Models.User;
|
||||
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Federation.Inbox
|
||||
{
|
||||
// A remote account that deleted itself keeps its posts, hidden (owner decision 2026-10-03), but leaves nothing in our
|
||||
// numbers: its likes, downvotes, reactions and poll votes go, and the counters they had moved come back, as do the
|
||||
// reblog, reply and quote counts its own posts had added. Its notifications go too. ForgottenAt makes it run once.
|
||||
public static class GoneActors
|
||||
{
|
||||
public static async Task Forget(ForeignAvatar actor, CancellationToken token)
|
||||
{
|
||||
if (actor == default || actor.ForgottenAt.HasValue)
|
||||
return;
|
||||
var uri = actor.ActorURI;
|
||||
|
||||
foreach (var like in await DB.Default.Find<Favourite>().Match(f => f.ActorURI == uri).ExecuteAsync(token))
|
||||
{
|
||||
await DB.Default.Update<PostEntity>().MatchID(like.PostId).Modify(b => b.Inc(p => p.FavouritesCount, -1)).ExecuteAsync(token);
|
||||
await DB.Default.DeleteAsync<Favourite>(like.ID);
|
||||
}
|
||||
foreach (var down in await DB.Default.Find<Downvote>().Match(d => d.ActorURI == uri).ExecuteAsync(token))
|
||||
{
|
||||
await DB.Default.Update<PostEntity>().MatchID(down.PostId).Modify(b => b.Inc(p => p.DownvotesCount, -1)).ExecuteAsync(token);
|
||||
await DB.Default.DeleteAsync<Downvote>(down.ID);
|
||||
}
|
||||
await DB.Default.DeleteAsync<Reaction>(r => r.ActorURI == uri);
|
||||
foreach (var ballot in (await DB.Default.Find<PollVote>().Match(v => v.ActorURI == uri).ExecuteAsync(token)).GroupBy(v => v.PostId))
|
||||
{
|
||||
var poll = await DB.Default.Find<PostEntity>().MatchID(ballot.Key).ExecuteFirstAsync(token);
|
||||
if (poll?.Poll != default)
|
||||
{
|
||||
var update = DB.Default.Update<PostEntity>().MatchID(poll.ID);
|
||||
foreach (var vote in ballot.Where(v => v.Choice >= 0 && v.Choice < poll.Poll.Options.Count))
|
||||
update = update.Modify(b => b.Inc($"{nameof(PostEntity.Poll)}.{nameof(PostPoll.Options)}.{vote.Choice}.{nameof(PollOption.Votes)}", -1));
|
||||
if (poll.Poll.VotersCount.HasValue)
|
||||
update = update.Modify(b => b.Inc($"{nameof(PostEntity.Poll)}.{nameof(PostPoll.VotersCount)}", -1));
|
||||
await update.ExecuteAsync(token);
|
||||
}
|
||||
await DB.Default.DeleteAsync<PollVote>(v => v.PostId == ballot.Key && v.ActorURI == uri);
|
||||
}
|
||||
|
||||
foreach (var post in await DB.Default.Find<PostEntity>().Match(p => p.ActorURI == uri && !p.DeletedAt.HasValue).ExecuteAsync(token))
|
||||
{
|
||||
if (!string.IsNullOrEmpty(post.ReblogOfPostId))
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ReblogOfPostId).Modify(b => b.Inc(p => p.ReblogsCount, -1)).ExecuteAsync(token);
|
||||
if (!string.IsNullOrEmpty(post.AnsweringToPostId) && Counted.Reply(post))
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token);
|
||||
if (!string.IsNullOrEmpty(post.QuotedPostId) && post.QuoteState == QuoteState.Accepted)
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
|
||||
}
|
||||
|
||||
await DB.Default.DeleteAsync<Notification>(n => n.FromActorURI == uri);
|
||||
await DB.Default.Update<ForeignAvatar>().MatchID(actor.ID).Modify(a => a.ForgottenAt, DateTime.UtcNow).ExecuteAsync(token);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -182,7 +182,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
await _previews.Wanted(post, token);
|
||||
await _quotes.Resolve(post, note, token);
|
||||
|
||||
if (parent != default)
|
||||
if (parent != default && Domain.Privacy.Counted.Reply(post))
|
||||
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
|
||||
await _fanout.Distribute(post, token);
|
||||
if (conversation != default)
|
||||
|
||||
@@ -71,6 +71,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
await DB.Default.DeleteAsync<Following>(f => f.TargetActorURI == actor.ActorURI);
|
||||
await DB.Default.DeleteAsync<TimelineEntry>(e => e.AuthorAccountId == actor.ID);
|
||||
await DB.Default.Update<PostEntity>().Match(p => p.ActorURI == actor.ActorURI).Modify(p => p.AuthorGone, true).ExecuteAsync(token);
|
||||
await GoneActors.Forget(actor, token);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Entities;
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Federation.Actors;
|
||||
@@ -33,6 +33,7 @@ namespace PrivaPub.Federation.Inbox
|
||||
post.Video = note.Video ?? post.Video;
|
||||
post.Audio = note.Audio ?? post.Audio;
|
||||
post.Event = note.Event ?? post.Event;
|
||||
post.Place = note.Place;
|
||||
if (!IsEdit(note, post))
|
||||
{
|
||||
await DB.Default.SaveAsync(post, token);
|
||||
@@ -104,7 +105,7 @@ namespace PrivaPub.Federation.Inbox
|
||||
await DB.Default.DeleteAsync<ObjectRecord>(r => r.PostId == post.ID || r.ObjectURI == objectUri);
|
||||
await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID);
|
||||
await DB.Default.DeleteAsync<PostEntity>(p => p.ReblogOfPostId == post.ID);
|
||||
if (!string.IsNullOrEmpty(post.AnsweringToPostId))
|
||||
if (!string.IsNullOrEmpty(post.AnsweringToPostId) && Domain.Privacy.Counted.Reply(post))
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.AnsweringToPostId).Modify(b => b.Inc(p => p.RepliesCount, -1)).ExecuteAsync(token);
|
||||
if (post.QuoteState == QuoteState.Accepted && !string.IsNullOrEmpty(post.QuotedPostId))
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Driver;
|
||||
using MongoDB.Driver;
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
@@ -90,6 +90,7 @@ namespace PrivaPub.Federation.Inbox
|
||||
Video = note.Video,
|
||||
Audio = note.Audio,
|
||||
Event = note.Event,
|
||||
Place = note.Place,
|
||||
HasContentWarning = note.Sensitive,
|
||||
Text = note.ContentHtml,
|
||||
ContentHtml = note.ContentHtml,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Entities;
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
@@ -18,12 +18,17 @@ namespace PrivaPub.Federation.Objects
|
||||
readonly IFederationHttp _http;
|
||||
readonly IGeoLocator _geo;
|
||||
readonly IConnectedAddresses _addresses;
|
||||
readonly ICdnRanges _cdnRanges;
|
||||
readonly IEdgeHints _edges;
|
||||
|
||||
public InstanceDescriber(IFederationHttp http, IGeoLocator geo = default, IConnectedAddresses addresses = default)
|
||||
public InstanceDescriber(IFederationHttp http, IGeoLocator geo = default, IConnectedAddresses addresses = default, ICdnRanges cdnRanges = default,
|
||||
IEdgeHints edges = default)
|
||||
{
|
||||
_http = http;
|
||||
_geo = geo;
|
||||
_addresses = addresses;
|
||||
_cdnRanges = cdnRanges;
|
||||
_edges = edges;
|
||||
}
|
||||
|
||||
public JobKind Kind => JobKind.DescribeInstance;
|
||||
@@ -145,35 +150,57 @@ namespace PrivaPub.Federation.Objects
|
||||
.Modify(s => s.RegistrationMode, instance?.RegistrationMode)
|
||||
.Modify(s => s.Languages, instance?.Languages)
|
||||
.Modify(s => s.Country, instance?.Geo?.Country)
|
||||
.Modify(s => s.City, instance?.Geo?.City)
|
||||
.Modify(s => s.Latitude, instance?.Geo?.Latitude)
|
||||
.Modify(s => s.Longitude, instance?.Geo?.Longitude)
|
||||
.Modify(s => s.Asn, instance?.Geo?.Asn)
|
||||
.Modify(s => s.Cdn, instance?.Geo?.Cdn)
|
||||
.Modify(s => s.CdnDomain, instance?.Geo?.CdnDomain)
|
||||
.Modify(s => s.CdnSource, instance?.Geo?.CdnSource)
|
||||
.Modify(s => s.Ipv6, instance?.Geo?.Ipv6)
|
||||
.Option(o => o.IsUpsert = true)
|
||||
.ExecuteAsync(token);
|
||||
return facts != default;
|
||||
}
|
||||
|
||||
// Where the address we reached sits, and the CDN in front of it, found by its published ranges, then by the
|
||||
// fingerprint in the responses we just got, then by a network that carries only a CDN.
|
||||
InstanceGeo Locate(string host)
|
||||
{
|
||||
var reached = _addresses?.Of(new Uri(Address($"https://{host}/")).Host);
|
||||
var reachedHost = new Uri(Address($"https://{host}/")).Host;
|
||||
var reached = _addresses?.Of(reachedHost);
|
||||
var fix = reached == default ? default : _geo?.Locate(reached);
|
||||
if (fix == default)
|
||||
var (cdn, cdnSource) = Cdn(reached, reachedHost, fix?.Asn, _cdnRanges, _edges);
|
||||
if (fix == default && cdn == default)
|
||||
return default;
|
||||
return new InstanceGeo
|
||||
{
|
||||
Country = fix.Country,
|
||||
City = fix.City,
|
||||
Latitude = fix.Latitude,
|
||||
Longitude = fix.Longitude,
|
||||
Asn = fix.Asn,
|
||||
AsnOrg = fix.AsnOrg,
|
||||
Cdn = CdnNetworks.Of(fix.Asn),
|
||||
Ipv6 = reached.AddressFamily == AddressFamily.InterNetworkV6,
|
||||
Source = _geo.Source,
|
||||
Country = fix?.Country,
|
||||
City = fix?.City,
|
||||
Latitude = fix?.Latitude,
|
||||
Longitude = fix?.Longitude,
|
||||
Asn = fix?.Asn,
|
||||
AsnOrg = fix?.AsnOrg,
|
||||
Cdn = cdn?.Name,
|
||||
CdnDomain = cdn?.Domain,
|
||||
CdnSource = cdnSource,
|
||||
Ipv6 = reached?.AddressFamily == AddressFamily.InterNetworkV6,
|
||||
Source = _geo?.Source,
|
||||
LocatedAt = DateTime.UtcNow
|
||||
};
|
||||
}
|
||||
|
||||
public static (CdnProvider Cdn, string Source) Cdn(System.Net.IPAddress address, string host, int? asn, ICdnRanges ranges, IEdgeHints edges)
|
||||
{
|
||||
if (ranges?.Of(address) is { } listed)
|
||||
return (listed, "ranges");
|
||||
if (edges?.Of(host) is { } hinted)
|
||||
return (hinted, "headers");
|
||||
if (CdnCatalog.OfAsn(asn) is { } network)
|
||||
return (network, "asn");
|
||||
return default;
|
||||
}
|
||||
|
||||
//where an https address is fetched from: itself, except in tests, which point it at a plain-http peer
|
||||
protected virtual string Address(string url) => url;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.Post;
|
||||
|
||||
using System.Globalization;
|
||||
using System.Text;
|
||||
@@ -44,6 +44,7 @@ namespace PrivaPub.Federation.Objects
|
||||
public VideoDetails Video { get; init; }
|
||||
public AudioDetails Audio { get; init; }
|
||||
public EventDetails Event { get; init; }
|
||||
public PostPlace Place { get; init; }
|
||||
}
|
||||
|
||||
public static class NoteParser
|
||||
@@ -110,7 +111,8 @@ namespace PrivaPub.Federation.Objects
|
||||
Link = ObjectShapes.Link(note),
|
||||
Video = ObjectShapes.Video(note),
|
||||
Audio = ObjectShapes.Audio(note),
|
||||
Event = ObjectShapes.Event(note)
|
||||
Event = ObjectShapes.Event(note),
|
||||
Place = ObjectShapes.NotePlace(note)
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.Post;
|
||||
|
||||
using System.Globalization;
|
||||
using System.Net;
|
||||
@@ -293,6 +293,31 @@ namespace PrivaPub.Federation.Objects
|
||||
};
|
||||
}
|
||||
|
||||
//a post's own place, as Pixelfed sends it: a Place with coordinates (numbers, or numbers as strings); an Event's
|
||||
//places are its own (Event)
|
||||
public static PostPlace NotePlace(JsonObject note)
|
||||
{
|
||||
if (Value(note, "type") == "Event")
|
||||
return default;
|
||||
var place = Objects(note["location"]).FirstOrDefault(l => Value(l, "type") == "Place");
|
||||
var latitude = Coordinate(place, "latitude");
|
||||
var longitude = Coordinate(place, "longitude");
|
||||
if (latitude is not (>= -90 and <= 90) || longitude is not (>= -180 and <= 180))
|
||||
return default;
|
||||
return new PostPlace
|
||||
{
|
||||
Name = Text(Value(place, "name"), 200),
|
||||
Latitude = Math.Round(latitude.Value, 4),
|
||||
Longitude = Math.Round(longitude.Value, 4),
|
||||
Country = Text(Value(place, "country"), 100)
|
||||
};
|
||||
}
|
||||
|
||||
static double? Coordinate(JsonObject node, string property) =>
|
||||
Double(node, property) ?? (double.TryParse(Value(node, property), NumberStyles.Float, CultureInfo.InvariantCulture, out var number) && double.IsFinite(number)
|
||||
? number
|
||||
: (double?)null);
|
||||
|
||||
static EventPlace Place(JsonObject location)
|
||||
{
|
||||
var type = Value(location, "type");
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Rendering;
|
||||
using PrivaPub.Models.Federation;
|
||||
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
@@ -14,6 +17,26 @@ namespace PrivaPub.Federation.Outbox
|
||||
|
||||
public class GroupDistributor : IGroupDistributor
|
||||
{
|
||||
// so the announce's id resolves (PeasantsController.Grunt); a repeat of the same announce keeps the first
|
||||
static async Task Keep(LocalActor group, JsonObject announce, string objectUri, CancellationToken token)
|
||||
{
|
||||
var body = (JsonObject)announce.DeepClone();
|
||||
body.Remove("@context");
|
||||
try
|
||||
{
|
||||
await DB.Default.SaveAsync(new GroupAnnouncement
|
||||
{
|
||||
GroupId = group.Id,
|
||||
ActivityURI = body["id"]!.GetValue<string>(),
|
||||
ObjectURI = objectUri,
|
||||
Body = body.ToJsonString()
|
||||
}, token);
|
||||
}
|
||||
catch (MongoDB.Driver.MongoWriteException ex) when (ex.WriteError?.Category == MongoDB.Driver.ServerErrorCategory.DuplicateKey)
|
||||
{
|
||||
}
|
||||
}
|
||||
|
||||
readonly IDeliveryService _delivery;
|
||||
|
||||
public GroupDistributor(IDeliveryService delivery)
|
||||
@@ -43,8 +66,13 @@ namespace PrivaPub.Federation.Outbox
|
||||
// Every follower, the author's own server included: Lemmy 1.0 keeps its user's post pending until the community
|
||||
// announces it back, and clears that before it answers the echo 400 ("Object is not remote").
|
||||
await _delivery.EnqueueToFollowers(group, announce, token);
|
||||
await Keep(group, announce, objectUri, token);
|
||||
if (isNewPost && !string.IsNullOrEmpty(objectUri))
|
||||
await _delivery.EnqueueToFollowers(group, ActivityPubRenderer.Announce(group, objectUri, $"announce-object-{key}"), token);
|
||||
{
|
||||
var boost = ActivityPubRenderer.Announce(group, objectUri, $"announce-object-{key}");
|
||||
await _delivery.EnqueueToFollowers(group, boost, token);
|
||||
await Keep(group, boost, objectUri, token);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Rendering;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Text.Json.Nodes;
|
||||
@@ -14,6 +15,7 @@ namespace PrivaPub.Federation.Outbox
|
||||
{
|
||||
Task<IReadOnlyList<string>> Audience(LocalActor author, PostEntity post, CancellationToken token);
|
||||
Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token);
|
||||
Task PublishUpdate(LocalActor author, PostEntity post, string reason, CancellationToken token);
|
||||
Task PublishProfile(LocalActor actor, CancellationToken token);
|
||||
}
|
||||
|
||||
@@ -73,25 +75,79 @@ namespace PrivaPub.Federation.Outbox
|
||||
return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList();
|
||||
}
|
||||
|
||||
async Task<IReadOnlyList<string>> CircleMembers(string groupId, CancellationToken token)
|
||||
async Task<IReadOnlyList<string>> CircleMembers(string groupId, CancellationToken token) =>
|
||||
(await CircleRecipients(groupId, token)).Select(r => r.InboxURL).Distinct(StringComparer.Ordinal).ToList();
|
||||
|
||||
async Task<IReadOnlyList<ForeignAvatar>> CircleRecipients(string groupId, CancellationToken token)
|
||||
{
|
||||
var circle = string.IsNullOrEmpty(groupId) ? default : await _dbEntities.Groups.MatchID(groupId).ExecuteFirstAsync(token);
|
||||
if (circle == default)
|
||||
return Array.Empty<string>();
|
||||
return Array.Empty<ForeignAvatar>();
|
||||
var remote = circle.Members.Where(m => m.IsForeign).Select(m => m.AvatarId).ToList();
|
||||
if (remote.Count == 0)
|
||||
return Array.Empty<string>();
|
||||
return Array.Empty<ForeignAvatar>();
|
||||
return (await _dbEntities.ForeignAvatars.Match(a => remote.Contains(a.ActorURI)).ExecuteAsync(token))
|
||||
.Select(a => a.InboxURL).Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList();
|
||||
.Where(a => !string.IsNullOrEmpty(a.InboxURL))
|
||||
.ToList();
|
||||
}
|
||||
|
||||
public async Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token)
|
||||
{
|
||||
if (post.Visibility == PostVisibility.Circle)
|
||||
{
|
||||
foreach (var member in await CircleRecipients(post.GroupId, token))
|
||||
await _delivery.Enqueue(author, new[] { member.InboxURL }, Naming(activity, new[] { member }), token);
|
||||
return;
|
||||
}
|
||||
var inboxes = await Audience(author, post, token);
|
||||
if (inboxes.Count > 0)
|
||||
await _delivery.Enqueue(author, inboxes, activity, token);
|
||||
}
|
||||
|
||||
public async Task PublishUpdate(LocalActor author, PostEntity post, string reason, CancellationToken token)
|
||||
{
|
||||
var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token);
|
||||
await Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, group, reason), token);
|
||||
}
|
||||
|
||||
// Mastodon keeps a post only when it names one of its own accounts, and GoToSocial shows a post that is neither
|
||||
// public nor for followers only to the accounts it mentions; a circle post names only the circle. So each member's
|
||||
// copy, or a member's refetch, also names that member in cc and mentions them, silently, in its tags (owner decision
|
||||
// 2026-10-04): it tells each member nothing but that they are in the circle.
|
||||
public static JsonObject Naming(JsonObject activityOrObject, IEnumerable<ForeignAvatar> members)
|
||||
{
|
||||
var named = members.ToList();
|
||||
var copy = (JsonObject)activityOrObject.DeepClone();
|
||||
if (copy["object"] is JsonObject inner)
|
||||
{
|
||||
Name(copy, named, mention: false);
|
||||
if (inner.ContainsKey("to"))
|
||||
Name(inner, named, mention: true);
|
||||
}
|
||||
else
|
||||
Name(copy, named, mention: copy.ContainsKey("attributedTo"));
|
||||
return copy;
|
||||
}
|
||||
|
||||
static void Name(JsonObject node, IReadOnlyList<ForeignAvatar> members, bool mention)
|
||||
{
|
||||
var cc = node["cc"] as JsonArray ?? new JsonArray();
|
||||
var tags = node["tag"] as JsonArray ?? new JsonArray();
|
||||
foreach (var member in members)
|
||||
{
|
||||
if (!cc.Any(c => c?.GetValue<string>() == member.ActorURI))
|
||||
cc.Add(member.ActorURI);
|
||||
if (mention && !tags.Any(t => t?["href"]?.GetValue<string>() == member.ActorURI))
|
||||
tags.Add(new JsonObject { ["type"] = "Mention", ["href"] = member.ActorURI, ["name"] = Handle(member) });
|
||||
}
|
||||
node["cc"] = cc;
|
||||
if (mention)
|
||||
node["tag"] = tags;
|
||||
}
|
||||
|
||||
static string Handle(ForeignAvatar member) =>
|
||||
string.IsNullOrEmpty(member.UserName) ? member.ActorURI : $"@{member.UserName}@{new Uri(member.ActorURI).Authority}";
|
||||
|
||||
public async Task PublishProfile(LocalActor actor, CancellationToken token)
|
||||
{
|
||||
var document = ActivityPubRenderer.Actor(actor);
|
||||
|
||||
@@ -178,11 +178,11 @@ namespace PrivaPub.Federation.Rendering
|
||||
return note;
|
||||
}
|
||||
|
||||
public static JsonObject UpdateOf(PostEntity post, LocalActor author, string reason)
|
||||
public static JsonObject UpdateOf(PostEntity post, LocalActor author, LocalActor group, string reason)
|
||||
{
|
||||
var note = post.Visibility == PostVisibility.Direct
|
||||
? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI)
|
||||
: Note(post, author, default, post.InReplyToURI);
|
||||
: Note(post, author, group, post.InReplyToURI);
|
||||
note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
|
||||
note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
|
||||
return new JsonObject
|
||||
@@ -357,13 +357,13 @@ namespace PrivaPub.Federation.Rendering
|
||||
["object"] = note
|
||||
};
|
||||
|
||||
public static JsonObject Announce(LocalActor group, string objectUri, string activityId) => new()
|
||||
public static JsonObject Announce(LocalActor group, string objectUri, string activityId, DateTime? published = default) => new()
|
||||
{
|
||||
["@context"] = ActivityStreams,
|
||||
["id"] = group.ActivityUri(activityId),
|
||||
["type"] = "Announce",
|
||||
["actor"] = group.Uri,
|
||||
["published"] = Timestamp(DateTime.UtcNow),
|
||||
["published"] = Timestamp(published ?? DateTime.UtcNow),
|
||||
["to"] = new JsonArray(Public),
|
||||
["cc"] = new JsonArray(group.Followers),
|
||||
["object"] = objectUri
|
||||
|
||||
@@ -1,23 +1,34 @@
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using GroupEntity = PrivaPub.Models.Group.Group;
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Federation.Signing
|
||||
{
|
||||
public interface ISignedFetchAuthorizer
|
||||
{
|
||||
Task<ForeignAvatar> Requester(HttpRequest request, CancellationToken token);
|
||||
Task<bool> MayRead(PostEntity post, ForeignAvatar requester, CancellationToken token);
|
||||
}
|
||||
|
||||
// Who may refetch a post that is not public (owner decision 2026-10-04). A server that received a followers-only post,
|
||||
// a DM or a circle post refetches it, signed by the account it was for or by its instance actor, and Mastodon deletes its
|
||||
// copy when the refetch answers 404. So the post is served to a signed request from someone it was for, or from the
|
||||
// instance actor of a server where someone it was for lives; everyone else still gets 404.
|
||||
public class SignedFetchAuthorizer : ISignedFetchAuthorizer
|
||||
{
|
||||
readonly IRemoteActorService _remoteActors;
|
||||
readonly DbEntities _dbEntities;
|
||||
|
||||
public SignedFetchAuthorizer(IRemoteActorService remoteActors)
|
||||
public SignedFetchAuthorizer(IRemoteActorService remoteActors, DbEntities dbEntities)
|
||||
{
|
||||
_remoteActors = remoteActors;
|
||||
_dbEntities = dbEntities;
|
||||
}
|
||||
|
||||
public async Task<ForeignAvatar> Requester(HttpRequest request, CancellationToken token)
|
||||
@@ -33,8 +44,48 @@ namespace PrivaPub.Federation.Signing
|
||||
return actor != default && HttpSignatures.Verify(actor.PublicKey, signingString, parameters.Signature) ? actor : default;
|
||||
}
|
||||
|
||||
public async Task<bool> MayRead(PostEntity post, ForeignAvatar requester, CancellationToken token)
|
||||
{
|
||||
if (post == default || requester == default)
|
||||
return false;
|
||||
switch (post.Visibility)
|
||||
{
|
||||
case PostVisibility.Public or PostVisibility.Unlisted:
|
||||
return true;
|
||||
case PostVisibility.Circle:
|
||||
var circle = string.IsNullOrEmpty(post.GroupId) ? default : await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token);
|
||||
return circle != default && MayReadCircle(circle, requester);
|
||||
case PostVisibility.Direct:
|
||||
return Addressed(post).Any(uri => Is(requester, uri));
|
||||
case PostVisibility.FollowersOnly:
|
||||
if (Addressed(post).Any(uri => Is(requester, uri)))
|
||||
return true;
|
||||
var followers = await _dbEntities.Followers
|
||||
.Match(f => f.LocalActorId == post.GroupUserId && f.LocalActorKind == LocalActorKind.Person && f.IsAccepted)
|
||||
.ExecuteAsync(token);
|
||||
return followers.Any(f => Is(requester, f.ActorURI));
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
static IEnumerable<string> Addressed(PostEntity post) =>
|
||||
post.To.Concat(post.Cc).Concat(post.Mentions.Select(m => m.ActorURI)).Where(uri => !string.IsNullOrEmpty(uri));
|
||||
|
||||
// the account itself, or the instance actor of the server it lives on
|
||||
static bool Is(ForeignAvatar requester, string actorUri) =>
|
||||
actorUri == requester.ActorURI || requester.AvatarType == AvatarType.Application && Origin.Same(actorUri, requester.ActorURI);
|
||||
|
||||
public static bool MayReadCircle(GroupEntity circle, ForeignAvatar requester) =>
|
||||
requester != default && circle.Members.Any(m => m.IsForeign
|
||||
&& (m.AvatarId == requester.ActorURI || requester.AvatarType == AvatarType.Application && Origin.Same(m.AvatarId, requester.ActorURI)));
|
||||
requester != default && circle.Members.Any(m => m.IsForeign && Is(requester, m.AvatarId));
|
||||
|
||||
public static bool MayReadConversation(Models.Group.DmGroup conversation, ForeignAvatar requester) =>
|
||||
requester != default && conversation.Members.Any(m => m.IsForeign && Is(requester, m.AvatarId));
|
||||
|
||||
// the members a refetch names in cc: the requesting member, or the members on an instance actor's server
|
||||
public static IReadOnlyList<string> CircleReaders(GroupEntity circle, ForeignAvatar requester) =>
|
||||
requester == default
|
||||
? Array.Empty<string>()
|
||||
: circle.Members.Where(m => m.IsForeign && Is(requester, m.AvatarId)).Select(m => m.AvatarId).ToList();
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Bson;
|
||||
using MongoDB.Bson;
|
||||
using MongoDB.Driver;
|
||||
using MongoDB.Entities;
|
||||
|
||||
@@ -54,6 +54,7 @@ namespace PrivaPub.Infrastructure.Data
|
||||
await Plain<Group>(token, g => g.InvitationCode);
|
||||
|
||||
await Plain<DmGroup>(token, g => g.ParticipantsKey);
|
||||
await Unique<GroupAnnouncement>(a => a.ActivityURI, Builders<GroupAnnouncement>.Filter.Type(a => a.ActivityURI, BsonType.String), token);
|
||||
|
||||
await Plain<Delivery>(token, d => d.DeliveredAt, d => d.AbandonedAt, d => d.NextAttemptAt);
|
||||
|
||||
@@ -130,6 +131,9 @@ namespace PrivaPub.Infrastructure.Data
|
||||
await DB.Default.Index<RemoteInstanceSnapshot>().Key(s => s.Host, KeyType.Ascending).Key(s => s.Week, KeyType.Ascending)
|
||||
.Option(o => o.Unique = true).CreateAsync(token);
|
||||
await Plain<RemoteInstanceSnapshot>(token, s => s.Week);
|
||||
await Plain<RemoteInstanceSnapshot>(token, s => s.CdnDomain, s => s.Week);
|
||||
await Plain<RemoteInstanceSnapshot>(token, s => s.Cdn, s => s.Week);
|
||||
await Plain<CdnRangeSet>(token, s => s.Key);
|
||||
await Plain<RemoteInstance>(token, i => i.Seen);
|
||||
await Plain<RemoteInstance>(token, i => i.LastCrawledAt);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Models.User;
|
||||
|
||||
namespace PrivaPub.Infrastructure.Data.Migrations
|
||||
{
|
||||
// Recovery codes were stored in plain text and never expired; from now on only a hash with a one-hour expiry is kept.
|
||||
// The old codes are dropped: anyone mid-recovery asks again.
|
||||
public class _011_recovery_codes_are_hashed : IMigration
|
||||
{
|
||||
public async Task UpgradeAsync() =>
|
||||
await DB.Default.DeleteAsync<EmailRecovery>(r => r.CodeHash == null);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
using MongoDB.Driver;
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Inbox;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.User;
|
||||
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Infrastructure.Data.Migrations
|
||||
{
|
||||
// replies_count now counts only public and unlisted replies, and accounts that deleted themselves leave nothing in the
|
||||
// counts (GoneActors). Both are recounted once for what was stored before.
|
||||
public class _012_counts_follow_one_rule : IMigration
|
||||
{
|
||||
public async Task UpgradeAsync()
|
||||
{
|
||||
var counted = (await DB.Default.Find<PostEntity, string>()
|
||||
.Match(p => p.AnsweringToPostId != null && !p.DeletedAt.HasValue && !p.AuthorGone
|
||||
&& (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted))
|
||||
.Project(p => p.AnsweringToPostId)
|
||||
.ExecuteAsync())
|
||||
.GroupBy(id => id)
|
||||
.ToDictionary(g => g.Key, g => g.Count());
|
||||
var stale = await DB.Default.Find<PostEntity>().Match(p => p.RepliesCount > 0).Project(p => p.Include(x => x.ID).Include(x => x.RepliesCount)).ExecuteAsync();
|
||||
foreach (var post in stale.Where(p => counted.GetValueOrDefault(p.ID) != p.RepliesCount))
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.RepliesCount, counted.GetValueOrDefault(post.ID)).ExecuteAsync();
|
||||
foreach (var (id, count) in counted.Where(c => stale.All(p => p.ID != c.Key)))
|
||||
await DB.Default.Update<PostEntity>().MatchID(id).Modify(p => p.RepliesCount, count).ExecuteAsync();
|
||||
|
||||
foreach (var gone in await DB.Default.Find<ForeignAvatar>().Match(a => a.DeletionAt != null && a.ForgottenAt == null).ExecuteAsync())
|
||||
await GoneActors.Forget(gone, CancellationToken.None);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
using System.Net;
|
||||
using System.Text.Json;
|
||||
|
||||
namespace PrivaPub.Infrastructure.Geo
|
||||
{
|
||||
// How a published list of a CDN's addresses is written.
|
||||
public enum CdnListFormat
|
||||
{
|
||||
Lines,//one CIDR per line (Cloudflare)
|
||||
StringArray,//a JSON array of addresses or CIDRs (Bunny)
|
||||
AddressesJson,//{"addresses": [...], "ipv6_addresses"/"addresses_v6": [...]} (Fastly, Gcore)
|
||||
AwsCloudFront,//AWS ip-ranges.json, CLOUDFRONT entries only
|
||||
ImpervaJson//{"ipRanges": [...], "ipv6Ranges": [...]}
|
||||
}
|
||||
|
||||
public sealed record CdnList(string Url, CdnListFormat Format, string Method = "GET", string Body = default);
|
||||
|
||||
// A CDN, or any edge network whose addresses say nothing about where a server runs. Domain names it in statistics.
|
||||
public sealed record CdnProvider(string Name, string Domain, int[] Asns, CdnList[] Lists, Func<HttpResponseMessage, bool> Fingerprint)
|
||||
{
|
||||
public string Key => Domain ?? Name.ToLowerInvariant();
|
||||
}
|
||||
|
||||
// owner decision (2026-10-04): PrivaPub finds CDNs by itself, three ways, best first. The address ranges a CDN
|
||||
// publishes (downloaded daily by CdnUpdater), the CDN's fingerprint in the responses PrivaPub already gets from a server
|
||||
// (EdgeHintsHandler), and the networks (ASN) that carry nothing but a CDN. A network shared with plain hosting (AWS,
|
||||
// Google, Microsoft, DataPacket) is never listed by ASN: only ranges or fingerprints name those CDNs.
|
||||
public static class CdnCatalog
|
||||
{
|
||||
public static readonly IReadOnlyList<CdnProvider> All = new CdnProvider[]
|
||||
{
|
||||
new("Cloudflare", "cloudflare.com", new[] { 13335, 209242 },
|
||||
new[] { new CdnList("https://www.cloudflare.com/ips-v4", CdnListFormat.Lines), new CdnList("https://www.cloudflare.com/ips-v6", CdnListFormat.Lines) },
|
||||
r => Has(r, "cf-ray") || ServerIs(r, "cloudflare")),
|
||||
new("Fastly", "fastly.com", new[] { 54113 },
|
||||
new[] { new CdnList("https://api.fastly.com/public-ip-list", CdnListFormat.AddressesJson) },
|
||||
r => Has(r, "x-fastly-request-id") || Has(r, "fastly-restarts") || Value(r, "x-served-by")?.StartsWith("cache-", StringComparison.OrdinalIgnoreCase) == true),
|
||||
new("Amazon CloudFront", "cloudfront.net", Array.Empty<int>(),
|
||||
new[] { new CdnList("https://ip-ranges.amazonaws.com/ip-ranges.json", CdnListFormat.AwsCloudFront) },
|
||||
r => Has(r, "x-amz-cf-id") || Has(r, "x-amz-cf-pop") || Value(r, "via")?.Contains("cloudfront", StringComparison.OrdinalIgnoreCase) == true),
|
||||
new("Akamai", "akamai.com", new[] { 20940, 16625, 16702, 21342 }, Array.Empty<CdnList>(),
|
||||
r => Has(r, "akamai-grn") || Has(r, "x-akamai-transformed") || Has(r, "akamai-cache-status") || ServerStarts(r, "AkamaiGHost") || ServerStarts(r, "AkamaiNetStorage")),
|
||||
new("Bunny", "bunny.net", new[] { 200325 },
|
||||
new[] { new CdnList("https://api.bunny.net/system/edgeserverlist", CdnListFormat.StringArray), new CdnList("https://api.bunny.net/system/edgeserverlist/IPv6", CdnListFormat.StringArray) },
|
||||
r => ServerStarts(r, "BunnyCDN") || Has(r, "cdn-pullzone") || Has(r, "cdn-requestid")),
|
||||
new("Gcore", "gcore.com", Array.Empty<int>(),
|
||||
new[] { new CdnList("https://api.gcore.com/cdn/public-ip-list", CdnListFormat.AddressesJson) },
|
||||
r => ServerStarts(r, "gcore")),
|
||||
new("Imperva", "imperva.com", new[] { 19551 },
|
||||
new[] { new CdnList("https://my.imperva.com/api/integration/v1/ips", CdnListFormat.ImpervaJson, "POST", "resp_format=json") },
|
||||
r => Has(r, "x-iinfo") || Value(r, "x-cdn") is "Imperva" or "Incapsula"),
|
||||
new("CDN77", "cdn77.com", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => Has(r, "x-77-nzt") || Has(r, "x-77-cache") || ServerStarts(r, "CDN77")),
|
||||
new("Edgio", "edg.io", new[] { 15133 }, Array.Empty<CdnList>(),
|
||||
r => ServerStarts(r, "ECAcc") || ServerStarts(r, "ECS (") || Has(r, "x-ec-custom-error")),
|
||||
new("Azure Front Door", "azure.microsoft.com", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => Has(r, "x-azure-ref")),
|
||||
new("Google Cloud", "cloud.google.com", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => Value(r, "via")?.Contains("google", StringComparison.OrdinalIgnoreCase) == true),
|
||||
new("Vercel", "vercel.com", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => Has(r, "x-vercel-id") || ServerIs(r, "Vercel")),
|
||||
new("Netlify", "netlify.com", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => Has(r, "x-nf-request-id") || ServerIs(r, "Netlify")),
|
||||
new("Sucuri", "sucuri.net", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => Has(r, "x-sucuri-id") || ServerStarts(r, "Sucuri")),
|
||||
new("DDoS-Guard", "ddos-guard.net", Array.Empty<int>(), Array.Empty<CdnList>(),
|
||||
r => ServerIs(r, "ddos-guard"))
|
||||
};
|
||||
|
||||
public static CdnProvider ByKey(string key) =>
|
||||
key == default ? default : All.FirstOrDefault(p => string.Equals(p.Key, key, StringComparison.OrdinalIgnoreCase) || string.Equals(p.Name, key, StringComparison.OrdinalIgnoreCase));
|
||||
|
||||
public static CdnProvider OfAsn(int? asn) => asn is { } number ? All.FirstOrDefault(p => p.Asns.Contains(number)) : default;
|
||||
|
||||
// The CDN a response passed through: a known fingerprint, or a CDN that names itself in X-CDN.
|
||||
public static CdnProvider FromResponse(HttpResponseMessage response)
|
||||
{
|
||||
if (response == default)
|
||||
return default;
|
||||
var known = All.FirstOrDefault(p => p.Fingerprint(response));
|
||||
if (known != default)
|
||||
return known;
|
||||
var named = Value(response, "x-cdn")?.Trim();
|
||||
return named is { Length: > 1 and <= 40 } && named.All(c => char.IsLetterOrDigit(c) || c is ' ' or '-' or '.' or '_')
|
||||
? new CdnProvider(named, default, Array.Empty<int>(), Array.Empty<CdnList>(), _ => false)
|
||||
: default;
|
||||
}
|
||||
|
||||
// The networks a published list holds; an entry that is not an address or a network is an error, never skipped.
|
||||
public static List<IPNetwork> Parse(CdnListFormat format, string body)
|
||||
{
|
||||
IEnumerable<string> entries = format switch
|
||||
{
|
||||
CdnListFormat.Lines => body.Split('\n', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries).Where(l => !l.StartsWith('#')),
|
||||
CdnListFormat.StringArray => Root(body),
|
||||
CdnListFormat.AddressesJson => Json(body, "addresses", "ipv6_addresses", "addresses_v6"),
|
||||
CdnListFormat.ImpervaJson => Json(body, "ipRanges", "ipv6Ranges"),
|
||||
CdnListFormat.AwsCloudFront => CloudFront(body),
|
||||
_ => Enumerable.Empty<string>()
|
||||
};
|
||||
return entries.Select(Network).ToList();
|
||||
}
|
||||
|
||||
static IPNetwork Network(string entry)
|
||||
{
|
||||
if (IPNetwork.TryParse(entry, out var network))
|
||||
return network;
|
||||
if (IPAddress.TryParse(entry, out var address))
|
||||
return new IPNetwork(address, address.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork ? 32 : 128);
|
||||
throw new FormatException($"not an address or a network: {entry}");
|
||||
}
|
||||
|
||||
static List<string> Root(string body)
|
||||
{
|
||||
using var document = JsonDocument.Parse(body);
|
||||
return Strings(document.RootElement);
|
||||
}
|
||||
|
||||
static IEnumerable<string> Json(string body, params string[] properties)
|
||||
{
|
||||
using var document = JsonDocument.Parse(body);
|
||||
var entries = new List<string>();
|
||||
foreach (var property in properties)
|
||||
if (document.RootElement.TryGetProperty(property, out var array))
|
||||
entries.AddRange(Strings(array));
|
||||
return entries;
|
||||
}
|
||||
|
||||
static IEnumerable<string> CloudFront(string body)
|
||||
{
|
||||
using var document = JsonDocument.Parse(body);
|
||||
var entries = new List<string>();
|
||||
foreach (var (array, field) in new[] { ("prefixes", "ip_prefix"), ("ipv6_prefixes", "ipv6_prefix") })
|
||||
if (document.RootElement.TryGetProperty(array, out var prefixes))
|
||||
foreach (var prefix in prefixes.EnumerateArray())
|
||||
if (prefix.TryGetProperty("service", out var service) && service.GetString() == "CLOUDFRONT" && prefix.TryGetProperty(field, out var value))
|
||||
entries.Add(value.GetString());
|
||||
return entries;
|
||||
}
|
||||
|
||||
static List<string> Strings(JsonElement array) =>
|
||||
array.ValueKind == JsonValueKind.Array ? array.EnumerateArray().Select(e => e.GetString()).ToList() : throw new FormatException("not a list");
|
||||
|
||||
static bool Has(HttpResponseMessage response, string header) =>
|
||||
response.Headers.Contains(header) || response.Content?.Headers.Contains(header) == true;
|
||||
|
||||
static string Value(HttpResponseMessage response, string header) =>
|
||||
response.Headers.TryGetValues(header, out var values) || response.Content?.Headers.TryGetValues(header, out values) == true
|
||||
? string.Join(", ", values)
|
||||
: default;
|
||||
|
||||
static bool ServerIs(HttpResponseMessage response, string name) => string.Equals(Value(response, "server")?.Trim(), name, StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
static bool ServerStarts(HttpResponseMessage response, string prefix) => Value(response, "server")?.TrimStart().StartsWith(prefix, StringComparison.OrdinalIgnoreCase) == true;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Models.Jobs;
|
||||
|
||||
using System.Net;
|
||||
|
||||
namespace PrivaPub.Infrastructure.Geo
|
||||
{
|
||||
public interface ICdnRanges
|
||||
{
|
||||
// the CDN whose published ranges hold the address, or default
|
||||
CdnProvider Of(IPAddress address);
|
||||
void Replace(CdnRangeSet set);
|
||||
Task Load(CancellationToken token);
|
||||
// per CDN key: how many ranges are known and when they were downloaded
|
||||
IReadOnlyDictionary<string, (int Ranges, DateTime FetchedAt)> Status { get; }
|
||||
}
|
||||
|
||||
// The published CDN ranges in memory, rebuilt from the stored CdnRangeSets at start and replaced set by set as
|
||||
// CdnUpdater downloads them. A few thousand networks: a linear scan per describe is cheap, and describes are weekly.
|
||||
public class CdnRanges : ICdnRanges
|
||||
{
|
||||
readonly object _gate = new();
|
||||
Dictionary<string, (CdnProvider Provider, IPNetwork[] Networks, DateTime FetchedAt)> _sets = new(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
public IReadOnlyDictionary<string, (int Ranges, DateTime FetchedAt)> Status
|
||||
{
|
||||
get
|
||||
{
|
||||
lock (_gate)
|
||||
return _sets.ToDictionary(s => s.Key, s => (s.Value.Networks.Length, s.Value.FetchedAt), StringComparer.OrdinalIgnoreCase);
|
||||
}
|
||||
}
|
||||
|
||||
public CdnProvider Of(IPAddress address)
|
||||
{
|
||||
if (address == default)
|
||||
return default;
|
||||
if (address.IsIPv4MappedToIPv6)
|
||||
address = address.MapToIPv4();
|
||||
Dictionary<string, (CdnProvider Provider, IPNetwork[] Networks, DateTime FetchedAt)> sets;
|
||||
lock (_gate)
|
||||
sets = _sets;
|
||||
foreach (var (provider, networks, _) in sets.Values)
|
||||
foreach (var network in networks)
|
||||
if (network.BaseAddress.AddressFamily == address.AddressFamily && network.Contains(address))
|
||||
return provider;
|
||||
return default;
|
||||
}
|
||||
|
||||
public void Replace(CdnRangeSet set)
|
||||
{
|
||||
var provider = CdnCatalog.ByKey(set.Key) ?? new CdnProvider(set.Name, set.Key, Array.Empty<int>(), Array.Empty<CdnList>(), _ => false);
|
||||
var networks = set.Ranges.Select(r => IPNetwork.TryParse(r, out var n) ? n : (IPNetwork?)null).Where(n => n != default).Select(n => n!.Value).ToArray();
|
||||
lock (_gate)
|
||||
_sets = new(_sets, StringComparer.OrdinalIgnoreCase) { [set.Key] = (provider, networks, set.FetchedAt) };
|
||||
}
|
||||
|
||||
public async Task Load(CancellationToken token)
|
||||
{
|
||||
foreach (var set in await DB.Default.Find<CdnRangeSet>().ExecuteAsync(token))
|
||||
Replace(set);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Infrastructure.Statistics;
|
||||
using PrivaPub.Models.Jobs;
|
||||
|
||||
using System.Text;
|
||||
|
||||
namespace PrivaPub.Infrastructure.Geo
|
||||
{
|
||||
// Keeps the CDNs' published address ranges current by itself (owner decision 2026-10-04): at start it loads the stored
|
||||
// ones, then once a day it downloads each list again. A list that fails to download or to parse, or comes back empty,
|
||||
// leaves the stored one in place and is tried the next day. Like GeoUpdater, plain HTTPS to fixed hosts, not federation.
|
||||
public sealed class CdnUpdater : BackgroundService
|
||||
{
|
||||
public const string ClientName = "cdn";
|
||||
static readonly TimeSpan FirstWait = TimeSpan.FromMinutes(1);
|
||||
static readonly TimeSpan Interval = TimeSpan.FromHours(24);
|
||||
const int MaxBytes = 16 * 1024 * 1024;
|
||||
const int MaxRanges = 200_000;
|
||||
|
||||
readonly IHttpClientFactory _http;
|
||||
readonly IOptionsMonitor<StatisticsOptions> _options;
|
||||
readonly ICdnRanges _ranges;
|
||||
readonly ILogger<CdnUpdater> _logger;
|
||||
|
||||
public CdnUpdater(IHttpClientFactory http, IOptionsMonitor<StatisticsOptions> options, ICdnRanges ranges, ILogger<CdnUpdater> logger)
|
||||
{
|
||||
_http = http;
|
||||
_options = options;
|
||||
_ranges = ranges;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
||||
{
|
||||
try
|
||||
{
|
||||
await _ranges.Load(stoppingToken);
|
||||
await Task.Delay(FirstWait, stoppingToken);
|
||||
while (!stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
if (_options.CurrentValue.Cdn.AutoUpdate)
|
||||
await Update(CdnCatalog.All, stoppingToken);
|
||||
await Task.Delay(Interval + TimeSpan.FromMinutes(Random.Shared.Next(60)), stoppingToken);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "{Service} stopped", nameof(CdnUpdater));
|
||||
}
|
||||
}
|
||||
|
||||
// The number of CDNs whose ranges were replaced.
|
||||
public async Task<int> Update(IEnumerable<CdnProvider> providers, CancellationToken token)
|
||||
{
|
||||
var replaced = 0;
|
||||
foreach (var provider in providers.Where(p => p.Lists.Length > 0))
|
||||
{
|
||||
try
|
||||
{
|
||||
var ranges = new List<string>();
|
||||
foreach (var list in provider.Lists)
|
||||
ranges.AddRange(CdnCatalog.Parse(list.Format, await Download(list, token)).Select(n => n.ToString()));
|
||||
ranges = ranges.Distinct().ToList();
|
||||
if (ranges.Count == 0 || ranges.Count > MaxRanges)
|
||||
throw new InvalidDataException($"{ranges.Count} ranges");
|
||||
|
||||
var set = await DB.Default.Find<CdnRangeSet>().Match(s => s.Key == provider.Key).ExecuteFirstAsync(token) ?? new CdnRangeSet { Key = provider.Key };
|
||||
set.Name = provider.Name;
|
||||
set.Ranges = ranges;
|
||||
set.Sources = provider.Lists.Select(l => l.Url).ToList();
|
||||
set.FetchedAt = DateTime.UtcNow;
|
||||
await DB.Default.SaveAsync(set, token);
|
||||
_ranges.Replace(set);
|
||||
replaced++;
|
||||
}
|
||||
catch (Exception ex) when (ex is HttpRequestException or InvalidDataException or FormatException or System.Text.Json.JsonException
|
||||
or TaskCanceledException && !token.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogWarning(ex, "Could not update the published ranges of {Cdn}", provider.Name);
|
||||
}
|
||||
}
|
||||
return replaced;
|
||||
}
|
||||
|
||||
async Task<string> Download(CdnList list, CancellationToken token)
|
||||
{
|
||||
using var request = new HttpRequestMessage(list.Method == "POST" ? HttpMethod.Post : HttpMethod.Get, list.Url);
|
||||
request.Headers.Accept.ParseAdd("application/json, text/plain;q=0.9");
|
||||
if (list.Body != default)
|
||||
request.Content = new StringContent(list.Body, Encoding.UTF8, "application/x-www-form-urlencoded");
|
||||
using var response = await _http.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
|
||||
response.EnsureSuccessStatusCode();
|
||||
if (response.Content.Headers.ContentLength > MaxBytes)
|
||||
throw new InvalidDataException($"more than {MaxBytes} bytes");
|
||||
await using var stream = await response.Content.ReadAsStreamAsync(token);
|
||||
using var buffer = new MemoryStream();
|
||||
var chunk = new byte[81920];
|
||||
int read;
|
||||
while ((read = await stream.ReadAsync(chunk, token)) > 0)
|
||||
{
|
||||
if (buffer.Length + read > MaxBytes)
|
||||
throw new InvalidDataException($"more than {MaxBytes} bytes");
|
||||
buffer.Write(chunk, 0, read);
|
||||
}
|
||||
return Encoding.UTF8.GetString(buffer.ToArray());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using MaxMind.Db;
|
||||
using MaxMind.Db;
|
||||
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
@@ -165,23 +165,4 @@ namespace PrivaPub.Infrastructure.Geo
|
||||
_asn?.Dispose();
|
||||
}
|
||||
}
|
||||
|
||||
public static class CdnNetworks
|
||||
{
|
||||
static readonly Dictionary<int, string> Known = new()
|
||||
{
|
||||
[13335] = "Cloudflare",
|
||||
[209242] = "Cloudflare",
|
||||
[54113] = "Fastly",
|
||||
[20940] = "Akamai",
|
||||
[16625] = "Akamai",
|
||||
[16702] = "Akamai",
|
||||
[21342] = "Akamai",
|
||||
[200325] = "Bunny",
|
||||
[60068] = "CDN77",
|
||||
[15133] = "Edgio"
|
||||
};
|
||||
|
||||
public static string Of(int? asn) => asn is { } number && Known.TryGetValue(number, out var name) ? name : default;
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using MaxMind.Db;
|
||||
using MaxMind.Db;
|
||||
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
@@ -11,8 +11,8 @@ namespace PrivaPub.Infrastructure.Geo
|
||||
{
|
||||
// Keeps the DB-IP Lite databases current by itself, so a deploy needs no timer and no root: once a day it checks
|
||||
// whether each database was built this month and, if not, fetches this month's (or, early in the month, last month's),
|
||||
// checks it opens as the right kind of database and swaps it in. This and SMTP are the server's only traffic that is not
|
||||
// federation, which is why it has its own client instead of IFederationHttp.
|
||||
// checks it opens as the right kind of database and swaps it in. This, CdnUpdater and SMTP are the server's only traffic
|
||||
// that is not federation, which is why each has its own client instead of IFederationHttp.
|
||||
public sealed class GeoUpdater : BackgroundService
|
||||
{
|
||||
public const string ClientName = "geo";
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
using PrivaPub.Infrastructure.Geo;
|
||||
|
||||
using System.Collections.Concurrent;
|
||||
|
||||
namespace PrivaPub.Infrastructure.Http
|
||||
{
|
||||
public interface IEdgeHints
|
||||
{
|
||||
void Remember(string host, CdnProvider cdn);
|
||||
CdnProvider Of(string host);
|
||||
}
|
||||
|
||||
// The CDN each remote host's responses came through, as their headers tell it (CdnCatalog fingerprints), for the
|
||||
// weekly description of the server. Like ConnectedAddresses, memory only: what a host answers is never stored per
|
||||
// request.
|
||||
public class EdgeHints : IEdgeHints
|
||||
{
|
||||
const int MaxHosts = 50_000;
|
||||
|
||||
readonly ConcurrentDictionary<string, CdnProvider> _hints = new(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
public void Remember(string host, CdnProvider cdn)
|
||||
{
|
||||
if (string.IsNullOrEmpty(host))
|
||||
return;
|
||||
if (cdn == default)
|
||||
{
|
||||
_hints.TryRemove(host, out _);
|
||||
return;
|
||||
}
|
||||
if (_hints.Count >= MaxHosts)
|
||||
_hints.Clear();
|
||||
_hints[host] = cdn;
|
||||
}
|
||||
|
||||
public CdnProvider Of(string host) => host != default && _hints.TryGetValue(host, out var cdn) ? cdn : default;
|
||||
}
|
||||
|
||||
// On the federation client: reads each response's CDN fingerprint for its host. A response without one forgets the
|
||||
// host's hint, so a server that leaves its CDN is seen as itself again.
|
||||
public class EdgeHintsHandler : DelegatingHandler
|
||||
{
|
||||
readonly IEdgeHints _hints;
|
||||
|
||||
public EdgeHintsHandler(IEdgeHints hints) => _hints = hints;
|
||||
|
||||
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
|
||||
{
|
||||
var response = await base.SendAsync(request, cancellationToken);
|
||||
if (request.RequestUri is { } uri && (int)response.StatusCode is >= 200 and < 500)
|
||||
_hints.Remember(uri.Host, CdnCatalog.FromResponse(response));
|
||||
return response;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,18 @@
|
||||
namespace PrivaPub.Infrastructure.Statistics
|
||||
namespace PrivaPub.Infrastructure.Statistics
|
||||
{
|
||||
public class StatisticsOptions
|
||||
{
|
||||
public bool Enabled { get; set; } = true;
|
||||
public string GeoDirectory { get; set; } = "/var/lib/privapub/geo";
|
||||
public int PublicCityMinUsers { get; set; } = 10;
|
||||
public CrawlerOptions Crawler { get; set; } = new();
|
||||
public GeoOptions Geo { get; set; } = new();
|
||||
public CdnOptions Cdn { get; set; } = new();
|
||||
}
|
||||
|
||||
//owner decision (2026-10-04): the server downloads the CDNs' published address ranges daily (CdnUpdater)
|
||||
public class CdnOptions
|
||||
{
|
||||
public bool AutoUpdate { get; set; } = true;
|
||||
}
|
||||
|
||||
//owner decision (2026-10-04): the server keeps its DB-IP Lite databases current by itself
|
||||
@@ -16,6 +22,16 @@ namespace PrivaPub.Infrastructure.Statistics
|
||||
public string DownloadBase { get; set; } = "https://download.db-ip.com/free";
|
||||
public long MaxDownloadBytes { get; set; } = 300L * 1024 * 1024;
|
||||
public long MaxDatabaseBytes { get; set; } = 1024L * 1024 * 1024;
|
||||
//this server's place, when locating its own address would be wrong (behind a proxy or a CDN): the owner's call
|
||||
public SelfGeoOptions Self { get; set; } = new();
|
||||
}
|
||||
|
||||
public class SelfGeoOptions
|
||||
{
|
||||
public double? Latitude { get; set; }
|
||||
public double? Longitude { get; set; }
|
||||
public string City { get; set; }
|
||||
public string Country { get; set; }
|
||||
}
|
||||
|
||||
//owner decision: off by default (on in production since 2026-10-04); when on, one server a minute, each at most weekly, at most MaxHosts servers
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using Microsoft.AspNetCore.Authentication.JwtBearer;
|
||||
using Microsoft.AspNetCore.Authentication.JwtBearer;
|
||||
using Microsoft.AspNetCore.ResponseCompression;
|
||||
|
||||
using OpenIddict.Validation.AspNetCore;
|
||||
@@ -61,9 +61,11 @@ namespace PrivaPub.Middleware
|
||||
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref} (+{baseAddress}/)");
|
||||
})
|
||||
.ConfigurePrimaryHttpMessageHandler(provider =>
|
||||
SafeHttpHandlerFactory.Create(provider.GetRequiredService<IOptions<FederationOptions>>().Value, provider.GetRequiredService<IConnectedAddresses>()));
|
||||
SafeHttpHandlerFactory.Create(provider.GetRequiredService<IOptions<FederationOptions>>().Value, provider.GetRequiredService<IConnectedAddresses>()))
|
||||
.AddHttpMessageHandler(provider => new EdgeHintsHandler(provider.GetRequiredService<IEdgeHints>()));
|
||||
return service
|
||||
.AddSingleton<IConnectedAddresses, ConnectedAddresses>()
|
||||
.AddSingleton<IEdgeHints, EdgeHints>()
|
||||
.AddSingleton<IFederationHttp, FederationHttp>()
|
||||
.AddSingleton<IDomainBlocks, DomainBlocks>()
|
||||
.AddSingleton<IContentRenderer, ContentRenderer>()
|
||||
@@ -99,6 +101,8 @@ namespace PrivaPub.Middleware
|
||||
.AddSingleton<IPollService, PollService>()
|
||||
.AddSingleton<IQuoteService, QuoteService>()
|
||||
.AddSingleton<IJobHandler, PollRefreshJob>()
|
||||
.AddSingleton<IJobHandler, RecoveryJob>()
|
||||
.AddSingleton<IJobHandler, Federation.Actors.AccountCountsJob>()
|
||||
.AddSingleton<IJobHandler, PollCloseJob>()
|
||||
.AddSingleton<InstanceDescriber>()
|
||||
.AddSingleton<IJobHandler>(services => services.GetRequiredService<InstanceDescriber>())
|
||||
@@ -117,6 +121,13 @@ namespace PrivaPub.Middleware
|
||||
client.Timeout = TimeSpan.FromMinutes(15);
|
||||
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref} (+{baseAddress}/)");
|
||||
});
|
||||
// not federation either: the CDNs' published address ranges (CdnUpdater), plain HTTPS to the CDNs' own hosts
|
||||
service.AddHttpClient(CdnUpdater.ClientName, (provider, client) =>
|
||||
{
|
||||
var baseAddress = provider.GetRequiredService<IOptionsMonitor<AppConfiguration>>().CurrentValue.BackendBaseAddress?.TrimEnd('/');
|
||||
client.Timeout = TimeSpan.FromMinutes(2);
|
||||
client.DefaultRequestHeaders.UserAgent.ParseAdd($"PrivaPub/{BuildInfo.Ref} (+{baseAddress}/)");
|
||||
});
|
||||
return service
|
||||
.Configure<StatisticsOptions>(configuration.GetSection("Statistics"))
|
||||
.AddSingleton<InteractionSalts>()
|
||||
@@ -125,6 +136,10 @@ namespace PrivaPub.Middleware
|
||||
.AddHostedService(services => services.GetRequiredService<InteractionLedger>())
|
||||
.AddSingleton<IJobHandler, RollupJob>()
|
||||
.AddSingleton<IGeoLocator, DbIpLocator>()
|
||||
.AddSingleton<ICdnRanges, CdnRanges>()
|
||||
.AddHostedService<CdnUpdater>()
|
||||
.AddSingleton<Domain.Statistics.ISelfLocation, Domain.Statistics.SelfLocation>()
|
||||
.AddSingleton<Domain.Statistics.IServerPlaces, Domain.Statistics.ServerPlaces>()
|
||||
.AddHostedService<GeoUpdater>()
|
||||
.AddSingleton<Domain.Statistics.StatisticsQueries>()
|
||||
.AddSingleton<IJobHandler, Federation.Crawler.CrawlPlanner>()
|
||||
@@ -182,6 +197,8 @@ namespace PrivaPub.Middleware
|
||||
return service
|
||||
.AddTransient<IDataService, DataService>()
|
||||
.AddTransient<IRootUsersService, RootUsersService>()
|
||||
.AddScoped<IRootSessions, RootSessions>()
|
||||
.AddScoped<IRootRemoval, RootRemoval>()
|
||||
.AddTransient<IPublicAvatarUsersService, PublicAvatarUsersService>()
|
||||
.AddTransient<IPrivateAvatarUsersService, PrivateAvatarUsersService>()
|
||||
.AddTransient<IGroupUsersService, GroupUsersService>()
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
namespace PrivaPub.Models.Federation
|
||||
{
|
||||
// An Announce a community sent (GroupDistributor), kept so its id resolves as both docs promise: announces are public,
|
||||
// unlike follows, likes and blocks. Served only while the post it is about is still shown.
|
||||
public class GroupAnnouncement : Entity
|
||||
{
|
||||
public string GroupId { get; set; }
|
||||
public string ActivityURI { get; set; }
|
||||
public string ObjectURI { get; set; }//the post it is about
|
||||
public string Body { get; set; }//the Announce as sent, without @context
|
||||
public DateTime PublishedAt { get; set; } = DateTime.UtcNow;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
namespace PrivaPub.Models.Jobs
|
||||
{
|
||||
// The address ranges one CDN publishes, as CdnUpdater last downloaded them: kept, so a restart needs no download and a
|
||||
// failed download keeps the previous list.
|
||||
public class CdnRangeSet : Entity
|
||||
{
|
||||
public string Key { get; set; }//the CDN's domain (CdnProvider.Key)
|
||||
public string Name { get; set; }
|
||||
public List<string> Ranges { get; set; } = new();
|
||||
public List<string> Sources { get; set; } = new();
|
||||
public DateTime FetchedAt { get; set; }
|
||||
}
|
||||
}
|
||||
@@ -29,7 +29,9 @@ namespace PrivaPub.Models.Jobs
|
||||
FetchPreview,
|
||||
RollupDay,
|
||||
CrawlPlan,
|
||||
CrawlInstance
|
||||
CrawlInstance,
|
||||
SendRecovery,
|
||||
CountAccount
|
||||
}
|
||||
|
||||
public enum JobState
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Entities;
|
||||
using MongoDB.Entities;
|
||||
|
||||
namespace PrivaPub.Models.Jobs
|
||||
{
|
||||
@@ -57,6 +57,8 @@ namespace PrivaPub.Models.Jobs
|
||||
public int? Asn { get; set; }
|
||||
public string AsnOrg { get; set; }
|
||||
public string Cdn { get; set; }//the CDN in front of it, whose edge address says nothing about where it runs
|
||||
public string CdnDomain { get; set; }//the CDN's own domain, which groups its servers (CdnProvider.Key)
|
||||
public string CdnSource { get; set; }//how the CDN was found: "ranges", "headers" or "asn"
|
||||
public bool? Ipv6 { get; set; }
|
||||
public string Source { get; set; }
|
||||
public DateTime? LocatedAt { get; set; }
|
||||
@@ -79,8 +81,13 @@ namespace PrivaPub.Models.Jobs
|
||||
public string RegistrationMode { get; set; }
|
||||
public List<string> Languages { get; set; }
|
||||
public string Country { get; set; }
|
||||
public string City { get; set; }
|
||||
public double? Latitude { get; set; }
|
||||
public double? Longitude { get; set; }
|
||||
public int? Asn { get; set; }
|
||||
public string Cdn { get; set; }
|
||||
public string CdnDomain { get; set; }
|
||||
public string CdnSource { get; set; }
|
||||
public bool? Ipv6 { get; set; }
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
using MongoDB.Bson.Serialization.Attributes;
|
||||
using MongoDB.Bson.Serialization.Attributes;
|
||||
using MongoDB.Entities;
|
||||
|
||||
namespace PrivaPub.Models.Post
|
||||
@@ -39,6 +39,8 @@ namespace PrivaPub.Models.Post
|
||||
[BsonIgnoreIfNull]
|
||||
public EventDetails Event { get; set; }
|
||||
[BsonIgnoreIfNull]
|
||||
public PostPlace Place { get; set; }
|
||||
[BsonIgnoreIfNull]
|
||||
public PostPoll Poll { get; set; }
|
||||
public List<float> Location { get; set; } = new();
|
||||
public float RangeKm { get; set; } = 5.0f;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
namespace PrivaPub.Models.Post
|
||||
namespace PrivaPub.Models.Post
|
||||
{
|
||||
public class CustomEmoji
|
||||
{
|
||||
@@ -104,6 +104,15 @@ namespace PrivaPub.Models.Post
|
||||
public string Category { get; set; }
|
||||
}
|
||||
|
||||
//a remote post's own place (Pixelfed's location): shown, never federated again
|
||||
public class PostPlace
|
||||
{
|
||||
public string Name { get; set; }
|
||||
public double Latitude { get; set; }
|
||||
public double Longitude { get; set; }
|
||||
public string Country { get; set; }
|
||||
}
|
||||
|
||||
public class EventPlace
|
||||
{
|
||||
public string Name { get; set; }
|
||||
|
||||
@@ -87,6 +87,11 @@ namespace PrivaPub.Models.User
|
||||
public DateTime? SuspendedAt { get; set; }
|
||||
public DateTime? BannedAt { get; set; }
|
||||
public DateTime? DeletionAt { get; set; }
|
||||
public DateTime? ForgottenAt { get; set; }//its likes, votes, boosts and replies were taken out of our counts (GoneActors)
|
||||
public int? FollowersCount { get; set; }//its own collections' totalItems, read at most daily (AccountCountsJob)
|
||||
public int? FollowingCount { get; set; }
|
||||
public int? StatusesCount { get; set; }
|
||||
public DateTime? CountedAt { get; set; }
|
||||
}
|
||||
|
||||
public class AvatarSettings
|
||||
|
||||
@@ -5,7 +5,9 @@ namespace PrivaPub.Models.User
|
||||
public class EmailRecovery : Entity
|
||||
{
|
||||
public string RootUserId { get; set; }
|
||||
public string RecoveryCode { get; set; }
|
||||
public string RecoveryCode { get; set; }//plaintext codes before 2026-10-04; migration _011 removed them, never written now
|
||||
public string CodeHash { get; set; }//sha-256 of the code, hex; the code itself exists only in the email
|
||||
public DateTime ExpiresAt { get; set; }
|
||||
public string RequestIP { get; set; }
|
||||
public DateTime CreationDate { get; set; } = DateTime.UtcNow;
|
||||
}
|
||||
|
||||
@@ -23,6 +23,8 @@ namespace PrivaPub.Models.User
|
||||
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
|
||||
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
|
||||
public DateTime? DeletedAt { get; set; }
|
||||
public DateTime? CredentialsChangedAt { get; set; }//when RootSessions last ended every session
|
||||
public string SessionStamp { get; set; }//carried by every /clientapi token; a new one ends the tokens issued before
|
||||
}
|
||||
|
||||
public class ContactItem
|
||||
|
||||
+2
-1
@@ -1,4 +1,4 @@
|
||||
using Microsoft.AspNetCore.HttpOverrides;
|
||||
using Microsoft.AspNetCore.HttpOverrides;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
using MongoDB.Bson;
|
||||
@@ -186,6 +186,7 @@ try
|
||||
var passwordHasher = app.Services.GetService(typeof(IPasswordHasher)) as IPasswordHasher;
|
||||
await dbClient.Init(passwordHasher);
|
||||
await app.Services.GetRequiredService<PrivaPub.Federation.Moderation.IDomainBlocks>().Reload(CancellationToken.None);
|
||||
await PrivaPub.Api.Mastodon.Auth.PersonaExchange.EnsureFirstPartyClient(app.Services, CancellationToken.None);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
|
||||
@@ -210,6 +210,7 @@ namespace PrivaPub.Services
|
||||
group.UpdatedAt = DateTime.UtcNow;
|
||||
await DB.Default.SaveAsync(group, token);
|
||||
}
|
||||
await FollowAsMember(group, form.AvatarId, token);
|
||||
|
||||
result.Data = ToView(group, form.AvatarId);
|
||||
return result;
|
||||
@@ -221,6 +222,33 @@ namespace PrivaPub.Services
|
||||
}
|
||||
}
|
||||
|
||||
// Joining by invitation is following, as a local follow of the group would be: the member counts among its
|
||||
// followers (so /groupies and /flock agree) and gets its posts in the home timeline.
|
||||
async Task FollowAsMember(GroupEntity group, string avatarId, CancellationToken token)
|
||||
{
|
||||
var persona = await _localActors.FindById(LocalActorKind.Person, avatarId, token);
|
||||
var target = _localActors.FromGroup(group);
|
||||
if (persona == default)
|
||||
return;
|
||||
await DB.Default.Update<Follower>()
|
||||
.Match(f => f.LocalActorId == group.ID && f.LocalActorKind == LocalActorKind.Group && f.ActorURI == persona.Uri)
|
||||
.Modify(f => f.InboxURL, persona.Inbox)
|
||||
.Modify(f => f.SharedInboxURL, persona.SharedInbox)
|
||||
.Modify(f => f.IsAccepted, true)
|
||||
.Modify(b => b.SetOnInsert(f => f.CreationDate, DateTime.UtcNow))
|
||||
.Option(o => o.IsUpsert = true)
|
||||
.ExecuteAsync(token);
|
||||
await DB.Default.Update<Models.Social.Following>()
|
||||
.Match(f => f.AvatarId == persona.Id && f.TargetActorURI == target.Uri)
|
||||
.Modify(f => f.TargetAccountId, group.ID)
|
||||
.Modify(f => f.TargetIsLocal, true)
|
||||
.Modify(f => f.TargetInboxURL, target.Inbox)
|
||||
.Modify(f => f.State, Models.Social.FollowState.Accepted)
|
||||
.Modify(b => b.SetOnInsert(f => f.CreatedAt, DateTime.UtcNow))
|
||||
.Option(o => o.IsUpsert = true)
|
||||
.ExecuteAsync(token);
|
||||
}
|
||||
|
||||
public async Task<WebResult> LeaveGroup(string rootUserId, GroupMembershipForm form, CancellationToken token)
|
||||
{
|
||||
var result = new WebResult();
|
||||
@@ -238,6 +266,12 @@ namespace PrivaPub.Services
|
||||
group.Members.RemoveAll(m => !m.IsForeign && m.AvatarId == form.AvatarId);
|
||||
group.UpdatedAt = DateTime.UtcNow;
|
||||
await DB.Default.SaveAsync(group, token);
|
||||
// leaving is unfollowing, as joining was following
|
||||
var groupUri = _localActors.FromGroup(group).Uri;
|
||||
var personaUri = (await _localActors.FindById(LocalActorKind.Person, form.AvatarId, token))?.Uri;
|
||||
await DB.Default.DeleteAsync<Models.Social.Following>(f => f.AvatarId == form.AvatarId && f.TargetActorURI == groupUri);
|
||||
if (personaUri != default)
|
||||
await DB.Default.DeleteAsync<Follower>(f => f.LocalActorId == group.ID && f.LocalActorKind == LocalActorKind.Group && f.ActorURI == personaUri);
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -16,6 +16,7 @@ namespace PrivaPub.Services
|
||||
Task<WebResult> UnbanUserAsync(UsersIds usersIds);
|
||||
|
||||
Task<WebResult> RemoveUserAsync(UsersIds usersIds);
|
||||
Task<WebResult> RemoveSelfAsync(string rootId, string password);
|
||||
|
||||
Task<WebResult> UpdateUserAsync(UserForm userEmailForm, string userId);
|
||||
|
||||
|
||||
@@ -28,9 +28,10 @@ namespace PrivaPub.Services
|
||||
? default
|
||||
: await context.HttpContext.RequestServices.GetRequiredService<DbEntities>().RootUsers.MatchID(rootId)
|
||||
.ExecuteFirstAsync(context.HttpContext.RequestAborted);
|
||||
if (root is not { IsBanned: false, DeletedAt: null })
|
||||
var refusal = RootJwt.Refusal(root, context.Principal);
|
||||
if (refusal != default)
|
||||
{
|
||||
context.Fail("The account can no longer be used.");
|
||||
context.Fail(refusal);
|
||||
return;
|
||||
}
|
||||
if (context.Principal.Identity is not ClaimsIdentity identity)
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
using MailKit.Net.Smtp;
|
||||
|
||||
using Microsoft.Extensions.Localization;
|
||||
|
||||
using MimeKit;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Infrastructure.Jobs;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.Resources;
|
||||
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
|
||||
namespace PrivaPub.Services
|
||||
{
|
||||
public sealed record RecoveryPayload(string RootUserId, string Host);
|
||||
|
||||
// Sends a password recovery link. Every request queues one of these, for an account that exists or not, so the answer
|
||||
// to the request and its timing say nothing; SMTP's slowness and failures happen here, where nobody waits on them.
|
||||
// The code exists only in the email: the database keeps its hash, for an hour.
|
||||
public class RecoveryJob : IJobHandler
|
||||
{
|
||||
public const string Host = "recovery";
|
||||
public static readonly TimeSpan CodeLifetime = TimeSpan.FromHours(1);
|
||||
|
||||
readonly AppConfigurationService _app;
|
||||
readonly IStringLocalizer<GenericRes> _localizer;
|
||||
readonly ILogger<RecoveryJob> _logger;
|
||||
|
||||
public RecoveryJob(AppConfigurationService app, IStringLocalizer<GenericRes> localizer, ILogger<RecoveryJob> logger)
|
||||
{
|
||||
_app = app;
|
||||
_localizer = localizer;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public JobKind Kind => JobKind.SendRecovery;
|
||||
public int Concurrency => 1;
|
||||
public int MaxAttempts => 3;
|
||||
public int PerHostLimit => 1;
|
||||
|
||||
public static string Hash(string code) => Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(code ?? string.Empty)));
|
||||
|
||||
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
|
||||
{
|
||||
var payload = JsonSerializer.Deserialize<RecoveryPayload>(job.Payload);
|
||||
if (string.IsNullOrEmpty(payload?.RootUserId))
|
||||
return JobOutcome.Done;//asked for an account that does not exist: there is nobody to write to
|
||||
var user = await DB.Default.Find<RootUser>().MatchID(payload.RootUserId).ExecuteFirstAsync(token);
|
||||
if (user is not { DeletedAt: null, IsBanned: false } || string.IsNullOrEmpty(user.Email))
|
||||
return JobOutcome.Done;
|
||||
|
||||
var code = Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(32));
|
||||
await DB.Default.DeleteAsync<EmailRecovery>(r => r.RootUserId == user.ID);
|
||||
await DB.Default.SaveAsync(new EmailRecovery { RootUserId = user.ID, CodeHash = Hash(code), ExpiresAt = DateTime.UtcNow + CodeLifetime }, token);
|
||||
|
||||
var email = _app.AppConfiguration.EmailConfiguration;
|
||||
var message = new MimeMessage();
|
||||
message.From.Add(new MailboxAddress("PrivaPub", email.SmtpUsername));
|
||||
message.To.Add(MailboxAddress.Parse(user.Email));
|
||||
message.Subject = _localizer["PrivaPub - Password recovery link"];
|
||||
message.Body = new TextPart("plain")
|
||||
{
|
||||
Text = string.Format(_localizer[
|
||||
"Hello,\n\nSomeone asked to reset the password of the PrivaPub login {0}. If it was you, open this link within an hour:\n{1}\n\nIf it was not you, ignore this email: nothing changes."],
|
||||
user.UserName, $"{payload.Host}/password-recovery?rc={code}")
|
||||
};
|
||||
try
|
||||
{
|
||||
using var smtp = new SmtpClient();
|
||||
await smtp.ConnectAsync(email.SmtpServer, email.SmtpPort, email.UseSSL, token);
|
||||
await smtp.AuthenticateAsync(email.SmtpUsername, email.SmtpPassword, token);
|
||||
await smtp.SendAsync(message, token);
|
||||
await smtp.DisconnectAsync(quit: true, token);
|
||||
return JobOutcome.Done;
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException || !token.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogWarning(ex, "The recovery email could not be sent");
|
||||
return JobOutcome.Retry("smtp");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
using Microsoft.IdentityModel.JsonWebTokens;
|
||||
using Microsoft.IdentityModel.Tokens;
|
||||
|
||||
using PrivaPub.Extensions;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Security.Claims;
|
||||
using System.Text;
|
||||
|
||||
namespace PrivaPub.Services
|
||||
{
|
||||
// The /clientapi token: how it is validated (JwtBearer through AddPrivaPubAuth, and the persona token exchange on
|
||||
// /oauth/token), and when a valid one can no longer be used. One place, so the two never drift apart.
|
||||
public static class RootJwt
|
||||
{
|
||||
public static TokenValidationParameters Parameters(IConfiguration configuration) => new()
|
||||
{
|
||||
ValidateIssuer = true,
|
||||
ValidateAudience = true,
|
||||
ValidateLifetime = true,
|
||||
ValidateIssuerSigningKey = true,
|
||||
ValidIssuer = configuration["AppConfiguration:Jwt:Issuer"],
|
||||
ValidAudience = configuration["AppConfiguration:Jwt:Audience"],
|
||||
IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(configuration["AppConfiguration:Jwt:Key"]))
|
||||
};
|
||||
|
||||
//why a token that validated can no longer be used, or default when it can: a token outlives a ban, a removal and a
|
||||
//password recovery (RootSessions gives the root a new stamp), so the database has the last word
|
||||
public static string Refusal(RootUser root, ClaimsPrincipal principal)
|
||||
{
|
||||
if (root is not { IsBanned: false, DeletedAt: null })
|
||||
return "The account can no longer be used.";
|
||||
if ((root.SessionStamp ?? string.Empty) != (principal.FindFirst(AuthTokenManager.SessionStamp)?.Value ?? string.Empty))
|
||||
return "The account's sessions were ended.";
|
||||
return default;
|
||||
}
|
||||
|
||||
//the root a token belongs to, when the token is valid and can still be used; default otherwise, never an exception
|
||||
public static async Task<RootUser> Root(string jwt, IConfiguration configuration, DbEntities dbEntities, CancellationToken token)
|
||||
{
|
||||
if (string.IsNullOrEmpty(jwt))
|
||||
return default;
|
||||
var result = await new JsonWebTokenHandler { MapInboundClaims = false }.ValidateTokenAsync(jwt, Parameters(configuration));
|
||||
if (!result.IsValid)
|
||||
return default;
|
||||
var principal = new ClaimsPrincipal(result.ClaimsIdentity);
|
||||
var rootId = principal.GetUserId();
|
||||
var root = string.IsNullOrEmpty(rootId) ? default : await dbEntities.RootUsers.MatchID(rootId).ExecuteFirstAsync(token);
|
||||
return Refusal(root, principal) == default ? root : default;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Outbox;
|
||||
using PrivaPub.Federation.Rendering;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Social;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
using GroupEntity = PrivaPub.Models.Group.Group;
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Services
|
||||
{
|
||||
public interface IRootRemoval
|
||||
{
|
||||
Task<bool> Remove(string rootId, CancellationToken token);
|
||||
}
|
||||
|
||||
// Deleting a root deletes everything public it had (owner decision 2026-10-04): each persona, and each group a persona
|
||||
// owns, is announced deleted with a Delete of the actor to everyone who follows it, every member and everyone it
|
||||
// follows; their posts are emptied; their names stay reserved, and their documents answer 410 from then on. The root's
|
||||
// sessions end first, so nothing acts as it while it goes.
|
||||
public class RootRemoval : IRootRemoval
|
||||
{
|
||||
readonly DbEntities _dbEntities;
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly IDeliveryService _delivery;
|
||||
readonly IRootSessions _sessions;
|
||||
|
||||
public RootRemoval(DbEntities dbEntities, ILocalActorService localActors, IDeliveryService delivery, IRootSessions sessions)
|
||||
{
|
||||
_dbEntities = dbEntities;
|
||||
_localActors = localActors;
|
||||
_delivery = delivery;
|
||||
_sessions = sessions;
|
||||
}
|
||||
|
||||
public async Task<bool> Remove(string rootId, CancellationToken token)
|
||||
{
|
||||
var root = await _dbEntities.RootUsers.MatchID(rootId).Match(u => u.DeletedAt == null).ExecuteFirstAsync(token);
|
||||
if (root == default)
|
||||
return false;
|
||||
await _sessions.Revoke(root.ID, token);
|
||||
var now = DateTime.UtcNow;
|
||||
|
||||
var avatarIds = (await _dbEntities.RootToAvatars.Match(ra => ra.RootId == root.ID).ExecuteAsync(token)).Select(ra => ra.AvatarId).ToList();
|
||||
var avatars = await _dbEntities.Avatars.Match(a => avatarIds.Contains(a.ID) && !a.DeletionAt.HasValue).ExecuteAsync(token);
|
||||
foreach (var avatar in avatars)
|
||||
{
|
||||
foreach (var group in await _dbEntities.Groups.Match(g => g.OwnerAvatarId == avatar.ID && !g.DeletionAt.HasValue).ExecuteAsync(token))
|
||||
{
|
||||
await Announce(_localActors.FromGroup(group), group.Members.Where(m => m.IsForeign).Select(m => m.AvatarId), token);
|
||||
await DB.Default.Update<GroupEntity>().MatchID(group.ID).Modify(g => g.DeletionAt, now).ExecuteAsync(token);
|
||||
}
|
||||
var persona = _localActors.FromAvatar(avatar);
|
||||
var followed = (await _dbEntities.Followings.Match(f => f.AvatarId == avatar.ID && !f.TargetIsLocal).ExecuteAsync(token))
|
||||
.Select(f => f.TargetActorURI);
|
||||
await Announce(persona, followed, token);
|
||||
await DB.Default.Update<Avatar>().MatchID(avatar.ID).Modify(a => a.DeletionAt, now).ExecuteAsync(token);
|
||||
await Empty(avatar.ID, now, token);
|
||||
}
|
||||
|
||||
await DB.Default.Update<RootUser>().MatchID(root.ID)
|
||||
.Modify(u => u.UserName, $"deleted-{root.ID}")//unique, so a second deletion never collides with the first
|
||||
.Modify(u => u.Email, null)
|
||||
.Modify(u => u.HashedPassword, null)
|
||||
.Modify(u => u.Policies, new List<string>())
|
||||
.Modify(u => u.IsBanned, false)
|
||||
.Modify(u => u.IsEmailValidated, false)
|
||||
.Modify(u => u.DeletedAt, now)
|
||||
.ExecuteAsync(token);
|
||||
await DB.Default.DeleteAsync<EmailRecovery>(r => r.RootUserId == root.ID);
|
||||
return true;
|
||||
}
|
||||
|
||||
// Delete{Actor}: to its followers' (shared) inboxes, and to the inboxes of the other accounts named
|
||||
async Task Announce(LocalActor actor, IEnumerable<string> others, CancellationToken token)
|
||||
{
|
||||
var inboxes = (await _delivery.FollowerInboxes(actor, token)).ToList();
|
||||
var named = others.Where(uri => !string.IsNullOrEmpty(uri)).Distinct().ToList();
|
||||
if (named.Count > 0)
|
||||
inboxes.AddRange((await _dbEntities.ForeignAvatars.Match(a => named.Contains(a.ActorURI)).ExecuteAsync(token))
|
||||
.Select(a => string.IsNullOrEmpty(a.SharedInboxURL) ? a.InboxURL : a.SharedInboxURL));
|
||||
await _delivery.Enqueue(actor, inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(), new JsonObject
|
||||
{
|
||||
["@context"] = ActivityPubRenderer.Context(),
|
||||
["id"] = actor.ActivityUri("delete-actor"),
|
||||
["type"] = "Delete",
|
||||
["actor"] = actor.Uri,
|
||||
["object"] = actor.Uri,
|
||||
["to"] = new JsonArray(ActivityPubRenderer.Public),
|
||||
["cc"] = new JsonArray(actor.Followers)
|
||||
}, token);
|
||||
}
|
||||
|
||||
// a persona's posts keep their ids and lose their content, like a single deleted post; a group writes none of its own
|
||||
static async Task Empty(string avatarId, DateTime now, CancellationToken token)
|
||||
{
|
||||
var postIds = (await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == avatarId && !p.IsFederatedCopy && !p.DeletedAt.HasValue)
|
||||
.Project(p => p.Include(x => x.ID)).ExecuteAsync(token)).Select(p => p.ID).ToList();
|
||||
if (postIds.Count == 0)
|
||||
return;
|
||||
await DB.Default.Update<PostEntity>().Match(p => postIds.Contains(p.ID))
|
||||
.Modify(p => p.DeletedAt, now)
|
||||
.Modify(p => p.Text, null)
|
||||
.Modify(p => p.ContentHtml, null)
|
||||
.Modify(p => p.Title, null)
|
||||
.Modify(p => p.SpoilerText, null)
|
||||
.Modify(p => p.Media, new List<Models.Post.PostMedia>())
|
||||
.Modify(p => p.Revisions, new List<Models.Post.PostRevision>())
|
||||
.ExecuteAsync(token);
|
||||
await DB.Default.DeleteAsync<TimelineEntry>(e => postIds.Contains(e.PostId) || postIds.Contains(e.ReblogOfPostId));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using OpenIddict.Abstractions;
|
||||
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
namespace PrivaPub.Services
|
||||
{
|
||||
public interface IRootSessions
|
||||
{
|
||||
Task Revoke(string rootId, CancellationToken token);
|
||||
}
|
||||
|
||||
// Ends everything a root is signed in with: its /clientapi tokens, through a new SessionStamp (checked by JwtEvents),
|
||||
// and the Mastodon API tokens and authorizations of each of its personas. Used when its password is recovered and when
|
||||
// it is deleted.
|
||||
public class RootSessions : IRootSessions
|
||||
{
|
||||
readonly DbEntities _dbEntities;
|
||||
readonly IOpenIddictTokenManager _tokens;
|
||||
readonly IOpenIddictAuthorizationManager _authorizations;
|
||||
|
||||
public RootSessions(DbEntities dbEntities, IOpenIddictTokenManager tokens, IOpenIddictAuthorizationManager authorizations)
|
||||
{
|
||||
_dbEntities = dbEntities;
|
||||
_tokens = tokens;
|
||||
_authorizations = authorizations;
|
||||
}
|
||||
|
||||
public async Task Revoke(string rootId, CancellationToken token)
|
||||
{
|
||||
await DB.Default.Update<RootUser>().MatchID(rootId)
|
||||
.Modify(u => u.CredentialsChangedAt, DateTime.UtcNow)
|
||||
.Modify(u => u.SessionStamp, Guid.NewGuid().ToString("N"))
|
||||
.ExecuteAsync(token);
|
||||
foreach (var link in await _dbEntities.RootToAvatars.Match(ra => ra.RootId == rootId).ExecuteAsync(token))
|
||||
{
|
||||
await _tokens.RevokeBySubjectAsync(link.AvatarId, token);
|
||||
await _authorizations.RevokeBySubjectAsync(link.AvatarId, token);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -14,9 +14,12 @@ using PrivaPub.ClientModels.User;
|
||||
using PrivaPub.Models;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.Resources;
|
||||
using PrivaPub.Infrastructure.Jobs;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Globalization;
|
||||
using System.Text.Json;
|
||||
|
||||
#pragma warning disable 8603
|
||||
#pragma warning disable 8625
|
||||
@@ -31,8 +34,14 @@ namespace PrivaPub.Services
|
||||
readonly ILogger<RootUsersService> Logger;
|
||||
readonly AppConfigurationService AppConfigurationService;
|
||||
readonly AuthTokenManager AuthTokenManager;
|
||||
readonly IJobQueue Jobs;
|
||||
readonly IRootSessions Sessions;
|
||||
readonly IRootRemoval Removal;
|
||||
|
||||
public RootUsersService(
|
||||
IJobQueue jobs,
|
||||
IRootSessions sessions,
|
||||
IRootRemoval removal,
|
||||
IStringLocalizer<GenericRes> localizer,
|
||||
ILogger<RootUsersService> logger,
|
||||
IPasswordHasher passwordHasher,
|
||||
@@ -40,6 +49,9 @@ namespace PrivaPub.Services
|
||||
AppConfigurationService appConfigurationService,
|
||||
AuthTokenManager authTokenManager)
|
||||
{
|
||||
Jobs = jobs;
|
||||
Sessions = sessions;
|
||||
Removal = removal;
|
||||
DbEntities = dbEntities;
|
||||
AuthTokenManager = authTokenManager;
|
||||
PasswordHasher = passwordHasher;
|
||||
@@ -48,6 +60,12 @@ namespace PrivaPub.Services
|
||||
AppConfigurationService = appConfigurationService;
|
||||
}
|
||||
|
||||
public const string NoMatch = "That username and password do not match.";
|
||||
static string _decoy;
|
||||
|
||||
// a hash nobody's password matches, so an unknown login costs the same hashing as a wrong password
|
||||
string Decoy => _decoy ??= PasswordHasher.Hash(Convert.ToHexString(System.Security.Cryptography.RandomNumberGenerator.GetBytes(16)));
|
||||
|
||||
public async Task<WebResult> SignUpAsync(LoginForm signUpForm, string invitationCode = default,
|
||||
bool isPasswordRequired = false)
|
||||
{
|
||||
@@ -114,20 +132,16 @@ namespace PrivaPub.Services
|
||||
var result = new WebResult();
|
||||
try
|
||||
{
|
||||
// One answer, after the same work, whether the login is unknown, deleted or the password wrong: sign-in must not
|
||||
// tell anyone which logins exist. Only someone who knows the password learns the login is banned.
|
||||
loginForm.UserName = loginForm.UserName.ToLower();
|
||||
if (!await DbEntities.RootUsers.Match(u => u.UserName == loginForm.UserName && u.DeletedAt == null)
|
||||
.ExecuteAnyAsync())
|
||||
return result.Invalidate(Localizer["Username '{0}' not found.", loginForm.UserName]);
|
||||
|
||||
var user = await DbEntities.RootUsers.Match(u => u.UserName == loginForm.UserName).ExecuteFirstAsync();
|
||||
var user = await DbEntities.RootUsers.Match(u => u.UserName == loginForm.UserName && u.DeletedAt == null).ExecuteFirstAsync();
|
||||
var (verified, needsUpgrade) = PasswordHasher.Check(user?.HashedPassword ?? Decoy, loginForm.Password);
|
||||
if (user?.HashedPassword == default || !verified)
|
||||
return result.Invalidate(Localizer[NoMatch]);
|
||||
if (user.IsBanned)
|
||||
return result.Invalidate(Localizer["User '{0}' banned.", user.UserName]);
|
||||
|
||||
var (verified, needsUpgrade) = PasswordHasher.Check(user.HashedPassword, loginForm.Password);
|
||||
|
||||
if (!verified)
|
||||
return result.Invalidate(Localizer["Wrong password."]);
|
||||
|
||||
if (needsUpgrade)
|
||||
result.ErrorMessage = Localizer["Needs upgrade!"];
|
||||
|
||||
@@ -261,24 +275,12 @@ namespace PrivaPub.Services
|
||||
var result = new WebResult();
|
||||
try
|
||||
{
|
||||
var users = await DbEntities.RootUsers.Match(u => usersIds.UserIdList.Contains(u.ID) && u.DeletedAt == default).ExecuteAsync();
|
||||
if (users == null || users.Count == 0)
|
||||
var removed = 0;
|
||||
foreach (var id in usersIds.UserIdList.Distinct())
|
||||
if (await Removal.Remove(id, CancellationToken.None))
|
||||
removed++;
|
||||
if (removed == 0)
|
||||
return result.Invalidate(Localizer["User already deleted."]);
|
||||
|
||||
foreach (var user in users)
|
||||
{
|
||||
user.Email = Localizer["Deleted user"];
|
||||
user.HashedPassword = null;
|
||||
user.Policies.Clear();
|
||||
user.IsBanned = false;
|
||||
user.IsEmailValidated = false;
|
||||
//user.TempSecret = null;
|
||||
user.UserName = Localizer["Deleted user"];
|
||||
user.DeletedAt = DateTime.UtcNow;
|
||||
|
||||
await DB.Default.SaveAsync(user);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
@@ -288,6 +290,26 @@ namespace PrivaPub.Services
|
||||
}
|
||||
}
|
||||
|
||||
// A root deletes itself only with its password, so a stolen session cannot.
|
||||
public async Task<WebResult> RemoveSelfAsync(string rootId, string password)
|
||||
{
|
||||
var result = new WebResult();
|
||||
try
|
||||
{
|
||||
var user = await DbEntities.RootUsers.MatchID(rootId).Match(u => u.DeletedAt == null).ExecuteFirstAsync();
|
||||
var (verified, _) = PasswordHasher.Check(user?.HashedPassword ?? Decoy, password ?? string.Empty);
|
||||
if (user?.HashedPassword == default || !verified)
|
||||
return result.Invalidate(Localizer[NoMatch], StatusCodes.Status403Forbidden);
|
||||
await Removal.Remove(user.ID, CancellationToken.None);
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogError(ex, $"{nameof(RootUsersService)}.{nameof(RemoveSelfAsync)}()");
|
||||
return result.Invalidate(Localizer["Something went wrong."], exception: ex);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<WebResult> BanUserAsync(UsersIds usersIds)
|
||||
{
|
||||
var result = new WebResult();
|
||||
@@ -354,141 +376,22 @@ namespace PrivaPub.Services
|
||||
}
|
||||
}
|
||||
|
||||
public const string RecoverySent = "If that account has an email address, a recovery link is on its way. It works for one hour.";
|
||||
|
||||
// The same answer, after the same work, whether the account exists, has an email or can be written to: recovery must
|
||||
// not tell anyone which logins or addresses exist. RecoveryJob does the rest, out of the request.
|
||||
public async Task<WebResult> SetupAndSendRecoveryEmail(PasswordRecoveryForm passwordRecoveryForm, string host)
|
||||
{
|
||||
var result = new WebResult();
|
||||
try
|
||||
{
|
||||
var usernameExists = false;
|
||||
if (passwordRecoveryForm.IsEmailDisabled)
|
||||
{
|
||||
if (!await DbEntities.RootUsers.Match(u => u.UserName == passwordRecoveryForm.UserName && u.DeletedAt == null)
|
||||
.ExecuteAnyAsync())
|
||||
return result.Invalidate(Localizer["Username '{0}' not found.", passwordRecoveryForm.UserName],
|
||||
StatusCodes.Status404NotFound);
|
||||
usernameExists = true;
|
||||
}
|
||||
else
|
||||
{
|
||||
if (!await DbEntities.RootUsers.Match(u => u.Email == passwordRecoveryForm.Email && u.DeletedAt == null)
|
||||
.ExecuteAnyAsync())
|
||||
return result.Invalidate(Localizer["Username '{0}' not found.", passwordRecoveryForm.UserName],
|
||||
StatusCodes.Status404NotFound);
|
||||
}
|
||||
|
||||
var user = default(RootUser);
|
||||
if (usernameExists)
|
||||
user = await DbEntities.RootUsers.Match(u => u.UserName == passwordRecoveryForm.UserName).ExecuteFirstAsync();
|
||||
else
|
||||
user = await DbEntities.RootUsers.Match(u => u.Email == passwordRecoveryForm.Email).ExecuteFirstAsync();
|
||||
|
||||
if (string.IsNullOrEmpty(user.Email))
|
||||
return result.Invalidate(Localizer["This User doesn't have an email, no way to recover."],
|
||||
StatusCodes.Status423Locked);
|
||||
|
||||
var emailRecovery = await DbEntities.EmailRecoveries
|
||||
.Match(er => er.RootUserId == user.ID).ExecuteFirstAsync();
|
||||
|
||||
if (emailRecovery is null)
|
||||
{
|
||||
emailRecovery = new()
|
||||
{
|
||||
RootUserId = user.ID
|
||||
};
|
||||
await DB.Default.SaveAsync(emailRecovery);
|
||||
}
|
||||
|
||||
using var recoveryCodeGenerator = new Password(true, true, true, false, 127);
|
||||
emailRecovery.RecoveryCode = recoveryCodeGenerator.Next();
|
||||
await DB.Default.SaveAsync(emailRecovery);
|
||||
|
||||
using (var smtpClient = new SmtpClient())
|
||||
{
|
||||
try
|
||||
{
|
||||
await smtpClient.ConnectAsync(AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpServer, AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpPort,
|
||||
AppConfigurationService.AppConfiguration.EmailConfiguration.UseSSL);
|
||||
if (!smtpClient.IsConnected)
|
||||
{
|
||||
Logger.LogError($"Failed to connect to the SMTP server({AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpServer}).");
|
||||
return result.Invalidate(Localizer["Failed to send email."], StatusCodes.Status503ServiceUnavailable);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogError(ex, $"Error at connection to the SMTP server({AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpServer}).");
|
||||
return result.Invalidate(Localizer["Failed to send email."], StatusCodes.Status503ServiceUnavailable, exception: ex);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
await smtpClient.AuthenticateAsync(AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername, AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpPassword);
|
||||
if (!smtpClient.IsAuthenticated)
|
||||
{
|
||||
Logger.LogError($"Failed SMTP authentication of {AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername}.");
|
||||
return result.Invalidate(Localizer["Failed to send email."],
|
||||
StatusCodes.Status503ServiceUnavailable);
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogError(ex, $"Failed SMTP authentication of {AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername}.");
|
||||
return result.Invalidate(Localizer["Failed to send email."],
|
||||
StatusCodes.Status503ServiceUnavailable, exception: ex);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
var toParsed = await smtpClient.VerifyAsync(user.Email);
|
||||
if (toParsed == null)
|
||||
return result.Invalidate(Localizer["Invalid email."], StatusCodes.Status400BadRequest);
|
||||
}
|
||||
catch (OperationCanceledException ex)
|
||||
{
|
||||
Logger.LogWarning(
|
||||
"SMTP operation canceled at email verification: {Error}", ex.Message);
|
||||
}
|
||||
catch (SmtpCommandException ex)
|
||||
{
|
||||
Logger.LogWarning(
|
||||
"SMTP command exception at email verification: {Error}", ex.Message);
|
||||
}
|
||||
catch (SmtpProtocolException ex)
|
||||
{
|
||||
Logger.LogWarning(
|
||||
"SMTP protocol exception at email verification: {Error}", ex.Message);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogWarning("Exception at email verification: {Error}", ex.Message);
|
||||
}
|
||||
|
||||
var message = new MimeMessage();
|
||||
message.From.Add(new MailboxAddress("PrivaPub", AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername));
|
||||
message.To.Add(MailboxAddress.Parse(user.Email));
|
||||
message.Subject = Localizer["PrivaPub - Password recovery link"];
|
||||
message.Body = new TextPart("plain")
|
||||
{
|
||||
Text = string.Format(Localizer[@"Hey {0},
|
||||
|
||||
Eugene from collAnon, following is the password recovery link:
|
||||
{1}
|
||||
|
||||
-- Eugene"], user.UserName, $"{host}/password-recovery?rc={emailRecovery.RecoveryCode}")
|
||||
};
|
||||
try
|
||||
{
|
||||
await smtpClient.SendAsync(message);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Logger.LogError(ex, $"Error at email sending to {user.Email} from {AppConfigurationService.AppConfiguration.EmailConfiguration.SmtpUsername}.");
|
||||
return result.Invalidate(Localizer["Failed to send email."], StatusCodes.Status503ServiceUnavailable, exception: ex);
|
||||
}
|
||||
|
||||
await smtpClient.DisconnectAsync(quit: true);
|
||||
}
|
||||
|
||||
var userName = passwordRecoveryForm.UserName?.ToLowerInvariant();
|
||||
var user = passwordRecoveryForm.IsEmailDisabled
|
||||
? await DbEntities.RootUsers.Match(u => u.UserName == userName && u.DeletedAt == null).ExecuteFirstAsync()
|
||||
: await DbEntities.RootUsers.Match(u => u.Email == passwordRecoveryForm.Email && u.DeletedAt == null).ExecuteFirstAsync();
|
||||
await Jobs.Enqueue(JobKind.SendRecovery, JsonSerializer.Serialize(new RecoveryPayload(user?.ID, host)), RecoveryJob.Host,
|
||||
dedupeKey: default, CancellationToken.None);
|
||||
result.Data = Localizer[RecoverySent].Value;
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
@@ -498,16 +401,19 @@ Eugene from collAnon, following is the password recovery link:
|
||||
}
|
||||
}
|
||||
|
||||
Task<EmailRecovery> LiveRecovery(string recoveryCode)
|
||||
{
|
||||
var hash = RecoveryJob.Hash(recoveryCode);
|
||||
var now = DateTime.UtcNow;
|
||||
return DbEntities.EmailRecoveries.Match(er => er.CodeHash == hash && er.ExpiresAt > now).ExecuteFirstAsync();
|
||||
}
|
||||
|
||||
public async Task<WebResult> IsValidRecoveryCode(string recoveryCode)
|
||||
{
|
||||
var result = new WebResult();
|
||||
try
|
||||
{
|
||||
var isValidRecoveryCode = await DbEntities.EmailRecoveries
|
||||
.Match(er => er.RecoveryCode == recoveryCode).ExecuteAnyAsync();
|
||||
|
||||
result.Data = isValidRecoveryCode;
|
||||
|
||||
result.Data = await LiveRecovery(recoveryCode) != default;
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
@@ -516,33 +422,24 @@ Eugene from collAnon, following is the password recovery link:
|
||||
}
|
||||
}
|
||||
|
||||
// A recovered password ends every session the root had: whoever had them may be why it was recovered.
|
||||
public async Task<WebResult> ChangePassword(NewPasswordForm newPasswordForm)
|
||||
{
|
||||
var result = new WebResult();
|
||||
try
|
||||
{
|
||||
var isValidRecoveryCode = await DbEntities.EmailRecoveries
|
||||
.Match(er => er.RecoveryCode == newPasswordForm.RecoveryCode).ExecuteAnyAsync();
|
||||
if (!isValidRecoveryCode)
|
||||
var recovery = await LiveRecovery(newPasswordForm.RecoveryCode);
|
||||
var user = recovery == default ? default
|
||||
: await DbEntities.RootUsers.MatchID(recovery.RootUserId).Match(u => u.DeletedAt == null && !u.IsBanned).ExecuteFirstAsync();
|
||||
if (user == default)
|
||||
return result.Invalidate(Localizer["Invalid recovery code."], StatusCodes.Status404NotFound);
|
||||
|
||||
var emailRecovery = await DbEntities.EmailRecoveries
|
||||
.Match(er => er.RecoveryCode == newPasswordForm.RecoveryCode)
|
||||
.Project(er => er.Include(nameof(EmailRecovery.RootUserId)))
|
||||
.ExecuteFirstAsync();
|
||||
|
||||
if (emailRecovery is null || emailRecovery.RootUserId is null)
|
||||
return result.Invalidate(Localizer["User not found."]);
|
||||
|
||||
var newHashedPassword = PasswordHasher.Hash(newPasswordForm.NewPassword);
|
||||
|
||||
_ = await DB.Default.Update<RootUser>()
|
||||
.Match(u => u.ID == emailRecovery.RootUserId && u.DeletedAt == null)
|
||||
.Modify(u => u.HashedPassword, newHashedPassword)
|
||||
await DB.Default.Update<RootUser>().MatchID(user.ID)
|
||||
.Modify(u => u.HashedPassword, PasswordHasher.Hash(newPasswordForm.NewPassword))
|
||||
.Modify(u => u.UpdatedAt, DateTime.UtcNow)
|
||||
.ExecuteAsync();
|
||||
|
||||
_ = await DB.Default.DeleteAsync<EmailRecovery>(er => er.RecoveryCode == newPasswordForm.RecoveryCode);
|
||||
|
||||
await DB.Default.DeleteAsync<EmailRecovery>(er => er.RootUserId == user.ID);
|
||||
await Sessions.Revoke(user.ID, CancellationToken.None);
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -12,6 +12,9 @@ namespace PrivaPub.StaticServices
|
||||
{
|
||||
public class AuthTokenManager
|
||||
{
|
||||
public const string SessionStamp = "ss";
|
||||
const string SessionStampClaim = SessionStamp;//the root's RootUser.SessionStamp when issued; JwtEvents refuses a stale one
|
||||
|
||||
readonly IConfiguration Configuration;
|
||||
|
||||
public AuthTokenManager(IConfiguration configuration)
|
||||
@@ -35,7 +38,8 @@ namespace PrivaPub.StaticServices
|
||||
var claims = new List<Claim>
|
||||
{
|
||||
new(ClaimTypes.UserData, user.ID),
|
||||
new(ClaimTypes.Name, user.UserName)
|
||||
new(ClaimTypes.Name, user.UserName),
|
||||
new(SessionStampClaim, user.SessionStamp ?? string.Empty)
|
||||
};
|
||||
|
||||
claims.Add(new(Policies.IsUser, $"{user.Policies.Contains(Policies.IsUser)}".ToLower()));
|
||||
@@ -44,6 +48,7 @@ namespace PrivaPub.StaticServices
|
||||
|
||||
var token = new JwtSecurityToken(issuer: Configuration["AppConfiguration:Jwt:Issuer"], audience: Configuration["AppConfiguration:Jwt:Audience"],
|
||||
claims: claims,
|
||||
notBefore: DateTime.UtcNow,
|
||||
expires: expiration,
|
||||
signingCredentials: new(securityKey, SecurityAlgorithms.HmacSha512)
|
||||
);
|
||||
|
||||
@@ -31,7 +31,7 @@ namespace PrivaPub.StaticServices
|
||||
using (var algorithm = new Rfc2898DeriveBytes(password, salt, iterations, HashAlgorithmName.SHA512))
|
||||
{
|
||||
var keyToCheck = algorithm.GetBytes(KeySize);
|
||||
var verified = keyToCheck.SequenceEqual(key);
|
||||
var verified = CryptographicOperations.FixedTimeEquals(keyToCheck, key);
|
||||
return (verified, needsUpgrade);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -92,6 +92,49 @@ namespace PrivaPub.Web.Pages
|
||||
}
|
||||
}
|
||||
|
||||
// The hashtag links our posts carry (/tags/{tag}): this server's public posts with that tag, as a page like a profile's.
|
||||
public class TagModel : PublicPageModel
|
||||
{
|
||||
const int PageSize = 20;
|
||||
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly DbEntities _dbEntities;
|
||||
|
||||
public TagModel(ILocalActorService localActors, DbEntities dbEntities)
|
||||
{
|
||||
_localActors = localActors;
|
||||
_dbEntities = dbEntities;
|
||||
}
|
||||
|
||||
public string Tag { get; private set; }
|
||||
public IReadOnlyList<PostView> Posts { get; private set; } = Array.Empty<PostView>();
|
||||
|
||||
public async Task<IActionResult> OnGetAsync(string tag, CancellationToken token)
|
||||
{
|
||||
Tag = tag?.TrimStart('#').ToLowerInvariant();
|
||||
if (string.IsNullOrEmpty(Tag) || Tag.Length > 100)
|
||||
return NotFound();
|
||||
Harden();
|
||||
var posts = await _dbEntities.Posts
|
||||
.Match(p => p.Tags.Contains(Tag) && !p.IsFederatedCopy && p.ReblogOfPostId == null && p.Visibility == PostVisibility.Public)
|
||||
.Match(VisibilityPolicy.IsPublic)
|
||||
.Sort(p => p.ID, Order.Descending)
|
||||
.Limit(PageSize)
|
||||
.ExecuteAsync(token);
|
||||
var authors = new Dictionary<string, LocalActor>();
|
||||
var views = new List<PostView>();
|
||||
foreach (var post in posts)
|
||||
{
|
||||
if (!authors.TryGetValue(post.GroupUserId, out var author))
|
||||
authors[post.GroupUserId] = author = await _localActors.FindById(LocalActorKind.Person, post.GroupUserId, token);
|
||||
if (author is { IsFederated: true })
|
||||
views.Add(PostView.From(post, author));
|
||||
}
|
||||
Posts = views;
|
||||
return Page();
|
||||
}
|
||||
}
|
||||
|
||||
public class StatusModel : PublicPageModel
|
||||
{
|
||||
readonly ILocalActorService _localActors;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
@page "/stargazing"
|
||||
@page "/stargazing"
|
||||
@model PrivaPub.Web.Pages.StargazingModel
|
||||
@{
|
||||
ViewData["Title"] = "Stargazing";
|
||||
@@ -13,8 +13,10 @@
|
||||
never accounts: what kind of software a server runs, what it exchanges with us, and how reliably. Distinct accounts are
|
||||
only counted, through a key that is destroyed at the end of each day.</p>
|
||||
<p>A server we exchange activities with is described once a week, from its public NodeInfo and, when it has one, its
|
||||
Mastodon instance API. Its location comes from the address we reached, looked up in an offline database: only the
|
||||
country is shown publicly for small servers, and only the CDN for servers behind one.</p>
|
||||
Mastodon instance API. Its location comes from the address we reached, looked up in an offline database, and is shown
|
||||
to the city; for a server behind a CDN, only the CDN is shown, since the address is the CDN's. CDNs are recognised from
|
||||
the address lists they publish and from their marks in the answers we get, and servers are grouped by their CDN week
|
||||
by week.</p>
|
||||
@if (Model.GeoSource is { } geo)
|
||||
{
|
||||
<p>Locations come from <a href="https://db-ip.com" rel="nofollow noopener">@geo</a>, licensed under
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
@page "/tags/{tag}"
|
||||
@model PrivaPub.Web.Pages.TagModel
|
||||
@{
|
||||
ViewData["Title"] = $"#{Model.Tag}";
|
||||
}
|
||||
<header>
|
||||
<h1>#@Model.Tag</h1>
|
||||
<div class="meta">Public posts on this server with this tag.</div>
|
||||
</header>
|
||||
@foreach (var post in Model.Posts)
|
||||
{
|
||||
<partial name="_Post" model="post" />
|
||||
}
|
||||
@if (Model.Posts.Count == 0)
|
||||
{
|
||||
<p class="meta">Nothing public with this tag yet.</p>
|
||||
}
|
||||
@@ -5,6 +5,9 @@
|
||||
"Registrations": {
|
||||
"Mode": "Invitations"
|
||||
},
|
||||
"Federation": {
|
||||
"SecureMode": true
|
||||
},
|
||||
"Statistics": {
|
||||
"Crawler": {
|
||||
"Enabled": true,
|
||||
|
||||
+21
-8
@@ -1,4 +1,4 @@
|
||||
# Interop: what every peer sends, what it expects, and what PrivaPub still drops
|
||||
# Interop: what every peer sends, what it expects, and what PrivaPub still drops
|
||||
|
||||
Research as of **2026-10-01**: the platforms' source on their default branches, live ActivityPub fetches from large
|
||||
instances, release notes and the FEP repository. Version numbers are what was current that day. Claims the
|
||||
@@ -169,15 +169,19 @@ Priorities, used throughout:
|
||||
Findings:
|
||||
- **Mastodon 4.7 names its actors by number**: `https://mastodon.test/ap/users/<id>`, inbox `…/ap/users/<id>/inbox`. Nothing here may assume
|
||||
`/users/<name>`.
|
||||
- **It drops circle posts** (expected failure in the scenario).
|
||||
- **It dropped circle posts** until each member's copy named that member (owner decision 2026-10-04, v1.19.0).
|
||||
- A post addressed only to `[circle, circle/flock]` parses as `direct` there, and Mastodon keeps a `direct` post
|
||||
only if it names a local account or arrived in a known account's inbox (`Create#addresses_local_accounts?`).
|
||||
- But `ActivityPub::InboxesController#account_required?` looks only at `params[:account_username]`. A delivery to
|
||||
the numeric `/ap/users/:account_id/inbox` it now advertises therefore reaches the worker with no recipient and
|
||||
is rejected.
|
||||
- DMs are unaffected because they name the recipient.
|
||||
- Fixing it on our side means naming the member (or that server's members) in each copy's `cc`, which changes what a
|
||||
circle reveals: **owner decision pending**. Reporting it upstream is the other half.
|
||||
- Each member's copy now names that member in `cc` and mentions them silently, so Mastodon keeps it, and its signed
|
||||
refetch (`ActivityPub::FetchRemoteStatusService`, by its instance actor) gets the post back naming the members on
|
||||
that server. The same holds for a followers-only post's refetch, which used to answer 404, and a 404 on refetch
|
||||
makes Mastodon delete its copy. Reporting the numeric-inbox recipient loss upstream is still worth doing.
|
||||
- **With SecureMode on** (all six peers, 2026-10-04) every federation check passes: Mastodon, GoToSocial, Misskey,
|
||||
Sharkey, Akkoma and Lemmy all sign their fetches, and fetch our instance actor's key unsigned first.
|
||||
- Inbound `Block` from Mastodon is not enforced yet (P7).
|
||||
|
||||
### GoToSocial: 0.22.1 (2026-07-20)
|
||||
@@ -233,7 +237,10 @@ Findings:
|
||||
|
||||
**Pasture evidence (2026-10-03, GoToSocial 0.22.1, `tools/pasture/scenarios/gts.sh`):** 37 checks pass, three runs in a
|
||||
row. That is the original 33 plus four on statistics: described as gotosocial, inbound and outbound traffic counted,
|
||||
no account named.
|
||||
no account named. Since 2026-10-04 (v1.19.0) circle posts reach a GoToSocial member too. GoToSocial files a post for
|
||||
neither the public nor the author's followers as a direct message, like our DMs, and shows it only to the accounts it
|
||||
mentions. Being in `cc` stored it but left it invisible, so each member's copy also mentions that member silently.
|
||||
Such posts are then found in the member's conversations, never by a search on their URI.
|
||||
|
||||
### Misskey family: Misskey 2026.10.0, Sharkey 2025.4.7, Iceshrimp.NET 2026.1.2-beta, CherryPick 4.17
|
||||
|
||||
@@ -624,8 +631,10 @@ attachment. The player is a card whose iframe loads from the remote host, which
|
||||
- **Gaps:**
|
||||
- **P1 outbound:** keep JPEG/PNG renditions with an explicit `mediaType`.
|
||||
- **P1 inbound:** Loops' `preview` poster.
|
||||
- **P2:** Pixelfed `location` → own `privapub.place`, display only and never re-federated; `commentsEnabled: false`
|
||||
disables replies.
|
||||
- ~~**P2:** Pixelfed `location` → own `privapub.place`, display only and never re-federated~~ done:
|
||||
`ObjectShapes.NotePlace` keeps a `Place` with coordinates (Pixelfed sends them as strings) in `Post.Place`, an edit
|
||||
replaces it, and nothing renders it back out.
|
||||
- **P2:** `commentsEnabled: false` disables replies.
|
||||
- **P3:** ignore `Add{Story}` without an error; answer `FeatureRequest` with `Reject`.
|
||||
|
||||
### Long-form: WordPress plugin 9.3.1, Ghost 6, WriteFreely 0.17.2
|
||||
@@ -800,7 +809,11 @@ a remote host.
|
||||
|
||||
`GET /api/privapub/v1/instances/:host` returns cached NodeInfo (`metadata` extras: Misskey `themeColor`/`maxNoteTextLength`,
|
||||
Pleroma `features[]`/`federation.mrf_policies`), the instance API's icon and description, the delivery health our
|
||||
circuit breaker sees, and which signature scheme worked.
|
||||
circuit breaker sees, and which signature scheme worked. Its `geo` is the public projection of where the server is
|
||||
(`precision` `city`, `country` or `cdn`, ROADMAP owner decision on server locations); a CDN-fronted server's names the
|
||||
CDN's domain, how it was found (`ranges`, `headers`, `asn`) and `before_cdn`, its last place before the CDN. `?host[]=`
|
||||
answers up to 40 at once, and this server describes itself the same way. `/instances/:host/history` lists its weekly
|
||||
snapshots; `/api/privapub/v1/cdns` and `/cdns/:domain` group servers by CDN, week by week.
|
||||
|
||||
`software.name` → family, for display:
|
||||
|
||||
|
||||
+38
-11
@@ -1,4 +1,4 @@
|
||||
# PrivaPub roadmap
|
||||
# PrivaPub roadmap
|
||||
|
||||
Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federation gap audit of commit `075c222`, research on .NET ActivityPub libraries, and the owner's decisions. Each phase ends in a tagged deploy plus verification; tick phases off here as they land.
|
||||
|
||||
@@ -31,9 +31,25 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
|
||||
|
||||
Both open items were closed without a root step (owner decisions, 2026-10-04): the server fetches its geolocation
|
||||
databases itself, and the deploy makes and signs in as @thepra.
|
||||
- [ ] Everything on in production (owner decisions 2026-10-04): v1.18.0 self-updating geolocation, @thepra, the crawler
|
||||
on, sign-up by invitation, one registrations switch; then signed audiences with circles for everyone and SecureMode on,
|
||||
account privacy, and one answer everywhere (the mismatch sweep).
|
||||
- [x] Everything on in production (owner decisions 2026-10-04):
|
||||
- v1.18.0, deployed and verified 2026-10-04: geolocation that updates itself (DB-IP Lite 2026-10 loaded), the deploy
|
||||
signing in as @thepra (undiscoverable), the crawler on (1010 servers known within the hour), sign-up by invitation
|
||||
with one registrations switch;
|
||||
- v1.19.1, deployed and verified 2026-10-04 (v1.19.0's deploy stopped at a flaky test, fixed in v1.19.1):
|
||||
- circle posts reach Mastodon and GoToSocial members, each copy naming and mentioning its member;
|
||||
- followers-only, direct and circle posts are served to signed refetches from those they were for;
|
||||
- SecureMode is on: a persona answers 401 unsigned, a browser is redirected, and all six pasture peers pass;
|
||||
- account privacy: sign-in and recovery say nothing, recovery codes are hashed for an hour, a recovered password
|
||||
ends every session, and a deleted root's personas and groups are deleted everywhere;
|
||||
- v1.20.0, deployed and verified 2026-10-04: one answer everywhere:
|
||||
- one counting rule for posts and users;
|
||||
- gone accounts out of every count;
|
||||
- replies_count only for public replies;
|
||||
- search anyone may use;
|
||||
- the instance API advertising what is enforced;
|
||||
- Mastodon routes that answer, grouped notifications;
|
||||
- community announce ids that resolve, a hashtag page;
|
||||
- remote accounts' real counts.
|
||||
- [ ] P7 Threads, communities, moderation, the social graph
|
||||
- [ ] P8 Signatures, discovery, the long tail
|
||||
|
||||
@@ -135,7 +151,7 @@ and circles (see Owner decisions).
|
||||
|
||||
| Question | Decision |
|
||||
|---|---|
|
||||
| Client interface | **Mastodon client API**, so Tusky, Elk, Phanpy, Ivory and the official apps work. Each avatar is its own Mastodon account: at OAuth authorize, the logged-in RootUser picks the avatar the token is for. PrivaPub-only features (avatars, groups, contacts, range posts) stay on `/clientapi`. Moving decePubClient onto the Mastodon API is out of scope. |
|
||||
| Client interface | **Mastodon client API**, so Tusky, Elk, Phanpy, Ivory and the official apps work. Each avatar is its own Mastodon account: at OAuth authorize, the logged-in RootUser picks the avatar the token is for. PrivaPub-only features (avatars, groups, contacts, range posts) stay on `/clientapi`. ~~Moving decePubClient onto the Mastodon API is out of scope.~~ *Superseded 2026-10-04: decePubClient uses both APIs with one sign-in, see "Owner decisions on decePubClient".* |
|
||||
| Personas | **Unlinkable to other users and servers.** The admin can still see the link in the database. No root IDs next to IPs in logs, NodeInfo counts nothing that links personas, blocks, mutes and notifications are per avatar, and invitation signup no longer names the avatar after the root username. |
|
||||
| Location-ranged posts | **Local only, never federated.** Shown to local users within the radius, with coordinates rounded on storage. |
|
||||
| Groups | **Per group, two kinds.** A *community* federates per FEP-1b12, Lemmy-compatible: it Announces the activity, uses `audience`, and accepts posts from non-followers. A *circle* is invitation-only: posts are addressed to the members collection, and objects are served only to signed requests from members. |
|
||||
@@ -162,7 +178,7 @@ and circles (see Owner decisions).
|
||||
| Local side in statistics | **Only the kind of local actor** (person, group, application), **and only on public and unlisted traffic.** DMs, followers-only and circle traffic are one "private" class, never broken out per server in public. Circles are never named, whether as a kind or as a reason. Fetches of our own documents are counted per day, never per server. |
|
||||
| Reading-driven traffic | **Counted per day, never logged per event:** the media proxy, lookups a client asks for, and the client API per endpoint group (admin only). Client app names are not recorded. |
|
||||
| Describing servers | **Every server we exchange activities with is described weekly**, from its NodeInfo (including the user counts it publishes) and its Mastodon instance API, never its contact account. These requests are unsigned, because they are not ActivityPub documents. Never on read. |
|
||||
| Server locations | **City and network (ASN) from the offline DB-IP Lite databases** (CC BY 4.0, attributed), downloaded monthly (by the server itself since 2026-10-04). The location comes from the address we connected to; an inbound sender's address is never recorded, and no address is stored. In public: city and network only for servers reporting at least 10 users and not behind a CDN; the country otherwise; only the CDN's name for CDN-fronted servers. The admin sees everything. |
|
||||
| Server locations | **City and network (ASN) from the offline DB-IP Lite databases** (CC BY 4.0, attributed), downloaded monthly (by the server itself since 2026-10-04). The location comes from the address we connected to; an inbound sender's address is never recorded, and no address is stored. In public: the city, coordinates and network the server was located to; only the CDN's name for CDN-fronted servers. The admin sees everything. *Corrected 2026-10-04: this row used to show only the country for servers reporting fewer than 10 users; the owner never decided that threshold, and it is gone.* |
|
||||
| Crawler | **Off by default** (`Statistics:Crawler:Enabled`); **on in production since 2026-10-04**, see below. When on, it identifies as `PrivaPub-Stargazer/<version> (+https://privapub.thepra.dev/stargazing)`, where `/stargazing` explains it and how to opt out. It honours robots.txt (an unreachable robots.txt means "keep out") and domain blocks. It visits one server a minute, each at most weekly, and at most 5000 servers. It reads only robots.txt, NodeInfo, the instance API and the peers list, never accounts, posts or directories. Crawled servers stay marked as crawled. |
|
||||
| A remote account deletes itself | **Its posts are kept but hidden everywhere** (`Post.AuthorGone`): from timelines, profiles, search and lookups by id. Its follows and timeline rows go, as before. |
|
||||
| Signed-in smoke check in production | **An undiscoverable persona**, the deploy checks `verify_credentials`, home and notifications with it. *Superseded 2026-10-04: the deploy makes and keeps the persona itself, below.* |
|
||||
@@ -181,6 +197,15 @@ and circles (see Owner decisions).
|
||||
| What circles and located posts reveal | **Unchanged**: circles still answer WebFinger, and circle and located posts still count in a persona's post count, "a good balance for the fediverse to work". |
|
||||
| Public `/stargazing` statistics | **Later**, as decided on 2026-10-03; the crawler and the admin API keep collecting meanwhile. |
|
||||
|
||||
### Owner decisions on decePubClient (2026-10-04)
|
||||
|
||||
| Question | Decision |
|
||||
|---|---|
|
||||
| Sign-in | **One login.** decePubClient signs in on `/clientapi` and exchanges that JWT for one persona's Mastodon token (RFC 8693 token exchange on `/oauth/token`, `PersonaExchange`), one token per persona it uses. Only the seeded first-party application may exchange; the token names the persona, never the root, like any other. |
|
||||
| CDN detection | **The server finds CDNs by itself.** It downloads the address ranges CDNs publish (Cloudflare, Fastly, Amazon CloudFront, Bunny, Gcore, Imperva) once a day from the CDNs' own hosts, reads CDN fingerprints in the responses it already gets from each server (`cf-ray`, `x-served-by`, `x-amz-cf-id`…), and keeps a short list of networks that carry nothing but a CDN (Akamai, Edgio…). |
|
||||
| CDN-fronted servers | **Kept apart, and followed through time.** Their place stays hidden (the address is the CDN's edge). They are grouped per CDN, named by the CDN's own domain, with week by week how many servers were behind it and who joined or left. Each server's weekly history (place, CDN, size) is public, including where it was before it went behind a CDN. |
|
||||
| Server locations on the instance API | **Public, as decided for public server locations** (2026-10-03, corrected): city, coordinates to 0.1° and network for every located server whatever its size, the CDN's name for a CDN-fronted one, with DB-IP's attribution. decePubClient's globe draws posts at their author's server. This server's own place comes from its host's address, or from `Statistics:Geo:Self` when the owner sets it (a server behind a proxy). |
|
||||
|
||||
## Libraries (researched; no maintained .NET ActivityPub library exists, so Letterbook and Iceshrimp.NET both wrote their own)
|
||||
|
||||
| Area | Choice |
|
||||
@@ -498,7 +523,10 @@ it, raw where it doesn't.
|
||||
so Mastodon 4.7 retries instead of switching to RFC 9421.
|
||||
- **Deliberately not done: dereferenceable `Follow`, `Like`, `Block`, `Accept`, `Reject` and `Undo` ids.** Serving them
|
||||
would publish who follows, likes and blocks whom, which our collections deliberately hide. Every one of them is sent
|
||||
with its object embedded, which is all Misskey needs. `Create` and `Announce` ids do dereference.
|
||||
with its object embedded, which is all Misskey needs. `Create` and `Announce` ids do dereference, a community's
|
||||
announces included (kept since 2026-10-04 while the post is shown, 410 after). Update, Delete, EmojiReact,
|
||||
QuoteRequest and its answers, Flag, Ignore and poll votes do not either: like Mastodon's, they are sent whole and
|
||||
never fetched.
|
||||
|
||||
#### P6 What people see: emoji, polls, quotes, reactions, cards, players
|
||||
- **Custom emoji** on posts, names, fields and poll options, proxied (done in v1.10.0), together with fuller remote
|
||||
@@ -568,7 +596,7 @@ it, raw where it doesn't.
|
||||
- Misskey actor extras;
|
||||
- book reviews (`relatedWith`, `rating`);
|
||||
- Funkwhale and Castopod metadata;
|
||||
- Pixelfed `place` (display only, never re-federated);
|
||||
- ~~Pixelfed `place` (display only, never re-federated)~~ done 2026-10-04: `Post.Place`, `privapub.place`;
|
||||
- Bluesky bridging per persona.
|
||||
|
||||
### Sweep and statistics (T and M, planned 2026-10-03, between P6 and P7)
|
||||
@@ -587,8 +615,8 @@ statistics gain value with every day recorded.
|
||||
|
||||
Later, and not part of these steps: the public `/stargazing` statistics. It is anonymous, cached and rate-limited, and
|
||||
covers per-server software, self-published counts, location as projected by the rule above, availability buckets and
|
||||
the public activity mix. Per-server activity is shown only as order-of-magnitude buckets, and only for servers that
|
||||
report at least 10 users; smaller ones fold into one "small servers" aggregate.
|
||||
the public activity mix. *Corrected 2026-10-04: an earlier version showed per-server activity only as order-of-magnitude
|
||||
buckets and only for servers reporting at least 10 users; the owner never intended either restriction.*
|
||||
|
||||
### Cut or deferred (deliberately)
|
||||
- **Cut:**
|
||||
@@ -602,7 +630,6 @@ report at least 10 users; smaller ones fold into one "small servers" aggregate.
|
||||
- S3 storage.
|
||||
- JSON-LD and LD-signature processing. FEP-8b32 proofs need neither (JCS), so they are in P8.
|
||||
- **Deferred:** video transcoding (remux only for now).
|
||||
- **Out of scope:** moving decePubClient onto the Mastodon API.
|
||||
|
||||
## Verification (per phase)
|
||||
|
||||
|
||||
@@ -15,6 +15,13 @@ site() { curl -k --resolve "$1:6443:127.0.0.1" "${@:2}"; }
|
||||
pfetch() { curl -sk --connect-to privapub.test:443:127.0.0.1:6443 "$@"; }
|
||||
|
||||
# make_png <path>: an 8x8 red PNG, for uploads
|
||||
# the status an unsigned ActivityPub GET of a PrivaPub document gets
|
||||
pstatus() { pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$1"; }
|
||||
# SecureMode (PRIVAPUB_ENV="Federation__SecureMode=true") answers every unsigned GET but the instance actor's 401, so a
|
||||
# post's author answering 401 means it is on; then 401 is what "not served" and "gone" look like to an unsigned reader
|
||||
secure_mode() { [ "$(pstatus "${1%%/scribbles/*}")" = "401" ]; }
|
||||
unserved() { local code; code=$(pstatus "$1"); if secure_mode "$1"; then [ "$code" = 401 ]; else [ "$code" = 404 ]; fi; }
|
||||
gone_unsigned() { local code; code=$(pstatus "$1"); if secure_mode "$1"; then [ "$code" = 401 ]; else [ "$code" = 410 ]; fi; }
|
||||
make_png() { python3 -c "
|
||||
import struct,zlib
|
||||
w=h=8
|
||||
|
||||
@@ -57,7 +57,8 @@ until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id/context" | j
|
||||
echo " likes, boosts and reactions"
|
||||
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/favourite"
|
||||
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/reblog"
|
||||
until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "1 1" ]' && ok "alice_akkoma's like and boost count on Akkoma" || ko "like or boost not counted on Akkoma"
|
||||
# Akkoma counts them from its own job queue, which can lag behind under the load of a six-peer run: two minutes
|
||||
until_true 60 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "1 1" ]' && ok "alice_akkoma's like and boost count on Akkoma" || ko "like or boost not counted on Akkoma"
|
||||
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/unfavourite"
|
||||
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$a_post_on_p/unreblog"
|
||||
until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "0 0" ]' && ok "alice_akkoma's unlike and unboost reach Akkoma" || ko "undo of like or boost not applied on Akkoma"
|
||||
|
||||
Loaded 100 of 102 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user