Compare commits

...
6 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 f9658a8e7a P6: remote video and audio play through the proxy
Build / Build (push) Successful in 37s
Deploy / privapub.thepra.dev (push) Successful in 57s
- The media proxy streams ranged requests from the origin (passing the range on, never caching), downloads and caches
  whole files otherwise, and serves cached files with range support. A PeerTube video is never fetched whole for one
  viewer, and clients still never contact the remote host.
- PeerTube's fragmented MP4 files inside an HLS entry are read as variants, so HLS-only instances play too.
- A remote Video or Audio post becomes one playable Mastodon attachment: the best MP4 up to 720p that carries both
  sound and picture, with its poster and duration; the card is kept only when nothing is playable.
- nginx: /media/proxy/ with proxy_buffering off and a 600 s read timeout (applied on the box, with a backup).

Checked live: a GoToSocial image through the proxy answers 206 with exactly the asked range when streamed, 200 when
cached, and 206 with the right Content-Range from the cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:59:14 +02:00
thepraandClaude Opus 5.5 6f1ab0073c P6: quote posts, received and sent (FEP-044f and the older keys)
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 54s
- Received quotes: read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 Link tag; the quoted post
  is fetched once; a quoteAuthorization stamp is verified field by field on the quoted author's origin; a consent
  quote without a stamp is pending; an older-key quote of a public post is shown; Delete of a stamp revokes. Counts
  and a `quote` notification follow the accepted state. The `quote-inline` fallback survives sanitising and is removed
  from content when the real quote is shown.
- Personas quote through `quoted_status_id`: posts that state a quote policy get a QuoteRequest and stay pending until
  an Accept brings a stamp we can verify, then an Update adds quoteAuthorization; posts that state none are quoted
  the older way, without `quote`; another persona's posts cannot be quoted yet (we issue no stamps). Quoting posts
  are delivered to the quoted author too.
- Mastodon API: Status.quote (with the quoted status one level deep), quotes_count, quote_approval from the remote
  policy, GET /api/v1/statuses/:id/quotes, `quote` notifications, and api_versions.mastodon = 7.

Checked live: GoToSocial's author-only quote policy is respected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:53:28 +02:00
thepraandClaude Opus 5.5 001fdac3be P6: link previews read by the server, as the owner decided
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 56s
- A public post that links to a page gets a preview card: from the post's own data first (FEP-8967 Link preview, the
  object's image, title and summary); otherwise the server reads the page once, 0-60 s after the post arrives, never
  when someone reads it. OpenGraph and Twitter tags give title, description and image; one LinkPreview per address
  is cached for 7 days and shared by the whole server, so a fetch never points at a persona.
- Lemmy link posts, which carry no title or description, get them filled in.
- The page fetch uses the guarded client (public addresses, three redirects, HTML only, first 512 KB).
- Federation:FetchLinkPreviews switches page fetching off.
- Local public posts get cards too.

Checked live: a link to a GoToSocial profile page becomes a card with its title, description and proxied image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:43:39 +02:00
thepraandClaude Opus 5.5 fdcf5176bc P6: emoji reactions in and out
Build / Build (push) Successful in 35s
Deploy / privapub.thepra.dev (push) Successful in 57s
- Incoming reactions in all three shapes: Misskey and Sharkey Likes carrying an emoji (`content` or
  `_misskey_reaction`), Pleroma, Akkoma and Iceshrimp.NET EmojiReacts, and their Undo. Unicode reactions are one
  grapheme; custom ones need their Emoji tag and keep its image; `:name@host:` is read as `name`. A Like whose
  content is a heart stays a favourite.
- Reactions are kept per account and emoji on our posts and on remote ones we hold, and the author of a local post
  gets a `pleroma:emoji_reaction` notification with the emoji.
- Personas react through Pleroma's API (PUT/DELETE /api/v1/pleroma/statuses/:id/reactions/:emoji, GET for the list),
  which sends an EmojiReact (or its Undo) to the post's author.
- Statuses carry `emoji_reactions` (read by Phanpy) and `pleroma.emoji_reactions`, with counts and whether the viewer
  reacted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:39:18 +02:00
thepraandClaude Opus 5.5 7be6749a23 P6: custom emoji, fuller remote profiles, and polls both ways
Build / Build (push) Successful in 39s
Deploy / privapub.thepra.dev (push) Successful in 57s
- Custom emoji (Emoji tags) on posts, display names, bios and profile fields, at most 64 per object, proxied, in
  Status.emojis and Account.emojis.
- Remote profiles keep their header, profile fields, locked flag, published date, movedTo, indexable, memorial and
  image descriptions (a locked GoToSocial account no longer shows as open).
- Polls: incoming Questions (Mastodon, Misskey, Pleroma, GoToSocial shapes) with counts, voters, end and closed;
  our own polls from the Mastodon API go out as Questions; votes in are counted once per voter and never become
  replies; personas vote on other servers' polls with one Note per choice; counts refresh with an Update at most every
  three minutes; a poll closes on time and tells its voters and its author. GET /api/v1/polls/:id and POST
  /api/v1/polls/:id/votes.
- An Update without a newer `updated` only refreshes poll, video, audio and event details and leaves no revision.

Checked live against GoToSocial: each side's poll reaches the other as a poll and each side's vote is counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:35:20 +02:00
thepraandClaude Opus 5.5 81de470f64 P5 done: a key we cannot fetch for now gets 503, and follow/like/block ids stay private on purpose
Build / Build (push) Successful in 59s
Deploy / privapub.thepra.dev (push) Successful in 1m11s
- When a sender's key cannot be fetched because its server timed out or answered 5xx, the inbox answers 503 with
  Retry-After: 300 instead of 401, so Mastodon 4.7 retries rather than switching to RFC 9421 signatures we do not
  verify yet. The fetcher's failure cache now remembers whether a failure was temporary.
- Follow, Like, Block, Accept, Reject and Undo ids are deliberately not dereferenceable: serving them would publish
  who follows, likes and blocks whom. They are always sent with their object embedded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:25:18 +02:00
57 changed files with 2643 additions and 92 deletions

No files matched your search

+27
View File
@@ -212,6 +212,16 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
sibling `media-proxy`, which `/media/files` does not serve.
3. **A client never contacts a remote server for media:** every remote URL the API returns goes through
`IMediaProxy.Wrap`, an HMAC-signed `/media/proxy/` URL fetched by `IFederationHttp.GetMedia`.
4. **The proxy serves three ways:**
- **Cached:** a file already cached is served from disk, ranges included.
- **Downloaded:** a request without a `Range` is downloaded whole, up to `Media:MaxProxiedBytes`, then cached.
- **Streamed:** a ranged request, or anything too big to cache, is streamed from the origin with the range passed on,
and never cached. That is how remote video plays.
nginx has a `/media/proxy/` location with `proxy_buffering off` and a 600 s read timeout for those streams.
5. **Remote video and audio become one playable attachment** in the Mastodon API (`MastodonMapper.Playable`): the best MP4
up to 720p that carries both sound and picture, including PeerTube's fragmented files inside an HLS entry. HLS
playlists themselves are not rewritten.
## Privacy invariants
@@ -252,6 +262,23 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
array; Markdown `content`; missing `mediaType`s inferred; thumbnails from any of their five places; and the typed
`Link`, `Video`, `Audio` and `Event` details. A Mastodon API card is built from those, never by fetching the linked page
(that fetch is the owner's decision 1, for P6).
- **An `Update` is an edit only when its `updated` is newer than ours** (`UpdateHandler.IsEdit`). Otherwise it refreshes
the poll, video, audio and event details and nothing else, and leaves no revision: Mastodon and Misskey refresh poll
counts with bare Updates.
- **A poll vote is a `Note` with a `name`, an `inReplyTo` that is a poll we hold, and no content.** `CreateHandler`
hands it to `PollService.Receive` before anything else, so it never becomes a reply. Our votes on other servers' polls
go only to the poll's author, without `published` (PieFed counts a vote only then).
- **Link previews follow owner decision 1** (`Domain/Content/LinkPreviews.cs`): only public posts, queued on arrival with
0–60 s of jitter, one cached `LinkPreview` per address for the whole server, never fetched when someone reads.
`LinkPreviews.Wanted` is called where posts are saved (`CreateHandler`, `StoreContext`, `StatusService.Publish`).
- **Quote states come from `QuoteService.Resolve`:**
- with a stamp: accepted only if `Verified` (fetched, on the quoted author's origin, naming both posts exactly);
- a FEP-044f `quote` without a stamp: pending;
- older keys only: accepted when the quoted post is public.
`QuotesCount` moves with the accepted state, never with the raw key. When a persona quotes, `QuotePermission` decides:
asking first for posts that state a policy, quoting at once for posts that state none, refusing otherwise. Only `quote`
of a post that asks for consent puts `quote` in our JSON; older-key quotes leave it out, as Sharkey learned they must.
- **A server is described on arrival, never on read.** The first record from a host enqueues `DescribeInstance` (its
NodeInfo, at most once a week, into `RemoteInstance`), so opening the details view tells nobody anything.
- **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post
+48 -4
View File
@@ -92,12 +92,13 @@ Received:
| `Update{Person}` | refetches the actor |
| `Like` | counted and notified, on posts the liker could see |
| `Dislike` | counted as a downvote (Lemmy, PieFed, Mbin, Friendica); `Undo` takes it back |
| `EmojiReact`, `Like` with an emoji `content` | an emoji reaction (FEP-c0e0; Pleroma, Akkoma, Iceshrimp.NET, Misskey, Sharkey), Unicode or a custom emoji from its `tag`; a `Like` whose content is ❤ stays a favourite; `Undo` takes it back |
| `Join` | answered with `Ignore`: PrivaPub hosts no events yet (FEP-8a8e) |
| `Announce` | counted and notified for local posts; shown to followers of the announcer, with the original refetched from its origin |
| `Delete` | deletes the object, or the actor and its follows; a deleted object id is remembered for 90 days, so a late `Create` cannot bring it back |
| `Flag` | becomes a report for this server's moderators |
Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`,
Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Create{Question}` and poll votes, `EmojiReact` and its `Undo`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`,
`Reject{Follow}`, `Like`, `Announce` and their `Undo`, `Flag`. A deleted post answers 410 with a `Tombstone`.
- **Attachments** are `Document`s with `mediaType`, `name` (alt text), `blurhash`, `focalPoint`, `width` and `height`.
@@ -108,7 +109,32 @@ Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Update{Note}`, `Update{Person}`
- **Reports** are sent as `Flag` by the instance actor, never by the reporting account.
A `Create`'s `Note` carries Mastodon's `content`, `contentMap`, `summary` and `sensitive`, plus `Mention` and `Hashtag`
tags. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
tags.
**Polls** (FEP-9967) go out as a `Question` with `oneOf` or `anyOf`, each option's count in `replies.totalItems`,
`endTime`, `votersCount`, and `closed` once it has ended. Incoming votes are `Create{Note{name, inReplyTo}}` with no
content, one per choice, counted once per voter. Counts are refreshed with an `Update{Question}` at most every three
minutes. Our own votes on other servers' polls are sent the same way to the poll's author only, without `published`.
An incoming `Update` without a newer `updated` only refreshes counts and details; it is never recorded as an edit.
**Quotes** (FEP-044f, with the older keys):
- **Received.** A quote is read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 `Link` tag.
- With a `quoteAuthorization`, the stamp is fetched and must be a `QuoteAuthorization` on the quoted author's origin,
attributed to them, naming exactly the two posts.
- A FEP-044f `quote` without a stamp is pending until an `Update` brings one.
- A quote made with the older keys only, by software that asks nobody, is shown when the quoted post is public.
- A `Delete` of a stamp revokes the quote.
- **Sent.** Our accounts quote with every key, plus the `RE:` fallback.
- When the quoted post states an `interactionPolicy.canQuote` that allows us, `quote` is included and a `QuoteRequest`
(with the quoting post as `instrument`) goes to the quoted author. The quote stays pending until their `Accept` brings a
stamp we can verify; the post is then updated with `quoteAuthorization`.
- Posts that state no policy are quoted the older way, without `quote`.
- Quoting posts are also delivered to the quoted author.
- **Not yet.** Our own posts cannot be quoted by others yet: we do not issue stamps.
**Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object.
**Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`,
`memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
`name`. A content warning without its own text uses the title, or "Content warning".
Received `summary` is a content warning only on a `Note` or `Question`, or when `sensitive` is `true`. On an `Article`,
@@ -117,6 +143,18 @@ Received `summary` is a content warning only on a `Note` or `Question`, or when
Visibility is expressed in `to`/`cc` the way Mastodon does it: public, unlisted, followers-only and direct. Inbound
followers-only posts are recognised by the author's own `followers` collection.
## Link previews
For a public post that links to a page, PrivaPub builds a preview card. It uses, in order:
1. a Link attachment's own `preview` (FEP-8967) or the post's own image, title and summary;
2. otherwise, the linked page, read once by the server between 0 and 60 seconds after the post arrives (never when
someone reads it), with an `Accept: text/html` request that reads at most 512 KB, through the same address checks as
every other fetch.
The page's OpenGraph and Twitter tags give the title, description and image; the result is cached per address for 7
days and shared by every account on the server. Images are served to clients only through PrivaPub's media proxy.
`Federation:FetchLinkPreviews=false` turns page fetching off.
## Local-only posts
Posts with a location (shown to nearby users of this server) never leave the server, in any form.
@@ -132,12 +170,18 @@ Posts with a location (shown to nearby users of this server) never leave the ser
- **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three
redirects and read at most 1 MB.
- **HTML.** Received HTML is sanitised to Mastodon's allowlist.
- **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered
5xx, the inbox answers 503 with `Retry-After: 300` rather than 401.
- **Activity ids.** `Create` and `Announce` ids dereference. `Follow`, `Like`, `Block` and the `Accept`, `Reject` and
`Undo` that answer them do not, because serving them would reveal who follows, likes and blocks whom; they are always
sent with their object embedded.
- **Delivery.** Failed deliveries are retried with Mastodon's backoff (16 attempts). A host that keeps failing is paused,
starting at an hour and growing to a week.
## Known limitations
- Polls and custom emoji are not implemented.
- Remote media is fetched through this server's proxy when a local client displays it.
- Our own posts carry no custom emoji: emoji are received and shown, not offered.
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
- Collections expose counts, not members.
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
+43
View File
@@ -0,0 +1,43 @@
using PrivaPub.Domain.Content;
using PrivaPub.Models.Post;
namespace PrivaPub.Tests.Domain
{
public class LinkPreviewTests
{
[Fact]
public void Reads_open_graph_then_twitter_then_the_page_title()
{
var preview = LinkPreviews.Read("https://news.example/a?utm_source=x", new Uri("https://news.example/a"), """
<html><head><title>Fallback title</title>
<meta property="og:title" content="Rain &amp; sun">
<meta name="twitter:description" content="A <b>long</b> story">
<meta property="og:image" content="/img/cover.jpg">
<meta property="og:site_name" content="News">
</head><body></body></html>
""");
Assert.Equal("Rain & sun", preview.Title);
Assert.Equal("A long story", preview.Description);
Assert.Equal("https://news.example/img/cover.jpg", preview.ImageURL);
Assert.Equal("News", preview.SiteName);
Assert.False(preview.Failed);
Assert.True(LinkPreviews.Read("https://x.example", new Uri("https://x.example"), "<html><body>nothing</body></html>").Failed);
}
[Fact]
public void Picks_the_first_real_link_and_skips_mentions_tags_quotes_and_ourselves() =>
Assert.Equal("https://blog.example/post", LinkPreviews.FirstLink(new Post
{
Url = "https://m.example/@a/1",
ContentHtml = """
<p><span class="h-card"><a href="https://m.example/@bob" class="u-url mention">@bob</a></span>
<a href="https://m.example/tags/x" class="mention hashtag" rel="tag">#x</a>
<span class="quote-inline">RE: <a href="https://k.example/notes/1">link</a></span>
<a href="https://privapub.test/@me">me</a>
<a href="https://m.example/@a/1">self</a>
<a href="https://blog.example/post">read this</a></p>
"""
}, "https://privapub.test"));
}
}
@@ -0,0 +1,41 @@
using PrivaPub.Federation.Actors;
using System.Text.Json;
namespace PrivaPub.Tests.Federation
{
public class ActorProfileTests
{
[Fact]
public void Reads_a_mastodon_profile_with_its_header_fields_emoji_and_lock()
{
using var json = JsonDocument.Parse("""
{
"id": "https://m.example/users/ann", "type": "Person", "preferredUsername": "ann", "name": "Ann :sparkle:",
"inbox": "https://m.example/users/ann/inbox", "manuallyApprovesFollowers": true, "indexable": true, "memorial": false,
"published": "2023-04-01T00:00:00Z", "movedTo": "https://n.example/users/ann",
"icon": { "type": "Image", "url": "https://m.example/a.png", "summary": "a cat" },
"image": { "type": "Image", "url": "https://m.example/h.png", "name": "mountains" },
"tag": [{ "type": "Emoji", "name": ":sparkle:", "icon": { "type": "Image", "url": "https://m.example/emoji/sparkle.png" } }],
"attachment": [
{ "type": "PropertyValue", "name": "Site", "value": "<a href=\"https://ann.example\" rel=\"me\">ann.example</a><script>x</script>" },
{ "type": "PropertyValue", "name": "Pronouns", "value": "she/her" }
]
}
""");
var actor = ActorDocument.Parse(json.RootElement);
Assert.True(actor.Locked);
Assert.True(actor.Indexable);
Assert.Equal("https://m.example/h.png", actor.Header);
Assert.Equal("a cat", actor.IconDescription);
Assert.Equal("mountains", actor.HeaderDescription);
Assert.Equal("https://n.example/users/ann", actor.MovedTo);
Assert.Equal(new DateTime(2023, 4, 1, 0, 0, 0, DateTimeKind.Utc), actor.Published);
Assert.Equal("sparkle", Assert.Single(actor.Emojis).Shortcode);
Assert.Equal(new[] { "Site", "Pronouns" }, actor.Fields.Keys);
Assert.DoesNotContain("<script>", actor.Fields["Site"]);
}
}
}
@@ -106,6 +106,21 @@ namespace PrivaPub.Tests.Federation
Assert.NotNull(message.ConversationId);
}
[Fact]
public async Task An_unreachable_key_asks_the_sender_to_retry_instead_of_refusing()
{
var (_, alice) = await _harness.Persona("alice");
var offline = new RemoteActor(_harness.Peer, "offline", origin: "http://127.0.0.1:9");
var result = await _harness.Deliver(offline, "/human-centipede", new JsonObject
{
["id"] = NewId(offline, "follows"), ["type"] = "Follow", ["actor"] = offline.Id, ["object"] = alice.Uri
});
Assert.Equal(503, result.StatusCode);
Assert.Equal(300, result.RetryAfterSeconds);
}
[Fact]
public async Task A_join_of_a_local_post_is_answered_with_ignore()
{
@@ -8,6 +8,8 @@ using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Moderation;
using PrivaPub.Domain.Social;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Jobs;
@@ -50,13 +52,15 @@ namespace PrivaPub.Tests.Federation
var db = new DbEntities();
_blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
_receiver = new InboxReceiver(_local, remote, queue, _blocks, NullLogger<InboxReceiver>.Instance);
var remotePosts = new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue), new NoPreviews());
var quotes = new QuoteService(db, remote, remotePosts, _local, delivery, new OutboxPublisher(db, _local, delivery));
_processor = new InboxProcessor(remote, new IActivityHandler[]
{
new FollowHandler(db, _local, remote, delivery),
new UndoHandler(db, _local),
new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue)), new GroupDistributor(delivery), new ObjectRecords(queue)),
new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery)),
new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue))
new UndoHandler(db, _local, new Reactions(db, delivery)),
new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), remotePosts, new GroupDistributor(delivery), new ObjectRecords(queue), new PollService(db, _local, delivery, queue), new NoPreviews(), quotes),
new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery), quotes),
new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue), quotes)
}, NullLogger<InboxProcessor>.Instance);
}
@@ -169,6 +169,75 @@ namespace PrivaPub.Tests.Federation
Assert.Equal("https://loops.example/poster.jpg", loop.CoverURL);
}
[Fact]
public void Keeps_custom_emoji_and_drops_ones_that_could_not_render()
{
var note = Parse("""
{
"id": "https://k.example/notes/1", "type": "Note", "attributedTo": "https://k.example/users/1", "content": ":blobcat: hi :bad:",
"tag": [
{ "type": "Emoji", "name": ":blobcat:", "updated": "2026-01-01T00:00:00Z", "icon": { "type": "Image", "mediaType": "image/png", "url": "https://k.example/emoji/blobcat.png" } },
{ "type": "Emoji", "name": ":blobcat:", "icon": { "type": "Image", "url": "https://k.example/emoji/duplicate.png" } },
{ "type": "Emoji", "name": ":bad name:", "icon": { "type": "Image", "url": "https://k.example/emoji/x.png" } },
{ "type": "Emoji", "name": ":nourl:", "icon": { "type": "Image", "url": "data:image/png;base64,AAAA" } }
]
}
""");
var emoji = Assert.Single(note.Emojis);
Assert.Equal("blobcat", emoji.Shortcode);
Assert.Equal("https://k.example/emoji/blobcat.png", emoji.URL);
}
[Fact]
public void Reads_polls_from_mastodon_misskey_and_closed_ones()
{
var mastodon = Parse("""
{ "id": "https://m.example/q/1", "type": "Question", "attributedTo": "https://m.example/u", "content": "?", "endTime": "2026-10-02T00:00:00Z", "votersCount": 7,
"oneOf": [{ "type": "Note", "name": "a", "replies": { "type": "Collection", "totalItems": 4 } }, { "type": "Note", "name": "b", "replies": { "type": "Collection", "totalItems": 3 } }] }
""");
var misskey = Parse("""
{ "id": "https://k.example/q/1", "type": "Question", "attributedTo": "https://k.example/u", "_misskey_content": "?",
"anyOf": [{ "type": "Note", "name": "x", "_misskey_votes": 2 }, { "type": "Note", "name": "y", "_misskey_votes": 5 }] }
""");
var closed = Parse("""
{ "id": "https://g.example/q/1", "type": "Question", "attributedTo": "https://g.example/u", "content": "?", "closed": "2026-09-30T12:00:00Z",
"oneOf": [{ "type": "Note", "name": "p" }, { "type": "Note", "name": "q" }] }
""");
Assert.False(mastodon.Poll.Multiple);
Assert.Equal(new[] { 4, 3 }, mastodon.Poll.Options.Select(o => o.Votes));
Assert.Equal(7, mastodon.Poll.VotersCount);
Assert.True(misskey.Poll.Multiple);
Assert.Equal(new[] { 2, 5 }, misskey.Poll.Options.Select(o => o.Votes));
Assert.Null(misskey.Poll.VotersCount);
Assert.Equal(new DateTime(2026, 9, 30, 12, 0, 0, DateTimeKind.Utc), closed.Poll.ClosedAt);
}
[Fact]
public void Finds_playable_files_inside_an_hls_only_peertube_video_and_picks_one_with_sound()
{
var note = Parse("""
{
"id": "https://tube.example/videos/watch/2", "type": "Video", "name": "HLS only", "attributedTo": "https://tube.example/accounts/ann",
"url": [
{ "type": "Link", "mediaType": "text/html", "href": "https://tube.example/w/def" },
{ "type": "Link", "mediaType": "application/x-mpegURL", "href": "https://tube.example/master.m3u8", "tag": [
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/1080-fragmented.mp4", "height": 1080,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }] },
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/720-fragmented.mp4", "height": 720,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }] },
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/480-fragmented.mp4", "height": 480,
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }] }
] }
]
}
""");
Assert.Equal(3, note.Video.Variants.Count);
Assert.Equal("https://tube.example/480-fragmented.mp4", MastodonMapper.BestVideo(note.Video.Variants).Url);
}
[Fact]
public void Normalises_hashtags_like_mastodon() =>
Assert.Equal("fedi".Normalize(System.Text.NormalizationForm.FormKC).ToLowerInvariant(), Assert.Single(Parse("""
+138
View File
@@ -0,0 +1,138 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class PollTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task Follows(RemoteActor follower, string target) =>
await _harness.Deliver(follower, "/human-centipede", new JsonObject
{
["id"] = NewId(follower, "follows"), ["type"] = "Follow", ["actor"] = follower.Id, ["object"] = target
});
JsonObject Vote(RemoteActor voter, string question, string owner, string choice) => new()
{
["id"] = NewId(voter, "activities"), ["type"] = "Create", ["actor"] = voter.Id, ["to"] = new JsonArray(owner),
["object"] = new JsonObject
{
["id"] = NewId(voter, "votes"), ["type"] = "Note", ["name"] = choice, ["attributedTo"] = voter.Id,
["inReplyTo"] = question, ["to"] = new JsonArray(owner)
}
};
[Fact]
public async Task A_local_poll_goes_out_as_a_question_and_counts_each_remote_voter_once()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
await Follows(mallory, alice.Uri);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "which?", Poll = new PollDraft(new[] { "tea", "coffee" }, 3600, false, false) }, token);
Assert.True(outcome.Ok);
var question = (await _harness.Outgoing(mallory.Id + "/inbox")).Single(a => a["type"]!.GetValue<string>() == "Create")["object"]!;
Assert.Equal("Question", question["type"]!.GetValue<string>());
Assert.Equal(new[] { "tea", "coffee" }, question["oneOf"]!.AsArray().Select(o => o!["name"]!.GetValue<string>()));
await _harness.Deliver(mallory, "/human-centipede", Vote(mallory, outcome.Post.ObjectURI, alice.Uri, "coffee"));
await _harness.Deliver(mallory, "/human-centipede", Vote(mallory, outcome.Post.ObjectURI, alice.Uri, "tea"));
var poll = (await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token)).Poll;
Assert.Equal(new[] { 0, 1 }, poll.Options.Select(o => o.Votes));
Assert.Equal(1, poll.VotersCount);
Assert.False(await DB.Default.Find<Post>().Match(p => p.InReplyToURI == outcome.Post.ObjectURI).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollRefresh && j.Payload == outcome.Post.ID).ExecuteAnyAsync(token));
Assert.True(await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollClose && j.Payload == outcome.Post.ID).ExecuteAnyAsync(token));
}
[Fact]
public async Task A_persona_votes_on_a_remote_poll_and_a_count_refresh_is_not_an_edit()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var questionId = NewId(mallory, "questions");
JsonObject Question(int first, int second) => new()
{
["id"] = questionId, ["type"] = "Question", ["attributedTo"] = mallory.Id, ["content"] = "<p>pick</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(alice.Uri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" }),
["endTime"] = DateTime.UtcNow.AddDays(1).ToString("O"),
["anyOf"] = new JsonArray(
new JsonObject { ["type"] = "Note", ["name"] = "red", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = first } },
new JsonObject { ["type"] = "Note", ["name"] = "blue", ["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = second } })
};
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities"), ["type"] = "Create", ["actor"] = mallory.Id, ["object"] = Question(3, 1)
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == questionId).ExecuteFirstAsync(token);
Assert.True(post.Poll.Multiple);
Assert.True((await _harness.Polls.Vote(alice, post, new[] { 0, 1 }, token)).Ok);
Assert.False((await _harness.Polls.Vote(alice, post, new[] { 0 }, token)).Ok);
var votes = (await _harness.Outgoing(mallory.Id + "/inbox")).Where(a => a["type"]!.GetValue<string>() == "Create").Select(a => a["object"]!).ToList();
Assert.Equal(new[] { "red", "blue" }, votes.Select(v => v["name"]!.GetValue<string>()).Order().Reverse());
Assert.All(votes, v => Assert.Equal(questionId, v["inReplyTo"]!.GetValue<string>()));
Assert.All(votes, v => Assert.Null(v["published"]));
Assert.Equal(new[] { 0, 1 }, (await _harness.Polls.OwnVotes(post.ID, alice.Id, token)).Order());
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities"), ["type"] = "Update", ["actor"] = mallory.Id, ["object"] = Question(9, 5)
});
var refreshed = await DB.Default.Find<Post>().OneAsync(post.ID, token);
Assert.Equal(new[] { 9, 5 }, refreshed.Poll.Options.Select(o => o.Votes));
Assert.Empty(refreshed.Revisions);
Assert.Null(refreshed.EditedAt);
}
[Fact]
public async Task Closing_a_poll_tells_its_voters_and_its_author()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var (_, bob) = await _harness.Persona("bob");
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "quick", Poll = new PollDraft(new[] { "yes", "no" }, 300, false, false) }, token);
Assert.True((await _harness.Polls.Vote(bob, outcome.Post, new[] { 0 }, token)).Ok);
Assert.False((await _harness.Polls.Vote(alice, outcome.Post, new[] { 0 }, token)).Ok);
var job = await DB.Default.Find<Job>().Match(j => j.Kind == JobKind.PollClose && j.Payload == outcome.Post.ID).ExecuteFirstAsync(token);
await new PollCloseJob(_harness.Db, _harness.Local, _harness.Outbox).Handle(job, token);
Assert.NotNull((await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token)).Poll.ClosedAt);
var told = await DB.Default.Find<Notification>().Match(n => n.Type == NotificationType.Poll && n.PostId == outcome.Post.ID).ExecuteAsync(token);
Assert.Equal(new[] { alice.Id, bob.Id }.Order(), told.Select(n => n.AvatarId).Order());
Assert.False((await _harness.Polls.Vote(bob, await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token), new[] { 1 }, token)).Ok);
}
}
}
+159
View File
@@ -0,0 +1,159 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
[Trait("Category", "Integration")]
public sealed class QuoteTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task<Post> Delivered(RemoteActor author, string mentioned, Action<JsonObject> shape = default)
{
var note = new JsonObject
{
["id"] = NewId(author, "notes"), ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = "<p>hello</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(mentioned),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = mentioned, ["name"] = "@someone" })
};
shape?.Invoke(note);
await _harness.Deliver(author, "/human-centipede", new JsonObject { ["id"] = NewId(author, "activities"), ["type"] = "Create", ["actor"] = author.Id, ["object"] = note });
return await DB.Default.Find<Post>().Match(p => p.ObjectURI == note["id"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
}
string Stamp(RemoteActor quotedAuthor, string quoting, string quoted)
{
var path = $"/stamps/{Guid.NewGuid():N}";
var id = Origin(quotedAuthor) + path;
_harness.Peer.Serve(path, new JsonObject
{
["@context"] = "https://www.w3.org/ns/activitystreams", ["id"] = id, ["type"] = "QuoteAuthorization",
["attributedTo"] = quotedAuthor.Id, ["interactingObject"] = quoting, ["interactionTarget"] = quoted
}.ToJsonString());
return id;
}
[Fact]
public async Task A_misskey_quote_of_a_public_post_is_shown_and_counted()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var kitty = new RemoteActor(_harness.Peer, "kitty");
var original = await Delivered(ann, alice.Uri);
var quoting = await Delivered(kitty, alice.Uri, n =>
{
n["_misskey_quote"] = original.ObjectURI;
n["content"] = $"<p>look<span class=\"quote-inline\"><br>RE: <a href=\"{original.ObjectURI}\">{original.ObjectURI}</a></span></p>";
});
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.Equal(original.ID, quoting.QuotedPostId);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
var status = await new PrivaPub.Api.Mastodon.Mappers.MastodonMapper(_harness.Db, _harness.Local).Status(quoting, alice.Id, token);
Assert.Equal("accepted", status.Quote.State);
Assert.Equal(original.ID, status.Quote.QuotedStatus.Id);
Assert.DoesNotContain("RE:", status.Content);
}
[Fact]
public async Task A_stamped_quote_is_accepted_a_forged_one_is_not_and_revoking_takes_it_back()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var bob = new RemoteActor(_harness.Peer, "bob");
var original = await Delivered(ann, alice.Uri);
var quotingId = NewId(bob, "notes");
var stamp = Stamp(ann, quotingId, original.ObjectURI);
var forged = Stamp(ann, NewId(bob, "notes"), original.ObjectURI);
var quoting = await Delivered(bob, alice.Uri, n => { n["id"] = quotingId; n["quote"] = original.ObjectURI; n["quoteAuthorization"] = stamp; });
var cheat = await Delivered(bob, alice.Uri, n => { n["quote"] = original.ObjectURI; n["quoteAuthorization"] = forged; });
var waiting = await Delivered(bob, alice.Uri, n => n["quote"] = original.ObjectURI);
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.Equal(QuoteState.Unauthorized, cheat.QuoteState);
Assert.Equal(QuoteState.Pending, waiting.QuoteState);
await _harness.Deliver(ann, "/human-centipede", new JsonObject
{
["id"] = NewId(ann, "activities"), ["type"] = "Delete", ["actor"] = ann.Id, ["object"] = stamp
});
Assert.Equal(QuoteState.Revoked, (await DB.Default.Find<Post>().OneAsync(quoting.ID, token)).QuoteState);
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
}
[Fact]
public async Task A_persona_asks_before_quoting_and_is_stamped_when_accepted()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var ann = new RemoteActor(_harness.Peer, "ann");
var original = await Delivered(ann, alice.Uri, n => n["interactionPolicy"] = new JsonObject
{
["canQuote"] = new JsonObject { ["automaticApproval"] = new JsonArray(Addressing.Public) }
});
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "this", QuotedStatusId = original.ID }, token);
Assert.Equal(QuoteState.Pending, outcome.Post.QuoteState);
var request = Assert.Single(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "QuoteRequest");
Assert.Equal(original.ObjectURI, request["object"]!.GetValue<string>());
Assert.Equal(original.ObjectURI, request["instrument"]!["quote"]!.GetValue<string>());
await _harness.Deliver(ann, "/human-centipede", new JsonObject
{
["id"] = NewId(ann, "activities"), ["type"] = "Accept", ["actor"] = ann.Id,
["object"] = request["id"]!.GetValue<string>(), ["result"] = Stamp(ann, outcome.Post.ObjectURI, original.ObjectURI)
});
var quoting = await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token);
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
Assert.NotNull(quoting.QuoteAuthorizationURI);
Assert.Contains(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Update"
&& a["object"]!["quoteAuthorization"]?.GetValue<string>() == quoting.QuoteAuthorizationURI);
}
[Fact]
public async Task A_legacy_post_is_quoted_without_asking_and_another_persona_cannot_be_quoted_yet()
{
var token = TestContext.Current.CancellationToken;
var (aliceRoot, alice) = await _harness.Persona("alice");
var (bobRoot, bob) = await _harness.Persona("bob");
var kitty = new RemoteActor(_harness.Peer, "kitty");
var legacy = await Delivered(kitty, alice.Uri);
await _harness.Posts.InsertPost(bobRoot, new InsertPostForm { AvatarId = bob.Id, Text = "mine" }, token);
var bobs = await DB.Default.Find<Post>().Match(p => p.GroupUserId == bob.Id).ExecuteFirstAsync(token);
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "nice", QuotedStatusId = legacy.ID }, token);
Assert.Equal(QuoteState.Accepted, outcome.Post.QuoteState);
Assert.False(outcome.Post.QuoteByConsent);
Assert.False((await _harness.Statuses.Publish(alice, new StatusDraft { Text = "and this", QuotedStatusId = bobs.ID }, token)).Ok);
Assert.True((await _harness.Statuses.Publish(bob, new StatusDraft { Text = "me again", QuotedStatusId = bobs.ID }, token)).Ok);
}
}
}
+119
View File
@@ -0,0 +1,119 @@
using MongoDB.Entities;
using PrivaPub.ClientModels.Post;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Tests.Support;
using System.Text.Json.Nodes;
namespace PrivaPub.Tests.Federation
{
public class ReactionNamesTests
{
[Theory]
[InlineData(":blobcat@k.example:", "blobcat")]
[InlineData("👍🏽", "👍🏽")]
[InlineData("❤️", "❤️")]
[InlineData("nope", null)]
[InlineData("👍👍", null)]
[InlineData(":unknown:", null)]
public void Normalises_what_peers_send(string content, string expected) =>
Assert.Equal(expected, Reactions.Normalise(content, new List<CustomEmoji> { new() { Shortcode = "blobcat", URL = "https://k.example/e.png" } })?.Name);
}
[Trait("Category", "Integration")]
public sealed class ReactionTests : IAsyncLifetime
{
Harness _harness;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
_harness = await Harness.Start();
}
public async ValueTask DisposeAsync()
{
if (_harness != default)
await _harness.DisposeAsync();
}
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
async Task<Post> LocalPost()
{
var (root, alice) = await _harness.Persona("alice");
await _harness.Posts.InsertPost(root, new InsertPostForm { AvatarId = alice.Id, Text = "react to me" }, TestContext.Current.CancellationToken);
return await DB.Default.Find<Post>().Match(p => p.GroupUserId == alice.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken);
}
[Fact]
public async Task Misskey_and_pleroma_reactions_are_kept_apart_from_favourites()
{
var token = TestContext.Current.CancellationToken;
var post = await LocalPost();
var misskey = new RemoteActor(_harness.Peer, "kitty");
var pleroma = new RemoteActor(_harness.Peer, "pleroma");
await _harness.Deliver(misskey, "/human-centipede", new JsonObject
{
["id"] = NewId(misskey, "likes"), ["type"] = "Like", ["actor"] = misskey.Id, ["object"] = post.ObjectURI, ["content"] = ":blobcat:",
["_misskey_reaction"] = ":blobcat:",
["tag"] = new JsonArray(new JsonObject { ["type"] = "Emoji", ["name"] = ":blobcat:", ["icon"] = new JsonObject { ["type"] = "Image", ["url"] = $"{Origin(misskey)}/emoji/blobcat.png" } })
});
await _harness.Deliver(misskey, "/human-centipede", new JsonObject
{
["id"] = NewId(misskey, "likes"), ["type"] = "Like", ["actor"] = misskey.Id, ["object"] = post.ObjectURI, ["content"] = "❤"
});
var fire = new JsonObject { ["id"] = NewId(pleroma, "activities"), ["type"] = "EmojiReact", ["actor"] = pleroma.Id, ["object"] = post.ObjectURI, ["content"] = "🔥" };
await _harness.Deliver(pleroma, "/human-centipede", fire);
var kept = await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID).ExecuteAsync(token);
Assert.Equal(new HashSet<string> { "blobcat", "🔥" }, kept.Select(r => r.Emoji).ToHashSet());
Assert.Equal($"{Origin(misskey)}/emoji/blobcat.png", kept.Single(r => r.Emoji == "blobcat").EmojiURL);
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(post.ID, token)).FavouritesCount);
Assert.True(await DB.Default.Find<Notification>().Match(n => n.Type == NotificationType.Reaction && n.Emoji == "🔥").ExecuteAnyAsync(token));
await _harness.Deliver(pleroma, "/human-centipede", new JsonObject
{
["id"] = NewId(pleroma, "activities"), ["type"] = "Undo", ["actor"] = pleroma.Id, ["object"] = fire.DeepClone()
});
Assert.Equal("blobcat", Assert.Single(await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID).ExecuteAsync(token)).Emoji);
}
[Fact]
public async Task A_persona_reacts_to_a_remote_post_and_takes_it_back()
{
var token = TestContext.Current.CancellationToken;
var (_, alice) = await _harness.Persona("alice");
var mallory = new RemoteActor(_harness.Peer, "mallory");
var noteId = NewId(mallory, "notes");
await _harness.Deliver(mallory, "/human-centipede", new JsonObject
{
["id"] = NewId(mallory, "activities"), ["type"] = "Create", ["actor"] = mallory.Id,
["object"] = new JsonObject
{
["id"] = noteId, ["type"] = "Note", ["attributedTo"] = mallory.Id, ["content"] = "<p>hi</p>",
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(alice.Uri),
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = alice.Uri, ["name"] = "@alice" })
}
});
var post = await DB.Default.Find<Post>().Match(p => p.ObjectURI == noteId).ExecuteFirstAsync(token);
Assert.True(await _harness.Reactions.React(alice, post, "🎉", token));
Assert.False(await _harness.Reactions.React(alice, post, "not an emoji", token));
Assert.True(await _harness.Reactions.Unreact(alice, post, "🎉", token));
var sent = await _harness.Outgoing(mallory.Id + "/inbox");
var react = Assert.Single(sent, a => a["type"]!.GetValue<string>() == "EmojiReact");
Assert.Equal("🎉", react["content"]!.GetValue<string>());
Assert.Equal(noteId, react["object"]!.GetValue<string>());
Assert.Equal(react["id"]!.GetValue<string>(), Assert.Single(sent, a => a["type"]!.GetValue<string>() == "Undo")["object"]!["id"]!.GetValue<string>());
}
}
}
+22 -10
View File
@@ -44,29 +44,33 @@ namespace PrivaPub.Tests.Support
Fanout = new Fanout(Db);
Groups = new GroupDistributor(Delivery);
Records = new ObjectRecords(Queue);
RemotePosts = new RemotePosts(Db, Local, Remote, new NoBlocks(), Queue, Records);
Polls = new PollService(Db, Local, Delivery, Queue);
Reactions = new Reactions(Db, Delivery);
RemotePosts = new RemotePosts(Db, Local, Remote, new NoBlocks(), Queue, Records, new NoPreviews());
Outbox = new OutboxPublisher(Db, Local, Delivery);
Quotes = new QuoteService(Db, Remote, RemotePosts, Local, Delivery, Outbox);
Receiver = new InboxReceiver(Local, Remote, Queue, new NoBlocks(), NullLogger<InboxReceiver>.Instance);
Processor = new InboxProcessor(Remote, new IActivityHandler[]
{
new FollowHandler(Db, Local, Remote, Delivery),
new AcceptHandler(Db, Local),
new RejectHandler(Db, Local),
new UndoHandler(Db, Local),
new LikeHandler(Db),
new AcceptHandler(Db, Local, Quotes),
new RejectHandler(Db, Local, Quotes),
new UndoHandler(Db, Local, Reactions),
new LikeHandler(Db, Reactions),
new EmojiReactHandler(Db, Reactions),
new DislikeHandler(Db),
new JoinHandler(Db, Local, Delivery),
new AnnounceHandler(Db, Local, RemotePosts, Fanout, Remote),
new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records),
new DeleteHandler(Db, Local, Remote, Delivery, Groups),
new UpdateHandler(Db, Local, Remote, Groups, Records),
new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records, Polls, new NoPreviews(), Quotes),
new DeleteHandler(Db, Local, Remote, Delivery, Groups, Quotes),
new UpdateHandler(Db, Local, Remote, Groups, Records, Quotes),
new FlagHandler(Db, Local)
}, NullLogger<InboxProcessor>.Instance);
Follows = new FollowService(Db, Local, Remote, Delivery, new KeyLocalizer<GenericRes>(), NullLogger<FollowService>.Instance);
Content = new ContentRenderer(Local, Remote);
Outbox = new OutboxPublisher(Db, Local, Delivery);
Media = new MediaService(new StaticOptions<MediaOptions>(new MediaOptions { Root = Path.Combine(Path.GetTempPath(), $"privapub-media-{Guid.NewGuid():N}") }),
Local, default, NullLogger<MediaService>.Instance);
Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups);
Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups, Polls, new NoPreviews(), Quotes);
Posts = new PostsService(Db, Local, Statuses, new KeyLocalizer<GenericRes>(), NullLogger<PostsService>.Instance);
Timelines = new TimelineService(Db, new KeyLocalizer<GenericRes>());
Relationships = new RelationshipService(Db, Follows, Delivery);
@@ -91,6 +95,9 @@ namespace PrivaPub.Tests.Support
public Fanout Fanout { get; }
public RemotePosts RemotePosts { get; }
public ObjectRecords Records { get; }
public PollService Polls { get; }
public Reactions Reactions { get; }
public QuoteService Quotes { get; }
public TimelineService Timelines { get; }
public RelationshipService Relationships { get; }
public ReportService Reports { get; }
@@ -143,4 +150,9 @@ namespace PrivaPub.Tests.Support
public LocalizedString this[string name, params object[] arguments] => new(name, string.Format(name, arguments));
public IEnumerable<LocalizedString> GetAllStrings(bool includeParentCultures) => Enumerable.Empty<LocalizedString>();
}
public sealed class NoPreviews : ILinkPreviews
{
public Task Wanted(PrivaPub.Models.Post.Post post, CancellationToken token) => Task.CompletedTask;
}
}
@@ -73,6 +73,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
domain = Domain,
title = "PrivaPub",
version = Version,
api_versions = new { mastodon = 7 },
source_url = "https://git.thepra.dev/thepra/SocialPub",
description = Description,
usage = new { users = new { active_month = 0 } },
@@ -60,13 +60,38 @@ namespace PrivaPub.Api.Mastodon.Controllers
[HttpGet("/media/proxy/{signature}/{encoded}"), AllowAnonymous, ApiExplorerSettings(IgnoreApi = true)]
public async Task<IActionResult> Proxy(string signature, string encoded, CancellationToken token)
{
var (path, contentType) = await _proxy.Fetch(signature, encoded, token);
if (path == default)
var url = _proxy.Verified(signature, encoded);
if (url == default)
return NotFound();
Response.Headers["X-Content-Type-Options"] = "nosniff";
Response.Headers["Content-Security-Policy"] = "default-src 'none'; sandbox";
Response.Headers["Cache-Control"] = "public, max-age=604800";
return PhysicalFile(path, contentType);
if (_proxy.Cached(url) is { Path: not null } cached)
return PhysicalFile(cached.Path, cached.ContentType, enableRangeProcessing: true);
if (Request.Headers.Range.Count == 0)
{
var (path, contentType) = await _proxy.Fetch(signature, encoded, token);
if (path != default)
return PhysicalFile(path, contentType, enableRangeProcessing: true);
}
return await Stream(url, token);
}
async Task<IActionResult> Stream(string url, CancellationToken token)
{
var range = System.Net.Http.Headers.RangeHeaderValue.TryParse(Request.Headers.Range.ToString(), out var asked) ? asked : default;
using var upstream = await _proxy.Open(url, range, token);
if (upstream == default)
return NotFound();
Response.StatusCode = (int)upstream.StatusCode;
Response.ContentType = upstream.Content.Headers.ContentType?.ToString() ?? "application/octet-stream";
if (upstream.Content.Headers.ContentLength is { } length)
Response.ContentLength = length;
if (upstream.Content.Headers.ContentRange is { } contentRange)
Response.Headers.ContentRange = contentRange.ToString();
Response.Headers.AcceptRanges = "bytes";
await upstream.Content.CopyToAsync(Response.Body, token);
return new EmptyResult();
}
object View(MediaAttachment attachment) => new
@@ -0,0 +1,54 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Post;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class PollsController : MastodonController
{
readonly DbEntities _dbEntities;
readonly MastodonMapper _mapper;
readonly IPollService _polls;
public PollsController(DbEntities dbEntities, MastodonMapper mapper, IPollService polls)
{
_dbEntities = dbEntities;
_mapper = mapper;
_polls = polls;
}
[HttpGet("/api/v1/polls/{id}"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
public async Task<IActionResult> Get(string id, CancellationToken token)
{
var post = await Visible(id, token);
return post?.Poll == default ? NotFoundError() : Json(_mapper.Poll(post, MyId, await _polls.OwnVotes(post.ID, MyId, token)));
}
[HttpPost("/api/v1/polls/{id}/votes"), Scope("write:statuses")]
public async Task<IActionResult> Vote(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post?.Poll == default)
return NotFoundError();
var choices = Params.List("choices").Select(c => int.TryParse(c, out var index) ? index : -1).ToList();
var outcome = await _polls.Vote(Me, post, choices, token);
if (!outcome.Ok)
return Error(outcome.Status, outcome.Error);
return Json(_mapper.Poll(outcome.Post, MyId, await _polls.OwnVotes(post.ID, MyId, token)));
}
async Task<PostEntity> Visible(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
return post != default && post.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(post, MyId, token) ? post : default;
}
}
}
@@ -0,0 +1,76 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Privacy;
using PrivaPub.Domain.Social;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Api.Mastodon.Controllers
{
public class ReactionsController : MastodonController
{
readonly DbEntities _dbEntities;
readonly MastodonMapper _mapper;
readonly IReactions _reactions;
public ReactionsController(DbEntities dbEntities, MastodonMapper mapper, IReactions reactions)
{
_dbEntities = dbEntities;
_mapper = mapper;
_reactions = reactions;
}
[HttpGet("/api/v1/pleroma/statuses/{id}/reactions"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
public async Task<IActionResult> List(string id, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
var reactions = await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID).ExecuteAsync(token);
var accounts = await _mapper.Accounts(reactions.Select(r => r.AccountId), token);
return Json(reactions.GroupBy(r => r.Emoji).Select(group => new
{
name = group.Key,
count = group.Count(),
me = MyId != default && group.Any(r => r.AccountId == MyId),
url = _mapper.ProxiedUrl(group.First().EmojiURL),
accounts = group.Select(r => accounts.GetValueOrDefault(r.AccountId)).Where(a => a != default).ToList()
}).OrderByDescending(r => r.count).ToList());
}
[HttpPut("/api/v1/pleroma/statuses/{id}/reactions/{emoji}"), Scope("write:favourites")]
public async Task<IActionResult> React(string id, string emoji, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
if (!await _reactions.React(Me, post, Uri.UnescapeDataString(emoji), token))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: That emoji cannot be used here");
return Json(await _mapper.Status(post, MyId, token));
}
[HttpDelete("/api/v1/pleroma/statuses/{id}/reactions/{emoji}"), Scope("write:favourites")]
public async Task<IActionResult> Unreact(string id, string emoji, CancellationToken token)
{
var post = await Visible(id, token);
if (post == default)
return NotFoundError();
await _reactions.Unreact(Me, post, Uri.UnescapeDataString(emoji), token);
return Json(await _mapper.Status(post, MyId, token));
}
async Task<PostEntity> Visible(string id, CancellationToken token)
{
var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token);
return post != default && post.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(post, MyId, token) ? post : default;
}
}
}
@@ -44,9 +44,6 @@ namespace PrivaPub.Api.Mastodon.Controllers
if (existing != default)
return Json(await _mapper.Status(existing, MyId, token));
}
if (Params.Has("poll[options]"))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: Polls are not supported yet");
var outcome = await _statuses.Publish(Me, new StatusDraft
{
Text = Params.Get("status"),
@@ -56,7 +53,12 @@ namespace PrivaPub.Api.Mastodon.Controllers
Visibility = Visibility(Params.Get("visibility") ?? Me.Settings.DefaultVisibility),
InReplyTo = Params.Get("in_reply_to_id"),
Language = Params.Get("language") ?? Me.Settings.DefaultLanguage,
MediaIds = Params.List("media_ids")
MediaIds = Params.List("media_ids"),
QuotedStatusId = Params.Get("quoted_status_id"),
Poll = Params.Has("poll[options]")
? new PollDraft(Params.List("poll[options]"), Params.Int("poll[expires_in]") ?? 0, Params.Bool("poll[multiple]") == true,
Params.Bool("poll[hide_totals]") == true)
: default
}, token);
if (!outcome.Ok)
return Error(outcome.Status, outcome.Error);
@@ -225,6 +227,21 @@ namespace PrivaPub.Api.Mastodon.Controllers
return Json(reblogs.Select(r => accounts.GetValueOrDefault(MastodonMapper.AuthorOf(r))).Where(a => a != default).ToList());
}
[HttpGet("/api/v1/statuses/{id}/quotes"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> Quotes(string id, CancellationToken token)
{
if (await Visible(id, token) == default)
return NotFoundError();
var query = _dbEntities.Posts.Match(p => p.QuotedPostId == id && p.QuoteState == QuoteState.Accepted && !p.DeletedAt.HasValue);
var quotes = await Page.From(Params, Limit()).Fetch(query, p => p.ID, token);
var visible = new List<PostEntity>();
foreach (var quote in quotes)
if (quote.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quote, MyId, token))
visible.Add(quote);
Link($"/api/v1/statuses/{id}/quotes", quotes.LastOrDefault()?.ID, quotes.FirstOrDefault()?.ID);
return Json(await _mapper.Statuses(visible, MyId, token));
}
[HttpPost("/api/v1/statuses/{id}/bookmark"), Scope("write:bookmarks")]
public async Task<IActionResult> Bookmark(string id, CancellationToken token)
{
@@ -162,7 +162,10 @@ namespace PrivaPub.Api.Mastodon.Controllers
[NotificationType.FollowRequest] = "follow_request",
[NotificationType.Favourite] = "favourite",
[NotificationType.Reblog] = "reblog",
[NotificationType.Update] = "update"
[NotificationType.Update] = "update",
[NotificationType.Poll] = "poll",
[NotificationType.Reaction] = "pleroma:emoji_reaction",
[NotificationType.Quote] = "quote"
};
readonly MastodonMapper _mapper;
@@ -233,7 +236,9 @@ namespace PrivaPub.Api.Mastodon.Controllers
CreatedAt = MastodonJson.Time(n.CreatedAt),
GroupKey = $"ungrouped-{n.ID}",
Account = accounts[n.FromAccountId],
Status = n.PostId == default ? default : statuses[n.PostId]
Status = n.PostId == default ? default : statuses[n.PostId],
Emoji = n.Emoji == default ? default : n.EmojiURL == default ? n.Emoji : $":{n.Emoji}:",
EmojiUrl = _mapper.ProxiedUrl(n.EmojiURL)
})
.ToList();
}
+71 -4
View File
@@ -25,12 +25,24 @@ namespace PrivaPub.Api.Mastodon.Entities
public string LastStatusAt { get; set; }
public bool? HideCollections { get; set; }
public bool Noindex { get; set; } = true;
public List<object> Emojis { get; set; } = new();
public List<CustomEmojiEntity> Emojis { get; set; } = new();
public List<object> Roles { get; set; } = new();
public bool? Memorial { get; set; }
public string AvatarDescription { get; set; } = string.Empty;
public string HeaderDescription { get; set; } = string.Empty;
public List<Field> Fields { get; set; } = new();
public Source Source { get; set; }
}
public class CustomEmojiEntity
{
public string Shortcode { get; set; }
public string Url { get; set; }
public string StaticUrl { get; set; }
public bool VisibleInPicker { get; set; }
public string Category { get; set; }
}
public class Field
{
public string Name { get; set; }
@@ -149,6 +161,54 @@ namespace PrivaPub.Api.Mastodon.Entities
public int Down { get; set; }
}
public class QuoteEntity
{
public string State { get; set; }
public Status QuotedStatus { get; set; }
public string QuotedStatusId { get; set; }
}
public class QuoteApprovalEntity
{
public List<string> Automatic { get; set; } = new();
public List<string> Manual { get; set; } = new();
public string CurrentUser { get; set; } = "denied";
}
public class EmojiReactionEntity
{
public string Name { get; set; }
public int Count { get; set; }
public bool Me { get; set; }
public string Url { get; set; }
public string StaticUrl { get; set; }
}
public class PleromaStatus
{
public List<EmojiReactionEntity> EmojiReactions { get; set; } = new();
}
public class PollEntity
{
public string Id { get; set; }
public string ExpiresAt { get; set; }
public bool Expired { get; set; }
public bool Multiple { get; set; }
public int VotesCount { get; set; }
public int? VotersCount { get; set; }
public bool? Voted { get; set; }
public List<int> OwnVotes { get; set; }
public List<PollOptionEntity> Options { get; set; } = new();
public List<CustomEmojiEntity> Emojis { get; set; } = new();
}
public class PollOptionEntity
{
public string Title { get; set; }
public int? VotesCount { get; set; }
}
public class PreviewCard
{
public string Url { get; set; }
@@ -183,7 +243,7 @@ namespace PrivaPub.Api.Mastodon.Entities
public string Content { get; set; } = string.Empty;
public string CreatedAt { get; set; }
public string EditedAt { get; set; }
public List<object> Emojis { get; set; } = new();
public List<CustomEmojiEntity> Emojis { get; set; } = new();
public int RepliesCount { get; set; }
public int ReblogsCount { get; set; }
public int FavouritesCount { get; set; }
@@ -199,12 +259,17 @@ namespace PrivaPub.Api.Mastodon.Entities
public List<StatusMention> Mentions { get; set; } = new();
public List<StatusTag> Tags { get; set; } = new();
public PreviewCard Card { get; set; }
public object Poll { get; set; }
public PollEntity Poll { get; set; }
public Application Application { get; set; }
public string Language { get; set; }
public string Text { get; set; }
public List<object> Filtered { get; set; } = new();
public PrivaPubStatus Privapub { get; set; }
public List<EmojiReactionEntity> EmojiReactions { get; set; } = new();
public PleromaStatus Pleroma { get; set; }
public QuoteEntity Quote { get; set; }
public int QuotesCount { get; set; }
public QuoteApprovalEntity QuoteApproval { get; set; }
}
public class MediaAttachment
@@ -273,6 +338,8 @@ namespace PrivaPub.Api.Mastodon.Entities
public string GroupKey { get; set; }
public Account Account { get; set; }
public Status Status { get; set; }
public string Emoji { get; set; }
public string EmojiUrl { get; set; }
}
public class Context
@@ -289,7 +356,7 @@ namespace PrivaPub.Api.Mastodon.Entities
public string CreatedAt { get; set; }
public Account Account { get; set; }
public List<MediaAttachment> MediaAttachments { get; set; } = new();
public List<object> Emojis { get; set; } = new();
public List<CustomEmojiEntity> Emojis { get; set; } = new();
}
public class StatusSource
+149 -6
View File
@@ -7,7 +7,10 @@ using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Group;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
using PrivaPub.Domain.Statuses;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
@@ -35,6 +38,8 @@ namespace PrivaPub.Api.Mastodon.Mappers
string Proxied(string url) => _proxy == default ? url : _proxy.Wrap(url);
public string ProxiedUrl(string url) => Proxied(url);
string MissingAvatar => $"{_localActors.BaseAddress}/media/missing-avatar.png";
string MissingHeader => $"{_localActors.BaseAddress}/media/missing-header.png";
@@ -103,8 +108,15 @@ namespace PrivaPub.Api.Mastodon.Mappers
DisplayName = foreign.Name ?? foreign.UserName ?? string.Empty,
Bot = foreign.AvatarType is AvatarType.Service or AvatarType.Application,
Group = foreign.AvatarType == AvatarType.Group,
Locked = foreign.IsLocked,
Discoverable = foreign.IsDiscoverable,
CreatedAt = MastodonJson.Day(foreign.CreatedAt),
Indexable = foreign.IsIndexable,
Memorial = foreign.IsMemorial ? true : default(bool?),
CreatedAt = MastodonJson.Day(foreign.Published ?? foreign.CreatedAt),
Emojis = Emojis(foreign.Emojis),
Fields = foreign.Fields.Select(f => new Field { Name = f.Key, Value = f.Value }).ToList(),
AvatarDescription = foreign.PictureDescription ?? string.Empty,
HeaderDescription = foreign.HeaderDescription ?? string.Empty,
Note = foreign.Biography ?? string.Empty,
Url = foreign.Url ?? foreign.ActorURI,
Uri = foreign.ActorURI,
@@ -136,7 +148,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
public async Task<Status> Status(PostEntity post, string viewerId, CancellationToken token) =>
(await Statuses(new[] { post }, viewerId, token)).FirstOrDefault();
public async Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, CancellationToken token)
public Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, CancellationToken token) =>
Statuses(posts, viewerId, nested: false, token);
async Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, bool nested, CancellationToken token)
{
var originalIds = posts.Where(p => p.ReblogOfPostId != default).Select(p => p.ReblogOfPostId).Distinct().ToList();
var originals = originalIds.Count == 0
@@ -158,6 +173,24 @@ namespace PrivaPub.Api.Mastodon.Mappers
? new HashSet<string>()
: (await DB.Default.Find<Bookmark>().Match(b => b.AvatarId == viewerId && ids.Contains(b.PostId)).ExecuteAsync(token)).Select(b => b.PostId).ToHashSet();
var pinned = (await DB.Default.Find<Pin>().Match(p => ids.Contains(p.PostId)).ExecuteAsync(token)).Select(p => p.PostId).ToHashSet();
var reactions = (await DB.Default.Find<Reaction>().Match(r => ids.Contains(r.PostId)).ExecuteAsync(token))
.GroupBy(r => r.PostId)
.ToDictionary(g => g.Key, g => g.GroupBy(r => r.Emoji)
.Select(e => new EmojiReactionEntity
{
Name = e.Key,
Count = e.Count(),
Me = viewerId != default && e.Any(r => r.AccountId == viewerId),
Url = Proxied(e.First().EmojiURL),
StaticUrl = Proxied(e.First().EmojiURL)
})
.OrderByDescending(e => e.Count)
.ToList());
var pollIds = all.Where(p => p.Poll != default).Select(p => p.ID).ToList();
var ownVotes = viewerId == default || pollIds.Count == 0
? new Dictionary<string, List<int>>()
: (await DB.Default.Find<PollVote>().Match(v => pollIds.Contains(v.PostId) && v.VoterAccountId == viewerId && v.IsLocalVoter).ExecuteAsync(token))
.GroupBy(v => v.PostId).ToDictionary(g => g.Key, g => g.Select(v => v.Choice).ToList());
var hidden = viewerId == default
? new HashSet<string>()
: await Domain.Relationships.Hidden.AuthorsHiddenFrom(viewerId, all.Select(p => p.ActorURI), forNotifications: false, token);
@@ -192,15 +225,43 @@ namespace PrivaPub.Api.Mastodon.Mappers
SpoilerText = post.SpoilerText ?? (post.HasContentWarning ? post.Title ?? ActivityPubRenderer.ContentWarning : string.Empty),
Visibility = Visibility(post.Visibility),
Language = post.Language,
MediaAttachments = post.Media.Select(Media).ToList(),
MediaAttachments = post.Media.Count > 0 ? post.Media.Select(Media).ToList() : Playable(post),
Card = Card(post),
Poll = Poll(post, viewerId, ownVotes.GetValueOrDefault(post.ID)),
Emojis = Emojis(post.Emojis),
Privapub = Extension(post),
EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new(),
Pleroma = new PleromaStatus { EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new() },
Mentions = post.Mentions.Select(m => Mention(m, mentionAccounts)).Where(m => m != default).ToList(),
Tags = post.Tags.Select(t => new StatusTag { Name = t, Url = ActivityPubRenderer.TagUrl(_localActors.BaseAddress, t) }).ToList()
};
return status;
}
var quotedIds = nested ? new List<string>() : all.Where(p => p.QuoteState == QuoteState.Accepted && p.QuotedPostId != default).Select(p => p.QuotedPostId).Distinct().ToList();
var quotedPosts = new List<PostEntity>();
foreach (var quotedPost in quotedIds.Count == 0 ? new List<PostEntity>() : await _dbEntities.Posts.Match(p => quotedIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token))
if (quotedPost.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quotedPost, viewerId, token))
quotedPosts.Add(quotedPost);
var quotedStatuses = quotedPosts.Count == 0
? new Dictionary<string, Status>()
: (await Statuses(quotedPosts, viewerId, nested: true, token)).ToDictionary(q => q.Id);
void Quote(Status status, PostEntity post)
{
status.QuotesCount = post.QuotesCount;
status.QuoteApproval = Approval(post, viewerId);
if (post.QuoteState == QuoteState.None)
return;
var state = post.QuoteState.ToString().ToLowerInvariant();
var quoted = post.QuotedPostId != default && quotedStatuses.TryGetValue(post.QuotedPostId, out var shown) ? shown : default;
status.Quote = nested
? new QuoteEntity { State = state, QuotedStatusId = post.QuotedPostId }
: new QuoteEntity { State = state, QuotedStatus = post.QuoteState == QuoteState.Accepted ? quoted : default, QuotedStatusId = post.QuotedPostId };
if (!nested && post.QuoteState == QuoteState.Accepted && quoted != default)
status.Content = WithoutQuoteFallback(status.Content);
}
var mapped = new List<Status>();
foreach (var post in posts)
{
@@ -210,10 +271,12 @@ namespace PrivaPub.Api.Mastodon.Mappers
var status = Map(post);
if (status == default)
continue;
Quote(status, post);
if (post.ReblogOfPostId != default)
{
if (!originals.TryGetValue(post.ReblogOfPostId, out var original) || Map(original) is not { } inner)
continue;
Quote(inner, original);
status.Reblog = inner;
status.Content = string.Empty;
status.Reblogged = reblogged.Contains(original.ID);
@@ -225,6 +288,33 @@ namespace PrivaPub.Api.Mastodon.Mappers
public static string AuthorOf(PostEntity post) => post.AuthorAccountId ?? post.GroupUserId;
static readonly AngleSharp.Html.Parser.HtmlParser Fragments = new();
public static string WithoutQuoteFallback(string html)
{
if (string.IsNullOrEmpty(html) || !html.Contains("quote-inline", StringComparison.Ordinal))
return html;
var document = Fragments.ParseDocument($"<body>{html}</body>");
foreach (var fallback in document.QuerySelectorAll(".quote-inline").ToList())
fallback.Remove();
return document.Body!.InnerHtml;
}
static QuoteApprovalEntity Approval(PostEntity post, string viewerId)
{
if (!post.IsFederatedCopy || post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
return new QuoteApprovalEntity();
if (post.QuotePolicy is not { } policy)
return new QuoteApprovalEntity { Automatic = new List<string> { "public" }, CurrentUser = viewerId == default ? "unknown" : "automatic" };
static List<string> Named(IEnumerable<string> who) => who.Select(w => Addressing.IsPublic(w) ? "public" : w.EndsWith("/followers") ? "followers" : "unsupported_policy").Distinct().ToList();
return new QuoteApprovalEntity
{
Automatic = Named(policy.Automatic),
Manual = Named(policy.Manual),
CurrentUser = viewerId == default ? "unknown" : policy.Automatic.Any(Addressing.IsPublic) ? "automatic" : policy.Manual.Any(Addressing.IsPublic) ? "manual" : "denied"
};
}
public static string Content(PostEntity post)
{
var content = post.ContentHtml ?? ActivityPubRenderer.Html(post.Text);
@@ -270,6 +360,30 @@ namespace PrivaPub.Api.Mastodon.Mappers
}
}
public PollEntity Poll(PostEntity post, string viewerId, IReadOnlyCollection<int> ownVotes)
{
if (post.Poll is not { } poll)
return default;
var expired = PollService.Ended(poll);
var hidden = poll.HideTotals && !expired && post.GroupUserId != viewerId;
return new PollEntity
{
Id = post.ID,
ExpiresAt = poll.ExpiresAt is { } ends ? MastodonJson.Time(ends) : default,
Expired = expired,
Multiple = poll.Multiple,
VotesCount = poll.Options.Sum(o => o.Votes),
VotersCount = poll.Multiple ? poll.VotersCount : poll.VotersCount ?? poll.Options.Sum(o => o.Votes),
Voted = viewerId == default ? default(bool?) : ownVotes?.Count > 0 || post.GroupUserId == viewerId,
OwnVotes = viewerId == default ? default : ownVotes?.ToList() ?? new List<int>(),
Options = poll.Options.Select(o => new PollOptionEntity { Title = o.Title, VotesCount = hidden ? default(int?) : o.Votes }).ToList(),
Emojis = Emojis(post.Emojis.Where(e => poll.Options.Any(o => o.Title.Contains($":{e.Shortcode}:"))))
};
}
List<CustomEmojiEntity> Emojis(IEnumerable<CustomEmoji> emojis) =>
emojis?.Select(e => new CustomEmojiEntity { Shortcode = e.Shortcode, Url = Proxied(e.URL), StaticUrl = Proxied(e.URL) }).ToList() ?? new();
PrivaPubStatus Extension(PostEntity post) => new()
{
ObjectType = post.ObjectType,
@@ -331,13 +445,42 @@ namespace PrivaPub.Api.Mastodon.Mappers
Streams = variant.Streams
};
List<MediaAttachment> Playable(PostEntity post)
{
var (variant, kind, duration) = post.Video is { } video ? (BestVideo(video.Variants), "video", video.DurationSeconds)
: post.Audio is { } audio ? (audio.Variants.FirstOrDefault(), "audio", audio.DurationSeconds)
: default;
if (variant == default)
return new List<MediaAttachment>();
return new List<MediaAttachment>
{
Media(new PostMedia
{
Id = new Guid(System.Security.Cryptography.MD5.HashData(System.Text.Encoding.UTF8.GetBytes(variant.Url))),
RemoteURL = variant.Url,
ContentType = variant.MediaType ?? kind + "/*",
RemotePreviewURL = post.CoverURL,
Width = variant.Width,
Height = variant.Height,
DurationSeconds = duration,
Description = post.Title
})
};
}
public static MediaVariant BestVideo(IReadOnlyCollection<MediaVariant> variants)
{
var playable = variants.Where(v => v.MediaType?.StartsWith("video/") == true && (v.Streams.Count == 0 || v.Streams.Contains("audio") && v.Streams.Contains("video"))).ToList();
return playable.Where(v => v.Height <= 720).OrderByDescending(v => v.Height).FirstOrDefault()
?? playable.OrderBy(v => v.Height ?? int.MaxValue).FirstOrDefault();
}
PreviewCard Card(PostEntity post)
{
if (!post.IsFederatedCopy)
return default;
if (post.Link is { } link)
return Card(link.Href, link.Title ?? post.Title, link.Description ?? (post.Media.Count == 0 ? post.Excerpt : default), link.ImageURL);
if (post.ObjectType is "Article" or "Event" or "Video" or "Audio" && post.Url != default && post.Media.Count == 0)
if (post.IsFederatedCopy && post.ObjectType is "Article" or "Event" && post.Url != default && post.Media.Count == 0
|| post.IsFederatedCopy && post.ObjectType is "Video" or "Audio" && post.Url != default && post.Media.Count == 0 && Playable(post).Count == 0)
return Card(post.Url, post.Title, post.Excerpt, post.CoverURL);
return default;
}
+155
View File
@@ -0,0 +1,155 @@
using AngleSharp.Html.Parser;
using Microsoft.Extensions.Options;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.StaticServices;
using System.Security.Cryptography;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Content
{
public class LinkPreview : Entity
{
public string Url { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public string ImageURL { get; set; }
public string SiteName { get; set; }
public bool Failed { get; set; }
public DateTime FetchedAt { get; set; } = DateTime.UtcNow;
}
public interface ILinkPreviews
{
Task Wanted(PostEntity post, CancellationToken token);
}
public class LinkPreviews : ILinkPreviews, IJobHandler
{
const int MaxJitterSeconds = 60;
static readonly TimeSpan Freshness = TimeSpan.FromDays(7);
static readonly HtmlParser Parser = new();
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IFederationHttp _http;
readonly IJobQueue _queue;
readonly IOptionsMonitor<FederationOptions> _options;
public LinkPreviews(DbEntities dbEntities, ILocalActorService localActors, IFederationHttp http, IJobQueue queue,
IOptionsMonitor<FederationOptions> options)
{
_dbEntities = dbEntities;
_localActors = localActors;
_http = http;
_queue = queue;
_options = options;
}
public JobKind Kind => JobKind.FetchPreview;
public int Concurrency => 2;
public int MaxAttempts => 1;
public int PerHostLimit => 1;
public async Task Wanted(PostEntity post, CancellationToken token)
{
if (!_options.CurrentValue.FetchLinkPreviews || post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted) || post.IsLocalOnly
|| post.ReblogOfPostId != default || post.Link?.Title != default || post.Link == default && post.Media.Count > 0)
return;
var url = post.Link?.Href ?? FirstLink(post, _localActors.BaseAddress);
if (url == default)
return;
await _queue.EnqueueMany(new[]
{
new Job
{
Kind = JobKind.FetchPreview, Payload = post.ID, Host = new Uri(url).Host.ToLowerInvariant(), DedupeKey = $"preview|{post.ID}",
RunAt = DateTime.UtcNow.AddSeconds(RandomNumberGenerator.GetInt32(0, MaxJitterSeconds + 1))
}
}, token);
}
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var post = await _dbEntities.Posts.MatchID(job.Payload).ExecuteFirstAsync(token);
if (post == default || post.DeletedAt.HasValue || post.Link?.Title != default)
return JobOutcome.Done;
var url = post.Link?.Href ?? FirstLink(post, _localActors.BaseAddress);
if (url == default)
return JobOutcome.Done;
var preview = await DB.Default.Find<LinkPreview>().Match(p => p.Url == url).ExecuteFirstAsync(token);
if (preview == default || preview.FetchedAt < DateTime.UtcNow - Freshness)
preview = await Fetch(url, token);
if (preview == default || preview.Failed)
return JobOutcome.Done;
var link = post.Link ?? new PostLink { Href = url };
link.Title ??= preview.Title;
link.Description ??= preview.Description;
link.ImageURL ??= preview.ImageURL;
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.Link, link).ExecuteAsync(token);
return JobOutcome.Done;
}
async Task<LinkPreview> Fetch(string url, CancellationToken token)
{
var (finalUri, html) = await _http.GetPage(url, token);
var preview = html == default ? new LinkPreview { Url = url, Failed = true } : Read(url, finalUri, html);
await DB.Default.Update<LinkPreview>()
.Match(p => p.Url == url)
.Modify(p => p.Title, preview.Title)
.Modify(p => p.Description, preview.Description)
.Modify(p => p.ImageURL, preview.ImageURL)
.Modify(p => p.SiteName, preview.SiteName)
.Modify(p => p.Failed, preview.Failed)
.Modify(p => p.FetchedAt, DateTime.UtcNow)
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
return preview;
}
public static LinkPreview Read(string url, Uri finalUri, string html)
{
var document = Parser.ParseDocument(html);
string Meta(params string[] names) => names
.Select(name => document.QuerySelector($"meta[property='{name}'], meta[name='{name}']")?.GetAttribute("content"))
.FirstOrDefault(value => !string.IsNullOrWhiteSpace(value));
var image = Meta("og:image:secure_url", "og:image", "og:image:url", "twitter:image", "twitter:image:src");
var imageUri = image != default && Uri.TryCreate(finalUri, image, out var resolved) && resolved.Scheme is "https" or "http" ? resolved.AbsoluteUri : default;
var preview = new LinkPreview
{
Url = url,
Title = ObjectShapes.Text(Meta("og:title", "twitter:title") ?? document.Title, 300),
Description = ObjectShapes.Text(Meta("og:description", "twitter:description", "description"), 1000),
SiteName = ObjectShapes.Text(Meta("og:site_name"), 200),
ImageURL = imageUri
};
preview.Failed = preview.Title == default && preview.Description == default && preview.ImageURL == default;
return preview;
}
public static string FirstLink(PostEntity post, string ownBase)
{
if (string.IsNullOrEmpty(post.ContentHtml))
return default;
var document = Parser.ParseDocument(post.ContentHtml);
return document.QuerySelectorAll("a[href]")
.Where(a => !(a.ClassList.Contains("mention") || a.ClassList.Contains("hashtag") || a.GetAttribute("rel")?.Contains("tag") == true
|| a.Closest(".quote-inline") != default))
.Select(a => a.GetAttribute("href"))
.FirstOrDefault(href => Uri.TryCreate(href, UriKind.Absolute, out var uri) && uri.Scheme is "https" or "http"
&& href != post.Url && href != post.ObjectURI && !href.StartsWith(ownBase + "/", StringComparison.OrdinalIgnoreCase));
}
}
}
+33 -11
View File
@@ -15,6 +15,9 @@ namespace PrivaPub.Domain.Media
{
string Wrap(string remoteUrl);
Task<(string Path, string ContentType)> Fetch(string signature, string encodedUrl, CancellationToken token);
string Verified(string signature, string encodedUrl);
(string Path, string ContentType) Cached(string url);
Task<HttpResponseMessage> Open(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token);
}
public class MediaProxy : IMediaProxy
@@ -43,7 +46,7 @@ namespace PrivaPub.Domain.Media
return $"{_localActors.BaseAddress}/media/proxy/{Sign(remoteUrl)}/{encoded}";
}
public async Task<(string Path, string ContentType)> Fetch(string signature, string encodedUrl, CancellationToken token)
public string Verified(string signature, string encodedUrl)
{
string url;
try
@@ -54,18 +57,37 @@ namespace PrivaPub.Domain.Media
{
return default;
}
if (!CryptographicOperations.FixedTimeEquals(Encoding.ASCII.GetBytes(signature ?? string.Empty), Encoding.ASCII.GetBytes(Sign(url))))
return default;
return CryptographicOperations.FixedTimeEquals(Encoding.ASCII.GetBytes(signature ?? string.Empty), Encoding.ASCII.GetBytes(Sign(url))) ? url : default;
}
public (string Path, string ContentType) Cached(string url)
{
var (path, typePath) = CachePaths(url);
if (!File.Exists(path) || !File.Exists(typePath))
return default;
File.SetLastWriteTimeUtc(path, DateTime.UtcNow);
return (path, File.ReadAllText(typePath));
}
public Task<HttpResponseMessage> Open(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token) =>
_http.OpenMedia(url, range, token);
(string Path, string TypePath) CachePaths(string url)
{
var name = Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(url)));
var directory = System.IO.Path.Combine(_media.ProxyRoot, name[..2]);
var path = System.IO.Path.Combine(directory, name);
var typePath = path + ".type";
if (File.Exists(path) && File.Exists(typePath))
{
File.SetLastWriteTimeUtc(path, DateTime.UtcNow);
return (path, await File.ReadAllTextAsync(typePath, token));
}
var path = System.IO.Path.Combine(_media.ProxyRoot, name[..2], name);
return (path, path + ".type");
}
public async Task<(string Path, string ContentType)> Fetch(string signature, string encodedUrl, CancellationToken token)
{
var url = Verified(signature, encodedUrl);
if (url == default)
return default;
if (Cached(url) is { Path: not null } cached)
return cached;
var (path, typePath) = CachePaths(url);
var directory = System.IO.Path.GetDirectoryName(path);
var (bytes, contentType) = await _http.GetMedia(url, _options.CurrentValue.MaxProxiedBytes, token);
if (bytes == default)
+5 -3
View File
@@ -8,9 +8,9 @@ namespace PrivaPub.Domain.Social
public static class Notifications
{
public static async Task Add(string avatarId, NotificationType type, string fromAccountId, string fromActorUri, string postId,
CancellationToken token)
CancellationToken token, string emoji = default, string emojiUrl = default)
{
if (string.IsNullOrEmpty(avatarId) || avatarId == fromAccountId)
if (string.IsNullOrEmpty(avatarId) || avatarId == fromAccountId && type != NotificationType.Poll)
return;
if ((await Relationships.Hidden.AuthorsHiddenFrom(avatarId, new[] { fromActorUri }, forNotifications: true, token)).Count > 0)
return;
@@ -23,7 +23,9 @@ namespace PrivaPub.Domain.Social
FromAccountId = fromAccountId,
FromActorURI = fromActorUri,
PostId = postId,
DedupeKey = $"{type}|{avatarId}|{fromActorUri}|{postId}"
Emoji = emoji,
EmojiURL = emojiUrl,
DedupeKey = emoji == default ? $"{type}|{avatarId}|{fromActorUri}|{postId}" : $"{type}|{avatarId}|{fromActorUri}|{postId}|{emoji}"
}, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
+179
View File
@@ -0,0 +1,179 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Globalization;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Social
{
public interface IReactions
{
Task Receive(PostEntity post, ForeignAvatar actor, string content, JsonNode tags, string activityId, CancellationToken token);
Task<bool> Withdraw(ForeignAvatar actor, string activityId, string objectUri, string content, CancellationToken token);
Task<bool> React(LocalActor me, PostEntity post, string emoji, CancellationToken token);
Task<bool> Unreact(LocalActor me, PostEntity post, string emoji, CancellationToken token);
}
public class Reactions : IReactions
{
static readonly string[] Hearts = { "❤", "❤️", "♥", "♥️" };
readonly DbEntities _dbEntities;
readonly IDeliveryService _delivery;
public Reactions(DbEntities dbEntities, IDeliveryService delivery)
{
_dbEntities = dbEntities;
_delivery = delivery;
}
public static bool IsHeart(string content) => content == default || Hearts.Contains(content.Trim());
public async Task Receive(PostEntity post, ForeignAvatar actor, string content, JsonNode tags, string activityId, CancellationToken token)
{
if (!LikeHandler.MaySee(post, actor) || Normalise(content, ObjectShapes.Emojis(tags)) is not { } emoji)
return;
try
{
await DB.Default.SaveAsync(new Reaction
{
PostId = post.ID, AccountId = actor.ID, ActorURI = actor.ActorURI, Emoji = emoji.Name, EmojiURL = emoji.Url, ActivityURI = activityId
}, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return;
}
if (!post.IsFederatedCopy)
await Notifications.Add(post.GroupUserId, NotificationType.Reaction, actor.ID, actor.ActorURI, post.ID, token, emoji.Name, emoji.Url);
}
public async Task<bool> Withdraw(ForeignAvatar actor, string activityId, string objectUri, string content, CancellationToken token)
{
var reaction = activityId == default
? default
: await DB.Default.Find<Reaction>().Match(r => r.ActivityURI == activityId && r.ActorURI == actor.ActorURI).ExecuteFirstAsync(token);
if (reaction == default && objectUri != default && Normalise(content, new()) is { } emoji)
{
var post = await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri).ExecuteFirstAsync(token);
reaction = post == default
? default
: await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.ActorURI == actor.ActorURI && r.Emoji == emoji.Name).ExecuteFirstAsync(token);
}
if (reaction == default)
return false;
await DB.Default.DeleteAsync<Reaction>(reaction.ID);
return true;
}
public async Task<bool> React(LocalActor me, PostEntity post, string emoji, CancellationToken token)
{
var known = post.Emojis.Select(e => (e.Shortcode, e.URL))
.Concat((await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.EmojiURL != null).ExecuteAsync(token)).Select(r => (r.Emoji, r.EmojiURL)))
.Select(e => new Models.Post.CustomEmoji { Shortcode = e.Item1, URL = e.Item2 })
.ToList();
if (Normalise(emoji, known) is not { } picked)
return false;
var reaction = new Reaction
{
PostId = post.ID, AccountId = me.Id, ActorURI = me.Uri, IsLocal = true, Emoji = picked.Name, EmojiURL = picked.Url,
ActivityURI = me.ActivityUri($"react-{post.ID}-{Guid.NewGuid():N}")
};
try
{
await DB.Default.SaveAsync(reaction, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return true;
}
if (!post.IsFederatedCopy)
await Notifications.Add(post.GroupUserId, NotificationType.Reaction, me.Id, me.Uri, post.ID, token, picked.Name, picked.Url);
else
await Tell(me, post, Activity(me, post, reaction), token);
return true;
}
public async Task<bool> Unreact(LocalActor me, PostEntity post, string emoji, CancellationToken token)
{
var name = Normalise(emoji, post.Emojis.ToList(), requireKnownCustom: false)?.Name;
var reaction = name == default
? default
: await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.ActorURI == me.Uri && r.Emoji == name).ExecuteFirstAsync(token);
if (reaction == default)
return false;
await DB.Default.DeleteAsync<Reaction>(reaction.ID);
if (post.IsFederatedCopy)
await Tell(me, post, new JsonObject
{
["@context"] = ActivityPubRenderer.Context(),
["id"] = me.ActivityUri($"undo-{reaction.ID}"),
["type"] = "Undo",
["actor"] = me.Uri,
["object"] = Activity(me, post, reaction)
}, token);
return true;
}
static JsonObject Activity(LocalActor me, PostEntity post, Reaction reaction)
{
var react = new JsonObject
{
["@context"] = ActivityPubRenderer.Context(),
["id"] = reaction.ActivityURI,
["type"] = "EmojiReact",
["actor"] = me.Uri,
["object"] = post.ObjectURI,
["to"] = new JsonArray(post.ActorURI),
["content"] = reaction.EmojiURL == default ? reaction.Emoji : $":{reaction.Emoji}:"
};
if (reaction.EmojiURL != default)
react["tag"] = new JsonArray(new JsonObject
{
["type"] = "Emoji",
["name"] = $":{reaction.Emoji}:",
["icon"] = new JsonObject { ["type"] = "Image", ["url"] = reaction.EmojiURL }
});
return react;
}
async Task Tell(LocalActor me, PostEntity post, JsonObject activity, CancellationToken token)
{
var author = await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == post.ActorURI).ExecuteFirstAsync(token);
if (!string.IsNullOrEmpty(author?.InboxURL))
await _delivery.Enqueue(me, new[] { author.InboxURL }, activity, token);
}
public sealed record Picked(string Name, string Url);
public static Picked Normalise(string content, List<Models.Post.CustomEmoji> emojis, bool requireKnownCustom = true)
{
var text = content?.Trim();
if (string.IsNullOrEmpty(text) || text.Length > 100)
return default;
if (text.StartsWith(':') && text.EndsWith(':') && text.Length > 2)
{
var shortcode = text.Trim(':');
var bare = shortcode.Split('@')[0];
var emoji = emojis.FirstOrDefault(e => e.Shortcode == shortcode) ?? emojis.FirstOrDefault(e => e.Shortcode.Split('@')[0] == bare);
if (emoji == default)
return requireKnownCustom ? default : new Picked(bare, default);
return new Picked(bare, emoji.URL);
}
if (text.Any(c => c < 128 || char.IsWhiteSpace(c)) || new StringInfo(text).LengthInTextElements != 1)
return default;
return new Picked(text, default);
}
}
}
+281
View File
@@ -0,0 +1,281 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox.Handlers;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Infrastructure.Jobs;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Jobs;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Statuses
{
public sealed record PollDraft(IReadOnlyList<string> Options, int ExpiresIn, bool Multiple, bool HideTotals);
public interface IPollService
{
string Invalid(PollDraft draft);
PostPoll Create(PollDraft draft);
Task Scheduled(PostEntity post, CancellationToken token);
Task<StatusOutcome> Vote(LocalActor me, PostEntity post, IReadOnlyList<int> choices, CancellationToken token);
Task Receive(PostEntity post, ForeignAvatar voter, string choice, string activityId, CancellationToken token);
Task<HashSet<int>> OwnVotes(string postId, string avatarId, CancellationToken token);
}
public class PollService : IPollService
{
public const int MaxOptions = 4;
public const int MaxOptionLength = 50;
public const int MinExpiration = 300;
public const int MaxExpiration = 2_629_746;
static readonly TimeSpan RefreshEvery = TimeSpan.FromMinutes(3);
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IDeliveryService _delivery;
readonly IJobQueue _queue;
public PollService(DbEntities dbEntities, ILocalActorService localActors, IDeliveryService delivery, IJobQueue queue)
{
_dbEntities = dbEntities;
_localActors = localActors;
_delivery = delivery;
_queue = queue;
}
public string Invalid(PollDraft draft)
{
var options = draft.Options.Select(o => o?.Trim()).ToList();
if (options.Count is < 2 or > MaxOptions)
return $"Validation failed: A poll needs between 2 and {MaxOptions} options";
if (options.Any(o => string.IsNullOrEmpty(o) || o.Length > MaxOptionLength))
return $"Validation failed: Poll options must be between 1 and {MaxOptionLength} characters";
if (options.Distinct(StringComparer.Ordinal).Count() != options.Count)
return "Validation failed: Poll options must be unique";
if (draft.ExpiresIn is < MinExpiration or > MaxExpiration)
return "Validation failed: The poll's duration is out of range";
return default;
}
public PostPoll Create(PollDraft draft) => new()
{
Options = draft.Options.Select(o => new PollOption { Title = o.Trim() }).ToList(),
Multiple = draft.Multiple,
HideTotals = draft.HideTotals,
ExpiresAt = DateTime.UtcNow.AddSeconds(draft.ExpiresIn),
VotersCount = 0
};
public async Task Scheduled(PostEntity post, CancellationToken token)
{
if (post.Poll?.ExpiresAt is not { } ends)
return;
await _queue.EnqueueMany(new[]
{
new Job { Kind = JobKind.PollClose, Payload = post.ID, Host = LocalHost, DedupeKey = $"poll-close|{post.ID}", RunAt = ends }
}, token);
}
public async Task<StatusOutcome> Vote(LocalActor me, PostEntity post, IReadOnlyList<int> choices, CancellationToken token)
{
var poll = post.Poll;
if (poll == default)
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
if (Ended(poll))
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: The poll has already ended");
if (!post.IsFederatedCopy && post.GroupUserId == me.Id)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: You cannot vote in your own poll");
var picked = choices.Distinct().ToList();
if (picked.Count == 0 || picked.Any(c => c < 0 || c >= poll.Options.Count) || !poll.Multiple && picked.Count > 1)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: The choices are invalid");
if (await DB.Default.Find<PollVote>().Match(v => v.PostId == post.ID && v.ActorURI == me.Uri).ExecuteAnyAsync(token))
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: You have already voted on this poll");
foreach (var choice in picked)
await DB.Default.SaveAsync(new PollVote
{
PostId = post.ID, VoterAccountId = me.Id, ActorURI = me.Uri, IsLocalVoter = true, Choice = choice,
ActivityURI = me.Uri + $"#votes/{post.ID}/{choice}"
}, token);
await Count(post, picked, newVoter: true, token);
if (post.IsFederatedCopy)
await SendVotes(me, post, picked, token);
else
await Refreshed(post, token);
await Scheduled(post, token);
return new StatusOutcome(await _dbEntities.Posts.MatchID(post.ID).ExecuteFirstAsync(token));
}
public async Task Receive(PostEntity post, ForeignAvatar voter, string choice, string activityId, CancellationToken token)
{
var poll = post.Poll;
if (poll == default || post.IsFederatedCopy || Ended(poll) || !LikeHandler.MaySee(post, voter))
return;
var index = poll.Options.FindIndex(o => o.Title == choice);
if (index < 0)
return;
var earlier = await DB.Default.Find<PollVote>().Match(v => v.PostId == post.ID && v.ActorURI == voter.ActorURI).ExecuteAsync(token);
if (earlier.Count > 0 && (!poll.Multiple || earlier.Any(v => v.Choice == index)))
return;
try
{
await DB.Default.SaveAsync(new PollVote
{
PostId = post.ID, VoterAccountId = voter.ID, ActorURI = voter.ActorURI, Choice = index, ActivityURI = activityId
}, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return;
}
await Count(post, new[] { index }, newVoter: earlier.Count == 0, token);
await Refreshed(post, token);
}
public async Task<HashSet<int>> OwnVotes(string postId, string avatarId, CancellationToken token) =>
avatarId == default
? new HashSet<int>()
: (await DB.Default.Find<PollVote>().Match(v => v.PostId == postId && v.VoterAccountId == avatarId && v.IsLocalVoter).ExecuteAsync(token))
.Select(v => v.Choice).ToHashSet();
public static bool Ended(PostPoll poll) => poll.ClosedAt.HasValue || poll.ExpiresAt <= DateTime.UtcNow;
async Task Count(PostEntity post, IReadOnlyList<int> choices, bool newVoter, CancellationToken token)
{
var update = DB.Default.Update<PostEntity>().MatchID(post.ID);
foreach (var choice in choices)
update = update.Modify(b => b.Inc($"{nameof(PostEntity.Poll)}.{nameof(PostPoll.Options)}.{choice}.{nameof(PollOption.Votes)}", 1));
if (newVoter && post.Poll.VotersCount.HasValue)
update = update.Modify(b => b.Inc($"{nameof(PostEntity.Poll)}.{nameof(PostPoll.VotersCount)}", 1));
await update.ExecuteAsync(token);
}
async Task SendVotes(LocalActor me, PostEntity post, IReadOnlyList<int> choices, CancellationToken token)
{
var owner = await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == post.ActorURI).ExecuteFirstAsync(token);
if (string.IsNullOrEmpty(owner?.InboxURL))
return;
foreach (var choice in choices)
{
var voteId = me.Uri + $"#votes/{post.ID}/{choice}";
await _delivery.Enqueue(me, new[] { owner.InboxURL }, new JsonObject
{
["@context"] = ActivityPubRenderer.ActivityStreams,
["id"] = voteId + "/activity",
["type"] = "Create",
["actor"] = me.Uri,
["to"] = new JsonArray(owner.ActorURI),
["object"] = new JsonObject
{
["id"] = voteId,
["type"] = "Note",
["name"] = post.Poll.Options[choice].Title,
["attributedTo"] = me.Uri,
["to"] = new JsonArray(owner.ActorURI),
["inReplyTo"] = post.ObjectURI
}
}, token);
}
}
async Task Refreshed(PostEntity post, CancellationToken token)
{
var bucket = DateTime.UtcNow.Ticks / RefreshEvery.Ticks;
await _queue.EnqueueMany(new[]
{
new Job
{
Kind = JobKind.PollRefresh, Payload = post.ID, Host = LocalHost, DedupeKey = $"poll-refresh|{post.ID}|{bucket}",
RunAt = new DateTime((bucket + 1) * RefreshEvery.Ticks, DateTimeKind.Utc)
}
}, token);
}
string LocalHost => new Uri(_localActors.BaseAddress).Host;
}
public class PollRefreshJob : IJobHandler
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IOutboxPublisher _outbox;
public PollRefreshJob(DbEntities dbEntities, ILocalActorService localActors, IOutboxPublisher outbox)
{
_dbEntities = dbEntities;
_localActors = localActors;
_outbox = outbox;
}
public virtual JobKind Kind => JobKind.PollRefresh;
public int Concurrency => 1;
public int MaxAttempts => 3;
public int PerHostLimit => 1;
public virtual async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var post = await _dbEntities.Posts.MatchID(job.Payload).ExecuteFirstAsync(token);
if (post?.Poll == default || post.IsFederatedCopy || post.DeletedAt.HasValue)
return JobOutcome.Done;
var author = await _localActors.FindById(LocalActorKind.Person, post.GroupUserId, token);
if (author == default)
return JobOutcome.Done;
await Closing(post, author, token);
if (!post.IsLocalOnly)
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, $"poll-{DateTime.UtcNow.Ticks}"), token);
return JobOutcome.Done;
}
protected virtual Task Closing(PostEntity post, LocalActor author, CancellationToken token) => Task.CompletedTask;
}
public class PollCloseJob : PollRefreshJob
{
readonly DbEntities _posts;
public PollCloseJob(DbEntities dbEntities, ILocalActorService localActors, IOutboxPublisher outbox) : base(dbEntities, localActors, outbox)
{
_posts = dbEntities;
}
public override JobKind Kind => JobKind.PollClose;
public override async Task<JobOutcome> Handle(Job job, CancellationToken token)
{
var post = await _posts.Posts.MatchID(job.Payload).ExecuteFirstAsync(token);
if (post?.Poll == default || post.DeletedAt.HasValue)
return JobOutcome.Done;
if (!post.IsFederatedCopy)
return await base.Handle(job, token);
await Notify(post, post.AuthorAccountId, post.ActorURI, token);
return JobOutcome.Done;
}
protected override async Task Closing(PostEntity post, LocalActor author, CancellationToken token)
{
post.Poll.ClosedAt ??= DateTime.UtcNow;
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.Poll.ClosedAt, post.Poll.ClosedAt).ExecuteAsync(token);
await Notify(post, author.Id, author.Uri, token);
}
public static async Task Notify(PostEntity post, string authorAccountId, string authorUri, CancellationToken token)
{
var voters = await DB.Default.Find<PollVote>().Match(v => v.PostId == post.ID && v.IsLocalVoter).ExecuteAsync(token);
foreach (var voter in voters.Select(v => v.VoterAccountId).Distinct())
await Notifications.Add(voter, NotificationType.Poll, authorAccountId, authorUri, post.ID, token);
if (!post.IsFederatedCopy)
await Notifications.Add(post.GroupUserId, NotificationType.Poll, post.GroupUserId, authorUri, post.ID, token);
}
}
}
+196
View File
@@ -0,0 +1,196 @@
using MongoDB.Entities;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.User;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Statuses
{
public interface IQuoteService
{
Task Resolve(PostEntity post, NoteDocument note, CancellationToken token);
Task Revoke(string stampUri, CancellationToken token);
Task<bool> Verified(string stampUri, string quotingUri, PostEntity quoted, CancellationToken token);
QuotePermission Permission(PostEntity quoted, LocalActor author);
Task Request(LocalActor author, PostEntity post, PostEntity quoted, JsonObject note, CancellationToken token);
Task<bool> Answered(JsonNode answer, ForeignAvatar actor, bool accepted, CancellationToken token);
}
public enum QuotePermission
{
Denied,
Granted,
AskFirst
}
public class QuoteService : IQuoteService
{
readonly DbEntities _dbEntities;
readonly IRemoteActorService _remoteActors;
readonly IRemotePosts _remotePosts;
readonly ILocalActorService _localActors;
readonly IDeliveryService _delivery;
readonly IOutboxPublisher _outbox;
public QuoteService(DbEntities dbEntities, IRemoteActorService remoteActors, IRemotePosts remotePosts, ILocalActorService localActors,
IDeliveryService delivery, IOutboxPublisher outbox)
{
_dbEntities = dbEntities;
_remoteActors = remoteActors;
_remotePosts = remotePosts;
_localActors = localActors;
_delivery = delivery;
_outbox = outbox;
}
const string RequestPrefix = "quote-request-";
public QuotePermission Permission(PostEntity quoted, LocalActor author)
{
if (!quoted.IsFederatedCopy)
return quoted.GroupUserId == author.Id && quoted.Visibility is PostVisibility.Public or PostVisibility.Unlisted
? QuotePermission.Granted
: QuotePermission.Denied;
if (quoted.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
return QuotePermission.Denied;
if (quoted.QuotePolicy is not { } policy)
return QuotePermission.Granted;
return policy.Automatic.Concat(policy.Manual).Any(Addressing.IsPublic) || policy.Automatic.Concat(policy.Manual).Contains(author.Uri)
? QuotePermission.AskFirst
: QuotePermission.Denied;
}
public async Task Request(LocalActor author, PostEntity post, PostEntity quoted, JsonObject note, CancellationToken token)
{
var owner = await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == quoted.ActorURI).ExecuteFirstAsync(token);
if (string.IsNullOrEmpty(owner?.InboxURL))
return;
await _delivery.Enqueue(author, new[] { owner.InboxURL }, new JsonObject
{
["@context"] = ActivityPubRenderer.Context(),
["id"] = author.ActivityUri(RequestPrefix + post.ID),
["type"] = "QuoteRequest",
["actor"] = author.Uri,
["object"] = quoted.ObjectURI,
["instrument"] = note.DeepClone(),
["to"] = new JsonArray(quoted.ActorURI)
}, token);
}
public async Task<bool> Answered(JsonNode answer, ForeignAvatar actor, bool accepted, CancellationToken token)
{
var request = answer["object"];
var requestId = Id(request);
var marker = requestId?.LastIndexOf("/grunts/" + RequestPrefix, StringComparison.Ordinal) ?? -1;
if (marker < 0 && !(request is JsonObject && Value(request, "type") == "QuoteRequest"))
return false;
if (marker < 0)
return true;
var postId = requestId[(marker + "/grunts/".Length + RequestPrefix.Length)..];
var post = await _dbEntities.Posts.Match(p => p.ID == postId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
var author = post == default ? default : await _localActors.FindById(LocalActorKind.Person, post.GroupUserId, token);
if (author == default || author.ActivityUri(RequestPrefix + post.ID) != requestId || post.QuoteState != QuoteState.Pending)
return true;
var quoted = await _dbEntities.Posts.MatchID(post.QuotedPostId).ExecuteFirstAsync(token);
if (quoted?.ActorURI != actor.ActorURI)
return true;
if (!accepted)
{
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.QuoteState, QuoteState.Rejected).ExecuteAsync(token);
return true;
}
var stamp = Id(answer["result"]);
if (stamp == default || !await Verified(stamp, post.ObjectURI, quoted, token))
return true;
post.QuoteAuthorizationURI = stamp;
post.QuoteState = QuoteState.Accepted;
await DB.Default.Update<PostEntity>().MatchID(post.ID)
.Modify(p => p.QuoteAuthorizationURI, stamp)
.Modify(p => p.QuoteState, QuoteState.Accepted)
.ExecuteAsync(token);
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
if (!post.IsLocalOnly)
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, "quote-approved"), token);
return true;
}
public async Task Resolve(PostEntity post, NoteDocument note, CancellationToken token)
{
if (note.QuoteUri == default && !note.QuoteDeleted)
return;
var quoted = note.QuoteUri == default
? default
: await _dbEntities.Posts.Match(p => p.ObjectURI == note.QuoteUri && !p.DeletedAt.HasValue).ExecuteFirstAsync(token)
?? await _remotePosts.StoreContext(note.QuoteUri, RemotePosts.MaxDepth, token);
var state = note.QuoteDeleted ? QuoteState.Deleted
: quoted == default ? QuoteState.Pending
: note.QuoteAuthorization != default
? !quoted.IsFederatedCopy || !await Verified(note.QuoteAuthorization, post.ObjectURI, quoted, token) ? QuoteState.Unauthorized : QuoteState.Accepted
: note.QuotesByConsent ? QuoteState.Pending
: quoted.Visibility is PostVisibility.Public or PostVisibility.Unlisted ? QuoteState.Accepted
: QuoteState.Unauthorized;
var wasCounted = post.QuoteState == QuoteState.Accepted ? post.QuotedPostId : default;
await DB.Default.Update<PostEntity>().MatchID(post.ID)
.Modify(p => p.QuoteURI, note.QuoteUri)
.Modify(p => p.QuotedPostId, quoted?.ID)
.Modify(p => p.QuoteState, state)
.Modify(p => p.QuoteAuthorizationURI, state == QuoteState.Accepted ? note.QuoteAuthorization : default)
.ExecuteAsync(token);
post.QuotedPostId = quoted?.ID;
post.QuoteState = state;
var nowCounted = state == QuoteState.Accepted ? quoted?.ID : default;
if (wasCounted == nowCounted)
return;
if (wasCounted != default)
await DB.Default.Update<PostEntity>().MatchID(wasCounted).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
if (nowCounted != default)
{
await DB.Default.Update<PostEntity>().MatchID(nowCounted).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
if (!quoted.IsFederatedCopy)
await Notifications.Add(quoted.GroupUserId, NotificationType.Quote, post.AuthorAccountId, post.ActorURI, post.ID, token);
}
}
public async Task Revoke(string stampUri, CancellationToken token)
{
var revoked = await _dbEntities.Posts.Match(p => p.QuoteAuthorizationURI == stampUri && p.QuoteState == QuoteState.Accepted).ExecuteAsync(token);
foreach (var post in revoked)
{
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.QuoteState, QuoteState.Revoked).ExecuteAsync(token);
if (post.QuotedPostId != default)
await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
}
}
public async Task<bool> Verified(string stampUri, string quotingUri, PostEntity quoted, CancellationToken token)
{
if (!Origin.Same(stampUri, quoted.ActorURI))
return false;
using var fetched = await _remoteActors.FetchObject(stampUri, token);
if (fetched == default)
return false;
var stamp = JsonNode.Parse(fetched.Root.GetRawText());
return Value(stamp, "type") == "QuoteAuthorization"
&& Id(stamp["attributedTo"]) == quoted.ActorURI
&& Id(stamp["interactingObject"]) == quotingUri
&& Id(stamp["interactionTarget"]) == quoted.ObjectURI;
}
}
}
+37 -3
View File
@@ -39,6 +39,8 @@ namespace PrivaPub.Domain.Statuses
public double? Latitude { get; init; }
public double? Longitude { get; init; }
public double? RangeKm { get; init; }
public PollDraft Poll { get; init; }
public string QuotedStatusId { get; init; }
}
public sealed record StatusOutcome(PostEntity Post, int Status = StatusCodes.Status200OK, string Error = default)
@@ -69,10 +71,17 @@ namespace PrivaPub.Domain.Statuses
readonly IFanout _fanout;
readonly IMediaService _media;
readonly IGroupDistributor _groups;
readonly IPollService _polls;
readonly ILinkPreviews _previews;
readonly IQuoteService _quotes;
public StatusService(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
IContentRenderer content, IOutboxPublisher outbox, IFanout fanout, IMediaService media, IGroupDistributor groups)
IContentRenderer content, IOutboxPublisher outbox, IFanout fanout, IMediaService media, IGroupDistributor groups, IPollService polls,
ILinkPreviews previews, IQuoteService quotes)
{
_previews = previews;
_quotes = quotes;
_polls = polls;
_media = media;
_groups = groups;
_dbEntities = dbEntities;
@@ -89,8 +98,10 @@ namespace PrivaPub.Domain.Statuses
var media = await Media(author, draft.MediaIds, default, token);
if (media == default)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Media attachments are invalid");
if (string.IsNullOrWhiteSpace(draft.Text) && media.Count == 0)
if (string.IsNullOrWhiteSpace(draft.Text) && media.Count == 0 && draft.Poll == default)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Text can't be blank");
if (draft.Poll != default && _polls.Invalid(draft.Poll) is { } invalidPoll)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, invalidPoll);
LocalActor group = default;
if (!string.IsNullOrEmpty(draft.GroupId))
@@ -107,6 +118,18 @@ namespace PrivaPub.Domain.Statuses
if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token))
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
PostEntity quoted = default;
var quotePermission = QuotePermission.Denied;
if (!string.IsNullOrEmpty(draft.QuotedStatusId))
{
quoted = await _dbEntities.Posts.Match(p => p.ID == draft.QuotedStatusId && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token);
if (quoted == default || !await VisibilityPolicy.CanSee(quoted, author.Id, token))
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
quotePermission = _quotes.Permission(quoted, author);
if (quotePermission == QuotePermission.Denied)
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: This post cannot be quoted");
}
var located = draft.Latitude.HasValue || draft.Longitude.HasValue;
if (located && (draft.Latitude is not (>= -90 and <= 90) || draft.Longitude is not (>= -180 and <= 180) || group != default
|| draft.Visibility == PostVisibility.Direct))
@@ -157,7 +180,12 @@ namespace PrivaPub.Domain.Statuses
InReplyToURI = parent?.ObjectURI ?? (IsRemoteUri(draft.InReplyTo) ? draft.InReplyTo : default),
InReplyToAccountId = parent?.AuthorAccountId ?? parent?.GroupUserId,
IsLocalOnly = isLocalOnly,
ActorURI = author.Uri
ActorURI = author.Uri,
Poll = draft.Poll == default ? default : _polls.Create(draft.Poll),
QuoteURI = quoted?.ObjectURI,
QuotedPostId = quoted?.ID,
QuoteByConsent = quoted != default && (quoted.QuotePolicy != default || !quoted.IsFederatedCopy),
QuoteState = quoted == default ? QuoteState.None : quotePermission == QuotePermission.Granted ? QuoteState.Accepted : QuoteState.Pending
};
post.ID = (string)post.GenerateNewID();
post.ObjectURI = author.PostUri(post.ID);
@@ -200,6 +228,12 @@ namespace PrivaPub.Domain.Statuses
if (parent != default)
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
await _fanout.Distribute(post, token);
await _polls.Scheduled(post, token);
if (post.QuoteState == QuoteState.Accepted)
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
if (post.QuoteState == QuoteState.Pending && create?["object"] is JsonObject quotingNote)
await _quotes.Request(author, post, quoted, quotingNote, token);
await _previews.Wanted(post, token);
if (create != default)
await _outbox.Publish(author, post, create, token);
if (create != default && group is { IsCircle: false } && visibility is PostVisibility.Public or PostVisibility.Unlisted)
+50 -1
View File
@@ -1,6 +1,9 @@
using System.Globalization;
using System.Text.Json;
using System.Text.Json.Nodes;
using PrivaPub.Federation.Objects;
using PrivaPub.Models.Post;
namespace PrivaPub.Federation.Actors
{
@@ -23,6 +26,16 @@ namespace PrivaPub.Federation.Actors
public string SharedInbox { get; init; }
public string Icon { get; init; }
public bool Discoverable { get; init; } = true;
public bool Indexable { get; init; }
public bool Locked { get; init; }
public bool Memorial { get; init; }
public string Header { get; init; }
public string IconDescription { get; init; }
public string HeaderDescription { get; init; }
public string MovedTo { get; init; }
public DateTime? Published { get; init; }
public List<CustomEmoji> Emojis { get; init; } = new();
public Dictionary<string, string> Fields { get; init; } = new();
public IReadOnlyList<ActorKey> Keys { get; init; } = Array.Empty<ActorKey>();
public ActorKey Key(string keyId) => Keys.FirstOrDefault(k => k.Id == keyId);
@@ -52,10 +65,46 @@ namespace PrivaPub.Federation.Actors
SharedInbox = root.TryGetProperty("endpoints", out var endpoints) ? RemoteActorService.Text(endpoints, "sharedInbox") : default,
Icon = root.TryGetProperty("icon", out var icon) ? RemoteActorService.Text(icon, "url") : default,
Discoverable = !root.TryGetProperty("discoverable", out var discoverable) || discoverable.ValueKind != JsonValueKind.False,
Keys = ParseKeys(root, id)
Keys = ParseKeys(root, id),
Indexable = Flag(root, "indexable"),
Locked = Flag(root, "manuallyApprovesFollowers"),
Memorial = Flag(root, "memorial"),
Header = root.TryGetProperty("image", out var image) ? ObjectShapes.Image(JsonNode.Parse(image.GetRawText())) : default,
IconDescription = root.TryGetProperty("icon", out var described) ? Alt(described) : default,
HeaderDescription = root.TryGetProperty("image", out var headerImage) ? Alt(headerImage) : default,
MovedTo = RemoteActorService.Text(root, "movedTo"),
Published = DateTimeOffset.TryParse(RemoteActorService.Text(root, "published"), CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out var published)
? published.UtcDateTime
: default(DateTime?),
Emojis = root.TryGetProperty("tag", out var tags) ? ObjectShapes.Emojis(JsonNode.Parse(tags.GetRawText())) : new(),
Fields = root.TryGetProperty("attachment", out var attachment) ? ParseFields(JsonNode.Parse(attachment.GetRawText())) : new()
};
}
const int MaxFields = 16;
static bool Flag(JsonElement root, string property) =>
root.TryGetProperty(property, out var value) && value.ValueKind == JsonValueKind.True;
static string Alt(JsonElement image) =>
image.ValueKind == JsonValueKind.Object ? ObjectShapes.Text(RemoteActorService.Text(image, "name") ?? RemoteActorService.Text(image, "summary"), 1500) : default;
static Dictionary<string, string> ParseFields(JsonNode attachment)
{
var fields = new Dictionary<string, string>();
foreach (var field in ObjectShapes.Objects(attachment).Where(a => ActivityJson.Value(a, "type") is "PropertyValue" or "Note"))
{
var name = ObjectShapes.Text(ActivityJson.Value(field, "name"), 255);
var value = ActivityJson.Value(field, "value") ?? ActivityJson.Value(field, "content");
if (name == default || value == default || fields.ContainsKey(name))
continue;
fields[name] = ContentSanitizer.Html(value.Length <= 2048 ? value : value[..2048]);
if (fields.Count == MaxFields)
break;
}
return fields;
}
static List<ActorKey> ParseKeys(JsonElement root, string actorId)
{
var keys = new List<ActorKey>();
@@ -19,6 +19,7 @@ namespace PrivaPub.Federation.Actors
Task<FetchedJson> FetchObject(string uri, CancellationToken token);
Task<ForeignAvatar> GetActor(string actorUri, bool refresh, CancellationToken token);
Task<ForeignAvatar> GetActorByKeyId(string keyId, bool refresh, CancellationToken token);
bool KeyTemporarilyUnavailable(string keyId) => false;
Task<string> ResolveHandle(string handle, CancellationToken token);
}
@@ -120,6 +121,8 @@ namespace PrivaPub.Federation.Actors
return await Upsert(actor, key, token);
}
public bool KeyTemporarilyUnavailable(string keyId) => Origin.Of(keyId) != default && _http.FailedTemporarily(StripFragment(keyId));
public async Task<string> ResolveHandle(string handle, CancellationToken token)
{
var parts = handle?.TrimStart('@').Split('@');
@@ -174,7 +177,17 @@ namespace PrivaPub.Federation.Actors
.Modify(a => a.FollowingURL, actor.Following)
.Modify(a => a.SharedInboxURL, actor.SharedInbox)
.Modify(a => a.PictureURL, actor.Icon)
.Modify(a => a.ThumbnailURL, actor.Header)
.Modify(a => a.PictureDescription, actor.IconDescription)
.Modify(a => a.HeaderDescription, actor.HeaderDescription)
.Modify(a => a.IsDiscoverable, actor.Discoverable)
.Modify(a => a.IsIndexable, actor.Indexable)
.Modify(a => a.IsLocked, actor.Locked)
.Modify(a => a.IsMemorial, actor.Memorial)
.Modify(a => a.MovedToURL, Origin.Of(actor.MovedTo) == default ? default : actor.MovedTo)
.Modify(a => a.Published, actor.Published)
.Modify(a => a.Emojis, actor.Emojis)
.Modify(a => a.Fields, actor.Fields)
.Modify(a => a.AvatarType, Enum.TryParse<AvatarType>(actor.Type, out var type) ? type : AvatarType.Person)
.Modify(a => a.PublicKeyId, key?.Id)
.Modify(a => a.PublicKey, key?.Pem)
@@ -333,6 +333,8 @@ namespace PrivaPub.Federation.Controllers
{
if (result.Error != default)
_logger.LogInformation("Inbox refused with {Status}: {Error}", result.StatusCode, result.Error);
if (result.RetryAfterSeconds is { } seconds)
Response.Headers.RetryAfter = seconds.ToString(System.Globalization.CultureInfo.InvariantCulture);
return result.Error == default ? StatusCode(result.StatusCode) : StatusCode(result.StatusCode, result.Error);
}
@@ -1,5 +1,6 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
@@ -15,17 +16,21 @@ namespace PrivaPub.Federation.Inbox.Handlers
{
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
readonly IQuoteService _quotes;
public AcceptHandler(DbEntities dbEntities, ILocalActorService localActors)
public AcceptHandler(DbEntities dbEntities, ILocalActorService localActors, IQuoteService quotes)
{
_dbEntities = dbEntities;
_localActors = localActors;
_quotes = quotes;
}
public virtual string Type => "Accept";
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
{
if (await _quotes.Answered(activity, actor, accepted: Type == "Accept", token))
return;
var following = await FindFollowing(activity["object"], actor, _dbEntities, _localActors, token);
if (following == default)
return;
@@ -56,7 +61,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
public class RejectHandler : AcceptHandler
{
public RejectHandler(DbEntities dbEntities, ILocalActorService localActors) : base(dbEntities, localActors)
public RejectHandler(DbEntities dbEntities, ILocalActorService localActors, IQuoteService quotes) : base(dbEntities, localActors, quotes)
{
}
@@ -4,6 +4,8 @@ using MongoDB.Entities;
using PrivaPub.Domain.Timelines;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Moderation;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Infrastructure.Ids;
@@ -35,11 +37,17 @@ namespace PrivaPub.Federation.Inbox.Handlers
readonly IRemotePosts _remotePosts;
readonly IGroupDistributor _groups;
readonly IObjectRecords _records;
readonly IPollService _polls;
readonly ILinkPreviews _previews;
readonly IQuoteService _quotes;
public CreateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
IDomainBlocks domainBlocks, IFanout fanout, IRemotePosts remotePosts, IGroupDistributor groups, IObjectRecords records)
IDomainBlocks domainBlocks, IFanout fanout, IRemotePosts remotePosts, IGroupDistributor groups, IObjectRecords records, IPollService polls, ILinkPreviews previews, IQuoteService quotes)
{
_quotes = quotes;
_previews = previews;
_records = records;
_polls = polls;
_groups = groups;
_fanout = fanout;
_remotePosts = remotePosts;
@@ -64,6 +72,15 @@ namespace PrivaPub.Federation.Inbox.Handlers
var note = NoteParser.Parse(node);
if (note == default || note.AttributedTo != author.ActorURI)
return;
if (note.Title != default && note.InReplyTo != default && ObjectShapes.Text(note.ContentHtml, 1) == default)
{
var question = await _dbEntities.Posts.Match(p => p.ObjectURI == note.InReplyTo && p.Poll != null).ExecuteFirstAsync(token);
if (question != default)
{
await _polls.Receive(question, author, note.Title, Id(activity), token);
return;
}
}
if (await _dbEntities.Posts.Match(p => p.ObjectURI == note.Id).ExecuteAnyAsync(token)
|| await DB.Default.Find<DeletedObject>().Match(d => d.ObjectURI == note.Id).ExecuteAnyAsync(token))
return;
@@ -136,6 +153,8 @@ namespace PrivaPub.Federation.Inbox.Handlers
return;
}
await _records.Record(note, post, ObjectPath.Delivered, refetched, token);
await _previews.Wanted(post, token);
await _quotes.Resolve(post, note, token);
if (parent != default)
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
@@ -1,5 +1,6 @@
using MongoDB.Entities;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
@@ -27,10 +28,12 @@ namespace PrivaPub.Federation.Inbox.Handlers
readonly IRemoteActorService _remoteActors;
readonly IDeliveryService _delivery;
readonly IGroupDistributor _groups;
readonly IQuoteService _quotes;
public DeleteHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
IGroupDistributor groups)
IGroupDistributor groups, IQuoteService quotes)
{
_quotes = quotes;
_groups = groups;
_dbEntities = dbEntities;
_localActors = localActors;
@@ -72,7 +75,10 @@ namespace PrivaPub.Federation.Inbox.Handlers
.ExecuteAsync(token);
var post = await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI).ExecuteFirstAsync(token);
if (post == default)
{
await _quotes.Revoke(objectUri, token);
return;
}
await DB.Default.DeleteAsync<PostEntity>(post.ID);
await DB.Default.DeleteAsync<ObjectRecord>(r => r.PostId == post.ID || r.ObjectURI == objectUri);
await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID);
@@ -0,0 +1,34 @@
using PrivaPub.Domain.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
namespace PrivaPub.Federation.Inbox.Handlers
{
public class EmojiReactHandler : IActivityHandler
{
readonly DbEntities _dbEntities;
readonly IReactions _reactions;
public EmojiReactHandler(DbEntities dbEntities, IReactions reactions)
{
_dbEntities = dbEntities;
_reactions = reactions;
}
public string Type => "EmojiReact";
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
{
var objectUri = Id(activity["object"]);
var post = objectUri == default
? default
: await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token);
if (post != default)
await _reactions.Receive(post, actor, Value(activity, "content"), activity["tag"], Id(activity), token);
}
}
}
@@ -18,10 +18,12 @@ namespace PrivaPub.Federation.Inbox.Handlers
public class LikeHandler : IActivityHandler
{
readonly DbEntities _dbEntities;
readonly IReactions _reactions;
public LikeHandler(DbEntities dbEntities)
public LikeHandler(DbEntities dbEntities, IReactions reactions)
{
_dbEntities = dbEntities;
_reactions = reactions;
}
public string Type => "Like";
@@ -34,6 +36,12 @@ namespace PrivaPub.Federation.Inbox.Handlers
: await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token);
if (post == default || !MaySee(post, actor))
return;
var content = Value(activity, "_misskey_reaction") ?? Value(activity, "content");
if (!Reactions.IsHeart(content))
{
await _reactions.Receive(post, actor, content, activity["tag"], Id(activity), token);
return;
}
try
{
@@ -1,5 +1,6 @@
using MongoDB.Entities;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Social;
@@ -17,11 +18,13 @@ namespace PrivaPub.Federation.Inbox.Handlers
{
public class UndoHandler : IActivityHandler
{
readonly IReactions _reactions;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
public UndoHandler(DbEntities dbEntities, ILocalActorService localActors)
public UndoHandler(DbEntities dbEntities, ILocalActorService localActors, IReactions reactions)
{
_reactions = reactions;
_dbEntities = dbEntities;
_localActors = localActors;
}
@@ -42,6 +45,9 @@ namespace PrivaPub.Federation.Inbox.Handlers
await UndoAnnounce(innerId, actor, token);
if (innerType is null or "Dislike")
await UndoDislike(inner, innerId, actor, token);
if (innerType is null or "Like" or "EmojiReact")
await _reactions.Withdraw(actor, innerId, inner is JsonObject ? Id(inner["object"]) : default,
inner is JsonObject ? Value(inner, "_misskey_reaction") ?? Value(inner, "content") : default, token);
}
async Task UndoDislike(JsonNode inner, string innerId, ForeignAvatar actor, CancellationToken token)
@@ -1,6 +1,7 @@
using MongoDB.Entities;
using PrivaPub.Federation.Actors;
using PrivaPub.Domain.Statuses;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Models.Post;
@@ -24,10 +25,12 @@ namespace PrivaPub.Federation.Inbox.Handlers
readonly IRemoteActorService _remoteActors;
readonly IGroupDistributor _groups;
readonly IObjectRecords _records;
readonly IQuoteService _quotes;
public UpdateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IGroupDistributor groups,
IObjectRecords records)
IObjectRecords records, IQuoteService quotes)
{
_quotes = quotes;
_records = records;
_groups = groups;
_dbEntities = dbEntities;
@@ -55,6 +58,19 @@ namespace PrivaPub.Federation.Inbox.Handlers
if (post == default || post.DeletedAt.HasValue)
return;
post.Poll = note.Poll ?? post.Poll;
post.QuotePolicy = note.QuotePolicy;
post.Video = note.Video ?? post.Video;
post.Audio = note.Audio ?? post.Audio;
post.Event = note.Event ?? post.Event;
if (!IsEdit(note, post))
{
await DB.Default.SaveAsync(post, token);
await _records.Revise(note, Id(activity), token);
await Requote(post, note, token);
return;
}
post.Revisions.Add(new PostRevision
{
Title = post.Title,
@@ -77,11 +93,9 @@ namespace PrivaPub.Federation.Inbox.Handlers
post.SpoilerText = note.SpoilerText;
post.Excerpt = note.Excerpt;
post.Source = note.Source;
post.Emojis = note.Emojis.ToList();
post.CoverURL = note.CoverURL;
post.Link = note.Link;
post.Video = note.Video;
post.Audio = note.Audio;
post.Event = note.Event;
post.HasContentWarning = note.Sensitive;
post.Text = note.ContentHtml;
post.ContentHtml = note.ContentHtml;
@@ -93,8 +107,18 @@ namespace PrivaPub.Federation.Inbox.Handlers
post.UpdateDate = DateTime.UtcNow;
await DB.Default.SaveAsync(post, token);
await _records.Revise(note, Id(activity), token);
await Requote(post, note, token);
if (!string.IsNullOrEmpty(post.GroupId) && await _localActors.FindById(Models.Federation.LocalActorKind.Group, post.GroupId, token) is { IsCircle: false } community)
await _groups.Announce(community, activity.AsObject(), post.ObjectURI, isNewPost: false, token);
}
static bool IsEdit(NoteDocument note, PostEntity post) =>
note.Updated is { } updated && updated > (post.EditedAt ?? post.CreationDate);
async Task Requote(PostEntity post, NoteDocument note, CancellationToken token)
{
if (note.QuoteUri != post.QuoteURI || note.QuoteAuthorization != post.QuoteAuthorizationURI || post.QuoteState != QuoteState.Accepted)
await _quotes.Resolve(post, note, token);
}
}
}
+5 -1
View File
@@ -13,7 +13,7 @@ using static PrivaPub.Federation.Objects.ActivityJson;
namespace PrivaPub.Federation.Inbox
{
public sealed record InboxResult(int StatusCode, string Error = default);
public sealed record InboxResult(int StatusCode, string Error = default, int? RetryAfterSeconds = default);
public sealed record InboxPayload(string ActorURI, string Activity, string Inbox = default, string KeyId = default, string Algorithm = default,
string[] SignedHeaders = default, DateTime? ReceivedAt = default);
@@ -25,6 +25,8 @@ namespace PrivaPub.Federation.Inbox
public class InboxReceiver : IInboxReceiver
{
const int KeyRetrySeconds = 300;
const int MaxBodyBytes = 1024 * 1024;
readonly ILocalActorService _localActors;
@@ -88,6 +90,8 @@ namespace PrivaPub.Federation.Inbox
if (keyOwner == default || !HttpSignatures.Verify(keyOwner.PublicKey, signingString, parameters.Signature))
{
keyOwner = await _remoteActors.GetActorByKeyId(parameters.KeyId, refresh: true, token);
if (keyOwner == default && _remoteActors.KeyTemporarilyUnavailable(parameters.KeyId))
return new(StatusCodes.Status503ServiceUnavailable, "the signing key could not be fetched; try again later", KeyRetrySeconds);
if (keyOwner == default || !HttpSignatures.Verify(keyOwner.PublicKey, signingString, parameters.Signature))
return new(StatusCodes.Status401Unauthorized, "the signature does not verify");
}
+8 -1
View File
@@ -1,6 +1,7 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Domain.Content;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Moderation;
using PrivaPub.Federation.Objects;
@@ -39,10 +40,12 @@ namespace PrivaPub.Federation.Inbox
readonly IDomainBlocks _domainBlocks;
readonly IJobQueue _queue;
readonly IObjectRecords _records;
readonly ILinkPreviews _previews;
public RemotePosts(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDomainBlocks domainBlocks,
IJobQueue queue, IObjectRecords records)
IJobQueue queue, IObjectRecords records, ILinkPreviews previews)
{
_previews = previews;
_records = records;
_dbEntities = dbEntities;
_localActors = localActors;
@@ -78,6 +81,9 @@ namespace PrivaPub.Federation.Inbox
SpoilerText = note.SpoilerText,
Excerpt = note.Excerpt,
Source = note.Source,
Poll = note.Poll,
QuotePolicy = note.QuotePolicy,
Emojis = note.Emojis.ToList(),
CoverURL = note.CoverURL,
Link = note.Link,
Video = note.Video,
@@ -141,6 +147,7 @@ namespace PrivaPub.Federation.Inbox
return await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri).ExecuteFirstAsync(token);
}
await _records.Record(note, post, ObjectPath.Fetched, refetched: false, token);
await _previews.Wanted(post, token);
if (grandparent == default && !string.IsNullOrEmpty(note.InReplyTo) && depth < MaxDepth)
await _queue.Enqueue(JobKind.FetchAncestors, JsonSerializer.Serialize(new AncestorsPayload(post.ID, depth + 1)), new Uri(note.InReplyTo).Host,
@@ -38,7 +38,7 @@ namespace PrivaPub.Federation.Objects
{
KeepChildNodes = true
};
foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible" })
foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible", "quote-inline" })
sanitizer.AllowedClasses.Add(allowed);
sanitizer.RemovingCssClass += (_, e) => e.Cancel = MicroformatClass().IsMatch(e.CssClass);
sanitizer.RemovingTag += (_, e) =>
+19 -1
View File
@@ -25,6 +25,9 @@ namespace PrivaPub.Federation.Objects
public string Context { get; init; }
public string Audience { get; init; }
public string QuoteUri { get; init; }
public string QuoteAuthorization { get; init; }
public bool QuotesByConsent { get; init; }//FEP-044f `quote`, as opposed to the legacy keys that ask nobody
public bool QuoteDeleted { get; init; }
public DateTime Published { get; init; }
public DateTime? Updated { get; init; }
public IReadOnlyList<string> To { get; init; } = Array.Empty<string>();
@@ -33,6 +36,9 @@ namespace PrivaPub.Federation.Objects
public IReadOnlyList<string> Tags { get; init; } = Array.Empty<string>();
public IReadOnlyList<PostMedia> Attachments { get; init; } = Array.Empty<PostMedia>();
public PostSource Source { get; init; }
public PostPoll Poll { get; init; }
public InteractionRule QuotePolicy { get; init; }
public IReadOnlyList<CustomEmoji> Emojis { get; init; } = Array.Empty<CustomEmoji>();
public string CoverURL { get; init; }
public PostLink Link { get; init; }
public VideoDetails Video { get; init; }
@@ -77,7 +83,10 @@ namespace PrivaPub.Federation.Objects
InReplyTo = Id(note["inReplyTo"]),
Context = Id(note["context"]) ?? Value(note, "conversation"),
Audience = Id(note["audience"]),
QuoteUri = Value(note, "quote") ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote"),
QuoteUri = Id(note["quote"]) ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote") ?? QuoteLink(note),
QuoteAuthorization = Id(note["quoteAuthorization"]),
QuotesByConsent = note.ContainsKey("quote"),
QuoteDeleted = note["quote"] is JsonObject quoted && Value(quoted, "type") == "Tombstone",
Published = Time(Value(note, "published")) ?? DateTime.UtcNow,
Updated = Time(Value(note, "updated")),
To = List(note["to"]),
@@ -94,6 +103,9 @@ namespace PrivaPub.Federation.Objects
.ToList(),
Attachments = Attachments(note),
Source = ObjectShapes.Source(note),
Poll = ObjectShapes.Poll(note),
QuotePolicy = ObjectShapes.QuotePolicy(note),
Emojis = ObjectShapes.Emojis(note["tag"]),
CoverURL = ObjectShapes.Cover(note),
Link = ObjectShapes.Link(note),
Video = ObjectShapes.Video(note),
@@ -165,6 +177,12 @@ namespace PrivaPub.Federation.Objects
};
}
static string QuoteLink(JsonObject note) =>
Tags(note, "Link")
.Where(t => Value(t, "mediaType") is { } type && (type.StartsWith("application/activity+json") || type.StartsWith("application/ld+json")))
.Select(t => Value(t, "href"))
.FirstOrDefault(href => Origin.Of(href) != default);
static string FirstOf(JsonNode map) =>
map is JsonObject languages ? languages.Select(l => l.Value is JsonValue v && v.TryGetValue<string>(out var text) ? text : default).FirstOrDefault(t => t != default) : default;
+73 -1
View File
@@ -16,6 +16,9 @@ namespace PrivaPub.Federation.Objects
const int MaxCaptions = 20;
const int MaxPlaces = 4;
const int CoverMaxWidth = 1280;
const int MaxEmojis = 64;
const int MaxPollOptions = 20;
const int MaxAudience = 32;
static readonly Dictionary<string, string> MediaTypesByExtension = new(StringComparer.OrdinalIgnoreCase)
{
@@ -81,6 +84,69 @@ namespace PrivaPub.Federation.Objects
?? usable.Select(c => c.Url).FirstOrDefault();
}
public static List<CustomEmoji> Emojis(JsonNode tags) =>
Objects(tags)
.Where(t => Value(t, "type") == "Emoji")
.Select(t => new CustomEmoji
{
Shortcode = Value(t, "name")?.Trim(':'),
URL = MediaLink(t["icon"]).Url,
Updated = Moment(Value(t, "updated"))
})
.Where(e => !string.IsNullOrEmpty(e.Shortcode) && e.Shortcode.Length <= 100 && Shortcode().IsMatch(e.Shortcode) && Origin.Of(e.URL) != default)
.DistinctBy(e => e.Shortcode)
.Take(MaxEmojis)
.ToList();
public static InteractionRule QuotePolicy(JsonObject note)
{
if (note["interactionPolicy"] is not JsonObject policy || policy["canQuote"] is not JsonObject canQuote)
return default;
static List<string> Who(JsonNode node) => node switch
{
JsonArray array => array.Select(Id).Where(id => id != default).Take(MaxAudience).ToList(),
JsonNode single when Id(single) is { } id => new List<string> { id },
_ => new List<string>()
};
return new InteractionRule
{
Automatic = Who(canQuote["automaticApproval"] ?? canQuote["always"]),
Manual = Who(canQuote["manualApproval"] ?? canQuote["approvalRequired"])
};
}
public static PostPoll Poll(JsonObject note)
{
if (Value(note, "type") != "Question")
return default;
var multiple = note["anyOf"] is JsonArray;
var options = Objects(note[multiple ? "anyOf" : "oneOf"])
.Select(o => new PollOption
{
Title = Text(Value(o, "name"), 200),
Votes = o["replies"] is JsonObject replies && Int(replies, "totalItems") is { } total ? total : Int(o, "_misskey_votes") ?? 0
})
.Where(o => o.Title != default)
.Take(MaxPollOptions)
.ToList();
if (options.Count < 2)
return default;
var closedAt = note["closed"] switch
{
JsonValue value when value.TryGetValue<bool>(out var closed) => closed ? Moment(Value(note, "endTime")) ?? DateTime.UtcNow : default(DateTime?),
JsonValue value when value.TryGetValue<string>(out var at) => Moment(at),
_ => default
};
return new PostPoll
{
Options = options,
Multiple = multiple,
ExpiresAt = Moment(Value(note, "endTime")),
ClosedAt = closedAt,
VotersCount = Int(note, "votersCount")
};
}
public static string Cover(JsonObject note) =>
Image(note["image"]) ?? Image(note["icon"]) ?? (note["preview"] is JsonObject preview && Value(preview, "type") == "Image" ? Image(preview) : default);
@@ -151,10 +217,13 @@ namespace PrivaPub.Federation.Objects
if (Value(note, "type") != "Video")
return default;
var links = Objects(note["url"]).ToList();
var nested = links.Where(l => Value(l, "mediaType")?.Equals("application/x-mpegURL", StringComparison.OrdinalIgnoreCase) == true)
.SelectMany(l => Objects(l["tag"]))
.ToList();
return new VideoDetails
{
DurationSeconds = Seconds(note["duration"]),
Variants = Variants(links, "video/", "audio/"),
Variants = Variants(links.Concat(nested).ToList(), "video/", "audio/"),
PlaylistURL = links.Where(l => Value(l, "mediaType")?.Equals("application/x-mpegURL", StringComparison.OrdinalIgnoreCase) == true)
.Select(l => Value(l, "href")).FirstOrDefault(IsWeb),
IsLive = Bool(note, "isLiveBroadcast") == true,
@@ -284,6 +353,9 @@ namespace PrivaPub.Federation.Objects
static bool? Bool(JsonObject node, string property) =>
node?[property] is JsonValue value && value.TryGetValue<bool>(out var flag) ? flag : (bool?)null;
[GeneratedRegex(@"^[\w+\-.@]+$")]
private static partial Regex Shortcode();
[GeneratedRegex("<[^>]*>")]
private static partial Regex Tags();
@@ -60,6 +60,16 @@ namespace PrivaPub.Federation.Outbox
inboxes.Add(parentAuthor.InboxURL);
}
if (post.Visibility is PostVisibility.Public or PostVisibility.Unlisted && !string.IsNullOrEmpty(post.QuotedPostId))
{
var quoted = await _dbEntities.Posts.MatchID(post.QuotedPostId).ExecuteFirstAsync(token);
var quotedAuthor = quoted is { IsFederatedCopy: true }
? await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == quoted.ActorURI).ExecuteFirstAsync(token)
: default;
if (!string.IsNullOrEmpty(quotedAuthor?.InboxURL))
inboxes.Add(quotedAuthor.InboxURL);
}
return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList();
}
@@ -37,6 +37,19 @@ namespace PrivaPub.Federation.Rendering
["discoverable"] = "toot:discoverable",
["indexable"] = "toot:indexable",
["blurhash"] = "toot:blurhash",
["votersCount"] = "toot:votersCount",
["Emoji"] = "toot:Emoji",
["quote"] = new JsonObject { ["@id"] = "https://w3id.org/fep/044f#quote", ["@type"] = "@id" },
["quoteAuthorization"] = new JsonObject { ["@id"] = "https://w3id.org/fep/044f#quoteAuthorization", ["@type"] = "@id" },
["QuoteRequest"] = "https://w3id.org/fep/044f#QuoteRequest",
["QuoteAuthorization"] = "https://w3id.org/fep/044f#QuoteAuthorization",
["misskey"] = "https://misskey-hub.net/ns#",
["_misskey_quote"] = "misskey:_misskey_quote",
["fedibird"] = "http://fedibird.com/ns#",
["quoteUri"] = "fedibird:quoteUri",
["quoteUrl"] = "as:quoteUrl",
["litepub"] = "http://litepub.social/ns#",
["EmojiReact"] = "litepub:EmojiReact",
["focalPoint"] = new JsonObject { ["@container"] = "@list", ["@id"] = "toot:focalPoint" },
["schema"] = "http://schema.org#",
["PropertyValue"] = "schema:PropertyValue",
@@ -156,6 +169,64 @@ namespace PrivaPub.Federation.Rendering
return note;
}
public static JsonObject UpdateOf(PostEntity post, LocalActor author, string reason)
{
var note = post.Visibility == PostVisibility.Direct
? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI)
: Note(post, author, default, post.InReplyToURI);
note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
return new JsonObject
{
["@context"] = Context(),
["id"] = author.ActivityUri($"update-{post.ID}-{reason}"),
["type"] = "Update",
["actor"] = author.Uri,
["to"] = note["to"]!.DeepClone(),
["cc"] = note["cc"]!.DeepClone(),
["object"] = note
};
}
static void AddQuote(JsonObject note, PostEntity post, ref string content)
{
if (string.IsNullOrEmpty(post.QuoteURI))
return;
var link = WebUtility.HtmlEncode(post.QuoteURI);
content += $"<p class=\"quote-inline\">RE: <a href=\"{link}\">{link}</a></p>";
if (post.QuoteByConsent)
note["quote"] = post.QuoteURI;
if (!string.IsNullOrEmpty(post.QuoteAuthorizationURI))
note["quoteAuthorization"] = post.QuoteAuthorizationURI;
note["_misskey_quote"] = post.QuoteURI;
note["quoteUri"] = post.QuoteURI;
note["quoteUrl"] = post.QuoteURI;
(note["tag"] as JsonArray)?.Add(new JsonObject
{
["type"] = "Link",
["mediaType"] = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"",
["href"] = post.QuoteURI,
["name"] = $"RE: {post.QuoteURI}"
});
}
static void AddPoll(JsonObject note, PostPoll poll)
{
note["type"] = "Question";
note[poll.Multiple ? "anyOf" : "oneOf"] = new JsonArray(poll.Options.Select(option => (JsonNode)new JsonObject
{
["type"] = "Note",
["name"] = option.Title,
["replies"] = new JsonObject { ["type"] = "Collection", ["totalItems"] = option.Votes }
}).ToArray());
if (poll.ExpiresAt is { } ends)
note["endTime"] = Timestamp(ends);
if (poll.ClosedAt is { } closed)
note["closed"] = Timestamp(closed);
if (poll.VotersCount is { } voters)
note["votersCount"] = voters;
}
static string Headline(PostEntity post)
{
var text = System.Text.RegularExpressions.Regex.Replace(post.ContentHtml ?? post.Text ?? string.Empty, "<[^>]+>", " ");
@@ -190,8 +261,12 @@ namespace PrivaPub.Federation.Rendering
}))
.ToArray())
};
AddQuote(note, post, ref content);
note["content"] = content;
if (!string.IsNullOrEmpty(post.Language))
note["contentMap"] = new JsonObject { [post.Language] = content };
if (post.Poll is { } poll)
AddPoll(note, poll);
var attachments = post.Media.Where(m => !string.IsNullOrEmpty(m.URL)).Select(m =>
{
var attachment = new JsonObject
+7
View File
@@ -110,6 +110,13 @@ namespace PrivaPub.Infrastructure.Data
await DB.Default.Index<DeletedObject>().Key(d => d.DeletedAt, KeyType.Ascending).Option(o => o.ExpireAfter = TombstoneLifetime).CreateAsync(token);
await DB.Default.Index<Downvote>().Key(d => d.PostId, KeyType.Ascending).Key(d => d.ActorURI, KeyType.Ascending).Option(o => o.Unique = true).CreateAsync(token);
await Plain<Downvote>(token, d => d.ActivityURI);
await DB.Default.Index<PollVote>().Key(v => v.PostId, KeyType.Ascending).Key(v => v.ActorURI, KeyType.Ascending).Key(v => v.Choice, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
await DB.Default.Index<PollVote>().Key(v => v.PostId, KeyType.Ascending).Key(v => v.VoterAccountId, KeyType.Ascending).CreateAsync(token);
await DB.Default.Index<Reaction>().Key(r => r.PostId, KeyType.Ascending).Key(r => r.ActorURI, KeyType.Ascending).Key(r => r.Emoji, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
await Plain<Reaction>(token, r => r.ActivityURI);
await Unique<Domain.Content.LinkPreview>(p => p.Url, Builders<Domain.Content.LinkPreview>.Filter.Type(p => p.Url, BsonType.String), token);
}
static async Task Unique<T>(System.Linq.Expressions.Expression<Func<T, object>> key, FilterDefinition<T> partial,
+116 -5
View File
@@ -21,6 +21,9 @@ namespace PrivaPub.Infrastructure.Http
{
bool IsAllowed(Uri target);
Task<FetchedJson> GetJson(string url, string accept, Action<HttpRequestMessage> sign, CancellationToken token);
bool FailedTemporarily(string url);
Task<(Uri FinalUri, string Html)> GetPage(string url, CancellationToken token);
Task<HttpResponseMessage> OpenMedia(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token);
Task<HttpResponseMessage> Send(HttpRequestMessage request, CancellationToken token);
Task<(byte[] Bytes, string ContentType)> GetMedia(string url, long maxBytes, CancellationToken token);
}
@@ -107,7 +110,8 @@ namespace PrivaPub.Infrastructure.Http
}
if (!response.IsSuccessStatusCode)
return Refuse(negativeKey, url, $"status {(int)response.StatusCode}");
return Refuse(negativeKey, url, $"status {(int)response.StatusCode}",
transient: (int)response.StatusCode is >= 500 or 429 or 408);
var mediaType = response.Content.Headers.ContentType?.MediaType;
if (mediaType == default || !JsonMediaTypes.Contains(mediaType, StringComparer.OrdinalIgnoreCase))
@@ -125,14 +129,21 @@ namespace PrivaPub.Infrastructure.Http
}
catch (OperationCanceledException) when (!token.IsCancellationRequested)
{
return Refuse(negativeKey, url, "a timeout");
return Refuse(negativeKey, url, "a timeout", transient: true);
}
catch (Exception ex) when (ex is HttpRequestException or JsonException or BlockedDestinationException)
catch (HttpRequestException ex)
{
return Refuse(negativeKey, url, ex.Message, transient: true);
}
catch (Exception ex) when (ex is JsonException or BlockedDestinationException)
{
return Refuse(negativeKey, url, ex.Message);
}
}
public bool FailedTemporarily(string url) =>
Uri.TryCreate(url, UriKind.Absolute, out var target) && _cache.TryGetValue(NegativeKey(target), out bool transient) && transient;
public async Task<(byte[] Bytes, string ContentType)> GetMedia(string url, long maxBytes, CancellationToken token)
{
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsAllowed(target))
@@ -172,6 +183,106 @@ namespace PrivaPub.Infrastructure.Http
}
}
public async Task<(Uri FinalUri, string Html)> GetPage(string url, CancellationToken token)
{
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsAllowed(target))
return default;
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(token);
timeout.CancelAfter(RequestTimeout);
try
{
for (var hop = 0; hop <= MaxRedirects; hop++)
{
using var request = new HttpRequestMessage(HttpMethod.Get, target);
request.Headers.Accept.ParseAdd("text/html, application/xhtml+xml");
using var response = await _httpClientFactory.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, timeout.Token);
if (IsRedirect(response.StatusCode))
{
var location = response.Headers.Location;
var next = location == default ? default : location.IsAbsoluteUri ? location : new Uri(target, location);
if (!IsAllowed(next))
return default;
target = next;
continue;
}
var mediaType = response.Content.Headers.ContentType?.MediaType?.ToLowerInvariant();
if (!response.IsSuccessStatusCode || mediaType is not ("text/html" or "application/xhtml+xml"))
return default;
var bytes = await ReadPrefix(response.Content, MaxPageBytes, timeout.Token);
var charset = response.Content.Headers.ContentType?.CharSet?.Trim('"');
var encoding = System.Text.Encoding.UTF8;
try
{
if (!string.IsNullOrEmpty(charset))
encoding = System.Text.Encoding.GetEncoding(charset);
}
catch (ArgumentException)
{
}
return (target, encoding.GetString(bytes));
}
return default;
}
catch (Exception ex) when (ex is HttpRequestException or BlockedDestinationException or OperationCanceledException && !token.IsCancellationRequested)
{
_logger.LogInformation("Page {Url} refused: {Reason}", url, ex.Message);
return default;
}
}
public async Task<HttpResponseMessage> OpenMedia(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token)
{
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsAllowed(target))
return default;
try
{
for (var hop = 0; hop <= MaxRedirects; hop++)
{
using var request = new HttpRequestMessage(HttpMethod.Get, target);
request.Headers.Accept.ParseAdd("video/*, audio/*, image/*");
request.Headers.Range = range;
var response = await _httpClientFactory.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
if (IsRedirect(response.StatusCode))
{
var location = response.Headers.Location;
response.Dispose();
var next = location == default ? default : location.IsAbsoluteUri ? location : new Uri(target, location);
if (!IsAllowed(next))
return default;
target = next;
continue;
}
var mediaType = response.Content.Headers.ContentType?.MediaType?.ToLowerInvariant();
if (!response.IsSuccessStatusCode || mediaType == default || mediaType.Contains("svg")
|| !(mediaType.StartsWith("video/") || mediaType.StartsWith("audio/") || mediaType.StartsWith("image/") || mediaType == "application/octet-stream"))
{
response.Dispose();
return default;
}
return response;
}
return default;
}
catch (Exception ex) when (ex is HttpRequestException or BlockedDestinationException or OperationCanceledException && !token.IsCancellationRequested)
{
_logger.LogInformation("Media stream {Url} refused: {Reason}", url, ex.Message);
return default;
}
}
const int MaxPageBytes = 512 * 1024;
static async Task<byte[]> ReadPrefix(HttpContent content, int limit, CancellationToken token)
{
await using var stream = await content.ReadAsStreamAsync(token);
var buffer = new byte[limit];
var total = 0;
int read;
while (total < limit && (read = await stream.ReadAsync(buffer.AsMemory(total, limit - total), token)) > 0)
total += read;
return buffer[..total];
}
public async Task<HttpResponseMessage> Send(HttpRequestMessage request, CancellationToken token)
{
if (!IsAllowed(request.RequestUri))
@@ -200,9 +311,9 @@ namespace PrivaPub.Infrastructure.Http
static string NegativeKey(Uri target) => "federation-http:refused:" + target.AbsoluteUri;
FetchedJson Refuse(string negativeKey, string url, string reason)
FetchedJson Refuse(string negativeKey, string url, string reason, bool transient = false)
{
_cache.Set(negativeKey, true, NegativeCacheLifetime);
_cache.Set(negativeKey, transient, NegativeCacheLifetime);
_logger.LogInformation("GET {Url} refused: {Reason}", url, reason);
return default;
}
@@ -6,5 +6,6 @@ namespace PrivaPub.Infrastructure.Http
public bool AllowPlainHttp { get; set; }
public bool SecureMode { get; set; }
public bool AcceptAnyCertificate { get; set; }
public bool FetchLinkPreviews { get; set; } = true;//owner decision 1: the server reads linked pages of public posts
}
}
@@ -77,6 +77,11 @@ namespace PrivaPub.Middleware
.AddSingleton<IActivityHandler, UndoHandler>()
.AddSingleton<IActivityHandler, LikeHandler>()
.AddSingleton<IActivityHandler, DislikeHandler>()
.AddSingleton<IActivityHandler, EmojiReactHandler>()
.AddSingleton<IReactions, Reactions>()
.AddSingleton<LinkPreviews>()
.AddSingleton<ILinkPreviews>(services => services.GetRequiredService<LinkPreviews>())
.AddSingleton<IJobHandler>(services => services.GetRequiredService<LinkPreviews>())
.AddSingleton<IActivityHandler, JoinHandler>()
.AddSingleton<IActivityHandler, AnnounceHandler>()
.AddSingleton<IActivityHandler, FlagHandler>()
@@ -86,6 +91,10 @@ namespace PrivaPub.Middleware
.AddSingleton<IJobHandler, InboxProcessor>()
.AddSingleton<IRemotePosts, RemotePosts>()
.AddSingleton<IObjectRecords, ObjectRecords>()
.AddSingleton<IPollService, PollService>()
.AddSingleton<IQuoteService, QuoteService>()
.AddSingleton<IJobHandler, PollRefreshJob>()
.AddSingleton<IJobHandler, PollCloseJob>()
.AddSingleton<IJobHandler, InstanceDescriber>()
.AddSingleton<IJobHandler, AncestorsJobHandler>()
.AddSingleton<IJobQueue, JobQueue>()
+4 -1
View File
@@ -23,7 +23,10 @@ namespace PrivaPub.Models.Jobs
Deliver,
ProcessInbox,
FetchAncestors,
DescribeInstance
DescribeInstance,
PollRefresh,
PollClose,
FetchPreview
}
public enum JobState
+21
View File
@@ -25,6 +25,7 @@ namespace PrivaPub.Models.Post
public List<PostMention> Mentions { get; set; } = new();
public List<string> Tags { get; set; } = new();
public List<PostMedia> Media { get; set; } = new();
public List<CustomEmoji> Emojis { get; set; } = new();
[BsonIgnoreIfNull]
public PostSource Source { get; set; }//the remote author's own text (Markdown, MFM, BBCode)
[BsonIgnoreIfNull]
@@ -37,6 +38,8 @@ namespace PrivaPub.Models.Post
public AudioDetails Audio { get; set; }
[BsonIgnoreIfNull]
public EventDetails Event { get; set; }
[BsonIgnoreIfNull]
public PostPoll Poll { get; set; }
public List<float> Location { get; set; } = new();
public float RangeKm { get; set; } = 5.0f;
[BsonIgnoreIfNull]
@@ -54,6 +57,13 @@ namespace PrivaPub.Models.Post
public string Url { get; set; }
public string ContextURI { get; set; }
public string QuoteURI { get; set; }
public string QuotedPostId { get; set; }//our copy of the quoted post, once fetched
public QuoteState QuoteState { get; set; }
public string QuoteAuthorizationURI { get; set; }//FEP-044f: the quoted author's stamp
public bool QuoteByConsent { get; set; }//our quote of a post whose server asks for consent (FEP-044f), so `quote` is sent
public int QuotesCount { get; set; }
[BsonIgnoreIfNull]
public InteractionRule QuotePolicy { get; set; }//a remote post's interactionPolicy.canQuote; null when it states none
public List<string> To { get; set; } = new();
public List<string> Cc { get; set; } = new();
@@ -70,6 +80,17 @@ namespace PrivaPub.Models.Post
public string UpdateReason { get; set; }
}
public enum QuoteState
{
None,
Pending,
Accepted,
Rejected,
Revoked,
Deleted,
Unauthorized
}
public enum PostVisibility
{
Public,
+29
View File
@@ -1,5 +1,34 @@
namespace PrivaPub.Models.Post
{
public class CustomEmoji
{
public string Shortcode { get; set; }//without the colons
public string URL { get; set; }
public DateTime? Updated { get; set; }
}
public class PostPoll
{
public List<PollOption> Options { get; set; } = new();
public bool Multiple { get; set; }
public DateTime? ExpiresAt { get; set; }
public DateTime? ClosedAt { get; set; }
public int? VotersCount { get; set; }//null when the origin does not say (Misskey)
public bool HideTotals { get; set; }
}
public class PollOption
{
public string Title { get; set; }
public int Votes { get; set; }
}
public class InteractionRule
{
public List<string> Automatic { get; set; } = new();//actor or collection URIs; Public for anyone
public List<string> Manual { get; set; } = new();
}
public class PostSource
{
public string Content { get; set; }
+6 -1
View File
@@ -10,6 +10,8 @@ namespace PrivaPub.Models.Social
public string FromActorURI { get; set; }
public string PostId { get; set; }
public string DedupeKey { get; set; }
public string Emoji { get; set; }//a reaction's emoji
public string EmojiURL { get; set; }
public bool IsRead { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
@@ -21,6 +23,9 @@ namespace PrivaPub.Models.Social
FollowRequest,
Favourite,
Reblog,
Update
Update,
Poll,
Reaction,
Quote
}
}
+15
View File
@@ -0,0 +1,15 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Social
{
public class PollVote : Entity
{
public string PostId { get; set; }
public string VoterAccountId { get; set; }//Avatar.ID for a local voter, ForeignAvatar.ID for a remote one
public string ActorURI { get; set; }
public bool IsLocalVoter { get; set; }
public int Choice { get; set; }
public string ActivityURI { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
}
+16
View File
@@ -0,0 +1,16 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Social
{
public class Reaction : Entity
{
public string PostId { get; set; }
public string AccountId { get; set; }//Avatar.ID for a local persona, ForeignAvatar.ID for a remote account
public string ActorURI { get; set; }
public bool IsLocal { get; set; }
public string Emoji { get; set; }//a single Unicode emoji, or a custom emoji's shortcode without colons or host
public string EmojiURL { get; set; }//set for a custom emoji
public string ActivityURI { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
}
+7
View File
@@ -59,6 +59,13 @@ namespace PrivaPub.Models.User
public string PersonalNote { get; set; }
public string ModerationNote { get; set; }
public bool IsDiscoverable { get; set; } = true;
public bool IsIndexable { get; set; }
public bool IsLocked { get; set; }//manuallyApprovesFollowers
public bool IsMemorial { get; set; }
public List<PrivaPub.Models.Post.CustomEmoji> Emojis { get; set; } = new();
public string PictureDescription { get; set; }
public string HeaderDescription { get; set; }
public DateTime? Published { get; set; }//the actor's own published
public string ActorURI { get; set; }//id
public string PublicKeyId { get; set; }
+12
View File
@@ -48,6 +48,18 @@ server {
proxy_read_timeout 300s;
}
location ^~ /media/proxy/ {
proxy_buffering off;
proxy_pass http://127.0.0.1:6970;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 600s;
}
location / {
proxy_pass http://127.0.0.1:6970;
proxy_http_version 1.1;
+28 -15
View File
@@ -10,7 +10,7 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
- [x] P2 Mastodon client API: v1.4.0, deployed 2026-10-01; OAuth and the anonymous API verified on production, the signed-in API verified locally (no real-client login on production yet)
- [x] P3 Social features: v1.5.0, deployed 2026-10-01; media, proxy, blocks, mutes, bookmarks, pins and reports verified by tests and locally (no upload on production yet)
- [x] P4 Groups and privacy features: v1.6.0, deployed 2026-10-01; communities, circles and local-only located posts verified by tests. v1.6.1 adds the pasture (`tools/pasture/`): live interop with GoToSocial 0.22.1 passes all 25 checks, three runs in a row. Lemmy and a live Mastodon circle member are not run yet; the pasture has GoToSocial only
- [ ] P5 Lose nothing: wire tolerance, full objects, raw capture (see `docs/INTEROP.md`)
- [x] P5 Lose nothing: v1.7.0 to v1.9.1, deployed 2026-10-01; parsing, typed details, provenance, downvotes, tombstones, federated blocks, all checked live against GoToSocial. Book reviews and forum threads keep their raw form only (typed in P7 and P8)
- [ ] P6 Emoji, polls, quotes, reactions, cards, players
- [ ] P7 Threads, communities, moderation, the social graph
- [ ] P8 Signatures, discovery, the long tail
@@ -414,7 +414,7 @@ it, raw where it doesn't.
- Persona usernames match Mastodon's and Misskey's pattern (groups already did).
- Every term we emit is defined in our JSON-LD context.
- `Vary: Accept`.
- Still open: every activity id dereferences.
- Not done on purpose: see "Deliberately not done" below.
- **Owner decisions that were small** (done in v1.7.0): blocks federate (`Block`, `Undo{Block}`); a persona's and a
group's `published` is a random day up to two weeks before its creation (migration `_007`).
- **Parsing every shape** (done in v1.8.0, `Federation/Objects/ObjectShapes.cs`):
@@ -441,24 +441,37 @@ it, raw where it doesn't.
- A deleted object's id is kept for 90 days (W9), and its `ObjectRecord` is removed with the post.
- **Typed details reach clients** (done in v1.9.0) as `Status.privapub`.
- **Delivery:** a 503 with `Retry-After` is waited out like a 429 and no longer counts against the host (done in v1.9.0).
Still open: answering 503 ourselves when a key fetch fails temporarily, and making every activity id dereference.
We answer 503 with `Retry-After` ourselves when a sender's key cannot be fetched for a temporary reason (v1.9.1),
so Mastodon 4.7 retries instead of switching to RFC 9421.
- **Deliberately not done: dereferenceable `Follow`, `Like`, `Block`, `Accept`, `Reject` and `Undo` ids.** Serving them
would publish who follows, likes and blocks whom, which our collections deliberately hide. Every one of them is sent
with its object embedded, which is all Misskey needs. `Create` and `Announce` ids do dereference.
#### P6 What people see: emoji, polls, quotes, reactions, cards, players
- **Custom emoji** on posts, names, fields and poll options, proxied.
- **Polls** in and out, with the Misskey, Pleroma and PieFed vote shapes. A count refresh is never an edit.
- **Custom emoji** on posts, names, fields and poll options, proxied (done in v1.10.0), together with fuller remote
profiles: header, fields, locked, published, moved-to, indexable, memorial, image descriptions.
- **Polls** in and out, with the Misskey, Pleroma and PieFed vote shapes. A count refresh is never an edit (done in
v1.10.0; checked live both ways against GoToSocial).
- **Quotes (FEP-044f):**
- read every key; verify `QuoteAuthorization` on all of its fields; handle revocation;
- be quotable: `canQuote`, answer `QuoteRequest`, serve and revoke stamps;
- then advertise `api_versions.mastodon ≥ 7` (4.5.0).
- **Emoji reactions** in all three inbound forms, plus outbound `EmojiReact`, exposed as `emoji_reactions`.
- **Link cards:**
- read every key; verify `QuoteAuthorization` on all of its fields; handle revocation (done in v1.13.0);
- personas quote others: `quoted_status_id`, `QuoteRequest`, `Accept{result}` verified, `Update` with
`quoteAuthorization`; legacy quotes for posts that state no policy (done in v1.13.0); `api_versions.mastodon = 7`;
- still open: be quotable (`canQuote` on our posts, answer `QuoteRequest`, serve and revoke stamps). It needs a themed
route name for stamps, agreed with the owner, and a default quote policy.
- **Emoji reactions** in all three inbound forms, plus outbound `EmojiReact`, exposed as `emoji_reactions` (done in
v1.11.0; Pleroma's `/api/v1/pleroma/statuses/:id/reactions` endpoints and `pleroma:emoji_reaction` notifications).
- **Link cards** (done in v1.12.0):
- from the object, or from FEP-8967 `preview`, without fetching;
- fetching the page itself is the owner's decision (INTEROP §6.1).
- **Media:**
- video playback through the proxy (Range requests, HLS playlist rewriting, the poster), with a `video` card;
- otherwise the server reads the page (owner decision 1): public posts only, 0–60 s after arrival, cached per address
for 7 days across the whole server, `Federation:FetchLinkPreviews` to switch it off. Checked live against a
GoToSocial profile page.
- **Media** (done in v1.14.0 except where noted):
- video playback through the proxy: range requests streamed from the origin, the poster, a playable attachment picked
from MP4 or fragmented MP4 files (HLS playlists are not rewritten; not needed while PeerTube publishes the files);
- audio attachments;
- an article reader view;
- JPEG/PNG renditions kept for Pixelfed.
- JPEG/PNG for Pixelfed: uploads were already re-encoded to JPEG, PNG or GIF;
- an article reader view: the data is in `Status.privapub` (title, excerpt, cover, full HTML in `content`); the view
itself belongs to the client.
#### P7 Threads, communities and the social graph
- **Thread backfill:**
+22
View File
@@ -101,6 +101,28 @@ gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses -d 'status=@alice@priv
until_true 20 'curl -s -H "$PH" "$P/api/v1/conversations" | grep -q "a secret back"' && ok "gtsuser's DM reaches alice" || ko "DM missing on PrivaPub"
until_true 5 '! gcurl -s -H "$GH" "$G/api/v1/timelines/public?local=false" | grep -q "a secret"' && ok "the DM is not public on GoToSocial" || ko "DM leaked to a public timeline"
echo "polls"
pp_poll=$(curl -s -X POST -H "$PH" $P/api/v1/statuses -d 'status=tea or coffee&visibility=public&poll[options][]=tea&poll[options][]=coffee&poll[expires_in]=3600' | j "print(d['uri'])")
until_true 20 'on_gts "$pp_poll" | j "print(len(d[\"statuses\"][0][\"poll\"][\"options\"]))" | grep -q 2' && ok "alice's poll reaches GoToSocial as a poll" || ko "poll missing on GoToSocial"
pp_poll_on_gts=$(on_gts "$pp_poll" | j "print(d['statuses'][0]['poll']['id'])")
gcurl -s -o /dev/null -X POST -H "$GH" "$G/api/v1/polls/$pp_poll_on_gts/votes" -d 'choices[]=1'
pp_poll_id=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if s['poll'] and s['uri']=='$pp_poll'))")
until_true 20 '[ "$(curl -s -H "$PH" "$P/api/v1/polls/$pp_poll_id" | j "print(d[\"options\"][1][\"votes_count\"])")" = "1" ]' && ok "gtsuser's vote counts on PrivaPub" || ko "vote not counted on PrivaPub"
gts_poll=$(gcurl -s -X POST -H "$GH" $G/api/v1/statuses -d 'status=red or blue&visibility=public&poll[options][]=red&poll[options][]=blue&poll[expires_in]=3600' | j "print(d['id'])")
until_true 20 'curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(any(s[\"poll\"] and s[\"content\"].find(\"red or blue\")>=0 for s in d))" | grep -q True' && ok "gtsuser's poll reaches alice as a poll" || ko "poll missing on PrivaPub"
gts_poll_on_pp=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['poll']['id'] for s in d if s['poll'] and 'red or blue' in s['content']))")
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/polls/$gts_poll_on_pp/votes" -d 'choices[]=0'
until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/statuses/$gts_poll" | j "print(d[\"poll\"][\"options\"][0][\"votes_count\"])")" = "1" ]' && ok "alice's vote counts on GoToSocial" || ko "vote not counted on GoToSocial"
echo "quotes"
quote_target=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'red or blue' in s['content']))")
[ "$(curl -s -o /dev/null -w '%{http_code}' -X POST -H "$PH" $P/api/v1/statuses -d "status=quoting&quoted_status_id=$quote_target")" = "422" ] \
&& ok "GoToSocial's author-only quote policy is respected" || ko "quoted a post whose author forbids it"
echo "link previews"
linked=$(curl -s -X POST -H "$PH" $P/api/v1/statuses --data-urlencode 'status=have a look https://gts.test/@gtsuser' -d 'visibility=public' | j "print(d['id'])")
until_true 45 '[ -n "$(curl -s -H "$PH" "$P/api/v1/statuses/$linked" | j "print((d[\"card\"] or {}).get(\"title\") or \"\")")" ]' && ok "the server builds a card for a linked page" || ko "no card for the linked page"
echo "edits and deletes"
curl -s -o /dev/null -X PUT -H "$PH" $P/api/v1/statuses/$pp_post -d 'status=Hello from PrivaPub, edited'
until_true 20 'gcurl -s -H "$GH" "$G/api/v1/statuses/$pp_on_gts" | grep -q "edited"' && ok "alice's edit reaches GoToSocial" || ko "edit not applied"