Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f9658a8e7a | ||
|
|
6f1ab0073c | ||
|
|
001fdac3be |
No files matched your search
@@ -212,6 +212,16 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
|
||||
sibling `media-proxy`, which `/media/files` does not serve.
|
||||
3. **A client never contacts a remote server for media:** every remote URL the API returns goes through
|
||||
`IMediaProxy.Wrap`, an HMAC-signed `/media/proxy/` URL fetched by `IFederationHttp.GetMedia`.
|
||||
4. **The proxy serves three ways:**
|
||||
- **Cached:** a file already cached is served from disk, ranges included.
|
||||
- **Downloaded:** a request without a `Range` is downloaded whole, up to `Media:MaxProxiedBytes`, then cached.
|
||||
- **Streamed:** a ranged request, or anything too big to cache, is streamed from the origin with the range passed on,
|
||||
and never cached. That is how remote video plays.
|
||||
|
||||
nginx has a `/media/proxy/` location with `proxy_buffering off` and a 600 s read timeout for those streams.
|
||||
5. **Remote video and audio become one playable attachment** in the Mastodon API (`MastodonMapper.Playable`): the best MP4
|
||||
up to 720p that carries both sound and picture, including PeerTube's fragmented files inside an HLS entry. HLS
|
||||
playlists themselves are not rewritten.
|
||||
|
||||
## Privacy invariants
|
||||
|
||||
@@ -258,6 +268,17 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
|
||||
- **A poll vote is a `Note` with a `name`, an `inReplyTo` that is a poll we hold, and no content.** `CreateHandler`
|
||||
hands it to `PollService.Receive` before anything else, so it never becomes a reply. Our votes on other servers' polls
|
||||
go only to the poll's author, without `published` (PieFed counts a vote only then).
|
||||
- **Link previews follow owner decision 1** (`Domain/Content/LinkPreviews.cs`): only public posts, queued on arrival with
|
||||
0–60 s of jitter, one cached `LinkPreview` per address for the whole server, never fetched when someone reads.
|
||||
`LinkPreviews.Wanted` is called where posts are saved (`CreateHandler`, `StoreContext`, `StatusService.Publish`).
|
||||
- **Quote states come from `QuoteService.Resolve`:**
|
||||
- with a stamp: accepted only if `Verified` (fetched, on the quoted author's origin, naming both posts exactly);
|
||||
- a FEP-044f `quote` without a stamp: pending;
|
||||
- older keys only: accepted when the quoted post is public.
|
||||
|
||||
`QuotesCount` moves with the accepted state, never with the raw key. When a persona quotes, `QuotePermission` decides:
|
||||
asking first for posts that state a policy, quoting at once for posts that state none, refusing otherwise. Only `quote`
|
||||
of a post that asks for consent puts `quote` in our JSON; older-key quotes leave it out, as Sharkey learned they must.
|
||||
- **A server is described on arrival, never on read.** The first record from a host enqueues `DescribeInstance` (its
|
||||
NodeInfo, at most once a week, into `RemoteInstance`), so opening the details view tells nobody anything.
|
||||
- **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post
|
||||
|
||||
+29
-1
@@ -117,6 +117,21 @@ content, one per choice, counted once per voter. Counts are refreshed with an `U
|
||||
minutes. Our own votes on other servers' polls are sent the same way to the poll's author only, without `published`.
|
||||
An incoming `Update` without a newer `updated` only refreshes counts and details; it is never recorded as an edit.
|
||||
|
||||
**Quotes** (FEP-044f, with the older keys):
|
||||
- **Received.** A quote is read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 `Link` tag.
|
||||
- With a `quoteAuthorization`, the stamp is fetched and must be a `QuoteAuthorization` on the quoted author's origin,
|
||||
attributed to them, naming exactly the two posts.
|
||||
- A FEP-044f `quote` without a stamp is pending until an `Update` brings one.
|
||||
- A quote made with the older keys only, by software that asks nobody, is shown when the quoted post is public.
|
||||
- A `Delete` of a stamp revokes the quote.
|
||||
- **Sent.** Our accounts quote with every key, plus the `RE:` fallback.
|
||||
- When the quoted post states an `interactionPolicy.canQuote` that allows us, `quote` is included and a `QuoteRequest`
|
||||
(with the quoting post as `instrument`) goes to the quoted author. The quote stays pending until their `Accept` brings a
|
||||
stamp we can verify; the post is then updated with `quoteAuthorization`.
|
||||
- Posts that state no policy are quoted the older way, without `quote`.
|
||||
- Quoting posts are also delivered to the quoted author.
|
||||
- **Not yet.** Our own posts cannot be quoted by others yet: we do not issue stamps.
|
||||
|
||||
**Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object.
|
||||
**Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`,
|
||||
`memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
|
||||
@@ -128,6 +143,18 @@ Received `summary` is a content warning only on a `Note` or `Question`, or when
|
||||
Visibility is expressed in `to`/`cc` the way Mastodon does it: public, unlisted, followers-only and direct. Inbound
|
||||
followers-only posts are recognised by the author's own `followers` collection.
|
||||
|
||||
## Link previews
|
||||
|
||||
For a public post that links to a page, PrivaPub builds a preview card. It uses, in order:
|
||||
1. a Link attachment's own `preview` (FEP-8967) or the post's own image, title and summary;
|
||||
2. otherwise, the linked page, read once by the server between 0 and 60 seconds after the post arrives (never when
|
||||
someone reads it), with an `Accept: text/html` request that reads at most 512 KB, through the same address checks as
|
||||
every other fetch.
|
||||
|
||||
The page's OpenGraph and Twitter tags give the title, description and image; the result is cached per address for 7
|
||||
days and shared by every account on the server. Images are served to clients only through PrivaPub's media proxy.
|
||||
`Federation:FetchLinkPreviews=false` turns page fetching off.
|
||||
|
||||
## Local-only posts
|
||||
|
||||
Posts with a location (shown to nearby users of this server) never leave the server, in any form.
|
||||
@@ -154,6 +181,7 @@ Posts with a location (shown to nearby users of this server) never leave the ser
|
||||
## Known limitations
|
||||
|
||||
- Our own posts carry no custom emoji: emoji are received and shown, not offered.
|
||||
- Remote media is fetched through this server's proxy when a local client displays it.
|
||||
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
|
||||
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
|
||||
- Collections expose counts, not members.
|
||||
- Only `rsa-sha256`-style keys are verified. RFC 9421 signatures are planned.
|
||||
@@ -0,0 +1,43 @@
|
||||
using PrivaPub.Domain.Content;
|
||||
using PrivaPub.Models.Post;
|
||||
|
||||
namespace PrivaPub.Tests.Domain
|
||||
{
|
||||
public class LinkPreviewTests
|
||||
{
|
||||
[Fact]
|
||||
public void Reads_open_graph_then_twitter_then_the_page_title()
|
||||
{
|
||||
var preview = LinkPreviews.Read("https://news.example/a?utm_source=x", new Uri("https://news.example/a"), """
|
||||
<html><head><title>Fallback title</title>
|
||||
<meta property="og:title" content="Rain & sun">
|
||||
<meta name="twitter:description" content="A <b>long</b> story">
|
||||
<meta property="og:image" content="/img/cover.jpg">
|
||||
<meta property="og:site_name" content="News">
|
||||
</head><body></body></html>
|
||||
""");
|
||||
|
||||
Assert.Equal("Rain & sun", preview.Title);
|
||||
Assert.Equal("A long story", preview.Description);
|
||||
Assert.Equal("https://news.example/img/cover.jpg", preview.ImageURL);
|
||||
Assert.Equal("News", preview.SiteName);
|
||||
Assert.False(preview.Failed);
|
||||
Assert.True(LinkPreviews.Read("https://x.example", new Uri("https://x.example"), "<html><body>nothing</body></html>").Failed);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Picks_the_first_real_link_and_skips_mentions_tags_quotes_and_ourselves() =>
|
||||
Assert.Equal("https://blog.example/post", LinkPreviews.FirstLink(new Post
|
||||
{
|
||||
Url = "https://m.example/@a/1",
|
||||
ContentHtml = """
|
||||
<p><span class="h-card"><a href="https://m.example/@bob" class="u-url mention">@bob</a></span>
|
||||
<a href="https://m.example/tags/x" class="mention hashtag" rel="tag">#x</a>
|
||||
<span class="quote-inline">RE: <a href="https://k.example/notes/1">link</a></span>
|
||||
<a href="https://privapub.test/@me">me</a>
|
||||
<a href="https://m.example/@a/1">self</a>
|
||||
<a href="https://blog.example/post">read this</a></p>
|
||||
"""
|
||||
}, "https://privapub.test"));
|
||||
}
|
||||
}
|
||||
@@ -52,13 +52,15 @@ namespace PrivaPub.Tests.Federation
|
||||
var db = new DbEntities();
|
||||
_blocks = new DomainBlocks(NullLogger<DomainBlocks>.Instance);
|
||||
_receiver = new InboxReceiver(_local, remote, queue, _blocks, NullLogger<InboxReceiver>.Instance);
|
||||
var remotePosts = new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue), new NoPreviews());
|
||||
var quotes = new QuoteService(db, remote, remotePosts, _local, delivery, new OutboxPublisher(db, _local, delivery));
|
||||
_processor = new InboxProcessor(remote, new IActivityHandler[]
|
||||
{
|
||||
new FollowHandler(db, _local, remote, delivery),
|
||||
new UndoHandler(db, _local, new Reactions(db, delivery)),
|
||||
new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue)), new GroupDistributor(delivery), new ObjectRecords(queue), new PollService(db, _local, delivery, queue)),
|
||||
new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery)),
|
||||
new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue))
|
||||
new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), remotePosts, new GroupDistributor(delivery), new ObjectRecords(queue), new PollService(db, _local, delivery, queue), new NoPreviews(), quotes),
|
||||
new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery), quotes),
|
||||
new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue), quotes)
|
||||
}, NullLogger<InboxProcessor>.Instance);
|
||||
}
|
||||
|
||||
|
||||
@@ -214,6 +214,30 @@ namespace PrivaPub.Tests.Federation
|
||||
Assert.Equal(new DateTime(2026, 9, 30, 12, 0, 0, DateTimeKind.Utc), closed.Poll.ClosedAt);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Finds_playable_files_inside_an_hls_only_peertube_video_and_picks_one_with_sound()
|
||||
{
|
||||
var note = Parse("""
|
||||
{
|
||||
"id": "https://tube.example/videos/watch/2", "type": "Video", "name": "HLS only", "attributedTo": "https://tube.example/accounts/ann",
|
||||
"url": [
|
||||
{ "type": "Link", "mediaType": "text/html", "href": "https://tube.example/w/def" },
|
||||
{ "type": "Link", "mediaType": "application/x-mpegURL", "href": "https://tube.example/master.m3u8", "tag": [
|
||||
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/1080-fragmented.mp4", "height": 1080,
|
||||
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }] },
|
||||
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/720-fragmented.mp4", "height": 720,
|
||||
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }] },
|
||||
{ "type": "Link", "mediaType": "video/mp4", "href": "https://tube.example/480-fragmented.mp4", "height": 480,
|
||||
"attachment": [{ "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "video" }, { "type": "PropertyValue", "name": "ffprobe_codec_type", "value": "audio" }] }
|
||||
] }
|
||||
]
|
||||
}
|
||||
""");
|
||||
|
||||
Assert.Equal(3, note.Video.Variants.Count);
|
||||
Assert.Equal("https://tube.example/480-fragmented.mp4", MastodonMapper.BestVideo(note.Video.Variants).Url);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Normalises_hashtags_like_mastodon() =>
|
||||
Assert.Equal("fedi".Normalize(System.Text.NormalizationForm.FormKC).ToLowerInvariant(), Assert.Single(Parse("""
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.ClientModels.Post;
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Tests.Support;
|
||||
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
namespace PrivaPub.Tests.Federation
|
||||
{
|
||||
[Trait("Category", "Integration")]
|
||||
public sealed class QuoteTests : IAsyncLifetime
|
||||
{
|
||||
Harness _harness;
|
||||
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||
_harness = await Harness.Start();
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_harness != default)
|
||||
await _harness.DisposeAsync();
|
||||
}
|
||||
|
||||
static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority);
|
||||
|
||||
static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}";
|
||||
|
||||
async Task<Post> Delivered(RemoteActor author, string mentioned, Action<JsonObject> shape = default)
|
||||
{
|
||||
var note = new JsonObject
|
||||
{
|
||||
["id"] = NewId(author, "notes"), ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = "<p>hello</p>",
|
||||
["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(mentioned),
|
||||
["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = mentioned, ["name"] = "@someone" })
|
||||
};
|
||||
shape?.Invoke(note);
|
||||
await _harness.Deliver(author, "/human-centipede", new JsonObject { ["id"] = NewId(author, "activities"), ["type"] = "Create", ["actor"] = author.Id, ["object"] = note });
|
||||
return await DB.Default.Find<Post>().Match(p => p.ObjectURI == note["id"]!.GetValue<string>()).ExecuteFirstAsync(TestContext.Current.CancellationToken);
|
||||
}
|
||||
|
||||
string Stamp(RemoteActor quotedAuthor, string quoting, string quoted)
|
||||
{
|
||||
var path = $"/stamps/{Guid.NewGuid():N}";
|
||||
var id = Origin(quotedAuthor) + path;
|
||||
_harness.Peer.Serve(path, new JsonObject
|
||||
{
|
||||
["@context"] = "https://www.w3.org/ns/activitystreams", ["id"] = id, ["type"] = "QuoteAuthorization",
|
||||
["attributedTo"] = quotedAuthor.Id, ["interactingObject"] = quoting, ["interactionTarget"] = quoted
|
||||
}.ToJsonString());
|
||||
return id;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_misskey_quote_of_a_public_post_is_shown_and_counted()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var (_, alice) = await _harness.Persona("alice");
|
||||
var ann = new RemoteActor(_harness.Peer, "ann");
|
||||
var kitty = new RemoteActor(_harness.Peer, "kitty");
|
||||
var original = await Delivered(ann, alice.Uri);
|
||||
|
||||
var quoting = await Delivered(kitty, alice.Uri, n =>
|
||||
{
|
||||
n["_misskey_quote"] = original.ObjectURI;
|
||||
n["content"] = $"<p>look<span class=\"quote-inline\"><br>RE: <a href=\"{original.ObjectURI}\">{original.ObjectURI}</a></span></p>";
|
||||
});
|
||||
|
||||
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
|
||||
Assert.Equal(original.ID, quoting.QuotedPostId);
|
||||
Assert.Equal(1, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
|
||||
var status = await new PrivaPub.Api.Mastodon.Mappers.MastodonMapper(_harness.Db, _harness.Local).Status(quoting, alice.Id, token);
|
||||
Assert.Equal("accepted", status.Quote.State);
|
||||
Assert.Equal(original.ID, status.Quote.QuotedStatus.Id);
|
||||
Assert.DoesNotContain("RE:", status.Content);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_stamped_quote_is_accepted_a_forged_one_is_not_and_revoking_takes_it_back()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var (_, alice) = await _harness.Persona("alice");
|
||||
var ann = new RemoteActor(_harness.Peer, "ann");
|
||||
var bob = new RemoteActor(_harness.Peer, "bob");
|
||||
var original = await Delivered(ann, alice.Uri);
|
||||
var quotingId = NewId(bob, "notes");
|
||||
var stamp = Stamp(ann, quotingId, original.ObjectURI);
|
||||
var forged = Stamp(ann, NewId(bob, "notes"), original.ObjectURI);
|
||||
|
||||
var quoting = await Delivered(bob, alice.Uri, n => { n["id"] = quotingId; n["quote"] = original.ObjectURI; n["quoteAuthorization"] = stamp; });
|
||||
var cheat = await Delivered(bob, alice.Uri, n => { n["quote"] = original.ObjectURI; n["quoteAuthorization"] = forged; });
|
||||
var waiting = await Delivered(bob, alice.Uri, n => n["quote"] = original.ObjectURI);
|
||||
|
||||
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
|
||||
Assert.Equal(QuoteState.Unauthorized, cheat.QuoteState);
|
||||
Assert.Equal(QuoteState.Pending, waiting.QuoteState);
|
||||
|
||||
await _harness.Deliver(ann, "/human-centipede", new JsonObject
|
||||
{
|
||||
["id"] = NewId(ann, "activities"), ["type"] = "Delete", ["actor"] = ann.Id, ["object"] = stamp
|
||||
});
|
||||
Assert.Equal(QuoteState.Revoked, (await DB.Default.Find<Post>().OneAsync(quoting.ID, token)).QuoteState);
|
||||
Assert.Equal(0, (await DB.Default.Find<Post>().OneAsync(original.ID, token)).QuotesCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_persona_asks_before_quoting_and_is_stamped_when_accepted()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var (_, alice) = await _harness.Persona("alice");
|
||||
var ann = new RemoteActor(_harness.Peer, "ann");
|
||||
var original = await Delivered(ann, alice.Uri, n => n["interactionPolicy"] = new JsonObject
|
||||
{
|
||||
["canQuote"] = new JsonObject { ["automaticApproval"] = new JsonArray(Addressing.Public) }
|
||||
});
|
||||
|
||||
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "this", QuotedStatusId = original.ID }, token);
|
||||
Assert.Equal(QuoteState.Pending, outcome.Post.QuoteState);
|
||||
var request = Assert.Single(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "QuoteRequest");
|
||||
Assert.Equal(original.ObjectURI, request["object"]!.GetValue<string>());
|
||||
Assert.Equal(original.ObjectURI, request["instrument"]!["quote"]!.GetValue<string>());
|
||||
|
||||
await _harness.Deliver(ann, "/human-centipede", new JsonObject
|
||||
{
|
||||
["id"] = NewId(ann, "activities"), ["type"] = "Accept", ["actor"] = ann.Id,
|
||||
["object"] = request["id"]!.GetValue<string>(), ["result"] = Stamp(ann, outcome.Post.ObjectURI, original.ObjectURI)
|
||||
});
|
||||
|
||||
var quoting = await DB.Default.Find<Post>().OneAsync(outcome.Post.ID, token);
|
||||
Assert.Equal(QuoteState.Accepted, quoting.QuoteState);
|
||||
Assert.NotNull(quoting.QuoteAuthorizationURI);
|
||||
Assert.Contains(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue<string>() == "Update"
|
||||
&& a["object"]!["quoteAuthorization"]?.GetValue<string>() == quoting.QuoteAuthorizationURI);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_legacy_post_is_quoted_without_asking_and_another_persona_cannot_be_quoted_yet()
|
||||
{
|
||||
var token = TestContext.Current.CancellationToken;
|
||||
var (aliceRoot, alice) = await _harness.Persona("alice");
|
||||
var (bobRoot, bob) = await _harness.Persona("bob");
|
||||
var kitty = new RemoteActor(_harness.Peer, "kitty");
|
||||
var legacy = await Delivered(kitty, alice.Uri);
|
||||
await _harness.Posts.InsertPost(bobRoot, new InsertPostForm { AvatarId = bob.Id, Text = "mine" }, token);
|
||||
var bobs = await DB.Default.Find<Post>().Match(p => p.GroupUserId == bob.Id).ExecuteFirstAsync(token);
|
||||
|
||||
var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "nice", QuotedStatusId = legacy.ID }, token);
|
||||
Assert.Equal(QuoteState.Accepted, outcome.Post.QuoteState);
|
||||
Assert.False(outcome.Post.QuoteByConsent);
|
||||
Assert.False((await _harness.Statuses.Publish(alice, new StatusDraft { Text = "and this", QuotedStatusId = bobs.ID }, token)).Ok);
|
||||
Assert.True((await _harness.Statuses.Publish(bob, new StatusDraft { Text = "me again", QuotedStatusId = bobs.ID }, token)).Ok);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -46,30 +46,31 @@ namespace PrivaPub.Tests.Support
|
||||
Records = new ObjectRecords(Queue);
|
||||
Polls = new PollService(Db, Local, Delivery, Queue);
|
||||
Reactions = new Reactions(Db, Delivery);
|
||||
RemotePosts = new RemotePosts(Db, Local, Remote, new NoBlocks(), Queue, Records);
|
||||
RemotePosts = new RemotePosts(Db, Local, Remote, new NoBlocks(), Queue, Records, new NoPreviews());
|
||||
Outbox = new OutboxPublisher(Db, Local, Delivery);
|
||||
Quotes = new QuoteService(Db, Remote, RemotePosts, Local, Delivery, Outbox);
|
||||
Receiver = new InboxReceiver(Local, Remote, Queue, new NoBlocks(), NullLogger<InboxReceiver>.Instance);
|
||||
Processor = new InboxProcessor(Remote, new IActivityHandler[]
|
||||
{
|
||||
new FollowHandler(Db, Local, Remote, Delivery),
|
||||
new AcceptHandler(Db, Local),
|
||||
new RejectHandler(Db, Local),
|
||||
new AcceptHandler(Db, Local, Quotes),
|
||||
new RejectHandler(Db, Local, Quotes),
|
||||
new UndoHandler(Db, Local, Reactions),
|
||||
new LikeHandler(Db, Reactions),
|
||||
new EmojiReactHandler(Db, Reactions),
|
||||
new DislikeHandler(Db),
|
||||
new JoinHandler(Db, Local, Delivery),
|
||||
new AnnounceHandler(Db, Local, RemotePosts, Fanout, Remote),
|
||||
new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records, Polls),
|
||||
new DeleteHandler(Db, Local, Remote, Delivery, Groups),
|
||||
new UpdateHandler(Db, Local, Remote, Groups, Records),
|
||||
new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records, Polls, new NoPreviews(), Quotes),
|
||||
new DeleteHandler(Db, Local, Remote, Delivery, Groups, Quotes),
|
||||
new UpdateHandler(Db, Local, Remote, Groups, Records, Quotes),
|
||||
new FlagHandler(Db, Local)
|
||||
}, NullLogger<InboxProcessor>.Instance);
|
||||
Follows = new FollowService(Db, Local, Remote, Delivery, new KeyLocalizer<GenericRes>(), NullLogger<FollowService>.Instance);
|
||||
Content = new ContentRenderer(Local, Remote);
|
||||
Outbox = new OutboxPublisher(Db, Local, Delivery);
|
||||
Media = new MediaService(new StaticOptions<MediaOptions>(new MediaOptions { Root = Path.Combine(Path.GetTempPath(), $"privapub-media-{Guid.NewGuid():N}") }),
|
||||
Local, default, NullLogger<MediaService>.Instance);
|
||||
Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups, Polls);
|
||||
Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups, Polls, new NoPreviews(), Quotes);
|
||||
Posts = new PostsService(Db, Local, Statuses, new KeyLocalizer<GenericRes>(), NullLogger<PostsService>.Instance);
|
||||
Timelines = new TimelineService(Db, new KeyLocalizer<GenericRes>());
|
||||
Relationships = new RelationshipService(Db, Follows, Delivery);
|
||||
@@ -96,6 +97,7 @@ namespace PrivaPub.Tests.Support
|
||||
public ObjectRecords Records { get; }
|
||||
public PollService Polls { get; }
|
||||
public Reactions Reactions { get; }
|
||||
public QuoteService Quotes { get; }
|
||||
public TimelineService Timelines { get; }
|
||||
public RelationshipService Relationships { get; }
|
||||
public ReportService Reports { get; }
|
||||
@@ -148,4 +150,9 @@ namespace PrivaPub.Tests.Support
|
||||
public LocalizedString this[string name, params object[] arguments] => new(name, string.Format(name, arguments));
|
||||
public IEnumerable<LocalizedString> GetAllStrings(bool includeParentCultures) => Enumerable.Empty<LocalizedString>();
|
||||
}
|
||||
|
||||
public sealed class NoPreviews : ILinkPreviews
|
||||
{
|
||||
public Task Wanted(PrivaPub.Models.Post.Post post, CancellationToken token) => Task.CompletedTask;
|
||||
}
|
||||
}
|
||||
@@ -73,6 +73,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
domain = Domain,
|
||||
title = "PrivaPub",
|
||||
version = Version,
|
||||
api_versions = new { mastodon = 7 },
|
||||
source_url = "https://git.thepra.dev/thepra/SocialPub",
|
||||
description = Description,
|
||||
usage = new { users = new { active_month = 0 } },
|
||||
|
||||
@@ -60,13 +60,38 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
[HttpGet("/media/proxy/{signature}/{encoded}"), AllowAnonymous, ApiExplorerSettings(IgnoreApi = true)]
|
||||
public async Task<IActionResult> Proxy(string signature, string encoded, CancellationToken token)
|
||||
{
|
||||
var (path, contentType) = await _proxy.Fetch(signature, encoded, token);
|
||||
if (path == default)
|
||||
var url = _proxy.Verified(signature, encoded);
|
||||
if (url == default)
|
||||
return NotFound();
|
||||
Response.Headers["X-Content-Type-Options"] = "nosniff";
|
||||
Response.Headers["Content-Security-Policy"] = "default-src 'none'; sandbox";
|
||||
Response.Headers["Cache-Control"] = "public, max-age=604800";
|
||||
return PhysicalFile(path, contentType);
|
||||
if (_proxy.Cached(url) is { Path: not null } cached)
|
||||
return PhysicalFile(cached.Path, cached.ContentType, enableRangeProcessing: true);
|
||||
if (Request.Headers.Range.Count == 0)
|
||||
{
|
||||
var (path, contentType) = await _proxy.Fetch(signature, encoded, token);
|
||||
if (path != default)
|
||||
return PhysicalFile(path, contentType, enableRangeProcessing: true);
|
||||
}
|
||||
return await Stream(url, token);
|
||||
}
|
||||
|
||||
async Task<IActionResult> Stream(string url, CancellationToken token)
|
||||
{
|
||||
var range = System.Net.Http.Headers.RangeHeaderValue.TryParse(Request.Headers.Range.ToString(), out var asked) ? asked : default;
|
||||
using var upstream = await _proxy.Open(url, range, token);
|
||||
if (upstream == default)
|
||||
return NotFound();
|
||||
Response.StatusCode = (int)upstream.StatusCode;
|
||||
Response.ContentType = upstream.Content.Headers.ContentType?.ToString() ?? "application/octet-stream";
|
||||
if (upstream.Content.Headers.ContentLength is { } length)
|
||||
Response.ContentLength = length;
|
||||
if (upstream.Content.Headers.ContentRange is { } contentRange)
|
||||
Response.Headers.ContentRange = contentRange.ToString();
|
||||
Response.Headers.AcceptRanges = "bytes";
|
||||
await upstream.Content.CopyToAsync(Response.Body, token);
|
||||
return new EmptyResult();
|
||||
}
|
||||
|
||||
object View(MediaAttachment attachment) => new
|
||||
|
||||
@@ -54,6 +54,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
InReplyTo = Params.Get("in_reply_to_id"),
|
||||
Language = Params.Get("language") ?? Me.Settings.DefaultLanguage,
|
||||
MediaIds = Params.List("media_ids"),
|
||||
QuotedStatusId = Params.Get("quoted_status_id"),
|
||||
Poll = Params.Has("poll[options]")
|
||||
? new PollDraft(Params.List("poll[options]"), Params.Int("poll[expires_in]") ?? 0, Params.Bool("poll[multiple]") == true,
|
||||
Params.Bool("poll[hide_totals]") == true)
|
||||
@@ -226,6 +227,21 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
return Json(reblogs.Select(r => accounts.GetValueOrDefault(MastodonMapper.AuthorOf(r))).Where(a => a != default).ToList());
|
||||
}
|
||||
|
||||
[HttpGet("/api/v1/statuses/{id}/quotes"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
|
||||
public async Task<IActionResult> Quotes(string id, CancellationToken token)
|
||||
{
|
||||
if (await Visible(id, token) == default)
|
||||
return NotFoundError();
|
||||
var query = _dbEntities.Posts.Match(p => p.QuotedPostId == id && p.QuoteState == QuoteState.Accepted && !p.DeletedAt.HasValue);
|
||||
var quotes = await Page.From(Params, Limit()).Fetch(query, p => p.ID, token);
|
||||
var visible = new List<PostEntity>();
|
||||
foreach (var quote in quotes)
|
||||
if (quote.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quote, MyId, token))
|
||||
visible.Add(quote);
|
||||
Link($"/api/v1/statuses/{id}/quotes", quotes.LastOrDefault()?.ID, quotes.FirstOrDefault()?.ID);
|
||||
return Json(await _mapper.Statuses(visible, MyId, token));
|
||||
}
|
||||
|
||||
[HttpPost("/api/v1/statuses/{id}/bookmark"), Scope("write:bookmarks")]
|
||||
public async Task<IActionResult> Bookmark(string id, CancellationToken token)
|
||||
{
|
||||
|
||||
@@ -164,7 +164,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
|
||||
[NotificationType.Reblog] = "reblog",
|
||||
[NotificationType.Update] = "update",
|
||||
[NotificationType.Poll] = "poll",
|
||||
[NotificationType.Reaction] = "pleroma:emoji_reaction"
|
||||
[NotificationType.Reaction] = "pleroma:emoji_reaction",
|
||||
[NotificationType.Quote] = "quote"
|
||||
};
|
||||
|
||||
readonly MastodonMapper _mapper;
|
||||
|
||||
@@ -161,6 +161,20 @@ namespace PrivaPub.Api.Mastodon.Entities
|
||||
public int Down { get; set; }
|
||||
}
|
||||
|
||||
public class QuoteEntity
|
||||
{
|
||||
public string State { get; set; }
|
||||
public Status QuotedStatus { get; set; }
|
||||
public string QuotedStatusId { get; set; }
|
||||
}
|
||||
|
||||
public class QuoteApprovalEntity
|
||||
{
|
||||
public List<string> Automatic { get; set; } = new();
|
||||
public List<string> Manual { get; set; } = new();
|
||||
public string CurrentUser { get; set; } = "denied";
|
||||
}
|
||||
|
||||
public class EmojiReactionEntity
|
||||
{
|
||||
public string Name { get; set; }
|
||||
@@ -253,6 +267,9 @@ namespace PrivaPub.Api.Mastodon.Entities
|
||||
public PrivaPubStatus Privapub { get; set; }
|
||||
public List<EmojiReactionEntity> EmojiReactions { get; set; } = new();
|
||||
public PleromaStatus Pleroma { get; set; }
|
||||
public QuoteEntity Quote { get; set; }
|
||||
public int QuotesCount { get; set; }
|
||||
public QuoteApprovalEntity QuoteApproval { get; set; }
|
||||
}
|
||||
|
||||
public class MediaAttachment
|
||||
|
||||
@@ -7,6 +7,8 @@ using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Rendering;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.Group;
|
||||
using PrivaPub.Domain.Privacy;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Models.Social;
|
||||
@@ -146,7 +148,10 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
public async Task<Status> Status(PostEntity post, string viewerId, CancellationToken token) =>
|
||||
(await Statuses(new[] { post }, viewerId, token)).FirstOrDefault();
|
||||
|
||||
public async Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, CancellationToken token)
|
||||
public Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, CancellationToken token) =>
|
||||
Statuses(posts, viewerId, nested: false, token);
|
||||
|
||||
async Task<List<Status>> Statuses(IReadOnlyCollection<PostEntity> posts, string viewerId, bool nested, CancellationToken token)
|
||||
{
|
||||
var originalIds = posts.Where(p => p.ReblogOfPostId != default).Select(p => p.ReblogOfPostId).Distinct().ToList();
|
||||
var originals = originalIds.Count == 0
|
||||
@@ -220,7 +225,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
SpoilerText = post.SpoilerText ?? (post.HasContentWarning ? post.Title ?? ActivityPubRenderer.ContentWarning : string.Empty),
|
||||
Visibility = Visibility(post.Visibility),
|
||||
Language = post.Language,
|
||||
MediaAttachments = post.Media.Select(Media).ToList(),
|
||||
MediaAttachments = post.Media.Count > 0 ? post.Media.Select(Media).ToList() : Playable(post),
|
||||
Card = Card(post),
|
||||
Poll = Poll(post, viewerId, ownVotes.GetValueOrDefault(post.ID)),
|
||||
Emojis = Emojis(post.Emojis),
|
||||
@@ -233,6 +238,30 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
return status;
|
||||
}
|
||||
|
||||
var quotedIds = nested ? new List<string>() : all.Where(p => p.QuoteState == QuoteState.Accepted && p.QuotedPostId != default).Select(p => p.QuotedPostId).Distinct().ToList();
|
||||
var quotedPosts = new List<PostEntity>();
|
||||
foreach (var quotedPost in quotedIds.Count == 0 ? new List<PostEntity>() : await _dbEntities.Posts.Match(p => quotedIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token))
|
||||
if (quotedPost.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quotedPost, viewerId, token))
|
||||
quotedPosts.Add(quotedPost);
|
||||
var quotedStatuses = quotedPosts.Count == 0
|
||||
? new Dictionary<string, Status>()
|
||||
: (await Statuses(quotedPosts, viewerId, nested: true, token)).ToDictionary(q => q.Id);
|
||||
|
||||
void Quote(Status status, PostEntity post)
|
||||
{
|
||||
status.QuotesCount = post.QuotesCount;
|
||||
status.QuoteApproval = Approval(post, viewerId);
|
||||
if (post.QuoteState == QuoteState.None)
|
||||
return;
|
||||
var state = post.QuoteState.ToString().ToLowerInvariant();
|
||||
var quoted = post.QuotedPostId != default && quotedStatuses.TryGetValue(post.QuotedPostId, out var shown) ? shown : default;
|
||||
status.Quote = nested
|
||||
? new QuoteEntity { State = state, QuotedStatusId = post.QuotedPostId }
|
||||
: new QuoteEntity { State = state, QuotedStatus = post.QuoteState == QuoteState.Accepted ? quoted : default, QuotedStatusId = post.QuotedPostId };
|
||||
if (!nested && post.QuoteState == QuoteState.Accepted && quoted != default)
|
||||
status.Content = WithoutQuoteFallback(status.Content);
|
||||
}
|
||||
|
||||
var mapped = new List<Status>();
|
||||
foreach (var post in posts)
|
||||
{
|
||||
@@ -242,10 +271,12 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
var status = Map(post);
|
||||
if (status == default)
|
||||
continue;
|
||||
Quote(status, post);
|
||||
if (post.ReblogOfPostId != default)
|
||||
{
|
||||
if (!originals.TryGetValue(post.ReblogOfPostId, out var original) || Map(original) is not { } inner)
|
||||
continue;
|
||||
Quote(inner, original);
|
||||
status.Reblog = inner;
|
||||
status.Content = string.Empty;
|
||||
status.Reblogged = reblogged.Contains(original.ID);
|
||||
@@ -257,6 +288,33 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
|
||||
public static string AuthorOf(PostEntity post) => post.AuthorAccountId ?? post.GroupUserId;
|
||||
|
||||
static readonly AngleSharp.Html.Parser.HtmlParser Fragments = new();
|
||||
|
||||
public static string WithoutQuoteFallback(string html)
|
||||
{
|
||||
if (string.IsNullOrEmpty(html) || !html.Contains("quote-inline", StringComparison.Ordinal))
|
||||
return html;
|
||||
var document = Fragments.ParseDocument($"<body>{html}</body>");
|
||||
foreach (var fallback in document.QuerySelectorAll(".quote-inline").ToList())
|
||||
fallback.Remove();
|
||||
return document.Body!.InnerHtml;
|
||||
}
|
||||
|
||||
static QuoteApprovalEntity Approval(PostEntity post, string viewerId)
|
||||
{
|
||||
if (!post.IsFederatedCopy || post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
|
||||
return new QuoteApprovalEntity();
|
||||
if (post.QuotePolicy is not { } policy)
|
||||
return new QuoteApprovalEntity { Automatic = new List<string> { "public" }, CurrentUser = viewerId == default ? "unknown" : "automatic" };
|
||||
static List<string> Named(IEnumerable<string> who) => who.Select(w => Addressing.IsPublic(w) ? "public" : w.EndsWith("/followers") ? "followers" : "unsupported_policy").Distinct().ToList();
|
||||
return new QuoteApprovalEntity
|
||||
{
|
||||
Automatic = Named(policy.Automatic),
|
||||
Manual = Named(policy.Manual),
|
||||
CurrentUser = viewerId == default ? "unknown" : policy.Automatic.Any(Addressing.IsPublic) ? "automatic" : policy.Manual.Any(Addressing.IsPublic) ? "manual" : "denied"
|
||||
};
|
||||
}
|
||||
|
||||
public static string Content(PostEntity post)
|
||||
{
|
||||
var content = post.ContentHtml ?? ActivityPubRenderer.Html(post.Text);
|
||||
@@ -387,13 +445,42 @@ namespace PrivaPub.Api.Mastodon.Mappers
|
||||
Streams = variant.Streams
|
||||
};
|
||||
|
||||
List<MediaAttachment> Playable(PostEntity post)
|
||||
{
|
||||
var (variant, kind, duration) = post.Video is { } video ? (BestVideo(video.Variants), "video", video.DurationSeconds)
|
||||
: post.Audio is { } audio ? (audio.Variants.FirstOrDefault(), "audio", audio.DurationSeconds)
|
||||
: default;
|
||||
if (variant == default)
|
||||
return new List<MediaAttachment>();
|
||||
return new List<MediaAttachment>
|
||||
{
|
||||
Media(new PostMedia
|
||||
{
|
||||
Id = new Guid(System.Security.Cryptography.MD5.HashData(System.Text.Encoding.UTF8.GetBytes(variant.Url))),
|
||||
RemoteURL = variant.Url,
|
||||
ContentType = variant.MediaType ?? kind + "/*",
|
||||
RemotePreviewURL = post.CoverURL,
|
||||
Width = variant.Width,
|
||||
Height = variant.Height,
|
||||
DurationSeconds = duration,
|
||||
Description = post.Title
|
||||
})
|
||||
};
|
||||
}
|
||||
|
||||
public static MediaVariant BestVideo(IReadOnlyCollection<MediaVariant> variants)
|
||||
{
|
||||
var playable = variants.Where(v => v.MediaType?.StartsWith("video/") == true && (v.Streams.Count == 0 || v.Streams.Contains("audio") && v.Streams.Contains("video"))).ToList();
|
||||
return playable.Where(v => v.Height <= 720).OrderByDescending(v => v.Height).FirstOrDefault()
|
||||
?? playable.OrderBy(v => v.Height ?? int.MaxValue).FirstOrDefault();
|
||||
}
|
||||
|
||||
PreviewCard Card(PostEntity post)
|
||||
{
|
||||
if (!post.IsFederatedCopy)
|
||||
return default;
|
||||
if (post.Link is { } link)
|
||||
return Card(link.Href, link.Title ?? post.Title, link.Description ?? (post.Media.Count == 0 ? post.Excerpt : default), link.ImageURL);
|
||||
if (post.ObjectType is "Article" or "Event" or "Video" or "Audio" && post.Url != default && post.Media.Count == 0)
|
||||
if (post.IsFederatedCopy && post.ObjectType is "Article" or "Event" && post.Url != default && post.Media.Count == 0
|
||||
|| post.IsFederatedCopy && post.ObjectType is "Video" or "Audio" && post.Url != default && post.Media.Count == 0 && Playable(post).Count == 0)
|
||||
return Card(post.Url, post.Title, post.Excerpt, post.CoverURL);
|
||||
return default;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
using AngleSharp.Html.Parser;
|
||||
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Infrastructure.Http;
|
||||
using PrivaPub.Infrastructure.Jobs;
|
||||
using PrivaPub.Models.Jobs;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Security.Cryptography;
|
||||
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Domain.Content
|
||||
{
|
||||
public class LinkPreview : Entity
|
||||
{
|
||||
public string Url { get; set; }
|
||||
public string Title { get; set; }
|
||||
public string Description { get; set; }
|
||||
public string ImageURL { get; set; }
|
||||
public string SiteName { get; set; }
|
||||
public bool Failed { get; set; }
|
||||
public DateTime FetchedAt { get; set; } = DateTime.UtcNow;
|
||||
}
|
||||
|
||||
public interface ILinkPreviews
|
||||
{
|
||||
Task Wanted(PostEntity post, CancellationToken token);
|
||||
}
|
||||
|
||||
public class LinkPreviews : ILinkPreviews, IJobHandler
|
||||
{
|
||||
const int MaxJitterSeconds = 60;
|
||||
static readonly TimeSpan Freshness = TimeSpan.FromDays(7);
|
||||
static readonly HtmlParser Parser = new();
|
||||
|
||||
readonly DbEntities _dbEntities;
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly IFederationHttp _http;
|
||||
readonly IJobQueue _queue;
|
||||
readonly IOptionsMonitor<FederationOptions> _options;
|
||||
|
||||
public LinkPreviews(DbEntities dbEntities, ILocalActorService localActors, IFederationHttp http, IJobQueue queue,
|
||||
IOptionsMonitor<FederationOptions> options)
|
||||
{
|
||||
_dbEntities = dbEntities;
|
||||
_localActors = localActors;
|
||||
_http = http;
|
||||
_queue = queue;
|
||||
_options = options;
|
||||
}
|
||||
|
||||
public JobKind Kind => JobKind.FetchPreview;
|
||||
public int Concurrency => 2;
|
||||
public int MaxAttempts => 1;
|
||||
public int PerHostLimit => 1;
|
||||
|
||||
public async Task Wanted(PostEntity post, CancellationToken token)
|
||||
{
|
||||
if (!_options.CurrentValue.FetchLinkPreviews || post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted) || post.IsLocalOnly
|
||||
|| post.ReblogOfPostId != default || post.Link?.Title != default || post.Link == default && post.Media.Count > 0)
|
||||
return;
|
||||
var url = post.Link?.Href ?? FirstLink(post, _localActors.BaseAddress);
|
||||
if (url == default)
|
||||
return;
|
||||
await _queue.EnqueueMany(new[]
|
||||
{
|
||||
new Job
|
||||
{
|
||||
Kind = JobKind.FetchPreview, Payload = post.ID, Host = new Uri(url).Host.ToLowerInvariant(), DedupeKey = $"preview|{post.ID}",
|
||||
RunAt = DateTime.UtcNow.AddSeconds(RandomNumberGenerator.GetInt32(0, MaxJitterSeconds + 1))
|
||||
}
|
||||
}, token);
|
||||
}
|
||||
|
||||
public async Task<JobOutcome> Handle(Job job, CancellationToken token)
|
||||
{
|
||||
var post = await _dbEntities.Posts.MatchID(job.Payload).ExecuteFirstAsync(token);
|
||||
if (post == default || post.DeletedAt.HasValue || post.Link?.Title != default)
|
||||
return JobOutcome.Done;
|
||||
var url = post.Link?.Href ?? FirstLink(post, _localActors.BaseAddress);
|
||||
if (url == default)
|
||||
return JobOutcome.Done;
|
||||
|
||||
var preview = await DB.Default.Find<LinkPreview>().Match(p => p.Url == url).ExecuteFirstAsync(token);
|
||||
if (preview == default || preview.FetchedAt < DateTime.UtcNow - Freshness)
|
||||
preview = await Fetch(url, token);
|
||||
if (preview == default || preview.Failed)
|
||||
return JobOutcome.Done;
|
||||
|
||||
var link = post.Link ?? new PostLink { Href = url };
|
||||
link.Title ??= preview.Title;
|
||||
link.Description ??= preview.Description;
|
||||
link.ImageURL ??= preview.ImageURL;
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.Link, link).ExecuteAsync(token);
|
||||
return JobOutcome.Done;
|
||||
}
|
||||
|
||||
async Task<LinkPreview> Fetch(string url, CancellationToken token)
|
||||
{
|
||||
var (finalUri, html) = await _http.GetPage(url, token);
|
||||
var preview = html == default ? new LinkPreview { Url = url, Failed = true } : Read(url, finalUri, html);
|
||||
await DB.Default.Update<LinkPreview>()
|
||||
.Match(p => p.Url == url)
|
||||
.Modify(p => p.Title, preview.Title)
|
||||
.Modify(p => p.Description, preview.Description)
|
||||
.Modify(p => p.ImageURL, preview.ImageURL)
|
||||
.Modify(p => p.SiteName, preview.SiteName)
|
||||
.Modify(p => p.Failed, preview.Failed)
|
||||
.Modify(p => p.FetchedAt, DateTime.UtcNow)
|
||||
.Option(o => o.IsUpsert = true)
|
||||
.ExecuteAsync(token);
|
||||
return preview;
|
||||
}
|
||||
|
||||
public static LinkPreview Read(string url, Uri finalUri, string html)
|
||||
{
|
||||
var document = Parser.ParseDocument(html);
|
||||
string Meta(params string[] names) => names
|
||||
.Select(name => document.QuerySelector($"meta[property='{name}'], meta[name='{name}']")?.GetAttribute("content"))
|
||||
.FirstOrDefault(value => !string.IsNullOrWhiteSpace(value));
|
||||
var image = Meta("og:image:secure_url", "og:image", "og:image:url", "twitter:image", "twitter:image:src");
|
||||
var imageUri = image != default && Uri.TryCreate(finalUri, image, out var resolved) && resolved.Scheme is "https" or "http" ? resolved.AbsoluteUri : default;
|
||||
var preview = new LinkPreview
|
||||
{
|
||||
Url = url,
|
||||
Title = ObjectShapes.Text(Meta("og:title", "twitter:title") ?? document.Title, 300),
|
||||
Description = ObjectShapes.Text(Meta("og:description", "twitter:description", "description"), 1000),
|
||||
SiteName = ObjectShapes.Text(Meta("og:site_name"), 200),
|
||||
ImageURL = imageUri
|
||||
};
|
||||
preview.Failed = preview.Title == default && preview.Description == default && preview.ImageURL == default;
|
||||
return preview;
|
||||
}
|
||||
|
||||
public static string FirstLink(PostEntity post, string ownBase)
|
||||
{
|
||||
if (string.IsNullOrEmpty(post.ContentHtml))
|
||||
return default;
|
||||
var document = Parser.ParseDocument(post.ContentHtml);
|
||||
return document.QuerySelectorAll("a[href]")
|
||||
.Where(a => !(a.ClassList.Contains("mention") || a.ClassList.Contains("hashtag") || a.GetAttribute("rel")?.Contains("tag") == true
|
||||
|| a.Closest(".quote-inline") != default))
|
||||
.Select(a => a.GetAttribute("href"))
|
||||
.FirstOrDefault(href => Uri.TryCreate(href, UriKind.Absolute, out var uri) && uri.Scheme is "https" or "http"
|
||||
&& href != post.Url && href != post.ObjectURI && !href.StartsWith(ownBase + "/", StringComparison.OrdinalIgnoreCase));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,9 @@ namespace PrivaPub.Domain.Media
|
||||
{
|
||||
string Wrap(string remoteUrl);
|
||||
Task<(string Path, string ContentType)> Fetch(string signature, string encodedUrl, CancellationToken token);
|
||||
string Verified(string signature, string encodedUrl);
|
||||
(string Path, string ContentType) Cached(string url);
|
||||
Task<HttpResponseMessage> Open(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token);
|
||||
}
|
||||
|
||||
public class MediaProxy : IMediaProxy
|
||||
@@ -43,7 +46,7 @@ namespace PrivaPub.Domain.Media
|
||||
return $"{_localActors.BaseAddress}/media/proxy/{Sign(remoteUrl)}/{encoded}";
|
||||
}
|
||||
|
||||
public async Task<(string Path, string ContentType)> Fetch(string signature, string encodedUrl, CancellationToken token)
|
||||
public string Verified(string signature, string encodedUrl)
|
||||
{
|
||||
string url;
|
||||
try
|
||||
@@ -54,19 +57,38 @@ namespace PrivaPub.Domain.Media
|
||||
{
|
||||
return default;
|
||||
}
|
||||
if (!CryptographicOperations.FixedTimeEquals(Encoding.ASCII.GetBytes(signature ?? string.Empty), Encoding.ASCII.GetBytes(Sign(url))))
|
||||
return default;
|
||||
|
||||
var name = Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(url)));
|
||||
var directory = System.IO.Path.Combine(_media.ProxyRoot, name[..2]);
|
||||
var path = System.IO.Path.Combine(directory, name);
|
||||
var typePath = path + ".type";
|
||||
if (File.Exists(path) && File.Exists(typePath))
|
||||
{
|
||||
File.SetLastWriteTimeUtc(path, DateTime.UtcNow);
|
||||
return (path, await File.ReadAllTextAsync(typePath, token));
|
||||
return CryptographicOperations.FixedTimeEquals(Encoding.ASCII.GetBytes(signature ?? string.Empty), Encoding.ASCII.GetBytes(Sign(url))) ? url : default;
|
||||
}
|
||||
|
||||
public (string Path, string ContentType) Cached(string url)
|
||||
{
|
||||
var (path, typePath) = CachePaths(url);
|
||||
if (!File.Exists(path) || !File.Exists(typePath))
|
||||
return default;
|
||||
File.SetLastWriteTimeUtc(path, DateTime.UtcNow);
|
||||
return (path, File.ReadAllText(typePath));
|
||||
}
|
||||
|
||||
public Task<HttpResponseMessage> Open(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token) =>
|
||||
_http.OpenMedia(url, range, token);
|
||||
|
||||
(string Path, string TypePath) CachePaths(string url)
|
||||
{
|
||||
var name = Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(url)));
|
||||
var path = System.IO.Path.Combine(_media.ProxyRoot, name[..2], name);
|
||||
return (path, path + ".type");
|
||||
}
|
||||
|
||||
public async Task<(string Path, string ContentType)> Fetch(string signature, string encodedUrl, CancellationToken token)
|
||||
{
|
||||
var url = Verified(signature, encodedUrl);
|
||||
if (url == default)
|
||||
return default;
|
||||
if (Cached(url) is { Path: not null } cached)
|
||||
return cached;
|
||||
var (path, typePath) = CachePaths(url);
|
||||
var directory = System.IO.Path.GetDirectoryName(path);
|
||||
|
||||
var (bytes, contentType) = await _http.GetMedia(url, _options.CurrentValue.MaxProxiedBytes, token);
|
||||
if (bytes == default)
|
||||
return default;
|
||||
|
||||
@@ -233,30 +233,11 @@ namespace PrivaPub.Domain.Statuses
|
||||
return JobOutcome.Done;
|
||||
await Closing(post, author, token);
|
||||
if (!post.IsLocalOnly)
|
||||
await _outbox.Publish(author, post, Update(post, author), token);
|
||||
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, $"poll-{DateTime.UtcNow.Ticks}"), token);
|
||||
return JobOutcome.Done;
|
||||
}
|
||||
|
||||
protected virtual Task Closing(PostEntity post, LocalActor author, CancellationToken token) => Task.CompletedTask;
|
||||
|
||||
static JsonObject Update(PostEntity post, LocalActor author)
|
||||
{
|
||||
var question = post.Visibility == PostVisibility.Direct
|
||||
? ActivityPubRenderer.DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI)
|
||||
: ActivityPubRenderer.Note(post, author, default, post.InReplyToURI);
|
||||
question["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
|
||||
question["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
|
||||
return new JsonObject
|
||||
{
|
||||
["@context"] = ActivityPubRenderer.Context(),
|
||||
["id"] = author.ActivityUri($"update-{post.ID}-poll-{DateTime.UtcNow.Ticks}"),
|
||||
["type"] = "Update",
|
||||
["actor"] = author.Uri,
|
||||
["to"] = question["to"]!.DeepClone(),
|
||||
["cc"] = question["cc"]!.DeepClone(),
|
||||
["object"] = question
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
public class PollCloseJob : PollRefreshJob
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Domain.Social;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Inbox;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Federation.Outbox;
|
||||
using PrivaPub.Federation.Rendering;
|
||||
using PrivaPub.Models.Federation;
|
||||
using PrivaPub.Models.User;
|
||||
using PrivaPub.Models.Post;
|
||||
using PrivaPub.Models.Social;
|
||||
using PrivaPub.StaticServices;
|
||||
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
using static PrivaPub.Federation.Objects.ActivityJson;
|
||||
|
||||
using PostEntity = PrivaPub.Models.Post.Post;
|
||||
|
||||
namespace PrivaPub.Domain.Statuses
|
||||
{
|
||||
public interface IQuoteService
|
||||
{
|
||||
Task Resolve(PostEntity post, NoteDocument note, CancellationToken token);
|
||||
Task Revoke(string stampUri, CancellationToken token);
|
||||
Task<bool> Verified(string stampUri, string quotingUri, PostEntity quoted, CancellationToken token);
|
||||
QuotePermission Permission(PostEntity quoted, LocalActor author);
|
||||
Task Request(LocalActor author, PostEntity post, PostEntity quoted, JsonObject note, CancellationToken token);
|
||||
Task<bool> Answered(JsonNode answer, ForeignAvatar actor, bool accepted, CancellationToken token);
|
||||
}
|
||||
|
||||
public enum QuotePermission
|
||||
{
|
||||
Denied,
|
||||
Granted,
|
||||
AskFirst
|
||||
}
|
||||
|
||||
public class QuoteService : IQuoteService
|
||||
{
|
||||
readonly DbEntities _dbEntities;
|
||||
readonly IRemoteActorService _remoteActors;
|
||||
readonly IRemotePosts _remotePosts;
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly IDeliveryService _delivery;
|
||||
readonly IOutboxPublisher _outbox;
|
||||
|
||||
public QuoteService(DbEntities dbEntities, IRemoteActorService remoteActors, IRemotePosts remotePosts, ILocalActorService localActors,
|
||||
IDeliveryService delivery, IOutboxPublisher outbox)
|
||||
{
|
||||
_dbEntities = dbEntities;
|
||||
_remoteActors = remoteActors;
|
||||
_remotePosts = remotePosts;
|
||||
_localActors = localActors;
|
||||
_delivery = delivery;
|
||||
_outbox = outbox;
|
||||
}
|
||||
|
||||
const string RequestPrefix = "quote-request-";
|
||||
|
||||
public QuotePermission Permission(PostEntity quoted, LocalActor author)
|
||||
{
|
||||
if (!quoted.IsFederatedCopy)
|
||||
return quoted.GroupUserId == author.Id && quoted.Visibility is PostVisibility.Public or PostVisibility.Unlisted
|
||||
? QuotePermission.Granted
|
||||
: QuotePermission.Denied;
|
||||
if (quoted.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted))
|
||||
return QuotePermission.Denied;
|
||||
if (quoted.QuotePolicy is not { } policy)
|
||||
return QuotePermission.Granted;
|
||||
return policy.Automatic.Concat(policy.Manual).Any(Addressing.IsPublic) || policy.Automatic.Concat(policy.Manual).Contains(author.Uri)
|
||||
? QuotePermission.AskFirst
|
||||
: QuotePermission.Denied;
|
||||
}
|
||||
|
||||
public async Task Request(LocalActor author, PostEntity post, PostEntity quoted, JsonObject note, CancellationToken token)
|
||||
{
|
||||
var owner = await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == quoted.ActorURI).ExecuteFirstAsync(token);
|
||||
if (string.IsNullOrEmpty(owner?.InboxURL))
|
||||
return;
|
||||
await _delivery.Enqueue(author, new[] { owner.InboxURL }, new JsonObject
|
||||
{
|
||||
["@context"] = ActivityPubRenderer.Context(),
|
||||
["id"] = author.ActivityUri(RequestPrefix + post.ID),
|
||||
["type"] = "QuoteRequest",
|
||||
["actor"] = author.Uri,
|
||||
["object"] = quoted.ObjectURI,
|
||||
["instrument"] = note.DeepClone(),
|
||||
["to"] = new JsonArray(quoted.ActorURI)
|
||||
}, token);
|
||||
}
|
||||
|
||||
public async Task<bool> Answered(JsonNode answer, ForeignAvatar actor, bool accepted, CancellationToken token)
|
||||
{
|
||||
var request = answer["object"];
|
||||
var requestId = Id(request);
|
||||
var marker = requestId?.LastIndexOf("/grunts/" + RequestPrefix, StringComparison.Ordinal) ?? -1;
|
||||
if (marker < 0 && !(request is JsonObject && Value(request, "type") == "QuoteRequest"))
|
||||
return false;
|
||||
if (marker < 0)
|
||||
return true;
|
||||
var postId = requestId[(marker + "/grunts/".Length + RequestPrefix.Length)..];
|
||||
var post = await _dbEntities.Posts.Match(p => p.ID == postId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
|
||||
var author = post == default ? default : await _localActors.FindById(LocalActorKind.Person, post.GroupUserId, token);
|
||||
if (author == default || author.ActivityUri(RequestPrefix + post.ID) != requestId || post.QuoteState != QuoteState.Pending)
|
||||
return true;
|
||||
var quoted = await _dbEntities.Posts.MatchID(post.QuotedPostId).ExecuteFirstAsync(token);
|
||||
if (quoted?.ActorURI != actor.ActorURI)
|
||||
return true;
|
||||
if (!accepted)
|
||||
{
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.QuoteState, QuoteState.Rejected).ExecuteAsync(token);
|
||||
return true;
|
||||
}
|
||||
var stamp = Id(answer["result"]);
|
||||
if (stamp == default || !await Verified(stamp, post.ObjectURI, quoted, token))
|
||||
return true;
|
||||
post.QuoteAuthorizationURI = stamp;
|
||||
post.QuoteState = QuoteState.Accepted;
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID)
|
||||
.Modify(p => p.QuoteAuthorizationURI, stamp)
|
||||
.Modify(p => p.QuoteState, QuoteState.Accepted)
|
||||
.ExecuteAsync(token);
|
||||
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
|
||||
if (!post.IsLocalOnly)
|
||||
await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, "quote-approved"), token);
|
||||
return true;
|
||||
}
|
||||
|
||||
public async Task Resolve(PostEntity post, NoteDocument note, CancellationToken token)
|
||||
{
|
||||
if (note.QuoteUri == default && !note.QuoteDeleted)
|
||||
return;
|
||||
var quoted = note.QuoteUri == default
|
||||
? default
|
||||
: await _dbEntities.Posts.Match(p => p.ObjectURI == note.QuoteUri && !p.DeletedAt.HasValue).ExecuteFirstAsync(token)
|
||||
?? await _remotePosts.StoreContext(note.QuoteUri, RemotePosts.MaxDepth, token);
|
||||
|
||||
var state = note.QuoteDeleted ? QuoteState.Deleted
|
||||
: quoted == default ? QuoteState.Pending
|
||||
: note.QuoteAuthorization != default
|
||||
? !quoted.IsFederatedCopy || !await Verified(note.QuoteAuthorization, post.ObjectURI, quoted, token) ? QuoteState.Unauthorized : QuoteState.Accepted
|
||||
: note.QuotesByConsent ? QuoteState.Pending
|
||||
: quoted.Visibility is PostVisibility.Public or PostVisibility.Unlisted ? QuoteState.Accepted
|
||||
: QuoteState.Unauthorized;
|
||||
|
||||
var wasCounted = post.QuoteState == QuoteState.Accepted ? post.QuotedPostId : default;
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID)
|
||||
.Modify(p => p.QuoteURI, note.QuoteUri)
|
||||
.Modify(p => p.QuotedPostId, quoted?.ID)
|
||||
.Modify(p => p.QuoteState, state)
|
||||
.Modify(p => p.QuoteAuthorizationURI, state == QuoteState.Accepted ? note.QuoteAuthorization : default)
|
||||
.ExecuteAsync(token);
|
||||
post.QuotedPostId = quoted?.ID;
|
||||
post.QuoteState = state;
|
||||
|
||||
var nowCounted = state == QuoteState.Accepted ? quoted?.ID : default;
|
||||
if (wasCounted == nowCounted)
|
||||
return;
|
||||
if (wasCounted != default)
|
||||
await DB.Default.Update<PostEntity>().MatchID(wasCounted).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
|
||||
if (nowCounted != default)
|
||||
{
|
||||
await DB.Default.Update<PostEntity>().MatchID(nowCounted).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
|
||||
if (!quoted.IsFederatedCopy)
|
||||
await Notifications.Add(quoted.GroupUserId, NotificationType.Quote, post.AuthorAccountId, post.ActorURI, post.ID, token);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task Revoke(string stampUri, CancellationToken token)
|
||||
{
|
||||
var revoked = await _dbEntities.Posts.Match(p => p.QuoteAuthorizationURI == stampUri && p.QuoteState == QuoteState.Accepted).ExecuteAsync(token);
|
||||
foreach (var post in revoked)
|
||||
{
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.ID).Modify(p => p.QuoteState, QuoteState.Revoked).ExecuteAsync(token);
|
||||
if (post.QuotedPostId != default)
|
||||
await DB.Default.Update<PostEntity>().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<bool> Verified(string stampUri, string quotingUri, PostEntity quoted, CancellationToken token)
|
||||
{
|
||||
if (!Origin.Same(stampUri, quoted.ActorURI))
|
||||
return false;
|
||||
using var fetched = await _remoteActors.FetchObject(stampUri, token);
|
||||
if (fetched == default)
|
||||
return false;
|
||||
var stamp = JsonNode.Parse(fetched.Root.GetRawText());
|
||||
return Value(stamp, "type") == "QuoteAuthorization"
|
||||
&& Id(stamp["attributedTo"]) == quoted.ActorURI
|
||||
&& Id(stamp["interactingObject"]) == quotingUri
|
||||
&& Id(stamp["interactionTarget"]) == quoted.ObjectURI;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -40,6 +40,7 @@ namespace PrivaPub.Domain.Statuses
|
||||
public double? Longitude { get; init; }
|
||||
public double? RangeKm { get; init; }
|
||||
public PollDraft Poll { get; init; }
|
||||
public string QuotedStatusId { get; init; }
|
||||
}
|
||||
|
||||
public sealed record StatusOutcome(PostEntity Post, int Status = StatusCodes.Status200OK, string Error = default)
|
||||
@@ -71,10 +72,15 @@ namespace PrivaPub.Domain.Statuses
|
||||
readonly IMediaService _media;
|
||||
readonly IGroupDistributor _groups;
|
||||
readonly IPollService _polls;
|
||||
readonly ILinkPreviews _previews;
|
||||
readonly IQuoteService _quotes;
|
||||
|
||||
public StatusService(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
|
||||
IContentRenderer content, IOutboxPublisher outbox, IFanout fanout, IMediaService media, IGroupDistributor groups, IPollService polls)
|
||||
IContentRenderer content, IOutboxPublisher outbox, IFanout fanout, IMediaService media, IGroupDistributor groups, IPollService polls,
|
||||
ILinkPreviews previews, IQuoteService quotes)
|
||||
{
|
||||
_previews = previews;
|
||||
_quotes = quotes;
|
||||
_polls = polls;
|
||||
_media = media;
|
||||
_groups = groups;
|
||||
@@ -112,6 +118,18 @@ namespace PrivaPub.Domain.Statuses
|
||||
if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token))
|
||||
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
|
||||
|
||||
PostEntity quoted = default;
|
||||
var quotePermission = QuotePermission.Denied;
|
||||
if (!string.IsNullOrEmpty(draft.QuotedStatusId))
|
||||
{
|
||||
quoted = await _dbEntities.Posts.Match(p => p.ID == draft.QuotedStatusId && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token);
|
||||
if (quoted == default || !await VisibilityPolicy.CanSee(quoted, author.Id, token))
|
||||
return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found");
|
||||
quotePermission = _quotes.Permission(quoted, author);
|
||||
if (quotePermission == QuotePermission.Denied)
|
||||
return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: This post cannot be quoted");
|
||||
}
|
||||
|
||||
var located = draft.Latitude.HasValue || draft.Longitude.HasValue;
|
||||
if (located && (draft.Latitude is not (>= -90 and <= 90) || draft.Longitude is not (>= -180 and <= 180) || group != default
|
||||
|| draft.Visibility == PostVisibility.Direct))
|
||||
@@ -163,7 +181,11 @@ namespace PrivaPub.Domain.Statuses
|
||||
InReplyToAccountId = parent?.AuthorAccountId ?? parent?.GroupUserId,
|
||||
IsLocalOnly = isLocalOnly,
|
||||
ActorURI = author.Uri,
|
||||
Poll = draft.Poll == default ? default : _polls.Create(draft.Poll)
|
||||
Poll = draft.Poll == default ? default : _polls.Create(draft.Poll),
|
||||
QuoteURI = quoted?.ObjectURI,
|
||||
QuotedPostId = quoted?.ID,
|
||||
QuoteByConsent = quoted != default && (quoted.QuotePolicy != default || !quoted.IsFederatedCopy),
|
||||
QuoteState = quoted == default ? QuoteState.None : quotePermission == QuotePermission.Granted ? QuoteState.Accepted : QuoteState.Pending
|
||||
};
|
||||
post.ID = (string)post.GenerateNewID();
|
||||
post.ObjectURI = author.PostUri(post.ID);
|
||||
@@ -207,6 +229,11 @@ namespace PrivaPub.Domain.Statuses
|
||||
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
|
||||
await _fanout.Distribute(post, token);
|
||||
await _polls.Scheduled(post, token);
|
||||
if (post.QuoteState == QuoteState.Accepted)
|
||||
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
|
||||
if (post.QuoteState == QuoteState.Pending && create?["object"] is JsonObject quotingNote)
|
||||
await _quotes.Request(author, post, quoted, quotingNote, token);
|
||||
await _previews.Wanted(post, token);
|
||||
if (create != default)
|
||||
await _outbox.Publish(author, post, create, token);
|
||||
if (create != default && group is { IsCircle: false } && visibility is PostVisibility.Public or PostVisibility.Unlisted)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Models.Social;
|
||||
using PrivaPub.Models.User;
|
||||
@@ -15,17 +16,21 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
{
|
||||
readonly DbEntities _dbEntities;
|
||||
readonly ILocalActorService _localActors;
|
||||
readonly IQuoteService _quotes;
|
||||
|
||||
public AcceptHandler(DbEntities dbEntities, ILocalActorService localActors)
|
||||
public AcceptHandler(DbEntities dbEntities, ILocalActorService localActors, IQuoteService quotes)
|
||||
{
|
||||
_dbEntities = dbEntities;
|
||||
_localActors = localActors;
|
||||
_quotes = quotes;
|
||||
}
|
||||
|
||||
public virtual string Type => "Accept";
|
||||
|
||||
public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token)
|
||||
{
|
||||
if (await _quotes.Answered(activity, actor, accepted: Type == "Accept", token))
|
||||
return;
|
||||
var following = await FindFollowing(activity["object"], actor, _dbEntities, _localActors, token);
|
||||
if (following == default)
|
||||
return;
|
||||
@@ -56,7 +61,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
|
||||
public class RejectHandler : AcceptHandler
|
||||
{
|
||||
public RejectHandler(DbEntities dbEntities, ILocalActorService localActors) : base(dbEntities, localActors)
|
||||
public RejectHandler(DbEntities dbEntities, ILocalActorService localActors, IQuoteService quotes) : base(dbEntities, localActors, quotes)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ using MongoDB.Entities;
|
||||
using PrivaPub.Domain.Timelines;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Moderation;
|
||||
using PrivaPub.Domain.Content;
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Federation.Outbox;
|
||||
@@ -37,10 +38,14 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
readonly IGroupDistributor _groups;
|
||||
readonly IObjectRecords _records;
|
||||
readonly IPollService _polls;
|
||||
readonly ILinkPreviews _previews;
|
||||
readonly IQuoteService _quotes;
|
||||
|
||||
public CreateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
|
||||
IDomainBlocks domainBlocks, IFanout fanout, IRemotePosts remotePosts, IGroupDistributor groups, IObjectRecords records, IPollService polls)
|
||||
IDomainBlocks domainBlocks, IFanout fanout, IRemotePosts remotePosts, IGroupDistributor groups, IObjectRecords records, IPollService polls, ILinkPreviews previews, IQuoteService quotes)
|
||||
{
|
||||
_quotes = quotes;
|
||||
_previews = previews;
|
||||
_records = records;
|
||||
_polls = polls;
|
||||
_groups = groups;
|
||||
@@ -148,6 +153,8 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
return;
|
||||
}
|
||||
await _records.Record(note, post, ObjectPath.Delivered, refetched, token);
|
||||
await _previews.Wanted(post, token);
|
||||
await _quotes.Resolve(post, note, token);
|
||||
|
||||
if (parent != default)
|
||||
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Federation.Outbox;
|
||||
@@ -27,10 +28,12 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
readonly IRemoteActorService _remoteActors;
|
||||
readonly IDeliveryService _delivery;
|
||||
readonly IGroupDistributor _groups;
|
||||
readonly IQuoteService _quotes;
|
||||
|
||||
public DeleteHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery,
|
||||
IGroupDistributor groups)
|
||||
IGroupDistributor groups, IQuoteService quotes)
|
||||
{
|
||||
_quotes = quotes;
|
||||
_groups = groups;
|
||||
_dbEntities = dbEntities;
|
||||
_localActors = localActors;
|
||||
@@ -72,7 +75,10 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
.ExecuteAsync(token);
|
||||
var post = await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI).ExecuteFirstAsync(token);
|
||||
if (post == default)
|
||||
{
|
||||
await _quotes.Revoke(objectUri, token);
|
||||
return;
|
||||
}
|
||||
await DB.Default.DeleteAsync<PostEntity>(post.ID);
|
||||
await DB.Default.DeleteAsync<ObjectRecord>(r => r.PostId == post.ID || r.ObjectURI == objectUri);
|
||||
await DB.Default.DeleteAsync<TimelineEntry>(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID);
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Domain.Statuses;
|
||||
using PrivaPub.Federation.Objects;
|
||||
using PrivaPub.Federation.Outbox;
|
||||
using PrivaPub.Models.Post;
|
||||
@@ -24,10 +25,12 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
readonly IRemoteActorService _remoteActors;
|
||||
readonly IGroupDistributor _groups;
|
||||
readonly IObjectRecords _records;
|
||||
readonly IQuoteService _quotes;
|
||||
|
||||
public UpdateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IGroupDistributor groups,
|
||||
IObjectRecords records)
|
||||
IObjectRecords records, IQuoteService quotes)
|
||||
{
|
||||
_quotes = quotes;
|
||||
_records = records;
|
||||
_groups = groups;
|
||||
_dbEntities = dbEntities;
|
||||
@@ -56,6 +59,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
return;
|
||||
|
||||
post.Poll = note.Poll ?? post.Poll;
|
||||
post.QuotePolicy = note.QuotePolicy;
|
||||
post.Video = note.Video ?? post.Video;
|
||||
post.Audio = note.Audio ?? post.Audio;
|
||||
post.Event = note.Event ?? post.Event;
|
||||
@@ -63,6 +67,7 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
{
|
||||
await DB.Default.SaveAsync(post, token);
|
||||
await _records.Revise(note, Id(activity), token);
|
||||
await Requote(post, note, token);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -102,11 +107,18 @@ namespace PrivaPub.Federation.Inbox.Handlers
|
||||
post.UpdateDate = DateTime.UtcNow;
|
||||
await DB.Default.SaveAsync(post, token);
|
||||
await _records.Revise(note, Id(activity), token);
|
||||
await Requote(post, note, token);
|
||||
if (!string.IsNullOrEmpty(post.GroupId) && await _localActors.FindById(Models.Federation.LocalActorKind.Group, post.GroupId, token) is { IsCircle: false } community)
|
||||
await _groups.Announce(community, activity.AsObject(), post.ObjectURI, isNewPost: false, token);
|
||||
}
|
||||
|
||||
static bool IsEdit(NoteDocument note, PostEntity post) =>
|
||||
note.Updated is { } updated && updated > (post.EditedAt ?? post.CreationDate);
|
||||
|
||||
async Task Requote(PostEntity post, NoteDocument note, CancellationToken token)
|
||||
{
|
||||
if (note.QuoteUri != post.QuoteURI || note.QuoteAuthorization != post.QuoteAuthorizationURI || post.QuoteState != QuoteState.Accepted)
|
||||
await _quotes.Resolve(post, note, token);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
using MongoDB.Driver;
|
||||
using MongoDB.Entities;
|
||||
|
||||
using PrivaPub.Domain.Content;
|
||||
using PrivaPub.Federation.Actors;
|
||||
using PrivaPub.Federation.Moderation;
|
||||
using PrivaPub.Federation.Objects;
|
||||
@@ -39,10 +40,12 @@ namespace PrivaPub.Federation.Inbox
|
||||
readonly IDomainBlocks _domainBlocks;
|
||||
readonly IJobQueue _queue;
|
||||
readonly IObjectRecords _records;
|
||||
readonly ILinkPreviews _previews;
|
||||
|
||||
public RemotePosts(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDomainBlocks domainBlocks,
|
||||
IJobQueue queue, IObjectRecords records)
|
||||
IJobQueue queue, IObjectRecords records, ILinkPreviews previews)
|
||||
{
|
||||
_previews = previews;
|
||||
_records = records;
|
||||
_dbEntities = dbEntities;
|
||||
_localActors = localActors;
|
||||
@@ -79,6 +82,7 @@ namespace PrivaPub.Federation.Inbox
|
||||
Excerpt = note.Excerpt,
|
||||
Source = note.Source,
|
||||
Poll = note.Poll,
|
||||
QuotePolicy = note.QuotePolicy,
|
||||
Emojis = note.Emojis.ToList(),
|
||||
CoverURL = note.CoverURL,
|
||||
Link = note.Link,
|
||||
@@ -143,6 +147,7 @@ namespace PrivaPub.Federation.Inbox
|
||||
return await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri).ExecuteFirstAsync(token);
|
||||
}
|
||||
await _records.Record(note, post, ObjectPath.Fetched, refetched: false, token);
|
||||
await _previews.Wanted(post, token);
|
||||
|
||||
if (grandparent == default && !string.IsNullOrEmpty(note.InReplyTo) && depth < MaxDepth)
|
||||
await _queue.Enqueue(JobKind.FetchAncestors, JsonSerializer.Serialize(new AncestorsPayload(post.ID, depth + 1)), new Uri(note.InReplyTo).Host,
|
||||
|
||||
@@ -38,7 +38,7 @@ namespace PrivaPub.Federation.Objects
|
||||
{
|
||||
KeepChildNodes = true
|
||||
};
|
||||
foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible" })
|
||||
foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible", "quote-inline" })
|
||||
sanitizer.AllowedClasses.Add(allowed);
|
||||
sanitizer.RemovingCssClass += (_, e) => e.Cancel = MicroformatClass().IsMatch(e.CssClass);
|
||||
sanitizer.RemovingTag += (_, e) =>
|
||||
|
||||
@@ -25,6 +25,9 @@ namespace PrivaPub.Federation.Objects
|
||||
public string Context { get; init; }
|
||||
public string Audience { get; init; }
|
||||
public string QuoteUri { get; init; }
|
||||
public string QuoteAuthorization { get; init; }
|
||||
public bool QuotesByConsent { get; init; }//FEP-044f `quote`, as opposed to the legacy keys that ask nobody
|
||||
public bool QuoteDeleted { get; init; }
|
||||
public DateTime Published { get; init; }
|
||||
public DateTime? Updated { get; init; }
|
||||
public IReadOnlyList<string> To { get; init; } = Array.Empty<string>();
|
||||
@@ -34,6 +37,7 @@ namespace PrivaPub.Federation.Objects
|
||||
public IReadOnlyList<PostMedia> Attachments { get; init; } = Array.Empty<PostMedia>();
|
||||
public PostSource Source { get; init; }
|
||||
public PostPoll Poll { get; init; }
|
||||
public InteractionRule QuotePolicy { get; init; }
|
||||
public IReadOnlyList<CustomEmoji> Emojis { get; init; } = Array.Empty<CustomEmoji>();
|
||||
public string CoverURL { get; init; }
|
||||
public PostLink Link { get; init; }
|
||||
@@ -79,7 +83,10 @@ namespace PrivaPub.Federation.Objects
|
||||
InReplyTo = Id(note["inReplyTo"]),
|
||||
Context = Id(note["context"]) ?? Value(note, "conversation"),
|
||||
Audience = Id(note["audience"]),
|
||||
QuoteUri = Value(note, "quote") ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote"),
|
||||
QuoteUri = Id(note["quote"]) ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote") ?? QuoteLink(note),
|
||||
QuoteAuthorization = Id(note["quoteAuthorization"]),
|
||||
QuotesByConsent = note.ContainsKey("quote"),
|
||||
QuoteDeleted = note["quote"] is JsonObject quoted && Value(quoted, "type") == "Tombstone",
|
||||
Published = Time(Value(note, "published")) ?? DateTime.UtcNow,
|
||||
Updated = Time(Value(note, "updated")),
|
||||
To = List(note["to"]),
|
||||
@@ -97,6 +104,7 @@ namespace PrivaPub.Federation.Objects
|
||||
Attachments = Attachments(note),
|
||||
Source = ObjectShapes.Source(note),
|
||||
Poll = ObjectShapes.Poll(note),
|
||||
QuotePolicy = ObjectShapes.QuotePolicy(note),
|
||||
Emojis = ObjectShapes.Emojis(note["tag"]),
|
||||
CoverURL = ObjectShapes.Cover(note),
|
||||
Link = ObjectShapes.Link(note),
|
||||
@@ -169,6 +177,12 @@ namespace PrivaPub.Federation.Objects
|
||||
};
|
||||
}
|
||||
|
||||
static string QuoteLink(JsonObject note) =>
|
||||
Tags(note, "Link")
|
||||
.Where(t => Value(t, "mediaType") is { } type && (type.StartsWith("application/activity+json") || type.StartsWith("application/ld+json")))
|
||||
.Select(t => Value(t, "href"))
|
||||
.FirstOrDefault(href => Origin.Of(href) != default);
|
||||
|
||||
static string FirstOf(JsonNode map) =>
|
||||
map is JsonObject languages ? languages.Select(l => l.Value is JsonValue v && v.TryGetValue<string>(out var text) ? text : default).FirstOrDefault(t => t != default) : default;
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ namespace PrivaPub.Federation.Objects
|
||||
const int CoverMaxWidth = 1280;
|
||||
const int MaxEmojis = 64;
|
||||
const int MaxPollOptions = 20;
|
||||
const int MaxAudience = 32;
|
||||
|
||||
static readonly Dictionary<string, string> MediaTypesByExtension = new(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
@@ -97,6 +98,23 @@ namespace PrivaPub.Federation.Objects
|
||||
.Take(MaxEmojis)
|
||||
.ToList();
|
||||
|
||||
public static InteractionRule QuotePolicy(JsonObject note)
|
||||
{
|
||||
if (note["interactionPolicy"] is not JsonObject policy || policy["canQuote"] is not JsonObject canQuote)
|
||||
return default;
|
||||
static List<string> Who(JsonNode node) => node switch
|
||||
{
|
||||
JsonArray array => array.Select(Id).Where(id => id != default).Take(MaxAudience).ToList(),
|
||||
JsonNode single when Id(single) is { } id => new List<string> { id },
|
||||
_ => new List<string>()
|
||||
};
|
||||
return new InteractionRule
|
||||
{
|
||||
Automatic = Who(canQuote["automaticApproval"] ?? canQuote["always"]),
|
||||
Manual = Who(canQuote["manualApproval"] ?? canQuote["approvalRequired"])
|
||||
};
|
||||
}
|
||||
|
||||
public static PostPoll Poll(JsonObject note)
|
||||
{
|
||||
if (Value(note, "type") != "Question")
|
||||
@@ -199,10 +217,13 @@ namespace PrivaPub.Federation.Objects
|
||||
if (Value(note, "type") != "Video")
|
||||
return default;
|
||||
var links = Objects(note["url"]).ToList();
|
||||
var nested = links.Where(l => Value(l, "mediaType")?.Equals("application/x-mpegURL", StringComparison.OrdinalIgnoreCase) == true)
|
||||
.SelectMany(l => Objects(l["tag"]))
|
||||
.ToList();
|
||||
return new VideoDetails
|
||||
{
|
||||
DurationSeconds = Seconds(note["duration"]),
|
||||
Variants = Variants(links, "video/", "audio/"),
|
||||
Variants = Variants(links.Concat(nested).ToList(), "video/", "audio/"),
|
||||
PlaylistURL = links.Where(l => Value(l, "mediaType")?.Equals("application/x-mpegURL", StringComparison.OrdinalIgnoreCase) == true)
|
||||
.Select(l => Value(l, "href")).FirstOrDefault(IsWeb),
|
||||
IsLive = Bool(note, "isLiveBroadcast") == true,
|
||||
|
||||
@@ -60,6 +60,16 @@ namespace PrivaPub.Federation.Outbox
|
||||
inboxes.Add(parentAuthor.InboxURL);
|
||||
}
|
||||
|
||||
if (post.Visibility is PostVisibility.Public or PostVisibility.Unlisted && !string.IsNullOrEmpty(post.QuotedPostId))
|
||||
{
|
||||
var quoted = await _dbEntities.Posts.MatchID(post.QuotedPostId).ExecuteFirstAsync(token);
|
||||
var quotedAuthor = quoted is { IsFederatedCopy: true }
|
||||
? await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == quoted.ActorURI).ExecuteFirstAsync(token)
|
||||
: default;
|
||||
if (!string.IsNullOrEmpty(quotedAuthor?.InboxURL))
|
||||
inboxes.Add(quotedAuthor.InboxURL);
|
||||
}
|
||||
|
||||
return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList();
|
||||
}
|
||||
|
||||
|
||||
@@ -39,6 +39,15 @@ namespace PrivaPub.Federation.Rendering
|
||||
["blurhash"] = "toot:blurhash",
|
||||
["votersCount"] = "toot:votersCount",
|
||||
["Emoji"] = "toot:Emoji",
|
||||
["quote"] = new JsonObject { ["@id"] = "https://w3id.org/fep/044f#quote", ["@type"] = "@id" },
|
||||
["quoteAuthorization"] = new JsonObject { ["@id"] = "https://w3id.org/fep/044f#quoteAuthorization", ["@type"] = "@id" },
|
||||
["QuoteRequest"] = "https://w3id.org/fep/044f#QuoteRequest",
|
||||
["QuoteAuthorization"] = "https://w3id.org/fep/044f#QuoteAuthorization",
|
||||
["misskey"] = "https://misskey-hub.net/ns#",
|
||||
["_misskey_quote"] = "misskey:_misskey_quote",
|
||||
["fedibird"] = "http://fedibird.com/ns#",
|
||||
["quoteUri"] = "fedibird:quoteUri",
|
||||
["quoteUrl"] = "as:quoteUrl",
|
||||
["litepub"] = "http://litepub.social/ns#",
|
||||
["EmojiReact"] = "litepub:EmojiReact",
|
||||
["focalPoint"] = new JsonObject { ["@container"] = "@list", ["@id"] = "toot:focalPoint" },
|
||||
@@ -160,6 +169,47 @@ namespace PrivaPub.Federation.Rendering
|
||||
return note;
|
||||
}
|
||||
|
||||
public static JsonObject UpdateOf(PostEntity post, LocalActor author, string reason)
|
||||
{
|
||||
var note = post.Visibility == PostVisibility.Direct
|
||||
? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI)
|
||||
: Note(post, author, default, post.InReplyToURI);
|
||||
note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray());
|
||||
note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray());
|
||||
return new JsonObject
|
||||
{
|
||||
["@context"] = Context(),
|
||||
["id"] = author.ActivityUri($"update-{post.ID}-{reason}"),
|
||||
["type"] = "Update",
|
||||
["actor"] = author.Uri,
|
||||
["to"] = note["to"]!.DeepClone(),
|
||||
["cc"] = note["cc"]!.DeepClone(),
|
||||
["object"] = note
|
||||
};
|
||||
}
|
||||
|
||||
static void AddQuote(JsonObject note, PostEntity post, ref string content)
|
||||
{
|
||||
if (string.IsNullOrEmpty(post.QuoteURI))
|
||||
return;
|
||||
var link = WebUtility.HtmlEncode(post.QuoteURI);
|
||||
content += $"<p class=\"quote-inline\">RE: <a href=\"{link}\">{link}</a></p>";
|
||||
if (post.QuoteByConsent)
|
||||
note["quote"] = post.QuoteURI;
|
||||
if (!string.IsNullOrEmpty(post.QuoteAuthorizationURI))
|
||||
note["quoteAuthorization"] = post.QuoteAuthorizationURI;
|
||||
note["_misskey_quote"] = post.QuoteURI;
|
||||
note["quoteUri"] = post.QuoteURI;
|
||||
note["quoteUrl"] = post.QuoteURI;
|
||||
(note["tag"] as JsonArray)?.Add(new JsonObject
|
||||
{
|
||||
["type"] = "Link",
|
||||
["mediaType"] = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"",
|
||||
["href"] = post.QuoteURI,
|
||||
["name"] = $"RE: {post.QuoteURI}"
|
||||
});
|
||||
}
|
||||
|
||||
static void AddPoll(JsonObject note, PostPoll poll)
|
||||
{
|
||||
note["type"] = "Question";
|
||||
@@ -211,6 +261,8 @@ namespace PrivaPub.Federation.Rendering
|
||||
}))
|
||||
.ToArray())
|
||||
};
|
||||
AddQuote(note, post, ref content);
|
||||
note["content"] = content;
|
||||
if (!string.IsNullOrEmpty(post.Language))
|
||||
note["contentMap"] = new JsonObject { [post.Language] = content };
|
||||
if (post.Poll is { } poll)
|
||||
|
||||
@@ -116,6 +116,7 @@ namespace PrivaPub.Infrastructure.Data
|
||||
await DB.Default.Index<Reaction>().Key(r => r.PostId, KeyType.Ascending).Key(r => r.ActorURI, KeyType.Ascending).Key(r => r.Emoji, KeyType.Ascending)
|
||||
.Option(o => o.Unique = true).CreateAsync(token);
|
||||
await Plain<Reaction>(token, r => r.ActivityURI);
|
||||
await Unique<Domain.Content.LinkPreview>(p => p.Url, Builders<Domain.Content.LinkPreview>.Filter.Type(p => p.Url, BsonType.String), token);
|
||||
}
|
||||
|
||||
static async Task Unique<T>(System.Linq.Expressions.Expression<Func<T, object>> key, FilterDefinition<T> partial,
|
||||
|
||||
@@ -22,6 +22,8 @@ namespace PrivaPub.Infrastructure.Http
|
||||
bool IsAllowed(Uri target);
|
||||
Task<FetchedJson> GetJson(string url, string accept, Action<HttpRequestMessage> sign, CancellationToken token);
|
||||
bool FailedTemporarily(string url);
|
||||
Task<(Uri FinalUri, string Html)> GetPage(string url, CancellationToken token);
|
||||
Task<HttpResponseMessage> OpenMedia(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token);
|
||||
Task<HttpResponseMessage> Send(HttpRequestMessage request, CancellationToken token);
|
||||
Task<(byte[] Bytes, string ContentType)> GetMedia(string url, long maxBytes, CancellationToken token);
|
||||
}
|
||||
@@ -181,6 +183,106 @@ namespace PrivaPub.Infrastructure.Http
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<(Uri FinalUri, string Html)> GetPage(string url, CancellationToken token)
|
||||
{
|
||||
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsAllowed(target))
|
||||
return default;
|
||||
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(token);
|
||||
timeout.CancelAfter(RequestTimeout);
|
||||
try
|
||||
{
|
||||
for (var hop = 0; hop <= MaxRedirects; hop++)
|
||||
{
|
||||
using var request = new HttpRequestMessage(HttpMethod.Get, target);
|
||||
request.Headers.Accept.ParseAdd("text/html, application/xhtml+xml");
|
||||
using var response = await _httpClientFactory.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, timeout.Token);
|
||||
if (IsRedirect(response.StatusCode))
|
||||
{
|
||||
var location = response.Headers.Location;
|
||||
var next = location == default ? default : location.IsAbsoluteUri ? location : new Uri(target, location);
|
||||
if (!IsAllowed(next))
|
||||
return default;
|
||||
target = next;
|
||||
continue;
|
||||
}
|
||||
var mediaType = response.Content.Headers.ContentType?.MediaType?.ToLowerInvariant();
|
||||
if (!response.IsSuccessStatusCode || mediaType is not ("text/html" or "application/xhtml+xml"))
|
||||
return default;
|
||||
var bytes = await ReadPrefix(response.Content, MaxPageBytes, timeout.Token);
|
||||
var charset = response.Content.Headers.ContentType?.CharSet?.Trim('"');
|
||||
var encoding = System.Text.Encoding.UTF8;
|
||||
try
|
||||
{
|
||||
if (!string.IsNullOrEmpty(charset))
|
||||
encoding = System.Text.Encoding.GetEncoding(charset);
|
||||
}
|
||||
catch (ArgumentException)
|
||||
{
|
||||
}
|
||||
return (target, encoding.GetString(bytes));
|
||||
}
|
||||
return default;
|
||||
}
|
||||
catch (Exception ex) when (ex is HttpRequestException or BlockedDestinationException or OperationCanceledException && !token.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogInformation("Page {Url} refused: {Reason}", url, ex.Message);
|
||||
return default;
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<HttpResponseMessage> OpenMedia(string url, System.Net.Http.Headers.RangeHeaderValue range, CancellationToken token)
|
||||
{
|
||||
if (!Uri.TryCreate(url, UriKind.Absolute, out var target) || !IsAllowed(target))
|
||||
return default;
|
||||
try
|
||||
{
|
||||
for (var hop = 0; hop <= MaxRedirects; hop++)
|
||||
{
|
||||
using var request = new HttpRequestMessage(HttpMethod.Get, target);
|
||||
request.Headers.Accept.ParseAdd("video/*, audio/*, image/*");
|
||||
request.Headers.Range = range;
|
||||
var response = await _httpClientFactory.CreateClient(ClientName).SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
|
||||
if (IsRedirect(response.StatusCode))
|
||||
{
|
||||
var location = response.Headers.Location;
|
||||
response.Dispose();
|
||||
var next = location == default ? default : location.IsAbsoluteUri ? location : new Uri(target, location);
|
||||
if (!IsAllowed(next))
|
||||
return default;
|
||||
target = next;
|
||||
continue;
|
||||
}
|
||||
var mediaType = response.Content.Headers.ContentType?.MediaType?.ToLowerInvariant();
|
||||
if (!response.IsSuccessStatusCode || mediaType == default || mediaType.Contains("svg")
|
||||
|| !(mediaType.StartsWith("video/") || mediaType.StartsWith("audio/") || mediaType.StartsWith("image/") || mediaType == "application/octet-stream"))
|
||||
{
|
||||
response.Dispose();
|
||||
return default;
|
||||
}
|
||||
return response;
|
||||
}
|
||||
return default;
|
||||
}
|
||||
catch (Exception ex) when (ex is HttpRequestException or BlockedDestinationException or OperationCanceledException && !token.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogInformation("Media stream {Url} refused: {Reason}", url, ex.Message);
|
||||
return default;
|
||||
}
|
||||
}
|
||||
|
||||
const int MaxPageBytes = 512 * 1024;
|
||||
|
||||
static async Task<byte[]> ReadPrefix(HttpContent content, int limit, CancellationToken token)
|
||||
{
|
||||
await using var stream = await content.ReadAsStreamAsync(token);
|
||||
var buffer = new byte[limit];
|
||||
var total = 0;
|
||||
int read;
|
||||
while (total < limit && (read = await stream.ReadAsync(buffer.AsMemory(total, limit - total), token)) > 0)
|
||||
total += read;
|
||||
return buffer[..total];
|
||||
}
|
||||
|
||||
public async Task<HttpResponseMessage> Send(HttpRequestMessage request, CancellationToken token)
|
||||
{
|
||||
if (!IsAllowed(request.RequestUri))
|
||||
|
||||
@@ -6,5 +6,6 @@ namespace PrivaPub.Infrastructure.Http
|
||||
public bool AllowPlainHttp { get; set; }
|
||||
public bool SecureMode { get; set; }
|
||||
public bool AcceptAnyCertificate { get; set; }
|
||||
public bool FetchLinkPreviews { get; set; } = true;//owner decision 1: the server reads linked pages of public posts
|
||||
}
|
||||
}
|
||||
@@ -79,6 +79,9 @@ namespace PrivaPub.Middleware
|
||||
.AddSingleton<IActivityHandler, DislikeHandler>()
|
||||
.AddSingleton<IActivityHandler, EmojiReactHandler>()
|
||||
.AddSingleton<IReactions, Reactions>()
|
||||
.AddSingleton<LinkPreviews>()
|
||||
.AddSingleton<ILinkPreviews>(services => services.GetRequiredService<LinkPreviews>())
|
||||
.AddSingleton<IJobHandler>(services => services.GetRequiredService<LinkPreviews>())
|
||||
.AddSingleton<IActivityHandler, JoinHandler>()
|
||||
.AddSingleton<IActivityHandler, AnnounceHandler>()
|
||||
.AddSingleton<IActivityHandler, FlagHandler>()
|
||||
@@ -89,6 +92,7 @@ namespace PrivaPub.Middleware
|
||||
.AddSingleton<IRemotePosts, RemotePosts>()
|
||||
.AddSingleton<IObjectRecords, ObjectRecords>()
|
||||
.AddSingleton<IPollService, PollService>()
|
||||
.AddSingleton<IQuoteService, QuoteService>()
|
||||
.AddSingleton<IJobHandler, PollRefreshJob>()
|
||||
.AddSingleton<IJobHandler, PollCloseJob>()
|
||||
.AddSingleton<IJobHandler, InstanceDescriber>()
|
||||
|
||||
@@ -25,7 +25,8 @@ namespace PrivaPub.Models.Jobs
|
||||
FetchAncestors,
|
||||
DescribeInstance,
|
||||
PollRefresh,
|
||||
PollClose
|
||||
PollClose,
|
||||
FetchPreview
|
||||
}
|
||||
|
||||
public enum JobState
|
||||
|
||||
@@ -57,6 +57,13 @@ namespace PrivaPub.Models.Post
|
||||
public string Url { get; set; }
|
||||
public string ContextURI { get; set; }
|
||||
public string QuoteURI { get; set; }
|
||||
public string QuotedPostId { get; set; }//our copy of the quoted post, once fetched
|
||||
public QuoteState QuoteState { get; set; }
|
||||
public string QuoteAuthorizationURI { get; set; }//FEP-044f: the quoted author's stamp
|
||||
public bool QuoteByConsent { get; set; }//our quote of a post whose server asks for consent (FEP-044f), so `quote` is sent
|
||||
public int QuotesCount { get; set; }
|
||||
[BsonIgnoreIfNull]
|
||||
public InteractionRule QuotePolicy { get; set; }//a remote post's interactionPolicy.canQuote; null when it states none
|
||||
public List<string> To { get; set; } = new();
|
||||
public List<string> Cc { get; set; } = new();
|
||||
|
||||
@@ -73,6 +80,17 @@ namespace PrivaPub.Models.Post
|
||||
public string UpdateReason { get; set; }
|
||||
}
|
||||
|
||||
public enum QuoteState
|
||||
{
|
||||
None,
|
||||
Pending,
|
||||
Accepted,
|
||||
Rejected,
|
||||
Revoked,
|
||||
Deleted,
|
||||
Unauthorized
|
||||
}
|
||||
|
||||
public enum PostVisibility
|
||||
{
|
||||
Public,
|
||||
|
||||
@@ -23,6 +23,12 @@ namespace PrivaPub.Models.Post
|
||||
public int Votes { get; set; }
|
||||
}
|
||||
|
||||
public class InteractionRule
|
||||
{
|
||||
public List<string> Automatic { get; set; } = new();//actor or collection URIs; Public for anyone
|
||||
public List<string> Manual { get; set; } = new();
|
||||
}
|
||||
|
||||
public class PostSource
|
||||
{
|
||||
public string Content { get; set; }
|
||||
|
||||
@@ -25,6 +25,7 @@ namespace PrivaPub.Models.Social
|
||||
Reblog,
|
||||
Update,
|
||||
Poll,
|
||||
Reaction
|
||||
Reaction,
|
||||
Quote
|
||||
}
|
||||
}
|
||||
@@ -48,6 +48,18 @@ server {
|
||||
proxy_read_timeout 300s;
|
||||
}
|
||||
|
||||
location ^~ /media/proxy/ {
|
||||
proxy_buffering off;
|
||||
proxy_pass http://127.0.0.1:6970;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Connection "";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_read_timeout 600s;
|
||||
}
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:6970;
|
||||
proxy_http_version 1.1;
|
||||
|
||||
+15
-9
@@ -453,19 +453,25 @@ it, raw where it doesn't.
|
||||
- **Polls** in and out, with the Misskey, Pleroma and PieFed vote shapes. A count refresh is never an edit (done in
|
||||
v1.10.0; checked live both ways against GoToSocial).
|
||||
- **Quotes (FEP-044f):**
|
||||
- read every key; verify `QuoteAuthorization` on all of its fields; handle revocation;
|
||||
- be quotable: `canQuote`, answer `QuoteRequest`, serve and revoke stamps;
|
||||
- then advertise `api_versions.mastodon ≥ 7` (4.5.0).
|
||||
- read every key; verify `QuoteAuthorization` on all of its fields; handle revocation (done in v1.13.0);
|
||||
- personas quote others: `quoted_status_id`, `QuoteRequest`, `Accept{result}` verified, `Update` with
|
||||
`quoteAuthorization`; legacy quotes for posts that state no policy (done in v1.13.0); `api_versions.mastodon = 7`;
|
||||
- still open: be quotable (`canQuote` on our posts, answer `QuoteRequest`, serve and revoke stamps). It needs a themed
|
||||
route name for stamps, agreed with the owner, and a default quote policy.
|
||||
- **Emoji reactions** in all three inbound forms, plus outbound `EmojiReact`, exposed as `emoji_reactions` (done in
|
||||
v1.11.0; Pleroma's `/api/v1/pleroma/statuses/:id/reactions` endpoints and `pleroma:emoji_reaction` notifications).
|
||||
- **Link cards:**
|
||||
- **Link cards** (done in v1.12.0):
|
||||
- from the object, or from FEP-8967 `preview`, without fetching;
|
||||
- fetching the page itself is the owner's decision (INTEROP §6.1).
|
||||
- **Media:**
|
||||
- video playback through the proxy (Range requests, HLS playlist rewriting, the poster), with a `video` card;
|
||||
- otherwise the server reads the page (owner decision 1): public posts only, 0–60 s after arrival, cached per address
|
||||
for 7 days across the whole server, `Federation:FetchLinkPreviews` to switch it off. Checked live against a
|
||||
GoToSocial profile page.
|
||||
- **Media** (done in v1.14.0 except where noted):
|
||||
- video playback through the proxy: range requests streamed from the origin, the poster, a playable attachment picked
|
||||
from MP4 or fragmented MP4 files (HLS playlists are not rewritten; not needed while PeerTube publishes the files);
|
||||
- audio attachments;
|
||||
- an article reader view;
|
||||
- JPEG/PNG renditions kept for Pixelfed.
|
||||
- JPEG/PNG for Pixelfed: uploads were already re-encoded to JPEG, PNG or GIF;
|
||||
- an article reader view: the data is in `Status.privapub` (title, excerpt, cover, full HTML in `content`); the view
|
||||
itself belongs to the client.
|
||||
|
||||
#### P7 Threads, communities and the social graph
|
||||
- **Thread backfill:**
|
||||
|
||||
@@ -114,6 +114,15 @@ gts_poll_on_pp=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['
|
||||
curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/polls/$gts_poll_on_pp/votes" -d 'choices[]=0'
|
||||
until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/statuses/$gts_poll" | j "print(d[\"poll\"][\"options\"][0][\"votes_count\"])")" = "1" ]' && ok "alice's vote counts on GoToSocial" || ko "vote not counted on GoToSocial"
|
||||
|
||||
echo "quotes"
|
||||
quote_target=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'red or blue' in s['content']))")
|
||||
[ "$(curl -s -o /dev/null -w '%{http_code}' -X POST -H "$PH" $P/api/v1/statuses -d "status=quoting"ed_status_id=$quote_target")" = "422" ] \
|
||||
&& ok "GoToSocial's author-only quote policy is respected" || ko "quoted a post whose author forbids it"
|
||||
|
||||
echo "link previews"
|
||||
linked=$(curl -s -X POST -H "$PH" $P/api/v1/statuses --data-urlencode 'status=have a look https://gts.test/@gtsuser' -d 'visibility=public' | j "print(d['id'])")
|
||||
until_true 45 '[ -n "$(curl -s -H "$PH" "$P/api/v1/statuses/$linked" | j "print((d[\"card\"] or {}).get(\"title\") or \"\")")" ]' && ok "the server builds a card for a linked page" || ko "no card for the linked page"
|
||||
|
||||
echo "edits and deletes"
|
||||
curl -s -o /dev/null -X PUT -H "$PH" $P/api/v1/statuses/$pp_post -d 'status=Hello from PrivaPub, edited'
|
||||
until_true 20 'gcurl -s -H "$GH" "$G/api/v1/statuses/$pp_on_gts" | grep -q "edited"' && ok "alice's edit reaches GoToSocial" || ko "edit not applied"
|
||||
|
||||
Reference in new issue
Block a user