R8 the Mastodon API made real: featured tags, endorsements, notes, notify and languages, suggestions, the direct stream

Featured tags per persona (/api/v1/featured_tags, suggestions, tags/:name/feature, accounts' featured tags, max 10),
listed at /tattoos as Hashtags and sent as Add/Remove{Hashtag} to followers on the featured collection, as Mastodon
sends them; remote accounts' tags received and read from featuredTags. Endorsements (pin/endorse) and private notes
(PersonaNote) kept per persona and never federated; the relationship's notifying, languages, endorsed and note, with
follow's notify (status notifications) and languages (home filter). Reading preferences set on /clientapi. Suggestions
v1/v2 with dismissals and staff picks, never the persona or a sibling. conversation events on the direct stream.
translate, annual reports and the admin API answer. New collections backed up and removed with their persona.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WpVCoDeM1qUTRp4fXQdfof
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-18 08:38:11 +02:00
1 parent d08fbabaa5
commit 87529bca49
42 files changed
+1505 -77

No files matched your search

+99 -6
View File
@@ -31,8 +31,10 @@ Its defining idea: **one private login owns several public personas.**
Privacy features in the model:
- location-ranged posts (`Post.Location`, `RangeKm`), local-only and never federated;
- contacts a persona can share (`Avatar.SharedPersonalContacts`) and private notes (`RootUserNote`, `PersonalNote`)
exist only as model classes, with no endpoint.
- contacts a persona can share (`Avatar.SharedPersonalContacts`) and the root's notes (`RootUserNote`, `PersonalNote`)
exist only as model classes, with no endpoint. A persona's private note on an account is `PersonaNote` (R8), its own:
`RootUserNote` is keyed by the root and would show a note made as one persona to its siblings, so it is never used for
it.
The intended client is the Mastodon client API (Tusky, Elk, Phanpy, Ivory), where each avatar logs in as its own
account. The private `/clientapi` covers what Mastodon can't express. `thepra/decePubClient`
@@ -57,7 +59,7 @@ actor-scoped route gets a name in the same spirit, agreed with the owner, and a
| DM context | `/peasants/{name}/whispers/{id}` |
| Quote permission (FEP-044f `QuoteAuthorization`) | `/peasants/{name}/parrot-licences/{id}` |
| Featured (pinned posts) | `/peasants/{name}/trophies` |
| Featured tags | `/peasants/{name}/tattoos` |
| Featured tags (an `OrderedCollection` of `Hashtag`s, R8) | `/peasants/{name}/tattoos` |
| Wall (FEP-400e) | `/peasants/{name}/graffiti` |
| Followers synchronisation (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` |
| A group's members and moderators | `/peasants/{name}/flock`, `/peasants/{name}/wardens` |
@@ -127,7 +129,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Instance/ InstanceSettings (the server's own settings in memory), MediaLimitsFromSettings
Moderation/ ModerationLog (the staff's log, AdminAction), AccountModeration (silence, suspend, posts
removed), Restrictions (who is held back, in memory), ModerationFlags (posts' flags again)
Social/ FollowService (local in-process, remote Follow/Accept), Notifications, Announcements
Social/ FollowService (local in-process, remote Follow/Accept), Notifications, Announcements,
FeaturedTags (a persona's featured hashtags, sent and received), Suggestions (who to follow)
Timelines/ Fanout (TimelineEntry rows, Mastodon's home rules), TimelineService
Privacy/ VisibilityPolicy (IsPublic expression, CanSee)
Relationships/ RelationshipService (blocks, mutes, account domain blocks; Hidden), ReportService
@@ -143,7 +146,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch
Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, lists, filters,
media, polls, reactions, reports, scheduled statuses, streaming, tags, trends, cdns,
provenance, custom emojis; the stubs are `StubsController` in `SearchController.cs`
provenance, custom emojis, featured tags, suggestions; the stubs are `StubsController`
in `SearchController.cs`
Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex);
OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent)
Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, RootRemoval,
@@ -350,6 +354,81 @@ client's role route never does.
here and elsewhere, and the posts taken down, before it does; indexes on `SilencedAt` and `SuspendedAt` of both account
collections and a partial one on `Post.ModeratedAt`. No new collection, so backups hold it all as they did.
## The Mastodon API made real (completion program R8)
- **Every row is one persona's** (`Models/Social/Featuring.cs`, new collections): `FeaturedTag`, `Endorsement`,
`PersonaNote` and `SuggestionDismissal` are keyed by the persona (`AvatarId`), never by its root, with unique indexes on
(persona, name or account); `RemoteFeaturedTag` is an account elsewhere's (by `ActorURI`). `Following` gained `Notify`
and `Languages` (null: every language), `ForeignAvatar` `FeaturedTagsURL`, `AvatarSettings` `ExpandMedia`
(`default`, `show_all`, `hide_all`) and `ExpandSpoilers`, `NotificationType` `Status` (last, as enum members are stored
as ints).
- **Featured tags** (`Domain/Social/FeaturedTags.cs`, `FeaturedTagsController`, `TagsController`): `GET`/`POST
/api/v1/featured_tags` (`name`; 422 for a name that is no hashtag or past `FeaturedTags.MaxFeatured`, 10, which both
instance APIs advertise as `max_featured_tags`; one featured already is answered as it is), `DELETE
/api/v1/featured_tags/{id}` (`{}`, 404), `GET /api/v1/featured_tags/suggestions` (Mastodon's `Tag`s: the tags of the
persona's last 200 posts it does not feature, the most used first, ten), `POST /api/v1/tags/{name}/feature|unfeature`
(the `Tag`, with Mastodon 4.4's `featuring`, which every `Tag` now carries for a signed-in persona), `GET
/api/v1/accounts/{id}/featured_tags` (anonymous too: a persona's own, an account elsewhere's from `RemoteFeaturedTag`, a
community none, a suspended account 404). Mastodon's `FeaturedTag`: `statuses_count` a string and `last_status_at` a
day, counted from the account's public and unlisted posts held here with the tag. They are public, so they federate:
`/tattoos` lists them as `Hashtag`s, and featuring one sends `Add{Hashtag}`, unfeaturing `Remove{Hashtag}`, to the
persona's followers, **with its `featured` collection (`/trophies`) as the target**, as Mastodon sends them
(`AddHashtagSerializer`) and the only target Mastodon takes a Hashtag from. Received: `FeaturedHandler` takes a `Hashtag`
added to or removed from an account's `featured` or `featuredTags` (an object with no id, read before anything takes its
`href` for a post), and `AccountCountsJob` reads the `featuredTags` collection (its own server only, three pages, 20 tags
at most) with the account's counts.
- **Endorsements** (`AccountsController`): `POST /api/v1/accounts/{id}/pin|unpin` (Mastodon's old names) and
`/endorse|/unendorse` (4.4) answer the relationship; only an account the persona follows (422 otherwise, as Mastodon
says it), 500 at most. `GET /api/v1/endorsements` lists the persona's own, newest first, those it still follows (paged
by `Link`); `GET /api/v1/accounts/{id}/endorsements` answers them to the persona itself and `[]` for every other account:
**endorsements stay private** (the owner's default), never federated as featured accounts. Unfollowing drops the
endorsement, as on Mastodon; a `Move` carries the endorsement and the note over to the new account.
- **Private notes:** `POST /api/v1/accounts/{id}/note` (`comment`, 2000 characters at most, 422 beyond; empty takes it
away) answers the relationship, whose `note` is the persona's own `PersonaNote`. Never federated.
- **The relationship** fills `notifying` and `languages` from the follow, `endorsed` (only while it follows) and `note`.
`POST /api/v1/accounts/{id}/follow` takes `notify` and `languages[]` (each changed only when given, as on Mastodon; an
empty list is every language). `Fanout` keeps a post in another language out of that follower's home (a post whose
language is unknown comes in; languages compare by their primary subtag) and gives a follower with `notify` a `status`
notification for the account's own posts and its replies to itself, never its boosts, once the post reached that home
(Mastodon's `FeedInsertWorker`). Archives carry both in `following_accounts.csv`'s own columns.
- **Preferences:** `GET /api/v1/preferences` answers `reading:expand:media` and `reading:expand:spoilers` from the
persona's settings, which Mastodon's API sets nowhere, so `GET`/`POST /clientapi/avatar/private/preferences`
(`ViewPersonaPreferences`, `PersonaPreferencesForm`, the login's own persona; 404 for any other, 422 for another mode)
sets them.
- **Suggestions** (`Domain/Social/Suggestions.cs`, `SuggestionsController`): `GET /api/v1/suggestions` (accounts) and
`/api/v2/suggestions` (`source`, `sources`, `account`), `limit` 40 (80 at most); `DELETE /api/v1/suggestions/{account_id}`
(`{}`) dismisses one for that persona for good. In order: the staff's picks (`ServerSettings.StaffPicks`, `staff` /
`featured`), past interactions (accounts the persona favourited, boosted or answered lately, `past_interactions`, which
4.3's `sources` has no word for), then global: accounts followed by at least three personas here (`most_followed`) and
personas here whose public posts of the last 30 days were favourited and boosted most (`most_interactions`),
discoverable ones only, worked out at most every ten minutes. **Never the persona itself and never a sibling persona of
its login** (`RootToAvatar` read server-side only: suggesting a sibling would pair them the moment the persona followed
it), nor what it follows, asked to follow, blocked, muted, was blocked by (here or elsewhere), dismissed or hides by
server, nor an account held back (`Counted.HeldBackPersonas`), suspended, silenced or shown `limited`; a community only
as a staff pick.
- **Staff picks** (`InstanceAdminController`, administrators, checked against the database): `GET
/clientapi/admin/staff-picks`, `POST /staff-picks/insert` (`StaffPickForm`: a persona here or a person elsewhere by its
Mastodon id; 404 for no such account, 409 for one picked already, 422 past 40), `POST /staff-picks/delete?accountId=`;
each answers `List<ViewStaffPick>`, which `ViewServerSettings.StaffPicks` shows too (`IsShown` false for one deleted,
suspended or silenced since). They are kept in the settings but never changed by the settings form, so an older page
saving the form keeps them. Logged as `staff_pick.insert` and `staff_pick.delete` (target `persona` or `account`, so an
entry about a persona of the acting login keeps no actor).
- **familiar_followers stays empty by design:** which accounts the persona follows also follow another would tell who
follows whom among the personas here; every account answers `accounts: []`.
- **The direct stream:** `StreamingController` takes `stream=direct` (and the WebSocket's `subscribe` to `direct`); a
direct message saved here or arriving (`StatusService`, `CreateHandler`, after `ConversationStates.Posted`) tells the
`direct` streams of the conversation's personas a `conversation` event (`Fanout.Conversation`), mapped per persona as
`/api/v1/conversations` maps it.
- **Stubs that answer:** `POST /api/v1/statuses/{id}/translate` 422 (translation is cut), `GET /api/v1/annual_reports`
`{"annual_reports":[],"accounts":[],"statuses":[]}`, every verb under `/api/v1/admin/**` (and `GET`/`POST` under
`/api/v2/admin/**`) 403 `{"error":"This action is not allowed"}`, signed in or not (moderation stays on `/clientapi`).
- **Removal, backups, archives:** `PersonaRemoval` deletes the persona's featured tags, endorsements, notes and
dismissals, and what other personas kept about it (their endorsements of it, notes on it, dismissals of it); the Delete
of the actor tells other servers. Every new collection is in every backup (none is in `ServerBackup.Excluded`: they are
personas' own data, the remote featured tags a cache like `RemoteFeatured`); a restore keeps them as the backup has them,
and `ProtectiveMerge` deletes those of a persona deleted since. Mastodon's account archive has no featured tags, notes or
endorsements, so the persona archive leaves them out.
## Custom emoji (completion program R7)
- **The server's own emoji** (`Models/Post/LocalEmoji.cs`, a collection of its own): a shortcode (`[a-zA-Z0-9_]{2,64}`,
@@ -541,6 +620,10 @@ client's role route never does.
(`ObjectShapes.Flairs`, both dialects). Every media URL in it goes
through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following
one is the reader's choice.
8. **What a persona keeps through the API is that persona's** (R8): featured tags (public, federated), endorsements and
private notes (shown to it alone, never federated), dismissed suggestions. Suggestions never offer the persona itself
or a sibling persona of its login, and `familiar_followers` answers no one for every account, since it would tell who
follows whom here.
## Media invariants
@@ -673,7 +756,9 @@ client's role route never does.
- **An actor's `published` is `PublishedOn`**, a random whole day up to two weeks before creation, so personas made the
same day do not share a date. New persona and group ids carry that same day (`GenerateNewID`), because local clients
see ids. Never emit `CreatedAt`.
- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows. Nothing may relate sibling avatars.
- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows, and since R8 featured tags,
endorsements, private notes (`PersonaNote`, never `RootUserNote`) and dismissed suggestions. Nothing may relate sibling
avatars: suggestions never offer a sibling, and familiar followers stay empty.
- **Blocks federate** (owner decision, 2026-10-01): a block is sent as `Block` from the blocking avatar, an unblock as
`Undo{Block}`. Reports still leave as `Flag` from the server's own actors (the instance actor, or the reporter for
communities on Lemmy, PieFed and Mbin), never from the reporting avatar.
@@ -1061,6 +1146,14 @@ tools/pasture/run.sh down # removes e
in the status's `emojis` on Mastodon, GoToSocial (by URI), Misskey (the note's `emojis`) and Akkoma; her display name
shows it on Mastodon; her `EmojiReact` with it reaches Misskey and Akkoma as that emoji; deleted, it leaves the list,
its id answers 410 and posts here carry it no more; the staff's log keeps the insert and the delete.
- **Featured tags and the API made real (`scenarios/featured.sh`, needs mastodon; after emoji in batch A):** mastouser
follows feat_alice, who features a tag: Mastodon's `featured_tags` of her account shows it (her `Add{Hashtag}`, the
`/tattoos` collection serving it), unfeaturing takes it away there; mastouser features a tag, which feat_alice reads
(following him) through `/api/v1/accounts/:id/featured_tags`; following mastouser with `notify=true` gives feat_alice a
`status` notification for his next post; her SSE and WebSocket `direct` streams tell a `conversation` event for his
DM; her endorsement of him and her note on him are hers alone (feat_bob, a sibling persona of the same login, sees
neither, and `GET /api/v1/accounts/:id/endorsements` answers nobody else), and feat_bob is never suggested to her; the
stubs (translate 422, annual reports, the admin API 403) answer.
- **Mailpit (1.31.4, `peers/mailpit.sh`, as `mailpit.test`):** the pasture's mail server. `appsettings.Pasture.json`
sends PrivaPub's mail to `pasture-mailpit:1025` (plain SMTP, any login accepted) and its recovery links to
`https://decepub.test`. `scenarios/accounts.sh` reads what arrived through Mailpit's API: an address set with the
+19 -4
View File
@@ -132,6 +132,7 @@ actor document, a collection, NodeInfo or a delivery relates two avatars of the
| Direct-message context | `/peasants/{name}/whispers/{id}` |
| A persona's wall (FEP-400e) | `/peasants/{name}/graffiti` |
| Followers on the asking server (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` |
| Featured (pinned) posts, featured hashtags | `/peasants/{name}/trophies`, `/peasants/{name}/tattoos` |
| A custom emoji of this server (`Emoji`) | `/emojis/{shortcode}` |
| Profile and post pages | `/@{name}`, `/@{name}/{id}` |
@@ -216,12 +217,13 @@ Received:
| `Delete` | deletes the object, or the actor and its follows; a deleted object id is remembered for 90 days, so a late `Create` cannot bring it back, though a new post its server publishes under that id after the deletion (Gancio numbers events from the last one kept) is kept |
| `Flag` | becomes a report for this server's moderators |
| `Move` | an account moving: believed as Mastodon believes it, when the account sends it about itself and the new account, read again from its server, names it in `alsoKnownAs`. The old account then shows where it went (`moved`), and, as Mastodon does it (owner decision 2026-10-05), the personas following it follow the new one instead (a Follow to its server, an Undo to the old), in the same lists; a mute or a block of the old account carries over |
| `Add`/`Remove` of a `Hashtag` on the actor's `featured` or `featuredTags` | the account features a hashtag on its profile, or stops (Mastodon sends them to `featured`); shown as its featured tags (`GET /api/v1/accounts/:id/featured_tags`). Its `featuredTags` collection itself is read with its counts, at most once a day (20 tags at most) |
| `Add`/`Remove` on the actor's `featured` | the account pins or unpins one of its own posts (fetched from its server when not held); inside a community's announce, the community features a post made in it. Its profile shows them first (`pinned=true`). The collection itself is read with the account's counts, at most once a day. On the account's wall (`sm:wall`), see "Walls". Any other target (a community's moderators) is dropped; one on the account's own server that PrivaPub does not know has its document read again first, at most hourly |
| `Block` of a persona | the follows between them end, the blocker's posts and notifications are hidden from the persona, nothing of the persona's is addressed to the blocker, and the relationship says `blocked_by`; `Undo{Block}` lifts it |
Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Create{Question}` and poll votes, `EmojiReact` and its `Undo`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`,
`Reject{Follow}`, `Like`, `Dislike`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event, and the
replies to a persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`.
`Reject{Follow}`, `Like`, `Dislike`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event,
`Add`/`Remove` of a pinned post or a featured `Hashtag`, and the replies to a persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`.
- **Votes.** A favourite is a `Like`, a downvote a `Dislike`; a persona has one vote a post, and a changed vote is sent
as the new vote alone, as Lemmy sends one (an `Undo` of the old one beside it could arrive after it and take the new
@@ -256,8 +258,21 @@ replies to a persona's posts passed on to its followers. A deleted post answers
post on its wall. The wall lists the persona's public posts that start a thread and what was written on it, newest
first. An account elsewhere that tells its followers here, with `Add{Note}` on its own wall, of a post written there
has it shown to them as on its wall (`privapub.wall` on the status); a `Remove` there takes it away.
- **Pinned posts** are the actor's `featured` collection (`/trophies`); `featuredTags` is `/tattoos`. A pin or an unpin
is told to the post's audience as `Add` or `Remove` on `featured`, as Mastodon tells it.
- **Pinned posts** are the actor's `featured` collection (`/trophies`). A pin or an unpin is told to the post's audience
as `Add` or `Remove` on `featured`, as Mastodon tells it.
- **Featured hashtags** are the actor's `featuredTags` collection (`/tattoos`): an `OrderedCollection` of `Hashtag`s, in
the order the persona featured them (ten at most), each `{"type": "Hashtag", "href": "<this server>/tags/<tag>", "name":
"#Tag"}`. Featuring one sends its followers
```json
{"id": "…/peasants/alice/grunts/feature-tag-…", "type": "Add", "actor": "…/peasants/alice",
"object": {"type": "Hashtag", "href": "https://privapub.thepra.dev/tags/gardening", "name": "#Gardening"},
"target": "…/peasants/alice/trophies", "to": ["…#Public"], "cc": ["…/peasants/alice/groupies"]}
```
and unfeaturing it the same `Remove`. The target is the `featured` collection, not `featuredTags`: that is what Mastodon
sends and the only target it takes a Hashtag from (it tells a pin from a tag by the object's type).
- **What is never sent:** a persona's endorsements of other accounts (Mastodon's account pins and 4.4's endorsements) stay
private, never published as featured accounts or a collection; its private notes on accounts and its dismissed follow
suggestions are its own. No collection lists who follows whom (familiar followers are never shown either).
- **Blocks are sent.** A blocked remote account receives `Block` from the blocking account (and `Reject{Follow}` if it
followed); an unblock sends `Undo{Block}`. A server's block of a whole server sends that server nothing; the
administrator may list it at `/api/v1/instance/domain_blocks` (off for each block by default, its name partly hidden
@@ -25,6 +25,7 @@ namespace PrivaPub.ClientModels.Admin
public DateTime? TermsEffectiveAt { get; set; }
public List<ViewServerRule> Rules { get; set; } = [];//deleted ones too (IsDeleted), as old reports name them
public List<ViewRelay> Relays { get; set; } = [];
public List<ViewStaffPick> StaffPicks { get; set; } = [];//the accounts suggested first to every persona (R8), their own routes
public ViewMediaLimits Media { get; set; } = new();
public string RegistrationMode { get; set; }//"Invitations" or "Open", from the configuration
public bool PublishActivity { get; set; } = false;
@@ -158,6 +159,26 @@ namespace PrivaPub.ClientModels.Admin
public DateTime? AnsweredAt { get; set; }
}
// an account the administrators suggest to every persona here, first (R8: Mastodon's staff picks, `featured` among a
// suggestion's sources): a persona of this server or an account elsewhere, by its Mastodon account id. Added and removed
// on their own routes (/clientapi/admin/staff-picks), never through the settings form, so an older page saving the form
// never drops them.
public class ViewStaffPick
{
public string AccountId { get; set; }
public bool IsLocal { get; set; } = false;
public string Handle { get; set; }//user@host
public string Name { get; set; }
public bool IsShown { get; set; } = true;//false: deleted, suspended or silenced since, so suggested to nobody
}
public class StaffPickForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(64, ErrorMessageResourceName = "StringLength", ErrorMessageResourceType = typeof(ErrorsResource))]
public string AccountId { get; set; }
}
public class RelayForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
@@ -0,0 +1,29 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.User.Avatar
{
// How a persona's clients show what others post (R8): Mastodon's reading:expand:media ("default" hides media marked
// sensitive, "show_all", "hide_all") and reading:expand:spoilers, which GET /api/v1/preferences answers. Mastodon's API
// sets neither, so they are set here: GET and POST /clientapi/avatar/private/preferences, for the login's own persona.
public class ViewPersonaPreferences
{
public string AvatarId { get; set; }
public string ExpandMedia { get; set; }
public bool ExpandSpoilers { get; set; } = false;
}
public class PersonaPreferencesForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
Display(Name = nameof(AvatarId), ResourceType = typeof(FieldsNameResource))]
public string AvatarId { get; set; }
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
StringLength(16, ErrorMessageResourceName = "StringLength", ErrorMessageResourceType = typeof(ErrorsResource))]
public string ExpandMedia { get; set; }//"default", "show_all" or "hide_all"
public bool ExpandSpoilers { get; set; } = false;
}
}
@@ -304,7 +304,10 @@ namespace PrivaPub.Api.Mastodon.Controllers
await _feeds.Subscribe(Me, remote, true, token);
return Json(await Relationship(id, token));
}
var following = await _follows.FollowAs(Me, local?.Uri ?? remote.ActorURI, Params.Bool("reblogs") != false, token);
// notify: a `status` notification for each of its new posts; languages: only its posts in those reach the home (R8).
// Either changes only when given, as on Mastodon
var following = await _follows.FollowAs(Me, local?.Uri ?? remote.ActorURI, Params.Bool("reblogs") != false, token, Params.Bool("notify"),
Params.Has("languages") ? Params.List("languages") : default);
return following == default ? Error(StatusCodes.Status403Forbidden, "This action is not allowed") : Json(await Relationship(id, token));
}
@@ -425,12 +428,112 @@ namespace PrivaPub.Api.Mastodon.Controllers
public async Task<IActionResult> Reject(string id, CancellationToken token) =>
await _follows.Decide(Me, id, accept: false, token) ? Json(await Relationship(id, token)) : NotFoundError();
[HttpGet("/api/v1/accounts/{id}/featured_tags"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult FeaturedTags(string id) => Json(Array.Empty<object>());
// Familiar followers stay empty by design: which of the accounts the persona follows also follow another account would
// tell who follows whom among the personas here (and their followers lists are never shown, hide_collections), so
// every account answers none
[HttpGet("/api/v1/accounts/familiar_followers"), Scope("read:follows")]
public IActionResult FamiliarFollowers() => Json(Params.List("id").Select(id => new { id, accounts = Array.Empty<Account>() }).ToList());
// Endorsements (R8): Mastodon's account pins under their old names (pin, unpin) and 4.4's (endorse, unendorse). Only an
// account the persona follows may be endorsed, as Mastodon requires; they are the persona's alone, never federated as
// featured accounts and never listed to anyone else (the owner's default: endorsements stay private)
[HttpPost("/api/v1/accounts/{id}/pin"), HttpPost("/api/v1/accounts/{id}/endorse"), Scope("write:accounts")]
public async Task<IActionResult> Endorse(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default || local?.Id == MyId || local is { Suspended: true } || remote is { SuspendedAt: not null })
return NotFoundError();
var uri = local?.Uri ?? remote.ActorURI;
if (!await Follows(uri, token))
return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: You must be already following the person you want to endorse");
if (!await DB.Default.Find<Endorsement>().Match(e => e.AvatarId == MyId && e.TargetActorURI == uri).ExecuteAnyAsync(token))
{
if (await DB.Default.CountAsync<Endorsement>(e => e.AvatarId == MyId, token) >= MaxEndorsements)
return Error(StatusCodes.Status422UnprocessableEntity, $"Validation failed: You can endorse at most {MaxEndorsements} accounts");
try
{
await DB.Default.SaveAsync(new Endorsement { AvatarId = MyId, TargetAccountId = id, TargetActorURI = uri }, token);
}
catch (MongoDB.Driver.MongoWriteException ex) when (ex.WriteError?.Category == MongoDB.Driver.ServerErrorCategory.DuplicateKey)
{
}
}
return Json(await Relationship(id, token));
}
[HttpPost("/api/v1/accounts/{id}/unpin"), HttpPost("/api/v1/accounts/{id}/unendorse"), Scope("write:accounts")]
public async Task<IActionResult> Unendorse(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
var uri = local?.Uri ?? remote.ActorURI;
await DB.Default.DeleteAsync<Endorsement>(e => e.AvatarId == MyId && (e.TargetActorURI == uri || e.TargetAccountId == id));
return Json(await Relationship(id, token));
}
// the persona's own endorsements, newest first, those it still follows
[HttpGet("/api/v1/endorsements"), Scope("read:accounts")]
public Task<IActionResult> Endorsements(CancellationToken token) => EndorsementsOf("/api/v1/endorsements", token);
// Mastodon 4.4: an account's endorsements. Only the persona's own are answered, to the persona itself: endorsements
// stay private, so every other account answers none
[HttpGet("/api/v1/accounts/{id}/endorsements"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> EndorsementsOfAccount(string id, CancellationToken token) =>
MyId != default && id == MyId ? await EndorsementsOf($"/api/v1/accounts/{id}/endorsements", token) : Json(Array.Empty<Account>());
async Task<IActionResult> EndorsementsOf(string path, CancellationToken token)
{
var followed = (await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.State == FollowState.Accepted && f.FollowerKind == null)
.Project(f => f.Include(x => x.TargetActorURI)).ExecuteAsync(token)).Select(f => f.TargetActorURI).ToList();
var page = await Page.From(Params, Limit(40, 80))
.Fetch(DB.Default.Find<Endorsement>().Match(e => e.AvatarId == MyId && followed.Contains(e.TargetActorURI)), e => e.ID, token);
Link(path, page.LastOrDefault()?.ID, page.FirstOrDefault()?.ID);
var accounts = await _mapper.Accounts(page.Select(e => e.TargetAccountId), token);
return Json(page.Select(e => accounts.GetValueOrDefault(e.TargetAccountId)).Where(a => a != default).ToList());
}
// A private note on an account (R8): the persona's own (PersonaNote, never its login's), shown only to it in the
// relationship's `note`, never federated; an empty comment takes it away
[HttpPost("/api/v1/accounts/{id}/note"), Scope("write:accounts")]
public async Task<IActionResult> Note(string id, CancellationToken token)
{
var (local, remote) = await Find(id, token);
if (local == default && remote == default)
return NotFoundError();
var comment = Params.Get("comment")?.Trim();
if (comment?.Length > PersonaNote.MaxLength)
return Error(StatusCodes.Status422UnprocessableEntity, $"Validation failed: Comment is too long (maximum is {PersonaNote.MaxLength} characters)");
if (string.IsNullOrEmpty(comment))
await DB.Default.DeleteAsync<PersonaNote>(n => n.AvatarId == MyId && n.TargetAccountId == id);
else
await DB.Default.Update<PersonaNote>().Match(n => n.AvatarId == MyId && n.TargetAccountId == id)
.Modify(n => n.Text, comment)
.Modify(n => n.UpdatedAt, DateTime.UtcNow)
.Modify(b => b.SetOnInsert(n => n.CreatedAt, DateTime.UtcNow))
.Option(o => o.IsUpsert = true)
.ExecuteAsync(token);
return Json(await Relationship(id, token));
}
// Mastodon's preferences: posting defaults (update_credentials' source) and reading ones (R8: AvatarSettings, set
// through /clientapi/avatar/private/preferences, since Mastodon's API sets none)
[HttpGet("/api/v1/preferences"), Scope("read:accounts")]
public IActionResult Preferences() => Json(new Dictionary<string, object>
{
["posting:default:visibility"] = Me.Settings.DefaultVisibility ?? "public",
["posting:default:sensitive"] = Me.Settings.DefaultSensitive,
["posting:default:language"] = Me.Settings.DefaultLanguage,
["reading:expand:media"] = ExpandMediaModes.IsKnown(Me.Settings.ExpandMedia) ? Me.Settings.ExpandMedia : ExpandMediaModes.Default,
["reading:expand:spoilers"] = Me.Settings.ExpandSpoilers
});
const int MaxEndorsements = 500;
async Task<bool> Follows(string uri, CancellationToken token) =>
await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.TargetActorURI == uri && f.State == FollowState.Accepted && f.FollowerKind == null)
.ExecuteAnyAsync(token);
async Task<List<Account>> AccountsFor(IEnumerable<string> actorUris, CancellationToken token)
{
var accounts = new List<Account>();
@@ -474,6 +577,10 @@ namespace PrivaPub.Api.Mastodon.Controllers
: remote != default && await DB.Default.Find<BlockedBy>().Match(b => b.AvatarId == MyId && b.ActorURI == remote.ActorURI).ExecuteAnyAsync(token);
var domainBlocked = remote != default && (await DB.Default.Find<AccountDomainBlock>().Match(b => b.AvatarId == MyId).ExecuteAsync(token))
.Any(b => Hidden.Blocks(b.Domain, remote.ActorURI, remote.Domain));
// the persona's own endorsement and note (R8), never another persona's: a note made as a sibling stays its own
var endorsed = uri != default && following?.State == FollowState.Accepted
&& await DB.Default.Find<Endorsement>().Match(e => e.AvatarId == MyId && e.TargetActorURI == uri).ExecuteAnyAsync(token);
var note = await DB.Default.Find<PersonaNote>().Match(n => n.AvatarId == MyId && n.TargetAccountId == id).ExecuteFirstAsync(token);
return new Relationship
{
Id = id,
@@ -485,8 +592,12 @@ namespace PrivaPub.Api.Mastodon.Controllers
Following = following?.State == FollowState.Accepted || reading,
Requested = following?.State == FollowState.Requested,
ShowingReblogs = following?.ShowReblogs ?? false,
Notifying = following?.Notify ?? false,
Languages = following?.Languages,
FollowedBy = followedBy?.IsAccepted == true,
RequestedBy = followedBy is { IsAccepted: false }
RequestedBy = followedBy is { IsAccepted: false },
Endorsed = endorsed,
Note = note?.Text ?? string.Empty
};
}
}
@@ -0,0 +1,113 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Social;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Federation;
using PrivaPub.Models.Social;
using PrivaPub.StaticServices;
namespace PrivaPub.Api.Mastodon.Controllers
{
// Featured hashtags as Mastodon's API has them (completion program R8, Domain/Social/FeaturedTags): the persona's own,
// featured and unfeatured (its followers are told), suggested from the tags it uses, and any account's: a persona's own
// here, or those an account elsewhere features as its server tells them. They are public, as on Mastodon.
public class FeaturedTagsController : MastodonController
{
const int MaxSuggested = 10;//Mastodon's RECENT_TAGS_LIMIT
const int SuggestionPosts = 200;
readonly IFeaturedTags _featured;
readonly DbEntities _dbEntities;
readonly ILocalActorService _localActors;
public FeaturedTagsController(IFeaturedTags featured, DbEntities dbEntities, ILocalActorService localActors)
{
_featured = featured;
_dbEntities = dbEntities;
_localActors = localActors;
}
[HttpGet("/api/v1/featured_tags"), Scope("read:accounts")]
public async Task<IActionResult> Mine(CancellationToken token) => Json(await Entities(await _featured.Of(MyId, token), token));
[HttpPost("/api/v1/featured_tags"), Scope("write:accounts")]
public async Task<IActionResult> Feature(CancellationToken token)
{
var (tag, error) = await _featured.Feature(Me, Params.Get("name"), token);
return tag == default ? Error(StatusCodes.Status422UnprocessableEntity, error) : Json((await Entities(new List<FeaturedTag> { tag }, token))[0]);
}
[HttpDelete("/api/v1/featured_tags/{id}"), Scope("write:accounts")]
public async Task<IActionResult> Unfeature(string id, CancellationToken token) =>
await _featured.Unfeature(Me, id, default, token) == default ? NotFoundError() : Json(new { });
// the tags the persona used lately and does not feature yet, the most used first, as Mastodon's Tag
[HttpGet("/api/v1/featured_tags/suggestions"), Scope("read:accounts")]
public async Task<IActionResult> Suggestions(CancellationToken token)
{
var featured = (await _featured.Of(MyId, token)).Select(t => t.Name).ToHashSet(StringComparer.Ordinal);
var recent = await _dbEntities.Posts.Match(p => p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.ReblogOfPostId == null
&& p.Tags.Count > 0)
.Sort(p => p.ID, Order.Descending).Limit(SuggestionPosts).Project(p => p.Include(x => x.Tags)).ExecuteAsync(token);
var used = recent.SelectMany((p, age) => p.Tags.Distinct().Select(tag => (tag, age)))
.Where(t => !featured.Contains(t.tag) && TagsController.IsHashtag(t.tag))
.GroupBy(t => t.tag)
.OrderByDescending(g => g.Count()).ThenBy(g => g.Min(t => t.age))
.Take(MaxSuggested)
.Select(g => g.Key)
.ToList();
var views = new List<object>();
foreach (var tag in used)
views.Add(await TagsController.TagView(_dbEntities, _localActors.BaseAddress, tag, MyId, default, false, token));
return Json(views);
}
// what an account features: a persona's own hashtags, or what an account elsewhere tells (its featuredTags, its Add
// and Remove); a community features none. A suspended account's answer 404, as its account does (R6).
[HttpGet("/api/v1/accounts/{id}/featured_tags"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public async Task<IActionResult> OfAccount(string id, CancellationToken token)
{
var avatar = await _dbEntities.Avatars.MatchID(id).Match(a => !a.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (avatar != default)
return _localActors.FromAvatar(avatar) is { Suspended: false }
? Json(await Entities(await _featured.Of(avatar.ID, token), token))
: NotFoundError();
if (await _dbEntities.Groups.MatchID(id).Match(g => !g.DeletionAt.HasValue).ExecuteAnyAsync(token))
return Json(Array.Empty<FeaturedTagEntity>());
var remote = await _dbEntities.ForeignAvatars.MatchID(id).Match(f => !f.DeletionAt.HasValue).ExecuteFirstAsync(token);
if (remote == default || remote.SuspendedAt.HasValue)
return NotFoundError();
var tags = await DB.Default.Find<RemoteFeaturedTag>().Match(t => t.ActorURI == remote.ActorURI).Sort(t => t.Position, Order.Ascending)
.Limit(FeaturedTags.MaxRemote).ExecuteAsync(token);
var usage = await FeaturedTags.Usage(default, remote.ActorURI, tags.Select(t => t.Name).ToList(), _dbEntities, token);
return Json(tags.Select(t => Entity(t.ID, t.Name, t.DisplayName, t.Href, usage.GetValueOrDefault(t.Name))).ToList());
}
async Task<List<FeaturedTagEntity>> Entities(List<FeaturedTag> tags, CancellationToken token)
{
var byPersona = tags.GroupBy(t => t.AvatarId).ToList();
var entities = new List<FeaturedTagEntity>();
foreach (var persona in byPersona)
{
var usage = await FeaturedTags.Usage(persona.Key, default, persona.Select(t => t.Name).ToList(), _dbEntities, token);
entities.AddRange(persona.Select(t => Entity(t.ID, t.Name, t.DisplayName, default, usage.GetValueOrDefault(t.Name))));
}
return entities;
}
FeaturedTagEntity Entity(string id, string name, string displayName, string href, (long Count, DateTime? Last) usage) => new()
{
Id = id,
Name = string.IsNullOrEmpty(displayName) ? name : displayName,
// (a link to the remote site is only a link, which the reader follows or not; ours is this server's tag page)
Url = href ?? ActivityPubRenderer.TagUrl(_localActors.BaseAddress, name),
StatusesCount = usage.Count.ToString(System.Globalization.CultureInfo.InvariantCulture),
LastStatusAt = usage.Last?.ToString("yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture)
};
}
}
@@ -134,7 +134,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
registrations = Open,
approval_required = false,
invites_enabled = true,//a group invitation always creates an account
configuration = new { accounts = new { max_featured_tags = 0 }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls },
configuration = new { accounts = new { max_featured_tags = PrivaPub.Domain.Social.FeaturedTags.MaxFeatured }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls },
contact_account = await ContactAccount(settings, token),
rules = Rules(settings)
});
@@ -161,7 +161,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
configuration = new
{
urls = new { streaming = Streaming },
accounts = new { max_featured_tags = 0, max_pinned_statuses = StatusesController.MaxPins },
accounts = new { max_featured_tags = PrivaPub.Domain.Social.FeaturedTags.MaxFeatured, max_pinned_statuses = StatusesController.MaxPins },
statuses = Statuses,
media_attachments = MediaAttachments,
polls = Polls,
@@ -140,28 +140,6 @@ namespace PrivaPub.Api.Mastodon.Controllers
public class StubsController : MastodonController
{
[HttpGet("/api/v1/suggestions"), Scope("read")]
public IActionResult SuggestionsV1() => Json(Array.Empty<object>());
[HttpGet("/api/v2/suggestions"), Scope("read")]
public IActionResult SuggestionsV2() => Json(Array.Empty<object>());
[HttpGet("/api/v1/endorsements"), Scope("read:accounts")]
public IActionResult Endorsements() => Json(Array.Empty<object>());
[HttpGet("/api/v1/featured_tags"), Scope("read:accounts")]
public IActionResult FeaturedTags() => Json(Array.Empty<object>());
[HttpGet("/api/v1/preferences"), Scope("read:accounts")]
public IActionResult Preferences() => Json(new Dictionary<string, object>
{
["posting:default:visibility"] = Me.Settings.DefaultVisibility ?? "public",
["posting:default:sensitive"] = Me.Settings.DefaultSensitive,
["posting:default:language"] = Me.Settings.DefaultLanguage,
["reading:expand:media"] = "default",
["reading:expand:spoilers"] = false
});
[HttpGet("/api/v1/push/subscription"), Scope("push")]
public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found");
@@ -179,5 +157,21 @@ namespace PrivaPub.Api.Mastodon.Controllers
[HttpGet("/api/v1/instance/translation_languages"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult TranslationLanguages() => Json(new { });
// translation is cut (owner decision 2026-10-17): no language offers any (translation_languages), and a client that
// asks anyway is told so, as Mastodon tells it when it can't translate
[HttpPost("/api/v1/statuses/{id}/translate"), Scope("read:statuses")]
public IActionResult Translate(string id) => Error(StatusCodes.Status422UnprocessableEntity, "Translation is not available on this server");
// Mastodon 4.3's yearly reports (Wrapstodon): none are made here, so there are never any to show
[HttpGet("/api/v1/annual_reports"), Scope("read:accounts")]
public IActionResult AnnualReports() => Json(new { annual_reports = Array.Empty<object>(), accounts = Array.Empty<object>(), statuses = Array.Empty<object>() });
// the Mastodon admin API is cut: moderation stays on /clientapi (ModeratorController, AdminController), so every
// admin route answers as Mastodon answers an account that may not use it, whoever asks
[HttpGet("/api/v1/admin/{**rest}"), HttpPost("/api/v1/admin/{**rest}"), HttpPut("/api/v1/admin/{**rest}"), HttpPatch("/api/v1/admin/{**rest}"),
HttpDelete("/api/v1/admin/{**rest}"), HttpGet("/api/v2/admin/{**rest}"), HttpPost("/api/v2/admin/{**rest}"),
ApiExplorerSettings(IgnoreApi = true), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult Admin(string rest) => Error(StatusCodes.Status403Forbidden, "This action is not allowed");
}
}
@@ -41,7 +41,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
{
switch (name)
{
case Streams.User or Streams.UserNotification or Streams.Public or Streams.PublicLocal or Streams.PublicRemote:
case Streams.User or Streams.UserNotification or Streams.Public or Streams.PublicLocal or Streams.PublicRemote or Streams.Direct:
return (new Streams.Key(name), default);
case Streams.Hashtag or Streams.HashtagLocal:
var normal = TagsController.Normalise(tag);
@@ -69,6 +69,15 @@ namespace PrivaPub.Api.Mastodon.Controllers
}
if (e.Name == "announcement.reaction")
return JsonSerializer.Serialize(await AnnouncementsController.ReactionEvent(e.Id, e.Data, token), MastodonJson.Options);
// a direct conversation the persona takes part in (R8): Mastodon's Conversation, as /api/v1/conversations lists it
if (e.Name == "conversation")
{
var group = await _dbEntities.DmGroups.Match(g => g.ID == e.Id && !g.DeletionAt.HasValue && g.Members.Any(m => !m.IsForeign && m.AvatarId == MyId))
.ExecuteFirstAsync(token);
var conversation = group == default ? default
: (await TimelinesController.ConversationsOf(_mapper, _dbEntities, MyId, new List<Models.Group.DmGroup> { group }, token)).FirstOrDefault();
return conversation == default ? default : JsonSerializer.Serialize(conversation, MastodonJson.Options);
}
if (e.Name == "notification")
{
var notification = await _dbEntities.Notifications.Match(n => n.ID == e.Id && n.AvatarId == MyId).ExecuteFirstAsync(token);
@@ -0,0 +1,76 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Api.Mastodon.Mappers;
using PrivaPub.Domain.Instance;
using PrivaPub.Domain.Relationships;
using PrivaPub.Domain.Social;
using PrivaPub.Models.Social;
namespace PrivaPub.Api.Mastodon.Controllers
{
// Follow suggestions as Mastodon's API has them (completion program R8; who and why in Domain/Social/Suggestions): v1 the
// accounts, v2 each with its reasons; a dismissed account is never suggested to that persona again.
public class SuggestionsController : MastodonController
{
readonly MastodonMapper _mapper;
readonly IInstanceSettings _settings;
public SuggestionsController(MastodonMapper mapper, IInstanceSettings settings)
{
_mapper = mapper;
_settings = settings;
}
[HttpGet("/api/v1/suggestions"), Scope("read")]
public async Task<IActionResult> V1(CancellationToken token) => Json((await Suggested(token)).Select(s => s.Account).ToList());
[HttpGet("/api/v2/suggestions"), Scope("read")]
public async Task<IActionResult> V2(CancellationToken token) => Json(await Suggested(token));
[HttpDelete("/api/v1/suggestions/{accountId}"), Scope("read")]
public async Task<IActionResult> Dismiss(string accountId, CancellationToken token)
{
if (!string.IsNullOrEmpty(accountId) && accountId.Length <= 64)
try
{
await DB.Default.Update<SuggestionDismissal>().Match(d => d.AvatarId == MyId && d.AccountId == accountId)
.Modify(b => b.SetOnInsert(d => d.CreatedAt, DateTime.UtcNow)).Option(o => o.IsUpsert = true).ExecuteAsync(token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
}
return Json(new { });
}
// the candidates mapped a few at a time, until the page is full: an account no longer shown (deleted, suspended), one
// shown limited (silenced, or its server), a community outside the staff's picks, and one the persona hides (its
// blocked servers) are passed over
async Task<List<Suggestion>> Suggested(CancellationToken token)
{
var limit = Limit(40, 80);
var candidates = await Suggestions.For(Me, _settings.Current.StaffPicks, token);
var suggested = new List<Suggestion>();
for (var start = 0; start < candidates.Count && suggested.Count < limit; start += limit * 2)
{
var chunk = candidates.Skip(start).Take(limit * 2).ToList();
var accounts = await _mapper.Accounts(chunk.Select(c => c.AccountId), token);
var hidden = await Hidden.AuthorsHiddenFrom(MyId, accounts.Values.Select(a => a.Uri), forNotifications: false, token);
foreach (var candidate in chunk)
{
if (accounts.GetValueOrDefault(candidate.AccountId) is not { } account || account.Limited == true || hidden.Contains(account.Uri)
|| account.Group && candidate.Source != Suggestions.Staff)
continue;
suggested.Add(new Suggestion { Source = candidate.Source, Sources = candidate.Sources, Account = account });
if (suggested.Count >= limit)
break;
}
}
return suggested;
}
}
}
@@ -59,12 +59,19 @@ namespace PrivaPub.Api.Mastodon.Controllers
}).ToList();
}
async Task<object> View(string tag, bool? following, CancellationToken token) => new
Task<object> View(string tag, bool? following, CancellationToken token, bool? featuring = default) =>
TagView(_dbEntities, _localActors.BaseAddress, tag, MyId, following, featuring, token);
// Mastodon's Tag: its week of use, and for a signed-in persona whether it follows it and (Mastodon 4.4) features it
internal static async Task<object> TagView(DbEntities dbEntities, string baseAddress, string tag, string viewerId, bool? following, bool? featuring,
CancellationToken token) => new
{
name = tag,
url = $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag)}",
history = await History(_dbEntities, tag, token),
following = following ?? (MyId != default && await DB.Default.Find<FollowedTag>().Match(t => t.AvatarId == MyId && t.Name == tag).ExecuteAnyAsync(token))
url = $"{baseAddress}/tags/{Uri.EscapeDataString(tag)}",
history = await History(dbEntities, tag, token),
following = following ?? (viewerId != default && await DB.Default.Find<FollowedTag>().Match(t => t.AvatarId == viewerId && t.Name == tag).ExecuteAnyAsync(token)),
featuring = viewerId == default ? default(bool?)
: featuring ?? await DB.Default.Find<FeaturedTag>().Match(t => t.AvatarId == viewerId && t.Name == tag).ExecuteAnyAsync(token)
};
[HttpGet("/api/v1/tags/{name}"), Scope("read", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
@@ -103,6 +110,24 @@ namespace PrivaPub.Api.Mastodon.Controllers
return Json(await View(tag, false, token));
}
// Mastodon 4.4: a hashtag featured on the persona's profile, or no longer (R8, Domain/Social/FeaturedTags)
[HttpPost("/api/v1/tags/{name}/feature"), Scope("write:accounts")]
public async Task<IActionResult> Feature(string name, [FromServices] Domain.Social.IFeaturedTags featured, CancellationToken token)
{
var (tag, error) = await featured.Feature(Me, name, token);
return tag == default ? Error(StatusCodes.Status422UnprocessableEntity, error) : Json(await View(tag.Name, default, token, featuring: true));
}
[HttpPost("/api/v1/tags/{name}/unfeature"), Scope("write:accounts")]
public async Task<IActionResult> Unfeature(string name, [FromServices] Domain.Social.IFeaturedTags featured, CancellationToken token)
{
var tag = Normalise(name);
if (!IsHashtag(tag))
return NotFoundError();
await featured.Unfeature(Me, default, tag, token);
return Json(await View(tag, default, token, featuring: false));
}
[HttpGet("/api/v1/followed_tags"), Scope("read:follows")]
public async Task<IActionResult> Followed(CancellationToken token)
{
@@ -123,35 +123,39 @@ namespace PrivaPub.Api.Mastodon.Controllers
return Json(await ConversationsOf(groups, token));
}
async Task<List<Conversation>> ConversationsOf(List<DmGroup> groups, CancellationToken token)
Task<List<Conversation>> ConversationsOf(List<DmGroup> groups, CancellationToken token) => ConversationsOf(_mapper, _dbEntities, MyId, groups, token);
// (also the `direct` stream's conversation event, StreamingController)
internal static async Task<List<Conversation>> ConversationsOf(MastodonMapper mapper, DbEntities dbEntities, string viewerId, List<DmGroup> groups,
CancellationToken token)
{
var ids = groups.Select(g => g.ID).ToList();
var states = (await DB.Default.Find<ConversationState>().Match(s => s.AvatarId == MyId && ids.Contains(s.ConversationId)).ExecuteAsync(token))
var states = (await DB.Default.Find<ConversationState>().Match(s => s.AvatarId == viewerId && ids.Contains(s.ConversationId)).ExecuteAsync(token))
.ToDictionary(s => s.ConversationId);
groups = groups.Where(g => !ConversationStates.IsHidden(states.GetValueOrDefault(g.ID), g.LastPostId)).ToList();
// the newest post each conversation still shows: usually its LastPostId, else (deleted) the one before
var newestIds = groups.Select(g => g.LastPostId).ToList();
var newest = (await _dbEntities.Posts.Match(p => newestIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token))
var newest = (await dbEntities.Posts.Match(p => newestIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token))
.ToDictionary(p => p.ConversationId);
foreach (var group in groups.Where(g => !newest.ContainsKey(g.ID)))
if (await _dbEntities.Posts.Match(p => p.ConversationId == group.ID).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending)
if (await dbEntities.Posts.Match(p => p.ConversationId == group.ID).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending)
.ExecuteFirstAsync(token) is { } shown)
newest[group.ID] = shown;
var foreignUris = groups.SelectMany(g => g.Members).Where(m => m.IsForeign).Select(m => m.AvatarId).Distinct().ToList();
var foreignIds = (await _dbEntities.ForeignAvatars.Match(f => foreignUris.Contains(f.ActorURI)).ExecuteAsync(token))
var foreignIds = (await dbEntities.ForeignAvatars.Match(f => foreignUris.Contains(f.ActorURI)).ExecuteAsync(token))
.GroupBy(f => f.ActorURI).ToDictionary(g => g.Key, g => g.First().ID);
List<string> Others(DmGroup group) => group.Members.Where(m => !(m.AvatarId == MyId && !m.IsForeign))
List<string> Others(DmGroup group) => group.Members.Where(m => !(m.AvatarId == viewerId && !m.IsForeign))
.Select(m => m.IsForeign ? foreignIds.GetValueOrDefault(m.AvatarId) : m.AvatarId).Where(id => id != default).ToList();
var accounts = await _mapper.Accounts(groups.SelectMany(Others).Distinct(), token);
var statuses = (await _mapper.Statuses(newest.Values.ToList(), MyId, token)).ToDictionary(s => s.Id);
var accounts = await mapper.Accounts(groups.SelectMany(Others).Distinct(), token);
var statuses = (await mapper.Statuses(newest.Values.ToList(), viewerId, token)).ToDictionary(s => s.Id);
return groups.Where(g => newest.ContainsKey(g.ID) && statuses.ContainsKey(newest[g.ID].ID))
.Select(g => new Conversation
{
Id = g.ID,
Unread = ConversationStates.IsUnread(states.GetValueOrDefault(g.ID), newest[g.ID].ID, newest[g.ID].AuthorAccountId, MyId),
Unread = ConversationStates.IsUnread(states.GetValueOrDefault(g.ID), newest[g.ID].ID, newest[g.ID].AuthorAccountId, viewerId),
Accounts = Others(g).Select(id => accounts.GetValueOrDefault(id)).Where(a => a != default).ToList(),
LastStatus = statuses[newest[g.ID].ID]
})
@@ -240,7 +244,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
[NotificationType.Poll] = "poll",
[NotificationType.Reaction] = "pleroma:emoji_reaction",
[NotificationType.Quote] = "quote",
[NotificationType.ModerationWarning] = "moderation_warning"
[NotificationType.ModerationWarning] = "moderation_warning",
[NotificationType.Status] = "status"//an account followed with notify posted (R8)
};
readonly MastodonMapper _mapper;
@@ -410,6 +410,24 @@
public string Note { get; set; } = string.Empty;
}
// Mastodon's FeaturedTag: its count as a string and its last use as a day, as Mastodon's serializer writes them
public class FeaturedTagEntity
{
public string Id { get; set; }
public string Name { get; set; }
public string Url { get; set; }
public string StatusesCount { get; set; } = "0";
public string LastStatusAt { get; set; }
}
// Mastodon's v2 Suggestion: why (source, deprecated since 4.3, and sources) and whom
public class Suggestion
{
public string Source { get; set; }
public List<string> Sources { get; set; } = new();
public Account Account { get; set; }
}
public class Notification
{
public string Id { get; set; }
@@ -275,6 +275,7 @@ namespace PrivaPub.Controllers.ClientToServer
DeletedAt = r.DeletedAt
}).ToList(),
Relays = await RelayViews(settings, token),
StaffPicks = await StaffPickViews(settings, token),
Media = new ViewMediaLimits
{
QuotaBytesPerRoot = limits.QuotaBytesPerRoot,
@@ -387,6 +388,103 @@ namespace PrivaPub.Controllers.ClientToServer
#endregion
#region staff picks
// the accounts suggested first to every persona (completion program R8, Domain/Social/Suggestions): a persona here or an
// account elsewhere the administrators choose, in their order; kept in the settings, changed only here, every change in
// the staff's log (an entry about a persona of the acting login keeps no actor, ModerationLog)
[HttpGet, Route("/clientapi/admin/staff-picks")]
public async Task<IActionResult> StaffPicks(CancellationToken token)
{
if (await Administrator(token) == default)
return Forbid();
return Ok(await StaffPickViews(_settings.Current, token));
}
[HttpPost, Route("/clientapi/admin/staff-picks/insert")]
public async Task<IActionResult> InsertStaffPick(StaffPickForm form, CancellationToken token)
{
var admin = await Administrator(token);
if (admin == default)
return Forbid();
var id = form.AccountId?.Trim();
var picked = await Pickable(id, token);
if (picked == default)
return Refused(StatusCodes.Status404NotFound, "No such account to suggest.");
if (_settings.Current.StaffPicks.Contains(id))
return Refused(StatusCodes.Status409Conflict, "This account is suggested already.");
if (_settings.Current.StaffPicks.Count >= MaxStaffPicks)
return Refused(StatusCodes.Status422UnprocessableEntity, "40 accounts at most are suggested.");
var saved = await _settings.Update(settings =>
{
if (!settings.StaffPicks.Contains(id))
settings.StaffPicks.Add(id);
}, admin.ID, token);
await _log.Write(admin.ID, AdminActions.StaffPickInsert, picked.Value.IsLocal ? AdminTargets.Persona : AdminTargets.Account, id, picked.Value.Handle,
token: token);
return Ok(await StaffPickViews(saved, token));
}
[HttpPost, Route("/clientapi/admin/staff-picks/delete")]
public async Task<IActionResult> DeleteStaffPick([FromQuery] string accountId, CancellationToken token)
{
var admin = await Administrator(token);
if (admin == default)
return Forbid();
accountId = accountId?.Trim();
if (string.IsNullOrEmpty(accountId) || !_settings.Current.StaffPicks.Contains(accountId))
return NotFound();
var view = (await StaffPickViews(_settings.Current, token)).FirstOrDefault(p => p.AccountId == accountId);
var saved = await _settings.Update(settings => settings.StaffPicks.Remove(accountId), admin.ID, token);
await _log.Write(admin.ID, AdminActions.StaffPickDelete, view?.IsLocal == true ? AdminTargets.Persona : AdminTargets.Account, accountId,
view?.Handle ?? accountId, token: token);
return Ok(await StaffPickViews(saved, token));
}
const int MaxStaffPicks = 40;
// a persona that may be shown, or an account elsewhere that is a person (no group, feed or server's actor), neither
// suspended nor deleted
async Task<(bool IsLocal, string Handle)?> Pickable(string id, CancellationToken token)
{
if (string.IsNullOrEmpty(id) || id.Length > 64)
return default;
var avatar = await DB.Default.Find<Avatar>().Match(a => a.ID == id && !a.DeletionAt.HasValue && !a.SuspendedAt.HasValue).ExecuteFirstAsync(token);
if (avatar != default)
return (await Counted.HeldBackPersonas(token)).Contains(avatar.ID) ? ((bool, string)?)null : (true, _localActors.FromAvatar(avatar).Handle);
var foreign = await DB.Default.Find<ForeignAvatar>().Match(f => f.ID == id && !f.DeletionAt.HasValue && !f.SuspendedAt.HasValue
&& (f.AvatarType == AvatarType.Person || f.AvatarType == AvatarType.Service || f.AvatarType == AvatarType.Organization)).ExecuteFirstAsync(token);
return foreign == default ? ((bool, string)?)null : (false, $"{foreign.UserName}@{foreign.Domain}");
}
async Task<List<ViewStaffPick>> StaffPickViews(ServerSettings settings, CancellationToken token)
{
var ids = (settings.StaffPicks ?? new()).ToList();
var held = await Counted.HeldBackPersonas(token);
var avatars = (await DB.Default.Find<Avatar>().Match(a => ids.Contains(a.ID)).ExecuteAsync(token)).ToDictionary(a => a.ID);
var foreigns = (await DB.Default.Find<ForeignAvatar>().Match(f => ids.Contains(f.ID)).ExecuteAsync(token)).ToDictionary(f => f.ID);
return ids.Select(id => avatars.TryGetValue(id, out var avatar)
? new ViewStaffPick
{
AccountId = id,
IsLocal = true,
Handle = _localActors.FromAvatar(avatar).Handle,
Name = avatar.Name,
IsShown = avatar.DeletionAt == default && avatar.SuspendedAt == default && avatar.SilencedAt == default && !held.Contains(id)
}
: foreigns.TryGetValue(id, out var foreign)
? new ViewStaffPick
{
AccountId = id,
Handle = $"{foreign.UserName}@{foreign.Domain}",
Name = foreign.Name,
IsShown = foreign.DeletionAt == default && foreign.SuspendedAt == default && foreign.SilencedAt == default
}
: new ViewStaffPick { AccountId = id, IsShown = false }).ToList();
}
#endregion
#region relays
[HttpGet, Route("/clientapi/admin/relays")]
@@ -74,6 +74,51 @@ namespace PrivaPub.Controllers.ClientToServer
return Ok();
}
// how the persona's clients show what others post (R8): Mastodon's reading preferences, which its API only reads
[HttpGet, Route("/clientapi/avatar/private/preferences")]
public async Task<IActionResult> Preferences([FromQuery] string avatarId, [FromServices] StaticServices.DbEntities dbEntities, CancellationToken token)
{
var avatar = await OwnPersona(avatarId, dbEntities, token);
return avatar == default
? NotFound(new WebResult().Invalidate(_localizer["Avatar not found."], StatusCodes.Status404NotFound))
: Ok(PreferencesOf(avatar));
}
[HttpPost, Route("/clientapi/avatar/private/preferences")]
public async Task<IActionResult> SavePreferences(PersonaPreferencesForm form, [FromServices] StaticServices.DbEntities dbEntities, CancellationToken token)
{
if (!ModelState.IsValid)
return BadRequest(new WebResult().Invalidate(_localizer["Invalid model."]));
if (!Models.User.ExpandMediaModes.IsKnown(form.ExpandMedia))
return UnprocessableEntity(new WebResult().Invalidate(_localizer["Media are shown as default, show_all or hide_all."], StatusCodes.Status422UnprocessableEntity));
var avatar = await OwnPersona(form.AvatarId, dbEntities, token);
if (avatar == default)
return NotFound(new WebResult().Invalidate(_localizer["Avatar not found."], StatusCodes.Status404NotFound));
await MongoDB.Entities.DB.Default.Update<Models.User.Avatar>().MatchID(avatar.ID)
.Modify(a => a.Settings.ExpandMedia, form.ExpandMedia)
.Modify(a => a.Settings.ExpandSpoilers, form.ExpandSpoilers)
.ExecuteAsync(token);
avatar.Settings.ExpandMedia = form.ExpandMedia;
avatar.Settings.ExpandSpoilers = form.ExpandSpoilers;
return Ok(PreferencesOf(avatar));
}
// one of the login's own personas, not deleted (a suspended one keeps its preferences for when it is back)
async Task<Models.User.Avatar> OwnPersona(string avatarId, StaticServices.DbEntities dbEntities, CancellationToken token)
{
var rootId = User.GetUserId();
if (string.IsNullOrEmpty(avatarId) || !await dbEntities.RootToAvatars.Match(ra => ra.RootId == rootId && ra.AvatarId == avatarId).ExecuteAnyAsync(token))
return default;
return await dbEntities.Avatars.MatchID(avatarId).Match(a => !a.DeletionAt.HasValue).ExecuteFirstAsync(token);
}
static ViewPersonaPreferences PreferencesOf(Models.User.Avatar avatar) => new()
{
AvatarId = avatar.ID,
ExpandMedia = Models.User.ExpandMediaModes.IsKnown(avatar.Settings.ExpandMedia) ? avatar.Settings.ExpandMedia : Models.User.ExpandMediaModes.Default,
ExpandSpoilers = avatar.Settings.ExpandSpoilers
};
[HttpPost, Route("/clientapi/avatar/private/update")]
public async Task<IActionResult> UpdateAvatar(UpdateAvatarForm model)
{
@@ -128,7 +128,8 @@ namespace PrivaPub.Domain.Portability
var follows = await DB.Default.Find<Following>().Match(f => f.AvatarId == actor.Id).ExecuteAsync(token);
var csv = new StringBuilder("Account address,Show boosts,Notify on new posts,Languages\n");
foreach (var follow in follows)
csv.Append($"{Csv(await handles.Of(follow.TargetActorURI, token))},{(follow.ShowReblogs ? "true" : "false")},false,\n");
csv.Append($"{Csv(await handles.Of(follow.TargetActorURI, token))},{(follow.ShowReblogs ? "true" : "false")},{(follow.Notify ? "true" : "false")},"
+ $"{Csv(string.Join(", ", follow.Languages ?? new List<string>()))}\n");
await Text(zip, "following_accounts.csv", csv.ToString(), token);
csv.Clear();
@@ -707,7 +707,10 @@ namespace PrivaPub.Domain.Portability
|| await DB.Default.Find<Following>().Match(f => f.AvatarId == run.Me.Id && f.TargetActorURI == uri).ExecuteAnyAsync(token))
return false;
var boosts = cells.Length < 2 || !cells[1].Trim().Equals("false", StringComparison.OrdinalIgnoreCase);
return await run.Services.GetRequiredService<IFollowService>().FollowAs(run.Me, uri, boosts, token) != default;
// Mastodon's columns: notify on new posts, and the languages (", " between them) its posts reach the home in (R8)
var notify = cells.Length >= 3 && cells[2].Trim().Equals("true", StringComparison.OrdinalIgnoreCase);
var languages = cells.Length >= 4 ? cells[3].Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries) : [];
return await run.Services.GetRequiredService<IFollowService>().FollowAs(run.Me, uri, boosts, token, notify, languages) != default;
}
// lists by title, with only the accounts the persona follows
+264
View File
@@ -0,0 +1,264 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Controllers;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox;
using PrivaPub.Federation.Objects;
using PrivaPub.Federation.Outbox;
using PrivaPub.Federation.Rendering;
using PrivaPub.Infrastructure.Http;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.StaticServices;
using System.Text.Json;
using System.Text.Json.Nodes;
using static PrivaPub.Federation.Objects.ActivityJson;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Social
{
public interface IFeaturedTags
{
/// <summary>The persona features a hashtag (already featured, it is as it was): the row, or why not.</summary>
Task<(FeaturedTag Tag, string Error)> Feature(LocalActor persona, string name, CancellationToken token);
/// <summary>The persona stops featuring a hashtag, named by the row's id or by its name: the row taken away, if any.</summary>
Task<FeaturedTag> Unfeature(LocalActor persona, string id, string name, CancellationToken token);
Task<List<FeaturedTag>> Of(string avatarId, CancellationToken token);
/// <summary>An account elsewhere features a hashtag (Add of a Hashtag) or stops (Remove).</summary>
Task Receive(JsonNode activity, ForeignAvatar owner, bool add, CancellationToken token);
/// <summary>An account elsewhere's featuredTags collection, read again (with its counts, at most daily).</summary>
Task Sync(ForeignAvatar owner, CancellationToken token);
}
// Featured hashtags (Mastodon's featured tags, completion program R8). A persona's are public: its featuredTags
// collection (/peasants/{name}/tattoos) lists them as Hashtag objects, and featuring one sends Add{Hashtag} to its
// followers, unfeaturing it Remove{Hashtag}, both with its `featured` collection (/trophies) as the target: that is how
// Mastodon sends them (its AddHashtagSerializer), and the only target Mastodon takes a Hashtag from. Received the same
// way, with either collection as the target, and read from an account's featuredTags collection with its counts.
public class FeaturedTags : IFeaturedTags
{
public const int MaxFeatured = 10;//the instance API's max_featured_tags, Mastodon's FeaturedTag::LIMIT
public const int MaxRemote = 20;//what we keep of an account elsewhere, whatever its server allows
readonly IDeliveryService _delivery;
readonly IRemoteActorService _remoteActors;
public FeaturedTags(IDeliveryService delivery, IRemoteActorService remoteActors)
{
_delivery = delivery;
_remoteActors = remoteActors;
}
// a featured hashtag as Mastodon renders it (its HashtagSerializer): the link to this server's page of the tag, which
// is what our posts' Hashtag tags link to, and the name as the persona wrote it
public static JsonObject Hashtag(string baseAddress, FeaturedTag tag) => new()
{
["type"] = "Hashtag",
["href"] = ActivityPubRenderer.TagUrl(baseAddress, tag.Name),
["name"] = "#" + (string.IsNullOrEmpty(tag.DisplayName) ? tag.Name : tag.DisplayName)
};
public async Task<List<FeaturedTag>> Of(string avatarId, CancellationToken token) =>
await DB.Default.Find<FeaturedTag>().Match(t => t.AvatarId == avatarId).Sort(t => t.ID, Order.Ascending).ExecuteAsync(token);
public async Task<(FeaturedTag Tag, string Error)> Feature(LocalActor persona, string name, CancellationToken token)
{
var display = name?.Trim().TrimStart('#');
var normal = TagsController.Normalise(display);
if (!TagsController.IsHashtag(normal))
return (default, "Validation failed: Tag is invalid");
var existing = await DB.Default.Find<FeaturedTag>().Match(t => t.AvatarId == persona.Id && t.Name == normal).ExecuteFirstAsync(token);
if (existing != default)
return (existing, default);
if (await DB.Default.CountAsync<FeaturedTag>(t => t.AvatarId == persona.Id, token) >= MaxFeatured)
return (default, $"Validation failed: You can feature at most {MaxFeatured} hashtags");
var tag = new FeaturedTag
{
AvatarId = persona.Id,
Name = normal,
DisplayName = display.Length <= 100 ? display : normal
};
try
{
await DB.Default.SaveAsync(tag, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
return (await DB.Default.Find<FeaturedTag>().Match(t => t.AvatarId == persona.Id && t.Name == normal).ExecuteFirstAsync(token), default);
}
await Tell(persona, tag, add: true, token);
return (tag, default);
}
public async Task<FeaturedTag> Unfeature(LocalActor persona, string id, string name, CancellationToken token)
{
var normal = id == default ? TagsController.Normalise(name) : default;
var tag = id != default
? await DB.Default.Find<FeaturedTag>().Match(t => t.ID == id && t.AvatarId == persona.Id).ExecuteFirstAsync(token)
: await DB.Default.Find<FeaturedTag>().Match(t => t.AvatarId == persona.Id && t.Name == normal).ExecuteFirstAsync(token);
if (tag == default)
return default;
var removed = await DB.Default.DeleteAsync<FeaturedTag>(t => t.ID == tag.ID);
if (removed.DeletedCount > 0)
await Tell(persona, tag, add: false, token);
return tag;
}
// to the persona's followers, as Mastodon distributes it (AccountRawDistributionWorker); a suspended persona's goes
// nowhere (DeliveryService.HeldBack)
async Task Tell(LocalActor persona, FeaturedTag tag, bool add, CancellationToken token)
{
if (persona is not { IsFederated: true, Kind: Models.Federation.LocalActorKind.Person })
return;
await _delivery.EnqueueToFollowers(persona, new JsonObject
{
["@context"] = ActivityPubRenderer.Context(),
["id"] = persona.ActivityUri($"{(add ? "feature" : "unfeature")}-tag-{tag.ID}-{DateTime.UtcNow.Ticks}"),
["type"] = add ? "Add" : "Remove",
["actor"] = persona.Uri,
["object"] = Hashtag(persona.BaseAddress, tag),
["target"] = persona.Featured,
["to"] = new JsonArray(ActivityPubRenderer.Public),
["cc"] = new JsonArray(persona.Followers)
}, token);
}
// what posts carry the tag: a persona's own public and unlisted posts, or the public and unlisted posts we hold of an
// account elsewhere, as Mastodon counts a featured tag (statuses_count, last_status_at)
public static async Task<Dictionary<string, (long Count, DateTime? Last)>> Usage(string avatarId, string actorUri, IReadOnlyCollection<string> names,
DbEntities dbEntities, CancellationToken token)
{
var usage = new Dictionary<string, (long, DateTime?)>(StringComparer.Ordinal);
foreach (var name in names.Distinct(StringComparer.Ordinal))
{
var tag = name;
var query = avatarId != default
? dbEntities.Posts.Match(p => p.GroupUserId == avatarId && !p.IsFederatedCopy)
: dbEntities.Posts.Match(p => p.ActorURI == actorUri && p.IsFederatedCopy);
query.Match(p => p.Tags.Contains(tag) && p.ReblogOfPostId == null
&& (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted)).Match(Privacy.VisibilityPolicy.IsShown);
var latest = await query.Sort(p => p.ID, Order.Descending).Limit(1).ExecuteFirstAsync(token);
if (latest == default)
{
usage[name] = (0, default);
continue;
}
var counted = avatarId != default
? await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.GroupUserId == avatarId && !p.IsFederatedCopy && p.Tags.Contains(tag)
&& p.ReblogOfPostId == null && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted))
& f.Where(Privacy.VisibilityPolicy.IsShown), token)
: await DB.Default.CountAsync<PostEntity>(f => f.Where(p => p.ActorURI == actorUri && p.IsFederatedCopy && p.Tags.Contains(tag)
&& p.ReblogOfPostId == null && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted))
& f.Where(Privacy.VisibilityPolicy.IsShown), token);
usage[name] = (counted, latest.CreationDate);
}
return usage;
}
public async Task Receive(JsonNode activity, ForeignAvatar owner, bool add, CancellationToken token)
{
var (normal, display) = Named(Value(activity["object"], "name"));
if (normal == default)
{
Arrival.Drop("unparseable");
return;
}
if (!add)
{
var removed = await DB.Default.DeleteAsync<RemoteFeaturedTag>(t => t.ActorURI == owner.ActorURI && t.Name == normal);
if (removed is { IsAcknowledged: true, DeletedCount: > 0 })
Arrival.Accept("tag-unfeatured");
else
Arrival.Drop("unknown-tag");
return;
}
if (await DB.Default.CountAsync<RemoteFeaturedTag>(t => t.ActorURI == owner.ActorURI, token) >= MaxRemote)
{
Arrival.Drop("too-many-tags");
return;
}
var first = await DB.Default.Find<RemoteFeaturedTag>().Match(t => t.ActorURI == owner.ActorURI).Sort(t => t.Position, Order.Ascending).ExecuteFirstAsync(token);
try
{
await DB.Default.SaveAsync(new RemoteFeaturedTag
{
ActorURI = owner.ActorURI,
Name = normal,
DisplayName = display,
Href = Link(Value(activity["object"], "href")),
Position = (first?.Position ?? 0) - 1
}, token);
Arrival.Accept("tag-featured");
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
Arrival.Drop("duplicate");
}
}
public async Task Sync(ForeignAvatar owner, CancellationToken token)
{
if (string.IsNullOrEmpty(owner.FeaturedTagsURL) || !Origin.Same(owner.FeaturedTagsURL, owner.ActorURI))
return;
var items = await Items(owner, token);
if (items == default)
return;
var tags = new List<RemoteFeaturedTag>();
foreach (var item in items)
{
if (Value(item, "type") != "Hashtag" || Named(Value(item, "name")) is not ({ } normal, var display) || tags.Any(t => t.Name == normal))
continue;
tags.Add(new RemoteFeaturedTag { ActorURI = owner.ActorURI, Name = normal, DisplayName = display, Href = Link(Value(item, "href")), Position = tags.Count });
if (tags.Count >= MaxRemote)
break;
}
await DB.Default.DeleteAsync<RemoteFeaturedTag>(t => t.ActorURI == owner.ActorURI);
if (tags.Count > 0)
await DB.Default.SaveAsync(tags, token);
}
static (string Normal, string Display) Named(string name)
{
var display = name?.Trim().TrimStart('#');
var normal = TagsController.Normalise(display);
return TagsController.IsHashtag(normal) ? (normal, display.Length <= 100 ? display : normal) : (default, default);
}
static string Link(string href) => Uri.TryCreate(href, UriKind.Absolute, out var uri) && uri.Scheme is "https" or "http" ? href : default;
// the collection's items, from it or its first page (its own server only), and the pages after, as Mastodon reads them
async Task<List<JsonNode>> Items(ForeignAvatar owner, CancellationToken token)
{
using var scope = HttpScope.For("collection");
var collection = await Fetch(owner.FeaturedTagsURL, token);
if (collection == default)
return default;
var items = new List<JsonNode>();
var page = collection["first"] is JsonObject embedded ? embedded
: Id(collection["first"]) is { } first && Origin.Same(first, owner.ActorURI) ? await Fetch(first, token)
: collection;
for (var pages = 0; page != default && pages < 3 && items.Count < MaxRemote; pages++)
{
if ((page["orderedItems"] ?? page["items"]) is JsonArray list)
items.AddRange(list.Where(i => i is JsonObject));
page = Id(page["next"]) is { } next && Origin.Same(next, owner.ActorURI) ? await Fetch(next, token) : default;
}
return items;
}
async Task<JsonObject> Fetch(string uri, CancellationToken token)
{
using var fetched = await _remoteActors.FetchObject(uri, token);
return fetched == default || fetched.Root.ValueKind != JsonValueKind.Object ? default : JsonNode.Parse(fetched.Root.GetRawText()) as JsonObject;
}
}
}
+31 -2
View File
@@ -26,7 +26,10 @@ namespace PrivaPub.Domain.Social
Task<WebResult> Follow(string rootUserId, FollowForm form, CancellationToken token);
Task<WebResult> Unfollow(string rootUserId, FollowForm form, CancellationToken token);
Task<WebResult> Followings(string rootUserId, string avatarId, CancellationToken token);
Task<Following> FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token);
/// <summary>Follows an account, or changes how a follow already made shows it: notify and languages (R8) change only
/// when given (languages empty: every language).</summary>
Task<Following> FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token, bool? notify = default,
IReadOnlyList<string> languages = default);
Task UnfollowAs(LocalActor follower, string target, CancellationToken token);
Task<bool> Decide(LocalActor me, string followerAccountId, bool accept, CancellationToken token);
Task<int> ResendPending(DateTime now, CancellationToken token);
@@ -93,7 +96,18 @@ namespace PrivaPub.Domain.Social
}
}
public async Task<Following> FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token)
// a follow's languages as kept: their primary subtags, lower case; none, every language (null)
public static List<string> Languages(IReadOnlyList<string> languages)
{
var kept = (languages ?? Array.Empty<string>())
.Select(l => l?.Trim().Replace('_', '-').Split('-')[0].ToLowerInvariant())
.Where(l => l is { Length: >= 2 and <= 3 } && l.All(char.IsAsciiLetterLower))
.Distinct().Take(20).ToList();
return kept.Count == 0 ? null : kept;
}
public async Task<Following> FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token, bool? notify = default,
IReadOnlyList<string> languages = default)
{
var (local, remote) = await ResolveTarget(target, token);
if (local == default && remote == default)
@@ -108,6 +122,17 @@ namespace PrivaPub.Domain.Social
if (existing.ShowReblogs != showReblogs)
await DB.Default.Update<Following>().MatchID(existing.ID).Modify(f => f.ShowReblogs, showReblogs).ExecuteAsync(token);
existing.ShowReblogs = showReblogs;
if (notify is { } notifying && existing.Notify != notifying)
{
await DB.Default.Update<Following>().MatchID(existing.ID).Modify(f => f.Notify, notifying).ExecuteAsync(token);
existing.Notify = notifying;
}
if (languages != default)
{
var kept = Languages(languages);
await DB.Default.Update<Following>().MatchID(existing.ID).Modify(f => f.Languages, kept).ExecuteAsync(token);
existing.Languages = kept;
}
// a request still unanswered is sent again, at most once an hour: a server can take a Follow (202) and
// drop it later, as one that cannot yet read our actor does; one that holds it already ignores the copy
if (existing.State == FollowState.Requested && remote != default)
@@ -125,6 +150,8 @@ namespace PrivaPub.Domain.Social
TargetIsLocal = local != default,
TargetInboxURL = local?.Inbox ?? remote.InboxURL,
ShowReblogs = showReblogs,
Notify = notify == true,
Languages = Languages(languages),
State = local is { ManuallyApprovesFollowers: false } ? FollowState.Accepted : FollowState.Requested
};
following.ID = (string)following.GenerateNewID();
@@ -157,6 +184,8 @@ namespace PrivaPub.Domain.Social
await DB.Default.DeleteAsync<Following>(following.ID);
await DB.Default.DeleteAsync<PersonaListMember>(m => m.AvatarId == follower.Id && m.AccountId == following.TargetAccountId);
// an endorsement holds only while the follow does, as on Mastodon (R8)
await DB.Default.DeleteAsync<Endorsement>(e => e.AvatarId == follower.Id && e.TargetActorURI == targetUri);
if (following.TargetIsLocal)
{
await DB.Default.DeleteAsync<Follower>(f => f.LocalActorId == following.TargetAccountId && f.ActorURI == follower.Uri);
+149
View File
@@ -0,0 +1,149 @@
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PostEntity = PrivaPub.Models.Post.Post;
namespace PrivaPub.Domain.Social
{
// Follow suggestions (completion program R8, Mastodon's /api/v1 and /api/v2/suggestions), best first:
// - staff: the accounts the administrators pick (ServerSettings.StaffPicks), Mastodon's `featured`;
// - past interactions: accounts the persona favourited, boosted or answered and does not follow;
// - global: accounts followed by at least GlobalMinimumFollowers personas here (`most_followed`), and personas here whose
// public posts of the last 30 days were favourited and boosted the most (`most_interactions`); discoverable ones only.
// Never the persona itself, never a sibling persona of its login (suggesting one would pair them the moment the persona
// followed it; the server knows the pair, the persona's suggestions must not use it), never an account it follows, asked
// to follow, blocked, muted, was blocked by or dismissed, and never an account held back (suspended, a banned login's) or
// silenced (the API mapper shows it `limited`). Nothing here is shown to anyone but the persona asking.
public static class Suggestions
{
public const string Staff = "staff";
public const string PastInteractions = "past_interactions";
public const string Global = "global";
public const int GlobalMinimumFollowers = 3;
const int Recent = 200;
const int GlobalSize = 100;
static readonly TimeSpan GlobalLifetime = TimeSpan.FromMinutes(10);
static (DateTime At, List<(string Id, string Why)> Ids) _global = (DateTime.MinValue, new());
public sealed record Candidate(string AccountId, string Source, List<string> Sources);
// every candidate for the persona, in order, each once (with every reason it has), none of those it must never see
public static async Task<List<Candidate>> For(LocalActor me, IReadOnlyList<string> staffPicks, CancellationToken token)
{
var excluded = await Excluded(me, token);
var found = new List<Candidate>();
var byId = new Dictionary<string, Candidate>(StringComparer.Ordinal);
void Add(string id, string source, string why)
{
if (string.IsNullOrEmpty(id) || excluded.Contains(id))
return;
if (byId.TryGetValue(id, out var known))
{
if (!known.Sources.Contains(why))
known.Sources.Add(why);
return;
}
var candidate = new Candidate(id, source, new List<string> { why });
byId[id] = candidate;
found.Add(candidate);
}
foreach (var id in staffPicks ?? Array.Empty<string>())
Add(id, Staff, "featured");
foreach (var id in await Interacted(me.Id, token))
Add(id, PastInteractions, PastInteractions);//(Mastodon's sources has no word for it since 4.3; its source does)
foreach (var (id, why) in await GlobalCandidates(token))
Add(id, Global, why);
return found;
}
static async Task<HashSet<string>> Excluded(LocalActor me, CancellationToken token)
{
var excluded = new HashSet<string>(StringComparer.Ordinal) { me.Id };
var roots = (await DB.Default.Find<RootToAvatar>().Match(r => r.AvatarId == me.Id).ExecuteAsync(token)).Select(r => r.RootId).ToList();
if (roots.Count > 0)
excluded.UnionWith((await DB.Default.Find<RootToAvatar>().Match(r => roots.Contains(r.RootId)).ExecuteAsync(token)).Select(r => r.AvatarId));
excluded.UnionWith((await DB.Default.Find<Following>().Match(f => f.AvatarId == me.Id).Project(f => f.Include(x => x.TargetAccountId))
.ExecuteAsync(token)).Select(f => f.TargetAccountId));
excluded.UnionWith((await DB.Default.Find<Block>().Match(b => b.AvatarId == me.Id).ExecuteAsync(token)).Select(b => b.TargetAccountId));
// (a persona here that blocked this one)
excluded.UnionWith((await DB.Default.Find<Block>().Match(b => b.TargetActorURI == me.Uri).ExecuteAsync(token)).Select(b => b.AvatarId));
excluded.UnionWith((await DB.Default.Find<BlockedBy>().Match(b => b.AvatarId == me.Id).ExecuteAsync(token)).Select(b => b.AccountId));
excluded.UnionWith((await DB.Default.Find<Mute>().Match(m => m.AvatarId == me.Id).ExecuteAsync(token)).Select(m => m.TargetAccountId));
excluded.UnionWith((await DB.Default.Find<SuggestionDismissal>().Match(d => d.AvatarId == me.Id).ExecuteAsync(token)).Select(d => d.AccountId));
excluded.UnionWith(await Counted.HeldBackPersonas(token));
excluded.Remove(default);
return excluded;
}
// the accounts whose posts the persona favourited, boosted or answered lately, the most often first
static async Task<List<string>> Interacted(string avatarId, CancellationToken token)
{
var authors = new List<(string Id, int Age)>();
var liked = (await DB.Default.Find<Favourite>().Match(f => f.AccountId == avatarId).Sort(f => f.ID, Order.Descending).Limit(Recent)
.ExecuteAsync(token)).Select(f => f.PostId).ToList();
var boosted = (await DB.Default.Find<PostEntity>().Match(p => p.AuthorAccountId == avatarId && p.ReblogOfPostId != null && !p.DeletedAt.HasValue)
.Sort(p => p.ID, Order.Descending).Limit(Recent).Project(p => p.Include(x => x.ReblogOfPostId)).ExecuteAsync(token)).Select(p => p.ReblogOfPostId).ToList();
var interacted = liked.Concat(boosted).Distinct().ToList();
var posts = interacted.Count == 0 ? new Dictionary<string, PostEntity>()
: (await DB.Default.Find<PostEntity>().Match(p => interacted.Contains(p.ID))
.Project(p => p.Include(x => x.AuthorAccountId).Include(x => x.GroupUserId).Include(x => x.ReblogOfPostId)).ExecuteAsync(token))
.ToDictionary(p => p.ID);
authors.AddRange(interacted.Select((id, age) => (posts.GetValueOrDefault(id) is { ReblogOfPostId: null } post ? post.AuthorAccountId ?? post.GroupUserId : default, age)));
var answered = await DB.Default.Find<PostEntity>().Match(p => p.GroupUserId == avatarId && !p.IsFederatedCopy && p.InReplyToAccountId != null && !p.DeletedAt.HasValue)
.Sort(p => p.ID, Order.Descending).Limit(Recent).Project(p => p.Include(x => x.InReplyToAccountId)).ExecuteAsync(token);
authors.AddRange(answered.Select((p, age) => (p.InReplyToAccountId, age)));
return authors.Where(a => !string.IsNullOrEmpty(a.Id) && a.Id != avatarId)
.GroupBy(a => a.Id).OrderByDescending(g => g.Count()).ThenBy(g => g.Min(a => a.Age)).Select(g => g.Key).ToList();
}
// the same for every persona, worked out at most every ten minutes
static async Task<List<(string Id, string Why)>> GlobalCandidates(CancellationToken token)
{
var cached = _global;
if (DateTime.UtcNow - cached.At < GlobalLifetime)
return cached.Ids;
var ids = new List<(string, string)>();
var followed = await DB.Default.Fluent<Following>()
.Match(f => f.State == FollowState.Accepted && f.FollowerKind == null)
.Group(f => f.TargetAccountId, g => new { Id = g.Key, Count = g.Count() })
.Match(g => g.Count >= GlobalMinimumFollowers)
.SortByDescending(g => g.Count)
.Limit(GlobalSize)
.ToListAsync(token);
var followedIds = followed.Select(f => f.Id).ToList();
var discoverable = (await DB.Default.Find<Avatar>().Match(a => followedIds.Contains(a.ID) && a.Settings.IsDiscoverable && !a.DeletionAt.HasValue
&& !a.SilencedAt.HasValue && !a.SuspendedAt.HasValue).Project(a => a.Include(x => x.ID)).ExecuteAsync(token)).Select(a => a.ID)
.Concat((await DB.Default.Find<ForeignAvatar>().Match(f => followedIds.Contains(f.ID) && f.IsDiscoverable && !f.DeletionAt.HasValue
&& !f.SilencedAt.HasValue && !f.SuspendedAt.HasValue
&& (f.AvatarType == AvatarType.Person || f.AvatarType == AvatarType.Service || f.AvatarType == AvatarType.Organization))
.Project(f => f.Include(x => x.ID)).ExecuteAsync(token)).Select(f => f.ID))
.ToHashSet(StringComparer.Ordinal);
ids.AddRange(followed.Where(f => discoverable.Contains(f.Id)).Select(f => (f.Id, "most_followed")));
var since = DateTime.UtcNow.AddDays(-30);
var interactions = await DB.Default.Fluent<PostEntity>()
.Match(p => !p.IsFederatedCopy && p.GroupUserId != null && p.Visibility == PostVisibility.Public && p.ReblogOfPostId == null
&& !p.DeletedAt.HasValue && !p.AuthorLimited && !p.AuthorSuspended && p.CreationDate >= since)
.Group(p => p.GroupUserId, g => new { Id = g.Key, Count = g.Sum(p => p.FavouritesCount + p.ReblogsCount) })
.Match(g => g.Count > 0)
.SortByDescending(g => g.Count)
.Limit(GlobalSize)
.ToListAsync(token);
var active = interactions.Select(i => i.Id).ToList();
var shown = (await DB.Default.Find<Avatar>().Match(a => active.Contains(a.ID) && a.Settings.IsDiscoverable && !a.DeletionAt.HasValue
&& !a.SilencedAt.HasValue && !a.SuspendedAt.HasValue).Project(a => a.Include(x => x.ID)).ExecuteAsync(token)).Select(a => a.ID).ToHashSet();
ids.AddRange(interactions.Where(i => shown.Contains(i.Id)).Select(i => (i.Id, "most_interactions")));
_global = (DateTime.UtcNow, ids);
return ids;
}
}
}
@@ -285,6 +285,8 @@ namespace PrivaPub.Domain.Statuses
if (parent != default && Counted.Reply(post))
await DB.Default.Update<PostEntity>().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token);
await _fanout.Distribute(post, token);
if (post.Visibility == PostVisibility.Direct)
await Fanout.Conversation(post.ConversationId, token);//the `direct` streams (R8), now that it is saved
await _polls.Scheduled(post, token);
if (post.QuoteState == QuoteState.Accepted)
await DB.Default.Update<PostEntity>().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token);
+34
View File
@@ -35,6 +35,8 @@ namespace PrivaPub.Domain.Timelines
var recipients = new HashSet<string>(StringComparer.Ordinal);
if (!post.IsFederatedCopy && !string.IsNullOrEmpty(post.GroupUserId))
recipients.Add(post.GroupUserId);
// those following the author with notify (R8), told of the post once it reached their home
var notifying = new List<string>();
switch (post.Visibility)
{
@@ -44,7 +46,11 @@ namespace PrivaPub.Domain.Timelines
: await _dbEntities.Followings.Match(f => f.TargetAccountId == post.GroupUserId && f.TargetIsLocal && f.State == FollowState.Accepted).ExecuteAsync(token);
foreach (var following in followers)
if (await Shows(following, post, token))
{
recipients.Add(following.AvatarId);
if (following.Notify && Notifies(post))
notifying.Add(following.AvatarId);
}
if (!string.IsNullOrEmpty(post.AudienceURI))
// (the server's reader of feeds follows communities for nobody's home)
foreach (var member in await _dbEntities.Followings.Match(f => f.TargetActorURI == post.AudienceURI && f.State == FollowState.Accepted && f.FollowerKind == null).ExecuteAsync(token))
@@ -100,6 +106,8 @@ namespace PrivaPub.Domain.Timelines
if (string.IsNullOrEmpty(post.ReblogOfPostId))
foreach (var mention in post.Mentions.Where(m => m.IsLocal && !string.IsNullOrEmpty(m.AccountId)))
await Notifications.Add(mention.AccountId, NotificationType.Mention, post.AuthorAccountId ?? post.GroupUserId, post.ActorURI, post.ID, token);
foreach (var avatarId in notifying.Where(recipients.Contains).Distinct())
await Notifications.Add(avatarId, NotificationType.Status, post.AuthorAccountId ?? post.GroupUserId, post.ActorURI, post.ID, token);
if (Streams.Anyone)
await Stream(post, recipients, token);
}
@@ -186,10 +194,36 @@ namespace PrivaPub.Domain.Timelines
}
}
// Mastodon's notify: the account's own posts and its replies to itself, never its boosts (FeedInsertWorker)
static bool Notifies(Post post) =>
string.IsNullOrEmpty(post.ReblogOfPostId) && post.Visibility is PostVisibility.Public or PostVisibility.Unlisted or PostVisibility.FollowersOnly
&& (string.IsNullOrEmpty(post.InReplyToURI) || post.InReplyToAccountId == (post.AuthorAccountId ?? post.GroupUserId));
// a follow's languages (R8): a post in another language stays out of that home, as on Mastodon; one whose language
// is not known comes in
static bool InLanguages(Following following, Post post) =>
following.Languages is not { Count: > 0 } || string.IsNullOrEmpty(post.Language)
|| following.Languages.Contains(post.Language.Replace('_', '-').Split('-')[0].ToLowerInvariant());
// a direct conversation that has a new post, to the `direct` streams of the personas taking part in it (R8), as
// Mastodon's `conversation` event: each connection maps it for its own persona when it sends it
public static async Task Conversation(string conversationId, CancellationToken token)
{
if (!Streams.Anyone || string.IsNullOrEmpty(conversationId))
return;
var conversation = await DB.Default.Find<DmGroup>().MatchID(conversationId).ExecuteFirstAsync(token);
var told = new Streams.Event("conversation", conversationId);
foreach (var member in conversation?.Members.Where(m => !m.IsForeign).Select(m => m.AvatarId).Distinct() ?? Enumerable.Empty<string>())
if (Streams.Listening(member))
Streams.ToPersona(member, new Streams.Key(Streams.Direct), told);
}
async Task<bool> Shows(Following following, Post post, CancellationToken token)
{
if (!string.IsNullOrEmpty(post.ReblogOfPostId))
return following.ShowReblogs;
if (!InLanguages(following, post))
return false;
var repliedTo = post.InReplyToAccountId;
var author = post.AuthorAccountId ?? post.GroupUserId;
if (string.IsNullOrEmpty(post.InReplyToURI) || repliedTo == author || repliedTo == following.AvatarId)
@@ -13,6 +13,7 @@ namespace PrivaPub.Federation.Actors
// followers, following and outbox collections), and the posts it features. Read when the account is fetched, at most
// once a day, never when someone looks at it; a hidden or missing collection leaves its count unknown. Mastodon and
// GoToSocial do the same. A feed's following is its communities: they are kept, and read in turn, so its list shows.
// Its featured hashtags (its featuredTags collection, R8) are read with them.
public class AccountCountsJob : IJobHandler
{
readonly IRemoteActorService _remoteActors;
@@ -20,12 +21,14 @@ namespace PrivaPub.Federation.Actors
readonly IFeaturedPosts _featured;
readonly ILocalActorService _localActors;
readonly IJobQueue _jobs;
readonly Domain.Social.IFeaturedTags _featuredTags;
public const int MaxFeedCommunities = 50;
public AccountCountsJob(IRemoteActorService remoteActors, DbEntities dbEntities, IFeaturedPosts featured = default,
ILocalActorService localActors = default, IJobQueue jobs = default)
ILocalActorService localActors = default, IJobQueue jobs = default, Domain.Social.IFeaturedTags featuredTags = default)
{
_featuredTags = featuredTags;
_remoteActors = remoteActors;
_dbEntities = dbEntities;
_featured = featured;
@@ -66,6 +69,8 @@ namespace PrivaPub.Federation.Actors
}
if (_featured != default)
await _featured.Sync(actor, token);
if (_featuredTags != default)
await _featuredTags.Sync(actor, token);
return JobOutcome.Done;
}
@@ -24,6 +24,7 @@ namespace PrivaPub.Federation.Actors
public string Followers { get; init; }
public string Following { get; init; }
public string Featured { get; init; }
public string FeaturedTags { get; init; }
public string Wall { get; init; }
public string WebFinger { get; init; }//FEP-2c59: its handle, user@domain, where the domain is not its host's
public string SharedInbox { get; init; }
@@ -82,6 +83,7 @@ namespace PrivaPub.Federation.Actors
Followers = RemoteActorService.Text(root, "followers"),
Following = RemoteActorService.Text(root, "following"),
Featured = RemoteActorService.Text(root, "featured"),
FeaturedTags = RemoteActorService.Text(root, "featuredTags"),
Wall = RemoteActorService.Text(root, "wall") ?? RemoteActorService.Text(root, "sm:wall"),
WebFinger = RemoteActorService.Text(root, "webfinger")?.Replace("acct:", "", StringComparison.OrdinalIgnoreCase).TrimStart('@'),
SharedInbox = root.TryGetProperty("endpoints", out var endpoints) ? RemoteActorService.Text(endpoints, "sharedInbox") : default,
@@ -228,6 +228,7 @@ namespace PrivaPub.Federation.Actors
.Modify(a => a.FollowersURL, actor.Followers)
.Modify(a => a.FollowingURL, actor.Following)
.Modify(a => a.FeaturedURL, Origin.Same(actor.Featured, actor.Id) ? actor.Featured : default)
.Modify(a => a.FeaturedTagsURL, Origin.Same(actor.FeaturedTags, actor.Id) ? actor.FeaturedTags : default)
.Modify(a => a.WallURL, Origin.Same(actor.Wall, actor.Id) ? actor.Wall : default)
.Modify(a => a.AssertionKeys, actor.AssertionKeys)
.Modify(a => a.SharedInboxURL, actor.SharedInbox)
@@ -244,13 +244,21 @@ namespace PrivaPub.Federation.Controllers
return Activity(ActivityPubRenderer.OrderedCollection(local.Featured, notes.Count, notes));
}
// the hashtags a persona features (R8), as Hashtag objects in the order it featured them, as Mastodon's featured tags
// collection lists them (Mastodon reads Collection and OrderedCollection alike); a group features none
[HttpGet, Route("{actor}/tattoos")]
public async Task<IActionResult> FeaturedTags(string actor, CancellationToken token)
{
var local = await _localActors.FindByUserName(actor, token);
return local is not { IsFederated: true }
? NotFound()
: Activity(ActivityPubRenderer.OrderedCollection(local.FeaturedTags, 0, Enumerable.Empty<JsonNode>()));
if (local is not { IsFederated: true })
return NotFound();
var tags = local.Kind == LocalActorKind.Person
? await DB.Default.Find<FeaturedTag>().Match(t => t.AvatarId == local.Id).Sort(t => t.ID, Order.Ascending).ExecuteAsync(token)
: new List<FeaturedTag>();
var collection = ActivityPubRenderer.OrderedCollection(local.FeaturedTags, tags.Count,
tags.Select(t => (JsonNode)Domain.Social.FeaturedTags.Hashtag(local.BaseAddress, t)));
collection["@context"] = ActivityPubRenderer.Context();//(Hashtag is Mastodon's term, defined there)
return Activity(collection);
}
[HttpGet, Route("{actor}/scribbles/{postId}")]
@@ -256,7 +256,10 @@ namespace PrivaPub.Federation.Inbox.Handlers
await _fanout.Distribute(post, token);
await ReplyRelay.Pass(post, _dbEntities, _localActors, _delivery, token);
if (conversation != default)
{
await Domain.Statuses.ConversationStates.Posted(conversation.ID, post.ID, default, token);
await Domain.Timelines.Fanout.Conversation(conversation.ID, token);//the `direct` streams (R8), once it is the newest
}
if (group is { IsCircle: false })
await _groups.Announce(group, activity.AsObject(), note.Id, isNewPost: true, token);
if (wall != default)
@@ -6,20 +6,23 @@ using System.Text.Json.Nodes;
namespace PrivaPub.Federation.Inbox.Handlers
{
// An account pins one of its posts (Add to its `featured` collection) or unpins it (Remove), as Mastodon and Akkoma
// send it; inside a community's announce, the community features a post made in it. On its wall (Smithereen's, FEP-400e)
// it tells of a post someone wrote there, or takes it off. Any other target (a community's moderators) is not ours to
// keep.
// send it; inside a community's announce, the community features a post made in it. A Hashtag added to or removed from
// its `featured` collection (as Mastodon sends it) or its `featuredTags` is a featured tag (R8). On its wall
// (Smithereen's, FEP-400e) it tells of a post someone wrote there, or takes it off. Any other target (a community's
// moderators) is not ours to keep.
public class FeaturedHandler : IActivityHandler
{
readonly IFeaturedPosts _featured;
readonly IWalls _walls;
readonly IRemoteActorService _remoteActors;
readonly Infrastructure.Jobs.IJobQueue _queue;
readonly Domain.Social.IFeaturedTags _tags;
readonly bool _add;
public FeaturedHandler(IFeaturedPosts featured, bool add, IWalls walls = default, IRemoteActorService remoteActors = default,
Infrastructure.Jobs.IJobQueue queue = default)
Infrastructure.Jobs.IJobQueue queue = default, Domain.Social.IFeaturedTags tags = default)
{
_tags = tags;
_featured = featured;
_walls = walls;
_remoteActors = remoteActors;
@@ -36,9 +39,18 @@ namespace PrivaPub.Federation.Inbox.Handlers
var target = Objects.ActivityJson.Id(activity["target"]);
// a collection of its own we do not know of: its document read again, at most hourly, in case it was kept before
// we read that collection (a wall)
if (_remoteActors != default && target != default && target != actor.FeaturedURL && target != actor.WallURL
if (_remoteActors != default && target != default && target != actor.FeaturedURL && target != actor.WallURL && target != actor.FeaturedTagsURL
&& Objects.Origin.Same(target, actor.ActorURI) && actor.UpdatedAt < DateTime.UtcNow.AddHours(-1))
actor = await _remoteActors.GetActor(actor.ActorURI, refresh: true, token) ?? actor;
// a featured hashtag: an object of its own, with no id (its href is no post), so before anything reads it as one
if (Objects.ActivityJson.Value(activity["object"], "type") == "Hashtag")
{
if (_tags != default && target != default && (target == actor.FeaturedURL || target == actor.FeaturedTagsURL))
await _tags.Receive(activity, actor, _add, token);
else
Arrival.Drop("unknown-target");
return;
}
if (_walls != default && !string.IsNullOrEmpty(actor.WallURL) && target == actor.WallURL)
await _walls.Receive(activity, actor, _add, token);
else
@@ -73,10 +73,19 @@ namespace PrivaPub.Federation.Inbox.Handlers
if (persona == default)
continue;
var lists = await DB.Default.Find<PersonaListMember>().Match(m => m.AvatarId == persona.Id && m.AccountId == old.ID).ExecuteAsync(token);
var followed = await _follows.FollowAs(persona, moved.ActorURI, following.ShowReblogs, token);
var endorsed = await DB.Default.Find<Endorsement>().Match(e => e.AvatarId == persona.Id && e.TargetActorURI == old.ActorURI).ExecuteAnyAsync(token);
var followed = await _follows.FollowAs(persona, moved.ActorURI, following.ShowReblogs, token, following.Notify, following.Languages ?? new List<string>());
await _follows.UnfollowAs(persona, old.ActorURI, token);
if (followed == default)
continue;
// the persona's endorsement and private note go with the follow, as Mastodon carries them over (R8)
if (endorsed)
await DB.Default.Update<Endorsement>().Match(e => e.AvatarId == persona.Id && e.TargetAccountId == moved.ID)
.Modify(e => e.TargetActorURI, moved.ActorURI).Modify(b => b.SetOnInsert(e => e.CreatedAt, DateTime.UtcNow))
.Option(o => o.IsUpsert = true).ExecuteAsync(token);
if (await DB.Default.Find<PersonaNote>().Match(n => n.AvatarId == persona.Id && n.TargetAccountId == old.ID).ExecuteFirstAsync(token) is { } note
&& !await DB.Default.Find<PersonaNote>().Match(n => n.AvatarId == persona.Id && n.TargetAccountId == moved.ID).ExecuteAnyAsync(token))
await DB.Default.SaveAsync(new PersonaNote { AvatarId = persona.Id, TargetAccountId = moved.ID, Text = note.Text }, token);
foreach (var member in lists)
try
{
@@ -102,7 +102,8 @@ namespace PrivaPub.Infrastructure.Backup
report.PersonasDeleted++;
await avatars.ReplaceOneAsync(Id(avatar["_id"]), avatar, cancellationToken: token);
// what RootRemoval takes with a persona, which the backup still had
foreach (var collection in new[] { "PersonaListMember", "PersonaList", "PersonaFilter", "FollowedTag", "ScheduledStatus" })
foreach (var collection in new[] { "PersonaListMember", "PersonaList", "PersonaFilter", "FollowedTag", "ScheduledStatus",
"FeaturedTag", "Endorsement", "PersonaNote", "SuggestionDismissal" })
await database.GetCollection<BsonDocument>(collection).DeleteManyAsync(new BsonDocument("AvatarId", avatar["_id"].ToString()), token);
continue;
}
+14
View File
@@ -59,6 +59,20 @@ namespace PrivaPub.Infrastructure.Data
await Plain<AnnouncementReaction>(token, r => r.AvatarId);
// the server's custom emoji, one per shortcode whatever its case (a new collection, R7)
await Unique<LocalEmoji>(e => e.Key, Builders<LocalEmoji>.Filter.Gt(e => e.Key, ""), token);
// what the Mastodon API keeps per persona (new collections, R8): one featured hashtag per persona and name, one per
// account elsewhere and name, one endorsement, note and dismissal per persona and account
await DB.Default.Index<FeaturedTag>().Key(t => t.AvatarId, KeyType.Ascending).Key(t => t.Name, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
await DB.Default.Index<RemoteFeaturedTag>().Key(t => t.ActorURI, KeyType.Ascending).Key(t => t.Name, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
await DB.Default.Index<Endorsement>().Key(e => e.AvatarId, KeyType.Ascending).Key(e => e.TargetAccountId, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
await Plain<Endorsement>(token, e => e.TargetAccountId);
await DB.Default.Index<PersonaNote>().Key(n => n.AvatarId, KeyType.Ascending).Key(n => n.TargetAccountId, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
await Plain<PersonaNote>(token, n => n.TargetAccountId);
await DB.Default.Index<SuggestionDismissal>().Key(d => d.AvatarId, KeyType.Ascending).Key(d => d.AccountId, KeyType.Ascending)
.Option(o => o.Unique = true).CreateAsync(token);
// the staff's log, read newest first
await DB.Default.Index<Models.AdminAction>().Key(a => a.At, KeyType.Descending).CreateAsync(token);
@@ -94,12 +94,13 @@ namespace PrivaPub.Middleware
.AddSingleton<IActivityHandler, LockHandler>()
.AddSingleton<Federation.Actors.IFeaturedPosts, Federation.Actors.FeaturedPosts>()
.AddSingleton<Federation.Actors.IWalls, Federation.Actors.Walls>()
.AddSingleton<Domain.Social.IFeaturedTags, Domain.Social.FeaturedTags>()
.AddSingleton<IActivityHandler>(services => new FeaturedHandler(services.GetRequiredService<Federation.Actors.IFeaturedPosts>(), add: true,
services.GetRequiredService<Federation.Actors.IWalls>(), services.GetRequiredService<Federation.Actors.IRemoteActorService>(),
services.GetRequiredService<IJobQueue>()))
services.GetRequiredService<IJobQueue>(), services.GetRequiredService<Domain.Social.IFeaturedTags>()))
.AddSingleton<IActivityHandler>(services => new FeaturedHandler(services.GetRequiredService<Federation.Actors.IFeaturedPosts>(), add: false,
services.GetRequiredService<Federation.Actors.IWalls>(), services.GetRequiredService<Federation.Actors.IRemoteActorService>(),
services.GetRequiredService<IJobQueue>()))
services.GetRequiredService<IJobQueue>(), services.GetRequiredService<Domain.Social.IFeaturedTags>()))
.AddSingleton<IActivityHandler, MoveHandler>()
.AddSingleton<IActivityHandler, CreateHandler>()
.AddSingleton<IActivityHandler, DeleteHandler>()
+3
View File
@@ -64,6 +64,9 @@ namespace PrivaPub.Models
public const string EmojiInsert = "emoji.insert";
public const string EmojiUpdate = "emoji.update";
public const string EmojiDelete = "emoji.delete";
// the accounts suggested to every persona (completion program R8)
public const string StaffPickInsert = "staff_pick.insert";
public const string StaffPickDelete = "staff_pick.delete";
}
public static class AdminTargets
+1 -1
View File
@@ -33,7 +33,7 @@ namespace PrivaPub.Models
public List<ServerRule> Rules { get; set; } = new();//soft-deleted ones stay, so old reports keep their meaning
public List<string> Relays { get; set; } = new();//besides Federation:Relays
public ServerMediaLimits Media { get; set; } = new();
public List<string> StaffPicks { get; set; } = new();//account ids, for later
public List<string> StaffPicks { get; set; } = new();//Mastodon account ids (Avatar.ID, ForeignAvatar.ID) suggested first to every persona (R8)
public bool PublishActivity { get; set; }//GET /api/v1/instance/activity; off by default
public DateTime? UpdatedAt { get; set; }//null: never saved
public string UpdatedBy { get; set; }//the administrator's root id, or "cli"
+64
View File
@@ -0,0 +1,64 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Social
{
// What the Mastodon API keeps per persona since the completion program's R8. Every row here is one persona's, never its
// login's: siblings of one login must never be linkable, so nothing is kept per root.
// A hashtag a persona features on its profile (Mastodon's featured tags). Public: its featuredTags collection
// (/peasants/{name}/tattoos) lists it, and featuring or unfeaturing it is told to its followers as Add or Remove of a
// Hashtag. One per persona and name.
public class FeaturedTag : Entity
{
public string AvatarId { get; set; }
public string Name { get; set; }//normalised: no #, NFKC, lower case (TagsController.Normalise), as posts keep their tags
public string DisplayName { get; set; }//as the persona wrote it (it normalises to Name)
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
// A hashtag an account elsewhere features: its featuredTags collection, read with its counts at most once a day
// (AccountCountsJob), and the Add and Remove of a Hashtag it sends. Shown as its featured tags here; the lowest Position
// first.
public class RemoteFeaturedTag : Entity
{
public string ActorURI { get; set; }
public string Name { get; set; }//normalised, as FeaturedTag.Name
public string DisplayName { get; set; }
public string Href { get; set; }//the Hashtag's own link (its server's page of the account's posts with it), when it gave one
public int Position { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
// An account a persona endorses (Mastodon's account pins, 4.4's endorsements): only one it follows, and only while it
// does. Kept for the persona alone: never federated as featured accounts and never shown to anyone else (the owner's
// default: endorsements stay private).
public class Endorsement : Entity
{
public string AvatarId { get; set; }
public string TargetAccountId { get; set; }//Avatar.ID, Group.ID or ForeignAvatar.ID
public string TargetActorURI { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
// A persona's private note on an account (POST /api/v1/accounts/:id/note): its own, never its login's. RootUserNote is
// keyed by the root and would show a note made as one persona to its siblings. Never federated.
public class PersonaNote : Entity
{
public const int MaxLength = 2000;//Mastodon's AccountNote limit
public string AvatarId { get; set; }
public string TargetAccountId { get; set; }
public string Text { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
}
// An account a persona took out of its follow suggestions (DELETE /api/v1/suggestions/:account_id): never suggested to
// that persona again.
public class SuggestionDismissal : Entity
{
public string AvatarId { get; set; }
public string AccountId { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
}
}
+2
View File
@@ -13,6 +13,8 @@ namespace PrivaPub.Models.Social
public FollowState State { get; set; }
public string FollowActivityURI { get; set; }
public bool ShowReblogs { get; set; } = true;
public bool Notify { get; set; }//Mastodon's notify (R8): a `status` notification for each new post of the account
public List<string> Languages { get; set; }//Mastodon's languages (R8): only its posts in these reach the home; null, every language
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public int Resent { get; set; }//how many times a request still unanswered was sent again (FollowService.ResendPending)
public DateTime? ResentAt { get; set; }
+2 -1
View File
@@ -33,6 +33,7 @@ namespace PrivaPub.Models.Social
Poll,
Reaction,
Quote,
ModerationWarning
ModerationWarning,
Status//a new post of an account the persona follows with notify (R8)
}
}
+14
View File
@@ -78,6 +78,7 @@ namespace PrivaPub.Models.User
public string InboxURL { get; set; }
public string OutboxURL { get; set; }
public string FeaturedURL { get; set; }//featured: the posts it pins
public string FeaturedTagsURL { get; set; }//featuredTags: the hashtags it features (R8)
public List<PrivaPub.Models.Post.PostFlair> Flairs { get; set; } = new();//a community's labels for its posts
public List<string> FeedCommunities { get; set; } = new();//a feed's communities, its `following` (Lemmy's multi-communities, PieFed's feeds)
public List<string> SearchableBy { get; set; }//FEP-268d, when the account states it: outranks IsIndexable
@@ -127,6 +128,19 @@ namespace PrivaPub.Models.User
public bool DefaultSensitive { get; set; }
public string DefaultLanguage { get; set; }
public string QuotePolicy { get; set; } = QuotePolicies.Public;//owner decision: anyone may quote a public post, automatically
// how its clients show what others post (R8, Mastodon's reading:expand:media and :spoilers in GET /api/v1/preferences),
// set through /clientapi/avatar/private/preferences, since Mastodon's API sets neither
public string ExpandMedia { get; set; } = ExpandMediaModes.Default;
public bool ExpandSpoilers { get; set; }
}
public static class ExpandMediaModes
{
public const string Default = "default";//hide media marked sensitive
public const string ShowAll = "show_all";
public const string HideAll = "hide_all";
public static bool IsKnown(string value) => value is Default or ShowAll or HideAll;
}
public static class QuotePolicies
+9 -1
View File
@@ -61,7 +61,8 @@ namespace PrivaPub.Services
// personas always were: its Mastodon tokens end first, each group it owns goes (GroupRemoval), a Delete of the actor goes
// to its followers and to the accounts it follows, its posts are emptied, and its name stays reserved while its documents
// answer 410. What it kept for itself (lists, filters, followed hashtags, scheduled posts, what it did with the
// announcements, the feeds it read, its media) goes with it. Its link to its login stays, so the login keeps no other
// announcements, the feeds it read, its featured hashtags, endorsements, notes and dismissed suggestions, its media)
// goes with it. Its link to its login stays, so the login keeps no other
// trace than that.
public class PersonaRemoval : IPersonaRemoval
{
@@ -113,6 +114,13 @@ namespace PrivaPub.Services
await DB.Default.DeleteAsync<AnnouncementReaction>(r => r.AvatarId == avatar.ID);
// (the reader of feeds stops following what nobody reads any more at its next round)
await DB.Default.DeleteAsync<FeedSubscription>(s => s.AvatarId == avatar.ID);
// what the Mastodon API kept for it (R8): its featured hashtags (the Delete of the actor takes them from other
// servers), its endorsements, its private notes and its dismissed suggestions, and what other personas here kept
// about it: their endorsements of it and their notes on it
await DB.Default.DeleteAsync<FeaturedTag>(t => t.AvatarId == avatar.ID);
await DB.Default.DeleteAsync<Endorsement>(e => e.AvatarId == avatar.ID || e.TargetAccountId == avatar.ID);
await DB.Default.DeleteAsync<PersonaNote>(n => n.AvatarId == avatar.ID || n.TargetAccountId == avatar.ID);
await DB.Default.DeleteAsync<SuggestionDismissal>(d => d.AvatarId == avatar.ID || d.AccountId == avatar.ID);
// nothing of it publishes later, and none of its files stays served: its posts' media, its pictures and what its
// scheduled posts held
await DB.Default.DeleteAsync<ScheduledStatus>(s => s.AvatarId == avatar.ID);
+18 -1
View File
@@ -138,6 +138,22 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
410; what posts and profiles render is the emoji as they are now, and no `Update` is sent when one changes; its id is
served unsigned under SecureMode; a restore keeps the live emoji, as it keeps the settings (one deleted since stays
deleted).
- R8 the Mastodon API made real (built 2026-10-18, not yet deployed; `tools/pasture/scenarios/featured.sh`): featured
tags (`/api/v1/featured_tags`, suggestions, `tags/:name/feature`, accounts' featured tags, `max_featured_tags` 10;
`/tattoos` a collection of `Hashtag`s, `Add`/`Remove{Hashtag}` sent to followers and received, remote accounts'
`featuredTags` read with their counts); endorsements (`pin`/`unpin`, `endorse`/`unendorse`, `/api/v1/endorsements`);
private notes per persona (`PersonaNote`); the relationship's `notifying`, `languages`, `endorsed` and `note`, with
`follow`'s `notify` (a `status` notification) and `languages` (filtering the home); preferences' reading settings;
suggestions v1 and v2 with dismissals and the administrators' staff picks; the `direct` stream's `conversation`
events; translate, annual reports and the admin API answering. Chosen while building it, for the owner to confirm:
**featured tags federate** (they are public on Mastodon; `Add`/`Remove{Hashtag}` target `featured`, as Mastodon
sends them, rather than `featuredTags`, which Mastodon ignores); endorsements are never federated and
`/accounts/:id/endorsements` answers only the persona's own, to itself; suggestions never name a sibling persona and
count `most_followed` only from three followers here; `familiar_followers` stays empty; the reading preferences are
set on `/clientapi/avatar/private/preferences` (Mastodon's API sets none); staff picks have their own routes, not the
settings form; a `Move` carries the persona's endorsement and note to the new account; featured tags, notes and
endorsements are left out of the persona archive (Mastodon's layout has none), its `following_accounts.csv` now
carries notify and languages.
## Intent
@@ -601,7 +617,8 @@ The first refactor commit is a pure move with namespaces only. Logic changes fol
`hashtag:local`, `list`; events `update`, `notification`, `status.update`, `delete`; `health`. Advertised as
`urls.streaming`.
- **Stubs:** `custom_emojis`, announcements, suggestions, preferences. *Announcements are real since R5,
`custom_emojis` since R7 (2026-10-17).*
`custom_emojis` since R7 (2026-10-17), suggestions, preferences, featured tags and endorsements since R8
(2026-10-18).*
- **Mapping:**
- Avatar, ForeignAvatar and Group all map to Account (`group: true` for groups). `acct` uses a WebFinger-verified
handle; `created_at` is truncated to the day; avatar and header always have a placeholder URL.
+128
View File
@@ -0,0 +1,128 @@
# The Mastodon API made real (completion program R8), with Mastodon. mastouser follows feat_alice, who features a hashtag:
# her /tattoos serves it and Mastodon shows it among her account's featured tags (her Add{Hashtag}), and unfeaturing it
# takes it away there (Remove{Hashtag}). feat_alice follows mastouser with notify=true, and a tag he features shows among
# his featured tags here; his next post gives her a `status` notification; his direct message to her is told on her
# `direct` stream, as server-sent events and on the WebSocket, as a `conversation` event. Her endorsement of him and her
# private note on him are hers alone: feat_bob, a sibling persona of the same login, sees neither, nobody else is told her
# endorsements, and feat_bob is never suggested to her, though she favourited his post. The stubs answer: translate 422,
# annual reports empty, the Mastodon admin API 403. Needs the mastodon peer (batch A); run after emoji.
M=https://mastodon.test:6443
mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; }
. "$here/peers/mastodon.sh"
code() { curl -s -o /dev/null -w '%{http_code}' "$@"; }
# m_signed_body <url>: one of our documents as Mastodon's instance actor reads it, signed (SecureMode answers unsigned 401)
m_signed_body() { podman exec pasture-mastodon bin/rails runner "Request.new(:get, '$1').on_behalf_of(Account.representative).add_headers('Accept' => 'application/activity+json').perform { |r| puts r.body_with_limit }" 2>/dev/null | tail -1; }
m_featured() { mcurl -H "$MH" "$M/api/v1/accounts/$1/featured_tags" | j "print(any((t.get('name') or '').lower() == '$2' for t in d or []))"; }
p_featured() { curl -s -H "$AH" "$P/api/v1/accounts/$1/featured_tags" | j "print(any((t.get('name') or '').lower() == '$2' for t in d or []))"; }
echo "featured"
run=$(date +%s)
tag="pasture$run"; mtag="mastopasture$run"
AT=$(privapub_token feat_alice); AH="Authorization: Bearer $AT"
BT=$(privapub_token feat_bob); BH="Authorization: Bearer $BT"
MT=$(mastodon_token); MH="Authorization: Bearer $MT"
[ -n "$AT" ] && [ -n "$BT" ] && [ -n "$MT" ] && ok "tokens for feat_alice, feat_bob (her sibling, the same login) and mastouser" \
|| { ko "a token is missing"; return 1; }
alice_id=$(curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])")
bob_id=$(curl -s -H "$BH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])")
echo " follows"
alice_on_m=$(mcurl -H "$MH" "$M/api/v2/search?q=@feat_alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$alice_on_m/follow"
until_true 30 '[ "$(curl -s "$P/api/v1/accounts/$alice_id" | j "print(d[\"followers_count\"])")" -ge 1 ]' \
&& ok "mastouser follows feat_alice" || ko "mastouser's follow never arrived"
m_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=mastouser@mastodon.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
rel=$(curl -s -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/follow" -d notify=true -d 'languages[]=en')
[ "$(echo "$rel" | j "print(d['notifying'] is True and d['languages'] == ['en'])")" = True ] \
&& ok "feat_alice follows mastouser with notify and languages (the relationship says so)" || ko "the follow's options: $(echo "$rel" | head -c 300)"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/relationships?id[]=$m_on_p" | j "print(d[0][\"following\"])")" = True ]' \
&& ok "mastouser accepts feat_alice" || ko "feat_alice's follow was not accepted"
echo " her featured tag"
featured=$(curl -s -X POST -H "$AH" "$P/api/v1/featured_tags" -d "name=$tag")
featured_id=$(echo "$featured" | j "print(d['id'])")
[ "$(echo "$featured" | j "print(d['name'] == '$tag' and d['statuses_count'] == '0')")" = True ] && ok "feat_alice features #$tag" \
|| ko "featuring was refused: $(echo "$featured" | head -c 300)"
[ "$(curl -s -H "$AH" "$P/api/v1/featured_tags" | j "print(any(t['id'] == '$featured_id' for t in d))")" = True ] \
&& [ "$(curl -s "$P/api/v1/accounts/$alice_id/featured_tags" | j "print(any(t['name'] == '$tag' for t in d))")" = True ] \
&& ok "her featured tags list it, anonymously too" || ko "her featured tags miss it"
[ "$(curl -s "$P/api/v2/instance" | j "print(d['configuration']['accounts']['max_featured_tags'])")" = 10 ] \
&& ok "the instance API allows ten" || ko "max_featured_tags: $(curl -s "$P/api/v2/instance" | j "print(d['configuration']['accounts'])")"
[ "$(m_signed_body https://privapub.test/peasants/feat_alice/tattoos | j "print(any(i['type'] == 'Hashtag' and i['name'] == '#$tag' for i in d['orderedItems']))")" = True ] \
&& ok "/tattoos serves it as a Hashtag (signed)" || ko "/tattoos: $(m_signed_body https://privapub.test/peasants/feat_alice/tattoos | head -c 300)"
until_true 30 '[ "$(m_featured "$alice_on_m" "$tag")" = True ]' \
&& ok "Mastodon shows #$tag among feat_alice's featured tags" || ko "Mastodon's featured tags of feat_alice: $(mcurl -H "$MH" "$M/api/v1/accounts/$alice_on_m/featured_tags" | head -c 300)"
[ "$(code -X DELETE -H "$AH" "$P/api/v1/featured_tags/$featured_id")" = 200 ] && ok "feat_alice unfeatures it" || ko "unfeaturing was refused"
until_true 30 '[ "$(m_featured "$alice_on_m" "$tag")" = False ]' \
&& ok "Mastodon no longer shows it" || ko "Mastodon still shows #$tag"
echo " his featured tag"
m_featured_id=$(mcurl -X POST -H "$MH" "$M/api/v1/featured_tags" -d "name=$mtag" | j "print(d['id'])")
[ -n "$m_featured_id" ] && ok "mastouser features #$mtag" || ko "Mastodon refused the featured tag"
until_true 30 '[ "$(p_featured "$m_on_p" "$mtag")" = True ]' \
&& ok "feat_alice reads #$mtag among mastouser's featured tags" || ko "his featured tags here: $(curl -s -H "$AH" "$P/api/v1/accounts/$m_on_p/featured_tags" | head -c 300)"
mcurl -o /dev/null -X DELETE -H "$MH" "$M/api/v1/featured_tags/$m_featured_id"
until_true 30 '[ "$(p_featured "$m_on_p" "$mtag")" = False ]' \
&& ok "his unfeaturing reaches PrivaPub" || ko "#$mtag is still featured here"
echo " notify"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" --data-urlencode "status=notify me, PrivaPub $run" -d visibility=public -d language=en
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/notifications?types[]=status" | j "print(any(\"notify me, PrivaPub $run\" in n[\"status\"][\"content\"] for n in d))")" = True ]' \
&& ok "his post gives feat_alice a status notification" || ko "no status notification: $(curl -s -H "$AH" "$P/api/v1/notifications?limit=5" | head -c 300)"
echo " the direct stream"
curl -s -N -H "$AH" "$P/api/v1/streaming/direct" > "$work/sse.txt" & sse=$!
cat > "$work/ws.py" <<'PY'
import asyncio, json, sys, websockets
async def main(url, token, seconds):
async with websockets.connect(url, subprotocols=[token]) as ws:
await ws.send(json.dumps({"type": "subscribe", "stream": "direct"}))
try:
while True:
message = json.loads(await asyncio.wait_for(ws.recv(), timeout=seconds))
print(json.dumps(message), flush=True)
except asyncio.TimeoutError:
pass
asyncio.run(main(sys.argv[1], sys.argv[2], float(sys.argv[3])))
PY
python3 -I "$work/ws.py" "ws://127.0.0.1:${PASTURE_PORT:-6971}/api/v1/streaming" "$AT" 40 > "$work/ws.txt" 2>&1 & ws=$!
sleep 3
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" --data-urlencode "status=@feat_alice@privapub.test a secret, PrivaPub $run" -d visibility=direct
until_true 30 'grep -q "^event: conversation" "$work/sse.txt"' && ok "the SSE direct stream tells a conversation event" \
|| ko "the SSE direct stream: $(head -c 300 "$work/sse.txt")"
until_true 20 '[ "$(python3 -c "import json,sys
for l in open(sys.argv[1]):
m=json.loads(l)
if m.get(\"event\")==\"conversation\" and m.get(\"stream\")==[\"direct\"] and \"a secret, PrivaPub $run\" in json.loads(m[\"payload\"])[\"last_status\"][\"content\"]: print(True); break" "$work/ws.txt" 2>/dev/null)" = True ]' \
&& ok "the WebSocket's direct subscription tells it, with the message" || ko "the WebSocket: $(head -c 300 "$work/ws.txt")"
kill "$sse" "$ws" 2>/dev/null; wait "$sse" "$ws" 2>/dev/null
echo " endorsements and notes stay hers"
[ "$(curl -s -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/endorse" | j "print(d['endorsed'])")" = True ] \
&& [ "$(curl -s -H "$AH" "$P/api/v1/endorsements" | j "print(any(a['id'] == '$m_on_p' for a in d))")" = True ] \
&& ok "feat_alice endorses mastouser" || ko "the endorsement did not hold"
[ "$(curl -s -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/note" --data-urlencode "comment=met in the pasture $run" | j "print(d['note'])")" = "met in the pasture $run" ] \
&& ok "her private note on him" || ko "the note was not kept"
[ "$(curl -s -H "$BH" "$P/api/v1/accounts/relationships?id[]=$m_on_p" | j "print(d[0]['note'] == '' and d[0]['endorsed'] is False)")" = True ] \
&& [ "$(curl -s -H "$BH" "$P/api/v1/endorsements" | j "print(d == [])")" = True ] \
&& ok "feat_bob, of the same login, sees neither her note nor her endorsement" || ko "a sibling sees her note or endorsement"
[ "$(curl -s -H "$BH" "$P/api/v1/accounts/$alice_id/endorsements" | j "print(d == [])")" = True ] \
&& [ "$(curl -s "$P/api/v1/accounts/$alice_id/endorsements" | j "print(d == [])")" = True ] \
&& [ "$(curl -s -H "$AH" "$P/api/v1/accounts/$alice_id/endorsements" | j "print(any(a['id'] == '$m_on_p' for a in d))")" = True ] \
&& ok "her endorsements are told to her alone" || ko "her endorsements are told to others"
bob_post=$(curl -s -X POST -H "$BH" "$P/api/v1/statuses" --data-urlencode "status=bob's own post $run" -d visibility=public | j "print(d['id'])")
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses/$bob_post/favourite"
[ "$(curl -s -H "$AH" "$P/api/v2/suggestions?limit=80" | j "print(all(s['account']['id'] not in ('$bob_id', '$alice_id') for s in d))")" = True ] \
&& ok "feat_bob is never suggested to her, though she favourited his post" || ko "a sibling is suggested: $(curl -s -H "$AH" "$P/api/v2/suggestions" | head -c 300)"
[ "$(curl -s -H "$AH" "$P/api/v1/accounts/familiar_followers?id[]=$m_on_p" | j "print(d == [{'id': '$m_on_p', 'accounts': []}])")" = True ] \
&& ok "familiar followers stay empty" || ko "familiar followers: $(curl -s -H "$AH" "$P/api/v1/accounts/familiar_followers?id[]=$m_on_p")"
echo " stubs"
[ "$(code -X POST -H "$AH" "$P/api/v1/statuses/$bob_post/translate")" = 422 ] && ok "translate answers 422" || ko "translate answers otherwise"
[ "$(curl -s -H "$AH" "$P/api/v1/annual_reports" | j "print(d == {'annual_reports': [], 'accounts': [], 'statuses': []})")" = True ] \
&& ok "annual reports answer none" || ko "annual reports: $(curl -s -H "$AH" "$P/api/v1/annual_reports")"
[ "$(code -H "$AH" "$P/api/v1/admin/accounts")" = 403 ] && [ "$(curl -s "$P/api/v1/admin/reports" | j "print(d['error'])")" = "This action is not allowed" ] \
&& ok "the Mastodon admin API answers 403" || ko "the admin API answers $(code -H "$AH" "$P/api/v1/admin/accounts")"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/unendorse"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/note" -d comment=
+3 -2
View File
@@ -9,7 +9,8 @@
# A gts mastodon misskey sharkey akkoma, with relay, aoderelay and mailpit; scenarios: those five, threads followsync
# pins moves relay (relay's covers both relays) accounts (mailpit's) instance (the server's own settings: it changes
# them all and puts them back) moderation (gts and mastodon; after instance, whose log check expects only the
# administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation)
# administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation) featured (mastodon: featured
# tags, notify, the direct stream, endorsements and notes per persona; after emoji)
# B lemmy lemmy19 piefed mbin nodebb
# C peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale
# D friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla
@@ -29,7 +30,7 @@ batch_peers() {
}
batch_scenarios() {
case "$1" in
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji" ;;
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji featured" ;;
*) batch_peers "$1" ;;
esac
}