diff --git a/CLAUDE.md b/CLAUDE.md index 073b539..4937e80 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,8 +31,10 @@ Its defining idea: **one private login owns several public personas.** Privacy features in the model: - location-ranged posts (`Post.Location`, `RangeKm`), local-only and never federated; -- contacts a persona can share (`Avatar.SharedPersonalContacts`) and private notes (`RootUserNote`, `PersonalNote`) - exist only as model classes, with no endpoint. +- contacts a persona can share (`Avatar.SharedPersonalContacts`) and the root's notes (`RootUserNote`, `PersonalNote`) + exist only as model classes, with no endpoint. A persona's private note on an account is `PersonaNote` (R8), its own: + `RootUserNote` is keyed by the root and would show a note made as one persona to its siblings, so it is never used for + it. The intended client is the Mastodon client API (Tusky, Elk, Phanpy, Ivory), where each avatar logs in as its own account. The private `/clientapi` covers what Mastodon can't express. `thepra/decePubClient` @@ -57,7 +59,7 @@ actor-scoped route gets a name in the same spirit, agreed with the owner, and a | DM context | `/peasants/{name}/whispers/{id}` | | Quote permission (FEP-044f `QuoteAuthorization`) | `/peasants/{name}/parrot-licences/{id}` | | Featured (pinned posts) | `/peasants/{name}/trophies` | -| Featured tags | `/peasants/{name}/tattoos` | +| Featured tags (an `OrderedCollection` of `Hashtag`s, R8) | `/peasants/{name}/tattoos` | | Wall (FEP-400e) | `/peasants/{name}/graffiti` | | Followers synchronisation (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` | | A group's members and moderators | `/peasants/{name}/flock`, `/peasants/{name}/wardens` | @@ -127,7 +129,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0 Instance/ InstanceSettings (the server's own settings in memory), MediaLimitsFromSettings Moderation/ ModerationLog (the staff's log, AdminAction), AccountModeration (silence, suspend, posts removed), Restrictions (who is held back, in memory), ModerationFlags (posts' flags again) - Social/ FollowService (local in-process, remote Follow/Accept), Notifications, Announcements + Social/ FollowService (local in-process, remote Follow/Accept), Notifications, Announcements, + FeaturedTags (a persona's featured hashtags, sent and received), Suggestions (who to follow) Timelines/ Fanout (TimelineEntry rows, Mastodon's home rules), TimelineService Privacy/ VisibilityPolicy (IsPublic expression, CanSee) Relationships/ RelationshipService (blocks, mutes, account domain blocks; Hidden), ReportService @@ -143,7 +146,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0 Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, lists, filters, media, polls, reactions, reports, scheduled statuses, streaming, tags, trends, cdns, - provenance, custom emojis; the stubs are `StubsController` in `SearchController.cs` + provenance, custom emojis, featured tags, suggestions; the stubs are `StubsController` + in `SearchController.cs` Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex); OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent) Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, RootRemoval, @@ -350,6 +354,81 @@ client's role route never does. here and elsewhere, and the posts taken down, before it does; indexes on `SilencedAt` and `SuspendedAt` of both account collections and a partial one on `Post.ModeratedAt`. No new collection, so backups hold it all as they did. +## The Mastodon API made real (completion program R8) + +- **Every row is one persona's** (`Models/Social/Featuring.cs`, new collections): `FeaturedTag`, `Endorsement`, + `PersonaNote` and `SuggestionDismissal` are keyed by the persona (`AvatarId`), never by its root, with unique indexes on + (persona, name or account); `RemoteFeaturedTag` is an account elsewhere's (by `ActorURI`). `Following` gained `Notify` + and `Languages` (null: every language), `ForeignAvatar` `FeaturedTagsURL`, `AvatarSettings` `ExpandMedia` + (`default`, `show_all`, `hide_all`) and `ExpandSpoilers`, `NotificationType` `Status` (last, as enum members are stored + as ints). +- **Featured tags** (`Domain/Social/FeaturedTags.cs`, `FeaturedTagsController`, `TagsController`): `GET`/`POST + /api/v1/featured_tags` (`name`; 422 for a name that is no hashtag or past `FeaturedTags.MaxFeatured`, 10, which both + instance APIs advertise as `max_featured_tags`; one featured already is answered as it is), `DELETE + /api/v1/featured_tags/{id}` (`{}`, 404), `GET /api/v1/featured_tags/suggestions` (Mastodon's `Tag`s: the tags of the + persona's last 200 posts it does not feature, the most used first, ten), `POST /api/v1/tags/{name}/feature|unfeature` + (the `Tag`, with Mastodon 4.4's `featuring`, which every `Tag` now carries for a signed-in persona), `GET + /api/v1/accounts/{id}/featured_tags` (anonymous too: a persona's own, an account elsewhere's from `RemoteFeaturedTag`, a + community none, a suspended account 404). Mastodon's `FeaturedTag`: `statuses_count` a string and `last_status_at` a + day, counted from the account's public and unlisted posts held here with the tag. They are public, so they federate: + `/tattoos` lists them as `Hashtag`s, and featuring one sends `Add{Hashtag}`, unfeaturing `Remove{Hashtag}`, to the + persona's followers, **with its `featured` collection (`/trophies`) as the target**, as Mastodon sends them + (`AddHashtagSerializer`) and the only target Mastodon takes a Hashtag from. Received: `FeaturedHandler` takes a `Hashtag` + added to or removed from an account's `featured` or `featuredTags` (an object with no id, read before anything takes its + `href` for a post), and `AccountCountsJob` reads the `featuredTags` collection (its own server only, three pages, 20 tags + at most) with the account's counts. +- **Endorsements** (`AccountsController`): `POST /api/v1/accounts/{id}/pin|unpin` (Mastodon's old names) and + `/endorse|/unendorse` (4.4) answer the relationship; only an account the persona follows (422 otherwise, as Mastodon + says it), 500 at most. `GET /api/v1/endorsements` lists the persona's own, newest first, those it still follows (paged + by `Link`); `GET /api/v1/accounts/{id}/endorsements` answers them to the persona itself and `[]` for every other account: + **endorsements stay private** (the owner's default), never federated as featured accounts. Unfollowing drops the + endorsement, as on Mastodon; a `Move` carries the endorsement and the note over to the new account. +- **Private notes:** `POST /api/v1/accounts/{id}/note` (`comment`, 2000 characters at most, 422 beyond; empty takes it + away) answers the relationship, whose `note` is the persona's own `PersonaNote`. Never federated. +- **The relationship** fills `notifying` and `languages` from the follow, `endorsed` (only while it follows) and `note`. + `POST /api/v1/accounts/{id}/follow` takes `notify` and `languages[]` (each changed only when given, as on Mastodon; an + empty list is every language). `Fanout` keeps a post in another language out of that follower's home (a post whose + language is unknown comes in; languages compare by their primary subtag) and gives a follower with `notify` a `status` + notification for the account's own posts and its replies to itself, never its boosts, once the post reached that home + (Mastodon's `FeedInsertWorker`). Archives carry both in `following_accounts.csv`'s own columns. +- **Preferences:** `GET /api/v1/preferences` answers `reading:expand:media` and `reading:expand:spoilers` from the + persona's settings, which Mastodon's API sets nowhere, so `GET`/`POST /clientapi/avatar/private/preferences` + (`ViewPersonaPreferences`, `PersonaPreferencesForm`, the login's own persona; 404 for any other, 422 for another mode) + sets them. +- **Suggestions** (`Domain/Social/Suggestions.cs`, `SuggestionsController`): `GET /api/v1/suggestions` (accounts) and + `/api/v2/suggestions` (`source`, `sources`, `account`), `limit` 40 (80 at most); `DELETE /api/v1/suggestions/{account_id}` + (`{}`) dismisses one for that persona for good. In order: the staff's picks (`ServerSettings.StaffPicks`, `staff` / + `featured`), past interactions (accounts the persona favourited, boosted or answered lately, `past_interactions`, which + 4.3's `sources` has no word for), then global: accounts followed by at least three personas here (`most_followed`) and + personas here whose public posts of the last 30 days were favourited and boosted most (`most_interactions`), + discoverable ones only, worked out at most every ten minutes. **Never the persona itself and never a sibling persona of + its login** (`RootToAvatar` read server-side only: suggesting a sibling would pair them the moment the persona followed + it), nor what it follows, asked to follow, blocked, muted, was blocked by (here or elsewhere), dismissed or hides by + server, nor an account held back (`Counted.HeldBackPersonas`), suspended, silenced or shown `limited`; a community only + as a staff pick. +- **Staff picks** (`InstanceAdminController`, administrators, checked against the database): `GET + /clientapi/admin/staff-picks`, `POST /staff-picks/insert` (`StaffPickForm`: a persona here or a person elsewhere by its + Mastodon id; 404 for no such account, 409 for one picked already, 422 past 40), `POST /staff-picks/delete?accountId=`; + each answers `List`, which `ViewServerSettings.StaffPicks` shows too (`IsShown` false for one deleted, + suspended or silenced since). They are kept in the settings but never changed by the settings form, so an older page + saving the form keeps them. Logged as `staff_pick.insert` and `staff_pick.delete` (target `persona` or `account`, so an + entry about a persona of the acting login keeps no actor). +- **familiar_followers stays empty by design:** which accounts the persona follows also follow another would tell who + follows whom among the personas here; every account answers `accounts: []`. +- **The direct stream:** `StreamingController` takes `stream=direct` (and the WebSocket's `subscribe` to `direct`); a + direct message saved here or arriving (`StatusService`, `CreateHandler`, after `ConversationStates.Posted`) tells the + `direct` streams of the conversation's personas a `conversation` event (`Fanout.Conversation`), mapped per persona as + `/api/v1/conversations` maps it. +- **Stubs that answer:** `POST /api/v1/statuses/{id}/translate` 422 (translation is cut), `GET /api/v1/annual_reports` + `{"annual_reports":[],"accounts":[],"statuses":[]}`, every verb under `/api/v1/admin/**` (and `GET`/`POST` under + `/api/v2/admin/**`) 403 `{"error":"This action is not allowed"}`, signed in or not (moderation stays on `/clientapi`). +- **Removal, backups, archives:** `PersonaRemoval` deletes the persona's featured tags, endorsements, notes and + dismissals, and what other personas kept about it (their endorsements of it, notes on it, dismissals of it); the Delete + of the actor tells other servers. Every new collection is in every backup (none is in `ServerBackup.Excluded`: they are + personas' own data, the remote featured tags a cache like `RemoteFeatured`); a restore keeps them as the backup has them, + and `ProtectiveMerge` deletes those of a persona deleted since. Mastodon's account archive has no featured tags, notes or + endorsements, so the persona archive leaves them out. + ## Custom emoji (completion program R7) - **The server's own emoji** (`Models/Post/LocalEmoji.cs`, a collection of its own): a shortcode (`[a-zA-Z0-9_]{2,64}`, @@ -541,6 +620,10 @@ client's role route never does. (`ObjectShapes.Flairs`, both dialects). Every media URL in it goes through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following one is the reader's choice. +8. **What a persona keeps through the API is that persona's** (R8): featured tags (public, federated), endorsements and + private notes (shown to it alone, never federated), dismissed suggestions. Suggestions never offer the persona itself + or a sibling persona of its login, and `familiar_followers` answers no one for every account, since it would tell who + follows whom here. ## Media invariants @@ -673,7 +756,9 @@ client's role route never does. - **An actor's `published` is `PublishedOn`**, a random whole day up to two weeks before creation, so personas made the same day do not share a date. New persona and group ids carry that same day (`GenerateNewID`), because local clients see ids. Never emit `CreatedAt`. -- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows. Nothing may relate sibling avatars. +- **Per-avatar state stays per avatar:** blocks, mutes, notifications, follows, and since R8 featured tags, + endorsements, private notes (`PersonaNote`, never `RootUserNote`) and dismissed suggestions. Nothing may relate sibling + avatars: suggestions never offer a sibling, and familiar followers stay empty. - **Blocks federate** (owner decision, 2026-10-01): a block is sent as `Block` from the blocking avatar, an unblock as `Undo{Block}`. Reports still leave as `Flag` from the server's own actors (the instance actor, or the reporter for communities on Lemmy, PieFed and Mbin), never from the reporting avatar. @@ -1061,6 +1146,14 @@ tools/pasture/run.sh down # removes e in the status's `emojis` on Mastodon, GoToSocial (by URI), Misskey (the note's `emojis`) and Akkoma; her display name shows it on Mastodon; her `EmojiReact` with it reaches Misskey and Akkoma as that emoji; deleted, it leaves the list, its id answers 410 and posts here carry it no more; the staff's log keeps the insert and the delete. +- **Featured tags and the API made real (`scenarios/featured.sh`, needs mastodon; after emoji in batch A):** mastouser + follows feat_alice, who features a tag: Mastodon's `featured_tags` of her account shows it (her `Add{Hashtag}`, the + `/tattoos` collection serving it), unfeaturing takes it away there; mastouser features a tag, which feat_alice reads + (following him) through `/api/v1/accounts/:id/featured_tags`; following mastouser with `notify=true` gives feat_alice a + `status` notification for his next post; her SSE and WebSocket `direct` streams tell a `conversation` event for his + DM; her endorsement of him and her note on him are hers alone (feat_bob, a sibling persona of the same login, sees + neither, and `GET /api/v1/accounts/:id/endorsements` answers nobody else), and feat_bob is never suggested to her; the + stubs (translate 422, annual reports, the admin API 403) answer. - **Mailpit (1.31.4, `peers/mailpit.sh`, as `mailpit.test`):** the pasture's mail server. `appsettings.Pasture.json` sends PrivaPub's mail to `pasture-mailpit:1025` (plain SMTP, any login accepted) and its recovery links to `https://decepub.test`. `scenarios/accounts.sh` reads what arrived through Mailpit's API: an address set with the diff --git a/FEDERATION.md b/FEDERATION.md index 00b0849..559277d 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -132,6 +132,7 @@ actor document, a collection, NodeInfo or a delivery relates two avatars of the | Direct-message context | `/peasants/{name}/whispers/{id}` | | A persona's wall (FEP-400e) | `/peasants/{name}/graffiti` | | Followers on the asking server (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` | +| Featured (pinned) posts, featured hashtags | `/peasants/{name}/trophies`, `/peasants/{name}/tattoos` | | A custom emoji of this server (`Emoji`) | `/emojis/{shortcode}` | | Profile and post pages | `/@{name}`, `/@{name}/{id}` | @@ -216,12 +217,13 @@ Received: | `Delete` | deletes the object, or the actor and its follows; a deleted object id is remembered for 90 days, so a late `Create` cannot bring it back, though a new post its server publishes under that id after the deletion (Gancio numbers events from the last one kept) is kept | | `Flag` | becomes a report for this server's moderators | | `Move` | an account moving: believed as Mastodon believes it, when the account sends it about itself and the new account, read again from its server, names it in `alsoKnownAs`. The old account then shows where it went (`moved`), and, as Mastodon does it (owner decision 2026-10-05), the personas following it follow the new one instead (a Follow to its server, an Undo to the old), in the same lists; a mute or a block of the old account carries over | +| `Add`/`Remove` of a `Hashtag` on the actor's `featured` or `featuredTags` | the account features a hashtag on its profile, or stops (Mastodon sends them to `featured`); shown as its featured tags (`GET /api/v1/accounts/:id/featured_tags`). Its `featuredTags` collection itself is read with its counts, at most once a day (20 tags at most) | | `Add`/`Remove` on the actor's `featured` | the account pins or unpins one of its own posts (fetched from its server when not held); inside a community's announce, the community features a post made in it. Its profile shows them first (`pinned=true`). The collection itself is read with the account's counts, at most once a day. On the account's wall (`sm:wall`), see "Walls". Any other target (a community's moderators) is dropped; one on the account's own server that PrivaPub does not know has its document read again first, at most hourly | | `Block` of a persona | the follows between them end, the blocker's posts and notifications are hidden from the persona, nothing of the persona's is addressed to the blocker, and the relationship says `blocked_by`; `Undo{Block}` lifts it | Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Create{Question}` and poll votes, `EmojiReact` and its `Undo`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`, -`Reject{Follow}`, `Like`, `Dislike`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event, and the -replies to a persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`. +`Reject{Follow}`, `Like`, `Dislike`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event, +`Add`/`Remove` of a pinned post or a featured `Hashtag`, and the replies to a persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`. - **Votes.** A favourite is a `Like`, a downvote a `Dislike`; a persona has one vote a post, and a changed vote is sent as the new vote alone, as Lemmy sends one (an `Undo` of the old one beside it could arrive after it and take the new @@ -256,8 +258,21 @@ replies to a persona's posts passed on to its followers. A deleted post answers post on its wall. The wall lists the persona's public posts that start a thread and what was written on it, newest first. An account elsewhere that tells its followers here, with `Add{Note}` on its own wall, of a post written there has it shown to them as on its wall (`privapub.wall` on the status); a `Remove` there takes it away. -- **Pinned posts** are the actor's `featured` collection (`/trophies`); `featuredTags` is `/tattoos`. A pin or an unpin - is told to the post's audience as `Add` or `Remove` on `featured`, as Mastodon tells it. +- **Pinned posts** are the actor's `featured` collection (`/trophies`). A pin or an unpin is told to the post's audience + as `Add` or `Remove` on `featured`, as Mastodon tells it. +- **Featured hashtags** are the actor's `featuredTags` collection (`/tattoos`): an `OrderedCollection` of `Hashtag`s, in + the order the persona featured them (ten at most), each `{"type": "Hashtag", "href": "/tags/", "name": + "#Tag"}`. Featuring one sends its followers + ```json + {"id": "…/peasants/alice/grunts/feature-tag-…", "type": "Add", "actor": "…/peasants/alice", + "object": {"type": "Hashtag", "href": "https://privapub.thepra.dev/tags/gardening", "name": "#Gardening"}, + "target": "…/peasants/alice/trophies", "to": ["…#Public"], "cc": ["…/peasants/alice/groupies"]} + ``` + and unfeaturing it the same `Remove`. The target is the `featured` collection, not `featuredTags`: that is what Mastodon + sends and the only target it takes a Hashtag from (it tells a pin from a tag by the object's type). +- **What is never sent:** a persona's endorsements of other accounts (Mastodon's account pins and 4.4's endorsements) stay + private, never published as featured accounts or a collection; its private notes on accounts and its dismissed follow + suggestions are its own. No collection lists who follows whom (familiar followers are never shown either). - **Blocks are sent.** A blocked remote account receives `Block` from the blocking account (and `Reject{Follow}` if it followed); an unblock sends `Undo{Block}`. A server's block of a whole server sends that server nothing; the administrator may list it at `/api/v1/instance/domain_blocks` (off for each block by default, its name partly hidden diff --git a/PrivaPub.ClientModels/Admin/ViewServerSettings.cs b/PrivaPub.ClientModels/Admin/ViewServerSettings.cs index 984d8af..6f8b048 100644 --- a/PrivaPub.ClientModels/Admin/ViewServerSettings.cs +++ b/PrivaPub.ClientModels/Admin/ViewServerSettings.cs @@ -25,6 +25,7 @@ namespace PrivaPub.ClientModels.Admin public DateTime? TermsEffectiveAt { get; set; } public List Rules { get; set; } = [];//deleted ones too (IsDeleted), as old reports name them public List Relays { get; set; } = []; + public List StaffPicks { get; set; } = [];//the accounts suggested first to every persona (R8), their own routes public ViewMediaLimits Media { get; set; } = new(); public string RegistrationMode { get; set; }//"Invitations" or "Open", from the configuration public bool PublishActivity { get; set; } = false; @@ -158,6 +159,26 @@ namespace PrivaPub.ClientModels.Admin public DateTime? AnsweredAt { get; set; } } + // an account the administrators suggest to every persona here, first (R8: Mastodon's staff picks, `featured` among a + // suggestion's sources): a persona of this server or an account elsewhere, by its Mastodon account id. Added and removed + // on their own routes (/clientapi/admin/staff-picks), never through the settings form, so an older page saving the form + // never drops them. + public class ViewStaffPick + { + public string AccountId { get; set; } + public bool IsLocal { get; set; } = false; + public string Handle { get; set; }//user@host + public string Name { get; set; } + public bool IsShown { get; set; } = true;//false: deleted, suspended or silenced since, so suggested to nobody + } + + public class StaffPickForm + { + [Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)), + StringLength(64, ErrorMessageResourceName = "StringLength", ErrorMessageResourceType = typeof(ErrorsResource))] + public string AccountId { get; set; } + } + public class RelayForm { [Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)), diff --git a/PrivaPub.ClientModels/User/Avatar/PersonaPreferences.cs b/PrivaPub.ClientModels/User/Avatar/PersonaPreferences.cs new file mode 100644 index 0000000..c0b6b66 --- /dev/null +++ b/PrivaPub.ClientModels/User/Avatar/PersonaPreferences.cs @@ -0,0 +1,29 @@ +using PrivaPub.ClientModels.Resources; + +using System.ComponentModel.DataAnnotations; + +namespace PrivaPub.ClientModels.User.Avatar +{ + // How a persona's clients show what others post (R8): Mastodon's reading:expand:media ("default" hides media marked + // sensitive, "show_all", "hide_all") and reading:expand:spoilers, which GET /api/v1/preferences answers. Mastodon's API + // sets neither, so they are set here: GET and POST /clientapi/avatar/private/preferences, for the login's own persona. + public class ViewPersonaPreferences + { + public string AvatarId { get; set; } + public string ExpandMedia { get; set; } + public bool ExpandSpoilers { get; set; } = false; + } + + public class PersonaPreferencesForm + { + [Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)), + Display(Name = nameof(AvatarId), ResourceType = typeof(FieldsNameResource))] + public string AvatarId { get; set; } + + [Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)), + StringLength(16, ErrorMessageResourceName = "StringLength", ErrorMessageResourceType = typeof(ErrorsResource))] + public string ExpandMedia { get; set; }//"default", "show_all" or "hide_all" + + public bool ExpandSpoilers { get; set; } = false; + } +} diff --git a/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs b/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs index 2cf6f4c..3e571f2 100644 --- a/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs @@ -304,7 +304,10 @@ namespace PrivaPub.Api.Mastodon.Controllers await _feeds.Subscribe(Me, remote, true, token); return Json(await Relationship(id, token)); } - var following = await _follows.FollowAs(Me, local?.Uri ?? remote.ActorURI, Params.Bool("reblogs") != false, token); + // notify: a `status` notification for each of its new posts; languages: only its posts in those reach the home (R8). + // Either changes only when given, as on Mastodon + var following = await _follows.FollowAs(Me, local?.Uri ?? remote.ActorURI, Params.Bool("reblogs") != false, token, Params.Bool("notify"), + Params.Has("languages") ? Params.List("languages") : default); return following == default ? Error(StatusCodes.Status403Forbidden, "This action is not allowed") : Json(await Relationship(id, token)); } @@ -425,12 +428,112 @@ namespace PrivaPub.Api.Mastodon.Controllers public async Task Reject(string id, CancellationToken token) => await _follows.Decide(Me, id, accept: false, token) ? Json(await Relationship(id, token)) : NotFoundError(); - [HttpGet("/api/v1/accounts/{id}/featured_tags"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous] - public IActionResult FeaturedTags(string id) => Json(Array.Empty()); - + // Familiar followers stay empty by design: which of the accounts the persona follows also follow another account would + // tell who follows whom among the personas here (and their followers lists are never shown, hide_collections), so + // every account answers none [HttpGet("/api/v1/accounts/familiar_followers"), Scope("read:follows")] public IActionResult FamiliarFollowers() => Json(Params.List("id").Select(id => new { id, accounts = Array.Empty() }).ToList()); + // Endorsements (R8): Mastodon's account pins under their old names (pin, unpin) and 4.4's (endorse, unendorse). Only an + // account the persona follows may be endorsed, as Mastodon requires; they are the persona's alone, never federated as + // featured accounts and never listed to anyone else (the owner's default: endorsements stay private) + [HttpPost("/api/v1/accounts/{id}/pin"), HttpPost("/api/v1/accounts/{id}/endorse"), Scope("write:accounts")] + public async Task Endorse(string id, CancellationToken token) + { + var (local, remote) = await Find(id, token); + if (local == default && remote == default || local?.Id == MyId || local is { Suspended: true } || remote is { SuspendedAt: not null }) + return NotFoundError(); + var uri = local?.Uri ?? remote.ActorURI; + if (!await Follows(uri, token)) + return Error(StatusCodes.Status422UnprocessableEntity, "Validation failed: You must be already following the person you want to endorse"); + if (!await DB.Default.Find().Match(e => e.AvatarId == MyId && e.TargetActorURI == uri).ExecuteAnyAsync(token)) + { + if (await DB.Default.CountAsync(e => e.AvatarId == MyId, token) >= MaxEndorsements) + return Error(StatusCodes.Status422UnprocessableEntity, $"Validation failed: You can endorse at most {MaxEndorsements} accounts"); + try + { + await DB.Default.SaveAsync(new Endorsement { AvatarId = MyId, TargetAccountId = id, TargetActorURI = uri }, token); + } + catch (MongoDB.Driver.MongoWriteException ex) when (ex.WriteError?.Category == MongoDB.Driver.ServerErrorCategory.DuplicateKey) + { + } + } + return Json(await Relationship(id, token)); + } + + [HttpPost("/api/v1/accounts/{id}/unpin"), HttpPost("/api/v1/accounts/{id}/unendorse"), Scope("write:accounts")] + public async Task Unendorse(string id, CancellationToken token) + { + var (local, remote) = await Find(id, token); + if (local == default && remote == default) + return NotFoundError(); + var uri = local?.Uri ?? remote.ActorURI; + await DB.Default.DeleteAsync(e => e.AvatarId == MyId && (e.TargetActorURI == uri || e.TargetAccountId == id)); + return Json(await Relationship(id, token)); + } + + // the persona's own endorsements, newest first, those it still follows + [HttpGet("/api/v1/endorsements"), Scope("read:accounts")] + public Task Endorsements(CancellationToken token) => EndorsementsOf("/api/v1/endorsements", token); + + // Mastodon 4.4: an account's endorsements. Only the persona's own are answered, to the persona itself: endorsements + // stay private, so every other account answers none + [HttpGet("/api/v1/accounts/{id}/endorsements"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous] + public async Task EndorsementsOfAccount(string id, CancellationToken token) => + MyId != default && id == MyId ? await EndorsementsOf($"/api/v1/accounts/{id}/endorsements", token) : Json(Array.Empty()); + + async Task EndorsementsOf(string path, CancellationToken token) + { + var followed = (await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.State == FollowState.Accepted && f.FollowerKind == null) + .Project(f => f.Include(x => x.TargetActorURI)).ExecuteAsync(token)).Select(f => f.TargetActorURI).ToList(); + var page = await Page.From(Params, Limit(40, 80)) + .Fetch(DB.Default.Find().Match(e => e.AvatarId == MyId && followed.Contains(e.TargetActorURI)), e => e.ID, token); + Link(path, page.LastOrDefault()?.ID, page.FirstOrDefault()?.ID); + var accounts = await _mapper.Accounts(page.Select(e => e.TargetAccountId), token); + return Json(page.Select(e => accounts.GetValueOrDefault(e.TargetAccountId)).Where(a => a != default).ToList()); + } + + // A private note on an account (R8): the persona's own (PersonaNote, never its login's), shown only to it in the + // relationship's `note`, never federated; an empty comment takes it away + [HttpPost("/api/v1/accounts/{id}/note"), Scope("write:accounts")] + public async Task Note(string id, CancellationToken token) + { + var (local, remote) = await Find(id, token); + if (local == default && remote == default) + return NotFoundError(); + var comment = Params.Get("comment")?.Trim(); + if (comment?.Length > PersonaNote.MaxLength) + return Error(StatusCodes.Status422UnprocessableEntity, $"Validation failed: Comment is too long (maximum is {PersonaNote.MaxLength} characters)"); + if (string.IsNullOrEmpty(comment)) + await DB.Default.DeleteAsync(n => n.AvatarId == MyId && n.TargetAccountId == id); + else + await DB.Default.Update().Match(n => n.AvatarId == MyId && n.TargetAccountId == id) + .Modify(n => n.Text, comment) + .Modify(n => n.UpdatedAt, DateTime.UtcNow) + .Modify(b => b.SetOnInsert(n => n.CreatedAt, DateTime.UtcNow)) + .Option(o => o.IsUpsert = true) + .ExecuteAsync(token); + return Json(await Relationship(id, token)); + } + + // Mastodon's preferences: posting defaults (update_credentials' source) and reading ones (R8: AvatarSettings, set + // through /clientapi/avatar/private/preferences, since Mastodon's API sets none) + [HttpGet("/api/v1/preferences"), Scope("read:accounts")] + public IActionResult Preferences() => Json(new Dictionary + { + ["posting:default:visibility"] = Me.Settings.DefaultVisibility ?? "public", + ["posting:default:sensitive"] = Me.Settings.DefaultSensitive, + ["posting:default:language"] = Me.Settings.DefaultLanguage, + ["reading:expand:media"] = ExpandMediaModes.IsKnown(Me.Settings.ExpandMedia) ? Me.Settings.ExpandMedia : ExpandMediaModes.Default, + ["reading:expand:spoilers"] = Me.Settings.ExpandSpoilers + }); + + const int MaxEndorsements = 500; + + async Task Follows(string uri, CancellationToken token) => + await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.TargetActorURI == uri && f.State == FollowState.Accepted && f.FollowerKind == null) + .ExecuteAnyAsync(token); + async Task> AccountsFor(IEnumerable actorUris, CancellationToken token) { var accounts = new List(); @@ -474,6 +577,10 @@ namespace PrivaPub.Api.Mastodon.Controllers : remote != default && await DB.Default.Find().Match(b => b.AvatarId == MyId && b.ActorURI == remote.ActorURI).ExecuteAnyAsync(token); var domainBlocked = remote != default && (await DB.Default.Find().Match(b => b.AvatarId == MyId).ExecuteAsync(token)) .Any(b => Hidden.Blocks(b.Domain, remote.ActorURI, remote.Domain)); + // the persona's own endorsement and note (R8), never another persona's: a note made as a sibling stays its own + var endorsed = uri != default && following?.State == FollowState.Accepted + && await DB.Default.Find().Match(e => e.AvatarId == MyId && e.TargetActorURI == uri).ExecuteAnyAsync(token); + var note = await DB.Default.Find().Match(n => n.AvatarId == MyId && n.TargetAccountId == id).ExecuteFirstAsync(token); return new Relationship { Id = id, @@ -485,8 +592,12 @@ namespace PrivaPub.Api.Mastodon.Controllers Following = following?.State == FollowState.Accepted || reading, Requested = following?.State == FollowState.Requested, ShowingReblogs = following?.ShowReblogs ?? false, + Notifying = following?.Notify ?? false, + Languages = following?.Languages, FollowedBy = followedBy?.IsAccepted == true, - RequestedBy = followedBy is { IsAccepted: false } + RequestedBy = followedBy is { IsAccepted: false }, + Endorsed = endorsed, + Note = note?.Text ?? string.Empty }; } } diff --git a/PrivaPub/Api/Mastodon/Controllers/FeaturedTagsController.cs b/PrivaPub/Api/Mastodon/Controllers/FeaturedTagsController.cs new file mode 100644 index 0000000..afe3ee2 --- /dev/null +++ b/PrivaPub/Api/Mastodon/Controllers/FeaturedTagsController.cs @@ -0,0 +1,113 @@ +using Microsoft.AspNetCore.Mvc; + +using MongoDB.Entities; + +using PrivaPub.Api.Mastodon.Entities; +using PrivaPub.Api.Mastodon.Infrastructure; +using PrivaPub.Domain.Social; +using PrivaPub.Federation.Actors; +using PrivaPub.Federation.Rendering; +using PrivaPub.Models.Federation; +using PrivaPub.Models.Social; +using PrivaPub.StaticServices; + +namespace PrivaPub.Api.Mastodon.Controllers +{ + // Featured hashtags as Mastodon's API has them (completion program R8, Domain/Social/FeaturedTags): the persona's own, + // featured and unfeatured (its followers are told), suggested from the tags it uses, and any account's: a persona's own + // here, or those an account elsewhere features as its server tells them. They are public, as on Mastodon. + public class FeaturedTagsController : MastodonController + { + const int MaxSuggested = 10;//Mastodon's RECENT_TAGS_LIMIT + const int SuggestionPosts = 200; + + readonly IFeaturedTags _featured; + readonly DbEntities _dbEntities; + readonly ILocalActorService _localActors; + + public FeaturedTagsController(IFeaturedTags featured, DbEntities dbEntities, ILocalActorService localActors) + { + _featured = featured; + _dbEntities = dbEntities; + _localActors = localActors; + } + + [HttpGet("/api/v1/featured_tags"), Scope("read:accounts")] + public async Task Mine(CancellationToken token) => Json(await Entities(await _featured.Of(MyId, token), token)); + + [HttpPost("/api/v1/featured_tags"), Scope("write:accounts")] + public async Task Feature(CancellationToken token) + { + var (tag, error) = await _featured.Feature(Me, Params.Get("name"), token); + return tag == default ? Error(StatusCodes.Status422UnprocessableEntity, error) : Json((await Entities(new List { tag }, token))[0]); + } + + [HttpDelete("/api/v1/featured_tags/{id}"), Scope("write:accounts")] + public async Task Unfeature(string id, CancellationToken token) => + await _featured.Unfeature(Me, id, default, token) == default ? NotFoundError() : Json(new { }); + + // the tags the persona used lately and does not feature yet, the most used first, as Mastodon's Tag + [HttpGet("/api/v1/featured_tags/suggestions"), Scope("read:accounts")] + public async Task Suggestions(CancellationToken token) + { + var featured = (await _featured.Of(MyId, token)).Select(t => t.Name).ToHashSet(StringComparer.Ordinal); + var recent = await _dbEntities.Posts.Match(p => p.GroupUserId == MyId && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.ReblogOfPostId == null + && p.Tags.Count > 0) + .Sort(p => p.ID, Order.Descending).Limit(SuggestionPosts).Project(p => p.Include(x => x.Tags)).ExecuteAsync(token); + var used = recent.SelectMany((p, age) => p.Tags.Distinct().Select(tag => (tag, age))) + .Where(t => !featured.Contains(t.tag) && TagsController.IsHashtag(t.tag)) + .GroupBy(t => t.tag) + .OrderByDescending(g => g.Count()).ThenBy(g => g.Min(t => t.age)) + .Take(MaxSuggested) + .Select(g => g.Key) + .ToList(); + var views = new List(); + foreach (var tag in used) + views.Add(await TagsController.TagView(_dbEntities, _localActors.BaseAddress, tag, MyId, default, false, token)); + return Json(views); + } + + // what an account features: a persona's own hashtags, or what an account elsewhere tells (its featuredTags, its Add + // and Remove); a community features none. A suspended account's answer 404, as its account does (R6). + [HttpGet("/api/v1/accounts/{id}/featured_tags"), Scope("read:accounts", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous] + public async Task OfAccount(string id, CancellationToken token) + { + var avatar = await _dbEntities.Avatars.MatchID(id).Match(a => !a.DeletionAt.HasValue).ExecuteFirstAsync(token); + if (avatar != default) + return _localActors.FromAvatar(avatar) is { Suspended: false } + ? Json(await Entities(await _featured.Of(avatar.ID, token), token)) + : NotFoundError(); + if (await _dbEntities.Groups.MatchID(id).Match(g => !g.DeletionAt.HasValue).ExecuteAnyAsync(token)) + return Json(Array.Empty()); + var remote = await _dbEntities.ForeignAvatars.MatchID(id).Match(f => !f.DeletionAt.HasValue).ExecuteFirstAsync(token); + if (remote == default || remote.SuspendedAt.HasValue) + return NotFoundError(); + var tags = await DB.Default.Find().Match(t => t.ActorURI == remote.ActorURI).Sort(t => t.Position, Order.Ascending) + .Limit(FeaturedTags.MaxRemote).ExecuteAsync(token); + var usage = await FeaturedTags.Usage(default, remote.ActorURI, tags.Select(t => t.Name).ToList(), _dbEntities, token); + return Json(tags.Select(t => Entity(t.ID, t.Name, t.DisplayName, t.Href, usage.GetValueOrDefault(t.Name))).ToList()); + } + + async Task> Entities(List tags, CancellationToken token) + { + var byPersona = tags.GroupBy(t => t.AvatarId).ToList(); + var entities = new List(); + foreach (var persona in byPersona) + { + var usage = await FeaturedTags.Usage(persona.Key, default, persona.Select(t => t.Name).ToList(), _dbEntities, token); + entities.AddRange(persona.Select(t => Entity(t.ID, t.Name, t.DisplayName, default, usage.GetValueOrDefault(t.Name)))); + } + return entities; + } + + FeaturedTagEntity Entity(string id, string name, string displayName, string href, (long Count, DateTime? Last) usage) => new() + { + Id = id, + Name = string.IsNullOrEmpty(displayName) ? name : displayName, + // (a link to the remote site is only a link, which the reader follows or not; ours is this server's tag page) + Url = href ?? ActivityPubRenderer.TagUrl(_localActors.BaseAddress, name), + StatusesCount = usage.Count.ToString(System.Globalization.CultureInfo.InvariantCulture), + LastStatusAt = usage.Last?.ToString("yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture) + }; + } +} diff --git a/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs b/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs index 684da3a..735349e 100644 --- a/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs @@ -134,7 +134,7 @@ namespace PrivaPub.Api.Mastodon.Controllers registrations = Open, approval_required = false, invites_enabled = true,//a group invitation always creates an account - configuration = new { accounts = new { max_featured_tags = 0 }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls }, + configuration = new { accounts = new { max_featured_tags = PrivaPub.Domain.Social.FeaturedTags.MaxFeatured }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls }, contact_account = await ContactAccount(settings, token), rules = Rules(settings) }); @@ -161,7 +161,7 @@ namespace PrivaPub.Api.Mastodon.Controllers configuration = new { urls = new { streaming = Streaming }, - accounts = new { max_featured_tags = 0, max_pinned_statuses = StatusesController.MaxPins }, + accounts = new { max_featured_tags = PrivaPub.Domain.Social.FeaturedTags.MaxFeatured, max_pinned_statuses = StatusesController.MaxPins }, statuses = Statuses, media_attachments = MediaAttachments, polls = Polls, diff --git a/PrivaPub/Api/Mastodon/Controllers/SearchController.cs b/PrivaPub/Api/Mastodon/Controllers/SearchController.cs index 2695d0d..7aaee9e 100644 --- a/PrivaPub/Api/Mastodon/Controllers/SearchController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/SearchController.cs @@ -140,28 +140,6 @@ namespace PrivaPub.Api.Mastodon.Controllers public class StubsController : MastodonController { - [HttpGet("/api/v1/suggestions"), Scope("read")] - public IActionResult SuggestionsV1() => Json(Array.Empty()); - - [HttpGet("/api/v2/suggestions"), Scope("read")] - public IActionResult SuggestionsV2() => Json(Array.Empty()); - - [HttpGet("/api/v1/endorsements"), Scope("read:accounts")] - public IActionResult Endorsements() => Json(Array.Empty()); - - [HttpGet("/api/v1/featured_tags"), Scope("read:accounts")] - public IActionResult FeaturedTags() => Json(Array.Empty()); - - [HttpGet("/api/v1/preferences"), Scope("read:accounts")] - public IActionResult Preferences() => Json(new Dictionary - { - ["posting:default:visibility"] = Me.Settings.DefaultVisibility ?? "public", - ["posting:default:sensitive"] = Me.Settings.DefaultSensitive, - ["posting:default:language"] = Me.Settings.DefaultLanguage, - ["reading:expand:media"] = "default", - ["reading:expand:spoilers"] = false - }); - [HttpGet("/api/v1/push/subscription"), Scope("push")] public IActionResult PushSubscription() => Error(StatusCodes.Status404NotFound, "Record not found"); @@ -179,5 +157,21 @@ namespace PrivaPub.Api.Mastodon.Controllers [HttpGet("/api/v1/instance/translation_languages"), Microsoft.AspNetCore.Authorization.AllowAnonymous] public IActionResult TranslationLanguages() => Json(new { }); + + // translation is cut (owner decision 2026-10-17): no language offers any (translation_languages), and a client that + // asks anyway is told so, as Mastodon tells it when it can't translate + [HttpPost("/api/v1/statuses/{id}/translate"), Scope("read:statuses")] + public IActionResult Translate(string id) => Error(StatusCodes.Status422UnprocessableEntity, "Translation is not available on this server"); + + // Mastodon 4.3's yearly reports (Wrapstodon): none are made here, so there are never any to show + [HttpGet("/api/v1/annual_reports"), Scope("read:accounts")] + public IActionResult AnnualReports() => Json(new { annual_reports = Array.Empty(), accounts = Array.Empty(), statuses = Array.Empty() }); + + // the Mastodon admin API is cut: moderation stays on /clientapi (ModeratorController, AdminController), so every + // admin route answers as Mastodon answers an account that may not use it, whoever asks + [HttpGet("/api/v1/admin/{**rest}"), HttpPost("/api/v1/admin/{**rest}"), HttpPut("/api/v1/admin/{**rest}"), HttpPatch("/api/v1/admin/{**rest}"), + HttpDelete("/api/v1/admin/{**rest}"), HttpGet("/api/v2/admin/{**rest}"), HttpPost("/api/v2/admin/{**rest}"), + ApiExplorerSettings(IgnoreApi = true), Microsoft.AspNetCore.Authorization.AllowAnonymous] + public IActionResult Admin(string rest) => Error(StatusCodes.Status403Forbidden, "This action is not allowed"); } } diff --git a/PrivaPub/Api/Mastodon/Controllers/StreamingController.cs b/PrivaPub/Api/Mastodon/Controllers/StreamingController.cs index c27bae3..cef62bd 100644 --- a/PrivaPub/Api/Mastodon/Controllers/StreamingController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/StreamingController.cs @@ -41,7 +41,7 @@ namespace PrivaPub.Api.Mastodon.Controllers { switch (name) { - case Streams.User or Streams.UserNotification or Streams.Public or Streams.PublicLocal or Streams.PublicRemote: + case Streams.User or Streams.UserNotification or Streams.Public or Streams.PublicLocal or Streams.PublicRemote or Streams.Direct: return (new Streams.Key(name), default); case Streams.Hashtag or Streams.HashtagLocal: var normal = TagsController.Normalise(tag); @@ -69,6 +69,15 @@ namespace PrivaPub.Api.Mastodon.Controllers } if (e.Name == "announcement.reaction") return JsonSerializer.Serialize(await AnnouncementsController.ReactionEvent(e.Id, e.Data, token), MastodonJson.Options); + // a direct conversation the persona takes part in (R8): Mastodon's Conversation, as /api/v1/conversations lists it + if (e.Name == "conversation") + { + var group = await _dbEntities.DmGroups.Match(g => g.ID == e.Id && !g.DeletionAt.HasValue && g.Members.Any(m => !m.IsForeign && m.AvatarId == MyId)) + .ExecuteFirstAsync(token); + var conversation = group == default ? default + : (await TimelinesController.ConversationsOf(_mapper, _dbEntities, MyId, new List { group }, token)).FirstOrDefault(); + return conversation == default ? default : JsonSerializer.Serialize(conversation, MastodonJson.Options); + } if (e.Name == "notification") { var notification = await _dbEntities.Notifications.Match(n => n.ID == e.Id && n.AvatarId == MyId).ExecuteFirstAsync(token); diff --git a/PrivaPub/Api/Mastodon/Controllers/SuggestionsController.cs b/PrivaPub/Api/Mastodon/Controllers/SuggestionsController.cs new file mode 100644 index 0000000..a08da1d --- /dev/null +++ b/PrivaPub/Api/Mastodon/Controllers/SuggestionsController.cs @@ -0,0 +1,76 @@ +using Microsoft.AspNetCore.Mvc; + +using MongoDB.Driver; +using MongoDB.Entities; + +using PrivaPub.Api.Mastodon.Entities; +using PrivaPub.Api.Mastodon.Infrastructure; +using PrivaPub.Api.Mastodon.Mappers; +using PrivaPub.Domain.Instance; +using PrivaPub.Domain.Relationships; +using PrivaPub.Domain.Social; +using PrivaPub.Models.Social; + +namespace PrivaPub.Api.Mastodon.Controllers +{ + // Follow suggestions as Mastodon's API has them (completion program R8; who and why in Domain/Social/Suggestions): v1 the + // accounts, v2 each with its reasons; a dismissed account is never suggested to that persona again. + public class SuggestionsController : MastodonController + { + readonly MastodonMapper _mapper; + readonly IInstanceSettings _settings; + + public SuggestionsController(MastodonMapper mapper, IInstanceSettings settings) + { + _mapper = mapper; + _settings = settings; + } + + [HttpGet("/api/v1/suggestions"), Scope("read")] + public async Task V1(CancellationToken token) => Json((await Suggested(token)).Select(s => s.Account).ToList()); + + [HttpGet("/api/v2/suggestions"), Scope("read")] + public async Task V2(CancellationToken token) => Json(await Suggested(token)); + + [HttpDelete("/api/v1/suggestions/{accountId}"), Scope("read")] + public async Task Dismiss(string accountId, CancellationToken token) + { + if (!string.IsNullOrEmpty(accountId) && accountId.Length <= 64) + try + { + await DB.Default.Update().Match(d => d.AvatarId == MyId && d.AccountId == accountId) + .Modify(b => b.SetOnInsert(d => d.CreatedAt, DateTime.UtcNow)).Option(o => o.IsUpsert = true).ExecuteAsync(token); + } + catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey) + { + } + return Json(new { }); + } + + // the candidates mapped a few at a time, until the page is full: an account no longer shown (deleted, suspended), one + // shown limited (silenced, or its server), a community outside the staff's picks, and one the persona hides (its + // blocked servers) are passed over + async Task> Suggested(CancellationToken token) + { + var limit = Limit(40, 80); + var candidates = await Suggestions.For(Me, _settings.Current.StaffPicks, token); + var suggested = new List(); + for (var start = 0; start < candidates.Count && suggested.Count < limit; start += limit * 2) + { + var chunk = candidates.Skip(start).Take(limit * 2).ToList(); + var accounts = await _mapper.Accounts(chunk.Select(c => c.AccountId), token); + var hidden = await Hidden.AuthorsHiddenFrom(MyId, accounts.Values.Select(a => a.Uri), forNotifications: false, token); + foreach (var candidate in chunk) + { + if (accounts.GetValueOrDefault(candidate.AccountId) is not { } account || account.Limited == true || hidden.Contains(account.Uri) + || account.Group && candidate.Source != Suggestions.Staff) + continue; + suggested.Add(new Suggestion { Source = candidate.Source, Sources = candidate.Sources, Account = account }); + if (suggested.Count >= limit) + break; + } + } + return suggested; + } + } +} diff --git a/PrivaPub/Api/Mastodon/Controllers/TagsController.cs b/PrivaPub/Api/Mastodon/Controllers/TagsController.cs index e528e83..a11b3a8 100644 --- a/PrivaPub/Api/Mastodon/Controllers/TagsController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/TagsController.cs @@ -59,12 +59,19 @@ namespace PrivaPub.Api.Mastodon.Controllers }).ToList(); } - async Task View(string tag, bool? following, CancellationToken token) => new + Task View(string tag, bool? following, CancellationToken token, bool? featuring = default) => + TagView(_dbEntities, _localActors.BaseAddress, tag, MyId, following, featuring, token); + + // Mastodon's Tag: its week of use, and for a signed-in persona whether it follows it and (Mastodon 4.4) features it + internal static async Task TagView(DbEntities dbEntities, string baseAddress, string tag, string viewerId, bool? following, bool? featuring, + CancellationToken token) => new { name = tag, - url = $"{_localActors.BaseAddress}/tags/{Uri.EscapeDataString(tag)}", - history = await History(_dbEntities, tag, token), - following = following ?? (MyId != default && await DB.Default.Find().Match(t => t.AvatarId == MyId && t.Name == tag).ExecuteAnyAsync(token)) + url = $"{baseAddress}/tags/{Uri.EscapeDataString(tag)}", + history = await History(dbEntities, tag, token), + following = following ?? (viewerId != default && await DB.Default.Find().Match(t => t.AvatarId == viewerId && t.Name == tag).ExecuteAnyAsync(token)), + featuring = viewerId == default ? default(bool?) + : featuring ?? await DB.Default.Find().Match(t => t.AvatarId == viewerId && t.Name == tag).ExecuteAnyAsync(token) }; [HttpGet("/api/v1/tags/{name}"), Scope("read", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous] @@ -103,6 +110,24 @@ namespace PrivaPub.Api.Mastodon.Controllers return Json(await View(tag, false, token)); } + // Mastodon 4.4: a hashtag featured on the persona's profile, or no longer (R8, Domain/Social/FeaturedTags) + [HttpPost("/api/v1/tags/{name}/feature"), Scope("write:accounts")] + public async Task Feature(string name, [FromServices] Domain.Social.IFeaturedTags featured, CancellationToken token) + { + var (tag, error) = await featured.Feature(Me, name, token); + return tag == default ? Error(StatusCodes.Status422UnprocessableEntity, error) : Json(await View(tag.Name, default, token, featuring: true)); + } + + [HttpPost("/api/v1/tags/{name}/unfeature"), Scope("write:accounts")] + public async Task Unfeature(string name, [FromServices] Domain.Social.IFeaturedTags featured, CancellationToken token) + { + var tag = Normalise(name); + if (!IsHashtag(tag)) + return NotFoundError(); + await featured.Unfeature(Me, default, tag, token); + return Json(await View(tag, default, token, featuring: false)); + } + [HttpGet("/api/v1/followed_tags"), Scope("read:follows")] public async Task Followed(CancellationToken token) { diff --git a/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs b/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs index e96735d..84845d8 100644 --- a/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs @@ -123,35 +123,39 @@ namespace PrivaPub.Api.Mastodon.Controllers return Json(await ConversationsOf(groups, token)); } - async Task> ConversationsOf(List groups, CancellationToken token) + Task> ConversationsOf(List groups, CancellationToken token) => ConversationsOf(_mapper, _dbEntities, MyId, groups, token); + + // (also the `direct` stream's conversation event, StreamingController) + internal static async Task> ConversationsOf(MastodonMapper mapper, DbEntities dbEntities, string viewerId, List groups, + CancellationToken token) { var ids = groups.Select(g => g.ID).ToList(); - var states = (await DB.Default.Find().Match(s => s.AvatarId == MyId && ids.Contains(s.ConversationId)).ExecuteAsync(token)) + var states = (await DB.Default.Find().Match(s => s.AvatarId == viewerId && ids.Contains(s.ConversationId)).ExecuteAsync(token)) .ToDictionary(s => s.ConversationId); groups = groups.Where(g => !ConversationStates.IsHidden(states.GetValueOrDefault(g.ID), g.LastPostId)).ToList(); // the newest post each conversation still shows: usually its LastPostId, else (deleted) the one before var newestIds = groups.Select(g => g.LastPostId).ToList(); - var newest = (await _dbEntities.Posts.Match(p => newestIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token)) + var newest = (await dbEntities.Posts.Match(p => newestIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token)) .ToDictionary(p => p.ConversationId); foreach (var group in groups.Where(g => !newest.ContainsKey(g.ID))) - if (await _dbEntities.Posts.Match(p => p.ConversationId == group.ID).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending) + if (await dbEntities.Posts.Match(p => p.ConversationId == group.ID).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending) .ExecuteFirstAsync(token) is { } shown) newest[group.ID] = shown; var foreignUris = groups.SelectMany(g => g.Members).Where(m => m.IsForeign).Select(m => m.AvatarId).Distinct().ToList(); - var foreignIds = (await _dbEntities.ForeignAvatars.Match(f => foreignUris.Contains(f.ActorURI)).ExecuteAsync(token)) + var foreignIds = (await dbEntities.ForeignAvatars.Match(f => foreignUris.Contains(f.ActorURI)).ExecuteAsync(token)) .GroupBy(f => f.ActorURI).ToDictionary(g => g.Key, g => g.First().ID); - List Others(DmGroup group) => group.Members.Where(m => !(m.AvatarId == MyId && !m.IsForeign)) + List Others(DmGroup group) => group.Members.Where(m => !(m.AvatarId == viewerId && !m.IsForeign)) .Select(m => m.IsForeign ? foreignIds.GetValueOrDefault(m.AvatarId) : m.AvatarId).Where(id => id != default).ToList(); - var accounts = await _mapper.Accounts(groups.SelectMany(Others).Distinct(), token); - var statuses = (await _mapper.Statuses(newest.Values.ToList(), MyId, token)).ToDictionary(s => s.Id); + var accounts = await mapper.Accounts(groups.SelectMany(Others).Distinct(), token); + var statuses = (await mapper.Statuses(newest.Values.ToList(), viewerId, token)).ToDictionary(s => s.Id); return groups.Where(g => newest.ContainsKey(g.ID) && statuses.ContainsKey(newest[g.ID].ID)) .Select(g => new Conversation { Id = g.ID, - Unread = ConversationStates.IsUnread(states.GetValueOrDefault(g.ID), newest[g.ID].ID, newest[g.ID].AuthorAccountId, MyId), + Unread = ConversationStates.IsUnread(states.GetValueOrDefault(g.ID), newest[g.ID].ID, newest[g.ID].AuthorAccountId, viewerId), Accounts = Others(g).Select(id => accounts.GetValueOrDefault(id)).Where(a => a != default).ToList(), LastStatus = statuses[newest[g.ID].ID] }) @@ -240,7 +244,8 @@ namespace PrivaPub.Api.Mastodon.Controllers [NotificationType.Poll] = "poll", [NotificationType.Reaction] = "pleroma:emoji_reaction", [NotificationType.Quote] = "quote", - [NotificationType.ModerationWarning] = "moderation_warning" + [NotificationType.ModerationWarning] = "moderation_warning", + [NotificationType.Status] = "status"//an account followed with notify posted (R8) }; readonly MastodonMapper _mapper; diff --git a/PrivaPub/Api/Mastodon/Entities/Entities.cs b/PrivaPub/Api/Mastodon/Entities/Entities.cs index 00f54c6..83900d9 100644 --- a/PrivaPub/Api/Mastodon/Entities/Entities.cs +++ b/PrivaPub/Api/Mastodon/Entities/Entities.cs @@ -410,6 +410,24 @@ public string Note { get; set; } = string.Empty; } + // Mastodon's FeaturedTag: its count as a string and its last use as a day, as Mastodon's serializer writes them + public class FeaturedTagEntity + { + public string Id { get; set; } + public string Name { get; set; } + public string Url { get; set; } + public string StatusesCount { get; set; } = "0"; + public string LastStatusAt { get; set; } + } + + // Mastodon's v2 Suggestion: why (source, deprecated since 4.3, and sources) and whom + public class Suggestion + { + public string Source { get; set; } + public List Sources { get; set; } = new(); + public Account Account { get; set; } + } + public class Notification { public string Id { get; set; } diff --git a/PrivaPub/Controllers/ClientToServer/InstanceAdminController.cs b/PrivaPub/Controllers/ClientToServer/InstanceAdminController.cs index 8090499..1c689d1 100644 --- a/PrivaPub/Controllers/ClientToServer/InstanceAdminController.cs +++ b/PrivaPub/Controllers/ClientToServer/InstanceAdminController.cs @@ -275,6 +275,7 @@ namespace PrivaPub.Controllers.ClientToServer DeletedAt = r.DeletedAt }).ToList(), Relays = await RelayViews(settings, token), + StaffPicks = await StaffPickViews(settings, token), Media = new ViewMediaLimits { QuotaBytesPerRoot = limits.QuotaBytesPerRoot, @@ -387,6 +388,103 @@ namespace PrivaPub.Controllers.ClientToServer #endregion + #region staff picks + + // the accounts suggested first to every persona (completion program R8, Domain/Social/Suggestions): a persona here or an + // account elsewhere the administrators choose, in their order; kept in the settings, changed only here, every change in + // the staff's log (an entry about a persona of the acting login keeps no actor, ModerationLog) + [HttpGet, Route("/clientapi/admin/staff-picks")] + public async Task StaffPicks(CancellationToken token) + { + if (await Administrator(token) == default) + return Forbid(); + return Ok(await StaffPickViews(_settings.Current, token)); + } + + [HttpPost, Route("/clientapi/admin/staff-picks/insert")] + public async Task InsertStaffPick(StaffPickForm form, CancellationToken token) + { + var admin = await Administrator(token); + if (admin == default) + return Forbid(); + var id = form.AccountId?.Trim(); + var picked = await Pickable(id, token); + if (picked == default) + return Refused(StatusCodes.Status404NotFound, "No such account to suggest."); + if (_settings.Current.StaffPicks.Contains(id)) + return Refused(StatusCodes.Status409Conflict, "This account is suggested already."); + if (_settings.Current.StaffPicks.Count >= MaxStaffPicks) + return Refused(StatusCodes.Status422UnprocessableEntity, "40 accounts at most are suggested."); + var saved = await _settings.Update(settings => + { + if (!settings.StaffPicks.Contains(id)) + settings.StaffPicks.Add(id); + }, admin.ID, token); + await _log.Write(admin.ID, AdminActions.StaffPickInsert, picked.Value.IsLocal ? AdminTargets.Persona : AdminTargets.Account, id, picked.Value.Handle, + token: token); + return Ok(await StaffPickViews(saved, token)); + } + + [HttpPost, Route("/clientapi/admin/staff-picks/delete")] + public async Task DeleteStaffPick([FromQuery] string accountId, CancellationToken token) + { + var admin = await Administrator(token); + if (admin == default) + return Forbid(); + accountId = accountId?.Trim(); + if (string.IsNullOrEmpty(accountId) || !_settings.Current.StaffPicks.Contains(accountId)) + return NotFound(); + var view = (await StaffPickViews(_settings.Current, token)).FirstOrDefault(p => p.AccountId == accountId); + var saved = await _settings.Update(settings => settings.StaffPicks.Remove(accountId), admin.ID, token); + await _log.Write(admin.ID, AdminActions.StaffPickDelete, view?.IsLocal == true ? AdminTargets.Persona : AdminTargets.Account, accountId, + view?.Handle ?? accountId, token: token); + return Ok(await StaffPickViews(saved, token)); + } + + const int MaxStaffPicks = 40; + + // a persona that may be shown, or an account elsewhere that is a person (no group, feed or server's actor), neither + // suspended nor deleted + async Task<(bool IsLocal, string Handle)?> Pickable(string id, CancellationToken token) + { + if (string.IsNullOrEmpty(id) || id.Length > 64) + return default; + var avatar = await DB.Default.Find().Match(a => a.ID == id && !a.DeletionAt.HasValue && !a.SuspendedAt.HasValue).ExecuteFirstAsync(token); + if (avatar != default) + return (await Counted.HeldBackPersonas(token)).Contains(avatar.ID) ? ((bool, string)?)null : (true, _localActors.FromAvatar(avatar).Handle); + var foreign = await DB.Default.Find().Match(f => f.ID == id && !f.DeletionAt.HasValue && !f.SuspendedAt.HasValue + && (f.AvatarType == AvatarType.Person || f.AvatarType == AvatarType.Service || f.AvatarType == AvatarType.Organization)).ExecuteFirstAsync(token); + return foreign == default ? ((bool, string)?)null : (false, $"{foreign.UserName}@{foreign.Domain}"); + } + + async Task> StaffPickViews(ServerSettings settings, CancellationToken token) + { + var ids = (settings.StaffPicks ?? new()).ToList(); + var held = await Counted.HeldBackPersonas(token); + var avatars = (await DB.Default.Find().Match(a => ids.Contains(a.ID)).ExecuteAsync(token)).ToDictionary(a => a.ID); + var foreigns = (await DB.Default.Find().Match(f => ids.Contains(f.ID)).ExecuteAsync(token)).ToDictionary(f => f.ID); + return ids.Select(id => avatars.TryGetValue(id, out var avatar) + ? new ViewStaffPick + { + AccountId = id, + IsLocal = true, + Handle = _localActors.FromAvatar(avatar).Handle, + Name = avatar.Name, + IsShown = avatar.DeletionAt == default && avatar.SuspendedAt == default && avatar.SilencedAt == default && !held.Contains(id) + } + : foreigns.TryGetValue(id, out var foreign) + ? new ViewStaffPick + { + AccountId = id, + Handle = $"{foreign.UserName}@{foreign.Domain}", + Name = foreign.Name, + IsShown = foreign.DeletionAt == default && foreign.SuspendedAt == default && foreign.SilencedAt == default + } + : new ViewStaffPick { AccountId = id, IsShown = false }).ToList(); + } + + #endregion + #region relays [HttpGet, Route("/clientapi/admin/relays")] diff --git a/PrivaPub/Controllers/ClientToServer/PrivateAvatarController.cs b/PrivaPub/Controllers/ClientToServer/PrivateAvatarController.cs index 50a87cc..ad57121 100644 --- a/PrivaPub/Controllers/ClientToServer/PrivateAvatarController.cs +++ b/PrivaPub/Controllers/ClientToServer/PrivateAvatarController.cs @@ -74,6 +74,51 @@ namespace PrivaPub.Controllers.ClientToServer return Ok(); } + // how the persona's clients show what others post (R8): Mastodon's reading preferences, which its API only reads + [HttpGet, Route("/clientapi/avatar/private/preferences")] + public async Task Preferences([FromQuery] string avatarId, [FromServices] StaticServices.DbEntities dbEntities, CancellationToken token) + { + var avatar = await OwnPersona(avatarId, dbEntities, token); + return avatar == default + ? NotFound(new WebResult().Invalidate(_localizer["Avatar not found."], StatusCodes.Status404NotFound)) + : Ok(PreferencesOf(avatar)); + } + + [HttpPost, Route("/clientapi/avatar/private/preferences")] + public async Task SavePreferences(PersonaPreferencesForm form, [FromServices] StaticServices.DbEntities dbEntities, CancellationToken token) + { + if (!ModelState.IsValid) + return BadRequest(new WebResult().Invalidate(_localizer["Invalid model."])); + if (!Models.User.ExpandMediaModes.IsKnown(form.ExpandMedia)) + return UnprocessableEntity(new WebResult().Invalidate(_localizer["Media are shown as default, show_all or hide_all."], StatusCodes.Status422UnprocessableEntity)); + var avatar = await OwnPersona(form.AvatarId, dbEntities, token); + if (avatar == default) + return NotFound(new WebResult().Invalidate(_localizer["Avatar not found."], StatusCodes.Status404NotFound)); + await MongoDB.Entities.DB.Default.Update().MatchID(avatar.ID) + .Modify(a => a.Settings.ExpandMedia, form.ExpandMedia) + .Modify(a => a.Settings.ExpandSpoilers, form.ExpandSpoilers) + .ExecuteAsync(token); + avatar.Settings.ExpandMedia = form.ExpandMedia; + avatar.Settings.ExpandSpoilers = form.ExpandSpoilers; + return Ok(PreferencesOf(avatar)); + } + + // one of the login's own personas, not deleted (a suspended one keeps its preferences for when it is back) + async Task OwnPersona(string avatarId, StaticServices.DbEntities dbEntities, CancellationToken token) + { + var rootId = User.GetUserId(); + if (string.IsNullOrEmpty(avatarId) || !await dbEntities.RootToAvatars.Match(ra => ra.RootId == rootId && ra.AvatarId == avatarId).ExecuteAnyAsync(token)) + return default; + return await dbEntities.Avatars.MatchID(avatarId).Match(a => !a.DeletionAt.HasValue).ExecuteFirstAsync(token); + } + + static ViewPersonaPreferences PreferencesOf(Models.User.Avatar avatar) => new() + { + AvatarId = avatar.ID, + ExpandMedia = Models.User.ExpandMediaModes.IsKnown(avatar.Settings.ExpandMedia) ? avatar.Settings.ExpandMedia : Models.User.ExpandMediaModes.Default, + ExpandSpoilers = avatar.Settings.ExpandSpoilers + }; + [HttpPost, Route("/clientapi/avatar/private/update")] public async Task UpdateAvatar(UpdateAvatarForm model) { diff --git a/PrivaPub/Domain/Portability/ExportArchiveJob.cs b/PrivaPub/Domain/Portability/ExportArchiveJob.cs index 06fe603..258ceea 100644 --- a/PrivaPub/Domain/Portability/ExportArchiveJob.cs +++ b/PrivaPub/Domain/Portability/ExportArchiveJob.cs @@ -128,7 +128,8 @@ namespace PrivaPub.Domain.Portability var follows = await DB.Default.Find().Match(f => f.AvatarId == actor.Id).ExecuteAsync(token); var csv = new StringBuilder("Account address,Show boosts,Notify on new posts,Languages\n"); foreach (var follow in follows) - csv.Append($"{Csv(await handles.Of(follow.TargetActorURI, token))},{(follow.ShowReblogs ? "true" : "false")},false,\n"); + csv.Append($"{Csv(await handles.Of(follow.TargetActorURI, token))},{(follow.ShowReblogs ? "true" : "false")},{(follow.Notify ? "true" : "false")}," + + $"{Csv(string.Join(", ", follow.Languages ?? new List()))}\n"); await Text(zip, "following_accounts.csv", csv.ToString(), token); csv.Clear(); diff --git a/PrivaPub/Domain/Portability/ImportArchiveJob.cs b/PrivaPub/Domain/Portability/ImportArchiveJob.cs index 1cb5c1e..ac66c18 100644 --- a/PrivaPub/Domain/Portability/ImportArchiveJob.cs +++ b/PrivaPub/Domain/Portability/ImportArchiveJob.cs @@ -707,7 +707,10 @@ namespace PrivaPub.Domain.Portability || await DB.Default.Find().Match(f => f.AvatarId == run.Me.Id && f.TargetActorURI == uri).ExecuteAnyAsync(token)) return false; var boosts = cells.Length < 2 || !cells[1].Trim().Equals("false", StringComparison.OrdinalIgnoreCase); - return await run.Services.GetRequiredService().FollowAs(run.Me, uri, boosts, token) != default; + // Mastodon's columns: notify on new posts, and the languages (", " between them) its posts reach the home in (R8) + var notify = cells.Length >= 3 && cells[2].Trim().Equals("true", StringComparison.OrdinalIgnoreCase); + var languages = cells.Length >= 4 ? cells[3].Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries) : []; + return await run.Services.GetRequiredService().FollowAs(run.Me, uri, boosts, token, notify, languages) != default; } // lists by title, with only the accounts the persona follows diff --git a/PrivaPub/Domain/Social/FeaturedTags.cs b/PrivaPub/Domain/Social/FeaturedTags.cs new file mode 100644 index 0000000..e0ae3c5 --- /dev/null +++ b/PrivaPub/Domain/Social/FeaturedTags.cs @@ -0,0 +1,264 @@ +using MongoDB.Driver; +using MongoDB.Entities; + +using PrivaPub.Api.Mastodon.Controllers; +using PrivaPub.Federation.Actors; +using PrivaPub.Federation.Inbox; +using PrivaPub.Federation.Objects; +using PrivaPub.Federation.Outbox; +using PrivaPub.Federation.Rendering; +using PrivaPub.Infrastructure.Http; +using PrivaPub.Models.Post; +using PrivaPub.Models.Social; +using PrivaPub.Models.User; +using PrivaPub.StaticServices; + +using System.Text.Json; +using System.Text.Json.Nodes; + +using static PrivaPub.Federation.Objects.ActivityJson; + +using PostEntity = PrivaPub.Models.Post.Post; + +namespace PrivaPub.Domain.Social +{ + public interface IFeaturedTags + { + /// The persona features a hashtag (already featured, it is as it was): the row, or why not. + Task<(FeaturedTag Tag, string Error)> Feature(LocalActor persona, string name, CancellationToken token); + + /// The persona stops featuring a hashtag, named by the row's id or by its name: the row taken away, if any. + Task Unfeature(LocalActor persona, string id, string name, CancellationToken token); + + Task> Of(string avatarId, CancellationToken token); + + /// An account elsewhere features a hashtag (Add of a Hashtag) or stops (Remove). + Task Receive(JsonNode activity, ForeignAvatar owner, bool add, CancellationToken token); + + /// An account elsewhere's featuredTags collection, read again (with its counts, at most daily). + Task Sync(ForeignAvatar owner, CancellationToken token); + } + + // Featured hashtags (Mastodon's featured tags, completion program R8). A persona's are public: its featuredTags + // collection (/peasants/{name}/tattoos) lists them as Hashtag objects, and featuring one sends Add{Hashtag} to its + // followers, unfeaturing it Remove{Hashtag}, both with its `featured` collection (/trophies) as the target: that is how + // Mastodon sends them (its AddHashtagSerializer), and the only target Mastodon takes a Hashtag from. Received the same + // way, with either collection as the target, and read from an account's featuredTags collection with its counts. + public class FeaturedTags : IFeaturedTags + { + public const int MaxFeatured = 10;//the instance API's max_featured_tags, Mastodon's FeaturedTag::LIMIT + public const int MaxRemote = 20;//what we keep of an account elsewhere, whatever its server allows + + readonly IDeliveryService _delivery; + readonly IRemoteActorService _remoteActors; + + public FeaturedTags(IDeliveryService delivery, IRemoteActorService remoteActors) + { + _delivery = delivery; + _remoteActors = remoteActors; + } + + // a featured hashtag as Mastodon renders it (its HashtagSerializer): the link to this server's page of the tag, which + // is what our posts' Hashtag tags link to, and the name as the persona wrote it + public static JsonObject Hashtag(string baseAddress, FeaturedTag tag) => new() + { + ["type"] = "Hashtag", + ["href"] = ActivityPubRenderer.TagUrl(baseAddress, tag.Name), + ["name"] = "#" + (string.IsNullOrEmpty(tag.DisplayName) ? tag.Name : tag.DisplayName) + }; + + public async Task> Of(string avatarId, CancellationToken token) => + await DB.Default.Find().Match(t => t.AvatarId == avatarId).Sort(t => t.ID, Order.Ascending).ExecuteAsync(token); + + public async Task<(FeaturedTag Tag, string Error)> Feature(LocalActor persona, string name, CancellationToken token) + { + var display = name?.Trim().TrimStart('#'); + var normal = TagsController.Normalise(display); + if (!TagsController.IsHashtag(normal)) + return (default, "Validation failed: Tag is invalid"); + var existing = await DB.Default.Find().Match(t => t.AvatarId == persona.Id && t.Name == normal).ExecuteFirstAsync(token); + if (existing != default) + return (existing, default); + if (await DB.Default.CountAsync(t => t.AvatarId == persona.Id, token) >= MaxFeatured) + return (default, $"Validation failed: You can feature at most {MaxFeatured} hashtags"); + var tag = new FeaturedTag + { + AvatarId = persona.Id, + Name = normal, + DisplayName = display.Length <= 100 ? display : normal + }; + try + { + await DB.Default.SaveAsync(tag, token); + } + catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey) + { + return (await DB.Default.Find().Match(t => t.AvatarId == persona.Id && t.Name == normal).ExecuteFirstAsync(token), default); + } + await Tell(persona, tag, add: true, token); + return (tag, default); + } + + public async Task Unfeature(LocalActor persona, string id, string name, CancellationToken token) + { + var normal = id == default ? TagsController.Normalise(name) : default; + var tag = id != default + ? await DB.Default.Find().Match(t => t.ID == id && t.AvatarId == persona.Id).ExecuteFirstAsync(token) + : await DB.Default.Find().Match(t => t.AvatarId == persona.Id && t.Name == normal).ExecuteFirstAsync(token); + if (tag == default) + return default; + var removed = await DB.Default.DeleteAsync(t => t.ID == tag.ID); + if (removed.DeletedCount > 0) + await Tell(persona, tag, add: false, token); + return tag; + } + + // to the persona's followers, as Mastodon distributes it (AccountRawDistributionWorker); a suspended persona's goes + // nowhere (DeliveryService.HeldBack) + async Task Tell(LocalActor persona, FeaturedTag tag, bool add, CancellationToken token) + { + if (persona is not { IsFederated: true, Kind: Models.Federation.LocalActorKind.Person }) + return; + await _delivery.EnqueueToFollowers(persona, new JsonObject + { + ["@context"] = ActivityPubRenderer.Context(), + ["id"] = persona.ActivityUri($"{(add ? "feature" : "unfeature")}-tag-{tag.ID}-{DateTime.UtcNow.Ticks}"), + ["type"] = add ? "Add" : "Remove", + ["actor"] = persona.Uri, + ["object"] = Hashtag(persona.BaseAddress, tag), + ["target"] = persona.Featured, + ["to"] = new JsonArray(ActivityPubRenderer.Public), + ["cc"] = new JsonArray(persona.Followers) + }, token); + } + + // what posts carry the tag: a persona's own public and unlisted posts, or the public and unlisted posts we hold of an + // account elsewhere, as Mastodon counts a featured tag (statuses_count, last_status_at) + public static async Task> Usage(string avatarId, string actorUri, IReadOnlyCollection names, + DbEntities dbEntities, CancellationToken token) + { + var usage = new Dictionary(StringComparer.Ordinal); + foreach (var name in names.Distinct(StringComparer.Ordinal)) + { + var tag = name; + var query = avatarId != default + ? dbEntities.Posts.Match(p => p.GroupUserId == avatarId && !p.IsFederatedCopy) + : dbEntities.Posts.Match(p => p.ActorURI == actorUri && p.IsFederatedCopy); + query.Match(p => p.Tags.Contains(tag) && p.ReblogOfPostId == null + && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted)).Match(Privacy.VisibilityPolicy.IsShown); + var latest = await query.Sort(p => p.ID, Order.Descending).Limit(1).ExecuteFirstAsync(token); + if (latest == default) + { + usage[name] = (0, default); + continue; + } + var counted = avatarId != default + ? await DB.Default.CountAsync(f => f.Where(p => p.GroupUserId == avatarId && !p.IsFederatedCopy && p.Tags.Contains(tag) + && p.ReblogOfPostId == null && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted)) + & f.Where(Privacy.VisibilityPolicy.IsShown), token) + : await DB.Default.CountAsync(f => f.Where(p => p.ActorURI == actorUri && p.IsFederatedCopy && p.Tags.Contains(tag) + && p.ReblogOfPostId == null && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted)) + & f.Where(Privacy.VisibilityPolicy.IsShown), token); + usage[name] = (counted, latest.CreationDate); + } + return usage; + } + + public async Task Receive(JsonNode activity, ForeignAvatar owner, bool add, CancellationToken token) + { + var (normal, display) = Named(Value(activity["object"], "name")); + if (normal == default) + { + Arrival.Drop("unparseable"); + return; + } + if (!add) + { + var removed = await DB.Default.DeleteAsync(t => t.ActorURI == owner.ActorURI && t.Name == normal); + if (removed is { IsAcknowledged: true, DeletedCount: > 0 }) + Arrival.Accept("tag-unfeatured"); + else + Arrival.Drop("unknown-tag"); + return; + } + if (await DB.Default.CountAsync(t => t.ActorURI == owner.ActorURI, token) >= MaxRemote) + { + Arrival.Drop("too-many-tags"); + return; + } + var first = await DB.Default.Find().Match(t => t.ActorURI == owner.ActorURI).Sort(t => t.Position, Order.Ascending).ExecuteFirstAsync(token); + try + { + await DB.Default.SaveAsync(new RemoteFeaturedTag + { + ActorURI = owner.ActorURI, + Name = normal, + DisplayName = display, + Href = Link(Value(activity["object"], "href")), + Position = (first?.Position ?? 0) - 1 + }, token); + Arrival.Accept("tag-featured"); + } + catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey) + { + Arrival.Drop("duplicate"); + } + } + + public async Task Sync(ForeignAvatar owner, CancellationToken token) + { + if (string.IsNullOrEmpty(owner.FeaturedTagsURL) || !Origin.Same(owner.FeaturedTagsURL, owner.ActorURI)) + return; + var items = await Items(owner, token); + if (items == default) + return; + var tags = new List(); + foreach (var item in items) + { + if (Value(item, "type") != "Hashtag" || Named(Value(item, "name")) is not ({ } normal, var display) || tags.Any(t => t.Name == normal)) + continue; + tags.Add(new RemoteFeaturedTag { ActorURI = owner.ActorURI, Name = normal, DisplayName = display, Href = Link(Value(item, "href")), Position = tags.Count }); + if (tags.Count >= MaxRemote) + break; + } + await DB.Default.DeleteAsync(t => t.ActorURI == owner.ActorURI); + if (tags.Count > 0) + await DB.Default.SaveAsync(tags, token); + } + + static (string Normal, string Display) Named(string name) + { + var display = name?.Trim().TrimStart('#'); + var normal = TagsController.Normalise(display); + return TagsController.IsHashtag(normal) ? (normal, display.Length <= 100 ? display : normal) : (default, default); + } + + static string Link(string href) => Uri.TryCreate(href, UriKind.Absolute, out var uri) && uri.Scheme is "https" or "http" ? href : default; + + // the collection's items, from it or its first page (its own server only), and the pages after, as Mastodon reads them + async Task> Items(ForeignAvatar owner, CancellationToken token) + { + using var scope = HttpScope.For("collection"); + var collection = await Fetch(owner.FeaturedTagsURL, token); + if (collection == default) + return default; + var items = new List(); + var page = collection["first"] is JsonObject embedded ? embedded + : Id(collection["first"]) is { } first && Origin.Same(first, owner.ActorURI) ? await Fetch(first, token) + : collection; + for (var pages = 0; page != default && pages < 3 && items.Count < MaxRemote; pages++) + { + if ((page["orderedItems"] ?? page["items"]) is JsonArray list) + items.AddRange(list.Where(i => i is JsonObject)); + page = Id(page["next"]) is { } next && Origin.Same(next, owner.ActorURI) ? await Fetch(next, token) : default; + } + return items; + } + + async Task Fetch(string uri, CancellationToken token) + { + using var fetched = await _remoteActors.FetchObject(uri, token); + return fetched == default || fetched.Root.ValueKind != JsonValueKind.Object ? default : JsonNode.Parse(fetched.Root.GetRawText()) as JsonObject; + } + } +} diff --git a/PrivaPub/Domain/Social/FollowService.cs b/PrivaPub/Domain/Social/FollowService.cs index bd81a9b..500e389 100644 --- a/PrivaPub/Domain/Social/FollowService.cs +++ b/PrivaPub/Domain/Social/FollowService.cs @@ -26,7 +26,10 @@ namespace PrivaPub.Domain.Social Task Follow(string rootUserId, FollowForm form, CancellationToken token); Task Unfollow(string rootUserId, FollowForm form, CancellationToken token); Task Followings(string rootUserId, string avatarId, CancellationToken token); - Task FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token); + /// Follows an account, or changes how a follow already made shows it: notify and languages (R8) change only + /// when given (languages empty: every language). + Task FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token, bool? notify = default, + IReadOnlyList languages = default); Task UnfollowAs(LocalActor follower, string target, CancellationToken token); Task Decide(LocalActor me, string followerAccountId, bool accept, CancellationToken token); Task ResendPending(DateTime now, CancellationToken token); @@ -93,7 +96,18 @@ namespace PrivaPub.Domain.Social } } - public async Task FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token) + // a follow's languages as kept: their primary subtags, lower case; none, every language (null) + public static List Languages(IReadOnlyList languages) + { + var kept = (languages ?? Array.Empty()) + .Select(l => l?.Trim().Replace('_', '-').Split('-')[0].ToLowerInvariant()) + .Where(l => l is { Length: >= 2 and <= 3 } && l.All(char.IsAsciiLetterLower)) + .Distinct().Take(20).ToList(); + return kept.Count == 0 ? null : kept; + } + + public async Task FollowAs(LocalActor follower, string target, bool showReblogs, CancellationToken token, bool? notify = default, + IReadOnlyList languages = default) { var (local, remote) = await ResolveTarget(target, token); if (local == default && remote == default) @@ -108,6 +122,17 @@ namespace PrivaPub.Domain.Social if (existing.ShowReblogs != showReblogs) await DB.Default.Update().MatchID(existing.ID).Modify(f => f.ShowReblogs, showReblogs).ExecuteAsync(token); existing.ShowReblogs = showReblogs; + if (notify is { } notifying && existing.Notify != notifying) + { + await DB.Default.Update().MatchID(existing.ID).Modify(f => f.Notify, notifying).ExecuteAsync(token); + existing.Notify = notifying; + } + if (languages != default) + { + var kept = Languages(languages); + await DB.Default.Update().MatchID(existing.ID).Modify(f => f.Languages, kept).ExecuteAsync(token); + existing.Languages = kept; + } // a request still unanswered is sent again, at most once an hour: a server can take a Follow (202) and // drop it later, as one that cannot yet read our actor does; one that holds it already ignores the copy if (existing.State == FollowState.Requested && remote != default) @@ -125,6 +150,8 @@ namespace PrivaPub.Domain.Social TargetIsLocal = local != default, TargetInboxURL = local?.Inbox ?? remote.InboxURL, ShowReblogs = showReblogs, + Notify = notify == true, + Languages = Languages(languages), State = local is { ManuallyApprovesFollowers: false } ? FollowState.Accepted : FollowState.Requested }; following.ID = (string)following.GenerateNewID(); @@ -157,6 +184,8 @@ namespace PrivaPub.Domain.Social await DB.Default.DeleteAsync(following.ID); await DB.Default.DeleteAsync(m => m.AvatarId == follower.Id && m.AccountId == following.TargetAccountId); + // an endorsement holds only while the follow does, as on Mastodon (R8) + await DB.Default.DeleteAsync(e => e.AvatarId == follower.Id && e.TargetActorURI == targetUri); if (following.TargetIsLocal) { await DB.Default.DeleteAsync(f => f.LocalActorId == following.TargetAccountId && f.ActorURI == follower.Uri); diff --git a/PrivaPub/Domain/Social/Suggestions.cs b/PrivaPub/Domain/Social/Suggestions.cs new file mode 100644 index 0000000..748beb5 --- /dev/null +++ b/PrivaPub/Domain/Social/Suggestions.cs @@ -0,0 +1,149 @@ +using MongoDB.Driver; +using MongoDB.Entities; + +using PrivaPub.Domain.Privacy; +using PrivaPub.Federation.Actors; +using PrivaPub.Models.Post; +using PrivaPub.Models.Social; +using PrivaPub.Models.User; + +using PostEntity = PrivaPub.Models.Post.Post; + +namespace PrivaPub.Domain.Social +{ + // Follow suggestions (completion program R8, Mastodon's /api/v1 and /api/v2/suggestions), best first: + // - staff: the accounts the administrators pick (ServerSettings.StaffPicks), Mastodon's `featured`; + // - past interactions: accounts the persona favourited, boosted or answered and does not follow; + // - global: accounts followed by at least GlobalMinimumFollowers personas here (`most_followed`), and personas here whose + // public posts of the last 30 days were favourited and boosted the most (`most_interactions`); discoverable ones only. + // Never the persona itself, never a sibling persona of its login (suggesting one would pair them the moment the persona + // followed it; the server knows the pair, the persona's suggestions must not use it), never an account it follows, asked + // to follow, blocked, muted, was blocked by or dismissed, and never an account held back (suspended, a banned login's) or + // silenced (the API mapper shows it `limited`). Nothing here is shown to anyone but the persona asking. + public static class Suggestions + { + public const string Staff = "staff"; + public const string PastInteractions = "past_interactions"; + public const string Global = "global"; + public const int GlobalMinimumFollowers = 3; + + const int Recent = 200; + const int GlobalSize = 100; + static readonly TimeSpan GlobalLifetime = TimeSpan.FromMinutes(10); + static (DateTime At, List<(string Id, string Why)> Ids) _global = (DateTime.MinValue, new()); + + public sealed record Candidate(string AccountId, string Source, List Sources); + + // every candidate for the persona, in order, each once (with every reason it has), none of those it must never see + public static async Task> For(LocalActor me, IReadOnlyList staffPicks, CancellationToken token) + { + var excluded = await Excluded(me, token); + var found = new List(); + var byId = new Dictionary(StringComparer.Ordinal); + void Add(string id, string source, string why) + { + if (string.IsNullOrEmpty(id) || excluded.Contains(id)) + return; + if (byId.TryGetValue(id, out var known)) + { + if (!known.Sources.Contains(why)) + known.Sources.Add(why); + return; + } + var candidate = new Candidate(id, source, new List { why }); + byId[id] = candidate; + found.Add(candidate); + } + foreach (var id in staffPicks ?? Array.Empty()) + Add(id, Staff, "featured"); + foreach (var id in await Interacted(me.Id, token)) + Add(id, PastInteractions, PastInteractions);//(Mastodon's sources has no word for it since 4.3; its source does) + foreach (var (id, why) in await GlobalCandidates(token)) + Add(id, Global, why); + return found; + } + + static async Task> Excluded(LocalActor me, CancellationToken token) + { + var excluded = new HashSet(StringComparer.Ordinal) { me.Id }; + var roots = (await DB.Default.Find().Match(r => r.AvatarId == me.Id).ExecuteAsync(token)).Select(r => r.RootId).ToList(); + if (roots.Count > 0) + excluded.UnionWith((await DB.Default.Find().Match(r => roots.Contains(r.RootId)).ExecuteAsync(token)).Select(r => r.AvatarId)); + excluded.UnionWith((await DB.Default.Find().Match(f => f.AvatarId == me.Id).Project(f => f.Include(x => x.TargetAccountId)) + .ExecuteAsync(token)).Select(f => f.TargetAccountId)); + excluded.UnionWith((await DB.Default.Find().Match(b => b.AvatarId == me.Id).ExecuteAsync(token)).Select(b => b.TargetAccountId)); + // (a persona here that blocked this one) + excluded.UnionWith((await DB.Default.Find().Match(b => b.TargetActorURI == me.Uri).ExecuteAsync(token)).Select(b => b.AvatarId)); + excluded.UnionWith((await DB.Default.Find().Match(b => b.AvatarId == me.Id).ExecuteAsync(token)).Select(b => b.AccountId)); + excluded.UnionWith((await DB.Default.Find().Match(m => m.AvatarId == me.Id).ExecuteAsync(token)).Select(m => m.TargetAccountId)); + excluded.UnionWith((await DB.Default.Find().Match(d => d.AvatarId == me.Id).ExecuteAsync(token)).Select(d => d.AccountId)); + excluded.UnionWith(await Counted.HeldBackPersonas(token)); + excluded.Remove(default); + return excluded; + } + + // the accounts whose posts the persona favourited, boosted or answered lately, the most often first + static async Task> Interacted(string avatarId, CancellationToken token) + { + var authors = new List<(string Id, int Age)>(); + var liked = (await DB.Default.Find().Match(f => f.AccountId == avatarId).Sort(f => f.ID, Order.Descending).Limit(Recent) + .ExecuteAsync(token)).Select(f => f.PostId).ToList(); + var boosted = (await DB.Default.Find().Match(p => p.AuthorAccountId == avatarId && p.ReblogOfPostId != null && !p.DeletedAt.HasValue) + .Sort(p => p.ID, Order.Descending).Limit(Recent).Project(p => p.Include(x => x.ReblogOfPostId)).ExecuteAsync(token)).Select(p => p.ReblogOfPostId).ToList(); + var interacted = liked.Concat(boosted).Distinct().ToList(); + var posts = interacted.Count == 0 ? new Dictionary() + : (await DB.Default.Find().Match(p => interacted.Contains(p.ID)) + .Project(p => p.Include(x => x.AuthorAccountId).Include(x => x.GroupUserId).Include(x => x.ReblogOfPostId)).ExecuteAsync(token)) + .ToDictionary(p => p.ID); + authors.AddRange(interacted.Select((id, age) => (posts.GetValueOrDefault(id) is { ReblogOfPostId: null } post ? post.AuthorAccountId ?? post.GroupUserId : default, age))); + var answered = await DB.Default.Find().Match(p => p.GroupUserId == avatarId && !p.IsFederatedCopy && p.InReplyToAccountId != null && !p.DeletedAt.HasValue) + .Sort(p => p.ID, Order.Descending).Limit(Recent).Project(p => p.Include(x => x.InReplyToAccountId)).ExecuteAsync(token); + authors.AddRange(answered.Select((p, age) => (p.InReplyToAccountId, age))); + return authors.Where(a => !string.IsNullOrEmpty(a.Id) && a.Id != avatarId) + .GroupBy(a => a.Id).OrderByDescending(g => g.Count()).ThenBy(g => g.Min(a => a.Age)).Select(g => g.Key).ToList(); + } + + // the same for every persona, worked out at most every ten minutes + static async Task> GlobalCandidates(CancellationToken token) + { + var cached = _global; + if (DateTime.UtcNow - cached.At < GlobalLifetime) + return cached.Ids; + var ids = new List<(string, string)>(); + + var followed = await DB.Default.Fluent() + .Match(f => f.State == FollowState.Accepted && f.FollowerKind == null) + .Group(f => f.TargetAccountId, g => new { Id = g.Key, Count = g.Count() }) + .Match(g => g.Count >= GlobalMinimumFollowers) + .SortByDescending(g => g.Count) + .Limit(GlobalSize) + .ToListAsync(token); + var followedIds = followed.Select(f => f.Id).ToList(); + var discoverable = (await DB.Default.Find().Match(a => followedIds.Contains(a.ID) && a.Settings.IsDiscoverable && !a.DeletionAt.HasValue + && !a.SilencedAt.HasValue && !a.SuspendedAt.HasValue).Project(a => a.Include(x => x.ID)).ExecuteAsync(token)).Select(a => a.ID) + .Concat((await DB.Default.Find().Match(f => followedIds.Contains(f.ID) && f.IsDiscoverable && !f.DeletionAt.HasValue + && !f.SilencedAt.HasValue && !f.SuspendedAt.HasValue + && (f.AvatarType == AvatarType.Person || f.AvatarType == AvatarType.Service || f.AvatarType == AvatarType.Organization)) + .Project(f => f.Include(x => x.ID)).ExecuteAsync(token)).Select(f => f.ID)) + .ToHashSet(StringComparer.Ordinal); + ids.AddRange(followed.Where(f => discoverable.Contains(f.Id)).Select(f => (f.Id, "most_followed"))); + + var since = DateTime.UtcNow.AddDays(-30); + var interactions = await DB.Default.Fluent() + .Match(p => !p.IsFederatedCopy && p.GroupUserId != null && p.Visibility == PostVisibility.Public && p.ReblogOfPostId == null + && !p.DeletedAt.HasValue && !p.AuthorLimited && !p.AuthorSuspended && p.CreationDate >= since) + .Group(p => p.GroupUserId, g => new { Id = g.Key, Count = g.Sum(p => p.FavouritesCount + p.ReblogsCount) }) + .Match(g => g.Count > 0) + .SortByDescending(g => g.Count) + .Limit(GlobalSize) + .ToListAsync(token); + var active = interactions.Select(i => i.Id).ToList(); + var shown = (await DB.Default.Find().Match(a => active.Contains(a.ID) && a.Settings.IsDiscoverable && !a.DeletionAt.HasValue + && !a.SilencedAt.HasValue && !a.SuspendedAt.HasValue).Project(a => a.Include(x => x.ID)).ExecuteAsync(token)).Select(a => a.ID).ToHashSet(); + ids.AddRange(interactions.Where(i => shown.Contains(i.Id)).Select(i => (i.Id, "most_interactions"))); + + _global = (DateTime.UtcNow, ids); + return ids; + } + } +} diff --git a/PrivaPub/Domain/Statuses/StatusService.cs b/PrivaPub/Domain/Statuses/StatusService.cs index 28412f5..396ad7f 100644 --- a/PrivaPub/Domain/Statuses/StatusService.cs +++ b/PrivaPub/Domain/Statuses/StatusService.cs @@ -285,6 +285,8 @@ namespace PrivaPub.Domain.Statuses if (parent != default && Counted.Reply(post)) await DB.Default.Update().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token); await _fanout.Distribute(post, token); + if (post.Visibility == PostVisibility.Direct) + await Fanout.Conversation(post.ConversationId, token);//the `direct` streams (R8), now that it is saved await _polls.Scheduled(post, token); if (post.QuoteState == QuoteState.Accepted) await DB.Default.Update().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token); diff --git a/PrivaPub/Domain/Timelines/Fanout.cs b/PrivaPub/Domain/Timelines/Fanout.cs index 789e85d..014d14e 100644 --- a/PrivaPub/Domain/Timelines/Fanout.cs +++ b/PrivaPub/Domain/Timelines/Fanout.cs @@ -35,6 +35,8 @@ namespace PrivaPub.Domain.Timelines var recipients = new HashSet(StringComparer.Ordinal); if (!post.IsFederatedCopy && !string.IsNullOrEmpty(post.GroupUserId)) recipients.Add(post.GroupUserId); + // those following the author with notify (R8), told of the post once it reached their home + var notifying = new List(); switch (post.Visibility) { @@ -44,7 +46,11 @@ namespace PrivaPub.Domain.Timelines : await _dbEntities.Followings.Match(f => f.TargetAccountId == post.GroupUserId && f.TargetIsLocal && f.State == FollowState.Accepted).ExecuteAsync(token); foreach (var following in followers) if (await Shows(following, post, token)) + { recipients.Add(following.AvatarId); + if (following.Notify && Notifies(post)) + notifying.Add(following.AvatarId); + } if (!string.IsNullOrEmpty(post.AudienceURI)) // (the server's reader of feeds follows communities for nobody's home) foreach (var member in await _dbEntities.Followings.Match(f => f.TargetActorURI == post.AudienceURI && f.State == FollowState.Accepted && f.FollowerKind == null).ExecuteAsync(token)) @@ -100,6 +106,8 @@ namespace PrivaPub.Domain.Timelines if (string.IsNullOrEmpty(post.ReblogOfPostId)) foreach (var mention in post.Mentions.Where(m => m.IsLocal && !string.IsNullOrEmpty(m.AccountId))) await Notifications.Add(mention.AccountId, NotificationType.Mention, post.AuthorAccountId ?? post.GroupUserId, post.ActorURI, post.ID, token); + foreach (var avatarId in notifying.Where(recipients.Contains).Distinct()) + await Notifications.Add(avatarId, NotificationType.Status, post.AuthorAccountId ?? post.GroupUserId, post.ActorURI, post.ID, token); if (Streams.Anyone) await Stream(post, recipients, token); } @@ -186,10 +194,36 @@ namespace PrivaPub.Domain.Timelines } } + // Mastodon's notify: the account's own posts and its replies to itself, never its boosts (FeedInsertWorker) + static bool Notifies(Post post) => + string.IsNullOrEmpty(post.ReblogOfPostId) && post.Visibility is PostVisibility.Public or PostVisibility.Unlisted or PostVisibility.FollowersOnly + && (string.IsNullOrEmpty(post.InReplyToURI) || post.InReplyToAccountId == (post.AuthorAccountId ?? post.GroupUserId)); + + // a follow's languages (R8): a post in another language stays out of that home, as on Mastodon; one whose language + // is not known comes in + static bool InLanguages(Following following, Post post) => + following.Languages is not { Count: > 0 } || string.IsNullOrEmpty(post.Language) + || following.Languages.Contains(post.Language.Replace('_', '-').Split('-')[0].ToLowerInvariant()); + + // a direct conversation that has a new post, to the `direct` streams of the personas taking part in it (R8), as + // Mastodon's `conversation` event: each connection maps it for its own persona when it sends it + public static async Task Conversation(string conversationId, CancellationToken token) + { + if (!Streams.Anyone || string.IsNullOrEmpty(conversationId)) + return; + var conversation = await DB.Default.Find().MatchID(conversationId).ExecuteFirstAsync(token); + var told = new Streams.Event("conversation", conversationId); + foreach (var member in conversation?.Members.Where(m => !m.IsForeign).Select(m => m.AvatarId).Distinct() ?? Enumerable.Empty()) + if (Streams.Listening(member)) + Streams.ToPersona(member, new Streams.Key(Streams.Direct), told); + } + async Task Shows(Following following, Post post, CancellationToken token) { if (!string.IsNullOrEmpty(post.ReblogOfPostId)) return following.ShowReblogs; + if (!InLanguages(following, post)) + return false; var repliedTo = post.InReplyToAccountId; var author = post.AuthorAccountId ?? post.GroupUserId; if (string.IsNullOrEmpty(post.InReplyToURI) || repliedTo == author || repliedTo == following.AvatarId) diff --git a/PrivaPub/Federation/Actors/AccountCountsJob.cs b/PrivaPub/Federation/Actors/AccountCountsJob.cs index 7471dec..a523e02 100644 --- a/PrivaPub/Federation/Actors/AccountCountsJob.cs +++ b/PrivaPub/Federation/Actors/AccountCountsJob.cs @@ -13,6 +13,7 @@ namespace PrivaPub.Federation.Actors // followers, following and outbox collections), and the posts it features. Read when the account is fetched, at most // once a day, never when someone looks at it; a hidden or missing collection leaves its count unknown. Mastodon and // GoToSocial do the same. A feed's following is its communities: they are kept, and read in turn, so its list shows. + // Its featured hashtags (its featuredTags collection, R8) are read with them. public class AccountCountsJob : IJobHandler { readonly IRemoteActorService _remoteActors; @@ -20,12 +21,14 @@ namespace PrivaPub.Federation.Actors readonly IFeaturedPosts _featured; readonly ILocalActorService _localActors; readonly IJobQueue _jobs; + readonly Domain.Social.IFeaturedTags _featuredTags; public const int MaxFeedCommunities = 50; public AccountCountsJob(IRemoteActorService remoteActors, DbEntities dbEntities, IFeaturedPosts featured = default, - ILocalActorService localActors = default, IJobQueue jobs = default) + ILocalActorService localActors = default, IJobQueue jobs = default, Domain.Social.IFeaturedTags featuredTags = default) { + _featuredTags = featuredTags; _remoteActors = remoteActors; _dbEntities = dbEntities; _featured = featured; @@ -66,6 +69,8 @@ namespace PrivaPub.Federation.Actors } if (_featured != default) await _featured.Sync(actor, token); + if (_featuredTags != default) + await _featuredTags.Sync(actor, token); return JobOutcome.Done; } diff --git a/PrivaPub/Federation/Actors/ActorDocument.cs b/PrivaPub/Federation/Actors/ActorDocument.cs index 6532753..bf5d049 100644 --- a/PrivaPub/Federation/Actors/ActorDocument.cs +++ b/PrivaPub/Federation/Actors/ActorDocument.cs @@ -24,6 +24,7 @@ namespace PrivaPub.Federation.Actors public string Followers { get; init; } public string Following { get; init; } public string Featured { get; init; } + public string FeaturedTags { get; init; } public string Wall { get; init; } public string WebFinger { get; init; }//FEP-2c59: its handle, user@domain, where the domain is not its host's public string SharedInbox { get; init; } @@ -82,6 +83,7 @@ namespace PrivaPub.Federation.Actors Followers = RemoteActorService.Text(root, "followers"), Following = RemoteActorService.Text(root, "following"), Featured = RemoteActorService.Text(root, "featured"), + FeaturedTags = RemoteActorService.Text(root, "featuredTags"), Wall = RemoteActorService.Text(root, "wall") ?? RemoteActorService.Text(root, "sm:wall"), WebFinger = RemoteActorService.Text(root, "webfinger")?.Replace("acct:", "", StringComparison.OrdinalIgnoreCase).TrimStart('@'), SharedInbox = root.TryGetProperty("endpoints", out var endpoints) ? RemoteActorService.Text(endpoints, "sharedInbox") : default, diff --git a/PrivaPub/Federation/Actors/RemoteActorService.cs b/PrivaPub/Federation/Actors/RemoteActorService.cs index efd412b..379331d 100644 --- a/PrivaPub/Federation/Actors/RemoteActorService.cs +++ b/PrivaPub/Federation/Actors/RemoteActorService.cs @@ -228,6 +228,7 @@ namespace PrivaPub.Federation.Actors .Modify(a => a.FollowersURL, actor.Followers) .Modify(a => a.FollowingURL, actor.Following) .Modify(a => a.FeaturedURL, Origin.Same(actor.Featured, actor.Id) ? actor.Featured : default) + .Modify(a => a.FeaturedTagsURL, Origin.Same(actor.FeaturedTags, actor.Id) ? actor.FeaturedTags : default) .Modify(a => a.WallURL, Origin.Same(actor.Wall, actor.Id) ? actor.Wall : default) .Modify(a => a.AssertionKeys, actor.AssertionKeys) .Modify(a => a.SharedInboxURL, actor.SharedInbox) diff --git a/PrivaPub/Federation/Controllers/PeasantsController.cs b/PrivaPub/Federation/Controllers/PeasantsController.cs index 0b15768..223e2f2 100644 --- a/PrivaPub/Federation/Controllers/PeasantsController.cs +++ b/PrivaPub/Federation/Controllers/PeasantsController.cs @@ -244,13 +244,21 @@ namespace PrivaPub.Federation.Controllers return Activity(ActivityPubRenderer.OrderedCollection(local.Featured, notes.Count, notes)); } + // the hashtags a persona features (R8), as Hashtag objects in the order it featured them, as Mastodon's featured tags + // collection lists them (Mastodon reads Collection and OrderedCollection alike); a group features none [HttpGet, Route("{actor}/tattoos")] public async Task FeaturedTags(string actor, CancellationToken token) { var local = await _localActors.FindByUserName(actor, token); - return local is not { IsFederated: true } - ? NotFound() - : Activity(ActivityPubRenderer.OrderedCollection(local.FeaturedTags, 0, Enumerable.Empty())); + if (local is not { IsFederated: true }) + return NotFound(); + var tags = local.Kind == LocalActorKind.Person + ? await DB.Default.Find().Match(t => t.AvatarId == local.Id).Sort(t => t.ID, Order.Ascending).ExecuteAsync(token) + : new List(); + var collection = ActivityPubRenderer.OrderedCollection(local.FeaturedTags, tags.Count, + tags.Select(t => (JsonNode)Domain.Social.FeaturedTags.Hashtag(local.BaseAddress, t))); + collection["@context"] = ActivityPubRenderer.Context();//(Hashtag is Mastodon's term, defined there) + return Activity(collection); } [HttpGet, Route("{actor}/scribbles/{postId}")] diff --git a/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs b/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs index 4e333c1..50d0512 100644 --- a/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs @@ -256,7 +256,10 @@ namespace PrivaPub.Federation.Inbox.Handlers await _fanout.Distribute(post, token); await ReplyRelay.Pass(post, _dbEntities, _localActors, _delivery, token); if (conversation != default) + { await Domain.Statuses.ConversationStates.Posted(conversation.ID, post.ID, default, token); + await Domain.Timelines.Fanout.Conversation(conversation.ID, token);//the `direct` streams (R8), once it is the newest + } if (group is { IsCircle: false }) await _groups.Announce(group, activity.AsObject(), note.Id, isNewPost: true, token); if (wall != default) diff --git a/PrivaPub/Federation/Inbox/Handlers/FeaturedHandler.cs b/PrivaPub/Federation/Inbox/Handlers/FeaturedHandler.cs index 13597d7..70e80d5 100644 --- a/PrivaPub/Federation/Inbox/Handlers/FeaturedHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/FeaturedHandler.cs @@ -6,20 +6,23 @@ using System.Text.Json.Nodes; namespace PrivaPub.Federation.Inbox.Handlers { // An account pins one of its posts (Add to its `featured` collection) or unpins it (Remove), as Mastodon and Akkoma - // send it; inside a community's announce, the community features a post made in it. On its wall (Smithereen's, FEP-400e) - // it tells of a post someone wrote there, or takes it off. Any other target (a community's moderators) is not ours to - // keep. + // send it; inside a community's announce, the community features a post made in it. A Hashtag added to or removed from + // its `featured` collection (as Mastodon sends it) or its `featuredTags` is a featured tag (R8). On its wall + // (Smithereen's, FEP-400e) it tells of a post someone wrote there, or takes it off. Any other target (a community's + // moderators) is not ours to keep. public class FeaturedHandler : IActivityHandler { readonly IFeaturedPosts _featured; readonly IWalls _walls; readonly IRemoteActorService _remoteActors; readonly Infrastructure.Jobs.IJobQueue _queue; + readonly Domain.Social.IFeaturedTags _tags; readonly bool _add; public FeaturedHandler(IFeaturedPosts featured, bool add, IWalls walls = default, IRemoteActorService remoteActors = default, - Infrastructure.Jobs.IJobQueue queue = default) + Infrastructure.Jobs.IJobQueue queue = default, Domain.Social.IFeaturedTags tags = default) { + _tags = tags; _featured = featured; _walls = walls; _remoteActors = remoteActors; @@ -36,9 +39,18 @@ namespace PrivaPub.Federation.Inbox.Handlers var target = Objects.ActivityJson.Id(activity["target"]); // a collection of its own we do not know of: its document read again, at most hourly, in case it was kept before // we read that collection (a wall) - if (_remoteActors != default && target != default && target != actor.FeaturedURL && target != actor.WallURL + if (_remoteActors != default && target != default && target != actor.FeaturedURL && target != actor.WallURL && target != actor.FeaturedTagsURL && Objects.Origin.Same(target, actor.ActorURI) && actor.UpdatedAt < DateTime.UtcNow.AddHours(-1)) actor = await _remoteActors.GetActor(actor.ActorURI, refresh: true, token) ?? actor; + // a featured hashtag: an object of its own, with no id (its href is no post), so before anything reads it as one + if (Objects.ActivityJson.Value(activity["object"], "type") == "Hashtag") + { + if (_tags != default && target != default && (target == actor.FeaturedURL || target == actor.FeaturedTagsURL)) + await _tags.Receive(activity, actor, _add, token); + else + Arrival.Drop("unknown-target"); + return; + } if (_walls != default && !string.IsNullOrEmpty(actor.WallURL) && target == actor.WallURL) await _walls.Receive(activity, actor, _add, token); else diff --git a/PrivaPub/Federation/Inbox/Handlers/MoveHandler.cs b/PrivaPub/Federation/Inbox/Handlers/MoveHandler.cs index 232ddf7..d0c099e 100644 --- a/PrivaPub/Federation/Inbox/Handlers/MoveHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/MoveHandler.cs @@ -73,10 +73,19 @@ namespace PrivaPub.Federation.Inbox.Handlers if (persona == default) continue; var lists = await DB.Default.Find().Match(m => m.AvatarId == persona.Id && m.AccountId == old.ID).ExecuteAsync(token); - var followed = await _follows.FollowAs(persona, moved.ActorURI, following.ShowReblogs, token); + var endorsed = await DB.Default.Find().Match(e => e.AvatarId == persona.Id && e.TargetActorURI == old.ActorURI).ExecuteAnyAsync(token); + var followed = await _follows.FollowAs(persona, moved.ActorURI, following.ShowReblogs, token, following.Notify, following.Languages ?? new List()); await _follows.UnfollowAs(persona, old.ActorURI, token); if (followed == default) continue; + // the persona's endorsement and private note go with the follow, as Mastodon carries them over (R8) + if (endorsed) + await DB.Default.Update().Match(e => e.AvatarId == persona.Id && e.TargetAccountId == moved.ID) + .Modify(e => e.TargetActorURI, moved.ActorURI).Modify(b => b.SetOnInsert(e => e.CreatedAt, DateTime.UtcNow)) + .Option(o => o.IsUpsert = true).ExecuteAsync(token); + if (await DB.Default.Find().Match(n => n.AvatarId == persona.Id && n.TargetAccountId == old.ID).ExecuteFirstAsync(token) is { } note + && !await DB.Default.Find().Match(n => n.AvatarId == persona.Id && n.TargetAccountId == moved.ID).ExecuteAnyAsync(token)) + await DB.Default.SaveAsync(new PersonaNote { AvatarId = persona.Id, TargetAccountId = moved.ID, Text = note.Text }, token); foreach (var member in lists) try { diff --git a/PrivaPub/Infrastructure/Backup/ProtectiveMerge.cs b/PrivaPub/Infrastructure/Backup/ProtectiveMerge.cs index bcfa1c4..deb40c0 100644 --- a/PrivaPub/Infrastructure/Backup/ProtectiveMerge.cs +++ b/PrivaPub/Infrastructure/Backup/ProtectiveMerge.cs @@ -102,7 +102,8 @@ namespace PrivaPub.Infrastructure.Backup report.PersonasDeleted++; await avatars.ReplaceOneAsync(Id(avatar["_id"]), avatar, cancellationToken: token); // what RootRemoval takes with a persona, which the backup still had - foreach (var collection in new[] { "PersonaListMember", "PersonaList", "PersonaFilter", "FollowedTag", "ScheduledStatus" }) + foreach (var collection in new[] { "PersonaListMember", "PersonaList", "PersonaFilter", "FollowedTag", "ScheduledStatus", + "FeaturedTag", "Endorsement", "PersonaNote", "SuggestionDismissal" }) await database.GetCollection(collection).DeleteManyAsync(new BsonDocument("AvatarId", avatar["_id"].ToString()), token); continue; } diff --git a/PrivaPub/Infrastructure/Data/Indexes.cs b/PrivaPub/Infrastructure/Data/Indexes.cs index 9f9098e..379f42e 100644 --- a/PrivaPub/Infrastructure/Data/Indexes.cs +++ b/PrivaPub/Infrastructure/Data/Indexes.cs @@ -59,6 +59,20 @@ namespace PrivaPub.Infrastructure.Data await Plain(token, r => r.AvatarId); // the server's custom emoji, one per shortcode whatever its case (a new collection, R7) await Unique(e => e.Key, Builders.Filter.Gt(e => e.Key, ""), token); + // what the Mastodon API keeps per persona (new collections, R8): one featured hashtag per persona and name, one per + // account elsewhere and name, one endorsement, note and dismissal per persona and account + await DB.Default.Index().Key(t => t.AvatarId, KeyType.Ascending).Key(t => t.Name, KeyType.Ascending) + .Option(o => o.Unique = true).CreateAsync(token); + await DB.Default.Index().Key(t => t.ActorURI, KeyType.Ascending).Key(t => t.Name, KeyType.Ascending) + .Option(o => o.Unique = true).CreateAsync(token); + await DB.Default.Index().Key(e => e.AvatarId, KeyType.Ascending).Key(e => e.TargetAccountId, KeyType.Ascending) + .Option(o => o.Unique = true).CreateAsync(token); + await Plain(token, e => e.TargetAccountId); + await DB.Default.Index().Key(n => n.AvatarId, KeyType.Ascending).Key(n => n.TargetAccountId, KeyType.Ascending) + .Option(o => o.Unique = true).CreateAsync(token); + await Plain(token, n => n.TargetAccountId); + await DB.Default.Index().Key(d => d.AvatarId, KeyType.Ascending).Key(d => d.AccountId, KeyType.Ascending) + .Option(o => o.Unique = true).CreateAsync(token); // the staff's log, read newest first await DB.Default.Index().Key(a => a.At, KeyType.Descending).CreateAsync(token); diff --git a/PrivaPub/Middleware/SocialPubConfigurations.cs b/PrivaPub/Middleware/SocialPubConfigurations.cs index 9ff6e92..1e49039 100644 --- a/PrivaPub/Middleware/SocialPubConfigurations.cs +++ b/PrivaPub/Middleware/SocialPubConfigurations.cs @@ -94,12 +94,13 @@ namespace PrivaPub.Middleware .AddSingleton() .AddSingleton() .AddSingleton() + .AddSingleton() .AddSingleton(services => new FeaturedHandler(services.GetRequiredService(), add: true, services.GetRequiredService(), services.GetRequiredService(), - services.GetRequiredService())) + services.GetRequiredService(), services.GetRequiredService())) .AddSingleton(services => new FeaturedHandler(services.GetRequiredService(), add: false, services.GetRequiredService(), services.GetRequiredService(), - services.GetRequiredService())) + services.GetRequiredService(), services.GetRequiredService())) .AddSingleton() .AddSingleton() .AddSingleton() diff --git a/PrivaPub/Models/AdminAction.cs b/PrivaPub/Models/AdminAction.cs index e86b392..da5ca93 100644 --- a/PrivaPub/Models/AdminAction.cs +++ b/PrivaPub/Models/AdminAction.cs @@ -64,6 +64,9 @@ namespace PrivaPub.Models public const string EmojiInsert = "emoji.insert"; public const string EmojiUpdate = "emoji.update"; public const string EmojiDelete = "emoji.delete"; + // the accounts suggested to every persona (completion program R8) + public const string StaffPickInsert = "staff_pick.insert"; + public const string StaffPickDelete = "staff_pick.delete"; } public static class AdminTargets diff --git a/PrivaPub/Models/ServerSettings.cs b/PrivaPub/Models/ServerSettings.cs index 28a7d77..8773997 100644 --- a/PrivaPub/Models/ServerSettings.cs +++ b/PrivaPub/Models/ServerSettings.cs @@ -33,7 +33,7 @@ namespace PrivaPub.Models public List Rules { get; set; } = new();//soft-deleted ones stay, so old reports keep their meaning public List Relays { get; set; } = new();//besides Federation:Relays public ServerMediaLimits Media { get; set; } = new(); - public List StaffPicks { get; set; } = new();//account ids, for later + public List StaffPicks { get; set; } = new();//Mastodon account ids (Avatar.ID, ForeignAvatar.ID) suggested first to every persona (R8) public bool PublishActivity { get; set; }//GET /api/v1/instance/activity; off by default public DateTime? UpdatedAt { get; set; }//null: never saved public string UpdatedBy { get; set; }//the administrator's root id, or "cli" diff --git a/PrivaPub/Models/Social/Featuring.cs b/PrivaPub/Models/Social/Featuring.cs new file mode 100644 index 0000000..ed4cf77 --- /dev/null +++ b/PrivaPub/Models/Social/Featuring.cs @@ -0,0 +1,64 @@ +using MongoDB.Entities; + +namespace PrivaPub.Models.Social +{ + // What the Mastodon API keeps per persona since the completion program's R8. Every row here is one persona's, never its + // login's: siblings of one login must never be linkable, so nothing is kept per root. + + // A hashtag a persona features on its profile (Mastodon's featured tags). Public: its featuredTags collection + // (/peasants/{name}/tattoos) lists it, and featuring or unfeaturing it is told to its followers as Add or Remove of a + // Hashtag. One per persona and name. + public class FeaturedTag : Entity + { + public string AvatarId { get; set; } + public string Name { get; set; }//normalised: no #, NFKC, lower case (TagsController.Normalise), as posts keep their tags + public string DisplayName { get; set; }//as the persona wrote it (it normalises to Name) + public DateTime CreatedAt { get; set; } = DateTime.UtcNow; + } + + // A hashtag an account elsewhere features: its featuredTags collection, read with its counts at most once a day + // (AccountCountsJob), and the Add and Remove of a Hashtag it sends. Shown as its featured tags here; the lowest Position + // first. + public class RemoteFeaturedTag : Entity + { + public string ActorURI { get; set; } + public string Name { get; set; }//normalised, as FeaturedTag.Name + public string DisplayName { get; set; } + public string Href { get; set; }//the Hashtag's own link (its server's page of the account's posts with it), when it gave one + public int Position { get; set; } + public DateTime CreatedAt { get; set; } = DateTime.UtcNow; + } + + // An account a persona endorses (Mastodon's account pins, 4.4's endorsements): only one it follows, and only while it + // does. Kept for the persona alone: never federated as featured accounts and never shown to anyone else (the owner's + // default: endorsements stay private). + public class Endorsement : Entity + { + public string AvatarId { get; set; } + public string TargetAccountId { get; set; }//Avatar.ID, Group.ID or ForeignAvatar.ID + public string TargetActorURI { get; set; } + public DateTime CreatedAt { get; set; } = DateTime.UtcNow; + } + + // A persona's private note on an account (POST /api/v1/accounts/:id/note): its own, never its login's. RootUserNote is + // keyed by the root and would show a note made as one persona to its siblings. Never federated. + public class PersonaNote : Entity + { + public const int MaxLength = 2000;//Mastodon's AccountNote limit + + public string AvatarId { get; set; } + public string TargetAccountId { get; set; } + public string Text { get; set; } + public DateTime CreatedAt { get; set; } = DateTime.UtcNow; + public DateTime UpdatedAt { get; set; } = DateTime.UtcNow; + } + + // An account a persona took out of its follow suggestions (DELETE /api/v1/suggestions/:account_id): never suggested to + // that persona again. + public class SuggestionDismissal : Entity + { + public string AvatarId { get; set; } + public string AccountId { get; set; } + public DateTime CreatedAt { get; set; } = DateTime.UtcNow; + } +} diff --git a/PrivaPub/Models/Social/Following.cs b/PrivaPub/Models/Social/Following.cs index 582e2a7..ad20d18 100644 --- a/PrivaPub/Models/Social/Following.cs +++ b/PrivaPub/Models/Social/Following.cs @@ -13,6 +13,8 @@ namespace PrivaPub.Models.Social public FollowState State { get; set; } public string FollowActivityURI { get; set; } public bool ShowReblogs { get; set; } = true; + public bool Notify { get; set; }//Mastodon's notify (R8): a `status` notification for each new post of the account + public List Languages { get; set; }//Mastodon's languages (R8): only its posts in these reach the home; null, every language public DateTime CreatedAt { get; set; } = DateTime.UtcNow; public int Resent { get; set; }//how many times a request still unanswered was sent again (FollowService.ResendPending) public DateTime? ResentAt { get; set; } diff --git a/PrivaPub/Models/Social/Notification.cs b/PrivaPub/Models/Social/Notification.cs index b506e41..608e784 100644 --- a/PrivaPub/Models/Social/Notification.cs +++ b/PrivaPub/Models/Social/Notification.cs @@ -33,6 +33,7 @@ namespace PrivaPub.Models.Social Poll, Reaction, Quote, - ModerationWarning + ModerationWarning, + Status//a new post of an account the persona follows with notify (R8) } } diff --git a/PrivaPub/Models/User/Avatar.cs b/PrivaPub/Models/User/Avatar.cs index 3687f08..898b282 100644 --- a/PrivaPub/Models/User/Avatar.cs +++ b/PrivaPub/Models/User/Avatar.cs @@ -78,6 +78,7 @@ namespace PrivaPub.Models.User public string InboxURL { get; set; } public string OutboxURL { get; set; } public string FeaturedURL { get; set; }//featured: the posts it pins + public string FeaturedTagsURL { get; set; }//featuredTags: the hashtags it features (R8) public List Flairs { get; set; } = new();//a community's labels for its posts public List FeedCommunities { get; set; } = new();//a feed's communities, its `following` (Lemmy's multi-communities, PieFed's feeds) public List SearchableBy { get; set; }//FEP-268d, when the account states it: outranks IsIndexable @@ -127,6 +128,19 @@ namespace PrivaPub.Models.User public bool DefaultSensitive { get; set; } public string DefaultLanguage { get; set; } public string QuotePolicy { get; set; } = QuotePolicies.Public;//owner decision: anyone may quote a public post, automatically + // how its clients show what others post (R8, Mastodon's reading:expand:media and :spoilers in GET /api/v1/preferences), + // set through /clientapi/avatar/private/preferences, since Mastodon's API sets neither + public string ExpandMedia { get; set; } = ExpandMediaModes.Default; + public bool ExpandSpoilers { get; set; } + } + + public static class ExpandMediaModes + { + public const string Default = "default";//hide media marked sensitive + public const string ShowAll = "show_all"; + public const string HideAll = "hide_all"; + + public static bool IsKnown(string value) => value is Default or ShowAll or HideAll; } public static class QuotePolicies diff --git a/PrivaPub/Services/PersonaRemoval.cs b/PrivaPub/Services/PersonaRemoval.cs index afc77d5..c8114b1 100644 --- a/PrivaPub/Services/PersonaRemoval.cs +++ b/PrivaPub/Services/PersonaRemoval.cs @@ -61,7 +61,8 @@ namespace PrivaPub.Services // personas always were: its Mastodon tokens end first, each group it owns goes (GroupRemoval), a Delete of the actor goes // to its followers and to the accounts it follows, its posts are emptied, and its name stays reserved while its documents // answer 410. What it kept for itself (lists, filters, followed hashtags, scheduled posts, what it did with the - // announcements, the feeds it read, its media) goes with it. Its link to its login stays, so the login keeps no other + // announcements, the feeds it read, its featured hashtags, endorsements, notes and dismissed suggestions, its media) + // goes with it. Its link to its login stays, so the login keeps no other // trace than that. public class PersonaRemoval : IPersonaRemoval { @@ -113,6 +114,13 @@ namespace PrivaPub.Services await DB.Default.DeleteAsync(r => r.AvatarId == avatar.ID); // (the reader of feeds stops following what nobody reads any more at its next round) await DB.Default.DeleteAsync(s => s.AvatarId == avatar.ID); + // what the Mastodon API kept for it (R8): its featured hashtags (the Delete of the actor takes them from other + // servers), its endorsements, its private notes and its dismissed suggestions, and what other personas here kept + // about it: their endorsements of it and their notes on it + await DB.Default.DeleteAsync(t => t.AvatarId == avatar.ID); + await DB.Default.DeleteAsync(e => e.AvatarId == avatar.ID || e.TargetAccountId == avatar.ID); + await DB.Default.DeleteAsync(n => n.AvatarId == avatar.ID || n.TargetAccountId == avatar.ID); + await DB.Default.DeleteAsync(d => d.AvatarId == avatar.ID || d.AccountId == avatar.ID); // nothing of it publishes later, and none of its files stays served: its posts' media, its pictures and what its // scheduled posts held await DB.Default.DeleteAsync(s => s.AvatarId == avatar.ID); diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 4abf240..7fc2eb3 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -138,6 +138,22 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati 410; what posts and profiles render is the emoji as they are now, and no `Update` is sent when one changes; its id is served unsigned under SecureMode; a restore keeps the live emoji, as it keeps the settings (one deleted since stays deleted). + - R8 the Mastodon API made real (built 2026-10-18, not yet deployed; `tools/pasture/scenarios/featured.sh`): featured + tags (`/api/v1/featured_tags`, suggestions, `tags/:name/feature`, accounts' featured tags, `max_featured_tags` 10; + `/tattoos` a collection of `Hashtag`s, `Add`/`Remove{Hashtag}` sent to followers and received, remote accounts' + `featuredTags` read with their counts); endorsements (`pin`/`unpin`, `endorse`/`unendorse`, `/api/v1/endorsements`); + private notes per persona (`PersonaNote`); the relationship's `notifying`, `languages`, `endorsed` and `note`, with + `follow`'s `notify` (a `status` notification) and `languages` (filtering the home); preferences' reading settings; + suggestions v1 and v2 with dismissals and the administrators' staff picks; the `direct` stream's `conversation` + events; translate, annual reports and the admin API answering. Chosen while building it, for the owner to confirm: + **featured tags federate** (they are public on Mastodon; `Add`/`Remove{Hashtag}` target `featured`, as Mastodon + sends them, rather than `featuredTags`, which Mastodon ignores); endorsements are never federated and + `/accounts/:id/endorsements` answers only the persona's own, to itself; suggestions never name a sibling persona and + count `most_followed` only from three followers here; `familiar_followers` stays empty; the reading preferences are + set on `/clientapi/avatar/private/preferences` (Mastodon's API sets none); staff picks have their own routes, not the + settings form; a `Move` carries the persona's endorsement and note to the new account; featured tags, notes and + endorsements are left out of the persona archive (Mastodon's layout has none), its `following_accounts.csv` now + carries notify and languages. ## Intent @@ -601,7 +617,8 @@ The first refactor commit is a pure move with namespaces only. Logic changes fol `hashtag:local`, `list`; events `update`, `notification`, `status.update`, `delete`; `health`. Advertised as `urls.streaming`. - **Stubs:** `custom_emojis`, announcements, suggestions, preferences. *Announcements are real since R5, - `custom_emojis` since R7 (2026-10-17).* + `custom_emojis` since R7 (2026-10-17), suggestions, preferences, featured tags and endorsements since R8 + (2026-10-18).* - **Mapping:** - Avatar, ForeignAvatar and Group all map to Account (`group: true` for groups). `acct` uses a WebFinger-verified handle; `created_at` is truncated to the day; avatar and header always have a placeholder URL. diff --git a/tools/pasture/scenarios/featured.sh b/tools/pasture/scenarios/featured.sh new file mode 100644 index 0000000..2b7fb7e --- /dev/null +++ b/tools/pasture/scenarios/featured.sh @@ -0,0 +1,128 @@ +# The Mastodon API made real (completion program R8), with Mastodon. mastouser follows feat_alice, who features a hashtag: +# her /tattoos serves it and Mastodon shows it among her account's featured tags (her Add{Hashtag}), and unfeaturing it +# takes it away there (Remove{Hashtag}). feat_alice follows mastouser with notify=true, and a tag he features shows among +# his featured tags here; his next post gives her a `status` notification; his direct message to her is told on her +# `direct` stream, as server-sent events and on the WebSocket, as a `conversation` event. Her endorsement of him and her +# private note on him are hers alone: feat_bob, a sibling persona of the same login, sees neither, nobody else is told her +# endorsements, and feat_bob is never suggested to her, though she favourited his post. The stubs answer: translate 422, +# annual reports empty, the Mastodon admin API 403. Needs the mastodon peer (batch A); run after emoji. +M=https://mastodon.test:6443 +mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; } +. "$here/peers/mastodon.sh" +code() { curl -s -o /dev/null -w '%{http_code}' "$@"; } +# m_signed_body : one of our documents as Mastodon's instance actor reads it, signed (SecureMode answers unsigned 401) +m_signed_body() { podman exec pasture-mastodon bin/rails runner "Request.new(:get, '$1').on_behalf_of(Account.representative).add_headers('Accept' => 'application/activity+json').perform { |r| puts r.body_with_limit }" 2>/dev/null | tail -1; } +m_featured() { mcurl -H "$MH" "$M/api/v1/accounts/$1/featured_tags" | j "print(any((t.get('name') or '').lower() == '$2' for t in d or []))"; } +p_featured() { curl -s -H "$AH" "$P/api/v1/accounts/$1/featured_tags" | j "print(any((t.get('name') or '').lower() == '$2' for t in d or []))"; } + +echo "featured" +run=$(date +%s) +tag="pasture$run"; mtag="mastopasture$run" +AT=$(privapub_token feat_alice); AH="Authorization: Bearer $AT" +BT=$(privapub_token feat_bob); BH="Authorization: Bearer $BT" +MT=$(mastodon_token); MH="Authorization: Bearer $MT" +[ -n "$AT" ] && [ -n "$BT" ] && [ -n "$MT" ] && ok "tokens for feat_alice, feat_bob (her sibling, the same login) and mastouser" \ + || { ko "a token is missing"; return 1; } +alice_id=$(curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])") +bob_id=$(curl -s -H "$BH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])") + +echo " follows" +alice_on_m=$(mcurl -H "$MH" "$M/api/v2/search?q=@feat_alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$alice_on_m/follow" +until_true 30 '[ "$(curl -s "$P/api/v1/accounts/$alice_id" | j "print(d[\"followers_count\"])")" -ge 1 ]' \ + && ok "mastouser follows feat_alice" || ko "mastouser's follow never arrived" +m_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=mastouser@mastodon.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +rel=$(curl -s -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/follow" -d notify=true -d 'languages[]=en') +[ "$(echo "$rel" | j "print(d['notifying'] is True and d['languages'] == ['en'])")" = True ] \ + && ok "feat_alice follows mastouser with notify and languages (the relationship says so)" || ko "the follow's options: $(echo "$rel" | head -c 300)" +until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/relationships?id[]=$m_on_p" | j "print(d[0][\"following\"])")" = True ]' \ + && ok "mastouser accepts feat_alice" || ko "feat_alice's follow was not accepted" + +echo " her featured tag" +featured=$(curl -s -X POST -H "$AH" "$P/api/v1/featured_tags" -d "name=$tag") +featured_id=$(echo "$featured" | j "print(d['id'])") +[ "$(echo "$featured" | j "print(d['name'] == '$tag' and d['statuses_count'] == '0')")" = True ] && ok "feat_alice features #$tag" \ + || ko "featuring was refused: $(echo "$featured" | head -c 300)" +[ "$(curl -s -H "$AH" "$P/api/v1/featured_tags" | j "print(any(t['id'] == '$featured_id' for t in d))")" = True ] \ + && [ "$(curl -s "$P/api/v1/accounts/$alice_id/featured_tags" | j "print(any(t['name'] == '$tag' for t in d))")" = True ] \ + && ok "her featured tags list it, anonymously too" || ko "her featured tags miss it" +[ "$(curl -s "$P/api/v2/instance" | j "print(d['configuration']['accounts']['max_featured_tags'])")" = 10 ] \ + && ok "the instance API allows ten" || ko "max_featured_tags: $(curl -s "$P/api/v2/instance" | j "print(d['configuration']['accounts'])")" +[ "$(m_signed_body https://privapub.test/peasants/feat_alice/tattoos | j "print(any(i['type'] == 'Hashtag' and i['name'] == '#$tag' for i in d['orderedItems']))")" = True ] \ + && ok "/tattoos serves it as a Hashtag (signed)" || ko "/tattoos: $(m_signed_body https://privapub.test/peasants/feat_alice/tattoos | head -c 300)" +until_true 30 '[ "$(m_featured "$alice_on_m" "$tag")" = True ]' \ + && ok "Mastodon shows #$tag among feat_alice's featured tags" || ko "Mastodon's featured tags of feat_alice: $(mcurl -H "$MH" "$M/api/v1/accounts/$alice_on_m/featured_tags" | head -c 300)" +[ "$(code -X DELETE -H "$AH" "$P/api/v1/featured_tags/$featured_id")" = 200 ] && ok "feat_alice unfeatures it" || ko "unfeaturing was refused" +until_true 30 '[ "$(m_featured "$alice_on_m" "$tag")" = False ]' \ + && ok "Mastodon no longer shows it" || ko "Mastodon still shows #$tag" + +echo " his featured tag" +m_featured_id=$(mcurl -X POST -H "$MH" "$M/api/v1/featured_tags" -d "name=$mtag" | j "print(d['id'])") +[ -n "$m_featured_id" ] && ok "mastouser features #$mtag" || ko "Mastodon refused the featured tag" +until_true 30 '[ "$(p_featured "$m_on_p" "$mtag")" = True ]' \ + && ok "feat_alice reads #$mtag among mastouser's featured tags" || ko "his featured tags here: $(curl -s -H "$AH" "$P/api/v1/accounts/$m_on_p/featured_tags" | head -c 300)" +mcurl -o /dev/null -X DELETE -H "$MH" "$M/api/v1/featured_tags/$m_featured_id" +until_true 30 '[ "$(p_featured "$m_on_p" "$mtag")" = False ]' \ + && ok "his unfeaturing reaches PrivaPub" || ko "#$mtag is still featured here" + +echo " notify" +mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" --data-urlencode "status=notify me, PrivaPub $run" -d visibility=public -d language=en +until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/notifications?types[]=status" | j "print(any(\"notify me, PrivaPub $run\" in n[\"status\"][\"content\"] for n in d))")" = True ]' \ + && ok "his post gives feat_alice a status notification" || ko "no status notification: $(curl -s -H "$AH" "$P/api/v1/notifications?limit=5" | head -c 300)" + +echo " the direct stream" +curl -s -N -H "$AH" "$P/api/v1/streaming/direct" > "$work/sse.txt" & sse=$! +cat > "$work/ws.py" <<'PY' +import asyncio, json, sys, websockets +async def main(url, token, seconds): + async with websockets.connect(url, subprotocols=[token]) as ws: + await ws.send(json.dumps({"type": "subscribe", "stream": "direct"})) + try: + while True: + message = json.loads(await asyncio.wait_for(ws.recv(), timeout=seconds)) + print(json.dumps(message), flush=True) + except asyncio.TimeoutError: + pass +asyncio.run(main(sys.argv[1], sys.argv[2], float(sys.argv[3]))) +PY +python3 -I "$work/ws.py" "ws://127.0.0.1:${PASTURE_PORT:-6971}/api/v1/streaming" "$AT" 40 > "$work/ws.txt" 2>&1 & ws=$! +sleep 3 +mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" --data-urlencode "status=@feat_alice@privapub.test a secret, PrivaPub $run" -d visibility=direct +until_true 30 'grep -q "^event: conversation" "$work/sse.txt"' && ok "the SSE direct stream tells a conversation event" \ + || ko "the SSE direct stream: $(head -c 300 "$work/sse.txt")" +until_true 20 '[ "$(python3 -c "import json,sys +for l in open(sys.argv[1]): + m=json.loads(l) + if m.get(\"event\")==\"conversation\" and m.get(\"stream\")==[\"direct\"] and \"a secret, PrivaPub $run\" in json.loads(m[\"payload\"])[\"last_status\"][\"content\"]: print(True); break" "$work/ws.txt" 2>/dev/null)" = True ]' \ + && ok "the WebSocket's direct subscription tells it, with the message" || ko "the WebSocket: $(head -c 300 "$work/ws.txt")" +kill "$sse" "$ws" 2>/dev/null; wait "$sse" "$ws" 2>/dev/null + +echo " endorsements and notes stay hers" +[ "$(curl -s -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/endorse" | j "print(d['endorsed'])")" = True ] \ + && [ "$(curl -s -H "$AH" "$P/api/v1/endorsements" | j "print(any(a['id'] == '$m_on_p' for a in d))")" = True ] \ + && ok "feat_alice endorses mastouser" || ko "the endorsement did not hold" +[ "$(curl -s -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/note" --data-urlencode "comment=met in the pasture $run" | j "print(d['note'])")" = "met in the pasture $run" ] \ + && ok "her private note on him" || ko "the note was not kept" +[ "$(curl -s -H "$BH" "$P/api/v1/accounts/relationships?id[]=$m_on_p" | j "print(d[0]['note'] == '' and d[0]['endorsed'] is False)")" = True ] \ + && [ "$(curl -s -H "$BH" "$P/api/v1/endorsements" | j "print(d == [])")" = True ] \ + && ok "feat_bob, of the same login, sees neither her note nor her endorsement" || ko "a sibling sees her note or endorsement" +[ "$(curl -s -H "$BH" "$P/api/v1/accounts/$alice_id/endorsements" | j "print(d == [])")" = True ] \ + && [ "$(curl -s "$P/api/v1/accounts/$alice_id/endorsements" | j "print(d == [])")" = True ] \ + && [ "$(curl -s -H "$AH" "$P/api/v1/accounts/$alice_id/endorsements" | j "print(any(a['id'] == '$m_on_p' for a in d))")" = True ] \ + && ok "her endorsements are told to her alone" || ko "her endorsements are told to others" +bob_post=$(curl -s -X POST -H "$BH" "$P/api/v1/statuses" --data-urlencode "status=bob's own post $run" -d visibility=public | j "print(d['id'])") +curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses/$bob_post/favourite" +[ "$(curl -s -H "$AH" "$P/api/v2/suggestions?limit=80" | j "print(all(s['account']['id'] not in ('$bob_id', '$alice_id') for s in d))")" = True ] \ + && ok "feat_bob is never suggested to her, though she favourited his post" || ko "a sibling is suggested: $(curl -s -H "$AH" "$P/api/v2/suggestions" | head -c 300)" +[ "$(curl -s -H "$AH" "$P/api/v1/accounts/familiar_followers?id[]=$m_on_p" | j "print(d == [{'id': '$m_on_p', 'accounts': []}])")" = True ] \ + && ok "familiar followers stay empty" || ko "familiar followers: $(curl -s -H "$AH" "$P/api/v1/accounts/familiar_followers?id[]=$m_on_p")" + +echo " stubs" +[ "$(code -X POST -H "$AH" "$P/api/v1/statuses/$bob_post/translate")" = 422 ] && ok "translate answers 422" || ko "translate answers otherwise" +[ "$(curl -s -H "$AH" "$P/api/v1/annual_reports" | j "print(d == {'annual_reports': [], 'accounts': [], 'statuses': []})")" = True ] \ + && ok "annual reports answer none" || ko "annual reports: $(curl -s -H "$AH" "$P/api/v1/annual_reports")" +[ "$(code -H "$AH" "$P/api/v1/admin/accounts")" = 403 ] && [ "$(curl -s "$P/api/v1/admin/reports" | j "print(d['error'])")" = "This action is not allowed" ] \ + && ok "the Mastodon admin API answers 403" || ko "the admin API answers $(code -H "$AH" "$P/api/v1/admin/accounts")" + +curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/unendorse" +curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/note" -d comment= diff --git a/tools/pasture/sweep.sh b/tools/pasture/sweep.sh index bc7f74d..fb1dcb7 100755 --- a/tools/pasture/sweep.sh +++ b/tools/pasture/sweep.sh @@ -9,7 +9,8 @@ # A gts mastodon misskey sharkey akkoma, with relay, aoderelay and mailpit; scenarios: those five, threads followsync # pins moves relay (relay's covers both relays) accounts (mailpit's) instance (the server's own settings: it changes # them all and puts them back) moderation (gts and mastodon; after instance, whose log check expects only the -# administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation) +# administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation) featured (mastodon: featured +# tags, notify, the direct stream, endorsements and notes per persona; after emoji) # B lemmy lemmy19 piefed mbin nodebb # C peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale # D friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla @@ -29,7 +30,7 @@ batch_peers() { } batch_scenarios() { case "$1" in - A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji" ;; + A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji featured" ;; *) batch_peers "$1" ;; esac }