R7 custom emoji: the server's own, added by administrators, sent on posts, profiles and reactions

LocalEmoji, a new collection (unique shortcode whatever its case, never renamed: the shortcode is its id elsewhere),
managed on /clientapi/admin/emoji/list|insert|update|delete and kept in the staff's log (emoji.insert, .update,
.delete). Its picture goes through IMediaService.EmojiImage: PNG, GIF, WebP or JPEG up to 256 KB, a PNG within 256 px
with alpha kept (an animated GIF or WebP stays an animated GIF, with a still PNG), keep=none, shrunk until it fits
GoToSocial's 100 KiB, held by ServerUse "emoji:<shortcode>" so the janitor and the audit leave it. LocalEmojis keeps the
offered ones in memory, reloaded on every change: GET /api/v1/custom_emojis is real; local posts (words, title, warning,
poll options, each revision) and profiles keep the emoji they name; Notes, Questions, actors and EmojiReacts carry
Mastodon's Emoji tag with its id, served as an Emoji at /emojis/{shortcode} (410 once deleted; the path awaits the
owner's confirmation); a persona reacts with one (Pleroma's reactions, announcements). A restore keeps the live emoji
as it keeps the settings, and trashes server pictures nothing holds. tools/pasture/scenarios/emoji.sh joins batch A.

Verified: built, and run locally against a scratch database (uploads of every type, animation kept, metadata gone,
refusals, rendering, reactions, history, announcements, delete and revive, backup and restore, media audit). The
pasture scenario and the sweep were not run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WpVCoDeM1qUTRp4fXQdfof
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-17 05:57:25 +02:00
1 parent 9f68ca3ed0
commit d08fbabaa5
38 files changed
+1117 -54

No files matched your search

+81 -10
View File
@@ -63,8 +63,11 @@ actor-scoped route gets a name in the same spirit, agreed with the owner, and a
| A group's members and moderators | `/peasants/{name}/flock`, `/peasants/{name}/wardens` | | A group's members and moderators | `/peasants/{name}/flock`, `/peasants/{name}/wardens` |
| A post's replies and thread | `/peasants/{name}/scribbles/{id}/replies`, `/peasants/{name}/scribbles/{id}/context` | | A post's replies and thread | `/peasants/{name}/scribbles/{id}/replies`, `/peasants/{name}/scribbles/{id}/context` |
| Token refresh | `/clientapi/user/sniff/again` | | Token refresh | `/clientapi/user/sniff/again` |
| A custom emoji (`Emoji`, R7) | `/emojis/{shortcode}` (`LocalEmojis.Path`, the one place it is written) |
All of these have federated, so they are frozen. Agreed for later: `/drool` and `/echoes` (likes, shares). `/gossip` was All of these have federated, so they are frozen, but `/emojis/{shortcode}`: it is the plan's name for a custom emoji's
id (Mastodon's own path, with the shortcode for its number) and waits for the owner's confirmation before R7 deploys; it
freezes once it has federated, and every emoji id elsewhere names it. Agreed for later: `/drool` and `/echoes` (likes, shares). `/gossip` was
agreed for replies before they were served at `…/scribbles/{id}/replies` (2026-10-06); whether it is still wanted is agreed for replies before they were served at `…/scribbles/{id}/replies` (2026-10-06); whether it is still wanted is
asked in the completion program's R10. asked in the completion program's R10.
@@ -85,7 +88,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Group, Post (posts + DMs), Social, Timeline, Data, PersonaArchive, Moderator (reports, Group, Post (posts + DMs), Social, Timeline, Data, PersonaArchive, Moderator (reports,
accounts and posts moderated, the staff's log), Admin (logins: roles, bans, deletions, accounts and posts moderated, the staff's log), Admin (logins: roles, bans, deletions,
two-factor resets), DomainBlock, Statistics, Backup, InstanceAdmin (the server's two-factor resets), DomainBlock, Statistics, Backup, InstanceAdmin (the server's
settings, thumbnail, announcements, relays) settings, thumbnail, announcements, relays), EmojiAdmin (the server's custom emoji)
Infrastructure/ Infrastructure/
Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out
Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker
@@ -100,7 +103,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
RegistrationOptions.cs who may make a login (`Registrations:Mode`) RegistrationOptions.cs who may make a login (`Registrations:Mode`)
Federation/ Federation/
Controllers/ PeasantsController (actor, outbox, followers, following, posts, inboxes), Controllers/ PeasantsController (actor, outbox, followers, following, posts, inboxes),
WellKnownController (webfinger, nodeinfo), UsersController (redirect) WellKnownController (webfinger, nodeinfo), UsersController (redirect),
EmojisController (a custom emoji's `Emoji` at /emojis/{shortcode})
Actors/ LocalActorService (LocalActor, Keys, ReservedName), RemoteActorService Actors/ LocalActorService (LocalActor, Keys, ReservedName), RemoteActorService
(authoritative fetch, key verification, WebFinger), ActorDocument (parser) (authoritative fetch, key verification, WebFinger), ActorDocument (parser)
Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer
@@ -119,7 +123,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Relays/ Relays (the instance actor's relay subscriptions) Relays/ Relays (the instance actor's relay subscriptions)
Domain/ Domain/
Content/ ContentRenderer (Markdown or plain text → HTML with h-card mentions and hashtags; Content/ ContentRenderer (Markdown or plain text → HTML with h-card mentions and hashtags;
Document: the server's own texts) Document: the server's own texts), LocalEmojis (the server's custom emoji in memory)
Instance/ InstanceSettings (the server's own settings in memory), MediaLimitsFromSettings Instance/ InstanceSettings (the server's own settings in memory), MediaLimitsFromSettings
Moderation/ ModerationLog (the staff's log, AdminAction), AccountModeration (silence, suspend, posts Moderation/ ModerationLog (the staff's log, AdminAction), AccountModeration (silence, suspend, posts
removed), Restrictions (who is held back, in memory), ModerationFlags (posts' flags again) removed), Restrictions (who is held back, in memory), ModerationFlags (posts' flags again)
@@ -139,7 +143,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch
Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, lists, filters, Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, lists, filters,
media, polls, reactions, reports, scheduled statuses, streaming, tags, trends, cdns, media, polls, reactions, reports, scheduled statuses, streaming, tags, trends, cdns,
provenance; the stubs are `StubsController` in `SearchController.cs` provenance, custom emojis; the stubs are `StubsController` in `SearchController.cs`
Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex); Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex);
OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent) OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent)
Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, RootRemoval, Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, RootRemoval,
@@ -346,6 +350,59 @@ client's role route never does.
here and elsewhere, and the posts taken down, before it does; indexes on `SilencedAt` and `SuspendedAt` of both account here and elsewhere, and the posts taken down, before it does; indexes on `SilencedAt` and `SuspendedAt` of both account
collections and a partial one on `Post.ModeratedAt`. No new collection, so backups hold it all as they did. collections and a partial one on `Post.ModeratedAt`. No new collection, so backups hold it all as they did.
## Custom emoji (completion program R7)
- **The server's own emoji** (`Models/Post/LocalEmoji.cs`, a collection of its own): a shortcode (`[a-zA-Z0-9_]{2,64}`,
Mastodon's rule, `Constants.ShortcodeRegex`; unique whatever its case through `Key`, the lower-cased shortcode, while
texts match it exactly), a category, `VisibleInPicker`, `Disabled` (kept, but offered nowhere and taken into nothing
new), the picture (`MediaId`, `Url`, `StaticUrl`, `ContentType`), `CreatedAt`, `UpdatedAt` (any change),
`ImageUpdatedAt` (the picture's: the `updated` that federates) and `DeletedAt`. **It is never renamed**: its shortcode
is its id elsewhere, so an update names the emoji by it, and a new name is a new emoji. Deleted, the row stays, so its
id answers 410; adding the same shortcode again (written the same way) brings the row back with a new picture.
- **In memory** (`Domain/Content/LocalEmojis.cs`, static, as `Restrictions` is): the emoji offered now (neither disabled
nor deleted), read at start, after every change and when five minutes old. Whatever renders or matches emoji reads it,
never the database.
- **Where they are taken:** a local post keeps the emoji its words, title, warning and poll options name when it is
published or edited (`StatusService`, also archive imports; `Post.Emojis`, and each `PostRevision` its own), a persona
those its name, bio and fields name when its profile is saved (`update_credentials`, `/clientapi`'s avatar insert and
update, archive imports; `Avatar.Emojis`, `LocalActor.Emojis`). Only this server's, only those offered: `:name:` with no
letter or digit right before or after it, as Mastodon scans, at most 64 an object. Remote shortcodes are never ours.
- **What is rendered is what they are now** (`LocalEmojis.Of`): a Note, a Question and an actor carry an `Emoji` tag per
kept emoji still offered, with its current picture (an actor only when it has one); the Mastodon API's `emojis` of a
local status, poll, account and history entry come from the same list. An emoji disabled or deleted since is left out,
a new picture shows (its `updated` moves, so servers holding the old one fetch it again). No `Update` is sent when an
emoji changes (Mastodon sends none either).
- **The `Emoji` sent** (`LocalEmojis.Tag`, Mastodon's shape; GoToSocial takes none without its `id`):
`{"id": "{base}/emojis/{shortcode}", "type": "Emoji", "name": ":shortcode:", "updated": <ImageUpdatedAt, ms>,
"icon": {"type": "Image", "mediaType": "image/png" | "image/gif", "url": <the picture>}}`, and the same with the
`@context` at its id (`Federation/Controllers/EmojisController.cs`, `application/activity+json`, unsigned under
SecureMode too, since an emoji tells nothing about anyone; a disabled one 404, a deleted one 410 with a Tombstone; a
browser is sent to the picture). Statistics count its fetches as served `emoji`.
- **Reactions:** a persona reacts with one of ours as `:shortcode:`, `:shortcode@this-host:` or the bare shortcode
(`Reactions.React`, before the post's own emoji, so `:code:` is ours and `:code@host:` another server's); the
`EmojiReact` carries `content` `:shortcode:` and the full tag, which Misskey and Akkoma register before they count it.
Announcements take our emoji as reactions too (Mastodon's `name`, the shortcode, with `url` and `static_url`), and list
those their text uses in `emojis`. A new picture moves the `EmojiURL` of the reactions and notifications made with it.
- **The Mastodon API:** `GET /api/v1/custom_emojis` (anonymous) lists, as Mastodon does, those offered and in the picker,
by shortcode (`shortcode`, `url`, `static_url`, `visible_in_picker`, `category`), from memory.
- **Administrator routes** (`EmojiAdminController`, `[Authorize(Policy = Policies.IsAdmin)]` and checked again against
the database; every act in the staff's log as `emoji.insert`, `emoji.update` (its note names what changed) or
`emoji.delete`, target kind `emoji`, labelled `:shortcode:`):
- `GET /clientapi/admin/emoji/list`: `List<ViewLocalEmoji>`, disabled ones too, deleted ones never;
- `POST /clientapi/admin/emoji/insert`: multipart, `LocalEmojiForm` (`shortcode`, `category`, `visibleInPicker`,
`disabled`) and the picture as `image` (or `file`): 200 `ViewLocalEmoji`, 400 for the form, 409 for a shortcode taken
(whatever its case), 422 for the picture;
- `POST /clientapi/admin/emoji/update`: the same form, the emoji as it should be (its category none when empty), a new
picture optional: 200, 400, 404 for no such emoji, 422;
- `POST /clientapi/admin/emoji/delete?shortcode=`: 200, 404.
- **Its picture** (`IMediaService.EmojiImage`, see Media invariants): held by `ServerUse = "emoji:<shortcode>"` like the
thumbnail (the janitor and the quota leave it, `admin media audit` keeps the pictures of the emoji not deleted, disabled
ones too); a replaced picture and a deleted emoji's are trashed.
- **Backups and restores:** `LocalEmoji` is in every backup. A restore keeps the live emoji as it keeps the settings
(`ProtectiveMerge`: P's rows): one deleted since the backup stays deleted (its id answers 410, its picture stays in the
trash), one added or changed since stays as it is; then a picture of the server's that neither the settings nor an
emoji holds goes to the trash.
## Federation invariants ## Federation invariants
1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only 1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only
@@ -498,6 +555,13 @@ client's role route never does.
sRGB (`thumbnail`). sRGB (`thumbnail`).
- **GIFs:** an animated GIF becomes a looping silent H.264 mp4 typed `gifv`, as on Mastodon (ffmpeg with libx264; - **GIFs:** an animated GIF becomes a looping silent H.264 mp4 typed `gifv`, as on Mastodon (ffmpeg with libx264;
`PostMedia.Kind` says so), and a still GIF is an image. `PostMedia.Kind` says so), and a still GIF is an image.
- **A custom emoji's picture** (R7, `EmojiImage`): PNG, GIF, WebP or JPEG up to 256 KB, fitted within 256 px with its
transparency kept, written by libvips with `keep=none`: a still picture as a PNG, an animated GIF or WebP as an
animated GIF (libvips' `thumbnail` with every page, then `gifsave`: a PNG would lose the animation, and GIF is the
animated picture every peer takes as an emoji) with a still PNG of its first frame for `static_url`. It must end at
most 100 KiB, GoToSocial's default for another server's emoji (the strictest peer; Mastodon's is 256 KB): a still
picture is tried as it is, then with a palette, then at 192, 128, 96 and 64 px; an animation at those sides; one that
never fits is refused (422).
- **Types:** HEIC and HEIF are not accepted, since the bundled libvips has no HEVC decoder. - **Types:** HEIC and HEIF are not accepted, since the bundled libvips has no HEVC decoder.
- **Load:** processing runs `Media:Concurrency` at a time, and uploads have their own rate limit (`uploads`, per - **Load:** processing runs `Media:Concurrency` at a time, and uploads have their own rate limit (`uploads`, per
credential). credential).
@@ -526,8 +590,8 @@ client's role route never does.
What a login's media take (every persona's, trashed ones no longer) is `Counted.MediaOfRoot`, shown to the login at What a login's media take (every persona's, trashed ones no longer) is `Counted.MediaOfRoot`, shown to the login at
`/clientapi/user/storage`; `Media:QuotaBytesPerRoot` (0, none, by default, or the administrator's override in the `/clientapi/user/storage`; `Media:QuotaBytesPerRoot` (0, none, by default, or the administrator's override in the
server's settings) refuses an upload over it with 422; the administrator's statistics overview has the totals, the server's settings) refuses an upload over it with 422; the administrator's statistics overview has the totals, the
proxy cache and the trash. The server's own pictures (`ServerUse`, its thumbnail) belong to nobody and are held while proxy cache and the trash. The server's own pictures (`ServerUse`: its thumbnail, `emoji:<shortcode>` for a custom
the settings name them. emoji's) belong to nobody and are held while the settings name them, or while their emoji is not deleted.
Nothing is deleted because it looks unused. `PrivaPub admin media audit [--fix]` compares disk and database: with Nothing is deleted because it looks unused. `PrivaPub admin media audit [--fix]` compares disk and database: with
`--fix` (as www-data) it gives pictures shown from before their rows a row, and trashes media of deleted posts or `--fix` (as www-data) it gives pictures shown from before their rows a row, and trashes media of deleted posts or
@@ -742,9 +806,10 @@ client's role route never does.
the staff took down, and a root's password, e-mail, ban and policies stay, and every post's moderation flags are the staff took down, and a root's password, e-mail, ban and policies stay, and every post's moderation flags are
worked out again (`ModerationFlags`); worked out again (`ModerationFlags`);
roots, personas and groups made since become tombstones (deleted, names kept), local posts made since answer 410, roots, personas and groups made since become tombstones (deleted, names kept), local posts made since answer 410,
media made or trashed since go to the trash (but the server's own pictures, `ServerUse`); the staff's log media made or trashed since go to the trash (but the server's own pictures, `ServerUse`, though one of those that
(`AdminAction`) is the union, so a restore never loses an entry, its own request included, and the server's settings nothing holds any more goes there too); the staff's log (`AdminAction`) is the union, so a restore never loses an
are P's: the live settings outlive a restore; entry, its own request included, and the server's settings and custom emoji (`LocalEmoji`, R7) are P's: the live
settings and emoji outlive a restore, and an emoji deleted since stays deleted;
5. every session ends (a new `SessionStamp` for each root, persona tokens deleted), circuits close, and a 5. every session ends (a new `SessionStamp` for each root, persona tokens deleted), circuits close, and a
`RestoreRecord` (never in a backup) tells what happened (`admin restore --status`). `RestoreRecord` (never in a backup) tells what happened (`admin restore --status`).
@@ -990,6 +1055,12 @@ tools/pasture/run.sh down # removes e
directly and through a report gone on Mastodon, the author warned, a GoToSocial copy taken down here; the staff's log directly and through a report gone on Mastodon, the author warned, a GoToSocial copy taken down here; the staff's log
listing every act, the report's action in its note, no login entry naming a persona, none shown to the moderator, and listing every act, the report's action in its note, no login entry naming a persona, none shown to the moderator, and
no address. no address.
- **Custom emoji (`scenarios/emoji.sh`, needs gts, mastodon, misskey and akkoma; after moderation in batch A):** the
administrator adds `:pasture_<run>:` (409 for it again, a persona refused the list); `/api/v1/custom_emojis` lists it
and `/emojis/{shortcode}` serves the `Emoji` unsigned; the four peers follow emoji_alice, whose post using it shows it
in the status's `emojis` on Mastodon, GoToSocial (by URI), Misskey (the note's `emojis`) and Akkoma; her display name
shows it on Mastodon; her `EmojiReact` with it reaches Misskey and Akkoma as that emoji; deleted, it leaves the list,
its id answers 410 and posts here carry it no more; the staff's log keeps the insert and the delete.
- **Mailpit (1.31.4, `peers/mailpit.sh`, as `mailpit.test`):** the pasture's mail server. `appsettings.Pasture.json` - **Mailpit (1.31.4, `peers/mailpit.sh`, as `mailpit.test`):** the pasture's mail server. `appsettings.Pasture.json`
sends PrivaPub's mail to `pasture-mailpit:1025` (plain SMTP, any login accepted) and its recovery links to sends PrivaPub's mail to `pasture-mailpit:1025` (plain SMTP, any login accepted) and its recovery links to
`https://decepub.test`. `scenarios/accounts.sh` reads what arrived through Mailpit's API: an address set with the `https://decepub.test`. `scenarios/accounts.sh` reads what arrived through Mailpit's API: an address set with the
+23 -4
View File
@@ -108,9 +108,11 @@ read from documents in their published shape (`docs/INTEROP.md` records each one
instance actor follows the relays configured, and their `Announce`s are taken as their origin vouches (see "Relays") instance actor follows the relays configured, and their `Announce`s are taken as their origin vouches (see "Relays")
- [FEP-8967: Generic attachments, link previews](https://codeberg.org/fediverse/fep/src/branch/main/fep/8967/fep-8967.md): - [FEP-8967: Generic attachments, link previews](https://codeberg.org/fediverse/fep/src/branch/main/fep/8967/fep-8967.md):
received; a `preview` builds the card received; a `preview` builds the card
- [FEP-9098: Custom emojis](https://codeberg.org/fediverse/fep/src/branch/main/fep/9098/fep-9098.md): received, and this
server's own sent on posts, polls, actors and reactions, each served at its id (see "Custom emoji")
Planned (see `docs/ROADMAP.md`, the completion program, and the per-platform notes in `docs/INTEROP.md`): FEP-9098 Planned (see `docs/ROADMAP.md`, the completion program, and the per-platform notes in `docs/INTEROP.md`): FEP-8967 link
custom emoji offered (R7) and FEP-8967 link attachments sent (R10). attachments sent (R10).
## Actors ## Actors
@@ -130,6 +132,7 @@ actor document, a collection, NodeInfo or a delivery relates two avatars of the
| Direct-message context | `/peasants/{name}/whispers/{id}` | | Direct-message context | `/peasants/{name}/whispers/{id}` |
| A persona's wall (FEP-400e) | `/peasants/{name}/graffiti` | | A persona's wall (FEP-400e) | `/peasants/{name}/graffiti` |
| Followers on the asking server (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` | | Followers on the asking server (FEP-8fcf) | `/peasants/{name}/groupies/roll-call` |
| A custom emoji of this server (`Emoji`) | `/emojis/{shortcode}` |
| Profile and post pages | `/@{name}`, `/@{name}/{id}` | | Profile and post pages | `/@{name}`, `/@{name}/{id}` |
The names are the project's own and are stable; resolve actors through WebFinger, not by guessing a path. The names are the project's own and are stable; resolve actors through WebFinger, not by guessing a path.
@@ -344,6 +347,23 @@ and naming the post answered, lets it out to its audience with `replyApproval`;
(`rejected`). An empty `canReply` refuses every reply (422). (`rejected`). An empty `canReply` refuses every reply (422).
**Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object. **Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object.
This server's own (its administrator adds them; shortcodes of letters, digits and underscores) are sent the same way, as
Mastodon sends them, on a post or poll whose words, title, warning or options name them as `:shortcode:`, on an actor
whose name, bio or fields do, and on an `EmojiReact`:
```json
{"id": "https://privapub.thepra.dev/emojis/blobcat", "type": "Emoji", "name": ":blobcat:",
"updated": "2026-10-17T12:00:00.000Z",
"icon": {"type": "Image", "mediaType": "image/png", "url": "https://privapub.thepra.dev/media/files/2026/10/….png"}}
```
The picture is a PNG of at most 256 px a side, or an animated GIF, and at most 100 KiB (GoToSocial's limit for another
server's emoji). Its `id` serves the same object with its `@context` (`application/activity+json`, signed or not, since
an emoji tells nothing about anyone); a deleted emoji answers 410 with a `Tombstone`, and one added again under the same
shortcode comes back at the same id with a newer `updated`. A post or an actor rendered again carries the emoji as they
are then: one deleted or disabled since is left out, and a new picture has a newer `updated`. No activity is sent when
an emoji changes. A reaction with one of them has `content` `:shortcode:` and that tag.
**Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`, **Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`,
`memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show `memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show
`name`. A content warning without its own text uses the title, or "Content warning". `name`. A content warning without its own text uses the title, or "Content warning".
@@ -457,7 +477,7 @@ Posts with a location (shown to nearby users of this server) never leave the ser
- **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like - **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like
Mastodon's authorized fetch; the server's own actors (the instance actor `/peasants/privapub`, the reporter Mastodon's authorized fetch; the server's own actors (the instance actor `/peasants/privapub`, the reporter
`/peasants/privapub_reports` and the reader of feeds `/peasants/privapub_feeds`) are the exception, since their keys `/peasants/privapub_reports` and the reader of feeds `/peasants/privapub_feeds`) are the exception, since their keys
are needed first. are needed first, and so are its custom emoji at `/emojis/{shortcode}`.
A browser asking for HTML is redirected to the public page instead. A browser asking for HTML is redirected to the public page instead.
- **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were - **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were
for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted
@@ -483,7 +503,6 @@ Posts with a location (shown to nearby users of this server) never leave the ser
## Known limitations ## Known limitations
- Our own posts carry no custom emoji: emoji are received and shown, not offered.
- Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed - Remote media is fetched through this server's proxy when a local client displays it. Video and audio are streamed
through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer. through it with byte ranges passed on, so a PeerTube file is never fetched whole for one viewer.
- Collections expose counts, not members: `/groupies` and `/stalking` give the same totals as the Mastodon API's - Collections expose counts, not members: `/groupies` and `/stalking` give the same totals as the Mastodon API's
@@ -0,0 +1,45 @@
using PrivaPub.ClientModels.Resources;
using System.ComponentModel.DataAnnotations;
namespace PrivaPub.ClientModels.Admin
{
// one of the server's custom emoji as the administrator's page shows it (/clientapi/admin/emoji/list, completion program
// R7): disabled ones too, deleted ones never
public class ViewLocalEmoji
{
public string Id { get; set; }
public string Shortcode { get; set; }
public string Category { get; set; }
public bool VisibleInPicker { get; set; } = true;
public bool Disabled { get; set; } = false;
public string Url { get; set; }//its picture: a PNG, or an animated GIF
public string StaticUrl { get; set; }//a still PNG of it
public string ContentType { get; set; }
public int? Width { get; set; }
public int? Height { get; set; }
public long Size { get; set; } = 0;
public string Uri { get; set; }//its id on other servers (/emojis/{shortcode}): why a shortcode never changes
public DateTime CreatedAt { get; set; }
public DateTime UpdatedAt { get; set; }
public DateTime ImageUpdatedAt { get; set; }
}
// POST /clientapi/admin/emoji/insert and /emoji/update, as multipart/form-data with the picture as `image` (PNG, GIF,
// WebP or JPEG, 256 KB at most; required to insert, optional to update). The shortcode names the emoji: an update
// changes the emoji with that shortcode, and an emoji is never renamed (its shortcode is its id elsewhere); an update
// sends the emoji as it should be, its category (none when empty), whether it is in the picker and whether it is
// disabled.
public class LocalEmojiForm
{
[Required(ErrorMessageResourceName = "Required", ErrorMessageResourceType = typeof(ErrorsResource)),
RegularExpression(Constants.ShortcodeRegex, ErrorMessageResourceName = "InvalidShortcode", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Shortcode { get; set; }
[StringLength(64, ErrorMessageResourceName = "MaxLengthString", ErrorMessageResourceType = typeof(ErrorsResource))]
public string Category { get; set; }
public bool VisibleInPicker { get; set; } = true;
public bool Disabled { get; set; } = false;//kept, but offered nowhere and taken into no new post
}
}
+2
View File
@@ -10,5 +10,7 @@
public const string PasswordRegex = @"^(?=.*)(?=.*[a-z])(?=.*[A-Z])(?!.*\s).*$"; public const string PasswordRegex = @"^(?=.*)(?=.*[a-z])(?=.*[A-Z])(?!.*\s).*$";
public const string UserNameRegex = "^[a-z0-9_]+$"; public const string UserNameRegex = "^[a-z0-9_]+$";
// a custom emoji's shortcode: Mastodon's rule (letters, digits and underscores, two at least), 64 at most
public const string ShortcodeRegex = "^[a-zA-Z0-9_]{2,64}$";
} }
} }
@@ -114,6 +114,15 @@ namespace PrivaPub.ClientModels.Resources {
} }
} }
/// <summary>
/// Looks up a localized string similar to Use only letters, digits and underscores, from 2 to 64 of them..
/// </summary>
public static string InvalidShortcode {
get {
return ResourceManager.GetString("InvalidShortcode", resourceCulture);
}
}
/// <summary> /// <summary>
/// Looks up a localized string similar to Invalid password, upper-case, lower-case and number characters required.. /// Looks up a localized string similar to Invalid password, upper-case, lower-case and number characters required..
/// </summary> /// </summary>
@@ -135,6 +135,9 @@
<data name="InvalidUserName" xml:space="preserve"> <data name="InvalidUserName" xml:space="preserve">
<value>Usa solo lettere minuscole, cifre e trattini bassi.</value> <value>Usa solo lettere minuscole, cifre e trattini bassi.</value>
</data> </data>
<data name="InvalidShortcode" xml:space="preserve">
<value>Usa solo lettere, cifre e trattini bassi, da 2 a 64.</value>
</data>
<data name="InvalidPassword" xml:space="preserve"> <data name="InvalidPassword" xml:space="preserve">
<value>Password non valida, deve avere almeno un carattere grande, un carattere piccolo ed un numero.</value> <value>Password non valida, deve avere almeno un carattere grande, un carattere piccolo ed un numero.</value>
</data> </data>
@@ -135,6 +135,9 @@
<data name="InvalidUserName" xml:space="preserve"> <data name="InvalidUserName" xml:space="preserve">
<value>Use only lower-case letters, digits and underscores.</value> <value>Use only lower-case letters, digits and underscores.</value>
</data> </data>
<data name="InvalidShortcode" xml:space="preserve">
<value>Use only letters, digits and underscores, from 2 to 64 of them.</value>
</data>
<data name="InvalidPassword" xml:space="preserve"> <data name="InvalidPassword" xml:space="preserve">
<value>Invalid password, upper-case, lower-case and number characters required.</value> <value>Invalid password, upper-case, lower-case and number characters required.</value>
</data> </data>
@@ -115,6 +115,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
} }
if (fields.Count > 0 || Params.Has("fields_attributes[0][name]")) if (fields.Count > 0 || Params.Has("fields_attributes[0][name]"))
avatar.Fields = fields; avatar.Fields = fields;
avatar.Emojis = PrivaPub.Domain.Content.LocalEmojis.OfProfile(avatar);//the server's custom emoji its name, note and fields use (R7)
avatar.UpdatedAt = DateTime.UtcNow; avatar.UpdatedAt = DateTime.UtcNow;
await DB.Default.SaveAsync(avatar, token); await DB.Default.SaveAsync(avatar, token);
foreach (var made in replaced) foreach (var made in replaced)
@@ -4,6 +4,7 @@ using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities; using MongoDB.Entities;
using PrivaPub.Api.Mastodon.Infrastructure; using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Social; using PrivaPub.Domain.Social;
using PrivaPub.Models.Social; using PrivaPub.Models.Social;
@@ -65,7 +66,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
["mentions"] = Array.Empty<object>(), ["mentions"] = Array.Empty<object>(),
["statuses"] = Array.Empty<object>(), ["statuses"] = Array.Empty<object>(),
["tags"] = Array.Empty<object>(), ["tags"] = Array.Empty<object>(),
["emojis"] = Array.Empty<object>(), ["emojis"] = Mappers.MastodonMapper.LocalEmojiEntities(LocalEmojis.In(a.Text)),//the server's custom emoji its text uses (R7)
["reactions"] = reactions.Where(r => r.AnnouncementId == a.ID) ["reactions"] = reactions.Where(r => r.AnnouncementId == a.ID)
.GroupBy(r => r.Name) .GroupBy(r => r.Name)
.OrderBy(g => g.Min(r => r.CreatedAt)) .OrderBy(g => g.Min(r => r.CreatedAt))
@@ -78,12 +79,18 @@ namespace PrivaPub.Api.Mastodon.Controllers
}).ToList(); }).ToList();
} }
// a unicode emoji's reaction: no url (custom emoji come later); `me` only for a persona // a reaction: a unicode emoji's has no url, one of the server's custom emoji's has its pictures (R7); `me` only for a
// persona
static Dictionary<string, object> Reaction(string name, int count, bool? me) static Dictionary<string, object> Reaction(string name, int count, bool? me)
{ {
var reaction = new Dictionary<string, object> { ["name"] = name, ["count"] = count }; var reaction = new Dictionary<string, object> { ["name"] = name, ["count"] = count };
if (me is { } mine) if (me is { } mine)
reaction["me"] = mine; reaction["me"] = mine;
if (LocalEmojis.Find(name) is { } custom)
{
reaction["url"] = custom.Url;
reaction["static_url"] = custom.StaticUrl;
}
return reaction; return reaction;
} }
@@ -0,0 +1,22 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Content;
namespace PrivaPub.Api.Mastodon.Controllers
{
// Mastodon's list of the server's custom emoji (completion program R7), for clients' pickers: as Mastodon lists them,
// those offered (not disabled) and shown in the picker, by shortcode. Read from LocalEmojis, the copy in memory that every
// change reloads at once (and that is read again when five minutes old), so no request reads the database.
public class CustomEmojisController : MastodonController
{
[HttpGet("/api/v1/custom_emojis"), AllowAnonymous]
public IActionResult List() => Json(LocalEmojis.Current.Values
.Where(e => e.VisibleInPicker)
.OrderBy(e => e.Shortcode, StringComparer.Ordinal)
.Select(e => new CustomEmojiEntity { Shortcode = e.Shortcode, Url = e.Url, StaticUrl = e.StaticUrl, VisibleInPicker = true, Category = e.Category })
.ToList());
}
}
@@ -140,9 +140,6 @@ namespace PrivaPub.Api.Mastodon.Controllers
public class StubsController : MastodonController public class StubsController : MastodonController
{ {
[HttpGet("/api/v1/custom_emojis"), Microsoft.AspNetCore.Authorization.AllowAnonymous]
public IActionResult CustomEmojis() => Json(Array.Empty<object>());
[HttpGet("/api/v1/suggestions"), Scope("read")] [HttpGet("/api/v1/suggestions"), Scope("read")]
public IActionResult SuggestionsV1() => Json(Array.Empty<object>()); public IActionResult SuggestionsV1() => Json(Array.Empty<object>());
@@ -291,7 +291,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
SpoilerText = r.SpoilerText ?? string.Empty, SpoilerText = r.SpoilerText ?? string.Empty,
Sensitive = r.HasContentWarning, Sensitive = r.HasContentWarning,
CreatedAt = MastodonJson.Time(r.EditedAt), CreatedAt = MastodonJson.Time(r.EditedAt),
Account = account Account = account,
Emojis = _mapper.EmojisOf(post, r.Emojis, r.ContentHtml, r.SpoilerText, r.Title)
}).ToList(); }).ToList();
edits.Add(new StatusEdit edits.Add(new StatusEdit
{ {
@@ -299,7 +300,8 @@ namespace PrivaPub.Api.Mastodon.Controllers
SpoilerText = post.SpoilerText ?? string.Empty, SpoilerText = post.SpoilerText ?? string.Empty,
Sensitive = post.HasContentWarning, Sensitive = post.HasContentWarning,
CreatedAt = MastodonJson.Time(post.EditedAt ?? post.CreationDate), CreatedAt = MastodonJson.Time(post.EditedAt ?? post.CreationDate),
Account = account Account = account,
Emojis = _mapper.EmojisOf(post, post.Emojis, post.ContentHtml, post.Text, post.SpoilerText, post.Title)
}); });
return Json(edits); return Json(edits);
} }
@@ -2,6 +2,7 @@
using PrivaPub.Api.Mastodon.Entities; using PrivaPub.Api.Mastodon.Entities;
using PrivaPub.Api.Mastodon.Infrastructure; using PrivaPub.Api.Mastodon.Infrastructure;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Media; using PrivaPub.Domain.Media;
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering; using PrivaPub.Federation.Rendering;
@@ -123,6 +124,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
FollowingCount = (int)following, FollowingCount = (int)following,
StatusesCount = (int)statuses, StatusesCount = (int)statuses,
LastStatusAt = latest == default ? default : latest.CreationDate.ToString("yyyy-MM-dd"), LastStatusAt = latest == default ? default : latest.CreationDate.ToString("yyyy-MM-dd"),
Emojis = LocalEmojiEntities(actor.Emojis),
Fields = actor.Fields.Select(f => new Field { Name = f.Key, Value = ActivityPubRenderer.FieldValue(f.Value) }).ToList() Fields = actor.Fields.Select(f => new Field { Name = f.Key, Value = ActivityPubRenderer.FieldValue(f.Value) }).ToList()
}; };
if (withSource) if (withSource)
@@ -303,7 +305,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
MediaAttachments = post.Media.Count > 0 ? post.Media.Select(Media).ToList() : Playable(post), MediaAttachments = post.Media.Count > 0 ? post.Media.Select(Media).ToList() : Playable(post),
Card = Card(post), Card = Card(post),
Poll = Poll(post, viewerId, ownVotes.GetValueOrDefault(post.ID)), Poll = Poll(post, viewerId, ownVotes.GetValueOrDefault(post.ID)),
Emojis = Emojis(post.Emojis), Emojis = post.IsFederatedCopy ? Emojis(post.Emojis) : LocalEmojiEntities(post.Emojis),
Privapub = Extension(post, accounts, viewerId == default ? default(bool?) : downvoted.Contains(post.ID)), Privapub = Extension(post, accounts, viewerId == default ? default(bool?) : downvoted.Contains(post.ID)),
EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new(), EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new(),
Pleroma = new PleromaStatus { EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new() }, Pleroma = new PleromaStatus { EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new() },
@@ -500,13 +502,31 @@ namespace PrivaPub.Api.Mastodon.Mappers
Voted = viewerId == default ? default(bool?) : ownVotes?.Count > 0 || post.GroupUserId == viewerId, Voted = viewerId == default ? default(bool?) : ownVotes?.Count > 0 || post.GroupUserId == viewerId,
OwnVotes = viewerId == default ? default : ownVotes?.ToList() ?? new List<int>(), OwnVotes = viewerId == default ? default : ownVotes?.ToList() ?? new List<int>(),
Options = poll.Options.Select(o => new PollOptionEntity { Title = o.Title, VotesCount = hidden ? default(int?) : o.Votes }).ToList(), Options = poll.Options.Select(o => new PollOptionEntity { Title = o.Title, VotesCount = hidden ? default(int?) : o.Votes }).ToList(),
Emojis = Emojis(post.Emojis.Where(e => poll.Options.Any(o => o.Title.Contains($":{e.Shortcode}:")))) Emojis = post.IsFederatedCopy
? Emojis(post.Emojis.Where(e => poll.Options.Any(o => o.Title.Contains($":{e.Shortcode}:"))))
: LocalEmojiEntities(post.Emojis.Where(e => poll.Options.Any(o => o.Title.Contains($":{e.Shortcode}:"))))
}; };
} }
List<CustomEmojiEntity> Emojis(IEnumerable<CustomEmoji> emojis) => List<CustomEmojiEntity> Emojis(IEnumerable<CustomEmoji> emojis) =>
emojis?.Select(e => new CustomEmojiEntity { Shortcode = e.Shortcode, Url = Proxied(e.URL), StaticUrl = Proxied(e.URL) }).ToList() ?? new(); emojis?.Select(e => new CustomEmojiEntity { Shortcode = e.Shortcode, Url = Proxied(e.URL), StaticUrl = Proxied(e.URL) }).ToList() ?? new();
/// <summary>The custom emoji one version of a post names (its history's `emojis`): those the version kept, else the
/// post's that its texts name.</summary>
public List<CustomEmojiEntity> EmojisOf(PostEntity post, IReadOnlyCollection<CustomEmoji> kept, params string[] texts)
{
var named = (kept is { Count: > 0 } ? kept : post.Emojis ?? new List<CustomEmoji>()).Where(e => texts.Any(t => t?.Contains($":{e.Shortcode}:") == true));
return post.IsFederatedCopy ? Emojis(named) : LocalEmojiEntities(named);
}
// a local post's or persona's emoji are this server's, as they are now: one disabled or deleted since is left out, and
// a new picture shows (R7)
public static List<CustomEmojiEntity> LocalEmojiEntities(IEnumerable<CustomEmoji> kept) =>
LocalEmojis.Of(kept).Select(e => new CustomEmojiEntity
{
Shortcode = e.Shortcode, Url = e.Url, StaticUrl = e.StaticUrl, VisibleInPicker = e.VisibleInPicker, Category = e.Category
}).ToList();
PrivaPubStatus Extension(PostEntity post, Dictionary<string, Account> accounts, bool? downvoted) => new() PrivaPubStatus Extension(PostEntity post, Dictionary<string, Account> accounts, bool? downvoted) => new()
{ {
Wall = post.WallOwnerAccountId == default || post.WallOwnerAccountId == AuthorOf(post) ? default : accounts.GetValueOrDefault(post.WallOwnerAccountId), Wall = post.WallOwnerAccountId == default || post.WallOwnerAccountId == AuthorOf(post) ? default : accounts.GetValueOrDefault(post.WallOwnerAccountId),
@@ -0,0 +1,246 @@
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Localization;
using MongoDB.Driver;
using MongoDB.Entities;
using PrivaPub.ClientModels;
using PrivaPub.ClientModels.Admin;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Media;
using PrivaPub.Domain.Moderation;
using PrivaPub.Extensions;
using PrivaPub.Federation.Actors;
using PrivaPub.Models;
using PrivaPub.Models.Post;
using PrivaPub.Models.Social;
using PrivaPub.Models.User;
using PrivaPub.Resources;
namespace PrivaPub.Controllers.ClientToServer
{
// The server's custom emoji from the administrator's page (completion program R7): listed, added, changed and deleted
// by an administrator, checked again against the database. Every act is kept in the staff's log (emoji.insert,
// emoji.update, emoji.delete). An emoji is never renamed: its shortcode is its id on other servers (/emojis/{shortcode}).
[ApiController,
Route("clientapi/admin/emoji"),
Authorize(Policy = Policies.IsAdmin)]
public class EmojiAdminController : ControllerBase
{
// a picture up to a megabyte reaches EmojiImage, which refuses one over 256 KB with its reason
const long MaxRequestBytes = 1024 * 1024;
readonly IModerationLog _log;
readonly IStringLocalizer _localizer;
readonly IMediaService _media;
readonly ILocalActorService _localActors;
public EmojiAdminController(IModerationLog log, IStringLocalizer<GenericRes> localizer, IMediaService media, ILocalActorService localActors)
{
_log = log;
_localizer = localizer;
_media = media;
_localActors = localActors;
}
// the token says administrator; the database has the last word
async Task<RootUser> Administrator(CancellationToken token)
{
var root = await DB.Default.Find<RootUser>().MatchID(User.GetUserId()).ExecuteFirstAsync(token);
return root is { IsBanned: false, DeletedAt: null } && root.Policies.Contains(Policies.IsAdmin) ? root : default;
}
IActionResult Refused(int status, string message) => StatusCode(status, new WebResult().Invalidate(_localizer[message], status));
[HttpGet, Route("list")]
public async Task<IActionResult> List(CancellationToken token)
{
if (await Administrator(token) == default)
return Forbid();
var emojis = await DB.Default.Find<LocalEmoji>().Match(e => e.DeletedAt == null).Sort(e => e.Key, Order.Ascending).ExecuteAsync(token);
return Ok(emojis.Select(View).ToList());
}
// a new emoji; a shortcode deleted before comes back as it was written, with its new picture (the same id elsewhere)
[HttpPost, Route("insert"), RequestSizeLimit(MaxRequestBytes), RequestFormLimits(MultipartBodyLengthLimit = MaxRequestBytes)]
public async Task<IActionResult> Insert([FromForm] LocalEmojiForm form, CancellationToken token)
{
var admin = await Administrator(token);
if (admin == default)
return Forbid();
var key = form.Shortcode.ToLowerInvariant();
var existing = await DB.Default.Find<LocalEmoji>().Match(e => e.Key == key).ExecuteFirstAsync(token);
if (existing != default && (existing.DeletedAt == default || existing.Shortcode != form.Shortcode))
return Refused(StatusCodes.Status409Conflict, "This shortcode is taken.");
var outcome = await _media.EmojiImage(form.Shortcode, await Picture(token), token);
if (!outcome.Ok)
return StatusCode(outcome.Status, new WebResult().Invalidate(outcome.Error, outcome.Status));
var now = DateTime.UtcNow;
var emoji = existing ?? new LocalEmoji { Shortcode = form.Shortcode, Key = key, CreatedAt = now };
emoji.Category = Clean(form.Category);
emoji.VisibleInPicker = form.VisibleInPicker;
emoji.Disabled = form.Disabled;
emoji.DeletedAt = default;
emoji.UpdatedAt = now;
Picture(emoji, outcome.Attachment, now);
var saved = true;
if (existing == default)
try
{
await DB.Default.SaveAsync(emoji, token);
}
catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey)
{
saved = false;//the same shortcode added at the same moment
}
else
saved = (await DB.Default.Update<LocalEmoji>().Match(e => e.ID == existing.ID && e.DeletedAt != null).ModifyWith(emoji).ExecuteAsync(token))
.ModifiedCount > 0;
if (!saved)
{
await _media.Trash(m => m.ID == outcome.Attachment.ID, "emoji not saved", token);
return Refused(StatusCodes.Status409Conflict, "This shortcode is taken.");
}
await LocalEmojis.Reload(CancellationToken.None);
var notes = new[] { existing != default ? "added again" : default, emoji.Disabled ? "disabled" : default, Animated(emoji) }.Where(n => n != default).ToList();
await _log.Write(admin.ID, AdminActions.EmojiInsert, AdminTargets.Emoji, emoji.ID, $":{emoji.Shortcode}:",
note: notes.Count > 0 ? string.Join(", ", notes) : default, token: token);
return Ok(View(emoji));
}
// the emoji with that shortcode as it should be now: its category, the picker, disabled or not, and a new picture if
// one is sent
[HttpPost, Route("update"), RequestSizeLimit(MaxRequestBytes), RequestFormLimits(MultipartBodyLengthLimit = MaxRequestBytes)]
public async Task<IActionResult> Update([FromForm] LocalEmojiForm form, CancellationToken token)
{
var admin = await Administrator(token);
if (admin == default)
return Forbid();
var emoji = await Live(form.Shortcode, token);
if (emoji == default)
return NotFound();
var changed = new List<string>();
var category = Clean(form.Category);
if (emoji.Category != category)
changed.Add("category");
if (emoji.VisibleInPicker != form.VisibleInPicker)
changed.Add(form.VisibleInPicker ? "in the picker" : "out of the picker");
if (emoji.Disabled != form.Disabled)
changed.Add(form.Disabled ? "disabled" : "enabled");
emoji.Category = category;
emoji.VisibleInPicker = form.VisibleInPicker;
emoji.Disabled = form.Disabled;
var image = await Picture(token);
var previous = (MediaId: emoji.MediaId, Url: emoji.Url);
var now = DateTime.UtcNow;
MediaOutcome outcome = default;
if (image != default)
{
outcome = await _media.EmojiImage(emoji.Shortcode, image, token);
if (!outcome.Ok)
return StatusCode(outcome.Status, new WebResult().Invalidate(outcome.Error, outcome.Status));
Picture(emoji, outcome.Attachment, now);
changed.Add("image");
}
if (changed.Count == 0)
return Ok(View(emoji));
emoji.UpdatedAt = now;
if ((await DB.Default.Update<LocalEmoji>().Match(e => e.ID == emoji.ID && e.DeletedAt == null).ModifyWith(emoji).ExecuteAsync(token)).ModifiedCount == 0)
{
if (outcome != default)
await _media.Trash(m => m.ID == outcome.Attachment.ID, "emoji not saved", token);
return NotFound();//deleted meanwhile
}
if (outcome != default)
{
var use = LocalEmojis.ServerUse(emoji.Shortcode);
if (previous.MediaId != default)
await _media.Trash(m => m.ID == previous.MediaId && m.ServerUse == use, "emoji image replaced", token);
// the reactions made with it show the picture it has now
if (previous.Url != default)
{
await DB.Default.Update<Reaction>().Match(r => r.Emoji == emoji.Shortcode && r.EmojiURL == previous.Url)
.Modify(r => r.EmojiURL, emoji.Url).ExecuteAsync(token);
await DB.Default.Update<Notification>().Match(n => n.Emoji == emoji.Shortcode && n.EmojiURL == previous.Url)
.Modify(n => n.EmojiURL, emoji.Url).ExecuteAsync(token);
}
}
await LocalEmojis.Reload(CancellationToken.None);
await _log.Write(admin.ID, AdminActions.EmojiUpdate, AdminTargets.Emoji, emoji.ID, $":{emoji.Shortcode}:", note: string.Join(", ", changed), token: token);
return Ok(View(emoji));
}
// deleted: offered nowhere, left out of what posts and profiles render, its picture trashed, and its id answers 410.
// No other server is told (Mastodon tells none either); they keep their copy until they read a post again.
[HttpPost, Route("delete")]
public async Task<IActionResult> Delete([FromQuery] string shortcode, CancellationToken token)
{
var admin = await Administrator(token);
if (admin == default)
return Forbid();
var emoji = await Live(shortcode, token);
if (emoji == default)
return NotFound();
var now = DateTime.UtcNow;
if ((await DB.Default.Update<LocalEmoji>().Match(e => e.ID == emoji.ID && e.DeletedAt == null)
.Modify(e => e.DeletedAt, now).Modify(e => e.UpdatedAt, now).ExecuteAsync(token)).ModifiedCount == 0)
return NotFound();
var use = LocalEmojis.ServerUse(emoji.Shortcode);
await _media.Trash(m => m.ID == emoji.MediaId && m.ServerUse == use, "emoji deleted", token);
await LocalEmojis.Reload(CancellationToken.None);
await _log.Write(admin.ID, AdminActions.EmojiDelete, AdminTargets.Emoji, emoji.ID, $":{emoji.Shortcode}:", token: token);
return Ok();
}
static async Task<LocalEmoji> Live(string shortcode, CancellationToken token)
{
if (!LocalEmojis.IsShortcode(shortcode))
return default;
var key = shortcode.ToLowerInvariant();
var emoji = await DB.Default.Find<LocalEmoji>().Match(e => e.Key == key && e.DeletedAt == null).ExecuteFirstAsync(token);
return emoji?.Shortcode == shortcode ? emoji : default;
}
// the picture sent as `image` (or `file`, as the thumbnail's route names it)
async Task<IFormFile> Picture(CancellationToken token) =>
Request.HasFormContentType && await Request.ReadFormAsync(token) is { } form ? form.Files["image"] ?? form.Files["file"] : default;
void Picture(LocalEmoji emoji, Models.Media.MediaAttachment attachment, DateTime now)
{
emoji.MediaId = attachment.ID;
emoji.Url = _media.Url(attachment.FilePath);
emoji.StaticUrl = attachment.PreviewPath == default ? emoji.Url : _media.Url(attachment.PreviewPath);
emoji.ContentType = attachment.ContentType;
emoji.Width = attachment.Width;
emoji.Height = attachment.Height;
emoji.Size = attachment.Size;
emoji.ImageUpdatedAt = now;
}
static string Animated(LocalEmoji emoji) => emoji.ContentType == "image/gif" ? "animated" : default;
static string Clean(string text) => string.IsNullOrWhiteSpace(text) ? default : text.Trim();
ViewLocalEmoji View(LocalEmoji emoji) => new()
{
Id = emoji.ID,
Shortcode = emoji.Shortcode,
Category = emoji.Category,
VisibleInPicker = emoji.VisibleInPicker,
Disabled = emoji.Disabled,
Url = emoji.Url,
StaticUrl = emoji.StaticUrl ?? emoji.Url,
ContentType = emoji.ContentType,
Width = emoji.Width,
Height = emoji.Height,
Size = emoji.Size,
Uri = LocalEmojis.Uri(_localActors.BaseAddress, emoji.Shortcode),
CreatedAt = emoji.CreatedAt,
UpdatedAt = emoji.UpdatedAt,
ImageUpdatedAt = emoji.ImageUpdatedAt
};
}
}
+164
View File
@@ -0,0 +1,164 @@
using MongoDB.Entities;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
using System.Text.RegularExpressions;
namespace PrivaPub.Domain.Content
{
// This server's own custom emoji (completion program R7), in memory: the ones offered now (neither disabled nor deleted),
// for the places that read them one text at a time (a post published or edited, a profile saved, a Note or an actor
// rendered, a reaction, the Mastodon API's lists). Read at start, again after every change an administrator makes
// (EmojiAdminController) and when five minutes old; a process that never loaded it starts with none and loads it on
// first use.
//
// A local post or profile keeps the emoji its text named when it was written (Post.Emojis, Avatar.Emojis); what is
// rendered of them is what they are now: one disabled or deleted since is left out, and one whose picture changed is
// shown with the new one, its `updated` telling the servers that hold the old one to fetch it again.
public static class LocalEmojis
{
/// <summary>Where each emoji is served as an ActivityPub `Emoji` object: /emojis/{shortcode}, its id on other servers.
/// The plan's name, waiting for the owner's confirmation; it is frozen once it has federated (CLAUDE.md, routes).</summary>
public const string Path = "emojis";
public const int MaxShortcodeLength = 64;
public const int MaxPerObject = 64;//what ObjectShapes reads of anyone's object
const string ServerUsePrefix = "emoji:";
static readonly TimeSpan Staleness = TimeSpan.FromMinutes(5);
static readonly Regex Shortcode = new("^[a-zA-Z0-9_]{2,64}$", RegexOptions.CultureInvariant);
// `:name:` with no letter or digit right before or after it, as Mastodon scans texts (`:a::b:` is two)
static readonly Regex Named = new(@"(?<![a-zA-Z0-9]):([a-zA-Z0-9_]{2,64}):(?![a-zA-Z0-9])", RegexOptions.CultureInvariant);
public sealed record Live(string Id, string Shortcode, string Url, string StaticUrl, string ContentType, DateTime ImageUpdatedAt, string Category,
bool VisibleInPicker);
static volatile IReadOnlyDictionary<string, Live> _current = new Dictionary<string, Live>(StringComparer.Ordinal);
static DateTime _loadedAt = DateTime.MinValue;
static int _reloading;
/// <summary>The emoji offered now, by their exact shortcode.</summary>
public static IReadOnlyDictionary<string, Live> Current
{
get
{
RefreshIfStale();
return _current;
}
}
public static bool IsShortcode(string text) => !string.IsNullOrEmpty(text) && Shortcode.IsMatch(text);
/// <summary>The id an emoji federates under: {base}/emojis/{shortcode}.</summary>
public static string Uri(string baseAddress, string shortcode) => $"{baseAddress}/{Path}/{shortcode}";
/// <summary>What holds its picture (MediaAttachment.ServerUse): the janitor and the media audit leave such rows alone.</summary>
public static string ServerUse(string shortcode) => ServerUsePrefix + shortcode;
public static bool IsEmojiUse(string serverUse) => serverUse?.StartsWith(ServerUsePrefix, StringComparison.Ordinal) == true;
public static Live Find(string shortcode) =>
shortcode != default && Current.TryGetValue(shortcode, out var emoji) ? emoji : default;
public static async Task Reload(CancellationToken token = default)
{
var offered = await DB.Default.Find<LocalEmoji>().Match(e => e.DeletedAt == null && !e.Disabled).ExecuteAsync(token);
_current = offered.Where(e => IsShortcode(e.Shortcode) && !string.IsNullOrEmpty(e.Url))
.GroupBy(e => e.Shortcode, StringComparer.Ordinal)
.ToDictionary(g => g.Key, g => g.Select(e => new Live(e.ID, e.Shortcode, e.Url, e.StaticUrl ?? e.Url, e.ContentType ?? "image/png", e.ImageUpdatedAt,
e.Category, e.VisibleInPicker)).First(), StringComparer.Ordinal);
_loadedAt = DateTime.UtcNow;
}
static void RefreshIfStale()
{
if (DateTime.UtcNow - _loadedAt < Staleness || Interlocked.Exchange(ref _reloading, 1) == 1)
return;
_ = Task.Run(async () =>
{
try
{
await Reload(CancellationToken.None);
}
catch (Exception ex)
{
Serilog.Log.ForContext(typeof(LocalEmojis)).Warning(ex, "The custom emoji could not be reloaded");
}
finally
{
Interlocked.Exchange(ref _reloading, 0);
}
});
}
/// <summary>The emoji offered now that the texts name as `:shortcode:`, as a post or a profile keeps them; remote
/// shortcodes are not ours and are never taken.</summary>
public static List<CustomEmoji> In(IEnumerable<string> texts)
{
var current = Current;
if (current.Count == 0)
return new List<CustomEmoji>();
return texts.Where(t => !string.IsNullOrEmpty(t))
.SelectMany(t => Named.Matches(t).Select(m => m.Groups[1].Value))
.Distinct(StringComparer.Ordinal)
.Select(code => current.GetValueOrDefault(code))
.Where(e => e != default)
.Take(MaxPerObject)
.Select(e => new CustomEmoji { Shortcode = e.Shortcode, URL = e.Url, Updated = e.ImageUpdatedAt })
.ToList();
}
public static List<CustomEmoji> In(params string[] texts) => In((IEnumerable<string>)texts);
/// <summary>A persona's emoji: those its name, its bio and its fields name.</summary>
public static List<CustomEmoji> OfProfile(Models.User.Avatar avatar) =>
In(new[] { avatar.Name, avatar.Biography }.Concat((avatar.Fields ?? new()).SelectMany(f => new[] { f.Key, f.Value })));
/// <summary>What a local post's or persona's kept emoji are now: those still offered, as they are now.</summary>
public static List<Live> Of(IEnumerable<CustomEmoji> kept)
{
var current = Current;
if (kept == default || current.Count == 0)
return new List<Live>();
return kept.Where(e => e?.Shortcode != default).Select(e => current.GetValueOrDefault(e.Shortcode)).Where(e => e != default)
.DistinctBy(e => e.Shortcode).Take(MaxPerObject).ToList();
}
/// <summary>The local emoji a reaction names: `:shortcode:`, `:shortcode@this-host:` or the bare shortcode (Mastodon's
/// announcements, Pleroma's clients); none for a unicode emoji or another server's.</summary>
public static Live ForReaction(string content, string domain)
{
var text = content?.Trim();
if (string.IsNullOrEmpty(text) || text.Length > 300)
return default;
if (text.Length > 2 && text.StartsWith(':') && text.EndsWith(':'))
text = text[1..^1];
var at = text.IndexOf('@');
if (at >= 0)
{
if (!string.Equals(text[(at + 1)..], domain, StringComparison.OrdinalIgnoreCase))
return default;
text = text[..at];
}
return IsShortcode(text) ? Find(text) : default;
}
/// <summary>An emoji as a tag of a Note, Question, actor or EmojiReact, in Mastodon's shape (GoToSocial needs its id).</summary>
public static JsonObject Tag(Live emoji, string baseAddress) => Build(emoji, baseAddress, new JsonObject());
/// <summary>The document served at the emoji's id: the same object, with the context.</summary>
public static JsonObject Document(Live emoji, string baseAddress) =>
Build(emoji, baseAddress, new JsonObject { ["@context"] = ActivityPubRenderer.Context() });
static JsonObject Build(Live emoji, string baseAddress, JsonObject into)
{
into["id"] = Uri(baseAddress, emoji.Shortcode);
into["type"] = "Emoji";
into["name"] = $":{emoji.Shortcode}:";
into["updated"] = ActivityPubRenderer.Timestamp(emoji.ImageUpdatedAt);
into["icon"] = new JsonObject { ["type"] = "Image", ["mediaType"] = emoji.ContentType, ["url"] = emoji.Url };
return into;
}
}
}
+5 -2
View File
@@ -10,7 +10,7 @@ namespace PrivaPub.Domain.Media
// What the media directory holds against what the database says holds it, and, with fix, the two made to agree: // What the media directory holds against what the database says holds it, and, with fix, the two made to agree:
// - a persona's current avatar or header with no row (they had none before) becomes a row of its own; // - a persona's current avatar or header with no row (they had none before) becomes a row of its own;
// - media of a deleted post or of a deleted persona go to the trash, and a picture of the server's own (ServerUse) that // - media of a deleted post or of a deleted persona go to the trash, and a picture of the server's own (ServerUse) that
// its settings no longer name; // its settings no longer name (its thumbnail) or no custom emoji holds;
// - rows whose files are gone go to the trash; // - rows whose files are gone go to the trash;
// - files nothing holds (deleted posts' media and replaced pictures, from before media were trashed) get a trashed row // - files nothing holds (deleted posts' media and replaced pictures, from before media were trashed) get a trashed row
// each. // each.
@@ -60,9 +60,12 @@ namespace PrivaPub.Domain.Media
var ownerIds = rows.Select(r => r.OwnerAvatarId).Where(id => id != null).Distinct().ToList(); var ownerIds = rows.Select(r => r.OwnerAvatarId).Where(id => id != null).Distinct().ToList();
var liveOwners = (await DB.Default.Find<Avatar>().Match(a => ownerIds.Contains(a.ID) && !a.DeletionAt.HasValue).Project(a => a.Include(x => x.ID)).ExecuteAsync(token)) var liveOwners = (await DB.Default.Find<Avatar>().Match(a => ownerIds.Contains(a.ID) && !a.DeletionAt.HasValue).Project(a => a.Include(x => x.ID)).ExecuteAsync(token))
.Select(a => a.ID).ToHashSet(); .Select(a => a.ID).ToHashSet();
// (the server's own pictures belong to nobody: they are held while the settings name them) // (the server's own pictures belong to nobody: they are held while the settings name them, or a custom emoji that is
// not deleted, disabled ones too)
var settings = await DB.Default.Find<Models.ServerSettings>().Match(x => x.ID == Models.ServerSettings.Name).ExecuteFirstAsync(token); var settings = await DB.Default.Find<Models.ServerSettings>().Match(x => x.ID == Models.ServerSettings.Name).ExecuteFirstAsync(token);
var serverHeld = new HashSet<string> { settings?.ThumbnailMediaId }; var serverHeld = new HashSet<string> { settings?.ThumbnailMediaId };
serverHeld.UnionWith((await DB.Default.Find<Models.Post.LocalEmoji>().Match(e => e.DeletedAt == null).Project(e => e.Include(x => x.MediaId))
.ExecuteAsync(token)).Select(e => e.MediaId).Where(id => id != null));
var ofDeleted = rows.Where(r => r.ServerUse != null ? !serverHeld.Contains(r.ID) var ofDeleted = rows.Where(r => r.ServerUse != null ? !serverHeld.Contains(r.ID)
: (r.PostId != null && !livePosts.Contains(r.PostId)) || (r.OwnerAvatarId != null && !liveOwners.Contains(r.OwnerAvatarId))).ToList(); : (r.PostId != null && !livePosts.Contains(r.PostId)) || (r.OwnerAvatarId != null && !liveOwners.Contains(r.OwnerAvatarId))).ToList();
var missing = rows.Except(ofDeleted).Where(r => !string.IsNullOrEmpty(r.FilePath) && !files.Contains(r.FilePath)).ToList(); var missing = rows.Except(ofDeleted).Where(r => !string.IsNullOrEmpty(r.FilePath) && !files.Contains(r.FilePath)).ToList();
+106
View File
@@ -49,6 +49,10 @@ namespace PrivaPub.Domain.Media
/// <summary>A picture of the server's own (its thumbnail, "thumbnail"), fitted within 1200×630, as a row nobody owns, /// <summary>A picture of the server's own (its thumbnail, "thumbnail"), fitted within 1200×630, as a row nobody owns,
/// held by its <see cref="MediaAttachment.ServerUse"/>.</summary> /// held by its <see cref="MediaAttachment.ServerUse"/>.</summary>
Task<MediaOutcome> ServerImage(string use, IFormFile file, CancellationToken token); Task<MediaOutcome> ServerImage(string use, IFormFile file, CancellationToken token);
/// <summary>A custom emoji's picture (R7): a PNG at most 256 px on its longer side, alpha kept, or an animated GIF
/// that stays one, small enough for every peer (EmojiMaxBytes); a row nobody owns, held by ServerUse
/// "emoji:<shortcode>".</summary>
Task<MediaOutcome> EmojiImage(string shortcode, IFormFile file, CancellationToken token);
/// <summary>Trashes the media <paramref name="which"/> matches (and that isn't trashed yet): each row is marked in one /// <summary>Trashes the media <paramref name="which"/> matches (and that isn't trashed yet): each row is marked in one
/// conditional update, so a row attached meanwhile is left alone, and its files stop being served at once.</summary> /// conditional update, so a row attached meanwhile is left alone, and its files stop being served at once.</summary>
Task<long> Trash(System.Linq.Expressions.Expression<Func<MediaAttachment, bool>> which, string reason, CancellationToken token); Task<long> Trash(System.Linq.Expressions.Expression<Func<MediaAttachment, bool>> which, string reason, CancellationToken token);
@@ -299,6 +303,108 @@ namespace PrivaPub.Domain.Media
return new MediaOutcome(attachment); return new MediaOutcome(attachment);
} }
// what an emoji's picture may be sent as, and how big (Mastodon's limit for one, 256 KB)
static readonly string[] EmojiTypes = { "image/png", "image/gif", "image/webp", "image/jpeg" };
public const long EmojiMaxUploadBytes = 256 * 1024;
// what it may be once made: GoToSocial's default for another server's emoji (media-emoji-remote-max-size, 100 KiB) is
// the smallest any peer takes; Mastodon's is 256 KB
public const long EmojiMaxBytes = 100 * 1024;
public const int EmojiSide = 256;
// tried in turn until the picture fits EmojiMaxBytes: 256 px as it is, then with a palette, then fewer pixels
static readonly (int Side, bool Palette)[] EmojiAttempts = { (256, false), (256, true), (192, true), (128, true), (96, true), (64, true) };
// As safe as any upload (checked before it is decoded, the allowed loaders alone, written without its metadata).
// A still picture becomes a PNG, its transparency kept; an animated GIF or WebP becomes an animated GIF, since a PNG
// would lose the animation and GIF is the animated picture every peer shows as an emoji, with a still PNG of its first
// frame for static_url.
public async Task<MediaOutcome> EmojiImage(string shortcode, IFormFile file, CancellationToken token)
{
var contentType = file?.ContentType?.Split(';')[0].Trim().ToLowerInvariant();
if (file == default || file.Length == 0)
return MediaOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Image can't be blank");
if (!EmojiTypes.Contains(contentType))
return MediaOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: An emoji's image is a PNG, GIF, WebP or JPEG");
if (file.Length > EmojiMaxUploadBytes)
return MediaOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: An emoji's image is at most 256 KB");
var input = await Read(file, token);
if (Refusal(input, _options.CurrentValue) is { } refused)
return refused;
byte[] bytes, still = default;
int width, height;
bool animated;
await _processing.WaitAsync(token);
try
{
var header = Inspect(input);
animated = header.Pages > 1 && (header.Loader?.StartsWith("gifload", StringComparison.Ordinal) == true
|| header.Loader?.StartsWith("webpload", StringComparison.Ordinal) == true);
(bytes, still, width, height) = animated ? EmojiAnimation(input) : EmojiStill(input);
}
catch (VipsException ex)
{
_logger.LogInformation("Refused an emoji's picture: {Error}", ex.Message);
return MediaOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: The file is not a readable image");
}
finally
{
_processing.Release();
}
if (bytes == default)
return MediaOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: The image can't be made small enough for an emoji (100 KB)");
var attachment = new MediaAttachment
{
Kind = "emoji",
ServerUse = Content.LocalEmojis.ServerUse(shortcode),
ContentType = animated ? "image/gif" : "image/png",
FilePath = await Save(bytes, animated ? "gif" : "png", token),
PreviewPath = still == default ? default : await Save(still, "png", token),
Size = bytes.Length,
Width = width,
Height = height,
AttachedAt = DateTime.UtcNow
};
await DB.Default.SaveAsync(attachment, token);
return new MediaOutcome(attachment);
}
static (byte[] Bytes, byte[] Still, int Width, int Height) EmojiStill(byte[] input)
{
foreach (var (side, palette) in EmojiAttempts)
{
using var image = Image.ThumbnailBuffer(input, side, height: side, size: Enums.Size.Down);
using var pixels = EmojiPixels(image);
var bytes = pixels.WriteToBuffer(palette ? ".png[keep=none,compression=9,palette=true]" : ".png[keep=none,compression=9]");
if (bytes.Length <= EmojiMaxBytes)
return (bytes, default, pixels.Width, pixels.Height);
}
return default;
}
// every frame shrunk to the side (thumbnail with n=-1 resizes each page), written as a GIF; its first frame as a PNG
static (byte[] Bytes, byte[] Still, int Width, int Height) EmojiAnimation(byte[] input)
{
foreach (var side in EmojiAttempts.Select(a => a.Side).Distinct())
{
using var frames = Image.ThumbnailBuffer(input, side, height: side, optionString: "n=-1", size: Enums.Size.Down);
using var pixels = EmojiPixels(frames);
var bytes = pixels.WriteToBuffer(".gif[keep=none]");
if (bytes.Length > EmojiMaxBytes)
continue;
using var first = Image.ThumbnailBuffer(input, side, height: side, size: Enums.Size.Down);
using var still = EmojiPixels(first);
var height = pixels.Contains("page-height") ? (int)pixels.Get("page-height") : pixels.Height;
return (bytes, still.WriteToBuffer(".png[keep=none,compression=9,palette=true]"), pixels.Width, height);
}
return default;
}
// 8-bit sRGB, its alpha kept (thumbnail already brought its colours into sRGB; a 16-bit or grey picture is made so)
static Image EmojiPixels(Image image)
{
using var srgb = image.Interpretation == Enums.Interpretation.Srgb ? image.Copy() : image.Colourspace(Enums.Interpretation.Srgb);
return srgb.Format == Enums.BandFormat.Uchar ? srgb.Copy() : srgb.Cast(Enums.BandFormat.Uchar);
}
// a login over its quota (Media:QuotaBytesPerRoot; none when 0) uploads nothing more // a login over its quota (Media:QuotaBytesPerRoot; none when 0) uploads nothing more
async Task<MediaOutcome> OverQuota(string avatarId, long incoming, CancellationToken token) async Task<MediaOutcome> OverQuota(string avatarId, long incoming, CancellationToken token)
{ {
@@ -367,6 +367,9 @@ namespace PrivaPub.Domain.Portability
} }
if (Text(note["type"]) == "Question") if (Text(note["type"]) == "Question")
post.Poll = Poll(note); post.Poll = Poll(note);
// this server's custom emoji the words name, as a post written here gets them (R7)
post.Emojis = Content.LocalEmojis.In(new[] { post.Text ?? post.ContentHtml, post.Title, post.SpoilerText }
.Concat(post.Poll?.Options.Select(o => o.Title) ?? Enumerable.Empty<string>()));
// the conversation (FEP-7888) of its thread's first post, when the thread starts in the archive // the conversation (FEP-7888) of its thread's first post, when the thread starts in the archive
if (visibility is PostVisibility.Public or PostVisibility.Unlisted && Root(run, original) is { } root && run.Posts.TryGetValue(root, out var first)) if (visibility is PostVisibility.Public or PostVisibility.Unlisted && Root(run, original) is { } root && run.Posts.TryGetValue(root, out var first))
post.ContextURI = first.Uri + "/context"; post.ContextURI = first.Uri + "/context";
@@ -812,6 +815,7 @@ namespace PrivaPub.Domain.Portability
.Take(4).ToDictionary(f => Text(f["name"]).Trim(), f => Plain(Text(f["value"]) ?? string.Empty)); .Take(4).ToDictionary(f => Text(f["name"]).Trim(), f => Plain(Text(f["value"]) ?? string.Empty));
if (fields.Count > 0) if (fields.Count > 0)
avatar.Fields = fields; avatar.Fields = fields;
avatar.Emojis = Content.LocalEmojis.OfProfile(avatar);//this server's custom emoji the profile names (R7)
if (actor["manuallyApprovesFollowers"]?.GetValueKind() is { } locked and (System.Text.Json.JsonValueKind.True or System.Text.Json.JsonValueKind.False)) if (actor["manuallyApprovesFollowers"]?.GetValueKind() is { } locked and (System.Text.Json.JsonValueKind.True or System.Text.Json.JsonValueKind.False))
avatar.Settings.IsLocked = locked == System.Text.Json.JsonValueKind.True; avatar.Settings.IsLocked = locked == System.Text.Json.JsonValueKind.True;
if (actor["discoverable"]?.GetValueKind() is { } discoverable and (System.Text.Json.JsonValueKind.True or System.Text.Json.JsonValueKind.False)) if (actor["discoverable"]?.GetValueKind() is { } discoverable and (System.Text.Json.JsonValueKind.True or System.Text.Json.JsonValueKind.False))
+11 -5
View File
@@ -73,11 +73,17 @@ namespace PrivaPub.Domain.Social
{ {
if (await FindShown(id, token) == default) if (await FindShown(id, token) == default)
return (StatusCodes.Status404NotFound, "Record not found"); return (StatusCodes.Status404NotFound, "Record not found");
// one unicode emoji (custom emoji come with R7): the same normalisation as a post's reactions // one unicode emoji, normalised as a post's reactions are, or one of the server's custom emoji that is offered, by its
var picked = Reactions.Normalise(name, new List<Models.Post.CustomEmoji>(), requireKnownCustom: true); // shortcode (R7), as Mastodon takes them
if (picked == default || picked.Url != default || picked.Name.Length > MaxNameLength) if (LocalEmojis.ForReaction(name, default) is { } custom)
return (StatusCodes.Status422UnprocessableEntity, "Validation failed: Name is not a recognized emoji"); name = custom.Shortcode;
name = picked.Name; else
{
var picked = Reactions.Normalise(name, new List<Models.Post.CustomEmoji>(), requireKnownCustom: true);
if (picked == default || picked.Url != default || picked.Name.Length > MaxNameLength)
return (StatusCodes.Status422UnprocessableEntity, "Validation failed: Name is not a recognized emoji");
name = picked.Name;
}
if (!add) if (!add)
{ {
var removed = await DB.Default.DeleteAsync<AnnouncementReaction>(r => r.AnnouncementId == id && r.AvatarId == avatarId && r.Name == name); var removed = await DB.Default.DeleteAsync<AnnouncementReaction>(r => r.AnnouncementId == id && r.AvatarId == avatarId && r.Name == name);
+24 -12
View File
@@ -1,6 +1,7 @@
using MongoDB.Driver; using MongoDB.Driver;
using MongoDB.Entities; using MongoDB.Entities;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Privacy; using PrivaPub.Domain.Privacy;
using PrivaPub.Federation.Actors; using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Inbox; using PrivaPub.Federation.Inbox;
@@ -84,13 +85,20 @@ namespace PrivaPub.Domain.Social
return true; return true;
} }
// a unicode emoji, one of this server's custom emoji that is offered (`:code:`, as Akkoma names its own; R7), or a custom
// emoji the post or its reactions already show (`:code@host:` for another server's)
public async Task<bool> React(LocalActor me, PostEntity post, string emoji, CancellationToken token) public async Task<bool> React(LocalActor me, PostEntity post, string emoji, CancellationToken token)
{ {
var known = post.Emojis.Select(e => (e.Shortcode, e.URL)) var picked = LocalEmojis.ForReaction(emoji, me.Domain) is { } local ? new Picked(local.Shortcode, local.Url) : default;
.Concat((await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.EmojiURL != null).ExecuteAsync(token)).Select(r => (r.Emoji, r.EmojiURL))) if (picked == default)
.Select(e => new Models.Post.CustomEmoji { Shortcode = e.Item1, URL = e.Item2 }) {
.ToList(); var known = post.Emojis.Select(e => (e.Shortcode, e.URL))
if (Normalise(emoji, known) is not { } picked) .Concat((await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.EmojiURL != null).ExecuteAsync(token)).Select(r => (r.Emoji, r.EmojiURL)))
.Select(e => new Models.Post.CustomEmoji { Shortcode = e.Item1, URL = e.Item2 })
.ToList();
picked = Normalise(emoji, known);
}
if (picked == default)
return false; return false;
var reaction = new Reaction var reaction = new Reaction
{ {
@@ -114,7 +122,7 @@ namespace PrivaPub.Domain.Social
public async Task<bool> Unreact(LocalActor me, PostEntity post, string emoji, CancellationToken token) public async Task<bool> Unreact(LocalActor me, PostEntity post, string emoji, CancellationToken token)
{ {
var name = Normalise(emoji, post.Emojis.ToList(), requireKnownCustom: false)?.Name; var name = LocalEmojis.ForReaction(emoji, me.Domain)?.Shortcode ?? Normalise(emoji, post.Emojis.ToList(), requireKnownCustom: false)?.Name;
var reaction = name == default var reaction = name == default
? default ? default
: await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.ActorURI == me.Uri && r.Emoji == name).ExecuteFirstAsync(token); : await DB.Default.Find<Reaction>().Match(r => r.PostId == post.ID && r.ActorURI == me.Uri && r.Emoji == name).ExecuteFirstAsync(token);
@@ -145,13 +153,17 @@ namespace PrivaPub.Domain.Social
["to"] = new JsonArray(post.ActorURI), ["to"] = new JsonArray(post.ActorURI),
["content"] = reaction.EmojiURL == default ? reaction.Emoji : $":{reaction.Emoji}:" ["content"] = reaction.EmojiURL == default ? reaction.Emoji : $":{reaction.Emoji}:"
}; };
// one of this server's emoji goes as posts carry it (its id, `updated`, the picture's type), as Misskey and Akkoma
// register it; another server's as the post showed it
if (reaction.EmojiURL != default) if (reaction.EmojiURL != default)
react["tag"] = new JsonArray(new JsonObject react["tag"] = new JsonArray(LocalEmojis.Find(reaction.Emoji) is { } local && local.Url == reaction.EmojiURL
{ ? LocalEmojis.Tag(local, me.BaseAddress)
["type"] = "Emoji", : new JsonObject
["name"] = $":{reaction.Emoji}:", {
["icon"] = new JsonObject { ["type"] = "Image", ["url"] = reaction.EmojiURL } ["type"] = "Emoji",
}); ["name"] = $":{reaction.Emoji}:",
["icon"] = new JsonObject { ["type"] = "Image", ["url"] = reaction.EmojiURL }
});
return react; return react;
} }
+8 -1
View File
@@ -230,6 +230,7 @@ namespace PrivaPub.Domain.Statuses
QuoteState = quoted == default ? QuoteState.None : quotePermission == QuotePermission.Granted ? QuoteState.Accepted : QuoteState.Pending, QuoteState = quoted == default ? QuoteState.None : quotePermission == QuotePermission.Granted ? QuoteState.Accepted : QuoteState.Pending,
Approval = replyPermission == QuotePermission.AskFirst ? ApprovalState.Pending : ApprovalState.None Approval = replyPermission == QuotePermission.AskFirst ? ApprovalState.Pending : ApprovalState.None
}; };
post.Emojis = Emojis(post);
post.ID = (string)post.GenerateNewID(); post.ID = (string)post.GenerateNewID();
post.ObjectURI = author.PostUri(post.ID); post.ObjectURI = author.PostUri(post.ID);
post.Url = author.PostHtmlUrl(post.ID); post.Url = author.PostHtmlUrl(post.ID);
@@ -300,6 +301,10 @@ namespace PrivaPub.Domain.Statuses
return new StatusOutcome(post); return new StatusOutcome(post);
} }
// the server's custom emoji its words, title, warning and poll options name (R7): only ours that are offered now
static List<CustomEmoji> Emojis(PostEntity post) =>
LocalEmojis.In(new[] { post.Text, post.Title, post.SpoilerText }.Concat(post.Poll?.Options.Select(o => o.Title) ?? Enumerable.Empty<string>()));
public async Task<StatusOutcome> Edit(LocalActor author, string postId, StatusDraft draft, CancellationToken token) public async Task<StatusOutcome> Edit(LocalActor author, string postId, StatusDraft draft, CancellationToken token)
{ {
var post = await Own(author, postId, token); var post = await Own(author, postId, token);
@@ -319,7 +324,8 @@ namespace PrivaPub.Domain.Statuses
SpoilerText = post.SpoilerText, SpoilerText = post.SpoilerText,
ContentHtml = post.ContentHtml, ContentHtml = post.ContentHtml,
HasContentWarning = post.HasContentWarning, HasContentWarning = post.HasContentWarning,
EditedAt = post.EditedAt ?? post.CreationDate EditedAt = post.EditedAt ?? post.CreationDate,
Emojis = post.Emojis.ToList()
}); });
if (post.Revisions.Count > MaxRevisions) if (post.Revisions.Count > MaxRevisions)
post.Revisions.RemoveRange(0, post.Revisions.Count - MaxRevisions); post.Revisions.RemoveRange(0, post.Revisions.Count - MaxRevisions);
@@ -363,6 +369,7 @@ namespace PrivaPub.Domain.Statuses
? post.Mentions.Concat(rendered.Mentions.Select(ToMention)).DistinctBy(m => m.ActorURI).ToList() ? post.Mentions.Concat(rendered.Mentions.Select(ToMention)).DistinctBy(m => m.ActorURI).ToList()
: rendered.Mentions.Select(ToMention).ToList(); : rendered.Mentions.Select(ToMention).ToList();
post.Tags = rendered.Tags.ToList(); post.Tags = rendered.Tags.ToList();
post.Emojis = Emojis(post);
post.EditedAt = DateTime.UtcNow; post.EditedAt = DateTime.UtcNow;
post.UpdateDate = post.EditedAt; post.UpdateDate = post.EditedAt;
await DB.Default.SaveAsync(post, token); await DB.Default.SaveAsync(post, token);
@@ -43,6 +43,8 @@ namespace PrivaPub.Federation.Actors
public DateTime Published { get; init; } public DateTime Published { get; init; }
public string BaseAddress { get; init; } public string BaseAddress { get; init; }
public IReadOnlyDictionary<string, string> Fields { get; init; } = new Dictionary<string, string>(); public IReadOnlyDictionary<string, string> Fields { get; init; } = new Dictionary<string, string>();
// the custom emoji of this server its profile named (R7); rendered as they are now (LocalEmojis.Of)
public IReadOnlyList<Models.Post.CustomEmoji> Emojis { get; init; } = Array.Empty<Models.Post.CustomEmoji>();
public string Uri => $"{BaseAddress}/peasants/{UserName}"; public string Uri => $"{BaseAddress}/peasants/{UserName}";
public string KeyId => $"{Uri}#main-key"; public string KeyId => $"{Uri}#main-key";
@@ -335,6 +337,7 @@ namespace PrivaPub.Federation.Actors
SigningKey = avatar.SigningKey, SigningKey = avatar.SigningKey,
Published = avatar.PublishedOn, Published = avatar.PublishedOn,
Fields = avatar.Fields ?? new Dictionary<string, string>(), Fields = avatar.Fields ?? new Dictionary<string, string>(),
Emojis = avatar.Emojis ?? new List<Models.Post.CustomEmoji>(),
ManuallyApprovesFollowers = avatar.Settings?.IsLocked == true, ManuallyApprovesFollowers = avatar.Settings?.IsLocked == true,
Discoverable = avatar.Settings?.IsDiscoverable != false, Discoverable = avatar.Settings?.IsDiscoverable != false,
IsBot = avatar.Settings?.IsBot == true, IsBot = avatar.Settings?.IsBot == true,
@@ -0,0 +1,77 @@
using Microsoft.AspNetCore.Mvc;
using MongoDB.Entities;
using PrivaPub.Domain.Content;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Rendering;
using PrivaPub.Models.Post;
using System.Text.Json.Nodes;
namespace PrivaPub.Federation.Controllers
{
// Each of the server's custom emoji as an ActivityPub `Emoji` object at its id, /emojis/{shortcode} (completion program
// R7; the path is LocalEmojis.Path): the tag that posts, actors and reactions carry, with its @context. GoToSocial takes
// an emoji only with an id, and may read it there. A disabled emoji is not served (404, it may come back); a deleted one
// answers 410 with a Tombstone. An emoji tells nothing about anyone, so it is served unsigned under SecureMode too, as the
// server's own actors are; a browser is sent to the picture.
[ApiController]
public class EmojisController : ControllerBase
{
const string ActivityContentType = "application/activity+json; charset=utf-8";
readonly ILocalActorService _localActors;
public EmojisController(ILocalActorService localActors)
{
_localActors = localActors;
}
[HttpGet, Route("/" + LocalEmojis.Path + "/{shortcode}")]
public async Task<IActionResult> Get(string shortcode, CancellationToken token)
{
Response.Headers.Vary = "Accept";
if (!LocalEmojis.IsShortcode(shortcode))
return NotFound();
var key = shortcode.ToLowerInvariant();
var emoji = await DB.Default.Find<LocalEmoji>().Match(e => e.Key == key).ExecuteFirstAsync(token);
if (emoji == default || emoji.Shortcode != shortcode)
return NotFound();
if (emoji.DeletedAt is { } deleted)
return new ContentResult
{
Content = new JsonObject
{
["@context"] = ActivityPubRenderer.ActivityStreams,
["id"] = LocalEmojis.Uri(_localActors.BaseAddress, emoji.Shortcode),
["type"] = "Tombstone",
["formerType"] = "Emoji",
["deleted"] = ActivityPubRenderer.Timestamp(deleted)
}.ToJsonString(),
ContentType = ActivityContentType,
StatusCode = StatusCodes.Status410Gone
};
if (emoji.Disabled || string.IsNullOrEmpty(emoji.Url))
return NotFound();
if (WantsHtml())
return Redirect(emoji.Url);
var live = new LocalEmojis.Live(emoji.ID, emoji.Shortcode, emoji.Url, emoji.StaticUrl ?? emoji.Url, emoji.ContentType ?? "image/png",
emoji.ImageUpdatedAt, emoji.Category, emoji.VisibleInPicker);
return new ContentResult
{
Content = LocalEmojis.Document(live, _localActors.BaseAddress).ToJsonString(),
ContentType = ActivityContentType,
StatusCode = StatusCodes.Status200OK
};
}
bool WantsHtml()
{
var accept = Request.Headers.Accept.ToString();
return accept.Contains("text/html", StringComparison.OrdinalIgnoreCase)
&& !accept.Contains("activity+json", StringComparison.OrdinalIgnoreCase)
&& !accept.Contains("ld+json", StringComparison.OrdinalIgnoreCase);
}
}
}
+2 -1
View File
@@ -60,7 +60,8 @@ namespace PrivaPub.Federation.Inbox
SpoilerText = post.SpoilerText, SpoilerText = post.SpoilerText,
ContentHtml = post.ContentHtml, ContentHtml = post.ContentHtml,
HasContentWarning = post.HasContentWarning, HasContentWarning = post.HasContentWarning,
EditedAt = post.EditedAt ?? post.CreationDate EditedAt = post.EditedAt ?? post.CreationDate,
Emojis = post.Emojis.ToList()
}); });
if (post.Revisions.Count > MaxRevisions) if (post.Revisions.Count > MaxRevisions)
post.Revisions.RemoveRange(0, post.Revisions.Count - MaxRevisions); post.Revisions.RemoveRange(0, post.Revisions.Count - MaxRevisions);
+2 -1
View File
@@ -165,7 +165,8 @@ namespace PrivaPub.Federation.Objects
SpoilerText = warns ? Plain(summary, 500) : default, SpoilerText = warns ? Plain(summary, 500) : default,
ContentHtml = ContentSanitizer.Html(Content(version).Html), ContentHtml = ContentSanitizer.Html(Content(version).Html),
HasContentWarning = sensitive, HasContentWarning = sensitive,
EditedAt = Time(Value(version, "updated")) ?? Time(Value(version, "published")) ?? default EditedAt = Time(Value(version, "updated")) ?? Time(Value(version, "published")) ?? default,
Emojis = ObjectShapes.Emojis(version["tag"])
}; };
}) })
.Reverse() .Reverse()
@@ -4,6 +4,7 @@ using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects; using PrivaPub.Federation.Objects;
using PrivaPub.Models.Federation; using PrivaPub.Models.Federation;
using PrivaPub.Models.Post; using PrivaPub.Models.Post;
using PrivaPub.Domain.Content;
using PrivaPub.Domain.Media; using PrivaPub.Domain.Media;
using System.Globalization; using System.Globalization;
@@ -188,6 +189,10 @@ namespace PrivaPub.Federation.Rendering
["controller"] = actor.Uri, ["controller"] = actor.Uri,
["publicKeyMultibase"] = Signing.IntegrityProofs.Multikey(Signing.IntegrityProofs.PublicKey(actor.SigningKey)) ["publicKeyMultibase"] = Signing.IntegrityProofs.Multikey(Signing.IntegrityProofs.PublicKey(actor.SigningKey))
}); });
// the server's custom emoji its name, bio and fields use (R7), only when there are some
var emojis = LocalEmojis.Of(actor.Emojis);
if (emojis.Count > 0)
document["tag"] = new JsonArray(emojis.Select(e => (JsonNode)LocalEmojis.Tag(e, actor.BaseAddress)).ToArray());
if (!string.IsNullOrEmpty(actor.PictureURL)) if (!string.IsNullOrEmpty(actor.PictureURL))
document["icon"] = new JsonObject { ["type"] = "Image", ["url"] = actor.PictureURL }; document["icon"] = new JsonObject { ["type"] = "Image", ["url"] = actor.PictureURL };
if (!string.IsNullOrEmpty(actor.ThumbnailURL)) if (!string.IsNullOrEmpty(actor.ThumbnailURL))
@@ -355,6 +360,10 @@ namespace PrivaPub.Federation.Rendering
["href"] = TagUrl(author.BaseAddress, t), ["href"] = TagUrl(author.BaseAddress, t),
["name"] = "#" + t ["name"] = "#" + t
})) }))
// this server's custom emoji its words, warning and poll options use, as they are now (R7); another server's
// post (a copy we hold) carries its own
.Concat(post.IsFederatedCopy ? Enumerable.Empty<JsonNode>()
: LocalEmojis.Of(post.Emojis).Select(e => (JsonNode)LocalEmojis.Tag(e, author.BaseAddress)))
.ToArray()) .ToArray())
}; };
AddQuote(note, post, ref content); AddQuote(note, post, ref content);
@@ -15,9 +15,11 @@ namespace PrivaPub.Infrastructure.Backup
// - a root keeps P's password, e-mail, ban and policies; // - a root keeps P's password, e-mail, ban and policies;
// - roots, personas and groups made since become tombstones: deleted, their names kept; local posts made since answer // - roots, personas and groups made since become tombstones: deleted, their names kept; local posts made since answer
// 410; media made since go to the trash, as do media trashed since (and they leave the posts restored with them), // 410; media made since go to the trash, as do media trashed since (and they leave the posts restored with them),
// but the server's own pictures the live settings hold; // but the server's own pictures the live settings and custom emoji hold; a picture of the server's that neither holds
// any more goes to the trash;
// - the staff's log is the union (a restore never loses an entry, its own request included), and the server's // - the staff's log is the union (a restore never loses an entry, its own request included), and the server's
// settings are P's: the live settings outlive a restore; // settings are P's: the live settings outlive a restore; so do its custom emoji (R7), as P has them: one deleted since
// stays deleted (its id answers 410), one added or changed since stays as it is;
// - every session ends, and every server's circuit is closed again. // - every session ends, and every server's circuit is closed again.
// Run on the imported database, it reads P's files only, so it can run again after a failed attempt. // Run on the imported database, it reads P's files only, so it can run again after a failed attempt.
public static class ProtectiveMerge public static class ProtectiveMerge
@@ -44,8 +46,8 @@ namespace PrivaPub.Infrastructure.Backup
IEnumerable<BsonDocument> P(string collection) => ServerRestore.Read(Path.Combine(previous, collection + ".jsonl.gz")); IEnumerable<BsonDocument> P(string collection) => ServerRestore.Read(Path.Combine(previous, collection + ".jsonl.gz"));
var now = DateTime.UtcNow; var now = DateTime.UtcNow;
// who follows whom: P's; and the server's settings as they were live // who follows whom: P's; and the server's settings and custom emoji as they were live
foreach (var collection in new[] { "Follower", "Following", "ServerSettings" }) foreach (var collection in new[] { "Follower", "Following", "ServerSettings", "LocalEmoji" })
{ {
await database.DropCollectionAsync(collection, token); await database.DropCollectionAsync(collection, token);
await ServerRestore.Insert(database.GetCollection<BsonDocument>(collection), P(collection), token); await ServerRestore.Insert(database.GetCollection<BsonDocument>(collection), P(collection), token);
@@ -151,6 +153,7 @@ namespace PrivaPub.Infrastructure.Backup
await Posts(database, P("Post"), report, token); await Posts(database, P("Post"), report, token);
await Media(database, P("MediaAttachment"), report, token); await Media(database, P("MediaAttachment"), report, token);
await ServerPictures(database, report, token);
// what the staff took down stays down, and every post's flags follow the moderation now kept (R6) // what the staff took down stays down, and every post's flags follow the moderation now kept (R6)
foreach (var post in P("Post").Where(p => IsSet(p, "ModeratedAt"))) foreach (var post in P("Post").Where(p => IsSet(p, "ModeratedAt")))
await database.GetCollection<BsonDocument>("Post").UpdateOneAsync( await database.GetCollection<BsonDocument>("Post").UpdateOneAsync(
@@ -228,8 +231,8 @@ namespace PrivaPub.Infrastructure.Backup
{ {
if (!restored.TryGetValue(row["_id"], out var mine)) if (!restored.TryGetValue(row["_id"], out var mine))
{ {
// made since, and to the trash, but a picture of the server's own (its thumbnail): the live settings, // made since, and to the trash, but a picture of the server's own (its thumbnail, a custom emoji's):
// which a restore keeps, still show it // the live settings and emoji, which a restore keeps, still show it
if (!IsSet(row, "TrashedAt") && !IsSet(row, "ServerUse")) if (!IsSet(row, "TrashedAt") && !IsSet(row, "ServerUse"))
{ {
row["TrashedAt"] = now; row["TrashedAt"] = now;
@@ -258,6 +261,31 @@ namespace PrivaPub.Infrastructure.Backup
Builders<BsonDocument>.Update.Set("TrashedAt", now).Set("TrashReason", "restore: its post was deleted"), cancellationToken: token)).ModifiedCount; Builders<BsonDocument>.Update.Set("TrashedAt", now).Set("TrashReason", "restore: its post was deleted"), cancellationToken: token)).ModifiedCount;
} }
// the server's own pictures (ServerUse) that neither the settings nor a custom emoji, both P's now, hold: to the trash.
// One the backup still had and that was replaced and purged since would otherwise be held by nothing for ever.
static async Task ServerPictures(IMongoDatabase database, RestoreReport report, CancellationToken token)
{
var held = new HashSet<string>(StringComparer.Ordinal);
foreach (var settings in await (await database.GetCollection<BsonDocument>("ServerSettings").FindAsync(FilterDefinition<BsonDocument>.Empty,
cancellationToken: token)).ToListAsync(token))
if (settings.GetValue("ThumbnailMediaId", BsonNull.Value) is BsonString thumbnail)
held.Add(thumbnail.AsString);
foreach (var emoji in await (await database.GetCollection<BsonDocument>("LocalEmoji").FindAsync(new BsonDocument("DeletedAt", BsonNull.Value),
cancellationToken: token)).ToListAsync(token))
if (emoji.GetValue("MediaId", BsonNull.Value) is BsonString picture)
held.Add(picture.AsString);
var media = database.GetCollection<BsonDocument>("MediaAttachment");
var rows = await (await media.FindAsync(new BsonDocument { { "ServerUse", new BsonDocument("$ne", BsonNull.Value) }, { "TrashedAt", BsonNull.Value } },
new FindOptions<BsonDocument> { Projection = new BsonDocument("_id", 1) }, token)).ToListAsync(token);
var now = DateTime.UtcNow;
foreach (var row in rows.Where(r => !held.Contains(r["_id"].ToString())))
{
await media.UpdateOneAsync(Id(row["_id"]), Builders<BsonDocument>.Update.Set("TrashedAt", now).Set("TrashReason", "restore: no longer the server's"),
cancellationToken: token);
report.MediaTrashed++;
}
}
// P's rows added, each replacing the restored row with the same key: how many P had // P's rows added, each replacing the restored row with the same key: how many P had
static async Task<int> Union(IMongoCollection<BsonDocument> collection, IEnumerable<BsonDocument> previous, string[] key, CancellationToken token) static async Task<int> Union(IMongoCollection<BsonDocument> collection, IEnumerable<BsonDocument> previous, string[] key, CancellationToken token)
{ {
+2
View File
@@ -57,6 +57,8 @@ namespace PrivaPub.Infrastructure.Data
await DB.Default.Index<AnnouncementReaction>().Key(r => r.AnnouncementId, KeyType.Ascending).Key(r => r.AvatarId, KeyType.Ascending) await DB.Default.Index<AnnouncementReaction>().Key(r => r.AnnouncementId, KeyType.Ascending).Key(r => r.AvatarId, KeyType.Ascending)
.Key(r => r.Name, KeyType.Ascending).Option(o => o.Unique = true).CreateAsync(token); .Key(r => r.Name, KeyType.Ascending).Option(o => o.Unique = true).CreateAsync(token);
await Plain<AnnouncementReaction>(token, r => r.AvatarId); await Plain<AnnouncementReaction>(token, r => r.AvatarId);
// the server's custom emoji, one per shortcode whatever its case (a new collection, R7)
await Unique<LocalEmoji>(e => e.Key, Builders<LocalEmoji>.Filter.Gt(e => e.Key, ""), token);
// the staff's log, read newest first // the staff's log, read newest first
await DB.Default.Index<Models.AdminAction>().Key(a => a.At, KeyType.Descending).CreateAsync(token); await DB.Default.Index<Models.AdminAction>().Key(a => a.At, KeyType.Descending).CreateAsync(token);
@@ -57,6 +57,8 @@ namespace PrivaPub.Infrastructure.Statistics
return "webfinger"; return "webfinger";
if (path.StartsWithSegments("/.well-known/nodeinfo") || path.StartsWithSegments("/nodeinfo")) if (path.StartsWithSegments("/.well-known/nodeinfo") || path.StartsWithSegments("/nodeinfo"))
return "nodeinfo"; return "nodeinfo";
if (path.StartsWithSegments("/" + Domain.Content.LocalEmojis.Path))
return "emoji";//a custom emoji's document (R7)
if (!path.StartsWithSegments("/peasants") && !path.StartsWithSegments("/users")) if (!path.StartsWithSegments("/peasants") && !path.StartsWithSegments("/users"))
return default; return default;
var segments = (template ?? path.Value ?? string.Empty).Trim('/').Split('/'); var segments = (template ?? path.Value ?? string.Empty).Trim('/').Split('/');
+4
View File
@@ -60,6 +60,10 @@ namespace PrivaPub.Models
public const string AccountUnsuspend = "account.unsuspend"; public const string AccountUnsuspend = "account.unsuspend";
public const string GroupDelete = "group.delete"; public const string GroupDelete = "group.delete";
public const string PostRemove = "post.remove"; public const string PostRemove = "post.remove";
// the server's custom emoji (completion program R7)
public const string EmojiInsert = "emoji.insert";
public const string EmojiUpdate = "emoji.update";
public const string EmojiDelete = "emoji.delete";
} }
public static class AdminTargets public static class AdminTargets
+29
View File
@@ -0,0 +1,29 @@
using MongoDB.Entities;
namespace PrivaPub.Models.Post
{
// One of this server's own custom emoji (completion program R7), added by an administrator
// (/clientapi/admin/emoji, EmojiAdminController). Its shortcode is its name everywhere: `:shortcode:` in a post, and
// its federated id (/emojis/{shortcode}, Domain/Content/LocalEmojis.Path), which is why it is never renamed. Its picture
// is a MediaAttachment nobody owns, held by ServerUse "emoji:<shortcode>" while the emoji exists. Deleted, the row stays
// (DeletedAt) so its id answers 410; adding the same shortcode again brings the row back with a new picture.
public class LocalEmoji : Entity
{
public string Shortcode { get; set; }//as written, [a-zA-Z0-9_]{2,64} (Mastodon's rule); matched exactly in texts
public string Key { get; set; }//the shortcode in lower case: unique, so `Blob` and `blob` are never two emoji
public string Category { get; set; }
public bool VisibleInPicker { get; set; } = true;
public bool Disabled { get; set; } = false;//kept, but offered nowhere and taken into no new post
public string MediaId { get; set; }//its picture's MediaAttachment (ServerUse "emoji:<shortcode>")
public string Url { get; set; }//the picture: a PNG, or an animated GIF
public string StaticUrl { get; set; }//a still PNG of it (the first frame of a GIF; the picture itself otherwise)
public string ContentType { get; set; }//image/png or image/gif
public int? Width { get; set; }
public int? Height { get; set; }
public long Size { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;//any change
public DateTime ImageUpdatedAt { get; set; } = DateTime.UtcNow;//its picture's: the Emoji's `updated`, which tells peers to fetch it again
public DateTime? DeletedAt { get; set; }
}
}
+1
View File
@@ -160,5 +160,6 @@ namespace PrivaPub.Models.Post
public string ContentHtml { get; set; } public string ContentHtml { get; set; }
public bool HasContentWarning { get; set; } public bool HasContentWarning { get; set; }
public DateTime EditedAt { get; set; } public DateTime EditedAt { get; set; }
public List<CustomEmoji> Emojis { get; set; } = new();//the post's custom emoji in that version (none kept before R7)
} }
} }
+2
View File
@@ -18,6 +18,8 @@ namespace PrivaPub.Models.User
public AvatarAccountState AccountState { get; set; } = AvatarAccountState.Normal; public AvatarAccountState AccountState { get; set; } = AvatarAccountState.Normal;
public AvatarSettings Settings { get; set; } = new(); public AvatarSettings Settings { get; set; } = new();
public Dictionary<string, string> Fields { get; set; } = new(); public Dictionary<string, string> Fields { get; set; } = new();
// this server's custom emoji its name, bio and fields named when they were saved (R7, Domain/Content/LocalEmojis)
public List<PrivaPub.Models.Post.CustomEmoji> Emojis { get; set; } = new();
public string Domain { get; set; } public string Domain { get; set; }
public string PersonalNote { get; set; } public string PersonalNote { get; set; }
+1
View File
@@ -232,6 +232,7 @@ try
await dbClient.Init(); await dbClient.Init();
await app.Services.GetRequiredService<PrivaPub.Federation.Moderation.IDomainBlocks>().Reload(CancellationToken.None); await app.Services.GetRequiredService<PrivaPub.Federation.Moderation.IDomainBlocks>().Reload(CancellationToken.None);
await PrivaPub.Domain.Moderation.Restrictions.Reload(CancellationToken.None);//who the staff's moderation holds back (R6) await PrivaPub.Domain.Moderation.Restrictions.Reload(CancellationToken.None);//who the staff's moderation holds back (R6)
await PrivaPub.Domain.Content.LocalEmojis.Reload(CancellationToken.None);//the server's custom emoji (R7)
await PrivaPub.Api.Mastodon.Auth.PersonaExchange.EnsureFirstPartyClient(app.Services, CancellationToken.None); await PrivaPub.Api.Mastodon.Auth.PersonaExchange.EnsureFirstPartyClient(app.Services, CancellationToken.None);
} }
catch (Exception ex) catch (Exception ex)
@@ -70,6 +70,7 @@ namespace PrivaPub.Services.ClientToServer.Private
SigningKey = Federation.Signing.IntegrityProofs.NewSeed(), SigningKey = Federation.Signing.IntegrityProofs.NewSeed(),
Domain = _localActors.BaseAddress Domain = _localActors.BaseAddress
}; };
newAvatar.Emojis = PrivaPub.Domain.Content.LocalEmojis.OfProfile(newAvatar);
newAvatar.ID = (string)newAvatar.GenerateNewID(); newAvatar.ID = (string)newAvatar.GenerateNewID();
if (!await _localActors.TryReserveUserName(userName, LocalActorKind.Person, newAvatar.ID, default)) if (!await _localActors.TryReserveUserName(userName, LocalActorKind.Person, newAvatar.ID, default))
return result.Invalidate(_localizer["The username '{0}' is already taken.", userName]); return result.Invalidate(_localizer["The username '{0}' is already taken.", userName]);
@@ -119,6 +120,7 @@ namespace PrivaPub.Services.ClientToServer.Private
avatar.Biography = form.Biography; avatar.Biography = form.Biography;
avatar.Fields = form.Fields ?? new(); avatar.Fields = form.Fields ?? new();
avatar.PersonalNote = form.PersonalNote; avatar.PersonalNote = form.PersonalNote;
avatar.Emojis = PrivaPub.Domain.Content.LocalEmojis.OfProfile(avatar);//the server's custom emoji its profile uses (R7)
if (form.Settings != default) if (form.Settings != default)
Apply(avatar.Settings ??= new(), form.Settings); Apply(avatar.Settings ??= new(), form.Settings);
avatar.UpdatedAt = DateTime.UtcNow; avatar.UpdatedAt = DateTime.UtcNow;
+14 -1
View File
@@ -126,6 +126,18 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
meanwhile), still counts among the users (counts tell nothing new), leaves a follow request unanswered, and its meanwhile), still counts among the users (counts tell nothing new), leaves a follow request unanswered, and its
scheduled posts wait; a suspended account elsewhere may still delete or undo what it sent; deleting a group asks its scheduled posts wait; a suspended account elsewhere may still delete or undo what it sent; deleting a group asks its
owner's password, like deleting a persona. owner's password, like deleting a persona.
- R7 custom emoji (built 2026-10-17, not yet deployed; `tools/pasture/scenarios/emoji.sh`): the server's own emoji
(`LocalEmoji`), added, changed and deleted by an administrator on `/clientapi/admin/emoji` and kept in the staff's log;
`GET /api/v1/custom_emojis`; taken into local posts (words, title, warning, poll options), profiles (name, bio, fields)
and reactions (Pleroma's reactions and announcements), sent as Mastodon's `Emoji` tags on Notes, Questions, actors and
`EmojiReact`s, each served as an `Emoji` at its id `/emojis/{shortcode}`. Chosen while building it, for the owner to
confirm: **the id's path `/emojis/{shortcode}`** (the plan's name, defined once as `LocalEmojis.Path`; it freezes once
federated); an emoji is never renamed (its shortcode is its id); its picture is a PNG within 256 px, or an animated GIF
(an animated WebP becomes one too), made smaller until it fits GoToSocial's 100 KiB; `custom_emojis` lists only those
in the picker, as Mastodon does; a disabled emoji is kept but offered and rendered nowhere (its id 404), a deleted one
410; what posts and profiles render is the emoji as they are now, and no `Update` is sent when one changes; its id is
served unsigned under SecureMode; a restore keeps the live emoji, as it keeps the settings (one deleted since stays
deleted).
## Intent ## Intent
@@ -588,7 +600,8 @@ The first refactor commit is a pure move with namespaces only. Logic changes fol
(server-sent events): `user`, `user:notification`, `public`, `public:local`, `public:remote`, `hashtag`, (server-sent events): `user`, `user:notification`, `public`, `public:local`, `public:remote`, `hashtag`,
`hashtag:local`, `list`; events `update`, `notification`, `status.update`, `delete`; `health`. Advertised as `hashtag:local`, `list`; events `update`, `notification`, `status.update`, `delete`; `health`. Advertised as
`urls.streaming`. `urls.streaming`.
- **Stubs:** `custom_emojis`, announcements, suggestions, preferences. - **Stubs:** `custom_emojis`, announcements, suggestions, preferences. *Announcements are real since R5,
`custom_emojis` since R7 (2026-10-17).*
- **Mapping:** - **Mapping:**
- Avatar, ForeignAvatar and Group all map to Account (`group: true` for groups). `acct` uses a WebFinger-verified - Avatar, ForeignAvatar and Group all map to Account (`group: true` for groups). `acct` uses a WebFinger-verified
handle; `created_at` is truncated to the day; avatar and header always have a placeholder URL. handle; `created_at` is truncated to the day; avatar and header always have a placeholder URL.
+139
View File
@@ -0,0 +1,139 @@
# Custom emoji (completion program R7). The administrator adds one of the server's own emoji through
# /clientapi/admin/emoji, which the Mastodon API lists and /emojis/{shortcode} serves as an ActivityPub Emoji (signed or
# not); a public post using it reaches Mastodon, GoToSocial, Misskey and Akkoma with the emoji (their APIs list it among
# the status's emojis); a display name using it shows it on Mastodon; an EmojiReact with it reaches Akkoma and Misskey
# as that emoji; deleting it takes it out of the list and answers 410 at its id, and a post written after names it no
# more. The staff's log keeps the insert and the delete. Needs the gts, mastodon, misskey and akkoma peers (batch A);
# run it after moderation, as every scenario that edits the server as its administrator.
G=https://gts.test:6443
gcurl() { curl -sk --resolve gts.test:6443:127.0.0.1 "$@"; }
M=https://mastodon.test:6443
mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; }
AK=https://akkoma.test:6443
acurl() { curl -sk --resolve akkoma.test:6443:127.0.0.1 "$@"; }
MK=https://misskey.test:6443
MKT=$(cat "$here/.state/misskey.token" 2>/dev/null)
KT=$(cat "$here/.state/akkoma.token" 2>/dev/null); KH="Authorization: Bearer $KT"
. "$here/peers/mastodon.sh"
# mk <endpoint> [json]: a Misskey API call as mkuser
mk() {
local body=${2:-'{}'}
site misskey.test -s -X POST "$MK/api/$1" -H 'Content-Type: application/json' \
-d "$(python3 -c "import sys,json; d=json.loads(sys.argv[1]); d['i']=sys.argv[2]; print(json.dumps(d))" "$body" "$MKT")"
}
mk_note_by_uri() { mk users/notes "{\"userId\":\"$1\",\"limit\":50,\"withReplies\":true}" | j "print(json.dumps(next((n for n in d if n.get('uri')=='$2'), None)))"; }
login() { curl -s -X POST "$P/clientapi/user/login" -H 'Content-Type: application/json' -d "{\"userName\":\"$1\",\"password\":\"$2\"}" | j "print(d['token'])"; }
code_of() { local jwt=$1 method=$2 path=$3; shift 3; curl -s -o /dev/null -w '%{http_code}' -X "$method" -H "Authorization: Bearer $jwt" "$P$path" "$@"; }
gts_token() {
local app id secret code jar
jar=$(mktemp); app=$(gcurl -X POST $G/api/v1/apps -d 'client_name=pasture&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read+write+follow')
id=$(echo "$app" | j "print(d['client_id'])"); secret=$(echo "$app" | j "print(d['client_secret'])")
gcurl -o /dev/null -c "$jar" -b "$jar" "$G/oauth/authorize?client_id=$id&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=read+write+follow"
gcurl -o /dev/null -c "$jar" -b "$jar" -X POST $G/auth/sign_in --data-urlencode 'username=gtsuser@gts.test' --data-urlencode 'password=Gts-Pasture-Pass-1!'
code=$(gcurl -o /dev/null -w '%{redirect_url}' -c "$jar" -b "$jar" -X POST "$G/oauth/authorize" | sed -n 's/.*[?&]code=\([^&]*\).*/\1/p')
rm -f "$jar"
gcurl -X POST $G/oauth/token -d "grant_type=authorization_code&code=$code&client_id=$id&client_secret=$secret&redirect_uri=urn:ietf:wg:oauth:2.0:oob&scope=read+write+follow" | j "print(d['access_token'])"
}
# GoToSocial answers a lookup by URI with resolve=false from its database alone (its home timeline can't be trusted)
on_gts() { gcurl -G -H "$GH" "$G/api/v2/search" --data-urlencode "q=$1" -d resolve=false -d type=statuses; }
# names <json of emojis>: the shortcodes a list of emoji entities, or Misskey's name-to-url map, holds (hosts left out)
has_code() { j "e=d or []
names=[x.get('shortcode') or x.get('name') for x in e] if isinstance(e, list) else list(e.keys())
print(any((n or '').strip(':').split('@')[0] == '$1' for n in names))"; }
p_home_id() { curl -s -H "$AH" "$P/api/v1/timelines/home?limit=40" | j "print(next((s['id'] for s in d if '$1' in s['content']), ''))"; }
echo "emoji"
run=$(date +%s)
code="pasture_$run"
privapub_root >/dev/null
podman exec -w /app pasture-privapub /app/PrivaPub admin promote "$ROOT_USER" >/dev/null 2>&1 || true
ADMIN=$(login "$ROOT_USER" "$ROOT_PASS")
[ -n "$ADMIN" ] && ok "the pasture's root signs in as the administrator" || { ko "no administrator's login"; return 1; }
AT=$(privapub_token emoji_alice); AH="Authorization: Bearer $AT"
MT=$(mastodon_token); MH="Authorization: Bearer $MT"
GT=$(gts_token); GH="Authorization: Bearer $GT"
[ -n "$AT" ] && [ -n "$MT" ] && [ -n "$GT" ] && [ -n "$MKT" ] && [ -n "$KT" ] && ok "tokens for emoji_alice, mastouser, gtsuser, mkuser and akuser" \
|| { ko "a token is missing"; return 1; }
echo " the administrator adds it"
make_png "$work/emoji.png"
added=$(curl -s -X POST -H "Authorization: Bearer $ADMIN" "$P/clientapi/admin/emoji/insert" -F "shortcode=$code" -F 'category=Pasture' \
-F "image=@$work/emoji.png;type=image/png")
emoji_url=$(echo "$added" | j "print(d['url'])"); emoji_uri=$(echo "$added" | j "print(d['uri'])")
[ "$(echo "$added" | j "print(d['shortcode'] == '$code' and d['contentType'] == 'image/png' and d['uri'] == 'https://privapub.test/emojis/$code')")" = True ] \
&& ok "an administrator adds :$code: (a PNG, its id /emojis/$code)" || ko "the emoji was not added: $(echo "$added" | head -c 300)"
[ "$(code_of "$ADMIN" POST /clientapi/admin/emoji/insert -F "shortcode=$code" -F "image=@$work/emoji.png;type=image/png")" = 409 ] \
&& ok "the same shortcode is refused (409)" || ko "a second :$code: was taken"
[ "$(code_of "$AT" GET /clientapi/admin/emoji/list)" != 200 ] && ok "a persona's token can't list them" || ko "a persona read the administrator's list"
[ "$(curl -s "$P/api/v1/custom_emojis" | j "print(any(e['shortcode'] == '$code' and e['url'] == '$emoji_url' and e['category'] == 'Pasture' for e in d))")" = True ] \
&& ok "/api/v1/custom_emojis lists it" || ko "custom_emojis misses it: $(curl -s "$P/api/v1/custom_emojis" | head -c 300)"
document=$(pfetch -H 'Accept: application/activity+json' "https://privapub.test/emojis/$code")
[ "$(echo "$document" | j "print(d['type'] == 'Emoji' and d['id'] == '$emoji_uri' and d['name'] == ':$code:' and d['icon']['url'] == '$emoji_url' and d['icon']['mediaType'] == 'image/png' and bool(d.get('updated')))")" = True ] \
&& ok "/emojis/$code serves the Emoji, unsigned too" || ko "the Emoji document: $(echo "$document" | head -c 300)"
[ "$(pfetch -o /dev/null -w '%{http_code}' "$emoji_url")" = 200 ] && ok "its picture is served" || ko "its picture is not served"
echo " follows"
alice_on_m=$(mcurl -H "$MH" "$M/api/v2/search?q=@emoji_alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$alice_on_m/follow"
alice_on_g=$(gcurl -H "$GH" "$G/api/v2/search?q=@emoji_alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
gcurl -o /dev/null -X POST -H "$GH" "$G/api/v1/accounts/$alice_on_g/follow"
alice_on_mk=$(mk users/show '{"username":"emoji_alice","host":"privapub.test"}' | j "print(d['id'])")
mk following/create "{\"userId\":\"$alice_on_mk\"}" >/dev/null
alice_on_a=$(acurl -H "$KH" "$AK/api/v2/search?q=@emoji_alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
acurl -o /dev/null -X POST -H "$KH" "$AK/api/v1/accounts/$alice_on_a/follow"
alice_id=$(curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])")
until_true 30 '[ "$(curl -s "$P/api/v1/accounts/$alice_id" | j "print(d[\"followers_count\"])")" -ge 4 ]' \
&& ok "mastouser, gtsuser, mkuser and akuser follow emoji_alice" || ko "not every peer follows emoji_alice"
mk_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=mkuser@misskey.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
ak_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=akuser@akkoma.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$mk_on_p/follow"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$ak_on_p/follow"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/relationships?id[]=$mk_on_p&id[]=$ak_on_p" | j "print(all(r[\"following\"] for r in d))")" = True ]' \
&& ok "emoji_alice follows mkuser and akuser" || ko "emoji_alice's follows were not accepted"
echo " a post using it"
post=$(curl -s -X POST -H "$AH" "$P/api/v1/statuses" --data-urlencode "status=Hello :$code: from PrivaPub $run" -d visibility=public)
post_id=$(echo "$post" | j "print(d['id'])"); post_uri=$(echo "$post" | j "print(d['uri'])")
[ "$(echo "$post" | has_code "$code")" = True ] && ok "the status lists :$code: among its emojis here" || ko "the status has no emoji here: $(echo "$post" | head -c 300)"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/$alice_on_m/statuses?limit=20" | j "s=next((s for s in d if s[\"uri\"]==\"$post_uri\"), None); print(json.dumps(s[\"emojis\"]) if s else \"null\")" | has_code "$code")" = True ]' \
&& ok "Mastodon shows the post with :$code:" || ko "Mastodon's copy has no emoji: $(mcurl -H "$MH" "$M/api/v1/accounts/$alice_on_m/statuses?limit=5" | head -c 300)"
until_true 30 '[ "$(on_gts "$post_uri" | j "print(json.dumps(d[\"statuses\"][0][\"emojis\"]) if d[\"statuses\"] else \"null\")" | has_code "$code")" = True ]' \
&& ok "GoToSocial shows the post with :$code:" || ko "GoToSocial's copy has no emoji: $(on_gts "$post_uri" | head -c 300)"
until_true 30 '[ "$(mk_note_by_uri "$alice_on_mk" "$post_uri" | j "print(json.dumps((d or {}).get(\"emojis\")))" | has_code "$code")" = True ]' \
&& ok "Misskey shows the note with :$code:" || ko "Misskey's copy has no emoji: $(mk_note_by_uri "$alice_on_mk" "$post_uri" | head -c 300)"
until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/accounts/$alice_on_a/statuses?limit=20" | j "s=next((s for s in d if s[\"uri\"]==\"$post_uri\"), None); print(json.dumps(s[\"emojis\"]) if s else \"null\")" | has_code "$code")" = True ]' \
&& ok "Akkoma shows the post with :$code:" || ko "Akkoma's copy has no emoji"
echo " a display name using it"
curl -s -o /dev/null -X PATCH -H "$AH" "$P/api/v1/accounts/update_credentials" --data-urlencode "display_name=Emoji Alice :$code:"
[ "$(curl -s "$P/api/v1/accounts/$alice_id" | j "print(json.dumps(d['emojis']))" | has_code "$code")" = True ] \
&& ok "emoji_alice's account lists :$code: here" || ko "the account here has no emoji"
until_true 45 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/$alice_on_m" | j "print(json.dumps(d[\"emojis\"]) if \"Emoji Alice\" in d[\"display_name\"] else \"null\")" | has_code "$code")" = True ]' \
&& ok "Mastodon shows the display name with :$code:" || ko "Mastodon's account: $(mcurl -H "$MH" "$M/api/v1/accounts/$alice_on_m" | j "print(d['display_name'], d['emojis'])")"
echo " reactions with it"
mk_post=$(mk notes/create "{\"text\":\"react to this, PrivaPub $run\",\"visibility\":\"public\"}" | j "print(d['createdNote']['id'])")
a_post=$(acurl -X POST -H "$KH" "$AK/api/v1/statuses" --data-urlencode "status=react to this too, PrivaPub $run" -d visibility=public | j "print(d['id'])")
until_true 30 '[ -n "$(p_home_id "react to this, PrivaPub $run")" ] && [ -n "$(p_home_id "react to this too, PrivaPub $run")" ]' \
&& ok "mkuser's and akuser's posts reach emoji_alice's home" || ko "the posts to react to never arrived"
mk_on_home=$(p_home_id "react to this, PrivaPub $run"); a_on_home=$(p_home_id "react to this too, PrivaPub $run")
[ "$(curl -s -o /dev/null -w '%{http_code}' -X PUT -H "$AH" "$P/api/v1/pleroma/statuses/$mk_on_home/reactions/%3A$code%3A")" = 200 ] \
&& [ "$(curl -s -o /dev/null -w '%{http_code}' -X PUT -H "$AH" "$P/api/v1/pleroma/statuses/$a_on_home/reactions/%3A$code%3A")" = 200 ] \
&& ok "emoji_alice reacts with :$code: to both" || ko "the reaction with the local emoji was refused"
until_true 30 '[ "$(mk notes/show "{\"noteId\":\"$mk_post\"}" | j "print(json.dumps(list((d.get(\"reactions\") or {}).keys())))" | has_code "$code")" = True ]' \
&& ok "the EmojiReact reaches Misskey as :$code:" || ko "Misskey's reactions: $(mk notes/show "{\"noteId\":\"$mk_post\"}" | j "print(d.get('reactions'), d.get('reactionEmojis'))")"
until_true 30 '[ "$(acurl -H "$KH" "$AK/api/v1/statuses/$a_post" | j "print(json.dumps((d.get(\"pleroma\") or {}).get(\"emoji_reactions\") or d.get(\"emoji_reactions\") or []))" | has_code "$code")" = True ]' \
&& ok "the EmojiReact reaches Akkoma as :$code:" || ko "Akkoma's reactions: $(acurl -H "$KH" "$AK/api/v1/statuses/$a_post" | j "print((d.get('pleroma') or {}).get('emoji_reactions'))")"
echo " deleted"
[ "$(code_of "$ADMIN" POST "/clientapi/admin/emoji/delete?shortcode=$code")" = 200 ] && ok "the administrator deletes it" || ko "the emoji was not deleted"
[ "$(curl -s "$P/api/v1/custom_emojis" | j "print(any(e['shortcode'] == '$code' for e in d))")" = False ] && ok "custom_emojis no longer lists it" || ko "custom_emojis still lists it"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "https://privapub.test/emojis/$code")" = 410 ] \
&& ok "/emojis/$code answers 410" || ko "/emojis/$code answers $(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "https://privapub.test/emojis/$code")"
[ "$(curl -s -X POST -H "$AH" "$P/api/v1/statuses" --data-urlencode "status=No more :$code: $run" -d visibility=public | has_code "$code")" = False ] \
&& [ "$(curl -s "$P/api/v1/statuses/$post_id" | j "print(json.dumps(d['emojis']))" | has_code "$code")" = False ] \
&& ok "posts here name it no more" || ko "a post still carries the deleted emoji"
log=$(curl -s -H "Authorization: Bearer $ADMIN" "$P/clientapi/moderator/log?limit=100")
[ "$(echo "$log" | j "print(all(any(i['action'] == a and i['targetKind'] == 'emoji' and i.get('targetLabel') == ':$code:' for i in d['items']) for a in ('emoji.insert', 'emoji.delete')))")" = True ] \
&& ok "the staff's log keeps the insert and the delete" || ko "the staff's log misses the emoji's acts"
curl -s -o /dev/null -X PATCH -H "$AH" "$P/api/v1/accounts/update_credentials" --data-urlencode 'display_name=emoji_alice'
+2 -2
View File
@@ -9,7 +9,7 @@
# A gts mastodon misskey sharkey akkoma, with relay, aoderelay and mailpit; scenarios: those five, threads followsync # A gts mastodon misskey sharkey akkoma, with relay, aoderelay and mailpit; scenarios: those five, threads followsync
# pins moves relay (relay's covers both relays) accounts (mailpit's) instance (the server's own settings: it changes # pins moves relay (relay's covers both relays) accounts (mailpit's) instance (the server's own settings: it changes
# them all and puts them back) moderation (gts and mastodon; after instance, whose log check expects only the # them all and puts them back) moderation (gts and mastodon; after instance, whose log check expects only the
# administrator's acts) # administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation)
# B lemmy lemmy19 piefed mbin nodebb # B lemmy lemmy19 piefed mbin nodebb
# C peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale # C peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale
# D friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla # D friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla
@@ -29,7 +29,7 @@ batch_peers() {
} }
batch_scenarios() { batch_scenarios() {
case "$1" in case "$1" in
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation" ;; A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji" ;;
*) batch_peers "$1" ;; *) batch_peers "$1" ;;
esac esac
} }