R7 custom emoji: the server's own, added by administrators, sent on posts, profiles and reactions
LocalEmoji, a new collection (unique shortcode whatever its case, never renamed: the shortcode is its id elsewhere),
managed on /clientapi/admin/emoji/list|insert|update|delete and kept in the staff's log (emoji.insert, .update,
.delete). Its picture goes through IMediaService.EmojiImage: PNG, GIF, WebP or JPEG up to 256 KB, a PNG within 256 px
with alpha kept (an animated GIF or WebP stays an animated GIF, with a still PNG), keep=none, shrunk until it fits
GoToSocial's 100 KiB, held by ServerUse "emoji:<shortcode>" so the janitor and the audit leave it. LocalEmojis keeps the
offered ones in memory, reloaded on every change: GET /api/v1/custom_emojis is real; local posts (words, title, warning,
poll options, each revision) and profiles keep the emoji they name; Notes, Questions, actors and EmojiReacts carry
Mastodon's Emoji tag with its id, served as an Emoji at /emojis/{shortcode} (410 once deleted; the path awaits the
owner's confirmation); a persona reacts with one (Pleroma's reactions, announcements). A restore keeps the live emoji
as it keeps the settings, and trashes server pictures nothing holds. tools/pasture/scenarios/emoji.sh joins batch A.
Verified: built, and run locally against a scratch database (uploads of every type, animation kept, metadata gone,
refusals, rendering, reactions, history, announcements, delete and revive, backup and restore, media audit). The
pasture scenario and the sweep were not run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WpVCoDeM1qUTRp4fXQdfof
This commit is contained in:
1 parent
9f68ca3ed0
commit
d08fbabaa5
38 files changed
+1117
-54
No files matched your search
@@ -63,8 +63,11 @@ actor-scoped route gets a name in the same spirit, agreed with the owner, and a
|
||||
| A group's members and moderators | `/peasants/{name}/flock`, `/peasants/{name}/wardens` |
|
||||
| A post's replies and thread | `/peasants/{name}/scribbles/{id}/replies`, `/peasants/{name}/scribbles/{id}/context` |
|
||||
| Token refresh | `/clientapi/user/sniff/again` |
|
||||
| A custom emoji (`Emoji`, R7) | `/emojis/{shortcode}` (`LocalEmojis.Path`, the one place it is written) |
|
||||
|
||||
All of these have federated, so they are frozen. Agreed for later: `/drool` and `/echoes` (likes, shares). `/gossip` was
|
||||
All of these have federated, so they are frozen, but `/emojis/{shortcode}`: it is the plan's name for a custom emoji's
|
||||
id (Mastodon's own path, with the shortcode for its number) and waits for the owner's confirmation before R7 deploys; it
|
||||
freezes once it has federated, and every emoji id elsewhere names it. Agreed for later: `/drool` and `/echoes` (likes, shares). `/gossip` was
|
||||
agreed for replies before they were served at `…/scribbles/{id}/replies` (2026-10-06); whether it is still wanted is
|
||||
asked in the completion program's R10.
|
||||
|
||||
@@ -85,7 +88,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
|
||||
Group, Post (posts + DMs), Social, Timeline, Data, PersonaArchive, Moderator (reports,
|
||||
accounts and posts moderated, the staff's log), Admin (logins: roles, bans, deletions,
|
||||
two-factor resets), DomainBlock, Statistics, Backup, InstanceAdmin (the server's
|
||||
settings, thumbnail, announcements, relays)
|
||||
settings, thumbnail, announcements, relays), EmojiAdmin (the server's custom emoji)
|
||||
Infrastructure/
|
||||
Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out
|
||||
Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker
|
||||
@@ -100,7 +103,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0
|
||||
RegistrationOptions.cs who may make a login (`Registrations:Mode`)
|
||||
Federation/
|
||||
Controllers/ PeasantsController (actor, outbox, followers, following, posts, inboxes),
|
||||
WellKnownController (webfinger, nodeinfo), UsersController (redirect)
|
||||
WellKnownController (webfinger, nodeinfo), UsersController (redirect),
|
||||
EmojisController (a custom emoji's `Emoji` at /emojis/{shortcode})
|
||||
Actors/ LocalActorService (LocalActor, Keys, ReservedName), RemoteActorService
|
||||
(authoritative fetch, key verification, WebFinger), ActorDocument (parser)
|
||||
Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer
|
||||
@@ -119,7 +123,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
|
||||
Relays/ Relays (the instance actor's relay subscriptions)
|
||||
Domain/
|
||||
Content/ ContentRenderer (Markdown or plain text → HTML with h-card mentions and hashtags;
|
||||
Document: the server's own texts)
|
||||
Document: the server's own texts), LocalEmojis (the server's custom emoji in memory)
|
||||
Instance/ InstanceSettings (the server's own settings in memory), MediaLimitsFromSettings
|
||||
Moderation/ ModerationLog (the staff's log, AdminAction), AccountModeration (silence, suspend, posts
|
||||
removed), Restrictions (who is held back, in memory), ModerationFlags (posts' flags again)
|
||||
@@ -139,7 +143,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0
|
||||
Entities/ Mappers/ Mastodon entities; MastodonMapper (Account, Status), AccountSearch
|
||||
Controllers/ apps, instance, accounts, statuses, timelines, notifications, search, lists, filters,
|
||||
media, polls, reactions, reports, scheduled statuses, streaming, tags, trends, cdns,
|
||||
provenance; the stubs are `StubsController` in `SearchController.cs`
|
||||
provenance, custom emojis; the stubs are `StubsController` in `SearchController.cs`
|
||||
Web/Pages/ Razor: /@{user}, /@{user}/{id} (public posts only, strict CSP, noindex);
|
||||
OAuth/: /oauth/login (root password), /oauth/authorize (choose persona, consent)
|
||||
Services/ RootUsersService, GroupUsersService, PostsService, AppConfigurationService, RootRemoval,
|
||||
@@ -346,6 +350,59 @@ client's role route never does.
|
||||
here and elsewhere, and the posts taken down, before it does; indexes on `SilencedAt` and `SuspendedAt` of both account
|
||||
collections and a partial one on `Post.ModeratedAt`. No new collection, so backups hold it all as they did.
|
||||
|
||||
## Custom emoji (completion program R7)
|
||||
|
||||
- **The server's own emoji** (`Models/Post/LocalEmoji.cs`, a collection of its own): a shortcode (`[a-zA-Z0-9_]{2,64}`,
|
||||
Mastodon's rule, `Constants.ShortcodeRegex`; unique whatever its case through `Key`, the lower-cased shortcode, while
|
||||
texts match it exactly), a category, `VisibleInPicker`, `Disabled` (kept, but offered nowhere and taken into nothing
|
||||
new), the picture (`MediaId`, `Url`, `StaticUrl`, `ContentType`), `CreatedAt`, `UpdatedAt` (any change),
|
||||
`ImageUpdatedAt` (the picture's: the `updated` that federates) and `DeletedAt`. **It is never renamed**: its shortcode
|
||||
is its id elsewhere, so an update names the emoji by it, and a new name is a new emoji. Deleted, the row stays, so its
|
||||
id answers 410; adding the same shortcode again (written the same way) brings the row back with a new picture.
|
||||
- **In memory** (`Domain/Content/LocalEmojis.cs`, static, as `Restrictions` is): the emoji offered now (neither disabled
|
||||
nor deleted), read at start, after every change and when five minutes old. Whatever renders or matches emoji reads it,
|
||||
never the database.
|
||||
- **Where they are taken:** a local post keeps the emoji its words, title, warning and poll options name when it is
|
||||
published or edited (`StatusService`, also archive imports; `Post.Emojis`, and each `PostRevision` its own), a persona
|
||||
those its name, bio and fields name when its profile is saved (`update_credentials`, `/clientapi`'s avatar insert and
|
||||
update, archive imports; `Avatar.Emojis`, `LocalActor.Emojis`). Only this server's, only those offered: `:name:` with no
|
||||
letter or digit right before or after it, as Mastodon scans, at most 64 an object. Remote shortcodes are never ours.
|
||||
- **What is rendered is what they are now** (`LocalEmojis.Of`): a Note, a Question and an actor carry an `Emoji` tag per
|
||||
kept emoji still offered, with its current picture (an actor only when it has one); the Mastodon API's `emojis` of a
|
||||
local status, poll, account and history entry come from the same list. An emoji disabled or deleted since is left out,
|
||||
a new picture shows (its `updated` moves, so servers holding the old one fetch it again). No `Update` is sent when an
|
||||
emoji changes (Mastodon sends none either).
|
||||
- **The `Emoji` sent** (`LocalEmojis.Tag`, Mastodon's shape; GoToSocial takes none without its `id`):
|
||||
`{"id": "{base}/emojis/{shortcode}", "type": "Emoji", "name": ":shortcode:", "updated": <ImageUpdatedAt, ms>,
|
||||
"icon": {"type": "Image", "mediaType": "image/png" | "image/gif", "url": <the picture>}}`, and the same with the
|
||||
`@context` at its id (`Federation/Controllers/EmojisController.cs`, `application/activity+json`, unsigned under
|
||||
SecureMode too, since an emoji tells nothing about anyone; a disabled one 404, a deleted one 410 with a Tombstone; a
|
||||
browser is sent to the picture). Statistics count its fetches as served `emoji`.
|
||||
- **Reactions:** a persona reacts with one of ours as `:shortcode:`, `:shortcode@this-host:` or the bare shortcode
|
||||
(`Reactions.React`, before the post's own emoji, so `:code:` is ours and `:code@host:` another server's); the
|
||||
`EmojiReact` carries `content` `:shortcode:` and the full tag, which Misskey and Akkoma register before they count it.
|
||||
Announcements take our emoji as reactions too (Mastodon's `name`, the shortcode, with `url` and `static_url`), and list
|
||||
those their text uses in `emojis`. A new picture moves the `EmojiURL` of the reactions and notifications made with it.
|
||||
- **The Mastodon API:** `GET /api/v1/custom_emojis` (anonymous) lists, as Mastodon does, those offered and in the picker,
|
||||
by shortcode (`shortcode`, `url`, `static_url`, `visible_in_picker`, `category`), from memory.
|
||||
- **Administrator routes** (`EmojiAdminController`, `[Authorize(Policy = Policies.IsAdmin)]` and checked again against
|
||||
the database; every act in the staff's log as `emoji.insert`, `emoji.update` (its note names what changed) or
|
||||
`emoji.delete`, target kind `emoji`, labelled `:shortcode:`):
|
||||
- `GET /clientapi/admin/emoji/list`: `List<ViewLocalEmoji>`, disabled ones too, deleted ones never;
|
||||
- `POST /clientapi/admin/emoji/insert`: multipart, `LocalEmojiForm` (`shortcode`, `category`, `visibleInPicker`,
|
||||
`disabled`) and the picture as `image` (or `file`): 200 `ViewLocalEmoji`, 400 for the form, 409 for a shortcode taken
|
||||
(whatever its case), 422 for the picture;
|
||||
- `POST /clientapi/admin/emoji/update`: the same form, the emoji as it should be (its category none when empty), a new
|
||||
picture optional: 200, 400, 404 for no such emoji, 422;
|
||||
- `POST /clientapi/admin/emoji/delete?shortcode=`: 200, 404.
|
||||
- **Its picture** (`IMediaService.EmojiImage`, see Media invariants): held by `ServerUse = "emoji:<shortcode>"` like the
|
||||
thumbnail (the janitor and the quota leave it, `admin media audit` keeps the pictures of the emoji not deleted, disabled
|
||||
ones too); a replaced picture and a deleted emoji's are trashed.
|
||||
- **Backups and restores:** `LocalEmoji` is in every backup. A restore keeps the live emoji as it keeps the settings
|
||||
(`ProtectiveMerge`: P's rows): one deleted since the backup stays deleted (its id answers 410, its picture stays in the
|
||||
trash), one added or changed since stays as it is; then a picture of the server's that neither the settings nor an
|
||||
emoji holds goes to the trash.
|
||||
|
||||
## Federation invariants
|
||||
|
||||
1. **Every outbound request goes through `IFederationHttp`.** Its handler resolves the name itself and connects only
|
||||
@@ -498,6 +555,13 @@ client's role route never does.
|
||||
sRGB (`thumbnail`).
|
||||
- **GIFs:** an animated GIF becomes a looping silent H.264 mp4 typed `gifv`, as on Mastodon (ffmpeg with libx264;
|
||||
`PostMedia.Kind` says so), and a still GIF is an image.
|
||||
- **A custom emoji's picture** (R7, `EmojiImage`): PNG, GIF, WebP or JPEG up to 256 KB, fitted within 256 px with its
|
||||
transparency kept, written by libvips with `keep=none`: a still picture as a PNG, an animated GIF or WebP as an
|
||||
animated GIF (libvips' `thumbnail` with every page, then `gifsave`: a PNG would lose the animation, and GIF is the
|
||||
animated picture every peer takes as an emoji) with a still PNG of its first frame for `static_url`. It must end at
|
||||
most 100 KiB, GoToSocial's default for another server's emoji (the strictest peer; Mastodon's is 256 KB): a still
|
||||
picture is tried as it is, then with a palette, then at 192, 128, 96 and 64 px; an animation at those sides; one that
|
||||
never fits is refused (422).
|
||||
- **Types:** HEIC and HEIF are not accepted, since the bundled libvips has no HEVC decoder.
|
||||
- **Load:** processing runs `Media:Concurrency` at a time, and uploads have their own rate limit (`uploads`, per
|
||||
credential).
|
||||
@@ -526,8 +590,8 @@ client's role route never does.
|
||||
What a login's media take (every persona's, trashed ones no longer) is `Counted.MediaOfRoot`, shown to the login at
|
||||
`/clientapi/user/storage`; `Media:QuotaBytesPerRoot` (0, none, by default, or the administrator's override in the
|
||||
server's settings) refuses an upload over it with 422; the administrator's statistics overview has the totals, the
|
||||
proxy cache and the trash. The server's own pictures (`ServerUse`, its thumbnail) belong to nobody and are held while
|
||||
the settings name them.
|
||||
proxy cache and the trash. The server's own pictures (`ServerUse`: its thumbnail, `emoji:<shortcode>` for a custom
|
||||
emoji's) belong to nobody and are held while the settings name them, or while their emoji is not deleted.
|
||||
|
||||
Nothing is deleted because it looks unused. `PrivaPub admin media audit [--fix]` compares disk and database: with
|
||||
`--fix` (as www-data) it gives pictures shown from before their rows a row, and trashes media of deleted posts or
|
||||
@@ -742,9 +806,10 @@ client's role route never does.
|
||||
the staff took down, and a root's password, e-mail, ban and policies stay, and every post's moderation flags are
|
||||
worked out again (`ModerationFlags`);
|
||||
roots, personas and groups made since become tombstones (deleted, names kept), local posts made since answer 410,
|
||||
media made or trashed since go to the trash (but the server's own pictures, `ServerUse`); the staff's log
|
||||
(`AdminAction`) is the union, so a restore never loses an entry, its own request included, and the server's settings
|
||||
are P's: the live settings outlive a restore;
|
||||
media made or trashed since go to the trash (but the server's own pictures, `ServerUse`, though one of those that
|
||||
nothing holds any more goes there too); the staff's log (`AdminAction`) is the union, so a restore never loses an
|
||||
entry, its own request included, and the server's settings and custom emoji (`LocalEmoji`, R7) are P's: the live
|
||||
settings and emoji outlive a restore, and an emoji deleted since stays deleted;
|
||||
5. every session ends (a new `SessionStamp` for each root, persona tokens deleted), circuits close, and a
|
||||
`RestoreRecord` (never in a backup) tells what happened (`admin restore --status`).
|
||||
|
||||
@@ -990,6 +1055,12 @@ tools/pasture/run.sh down # removes e
|
||||
directly and through a report gone on Mastodon, the author warned, a GoToSocial copy taken down here; the staff's log
|
||||
listing every act, the report's action in its note, no login entry naming a persona, none shown to the moderator, and
|
||||
no address.
|
||||
- **Custom emoji (`scenarios/emoji.sh`, needs gts, mastodon, misskey and akkoma; after moderation in batch A):** the
|
||||
administrator adds `:pasture_<run>:` (409 for it again, a persona refused the list); `/api/v1/custom_emojis` lists it
|
||||
and `/emojis/{shortcode}` serves the `Emoji` unsigned; the four peers follow emoji_alice, whose post using it shows it
|
||||
in the status's `emojis` on Mastodon, GoToSocial (by URI), Misskey (the note's `emojis`) and Akkoma; her display name
|
||||
shows it on Mastodon; her `EmojiReact` with it reaches Misskey and Akkoma as that emoji; deleted, it leaves the list,
|
||||
its id answers 410 and posts here carry it no more; the staff's log keeps the insert and the delete.
|
||||
- **Mailpit (1.31.4, `peers/mailpit.sh`, as `mailpit.test`):** the pasture's mail server. `appsettings.Pasture.json`
|
||||
sends PrivaPub's mail to `pasture-mailpit:1025` (plain SMTP, any login accepted) and its recovery links to
|
||||
`https://decepub.test`. `scenarios/accounts.sh` reads what arrived through Mailpit's API: an address set with the
|
||||
|
||||
Reference in new issue
Block a user