thepra/SocialPub
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

R9 Web Push, then the honest version: VAPID key, push subscriptions, encrypted pushes, 4.5.0 advertised

Browse Source 
The server's VAPID key (PushKeys, made on first need, in Mongo) is the instance API's vapid.public_key and every app's
vapid_key. /api/v1/push/subscription (POST, GET, PUT, DELETE) keeps one subscription per access token, so per persona;
every notification kept (never a filtered one) is queued as a SendPush job for the subscriptions whose alert and policy
admit it, and sent through the `push` client (the safe handler, https only) encrypted as RFC 8291 aes128gcm or the
aesgcm draft with an ES256 VAPID JWT. 404/410 delete the subscription, 429/5xx/timeouts retry, other answers drop the
push. Subscriptions end with their tokens (revocation, sessions ended, suspension, deletion, a restore); backups hold
neither keys nor subscriptions. The walk of every Mastodon 4.3-4.5 client route (docs/MASTODON-API.md) added
conversations/:id/unread, domain_blocks/preview, v2 notifications clear and PUT policy, /oauth/userinfo and stubs, so both
instance APIs now say 4.5.0 (compatible; PrivaPub <release>). The pasture gets a push sink (pushsink.test) and
scenarios/push.sh in batch A.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WpVCoDeM1qUTRp4fXQdfof
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-18 09:18:26 +02:00
1 parent 87529bca49
commit 9b792042e6
43 files changed
+2079 -49
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tools/pasture/Caddyfile
+5
View File
@@ -207,6 +207,11 @@ lemmy19.test {
reverse_proxy pasture-lemmy19:8536
}
pushsink.test {
tls internal
reverse_proxy pasture-pushsink:8080
}
flood1.test, flood2.test, flood3.test, flood4.test, flood5.test, flood6.test, flood7.test, flood8.test, flood9.test, flood10.test, flood11.test, flood12.test, flood13.test, flood14.test, flood15.test, flood16.test, flood17.test, flood18.test, flood19.test, flood20.test {
tls internal
reverse_proxy pasture-flood:8080
tools/pasture/lib/interop.sh
+4 -4
View File
@@ -42,14 +42,14 @@ privapub_root() {
echo "$root" | j "print(d['token'])"
}
# privapub_token <persona>: creates the persona under the pasture root if needed and returns a Mastodon token for it,
# through the real OAuth flow (tools/smoke/oauth.sh).
# privapub_token <persona> [scopes]: creates the persona under the pasture root if needed and returns a Mastodon token for
# it, through the real OAuth flow (tools/smoke/oauth.sh); "read write follow" unless other scopes are asked for
privapub_token() {
local persona=$1 jwt
local persona=$1 scopes=${2:-read write follow} jwt
jwt=$(privapub_root)
curl -s -o /dev/null -X POST $P/clientapi/avatar/private/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"userName\":\"$persona\",\"name\":\"$persona of PrivaPub\",\"biography\":\"testing federation\"}"
"$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" "$persona" "read write follow" | cut -d' ' -f1
"$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" "$persona" "$scopes" | cut -d' ' -f1
}
# p_report <authorization header> <account id> <reason> <status id...>: a persona reports an account's posts and asks for
tools/pasture/peers/pushsink.sh
+22
View File
@@ -0,0 +1,22 @@
# pushsink: a Web Push service as pushsink.test, for scenarios/push.sh (tools/pasture/pushsink/pushsink.cs, a .NET
# file-based app published once into .pushsink and run on the runtime image PrivaPub uses). It makes each subscription's
# keys itself, takes PrivaPub's pushes as a push service would (the VAPID JWT checked against the instance API's key, the
# aes128gcm or aesgcm body decrypted) and keeps what it took for the scenario to read: https://pushsink.test:6443/
# subscriptions/<id>/received. It can be told to answer the next push otherwise (/subscriptions/<id>/answer, a 410).
# PrivaPub reaches it through Caddy as it reaches every peer: its push client honours appsettings.Pasture.json's
# Federation exceptions (AcceptAnyCertificate for Caddy's CA, AllowPrivateNetworks), which production refuses at start.
pushsink_up() {
local dotnet=${DOTNET:-$(command -v dotnet || echo ~/.dotnet/dotnet)}
if [ ! -x "$here/.pushsink/pushsink" ] || [ "$here/pushsink/pushsink.cs" -nt "$here/.pushsink/pushsink" ]; then
"$dotnet" publish "$here/pushsink/pushsink.cs" -r linux-x64 --self-contained true -o "$here/.pushsink" -v quiet
fi
podman run -d --replace --name pasture-pushsink --network $net --label pasture=1 -w /app -v "$here/.pushsink:/app:Z,ro" \
-v "$ca/bundle.pem:/etc/ssl/certs/ca-certificates.crt:z,ro" -e SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
-e PUSHSINK_BASE=https://pushsink.test -e PUSHSINK_INSTANCE=https://privapub.test \
mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/pushsink >/dev/null
for _ in $(seq 1 60); do
site pushsink.test -s -o /dev/null -w '%{http_code}' https://pushsink.test:6443/health 2>/dev/null | grep -q 200 && break
sleep 1
done
echo "pushsink: https://pushsink.test:6443"
}
tools/pasture/pushsink/pushsink.cs
+465
View File
@@ -0,0 +1,465 @@
#:sdk Microsoft.NET.Sdk.Web
#:property PublishAot=false
#:property InvariantGlobalization=true
#:property Nullable=disable
// pushsink: a Web Push service for the pasture (tools/pasture/scenarios/push.sh), behind Caddy as pushsink.test, which
// takes PrivaPub's pushes as a browser's push service would and keeps what it took for the scenario to read. It makes the
// keys of each subscription itself, so it can read what PrivaPub encrypted to them, and it checks what a push service
// checks: the VAPID JWT (ES256 by the server's key, the push service's origin as aud, exp in the future and at most a day
// ahead, a mailto: or https: sub) and the encryption (RFC 8291 aes128gcm, or the aesgcm draft with Encryption and
// Crypto-Key). Its own code, not PrivaPub's, so a round trip proves the two agree.
// POST /subscriptions {"server_key": "<the instance's vapid.public_key>"} (optional; without it the
// key is read from $PUSHSINK_INSTANCE/api/v2/instance): a new subscription, its
// keys made here: {"id", "endpoint", "keys": {"p256dh", "auth"}}
// GET /subscriptions/{id} the subscription, with what it answers next
// POST /subscriptions/{id}/answer {"status": 410, "times": 1}: the next push(es) are answered so, after being read
// GET /subscriptions/{id}/received every push it took or refused, oldest first: when, the status it answered,
// whether every check passed (ok) and the errors, the headers, the JWT's claims,
// the decrypted payload (parsed JSON)
// DELETE /subscriptions/{id}/received forgets them
// POST /push/{id} the push service: 201 when every check passes, 403 for the VAPID's failures, 400
// for the rest, 404 for no such subscription, or the status asked for
// GET /health
// Everything is in memory: a new container starts empty.
using System.Buffers.Binary;
using System.Collections.Concurrent;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
var publicBase = (Environment.GetEnvironmentVariable("PUSHSINK_BASE") ?? "https://pushsink.test").TrimEnd('/');
var instance = (Environment.GetEnvironmentVariable("PUSHSINK_INSTANCE") ?? "https://privapub.test").TrimEnd('/');
var listen = Environment.GetEnvironmentVariable("PUSHSINK_LISTEN") ?? "http://0.0.0.0:8080";
var subscriptions = new ConcurrentDictionary<string, Subscription>();
var http = new HttpClient { Timeout = TimeSpan.FromSeconds(10) };
var builder = WebApplication.CreateBuilder();
builder.Logging.SetMinimumLevel(LogLevel.Warning);
builder.WebHost.UseUrls(listen);
builder.WebHost.ConfigureKestrel(k => k.Limits.MaxRequestBodySize = 64 * 1024);
var app = builder.Build();
app.MapGet("/health", () => Results.Json(new { ok = true, subscriptions = subscriptions.Count }));
app.MapPost("/subscriptions", async (HttpContext context) =>
{
var request = context.Request.ContentLength > 0 ? await JsonNode.ParseAsync(context.Request.Body) : null;
var key = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
var subscription = new Subscription(Convert.ToHexStringLower(RandomNumberGenerator.GetBytes(12)), key, RandomNumberGenerator.GetBytes(16),
request?["server_key"]?.GetValue<string>());
subscriptions[subscription.Id] = subscription;
return Results.Json(subscription.View(publicBase));
});
app.MapGet("/subscriptions/{id}", (string id) =>
subscriptions.TryGetValue(id, out var subscription) ? Results.Json(subscription.View(publicBase)) : Results.NotFound());
app.MapPost("/subscriptions/{id}/answer", async (HttpContext context, string id) =>
{
if (!subscriptions.TryGetValue(id, out var subscription))
return Results.NotFound();
var request = await JsonNode.ParseAsync(context.Request.Body);
var status = request?["status"]?.GetValue<int>() ?? 0;
if (status is < 200 or > 599)
return Results.BadRequest(new { error = "status: 200 to 599" });
subscription.Answer(status, request?["times"]?.GetValue<int>() ?? 1);
return Results.Json(subscription.View(publicBase));
});
app.MapGet("/subscriptions/{id}/received", (string id) =>
subscriptions.TryGetValue(id, out var subscription) ? Results.Text(subscription.Received().ToJsonString(), "application/json") : Results.NotFound());
app.MapDelete("/subscriptions/{id}/received", (string id) =>
{
if (!subscriptions.TryGetValue(id, out var subscription))
return Results.NotFound();
subscription.Forget();
return Results.Json(new { });
});
app.MapPost("/push/{id}", async (HttpContext context, string id) =>
{
if (!subscriptions.TryGetValue(id, out var subscription))
return Results.NotFound();
using var buffer = new MemoryStream();
await context.Request.Body.CopyToAsync(buffer);
var check = new Check(context.Request, buffer.ToArray(), subscription, publicBase);
var serverKey = subscription.ServerKey ?? await InstanceKey();
check.Run(serverKey);
var forced = subscription.NextAnswer();
var status = forced ?? (check.Errors.Count == 0 ? 201 : check.VapidFailed ? 403 : 400);
subscription.Keep(check.Record(status));
if (status == 201)
context.Response.Headers.Location = $"{publicBase}/messages/{Guid.NewGuid():N}";
return Results.StatusCode(status);
});
app.Run();
return 0;
// the server's key as its instance API gives it (configuration.vapid.public_key), read once a minute at most
async Task<string> InstanceKey()
{
if (InstanceCache.Key is { } known && DateTime.UtcNow - InstanceCache.At < TimeSpan.FromMinutes(1))
return known;
try
{
var document = JsonNode.Parse(await http.GetStringAsync(instance + "/api/v2/instance"));
InstanceCache.Key = document?["configuration"]?["vapid"]?["public_key"]?.GetValue<string>();
InstanceCache.At = DateTime.UtcNow;
}
catch (Exception)
{
InstanceCache.Key = null;
}
return InstanceCache.Key;
}
static class InstanceCache
{
public static string Key;
public static DateTime At;
}
sealed class Subscription(string id, ECDiffieHellman key, byte[] auth, string serverKey)
{
readonly Lock _gate = new();
readonly List<JsonObject> _received = new();
int _answer;
int _answerTimes;
public string Id => id;
public ECDiffieHellman Key => key;
public byte[] Auth => auth;
public string ServerKey => serverKey;
public byte[] PublicPoint
{
get
{
var parameters = key.ExportParameters(false);
return [0x04, .. parameters.Q.X, .. parameters.Q.Y];
}
}
public JsonObject View(string publicBase)
{
lock (_gate)
return new JsonObject
{
["id"] = id,
["endpoint"] = $"{publicBase}/push/{id}",
["keys"] = new JsonObject { ["p256dh"] = B64.Url(PublicPoint), ["auth"] = B64.Url(auth) },
["server_key"] = serverKey,
["next_answer"] = _answerTimes > 0 ? _answer : null,
["next_answer_times"] = _answerTimes,
["received"] = _received.Count
};
}
public void Answer(int status, int times)
{
lock (_gate)
{
_answer = status;
_answerTimes = Math.Max(1, times);
}
}
public int? NextAnswer()
{
lock (_gate)
{
if (_answerTimes <= 0)
return null;
_answerTimes--;
return _answer;
}
}
public void Keep(JsonObject record)
{
lock (_gate)
_received.Add(record);
}
public JsonArray Received()
{
lock (_gate)
return new JsonArray(_received.Select(r => (JsonNode)r.DeepClone()).ToArray());
}
public void Forget()
{
lock (_gate)
_received.Clear();
}
}
// what a push service checks of one push, and what the sink keeps of it
sealed class Check(HttpRequest request, byte[] body, Subscription subscription, string publicBase)
{
public List<string> Errors { get; } = new();
public bool VapidFailed { get; private set; }
JsonObject _claims;
string _scheme;
string _k;
JsonNode _payload;
string _encoding;
public void Run(string expectedServerKey)
{
_encoding = request.Headers.ContentEncoding.ToString().Trim().ToLowerInvariant();
if (_encoding is not ("aes128gcm" or "aesgcm"))
Errors.Add($"Content-Encoding is '{_encoding}', neither aes128gcm nor aesgcm");
if (!int.TryParse(request.Headers["TTL"].ToString(), out var ttl) || ttl < 0)
Errors.Add("no TTL header (RFC 8030 5.2)");
Vapid(expectedServerKey);
try
{
var plaintext = _encoding switch
{
"aes128gcm" => Decrypt.Aes128Gcm(body, subscription.Key, subscription.PublicPoint, subscription.Auth),
"aesgcm" => Decrypt.AesGcm(body, Param(request.Headers["Encryption"].ToString(), "salt"), Param(request.Headers["Crypto-Key"].ToString(), "dh"),
subscription.Key, subscription.PublicPoint, subscription.Auth),
_ => null
};
if (plaintext != null)
_payload = JsonNode.Parse(plaintext);
}
catch (Exception ex) when (ex is CryptographicException or FormatException or JsonException or ArgumentException or IndexOutOfRangeException)
{
Errors.Add("the body could not be decrypted: " + ex.Message);
}
}
// RFC 8292 3: Authorization: vapid t=<jwt>, k=<key>; the draft's: Authorization: WebPush <jwt> and Crypto-Key's p256ecdsa
void Vapid(string expectedServerKey)
{
var authorization = request.Headers.Authorization.ToString().Trim();
var space = authorization.IndexOf(' ');
_scheme = space > 0 ? authorization[..space] : authorization;
var rest = space > 0 ? authorization[(space + 1)..].Trim() : string.Empty;
string jwt;
if (_scheme.Equals("vapid", StringComparison.OrdinalIgnoreCase))
{
jwt = Param(rest, "t");
_k = Param(rest, "k");
}
else if (_scheme.Equals("WebPush", StringComparison.OrdinalIgnoreCase))
{
jwt = rest;
_k = Param(request.Headers["Crypto-Key"].ToString(), "p256ecdsa");
}
else
{
Fail($"Authorization's scheme is '{_scheme}', neither vapid nor WebPush");
return;
}
if (_encoding == "aes128gcm" && !_scheme.Equals("vapid", StringComparison.OrdinalIgnoreCase))
Fail("an aes128gcm push carries RFC 8292's vapid authorization");
var key = B64.Decode(_k);
if (key is not { Length: 65 } || key[0] != 0x04)
{
Fail("the VAPID key (k or p256ecdsa) is no uncompressed P-256 point");
return;
}
if (string.IsNullOrEmpty(expectedServerKey))
Fail("the server's key is unknown (no server_key given, and the instance API did not say)");
else if (!B64.Decode(expectedServerKey).AsSpan().SequenceEqual(key))
Fail("the VAPID key is not the server's (the instance API's vapid.public_key)");
var parts = jwt?.Split('.') ?? [];
if (parts.Length != 3)
{
Fail("the VAPID token is no JWT");
return;
}
JsonNode header;
try
{
header = JsonNode.Parse(B64.Decode(parts[0]));
_claims = JsonNode.Parse(B64.Decode(parts[1])) as JsonObject;
}
catch (Exception)
{
Fail("the JWT's header or claims are not JSON");
return;
}
if (header?["alg"]?.GetValue<string>() != "ES256")
Fail($"the JWT's alg is '{header?["alg"]}', not ES256");
using (var verifier = ECDsa.Create(new ECParameters { Curve = ECCurve.NamedCurves.nistP256, Q = new ECPoint { X = key[1..33], Y = key[33..] } }))
{
var signature = B64.Decode(parts[2]);
if (signature is not { Length: 64 } || !verifier.VerifyData(Encoding.ASCII.GetBytes(parts[0] + "." + parts[1]), signature, HashAlgorithmName.SHA256,
DSASignatureFormat.IeeeP1363FixedFieldConcatenation))
Fail("the JWT's ES256 signature does not verify with the VAPID key");
}
var audience = new Uri($"{publicBase}/push/{subscription.Id}").GetLeftPart(UriPartial.Authority);
if (_claims?["aud"]?.GetValue<string>() != audience)
Fail($"the JWT's aud is '{_claims?["aud"]}', not the push service's origin {audience}");
var now = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
var exp = _claims?["exp"]?.GetValue<long>() ?? 0;
if (exp <= now)
Fail("the JWT has expired (exp)");
else if (exp > now + 24 * 3600 + 60)
Fail("the JWT's exp is more than 24 hours ahead (RFC 8292 2)");
var sub = _claims?["sub"]?.GetValue<string>() ?? string.Empty;
if (!sub.StartsWith("mailto:", StringComparison.Ordinal) && !sub.StartsWith("https:", StringComparison.Ordinal))
Fail($"the JWT's sub is '{sub}', neither mailto: nor https:");
}
void Fail(string error)
{
VapidFailed = true;
Errors.Add(error);
}
// a parameter of a header written a=b, c=d or a=b;c=d (Crypto-Key may hold several)
static string Param(string header, string name)
{
foreach (var part in (header ?? string.Empty).Split([',', ';'], StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
var equals = part.IndexOf('=');
if (equals > 0 && part[..equals].Trim().Equals(name, StringComparison.OrdinalIgnoreCase))
return part[(equals + 1)..].Trim().Trim('"');
}
return null;
}
public JsonObject Record(int status) => new()
{
["at"] = DateTime.UtcNow.ToString("O"),
["status"] = status,
["ok"] = Errors.Count == 0,
["errors"] = new JsonArray(Errors.Select(e => (JsonNode)e).ToArray()),
["encoding"] = _encoding,
["headers"] = new JsonObject
{
["ttl"] = request.Headers["TTL"].ToString(),
["urgency"] = request.Headers["Urgency"].ToString(),
["content_type"] = request.ContentType,
["content_encoding"] = request.Headers.ContentEncoding.ToString(),
["authorization_scheme"] = _scheme,
["encryption"] = request.Headers["Encryption"].ToString(),
["crypto_key"] = request.Headers["Crypto-Key"].ToString(),
["user_agent"] = request.Headers.UserAgent.ToString()
},
["vapid"] = new JsonObject
{
["k"] = _k,
["aud"] = _claims?["aud"]?.DeepClone(),
["exp"] = _claims?["exp"]?.DeepClone(),
["sub"] = _claims?["sub"]?.DeepClone()
},
["bytes"] = body.Length,
["payload"] = _payload?.DeepClone()
};
}
// the client's side of RFC 8291 and of the aesgcm draft
static class Decrypt
{
public static byte[] Aes128Gcm(byte[] body, ECDiffieHellman client, byte[] clientPublic, byte[] auth)
{
// RFC 8188 2.1: salt (16), record size (4), key id length (1), key id: the sender's public key
var salt = body[..16];
var recordSize = (int)BinaryPrimitives.ReadUInt32BigEndian(body.AsSpan(16, 4));
var idLength = body[20];
var serverPublic = body[21..(21 + idLength)];
if (recordSize < 18)
throw new CryptographicException($"record size {recordSize} is below 18");
var shared = Agree(client, serverPublic);
var keyInfo = Concat(Encoding.ASCII.GetBytes("WebPush: info\0"), clientPublic, serverPublic);
var ikm = HKDF.Expand(HashAlgorithmName.SHA256, HKDF.Extract(HashAlgorithmName.SHA256, shared, auth), 32, keyInfo);
var prk = HKDF.Extract(HashAlgorithmName.SHA256, ikm, salt);
var cek = HKDF.Expand(HashAlgorithmName.SHA256, prk, 16, Encoding.ASCII.GetBytes("Content-Encoding: aes128gcm\0"));
var nonce = HKDF.Expand(HashAlgorithmName.SHA256, prk, 12, Encoding.ASCII.GetBytes("Content-Encoding: nonce\0"));
var content = body[(21 + idLength)..];
var plaintext = new List<byte>();
var records = (content.Length + recordSize - 1) / recordSize;
for (var i = 0; i < records; i++)
{
var record = content[(i * recordSize)..Math.Min(content.Length, (i + 1) * recordSize)];
var recordNonce = (byte[])nonce.Clone();
for (var b = 0; b < 8; b++)//the record's sequence number, XORed into the nonce's last bytes
recordNonce[11 - b] ^= (byte)((long)i >> (8 * b));
var opened = Open(cek, recordNonce, record);
var end = opened.Length - 1;
while (end >= 0 && opened[end] == 0)
end--;
var last = i == records - 1;
if (end < 0 || opened[end] != (last ? 0x02 : 0x01))
throw new CryptographicException("a record's delimiter is wrong");
plaintext.AddRange(opened[..end]);
}
return plaintext.ToArray();
}
public static byte[] AesGcm(byte[] body, string salt, string serverKey, ECDiffieHellman client, byte[] clientPublic, byte[] auth)
{
var saltBytes = B64.Decode(salt) ?? throw new FormatException("no salt in the Encryption header");
var serverPublic = B64.Decode(serverKey) ?? throw new FormatException("no dh in the Crypto-Key header");
var shared = Agree(client, serverPublic);
var ikm = HKDF.Expand(HashAlgorithmName.SHA256, HKDF.Extract(HashAlgorithmName.SHA256, shared, auth), 32, Encoding.ASCII.GetBytes("Content-Encoding: auth\0"));
var context = Concat(Encoding.ASCII.GetBytes("P-256\0"), Length16(clientPublic), clientPublic, Length16(serverPublic), serverPublic);
var prk = HKDF.Extract(HashAlgorithmName.SHA256, ikm, saltBytes);
var cek = HKDF.Expand(HashAlgorithmName.SHA256, prk, 16, Concat(Encoding.ASCII.GetBytes("Content-Encoding: aesgcm\0"), context));
var nonce = HKDF.Expand(HashAlgorithmName.SHA256, prk, 12, Concat(Encoding.ASCII.GetBytes("Content-Encoding: nonce\0"), context));
var opened = Open(cek, nonce, body);
var padding = BinaryPrimitives.ReadUInt16BigEndian(opened);
return opened[(2 + padding)..];
}
static byte[] Agree(ECDiffieHellman client, byte[] serverPublic)
{
if (serverPublic is not { Length: 65 } || serverPublic[0] != 0x04)
throw new CryptographicException("the sender's key is no uncompressed P-256 point");
using var server = ECDiffieHellman.Create(new ECParameters
{
Curve = ECCurve.NamedCurves.nistP256,
Q = new ECPoint { X = serverPublic[1..33], Y = serverPublic[33..] }
});
return client.DeriveRawSecretAgreement(server.PublicKey);
}
static byte[] Open(byte[] key, byte[] nonce, byte[] sealedRecord)
{
if (sealedRecord.Length < 16)
throw new CryptographicException("a record shorter than its tag");
var plaintext = new byte[sealedRecord.Length - 16];
using var aes = new AesGcm(key, 16);
aes.Decrypt(nonce, sealedRecord.AsSpan(0, plaintext.Length), sealedRecord.AsSpan(plaintext.Length), plaintext);
return plaintext;
}
static byte[] Length16(byte[] value)
{
var length = new byte[2];
BinaryPrimitives.WriteUInt16BigEndian(length, (ushort)value.Length);
return length;
}
static byte[] Concat(params byte[][] parts) => parts.SelectMany(p => p).ToArray();
}
static class B64
{
public static string Url(byte[] bytes) => Convert.ToBase64String(bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_');
public static byte[] Decode(string text)
{
if (string.IsNullOrWhiteSpace(text))
return null;
var normal = text.Trim().TrimEnd('=').Replace('-', '+').Replace('_', '/');
normal = normal.PadRight(normal.Length + (4 - normal.Length % 4) % 4, '=');
try
{
return Convert.FromBase64String(normal);
}
catch (FormatException)
{
return null;
}
}
}
tools/pasture/scenarios/push.sh
+168
View File
@@ -0,0 +1,168 @@
# Web Push (completion program R9), with Mastodon and the pushsink. push_alice subscribes with keys the sink made for
# her: a mention from mastouser reaches the sink encrypted (aes128gcm), signed by the server's VAPID key for the sink's
# origin, and decrypts to Mastodon's payload with notification_type mention and her own access token. Turning the mention
# alert off stops it while a favourite still comes; policy=followed stops mastouser's mention until she follows him;
# standard=false is the aesgcm draft and decrypts too, and a new subscription replaces the old; push_bob, a sibling persona
# of the same login, never gets hers, nor she his; DELETE stops the pushes; the sink answering 410 removes the
# subscription; a revoked token takes its subscription with it. Needs the mastodon and pushsink peers (batch A); run
# after featured.
M=https://mastodon.test:6443
mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; }
PS=https://pushsink.test:6443
scurl() { curl -sk --resolve pushsink.test:6443:127.0.0.1 "$@"; }
. "$here/peers/mastodon.sh"
code() { curl -s -o /dev/null -w '%{http_code}' "$@"; }
# sink_new: a subscription of the sink's own, its keys made there, told the server's key to check the VAPID JWT with
sink_new() { scurl -X POST "$PS/subscriptions" -H 'Content-Type: application/json' -d "{\"server_key\":\"$server_key\"}"; }
sink_count() { scurl "$PS/subscriptions/$1" | j "print(d['received'])"; }
# sink_last <id> <python over d, the pushes it took, oldest first>
sink_last() { scurl "$PS/subscriptions/$1/received" | j "$2"; }
sink_waits() { local sid=$1 want=$2; until_true 30 '[ "$(sink_count "$sid")" -ge "$want" ]'; }
# subscribe <authorization header> <sink subscription json> <standard> <alert...> [policy=<policy>]: Mastodon's form
subscribe() {
local auth=$1 sink=$2 standard=$3 args=() a
shift 3
for a in "$@"; do
case $a in policy=*) args+=(-d "data[policy]=${a#policy=}") ;; *) args+=(-d "data[alerts][$a]=true") ;; esac
done
curl -s -X POST -H "$auth" "$P/api/v1/push/subscription" \
--data-urlencode "subscription[endpoint]=$(echo "$sink" | j "print(d['endpoint'])")" \
--data-urlencode "subscription[keys][p256dh]=$(echo "$sink" | j "print(d['keys']['p256dh'])")" \
--data-urlencode "subscription[keys][auth]=$(echo "$sink" | j "print(d['keys']['auth'])")" \
-d "subscription[standard]=$standard" "${args[@]}"
}
# m_mention <handle> <words>: mastouser's public post naming a persona here
m_mention() { mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" --data-urlencode "status=@$1@privapub.test $2" -d visibility=public; }
# notified <authorization header> <words>: the persona holds a mention with these words (it arrived, pushed or not)
notified() { curl -s -H "$1" "$P/api/v1/notifications?types[]=mention&limit=30" | j "print(any('$2' in (n.get('status') or {}).get('content', '') for n in d))"; }
# arrives_unpushed <authorization header> <words> <sink id> <count before>: the mention arrived and the sink got nothing
arrives_unpushed() {
local auth=$1 words=$2 sid=$3 had=$4
until_true 30 '[ "$(notified "$auth" "$words")" = True ]' || return 1
sleep 6
[ "$(sink_count "$sid")" = "$had" ]
}
echo "push"
run=$(date +%s)
AT=$(privapub_token push_alice "read write follow push"); AH="Authorization: Bearer $AT"
BT=$(privapub_token push_bob "read write follow push"); BH="Authorization: Bearer $BT"
MT=$(mastodon_token); MH="Authorization: Bearer $MT"
[ -n "$AT" ] && [ -n "$BT" ] && [ -n "$MT" ] && ok "tokens for push_alice, push_bob (her sibling, the same login) and mastouser" \
|| { ko "a token is missing"; return 1; }
echo " the server's key"
server_key=$(curl -s "$P/api/v2/instance" | j "print(d['configuration']['vapid']['public_key'])")
[ "$(python3 -c "import base64,sys; k=base64.urlsafe_b64decode(sys.argv[1]); print(len(k) == 65 and k[0] == 4)" "$server_key" 2>/dev/null)" = True ] \
&& ok "the instance API gives the VAPID key (an uncompressed P-256 point)" || ko "configuration.vapid: $server_key"
[ "$(curl -s -X POST "$P/api/v1/apps" -d client_name=push-check -d redirect_uris=urn:ietf:wg:oauth:2.0:oob -d scopes=push | j "print(d['vapid_key'])")" = "$server_key" ] \
&& ok "an app's vapid_key is the same key" || ko "the app's vapid_key differs"
[ "$(curl -s "$P/api/v1/instance" | j "print(d['version'].startswith('4.5.0 (compatible; PrivaPub'))")" = True ] \
&& [ "$(curl -s "$P/api/v2/instance" | j "print(d['version'].startswith('4.5.0 (compatible; PrivaPub') and d['api_versions']['mastodon'] == 7)")" = True ] \
&& ok "both instance APIs say 4.5.0 (compatible; PrivaPub ...)" || ko "the version: $(curl -s "$P/api/v2/instance" | j "print(d['version'])")"
echo " a mention, pushed"
m_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=mastouser@mastodon.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
# (a run before may have left the follow the policy check needs gone)
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/unfollow"
for persona in push_alice push_bob; do
mcurl -o /dev/null -H "$MH" "$M/api/v2/search?q=@$persona@privapub.test&resolve=true&type=accounts"
done
sa=$(sink_new); sa_id=$(echo "$sa" | j "print(d['id'])")
made=$(subscribe "$AH" "$sa" true mention favourite policy=all)
[ "$(echo "$made" | j "print(d['standard'] is True and d['alerts']['mention'] is True and d['alerts']['reblog'] is False and d['policy'] == 'all' and d['server_key'] == '$server_key' and d['endpoint'].startswith('https://pushsink.test/push/'))")" = True ] \
&& ok "push_alice subscribes with the sink's keys (standard)" || ko "the subscription: $(echo "$made" | head -c 400)"
[ "$(curl -s -H "$AH" "$P/api/v1/push/subscription" | j "print(d['id'])")" = "$(echo "$made" | j "print(d['id'])")" ] \
&& ok "GET answers it" || ko "GET: $(curl -s -H "$AH" "$P/api/v1/push/subscription" | head -c 300)"
[ "$(code -H "$BH" "$P/api/v1/push/subscription")" = 404 ] && ok "push_bob's token has none" || ko "push_bob's token sees a subscription"
m_mention push_alice "first push $run"
sink_waits "$sa_id" 1 \
&& [ "$(sink_last "$sa_id" "r = d[-1]; p = r['payload'] or {}; print(r['ok'] and r['status'] == 201 and r['encoding'] == 'aes128gcm' and p.get('notification_type') == 'mention' and 'first push $run' in p.get('body', ''))")" = True ] \
&& ok "mastouser's mention reaches the sink, aes128gcm, and decrypts to a mention" || ko "the push: $(scurl "$PS/subscriptions/$sa_id/received" | head -c 800)"
[ "$(sink_last "$sa_id" "print(d[-1]['payload']['access_token'] == '$AT')")" = True ] && ok "it carries push_alice's own access token" \
|| ko "the push's access_token is not hers"
[ "$(sink_last "$sa_id" "r = d[-1]; v = r['vapid']; print(v['aud'] == 'https://pushsink.test' and v['sub'].startswith(('mailto:', 'https:')) and 'push_alice' not in v['sub'] and r['headers']['authorization_scheme'] == 'vapid' and int(r['headers']['ttl']) > 0 and r['headers']['urgency'] == 'normal')")" = True ] \
&& ok "signed for the sink's origin by the server, TTL and Urgency set, no persona in the sub" || ko "the VAPID claims: $(sink_last "$sa_id" "print(d[-1]['vapid'], d[-1]['headers'])")"
pushed_id=$(sink_last "$sa_id" "print(d[-1]['payload']['notification_id'])")
[ "$(curl -s -H "$AH" "$P/api/v1/notifications/$pushed_id" | j "print(d['type'] == 'mention' and 'first push $run' in d['status']['content'])")" = True ] \
&& ok "its notification_id is her notification" || ko "notification $pushed_id: $(curl -s -H "$AH" "$P/api/v1/notifications/$pushed_id" | head -c 200)"
[ "$(sink_last "$sa_id" "p = d[-1]['payload']; print(p['icon'].startswith('https://privapub.test/') and 'mastouser' in p['title'] and p['preferred_locale'] != '')")" = True ] \
&& ok "its icon goes through PrivaPub, its title names mastouser" || ko "icon and title: $(sink_last "$sa_id" "print(d[-1]['payload'])")"
echo " alerts and policy"
changed=$(curl -s -X PUT -H "$AH" "$P/api/v1/push/subscription" -d "data[alerts][favourite]=true")
[ "$(echo "$changed" | j "print(d['alerts']['mention'] is False and d['alerts']['favourite'] is True and d['policy'] == 'all')")" = True ] \
&& ok "PUT turns the mention alert off" || ko "PUT: $(echo "$changed" | head -c 300)"
before=$(sink_count "$sa_id")
m_mention push_alice "unwanted $run"
arrives_unpushed "$AH" "unwanted $run" "$sa_id" "$before" && ok "with the alert off, the mention arrives unpushed" || ko "a mention was pushed with its alert off"
alice_post=$(curl -s -X POST -H "$AH" "$P/api/v1/statuses" --data-urlencode "status=push_alice posts $run" -d visibility=public | j "print(d['uri'])")
m_post=$(mcurl -H "$MH" "$M/api/v2/search?q=$alice_post&resolve=true&type=statuses" | j "print(d['statuses'][0]['id'])")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses/$m_post/favourite"
sink_waits "$sa_id" $((before + 1)) && [ "$(sink_last "$sa_id" "print(d[-1]['ok'] and d[-1]['payload']['notification_type'])")" = favourite ] \
&& ok "his favourite of her post is pushed" || ko "no favourite push: $(sink_last "$sa_id" "print(d[-1])")"
curl -s -o /dev/null -X PUT -H "$AH" "$P/api/v1/push/subscription" -d "data[alerts][mention]=true" -d "data[policy]=followed"
before=$(sink_count "$sa_id")
m_mention push_alice "a stranger $run"
arrives_unpushed "$AH" "a stranger $run" "$sa_id" "$before" && ok "policy=followed: mastouser, whom she does not follow, is not pushed" \
|| ko "a non-followed sender was pushed"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/follow"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/relationships?id[]=$m_on_p" | j "print(d[0][\"following\"])")" = True ]' \
&& ok "push_alice follows mastouser" || ko "her follow was not accepted"
m_mention push_alice "a friend now $run"
sink_waits "$sa_id" $((before + 1)) && [ "$(sink_last "$sa_id" "print('a friend now $run' in d[-1]['payload']['body'])")" = True ] \
&& ok "once followed, his mention is pushed" || ko "a followed sender was not pushed"
echo " the aesgcm draft"
sl=$(sink_new); sl_id=$(echo "$sl" | j "print(d['id'])")
[ "$(subscribe "$AH" "$sl" false mention | j "print(d['standard'])")" = False ] && ok "push_alice subscribes again, standard=false" || ko "the legacy subscription"
before=$(sink_count "$sa_id")
m_mention push_alice "the draft $run"
sink_waits "$sl_id" 1 \
&& [ "$(sink_last "$sl_id" "r = d[-1]; print(r['ok'] and r['encoding'] == 'aesgcm' and r['headers']['authorization_scheme'] == 'WebPush' and 'p256ecdsa=' in r['headers']['crypto_key'] and r['headers']['encryption'].startswith('salt=') and r['payload']['notification_type'] == 'mention' and r['payload']['access_token'] == '$AT')")" = True ] \
&& ok "aesgcm, with Encryption and Crypto-Key, decrypts to her mention" || ko "the aesgcm push: $(scurl "$PS/subscriptions/$sl_id/received" | head -c 800)"
sleep 3
[ "$(sink_count "$sa_id")" = "$before" ] && ok "the subscription it replaced gets nothing" || ko "the replaced subscription is still pushed"
echo " siblings"
sb=$(sink_new); sb_id=$(echo "$sb" | j "print(d['id'])")
subscribe "$BH" "$sb" true mention policy=all >/dev/null
alice_before=$(sink_count "$sl_id")
m_mention push_alice "for alice only $run"
sink_waits "$sl_id" $((alice_before + 1)); sleep 4
[ "$(sink_count "$sb_id")" = 0 ] && ok "push_bob's subscription never gets push_alice's mention" || ko "a sibling's subscription got her push"
m_mention push_bob "for bob only $run"
sink_waits "$sb_id" 1 && [ "$(sink_last "$sb_id" "p = d[-1]['payload']; print(p['access_token'] == '$BT' and 'for bob only $run' in p['body'])")" = True ] \
&& ok "push_bob's own mention, with his own token" || ko "push_bob's push: $(scurl "$PS/subscriptions/$sb_id/received" | head -c 400)"
sleep 3
[ "$(sink_count "$sl_id")" = "$((alice_before + 1))" ] && [ "$(sink_last "$sl_id" "print(all(r['payload']['access_token'] == '$AT' for r in d))")" = True ] \
&& ok "push_alice gets nothing of his, and only her token" || ko "push_alice's subscription got push_bob's"
[ "$(curl -s -X DELETE -H "$BH" "$P/api/v1/push/subscription")" = "{}" ] && ok "push_bob unsubscribes" || ko "push_bob's DELETE"
echo " DELETE, 410, revocation"
[ "$(curl -s -X DELETE -H "$AH" "$P/api/v1/push/subscription")" = "{}" ] && [ "$(code -H "$AH" "$P/api/v1/push/subscription")" = 404 ] \
&& ok "DELETE: the subscription is gone" || ko "DELETE left it"
before=$(sink_count "$sl_id")
m_mention push_alice "after delete $run"
arrives_unpushed "$AH" "after delete $run" "$sl_id" "$before" && ok "nothing is pushed after DELETE" || ko "pushed after DELETE"
sg=$(sink_new); sg_id=$(echo "$sg" | j "print(d['id'])")
subscribe "$AH" "$sg" true mention >/dev/null
scurl -o /dev/null -X POST "$PS/subscriptions/$sg_id/answer" -H 'Content-Type: application/json' -d '{"status": 410}'
m_mention push_alice "gone $run"
sink_waits "$sg_id" 1 && until_true 15 '[ "$(code -H "$AH" "$P/api/v1/push/subscription")" = 404 ]' \
&& ok "the sink answering 410 removes the subscription" || ko "after a 410: $(curl -s -H "$AH" "$P/api/v1/push/subscription" | head -c 200)"
before=$(sink_count "$sg_id")
m_mention push_alice "still gone $run"
arrives_unpushed "$AH" "still gone $run" "$sg_id" "$before" && ok "and nothing more goes there" || ko "pushed after the 410"
oauth=$("$here/../smoke/oauth.sh" "$P" "$ROOT_USER" "$ROOT_PASS" push_alice "read write follow push")
RT=$(echo "$oauth" | cut -d' ' -f1); rcid=$(echo "$oauth" | cut -d' ' -f2); rcs=$(echo "$oauth" | cut -d' ' -f3)
sr=$(sink_new); sr_id=$(echo "$sr" | j "print(d['id'])")
subscribe "Authorization: Bearer $RT" "$sr" true mention >/dev/null
curl -s -o /dev/null -X POST "$P/oauth/revoke" -d "token=$RT" -d "client_id=$rcid" -d "client_secret=$rcs"
[ "$(code -H "Authorization: Bearer $RT" "$P/api/v1/push/subscription")" = 401 ] && ok "the revoked token is refused" || ko "the revoked token still works"
m_mention push_alice "revoked $run"
arrives_unpushed "$AH" "revoked $run" "$sr_id" 0 && ok "a revoked token's subscription is pushed nothing" || ko "pushed to a revoked token's subscription"
[ "$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'db.PushSubscription.countDocuments()' 2>/dev/null | tail -1)" = 0 ] \
&& ok "no subscription is left" || ko "subscriptions left: $(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'db.PushSubscription.countDocuments()' 2>/dev/null | tail -1)"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$m_on_p/unfollow"
tools/pasture/sweep.sh
+5 -4
View File
@@ -6,11 +6,12 @@
# PrivaPub runs with SecureMode on, as in production, unless PRIVAPUB_ENV says otherwise; DOTNET and PASTURE_PORT reach
# run.sh as they are. Each batch's output is kept in out/sweep-<batch>.log.
# usage: tools/pasture/sweep.sh [A|B|C|D ...] (default: every batch, in that order)
# A gts mastodon misskey sharkey akkoma, with relay, aoderelay and mailpit; scenarios: those five, threads followsync
# A gts mastodon misskey sharkey akkoma, with relay, aoderelay, mailpit and pushsink; scenarios: those five, threads followsync
# pins moves relay (relay's covers both relays) accounts (mailpit's) instance (the server's own settings: it changes
# them all and puts them back) moderation (gts and mastodon; after instance, whose log check expects only the
# administrator's acts) emoji (gts, mastodon, misskey and akkoma; after moderation) featured (mastodon: featured
# tags, notify, the direct stream, endorsements and notes per persona; after emoji)
# tags, notify, the direct stream, endorsements and notes per persona; after emoji) push (mastodon and pushsink: Web
# Push, encrypted both ways, alerts, policies, siblings, DELETE, 410 and revocation; after featured)
# B lemmy lemmy19 piefed mbin nodebb
# C peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale
# D friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla
@@ -22,7 +23,7 @@ export PRIVAPUB_ENV=${PRIVAPUB_ENV:-Federation__SecureMode=true}
batch_peers() {
case "$1" in
A) echo "gts mastodon misskey sharkey akkoma relay aoderelay mailpit" ;;
A) echo "gts mastodon misskey sharkey akkoma relay aoderelay mailpit pushsink" ;;
B) echo "lemmy lemmy19 piefed mbin nodebb" ;;
C) echo "peertube pixelfed vernissage wordpress ghost writefreely owncast castopod funkwhale" ;;
D) echo "friendica mobilizon gancio bookwyrm smithereen mitra snac ktistec forte hubzilla" ;;
@@ -30,7 +31,7 @@ batch_peers() {
}
batch_scenarios() {
case "$1" in
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji featured" ;;
A) echo "gts mastodon misskey sharkey akkoma threads followsync pins moves relay accounts instance moderation emoji featured push" ;;
*) batch_peers "$1" ;;
esac
}