I added a migration that creates an ugly fake OutboxItem for messages that
don't have one. I'll try to turn it into a real one. And then very possibly
remove the whole ugly migration, replacing it with addFielfRefRequiredEmpty,
which should work for empty instances.
- Allow client to specify recipients that don't need to be delivered to
- When fetching recipient, recognize collections and don't try to deliver to
them
- Remember collections in DB, and use that to skip HTTP delivery
It runs checks against all the relevant tables, but ultimately just inserts the
activity into the recipient's inbox and nothing more, leaving the RemoteMessage
creation and inbox forwarding to the project inbox handler.
Inbox post is disabled but in the next patches I'll code and integrate a fixed
complete one, hopefully finally getting ticket comment federation ready for
testing.
I'm making this change because if an actor receives an activity due to being
addressed in bto, ot bcc, or being listed in some remote collection, the server
doesn't have a way to tell which actor(s) are the intended recipients, without
having an individual inbox URL for each actor. I could use a different hack for
this, but it wouldn't be compatible with other AP servers (unless the whole
fediverse agrees on a method).
I wasn't using sharedInbox anyway, and it's an optimization either way.
I wrote a function handleOutboxNote that's supposed to do the whole outbox POST
handler process. There's an outbox item table in the DB now, I adapted things
in various source files. Ticket comment federation work is still in progress.
The custom module provides a parametric wrapper, allowing any specific
FromJSON/ToJSON instance to be used. It's a standalone module though, and not a
wrapper of persistent-postgresql, because persistent-postgresql uses aeson
Value and it prevents using toEncoding to get from the value directly to a
string.
Before, things worked like this:
* Only signatures of Ed25519 keys could be verified
* Key encoding placed the plain binary Ed25519 key in the PEM, instead of the
key's ASN1 encoding
With this patch it now works like this:
* Ed25519 signatures are supported as before
* RSA keys are now supported too, assuming RSA-SHA256 signatures
* Both Ed25519 and RSA keys are encoded and decoded using actual PEM with ASN1
When we verify an HTTP signature,
* If we know the key, check in the DB whether we know the actor lists it. If it
doesn't, and there's room left for keys, HTTP GET the actor and update the DB
accordingly.
* If we know the key but had to update it, do the same, check usage in DB and
update DB if needed
* If we don't know the key, record usage in DB
However,
* If we're GETing a key and discovering it's a shared key, we GET the actor to
verify it lists the key. When we don't know the key at all yet, that's fine
(can be further optimized but it's marginal), but if it's a key we do know,
it means we already know the actor and for now it's enough for us to rely
only on the DB to test usage.
When a local user wants to publish an activity, we were always GETing the
recipient actor, so that we could determine their inbox and POST the activity
to it. But now, instead, whenever we GET an actor (whether it's for the key sig
verification or for determining inbox URI), we keep their inbox URI in the
database, and we don't need to GET it again next time.
I'm not sure what the best balance is, but once an hour may end up causing a
lot more key re-fetch requests coming from other servers. I prefer to default
to once a day for now (maybe even once a week) and tighten it later if needed.
Caveat: If an instance key is rotated once a day, there's no
change-key-right-after-toot-deletion thing for deniability. Potentially,
rotation may happen only 24 hours after that deletion, which is much more than
1 hour. On the other hand, it's a whole instance key, not personal key of the
actor.
Using a dedicated type allows to record in the type the guarantees that we
provide, such as scheme being HTTPS and authority being present. Allows to
replace ugly `fromJust` and such with direct field access.
Before, there was a single key used as a personal key for all actors. Now,
things work like this:
- There are 2 keys, each time one is rotated, this way the old key remains
valid and we can freely rotate without a risk of race conditions on other
servers and end up with our posts being rejected
- The keys are explicitly instance-scope keys, all actors refer to them
- We add the ActivityPub-Actor header to all activity POSTs we send, to declare
for which specific actor our signature applies. Activities and otherwise
different payloads may have varying ways to specify attribution; using this
header will be a standard uniform way to specify the actor, regardless of
payload format. Of course, servers should make sure the actual activity is
attributed to the same actor we specified in the header. (This is important
with instance-scope keys; for personal keys it's not critical)
Allow keys to specify expiration time using w3c security vocabulary. If a key
has expired, we treat it like sig validation failure and re-fetch the key from
the other server. And we never accept a sig, even a valid sig, if the key has
expired.
Since servers keep actors and keys in the DB, expiration can be a nice way to
ask that keys aren't used more than we want them to. The security vocab spec
also recommends to set expiration time on keys, so it's nice to support this
feature.
It's now possible for activities we be attributed to actors that have more than
one key. We allow up to 2 keys. We also store in the DB. Scaling to support any
number of keys is trivial, but I'm limiting to 2 to avoid potential trouble and
because 2 is the actual number we need.
By having 2 keys, and replacing only one of them in each rotation, we avoid
race conditions. With 1 key, the following can happen:
1. We send an activity to another server
2. We rotate our key
3. The server reaches the activity in its processing queue, tries to verify our
request signature, but fails because it can't fetch the key. It's the old
key and we discarded it already, replaced it with the new one
When we use 2 keys, the previous key remains available and other servers have
time to finish processing our requests signed with that key. We can safely
rotate, without worrying about whether the user sent anything right before the
rotation time.
Caveat: With this feature, we allow OTHER servers to rotate freely. It's safe
because it's optional, but it's just Vervis right now. Once Vervis itself
starts using 2 keys, it will be able to rotate freely without race condition
risk, but probably Mastodon etc. won't accept its signatures because of the use
of 2 keys and because they're server-scope keys.
Maybe I can get these features adopted by the fediverse?
