serai/docs/processor/UTXO Management.md

UTXO Management

UTXO-based chains have practical requirements for efficient operation which can effectively be guaranteed to terminate with a safe end state. This document attempts to detail such requirements, and the implementations in Serai resolving them.

Fees From Effecting Transactions Out

When sriXYZ is burnt, Serai is expected to create an output for XYZ as instructed. The transaction containing this output will presumably have some fee necessitating payment. Serai linearly amortizes this fee over all outputs this transaction intends to create in response to burns.

While Serai could charge a fee in advance, either static or dynamic to views of the fee market, it'd risk the fee being inaccurate. If it's too high, users have paid fees they shouldn't have. If it's too low, Serai is insolvent. This is why the actual fee is amortized, rather than an estimation being prepaid.

Serai could report a view, and when burning occurred, that view could be locked in as the basis for transaction fees as used to fulfill the output in question. This would require burns specify the most recent fee market view they're aware of, signifying their agreeance, with Serai erroring is a new view is published before the burn is included on-chain. Not only would this require more data be published to Serai (widening data pipeline requirements), it'd prevent any RBF-based solutions to dynamic fee markets causing transactions to get stuck.

Output Frequency

Outputs can be created on an external network at rate max_outputs_per_tx / external_tick_rate, where external_tick_rate is the external's network limitations on spending outputs. While external_tick_rate is generally writable as zero, due to mempool chaining, some external networks may not allow spending outputs from transactions which have yet to be ordered. Monero only allows spending outputs from transactions who have 10 confirmations, for its own security.

Serai defines its own tick rate per external network, such that serai_tick_rate >= external_tick_rate. This ensures that Serai never assumes availability before actual availability. serai_tick_rate is also > 0. This is since a zero external_tick_rate generally does not truly allow an infinite output creation rate due to limitations on the amount of transactions allowed in the mempool.

Define output_creation_rate as max_outputs_per_tx / serai_tick_rate. Under a naive system which greedily accumulates inputs and linearly processes outputs, this is the highest speed at which outputs which may be processed.

If the Serai blockchain enables burning sriXYZ at a rate exceeding output_creation_rate, a backlog would form. This backlog could linearly grow at a rate larger than the outputs could linearly shrink, creating an ever-growing backlog, performing a DoS against Serai.

One solution would be to increase the fee associated with burning sriXYZ when approaching output_creation_rate, making such a DoS unsustainable. This would require the Serai blockchain be aware of each external network's output_creation_rate and implement such a sliding fee. This 'solution' isn't preferred as it still temporarily has a growing queue, and normal users would also be affected by the increased fees.

The solution implemented into Serai is to consume all burns from the start of a global queue which can be satisfied under currently available inputs. While the consumed queue may have 256 items, which can't be processed within a single tick by an external network whose output_creation_rate is 16, Serai can immediately set a finite bound on execution duration.

For the above example parameters, Serai would create 16 outputs within its tick, ignoring the necessity of a change output. These 16 outputs would not create any outputs Serai is expected to create in response to burns, yet instead create 16 "branch" outputs. One tick later, when the branch outputs are available to spend, each would fund creating of 16 expected outputs.

For e expected outputs, the execution duration is just log e ticks with the base of the logarithm being output_creation_rate. Since these e expected outputs are consumed from the linearly-implemented global queue into their own tree structure, execution duration cannot be extended. We can also re-consume the entire global queue (barring input availability, see next section) after just one tick, when the change output becomes available again.

Due to the logarithmic complexity of fulfilling burns, attacks require exponential growth (which is infeasible to scale). This solution does not require a sliding fee on Serai's side due to not needing to limit the on-chain rate of burns, which means it doesn't so adversely affect normal users. While an increased tree depth will increase the amount of transactions needed to fulfill an output, increasing the fee amortized over the output and its siblings, this fee scales linearly with the logarithmically scaling tree depth. This is considered acceptable.

Input Availability

The following section refers to spending an output, and then spending it again. Spending it again, which is impossible under the UTXO model, refers to spending the change output of the transaction it was spent in. The following section also assumes any published transaction is immediately ordered on-chain, ignoring the potential for latency from mempool to blockchain (as it is assumed to have a negligible effect in practice).

When a burn for amount a is issued, the sum amount of immediately available inputs may be < a. This is because despite each output being considered usable on a tick basis, there is no global tick. Each output may or may not be spendable at some moment, and spending it will prevent its availability for one tick of a clock newly started.

This means all outputs will become available by simply waiting a single tick, without spending any outputs during the waited tick. Any outputs unlocked at the start of the tick will carry, and within the tick the rest of the outputs will become unlocked.

This means that within a tick of operations, the full balance of Serai can be considered unlocked and used to consume the entire global queue. While Serai could wait for all its outputs to be available before popping from the front of the global queue, eager execution as enough inputs become available provides lower latency. Considering the tick may be an hour (as in the case of Bitcoin), this is very appreciated.

If a full tick is waited for, due to the front of the global queue having a notably large burn, then the entire global queue will be consumed as full input availability means the ability to satisfy all potential burns in a solvent system.

Fees Incurred During Operations

While fees incurred when satisfying burn were covered above, with documentation on how solvency is maintained, two other operating costs exists.

1. Input accumulation
2. Multisig rotations

Input accumulation refers to transactions which exist to merge inputs. Just as there is a max_outputs_per_tx, there is a max_inputs_per_tx. When the amount of inputs belonging to Serai exceeds max_inputs_per_tx, a TX merging them is created. This TX incurs fees yet has no outputs mapping to burns to amortize them over, creating an insolvency.

Please note that this merging occurs in parallel to create a logarithmic execution, similar to how outputs are also processed in parallel.

As for multisig rotation, multisig rotation occurs when a new multisig for an external network is created and the old multisig must transfer its inputs in order for Serai to continue its operations. This operation also incurs fees without having outputs immediately available to amortize over.

Serai could charge fees on received outputs, deducting from the amount of sriXYZ minted in order to cover these operating fees. An overt amount would be deducted to practically ensure solvency, forming a buffer. Once the buffer is filled, fees would be reduced. As the buffer drains, fees would go back up.

This would keep charged fees in line with actual fees, once the buffer is initially filled, yet requires:

1. Creating and tracking a buffer
2. Overcharging some users on fees

while still risking insolvency, if the actual fees keep increasing in a way preventing successful estimation.

The solution Serai implements is to accrue insolvency, tracking each output with a virtual amount (the amount it represents on Serai) and the actual amount. When the output, or a descendant of it, is used to handle burns, the discrepancy between the virtual amount and the amount is amortized over outputs. This restores solvency while solely charging the actual fees, making Serai a generally insolvent, always eventually solvent system.

There is the concern that a significant amount of outputs could be created, which when merged as inputs, create a significant amount of fees as an insolvency. This would then be forced onto random users, while the party who created the insolvency would then be able to burn their own sriXYZ without the notable insolvency.

To describe this attack in its optimal form, assume a sole malicious block producer for an external network where max_inputs_per_tx is 16. The malicious miner adds 256 outputs to Serai, not paying any fees as the block producer. Serai must create 16 transactions to produce a set of 16 inputs, paying for 16 transaction fees in the process (the fees of which go to the malicious miner).

When Serai users burn sriXYZ, they are hit with the 16 transaction fees plus the normally amortized fee. Then, the malicious miner burns their sriXYZ, having the fee they capture be amortized over their output. In this process, they remain net except for the 16 transaction fees they gain from other users, which they profit.

A miner only has to have 7% of the external network's hash power to execute this attack profitably. By only minting sriXYZ during their blocks, they pay no fees. Then, a miner, which has a 7% chance of being themselves, collects the 16 transaction fees. Finally, they burn, with a 7% chance of collecting their own fee, or a 93% chance of losing a single transaction fee.

16 attempts, costing 16 transaction fees if they always lose their single transaction fee, will cause a slight edge they gain the 16 transaction fees at least once, offsetting their costs.

To limit this attack vector, a flat fee of 2 * (the estimation of an input-merging transaction fee) / max_inputs_per_tx is applied to each input. This means, assuming an inability to manipulate Serai's fee estimations, creating 16 outputs to force a merge transaction (and the associated fee) costs the attacker twice as much as the associated fee.

Even without the above flat fee, Serai remains solvent. With the above flat fee, malicious miners on external networks can only steal from other users if they can manipulate Serai's fee estimations so that the merge transaction fee used is twice as high as the fees charged for causing a merge transaction. This is assumed infeasible to perform at scale, yet even if demonstrated feasible, it would not be a critical vulnerability against Serai. Solely a low/medium/high vulnerability against the users (though one it would still be our responsibility to rectify).