* Move logic for evaluating if a cosign should occur to its own file
Cleans it up and makes it more robust.
* Have expected_next_batch return an error instead of retrying
While convenient to offer an error-free implementation, it potentially caused
very long lived lock acquisitions in handle_processor_message.
* Unify and clean DkgConfirmer and DkgRemoval
Does so via adding a new file for the common code, SigningProtocol.
Modifies from_cache to return the preprocess with the machine, as there's no
reason not to. Also removes an unused Result around the type.
Clarifies the security around deterministic nonces, removing them for
saved-to-disk cached preprocesses. The cached preprocesses are encrypted as the
DB is not a proper secret store.
Moves arguments always present in the protocol from function arguments into the
struct itself.
Removes the horribly ugly code in DkgRemoval, fixing multiple issues present
with it which would cause it to fail on use.
* Set SeraiBlockNumber in cosign.rs as it's used by the cosigning protocol
* Remove unnecessary Clone from lambdas in coordinator
* Remove the EventDb from Tributary scanner
We used per-Transaction DB TXNs so on error, we don't have to rescan the entire
block yet only the rest of it. We prevented scanning multiple transactions by
tracking which we already had.
This is over-engineered and not worth it.
* Implement borsh for HasEvents, removing the manual encoding
* Merge DkgConfirmer and DkgRemoval into signing_protocol.rs
Fixes a bug in DkgConfirmer which would cause it to improperly handle indexes
if any validator had multiple key shares.
* Strictly type DataSpecification's Label
* Correct threshold_i_map_to_keys_and_musig_i_map
It didn't include the participant's own index and accordingly was offset.
* Create TributaryBlockHandler
This struct contains all variables prior passed to handle_block and stops them
from being passed around again and again.
This also ensures fatal_slash is only called while handling a block, as needed
as it expects to operate under perfect consensus.
* Inline accumulate, store confirmation nonces with shares
Inlining accumulate makes sense due to the amount of data accumulate needed to
be passed.
Storing confirmation nonces with shares ensures that both are available or
neither. Prior, one could be yet the other may not have been (requiring an
assert in runtime to ensure we didn't bungle it somehow).
* Create helper functions for handling DkgRemoval/SubstrateSign/Sign Tributary TXs
* Move Label into SignData
All of our transactions which use SignData end up with the same common usage
pattern for Label, justifying this.
Removes 3 transactions, explicitly de-duplicating their handlers.
* Remove CurrentlyCompletingKeyPair for the non-contextual DkgKeyPair
* Remove the manual read/write for TributarySpec for borsh
This struct doesn't have any optimizations booned by the manual impl. Using
borsh reduces our scope.
* Use temporary variables to further minimize LoC in tributary handler
* Remove usage of tuples for non-trivial Tributary transactions
* Remove serde from dkg
serde could be used to deserialize intenrally inconsistent objects which could
lead to panics or faults.
The BorshDeserialize derives have been replaced with a manual implementation
which won't produce inconsistent objects.
* Abstract Future generics using new trait definitions in coordinator
* Move published_signed_transaction to tributary/mod.rs to reduce the size of main.rs
* Split coordinator/src/tributary/mod.rs into spec.rs and transaction.rs
Event retrieval was prior:
- Retrieve all events in the block, which may be hundreds of KB
- Filter to just a few
Since it's frequent to want multiple sets of events, each filtered in their own
way, this caused the retrieval to happen multiple times. Now, it only will
happen once.
Also has the scoped clients take a reference, not an owned TemporalSerai.
* Make it clear not providing a change address is fingerprintable
When no change address is provided, all change is shunted to the
fee. This PR makes it clear to the caller that it is fingerprintable
when the caller does this.
* Review comments
* chore: implement create_db for substrate (fix broken branch)
* Correct rebase artifacts
* chore: remove todo statement
* chore: rename BlockDb to NextBlock
* chore: return empty tuple instead of empty array for event storage
* Finish rebasing
* .Minor tweaks to remove leftover variables
These may be rebase artifacts.
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
* coordinator/src/db.rs db macro implimentation
* fixed fmt errors
* converted txn functions to get/set counterparts
* use take_signed_transaction function
* fix for two fo the tests
* Misc tweaks
* Minor tweaks
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
Uses a full-fledged serai-abi to do so.
Removes use of UncheckedExtrinsic as a pointlessly (for us) length-prefixed
block with a more complicated signing algorithm than advantageous.
In the future, we should considering consolidating the various primitives
crates. I'm not convinced we benefit from one primitives crate per pallet.
Call and Event are both from the pallets, which are AGPL licensed. Accordingly,
they make serai-client AGPL licensed when serai-client must end up MIT
licensed. This creates a MIT-licensed variant of Calls and Events such that
they can be used by serai-client, enabling transitioning it to MIT.
Relevant to https://github.com/serai-dex/serai/issues/337.
Updates off a yanked version of zerocopy, fixing the failing deny CI.
Bites the bullet on windows-sys 0.52. While I was hoping to update everything
at once, unfortunately tokio won't update until March (see
https://github.com/tokio-rs/mio/pull/1725). I don't want to withold these
updates for that long.
They're a bit more binding, smaller, provided by the Rust bitcoin library,
sane, and we don't have to worry about malleability since all of our inputs are
SegWit.
The test failures were caused by not inserting any price, causing the first
price to immediately become the oraclized price. While that's not inherently
invalid, suggesting the tests should've been the ones updated, it opens an
exploit where whoever first adds liquidity has the opportunity to set a
ridiculous price and DoS the set. Not oraclizing until we have an entire
period, achieved by inserting 0s during the initial blocks, ensures an open
launch for such discovery.
There's an exploit where the prior set improperly mints coins, the new set
occurs (resetting the oracle), and they immediately deallocate 49.9% of their
coins (which is more than enough to achieve profitability).
Now, anyone in set must wait until after the next set completes to perform any
deallocation, enabling time to halt upon improper mints.
* Update ValidatorSets with a remove_participant call
* Add DkgRemoval, a sign machine for producing the relevant MuSig signatures
* Don't use position-dependent u8s yet Public when removing validators from the DKG
* Add DkgRemovalPreprocess, DkgRemovalShares
Implementation is via a new publish_tributary_tx lambda.
This is code is a copy-pasted mess which will need to be cleaned up.
* Only allow non-removed validators to vote for removals
Otherwise, it's risked that the remaining validators fall below 67% of the
original set.
* Correct publish_serai_tx, which was prior publish_set_keys in practice
This mirrors how Provided TXs handle topics.
Now, instead of managing a global nonce stream, we can use items such as plan
IDs as topics.
This massively benefits re-attempts, as else we'd need a NOP TX to clear unused
nonces.
* Use redb and in Dockerfiles
The motivation for redb was to remove the multiple rocksdb compile times from
CI.
* Correct feature flagging of coordinator and message-queue in Dockerfiles
* Correct message-queue DB type alias
* Use consistent table typing in redb
* Correct rebase artifacts
* Correct removal of binaries feature from message-queue
* Correct processor feature flagging
* Replace redb with parity-db
It still has much better compile times yet doesn't block when creating multiple
transactions. It also is actively maintained and doesn't grow our tree. The MPT
aspects are irrelevant.
* Correct stray Redb
* clippy warning
* Correct txn get
* Use debug builds in our Dockerfiles to reduce CI times
Also enables only spawning the mdns service when debug in the coordinator.
* Correct underflow in processor
Prior undetected due to relase builds not having bounds checks enabled.
* Restore Serai release due to CI/RPC failures caused by compiling it in debug mode
This is *probably* worth an issue filed upstream, if it can be tracked down.
* Correct failing debug asserts in Monero
These debug asserts assumed there was a change address to take the remainder.
If there's no change address, the remainder is shunted to the fee, causing the
fee to be distinct from the estimate.
We presumably need to modify monero-serai such that change: None isn't valid,
and users must use Change::Fingerprintable(None).
* Remove subxt
Removes ~20 crates from our Cargo.lock.
Removes downloading the metadata and enables removing the getMetadata RPC route
(relevant to #379).
Moves forward #337.
Done now due to distinctions in the subxt 0.32 API surface which make it
justifiable to not update.
* fmt, update due to deny triggering on a yanked crate
* Correct the handling of substrate_block_notifier now that it's ephemeral, not long-lived
* Correct URL in tests/coordinator from ws to http
* Remove NetworkId from processor-messages
Because intent binds to the sender/receiver, it's not needed for intent.
The processor knows what the network is.
The coordinator knows which to use because it's sending this message to the
processor for that network.
Also removes the unused zeroize.
* ProcessorMessage::Completed use Session instead of key
* Move SubstrateSignId to Session
* Finish replacing key with session
* Move message-queue to a fully binary representation
Additionally adds a timeout to the message queue test.
* coordinator clippy
* Remove contention for the message-queue socket by using per-request sockets
* clippy
For some reason, these constantly failed for me while waiting for the key pair
to confirm. This adds a sleep during the mining process, to ensure blocks
actually have time between them, and mines several more blocks to handle the
median code recently added.
Monero doesn't assert the time increases with each block, solely that it
doesn't decrease. Now, the block number is added to the time to ensure it
increases.
processor isn't intended to be used as a library, yet serai-processor-tests
does pull it in as a lib. This caused serai-processor-tests to need to compile
rocksdb, which added multiple minutes to the compilation time.
* Add SignalsConfig to chain_spec
* Correct multiexp feature flagging for rand_core std
* Remove bincode for borsh
Replaces a non-canonical encoding with a canonical encoding which additionally
should be faster.
Also fixes an issue where we used bincode in transcripts where it cannot be
trusted.
This ended up fixing a myriad of other bugs observed, unfortunately.
Accordingly, it either has to be merged or the bug fixes from it must be ported
to a new PR.
* Make serde optional, minimize usage
* Make borsh an optional dependency of substrate/ crates
* Remove unused dependencies
* Use [u8; 64] where possible in the processor messages
* Correct borsh feature flagging
Relevant to #394.
Prevents hand-over due to hand-over occurring via a `Batch` publication.
Expects a new protocol to restore functionality (after a retirement of the
current protocol).
