* Remove subxt
Removes ~20 crates from our Cargo.lock.
Removes downloading the metadata and enables removing the getMetadata RPC route
(relevant to #379).
Moves forward #337.
Done now due to distinctions in the subxt 0.32 API surface which make it
justifiable to not update.
* fmt, update due to deny triggering on a yanked crate
* Correct the handling of substrate_block_notifier now that it's ephemeral, not long-lived
* Correct URL in tests/coordinator from ws to http
* Remove NetworkId from processor-messages
Because intent binds to the sender/receiver, it's not needed for intent.
The processor knows what the network is.
The coordinator knows which to use because it's sending this message to the
processor for that network.
Also removes the unused zeroize.
* ProcessorMessage::Completed use Session instead of key
* Move SubstrateSignId to Session
* Finish replacing key with session
* Move message-queue to a fully binary representation
Additionally adds a timeout to the message queue test.
* coordinator clippy
* Remove contention for the message-queue socket by using per-request sockets
* clippy
For some reason, these constantly failed for me while waiting for the key pair
to confirm. This adds a sleep during the mining process, to ensure blocks
actually have time between them, and mines several more blocks to handle the
median code recently added.
Monero doesn't assert the time increases with each block, solely that it
doesn't decrease. Now, the block number is added to the time to ensure it
increases.
processor isn't intended to be used as a library, yet serai-processor-tests
does pull it in as a lib. This caused serai-processor-tests to need to compile
rocksdb, which added multiple minutes to the compilation time.
* Add SignalsConfig to chain_spec
* Correct multiexp feature flagging for rand_core std
* Remove bincode for borsh
Replaces a non-canonical encoding with a canonical encoding which additionally
should be faster.
Also fixes an issue where we used bincode in transcripts where it cannot be
trusted.
This ended up fixing a myriad of other bugs observed, unfortunately.
Accordingly, it either has to be merged or the bug fixes from it must be ported
to a new PR.
* Make serde optional, minimize usage
* Make borsh an optional dependency of substrate/ crates
* Remove unused dependencies
* Use [u8; 64] where possible in the processor messages
* Correct borsh feature flagging
Relevant to #394.
Prevents hand-over due to hand-over occurring via a `Batch` publication.
Expects a new protocol to restore functionality (after a retirement of the
current protocol).
* implement db macro for processor/substrate_signer
* Use ()
* Correct AttemptDb usage of ()
* () -> &()
---------
Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
* Remove dtolnay's rust-toolchain action
I believe our rust-toolchain.toml handles its use case exactly.
I don't believe this'll work, as it'd require rustup install a cargo stub
before any toolchain is installed, yet I want to confirm it doesn't.
* Place quotes around nightly toolchain version
* Put toolchain before options to resolve what appears to be a bug in rustup's help strings
* Add wasm32-unkknown-unknown to clippy workflow
Adds a Rust toolchain file to be less disruptive to developers who don't keep
their toolchain synchronized (by now having rustup automatically synchronize).
Hopefully helps resolve how +nightly clippy may pass for the coordinator, yet
building would fail due to stable's (hopefully prior?) failure to model some
async functions re: Send/Sync.
Also adds rust-src as a component in preparation of
https://github.com/paritytech/polkadot-sdk/pull/2217
The coordinator already had one of these, albeit implemented much worse than
the one now properly introduced. It had to either be sending or receiving,
whereas the new one can do both at the same time.
This replaces said instance and enables pleasant patterns when implementing the
processor/coordinator.
They were already banned form publishing `Batch`s, yet the ability to set keys
enabled changing how certain functionality in the coordinator operated.
Removing this malleability ensures operation as expected.
* Add a function to deterministically decide which Serai blocks should be co-signed
Has a 5 minute latency between co-signs, also used as the maximal latency
before a co-sign is started.
* Get all active tributaries we're in at a specific block
* Add and route CosignSubstrateBlock, a new provided TX
* Split queued cosigns per network
* Rename BatchSignId to SubstrateSignId
* Add SubstrateSignableId, a meta-type for either Batch or Block, and modularize around it
* Handle the CosignSubstrateBlock provided TX
* Revert substrate_signer.rs to develop (and patch to still work)
Due to SubstrateSigner moving when the prior multisig closes, yet cosigning
occurring with the most recent key, a single SubstrateSigner can be reused.
We could manage multiple SubstrateSigners, yet considering the much lower
specifications for cosigning, I'd rather treat it distinctly.
* Route cosigning through the processor
* Add note to rename SubstrateSigner post-PR
I don't want to do so now in order to preserve the diff's clarity.
* Implement cosign evaluation into the coordinator
* Get tests to compile
* Bug fixes, mark blocks without cosigners available as cosigned
* Correct the ID Batch preprocesses are saved under, add log statements
* Create a dedicated function to handle cosigns
* Correct the flow around Batch verification/queueing
Verifying `Batch`s could stall when a `Batch` was signed before its
predecessors/before the block it's contained in was cosigned (the latter being
inevitable as we can't sign a block containing a signed batch before signing
the batch).
Now, Batch verification happens on a distinct async task in order to not block
the handling of processor messages. This task is the sole caller of verify in
order to ensure last_verified_batch isn't unexpectedly mutated.
When the processor message handler needs to access it, or needs to queue a
Batch, it associates the DB TXN with a lock preventing the other task from
doing so.
This lock, as currently implemented, is a poor and inefficient design. It
should be modified to the pattern used for cosign management. Additionally, a
new primitive of a DB-backed channel may be immensely valuable.
Fixes a standing potential deadlock and a deadlock introduced with the
cosigning protocol.
* Working full-stack tests
After the last commit, this only required extending a timeout.
* Replace "co-sign" with "cosign" to make finding text easier
* Update the coordinator tests to support cosigning
* Inline prior_batch calculation to prevent panic on rotation
Noticed when doing a final review of the branch.
