Ecosystem/serai
Ecosystem/serai
1
0
Fork 0
mirror of https://github.com/serai-dex/serai.git synced 2024-12-23 03:59:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
serai/coordinator/Cargo.toml

65 lines
3 KiB
TOML
Raw Permalink Normal View History

Add empty coordinator
2023-04-11 13:21:35 +00:00
[package]
Use serai- prefixes on Serai-specific packages Fixes deny.toml, also runs a minor cargo update shrinking the tree.
2023-07-03 12:09:41 +00:00
name = "serai-coordinator"
Add empty coordinator
2023-04-11 13:21:35 +00:00
version = "0.1.0"
description = "Serai coordinator to prepare batches and sign transactions"
license = "AGPL-3.0-only"
repository = "https://github.com/serai-dex/serai/tree/develop/coordinator"
authors = ["Luke Parker <lukeparker5132@gmail.com>"]
keywords = []
edition = "2021"
publish = false
[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "docsrs"]
Add workspace lints
2023-12-17 01:54:24 +00:00
[lints]
workspace = true
Add empty coordinator
2023-04-11 13:21:35 +00:00
[dependencies]
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
async-trait = { version = "0.1", default-features = false }
Make progres on handling NewSet events Further bones out the coordinator.
2023-04-16 04:51:56 +00:00
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
zeroize = { version = "^1.5", default-features = false, features = ["std"] }
rand_core = { version = "0.6", default-features = false, features = ["std"] }
Start defining the coordinator
2023-04-15 21:38:47 +00:00
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
blake2 = { version = "0.10", default-features = false, features = ["std"] }
Define all coordinator transaction types
2023-04-11 23:04:53 +00:00
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
transcript = { package = "flexible-transcript", path = "../crypto/transcript", default-features = false, features = ["std", "recommended"] }
ciphersuite = { path = "../crypto/ciphersuite", default-features = false, features = ["std"] }
schnorr = { package = "schnorr-signatures", path = "../crypto/schnorr", default-features = false, features = ["std"] }
Define all coordinator transaction types
2023-04-11 23:04:53 +00:00
frost = { package = "modular-frost", path = "../crypto/frost" }
Perform MuSig signing of generated keys
2023-08-14 10:08:55 +00:00
frost-schnorrkel = { path = "../crypto/schnorrkel" }
Define all coordinator transaction types
2023-04-11 23:04:53 +00:00
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
scale = { package = "parity-scale-codec", version = "3", default-features = false, features = ["std", "derive"] }
Add a test to the coordinator for running a Tributary Impls a LocalP2p for testing. Moves rebroadcasting into Tendermint, since it's what knows if a message is fully valid + original. Removes TributarySpec::validators() HashMap, as its non-determinism caused different instances to have different round robin schedules. It was already prior moved to a Vec for this issue, so I'm unsure why this remnant existed. Also renames the GH no-std workflow from the prior commit.
2023-04-22 14:49:52 +00:00
PR to track down CI failures (#501) * Use an extended timeout for DKGs specifically * Add a log statement when message-queue connection fails * Add a 60 second keep-alive to connections * Use zalloc for processor/message-queue/coordinator An additional layer which protects us against edge cases with Zeroizing (objects which don't support it or don't miss it). * Add further logs to message-queue * Further increase re-attempt timeouts in CI * Remove misplaced continue inmessage-queue client Fixes observed CI failures. * Revert "Further increase re-attempt timeouts in CI" This reverts commit 3723530cf610e784adf5c94ec1c723b58ce7b193.
2024-01-04 06:08:13 +00:00
zalloc = { path = "../common/zalloc" }
Use parity-db in current Dockerfiles (#455) * Use redb and in Dockerfiles The motivation for redb was to remove the multiple rocksdb compile times from CI. * Correct feature flagging of coordinator and message-queue in Dockerfiles * Correct message-queue DB type alias * Use consistent table typing in redb * Correct rebase artifacts * Correct removal of binaries feature from message-queue * Correct processor feature flagging * Replace redb with parity-db It still has much better compile times yet doesn't block when creating multiple transactions. It also is actively maintained and doesn't grow our tree. The MPT aspects are irrelevant. * Correct stray Redb * clippy warning * Correct txn get
2023-11-30 09:22:37 +00:00
serai-db = { path = "../common/db" }
Integrate coordinator with MessageQueue and RocksDB Also resolves a couple TODOs.
2023-07-18 05:53:51 +00:00
serai-env = { path = "../common/env" }
Start defining the coordinator
2023-04-15 21:38:47 +00:00
Use serai- prefixes on Serai-specific packages Fixes deny.toml, also runs a minor cargo update shrinking the tree.
2023-07-03 12:09:41 +00:00
processor-messages = { package = "serai-processor-messages", path = "../processor/messages" }
Integrate coordinator with MessageQueue and RocksDB Also resolves a couple TODOs.
2023-07-18 05:53:51 +00:00
message-queue = { package = "serai-message-queue", path = "../message-queue" }
Define all coordinator transaction types
2023-04-11 23:04:53 +00:00
tributary = { package = "tributary-chain", path = "./tributary" }
Add a cosigning protocol to ensure finalizations are unique (#433) * Add a function to deterministically decide which Serai blocks should be co-signed Has a 5 minute latency between co-signs, also used as the maximal latency before a co-sign is started. * Get all active tributaries we're in at a specific block * Add and route CosignSubstrateBlock, a new provided TX * Split queued cosigns per network * Rename BatchSignId to SubstrateSignId * Add SubstrateSignableId, a meta-type for either Batch or Block, and modularize around it * Handle the CosignSubstrateBlock provided TX * Revert substrate_signer.rs to develop (and patch to still work) Due to SubstrateSigner moving when the prior multisig closes, yet cosigning occurring with the most recent key, a single SubstrateSigner can be reused. We could manage multiple SubstrateSigners, yet considering the much lower specifications for cosigning, I'd rather treat it distinctly. * Route cosigning through the processor * Add note to rename SubstrateSigner post-PR I don't want to do so now in order to preserve the diff's clarity. * Implement cosign evaluation into the coordinator * Get tests to compile * Bug fixes, mark blocks without cosigners available as cosigned * Correct the ID Batch preprocesses are saved under, add log statements * Create a dedicated function to handle cosigns * Correct the flow around Batch verification/queueing Verifying `Batch`s could stall when a `Batch` was signed before its predecessors/before the block it's contained in was cosigned (the latter being inevitable as we can't sign a block containing a signed batch before signing the batch). Now, Batch verification happens on a distinct async task in order to not block the handling of processor messages. This task is the sole caller of verify in order to ensure last_verified_batch isn't unexpectedly mutated. When the processor message handler needs to access it, or needs to queue a Batch, it associates the DB TXN with a lock preventing the other task from doing so. This lock, as currently implemented, is a poor and inefficient design. It should be modified to the pattern used for cosign management. Additionally, a new primitive of a DB-backed channel may be immensely valuable. Fixes a standing potential deadlock and a deadlock introduced with the cosigning protocol. * Working full-stack tests After the last commit, this only required extending a timeout. * Replace "co-sign" with "cosign" to make finding text easier * Update the coordinator tests to support cosigning * Inline prior_batch calculation to prevent panic on rotation Noticed when doing a final review of the branch.
2023-11-15 21:57:21 +00:00
sp-application-crypto = { git = "https://github.com/serai-dex/substrate", default-features = false, features = ["std"] }
Coordinator Cleanup (#481) * Move logic for evaluating if a cosign should occur to its own file Cleans it up and makes it more robust. * Have expected_next_batch return an error instead of retrying While convenient to offer an error-free implementation, it potentially caused very long lived lock acquisitions in handle_processor_message. * Unify and clean DkgConfirmer and DkgRemoval Does so via adding a new file for the common code, SigningProtocol. Modifies from_cache to return the preprocess with the machine, as there's no reason not to. Also removes an unused Result around the type. Clarifies the security around deterministic nonces, removing them for saved-to-disk cached preprocesses. The cached preprocesses are encrypted as the DB is not a proper secret store. Moves arguments always present in the protocol from function arguments into the struct itself. Removes the horribly ugly code in DkgRemoval, fixing multiple issues present with it which would cause it to fail on use. * Set SeraiBlockNumber in cosign.rs as it's used by the cosigning protocol * Remove unnecessary Clone from lambdas in coordinator * Remove the EventDb from Tributary scanner We used per-Transaction DB TXNs so on error, we don't have to rescan the entire block yet only the rest of it. We prevented scanning multiple transactions by tracking which we already had. This is over-engineered and not worth it. * Implement borsh for HasEvents, removing the manual encoding * Merge DkgConfirmer and DkgRemoval into signing_protocol.rs Fixes a bug in DkgConfirmer which would cause it to improperly handle indexes if any validator had multiple key shares. * Strictly type DataSpecification's Label * Correct threshold_i_map_to_keys_and_musig_i_map It didn't include the participant's own index and accordingly was offset. * Create TributaryBlockHandler This struct contains all variables prior passed to handle_block and stops them from being passed around again and again. This also ensures fatal_slash is only called while handling a block, as needed as it expects to operate under perfect consensus. * Inline accumulate, store confirmation nonces with shares Inlining accumulate makes sense due to the amount of data accumulate needed to be passed. Storing confirmation nonces with shares ensures that both are available or neither. Prior, one could be yet the other may not have been (requiring an assert in runtime to ensure we didn't bungle it somehow). * Create helper functions for handling DkgRemoval/SubstrateSign/Sign Tributary TXs * Move Label into SignData All of our transactions which use SignData end up with the same common usage pattern for Label, justifying this. Removes 3 transactions, explicitly de-duplicating their handlers. * Remove CurrentlyCompletingKeyPair for the non-contextual DkgKeyPair * Remove the manual read/write for TributarySpec for borsh This struct doesn't have any optimizations booned by the manual impl. Using borsh reduces our scope. * Use temporary variables to further minimize LoC in tributary handler * Remove usage of tuples for non-trivial Tributary transactions * Remove serde from dkg serde could be used to deserialize intenrally inconsistent objects which could lead to panics or faults. The BorshDeserialize derives have been replaced with a manual implementation which won't produce inconsistent objects. * Abstract Future generics using new trait definitions in coordinator * Move published_signed_transaction to tributary/mod.rs to reduce the size of main.rs * Split coordinator/src/tributary/mod.rs into spec.rs and transaction.rs
2023-12-11 01:21:44 +00:00
serai-client = { path = "../substrate/client", default-features = false, features = ["serai", "borsh"] }
Start defining the coordinator
2023-04-15 21:38:47 +00:00
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
hex = { version = "0.4", default-features = false, features = ["std"] }
Replace bincode with borsh (#452) * Add SignalsConfig to chain_spec * Correct multiexp feature flagging for rand_core std * Remove bincode for borsh Replaces a non-canonical encoding with a canonical encoding which additionally should be faster. Also fixes an issue where we used bincode in transcripts where it cannot be trusted. This ended up fixing a myriad of other bugs observed, unfortunately. Accordingly, it either has to be merged or the bug fixes from it must be ported to a new PR. * Make serde optional, minimize usage * Make borsh an optional dependency of substrate/ crates * Remove unused dependencies * Use [u8; 64] where possible in the processor messages * Correct borsh feature flagging
2023-11-25 09:01:11 +00:00
borsh = { version = "1", default-features = false, features = ["std", "derive", "de_strict_order"] }
Add initial coordinator e2e tests
2023-08-01 23:00:48 +00:00
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
log = { version = "0.4", default-features = false, features = ["std"] }
env_logger = { version = "0.10", default-features = false, features = ["humantime"] }
Add initial coordinator e2e tests
2023-08-01 23:00:48 +00:00
Replace remaining direct uses of futures with futures_util Slight downscope which helps combat the antipattern which is the futures glob crate. While futures_util is still a large crate, it has better defaults and is smaller by virtue of not pulling the executor.
2023-12-19 00:45:08 +00:00
futures-util = { version = "0.3", default-features = false, features = ["std"] }
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
tokio = { version = "1", default-features = false, features = ["rt-multi-thread", "sync", "time", "macros"] }
Remove cbor
2024-04-23 11:01:07 +00:00
libp2p = { version = "0.52", default-features = false, features = ["tokio", "tcp", "noise", "yamux", "request-response", "gossipsub", "macros"] }
Define all coordinator transaction types
2023-04-11 23:04:53 +00:00
[dev-dependencies]
tributary = { package = "tributary-chain", path = "./tributary", features = ["tests"] }
Correct std feature-flagging If a crate has std set, it should enable std for all dependencies in order to let them properly select which algorithms to use. Some crates fallback to slower/worse algorithms on no-std. Also more aggressively sets default-features = false leading to a *10%* reduction in the amount of crates coordinator builds.
2023-10-31 11:41:23 +00:00
sp-application-crypto = { git = "https://github.com/serai-dex/substrate", default-features = false, features = ["std"] }
sp-runtime = { git = "https://github.com/serai-dex/substrate", default-features = false, features = ["std"] }
Use parity-db in current Dockerfiles (#455) * Use redb and in Dockerfiles The motivation for redb was to remove the multiple rocksdb compile times from CI. * Correct feature flagging of coordinator and message-queue in Dockerfiles * Correct message-queue DB type alias * Use consistent table typing in redb * Correct rebase artifacts * Correct removal of binaries feature from message-queue * Correct processor feature flagging * Replace redb with parity-db It still has much better compile times yet doesn't block when creating multiple transactions. It also is actively maintained and doesn't grow our tree. The MPT aspects are irrelevant. * Correct stray Redb * clippy warning * Correct txn get
2023-11-30 09:22:37 +00:00
[features]
Increase reattempt delays in the GH CI, which is extremely latent
2023-12-31 03:11:04 +00:00
longer-reattempts = []
Use parity-db in current Dockerfiles (#455) * Use redb and in Dockerfiles The motivation for redb was to remove the multiple rocksdb compile times from CI. * Correct feature flagging of coordinator and message-queue in Dockerfiles * Correct message-queue DB type alias * Use consistent table typing in redb * Correct rebase artifacts * Correct removal of binaries feature from message-queue * Correct processor feature flagging * Replace redb with parity-db It still has much better compile times yet doesn't block when creating multiple transactions. It also is actively maintained and doesn't grow our tree. The MPT aspects are irrelevant. * Correct stray Redb * clippy warning * Correct txn get
2023-11-30 09:22:37 +00:00
parity-db = ["serai-db/parity-db"]
rocksdb = ["serai-db/rocksdb"]
Reference in a new issue Copy permalink