Commit graph

1396 commits

Author SHA1 Message Date
Luke Parker
88a1726399
Fixes for prior commit 2023-11-22 16:24:50 -05:00
Luke Parker
08e6669403
Replace substrate/client's use of Payload with usage of RuntimeCall
Gains explicit typing.
2023-11-22 11:23:04 -05:00
akildemir
fcfdadc791
Integrate session pallet into validator-sets pallet (#440)
* remove pallet-session

* Store key shares in InSet

* integrate grandpa to vs-pallet

* integrate pallet babe

* remove pallet-session & authority discovery from runtime

* update the grandpa pallet path

* cargo update grandpa

* cargo update substrate

* Misc tweaks

Sets validators for BABE/GRANDPA in chain_spec, per Akil's realization that was
the missing piece.

* fix pr comments

* bug fix & tidy up

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-22 06:22:46 -05:00
Luke Parker
07c657306b
Remove duplicated genesis presence in Tributary Scanner DB keys
This wasted 32-bytes per every single entry in the DB (ignoring de-duplication
possible by the DB layer).
2023-11-22 04:43:49 -05:00
David Bell
9df8c9476e
Convert tributary to use create_db macro (#448)
* chore: convert tributary to use create_db macro

* chore: fix fmt

* chore: break long line
2023-11-22 04:17:51 -05:00
econsta
9ab2a2cfe0
processor/db.rs macro implimentation (#437)
* processor/db.rs macro implimentation

* ran clippy and fmt

* incorporated recommendations

* used empty uple instead of [u8; 0]

* ran fmt
2023-11-22 03:58:26 -05:00
Luke Parker
a315040aee
if-watch 3.2.0 2023-11-22 01:24:17 -05:00
Luke Parker
1822e31142
Grow the yamux buffers to exceed the maximum message size 2023-11-21 02:01:41 -05:00
Luke Parker
6efc313d76
Add/update msrv for common/*, crypto/*, coins/*, and substrate/*
This includes all published crates.
2023-11-21 01:19:40 -05:00
Luke Parker
4b85d4b03b
Rename SubstrateSigner to BatchSigner 2023-11-20 22:21:52 -05:00
econsta
05d8c32be8
Multisig db (#444)
* implement db macro for processor/multisigs/db.rs

* ran fmt

* cargo +nightly fmt

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-20 21:40:54 -05:00
econsta
8634d90b6b
implement db macro for processor/signer.rs (#438)
* implement db macro for processor/signer.rs

* Use ()

* CompletedDb -> CompletionsDb

* () -> &()

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-20 03:03:45 -05:00
econsta
aa9daea6b0
implement db macro for processor/substrate_signer (#439)
* implement db macro for processor/substrate_signer

* Use ()

* Correct AttemptDb usage of ()

* () -> &()

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-20 03:03:35 -05:00
Luke Parker
942873f99d Non-intrusive cargo update 2023-11-20 02:31:47 -05:00
Luke Parker
7c9b581723
Remove dtolnay's rust-toolchain action (#442)
* Remove dtolnay's rust-toolchain action

I believe our rust-toolchain.toml handles its use case exactly.

I don't believe this'll work, as it'd require rustup install a cargo stub
before any toolchain is installed, yet I want to confirm it doesn't.

* Place quotes around nightly toolchain version

* Put toolchain before options to resolve what appears to be a bug in rustup's help strings

* Add wasm32-unkknown-unknown to clippy workflow
2023-11-20 02:31:22 -05:00
Luke Parker
b37a0db538
Move where the RISC-V toolchain is installed during the no-std workflow 2023-11-19 22:45:51 -05:00
Luke Parker
797604ad73
Replace usage of io::Error::new(io::ErrorKind::Other, with io::Error::other
Newly possible with Rust 1.74.
2023-11-19 18:31:37 -05:00
Luke Parker
05b975dff9
Rust 1.74
Adds a Rust toolchain file to be less disruptive to developers who don't keep
their toolchain synchronized (by now having rustup automatically synchronize).

Hopefully helps resolve how +nightly clippy may pass for the coordinator, yet
building would fail due to stable's (hopefully prior?) failure to model some
async functions re: Send/Sync.

Also adds rust-src as a component in preparation of
https://github.com/paritytech/polkadot-sdk/pull/2217
2023-11-19 17:47:46 -05:00
Luke Parker
74a8df4c7b
Add a new primitive of a DB-backed channel
The coordinator already had one of these, albeit implemented much worse than
the one now properly introduced. It had to either be sending or receiving,
whereas the new one can do both at the same time.

This replaces said instance and enables pleasant patterns when implementing the
processor/coordinator.
2023-11-19 02:05:01 -05:00
Luke Parker
bd14db9f76
Version pin foundry
Fixes #116 and #441.
2023-11-19 01:21:43 -05:00
Luke Parker
25c02c1311
Ban Serai from setting keys
They were already banned form publishing `Batch`s, yet the ability to set keys
enabled changing how certain functionality in the coordinator operated.
Removing this malleability ensures operation as expected.
2023-11-18 21:28:16 -05:00
Luke Parker
b018fc432c
Add deletion to create_db 2023-11-18 20:56:58 -05:00
Luke Parker
6e4ecbc90c
Support trailing commas in create_db 2023-11-18 20:54:37 -05:00
Luke Parker
be48dcc4a4
Use multiple LibP2P topics
We still need a peering protocol... Hopefully, we can read peers off of the
Substrate node's DHT.
2023-11-18 20:37:55 -05:00
Luke Parker
25066437da
Attempt to resolve #434
Worsens the coordinator tests' ensuring of the validity of the cosigning
protocol, yet should actually properly model it.
2023-11-18 00:12:36 -05:00
Luke Parker
db49a63c2b
Bump zeroize to appease cargo deny 2023-11-16 14:18:00 -05:00
Luke Parker
ee50f584aa
Add saner log statements to the coordinator
Disables trace on every single P2P message.

Logs a short-form of each transaction.
2023-11-16 13:37:39 -05:00
Luke Parker
14f3f330db
Remove deadlock in cosign_evaluator 2023-11-16 13:32:15 -05:00
Luke Parker
30a77d863f
Include non-JSON response from Monero node in error 2023-11-15 22:57:37 -05:00
Luke Parker
a03a1edbff
Remove needless transposition in coordinator 2023-11-15 22:49:58 -05:00
Luke Parker
c03afbe03e
Downgrade RustCrypto packages which violate Rust's interpretation of semver 2023-11-15 21:24:52 -05:00
Luke Parker
af611a39bf
Restore runtime feature to hyper 2023-11-15 20:40:24 -05:00
Luke Parker
0d080e6d34
Remove unused dependency from dex-pallet 2023-11-15 20:24:33 -05:00
Luke Parker
369af0fab5
\#339 addendum 2023-11-15 20:23:19 -05:00
Luke Parker
d25e3d86a2
Make TLS an optional feature of simple-request
Removes 14 crates from the tree when compiling the message-queue client.

Also performs a non-intrusive cargo update.
2023-11-15 17:24:11 -05:00
Luke Parker
96f1d26f7a
Add a cosigning protocol to ensure finalizations are unique (#433)
* Add a function to deterministically decide which Serai blocks should be co-signed

Has a 5 minute latency between co-signs, also used as the maximal latency
before a co-sign is started.

* Get all active tributaries we're in at a specific block

* Add and route CosignSubstrateBlock, a new provided TX

* Split queued cosigns per network

* Rename BatchSignId to SubstrateSignId

* Add SubstrateSignableId, a meta-type for either Batch or Block, and modularize around it

* Handle the CosignSubstrateBlock provided TX

* Revert substrate_signer.rs to develop (and patch to still work)

Due to SubstrateSigner moving when the prior multisig closes, yet cosigning
occurring with the most recent key, a single SubstrateSigner can be reused.
We could manage multiple SubstrateSigners, yet considering the much lower
specifications for cosigning, I'd rather treat it distinctly.

* Route cosigning through the processor

* Add note to rename SubstrateSigner post-PR

I don't want to do so now in order to preserve the diff's clarity.

* Implement cosign evaluation into the coordinator

* Get tests to compile

* Bug fixes, mark blocks without cosigners available as cosigned

* Correct the ID Batch preprocesses are saved under, add log statements

* Create a dedicated function to handle cosigns

* Correct the flow around Batch verification/queueing

Verifying `Batch`s could stall when a `Batch` was signed before its
predecessors/before the block it's contained in was cosigned (the latter being
inevitable as we can't sign a block containing a signed batch before signing
the batch).

Now, Batch verification happens on a distinct async task in order to not block
the handling of processor messages. This task is the sole caller of verify in
order to ensure last_verified_batch isn't unexpectedly mutated.

When the processor message handler needs to access it, or needs to queue a
Batch, it associates the DB TXN with a lock preventing the other task from
doing so.

This lock, as currently implemented, is a poor and inefficient design. It
should be modified to the pattern used for cosign management. Additionally, a
new primitive of a DB-backed channel may be immensely valuable.

Fixes a standing potential deadlock and a deadlock introduced with the
cosigning protocol.

* Working full-stack tests

After the last commit, this only required extending a timeout.

* Replace "co-sign" with "cosign" to make finding text easier

* Update the coordinator tests to support cosigning

* Inline prior_batch calculation to prevent panic on rotation

Noticed when doing a final review of the branch.
2023-11-15 16:57:21 -05:00
Luke Parker
79e4cce2f6
Explicitly set features in modular-frost 2023-11-13 05:31:47 -05:00
Luke Parker
0c341e3546
Fix no-std builds 2023-11-13 05:19:53 -05:00
Luke Parker
9f0790fb83
Remove RecommendedTranscript from DKG MuSig
Resolves #391.

Given this code already wasn't modular/composable, this should be overall
equivalent regarding functionality and security. It's much less opinionated
though and has fewer dependencies.
2023-11-13 05:11:40 -05:00
Luke Parker
bb8e034e68
Test historic start times in tendermint-machine
Closes https://github.com/serai-dex/serai/issues/342.

Under ideal network conditions, this is fine. While I won't claim ideal network
conditions will occur IRL, b0fcdd3367 has the
Tributary rebroadcast messages and should brute-force its way into a
functioning system.
2023-11-13 00:43:35 -05:00
Luke Parker
3f7bdaa64b
Make the output distribution cache only available under a feature
Enables a mode with reduced memory usage *and* increased safety given current
unsafety of the cache.

Relevant to https://github.com/serai-dex/serai/issues/415.
2023-11-13 00:24:54 -05:00
Luke Parker
351436a258
Dockerfile Parts (#428)
* De-duplicate Dockerfiles by using a bash file to concatenate common parts

Resolves #375.

Dockerfiles are still committed to the repo to avoid a dependency on bash.

* Add a CI job to confirm the committed dockerfiles are the currently generated ones

* Create dedicated Dockerfiles per processor network

Ensures the compromising of network-specific dependencies doesn't lead to a
compromise of the build process for all processors.

* Dockerfile corrections

* Correct call to build processor Docker image in tests/processor
2023-11-12 23:55:15 -05:00
David Bell
c328e5ea68
Convert coordinator/tributary/nonce_decider to use create_db macro (#423)
* chore: convert nonce_deicer to use create_db macro

* Restore pub NonceDecider

* Remove extraneous comma

I forgot to run git commit --amend on the prior commit :/

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-12 12:04:34 -05:00
Boog900
995734c960
Monero: add more legacy verify functions (#383)
* Add v1 ring sig verifying

* allow calculating signature hash for v1 txs

* add unreduced scalar type with recovery

I have added this type for borromen sigs, the ee field can be a normal
scalar as in the verify function the ee
field is checked against a reduced scalar mean for it to verify as
correct ee must be reduced

* change block major/ minor versions to u8

this matches Monero

I have also changed a couple varint functions to accept the `VarInt`
trait

* expose `serialize_hashable` on `Block`

* add back MLSAG verifying functions

I still need to revert the commit removing support for >1 input MLSAG FULL

This adds a new rct type to separate Full and simple rct

* add back support for multiple inputs for RCT FULL

* comment `non_adjacent_form` function

also added `#[allow(clippy::needless_range_loop)]` around a loop as without a re-write satisfying clippy without it will make the function worse.

* Improve Mlsag verifying API

* fix rebase errors

* revert the changes on `reserialize_chain`
plus other misc changes

* fix no-std

* Reduce the amount of rpc calls needed for `get_block_by_number`.
This function was causing me problems, every now and then a node would return a block with a different number than requested.

* change `serialize_hashable` to give the POW hashing blob.

Monero calculates the POW hash and the block hash using *slightly* different blobs :/

* make ring_signatures public and add length check when verifying.

* Misc improvements and bug fixes

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-12 10:18:18 -05:00
Luke Parker
54f1929078
Route blame between Processor and Coordinator (#427)
* Have processor report errors during the DKG to the coordinator

* Add RemoveParticipant, InvalidDkgShare to coordinator

* Route DKG blame around coordinator

* Allow public construction of AdditionalBlameMachine

Necessary for upcoming work on handling DKG blame in the processor and
coordinator.

Additionally fixes a publicly reachable panic when commitments parsed with one
ThresholdParams are used in a machine using another set of ThresholdParams.

Renames InvalidProofOfKnowledge to InvalidCommitments.

* Remove unused error from dleq

* Implement support for VerifyBlame in the processor

* Have coordinator send the processor share message relevant to Blame

* Remove desync between processors reporting InvalidShare and ones reporting GeneratedKeyPair

* Route blame on sign between processor and coordinator

Doesn't yet act on it in coordinator.

* Move txn usage as needed for stable Rust to build

* Correct InvalidDkgShare serialization
2023-11-12 07:24:41 -05:00
akildemir
d015ee96a3
Dex improvements (#422)
* remove dex traits&balance types

* remove liq tokens pallet in favor of coins-pallet instance

* fix tests & benchmarks

* remove liquidity tokens trait

* fix CI

* fix pr comments

* Slight renamings

* Add burn_with_instruction as a negative to LiquidityTokens CallFilter

* Remove use of One, Zero, Saturating taits in dex pallet

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-12 06:37:31 -05:00
Luke Parker
a43815f101
Restore Foundry to a test dependency via direct usage of solc 2023-11-12 04:34:45 -05:00
Luke Parker
7f1732c8c0
cargo update to snow 0.9.4 2023-11-12 00:40:32 -05:00
Luke Parker
ed2445390f
Replace post-detection of if a Plan is forwarded by noting if it's from the scanner 2023-11-09 14:54:38 -05:00
Luke Parker
52a0c56016
Rename Network::address to Network::external_address
Improves clarity since we now have 4 addresses.
2023-11-09 14:31:46 -05:00