Commit graph

1530 commits

Author SHA1 Message Date
Luke Parker
797604ad73
Replace usage of io::Error::new(io::ErrorKind::Other, with io::Error::other
Newly possible with Rust 1.74.
2023-11-19 18:31:37 -05:00
Luke Parker
05b975dff9
Rust 1.74
Adds a Rust toolchain file to be less disruptive to developers who don't keep
their toolchain synchronized (by now having rustup automatically synchronize).

Hopefully helps resolve how +nightly clippy may pass for the coordinator, yet
building would fail due to stable's (hopefully prior?) failure to model some
async functions re: Send/Sync.

Also adds rust-src as a component in preparation of
https://github.com/paritytech/polkadot-sdk/pull/2217
2023-11-19 17:47:46 -05:00
Luke Parker
74a8df4c7b
Add a new primitive of a DB-backed channel
The coordinator already had one of these, albeit implemented much worse than
the one now properly introduced. It had to either be sending or receiving,
whereas the new one can do both at the same time.

This replaces said instance and enables pleasant patterns when implementing the
processor/coordinator.
2023-11-19 02:05:01 -05:00
Luke Parker
bd14db9f76
Version pin foundry
Fixes #116 and #441.
2023-11-19 01:21:43 -05:00
Luke Parker
25c02c1311
Ban Serai from setting keys
They were already banned form publishing `Batch`s, yet the ability to set keys
enabled changing how certain functionality in the coordinator operated.
Removing this malleability ensures operation as expected.
2023-11-18 21:28:16 -05:00
Luke Parker
b018fc432c
Add deletion to create_db 2023-11-18 20:56:58 -05:00
Luke Parker
6e4ecbc90c
Support trailing commas in create_db 2023-11-18 20:54:37 -05:00
Luke Parker
be48dcc4a4
Use multiple LibP2P topics
We still need a peering protocol... Hopefully, we can read peers off of the
Substrate node's DHT.
2023-11-18 20:37:55 -05:00
Luke Parker
25066437da
Attempt to resolve #434
Worsens the coordinator tests' ensuring of the validity of the cosigning
protocol, yet should actually properly model it.
2023-11-18 00:12:36 -05:00
Luke Parker
db49a63c2b
Bump zeroize to appease cargo deny 2023-11-16 14:18:00 -05:00
Luke Parker
ee50f584aa
Add saner log statements to the coordinator
Disables trace on every single P2P message.

Logs a short-form of each transaction.
2023-11-16 13:37:39 -05:00
Luke Parker
14f3f330db
Remove deadlock in cosign_evaluator 2023-11-16 13:32:15 -05:00
Luke Parker
30a77d863f
Include non-JSON response from Monero node in error 2023-11-15 22:57:37 -05:00
Luke Parker
a03a1edbff
Remove needless transposition in coordinator 2023-11-15 22:49:58 -05:00
Luke Parker
c03afbe03e
Downgrade RustCrypto packages which violate Rust's interpretation of semver 2023-11-15 21:24:52 -05:00
Luke Parker
af611a39bf
Restore runtime feature to hyper 2023-11-15 20:40:24 -05:00
Luke Parker
0d080e6d34
Remove unused dependency from dex-pallet 2023-11-15 20:24:33 -05:00
Luke Parker
369af0fab5
\#339 addendum 2023-11-15 20:23:19 -05:00
Luke Parker
d25e3d86a2
Make TLS an optional feature of simple-request
Removes 14 crates from the tree when compiling the message-queue client.

Also performs a non-intrusive cargo update.
2023-11-15 17:24:11 -05:00
Luke Parker
96f1d26f7a
Add a cosigning protocol to ensure finalizations are unique (#433)
* Add a function to deterministically decide which Serai blocks should be co-signed

Has a 5 minute latency between co-signs, also used as the maximal latency
before a co-sign is started.

* Get all active tributaries we're in at a specific block

* Add and route CosignSubstrateBlock, a new provided TX

* Split queued cosigns per network

* Rename BatchSignId to SubstrateSignId

* Add SubstrateSignableId, a meta-type for either Batch or Block, and modularize around it

* Handle the CosignSubstrateBlock provided TX

* Revert substrate_signer.rs to develop (and patch to still work)

Due to SubstrateSigner moving when the prior multisig closes, yet cosigning
occurring with the most recent key, a single SubstrateSigner can be reused.
We could manage multiple SubstrateSigners, yet considering the much lower
specifications for cosigning, I'd rather treat it distinctly.

* Route cosigning through the processor

* Add note to rename SubstrateSigner post-PR

I don't want to do so now in order to preserve the diff's clarity.

* Implement cosign evaluation into the coordinator

* Get tests to compile

* Bug fixes, mark blocks without cosigners available as cosigned

* Correct the ID Batch preprocesses are saved under, add log statements

* Create a dedicated function to handle cosigns

* Correct the flow around Batch verification/queueing

Verifying `Batch`s could stall when a `Batch` was signed before its
predecessors/before the block it's contained in was cosigned (the latter being
inevitable as we can't sign a block containing a signed batch before signing
the batch).

Now, Batch verification happens on a distinct async task in order to not block
the handling of processor messages. This task is the sole caller of verify in
order to ensure last_verified_batch isn't unexpectedly mutated.

When the processor message handler needs to access it, or needs to queue a
Batch, it associates the DB TXN with a lock preventing the other task from
doing so.

This lock, as currently implemented, is a poor and inefficient design. It
should be modified to the pattern used for cosign management. Additionally, a
new primitive of a DB-backed channel may be immensely valuable.

Fixes a standing potential deadlock and a deadlock introduced with the
cosigning protocol.

* Working full-stack tests

After the last commit, this only required extending a timeout.

* Replace "co-sign" with "cosign" to make finding text easier

* Update the coordinator tests to support cosigning

* Inline prior_batch calculation to prevent panic on rotation

Noticed when doing a final review of the branch.
2023-11-15 16:57:21 -05:00
Luke Parker
79e4cce2f6
Explicitly set features in modular-frost 2023-11-13 05:31:47 -05:00
Luke Parker
0c341e3546
Fix no-std builds 2023-11-13 05:19:53 -05:00
Luke Parker
9f0790fb83
Remove RecommendedTranscript from DKG MuSig
Resolves #391.

Given this code already wasn't modular/composable, this should be overall
equivalent regarding functionality and security. It's much less opinionated
though and has fewer dependencies.
2023-11-13 05:11:40 -05:00
Luke Parker
bb8e034e68
Test historic start times in tendermint-machine
Closes https://github.com/serai-dex/serai/issues/342.

Under ideal network conditions, this is fine. While I won't claim ideal network
conditions will occur IRL, b0fcdd3367 has the
Tributary rebroadcast messages and should brute-force its way into a
functioning system.
2023-11-13 00:43:35 -05:00
Luke Parker
3f7bdaa64b
Make the output distribution cache only available under a feature
Enables a mode with reduced memory usage *and* increased safety given current
unsafety of the cache.

Relevant to https://github.com/serai-dex/serai/issues/415.
2023-11-13 00:24:54 -05:00
Luke Parker
351436a258
Dockerfile Parts (#428)
* De-duplicate Dockerfiles by using a bash file to concatenate common parts

Resolves #375.

Dockerfiles are still committed to the repo to avoid a dependency on bash.

* Add a CI job to confirm the committed dockerfiles are the currently generated ones

* Create dedicated Dockerfiles per processor network

Ensures the compromising of network-specific dependencies doesn't lead to a
compromise of the build process for all processors.

* Dockerfile corrections

* Correct call to build processor Docker image in tests/processor
2023-11-12 23:55:15 -05:00
David Bell
c328e5ea68
Convert coordinator/tributary/nonce_decider to use create_db macro (#423)
* chore: convert nonce_deicer to use create_db macro

* Restore pub NonceDecider

* Remove extraneous comma

I forgot to run git commit --amend on the prior commit :/

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-12 12:04:34 -05:00
Boog900
995734c960
Monero: add more legacy verify functions (#383)
* Add v1 ring sig verifying

* allow calculating signature hash for v1 txs

* add unreduced scalar type with recovery

I have added this type for borromen sigs, the ee field can be a normal
scalar as in the verify function the ee
field is checked against a reduced scalar mean for it to verify as
correct ee must be reduced

* change block major/ minor versions to u8

this matches Monero

I have also changed a couple varint functions to accept the `VarInt`
trait

* expose `serialize_hashable` on `Block`

* add back MLSAG verifying functions

I still need to revert the commit removing support for >1 input MLSAG FULL

This adds a new rct type to separate Full and simple rct

* add back support for multiple inputs for RCT FULL

* comment `non_adjacent_form` function

also added `#[allow(clippy::needless_range_loop)]` around a loop as without a re-write satisfying clippy without it will make the function worse.

* Improve Mlsag verifying API

* fix rebase errors

* revert the changes on `reserialize_chain`
plus other misc changes

* fix no-std

* Reduce the amount of rpc calls needed for `get_block_by_number`.
This function was causing me problems, every now and then a node would return a block with a different number than requested.

* change `serialize_hashable` to give the POW hashing blob.

Monero calculates the POW hash and the block hash using *slightly* different blobs :/

* make ring_signatures public and add length check when verifying.

* Misc improvements and bug fixes

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-12 10:18:18 -05:00
Luke Parker
54f1929078
Route blame between Processor and Coordinator (#427)
* Have processor report errors during the DKG to the coordinator

* Add RemoveParticipant, InvalidDkgShare to coordinator

* Route DKG blame around coordinator

* Allow public construction of AdditionalBlameMachine

Necessary for upcoming work on handling DKG blame in the processor and
coordinator.

Additionally fixes a publicly reachable panic when commitments parsed with one
ThresholdParams are used in a machine using another set of ThresholdParams.

Renames InvalidProofOfKnowledge to InvalidCommitments.

* Remove unused error from dleq

* Implement support for VerifyBlame in the processor

* Have coordinator send the processor share message relevant to Blame

* Remove desync between processors reporting InvalidShare and ones reporting GeneratedKeyPair

* Route blame on sign between processor and coordinator

Doesn't yet act on it in coordinator.

* Move txn usage as needed for stable Rust to build

* Correct InvalidDkgShare serialization
2023-11-12 07:24:41 -05:00
akildemir
d015ee96a3
Dex improvements (#422)
* remove dex traits&balance types

* remove liq tokens pallet in favor of coins-pallet instance

* fix tests & benchmarks

* remove liquidity tokens trait

* fix CI

* fix pr comments

* Slight renamings

* Add burn_with_instruction as a negative to LiquidityTokens CallFilter

* Remove use of One, Zero, Saturating taits in dex pallet

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-12 06:37:31 -05:00
Luke Parker
a43815f101
Restore Foundry to a test dependency via direct usage of solc 2023-11-12 04:34:45 -05:00
Luke Parker
7f1732c8c0
cargo update to snow 0.9.4 2023-11-12 00:40:32 -05:00
Luke Parker
ed2445390f
Replace post-detection of if a Plan is forwarded by noting if it's from the scanner 2023-11-09 14:54:38 -05:00
Luke Parker
52a0c56016
Rename Network::address to Network::external_address
Improves clarity since we now have 4 addresses.
2023-11-09 14:31:46 -05:00
Luke Parker
42e8f2c8d8
Add OutputType::Forwarded to ensure a user's transfer in isn't misclassified
If a user transferred in without an InInstruction, and the amount exactly
matched a forwarded output, the user's output would fulfill the
forwarding. Then the forwarded output would come along, have no InInstruction,
and be refunded (to the prior multisig) when the user should've been refunded.

Adding this new address type resolves such concerns.
2023-11-09 14:24:13 -05:00
Luke Parker
b51204a4eb
Replace usage of ethers-signers with 11 lines of ECDSA code 2023-11-09 13:22:43 -05:00
Luke Parker
ec51fa233a
Document an accepted false positive 2023-11-09 12:41:15 -05:00
Luke Parker
ce4091695f
Document a choice of variable name 2023-11-09 12:38:06 -05:00
Luke Parker
24919cfc54
Resolve race condition regarding when forwarded output is set
The higher-level scanner code in multisigs/mod.rs now creates a series of plans
with limited context. These include forwarding and refunding plans, moving all
handling of forwarding flags on the scanner's clock and therefore safe.

Also simplifies the refunding a decent bit.
2023-11-09 12:37:07 -05:00
Luke Parker
bf41009c5a
Document critical race condition due to two distinct clocks operating over the same data 2023-11-09 08:41:22 -05:00
Luke Parker
e8e9e212df
Move additional functions which retry until success into Network trait 2023-11-09 07:16:15 -05:00
Luke Parker
19187d2c30
Implement calculation of monotonic network times for Bitcoin and Monero 2023-11-09 07:02:52 -05:00
Luke Parker
43ae6794db
Remove invalid TODOs from processor signers 2023-11-09 03:53:30 -05:00
Luke Parker
978134a9d1
Remove events from SubstrateSigner
Same vibes as prior commit.
2023-11-09 01:56:09 -05:00
Luke Parker
2eb155753a
Remove the Signer events pseudo-channel for a returned message
Also replaces SignerEvent with usage of ProcessorMessage directly.
2023-11-09 01:26:30 -05:00
Luke Parker
7d72e224f0
Remove Output::amount and move Payment from Amount to Balance
This code is still largely designed around the idea a payment for a network is
fungible with any other, which isn't true. This starts moving past that.

Asserts are added to ensure the integrity of coin to the scheduler (which is
now per key per coin, not per key alone) and in Bitcoin/Monero prepare_send.
2023-11-08 23:33:25 -05:00
Luke Parker
ffedba7a05
Update processor tests to refund logic 2023-11-08 21:59:11 -05:00
Luke Parker
06e627a562
Support refunds as possible for invalidly received outputs on Serai 2023-11-08 11:26:28 -05:00
Luke Parker
11f66c741d
Remove ethers-middleware 2023-11-08 08:19:12 -05:00
Luke Parker
a0a2ef22e4
Remove ethers-solc
ethers-solc was used for a type (now manually specified) and to call out to
solc. Since Foundry was already a documented dependency, a call to it now
handles building.

Removing this single crate removes a total of 17 crates from our dependency
tree. While these may still be around due to Foundry, they at least may not
be.

Further work to remove the requirement on Foundry for solc alone would be
appreciated.
2023-11-08 06:25:35 -05:00