Commit graph

1262 commits

Author SHA1 Message Date
Luke Parker
06a6cd29b0
Set nodelay on coordinator's P2P sockets 2023-09-06 22:57:33 -04:00
Luke Parker
2472ec7ba8
Don't attempt parsing truncated InInstructions 2023-09-02 17:18:04 -04:00
Luke Parker
69c3fad7ce
cargo fmt 2023-09-02 16:32:42 -04:00
Luke Parker
bd9a05feef
Document UTXO solvency modeling 2023-09-02 16:11:01 -04:00
Luke Parker
7d8e08d5b4
Use scale instead of bincode throughout processor-messages/processor DB
scale is canonical, bincode is not.
2023-09-02 07:54:09 -04:00
Luke Parker
f7e49e1f90
Update Rust nightly
Supersedes #368.

Adds exceptions for unwrap_or_default due to preference against Default's
ambiguity.
2023-09-02 01:24:09 -04:00
Luke Parker
cd4c3a6c88
Correct publication of Completed Tributary TXs 2023-09-02 00:50:54 -04:00
Luke Parker
fddc605c65
Define a proper Topic (Zone + ID)
Removes the inefficiency of recognized_topic for attempt returning None if the
topic isn't recognized.
2023-09-01 01:21:15 -04:00
Luke Parker
2ad6b38be9
Prefix root keys in coordinator with "coordinator" to prevent conflicts with tributary 2023-09-01 01:00:24 -04:00
Luke Parker
fda90e23c9
Reduce and clarify data flow in Tributary scanner 2023-09-01 00:59:10 -04:00
Luke Parker
3f3f6b2d0c
Properly route attempt around DkgConfirmer 2023-09-01 00:16:43 -04:00
Luke Parker
fa8ff62b09
Remove sender_i from DkgShares
It was a piece of duplicated data used to achieve context-less
de)serialization. This new Vec code is a bit tricker to first read, yet overall
clean and removes a potential fault.

Saves 2 bytes from DkgShares messages.
2023-09-01 00:03:56 -04:00
Luke Parker
5113ab9ec4
Move SignCompleted to Unsigned to cause de-duplication amongst honest validators 2023-08-31 23:39:36 -04:00
Luke Parker
9b7cb688ed
Have Batch contain Block and batch ID, ensuring eclipsed validators don't publish invalid shares
See prior commit message for more info.

With the plan for the batch sign ID to be just 5 bytes (potentially 4), this
does incur a +5 bytes cost compared to the ExternalBlock system *even in the
standard case*. The simplicity remains preferred at this time.
2023-08-31 23:04:39 -04:00
Luke Parker
9a5f8fc5dd
Replace ExternalBlock with Batch
The initial TODO was simply to use one ExternalBlock per all batches in the
block. This would require publishing ExternalBlock after the last batch,
requiring knowing the last batch. While we could add such a pipeline, it'd
require:

1) Initial preprocesses using a distinct message from BatchPreprocess
2) An additional message sent after all BatchPreprocess are sent

Unfortunately, both would require tweaks to the SubstrateSigner which aren't
worth the complexity compared to the solution here, at least, not at this time.

While this will cause, if a Tributary is signing a block whose total batch data
exceeds 25 kB, to use multiple transactions which could be optimized out by
'better' local data pipelining, that's an extreme edge case. Given the temporal
nature of each Tributary, it's also an acceptable edge.

This does no longer achieve synchrony over external blocks accordingly. While
signed batches have synchrony, as they embed their block hash, batches being
signed don't have cryptographic synchrony on their contents. This means
validators who are eclipsed may produce invalid shares, as they sign a
different batch. This will be introduced in a follow-up commit.
2023-08-31 23:00:25 -04:00
Luke Parker
2dc35193c9
Handle batch n+1 being signed before batch n is 2023-08-31 22:09:34 -04:00
Luke Parker
9bf24480f4
Spawn an async test per P2P message to try and resolve latency issues 2023-08-31 02:35:50 -04:00
Luke Parker
3af9dc5d6f
Tweak Heartbeat configuration so LibP2P can be expected to deliver messages within latency window 2023-08-31 01:33:52 -04:00
Luke Parker
148bc380fe
Add assert on edge case requiring a validator with 34% and a broken invariant 2023-08-31 01:08:40 -04:00
Luke Parker
8dad62f300
Fix panic when post-verifying Precommits in log
Notes an edge case which enables invalid commit production.
2023-08-31 01:02:57 -04:00
Luke Parker
1e79de87e8
Remove contention between LibP2p spawned task and consumers via channels 2023-08-30 23:31:09 -04:00
Luke Parker
2f57a69cb6
Define BLOCK_PROCESSING_TIME, LATENCY_TIME in ms
Updates Tributary values to allow 999ms for block processing (from 2s) and
1667ms for latency (up from 1s).

The intent is to resolve #365. I don't know if this will, but it increases the
chances of success and these values should be fine in prod since Tributary is a
post-execution chain (making block procesisng time minimal).

Does embed the dagger of N::block_time() panicking if the block time in ms
doesn't cleanly divide by 1000.
2023-08-30 22:58:42 -04:00
Luke Parker
493a222421
Use a timeout in case the JSON-RPC notifications have unexpected behavior 2023-08-30 17:57:33 -04:00
Luke Parker
d5a19eca8c
Add a notification system for finalizations to serai-client, use in coordinator 2023-08-30 17:25:04 -04:00
Luke Parker
e9fca37181
Intent based de-duplication in MessageQueue 2023-08-29 17:05:01 -04:00
Luke Parker
83c25eff03
Remove no longer necessary async from monero SignatableTransaction::sign 2023-08-29 16:20:21 -04:00
Luke Parker
285422f71a
Add a full-stack mint and burn test for Bitcoin and Monero
Fixes where ram_scanned is updated in processor. The prior version, while safe,
would redo massive amounts of work during periods of inactivity. It also hit an
undocumented invariant where get_eventuality_completions assumes new blocks,
yet redone work wouldn't have new blocks.

Modifies Monero's generate_blocks to return the hashes of the generated blocks.
2023-08-28 21:17:22 -04:00
Luke Parker
1838c37ecf
Full stack test framework 2023-08-27 18:37:12 -04:00
Luke Parker
a3649b2062
Move where we trigger KeyGen to avoid a race condition
We only expect processor messages when we have the relevant Tributary. We
queued Tributary creation, yet then kicked off processor messages. We need to
wait until the Tributary is actually created to kick off processor messages.
2023-08-27 18:23:45 -04:00
Luke Parker
1bd14163a0
Log all output to console when running in CI
Attempts to debug https://github.com/serai-dex/serai/actions/runs/5992819682/job/16252622433,
which seems to be a legitimate test failure.
2023-08-27 17:51:36 -04:00
Luke Parker
89a6ee9290
Silence warning when building in release 2023-08-27 15:39:09 -04:00
Luke Parker
a66994aade
Use FCMP implementation of BP+ in monero-serai (#344)
* Add in an implementation of BP+ based off the paper, intended for clarity and review

This was done as part of my work on FCMPs from Monero, and is copied from https://github.com/kayabaNerve/full-chain-membership-proofs

* Remove crate structure of BP+

* Remove arithmetic circuit code

* Remove AC/VC generators code

* Remove generator transcript

Monero uses non-transcripted static generators.

* Further trimming of generators

* Remove the single range proof

It's unused by Monero and accordingly unhelpful.

* Work on getting BP+ to compile in its new env

* Correct BP+ folder name

* Further tweaks to get closer to compiling

* Remove the ScalarMatrix file

It's only used for AC proofs

* Compiles, with tests passing

* Lock BP+ to Ed25519 instead of the generic Ciphersuite

* Resolve most warnings in BP+

* Make existing bulletproofs test easier to read

* Further strip generators

* Swap G/H as Monero did

* Replace RangeCommitment with Commitment

* Hard-code BP+ h to Ed25519's generator

* Use pub(crate) for BP+, not pub

* Replace initial_transcript with hash_plus

* Rename hash_plus to initial_transcript

* Finish integrating the FCMP BP+ impl

* Move BP+ folder

* Correct no-std support

* Rename "long_n" to eta

* Add note on non-prime order dfg points
2023-08-27 15:33:17 -04:00
Luke Parker
34ffd2fa76
Run coordinator e2e tests one at a time due to usage of mdns causing cross-talk 2023-08-27 05:40:36 -04:00
Luke Parker
775353f8cd
Add testing for Completed to coordinator e2e tests
Two commits ago stubbed in support, sufficient for a honest-validator
environment.
2023-08-27 05:34:04 -04:00
Luke Parker
c9b2490ab9
Tweak tributary_test to handle a one-block variance
Prior to the previous commit, whatever async scheduling occurred caused them to
all have the same tip. Now, some are one block ahead of others. This adds
tolerance for that, as it's an acceptable variance, so long as it's solely one
block.
2023-08-27 05:28:36 -04:00
Luke Parker
2db53d5434
Use &self for handle_message and sync_block in Tributary
They used &mut self to prevent execution at the same time. This uses a lock
over the channel to achieve the same security, without requiring a lock over
the entire tributary.

This fixes post-provided Provided transactions. sync_block waited for the TX to
be provided, yet it never would as sync_block held a mutable reference over the
entire Tributary, preventing any other read/write operations of any scope.

A timeout increased (bc2f23f72b) due to this bug
not being identified has been decreased back, thankfully.

Also shims in basic support for Completed, which was the WIP before this bug
was identified.
2023-08-27 05:07:11 -04:00
Luke Parker
6268bbd7c8
Replace "connected" with "external" in Processor doc 2023-08-27 01:57:17 -04:00
Luke Parker
bc2f23f72b
Further increase GH timeout in response to https://github.com/serai-dex/serai/actions/runs/5988202109/job/16243154650
This should be egregious unless the GitHub CI is so inperformant it's breaking
Tendermint consensus's synchrony expectations, which likely points to our own
code being unviable.

This solely serves as an immediate fix to the problem, not a justification of
the unevaluated performance.
2023-08-27 01:26:42 -04:00
Luke Parker
72337b17f5
Test the Sign flow across the Coordinators 2023-08-26 21:45:53 -04:00
Luke Parker
36b193992f
Correct handling of known signers in batch test
Putting a signer first doesn't work because signers can only publish once a
supermajority sync. Now, the code uses an excluded signer (instead of an
included signer) to determine signing set.

Further simplifications are available. Also adds accurate documentation on
latency/sleep reasoning.
2023-08-26 21:38:58 -04:00
Luke Parker
37b6de9c0c
Add SRI balanace/transfer functions to serai_client 2023-08-26 21:37:54 -04:00
Luke Parker
22f3c9e58f
Stop trying to publish a Batch if another node does 2023-08-26 21:37:21 -04:00
Luke Parker
9adefa4c2c
Add code to handle a race condition around first_preprocess 2023-08-26 21:35:43 -04:00
Luke Parker
c245bcdc9b
Extend Batch test with SubstrateBlock/arb Batch signing 2023-08-25 21:37:36 -04:00
Luke Parker
f249e20028
Route KeyPair so Tributary can construct SignId as needed 2023-08-25 18:37:22 -04:00
Luke Parker
3745f8b6af
Increase GitHub CI timeouts for the coordinator tests 2023-08-25 14:09:44 -04:00
Luke Parker
ba46aa76f0
Update libp2p
Adds 17 new crates, which I'm extremely unhappy about. Unfortunately, it's
needed to resolve a security issue (RUSTSEC-2023-0052) and is inevitable.

Closes #355.
2023-08-25 00:01:58 -04:00
Luke Parker
8c1d8a2658
Only emit Preprocesses/Shares when participating 2023-08-24 23:50:19 -04:00
Luke Parker
2702384c70
Coordinator Batch signing test 2023-08-24 23:48:50 -04:00
Luke Parker
d3093c92dc
Uncontroversial cargo update 2023-08-24 22:06:08 -04:00