Commit graph

269 commits

Author SHA1 Message Date
Luke Parker
e2901cab06
Revert round-advance on TendermintMachine::new if local clock is ahead of block start
It was improperly implemented, as it assumed rounds had a constant time
interval, which they do not. It also is against the spec and was meant to
absolve us of issues with poor performance when post-starting blockchains. The
new, and much more proper, workaround for the latter is a 120-second delay
between the Substrate time and the Tributary start time.
2023-08-13 04:35:46 -04:00
Luke Parker
13a8b0afc1
Add panic-handlers which exit on any panic
By default, tokio-spawned worker panics will only kill the task, not the
program. Due to our extensive use of panicking on invariants, we should ensure
the program exits.
2023-08-13 04:30:49 -04:00
Luke Parker
7e71450dc4
Bug fixes and log statements
Also shims next nonce code with a fine-for-now piece of code which is unviable
in production, yet should survive testnet.
2023-08-13 04:03:59 -04:00
Luke Parker
f6f945e747
Add a LibP2P instantiation to coordinator
It's largely unoptimized, and not yet exclusive to validators, yet has basic
sanity (using message content for ID instead of sender + index).

Fixes bugs as found. Notably, we used a time in milliseconds where the
Tributary expected  seconds.

Also has Tributary::new jump to the presumed round number. This reduces slashes
when starting new chains (whose times will be before the current time) and was
the only way I was able to observe successful confirmations given current
surrounding infrastructure.
2023-08-08 15:12:47 -04:00
Luke Parker
cee788eac3
Test the Coordinator emits KeyGen
Mainly just a test that the full stack is properly set up and we've hit basic
functioning for further testing.
2023-08-06 12:38:44 -04:00
Luke Parker
aab8a417db
Have the Coordinator scan the Substrate genesis block
Also adds a workflow for running tests/coordinator.
2023-08-02 12:18:50 -04:00
Luke Parker
d5c787fea2
Add initial coordinator e2e tests 2023-08-01 19:00:48 -04:00
Luke Parker
53d86e2a29
Latest clippy 2023-08-01 02:49:31 -04:00
Luke Parker
3c38a0ec11
cargo +nightly fmt 2023-08-01 00:47:36 -04:00
Luke Parker
9f143a9742
Replace "coin" with "network"
The Processor's coins folder referred to the networks it could process, as did
its Coin trait. This, and other similar cases throughout the codebase, have now
been corrected.

Also corrects dated documentation for a key pair is confirmed under the
validator-sets pallet.
2023-07-30 16:11:30 -04:00
Luke Parker
32435d8a4c
Consolidate RockDB code
Moves explicitly to zstd. RockDB recommends zstd, or at least lz4 over snappy,
and this minimizes which dependencies we pull in.
2023-07-25 21:43:27 -04:00
Luke Parker
624fb2781d
Update how RPCs are handled
The processor now takes three vars and joins them itself. message-queue uses a
single argument, with defaults, as it's a service we control.
2023-07-21 14:01:42 -04:00
Luke Parker
a7c9c1ef55
Integrate coordinator with MessageQueue and RocksDB
Also resolves a couple TODOs.
2023-07-18 01:53:51 -04:00
Luke Parker
807ec30762
Update the flow for completed signing processes
Now, an on-chain transaction exists. This resolves some ambiguities and
provides greater coordination.
2023-07-14 14:05:12 -04:00
Luke Parker
f6a497f3ac
Slight terminology correction in sync test
Also correct a mistake from merging the most recent polkadot version.
2023-06-28 15:20:50 -04:00
akildemir
790fe7ee23 fix tributary sync test 2023-06-28 15:01:55 -04:00
Luke Parker
dfa3106a38
Fix incorrect sig_hash generation
sig_hash was used as a challenge. challenges should be of the form H(R, A, m).
These sig hashes were solely H(A, m), allowing trivial forgeries.
2023-06-08 06:38:25 -04:00
Luke Parker
40b2920412
Remove signed Substrate TXs from Coordinator 2023-05-13 22:43:13 -04:00
Luke Parker
89974c529a
Correct 2/3rds definitions throughout the codebase
The prior formula failed for some values, such as 20.
20 / 3 = 6, * 2 = 12, + 1 = 13. 13 is 65%, not >= 67.
2023-05-10 06:29:21 -04:00
Luke Parker
f55e9b40e6
Have coordinator publish batches to Substrate 2023-05-10 01:46:20 -04:00
Luke Parker
a70df6a449
Remove TODO about code de-duplication
It's infeasible to write a macro/function there. Does add a type alias which
makes things cleaner.
2023-05-10 01:19:01 -04:00
Luke Parker
168f2899f0
Create a vote transaction upon GeneratedKeyPair 2023-05-10 00:46:51 -04:00
Luke Parker
c95bdb6752
Properly get genesis for a Processor message 2023-05-09 23:51:05 -04:00
Luke Parker
7b7ddbdd97
Move the coordinator to a n-processor design 2023-05-09 23:44:41 -04:00
Luke Parker
9175383e89
Spawn a new async task for each block message
This probably should be done with n-long lived tasks, one per Tributary. While
this may not be suitably performant long-term (potential DoS vector), this at
least resolves the halting concerns.
2023-05-09 17:05:33 -04:00
Luke Parker
964fdee175
Publish ExternablBlock/SubstrateBlock, delay *Preprocess until ID acknowledged
Adds a channel for the Tributary scanner to communicate when an ID has been
acknowledged.
2023-05-08 22:20:51 -04:00
Luke Parker
6032af6692
Have Coordinator MainDb take a mutable borrow 2023-04-26 00:10:06 -04:00
Luke Parker
7824b6cb8b
Document the processor/tributary/coordinator/serai flow 2023-04-25 15:05:58 -04:00
Luke Parker
78d5372fb7
Initial code to handle messages from processors 2023-04-25 03:14:42 -04:00
Luke Parker
e74b4ab94f
Add a TributaryReader which doesn't require a borrow to operate
Reduces lock contention.

Additionally changes block_key to include the genesis. While not technically
needed, the lack of genesis introduced a side effect where any Tributary on the
the database could return the block of any other Tributary. While that wasn't a
security issue, returning it suggested it was on-chain when it wasn't. This may
have been usable to create issues.
2023-04-24 07:02:00 -04:00
Luke Parker
2feebe536e
Test handle_p2p and Tributary syncing
Includes bug fixes.
2023-04-24 03:30:19 -04:00
Luke Parker
14388e746c
Implement Tributary syncing
Also adds a forwards-lookup to the Tributary blockchain.
2023-04-24 00:53:18 -04:00
Luke Parker
215155f84b
Remove reliance on a blockchain read lock from block/commit 2023-04-23 23:51:10 -04:00
Luke Parker
c476f9b640
Break coordinator main into multiple functions
Also moves from std::sync::RwLock to tokio::sync::RwLock to prevent wasting
cycles on spinning.
2023-04-23 23:15:15 -04:00
Luke Parker
05b1fc5f05
Send a heartbeat message when a Tributary falls behind 2023-04-23 18:55:43 -04:00
Luke Parker
72633d6421
Clarify Arc RwLocks and sleeps in coordinator 2023-04-23 18:29:50 -04:00
Luke Parker
ad5522d854
Start handling P2P messages
This defines the tart of a very complex series of locks I'm really unhappy
with. At the same time, there's not immediately a better solution. This also
should work without issue.
2023-04-23 17:01:30 -04:00
Luke Parker
f2d9d70068
Reload Tributaries
add_active_tributary writes the spec to disk before it returns, so even if the
VecDeque it pushes to isn't popped, the tributary will still be loaded on boot.
2023-04-23 04:31:00 -04:00
Luke Parker
2b09309adc
Handle adding new Tributaries
Removes last_block as an argument from Tendermint. It now loads from the DB as
needed. While slightly less performant, it's easiest and should be fine.
2023-04-23 03:51:26 -04:00
Luke Parker
bf9ec410db
Additionally test DKGShares 2023-04-23 02:18:46 -04:00
Luke Parker
e0dc5d29ad
Tributary test wait_for_tx_inclusion function 2023-04-23 01:52:19 -04:00
Luke Parker
710e6e5217
Add Transaction::sign.
While I don't love the introduction of empty_signed, it's practically fine.
2023-04-23 01:25:45 -04:00
Luke Parker
3f6565588f
Test handling of DKG commitments transactions 2023-04-23 01:00:46 -04:00
Luke Parker
af84b7f707
Add a test for Tributary
Further fleshes out the Tributary testing code.
2023-04-22 22:28:20 -04:00
Luke Parker
8c74576cf0
Add a test to the coordinator for running a Tributary
Impls a LocalP2p for testing.

Moves rebroadcasting into Tendermint, since it's what knows if a message is
fully valid + original.

Removes TributarySpec::validators() HashMap, as its non-determinism caused
different instances to have different round robin schedules. It was already
prior moved to a Vec for this issue, so I'm unsure why this remnant existed.

Also renames the GH no-std workflow from the prior commit.
2023-04-22 10:49:52 -04:00
Luke Parker
09c3c9cc9e
Route the SubstrateBlock message, which is the last Tributary transaction type 2023-04-20 15:37:22 -04:00
Luke Parker
70d866af6a
ExternalBlock handler 2023-04-20 14:51:33 -04:00
Luke Parker
f99a91b34d
Slash on unrecognized ID 2023-04-20 14:33:19 -04:00
Luke Parker
294ad08e00
Add support for multiple orderings in Provided
Necessary as our Tributary chains needed to agree when a Serai block has
occurred, and when a Monero block has occurred. Since those could happen at the
same time, some validators may put SeraiBlock before ExternalBlock and vice
versa, causing a chain halt. Now they can have distinct ordering queues.
2023-04-20 07:32:40 -04:00
Luke Parker
a26ca1a92f
Split FinalizedBlock into ExternalBlock and SeraiBlock
Also re-arranges their orders.
2023-04-20 06:59:42 -04:00
Luke Parker
9c2a44f9df
Apply DKG TX handling code to all sign TXs
The existing code was almost entirely applicable. It just needed to be scoped
with an ID. While the handle function is now a bit convoluted, I don't see a
better option.
2023-04-20 06:27:05 -04:00
Luke Parker
8041a0d845
Initial Tributary handling 2023-04-20 05:05:17 -04:00
Luke Parker
9e1f3fc85c
Make MainDB into SubstrateDB 2023-04-20 05:04:08 -04:00
Luke Parker
396e5322b4
Code a method to determine the activation block before any block has consensus
[0; 32] is a magic for no block has been set yet due to this being the first
key pair. If [0; 32] is the latest finalized block, the processor determines
an activation block based on timestamps.

This doesn't use an Option for ergonomic reasons.
2023-04-18 03:04:52 -04:00
Luke Parker
9da0eb69c7
Use an enum for Coin/NetworkId
It originally wasn't an enum so software which had yet to update before an
integration wouldn't error (as now enums are strictly typed). The strict typing
is preferable though.
2023-04-18 02:04:47 -04:00
Luke Parker
6f3b5f4535
Tweak ConfirmKeyPair to alleviate database requirements of coordinator 2023-04-18 01:09:22 -04:00
Luke Parker
5a499de4ca
Remove BatchSigned
SubstrateBlock's provision of the most recently acknowledged block has
equivalent information with the same latency. Accordingly, there's no need for
it.
2023-04-17 20:19:15 -04:00
Luke Parker
e26b861d25
Move ConfirmKeyPair from key_gen to substrate
Clarifies the emitter and accordingly why its mutations are justified.
2023-04-17 19:40:17 -04:00
Luke Parker
92a868e574
Add a processor API to the coordinator 2023-04-17 02:10:33 -04:00
Luke Parker
595cd6d404
Rename transaction file to tributary, add function for genesis 2023-04-17 02:09:29 -04:00
Luke Parker
4d43c04916
Clean up the Substrate block processing code 2023-04-17 00:50:56 -04:00
Luke Parker
2604746586
Fill out code for the rest of the Substrate events 2023-04-16 03:18:52 -04:00
Luke Parker
79655672ef
Make progres on handling NewSet events
Further bones out the coordinator.
2023-04-16 00:51:56 -04:00
Luke Parker
eafd054296
Start defining the coordinator 2023-04-15 17:38:47 -04:00
Luke Parker
d5a12a9b97
Make TransactionKind have a reference to Signed
Broken commit due to partial staging of one file.
2023-04-12 09:38:20 -04:00
Luke Parker
402a7be966
Block contructor and tests 2023-04-11 20:24:27 -04:00
Luke Parker
119d25be49
Clarify transaction length sizing 2023-04-11 19:18:26 -04:00
Luke Parker
2cfee536f6
Define all coordinator transaction types 2023-04-11 19:04:53 -04:00
Luke Parker
de52c4db7f
Add empty coordinator 2023-04-11 09:21:35 -04:00