Commit graph

209 commits

Author SHA1 Message Date
akildemir
899a9604e1
Add Dex pallet (#407)
* Move pallet-asset-conversion

* update licensing

* initial integration

* Integrate Currency & Assets types

* integrate liquidity tokens

* fmt

* integrate dex pallet tests

* fmt

* compilation error fixes

* integrate dex benchmarks

* fmt

* cargo clippy

* replace all occurrences of "asset" with "coin"

* add the actual add liq/swap logic to in-instructions

* add client side & tests

* fix deny

* Lint and changes

- Renames InInstruction::AddLiquidity to InInstruction::SwapAndAddLiquidity
- Makes create_pool an internal function
- Makes dex-pallet exclusively create pools against a native coin
- Removes various fees
- Adds new crates to GH workflow

* Fix rebase artifacts

* Correct other rebase artifact

* Correct CI specification for liquidity-tokens

* Correct primitives' test to the standardized pallet account scheme

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-05 12:02:34 -05:00
Luke Parker
e05b77d830
Support multiple key shares per validator (#416)
* Update the coordinator to give key shares based on weight, not based on existence

Participants are now identified by their starting index. While this compiles,
the following is unimplemented:

1) A conversion for DKG `i` values. It assumes the threshold `i` values used
will be identical for the MuSig signature used to confirm the DKG.
2) Expansion from compressed values to full values before forwarding to the
processor.

* Add a fn to the DkgConfirmer to convert `i` values as needed

Also removes TODOs regarding Serai ensuring validator key uniqueness +
validity. The current infra achieves both.

* Have the Tributary DB track participation by shares, not by count

* Prevent a node from obtaining 34% of the maximum amount of key shares

This is actually mainly intended to set a bound on message sizes in the
coordinator. Message sizes are amplified by the amount of key shares held, so
setting an upper bound on said amount lets it determine constants. While that
upper bound could be 150, that'd be unreasonable and increase the potential for
DoS attacks.

* Correct the mechanism to detect if sufficient accumulation has occured

It used to check if the latest accumulation hit the required threshold. Now,
accumulations may jump past the required threshold. The required mechanism is
to check the threshold wasn't prior met and is now met.

* Finish updating the coordinator to handle a multiple key share per validator environment

* Adjust stategy re: preventing noce reuse in DKG Confirmer

* Add TODOs regarding dropped transactions, add possible TODO fix

* Update tests/coordinator

This doesn't add new multi-key-share tests, it solely updates the existing
single key-share tests to compile and run, with the necessary fixes to the
coordinator.

* Update processor key_gen to handle generating multiple key shares at once

* Update SubstrateSigner

* Update signer, clippy

* Update processor tests

* Update processor docker tests
2023-11-04 19:26:13 -04:00
Luke Parker
34bcb9eb01 bitcoin 0.31 2023-10-31 03:47:45 -04:00
Luke Parker
c4bdbdde11
dockertest 0.4 (#406)
* Updates to modern dockertest

* More updates to latest dockertest

* Update Cargo.lock to dockertest with handle restored

* clippy coordinator tests

* clippy full-stack tests

* Remove kayabaNerve branch for official repo's latest commit hash

* Update serai-client, remove reliance on the existence of a handle fn

* Don't use the hex encoding of unique_id in dockertests

Gets our hostnames just below 64 bytes, resolving test failures on at least
Debian-based systems.

* Use Network::Isolated for all dockertest instances

* Correct error from prior commit's edits
2023-10-23 06:59:38 -04:00
Luke Parker
fbf51e53ec
Resolve #327
Also runs `cargo update` and moves where we install the wasm toolchain in the
Dockerfile for better caching properties.
2023-10-23 00:45:00 -04:00
Luke Parker
e5113c333e
Move LastBatchBlock, LastBatch sets by their checks 2023-10-22 05:49:19 -04:00
Luke Parker
6068978676
Use a transaction layer when executing each InInstruction 2023-10-22 05:46:03 -04:00
Luke Parker
e7e30150f0
Don't let the Serai set publish batches 2023-10-22 05:38:44 -04:00
Luke Parker
55fe27f41a
Don't allow the Bitcoin set to mint sriETH 2023-10-22 05:37:23 -04:00
Luke Parker
d66a7ee43e
Remove the staking pallet for validator-sets alone
The staking pallet is an indirection which offered no practical benefit yet
increased the overhead of every call.
2023-10-22 04:00:42 -04:00
Luke Parker
a702d65c3d
validator-sets pallet event expansion 2023-10-22 03:28:42 -04:00
Luke Parker
52eb68677a
pallet-staking event 2023-10-22 03:19:01 -04:00
Luke Parker
d29d19bdfe
Ensure pallet-session is initialized after staking 2023-10-22 02:52:34 -04:00
Luke Parker
c3fdb9d9df
Use a comprehensive call filter in the runtime 2023-10-21 21:35:14 -04:00
Luke Parker
1bff2a0447
Add signals pallet
Resolves #353

Implements code such that:

- 80% of validators (by stake) must be in favor of a signal for the network to
  be
- 80% of networks (by stake) must be in favor of a signal for it to be locked
  in
- After a signal has been locked in for two weeks, the network halts

The intention is to:

1) Not allow validators to unilaterally declare new consensus rules.

No method of declaring new consensus rules is provided by this pallet. Solely a
way to deprecate the current rules, with a signaled for successor. All nodes
must then individually decide whether or not to download and run a new node
which has new rules, and if so, which rules.

2) Not place blobs on chain.

Even if they'd be reproducible, it's just a lot of data to chuck on the
blockchain.
2023-10-21 20:06:55 -04:00
Luke Parker
8ead7a2581
Add missing std feature flags to a couple Substrate dependencies 2023-10-20 17:53:07 -04:00
Luke Parker
3797679755
Track total allocated stake in validator-sets pallet 2023-10-20 16:58:44 -04:00
Luke Parker
d833254b84
Other clippy fixes 2023-10-19 08:01:15 -04:00
Luke Parker
a1b2bdf0a2
clippy fixes 2023-10-19 06:30:58 -04:00
Luke Parker
43841f95fc
cargo fmt 2023-10-19 06:24:52 -04:00
akildemir
fdfce9e207
Coins pallet (#399)
* initial implementation

* add function to get a balance of an account

* add support for multiple coins

* rename pallet to "coins-pallet"

* replace balances, assets and tokens pallet with coins pallet in runtime

* add total supply info

* update client side for new Coins pallet

* handle fees

* bug fixes

* Update FeeAccount test

* Fmt

* fix pr comments

* remove extraneous Imbalance type

* Minor tweaks

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-10-19 06:22:21 -04:00
Luke Parker
e4d59eeeca
Remove hashbrown from validator-sets pallet 2023-10-16 01:47:15 -04:00
Luke Parker
3b3fdd104b
Most of coordinator Tributary retiry
Adds Event::SetRetired to validator-sets.

Emit TributaryRetired.

Replaces is_active_set, which made multiple network requests, with
is_retired_tributary, a DB read.

Performs most of the removals necessary upon TributaryRetired.

Still needs to clean up the actual Tributary/Tendermint tasks.
2023-10-14 16:47:25 -04:00
Luke Parker
5c5c097da9 Tweaks for processor to work with the new serai-client 2023-10-14 15:26:36 -04:00
Luke Parker
7d4e8b59db Update dockertests to new serai-client 2023-10-14 15:26:36 -04:00
Luke Parker
cb61c9052a Reorganize serai-client
Instead of functions taking a block hash, has a scope to a block hash before
functions can be called.

Separates functions by pallets.
2023-10-14 15:26:36 -04:00
Luke Parker
f6e8bc3352
Alternate handover batch TOCTOU fix (#397)
* Revert "Correct the prior documented TOCTOU"

This reverts commit d50fe87801.

* Correct the prior documented TOCTOU

d50fe87801 edited the challenge for the Batch to
fix it. This won't produce Batch n+1 until Batch n is successfully published
and verified. It's an alternative strategy able to be reviewed, with a much
smaller impact to scope.
2023-10-13 12:14:59 -04:00
Luke Parker
d50fe87801
Correct the prior documented TOCTOU
Now, if a malicious validator set publishes a malicious `Batch` at the last
moment, it'll cause all future `Batch`s signed by the next validator set to
require a bool being set (yet they never will set it).

This will prevent the handover.

The only overhead is having two distinct `batch_message` calls on-chain.
2023-10-13 04:41:01 -04:00
Luke Parker
02edfd2935
Verify all Batchs published by the prior set
The new set publishing a `Batch` completes the handover protocol. The new set
should only publish a `Batch` once it believes the old set has completed all of
its on-external-chain activity, marking it honest and finite.

With the handover comes the acceptance of liability, hence the requirement for
all of the on-Serai-chain activity also needing verification. While most
activity would be verified in-real-time (upon ::Batch messages), the new set
will now explicitly verify the complete set of `Batch`s before beginning its
preprocess for its own `Batch` (the one accepting the handover).
2023-10-13 04:12:21 -04:00
Luke Parker
9aeece5bf6
Give one weight per key share to validators in Tributary 2023-10-13 02:29:22 -04:00
Luke Parker
bb84f7cf1d
Correct ValidatorSets genesis 2023-10-13 01:42:26 -04:00
Luke Parker
bb25baf3bc
Add logic to amortize excess key shares, correcting is_bft 2023-10-13 01:04:41 -04:00
Luke Parker
013a0cddfc
MAX_VALIDATORS_PER_SET -> MAX_KEY_SHARES_PER_SET 2023-10-13 00:50:07 -04:00
Luke Parker
ed7300b406
Explicitly provide a pre_dispatch which calls validate_unsigned
pre_dispatch is guaranteed by documentation to be called and persisted.
validate_unsigned is not, though the provided pre_dispatch does by default call
validate_unsigned. By explicitly providing our own pre_dispatch, we accomplish
the bounds we require and expect, only being invalidated on Substrate
redefining their API.

We should still test this, yet since we call retire_session in
validate_unsigned, any test of rotation will test it's being properly called.
2023-10-13 00:31:23 -04:00
Luke Parker
88b5efda99
cargo fmt 2023-10-13 00:12:10 -04:00
Luke Parker
0712e6f107
Localize stake into networks
Sets a stake requirement of 100k for Serai and Monero, as Serai doesn't have
stake requirements and Monero isn't expected to see as much
volume/institutional support as Bitcoin/Ethereum.
2023-10-13 00:04:30 -04:00
Luke Parker
6a4c57e86f
Define an array of all NetworkIds in serai_primitives 2023-10-12 23:59:21 -04:00
Luke Parker
b7746aa71d
Don't allow (de)allocations which remove fault tolerance 2023-10-12 23:47:00 -04:00
Luke Parker
8dd41ee798
Allow immediate deallocation if the decrease doesn't cross a key-share threshold 2023-10-12 23:06:20 -04:00
Luke Parker
9a1d10f4ea
Error if deallocation would remove fault tolerance 2023-10-12 23:05:29 -04:00
Luke Parker
6587590986
Grab up to 150 key shares of validators, not 150 validators 2023-10-12 22:44:10 -04:00
Luke Parker
29fcf6be4d
Support immediate deallocations for non-active validators 2023-10-12 00:51:18 -04:00
Luke Parker
108e2b57d9
Add claim_deallocation to the staking pallet 2023-10-12 00:26:35 -04:00
Luke Parker
3da5577950
Only allow deallocations after the next set after the validator's inclusion starts, plus a one session cooldown period
Part of #394.
2023-10-11 23:42:15 -04:00
Luke Parker
f692047b8b
Rename validators to select_validators 2023-10-11 17:23:09 -04:00
Luke Parker
2401266374
Replace mutate with get + set
I'm legitimately unsure why mutate doesn't work. Reading the impls, it should...
2023-10-11 02:11:44 -04:00
Luke Parker
04f9a1fa31
Correct handling of InSet's keys 2023-10-11 01:05:48 -04:00
Luke Parker
13cbc99149
Properly define the on-chain handover protocol
The new key publishing `Batch`s is more than sufficient.

Also uses the correct key to verify the published `Batch`s authenticity.
2023-10-10 23:55:59 -04:00
Luke Parker
1a0b4198ba
Correct the check for if we still need to set keys
The prior check had an edge case where once keys were pruned, it'd believe the
keys needed to be set ad infinitum.
2023-10-10 22:53:15 -04:00
Luke Parker
ab5af57dae
Fix a pair of bugs in SortedAllocations and add further documentation
We took with `amount`, not `prior`, allowing multiple presences in
SortedAllocations.

While spam is limited by the amount of nibbles in the amount, the key provides
a much larger space to abuse. Inserting a cryptographic hash prevents its use
for abuse.
2023-10-10 16:50:48 -04:00
akildemir
98190b7b83
Staking pallet (#373)
* initial staking pallet

* add staking pallet to runtime

* support session rotation for serai

* optimizations & cleaning

* fix deny

* add serai network to initial networks

* a few tweaks & comments

* fix some pr comments

* Rewrite validator-sets with logarithmic algorithms

Uses the fact the underlying DB is sorted to achieve sorting of potential
validators by stake.

Removes release of deallocated stake for now.

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-10-10 06:53:24 -04:00
Luke Parker
0eff3d9453
Add Batch messages from processor, verify Batchs published on-chain
Renames Update to SignedBatch.

Checks Batch equality via a hash of the InInstructions. That prevents needing
to keep the Batch in node state or TX introspect.
2023-09-29 03:51:01 -04:00
Luke Parker
f7e49e1f90
Update Rust nightly
Supersedes #368.

Adds exceptions for unwrap_or_default due to preference against Default's
ambiguity.
2023-09-02 01:24:09 -04:00
Luke Parker
d5a19eca8c
Add a notification system for finalizations to serai-client, use in coordinator 2023-08-30 17:25:04 -04:00
Luke Parker
285422f71a
Add a full-stack mint and burn test for Bitcoin and Monero
Fixes where ram_scanned is updated in processor. The prior version, while safe,
would redo massive amounts of work during periods of inactivity. It also hit an
undocumented invariant where get_eventuality_completions assumes new blocks,
yet redone work wouldn't have new blocks.

Modifies Monero's generate_blocks to return the hashes of the generated blocks.
2023-08-28 21:17:22 -04:00
Luke Parker
37b6de9c0c
Add SRI balanace/transfer functions to serai_client 2023-08-26 21:37:54 -04:00
Luke Parker
22f3c9e58f
Stop trying to publish a Batch if another node does 2023-08-26 21:37:21 -04:00
Luke Parker
2702384c70
Coordinator Batch signing test 2023-08-24 23:48:50 -04:00
Luke Parker
108504d6e2
Increase MAX_ADDRESS_LEN
In response to
https://gist.github.com/tevador/50160d160d24cfc6c52ae02eb3d17024?permalink_comment_id=4665372#gistcomment-4665372
2023-08-21 02:56:10 -04:00
Luke Parker
906d3b9a7c
Merge pull request #348 from serai-dex/current-crypto-crates
Current crypto crates
2023-08-21 01:24:16 -04:00
Luke Parker
34c6974311
Merge branch 'dalek-4.0' into develop 2023-08-17 02:00:36 -04:00
akildemir
e680eabb62
Improve batch handling (#316)
* restrict batch size to ~25kb

* add batch size check to node

* rate limit batches to 1 per serai block

* add support for multiple batches for block

* fix review comments

* Misc fixes

Doesn't yet update tests/processor until data flow is inspected.

* Move the block from SignId to ProcessorMessage::BatchPreprocesses

* Misc clean up

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-08-14 11:57:38 -04:00
Luke Parker
a3441a6871
Don't have publish return the 'hash' 2023-08-14 08:18:19 -04:00
Luke Parker
5e02f936e4
Perform MuSig signing of generated keys 2023-08-14 06:08:55 -04:00
Luke Parker
fa406c507f
Update crypto/ package versions
On a branch while bitcoin-serai wraps up its audit.
2023-08-08 18:19:01 -04:00
Luke Parker
f6f945e747
Add a LibP2P instantiation to coordinator
It's largely unoptimized, and not yet exclusive to validators, yet has basic
sanity (using message content for ID instead of sender + index).

Fixes bugs as found. Notably, we used a time in milliseconds where the
Tributary expected  seconds.

Also has Tributary::new jump to the presumed round number. This reduces slashes
when starting new chains (whose times will be before the current time) and was
the only way I was able to observe successful confirmations given current
surrounding infrastructure.
2023-08-08 15:12:47 -04:00
Luke Parker
0dd8aed134
Expand cluster-sm/local testnet to 4 validators for BFT where f=1 2023-08-06 13:42:16 -04:00
Luke Parker
3c38a0ec11
cargo +nightly fmt 2023-08-01 00:47:36 -04:00
Luke Parker
9f143a9742
Replace "coin" with "network"
The Processor's coins folder referred to the networks it could process, as did
its Coin trait. This, and other similar cases throughout the codebase, have now
been corrected.

Also corrects dated documentation for a key pair is confirmed under the
validator-sets pallet.
2023-07-30 16:11:30 -04:00
Luke Parker
a8c7bb96c8
Add a crate to test the runtime can be reproducibly built 2023-07-27 21:42:26 -04:00
Luke Parker
e00aa3031c
Have Shorthand::Raw contain RefundableInInstruction, not an encoded RII 2023-07-26 14:00:30 -04:00
Luke Parker
f8afb040dc
Remove ApplicationCall
We can simply inline `Dex` into the InInstruction enum.
2023-07-26 12:45:51 -04:00
Luke Parker
88a1fce15c
Test the processor's batch signing
Updates message-queue ot try recv every second, not 5.
2023-07-25 18:09:23 -04:00
Luke Parker
23e1c9769c
dalek 4.0 2023-07-23 14:32:14 -04:00
Luke Parker
37af8b51b3
Fallback to pgrep if pidof is unavailable 2023-07-21 03:37:48 -04:00
Luke Parker
d5cfb3fb25
Corrections from renaming Index to Nonce 2023-07-18 23:52:37 -04:00
Luke Parker
b1dbe1f50d
hashbrown 0.14 in validator-sets 2023-07-18 23:25:15 -04:00
Luke Parker
1b57d655ed
Update to subxt 0.29 2023-07-18 23:01:51 -04:00
Luke Parker
64402914ba
Update substrate 2023-07-18 22:30:55 -04:00
Luke Parker
d49c636f0f
Use serai- prefixes on Serai-specific packages
Fixes deny.toml, also runs a minor cargo update shrinking the tree.
2023-07-03 08:50:23 -04:00
Luke Parker
f6a497f3ac
Slight terminology correction in sync test
Also correct a mistake from merging the most recent polkadot version.
2023-06-28 15:20:50 -04:00
Luke Parker
8c020abb86
Update to substrate polkadot-v0.9.43 2023-06-28 14:57:58 -04:00
Luke Parker
40b2920412
Remove signed Substrate TXs from Coordinator 2023-05-13 22:43:13 -04:00
Luke Parker
47f8766da6
Use proper messages for ValidatorSets/InInstructions pallet
Provides a DST, and associated metadata as beneficial.

Also utilizes MuSig's context to session-bind. Since set_keys_messages also
binds to set, this is semi-redundant, yet that's appreciated.
2023-05-13 04:40:16 -04:00
Luke Parker
f069567f12
Use a MuSig signature to publish validator set key pairs to Serai
The processor/coordinator flow still has to be rewritten.
2023-05-13 02:15:41 -04:00
Luke Parker
f55e9b40e6
Have coordinator publish batches to Substrate 2023-05-10 01:46:20 -04:00
Luke Parker
168f2899f0
Create a vote transaction upon GeneratedKeyPair 2023-05-10 00:46:51 -04:00
Luke Parker
c0001f5ff2
Update to substrate polkadot-v0.9.42 2023-05-01 03:17:37 -04:00
Luke Parker
9da0eb69c7
Use an enum for Coin/NetworkId
It originally wasn't an enum so software which had yet to update before an
integration wouldn't error (as now enums are strictly typed). The strict typing
is preferable though.
2023-04-18 02:04:47 -04:00
Luke Parker
36cdf6d4bf
Have InInstructions track the latest block for a network in storage 2023-04-16 02:57:19 -04:00
Luke Parker
fa2cf03e61
Support extracting timestamps from blocks 2023-04-16 00:31:54 -04:00
Luke Parker
eafd054296
Start defining the coordinator 2023-04-15 17:38:47 -04:00
Luke Parker
51bf51ae1e
Make unsigned private due to unsafe calling potential 2023-04-15 16:37:59 -04:00
Luke Parker
28b6bc99ac
Update to the latest subxt
Writes a custom unsigned extrinic creator due to subxt having an internal error
with the scale metadata. While the code in our scope increased, it's much more
ergonomic to our usage. We may end up rewriting most of subxt, eventually.
2023-04-15 05:23:57 -04:00
Luke Parker
124b994c23
Add a NewSet event to validator-sets
Updates to the latest serai-dex/substrate due to depending on
10ccaca0eb498a2316bbf627d419b29b1a75933a.
2023-04-15 00:40:33 -04:00
Luke Parker
9e78c8fc9e
Test the processor's Substrate signer 2023-04-10 12:48:48 -04:00
Luke Parker
f6206b60ec
Update to bitcoin 0.30
Also performs a general update with a variety of upgraded Substrate depends.
2023-04-09 02:31:13 -04:00
Luke Parker
96525330c2
cargo update 2023-04-08 04:44:28 -04:00
Luke Parker
7abc8f19cd
Move substrate/serai/* to substrate/* 2023-04-08 03:01:14 -04:00
Luke Parker
a4f64e2651
Expand work done in provide_batch 2023-03-31 08:13:45 -04:00