Commit graph

90 commits

Author SHA1 Message Date
Luke Parker
f374cd7398
Update to ethers 2 2023-03-16 20:16:57 -04:00
Luke Parker
67da08705e
cargo update 2023-03-16 19:37:32 -04:00
Luke Parker
0d4b66dc2a
Bump package versions 2023-03-16 19:29:22 -04:00
Luke Parker
d2c1592c61
Resolve merging crypto-{audit, tweaks} and use the proper transcript in Bitcoin 2023-03-16 16:59:20 -04:00
Luke Parker
caf37527eb
Merge branch 'develop' into crypto-tweaks 2023-03-16 16:43:04 -04:00
Luke Parker
669d2dbffc
3.10.2 Explicitly test RecommendedTranscript 2023-03-15 19:55:07 -04:00
Luke Parker
14203bbb46
Use an async Mutex for the Monero distribution
Enables safe async/thread-safe usage.
2023-03-12 04:13:43 -04:00
Luke Parker
4335baa43f
cargo update 2023-03-11 04:49:05 -05:00
akildemir
77de28f77a
add monero seed support (#252)
* add monero seed support

* fix some of the pr comments

* remove languages module and unnecessary error returns

* Clean classic seed impl

Fixes a few issues regarding Zeroize usage/API safety. Mainly a cleanup.

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-03-10 14:16:00 -05:00
Luke Parker
6bff3866ea
Correct Ethereum 2023-03-07 05:25:25 -05:00
Luke Parker
e08adcc1ac
Have Ciphersuite re-export Group 2023-03-07 03:46:16 -05:00
Luke Parker
1a99629a4a
Add feature-gated serde support for Participant/ThresholdParams
These don't have secret data yet sometimes have value to be communicated.
2023-03-07 03:13:55 -05:00
Luke Parker
0e8c55e050
Update and remove unused dependencies 2023-03-07 03:06:46 -05:00
Luke Parker
2729882d65
Update to {k, p}256 0.12 2023-03-07 02:34:10 -05:00
Luke Parker
943438628d
cargo update 2023-03-06 07:39:43 -05:00
Luke Parker
97374a3e24
3.8.6 Correct transcript to scalar derivation
Replaces the externally passed in Digest with C::H since C is available.
2023-03-02 10:04:18 -05:00
Luke Parker
93f7afec8b
3.5.2 Add more tests to ff-group-tests
The audit recommends checking failure cases for from_bytes,
from_bytes_unechecked, and from_repr. This isn't feasible.

from_bytes is allowed to have non-canonical values. [0xff; 32] may accordingly
be a valid point for non-SEC1-encoded curves.

from_bytes_unchecked doesn't have a defined failure mode, and by name,
unchecked, shouldn't necessarily fail. The audit acknowledges the tests should
test for whatever result is 'appropriate', yet any result which isn't a failure
on a valid element is appropriate.

from_repr must be canonical, yet for a binary field of 2^n where n % 8 == 0, a
[0xff; n / 8] repr would be valid.
2023-02-24 06:03:56 -05:00
Luke Parker
d929a8d96e
3.2.2 Use a hash to point for random points in dfg 2023-02-23 04:29:17 -05:00
VRx
c6bd00e778
Bitcoin processor (#232)
* serai Dockerfile & Makefile fixed

* added new bitcoin mod & bitcoinhram

* couple changes

* added odd&even check for bitcoin signing

* sign message updated

* print_keys commented out

* fixed signing process

* Added new bitcoin library & added most of bitcoin processor logic

* added new crate and refactored the bitcoin coin library

* added signing test function

* moved signature.rs

* publish set to false

* tests moved back to the root

* added new functions to rpc

* added utxo test

* added new rpc methods and refactored bitcoin processor

* added spendable output & fixed errors & added new logic for sighash & opened port 18443 for bitcoin docker

* changed tweak keys

* added tweak_keys & publish transaction and refactored bitcoin processor

* added new structs and fixed problems for testing purposes

* reverted dockerfile back its original

* reverted block generation of bitcoin to 5 seconds

* deleted unnecessary test function

* added new sighash & added new dbg messages & fixed couple errors

* fixed couple issue & removed unused functions

* fix for signing process

* crypto file for bitcoin refactored

* disabled test_send & removed some of the debug logs

* signing implemented & transaction weight calculation added & change address logic added

* refactored tweak_keys

* refactored mine_block & fixed change_address logic

* implemented new traits to bitcoin processor& refactored bitcoin processor

* added new line to tests file

* added new line to bitcoin's wallet.rs

* deleted Cargo.toml from coins folder

* edited bitcoin's Cargo.toml and added LICENSE

* added new line to bitcoin's Cargo.toml

* added spaces

* added spaces

* deleted unnecessary object

* added spaces

* deleted patch numbers

* updated sha256 parameter for message

* updated tag as const

* deleted unnecessary brackets and imports

* updated rpc.rs to 2 space indent

* deleted unnecessary brackers

* deleted unnecessary brackets

* changed it to explicit

* updated to explicit

* deleted unnecessary parsing

* added ? for easy return

* updated imports

* updated height to number

* deleted unnecessary brackets

* updated clsag to sig & to_vec to as_ref

* updated _sig to schnorr_signature

* deleted unnecessary variable

* updated Cargo.toml of processor and bitcoin

* updated imports of bitcoin processor

* updated MBlock to BBlock

* updated MSignable to BSignable

* updated imports

* deleted mask from Fee

* updated get_block function return

* updated comparison logic for scripts

* updated assert to debug_assert

* updated height to number

* updated txid logic

* updated tweak_keys definition

* updated imports

* deleted new line

* delete HashMap from monero

* deleted old test code parts

* updated test amount to a round number

* changed the test code part back to its original

* updated imports of rpc.rs

* deleted unnecessary return assignments

* deleted get_fee_per_byte

* deleted create_raw_transaction

* deleted fund_raw_transaction

* deleted sign transaction rpc

* delete verify_message rpc

* deleted get_balance

* deleted decode_raw_transaction rpc

* deleted list_transactions rpc

* changed test_send to p2wpkh

* updated imports of test_send

* fixed imports of test_send

* updated bitcoin's mine_block function

* updated bitcoin's test_send

* updated bitcoin's hram and test_signing

* deleted 2 rpc function (is_confirmed & get_transaction_block_number)

* deleted get_raw_transaction_hex

* deleted get_raw_transaction_info

* deleted new_address

* deleted test_mempool_accept

* updated remove(0) to remove(index)

* deleted ger_raw_transaction

* deleted RawTx trait and converted type to Transaction

* reverted raw_hex feature back

* added NotEnoughFunds to CoinError

* changed Sighash to all

* removed lifetime of RpcParams

* changed pub to pub(crate) & changed sig_hash line

* changed taproot_key_spend_signature_hash to internal

* added Clone to RpcError & deleted get_utxo_for

* changed to_hex to as_bytes for weight calculation

* updated SpendableOutput

* deleted unnecessary parentheses

* updated serialize of Output s id field

* deleted unused crate & added lazy_static

* updated RPC init function

* added lazy_static for TAG_HASH & updated imported crates

* changed get_block_index to get_block_number

* deleted get_block_info

* updated get_height to get_latest_block_number

* removed GetBlockWithDetailResult and get_block_with_transactions

* deleted unnecessary imports from rpc_helper

* removed lock and unlock_unspent

* deleted get_transactions and get_transaction and renamed get_raw_transaction to get_transaction

* updated opt_into_json

* changed payment_address and amount to output_script and amount for transcript

* refactored error logic for rpc & deleted anyhow crate

* added a dedicated file for json helper functions

* refactored imports and deleted unused code

* added clippy::non_snake_case

* removed unused Error items

* added new line to Cargo

* rekmoved Block and used bitcoin::Block direcetly

* removed added println and futures.len check

* removed HashMap from coin mod.rs

* updated Testnet to Regtest

* removed unnecessary variable

* updated as_str to &

* removed RawTx trait

* added newline

* changed test transaction to p2pkh

* updated test_send

* updated test_send

* updated test_send

* reformatted bitcoin processor

* moved sighash logic into signmachine

* removed generate_to_address

* added test_address function to bitcoin processor

* updated RpcResponse to enum and added Clone trait

* removed old RpcResponse

* updated shared_key to internal_key

* updated fee part

* updated test_send block logic

* added a test function for getting spendables

* updated tweaking keys logic

* updated calculate_weight logic

* added todo for BitcoinSchnorr Algorithm

* updated calculate_weight

* updated calculate_weight

* updated calculate_weight

* added a TODO for bitcoin's signing process

* removed unused code

* Finish merging develop

* cargo fmt

* cargo machete

* Handle most clippy lints on bitcoin

Doesn't handle the unused transcript due to pending cryptographic considerations.

* Rearrange imports and clippy tests

* Misc processor lint

* Update deny.toml

* Remove unnecessary RPC code

* updated test_send

* added bitcoin ci & updated test-dependencies yml

* fixed bitcoin ci

* updated bitcoin ci yml

* Remove mining from the bitcoin/monero docker files

The tests should control block production in order to test various
circumstances. The automatic mining disrupts assumptions made in testing. Since
we're now using the Bitcoin docker container for testing...

* Multiple fixes to the Bitcoin processor

Doesn't unwrap on RPC errors. Returns the expected connection error.

Fee calculation has a random - 1. This has been removed.

Supports the change address being an Option, as it is. This should not have
been blindly unwrapped.

* Remove unnecessary RPC code

* Further RPC simplifications

* Simplify Bitcoin action

It should not be mining.

* cargo fmt

* Finish RPC simplifications

* Run bitcoind as a daemon

* Remove the requirement on txindex

Saves tens of GB.

Also has attempt_send no longer return a list of outputs. That's incompatible
with this and only relevant to old scheduling designs.

* Remove number from Bitcoin SignableTransaction

Monero requires the current block number for decoy selection. Bitcoin doesn't
have a use.

* Ban coinbase transactions

These are burdened by maturity, so it's critically flawed to support them.

This causes the test_send function to fail as its working was premised on
a coinbase output. While it does make an actual output, it had insufficient
funds for the test's expectations due to regtest halving every 150 blocks.

In order to workaround this, the test will invalidate any existing chain,
offering a fresh start.

Also removes test_get_spendables and simplifies test_send.

* Various simplifications

Modifies SpendableOutput further to not require RPC calls at time of sign.

Removes the need to have get_transaction in the RPC.

* Clean prepare_send

* Update the Bitcoin TransactionMachine to output a Transaction

* Bitcoin TransactionMachine simplifications

* Update XOnly key handling

* Use a single sighash cache

* Move tweak_keys

* Remove unnecessary PSBT sets

* Restore removed newlines

* Other newlines

* Replace calculate_weight's custom math with a dummy TX serialize

* Move BTC TX construction code from processor to bitcoin

* Rename transactions.rs to wallet.rs

* Remove unused crate

* Note TODO

* Clean bitcoin signature test

* Make unit test out of BTC FROST signing test

* Final lint

* Remove usage of PartiallySignedTransaction

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-01-31 07:48:14 -05:00
Luke Parker
2ace339975
Tokens pallet (#243)
* Use Monero-compatible additional TX keys

This still sends a fingerprinting flare up if you send to a subaddress which
needs to be fixed. Despite that, Monero no should no longer fail to scan TXs
from monero-serai regarding additional keys.

Previously it failed becuase we supplied one key as THE key, and n-1 as
additional. Monero expects n for additional.

This does correctly select when to use THE key versus when to use the additional
key when sending. That removes the ability for recipients to fingerprint
monero-serai by receiving to a standard address yet needing to use an additional
key.

* Add tokens_primitives

Moves OutInstruction from in-instructions.

Turns Destination into OutInstruction.

* Correct in-instructions DispatchClass

* Add initial tokens pallet

* Don't allow pallet addresses to equal identity

* Add support for InInstruction::transfer

Requires a cargo update due to modifications made to serai-dex/substrate.

Successfully mints a token to a SeraiAddress.

* Bind InInstructions to an amount

* Add a call filter to the runtime

Prevents worrying about calls to the assets pallet/generally tightens things
up.

* Restore Destination

It was meged into OutInstruction, yet it didn't make sense for OutInstruction
to contain a SeraiAddress.

Also deletes the excessively dated Scenarios doc.

* Split PublicKey/SeraiAddress

Lets us define a custom Display/ToString for SeraiAddress.

Also resolves an oddity where PublicKey would be encoded as String, not
[u8; 32].

* Test burning tokens/retrieving OutInstructions

Modularizes processor_coinUpdates into a shared testing utility.

* Misc lint

* Don't use PolkadotExtrinsicParams
2023-01-28 01:47:13 -05:00
Luke Parker
8ca90e7905
Initial In Instructions pallet and Serai client lib (#233)
* Initial work on an In Inherents pallet

* Add an event for when a batch is executed

* Add a dummy provider for InInstructions

* Add in-instructions to the node

* Add the Serai runtime API to the processor

* Move processor tests around

* Build a subxt Client around Serai

* Successfully get Batch events from Serai

Renamed processor/substrate to processor/serai.

* Much more robust InInstruction pallet

* Implement the workaround from https://github.com/paritytech/subxt/issues/602

* Initial prototype of processor generated InInstructions

* Correct PendingCoins data flow for InInstructions

* Minor lint to in-instructions

* Remove the global Serai connection for a partial re-impl

* Correct ID handling of the processor test

* Workaround the delay in the subscription

* Make an unwrap an if let Some, remove old comments

* Lint the processor toml

* Rebase and update

* Move substrate/in-instructions to substrate/in-instructions/pallet

* Start an in-instructions primitives lib

* Properly update processor to subxt 0.24

Also corrects failures from the rebase.

* in-instructions cargo update

* Implement IsFatalError

* is_inherent -> true

* Rename in-instructions crates and misc cleanup

* Update documentation

* cargo update

* Misc update fixes

* Replace height with block_number

* Update processor src to latest subxt

* Correct pipeline for InInstructions testing

* Remove runtime::AccountId for serai_primitives::NativeAddress

* Rewrite the in-instructions pallet

Complete with respect to the currently written docs.

Drops the custom serializer for just using SCALE.

Makes slight tweaks as relevant.

* Move instructions' InherentDataProvider to a client crate

* Correct doc gen

* Add serde to in-instructions-primitives

* Add in-instructions-primitives to pallet

* Heights -> BlockNumbers

* Get batch pub test loop working

* Update in instructions pallet terminology

Removes the ambiguous Coin for Update.

Removes pending/artificial latency for furture client work.

Also moves to using serai_primitives::Coin.

* Add a BlockNumber primitive

* Belated cargo fmt

* Further document why DifferentBatch isn't fatal

* Correct processor sleeps

* Remove metadata at compile time, add test framework for Serai nodes

* Remove manual RPC client

* Simplify update test

* Improve re-exporting behavior of serai-runtime

It now re-exports all pallets underneath it.

* Add a function to get storage values to the Serai RPC

* Update substrate/ to latest substrate

* Create a dedicated crate for the Serai RPC

* Remove unused dependencies in substrate/

* Remove unused dependencies in coins/

Out of scope for this branch, just minor and path of least resistance.

* Use substrate/serai/client for the Serai RPC lib

It's a bit out of place, since these client folders are intended for the node to
access pallets and so on. This is for end-users to access Serai as a whole.

In that sense, it made more sense as a top level folder, yet that also felt
out of place.

* Move InInstructions test to serai-client for now

* Final cleanup

* Update deny.toml

* Cargo.lock update from merging develop

* Update nightly

Attempt to work around the current CI failure, which is a Rust ICE.

We previously didn't upgrade due to clippy 10134, yet that's been reverted.

* clippy

* clippy

* fmt

* NativeAddress -> SeraiAddress

* Sec fix on non-provided updates and doc fixes

* Add Serai as a Coin

Necessary in order to swap to Serai.

* Add a BlockHash type, used for batch IDs

* Remove origin from InInstruction

Makes InInstructionTarget. Adds RefundableInInstruction with origin.

* Document storage items in in-instructions

* Rename serai/client/tests/serai.rs to updates.rs

It only tested publishing updates and their successful acceptance.
2023-01-20 11:00:18 -05:00
Luke Parker
e13cf52c49
Update Cargo.lock
It appears to have changed with the recent Monero tests yet not have been
included in that PR.
2023-01-17 02:35:22 -05:00
Luke Parker
ced89332d2
cargo update
Necessary due to https://github.com/RustCrypto/signatures/issues/615.

Opportunity taken to update Substrate.
2023-01-16 10:42:31 -05:00
Luke Parker
be05e0dd47
Revert "Implement a FROST algorithm for Schnorrkel"
This reverts commit 8ef8b5ca6f.
2023-01-13 18:57:07 -05:00
Luke Parker
8ef8b5ca6f
Implement a FROST algorithm for Schnorrkel 2023-01-13 18:52:38 -05:00
Luke Parker
97a94a0bf8
cargo update
Fixes an irrelevant security issue in tokio.
2023-01-08 09:10:38 -05:00
Luke Parker
f760c09006
Add Substrate "assets" pallet
While over-engineered for our purposes, it's still usable.

Also cleans the runtime a bit.
2023-01-05 19:45:19 -05:00
Luke Parker
daa88a051f
Remove the timestamp pallet
It was needed for contracts, which has since been removed. We now no longer
need it.
2023-01-04 23:09:58 -05:00
Luke Parker
e979883f2d
Initial validator sets pallet (#187)
* Initial work on a Validator Sets pallet

* Update Validator Set docs per current discussions

* Update validator-sets primitives and storage handling

* Add validator set pallets to deny.toml

* Remove Curve from primitives

Since we aren't reusing keys across coins, there's no reason for it to be
on-chain (as previously planned).

* Update documentation on Validator Sets

* Use Twox64Concat instead of Identity

Ensures an even distribution of keys. While xxhash is breakable, these keys
aren't manipulatable by users.

* Add math ops on Amount and define a coin as 1e8

* Add validator-sets to the runtime and remove contracts

Also removes the randomness pallet which was only required by the contracts
runtime.

Does not remove the contracts folder yet so they can still be referred to while
validator-sets is under development. Does remove them from Cargo.toml.

* Add vote function to validator-sets

* Remove contracts folder

* Create an event for the Validator Sets pallet

* Remove old contracts crates from deny.toml

* Remove line from staking branch

* Remove staking from runtime

* Correct VS Config in runtime

* cargo update

* Resolve a few PR comments on terminology

* Create a serai-primitives crate

Move types such as Amount/Coin out of validator-sets. Will be expanded in the
future.

* Fixes for last commit

* Don't reserve set 0

* Further fixes

* Add files meant for last commit

* Remove Staking transfer
2023-01-04 22:52:41 -05:00
Luke Parker
bf09b5cc7e
cargo update 2023-01-04 03:17:36 -05:00
Luke Parker
bacf31378d
Add test vectors for Ciphersuite::hash_to_F 2022-12-25 02:50:10 -05:00
Luke Parker
445bb3786e
Add a dedicated crate for testing ff/group implementors
Provides extensive testing for dalek-ff-group and ed448.

Also includes a fix for an observed bug in ed448.
2022-12-24 15:09:09 -05:00
Luke Parker
6e518f5c22
cargo update due to yanked openssl crate 2022-12-20 23:12:26 -05:00
Luke Parker
25f1549c6c
Move verify_share to return batch-verifiable statements
While the previous construction achieved n/2 average detection,
this will run in log2(n). Unfortunately, the need to keep entropy
around (or take in an RNG here) remains.
2022-12-13 20:31:00 -05:00
Luke Parker
783a445a3e
Use a challenge from the FROST transcript as context in the DLEq proofs 2022-12-13 19:27:09 -05:00
Luke Parker
af86b7a499
Support caching preprocesses in FROST (#190)
* Remove the explicit included participants from FROST

Now, whoever submits preprocesses becomes the signing set. Better separates
preprocess from sign, at the cost of slightly more annoying integrations
(Monero needs to now independently lagrange/offset its key images).

* Support caching preprocesses

Closes https://github.com/serai-dex/serai/issues/40.

I *could* have added a serialization trait to Algorithm and written a ton of
data to disk, while requiring Algorithm implementors also accept such work.
Instead, I moved preprocess to a seeded RNG (Chacha20) which should be as
secure as the regular RNG. Rebuilding from cache simply loads the previously
used Chacha seed, making the Algorithm oblivious to the fact it's being
rebuilt from a cache. This removes any requirements for it to be modified
while guaranteeing equivalency.

This builds on the last commit which delayed determining the signing set till
post-preprocess acquisition. Unfortunately, that commit did force preprocess
from ThresholdView to ThresholdKeys which had visible effects on Monero.

Serai will actually need delayed set determination for #163, and overall,
it remains better, hence it's inclusion.

* Document FROST preprocess caching

* Update ethereum to new FROST

* Fix bug in Monero offset calculation and update processor
2022-12-08 19:04:35 -05:00
Luke Parker
bec92c10ad
Update to ethers 1.0
Removes rust_decimal as a depend, which added borsh and multiple other misc
packages in the previous commit.
2022-12-07 18:05:06 -05:00
Luke Parker
bade7a504e
cargo update 2022-12-07 17:56:53 -05:00
Luke Parker
13977f6287
Clean and document the DKG library's encryption
Encryption used to be inlined into FROST. When writing the documentation, I
realized it was decently hard to review. It also was antagonistic to other
hosted DKG algorithms by not allowing code re-use.

Encryption is now a standalone module, providing clear boundaries and
reusability.

Additionally, the DKG protocol itself used to use the ciphersuite's specified
hash function (with an HKDF to prevent length extension attacks). Now,
RecommendedTranscript is used to achieve much more robust transcripting and
remove the HKDF dependency. This does add Blake2 into all consumers yet is
preferred for its security properties and ease of review.
2022-12-07 17:30:42 -05:00
Luke Parker
f136328c08
Fix tendermint-machine's version in Cargo.lock 2022-12-05 17:26:28 -05:00
Luke Parker
8f4d6f79f3
Initial Tendermint implementation (#145)
* Machine without timeouts

* Time code

* Move substrate/consensus/tendermint to substrate/tendermint

* Delete the old paper doc

* Refactor out external parts to generics

Also creates a dedicated file for the message log.

* Refactor <V, B> to type V, type B

* Successfully compiling

* Calculate timeouts

* Fix test

* Finish timeouts

* Misc cleanup

* Define a signature scheme trait

* Implement serialization via parity's scale codec

Ideally, this would be generic. Unfortunately, the generic API serde 
doesn't natively support borsh, nor SCALE, and while there is a serde 
SCALE crate, it's old. While it may be complete, it's not worth working 
with.

While we could still grab bincode, and a variety of other formats, it 
wasn't worth it to go custom and for Serai, we'll be using SCALE almost 
everywhere anyways.

* Implement usage of the signature scheme

* Make the infinite test non-infinite

* Provide a dedicated signature in Precommit of just the block hash

Greatly simplifies verifying when syncing.

* Dedicated Commit object

Restores sig aggregation API.

* Tidy README

* Document tendermint

* Sign the ID directly instead of its SCALE encoding

For a hash, which is fixed-size, these should be the same yet this helps 
move past the dependency on SCALE. It also, for any type where the two 
values are different, smooths integration.

* Litany of bug fixes

Also attempts to make the code more readable while updating/correcting 
documentation.

* Remove async recursion

Greatly increases safety as well by ensuring only one message is 
processed at once.

* Correct timing issues

1) Commit didn't include the round, leaving the clock in question.

2) Machines started with a local time, instead of a proper start time.

3) Machines immediately started the next block instead of waiting for 
the block time.

* Replace MultiSignature with sr25519::Signature

* Minor SignatureScheme API changes

* Map TM SignatureScheme to Substrate's sr25519

* Initial work on an import queue

* Properly use check_block

* Rename import to import_queue

* Implement tendermint_machine::Block for Substrate Blocks

Unfortunately, this immediately makes Tendermint machine capable of 
deployment as  crate since it uses a git reference. In the future, a 
Cargo.toml patch section for serai/substrate should be investigated. 
This is being done regardless as it's the quickest way forward and this 
is for Serai.

* Dummy Weights

* Move documentation to the top of the file

* Move logic into TendermintImport itself

Multiple traits exist to verify/handle blocks. I'm unsure exactly when 
each will be called in the pipeline, so the easiest solution is to have 
every step run every check.

That would be extremely computationally expensive if we ran EVERY check, 
yet we rely on Substrate for execution (and according checks), which are 
limited to just the actual import function.

Since we're calling this code from many places, it makes sense for it to 
be consolidated under TendermintImport.

* BlockImport, JustificationImport, Verifier, and import_queue function

* Update consensus/lib.rs from PoW to Tendermint

Not possible to be used as the previous consensus could. It will not
produce blocks nor does it currenly even instantiate a machine. This is
just he next step.

* Update Cargo.tomls for substrate packages

* Tendermint SelectChain

This is incompatible with Substrate's expectations, yet should be valid 
for ours

* Move the node over to the new SelectChain

* Minor tweaks

* Update SelectChain documentation

* Remove substrate/node lib.rs

This shouldn't be used as a library AFAIK. While runtime should be, and 
arguably should even be published, I have yet to see node in the same 
way. Helps tighten API boundaries.

* Remove unused macro_use

* Replace panicking todos with stubs and // TODO

Enables progress.

* Reduce chain_spec and use more accurate naming

* Implement block proposal logic

* Modularize to get_proposal

* Trigger block importing

Doesn't wait for the response yet, which it needs to.

* Get the result of block importing

* Split import_queue into a series of files

* Provide a way to create the machine

The BasicQueue returned obscures the TendermintImport struct. 
Accordingly, a Future scoped with access is returned upwards, which when 
awaited will create the machine. This makes creating the machine 
optional while maintaining scope boundaries.

Is sufficient to create a 1-node net which produces and finalizes 
blocks.

* Don't import justifications multiple times

Also don't broadcast blocks which were solely proposed.

* Correct justication import pipeline

Removes JustificationImport as it should never be used.

* Announce blocks

By claiming File, they're not sent ovber the P2P network before they 
have a justification, as desired. Unfortunately, they never were. This 
works around that.

* Add an assert to verify proposed children aren't best

* Consolidate C and I generics into a TendermintClient trait alias

* Expand sanity checks

Substrate doesn't expect nor officially support children with less work 
than their parents. It's a trick used here. Accordingly, ensure the 
trick's validity.

* When resetting, use the end time of the round which was committed to

The machine reset to the end time of the current round. For a delayed 
network connection, a machine may move ahead in rounds and only later 
realize a prior round succeeded. Despite acknowledging that round's 
success, it would maintain its delay when moving to the next block, 
bricking it.

Done by tracking the end time for each round as they occur.

* Move Commit from including the round to including the round's end_time

The round was usable to build the current clock in an accumulated 
fashion, relative to the previous round. The end time is the absolute 
metric of it, which can be used to calculate the round number (with all 
previous end times).

Substrate now builds off the best block, not genesis, using the end time 
included in the justification to start its machine in a synchronized 
state.

Knowing the end time of a round, or the round in which block was 
committed to, is necessary for nodes to sync up with Tendermint. 
Encoding it in the commit ensures it's long lasting and makes it readily 
available, without the load of an entire transaction.

* Add a TODO on Tendermint

* Misc bug fixes

* More misc bug fixes

* Clean up lock acquisition

* Merge weights and signing scheme into validators, documenting needed changes

* Add pallet sessions to runtime, create pallet-tendermint

* Update node to use pallet sessions

* Update support URL

* Partial work on correcting pallet calls

* Redo Tendermint folder structure

* TendermintApi, compilation fixes

* Fix the stub round robin

At some point, the modulus was removed causing it to exceed the 
validators list and stop proposing.

* Use the validators list from the session pallet

* Basic Gossip Validator

* Correct Substrate Tendermint start block

The Tendermint machine uses the passed in number as the block's being 
worked on number. Substrate passed in the already finalized block's 
number.

Also updates misc comments.

* Clean generics in Tendermint with a monolith with associated types

* Remove the Future triggering the machine for an async fn

Enables passing data in, such as the network.

* Move TendermintMachine from start_num, time to last_num, time

Provides an explicitly clear API clearer to program around.

Also adds additional time code to handle an edge case.

* Connect the Tendermint machine to a GossipEngine

* Connect broadcast

* Remove machine from TendermintImport

It's not used there at all.

* Merge Verifier into block_import.rs

These two files were largely the same, just hooking into sync structs 
with almost identical imports. As this project shapes up, removing dead 
weight is appreciated.

* Create a dedicated file for being a Tendermint authority

* Deleted comment code related to PoW

* Move serai_runtime specific code from tendermint/client to node

Renames serai-consensus to sc_tendermint

* Consolidate file structure in sc_tendermint

* Replace best_* with finalized_*

We test their equivalency yet still better to use finalized_* in 
general.

* Consolidate references to sr25519 in sc_tendermint

* Add documentation to public structs/functions in sc_tendermint

* Add another missing comment

* Make sign asynchronous

Some relation to https://github.com/serai-dex/serai/issues/95.

* Move sc_tendermint to async sign

* Implement proper checking of inherents

* Take in a Keystore and validator ID

* Remove unnecessary PhantomDatas

* Update node to latest sc_tendermint

* Configure node for a multi-node testnet

* Fix handling of the GossipEngine

* Use a rounded genesis to obtain sufficient synchrony within the Docker env

* Correct Serai d-f names in Docker

* Remove an attempt at caching I don't believe would ever hit

* Add an already in chain check to block import

While the inner should do this for us, we call verify_order on our end 
*before* inner to ensure sequential import. Accordingly, we need to 
provide our own check.

Removes errors of "non-sequential import" when trying to re-import an 
existing block.

* Update the consensus documentation

It was incredibly out of date.

* Add a _ to the validator arg in slash

* Make the dev profile a local testnet profile

Restores a dev profile which only has one validator, locally running.

* Reduce Arcs in TendermintMachine, split Signer from SignatureScheme

* Update sc_tendermint per previous commit

* Restore cache

* Remove error case which shouldn't be an error

* Stop returning errors on already existing blocks entirely

* Correct Dave, Eve, and Ferdie to not run as validators

* Rename dev to devnet

--dev still works thanks to the |. Acheieves a personal preference of 
mine with some historical meaning.

* Add message expiry to the Tendermint gossip

* Localize the LibP2P protocol to the blockchain

Follows convention by doing so. Theoretically enables running multiple 
blockchains over a single LibP2P connection.

* Add a version to sp-runtime in tendermint-machine

* Add missing trait

* Bump Substrate dependency

Fixes #147.

* Implement Schnorr half-aggregation from https://eprint.iacr.org/2021/350.pdf

Relevant to https://github.com/serai-dex/serai/issues/99.

* cargo update (tendermint)

* Move from polling loops to a pure IO model for sc_tendermint's gossip

* Correct protocol name handling

* Use futures mpsc instead of tokio

* Timeout futures

* Move from a yielding loop to select in tendermint-machine

* Update Substrate to the new TendermintHandle

* Use futures pin instead of tokio

* Only recheck blocks with non-fatal inherent transaction errors

* Update to the latest substrate

* Separate the block processing time from the latency

* Add notes to the runtime

* Don't spam slash

Also adds a slash condition of failing to propose.

* Support running TendermintMachine when not a validator

This supports validators who leave the current set, without crashing 
their nodes, along with nodes trying to become validators (who will now 
seamlessly transition in).

* Properly define and pass around the block size

* Correct the Duration timing

The proposer will build it, send it, then process it (on the first 
round). Accordingly, it's / 3, not / 2, as / 2 only accounted for the 
latter events.

* Correct time-adjustment code on round skip

* Have the machine respond to advances made by an external sync loop

* Clean up time code in tendermint-machine

* BlockData and RoundData structs

* Rename Round to RoundNumber

* Move BlockData to a new file

* Move Round to an Option due to the pseudo-uninitialized state we create

Before the addition of RoundData, we always created the round, and on 
.round(0), simply created it again. With RoundData, and the changes to 
the time code, we used round 0, time 0, the latter being incorrect yet 
not an issue due to lack of misuse.

Now, if we do misuse it, it'll panic.

* Clear the Queue instead of draining and filtering

There shouldn't ever be a message which passes the filter under the 
current design.

* BlockData::new

* Move more code into block.rs

Introduces type-aliases to obtain Data/Message/SignedMessage solely from 
a Network object.

Fixes a bug regarding stepping when you're not an active validator.

* Have verify_precommit_signature return if it verified the signature

Also fixes a bug where invalid precommit signatures were left standing 
and therefore contributing to commits.

* Remove the precommit signature hash

It cached signatures per-block. Precommit signatures are bound to each 
round. This would lead to forming invalid commits when a commit should 
be formed. Under debug, the machine would catch that and panic. On 
release, it'd have everyone who wasn't a validator fail to continue 
syncing.

* Slight doc changes

Also flattens the message handling function by replacing an if 
containing all following code in the function with an early return for 
the else case.

* Always produce notifications for finalized blocks via origin overrides

* Correct weird formatting

* Update to the latest tendermint-machine

* Manually step the Tendermint machine when we synced a block over the network

* Ignore finality notifications for old blocks

* Remove a TODO resolved in 8c51bc011d

* Add a TODO comment to slash

Enables searching for the case-sensitive phrase and finding it.

* cargo fmt

* Use a tmp DB for Serai in Docker

* Remove panic on slash

As we move towards protonet, this can happen (if a node goes offline), 
yet it happening brings down the entire net right now.

* Add log::error on slash

* created shared volume between containers

* Complete the sh scripts

* Pass in the genesis time to Substrate

* Correct block announcements

They were announced, yet not marked best.

* Correct pupulate_end_time

It was used as inclusive yet didn't work inclusively.

* Correct gossip channel jumping when a block is synced via Substrate

* Use a looser check in import_future

This triggered so it needs to be accordingly relaxed.

* Correct race conditions between add_block and step

Also corrects a <= to <.

* Update cargo deny

* rename genesis-service to genesis

* Update Cargo.lock

* Correct runtime Cargo.toml whitespace

* Correct typo

* Document recheck

* Misc lints

* Fix prev commit

* Resolve low-hanging review comments

* Mark genesis/entry-dev.sh as executable

* Prevent a commit from including the same signature multiple times

Yanks tendermint-machine 0.1.0 accordingly.

* Update to latest nightly clippy

* Improve documentation

* Use clearer variable names

* Add log statements

* Pair more log statements

* Clean TendermintAuthority::authority as possible

Merges it into new. It has way too many arguments, yet there's no clear path at
consolidation there, unfortunately.

Additionally provides better scoping within itself.

* Fix #158

Doesn't use lock_import_and_run for reasons commented (lack of async).

* Rename guard to lock

* Have the devnet use the current time as the genesis

Possible since it's only a single node, not requiring synchronization.

* Fix gossiping

I really don't know what side effect this avoids and I can't say I care at this
point.

* Misc lints

Co-authored-by: vrx00 <vrx00@proton.me>
Co-authored-by: TheArchitect108 <TheArchitect108@protonmail.com>
2022-12-03 18:38:02 -05:00
Luke Parker
00c1f21224
cargo update
Resolves yanking of crossbeam (see https://github.com/crossbeam-rs/crossbeam/issues/931).
2022-11-23 07:44:30 -05:00
Luke Parker
138bb46e19
cargo update
Mandated by https://github.com/dylni/os_str_bytes/issues/14.
2022-11-21 02:23:38 -05:00
Luke Parker
56574f2f5b
Add a cargo deny workflow (#89)
* Add a cargo deny workflow

Also trims out a pointless submodule checkout (we have none).

* Remove no longer relevant advisories/allowances

* Patch for array-bytes

* Remove unused properties

* Restore chrono advisory

* Allow MPL-2.0, correct GPL-3.0 allowance specification

* Properly ban copyleft, run on all crates

* Exceptions for Serai crates (AGPL-3.0)

* Remove top comments

* Clarify reasoning for not checking advisories in CI

* Run all checks in CI

While this may bring down an unrelated commit, we can manually review, before creating a followup commit allowing it. If it's critical, then this did its job.
2022-11-16 20:53:35 -06:00
Luke Parker
6f9cf510da
Support an authenticated Monero RPC
Closes https://github.com/serai-dex/serai/issues/143.
2022-11-14 23:24:35 -05:00
Luke Parker
fe1f28c77a
cargo update 2022-11-14 04:55:00 -05:00
Luke Parker
3d9b9b178c
Zeroizing allocator (#154)
* Add a zeroizing allocator

* Also implement the allocator API

* Add misisng license file to zalloc

* Slight change to zalloc description
2022-11-10 23:34:40 -06:00
Luke Parker
7334ed1f43
cargo update
Updates Substrate to polkadot-v0.9.33
2022-11-10 23:59:20 -05:00
Luke Parker
84de427d72
Fix https://github.com/serai-dex/serai/issues/150 2022-11-10 22:35:09 -05:00
Luke Parker
d714f2202d
Document multiexp
Bumps the crate version to enable publishing.
2022-11-07 18:31:20 -05:00