Commit graph

1465 commits

Author SHA1 Message Date
GitHub Actions
cb1ef0d71f Update nightly 2023-12-01 00:24:51 -05:00
Luke Parker
b823413c9b
Use parity-db in current Dockerfiles (#455)
* Use redb and in Dockerfiles

The motivation for redb was to remove the multiple rocksdb compile times from
CI.

* Correct feature flagging of coordinator and message-queue in Dockerfiles

* Correct message-queue DB type alias

* Use consistent table typing in redb

* Correct rebase artifacts

* Correct removal of binaries feature from message-queue

* Correct processor feature flagging

* Replace redb with parity-db

It still has much better compile times yet doesn't block when creating multiple
transactions. It also is actively maintained and doesn't grow our tree. The MPT
aspects are irrelevant.

* Correct stray Redb

* clippy warning

* Correct txn get
2023-11-30 04:22:37 -05:00
Luke Parker
d1122a6535
Fix no-std builds 2023-11-29 03:20:49 -05:00
Luke Parker
8040fedddf
Have simple-request Response's borrow the Client to ensure it's not prematurely dropped 2023-11-29 01:16:18 -05:00
Luke Parker
51bb434239
Properly include the non-JSON HTTP result in Monero's RpcError
The CI for 695d1f0ecf actually errored with a
non-JSON response, hence the value in this.
2023-11-29 00:43:59 -05:00
Luke Parker
f0ff3a18d2
Use debug builds in our Dockerfiles to reduce CI times (#462)
* Use debug builds in our Dockerfiles to reduce CI times

Also enables only spawning the mdns service when debug in the coordinator.

* Correct underflow in processor

Prior undetected due to relase builds not having bounds checks enabled.

* Restore Serai release due to CI/RPC failures caused by compiling it in debug mode

This is *probably* worth an issue filed upstream, if it can be tracked down.

* Correct failing debug asserts in Monero

These debug asserts assumed there was a change address to take the remainder.
If there's no change address, the remainder is shunted to the fee, causing the
fee to be distinct from the estimate.

We presumably need to modify monero-serai such that change: None isn't valid,
and users must use Change::Fingerprintable(None).
2023-11-29 00:24:37 -05:00
Luke Parker
695d1f0ecf
Remove subxt (#460)
* Remove subxt

Removes ~20 crates from our Cargo.lock.

Removes downloading the metadata and enables removing the getMetadata RPC route
(relevant to #379).

Moves forward #337.

Done now due to distinctions in the subxt 0.32 API surface which make it
justifiable to not update.

* fmt, update due to deny triggering on a yanked crate

* Correct the handling of substrate_block_notifier now that it's ephemeral, not long-lived

* Correct URL in tests/coordinator from ws to http
2023-11-28 02:29:50 -05:00
Luke Parker
571195bfda
Resolve #360 (#456)
* Remove NetworkId from processor-messages

Because intent binds to the sender/receiver, it's not needed for intent.

The processor knows what the network is.

The coordinator knows which to use because it's sending this message to the
processor for that network.

Also removes the unused zeroize.

* ProcessorMessage::Completed use Session instead of key

* Move SubstrateSignId to Session

* Finish replacing key with session
2023-11-26 12:14:23 -05:00
Luke Parker
b79cf8abde
Move message-queue to a fully binary representation (#454)
* Move message-queue to a fully binary representation

Additionally adds a timeout to the message queue test.

* coordinator clippy

* Remove contention for the message-queue socket by using per-request sockets

* clippy
2023-11-26 11:22:18 -05:00
Luke Parker
c6c74684c9
Increase timeouts in processor tests
For some reason, these constantly failed for me while waiting for the key pair
to confirm. This adds a sleep during the mining process, to ensure blocks
actually have time between them, and mines several more blocks to handle the
median code recently added.
2023-11-25 04:09:07 -05:00
Luke Parker
de14687a0d
Fix the processor's Monero time monotonicity
Monero doesn't assert the time increases with each block, solely that it
doesn't decrease. Now, the block number is added to the time to ensure it
increases.
2023-11-25 04:07:31 -05:00
Luke Parker
d60e007126
Add a binaries feature to the processor to reduce dependencies when used as a lib
processor isn't intended to be used as a library, yet serai-processor-tests
does pull it in as a lib. This caused serai-processor-tests to need to compile
rocksdb, which added multiple minutes to the compilation time.
2023-11-25 04:04:52 -05:00
Luke Parker
b296be8515
Replace bincode with borsh (#452)
* Add SignalsConfig to chain_spec

* Correct multiexp feature flagging for rand_core std

* Remove bincode for borsh

Replaces a non-canonical encoding with a canonical encoding which additionally
should be faster.

Also fixes an issue where we used bincode in transcripts where it cannot be
trusted.

This ended up fixing a myriad of other bugs observed, unfortunately.
Accordingly, it either has to be merged or the bug fixes from it must be ported
to a new PR.

* Make serde optional, minimize usage

* Make borsh an optional dependency of substrate/ crates

* Remove unused dependencies

* Use [u8; 64] where possible in the processor messages

* Correct borsh feature flagging
2023-11-25 04:01:11 -05:00
Luke Parker
6b2876351e
Add file meant for prior commit 2023-11-24 21:41:59 -05:00
Luke Parker
0ea90d054d
Enable the signals pallet to halt networks' publication of Batchs
Relevant to #394.

Prevents hand-over due to hand-over occurring via a `Batch` publication.

Expects a new protocol to restore functionality (after a retirement of the
current protocol).
2023-11-24 19:56:57 -05:00
Luke Parker
eb1d00aa55
Update TX format in coordinator test 2023-11-23 11:45:00 -05:00
Luke Parker
372149c2cc
Various simplifications re: Serai transactions
Removes PairSigner for the pair directly.

Resets spec_version to 1.

Defines the extrinsic without its length prefix, only prefixing during publish.
2023-11-23 00:02:01 -05:00
Luke Parker
c6cf33e370
Install wasm toolchain before ADDing files to docker images
Enables caching the wasm toolchain.
2023-11-22 18:07:58 -05:00
Luke Parker
f58478ad87
Add hex as a dependency to serai-client 2023-11-22 18:06:10 -05:00
Luke Parker
88a1726399
Fixes for prior commit 2023-11-22 16:24:50 -05:00
Luke Parker
08e6669403
Replace substrate/client's use of Payload with usage of RuntimeCall
Gains explicit typing.
2023-11-22 11:23:04 -05:00
akildemir
fcfdadc791
Integrate session pallet into validator-sets pallet (#440)
* remove pallet-session

* Store key shares in InSet

* integrate grandpa to vs-pallet

* integrate pallet babe

* remove pallet-session & authority discovery from runtime

* update the grandpa pallet path

* cargo update grandpa

* cargo update substrate

* Misc tweaks

Sets validators for BABE/GRANDPA in chain_spec, per Akil's realization that was
the missing piece.

* fix pr comments

* bug fix & tidy up

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-22 06:22:46 -05:00
Luke Parker
07c657306b
Remove duplicated genesis presence in Tributary Scanner DB keys
This wasted 32-bytes per every single entry in the DB (ignoring de-duplication
possible by the DB layer).
2023-11-22 04:43:49 -05:00
David Bell
9df8c9476e
Convert tributary to use create_db macro (#448)
* chore: convert tributary to use create_db macro

* chore: fix fmt

* chore: break long line
2023-11-22 04:17:51 -05:00
econsta
9ab2a2cfe0
processor/db.rs macro implimentation (#437)
* processor/db.rs macro implimentation

* ran clippy and fmt

* incorporated recommendations

* used empty uple instead of [u8; 0]

* ran fmt
2023-11-22 03:58:26 -05:00
Luke Parker
a315040aee
if-watch 3.2.0 2023-11-22 01:24:17 -05:00
Luke Parker
1822e31142
Grow the yamux buffers to exceed the maximum message size 2023-11-21 02:01:41 -05:00
Luke Parker
6efc313d76
Add/update msrv for common/*, crypto/*, coins/*, and substrate/*
This includes all published crates.
2023-11-21 01:19:40 -05:00
Luke Parker
4b85d4b03b
Rename SubstrateSigner to BatchSigner 2023-11-20 22:21:52 -05:00
econsta
05d8c32be8
Multisig db (#444)
* implement db macro for processor/multisigs/db.rs

* ran fmt

* cargo +nightly fmt

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-20 21:40:54 -05:00
econsta
8634d90b6b
implement db macro for processor/signer.rs (#438)
* implement db macro for processor/signer.rs

* Use ()

* CompletedDb -> CompletionsDb

* () -> &()

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-20 03:03:45 -05:00
econsta
aa9daea6b0
implement db macro for processor/substrate_signer (#439)
* implement db macro for processor/substrate_signer

* Use ()

* Correct AttemptDb usage of ()

* () -> &()

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2023-11-20 03:03:35 -05:00
Luke Parker
942873f99d Non-intrusive cargo update 2023-11-20 02:31:47 -05:00
Luke Parker
7c9b581723
Remove dtolnay's rust-toolchain action (#442)
* Remove dtolnay's rust-toolchain action

I believe our rust-toolchain.toml handles its use case exactly.

I don't believe this'll work, as it'd require rustup install a cargo stub
before any toolchain is installed, yet I want to confirm it doesn't.

* Place quotes around nightly toolchain version

* Put toolchain before options to resolve what appears to be a bug in rustup's help strings

* Add wasm32-unkknown-unknown to clippy workflow
2023-11-20 02:31:22 -05:00
Luke Parker
b37a0db538
Move where the RISC-V toolchain is installed during the no-std workflow 2023-11-19 22:45:51 -05:00
Luke Parker
797604ad73
Replace usage of io::Error::new(io::ErrorKind::Other, with io::Error::other
Newly possible with Rust 1.74.
2023-11-19 18:31:37 -05:00
Luke Parker
05b975dff9
Rust 1.74
Adds a Rust toolchain file to be less disruptive to developers who don't keep
their toolchain synchronized (by now having rustup automatically synchronize).

Hopefully helps resolve how +nightly clippy may pass for the coordinator, yet
building would fail due to stable's (hopefully prior?) failure to model some
async functions re: Send/Sync.

Also adds rust-src as a component in preparation of
https://github.com/paritytech/polkadot-sdk/pull/2217
2023-11-19 17:47:46 -05:00
Luke Parker
74a8df4c7b
Add a new primitive of a DB-backed channel
The coordinator already had one of these, albeit implemented much worse than
the one now properly introduced. It had to either be sending or receiving,
whereas the new one can do both at the same time.

This replaces said instance and enables pleasant patterns when implementing the
processor/coordinator.
2023-11-19 02:05:01 -05:00
Luke Parker
bd14db9f76
Version pin foundry
Fixes #116 and #441.
2023-11-19 01:21:43 -05:00
Luke Parker
25c02c1311
Ban Serai from setting keys
They were already banned form publishing `Batch`s, yet the ability to set keys
enabled changing how certain functionality in the coordinator operated.
Removing this malleability ensures operation as expected.
2023-11-18 21:28:16 -05:00
Luke Parker
b018fc432c
Add deletion to create_db 2023-11-18 20:56:58 -05:00
Luke Parker
6e4ecbc90c
Support trailing commas in create_db 2023-11-18 20:54:37 -05:00
Luke Parker
be48dcc4a4
Use multiple LibP2P topics
We still need a peering protocol... Hopefully, we can read peers off of the
Substrate node's DHT.
2023-11-18 20:37:55 -05:00
Luke Parker
25066437da
Attempt to resolve #434
Worsens the coordinator tests' ensuring of the validity of the cosigning
protocol, yet should actually properly model it.
2023-11-18 00:12:36 -05:00
Luke Parker
db49a63c2b
Bump zeroize to appease cargo deny 2023-11-16 14:18:00 -05:00
Luke Parker
ee50f584aa
Add saner log statements to the coordinator
Disables trace on every single P2P message.

Logs a short-form of each transaction.
2023-11-16 13:37:39 -05:00
Luke Parker
14f3f330db
Remove deadlock in cosign_evaluator 2023-11-16 13:32:15 -05:00
Luke Parker
30a77d863f
Include non-JSON response from Monero node in error 2023-11-15 22:57:37 -05:00
Luke Parker
a03a1edbff
Remove needless transposition in coordinator 2023-11-15 22:49:58 -05:00
Luke Parker
c03afbe03e
Downgrade RustCrypto packages which violate Rust's interpretation of semver 2023-11-15 21:24:52 -05:00