Commit graph

1770 commits

Author SHA1 Message Date
akildemir
d88aa90ec2
support input encoded data for bitcoin network (#486)
* add input script check

* add test

* optimizations

* bug fix

* fix pr comments

* Test SegWit-encoded data using a single output (not two)

* Remove TODO used as a question, document origins when SegWit encoding

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-18 07:43:44 -05:00
Luke Parker
c05c511938
rocksdb 0.22 via a patch
patch is removeable once https://github.com/paritytech/parity-common/pull/828
is merged and released.
2024-02-18 05:32:04 -05:00
Justin Berman
df85c09435
monero: match monero's stricter check when decompressing points (#515)
* monero: match monero's stricter check when decompressing points

* Reverted type change for output key
2024-02-17 23:16:16 -05:00
Luke Parker
62a619a312
Have monerod be chown'd to monero:nogroup
On some Docker setups, the monero user doesn't have a monero group for some
reason. This handles that edge case.
2024-02-10 20:58:04 -05:00
Luke Parker
95b7460907
Use Debian instead of Alpine for monero on testnet 2024-02-10 20:57:55 -05:00
Luke Parker
95c3cfc52e
Add restart policy to Docker containers 2024-02-09 08:43:33 -05:00
Luke Parker
f0694172ef
Fix potential generation of invalid SignData in shim 2024-02-09 02:52:08 -05:00
Luke Parker
29633ada1b
Rust 1.76 2024-02-09 02:51:24 -05:00
Luke Parker
337e54c672
Redo Dockerfile generation (#530)
Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program.

Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again.

Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles.

Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure.

---

* Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles

Enables greater templating.

Also delete the unmaintained kubernetes folder *for now*. This should be
restored in the future.

* Use Dockerfiles from the orchestator

* Ignore Dockerfiles in the git repo

* Remove CI job to check Dockerfiles are as expected now that they're no longer committed

* Remove old Dockerfiles from repo

* Use Debian for monero-wallet-rpc

* Remove replace_cmds for proper usage of entry-dev

Consolidates ports a bit.

Updates serai-docker-tests from "compose" to "build".

* Only write a new dockerfile if it's distinct

Preserves the updated time metadata.

* Update serai-docker-tests

* Correct the path Dockerfiles are built from

* Correct inclusion of orchestration folder in Docker builds

* Correct debug/release flagging in the cargo command

Apparently, --debug isn't an effective NOP yet an error.

* Correct path used to run the Serai node within a Dockerfile

* Correct path in Monero Dockerfile

* Attempt storing monerod in /usr/bin

* Use sudo to move into /usr/bin in CI

* Correct 18.3.0 to 18.3.1

* Escape * with quotes

* Update deny.toml, ADD orchestration in runtime Dockerfile

* Add --detach to the Monero GH CI

* Diversify dockerfiles by network

* Fixes to network-diversified orchestration

* Bitcoin and Monero testnet scripts

* Permissions and tweaks

* Flatten scripts folders

* Add missing folder specification to Monero Dockerfile

* Have monero-wallet-rpc specify the monerod login

* Have the Docker CMD specify env variables inserted at time of Dockerfile generation

They're overrideable with the global enviornment as for tests. This enables
variable generation in orchestrator and output to productionized Docker files
without creating a life-long file within the Docker container.

* Don't add Dockerfiles into Docker containers now that they have secrets

Solely add the source code for them as needed to satisfy the workspace bounds.

* Download arm64 Monero on arm64

* Ensure constant host architecture when reproducibly building the wasm

Host architecture, for some reason, can effect the generated code despite the
target architecture always being foreign to the host architecture.

* Randomly generate infrastructure keys

* Have orchestrator generate a key, be able to create/start containers

* Ensure bash is used over sh

* Clean dated docs

* Change how quoting occurs

* Standardize to sh

* Have Docker test build the dev Dockerfiles

* Only key_gen once

* cargo update

Adds a patch for zstd and reconciles the breaking nightly change which just
occurred.

* Use a dedicated network for Serai

Also fixes SERAI_HOSTNAME passed to coordinator.

* Support providing a key over the env for the Serai node

* Enable and document running daemons for tests via serai-orchestrator

Has running containers under the dev network port forward the RPC ports.

* Use volumes for bitcoin/monero

* Use bitcoin's run.sh in GH CI

* Only use the volume for testnet (not dev)
2024-02-09 02:48:44 -05:00
akildemir
347d4cf413
Fix tendermint distinct precommit bug (#517)
* fix tendermint distinct precommit bug

* remove conflicting precommit error
2024-02-08 13:47:37 -05:00
Luke Parker
aaff74575f
Remove unused brew packages on macOS (#531)
* Remove unused brew packages on macOS

* Remove reference to Docker in macOS CI

* Remove gems, explicitly test Intel and m1 macOS

* Allow gem to error since it still mostly runs
2024-02-05 23:53:57 -05:00
akildemir
ad0ecc5185
complete various todos in tributary (#520)
* complete various todos

* fix pr comments

* Document bounds on unique hashes in TransactionKind

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-05 03:50:55 -05:00
Luke Parker
af12cec3b9
cargo update
Resolves deny warning around unintended behavior change (without semver bump).
2024-02-04 03:50:48 -05:00
Luke Parker
89788be034
macOS clippy (#526)
* Specifically use bash as a shell to try and get rustup to work on Windows

* Use bash for the call to echo

* Add macOS clippy

* Debug why git diff failed

* Restore macos-latest to matrix

* Allow whitespace before the fact 0 lines were modified

* Add LC_ALL env variable to grep

* Replace usage of -P with -e
2024-02-01 21:31:02 -05:00
GitHub Actions
745075af6e Update nightly 2024-02-01 21:03:27 -05:00
Luke Parker
9b25d0dad7
Update to node 20 GitHub cache action 2024-02-01 20:52:17 -05:00
Luke Parker
2b76e41c9a
Directly install protobuf-compiler without using an external action (#524)
* Directly install protobuf-compiler without using an external action

* Remove unused "github-token" input
2024-01-31 19:21:26 -05:00
Luke Parker
05219c3ce8
Windows Clippy (#525)
* Add windows clippy

* Adjust build-dependencies for Linux/Windows

* Specifically use bash as a shell to try and get rustup to work on Windows

* Use bash for the call to echo
2024-01-31 19:10:39 -05:00
Luke Parker
cc75b52a43
Don't allow constructing unusable serai_client::bitcoin::Address es 2024-01-31 17:54:43 -05:00
Luke Parker
4913873b10
Slash reports (#523)
* report_slashes plumbing in Substrate

Notably delays the SetRetired event until it provides a slash report or the set
after it becomes the set to report its slashes.

* Add dedicated AcceptedHandover event

* Add SlashReport TX to Tributary

* Create SlashReport TXs

* Handle SlashReport TXs

* Add logic to generate a SlashReport to the coordinator

* Route SlashReportSigner into the processor

* Finish routing the SlashReport signing/TX publication

* Add serai feature to processor's serai-client
2024-01-29 03:48:53 -05:00
rlking
0b8c7ade6e
Add scripts to create monero wallet rpc container (#521)
* create Dockerfile for monero wallet rpc with dockerfiles.sh

* make monero wallet rpc docker accessible from outside

* connect wallet-rpc with monerod

* add generated Dockerfile for monero wallet rpc

* add monero wallet rpcs to docker profiles

* update getting started guide to refer to wallet rpc docker
2024-01-28 20:58:23 -05:00
Luke Parker
21262d41e6
Resolve latest clippy and a couple no longer needed fmt notes 2024-01-22 22:13:37 -05:00
Luke Parker
508f7eb23a
cargo update
Pseudo-resolves shlex advisory (due to the deprecation of the vulnerable
functions, which hopefully should prevent their use). shlex is only used by
bindgen, a sufficiently trusted dependency.
2024-01-22 22:08:37 -05:00
Luke Parker
90df391170
cargo update
Resolves h2 disclosure (which shouldn't have affected us).
2024-01-19 11:44:49 -05:00
Luke Parker
9d3d47fc9f
hyper-rustls 0.26 (#519)
* hyper-rustls 0.25

This isn't worth it until our dependencies update to rustls 0.22 as well.

* hyper-rustls 0.26 and hyper 1.0
2024-01-16 19:32:30 -05:00
akildemir
6691f16292
remove mach patch 2024-01-16 12:06:50 -05:00
Luke Parker
9c06cbccad
Document immunity to https://github.com/paritytech/polkadot-sdk/issues/2947 now that I have permission to disclose it 2024-01-16 12:06:08 -05:00
Justin Berman
c507ab9fd6
monero: match varint decoding (#513)
* monero: match varint decoding

* Fix build and clippy
2024-01-11 03:15:11 -05:00
Luke Parker
3aa8007700
Add missing unwap to processor's test fn 2024-01-06 01:01:19 -05:00
Luke Parker
1ba2d8d832
Make monero-serai Block::number not panic on invalid blocks 2024-01-06 00:03:14 -05:00
Boog900
e7b0ed3e7e Check miner tx has a miner input when deserializing. 2024-01-05 23:49:43 -05:00
Luke Parker
f3429ec1ef
Inside publish (for a Serai transaction from the coordinator), use RetiredDb over latest session
Not only is this more performant, the definition of retired won't be if a newer
session is active. It will be if the session has posted a slash report or the
stake for that session has unlocked.

Initial commit towards implementing SlashReports.
2024-01-05 23:40:15 -05:00
Luke Parker
1cff9b4264
Patch proc-macro-crate 2 to proc-macro-crate 3
Updates toml_edit to 0.21.
2024-01-05 23:40:15 -05:00
j-berman
3c5a82e915 monero: investigated TODO and can remove it
The behavior appears to match monero core. monero core isn't
throwing an exception in the linked code, it's returning
boost::none (and logging an error) which is the same functional
behavior as finding that the output does not belong to the user.
2024-01-05 12:18:10 -05:00
Boog900
93e85c5ce6
Monero: use only the first input ring length for RCT deserialization. (#504)
* Use only the first input ring length for all RCT input signatures.

This is what Monero does:
ac02af9286/src/ringct/rctTypes.h (L422)

https://github.com/monero-project/monero/blob/master/src/cryptonote_basic/cryptonote_basic.h#L308-L309

This isn't an issue for current transactions as from hf 12 Monero requires
all inputs to have the same number of decoys but for transactions before
that Monero would reject RCT txs with differing ring lengths. Monero would
deserialize each inputs signature using the ring length of the first so the
signatures for inputs other than the first would have a different
(wrong) number of elements for that input meaning the signature is invalid.

But as we are using the ring length of each input, which arguably is the
*correct* way, we would approve of transactions with inputs differing in
ring lengths.

* Check that there is more than one ring member for MLSAG signatures.

ac02af9286/src/ringct/rctSigs.cpp (L462)
2024-01-05 00:02:16 -05:00
Luke Parker
617ec604ee
cargo update
Resolves the deny CI failure.
2024-01-04 01:46:26 -05:00
Justin Berman
265261d3ba
monero: require seed lang when decoding seed (#502)
* monero: require seed lang when decoding seed

- Require the seed language when decoding a Classic|Polyseed seed string
	- As per https://github.com/monero-project/monero/issues/9089 and https://github.com/tevador/polyseed/issues/11
	- Fixes #478
	- Implementation note: I reused the `SeedType` enum and required it as a param to `Seed::from_string` because it seemed simplest, but perhaps there is a cleaner way to require the seed lang.
- Made sure the print statements from #487 print the seed as early as possible to help debug future issues
- A future PR could support deducing which languages a seed decodes to in order to support the UX @kayabaNerve suggested in https://github.com/monero-project/monero/issues/9089:
	- "Wallets can also try to abstract [language specification], by decoding with all languages, and only asking the user if/when multiple valid options show up ("Is this seed Spanish or Italian?")."

* Lint
2024-01-04 01:32:42 -05:00
Luke Parker
7eb388e546
PR to track down CI failures (#501)
* Use an extended timeout for DKGs specifically

* Add a log statement when message-queue connection fails

* Add a 60 second keep-alive to connections

* Use zalloc for processor/message-queue/coordinator

An additional layer which protects us against edge cases with Zeroizing
(objects which don't support it or don't miss it).

* Add further logs to message-queue

* Further increase re-attempt timeouts in CI

* Remove misplaced continue inmessage-queue client

Fixes observed CI failures.

* Revert "Further increase re-attempt timeouts in CI"

This reverts commit 3723530cf6.
2024-01-04 01:08:13 -05:00
Luke Parker
6c8040f723
Restore release for serai-node to obtain sane bootup times 2023-12-30 23:59:00 -05:00
Luke Parker
02776c54a8
Increase reattempt delays in the GH CI, which is extremely latent 2023-12-30 22:11:04 -05:00
Luke Parker
ec8dfd4639
Correct SignData serialization test from creating 256 signers of data
This overflows the u8 allowed and caused a CI failure. The actual
code/assumption is fine.
2023-12-30 19:08:29 -05:00
Luke Parker
99e05e4e5e
Add patches folder to runtime Dockerfile 2023-12-30 18:36:43 -05:00
Luke Parker
a72b547824
Add patches folder to Dockerfiles 2023-12-30 13:49:41 -05:00
Luke Parker
bad3d210ba
rust 1.75 2023-12-30 03:26:32 -05:00
Luke Parker
8c676d98c5
Tweaks from cargo update and patches 2023-12-30 03:26:11 -05:00
Luke Parker
890b70212a
Patch matches, mach 2023-12-30 02:52:05 -05:00
Luke Parker
9f7140c3db
Patch is-terminal to the std-included IsTerminal 2023-12-30 02:48:26 -05:00
Luke Parker
8b26a85faa
Add patches for directories-next/option-ext
The rational is detailed in the root Cargo.toml.

While I don't personally mind MPL dependencies, even if I don't prefer them
(they're allowed in the deny.toml for a reason), I do mind the pointless scope
creep and wish to highlight how little it actually used from the crate by
re-defining it as the single function.

We could also fork directories-next, or directories, and remove the usage of
option-ext per https://github.com/dirs-dev/dirs-sys-rs/issues/24, yet that'd be
a much larger task than what was done here.

In the future, it may be beneficial to submit a PR to wasmtime replacing
directories-next with home, a cargo-team maintained library to get the home
directory and associated folders. An example migration can be found at
https://github.com/harryfei/which-rs/pull/80.
2023-12-30 02:44:33 -05:00
Luke Parker
24ea65eae9
cargo update 2023-12-30 02:36:51 -05:00
Luke Parker
fff8dcb827
Document usage of latest_decided in AuthorityDiscoveryApi 2023-12-23 21:28:50 -05:00