Commit graph

1532 commits

Author SHA1 Message Date
akildemir
627e7e6210
Add validator set rotation test for the node side (#532)
* add node side unit test

* complete rotation test for all networks

* set up the fast-epoch docker file

* fix pr comments
2024-02-24 14:51:06 -05:00
Luke Parker
019b42c0e0
fmt/clippy fixes 2024-02-19 22:33:56 -05:00
Justin Berman
079fddbaa6
monero: only mask user features on new polyseed, not on decode (#503)
* monero: only mask user features on new polyseed, not on decode

- This commit ensures a polyseed string that has unsupported features correctly errors on decode (rather than panic in debug build or return an incorrect successful response in prod build)
- Also avoids panicking when checksum calculation is unexpectedly wrong

Polyseed reference impl for feature masking:
- polyseed_create: b7c35bb3c6/src/polyseed.c (L61)
- polyseed_decode: b7c35bb3c6/src/polyseed.c (L212)

* PR comments

* Make from_internal a member of Polyseed

* Add accidentally removed newline

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-19 22:03:02 -05:00
Justin Berman
92d8b91be9
Monero: fix decoy selection algo and add test for latest spendable (#384)
* Monero: fix decoy selection algo and add test for latest spendable

- DSA only selected coinbase outputs and didn't match the wallet2
implementation
- Added test to make sure DSA will select a decoy output from the
most recent unlocked block
- Made usage of "height" in DSA consistent with other usage of
"height" in Monero code (height == num blocks in chain)
- Rely on monerod RPC response for output's unlocked status

* xmr runner tests mine until outputs are unlocked

* fingerprintable canoncial select decoys

* Separate fingerprintable canonical function

Makes it simpler for callers who are unconcered with consistent
canonical output selection across multiple clients to rely on
the simpler Decoy::select and not worry about fingerprintable
canonical

* fix merge conflicts

* Put back TODO for issue #104

* Fix incorrect check on distribution len

The RingCT distribution on mainnet doesn't start until well after
genesis, so the distribution length is expected to be < height.

To be clear, this was my mistake from this series of changes
to the DSA. I noticed this mistake because the DSA would error
when running on mainnet.
2024-02-19 21:34:10 -05:00
Justin Berman
4f1f7984a6
monero: added tx extra variants padding and mysterious minergate (#510)
* monero: read/write tx extra padding

* monero: read/write tx extra mysterious minergate variant

* Clippy

* monero: add tx extra test for minergate + pub key

* BufRead

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-19 21:22:00 -05:00
Justin Berman
cda14ac8b9
monero: Use fee priority enums from monero repo CLI/RPC wallets (#499)
* monero: Use fee priority enums from monero repo CLI/RPC wallets

* Update processor for fee priority change

* Remove FeePriority::Default

Done in consultation with @j-berman.

The RPC/CLI/GUI almost always adjust up except barring very explicit commands,
hence why FeePriority 0 is now only exposed via the explicit command of
FeePriority::Custom { priority: 0 }.

Also helps with terminology.

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-19 21:03:27 -05:00
Luke Parker
6f5d794f10
Median by Position (#533)
* use median price instead of the highest sustained

* add test for lexicographically reversing a byte slice

* fix pr comments

* fix CI fail

* fix dex tests

* Use a fuzz-tested list of prices

* Working median algorithm based on position + lints

---------

Co-authored-by: akildemir <aeg_asd@hotmail.com>
2024-02-19 20:50:04 -05:00
j-berman
34b93b882c monero: scan all tx pub keys (not additional) for every tx
wallet2's behavior is explained more fully here:
https://github.com/UkoeHB/monero/issues/27
2024-02-19 20:48:37 -05:00
Justin Berman
0880453f82
monero: make dummy payment ID zeroes when it's included in a tx (#514)
* monero: make dummy payment ID zeroes when it's included in a tx

Also did some minor cleaning of InternalPayment::Change

* Lint

* Clarify comment
2024-02-19 20:45:50 -05:00
Justin Berman
ebdfc9afb4
monero: test xmr send that requires additional pub keys (#516)
* Test xmr send that requires additional pub keys

* Clippy
2024-02-19 20:18:31 -05:00
Luke Parker
f6409d08f3
Increase timeout in coordinator tests 2024-02-18 08:19:07 -05:00
Luke Parker
c41a8ac8f2
Revert "rocksdb 0.22 via a patch"
This reverts commit c05c511938.

rocksdb 0.22 does not work on Windows at this time.
2024-02-18 08:17:26 -05:00
akildemir
d88aa90ec2
support input encoded data for bitcoin network (#486)
* add input script check

* add test

* optimizations

* bug fix

* fix pr comments

* Test SegWit-encoded data using a single output (not two)

* Remove TODO used as a question, document origins when SegWit encoding

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-18 07:43:44 -05:00
Luke Parker
c05c511938
rocksdb 0.22 via a patch
patch is removeable once https://github.com/paritytech/parity-common/pull/828
is merged and released.
2024-02-18 05:32:04 -05:00
Justin Berman
df85c09435
monero: match monero's stricter check when decompressing points (#515)
* monero: match monero's stricter check when decompressing points

* Reverted type change for output key
2024-02-17 23:16:16 -05:00
Luke Parker
62a619a312
Have monerod be chown'd to monero:nogroup
On some Docker setups, the monero user doesn't have a monero group for some
reason. This handles that edge case.
2024-02-10 20:58:04 -05:00
Luke Parker
95b7460907
Use Debian instead of Alpine for monero on testnet 2024-02-10 20:57:55 -05:00
Luke Parker
95c3cfc52e
Add restart policy to Docker containers 2024-02-09 08:43:33 -05:00
Luke Parker
f0694172ef
Fix potential generation of invalid SignData in shim 2024-02-09 02:52:08 -05:00
Luke Parker
29633ada1b
Rust 1.76 2024-02-09 02:51:24 -05:00
Luke Parker
337e54c672
Redo Dockerfile generation (#530)
Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program.

Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again.

Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles.

Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure.

---

* Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles

Enables greater templating.

Also delete the unmaintained kubernetes folder *for now*. This should be
restored in the future.

* Use Dockerfiles from the orchestator

* Ignore Dockerfiles in the git repo

* Remove CI job to check Dockerfiles are as expected now that they're no longer committed

* Remove old Dockerfiles from repo

* Use Debian for monero-wallet-rpc

* Remove replace_cmds for proper usage of entry-dev

Consolidates ports a bit.

Updates serai-docker-tests from "compose" to "build".

* Only write a new dockerfile if it's distinct

Preserves the updated time metadata.

* Update serai-docker-tests

* Correct the path Dockerfiles are built from

* Correct inclusion of orchestration folder in Docker builds

* Correct debug/release flagging in the cargo command

Apparently, --debug isn't an effective NOP yet an error.

* Correct path used to run the Serai node within a Dockerfile

* Correct path in Monero Dockerfile

* Attempt storing monerod in /usr/bin

* Use sudo to move into /usr/bin in CI

* Correct 18.3.0 to 18.3.1

* Escape * with quotes

* Update deny.toml, ADD orchestration in runtime Dockerfile

* Add --detach to the Monero GH CI

* Diversify dockerfiles by network

* Fixes to network-diversified orchestration

* Bitcoin and Monero testnet scripts

* Permissions and tweaks

* Flatten scripts folders

* Add missing folder specification to Monero Dockerfile

* Have monero-wallet-rpc specify the monerod login

* Have the Docker CMD specify env variables inserted at time of Dockerfile generation

They're overrideable with the global enviornment as for tests. This enables
variable generation in orchestrator and output to productionized Docker files
without creating a life-long file within the Docker container.

* Don't add Dockerfiles into Docker containers now that they have secrets

Solely add the source code for them as needed to satisfy the workspace bounds.

* Download arm64 Monero on arm64

* Ensure constant host architecture when reproducibly building the wasm

Host architecture, for some reason, can effect the generated code despite the
target architecture always being foreign to the host architecture.

* Randomly generate infrastructure keys

* Have orchestrator generate a key, be able to create/start containers

* Ensure bash is used over sh

* Clean dated docs

* Change how quoting occurs

* Standardize to sh

* Have Docker test build the dev Dockerfiles

* Only key_gen once

* cargo update

Adds a patch for zstd and reconciles the breaking nightly change which just
occurred.

* Use a dedicated network for Serai

Also fixes SERAI_HOSTNAME passed to coordinator.

* Support providing a key over the env for the Serai node

* Enable and document running daemons for tests via serai-orchestrator

Has running containers under the dev network port forward the RPC ports.

* Use volumes for bitcoin/monero

* Use bitcoin's run.sh in GH CI

* Only use the volume for testnet (not dev)
2024-02-09 02:48:44 -05:00
akildemir
347d4cf413
Fix tendermint distinct precommit bug (#517)
* fix tendermint distinct precommit bug

* remove conflicting precommit error
2024-02-08 13:47:37 -05:00
Luke Parker
aaff74575f
Remove unused brew packages on macOS (#531)
* Remove unused brew packages on macOS

* Remove reference to Docker in macOS CI

* Remove gems, explicitly test Intel and m1 macOS

* Allow gem to error since it still mostly runs
2024-02-05 23:53:57 -05:00
akildemir
ad0ecc5185
complete various todos in tributary (#520)
* complete various todos

* fix pr comments

* Document bounds on unique hashes in TransactionKind

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-05 03:50:55 -05:00
Luke Parker
af12cec3b9
cargo update
Resolves deny warning around unintended behavior change (without semver bump).
2024-02-04 03:50:48 -05:00
Luke Parker
89788be034
macOS clippy (#526)
* Specifically use bash as a shell to try and get rustup to work on Windows

* Use bash for the call to echo

* Add macOS clippy

* Debug why git diff failed

* Restore macos-latest to matrix

* Allow whitespace before the fact 0 lines were modified

* Add LC_ALL env variable to grep

* Replace usage of -P with -e
2024-02-01 21:31:02 -05:00
GitHub Actions
745075af6e Update nightly 2024-02-01 21:03:27 -05:00
Luke Parker
9b25d0dad7
Update to node 20 GitHub cache action 2024-02-01 20:52:17 -05:00
Luke Parker
2b76e41c9a
Directly install protobuf-compiler without using an external action (#524)
* Directly install protobuf-compiler without using an external action

* Remove unused "github-token" input
2024-01-31 19:21:26 -05:00
Luke Parker
05219c3ce8
Windows Clippy (#525)
* Add windows clippy

* Adjust build-dependencies for Linux/Windows

* Specifically use bash as a shell to try and get rustup to work on Windows

* Use bash for the call to echo
2024-01-31 19:10:39 -05:00
Luke Parker
cc75b52a43
Don't allow constructing unusable serai_client::bitcoin::Address es 2024-01-31 17:54:43 -05:00
Luke Parker
4913873b10
Slash reports (#523)
* report_slashes plumbing in Substrate

Notably delays the SetRetired event until it provides a slash report or the set
after it becomes the set to report its slashes.

* Add dedicated AcceptedHandover event

* Add SlashReport TX to Tributary

* Create SlashReport TXs

* Handle SlashReport TXs

* Add logic to generate a SlashReport to the coordinator

* Route SlashReportSigner into the processor

* Finish routing the SlashReport signing/TX publication

* Add serai feature to processor's serai-client
2024-01-29 03:48:53 -05:00
rlking
0b8c7ade6e
Add scripts to create monero wallet rpc container (#521)
* create Dockerfile for monero wallet rpc with dockerfiles.sh

* make monero wallet rpc docker accessible from outside

* connect wallet-rpc with monerod

* add generated Dockerfile for monero wallet rpc

* add monero wallet rpcs to docker profiles

* update getting started guide to refer to wallet rpc docker
2024-01-28 20:58:23 -05:00
Luke Parker
21262d41e6
Resolve latest clippy and a couple no longer needed fmt notes 2024-01-22 22:13:37 -05:00
Luke Parker
508f7eb23a
cargo update
Pseudo-resolves shlex advisory (due to the deprecation of the vulnerable
functions, which hopefully should prevent their use). shlex is only used by
bindgen, a sufficiently trusted dependency.
2024-01-22 22:08:37 -05:00
Luke Parker
90df391170
cargo update
Resolves h2 disclosure (which shouldn't have affected us).
2024-01-19 11:44:49 -05:00
Luke Parker
9d3d47fc9f
hyper-rustls 0.26 (#519)
* hyper-rustls 0.25

This isn't worth it until our dependencies update to rustls 0.22 as well.

* hyper-rustls 0.26 and hyper 1.0
2024-01-16 19:32:30 -05:00
akildemir
6691f16292
remove mach patch 2024-01-16 12:06:50 -05:00
Luke Parker
9c06cbccad
Document immunity to https://github.com/paritytech/polkadot-sdk/issues/2947 now that I have permission to disclose it 2024-01-16 12:06:08 -05:00
Justin Berman
c507ab9fd6
monero: match varint decoding (#513)
* monero: match varint decoding

* Fix build and clippy
2024-01-11 03:15:11 -05:00
Luke Parker
3aa8007700
Add missing unwap to processor's test fn 2024-01-06 01:01:19 -05:00
Luke Parker
1ba2d8d832
Make monero-serai Block::number not panic on invalid blocks 2024-01-06 00:03:14 -05:00
Boog900
e7b0ed3e7e Check miner tx has a miner input when deserializing. 2024-01-05 23:49:43 -05:00
Luke Parker
f3429ec1ef
Inside publish (for a Serai transaction from the coordinator), use RetiredDb over latest session
Not only is this more performant, the definition of retired won't be if a newer
session is active. It will be if the session has posted a slash report or the
stake for that session has unlocked.

Initial commit towards implementing SlashReports.
2024-01-05 23:40:15 -05:00
Luke Parker
1cff9b4264
Patch proc-macro-crate 2 to proc-macro-crate 3
Updates toml_edit to 0.21.
2024-01-05 23:40:15 -05:00
j-berman
3c5a82e915 monero: investigated TODO and can remove it
The behavior appears to match monero core. monero core isn't
throwing an exception in the linked code, it's returning
boost::none (and logging an error) which is the same functional
behavior as finding that the output does not belong to the user.
2024-01-05 12:18:10 -05:00
Boog900
93e85c5ce6
Monero: use only the first input ring length for RCT deserialization. (#504)
* Use only the first input ring length for all RCT input signatures.

This is what Monero does:
ac02af9286/src/ringct/rctTypes.h (L422)

https://github.com/monero-project/monero/blob/master/src/cryptonote_basic/cryptonote_basic.h#L308-L309

This isn't an issue for current transactions as from hf 12 Monero requires
all inputs to have the same number of decoys but for transactions before
that Monero would reject RCT txs with differing ring lengths. Monero would
deserialize each inputs signature using the ring length of the first so the
signatures for inputs other than the first would have a different
(wrong) number of elements for that input meaning the signature is invalid.

But as we are using the ring length of each input, which arguably is the
*correct* way, we would approve of transactions with inputs differing in
ring lengths.

* Check that there is more than one ring member for MLSAG signatures.

ac02af9286/src/ringct/rctSigs.cpp (L462)
2024-01-05 00:02:16 -05:00
Luke Parker
617ec604ee
cargo update
Resolves the deny CI failure.
2024-01-04 01:46:26 -05:00
Justin Berman
265261d3ba
monero: require seed lang when decoding seed (#502)
* monero: require seed lang when decoding seed

- Require the seed language when decoding a Classic|Polyseed seed string
	- As per https://github.com/monero-project/monero/issues/9089 and https://github.com/tevador/polyseed/issues/11
	- Fixes #478
	- Implementation note: I reused the `SeedType` enum and required it as a param to `Seed::from_string` because it seemed simplest, but perhaps there is a cleaner way to require the seed lang.
- Made sure the print statements from #487 print the seed as early as possible to help debug future issues
- A future PR could support deducing which languages a seed decodes to in order to support the UX @kayabaNerve suggested in https://github.com/monero-project/monero/issues/9089:
	- "Wallets can also try to abstract [language specification], by decoding with all languages, and only asking the user if/when multiple valid options show up ("Is this seed Spanish or Italian?")."

* Lint
2024-01-04 01:32:42 -05:00
Luke Parker
7eb388e546
PR to track down CI failures (#501)
* Use an extended timeout for DKGs specifically

* Add a log statement when message-queue connection fails

* Add a 60 second keep-alive to connections

* Use zalloc for processor/message-queue/coordinator

An additional layer which protects us against edge cases with Zeroizing
(objects which don't support it or don't miss it).

* Add further logs to message-queue

* Further increase re-attempt timeouts in CI

* Remove misplaced continue inmessage-queue client

Fixes observed CI failures.

* Revert "Further increase re-attempt timeouts in CI"

This reverts commit 3723530cf6.
2024-01-04 01:08:13 -05:00