Commit graph

1688 commits

Author SHA1 Message Date
Luke Parker
6c8a0bfda6
Limit docker logs to 300MB per container 2024-03-06 21:49:55 -05:00
Luke Parker
06c23368f2
Mitigate https://github.com/serai-dex/serai/issues/539 by making keystore deterministic 2024-03-06 21:37:40 -05:00
Luke Parker
5629c94b8b
Reconcile the two copies of scalar_vector.rs in monero-serai 2024-03-02 17:15:16 -05:00
Luke Parker
b427f4b8ab
Revert "Add bootnodes"
This reverts commit 1096ddb7ea.

This commit was intended for the testnet branch alone.
2024-02-26 10:47:47 -05:00
Luke Parker
1096ddb7ea
Add bootnodes 2024-02-26 10:46:16 -05:00
Luke Parker
5487844b9e
clippy && cargo update 2024-02-25 18:37:15 -05:00
akildemir
627e7e6210
Add validator set rotation test for the node side (#532)
* add node side unit test

* complete rotation test for all networks

* set up the fast-epoch docker file

* fix pr comments
2024-02-24 14:51:06 -05:00
Luke Parker
019b42c0e0
fmt/clippy fixes 2024-02-19 22:33:56 -05:00
Justin Berman
079fddbaa6
monero: only mask user features on new polyseed, not on decode (#503)
* monero: only mask user features on new polyseed, not on decode

- This commit ensures a polyseed string that has unsupported features correctly errors on decode (rather than panic in debug build or return an incorrect successful response in prod build)
- Also avoids panicking when checksum calculation is unexpectedly wrong

Polyseed reference impl for feature masking:
- polyseed_create: b7c35bb3c6/src/polyseed.c (L61)
- polyseed_decode: b7c35bb3c6/src/polyseed.c (L212)

* PR comments

* Make from_internal a member of Polyseed

* Add accidentally removed newline

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-19 22:03:02 -05:00
Justin Berman
92d8b91be9
Monero: fix decoy selection algo and add test for latest spendable (#384)
* Monero: fix decoy selection algo and add test for latest spendable

- DSA only selected coinbase outputs and didn't match the wallet2
implementation
- Added test to make sure DSA will select a decoy output from the
most recent unlocked block
- Made usage of "height" in DSA consistent with other usage of
"height" in Monero code (height == num blocks in chain)
- Rely on monerod RPC response for output's unlocked status

* xmr runner tests mine until outputs are unlocked

* fingerprintable canoncial select decoys

* Separate fingerprintable canonical function

Makes it simpler for callers who are unconcered with consistent
canonical output selection across multiple clients to rely on
the simpler Decoy::select and not worry about fingerprintable
canonical

* fix merge conflicts

* Put back TODO for issue #104

* Fix incorrect check on distribution len

The RingCT distribution on mainnet doesn't start until well after
genesis, so the distribution length is expected to be < height.

To be clear, this was my mistake from this series of changes
to the DSA. I noticed this mistake because the DSA would error
when running on mainnet.
2024-02-19 21:34:10 -05:00
Justin Berman
4f1f7984a6
monero: added tx extra variants padding and mysterious minergate (#510)
* monero: read/write tx extra padding

* monero: read/write tx extra mysterious minergate variant

* Clippy

* monero: add tx extra test for minergate + pub key

* BufRead

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-19 21:22:00 -05:00
Justin Berman
cda14ac8b9
monero: Use fee priority enums from monero repo CLI/RPC wallets (#499)
* monero: Use fee priority enums from monero repo CLI/RPC wallets

* Update processor for fee priority change

* Remove FeePriority::Default

Done in consultation with @j-berman.

The RPC/CLI/GUI almost always adjust up except barring very explicit commands,
hence why FeePriority 0 is now only exposed via the explicit command of
FeePriority::Custom { priority: 0 }.

Also helps with terminology.

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-19 21:03:27 -05:00
Luke Parker
6f5d794f10
Median by Position (#533)
* use median price instead of the highest sustained

* add test for lexicographically reversing a byte slice

* fix pr comments

* fix CI fail

* fix dex tests

* Use a fuzz-tested list of prices

* Working median algorithm based on position + lints

---------

Co-authored-by: akildemir <aeg_asd@hotmail.com>
2024-02-19 20:50:04 -05:00
j-berman
34b93b882c monero: scan all tx pub keys (not additional) for every tx
wallet2's behavior is explained more fully here:
https://github.com/UkoeHB/monero/issues/27
2024-02-19 20:48:37 -05:00
Justin Berman
0880453f82
monero: make dummy payment ID zeroes when it's included in a tx (#514)
* monero: make dummy payment ID zeroes when it's included in a tx

Also did some minor cleaning of InternalPayment::Change

* Lint

* Clarify comment
2024-02-19 20:45:50 -05:00
Justin Berman
ebdfc9afb4
monero: test xmr send that requires additional pub keys (#516)
* Test xmr send that requires additional pub keys

* Clippy
2024-02-19 20:18:31 -05:00
Luke Parker
f6409d08f3
Increase timeout in coordinator tests 2024-02-18 08:19:07 -05:00
Luke Parker
c41a8ac8f2
Revert "rocksdb 0.22 via a patch"
This reverts commit c05c511938.

rocksdb 0.22 does not work on Windows at this time.
2024-02-18 08:17:26 -05:00
akildemir
d88aa90ec2
support input encoded data for bitcoin network (#486)
* add input script check

* add test

* optimizations

* bug fix

* fix pr comments

* Test SegWit-encoded data using a single output (not two)

* Remove TODO used as a question, document origins when SegWit encoding

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-18 07:43:44 -05:00
Luke Parker
c05c511938
rocksdb 0.22 via a patch
patch is removeable once https://github.com/paritytech/parity-common/pull/828
is merged and released.
2024-02-18 05:32:04 -05:00
Justin Berman
df85c09435
monero: match monero's stricter check when decompressing points (#515)
* monero: match monero's stricter check when decompressing points

* Reverted type change for output key
2024-02-17 23:16:16 -05:00
Luke Parker
62a619a312
Have monerod be chown'd to monero:nogroup
On some Docker setups, the monero user doesn't have a monero group for some
reason. This handles that edge case.
2024-02-10 20:58:04 -05:00
Luke Parker
95b7460907
Use Debian instead of Alpine for monero on testnet 2024-02-10 20:57:55 -05:00
Luke Parker
95c3cfc52e
Add restart policy to Docker containers 2024-02-09 08:43:33 -05:00
Luke Parker
f0694172ef
Fix potential generation of invalid SignData in shim 2024-02-09 02:52:08 -05:00
Luke Parker
29633ada1b
Rust 1.76 2024-02-09 02:51:24 -05:00
Luke Parker
337e54c672
Redo Dockerfile generation (#530)
Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program.

Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again.

Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles.

Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure.

---

* Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles

Enables greater templating.

Also delete the unmaintained kubernetes folder *for now*. This should be
restored in the future.

* Use Dockerfiles from the orchestator

* Ignore Dockerfiles in the git repo

* Remove CI job to check Dockerfiles are as expected now that they're no longer committed

* Remove old Dockerfiles from repo

* Use Debian for monero-wallet-rpc

* Remove replace_cmds for proper usage of entry-dev

Consolidates ports a bit.

Updates serai-docker-tests from "compose" to "build".

* Only write a new dockerfile if it's distinct

Preserves the updated time metadata.

* Update serai-docker-tests

* Correct the path Dockerfiles are built from

* Correct inclusion of orchestration folder in Docker builds

* Correct debug/release flagging in the cargo command

Apparently, --debug isn't an effective NOP yet an error.

* Correct path used to run the Serai node within a Dockerfile

* Correct path in Monero Dockerfile

* Attempt storing monerod in /usr/bin

* Use sudo to move into /usr/bin in CI

* Correct 18.3.0 to 18.3.1

* Escape * with quotes

* Update deny.toml, ADD orchestration in runtime Dockerfile

* Add --detach to the Monero GH CI

* Diversify dockerfiles by network

* Fixes to network-diversified orchestration

* Bitcoin and Monero testnet scripts

* Permissions and tweaks

* Flatten scripts folders

* Add missing folder specification to Monero Dockerfile

* Have monero-wallet-rpc specify the monerod login

* Have the Docker CMD specify env variables inserted at time of Dockerfile generation

They're overrideable with the global enviornment as for tests. This enables
variable generation in orchestrator and output to productionized Docker files
without creating a life-long file within the Docker container.

* Don't add Dockerfiles into Docker containers now that they have secrets

Solely add the source code for them as needed to satisfy the workspace bounds.

* Download arm64 Monero on arm64

* Ensure constant host architecture when reproducibly building the wasm

Host architecture, for some reason, can effect the generated code despite the
target architecture always being foreign to the host architecture.

* Randomly generate infrastructure keys

* Have orchestrator generate a key, be able to create/start containers

* Ensure bash is used over sh

* Clean dated docs

* Change how quoting occurs

* Standardize to sh

* Have Docker test build the dev Dockerfiles

* Only key_gen once

* cargo update

Adds a patch for zstd and reconciles the breaking nightly change which just
occurred.

* Use a dedicated network for Serai

Also fixes SERAI_HOSTNAME passed to coordinator.

* Support providing a key over the env for the Serai node

* Enable and document running daemons for tests via serai-orchestrator

Has running containers under the dev network port forward the RPC ports.

* Use volumes for bitcoin/monero

* Use bitcoin's run.sh in GH CI

* Only use the volume for testnet (not dev)
2024-02-09 02:48:44 -05:00
akildemir
347d4cf413
Fix tendermint distinct precommit bug (#517)
* fix tendermint distinct precommit bug

* remove conflicting precommit error
2024-02-08 13:47:37 -05:00
Luke Parker
aaff74575f
Remove unused brew packages on macOS (#531)
* Remove unused brew packages on macOS

* Remove reference to Docker in macOS CI

* Remove gems, explicitly test Intel and m1 macOS

* Allow gem to error since it still mostly runs
2024-02-05 23:53:57 -05:00
akildemir
ad0ecc5185
complete various todos in tributary (#520)
* complete various todos

* fix pr comments

* Document bounds on unique hashes in TransactionKind

---------

Co-authored-by: Luke Parker <lukeparker5132@gmail.com>
2024-02-05 03:50:55 -05:00
Luke Parker
af12cec3b9
cargo update
Resolves deny warning around unintended behavior change (without semver bump).
2024-02-04 03:50:48 -05:00
Luke Parker
89788be034
macOS clippy (#526)
* Specifically use bash as a shell to try and get rustup to work on Windows

* Use bash for the call to echo

* Add macOS clippy

* Debug why git diff failed

* Restore macos-latest to matrix

* Allow whitespace before the fact 0 lines were modified

* Add LC_ALL env variable to grep

* Replace usage of -P with -e
2024-02-01 21:31:02 -05:00
GitHub Actions
745075af6e Update nightly 2024-02-01 21:03:27 -05:00
Luke Parker
9b25d0dad7
Update to node 20 GitHub cache action 2024-02-01 20:52:17 -05:00
Luke Parker
2b76e41c9a
Directly install protobuf-compiler without using an external action (#524)
* Directly install protobuf-compiler without using an external action

* Remove unused "github-token" input
2024-01-31 19:21:26 -05:00
Luke Parker
05219c3ce8
Windows Clippy (#525)
* Add windows clippy

* Adjust build-dependencies for Linux/Windows

* Specifically use bash as a shell to try and get rustup to work on Windows

* Use bash for the call to echo
2024-01-31 19:10:39 -05:00
Luke Parker
cc75b52a43
Don't allow constructing unusable serai_client::bitcoin::Address es 2024-01-31 17:54:43 -05:00
Luke Parker
4913873b10
Slash reports (#523)
* report_slashes plumbing in Substrate

Notably delays the SetRetired event until it provides a slash report or the set
after it becomes the set to report its slashes.

* Add dedicated AcceptedHandover event

* Add SlashReport TX to Tributary

* Create SlashReport TXs

* Handle SlashReport TXs

* Add logic to generate a SlashReport to the coordinator

* Route SlashReportSigner into the processor

* Finish routing the SlashReport signing/TX publication

* Add serai feature to processor's serai-client
2024-01-29 03:48:53 -05:00
rlking
0b8c7ade6e
Add scripts to create monero wallet rpc container (#521)
* create Dockerfile for monero wallet rpc with dockerfiles.sh

* make monero wallet rpc docker accessible from outside

* connect wallet-rpc with monerod

* add generated Dockerfile for monero wallet rpc

* add monero wallet rpcs to docker profiles

* update getting started guide to refer to wallet rpc docker
2024-01-28 20:58:23 -05:00
Luke Parker
21262d41e6
Resolve latest clippy and a couple no longer needed fmt notes 2024-01-22 22:13:37 -05:00
Luke Parker
508f7eb23a
cargo update
Pseudo-resolves shlex advisory (due to the deprecation of the vulnerable
functions, which hopefully should prevent their use). shlex is only used by
bindgen, a sufficiently trusted dependency.
2024-01-22 22:08:37 -05:00
Luke Parker
90df391170
cargo update
Resolves h2 disclosure (which shouldn't have affected us).
2024-01-19 11:44:49 -05:00
Luke Parker
9d3d47fc9f
hyper-rustls 0.26 (#519)
* hyper-rustls 0.25

This isn't worth it until our dependencies update to rustls 0.22 as well.

* hyper-rustls 0.26 and hyper 1.0
2024-01-16 19:32:30 -05:00
akildemir
6691f16292
remove mach patch 2024-01-16 12:06:50 -05:00
Luke Parker
9c06cbccad
Document immunity to https://github.com/paritytech/polkadot-sdk/issues/2947 now that I have permission to disclose it 2024-01-16 12:06:08 -05:00
Justin Berman
c507ab9fd6
monero: match varint decoding (#513)
* monero: match varint decoding

* Fix build and clippy
2024-01-11 03:15:11 -05:00
Luke Parker
3aa8007700
Add missing unwap to processor's test fn 2024-01-06 01:01:19 -05:00
Luke Parker
1ba2d8d832
Make monero-serai Block::number not panic on invalid blocks 2024-01-06 00:03:14 -05:00
Boog900
e7b0ed3e7e Check miner tx has a miner input when deserializing. 2024-01-05 23:49:43 -05:00
Luke Parker
f3429ec1ef
Inside publish (for a Serai transaction from the coordinator), use RetiredDb over latest session
Not only is this more performant, the definition of retired won't be if a newer
session is active. It will be if the session has posted a slash report or the
stake for that session has unlocked.

Initial commit towards implementing SlashReports.
2024-01-05 23:40:15 -05:00