serai/spec/Getting Started.md

92 lines
1.6 KiB
Markdown
Raw Normal View History

# Getting Started
2023-03-25 05:44:07 +00:00
### Dependencies
##### Ubuntu
```
sudo apt-get install -y build-essential clang-11 pkg-config cmake git curl protobuf-compiler
```
### Install rustup
##### Linux
```
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
##### macOS
```
brew install rustup
```
### Install Rust
```
rustup update
2023-03-25 05:44:07 +00:00
rustup toolchain install stable
rustup target add wasm32-unknown-unknown
rustup toolchain install nightly
rustup target add wasm32-unknown-unknown --toolchain nightly
```
### Install Solidity
```
cargo install svm-rs
svm install 0.8.25
svm use 0.8.25
```
### Install Solidity Compiler Version Manager
```
cargo install svm-rs
svm install 0.8.25
svm use 0.8.25
```
### Install foundry (for tests)
```
cargo install --git https://github.com/foundry-rs/foundry --profile local --locked forge cast chisel anvil
```
### Clone and Build Serai
```
git clone https://github.com/serai-dex/serai
cd serai
cargo build --release --all-features
```
### Run Tests
2023-03-25 05:44:07 +00:00
Running tests requires:
- [A rootless Docker setup](https://docs.docker.com/engine/security/rootless/)
- A properly configured Bitcoin regtest node (available via Docker)
- A properly configured Monero regtest node (available via Docker)
- A properly configured monero-wallet-rpc instance (available via Docker)
Redo Dockerfile generation (#530) Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program. Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again. Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles. Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure. --- * Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles Enables greater templating. Also delete the unmaintained kubernetes folder *for now*. This should be restored in the future. * Use Dockerfiles from the orchestator * Ignore Dockerfiles in the git repo * Remove CI job to check Dockerfiles are as expected now that they're no longer committed * Remove old Dockerfiles from repo * Use Debian for monero-wallet-rpc * Remove replace_cmds for proper usage of entry-dev Consolidates ports a bit. Updates serai-docker-tests from "compose" to "build". * Only write a new dockerfile if it's distinct Preserves the updated time metadata. * Update serai-docker-tests * Correct the path Dockerfiles are built from * Correct inclusion of orchestration folder in Docker builds * Correct debug/release flagging in the cargo command Apparently, --debug isn't an effective NOP yet an error. * Correct path used to run the Serai node within a Dockerfile * Correct path in Monero Dockerfile * Attempt storing monerod in /usr/bin * Use sudo to move into /usr/bin in CI * Correct 18.3.0 to 18.3.1 * Escape * with quotes * Update deny.toml, ADD orchestration in runtime Dockerfile * Add --detach to the Monero GH CI * Diversify dockerfiles by network * Fixes to network-diversified orchestration * Bitcoin and Monero testnet scripts * Permissions and tweaks * Flatten scripts folders * Add missing folder specification to Monero Dockerfile * Have monero-wallet-rpc specify the monerod login * Have the Docker CMD specify env variables inserted at time of Dockerfile generation They're overrideable with the global enviornment as for tests. This enables variable generation in orchestrator and output to productionized Docker files without creating a life-long file within the Docker container. * Don't add Dockerfiles into Docker containers now that they have secrets Solely add the source code for them as needed to satisfy the workspace bounds. * Download arm64 Monero on arm64 * Ensure constant host architecture when reproducibly building the wasm Host architecture, for some reason, can effect the generated code despite the target architecture always being foreign to the host architecture. * Randomly generate infrastructure keys * Have orchestrator generate a key, be able to create/start containers * Ensure bash is used over sh * Clean dated docs * Change how quoting occurs * Standardize to sh * Have Docker test build the dev Dockerfiles * Only key_gen once * cargo update Adds a patch for zstd and reconciles the breaking nightly change which just occurred. * Use a dedicated network for Serai Also fixes SERAI_HOSTNAME passed to coordinator. * Support providing a key over the env for the Serai node * Enable and document running daemons for tests via serai-orchestrator Has running containers under the dev network port forward the RPC ports. * Use volumes for bitcoin/monero * Use bitcoin's run.sh in GH CI * Only use the volume for testnet (not dev)
2024-02-09 07:48:44 +00:00
To start the required daemons, one may run:
```
Redo Dockerfile generation (#530) Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program. Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again. Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles. Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure. --- * Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles Enables greater templating. Also delete the unmaintained kubernetes folder *for now*. This should be restored in the future. * Use Dockerfiles from the orchestator * Ignore Dockerfiles in the git repo * Remove CI job to check Dockerfiles are as expected now that they're no longer committed * Remove old Dockerfiles from repo * Use Debian for monero-wallet-rpc * Remove replace_cmds for proper usage of entry-dev Consolidates ports a bit. Updates serai-docker-tests from "compose" to "build". * Only write a new dockerfile if it's distinct Preserves the updated time metadata. * Update serai-docker-tests * Correct the path Dockerfiles are built from * Correct inclusion of orchestration folder in Docker builds * Correct debug/release flagging in the cargo command Apparently, --debug isn't an effective NOP yet an error. * Correct path used to run the Serai node within a Dockerfile * Correct path in Monero Dockerfile * Attempt storing monerod in /usr/bin * Use sudo to move into /usr/bin in CI * Correct 18.3.0 to 18.3.1 * Escape * with quotes * Update deny.toml, ADD orchestration in runtime Dockerfile * Add --detach to the Monero GH CI * Diversify dockerfiles by network * Fixes to network-diversified orchestration * Bitcoin and Monero testnet scripts * Permissions and tweaks * Flatten scripts folders * Add missing folder specification to Monero Dockerfile * Have monero-wallet-rpc specify the monerod login * Have the Docker CMD specify env variables inserted at time of Dockerfile generation They're overrideable with the global enviornment as for tests. This enables variable generation in orchestrator and output to productionized Docker files without creating a life-long file within the Docker container. * Don't add Dockerfiles into Docker containers now that they have secrets Solely add the source code for them as needed to satisfy the workspace bounds. * Download arm64 Monero on arm64 * Ensure constant host architecture when reproducibly building the wasm Host architecture, for some reason, can effect the generated code despite the target architecture always being foreign to the host architecture. * Randomly generate infrastructure keys * Have orchestrator generate a key, be able to create/start containers * Ensure bash is used over sh * Clean dated docs * Change how quoting occurs * Standardize to sh * Have Docker test build the dev Dockerfiles * Only key_gen once * cargo update Adds a patch for zstd and reconciles the breaking nightly change which just occurred. * Use a dedicated network for Serai Also fixes SERAI_HOSTNAME passed to coordinator. * Support providing a key over the env for the Serai node * Enable and document running daemons for tests via serai-orchestrator Has running containers under the dev network port forward the RPC ports. * Use volumes for bitcoin/monero * Use bitcoin's run.sh in GH CI * Only use the volume for testnet (not dev)
2024-02-09 07:48:44 +00:00
cargo run -p serai-orchestrator -- key_gen dev
cargo run -p serai-orchestrator -- setup dev
```
Redo Dockerfile generation (#530) Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program. Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again. Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles. Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure. --- * Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles Enables greater templating. Also delete the unmaintained kubernetes folder *for now*. This should be restored in the future. * Use Dockerfiles from the orchestator * Ignore Dockerfiles in the git repo * Remove CI job to check Dockerfiles are as expected now that they're no longer committed * Remove old Dockerfiles from repo * Use Debian for monero-wallet-rpc * Remove replace_cmds for proper usage of entry-dev Consolidates ports a bit. Updates serai-docker-tests from "compose" to "build". * Only write a new dockerfile if it's distinct Preserves the updated time metadata. * Update serai-docker-tests * Correct the path Dockerfiles are built from * Correct inclusion of orchestration folder in Docker builds * Correct debug/release flagging in the cargo command Apparently, --debug isn't an effective NOP yet an error. * Correct path used to run the Serai node within a Dockerfile * Correct path in Monero Dockerfile * Attempt storing monerod in /usr/bin * Use sudo to move into /usr/bin in CI * Correct 18.3.0 to 18.3.1 * Escape * with quotes * Update deny.toml, ADD orchestration in runtime Dockerfile * Add --detach to the Monero GH CI * Diversify dockerfiles by network * Fixes to network-diversified orchestration * Bitcoin and Monero testnet scripts * Permissions and tweaks * Flatten scripts folders * Add missing folder specification to Monero Dockerfile * Have monero-wallet-rpc specify the monerod login * Have the Docker CMD specify env variables inserted at time of Dockerfile generation They're overrideable with the global enviornment as for tests. This enables variable generation in orchestrator and output to productionized Docker files without creating a life-long file within the Docker container. * Don't add Dockerfiles into Docker containers now that they have secrets Solely add the source code for them as needed to satisfy the workspace bounds. * Download arm64 Monero on arm64 * Ensure constant host architecture when reproducibly building the wasm Host architecture, for some reason, can effect the generated code despite the target architecture always being foreign to the host architecture. * Randomly generate infrastructure keys * Have orchestrator generate a key, be able to create/start containers * Ensure bash is used over sh * Clean dated docs * Change how quoting occurs * Standardize to sh * Have Docker test build the dev Dockerfiles * Only key_gen once * cargo update Adds a patch for zstd and reconciles the breaking nightly change which just occurred. * Use a dedicated network for Serai Also fixes SERAI_HOSTNAME passed to coordinator. * Support providing a key over the env for the Serai node * Enable and document running daemons for tests via serai-orchestrator Has running containers under the dev network port forward the RPC ports. * Use volumes for bitcoin/monero * Use bitcoin's run.sh in GH CI * Only use the volume for testnet (not dev)
2024-02-09 07:48:44 +00:00
and then:
```
Redo Dockerfile generation (#530) Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program. Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again. Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles. Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure. --- * Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles Enables greater templating. Also delete the unmaintained kubernetes folder *for now*. This should be restored in the future. * Use Dockerfiles from the orchestator * Ignore Dockerfiles in the git repo * Remove CI job to check Dockerfiles are as expected now that they're no longer committed * Remove old Dockerfiles from repo * Use Debian for monero-wallet-rpc * Remove replace_cmds for proper usage of entry-dev Consolidates ports a bit. Updates serai-docker-tests from "compose" to "build". * Only write a new dockerfile if it's distinct Preserves the updated time metadata. * Update serai-docker-tests * Correct the path Dockerfiles are built from * Correct inclusion of orchestration folder in Docker builds * Correct debug/release flagging in the cargo command Apparently, --debug isn't an effective NOP yet an error. * Correct path used to run the Serai node within a Dockerfile * Correct path in Monero Dockerfile * Attempt storing monerod in /usr/bin * Use sudo to move into /usr/bin in CI * Correct 18.3.0 to 18.3.1 * Escape * with quotes * Update deny.toml, ADD orchestration in runtime Dockerfile * Add --detach to the Monero GH CI * Diversify dockerfiles by network * Fixes to network-diversified orchestration * Bitcoin and Monero testnet scripts * Permissions and tweaks * Flatten scripts folders * Add missing folder specification to Monero Dockerfile * Have monero-wallet-rpc specify the monerod login * Have the Docker CMD specify env variables inserted at time of Dockerfile generation They're overrideable with the global enviornment as for tests. This enables variable generation in orchestrator and output to productionized Docker files without creating a life-long file within the Docker container. * Don't add Dockerfiles into Docker containers now that they have secrets Solely add the source code for them as needed to satisfy the workspace bounds. * Download arm64 Monero on arm64 * Ensure constant host architecture when reproducibly building the wasm Host architecture, for some reason, can effect the generated code despite the target architecture always being foreign to the host architecture. * Randomly generate infrastructure keys * Have orchestrator generate a key, be able to create/start containers * Ensure bash is used over sh * Clean dated docs * Change how quoting occurs * Standardize to sh * Have Docker test build the dev Dockerfiles * Only key_gen once * cargo update Adds a patch for zstd and reconciles the breaking nightly change which just occurred. * Use a dedicated network for Serai Also fixes SERAI_HOSTNAME passed to coordinator. * Support providing a key over the env for the Serai node * Enable and document running daemons for tests via serai-orchestrator Has running containers under the dev network port forward the RPC ports. * Use volumes for bitcoin/monero * Use bitcoin's run.sh in GH CI * Only use the volume for testnet (not dev)
2024-02-09 07:48:44 +00:00
cargo run -p serai-orchestrator -- start dev bitcoin-daemon monero-daemon monero-wallet-rpc
```
Redo Dockerfile generation (#530) Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program. Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again. Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles. Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure. --- * Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles Enables greater templating. Also delete the unmaintained kubernetes folder *for now*. This should be restored in the future. * Use Dockerfiles from the orchestator * Ignore Dockerfiles in the git repo * Remove CI job to check Dockerfiles are as expected now that they're no longer committed * Remove old Dockerfiles from repo * Use Debian for monero-wallet-rpc * Remove replace_cmds for proper usage of entry-dev Consolidates ports a bit. Updates serai-docker-tests from "compose" to "build". * Only write a new dockerfile if it's distinct Preserves the updated time metadata. * Update serai-docker-tests * Correct the path Dockerfiles are built from * Correct inclusion of orchestration folder in Docker builds * Correct debug/release flagging in the cargo command Apparently, --debug isn't an effective NOP yet an error. * Correct path used to run the Serai node within a Dockerfile * Correct path in Monero Dockerfile * Attempt storing monerod in /usr/bin * Use sudo to move into /usr/bin in CI * Correct 18.3.0 to 18.3.1 * Escape * with quotes * Update deny.toml, ADD orchestration in runtime Dockerfile * Add --detach to the Monero GH CI * Diversify dockerfiles by network * Fixes to network-diversified orchestration * Bitcoin and Monero testnet scripts * Permissions and tweaks * Flatten scripts folders * Add missing folder specification to Monero Dockerfile * Have monero-wallet-rpc specify the monerod login * Have the Docker CMD specify env variables inserted at time of Dockerfile generation They're overrideable with the global enviornment as for tests. This enables variable generation in orchestrator and output to productionized Docker files without creating a life-long file within the Docker container. * Don't add Dockerfiles into Docker containers now that they have secrets Solely add the source code for them as needed to satisfy the workspace bounds. * Download arm64 Monero on arm64 * Ensure constant host architecture when reproducibly building the wasm Host architecture, for some reason, can effect the generated code despite the target architecture always being foreign to the host architecture. * Randomly generate infrastructure keys * Have orchestrator generate a key, be able to create/start containers * Ensure bash is used over sh * Clean dated docs * Change how quoting occurs * Standardize to sh * Have Docker test build the dev Dockerfiles * Only key_gen once * cargo update Adds a patch for zstd and reconciles the breaking nightly change which just occurred. * Use a dedicated network for Serai Also fixes SERAI_HOSTNAME passed to coordinator. * Support providing a key over the env for the Serai node * Enable and document running daemons for tests via serai-orchestrator Has running containers under the dev network port forward the RPC ports. * Use volumes for bitcoin/monero * Use bitcoin's run.sh in GH CI * Only use the volume for testnet (not dev)
2024-02-09 07:48:44 +00:00
Finally, to run the tests:
Redo Dockerfile generation (#530) Moves from concatted Dockerfiles to pseudo-templated Dockerfiles via a dedicated Rust program. Removes the unmaintained kubernetes, not because we shouldn't have/use it, but because it's unmaintained and needs to be reworked before it's present again. Replaces the compose with the work in the new orchestrator binary which spawns everything as expected. While this arguably re-invents the wheel, it correctly manages secrets and handles the variadic Dockerfiles. Also adds an unrelated patch for zstd and simplifies running services a bit by greater utilizing the existing infrastructure. --- * Delete all Dockerfile fragments, add new orchestator to generate Dockerfiles Enables greater templating. Also delete the unmaintained kubernetes folder *for now*. This should be restored in the future. * Use Dockerfiles from the orchestator * Ignore Dockerfiles in the git repo * Remove CI job to check Dockerfiles are as expected now that they're no longer committed * Remove old Dockerfiles from repo * Use Debian for monero-wallet-rpc * Remove replace_cmds for proper usage of entry-dev Consolidates ports a bit. Updates serai-docker-tests from "compose" to "build". * Only write a new dockerfile if it's distinct Preserves the updated time metadata. * Update serai-docker-tests * Correct the path Dockerfiles are built from * Correct inclusion of orchestration folder in Docker builds * Correct debug/release flagging in the cargo command Apparently, --debug isn't an effective NOP yet an error. * Correct path used to run the Serai node within a Dockerfile * Correct path in Monero Dockerfile * Attempt storing monerod in /usr/bin * Use sudo to move into /usr/bin in CI * Correct 18.3.0 to 18.3.1 * Escape * with quotes * Update deny.toml, ADD orchestration in runtime Dockerfile * Add --detach to the Monero GH CI * Diversify dockerfiles by network * Fixes to network-diversified orchestration * Bitcoin and Monero testnet scripts * Permissions and tweaks * Flatten scripts folders * Add missing folder specification to Monero Dockerfile * Have monero-wallet-rpc specify the monerod login * Have the Docker CMD specify env variables inserted at time of Dockerfile generation They're overrideable with the global enviornment as for tests. This enables variable generation in orchestrator and output to productionized Docker files without creating a life-long file within the Docker container. * Don't add Dockerfiles into Docker containers now that they have secrets Solely add the source code for them as needed to satisfy the workspace bounds. * Download arm64 Monero on arm64 * Ensure constant host architecture when reproducibly building the wasm Host architecture, for some reason, can effect the generated code despite the target architecture always being foreign to the host architecture. * Randomly generate infrastructure keys * Have orchestrator generate a key, be able to create/start containers * Ensure bash is used over sh * Clean dated docs * Change how quoting occurs * Standardize to sh * Have Docker test build the dev Dockerfiles * Only key_gen once * cargo update Adds a patch for zstd and reconciles the breaking nightly change which just occurred. * Use a dedicated network for Serai Also fixes SERAI_HOSTNAME passed to coordinator. * Support providing a key over the env for the Serai node * Enable and document running daemons for tests via serai-orchestrator Has running containers under the dev network port forward the RPC ports. * Use volumes for bitcoin/monero * Use bitcoin's run.sh in GH CI * Only use the volume for testnet (not dev)
2024-02-09 07:48:44 +00:00
```
cargo test --all-features
```