cuprate/.github/workflows/ci.yml

141 lines
4.1 KiB
YAML
Raw Normal View History

2023-10-26 00:39:57 +00:00
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
2023-10-26 00:39:57 +00:00
env:
# Show colored output in CI.
2023-10-26 00:39:57 +00:00
CARGO_TERM_COLOR: always
# Show full panics.
RUST_BACKTRACE: "full"
# Increase thread stack size to 8 megabytes.
RUST_MIN_STACK: 8000000
# Fail on documentation warnings.
RUSTDOCFLAGS: '-D warnings'
# Enable debug information generation for build dependencies.
CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG: true
2023-10-26 00:39:57 +00:00
jobs:
# Run format separately.
#
# This will fast-cancel other CI early if this fails.
#
# `cargo fmt` checks _all_ code, regardless of the OS
# or any `#[cfg]`'s, so this only needs to run on Linux.
2024-01-22 22:41:20 +00:00
fmt:
runs-on: ubuntu-latest
2023-10-26 00:39:57 +00:00
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Format
run: cargo fmt --all --check
# Run typo checker separately.
# This will fast-cancel other CI early if this fails.
typo:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Spell Check
uses: crate-ci/typos@master
# All other CI.
ci:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [windows-latest, macos-latest, ubuntu-latest]
include:
- os: windows-latest
shell: msys2 {0}
# GNU Windows is used as we need
# `unistd.h` and more in `cryptonight/`.
rust: stable-x86_64-pc-windows-gnu
- os: macos-latest
shell: bash
rust: stable
- os: ubuntu-latest
shell: bash
rust: stable
defaults:
run:
shell: ${{ matrix.shell }}
2023-12-03 00:29:12 +00:00
2024-01-22 22:30:19 +00:00
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Install Rust
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ matrix.rust }}
components: clippy
- name: Cache
uses: actions/cache@v3
with:
path: target
key: ${{ matrix.os }}
- name: Download monerod
uses: ./.github/actions/monerod-download
# Packages other than `Boost` used by `Monero` are listed here.
# https://github.com/monero-project/monero/blob/c444a7e002036e834bfb4c68f04a121ce1af5825/.github/workflows/build.yml#L71
- name: Install dependencies (Linux)
if: matrix.os == 'ubuntu-latest'
run: sudo apt install -y libboost-dev
- name: Install dependencies (macOS)
if: matrix.os == 'macos-latest'
run: HOMEBREW_NO_AUTO_UPDATE=1 brew install boost
- name: Install dependencies (Windows)
if: matrix.os == 'windows-latest'
uses: msys2/setup-msys2@v2
with:
path-type: inherit
update: true
install: mingw-w64-x86_64-toolchain mingw-w64-x86_64-boost msys2-runtime-devel git mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja
# HACK: 2024-05-14
# GCC 14+ fails to build `lmdb-master-sys` with no clear error message:
# <https://github.com/Cuprate/cuprate/pull/127>
#
# - MSYS2 repos carry older versions of packages
# - pacman lets us manually downgrade from package files
# - Note that `gcc` requires `gcc-libs`
- name: Downgrade to GCC 13.2 (Windows)
if: matrix.os == 'windows-latest'
run: |
wget https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-gcc-13.2.0-6-any.pkg.tar.zst https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-gcc-libs-13.2.0-6-any.pkg.tar.zst
pacman -U --noconfirm mingw-w64-x86_64-gcc-13.2.0-6-any.pkg.tar.zst mingw-w64-x86_64-gcc-libs-13.2.0-6-any.pkg.tar.zst
- name: Documentation
run: cargo doc --workspace --all-features --no-deps
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
- name: Clippy (fail on warnings)
run: cargo clippy --workspace --all-features --all-targets -- -D warnings
database: impl trait function bodies for `heed` & `redb` (#85) * env: remove `T: Table` for `Env::create_tables()` It creates _all_ tables, not a specific `T: Table` * heed: half impl `Env::open()`, some TODOs * heed: add `HeedTxR{o,w}<'env>` * workspace/cargo: add `parking_lot` * remove `parking_lot` `MappedRwLockGuard` doesn't solve the `returning reference to object owned by function` error when returning heed's lock guard + the tx so we'll be going with `std` * env: add `{EnvInner,TxRoInput,TxRwInput}` and getter `fn()`s * env: fix tx <-> table lifetimes, fix `Env::create_tables()` * heed: impl `DatabaseRo` * heed: impl `DatabaseRw` * database: add `src/backend/${BACKEND}/tests.rs` * heed: impl more of `Env::open()` * redb: fix trait signatures, add `trait ValueGuard` * accommodate `DatabaseRo` bounds for both `{heed,redb}` * fold `get_range()` generic + bounds * add `TxCreator`, add `heed` tests * env: `TxCreator` -> `EnvInner` which doubles as DB/Table opener * database: `DatabaseRw<'tx>` -> `DatabaseRw<'db, 'tx>` * heed: `db_read_write()` test * database: fix `get()` lifetimes, heed: add `db_read_write()` test * database: remove `'env` lifetime from `DatabaseRo<'env, 'tx>` not needed for immutable references * redb: fix new `{Env, EnvInner, DatabaseR{o,w}}` bounds * redb: impl `Database` traits * redb: impl `TxR{o,w}` traits * redb: impl `Env` * redb: open/create tables in `Env::open` * redb: enable tests, add tmp `Storable` printlns * redb: fix alignment issue with `Cow` and `from_bytes_unaligned()` * redb: only allocate bytes when alignment != 1 * heed: remove custom iterator from `range()` * storable: conditionally allocat on misaligned bytes in `from_bytes` * add database guard * database: AccessGuard -> ValueGuard * storable: add `ALIGN` and `from_bytes_unaligned()` * redb: 1 serde type `StorableRedb`, fix impl * cow serde, trait bounds, fix backend's where bounds - Uses Cow for `redb`'s deserialization - Conforms `heed` to use Cow (but not as the actual key/value) - Conforms the `cuprate_database` trait API to use Cow - Adds `ToOwned + Debug` (and permutation) trait bounds - Solves 23098573148968945687345349657398 compiler errors due to aforementioned trait bounds, causing `where` to be everywhere * fix docs, use fully qualified Tx functions for tests * backend: check value guard contains value in test * add `Storable::ALIGN` tests, doc TODOs * add `trait ToOwnedDebug` * traits: `ToOwned + Debug` -> `ToOwnedDebug` * heed: remove `ToOwned` + `Debug` bounds * redb: remove `ToOwned` + `Debug` bounds * add `ValueGuard`, replace signatures, fix `redb` * heed: fix for `ValueGuard` * docs, tests * redb: add `CowRange` for `T::Key` -> `Cow<'_, T::Key>` conversion * separate `config.rs` -> `config/` * ci: combine tests, run both `heed` and `redb` tests * ci: fix workflow * backend: add `resize()` test * ci: remove windows-specific update * ci: remove update + windows default set * backend: add `unreachable` tests, fix docs * trait docs * ci: fix * Update database/src/backend/heed/env.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/env.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/redb/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/redb/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * readme: fix `value_guard.rs` * heed: remove unneeded clippy + fix formatting * heed: create and use `create_table()` in `Env::open()` * redb: create and use `create_table()` in `Env::open()` * redb: remove unneeded clippy * fix clippy, remove `<[u8], [u8]>` docs --------- Co-authored-by: Boog900 <54e72d8a-345f-4599-bd90-c6b9bc7d0ec5@aleeas.com> Co-authored-by: Boog900 <boog900@tutanota.com>
2024-03-13 22:05:24 +00:00
# HACK: how to test both DB backends that are feature-gated?
- name: Test
database: impl trait function bodies for `heed` & `redb` (#85) * env: remove `T: Table` for `Env::create_tables()` It creates _all_ tables, not a specific `T: Table` * heed: half impl `Env::open()`, some TODOs * heed: add `HeedTxR{o,w}<'env>` * workspace/cargo: add `parking_lot` * remove `parking_lot` `MappedRwLockGuard` doesn't solve the `returning reference to object owned by function` error when returning heed's lock guard + the tx so we'll be going with `std` * env: add `{EnvInner,TxRoInput,TxRwInput}` and getter `fn()`s * env: fix tx <-> table lifetimes, fix `Env::create_tables()` * heed: impl `DatabaseRo` * heed: impl `DatabaseRw` * database: add `src/backend/${BACKEND}/tests.rs` * heed: impl more of `Env::open()` * redb: fix trait signatures, add `trait ValueGuard` * accommodate `DatabaseRo` bounds for both `{heed,redb}` * fold `get_range()` generic + bounds * add `TxCreator`, add `heed` tests * env: `TxCreator` -> `EnvInner` which doubles as DB/Table opener * database: `DatabaseRw<'tx>` -> `DatabaseRw<'db, 'tx>` * heed: `db_read_write()` test * database: fix `get()` lifetimes, heed: add `db_read_write()` test * database: remove `'env` lifetime from `DatabaseRo<'env, 'tx>` not needed for immutable references * redb: fix new `{Env, EnvInner, DatabaseR{o,w}}` bounds * redb: impl `Database` traits * redb: impl `TxR{o,w}` traits * redb: impl `Env` * redb: open/create tables in `Env::open` * redb: enable tests, add tmp `Storable` printlns * redb: fix alignment issue with `Cow` and `from_bytes_unaligned()` * redb: only allocate bytes when alignment != 1 * heed: remove custom iterator from `range()` * storable: conditionally allocat on misaligned bytes in `from_bytes` * add database guard * database: AccessGuard -> ValueGuard * storable: add `ALIGN` and `from_bytes_unaligned()` * redb: 1 serde type `StorableRedb`, fix impl * cow serde, trait bounds, fix backend's where bounds - Uses Cow for `redb`'s deserialization - Conforms `heed` to use Cow (but not as the actual key/value) - Conforms the `cuprate_database` trait API to use Cow - Adds `ToOwned + Debug` (and permutation) trait bounds - Solves 23098573148968945687345349657398 compiler errors due to aforementioned trait bounds, causing `where` to be everywhere * fix docs, use fully qualified Tx functions for tests * backend: check value guard contains value in test * add `Storable::ALIGN` tests, doc TODOs * add `trait ToOwnedDebug` * traits: `ToOwned + Debug` -> `ToOwnedDebug` * heed: remove `ToOwned` + `Debug` bounds * redb: remove `ToOwned` + `Debug` bounds * add `ValueGuard`, replace signatures, fix `redb` * heed: fix for `ValueGuard` * docs, tests * redb: add `CowRange` for `T::Key` -> `Cow<'_, T::Key>` conversion * separate `config.rs` -> `config/` * ci: combine tests, run both `heed` and `redb` tests * ci: fix workflow * backend: add `resize()` test * ci: remove windows-specific update * ci: remove update + windows default set * backend: add `unreachable` tests, fix docs * trait docs * ci: fix * Update database/src/backend/heed/env.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/env.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/redb/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/redb/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * readme: fix `value_guard.rs` * heed: remove unneeded clippy + fix formatting * heed: create and use `create_table()` in `Env::open()` * redb: create and use `create_table()` in `Env::open()` * redb: remove unneeded clippy * fix clippy, remove `<[u8], [u8]>` docs --------- Co-authored-by: Boog900 <54e72d8a-345f-4599-bd90-c6b9bc7d0ec5@aleeas.com> Co-authored-by: Boog900 <boog900@tutanota.com>
2024-03-13 22:05:24 +00:00
run: |
cargo test --all-features --workspace
cargo test --package cuprate-blockchain --no-default-features --features redb --features service
# TODO: upload binaries with `actions/upload-artifact@v3`
- name: Build
run: cargo build --all-features --all-targets --workspace