Commit graph

20 commits

Author SHA1 Message Date
hinto-janai
4876e159e7
repo: add skeleton directories, fix labels (#139)
* `database/` -> `storage/`, create skeleton `storage/` crates

* add skeleton root directories

* books: add skeleton

* ci: fix labeler.yml

* fix CONTRIBUTING.md

* fix workspace Cargo.toml

* fix `storage/`

* rename helper fs fn, `cuprate_database` -> `cuprate_blockchain`

* fix Cargo.toml

* cuprate-blockchain: revert find-and-replace

Since it has to be ported to the book anyway, there's not much
reason to update this and create review diffs.

* labeler.yml: remove `A-docs` from `books/` changes

Although books are documentation,
the `A-books` label already exists for this
2024-05-29 02:18:30 +01:00
hinto-janai
45656fe653
repo: adopt rust-lang label system (#136)
* contributing: add `Keeping track of issues and PRs` section

* add `labeler.yml`

* more labels

* update labels

* update `.github/labeler.yml`

* update `CONTRIBUTING.md`

* doc `.github` files

* wording fix

* add `A-dependency`
2024-05-27 00:57:21 +01:00
hinto-janai
d0ac94a813
ci: downgrade to gcc 13.2 for windows (#127) 2024-05-14 19:41:39 +01:00
hinto-janai
10d327ca13
ci: use dtolnay/rust-toolchain (#124)
* ci: don't cache `.cargo/`, `.rustup/`

* ci: use `dtolnay/rust-toolchain@master`
2024-05-05 15:22:05 +01:00
88f7d1f212
download monerod in CI (#123)
* download monerod in CI

* move action file

* add macOS arm support

* remove reqwest from workspace

* undo whitespace changes

* fix indentation

* update comments

* add monerod to .gitignore

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* add back spacing

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-05-01 01:21:08 +01:00
hinto-janai
2ac90420c6
ci: fix windows/macos (#116)
* cryptonight: include homebrew directories

* test-utils: add arm macos to `monerod` downloader

* ci: install windows-gnu
2024-04-27 00:49:55 +01:00
hinto-janai
8f22d8ab79
database: impl trait function bodies for heed & redb (#85)
* env: remove `T: Table` for `Env::create_tables()`

It creates _all_ tables, not a specific `T: Table`

* heed: half impl `Env::open()`, some TODOs

* heed: add `HeedTxR{o,w}<'env>`

* workspace/cargo: add `parking_lot`

* remove `parking_lot`

`MappedRwLockGuard` doesn't solve the `returning reference to
object owned by function` error when returning heed's lock guard
+ the tx so we'll be going with `std`

* env: add `{EnvInner,TxRoInput,TxRwInput}` and getter `fn()`s

* env: fix tx <-> table lifetimes, fix `Env::create_tables()`

* heed: impl `DatabaseRo`

* heed: impl `DatabaseRw`

* database: add `src/backend/${BACKEND}/tests.rs`

* heed: impl more of `Env::open()`

* redb: fix trait signatures, add `trait ValueGuard`

* accommodate `DatabaseRo` bounds for both `{heed,redb}`

* fold `get_range()` generic + bounds

* add `TxCreator`, add `heed` tests

* env: `TxCreator` -> `EnvInner` which doubles as DB/Table opener

* database: `DatabaseRw<'tx>` -> `DatabaseRw<'db, 'tx>`

* heed: `db_read_write()` test

* database: fix `get()` lifetimes, heed: add `db_read_write()` test

* database: remove `'env` lifetime from `DatabaseRo<'env, 'tx>`

not needed for immutable references

* redb: fix new `{Env, EnvInner, DatabaseR{o,w}}` bounds

* redb: impl `Database` traits

* redb: impl `TxR{o,w}` traits

* redb: impl `Env`

* redb: open/create tables in `Env::open`

* redb: enable tests, add tmp `Storable` printlns

* redb: fix alignment issue with `Cow` and `from_bytes_unaligned()`

* redb: only allocate bytes when alignment != 1

* heed: remove custom iterator from `range()`

* storable: conditionally allocat on misaligned bytes in `from_bytes`

* add database guard

* database: AccessGuard -> ValueGuard

* storable: add `ALIGN` and `from_bytes_unaligned()`

* redb: 1 serde type `StorableRedb`, fix impl

* cow serde, trait bounds, fix backend's where bounds

- Uses Cow for `redb`'s deserialization
- Conforms `heed` to use Cow (but not as the actual key/value)
- Conforms the `cuprate_database` trait API to use Cow
- Adds `ToOwned + Debug` (and permutation) trait bounds
- Solves 23098573148968945687345349657398 compiler errors due
  to aforementioned trait bounds, causing `where` to be everywhere

* fix docs, use fully qualified Tx functions for tests

* backend: check value guard contains value in test

* add `Storable::ALIGN` tests, doc TODOs

* add `trait ToOwnedDebug`

* traits: `ToOwned + Debug` -> `ToOwnedDebug`

* heed: remove `ToOwned` + `Debug` bounds

* redb: remove `ToOwned` + `Debug` bounds

* add `ValueGuard`, replace signatures, fix `redb`

* heed: fix for `ValueGuard`

* docs, tests

* redb: add `CowRange` for `T::Key` -> `Cow<'_, T::Key>` conversion

* separate `config.rs` -> `config/`

* ci: combine tests, run both `heed` and `redb` tests

* ci: fix workflow

* backend: add `resize()` test

* ci: remove windows-specific update

* ci: remove update + windows default set

* backend: add `unreachable` tests, fix docs

* trait docs

* ci: fix

* Update database/src/backend/heed/env.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/heed/env.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/heed/transaction.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/heed/transaction.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/heed/transaction.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/redb/database.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/redb/database.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update database/src/backend/heed/database.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* readme: fix `value_guard.rs`

* heed: remove unneeded clippy + fix formatting

* heed: create and use `create_table()` in `Env::open()`

* redb: create and use `create_table()` in `Env::open()`

* redb: remove unneeded clippy

* fix clippy, remove `<[u8], [u8]>` docs

---------

Co-authored-by: Boog900 <54e72d8a-345f-4599-bd90-c6b9bc7d0ec5@aleeas.com>
Co-authored-by: Boog900 <boog900@tutanota.com>
2024-03-13 22:05:24 +00:00
hinto-janai
9a8e306fd5
Add cargo deny to CI (#34)
* add `deny.toml`

* deny Copyleft licenses

* deny yanked crate usage

* add common targets

* allow common licenses

* allow AGPL-3.0 for Cuprate crates

* misc fixes
2024-02-29 16:44:17 +00:00
34bfe37673
CI: fix audit (#76)
* add `--locked`

* cargo update
2024-02-25 20:50:58 +00:00
hinto-janai
354ac9c2f6
Add typos + cargo doc CI (#32)
* ci: add separate `typo` job

* add `typos.toml` for false positives

* fix all typos

* ci: add `cargo doc` step

* fix doc errors

* contributing.md: update passing CI steps

* fix more typos, add exception to `cryptonight/`

* ci: move `cargo doc` step within `ci` job

It needs dependencies.

* ci: add https://github.com/Cuprate/cuprate/pull/63

* test-utils: fix typo

* ci: switch `rustup update` and switch order

* ci: only update rust on unix

* ci: set `RUSTDOCFLAGS` env earlier

* ci: only run `cargo doc` on linux

* ci: remove `bash` on `cargo doc`

* ci: remove `--all-targets`

We now have the target OS's in CI, no need to compile for each.

* contributing.md: update ci steps

* ci: add `--all-targets` back to clippy, build

* update contributing.md
2024-02-15 16:03:04 +00:00
hinto-janai
331d4d3e7f
Database (#35)
* rename `database` -> `old_database`

Keeping it around for reference until new implementation is complete.

* create new `database/` skeleton

* add `DATABASE.md` design doc skeleton

* move design doc to `database/README.md`

* add rough code

* `lib.rs` -> `gist.rs`

* database: use workspace deps

* workspace: include `database/` as member

CI will now include this crate.

* cargo fmt

* database: `AGPL` -> `MIT`

* readme: add `TODO`s

* add base files

* cargo.toml: add `heed` feature

* fix clippy

* lib.rs: add extremely pedantic lints

* readme: add `# Backends`

* cargo.toml: add `cfg-if`

* add `backend/` structure

* base `database.rs`

* cargo.toml: add `borsh`

* backend: add `DATABASE_BACKEND`

* base `error.rs`

* base `database.rs`

* base `transaction.rs`

* base `table.rs`

* lib.rs: add imports

* add `pod.rs`

* pod: use `Read/Write`, add tests for all primitive numbers

* pod: impl Pod for `Vec<u8>`, `[u8; N]`

* pod: add docs, add `private::Sealed`

* pod: `to_writer`, `from_reader`

The new `as_bytes` + `from_bytes` now allows (de)serializing from
bytes directly instead of going through Read/Write.

Different array return sizes are abstracted away with `-> impl AsRef<[u8]>`

* pod: impl Pod for `Box<[u8]>`

* pod: return `Err` on incorrect length in `from_bytes()`

* pod: docs

* pod: impl Pod for `Arc<[u8]>`

* readme: docs

* database: add `create_table()`, `get_table()`

* table: `Pod` bound

* backend: move into directories

* pod: add `into_bytes()`

* heed: impl `BytesEncode`, `BytesDecode`

* add `actor`, `service` features

The thread/actor system used will be gated behind `actor`,
and the `tower/tokio` integration will be gated behind `service`.

* add `lib.rs` docs

* service: add `service.rs`

* service: add `reader.rs`

* service: add `writer.rs`

* service: add `request.rs` & `response.rs`

* cargo.toml: add `crossbeam`

* service: add/use `enum Request`, `enum Response`

* service: basic signatures for thread-pools, `Writer` -> `Writers`

* service: split `tower::Service<ReadRequest/WriteRequest>`

* service: impl `tower::Service` for `Database(Reader|Writer)`

* service: add `init()`, impl basic `Reader/Writer` pools

* service: add example `Request`'s

* service: add example `ReadRequest` handling

* temporarily allow clippy lints

* readme: update file structure

* transaction: add `RoTx::get_range()`

* service: module docs

* cargo.toml: add `cuprate-helper`

* service: scale readers/writers based on thread count

* database: change lifetimes

* heed: impl Database for `heed`

* heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits

* docs

* service: `read.rs` docs

* service: `write.rs` docs

* service: request/response docs

* service: use `OnceLock` in `init()`, add `db_read()`, `db_write()`

* service: leak database into `&'static`

* database: add `#[inline]`, `#[cold]`

* service: `free.rs` docs, more `#[inline]` + `#[cold]`

* service: add `shutdown()`

* service: add `Request::Shutdown`

* service: `shutdown()` docs

* heed: hide concrete tx types

* lib.rs: add terms

* split `Env` <-> `Database`

* cargo.toml: add `paste`

* database: add `tables/`

* impl `serde/borsh` where possible

* tables: add `Tables`, add test docs

* make db backend mutually exclusive to fix `--all-features`

* tables: use `$()*` in `tables!()`

* cargo.toml: add `sanakirja 1.4.0`

* sanakirja: impl `Env`

* sanakirja: impl `Database`

* sanakirja: impl `Transaction`

* table: temporarily fix `sanakirja` K/V bounds

* table: fix `#[cfg]`

* cargo.toml: fix deps

* lib.rs: docs

* service: docs

* readme: add files, update `# Documentation`, add `# Layers`

* readme: `src/` file purpose

* readme: `src/service/` file purpose

* readme: `src/backend/` file purpose

* fix `Cargo.lock` merge conflict

* database: remove `gist.rs`

* add to `constants.rs`

* add top `//! comments` for files/modules

* constants: add sanity-check test

* service: add `only_one_database` test in `free.rs`

* service: add `tests.rs`

* remove unneeded markers + imports

* backend: fix `get_range()`'s trait `impl` return

* env: add `create_tables_if_needed()`, don't return `Option<Db>`

* sort imports by `CONTRIBUTING.md` rules

oops sorry boog

* add `monero.rs`

* monero: docs

* database: add missing `RoTx/RwTx` inputs

* backend: add missing `RoTx/RwTx` inputs

* `monero.rs` trait -> free functions in `ops/`

* pod: make methods infallible

* ci: add `rustup update` step

* service: use `Arc` instead of leaking, remove `db_read/db_write`

* service: use `InfallibleOneshotReceiver` for readers

* service: shutdown on error, add todos

* service: remove `Request`

* service: combine `ReadResponse` and `WriteResponse`

* service: use `InfallibleOneshotReceiver` for writer

* service: only spawn 1 writer, don't allow cloning writer handle

* table: add associated `const CONSTANT_SIZE`

* add `key.rs` + `trait Key`, add bound to `Table`

* fix typos
2024-02-13 17:43:25 +00:00
hinto-janai
630faed263
ci: include macos + windows (#52)
* ci: install boost, include macos + windows

* cryptonight: fix `MSVC`

* cryptonight: use `flag_if_supported()`

* fix cryptonight builds

* update randomX

* fix rx builds

* add memwipe

* include memwipe.c in build

* spawn monerod in msys2 for windows

* fix last commit

* install dependencies before spawning monerod

* remove --detach

* try another way of spawning monerod

* add /I

* download and spawn monerod as a part of tests

* add download.rs

* extend time for monerod spawn

* move sleep and show monerod output

* fix clippy

* change stdin to pipped

* #[cfg(unix)] on bytes::Buf

* fix macos capitalisation

* remove tar.bz2 on macos expected dir

* remove zip on windows expected dir

* fix todo

* add docs

* fix a couple typos

---------

Co-authored-by: Boog900 <54e72d8a-345f-4599-bd90-c6b9bc7d0ec5@aleeas.com>
Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-12 13:39:15 +00:00
Boog900
1b8bfb8a32
CI: add back in monerod spawn 2024-01-22 23:08:06 +00:00
Boog900
74e04056a9
CI: change cache key 2024-01-22 22:41:20 +00:00
Boog900
0662a125e1
update CI 2024-01-22 22:32:43 +00:00
Boog900
34dd105a0c
make CI more strict 2023-12-03 00:29:12 +00:00
8557073c15
p2p changes (#38)
* start re-working p2p to work with change monero-wire

* start re-working p2p to work with change monero-wire

adds back some changes from #22

* change the peer module to use the new API + fix a couple bugs

* remove peer set for now

* add try_from/from conversion between `Message` and
`Request`/`Response`

* Allow specifying other parameters in levin-cuprate

* add new `LevinCommand` enum and clean up monero-wire message de/encoding

* fix issues with merge

* start splitting up p2p crate into smaller crates.

* add monerod action from serai to test network code

* remove tracing in tests
2023-11-30 18:09:05 +00:00
Boog900
e1eaaf80d9
fix CI: Install dependencies (#33)
install boost, remove all targets except ubuntu
2023-10-29 01:11:55 +00:00
hinto.janai
39b029f3e7
add CI actions 2023-10-25 21:07:13 -04:00
Someone Else
61aa3e7121
setup basic Rust Github Actions 2023-03-20 22:51:25 +01:00