cuprate-hinto-janai/database/src/tables.rs

410 lines
14 KiB
Rust
Raw Normal View History

Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//! Database tables.
//!
//! This module contains all the table definitions used by `cuprate-database`.
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//---------------------------------------------------------------------------------------------------- Import
use crate::{
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
database::{DatabaseIter, DatabaseRo, DatabaseRw},
table::Table,
types::{
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfo, KeyImage,
Output, PreRctOutputId, PrunableBlob, PrunableHash, PrunedBlob, RctOutput, TxBlob, TxHash,
TxId, UnlockTime,
},
};
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
//---------------------------------------------------------------------------------------------------- Sealed
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
/// Private module, should not be accessible outside this crate.
pub(super) mod private {
/// Private sealed trait.
///
/// Cannot be implemented outside this crate.
pub trait Sealed {}
}
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
//---------------------------------------------------------------------------------------------------- `trait Tables[Mut]`
/// Creates:
/// - `pub trait Tables`
/// - `pub trait TablesMut`
/// - Blanket implementation for `(tuples, containing, all, open, database, tables, ...)`
///
/// For why this exists, see: <https://github.com/Cuprate/cuprate/pull/102#pullrequestreview-1978348871>.
macro_rules! define_trait_tables {
($(
// The `T: Table` type The index in a tuple
// | containing all tables
// v v
$table:ident => $index:literal
),* $(,)?) => { paste::paste! {
/// Object containing all opened [`Table`]s in read-only mode.
///
/// This is an encapsulated object that contains all
/// available [`Table`]'s in read-only mode.
///
/// It is a `Sealed` trait and is only implemented on a
/// `(tuple, containing, all, table, types, ...)`.
///
/// This is used to return a _single_ object from functions like
/// [`EnvInner::open_tables`](crate::EnvInner::open_tables) rather
/// than the tuple containing the tables itself.
///
/// To replace `tuple.0` style indexing, `field_accessor_functions()`
/// are provided on this trait, which essentially map the object to
/// fields containing the particular database table, for example:
/// ```rust,ignore
/// let tables = open_tables();
///
/// // The accessor function `block_info_v1s()` returns the field
/// // containing an open database table for `BlockInfoV1s`.
/// let _ = tables.block_info_v1s();
/// ```
pub trait Tables: private::Sealed {
// This expands to creating `fn field_accessor_functions()`
// for each passed `$table` type.
//
// It is essentially a mapping to the field
// containing the proper opened database table.
//
// The function name of the function is
// the table type in `snake_case`, e.g., `block_info_v1s()`.
$(
/// Access an opened
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake>](&self) -> &impl DatabaseRo<$table>;
)*
/// This returns `true` if all tables are empty.
///
/// # Errors
/// This returns errors on regular database errors.
fn all_tables_empty(&self) -> Result<bool, $crate::error::RuntimeError>;
}
/// Object containing all opened [`Table`]s in read + iter mode.
///
/// This is the same as [`Tables`] but includes `_iter()` variants.
///
/// See [`Tables`] for documentation - this trait exists for the same reasons.
pub trait TablesIter: private::Sealed + Tables {
$(
/// Access an opened read-only + iterable
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake _iter>](&self) -> &(impl DatabaseRo<$table> + DatabaseIter<$table>);
)*
}
/// Object containing all opened [`Table`]s in write mode.
///
/// This is the same as [`Tables`] but for mutable accesses.
///
/// See [`Tables`] for documentation - this trait exists for the same reasons.
pub trait TablesMut: private::Sealed + Tables {
$(
/// Access an opened
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake _mut>](&mut self) -> &mut impl DatabaseRw<$table>;
)*
}
// Implement `Sealed` for all table types.
impl<$([<$table:upper>]),*> private::Sealed for ($([<$table:upper>]),*) {}
// This creates a blanket-implementation for
// `(tuple, containing, all, table, types)`.
//
// There is a generic defined here _for each_ `$table` input.
// Specifically, the generic letters are just the table types in UPPERCASE.
// Concretely, this expands to something like:
// ```rust
// impl<BLOCKINFOSV1S, BLOCKINFOSV2S, BLOCKINFOSV3S, [...]>
// ```
impl<$([<$table:upper>]),*> Tables
// We are implementing `Tables` on a tuple that
// contains all those generics specified, i.e.,
// a tuple containing all open table types.
//
// Concretely, this expands to something like:
// ```rust
// (BLOCKINFOSV1S, BLOCKINFOSV2S, BLOCKINFOSV3S, [...])
// ```
// which is just a tuple of the generics defined above.
for ($([<$table:upper>]),*)
where
// This expands to a where bound that asserts each element
// in the tuple implements some database table type.
//
// Concretely, this expands to something like:
// ```rust
// BLOCKINFOSV1S: DatabaseRo<BlockInfoV1s> + DatabaseIter<BlockInfoV1s>,
// BLOCKINFOSV2S: DatabaseRo<BlockInfoV2s> + DatabaseIter<BlockInfoV2s>,
// [...]
// ```
$(
[<$table:upper>]: DatabaseRo<$table>,
)*
{
$(
// The function name of the accessor function is
// the table type in `snake_case`, e.g., `block_info_v1s()`.
#[inline]
fn [<$table:snake>](&self) -> &impl DatabaseRo<$table> {
// The index of the database table in
// the tuple implements the table trait.
&self.$index
}
)*
fn all_tables_empty(&self) -> Result<bool, $crate::error::RuntimeError> {
$(
if !DatabaseRo::is_empty(&self.$index)? {
return Ok(false);
}
)*
Ok(true)
}
}
// This is the same as the above
// `Tables`, but for `TablesIter`.
impl<$([<$table:upper>]),*> TablesIter
for ($([<$table:upper>]),*)
where
$(
[<$table:upper>]: DatabaseRo<$table> + DatabaseIter<$table>,
)*
{
$(
// The function name of the accessor function is
// the table type in `snake_case` + `_iter`, e.g., `block_info_v1s_iter()`.
#[inline]
fn [<$table:snake _iter>](&self) -> &(impl DatabaseRo<$table> + DatabaseIter<$table>) {
&self.$index
}
)*
}
// This is the same as the above
// `Tables`, but for `TablesMut`.
impl<$([<$table:upper>]),*> TablesMut
for ($([<$table:upper>]),*)
where
$(
[<$table:upper>]: DatabaseRw<$table>,
)*
{
$(
// The function name of the mutable accessor function is
// the table type in `snake_case` + `_mut`, e.g., `block_info_v1s_mut()`.
#[inline]
fn [<$table:snake _mut>](&mut self) -> &mut impl DatabaseRw<$table> {
&mut self.$index
}
)*
}
}};
}
// Format: $table_type => $index
//
// The $index:
// - Simply increments by 1 for each table
// - Must be 0..
// - Must end at the total amount of table types
//
// Compile errors will occur if these aren't satisfied.
define_trait_tables! {
BlockInfos => 0,
BlockBlobs => 1,
BlockHeights => 2,
KeyImages => 3,
NumOutputs => 4,
PrunedTxBlobs => 5,
PrunableHashes => 6,
Outputs => 7,
PrunableTxBlobs => 8,
RctOutputs => 9,
TxBlobs => 10,
TxIds => 11,
TxHeights => 12,
TxOutputs => 13,
TxUnlockTime => 14,
}
//---------------------------------------------------------------------------------------------------- Table function macro
/// `crate`-private macro for callings functions on all tables.
///
/// This calls the function `$fn` with the optional
/// arguments `$args` on all tables - returning early
/// (within whatever scope this is called) if any
/// of the function calls error.
///
/// Else, it evaluates to an `Ok((tuple, of, all, table, types, ...))`,
/// i.e., an `impl Table[Mut]` wrapped in `Ok`.
macro_rules! call_fn_on_all_tables_or_early_return {
(
$($fn:ident $(::)?)*
(
$($arg:ident),* $(,)?
)
) => {{
Ok((
$($fn ::)*<$crate::tables::BlockInfos>($($arg),*)?,
$($fn ::)*<$crate::tables::BlockBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::BlockHeights>($($arg),*)?,
$($fn ::)*<$crate::tables::KeyImages>($($arg),*)?,
$($fn ::)*<$crate::tables::NumOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunedTxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunableHashes>($($arg),*)?,
$($fn ::)*<$crate::tables::Outputs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunableTxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::RctOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxIds>($($arg),*)?,
$($fn ::)*<$crate::tables::TxHeights>($($arg),*)?,
$($fn ::)*<$crate::tables::TxOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxUnlockTime>($($arg),*)?,
))
}};
}
pub(crate) use call_fn_on_all_tables_or_early_return;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//---------------------------------------------------------------------------------------------------- Table macro
/// Create all tables, should be used _once_.
///
/// Generating this macro once and using `$()*` is probably
/// faster for compile times than calling the macro _per_ table.
///
/// All tables are zero-sized table structs, and implement the `Table` trait.
///
/// Table structs are automatically `CamelCase`,
/// and their static string names are automatically `snake_case`.
macro_rules! tables {
(
$(
$(#[$attr:meta])* // Documentation and any `derive`'s.
$table:ident, // The table name + doubles as the table struct name.
$key:ty => // Key type.
$value:ty // Value type.
),* $(,)?
) => {
paste::paste! { $(
// Table struct.
$(#[$attr])*
// The below test show the `snake_case` table name in cargo docs.
/// ## Table Name
/// ```rust
/// # use cuprate_database::{*,tables::*};
#[doc = concat!(
"assert_eq!(",
stringify!([<$table:camel>]),
"::NAME, \"",
stringify!([<$table:snake>]),
"\");",
)]
/// ```
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash)]
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
pub struct [<$table:camel>];
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
// Implement the `Sealed` in this file.
// Required by `Table`.
impl private::Sealed for [<$table:camel>] {}
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
// Table trait impl.
impl Table for [<$table:camel>] {
const NAME: &'static str = stringify!([<$table:snake>]);
type Key = $key;
type Value = $value;
}
)* }
};
}
//---------------------------------------------------------------------------------------------------- Tables
// Notes:
// - Keep this sorted A-Z (by table name)
// - Tables are defined in plural to avoid name conflicts with types
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
// - If adding/changing a table also edit:
// a) the tests in `src/backend/tests.rs`
// b) `Env::open` to make sure it creates the table (for all backends)
// c) `call_fn_on_all_tables_or_early_return!()` macro defined in this file
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
tables! {
/// TODO
BlockBlobs,
BlockHeight => BlockBlob,
/// TODO
BlockHeights,
BlockHash => BlockHeight,
/// TODO
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
BlockInfos,
BlockHeight => BlockInfo,
/// TODO
KeyImages,
KeyImage => (),
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
/// Maps an output's amount to the number of outputs with that amount.
///
/// For a new output the `AmountIndex` value from this
/// table will be its index in a list of duplicate outputs.
NumOutputs,
Amount => AmountIndex,
/// TODO
PrunedTxBlobs,
TxId => PrunedBlob,
/// TODO
Outputs,
PreRctOutputId => Output,
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
// SOMEDAY: impl when `monero-serai` supports pruning
PrunableTxBlobs,
TxId => PrunableBlob,
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
// SOMEDAY: impl when `monero-serai` supports pruning
PrunableHashes,
TxId => PrunableHash,
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
// SOMEDAY: impl a properties table:
// - db version
// - pruning seed
// Properties,
// StorableString => StorableVec,
/// TODO
RctOutputs,
AmountIndex => RctOutput,
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
/// SOMEDAY: remove when `monero-serai` supports pruning
TxBlobs,
TxId => TxBlob,
/// TODO
TxIds,
TxHash => TxId,
/// TODO
TxHeights,
TxId => BlockHeight,
database: implement `ops/` (#102) * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 01:53:29 +00:00
/// TODO
TxOutputs,
TxId => AmountIndices,
/// TODO
TxUnlockTime,
TxId => UnlockTime,
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
mod test {
// use super::*;
}