cuprate/database/src/env.rs

169 lines
5.3 KiB
Rust
Raw Normal View History

Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//! Abstracted database environment; `trait Env`.
//---------------------------------------------------------------------------------------------------- Import
use crate::{
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
config::Config,
database::{DatabaseRo, DatabaseRw},
database: Errors (#62) * error: add variants to `RuntimeError` * error: remove `<BackendError>` generic we can just map each backend error variant <-> our error as needed * backend: impl `From<heed::Error>` for `RuntimeError` * add `Never` type to allow foreign trait implementations This is a newtype to workaround `sanakirja::Storable` not being able to be implemented on `std::convert::Infallible` due to "foreign trait on foreign type" rules. * revert 0342848, fix `sanakirja` trait bounds K/V will always be `[u8]`, not the concrete type so it does not need to be bounded. * backend: fix `sanakijra` K/V generics * sanakirja: add `error.rs` * error: remove serde traits * heed: add `todo!()` for error mappings * error: add `Corrupt` variant * sanakirja: finish error mappings * heed: finish error mappings * error: add to error types * env: use `InitError` in `Env::open()` * error: docs * heed: remove `serde.rs` Not needed if all K/V's stored are `[u8]`. * heed: use `heed::types::Bytes` as K/V * database: docs * heed: impl `From<heed::Error>` for `InitError` * sanakirja: impl `From<sanakirja::Error>` for `InitError` * error: fix doc warnings * heed: fix `clippy::match_same_arms` in error match * readme: add TODO * error: add `BoxError`, and fatal/unknown variants * heed: use `Fatal/Unknown` variants in errors * sanakirja: use `Fatal/Unknown` variants in errors * clippy * sanakijra: remove `fallible_impl_from` * error: remove `RuntimeError::InvalidVersion` * error: remove `RuntimeError` variants that should panic * error: remove `InitError::Fatal` We will exit on all errors regardless. Any non-enumrated variants will use `InitError::Unknown`. * heed: fix error mappings * constants: add `CUPRATE_DATABASE_CORRUPT_MSG` * sanakijra: fix error mappings * heed: fix import * comments/docs * key: fix docs
2024-02-17 13:00:14 +00:00
error::{InitError, RuntimeError},
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
resize::ResizeAlgorithm,
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
table::Table,
transaction::{TxRo, TxRw},
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
};
//---------------------------------------------------------------------------------------------------- Env
/// Database environment abstraction.
///
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
/// Essentially, the functions that can be called on [`ConcreteEnv`](crate::ConcreteEnv).
///
/// # `Drop`
/// Objects that implement [`Env`] _should_ probably
/// [`Env::sync`] in their drop implementations,
/// although, no invariant relies on this (yet).
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
pub trait Env: Sized {
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
//------------------------------------------------ Constants
/// Does the database backend need to be manually
/// resized when the memory-map is full?
///
/// # Invariant
/// If this is `false`, that means this [`Env`]
/// can _never_ return a [`RuntimeError::ResizeNeeded`].
///
/// If this is `true`, [`Env::resize_map`] & [`Env::current_map_size`]
/// _must_ be re-implemented, as it just panics by default.
const MANUAL_RESIZE: bool;
/// Does the database backend forcefully sync/flush
/// to disk on every transaction commit?
///
/// This is used as an optimization.
const SYNCS_PER_TX: bool;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//------------------------------------------------ Types
/// TODO
type TxRo<'env>: TxRo<'env>;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// TODO
type TxRw<'env>: TxRw<'env>;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//------------------------------------------------ Required
/// TODO
/// # Errors
/// TODO
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
fn open(config: Config) -> Result<Self, InitError>;
/// TODO
/// # Errors
/// TODO
fn create_tables<T: Table>(&self, tx_rw: &mut Self::TxRw<'_>) -> Result<(), RuntimeError>;
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
/// Return the [`Config`] that this database was [`Env::open`]ed with.
fn config(&self) -> &Config;
/// Return the amount of actual of bytes the database is taking up on disk.
///
/// This is the current _disk_ value in bytes, not the memory map.
///
/// # Errors
/// This will error if either:
///
/// - [`std::fs::File::open`]
/// - [`std::fs::File::metadata`]
///
/// failed on the database file on disk.
fn disk_size_bytes(&self) -> std::io::Result<u64> {
// We have the direct PATH to the file,
// no need to use backend-specific functions.
//
// SAFETY: as we are only accessing the metadata of
// the file and not reading the bytes, it should be
// fine even with a memory mapped file being actively
// written to.
Ok(std::fs::File::open(&self.config().db_file)?
.metadata()?
.len())
}
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// TODO
///
/// # Invariant
/// This must **fully** and **synchronously** flush the database data to disk.
///
/// I.e., after this function returns, there must be no doubts
/// that the data isn't synced yet, it _must_ be synced.
///
/// TODO: either this invariant or `sync()` itself will most
/// likely be removed/changed after `SyncMode` is finalized.
///
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// # Errors
/// TODO
fn sync(&self) -> Result<(), RuntimeError>;
database: Resizes, Shutdown, Flushing (#68) * error: add `NeedsResize` accidently removed, was `MapFull` before. We need this because we as the writer thread must react to this error in order to resize. The writer thread doesn't have access to `heed::Error`, but `Runtime::Error`, so this variant must exist * env/backend: add `MANUAL_RESIZE` and `resize()` * heed: respect `ReadersFull`, comment errors * free: add `resize_memory_map()` * env: add `Env::current_map_size` * service: add `resize_map()` * database: make `Env` itself cheaply clonable + threadsafe `heed::Env` already uses `Arc` internally, but `sanakirja` does not, so abstract this at the `Env` level instead of wrapping `ConcreteEnv` in `Arc` ourselves. * service: `Arc<ConcreteEnv>` -> `ConcreteEnv: Clone` * env: add `SYNCS_PER_TX`, `path()`, `shutdown()` * database: add `src/service/state.rs` * service: add to `state.rs`, add `DatabaseState` to readers/writer * add `parking_lot` Okay, turns out we need to take locks in `database/src/service/state.rs`... `std`'s lock fairness policies are not well defined and depend on the OS implementation, `parking_lot` on the other hand has a fairness policy which is important when the writer needs the lock but readers keep pouring in, essentially never letting the writer do stuff. * state: use `crossbeam::atomic::AtomicCell` We have crossbeam as a dep anyway. * heed: `heed::Env` -> `Arc<RwLock<heed::Env>>` * service: add reader shutdown handle, use `Select` for msgs * service: remove `state.rs` We don't need this, we will represent shutdowns with channel messages and `Select`, and mutual exclusion with a `RwLock`. * service: fix misc reader/writer stuff * database: add `config.rs` * service: use `ReaderThreads` when spawning readers * service: impl `shutdown()` for readers/writer * service: return `DatabaseReaderReceivers` on shutdown via `JoinHandle` Solves the issue of unfortunately timed `Request`s that come in _right_ as we are shutting down. If we (Cuprate) drop the database channels too early the requesting thread will probably panic as they probably use `.unwrap()`, never expecting a channel failure. Returning this structure containing all the channels allows the final shutdown code to carry these channels until the very end of the program, at which point, all threads exit - no panics. * remove `parking_lot` Could be used as the database mutual exclusion lock. Needs to be tested against `std`. * config: add `path` * env: `path()` -> `config()`, backend: impl `Drop` * `Arc<ConcreteEnv>`, shutdown `service` on channel disconnect * backend: add `Config` to `ConcreteEnv` * service: use `std::thread::Builder` for readers/writer * config: `PathBuf` -> `Cow<'static, Path>` * misc docs, remove `RuntimeError::ShuttingDown` * service: init & shutdown docs * heed: impl `Env::resize_map()` * heed: impl `Env::current_map_size()` * lib.rs: add example crate usage test * heed: `RwLock` comment * helper: add `cuprate_database_dir()` * config: use `cuprate_database_dir()` * lib.rs: TODO example test * database: add `page_size` The database memory map size must be a multiple of the OS page size. Why doesn't `heed` re-expose this? It calls it when checking anyway... https://docs.rs/heed/0.20.0-alpha.9/src/heed/env.rs.html#772 * free: impl `resize_memory_map()` * free: docs * env: add `disk_size_bytes()` * config: impl `From<$num>` for `ReaderThreads` * move `fs`-related constants to `cuprate_helper::fs` * docs * add `resize.rs`, `ResizeAlgorithm` * env: use `ResizeAlgorithm` in `resize_map()` * TODO: account for LMDB reader limit * resize: docs, add `page_size()`, impl `fixed_bytes()` * resize: impl `percent()` * resize: docs * env: take `ResizeAlgorithm` by value (it impls `Copy`) * heed: TODO for `MDB_MAP_FULL` & `MDB_MAP_RESIZED` * config: `From<Into<usize>>` for `ReaderThreads` Co-authored-by: Boog900 <boog900@tutanota.com> * env: move mutual exclusion doc to backend * free: update invariant doc Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/service/mod.rs Co-authored-by: Boog900 <boog900@tutanota.com> * fix `[allow(unused_imports)] // docs` * move DB filename from `helper/` -> `database/` * config: use `DATABASE_FILENAME` * config: add `db_file_path()` * lib: add non-`service` usage invariant docs * table: seal `Table` trait, impl for all `crate::tables` * fix docs * fix docs pt.2 --------- Co-authored-by: Boog900 <boog900@tutanota.com>
2024-02-25 19:46:36 +00:00
/// Resize the database's memory map to a
/// new (bigger) size using a [`ResizeAlgorithm`].
///
/// By default, this function will use the `ResizeAlgorithm` in [`Env::config`].
///
/// If `resize_algorithm` is `Some`, that will be used instead.
///
/// # Invariant
/// This function _must_ be re-implemented if [`Env::MANUAL_RESIZE`] is `true`.
///
/// Otherwise, this function will panic with `unreachable!()`.
fn resize_map(&self, resize_algorithm: Option<ResizeAlgorithm>) {
unreachable!()
}
/// What is the _current_ size of the database's memory map in bytes?
///
/// # Invariant
/// 1. This function _must_ be re-implemented if [`Env::MANUAL_RESIZE`] is `true`.
/// 2. This function must be accurate, as [`Env::resize_map()`] may depend on it.
fn current_map_size(&self) -> usize {
unreachable!()
}
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// TODO
/// # Errors
/// TODO
fn tx_ro(&self) -> Result<Self::TxRo<'_>, RuntimeError>;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// TODO
/// # Errors
/// TODO
fn tx_rw(&self) -> Result<Self::TxRw<'_>, RuntimeError>;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// TODO
///
/// # TODO: Invariant
/// This should never panic the database because the table doesn't exist.
///
/// Opening/using the database [`Env`] should have an invariant
/// that it creates all the tables we need, such that this
/// never returns `None`.
///
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// # Errors
/// TODO
fn open_db_ro<T: Table>(
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
&self,
tx_ro: &Self::TxRo<'_>,
) -> Result<impl DatabaseRo<T>, RuntimeError>;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// TODO
///
/// # TODO: Invariant
/// This should never panic the database because the table doesn't exist.
///
/// Opening/using the database [`Env`] should have an invariant
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
/// that it creates all the tables we need, such that this
/// never returns `None`.
///
/// # Errors
/// TODO
fn open_db_rw<T: Table>(
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
&self,
tx_rw: &mut Self::TxRw<'_>,
) -> Result<impl DatabaseRw<T>, RuntimeError>;
Database (#35) * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos
2024-02-13 17:43:25 +00:00
//------------------------------------------------ Provided
}