4 commits

Author	SHA1	Message	Date
hinto-janai	8f22d8ab79	database: impl trait function bodies for heed & redb (#85) ... * env: remove `T: Table` for `Env::create_tables()` It creates _all_ tables, not a specific `T: Table` * heed: half impl `Env::open()`, some TODOs * heed: add `HeedTxR{o,w}<'env>` * workspace/cargo: add `parking_lot` * remove `parking_lot` `MappedRwLockGuard` doesn't solve the `returning reference to object owned by function` error when returning heed's lock guard + the tx so we'll be going with `std` * env: add `{EnvInner,TxRoInput,TxRwInput}` and getter `fn()`s * env: fix tx <-> table lifetimes, fix `Env::create_tables()` * heed: impl `DatabaseRo` * heed: impl `DatabaseRw` * database: add `src/backend/${BACKEND}/tests.rs` * heed: impl more of `Env::open()` * redb: fix trait signatures, add `trait ValueGuard` * accommodate `DatabaseRo` bounds for both `{heed,redb}` * fold `get_range()` generic + bounds * add `TxCreator`, add `heed` tests * env: `TxCreator` -> `EnvInner` which doubles as DB/Table opener * database: `DatabaseRw<'tx>` -> `DatabaseRw<'db, 'tx>` * heed: `db_read_write()` test * database: fix `get()` lifetimes, heed: add `db_read_write()` test * database: remove `'env` lifetime from `DatabaseRo<'env, 'tx>` not needed for immutable references * redb: fix new `{Env, EnvInner, DatabaseR{o,w}}` bounds * redb: impl `Database` traits * redb: impl `TxR{o,w}` traits * redb: impl `Env` * redb: open/create tables in `Env::open` * redb: enable tests, add tmp `Storable` printlns * redb: fix alignment issue with `Cow` and `from_bytes_unaligned()` * redb: only allocate bytes when alignment != 1 * heed: remove custom iterator from `range()` * storable: conditionally allocat on misaligned bytes in `from_bytes` * add database guard * database: AccessGuard -> ValueGuard * storable: add `ALIGN` and `from_bytes_unaligned()` * redb: 1 serde type `StorableRedb`, fix impl * cow serde, trait bounds, fix backend's where bounds - Uses Cow for `redb`'s deserialization - Conforms `heed` to use Cow (but not as the actual key/value) - Conforms the `cuprate_database` trait API to use Cow - Adds `ToOwned + Debug` (and permutation) trait bounds - Solves 23098573148968945687345349657398 compiler errors due to aforementioned trait bounds, causing `where` to be everywhere * fix docs, use fully qualified Tx functions for tests * backend: check value guard contains value in test * add `Storable::ALIGN` tests, doc TODOs * add `trait ToOwnedDebug` * traits: `ToOwned + Debug` -> `ToOwnedDebug` * heed: remove `ToOwned` + `Debug` bounds * redb: remove `ToOwned` + `Debug` bounds * add `ValueGuard`, replace signatures, fix `redb` * heed: fix for `ValueGuard` * docs, tests * redb: add `CowRange` for `T::Key` -> `Cow<'_, T::Key>` conversion * separate `config.rs` -> `config/` * ci: combine tests, run both `heed` and `redb` tests * ci: fix workflow * backend: add `resize()` test * ci: remove windows-specific update * ci: remove update + windows default set * backend: add `unreachable` tests, fix docs * trait docs * ci: fix * Update database/src/backend/heed/env.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/env.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/transaction.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/redb/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/redb/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * Update database/src/backend/heed/database.rs Co-authored-by: Boog900 <boog900@tutanota.com> * readme: fix `value_guard.rs` * heed: remove unneeded clippy + fix formatting * heed: create and use `create_table()` in `Env::open()` * redb: create and use `create_table()` in `Env::open()` * redb: remove unneeded clippy * fix clippy, remove `<[u8], [u8]>` docs --------- Co-authored-by: Boog900 <54e72d8a-345f-4599-bd90-c6b9bc7d0ec5@aleeas.com> Co-authored-by: Boog900 <boog900@tutanota.com>	2024-03-13 22:05:24 +00:00
hinto-janai	272ef18eb6	database: replace Pod with bytemuck (#81) ... * database: add `bytemuck` * database: add `types` module, replace `Pod` with `bytemuck` traits * types: docs * types: more docs * types: align safety msg * types: docs * misc docs * add `storable.rs` * add `slice.rs` * storable: impl `impl_storable_checked_bit_pattern!()` * database: TODO: fix `DatabaseRo::get_range` lifetimes * key/table: trait bound fixes * misc fixes * remove `borsh` - Doesn't work on must types - Probably won't use it anyway - Most things impl `serde` * key: add `new_with_max_secondary()` * key: add `new_with_max_secondary()` * heed: add `StorableHeed` for `Storable` compat * redb: add `StorableRedb{Key,Value}` for `Storable` compat * storable: add `Debug` bound and `fixed_width()` * redb: fix `'static` bound * storable: docs * `pod.rs` tests -> `storable.rs` * redb: add `Storable` tests * storable: add doc tests * redb: simplify `Storable` tests * heed: add `Storable` tests * misc docs/fixes * cargo: switch from forked `heed` -> `heed 0.20.0-alpha.9` * update readme * docs * fix README * table: remove `CONSTANT_SIZE` * database: `get()/delete() -> Err(KeyNotFound)` instead of `Option`	2024-03-03 22:26:39 +00:00
hinto-janai	240e579066	database: replace sanakirja with redb (#80) ... * cargo: replace `sanakirja` with `redb` * database: update docs `sanakirja` -> `redb` * lib: add TODO for `ConcreteEnv` generic replacement * database: split `trait Database` -> `trait Database{Read,Write}` * heed: add `struct HeedTable{Ro,Rw}` to match `redb` behavior * ops: remove imports for now * env: fix `&mut` bound on RwTx * database: impl `redb`, type-checks * fix heed trait impls, `Database{Read,Write}` -> `Database{Ro,Rw}` * redb: impl `From<_>` for `RuntimeError` * update readme * heed: document `HeedTableR{o,w}` types * env: doc `sync()` invariant * database: document data & lock filenames * misc docs, `redb` durability impl, `'db` -> `'env` * redb: fixes * misc docs and fixes * Update database/README.md Co-authored-by: Boog900 <boog900@tutanota.com> --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-02-29 17:40:15 +00:00
hinto-janai	331d4d3e7f	Database (#35) ... * rename `database` -> `old_database` Keeping it around for reference until new implementation is complete. * create new `database/` skeleton * add `DATABASE.md` design doc skeleton * move design doc to `database/README.md` * add rough code * `lib.rs` -> `gist.rs` * database: use workspace deps * workspace: include `database/` as member CI will now include this crate. * cargo fmt * database: `AGPL` -> `MIT` * readme: add `TODO`s * add base files * cargo.toml: add `heed` feature * fix clippy * lib.rs: add extremely pedantic lints * readme: add `# Backends` * cargo.toml: add `cfg-if` * add `backend/` structure * base `database.rs` * cargo.toml: add `borsh` * backend: add `DATABASE_BACKEND` * base `error.rs` * base `database.rs` * base `transaction.rs` * base `table.rs` * lib.rs: add imports * add `pod.rs` * pod: use `Read/Write`, add tests for all primitive numbers * pod: impl Pod for `Vec<u8>`, `[u8; N]` * pod: add docs, add `private::Sealed` * pod: `to_writer`, `from_reader` The new `as_bytes` + `from_bytes` now allows (de)serializing from bytes directly instead of going through Read/Write. Different array return sizes are abstracted away with `-> impl AsRef<[u8]>` * pod: impl Pod for `Box<[u8]>` * pod: return `Err` on incorrect length in `from_bytes()` * pod: docs * pod: impl Pod for `Arc<[u8]>` * readme: docs * database: add `create_table()`, `get_table()` * table: `Pod` bound * backend: move into directories * pod: add `into_bytes()` * heed: impl `BytesEncode`, `BytesDecode` * add `actor`, `service` features The thread/actor system used will be gated behind `actor`, and the `tower/tokio` integration will be gated behind `service`. * add `lib.rs` docs * service: add `service.rs` * service: add `reader.rs` * service: add `writer.rs` * service: add `request.rs` & `response.rs` * cargo.toml: add `crossbeam` * service: add/use `enum Request`, `enum Response` * service: basic signatures for thread-pools, `Writer` -> `Writers` * service: split `tower::Service<ReadRequest/WriteRequest>` * service: impl `tower::Service` for `Database(Reader|Writer)` * service: add `init()`, impl basic `Reader/Writer` pools * service: add example `Request`'s * service: add example `ReadRequest` handling * temporarily allow clippy lints * readme: update file structure * transaction: add `RoTx::get_range()` * service: module docs * cargo.toml: add `cuprate-helper` * service: scale readers/writers based on thread count * database: change lifetimes * heed: impl Database for `heed` * heed: add `ConcreteRoTx`, `ConcreteRwTx`, impl Tx traits * docs * service: `read.rs` docs * service: `write.rs` docs * service: request/response docs * service: use `OnceLock` in `init()`, add `db_read()`, `db_write()` * service: leak database into `&'static` * database: add `#[inline]`, `#[cold]` * service: `free.rs` docs, more `#[inline]` + `#[cold]` * service: add `shutdown()` * service: add `Request::Shutdown` * service: `shutdown()` docs * heed: hide concrete tx types * lib.rs: add terms * split `Env` <-> `Database` * cargo.toml: add `paste` * database: add `tables/` * impl `serde/borsh` where possible * tables: add `Tables`, add test docs * make db backend mutually exclusive to fix `--all-features` * tables: use `$()*` in `tables!()` * cargo.toml: add `sanakirja 1.4.0` * sanakirja: impl `Env` * sanakirja: impl `Database` * sanakirja: impl `Transaction` * table: temporarily fix `sanakirja` K/V bounds * table: fix `#[cfg]` * cargo.toml: fix deps * lib.rs: docs * service: docs * readme: add files, update `# Documentation`, add `# Layers` * readme: `src/` file purpose * readme: `src/service/` file purpose * readme: `src/backend/` file purpose * fix `Cargo.lock` merge conflict * database: remove `gist.rs` * add to `constants.rs` * add top `//! comments` for files/modules * constants: add sanity-check test * service: add `only_one_database` test in `free.rs` * service: add `tests.rs` * remove unneeded markers + imports * backend: fix `get_range()`'s trait `impl` return * env: add `create_tables_if_needed()`, don't return `Option<Db>` * sort imports by `CONTRIBUTING.md` rules oops sorry boog * add `monero.rs` * monero: docs * database: add missing `RoTx/RwTx` inputs * backend: add missing `RoTx/RwTx` inputs * `monero.rs` trait -> free functions in `ops/` * pod: make methods infallible * ci: add `rustup update` step * service: use `Arc` instead of leaking, remove `db_read/db_write` * service: use `InfallibleOneshotReceiver` for readers * service: shutdown on error, add todos * service: remove `Request` * service: combine `ReadResponse` and `WriteResponse` * service: use `InfallibleOneshotReceiver` for writer * service: only spawn 1 writer, don't allow cloning writer handle * table: add associated `const CONSTANT_SIZE` * add `key.rs` + `trait Key`, add bound to `Table` * fix typos	2024-02-13 17:43:25 +00:00