diff --git a/books/architecture/src/storage/db/backends.md b/books/architecture/src/storage/db/backends.md index b2ec6e4..74003bf 100644 --- a/books/architecture/src/storage/db/backends.md +++ b/books/architecture/src/storage/db/backends.md @@ -1 +1,48 @@ -# 🟢 Backends +# Backends +`cuprate-database`'s `trait`s allow abstracting over the actual database, such that any backend in particular could be used. + +This page is an enumeration of all the backends Cuprate has, has tried, and may try in the future. + +## `heed` +The default database used is [`heed`](https://github.com/meilisearch/heed) (LMDB). The upstream versions from [`crates.io`](https://crates.io/crates/heed) are used. `LMDB` should not need to be installed as `heed` has a build script that pulls it in automatically. + +`heed`'s filenames inside Cuprate's database folder (`~/.local/share/cuprate/database/`) are: + +| Filename | Purpose | +|------------|---------| +| `data.mdb` | Main data file +| `lock.mdb` | Database lock file + +`heed`-specific notes: +- [There is a maximum reader limit](https://github.com/monero-project/monero/blob/059028a30a8ae9752338a7897329fe8012a310d5/src/blockchain_db/lmdb/db_lmdb.cpp#L1372). Other potential processes (e.g. `xmrblocks`) that are also reading the `data.mdb` file need to be accounted for +- [LMDB does not work on remote filesystem](https://github.com/LMDB/lmdb/blob/b8e54b4c31378932b69f1298972de54a565185b1/libraries/liblmdb/lmdb.h#L129) + +## `redb` +The 2nd database backend is the 100% Rust [`redb`](https://github.com/cberner/redb). + +The upstream versions from [`crates.io`](https://crates.io/crates/redb) are used. + +`redb`'s filenames inside Cuprate's database folder (`~/.local/share/cuprate/database/`) are: + +| Filename | Purpose | +|-------------|---------| +| `data.redb` | Main data file + + + +## `redb-memory` +This backend is 100% the same as `redb`, although, it uses [`redb::backend::InMemoryBackend`](https://docs.rs/redb/2.1.2/redb/backends/struct.InMemoryBackend.html) which is a database that completely resides in memory instead of a file. + +All other details about this should be the same as the normal `redb` backend. + +## `sanakirja` +[`sanakirja`](https://docs.rs/sanakirja) was a candidate as a backend, however there were problems with maximum value sizes. + +The default maximum value size is [1012 bytes](https://docs.rs/sanakirja/1.4.1/sanakirja/trait.Storable.html) which was too small for our requirements. Using [`sanakirja::Slice`](https://docs.rs/sanakirja/1.4.1/sanakirja/union.Slice.html) and [sanakirja::UnsizedStorage](https://docs.rs/sanakirja/1.4.1/sanakirja/trait.UnsizedStorable.html) was attempted, but there were bugs found when inserting a value in-between `512..=4096` bytes. + +As such, it is not implemented. + +## `MDBX` +[`MDBX`](https://erthink.github.io/libmdbx) was a candidate as a backend, however MDBX deprecated the custom key/value comparison functions, this makes it a bit trickier to implement multimap tables. It is also quite similar to the main backend LMDB (of which it was originally a fork of). + +As such, it is not implemented (yet). diff --git a/books/architecture/src/storage/db/backends/intro.md b/books/architecture/src/storage/db/backends/intro.md deleted file mode 100644 index 250cdf3..0000000 --- a/books/architecture/src/storage/db/backends/intro.md +++ /dev/null @@ -1 +0,0 @@ -# ⚪️ Backends diff --git a/books/architecture/src/storage/db/common/initialization.md b/books/architecture/src/storage/db/common/initialization.md deleted file mode 100644 index 59f994c..0000000 --- a/books/architecture/src/storage/db/common/initialization.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Initialization diff --git a/books/architecture/src/storage/db/common/intro.md b/books/architecture/src/storage/db/common/intro.md deleted file mode 100644 index b29e971..0000000 --- a/books/architecture/src/storage/db/common/intro.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Common behavior diff --git a/books/architecture/src/storage/db/common/ops.md b/books/architecture/src/storage/db/common/ops.md deleted file mode 100644 index 2bfd3f9..0000000 --- a/books/architecture/src/storage/db/common/ops.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 ops diff --git a/books/architecture/src/storage/db/common/requests.md b/books/architecture/src/storage/db/common/requests.md deleted file mode 100644 index 7391375..0000000 --- a/books/architecture/src/storage/db/common/requests.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Requests diff --git a/books/architecture/src/storage/db/common/responses.md b/books/architecture/src/storage/db/common/responses.md deleted file mode 100644 index 3ba68e0..0000000 --- a/books/architecture/src/storage/db/common/responses.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Responses diff --git a/books/architecture/src/storage/db/common/shutdown.md b/books/architecture/src/storage/db/common/shutdown.md deleted file mode 100644 index cfe83db..0000000 --- a/books/architecture/src/storage/db/common/shutdown.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Shutdown diff --git a/books/architecture/src/storage/db/common/thread-model.md b/books/architecture/src/storage/db/common/thread-model.md deleted file mode 100644 index b2addfc..0000000 --- a/books/architecture/src/storage/db/common/thread-model.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Thread model diff --git a/books/architecture/src/storage/db/common/types.md b/books/architecture/src/storage/db/common/types.md deleted file mode 100644 index ba238d6..0000000 --- a/books/architecture/src/storage/db/common/types.md +++ /dev/null @@ -1 +0,0 @@ -# 🟢 Types