* error: add variants to `RuntimeError`
* error: remove `<BackendError>` generic
we can just map each backend error variant <-> our error as needed
* backend: impl `From<heed::Error>` for `RuntimeError`
* add `Never` type to allow foreign trait implementations
This is a newtype to workaround `sanakirja::Storable` not being
able to be implemented on `std::convert::Infallible` due to
"foreign trait on foreign type" rules.
* revert 0342848
, fix `sanakirja` trait bounds
K/V will always be `[u8]`, not the concrete type
so it does not need to be bounded.
* backend: fix `sanakijra` K/V generics
* sanakirja: add `error.rs`
* error: remove serde traits
* heed: add `todo!()` for error mappings
* error: add `Corrupt` variant
* sanakirja: finish error mappings
* heed: finish error mappings
* error: add to error types
* env: use `InitError` in `Env::open()`
* error: docs
* heed: remove `serde.rs`
Not needed if all K/V's stored are `[u8]`.
* heed: use `heed::types::Bytes` as K/V
* database: docs
* heed: impl `From<heed::Error>` for `InitError`
* sanakirja: impl `From<sanakirja::Error>` for `InitError`
* error: fix doc warnings
* heed: fix `clippy::match_same_arms` in error match
* readme: add TODO
* error: add `BoxError`, and fatal/unknown variants
* heed: use `Fatal/Unknown` variants in errors
* sanakirja: use `Fatal/Unknown` variants in errors
* clippy
* sanakijra: remove `fallible_impl_from`
* error: remove `RuntimeError::InvalidVersion`
* error: remove `RuntimeError` variants that should panic
* error: remove `InitError::Fatal`
We will exit on all errors regardless.
Any non-enumrated variants will use `InitError::Unknown`.
* heed: fix error mappings
* constants: add `CUPRATE_DATABASE_CORRUPT_MSG`
* sanakijra: fix error mappings
* heed: fix import
* comments/docs
* key: fix docs
5.6 KiB
Database
Cuprate's database implementation.
TODO: document Pod
and how databases use (de)serialize objects when storing/fetching, essentially using <[u8], [u8]>
.
Documentation
In general, documentation for database/
is split into 3:
Documentation location | Purpose |
---|---|
database/README.md |
High level design of cuprate-database |
cuprate-database |
Practical usage documentation/warnings/notes/etc |
Source file // comments |
Implementation-specific details (e.g, how many reader threads to spawn?) |
This README serves as the overview/design document.
For actual practical usage, cuprate-database
's types and general usage are documented via standard Rust tooling.
Run:
cargo doc --package cuprate-database --open
at the root of the repo to open/read the documentation.
If this documentation is too abstract, refer to any of the source files, they are heavily commented. There are many // Regular comments
that explain more implementation specific details that aren't present here or in the docs. Use the file reference below to find what you're looking for.
The code within src/
is also littered with some grep
-able comments containing some keywords:
Word | Meaning |
---|---|
INVARIANT |
This code makes an assumption that must be upheld for correctness |
SAFETY |
This unsafe code is okay, for x,y,z reasons |
FIXME |
This code works but isn't ideal |
HACK |
This code is a brittle workaround |
PERF |
This code is weird for performance reasons |
TODO |
This has to be implemented |
SOMEDAY |
This should be implemented... someday |
File Structure
A quick reference of the structure of the folders & files in cuprate-database
.
Note that lib.rs/mod.rs
files are purely for re-exporting/visibility/lints, and contain no code. Each sub-directory has a corresponding mod.rs
.
src/
The top-level src/
files.
File | Purpose |
---|---|
constants.rs |
General constants used throughout cuprate-database |
database.rs |
Abstracted database; trait Database |
env.rs |
Abstracted database environment; trait Env |
error.rs |
Database error types |
free.rs |
General free functions (related to the database) |
key.rs |
Abstracted database keys; trait Key |
pod.rs |
Data (de)serialization; trait Pod |
table.rs |
Database table abstraction; trait Table |
tables.rs |
All the table definitions used by cuprate-database |
transaction.rs |
Database transaction abstraction; trait RoTx , trait RwTx |
src/ops/
This folder contains the cupate_database::ops
module.
TODO: more detailed descriptions.
File | Purpose |
---|---|
alt_block.rs |
Alternative blocks |
block.rs |
Blocks |
blockchain.rs |
Blockchain-related |
output.rs |
Outputs |
property.rs |
Properties |
spent_key.rs |
Spent keys |
tx.rs |
Transactions |
src/service/
This folder contains the cupate_database::service
module.
File | Purpose |
---|---|
free.rs |
General free functions used (related to cuprate_database::service ) |
read.rs |
Read thread-pool definitions and logic |
request.rs |
Read/write Request s to the database |
response.rs |
Read/write Response 's from the database |
tests.rs |
Thread-pool tests and test helper functions |
write.rs |
Write thread-pool definitions and logic |
src/backend/
This folder contains the actual database crates used as the backend for cuprate-database
.
Each backend has its own folder.
Folder | Purpose |
---|---|
heed/ |
Backend using using forked heed |
sanakirja/ |
Backend using sanakirja |
All backends follow the same file structure:
File | Purpose |
---|---|
database.rs |
Implementation of trait Database |
env.rs |
Implementation of trait Env |
error.rs |
Implementation of backend's errors to cuprate_database 's error types |
transaction.rs |
Implementation of trait RoTx/RwTx |
types.rs |
Type aliases for long backend-specific types |
Backends
cuprate-database
's trait
s abstract over various actual databases.
Each database's implementation is located in its respective file in src/backend/${DATABASE_NAME}.rs
.
heed
The default database used is a modified fork of heed
, located at Cuprate/heed
.
To generate documentation of the fork for local use:
git clone --recursive https://github.com/Cuprate/heed
cargo doc
LMDB
should not need to be installed as heed
has a build script that pulls it in automatically.
sanakirja
TODO
Layers
TODO: update with accurate information when ready, update image.