diff --git a/books/architecture/src/SUMMARY.md b/books/architecture/src/SUMMARY.md index d0c641a1..a29ce902 100644 --- a/books/architecture/src/SUMMARY.md +++ b/books/architecture/src/SUMMARY.md @@ -27,33 +27,33 @@ --- -- [⚪️ Storage](storage/intro.md) - - [⚪️ Database abstraction](storage/db/intro.md) - - [⚪️ Backends](storage/db/backends/intro.md) - - [⚪️ Layers](storage/db/layers/intro.md) - - [⚪️ Backend](storage/db/layers/backend.md) - - [⚪️ ConcreteEnv](storage/db/layers/concrete_env.md) - - [⚪️ Trait](storage/db/layers/trait.md) - - [⚪️ `ops`](storage/db/layers/ops.md) - - [⚪️ Service](storage/db/layers/service.md) - - [⚪️ The service](storage/db/service/intro.md) - - [⚪️ Initialization](storage/db/service/initialization.md) - - [⚪️ Requests](storage/db/service/requests.md) - - [⚪️ Responses](storage/db/service/responses.md) - - [⚪️ Thread model](storage/db/service/thread-model.md) - - [⚪️ Shutdown](storage/db/service/shutdown.md) - - [⚪️ Syncing](storage/db/syncing.md) - - [⚪️ Resizing](storage/db/resizing.md) - - [⚪️ (De)serialization](storage/db/serde.md) - - [⚪️ Schema](storage/db/schema/intro.md) - - [⚪️ Tables](storage/db/schema/tables.md) - - [⚪️ Multimap tables](storage/db/schema/multimap.md) - - [⚪️ Known issues and tradeoffs](storage/db/issues/intro.md) - - [⚪️ Traits abstracting backends](storage/db/issues/traits.md) - - [⚪️ Hot-swappable backends](storage/db/issues/hot-swap.md) - - [⚪️ Copying unaligned bytes](storage/db/issues/copy.md) - - [⚪️ Endianness](storage/db/issues/endian.md) - - [⚪️ Extra table data](storage/db/issues/extra-table-data.md) +- [🟢 Storage](storage/intro.md) + - [🟢 Database abstraction](storage/db/intro.md) + - [🟢 Backends](storage/db/backends/intro.md) + - [🟢 Layers](storage/db/layers/intro.md) + - [🟢 Backend](storage/db/layers/backend.md) + - [🟢 ConcreteEnv](storage/db/layers/concrete_env.md) + - [🟢 Trait](storage/db/layers/trait.md) + - [🟢 Syncing](storage/db/syncing.md) + - [🟢 Resizing](storage/db/resizing.md) + - [🟢 (De)serialization](storage/db/serde.md) + - [🟢 Schema](storage/db/schema/intro.md) + - [🟢 Tables](storage/db/schema/tables.md) + - [🟢 Multimap tables](storage/db/schema/multimap.md) + - [🟢 Known issues and tradeoffs](storage/db/issues/intro.md) + - [🟢 Traits abstracting backends](storage/db/issues/traits.md) + - [🟢 Hot-swappable backends](storage/db/issues/hot-swap.md) + - [🟢 Copying unaligned bytes](storage/db/issues/copy.md) + - [🟢 Endianness](storage/db/issues/endian.md) + - [🟢 Extra table data](storage/db/issues/extra-table-data.md) + - [🟢 The layers](storage/db/layers/intro.md) + - [🟢 `ops`](storage/db/layers/ops.md) + - [🟢 `tower::Service`](storage/db/service/intro.md) + - [🟢 Initialization](storage/db/service/initialization.md) + - [🟢 Requests](storage/db/service/requests.md) + - [🟢 Responses](storage/db/service/responses.md) + - [🟢 Thread model](storage/db/service/thread-model.md) + - [🟢 Shutdown](storage/db/service/shutdown.md) - [⚪️ Blockchain](storage/blockchain.md) - [⚪️ Transaction pool](storage/transaction-pool.md) - [⚪️ Pruning](storage/pruning.md) diff --git a/books/architecture/src/storage/db/backends/intro.md b/books/architecture/src/storage/db/backends/intro.md new file mode 100644 index 00000000..250cdf34 --- /dev/null +++ b/books/architecture/src/storage/db/backends/intro.md @@ -0,0 +1 @@ +# ⚪️ Backends diff --git a/books/architecture/src/storage/db/intro.md b/books/architecture/src/storage/db/intro.md new file mode 100644 index 00000000..b21a192c --- /dev/null +++ b/books/architecture/src/storage/db/intro.md @@ -0,0 +1 @@ +# ⚪️ Database abstraction diff --git a/books/architecture/src/storage/db/issues/copy.md b/books/architecture/src/storage/db/issues/copy.md new file mode 100644 index 00000000..f13746d0 --- /dev/null +++ b/books/architecture/src/storage/db/issues/copy.md @@ -0,0 +1 @@ +# ⚪️ Copying unaligned bytes diff --git a/books/architecture/src/storage/db/issues/endian.md b/books/architecture/src/storage/db/issues/endian.md new file mode 100644 index 00000000..d1271ece --- /dev/null +++ b/books/architecture/src/storage/db/issues/endian.md @@ -0,0 +1 @@ +# ⚪️ Endianness diff --git a/books/architecture/src/storage/db/issues/extra-table-data.md b/books/architecture/src/storage/db/issues/extra-table-data.md new file mode 100644 index 00000000..b0215c2b --- /dev/null +++ b/books/architecture/src/storage/db/issues/extra-table-data.md @@ -0,0 +1 @@ +# ⚪️ Extra table data diff --git a/books/architecture/src/storage/db/issues/hot-swap.md b/books/architecture/src/storage/db/issues/hot-swap.md new file mode 100644 index 00000000..2cbab13e --- /dev/null +++ b/books/architecture/src/storage/db/issues/hot-swap.md @@ -0,0 +1 @@ +# ⚪️ Hot-swappable backends diff --git a/books/architecture/src/storage/db/issues/intro.md b/books/architecture/src/storage/db/issues/intro.md new file mode 100644 index 00000000..20ab7b5c --- /dev/null +++ b/books/architecture/src/storage/db/issues/intro.md @@ -0,0 +1 @@ +# ⚪️ Known issues and tradeoffs diff --git a/books/architecture/src/storage/db/issues/traits.md b/books/architecture/src/storage/db/issues/traits.md new file mode 100644 index 00000000..97401300 --- /dev/null +++ b/books/architecture/src/storage/db/issues/traits.md @@ -0,0 +1 @@ +# ⚪️ Traits abstracting backends diff --git a/books/architecture/src/storage/db/layers/backend.md b/books/architecture/src/storage/db/layers/backend.md new file mode 100644 index 00000000..568ecaff --- /dev/null +++ b/books/architecture/src/storage/db/layers/backend.md @@ -0,0 +1 @@ +# ⚪️ Backend diff --git a/books/architecture/src/storage/db/layers/concrete_env.md b/books/architecture/src/storage/db/layers/concrete_env.md new file mode 100644 index 00000000..98018a9d --- /dev/null +++ b/books/architecture/src/storage/db/layers/concrete_env.md @@ -0,0 +1 @@ +# ⚪️ ConcreteEnv diff --git a/books/architecture/src/storage/db/layers/intro.md b/books/architecture/src/storage/db/layers/intro.md new file mode 100644 index 00000000..701ca854 --- /dev/null +++ b/books/architecture/src/storage/db/layers/intro.md @@ -0,0 +1 @@ +# ⚪️ Layers diff --git a/books/architecture/src/storage/db/layers/ops.md b/books/architecture/src/storage/db/layers/ops.md new file mode 100644 index 00000000..f37989c0 --- /dev/null +++ b/books/architecture/src/storage/db/layers/ops.md @@ -0,0 +1 @@ +# ⚪️ ops diff --git a/books/architecture/src/storage/db/layers/service.md b/books/architecture/src/storage/db/layers/service.md new file mode 100644 index 00000000..1ec84798 --- /dev/null +++ b/books/architecture/src/storage/db/layers/service.md @@ -0,0 +1 @@ +# ⚪️ Service diff --git a/books/architecture/src/storage/db/layers/trait.md b/books/architecture/src/storage/db/layers/trait.md new file mode 100644 index 00000000..6a441d8d --- /dev/null +++ b/books/architecture/src/storage/db/layers/trait.md @@ -0,0 +1 @@ +# ⚪️ Trait diff --git a/books/architecture/src/storage/db/resizing.md b/books/architecture/src/storage/db/resizing.md new file mode 100644 index 00000000..777e2d44 --- /dev/null +++ b/books/architecture/src/storage/db/resizing.md @@ -0,0 +1 @@ +# ⚪️ Resizing diff --git a/books/architecture/src/storage/db/schema/intro.md b/books/architecture/src/storage/db/schema/intro.md new file mode 100644 index 00000000..010e736f --- /dev/null +++ b/books/architecture/src/storage/db/schema/intro.md @@ -0,0 +1 @@ +# ⚪️ Schema diff --git a/books/architecture/src/storage/db/schema/multimap.md b/books/architecture/src/storage/db/schema/multimap.md new file mode 100644 index 00000000..13dd36be --- /dev/null +++ b/books/architecture/src/storage/db/schema/multimap.md @@ -0,0 +1 @@ +# ⚪️ Multimap tables diff --git a/books/architecture/src/storage/db/schema/tables.md b/books/architecture/src/storage/db/schema/tables.md new file mode 100644 index 00000000..60c0465d --- /dev/null +++ b/books/architecture/src/storage/db/schema/tables.md @@ -0,0 +1 @@ +# ⚪️ Tables diff --git a/books/architecture/src/storage/db/serde.md b/books/architecture/src/storage/db/serde.md new file mode 100644 index 00000000..5c559e54 --- /dev/null +++ b/books/architecture/src/storage/db/serde.md @@ -0,0 +1 @@ +# ⚪️ (De)serialization diff --git a/books/architecture/src/storage/db/service/initialization.md b/books/architecture/src/storage/db/service/initialization.md new file mode 100644 index 00000000..d96fb300 --- /dev/null +++ b/books/architecture/src/storage/db/service/initialization.md @@ -0,0 +1 @@ +# ⚪️ Initialization diff --git a/books/architecture/src/storage/db/service/intro.md b/books/architecture/src/storage/db/service/intro.md new file mode 100644 index 00000000..c6b2c740 --- /dev/null +++ b/books/architecture/src/storage/db/service/intro.md @@ -0,0 +1 @@ +# ⚪️ The service diff --git a/books/architecture/src/storage/db/service/requests.md b/books/architecture/src/storage/db/service/requests.md new file mode 100644 index 00000000..3f6a1e9c --- /dev/null +++ b/books/architecture/src/storage/db/service/requests.md @@ -0,0 +1 @@ +# ⚪️ Requests diff --git a/books/architecture/src/storage/db/service/responses.md b/books/architecture/src/storage/db/service/responses.md new file mode 100644 index 00000000..404ae0c0 --- /dev/null +++ b/books/architecture/src/storage/db/service/responses.md @@ -0,0 +1 @@ +# ⚪️ Responses diff --git a/books/architecture/src/storage/db/service/shutdown.md b/books/architecture/src/storage/db/service/shutdown.md new file mode 100644 index 00000000..1bdc92de --- /dev/null +++ b/books/architecture/src/storage/db/service/shutdown.md @@ -0,0 +1 @@ +# ⚪️ Shutdown diff --git a/books/architecture/src/storage/db/service/thread-model.md b/books/architecture/src/storage/db/service/thread-model.md new file mode 100644 index 00000000..4bcfa01e --- /dev/null +++ b/books/architecture/src/storage/db/service/thread-model.md @@ -0,0 +1 @@ +# ⚪️ Thread model diff --git a/books/architecture/src/storage/db/syncing.md b/books/architecture/src/storage/db/syncing.md new file mode 100644 index 00000000..7cc80f15 --- /dev/null +++ b/books/architecture/src/storage/db/syncing.md @@ -0,0 +1 @@ +# ⚪️ Syncing diff --git a/books/architecture/src/storage/intro.md b/books/architecture/src/storage/intro.md index 214cf15d..e78538e5 100644 --- a/books/architecture/src/storage/intro.md +++ b/books/architecture/src/storage/intro.md @@ -1 +1,34 @@ -# ⚪️ Storage +# Storage +This section covers all things related to the long-term on-disk storage of data within Cuprate. + +## Overview +The quick overview is that Cuprate has a [database abstraction crate](./database-abstraction.md) +that handles "low-level" database details such as key and value (de)serialization, tables, transactions, etc. + +This database abstraction crate is then used by all crates that need on-disk storage, i.e. the +- [Blockchain database](./blockchain.md) +- [Transaction pool database](./transaction-pool.md) + +## Service +The interface provided by all crates building on-top of the +database abstraction is a [`tower::Service`](https://docs.rs/tower), i.e. +database requests are sent, and database responses are received asynchronously. + +As the interface details are similar across databases (threadpool, read operations, write operations), +the interface itself is abstracted in the [`cuprate_database_service`](./db/layers/intro.md) crate, +which is then used by the crates. + +## Diagram +This is a roughly how database crates are set up. + +```text + ┌─────────────────┐ +┌──────────────────────────────────┐ │ │ +│ Some crate that needs a database │ ┌────────────────┐ │ │ +│ │ │ Public │ │ │ +│ ┌──────────────────────────────┐ │─►│ tower::Service │◄─►│ Rest of Cuprate │ +│ │ Database abstraction │ │ │ API │ │ │ +│ └──────────────────────────────┘ │ └────────────────┘ │ │ +└──────────────────────────────────┘ │ │ + └─────────────────┘ +```