Cuprate/cuprate
Cuprate/cuprate
1
0
Fork 0
mirror of https://github.com/Cuprate/cuprate.git synced 2024-12-22 11:39:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

database: implement ops/ (#102)

Browse source 
* ops: add `trait MoneroR{o,w}`

* update `trait MoneroR{o,w}` bounds

* types: add `BlockInfoLatest` type alias

* block: impl most core functions

* types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916

* fix table type test

* cargo.toml: add `{cuprate-types, monero-serai}`

* add_block: add all other block data

* ops: remove unneeded `block` functions

* env: add `EnvInner::open_db_rw_all()`

* types: fix test

* block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()`

* add `trait Tables[Mut]` and use it in `EnvInner`

* block: use `TablesMut`

* tables: replace manual impl with `define_trait_tables!()`

* tables: docs for `trait Tables[Mut]`

* tables: doc functions

* create `call_fn_on_all_tables_or_early_return!()` macro

* block: cleanup signatures + bodies

* block: more fn's, docs

* block: add `doc_{error,single,bulk}!()`

* remove `ops/monero.rs`

* move `height()` to `ops/blockchain.rs`

* add `ops/macros.rs`

* tx: add fn signatures

* output: fix fn signatures

* ops: expose `_inner()` functions

* block: add `add_block_header{_bulk, _inner}()`

* ops: remove doc_{fn,inner}!()`

* ops: remove `_{inner,bulk}()`, lifetime + generics

* update lib/mod docs

* ops: add and use `doc_add_block_inner_invariant!()`

* ops: add docs/return to inner `add_block()` functions

* add_block(): extract and use fn for {key_image, output}

* ops: more fn body impl + `add_block()`

* cargo: add `monero-pruning`

* ops: extract out `tx` functions from `add_block()`

* property: add `db_version()`

* ops: `pop_block()` body, remove other `pop_block` fn's

* types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation`

* block: put `block_blob`, pass `Tables` to sub-functions

`impl TablesMut` can't mutably pass multiple tables since
it takes `&mut self`, so all functions unfortunately have to
take a full `&mut impl TablesMut` even though they only need
a few tables.

* database: add `DatabaseRw::take()`

useful for `pop_block()` where we need the value afterwards

* block: deserialize tx's from `block_blobs` in `pop_block()`

* blockchain: `height()` -> `chain_height()`

* output: fix `amount_index`

* ops: fix unlock_time, chain_height

* `BlockInfoV{1,2,3}` -> `BlockInfo`

* constants: add `DATABASE_VERSION`

* database: add `DatabaseRw::update()`

* output: use `DatabaseRw::update()` in `remove_output()`

* add `TxBlobs` table, ignore pruning tables

* block: mostly impl `add_block()` body

* ops: comments

* add_block: miner v2 tx commitment, height cast

* block: impl `pop_block()`

* block: mostly impl `get_block()`

* block: impl `get_block_{from_height,header,header_from_height}`

* add `OutputFlags` bitflags

* add_block: u32::try_into(height: u32), use `OutputFlags`

* tx: impl `get_{tx,tx_from_id}()`

* tx: move docs tests to `#[test]`

testing everything in 1 go is more natural since e.g:
`add_tx()` is followed by `get_tx()`

* tables: add `trait TablesIter`, `all_tables_empty()`

This allows `TablesMut` to be a superset of `Tables`
and use all its accessor functions.

* use cuprate-test-utils, fix tx tests

* block: `add_block()` take block by ref

* tx: use all txs in tests

* output: add `all_tx_functions()` test

* add_block: check current height against input

* block: map more fields in `get_block()`

* block: remove `get_block()`, doc tests, fix `get_block_header()`

* block: dummy values in test

* heed: use `last/first()` instead of `unsafe` cursors

We no longer have DUP semantics and also hard to debug errors
were popping up on `del_current()`...

* heed: fix `DatabaseRw::delete`

Ok(true) means the key did not exist, so we must return
Err(RuntimeError::KeyNotFound)

* block: `add_block()` (dummy value) test

* ops: `key_image` tests

* cleanup, docs, tests

* backend: test `take()` & `update()`

* docs

* remove `OutputFlags::NONE`

* add_block(): add asserts, panic docs, `should_panic` tests

* backend: remove `Ok(())` in `Database::delete` if already deleted

redb already does this, so heed so match

* block: move block operations after tx/outputs

* `amount == 0` -> `amount == 1`

* Nit: StorableVec::wrap_ref

* `saturating_sub(1)` -> `- 1`

* add `TxOutputs` table

* add_block(): add to `tx_outputs` table

* fix `DatabaseRo::update`

* add_tx(): take `block_height` as input

* tx: add/remove from `TxOutputs` table

* output: remove if `amount == 1` -> `amount_index == 0`

* output: fix `add_output()`'s `amount_index` calculation

* output: fix `add_output()`'s `amount_index` calculation again

* output: tests for `amount_index/num_outputs`

* block: `num_outputs - 1` and `take()` -> `get()`

We don't need to `take()` since the call afterwards to
`remove_output()` removes the entry

* block: swap `get_block_extended_header[_from_height]()`

* move `{key_image,output}` handling `add_block()` -> `add_tx()`

* blockchain: add doc to `top_block_height()`

* block: manual panic -> `assert_eq!()`

* test-utils: add `block_blob` to `VerifiedBlockInformation`

field introduced in this PR

* ops: use real block/tx data in tests

* block: `total_generated_coins` -> `cumulative_generated_coins`

* fix clippy, docs, TODOs

* `cumulative_generated_coins()`: `block/` -> `blockchain/`

* blockchain: add `cumulative_generated_coins()` tests

* Update database/src/ops/block.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* `cumulative_generated_coins()` docs for pre-block-0 special case

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
This commit is contained in:
hinto-janai 2024-04-22 21:53:29 -04:00 committed by GitHub
parent ee22e81c7e
commit c65eb0a3ca
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
26 changed files with 2480 additions and 637 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

532
Cargo.lock generated
View file

File diff suppressed because it is too large Load diff

22
database/Cargo.toml
View file

@ -9,26 +9,30 @@ repository = "https://github.com/Cuprate/cuprate/tree/main/database"
keywords = ["cuprate", "database"]
[features]
# default = ["heed", "redb", "service"]
default = ["heed", "redb", "service"]
# default = ["redb", "service"]
default = ["redb-memory", "service"]
# default = ["redb-memory", "service"]
heed = ["dep:heed"]
redb = ["dep:redb"]
redb-memory = ["redb"]
service = ["dep:crossbeam", "dep:futures", "dep:tokio", "dep:tokio-util", "dep:tower", "dep:rayon"]
[dependencies]
bitflags = { workspace = true, features = ["serde", "bytemuck"] }
bytemuck = { version = "1.14.3", features = ["must_cast", "derive", "min_const_generics", "extern_crate_alloc"] }
bytes = { workspace = true }
cfg-if = { workspace = true }
# FIXME:
# We only need the `thread` feature if `service` is enabled.
# Figure out how to enable features of an already pulled in dependency conditionally.
cuprate-helper = { path = "../helper", features = ["fs", "thread"] }
cuprate-types = { path = "../types", features = ["service"] }
paste = { workspace = true }
page_size = { version = "0.6.0" } # Needed for database resizes, they must be a multiple of the OS page size.
thiserror = { workspace = true }
cuprate-helper = { path = "../helper", features = ["fs", "thread"] }
cuprate-types = { path = "../types", features = ["service"] }
curve25519-dalek = { workspace = true }
monero-pruning = { path = "../pruning" }
monero-serai = { workspace = true, features = ["std"] }
paste = { workspace = true }
page_size = { version = "0.6.0" } # Needed for database resizes, they must be a multiple of the OS page size.
thiserror = { workspace = true }
# `service` feature.
crossbeam = { workspace = true, features = ["std"], optional = true }
@ -46,5 +50,9 @@ serde = { workspace = true, optional = true }
[dev-dependencies]
bytemuck = { version = "1.14.3", features = ["must_cast", "derive", "min_const_generics", "extern_crate_alloc"] }
cuprate-helper = { path = "../helper", features = ["thread"] }
cuprate-test-utils = { path = "../test-utils" }
page_size = { version = "0.6.0" }
tempfile = { version = "3.10.0" }
pretty_assertions = { version = "1.4.0" }
hex = { workspace = true }
hex-literal = { workspace = true }

61
database/src/backend/heed/database.rs
View file

@ -204,55 +204,56 @@ impl<T: Table> DatabaseRw<T> for HeedTableRw<'_, '_, T> {
Ok(())
}
#[inline]
fn take(&mut self, key: &T::Key) -> Result<T::Value, RuntimeError> {
// LMDB/heed does not return the value on deletion.
// So, fetch it first - then delete.
let value = get::<T>(&self.db, &self.tx_rw.borrow(), key)?;
match self.db.delete(&mut self.tx_rw.borrow_mut(), key) {
Ok(true) => Ok(value),
Err(e) => Err(e.into()),
// We just `get()`'ed the value - it is
// incorrect for it to suddenly not exist.
Ok(false) => unreachable!(),
}
}
#[inline]
fn pop_first(&mut self) -> Result<(T::Key, T::Value), RuntimeError> {
let tx_rw = &mut self.tx_rw.borrow_mut();
// Get the first value first...
let Some(first) = self.db.first(tx_rw)? else {
// Get the value first...
let Some((key, value)) = self.db.first(tx_rw)? else {
return Err(RuntimeError::KeyNotFound);
};
// ...then remove it.
//
// We use an iterator because we want to semantically
// remove the _first_ and only the first `(key, value)`.
// `delete()` removes all keys including duplicates which
// is slightly different behavior.
let mut iter = self.db.iter_mut(tx_rw)?;
// SAFETY:
// It is undefined behavior to keep a reference of
// a value from this database while modifying it.
// We are deleting the value and never accessing
// the iterator again so this should be safe.
unsafe {
iter.del_current()?;
match self.db.delete(tx_rw, &key) {
Ok(true) => Ok((key, value)),
Err(e) => Err(e.into()),
// We just `get()`'ed the value - it is
// incorrect for it to suddenly not exist.
Ok(false) => unreachable!(),
}
Ok(first)
}
#[inline]
fn pop_last(&mut self) -> Result<(T::Key, T::Value), RuntimeError> {
let tx_rw = &mut self.tx_rw.borrow_mut();
let Some(first) = self.db.last(tx_rw)? else {
// Get the value first...
let Some((key, value)) = self.db.last(tx_rw)? else {
return Err(RuntimeError::KeyNotFound);
};
let mut iter = self.db.rev_iter_mut(tx_rw)?;
// SAFETY:
// It is undefined behavior to keep a reference of
// a value from this database while modifying it.
// We are deleting the value and never accessing
// the iterator again so this should be safe.
unsafe {
iter.del_current()?;
// ...then remove it.
match self.db.delete(tx_rw, &key) {
Ok(true) => Ok((key, value)),
Err(e) => Err(e.into()),
// We just `get()`'ed the value - it is
// incorrect for it to suddenly not exist.
Ok(false) => unreachable!(),
}
Ok(first)
}
}

12
database/src/backend/heed/env.rs
View file

@ -207,17 +207,15 @@ impl Env for ConcreteEnv {
}
use crate::tables::{
BlockBlobs, BlockHeights, BlockInfoV1s, BlockInfoV2s, BlockInfoV3s, KeyImages,
NumOutputs, Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs,
TxHeights, TxIds, TxUnlockTime,
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxBlobs, TxHeights, TxIds, TxOutputs,
TxUnlockTime,
};
let mut tx_rw = env.write_txn()?;
create_table::<BlockBlobs>(&env, &mut tx_rw)?;
create_table::<BlockHeights>(&env, &mut tx_rw)?;
create_table::<BlockInfoV1s>(&env, &mut tx_rw)?;
create_table::<BlockInfoV2s>(&env, &mut tx_rw)?;
create_table::<BlockInfoV3s>(&env, &mut tx_rw)?;
create_table::<BlockInfos>(&env, &mut tx_rw)?;
create_table::<KeyImages>(&env, &mut tx_rw)?;
create_table::<NumOutputs>(&env, &mut tx_rw)?;
create_table::<Outputs>(&env, &mut tx_rw)?;
@ -225,8 +223,10 @@ impl Env for ConcreteEnv {
create_table::<PrunableTxBlobs>(&env, &mut tx_rw)?;
create_table::<PrunedTxBlobs>(&env, &mut tx_rw)?;
create_table::<RctOutputs>(&env, &mut tx_rw)?;
create_table::<TxBlobs>(&env, &mut tx_rw)?;
create_table::<TxHeights>(&env, &mut tx_rw)?;
create_table::<TxIds>(&env, &mut tx_rw)?;
create_table::<TxOutputs>(&env, &mut tx_rw)?;
create_table::<TxUnlockTime>(&env, &mut tx_rw)?;
// TODO: Set dupsort and comparison functions for certain tables

9
database/src/backend/redb/database.rs
View file

@ -188,6 +188,15 @@ impl<T: Table + 'static> DatabaseRw<T> for RedbTableRw<'_, T::Key, T::Value> {
Ok(())
}
#[inline]
fn take(&mut self, key: &T::Key) -> Result<T::Value, RuntimeError> {
if let Some(value) = redb::Table::remove(self, key)? {
Ok(value.value())
} else {
Err(RuntimeError::KeyNotFound)
}
}
#[inline]
fn pop_first(&mut self) -> Result<(T::Key, T::Value), RuntimeError> {
let (key, value) = redb::Table::pop_first(self)?.ok_or(RuntimeError::KeyNotFound)?;

12
database/src/backend/redb/env.rs
View file

@ -109,17 +109,15 @@ impl Env for ConcreteEnv {
}
use crate::tables::{
BlockBlobs, BlockHeights, BlockInfoV1s, BlockInfoV2s, BlockInfoV3s, KeyImages,
NumOutputs, Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs,
TxHeights, TxIds, TxUnlockTime,
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxBlobs, TxHeights, TxIds, TxOutputs,
TxUnlockTime,
};
let tx_rw = env.begin_write()?;
create_table::<BlockBlobs>(&tx_rw)?;
create_table::<BlockHeights>(&tx_rw)?;
create_table::<BlockInfoV1s>(&tx_rw)?;
create_table::<BlockInfoV2s>(&tx_rw)?;
create_table::<BlockInfoV3s>(&tx_rw)?;
create_table::<BlockInfos>(&tx_rw)?;
create_table::<KeyImages>(&tx_rw)?;
create_table::<NumOutputs>(&tx_rw)?;
create_table::<Outputs>(&tx_rw)?;
@ -127,8 +125,10 @@ impl Env for ConcreteEnv {
create_table::<PrunableTxBlobs>(&tx_rw)?;
create_table::<PrunedTxBlobs>(&tx_rw)?;
create_table::<RctOutputs>(&tx_rw)?;
create_table::<TxBlobs>(&tx_rw)?;
create_table::<TxHeights>(&tx_rw)?;
create_table::<TxIds>(&tx_rw)?;
create_table::<TxOutputs>(&tx_rw)?;
create_table::<TxUnlockTime>(&tx_rw)?;
tx_rw.commit()?;

112
database/src/backend/tests.rs
View file

@ -31,32 +31,21 @@ use crate::{
storable::StorableVec,
table::Table,
tables::{
BlockBlobs, BlockHeights, BlockInfoV1s, BlockInfoV2s, BlockInfoV3s, KeyImages, NumOutputs,
Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxHeights, TxIds,
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxBlobs, TxHeights, TxIds, TxOutputs,
TxUnlockTime,
},
tests::tmp_concrete_env,
transaction::{TxRo, TxRw},
types::{
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfoV1,
BlockInfoV2, BlockInfoV3, KeyImage, Output, PreRctOutputId, PrunableBlob, PrunableHash,
PrunedBlob, RctOutput, TxHash, TxId, UnlockTime,
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfo, KeyImage,
Output, OutputFlags, PreRctOutputId, PrunableBlob, PrunableHash, PrunedBlob, RctOutput,
TxBlob, TxHash, TxId, UnlockTime,
},
ConcreteEnv,
};
//---------------------------------------------------------------------------------------------------- Tests
/// Create an `Env` in a temporarily directory.
/// The directory is automatically removed after the `TempDir` is dropped.
///
/// TODO: changing this to `-> impl Env` causes lifetime errors...
fn tmp_concrete_env() -> (ConcreteEnv, tempfile::TempDir) {
let tempdir = tempfile::tempdir().unwrap();
let config = Config::low_power(Some(tempdir.path().into()));
let env = ConcreteEnv::open(config).unwrap();
(env, tempdir)
}
/// Simply call [`Env::open`]. If this fails, something is really wrong.
#[test]
fn open() {
@ -87,9 +76,7 @@ fn open_db() {
// This should be updated when tables are modified.
env_inner.open_db_ro::<BlockBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockHeights>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfoV1s>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfoV2s>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfoV3s>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfos>(&tx_ro).unwrap();
env_inner.open_db_ro::<KeyImages>(&tx_ro).unwrap();
env_inner.open_db_ro::<NumOutputs>(&tx_ro).unwrap();
env_inner.open_db_ro::<Outputs>(&tx_ro).unwrap();
@ -97,17 +84,17 @@ fn open_db() {
env_inner.open_db_ro::<PrunableTxBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<PrunedTxBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<RctOutputs>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxHeights>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxIds>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxOutputs>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxUnlockTime>(&tx_ro).unwrap();
TxRo::commit(tx_ro).unwrap();
// Open all tables in read/write mode.
env_inner.open_db_rw::<BlockBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockHeights>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfoV1s>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfoV2s>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfoV3s>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfos>(&tx_rw).unwrap();
env_inner.open_db_rw::<KeyImages>(&tx_rw).unwrap();
env_inner.open_db_rw::<NumOutputs>(&tx_rw).unwrap();
env_inner.open_db_rw::<Outputs>(&tx_rw).unwrap();
@ -115,8 +102,10 @@ fn open_db() {
env_inner.open_db_rw::<PrunableTxBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<PrunedTxBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<RctOutputs>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxHeights>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxIds>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxOutputs>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxUnlockTime>(&tx_rw).unwrap();
TxRw::commit(tx_rw).unwrap();
}
@ -182,7 +171,7 @@ fn db_read_write() {
const VALUE: Output = Output {
key: [35; 32],
height: 45_761_798,
output_flags: 0,
output_flags: OutputFlags::empty(),
tx_idx: 2_353_487,
};
/// How many `(key, value)` pairs will be inserted.
@ -271,6 +260,22 @@ fn db_read_write() {
}
}
// Assert `update()` works.
{
const HEIGHT: u32 = 999;
assert_ne!(table.get(&KEY).unwrap().height, HEIGHT);
table
.update(&KEY, |mut value| {
value.height = HEIGHT;
Some(value)
})
.unwrap();
assert_eq!(table.get(&KEY).unwrap().height, HEIGHT);
}
// Assert deleting works.
{
table.delete(&KEY).unwrap();
@ -284,6 +289,23 @@ fn db_read_write() {
assert_same(value);
}
// Assert `take()` works.
{
let mut key = KEY;
key.amount += 1;
let value = table.take(&key).unwrap();
assert_eq!(value, VALUE);
let get = table.get(&KEY);
assert!(!table.contains(&key).unwrap());
assert!(matches!(get, Err(RuntimeError::KeyNotFound)));
// Assert the other `(key, value)` pairs are still there.
key.amount += 1;
let value = table.get(&key).unwrap();
assert_same(value);
}
drop(table);
TxRw::commit(tx_rw).unwrap();
@ -406,36 +428,14 @@ test_tables! {
BlockHash => BlockHeight,
[32; 32] => 123,
BlockInfoV1s,
BlockHeight => BlockInfoV1,
123 => BlockInfoV1 {
BlockInfos,
BlockHeight => BlockInfo,
123 => BlockInfo {
timestamp: 1,
total_generated_coins: 123,
cumulative_generated_coins: 123,
weight: 321,
cumulative_difficulty: 111,
block_hash: [54; 32],
},
BlockInfoV2s,
BlockHeight => BlockInfoV2,
123 => BlockInfoV2 {
timestamp: 1,
total_generated_coins: 123,
weight: 321,
cumulative_difficulty: 111,
cumulative_rct_outs: 2389,
block_hash: [54; 32],
},
BlockInfoV3s,
BlockHeight => BlockInfoV3,
123 => BlockInfoV3 {
timestamp: 1,
total_generated_coins: 123,
weight: 321,
cumulative_difficulty_low: 111,
cumulative_difficulty_high: 112,
block_hash: [54; 32],
cumulative_rct_outs: 2389,
long_term_weight: 2389,
},
@ -448,6 +448,10 @@ test_tables! {
Amount => AmountIndex,
123 => 123,
TxBlobs,
TxId => TxBlob,
123 => StorableVec(vec![1,2,3,4,5,6,7,8]),
TxIds,
TxHash => TxId,
[32; 32] => 123,
@ -456,6 +460,10 @@ test_tables! {
TxId => BlockHeight,
123 => 123,
TxOutputs,
TxId => AmountIndices,
123 => StorableVec(vec![1,2,3,4,5,6,7,8]),
TxUnlockTime,
TxId => UnlockTime,
123 => 123,
@ -468,7 +476,7 @@ test_tables! {
} => Output {
key: [1; 32],
height: 1,
output_flags: 0,
output_flags: OutputFlags::empty(),
tx_idx: 3,
},
@ -489,7 +497,7 @@ test_tables! {
123 => RctOutput {
key: [1; 32],
height: 1,
output_flags: 0,
output_flags: OutputFlags::empty(),
tx_idx: 3,
commitment: [3; 32],
},

12
database/src/constants.rs
View file

@ -3,6 +3,18 @@
//---------------------------------------------------------------------------------------------------- Import
use cfg_if::cfg_if;
//---------------------------------------------------------------------------------------------------- Version
/// Current major version of the database.
///
/// Returned by [`crate::ops::property::db_version`].
///
/// This is incremented by 1 when `cuprate_database`'s
/// structure/schema/tables change.
///
/// This is akin to `VERSION` in `monerod`:
/// <https://github.com/monero-project/monero/blob/c8214782fb2a769c57382a999eaf099691c836e7/src/blockchain_db/lmdb/db_lmdb.cpp#L57>
pub const DATABASE_VERSION: u64 = 0;
//---------------------------------------------------------------------------------------------------- Error Messages
/// Corrupt database error message.
///

42
database/src/database.rs
View file

@ -13,7 +13,7 @@ use crate::{
transaction::{TxRo, TxRw},
};
//---------------------------------------------------------------------------------------------------- DatabaseRoIter
//---------------------------------------------------------------------------------------------------- DatabaseIter
/// Database (key-value store) read-only iteration abstraction.
///
/// These are read-only iteration-related operations that
@ -140,14 +140,50 @@ pub trait DatabaseRw<T: Table>: DatabaseRo<T> {
/// This will overwrite any existing key-value pairs.
///
/// # Errors
/// This will not return [`RuntimeError::KeyExists`].
/// This will never [`RuntimeError::KeyExists`].
fn put(&mut self, key: &T::Key, value: &T::Value) -> Result<(), RuntimeError>;
/// Delete a key-value pair in the database.
///
/// This will return `Ok(())` if the key does not exist.
///
/// # Errors
/// This will never [`RuntimeError::KeyNotFound`].
fn delete(&mut self, key: &T::Key) -> Result<(), RuntimeError>;
/// Delete and return a key-value pair in the database.
///
/// This is the same as [`DatabaseRw::delete`], however,
/// it will serialize the `T::Value` and return it.
///
/// # Errors
/// This will return [`RuntimeError::KeyNotFound`] wrapped in [`Err`] if `key` does not exist.
fn delete(&mut self, key: &T::Key) -> Result<(), RuntimeError>;
fn take(&mut self, key: &T::Key) -> Result<T::Value, RuntimeError>;
/// Fetch the value, and apply a function to it - or delete the entry.
///
/// This will call [`DatabaseRo::get`] and call your provided function `f` on it.
///
/// The [`Option`] `f` returns will dictate whether `update()`:
/// - Updates the current value OR
/// - Deletes the `(key, value)` pair
///
/// - If `f` returns `Some(value)`, that will be [`DatabaseRw::put`] as the new value
/// - If `f` returns `None`, the entry will be [`DatabaseRw::delete`]d
///
/// # Errors
/// This will return [`RuntimeError::KeyNotFound`] wrapped in [`Err`] if `key` does not exist.
fn update<F>(&mut self, key: &T::Key, mut f: F) -> Result<(), RuntimeError>
where
F: FnMut(T::Value) -> Option<T::Value>,
{
let value = DatabaseRo::get(self, key)?;
match f(value) {
Some(value) => DatabaseRw::put(self, key, &value),
None => DatabaseRw::delete(self, key),
}
}
/// TODO
///

25
database/src/env.rs
View file

@ -9,6 +9,11 @@ use crate::{
error::{InitError, RuntimeError},
resize::ResizeAlgorithm,
table::Table,
tables::{
call_fn_on_all_tables_or_early_return, BlockBlobs, BlockHeights, BlockInfos, KeyImages,
NumOutputs, Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables,
TablesMut, TxHeights, TxIds, TxUnlockTime,
},
transaction::{TxRo, TxRw},
};
@ -224,6 +229,26 @@ where
/// a table doesn't exist.
fn open_db_rw<T: Table>(&self, tx_rw: &Rw) -> Result<impl DatabaseRw<T>, RuntimeError>;
/// TODO
///
/// # Errors
/// TODO
fn open_tables(&self, tx_ro: &Ro) -> Result<impl Tables, RuntimeError> {
call_fn_on_all_tables_or_early_return! {
Self::open_db_ro(self, tx_ro)
}
}
/// TODO
///
/// # Errors
/// TODO
fn open_tables_mut(&self, tx_rw: &Rw) -> Result<impl TablesMut, RuntimeError> {
call_fn_on_all_tables_or_early_return! {
Self::open_db_rw(self, tx_rw)
}
}
/// Clear all `(key, value)`'s from a database table.
///
/// This will delete all key and values in the passed

34
database/src/lib.rs
View file

@ -1,4 +1,4 @@
//! Database abstraction and utilities.
//! Cuprate's database abstraction.
//!
//! This documentation is mostly for practical usage of `cuprate_database`.
//!
@ -8,28 +8,33 @@
//! # Purpose
//! This crate does 3 things:
//! 1. Abstracts various database backends with traits
//! 2. Implements various `Monero` related [functions](ops) & [tables] & [types]
//! 2. Implements various `Monero` related [operations](ops), [tables], and [types]
//! 3. Exposes a [`tower::Service`] backed by a thread-pool
//!
//! Each layer builds on-top of the previous.
//!
//! As a user of `cuprate_database`, consider using the higher-level [`service`],
//! or at the very least [`ops`] instead of interacting with the database traits directly.
//!
//! With that said, many database traits and internals (like [`DatabaseRo::get`]) are exposed.
//!
//! # Terminology
//! To be more clear on some terms used in this crate:
//!
//! | Term | Meaning |
//! |---------------|--------------------------------------|
//! | `Env` | The 1 database environment, the "whole" thing
//! | `DatabaseRo` | A read-only `key/value` store
//! | `DatabaseRw` | A readable/writable `key/value` store
//! | `Table` | Solely the metadata of a `Database` (the `key` and `value` types, and the name)
//! | `TxRo` | Read only transaction
//! | `TxRw` | Read/write transaction
//! | `Storable` | A data that type can be stored in the database
//! | Term | Meaning |
//! |------------------|--------------------------------------|
//! | `Env` | The 1 database environment, the "whole" thing
//! | `DatabaseR{o,w}` | A _actively open_ readable/writable `key/value` store
//! | `Table` | Solely the metadata of a `Database` (the `key` and `value` types, and the name)
//! | `TxR{o,w}` | A read/write transaction
//! | `Storable` | A data that type can be stored in the database
//!
//! The dataflow is `Env` -> `Tx` -> `Database`
//!
//! Which reads as:
//! 1. You have a database `Environment`
//! 1. You open up a `Transaction`
//! 1. You get a particular `Database` from that `Environment`
//! 1. You open a particular `Table` from that `Environment`, getting a `Database`
//! 1. You can now read/write data from/to that `Database`
//!
//! # `ConcreteEnv`
@ -138,7 +143,6 @@
unconditional_recursion,
for_loops_over_fallibles,
unused_braces,
unused_doc_comments,
unused_labels,
keyword_idents,
non_ascii_idents,
@ -169,6 +173,7 @@
clippy::pedantic,
clippy::nursery,
clippy::cargo,
unused_doc_comments,
unused_mut,
missing_docs,
deprecated,
@ -221,6 +226,7 @@ pub mod config;
mod constants;
pub use constants::{
DATABASE_BACKEND, DATABASE_CORRUPT_MSG, DATABASE_DATA_FILENAME, DATABASE_LOCK_FILENAME,
DATABASE_VERSION,
};
mod database;
@ -261,3 +267,5 @@ pub use transaction::{TxRo, TxRw};
pub mod service;
//---------------------------------------------------------------------------------------------------- Private
#[cfg(test)]
pub(crate) mod tests;

483
database/src/ops/block.rs
View file

@ -1,89 +1,450 @@
//! Blocks.
//---------------------------------------------------------------------------------------------------- Import
use std::sync::Arc;
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn add_block() {
todo!()
use bytemuck::TransparentWrapper;
use curve25519_dalek::{constants::ED25519_BASEPOINT_POINT, Scalar};
use monero_serai::{
block::Block,
transaction::{Input, Timelock, Transaction},
};
use cuprate_types::{ExtendedBlockHeader, TransactionVerificationData, VerifiedBlockInformation};
use crate::{
database::{DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::{
blockchain::{chain_height, cumulative_generated_coins},
key_image::{add_key_image, remove_key_image},
macros::doc_error,
output::{
add_output, add_rct_output, get_rct_num_outputs, remove_output, remove_rct_output,
},
tx::{add_tx, get_num_tx, remove_tx},
},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
AmountIndex, BlockHash, BlockHeight, BlockInfo, KeyImage, Output, OutputFlags,
PreRctOutputId, RctOutput, TxHash,
},
StorableVec,
};
//---------------------------------------------------------------------------------------------------- `add_block_*`
/// Add a [`VerifiedBlockInformation`] to the database.
///
/// This extracts all the data from the input block and
/// maps/adds them to the appropriate database tables.
///
#[doc = doc_error!()]
///
/// # Panics
/// This function will panic if:
/// - `block.height > u32::MAX` (not normally possible)
/// - `block.height` is not != [`chain_height`]
///
/// # Already exists
/// This function will operate normally even if `block` already
/// exists, i.e., this function will not return `Err` even if you
/// call this function infinitely with the same block.
// no inline, too big.
pub fn add_block(
block: &VerifiedBlockInformation,
tables: &mut impl TablesMut,
) -> Result<(), RuntimeError> {
//------------------------------------------------------ Check preconditions first
// Cast height to `u32` for storage (handled at top of function).
// Panic (should never happen) instead of allowing DB corruption.
// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1560020991>
let Ok(height) = u32::try_from(block.height) else {
panic!("block.height ({}) > u32::MAX", block.height);
};
let chain_height = chain_height(tables.block_heights())?;
assert_eq!(
block.height, chain_height,
"block.height ({}) != chain_height ({})",
block.height, chain_height,
);
// Expensive checks - debug only.
#[cfg(debug_assertions)]
{
assert_eq!(block.block.serialize(), block.block_blob);
assert_eq!(block.block.txs.len(), block.txs.len());
for (i, tx) in block.txs.iter().enumerate() {
assert_eq!(tx.tx_blob, tx.tx.serialize());
assert_eq!(tx.tx_hash, block.block.txs[i]);
}
}
//------------------------------------------------------ Transaction / Outputs / Key Images
for tx_verification_data in &block.txs {
add_tx(tx_verification_data, &chain_height, tables)?;
}
//------------------------------------------------------ Block Info
// INVARIANT: must be below the above transaction loop since this
// RCT output count needs account for _this_ block's outputs.
let cumulative_rct_outs = get_rct_num_outputs(tables.rct_outputs())?;
let cumulative_generated_coins =
cumulative_generated_coins(&block.height.saturating_sub(1), tables.block_infos())?
+ block.generated_coins;
// Block Info.
tables.block_infos_mut().put(
&block.height,
&BlockInfo {
cumulative_generated_coins,
cumulative_rct_outs,
timestamp: block.block.header.timestamp,
cumulative_difficulty: block.cumulative_difficulty,
block_hash: block.block_hash,
// INVARIANT: #[cfg] @ lib.rs asserts `usize == u64`
weight: block.weight as u64,
long_term_weight: block.long_term_weight as u64,
},
)?;
// Block blobs.
tables
.block_blobs_mut()
.put(&block.height, StorableVec::wrap_ref(&block.block_blob))?;
// Block heights.
tables
.block_heights_mut()
.put(&block.block_hash, &block.height)?;
Ok(())
}
/// TODO
pub fn add_block_data() {
todo!()
//---------------------------------------------------------------------------------------------------- `pop_block`
/// Remove the top/latest block from the database.
///
/// The removed block's height and hash are returned.
#[doc = doc_error!()]
// no inline, too big
pub fn pop_block(
tables: &mut impl TablesMut,
) -> Result<(BlockHeight, BlockHash, Block), RuntimeError> {
//------------------------------------------------------ Block Info
// Remove block data from tables.
let (block_height, block_hash) = {
let (block_height, block_info) = tables.block_infos_mut().pop_last()?;
(block_height, block_info.block_hash)
};
// Block heights.
tables.block_heights_mut().delete(&block_hash)?;
// Block blobs.
// We deserialize the block blob into a `Block`, such
// that we can remove the associated transactions later.
let block_blob = tables.block_blobs_mut().take(&block_height)?.0;
let block = Block::read(&mut block_blob.as_slice())?;
//------------------------------------------------------ Transaction / Outputs / Key Images
for tx_hash in &block.txs {
remove_tx(tx_hash, tables)?;
}
Ok((block_height, block_hash, block))
}
/// TODO
pub fn pop_block() {
todo!()
//---------------------------------------------------------------------------------------------------- `get_block_extended_header_*`
/// Retrieve a [`ExtendedBlockHeader`] from the database.
///
/// This extracts all the data from the database tables
/// needed to create a full `ExtendedBlockHeader`.
///
/// # Notes
/// This is slightly more expensive than [`get_block_extended_header_from_height`]
/// (1 more database lookup).
#[doc = doc_error!()]
#[inline]
pub fn get_block_extended_header(
block_hash: &BlockHash,
tables: &impl Tables,
) -> Result<ExtendedBlockHeader, RuntimeError> {
get_block_extended_header_from_height(&tables.block_heights().get(block_hash)?, tables)
}
/// TODO
pub fn block_exists() {
todo!()
/// Same as [`get_block_extended_header`] but with a [`BlockHeight`].
#[doc = doc_error!()]
#[inline]
pub fn get_block_extended_header_from_height(
block_height: &BlockHeight,
tables: &impl Tables,
) -> Result<ExtendedBlockHeader, RuntimeError> {
let block_info = tables.block_infos().get(block_height)?;
let block_blob = tables.block_blobs().get(block_height)?.0;
let block = Block::read(&mut block_blob.as_slice())?;
// INVARIANT: #[cfg] @ lib.rs asserts `usize == u64`
#[allow(clippy::cast_possible_truncation)]
Ok(ExtendedBlockHeader {
version: block.header.major_version,
vote: block.header.minor_version,
timestamp: block.header.timestamp,
cumulative_difficulty: block_info.cumulative_difficulty,
block_weight: block_info.weight as usize,
long_term_weight: block_info.long_term_weight as usize,
})
}
/// TODO
pub fn get_block_hash() {
todo!()
/// Return the top/latest [`ExtendedBlockHeader`] from the database.
#[doc = doc_error!()]
#[inline]
pub fn get_block_extended_header_top(
tables: &impl Tables,
) -> Result<(ExtendedBlockHeader, BlockHeight), RuntimeError> {
let height = chain_height(tables.block_heights())?.saturating_sub(1);
let header = get_block_extended_header_from_height(&height, tables)?;
Ok((header, height))
}
/// TODO
pub fn get_block_height() {
todo!()
//---------------------------------------------------------------------------------------------------- Misc
/// Retrieve a [`BlockHeight`] via its [`BlockHash`].
#[doc = doc_error!()]
#[inline]
pub fn get_block_height(
block_hash: &BlockHash,
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<BlockHeight, RuntimeError> {
table_block_heights.get(block_hash)
}
/// TODO
pub fn get_block_weight() {
todo!()
/// Check if a block exists in the database.
#[doc = doc_error!()]
#[inline]
pub fn block_exists(
block_hash: &BlockHash,
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<bool, RuntimeError> {
table_block_heights.contains(block_hash)
}
/// TODO
pub fn get_block_already_generated_coins() {
todo!()
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(
clippy::significant_drop_tightening,
clippy::cognitive_complexity,
clippy::too_many_lines
)]
mod test {
use hex_literal::hex;
use pretty_assertions::assert_eq;
/// TODO
pub fn get_block_long_term_weight() {
todo!()
}
use cuprate_test_utils::data::{block_v16_tx0, block_v1_tx2, block_v9_tx3, tx_v2_rct3};
/// TODO
pub fn get_block_timestamp() {
todo!()
}
use super::*;
use crate::{
ops::tx::{get_tx, tx_exists},
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
/// TODO
pub fn get_block_cumulative_rct_outputs() {
todo!()
}
/// Tests all above block functions.
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
fn all_block_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
/// TODO
pub fn get_block() {
todo!()
}
let mut blocks = [
block_v1_tx2().clone(),
block_v9_tx3().clone(),
block_v16_tx0().clone(),
];
// HACK: `add_block()` asserts blocks with non-sequential heights
// cannot be added, to get around this, manually edit the block height.
for (height, block) in blocks.iter_mut().enumerate() {
block.height = height as u64;
assert_eq!(block.block.serialize(), block.block_blob);
}
let generated_coins_sum = blocks
.iter()
.map(|block| block.generated_coins)
.sum::<u64>();
/// TODO
pub fn get_block_from_height() {
todo!()
}
// Add blocks.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
/// TODO
pub fn get_block_header() {
todo!()
}
for block in &blocks {
// println!("add_block: {block:#?}");
add_block(block, &mut tables).unwrap();
}
/// TODO
pub fn get_block_header_from_height() {
todo!()
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
/// TODO
pub fn get_top_block() {
todo!()
}
// Assert all reads are OK.
let block_hashes = {
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
/// TODO
pub fn get_top_block_hash() {
todo!()
// Assert only the proper tables were added to.
assert_eq!(tables.block_infos().len().unwrap(), 3);
assert_eq!(tables.block_blobs().len().unwrap(), 3);
assert_eq!(tables.block_heights().len().unwrap(), 3);
assert_eq!(tables.key_images().len().unwrap(), 69);
assert_eq!(tables.num_outputs().len().unwrap(), 38);
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.outputs().len().unwrap(), 107);
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 6);
assert_eq!(tables.tx_blobs().len().unwrap(), 5);
assert_eq!(tables.tx_ids().len().unwrap(), 5);
assert_eq!(tables.tx_heights().len().unwrap(), 5);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 0);
// Check `cumulative` functions work.
assert_eq!(
cumulative_generated_coins(&2, tables.block_infos()).unwrap(),
generated_coins_sum,
);
// Both height and hash should result in getting the same data.
let mut block_hashes = vec![];
for block in &blocks {
println!("blocks.iter(): hash: {}", hex::encode(block.block_hash));
let height = get_block_height(&block.block_hash, tables.block_heights()).unwrap();
println!("blocks.iter(): height: {height}");
assert!(block_exists(&block.block_hash, tables.block_heights()).unwrap());
let block_header_from_height =
get_block_extended_header_from_height(&height, &tables).unwrap();
let block_header_from_hash =
get_block_extended_header(&block.block_hash, &tables).unwrap();
// Just an alias, these names are long.
let b1 = block_header_from_hash;
let b2 = block;
assert_eq!(b1, block_header_from_height);
assert_eq!(b1.version, b2.block.header.major_version);
assert_eq!(b1.vote, b2.block.header.minor_version);
assert_eq!(b1.timestamp, b2.block.header.timestamp);
assert_eq!(b1.cumulative_difficulty, b2.cumulative_difficulty);
assert_eq!(b1.block_weight, b2.weight);
assert_eq!(b1.long_term_weight, b2.long_term_weight);
block_hashes.push(block.block_hash);
// Assert transaction reads are OK.
for (i, tx) in block.txs.iter().enumerate() {
println!("tx_hash: {:?}", hex::encode(tx.tx_hash));
assert!(tx_exists(&tx.tx_hash, tables.tx_ids()).unwrap());
let tx2 = get_tx(&tx.tx_hash, tables.tx_ids(), tables.tx_blobs()).unwrap();
assert_eq!(tx.tx_blob, tx2.serialize());
assert_eq!(tx.tx_weight, tx2.weight());
assert_eq!(tx.tx_hash, block.block.txs[i]);
assert_eq!(tx.tx_hash, tx2.hash());
}
}
block_hashes
};
{
let len = block_hashes.len();
let hashes: Vec<String> = block_hashes.iter().map(hex::encode).collect();
println!("block_hashes: len: {len}, hashes: {hashes:?}");
}
// Remove the blocks.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for block_hash in block_hashes.into_iter().rev() {
println!("pop_block(): block_hash: {}", hex::encode(block_hash));
let (popped_height, popped_hash, popped_block) = pop_block(&mut tables).unwrap();
assert_eq!(block_hash, popped_hash);
assert!(matches!(
get_block_extended_header(&block_hash, &tables),
Err(RuntimeError::KeyNotFound)
));
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
assert_all_tables_are_empty(&env);
}
/// We should panic if: `block.height` > `u32::MAX`
#[test]
#[should_panic(expected = "block.height (4294967296) > u32::MAX")]
fn block_height_gt_u32_max() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
let mut block = block_v9_tx3().clone();
block.height = u64::from(u32::MAX) + 1;
add_block(&block, &mut tables).unwrap();
}
/// We should panic if: `block.height` != the chain height
#[test]
#[should_panic(
expected = "assertion `left == right` failed: block.height (123) != chain_height (1)\n left: 123\n right: 1"
)]
fn block_height_not_chain_height() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
let mut block = block_v9_tx3().clone();
// HACK: `add_block()` asserts blocks with non-sequential heights
// cannot be added, to get around this, manually edit the block height.
block.height = 0;
// OK, `0 == 0`
assert_eq!(block.height, 0);
add_block(&block, &mut tables).unwrap();
// FAIL, `123 != 1`
block.height = 123;
add_block(&block, &mut tables).unwrap();
}
}

175
database/src/ops/blockchain.rs
View file

@ -1,9 +1,178 @@
//! Blockchain.
//---------------------------------------------------------------------------------------------------- Import
use monero_serai::transaction::Timelock;
use cuprate_types::VerifiedBlockInformation;
use crate::{
database::{DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::doc_error,
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId, RctOutput},
};
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn height() {
todo!()
/// Retrieve the height of the chain.
///
/// This returns the chain-tip, not the [`top_block_height`].
///
/// For example:
/// - The blockchain has 0 blocks => this returns `0`
/// - The blockchain has 1 block (height 0) => this returns `1`
/// - The blockchain has 2 blocks (height 1) => this returns `2`
///
/// So the height of a new block would be `chain_height()`.
#[doc = doc_error!()]
#[inline]
pub fn chain_height(
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<BlockHeight, RuntimeError> {
table_block_heights.len()
}
/// Retrieve the height of the top block.
///
/// This returns the height of the top block, not the [`chain_height`].
///
/// For example:
/// - The blockchain has 0 blocks => this returns `Err(RuntimeError::KeyNotFound)`
/// - The blockchain has 1 block (height 0) => this returns `Ok(0)`
/// - The blockchain has 2 blocks (height 1) => this returns `Ok(1)`
///
/// Note that in cases where no blocks have been written to the
/// database yet, an error is returned: `Err(RuntimeError::KeyNotFound)`.
///
#[doc = doc_error!()]
#[inline]
pub fn top_block_height(
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<BlockHeight, RuntimeError> {
match table_block_heights.len()? {
0 => Err(RuntimeError::KeyNotFound),
height => Ok(height - 1),
}
}
/// Check how many cumulative generated coins there have been until a certain [`BlockHeight`].
///
/// This returns the total amount of Monero generated up to `block_height`
/// (including the block itself) in atomic units.
///
/// For example:
/// - on the genesis block `0`, this returns the amount block `0` generated
/// - on the next block `1`, this returns the amount block `0` and `1` generated
///
/// If no blocks have been added and `block_height == 0`
/// (i.e., the cumulative generated coins before genesis block is being calculated),
/// this returns `Ok(0)`.
#[doc = doc_error!()]
#[inline]
pub fn cumulative_generated_coins(
block_height: &BlockHeight,
table_block_infos: &impl DatabaseRo<BlockInfos>,
) -> Result<u64, RuntimeError> {
match table_block_infos.get(block_height) {
Ok(block_info) => Ok(block_info.cumulative_generated_coins),
Err(RuntimeError::KeyNotFound) if block_height == &0 => Ok(0),
Err(e) => Err(e),
}
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening)]
mod test {
use hex_literal::hex;
use pretty_assertions::assert_eq;
use cuprate_test_utils::data::{block_v16_tx0, block_v1_tx2, block_v9_tx3, tx_v2_rct3};
use super::*;
use crate::{
ops::{
block::add_block,
tx::{get_tx, tx_exists},
},
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
/// Tests all above functions.
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
#[allow(clippy::cognitive_complexity, clippy::cast_possible_truncation)]
fn all_blockchain_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let mut blocks = [
block_v1_tx2().clone(),
block_v9_tx3().clone(),
block_v16_tx0().clone(),
];
let blocks_len = u64::try_from(blocks.len()).unwrap();
// Add blocks.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
assert!(matches!(
top_block_height(tables.block_heights()),
Err(RuntimeError::KeyNotFound),
));
assert_eq!(
0,
cumulative_generated_coins(&0, tables.block_infos()).unwrap()
);
for (i, block) in blocks.iter_mut().enumerate() {
let i = u64::try_from(i).unwrap();
// HACK: `add_block()` asserts blocks with non-sequential heights
// cannot be added, to get around this, manually edit the block height.
block.height = i;
add_block(block, &mut tables).unwrap();
}
// Assert reads are correct.
assert_eq!(blocks_len, chain_height(tables.block_heights()).unwrap());
assert_eq!(
blocks_len - 1,
top_block_height(tables.block_heights()).unwrap()
);
assert_eq!(
cumulative_generated_coins(&0, tables.block_infos()).unwrap(),
13_138_270_467_918,
);
assert_eq!(
cumulative_generated_coins(&1, tables.block_infos()).unwrap(),
16_542_044_490_081,
);
assert_eq!(
cumulative_generated_coins(&2, tables.block_infos()).unwrap(),
17_142_044_490_081,
);
assert!(matches!(
cumulative_generated_coins(&3, tables.block_infos()),
Err(RuntimeError::KeyNotFound),
));
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
}
}

152
database/src/ops/key_image.rs Normal file
View file

@ -0,0 +1,152 @@
//! Spent keys.
//---------------------------------------------------------------------------------------------------- Import
use monero_serai::transaction::{Timelock, Transaction};
use cuprate_types::{OutputOnChain, VerifiedBlockInformation};
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::{doc_add_block_inner_invariant, doc_error},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId, RctOutput, TxHash,
},
};
//---------------------------------------------------------------------------------------------------- Key image functions
/// Add a [`KeyImage`] to the "spent" set in the database.
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn add_key_image(
key_image: &KeyImage,
table_key_images: &mut impl DatabaseRw<KeyImages>,
) -> Result<(), RuntimeError> {
table_key_images.put(key_image, &())
}
/// Remove a [`KeyImage`] from the "spent" set in the database.
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn remove_key_image(
key_image: &KeyImage,
table_key_images: &mut impl DatabaseRw<KeyImages>,
) -> Result<(), RuntimeError> {
table_key_images.delete(key_image)
}
/// Check if a [`KeyImage`] exists - i.e. if it is "spent".
#[doc = doc_error!()]
#[inline]
pub fn key_image_exists(
key_image: &KeyImage,
table_key_images: &impl DatabaseRo<KeyImages>,
) -> Result<bool, RuntimeError> {
table_key_images.contains(key_image)
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening, clippy::cognitive_complexity)]
mod test {
use hex_literal::hex;
use pretty_assertions::assert_eq;
use super::*;
use crate::{
ops::tx::{get_tx, tx_exists},
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
/// Tests all above key-image functions.
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
fn all_key_image_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let key_images = [
hex!("be1c87fc8f958f68fbe346a18dfb314204dca7573f61aae14840b8037da5c286"),
hex!("c5e4a592c11f34a12e13516ab2883b7c580d47b286b8fe8b15d57d2a18ade275"),
hex!("93288b646f858edfb0997ae08d7c76f4599b04c127f108e8e69a0696ae7ba334"),
hex!("726e9e3d8f826d24811183f94ff53aeba766c9efe6274eb80806f69b06bfa3fc"),
];
// Add.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for key_image in &key_images {
println!("add_key_image(): {}", hex::encode(key_image));
add_key_image(key_image, tables.key_images_mut()).unwrap();
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
// Assert all reads are OK.
{
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
// Assert only the proper tables were added to.
assert_eq!(
tables.key_images().len().unwrap(),
u64::try_from(key_images.len()).unwrap()
);
assert_eq!(tables.block_infos().len().unwrap(), 0);
assert_eq!(tables.block_blobs().len().unwrap(), 0);
assert_eq!(tables.block_heights().len().unwrap(), 0);
assert_eq!(tables.num_outputs().len().unwrap(), 0);
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.outputs().len().unwrap(), 0);
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 0);
assert_eq!(tables.tx_blobs().len().unwrap(), 0);
assert_eq!(tables.tx_ids().len().unwrap(), 0);
assert_eq!(tables.tx_heights().len().unwrap(), 0);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 0);
for key_image in &key_images {
println!("key_image_exists(): {}", hex::encode(key_image));
key_image_exists(key_image, tables.key_images()).unwrap();
}
}
// Remove.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for key_image in key_images {
println!("remove_key_image(): {}", hex::encode(key_image));
remove_key_image(&key_image, tables.key_images_mut()).unwrap();
assert!(!key_image_exists(&key_image, tables.key_images()).unwrap());
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
assert_all_tables_are_empty(&env);
}
}

33
database/src/ops/macros.rs Normal file
View file

@ -0,0 +1,33 @@
//! Macros.
//!
//! These generate repetitive documentation
//! for all the functions defined in `ops/`.
//---------------------------------------------------------------------------------------------------- Documentation macros
/// Generate documentation for the required `# Error` section.
macro_rules! doc_error {
() => {
r#"# Errors
This function returns [`RuntimeError::KeyNotFound`] if the input doesn't exist or other `RuntimeError`'s on database errors."#
};
}
pub(super) use doc_error;
/// Generate `# Invariant` documentation for internal `fn`'s
/// that should be called directly with caution.
macro_rules! doc_add_block_inner_invariant {
() => {
r#"# ⚠️ Invariant ⚠️
This function mainly exists to be used internally by the parent function [`crate::ops::block::add_block`].
`add_block()` makes sure all data related to the input is mutated, while
this function _does not_, it specifically mutates _particular_ tables.
This is usually undesired - although this function is still available to call directly.
When calling this function, ensure that either:
1. This effect (incomplete database mutation) is what is desired, or that...
2. ...the other tables will also be mutated to a correct state"#
};
}
pub(super) use doc_add_block_inner_invariant;

46
database/src/ops/mod.rs
View file

@ -4,18 +4,48 @@
//! traits in this crate to generically call Monero-related
//! database operations.
//!
//! # TODO
//! TODO: These functions should pretty much map 1-1 to the `Request` enum.
//! # `impl Table`
//! `ops/` functions take [`Tables`](crate::tables::Tables) and
//! [`TablesMut`](crate::tables::TablesMut) directly - these are
//! _already opened_ database tables.
//!
//! TODO: These are function names from `old_database/` for now.
//! The actual underlying functions (e.g `get()`) aren't implemented.
//! As such, the function puts the responsibility
//! of transactions, tables, etc on the caller.
//!
//! TODO: All of these functions need to take in generic
//! database trait parameters (and their actual inputs).
//! This does mean these functions are mostly as lean
//! as possible, so calling them in a loop should be okay.
//!
//! # Atomicity
//! As transactions are handled by the _caller_ of these functions,
//! it is up to the caller to decide what happens if one them return
//! an error.
//!
//! To maintain atomicity, transactions should be [`abort`](crate::transaction::TxRw::abort)ed
//! if one of the functions failed.
//!
//! For example, if [`add_block()`](block::add_block) is called and returns an [`Err`],
//! `abort`ing the transaction that opened the input `TableMut` would reverse all tables
//! mutated by `add_block()` up until the error, leaving it in the state it was in before
//! `add_block()` was called.
//!
//! # Sub-functions
//! The main functions within this module are mostly within the [`block`] module.
//!
//! Practically speaking, you should only be using 2 functions:
//! - [`add_block`](block::add_block)
//! - [`pop_block`](block::pop_block)
//!
//! The `block` functions are "parent" functions, calling other
//! sub-functions such as [`add_output()`](output::add_output). `add_output()`
//! itself only modifies output-related tables, while the `block` "parent" functions
//! (like `add_block` and `pop_block`) modify _everything_ that is required.
pub mod alt_block;
// pub mod alt_block; // TODO: is this needed?
pub mod block;
pub mod blockchain;
pub mod key_image;
pub mod output;
pub mod property;
pub mod spent_key;
pub mod tx;
mod macros;

282
database/src/ops/output.rs
View file

@ -1,34 +1,278 @@
//! Outputs.
//---------------------------------------------------------------------------------------------------- Import
use monero_serai::transaction::{Timelock, Transaction};
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn add_output() {
todo!()
use cuprate_types::{OutputOnChain, VerifiedBlockInformation};
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::{doc_add_block_inner_invariant, doc_error},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
Amount, AmountIndex, BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId,
RctOutput, TxHash,
},
};
//---------------------------------------------------------------------------------------------------- Pre-RCT Outputs
/// Add a Pre-RCT [`Output`] to the database.
///
/// Upon [`Ok`], this function returns the [`PreRctOutputId`] that
/// can be used to lookup the `Output` in [`get_output()`].
///
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn add_output(
amount: Amount,
output: &Output,
tables: &mut impl TablesMut,
) -> Result<PreRctOutputId, RuntimeError> {
// FIXME: this would be much better expressed with a
// `btree_map::Entry`-like API, fix `trait DatabaseRw`.
let num_outputs = match tables.num_outputs().get(&amount) {
// Entry with `amount` already exists.
Ok(num_outputs) => num_outputs,
// Entry with `amount` didn't exist, this is
// the 1st output with this amount.
Err(RuntimeError::KeyNotFound) => 0,
Err(e) => return Err(e),
};
// Update the amount of outputs.
tables.num_outputs_mut().put(&amount, &(num_outputs + 1))?;
let pre_rct_output_id = PreRctOutputId {
amount,
// The new `amount_index` is the length of amount of outputs with same amount.
amount_index: num_outputs,
};
tables.outputs_mut().put(&pre_rct_output_id, output)?;
Ok(pre_rct_output_id)
}
/// TODO
pub fn remove_output() {
todo!()
/// Remove a Pre-RCT [`Output`] from the database.
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn remove_output(
pre_rct_output_id: &PreRctOutputId,
tables: &mut impl TablesMut,
) -> Result<(), RuntimeError> {
// Decrement the amount index by 1, or delete the entry out-right.
// FIXME: this would be much better expressed with a
// `btree_map::Entry`-like API, fix `trait DatabaseRw`.
tables
.num_outputs_mut()
.update(&pre_rct_output_id.amount, |num_outputs| {
// INVARIANT: Should never be 0.
if num_outputs == 1 {
None
} else {
Some(num_outputs - 1)
}
})?;
// Delete the output data itself.
tables.outputs_mut().delete(pre_rct_output_id)
}
/// TODO
pub fn get_output() {
todo!()
/// Retrieve a Pre-RCT [`Output`] from the database.
#[doc = doc_error!()]
#[inline]
pub fn get_output(
pre_rct_output_id: &PreRctOutputId,
table_outputs: &impl DatabaseRo<Outputs>,
) -> Result<Output, RuntimeError> {
table_outputs.get(pre_rct_output_id)
}
/// TODO
pub fn get_output_list() {
todo!()
/// How many pre-RCT [`Output`]s are there?
///
/// This returns the amount of pre-RCT outputs currently stored.
#[doc = doc_error!()]
#[inline]
pub fn get_num_outputs(table_outputs: &impl DatabaseRo<Outputs>) -> Result<u64, RuntimeError> {
table_outputs.len()
}
/// TODO
pub fn get_rct_num_outputs() {
todo!()
//---------------------------------------------------------------------------------------------------- RCT Outputs
/// Add an [`RctOutput`] to the database.
///
/// Upon [`Ok`], this function returns the [`AmountIndex`] that
/// can be used to lookup the `RctOutput` in [`get_rct_output()`].
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn add_rct_output(
rct_output: &RctOutput,
table_rct_outputs: &mut impl DatabaseRw<RctOutputs>,
) -> Result<AmountIndex, RuntimeError> {
let amount_index = get_rct_num_outputs(table_rct_outputs)?;
table_rct_outputs.put(&amount_index, rct_output)?;
Ok(amount_index)
}
/// TODO
pub fn get_pre_rct_num_outputs() {
todo!()
/// Remove an [`RctOutput`] from the database.
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn remove_rct_output(
amount_index: &AmountIndex,
table_rct_outputs: &mut impl DatabaseRw<RctOutputs>,
) -> Result<(), RuntimeError> {
table_rct_outputs.delete(amount_index)
}
/// Retrieve an [`RctOutput`] from the database.
#[doc = doc_error!()]
#[inline]
pub fn get_rct_output(
amount_index: &AmountIndex,
table_rct_outputs: &impl DatabaseRo<RctOutputs>,
) -> Result<RctOutput, RuntimeError> {
table_rct_outputs.get(amount_index)
}
/// How many [`RctOutput`]s are there?
///
/// This returns the amount of RCT outputs currently stored.
#[doc = doc_error!()]
#[inline]
pub fn get_rct_num_outputs(
table_rct_outputs: &impl DatabaseRo<RctOutputs>,
) -> Result<u64, RuntimeError> {
table_rct_outputs.len()
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening, clippy::cognitive_complexity)]
mod test {
use super::*;
use crate::{
tests::{assert_all_tables_are_empty, tmp_concrete_env},
types::OutputFlags,
Env,
};
use cuprate_test_utils::data::{tx_v1_sig2, tx_v2_rct3};
use pretty_assertions::assert_eq;
/// Dummy `Output`.
const OUTPUT: Output = Output {
key: [44; 32],
height: 0,
output_flags: OutputFlags::NON_ZERO_UNLOCK_TIME,
tx_idx: 0,
};
/// Dummy `RctOutput`.
const RCT_OUTPUT: RctOutput = RctOutput {
key: [88; 32],
height: 1,
output_flags: OutputFlags::empty(),
tx_idx: 1,
commitment: [100; 32],
};
/// Dummy `Amount`
const AMOUNT: Amount = 22;
/// Tests all above output functions when only inputting `Output` data (no Block).
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
fn all_output_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
// Assert length is correct.
assert_eq!(get_num_outputs(tables.outputs()).unwrap(), 0);
assert_eq!(get_rct_num_outputs(tables.rct_outputs()).unwrap(), 0);
// Add outputs.
let pre_rct_output_id = add_output(AMOUNT, &OUTPUT, &mut tables).unwrap();
let amount_index = add_rct_output(&RCT_OUTPUT, tables.rct_outputs_mut()).unwrap();
assert_eq!(
pre_rct_output_id,
PreRctOutputId {
amount: AMOUNT,
amount_index: 0,
}
);
// Assert all reads of the outputs are OK.
{
// Assert proper tables were added to.
assert_eq!(tables.block_infos().len().unwrap(), 0);
assert_eq!(tables.block_blobs().len().unwrap(), 0);
assert_eq!(tables.block_heights().len().unwrap(), 0);
assert_eq!(tables.key_images().len().unwrap(), 0);
assert_eq!(tables.num_outputs().len().unwrap(), 1);
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.outputs().len().unwrap(), 1);
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 1);
assert_eq!(tables.tx_blobs().len().unwrap(), 0);
assert_eq!(tables.tx_ids().len().unwrap(), 0);
assert_eq!(tables.tx_heights().len().unwrap(), 0);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 0);
// Assert length is correct.
assert_eq!(get_num_outputs(tables.outputs()).unwrap(), 1);
assert_eq!(get_rct_num_outputs(tables.rct_outputs()).unwrap(), 1);
assert_eq!(1, tables.num_outputs().get(&AMOUNT).unwrap());
// Assert value is save after retrieval.
assert_eq!(
OUTPUT,
get_output(&pre_rct_output_id, tables.outputs()).unwrap(),
);
assert_eq!(
RCT_OUTPUT,
get_rct_output(&amount_index, tables.rct_outputs()).unwrap(),
);
}
// Remove the outputs.
{
remove_output(&pre_rct_output_id, &mut tables).unwrap();
remove_rct_output(&amount_index, tables.rct_outputs_mut()).unwrap();
// Assert value no longer exists.
assert!(matches!(
get_output(&pre_rct_output_id, tables.outputs()),
Err(RuntimeError::KeyNotFound)
));
assert!(matches!(
get_rct_output(&amount_index, tables.rct_outputs()),
Err(RuntimeError::KeyNotFound)
));
// Assert length is correct.
assert_eq!(get_num_outputs(tables.outputs()).unwrap(), 0);
assert_eq!(get_rct_num_outputs(tables.rct_outputs()).unwrap(), 0);
}
assert_all_tables_are_empty(&env);
}
}

52
database/src/ops/property.rs
View file

@ -1,9 +1,57 @@
//! Properties.
//---------------------------------------------------------------------------------------------------- Import
use monero_pruning::PruningSeed;
use monero_serai::transaction::{Timelock, Transaction};
use cuprate_types::{OutputOnChain, VerifiedBlockInformation};
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::{doc_add_block_inner_invariant, doc_error},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId, RctOutput, TxHash,
TxId,
},
};
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn get_blockchain_pruning_seed() {
todo!()
///
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
///
/// # Example
/// ```rust
/// # use cuprate_database::{*, tables::*, ops::block::*, ops::tx::*};
/// // TODO
/// ```
#[inline]
pub const fn get_blockchain_pruning_seed() -> Result<PruningSeed, RuntimeError> {
// TODO: impl pruning.
// We need a DB properties table.
Ok(PruningSeed::NotPruned)
}
/// TODO
///
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
///
/// # Example
/// ```rust
/// # use cuprate_database::{*, tables::*, ops::block::*, ops::tx::*};
/// // TODO
/// ```
#[inline]
pub const fn db_version() -> Result<u64, RuntimeError> {
// TODO: We need a DB properties table.
Ok(crate::constants::DATABASE_VERSION)
}

19
database/src/ops/spent_key.rs
View file

@ -1,19 +0,0 @@
//! Spent keys.
//---------------------------------------------------------------------------------------------------- Import
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn add_spent_key() {
todo!()
}
/// TODO
pub fn remove_spent_key() {
todo!()
}
/// TODO
pub fn is_spent_key_recorded() {
todo!()
}

466
database/src/ops/tx.rs
View file

@ -1,64 +1,446 @@
//! Transactions.
//---------------------------------------------------------------------------------------------------- Import
use bytemuck::TransparentWrapper;
use curve25519_dalek::{constants::ED25519_BASEPOINT_POINT, Scalar};
use monero_serai::transaction::{Input, Timelock, Transaction};
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn add_transaction() {
todo!()
use cuprate_types::{OutputOnChain, TransactionVerificationData, VerifiedBlockInformation};
use monero_pruning::PruningSeed;
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::{
blockchain::chain_height,
macros::{doc_add_block_inner_invariant, doc_error},
property::get_blockchain_pruning_seed,
},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxBlobs, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
AmountIndices, BlockHash, BlockHeight, BlockInfo, KeyImage, Output, OutputFlags,
PreRctOutputId, RctOutput, TxBlob, TxHash, TxId,
},
StorableVec,
};
use super::{
key_image::{add_key_image, remove_key_image},
output::{add_output, add_rct_output, get_rct_num_outputs, remove_output, remove_rct_output},
};
//---------------------------------------------------------------------------------------------------- Private
/// Add a [`TransactionVerificationData`] to the database.
///
/// The `block_height` is the block that this `tx` belongs to.
///
/// Note that the caller's input is trusted implicitly and no checks
/// are done (in this function) whether the `block_height` is correct or not.
///
#[doc = doc_add_block_inner_invariant!()]
///
/// # Notes
/// This function is different from other sub-functions and slightly more similar to
/// [`add_block()`](crate::ops::block::add_block) in that it calls other sub-functions.
///
/// This function calls:
/// - [`add_output()`]
/// - [`add_rct_output()`]
/// - [`add_key_image()`]
///
/// Thus, after [`add_tx`], those values (outputs and key images)
/// will be added to database tables as well.
///
/// # Panics
/// This function will panic if:
/// - `block.height > u32::MAX` (not normally possible)
#[doc = doc_error!()]
#[inline]
pub fn add_tx(
tx: &TransactionVerificationData,
block_height: &BlockHeight,
tables: &mut impl TablesMut,
) -> Result<TxId, RuntimeError> {
let tx_id = get_num_tx(tables.tx_ids_mut())?;
//------------------------------------------------------ Transaction data
tables.tx_ids_mut().put(&tx.tx_hash, &tx_id)?;
tables.tx_heights_mut().put(&tx_id, block_height)?;
tables
.tx_blobs_mut()
.put(&tx_id, StorableVec::wrap_ref(&tx.tx_blob))?;
//------------------------------------------------------ Timelocks
// Height/time is not differentiated via type, but rather:
// "height is any value less than 500_000_000 and timestamp is any value above"
// so the `u64/usize` is stored without any tag.
//
// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1558504285>
match tx.tx.prefix.timelock {
Timelock::None => (),
Timelock::Block(height) => tables.tx_unlock_time_mut().put(&tx_id, &(height as u64))?,
Timelock::Time(time) => tables.tx_unlock_time_mut().put(&tx_id, &time)?,
}
//------------------------------------------------------ Pruning
// SOMEDAY: implement pruning after `monero-serai` does.
// if let PruningSeed::Pruned(decompressed_pruning_seed) = get_blockchain_pruning_seed()? {
// SOMEDAY: what to store here? which table?
// }
//------------------------------------------------------
// Refer to the inner transaction type from now on.
let tx: &Transaction = &tx.tx;
let Ok(height) = u32::try_from(*block_height) else {
panic!("add_tx(): block_height ({block_height}) > u32::MAX");
};
//------------------------------------------------------ Key Images
// Is this a miner transaction?
// Which table we add the output data to depends on this.
// <https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/blockchain_db/blockchain_db.cpp#L212-L216>
let mut miner_tx = false;
// Key images.
for inputs in &tx.prefix.inputs {
match inputs {
// Key images.
Input::ToKey { key_image, .. } => {
add_key_image(key_image.compress().as_bytes(), tables.key_images_mut())?;
}
// This is a miner transaction, set it for later use.
Input::Gen(_) => miner_tx = true,
}
}
//------------------------------------------------------ Outputs
// Output bit flags.
// Set to a non-zero bit value if the unlock time is non-zero.
let output_flags = match tx.prefix.timelock {
Timelock::None => OutputFlags::empty(),
Timelock::Block(_) | Timelock::Time(_) => OutputFlags::NON_ZERO_UNLOCK_TIME,
};
let mut amount_indices = Vec::with_capacity(tx.prefix.outputs.len());
let tx_idx = get_num_tx(tables.tx_ids_mut())?;
for (i, output) in tx.prefix.outputs.iter().enumerate() {
let key = *output.key.as_bytes();
// Outputs with clear amounts.
let amount_index = if let Some(amount) = output.amount {
// RingCT (v2 transaction) miner outputs.
if miner_tx && tx.prefix.version == 2 {
// Create commitment.
// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1559489302>
// FIXME: implement lookup table for common values:
// <https://github.com/monero-project/monero/blob/c8214782fb2a769c57382a999eaf099691c836e7/src/ringct/rctOps.cpp#L322>
let commitment = (ED25519_BASEPOINT_POINT
+ monero_serai::H() * Scalar::from(amount))
.compress()
.to_bytes();
add_rct_output(
&RctOutput {
key,
height,
output_flags,
tx_idx,
commitment,
},
tables.rct_outputs_mut(),
)?
// Pre-RingCT outputs.
} else {
add_output(
amount,
&Output {
key,
height,
output_flags,
tx_idx,
},
tables,
)?
.amount_index
}
// RingCT outputs.
} else {
let commitment = tx.rct_signatures.base.commitments[i].compress().to_bytes();
add_rct_output(
&RctOutput {
key,
height,
output_flags,
tx_idx,
commitment,
},
tables.rct_outputs_mut(),
)?
};
amount_indices.push(amount_index);
} // for each output
tables
.tx_outputs_mut()
.put(&tx_id, &StorableVec(amount_indices))?;
Ok(tx_id)
}
/// TODO
pub fn add_transaction_data() {
todo!()
/// Remove a transaction from the database with its [`TxHash`].
///
/// This returns the [`TxId`] and [`TxBlob`] of the removed transaction.
///
#[doc = doc_add_block_inner_invariant!()]
///
/// # Notes
/// As mentioned in [`add_tx`], this function will call other sub-functions:
/// - [`remove_output()`]
/// - [`remove_rct_output()`]
/// - [`remove_key_image()`]
///
/// Thus, after [`remove_tx`], those values (outputs and key images)
/// will be remove from database tables as well.
///
#[doc = doc_error!()]
#[inline]
pub fn remove_tx(
tx_hash: &TxHash,
tables: &mut impl TablesMut,
) -> Result<(TxId, Transaction), RuntimeError> {
//------------------------------------------------------ Transaction data
let tx_id = tables.tx_ids_mut().take(tx_hash)?;
let tx_blob = tables.tx_blobs_mut().take(&tx_id)?;
tables.tx_heights_mut().delete(&tx_id)?;
tables.tx_outputs_mut().delete(&tx_id)?;
//------------------------------------------------------ Pruning
// SOMEDAY: implement pruning after `monero-serai` does.
// table_prunable_hashes.delete(&tx_id)?;
// table_prunable_tx_blobs.delete(&tx_id)?;
// if let PruningSeed::Pruned(decompressed_pruning_seed) = get_blockchain_pruning_seed()? {
// SOMEDAY: what to remove here? which table?
// }
//------------------------------------------------------ Unlock Time
match tables.tx_unlock_time_mut().delete(&tx_id) {
Ok(()) | Err(RuntimeError::KeyNotFound) => (),
// An actual error occurred, return.
Err(e) => return Err(e),
}
//------------------------------------------------------
// Refer to the inner transaction type from now on.
let tx = Transaction::read(&mut tx_blob.0.as_slice())?;
//------------------------------------------------------ Key Images
// Is this a miner transaction?
let mut miner_tx = false;
for inputs in &tx.prefix.inputs {
match inputs {
// Key images.
Input::ToKey { key_image, .. } => {
remove_key_image(key_image.compress().as_bytes(), tables.key_images_mut())?;
}
// This is a miner transaction, set it for later use.
Input::Gen(_) => miner_tx = true,
}
} // for each input
//------------------------------------------------------ Outputs
// Remove each output in the transaction.
for (i, output) in tx.prefix.outputs.iter().enumerate() {
// Outputs with clear amounts.
if let Some(amount) = output.amount {
// RingCT miner outputs.
if miner_tx && tx.prefix.version == 2 {
let amount_index = get_rct_num_outputs(tables.rct_outputs())? - 1;
remove_rct_output(&amount_index, tables.rct_outputs_mut())?;
// Pre-RingCT outputs.
} else {
let amount_index = tables.num_outputs_mut().get(&amount)? - 1;
remove_output(
&PreRctOutputId {
amount,
amount_index,
},
tables,
)?;
}
// RingCT outputs.
} else {
let amount_index = get_rct_num_outputs(tables.rct_outputs())? - 1;
remove_rct_output(&amount_index, tables.rct_outputs_mut())?;
}
} // for each output
Ok((tx_id, tx))
}
/// TODO
pub fn remove_transaction() {
todo!()
//---------------------------------------------------------------------------------------------------- `get_tx_*`
/// Retrieve a [`Transaction`] from the database with its [`TxHash`].
#[doc = doc_error!()]
#[inline]
pub fn get_tx(
tx_hash: &TxHash,
table_tx_ids: &impl DatabaseRo<TxIds>,
table_tx_blobs: &impl DatabaseRo<TxBlobs>,
) -> Result<Transaction, RuntimeError> {
get_tx_from_id(&table_tx_ids.get(tx_hash)?, table_tx_blobs)
}
/// TODO
pub fn remove_transaction_data() {
todo!()
/// Retrieve a [`Transaction`] from the database with its [`TxId`].
#[doc = doc_error!()]
#[inline]
pub fn get_tx_from_id(
tx_id: &TxId,
table_tx_blobs: &impl DatabaseRo<TxBlobs>,
) -> Result<Transaction, RuntimeError> {
let tx_blob = table_tx_blobs.get(tx_id)?.0;
Ok(Transaction::read(&mut tx_blob.as_slice())?)
}
/// TODO
pub fn remove_tx_outputs() {
todo!()
//----------------------------------------------------------------------------------------------------
/// How many [`Transaction`]s are there?
///
/// This returns the amount of transactions currently stored.
///
/// For example:
/// - 0 transactions exist => returns 0
/// - 1 transactions exist => returns 1
/// - 5 transactions exist => returns 5
/// - etc
#[doc = doc_error!()]
#[inline]
pub fn get_num_tx(table_tx_ids: &impl DatabaseRo<TxIds>) -> Result<u64, RuntimeError> {
table_tx_ids.len()
}
/// TODO
pub fn get_num_tx() {
todo!()
//----------------------------------------------------------------------------------------------------
/// Check if a transaction exists in the database.
///
/// Returns `true` if it does, else `false`.
#[doc = doc_error!()]
#[inline]
pub fn tx_exists(
tx_hash: &TxHash,
table_tx_ids: &impl DatabaseRo<TxIds>,
) -> Result<bool, RuntimeError> {
table_tx_ids.contains(tx_hash)
}
/// TODO
pub fn tx_exists() {
todo!()
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening)]
mod test {
use super::*;
use crate::{
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
use cuprate_test_utils::data::{tx_v1_sig0, tx_v1_sig2, tx_v2_rct3};
use pretty_assertions::assert_eq;
/// TODO
pub fn get_tx_unlock_time() {
todo!()
}
/// Tests all above tx functions when only inputting `Transaction` data (no Block).
#[test]
fn all_tx_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
/// TODO
pub fn get_tx() {
todo!()
}
// Monero `Transaction`, not database tx.
let txs = [tx_v1_sig0(), tx_v1_sig2(), tx_v2_rct3()];
/// TODO
pub fn get_tx_list() {
todo!()
}
// Add transactions.
let tx_ids = {
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
/// TODO
pub fn get_pruned_tx() {
todo!()
}
let tx_ids = txs
.iter()
.map(|tx| {
println!("add_tx(): {tx:#?}");
add_tx(tx, &0, &mut tables).unwrap()
})
.collect::<Vec<TxId>>();
/// TODO
pub fn get_tx_block_height() {
todo!()
drop(tables);
TxRw::commit(tx_rw).unwrap();
tx_ids
};
// Assert all reads of the transactions are OK.
let tx_hashes = {
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
// Assert only the proper tables were added to.
assert_eq!(tables.block_infos().len().unwrap(), 0);
assert_eq!(tables.block_blobs().len().unwrap(), 0);
assert_eq!(tables.block_heights().len().unwrap(), 0);
assert_eq!(tables.key_images().len().unwrap(), 4); // added to key images
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.num_outputs().len().unwrap(), 9);
assert_eq!(tables.outputs().len().unwrap(), 10); // added to outputs
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 2);
assert_eq!(tables.tx_blobs().len().unwrap(), 3);
assert_eq!(tables.tx_ids().len().unwrap(), 3);
assert_eq!(tables.tx_heights().len().unwrap(), 3);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 1); // only 1 has a timelock
// Both from ID and hash should result in getting the same transaction.
let mut tx_hashes = vec![];
for (i, tx_id) in tx_ids.iter().enumerate() {
println!("tx_ids.iter(): i: {i}, tx_id: {tx_id}");
let tx_get_from_id = get_tx_from_id(tx_id, tables.tx_blobs()).unwrap();
let tx_hash = tx_get_from_id.hash();
let tx_get = get_tx(&tx_hash, tables.tx_ids(), tables.tx_blobs()).unwrap();
println!("tx_ids.iter(): tx_get_from_id: {tx_get_from_id:#?}, tx_get: {tx_get:#?}");
assert_eq!(tx_get_from_id.hash(), tx_get.hash());
assert_eq!(tx_get_from_id.hash(), txs[i].tx_hash);
assert_eq!(tx_get_from_id, tx_get);
assert_eq!(tx_get, txs[i].tx);
assert!(tx_exists(&tx_hash, tables.tx_ids()).unwrap());
tx_hashes.push(tx_hash);
}
tx_hashes
};
// Remove the transactions.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for tx_hash in tx_hashes {
println!("remove_tx(): tx_hash: {tx_hash:?}");
let (tx_id, _) = remove_tx(&tx_hash, &mut tables).unwrap();
assert!(matches!(
get_tx_from_id(&tx_id, tables.tx_blobs()),
Err(RuntimeError::KeyNotFound)
));
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
assert_all_tables_are_empty(&env);
}
}

302
database/src/tables.rs
View file

@ -4,19 +4,17 @@
//---------------------------------------------------------------------------------------------------- Import
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
table::Table,
types::{
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfoV1,
BlockInfoV2, BlockInfoV3, KeyImage, Output, PreRctOutputId, PrunableBlob, PrunableHash,
PrunedBlob, RctOutput, TxHash, TxId, UnlockTime,
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfo, KeyImage,
Output, PreRctOutputId, PrunableBlob, PrunableHash, PrunedBlob, RctOutput, TxBlob, TxHash,
TxId, UnlockTime,
},
};
//---------------------------------------------------------------------------------------------------- Tables
//---------------------------------------------------------------------------------------------------- Sealed
/// Private module, should not be accessible outside this crate.
///
/// Used to block outsiders implementing [`Table`].
/// All [`Table`] types must also implement [`Sealed`].
pub(super) mod private {
/// Private sealed trait.
///
@ -24,6 +22,255 @@ pub(super) mod private {
pub trait Sealed {}
}
//---------------------------------------------------------------------------------------------------- `trait Tables[Mut]`
/// Creates:
/// - `pub trait Tables`
/// - `pub trait TablesMut`
/// - Blanket implementation for `(tuples, containing, all, open, database, tables, ...)`
///
/// For why this exists, see: <https://github.com/Cuprate/cuprate/pull/102#pullrequestreview-1978348871>.
macro_rules! define_trait_tables {
($(
// The `T: Table` type The index in a tuple
// | containing all tables
// v v
$table:ident => $index:literal
),* $(,)?) => { paste::paste! {
/// Object containing all opened [`Table`]s in read-only mode.
///
/// This is an encapsulated object that contains all
/// available [`Table`]'s in read-only mode.
///
/// It is a `Sealed` trait and is only implemented on a
/// `(tuple, containing, all, table, types, ...)`.
///
/// This is used to return a _single_ object from functions like
/// [`EnvInner::open_tables`](crate::EnvInner::open_tables) rather
/// than the tuple containing the tables itself.
///
/// To replace `tuple.0` style indexing, `field_accessor_functions()`
/// are provided on this trait, which essentially map the object to
/// fields containing the particular database table, for example:
/// ```rust,ignore
/// let tables = open_tables();
///
/// // The accessor function `block_info_v1s()` returns the field
/// // containing an open database table for `BlockInfoV1s`.
/// let _ = tables.block_info_v1s();
/// ```
pub trait Tables: private::Sealed {
// This expands to creating `fn field_accessor_functions()`
// for each passed `$table` type.
//
// It is essentially a mapping to the field
// containing the proper opened database table.
//
// The function name of the function is
// the table type in `snake_case`, e.g., `block_info_v1s()`.
$(
/// Access an opened
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake>](&self) -> &impl DatabaseRo<$table>;
)*
/// This returns `true` if all tables are empty.
///
/// # Errors
/// This returns errors on regular database errors.
fn all_tables_empty(&self) -> Result<bool, $crate::error::RuntimeError>;
}
/// Object containing all opened [`Table`]s in read + iter mode.
///
/// This is the same as [`Tables`] but includes `_iter()` variants.
///
/// See [`Tables`] for documentation - this trait exists for the same reasons.
pub trait TablesIter: private::Sealed + Tables {
$(
/// Access an opened read-only + iterable
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake _iter>](&self) -> &(impl DatabaseRo<$table> + DatabaseIter<$table>);
)*
}
/// Object containing all opened [`Table`]s in write mode.
///
/// This is the same as [`Tables`] but for mutable accesses.
///
/// See [`Tables`] for documentation - this trait exists for the same reasons.
pub trait TablesMut: private::Sealed + Tables {
$(
/// Access an opened
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake _mut>](&mut self) -> &mut impl DatabaseRw<$table>;
)*
}
// Implement `Sealed` for all table types.
impl<$([<$table:upper>]),*> private::Sealed for ($([<$table:upper>]),*) {}
// This creates a blanket-implementation for
// `(tuple, containing, all, table, types)`.
//
// There is a generic defined here _for each_ `$table` input.
// Specifically, the generic letters are just the table types in UPPERCASE.
// Concretely, this expands to something like:
// ```rust
// impl<BLOCKINFOSV1S, BLOCKINFOSV2S, BLOCKINFOSV3S, [...]>
// ```
impl<$([<$table:upper>]),*> Tables
// We are implementing `Tables` on a tuple that
// contains all those generics specified, i.e.,
// a tuple containing all open table types.
//
// Concretely, this expands to something like:
// ```rust
// (BLOCKINFOSV1S, BLOCKINFOSV2S, BLOCKINFOSV3S, [...])
// ```
// which is just a tuple of the generics defined above.
for ($([<$table:upper>]),*)
where
// This expands to a where bound that asserts each element
// in the tuple implements some database table type.
//
// Concretely, this expands to something like:
// ```rust
// BLOCKINFOSV1S: DatabaseRo<BlockInfoV1s> + DatabaseIter<BlockInfoV1s>,
// BLOCKINFOSV2S: DatabaseRo<BlockInfoV2s> + DatabaseIter<BlockInfoV2s>,
// [...]
// ```
$(
[<$table:upper>]: DatabaseRo<$table>,
)*
{
$(
// The function name of the accessor function is
// the table type in `snake_case`, e.g., `block_info_v1s()`.
#[inline]
fn [<$table:snake>](&self) -> &impl DatabaseRo<$table> {
// The index of the database table in
// the tuple implements the table trait.
&self.$index
}
)*
fn all_tables_empty(&self) -> Result<bool, $crate::error::RuntimeError> {
$(
if !DatabaseRo::is_empty(&self.$index)? {
return Ok(false);
}
)*
Ok(true)
}
}
// This is the same as the above
// `Tables`, but for `TablesIter`.
impl<$([<$table:upper>]),*> TablesIter
for ($([<$table:upper>]),*)
where
$(
[<$table:upper>]: DatabaseRo<$table> + DatabaseIter<$table>,
)*
{
$(
// The function name of the accessor function is
// the table type in `snake_case` + `_iter`, e.g., `block_info_v1s_iter()`.
#[inline]
fn [<$table:snake _iter>](&self) -> &(impl DatabaseRo<$table> + DatabaseIter<$table>) {
&self.$index
}
)*
}
// This is the same as the above
// `Tables`, but for `TablesMut`.
impl<$([<$table:upper>]),*> TablesMut
for ($([<$table:upper>]),*)
where
$(
[<$table:upper>]: DatabaseRw<$table>,
)*
{
$(
// The function name of the mutable accessor function is
// the table type in `snake_case` + `_mut`, e.g., `block_info_v1s_mut()`.
#[inline]
fn [<$table:snake _mut>](&mut self) -> &mut impl DatabaseRw<$table> {
&mut self.$index
}
)*
}
}};
}
// Format: $table_type => $index
//
// The $index:
// - Simply increments by 1 for each table
// - Must be 0..
// - Must end at the total amount of table types
//
// Compile errors will occur if these aren't satisfied.
define_trait_tables! {
BlockInfos => 0,
BlockBlobs => 1,
BlockHeights => 2,
KeyImages => 3,
NumOutputs => 4,
PrunedTxBlobs => 5,
PrunableHashes => 6,
Outputs => 7,
PrunableTxBlobs => 8,
RctOutputs => 9,
TxBlobs => 10,
TxIds => 11,
TxHeights => 12,
TxOutputs => 13,
TxUnlockTime => 14,
}
//---------------------------------------------------------------------------------------------------- Table function macro
/// `crate`-private macro for callings functions on all tables.
///
/// This calls the function `$fn` with the optional
/// arguments `$args` on all tables - returning early
/// (within whatever scope this is called) if any
/// of the function calls error.
///
/// Else, it evaluates to an `Ok((tuple, of, all, table, types, ...))`,
/// i.e., an `impl Table[Mut]` wrapped in `Ok`.
macro_rules! call_fn_on_all_tables_or_early_return {
(
$($fn:ident $(::)?)*
(
$($arg:ident),* $(,)?
)
) => {{
Ok((
$($fn ::)*<$crate::tables::BlockInfos>($($arg),*)?,
$($fn ::)*<$crate::tables::BlockBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::BlockHeights>($($arg),*)?,
$($fn ::)*<$crate::tables::KeyImages>($($arg),*)?,
$($fn ::)*<$crate::tables::NumOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunedTxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunableHashes>($($arg),*)?,
$($fn ::)*<$crate::tables::Outputs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunableTxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::RctOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxIds>($($arg),*)?,
$($fn ::)*<$crate::tables::TxHeights>($($arg),*)?,
$($fn ::)*<$crate::tables::TxOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxUnlockTime>($($arg),*)?,
))
}};
}
pub(crate) use call_fn_on_all_tables_or_early_return;
//---------------------------------------------------------------------------------------------------- Table macro
/// Create all tables, should be used _once_.
///
@ -80,8 +327,10 @@ macro_rules! tables {
// Notes:
// - Keep this sorted A-Z (by table name)
// - Tables are defined in plural to avoid name conflicts with types
// - If adding/changing a table, also edit the tests in `src/backend/tests.rs`
// and edit `Env::open` to make sure it creates the table
// - If adding/changing a table also edit:
// a) the tests in `src/backend/tests.rs`
// b) `Env::open` to make sure it creates the table (for all backends)
// c) `call_fn_on_all_tables_or_early_return!()` macro defined in this file
tables! {
/// TODO
BlockBlobs,
@ -92,22 +341,17 @@ tables! {
BlockHash => BlockHeight,
/// TODO
BlockInfoV1s,
BlockHeight => BlockInfoV1,
/// TODO
BlockInfoV2s,
BlockHeight => BlockInfoV2,
/// TODO
BlockInfoV3s,
BlockHeight => BlockInfoV3,
BlockInfos,
BlockHeight => BlockInfo,
/// TODO
KeyImages,
KeyImage => (),
/// TODO
/// Maps an output's amount to the number of outputs with that amount.
///
/// For a new output the `AmountIndex` value from this
/// table will be its index in a list of duplicate outputs.
NumOutputs,
Amount => AmountIndex,
@ -119,18 +363,28 @@ tables! {
Outputs,
PreRctOutputId => Output,
/// TODO
// SOMEDAY: impl when `monero-serai` supports pruning
PrunableTxBlobs,
TxId => PrunableBlob,
/// TODO
// SOMEDAY: impl when `monero-serai` supports pruning
PrunableHashes,
TxId => PrunableHash,
// SOMEDAY: impl a properties table:
// - db version
// - pruning seed
// Properties,
// StorableString => StorableVec,
/// TODO
RctOutputs,
AmountIndex => RctOutput,
/// SOMEDAY: remove when `monero-serai` supports pruning
TxBlobs,
TxId => TxBlob,
/// TODO
TxIds,
TxHash => TxId,
@ -139,6 +393,10 @@ tables! {
TxHeights,
TxId => BlockHeight,
/// TODO
TxOutputs,
TxId => AmountIndices,
/// TODO
TxUnlockTime,
TxId => UnlockTime,

45
database/src/tests.rs Normal file
View file

@ -0,0 +1,45 @@
//! Utilities for `cuprate_database` testing.
//!
//! These types/fn's are only:
//! - enabled on #[cfg(test)]
//! - only used internally
#![allow(clippy::significant_drop_tightening)]
//---------------------------------------------------------------------------------------------------- Import
use std::{
fmt::Debug,
sync::{Arc, OnceLock},
};
use monero_serai::{
ringct::{RctPrunable, RctSignatures},
transaction::{Timelock, Transaction, TransactionPrefix},
};
use crate::{
config::Config, key::Key, storable::Storable, tables::Tables, transaction::TxRo, ConcreteEnv,
Env, EnvInner,
};
//---------------------------------------------------------------------------------------------------- fn
/// Create an `Env` in a temporarily directory.
/// The directory is automatically removed after the `TempDir` is dropped.
///
/// FIXME: changing this to `-> impl Env` causes lifetime errors...
pub(crate) fn tmp_concrete_env() -> (ConcreteEnv, tempfile::TempDir) {
let tempdir = tempfile::tempdir().unwrap();
let config = Config::low_power(Some(tempdir.path().into()));
let env = ConcreteEnv::open(config).unwrap();
(env, tempdir)
}
/// Assert all the tables in the environment are empty.
pub(crate) fn assert_all_tables_are_empty(env: &ConcreteEnv) {
let env_inner = env.env_inner();
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
assert!(tables.all_tables_empty().unwrap());
assert_eq!(crate::ops::tx::get_num_tx(tables.tx_ids()).unwrap(), 0);
}

153
database/src/types.rs
View file

@ -82,6 +82,9 @@ pub type PrunableBlob = StorableVec<u8>;
/// TODO
pub type PrunableHash = [u8; 32];
/// TODO
pub type TxBlob = StorableVec<u8>;
/// TODO
pub type TxId = u64;
@ -124,96 +127,6 @@ pub struct PreRctOutputId {
pub amount_index: AmountIndex,
}
//---------------------------------------------------------------------------------------------------- BlockInfoV1
/// TODO
///
/// ```rust
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = BlockInfoV1 {
/// timestamp: 1,
/// total_generated_coins: 123,
/// weight: 321,
/// cumulative_difficulty: 111,
/// block_hash: [54; 32],
/// };
/// let b = Storable::as_bytes(&a);
/// let c: BlockInfoV1 = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
/// # Size & Alignment
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<BlockInfoV1>(), 64);
/// assert_eq!(align_of::<BlockInfoV1>(), 8);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(C)]
pub struct BlockInfoV1 {
/// TODO
pub timestamp: u64,
/// TODO
pub total_generated_coins: u64,
/// TODO
pub weight: u64,
/// TODO
pub cumulative_difficulty: u64,
/// TODO
pub block_hash: [u8; 32],
}
//---------------------------------------------------------------------------------------------------- BlockInfoV2
/// TODO
///
/// ```rust
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = BlockInfoV2 {
/// timestamp: 1,
/// total_generated_coins: 123,
/// weight: 321,
/// block_hash: [54; 32],
/// cumulative_difficulty: 111,
/// cumulative_rct_outs: 2389,
/// };
/// let b = Storable::as_bytes(&a);
/// let c: BlockInfoV2 = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
/// # Size & Alignment
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<BlockInfoV2>(), 72);
/// assert_eq!(align_of::<BlockInfoV2>(), 8);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(C)]
pub struct BlockInfoV2 {
/// TODO
pub timestamp: u64,
/// TODO
pub total_generated_coins: u64,
/// TODO
pub weight: u64,
/// TODO
pub block_hash: [u8; 32],
/// TODO
pub cumulative_difficulty: u64,
/// TODO
///
/// TODO: note that this is originally u32,
/// but is u64 here for padding reasons.
pub cumulative_rct_outs: u64,
}
//---------------------------------------------------------------------------------------------------- BlockInfoV3
/// TODO
///
@ -221,18 +134,17 @@ pub struct BlockInfoV2 {
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = BlockInfoV3 {
/// let a = BlockInfo {
/// timestamp: 1,
/// total_generated_coins: 123,
/// cumulative_generated_coins: 123,
/// weight: 321,
/// cumulative_difficulty_low: 111,
/// cumulative_difficulty_high: 112,
/// cumulative_difficulty: 112,
/// block_hash: [54; 32],
/// cumulative_rct_outs: 2389,
/// long_term_weight: 2389,
/// };
/// let b = Storable::as_bytes(&a);
/// let c: BlockInfoV3 = Storable::from_bytes(b);
/// let c: BlockInfo = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
@ -240,24 +152,21 @@ pub struct BlockInfoV2 {
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<BlockInfoV3>(), 88);
/// assert_eq!(align_of::<BlockInfoV3>(), 8);
/// assert_eq!(size_of::<BlockInfo>(), 88);
/// assert_eq!(align_of::<BlockInfo>(), 8);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(C)]
pub struct BlockInfoV3 {
pub struct BlockInfo {
/// TODO
pub timestamp: u64,
/// TODO
pub total_generated_coins: u64,
pub cumulative_generated_coins: u64,
/// TODO
pub weight: u64,
// Maintain 8 byte alignment.
/// TODO
pub cumulative_difficulty_low: u64,
/// TODO
pub cumulative_difficulty_high: u64,
pub cumulative_difficulty: u128,
/// TODO
pub block_hash: [u8; 32],
/// TODO
@ -266,6 +175,36 @@ pub struct BlockInfoV3 {
pub long_term_weight: u64,
}
//---------------------------------------------------------------------------------------------------- OutputFlags
bitflags::bitflags! {
/// TODO
///
/// ```rust
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = OutputFlags::NON_ZERO_UNLOCK_TIME;
/// let b = Storable::as_bytes(&a);
/// let c: OutputFlags = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
/// # Size & Alignment
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<OutputFlags>(), 4);
/// assert_eq!(align_of::<OutputFlags>(), 4);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(transparent)]
pub struct OutputFlags: u32 {
/// This output has a non-zero unlock time.
const NON_ZERO_UNLOCK_TIME = 0b0000_0001;
}
}
//---------------------------------------------------------------------------------------------------- Output
/// TODO
///
@ -276,7 +215,7 @@ pub struct BlockInfoV3 {
/// let a = Output {
/// key: [1; 32],
/// height: 1,
/// output_flags: 0,
/// output_flags: OutputFlags::empty(),
/// tx_idx: 3,
/// };
/// let b = Storable::as_bytes(&a);
@ -300,7 +239,7 @@ pub struct Output {
/// We could get this from the tx_idx with the Tx Heights table but that would require another look up per out.
pub height: u32,
/// Bit flags for this output, currently only the first bit is used and, if set, it means this output has a non-zero unlock time.
pub output_flags: u32,
pub output_flags: OutputFlags,
/// TODO
pub tx_idx: u64,
}
@ -315,7 +254,7 @@ pub struct Output {
/// let a = RctOutput {
/// key: [1; 32],
/// height: 1,
/// output_flags: 0,
/// output_flags: OutputFlags::empty(),
/// tx_idx: 3,
/// commitment: [3; 32],
/// };
@ -340,7 +279,7 @@ pub struct RctOutput {
/// We could get this from the tx_idx with the Tx Heights table but that would require another look up per out.
pub height: u32,
/// Bit flags for this output, currently only the first bit is used and, if set, it means this output has a non-zero unlock time.
pub output_flags: u32,
pub output_flags: OutputFlags,
/// TODO
pub tx_idx: u64,
/// The amount commitment of this output.

24
test-utils/src/data/free.rs
View file

@ -27,8 +27,8 @@ use crate::data::constants::{
/// this struct represents that data that must be provided.
///
/// Consider using `cuprate_test_utils::rpc` to get this data easily.
struct VerifiedBlockMap<'a> {
block: Block,
struct VerifiedBlockMap {
block_blob: &'static [u8],
pow_hash: [u8; 32],
height: u64,
generated_coins: u64,
@ -37,10 +37,10 @@ struct VerifiedBlockMap<'a> {
cumulative_difficulty: u128,
// Vec of `tx_blob`'s, i.e. the data in `/test-utils/src/data/tx/`.
// This should the actual `tx_blob`'s of the transactions within this block.
txs: Vec<&'a [u8]>,
txs: &'static [&'static [u8]],
}
impl VerifiedBlockMap<'_> {
impl VerifiedBlockMap {
/// Turn the various static data bits in `self` into a `VerifiedBlockInformation`.
///
/// Transactions are verified that they at least match the block's,
@ -48,7 +48,7 @@ impl VerifiedBlockMap<'_> {
/// is not checked.
fn into_verified(self) -> VerifiedBlockInformation {
let Self {
block,
block_blob,
pow_hash,
height,
generated_coins,
@ -58,8 +58,11 @@ impl VerifiedBlockMap<'_> {
txs,
} = self;
let block_blob = block_blob.to_vec();
let block = Block::read(&mut block_blob.as_slice()).unwrap();
let txs: Vec<Arc<TransactionVerificationData>> = txs
.into_iter()
.iter()
.map(to_tx_verification_data)
.map(Arc::new)
.collect();
@ -79,6 +82,7 @@ impl VerifiedBlockMap<'_> {
VerifiedBlockInformation {
block_hash: block.hash(),
block_blob,
block,
txs,
pow_hash,
@ -92,8 +96,8 @@ impl VerifiedBlockMap<'_> {
}
// Same as [`VerifiedBlockMap`] but for [`TransactionVerificationData`].
fn to_tx_verification_data(tx_blob: &[u8]) -> TransactionVerificationData {
let tx_blob = tx_blob.to_vec();
fn to_tx_verification_data(tx_blob: impl AsRef<[u8]>) -> TransactionVerificationData {
let tx_blob = tx_blob.as_ref().to_vec();
let tx = Transaction::read(&mut tx_blob.as_slice()).unwrap();
TransactionVerificationData {
tx_weight: tx.weight(),
@ -158,14 +162,14 @@ macro_rules! verified_block_information_fn {
static BLOCK: OnceLock<VerifiedBlockInformation> = OnceLock::new();
BLOCK.get_or_init(|| {
VerifiedBlockMap {
block: Block::read(&mut $block_blob).unwrap(),
block_blob: $block_blob,
pow_hash: hex!($pow_hash),
height: $height,
generated_coins: $generated_coins,
weight: $weight,
long_term_weight: $long_term_weight,
cumulative_difficulty: $cumulative_difficulty,
txs: vec![$($tx_blob),*],
txs: &[$($tx_blob),*],
}
.into_verified()
})

8
test-utils/src/rpc/client.rs
View file

@ -102,9 +102,10 @@ impl HttpRpcClient {
let reward = result.block_header.reward;
let (block_hash, block) = spawn_blocking(|| {
let block = Block::read(&mut hex::decode(result.blob).unwrap().as_slice()).unwrap();
(block.hash(), block)
let (block_hash, block_blob, block) = spawn_blocking(|| {
let block_blob = hex::decode(result.blob).unwrap();
let block = Block::read(&mut block_blob.as_slice()).unwrap();
(block.hash(), block_blob, block)
})
.await
.unwrap();
@ -139,6 +140,7 @@ impl HttpRpcClient {
VerifiedBlockInformation {
block,
block_blob,
txs,
block_hash,
pow_hash,

4
types/src/types.rs
View file

@ -78,6 +78,10 @@ pub struct VerifiedBlockInformation {
pub long_term_weight: usize,
/// TODO
pub cumulative_difficulty: u128,
/// TODO
/// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1556694072>
/// <https://github.com/serai-dex/serai/blob/93be7a30674ecedfb325b6d09dc22d550d7c13f8/coins/monero/src/block.rs#L110>
pub block_blob: Vec<u8>,
}
//---------------------------------------------------------------------------------------------------- OutputOnChain