database: implement ops/ (#102)

* ops: add `trait MoneroR{o,w}`

* update `trait MoneroR{o,w}` bounds

* types: add `BlockInfoLatest` type alias

* block: impl most core functions

* types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916

* fix table type test

* cargo.toml: add `{cuprate-types, monero-serai}`

* add_block: add all other block data

* ops: remove unneeded `block` functions

* env: add `EnvInner::open_db_rw_all()`

* types: fix test

* block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()`

* add `trait Tables[Mut]` and use it in `EnvInner`

* block: use `TablesMut`

* tables: replace manual impl with `define_trait_tables!()`

* tables: docs for `trait Tables[Mut]`

* tables: doc functions

* create `call_fn_on_all_tables_or_early_return!()` macro

* block: cleanup signatures + bodies

* block: more fn's, docs

* block: add `doc_{error,single,bulk}!()`

* remove `ops/monero.rs`

* move `height()` to `ops/blockchain.rs`

* add `ops/macros.rs`

* tx: add fn signatures

* output: fix fn signatures

* ops: expose `_inner()` functions

* block: add `add_block_header{_bulk, _inner}()`

* ops: remove doc_{fn,inner}!()`

* ops: remove `_{inner,bulk}()`, lifetime + generics

* update lib/mod docs

* ops: add and use `doc_add_block_inner_invariant!()`

* ops: add docs/return to inner `add_block()` functions

* add_block(): extract and use fn for {key_image, output}

* ops: more fn body impl + `add_block()`

* cargo: add `monero-pruning`

* ops: extract out `tx` functions from `add_block()`

* property: add `db_version()`

* ops: `pop_block()` body, remove other `pop_block` fn's

* types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation`

* block: put `block_blob`, pass `Tables` to sub-functions

`impl TablesMut` can't mutably pass multiple tables since
it takes `&mut self`, so all functions unfortunately have to
take a full `&mut impl TablesMut` even though they only need
a few tables.

* database: add `DatabaseRw::take()`

useful for `pop_block()` where we need the value afterwards

* block: deserialize tx's from `block_blobs` in `pop_block()`

* blockchain: `height()` -> `chain_height()`

* output: fix `amount_index`

* ops: fix unlock_time, chain_height

* `BlockInfoV{1,2,3}` -> `BlockInfo`

* constants: add `DATABASE_VERSION`

* database: add `DatabaseRw::update()`

* output: use `DatabaseRw::update()` in `remove_output()`

* add `TxBlobs` table, ignore pruning tables

* block: mostly impl `add_block()` body

* ops: comments

* add_block: miner v2 tx commitment, height cast

* block: impl `pop_block()`

* block: mostly impl `get_block()`

* block: impl `get_block_{from_height,header,header_from_height}`

* add `OutputFlags` bitflags

* add_block: u32::try_into(height: u32), use `OutputFlags`

* tx: impl `get_{tx,tx_from_id}()`

* tx: move docs tests to `#[test]`

testing everything in 1 go is more natural since e.g:
`add_tx()` is followed by `get_tx()`

* tables: add `trait TablesIter`, `all_tables_empty()`

This allows `TablesMut` to be a superset of `Tables`
and use all its accessor functions.

* use cuprate-test-utils, fix tx tests

* block: `add_block()` take block by ref

* tx: use all txs in tests

* output: add `all_tx_functions()` test

* add_block: check current height against input

* block: map more fields in `get_block()`

* block: remove `get_block()`, doc tests, fix `get_block_header()`

* block: dummy values in test

* heed: use `last/first()` instead of `unsafe` cursors

We no longer have DUP semantics and also hard to debug errors
were popping up on `del_current()`...

* heed: fix `DatabaseRw::delete`

Ok(true) means the key did not exist, so we must return
Err(RuntimeError::KeyNotFound)

* block: `add_block()` (dummy value) test

* ops: `key_image` tests

* cleanup, docs, tests

* backend: test `take()` & `update()`

* docs

* remove `OutputFlags::NONE`

* add_block(): add asserts, panic docs, `should_panic` tests

* backend: remove `Ok(())` in `Database::delete` if already deleted

redb already does this, so heed so match

* block: move block operations after tx/outputs

* `amount == 0` -> `amount == 1`

* Nit: StorableVec::wrap_ref

* `saturating_sub(1)` -> `- 1`

* add `TxOutputs` table

* add_block(): add to `tx_outputs` table

* fix `DatabaseRo::update`

* add_tx(): take `block_height` as input

* tx: add/remove from `TxOutputs` table

* output: remove if `amount == 1` -> `amount_index == 0`

* output: fix `add_output()`'s `amount_index` calculation

* output: fix `add_output()`'s `amount_index` calculation again

* output: tests for `amount_index/num_outputs`

* block: `num_outputs - 1` and `take()` -> `get()`

We don't need to `take()` since the call afterwards to
`remove_output()` removes the entry

* block: swap `get_block_extended_header[_from_height]()`

* move `{key_image,output}` handling `add_block()` -> `add_tx()`

* blockchain: add doc to `top_block_height()`

* block: manual panic -> `assert_eq!()`

* test-utils: add `block_blob` to `VerifiedBlockInformation`

field introduced in this PR

* ops: use real block/tx data in tests

* block: `total_generated_coins` -> `cumulative_generated_coins`

* fix clippy, docs, TODOs

* `cumulative_generated_coins()`: `block/` -> `blockchain/`

* blockchain: add `cumulative_generated_coins()` tests

* Update database/src/ops/block.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* `cumulative_generated_coins()` docs for pre-block-0 special case

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
This commit is contained in:
hinto-janai 2024-04-22 21:53:29 -04:00 committed by GitHub
parent ee22e81c7e
commit c65eb0a3ca
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
26 changed files with 2480 additions and 637 deletions

532
Cargo.lock generated

File diff suppressed because it is too large Load diff

View file

@ -9,15 +9,16 @@ repository = "https://github.com/Cuprate/cuprate/tree/main/database"
keywords = ["cuprate", "database"] keywords = ["cuprate", "database"]
[features] [features]
# default = ["heed", "redb", "service"] default = ["heed", "redb", "service"]
# default = ["redb", "service"] # default = ["redb", "service"]
default = ["redb-memory", "service"] # default = ["redb-memory", "service"]
heed = ["dep:heed"] heed = ["dep:heed"]
redb = ["dep:redb"] redb = ["dep:redb"]
redb-memory = ["redb"] redb-memory = ["redb"]
service = ["dep:crossbeam", "dep:futures", "dep:tokio", "dep:tokio-util", "dep:tower", "dep:rayon"] service = ["dep:crossbeam", "dep:futures", "dep:tokio", "dep:tokio-util", "dep:tower", "dep:rayon"]
[dependencies] [dependencies]
bitflags = { workspace = true, features = ["serde", "bytemuck"] }
bytemuck = { version = "1.14.3", features = ["must_cast", "derive", "min_const_generics", "extern_crate_alloc"] } bytemuck = { version = "1.14.3", features = ["must_cast", "derive", "min_const_generics", "extern_crate_alloc"] }
bytes = { workspace = true } bytes = { workspace = true }
cfg-if = { workspace = true } cfg-if = { workspace = true }
@ -26,6 +27,9 @@ cfg-if = { workspace = true }
# Figure out how to enable features of an already pulled in dependency conditionally. # Figure out how to enable features of an already pulled in dependency conditionally.
cuprate-helper = { path = "../helper", features = ["fs", "thread"] } cuprate-helper = { path = "../helper", features = ["fs", "thread"] }
cuprate-types = { path = "../types", features = ["service"] } cuprate-types = { path = "../types", features = ["service"] }
curve25519-dalek = { workspace = true }
monero-pruning = { path = "../pruning" }
monero-serai = { workspace = true, features = ["std"] }
paste = { workspace = true } paste = { workspace = true }
page_size = { version = "0.6.0" } # Needed for database resizes, they must be a multiple of the OS page size. page_size = { version = "0.6.0" } # Needed for database resizes, they must be a multiple of the OS page size.
thiserror = { workspace = true } thiserror = { workspace = true }
@ -46,5 +50,9 @@ serde = { workspace = true, optional = true }
[dev-dependencies] [dev-dependencies]
bytemuck = { version = "1.14.3", features = ["must_cast", "derive", "min_const_generics", "extern_crate_alloc"] } bytemuck = { version = "1.14.3", features = ["must_cast", "derive", "min_const_generics", "extern_crate_alloc"] }
cuprate-helper = { path = "../helper", features = ["thread"] } cuprate-helper = { path = "../helper", features = ["thread"] }
cuprate-test-utils = { path = "../test-utils" }
page_size = { version = "0.6.0" } page_size = { version = "0.6.0" }
tempfile = { version = "3.10.0" } tempfile = { version = "3.10.0" }
pretty_assertions = { version = "1.4.0" }
hex = { workspace = true }
hex-literal = { workspace = true }

View file

@ -204,55 +204,56 @@ impl<T: Table> DatabaseRw<T> for HeedTableRw<'_, '_, T> {
Ok(()) Ok(())
} }
#[inline]
fn take(&mut self, key: &T::Key) -> Result<T::Value, RuntimeError> {
// LMDB/heed does not return the value on deletion.
// So, fetch it first - then delete.
let value = get::<T>(&self.db, &self.tx_rw.borrow(), key)?;
match self.db.delete(&mut self.tx_rw.borrow_mut(), key) {
Ok(true) => Ok(value),
Err(e) => Err(e.into()),
// We just `get()`'ed the value - it is
// incorrect for it to suddenly not exist.
Ok(false) => unreachable!(),
}
}
#[inline] #[inline]
fn pop_first(&mut self) -> Result<(T::Key, T::Value), RuntimeError> { fn pop_first(&mut self) -> Result<(T::Key, T::Value), RuntimeError> {
let tx_rw = &mut self.tx_rw.borrow_mut(); let tx_rw = &mut self.tx_rw.borrow_mut();
// Get the first value first... // Get the value first...
let Some(first) = self.db.first(tx_rw)? else { let Some((key, value)) = self.db.first(tx_rw)? else {
return Err(RuntimeError::KeyNotFound); return Err(RuntimeError::KeyNotFound);
}; };
// ...then remove it. // ...then remove it.
// match self.db.delete(tx_rw, &key) {
// We use an iterator because we want to semantically Ok(true) => Ok((key, value)),
// remove the _first_ and only the first `(key, value)`. Err(e) => Err(e.into()),
// `delete()` removes all keys including duplicates which // We just `get()`'ed the value - it is
// is slightly different behavior. // incorrect for it to suddenly not exist.
let mut iter = self.db.iter_mut(tx_rw)?; Ok(false) => unreachable!(),
// SAFETY:
// It is undefined behavior to keep a reference of
// a value from this database while modifying it.
// We are deleting the value and never accessing
// the iterator again so this should be safe.
unsafe {
iter.del_current()?;
} }
Ok(first)
} }
#[inline] #[inline]
fn pop_last(&mut self) -> Result<(T::Key, T::Value), RuntimeError> { fn pop_last(&mut self) -> Result<(T::Key, T::Value), RuntimeError> {
let tx_rw = &mut self.tx_rw.borrow_mut(); let tx_rw = &mut self.tx_rw.borrow_mut();
let Some(first) = self.db.last(tx_rw)? else { // Get the value first...
let Some((key, value)) = self.db.last(tx_rw)? else {
return Err(RuntimeError::KeyNotFound); return Err(RuntimeError::KeyNotFound);
}; };
let mut iter = self.db.rev_iter_mut(tx_rw)?; // ...then remove it.
match self.db.delete(tx_rw, &key) {
// SAFETY: Ok(true) => Ok((key, value)),
// It is undefined behavior to keep a reference of Err(e) => Err(e.into()),
// a value from this database while modifying it. // We just `get()`'ed the value - it is
// We are deleting the value and never accessing // incorrect for it to suddenly not exist.
// the iterator again so this should be safe. Ok(false) => unreachable!(),
unsafe {
iter.del_current()?;
} }
Ok(first)
} }
} }

View file

@ -207,17 +207,15 @@ impl Env for ConcreteEnv {
} }
use crate::tables::{ use crate::tables::{
BlockBlobs, BlockHeights, BlockInfoV1s, BlockInfoV2s, BlockInfoV3s, KeyImages, BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
NumOutputs, Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxBlobs, TxHeights, TxIds, TxOutputs,
TxHeights, TxIds, TxUnlockTime, TxUnlockTime,
}; };
let mut tx_rw = env.write_txn()?; let mut tx_rw = env.write_txn()?;
create_table::<BlockBlobs>(&env, &mut tx_rw)?; create_table::<BlockBlobs>(&env, &mut tx_rw)?;
create_table::<BlockHeights>(&env, &mut tx_rw)?; create_table::<BlockHeights>(&env, &mut tx_rw)?;
create_table::<BlockInfoV1s>(&env, &mut tx_rw)?; create_table::<BlockInfos>(&env, &mut tx_rw)?;
create_table::<BlockInfoV2s>(&env, &mut tx_rw)?;
create_table::<BlockInfoV3s>(&env, &mut tx_rw)?;
create_table::<KeyImages>(&env, &mut tx_rw)?; create_table::<KeyImages>(&env, &mut tx_rw)?;
create_table::<NumOutputs>(&env, &mut tx_rw)?; create_table::<NumOutputs>(&env, &mut tx_rw)?;
create_table::<Outputs>(&env, &mut tx_rw)?; create_table::<Outputs>(&env, &mut tx_rw)?;
@ -225,8 +223,10 @@ impl Env for ConcreteEnv {
create_table::<PrunableTxBlobs>(&env, &mut tx_rw)?; create_table::<PrunableTxBlobs>(&env, &mut tx_rw)?;
create_table::<PrunedTxBlobs>(&env, &mut tx_rw)?; create_table::<PrunedTxBlobs>(&env, &mut tx_rw)?;
create_table::<RctOutputs>(&env, &mut tx_rw)?; create_table::<RctOutputs>(&env, &mut tx_rw)?;
create_table::<TxBlobs>(&env, &mut tx_rw)?;
create_table::<TxHeights>(&env, &mut tx_rw)?; create_table::<TxHeights>(&env, &mut tx_rw)?;
create_table::<TxIds>(&env, &mut tx_rw)?; create_table::<TxIds>(&env, &mut tx_rw)?;
create_table::<TxOutputs>(&env, &mut tx_rw)?;
create_table::<TxUnlockTime>(&env, &mut tx_rw)?; create_table::<TxUnlockTime>(&env, &mut tx_rw)?;
// TODO: Set dupsort and comparison functions for certain tables // TODO: Set dupsort and comparison functions for certain tables

View file

@ -188,6 +188,15 @@ impl<T: Table + 'static> DatabaseRw<T> for RedbTableRw<'_, T::Key, T::Value> {
Ok(()) Ok(())
} }
#[inline]
fn take(&mut self, key: &T::Key) -> Result<T::Value, RuntimeError> {
if let Some(value) = redb::Table::remove(self, key)? {
Ok(value.value())
} else {
Err(RuntimeError::KeyNotFound)
}
}
#[inline] #[inline]
fn pop_first(&mut self) -> Result<(T::Key, T::Value), RuntimeError> { fn pop_first(&mut self) -> Result<(T::Key, T::Value), RuntimeError> {
let (key, value) = redb::Table::pop_first(self)?.ok_or(RuntimeError::KeyNotFound)?; let (key, value) = redb::Table::pop_first(self)?.ok_or(RuntimeError::KeyNotFound)?;

View file

@ -109,17 +109,15 @@ impl Env for ConcreteEnv {
} }
use crate::tables::{ use crate::tables::{
BlockBlobs, BlockHeights, BlockInfoV1s, BlockInfoV2s, BlockInfoV3s, KeyImages, BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
NumOutputs, Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxBlobs, TxHeights, TxIds, TxOutputs,
TxHeights, TxIds, TxUnlockTime, TxUnlockTime,
}; };
let tx_rw = env.begin_write()?; let tx_rw = env.begin_write()?;
create_table::<BlockBlobs>(&tx_rw)?; create_table::<BlockBlobs>(&tx_rw)?;
create_table::<BlockHeights>(&tx_rw)?; create_table::<BlockHeights>(&tx_rw)?;
create_table::<BlockInfoV1s>(&tx_rw)?; create_table::<BlockInfos>(&tx_rw)?;
create_table::<BlockInfoV2s>(&tx_rw)?;
create_table::<BlockInfoV3s>(&tx_rw)?;
create_table::<KeyImages>(&tx_rw)?; create_table::<KeyImages>(&tx_rw)?;
create_table::<NumOutputs>(&tx_rw)?; create_table::<NumOutputs>(&tx_rw)?;
create_table::<Outputs>(&tx_rw)?; create_table::<Outputs>(&tx_rw)?;
@ -127,8 +125,10 @@ impl Env for ConcreteEnv {
create_table::<PrunableTxBlobs>(&tx_rw)?; create_table::<PrunableTxBlobs>(&tx_rw)?;
create_table::<PrunedTxBlobs>(&tx_rw)?; create_table::<PrunedTxBlobs>(&tx_rw)?;
create_table::<RctOutputs>(&tx_rw)?; create_table::<RctOutputs>(&tx_rw)?;
create_table::<TxBlobs>(&tx_rw)?;
create_table::<TxHeights>(&tx_rw)?; create_table::<TxHeights>(&tx_rw)?;
create_table::<TxIds>(&tx_rw)?; create_table::<TxIds>(&tx_rw)?;
create_table::<TxOutputs>(&tx_rw)?;
create_table::<TxUnlockTime>(&tx_rw)?; create_table::<TxUnlockTime>(&tx_rw)?;
tx_rw.commit()?; tx_rw.commit()?;

View file

@ -31,32 +31,21 @@ use crate::{
storable::StorableVec, storable::StorableVec,
table::Table, table::Table,
tables::{ tables::{
BlockBlobs, BlockHeights, BlockInfoV1s, BlockInfoV2s, BlockInfoV3s, KeyImages, NumOutputs, BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxHeights, TxIds, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, TxBlobs, TxHeights, TxIds, TxOutputs,
TxUnlockTime, TxUnlockTime,
}, },
tests::tmp_concrete_env,
transaction::{TxRo, TxRw}, transaction::{TxRo, TxRw},
types::{ types::{
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfoV1, Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfo, KeyImage,
BlockInfoV2, BlockInfoV3, KeyImage, Output, PreRctOutputId, PrunableBlob, PrunableHash, Output, OutputFlags, PreRctOutputId, PrunableBlob, PrunableHash, PrunedBlob, RctOutput,
PrunedBlob, RctOutput, TxHash, TxId, UnlockTime, TxBlob, TxHash, TxId, UnlockTime,
}, },
ConcreteEnv, ConcreteEnv,
}; };
//---------------------------------------------------------------------------------------------------- Tests //---------------------------------------------------------------------------------------------------- Tests
/// Create an `Env` in a temporarily directory.
/// The directory is automatically removed after the `TempDir` is dropped.
///
/// TODO: changing this to `-> impl Env` causes lifetime errors...
fn tmp_concrete_env() -> (ConcreteEnv, tempfile::TempDir) {
let tempdir = tempfile::tempdir().unwrap();
let config = Config::low_power(Some(tempdir.path().into()));
let env = ConcreteEnv::open(config).unwrap();
(env, tempdir)
}
/// Simply call [`Env::open`]. If this fails, something is really wrong. /// Simply call [`Env::open`]. If this fails, something is really wrong.
#[test] #[test]
fn open() { fn open() {
@ -87,9 +76,7 @@ fn open_db() {
// This should be updated when tables are modified. // This should be updated when tables are modified.
env_inner.open_db_ro::<BlockBlobs>(&tx_ro).unwrap(); env_inner.open_db_ro::<BlockBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockHeights>(&tx_ro).unwrap(); env_inner.open_db_ro::<BlockHeights>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfoV1s>(&tx_ro).unwrap(); env_inner.open_db_ro::<BlockInfos>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfoV2s>(&tx_ro).unwrap();
env_inner.open_db_ro::<BlockInfoV3s>(&tx_ro).unwrap();
env_inner.open_db_ro::<KeyImages>(&tx_ro).unwrap(); env_inner.open_db_ro::<KeyImages>(&tx_ro).unwrap();
env_inner.open_db_ro::<NumOutputs>(&tx_ro).unwrap(); env_inner.open_db_ro::<NumOutputs>(&tx_ro).unwrap();
env_inner.open_db_ro::<Outputs>(&tx_ro).unwrap(); env_inner.open_db_ro::<Outputs>(&tx_ro).unwrap();
@ -97,17 +84,17 @@ fn open_db() {
env_inner.open_db_ro::<PrunableTxBlobs>(&tx_ro).unwrap(); env_inner.open_db_ro::<PrunableTxBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<PrunedTxBlobs>(&tx_ro).unwrap(); env_inner.open_db_ro::<PrunedTxBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<RctOutputs>(&tx_ro).unwrap(); env_inner.open_db_ro::<RctOutputs>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxBlobs>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxHeights>(&tx_ro).unwrap(); env_inner.open_db_ro::<TxHeights>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxIds>(&tx_ro).unwrap(); env_inner.open_db_ro::<TxIds>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxOutputs>(&tx_ro).unwrap();
env_inner.open_db_ro::<TxUnlockTime>(&tx_ro).unwrap(); env_inner.open_db_ro::<TxUnlockTime>(&tx_ro).unwrap();
TxRo::commit(tx_ro).unwrap(); TxRo::commit(tx_ro).unwrap();
// Open all tables in read/write mode. // Open all tables in read/write mode.
env_inner.open_db_rw::<BlockBlobs>(&tx_rw).unwrap(); env_inner.open_db_rw::<BlockBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockHeights>(&tx_rw).unwrap(); env_inner.open_db_rw::<BlockHeights>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfoV1s>(&tx_rw).unwrap(); env_inner.open_db_rw::<BlockInfos>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfoV2s>(&tx_rw).unwrap();
env_inner.open_db_rw::<BlockInfoV3s>(&tx_rw).unwrap();
env_inner.open_db_rw::<KeyImages>(&tx_rw).unwrap(); env_inner.open_db_rw::<KeyImages>(&tx_rw).unwrap();
env_inner.open_db_rw::<NumOutputs>(&tx_rw).unwrap(); env_inner.open_db_rw::<NumOutputs>(&tx_rw).unwrap();
env_inner.open_db_rw::<Outputs>(&tx_rw).unwrap(); env_inner.open_db_rw::<Outputs>(&tx_rw).unwrap();
@ -115,8 +102,10 @@ fn open_db() {
env_inner.open_db_rw::<PrunableTxBlobs>(&tx_rw).unwrap(); env_inner.open_db_rw::<PrunableTxBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<PrunedTxBlobs>(&tx_rw).unwrap(); env_inner.open_db_rw::<PrunedTxBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<RctOutputs>(&tx_rw).unwrap(); env_inner.open_db_rw::<RctOutputs>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxBlobs>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxHeights>(&tx_rw).unwrap(); env_inner.open_db_rw::<TxHeights>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxIds>(&tx_rw).unwrap(); env_inner.open_db_rw::<TxIds>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxOutputs>(&tx_rw).unwrap();
env_inner.open_db_rw::<TxUnlockTime>(&tx_rw).unwrap(); env_inner.open_db_rw::<TxUnlockTime>(&tx_rw).unwrap();
TxRw::commit(tx_rw).unwrap(); TxRw::commit(tx_rw).unwrap();
} }
@ -182,7 +171,7 @@ fn db_read_write() {
const VALUE: Output = Output { const VALUE: Output = Output {
key: [35; 32], key: [35; 32],
height: 45_761_798, height: 45_761_798,
output_flags: 0, output_flags: OutputFlags::empty(),
tx_idx: 2_353_487, tx_idx: 2_353_487,
}; };
/// How many `(key, value)` pairs will be inserted. /// How many `(key, value)` pairs will be inserted.
@ -271,6 +260,22 @@ fn db_read_write() {
} }
} }
// Assert `update()` works.
{
const HEIGHT: u32 = 999;
assert_ne!(table.get(&KEY).unwrap().height, HEIGHT);
table
.update(&KEY, |mut value| {
value.height = HEIGHT;
Some(value)
})
.unwrap();
assert_eq!(table.get(&KEY).unwrap().height, HEIGHT);
}
// Assert deleting works. // Assert deleting works.
{ {
table.delete(&KEY).unwrap(); table.delete(&KEY).unwrap();
@ -284,6 +289,23 @@ fn db_read_write() {
assert_same(value); assert_same(value);
} }
// Assert `take()` works.
{
let mut key = KEY;
key.amount += 1;
let value = table.take(&key).unwrap();
assert_eq!(value, VALUE);
let get = table.get(&KEY);
assert!(!table.contains(&key).unwrap());
assert!(matches!(get, Err(RuntimeError::KeyNotFound)));
// Assert the other `(key, value)` pairs are still there.
key.amount += 1;
let value = table.get(&key).unwrap();
assert_same(value);
}
drop(table); drop(table);
TxRw::commit(tx_rw).unwrap(); TxRw::commit(tx_rw).unwrap();
@ -406,36 +428,14 @@ test_tables! {
BlockHash => BlockHeight, BlockHash => BlockHeight,
[32; 32] => 123, [32; 32] => 123,
BlockInfoV1s, BlockInfos,
BlockHeight => BlockInfoV1, BlockHeight => BlockInfo,
123 => BlockInfoV1 { 123 => BlockInfo {
timestamp: 1, timestamp: 1,
total_generated_coins: 123, cumulative_generated_coins: 123,
weight: 321, weight: 321,
cumulative_difficulty: 111, cumulative_difficulty: 111,
block_hash: [54; 32], block_hash: [54; 32],
},
BlockInfoV2s,
BlockHeight => BlockInfoV2,
123 => BlockInfoV2 {
timestamp: 1,
total_generated_coins: 123,
weight: 321,
cumulative_difficulty: 111,
cumulative_rct_outs: 2389,
block_hash: [54; 32],
},
BlockInfoV3s,
BlockHeight => BlockInfoV3,
123 => BlockInfoV3 {
timestamp: 1,
total_generated_coins: 123,
weight: 321,
cumulative_difficulty_low: 111,
cumulative_difficulty_high: 112,
block_hash: [54; 32],
cumulative_rct_outs: 2389, cumulative_rct_outs: 2389,
long_term_weight: 2389, long_term_weight: 2389,
}, },
@ -448,6 +448,10 @@ test_tables! {
Amount => AmountIndex, Amount => AmountIndex,
123 => 123, 123 => 123,
TxBlobs,
TxId => TxBlob,
123 => StorableVec(vec![1,2,3,4,5,6,7,8]),
TxIds, TxIds,
TxHash => TxId, TxHash => TxId,
[32; 32] => 123, [32; 32] => 123,
@ -456,6 +460,10 @@ test_tables! {
TxId => BlockHeight, TxId => BlockHeight,
123 => 123, 123 => 123,
TxOutputs,
TxId => AmountIndices,
123 => StorableVec(vec![1,2,3,4,5,6,7,8]),
TxUnlockTime, TxUnlockTime,
TxId => UnlockTime, TxId => UnlockTime,
123 => 123, 123 => 123,
@ -468,7 +476,7 @@ test_tables! {
} => Output { } => Output {
key: [1; 32], key: [1; 32],
height: 1, height: 1,
output_flags: 0, output_flags: OutputFlags::empty(),
tx_idx: 3, tx_idx: 3,
}, },
@ -489,7 +497,7 @@ test_tables! {
123 => RctOutput { 123 => RctOutput {
key: [1; 32], key: [1; 32],
height: 1, height: 1,
output_flags: 0, output_flags: OutputFlags::empty(),
tx_idx: 3, tx_idx: 3,
commitment: [3; 32], commitment: [3; 32],
}, },

View file

@ -3,6 +3,18 @@
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use cfg_if::cfg_if; use cfg_if::cfg_if;
//---------------------------------------------------------------------------------------------------- Version
/// Current major version of the database.
///
/// Returned by [`crate::ops::property::db_version`].
///
/// This is incremented by 1 when `cuprate_database`'s
/// structure/schema/tables change.
///
/// This is akin to `VERSION` in `monerod`:
/// <https://github.com/monero-project/monero/blob/c8214782fb2a769c57382a999eaf099691c836e7/src/blockchain_db/lmdb/db_lmdb.cpp#L57>
pub const DATABASE_VERSION: u64 = 0;
//---------------------------------------------------------------------------------------------------- Error Messages //---------------------------------------------------------------------------------------------------- Error Messages
/// Corrupt database error message. /// Corrupt database error message.
/// ///

View file

@ -13,7 +13,7 @@ use crate::{
transaction::{TxRo, TxRw}, transaction::{TxRo, TxRw},
}; };
//---------------------------------------------------------------------------------------------------- DatabaseRoIter //---------------------------------------------------------------------------------------------------- DatabaseIter
/// Database (key-value store) read-only iteration abstraction. /// Database (key-value store) read-only iteration abstraction.
/// ///
/// These are read-only iteration-related operations that /// These are read-only iteration-related operations that
@ -140,14 +140,50 @@ pub trait DatabaseRw<T: Table>: DatabaseRo<T> {
/// This will overwrite any existing key-value pairs. /// This will overwrite any existing key-value pairs.
/// ///
/// # Errors /// # Errors
/// This will not return [`RuntimeError::KeyExists`]. /// This will never [`RuntimeError::KeyExists`].
fn put(&mut self, key: &T::Key, value: &T::Value) -> Result<(), RuntimeError>; fn put(&mut self, key: &T::Key, value: &T::Value) -> Result<(), RuntimeError>;
/// Delete a key-value pair in the database. /// Delete a key-value pair in the database.
/// ///
/// This will return `Ok(())` if the key does not exist.
///
/// # Errors
/// This will never [`RuntimeError::KeyNotFound`].
fn delete(&mut self, key: &T::Key) -> Result<(), RuntimeError>;
/// Delete and return a key-value pair in the database.
///
/// This is the same as [`DatabaseRw::delete`], however,
/// it will serialize the `T::Value` and return it.
///
/// # Errors /// # Errors
/// This will return [`RuntimeError::KeyNotFound`] wrapped in [`Err`] if `key` does not exist. /// This will return [`RuntimeError::KeyNotFound`] wrapped in [`Err`] if `key` does not exist.
fn delete(&mut self, key: &T::Key) -> Result<(), RuntimeError>; fn take(&mut self, key: &T::Key) -> Result<T::Value, RuntimeError>;
/// Fetch the value, and apply a function to it - or delete the entry.
///
/// This will call [`DatabaseRo::get`] and call your provided function `f` on it.
///
/// The [`Option`] `f` returns will dictate whether `update()`:
/// - Updates the current value OR
/// - Deletes the `(key, value)` pair
///
/// - If `f` returns `Some(value)`, that will be [`DatabaseRw::put`] as the new value
/// - If `f` returns `None`, the entry will be [`DatabaseRw::delete`]d
///
/// # Errors
/// This will return [`RuntimeError::KeyNotFound`] wrapped in [`Err`] if `key` does not exist.
fn update<F>(&mut self, key: &T::Key, mut f: F) -> Result<(), RuntimeError>
where
F: FnMut(T::Value) -> Option<T::Value>,
{
let value = DatabaseRo::get(self, key)?;
match f(value) {
Some(value) => DatabaseRw::put(self, key, &value),
None => DatabaseRw::delete(self, key),
}
}
/// TODO /// TODO
/// ///

View file

@ -9,6 +9,11 @@ use crate::{
error::{InitError, RuntimeError}, error::{InitError, RuntimeError},
resize::ResizeAlgorithm, resize::ResizeAlgorithm,
table::Table, table::Table,
tables::{
call_fn_on_all_tables_or_early_return, BlockBlobs, BlockHeights, BlockInfos, KeyImages,
NumOutputs, Outputs, PrunableHashes, PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables,
TablesMut, TxHeights, TxIds, TxUnlockTime,
},
transaction::{TxRo, TxRw}, transaction::{TxRo, TxRw},
}; };
@ -224,6 +229,26 @@ where
/// a table doesn't exist. /// a table doesn't exist.
fn open_db_rw<T: Table>(&self, tx_rw: &Rw) -> Result<impl DatabaseRw<T>, RuntimeError>; fn open_db_rw<T: Table>(&self, tx_rw: &Rw) -> Result<impl DatabaseRw<T>, RuntimeError>;
/// TODO
///
/// # Errors
/// TODO
fn open_tables(&self, tx_ro: &Ro) -> Result<impl Tables, RuntimeError> {
call_fn_on_all_tables_or_early_return! {
Self::open_db_ro(self, tx_ro)
}
}
/// TODO
///
/// # Errors
/// TODO
fn open_tables_mut(&self, tx_rw: &Rw) -> Result<impl TablesMut, RuntimeError> {
call_fn_on_all_tables_or_early_return! {
Self::open_db_rw(self, tx_rw)
}
}
/// Clear all `(key, value)`'s from a database table. /// Clear all `(key, value)`'s from a database table.
/// ///
/// This will delete all key and values in the passed /// This will delete all key and values in the passed

View file

@ -1,4 +1,4 @@
//! Database abstraction and utilities. //! Cuprate's database abstraction.
//! //!
//! This documentation is mostly for practical usage of `cuprate_database`. //! This documentation is mostly for practical usage of `cuprate_database`.
//! //!
@ -8,20 +8,25 @@
//! # Purpose //! # Purpose
//! This crate does 3 things: //! This crate does 3 things:
//! 1. Abstracts various database backends with traits //! 1. Abstracts various database backends with traits
//! 2. Implements various `Monero` related [functions](ops) & [tables] & [types] //! 2. Implements various `Monero` related [operations](ops), [tables], and [types]
//! 3. Exposes a [`tower::Service`] backed by a thread-pool //! 3. Exposes a [`tower::Service`] backed by a thread-pool
//! //!
//! Each layer builds on-top of the previous.
//!
//! As a user of `cuprate_database`, consider using the higher-level [`service`],
//! or at the very least [`ops`] instead of interacting with the database traits directly.
//!
//! With that said, many database traits and internals (like [`DatabaseRo::get`]) are exposed.
//!
//! # Terminology //! # Terminology
//! To be more clear on some terms used in this crate: //! To be more clear on some terms used in this crate:
//! //!
//! | Term | Meaning | //! | Term | Meaning |
//! |---------------|--------------------------------------| //! |------------------|--------------------------------------|
//! | `Env` | The 1 database environment, the "whole" thing //! | `Env` | The 1 database environment, the "whole" thing
//! | `DatabaseRo` | A read-only `key/value` store //! | `DatabaseR{o,w}` | A _actively open_ readable/writable `key/value` store
//! | `DatabaseRw` | A readable/writable `key/value` store
//! | `Table` | Solely the metadata of a `Database` (the `key` and `value` types, and the name) //! | `Table` | Solely the metadata of a `Database` (the `key` and `value` types, and the name)
//! | `TxRo` | Read only transaction //! | `TxR{o,w}` | A read/write transaction
//! | `TxRw` | Read/write transaction
//! | `Storable` | A data that type can be stored in the database //! | `Storable` | A data that type can be stored in the database
//! //!
//! The dataflow is `Env` -> `Tx` -> `Database` //! The dataflow is `Env` -> `Tx` -> `Database`
@ -29,7 +34,7 @@
//! Which reads as: //! Which reads as:
//! 1. You have a database `Environment` //! 1. You have a database `Environment`
//! 1. You open up a `Transaction` //! 1. You open up a `Transaction`
//! 1. You get a particular `Database` from that `Environment` //! 1. You open a particular `Table` from that `Environment`, getting a `Database`
//! 1. You can now read/write data from/to that `Database` //! 1. You can now read/write data from/to that `Database`
//! //!
//! # `ConcreteEnv` //! # `ConcreteEnv`
@ -138,7 +143,6 @@
unconditional_recursion, unconditional_recursion,
for_loops_over_fallibles, for_loops_over_fallibles,
unused_braces, unused_braces,
unused_doc_comments,
unused_labels, unused_labels,
keyword_idents, keyword_idents,
non_ascii_idents, non_ascii_idents,
@ -169,6 +173,7 @@
clippy::pedantic, clippy::pedantic,
clippy::nursery, clippy::nursery,
clippy::cargo, clippy::cargo,
unused_doc_comments,
unused_mut, unused_mut,
missing_docs, missing_docs,
deprecated, deprecated,
@ -221,6 +226,7 @@ pub mod config;
mod constants; mod constants;
pub use constants::{ pub use constants::{
DATABASE_BACKEND, DATABASE_CORRUPT_MSG, DATABASE_DATA_FILENAME, DATABASE_LOCK_FILENAME, DATABASE_BACKEND, DATABASE_CORRUPT_MSG, DATABASE_DATA_FILENAME, DATABASE_LOCK_FILENAME,
DATABASE_VERSION,
}; };
mod database; mod database;
@ -261,3 +267,5 @@ pub use transaction::{TxRo, TxRw};
pub mod service; pub mod service;
//---------------------------------------------------------------------------------------------------- Private //---------------------------------------------------------------------------------------------------- Private
#[cfg(test)]
pub(crate) mod tests;

View file

@ -1,89 +1,450 @@
//! Blocks. //! Blocks.
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use std::sync::Arc;
//---------------------------------------------------------------------------------------------------- Free Functions use bytemuck::TransparentWrapper;
/// TODO use curve25519_dalek::{constants::ED25519_BASEPOINT_POINT, Scalar};
pub fn add_block() { use monero_serai::{
todo!() block::Block,
transaction::{Input, Timelock, Transaction},
};
use cuprate_types::{ExtendedBlockHeader, TransactionVerificationData, VerifiedBlockInformation};
use crate::{
database::{DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::{
blockchain::{chain_height, cumulative_generated_coins},
key_image::{add_key_image, remove_key_image},
macros::doc_error,
output::{
add_output, add_rct_output, get_rct_num_outputs, remove_output, remove_rct_output,
},
tx::{add_tx, get_num_tx, remove_tx},
},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
AmountIndex, BlockHash, BlockHeight, BlockInfo, KeyImage, Output, OutputFlags,
PreRctOutputId, RctOutput, TxHash,
},
StorableVec,
};
//---------------------------------------------------------------------------------------------------- `add_block_*`
/// Add a [`VerifiedBlockInformation`] to the database.
///
/// This extracts all the data from the input block and
/// maps/adds them to the appropriate database tables.
///
#[doc = doc_error!()]
///
/// # Panics
/// This function will panic if:
/// - `block.height > u32::MAX` (not normally possible)
/// - `block.height` is not != [`chain_height`]
///
/// # Already exists
/// This function will operate normally even if `block` already
/// exists, i.e., this function will not return `Err` even if you
/// call this function infinitely with the same block.
// no inline, too big.
pub fn add_block(
block: &VerifiedBlockInformation,
tables: &mut impl TablesMut,
) -> Result<(), RuntimeError> {
//------------------------------------------------------ Check preconditions first
// Cast height to `u32` for storage (handled at top of function).
// Panic (should never happen) instead of allowing DB corruption.
// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1560020991>
let Ok(height) = u32::try_from(block.height) else {
panic!("block.height ({}) > u32::MAX", block.height);
};
let chain_height = chain_height(tables.block_heights())?;
assert_eq!(
block.height, chain_height,
"block.height ({}) != chain_height ({})",
block.height, chain_height,
);
// Expensive checks - debug only.
#[cfg(debug_assertions)]
{
assert_eq!(block.block.serialize(), block.block_blob);
assert_eq!(block.block.txs.len(), block.txs.len());
for (i, tx) in block.txs.iter().enumerate() {
assert_eq!(tx.tx_blob, tx.tx.serialize());
assert_eq!(tx.tx_hash, block.block.txs[i]);
}
}
//------------------------------------------------------ Transaction / Outputs / Key Images
for tx_verification_data in &block.txs {
add_tx(tx_verification_data, &chain_height, tables)?;
}
//------------------------------------------------------ Block Info
// INVARIANT: must be below the above transaction loop since this
// RCT output count needs account for _this_ block's outputs.
let cumulative_rct_outs = get_rct_num_outputs(tables.rct_outputs())?;
let cumulative_generated_coins =
cumulative_generated_coins(&block.height.saturating_sub(1), tables.block_infos())?
+ block.generated_coins;
// Block Info.
tables.block_infos_mut().put(
&block.height,
&BlockInfo {
cumulative_generated_coins,
cumulative_rct_outs,
timestamp: block.block.header.timestamp,
cumulative_difficulty: block.cumulative_difficulty,
block_hash: block.block_hash,
// INVARIANT: #[cfg] @ lib.rs asserts `usize == u64`
weight: block.weight as u64,
long_term_weight: block.long_term_weight as u64,
},
)?;
// Block blobs.
tables
.block_blobs_mut()
.put(&block.height, StorableVec::wrap_ref(&block.block_blob))?;
// Block heights.
tables
.block_heights_mut()
.put(&block.block_hash, &block.height)?;
Ok(())
} }
/// TODO //---------------------------------------------------------------------------------------------------- `pop_block`
pub fn add_block_data() { /// Remove the top/latest block from the database.
todo!() ///
/// The removed block's height and hash are returned.
#[doc = doc_error!()]
// no inline, too big
pub fn pop_block(
tables: &mut impl TablesMut,
) -> Result<(BlockHeight, BlockHash, Block), RuntimeError> {
//------------------------------------------------------ Block Info
// Remove block data from tables.
let (block_height, block_hash) = {
let (block_height, block_info) = tables.block_infos_mut().pop_last()?;
(block_height, block_info.block_hash)
};
// Block heights.
tables.block_heights_mut().delete(&block_hash)?;
// Block blobs.
// We deserialize the block blob into a `Block`, such
// that we can remove the associated transactions later.
let block_blob = tables.block_blobs_mut().take(&block_height)?.0;
let block = Block::read(&mut block_blob.as_slice())?;
//------------------------------------------------------ Transaction / Outputs / Key Images
for tx_hash in &block.txs {
remove_tx(tx_hash, tables)?;
}
Ok((block_height, block_hash, block))
} }
/// TODO //---------------------------------------------------------------------------------------------------- `get_block_extended_header_*`
pub fn pop_block() { /// Retrieve a [`ExtendedBlockHeader`] from the database.
todo!() ///
/// This extracts all the data from the database tables
/// needed to create a full `ExtendedBlockHeader`.
///
/// # Notes
/// This is slightly more expensive than [`get_block_extended_header_from_height`]
/// (1 more database lookup).
#[doc = doc_error!()]
#[inline]
pub fn get_block_extended_header(
block_hash: &BlockHash,
tables: &impl Tables,
) -> Result<ExtendedBlockHeader, RuntimeError> {
get_block_extended_header_from_height(&tables.block_heights().get(block_hash)?, tables)
} }
/// TODO /// Same as [`get_block_extended_header`] but with a [`BlockHeight`].
pub fn block_exists() { #[doc = doc_error!()]
todo!() #[inline]
pub fn get_block_extended_header_from_height(
block_height: &BlockHeight,
tables: &impl Tables,
) -> Result<ExtendedBlockHeader, RuntimeError> {
let block_info = tables.block_infos().get(block_height)?;
let block_blob = tables.block_blobs().get(block_height)?.0;
let block = Block::read(&mut block_blob.as_slice())?;
// INVARIANT: #[cfg] @ lib.rs asserts `usize == u64`
#[allow(clippy::cast_possible_truncation)]
Ok(ExtendedBlockHeader {
version: block.header.major_version,
vote: block.header.minor_version,
timestamp: block.header.timestamp,
cumulative_difficulty: block_info.cumulative_difficulty,
block_weight: block_info.weight as usize,
long_term_weight: block_info.long_term_weight as usize,
})
} }
/// TODO /// Return the top/latest [`ExtendedBlockHeader`] from the database.
pub fn get_block_hash() { #[doc = doc_error!()]
todo!() #[inline]
pub fn get_block_extended_header_top(
tables: &impl Tables,
) -> Result<(ExtendedBlockHeader, BlockHeight), RuntimeError> {
let height = chain_height(tables.block_heights())?.saturating_sub(1);
let header = get_block_extended_header_from_height(&height, tables)?;
Ok((header, height))
} }
/// TODO //---------------------------------------------------------------------------------------------------- Misc
pub fn get_block_height() { /// Retrieve a [`BlockHeight`] via its [`BlockHash`].
todo!() #[doc = doc_error!()]
#[inline]
pub fn get_block_height(
block_hash: &BlockHash,
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<BlockHeight, RuntimeError> {
table_block_heights.get(block_hash)
} }
/// TODO /// Check if a block exists in the database.
pub fn get_block_weight() { #[doc = doc_error!()]
todo!() #[inline]
pub fn block_exists(
block_hash: &BlockHash,
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<bool, RuntimeError> {
table_block_heights.contains(block_hash)
} }
/// TODO //---------------------------------------------------------------------------------------------------- Tests
pub fn get_block_already_generated_coins() { #[cfg(test)]
todo!() #[allow(
} clippy::significant_drop_tightening,
clippy::cognitive_complexity,
clippy::too_many_lines
)]
mod test {
use hex_literal::hex;
use pretty_assertions::assert_eq;
/// TODO use cuprate_test_utils::data::{block_v16_tx0, block_v1_tx2, block_v9_tx3, tx_v2_rct3};
pub fn get_block_long_term_weight() {
todo!()
}
/// TODO use super::*;
pub fn get_block_timestamp() { use crate::{
todo!() ops::tx::{get_tx, tx_exists},
} tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
/// TODO /// Tests all above block functions.
pub fn get_block_cumulative_rct_outputs() { ///
todo!() /// Note that this doesn't test the correctness of values added, as the
} /// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
fn all_block_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
/// TODO let mut blocks = [
pub fn get_block() { block_v1_tx2().clone(),
todo!() block_v9_tx3().clone(),
} block_v16_tx0().clone(),
];
// HACK: `add_block()` asserts blocks with non-sequential heights
// cannot be added, to get around this, manually edit the block height.
for (height, block) in blocks.iter_mut().enumerate() {
block.height = height as u64;
assert_eq!(block.block.serialize(), block.block_blob);
}
let generated_coins_sum = blocks
.iter()
.map(|block| block.generated_coins)
.sum::<u64>();
/// TODO // Add blocks.
pub fn get_block_from_height() { {
todo!() let tx_rw = env_inner.tx_rw().unwrap();
} let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
/// TODO for block in &blocks {
pub fn get_block_header() { // println!("add_block: {block:#?}");
todo!() add_block(block, &mut tables).unwrap();
} }
/// TODO drop(tables);
pub fn get_block_header_from_height() { TxRw::commit(tx_rw).unwrap();
todo!() }
}
/// TODO // Assert all reads are OK.
pub fn get_top_block() { let block_hashes = {
todo!() let tx_ro = env_inner.tx_ro().unwrap();
} let tables = env_inner.open_tables(&tx_ro).unwrap();
/// TODO // Assert only the proper tables were added to.
pub fn get_top_block_hash() { assert_eq!(tables.block_infos().len().unwrap(), 3);
todo!() assert_eq!(tables.block_blobs().len().unwrap(), 3);
assert_eq!(tables.block_heights().len().unwrap(), 3);
assert_eq!(tables.key_images().len().unwrap(), 69);
assert_eq!(tables.num_outputs().len().unwrap(), 38);
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.outputs().len().unwrap(), 107);
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 6);
assert_eq!(tables.tx_blobs().len().unwrap(), 5);
assert_eq!(tables.tx_ids().len().unwrap(), 5);
assert_eq!(tables.tx_heights().len().unwrap(), 5);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 0);
// Check `cumulative` functions work.
assert_eq!(
cumulative_generated_coins(&2, tables.block_infos()).unwrap(),
generated_coins_sum,
);
// Both height and hash should result in getting the same data.
let mut block_hashes = vec![];
for block in &blocks {
println!("blocks.iter(): hash: {}", hex::encode(block.block_hash));
let height = get_block_height(&block.block_hash, tables.block_heights()).unwrap();
println!("blocks.iter(): height: {height}");
assert!(block_exists(&block.block_hash, tables.block_heights()).unwrap());
let block_header_from_height =
get_block_extended_header_from_height(&height, &tables).unwrap();
let block_header_from_hash =
get_block_extended_header(&block.block_hash, &tables).unwrap();
// Just an alias, these names are long.
let b1 = block_header_from_hash;
let b2 = block;
assert_eq!(b1, block_header_from_height);
assert_eq!(b1.version, b2.block.header.major_version);
assert_eq!(b1.vote, b2.block.header.minor_version);
assert_eq!(b1.timestamp, b2.block.header.timestamp);
assert_eq!(b1.cumulative_difficulty, b2.cumulative_difficulty);
assert_eq!(b1.block_weight, b2.weight);
assert_eq!(b1.long_term_weight, b2.long_term_weight);
block_hashes.push(block.block_hash);
// Assert transaction reads are OK.
for (i, tx) in block.txs.iter().enumerate() {
println!("tx_hash: {:?}", hex::encode(tx.tx_hash));
assert!(tx_exists(&tx.tx_hash, tables.tx_ids()).unwrap());
let tx2 = get_tx(&tx.tx_hash, tables.tx_ids(), tables.tx_blobs()).unwrap();
assert_eq!(tx.tx_blob, tx2.serialize());
assert_eq!(tx.tx_weight, tx2.weight());
assert_eq!(tx.tx_hash, block.block.txs[i]);
assert_eq!(tx.tx_hash, tx2.hash());
}
}
block_hashes
};
{
let len = block_hashes.len();
let hashes: Vec<String> = block_hashes.iter().map(hex::encode).collect();
println!("block_hashes: len: {len}, hashes: {hashes:?}");
}
// Remove the blocks.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for block_hash in block_hashes.into_iter().rev() {
println!("pop_block(): block_hash: {}", hex::encode(block_hash));
let (popped_height, popped_hash, popped_block) = pop_block(&mut tables).unwrap();
assert_eq!(block_hash, popped_hash);
assert!(matches!(
get_block_extended_header(&block_hash, &tables),
Err(RuntimeError::KeyNotFound)
));
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
assert_all_tables_are_empty(&env);
}
/// We should panic if: `block.height` > `u32::MAX`
#[test]
#[should_panic(expected = "block.height (4294967296) > u32::MAX")]
fn block_height_gt_u32_max() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
let mut block = block_v9_tx3().clone();
block.height = u64::from(u32::MAX) + 1;
add_block(&block, &mut tables).unwrap();
}
/// We should panic if: `block.height` != the chain height
#[test]
#[should_panic(
expected = "assertion `left == right` failed: block.height (123) != chain_height (1)\n left: 123\n right: 1"
)]
fn block_height_not_chain_height() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
let mut block = block_v9_tx3().clone();
// HACK: `add_block()` asserts blocks with non-sequential heights
// cannot be added, to get around this, manually edit the block height.
block.height = 0;
// OK, `0 == 0`
assert_eq!(block.height, 0);
add_block(&block, &mut tables).unwrap();
// FAIL, `123 != 1`
block.height = 123;
add_block(&block, &mut tables).unwrap();
}
} }

View file

@ -1,9 +1,178 @@
//! Blockchain. //! Blockchain.
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use monero_serai::transaction::Timelock;
use cuprate_types::VerifiedBlockInformation;
use crate::{
database::{DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::doc_error,
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId, RctOutput},
};
//---------------------------------------------------------------------------------------------------- Free Functions //---------------------------------------------------------------------------------------------------- Free Functions
/// TODO /// Retrieve the height of the chain.
pub fn height() { ///
todo!() /// This returns the chain-tip, not the [`top_block_height`].
///
/// For example:
/// - The blockchain has 0 blocks => this returns `0`
/// - The blockchain has 1 block (height 0) => this returns `1`
/// - The blockchain has 2 blocks (height 1) => this returns `2`
///
/// So the height of a new block would be `chain_height()`.
#[doc = doc_error!()]
#[inline]
pub fn chain_height(
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<BlockHeight, RuntimeError> {
table_block_heights.len()
}
/// Retrieve the height of the top block.
///
/// This returns the height of the top block, not the [`chain_height`].
///
/// For example:
/// - The blockchain has 0 blocks => this returns `Err(RuntimeError::KeyNotFound)`
/// - The blockchain has 1 block (height 0) => this returns `Ok(0)`
/// - The blockchain has 2 blocks (height 1) => this returns `Ok(1)`
///
/// Note that in cases where no blocks have been written to the
/// database yet, an error is returned: `Err(RuntimeError::KeyNotFound)`.
///
#[doc = doc_error!()]
#[inline]
pub fn top_block_height(
table_block_heights: &impl DatabaseRo<BlockHeights>,
) -> Result<BlockHeight, RuntimeError> {
match table_block_heights.len()? {
0 => Err(RuntimeError::KeyNotFound),
height => Ok(height - 1),
}
}
/// Check how many cumulative generated coins there have been until a certain [`BlockHeight`].
///
/// This returns the total amount of Monero generated up to `block_height`
/// (including the block itself) in atomic units.
///
/// For example:
/// - on the genesis block `0`, this returns the amount block `0` generated
/// - on the next block `1`, this returns the amount block `0` and `1` generated
///
/// If no blocks have been added and `block_height == 0`
/// (i.e., the cumulative generated coins before genesis block is being calculated),
/// this returns `Ok(0)`.
#[doc = doc_error!()]
#[inline]
pub fn cumulative_generated_coins(
block_height: &BlockHeight,
table_block_infos: &impl DatabaseRo<BlockInfos>,
) -> Result<u64, RuntimeError> {
match table_block_infos.get(block_height) {
Ok(block_info) => Ok(block_info.cumulative_generated_coins),
Err(RuntimeError::KeyNotFound) if block_height == &0 => Ok(0),
Err(e) => Err(e),
}
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening)]
mod test {
use hex_literal::hex;
use pretty_assertions::assert_eq;
use cuprate_test_utils::data::{block_v16_tx0, block_v1_tx2, block_v9_tx3, tx_v2_rct3};
use super::*;
use crate::{
ops::{
block::add_block,
tx::{get_tx, tx_exists},
},
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
/// Tests all above functions.
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
#[allow(clippy::cognitive_complexity, clippy::cast_possible_truncation)]
fn all_blockchain_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let mut blocks = [
block_v1_tx2().clone(),
block_v9_tx3().clone(),
block_v16_tx0().clone(),
];
let blocks_len = u64::try_from(blocks.len()).unwrap();
// Add blocks.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
assert!(matches!(
top_block_height(tables.block_heights()),
Err(RuntimeError::KeyNotFound),
));
assert_eq!(
0,
cumulative_generated_coins(&0, tables.block_infos()).unwrap()
);
for (i, block) in blocks.iter_mut().enumerate() {
let i = u64::try_from(i).unwrap();
// HACK: `add_block()` asserts blocks with non-sequential heights
// cannot be added, to get around this, manually edit the block height.
block.height = i;
add_block(block, &mut tables).unwrap();
}
// Assert reads are correct.
assert_eq!(blocks_len, chain_height(tables.block_heights()).unwrap());
assert_eq!(
blocks_len - 1,
top_block_height(tables.block_heights()).unwrap()
);
assert_eq!(
cumulative_generated_coins(&0, tables.block_infos()).unwrap(),
13_138_270_467_918,
);
assert_eq!(
cumulative_generated_coins(&1, tables.block_infos()).unwrap(),
16_542_044_490_081,
);
assert_eq!(
cumulative_generated_coins(&2, tables.block_infos()).unwrap(),
17_142_044_490_081,
);
assert!(matches!(
cumulative_generated_coins(&3, tables.block_infos()),
Err(RuntimeError::KeyNotFound),
));
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
}
} }

View file

@ -0,0 +1,152 @@
//! Spent keys.
//---------------------------------------------------------------------------------------------------- Import
use monero_serai::transaction::{Timelock, Transaction};
use cuprate_types::{OutputOnChain, VerifiedBlockInformation};
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::{doc_add_block_inner_invariant, doc_error},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId, RctOutput, TxHash,
},
};
//---------------------------------------------------------------------------------------------------- Key image functions
/// Add a [`KeyImage`] to the "spent" set in the database.
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn add_key_image(
key_image: &KeyImage,
table_key_images: &mut impl DatabaseRw<KeyImages>,
) -> Result<(), RuntimeError> {
table_key_images.put(key_image, &())
}
/// Remove a [`KeyImage`] from the "spent" set in the database.
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn remove_key_image(
key_image: &KeyImage,
table_key_images: &mut impl DatabaseRw<KeyImages>,
) -> Result<(), RuntimeError> {
table_key_images.delete(key_image)
}
/// Check if a [`KeyImage`] exists - i.e. if it is "spent".
#[doc = doc_error!()]
#[inline]
pub fn key_image_exists(
key_image: &KeyImage,
table_key_images: &impl DatabaseRo<KeyImages>,
) -> Result<bool, RuntimeError> {
table_key_images.contains(key_image)
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening, clippy::cognitive_complexity)]
mod test {
use hex_literal::hex;
use pretty_assertions::assert_eq;
use super::*;
use crate::{
ops::tx::{get_tx, tx_exists},
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
/// Tests all above key-image functions.
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
fn all_key_image_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let key_images = [
hex!("be1c87fc8f958f68fbe346a18dfb314204dca7573f61aae14840b8037da5c286"),
hex!("c5e4a592c11f34a12e13516ab2883b7c580d47b286b8fe8b15d57d2a18ade275"),
hex!("93288b646f858edfb0997ae08d7c76f4599b04c127f108e8e69a0696ae7ba334"),
hex!("726e9e3d8f826d24811183f94ff53aeba766c9efe6274eb80806f69b06bfa3fc"),
];
// Add.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for key_image in &key_images {
println!("add_key_image(): {}", hex::encode(key_image));
add_key_image(key_image, tables.key_images_mut()).unwrap();
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
// Assert all reads are OK.
{
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
// Assert only the proper tables were added to.
assert_eq!(
tables.key_images().len().unwrap(),
u64::try_from(key_images.len()).unwrap()
);
assert_eq!(tables.block_infos().len().unwrap(), 0);
assert_eq!(tables.block_blobs().len().unwrap(), 0);
assert_eq!(tables.block_heights().len().unwrap(), 0);
assert_eq!(tables.num_outputs().len().unwrap(), 0);
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.outputs().len().unwrap(), 0);
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 0);
assert_eq!(tables.tx_blobs().len().unwrap(), 0);
assert_eq!(tables.tx_ids().len().unwrap(), 0);
assert_eq!(tables.tx_heights().len().unwrap(), 0);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 0);
for key_image in &key_images {
println!("key_image_exists(): {}", hex::encode(key_image));
key_image_exists(key_image, tables.key_images()).unwrap();
}
}
// Remove.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for key_image in key_images {
println!("remove_key_image(): {}", hex::encode(key_image));
remove_key_image(&key_image, tables.key_images_mut()).unwrap();
assert!(!key_image_exists(&key_image, tables.key_images()).unwrap());
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
assert_all_tables_are_empty(&env);
}
}

View file

@ -0,0 +1,33 @@
//! Macros.
//!
//! These generate repetitive documentation
//! for all the functions defined in `ops/`.
//---------------------------------------------------------------------------------------------------- Documentation macros
/// Generate documentation for the required `# Error` section.
macro_rules! doc_error {
() => {
r#"# Errors
This function returns [`RuntimeError::KeyNotFound`] if the input doesn't exist or other `RuntimeError`'s on database errors."#
};
}
pub(super) use doc_error;
/// Generate `# Invariant` documentation for internal `fn`'s
/// that should be called directly with caution.
macro_rules! doc_add_block_inner_invariant {
() => {
r#"# ⚠️ Invariant ⚠️
This function mainly exists to be used internally by the parent function [`crate::ops::block::add_block`].
`add_block()` makes sure all data related to the input is mutated, while
this function _does not_, it specifically mutates _particular_ tables.
This is usually undesired - although this function is still available to call directly.
When calling this function, ensure that either:
1. This effect (incomplete database mutation) is what is desired, or that...
2. ...the other tables will also be mutated to a correct state"#
};
}
pub(super) use doc_add_block_inner_invariant;

View file

@ -4,18 +4,48 @@
//! traits in this crate to generically call Monero-related //! traits in this crate to generically call Monero-related
//! database operations. //! database operations.
//! //!
//! # TODO //! # `impl Table`
//! TODO: These functions should pretty much map 1-1 to the `Request` enum. //! `ops/` functions take [`Tables`](crate::tables::Tables) and
//! [`TablesMut`](crate::tables::TablesMut) directly - these are
//! _already opened_ database tables.
//! //!
//! TODO: These are function names from `old_database/` for now. //! As such, the function puts the responsibility
//! The actual underlying functions (e.g `get()`) aren't implemented. //! of transactions, tables, etc on the caller.
//! //!
//! TODO: All of these functions need to take in generic //! This does mean these functions are mostly as lean
//! database trait parameters (and their actual inputs). //! as possible, so calling them in a loop should be okay.
//!
//! # Atomicity
//! As transactions are handled by the _caller_ of these functions,
//! it is up to the caller to decide what happens if one them return
//! an error.
//!
//! To maintain atomicity, transactions should be [`abort`](crate::transaction::TxRw::abort)ed
//! if one of the functions failed.
//!
//! For example, if [`add_block()`](block::add_block) is called and returns an [`Err`],
//! `abort`ing the transaction that opened the input `TableMut` would reverse all tables
//! mutated by `add_block()` up until the error, leaving it in the state it was in before
//! `add_block()` was called.
//!
//! # Sub-functions
//! The main functions within this module are mostly within the [`block`] module.
//!
//! Practically speaking, you should only be using 2 functions:
//! - [`add_block`](block::add_block)
//! - [`pop_block`](block::pop_block)
//!
//! The `block` functions are "parent" functions, calling other
//! sub-functions such as [`add_output()`](output::add_output). `add_output()`
//! itself only modifies output-related tables, while the `block` "parent" functions
//! (like `add_block` and `pop_block`) modify _everything_ that is required.
pub mod alt_block; // pub mod alt_block; // TODO: is this needed?
pub mod block; pub mod block;
pub mod blockchain;
pub mod key_image;
pub mod output; pub mod output;
pub mod property; pub mod property;
pub mod spent_key;
pub mod tx; pub mod tx;
mod macros;

View file

@ -1,34 +1,278 @@
//! Outputs. //! Outputs.
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use monero_serai::transaction::{Timelock, Transaction};
//---------------------------------------------------------------------------------------------------- Free Functions use cuprate_types::{OutputOnChain, VerifiedBlockInformation};
/// TODO
pub fn add_output() { use crate::{
todo!() database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::{doc_add_block_inner_invariant, doc_error},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
Amount, AmountIndex, BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId,
RctOutput, TxHash,
},
};
//---------------------------------------------------------------------------------------------------- Pre-RCT Outputs
/// Add a Pre-RCT [`Output`] to the database.
///
/// Upon [`Ok`], this function returns the [`PreRctOutputId`] that
/// can be used to lookup the `Output` in [`get_output()`].
///
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn add_output(
amount: Amount,
output: &Output,
tables: &mut impl TablesMut,
) -> Result<PreRctOutputId, RuntimeError> {
// FIXME: this would be much better expressed with a
// `btree_map::Entry`-like API, fix `trait DatabaseRw`.
let num_outputs = match tables.num_outputs().get(&amount) {
// Entry with `amount` already exists.
Ok(num_outputs) => num_outputs,
// Entry with `amount` didn't exist, this is
// the 1st output with this amount.
Err(RuntimeError::KeyNotFound) => 0,
Err(e) => return Err(e),
};
// Update the amount of outputs.
tables.num_outputs_mut().put(&amount, &(num_outputs + 1))?;
let pre_rct_output_id = PreRctOutputId {
amount,
// The new `amount_index` is the length of amount of outputs with same amount.
amount_index: num_outputs,
};
tables.outputs_mut().put(&pre_rct_output_id, output)?;
Ok(pre_rct_output_id)
} }
/// TODO /// Remove a Pre-RCT [`Output`] from the database.
pub fn remove_output() { #[doc = doc_add_block_inner_invariant!()]
todo!() #[doc = doc_error!()]
#[inline]
pub fn remove_output(
pre_rct_output_id: &PreRctOutputId,
tables: &mut impl TablesMut,
) -> Result<(), RuntimeError> {
// Decrement the amount index by 1, or delete the entry out-right.
// FIXME: this would be much better expressed with a
// `btree_map::Entry`-like API, fix `trait DatabaseRw`.
tables
.num_outputs_mut()
.update(&pre_rct_output_id.amount, |num_outputs| {
// INVARIANT: Should never be 0.
if num_outputs == 1 {
None
} else {
Some(num_outputs - 1)
}
})?;
// Delete the output data itself.
tables.outputs_mut().delete(pre_rct_output_id)
} }
/// TODO /// Retrieve a Pre-RCT [`Output`] from the database.
pub fn get_output() { #[doc = doc_error!()]
todo!() #[inline]
pub fn get_output(
pre_rct_output_id: &PreRctOutputId,
table_outputs: &impl DatabaseRo<Outputs>,
) -> Result<Output, RuntimeError> {
table_outputs.get(pre_rct_output_id)
} }
/// TODO /// How many pre-RCT [`Output`]s are there?
pub fn get_output_list() { ///
todo!() /// This returns the amount of pre-RCT outputs currently stored.
#[doc = doc_error!()]
#[inline]
pub fn get_num_outputs(table_outputs: &impl DatabaseRo<Outputs>) -> Result<u64, RuntimeError> {
table_outputs.len()
} }
/// TODO //---------------------------------------------------------------------------------------------------- RCT Outputs
pub fn get_rct_num_outputs() { /// Add an [`RctOutput`] to the database.
todo!() ///
/// Upon [`Ok`], this function returns the [`AmountIndex`] that
/// can be used to lookup the `RctOutput` in [`get_rct_output()`].
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
#[inline]
pub fn add_rct_output(
rct_output: &RctOutput,
table_rct_outputs: &mut impl DatabaseRw<RctOutputs>,
) -> Result<AmountIndex, RuntimeError> {
let amount_index = get_rct_num_outputs(table_rct_outputs)?;
table_rct_outputs.put(&amount_index, rct_output)?;
Ok(amount_index)
} }
/// TODO /// Remove an [`RctOutput`] from the database.
pub fn get_pre_rct_num_outputs() { #[doc = doc_add_block_inner_invariant!()]
todo!() #[doc = doc_error!()]
#[inline]
pub fn remove_rct_output(
amount_index: &AmountIndex,
table_rct_outputs: &mut impl DatabaseRw<RctOutputs>,
) -> Result<(), RuntimeError> {
table_rct_outputs.delete(amount_index)
}
/// Retrieve an [`RctOutput`] from the database.
#[doc = doc_error!()]
#[inline]
pub fn get_rct_output(
amount_index: &AmountIndex,
table_rct_outputs: &impl DatabaseRo<RctOutputs>,
) -> Result<RctOutput, RuntimeError> {
table_rct_outputs.get(amount_index)
}
/// How many [`RctOutput`]s are there?
///
/// This returns the amount of RCT outputs currently stored.
#[doc = doc_error!()]
#[inline]
pub fn get_rct_num_outputs(
table_rct_outputs: &impl DatabaseRo<RctOutputs>,
) -> Result<u64, RuntimeError> {
table_rct_outputs.len()
}
//---------------------------------------------------------------------------------------------------- Tests
#[cfg(test)]
#[allow(clippy::significant_drop_tightening, clippy::cognitive_complexity)]
mod test {
use super::*;
use crate::{
tests::{assert_all_tables_are_empty, tmp_concrete_env},
types::OutputFlags,
Env,
};
use cuprate_test_utils::data::{tx_v1_sig2, tx_v2_rct3};
use pretty_assertions::assert_eq;
/// Dummy `Output`.
const OUTPUT: Output = Output {
key: [44; 32],
height: 0,
output_flags: OutputFlags::NON_ZERO_UNLOCK_TIME,
tx_idx: 0,
};
/// Dummy `RctOutput`.
const RCT_OUTPUT: RctOutput = RctOutput {
key: [88; 32],
height: 1,
output_flags: OutputFlags::empty(),
tx_idx: 1,
commitment: [100; 32],
};
/// Dummy `Amount`
const AMOUNT: Amount = 22;
/// Tests all above output functions when only inputting `Output` data (no Block).
///
/// Note that this doesn't test the correctness of values added, as the
/// functions have a pre-condition that the caller handles this.
///
/// It simply tests if the proper tables are mutated, and if the data
/// stored and retrieved is the same.
#[test]
fn all_output_functions() {
let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
// Assert length is correct.
assert_eq!(get_num_outputs(tables.outputs()).unwrap(), 0);
assert_eq!(get_rct_num_outputs(tables.rct_outputs()).unwrap(), 0);
// Add outputs.
let pre_rct_output_id = add_output(AMOUNT, &OUTPUT, &mut tables).unwrap();
let amount_index = add_rct_output(&RCT_OUTPUT, tables.rct_outputs_mut()).unwrap();
assert_eq!(
pre_rct_output_id,
PreRctOutputId {
amount: AMOUNT,
amount_index: 0,
}
);
// Assert all reads of the outputs are OK.
{
// Assert proper tables were added to.
assert_eq!(tables.block_infos().len().unwrap(), 0);
assert_eq!(tables.block_blobs().len().unwrap(), 0);
assert_eq!(tables.block_heights().len().unwrap(), 0);
assert_eq!(tables.key_images().len().unwrap(), 0);
assert_eq!(tables.num_outputs().len().unwrap(), 1);
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.outputs().len().unwrap(), 1);
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 1);
assert_eq!(tables.tx_blobs().len().unwrap(), 0);
assert_eq!(tables.tx_ids().len().unwrap(), 0);
assert_eq!(tables.tx_heights().len().unwrap(), 0);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 0);
// Assert length is correct.
assert_eq!(get_num_outputs(tables.outputs()).unwrap(), 1);
assert_eq!(get_rct_num_outputs(tables.rct_outputs()).unwrap(), 1);
assert_eq!(1, tables.num_outputs().get(&AMOUNT).unwrap());
// Assert value is save after retrieval.
assert_eq!(
OUTPUT,
get_output(&pre_rct_output_id, tables.outputs()).unwrap(),
);
assert_eq!(
RCT_OUTPUT,
get_rct_output(&amount_index, tables.rct_outputs()).unwrap(),
);
}
// Remove the outputs.
{
remove_output(&pre_rct_output_id, &mut tables).unwrap();
remove_rct_output(&amount_index, tables.rct_outputs_mut()).unwrap();
// Assert value no longer exists.
assert!(matches!(
get_output(&pre_rct_output_id, tables.outputs()),
Err(RuntimeError::KeyNotFound)
));
assert!(matches!(
get_rct_output(&amount_index, tables.rct_outputs()),
Err(RuntimeError::KeyNotFound)
));
// Assert length is correct.
assert_eq!(get_num_outputs(tables.outputs()).unwrap(), 0);
assert_eq!(get_rct_num_outputs(tables.rct_outputs()).unwrap(), 0);
}
assert_all_tables_are_empty(&env);
}
} }

View file

@ -1,9 +1,57 @@
//! Properties. //! Properties.
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use monero_pruning::PruningSeed;
use monero_serai::transaction::{Timelock, Transaction};
use cuprate_types::{OutputOnChain, VerifiedBlockInformation};
use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::macros::{doc_add_block_inner_invariant, doc_error},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
BlockHash, BlockHeight, BlockInfo, KeyImage, Output, PreRctOutputId, RctOutput, TxHash,
TxId,
},
};
//---------------------------------------------------------------------------------------------------- Free Functions //---------------------------------------------------------------------------------------------------- Free Functions
/// TODO /// TODO
pub fn get_blockchain_pruning_seed() { ///
todo!() #[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
///
/// # Example
/// ```rust
/// # use cuprate_database::{*, tables::*, ops::block::*, ops::tx::*};
/// // TODO
/// ```
#[inline]
pub const fn get_blockchain_pruning_seed() -> Result<PruningSeed, RuntimeError> {
// TODO: impl pruning.
// We need a DB properties table.
Ok(PruningSeed::NotPruned)
}
/// TODO
///
#[doc = doc_add_block_inner_invariant!()]
#[doc = doc_error!()]
///
/// # Example
/// ```rust
/// # use cuprate_database::{*, tables::*, ops::block::*, ops::tx::*};
/// // TODO
/// ```
#[inline]
pub const fn db_version() -> Result<u64, RuntimeError> {
// TODO: We need a DB properties table.
Ok(crate::constants::DATABASE_VERSION)
} }

View file

@ -1,19 +0,0 @@
//! Spent keys.
//---------------------------------------------------------------------------------------------------- Import
//---------------------------------------------------------------------------------------------------- Free Functions
/// TODO
pub fn add_spent_key() {
todo!()
}
/// TODO
pub fn remove_spent_key() {
todo!()
}
/// TODO
pub fn is_spent_key_recorded() {
todo!()
}

View file

@ -1,64 +1,446 @@
//! Transactions. //! Transactions.
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use bytemuck::TransparentWrapper;
use curve25519_dalek::{constants::ED25519_BASEPOINT_POINT, Scalar};
use monero_serai::transaction::{Input, Timelock, Transaction};
//---------------------------------------------------------------------------------------------------- Free Functions use cuprate_types::{OutputOnChain, TransactionVerificationData, VerifiedBlockInformation};
/// TODO use monero_pruning::PruningSeed;
pub fn add_transaction() {
todo!() use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
env::EnvInner,
error::RuntimeError,
ops::{
blockchain::chain_height,
macros::{doc_add_block_inner_invariant, doc_error},
property::get_blockchain_pruning_seed,
},
tables::{
BlockBlobs, BlockHeights, BlockInfos, KeyImages, NumOutputs, Outputs, PrunableHashes,
PrunableTxBlobs, PrunedTxBlobs, RctOutputs, Tables, TablesMut, TxBlobs, TxHeights, TxIds,
TxUnlockTime,
},
transaction::{TxRo, TxRw},
types::{
AmountIndices, BlockHash, BlockHeight, BlockInfo, KeyImage, Output, OutputFlags,
PreRctOutputId, RctOutput, TxBlob, TxHash, TxId,
},
StorableVec,
};
use super::{
key_image::{add_key_image, remove_key_image},
output::{add_output, add_rct_output, get_rct_num_outputs, remove_output, remove_rct_output},
};
//---------------------------------------------------------------------------------------------------- Private
/// Add a [`TransactionVerificationData`] to the database.
///
/// The `block_height` is the block that this `tx` belongs to.
///
/// Note that the caller's input is trusted implicitly and no checks
/// are done (in this function) whether the `block_height` is correct or not.
///
#[doc = doc_add_block_inner_invariant!()]
///
/// # Notes
/// This function is different from other sub-functions and slightly more similar to
/// [`add_block()`](crate::ops::block::add_block) in that it calls other sub-functions.
///
/// This function calls:
/// - [`add_output()`]
/// - [`add_rct_output()`]
/// - [`add_key_image()`]
///
/// Thus, after [`add_tx`], those values (outputs and key images)
/// will be added to database tables as well.
///
/// # Panics
/// This function will panic if:
/// - `block.height > u32::MAX` (not normally possible)
#[doc = doc_error!()]
#[inline]
pub fn add_tx(
tx: &TransactionVerificationData,
block_height: &BlockHeight,
tables: &mut impl TablesMut,
) -> Result<TxId, RuntimeError> {
let tx_id = get_num_tx(tables.tx_ids_mut())?;
//------------------------------------------------------ Transaction data
tables.tx_ids_mut().put(&tx.tx_hash, &tx_id)?;
tables.tx_heights_mut().put(&tx_id, block_height)?;
tables
.tx_blobs_mut()
.put(&tx_id, StorableVec::wrap_ref(&tx.tx_blob))?;
//------------------------------------------------------ Timelocks
// Height/time is not differentiated via type, but rather:
// "height is any value less than 500_000_000 and timestamp is any value above"
// so the `u64/usize` is stored without any tag.
//
// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1558504285>
match tx.tx.prefix.timelock {
Timelock::None => (),
Timelock::Block(height) => tables.tx_unlock_time_mut().put(&tx_id, &(height as u64))?,
Timelock::Time(time) => tables.tx_unlock_time_mut().put(&tx_id, &time)?,
}
//------------------------------------------------------ Pruning
// SOMEDAY: implement pruning after `monero-serai` does.
// if let PruningSeed::Pruned(decompressed_pruning_seed) = get_blockchain_pruning_seed()? {
// SOMEDAY: what to store here? which table?
// }
//------------------------------------------------------
// Refer to the inner transaction type from now on.
let tx: &Transaction = &tx.tx;
let Ok(height) = u32::try_from(*block_height) else {
panic!("add_tx(): block_height ({block_height}) > u32::MAX");
};
//------------------------------------------------------ Key Images
// Is this a miner transaction?
// Which table we add the output data to depends on this.
// <https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/blockchain_db/blockchain_db.cpp#L212-L216>
let mut miner_tx = false;
// Key images.
for inputs in &tx.prefix.inputs {
match inputs {
// Key images.
Input::ToKey { key_image, .. } => {
add_key_image(key_image.compress().as_bytes(), tables.key_images_mut())?;
}
// This is a miner transaction, set it for later use.
Input::Gen(_) => miner_tx = true,
}
}
//------------------------------------------------------ Outputs
// Output bit flags.
// Set to a non-zero bit value if the unlock time is non-zero.
let output_flags = match tx.prefix.timelock {
Timelock::None => OutputFlags::empty(),
Timelock::Block(_) | Timelock::Time(_) => OutputFlags::NON_ZERO_UNLOCK_TIME,
};
let mut amount_indices = Vec::with_capacity(tx.prefix.outputs.len());
let tx_idx = get_num_tx(tables.tx_ids_mut())?;
for (i, output) in tx.prefix.outputs.iter().enumerate() {
let key = *output.key.as_bytes();
// Outputs with clear amounts.
let amount_index = if let Some(amount) = output.amount {
// RingCT (v2 transaction) miner outputs.
if miner_tx && tx.prefix.version == 2 {
// Create commitment.
// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1559489302>
// FIXME: implement lookup table for common values:
// <https://github.com/monero-project/monero/blob/c8214782fb2a769c57382a999eaf099691c836e7/src/ringct/rctOps.cpp#L322>
let commitment = (ED25519_BASEPOINT_POINT
+ monero_serai::H() * Scalar::from(amount))
.compress()
.to_bytes();
add_rct_output(
&RctOutput {
key,
height,
output_flags,
tx_idx,
commitment,
},
tables.rct_outputs_mut(),
)?
// Pre-RingCT outputs.
} else {
add_output(
amount,
&Output {
key,
height,
output_flags,
tx_idx,
},
tables,
)?
.amount_index
}
// RingCT outputs.
} else {
let commitment = tx.rct_signatures.base.commitments[i].compress().to_bytes();
add_rct_output(
&RctOutput {
key,
height,
output_flags,
tx_idx,
commitment,
},
tables.rct_outputs_mut(),
)?
};
amount_indices.push(amount_index);
} // for each output
tables
.tx_outputs_mut()
.put(&tx_id, &StorableVec(amount_indices))?;
Ok(tx_id)
} }
/// TODO /// Remove a transaction from the database with its [`TxHash`].
pub fn add_transaction_data() { ///
todo!() /// This returns the [`TxId`] and [`TxBlob`] of the removed transaction.
///
#[doc = doc_add_block_inner_invariant!()]
///
/// # Notes
/// As mentioned in [`add_tx`], this function will call other sub-functions:
/// - [`remove_output()`]
/// - [`remove_rct_output()`]
/// - [`remove_key_image()`]
///
/// Thus, after [`remove_tx`], those values (outputs and key images)
/// will be remove from database tables as well.
///
#[doc = doc_error!()]
#[inline]
pub fn remove_tx(
tx_hash: &TxHash,
tables: &mut impl TablesMut,
) -> Result<(TxId, Transaction), RuntimeError> {
//------------------------------------------------------ Transaction data
let tx_id = tables.tx_ids_mut().take(tx_hash)?;
let tx_blob = tables.tx_blobs_mut().take(&tx_id)?;
tables.tx_heights_mut().delete(&tx_id)?;
tables.tx_outputs_mut().delete(&tx_id)?;
//------------------------------------------------------ Pruning
// SOMEDAY: implement pruning after `monero-serai` does.
// table_prunable_hashes.delete(&tx_id)?;
// table_prunable_tx_blobs.delete(&tx_id)?;
// if let PruningSeed::Pruned(decompressed_pruning_seed) = get_blockchain_pruning_seed()? {
// SOMEDAY: what to remove here? which table?
// }
//------------------------------------------------------ Unlock Time
match tables.tx_unlock_time_mut().delete(&tx_id) {
Ok(()) | Err(RuntimeError::KeyNotFound) => (),
// An actual error occurred, return.
Err(e) => return Err(e),
}
//------------------------------------------------------
// Refer to the inner transaction type from now on.
let tx = Transaction::read(&mut tx_blob.0.as_slice())?;
//------------------------------------------------------ Key Images
// Is this a miner transaction?
let mut miner_tx = false;
for inputs in &tx.prefix.inputs {
match inputs {
// Key images.
Input::ToKey { key_image, .. } => {
remove_key_image(key_image.compress().as_bytes(), tables.key_images_mut())?;
}
// This is a miner transaction, set it for later use.
Input::Gen(_) => miner_tx = true,
}
} // for each input
//------------------------------------------------------ Outputs
// Remove each output in the transaction.
for (i, output) in tx.prefix.outputs.iter().enumerate() {
// Outputs with clear amounts.
if let Some(amount) = output.amount {
// RingCT miner outputs.
if miner_tx && tx.prefix.version == 2 {
let amount_index = get_rct_num_outputs(tables.rct_outputs())? - 1;
remove_rct_output(&amount_index, tables.rct_outputs_mut())?;
// Pre-RingCT outputs.
} else {
let amount_index = tables.num_outputs_mut().get(&amount)? - 1;
remove_output(
&PreRctOutputId {
amount,
amount_index,
},
tables,
)?;
}
// RingCT outputs.
} else {
let amount_index = get_rct_num_outputs(tables.rct_outputs())? - 1;
remove_rct_output(&amount_index, tables.rct_outputs_mut())?;
}
} // for each output
Ok((tx_id, tx))
} }
/// TODO //---------------------------------------------------------------------------------------------------- `get_tx_*`
pub fn remove_transaction() { /// Retrieve a [`Transaction`] from the database with its [`TxHash`].
todo!() #[doc = doc_error!()]
#[inline]
pub fn get_tx(
tx_hash: &TxHash,
table_tx_ids: &impl DatabaseRo<TxIds>,
table_tx_blobs: &impl DatabaseRo<TxBlobs>,
) -> Result<Transaction, RuntimeError> {
get_tx_from_id(&table_tx_ids.get(tx_hash)?, table_tx_blobs)
} }
/// TODO /// Retrieve a [`Transaction`] from the database with its [`TxId`].
pub fn remove_transaction_data() { #[doc = doc_error!()]
todo!() #[inline]
pub fn get_tx_from_id(
tx_id: &TxId,
table_tx_blobs: &impl DatabaseRo<TxBlobs>,
) -> Result<Transaction, RuntimeError> {
let tx_blob = table_tx_blobs.get(tx_id)?.0;
Ok(Transaction::read(&mut tx_blob.as_slice())?)
} }
/// TODO //----------------------------------------------------------------------------------------------------
pub fn remove_tx_outputs() { /// How many [`Transaction`]s are there?
todo!() ///
/// This returns the amount of transactions currently stored.
///
/// For example:
/// - 0 transactions exist => returns 0
/// - 1 transactions exist => returns 1
/// - 5 transactions exist => returns 5
/// - etc
#[doc = doc_error!()]
#[inline]
pub fn get_num_tx(table_tx_ids: &impl DatabaseRo<TxIds>) -> Result<u64, RuntimeError> {
table_tx_ids.len()
} }
/// TODO //----------------------------------------------------------------------------------------------------
pub fn get_num_tx() { /// Check if a transaction exists in the database.
todo!() ///
/// Returns `true` if it does, else `false`.
#[doc = doc_error!()]
#[inline]
pub fn tx_exists(
tx_hash: &TxHash,
table_tx_ids: &impl DatabaseRo<TxIds>,
) -> Result<bool, RuntimeError> {
table_tx_ids.contains(tx_hash)
} }
/// TODO //---------------------------------------------------------------------------------------------------- Tests
pub fn tx_exists() { #[cfg(test)]
todo!() #[allow(clippy::significant_drop_tightening)]
} mod test {
use super::*;
use crate::{
tests::{assert_all_tables_are_empty, tmp_concrete_env},
Env,
};
use cuprate_test_utils::data::{tx_v1_sig0, tx_v1_sig2, tx_v2_rct3};
use pretty_assertions::assert_eq;
/// TODO /// Tests all above tx functions when only inputting `Transaction` data (no Block).
pub fn get_tx_unlock_time() { #[test]
todo!() fn all_tx_functions() {
} let (env, tmp) = tmp_concrete_env();
let env_inner = env.env_inner();
assert_all_tables_are_empty(&env);
/// TODO // Monero `Transaction`, not database tx.
pub fn get_tx() { let txs = [tx_v1_sig0(), tx_v1_sig2(), tx_v2_rct3()];
todo!()
}
/// TODO // Add transactions.
pub fn get_tx_list() { let tx_ids = {
todo!() let tx_rw = env_inner.tx_rw().unwrap();
} let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
/// TODO let tx_ids = txs
pub fn get_pruned_tx() { .iter()
todo!() .map(|tx| {
} println!("add_tx(): {tx:#?}");
add_tx(tx, &0, &mut tables).unwrap()
})
.collect::<Vec<TxId>>();
/// TODO drop(tables);
pub fn get_tx_block_height() { TxRw::commit(tx_rw).unwrap();
todo!()
tx_ids
};
// Assert all reads of the transactions are OK.
let tx_hashes = {
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
// Assert only the proper tables were added to.
assert_eq!(tables.block_infos().len().unwrap(), 0);
assert_eq!(tables.block_blobs().len().unwrap(), 0);
assert_eq!(tables.block_heights().len().unwrap(), 0);
assert_eq!(tables.key_images().len().unwrap(), 4); // added to key images
assert_eq!(tables.pruned_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.prunable_hashes().len().unwrap(), 0);
assert_eq!(tables.num_outputs().len().unwrap(), 9);
assert_eq!(tables.outputs().len().unwrap(), 10); // added to outputs
assert_eq!(tables.prunable_tx_blobs().len().unwrap(), 0);
assert_eq!(tables.rct_outputs().len().unwrap(), 2);
assert_eq!(tables.tx_blobs().len().unwrap(), 3);
assert_eq!(tables.tx_ids().len().unwrap(), 3);
assert_eq!(tables.tx_heights().len().unwrap(), 3);
assert_eq!(tables.tx_unlock_time().len().unwrap(), 1); // only 1 has a timelock
// Both from ID and hash should result in getting the same transaction.
let mut tx_hashes = vec![];
for (i, tx_id) in tx_ids.iter().enumerate() {
println!("tx_ids.iter(): i: {i}, tx_id: {tx_id}");
let tx_get_from_id = get_tx_from_id(tx_id, tables.tx_blobs()).unwrap();
let tx_hash = tx_get_from_id.hash();
let tx_get = get_tx(&tx_hash, tables.tx_ids(), tables.tx_blobs()).unwrap();
println!("tx_ids.iter(): tx_get_from_id: {tx_get_from_id:#?}, tx_get: {tx_get:#?}");
assert_eq!(tx_get_from_id.hash(), tx_get.hash());
assert_eq!(tx_get_from_id.hash(), txs[i].tx_hash);
assert_eq!(tx_get_from_id, tx_get);
assert_eq!(tx_get, txs[i].tx);
assert!(tx_exists(&tx_hash, tables.tx_ids()).unwrap());
tx_hashes.push(tx_hash);
}
tx_hashes
};
// Remove the transactions.
{
let tx_rw = env_inner.tx_rw().unwrap();
let mut tables = env_inner.open_tables_mut(&tx_rw).unwrap();
for tx_hash in tx_hashes {
println!("remove_tx(): tx_hash: {tx_hash:?}");
let (tx_id, _) = remove_tx(&tx_hash, &mut tables).unwrap();
assert!(matches!(
get_tx_from_id(&tx_id, tables.tx_blobs()),
Err(RuntimeError::KeyNotFound)
));
}
drop(tables);
TxRw::commit(tx_rw).unwrap();
}
assert_all_tables_are_empty(&env);
}
} }

View file

@ -4,19 +4,17 @@
//---------------------------------------------------------------------------------------------------- Import //---------------------------------------------------------------------------------------------------- Import
use crate::{ use crate::{
database::{DatabaseIter, DatabaseRo, DatabaseRw},
table::Table, table::Table,
types::{ types::{
Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfoV1, Amount, AmountIndex, AmountIndices, BlockBlob, BlockHash, BlockHeight, BlockInfo, KeyImage,
BlockInfoV2, BlockInfoV3, KeyImage, Output, PreRctOutputId, PrunableBlob, PrunableHash, Output, PreRctOutputId, PrunableBlob, PrunableHash, PrunedBlob, RctOutput, TxBlob, TxHash,
PrunedBlob, RctOutput, TxHash, TxId, UnlockTime, TxId, UnlockTime,
}, },
}; };
//---------------------------------------------------------------------------------------------------- Tables //---------------------------------------------------------------------------------------------------- Sealed
/// Private module, should not be accessible outside this crate. /// Private module, should not be accessible outside this crate.
///
/// Used to block outsiders implementing [`Table`].
/// All [`Table`] types must also implement [`Sealed`].
pub(super) mod private { pub(super) mod private {
/// Private sealed trait. /// Private sealed trait.
/// ///
@ -24,6 +22,255 @@ pub(super) mod private {
pub trait Sealed {} pub trait Sealed {}
} }
//---------------------------------------------------------------------------------------------------- `trait Tables[Mut]`
/// Creates:
/// - `pub trait Tables`
/// - `pub trait TablesMut`
/// - Blanket implementation for `(tuples, containing, all, open, database, tables, ...)`
///
/// For why this exists, see: <https://github.com/Cuprate/cuprate/pull/102#pullrequestreview-1978348871>.
macro_rules! define_trait_tables {
($(
// The `T: Table` type The index in a tuple
// | containing all tables
// v v
$table:ident => $index:literal
),* $(,)?) => { paste::paste! {
/// Object containing all opened [`Table`]s in read-only mode.
///
/// This is an encapsulated object that contains all
/// available [`Table`]'s in read-only mode.
///
/// It is a `Sealed` trait and is only implemented on a
/// `(tuple, containing, all, table, types, ...)`.
///
/// This is used to return a _single_ object from functions like
/// [`EnvInner::open_tables`](crate::EnvInner::open_tables) rather
/// than the tuple containing the tables itself.
///
/// To replace `tuple.0` style indexing, `field_accessor_functions()`
/// are provided on this trait, which essentially map the object to
/// fields containing the particular database table, for example:
/// ```rust,ignore
/// let tables = open_tables();
///
/// // The accessor function `block_info_v1s()` returns the field
/// // containing an open database table for `BlockInfoV1s`.
/// let _ = tables.block_info_v1s();
/// ```
pub trait Tables: private::Sealed {
// This expands to creating `fn field_accessor_functions()`
// for each passed `$table` type.
//
// It is essentially a mapping to the field
// containing the proper opened database table.
//
// The function name of the function is
// the table type in `snake_case`, e.g., `block_info_v1s()`.
$(
/// Access an opened
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake>](&self) -> &impl DatabaseRo<$table>;
)*
/// This returns `true` if all tables are empty.
///
/// # Errors
/// This returns errors on regular database errors.
fn all_tables_empty(&self) -> Result<bool, $crate::error::RuntimeError>;
}
/// Object containing all opened [`Table`]s in read + iter mode.
///
/// This is the same as [`Tables`] but includes `_iter()` variants.
///
/// See [`Tables`] for documentation - this trait exists for the same reasons.
pub trait TablesIter: private::Sealed + Tables {
$(
/// Access an opened read-only + iterable
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake _iter>](&self) -> &(impl DatabaseRo<$table> + DatabaseIter<$table>);
)*
}
/// Object containing all opened [`Table`]s in write mode.
///
/// This is the same as [`Tables`] but for mutable accesses.
///
/// See [`Tables`] for documentation - this trait exists for the same reasons.
pub trait TablesMut: private::Sealed + Tables {
$(
/// Access an opened
#[doc = concat!("[`", stringify!($table), "`]")]
/// database.
fn [<$table:snake _mut>](&mut self) -> &mut impl DatabaseRw<$table>;
)*
}
// Implement `Sealed` for all table types.
impl<$([<$table:upper>]),*> private::Sealed for ($([<$table:upper>]),*) {}
// This creates a blanket-implementation for
// `(tuple, containing, all, table, types)`.
//
// There is a generic defined here _for each_ `$table` input.
// Specifically, the generic letters are just the table types in UPPERCASE.
// Concretely, this expands to something like:
// ```rust
// impl<BLOCKINFOSV1S, BLOCKINFOSV2S, BLOCKINFOSV3S, [...]>
// ```
impl<$([<$table:upper>]),*> Tables
// We are implementing `Tables` on a tuple that
// contains all those generics specified, i.e.,
// a tuple containing all open table types.
//
// Concretely, this expands to something like:
// ```rust
// (BLOCKINFOSV1S, BLOCKINFOSV2S, BLOCKINFOSV3S, [...])
// ```
// which is just a tuple of the generics defined above.
for ($([<$table:upper>]),*)
where
// This expands to a where bound that asserts each element
// in the tuple implements some database table type.
//
// Concretely, this expands to something like:
// ```rust
// BLOCKINFOSV1S: DatabaseRo<BlockInfoV1s> + DatabaseIter<BlockInfoV1s>,
// BLOCKINFOSV2S: DatabaseRo<BlockInfoV2s> + DatabaseIter<BlockInfoV2s>,
// [...]
// ```
$(
[<$table:upper>]: DatabaseRo<$table>,
)*
{
$(
// The function name of the accessor function is
// the table type in `snake_case`, e.g., `block_info_v1s()`.
#[inline]
fn [<$table:snake>](&self) -> &impl DatabaseRo<$table> {
// The index of the database table in
// the tuple implements the table trait.
&self.$index
}
)*
fn all_tables_empty(&self) -> Result<bool, $crate::error::RuntimeError> {
$(
if !DatabaseRo::is_empty(&self.$index)? {
return Ok(false);
}
)*
Ok(true)
}
}
// This is the same as the above
// `Tables`, but for `TablesIter`.
impl<$([<$table:upper>]),*> TablesIter
for ($([<$table:upper>]),*)
where
$(
[<$table:upper>]: DatabaseRo<$table> + DatabaseIter<$table>,
)*
{
$(
// The function name of the accessor function is
// the table type in `snake_case` + `_iter`, e.g., `block_info_v1s_iter()`.
#[inline]
fn [<$table:snake _iter>](&self) -> &(impl DatabaseRo<$table> + DatabaseIter<$table>) {
&self.$index
}
)*
}
// This is the same as the above
// `Tables`, but for `TablesMut`.
impl<$([<$table:upper>]),*> TablesMut
for ($([<$table:upper>]),*)
where
$(
[<$table:upper>]: DatabaseRw<$table>,
)*
{
$(
// The function name of the mutable accessor function is
// the table type in `snake_case` + `_mut`, e.g., `block_info_v1s_mut()`.
#[inline]
fn [<$table:snake _mut>](&mut self) -> &mut impl DatabaseRw<$table> {
&mut self.$index
}
)*
}
}};
}
// Format: $table_type => $index
//
// The $index:
// - Simply increments by 1 for each table
// - Must be 0..
// - Must end at the total amount of table types
//
// Compile errors will occur if these aren't satisfied.
define_trait_tables! {
BlockInfos => 0,
BlockBlobs => 1,
BlockHeights => 2,
KeyImages => 3,
NumOutputs => 4,
PrunedTxBlobs => 5,
PrunableHashes => 6,
Outputs => 7,
PrunableTxBlobs => 8,
RctOutputs => 9,
TxBlobs => 10,
TxIds => 11,
TxHeights => 12,
TxOutputs => 13,
TxUnlockTime => 14,
}
//---------------------------------------------------------------------------------------------------- Table function macro
/// `crate`-private macro for callings functions on all tables.
///
/// This calls the function `$fn` with the optional
/// arguments `$args` on all tables - returning early
/// (within whatever scope this is called) if any
/// of the function calls error.
///
/// Else, it evaluates to an `Ok((tuple, of, all, table, types, ...))`,
/// i.e., an `impl Table[Mut]` wrapped in `Ok`.
macro_rules! call_fn_on_all_tables_or_early_return {
(
$($fn:ident $(::)?)*
(
$($arg:ident),* $(,)?
)
) => {{
Ok((
$($fn ::)*<$crate::tables::BlockInfos>($($arg),*)?,
$($fn ::)*<$crate::tables::BlockBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::BlockHeights>($($arg),*)?,
$($fn ::)*<$crate::tables::KeyImages>($($arg),*)?,
$($fn ::)*<$crate::tables::NumOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunedTxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunableHashes>($($arg),*)?,
$($fn ::)*<$crate::tables::Outputs>($($arg),*)?,
$($fn ::)*<$crate::tables::PrunableTxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::RctOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxBlobs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxIds>($($arg),*)?,
$($fn ::)*<$crate::tables::TxHeights>($($arg),*)?,
$($fn ::)*<$crate::tables::TxOutputs>($($arg),*)?,
$($fn ::)*<$crate::tables::TxUnlockTime>($($arg),*)?,
))
}};
}
pub(crate) use call_fn_on_all_tables_or_early_return;
//---------------------------------------------------------------------------------------------------- Table macro //---------------------------------------------------------------------------------------------------- Table macro
/// Create all tables, should be used _once_. /// Create all tables, should be used _once_.
/// ///
@ -80,8 +327,10 @@ macro_rules! tables {
// Notes: // Notes:
// - Keep this sorted A-Z (by table name) // - Keep this sorted A-Z (by table name)
// - Tables are defined in plural to avoid name conflicts with types // - Tables are defined in plural to avoid name conflicts with types
// - If adding/changing a table, also edit the tests in `src/backend/tests.rs` // - If adding/changing a table also edit:
// and edit `Env::open` to make sure it creates the table // a) the tests in `src/backend/tests.rs`
// b) `Env::open` to make sure it creates the table (for all backends)
// c) `call_fn_on_all_tables_or_early_return!()` macro defined in this file
tables! { tables! {
/// TODO /// TODO
BlockBlobs, BlockBlobs,
@ -92,22 +341,17 @@ tables! {
BlockHash => BlockHeight, BlockHash => BlockHeight,
/// TODO /// TODO
BlockInfoV1s, BlockInfos,
BlockHeight => BlockInfoV1, BlockHeight => BlockInfo,
/// TODO
BlockInfoV2s,
BlockHeight => BlockInfoV2,
/// TODO
BlockInfoV3s,
BlockHeight => BlockInfoV3,
/// TODO /// TODO
KeyImages, KeyImages,
KeyImage => (), KeyImage => (),
/// TODO /// Maps an output's amount to the number of outputs with that amount.
///
/// For a new output the `AmountIndex` value from this
/// table will be its index in a list of duplicate outputs.
NumOutputs, NumOutputs,
Amount => AmountIndex, Amount => AmountIndex,
@ -119,18 +363,28 @@ tables! {
Outputs, Outputs,
PreRctOutputId => Output, PreRctOutputId => Output,
/// TODO // SOMEDAY: impl when `monero-serai` supports pruning
PrunableTxBlobs, PrunableTxBlobs,
TxId => PrunableBlob, TxId => PrunableBlob,
/// TODO // SOMEDAY: impl when `monero-serai` supports pruning
PrunableHashes, PrunableHashes,
TxId => PrunableHash, TxId => PrunableHash,
// SOMEDAY: impl a properties table:
// - db version
// - pruning seed
// Properties,
// StorableString => StorableVec,
/// TODO /// TODO
RctOutputs, RctOutputs,
AmountIndex => RctOutput, AmountIndex => RctOutput,
/// SOMEDAY: remove when `monero-serai` supports pruning
TxBlobs,
TxId => TxBlob,
/// TODO /// TODO
TxIds, TxIds,
TxHash => TxId, TxHash => TxId,
@ -139,6 +393,10 @@ tables! {
TxHeights, TxHeights,
TxId => BlockHeight, TxId => BlockHeight,
/// TODO
TxOutputs,
TxId => AmountIndices,
/// TODO /// TODO
TxUnlockTime, TxUnlockTime,
TxId => UnlockTime, TxId => UnlockTime,

45
database/src/tests.rs Normal file
View file

@ -0,0 +1,45 @@
//! Utilities for `cuprate_database` testing.
//!
//! These types/fn's are only:
//! - enabled on #[cfg(test)]
//! - only used internally
#![allow(clippy::significant_drop_tightening)]
//---------------------------------------------------------------------------------------------------- Import
use std::{
fmt::Debug,
sync::{Arc, OnceLock},
};
use monero_serai::{
ringct::{RctPrunable, RctSignatures},
transaction::{Timelock, Transaction, TransactionPrefix},
};
use crate::{
config::Config, key::Key, storable::Storable, tables::Tables, transaction::TxRo, ConcreteEnv,
Env, EnvInner,
};
//---------------------------------------------------------------------------------------------------- fn
/// Create an `Env` in a temporarily directory.
/// The directory is automatically removed after the `TempDir` is dropped.
///
/// FIXME: changing this to `-> impl Env` causes lifetime errors...
pub(crate) fn tmp_concrete_env() -> (ConcreteEnv, tempfile::TempDir) {
let tempdir = tempfile::tempdir().unwrap();
let config = Config::low_power(Some(tempdir.path().into()));
let env = ConcreteEnv::open(config).unwrap();
(env, tempdir)
}
/// Assert all the tables in the environment are empty.
pub(crate) fn assert_all_tables_are_empty(env: &ConcreteEnv) {
let env_inner = env.env_inner();
let tx_ro = env_inner.tx_ro().unwrap();
let tables = env_inner.open_tables(&tx_ro).unwrap();
assert!(tables.all_tables_empty().unwrap());
assert_eq!(crate::ops::tx::get_num_tx(tables.tx_ids()).unwrap(), 0);
}

View file

@ -82,6 +82,9 @@ pub type PrunableBlob = StorableVec<u8>;
/// TODO /// TODO
pub type PrunableHash = [u8; 32]; pub type PrunableHash = [u8; 32];
/// TODO
pub type TxBlob = StorableVec<u8>;
/// TODO /// TODO
pub type TxId = u64; pub type TxId = u64;
@ -124,96 +127,6 @@ pub struct PreRctOutputId {
pub amount_index: AmountIndex, pub amount_index: AmountIndex,
} }
//---------------------------------------------------------------------------------------------------- BlockInfoV1
/// TODO
///
/// ```rust
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = BlockInfoV1 {
/// timestamp: 1,
/// total_generated_coins: 123,
/// weight: 321,
/// cumulative_difficulty: 111,
/// block_hash: [54; 32],
/// };
/// let b = Storable::as_bytes(&a);
/// let c: BlockInfoV1 = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
/// # Size & Alignment
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<BlockInfoV1>(), 64);
/// assert_eq!(align_of::<BlockInfoV1>(), 8);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(C)]
pub struct BlockInfoV1 {
/// TODO
pub timestamp: u64,
/// TODO
pub total_generated_coins: u64,
/// TODO
pub weight: u64,
/// TODO
pub cumulative_difficulty: u64,
/// TODO
pub block_hash: [u8; 32],
}
//---------------------------------------------------------------------------------------------------- BlockInfoV2
/// TODO
///
/// ```rust
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = BlockInfoV2 {
/// timestamp: 1,
/// total_generated_coins: 123,
/// weight: 321,
/// block_hash: [54; 32],
/// cumulative_difficulty: 111,
/// cumulative_rct_outs: 2389,
/// };
/// let b = Storable::as_bytes(&a);
/// let c: BlockInfoV2 = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
/// # Size & Alignment
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<BlockInfoV2>(), 72);
/// assert_eq!(align_of::<BlockInfoV2>(), 8);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(C)]
pub struct BlockInfoV2 {
/// TODO
pub timestamp: u64,
/// TODO
pub total_generated_coins: u64,
/// TODO
pub weight: u64,
/// TODO
pub block_hash: [u8; 32],
/// TODO
pub cumulative_difficulty: u64,
/// TODO
///
/// TODO: note that this is originally u32,
/// but is u64 here for padding reasons.
pub cumulative_rct_outs: u64,
}
//---------------------------------------------------------------------------------------------------- BlockInfoV3 //---------------------------------------------------------------------------------------------------- BlockInfoV3
/// TODO /// TODO
/// ///
@ -221,18 +134,17 @@ pub struct BlockInfoV2 {
/// # use std::borrow::*; /// # use std::borrow::*;
/// # use cuprate_database::{*, types::*}; /// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct. /// // Assert Storable is correct.
/// let a = BlockInfoV3 { /// let a = BlockInfo {
/// timestamp: 1, /// timestamp: 1,
/// total_generated_coins: 123, /// cumulative_generated_coins: 123,
/// weight: 321, /// weight: 321,
/// cumulative_difficulty_low: 111, /// cumulative_difficulty: 112,
/// cumulative_difficulty_high: 112,
/// block_hash: [54; 32], /// block_hash: [54; 32],
/// cumulative_rct_outs: 2389, /// cumulative_rct_outs: 2389,
/// long_term_weight: 2389, /// long_term_weight: 2389,
/// }; /// };
/// let b = Storable::as_bytes(&a); /// let b = Storable::as_bytes(&a);
/// let c: BlockInfoV3 = Storable::from_bytes(b); /// let c: BlockInfo = Storable::from_bytes(b);
/// assert_eq!(a, c); /// assert_eq!(a, c);
/// ``` /// ```
/// ///
@ -240,24 +152,21 @@ pub struct BlockInfoV2 {
/// ```rust /// ```rust
/// # use cuprate_database::types::*; /// # use cuprate_database::types::*;
/// # use std::mem::*; /// # use std::mem::*;
/// assert_eq!(size_of::<BlockInfoV3>(), 88); /// assert_eq!(size_of::<BlockInfo>(), 88);
/// assert_eq!(align_of::<BlockInfoV3>(), 8); /// assert_eq!(align_of::<BlockInfo>(), 8);
/// ``` /// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))] #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)] #[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(C)] #[repr(C)]
pub struct BlockInfoV3 { pub struct BlockInfo {
/// TODO /// TODO
pub timestamp: u64, pub timestamp: u64,
/// TODO /// TODO
pub total_generated_coins: u64, pub cumulative_generated_coins: u64,
/// TODO /// TODO
pub weight: u64, pub weight: u64,
// Maintain 8 byte alignment.
/// TODO /// TODO
pub cumulative_difficulty_low: u64, pub cumulative_difficulty: u128,
/// TODO
pub cumulative_difficulty_high: u64,
/// TODO /// TODO
pub block_hash: [u8; 32], pub block_hash: [u8; 32],
/// TODO /// TODO
@ -266,6 +175,36 @@ pub struct BlockInfoV3 {
pub long_term_weight: u64, pub long_term_weight: u64,
} }
//---------------------------------------------------------------------------------------------------- OutputFlags
bitflags::bitflags! {
/// TODO
///
/// ```rust
/// # use std::borrow::*;
/// # use cuprate_database::{*, types::*};
/// // Assert Storable is correct.
/// let a = OutputFlags::NON_ZERO_UNLOCK_TIME;
/// let b = Storable::as_bytes(&a);
/// let c: OutputFlags = Storable::from_bytes(b);
/// assert_eq!(a, c);
/// ```
///
/// # Size & Alignment
/// ```rust
/// # use cuprate_database::types::*;
/// # use std::mem::*;
/// assert_eq!(size_of::<OutputFlags>(), 4);
/// assert_eq!(align_of::<OutputFlags>(), 4);
/// ```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd, Eq, Ord, Hash, Pod, Zeroable)]
#[repr(transparent)]
pub struct OutputFlags: u32 {
/// This output has a non-zero unlock time.
const NON_ZERO_UNLOCK_TIME = 0b0000_0001;
}
}
//---------------------------------------------------------------------------------------------------- Output //---------------------------------------------------------------------------------------------------- Output
/// TODO /// TODO
/// ///
@ -276,7 +215,7 @@ pub struct BlockInfoV3 {
/// let a = Output { /// let a = Output {
/// key: [1; 32], /// key: [1; 32],
/// height: 1, /// height: 1,
/// output_flags: 0, /// output_flags: OutputFlags::empty(),
/// tx_idx: 3, /// tx_idx: 3,
/// }; /// };
/// let b = Storable::as_bytes(&a); /// let b = Storable::as_bytes(&a);
@ -300,7 +239,7 @@ pub struct Output {
/// We could get this from the tx_idx with the Tx Heights table but that would require another look up per out. /// We could get this from the tx_idx with the Tx Heights table but that would require another look up per out.
pub height: u32, pub height: u32,
/// Bit flags for this output, currently only the first bit is used and, if set, it means this output has a non-zero unlock time. /// Bit flags for this output, currently only the first bit is used and, if set, it means this output has a non-zero unlock time.
pub output_flags: u32, pub output_flags: OutputFlags,
/// TODO /// TODO
pub tx_idx: u64, pub tx_idx: u64,
} }
@ -315,7 +254,7 @@ pub struct Output {
/// let a = RctOutput { /// let a = RctOutput {
/// key: [1; 32], /// key: [1; 32],
/// height: 1, /// height: 1,
/// output_flags: 0, /// output_flags: OutputFlags::empty(),
/// tx_idx: 3, /// tx_idx: 3,
/// commitment: [3; 32], /// commitment: [3; 32],
/// }; /// };
@ -340,7 +279,7 @@ pub struct RctOutput {
/// We could get this from the tx_idx with the Tx Heights table but that would require another look up per out. /// We could get this from the tx_idx with the Tx Heights table but that would require another look up per out.
pub height: u32, pub height: u32,
/// Bit flags for this output, currently only the first bit is used and, if set, it means this output has a non-zero unlock time. /// Bit flags for this output, currently only the first bit is used and, if set, it means this output has a non-zero unlock time.
pub output_flags: u32, pub output_flags: OutputFlags,
/// TODO /// TODO
pub tx_idx: u64, pub tx_idx: u64,
/// The amount commitment of this output. /// The amount commitment of this output.

View file

@ -27,8 +27,8 @@ use crate::data::constants::{
/// this struct represents that data that must be provided. /// this struct represents that data that must be provided.
/// ///
/// Consider using `cuprate_test_utils::rpc` to get this data easily. /// Consider using `cuprate_test_utils::rpc` to get this data easily.
struct VerifiedBlockMap<'a> { struct VerifiedBlockMap {
block: Block, block_blob: &'static [u8],
pow_hash: [u8; 32], pow_hash: [u8; 32],
height: u64, height: u64,
generated_coins: u64, generated_coins: u64,
@ -37,10 +37,10 @@ struct VerifiedBlockMap<'a> {
cumulative_difficulty: u128, cumulative_difficulty: u128,
// Vec of `tx_blob`'s, i.e. the data in `/test-utils/src/data/tx/`. // Vec of `tx_blob`'s, i.e. the data in `/test-utils/src/data/tx/`.
// This should the actual `tx_blob`'s of the transactions within this block. // This should the actual `tx_blob`'s of the transactions within this block.
txs: Vec<&'a [u8]>, txs: &'static [&'static [u8]],
} }
impl VerifiedBlockMap<'_> { impl VerifiedBlockMap {
/// Turn the various static data bits in `self` into a `VerifiedBlockInformation`. /// Turn the various static data bits in `self` into a `VerifiedBlockInformation`.
/// ///
/// Transactions are verified that they at least match the block's, /// Transactions are verified that they at least match the block's,
@ -48,7 +48,7 @@ impl VerifiedBlockMap<'_> {
/// is not checked. /// is not checked.
fn into_verified(self) -> VerifiedBlockInformation { fn into_verified(self) -> VerifiedBlockInformation {
let Self { let Self {
block, block_blob,
pow_hash, pow_hash,
height, height,
generated_coins, generated_coins,
@ -58,8 +58,11 @@ impl VerifiedBlockMap<'_> {
txs, txs,
} = self; } = self;
let block_blob = block_blob.to_vec();
let block = Block::read(&mut block_blob.as_slice()).unwrap();
let txs: Vec<Arc<TransactionVerificationData>> = txs let txs: Vec<Arc<TransactionVerificationData>> = txs
.into_iter() .iter()
.map(to_tx_verification_data) .map(to_tx_verification_data)
.map(Arc::new) .map(Arc::new)
.collect(); .collect();
@ -79,6 +82,7 @@ impl VerifiedBlockMap<'_> {
VerifiedBlockInformation { VerifiedBlockInformation {
block_hash: block.hash(), block_hash: block.hash(),
block_blob,
block, block,
txs, txs,
pow_hash, pow_hash,
@ -92,8 +96,8 @@ impl VerifiedBlockMap<'_> {
} }
// Same as [`VerifiedBlockMap`] but for [`TransactionVerificationData`]. // Same as [`VerifiedBlockMap`] but for [`TransactionVerificationData`].
fn to_tx_verification_data(tx_blob: &[u8]) -> TransactionVerificationData { fn to_tx_verification_data(tx_blob: impl AsRef<[u8]>) -> TransactionVerificationData {
let tx_blob = tx_blob.to_vec(); let tx_blob = tx_blob.as_ref().to_vec();
let tx = Transaction::read(&mut tx_blob.as_slice()).unwrap(); let tx = Transaction::read(&mut tx_blob.as_slice()).unwrap();
TransactionVerificationData { TransactionVerificationData {
tx_weight: tx.weight(), tx_weight: tx.weight(),
@ -158,14 +162,14 @@ macro_rules! verified_block_information_fn {
static BLOCK: OnceLock<VerifiedBlockInformation> = OnceLock::new(); static BLOCK: OnceLock<VerifiedBlockInformation> = OnceLock::new();
BLOCK.get_or_init(|| { BLOCK.get_or_init(|| {
VerifiedBlockMap { VerifiedBlockMap {
block: Block::read(&mut $block_blob).unwrap(), block_blob: $block_blob,
pow_hash: hex!($pow_hash), pow_hash: hex!($pow_hash),
height: $height, height: $height,
generated_coins: $generated_coins, generated_coins: $generated_coins,
weight: $weight, weight: $weight,
long_term_weight: $long_term_weight, long_term_weight: $long_term_weight,
cumulative_difficulty: $cumulative_difficulty, cumulative_difficulty: $cumulative_difficulty,
txs: vec![$($tx_blob),*], txs: &[$($tx_blob),*],
} }
.into_verified() .into_verified()
}) })

View file

@ -102,9 +102,10 @@ impl HttpRpcClient {
let reward = result.block_header.reward; let reward = result.block_header.reward;
let (block_hash, block) = spawn_blocking(|| { let (block_hash, block_blob, block) = spawn_blocking(|| {
let block = Block::read(&mut hex::decode(result.blob).unwrap().as_slice()).unwrap(); let block_blob = hex::decode(result.blob).unwrap();
(block.hash(), block) let block = Block::read(&mut block_blob.as_slice()).unwrap();
(block.hash(), block_blob, block)
}) })
.await .await
.unwrap(); .unwrap();
@ -139,6 +140,7 @@ impl HttpRpcClient {
VerifiedBlockInformation { VerifiedBlockInformation {
block, block,
block_blob,
txs, txs,
block_hash, block_hash,
pow_hash, pow_hash,

View file

@ -78,6 +78,10 @@ pub struct VerifiedBlockInformation {
pub long_term_weight: usize, pub long_term_weight: usize,
/// TODO /// TODO
pub cumulative_difficulty: u128, pub cumulative_difficulty: u128,
/// TODO
/// <https://github.com/Cuprate/cuprate/pull/102#discussion_r1556694072>
/// <https://github.com/serai-dex/serai/blob/93be7a30674ecedfb325b6d09dc22d550d7c13f8/coins/monero/src/block.rs#L110>
pub block_blob: Vec<u8>,
} }
//---------------------------------------------------------------------------------------------------- OutputOnChain //---------------------------------------------------------------------------------------------------- OutputOnChain