Commit graph

21 commits

Author SHA1 Message Date
hinto.janai
e15b7d42c0
pop_blocks 2024-09-10 21:00:05 -04:00
hinto.janai
2ac2b7e2a5
add BlockchainResponse::PopBlocks 2024-09-10 20:53:19 -04:00
hinto.janai
1aca03d319
blockchain: add KeyImageSpent request 2024-09-09 18:10:29 -04:00
hinto.janai
2d52c26bdc
get_block_header_by_height, use anyhow!() 2024-09-08 18:03:47 -04:00
hinto.janai
b8656961f3
add BlockchainReadRequest::Block[ByHash] 2024-09-06 20:45:25 -04:00
hinto.janai
51fd017c5b
blockchain: add BlockchainReadRequest::BlockExtendedHeaderByHash 2024-09-06 17:16:13 -04:00
hinto-janai
eead49beb0
lints: opt in manual lint crates (#263)
* cargo.toml: transfer existing lints

* rpc/interface: lints

* rpc/json-rpc: lints

* rpc/types: lints

* storage/blockchain: lints

* rpc/types: fix lints

* cargo.toml: fix lint group priority

* storage/blockchain: fix lints

* fix misc lints

* storage/database: fixes

* storage/txpool: opt in lints + fixes

* types: opt in + fixes

* helper: opt in + fixes

* types: remove borsh

* rpc/interface: fix test

* test fixes

* database: fix lints

* fix lint

* tabs -> spaces

* blockchain: `config/` -> `config.rs`
2024-09-02 18:12:54 +01:00
fdd1689665
Storage: tx-pool database (#238)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* split the DB service abstraction

* fix ci

* misc changes

* init tx-pool DBs

* add some comments

* move more types to `/types`

* add some ops

* add config & more ops functions & open function

* add read & write svcs

* add more docs

* add write functions + docs

* fix merge

* fix test

* fix ci

* move `TxPoolWriteError`

* add more docs

* fix toml formatting

* fix some docs

* fix clippy

* review fixes

* update docs

* fix merge

* fix docs

* fix tests

* fix tests

* add back lints

* Update storage/txpool/README.md

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-08-22 02:09:07 +01:00
be2f3f2672
Consensus: move more types to types (#250)
* move `HardFork` to `types`

* fmt

* fix tests & doc

* fmt

* fix clippy

* move transaction verification data

* misc fixes

* doc fixes

* update README.md

* review fixes
2024-08-09 00:56:13 +01:00
8227c28604
update monero-serai (#201)
* update monero-serai

* update monero-serai + change height to `usize`

* fix merge

* fix merge

* fix doc

* fix clippy take 2

* misc changes

* move RPC imports to dev deps

* handle miner txs when calculating fee

* Update consensus/rules/src/blocks.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update consensus/rules/src/transactions.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update storage/blockchain/src/ops/tx.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update test-utils/src/data/free.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* fixes

* fix clippy

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-08-07 00:48:53 +01:00
1a178381dd
Storage: split the DB service abstraction (#237)
* split the DB service abstraction

* fix ci

* misc changes

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* review fixes

* Update storage/service/Cargo.toml

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update storage/service/Cargo.toml

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* fix clippy

* fix doc

* `bc` -> `blockchain`

* doc fixes

* Update storage/service/README.md

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* cargo fmt

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-08-05 21:47:30 +01:00
b44c6b045b
Consensus: add alt-chain handling (#214)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* add `pop_blocks` to the difficulty cache

* add a rolling median struct

* use RollingMedian in weight cache

* add pop_blocks to weight cache

* add alt context cache

* add getting alt RX vms

* rework alt cache

* add alt block verify function

* keep alt caches around

* add alt checked alt blocks to the cache

* check the alt blocks timestamp

* add docs + cleanup code

* add popping blocks from the context cache

* finish popping blocks + fix tests

* fix doc

* add a test popping blocks from HF cache

* add a request to clear alt caches

* add back lint

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* review fixes

* small changes

* change panic doc

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-07-29 01:13:08 +01:00
hinto-janai
929d19c450
rpc: custom epee for misc/bin types (#229)
* fixed-bytes: add `serde`, document feature flags

* fixed-bytes: add derives

* rpc: add `as _` syntax to macro

* rpc: use `ByteArrayVec` and `ContainerAsBlob` for binary types

* fixed-bytes: re-add derives

* rpc-types: dedup default value within macro

* readme: fixed bytes section

* types: custom epee - `BlockCompleteEntry`

* types: custom epee - `KeyImageSpentStatus`

* types: custom epee - `PoolInfoExtent`

* types: add `Status::Other(String)` variant

* types: custom epee - `TxEntry`, add `read_epee_field` macro

* bin: custom epee - `GetBlocks`

* types: add `serde.rs`

* misc: make `TxEntry` an `enum`, impl serde

* misc: `unimplemented!()` for `TxEntry`'s epee

* types: add `BlockCompleteEntry`

* rpc: replace `BlockCompleteEntry` with `cuprate-types`

* types: document `BlockCompleteEntry`

* bin: fix `number_of_fields` for `GetBlocksResponse`

* misc: add `Distribution`

* distribution: add todo

* misc fixes

* readme: add `(De)serialization invariants`

* distribution: compress variants

* types: add `block_complete_entry.rs`

* net: fix imports

* p2p: fix imports

* turn off default-features

* p2p: fix imports

* misc fixes

* Update net/wire/Cargo.toml

Co-authored-by: Boog900 <boog900@tutanota.com>

* distribution: module doc

* wire: re-export types

* bin: use enum for `GetBlocksResponse`

* misc: use lowercase for stringify

* remove duplicated fields for custom epee

* types: remove `should_write()` for custom epee

* bin: split `GetBlocksResponse` variant fields into structs

* misc: split `Distribution` variant fields into structs

* small fixes

* put all fields in `read_epee_field!`

* distribution: (de)compress during epee/serde (de)serialization

* distribution: leave (de)compression functions as `todo!()`

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-07-25 16:46:41 +01:00
hinto-janai
6820da9848
types: add BlockCompleteEntry (#230)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* types: add `block_complete_entry.rs`

* net: fix imports

* p2p: fix imports

* turn off default-features

* p2p: fix imports

* Update net/wire/Cargo.toml

Co-authored-by: Boog900 <boog900@tutanota.com>

* wire: re-export types

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-07-16 22:47:50 +01:00
hinto-janai
71131a4836
ci: fix doc.yml (#213)
Some checks are pending
Audit / audit (push) Waiting to run
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Deny / audit (push) Waiting to run
Doc / build (push) Waiting to run
Doc / deploy (push) Blocked by required conditions
* types: remove borsh/serde

* blockchain: re-add optional serde

* doc.yml: remove `-D warnings`
2024-07-04 15:52:51 +01:00
7c8466f4ba
Storage: add blockchain history requests (#206)
Some checks failed
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Audit / audit (push) Has been cancelled
Deny / audit (push) Has been cancelled
* Add database requests for chain history

* misc fixes

* review comments

* fix clippy

* add link and fix typo

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* add comment

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-07-02 23:08:19 +01:00
0622237d19
Consensus: use cuprate-types types (#145)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
* move consensus database to /types

* fix `storage` builds

* unify `VerifiedBlockInformation`

* fix docs

* change `Database` trait wording

* order imports

* service -> blockchain

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* fix typo

* fix key_images_spent

* add back todo

* fix tests

* service -> blockchain 2

* update docs

* update docs 2

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-06-04 18:19:35 +01:00
hinto-janai
fb3d41ccbb
database: final docs + cleanup (#117)
* re-apply 'main' merge + doc patches

* fix redb lints

* update readme

* add `lib.rs, ops, service` doc-test examples

* docs for `config`, `ops`, add doc-tests

* remove merge error

incorrect leftover code from previous merge

* doc top-level types

* docs: error, tables, types

* misc docs, TODO, FIXME, SOMEDAY fixes

* change clippy lints

* tests: add `tables_are_sorted()`

* move `tables_are_sorted()` test to `backend/tests.rs`

* readme formatting

* small fixes

* readme fixes

* docs: `helper/`

* docs: `types/`

* database/README.md fixes

* doc fixes

* types: doc fixes

* fixes

* all review changes
2024-05-05 15:21:28 +01:00
hinto-janai
c65eb0a3ca
database: implement ops/ (#102)
* ops: add `trait MoneroR{o,w}`

* update `trait MoneroR{o,w}` bounds

* types: add `BlockInfoLatest` type alias

* block: impl most core functions

* types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916

* fix table type test

* cargo.toml: add `{cuprate-types, monero-serai}`

* add_block: add all other block data

* ops: remove unneeded `block` functions

* env: add `EnvInner::open_db_rw_all()`

* types: fix test

* block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()`

* add `trait Tables[Mut]` and use it in `EnvInner`

* block: use `TablesMut`

* tables: replace manual impl with `define_trait_tables!()`

* tables: docs for `trait Tables[Mut]`

* tables: doc functions

* create `call_fn_on_all_tables_or_early_return!()` macro

* block: cleanup signatures + bodies

* block: more fn's, docs

* block: add `doc_{error,single,bulk}!()`

* remove `ops/monero.rs`

* move `height()` to `ops/blockchain.rs`

* add `ops/macros.rs`

* tx: add fn signatures

* output: fix fn signatures

* ops: expose `_inner()` functions

* block: add `add_block_header{_bulk, _inner}()`

* ops: remove doc_{fn,inner}!()`

* ops: remove `_{inner,bulk}()`, lifetime + generics

* update lib/mod docs

* ops: add and use `doc_add_block_inner_invariant!()`

* ops: add docs/return to inner `add_block()` functions

* add_block(): extract and use fn for {key_image, output}

* ops: more fn body impl + `add_block()`

* cargo: add `monero-pruning`

* ops: extract out `tx` functions from `add_block()`

* property: add `db_version()`

* ops: `pop_block()` body, remove other `pop_block` fn's

* types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation`

* block: put `block_blob`, pass `Tables` to sub-functions

`impl TablesMut` can't mutably pass multiple tables since
it takes `&mut self`, so all functions unfortunately have to
take a full `&mut impl TablesMut` even though they only need
a few tables.

* database: add `DatabaseRw::take()`

useful for `pop_block()` where we need the value afterwards

* block: deserialize tx's from `block_blobs` in `pop_block()`

* blockchain: `height()` -> `chain_height()`

* output: fix `amount_index`

* ops: fix unlock_time, chain_height

* `BlockInfoV{1,2,3}` -> `BlockInfo`

* constants: add `DATABASE_VERSION`

* database: add `DatabaseRw::update()`

* output: use `DatabaseRw::update()` in `remove_output()`

* add `TxBlobs` table, ignore pruning tables

* block: mostly impl `add_block()` body

* ops: comments

* add_block: miner v2 tx commitment, height cast

* block: impl `pop_block()`

* block: mostly impl `get_block()`

* block: impl `get_block_{from_height,header,header_from_height}`

* add `OutputFlags` bitflags

* add_block: u32::try_into(height: u32), use `OutputFlags`

* tx: impl `get_{tx,tx_from_id}()`

* tx: move docs tests to `#[test]`

testing everything in 1 go is more natural since e.g:
`add_tx()` is followed by `get_tx()`

* tables: add `trait TablesIter`, `all_tables_empty()`

This allows `TablesMut` to be a superset of `Tables`
and use all its accessor functions.

* use cuprate-test-utils, fix tx tests

* block: `add_block()` take block by ref

* tx: use all txs in tests

* output: add `all_tx_functions()` test

* add_block: check current height against input

* block: map more fields in `get_block()`

* block: remove `get_block()`, doc tests, fix `get_block_header()`

* block: dummy values in test

* heed: use `last/first()` instead of `unsafe` cursors

We no longer have DUP semantics and also hard to debug errors
were popping up on `del_current()`...

* heed: fix `DatabaseRw::delete`

Ok(true) means the key did not exist, so we must return
Err(RuntimeError::KeyNotFound)

* block: `add_block()` (dummy value) test

* ops: `key_image` tests

* cleanup, docs, tests

* backend: test `take()` & `update()`

* docs

* remove `OutputFlags::NONE`

* add_block(): add asserts, panic docs, `should_panic` tests

* backend: remove `Ok(())` in `Database::delete` if already deleted

redb already does this, so heed so match

* block: move block operations after tx/outputs

* `amount == 0` -> `amount == 1`

* Nit: StorableVec::wrap_ref

* `saturating_sub(1)` -> `- 1`

* add `TxOutputs` table

* add_block(): add to `tx_outputs` table

* fix `DatabaseRo::update`

* add_tx(): take `block_height` as input

* tx: add/remove from `TxOutputs` table

* output: remove if `amount == 1` -> `amount_index == 0`

* output: fix `add_output()`'s `amount_index` calculation

* output: fix `add_output()`'s `amount_index` calculation again

* output: tests for `amount_index/num_outputs`

* block: `num_outputs - 1` and `take()` -> `get()`

We don't need to `take()` since the call afterwards to
`remove_output()` removes the entry

* block: swap `get_block_extended_header[_from_height]()`

* move `{key_image,output}` handling `add_block()` -> `add_tx()`

* blockchain: add doc to `top_block_height()`

* block: manual panic -> `assert_eq!()`

* test-utils: add `block_blob` to `VerifiedBlockInformation`

field introduced in this PR

* ops: use real block/tx data in tests

* block: `total_generated_coins` -> `cumulative_generated_coins`

* fix clippy, docs, TODOs

* `cumulative_generated_coins()`: `block/` -> `blockchain/`

* blockchain: add `cumulative_generated_coins()` tests

* Update database/src/ops/block.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* `cumulative_generated_coins()` docs for pre-block-0 special case

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 02:53:29 +01:00
hinto-janai
ad7b750d76
database: impl service::{Request,Response} mappings (#101)
* add `cuprate-types`

* remove `cuprate_database::service::{request,response}`

* use `cuprate_types::service::{request,response}`

* service: fix `Request` `match`'s

* service: create `ReadRequest` function mappings

* service: create `WriteRequest` function mappings

* service: add rough `WriteRequest` retry loop

* service: handle `RuntimeError::ResizeNeeded` in writer

* add `{R,r}o` exception to typos

* docs

* env: make `resize_map()` return new memory map byte size

* write: proactively handle resizes

`add_block()` takes `VerifiedBlockInformation` such
that it can just take the inner blobs of data.

This is a problem when reactively resizing since we no longer
have the block struct we just gave away so we're forced to `.clone()`
each retry.

Instead of that - we will proactively resize so the resize error
will never occur in the first place.

* read: use type aliases

* docs

* fix import

* write: handle resizes reactively

* service: panic if response can't be sent back

* write: add loop unreachable asserts

* service: print and drop error instead of panic

* write: fix retry loop off-by-1

* write: fix docs

* review changes

* update readme

* remove `BlockBatchInRange` request/response

* Update database/README.md

Co-authored-by: Boog900 <boog900@tutanota.com>

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-16 23:05:38 +01:00
hinto-janai
d503548716
Implement types/ (#94)
* workspace: add `bytemuck` to workspace

* add `types/`

* workspace: add `cuprate-types` to members

* copy `consensus/` types to `types/`

* remove `hard_fork/`

* extended_block_header: impl `Pod`, fix layout

* update `Request/Response`

* impl types

* fix `Response/Request`

* impl `borsh`

* workspace: add `strum`

* service: add `strum` traits

* remove `paste`, `serde_json`, `thiserror`

* remove `strum`

* VerifiedBlockInformation: remove `hf_vote`

* Update Cargo.toml

Co-authored-by: Boog900 <boog900@tutanota.com>

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-03-27 00:46:32 +00:00