* impl async buffer
* clippy
* p2p changes
* clippy
* a few more docs
* init cuprate-p2p
* remove some unrelated code and add some docs
* start documenting client_pool.rs
* add more docs
* typo
* fix docs
* use JoinSet in connection maintainer
* small changes
* add peer sync state svc
* add broadcast svc
* add more docs
* add some tests
* add a test
* fix merge
* add another test
* unify PeerDisconnectFut and add more docs
* start network init
* add an inbound connection server
* remove crate doc for now
* fix address book docs
* fix leak in client pool
* correct comment
* fix merge + add some docs
* review comments
* init block downloader
* fix doc
* initial chain search
* add chain_tracker
* move block downloader to struct
* spawn task whe getting blocks
* check for free peers and handle batch response
* add test bin
* working block downloader
* dynamic batch sizes
* dandelion_tower -> dandelion-tower
* fix async-buffer builds
* check if incoming peers are banned
* add interface methods
* update docs
* use a JoinSet for background network tasks
* dynamic batch size changes
* Keep a longer of queue of blocks to get
* more checks on incoming data
* fix merge
* fix imports
* add more docs
* add some limits on messages
* keep peers that dont have the current need data
* fix clippy
* fix .lock
* fix stopping the block downloader
* clean up API and add more docs
* tracing + bug fixes
* fix panic
* doc changes
* remove test_init
* remove spammy log
* fix previous merge
* add a test
* fix test
* remove test unwrap
* order imports correctly
* clean up test
* add a timeout
* fix tests
* review fixes
* make `BlockDownloader` pub
* make `initial_chain_search` pub
* make `block_downloader` private
* Apply suggestions from code review
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
* split some sections into separate modules
* split chain requests
* sort imports
* check previous ID is correct
* fix typos
* Apply suggestions from code review
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
---------
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
* change monero-consensus to cuprate-consensus-rules
* document the context service
* remove the mutex on blockchain context
* comment the context caches
* add back tokio
* document block checks
* typo
* keep tha amount of outputs with a certain amount in the output cache
* typo
* nuke cross-block batch verification
* remove RPC scanner
* change how contextual data is got.
* fmt & clippy fixes
* typo
* cargo update
* restore Cargo.lock
* add a verify tx test.
+ fixes an issue with verifying signatures after BPs
* clippy
* remove bad test
* add mores tests and fix a couple bugs
* typos
* move tests and add some more
* typo
* remove scan_chain docs
* fix check for duplicate txs when duplicates are not sequential
* add a proptest for dup txs
* cache tx verification state
* doc updates + move `Vec` to `Arc<[]>`
* clippy
* misc changes
* Apply suggestions from code review
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
Co-authored-by: SyntheticBird <118022351+SyntheticBird45@users.noreply.github.com>
* fix fmt
* review changes
---------
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
Co-authored-by: SyntheticBird <118022351+SyntheticBird45@users.noreply.github.com>
* `database/` -> `storage/`, create skeleton `storage/` crates
* add skeleton root directories
* books: add skeleton
* ci: fix labeler.yml
* fix CONTRIBUTING.md
* fix workspace Cargo.toml
* fix `storage/`
* rename helper fs fn, `cuprate_database` -> `cuprate_blockchain`
* fix Cargo.toml
* cuprate-blockchain: revert find-and-replace
Since it has to be ported to the book anyway, there's not much
reason to update this and create review diffs.
* labeler.yml: remove `A-docs` from `books/` changes
Although books are documentation,
the `A-books` label already exists for this
* cargo: add `pretty_assertions`
* test-utils: add tests to compare data against RPC data, and fix
* small fixes
* database: fix `cumulative_generated_coins` from test data
block had wrong data which offset all of the asserts
* ops: add `trait MoneroR{o,w}`
* update `trait MoneroR{o,w}` bounds
* types: add `BlockInfoLatest` type alias
* block: impl most core functions
* types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916
* fix table type test
* cargo.toml: add `{cuprate-types, monero-serai}`
* add_block: add all other block data
* ops: remove unneeded `block` functions
* env: add `EnvInner::open_db_rw_all()`
* types: fix test
* block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()`
* add `trait Tables[Mut]` and use it in `EnvInner`
* block: use `TablesMut`
* tables: replace manual impl with `define_trait_tables!()`
* tables: docs for `trait Tables[Mut]`
* tables: doc functions
* create `call_fn_on_all_tables_or_early_return!()` macro
* block: cleanup signatures + bodies
* block: more fn's, docs
* block: add `doc_{error,single,bulk}!()`
* remove `ops/monero.rs`
* move `height()` to `ops/blockchain.rs`
* add `ops/macros.rs`
* tx: add fn signatures
* output: fix fn signatures
* ops: expose `_inner()` functions
* block: add `add_block_header{_bulk, _inner}()`
* ops: remove doc_{fn,inner}!()`
* ops: remove `_{inner,bulk}()`, lifetime + generics
* update lib/mod docs
* ops: add and use `doc_add_block_inner_invariant!()`
* ops: add docs/return to inner `add_block()` functions
* add_block(): extract and use fn for {key_image, output}
* ops: more fn body impl + `add_block()`
* cargo: add `monero-pruning`
* ops: extract out `tx` functions from `add_block()`
* property: add `db_version()`
* ops: `pop_block()` body, remove other `pop_block` fn's
* types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation`
* block: put `block_blob`, pass `Tables` to sub-functions
`impl TablesMut` can't mutably pass multiple tables since
it takes `&mut self`, so all functions unfortunately have to
take a full `&mut impl TablesMut` even though they only need
a few tables.
* database: add `DatabaseRw::take()`
useful for `pop_block()` where we need the value afterwards
* block: deserialize tx's from `block_blobs` in `pop_block()`
* blockchain: `height()` -> `chain_height()`
* output: fix `amount_index`
* ops: fix unlock_time, chain_height
* `BlockInfoV{1,2,3}` -> `BlockInfo`
* constants: add `DATABASE_VERSION`
* database: add `DatabaseRw::update()`
* output: use `DatabaseRw::update()` in `remove_output()`
* add `TxBlobs` table, ignore pruning tables
* block: mostly impl `add_block()` body
* ops: comments
* add_block: miner v2 tx commitment, height cast
* block: impl `pop_block()`
* block: mostly impl `get_block()`
* block: impl `get_block_{from_height,header,header_from_height}`
* add `OutputFlags` bitflags
* add_block: u32::try_into(height: u32), use `OutputFlags`
* tx: impl `get_{tx,tx_from_id}()`
* tx: move docs tests to `#[test]`
testing everything in 1 go is more natural since e.g:
`add_tx()` is followed by `get_tx()`
* tables: add `trait TablesIter`, `all_tables_empty()`
This allows `TablesMut` to be a superset of `Tables`
and use all its accessor functions.
* use cuprate-test-utils, fix tx tests
* block: `add_block()` take block by ref
* tx: use all txs in tests
* output: add `all_tx_functions()` test
* add_block: check current height against input
* block: map more fields in `get_block()`
* block: remove `get_block()`, doc tests, fix `get_block_header()`
* block: dummy values in test
* heed: use `last/first()` instead of `unsafe` cursors
We no longer have DUP semantics and also hard to debug errors
were popping up on `del_current()`...
* heed: fix `DatabaseRw::delete`
Ok(true) means the key did not exist, so we must return
Err(RuntimeError::KeyNotFound)
* block: `add_block()` (dummy value) test
* ops: `key_image` tests
* cleanup, docs, tests
* backend: test `take()` & `update()`
* docs
* remove `OutputFlags::NONE`
* add_block(): add asserts, panic docs, `should_panic` tests
* backend: remove `Ok(())` in `Database::delete` if already deleted
redb already does this, so heed so match
* block: move block operations after tx/outputs
* `amount == 0` -> `amount == 1`
* Nit: StorableVec::wrap_ref
* `saturating_sub(1)` -> `- 1`
* add `TxOutputs` table
* add_block(): add to `tx_outputs` table
* fix `DatabaseRo::update`
* add_tx(): take `block_height` as input
* tx: add/remove from `TxOutputs` table
* output: remove if `amount == 1` -> `amount_index == 0`
* output: fix `add_output()`'s `amount_index` calculation
* output: fix `add_output()`'s `amount_index` calculation again
* output: tests for `amount_index/num_outputs`
* block: `num_outputs - 1` and `take()` -> `get()`
We don't need to `take()` since the call afterwards to
`remove_output()` removes the entry
* block: swap `get_block_extended_header[_from_height]()`
* move `{key_image,output}` handling `add_block()` -> `add_tx()`
* blockchain: add doc to `top_block_height()`
* block: manual panic -> `assert_eq!()`
* test-utils: add `block_blob` to `VerifiedBlockInformation`
field introduced in this PR
* ops: use real block/tx data in tests
* block: `total_generated_coins` -> `cumulative_generated_coins`
* fix clippy, docs, TODOs
* `cumulative_generated_coins()`: `block/` -> `blockchain/`
* blockchain: add `cumulative_generated_coins()` tests
* Update database/src/ops/block.rs
Co-authored-by: Boog900 <boog900@tutanota.com>
* `cumulative_generated_coins()` docs for pre-block-0 special case
---------
Co-authored-by: Boog900 <boog900@tutanota.com>
* add `cuprate-types`
* remove `cuprate_database::service::{request,response}`
* use `cuprate_types::service::{request,response}`
* service: fix `Request` `match`'s
* service: create `ReadRequest` function mappings
* service: create `WriteRequest` function mappings
* service: add rough `WriteRequest` retry loop
* service: handle `RuntimeError::ResizeNeeded` in writer
* add `{R,r}o` exception to typos
* docs
* env: make `resize_map()` return new memory map byte size
* write: proactively handle resizes
`add_block()` takes `VerifiedBlockInformation` such
that it can just take the inner blobs of data.
This is a problem when reactively resizing since we no longer
have the block struct we just gave away so we're forced to `.clone()`
each retry.
Instead of that - we will proactively resize so the resize error
will never occur in the first place.
* read: use type aliases
* docs
* fix import
* write: handle resizes reactively
* service: panic if response can't be sent back
* write: add loop unreachable asserts
* service: print and drop error instead of panic
* write: fix retry loop off-by-1
* write: fix docs
* review changes
* update readme
* remove `BlockBatchInRange` request/response
* Update database/README.md
Co-authored-by: Boog900 <boog900@tutanota.com>
---------
Co-authored-by: Boog900 <boog900@tutanota.com>
* use tokio's delay queue for bans
* document handles
* remove peers from address book when retrieving
* ping inbound peers during handshakes
* support receiving pings during handshakes
* add peer to anchor before reducing whit list
* clippy
* comment handshakes
* typos
* sort `use`
* use `rand::prelude::*`
* review comments
* update macro
* levin: fix fragmented messages & use bitflags
* levin: add a method to fragment a message
* levin: add tests for fragmented messages and fix issues
* fix docs
* tests: don't include bytes length
* levin: add support for sending fragmented
/ dummy messages
* fmt
* add fragmented handshake tests.
* fix handshake detection when fragmented
and alt (non-monero) protocol info
* add tracing logs
* remove `already_built`, this was an old way I was thinking of sending raw buckets
* clippy
* clippy 2
* Update net/levin/src/message.rs
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
* review comments
* add timeout to tests
* Update net/levin/src/header.rs
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
---------
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
* monerod: set a random DB and remove dir when done
* add a test for an inbound monerod handshake
* don't fail if can't remove dir on windows
* Update test-utils/src/monerod.rs
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
* use `temp_dir`
* use `tempfile`
---------
Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
