82 commits

Author	SHA1	Message	Date
Boog900	be2f3f2672	Consensus: move more types to types (#250) ... * move `HardFork` to `types` * fmt * fix tests & doc * fmt * fix clippy * move transaction verification data * misc fixes * doc fixes * update README.md * review fixes	2024-08-09 00:56:13 +01:00
Boog900	8227c28604	update monero-serai (#201) ... * update monero-serai * update monero-serai + change height to `usize` * fix merge * fix merge * fix doc * fix clippy take 2 * misc changes * move RPC imports to dev deps * handle miner txs when calculating fee * Update consensus/rules/src/blocks.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * Update consensus/rules/src/transactions.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * Update storage/blockchain/src/ops/tx.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * Update test-utils/src/data/free.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * fixes * fix clippy --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-08-07 00:48:53 +01:00
hinto-janai	27767690ca	rpc: impl cuprate-rpc-interface (#233) ... * fixed-bytes: add `serde`, document feature flags * fixed-bytes: add derives * rpc: add `as _` syntax to macro * rpc: use `ByteArrayVec` and `ContainerAsBlob` for binary types * fixed-bytes: re-add derives * rpc-types: dedup default value within macro * readme: fixed bytes section * types: custom epee - `BlockCompleteEntry` * types: custom epee - `KeyImageSpentStatus` * types: custom epee - `PoolInfoExtent` * types: add `Status::Other(String)` variant * types: custom epee - `TxEntry`, add `read_epee_field` macro * bin: custom epee - `GetBlocks` * types: add `serde.rs` * misc: make `TxEntry` an `enum`, impl serde * misc: `unimplemented!()` for `TxEntry`'s epee * types: add `BlockCompleteEntry` * rpc: replace `BlockCompleteEntry` with `cuprate-types` * types: document `BlockCompleteEntry` * bin: fix `number_of_fields` for `GetBlocksResponse` * misc: add `Distribution` * distribution: add todo * misc fixes * readme: add `(De)serialization invariants` * distribution: compress variants * types: add `block_complete_entry.rs` * net: fix imports * p2p: fix imports * turn off default-features * p2p: fix imports * misc fixes * Update net/wire/Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> * distribution: module doc * wire: re-export types * test-utils: add `crate::rpc::types` module * test-utils: conditional json doc-tests * bin: use enum for `GetBlocksResponse` * misc: use lowercase for stringify * json: add test data, fix macro doc tests * json: add all data * other: add all data * bin: add skeleton * docs * move type to correct file * remove duplicated fields for custom epee * rpc: `client/{client,constants}.rs` -> `client.rs` * lib.rs: remove `clippy::module_inception` * macros: add json doc test macro * json: add some tests * json: add doc-test for all types * add all other JSON doc-tests * move doc-test macros to files * base: add doc-tests * rpc: add `cuprate-rpc-interface` skeleton files * traits * json_rpc_method: add `.is_restricted()` * add route fn signatures * types: add rpc enums * interface: routes, types * interface: simplify routes * rewrite interface fns * types: remove `()` type alias, add `(restricted)` * types: add `other::InPeers` * interface: routes * types: fix `is_restricted()` * interface: reorder short-circuit bool * clean up traits/bounds * types: remove `axum` feature * interface: cleanup unused imports * interface: call handler in routes * json: TODO distribution test * interface: readme intro * combine `RpcHandler` + `RpcService`, add `RpcDummyHandler` * interface: readme docs + test * `IsRestricted` -> `RpcCall` * fix no input route problem * interface: `RpcHandlerDummy` docs * interface: crate docs * replace `create_router` with `RouterBuilder` * types: docs * types: doc `JsonRpc{Request,Response}` * types: readme docs * interface: doc `route/` * interface: fix `todo!()` * interface: allow customizing HTTP method on route functions * interface: fix tests * fix derives * Update rpc/interface/README.md Co-authored-by: Boog900 <boog900@tutanota.com> * Update rpc/interface/README.md Co-authored-by: Boog900 <boog900@tutanota.com> * interface: make `RpcHandler`'s `Future` generic * interface: add JSON-RPC notification todo * formatting * interface: use associated type bound for `RpcHandler`'s `Future` --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-08-06 00:50:38 +01:00
Boog900	1a178381dd	Storage: split the DB service abstraction (#237) ... * split the DB service abstraction * fix ci * misc changes * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * review fixes * Update storage/service/Cargo.toml Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * Update storage/service/Cargo.toml Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * fix clippy * fix doc * `bc` -> `blockchain` * doc fixes * Update storage/service/README.md Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * cargo fmt --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-08-05 21:47:30 +01:00
Boog900	b44c6b045b	Consensus: add alt-chain handling (#214) ... * add `pop_blocks` to the difficulty cache * add a rolling median struct * use RollingMedian in weight cache * add pop_blocks to weight cache * add alt context cache * add getting alt RX vms * rework alt cache * add alt block verify function * keep alt caches around * add alt checked alt blocks to the cache * check the alt blocks timestamp * add docs + cleanup code * add popping blocks from the context cache * finish popping blocks + fix tests * fix doc * add a test popping blocks from HF cache * add a request to clear alt caches * add back lint * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * review fixes * small changes * change panic doc --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-07-29 01:13:08 +01:00
hinto-janai	a2bca1b889	rpc: add JSON doc-tests (#232) ... * fixed-bytes: add `serde`, document feature flags * fixed-bytes: add derives * rpc: add `as _` syntax to macro * rpc: use `ByteArrayVec` and `ContainerAsBlob` for binary types * fixed-bytes: re-add derives * rpc-types: dedup default value within macro * readme: fixed bytes section * types: custom epee - `BlockCompleteEntry` * types: custom epee - `KeyImageSpentStatus` * types: custom epee - `PoolInfoExtent` * types: add `Status::Other(String)` variant * types: custom epee - `TxEntry`, add `read_epee_field` macro * bin: custom epee - `GetBlocks` * types: add `serde.rs` * misc: make `TxEntry` an `enum`, impl serde * misc: `unimplemented!()` for `TxEntry`'s epee * types: add `BlockCompleteEntry` * rpc: replace `BlockCompleteEntry` with `cuprate-types` * types: document `BlockCompleteEntry` * bin: fix `number_of_fields` for `GetBlocksResponse` * misc: add `Distribution` * distribution: add todo * misc fixes * readme: add `(De)serialization invariants` * distribution: compress variants * types: add `block_complete_entry.rs` * net: fix imports * p2p: fix imports * turn off default-features * p2p: fix imports * misc fixes * Update net/wire/Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> * distribution: module doc * wire: re-export types * test-utils: add `crate::rpc::types` module * test-utils: conditional json doc-tests * bin: use enum for `GetBlocksResponse` * misc: use lowercase for stringify * json: add test data, fix macro doc tests * json: add all data * other: add all data * bin: add skeleton * docs * move type to correct file * remove duplicated fields for custom epee * rpc: `client/{client,constants}.rs` -> `client.rs` * lib.rs: remove `clippy::module_inception` * macros: add json doc test macro * json: add some tests * json: add doc-test for all types * add all other JSON doc-tests * move doc-test macros to files * base: add doc-tests * json: TODO distribution test --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-07-28 21:43:16 +01:00
hinto-janai	929d19c450	rpc: custom epee for misc/bin types (#229) ... * fixed-bytes: add `serde`, document feature flags * fixed-bytes: add derives * rpc: add `as _` syntax to macro * rpc: use `ByteArrayVec` and `ContainerAsBlob` for binary types * fixed-bytes: re-add derives * rpc-types: dedup default value within macro * readme: fixed bytes section * types: custom epee - `BlockCompleteEntry` * types: custom epee - `KeyImageSpentStatus` * types: custom epee - `PoolInfoExtent` * types: add `Status::Other(String)` variant * types: custom epee - `TxEntry`, add `read_epee_field` macro * bin: custom epee - `GetBlocks` * types: add `serde.rs` * misc: make `TxEntry` an `enum`, impl serde * misc: `unimplemented!()` for `TxEntry`'s epee * types: add `BlockCompleteEntry` * rpc: replace `BlockCompleteEntry` with `cuprate-types` * types: document `BlockCompleteEntry` * bin: fix `number_of_fields` for `GetBlocksResponse` * misc: add `Distribution` * distribution: add todo * misc fixes * readme: add `(De)serialization invariants` * distribution: compress variants * types: add `block_complete_entry.rs` * net: fix imports * p2p: fix imports * turn off default-features * p2p: fix imports * misc fixes * Update net/wire/Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> * distribution: module doc * wire: re-export types * bin: use enum for `GetBlocksResponse` * misc: use lowercase for stringify * remove duplicated fields for custom epee * types: remove `should_write()` for custom epee * bin: split `GetBlocksResponse` variant fields into structs * misc: split `Distribution` variant fields into structs * small fixes * put all fields in `read_epee_field!` * distribution: (de)compress during epee/serde (de)serialization * distribution: leave (de)compression functions as `todo!()` --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-07-25 16:46:41 +01:00
hinto-janai	aa718e224f	test-utils: add crate::rpc::data module (#231) ... * test-utils: add `crate::rpc::types` module * test-utils: conditional json doc-tests * json: add test data, fix macro doc tests * json: add all data * other: add all data * bin: add skeleton * docs * move type to correct file * rpc: `client/{client,constants}.rs` -> `client.rs` * lib.rs: remove `clippy::module_inception`	2024-07-19 00:50:27 +01:00
hinto-janai	6820da9848	types: add BlockCompleteEntry (#230) ... * types: add `block_complete_entry.rs` * net: fix imports * p2p: fix imports * turn off default-features * p2p: fix imports * Update net/wire/Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> * wire: re-export types --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-07-16 22:47:50 +01:00
hinto-janai	0910c0a231	rpc: use ByteArrayVec and ContainerAsBlob (#227) ... * fixed-bytes: add `serde`, document feature flags * fixed-bytes: add derives * rpc: add `as _` syntax to macro * rpc: use `ByteArrayVec` and `ContainerAsBlob` for binary types * fixed-bytes: re-add derives * rpc-types: dedup default value within macro * readme: fixed bytes section	2024-07-13 01:26:11 +01:00
hinto-janai	fbae3df203	storage: move table generator macro blockchain -> database (#222) ... * move table generator macro `blockchain` -> `database` * blockchain: fix imports * docs * fix import ordering	2024-07-11 14:20:56 +01:00
hinto-janai	824651c8cf	fixed-bytes: add serde, document feature flags (#226) ... * fixed-bytes: add `serde`, document feature flags * manual impl `serde::Deserialize` * add serde tests	2024-07-11 02:00:47 +01:00
dependabot[bot]	0d7b86abe3	build(deps): bump zerovec from 0.10.2 to 0.10.4 (#217) ... Bumps [zerovec](https://github.com/unicode-org/icu4x) from 0.10.2 to 0.10.4. - [Release notes](https://github.com/unicode-org/icu4x/releases) - [Changelog](https://github.com/unicode-org/icu4x/blob/main/CHANGELOG.md) - [Commits](https://github.com/unicode-org/icu4x/commits/ind/zerovec@0.10.4) --- updated-dependencies: - dependency-name: zerovec dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-09 23:03:13 +01:00
dependabot[bot]	03815efe29	build(deps): bump zerovec-derive from 0.10.2 to 0.10.3 (#223) ... Bumps [zerovec-derive](https://github.com/unicode-org/icu4x) from 0.10.2 to 0.10.3. - [Release notes](https://github.com/unicode-org/icu4x/releases) - [Changelog](https://github.com/unicode-org/icu4x/blob/main/CHANGELOG.md) - [Commits](https://github.com/unicode-org/icu4x/commits/ind/zerovec-derive@0.10.3) --- updated-dependencies: - dependency-name: zerovec-derive dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-09 23:01:31 +01:00
Boog900	d5c8eba1d8	P2P: API Improvements (#168) ... * start handshaker builder * finish builder * remove borsh requirement * fix poll_ready :/ * add more docs to builder * more docs * fix clippy * merge fixes * fix doc test * fix imports * cargo fmt * split `PeerRequest` and `PeerResponse` enums up. This splits them both into a protocol and admin enum * add request handler file * integrate request handler into connection * fix docs * doc updates * add more docs * fix docs * misc changes * review fixes * fix merge * add dummy file * fix docs * Update p2p/dandelion-tower/src/router.rs * fix docs	2024-07-04 21:05:22 +01:00
hinto-janai	71131a4836	ci: fix doc.yml (#213) ... * types: remove borsh/serde * blockchain: re-add optional serde * doc.yml: remove `-D warnings`	2024-07-04 15:52:51 +01:00
Boog900	7c8466f4ba	Storage: add blockchain history requests (#206) ... * Add database requests for chain history * misc fixes * review comments * fix clippy * add link and fix typo * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * add comment --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-07-02 23:08:19 +01:00
dependabot[bot]	7e9891de5b	build(deps): bump curve25519-dalek from 4.1.2 to 4.1.3 (#197) ... Bumps [curve25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek) from 4.1.2 to 4.1.3. - [Release notes](https://github.com/dalek-cryptography/curve25519-dalek/releases) - [Commits](https://github.com/dalek-cryptography/curve25519-dalek/compare/curve25519-4.1.2...curve25519-4.1.3) --- updated-dependencies: - dependency-name: curve25519-dalek dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-06-26 22:59:11 +01:00
hinto-janai	a438279aa8	storage: split cuprate-blockchain <-> cuprate-database (#160) ... * storage: port some code `cuprate-blockchain` -> `database` * database: remove `Tables` references * database: remove old `cuprate-blockchain` type references * find/replace `cuprate_blockchain` -> `database`, add `create_db()` * database: fix redb * database: use readme for docs, link in `lib.rs` * database: fix `open_db_ro`, `open_db_rw`, `create_db` behavior * database: add open table tests * database: fix tests, remove blockchain specific references * database: remove `ReaderThreads`, make `db_directory` mandatory * initial `cuprate-blockchain` split * fix doc links * rename, fix database config * blockchain: create `crate::open()`, `OpenTables::create_tables()` * more compat fixes * fix imports * fix conflicts * align cargo.toml * docs * fixes * add `unused_crate_dependencies` lint, fix * blockchain: add open table tests	2024-06-26 22:51:06 +01:00
hinto-janai	e405786a73	rpc: start cuprate-rpc-types (#147) ... * rpc: add `monero-rpc-types` * lib.rs: add lints * add base files, deps * fix macro generation, doc test * add `strum`, add `misc` module * document struct generation macro * add `GetHeight` * lib.rs: create re-export macro * macro changes, add few more types * docs * `monero-rpc-types` -> `cuprate-rpc-types` * fix modules * specify commit in macro, add () type aliases * macro docs, fixes * add `Status::Other(String)` * add TODO for `strum` * Update rpc/types/Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> * add `BinaryString` * add `ResponseBase` * add `CORE_RPC_*` constants * fix status; use `CORE_RPC_*` constants * cargo.toml: add `epee_encoding` * rpc: add epee_encoding impl for `Status` * macro: add epee_encoding for every type * remove `strum` * add response bases * add `CORE_RPC_STATUS_UNKNOWN` * add response/request bases for epee * create `base` module * use different type for macro example * move base / root types around * docs, status serde test * status: use `Status::Unknown` for `epee_default_value` * json: add missing fields to `GetBlockTemplateRequest` not sure I missed these cc73fe7116/src/rpc/core_rpc_server_commands_defs.h (L947-L950) --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-06-26 22:24:05 +01:00
hinto-janai	4b93dbec4c	workspace: enforce crate/directory naming scheme (#164) ... * rename all directories and crates * fix all `use` * fix doc link * `dandelion/` -> `dandelion-tower/` * fix epee-encoding test * fix `json-rpc` * fix pruning * crate import fixes * fix leftover merge conflicts * fix `epee-encoding`	2024-06-24 02:30:47 +01:00
Boog900	10aac8cbb2	P2P: Block downloader (#132) ... * impl async buffer * clippy * p2p changes * clippy * a few more docs * init cuprate-p2p * remove some unrelated code and add some docs * start documenting client_pool.rs * add more docs * typo * fix docs * use JoinSet in connection maintainer * small changes * add peer sync state svc * add broadcast svc * add more docs * add some tests * add a test * fix merge * add another test * unify PeerDisconnectFut and add more docs * start network init * add an inbound connection server * remove crate doc for now * fix address book docs * fix leak in client pool * correct comment * fix merge + add some docs * review comments * init block downloader * fix doc * initial chain search * add chain_tracker * move block downloader to struct * spawn task whe getting blocks * check for free peers and handle batch response * add test bin * working block downloader * dynamic batch sizes * dandelion_tower -> dandelion-tower * fix async-buffer builds * check if incoming peers are banned * add interface methods * update docs * use a JoinSet for background network tasks * dynamic batch size changes * Keep a longer of queue of blocks to get * more checks on incoming data * fix merge * fix imports * add more docs * add some limits on messages * keep peers that dont have the current need data * fix clippy * fix .lock * fix stopping the block downloader * clean up API and add more docs * tracing + bug fixes * fix panic * doc changes * remove test_init * remove spammy log * fix previous merge * add a test * fix test * remove test unwrap * order imports correctly * clean up test * add a timeout * fix tests * review fixes * make `BlockDownloader` pub * make `initial_chain_search` pub * make `block_downloader` private * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * split some sections into separate modules * split chain requests * sort imports * check previous ID is correct * fix typos * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-06-22 01:29:40 +01:00
hinto-janai	bef2a2cbd4	epee: unseal trait EpeeValue (#184) ... * unseal `trait EpeeValue` * fix `container_as_blob.rs` * clippy * epee-encoding: remove `sealed`	2024-06-20 23:20:13 +01:00
Boog900	b76042a4e4	Cargo update (#171) ... cargo update	2024-06-18 22:43:35 +01:00
jomuel	c837f2f48e	Fast sync part 2 (#156) ... * boilerplate * Cargo.lock * Stub of block validation * Block validation (diff by @dllud) * Cargo.lock * Complete implementation of block validation request * Apply suggestions from code review Co-authored-by: Boog900 <boog900@tutanota.com> * More suggestions * Update consensus/fast-sync/src/fast_sync.rs * Update consensus/fast-sync/src/fast_sync.rs --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-06-14 23:00:18 +01:00
Boog900	acd5380256	Fix clippy & Cargo update (#161) ... * fix clippy * run cargo update * fix cargo deny * remove duplicate `#[test]`	2024-06-13 19:08:34 +01:00
hinto-janai	a3e34c3ba8	rpc: implement json-rpc crate (#148) ... * rpc: add `json-rpc` from https://github.com/Cuprate/cuprate/pull/43 Maintains all the changes made in that branch * workspace: add `rpc/json-rpc` * json-rpc: fix cargo.toml * add todo * satisfy clippy * `method/params` -> `body` switch, adjust input types * add test helpers, test tagged enums and flatten structs * fix id type `None` <-> `Some(Id::Null)` difference * lib.rs: add docs * impl `Version` * impl `Id` * impl `Request` * impl `Response` * impl `ErrorCode` * impl `ErrorObject` * fixes * add monero jsonrpc tests * response: add id test * add display docs to `ErrorObject` * remove `#[inline]` * add id null test * cleanup * code: clarify Monero's error code usage in docs * id: fix macro indentation * readme: fix `Response` -> `Request` * request: add `lowercase` test * tests: formatting, more string tests * readme: add `Serialization changes` * code: ugly `match` -> `if` * response: manual deserialization impl - lowercase keys only - enforce either `result/error` but not both * remove unneeded clone bounds * readme: add implementation comparison tests * request/response: more tests * readme: formatting, assert error messages are expected * request: add unknown field test * request/response: add unknown field and unicode test	2024-06-12 02:12:31 +01:00
jomuel	663c852b13	Fast sync (WIP) (#155) ... * Fast sync (work in progress) * Cargo.lock * Add missing hashes file * clippy warnings * Stub of database tool to create the fast sync hashes * Command line arg for target height, error handling * Cargo.lock * fmt and unused imports * fmt * Add license information to consensus/fast-sync/Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> * Order imports in consensus/fast-sync/src/create.rs Co-authored-by: Boog900 <boog900@tutanota.com> * beautify hex generation function & fmt * Reorder imports consensus/fast-sync/src/fast_sync.rs	2024-06-09 14:24:44 +01:00
Boog900	07f61bdb9c	Consensus: fix panic in batch verifier (#152) ... * fix panic in batch verifier * docs * review comments * Update consensus/rules/src/batch_verifier.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-06-07 13:34:53 +01:00
hinto-janai	6df67bb9d3	rpc: add initial crates (#146) ... * add `rpc/` crates * rpc: fix `monero-rpc-server` -> `monero-rpc-types` * rpc: add skeleton `rpc/json-rpc` * remove `cuprate-rpc-server`, add `json-rpc`	2024-06-05 15:35:08 +01:00
Boog900	0622237d19	Consensus: use cuprate-types types (#145) ... * move consensus database to /types * fix `storage` builds * unify `VerifiedBlockInformation` * fix docs * change `Database` trait wording * order imports * service -> blockchain * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * fix typo * fix key_images_spent * add back todo * fix tests * service -> blockchain 2 * update docs * update docs 2 --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-06-04 18:19:35 +01:00
Boog900	889e15738b	Cleanup & Document consensus (#65) ... * change monero-consensus to cuprate-consensus-rules * document the context service * remove the mutex on blockchain context * comment the context caches * add back tokio * document block checks * typo * keep tha amount of outputs with a certain amount in the output cache * typo * nuke cross-block batch verification * remove RPC scanner * change how contextual data is got. * fmt & clippy fixes * typo * cargo update * restore Cargo.lock * add a verify tx test. + fixes an issue with verifying signatures after BPs * clippy * remove bad test * add mores tests and fix a couple bugs * typos * move tests and add some more * typo * remove scan_chain docs * fix check for duplicate txs when duplicates are not sequential * add a proptest for dup txs * cache tx verification state * doc updates + move `Vec` to `Arc<[]>` * clippy * misc changes * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> Co-authored-by: SyntheticBird <118022351+SyntheticBird45@users.noreply.github.com> * fix fmt * review changes --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com> Co-authored-by: SyntheticBird <118022351+SyntheticBird45@users.noreply.github.com>	2024-05-31 01:52:12 +01:00
hinto-janai	4876e159e7	repo: add skeleton directories, fix labels (#139) ... * `database/` -> `storage/`, create skeleton `storage/` crates * add skeleton root directories * books: add skeleton * ci: fix labeler.yml * fix CONTRIBUTING.md * fix workspace Cargo.toml * fix `storage/` * rename helper fs fn, `cuprate_database` -> `cuprate_blockchain` * fix Cargo.toml * cuprate-blockchain: revert find-and-replace Since it has to be ported to the book anyway, there's not much reason to update this and create review diffs. * labeler.yml: remove `A-docs` from `books/` changes Although books are documentation, the `A-books` label already exists for this	2024-05-29 02:18:30 +01:00
Boog900	c5fbbcc6e8	P2P Client Pool (#121) ... * p2p changes * clippy * a few more docs * init cuprate-p2p * remove some unrelated code and add some docs * start documenting client_pool.rs * add more docs * typo * fix docs * use JoinSet in connection maintainer * small changes * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * review changes * Update p2p/cuprate-p2p/src/connection_maintainer.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-05-17 14:52:51 +01:00
Boog900	75306babf8	dandelion++ lib (#111) ... * init D++ * init D++ router * working D++ router * add test * D++ tx pool * add more txpool docs * add a txpool builder * add tracing * add more docs * fix doc * reduce test epoch (windows CI fail) * generate first state in config Windows seems to not allows taking a big value from an instant * extend tests * clippy * review comments + more docs * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * update Cargo.lock * rename txpool.rs -> pool.rs * review comments * Update p2p/dandelion/src/tests/router.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * Update p2p/dandelion/src/router.rs Co-authored-by: hinto-janai <hinto.janai@protonmail.com> --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-05-05 20:22:41 +01:00
Boog900	00c3692eac	P2p changes (#118) ... * p2p changes * clippy * a few more docs * review comments * fix peer list bug	2024-05-02 22:58:22 +00:00
hinto-janai	9c27ba5791	database: impl service fn bodies (#113) ... * write: impl write_block() * ops: add `get_block_info()` * read: impl block fn's * read: fix signatures * service: wrap `ConcreteEnv` in `RwLock` and doc why * heed: use `read-txn-no-tls` for `Send` read transactions * service: remove RwLock, impl some read functions * read: impl `outputs()` * read: flatten indentation, add `thread_local()` * read: impl `number_outputs_with_amount()` * tests: add `AssertTableLen` * ops: replace all table len asserts with `AssertTableLen` * service: initial tests * service: finish most tests * service: fix bad block data in test * tables: fix incorrect doc * service: add `ReadRequest::Outputs` test * read: use macros for set/getting `ThreadLocal`'s based on backend * small fixes * fix review * small fixes * read: fix ThreadLocal macros for `redb` * read: move `Output` mapping to `crate::free` it's needed in tests too * service: check output value correctness in tests * helper: add timelock <-> u64 mapping functions * free: use `u64_to_timelock()` * read: rct outputs * read: fix variable name * read: use ThreadLocal for both backends * heed: use Mutex for `HeedTableRo`'s read tx * block: add miner_tx * heed: remove Table bound oops * Revert "heed: use Mutex for `HeedTableRo`'s read tx" This reverts commit 7e8aae016c. * add `UnsafeSendable` * read: use `UnsafeSendable` for `heed`, branch on backend * read: safety docs * cargo.toml: re-add `read-txn-no-tls` for heed * ops: fix tests, remove miner_tx * fix tx_idx calculation, account for RCT outputs in tests * read: docs, fix `get_tables!()` for both backends * fix clippy * database: `unsafe trait DatabaseRo` * tx: use correct tx_id * free: remove miner_tx comment * free: remove `amount` input for rct outputs * ops: split `add_tx` inputs * read: use `UnsafeSendable` for all backends * heed: update safety comment * move output functions `free` -> `ops` * read: fix `chain_height()` handling * remove serde on `UnsafeSendable` * de-dup docs on `trait DatabaseRo`, `get_tables!()` * Update database/src/unsafe_sendable.rs Co-authored-by: Boog900 <boog900@tutanota.com> --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-05-01 18:52:20 +01:00
Boog900	88f7d1f212	download monerod in CI (#123) ... * download monerod in CI * move action file * add macOS arm support * remove reqwest from workspace * undo whitespace changes * fix indentation * update comments * add monerod to .gitignore * Apply suggestions from code review Co-authored-by: hinto-janai <hinto.janai@protonmail.com> * add back spacing --------- Co-authored-by: hinto-janai <hinto.janai@protonmail.com>	2024-05-01 01:21:08 +01:00
hinto-janai	3f2cc5144b	database: update backends (#119) ... * `heed 0.20.0-alpha.9 -> 0.20.0`, `redb 2.0.0 -> 2.1.0` * heed: fix * redb: fix	2024-04-29 01:12:42 +01:00
hinto-janai	32a44a2503	test-utils: compare data against RPC (#115) ... * cargo: add `pretty_assertions` * test-utils: add tests to compare data against RPC data, and fix * small fixes * database: fix `cumulative_generated_coins` from test data block had wrong data which offset all of the asserts	2024-04-25 19:58:45 +01:00
hinto-janai	c65eb0a3ca	database: implement ops/ (#102) ... * ops: add `trait MoneroR{o,w}` * update `trait MoneroR{o,w}` bounds * types: add `BlockInfoLatest` type alias * block: impl most core functions * types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916 * fix table type test * cargo.toml: add `{cuprate-types, monero-serai}` * add_block: add all other block data * ops: remove unneeded `block` functions * env: add `EnvInner::open_db_rw_all()` * types: fix test * block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()` * add `trait Tables[Mut]` and use it in `EnvInner` * block: use `TablesMut` * tables: replace manual impl with `define_trait_tables!()` * tables: docs for `trait Tables[Mut]` * tables: doc functions * create `call_fn_on_all_tables_or_early_return!()` macro * block: cleanup signatures + bodies * block: more fn's, docs * block: add `doc_{error,single,bulk}!()` * remove `ops/monero.rs` * move `height()` to `ops/blockchain.rs` * add `ops/macros.rs` * tx: add fn signatures * output: fix fn signatures * ops: expose `_inner()` functions * block: add `add_block_header{_bulk, _inner}()` * ops: remove doc_{fn,inner}!()` * ops: remove `_{inner,bulk}()`, lifetime + generics * update lib/mod docs * ops: add and use `doc_add_block_inner_invariant!()` * ops: add docs/return to inner `add_block()` functions * add_block(): extract and use fn for {key_image, output} * ops: more fn body impl + `add_block()` * cargo: add `monero-pruning` * ops: extract out `tx` functions from `add_block()` * property: add `db_version()` * ops: `pop_block()` body, remove other `pop_block` fn's * types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation` * block: put `block_blob`, pass `Tables` to sub-functions `impl TablesMut` can't mutably pass multiple tables since it takes `&mut self`, so all functions unfortunately have to take a full `&mut impl TablesMut` even though they only need a few tables. * database: add `DatabaseRw::take()` useful for `pop_block()` where we need the value afterwards * block: deserialize tx's from `block_blobs` in `pop_block()` * blockchain: `height()` -> `chain_height()` * output: fix `amount_index` * ops: fix unlock_time, chain_height * `BlockInfoV{1,2,3}` -> `BlockInfo` * constants: add `DATABASE_VERSION` * database: add `DatabaseRw::update()` * output: use `DatabaseRw::update()` in `remove_output()` * add `TxBlobs` table, ignore pruning tables * block: mostly impl `add_block()` body * ops: comments * add_block: miner v2 tx commitment, height cast * block: impl `pop_block()` * block: mostly impl `get_block()` * block: impl `get_block_{from_height,header,header_from_height}` * add `OutputFlags` bitflags * add_block: u32::try_into(height: u32), use `OutputFlags` * tx: impl `get_{tx,tx_from_id}()` * tx: move docs tests to `#[test]` testing everything in 1 go is more natural since e.g: `add_tx()` is followed by `get_tx()` * tables: add `trait TablesIter`, `all_tables_empty()` This allows `TablesMut` to be a superset of `Tables` and use all its accessor functions. * use cuprate-test-utils, fix tx tests * block: `add_block()` take block by ref * tx: use all txs in tests * output: add `all_tx_functions()` test * add_block: check current height against input * block: map more fields in `get_block()` * block: remove `get_block()`, doc tests, fix `get_block_header()` * block: dummy values in test * heed: use `last/first()` instead of `unsafe` cursors We no longer have DUP semantics and also hard to debug errors were popping up on `del_current()`... * heed: fix `DatabaseRw::delete` Ok(true) means the key did not exist, so we must return Err(RuntimeError::KeyNotFound) * block: `add_block()` (dummy value) test * ops: `key_image` tests * cleanup, docs, tests * backend: test `take()` & `update()` * docs * remove `OutputFlags::NONE` * add_block(): add asserts, panic docs, `should_panic` tests * backend: remove `Ok(())` in `Database::delete` if already deleted redb already does this, so heed so match * block: move block operations after tx/outputs * `amount == 0` -> `amount == 1` * Nit: StorableVec::wrap_ref * `saturating_sub(1)` -> `- 1` * add `TxOutputs` table * add_block(): add to `tx_outputs` table * fix `DatabaseRo::update` * add_tx(): take `block_height` as input * tx: add/remove from `TxOutputs` table * output: remove if `amount == 1` -> `amount_index == 0` * output: fix `add_output()`'s `amount_index` calculation * output: fix `add_output()`'s `amount_index` calculation again * output: tests for `amount_index/num_outputs` * block: `num_outputs - 1` and `take()` -> `get()` We don't need to `take()` since the call afterwards to `remove_output()` removes the entry * block: swap `get_block_extended_header[_from_height]()` * move `{key_image,output}` handling `add_block()` -> `add_tx()` * blockchain: add doc to `top_block_height()` * block: manual panic -> `assert_eq!()` * test-utils: add `block_blob` to `VerifiedBlockInformation` field introduced in this PR * ops: use real block/tx data in tests * block: `total_generated_coins` -> `cumulative_generated_coins` * fix clippy, docs, TODOs * `cumulative_generated_coins()`: `block/` -> `blockchain/` * blockchain: add `cumulative_generated_coins()` tests * Update database/src/ops/block.rs Co-authored-by: Boog900 <boog900@tutanota.com> * `cumulative_generated_coins()` docs for pre-block-0 special case --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-04-23 02:53:29 +01:00
hinto-janai	ee22e81c7e	test-utils: add rpc module (#110) ... * test-utils: impl `rpc` module * client: use `spawn_blocking` * client: add tests * ignore test for now * add example and `get_transaction_verification_data()` * client: calculate proper `generated_coins` * data: fix `generated_coins/reward` references * data: fix height	2024-04-21 17:11:23 +01:00
dependabot[bot]	9ad7ea3fa0	build(deps): bump rustls from 0.22.2 to 0.22.4 (#112) ... Bumps [rustls](https://github.com/rustls/rustls) from 0.22.2 to 0.22.4. - [Release notes](https://github.com/rustls/rustls/releases) - [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md) - [Commits](https://github.com/rustls/rustls/compare/v/0.22.2...v/0.22.4) --- updated-dependencies: - dependency-name: rustls dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-21 01:34:17 +01:00
hinto-janai	fe2366c5df	test-utils: return &'static verified superset types (#108) ... * data: return `&'static` verified superset types * docs * fix VerifiedBlockMap, use manual input, add block 5da0a3d * free: use macros to generate accessor functions * constants: use macro * docs * `tx_data` -> `tx_blob`	2024-04-21 01:34:04 +01:00
hinto-janai	ad7b750d76	database: impl service::{Request,Response} mappings (#101) ... * add `cuprate-types` * remove `cuprate_database::service::{request,response}` * use `cuprate_types::service::{request,response}` * service: fix `Request` `match`'s * service: create `ReadRequest` function mappings * service: create `WriteRequest` function mappings * service: add rough `WriteRequest` retry loop * service: handle `RuntimeError::ResizeNeeded` in writer * add `{R,r}o` exception to typos * docs * env: make `resize_map()` return new memory map byte size * write: proactively handle resizes `add_block()` takes `VerifiedBlockInformation` such that it can just take the inner blobs of data. This is a problem when reactively resizing since we no longer have the block struct we just gave away so we're forced to `.clone()` each retry. Instead of that - we will proactively resize so the resize error will never occur in the first place. * read: use type aliases * docs * fix import * write: handle resizes reactively * service: panic if response can't be sent back * write: add loop unreachable asserts * service: print and drop error instead of panic * write: fix retry loop off-by-1 * write: fix docs * review changes * update readme * remove `BlockBatchInRange` request/response * Update database/README.md Co-authored-by: Boog900 <boog900@tutanota.com> --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-04-16 23:05:38 +01:00
hinto-janai	e6465ec613	test-utils: add data/ (#107) ... * test-utils: add `data/` + add lints + `clippy --fix` * data: assert tx_hash is correct * data: add `BLOCK_202612` doctest * lib.rs: remove lints * data: add more tx data, fix names * revert `Cargo.toml` formatting * data: add more block data * free: add serialization tests * remove clippy allows	2024-04-13 02:13:46 +01:00
dependabot[bot]	eb3227ef2d	build(deps): bump h2 from 0.3.24 to 0.3.26 (#105) ... Bumps [h2](https://github.com/hyperium/h2) from 0.3.24 to 0.3.26. - [Release notes](https://github.com/hyperium/h2/releases) - [Changelog](https://github.com/hyperium/h2/blob/v0.3.26/CHANGELOG.md) - [Commits](https://github.com/hyperium/h2/compare/v0.3.24...v0.3.26) --- updated-dependencies: - dependency-name: h2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-05 23:45:06 +01:00
hinto-janai	d503548716	Implement types/ (#94) ... * workspace: add `bytemuck` to workspace * add `types/` * workspace: add `cuprate-types` to members * copy `consensus/` types to `types/` * remove `hard_fork/` * extended_block_header: impl `Pod`, fix layout * update `Request/Response` * impl types * fix `Response/Request` * impl `borsh` * workspace: add `strum` * service: add `strum` traits * remove `paste`, `serde_json`, `thiserror` * remove `strum` * VerifiedBlockInformation: remove `hf_vote` * Update Cargo.toml Co-authored-by: Boog900 <boog900@tutanota.com> --------- Co-authored-by: Boog900 <boog900@tutanota.com>	2024-03-27 00:46:32 +00:00
hinto-janai	5e7ee57482	database: redb 1.5.0 -> redb 2.0.0 (#95) ... * `redb 1.5.0` -> `redb 2.0.0` * `Redb{Key,Value}` -> `redb::{Key,Value}` * redb: remove unneeded lifetimes * database: remove read table lifetime * redb: remove read table lifetime * heed: remove `'tx` lifetime on read table * remove `'env, 'tx` lifetime from read/write table * redb: remove `'env, 'tx` lifetime from read/write table * heed: remove `'env, 'tx` lifetime from read/write table * redb: update `TxRo::commit()` docs	2024-03-24 22:15:26 +00:00
hinto-janai	3656a1ada7	database: return owned T in Storable (#92) ... * add `storable_slice.rs` * storable: `from_bytes(&[u8]) -> Self` * heed: return `T` in `Storable`, use `StorableSlice` in tests * remove `value_guard.rs` * redb: update `Storable` to return owned `T` * database: return `T` directly, remove `'a` lifetime * replace `[]` with `StorableSlice` * tables: add lifetime to `tables!()` and generated structs * backend: fix tests * heed: update fn signatures * storable: add `StorableVec`, remove `Storable::ALIGN` * tables/types: remove slice lifetimes, use `StorableVec` * backend: use `StorableVec`, fix tests * types: fix tests to use owned returned value * heed: fix `Storable` impl and tests * storable: `StorableVec` docs + tests * backend: assert values are detached from `get_range()` iterator * database: add docs * udpate docs, remove unneeded clippy `allow` * replace `ToOwnedDebug` with `std::fmt::Debug` * impl `StorableBytes` * backend: add `StorableBytes` to tests * readme: remove `to_owned_debug.rs` reference	2024-03-22 21:11:48 +00:00