Commit graph

236 commits

Author SHA1 Message Date
Boog900
8eaaac53ca
Merge branch 'broadcast-svc' into network-init 2024-05-18 18:38:39 +01:00
Boog900
191a1723a1
Merge branch 'main' into broadcast-svc 2024-05-18 14:01:16 +01:00
Boog900
41970d748a
unify PeerDisconnectFut and add more docs 2024-05-18 13:52:21 +01:00
Boog900
f4ef14ca18
add another test 2024-05-17 17:36:25 +01:00
Boog900
e1dfa3e605
fix merge 2024-05-17 15:23:51 +01:00
Boog900
065fa242d8
Merge branch 'main' into sync-states 2024-05-17 14:54:58 +01:00
c5fbbcc6e8
P2P Client Pool (#121)
* p2p changes

* clippy

* a few more docs

* init cuprate-p2p

* remove some unrelated code and add some docs

* start documenting client_pool.rs

* add more docs

* typo

* fix docs

* use JoinSet in connection maintainer

* small changes

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* review changes

* Update p2p/cuprate-p2p/src/connection_maintainer.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-05-17 14:52:51 +01:00
Boog900
b407377f52
add a test 2024-05-16 00:31:55 +01:00
Boog900
aedf87ec5b
add some tests 2024-05-15 17:15:39 +01:00
hinto-janai
38ede35468
database: more README docs (#126)
* tables: change order

* readme: add `Schema`

* readme: use type aliases

* readme: add `10. Known issues and tradeoffs`

* !!

* readme: add `The service` section

* readme: fill `Known issues and tradeoffs` section

* readme: fixes

* readme: fixes

* readme: review fixes
2024-05-15 16:58:18 +01:00
Boog900
df6d6ec187
add more docs 2024-05-15 16:30:46 +01:00
Boog900
7881013380
add broadcast svc 2024-05-15 01:47:44 +01:00
Boog900
020fd56673
add peer sync state svc 2024-05-15 01:33:05 +01:00
Boog900
cbb629d431
Merge branch 'main' into peer-set2 2024-05-14 20:04:03 +01:00
hinto-janai
d0ac94a813
ci: downgrade to gcc 13.2 for windows (#127) 2024-05-14 19:41:39 +01:00
Boog900
6d3096eb03
small changes 2024-05-13 23:56:00 +01:00
Boog900
e59bb0a1f5
use JoinSet in connection maintainer 2024-05-13 23:39:36 +01:00
SyntheticBird
31ce89412a
Removing old_database/ (#100)
remove old db
2024-05-09 21:06:33 +01:00
Boog900
37e7c29c48
fix docs 2024-05-07 01:18:29 +01:00
Boog900
1570844b0b
typo 2024-05-07 01:13:55 +01:00
Boog900
859d67d8b6
add more docs 2024-05-07 01:07:18 +01:00
Boog900
1b01336294
start documenting client_pool.rs 2024-05-06 02:22:30 +01:00
Boog900
63a3207316
Merge branch 'main' into peer-set2 2024-05-06 01:52:46 +01:00
75306babf8
dandelion++ lib (#111)
* init D++

* init D++ router

* working D++ router

* add test

* D++ tx pool

* add more txpool docs

* add a txpool builder

* add tracing

* add more docs

* fix doc

* reduce test epoch (windows CI fail)

* generate first state in config

Windows seems to not allows taking a big value from an instant

* extend tests

* clippy

* review comments + more docs

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* update Cargo.lock

* rename txpool.rs -> pool.rs

* review comments

* Update p2p/dandelion/src/tests/router.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update p2p/dandelion/src/router.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-05-05 20:22:41 +01:00
hinto-janai
10d327ca13
ci: use dtolnay/rust-toolchain (#124)
* ci: don't cache `.cargo/`, `.rustup/`

* ci: use `dtolnay/rust-toolchain@master`
2024-05-05 15:22:05 +01:00
hinto-janai
fb3d41ccbb
database: final docs + cleanup (#117)
* re-apply 'main' merge + doc patches

* fix redb lints

* update readme

* add `lib.rs, ops, service` doc-test examples

* docs for `config`, `ops`, add doc-tests

* remove merge error

incorrect leftover code from previous merge

* doc top-level types

* docs: error, tables, types

* misc docs, TODO, FIXME, SOMEDAY fixes

* change clippy lints

* tests: add `tables_are_sorted()`

* move `tables_are_sorted()` test to `backend/tests.rs`

* readme formatting

* small fixes

* readme fixes

* docs: `helper/`

* docs: `types/`

* database/README.md fixes

* doc fixes

* types: doc fixes

* fixes

* all review changes
2024-05-05 15:21:28 +01:00
Boog900
4b471a947f
Merge branch 'main' into peer-set2 2024-05-03 17:16:36 +01:00
00c3692eac
P2p changes (#118)
* p2p changes

* clippy

* a few more docs

* review comments

* fix peer list bug
2024-05-02 22:58:22 +00:00
hinto-janai
9c27ba5791
database: impl service fn bodies (#113)
* write: impl write_block()

* ops: add `get_block_info()`

* read: impl block fn's

* read: fix signatures

* service: wrap `ConcreteEnv` in `RwLock` and doc why

* heed: use `read-txn-no-tls` for `Send` read transactions

* service: remove RwLock, impl some read functions

* read: impl `outputs()`

* read: flatten indentation, add `thread_local()`

* read: impl `number_outputs_with_amount()`

* tests: add `AssertTableLen`

* ops: replace all table len asserts with `AssertTableLen`

* service: initial tests

* service: finish most tests

* service: fix bad block data in test

* tables: fix incorrect doc

* service: add `ReadRequest::Outputs` test

* read: use macros for set/getting `ThreadLocal`'s based on backend

* small fixes

* fix review

* small fixes

* read: fix ThreadLocal macros for `redb`

* read: move `Output` mapping to `crate::free`

it's needed in tests too

* service: check output value correctness in tests

* helper: add timelock <-> u64 mapping functions

* free: use `u64_to_timelock()`

* read: rct outputs

* read: fix variable name

* read: use ThreadLocal for both backends

* heed: use Mutex for `HeedTableRo`'s read tx

* block: add miner_tx

* heed: remove Table bound

oops

* Revert "heed: use Mutex for `HeedTableRo`'s read tx"

This reverts commit 7e8aae016c.

* add `UnsafeSendable`

* read: use `UnsafeSendable` for `heed`, branch on backend

* read: safety docs

* cargo.toml: re-add `read-txn-no-tls` for heed

* ops: fix tests, remove miner_tx

* fix tx_idx calculation, account for RCT outputs in tests

* read: docs, fix `get_tables!()` for both backends

* fix clippy

* database: `unsafe trait DatabaseRo`

* tx: use correct tx_id

* free: remove miner_tx comment

* free: remove `amount` input for rct outputs

* ops: split `add_tx` inputs

* read: use `UnsafeSendable` for all backends

* heed: update safety comment

* move output functions `free` -> `ops`

* read: fix `chain_height()` handling

* remove serde on `UnsafeSendable`

* de-dup docs on `trait DatabaseRo`, `get_tables!()`

* Update database/src/unsafe_sendable.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-05-01 18:52:20 +01:00
88f7d1f212
download monerod in CI (#123)
* download monerod in CI

* move action file

* add macOS arm support

* remove reqwest from workspace

* undo whitespace changes

* fix indentation

* update comments

* add monerod to .gitignore

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* add back spacing

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-05-01 01:21:08 +01:00
Boog900
4d61b100f5
remove some unrelated code and add some docs 2024-04-30 22:11:18 +01:00
SyntheticBird
0454f0ba2e
[Misc] Updating SyntheticBird's GPG Key (#122)
Updated SyntheticBird's GPG Key

I still have the old key but it's not the one I'm actively using for signing
nor distribute for communicating.
2024-04-30 20:09:53 +01:00
Boog900
b9caee9335
init cuprate-p2p 2024-04-30 02:02:12 +01:00
Boog900
75a1db3fdc
a few more docs 2024-04-29 01:13:39 +01:00
hinto-janai
3f2cc5144b
database: update backends (#119)
* `heed 0.20.0-alpha.9 -> 0.20.0`, `redb 2.0.0 -> 2.1.0`

* heed: fix

* redb: fix
2024-04-29 01:12:42 +01:00
Boog900
43001760aa
Merge branch 'main' into p2p-changes 2024-04-29 01:05:10 +01:00
hinto-janai
2ac90420c6
ci: fix windows/macos (#116)
* cryptonight: include homebrew directories

* test-utils: add arm macos to `monerod` downloader

* ci: install windows-gnu
2024-04-27 00:49:55 +01:00
Boog900
b4a2be2338
clippy 2024-04-26 21:17:47 +01:00
Boog900
1c72c0952f
p2p changes 2024-04-26 21:13:53 +01:00
hinto-janai
32a44a2503
test-utils: compare data against RPC (#115)
* cargo: add `pretty_assertions`

* test-utils: add tests to compare data against RPC data, and fix

* small fixes

* database: fix `cumulative_generated_coins` from test data

block had wrong data which offset all of the asserts
2024-04-25 19:58:45 +01:00
hinto-janai
bf6c21c71e
database: split cumulative_difficulty into low/high bits (#114)
* types: split `cumulative_difficulty` into low/high bits

* helper: add `map` module

* database: use `helper`'s cumulative_diff functions

* helper: rename functions

splitting bits isn't necessarily `cumulative_difficulty` specific

* database: fix tests

* helper: docs

* helper: test output is low-endian bits

* helper: docs

* Update helper/src/map.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update helper/src/map.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-24 21:47:48 +01:00
hinto-janai
c65eb0a3ca
database: implement ops/ (#102)
* ops: add `trait MoneroR{o,w}`

* update `trait MoneroR{o,w}` bounds

* types: add `BlockInfoLatest` type alias

* block: impl most core functions

* types: fix https://github.com/Cuprate/cuprate/pull/91#discussion_r1527668916

* fix table type test

* cargo.toml: add `{cuprate-types, monero-serai}`

* add_block: add all other block data

* ops: remove unneeded `block` functions

* env: add `EnvInner::open_db_rw_all()`

* types: fix test

* block: `&mut TxRw` -> `&TxRw`, use `open_db_rw_all()`

* add `trait Tables[Mut]` and use it in `EnvInner`

* block: use `TablesMut`

* tables: replace manual impl with `define_trait_tables!()`

* tables: docs for `trait Tables[Mut]`

* tables: doc functions

* create `call_fn_on_all_tables_or_early_return!()` macro

* block: cleanup signatures + bodies

* block: more fn's, docs

* block: add `doc_{error,single,bulk}!()`

* remove `ops/monero.rs`

* move `height()` to `ops/blockchain.rs`

* add `ops/macros.rs`

* tx: add fn signatures

* output: fix fn signatures

* ops: expose `_inner()` functions

* block: add `add_block_header{_bulk, _inner}()`

* ops: remove doc_{fn,inner}!()`

* ops: remove `_{inner,bulk}()`, lifetime + generics

* update lib/mod docs

* ops: add and use `doc_add_block_inner_invariant!()`

* ops: add docs/return to inner `add_block()` functions

* add_block(): extract and use fn for {key_image, output}

* ops: more fn body impl + `add_block()`

* cargo: add `monero-pruning`

* ops: extract out `tx` functions from `add_block()`

* property: add `db_version()`

* ops: `pop_block()` body, remove other `pop_block` fn's

* types: add `block_blob: Vec<u8>` to `VerifiedBlockInformation`

* block: put `block_blob`, pass `Tables` to sub-functions

`impl TablesMut` can't mutably pass multiple tables since
it takes `&mut self`, so all functions unfortunately have to
take a full `&mut impl TablesMut` even though they only need
a few tables.

* database: add `DatabaseRw::take()`

useful for `pop_block()` where we need the value afterwards

* block: deserialize tx's from `block_blobs` in `pop_block()`

* blockchain: `height()` -> `chain_height()`

* output: fix `amount_index`

* ops: fix unlock_time, chain_height

* `BlockInfoV{1,2,3}` -> `BlockInfo`

* constants: add `DATABASE_VERSION`

* database: add `DatabaseRw::update()`

* output: use `DatabaseRw::update()` in `remove_output()`

* add `TxBlobs` table, ignore pruning tables

* block: mostly impl `add_block()` body

* ops: comments

* add_block: miner v2 tx commitment, height cast

* block: impl `pop_block()`

* block: mostly impl `get_block()`

* block: impl `get_block_{from_height,header,header_from_height}`

* add `OutputFlags` bitflags

* add_block: u32::try_into(height: u32), use `OutputFlags`

* tx: impl `get_{tx,tx_from_id}()`

* tx: move docs tests to `#[test]`

testing everything in 1 go is more natural since e.g:
`add_tx()` is followed by `get_tx()`

* tables: add `trait TablesIter`, `all_tables_empty()`

This allows `TablesMut` to be a superset of `Tables`
and use all its accessor functions.

* use cuprate-test-utils, fix tx tests

* block: `add_block()` take block by ref

* tx: use all txs in tests

* output: add `all_tx_functions()` test

* add_block: check current height against input

* block: map more fields in `get_block()`

* block: remove `get_block()`, doc tests, fix `get_block_header()`

* block: dummy values in test

* heed: use `last/first()` instead of `unsafe` cursors

We no longer have DUP semantics and also hard to debug errors
were popping up on `del_current()`...

* heed: fix `DatabaseRw::delete`

Ok(true) means the key did not exist, so we must return
Err(RuntimeError::KeyNotFound)

* block: `add_block()` (dummy value) test

* ops: `key_image` tests

* cleanup, docs, tests

* backend: test `take()` & `update()`

* docs

* remove `OutputFlags::NONE`

* add_block(): add asserts, panic docs, `should_panic` tests

* backend: remove `Ok(())` in `Database::delete` if already deleted

redb already does this, so heed so match

* block: move block operations after tx/outputs

* `amount == 0` -> `amount == 1`

* Nit: StorableVec::wrap_ref

* `saturating_sub(1)` -> `- 1`

* add `TxOutputs` table

* add_block(): add to `tx_outputs` table

* fix `DatabaseRo::update`

* add_tx(): take `block_height` as input

* tx: add/remove from `TxOutputs` table

* output: remove if `amount == 1` -> `amount_index == 0`

* output: fix `add_output()`'s `amount_index` calculation

* output: fix `add_output()`'s `amount_index` calculation again

* output: tests for `amount_index/num_outputs`

* block: `num_outputs - 1` and `take()` -> `get()`

We don't need to `take()` since the call afterwards to
`remove_output()` removes the entry

* block: swap `get_block_extended_header[_from_height]()`

* move `{key_image,output}` handling `add_block()` -> `add_tx()`

* blockchain: add doc to `top_block_height()`

* block: manual panic -> `assert_eq!()`

* test-utils: add `block_blob` to `VerifiedBlockInformation`

field introduced in this PR

* ops: use real block/tx data in tests

* block: `total_generated_coins` -> `cumulative_generated_coins`

* fix clippy, docs, TODOs

* `cumulative_generated_coins()`: `block/` -> `blockchain/`

* blockchain: add `cumulative_generated_coins()` tests

* Update database/src/ops/block.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* `cumulative_generated_coins()` docs for pre-block-0 special case

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-23 02:53:29 +01:00
hinto-janai
ee22e81c7e
test-utils: add rpc module (#110)
* test-utils: impl `rpc` module

* client: use `spawn_blocking`

* client: add tests

* ignore test for now

* add example and `get_transaction_verification_data()`

* client: calculate proper `generated_coins`

* data: fix `generated_coins/reward` references

* data: fix height
2024-04-21 17:11:23 +01:00
dependabot[bot]
9ad7ea3fa0
build(deps): bump rustls from 0.22.2 to 0.22.4 (#112)
Bumps [rustls](https://github.com/rustls/rustls) from 0.22.2 to 0.22.4.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.22.2...v/0.22.4)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-21 01:34:17 +01:00
hinto-janai
fe2366c5df
test-utils: return &'static verified superset types (#108)
* data: return `&'static` verified superset types

* docs

* fix VerifiedBlockMap, use manual input, add block 5da0a3d

* free: use macros to generate accessor functions

* constants: use macro

* docs

* `tx_data` -> `tx_blob`
2024-04-21 01:34:04 +01:00
hinto-janai
ad7b750d76
database: impl service::{Request,Response} mappings (#101)
* add `cuprate-types`

* remove `cuprate_database::service::{request,response}`

* use `cuprate_types::service::{request,response}`

* service: fix `Request` `match`'s

* service: create `ReadRequest` function mappings

* service: create `WriteRequest` function mappings

* service: add rough `WriteRequest` retry loop

* service: handle `RuntimeError::ResizeNeeded` in writer

* add `{R,r}o` exception to typos

* docs

* env: make `resize_map()` return new memory map byte size

* write: proactively handle resizes

`add_block()` takes `VerifiedBlockInformation` such
that it can just take the inner blobs of data.

This is a problem when reactively resizing since we no longer
have the block struct we just gave away so we're forced to `.clone()`
each retry.

Instead of that - we will proactively resize so the resize error
will never occur in the first place.

* read: use type aliases

* docs

* fix import

* write: handle resizes reactively

* service: panic if response can't be sent back

* write: add loop unreachable asserts

* service: print and drop error instead of panic

* write: fix retry loop off-by-1

* write: fix docs

* review changes

* update readme

* remove `BlockBatchInRange` request/response

* Update database/README.md

Co-authored-by: Boog900 <boog900@tutanota.com>

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-16 23:05:38 +01:00
hinto-janai
e6465ec613
test-utils: add data/ (#107)
* test-utils: add `data/` + add lints + `clippy --fix`

* data: assert tx_hash is correct

* data: add `BLOCK_202612` doctest

* lib.rs: remove lints

* data: add more tx data, fix names

* revert `Cargo.toml` formatting

* data: add more block data

* free: add serialization tests

* remove clippy allows
2024-04-13 02:13:46 +01:00
dependabot[bot]
eb3227ef2d
build(deps): bump h2 from 0.3.24 to 0.3.26 (#105)
Bumps [h2](https://github.com/hyperium/h2) from 0.3.24 to 0.3.26.
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/v0.3.26/CHANGELOG.md)
- [Commits](https://github.com/hyperium/h2/compare/v0.3.24...v0.3.26)

---
updated-dependencies:
- dependency-name: h2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-05 23:45:06 +01:00
hinto-janai
2813c92505
database: make open_db_rw() take &TxRw (#104)
* env: take `&TxRw` in `open_db_rw()`

* heed: use `UnsafeCell` for write transactions

* backend: update tests

* add `trait DatabaseIter<T: Table>`

* heed: impl `trait DatabaseIter`, inline shared iter `fn`s

* env: make `open_db_ro()` return `impl DatabaseIter`

* heed: use full path for transaction fn calls

* tests: fix tx/table tests

* backend: fix redb

* heed: `UnsafeCell` -> `RefCell`

* docs

* remove unneeded `// SAFETY`

* Update database/src/backend/heed/env.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-04-03 22:44:25 +01:00
hinto-janai
21697b8af5
database: add more fn to trait DatabaseR{o,w} (#96)
* database: add more fn signatures

* database: fix docs

* heed: add new fn signatures with `todo!()`

* heed: add shared fn sigantures

* database: make `iter()` return `(key, value)`

* database: make `get_range()` return `(key, value)`

* database: make `first()/last()` return `(key, value)`

* heed: impl new fn signatures

* redb: impl new fn signatures

* backend: update tests for new `fn`s

* backend: update table tests

* heed: fix `retain()` predicate behavior

* database: add `keys()`, `values()`, `pop_first()`, `pop_last()`

* heed: impl `keys()`, `values()`, `pop_first()`, `pop_last()`

* redb: impl `keys()`, `values()`, `pop_first()`, `pop_last()`

* database: add `contains()`

* backend: add `contains()` to tests

* database: remove `retain()`

* database: only return value in `get_range()`

* database: `DatabaseRw::clear()` -> `EnvInner::clear_db()`

* redb: update `clear_db()` invariant doc

* fix redb

* cargo.toml: avoid merge conflict

* update `get_range()` docs

* redb: re-create table in `clear_db()`
2024-03-30 01:33:38 +00:00