Commit graph

23 commits

Author SHA1 Message Date
hinto-janai
88605b081f
books/architecture: port database design document (#267)
Some checks failed
Architecture mdBook / build (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* add chapters

* add files, intro

* db abstraction

* backends

* abstraction

* syncing

* serde

* issues

* common/types

* common/ops

* common/service

* service diagram

* service/resize

* service/thread-model

* service/shutdown

* storage/blockchain

* update md files

* cleanup

* fixes

* update for https://github.com/Cuprate/cuprate/pull/290

* review fix
2024-09-24 17:23:22 +01:00
Asurar
e7c6bba63d
Database: Split BlockBlobs table + Miscellaneous fixes (#290)
Some checks are pending
Audit / audit (push) Waiting to run
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Deny / audit (push) Waiting to run
Doc / build (push) Waiting to run
Doc / deploy (push) Blocked by required conditions
* Split `BlockBlobs` database table + misc fixes

- Split the `BlockBlobs` database table into two new tables: `BlockHeaderBlobs` and `BlockTxsHashes`.
- `add_block`, `pop_block` and `get_block_extended_header` have been edited consequently.
- `VerifiedBlockInformation` now have a `mining_tx_index: u64` field.
- Made `cuprate-helper`'s `thread` feature a dependency of the `service` feature
- Edited service test mapping of output. It is now a full iterator.

* fix fmt

* Update storage/blockchain/src/types.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* Update storage/blockchain/src/ops/block.rs

Co-authored-by: Boog900 <boog900@tutanota.com>

* fix warning

---------

Co-authored-by: Boog900 <boog900@tutanota.com>
2024-09-19 20:05:41 +01:00
Boog900
4169c45c58
Blockchain: add alt-block handling (#260)
Some checks are pending
Audit / audit (push) Waiting to run
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Deny / audit (push) Waiting to run
Doc / build (push) Waiting to run
Doc / deploy (push) Blocked by required conditions
* add new tables & types

* add function to fully add an alt block

* resolve current todo!s

* add new requests

* WIP: starting re-orgs

* add last service request

* commit Cargo.lock

* add test

* more docs + cleanup + alt blocks request

* clippy + fmt

* document types

* move tx_fee to helper

* more doc updates

* fmt

* fix imports

* fix fee

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* remove default features from `cuprate-helper`

* review fixes

* fix find_block

* add a test and fix some issues in chain history

* fix clippy

* fmt

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* add dev dep

* cargo update

* move `flush_alt_blocks`

* review fixes

* more review fixes

* fix clippy

* remove INVARIANT comments

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-09-19 16:55:28 +01:00
hinto-janai
6502729d8c
lints: replace allow with expect (#285)
* cargo.toml: add `allow_attributes` lint

* fix lints

* fixes

* fmt

* fix docs

* fix docs

* fix expect msg
2024-09-18 21:31:08 +01:00
Boog900
49d1344aa1
Storage: use saturating_add for cumulative_generated_coins (#275)
* use `saturating_add` for `cumulative_generated_coins`

* cargo fmt
2024-09-10 01:15:04 +01:00
hinto-janai
eead49beb0
lints: opt in manual lint crates (#263)
* cargo.toml: transfer existing lints

* rpc/interface: lints

* rpc/json-rpc: lints

* rpc/types: lints

* storage/blockchain: lints

* rpc/types: fix lints

* cargo.toml: fix lint group priority

* storage/blockchain: fix lints

* fix misc lints

* storage/database: fixes

* storage/txpool: opt in lints + fixes

* types: opt in + fixes

* helper: opt in + fixes

* types: remove borsh

* rpc/interface: fix test

* test fixes

* database: fix lints

* fix lint

* tabs -> spaces

* blockchain: `config/` -> `config.rs`
2024-09-02 18:12:54 +01:00
hinto-janai
bec8cc0aa4
helper: add and use cast module (#264)
* helper: add `cast` module

* fix crates

* spacing
2024-09-02 18:09:52 +01:00
Boog900
fdd1689665
Storage: tx-pool database (#238)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* split the DB service abstraction

* fix ci

* misc changes

* init tx-pool DBs

* add some comments

* move more types to `/types`

* add some ops

* add config & more ops functions & open function

* add read & write svcs

* add more docs

* add write functions + docs

* fix merge

* fix test

* fix ci

* move `TxPoolWriteError`

* add more docs

* fix toml formatting

* fix some docs

* fix clippy

* review fixes

* update docs

* fix merge

* fix docs

* fix tests

* fix tests

* add back lints

* Update storage/txpool/README.md

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-08-22 02:09:07 +01:00
hinto-janai
5648bf0da0
rpc: remove temporary lints (#255)
* rpc: remove temporary lints for types

* rpc: remove temporary lints for json-rpc

* rpc: remove temporary lints for interface

* cfgs `1 tab` -> `4 spaces`
2024-08-20 23:50:31 +01:00
hinto-janai
aeb070ae8d
Replace OnceLock + fn with LazyLock (#256)
* `consensus/`

* `helper/`

* `test-utils/`

* `storage/`

* fix docs + tests + lints

* decomposed_amount: remove `LazyLock`

* clippy
2024-08-20 22:53:32 +01:00
hinto-janai
59adf6dcf8
std::mem::{size,align}_of -> {size,align}_of (#254)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
2024-08-10 00:09:25 +01:00
Boog900
be2f3f2672
Consensus: move more types to types (#250)
* move `HardFork` to `types`

* fmt

* fix tests & doc

* fmt

* fix clippy

* move transaction verification data

* misc fixes

* doc fixes

* update README.md

* review fixes
2024-08-09 00:56:13 +01:00
Boog900
8227c28604
update monero-serai (#201)
* update monero-serai

* update monero-serai + change height to `usize`

* fix merge

* fix merge

* fix doc

* fix clippy take 2

* misc changes

* move RPC imports to dev deps

* handle miner txs when calculating fee

* Update consensus/rules/src/blocks.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update consensus/rules/src/transactions.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update storage/blockchain/src/ops/tx.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update test-utils/src/data/free.rs

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* fixes

* fix clippy

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-08-07 00:48:53 +01:00
Boog900
1a178381dd
Storage: split the DB service abstraction (#237)
* split the DB service abstraction

* fix ci

* misc changes

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* review fixes

* Update storage/service/Cargo.toml

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* Update storage/service/Cargo.toml

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* fix clippy

* fix doc

* `bc` -> `blockchain`

* doc fixes

* Update storage/service/README.md

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* cargo fmt

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-08-05 21:47:30 +01:00
Boog900
b44c6b045b
Consensus: add alt-chain handling (#214)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* add `pop_blocks` to the difficulty cache

* add a rolling median struct

* use RollingMedian in weight cache

* add pop_blocks to weight cache

* add alt context cache

* add getting alt RX vms

* rework alt cache

* add alt block verify function

* keep alt caches around

* add alt checked alt blocks to the cache

* check the alt blocks timestamp

* add docs + cleanup code

* add popping blocks from the context cache

* finish popping blocks + fix tests

* fix doc

* add a test popping blocks from HF cache

* add a request to clear alt caches

* add back lint

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* review fixes

* small changes

* change panic doc

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-07-29 01:13:08 +01:00
hinto-janai
0a390a362a
storage: doc fixes (#228)
Some checks are pending
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Doc / build (push) Waiting to run
Doc / deploy (push) Blocked by required conditions
* database: doc fixes

* blockchain: doc fixes

* database: fix doc test

* database: readme fixes

* blockchain: ops fix

* blockchain: readme fix
2024-07-12 22:15:02 +01:00
hinto-janai
fbae3df203
storage: move table generator macro blockchain -> database (#222)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* move table generator macro `blockchain` -> `database`

* blockchain: fix imports

* docs

* fix import ordering
2024-07-11 14:20:56 +01:00
Boog900
a82c08cc80
Storage: fix lifetimes (#215)
Some checks failed
Audit / audit (push) Has been cancelled
CI / fmt (push) Has been cancelled
CI / typo (push) Has been cancelled
CI / ci (macos-latest, stable, bash) (push) Has been cancelled
CI / ci (ubuntu-latest, stable, bash) (push) Has been cancelled
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Has been cancelled
Deny / audit (push) Has been cancelled
Doc / build (push) Has been cancelled
Doc / deploy (push) Has been cancelled
* fix db lifetimes

* fix redb

* fix blockchain with redb

* add docs
2024-07-06 13:21:46 +01:00
hinto-janai
71131a4836
ci: fix doc.yml (#213)
Some checks are pending
Audit / audit (push) Waiting to run
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Deny / audit (push) Waiting to run
Doc / build (push) Waiting to run
Doc / deploy (push) Blocked by required conditions
* types: remove borsh/serde

* blockchain: re-add optional serde

* doc.yml: remove `-D warnings`
2024-07-04 15:52:51 +01:00
Boog900
7c8466f4ba
Storage: add blockchain history requests (#206)
Some checks failed
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Audit / audit (push) Has been cancelled
Deny / audit (push) Has been cancelled
* Add database requests for chain history

* misc fixes

* review comments

* fix clippy

* add link and fix typo

* Apply suggestions from code review

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>

* add comment

---------

Co-authored-by: hinto-janai <hinto.janai@protonmail.com>
2024-07-02 23:08:19 +01:00
hinto-janai
6ce177aeca
storage: add key sorting (#198)
Some checks are pending
Audit / audit (push) Waiting to run
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Deny / audit (push) Waiting to run
* database: modify `trait Key`, don't blanket impl

* heed: create `KeyHeed<T>` wrapper type

* fix backend/tests

* blockchain: `impl Key PreRctOutputId`

* database: `StorableStr`, docs, tests

* key: docs, cleanup

* fixes

* heed: simplify types

* storable: remove doc

* heed: use `INTEGER_KEY` instead of custom compare fn

* add docs, tests

* database: document `create_db` invariant

* key: `Lexicographic` -> `Default`

* redb: fix `clear_db` behavior

* fix docs
2024-07-01 20:24:48 +01:00
hinto-janai
a438279aa8
storage: split cuprate-blockchain <-> cuprate-database (#160)
* storage: port some code `cuprate-blockchain` -> `database`

* database: remove `Tables` references

* database: remove old `cuprate-blockchain` type references

* find/replace `cuprate_blockchain` -> `database`, add `create_db()`

* database: fix redb

* database: use readme for docs, link in `lib.rs`

* database: fix `open_db_ro`, `open_db_rw`, `create_db` behavior

* database: add open table tests

* database: fix tests, remove blockchain specific references

* database: remove `ReaderThreads`, make `db_directory` mandatory

* initial `cuprate-blockchain` split

* fix doc links

* rename, fix database config

* blockchain: create `crate::open()`, `OpenTables::create_tables()`

* more compat fixes

* fix imports

* fix conflicts

* align cargo.toml

* docs

* fixes

* add `unused_crate_dependencies` lint, fix

* blockchain: add open table tests
2024-06-26 22:51:06 +01:00
hinto-janai
4b93dbec4c
workspace: enforce crate/directory naming scheme (#164)
Some checks failed
CI / fmt (push) Waiting to run
CI / typo (push) Waiting to run
CI / ci (macos-latest, stable, bash) (push) Waiting to run
CI / ci (ubuntu-latest, stable, bash) (push) Waiting to run
CI / ci (windows-latest, stable-x86_64-pc-windows-gnu, msys2 {0}) (push) Waiting to run
Audit / audit (push) Has been cancelled
Deny / audit (push) Has been cancelled
* rename all directories and crates

* fix all `use`

* fix doc link

* `dandelion/` -> `dandelion-tower/`

* fix epee-encoding test

* fix `json-rpc`

* fix pruning

* crate import fixes

* fix leftover merge conflicts

* fix `epee-encoding`
2024-06-24 02:30:47 +01:00