Commit graph

281 commits

Author SHA1 Message Date
moneromooo-monero
198f557d38
blockchain: use different hard fork settings for testnet and mainnet 2015-09-27 22:46:13 +01:00
moneromooo-monero
5b11a89a76
hardfork: most state now saved to the DB
There will be a delay on first load of an existing blockchain
as it gets reparsed for this state data.
2015-09-20 18:42:52 +01:00
moneromooo-monero
0a54c3a553
hardfork: remove the "parts are copyright cryptonote" notices
I coded the whole thing from scratch.
2015-09-20 10:05:15 +01:00
moneromooo-monero
e546f3724a
Add an RPC call and daemon command to get info on hard fork voting 2015-09-19 16:47:48 +01:00
moneromooo-monero
d06713199e
blockchain: force a hardfork recalculation at load time
Since the state isn't actually saved anywhere, as the archive
code isn't called in the new DB version.
2015-09-19 16:47:42 +01:00
moneromooo-monero
a7177610b3
core: add consts where appropriate 2015-09-19 16:47:35 +01:00
moneromooo-monero
8ffc508cef
core: moan when we think an update is needed to get latest hard fork info 2015-09-13 18:09:57 +01:00
moneromooo-monero
f85498422d
blockchain: use the new hardfork class 2015-09-12 11:15:53 +01:00
moneromooo-monero
62b1f74116
New hardfork class
This keeps track of voting via block version, in order to decide
when to enable a particular fork's code.
2015-09-12 11:14:44 +01:00
moneromooo-monero
bed9a44e56
blockchain: add a couple missing includes 2015-09-12 11:14:34 +01:00
moneromooo-monero
813e758b62
blockchain: remove obsolete call to libc srand
crypto::rand is now used for output selection
2015-08-24 21:58:19 +01:00
moneromooo-monero
d91eb8c7b4
wallet: only return tx keys via RPC if requested
To get the tx keys returned via RPC, set the "get_tx_key" or
"get_tx_keys" request field to true (defaults to false).
2015-08-24 17:36:44 +01:00
Riccardo Spagni
6b7be9f89b
Merge pull request #383
3b5330e use correct unsigned type (roman)
59cc92b removed some gcc warnings. mainly unused variables. (roman)
2015-08-24 19:21:04 +02:00
roman
59cc92b388 removed some gcc warnings. mainly unused variables. 2015-08-23 17:59:24 +02:00
moneromooo-monero
6c995710d8
make tx keys available to the user
They are also stored in the cache file, to be retrieved using
a new get_tx_key command.
2015-08-19 21:11:48 +01:00
moneromooo-monero
378d004b37
blockchain: mark two places where the new code differs from the old
And I'd like a comment from tewinget or someone else
2015-08-15 18:46:19 +01:00
moneromooo-monero
73d42a75d4
blockchain: update cumulative size after block addition
Block addition can fail, and the old code would not update the
cumulative size in that case.
2015-08-15 18:44:56 +01:00
moneromooo-monero
4a443775e8
blockchain: remove dead code 2015-08-15 18:44:31 +01:00
moneromooo-monero
3f9089a767
blockchain: do not try to add a tx the pool when it was nor taken out
This is an unintended difference from the old code. Though I don't
think it can actually happen in practice with the current take_tx
implementation.
2015-08-15 18:42:29 +01:00
moneromooo-monero
769d5ef0e6
blockchain: fix off by 1 in timestamp median calculations
The height function apparently used to return the index of
the last block, rather than the height of the chain. This now
seems to be incorrect, judging the the code, so we remove the
now wrong comment, as well as a couple +/- 1 adjustments
which now cause the median calculation to differ from the
original blockchain_storage version.
2015-08-15 12:37:23 +01:00
moneromooo-monero
83bbea4c7f
Add a is_key_image_spent daemon command and RPC call 2015-08-11 10:52:41 +01:00
moneromooo-monero
35abef1b92
blockchain: remove dead code 2015-08-11 10:48:51 +01:00
Riccardo Spagni
09cada45c4
Merge pull request #363
4f19e68 blockchain: factor get_num_outpouts(amount) calls (moneromooo-monero)
275894c blockchain: always select random outs using triangular distribution (moneromooo-monero)
2015-08-10 20:04:32 +02:00
moneromooo-monero
4f19e68476
blockchain: factor get_num_outpouts(amount) calls
It has to stay constant as we get the blockchain lock for the
entire function. Avoids some unnecessary DB accesses.
2015-08-09 18:14:30 +01:00
moneromooo-monero
275894cdef
blockchain: always select random outs using triangular distribution
It was only used by the older blockchain_storage.
We also move the code to the calling blockchain level, to avoid
replicating the code in every DB implementation. This also makes
the get_random_out method obsolete, and we delete it.
2015-08-09 18:07:44 +01:00
moneromooo-monero
a2d7a5fb49
encrypted payment ids are now 64 bit, instead of 256 bit
Pros:
 - smaller on the blockchain
 - shorter integrated addresses

Cons:
 - less sparseness
 - less ability to embed actual information

The boolean argument to encrypt payment ids is now gone from the
RPC calls, since the decision is made based on the length of the
payment id passed.
2015-08-09 10:13:51 +01:00
moneromooo-monero
e40cfc4e29
Encrypted payment IDs
A payment ID may be encrypted using the tx secret key and the
receiver's public view key. The receiver can decrypt it with
the tx public key and the receiver's secret view key.

Using integrated addresses now cause the payment IDs to be
encrypted. Payment IDs used manually are not encrypted by default,
but can be encrypted using the new 'encrypt_payment_id' field
in the transfer and transfer_split RPC calls. It is not possible
to use an encrypted payment ID by specifying a manual simplewallet
transfer/transfer_new command, though this is just a limitation
due to input parsing.
2015-08-09 10:13:41 +01:00
Riccardo Spagni
5a26676932
Merge pull request #343
e20a4dd blockchain: fix testnet syncing (to not use blocks.dat) (moneromooo-monero)
2015-07-18 22:59:02 +02:00
moneromooo-monero
e20a4ddc76
blockchain: fix testnet syncing (to not use blocks.dat)
These are mainnet blocks, and would cause syncing on testnet to
reject all incoming blocks.
2015-07-18 10:25:22 +01:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
70ae2ee711 Fixed threadpool bug when running on single core systems.
*Thanks to freshman for reporting bug.
2015-07-17 20:02:29 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
94ea3e8ed2 Removed on_idle() calls to Blockchain::store_blockchain() for lmdb.
Added option to cache tx-input verification results.
2015-07-15 23:20:25 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
2e293a563e Fixed binary size issue due to embedded checkpoint data.
Fixed OSX compilation issues due to random lmdb resize points.
Fixed infinite loop bug when calling core::get_block_template(..).
2015-07-15 23:20:20 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
e5d2680094 ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
2015-07-15 23:20:16 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
1f83444d3d Update blockchain.cpp
Fix compilation error
2015-07-15 23:20:15 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
da1d3c01de
Experimental BDB workaround optimizations 2015-07-15 21:13:42 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
5d304cabfd Fix loop bug when calling core::get_block_template, causing calling thread to lock up. 2015-07-10 22:09:21 -07:00
moneromooo-monero
6a0f61d800
account: allow creating an account from a public address and view secret key 2015-06-20 17:33:06 +01:00
moneromooo-monero
f6da25a32e
Fix standard address deserialization 2015-06-16 15:36:20 +01:00
Riccardo Spagni
2d9d6c7621
Merge pull request #321
bbc5475 Fix DNS checkpoint consensus code (moneromooo-monero)
2015-06-14 13:10:18 +02:00
moneromooo-monero
bbc5475418
Fix DNS checkpoint consensus code
It's supposed to load all records and pick one that it finds twice.
2015-06-14 11:05:57 +01:00
moneromooo-monero
7bd6efe313
account: add a couple consts 2015-06-12 16:48:52 +01:00
moneromooo-monero
63741d8264
Integrated addresses (standard address plus payment id) 2015-06-12 16:48:41 +01:00
Riccardo Spagni
5bee2d2edf
Merge pull request #303
c882af6 wallet: add watch only wallet support (moneromooo-monero)
f7767c6 account: add a forget_spend_key method (moneromooo-monero)
2015-06-11 10:27:00 +02:00
moneromooo-monero
f7767c6508
account: add a forget_spend_key method 2015-05-31 15:32:54 +01:00
Riccardo Spagni
e01d32e52d
cleaning up, removing redundant files, renaming, fixing incorrect licenses 2015-05-31 13:40:18 +02:00
warptangent
d1eac1b71c
Support debugging command --pop-blocks on in-memory blockchain
Add public method blockchain_storage::debug_pop_block_from_blockchain()

Ensure blockchain_import calls destructors before exit.

To test:

DATABASE=memory make release

// create blockchain.bin from blockchain.raw if needed
build/release/bin/blockchain_import --block-stop 1000

// try popping a single block
build/release/bin/blockchain_import --pop-blocks 1
2015-05-16 19:38:52 -07:00
Thomas Winget
b1d92bcc37
Fixes changes to sort tx by fee per kb 2015-05-13 20:27:06 -04:00
Riccardo Spagni
1d42deb767
Merge pull request #281
ac011b4 Rename src/blockchain_converter/ to src/blockchain_utilities/ (warptangent)
ed9c639 Add --block-number option to blockchain_import (warptangent)
1eb4c66 Update blockchain utilities with portable bootstrap file format (warptangent)
54bd9c1 Add MDB_NORDAHEAD as a supported LMDB flag for blockchain_import (warptangent)
a52496d Condense #if directives (warptangent)
8c1a188 Add basic "pop blocks" command to blockchain_import for debugging (warptangent)
71af046 Update log statements (warptangent)
2015-05-13 11:21:42 +02:00
Riccardo Spagni
012164fff8
resolved merge conflict in tx_pool.cpp 2015-05-13 11:18:22 +02:00
warptangent
71af04669c
Update log statements
Use filesystem path conversion to string() instead of c_str().
Windows may otherwise output an address.
2015-05-08 14:12:06 -07:00