Commit graph

84 commits

Author SHA1 Message Date
Howard Chu
30f92f5630 Fix hf when import with verify off
Delete the hf tables, so the next open will rescan and regenerate
2016-01-15 17:26:19 +00:00
Riccardo Spagni
de03926850
updated copyright year 2015-12-31 08:39:56 +02:00
Riccardo Spagni
810a11267c
fixed copyrights with bad year references 2015-12-31 08:37:27 +02:00
Riccardo Spagni
fd36eea6dd
Merge pull request #572
b39aae7 Tweak 45800a25e9 (hyc)
4a5a5ff blockchain: always stop the ioservice before returning (moneromooo-monero)
78b65cf db_lmdb: safety close db at exit (moneromooo-monero)
45800a2 db_lmdb: fix a strdup/delete[] mistmatch (moneromooo-monero)
2015-12-30 09:38:41 +02:00
hyc
b39aae7aa5 Tweak 45800a25e9
trivial cleanup
2015-12-29 00:11:35 +00:00
moneromooo-monero
78b65cf7e8
db_lmdb: safety close db at exit
Noticed by hyc

Also set m_open to false when closed
2015-12-28 19:23:02 +00:00
moneromooo-monero
45800a25e9
db_lmdb: fix a strdup/delete[] mistmatch 2015-12-28 19:22:37 +00:00
warptangent
ee9d71e9f9
BlockchainDB: skip fixup check if read-only database 2015-12-26 14:30:20 -08:00
Riccardo Spagni
95ceb715dc
Merge pull request #565
79beed2 tests: fix various tests by using parameters better suited to monero (moneromooo-monero)
d0a8362 tests: fix some double spending tests (moneromooo-monero)
2358d0d tests: use 255 as a "too high" block version (moneromooo-monero)
f33a88c blockchain: fix a few block addition bugs (moneromooo-monero)
a9ff11c blockchain: fix an off by one error in unlocked time check (moneromooo-monero)
f294be3 blockchain: reinstate double spending checks in check_tx_inputs (moneromooo-monero)
737b6d6 blockchain: make some flag twiddling code closer to the original (moneromooo-monero)
81cb0fc blockchain: fix bitflipping test with quantized block rewards (moneromooo-monero)
22ddf09 blockchain: add missing m_tx_pool.on_blockchain_dec (moneromooo-monero)
d837c0c blockchain: fix switch to alternative blockchain for more than one block (moneromooo-monero)
5cec076 blockchain: add a missing validity check to rollback_blockchain_switching (moneromooo-monero)
3cabdb5 core: catch exceptions from get_output_key (moneromooo-monero)
5eef645 db: throw when given a non txout_to_key output to add (moneromooo-monero)
2015-12-26 10:23:17 +02:00
moneromooo-monero
5eef64578b
db: throw when given a non txout_to_key output to add
The check was explicit in the original version, so it seems
safer to make it explicit here, especially as it is now done
implicitely in a different place, away from the original check.
2015-12-25 21:56:37 +00:00
warptangent
d4c2fae2fb
BlockchainDB: Remove txs in reverse order
Data should be removed in the reverse order it was added. Not doing so
breaks assumptions and can cause problems in other DB implementations.

This matches the order of tx removal in
blockchain_storage::purge_block_data_from_blockchain.
2015-12-24 15:09:09 -08:00
warptangent
ffcf6bdb95
BlockchainLMDB: When removing, find amount output index fast by starting at end
This improves blockchain reorganization time by allowing one of the more
expensive DB lookups when popping a block to not have to seek through a
long dup list in the "output_amounts" subdb. This is most noticeable for
HDDs.

As before, the dup list is still walked if necessary (but in reverse),
and the global output index still confirmed to be the one looked for.
But under proper use, the result will be found at the end of the dup
list, so we start there.

Removing an amount output index is always done in the context of popping
a block, so the global output index being looked for should be the last
one in that amount key's dup list. Even if the txs themselves aren't
removed in reverse order (supposed to be according to original
implementation), the specified amount output index will still be near
the end, because the txs are in the same block.

TEST:

Pop blocks with blockchain_import.

Blocks should be successfully removed with no errors shown.

bitmonerod should be able to start syncing from the reduced blockchain
height.
2015-12-24 13:32:35 -08:00
warptangent
f11def012f
BlockchainLMDB: Remove tx outputs in reverse order
Data should be removed in the reverse order it was added.

This matches the order of removal in
blockchain_storage::pop_transaction_from_global_index.
2015-12-24 13:30:12 -08:00
moneromooo-monero
3f2970fadb
Add missing semicolons after log statements 2015-12-19 14:47:37 +00:00
warptangent
725acc7f17
Replace tabs with two spaces for consistency with rest of codebase
Remove trailing whitespace in same files.
2015-12-15 06:22:06 -08:00
moneromooo-monero
d462c62343
db_lmdb: always use a batch tx if possible
Also wrap that code in two prefix/suffix macros so they are
easier to use and not forget. Also add consts to returned data
casts where applicable.
2015-12-14 18:47:13 +00:00
Riccardo Spagni
0eb3b6645c
Merge pull request #532
79ff6a5 Add LMDB error string to database transaction exception message (warptangent)
2015-12-13 22:16:05 +02:00
Riccardo Spagni
bdf738bc7f
Merge pull request #531
cbded43 core_tests: fix ring_signature_1 tests (moneromooo-monero)
c3d208f core_tests: bump default test fee to 0.02 monero (moneromooo-monero)
10da0a0 add a --fakechain argument for tests (moneromooo-monero)
eee44e6 unit_tests: fix block reward test using post hard fork settings (moneromooo-monero)
595893f blockchain: log block (not chain) height in "BLOCK SUCCESFULLY ADDED" (moneromooo-monero)
2369968 blockchain: fix off by one in get_blocks (moneromooo-monero)
8af913a db_lmdb: implement BlockchainLMDB::reset (moneromooo-monero)
4833f4f db_bdb: implement BlockchainBDB::reset (moneromooo-monero)
18bf06e tx_pool: fix "minumim" typo in message (moneromooo-monero)
44f1267 tests: fix a typo in test name (moneromooo-monero)
1494557 db_lmdb: create all needed directories, not just the leaf one (moneromooo-monero)
015b68a db_bdb: create all needed directories, not just the leaf one (moneromooo-monero)
f141869 tests: remove data-dir argument registration (moneromooo-monero)
2015-12-13 22:14:56 +02:00
warptangent
79ff6a5a56
Add LMDB error string to database transaction exception message
When throwing an exception from being unable to begin an LMDB
transaction, include the reason.

It's often been due to a write transaction attempted within a write
transaction (batch mode), but there can be other reasons such as write
transaction attempted while database was opened read only, or
environment's map needs to be resized.
2015-12-13 09:09:46 -08:00
moneromooo-monero
8af913a3d1
db_lmdb: implement BlockchainLMDB::reset
It is needed by the core tests
2015-12-13 11:14:06 +00:00
moneromooo-monero
4833f4f96f
db_bdb: implement BlockchainBDB::reset
It is needed by the core tests
2015-12-13 11:14:00 +00:00
moneromooo-monero
1494557633
db_lmdb: create all needed directories, not just the leaf one 2015-12-13 11:09:43 +00:00
moneromooo-monero
015b68a2be
db_bdb: create all needed directories, not just the leaf one 2015-12-13 11:08:59 +00:00
Howard Chu
d98d5e79cb Fix data refs in read-only txns
Data is only guaranteed to be valid within the lifetime of a txn.
You cannot use data returned from LMDB after the txn ends.
Also, fixed a missing txn.commit BlockchainLMDB::get_tx_unlock_time()
2015-12-10 01:45:40 +00:00
Riccardo Spagni
8a5ba475cb
Merge pull request #524
1c8262c db_bdb: move log_set_config call before open (moneromooo-monero)
2015-12-08 20:11:06 +02:00
Riccardo Spagni
1fc1bfb677
Merge pull request #523
8620b31 db_bdb: support for libdb without DB_FORCESYNC (moneromooo-monero)
2015-12-08 20:10:34 +02:00
moneromooo-monero
1c8262c527
db_bdb: move log_set_config call before open
This is a precaution for older Berkeley DB versions.

- smooth reports an issue running with 4.7:
  DB_ENV->log_set_config: DB_LOG_IN_MEMORY: method not permitted
  after handle's open method
- this works just fine with 5.3
- we do not use DB_LOG_IN_MEMORY, but we use DB_LOG_AUTO_REMOVE
- libdb docs say some flags must be set before open, and some
  may be set at any time, but never say some must be set after
  open
- moving the call to log_set_config before open works with 5.3

Therefore, it seems best to move the call before open.
2015-12-07 19:22:22 +00:00
moneromooo-monero
8620b31019
db_bdb: support for libdb without DB_FORCESYNC 2015-12-07 09:31:53 +00:00
moneromooo-monero
a98e976f9e
blockchain_db: fixup missing key images in early DB version
Early DB versions did not store key images for inputs if the
transaction spending them had no outputs (ie, all fee). This
is not correct, as this would allow these outputs to be double
spent. This was fixed in 533acc30ed
a few months ago, but databases having synced blocks 2021612 and
685498 with a faulty version will be missing those key images
in the spent keys database. This code checks for this, and adds
those key images if they are missing.
2015-12-06 21:55:05 +00:00
moneromooo-monero
a3c5ca077c
blockchain_db: make the indexing base a BlockchainDB virtual function 2015-12-05 18:41:29 +00:00
moneromooo-monero
3bf35e14e3
db_bdb: read 32 bit heights from keys
Keys in Berkeley DB are 32 bits. We don't want to read random
bits in the high part.
2015-12-04 22:25:00 +00:00
moneromooo-monero
1b40952754
Revert "db_bdb: record numbers for recno databases start at 1"
It looks like some of the indices passed to the DB access functions
are already bumped by 1. Moreover, the existing code was not
throwing DB errors with 0 keys, and this is unlikely if it really
was using 0 keys. Last, this patch broke sync from scratch in at
least one case. So I'm calling it bad and reverting it.

This reverts commit bfc97401ae.
2015-11-13 09:28:17 +00:00
moneromooo-monero
4f873bcbaa
Remove some old/obsolete/unused code
git history's here if needed to get any of this back
2015-10-27 10:01:20 +00:00
moneromooo-monero
ecf0b83406
db_bdb: add versioning, to detect incompatible format changes 2015-10-26 18:09:55 +00:00
moneromooo-monero
f7e99047e4
db_lmdb: add versioning, to detect incompatible format changes 2015-10-26 18:09:46 +00:00
moneromooo-monero
5f397e4412
Add functions to iterate through blocks, txes, outputs, key images 2015-10-25 12:36:11 +00:00
moneromooo-monero
0a5a5e8597
db_bdb: record numbers for recno databases start at 1 2015-10-25 12:36:06 +00:00
moneromooo-monero
50dfdc057a
db_bdb: DB_KEYEMPTY is also not found for non-top recon fields 2015-10-25 12:35:59 +00:00
moneromooo-monero
572780e335
blockchain_db: use the DNE exceptions where appropriate 2015-10-25 12:35:52 +00:00
moneromooo-monero
791d8cb038
db_bdb: fix hard fork keys
Berkeley DB requires RECNO keys to be 32 bits, and forbids a key
value of 0.
2015-10-04 19:00:37 +01:00
moneromooo-monero
5b11a89a76
hardfork: most state now saved to the DB
There will be a delay on first load of an existing blockchain
as it gets reparsed for this state data.
2015-09-20 18:42:52 +01:00
moneromooo-monero
e63b854967
blockchain_db: match tx addition semantics to original code
The original code removed key images from a tx from the blockchain
when an non to-key nor gen input was found in that tx. Additionally,
the remainder of the tx data was added to the blockchain only after
the double spend check passed.
2015-08-11 11:11:17 +01:00
Riccardo Spagni
b210b304a3
Merge pull request #364
533acc3 blockchain_db: store key images for tx without outputs too (moneromooo-monero)
2015-08-10 20:05:12 +02:00
Riccardo Spagni
09cada45c4
Merge pull request #363
4f19e68 blockchain: factor get_num_outpouts(amount) calls (moneromooo-monero)
275894c blockchain: always select random outs using triangular distribution (moneromooo-monero)
2015-08-10 20:04:32 +02:00
moneromooo-monero
533acc30ed
blockchain_db: store key images for tx without outputs too 2015-08-09 19:26:36 +01:00
moneromooo-monero
275894cdef
blockchain: always select random outs using triangular distribution
It was only used by the older blockchain_storage.
We also move the code to the calling blockchain level, to avoid
replicating the code in every DB implementation. This also makes
the get_random_out method obsolete, and we delete it.
2015-08-09 18:07:44 +01:00
warptangent
525bf5811f
Fix estimation of batch storage size when no blocks exist
If there's no blocks in database (m_height == 0):
  Don't assign incorrect block range to check.
  Skip average block size check.

Test:

Run blockchain_converter with an existing source blockchain.bin and
a non-existent LMDB destination database.

The converter creates a BlockchainLMDB instance with zero height, due to
not being initialized with a genesis block, normally done by
Blockchain::init(). While different than the behavior of bitmonerod,
blockchain_import, and blockchain_export, the initialization hasn't been
strictly necessary.

The db batch size estimation normally uses an average block size, or a
default minimum block size, whichever is greater. In this case, as
there's no existing blocks to check for an average block size, the
default should be used.
2015-08-04 17:11:30 -07:00
warptangent
71793ef43f Add batch support to BlockchainLMDB::get_output_key
This allows blockchain_import to work with batch and verify modes enabled
(the default).
2015-07-16 00:36:26 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
94ea3e8ed2 Removed on_idle() calls to Blockchain::store_blockchain() for lmdb.
Added option to cache tx-input verification results.
2015-07-15 23:20:25 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
2e293a563e Fixed binary size issue due to embedded checkpoint data.
Fixed OSX compilation issues due to random lmdb resize points.
Fixed infinite loop bug when calling core::get_block_template(..).
2015-07-15 23:20:20 -07:00