monero-project/monero
monero-project/monero
1
0
Fork 0
mirror of https://github.com/monero-project/monero.git synced 2024-12-22 19:49:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)

Browse source 
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
This commit is contained in:
NoodleDoodleNoodleDoodleNoodleDoodleNoo 2015-07-10 13:09:32 -07:00
parent 1f83444d3d
commit e5d2680094
33 changed files with 4052 additions and 2373 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

76
CMakeLists.txt
View file

@ -45,6 +45,35 @@ function (die msg)
message(FATAL_ERROR "${BoldRed}${msg}${ColourReset}")
endfunction ()
if (NOT ${ARCH} STREQUAL "")
string(SUBSTRING ${ARCH} 0 5 ARM_TEST)
string(TOLOWER ${ARM_TEST} ARM_TEST)
if (${ARM_TEST} STREQUAL "armv6")
set(ARM6 1)
else()
set(ARM6 0)
endif()
if (${ARM_TEST} STREQUAL "armv7")
set(ARM7 1)
else()
set(ARM7 0)
endif()
endif()
if(WIN32 OR ARM7 OR ARM6)
set(CMAKE_C_FLAGS_RELEASE "-O2 -DNDEBUG")
set(CMAKE_CXX_FLAGS_RELEASE "-O2 -DNDEBUG")
endif()
# set this to 0 if per-block checkpoint needs to be disabled
set(PER_BLOCK_CHECKPOINT 1)
if(PER_BLOCK_CHECKPOINT)
add_definitions("-DPER_BLOCK_CHECKPOINT")
endif()
list(INSERT CMAKE_MODULE_PATH 0
"${CMAKE_SOURCE_DIR}/cmake")
@ -156,14 +185,42 @@ if (DEFINED ENV{DATABASE})
else()
message(STATUS "Could not find DATABASE in env (not required unless you want to change database type from default: ${DATABASE})")
endif()
set(BERKELEY_DB 0)
if (DATABASE STREQUAL "lmdb")
set(BLOCKCHAIN_DB DB_LMDB)
# temporarily allow mingw to compile with berkeley_db,
# regardless if building static or not
if(NOT STATIC OR MINGW)
find_package(BerkeleyDB)
if(NOT BERKELEY_DB_LIBRARIES)
message(STATUS "BerkeleyDB not found and has been disabled.")
else()
message(STATUS "Found BerkeleyDB include (db.h) in ${BERKELEY_DB_INCLUDE_DIR}")
if(BERKELEY_DB_LIBRARIES)
message(STATUS "Found BerkeleyDB shared library")
set(BDB_STATIC false CACHE BOOL "BDB Static flag")
set(BDB_INCLUDE ${BERKELEY_DB_INCLUDE_DIR} CACHE STRING "BDB include path")
set(BDB_LIBRARY ${BERKELEY_DB_LIBRARIES} CACHE STRING "BDB library name")
set(BDB_LIBRARY_DIRS "" CACHE STRING "BDB Library dirs")
set(BERKELEY_DB 1)
else()
message(STATUS "Found BerkeleyDB includes, but could not find BerkeleyDB library. Please make sure you have installed libdb and libdb-dev or the equivalent")
endif()
endif()
endif()
elseif (DATABASE STREQUAL "memory")
set(BLOCKCHAIN_DB DB_MEMORY)
else()
die("Invalid database type: ${DATABASE}")
endif()
if(BERKELEY_DB)
add_definitions("-DBERKELEY_DB")
endif()
add_definitions("-DBLOCKCHAIN_DB=${BLOCKCHAIN_DB}")
if (UNIX AND NOT APPLE)
@ -192,7 +249,7 @@ include_directories(external/rapidjson)
include_directories(${LMDB_INCLUDE})
# Final setup for Berkeley DB
if (NOT STATIC)
if (BERKELEY_DB)
include_directories(${BDB_INCLUDE})
endif()
@ -208,13 +265,14 @@ if(MSVC)
include_directories(SYSTEM src/platform/msc)
else()
set(ARCH native CACHE STRING "CPU to build for: -march value or default")
if(ARCH STREQUAL "default")
# -march=armv7-a conflicts with -mcpu=cortex-a7
if(ARCH STREQUAL "default" OR ARM7)
set(ARCH_FLAG "")
else()
if(ARCH STREQUAL "x86_64")
set(ARCH_FLAG "-march=x86-64")
else()
set(ARCH_FLAG "-march=${ARCH}")
set(ARCH_FLAG "-march=${ARCH}")
endif()
endif()
set(WARNINGS "-Wall -Wextra -Wpointer-arith -Wundef -Wvla -Wwrite-strings -Wno-error=extra -Wno-error=deprecated-declarations -Wno-error=sign-compare -Wno-error=strict-aliasing -Wno-error=type-limits -Wno-unused-parameter -Wno-error=unused-variable -Wno-error=undef -Wno-error=uninitialized")
@ -258,14 +316,18 @@ else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -D_GNU_SOURCE ${MINGW_FLAG} ${WARNINGS} ${CXX_WARNINGS} ${ARCH_FLAG} -maes")
endif()
string(SUBSTRING ${ARCH} 0 3 ARM_TEST)
string(TOLOWER ${ARM_TEST} ARM_TEST)
if(${ARM_TEST} STREQUAL "arm")
message(STATUS "Setting ARM C and C++ flags")
if(ARM6)
message(STATUS "Setting ARM6 C and C++ flags")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mfpu=vfp -mfloat-abi=hard")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mfpu=vfp -mfloat-abi=hard")
endif()
if(ARM7)
message(STATUS "Setting ARM7 C and C++ flags")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -O2 -mcpu=cortex-a7 -mfloat-abi=hard -mfpu=vfpv4 -funsafe-math-optimizations -mtune=cortex-a7")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O2 -mcpu=cortex-a7 -mfloat-abi=hard -mfpu=vfpv4 -funsafe-math-optimizations -mtune=cortex-a7")
endif()
if(APPLE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DGTEST_HAS_TR1_TUPLE=0")
endif()

12
Makefile
View file

@ -58,6 +58,14 @@ release-all:
mkdir -p build/release
cd build/release && cmake -D BUILD_TESTS=ON -D CMAKE_BUILD_TYPE=release ../.. && $(MAKE)
release-arm6:
mkdir -p build/release
cd build/release && cmake -D BUILD_TESTS=OFF -D ARCH="armv6zk" -D BUILD_64=OFF -D NO_AES=ON -D CMAKE_BUILD_TYPE=release ../.. && $(MAKE)
release-arm7:
mkdir -p build/release
cd build/release && cmake -D BUILD_TESTS=OFF -D ARCH="armv7-a" -D BUILD_64=OFF -D NO_AES=ON -D CMAKE_BUILD_TYPE=release ../.. && $(MAKE)
release-static: release-static-64
release-static-64:
@ -68,10 +76,6 @@ release-static-32:
mkdir -p build/release
cd build/release && cmake -D STATIC=ON -D ARCH="i686" -D BUILD_64=OFF -D CMAKE_BUILD_TYPE=release ../.. && $(MAKE)
release-static-arm6:
mkdir -p build/release
cd build/release && cmake -D STATIC=ON -D ARCH="armv6zk" -D BUILD_64=OFF -D NO_AES=ON -D CMAKE_BUILD_TYPE=release ../.. && $(MAKE)
release-static-win64:
mkdir -p build/release
cd build/release && cmake -G "MSYS Makefiles" -D STATIC=ON -D ARCH="x86-64" -D BUILD_64=ON -D CMAKE_BUILD_TYPE=Release -D CMAKE_TOOLCHAIN_FILE=../../cmake/64-bit-toolchain.cmake -D MSYS2_FOLDER=c:/msys64 ../.. && $(MAKE)

20
external/db_drivers/CMakeLists.txt vendored
View file

@ -31,24 +31,4 @@
message(STATUS "Using ${ARCH_WIDTH}-bit LMDB from source tree")
add_subdirectory(liblmdb${ARCH_WIDTH})
set(LMDB_INCLUDE "${CMAKE_CURRENT_SOURCE_DIR}/liblmdb${ARCH_WIDTH}" CACHE STRING "LMDB Include path")
set(LMDB_LIBRARY "lmdb" CACHE STRING "LMDB Library name")
if (NOT STATIC)
find_package(BerkeleyDB)
if(NOT BERKELEY_DB_LIBRARIES)
die("BerkeleyDB not found. At this time it should be installed in your system for a non-static build.")
else()
message(STATUS "Found BerkeleyDB include (db.h) in ${BERKELEY_DB_INCLUDE_DIR}")
if(BERKELEY_DB_LIBRARIES)
message(STATUS "Found BerkeleyDB shared library")
set(BDB_STATIC false CACHE BOOL "BDB Static flag")
set(BDB_INCLUDE ${BERKELEY_DB_INCLUDE_DIR} CACHE STRING "BDB include path")
set(BDB_LIBRARY ${BERKELEY_DB_LIBRARIES} CACHE STRING "BDB library name")
set(BDB_LIBRARY_DIRS "" CACHE STRING "BDB Library dirs")
else()
die("Found BerkeleyDB includes, but could not find BerkeleyDB library. Please make sure you have installed libdb and libdb-dev or the equivalent")
endif()
endif()
endif()

4
src/CMakeLists.txt
View file

@ -103,3 +103,7 @@ add_subdirectory(daemonizer)
add_subdirectory(daemon)
add_subdirectory(blockchain_utilities)
if(PER_BLOCK_CHECKPOINT)
add_subdirectory(blocks)
endif()

4
src/blockchain_db/CMakeLists.txt
View file

@ -31,7 +31,7 @@ set(blockchain_db_sources
lmdb/db_lmdb.cpp
)
if (NOT STATIC)
if (BERKELEY_DB)
set(blockchain_db_sources
${blockchain_db_sources}
berkeleydb/db_bdb.cpp
@ -46,7 +46,7 @@ set(blockchain_db_private_headers
lmdb/db_lmdb.h
)
if (NOT STATIC)
if (BERKELEY_DB)
set(blockchain_db_private_headers
${blockchain_db_private_headers}
berkeleydb/db_bdb.h

904
src/blockchain_db/berkeleydb/db_bdb.cpp
View file

File diff suppressed because it is too large Load diff

172
src/blockchain_db/berkeleydb/db_bdb.h
View file

@ -30,6 +30,11 @@
#include "blockchain_db/blockchain_db.h"
#include "cryptonote_protocol/blobdatatype.h" // for type blobdata
#include <unordered_map>
// ND: Enables multi-threaded bulk reads for when getting indices.
// TODO: Disabled for now, as it doesn't seem to provide noticeable improvements (??. Reason: TBD.
// #define BDB_BULK_CAN_THREAD
namespace cryptonote
{
@ -83,10 +88,145 @@ struct bdb_txn_safe
{
return &m_txn;
}
private:
DbTxn* m_txn;
};
// ND: Class to handle buffer management when doing bulk queries
// (DB_MULTIPLE). Allocates buffers then handles thread queuing
// so a fixed set of buffers can be used (instead of allocating
// every time a bulk query is needed).
template <typename T>
class bdb_safe_buffer
{
// limit the number of buffers to 8
const size_t MaxAllowedBuffers = 8;
public:
bdb_safe_buffer(size_t num_buffers, size_t count)
{
if(num_buffers > MaxAllowedBuffers)
num_buffers = MaxAllowedBuffers;
set_count(num_buffers);
for (size_t i = 0; i < num_buffers; i++)
m_buffers.push_back((T) malloc(sizeof(T) * count));
m_buffer_count = count;
}
~bdb_safe_buffer()
{
for (size_t i = 0; i < m_buffers.size(); i++)
{
if (m_buffers[i])
{
free(m_buffers[i]);
m_buffers[i] = nullptr;
}
}
m_buffers.resize(0);
}
T acquire_buffer()
{
std::unique_lock<std::mutex> lock(m_lock);
m_cv.wait(lock, [&]{ return m_count > 0; });
--m_count;
size_t index = -1;
for (size_t i = 0; i < m_open_slot.size(); i++)
{
if (m_open_slot[i])
{
m_open_slot[i] = false;
index = i;
break;
}
}
assert(index >= 0);
T buffer = m_buffers[index];
m_buffer_map.emplace(buffer, index);
return buffer;
}
void release_buffer(T buffer)
{
std::unique_lock<std::mutex> lock(m_lock);
assert(buffer != nullptr);
auto it = m_buffer_map.find(buffer);
if (it != m_buffer_map.end())
{
auto index = it->second;
assert(index < m_open_slot.size());
assert(m_open_slot[index] == false);
assert(m_count < m_open_slot.size());
++m_count;
m_open_slot[index] = true;
m_buffer_map.erase(it);
m_cv.notify_one();
}
}
size_t get_buffer_size() const
{
return m_buffer_count * sizeof(T);
}
size_t get_buffer_count() const
{
return m_buffer_count;
}
typedef T type;
private:
void set_count(size_t count)
{
assert(count > 0);
m_open_slot.resize(count, true);
m_count = count;
}
std::vector<T> m_buffers;
std::unordered_map<T, size_t> m_buffer_map;
std::condition_variable m_cv;
std::vector<bool> m_open_slot;
size_t m_count;
std::mutex m_lock;
size_t m_buffer_count;
};
template <typename T>
class bdb_safe_buffer_autolock
{
public:
bdb_safe_buffer_autolock(T &safe_buffer, typename T::type &buffer) :
m_safe_buffer(safe_buffer), m_buffer(nullptr)
{
m_buffer = m_safe_buffer.acquire_buffer();
buffer = m_buffer;
}
~bdb_safe_buffer_autolock()
{
if (m_buffer != nullptr)
{
m_safe_buffer.release_buffer(m_buffer);
m_buffer = nullptr;
}
}
private:
T &m_safe_buffer;
typename T::type m_buffer;
};
class BlockchainBDB : public BlockchainDB
{
public:
@ -159,8 +299,9 @@ public:
virtual uint64_t get_num_outputs(const uint64_t& amount) const;
virtual crypto::public_key get_output_key(const uint64_t& amount, const uint64_t& index) const;
virtual output_data_t get_output_key(const uint64_t& amount, const uint64_t& index);
virtual output_data_t get_output_key(const uint64_t& global_index) const;
virtual void get_output_key(const uint64_t &amount, const std::vector<uint64_t> &offsets, std::vector<output_data_t> &outputs);
virtual tx_out get_output(const crypto::hash& h, const uint64_t& index) const;
/**
@ -175,9 +316,11 @@ public:
tx_out get_output(const uint64_t& index) const;
virtual tx_out_index get_output_tx_and_index_from_global(const uint64_t& index) const;
virtual void get_output_tx_and_index_from_global(const std::vector<uint64_t> &global_indices,
std::vector<tx_out_index> &tx_out_indices) const;
virtual tx_out_index get_output_tx_and_index(const uint64_t& amount, const uint64_t& index) const;
virtual void get_output_tx_and_index(const uint64_t& amount, std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices) const;
virtual tx_out_index get_output_tx_and_index(const uint64_t& amount, const uint64_t& index);
virtual void get_output_tx_and_index(const uint64_t& amount, const std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices);
virtual std::vector<uint64_t> get_tx_output_indices(const crypto::hash& h) const;
virtual std::vector<uint64_t> get_tx_amount_output_indices(const crypto::hash& h) const;
@ -198,7 +341,12 @@ public:
virtual void batch_abort();
virtual void pop_block(block& blk, std::vector<transaction>& txs);
virtual bool has_bulk_indices() const { return true; }
#if defined(BDB_BULK_CAN_THREAD)
virtual bool can_thread_bulk_indices() const { return true; }
#else
virtual bool can_thread_bulk_indices() const { return false; }
#endif
private:
virtual void add_block( const block& blk
@ -214,7 +362,7 @@ private:
virtual void remove_transaction_data(const crypto::hash& tx_hash, const transaction& tx);
virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index);
virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time);
virtual void remove_output(const tx_out& tx_output);
@ -227,6 +375,7 @@ private:
virtual void remove_spent_key(const crypto::key_image& k_image);
void get_output_global_indices(const uint64_t& amount, const std::vector<uint64_t> &offsets, std::vector<uint64_t> &global_indices);
/**
* @brief convert a tx output to a blob for storage
*
@ -253,10 +402,13 @@ private:
*
* @return the global index of the desired output
*/
uint64_t get_output_global_index(const uint64_t& amount, const uint64_t& index) const;
uint64_t get_output_global_index(const uint64_t& amount, const uint64_t& index);
void checkpoint_worker() const;
void check_open() const;
void *m_buffer;
bool m_run_checkpoint;
std::unique_ptr<boost::thread> m_checkpoint_thread;
typedef bdb_safe_buffer<void *> bdb_safe_buffer_t;
bdb_safe_buffer_t m_buffer;
DbEnv* m_env;

2
src/blockchain_db/blockchain_db.cpp
View file

@ -64,7 +64,7 @@ void BlockchainDB::add_transaction(const crypto::hash& blk_hash, const transacti
{
for (uint64_t i = 0; i < tx.vout.size(); ++i)
{
add_output(tx_hash, tx.vout[i], i);
add_output(tx_hash, tx.vout[i], i, tx.unlock_time);
}
for (const txin_v& tx_input : tx.vin)

27
src/blockchain_db/blockchain_db.h
View file

@ -138,6 +138,15 @@ namespace cryptonote
// typedef for convenience
typedef std::pair<crypto::hash, uint64_t> tx_out_index;
#pragma pack(push, 1)
struct output_data_t
{
crypto::public_key pubkey;
uint64_t unlock_time;
uint64_t height;
};
#pragma pack(pop)
/***********************************
* Exception Definitions
***********************************/
@ -279,7 +288,7 @@ private:
virtual void remove_transaction_data(const crypto::hash& tx_hash, const transaction& tx) = 0;
// tells the subclass to store an output
virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index) = 0;
virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time) = 0;
// tells the subclass to remove an output
virtual void remove_output(const tx_out& tx_output) = 0;
@ -313,7 +322,7 @@ protected:
mutable uint64_t time_tx_exists = 0;
uint64_t time_commit1 = 0;
bool m_auto_remove_logs = true;
public:
@ -461,7 +470,8 @@ public:
virtual uint64_t get_num_outputs(const uint64_t& amount) const = 0;
// return public key for output with global output amount <amount> and index <index>
virtual crypto::public_key get_output_key(const uint64_t& amount, const uint64_t& index) const = 0;
virtual output_data_t get_output_key(const uint64_t& amount, const uint64_t& index) = 0;
virtual output_data_t get_output_key(const uint64_t& global_index) const = 0;
// returns the output indexed by <index> in the transaction with hash <h>
virtual tx_out get_output(const crypto::hash& h, const uint64_t& index) const = 0;
@ -471,9 +481,11 @@ public:
// returns the transaction-local reference for the output with <amount> at <index>
// return type is pair of tx hash and index
virtual tx_out_index get_output_tx_and_index(const uint64_t& amount, const uint64_t& index) const = 0;
virtual void get_output_tx_and_index(const uint64_t& amount, std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices) const = 0;
virtual bool has_bulk_indices() const = 0;
virtual tx_out_index get_output_tx_and_index(const uint64_t& amount, const uint64_t& index) = 0;
virtual void get_output_tx_and_index(const uint64_t& amount, const std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices) = 0;
virtual void get_output_key(const uint64_t &amount, const std::vector<uint64_t> &offsets, std::vector<output_data_t> &outputs) = 0;
virtual bool can_thread_bulk_indices() const = 0;
// return a vector of indices corresponding to the global output index for
// each output in the transaction with hash <h>
@ -485,7 +497,10 @@ public:
// returns true if key image <img> is present in spent key images storage
virtual bool has_key_image(const crypto::key_image& img) const = 0;
void set_auto_remove_logs(bool auto_remove) { m_auto_remove_logs = auto_remove; }
bool m_open;
mutable epee::critical_section m_synchronization_lock;
}; // class BlockchainDB

464
src/blockchain_db/lmdb/db_lmdb.cpp
View file

@ -65,10 +65,19 @@ struct lmdb_cur
done = false;
}
~lmdb_cur() { close(); }
~lmdb_cur()
{
close();
}
operator MDB_cursor*() { return m_cur; }
operator MDB_cursor**() { return &m_cur; }
operator MDB_cursor*()
{
return m_cur;
}
operator MDB_cursor**()
{
return &m_cur;
}
void close()
{
@ -87,7 +96,8 @@ private:
template<typename T>
struct MDB_val_copy: public MDB_val
{
MDB_val_copy(const T &t): t_copy(t)
MDB_val_copy(const T &t) :
t_copy(t)
{
mv_size = sizeof (T);
mv_data = &t_copy;
@ -99,7 +109,8 @@ private:
template<>
struct MDB_val_copy<cryptonote::blobdata>: public MDB_val
{
MDB_val_copy(const cryptonote::blobdata &bd): data(new char[bd.size()])
MDB_val_copy(const cryptonote::blobdata &bd) :
data(new char[bd.size()])
{
memcpy(data.get(), bd.data(), bd.size());
mv_size = bd.size();
@ -109,7 +120,8 @@ private:
std::unique_ptr<char[]> data;
};
auto compare_uint64 = [](const MDB_val *a, const MDB_val *b) {
auto compare_uint64 = [](const MDB_val *a, const MDB_val *b)
{
const uint64_t va = *(const uint64_t*)a->mv_data;
const uint64_t vb = *(const uint64_t*)b->mv_data;
if (va < vb) return -1;
@ -117,6 +129,20 @@ auto compare_uint64 = [](const MDB_val *a, const MDB_val *b) {
else return 1;
};
int compare_hash32(const MDB_val *a, const MDB_val *b)
{
uint32_t *va = (uint32_t*) a->mv_data;
uint32_t *vb = (uint32_t*) b->mv_data;
for (int n = 7; n >= 0; n--)
{
if (va[n] == vb[n])
continue;
return va[n] < vb[n] ? -1 : 1;
}
return 0;
}
const char* const LMDB_BLOCKS = "blocks";
const char* const LMDB_BLOCK_TIMESTAMPS = "block_timestamps";
const char* const LMDB_BLOCK_HEIGHTS = "block_heights";
@ -235,6 +261,26 @@ void mdb_txn_safe::allow_new_txns()
void BlockchainLMDB::do_resize(uint64_t increase_size)
{
CRITICAL_REGION_LOCAL(m_synchronization_lock);
const uint64_t add_size = 1LL << 30;
// check disk capacity
try
{
boost::filesystem::path path(m_folder);
boost::filesystem::space_info si = boost::filesystem::space(path);
if(si.available < add_size)
{
LOG_PRINT_RED_L0("!! WARNING: Insufficient free space to extend database !!: " << si.available / 1LL << 20L);
return;
}
}
catch(...)
{
// print something but proceed.
LOG_PRINT_YELLOW("Unable to query free disk space.", LOG_LEVEL_0);
}
MDB_envinfo mei;
mdb_env_info(m_env, &mei);
@ -250,6 +296,9 @@ void BlockchainLMDB::do_resize(uint64_t increase_size)
if (increase_size > 0)
new_mapsize = mei.me_mapsize + increase_size;
// add 1Gb per resize, instead of doing a percentage increase
// uint64_t new_mapsize = (double) mei.me_mapsize + add_size;
new_mapsize += (new_mapsize % mst.ms_psize);
mdb_txn_safe::prevent_new_txns();
@ -270,9 +319,7 @@ void BlockchainLMDB::do_resize(uint64_t increase_size)
mdb_env_set_mapsize(m_env, new_mapsize);
LOG_PRINT_L0("LMDB Mapsize increased."
<< " Old: " << mei.me_mapsize / (1024 * 1024) << "MiB"
<< ", New: " << new_mapsize / (1024 * 1024) << "MiB");
LOG_PRINT_GREEN("LMDB Mapsize increased." << " Old: " << mei.me_mapsize / (1024 * 1024) << "MiB" << ", New: " << new_mapsize / (1024 * 1024) << "MiB", LOG_LEVEL_0);
mdb_txn_safe::allow_new_txns();
}
@ -280,6 +327,7 @@ void BlockchainLMDB::do_resize(uint64_t increase_size)
// threshold_size is used for batch transactions
bool BlockchainLMDB::need_resize(uint64_t threshold_size) const
{
#if defined(ENABLE_AUTO_RESIZE)
MDB_envinfo mei;
mdb_env_info(m_env, &mei);
@ -311,12 +359,19 @@ bool BlockchainLMDB::need_resize(uint64_t threshold_size) const
return false;
}
if ((double)size_used / mei.me_mapsize > RESIZE_PERCENT)
std::mt19937 engine(std::random_device{}());
std::uniform_real_distribution<double> fdis(0.6, 0.9);
double resize_percent = fdis(engine);
if ((double)size_used / mei.me_mapsize > resize_percent)
{
LOG_PRINT_L1("Threshold met (percent-based)");
return true;
}
return false;
#else
return false;
#endif
}
void BlockchainLMDB::check_and_resize_for_batch(uint64_t batch_num_blocks)
@ -389,12 +444,8 @@ uint64_t BlockchainLMDB::get_estimated_batch_size(uint64_t batch_num_blocks) con
return threshold_size;
}
void BlockchainLMDB::add_block( const block& blk
, const size_t& block_size
, const difficulty_type& cumulative_difficulty
, const uint64_t& coins_generated
, const crypto::hash& blk_hash
)
void BlockchainLMDB::add_block(const block& blk, const size_t& block_size, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated,
const crypto::hash& blk_hash)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
@ -545,7 +596,7 @@ void BlockchainLMDB::remove_transaction_data(const crypto::hash& tx_hash, const
}
void BlockchainLMDB::add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index)
void BlockchainLMDB::add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
@ -574,10 +625,15 @@ void BlockchainLMDB::add_output(const crypto::hash& tx_hash, const tx_out& tx_ou
if (tx_output.target.type() == typeid(txout_to_key))
{
MDB_val_copy<crypto::public_key> val_pubkey(boost::get<txout_to_key>(tx_output.target).key);
result = mdb_put(*m_write_txn, m_output_keys, &k, &val_pubkey, 0);
if (result)
throw0(DB_ERROR(std::string("Failed to add output pubkey to db transaction: ").append(mdb_strerror(result)).c_str()));
output_data_t od;
od.pubkey = boost::get < txout_to_key > (tx_output.target).key;
od.unlock_time = unlock_time;
od.height = m_height;
MDB_val_copy<output_data_t> data(od);
//MDB_val_copy<crypto::public_key> val_pubkey(boost::get<txout_to_key>(tx_output.target).key);
if (mdb_put(*m_write_txn, m_output_keys, &k, &data, 0))
throw0(DB_ERROR("Failed to add output pubkey to db transaction"));
}
@ -808,47 +864,17 @@ tx_out BlockchainLMDB::output_from_blob(const blobdata& blob) const
return o;
}
uint64_t BlockchainLMDB::get_output_global_index(const uint64_t& amount, const uint64_t& index) const
uint64_t BlockchainLMDB::get_output_global_index(const uint64_t& amount, const uint64_t& index)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
mdb_txn_safe txn;
if (mdb_txn_begin(m_env, NULL, MDB_RDONLY, txn))
throw0(DB_ERROR("Failed to create a transaction for the db"));
lmdb_cur cur(txn, m_output_amounts);
MDB_val_copy<uint64_t> k(amount);
MDB_val v;
auto result = mdb_cursor_get(cur, &k, &v, MDB_SET);
if (result == MDB_NOTFOUND)
std::vector <uint64_t> offsets;
std::vector <uint64_t> global_indices;
offsets.push_back(index);
get_output_global_indices(amount, offsets, global_indices);
if (!global_indices.size())
throw1(OUTPUT_DNE("Attempting to get an output index by amount and amount index, but amount not found"));
else if (result)
throw0(DB_ERROR("DB error attempting to get an output"));
size_t num_elems = 0;
mdb_cursor_count(cur, &num_elems);
if (num_elems <= index)
throw1(OUTPUT_DNE("Attempting to get an output index by amount and amount index, but output not found"));
mdb_cursor_get(cur, &k, &v, MDB_FIRST_DUP);
for (uint64_t i = 0; i < index; ++i)
{
mdb_cursor_get(cur, &k, &v, MDB_NEXT_DUP);
}
mdb_cursor_get(cur, &k, &v, MDB_GET_CURRENT);
uint64_t glob_index = *(const uint64_t*)v.mv_data;
cur.close();
txn.commit();
return glob_index;
return global_indices[0];
}
void BlockchainLMDB::check_open() const
@ -903,8 +929,7 @@ void BlockchainLMDB::open(const std::string& filename, const int mdb_flags)
// check for existing LMDB files in base directory
boost::filesystem::path old_files = direc.parent_path();
if (boost::filesystem::exists(old_files / "data.mdb") ||
boost::filesystem::exists(old_files / "lock.mdb"))
if (boost::filesystem::exists(old_files / "data.mdb") || boost::filesystem::exists(old_files / "lock.mdb"))
{
LOG_PRINT_L0("Found existing LMDB files in " << old_files.string());
LOG_PRINT_L0("Move data.mdb and/or lock.mdb to " << filename << ", or delete them, and then restart");
@ -970,7 +995,7 @@ void BlockchainLMDB::open(const std::string& filename, const int mdb_flags)
lmdb_db_open(txn, LMDB_OUTPUT_TXS, MDB_INTEGERKEY | MDB_CREATE, m_output_txs, "Failed to open db handle for m_output_txs");
lmdb_db_open(txn, LMDB_OUTPUT_INDICES, MDB_INTEGERKEY | MDB_CREATE, m_output_indices, "Failed to open db handle for m_output_indices");
lmdb_db_open(txn, LMDB_OUTPUT_AMOUNTS, MDB_INTEGERKEY | MDB_DUPSORT | MDB_CREATE, m_output_amounts, "Failed to open db handle for m_output_amounts");
lmdb_db_open(txn, LMDB_OUTPUT_AMOUNTS, MDB_INTEGERKEY | MDB_DUPSORT | MDB_DUPFIXED | MDB_CREATE, m_output_amounts, "Failed to open db handle for m_output_amounts");
lmdb_db_open(txn, LMDB_OUTPUT_KEYS, MDB_INTEGERKEY | MDB_CREATE, m_output_keys, "Failed to open db handle for m_output_keys");
/*************** not used, but kept for posterity
@ -982,6 +1007,11 @@ void BlockchainLMDB::open(const std::string& filename, const int mdb_flags)
mdb_set_dupsort(txn, m_output_amounts, compare_uint64);
mdb_set_dupsort(txn, m_tx_outputs, compare_uint64);
mdb_set_compare(txn, m_spent_keys, compare_hash32);
mdb_set_compare(txn, m_block_heights, compare_hash32);
mdb_set_compare(txn, m_txs, compare_hash32);
mdb_set_compare(txn, m_tx_unlocks, compare_hash32);
mdb_set_compare(txn, m_tx_heights, compare_hash32);
// get and keep current height
MDB_stat db_stats;
@ -995,6 +1025,34 @@ void BlockchainLMDB::open(const std::string& filename, const int mdb_flags)
throw0(DB_ERROR("Failed to query m_output_indices"));
m_num_outputs = db_stats.ms_entries;
// ND: This "new" version of the lmdb database is incompatible with
// the previous version. Ensure that the output_keys database is
// sizeof(output_data_t) in length. Otherwise, inform user and
// terminate.
if(m_height > 0)
{
MDB_val_copy<uint64_t> k(0);
MDB_val v;
auto get_result = mdb_get(txn, m_output_keys, &k, &v);
if(get_result != MDB_SUCCESS)
{
txn.abort();
m_open = false;
return;
}
// LOG_PRINT_L0("Output keys size: " << v.mv_size);
if(v.mv_size != sizeof(output_data_t))
{
txn.abort();
mdb_env_close(m_env);
m_open = false;
LOG_PRINT_RED_L0("Existing lmdb database is incompatible with this version.");
LOG_PRINT_RED_L0("Please delete the existing database and resync.");
return;
}
}
// commit the transaction
txn.commit();
@ -1604,26 +1662,33 @@ uint64_t BlockchainLMDB::get_num_outputs(const uint64_t& amount) const
return num_elems;
}
crypto::public_key BlockchainLMDB::get_output_key(const uint64_t& amount, const uint64_t& index) const
output_data_t BlockchainLMDB::get_output_key(const uint64_t &global_index) const
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
uint64_t glob_index = get_output_global_index(amount, index);
mdb_txn_safe txn;
if (mdb_txn_begin(m_env, NULL, MDB_RDONLY, txn))
throw0(DB_ERROR("Failed to create a transaction for the db"));
MDB_val_copy<uint64_t> k(glob_index);
MDB_val_copy<uint64_t> k(global_index);
MDB_val v;
auto get_result = mdb_get(txn, m_output_keys, &k, &v);
if (get_result == MDB_NOTFOUND)
throw0(DB_ERROR("Attempting to get output pubkey by global index, but key does not exist"));
else if (get_result)
throw0(DB_ERROR("Error attempting to retrieve an output pubkey from the db"));
txn.commit();
return *(output_data_t *) v.mv_data;
}
return *(crypto::public_key*)v.mv_data;
output_data_t BlockchainLMDB::get_output_key(const uint64_t& amount, const uint64_t& index)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
uint64_t glob_index = get_output_global_index(amount, index);
return get_output_key(glob_index);
}
tx_out BlockchainLMDB::get_output(const crypto::hash& h, const uint64_t& index) const
@ -1731,53 +1796,17 @@ tx_out_index BlockchainLMDB::get_output_tx_and_index_from_global(const uint64_t&
return tx_out_index(tx_hash, *(const uint64_t *)v.mv_data);
}
tx_out_index BlockchainLMDB::get_output_tx_and_index(const uint64_t& amount, const uint64_t& index) const
tx_out_index BlockchainLMDB::get_output_tx_and_index(const uint64_t& amount, const uint64_t& index)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
mdb_txn_safe txn;
mdb_txn_safe* txn_ptr = &txn;
if (m_batch_active)
txn_ptr = m_write_txn;
else
{
if (mdb_txn_begin(m_env, NULL, MDB_RDONLY, txn))
throw0(DB_ERROR("Failed to create a transaction for the db"));
}
lmdb_cur cur(*txn_ptr, m_output_amounts);
MDB_val_copy<uint64_t> k(amount);
MDB_val v;
auto result = mdb_cursor_get(cur, &k, &v, MDB_SET);
if (result == MDB_NOTFOUND)
std::vector < uint64_t > offsets;
std::vector<tx_out_index> indices;
offsets.push_back(index);
get_output_tx_and_index(amount, offsets, indices);
if (!indices.size())
throw1(OUTPUT_DNE("Attempting to get an output index by amount and amount index, but amount not found"));
else if (result)
throw0(DB_ERROR("DB error attempting to get an output"));
size_t num_elems = 0;
mdb_cursor_count(cur, &num_elems);
if (num_elems <= index)
throw1(OUTPUT_DNE("Attempting to get an output index by amount and amount index, but output not found"));
mdb_cursor_get(cur, &k, &v, MDB_FIRST_DUP);
for (uint64_t i = 0; i < index; ++i)
{
mdb_cursor_get(cur, &k, &v, MDB_NEXT_DUP);
}
mdb_cursor_get(cur, &k, &v, MDB_GET_CURRENT);
uint64_t glob_index = *(const uint64_t*)v.mv_data;
cur.close();
if (! m_batch_active)
txn.commit();
return get_output_tx_and_index_from_global(glob_index);
return indices[0];
}
std::vector<uint64_t> BlockchainLMDB::get_tx_output_indices(const crypto::hash& h) const
@ -2016,12 +2045,8 @@ void BlockchainLMDB::set_batch_transactions(bool batch_transactions)
LOG_PRINT_L3("batch transactions " << (m_batch_transactions ? "enabled" : "disabled"));
}
uint64_t BlockchainLMDB::add_block( const block& blk
, const size_t& block_size
, const difficulty_type& cumulative_difficulty
, const uint64_t& coins_generated
, const std::vector<transaction>& txs
)
uint64_t BlockchainLMDB::add_block(const block& blk, const size_t& block_size, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated,
const std::vector<transaction>& txs)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
@ -2103,4 +2128,213 @@ void BlockchainLMDB::pop_block(block& blk, std::vector<transaction>& txs)
--m_height;
}
void BlockchainLMDB::get_output_tx_and_index_from_global(const std::vector<uint64_t> &global_indices,
std::vector<tx_out_index> &tx_out_indices) const
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
tx_out_indices.clear();
mdb_txn_safe txn;
mdb_txn_safe* txn_ptr = &txn;
if (m_batch_active)
txn_ptr = m_write_txn;
else
{
if (mdb_txn_begin(m_env, NULL, MDB_RDONLY, txn))
throw0(DB_ERROR("Failed to create a transaction for the db"));
}
for (const uint64_t &index : global_indices)
{
MDB_val_copy<uint64_t> k(index);
MDB_val v;
auto get_result = mdb_get(*txn_ptr, m_output_txs, &k, &v);
if (get_result == MDB_NOTFOUND)
throw1(OUTPUT_DNE("output with given index not in db"));
else if (get_result)
throw0(DB_ERROR("DB error attempting to fetch output tx hash"));
crypto::hash tx_hash = *(crypto::hash*) v.mv_data;
get_result = mdb_get(*txn_ptr, m_output_indices, &k, &v);
if (get_result == MDB_NOTFOUND)
throw1(OUTPUT_DNE("output with given index not in db"));
else if (get_result)
throw0(DB_ERROR("DB error attempting to fetch output tx index"));
auto result = tx_out_index(tx_hash, *(const uint64_t *) v.mv_data);
tx_out_indices.push_back(result);
}
if (!m_batch_active)
txn.commit();
}
void BlockchainLMDB::get_output_global_indices(const uint64_t& amount, const std::vector<uint64_t> &offsets,
std::vector<uint64_t> &global_indices)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
TIME_MEASURE_START(txx);
check_open();
global_indices.clear();
uint64_t max = 0;
for (const uint64_t &index : offsets)
{
if (index > max)
max = index;
}
mdb_txn_safe txn;
mdb_txn_safe* txn_ptr = &txn;
if(m_batch_active)
txn_ptr = m_write_txn;
else
{
if (mdb_txn_begin(m_env, NULL, MDB_RDONLY, txn))
throw0(DB_ERROR("Failed to create a transaction for the db"));
}
lmdb_cur cur(*txn_ptr, m_output_amounts);
MDB_val_copy<uint64_t> k(amount);
MDB_val v;
auto result = mdb_cursor_get(cur, &k, &v, MDB_SET);
if (result == MDB_NOTFOUND)
throw1(OUTPUT_DNE("Attempting to get an output index by amount and amount index, but amount not found"));
else if (result)
throw0(DB_ERROR("DB error attempting to get an output"));
size_t num_elems = 0;
mdb_cursor_count(cur, &num_elems);
if (max <= 1 && num_elems <= max)
throw1(OUTPUT_DNE("Attempting to get an output index by amount and amount index, but output not found"));
uint64_t t_dbmul = 0;
uint64_t t_dbscan = 0;
if (max <= 1)
{
for (const uint64_t& index : offsets)
{
mdb_cursor_get(cur, &k, &v, MDB_FIRST_DUP);
for (uint64_t i = 0; i < index; ++i)
{
mdb_cursor_get(cur, &k, &v, MDB_NEXT_DUP);
}
mdb_cursor_get(cur, &k, &v, MDB_GET_CURRENT);
uint64_t glob_index = *(const uint64_t*) v.mv_data;
LOG_PRINT_L3("Amount: " << amount << " M0->v: " << glob_index);
global_indices.push_back(glob_index);
}
}
else
{
uint32_t curcount = 0;
uint32_t blockstart = 0;
for (const uint64_t& index : offsets)
{
if (index >= num_elems)
{
LOG_PRINT_L1("Index: " << index << " Elems: " << num_elems << " partial results found for get_output_tx_and_index");
break;
}
while (index >= curcount)
{
TIME_MEASURE_START(db1);
if (mdb_cursor_get(cur, &k, &v, curcount == 0 ? MDB_GET_MULTIPLE : MDB_NEXT_MULTIPLE) != 0)
{
// allow partial results
result = false;
break;
}
int count = v.mv_size / sizeof(uint64_t);
blockstart = curcount;
curcount += count;
TIME_MEASURE_FINISH(db1);
t_dbmul += db1;
}
LOG_PRINT_L3("Records returned: " << curcount << " Index: " << index);
TIME_MEASURE_START(db2);
uint64_t actual_index = index - blockstart;
uint64_t glob_index = ((const uint64_t*) v.mv_data)[actual_index];
LOG_PRINT_L3("Amount: " << amount << " M1->v: " << glob_index);
global_indices.push_back(glob_index);
TIME_MEASURE_FINISH(db2);
t_dbscan += db2;
}
}
cur.close();
if(!m_batch_active)
txn.commit();
TIME_MEASURE_FINISH(txx);
LOG_PRINT_L3("txx: " << txx << " db1: " << t_dbmul << " db2: " << t_dbscan);
}
void BlockchainLMDB::get_output_key(const uint64_t &amount, const std::vector<uint64_t> &offsets, std::vector<output_data_t> &outputs)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
TIME_MEASURE_START(db3);
check_open();
outputs.clear();
std::vector <uint64_t> global_indices;
get_output_global_indices(amount, offsets, global_indices);
if (global_indices.size() > 0)
{
mdb_txn_safe txn;
if (mdb_txn_begin(m_env, NULL, MDB_RDONLY, txn))
throw0(DB_ERROR("Failed to create a transaction for the db"));
for (const uint64_t &index : global_indices)
{
MDB_val_copy<uint64_t> k(index);
MDB_val v;
auto get_result = mdb_get(txn, m_output_keys, &k, &v);
if (get_result != 0)
throw0(DB_ERROR("Attempting to get output pubkey by global index, but key does not exist"));
else if (get_result)
throw0(DB_ERROR("Error attempting to retrieve an output pubkey from the db"));
output_data_t data = *(output_data_t *) v.mv_data;
outputs.push_back(data);
}
txn.commit();
}
TIME_MEASURE_FINISH(db3);
LOG_PRINT_L3("db3: " << db3);
}
void BlockchainLMDB::get_output_tx_and_index(const uint64_t& amount, const std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices)
{
LOG_PRINT_L3("BlockchainLMDB::" << __func__);
check_open();
indices.clear();
std::vector <uint64_t> global_indices;
get_output_global_indices(amount, offsets, global_indices);
TIME_MEASURE_START(db3);
if(global_indices.size() > 0)
{
get_output_tx_and_index_from_global(global_indices, indices);
}
TIME_MEASURE_FINISH(db3);
LOG_PRINT_L3("db3: " << db3);
}
} // namespace cryptonote

35
src/blockchain_db/lmdb/db_lmdb.h
View file

@ -33,6 +33,8 @@
#include <lmdb.h>
#define ENABLE_AUTO_RESIZE
namespace cryptonote
{
@ -159,7 +161,9 @@ public:
virtual uint64_t get_num_outputs(const uint64_t& amount) const;
virtual crypto::public_key get_output_key(const uint64_t& amount, const uint64_t& index) const;
virtual output_data_t get_output_key(const uint64_t& amount, const uint64_t& index);
virtual output_data_t get_output_key(const uint64_t& global_index) const;
virtual void get_output_key(const uint64_t &amount, const std::vector<uint64_t> &offsets, std::vector<output_data_t> &outputs);
virtual tx_out get_output(const crypto::hash& h, const uint64_t& index) const;
@ -175,12 +179,12 @@ public:
tx_out get_output(const uint64_t& index) const;
virtual tx_out_index get_output_tx_and_index_from_global(const uint64_t& index) const;
virtual void get_output_tx_and_index_from_global(const std::vector<uint64_t> &global_indices,
std::vector<tx_out_index> &tx_out_indices) const;
virtual tx_out_index get_output_tx_and_index(const uint64_t& amount, const uint64_t& index) const;
virtual void get_output_tx_and_index(const uint64_t& amount, std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices) const
{
// do nothing
};
virtual tx_out_index get_output_tx_and_index(const uint64_t& amount, const uint64_t& index);
virtual void get_output_tx_and_index(const uint64_t& amount, const std::vector<uint64_t> &offsets, std::vector<tx_out_index> &indices);
virtual void get_output_global_indices(const uint64_t& amount, const std::vector<uint64_t> &offsets, std::vector<uint64_t> &indices);
virtual std::vector<uint64_t> get_tx_output_indices(const crypto::hash& h) const;
virtual std::vector<uint64_t> get_tx_amount_output_indices(const crypto::hash& h) const;
@ -202,7 +206,7 @@ public:
virtual void pop_block(block& blk, std::vector<transaction>& txs);
virtual bool has_bulk_indices() const { return false; }
virtual bool can_thread_bulk_indices() const { return true; }
private:
void do_resize(uint64_t size_increase=0);
@ -223,7 +227,7 @@ private:
virtual void remove_transaction_data(const crypto::hash& tx_hash, const transaction& tx);
virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index);
virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time);
virtual void remove_output(const tx_out& tx_output);
@ -262,7 +266,7 @@ private:
*
* @return the global index of the desired output
*/
uint64_t get_output_global_index(const uint64_t& amount, const uint64_t& index) const;
uint64_t get_output_global_index(const uint64_t& amount, const uint64_t& index);
void check_open() const;
@ -299,9 +303,18 @@ private:
bool m_batch_transactions; // support for batch transactions
bool m_batch_active; // whether batch transaction is in progress
constexpr static uint64_t DEFAULT_MAPSIZE = 1 << 30;
#if defined(__arm__)
// force a value so it can compile with 32-bit ARM
constexpr static uint64_t DEFAULT_MAPSIZE = 1LL << 31;
#else
#if defined(ENABLE_AUTO_RESIZE)
constexpr static uint64_t DEFAULT_MAPSIZE = 1LL << 30;
#else
constexpr static uint64_t DEFAULT_MAPSIZE = 1LL << 33;
#endif
#endif
constexpr static float RESIZE_PERCENT = 0.8f;
constexpr static float RESIZE_FACTOR = 1.5f;
};
} // namespace cryptonote

37
src/blocks/CMakeLists.txt Normal file
View file

@ -0,0 +1,37 @@
# Copyright (c) 2014-2015, The Monero Project
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification, are
# permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this list of
# conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice, this list
# of conditions and the following disclaimer in the documentation and/or other
# materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors may be
# used to endorse or promote products derived from this software without specific
# prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
# THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
if(APPLE)
add_library(blocks STATIC blockexports.c)
set_target_properties(blocks PROPERTIES LINKER_LANGUAGE C)
else()
add_custom_command(OUTPUT blocks.o COMMAND cd ${CMAKE_CURRENT_SOURCE_DIR} && ld -r -b binary -o ${CMAKE_CURRENT_BINARY_DIR}/blocks.o blocks.dat)
add_library(blocks STATIC blocks.o blockexports.c)
set_target_properties(blocks PROPERTIES LINKER_LANGUAGE C)
endif()

48
src/blocks/blockexports.c Normal file
View file

@ -0,0 +1,48 @@
#include <stddef.h>
#if defined(__APPLE__)
#include <mach-o/getsect.h>
#if !defined(__LP64__)
extern const struct mach_header _mh_execute_header;
#else
extern const struct mach_header_64 _mh_execute_header;
#endif
const unsigned char *get_blocks_dat_start()
{
size_t size;
return getsectiondata(&_mh_execute_header, "__DATA", "__blocks_dat", &size);
}
size_t get_blocks_dat_size()
{
size_t size;
getsectiondata(&_mh_execute_header, "__DATA", "__blocks_dat", &size);
return size;
}
#else
#if defined(_WIN32) && !defined(_WIN64)
#define _binary_blocks_start binary_blocks_dat_start
#define _binary_blocks_end binary_blocks_dat_end
#else
#define _binary_blocks_start _binary_blocks_dat_start
#define _binary_blocks_end _binary_blocks_dat_end
#endif
extern const unsigned char _binary_blocks_start[];
extern const unsigned char _binary_blocks_end[];
const unsigned char *get_blocks_dat_start(void)
{
return _binary_blocks_start;
}
size_t get_blocks_dat_size(void)
{
return (size_t) (_binary_blocks_end - _binary_blocks_start);
}
#endif

BIN
src/blocks/blocks.dat Normal file
View file

Binary file not shown.

16
src/blocks/blocks.h Normal file
View file

@ -0,0 +1,16 @@
#ifndef SRC_BLOCKS_BLOCKS_H_
#define SRC_BLOCKS_BLOCKS_H_
#ifdef __cplusplus
extern "C" {
#endif
const unsigned char *get_blocks_dat_start();
size_t get_blocks_dat_size();
#ifdef __cplusplus
}
#endif
#endif /* SRC_BLOCKS_BLOCKS_H_ */

12
src/crypto/aesb.c
View file

@ -140,7 +140,15 @@ extern "C"
d_4(uint32_t, t_dec(f,n), sb_data, u0, u1, u2, u3);
void aesb_single_round(const uint8_t *in, uint8_t *out, uint8_t *expandedKey)
#if !defined(STATIC)
#define STATIC
#endif
#if !defined(INLINE)
#define INLINE
#endif
STATIC INLINE void aesb_single_round(const uint8_t *in, uint8_t *out, uint8_t *expandedKey)
{
uint32_t b0[4], b1[4];
const uint32_t *kp = (uint32_t *) expandedKey;
@ -151,7 +159,7 @@ void aesb_single_round(const uint8_t *in, uint8_t *out, uint8_t *expandedKey)
state_out(out, b1);
}
void aesb_pseudo_round(const uint8_t *in, uint8_t *out, uint8_t *expandedKey)
STATIC INLINE void aesb_pseudo_round(const uint8_t *in, uint8_t *out, uint8_t *expandedKey)
{
uint32_t b0[4], b1[4];
const uint32_t *kp = (uint32_t *) expandedKey;

190
src/crypto/slow-hash.c
View file

@ -624,6 +624,196 @@ void cn_slow_hash(const void *data, size_t length, char *hash)
extra_hashes[state.hs.b[0] & 3](&state, 200, hash);
}
#elif defined(__arm__)
// ND: Some minor optimizations for ARM7 (raspberrry pi 2), effect seems to be ~40-50% faster.
// Needs more work.
void slow_hash_allocate_state(void)
{
// Do nothing, this is just to maintain compatibility with the upgraded slow-hash.c
return;
}
void slow_hash_free_state(void)
{
// As above
return;
}
static void (*const extra_hashes[4])(const void *, size_t, char *) = {
hash_extra_blake, hash_extra_groestl, hash_extra_jh, hash_extra_skein
};
#define MEMORY (1 << 21) /* 2 MiB */
#define ITER (1 << 20)
#define AES_BLOCK_SIZE 16
#define AES_KEY_SIZE 32 /*16*/
#define INIT_SIZE_BLK 8
#define INIT_SIZE_BYTE (INIT_SIZE_BLK * AES_BLOCK_SIZE)
#if defined(__GNUC__)
#define RDATA_ALIGN16 __attribute__ ((aligned(16)))
#define STATIC static
#define INLINE inline
#else
#define RDATA_ALIGN16
#define STATIC static
#define INLINE
#endif
#define U64(x) ((uint64_t *) (x))
#include "aesb.c"
STATIC INLINE void ___mul128(uint32_t *a, uint32_t *b, uint32_t *h, uint32_t *l)
{
// ND: 64x64 multiplication for ARM7
__asm__ __volatile__
(
// lo hi
"umull %[r0], %[r1], %[b], %[d]\n\t" // bd [r0 = bd.lo]
"umull %[r2], %[r3], %[b], %[c]\n\t" // bc
"umull %[b], %[c], %[a], %[c]\n\t" // ac
"adds %[r1], %[r1], %[r2]\n\t" // r1 = bd.hi + bc.lo
"adcs %[r2], %[r3], %[b]\n\t" // r2 = ac.lo + bc.hi + carry
"adc %[r3], %[c], #0\n\t" // r3 = ac.hi + carry
"umull %[b], %[a], %[a], %[d]\n\t" // ad
"adds %[r1], %[r1], %[b]\n\t" // r1 = bd.hi + bc.lo + ad.lo
"adcs %[r2], %[r2], %[a]\n\t" // r2 = ac.lo + bc.hi + ad.hi + carry
"adc %[r3], %[r3], #0\n\t" // r3 = ac.hi + carry
: [r0]"=&r"(l[0]), [r1]"=&r"(l[1]), [r2]"=&r"(h[0]), [r3]"=&r"(h[1])
: [a]"r"(a[1]), [b]"r"(a[0]), [c]"r"(b[1]), [d]"r"(b[0])
: "cc"
);
}
STATIC INLINE void mul(const uint8_t* a, const uint8_t* b, uint8_t* res)
{
___mul128((uint32_t *) a, (uint32_t *) b, (uint32_t *) (res + 0), (uint32_t *) (res + 8));
}
STATIC INLINE void sum_half_blocks(uint8_t* a, const uint8_t* b)
{
uint64_t a0, a1, b0, b1;
a0 = U64(a)[0];
a1 = U64(a)[1];
b0 = U64(b)[0];
b1 = U64(b)[1];
a0 += b0;
a1 += b1;
U64(a)[0] = a0;
U64(a)[1] = a1;
}
STATIC INLINE void swap_blocks(uint8_t *a, uint8_t *b)
{
uint64_t t[2];
U64(t)[0] = U64(a)[0];
U64(t)[1] = U64(a)[1];
U64(a)[0] = U64(b)[0];
U64(a)[1] = U64(b)[1];
U64(b)[0] = U64(t)[0];
U64(b)[1] = U64(t)[1];
}
STATIC INLINE void xor_blocks(uint8_t* a, const uint8_t* b)
{
U64(a)[0] ^= U64(b)[0];
U64(a)[1] ^= U64(b)[1];
}
#pragma pack(push, 1)
union cn_slow_hash_state
{
union hash_state hs;
struct
{
uint8_t k[64];
uint8_t init[INIT_SIZE_BYTE];
};
};
#pragma pack(pop)
void cn_slow_hash(const void *data, size_t length, char *hash)
{
uint8_t long_state[MEMORY];
uint8_t text[INIT_SIZE_BYTE];
uint8_t a[AES_BLOCK_SIZE];
uint8_t b[AES_BLOCK_SIZE];
uint8_t d[AES_BLOCK_SIZE];
uint8_t aes_key[AES_KEY_SIZE];
RDATA_ALIGN16 uint8_t expandedKey[256];
union cn_slow_hash_state state;
size_t i, j;
uint8_t *p = NULL;
oaes_ctx *aes_ctx;
static void (*const extra_hashes[4])(const void *, size_t, char *) =
{
hash_extra_blake, hash_extra_groestl, hash_extra_jh, hash_extra_skein
};
hash_process(&state.hs, data, length);
memcpy(text, state.init, INIT_SIZE_BYTE);
aes_ctx = (oaes_ctx *) oaes_alloc();
oaes_key_import_data(aes_ctx, state.hs.b, AES_KEY_SIZE);
// use aligned data
memcpy(expandedKey, aes_ctx->key->exp_data, aes_ctx->key->exp_data_len);
for(i = 0; i < MEMORY / INIT_SIZE_BYTE; i++)
{
for(j = 0; j < INIT_SIZE_BLK; j++)
aesb_pseudo_round(&text[AES_BLOCK_SIZE * j], &text[AES_BLOCK_SIZE * j], expandedKey);
memcpy(&long_state[i * INIT_SIZE_BYTE], text, INIT_SIZE_BYTE);
}
U64(a)[0] = U64(&state.k[0])[0] ^ U64(&state.k[32])[0];
U64(a)[1] = U64(&state.k[0])[1] ^ U64(&state.k[32])[1];
U64(b)[0] = U64(&state.k[16])[0] ^ U64(&state.k[48])[0];
U64(b)[1] = U64(&state.k[16])[1] ^ U64(&state.k[48])[1];
for(i = 0; i < ITER / 2; i++)
{
#define MASK ((uint32_t)(((MEMORY / AES_BLOCK_SIZE) - 1) << 4))
#define state_index(x) ((*(uint32_t *) x) & MASK)
// Iteration 1
p = &long_state[state_index(a)];
aesb_single_round(p, p, a);
xor_blocks(b, p);
swap_blocks(b, p);
swap_blocks(a, b);
// Iteration 2
p = &long_state[state_index(a)];
mul(a, p, d);
sum_half_blocks(b, d);
swap_blocks(b, p);
xor_blocks(b, p);
swap_blocks(a, b);
}
memcpy(text, state.init, INIT_SIZE_BYTE);
oaes_key_import_data(aes_ctx, &state.hs.b[32], AES_KEY_SIZE);
memcpy(expandedKey, aes_ctx->key->exp_data, aes_ctx->key->exp_data_len);
for(i = 0; i < MEMORY / INIT_SIZE_BYTE; i++)
{
for(j = 0; j < INIT_SIZE_BLK; j++)
{
xor_blocks(&text[j * AES_BLOCK_SIZE], &long_state[i * INIT_SIZE_BYTE + j * AES_BLOCK_SIZE]);
aesb_pseudo_round(&text[AES_BLOCK_SIZE * j], &text[AES_BLOCK_SIZE * j], expandedKey);
}
}
oaes_free((OAES_CTX **) &aes_ctx);
memcpy(state.init, text, INIT_SIZE_BYTE);
hash_permutation(&state.hs);
extra_hashes[state.hs.b[0] & 3](&state, 200, hash);
}
#else
// Portable implementation as a fallback

7
src/cryptonote_core/CMakeLists.txt
View file

@ -62,6 +62,12 @@ set(cryptonote_core_private_headers
tx_pool.h
verification_context.h)
if(PER_BLOCK_CHECKPOINT)
set(Blocks "blocks")
else()
set(Blocks "")
endif()
bitmonero_private_headers(cryptonote_core
${crypto_private_headers})
bitmonero_add_library(cryptonote_core
@ -78,6 +84,7 @@ target_link_libraries(cryptonote_core
${Boost_PROGRAM_OPTIONS_LIBRARY}
${Boost_SERIALIZATION_LIBRARY}
LINK_PRIVATE
${Blocks}
${Boost_FILESYSTEM_LIBRARY}
${Boost_SYSTEM_LIBRARY}
${Boost_THREAD_LIBRARY}

4082
src/cryptonote_core/blockchain.cpp
View file

File diff suppressed because it is too large Load diff

64
src/cryptonote_core/blockchain.h
View file

@ -56,6 +56,13 @@ namespace cryptonote
{
class tx_memory_pool;
enum blockchain_db_sync_mode
{
db_sync,
db_async,
db_nosync
};
/************************************************************************/
/* */
/************************************************************************/
@ -94,6 +101,8 @@ namespace cryptonote
crypto::hash get_block_id_by_height(uint64_t height) const;
bool get_block_by_hash(const crypto::hash &h, block &blk) const;
void get_all_known_block_ids(std::list<crypto::hash> &main, std::list<crypto::hash> &alt, std::list<crypto::hash> &invalid) const;
bool prepare_handle_incoming_blocks(const std::list<block_complete_entry> &blocks);
bool cleanup_handle_incoming_blocks(bool force_sync = false);
template<class archive_t>
void serialize(archive_t & ar, const unsigned int version);
@ -102,16 +111,13 @@ namespace cryptonote
bool have_tx_keyimges_as_spent(const transaction &tx) const;
bool have_tx_keyimg_as_spent(const crypto::key_image &key_im) const;
template<class visitor_t>
bool scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t& vis, uint64_t* pmax_related_block_height = NULL) const;
uint64_t get_current_blockchain_height() const;
crypto::hash get_tail_id() const;
crypto::hash get_tail_id(uint64_t& height) const;
difficulty_type get_difficulty_for_next_block() const;
difficulty_type get_difficulty_for_next_block();
bool add_new_block(const block& bl_, block_verification_context& bvc);
bool reset_and_set_genesis_block(const block& b);
bool create_block_template(block& b, const account_public_address& miner_address, difficulty_type& di, uint64_t& height, const blobdata& ex_nonce) const;
bool create_block_template(block& b, const account_public_address& miner_address, difficulty_type& di, uint64_t& height, const blobdata& ex_nonce);
bool have_block(const crypto::hash& id) const;
size_t get_total_transactions() const;
bool get_short_chain_history(std::list<crypto::hash>& ids) const;
@ -123,9 +129,8 @@ namespace cryptonote
bool get_random_outs_for_amounts(const COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::request& req, COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::response& res) const;
bool get_tx_outputs_gindexs(const crypto::hash& tx_id, std::vector<uint64_t>& indexs) const;
bool store_blockchain();
bool check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, uint64_t* pmax_related_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, uint64_t* pmax_used_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, uint64_t& pmax_used_block_height, crypto::hash& max_used_block_id) const;
bool check_tx_inputs(const transaction& tx, uint64_t& pmax_used_block_height, crypto::hash& max_used_block_id, bool kept_by_block = false);
uint64_t get_current_cumulative_blocksize_limit() const;
bool is_storing_blockchain()const{return m_is_blockchain_storing;}
uint64_t block_difficulty(uint64_t i) const;
@ -145,11 +150,23 @@ namespace cryptonote
void set_enforce_dns_checkpoints(bool enforce);
bool update_checkpoints(const std::string& file_path, bool check_dns);
// user options, must be called before calling init()
void set_user_options(uint64_t block_threads, uint64_t blocks_per_sync,
blockchain_db_sync_mode sync_mode, bool fast_sync);
void set_show_time_stats(bool stats) { m_show_time_stats = stats; }
BlockchainDB& get_db()
{
return *m_db;
}
void output_scan_worker(const uint64_t amount,const std::vector<uint64_t> &offsets,
std::vector<output_data_t> &outputs, std::unordered_map<crypto::hash,
cryptonote::transaction> &txs) const;
void block_longhash_worker(const uint64_t height, const std::vector<block> &blocks,
std::unordered_map<crypto::hash, crypto::hash> &map) const;
private:
typedef std::unordered_map<crypto::hash, size_t> blocks_by_id_index;
typedef std::unordered_map<crypto::hash, transaction_chain_entry> transactions_container;
@ -171,6 +188,29 @@ namespace cryptonote
key_images_container m_spent_keys;
size_t m_current_block_cumul_sz_limit;
std::unordered_map<crypto::hash, std::unordered_map<crypto::key_image, std::vector<output_data_t>>> m_scan_table;
std::unordered_map<crypto::hash, std::pair<bool, uint64_t>> m_check_tx_inputs_table;
std::unordered_map<crypto::hash, crypto::hash> m_blocks_longhash_table;
// SHA-3 hashes for each block and for fast pow checking
std::vector<crypto::hash> m_blocks_hash_check;
std::vector<crypto::hash> m_blocks_txs_check;
blockchain_db_sync_mode m_db_sync_mode;
bool m_fast_sync;
bool m_show_time_stats;
uint64_t m_db_blocks_per_sync;
uint64_t m_max_prepare_blocks_threads;
uint64_t m_fake_pow_calc_time;
uint64_t m_fake_scan_time;
uint64_t m_sync_counter;
std::vector<uint64_t> m_timestamps;
std::vector<difficulty_type> m_difficulties;
uint64_t m_timestamps_and_difficulties_height;
boost::asio::io_service m_async_service;
boost::thread_group m_async_pool;
std::unique_ptr<boost::asio::io_service::work> m_async_work_idle;
// all alternative chains
blocks_ext_by_hash m_alternative_chains; // crypto::hash -> block_extended_info
@ -185,6 +225,11 @@ namespace cryptonote
std::atomic<bool> m_is_blockchain_storing;
bool m_enforce_dns_checkpoints;
template<class visitor_t>
inline bool scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t &vis, const crypto::hash &tx_prefix_hash, uint64_t* pmax_related_block_height = NULL) const;
bool check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, std::vector<crypto::public_key> &output_keys, uint64_t* pmax_related_block_height);
bool check_tx_inputs(const transaction& tx, uint64_t* pmax_used_block_height = NULL);
bool switch_to_alternative_blockchain(std::list<blocks_ext_by_hash::iterator>& alt_chain, bool discard_disconnected_chain);
block pop_block_from_blockchain();
bool purge_transaction_from_blockchain(const crypto::hash& tx_id);
@ -213,6 +258,9 @@ namespace cryptonote
bool update_next_cumulative_size_limit();
bool check_for_double_spend(const transaction& tx, key_images_container& keys_this_block) const;
void get_timestamp_and_difficulty(uint64_t &timestamp, difficulty_type &difficulty, const int offset) const;
void check_ring_signature(const crypto::hash &tx_prefix_hash, const crypto::key_image &key_image,
const std::vector<crypto::public_key> &pubkeys, const std::vector<crypto::signature> &sig, uint64_t &result);
};

14
src/cryptonote_core/blockchain_storage.h
View file

@ -104,9 +104,6 @@ namespace cryptonote
bool have_tx_keyimg_as_spent(const crypto::key_image &key_im) const;
const transaction *get_tx(const crypto::hash &id) const;
template<class visitor_t>
bool scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t& vis, uint64_t* pmax_related_block_height = NULL) const;
uint64_t get_current_blockchain_height() const;
crypto::hash get_tail_id() const;
crypto::hash get_tail_id(uint64_t& height) const;
@ -127,9 +124,7 @@ namespace cryptonote
bool get_backward_blocks_sizes(size_t from_height, std::vector<size_t>& sz, size_t count) const;
bool get_tx_outputs_gindexs(const crypto::hash& tx_id, std::vector<uint64_t>& indexs) const;
bool store_blockchain();
bool check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, uint64_t* pmax_related_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, const crypto::hash& tx_prefix_hash, uint64_t* pmax_used_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, uint64_t* pmax_used_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, uint64_t& pmax_used_block_height, crypto::hash& max_used_block_id) const;
uint64_t get_current_cumulative_blocksize_limit() const;
bool is_storing_blockchain()const{return m_is_blockchain_storing;}
@ -229,6 +224,13 @@ namespace cryptonote
bool m_enforce_dns_checkpoints;
bool m_testnet;
// made private for consistency with blockchain.h
template<class visitor_t>
bool scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t& vis, uint64_t* pmax_related_block_height = NULL) const;
bool check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, uint64_t* pmax_related_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, const crypto::hash& tx_prefix_hash, uint64_t* pmax_used_block_height = NULL) const;
bool check_tx_inputs(const transaction& tx, uint64_t* pmax_used_block_height = NULL) const;
bool switch_to_alternative_blockchain(std::list<blocks_ext_by_hash::iterator>& alt_chain, bool discard_disconnected_chain);
bool pop_block_from_blockchain();
bool purge_block_data_from_blockchain(const block& b, size_t processed_tx_count);

6
src/cryptonote_core/cryptonote_basic_impl.cpp
View file

@ -99,7 +99,11 @@ namespace cryptonote {
assert(current_block_size < std::numeric_limits<uint32_t>::max());
uint64_t product_hi;
uint64_t product_lo = mul128(base_reward, current_block_size * (2 * median_size - current_block_size), &product_hi);
// BUGFIX: 32-bit saturation bug (e.g. ARM7), the result was being
// treated as 32-bit by default.
uint64_t multiplicand = 2 * median_size - current_block_size;
multiplicand *= current_block_size;
uint64_t product_lo = mul128(base_reward, multiplicand, &product_hi);
uint64_t reward_hi;
uint64_t reward_lo;

113
src/cryptonote_core/cryptonote_core.cpp
View file

@ -46,7 +46,7 @@ using namespace epee;
#include "cryptonote_core/checkpoints_create.h"
#include "blockchain_db/blockchain_db.h"
#include "blockchain_db/lmdb/db_lmdb.h"
#ifndef STATICLIB
#if defined(BERKELEY_DB)
#include "blockchain_db/berkeleydb/db_bdb.h"
#endif
@ -212,18 +212,27 @@ namespace cryptonote
#if BLOCKCHAIN_DB == DB_LMDB
std::string db_type = command_line::get_arg(vm, daemon_args::arg_db_type);
std::string db_sync_mode = command_line::get_arg(vm, daemon_args::arg_db_sync_mode);
bool fast_sync = command_line::get_arg(vm, daemon_args::arg_fast_block_sync) != 0;
uint64_t blocks_threads = command_line::get_arg(vm, daemon_args::arg_prep_blocks_threads);
BlockchainDB* db = nullptr;
uint64_t BDB_FAST_MODE = 0;
uint64_t BDB_FASTEST_MODE = 0;
uint64_t BDB_SAFE_MODE = 0;
if (db_type == "lmdb")
{
db = new BlockchainLMDB();
}
else if (db_type == "berkeley")
{
#ifndef STATICLIB
#if defined(BERKELEY_DB)
db = new BlockchainBDB();
BDB_FAST_MODE = DB_TXN_WRITE_NOSYNC;
BDB_FASTEST_MODE = DB_TXN_NOSYNC;
BDB_SAFE_MODE = DB_TXN_SYNC;
#else
LOG_ERROR("BlockchainBDB not supported on STATIC builds");
LOG_ERROR("BerkeleyDB support disabled.");
return false;
#endif
}
@ -240,9 +249,71 @@ namespace cryptonote
LOG_PRINT_L0("Loading blockchain from folder " << folder.string() << " ...");
const std::string filename = folder.string();
// temporarily default to fastest:async:1000
blockchain_db_sync_mode sync_mode = db_async;
uint64_t blocks_per_sync = 1000;
try
{
db->open(filename);
uint64_t db_flags = 0;
bool islmdb = db_type == "lmdb";
std::vector<std::string> options;
boost::trim(db_sync_mode);
boost::split(options, db_sync_mode, boost::is_any_of(" :"));
for(const auto &option : options)
LOG_PRINT_L0("option: " << option);
// temporarily default to fastest:async:1000
uint64_t DEFAULT_FLAGS = islmdb ? MDB_WRITEMAP | MDB_MAPASYNC | MDB_NORDAHEAD | MDB_NOMETASYNC | MDB_NOSYNC :
BDB_FASTEST_MODE;
if(options.size() == 0)
{
// temporarily default to fastest:async:1000
db_flags = DEFAULT_FLAGS;
}
bool safemode = false;
if(options.size() >= 1)
{
if(options[0] == "safe")
{
safemode = true;
db_flags = islmdb ? MDB_NORDAHEAD : BDB_SAFE_MODE;
sync_mode = db_nosync;
}
else if(options[0] == "fast")
db_flags = islmdb ? MDB_NOMETASYNC | MDB_NOSYNC | MDB_NORDAHEAD : BDB_FAST_MODE;
else if(options[0] == "fastest")
db_flags = islmdb ? MDB_WRITEMAP | MDB_MAPASYNC | MDB_NORDAHEAD | MDB_NOMETASYNC | MDB_NOSYNC : BDB_FASTEST_MODE;
else
db_flags = DEFAULT_FLAGS;
}
if(options.size() >= 2 && !safemode)
{
if(options[1] == "sync")
sync_mode = db_sync;
else if(options[1] == "async")
sync_mode = db_async;
}
if(options.size() >= 3 && !safemode)
{
blocks_per_sync = atoll(options[2].c_str());
if(blocks_per_sync > 5000)
blocks_per_sync = 5000;
if(blocks_per_sync == 0)
blocks_per_sync = 1;
}
bool auto_remove_logs = command_line::get_arg(vm, daemon_args::arg_db_auto_remove_logs) != 0;
db->set_auto_remove_logs(auto_remove_logs);
db->open(filename, db_flags);
if(!db->m_open)
return false;
}
catch (const DB_ERROR& e)
{
@ -250,7 +321,13 @@ namespace cryptonote
return false;
}
m_blockchain_storage.set_user_options(blocks_threads,
blocks_per_sync, sync_mode, fast_sync);
r = m_blockchain_storage.init(db, m_testnet);
bool show_time_stats = command_line::get_arg(vm, daemon_args::arg_show_time_stats) != 0;
m_blockchain_storage.set_show_time_stats(show_time_stats);
#else
r = m_blockchain_storage.init(m_config_folder, m_testnet);
#endif
@ -587,6 +664,25 @@ namespace cryptonote
{
return m_blockchain_storage.add_new_block(b, bvc);
}
//-----------------------------------------------------------------------------------------------
bool core::prepare_handle_incoming_blocks(const std::list<block_complete_entry> &blocks)
{
#if BLOCKCHAIN_DB == DB_LMDB
m_blockchain_storage.prepare_handle_incoming_blocks(blocks);
#endif
return true;
}
//-----------------------------------------------------------------------------------------------
bool core::cleanup_handle_incoming_blocks(bool force_sync)
{
#if BLOCKCHAIN_DB == DB_LMDB
m_blockchain_storage.cleanup_handle_incoming_blocks(force_sync);
#endif
return true;
}
//-----------------------------------------------------------------------------------------------
bool core::handle_incoming_block(const blobdata& block_blob, block_verification_context& bvc, bool update_miner_blocktemplate)
{
@ -678,7 +774,8 @@ namespace cryptonote
return m_blockchain_storage.get_block_id_by_height(height);
}
//-----------------------------------------------------------------------------------------------
bool core::get_block_by_hash(const crypto::hash &h, block &blk) {
bool core::get_block_by_hash(const crypto::hash &h, block &blk)
{
return m_blockchain_storage.get_block_by_hash(h, blk);
}
//-----------------------------------------------------------------------------------------------
@ -723,11 +820,13 @@ namespace cryptonote
return true;
}
//-----------------------------------------------------------------------------------------------
void core::set_target_blockchain_height(uint64_t target_blockchain_height) {
void core::set_target_blockchain_height(uint64_t target_blockchain_height)
{
m_target_blockchain_height = target_blockchain_height;
}
//-----------------------------------------------------------------------------------------------
uint64_t core::get_target_blockchain_height() const {
uint64_t core::get_target_blockchain_height() const
{
return m_target_blockchain_height;
}
//-----------------------------------------------------------------------------------------------

3
src/cryptonote_core/cryptonote_core.h
View file

@ -66,6 +66,9 @@ namespace cryptonote
bool on_idle();
bool handle_incoming_tx(const blobdata& tx_blob, tx_verification_context& tvc, bool keeped_by_block);
bool handle_incoming_block(const blobdata& block_blob, block_verification_context& bvc, bool update_miner_blocktemplate = true);
bool prepare_handle_incoming_blocks(const std::list<block_complete_entry> &blocks);
bool cleanup_handle_incoming_blocks(bool force_sync = false);
bool check_incoming_block_size(const blobdata& block_blob);
i_cryptonote_protocol* get_protocol(){return m_pprotocol;}

5
src/cryptonote_core/difficulty.cpp
View file

@ -45,10 +45,7 @@ namespace cryptonote {
using std::uint64_t;
using std::vector;
#if defined(_MSC_VER) || defined(__MINGW32__)
#include <windows.h>
#include <winnt.h>
#if defined(__x86_64__)
static inline void mul(uint64_t a, uint64_t b, uint64_t &low, uint64_t &high) {
low = mul128(a, b, &high);
}

13
src/cryptonote_core/tx_pool.cpp
View file

@ -128,7 +128,11 @@ namespace cryptonote
crypto::hash max_used_block_id = null_hash;
uint64_t max_used_block_height = 0;
#if BLOCKCHAIN_DB == DB_LMDB
bool ch_inp_res = m_blockchain.check_tx_inputs(tx, max_used_block_height, max_used_block_id, kept_by_block);
#else
bool ch_inp_res = m_blockchain.check_tx_inputs(tx, max_used_block_height, max_used_block_id);
#endif
CRITICAL_REGION_LOCAL(m_transactions_lock);
if(!ch_inp_res)
{
@ -203,6 +207,9 @@ namespace cryptonote
bool tx_memory_pool::remove_transaction_keyimages(const transaction& tx)
{
CRITICAL_REGION_LOCAL(m_transactions_lock);
// ND: Speedup
// 1. Move transaction hash calcuation outside of loop. ._.
crypto::hash actual_hash = get_transaction_hash(tx);
BOOST_FOREACH(const txin_v& vi, tx.vin)
{
CHECKED_GET_SPECIFIC_VARIANT(vi, const txin_to_key, txin, false);
@ -211,11 +218,11 @@ namespace cryptonote
<< "transaction id = " << get_transaction_hash(tx));
std::unordered_set<crypto::hash>& key_image_set = it->second;
CHECK_AND_ASSERT_MES(key_image_set.size(), false, "empty key_image set, img=" << txin.k_image << ENDL
<< "transaction id = " << get_transaction_hash(tx));
<< "transaction id = " << actual_hash);
auto it_in_set = key_image_set.find(get_transaction_hash(tx));
auto it_in_set = key_image_set.find(actual_hash);
CHECK_AND_ASSERT_MES(it_in_set != key_image_set.end(), false, "transaction id not found in key_image set, img=" << txin.k_image << ENDL
<< "transaction id = " << get_transaction_hash(tx));
<< "transaction id = " << actual_hash);
key_image_set.erase(it_in_set);
if(!key_image_set.size())
{

12
src/cryptonote_protocol/cryptonote_protocol_handler.inl
View file

@ -315,6 +315,9 @@ namespace cryptonote
if(context.m_state != cryptonote_connection_context::state_normal)
return 1;
std::list<block_complete_entry> blocks;
blocks.push_back(arg.b);
m_core.prepare_handle_incoming_blocks(blocks);
for(auto tx_blob_it = arg.b.txs.begin(); tx_blob_it!=arg.b.txs.end();tx_blob_it++)
{
cryptonote::tx_verification_context tvc = AUTO_VAL_INIT(tvc);
@ -323,6 +326,7 @@ namespace cryptonote
{
LOG_PRINT_CCONTEXT_L1("Block verification failed: transaction verification failed, dropping connection");
m_p2p->drop_connection(context);
m_core.cleanup_handle_incoming_blocks();
return 1;
}
}
@ -331,6 +335,7 @@ namespace cryptonote
block_verification_context bvc = boost::value_initialized<block_verification_context>();
m_core.pause_mine();
m_core.handle_incoming_block(arg.b.block, bvc); // got block from handle_notify_new_block
m_core.cleanup_handle_incoming_blocks(true);
m_core.resume_mine();
if(bvc.m_verifivation_failed)
{
@ -536,7 +541,7 @@ namespace cryptonote
if (m_core.get_test_drop_download() && m_core.get_test_drop_download_height()) { // DISCARD BLOCKS for testing
m_core.prepare_handle_incoming_blocks(arg.blocks);
BOOST_FOREACH(const block_complete_entry& block_entry, arg.blocks)
{
// process transactions
@ -550,6 +555,7 @@ namespace cryptonote
LOG_ERROR_CCONTEXT("transaction verification failed on NOTIFY_RESPONSE_GET_OBJECTS, \r\ntx_id = "
<< epee::string_tools::pod_to_hex(get_blob_hash(tx_blob)) << ", dropping connection");
m_p2p->drop_connection(context);
m_core.cleanup_handle_incoming_blocks();
return 1;
}
}
@ -566,12 +572,14 @@ namespace cryptonote
{
LOG_PRINT_CCONTEXT_L1("Block verification failed, dropping connection");
m_p2p->drop_connection(context);
m_core.cleanup_handle_incoming_blocks();
return 1;
}
if(bvc.m_marked_as_orphaned)
{
LOG_PRINT_CCONTEXT_L1("Block received at sync phase was marked as orphaned, dropping connection");
m_p2p->drop_connection(context);
m_core.cleanup_handle_incoming_blocks();
return 1;
}
@ -582,7 +590,7 @@ namespace cryptonote
epee::net_utils::data_logger::get_instance().add_data("block_processing", 1);
} // each download block
m_core.cleanup_handle_incoming_blocks();
} // if not DISCARD BLOCK

13
src/daemon/CMakeLists.txt
View file

@ -26,6 +26,12 @@
# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
set(blocksdat "")
if(APPLE AND PER_BLOCK_CHECKPOINT)
add_custom_command(OUTPUT blocksdat.o COMMAND cd ${CMAKE_CURRENT_SOURCE_DIR} && ${CMAKE_C_COMPILER} -o stub.o -c stub.c COMMAND cd ${CMAKE_CURRENT_SOURCE_DIR} && ld -r -sectcreate __DATA __blocks_dat ../blocks/blocks.dat -o ${CMAKE_CURRENT_BINARY_DIR}/blocksdat.o stub.o)
set(blocksdat "blocksdat.o")
endif()
set(daemon_sources
command_parser_executor.cpp
command_server.cpp
@ -69,7 +75,9 @@ bitmonero_private_headers(daemon
bitmonero_add_executable(daemon
${daemon_sources}
${daemon_headers}
${daemon_private_headers})
${daemon_private_headers}
${blocksdat}
)
target_link_libraries(daemon
LINK_PRIVATE
rpc
@ -90,8 +98,7 @@ target_link_libraries(daemon
${CMAKE_THREAD_LIBS_INIT}
${UPNP_LIBRARIES}
${EXTRA_LIBRARIES})
add_dependencies(daemon
version)
add_dependencies(daemon version)
set_property(TARGET daemon
PROPERTY
OUTPUT_NAME "bitmonerod")

27
src/daemon/command_line_args.h
View file

@ -75,7 +75,32 @@ namespace daemon_args
, "Specify database type"
, "lmdb"
};
const command_line::arg_descriptor<uint64_t> arg_prep_blocks_threads = {
"prep-blocks-threads"
, "Max number of threads to use when preparing block hashes in groups."
, 4
};
const command_line::arg_descriptor<uint64_t> arg_fast_block_sync = {
"fast-block-sync"
, "Test fast block-sync option using temporarily embedded known block hashes."
, 1
};
const command_line::arg_descriptor<uint64_t> arg_show_time_stats = {
"show-time-stats"
, "Show time-stats when processing blocks/txs and disk synchronization."
, 1
};
const command_line::arg_descriptor<uint64_t> arg_db_auto_remove_logs = {
"db-auto-remove-logs"
, "For BerkeleyDB only. Remove transactions logs automatically."
, 1
};
const command_line::arg_descriptor<std::string> arg_db_sync_mode = {
"db-sync-mode"
, "Specify sync option, using format [safe|fast|fastest]:[sync|async]:[nblocks_per_sync]."
, "fastest:async:1000"
};
;
} // namespace daemon_args
#endif // DAEMON_COMMAND_LINE_ARGS_H

6
src/daemon/main.cpp
View file

@ -86,6 +86,12 @@ int main(int argc, char const * argv[])
command_line::add_arg(core_settings, daemon_args::arg_testnet_on);
command_line::add_arg(core_settings, daemon_args::arg_dns_checkpoints);
command_line::add_arg(core_settings, daemon_args::arg_db_type);
command_line::add_arg(core_settings, daemon_args::arg_prep_blocks_threads);
command_line::add_arg(core_settings, daemon_args::arg_fast_block_sync);
command_line::add_arg(core_settings, daemon_args::arg_db_sync_mode);
command_line::add_arg(core_settings, daemon_args::arg_show_time_stats);
command_line::add_arg(core_settings, daemon_args::arg_db_auto_remove_logs);
daemonizer::init_options(hidden_options, visible_options);
daemonize::t_executor::init_options(core_settings);

3
src/p2p/net_node.h
View file

@ -209,11 +209,12 @@ namespace nodetool
bool set_rate_up_limit(const boost::program_options::variables_map& vm, int64_t limit);
bool set_rate_down_limit(const boost::program_options::variables_map& vm, int64_t limit);
bool set_rate_limit(const boost::program_options::variables_map& vm, uint64_t limit);
bool set_rate_limit(const boost::program_options::variables_map& vm, int64_t limit);
void kill() { ///< will be called e.g. from deinit()
_info("Killing the net_node");
is_closing = true;
if(mPeersLoggerThread != nullptr)
mPeersLoggerThread->join(); // make sure the thread finishes
_info("Joined extra background net_node threads");
}

32
src/p2p/net_node.inl
View file

@ -67,6 +67,8 @@ namespace nodetool
{
namespace
{
const int64_t default_limit_up = 2048;
const int64_t default_limit_down = 8192;
const command_line::arg_descriptor<std::string> arg_p2p_bind_ip = {"p2p-bind-ip", "Interface for p2p network protocol", "0.0.0.0"};
const command_line::arg_descriptor<std::string> arg_p2p_bind_port = {
"p2p-bind-port"
@ -93,7 +95,7 @@ namespace nodetool
const command_line::arg_descriptor<int64_t> arg_limit_rate_up = {"limit-rate-up", "set limit-rate-up [kB/s]", -1};
const command_line::arg_descriptor<int64_t> arg_limit_rate_down = {"limit-rate-down", "set limit-rate-down [kB/s]", -1};
const command_line::arg_descriptor<uint64_t> arg_limit_rate = {"limit-rate", "set limit-rate [kB/s]", 128};
const command_line::arg_descriptor<int64_t> arg_limit_rate = {"limit-rate", "set limit-rate [kB/s]", -1};
const command_line::arg_descriptor<bool> arg_save_graph = {"save-graph", "Save data for dr monero", false};
}
@ -1446,7 +1448,7 @@ namespace nodetool
this->islimitup=true;
if (limit==-1) {
limit=128;
limit=default_limit_up;
this->islimitup=false;
}
@ -1461,7 +1463,7 @@ namespace nodetool
{
this->islimitdown=true;
if(limit==-1) {
limit=128;
limit=default_limit_down;
this->islimitdown=false;
}
limit *= 1024;
@ -1471,19 +1473,31 @@ namespace nodetool
}
template<class t_payload_net_handler>
bool node_server<t_payload_net_handler>::set_rate_limit(const boost::program_options::variables_map& vm, uint64_t limit)
bool node_server<t_payload_net_handler>::set_rate_limit(const boost::program_options::variables_map& vm, int64_t limit)
{
int64_t limit_up = 0;
int64_t limit_down = 0;
if(limit == -1)
{
limit_up = default_limit_up * 1024;
limit_down = default_limit_down * 1024;
}
else
{
limit_up = limit * 1024;
limit_down = limit * 1024;
}
limit *= 1024;
if(this->islimitdown==false && this->islimitup==false) {
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_up_limit( limit );
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_down_limit( limit );
LOG_PRINT_L0("Set limit to " << limit/1024 << " kB/s");
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_up_limit(limit_up);
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_down_limit(limit_down);
}
else if(this->islimitdown==false && this->islimitup==true ) {
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_down_limit( limit );
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_down_limit(limit_down);
}
else if(this->islimitdown==true && this->islimitup==false ) {
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_up_limit( limit );
epee::net_utils::connection<epee::levin::async_protocol_handler<p2p_connection_context> >::set_rate_up_limit(limit_up);
}
return true;

2
tests/core_proxy/core_proxy.h
View file

@ -85,5 +85,7 @@ namespace tests
cryptonote::blockchain_storage &get_blockchain_storage() { throw std::runtime_error("Called invalid member function: please never call get_blockchain_storage on the TESTING class proxy_core."); }
bool get_test_drop_download() {return true;}
bool get_test_drop_download_height() {return true;}
bool prepare_handle_incoming_blocks(const std::list<cryptonote::block_complete_entry> &blocks) { return true; }
bool cleanup_handle_incoming_blocks(bool force_sync = false) { return true; }
};
}