Commit graph

60 commits

Author SHA1 Message Date
moneromooo-monero
5d65a75b69
move checkpoints in a separate library 2017-09-25 21:16:26 +01:00
stoffu
e29282d208
build: auto update version info without manually deleting version.h 2017-09-21 07:47:37 +09:00
Thomas Winget
77986023c3
json serialization for rpc-relevant monero types
Structured {de-,}serialization methods for (many new) types
which are used for requests or responses in the RPC.

New types include RPC requests and responses, and structs which compose
types within those.

# Conflicts:
#	src/cryptonote_core/blockchain.cpp
2017-09-05 12:20:27 -04:00
moneromooo-monero
71e2876033
debug_utilities: only build for debug builds 2017-08-04 14:45:43 +01:00
moneromooo-monero
fbaf5375c3
cn_deserialize: move to new debug_utilities subdirectory 2017-08-03 16:20:40 +01:00
Jaquee
f5bd346573
IOS CMAKE build settings 2017-04-03 18:38:50 +02:00
Riccardo Spagni
c3599fa7b9
update copyright year, fix occasional lack of newline at line end 2017-02-21 19:38:18 +02:00
kenshi84
8027ce0c75 extract some basic code from libcryptonote_core into libcryptonote_basic 2017-02-08 22:45:15 +09:00
Randi Joseph
9e54616924 Dropped "bit" from bitmonero. 2016-09-26 17:22:30 -04:00
redfish
06bb6923c3 cmake: support BUILD_SHARED_LIBS built-in option
Support building internal libraries as shared. This reduces
development time by eliminating the need to re-link all
binaries every time non-interface code in the library changes.
Instead, can hack on libxyz, then `make libxyz`, and re-run
monerod.

By default BUILD_SHARED_LIBS is OFF in release build type,
and ON in debug build type, but can be overriden with -D.
2016-09-18 02:56:26 -04:00
redfish
e1c7af35d4 cmake: transitive deps and remove deprecated LINK_*
Keep the immediate direct deps at the library that depends on them,
declare deps as PUBLIC so that targets that link against that library
get the library's deps as transitive deps.

Break dep cycle between blockchain_db <-> crytonote_core.
No code refactoring, just hide cycle from cmake so that
it doesn't complain (cycles are allowed only between
static libs, not shared libs).

This is in preparation for supproting BUILD_SHARED_LIBS cmake
built-in option for building internal libs as shared.
2016-09-18 02:56:26 -04:00
redfish
b1d5189ae1 cmake: fix def propagation, fixes 32-bit build
32-bit build would fail to link with 'mdb_env_create undefined' (because
for 32-bit build, mdb_env_create_vl32 is defined instead).

This bug was introduced with the recent change to virtual object
libraries. The problem is that the COMPILE_DEFINITIONS property
was not propagated from dependee target (lmdb) to depedent
target's (blockchain_db) virtual object lib (obj_blockchain_db).
This patch makes that happen.

I chose to include INTERFACE_COMPILE_DEFINITIONS because there
should not be a need to propagate private defs, but it doesn't
make a difference in this case.
2016-09-03 17:44:28 -04:00
Riccardo Spagni
7825386468
fix leftover add_directory 2016-09-03 13:51:07 +02:00
redfish
1de812a92a cmake: merge libs via virtual object libraries
CMake issued a warming about policy CMP0026: access of LOCATION
target property at config time was disallowed. Offending code
was the code that merged static libraries to generate
libwallet_merged.a.

This patch does that same merge task in a much simpler way. And,
since it doesn't violate the policy, the warning went away.
2016-08-30 23:01:48 -04:00
moneromooo-monero
9b1afe5f2d
ringct: import of Shen Noether's ring confidential transactions 2016-08-28 21:26:54 +01:00
redfish
2c5e68fb73 cmake: fix typo in handling of STACK_TRACE
This fixes the log output not getting redirected to log file.
2016-08-19 23:38:44 -04:00
redfish
042db0bd0f cmake: cleanup logic that sets flags per target/subdir
The previous logic that used a COMMON_*_FLAGS intermediate variable
and then re-assigned CMAKE_*_FLAGS before including each subdirectory
was confusing and ugly. This PR is the right way to do it.

This commit is purely refactoring: built binaries unchanged.
2016-07-28 21:20:50 -04:00
Riccardo Spagni
287e88283c
remove connectivity tool, comment it out from munin plugins too 2016-03-25 14:52:19 +02:00
warptangent
3e72d97ca7
cmake: Fix overly broad use of dynamic link settings
Remove LINK_SEARCH_START_STATIC and LINK_SEARCH_END_STATIC. This is more
appropriate when the compiler flag -static is used.

This had been causing CMake to omit the linker flags necesssary to
distinguish static and dynamic library linking. CMake had assumed static
linking for the target, causing it to omit explicit static link flags.
That is problematic without the -static compile flag being set.

With a library located in system directories, like libboost_date_time,
the full static path (.a), though found correctly by CMake, was treated
by the linker as a dynamic library. This is because
target_link_libraries() transforms the full path to -l<libname> if it's
in a system directory.  Without -static or explicit linker flags, the
dynamic library (.so) is linked.

Removing the above two properties removes the assumption of static. So
-Wl;-Bstatic is inserted where needed. This causes -l<libname> to
properly refer to the static library instead of dynamic.
2016-02-12 14:49:43 -08:00
Riccardo Spagni
de03926850
updated copyright year 2015-12-31 08:39:56 +02:00
warptangent
725acc7f17
Replace tabs with two spaces for consistency with rest of codebase
Remove trailing whitespace in same files.
2015-12-15 06:22:06 -08:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
e5d2680094 ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
2015-07-15 23:20:16 -07:00
warptangent
ac011b4312
Rename src/blockchain_converter/ to src/blockchain_utilities/
Update appropriate files (CMakeLists.txt, README.md)
2015-05-08 14:24:51 -07:00
Thomas Winget
a8bc7182ea
Merge BlockchainDB into upstream 2015-04-07 17:56:18 -04:00
rfree2monero
3cbdf198f1 Merge remote-tracking branch 'monero-official/master' into network-1.6-work1 2015-04-01 18:24:45 +02:00
Thomas Winget
5c0bc0050c
Merge upstream updates into blockchain branch 2015-03-25 05:56:36 -04:00
Thomas Winget
5eab480cb1
Moved BlockchainDB into its own src/ subfolder
Ostensibly janitorial work, but should be more relevant later down the
line.  Things that depend on core cryptonote things (i.e.
    cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
2015-03-06 15:20:45 -05:00
Thomas Winget
9193d6fb5b
Daemonize changes pulled in -- daemon builds
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return).  Other than that, this *should*
work...at any rate, it builds, and that's something.
2015-02-24 00:05:19 -05:00
rfree2monero
eabb519605 2014 network limit 1.0a +utils +toc -doc -drmonero
commands and options for network limiting
works very well e.g. for 50 KiB/sec up and down
ToS (QoS) flag
peer number limit
TODO some spikes in ingress/download
TODO problems when other up and down limit
added "otshell utils" - simple logging (with colors, text files channels)
2015-02-20 22:13:00 +01:00
Thomas Winget
4af0918501
very, VERY primitive blockchain converter
hard-coded config folder, hard-coded BlockchainDB subclass.

Needs finessing, but should be testable this way.

update for rebase (warptangent 2015-01-04)
  fix conflicts with upstream CMakeLists.txt files

  src/CMakeLists.txt (edit original commit)
  src/blockchain_converter/CMakeLists.txt (add)
2015-01-04 19:38:56 -08:00
Riccardo Spagni
f4b69d553a
year updated in license 2015-01-02 18:52:46 +02:00
Ben Boeckel
ec54e2f82b cmake: place binaries together in the build tree
This allows Windows to copy binaries to one place rather than to each
executable directory.
2014-11-18 17:02:28 -05:00
Ben Boeckel
e3776876af cmake: Windows and static builds need this 2014-10-24 15:52:01 -04:00
Ben Boeckel
7d708e4223 cmake: support 2.8.7
Older versions of CMake support LINK_{PUBLIC,PRIVATE} while newer
versions prefer PUBLIC and PRIVATE instead, but still support the LINK_
prefix.
2014-10-24 15:29:51 -04:00
Ben Boeckel
464c2805e5 cmake: fix up miniupnpc's define
It's only necessary on Windows builds and new versions renamed the
define without any compatibility bridge.
2014-10-23 16:43:00 -04:00
Ben Boeckel
f53f04724c cmake: handle private vs. public headers 2014-10-23 16:42:34 -04:00
Ben Boeckel
55ca7d3b34 cmake: refactor common code with libraries 2014-10-23 16:42:34 -04:00
Ben Boeckel
c773f465ca cmake: refactor common code with executables 2014-10-23 16:42:34 -04:00
Ben Boeckel
89cff7bddc cmake: put each library into its own directory
This cleans up the CMake code and shows patterns more easily (to be
refactored in the next commit).
2014-10-23 16:42:34 -04:00
Riccardo Spagni
bc537ac095
miniupnpc static define change 2014-10-07 00:44:44 +02:00
Riccardo Spagni
2cf94c1321
fix for mingw not playing nicely with libunbound configure, fix for correctly finding static libs on various operating systems 2014-10-06 22:29:07 +02:00
Riccardo Spagni
fd52015a2d use the correct CMake variable for static builds 2014-10-06 18:08:33 +02:00
Riccardo Spagni
434095d1f7
fixed conflict in CMakeLists 2014-10-05 11:40:25 +02:00
Riccardo Spagni
8c37823d45 remove pthreads, successfully tested on gcc 4.9.1 without pthreads 2014-10-02 18:54:55 +02:00
Oran Juice
b80755063e
Stop copying word list files to build directory 2014-10-02 21:47:49 +05:30
Oran Juice
4f693d715c
Merge with origin/master 2014-09-28 15:52:05 +05:30
Oran Juice
91aa25e055
Informs about old style mnemonics from older wallet and provides a new one. CMakeLists.txt update. 2014-09-27 17:04:23 +05:30
Riccardo Spagni
bff1f9d4c4
redo FindUnbound.cmake 2014-09-24 20:38:24 +02:00
Oran Juice
26ea53d461
Copies word lists directory to the location of the executable 2014-09-24 22:44:36 +05:30
Thomas Winget
dea98df6b1 Updated CMake files -- added libunbound linker flag
CMake config file written, but was unable to test/get it working
properly because of a bug in CMake with functions related to
find_package.  Simple "-lunbound" flag used in its stead for now.  May
not build on non-Linux systems, not sure yet.
2014-09-23 22:58:02 +02:00