Commit graph

132 commits

Author SHA1 Message Date
Riccardo Spagni
fd32a3b52a
Merge pull request #869
03e545d More fixing for Windows (iDunk)
e8baa57 CMakeLists.txt: fix build for mingw (iDunk)
2016-06-21 09:32:12 +02:00
iDunk
03e545d323
More fixing for Windows 2016-06-20 22:38:36 +01:00
iDunk
e8baa57326
CMakeLists.txt: fix build for mingw
Fix from iDunk from IRC.
2016-06-20 21:39:13 +01:00
moneromooo-monero
f72388c1c6
CMakeLists: fix build without libunwind 2016-06-20 19:20:14 +01:00
moneromooo-monero
e409e59d29
Print stack trace on exceptions
if libunwind is found.

Useful for debugging logs.
2016-04-28 23:34:51 +01:00
moneromooo-monero
f7301c3563
Revert "Print stack trace upon exceptions"
Ain't nobody got time for link/cmake skullduggery.

This reverts commit fff238ec94.
2016-03-21 10:12:23 +00:00
moneromooo-monero
fff238ec94
Print stack trace upon exceptions
Useful for debugging users' logs
2016-03-19 21:48:36 +00:00
Riccardo Spagni
11de19266b
change to LMDB as default, even on 32-bit and ARM 2016-03-12 21:20:00 +02:00
warptangent
9832d18dca
cmake: Include OpenSSL libraries in static linking 2016-02-12 16:54:14 -08:00
warptangent
c2f2437586
cmake: Remove unused variable
DL is empty and unused elsewhere.

The intention at one point may have been to use CMAKE_DL_LIBS, but that
would more likely apply in some situations involving static linking.
2016-02-12 15:02:12 -08:00
warptangent
7205210b0f
cmake: Fix unbound config compile settings
This allows the OpenSSL function checks to compile in unbound's CMake
configuration.

Otherwise, the functions SHA256() and EVP_sha512() won't be called from
libunbound as possible algorithms.

They had not been compiling because static OpenSSL libraries were being
used, along with lack of -ldl. The static library preference is
unnecessary for the checks, so use default suffixes ordering for
CMAKE_FIND_LIBRARY_SUFFIXES when building unbound.

Related files:
configure_checks.cmake
external/unbound/validator/val_secalgo.c
  secalgo_ds_digest(), setup_key_digest()
2016-02-12 15:02:06 -08:00
warptangent
3d1e690cb2
Add make option to disable BerkeleyDB
Sample use:
BERKELEY_DB=0 make debug

This makes development with BlockchainLMDB easier when virtual methods
have changed and don't match BlockchainBDB.
2016-02-01 13:06:15 -08:00
me0wmix
34ebfe39cd Last fixups in the Cmake stuff and better default build instruction 2016-01-22 20:27:01 -07:00
me0wmix
c965bf4a7c Added/corrected OpenBSD build instructions. 2016-01-21 11:34:02 -07:00
me0wmix
28f95eb001 OpenBSD support for Monero. 2016-01-21 11:18:26 -07:00
Howard Chu
94de39fbad Tweak arm6 flags 2016-01-11 23:14:06 +00:00
Howard Chu
d26e7d0b3a ARMv7 optimization flags are too aggressive 2016-01-03 12:58:08 +00:00
Howard Chu
26aac68095 Fix release flags
flags were being accumulated instead of being set.
Win32 was getting -O2 -DNDEBUG -Ofast -DNDEBUG etc...
Also for Win32, stay compatible to WinXP
2016-01-02 21:53:02 +00:00
Riccardo Spagni
de03926850
updated copyright year 2015-12-31 08:39:56 +02:00
Howard Chu
cfe15b81bd Also set stacksize for Win32 not on MSVC 2015-12-20 18:41:36 +00:00
warptangent
725acc7f17
Replace tabs with two spaces for consistency with rest of codebase
Remove trailing whitespace in same files.
2015-12-15 06:22:06 -08:00
moneromooo-monero
180bcde866
build: default to Berkeley DB for 32 bit and ARM 2015-10-04 19:01:56 +01:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
464dbe7379 Fixed MINGW duplicate library errors when MSYS folder cannot be found.
*Thanks to luigi1111 for reporting.
2015-07-17 19:49:22 -07:00
warptangent
bbb936df0f
Use g++ flag "-pthread" instead of "-lpthread", where preferred
CMake supports this through THREADS_PREFER_PTHREAD_FLAG.

Remove inclusion of pthread library in EXTRA_LIBRARIES, as the
individual CMakeLists.txt files which need pthread already require it
with CMAKE_THREAD_LIBS_INIT.
2015-07-16 05:30:46 -07:00
Riccardo Spagni
54c785d0bb
fixed ARM arch test to prevent breaking on short arch strings 2015-07-16 12:46:38 +02:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
e5d2680094 ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
2015-07-15 23:20:16 -07:00
moneromooo-monero
8623492150
Interpret x86_64 as x86-64 for architecture 2015-07-09 14:56:20 +01:00
Riccardo Spagni
ce974949e2
fixed static assert test 2015-05-31 15:39:56 +02:00
Riccardo Spagni
5001489353
explicitly include libc++ with clang 2015-05-26 11:07:58 +02:00
warptangent
5c025f99a8
Check for Clang before adding compiler flag
See eb565a1ce2
2015-05-13 09:20:02 -07:00
Sergey Kazenyuk
867e98d739 Merge branch 'clang_fixes' into clang_fixes-master 2015-05-06 19:10:51 +03:00
Sergey Kazenyuk
eb565a1ce2 Suppress 'register storage class is deprecated' warning in boost dependency library 2015-04-20 20:02:11 +03:00
Thomas Winget
a8bc7182ea
Merge BlockchainDB into upstream 2015-04-07 17:56:18 -04:00
Thomas Winget
7292dea192
Merges PR #37
New/updated Makefile targets
2015-04-07 16:13:42 -04:00
Thomas Winget
9519526224
Only compile BerkeleyDB as an option in non-static 2015-04-07 15:02:20 -04:00
Riccardo Spagni
4222b372e4
add arm6 target and flags 2015-04-06 14:00:09 +02:00
rfree2monero
c511abf005 remerged; commands JSON. logging upgrade. doxygen 2015-04-01 19:00:45 +02:00
rfree2monero
3cbdf198f1 Merge remote-tracking branch 'monero-official/master' into network-1.6-work1 2015-04-01 18:24:45 +02:00
Thomas Winget
874f48bc82
update berkeleydb branch to blockchain branch 2015-03-25 07:30:23 -04:00
Thomas Winget
5c0bc0050c
Merge upstream updates into blockchain branch 2015-03-25 05:56:36 -04:00
Thomas Winget
51e3579a80
Fixed bug in static linking boost on MINGW
There was a workaround for linking to boost at all on MINGW, but
unfortunately this workaround would not correctly link to boost
statically.  This workaround for that workaround works around the issue
that that workaround had.
2015-03-22 06:25:53 -04:00
Thomas Winget
8e3347f310
Pull blockchain changes into berkeleydb branch 2015-03-17 19:52:53 -04:00
Thomas Winget
6e9e8ab620
Move db_drivers/ to external/
Also change LMDB Cmake variables to CACHE rather than upgrading them
through several parent scopes.
2015-03-17 18:55:24 -04:00
Riccardo Spagni
56a1bf527b
fixed BUILD_64 option 2015-03-17 18:26:58 -04:00
Riccardo Spagni
c97a685a37
fixed arch_width option 2015-03-17 18:26:48 -04:00
Riccardo Spagni
3a3c07c330
fixed msys2 / mingw folders based on architecture, added license to unbound CMakeList as that is not part of standard Unbound 2015-03-17 18:26:38 -04:00
Riccardo Spagni
3b3da86483
moved 32-bit/64-bit detection into main CMakeLists 2015-03-17 18:26:29 -04:00
Thomas Winget
11e815761f
Revert "Moved db_drivers/ into external/ for consistency"
This reverts commit b21335642e.
2015-03-17 18:03:11 -04:00
Thomas Winget
cade0da8f1
CMake wiring, minor cleanup, minor test addition
Make Cmake things aware of BerkeleyDB and BlockchainBDB

Make the BlockchainDB unit tests aware of BlockchainBDB
2015-03-16 04:17:53 -04:00
Thomas Winget
b21335642e
Moved db_drivers/ into external/ for consistency 2015-03-09 15:48:46 -04:00