Commit graph

65 commits

Author SHA1 Message Date
Javier Smooth
b19456dc6d Re-enable optimized slow-hash if someone is trying to compile w/MSVC (disabling it was unintentional) 2015-08-26 10:30:22 -07:00
Riccardo Spagni
32077d3810
Merge pull request #385
0a4bc84 Added ref10 shen_ed25519_ref code, which includes code that can replace crypto-ops with a version straight from Bernstein's ref 10 (ShenNoether)
0d70fdc revert to 776b4fc91a (ShenNoether)
b01f286 Added shen_ed25519_ref to crypto ops subfolder, the point is to directly have bitmonero's crypto code come from bernstein et al's ref 10 code (ShenNoether)
2015-08-24 19:21:31 +02:00
Riccardo Spagni
6b7be9f89b
Merge pull request #383
3b5330e use correct unsigned type (roman)
59cc92b removed some gcc warnings. mainly unused variables. (roman)
2015-08-24 19:21:04 +02:00
ShenNoether
0a4bc84b2f Added ref10 shen_ed25519_ref code, which includes code that can replace crypto-ops with a version straight from Bernstein's ref 10 2015-08-23 14:48:50 -06:00
ShenNoether
0d70fdca8c revert to 776b4fc91a 2015-08-23 14:46:44 -06:00
ShenNoether
b01f286473 Added shen_ed25519_ref to crypto ops subfolder, the point is to directly have bitmonero's crypto code come from bernstein et al's ref 10 code 2015-08-23 14:18:59 -06:00
roman
3b5330e895 use correct unsigned type 2015-08-23 21:10:24 +02:00
roman
59cc92b388 removed some gcc warnings. mainly unused variables. 2015-08-23 17:59:24 +02:00
moneromooo-monero
98c76a388c
chacha8: add a key generation variant that take a pointer and size 2015-08-22 17:41:03 +01:00
moneromooo-monero
a2d7a5fb49
encrypted payment ids are now 64 bit, instead of 256 bit
Pros:
 - smaller on the blockchain
 - shorter integrated addresses

Cons:
 - less sparseness
 - less ability to embed actual information

The boolean argument to encrypt payment ids is now gone from the
RPC calls, since the decision is made based on the length of the
payment id passed.
2015-08-09 10:13:51 +01:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
e5d2680094 ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
2015-07-15 23:20:16 -07:00
Javier Smooth
1b19ee040f disable non-working optimized slow_hash on 32 bit intel 2015-04-13 16:38:33 -07:00
Riccardo Spagni
b2b612794d
no newline at end of file 2015-04-05 17:29:54 +02:00
Riccardo Spagni
3ece1584f0
make fallback compatible with newer slow-hash 2015-01-28 11:32:52 +02:00
Riccardo Spagni
9b3673a54c
added portable slow-hash, updated makefile targets, fixed readme 2015-01-27 09:43:15 +02:00
Riccardo Spagni
e70bc3d369
fixed sse2 ifdef 2015-01-27 09:15:14 +02:00
Riccardo Spagni
2b3b70a261
remove aes flag, detect no sse2 2015-01-27 09:05:47 +02:00
Riccardo Spagni
f4b69d553a
year updated in license 2015-01-02 18:52:46 +02:00
Ben Boeckel
f53f04724c cmake: handle private vs. public headers 2014-10-23 16:42:34 -04:00
Ben Boeckel
55ca7d3b34 cmake: refactor common code with libraries 2014-10-23 16:42:34 -04:00
Ben Boeckel
89cff7bddc cmake: put each library into its own directory
This cleans up the CMake code and shows patterns more easily (to be
refactored in the next commit).
2014-10-23 16:42:34 -04:00
David G. Andersen
28dfb4b178 capitalize Keccak everywhere 2014-10-06 17:44:23 -04:00
David G. Andersen
ce2ad65599 typo fix - cryptonight, not cryptonote 2014-10-06 17:42:58 -04:00
David G. Andersen
e4fb913b2d Fix comment start for two functions 2014-10-06 10:15:21 -04:00
David G. Andersen
46f26ffe88 another typo fix 2014-10-05 20:32:49 -04:00
David G. Andersen
ac6bc4859f fix typo 2014-10-05 14:14:59 -04:00
David G. Andersen
d744dd1be5 More documentation 2014-10-05 13:47:13 -04:00
David G. Andersen
4d493f6d4f initial doxygen commenting of the CryptoNight proof-of-work code 2014-10-05 12:33:39 -04:00
Oran Juice
4f693d715c
Merge with origin/master 2014-09-28 15:52:05 +05:30
Oran Juice
6be139b511
Moved mnemonics code to src/mnemonics 2014-09-23 17:04:04 +05:30
Zachary Michaels
aba3497fdd More preprocessor fixes for slow-hash 2014-09-15 22:33:10 +02:00
Zachary Michaels
cf91545734 Correct includes for mingw in slow-hash
This needs testing
2014-09-15 22:33:10 +02:00
Zachary Michaels
2b76e9d8a3 Change Windows include to windows
mingw is case sensitive
2014-09-15 22:33:10 +02:00
fluffypony
ccb2ab2b7b mmap on FreeBSD doesn't have MAP_HUGETLB 2014-09-10 14:01:39 +02:00
fluffypony
110f110181 Include sys/time.h on BSD 2014-09-10 13:58:43 +02:00
fluffypony
a8d043b6dd replace ftime with gettimeofday on FreeBSD because lcompat is stupid 2014-09-10 13:55:39 +02:00
fluffypony
266c639f4f more FreeBSD fixes 2014-09-09 14:49:13 +02:00
fluffypony
e1d31e0a8b malloc.h reference fixed for FreeBSD 2014-09-09 14:07:57 +02:00
fluffypony
6afbdd9754 FreeBSD alloca.h reference fixed 2014-09-09 12:37:19 +02:00
rfree2monero
2ef0aee81d Fix tree-hash cnt n^2. Asserts, comment. Squash2 2014-09-06 21:56:24 +02:00
fluffypony
f8025cada9 fixed OS X malloc.h reference 2014-09-02 12:08:27 +02:00
fluffypony
6fc995fe5d License updated to BSD 3-clause 2014-07-23 15:03:52 +02:00
Zachary Michaels
849c52c7ed Remove unused array in cn_slow_hash 2014-07-01 14:57:25 -04:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
e12ccdf905 Update slow-hash.c
1. Fix for Mac OSX compilation errors.
2014-06-19 23:49:25 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
156312f64a Update slow-hash.c
1. Added multiplication support in 32-bit mode
2014-06-16 21:13:05 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
72643c47da Update slow-hash.c
1. Added huge pages support and optimized scratchpad twiddling. (credits to dga).
2. Added aes-ni key expansion support.
3. Minor speedup to scratchpad initialization/finalization.
2014-06-16 03:35:09 -07:00
Thomas Winget
537b911af1 stupid signed arithmetic. 2014-06-12 19:05:52 -04:00
Thomas Winget
8bc032ed09 more pointer-based derp 2014-06-12 19:05:51 -04:00
Thomas Winget
72c3f36ca4 fixed some pointer- and loop-based derps 2014-06-12 19:05:51 -04:00
Thomas Winget
d22e458c6c builds, but doesn't link. other than that, electrum-style recovery implemented (but not tested\!) 2014-06-12 19:05:51 -04:00