Commit graph

784 commits

Author SHA1 Message Date
Thomas Winget 8e3347f310
Pull blockchain changes into berkeleydb branch 2015-03-17 19:52:53 -04:00
Thomas Winget 38c54b79fd
Merges PR #35 2015-03-17 19:07:09 -04:00
Thomas Winget bd9da6d5a5
Merges #33
Because of some github fuckery I had to manually cherry-pick/rebase
fluffypony's 32- and 64-bit lmdb changes, hence this isn't actually a
merge commit.
2015-03-17 18:58:09 -04:00
Thomas Winget 6e9e8ab620
Move db_drivers/ to external/
Also change LMDB Cmake variables to CACHE rather than upgrading them
through several parent scopes.
2015-03-17 18:55:24 -04:00
Riccardo Spagni 56a1bf527b
fixed BUILD_64 option 2015-03-17 18:26:58 -04:00
Riccardo Spagni c97a685a37
fixed arch_width option 2015-03-17 18:26:48 -04:00
Riccardo Spagni 3a3c07c330
fixed msys2 / mingw folders based on architecture, added license to unbound CMakeList as that is not part of standard Unbound 2015-03-17 18:26:38 -04:00
Riccardo Spagni 3b3da86483
moved 32-bit/64-bit detection into main CMakeLists 2015-03-17 18:26:29 -04:00
Riccardo Spagni a0cbf733b6
removed findlmdb.cmake, no longer required 2015-03-17 18:26:18 -04:00
Riccardo Spagni 0e8bbdb0bd
32-bit LMDB, removed check for in-system LMDB as it is consensus-critical 2015-03-17 18:25:28 -04:00
Thomas Winget 11e815761f
Revert "Moved db_drivers/ into external/ for consistency"
This reverts commit b21335642e.
2015-03-17 18:03:11 -04:00
Thomas Winget daf2a8f6cb
Revert "Build fixed, goofed up some CMake"
This reverts commit 8b82f3c57f.
2015-03-17 18:01:25 -04:00
Thomas Winget e146027acd
BlockchainBDB passes unit tests 2015-03-17 17:18:45 -04:00
warptangent ffadb6571a
blockchain_export: Add compile-time support for BlockchainDB
This allows an LMDB database to be used as the blockchain to export.

Adjust SOURCE_DB in src/blockchain_converter/blockchain_export.h
depending on needs. Defaults to DB_MEMORY.

DB_MEMORY is a sensible default for users migrating to LMDB, as it
allows the exporter to use the in-memory blockchain while the other
binaries work with LMDB, without recompiling anything.
2015-03-16 11:57:26 -07:00
Thomas Winget 43477b7dac
BerkeleyDB Blockchain building, not working yet
Everything except actually *using* BlockchainBDB is wired up, but the db
itself is not yet working.  Some error about user mem not large enough.
I think I know what this error means, but I can't determine the cause.

Notes: BerkeleyDB does not allow 0-indexing in its recno type databases,
  so block numbers *in the database* will be 1-indexed.  Modifications
  to indexing have been made as needed.
2015-03-16 09:14:51 -04:00
Thomas Winget cade0da8f1
CMake wiring, minor cleanup, minor test addition
Make Cmake things aware of BerkeleyDB and BlockchainBDB

Make the BlockchainDB unit tests aware of BlockchainBDB
2015-03-16 04:17:53 -04:00
Thomas Winget 1bc89398b4
BerkeleyDB BlockchainDB impl copy/paste/modify
LMDB implementation code copy/paste/modified into the Berkeley DB
implementation.  Need to test if it builds, then if it works, and so on,
but the code is all there.
2015-03-16 04:17:48 -04:00
Thomas Winget 5112dc37d7
Try to not pollute cryptonote namespace 2015-03-16 04:17:44 -04:00
Thomas Winget edef0bb771
Initial commit of BDB BlockchainDB implementation
Basically verbatim copy of LMDB implementation, but with the guts ripped
out and includes changed, etc.
2015-03-16 04:17:32 -04:00
warptangent f6cbfb623b
Add blockchain_export utility
Based on work by tomerkon.
See https://github.com/tomerkon/bitmonero
  src/cryptonote_core/bootfilesaver.{h,cpp}
  src/bootfilegen/bootfilegen.cpp
2015-03-16 00:26:59 -07:00
warptangent 275cbd4348
Add support for database open with flags
Add support to:
  - BlockchainDB, BlockchainLMDB
  - blockchain_import utility to open LMDB database with one or more
    LMDB flags.

Sample use:
  $ blockchain_import --database lmdb#nosync
  $ blockchain_import --database lmdb#nosync,nometasync
2015-03-16 00:26:59 -07:00
warptangent cb862cb81a
Add mdb_flags variable to LMDB database open 2015-03-16 00:26:58 -07:00
warptangent 260cc56fae
Add blockchain_import utility
This imports to the blockchain database from an exported blockchain
file.

It can be used to bootstrap a new database or to add blocks to an
existing one.

Supports:
  - both the in-memory and LMDB implementations
  - optional: batching, verification, testnet

See help for usage.

Based on work by tomerkon.
See https://github.com/tomerkon
  src/cryptonote_core/bootfileloader.{h,cpp}
2015-03-16 00:26:51 -07:00
warptangent ca75b4789c
Blockchain: add get_db() accessor, needed for blockchain_import
This handling may be changed in the future.
2015-03-15 13:22:52 -07:00
warptangent a3dd9d10f3
blockchain_converter: Add support for batch transactions
Add log level support.

Add testnet support.

Add command-line options:
  --help
  --data-dir
  --testnet-data-dir
  --testnet
  --log-level
  --batch
  --batch-size
  --block-number

See help for usage. Run at log level 1 to see profiling stats.
2015-03-15 13:22:47 -07:00
warptangent acb5d291b8
Update and relocate comment that applies class wide 2015-03-15 13:22:32 -07:00
Thomas Winget eee3ee7073
BlockchainDB implementations have names now
In order to make things more general, BlockchainDB now has get_db_name()
which should return a string with the "name" of that type of db.
This "name" will be the subfolder name that holds that db type's files
within the monero folder.

Small bugfix: blockchain_converter was not correctly appending this in
the prior hard-coded-string implementation of the subfolder data
directory concept.
2015-03-13 21:39:27 -04:00
Thomas Winget 5d2a2b7e69
Fixed includes in BlockchainDB unit tests 2015-03-10 15:17:46 -04:00
Thomas Winget 8b82f3c57f
Build fixed, goofed up some CMake
Forgot that CMake vars set to PARENT_SCOPE will still vanish if that
parent scope goes...out of scope.  LMDB vars elevated one more scope to
compensate for moving db_drivers/ into external/
2015-03-10 12:51:23 -04:00
Thomas Winget b21335642e
Moved db_drivers/ into external/ for consistency 2015-03-09 15:48:46 -04:00
Thomas Winget 5eab480cb1
Moved BlockchainDB into its own src/ subfolder
Ostensibly janitorial work, but should be more relevant later down the
line.  Things that depend on core cryptonote things (i.e.
    cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
2015-03-06 15:20:45 -05:00
Thomas Winget 0ad0784f46
Changed log level of debug message -- too spammy 2015-02-23 18:28:20 -05:00
Thomas Winget b5796da0fa
Merge #32
Add support to BlockchainDB and BlockchainLMDB for batch transactions.

Add profiling to block and tx processing and DB operations.

Improve block and tx processing efficiency by less repeat hashing.

Move LMDB storage to "lmdb" subfolder.
 - Upon startup, if old LMDB files are detected, abort with a message for the user to move them to subfolder or delete them.

Update and fix log statements and formatting.
2015-02-23 18:24:59 -05:00
warptangent 6485dacc2f
BlockchainLMDB: Add profiling to tx_exists() 2015-02-23 00:33:39 -08:00
warptangent 83fb6d8d07
BlockchainLMDB: Add batch transaction support to tx_exists() 2015-02-23 00:33:38 -08:00
warptangent 8529c0ea9a
BlockchainDB, BlockchainLMDB: Add profiling for DB commits 2015-02-23 00:33:38 -08:00
warptangent 7a66b8bbcf
BlockchainDB: Add virtual function declarations for batch transactions 2015-02-23 00:33:38 -08:00
warptangent b7a2d84919
BlockchainLMDB: Add check for open database to two functions 2015-02-23 00:33:38 -08:00
warptangent 58ecc58be1
BlockchainLMDB: Add support for batch transactions 2015-02-23 00:33:37 -08:00
warptangent 8909d7d82e
Improve block and tx processing efficiency by less repeat hashing
BlockchainLMDB::add_block()
  BlockchainLMDB::add_transaction_data()

  BlockchainDB::add_transaction()
2015-02-23 00:33:37 -08:00
warptangent 3676ac5841
Add profiling to block and tx processing 2015-02-23 00:33:37 -08:00
warptangent ce71abd0fe
Move LMDB storage to subfolder 2015-02-23 00:33:37 -08:00
warptangent 42f8fe5c7f
Fix formatting 2015-02-23 00:33:36 -08:00
warptangent aa82f786c7
Fix log statement 2015-02-23 00:33:36 -08:00
warptangent 26873db199
Remove unused variable 2015-02-23 00:33:36 -08:00
warptangent 4b90fd389d
Add log statement 2015-02-23 00:33:36 -08:00
warptangent 2531aa31f8
Add and extend log statements 2015-02-23 00:33:35 -08:00
warptangent 59305d3137
Blockchain: match original function declaration from blockchain_storage 2015-02-23 00:33:35 -08:00
warptangent cd972bdcc2
Update year and formatting in license 2015-02-23 00:33:26 -08:00
Thomas Winget b7d112f601 Merge pull request #31 from warptangent/fix/fix_get_tail_id
Fix Blockchain::get_tail_id() to set parameter to last block number instead of height
2015-02-22 18:06:44 -05:00