Commit graph

537 commits

Author SHA1 Message Date
Thomas Winget 7b14d4a17f
Steps toward multiple dbs available -- working
There will need to be some more refactoring for these changes to be
considered complete/correct, but for now it's working.

new daemon cli argument "--db-type", works for LMDB and BerkeleyDB.

A good deal of refactoring is also present in this commit, namely
Blockchain no longer instantiates BlockchainDB, but rather is passed a
pointer to an already-instantiated BlockchainDB on init().
2015-03-25 12:09:44 -04:00
Thomas Winget 874f48bc82
update berkeleydb branch to blockchain branch 2015-03-25 07:30:23 -04:00
Thomas Winget 5c0bc0050c
Merge upstream updates into blockchain branch 2015-03-25 05:56:36 -04:00
Thomas Winget 8855a32044
Merge upstream to daemonize changes
Preparation for PR
2015-03-24 02:47:15 -04:00
warptangent dbdcf11778
blockchain_converter: Add support for resume from last block
Add option "--resume <on|off>" where default is on.
2015-03-23 18:52:55 -07:00
warptangent 488080326c
blockchain_import: lengthen string for line clear 2015-03-22 15:45:37 -07:00
warptangent 7476d2e253
blockchain_export: show progress during export 2015-03-22 15:45:37 -07:00
warptangent 4bedd68d2c
Update Blockchain::get_db() to return reference instead of pointer
Where this method is used, a BlockchainDB object is always expected, so
a pointer is unnecessary and less safe.
2015-03-22 15:45:36 -07:00
warptangent 0386e9925b
Add README for blockchain converter, importer, and exporter utilities 2015-03-22 15:45:25 -07:00
Thomas Winget f78bb00943
Hopefully fixes build on Windows for real this time 2015-03-20 16:56:55 -04:00
Thomas Winget 2b0583b2c6
Hopefully fixes build on Windows 2015-03-20 16:40:54 -04:00
Thomas Winget ead7fad552
BerkeleyDB implementation of BlockchainDB seems to be working! 2015-03-17 22:12:09 -04:00
Thomas Winget 8e3347f310
Pull blockchain changes into berkeleydb branch 2015-03-17 19:52:53 -04:00
Thomas Winget e146027acd
BlockchainBDB passes unit tests 2015-03-17 17:18:45 -04:00
warptangent ffadb6571a
blockchain_export: Add compile-time support for BlockchainDB
This allows an LMDB database to be used as the blockchain to export.

Adjust SOURCE_DB in src/blockchain_converter/blockchain_export.h
depending on needs. Defaults to DB_MEMORY.

DB_MEMORY is a sensible default for users migrating to LMDB, as it
allows the exporter to use the in-memory blockchain while the other
binaries work with LMDB, without recompiling anything.
2015-03-16 11:57:26 -07:00
Thomas Winget 43477b7dac
BerkeleyDB Blockchain building, not working yet
Everything except actually *using* BlockchainBDB is wired up, but the db
itself is not yet working.  Some error about user mem not large enough.
I think I know what this error means, but I can't determine the cause.

Notes: BerkeleyDB does not allow 0-indexing in its recno type databases,
  so block numbers *in the database* will be 1-indexed.  Modifications
  to indexing have been made as needed.
2015-03-16 09:14:51 -04:00
Thomas Winget cade0da8f1
CMake wiring, minor cleanup, minor test addition
Make Cmake things aware of BerkeleyDB and BlockchainBDB

Make the BlockchainDB unit tests aware of BlockchainBDB
2015-03-16 04:17:53 -04:00
Thomas Winget 1bc89398b4
BerkeleyDB BlockchainDB impl copy/paste/modify
LMDB implementation code copy/paste/modified into the Berkeley DB
implementation.  Need to test if it builds, then if it works, and so on,
but the code is all there.
2015-03-16 04:17:48 -04:00
Thomas Winget 5112dc37d7
Try to not pollute cryptonote namespace 2015-03-16 04:17:44 -04:00
Thomas Winget edef0bb771
Initial commit of BDB BlockchainDB implementation
Basically verbatim copy of LMDB implementation, but with the guts ripped
out and includes changed, etc.
2015-03-16 04:17:32 -04:00
warptangent f6cbfb623b
Add blockchain_export utility
Based on work by tomerkon.
See https://github.com/tomerkon/bitmonero
  src/cryptonote_core/bootfilesaver.{h,cpp}
  src/bootfilegen/bootfilegen.cpp
2015-03-16 00:26:59 -07:00
warptangent 275cbd4348
Add support for database open with flags
Add support to:
  - BlockchainDB, BlockchainLMDB
  - blockchain_import utility to open LMDB database with one or more
    LMDB flags.

Sample use:
  $ blockchain_import --database lmdb#nosync
  $ blockchain_import --database lmdb#nosync,nometasync
2015-03-16 00:26:59 -07:00
warptangent cb862cb81a
Add mdb_flags variable to LMDB database open 2015-03-16 00:26:58 -07:00
warptangent 260cc56fae
Add blockchain_import utility
This imports to the blockchain database from an exported blockchain
file.

It can be used to bootstrap a new database or to add blocks to an
existing one.

Supports:
  - both the in-memory and LMDB implementations
  - optional: batching, verification, testnet

See help for usage.

Based on work by tomerkon.
See https://github.com/tomerkon
  src/cryptonote_core/bootfileloader.{h,cpp}
2015-03-16 00:26:51 -07:00
warptangent ca75b4789c
Blockchain: add get_db() accessor, needed for blockchain_import
This handling may be changed in the future.
2015-03-15 13:22:52 -07:00
warptangent a3dd9d10f3
blockchain_converter: Add support for batch transactions
Add log level support.

Add testnet support.

Add command-line options:
  --help
  --data-dir
  --testnet-data-dir
  --testnet
  --log-level
  --batch
  --batch-size
  --block-number

See help for usage. Run at log level 1 to see profiling stats.
2015-03-15 13:22:47 -07:00
warptangent acb5d291b8
Update and relocate comment that applies class wide 2015-03-15 13:22:32 -07:00
Thomas Winget eee3ee7073
BlockchainDB implementations have names now
In order to make things more general, BlockchainDB now has get_db_name()
which should return a string with the "name" of that type of db.
This "name" will be the subfolder name that holds that db type's files
within the monero folder.

Small bugfix: blockchain_converter was not correctly appending this in
the prior hard-coded-string implementation of the subfolder data
directory concept.
2015-03-13 21:39:27 -04:00
Thomas Winget 5eab480cb1
Moved BlockchainDB into its own src/ subfolder
Ostensibly janitorial work, but should be more relevant later down the
line.  Things that depend on core cryptonote things (i.e.
    cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
2015-03-06 15:20:45 -05:00
smooth 7e72e943db update checkpoint 2015-03-06 02:38:06 -08:00
smooth 41a95e7b11 add comment about avoiding overflow 2015-03-05 00:45:54 -08:00
smooth 754a785ee1 minimum subsidy for mining incentives, remove unused LEGACY_FEE define 2015-03-04 23:35:04 -08:00
Riccardo Spagni c01069f352
fixed English word list issue: 'launchpad' should be 'ourselves' 2015-03-04 15:31:24 +02:00
Riccardo Spagni 10e4132e22
fixed English word list issue: 'incline' should have remained 'inline' 2015-03-04 15:29:20 +02:00
Thomas Winget 9dab105e2e
DNS checkpoint loading for testnet should now be correct 2015-03-01 03:36:46 -05:00
Thomas Winget 52f9629bd7
sending commands to forked daemon works on testnet now 2015-03-01 02:35:33 -05:00
Thomas Winget 76289d0e3b
Fix tests building -- function signatures changed 2015-03-01 02:21:19 -05:00
Thomas Winget db53e1956a
revert stop_daemon method to use correct exit
This was changed because sometimes the daemon does not complete its exit
routine with this method, but as it does correctly wind most things down
even if it gets stuck I've changed it back.
2015-02-24 00:54:50 -05:00
Thomas Winget 96cbecffd7
RPC calls for background daemon added in
The RPC calls the daemon executable uses to talk to the running daemon
instance have mostly been added back in.  Rate limiting has not been
added in upstream, but is on its way in a separate effort, so those
calls are still NOPed out.
2015-02-24 00:44:36 -05:00
Thomas Winget 9193d6fb5b
Daemonize changes pulled in -- daemon builds
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return).  Other than that, this *should*
work...at any rate, it builds, and that's something.
2015-02-24 00:05:19 -05:00
Thomas Winget 0ad0784f46
Changed log level of debug message -- too spammy 2015-02-23 18:28:20 -05:00
warptangent 6485dacc2f
BlockchainLMDB: Add profiling to tx_exists() 2015-02-23 00:33:39 -08:00
warptangent 83fb6d8d07
BlockchainLMDB: Add batch transaction support to tx_exists() 2015-02-23 00:33:38 -08:00
warptangent 8529c0ea9a
BlockchainDB, BlockchainLMDB: Add profiling for DB commits 2015-02-23 00:33:38 -08:00
warptangent 7a66b8bbcf
BlockchainDB: Add virtual function declarations for batch transactions 2015-02-23 00:33:38 -08:00
warptangent b7a2d84919
BlockchainLMDB: Add check for open database to two functions 2015-02-23 00:33:38 -08:00
warptangent 58ecc58be1
BlockchainLMDB: Add support for batch transactions 2015-02-23 00:33:37 -08:00
warptangent 8909d7d82e
Improve block and tx processing efficiency by less repeat hashing
BlockchainLMDB::add_block()
  BlockchainLMDB::add_transaction_data()

  BlockchainDB::add_transaction()
2015-02-23 00:33:37 -08:00
warptangent 3676ac5841
Add profiling to block and tx processing 2015-02-23 00:33:37 -08:00
warptangent ce71abd0fe
Move LMDB storage to subfolder 2015-02-23 00:33:37 -08:00