misc-research/Pre-fork-BCH-BTC-Spending

Pre-fork BCH/BTC Spending Analysis

See this article for commentary on the results

How to reproduce the analysis

Be advised that the analysis takes several days of computing time, upwards of 50 GB of available RAM, and about 2.5 Terabytes of storage space to execute. The scripts call this external script file.

Prerequisites

The main analysis is done with the R statistical programming language. R itself can be downloaded here. RStudio is a good IDE for R. Install the necessary R packages with:

install.packages(c("rbch","data.table","future.apply","RSQLite","DBI","igraph","stringr","curl","ggplot2","scales","Cairo","lubridate"))

Linux users may have difficulties installing the required packages due to external dependencies (as they must be compiled from source). Ubuntu users will require build-essential, libcurl4-openssl-dev and libgmp3-dev. Please raise an issue in this GitHub if you encounter any further issues.

You must have a Bitcoin Cash (BCH) full node synced with the transaction index enabled with the -txindex flag. As of now, the analysis has been tested with the Bitcoin Unlimited node implementation. You must also have a Bitcoin Core (BTC) full node synced with the transaction index enabled with the -txindex flag.

It is best to run the following script files successively in separate R sessions for better RAM management.

extract-tx-graphs.R

The extract-tx-graphs.R script file issues JSON-RPC queries to bitcoind, the BTC or BCH node daemon.

In the R script file you must set bitcoin.conf.file to the filepath of your bitcoin.conf file and data.dir to the directory where you want files to be stored. Note that the data.dir in the R file should be a different directory than the bitcoind data dir where the BCH blockchain is stored.

extract-tx-graphs.R should be run twice: once for BCH and once for BTC. data.dir should be different for BCH and BTC. current.block.height should be set to 733867 for BCH and 729896 for BTC to perform the analysis up to March 31, 2022. The bitcoind for BCH (i.e. Bitcoin Unlimited) should be running when the R script is running for BCH. The same goes for BTC. The same bitcoin.conf file can be used for both blockchains.

Your bitcoin.conf file should contain these lines at a minimum:

testnet=0
rpcuser=<USER>
rpcpassword=<PASSWORD>
rpcport=8332
txindex=1
rpcallowip=0.0.0.0/0
rpcbind=localhost
server=1

Input <USER> and <PASSWORD> of your choice. Perform a quick sanity check to ensure you can access your node via JSON-RPC. The following should return data, and not refuse your connection: (user being the value of rpcuser / pass the value of rpcpass)

curl --data-binary '{"jsonrpc":"1.0","id":"curltext","method":"getblockchaininfo","params":[]}' -H 'content-type:text/plain;' http://user:pass@localhost:8332/

The script spawns multiple R process threads to accelerate queries to bitcoind and will take several hours to execute. In the specified data directory, a set of files named tx_graph_height_BEGIN _to_END.rds will be created.

The R object format of these files is:

List of 2
 $ incoming: Classes 'data.table and 'data.frame': N obs. of 3 variables:
  ..$ txid : chr [txid of the transaction]
  ..$ origin.txid : chr [txid of the output that is being spent by the transaction]
  ..$ origin.position : num [the position (of the output being spent) within the output's transaction, indexed from one]
 $ outgoing: Classes 'data.table and 'data.frame': N obs. of 4 variables:
  ..$ txid : chr [txid of the transaction]
  ..$ address : chr [address that the transaction is being spent to]
  ..$ position : int [position of the output being produced by the transaction, indexed from one]
  ..$ value : num [quantity of BCH being spent to each address] 

Note that, unlike the underlying blockchain data, the position of outputs is indexed from one, not from zero.

construct-edgelist.R

The construct-edgelist.R script file produces a SQL database that contains the edge list of the transaction graph. extract-tx-graphs.R should be run twice: once for BCH and once for BTC. data.dir should be different for BCH and BTC. Set data.dir to the same directories as you used in extract-tx-graphs.R.

The script assigns an integer index to every output. Converting the transaction ID + output position to integer indices is necessary so that the transaction graph can be stored in RAM. This script is single-threaded so as to avoid conflicts when writing to the database. It should take a few hours to complete.

get-block-times.R

The get-block-times.R script file produces a .rds file that contains the time that each block was produced. The script should be run twice: once for BCH and once for BTC. bitcoind must be running for the corresponding blockchain. data.dir should be different for BCH and BTC. Set data.dir to the same directories as you used in extract-tx-graphs.R. current.block.height should be set to 733867 for BCH and 729896 for BTC to perform the analysis up to March 31, 2022.

aggregate-spent-status-data.R

The aggregate-spent-status-data.R merges and aggregates the BCH and BTC data that is produced by earlier scripts. bch.data.dir and btc.data.dir should be set to the same directories as you used in extract-tx-graphs.R.

Three data files are produced in the bch.data.dir directory:

pre-fork-BTC-BCH-spent_status.csv with data structure:

53658348 obs. of  6 variables
btc.spent.block_height: int  Block height that the output was spent on the BTC blockchain, if it was spent
bch.spent.block_height: int  Block height that the output was spent on the BCH blockchain, if it was spent
destination_index     : int  Unique integer index of the output that was created by the R script
value                 : num  Value of the output, in bitcoin units
bch.block_time        : POSIXct, format: "YYYY-MM-DD HH:MM:SS" Block time of the BCH block height
btc.block_time        : POSIXct, format: "YYYY-MM-DD HH:MM:SS" Block time of the BTC block height

spent_status_by_day.csv with data structure:

1704 obs. of  9 variables:
block_time.date                : Date, format: "YYYY-MM-DD"
value.btc.unspent.bch.unspent  : num  
outputs.btc.unspent.bch.unspent: int  
value.btc.spent.bch.unspent    : num  
outputs.btc.spent.bch.unspent  : int  
value.btc.unspent.bch.spent    : num  
outputs.btc.unspent.bch.spent  : int  
value.btc.spent.bch.spent      : num  
outputs.btc.spent.bch.spent    : int  

state_trans_by_day.csv with data structure:

1704 obs. of  9 variables:
block_time.date : Date, format: "YYYY-MM-DD"
value.ff.to.tf  : num  
outputs.ff.to.tf: int  
value.ff.to.ft  : num  
outputs.ff.to.ft: int  
value.ff.to.tt  : num  
outputs.ff.to.tt: int  
value.ft.to.tt  : num  
outputs.ft.to.tt: num  
value.tf.to.tt  : num  
outputs.tf.to.tt: num  

See the article linked at the top of this README for information about how to interpret these variables.

create-plots.R

Creates four PNG plots in the Pre-fork-BCH-BTC-Spending/images directory. You must specify bch.data.dir as before.

These plots are:

pre-fork-BTC-BCH-spent-status-by-value.png pre-fork-BTC-BCH-spent-status-by-outputs.png pre-fork-BTC-BCH-state-transition-by-value.png pre-fork-BTC-BCH-state-transition-by-outputs.png