Releases: ArweaveTeam/arweave
Release 2.1.0.1
This release provides a few patches and improvements.
Bug fixes
This release fixes several bugs, one from 2.1.0.0, another one from long ago, which does not have any negative side effects apart from producing a lot of error logs, mostly from the 2.1.0.0 bug.
Also fixed an issue which could make the node crash with a segmentation fault if RocksDB got corrupted. If you experience that, please start the miner with enable automatic_rocksdb_repair.
The ArQL tag index is now off by default
See the commit description for some explanation.
Upgrade instructions
N.2.1.0.1 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.1.0.1.linux-x86_64.tar.gz
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.
If you want to run the miner from the existing Git folder, execute the following command to upgrade:
git fetch --all --tags && git checkout -f N.2.1.0.1
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.1.0.0
This release introduces the capability of storing and sharing large volumes of data. The goal is to sync arbitrarily large portions of the weave efficiently and in a way that keeps the distribution of data across the network uniform, while making storage and processing overhead grow only with the size of the data stored, not with the size of the entire weave.
Large data is uploaded and downloaded in chunks, via the POST /chunk, GET /tx/<id>/offset, and GET /chunk/<offset> endpoints. Client library support is underway.
Transaction data below 12 MiB continues to be accepted and served via the old APIs.
A portion of data can be uploaded immediately, before the corresponding transaction has been mined. The data is stored on disk in a "disk pool". There are some new configuration parameters to tweak it:
disk_pool_data_root_expiration_time, the time in seconds of how long a pending or orphaned data root is kept in disk pool, 2 hours by default;max_disk_pool_buffer_mb, the maximum total size in mebibytes of the pending chunks in the disk pool, 2 GiB by default;max_disk_pool_data_root_buffer_mb, the maximum size in mebibytes of the pending chunks in the disk pool, per data root, default is 50 Mib.
A graceful shutdown script
This release introduces a new storage engine (RocksDB), which may end up in a corrupt state unless closed properly.
Therefore, we introduce a new ./bin/stop script for stopping the miner gracefully. Note that the miner shell does not react to ctrl+c ctrl+c anymore.
Alternatively, one can send a SIGTERM signal to the process or run init:stop(). in the miner shell.
A bugfix
A bug was fixed where the nodes would connect to fewer peers than they are supposed to.
Upgrade instructions
N.2.1.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.1.0.0.linux-x86_64.tar.gz
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.
If you want to run the miner from the existing Git folder, execute the following command to upgrade:
git fetch --all --tags && git checkout -f N.2.1.0.0
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.4
The release includes a bugfix, a new HTTP client, and an improvement.
A new HTTP client
The new client allows to strictly control how much data is downloaded and fetches big blobs of data more efficiently.
A bugfix
A bug is fixed which prevented fresh nodes from joining the network.
An improvement
Accepting data from peers is made more efficient.
Upgrade instructions
N.2.0.0.4 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.4.linux-x86_64.tar.gz
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.
If you want to run the miner from the existing Git folder, execute the following command to upgrade (note the submodule update and the cleaning step):
git fetch --all --tags && git checkout -f N.2.0.0.4
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.3
The release includes a bugfix, a cleanup made after the transition to 2.0, and support for specifying peers via DNS in the command line.
DNS peers
You can now use domain names of your trusted peers on startup. E.g. to use the Arweave peers, you may specify
peer sfo-1.na-west-1.arweave.net peer nyc-1.na-east-1.arweave.net peer tor-1.na-east-2.arweave.net peer ams-1.eu-central-1.arweave.net
A post-2.0 cleanup
There are changes to the miner's bookkeeping, which reduce the memory footprint and increase the reliability of the miner.
A bugfix
A bug was fixed which caused nodes under some very specific conditions to get stuck recovering from a fork.
Upgrade instructions
N.2.0.0.3 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.3.linux-x86_64.tar.gz
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.
If you want to run the miner from the existing Git folder, execute the following command to upgrade:
git fetch --all --tags && git checkout -f N.2.0.0.3
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.2
The release includes a bugfix, a major performance optimization, and a script for removing old wallet lists.
A performance optimization
After the 2.0 upgrade, some nodes were noticed to spend significant amount of time decoding recall data encoded in Base64URL and consequently lag behind.
The release introduces a new Base64URL decoding library, which is more than 20x faster than the previous one.
A script for removing old wallet lists
Starting from the version 2.0, the wallet lists are no longer required for block syncing and recall block verification. Therefore, the node will not download them again after removal. This allows to clean up a lot of disk space.
A syncing bugfix
A bug is fixed which prevented the v2 transaction data from syncing in a timely manner. The tooling for posting v2 transactions has not been published yet, so the bug had a mild effect on the network.
Upgrade instructions
N.2.0.0.2 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.2.linux-x86_64.tar.gz
It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.
If you want to run the miner from the existing Git folder, execute the following command to upgrade (note the submodule update and the cleaning step):
git fetch --all --tags && git checkout -f N.2.0.0.2
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.1
The release includes a RandomX upgrade and a memory usage optimization. Note that the upgrade instructions are slightly different this time if you are running the miner from Git.
RandomX is upgraded to v1.1.7
The new version contains major performance improvements, the increase is up to 20% on certain CPUs with the large pages enabled.
The updated RandomX commit is here.
It contains a few exra changes to make it compatible with the current Arweave miner.
The RAM usage is reduced significantly on big miners
The memory footprint is lowered by roughly 2 GB on the machines with ~20 mining threads.
Upgrade instructions
N.2.0.0.1 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.1.linux-x86_64.tar.gz
It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.
If you want to run the miner from the existing Git folder, install CMake (on Ubuntu, sudo apt install cmake) and execute the following command to upgrade (note the submodule update and the cleaning step):
git fetch --all --tags && git checkout -f N.2.0.0.1 && git submodule update && ./rebar3 clean
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.0
The release is a hard fork that activates at height 422250, approximately 2020-04-09 10:00 UTC. You will need to make sure you have upgraded your miner before this time in order to be connected to the 2.0 network. The release introduces **Fast Write**: a major modification to the core Arweave protocol that removes relevant limits to the write speed of the network. The new _protocol_, enforced by this _client_ release, supports enormous volumes of data. However, the _client_ does not yet accept data separately from transaction headers, constraining the upload speeds not only by the network speeds, but also memory pool and transaction queue limits. The extended _client_ capabilities for managing arbitrarily huge volumes of data are planned for 2.1, but will not require a hard fork. Nonetheless, this upgrade still dramatically increases the practical block size of the network. In addition to increasing the write speed, the new protocol makes block validation more efficient. This is achieved through using a new Merkle tree-based Succinct Proofs of Access scheme, and an iterative block hashing mechanism. Other notable changes: - The memory footprint is reduced by about 1 GB. - Mining bookkeeping costs are reduced. - Data syncing speeds are improved. - The auto-update feature has been dropped. - No firewall checks are made during fork recovery. Since the data is distributed separately from the headers for the new transactions, it is not feasible to do firewall checks on the fly. - It is no longer necessary to store historical block metadata and wallet lists for mining. **Upgrade instructions** N.2.0.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. Download the corresponding archive and unpack it: ``` tar -xzf arweave-2.0.0.0.linux-x86_64.tar.gz ``` It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument. The archive contents look like this: ``` bin/ data/ erts-10.3/ lib/ releases/ ``` To run the miner, execute ./bin/start <arguments>. See the mining guide for more details. If your OS/platform architecture is not in the list, check README for how to build the miner from sources. If you want to run the miner from the existing Git folder, execute the following command to upgrade: ``` git fetch --all --tags && git checkout -f N.2.0.0.0 ``` You can now run the miner using the arweave-server script. **Support** For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 1.9.4.0
**Improvements & bug fixes** - A bottleneck was resolved in the core processing module which slowed down block application in the presence of a large number of transactions. - Transaction propagation parallelisation is made configurable, to allow the network to easily switch to a more conservative distribution rate. **Upgrade instructions** N.1.9.4.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. Download the corresponding archive and unpack it: ``` tar -xzf arweave-1.9.4.0.linux-x86_64.tar.gz ``` It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument. The archive contents look like this: ``` bin/ data/ erts-10.3/ lib/ releases/ ``` To run the miner, execute ./bin/start <arguments>. See the mining guide for more details. If your OS/platform architecture is not in the list, check README for how to build the miner from sources. If you want to run the miner from the existing Git folder, execute the following command to upgrade: ``` git fetch --all --tags && git checkout -f N.1.9.4.0 ``` You can now run the miner using the arweave-server script. **Support** For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 1.9.3.0
**Improvements & bug fixes** The release includes a few improvements to transaction processing and gossiping mechanisms, including in the priority queue, plus miscellaneous bug fixes. - A bug was fixed in the priority queue which caused it to distribute transactions too quickly. - Improvements to the core gossiping mechanics of the network. - Miscellaneous bug fixes. **Upgrade instructions** N.1.9.3.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. Download the corresponding archive and unpack it: ``` tar -xzf arweave-1.9.3.0.linux-x86_64.tar.gz ``` It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument. The archive contents look like this: ``` bin/ data/ erts-10.3/ lib/ releases/ ``` To run the miner, execute ./bin/start <arguments>. See the mining guide for more details. If your OS/platform architecture is not in the list, check README for how to build the miner from sources. If you want to run the miner from the existing Git folder, execute the following command to upgrade: ``` git fetch --all --tags && git checkout -f N.1.9.3.0 ``` You can now run the miner using the arweave-server script. **Support** For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release N.1.9.2.0
The release comes with the new build system. No need to install any dependencies anymore - simply download and run the binary! Please, read the upgrade upgrade instructions below for details - the upgrade instructions for the previous releases would not apply here.
Improvements and bugfixes, the miner:
- The polling mode was made more efficient. In addition to that, even outside the polling mode the node will ask its trusted peers for blocks if it has not received any in the last minute.
- The number of best peers to propagate transactions and blocks to can be specified in the command line.
- The number of file descriptors to use for incoming connections can be specified on the command line. After the limit is reached, the requests are queued.
- Collecting transactions during fork recovery now respects protocol limits to avoid abuse.
- Requests with valid transactions are excluded from the IP-based throttling to ensure timely propagation of excessive transaction volumes.
- Metrics for uploaded and downloaded bytes per endpoint were added.
Improvements and bugfixes, the gateway:
- The SQLite driver was replaced, the previous one had an issue where it occasionally would fail to restart after a crash.
- The missing CORS headers were added to the /:path and /:path/:subpath endpoints.
- The number of file descriptors necessary for incoming gateway connections was reduced.
- Metadata of all transactions of a block is inserted in a single database transaction now.
Upgrade instructions
N.1.9.2.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it.
tar -xzf arweave-1.9.2.0.linux-x86_64.tar.gz
It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.
If you have been already mining before this update, you can either download and unpack the archive in your Git arweave folder and run ./bin/start ... or execute the following to continue building the miner from sources:
git fetch --all --tags && git checkout -f N.1.9.2.0
git submodule update
rm -rf lib _build
You can now run the miner using the arweave-server script.
You can also create a new directory and unpack the archive there, but make sure you use the same data directory (data_dir argument) as you did before. If you used the default, it would be data_dir <path-to-your-previous-arweave-folder> in the mining command. Check this section of the guide if you would like to move your data elsewhere.
If you operate a gateway, note that the TLS certificates need to be put into the apps/arweave/priv/tls folder.
Support
For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us on team@arweave.org.