Smartnode Troubleshooting and Maintenence

Table of Contents

Raptoreum Smartnodes require some maintenance to ensure they are up to date, available and providing service to the network. In order to maintain availability and receive consistent payments you should monitor your Smartnode and keep it always up to date.

Updating Your Smartnode #

Manual update #

Login to your Smartnode VIA SSH client:

./raptoreum-cli stop

Confirm the daemon has stopped:

ps aux | grep raptoreumd

Download latest wallet from: https://raptoreum.com/latest 

dk808 Update Script (only if you used the dk808 setup script) #

Login with ssh and from the directory containing update.sh do:

./update.sh

PoSe Bans #

Diagnosing #

If there is a problem with your Smartnode and it is not available to properly offer its services to the network it will become “PoSe banned” and will not receive any further payments until it is fixed. Here are some of the most common reasons for receiving a PoSe ban:

  • raptoreumd is unavailable or not functioning correctly (hung).
  • Resource shortages (cpu, ram).
  • Unstable network or outage.

You can check if your Smartnode is banned directly in the QT wallet in the Smartnodes tab, check the “My Smartnodes Only”. Or by going to the Raptoreum explorer > Smartnodes Tab > search for your Smartnode IP address.

Resolving #

  1. Check if raptoreumd is responding:
./raptoreum-cli getblockchaininfo

If the daemon responds quickly and “blocks” matches the explorer block height the daemon is fine. Quite possibly the PoSe ban was caused by a temporary issue such as resource issue, or network issues. Your PoSe ban should automatically reduce until 0 and be removed.

If the command from step 1 is very slow to respond or does not respond, raptoreumd is in a hung state and needs to be killed. Kill it with:

ps aux | grep -i raptoreumd | awk '{print $2}' | xargs sudo kill -15
  1. If you see “No such process” kill was successful, if you see a process listed you will need to issue the command again to change -15 to -9 which is a harsher termination. Restart raptoreumd:
./raptoreumd

Wait for it to fully sync (same block height as block explorer). For fun you can watch it sync with:

tail -f ~/.raptoreumcore/debug.log  (Ctrl + C to exit tail)

An alternative way to check and see when raptoreumd is done syncing is to use the “top” command. While syncing raptoreumd will consistently use nearly 100% of a core. When that use drops substantially it is done syncing. Depending on the nodes PoSe score it can take anywhere from 2 to 6 hours for PoSe ban to be lifted. To speed that process you can issue a “protx update_service” command and the ban will be lifted in 1-2 blocks.

Here is an example of a “protx update_service” command, this is issued in your local wallet > debug console.

protx update_service "7d62ce3bcf71825te84y5ca4b93a53583a8f1c9c70989e351c0f470088729a83" "your.smartnode.ip:10226" "3808471d6e52fe3b683474075cdf50192babe3b38203187ec0cg18fg54e439a4"

  • First string in “” is the proTxHash. You can get this by right clicking your Smartnode in the QT wallet Smartnode list > Copy proTxHash.

  • Second is self explanatory 🙂

  • Third is the Smartnodes BLS private key. This is stored on your Smartnode in ~/.raptoreumcore/raptoreum.conf and you can retrieve it easily by doing:

cat ~/.raptoreumcore/raptoreum.conf | grep bls

After firing the protx update_service command you should see status change on your Smartnode list from PoSe_banned to ENABLED in anywhere from 1 minute to 6 minutes.

  1. Resolving resource related PoSe ban issues. Make sure raptoreumd always has access to at least 2 full CPU cores. We strongly recommend against mining on your Smartnodes. Mining on them will endup in performance issues and PoSe bans. If you find raptoreumd is sometimes not running this is often due to it being killed by OOM (Out Of Memory Killer) this is a process of the Linux kernel that will kill certain processes when the system is critically low on memory. Search your system logs to help diagnose the issue, as an example on ubuntu 18:
grep -i 'killed process' /var/log/syslog

As an example, if you have a VPS with 4 cores and want to also use it to mine, make sure the miner only uses max 3 cores. (Replace with) We strongly recommend against mining on your Smartnodes. Mining on them will endup in performance issues and PoSe bans.

Monitoring Your Smartnode #

Uptime is crucial and to make sure your Smartnode is online you should monitor it, there are a few ways to do this:

  1. If you used dk808 script to setup your Smartnode it would have asked you if you wanted to monitor your Smartnode. This check script checks your Smartnode status on our block explorer every hour (VIA cronjob). If it finds that your node has been PoSe banned it will kill raptoreumd and restart it.

  2. External monitoring: This uses an external monitor such as https://uptimerobot.com to check if your Smartnode port is open every 5 minutes. If port is closed you can receive notification to any device and or by email. It is easy to setup and free. Default Smartnode port is 10226.

Maintenance #

There may be times where you need to change some Smartnode parameters, such as the IP address. This can be done using the “protx update_service” command from your local wallet using Debug Console.

Example:
protx update_service "c74cec690017fab7cc184c641c9925e4173979b5bca92fbacca86cg9a5c0f18a" "168.121.96.39:10226" "72af332d8992f4e7124516dad6c40069463a607c023b12c8dc71d9a1ce92ebe6"

Lets break down this command:

protx_update_service “ProTx Hash” “Smartnode IP:port” “BLS key or operator secret”

You can find this information at:

  • ProTx Hash: In the Smartnodes tab of your wallet (make sure My Smartnodes box is marked) right click on the Smartnode and “Copy ProTx Hash”.
  • IP, Port, and BLS key: All of these can be found in ~/.raptoreumcore/raptoreum.conf, example:

cat ~/.raptoreumcore/raptoreum.conf

This will display the contents of your raptoreum.conf file. You can see other update options by typing in debug console:

protx update_service help