Paxeer Validator Operations Guide

This document covers the complete lifecycle of a validator node, from initial setup through ongoing operations and maintenance. Understanding these concepts is crucial for maintaining a reliable and secure validator operation.

Understanding Validator Responsibilities

A validator in the Paxeer network serves several critical functions. As a validator, you are responsible for:

• Participating in consensus by proposing and validating blocks
• Maintaining high uptime and performance to avoid slashing
• Managing delegator relationships and maintaining transparent operations
• Participating in governance and network upgrades

Paxeer uses Twin Turbo Consensus with machineRFT for fast, reliable finality.

Initial Setup

Initialize node

Before key management and registration, initialize your node with validator mode so that RPC and P2P bind to localhost (recommended for validator security):

paxd init <moniker> --chain-id hyperpax_125-1 --mode validator

The default init mode is full, which binds RPC and P2P to all interfaces (0.0.0.0). For validator (and seed) nodes, use --mode validator or --mode seed so that RPC and P2P listen on localhost only.

Key Management

The security of your validator begins with proper key management. Your validator requires several distinct keys:

# Validator consensus key - Used for signing blocks
paxd tendermint show-validator

# Operator key - Used for managing validator operations
paxd keys add operator

These keys serve different purposes and should be managed with appropriate security measures. The consensus key, stored in priv_validator_key.json, is particularly critical as it's used to sign blocks and could result in slashing if compromised or mishandled.

Hardware Security Module (HSM) Integration

For production validators, using an HSM is strongly recommended to protect your consensus key. Here's how to configure an HSM with your validator:

# Install required libraries
sudo apt-get install opensc pkcs11-utils

# Configure YubiHSM2
yubihsm-connector -d

# Generate key in HSM
yubihsm-shell

# Configure paxd to use HSM
tee "$HOME/.paxeer/config/priv_validator_config.json" << EOF
{
    "chain_id": "hyperpax_125-1",
    "key_type": "yubihsm",
    "state_file": "$HOME/.paxeer/data/priv_validator_state.json",
    "hsm_serial": "YOUR_HSM_SERIAL",
    "hsm_key_id": "YOUR_KEY_ID"
}
EOF

Validator Registration

Before registering your validator, ensure your node is fully synced with the network. The creation of a validator is a crucial step that requires careful consideration of commission parameters.

paxd tx staking create-validator \
    --amount=1000000uhpx \
    --pubkey=$(paxd tendermint show-validator) \
    --moniker="choose_moniker" \
    --chain-id=hyperpax_125-1 \
    --commission-rate="0.10" \
    --commission-max-rate="0.20" \
    --commission-max-change-rate="0.01" \
    --min-self-delegation="1" \
    --gas="auto" \
    --gas-adjustment="1.5" \
    --gas-prices="0.02uhpx" \
    --from=operator

The commission parameters require strategic consideration:

• commission-rate: Your initial commission rate, which should be competitive while ensuring operational sustainability
• commission-max-rate: An upper limit that can never be exceeded, setting a permanent cap on your commission
• commission-max-change-rate: Maximum daily commission change, limiting how quickly you can adjust rates

Monitoring and Alerting

Please refer to the Advanced Operations section for details around monitoring + alerting for your validator and other nodes.

Security Practices

Network Security

Validators may choose to implement a sentry node architecture to protect the block signing node (the validator). This setup helps prevent DDoS attacks on your validator node by creating a layer of defensive proxies:

# Validator node config.toml
[p2p]
pex = false
persistent_peers = "sentry_node_id@sentry_node_ip:26656"
private_peer_ids = ""
addr_book_strict = false

# Sentry node config.toml
[p2p]
pex = true
persistent_peers = "validator_node_id@ip:port"
private_peer_ids = "validator_node_id"
addr_book_strict = true
unconditional_peer_ids = "validator_node_id"

Key Management Practices

Implement secure key backup procedures. Remember to choose the storage media carefully! Mechanical / flash based storage can fail unexpectedly, and cloud storage should never be used.

By default, the wallet key files are stored in your /.paxeer directory root, and the signer/consensus key in /.paxeer/config. Not only the priv_validator_key.json, but the wallet .info and .address file should also be saved.

Example script to encrypt backups of key files:

#!/bin/bash
# Create encrypted backup of validator keys
BACKUP_DIR="/secure/validator/backup"
DATE=$(date +%Y%m%d)

# Backup validator key
tar czf - $HOME/.paxeer/config/priv_validator_key.json | \
    gpg --symmetric --cipher-algo AES256 \
    -o $BACKUP_DIR/validator_key_$DATE.tar.gz.gpg

# Backup keyring
tar czf - $HOME/.paxeer/keyring-file | \
    gpg --symmetric --cipher-algo AES256 \
    -o $BACKUP_DIR/keyring_$DATE.tar.gz.gpg

# Create SHA256 checksums
sha256sum $BACKUP_DIR/*.gpg > $BACKUP_DIR/checksums_$DATE.txt

Maintenance Procedures

Planned Maintenance

When performing planned maintenance, to minimize potential impact:

• Notify delegators (recommended at least 24h in advance)

Don't forget to stop the service!

# Gracefully stop the node
sudo systemctl stop paxd

# Perform maintenance tasks

# Restart services
sudo systemctl start paxd

Governance Participation

As a validator, active participation in governance is required. Governance is the primary tool with which adjustments to various chain parameters are made. Another critical role for validators is to review, and ultimately approve or reject proposed software upgrades to the network.

Governance proposals may be submitted by anyone willing to provide the mandatory (refundable) deposit, and can be voted on by any network user. Only votes by accounts delegating HPX at the end of the voting period for a given proposal will be given weight.

Proposals currently on chain can be queried at any given time:

# List active proposals
paxd query gov proposals --status voting_period

# Vote on a proposal
paxd tx gov vote 1 yes \
    --from operator \
    --chain-id hyperpax_125-1 \
    --gas auto \
    --gas-prices 0.02uhpx

Recovery Procedures

Critical Warning: Double-Signing Prevention

Never run your validator keys on more than one machine simultaneously. If your primary validator goes offline:

1. DO NOT start another validator with the same keys
2. Either recover the original machine or properly migrate keys with absolute certainty the original is offline
3. If unsure about the state of your original validator, seek support before proceeding

The safe approach to recovery is:

1. Diagnose why the original validator is offline
2. If the original validator cannot be recovered, verify it is offline and powered down
3. Only then proceed with key migration to a new machine

Validator Recovery

When you need to recover your validator on a new machine, follow these steps carefully:

# 1. Set up new machine with Paxeer node
# 2. Copy secured backup files
# 3. Restore validator key
gpg -d validator_key_backup.tar.gz.gpg | tar xzf -
# 4. Restore keyring
gpg -d keyring_backup.tar.gz.gpg | tar xzf -
# 5. Start services
sudo systemctl start paxd

After recovery, verify your validator's status and performance:

# Check validator status
paxd status
# Verify signing is working
paxd query slashing signing-info $(paxd tendermint show-validator)