React to New Blocks and Transactions with ZMQ

1What the node publishes

When the node accepts a transaction into its mempool, or connects a new block, it can push a message on a ZeroMQ PUB socket. Your program keeps a SUB socket connected and reacts. There are four topics; each one is turned on separately.

You can see this exact stream from the site's testnet node, in a browser, at the live ZMQ viewer. It subscribes to all four topics of the same divid you are about to configure.

2Turn it on in divi.conf

Add one line per topic you want to your divi.conf (where that file lives is in the testnet node tutorial). All four can share one address. Then restart divid.

# divi.conf: ZeroMQ publisher. Any free port works; 28332 is the usual choice.
zmqpubhashblock=tcp://127.0.0.1:28332
zmqpubhashtx=tcp://127.0.0.1:28332
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28332

The same four settings work as command-line flags (-zmqpubhashblock=…). divid --help lists them under "Enable publish hash block in <address>" and so on. If none of the four is set, the node creates no ZMQ socket at all.

Check it bound:

ss -ltnp | grep 28332
LISTEN 0 100 127.0.0.1:28332 0.0.0.0:* users:(("divid",…))

There is no RPC call that reports the ZMQ settings. divi-cli getzmqnotifications (which Bitcoin Core has) answers Method not found on Divi; read divi.conf instead.

3The message format

Every message is a ZeroMQ multipart message with exactly three frames:

1. Topic: the ASCII string hashblock, hashtx, rawblock or rawtx. ZeroMQ matches your subscription against the start of this frame.
2. Body: 32 bytes for the hash topics, the serialized block or transaction for the raw ones.
3. Sequence: a 4-byte unsigned integer, little-endian. Each topic has its own counter that starts at 0 when divid starts and goes up by one per message.

The 32-byte hashes are sent in display order: the node reverses the bytes before sending, so body.hex() is the same string you paste into the explorer or pass to getblock. No flipping needed.

4Subscribe from Python

Needs pyzmq. Tested with Python 3.10 and pyzmq 27.2.0:

python3 -m venv venv && . venv/bin/activate
pip install pyzmq==27.2.0

Save as zmq_watch.py:

"""Print every new block and transaction the node sees. Needs pyzmq."""
import sys
import zmq

endpoint = sys.argv[1] if len(sys.argv) > 1 else "tcp://127.0.0.1:28332"
ctx = zmq.Context()
sub = ctx.socket(zmq.SUB)
sub.connect(endpoint)
for topic in ("hashblock", "hashtx"):
    sub.setsockopt_string(zmq.SUBSCRIBE, topic)

print("listening on", endpoint, flush=True)
while True:
    topic, body, seq = sub.recv_multipart()
    print(f"{topic.decode():9} {body.hex()}  #{int.from_bytes(seq, 'little')}", flush=True)

Run it and leave it. On testnet, with one block a minute, this is what we saw while a payment of ours went through (the node's ZMQ port was 53720 on our machine):

$ python3 zmq_watch.py tcp://127.0.0.1:53720
listening on tcp://127.0.0.1:53720
hashtx    36a1bb249bec76e38e5b4d67bcc13131b60952b782c705aea8f09df024c0f5d5  #721914
hashtx    1d031f2110fb4ed033eaa832f8375f581bf8fdc0531d2ba0407689b3be6c7a91  #721915
hashtx    e5be59414292bc118d9c91392c076de4da2d729da9b66a04dd0bb251218a7e79  #721916
hashtx    61aa675e4416a3b888324065fa08be55067f3eee8343e5482797f909a752f375  #721917
hashtx    8690d187995b86c35731b3481b0b13aeaf094d61ddea09a1adc07944182d64e0  #721918
hashblock 5e48f6ff7814ce9723ff34c49a16feaa9634e3c0e9e2b1a9e74f3bc90ec27c6f  #368
hashtx    b035900017e217f10469788a1ed59d09df7036cfbd31a8f41cbcd0ba57b79397  #721919
hashtx    de95d6a17ef119a6397bb1d36d2b2bd87aab5302c8604aa1ea27a81781533195  #721920
hashtx    36a1bb249bec76e38e5b4d67bcc13131b60952b782c705aea8f09df024c0f5d5  #721921
hashblock cd497a17a5355d44847ca5cc79162d4289f24fdb9cdbd2ac163e87463cfc5512  #369

Read it top to bottom. Our transaction 36a1bb24…f5d5 appears at #721914 the moment we broadcast it (mempool). A minute later block cd497a17…5512 (testnet height 359,948) is connected: its three transactions come as hashtx #721919 to #721921, ours last, then the hashblock #369. The two counters are independent, and here they tell a story: since it started, this node had published 721,921 transaction messages but only 369 block messages. The node does not send hashblock while it is still in its initial download, but it does send hashtx for every transaction in every block it connects, so a fresh sync shows up as a flood of hashtx and no blocks.

5Subscribe from JavaScript

Needs the zeromq package. Tested with Node v20 and zeromq 6.8.0:

npm init -y >/dev/null && npm install zeromq@6.8.0

Save as zmq_watch.mjs:

// Print every new block and transaction the node sees. Needs the zeromq package.
import { Subscriber } from "zeromq";

const endpoint = process.argv[2] ?? "tcp://127.0.0.1:28332";
const sock = new Subscriber();
sock.connect(endpoint);
sock.subscribe("hashblock", "hashtx");

console.log("listening on", endpoint);
for await (const [topic, body, seq] of sock) {
  console.log(topic.toString().padEnd(9), body.toString("hex"), "#" + seq.readUInt32LE(0));
}

Run with node zmq_watch.mjs tcp://127.0.0.1:28332. Against the same node it prints the same kind of lines as the Python script (these two came a few minutes before the Python capture above, hence the lower sequence numbers):

$ node zmq_watch.mjs tcp://127.0.0.1:53720
listening on tcp://127.0.0.1:53720
hashtx    14105a86f75931561690fc3e67784e414020d4f5285c21c37ed02e3d22a2112c #721893
hashtx    688c9620cd9191ed571eda04668cd550fd50d8325e0c574a23ae53be9e602f23 #721894

6Act on each event

The hash topics only tell you which block or transaction. Fetch the details over JSON-RPC on the same node (the client is explained in Talk to a Divi node from Python and JavaScript). For a block call getblock <hash>; for a transaction call getrawtransaction <txid> 1, or gettransaction <txid> if it touches your wallet. This version of the Python watcher does that and also checks the sequence numbers (next step):

"""Act on each event, and notice when the node's sequence numbers skip. Needs pyzmq."""
import base64, json, sys, urllib.request, zmq

endpoint = sys.argv[1] if len(sys.argv) > 1 else "tcp://127.0.0.1:28332"
rpc_url, rpc_auth = "http://127.0.0.1:51475/", ("rpcuser", "rpcpassword")  # testnet RPC port; read the values from divi.conf

def rpc(method, *params):
    req = urllib.request.Request(rpc_url, json.dumps({"method": method, "params": list(params), "id": 1}).encode(),
                                 {"Content-Type": "application/json"})
    req.add_header("Authorization", "Basic " + base64.b64encode(f"{rpc_auth[0]}:{rpc_auth[1]}".encode()).decode())
    try:
        body = urllib.request.urlopen(req).read()
    except urllib.error.HTTPError as e:
        body = e.read()          # divid answers RPC errors with an HTTP error status and a JSON body
    return json.loads(body)

sub = zmq.Context().socket(zmq.SUB)
sub.connect(endpoint)
sub.setsockopt_string(zmq.SUBSCRIBE, "hashblock")
sub.setsockopt_string(zmq.SUBSCRIBE, "hashtx")
last = {}
while True:
    topic, body, seq = sub.recv_multipart()
    topic, h, n = topic.decode(), body.hex(), int.from_bytes(seq, "little")
    if topic in last and n != last[topic] + 1:
        print(f"GAP on {topic}: expected #{last[topic] + 1}, got #{n}", flush=True)
    last[topic] = n
    if topic == "hashblock":
        b = rpc("getblock", h)["result"]
        print(f"block {b['height']} with {len(b['tx'])} tx", flush=True)
    else:
        t = rpc("getrawtransaction", h, 1)["result"]
        state = "confirmed" if t.get("confirmations") else "in mempool"
        print(f"tx {h[:16]}… {state}, {len(t['vout'])} outputs", flush=True)

Two and a half minutes of it on testnet:

tx d88156f6da67cd4b… in mempool, 2 outputs
tx e0efaf43fd176ab9… in mempool, 5 outputs
tx d4033f97eacfd166… confirmed, 1 outputs
tx 12d1fde897d592eb… confirmed, 2 outputs
tx d88156f6da67cd4b… confirmed, 2 outputs
block 359971 with 3 tx
tx acc874d220d00385… in mempool, 2 outputs
tx 99da0499f3e4926a… confirmed, 1 outputs
tx 5e63dedf75a03651… confirmed, 2 outputs
tx acc874d220d00385… confirmed, 2 outputs
tx e0efaf43fd176ab9… confirmed, 5 outputs
block 359972 with 4 tx

Notice the order: the block's transactions are published (and already report confirmed when you look them up) before the hashblock for that block. The one-output transaction that only ever appears as confirmed is each block's coinbase (empty on a proof-of-stake chain), and the two-output one right after it is the stake transaction; neither goes through the mempool. The transaction with 5 outputs sat in the mempool for a block before it was included.

This is exactly how a payment watcher works; Accept DIVI payments on your website builds one.

7Catch missed messages

ZeroMQ PUB sockets do not queue forever. If your subscriber is slow or disconnected, messages are dropped and you are not told, except by the sequence number skipping. The watcher above prints GAP on hashtx: expected #N, got #M when that happens (we saw no gap in our runs, so the line is not in the output above). When you see it:

1. Call getbestblockhash and compare it with the last block you handled.
2. Walk forward from your last block with getblock (its nextblockhash field) and process every transaction you skipped.
3. For mempool transactions you only missed the "seen" moment; they will still be published again when a block includes them.

The counters reset to 0 when divid restarts, so a sudden drop in the sequence number means the node restarted, not that you went backwards. Do the same catch-up walk.

If you subscribe to a raw topic, your subscriber sees exactly the same sequence as the matching hash topic, because rawtx and hashtx are sent from the same notification, each with its own counter.

If something goes wrong

Related: Watch new blocks live with the ZMQ viewer (the browser version), the node's config files, and installing divid. Still stuck? Ask the Divi chatbot, or see all tutorials.