Facilement récupérer les données des block chains en utilisant les messages P2P

Lola Rigaut-Luczak

  • Ingénieure en informatique

 

  • Intéressée par les crypto-monnaies depuis 2013

 

 

Les crypto-monnaies

P2P Network

Les noeuds communiquent entre elles en utilisant un réseau de communication dit pair à pair.

Les noeuds s'envoient des messages pour récupérer des blocks, des transactions, d'autres pairs...

It is not ... JSON RPC

Le serveur JSON RPC est utilisé lorsqu'un utilisateur souhaite parler à son noeud et uniquement son noeud.

{
    "jsonrpc": "1.0",
    "id": "curltest",
    "method": "getblock",
    "params": ["00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09"]
}

Tous les messages/commandes P2P

  • version
  • verack
  • addr
  • inv
  • getdata
  • notfound
  • getblocks
  • getheaders
  • tx
  • block
  • headers
  • getaddr
  • mempool
  • ping
  • pong
  • reject
  • filterload
  • filteradd
  • filterclear
  • merkleblock
  • sendheaders
  • feefilters
  • ...

Message structure

Field size Description Data type
4 magic bytes Unsigned Integer (32 bits)
12 command String (12 char)
4 length Unsigned Integer (32 bits)
4 checksum (double sha 256) Unsigned Integer (32 bits)
? payload Bytes

Exemple

C0C0C0C076657261636B000000000000000000005DF6E0E2
^ "verack" message

C0C0C0C0 <-- Dogecoin mainnet magic bytes
76657261636B000000000000 <-- "verack" command
00000000 <-- payload length 
5DF6E0E2 <-- checksum of payload

https://github.com/dogecoin/dogecoin/blob/master/src/chainparams.cpp#L147-L150

Ecrire notre noeud sur-mesure en python

Ce que notre noeud va faire :

- Se connecter à un autre noeud

- Faire une requête pour avoir les premiers blocks

- Deserialiser les blocks

https://github.com/rllola/dogecoin-p2p-workshop

Se connecter à un noeud

Se connecter à un noeud

import socket
import struct
import sys
import time

MSG_BLOCK_TYPE = 2
HOST = "51.158.74.86"
PORT = 18444
GENESIS_BLOCK_HASH = "3d2160a3b5dc4a9d62e7e66a295f70313ac808440ef7400d6c0772171ce973a5"

print("Attempt to connect to node")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))

# Prepare Version Message
version_message = prepareVersionMessage(HOST, PORT)
payload = preparePayload(version_message, b'version')
# Send Version Message
s.send(payload)

# Received Version Message
wait_for(s, b'version')

payload = preparePayload(b'', b'verack')
s.send(payload)
wait_for(s, b'verack')

Version message

Field size Description Data type
4 version Integer (32 bits)
8 services Unsigned Integer (64 bits)
8 timestamp
26 remote node address
26 local node address
8 nonce unsigned integer (64bits)
? user agent String
4 start height Integer (32 bits)
1 relay bool

Récupérer des blocks

message getblocks

Field size Description Data type
4 version Unsigned Integer (32 bits)
1+ hash count Compact Size
32+ hashes Char[32]
32 stop hash Char[32]

message inv/getdata

Field size Description Data type
1+ count Compact Size
36 x ? inventory (hash+type)

https://en.bitcoin.it/wiki/Protocol_documentation#Inventory_Vectors

message block

Field Size Description Data type
4 version Integer (32 bit)
32 previous block Char[32]
32 merkle root Char[32]
4 timestamp Unsigned Integer (32 bits)
4 bits Unsigned Integer (32 bits)
4 nonce Unsigned Integer (32 bits)
1+ transaction count Compact Size
? transactions Transaction 
blocks_count = 0
next_hash = GENESIS_BLOCK_HASH

print("Asking for a new batch")
get_blocks_message = prepareGetBlockMessage(next_hash)
payload = preparePayload(get_blocks_message, b'getblocks')
s.send(payload)

l = 0
while l <= 1:
    # if 1 count it means it just tell us about a new block
    inv_message = wait_for(s, b'inv')[24:]
    l, offset = getCompactSize(inv_message[0:9])

print("l value is {}".format(l))

if l == 501:
    # There are giving us an extra block (probably the latest block discover)
    l = 500

for i in range(l):
    (type, hash) = struct.unpack('I32s', inv_message[offset+(36*i):offset+(36*(i+1))])
    # do we really need that ?
    assert(type == MSG_BLOCK_TYPE)
    if i == l-1:
        next_hash = hash[::-1].hex()

# We can send the message now
payload = preparePayload(inv_message, b'getdata')
s.send(payload)
count = l
log = ''

# Wait for 500 responses !
while count > 0:

    # Received Inv Message
    data = wait_for(s, b'block')

    if data == 0:
        print("ERROR : missing block")
        sys.exit()

    # verify data
    magic, command, m_length, chcksm = struct.unpack('4s12sI4s', data[0:24])
    if b'block' not in command:
        print("ERROR : wrong command")
        sys.exit()

    if len(data[24:]) != m_length:
        print("ERROR : Wrong length")
        sys.exit()


    block_message = data[24:]
    count -= 1
    hash = checksum(block_message[0:80])[::-1].hex()
    print("Count : {} - Block hash {}".format(blocks_count + 500 - count, hash))
    log += "----- {} -----\n".format(hash)

Récupérer tous les blocks

Désérialiser les blocks

    print("Count : {} - Block hash {}".format(blocks_count + 500 - count, hash))
    log += "----- {} -----\n".format(hash)


    txs = unpackBlock(block_message, False)


    for tx in txs:
        log += "{}\n".format(tx.hex())


print(log)

Pour aller plus loin :

- Désérialiser les blocks AuxPoW (auxiliary proof of work)

- Collecter les données des noeuds (IPs, User Agent, localization, ...)

- Les transactions non confirmé

- Un système d'alerte

Conclusion

Venez me voir pour une démo

Facilement récupérer les données des block chains en utilisant les messages P2P

By Lola Rigaut-Luczak

Made with Slides.com

Facilement récupérer les données des block chains en utilisant les messages P2P

  • 90

More from Lola Rigaut-Luczak