Image transfer over the ethernet

You can download the source codes here(https://github.com/raspberry-pi-maker/OpenCV)

In this post, I'll show you how to display an image you've created or read from a file using OpenCV, on another computer. As a protocol for transmitting image data to another computer via Ethernet, UDP or TCP can be selected.
We will use UDP here. The reason for this is as follows.

• In the same subnet, even if UDP communication is used, packet loss or duplication does not occur. 
• It's faster and lighter than TCP, so it's even better if you need to transfer multiple images per second, such as video.

prerequisite

There are several protocols for transferring images, but in this post, I will send uncompressed image data directly in the form of numpy arrays.

Numpy over network

The most important part of numpy data transfer and recovery is byte sorting of numpy data. The sender sorts bytes using the numpy tobytes () function. The receiver then creates a one-dimensional numpy array using numpy's asarray or frombuffer functions.
The 1D array is then converted back into a 3D array using the height, width, and channel values ​​of the original image.

Sending large data over UDP

Converting an image to a numpy matrix can take up a fairly large number of bytes  because it is an uncompressed format. Therefore, it may be too large to transmit at one time. So there is a need for a technique of splitting byte-aligned data into a size that can be transmitted using the numpy function tobytes described above. In addition, it is good to create a safety algorithm that can prevent data loss and reversal in case of breaking the data into chunks.

chunks = [data[i:i+CHUNK_SIZE] for i in range(0, len(data), CHUNK_SIZE)]

The code above divides data by CHUNK_SIZE unit. The divided data is kept in the chunks list.

for i, chunk in enumerate(chunks):
        chunk = struct.pack("<I", 1) + struct.pack("<I", i) + struct.pack("<I", chunk_len) + chunk

Then send the divided data one by one. Note the addition of a header at the beginning of the data. This header carries information such as the total number of packets, the current packet number, and whether or not the last packet is included.

Sending Part Code

import numpy as np
import cv2
import socket, struct,time

CHUNK_SIZE = 8192 * 6
SERVER = ("localhost",4321) # Modify localhost to receiving part IP address

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF) 
print('Snd buf size:%d'%(bufsize))

img = cv2.imread('star.jpg')
H,W,C = img.shape
print(img.shape)

data = img.tobytes()
print('byte len:%d'%(len(data)))
total = 0
chunks = [data[i:i+CHUNK_SIZE] for i in range(0, len(data), CHUNK_SIZE)]
chunk_len = len(chunks)
s = time.time()
for i, chunk in enumerate(chunks):
    if(i == chunk_len - 1): #last
        chunk = struct.pack("<I", 1) + struct.pack("<I", i) + struct.pack("<I", chunk_len) + chunk    # len(data) + 12 bytes , "<I" : < means little-endian, I means 4 bytes integer
    else:    
        chunk = struct.pack("<I", 0) + struct.pack("<I", i) + struct.pack("<I", chunk_len) + chunk    # len(data) + 12 bytes , "<I" : < means little-endian, I means 4 bytes integer
    sock.sendto(chunk, SERVER)
    total += len(chunk)
    print('Total sent:%d'%(total))
    time.sleep(0.0001)
e = time.time()
print('time:%f', e - s)

<net_snd.py>

Receiving Part Code

You should set the H, W, C variables in the code to the receiving image size, channel information.

import numpy as np
import cv2
import socket, struct,time
 
CHUNK_SIZE = 8192 + 32
CHUNK_SIZE = 52000
SERVER = ("0.0.0.0",4321)
H = 570
W = 720
C = 3
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(SERVER)
bufsize = sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF) 
print('Rcv buf size:%d'%(bufsize))
if CHUNK_SIZE < bufsize:
    CHUNK_SIZE = bufsize

sock.settimeout(5)
total = 0
buf = []
packet_cnt = 0
def reset():
    global buf, total, packet_cnt
    total = 0
    buf = []
    packet_cnt = 0

while(True):
    try:
        data, addr = sock.recvfrom(CHUNK_SIZE)
        total += len(data)
        key = int.from_bytes(data[:4],byteorder="little")
        seq = int.from_bytes(data[4:8],byteorder="little")
        cnt = int.from_bytes(data[8:12],byteorder="little")
        buf += data[12:]
        packet_cnt += 1
        print('Total rcv:%d Key:%d, seq:%d total chunk:%d'%(total, key, seq, cnt))
        if key == 1:    #last
            if(packet_cnt != cnt):
                print('Total rcv cnt:%d total chunk:%d'%(packet_cnt, cnt))
                reset()
                continue

            img = np.asarray(buf, dtype=np.uint8)
            b = img.reshape(H,W,C)
            cv2.imshow('H', b)
            cv2.waitKey(0)
            cv2.destroyAllWindows()
            reset()

    except KeyboardInterrupt:
        break
    except socket.timeout:
        reset()
        continue    

<net_rcv.py>

Test

As the code shows, the sender opens the file star.jpg and sends it. The receiver collects the chunks, receives the last chunk, converts it to an image, and displays it. Let's test if it works.

It works well. Although the tests were conducted on one PC, the same results would be apparent if tested on two computers. In case of packet loss, test by adjusting CHUNK_SIZE or sleep time of sender.

Wrapping up

In the above example, there is information to check for errors in the header of the data sent using UDP communication. However, the above example does not use this information. If you want more accurate transmission, use this information.
This method is also useful if you need to transfer a numpy matrix over a network to another computer. With a little bit of application, you can easily transfer videos too.

You can download the source code at https://github.com/raspberry-pi-maker/OpenCV .