how to stream realtime data in Python socket

Question

I am working on a realtime application. The application takes realtime audio, send to a recogniser, and get results back. I am using a socket for both sending and receiving. For debugging, I use a wave file. So I read audio, I would like to send the audio as if it is being played in realtime. I use time.sleep trying to synchronise the send, it caused the send to be slower than realtime. Should I use two threads, one for send and one for receive?

while True:
    readable, writable, exceptional = select.select([s], [s], [])
    if readable:
        try:
            data = s.recv(1024)
        except:
            break
        if data == "":
            break
        else:
            for line in data.split("\n"):
                if not line: continue
                t, w = line.split()
                if w == "<EOL>":
                    w = ""
                addResult(t, w)
    if writable:
        now = time.time()
        if (now - start_time) / 0.01 >= frame_count - 5:
            frame_count += 1
            data = wav.readframes(80)
            if data is None:
                s.shutdown(s.SHUT_WR)
                continue
            s.send(struct.pack('I', len(data)) + data)
        else:
            time.sleep(0.005)
    if exceptional:
        print "Hangup"
        break

your code is buggy. with recv it is not given, that only whole lines are read, therefore your data-string could end between a "<EO" and the "L>" is in the next recv. — Daniel
– Daniel, Commented Apr 27, 2014 at 8:27
you have to adjust your sleeping time dynamically, giving now - start_time and the number of frames sent. — Daniel
– Daniel, Commented Apr 27, 2014 at 8:30
Hi Daniel, the reason I don't let it sleep to a dynamic time is that there is a read as well, I don't want this to block the read. So if no send, the code can still go to next read. — Yifan Zhang
– Yifan Zhang, Commented Apr 27, 2014 at 8:53

Steffen Ullrich · Accepted Answer · 2014-04-27 08:46:54Z

There are two approaches to sending kind of real-time data:

Sender sends data at a specific rate: in this case the sender sends the data for a specific time frame, then the next etc. Because each packet can have a different delay in transmission a some kind of time stamp will be added so that the receiver known when to play this packet. TCP is not a good idea in this case, because it will retransmit on packet loss and the resent packet will arrive too late. So in this cases usually UDP is used and the codec need to be able to deal with packet loss. A typical example for this kind of protocol is RTP, which is used in VoIP and other cases.
Receiver receives the data at a specific rate: This is only for soft real-time, e.g. where a delay of several seconds or more is acceptable. In this case the sender just sends the data with TCP and each "frame" inside the stream contains some time stamp so that the receiver knows when to play it. The receiver will just read the packets as fast or slow as needed. If the packets arrive slower than needed it will need to pause playing and wait for more data (as seen with youtube etc), if they arrive faster than needed it will just read them as slow as needed which will automatically cause the sender to slow down the sending (inherent behavior of TCP).

The original approach of your program was, that the sender transmits the data at a specific rate. To continue with this way you should:

use UDP instead of TCP so you get no delays caused by retransmission if packets get lost
use a codec which can deal with packet loss, e.g. G.711 or similar
add a time stamp to each data packet so the sender knows when to play it
add accurate timing: using sleep does not account by itself for the time you needed before sending the packet, so instead you should store the start time and calculate the times, when each of the packets need to be sent (e.g. next_time = last_time+time_span_in_packet). Then sleep only for next_time - current_time.

Collectives™ on Stack Overflow

how to stream realtime data in Python socket

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related