GStreamer Quick Start
note of https://gstreamer.freedesktop.org/documentation/tutorials/basic/hello-world.html
GStreamer is a framework for constructing graphs of various filters (termed elements here) that will handle streaming media.
Any discrete (packetizable) media type is supported, with provisions for automatically determining source type.
Formatting/framing information is provided with a powerful negotiation framework.
Plugins are heavily used to provide for all elements, allowing one to construct plugins outside of the GST library, even released binary-only if license require (please don't).
GStreamer covers a wide range of use cases including: playback, recording, editing, serving streams, voice over ip and video calls.
GStreamer Plugin
GstRtpBin
RTP bin combines the functions of GstRtpSession, GstRtpSsrcDemux, GstRtpJitterBuffer and GstRtpPtDemux in one element. It allows for multiple RTP sessions that will be synchronized together using RTCP SR packets.
GstRtpBin is configured with a number of request pads that define the functionality that is activated, similar to the GstRtpSession element.
To use GstRtpBin as an RTP receiver, request a recv_rtpsink%u pad. The session number must be specified in the pad name. Data received on the recv_rtpsink%u pad will be processed in the GstRtpSession manager and after being validated forwarded on GstRtpSsrcDemux element. Each RTP stream is demuxed based on the SSRC and send to a GstRtpJitterBuffer. After the packets are released from the jitterbuffer, they will be forwarded to a GstRtpPtDemux element. The GstRtpPtDemux element will demux the packets based on the payload type and will create a unique pad recv_rtpsrc%u%u%u on rtpbin with the session number, SSRC and payload type respectively as the pad name.
To also use GstRtpBin as an RTCP receiver, request a recv_rtcpsink%u pad. The session number must be specified in the pad name.
If you want the session manager to generate and send RTCP packets, request the send_rtcpsrc%u pad with the session number in the pad name. Packet pushed on this pad contain SR/RR RTCP reports that should be sent to all participants in the session.
To use GstRtpBin as a sender, request a send_rtpsink%u pad, which will automatically create a send_rtpsrc%u pad. If the session number is not provided, the pad from the lowest available session will be returned. The session manager will modify the SSRC in the RTP packets to its own SSRC and wil forward the packets on the send_rtpsrc%u pad after updating its internal state.
The session manager needs the clock-rate of the payload types it is handling and will signal the “request-pt-map” signal when it needs such a mapping. One can clear the cached values with the “clear-pt-map” signal.
Access to the internal statistics of rtpbin is provided with the get-internal-session property. This action signal gives access to the RTPSession object which further provides action signals to retrieve the internal source and other sources.
GstRtpBin also has signals (“request-rtp-encoder”, “request-rtp-decoder”, “request-rtcp-encoder” and “request-rtp-decoder”) to dynamically request for RTP and RTCP encoders and decoders in order to support SRTP. The encoders must provide the pads rtpsink%u and rtpsrc%u for RTP and rtcpsink%u and rtcpsrc%u for RTCP. The session number will be used in the pad name. The decoders must provide rtp_sink and rtp_src for RTP and rtcp_sink and rtcp_src for RTCP. The decoders will be placed before the GstRtpSession element, thus they must support SSRC demuxing internally.
GstRtpBin has signals (“request-aux-sender” and “request-aux-receiver” to dynamically request an element that can be used to create or merge additional RTP streams. AUX elements are needed to implement FEC or retransmission (such as RFC 4588). An AUX sender must have one sink%u pad that matches the sessionid in the signal and it should have 1 or more src%u pads. For each src_%\u pad, a session will be made (if needed) and the pad will be linked to the session send_rtp_sink pad. Each session will then expose its source pad as send_rtpsrc%u on GstRtpBin. An AUX receiver has 1 src%u pad that much match the sessionid in the signal and 1 or more sink%u pads. A session will be made for each sink_%u pad when the corresponding recv_rtpsink%u pad is requested on GstRtpBin.
RTP Sender
# A simple RTP server with retransmission
# sends the output of videotestsrc as h264 encoded RTP on port 5000, RTCP is sent on
# port 5001. The destination is 127.0.0.1.
# the video receiver RTCP reports are received on port 5005
#
# .-------. .-------. .-------. .----------. .-------.
# |videots| |h264enc| |h264pay| | rtpbin | |udpsink| RTP
# | src->sink src->sink src->send_rtp send_rtp->sink | port=5000
# '-------' '-------' '-------' | | '-------'
# | |
# | | .-------.
# | | |udpsink| RTCP
# | send_rtcp->sink | port=5001
# .-------. | | '-------' sync=false
# RTCP |udpsrc | | | async=false
# port=5005 | src->recv_rtcp |
# '-------' '----------'
#
# ideally we should transport the properties on the RTP udpsink pads to the
# receiver in order to transmit the SPS and PPS earlier.
# change this to send the RTP data and RTCP to another host
DEST=127.0.0.1
# tuning parameters to make the sender send the streams out of sync. Can be used
# ot test the client RTCP synchronisation.
#VOFFSET=900000000
VOFFSET=0
AOFFSET=0
# H264 encode from the source
VELEM="videotestsrc is-live=1 horizontal-speed=1"
VCAPS="video/x-raw,width=352,height=288,framerate=15/1"
VSOURCE="$VELEM ! $VCAPS"
VENC="x264enc tune=zerolatency bitrate=300 ! rtph264pay config-interval=2"
VRTPSINK="udpsink port=5000 host=$DEST ts-offset=$VOFFSET name=vrtpsink"
VRTCPSINK="udpsink port=5001 host=$DEST sync=false async=false name=vrtcpsink"
VRTCPSRC="udpsrc port=5005 name=vrtpsrc"
gst-launch-1.0 -v rtpbin name=rtpbin rtp-profile=avpf \
$VSOURCE ! $VENC ! rtprtxqueue ! rtpbin.send_rtp_sink_0 \
rtpbin.send_rtp_src_0 ! identity drop-probability=0.1 ! $VRTPSINK \
rtpbin.send_rtcp_src_0 ! $VRTCPSINK \
$VRTCPSRC ! rtpbin.recv_rtcp_sink_0
RTP Receiver
# A simple RTP receiver with retransmission
#
# receives H264 encoded RTP video on port 5000, RTCP is received on port 5001.
# the receiver RTCP reports are sent to port 5005
#
# .-------. .----------. .---------. .-------. .-----------.
# RTP |udpsrc | | rtpbin | |h264depay| |h264dec| |xvimagesink|
# port=5000 | src->recv_rtp recv_rtp->sink src->sink src->sink |
# '-------' | | '---------' '-------' '-----------'
# | |
# | | .-------.
# | | |udpsink| RTCP
# | send_rtcp->sink | port=5005
# .-------. | | '-------' sync=false
# RTCP |udpsrc | | | async=false
# port=5001 | src->recv_rtcp |
# '-------' '----------'
# the caps of the sender RTP stream. This is usually negotiated out of band with
# SDP or RTSP. normally these caps will also include SPS and PPS but we don't
# have a mechanism to get this from the sender with a -launch line.
VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264"
VIDEO_DEC="rtph264depay ! avdec_h264"
VIDEO_SINK="videoconvert ! autovideosink"
# the destination machine to send RTCP to. This is the address of the sender and
# is used to send back the RTCP reports of this receiver. If the data is sent
# from another machine, change this address.
DEST=127.0.0.1
LATENCY=200
gst-launch-1.0 -v rtpbin name=rtpbin rtp-profile=avpf latency=$LATENCY do-retransmission=1 \
udpsrc caps=$VIDEO_CAPS port=5000 ! rtpbin.recv_rtp_sink_0 \
rtpbin. ! $VIDEO_DEC ! $VIDEO_SINK \
udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 \
rtpbin.send_rtcp_src_0 ! udpsink port=5005 host=$DEST sync=false async=false