{"id":745,"date":"2022-05-29T09:25:23","date_gmt":"2022-05-29T01:25:23","guid":{"rendered":"https:\/\/www.fanyamin.com\/wordpress\/?p=745"},"modified":"2022-05-29T09:25:39","modified_gmt":"2022-05-29T01:25:39","slug":"video-pipeline","status":"publish","type":"post","link":"https:\/\/www.fanyamin.com\/wordpress\/?p=745","title":{"rendered":"WebRTC Video Pipeline"},"content":{"rendered":"<h1>\u6982\u8ff0<\/h1>\n<p>\u89c6\u9891\u6d41\u6c34\u7ebf\u662f\u6307\u89c6\u9891\u4ece\u53d1\u9001\u7aef\u6444\u50cf\u5934\u6355\u6349\u89c6\u9891\u56fe\u50cf\uff0c\u5c06\u89c6\u9891\u6d41\u8fdb\u884c\u8f6c\u6362\uff0c\u7f16\u7801\uff0c\u6253\u5305\uff0c\u53d1\u9001\uff0c\u76f4\u5230\u63a5\u6536\u7aef\u8fdb\u884c\u89e3\u5305\uff0c\u7f13\u5b58\uff0c\u89e3\u7801\u518d\u5230\u5c06\u89c6\u9891\u56fe\u50cf\u6e32\u67d3\u5230\u753b\u5e03\u4e0a\u6216\u8005\u89c6\u9891\u5143\u7d20\u4e2d<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.fanyamin.com\/wordpress\/wp-content\/uploads\/2022\/05\/image-1653745005158.png\" alt=\"\" \/><\/p>\n<pre><code> # linux send h264 rtp stream:\n   gst-launch-1.0 -v ximagesrc \\\n   ! video\/x-raw,framerate=20\/1 \\\n   ! videoscale ! videoconvert \\\n   ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast \\\n   ! rtph264pay \\\n   ! udpsink host=127.0.0.1 port=5000\n\n   # Macos send h264 rtp stream:\n   gst-launch-1.0 -v avfvideosrc capture-screen=true \\\n   ! video\/x-raw,framerate=20\/1 \\\n   ! videoscale ! videoconvert \\\n   ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast \\\n   ! rtph264pay \\\n   ! udpsink host=127.0.0.1 port=5000\n\n   # receive h264 rtp stream:\n   gst-launch-1.0 -v udpsrc port=5000 caps = &quot;application\/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96&quot; \\\n   ! rtph264depay \\\n   ! decodebin \\\n   ! videoconvert \\\n   ! autovideosink<\/code><\/pre>\n<p>\u5927\u4f53\u6b65\u9aa4\u5982\u4e0b<\/p>\n<ol>\n<li>capture \u6355\u6349\u5a92\u4f53\u6d41<\/li>\n<\/ol>\n<p>getUserMedia<br \/>\ngetDisplayMedia<\/p>\n<ol start=\"2\">\n<li>convert \u683c\u5f0f\u8f6c\u6362<\/li>\n<\/ol>\n<p>RGB --&gt; YUV<\/p>\n<ol start=\"3\">\n<li>Encode \u7f16\u7801<\/li>\n<\/ol>\n<p>H.264 Encode<\/p>\n<ol start=\"4\">\n<li>Packetize \u6253\u5305<\/li>\n<\/ol>\n<p>H264 NAL pack into RTP packet<\/p>\n<ol start=\"5\">\n<li>Transmit \u4f20\u8f93<\/li>\n<\/ol>\n<p>Transmit RTP packets from sender to receiver<\/p>\n<ol start=\"6\">\n<li>Depacketize \u89e3\u5305<\/li>\n<\/ol>\n<p>Depacketize RTP packets to H264 NAL<\/p>\n<ol start=\"7\">\n<li>Jitter Buffer \u7f13\u5b58<\/li>\n<\/ol>\n<p>Cache H264 NAL if there is no frame can be decoded<\/p>\n<ol start=\"9\">\n<li>Decode \u89e3\u7801<\/li>\n<\/ol>\n<p>Decode H264 NAL to VAL to image<\/p>\n<ol start=\"10\">\n<li>Render \u6e32\u67d3\u6216\u56de\u653e<\/li>\n<\/ol>\n<p>Render the image to Canvas, playback the images as video<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6982\u8ff0 \u89c6\u9891\u6d41\u6c34\u7ebf\u662f\u6307\u89c6\u9891\u4ece\u53d1\u9001\u7aef\u6444\u50cf\u5934\u6355\u6349\u89c6\u9891\u56fe\u50cf\uff0c\u5c06\u89c6\u9891\u6d41\u8fdb\u884c\u8f6c\u6362\uff0c\u7f16\u7801\uff0c\u6253\u5305\uff0c\u53d1\u9001\uff0c\u76f4\u5230\u63a5\u6536\u7aef\u8fdb\u884c\u89e3\u5305\uff0c\u7f13\u5b58\uff0c\u89e3\u7801\u518d\u5230\u5c06\u89c6\u9891\u56fe\u50cf\u6e32\u67d3\u5230\u753b\u5e03\u4e0a\u6216\u8005\u89c6\u9891\u5143\u7d20\u4e2d # linux send h264 rtp stream: gst-launch-1.0 -v ximagesrc \\ ! video\/x-raw,framerate=20\/1 \\ ! videoscale ! videoconvert \\ ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast \\ ! rtph264pay \\ ! udpsink host=127.0.0.1 port=5000 # Macos send h264 rtp stream: gst-launch-1.0 -v avfvideosrc capture-screen=true \\ ! video\/x-raw,framerate=20\/1 \\ ! videoscale ! videoconvert \\ ! x264enc tune=zerolatency [&hellip;] <a class=\"read-more\" href=\"https:\/\/www.fanyamin.com\/wordpress\/?p=745\" title=\"Permanent Link to: WebRTC Video Pipeline\">&rarr;Read&nbsp;more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-745","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/745"}],"collection":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=745"}],"version-history":[{"count":5,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/745\/revisions"}],"predecessor-version":[{"id":751,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/745\/revisions\/751"}],"wp:attachment":[{"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=745"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=745"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fanyamin.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=745"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}