C#开源实现MJPEG流传输

OStack程序员社区-中国程序员成长平台 › 门户 › 编程› Node.js›Node.js教程

原作者: [db:作者] 来自: [db:来源] 收藏邀请

本文为 Dennis Gao 原创技术文章，发表于博客园博客，未经作者本人允许禁止任何形式的转载。

许久以前写了篇文章《基于.NET打造IP智能网络视频监控系统》，记录和介绍了自己几年来积累和演练的一个系统。发现几个月过去了，没有任何进展。

目前已经实现了 UDP+RTP 方式在不同物理机之间的媒体流传输。当然，由于没有基于 .NET 的媒体流压缩实现，所以直接传输的裸图 Bitmap。不过要求不高，帧率低一些，机器性能强一些，看着也很流畅。

能在桌面客户端上看到视频图像的功能已经完成了。下面需要考虑，如何通过浏览器来查看视频。

在不考虑使用 Flash、ActiveX 的条件下，貌似只能选择 MJPEG 方式。目前还没有研究在 HTML5 下视频是如何处理的，以后有时间可以探索。

什么是 MJPEG？

看这里：

当然，我主要关注 MJPEG over HTTP 这段。

M-JPEG over HTTP
HTTP streaming separates each image into individual HTTP replies on a specified marker. RTP streaming creates packets of a sequence of JPEG images that can be received by clients such as QuickTime or VLC.
In response to a GET request for a MJPEG file or stream, the server streams the sequence of JPEG frames over HTTP. A special mime-type content type multipart/x-mixed-replace;boundary=<boundary-name> informs the client to expect several parts (frames) as an answer delimited by <boundary-name>. This boundary name is expressly disclosed within the MIME-type declaration itself. The TCP connection is not closed as long as the client wants to receive new frames and the server wants to provide new frames. Two basic implementations of a M-JPEG streaming server are cambozola and MJPG-Streamer. The more robust ffmpeg-server also provides M-JPEG streaming support.

也就是说，建立 HTTP 连接后，服务端在 Response 消息中先发一个数据头 Header 告诉客户端，我后面的都是 JPEG 图片。图片之间使用 boundary-name 来区分，每个图片前都有自己的数据头来描述图片数据长度。

MJPEG数据头定义

 1     /// <summary>
 2     /// 流头部
 3     /// </summary>
 4     public string StreamHeader
 5     {
 6       get
 7       {
 8         return "HTTP/1.1 200 OK" +
 9                "\r\n" +
10                "Content-Type: multipart/x-mixed-replace; boundary=" + this.Boundary +
11                "\r\n";
12       }
13     }

 1     /// <summary>
 2     /// 图片头部
 3     /// </summary>
 4     public string PayloadHeader
 5     {
 6       get
 7       {
 8         return "\r\n" +
 9                this.Boundary +
10                "\r\n" +
11                "Content-Type: image/jpeg" +
12                "\r\n" +
13                "Content-Length: " + _contentLengthString +
14                "\r\n\r\n";
15       }
16     }

这里的 Boundary 可以是任意字符串，只要你觉得唯一并能区分即可，比如我可以设置为“--dennisgao”。

服务器端实现

Http 服务器其实就是个支持 Tcp 连接的服务器。

1 private AsyncTcpServer _server;
2 
3 _server = new AsyncTcpServer(Port);
4 _server.Encoding = Encoding.ASCII;

 1     public void Start()
 2     {
 3       _server.Start(10);
 4       _server.ClientConnected += new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
 5       _server.ClientDisconnected += new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
 6     }
 7 
 8     public void Stop()
 9     {
10       _server.Stop();
11       _server.ClientConnected -= new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
12       _server.ClientDisconnected -= new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
13     }
14 
15     private void OnClientConnected(object sender, TcpClientConnectedEventArgs e)
16     {
17       _clients.AddOrUpdate(e.TcpClient.Client.RemoteEndPoint.ToString(), e.TcpClient, (n, o) => { return e.TcpClient; });
18     }
19 
20     private void OnClientDisconnected(object sender, TcpClientDisconnectedEventArgs e)
21     {
22       TcpClient clientToBeThrowAway;
23       _clients.TryRemove(e.TcpClient.Client.RemoteEndPoint.ToString(), out clientToBeThrowAway);
24     }

这里可以参考两篇文章中的实现。

发送图片数据

首先要保证，对一个HTTP连接只能发一次流头，因为后面是接连不断的图片数据。当然，发点别的数据客户端也不会解码。

 1     private void WriteStreamHeader()
 2     {
 3       if (_clients.Count > 0)
 4       {
 5         foreach (var item in _clients)
 6         {
 7           Logger.Debug(string.Format(CultureInfo.InvariantCulture,
 8             "Writing stream header, {0}, {1}{2}", item.Key, Environment.NewLine, StreamHeader));
 9 
10           _server.SyncSend(item.Value, StreamHeader);
11 
12           TcpClient clientToBeThrowAway;
13           _clients.TryRemove(item.Key, out clientToBeThrowAway);
14         }
15       }
16     }

发送图片数据时，要保证图片的前面是图片头和长度信息，数据尾部要有换行符。

 1     private void WritePayload(byte[] payload)
 2     {
 3       string payloadHeader = this.PayloadHeader.Replace(_contentLengthString, payload.Length.ToString());
 4       string payloadTail = "\r\n";
 5 
 6       Logger.Debug(string.Format(CultureInfo.InvariantCulture,
 7         "Writing payload header, {0}{1}", Environment.NewLine, payloadHeader));
 8 
 9       byte[] payloadHeaderBytes = _server.Encoding.GetBytes(payloadHeader);
10       byte[] payloadTailBytes = _server.Encoding.GetBytes(payloadTail);
11       byte[] packet = new byte[payloadHeaderBytes.Length + payload.Length + payloadTail.Length];
12       Buffer.BlockCopy(payloadHeaderBytes, 0, packet, 0, payloadHeaderBytes.Length);
13       Buffer.BlockCopy(payload, 0, packet, payloadHeaderBytes.Length, payload.Length);
14       Buffer.BlockCopy(payloadTailBytes, 0, packet, payloadHeaderBytes.Length + payload.Length, payloadTailBytes.Length);
15 
16       _server.SendToAll(packet);
17     }

结果演示

在可以成功发送流信息和图片信息后，就可以在浏览器上查看视频了。当然，我用的 Google Chrome 。IE10 好奇葩，它会把流当成文件不停的下载，搞不懂。

远程访问

局域网内的无线设备，只要浏览器支持 MJPEG ，均可以查看视频。我测试了 iPad 上的 Safari 是可以的，但 Chrome 却直接解析成乱码。

当然，如果在路由器上配置转发规则，就可以在外网访问了。

完整代码

  1   public class MJpegStreamingServer
  2   {
  3     private static string _contentLengthString = "__PayloadHeaderContentLength__";
  4     private AsyncTcpServer _server;
  5     private ConcurrentDictionary<string, TcpClient> _clients;
  6 
  7     public MJpegStreamingServer(int listenPort)
  8       : this(listenPort, "--dennisgao")
  9     {
 10     }
 11 
 12     public MJpegStreamingServer(int listenPort, string boundary)
 13     {
 14       Port = listenPort;
 15       Boundary = boundary;
 16 
 17       _server = new AsyncTcpServer(Port);
 18       _server.Encoding = Encoding.ASCII;
 19       _clients = new ConcurrentDictionary<string, TcpClient>();
 20     }
 21 
 22     /// <summary>
 23     /// 监听的端口
 24     /// </summary>
 25     public int Port { get; private set; }
 26 
 27     /// <summary>
 28     /// 分隔符
 29     /// </summary>
 30     public string Boundary { get; private set; }
 31 
 32     /// <summary>
 33     /// 流头部
 34     /// </summary>
 35     public string StreamHeader
 36     {
 37       get
 38       {
 39         return "HTTP/1.1 200 OK" +
 40                "\r\n" +
 41                "Content-Type: multipart/x-mixed-replace; boundary=" + this.Boundary +
 42                "\r\n";
 43       }
 44     }
 45 
 46     /// <summary>
 47     /// 图片头部
 48     /// </summary>
 49     public string PayloadHeader
 50     {
 51       get
 52       {
 53         return "\r\n" +
 54                this.Boundary +
 55                "\r\n" +
 56                "Content-Type: image/jpeg" +
 57                "\r\n" +
 58                "Content-Length: " + _contentLengthString +
 59                "\r\n\r\n";
 60       }
 61     }
 62 
 63     public void Start()
 64     {
 65       _server.Start(10);
 66       _server.ClientConnected += new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
 67       _server.ClientDisconnected += new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
 68     }
 69 
 70     public void Stop()
 71     {
 72       _server.Stop();
 73       _server.ClientConnected -= new EventHandler<TcpClientConnectedEventArgs>(OnClientConnected);
 74       _server.ClientDisconnected -= new EventHandler<TcpClientDisconnectedEventArgs>(OnClientDisconnected);
 75     }
 76 
 77     private void OnClientConnected(object sender, TcpClientConnectedEventArgs e)
 78     {
 79       _clients.AddOrUpdate(e.TcpClient.Client.RemoteEndPoint.ToString(), e.TcpClient, (n, o) => { return e.TcpClient; });
 80     }
 81 
 82     private void OnClientDisconnected(object sender, TcpClientDisconnectedEventArgs e)
 83     {
 84       TcpClient clientToBeThrowAway;
 85       _clients.TryRemove(e.TcpClient.Client.RemoteEndPoint.ToString(), out clientToBeThrowAway);
 86     }
 87 
 88     public void Write(Image image)
 89     {
 90       if (_server.IsRunning)
 91       {
 92         byte[] payload = BytesOf(image);
 93 
 94         WriteStreamHeader();
 95         WritePayload(payload);
 96       }
 97     }
 98 
 99     private void WriteStreamHeader()
100     {
101       if (_clients.Count > 0)
102       {
103         foreach (var item in _clients)
104         {
105           Logger.Debug(string.Format(CultureInfo.InvariantCulture,
106             "Writing stream header, {0}, {1}{2}", item.Key, Environment.NewLine, StreamHeader));
107 
108           _server.SyncSend(item.Value, StreamHeader);
109 
110           TcpClient clientToBeThrowAway;
111           _clients.TryRemove(item.Key, out clientToBeThrowAway);
112         }
113       }
114     }
115 
116     private void WritePayload(byte[] payload)
117     {
118       string payloadHeader = this.PayloadHeader.Replace(_contentLengthString, payload.Length.ToString());
119       string payloadTail = "\r\n";
120 
121       Logger.Debug(string.Format(CultureInfo.InvariantCulture,
122         "Writing payload header, {0}{1}", Environment.NewLine, payloadHeader));
123 
124       byte[] payloadHeaderBytes = _server.Encoding.GetBytes(payloadHeader);
125       byte[] payloadTailBytes = _server.Encoding.GetBytes(payloadTail);
126       byte[] packet = new byte[payloadHeaderBytes.Length + payload.Length + payloadTail.Length];
127       Buffer.BlockCopy(payloadHeaderBytes, 0, packet, 0, payloadHeaderBytes.Length);
128       Buffer.BlockCopy(payload, 0, packet, payloadHeaderBytes.Length, payload.Length);
129       Buffer.BlockCopy(payloadTailBytes, 0, packet, payloadHeaderBytes.Length + payload.Length, payloadTailBytes.Length);
130 
131       _server.SendToAll(packet);
132     }
133 
134     private byte[] BytesOf(Image image)
135     {
136       MemoryStream ms = new MemoryStream();
137       image.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
138 
139       byte[] payload = ms.ToArray();
140 
141       return payload;
142     }
143   }