c# - response size limitaiton of a WCF web serivces

Question

I am hosting a WCF service in IIS using basicHttpBinding. The WCF web services queries back end SQL Server 2008 by using ADO.Net and return a DataTable to client side of WCF service.

I find when the returned DataTable is big, there will be exception said http connection is closed by IIS. Any ideas what is wrong and how to set bigger response size? Another idea is whether there is any hard limitation for the serialization size of an object (I thought maybe the DataTable instance is too big to serialize?)

The error information returned by IIS is:

Exception message:

An error occurred while receiving the HTTP response to http://labmachine1/service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.

{"The underlying connection was closed: An unexpected error occurred on a receive."}

Here is my whole source code for server side, for the server I host in web.config, no change for default values. Since I host in IIS, I am using basicHttpBinding.

public class StudentManagement : IStudentManagement
{
    public DataTable Poll(int Id)
    {
        return MakeParentTable();
    }

    private DataTable MakeParentTable()
    {
        // Create a new DataTable.
        System.Data.DataTable table = new DataTable("ParentTable");
        // Declare variables for DataColumn and DataRow objects.
        DataColumn column;
        DataRow row;

        // Create new DataColumn, set DataType, 
        // ColumnName and add to DataTable.    
        column = new DataColumn();
        column.DataType = System.Type.GetType("System.Int32");
        column.ColumnName = "id";
        column.ReadOnly = true;
        column.Unique = true;
        // Add the Column to the DataColumnCollection.
        table.Columns.Add(column);

        // Create second column.
        column = new DataColumn();
        column.DataType = System.Type.GetType("System.String");
        column.ColumnName = "ParentItem";
        column.AutoIncrement = false;
        column.Caption = "ParentItem";
        column.ReadOnly = false;
        column.Unique = false;
        // Add the column to the table.
        table.Columns.Add(column);

        // Make the ID column the primary key column.
        DataColumn[] PrimaryKeyColumns = new DataColumn[1];
        PrimaryKeyColumns[0] = table.Columns["id"];
        table.PrimaryKey = PrimaryKeyColumns;

        // Create three new DataRow objects and add 
        // them to the DataTable
        for (int i = 0; i <= 1000000; i++)
        {
            row = table.NewRow();
            row["id"] = i;
            row["ParentItem"] = "ParentItem " + i;
            table.Rows.Add(row);
        }

        return table;
    }
}

Client side code:

static void Main(string[] args)
{
    StudentIdentifier identifier = new StudentIdentifier();
    identifier.Id = 100;
    StudentManagementClient client = new StudentManagementClient();

    DataTable student = client.Poll(identifier);

    Console.WriteLine(student.Rows.Count);
}

Client side web.config:

<binding name="BasicHttpBinding_IStudentManagement" closeTimeout="00:01:00"
    openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
    allowCookies="false" bypassProxyOnLocal="false" 
    hostNameComparisonMode="StrongWildcard"
    maxBufferSize="1000000000" maxBufferPoolSize="1000000000" 
    maxReceivedMessageSize="1000000000"
    messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
    useDefaultWebProxy="true">
    <readerQuotas maxDepth="32" maxStringContentLength="1000000000" 
        maxArrayLength="1000000000"
        maxBytesPerRead="4096" maxNameTableCharCount="16384" />
</binding>

EDIT 2:

Streaming mode configuration for client side app.config,

    <basicHttpBinding>
      <binding name="BasicHttpBinding_IStudentManagement" closeTimeout="00:01:00"
        openTimeout="00:20:00" receiveTimeout="01:00:00" sendTimeout="01:00:00"
        allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
        maxBufferSize="1500000000" maxBufferPoolSize="1500000000" maxReceivedMessageSize="1500000000"
        messageEncoding="Mtom" textEncoding="utf-8" transferMode="Streamed"
        useDefaultWebProxy="true">
        <readerQuotas maxDepth="1500000000" maxStringContentLength="1500000000"
          maxArrayLength="1500000000" maxBytesPerRead="1500000000" maxNameTableCharCount="1500000000" />
        <security mode="None">
          <transport clientCredentialType="None" proxyCredentialType="None"
            realm="" />
          <message clientCredentialType="UserName" algorithmSuite="Default" />
        </security>
      </binding>
    </basicHttpBinding>

Server side web.config for streaming mode,

  <basicHttpBinding>
    <binding name="BasicHttpBinding_IStudentManagement" closeTimeout="00:01:00"
        openTimeout="00:20:00" receiveTimeout="01:00:00" sendTimeout="01:00:00"
        allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
        maxBufferSize="1000000000" maxBufferPoolSize="1000000000" maxReceivedMessageSize="1000000000"
        messageEncoding="Mtom" textEncoding="utf-8" transferMode="Streamed"
        useDefaultWebProxy="true">
      <readerQuotas maxDepth="32" maxStringContentLength="1000000000" maxArrayLength="1000000000"
          maxBytesPerRead="4096" maxNameTableCharCount="16384" />
      <security mode="None">
        <transport clientCredentialType="None" proxyCredentialType="None"
            realm="" />
        <message clientCredentialType="UserName" algorithmSuite="Default" />
      </security>
    </binding>
  </basicHttpBinding>

Answer 1

There's a multitude of buffer sizes you can play with - they are kept fairly small (64K) by default in order to avoid Denial-of-service attacks, but if you need to, you can increase those:

 <system.serviceModel>
    <bindings>
        <basicHttpBinding>
            <binding name="largebuffers" closeTimeout="00:01:00"
                openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
                bypassProxyOnLocal="false" transactionFlow="false" 
                hostNameComparisonMode="StrongWildcard"
                maxBufferSize="524288" maxBufferPoolSize="524288" 
                maxReceivedMessageSize="65536"
                messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
                allowCookies="false">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" 
                              maxArrayLength="16384" maxBytesPerRead="4096" 
                              maxNameTableCharCount="16384" />
            </binding>
        </basicHttpBinding>
    </bindings>

Have a look at the "MaxBufferSize", "MaxBufferPoolSize", "MaxReceivedMessageSize" settingsin the binding, as well as the various settings in the <readerQuotas> section.

I would try to just increase the "MaxBufferSize" and "MaxBufferPoolSize" first and see if that alone helps - most of the other settings should really be more geared towards when the service receives and needs to process a message.

There are no hard limits as to how big an object can be that you serialize back. However, the DataTable does carry a significant overhead, and if you don't really need that DataTable functionality on the client side, you might also do a conversion on the server to send back just a collection or generic list of objects - instead of the heavy-lift DataTable object.

Furthermore, if you frequently send back large amounts of data, you might need to investigate the streaming capabilities of WCF, too (e.g. if you return pictures, videos, that kind of stuff). Marc