Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
441 views
in Technique[技术] by (71.8m points)

c# - IRS-A2A BulkRequestTransmitter message not formmatted properly and/or cannot be interpreted

I am receiving the following error when attempting to submit through the BulkRequestTransmitter Web Service. The Composition Guide is less than helpful as far as this message goes, and when I compare my SOAP XML with the SOAP from the Composition Guide, they seem to be apples-to-apples. I'm hoping that another set of eyes may be able to see where the problem is.

The message was not formatted properly and/or cannot be interpreted. Please review the XML standards outlined in Section 3 of the AIR Submission Composition and Reference Guide located at https://www.irs.gov/for-Tax-Pros/Software-Developers/Information-Returns/Affordable-Care-Act-Information-Return-AIR-Program, correct any issues, and try again.

What I've Tried:

  • Attempted to submit with (and without) whitespace in the SOAP Envelope.
  • Attempted to submit with the Form Data XML in XML format.
  • Attempted to submit with the Form Data in base64string format (as this submission was).
  • Added the ds prefix to the Signature elements. Used this SO post in order to add the prefix to the Signature elements.
  • Added the Form Data in "Pretty Print" format and as according to the updated Composition Guide (v4.2).
  • Copied the formatting of the MIME for the BulkTransmitterService request outlined in section 10.3 of the Composition Guide.
  • Created two solutions: 1.) Manually creating the XML necessary for the SOAP requests and sending via HttpWebRequest object; 2.) Sending a submission request via the WSDL imported to the project as a Service Reference, using custom encoders for GZip and Mtom Encoding and manually creating the XML necessary for the SOAP Status Request (sent via HttpWebRequest).

Update #1
Updated the request based on some new additions.

  • Added the ds prefix to the Signature elements.
  • Added the Form Data in "Pretty Print" format and as according to the updated Composition Guide (v4.2: Section 5.4.2).

Update #2
I began to manually create the SOAP .xml file within a new instance of Visual Studio importing the schema references as necessary. I'm doing this outside of any sort of application creation.

In doing so, I was able to find some additional bugs in the SOAP I was creating through my application (thank you for intellisense!). The bugs that I found were within the Manifest XML, as they didn't conform to the IRS schema.

I will be looking into these in the next 24 hours and update accordingly.

  • The urn:MailingAddressGrp should have a child of either urn:USAddressGrp or urn:ForeignAddressGrp. That child should then contain the proper address elements. My code is currently missing the direct child of the urn:MailingAddressGrp.
  • The value for urn1:DocumentSystemFileNm of Form1094C_Request_[TCC]_yyyyMMddThhmmssfffZ.xml is incorrect. I'm not entirely sure what it should be just yet.
  • The urn1:BulkExchangeFile element, is having an issue related to the xop:Include element I have within. The schema wants a base64Binary type.

Update #2.5

  • Updated my XML generation process to include the USAddressGrp element.
  • Discovered that I had one extra character in the milliseconds (four instead of three). Once I corrected this, along with removing the string "Form" from the beginning of the file name, the value for the urn1:DocumentSystemFileNm was able to validate against the schema successfully.

Update #3

  • Updated the Full Request based on the updates I have made. At this point, I am unable to deduce what is wrong with my request. If anyone sees anything glaring, please help!

Update #4

  • Updated the Full Request based on additional updates made. Removed the ds prefix from the Signature based on another SO user's feedback. This user has gotten these requests to work without having to append the ds prefix to the Signature after the fact and re-compute the signature.

    The SO user also confirmed that his requests are working with an <inc:Include> element being setup as a child element of the <BulkExchangeFile> element.

  • Confirmed the MIME headers are correct as per the sample in section 10.3 of the Composition Guide.

Update #5

  • I currently have two solutions: one which is sending manually creating the XML necessary for the SOAP requests and sending via HttpWebRequest; and one which is using the WSDL Service Reference for the Submission Request, using the custom encoders outlined below, and manually creating the XML necessary for the SOAP Request of the Status.

    As of this update, Solution 1 gives me the error above when making a Submission Request, and gives me the error below when making the Status Request. However, when using Solution 2, both requests (Submission and Status) give me the error below.

    I am looking into possible certificate issues to see if they make any progress with either of these solutions.

Update #6

There were a number of issues I ran into which caused me to be delayed. I'll spare you the nitty-gritty details, however, the short of it is that we did not have the Security Certificate registered with the IRS system, nor did we have the Certificate installed properly so that I could access the information through the X509Store. Finally these things got done, and I was able to test submitting data to the IRS from the server (vs. my localmachine which did not have the proper certificate). Unfortunately, I am still receiving the WS-Security error detailed below. I have updated the Full Request with what I am currently sending.

An Error Occurred with message: The WS Security Header in the message is invalid. Please review the transmission instructions outlined in Section 5 of the AIR Submission Composition and Reference Guide located at https://www.irs.gov/for-Tax-Pros/Software-Developers/Information-Returns/Affordable-Care-Act-Information-Return-AIR-Program, correct any issues, and try again.


All line breaks in the MIME headers are as-is, and I believe the line breaks are what is expected. The FormData attachment is being sent as Pretty Print while the SOAP Envelope is not; The SOAP Envelope in this post is formatted for readability.

UPDATE #7:

Thanks to users: jstill and fatherOfWine with what they have posted below, and to Bon for earlier assistance on this project. I have broken through one wall in getting the Submission to work. It is now working. The Status request is also working. However, I need to figure out how to process it in order to pull the status and the attachment (error data file) out of it.

Full Request:

Content-Encoding: gzip
Accept-Encoding: gzip, deflate
Content-Type: multipart/related; type="application/xop+xml"; start="<rootpart>"; start-info="text/xml"; boundary="MIME_boundary"
SOAPAction: BulkRequestTransmitter
MIME-Version: 1.0
Host: la.www4.irs.gov

--MIME_Boundary
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: 8bit
Content-Id: <root_part>

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
        <Security xmlns:h="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
            <Signature Id="SIG-E77c57b78ebc54e989bfc9e43604a04a4" xmlns="http://www.w3.org/2000/09/xmldsig#">
                <SignedInfo>
                    <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#WithComments" />
                    <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
                    <Reference URI="#TS-Eb4799bee41bb4df0a72f52832d283ef7">
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>[TimestampDigestValue]</DigestValue>
                    </Reference>
                    <Reference URI="#id-E5f1ed32aab8f4578adeee5debd851a62">
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>[ACABusinessHeaderDigestValue]</DigestValue>
                    </Reference>
                    <Reference URI="#id-E4a71164001994d7f865fc7ddb8055350">
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>[ManifestDigestValue]</DigestValue>
                    </Reference>
                </SignedInfo>
                <SignatureValue>[SignatureValue]</SignatureValue>
                <KeyInfo Id="KI-E2309cb142e1a4076a2e71373e6e6b75f">
                    <SecurityTokenReference d6p1:Id="STR-E2751169ee468470290fe5e8bfb34589e" xmlns:d6p1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
                        <KeyIdentifier EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3">[KeyIdentifier]</KeyIdentifier>
                    </SecurityTokenReference>
                </KeyInfo>
            </Signature>
            <a:Timestamp a:Id="TS-Eb4799bee41bb4df0a72f52832d283ef7" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
                <a:Created>2016-05-18T09:51:05.856Z</a:Created>
                <a:Expires>2016-05-18T10:01:05.856Z</a:Expires>
            </a:Timestamp>
        </Security>
        <ACATransmitterManifestReqDtl a:Id="id-E4a71164001994d7f865fc7ddb8055350" xmlns:h="urn:us:gov:treasury:irs:ext:aca:air:7.0" xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns="urn:us:gov:treasury:irs:ext:aca:air:7.0">
            <PaymentYr>2015</PaymentYr>
            <PriorYearDataInd>0</PriorYearDataI

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Don't know if it will resolve your issue, but nevertheless i give it a shot. Sometimes help comes from very unexpected sources :)

  1. First of all timestamp fields are in a wrong format: one in businessheader should NOT contain milliseconds at all. I know it for a fact.
  2. In security header timestamp only 3 digits are allowed to represent milliseconds.
  3. Remove empty elements like "OriginalReceiptId" from ACATransmitterManifestReqDtl element: they don't like those.
  4. I hope you are providing them with proper software id, because you have it empty in the payload, but I am sure they would love to have it, imho.:)
  5. And I think the message you've got in the response also have something to do with Signature element. I think they want Signature element to have some prefix("ds" preferably, I guess). But here I am not sure on 100%.

    You see, I am battling same battle as you. And my message security timestamp has prefix "u" and they do not complain about it. Though they didn't like binarysecuritytoken ever.:) I am struggling to generate signature to the IRS liking. WCF is very secretive and does not allow easy prefix changing on soap envelope or allow to choose CanonicalizationMethod algorithm for a signature.

UPDATE: Been able to successfully send request to the service. Tell you at once: prefixes are unimportant. What was important: CorrectedInd tag must be present in Form1095BUpstreamDetail and attributes recordType="String" lineNum="0" also must be present.

UPDATE2: Another thing that I've changed I've placed ACABusinessHeader before ManifestDtl. Here are my settings: I am using WCF as carrier and SignedXml to generate signature. Also I am using custom gZip encoder(for obvious reasons0 and custom MtomEncoder to read response from service(yes, yes it's MTOMed:)) can you believe those pokemons?!?!?) and that's not all: they send response as multipart document with only 1 part!:)) I had to adjust my encoder to handle that. And voilà, service started to behave. Hope it might help.

UPDATE3 First of all make sure data in attachment file correspond to the test scenario you are using as guinea pig. I, probably, sound like a broken record, but that's REALLY important. Now I'll cut the stuff and present what I have. It's a bit crude, but it does the trick.:)

1.Here is config file portion:
1.1.Make sure system.serviceModel element contains following portion:

<extensions>
  <bindingElementExtensions>
    <add name="gzipMessageEncoding" type="<namespaceWhereEncoderLives>.GZipMessageEncodingElement, GZipEncoder, Version=4.0.0.0, Culture=neutral, PublicKeyToken=null" />
  </bindingElementExtensions>
</extensions>  

1.2. Make sure binding element contains this:

  <customBinding>
    <binding name="BulkRequestTransmitterBinding">
      <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
      <httpsTransport />
    </binding>
  </customBinding>

1.3. Change binding of BulkRequestTransmitterPort endpoit under client element to "customBinding"(and change binding name to the name of the custom binding as well) and make sure it contains following portion:

    <identity>
      <dns value="domain from cert" />
    </identity>

Also client element should contain following portion:

  <metadata>
    <policyImporters>
      <extension type="NamespaceToToTheLocationOf.GZipMessageEncodingBindingElementImporter, GZipMessageEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
    </policyImporters>
  </metadata>
  1. GZip encoder you could get from following link: https://msdn.microsoft.com/en-us/library/cc138373(v=vs.90).aspx Just download WCF example and dully move whole GZipMessageEncoder project under your project.

  2. Get MTOMEncoder(which I renamed from SwaEncoder for clarity reasons) from this link: Soap-with-Attachments Move following classes into GZipMessageEncoder project:
    MimeContent, MimeParser, MimePart, MTOMEncoder

  3. Modify GZipMessageEncoder class like this:
    4.1. Add following code at the beginning of the class:

       //------------------- MTOM related stuff. Begin. ---------------------
        const string ATTCHMNT_PROP = "attachment_file_content";
        const string ATTCHMNT_CONTENT_ID = "Here goes content id";
    
        private string _ContentType;
        private string _MediaType;
    
        protected MimeContent _MyContent;
        protected MimePart _SoapMimeContent;
        protected MimePart _AttachmentMimeContent;
        protected GZipMessageEncoderFactory _Factory;
        protected MimeParser _MimeParser;
        private void SetupMTOM(GZipMessageEncoderFactory factory)
        {
            //
            _ContentType = "multipart/related";
            _MediaType = _ContentType;
    
            //
            // Create owned objects
            //
            _Factory = factory;
            _MimeParser = new MimeParser();
    
            //
            // Create object for the mime content message
            // 
            _SoapMimeContent = new MimePart()
            {
                ContentTypeStart = "application/xop+xml",
                ContentType = "text/xml",
                ContentId = "Here goes envelope MIME id from HTTP Content-Type header",   // TODO: make content id dynamic or configurable?
                CharSet = "UTF-8",                                  // TODO: make charset configurable?
                TransferEncoding = "8bit"                         // TODO: make transfer-encoding configurable?
            };
            _AttachmentMimeContent = new MimePart()
            {
                ContentType = "application/xml",                    // TODO: AttachmentMimeContent.ContentType configurable?
                ContentId = ATTCHMNT_CONTENT_ID,                    // TODO: AttachmentMimeContent.ContentId configurable/dynamic?
                TransferEncoding = "7bit"                         // TODO: AttachmentMimeContent.TransferEncoding dynamic/configurable?
            };
            _MyContent = new MimeContent()
            {
                Boundary = "here goes boundary id"  // TODO: MimeContent.Boundary configurable/dynamic?
           };
            _MyContent.Parts.Add(_SoapMimeContent);
            _MyContent.Parts.Add(_AttachmentMimeContent);
            _MyContent.SetAsStartPart(_SoapMimeContent);
        }
        //------------------- MTOM related stuff. End. ----------------------
    

4.2. Modify Method WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset) like this:

public override ArraySegment<byte> WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
        {
            ArraySegment<byte> buffer = innerEncoder.WriteMessage(message, maxMessageSize, bufferManager, 0);
            var requestSOAPEnvelopeXml = System.Text.Encoding.UTF8.GetString(buffer.Array);

            //Here you create Security node and sign the request. For ex:
            requestSOAPEnvelopeXml = SigngEnvelope(requestSOAPEnvelopeXml);
            //Here you are getting 10941095 forms xml payload.
            string fileContent = GetAttachmentFileContent();

            //Here comes the MTOMing...
            _SoapMimeContent.Content = System.Text.Encoding.UTF8.GetBytes(requestSOAPEnvelopeXml);
            _AttachmentMimeContent.Content = System.Text.Encoding.UTF8.GetBytes(fileContent);

            _MyContent.Parts.Where(m=> m.ContentId!=null && m.ContentId.Equals(ATTCHMNT_CONTENT_ID)).Single().ContentDisposition = GetFileName(envelope);
            // Now create the message content for the stream
            byte[] MimeContentBytes = _MimeParser.SerializeMimeContent(_MyContent);
            int MimeContentLength = MimeContentBytes.Length;

            // Write the mime content into the section of the buffer passed into the method
            byte[] TargetBuffer = bufferManager.TakeBuffer(MimeContentLength + messageOffset);
            Array.Copy(MimeContentBytes, 0, TargetBuffer, messageOffset, MimeContentLength);

            // Return the segment of the buffer to the framework
            return CompressBuffer(new ArraySegment<byte>(TargetBuffer, messageOffset, MimeContentLength), bufferManager, messageOffset);                
        }

4.3. Override couple more methods like this:

public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager, string contentType)
        {
            ArraySegment<byte> decompressedBuffer = DecompressBuffer(buffer, bufferManager);

            MtomEncoder mtomEncoder = new MtomEncoder(innerEncoder, _Factory);
            Message returnMessage = mtomEncoder.ReadMessage(buffer, bufferManager, contentType);
            returnMessage.Properties.Encoder = mtomEncoder;

            return returnMessage;
        }

        public override bool IsContentTypeSupported(string contentType)
        {
            return true;
        }

4.4. Make sure GZipMessage constructor looks like this:

        internal GZipMessageEncoder(MessageEncoder messageEncoder, GZipMessageEncoderFactory factory)
            : base()
        {
            if (messageEncoder == null)
                throw new ArgumentNullException("messageEncoder", "A valid message encoder must be passed to the GZipEncoder");
            innerEncoder = messageEncoder;

            SetupMTOM(factory);
        }

5. Make sure GZipMessageEncodingBindingElement class has following method:

    public override void ApplyConfiguration(BindingElement bindingElement)
    {
        GZipMessageEncodingBindingElement binding = (GZipMessageEncodingBindingElement)bindingElement;
        PropertyInformationCollection propertyInfo = this.ElementInformation.Properties;
        if (propertyInfo["innerMessageEncoding"].ValueOrigin != PropertyValueOrigin.Default)
        {
            switch (this.InnerMessageEncoding)
            {
                case "textMessageEncoding":
                    binding.InnerMessageEncodingBindingElement = 
                        new TextMessageEncodingBindingElement(MessageVersion.Soap11, Encoding.UTF8);
                    break;
                case "binaryMessageEncoding":
                    binding.InnerMessageEncodingBindingElement = new BinaryMessageEncodingBindingElement();
                    break;
            }
        }
    }
  1. Modify MTOMEncoder class. Make sure that following method looks like this:

    public override Message ReadMessage(System.IO.Stream stream, int maxSizeOfHeaders, string contentType)
    {
        VerifyOperationContext();
    
        if (contentType.ToLower().StartsWith("multipart/related"))
        {
            byte[] ContentBytes = new byte[stream.Length];
            stream.Read(ContentBytes, 0, ContentBytes.Length);
            MimeContent Content = _MimeParser.DeserializeMimeContent(contentType, ContentBytes);
    
            if (Content.Parts.Count >= 1)
            {
                MemoryStream ms = new MemoryStream(Content.Parts[0].Content);
                //At least for now IRS is sending SOAP envelope as 1st part(and only part(sic!) of MULTIpart response) as xml. 
                Message Msg = ReadMessage

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...