powershell - Why is Invoke-WebRequest and Invoke-RestMethod failing and succeeding at the same time?

Question

I wrote a small PowerShell script to send a request to a server and get a simple XML result back.

PowerShell Script

$xml = "<?xml version='1.0' encoding='utf-8'?><Search><ID>278A87E1-1BC2-4E19-82E9-8BBE31D67D20</ID></Search>"
$response = Invoke-RestMethod -Method Post -Uri "http://localhost/search" -ContentType "application/xml" -Body $xml

That's it, really simple and there's no reason that I can see for it to be failing. I have also tried the script with Invoke-WebRequest and both fail. The error returned is Invoke-RestMethod : Value cannot be null. Parameter name: name. The strange thing is that when I monitor this with Wireshark, I see the connection, I see the POST and I see the result from the server and all looks perfectly good but the cmdlet is saying it failed (and yes, the return code is 200).

If I run the Invoke-WebRequest/Invoke-RestMethod with the -OutFile parameter, it runs fine with no errors and saves the result to the specified file; -OutVariable fails just in case you're wondering.

The result is an xml file, the headers specify that it is xml and the xml is properly formatted.

Result when successful

<?xml version="1.0" encoding="UTF-8" ?>
<Result version="1.0" xmlns="urn:xmlns-org">
    <searchID>{278a87e1-1bc2-4e19-82e9-8bbe31d67d20}</searchID>
    <responseStatus>true</responseStatus>
    <responseStatusStrg>MORE</responseStatusStrg>
    <numOfMatches>40</numOfMatches>
</Result>

Does anyone know why the Invoke-XXX cmdlets are returning an error and what I can do to fix it? Again, it works perfectly fine when I use the -OutFile parameter and even when it fails I can see a proper conversation between the script and the server in Wireshark.

Also, if I use -Verbose it tells me the following:

VERBOSE: POST http://localhost/search with -1-byte payload
VERBOSE: received X-byte response of content type application/xml; charset="UTF-8"

Where X-byte is the actual size of the response but it obviously differs with each response depending on the data sent to the server. I just find it odd that the cmdlet fails but says it received a response with data and that it sent a -1-byte payload.

Answer 1

I went ahead and looked into the Invoke-WebRequest cmdlet code and found out why it's failing with this particular error.

It's failing on a call to System.Globalization.EncodingTable.GetCodePageFromName. The encoding is passed to that function as a parameter and the encoding is retrieved from the the cmdlet through the Content-Type header. In the case of this server the Content-Type was sent back in the response as Content-Type: application/xml; charset="UTF-8".

The problem with this is that quotes aren't standard for wrapping the value in charset so the cmdlet parses it out as "UTF-8" instead of the valid UTF-8. The cmdlet passes "UTF-8" to the function and the function throws an exception stating that the provided encoding is invalid. This is fine and would make so much more sense if that is what was reported in the final exception but it's not.

The Invalid encoding exception is caught by the Microsoft.PowerShell.Commands.ContentHelper.GetEncodingOrDefault function and in the exception handler it calls GetEncoding again but with a null parameter which results in the final ArgumentNullException for parameter name.

Microsoft.PowerShell.Commands.ContentHelper.GetEncodingOrDefault

internal static Encoding GetEncodingOrDefault(string characterSet)
{
  string name = string.IsNullOrEmpty(characterSet) ? "ISO-8859-1" : characterSet;
  try
  {
    return Encoding.GetEncoding(name);
  }
  catch (ArgumentException ex)
  {
    return Encoding.GetEncoding((string) null);
  }
}

The call to GetEncoding inside the catch statement triggers the following code inside GetCodePageFromName which is itself called from GetEncoding

if (name==null) { 
    throw new ArgumentNullException("name");
}

PowerShell is handling this properly since technically it is an invalid value but you'd think they would call Trim(""") just to be safe.