Msxml2.ServerXMLHTTP

Update: added information on Kerberos.

When using MSXML server-side to get XML documents (with ServerXMLHTTP of course, XMLHTTP is for client-side usage), I found a potential performance issue.

I am investigating inter-server communciation with XML over HTTP, so this article should be interpreted from that perspective. My solution may not be the preferred choice to your specific situation.

Summary

With Microsoft's ServerXMLHTTP class, the programmer can't specify a preferred authentication mechanism. The workaround is to specify a custom header manually.

HTTP authentication

When retrieving an HTTP resource, either with Basic, Digest or NTLM (also called Challenge Response) authentication, an HTTP request is issued without any credentials. Then the server responds with an 401 Not Authorized response including one or more WWW-authenticate response headers. Then the client can pick its preferred authentication method.

Windows 2000 (and XP, 2003) adds an extra authentication sub-scheme to Integrated Security, named Kerberos. This would be perfect for our situation, but good old NT4 doesn't support it. At IIS Faq you can read more about authentication models.

This is a powerful mechanism, also referred to as negotiation: the client doesn't need to know beforehand which authentication mechanism to use. Server and client should at least share one common authentication method, of course.

Our situation

The http-calls we issue are between servers or on the same server, depending configuration. This means, we control the authentication scheme.

The NTLM authentication method is a proprietry authentication mechanism. It uses a 4-way handshake to authenticate. Considering that fact plus it's only slightly safer than Basic authentication, we dismissed this scheme.

Digest authentication is also dismissed, because we run Windows NT4, so that's not an option either. So that leaves us with Basic authentication

Getting the XML

Our XML requests are generated from within ASP pages. The code is straightforward, we get an HTTP resource synchronously (blocking) with API provided authentication. The code below just copies the received XML to the HTTP response.

<%
'VBScript
Dim objSrvHTTP: Set http = Server.CreateObject("MSXML2.ServerXMLHTTP")

http.open "GET","http://url.local/page.asp", False, "username", "password"
http.send
Response.ContentType = "text/xml"
http.responseXML.save Response
%>

Each time above script is ran, two HTTP requests and responses are generated (as explained in the HTTP authentication introduction above). In a client-side situation (ie your browser) that's perfect, because for the next call, the client remembers the supported scheme. In a server environment, the ASP page directly forgets the authentication scheme when it has finished running.

This is bad for two reasons. Response times drop (latency), caused by the extra request and response. There is also extra load, because the 401 Not Authorized response also contains HTML for a user-readable explanation. Default configuration on IIS is a 4KB document, using four (4) TCP/IP packets.

The solution

The most logical solution is to tell MSXML2.ServerXMLHTTP directly to send an authentication header with the initial request. But unfortunately, the authentication scheme negotiation of the component can't be influenced (it actually favours NTLM over Basic authentication, when the server supports both).

Fortunately, the Basic authentication scheme is fairly simple. Below I include the code to directly use the Basic authentication scheme. Notice I still specify the username and password parameters on the open, so NTLM-only servers are still supported. The setRequestHeader method call should be after the open call.

The code should run in VB without modification. Robert Graham has written a javascript implementation.

Dim http: Set http = Server.CreateObject("MSXML2.ServerXMLHTTP")

'Always specify username/password here, for fail safety
http.open "GET", "http://url.local/page.asp", _
  False,"username","password"
http.setRequestHeader "Authorization", _
  "Basic " & Base64Encode("username:password")
http.send

Response.ContentType = "text/xml"
http.responseXML.save Response

Function Base64Encode(inData)
'ripped from: 
'http://www.pstruh.cz/tips/detpg_Base64Encode.htm
  'rfc1521
  '2001 Antonin Foller, PSTRUH Software, http://pstruh.cz
  Const Base64 = _
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
  Dim sOut, I
  
  'For each group of 3 bytes
  For I = 1 To Len(inData) Step 3
    Dim nGroup, pOut
    
    'Create one long from this 3 bytes.
    nGroup = &H10000 * Asc(Mid(inData, I, 1)) + _
      &H100 * MyASC(Mid(inData, I + 1, 1)) + _
      MyASC(Mid(inData, I + 2, 1))
    
    'Oct splits the long To 8 groups with 3 bits
    nGroup = Oct(nGroup)
    
    'Add leading zeros
    nGroup = String(8 - Len(nGroup), "0") & nGroup
    
    'Convert To base64
    pOut = Mid(Base64, CLng("&o" & Mid(nGroup, 1, 2)) + 1, 1) + _
      Mid(Base64, CLng("&o" & Mid(nGroup, 3, 2)) + 1, 1) + _
      Mid(Base64, CLng("&o" & Mid(nGroup, 5, 2)) + 1, 1) + _
      Mid(Base64, CLng("&o" & Mid(nGroup, 7, 2)) + 1, 1)
    
    'Add the part To OutPut string
    sOut = sOut + pOut
    
  Next
  Select Case Len(inData) Mod 3
    Case 1: '8 bit final
      sOut = Left(sOut, Len(sOut) - 2) + "=="
    Case 2: '16 bit final
      sOut = Left(sOut, Len(sOut) - 1) + "="
  End Select
  Base64Encode = sOut
End Function

Function MyASC(OneChar)
  If OneChar = "" Then MyASC = 0 Else MyASC = Asc(OneChar)
End Function

Data on performance gain will be posted here, when we have ran the tests.