The first is sent by the client
to introduce itself to the server. This is message type 1.
Amongst other things, it indicates
which version of Outlook is in use and whether the authentication is using the user's logon
password or mail password (more of which later).
The second is the server's challenge. This is message type 2.
It contains the 'nonce', a string of 8 bytes,
unique to the session,
which the client will encrypt with the user's password.
The format of this message varies depending
on whether the client is Outlook or Outlook Express. If the challenge format doesn't
correspond to the client then Outlook will terminate the session.
Finally, the client responds with message type 3.
This identifies the user, domain (or work group) and host name.
These may be ASCII or UNICODE strings.
The 'nonce' sent in the server's challenge is encrypted with the user's password and returned in this message. The server repeats the encryption process with the stored password for the user and if the response strings match, authentication is complete. The encryption process is explained at LanMan and NT Password Encryption in Samba 2.x and NTLM Authentication Scheme for HTTP. This message can contain the response string in either or both of the NT and LanManager formats.
Outlook Express uses ASCII and only sends the LanManager format response string.
E.g.
C : AUTH NTLM
S : +OK NTLM
C : TlRMTVNTUAABAAAABoI .... rest of client intro (message type 1)
S : + TlRMTVNTUAABAAAAA4I .... rest of server challenge (message type 2)
C : TlRMTVNTUAADAAAAGAA .... rest of client response (message type 3)
S : +OK logged on
Non M$ Clients
I've used Ximian Evolution in testing. Other Linux based clients seem to be similar. Almost all support APOP, use the CAPA command and use the AUTH command properly. And most support better authentication schemes than NTLM, e.g. CRAM-MD5. But Exchange Server only supports NTLM/SPA so you may be stuck with it. Note that Exchange Server doesn't support CAPA so your client may be unable to determine whether the server supports NTLM; it just has to try it.For completeness, just in case you want to use NTLM/SPA with a server other than Exchange Server, using the CAPA command will elicit a response like this:
C : CAPA
S : +OK eitha.net POP3 server capabilities
S : SASL CRAM-MD5 NTLM LOGIN
S : TOP
S : UIDL
S : IMPLEMENTATION eitha.net 1.1
S : .
The client issues the AUTH command, specifying NTLM as the authentication method, but, unlike Outlook we dive right in to the authentication exchange. So the client introduction, message type 1, is not used and the client expects the server to respond to the AUTH command with it's challenge, message type 2. This conforms to the SASL mechanism (RFC 2222).
E.g.
C : AUTH NTLM
S : + TlRMTVNTUAABAAAAA4I .... rest of server challenge (message type 2)
C : TlRMTVNTUAADAAAAGAA .... rest of client response (message type 3)
S : +OK logged on
NTLM/SPA and the SMTP Protocol
Outlook and Outlook Express
Here there's much more conformance to the standards. Outlook clients send the EHLO command (RFC 1651) if the "outgoing server requires authentication" option is checked in the mail account settings. The server should list NTLM as one of the supported authentication mechanisms.
Thus :
C : EHLO test
S : 250-mail.eitha.net supports the following extensions
S : 250 AUTH CRAM-MD5 NTLM LOGIN
Outlook then issues the command AUTH NTLM and waits for a positive intermediate
reply from the server. Any information in this reply, e.g the server challenge, is ignored.
Outlook then sends the client introduction message, the server replies with it's challenge
and the client sends it's response which is authenticated as above.
If NTLM does not appear in the list of authentication mechanisms listed by the
server, Outlook won't try to use it.
If LOGIN is listed, Outlook will use that as second best.
If none of the methods Outlook knows about
(NTLM, LOGIN or KERBEROS) appear, Outlook tries sending the message
anyway.
It's then for the server to apply it's authorisation rules to
an unauthenticated connection.
What's more, if the NTLM authentication fails, Outlook will try LOGIN.
So much for keeping that password a secret!
E.g.
C : AUTH NTLM
S : 334 OK
C : TlRMTVNTUAABAAAAB7I .... rest of client intro (message type 1)
S : 334 TlRMTVNTUAABAAAAA4I .... rest of server challenge (message type 2)
C : TlRMTVNTUAADAAAAGAA .... rest of client response (message type 3)
S : 235 AUTH OK
Note that Outlook gets upset if the intermediate reply codes aren't 334, e.g. 332 will not do.
Message Formats
All Types
All messages start with the same information :
char method[8] = "NTLMSSP\0"
short int type = 1, 2, or 3 - the message type
short int z[3] six null bytes
Thereafter they differ accoding to message type.
Type 1 : Client Introduction
Message type 1 is sent by the client to introduce itself to the server. It indicates which version of Outlook is in use and whether the authentication is using the user's logon or mail credentials. SASL conformant clients don't use this introductory message.
NTLM is integrated into the Windows system. When Outlook is asked to provide
NTLM authentication it first attempts to use the Windows logon information, i.e. the
username and password you used to logon. If your mail username/password is the same
as your logon username/password then this is fine and you'll go straight in.
This is typically the case for a Windows workstation connected to a Windows server domain.
If your mail credentials are different, although you have painstakingly entered
this information into the mail account settings, Outlook puts up
a dialog box and asks for the username and password information for the mail server.
No doubt there's an undocumented switch in Windows somewhere that modifies this behaviour
but I haven't been able to find it (if you know, please pass the info on).
For remote users who don't connect to a domain or for
domain-free setups, keeping passwords in sync. can be tiresome.
As a saving virtue, Outlook only does this when it's started, not every time it polls
the mail server.
You'll be pleased to know that non-MS clients behave in an altogether more reasonable fashion
and use the credentials you supply to the mail client.
flags[0] = 0x06 - Outlook Express ; 0x07 - Outlook
flags[1] = 0xb2 - Windows login ; 0x82 - mail client credentials
(The values for flags[1] suggest that this is some kind of bit mask with bit 3 set
for Windows login information. Which implies that there's some more information
in there. Anyone know?)
Type 2 : Server Challenge
The 'nonce' is a string of 8 bytes, unique to the session, which the client will encrypt using the user's password as the encryption key. It can contain non-ASCII bytes. Values should be unpredictable and, if it's repeated at all, only repeated over a long interval. Typically, the server will use the current date, time, and milli-seconds past midnight to generate the nonce. Alternatives might be a simple counter, perhaps combined with the clients IP address. Since the nonce is clearly visible, the objective is to prevent an attacker capturing the client authentication for a given nonce value and replaying it when that value is repeated.
Type 3 : Client Response
Null response
There is one final type of message that can be issued in reply to any of the
other messages, or indeed, the initial positive server reply to AUTH NTLM.
It's simply the single character '*' on a line by itself.
This means 'I don't want to play anymore'.
It's equivalent to QUIT and is usually issued by the client.
Outlook sends it if it doesn't like the format of the server challenge
or if the user cancels out of the password prompt dialog.
The action required is to send a reply similar to those below, and close the connection.
for POP3 :
S : -ERR AUTH terminated by client
and for SMTP :
S : 535 AUTH terminated by client
Stan Thomas / 24 June 2003 / stan@eitha.net.