v1.2.3 released

I just released v1.2.3 of Corporate Addressbook to the Android Market.

Features and fixes include:

  • Support for Zimbra (Issue #47)
    • Zimbra did not like the query string that was sent from the app, and threw and error 500 on each search request received. Turns out that it required that the optional range element be included in every request. Thanks Stephen and Martin for helping debug this issue.
  • Error Code 0 (Issue #49)
    • Looks like a lot of people were seeing an error code 0 returned when they were trying to log in. Debugging with a few folks led to two discoveries
      1. Spaces in username: I had failed to URL encode the URL that is generated by concatenating the server name with the username. As a result of this any spaces in the username caused HttpOptions to barf.
      2. Any exceptions thrown during the connection process were being handled, but the actual error string was not propagated up to the UI.  Added some code to make the actual error string more user friendly. Hopefully this should help in getting to the bottom of connection issues.
  • Login screen does not scroll in landscape mode (Issue #40)
    • The login screen worked fine in portrait but was completely broken in the landscape mode. Dale picked up this issue and added a Scroll View to the UI. This should allow users to scroll to the Login button.

ActiveSync Primer continued – The PROVISION command

The PROVISION command and response is used by Exchange servers to communicate security policy settings to client devices.  If a security policy has been setup for an Exchange server, Exchange will not process any requests received from a client, until the policy settings are requested and acknowledged by the client. Please see the ActiveSync Provisioning Protocol Specification for more information about this command.

Depending upon the version of the Exchange server one of the following error messages are returned when a client issues a request before accepting the security policies.

  • HTTP/1.1 449 : Retry after sending a PROVISION command
  • HTTP/1.1 200 with a global status code in the body of the response of 142 (see example here).

A lot of this is repeated from the ActiveSync provisioning protocol documentation. However there are a few gotchas here, so I think it might be better to step through it.

Step 1: Client requests security policy from the server

Header

// Send a POST request to the mail server. Append
// the string “Microsoft-Server-ActiveSync” to the URL, and add the
// the following query strings as parameters to the URL as shown below
// username – Username used to log into the server.
// deviceId: DeviceId for the device (not validated by server)
// deviceType: Type of device (not validated by server)
// cmd: The actual command being sent across (Provision in this case)

POST https://mail.example.com/Microsoft-Server-ActiveSync?User=username&DeviceId=123412341234&DeviceType=Android&Cmd=Provision HTTP/1.1

// Set the user-agent
User-Agent: Android

// Authorization: This is the text Basic followed by a
// base64 encode of the string DOMAIN\USERNAME:PASSWORD.
// Replace with appropriate values.
Authorization: Basic RE9NQUlOXFVTRVJOQU1FOlBBU1NXT1JE

// Set the content-length to the length of the WBXML being sent (more on this in a bit)
Content-Length: FIXME

// Indicate to the Exchange server that we are sending
// WBXML encoded content.
Content-Type: application/vnd.ms-sync.wbxml

// Identifies the protocol version the client (we) support
// NOTE: This cannot be higher than the highest ActiveSync protocol
// version supported by the server. See my Options primer
// for details.
// Also I would recommend setting this to 12.1, because some Exchange
// 2010 requires servers (v14.1) require 14.1 clients to send several
// optional fields (yep you read that right). Setting this to 12.1
// makes your life a wee bit easier.

MS-ASProtocolVersion: 12.1

// English language
Accept-Language: en-US

// Hostname the request is being sent to
Host: mail.example.com

// Set the PolicyKey to zero indicating to the server
// That you do not currently have a policy key
X-MS-PolicyKey: 0

Body

// Send the following XML string to the server.
// Note that the XML below has to be encoded
// into WBXML before sending it across to the
// Exchange server. More on this in a bit.
// The policyType should be set to
// MS-EAS-Provisioning-WBXML if the activeSync version greater
/ than or equal to 12.0, MS-WAP-Provisioning-XML otherwise

[code lang=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision:">
<Policies>
<Policy>
<!– If ActiveSync Version >= 12.0 –>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<!– If ActiveSync Version < 12.0 –>
<PolicyType>MS-WAP-Provisioning-XML</PolicyType>
</Policy>
</Policies>
</Provision>[/code]

Now the XML in the body above has to be encoded into WBXML, before it can be sent across to the server. This is true for all ActiveSync commands that have a body. I wrote a Java implementation of a WBXML encoder and decoder for my app, and you can find the source code here. The source code is based on a WBXML parser I found in the k9mail. I modified the source to add support for International languages.

Step 2: Server responds to client request with security policy and temporary policyKey

Once the request above is sent to the server, you should get a response that looks somewhat like this. Again the XML returned from the server is encoded in WBXML. It is the client’s responsibility to decode this WBXML and make sense of it.

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1069
Content-Type: application/vnd.ms-sync.wbxml
Server: Microsoft-IIS/7.5
MS-Server-ActiveSync: 8.3
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Cache-Control: private
Date: Mon, 01 Feb 2011 21:14:17 GMT

[code lang=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision">
<Status>1</Status>
<Policies>
<Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<Status>1</Status>
<PolicyKey>2152355410</PolicyKey>
<Data>
<EASProvisionDoc>
<DevicePasswordEnabled>0</DevicePasswordEnabled>
<AlphanumericDevicePasswordRequired>0</AlphanumericDevicePasswordRequired>
<PasswordRecoveryEnabled>0</PasswordRecoveryEnabled>
<DeviceEncryptionEnabled>0</DeviceEncryptionEnabled>
<AttachmentsEnabled>1</AttachmentsEnabled>
<MinDevicePasswordLength/>
<MaxInactivityTimeDeviceLock/>
<MaxDevicePasswordFailedAttempts/>
<MaxAttachmentSize/>
<AllowSimpleDevicePassword>0</AllowSimpleDevicePassword>
<DevicePasswordExpiration/>
<DevicePasswordHistory>0</DevicePasswordHistory>
<AllowStorageCard>1</AllowStorageCard>
<AllowCamera>1</AllowCamera>
<RequireDeviceEncryption>0</RequireDeviceEncryption>
<AllowUnsignedApplications>1</AllowUnsignedApplications>
<AllowUnsignedInstallationPackages>1</AllowUnsignedInstallationPackages>
<MinDevicePasswordComplexCharacters>3</MinDevicePasswordComplexCharacters>
<AllowWiFi>1</AllowWiFi>
<AllowTextMessaging>1</AllowTextMessaging>
<AllowPOPIMAPEmail>1</AllowPOPIMAPEmail>
<AllowBluetooth>2</AllowBluetooth>
<AllowIrDA>1</AllowIrDA>
<RequireManualSyncWhenRoaming>1</RequireManualSyncWhenRoaming>
<AllowDesktopSync>1</AllowDesktopSync>
<MaxCalendarAgeFilter>0</MaxCalendarAgeFilter>
<AllowHTMLEmail>1</AllowHTMLEmail>
<MaxEmailAgeFilter>0</MaxEmailAgeFilter>
<MaxEmailBodyTruncationSize>-1</MaxEmailBodyTruncationSize>
<MaxEmailHTMLBodyTruncationSize>-1</MaxEmailHTMLBodyTruncationSize>
<RequireSignedSMIMEMessages>0</RequireSignedSMIMEMessages>
<RequireEncryptedSMIMEMessages>0</RequireEncryptedSMIMEMessages>
<RequireSignedSMIMEAlgorithm>0</RequireSignedSMIMEAlgorithm>
<RequireEncryptionSMIMEAlgorithm>0</RequireEncryptionSMIMEAlgorithm>
<AllowSMIMEEncryptionAlgorithmNegotiation>2</AllowSMIMEEncryptionAlgorithmNegotiation>
<AllowSMIMESoftCerts>1</AllowSMIMESoftCerts>
<AllowBrowser>1</AllowBrowser>
<AllowConsumerEmail>1</AllowConsumerEmail>
<AllowRemoteDesktop>1</AllowRemoteDesktop>
<AllowInternetSharing>1</AllowInternetSharing>
<UnapprovedInROMApplicationList/>
<ApprovedApplicationList/>
</EASProvisionDoc>
</Data>
</Policy>
</Policies>
</Provision>
[/code]

The 200 response code indicates a success. The XML returned from the server contains the security policy that the server would like the client to implement. Once the client receives this security policy, it needs to implement the security policy indicated in the XML, and then acknowledge receipt of the security policy. Note that the policyKey (2152355410) returned by the server in the response is temporary and needs to be sent to the server in Step 3 (acknowledgement).

Step 3: Client acknowledges receipt and application of security policy

The client should now acknowledge the security policy and its application by using the temporary policyKey obtained in Step 2.

POST https://mail.example.com/Microsoft-Server-ActiveSync?User=username&DeviceId=123412341234&DeviceType=Android&Cmd=Provision HTTP/1.1
User-Agent: Android
Authorization: Basic RE9NQUlOXFVTRVJOQU1FOlBBU1NXT1JE
Content-Length: FIXME
Content-Type: application/vnd.ms-sync.wbxml
MS-ASProtocolVersion: 12.1
Accept-Language: en-US
Host: mail.example.com
X-MS-PolicyKey: 2152355410

[code lang=”xml”]<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision:">
<Policies>
<Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<PolicyKey>2152355410</PolicyKey>
<Status>1</Status>
</Policy>
</Policies>
</Provision>
[/code]

Step 4: Server responds with final policyKey

At this point the server will respond with the “final” policyKey which the client then uses in the X-MS-PolicyKey header of all successive command requests to the server.

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1069
Content-Type: application/vnd.ms-sync.wbxml
Server: Microsoft-IIS/7.5
MS-Server-ActiveSync: 8.3
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Cache-Control: private
Date: Mon, 01 Feb 2011 21:15:17 GMT

[code lang=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision:">
<Status>1</Status>
<Policies>
<Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<Status>1</Status>
<PolicyKey>12432432244</PolicyKey>
</Policy>
</Policies>
</Provision>
[/code]

A word of caution here. I have seen cases where a server processes a few client requests that contain the final policyKey obtained using the method above, and then start sending down a 449 or a 142 error code. This implies that the client needs to acknowledge the policy settings again. So make sure you always check for 142 and 449 error codes even if you have accepted the policy settings once and received a final policyKey.

An Activesync Primer – The Options request

Now that my app is out in the market (finally), I think it is time to write a brief how-to on ActiveSync and how I managed to get things working.

First off, I would like to acknowledge the help and support from the following individuals / websites, without  whom this app would not been possible

Ok lets get started.

The Activesync OPTIONS Command

This is the first command that you want to send to an ActiveSync server. If the message is correctly formatted, the server responds with with a list of commands supported, and also the ActiveSync version that is supported. Here is a sample request (RAW view from Fiddler). Please note that the text in bold will need to be modified, and actual values will need to be inserted. See comments prefixed by // for a better description of each field.

[EDIT 01/20/2011] Turns out if you are connecting to Google Apps (that runs on Linux) or an Exchange server that is protected by a reverse proxy, case matters. so the Activesync text below should be ActiveSync (note the camelcase)
// Send an Option request to the mail server. Append the
// following location to the URL Microsoft-Server-ActiveSsync
// and add the username used to log into the server.
OPTIONS https:/ /mail.example.com/Microsoft-Server-ActiveSsync?User=username HTTP/1.1

// Set this to anything. Since I wrote an Android app,
// I set this to Android
User-Agent: Android

// This is the text Basic followed by a
// base64 encode of the string DOMAIN\USERNAME:PASSWORD.
// Replace with appropriate values.
Authorization: Basic RE9NQUlOXFVTRVJOQU1FOlBBU1NXT1JE

[EDIT 01/20/2011] Exchange server 2010 does not like it if you send any additional headers. The only required headers are the auth and the user-agent. Hence none of the headers below are required for the OPTIONS request
// Since this is an Options request, there is no content
// just the headers. Hence this is set to zero
Content-Length: 0

// Indicate to the Exchange server that we are sending
// WBXML encoded content.
// Right now since is not really applicable.
Content-Type: application/vnd.ms-sync.wbxml

// Identifies the protocol version the client (we) support
MS-ASProtocolVersion: 12.1

// English language
Accept-Language: en-US

// Hostname the request is being sent to
Host: mail.example.com

If all goes well, you should get a response that looks somewhat like this

HTTP/1.1 200 OK
Cache-Control: private
Allow: OPTIONS,POST
Content-Length: 0
Server: Microsoft-IIS/7.5
MS-Server-ActiveSync: 8.3
MS-ASProtocolVersions: 1.0,2.0,2.1,2.5,12.0,12.1
MS-ASProtocolCommands: Sync, SendMail, SmartForward, SmartReply, GetAttachment, GetHierarchy, CreateCollection, DeleteCollection, MoveCollection, FolderSync, FolderCreate, FolderDelete, FolderUpdate, MoveItems, GetItemEstimate, MeetingResponse, Search, Settings, Ping, ItemOperations, Provision, ResolveRecipients, ValidateCert
Public: OPTIONS,POST
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Mon, 06 Dec 2010 21:14:17 GMT

The 200 response code indicates a success. The MS-ASProtocolVersions field indicates the version of the Activesync Protocol that is supported by the server. The MS-ASProtocolCommands field indicates what Activesync commands are supported by the server.

I will keep updating my blog with more information on how to use Activesync to query the GAL. Stay tuned.

Corporate Addressbook – now available on the Android market

My first Android app (Corporate Addressbook) is now available on the Android market.

I wrote this app since my Nexus One lacks a feature that I really needed – a true GAL lookup. The Global Address List (GAL) lookup currently available in vanilla Android (Froyo running on my Nexus One) is quite limited in functionality. It only looks up email addresses and does not return anything more. This is quite frustrating when you are trying to look up the address book for the contact information of a colleague. The only other option was to buy Touchdown. Touchdown is a great app, however it does much more than GAL lookups. It also fetches email and does a bunch of other things that are now included in Froyo. Hence I thought it was better to write this application to meet my requirement.

This application looks up the GAL and returns ALL data that is available on the Exchange server for the query. The app uses the ActiveSync protocol to communicate with the Exchange server, and supports both Exchange 2003 and 2007.

The source for this app is available on Google Code, and is available via the Apache 2.0 license.

Download it while it is hot 🙂

Appbrain
Android Market

Please do let me know if you come across and bugs / issues or if you have any comments.