v1.2.3 released

I just released v1.2.3 of Corporate Addressbook to the Android Market.

Features and fixes include:

  • Support for Zimbra (Issue #47)
    • Zimbra did not like the query string that was sent from the app, and threw and error 500 on each search request received. Turns out that it required that the optional range element be included in every request. Thanks Stephen and Martin for helping debug this issue.
  • Error Code 0 (Issue #49)
    • Looks like a lot of people were seeing an error code 0 returned when they were trying to log in. Debugging with a few folks led to two discoveries
      1. Spaces in username: I had failed to URL encode the URL that is generated by concatenating the server name with the username. As a result of this any spaces in the username caused HttpOptions to barf.
      2. Any exceptions thrown during the connection process were being handled, but the actual error string was not propagated up to the UI.  Added some code to make the actual error string more user friendly. Hopefully this should help in getting to the bottom of connection issues.
  • Login screen does not scroll in landscape mode (Issue #40)
    • The login screen worked fine in portrait but was completely broken in the landscape mode. Dale picked up this issue and added a Scroll View to the UI. This should allow users to scroll to the Login button.

ActiveSync Primer continued – The PROVISION command

The PROVISION command and response is used by Exchange servers to communicate security policy settings to client devices.  If a security policy has been setup for an Exchange server, Exchange will not process any requests received from a client, until the policy settings are requested and acknowledged by the client. Please see the ActiveSync Provisioning Protocol Specification for more information about this command.

Depending upon the version of the Exchange server one of the following error messages are returned when a client issues a request before accepting the security policies.

  • HTTP/1.1 449 : Retry after sending a PROVISION command
  • HTTP/1.1 200 with a global status code in the body of the response of 142 (see example here).

A lot of this is repeated from the ActiveSync provisioning protocol documentation. However there are a few gotchas here, so I think it might be better to step through it.

Step 1: Client requests security policy from the server

Header

// Send a POST request to the mail server. Append
// the string “Microsoft-Server-ActiveSync” to the URL, and add the
// the following query strings as parameters to the URL as shown below
// username – Username used to log into the server.
// deviceId: DeviceId for the device (not validated by server)
// deviceType: Type of device (not validated by server)
// cmd: The actual command being sent across (Provision in this case)

POST https://mail.example.com/Microsoft-Server-ActiveSync?User=username&DeviceId=123412341234&DeviceType=Android&Cmd=Provision HTTP/1.1

// Set the user-agent
User-Agent: Android

// Authorization: This is the text Basic followed by a
// base64 encode of the string DOMAIN\USERNAME:PASSWORD.
// Replace with appropriate values.
Authorization: Basic RE9NQUlOXFVTRVJOQU1FOlBBU1NXT1JE

// Set the content-length to the length of the WBXML being sent (more on this in a bit)
Content-Length: FIXME

// Indicate to the Exchange server that we are sending
// WBXML encoded content.
Content-Type: application/vnd.ms-sync.wbxml

// Identifies the protocol version the client (we) support
// NOTE: This cannot be higher than the highest ActiveSync protocol
// version supported by the server. See my Options primer
// for details.
// Also I would recommend setting this to 12.1, because some Exchange
// 2010 requires servers (v14.1) require 14.1 clients to send several
// optional fields (yep you read that right). Setting this to 12.1
// makes your life a wee bit easier.

MS-ASProtocolVersion: 12.1

// English language
Accept-Language: en-US

// Hostname the request is being sent to
Host: mail.example.com

// Set the PolicyKey to zero indicating to the server
// That you do not currently have a policy key
X-MS-PolicyKey: 0

Body

// Send the following XML string to the server.
// Note that the XML below has to be encoded
// into WBXML before sending it across to the
// Exchange server. More on this in a bit.
// The policyType should be set to
// MS-EAS-Provisioning-WBXML if the activeSync version greater
/ than or equal to 12.0, MS-WAP-Provisioning-XML otherwise

[code lang=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision:">
<Policies>
<Policy>
<!– If ActiveSync Version >= 12.0 –>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<!– If ActiveSync Version < 12.0 –>
<PolicyType>MS-WAP-Provisioning-XML</PolicyType>
</Policy>
</Policies>
</Provision>[/code]

Now the XML in the body above has to be encoded into WBXML, before it can be sent across to the server. This is true for all ActiveSync commands that have a body. I wrote a Java implementation of a WBXML encoder and decoder for my app, and you can find the source code here. The source code is based on a WBXML parser I found in the k9mail. I modified the source to add support for International languages.

Step 2: Server responds to client request with security policy and temporary policyKey

Once the request above is sent to the server, you should get a response that looks somewhat like this. Again the XML returned from the server is encoded in WBXML. It is the client’s responsibility to decode this WBXML and make sense of it.

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1069
Content-Type: application/vnd.ms-sync.wbxml
Server: Microsoft-IIS/7.5
MS-Server-ActiveSync: 8.3
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Cache-Control: private
Date: Mon, 01 Feb 2011 21:14:17 GMT

[code lang=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision">
<Status>1</Status>
<Policies>
<Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<Status>1</Status>
<PolicyKey>2152355410</PolicyKey>
<Data>
<EASProvisionDoc>
<DevicePasswordEnabled>0</DevicePasswordEnabled>
<AlphanumericDevicePasswordRequired>0</AlphanumericDevicePasswordRequired>
<PasswordRecoveryEnabled>0</PasswordRecoveryEnabled>
<DeviceEncryptionEnabled>0</DeviceEncryptionEnabled>
<AttachmentsEnabled>1</AttachmentsEnabled>
<MinDevicePasswordLength/>
<MaxInactivityTimeDeviceLock/>
<MaxDevicePasswordFailedAttempts/>
<MaxAttachmentSize/>
<AllowSimpleDevicePassword>0</AllowSimpleDevicePassword>
<DevicePasswordExpiration/>
<DevicePasswordHistory>0</DevicePasswordHistory>
<AllowStorageCard>1</AllowStorageCard>
<AllowCamera>1</AllowCamera>
<RequireDeviceEncryption>0</RequireDeviceEncryption>
<AllowUnsignedApplications>1</AllowUnsignedApplications>
<AllowUnsignedInstallationPackages>1</AllowUnsignedInstallationPackages>
<MinDevicePasswordComplexCharacters>3</MinDevicePasswordComplexCharacters>
<AllowWiFi>1</AllowWiFi>
<AllowTextMessaging>1</AllowTextMessaging>
<AllowPOPIMAPEmail>1</AllowPOPIMAPEmail>
<AllowBluetooth>2</AllowBluetooth>
<AllowIrDA>1</AllowIrDA>
<RequireManualSyncWhenRoaming>1</RequireManualSyncWhenRoaming>
<AllowDesktopSync>1</AllowDesktopSync>
<MaxCalendarAgeFilter>0</MaxCalendarAgeFilter>
<AllowHTMLEmail>1</AllowHTMLEmail>
<MaxEmailAgeFilter>0</MaxEmailAgeFilter>
<MaxEmailBodyTruncationSize>-1</MaxEmailBodyTruncationSize>
<MaxEmailHTMLBodyTruncationSize>-1</MaxEmailHTMLBodyTruncationSize>
<RequireSignedSMIMEMessages>0</RequireSignedSMIMEMessages>
<RequireEncryptedSMIMEMessages>0</RequireEncryptedSMIMEMessages>
<RequireSignedSMIMEAlgorithm>0</RequireSignedSMIMEAlgorithm>
<RequireEncryptionSMIMEAlgorithm>0</RequireEncryptionSMIMEAlgorithm>
<AllowSMIMEEncryptionAlgorithmNegotiation>2</AllowSMIMEEncryptionAlgorithmNegotiation>
<AllowSMIMESoftCerts>1</AllowSMIMESoftCerts>
<AllowBrowser>1</AllowBrowser>
<AllowConsumerEmail>1</AllowConsumerEmail>
<AllowRemoteDesktop>1</AllowRemoteDesktop>
<AllowInternetSharing>1</AllowInternetSharing>
<UnapprovedInROMApplicationList/>
<ApprovedApplicationList/>
</EASProvisionDoc>
</Data>
</Policy>
</Policies>
</Provision>
[/code]

The 200 response code indicates a success. The XML returned from the server contains the security policy that the server would like the client to implement. Once the client receives this security policy, it needs to implement the security policy indicated in the XML, and then acknowledge receipt of the security policy. Note that the policyKey (2152355410) returned by the server in the response is temporary and needs to be sent to the server in Step 3 (acknowledgement).

Step 3: Client acknowledges receipt and application of security policy

The client should now acknowledge the security policy and its application by using the temporary policyKey obtained in Step 2.

POST https://mail.example.com/Microsoft-Server-ActiveSync?User=username&DeviceId=123412341234&DeviceType=Android&Cmd=Provision HTTP/1.1
User-Agent: Android
Authorization: Basic RE9NQUlOXFVTRVJOQU1FOlBBU1NXT1JE
Content-Length: FIXME
Content-Type: application/vnd.ms-sync.wbxml
MS-ASProtocolVersion: 12.1
Accept-Language: en-US
Host: mail.example.com
X-MS-PolicyKey: 2152355410

[code lang=”xml”]<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision:">
<Policies>
<Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<PolicyKey>2152355410</PolicyKey>
<Status>1</Status>
</Policy>
</Policies>
</Provision>
[/code]

Step 4: Server responds with final policyKey

At this point the server will respond with the “final” policyKey which the client then uses in the X-MS-PolicyKey header of all successive command requests to the server.

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 1069
Content-Type: application/vnd.ms-sync.wbxml
Server: Microsoft-IIS/7.5
MS-Server-ActiveSync: 8.3
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Cache-Control: private
Date: Mon, 01 Feb 2011 21:15:17 GMT

[code lang=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<Provision xmlns="Provision:">
<Status>1</Status>
<Policies>
<Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
<Status>1</Status>
<PolicyKey>12432432244</PolicyKey>
</Policy>
</Policies>
</Provision>
[/code]

A word of caution here. I have seen cases where a server processes a few client requests that contain the final policyKey obtained using the method above, and then start sending down a 449 or a 142 error code. This implies that the client needs to acknowledge the policy settings again. So make sure you always check for 142 and 449 error codes even if you have accepted the policy settings once and received a final policyKey.

An Activesync Primer – The Options request

Now that my app is out in the market (finally), I think it is time to write a brief how-to on ActiveSync and how I managed to get things working.

First off, I would like to acknowledge the help and support from the following individuals / websites, without  whom this app would not been possible

Ok lets get started.

The Activesync OPTIONS Command

This is the first command that you want to send to an ActiveSync server. If the message is correctly formatted, the server responds with with a list of commands supported, and also the ActiveSync version that is supported. Here is a sample request (RAW view from Fiddler). Please note that the text in bold will need to be modified, and actual values will need to be inserted. See comments prefixed by // for a better description of each field.

[EDIT 01/20/2011] Turns out if you are connecting to Google Apps (that runs on Linux) or an Exchange server that is protected by a reverse proxy, case matters. so the Activesync text below should be ActiveSync (note the camelcase)
// Send an Option request to the mail server. Append the
// following location to the URL Microsoft-Server-ActiveSsync
// and add the username used to log into the server.
OPTIONS https:/ /mail.example.com/Microsoft-Server-ActiveSsync?User=username HTTP/1.1

// Set this to anything. Since I wrote an Android app,
// I set this to Android
User-Agent: Android

// This is the text Basic followed by a
// base64 encode of the string DOMAIN\USERNAME:PASSWORD.
// Replace with appropriate values.
Authorization: Basic RE9NQUlOXFVTRVJOQU1FOlBBU1NXT1JE

[EDIT 01/20/2011] Exchange server 2010 does not like it if you send any additional headers. The only required headers are the auth and the user-agent. Hence none of the headers below are required for the OPTIONS request
// Since this is an Options request, there is no content
// just the headers. Hence this is set to zero
Content-Length: 0

// Indicate to the Exchange server that we are sending
// WBXML encoded content.
// Right now since is not really applicable.
Content-Type: application/vnd.ms-sync.wbxml

// Identifies the protocol version the client (we) support
MS-ASProtocolVersion: 12.1

// English language
Accept-Language: en-US

// Hostname the request is being sent to
Host: mail.example.com

If all goes well, you should get a response that looks somewhat like this

HTTP/1.1 200 OK
Cache-Control: private
Allow: OPTIONS,POST
Content-Length: 0
Server: Microsoft-IIS/7.5
MS-Server-ActiveSync: 8.3
MS-ASProtocolVersions: 1.0,2.0,2.1,2.5,12.0,12.1
MS-ASProtocolCommands: Sync, SendMail, SmartForward, SmartReply, GetAttachment, GetHierarchy, CreateCollection, DeleteCollection, MoveCollection, FolderSync, FolderCreate, FolderDelete, FolderUpdate, MoveItems, GetItemEstimate, MeetingResponse, Search, Settings, Ping, ItemOperations, Provision, ResolveRecipients, ValidateCert
Public: OPTIONS,POST
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Mon, 06 Dec 2010 21:14:17 GMT

The 200 response code indicates a success. The MS-ASProtocolVersions field indicates the version of the Activesync Protocol that is supported by the server. The MS-ASProtocolCommands field indicates what Activesync commands are supported by the server.

I will keep updating my blog with more information on how to use Activesync to query the GAL. Stay tuned.

Corporate Addressbook – now available on the Android market

My first Android app (Corporate Addressbook) is now available on the Android market.

I wrote this app since my Nexus One lacks a feature that I really needed – a true GAL lookup. The Global Address List (GAL) lookup currently available in vanilla Android (Froyo running on my Nexus One) is quite limited in functionality. It only looks up email addresses and does not return anything more. This is quite frustrating when you are trying to look up the address book for the contact information of a colleague. The only other option was to buy Touchdown. Touchdown is a great app, however it does much more than GAL lookups. It also fetches email and does a bunch of other things that are now included in Froyo. Hence I thought it was better to write this application to meet my requirement.

This application looks up the GAL and returns ALL data that is available on the Exchange server for the query. The app uses the ActiveSync protocol to communicate with the Exchange server, and supports both Exchange 2003 and 2007.

The source for this app is available on Google Code, and is available via the Apache 2.0 license.

Download it while it is hot 🙂

Appbrain
Android Market

Please do let me know if you come across and bugs / issues or if you have any comments.

Exception #11 on Amazon MP3 Android App

If you see this error when using the Amazon MP3 Android app to download songs, don’t bother calling Amazon Customer Care (like I did). I was told that it was a problem with my ISP (Yeah, you heard me right). Any way you will see this error if you do not 1-Click ordering enabled on your Amazon account. Follow the instructions here to turn on -Click ordering, and you should be good to go.

Printing on NSLU2

Using a Linksys NSLU2, I’ve got it unslung using v6.10.

Originally I got the NSLU2 because it seemed like a good, cheap way of getting a big networked data drive on my home network.
I quickly realized that the factory firmware from Linux was rather limited.

You get A LOT more functionality when you use Unslung.

The real bonus was that I was also going to get a home print server AND a more powerful shared network drive.

I started by carefully following the instructions to get the NSLU2 working with Unslung.
I chose to use a 250GB external USB drive connected to Port 1 for my Unslung install.
Once Unslung was installed, I was able to use the drive as an networked storage device.

The next step was to install CUPS and get the NSLU2 working as a print server.
I have an HP Photosmart 7350 that I wanted to use with the NSLU2.
All the instructions to add CUPS are available here
.

The actual CUPS installation when pretty smoothly – according to plan.

However, I got stuck once CUPS was installed, trying to get the printer to do anything.
I connected the usb cable to port 2. The port 2 light on the NSLU2 never lights up.
The CUPS instructions say that you should “just try printing” once you get CUPS installed.
I did this by trying to click the “Print Test Page” button for the default hp990c printer.
I got garbage out on my page.

DON’T TRY CHANGING ANY SETTINGS ON THE NSLU2 AT THIS POINT.

  • I tried permissions changes, more software installs on the NSLU2… totally wrong approach.
  • Just leave the base CUPS install and config alone.

The hp990c should be setup as a RAW printer in CUPS on the NSLU2 at this point. If it isn’t (because you messed with it) – click “modifiy printer” and change the driver to RAW.

From either a Mac or Windows machine on your network you now setup an Internet Printer.
Windows

  1. Add Printer
  2. Networked Printer
  3. URL – the address should be the entire link to the CUPS address on your NSLU2
    • http://192.168.0.110:631/printers/hp990c
    • make sure to include the “http://”
    • also make sure to include the port “:631”
  4. When prompted, select the print driver that matches your printer
    • In my case for the Photosmart 7350 I had to click “Have Disk”
    • This also required having ALREADY INSTALLED the printer as a local usb connected printer so that the drivers were installed by the HP software in the directory c:\program files\HP Photosmart 11\enu\drivers
  5. Finish the Add Printer install and give it a try by printing a test page!

Mac

  1. DO NOT USE THE SYSTEM PREFERENCES PANEL TO DO THIS
  2. Open a web browser and go to http://localhost:631/admin
    • This is the local admin control panel for the CUPS system that is built in to your Mac
  3. Use the Add Printer wizard
  4. Install using Internet Printing (HTTP)
  5. when you’re asked for the printer location, use the entire CUPS address on your NSLU2
    • http://192.168.0.110:631/printers/hp990c
    • make sure to include the “http://”
    • also make sure to include the port “:631”
  6. Finish the wizard and give it a try by clicking “print test page”

Querying the Global Address List (GAL) via Exchange Web Services (EWS)

If you are looking for a way to search the Global Address List (GAL) for a particular contact via Exchange Web Services, read on. Please note that this only works for Exchange 2007 and above.

  1. Grab the WSDL and generate stubs. The normal location for the WSDL is
    [sourcecode language=”html”]http://Your_exchange_server_name/EWS/Services.wsdl.[/sourcecode]

    Note that when you plug this URL into your browser, you might be directed to login. Once you login with your credentials, you will be redirected to your regular email page. Despair not. Enter the URL in your address bar again and you should have the WSDL.

  2. Use the code below to query for a user with a name “test”. The code below is in C#, but should translate quite easily to other languages. Also take note of the URL. In my case, I needed to use https. Also note that the username is not the email but just the prefix i.e. the @domain.com is not required.
    [sourcecode language=”csharp”]
    static void Main(string[] args)
    {
    ExchangeServiceBinding esb = new ExchangeServiceBinding();
    esb.Url = @"https://myserver/EWS/Exchange.asmx";
    esb.Credentials = new NetworkCredential(
    "username",
    @"password",
    @"domain");

    // Create the ResolveNamesType and set
    // the unresolved entry.
    ResolveNamesType rnType = new ResolveNamesType();
    rnType.ReturnFullContactData = true;
    rnType.UnresolvedEntry = "test";

    // Resolve names.
    ResolveNamesResponseType resolveNamesResponse
    = esb.ResolveNames(rnType);
    ArrayOfResponseMessagesType responses
    = resolveNamesResponse.ResponseMessages;

    // Check the result.
    if (responses.Items.Length > 0 &&
    responses.Items[0].ResponseClass
    != ResponseClassType.Error)
    {
    ResolveNamesResponseMessageType responseMessage =
    responses.Items[0] as
    ResolveNamesResponseMessageType;

    // Display the resolution information.
    ResolutionType[] resolutions =
    responseMessage.ResolutionSet.Resolution;
    foreach (ResolutionType resolution
    in resolutions)
    {
    Console.WriteLine(
    "Name: " +
    resolution.Contact.DisplayName
    );
    Console.WriteLine(
    "EmailAddress: " +
    resolution.Mailbox.EmailAddress
    );

    if (resolution.Contact.PhoneNumbers != null)
    {
    foreach (
    PhoneNumberDictionaryEntryType phone
    in resolution.Contact.PhoneNumbers)
    {
    Console.WriteLine(
    phone.Key.ToString() +
    " : " +
    phone.Value
    );
    }
    }
    Console.WriteLine(
    "Office location:" +
    resolution.Contact.OfficeLocation
    );
    Console.WriteLine("\n");
    }
    }
    }
    [/sourcecode]

Aphorisms

An aphorism (literally distinction or definition, from Greek αφοριζειν “to define”) expresses a general truth in a pithy sentence. It is a short, pointed sentence expressing a wise, clever observation, a general truth or adage.

I got these in my email today and a they seemed to make a lot of sense (Thanks Papa). Enjoy

  • The nicest thing about the future is that it always starts tomorrow.
  • Money will buy a fine dog but only kindness will make him wag his tail.
  • If you don’t have a sense of humor, you probably don’t have any sense at all.
  • Seat belts are not as confining as wheelchairs.
  • A good time to keep your mouth shut is when you’re in deep water.
  • How come it takes so little time for a child who is afraid of the dark to become a teenager who wants to stay out all night?
  • Business conventions are important because they demonstrate how many people a company can operate without.
  • Why is it that at class reunions you feel younger than everyone else looks ?
  • Scratch a dog and you’ll find a permanent job.
  • No one has more driving ambition than the boy who wants to buy a car.
  • There are no new sins; the old ones just get more publicity.
  • There are worse things than getting a call for a wrong number at 4 AM. Like this: It could be a right number.
  • No one ever says “It’s only a game” when their team is winning.
  • I’ve reached the age where the happy hour is a nap.
  • Be careful reading the fine print. There’s no way you’re going to like it.
  • The trouble with bucket seats is that not everybody has the same size bucket.
  • Do you realize that in about 40 years we’ll have millions of old ladies running around with tattoos? (And rap music will be the Golden Oldies!)
  • Money can’t buy happiness – but somehow it’s more comfortable to cry in a Corvette than in a Yugo.
  • After 70 if you don’t wake up aching in every joint, you are probably dead.

Slingbox setup

What model of the slingbox should I buy?
Though Slingmedia does not advertise this, the Slingbox Pro supports both NTSC and PAL signals. So this means that you can buy a Slingbox Pro in the US and use it unmodified in India.

Wireless router
Another thing you will need is a wired or a wireless router. Since it is quite difficult to buy a wired router these days, I would recommend you buy a wireless router. Any router with multiple ethernet ports will do. This router will help you share your internet connection to multiple hosts.

Cable or satellite?
I assume that you plan to place shift a cable TV and not a satellite signal. The advantage of cable TV (cable over coax) over satellite (e.g. Tata Sky) is that the slingbox’s internal tuner can tune into the desired channel. This means that you could be watching a particular channel remotely while a completely different channel could be playing on your TV in India.

Connectors
In India the default connector used in TV’s seems to be the Belling-Lee connector. The slingbox however accepts input via an F-connector. You can easily find converters that go from PAL to F. However this is something you need to keep in mind.

Network setup
The slingbox itself is pretty easy to setup and the instructions provided by Slingmedia are excellent. Their tool does a great job of detecting network settings on routers (it supports almost all major routers) and modifying the firewall settings via UPnP to enable remote viewing. However in my case my ISP had installed a DSL modem which also had a firewall. This modem did not come with a user manual and it took me a couple of hours to configure its firewall.

More help
There seems to be a pretty big community setting up Slingbox’s in India. Go here for more information.