BigIP F5 as Reverse Proxy for Lync Server, Windows 8 and Lync Metro App

I worked on interesting case today. A large enterprise customer began testing the new Windows 8 OS (desktop and Surface) and received numerous reports for sign-in issues with Lync Metro App. The client will spin for a while and go back to login screen. This seemed strange, since Lync 2010 and 2013 desktop client would work just fine, as well as Lync 2010 Mobile. In this post I will not only explain the problem and provide solution, but also share my method of troubleshooting and some techniques as well.

My approach to every problem is half academic, half technical – in this precise order. I don’t know how you guys work, but my brain retains information relevant to specific case only for relatively short period of time, or to be precise, until I move to the next case and so, for this task, I went back to my notes and refreshed the knowledge as of how Lync Autodiscover works. I decided to concentrate on Autodiscover process because Lync Metro App does not honor SRV records. Instead, it relies on Autodiscover to receive sign-in FQDN and/or URL.

1. Client will try and resolve either lyncdiscoverinternal.contoso.com or lyncdiscover.contoso.com
2. Client will request https://lyncdiscoverinternal.contoso.com
3. Autodiscover will send back the Autodiscover service URL (typically Director Web Services)
4. Client will authenticate and web ticket.
5. Client will make new request while submitting the web ticket so that the Director can retrieve the user’s home pool information.
6. Director will redirect the client to the home pool Web Services.
7. Client will authenticate again and submit the web ticket
8. Home pool web services (via Autodiscover) will respond with internal and external Lync services for the user’s home pool.
9. Client (Lync Metro App in this case) will sign to the Registrar or the Edge server just like desktop client would after discover the registrar via SR record.

All right, how that we know how the process works, next step is to identify at which step Lync Metro App fails. I could only use Wireshark for this troubleshooting session. It is possible to use WS to decrypt SSL traffic indeed if… you have the server private key, which I did not in this case. So, the logical step was to just capture sign-in attempt and see if there is something I can hang on.

The first logical step was to examine the certificate returned from the server. I know, I know – all other service are working at the moment but… just in case. There are many ways to examine server certificate:

By using Internet Explorer. Simply visit the web site by using https protocol and review the certificate.

***While this is certificate from my lab, the process used was the same. Earlier today I verified that the required names (lyncdiscover.contoso.com, director_web.contosof.com and home_pool.contoso.com) were present. So, it is not certificate issue!

Alternatively, you can use tool called RUCT (Remote UC Troubleshooting Tool) http://www.insideocs.com/Tools/RUCT/RUCT.htm. Simply start it, go to Certificate Information tab, input the target URL and port, retrieve the certificate and examine it.

Because the issue was Metro App which runs on Windows 8 OS only, my next test will be from W8 machine. I opened IE10 and went to https://lyncdiscoverinternal.customer.com

I received the expected output and when opend it with notepad, everything looked just fine.

So far I’ve established that DNS resolves to correct IP, the certificate presented from the server have the correct names in the SAN list and the web service is functional.

This is the place to mention the fact that this customer uses F5 Hardware Load Balancer not only as HLB but Reverse Proxy for the External Lync web services. I checked with the network team earlier today and found that the current fimware on F5 is version 10.2

With this in mind, my next stop was… you guessed – F5 compatibility matrix (http://support.f5.com/kb/en-us/products/big-ip_apm/manuals/product/apm-clientcompmatrix-11-1-0.html)

Ouch! As one of my favorite standup comedians Russel Peters often say: "Somebody gonna get a hurt real bad".

I was really not sure what to make of this statement “Windows Phone versions 7 and 8, and Windows RT are not supported”. Any way, as stubborn as I am, next logical step was to "wireshak it" and see what’s up.

I see the normal “Client Hello” followed by “Server Hello”, “Certificate , “Server Hello Done” and “Change Cipher, Encrypted Handshake Message”. Absolutely normal handshake, nothing out of ordinary… handshake done via TLSv1. No RST (reset) i.e both client and server are “talking”. Since I don’t have the private key, I cannot decrypt the traffic and so, for now I will assume the client visited lyncdiscover URL.

I see second handshake in the sequence described above. This must be the authentication attempt. Remember, we must authenticate first in order to receive web ticket.

Oops. This is something new. Note that the TLS version the client requested is v1.2

The server responded with Server Hello TLS v1.2 ...

…but then… nothing. The client dropped the connection.

I already knew from my research that F5 at firmware 10.2 does not support Windows 8 (not that it is clear what exactly is not supported…). I also know that TLS v1.0 works fine (when I visit https://lyncdiscover.contoso.com Internet Explorer 10 does not bombs and so, my conclusion was – this has something to do with the interpretation of TLS v1.2 on both sides.

At this point, just for fun, I decided to disable TLS v1.2 on Windows OS level, which I hoped would force Lync Metro App to use TLS v1.0 and examine the SSL traffic again.

Protocols can be enabled or disabled via registry. To make your life easier, here is link to .reg file which will disable TLS v1.2 on your Windows 8 powered device.

After modifying the registry and reboot, Lync Metro App signed immediately. Wireshark trace confirmed the app used TLS v1.0 to communicate with the Web Services via F5 Reverse Proxy and signed successfully. Download, unzip, move the file to your Windows 8, Merge and reboot.

Simply said, we introduce two new registry keys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server

To revert to the default security settings when such "fix" will be no longer necessary, delete "TLS 1.2" key and reboot.

By no means I encourage you to reduce the level of security on your operation system. This “fix” is proposed with the sole purpose that you as IT professional can continue testing all features of Windows 8 and/or Lync Server until such “fix” will be no longer necessary.