I have just solved one interesting case. The customer has a great powerful RDP session-based application farm based on Windows 2012 R2. The farm runs several session collections with RemoteApps. The RDP RemoteApps are published through RDWeb and connected over RD Gateway when access from the internet. Everything is using trusted TLS/SSL certificates bought from a public CA such as GlobalSign or Symantec etc.
Everything worked smoothly and fast except for the application startup time when accessed from the internet. RDWeb itself was fast enough. But once you clicked an application it sometimes took even three minutes to start the application. After that, smooth play, no delays anymore. Apparently the RD Gateway was the problem because this didn't happen from LAN when you avoided the RD gateway, at least not that severely.
Digging deeper into the problem, both the RD Gateway and the RD Connection Broker were both had some of their own part in the problem.
The reason was certificate revocation checking which timed-out
The reason identified itself when I enabled auditing for windows firewall connections (the Filtering Platform Connection audit subcategory) and compared and correlated it with the events in the TerminalServices-Gateway/Operational event log. The Security event log showed repeated and frequent TCP connections to remote port 80 (HTTP apparently) started by system processes such as lsass.exe or svchost.exe. Weirdly the connections were going to public internet IP addresses.
And yes, when I checked the TLS/SSL certificate CRL paths and their URLs, these IP addresses showed to be the CRL distribution points of the public CAs which issued their RDP certificates.
Ok, it seemed like the system was trying to verify certificate revocation of its own server certificates (why the hell?) by downloading their respective CRL files. The problem was the farm servers didn't have internet access actually. So all the connections were only starting (SYN-SENT), then each was timing-out for 21 seconds (as is the standard TCP connection establishment timeout) and failed. And again and again with every client connection being established.
As it appears, the RDP gateway and sometimes even the RD connection broker servers are trying to verify its own server certificates revocation status by downloading CRLs.
The solution could be either to allow the servers download the public CRL files from internet over HTTP TCP 80 (if you need, you can configure them to use an HTTP proxy server with netsh winhttp commnad).
Or to make sure that they either cannot resolve their public DNS names at all.
Or if the servers must be able to resolve the public DNS names, then make sure that the following TCP connection fails immediatelly, instead of waiting for the 21 seconds timeout.
If you have a network firewall in place, you must change the blocked port setting from stealth or drop to reject. Or you can configure an explicit blocking Outbound Rule in the servers' Windows Firewall. The outbound blocking rules are good in this regard as they prevent the blocked TCP connection immediatelly and do not let the applications time-out.