Office 365 Access to the registry key

[Office 365] Access to the registry key “HKEY_LOCAL_MACHINE\Software\Microsoft\MSOLCoExistence” is denied

If you receive the error when configuring the Microsoft Directory Synchronization Tool to communicate with Office 365:
“Access to the registry key ‘HKEY_LOCAL_MACHINE\Software\Microsoft\MSOLCoExistence’ is denied”

Make you you right click and run the tool as an administrator 

Microsoft Online Service Directory Synchronization Configuration Wizard in the Configuration step. The error

How To Install ADFS 2012 R2 For Office 365

How To Install ADFS 2012 R2 For Office 365

When discussing and reviewing Office 365 with customers, I wanted to have a series of posts to illustrate the steps involved when deploying Office 365.   In the burgeoning drafts folder ADFS was at the top, so that got finished first!

The act of deploying and configuring ADFS 2012 R2 for Office 365 will be broken down into three separate blog posts

1. Install ADFS (this post)
2. Install ADFS Proxy
3. Leverage ADFS with Office 365

Identity, Identity, Identity

The IT security landscape keeps evolving.  One of the recent changes is a move away from ACLs on files in the NTFS file system to an access control system that is based on claims.  Claims based authentication is an industry standard security protocol to authenticate users.  This is the underlying WS-* standards that describe the usage of Security Assertion Mark-up Language (SAML) tokens.  Claims based auth requires these tokens, and by extension an entity that can issue the token.  This is the Secure Token Service (STS).  The STS server can be based on Active Directory Federation Services (ADFS) or other platforms that provide this service.

ADFS lights up one of the three options for Office 365 identity management, which is option #3 in the below list:

1. Cloud Identity – users are created, and managed,  in Windows Azure Active Directory (WAAD).  No connection to any other directory.  This is the simplest model as there is no integration to any other directory.  Each user has an account created in the cloud which does not synchronise anywhere else.  Note that you will still typically need additional on-premises credentials to gain access to a local workstation and local resources.
2. Directory Synchronisation – Users are created and managed in the on-premises directory and get synchronised up to Office 365 so they can access Office 365 resources.  Typically this means running the DirSync appliance, or in some cases FIM with the Windows Azure Active Directory Connector.  The newer builds of DirSync allow for the user’s password hash to be synchronised up to Office 365.  Note this does not say clear text password.    This allows user’s to logon to Office 365 using the same credentials as on-premises with no additional infrastructure.
3. Federated Identity – Federation relies on directory synchronisation so that WAAD is populated.  When the authentication request is presented to Office 365, the service will then contact the on-premises ADFS infrastructure so that AD is responsible for authenticating the request.

ADFS is the primary choice for customers who want to use federated identities with Office 365.  In addition to this there are a variety of qualified third party identity providers that can be connected with Office 365 to provide the necessary plumbing for federation.  The shortcut URL aka.ms/SSOProviders  links to the ‘Works With Office 365’ Identity program, and lists the identity providers that have been qualified with Office 365.  Please read the notes on the TechNet pagewith regards to the testing and support aspects of these services.

Some customers will use these services as they do not wish to invest in a fault tolerant and geographically dispersed ADFS implementation.  The availability of ADFS is a key discussion point when discussing federation.  For whatever reason if the ADFS infrastructure is unavailable, then Office 365 cannot complete the authentication process and thus users cannot get access to Office 365.

In addition since DirSync now replicates the user’s hashed password to WAAD, some customers now use DirSync to provide Same Sign On / Single Sign On (SSO).  DirSync version 1.0.6385.12, which was released in May 2013, and latter builds provide the ability to synchronise passwords.  DirSync can be downloaded here, and the TechNet Wiki has details on the release history.   When running the configuration wizard with this release you will get the shiny “Password Synchronization” window:

This is worthwhile to mention as there is still a perception that ADFS is a hard requirement to get SSO.  That is soooooooooooo  Q1 2013!

Anyway, I digress let’s get back to ADFS…..

We shall look at installing ADFS 2012 R2 since there are numerous compelling features in this release!

What’s New And Improved In ADFS 2012 R2

The quick answer is a lot!  Some examples include:

• IIS dependency removed
• Single server installation option removed and now have single farm install (recommended to install a farm always in prior release anyway)
• Separate ADFS proxy role removed.  ADFS proxy now based off Web Application Proxy (WAP), and is used to publish the ADFS server to the Internet.  WAP can publish many other applications, not just ADFS.
• ADFS extranet lockout – ADDS account lockout protection on the ADFS proxy
• Access control based on network location to control user authentication to ADFS

There are many others, but check here for them since we are focussing on Office 365 usage for ADFS.

Note that you will not see me  call this release ADFS 3.0.  Its full and proper name is  ADFS 2012 R2.  for reference here are the older versions and what some folks call them:

ADFS Build	Notes
ADFS 1.0	Released with Windows 2003 R2.  Built into OS.
ADFS 1.1	Released with Windows 2008 and 2008 R2.  Built into OS.
ADFS 2.0	Released After Windows 2008 / 2008 R2.  Separate download from here.
ADFS 2.1	Windows 2012
ADFS 3.0	Windows 2012 R2

Update 5-5-2014:    Please also see this post on exploring ADFS 2012 R2 Extranet Lockout protection.

Update 29-5-2014:  Please also review update 2948086 Update that improves AD FS proxy and STS reliability in Windows Server 2012 R2 when multiple clients sign in.

Update 9-9-2014:    For the other posts on ADFS, please view this tag cloud.

Planning And Prerequisites, And Other Fun Details

 

Prerequisites

The prerequisites are listed on TechNet.  Of course before jumping into the install the installation needs to be planned.

ADFS Role Planning

The ADFS role should be deployed within the corporate network, and not in the DMZ.  The ADFS proxy role is intended to be installed into the DMZ.

The default topology for Active Directory Federation Services (AD FS) is a federation server farm, using the Windows Internal Database (WID), that consists of up to five federation servers hosting your organization’s Federation Service. In this topology, AD FS uses WID as the store for the AD FS configuration database for all federation servers that are joined to that farm. The farm replicates and maintains the Federation Service data in the configuration database across each server in the farm.

Since the availability of Office 365 relies upon the availability of ADFS when the domain is federated there is a strong recommendation to have at least two ADFS servers with a redundant ADFS proxy infrastructure.

Please review the design guidance on TechNet.

ADFS Service Account

We can now use a standard service account or a Group Managed Service Account in ADFS 2012 R2.

In this case since the KDS root key was not configured, lets leverage a standard service account.

The installation process should set the required Service Principal Names (SPN) on the account.

ADFS Namespace

Select what name you are to use to access ADFS.  Typically this is along the lines of:

sts.wingtiptoys.ca

adfs.tailspintoys.ca

Note that this is the namespace for the ADFS service.  Since we will be using Kerberos to access ADFS internally, there must be a Service Principle Name (SPN) registered for this name.  This will be associated to the service account, and since SPNs operate in the “Highlander – there can be only one!” mode you do not want to duplicate the SPN on the ADFS server by naming the computer the same as the ADFS namespace.

You also want to discuss what display name should be chosen, as this will be visible to users.

Certificates

Since ADFS leverages SSL, we need to have a SSL certificate.  You could try three options, but only one will work:

1. Self-signed certificate
2. Certificate issued from internal PKI
3. Certificate from 3rd party public CA

Office 365 needs to see a valid Service Communication Certificate on your ADFS infrastructure, so you are going to have to buy a certificate from a public CA.  Office 365 will not trust a service communication certificate that is either self-signed or from your internal CA, which results in tears.  We can use self-signed certificates for the Token Decrypting and Token Signing Certificate.  These are separate from the service communication cert.

Please follow the documentation from your chosen CA to request, install and complete the certificate.  The steps required vary from vendor to vendor and also over time.  Make sure you are not missing any updated intermediate certificates!  How would you know?  Follow their  process!!

For the purposes of this post we shall deploy the initial ADFS server, and in the future add another ADFS server for redundancy.

Installing ADFS On Windows Server 2012 R2

After starting up server manager’s add roles and features wizard, select Active Directory Federation Services, then click next.

We don’t need to add any additional features.  Remember that the IIS dependency was removed in ADFS 2012 R2.

Clicking next takes us to the ADFS splash screen.  Note that it helpfully tells us that the specific ADFS proxy role has been removed in Windows 2012 R2 and how to go about installing it.  Shame I missed that the very first time  I ran this, and could not find the old school ADFS Proxy role…

Clicking next will then install the necessary bits.

Bits are being shuffled around…

Shuffling has been completed, and the installation is complete.   You can launch the ADFS configuration wizard from here, or alternatively if this window is closed it can be launched from server manager.

Before starting the ADFS configuration wizard I already installed my 3rd party certificate and tested that is was correctly installed.

Additionally a service account called ADFS-Service  was also pre-created.

The wizard also states that you must have access to Domain Admin (DA) credentials!

Note that you are only given an option to either make a new ADFS farm or add this box to an existing farm.  This saves the painful issue from older ADFS builds, where ADFS was not installed into a farm you were then unable to easily the add the second ADFS server for redundancy.

Provide your domain admin credentials.

We need to select the SSL certificate that we will use and also provide the ADFS name we selected in the design process.

In this case the name is adfs.tailspintoys.ca   -- note that there is no concept of an InternalURL or ExternalURL for the ADFS namespace.  Clients will use the same name on the intranet and internet to locate ADFS.  Thus split DNS will make life simple!

Provide your chosen display name, and click next.

As mentioned earlier it is possible to use a GMSA as the ADFS service account.  GMSA will automatically update the service account’s credentials and administrators will also be oblivious as to its password.

In this case a standard service account was used.

Select the database configuration as per the design.

The Tailspintoys corporation will use WID.

Review the options, and when happy pull the trigger!

For reference the PowerShell script is shown here:

#
# Windows PowerShell script for AD FS Deployment
#

Import-Module ADFS

# Get the credential used for the federation service account
$serviceAccountCredential = Get-Credential -Message "Enter the credential for the Federation Service Account."

Install-AdfsFarm `
-CertificateThumbprint:"5804746A7980C8682FBF408D48EF6C3B02A5ZORG" `
-FederationServiceDisplayName:"Tailspintoys STS" `
-FederationServiceName:"adfs.Tailspintoys.ca" `
-ServiceAccountCredential:$serviceAccountCredential

The ADFS pre-requisite checks are done, and we can proceed to the configuration:

One coffee later, we have a shiny new ADFS server – whoo!!

We are not quite done yet, and there a couple of additional things to do!

Next Steps

ADFS Update(s)

Update 29-5-2014:  Please also review update 2948086 Update that improves AD FS proxy and STS reliability in Windows Server 2012 R2 when multiple clients sign in.

Update 11-12-2014:  The above update 2948086  is now bundled in this rollup: May 2014 update rollup for Windows RT 8.1, Windows 8.1, and Windows Server 2012 R2

Update 16-7-2014:  Other updates you may want to review are at the bottom of this post.

When multiple clients (over 200 clients) try to sign in by using an Active Directory Federation Services (AD FS) proxy, the AD FS proxy consumes 100% usage of the CPU. In this situation, the AD FS proxy performance is slow, and causes a delay that exceeds 10seconds. This also causes STS to work under minimal load. Therefore, STS rejects the requests or serves only 5 to 10 requests per second.

 

DNS A Record

We must create the DNS record  for the ADFS instance.  This maps to the ADFS namespace that we previously planned.  Create this A record in your internal DNS infrastructure.
Once the DNS record has been created an propagated ensure that it resolves correctly.
One thing to mention here, if you create a CNAME and point that to the server hosting ADFS chances are that you will run into a never ending authentication prompt situation.
In the below example the ADFS namespace is called adfs.tailspintoys.ca and a CNAME was used to direct traffic to the ADFS server called tail-ca-sts.tailspintoys.ca.    This will likely cause the client to obtain a Kerberos ticket for the incorrect name.

The easiest way to stop this is to use  a regular A record, like so:

There is also an option contained in KB 911149  that some folks have mentioned.

Additional Steps

This topic covers additional steps to configure AD FS after you install the first federation server, including:

• Opening the ADFS Management Snap-in
• Configuring Name Resolution for AD FS Services
• Adding nodes to the farm
• Adding a Web Application Proxy
• Enabling Device Registration Service

For more information about how to deploy AD FS, see How to deploy AD FS in Windows Server 2012 R2.

Verify Federation Service Metadata

Open Internet Explorer and navigate to your ADFS server’s federation metadata URL.

This will be something like the below, just change the FQDN to match your environment.

https://adfs.tailspintoys.ca/federationmetadata/2007-06/federationmetadata.xml

https://sts.contoso.com/federationmetadata/2007-06/federationmetadata.xml

The result should show this:

Verify ADFS Sign-In Page

Browse to the ADFS sign-in page and test that you are able to authenticate.

The URL will be similar to the below, again change the FQDN to match your organisation’s.

https://adfs.tailspintoys.ca/adfs/ls/idpinitiatedsignon.htm

https://sts.contoso.com/adfs/ls/idpinitiatedsignon.htm

You should see the below, and be prompted to sign in:

Depending upon how IE is configured you will either be prompted to provide credentials or be automatically signed-in.

If you want to have users be automatically signed-in then configure your browser settings to trust the federation server role by adding your federation service name (for example, https://adfs.tailspintoys.ca) to the browser’s local intranet zone. This will enable seamless sign-in using Windows Integrated Authentication.

Once we are happy that the ADFS instance is functioning appropriately we can then move onto installing the ADFS proxy role.

This will be covered in a separate post, to prevent this one getting too long!

Office 365 Reporting web service and Windows PowerShell cmdlets

Reports and the corresponding cmdlets

Office 365 Reporting web service reference page	Office 365 reporting-related Windows PowerShell cmdlets
CsActiveUser* reports	Get-CsAVConferenceTimeReport
CsAVConferenceTime* reports	Get-CsActiveUserReport
CsConference* reports	Get-CsConferenceReport
CsP2PAVTime* reports	Get-CsP2PAVTimeReport
CsP2PSession* reports	Get-CsP2PSessionReport
ConnectionbyClientType* reports	Get-ConnectionByClientTypeReport
ConnectionbyClientTypeDetail* reports	Get-ConnectionByClientTypeDetailReport
GroupActivity* reports	Get-GroupActivityReport
MailboxActivity* reports	Get-MailboxActivityReport
MailboxUsage report	Get-MailboxUsageReport
MailboxUsageDetail report	Get-MailboxUsageDetailReport
MailDetail report	Get-MailDetailReport
MailDetailDlpPolicy report	Get-MailDetailDlpPolicyReport
MailDetailMalware report	Get-MailDetailMalwareReport
MailDetailSpam report	Get-MailDetailSpamReport
MailDetailTransportRule report	Get-MailDetailTransportRuleReport
MailFilterList report	Get-MailFilterListReport
MailTraffic report	Get-MailTrafficReport
MailTrafficPolicy report	Get-MailTrafficPolicyReport
MailTrafficSummary reports	Get-MailTrafficSummaryReport
MailTrafficTop report	Get-MailTrafficTopReport
MessageTrace report	Get-MessageTrace
MessageTraceDetail report	Get-MessageTraceDetail
MxRecordReport report	Get-MxRecordReport
OutboundConnectorReport report	Get-OutboundConnectorReport
ServiceDeliveryReport report	Get-ServiceDeliveryReport
StaleMailbox report	Get-StaleMailboxReport
StaleMailboxDetail report	Get-StaleMailboxDetailReport

Reporting Office 365

Microsoft has never equipped on-premises Exchange with good reporting facilities and administrators have had to create their own reports using whatever facilities were available to them or turn to third-party software products such as the long-livedPromodag reports.

The lack of a consistent API to access information about mailboxes and their contents was an issue for administrators and developers alike. That problem was finally addressed from Exchange 2007 onwards when Microsoft introduced PowerShell and Exchange Web Services (EWS). PowerShell is well suited for the quick retrieval of information about Exchange configuration and settings while EWS is better when more than fifty or so objects need to be processed. Many examples of how to use these PowerShell or EWS to extract and report Exchange data can be found on the web, including how to combine the two to good effect.

Other sources of data like messaging tracking logs have been mined to provide insight into how Exchange is used, but the reports found in the on-premises product have never been particularly useful and sometimes have been just plain weird, as in the insistence in using XML as the preferred output format. It still bemuses me that a product that’s been around for nearly 20 years doesn’t offer out-of-the-box basic reporting like mailbox sizes, mailboxes per database/server, used and unused distribution groups, public folder analysis (size, ownership, access), and so on. If Microsoft had added just a single useful report per version, Exchange 2013 would provide eight to administrators. Yet all it can offer are some ho-hum reports that don’t really help anyone. Crazy!

Office 365 is better in two very important respects. First, the Office 365 Administration Center includes a Reports section that offers a selection of reports covering different aspects of the service. However, the detail exposed in the reports is basic and seldom provides a real insight into how Exchange Online and the other Office 365 applications are used. Microsoft must have realized that the built-in reports are basic and therefore provide the Office 365 reporting web service to provide an interface to the Office 365 Reporting DataMart, a repository that consolidates data feeds from Exchange Online, Exchange Online Protection, Lync Online, Azure Active Directory, and other Office 365 sources.

A range of information about different aspects of Office 365 is extracted from the data sources on a regular basis. You can see the kind of information available in the DataMart by running the Office 365 PowerShell reporting cmdlets. For example, the Get-GroupActivityReport cmdlet lists the numbers of distribution groups created and deleted within a tenant on a day-by-day basis. One thing that’s obvious from an examination of the data returned from Office 365 is that the reporting service focuses on high-level aggregate data and doesn’t dive down into the detail that is often necessary to make good management decisions. In addition, the reporting service returns data with a limited horizon of between 14 and 90 days, so if you want to use the information retrieved from Office 365 for long-term analysis some arrangement usually has to be made to store the data in another repository.

PowerShell is a fantastic way to automate common administrative activities or to process a group of objects at one time, but no one would ever claim that PowerShell is the most efficient or effective way to generate reports. In addition, PowerShell is slow when it comes to processing large collections of objects, so using a script to create a report about mailbox sizes is acceptable when you have 100 mailboxes to process, but not such a great idea when 10,000 mailboxes are involved. Not only would the script be slow, its execution would create a strain on Office 365 resources that Microsoft is likely keen to avoid.

To complement the capabilities of PowerShell, the Office 365 reporting service supports full REST programmatic access to the contents of the Reporting DataMart. The intention is that third-party software developers can incorporate Office 365 reporting data into their products to provide Office 365 customers with a much better view of what’s actually happening inside a tenant than is found in the basic reports. Apart from the obvious goodness inherent in having a common repository to consult to retrieve operating data about Office 365, the combination of Reporting DataMart and APIs avoids people coming up with their own methods to extract information from Office 365.

The Office 365 reporting service provides snapshots of tenant activity. Slow as it might be, sometimes PowerShell is the only way to drill down and retrieve granular information about mailboxes and other objects. Figuring out the gaps in the data available from the Office 365 reporting service, filtering out any inconsistencies that are detected, and analyzing information to generate meaningful reports are how reporting products from independent software vendors distinguish themselves. For instance, the reporting service might tell you that your tenant has so many inactive mailboxes, but what you really need to know is who owns those mailboxes, when they were last accessed, and if they are holding valuable Office 365 licenses.

Examples of Office 365 reporting products include Cogmotive, 365 Command, 4Ward365, and the Governance Toolkit for Office 365. A slightly different take is provided by products that have evolved from the on-premises space, like ENow Software’s MailScape 365, as these implementations cover components such as hybrid connectivity, including Active Directory Federation Services, Azure Active Directory Sync Services, and DirSync, to make sure to provide a complete picture of what can be a pretty complex infrastructure.

Each product has its own strengths and specific focus, but all that use the reporting service follow the same basic principle of using a service account, a non-licensed user belonging to the tenant, to log on and periodically extract data from Office 365 for a tenant and then use that data for analysis and reports. The service account is authorized with the appropriate permissions (such as membership of the Exchange View-Only Organization Management role group) to retrieve information from Office 365 on behalf of a tenant. The data is downloaded from the Office 365 reporting service and stored for ongoing analysis. Although most products will create a service account for you, it’s good to know what happens behind the scenes. This Cogmotive description provides good information for how a service account is created.

The service account is used to periodically download information from the Office 365 reporting service along with whatever other additional data is needed from Office 365 or Azure Active Directory. The retrieved data is usually stored by the software vendor so that it can be used to generate trend reports such as the growth in mailboxes or messaging activity over time. Each product has its own set of reports and its own unique way to interpret and present information.

Naturally, because reporting products are built by independent software vendors, you have to pay extra for them. Pricing details are available from the vendors and range from a couple of hundred dollars per month upward, depending on the number of seats in the tenant. Cogmotive has a very nice offer for small tenants as it doesn’t charge anything for tenants with fewer than 50 seats. According to a company spokesperson, Cogmotive currently supports about 2,000 tenants worldwide amounting to some 1.7 million seats.

Given the attractiveness of the growing number of Office 365 tenants and the availability of the Office 365 reporting web service, it’s likely that more software vendors will enter the market over time. Most vendors allow time-limited trials so it is possible to run products against real tenant data to test just how suitable they are for your needs. The kind of issues to consider when assessing reporting products for Office 365 are:

• How well does the product scale up to deal with the predicted full load of the tenant? Good demos performed against a 5-user tenant might not be so impressive when confronted with the data generated by 20,000 users.
• What additional software is required to create the reports? Is the product fully web-based or do you have to install some software on a workstation or servers to access the reports?
• How many reports does the product generate?  How many of those reports do you think you actually need? Are any obvious reports missing?
• How often is data retrieved from Office 365? What other sources of Office 365 data are queried outside the reporting service and where is the information stored? What privacy and security policies govern the retention of data accumulated from Office 365?
• How well does the product handle hybrid Exchange deployments?
• Is the product focused on any particular part of Office 365 or does it offer good coverage across Exchange, SharePoint, and Lync?
• How quickly is the product updated to accommodate new Office 365 features such as Office Delve and modern groups – or something like clients that connect using a new version of a mobile operating system?
• Does the product integrate with reporting software already used for on-premises applications?
• How is support provided? Is support available locally?
• Does the product offer any other useful features in addition to reporting?
• How much will the product add to your monthly expenditure?

No single product is the best choice for every situation. The best approach is to spend the time to test the software in your own environment and make the decision based on what you discover there. Take your time!