Can't receive mail in a hybrid environment after you install a new certificate on the on-premises Exchange 2016 servers

We have established hybrid connection between O365 and on Premises Exchange 2016 CU9, everything has been working fine, until recently the mail flow between O365 and on premises Exchange 2016 stopped working. Here is what we have done recently, CU update from CU7 to CU9 at on premises Exchange 2016, switched Godaddy certificate with COMODO RSA Domain Validation Security Server CA. I opened support with O365 support, they didn't provide me much help. 

I went to O365 admin portal, mail flow, connectors, and outbound connector and do validation for the connector from O365 to on Premises Exchange 2016 server, and got the following error:

450 4.4.317 Cannot connect to remote server [Message=451 5.7.3 STARTTLS is required to send mail] [LastAttemptedServerName=mail.mydomain.com] [LastAttemptedIP=198.171.58.5:25] [BL2NAM02FT047.eop-nam02.prod.protection.outlook.com].
 

I did some community support search and find out a Microsoft KB that makes sense to me. here is the link

https://support.microsoft.com/en-us/help/2989382/can-t-receive-mail-in-a-hybrid-environment-after-you-install-a-new-cer

I checked our Default frontend transport receiveconnector, and found out the certificate was bound to our old Godaddy certification, so I followed Microsoft above link, 

1. Run the following commands:

   Get-ReceiveConnector "ServerName\Default Frontend ReceiveConnector" | Set-ReceiveConnector -TlsCertificateName $null 

   Get-ReceiveConnector "ServerName\Default Frontend ReceiveConnector" | Set-ReceiveConnector -TlsDomainCapabilities $null 

2. Rerun the Hybrid Configuration wizard to update the receive connector on the hybrid server with the certificate information.
3. recheck the receiveconnector's TLS binding, it has updated with new certificate. 
   

After that 

Get-ReceiveConnector "ServerName\Default Frontend ReceiveConnector" |ft identity, TlsCertificateName, TlsDomainCapabilities
 -TlsCertificateName

We found out the new certificate are updated on all Exchange 2016 receiveconnectors and right tls domain is bound on it. Bingo. All mail flow are starting working again. 

Cannot Export to a PST File from Exchange 2013/2016 eDiscovery

When you are using Office 365 or Exchange 2013/2016 on premises. You are in the Exchange Admin Center and you want to export your eDiscovery search results to a PST file.  You select "Export to a PST file" option and got with the following error:

In order to successfully export your mailbox Search results to a PST file from Exchange Online eDiscovery you must use and configure the following:

1) Use IE 10 or newer.  If you are using IE 9, then you must also install the .NET Framework 4.5

2)  Add https://*.outlook.com to the Local Intranet Zone in Internet Explorer

3)  Ensure that the following URL's are listed in the Trusted sites Zone:

4) If you use Exchange 2013/2016 on premises, just login from one of your Exchange servers to do the export to make the things much easier :-) 

• https://*.outlook.com
• https://r4.res.outlook.com
• https://*.res.outlook.com

This should make a successful export

Note: Microsoft eDiscovery is limited 100 mailboxes per search.

 

very useful filtered netstat command by destination IP address or subnet

C:\Windows\system32>netstat -ano | findstr 10.103.*

  TCP    10.105.12.40:135        10.103.71.156:17118    ESTABLISHED     908

  TCP    10.105.12.40:135        10.103.71.157:1782     ESTABLISHED     908

  TCP    10.105.12.40:135        10.103.71.157:1786     ESTABLISHED     908

  TCP    10.105.12.40:135        10.103.71.219:40169    ESTABLISHED     908

  TCP    10.105.12.40:10457      10.103.71.219:443      ESTABLISHED     51032

  TCP    10.105.12.40:14080      10.103.71.159:443      ESTABLISHED     51032

  TCP    10.105.12.40:15345      10.103.71.159:443      ESTABLISHED     51032

  TCP    10.105.12.40:16241      10.103.71.219:443      ESTABLISHED     51032

  TCP    10.105.12.40:23522      10.103.71.159:443      ESTABLISHED     64528

  TCP    10.105.12.40:25290      10.103.71.159:443      ESTABLISHED     51032

  TCP    10.105.12.40:26783      10.103.71.156:17121    ESTABLISHED     7900

  TCP    10.105.12.40:26783      10.103.71.156:17122    ESTABLISHED     7900

  TCP    10.105.12.40:26783      10.103.71.157:1789     ESTABLISHED     7900

  TCP    10.105.12.40:26783      10.103.71.157:1797     ESTABLISHED     7900

  TCP    10.105.12.40:26783      10.103.71.219:40171    ESTABLISHED     7900

  TCP    10.105.12.40:28611      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:28645      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:28860      10.103.71.182:30435    ESTABLISHED     51032

  TCP    10.105.12.40:29710      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:29711      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:30505      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:30506      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:31217      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:31218      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:31945      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:32739      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:33411      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:34221      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:34883      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:34884      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:35620      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:37020      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:37597      10.103.71.182:30435    ESTABLISHED     51032

  TCP    10.105.12.40:37891      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:38098      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:38099      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:38892      10.103.71.157:25       ESTABLISHED     24292

  TCP    10.105.12.40:38939      10.103.71.159:443      ESTABLISHED     64528

  TCP    10.105.12.40:40820      10.103.71.156:25       ESTABLISHED     24292

  TCP    10.105.12.40:41330      10.103.71.156:25       ESTABLISHED     24292

Exchange 2013 CU16 and Exchange 2016 CU5, both of these Exchange versions now require .NET framework 4.6.2 on all supported OS platforms.

As mentioned in the release posts for Exchange 2013 CU16 and Exchange 2016 CU5, both of these Exchange versions now require .NET framework 4.6.2 on all supported OS platforms.  This post focusses on Windows 2012 and 2012 R2 installations, see note below about Windows 2016.

Exchange setup will check for the presence of the required .NET framework, if not present setup will halt.

This is also true when extending the AD Schema and preparing Active Directory.  If the machine where you are running the AD preparation commands does not have the required .NET framework, then the installer will log an error.   In the below example a Windows 2012 R2 domain controller, the Schema Master FMSO role holder, was used to initiate the PrepareSchema command.  This failed due to .NET not being updated on that server where the Exchange 2013 CU16 setup command was being executed:

setup.exe /IAcceptExchangeServerLicenseTerms /PrepareSchema

Let us say if you are running Exchange 2013 CU 10 or Exchange 2016 CU2, you cannot apply the latest CU to your current Exchange 2013 or Exchange 2016 directly. You will go with two steps pf Exchange CU update or three Steps including the .net fromwork update:

For Exchange 2013

1. Apply Exchange 2013 CU 15 on your Exchange 2013 CU10 first,

2. Apply .net framework 4.62 and its security updates

3. Apply the latest Exchange 2013 CU, says CU 17 or CU18 coming soon in late 09/2017

For Exchange 2016

1. Apply Exchange 2016 CU 4 on your Exchange 2016 CU2 server first,

2. Apply .net framework 4.62 and its security updates

3. Apply the latest Exchange 2016 CU, says CU 6 or CU7 coming soon in late 09/2017

Reference:

https://blogs.technet.microsoft.com/rmilne/2017/03/27/exchange-2013-cu16-and-exchange-2016-cu5-net-framework-requirement/

How to determine which client to send email on Exchange

We had issue with meeting auto forward issue with Apple device bug that has apparently been fixed by the latest iOS 10.3.3. In order to figure out which client caused the auto forwarding,  here is the tip that explains how to determine which type of email client sent a particular email:

The good news is that the Message Tracking Logs, as expected, records this information. Every email sent has a SourceContext property which contains, amongst other information, the ClientType used to send the email. The important thing is to check this property for SUBMIT events, i.e., when the Mailbox Transport Submission service passes the email to the Transport service (in other words, when Exchange picks up the email from the mailbox's outbox folder and passes it on for delivery).

Please note that this only applies to emails sent by internal users! There is no SUBMIT event when an external sender sends an email to an internal user, meaning there is no ClientType property for these emails. 

 To check a particular email, we can run something like the following cmdlet and look at the SourceContext field:

Get-TransportService | Get-MessageTrackingLog -ResultSize Unlimited -Start 07/28/2017 -EventID SUBMIT -Sender user@xyz.com -MessageSubject "subject of the message" | ft SourceContext -auto -wrap

the output is something like:

MDB:5f3ad20c-4f7c-4336-b90b-80713daf208f, Mailbox:58d5fcea-e4eb-4546-b11e-4553bee5db46, Event:220387198,
MessageClass:IPM.Note, CreationTime:2017-07-28T09:19:48.649Z, ClientType:AirSync

So it's the iPad's activesync caused the forwarding for our case...

Get-TransportService | Get-MessageTrackingLog -ResultSize Unlimited -Start 07/28/2017 -EventID SUBMIT -Sender user@xyz.com -MessageSubject "subject of the message"  | ft SourceContext -auto -wrap

 This field will contain information like this:

MDB:34f3dc86-91bb-4ee7-a6a5-3d3ddc536050, Mailbox:a1de664f-9826-43a3-b9c8-3db019c86d8b, Event:29647741, MessageClass:IPM.Note, CreationTime:2017-07-28T07:17:14.922Z, ClientType:MOMT

 In this case, MOMT stands for MAPI on the Middle Tier, basically clients that connect using Outlook or any other application that connects using RPC/HTTP or MAPI/HTTP.

To count the number of emails sent using OWA for today, we can do something like:

(Get-TransportService | Get-MessageTrackingLog -ResultSize Unlimited -Start 07/28/2017 -EventID SUBMIT | ? {$_.SourceContext -match "OWA"}).Count

Exchange 2010 CAS server recoverserver failed due to different SID of the computer account

We shut down one of our Exchange 2010 CAS for a few months, Windows group thought we don't need it and killed VM and deleted Computer object. Although we can kill the server via ADSIeiditor, however,  that's the worst case of scenarios. Instead we asked Windows team to rebuild VM with the same version of OS, the same computer name and joint the domain,

we went over the prereqs for doing this, ran setup /mode:recoverserver option, the recovery mode finished 98% and failed to start MSExchangeADTopology services.

this is the error

 Client Access Server Role                                 FAILED
     The following error was generated when "$error.Clear();
          if ($exsSid -eq $null -or $exsSid -eq "")
          {
          $exsSid = get-ExchangeServerGroupSID -DomainController $RoleDomainCont
roller
          }
          start-setupservice -ServiceName MSExchangeADTopology -ServiceParameter
s $exsSid,$RoleDomainController
        " was run: "Service 'MSExchangeADTopology' failed to reach status 'Runni
ng' on this server.".


The Exchange Server setup operation didn't complete. More details can be found
in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

Exchange Server setup encountered an error.

Rebooted the server, though MSExchangeADTopology service started OK, however, other services cannot be restarted.

So I tried to uninstall the server, and got error that the action of uninstall cannot continue since previous disaster recovery setup was not completed, please resume the disaster recovery maintenance or something like that. If I run setup /mode:recoverserver again and got the dame error as previously

I knew SID will cause the issue as we did disaster recover for Exchange 2003 servers previously. I did search cannot find any, until this one:

https://blogs./technet.microsoft.com/exchange/2007/05/21/how-does-exchange-2007-setup-know-to-resume-a-failed-setup/

By comparing with other working servers, I edit the register key as followings

 (Note: before do any modification of the reg key, we had better export the reg key to a file and do some screenshot as well)

1. delete the WaterMark string for sure

2. Delete Action string with DisasterRecover value

3. Create new Sting Reg_SZ called ConfiguredVersion with version number from UnpackedVersion value

After that, go to control panel -Program Features, remove the Exchange 2010 software, it failed at the first try for failed to stop the MSExchangeADTopology service. I tried to stop the service manually, it restarted the service automatically even I set it start manually, then I tried to disabled it and stop it, it still complained failed to stop the service. so I re-enabled the service and started it manually. this time, uninstalled finished completely successfully. Double check get-Exchangeserver, or Adsieditor, all entries for this server is gone. Well done,

  

IPSec Site to Site VPN Debug Command

Trying to identify VPN issues between two sites.

I've got the following enabled:

logging enable
logging buffered informational
logging trap informational
logging asdm informational
logging host switch.link 192.168.x.x
logging host switch.link 192.168.x.x
logging rate-limit 50 1 level 6

Normally you only run the debug commands when you are actually troubleshooting something. What I will do if I don't have a loging server available is to change the logging buffered to debugging and in SecureCRT (which is the terminal emulator I'm using) save the output on the screen to a text file and then search through that text file for the peer IP of the VPN etc.

If you run debugging level on buffered, trap and host at the same time it will burden the ASA quite significantly if there is a lot of output.

The best thing would be to have a linux based log server where you can use tools like grep to search through the output.

Exchange 2010 SP3 mailbox role installation with error exit code 87. Default mailbox database cannot mounted and copy status shown as "Service Down"

• I am in the process of installing Exchange Server 2010 SP3 on a Windows Server 2012 server in order to rehome Exchange 2010 Public folder servers to new hardware. Everything went well with the installation process until I went to add the Mailbox server role, which is giving me the following error (from the log file):

  [01/04/2017 21:58:09.0090] [1] Executing: 

            $wevtutil= join-path (join-path $env:SystemRoot system32) wevtutil.exe;

            $manifestPath = [System.IO.Path]::Combine($RoleInstallPath, "Scripts\TSCrimsonManifest.man");

            Start-SetupProcess -Name:"$wevtutil" -Args:"im `"$manifestPath`" "

          

  [01/04/2017 21:58:09.0105] [2] Active Directory session settings for 'Start-SetupProcess' are: View Entire Forest: 'True', Configuration Domain Controller: 'TAM-FS2.tacomaartmuseum.org', Preferred Global Catalog: 'TAM-FS2.tacomaartmuseum.org', Preferred Domain Controllers: '{ TAM-FS2.tacomaartmuseum.org }'

  [01/04/2017 21:58:09.0105] [2] Beginning processing Start-SetupProcess -Name:'C:\Windows\system32\wevtutil.exe' -Args:'im "C:\Program Files\Microsoft\Exchange Server\V14\Scripts\TSCrimsonManifest.man" '

  [01/04/2017 21:58:09.0276] [2] Starting: C:\Windows\system32\wevtutil.exe with arguments: im "C:\Program Files\Microsoft\Exchange Server\V14\Scripts\TSCrimsonManifest.man" 

  [01/04/2017 21:58:09.0344] [2] Process standard output: 

  [01/04/2017 21:58:09.0344] [2] Process standard error: The value for channel property Type contains an invalid value. The parameter is incorrect.

  
  

  [01/04/2017 21:58:09.0347] [2] [ERROR] Unexpected Error

  [01/04/2017 21:58:09.0347] [2] [ERROR] Process execution failed with exit code 87.

  [01/04/2017 21:58:09.0350] [2] Ending processing Start-SetupProcess

  [01/04/2017 21:58:09.0350] [1] The following 1 error(s) occurred during task execution:

  [01/04/2017 21:58:09.0350] [1] 0.  ErrorRecord: Process execution failed with exit code 87.

  [01/04/2017 21:58:09.0350] [1] 0.  ErrorRecord: Microsoft.Exchange.Configuration.Tasks.TaskException: Process execution failed with exit code 87.

  [01/04/2017 21:58:09.0351] [1] [ERROR] The following error was generated when "$error.Clear(); 

            $wevtutil= join-path (join-path $env:SystemRoot system32) wevtutil.exe;

            $manifestPath = [System.IO.Path]::Combine($RoleInstallPath, "Scripts\TSCrimsonManifest.man");

            Start-SetupProcess -Name:"$wevtutil" -Args:"im `"$manifestPath`" "

          " was run: "Process execution failed with exit code 87.".

  [01/04/2017 21:58:09.0351] [1] [ERROR] Process execution failed with exit code 87.

  [01/04/2017 21:58:09.0351] [1] [ERROR-REFERENCE] Id=MailboxComponent___13A8A6B7DE0A4fe3BEB5CB1D86105DA3 Component=EXCHANGE14:\Current\Release\PIM Storage\Content Indexing

  [01/04/2017 21:58:09.0351] [1] Setup is stopping now because of one or more critical errors.

  [01/04/2017 21:58:09.0351] [1] Finished executing component tasks.

  [01/04/2017 21:58:09.0393] [1] Ending processing Install-MailboxRole
• 
  

Although the installation ended with error exit code 87, the mailbox role seems installed already since all mailbox role's services are up and running... information store, mailbox replication services, etc. are installed and can be started with no issue. However, the default mailbox database cannot be mounted, copy status shows services down. We will have to fix it.

And the solution was:

• Fix this by removing the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\eventlog\Microsoft-Exchange-Troubleshooters/Operational and sub keys, then reinstall the mailbox rule once more time.  

That's nice, Microsoft, there is no KB about this.

Outlook 2016 - cannot view contents in shared mailbox inbox folders, but visible from OWA

There is a shared mailbox. users connect to the mailbox by adding it as an additional mailbox. It works fine. All suddenly, users reported they are no longer able to view contents of Inbox folders. " we don;t have any thing to display"

Here is the fix:

The way to change this option is the following:
 - menu "File"

 - "Account settings" -> "account settings..."

 - in the "E-mail" tab, select your account and then push the Change... button

 - push the "More settings..." button

 - go to the "Advanced" tab

 - in the "Cached Exchange Mode Settings" section, untick "Download shared folders"

If this options are already check, you need to do the opposite way to enabled both "Cached Exchange Mode Settings  and Download shared folders" options.

Meetings appear in the calendar as Tentative but the user didn't receive the requests in the inbox or meeting responses from others don't appear in the inbox

We have had several calls recently. Majority of them are Mac Outlook 2016 users (we are running Exchange 2016). We found out solution 1 of the MS KB https://support.microsoft.com/en-us/kb/2966790 fixed the issues

 Users experience one or both the following symptoms in Outlook:

• The user doesn't receive meeting requests in their inbox. However, the meetings appear in the user's calendar as Tentative.
• When the user creates a meeting request, the user doesn't see meeting responses from attendees. However, tracking information for the meeting is updated in the user's calendar.

In this scenario, the user does not have a delegate set up.

CAUSE

This issue occurs if the meeting requests were processed incorrectly in the user's mailbox and they get delivered to the user's Deleted Items folder instead of the Inbox folder or the meeting requests were processed incorrectly as if a delegate was set up. 

This can occur if the Receive folder for the IPM.SCHEDULE.MEETING message class was changed to the /Schedule folder or if the PR_RULE_MSG_PROVIDER property of messages that have as message class of IPM.Rule.Version2.Message is set to Schedule+ EMS Interface.

SOLUTION

To resolve this issue, start by following the steps in Solution 1. Depending on the scenario, you may have to use the steps in Solution 2. You will not know which solution applies to the user until you start troubleshooting by using the steps in Solution 1. 

Note The exact steps will vary based on the version of the MFCMAPI tool that you're using. Use caution when you modify mailboxes by using MFCMAPI. Using this tool incorrectly can cause permanent damage to a mailbox.

Solution 1

1. Download MFCMAPI from http://mfcmapi.codeplex.com/.
2. Start MFCMAPI.
3. On the Session menu, click Logon.
4. Select the user's online mode Outlook profile, and then click OK.
   
   Note If the user doesn't have an online mode profile, create a profile. Or, on the Tools menu, click Options, and then make sure that both the Use the MBD_ONLINE flag when calling OpenMsgStore check box and the Use the MAPI_NO_CACHE flag when calling OpenEntry check box are selected.
5. In the list, double-click the user's primary mailbox.
6. In the new window that appears, on the MDB menu, point to Display, and then click Receive folder table.
7. In the window, look for IPM.SCHEDULE.MEETING. Then, do one of the following:
   • If IPM.SCHEDULE.MEETING is not present, go to Solution 2. 
   • If IPM.SCHEDULE.MEETING is present, go to step 8 of this procedure.
8. Expand the Root container.
9. Right-click Schedule, click Advanced, and then click Set Receive Folder.
10. Enter IPM.SCHEDULE.MEETING in the box.
11. Click Delete Association, and then click OK.
12. Repeat step 6 and 7 to make sure that the IPM.SCHEDULE.MEETING association is removed from the list.
13. Test to see whether the user can receive meeting responses and meeting requests in their inbox.

Solution 2

1. Download MFCMAPI from http://mfcmapi.codeplex.com/.
2. Start MFCMAPI.
3. On the Session menu, click Logon.
4. Select the user's online mode Outlook profile, and then click OK.
   
   Note If the user doesn't have an online mode profile, create a profile. Or, on the Tools menu, click Options, and then make sure that both the Use the MBD_ONLINE flag when calling OpenMsgStore check box and the Use the MAPI_NO_CACHE flag when calling OpenEntry check box are selected.
5. In the list, double-click the user's primary mailbox.
6. Expand the Root container, and then expand Top of Information Store.
7. Right-click Inbox, and then click Open Associated Contents Table.
8. In the upper part of the window, scroll to locate the Message Class column.
9. Click Message Class to sort the Message Class column.
10. Look for all messages that have a message class of IPM.Rule.Version2.Message.
11. Click each message that has a message class of IPM.Rule.Version2.Message, and then in the lower part of the window, look for a property that's called PR_RULE_MSG_PROVIDER.
12. Check whether the PR_RULE_MSG_PROVIDER property has a value of Schedule+ EMS Interface.
13. In the upper part of the window, delete messages whose PR_RULE_MSG_PROVIDER property have a value of Schedule+ EMS Interface.
14. Test to see whether the user can receive meeting responses and meeting requests in their inbox.

Create a DAG with Exchange 2013/2016 – Troubleshooting

Once you tried to create a DAG (database Availability Group) with Microsoft Exchange Server 2013 or Exchange 2016, you may get many errors when you tried to add DAG member. It happens to me since I tried to use no Exchange server as witness server and forgot to add Exchange Trusted system group to local Administrators group once I tried to create an Exchange 2016 DAG in the LAB

So you’ve installed two or more Exchange 2016 servers with the Mailbox role and you decide to create the DAG with those two servers. So you create a DAG with a name of LABDAG-16…so good so far, no errors

Now you decide to add the first member to your DAG. Here are the errors you may encounter:

First Issue:  ACCESS DENIED

A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API ‘”CreateCluster() failed with 0×5. Error: Access is denied”‘ failed.. [Server: MBX1.domain.int]

Cause: the ECP wizard tried to add the first mailbox server to the DAG. During this process the wizard create a computer object in Active Directory (the CNO). This object doesn’t have the good rights. Exchange Trusted Subsystem had special permissions assigned and not “Full Control” on CNO (LABDAG-16).

Resolution: assign “Full Control” to Exchange Trusted Subsystem on the CNO of the DAG from Security Tab.

Second Issue: DNS Host Name

An Active Manager operation failed with a transient error. Please retry the operation. Error: The fully qualified domain name for node '' could not be found.

Cause: if you go in the property of the CNO, in the tab ‘general’, you will see the field “DNS name” empty. The ECP cannot find this information, that’s why you have this message.

Resolution: go ADUC, expand to Computers OU, in the property of the CNO (LABDAG-16), in the tab ‘attribute Editor’ tab and search for the “dNSHostName” attribute. Enter the FQDN of the CNO (LABDAG-16.DOMAIN.COM, for my case) and apply.

Third Issue: CNO not disable

A computer account named ' LABDAG-16' already exists and is enabled. The account must be disabled in order to be used by the database availability group.

Cause: the CNO is enable. To operate the ECP need to have the CNO disable. The ECP will enable the CNO itself.
Resolution: GO in Active Directory and disable the CNO (LABDAG-16, for my case), and try it again.

How to Speed up mailbox migration to Exchange 2016

Moving mailboxes from 2010 to Exchange 2016 or Exchange 2016 can often go very slowly, even when the network and server resources are fast and abundant! 
The Exchange Mailbox Replication Service (MRS) has extensive resource throttling enabled by default in order to prevent mailbox moves from choking out the rest of the users.  Because of this you may see mailboxes with a status of RelinquishedWlmStall and if you look at the details of the Get-MoveRequestStatistics report you will see mailboxes have a lot of time sitting idle under the TotalStalledDueToWriteThrottle counter.

Microsoft tech support suggests making changes to the “MSExchangeMailboxReplication.exe.config” file located at “C:\Program Files\Microsoft\Exchange Server\V15\Bin”.  The values to look at, along with their default settings are:

MaxActiveMovesPerSourceMDB=”20″
MaxActiveMovesPerTargetMDB=”20″
MaxActiveMovesPerSourceServer=”100″
MaxActiveMovesPerTargetServer=”100″
MaxTotalRequestsPerMRS=”100″

ExportBufferSizeKB=”512″

We typically like to set these values so that about 10 mailboxes can be moved simultaneously.  The ExportBufferSizeKB we’ve used in the past is “10240”.  The Exchange Mailbox Replication Service should be restarted after these changes.

The other suggestion Microsoft has made is to disable content indexing on the target database so that the search index scanner isn’t overwhelmed by all the new messages needing to be indexed.  You’ll want to set it back once the migration is complete.

Set-MailboxDatabase “DB1” -IndexEnabled:$False

In our experience however, these first two suggestions do NOT have tremendous impact on the overall speed.  The following two options have proven to be the most effective for us.

Use the “-priority emergency” parameter on the mailbox moves.  This will give the move the highest priority in the MRS queue.  For example:

New-MoveRequest -Identity “user@domain.com” -TargetDatabase “DB1” -Priority emergency

If the priority flag and the MRS config editing doesn’t make the moves fast enough for you, then disable MRS throttling altogether!  To do this, change the “MRS” REG_DWORD key from 1 to 0 under this registry path:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\MSExchange ResourceHealth

Then restart the Exchange Mailbox Replication service.  Now your mailboxes will move without any throttling policy.  Once the mailbox migration is complete change the value back to 1 to re-enable MRS resource monitoring.

How to: Move Catalog Data in Exchange 2010

When you’ve moved an mailbox database, the old catalog data will stay on the old partition. This prevents deleting the specific partition. When you try to delete those files, you receive this error.

Error description:

The Catalog data folder stays in place after a database move path action. The folder couldn’t be cleaned up because it is in use by the Microsoft Exchange Search Indexing service

Solution:

First we have to temporarily disable the indexing on the particular mailbox database. Then we stop the indexing service of Exchange so we can delete the remaining files. The last step is to enable the indexing again.

Actions:
You can follow the next steps.

- Open the Exchange Management Shell and run the following command:

     Set-MailboxDatabase -Identity YourDBName -IndexEnabled $false

-Before moving / cleaning the catalog files, stop the service Microsoft Exchange Search Indexer
- Move the catalog files to the new database location or clean them up.
- Start the service Microsoft Exchange Search Indexer
- Open the Exchange Management Shell and run the following command:

Set-MailboxDatabase -Identity YourDBName  -IndexEnabled $true

How to get-MailboxDatabase size quickly

get-MailboxDatabase us-* -Status |select ServerName, Name, DatabaseSize > c:\usmbxDBsize.txt

This command will provide us a list of the mailbox DBs their names starting with "US-"

How to block spoofing email from Exchange 2013/2016 or Office 365

1. Sign in to Exchange Admin Center http://outlook.office365.com/ecp with an admin account.
2. Navigate to mail flow>rules and click the add icon to create a new rule.
3. Create a rule like this (don’t forget to click more options):
 
With this rule, emails from outside your organization with your domain to your organization will be blocked.

failed to uninstall last Exchange 2013

If you have to uninstall the first and last Exchange 2013 server in your org and failed with following errors:

Error:Uninstall can't continue. Database Mailbox Database XXXXXXXXX1 : This mailbox database contains one or more mailboxes, mailbox plans, archive mailboxes, public folder mailboxes or arbitration mailboxes. To get a list of all mailboxes in this database, run the command Get-Mailbox -Database . To get a list of all mailbox plans in this database, run the command Get-MailboxPlan. To get a list of archive mailboxes in this database, run the command Get-Mailbox -Database -Archive. To get a list of all public folder mailboxes in this database, run the command Get-Mailbox -Database -PublicFolder. To get a list of all arbitration mailboxes in this database, run the command Get-Mailbox -Database -Arbitration. To disable a non-arbitration mailbox so that you can delete the mailbox database, run the command Disable-Mailbox . To disable an archive mailbox so you can delete the mailbox database, run the command Disable-Mailbox -Archive. To disable a public folder mailbox so that you can delete the mailbox database, run the command Disable-Mailbox -PublicFolder. Arbitration mailboxes should be moved to another server; to do this, run the command New-MoveRequest . If this is the last server in the organization, run the command Disable-Mailbox -Arbitration -DisableLastArbitrationMailboxAllowed to disable the arbitration mailbox.

This is because you have one of the following:

• Still have folders in your Public Folders
• Still have Mailboxes in your Mailbox Datbases
• Still have Mailbox Databses

Exchange 2013 uses a number of different types of mailboxes as seen below. Even if you have removed all of your users mailboxes you must still remove these mailboxes:

DISCOVERY MAILBOX
DiscoverySearchMailbox {D919BA05-46A6-415f-80AD-7E09334BB852}
MONITORING MAILBOXES
HealthMailbox62256620620346798029c55bfe0fcc5d
HealthMailboxb67c72c285f54c30bad37dd37eec361a
HealthMailbox721771958fa64a53aa78527c02caf55b
HealthMailbox995793b82c2c4a14b6173e54371179ec
HealthMailboxc8603cb9bb41442e8b402db42f93dc16
ARBITRATION MAILBOXES
SystemMailbox{1f05a927-c403-4250-9f07-c5e43605c1ac}
SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}
SystemMailbox{e0dc1c29-89c3-4034-b678-e6c29d823ed9}
FederatedEmail.4c1f4d8b-8179-4148-93bf-00a95fa1e042
Migration.8f3e7716-2011-43e4-96b1-aba62d229136

These mailboxes must be removed before you can actually uninstall Exchange 2013  You can remove these mailboxes from the ECP however if you you may need to use the command below to allow you to remove the arbitration mailoxes, this command allows you to disable them

Get-Mailbox -Arbitration -Database <MailboxDatabase> | Disable-Mailbox -Arbitration -DisableLastArbitrationMailboxAllowed

Replacing the Mailboxdatabase information with your own. Once removed if you start to unistall Exchange 2013 the uninstallation should complete

and last

Get-mailbox  -Database <mailboxdatabase > |disable-mailbox

after that try to uninstall your Exchange 2013 server again.

Good Luck!

How to check DAG Replication Status

get-DatabaseAvailabilityGroup USDAG-10 | %{ $_.Servers | %{ GetMailboxDatabaseCopyStatus -Server $_ } }

Exchange 2010 How to re-index mailbox database

 Note: Do the below if it happens all users on the DB. If just for one user, move the user to other DB to see if the issue is gone.

Exchange 2010 re-index mailbox database

Users may report that searching Outlook for non-cached or online mode mailboxes isn't getting the right results or not returning results past a certain date. Also OWA may return "The action couldn't be completed.  Try again later." when trying to search. You need to reindex your mailbox database(s). 

1. Run Repair-ExchangeSearchSymLinks.ps1. It isn't included in the normal scripts folder under the Exchange program files, you can copy this to a text document and save it as Repair-ExchangeSearchSymLinks.ps1.

view source

01	write-host "Attempting to repair symbolic links"

02

03	write-host "Stopping msftesql-exchange and msexchangesearch services"

04	stop-service -Force msexchangesearch

05	stop-service -Force msftesql-exchange

06

07	$rval = get-PSSnapin -Name Microsoft.Exchange.Management.PowerShell.Setup -ea SilentlyContinue

08	if(!$rval)

09

{

10	add-PSSnapin -Name Microsoft.Exchange.Management.PowerShell.Setup -ea SilentlyContinue

11

}

12

13	write-host "Installing symbolic links"

14	Install-ContentIndexingService –HubOnlyFilterRegistration

15

16	if(!$rval)

17

{

18	Remove-PSSnapin -Name Microsoft.Exchange.Management.PowerShell.Setup -ea SilentlyContinue

19

}

20

21	write-host "Starting msftesql-exchange and msexchangesearch services"

22	start-service msexchangesearch

23	start-service msftesql-exchange

24

25	write-host "Symbolic links restored."

26	write-host "Please run test-exchangesearch to verify installation."

Or you can get it from Microsoft here if you don't trust me and place it in the Exchange Server\V14\Scripts folder. Open Exchange Management Shell (remember you might need to run EMS elevated and run Set-ExecutionPolicy Unrestricted first), change path to the scripts folder then run:

.\Repair-ExchangeSearchSymLinks.ps1

 

2. Next run the ResetSearchIndex.ps1 script, this one is included in the Exchange program files scripts folder so no need to download. You can run it against a specific database or against all of them as per the commands below:

.\ResetSearchIndex.ps1 -All    .\ResetSearchIndex.ps1 databasename

 

You should see event id 109 for each database stating that an index has been created and sometime later event id 110 for each database as the indexing finishes.

ref link: http://www.stopdoingitwrong.co.uk/exchange-2010-re-index-mailbox-database/

How to: Move Catalog Data in Exchange 2010

How to: Move Catalog Data in Exchange 2010

 

 

When you've moved an mailbox database, the old catalog data is still present on the old partition. This prevents deleting the specific partition. When you try to delete those files, you receive this error.

 

Error description:

The Catalog data folder stays in place after a database move path action. The folder couldn't be cleaned up because it is in use by the Microsoft Exchange Search Indexing service

 

Solution:

First we have to temporarely disable the indexing on the particular mailbox database. Then we stop the indexing service of Exchange so we can delete the remaining files. The last step is to enable the indexing again.

 

Actions:

You can follow the next steps.

 

- Open the Exchange Management Shell and run the following command:

         Set-MailboxDatabase -Identity databasename -IndexEnabled $false

 -Before moving / cleaning the catalog files, stop the service Microsoft Exchange Search Indexer

         - Move the catalog files to the new database location or clean them up.

         - Start the service Microsoft Exchange Search Indexer

        - Open the Exchange Management Shell and run the following command:

    Set-MailboxDatabase -Identity databasename -IndexEnabled $true

how to get mailbox counts per Mailbox Database quickly

How to get mailbox counts per Mailbox Database quickly

get-mailbox -resultsize unlimited | group-object -property Database -noelement

How to get public path via email address

You may have issue to find a public folder path that you know the email address of the PF:

Here is the cmdlet you need to find out the path:

1. Set-AdServerSettings -ViewEntireForest $true -PreferredGlobalCatalog gc1.contoso.com

2. Get-PublicFolder cmdlet to find the path via the PF's email address:

Get-MailPublicFolder AccountsPayable@domain.com | Get-PublicFolder

you will see output something like below:

Name                                                        Parent Path
----                                                             -----------
Accounts Payable                                     \Accounting\US\Americas Choice

Missing Autodiscover information in your Exchange Autodiscover XML Response?

We had issue with Outlook anywhere that cannot be configured automatically reported from remote site. When we use Remote Connectivity Analyzer (https://www.testexchangeconnectivity.com/) to test Outlook Autodiscover, all the External URL (items below the <Type>EXPR</Type> lines) was missing.

After carefully checking the output, our EXPR type output is as follow:

Type>expr</Type>
 <Server>exchange.contoso.com</Server>
 <ASUrl>https://exchange.contoso.com/EWS/Exchange.asmx</ASUrl>
 <OOFUrl>https://exchange.contoso.com/EWS/Exchange.asmx</OOFUrl>
 <OABUrl>https://exchange.contoso.com/OAB/83099da3-46bc-49e3-9e35-4353f0f95268/</OABUrl>
 <UMUrl>https://exchange.contoso.com/EWS/UM2007Legacy.asmx</UMUrl>
 <Port>0</Port>
 <DirectoryPort>0</DirectoryPort>
 <ReferralPort>0</ReferralPort>
 <SSL>On</SSL>
 <AuthPackage>Basic</AuthPackage>
 <EwsUrl>https://exchange.contoso.com/EWS/Exchange.asmx</EwsUrl>
 <SharingUrl>https://exchange.contoso.com/EWS/Exchange.asmx</SharingUrl>
 <EcpUrl>https://exchange.contoso.com/ecp/</EcpUrl>
 <EcpUrl-um>?p=customize/voicemail.aspx&amp;exsvurl=1</EcpUrl-um>
 <EcpUrl-aggr>?p=personalsettings/EmailSubscriptions.slab&amp;exsvurl=1</EcpUrl-aggr>
 <EcpUrl-mt>PersonalSettings/DeliveryReport.aspx?exsvurl=1&amp;IsOWA=&lt;IsOWA&gt;&amp;MsgID=&lt;MsgID&gt;&amp;Mbx=&lt;Mbx&gt;</EcpUrl-mt>
 <EcpUrl-ret>?p=organize/retentionpolicytags.slab&amp;exsvurl=1</EcpUrl-ret>
 <EcpUrl-sms>?p=sms/textmessaging.slab&amp;exsvurl=1</EcpUrl-sms>
 </Protocol>
 <Protocol>

and normal output should be as follows:

<Type>EXPR</Type>
 <Server>exchange.contoso.com</Server>
 <ASUrl>https://exchange.contoso.com/EWS/Exchange.asmx</ASUrl>
 <OOFUrl>https://exchange.contoso.com/EWS/Exchange.asmx</OOFUrl>
 <OABUrl>https://exchange.contoso.com/OAB/83099da3-46bc-49e3-9e35-4353f0f95268/</OABUrl>
 <UMUrl>https://exchange.contoso.com/EWS/UM2007Legacy.asmx</UMUrl>
 <Port>0</Port>
 <DirectoryPort>0</DirectoryPort>
 <ReferralPort>0</ReferralPort>
 <SSL>On</SSL>
 <AuthPackage>Basic</AuthPackage>
 <EwsUrl>https://exchange.contoso.com/EWS/Exchange.asmx</EwsUrl>
 <SharingUrl>https://exchange.contoso.com/EWS/Exchange.asmx</SharingUrl>
 <EcpUrl>https://exchange.contoso.com/ecp/</EcpUrl> ;
 <EcpUrl-um>?p=customize/voicemail.aspx&amp;exsvurl=1</EcpUrl-um>
 <EcpUrl-aggr>?p=personalsettings/EmailSubscriptions.slab&amp;exsvurl=1</EcpUrl-aggr>
 <EcpUrl-mt>PersonalSettings/DeliveryReport.aspx?exsvurl=1&amp;IsOWA=&lt;IsOWA&gt;&amp;MsgID=&lt;MsgID&gt;&amp;Mbx=&lt;Mbx&gt;</EcpUrl-mt>
 <EcpUrl-ret>?p=organize/retentionpolicytags.slab&amp;exsvurl=1</EcpUrl-ret>
 <EcpUrl-sms>?p=sms/textmessaging.slab&amp;exsvurl=1</EcpUrl-sms>
 </Protocol>
 <Protocol>

Do you see the difference between ours and normal Type>expr</Type> vs <Type>EXPR</Type>?

lowercase expr vs Uppercase EXPR.

Oups, EXPR is case sensitive here!!!

Check our Exchange configuration

>get-OutlookProvider -id expr |FL

RunspaceId           : a86dfb0a-3f3f-4b39-a264-cd6cd993853d
CertPrincipalName    :
Server               :
TTL                  : 1
OutlookProviderFlags : None
AdminDisplayName     :
ExchangeVersion      : 0.1 (8.0.535.0)
Name                 : expr
DistinguishedName    : CN=expr,CN=Outlook,CN=AutoDiscover,CN=Client Access,CN=contoso,CN=Microsoft Exchange,CN=Serv
                       ices,CN=Configuration,DC=contoso,DC=com
Identity             : expr
Guid                 : 57b60bde-e415-4ce2-b618-5a2425a230e7
ObjectCategory       : contoso.com/Configuration/Schema/ms-Exch-Auto-Discover-Config
ObjectClass          : {top, msExchAutoDiscoverConfig}
WhenChanged          : 3/3/2014 2:53:33 AM
WhenCreated          : 3/3/2014 2:41:58 AM
WhenChangedUTC       : 3/3/2014 7:53:33 AM
WhenCreatedUTC       : 3/3/2014 7:41:58 AM

As you can see above it seems some admin created Identity             : expr WhenCreated          : 3/3/2014 2:41:58 AM.

This is the root cause of the issue and the following cmdlet that helped us to fix the issue

Remove-OutlookProvider -Identity expr

New-OutlookProvider -Name EXPR

After that, run the following command to verify and test Outlook Anywhere works fine after that.

 >get-OutlookProvider -id expr |FL

RunspaceId           : 0c0cf3a8-ad17-432c-8b8d-0d6daabe64df
CertPrincipalName    :
Server               :
TTL                  : 1
OutlookProviderFlags : None
AdminDisplayName     :
ExchangeVersion      : 0.1 (8.0.535.0)
Name                 : EXPR
DistinguishedName    : CN=EXPR,CN=Outlook,CN=AutoDiscover,CN=Client Access,CN=contoso,CN=Microsoft Exchange,CN=Serv
                       ices,CN=Configuration,DC=randomhouse,DC=com
Identity             : EXPR
Guid                 : 49918614-cafb-4193-9f06-bdb2287c31f1
ObjectCategory       : contoso.com/Configuration/Schema/ms-Exch-Auto-Discover-Config
ObjectClass          : {top, msExchAutoDiscoverConfig}
WhenChanged          : 6/4/2014 2:55:02 PM
WhenCreated          : 6/4/2014 2:45:24 PM
WhenChangedUTC       : 6/4/2014 6:55:02 PM
WhenCreatedUTC       : 6/4/2014 6:45:24 PM

By the way, for those who ran the test and got the following error:

"The EXCH provider section is missing from the Autodiscover response" message and Exchange 2010 Outlook Anywhere (RPC over HTTP) not working

here is the fix for this:

The EXCH provider section is missing from the Autodiscover response"

 

I'm sure, you already checked all Internal / External URL settings

But, have you checked your Mailbox Database settings and see what do you have configured as RPCClientAccessServer? You can do this by running the following cmdlet:

 

Get-MailboxDatabase | fl Server, RPCClientAccessServer

 

If your RPCClientAccessServer is not set or set not exactly as it supposed to be, you can  correct this as illustrated in the following example:

 

Set-MailboxDatabase -Identity "MYMBXDB-1″ -RpcClientAccessServer outlook.contoso.com

 

Note: In this case outlook.contoso.com is a FQDN name of my Network Load balancer