Active Directory Disaster Recovery 3s3c1r

and press Enter where is the number corresponding to the domain in which the failed server was located. 10. Type list sites and press Enter. 11. Type select site and press Enter where refers to the number of the site in which the DC was a member. 12. Type list servers in site and press Enter. This will list all servers in that site with a corresponding number. 13. Type select server and press Enter where refers to the DC to be removed. 14. Type quit and press Enter. The Metadata cleanup menu is displayed. 15. Type remove selected server and press Enter. At this point you should receive confirmation that the DC was removed successfully. If you receive an error that the object could not be found, it might have already been removed from the Active Directory. 16. Type quit, and press Enter repeatedly to return to the command prompt.

Note: Because this procedure requires modifying the configuration naming context, it requires Enterprise permissions. If the new DC receives a different name than the failed DC, you should perform the following additional steps: ·

Removal of the failed server object from the Sites & Services snap-in: 1. Open the Sites & Services snap-in. 2. Select the appropriate site. 3. Delete the server object associated with the failed DC.

·

Removal of the failed computer from the s & Computers snap-in: 1. Open the s & Computers snap-in. 2. Select the domain controllers container. 3. Delete the computer object associated with the failed DC.

WARNING:

9/11/2013 11:27 PM

Active Directory Disaster Recovery

18 of 40

http://windows.uwaterloo.ca/server/disaster/ms_docs/disaster.htm

Do not perform the additional steps above if the new machine will have the same name as the failed machine. Make sure that hardware failure wasn’t the cause of the problem. If the faulty hardware isn’t changed, then restoring via reinstallation may not help.

Restore From Backup This method relies primarily on the last good backup taken of the DC before the failure. A restore process can be initiated using either the Windows 2000 backup utility or a ed third party utility selected by your organization. The restore process will return the DC to its state at the time of backup; the DC will then query it’s replication partner(s) for any updates since that time. If there are changes, they will be replicated, ensuring the DC has an accurate and up-to-date copy of the Active Directory database. In addition, by restoring Active Directory from backup you have two further options available: ·

Non-Authoritative Restore

·

Authoritative Restore

These two methods allow you to manipulate two important components of the System State during the restore process, the Active Directory and SYSVOL. Although these components are restored together, they are discussed separately here. Non-Authoritative Restore Active Directory

Non-authoritative restore is the default method for the restoration of Active Directory, and it will be used for the majority of restore operations. Using this method, settings and entries that existed in the domain, schema, configuration, and optionally the global catalog naming contexts maintain the version number they had at the time of backup. After a non-authoritative restore, the DC is updated using normal replication techniques. That is if the version number of an attribute is less than the version number of the same attribute stored in its replication partners database (indicating the object has changed since it was last backed up). The object on the restored server will be updated with the changes that were made to that object since the time of the last backup. This ensures an up-to-date version of the database. SYSVOL

By restoring the SYSVOL non-authoritatively, the local copy that is held on the restored DC will be compared with that of its replication partners (using MD5 Checksums). Once the DC reboots, it will its replication partner(s), compare SYSVOL information, and replicate the necessary changes, bringing it up to date with the other DCs within the domain. This method should be used when there is at least one other functioning DC in the domain. This is the default SYSVOL restoration method and will occur automatically if a non-authoritative restore of the Active Directory is carried out. If there is no other functioning DC in the domain, a PRIMARY restore of the SYSVOL should be done. A primary restore builds a new ntfrs (Windows NT File Replication Service) database by loading the data present under SYSVOL on the local DC. This method is the same as non-authoritative except that the SYSVOL should be marked PRIMARY.

Authoritative Restore Active Directory

An authoritative restore is in essence an extension of the non-authoritative restore process. It requires all the steps of a non-authoritative restore before it can be initiated. The primary difference between the two is that an authoritative restore has the ability to increment the version number of the attributes of all objects in an entire directory, all objects

9/11/2013 11:27 PM


19 of 40


in a subtree, or an individual object (provided that it is a leaf object) to make it authoritative in the directory. As with a non-authoritative restore, once a DC is back online it will its replication partners to see what has changed since the time of the last backup. However, because the version number of the object attributes you wish to be authoritative will be higher than the existing instances of the attribute held on replication partners, the objects on the restored DC will appear to be more recent and therefore be replicated out to the rest of the DCs within the environment. Unlike a non-authoritative restore, an authoritative restore requires the use of a separate tool (ntdsutil.exe) to make it work. No backup utilities—including the native Windows 2000 utility—can perform an authoritative restore. An authoritative restore should be used when human error is involved such as when an has accidentally deleted a number of objects; that change has replicated to all the DCs, existence of those objects is removed from the domain; and the is unable to easily recreate these objects. An authoritative restore will not overwrite new objects that have been created after the backup was taken. It can only be carried out on objects from the configuration and domain contexts. Authoritative restores of schema naming contexts are not ed. SYSVOL

By restoring the SYSVOL authoritatively, you are specifying that the copy of SYSVOL that was restored from backup is authoritative for the domain. Once the necessary configurations have been made, the local SYSVOL will be marked as authoritative and be replicated out to the other DCs within the domain. Similarly to the Active Directory authoritative restore, this method will typically be used when human error is involved and the error has propagated out to other domain controllers. For example, when an has accidentally deleted an object that resides in SYSVOL such as a Group Policy object. The authoritative restore of SYSVOL does not occur automatically after an authoritative restore of Active Directory, additional steps are required. The exact process for all these methods of restoration will be discussed later in this white paper. Required Rights for Active Directory Restore To restore the System State data, the person performing the procedure must be a Local . Considerations for Restoring a DC From Backup An obvious advantage of restoring a DC from backup instead of replication is the faster restore times available. To illustrate this, the graph in Figure 8 below shows the time taken to restore a DC from backup with databases varying from 500 MB to 2 GB) in size. The machine in the test was a Compaq Proliant 800, with a 400MHz processor, 256 MB RAM and a 4/8 GB DAT Drive.

Figure 8: Graph representing times to restore varying sized DIT files from backup.

Note: This graph represents the time taken to restore System State only; the restore of a good backup as defined earlier will take longer depending on the size of your system disk.

9/11/2013 11:27 PM


20 of 40


Useful Life of Active Directory Backup

Ensure that the backup you are restoring was taken within the tombstone lifecycle, by default this is set to 60 days. For more information on the useful life of an Active Directory backup, see the What is a Good Backup? section of this paper. Restore Backup onto Different Hardware

It is possible to restore a DC onto different hardware. However there are some issues to consider before doing this. Different HALs—By default, the Hal.dll is not backed up as part of System State, however the Kernel32.dll is. Therefore if you are trying to restore a backup onto a machine that requires a different HAL—to a multiprocessor environment, for example—you will run into compatibility issues with the new HAL and the original Kernel32.dll. The only workaround for this situation is to explicitly copy the Hal.dll from the original machine and install it on the new machine. The limitation is that the new machine will now be bound to using only a single processor. Incompatible Boot.ini File—If you backup and restore the boot.ini file, you may have some incompatibility with your new hardware configuration, resulting in a failure to boot. Before restore, ensure that the boot.ini file is correct for your new hardware environment. Different Network or Video Cards—If your new hardware has a different video adapter or multiple network adapters, uninstall them before you restore data. When you restart the computer; the normal Plug and Play functionality will make the necessary changes.

Disk Space and Partition Configuration

In addition to the issues with restoring a DC onto different hardware, it is also important that the partitions on the new machine match those on the original machine. Specifically, all the drive mappings must be the same and the partition size must be at least the same as on the original machine. Additional Considerations for an Authoritative Restore

In addition to the above considerations for restoring a DC from backup, the following points should be considered specifically when performing an authoritative restore. Impact on Group hip The impact on group hips is the most significant issue stemming from the authoritative restore method of disaster recovery. You risk possible loss of group hip information. Because group hip is a multi-valued attribute, and because of the way links, back links and deletions are dealt with in Active Directory, the results of an authoritative restore on the representation of group hip may vary. These variations are based on which objects replicate first after an authoritative restore: the object or the Group object. If the un-deletion of the replicates first, then the group hip information of both the group (the it contains) and the (the groups he/she belongs to) will be represented correctly. If the un-deletion of the group replicates first, the replication partners will drop the addition of the (locally) deleted from the group hip. The only exception to this is the ’s primary group, which is always represented correctly both from the and group reference. Unfortunately there is no way to define which objects replicate first after an authoritative restore has been carried out. If your environment is affected by this situation, the only option you have is to modify the group hip attribute of the effected groups on the DC where the authoritative restore was carried out. This issue stems not from the integrity of the restored data, but from the way in which the data is replicated. By looking at this DC, s can view the way the directory should look and take steps to replicate the accurate directory information out to the other DCs within the domain. The best way to do this is to add a dummy and then delete that same dummy to/from each group that was involved in the authoritative restore.

9/11/2013 11:27 PM


21 of 40


The definition of “involved” in this context means any group that was either authoritatively restored itself or which had restored who did not have that group defined as their “Primary Group.” By doing this, you will force the correct group hip information to be replicated out from the source DC (the DC that the original authoritative restore was carried out on) and update the group hip information on its replication partners. These updated objects will reflect the correct hips and will also correct the information represented in the Member of tab of the restored objects. You MUST make sure that no additions are made to group hip (for the effected groups and s) on any of the other DCs within the environment. If you do not adhere to this, you risk the accurate version of the directory (held on the DC where the restore took place) being corrupted by the incorrect hip information. Once this occurs, you must either update group hip manually or perform another authoritative restore of the objects using the verinc option, and perform the process defined above again.

Impact on Trusts and Computer s In Windows 2000, trust relationships and computer s are negotiated at a specified interval (by default 30 days for trust relationships and computer s). When using the authoritative restore method, previously used s for the objects in the Active Directory that maintain trust relationships and computer s may be restored. In the case of trust relationships, this may impact communication with other domain controllers from other domains, manifesting in permissions errors when trying to access resources across domain boundaries. To rectify this, NTLM trust relationships to Windows 2000 or down level domains must be removed and recreated. In the case of a computer , this could impact communications between the member workstation or server and a DC of its domain. This will usually manifest itself in a on Windows NT or Windows 2000 machine having issues with authentication due to an invalid machine . To help with both the recreation of trusts and the resetting of computer s, use the NETDOM utility included in the tools on the Windows 2000 CD. Note: A Windows NT 4.0 machine will change its every seven days.

Bandwidth Considerations of SYSVOL Replication When performing an authoritative restore of the Active Directory, you should also perform an authoritative restore of the SYSVOL. By doing this, you are telling the other DCs in the domain that the SYSVOL information on the restored DC is authoritative. As a result the entire SYSVOL contained on the restored DC will be replicated out to all other DCs in the domain (via replication partners). The bandwidth associated with such replication will only be a consideration in a domain where there is an extensive use of large Group Policies and scripts. In addition, unlike Active Directory replication, FRS replication is not compressed between sites. Steps Required for a Non-Authoritative Restore To perform a non-authoritative restore using the Windows 2000 native backup utility, follow the steps in this section.

WARNING: If reinstalling the operating system, you may or may not the machine to the domain and could give any name to the machine during setup of the operating system. Do not promote the machine to a domain controller. After reinstalling the operating system, go directly to Step 4 below.

1. Reboot target system into Directory Services Restore Mode by pressing the F8 key upon system startup. 2. Select Directory Services Restore Mode (Windows 2000 domain controllers only).

9/11/2013 11:27 PM


22 of 40


3. Select the operating system that you wish to start in restore mode. 4. as (local system , no domain selection is available). 5. Run the Windows 2000 Backup utility and select the Restore Wizard button. 6. Select the appropriate backup location and ensure that at least the System disk and System State containers are checked. 7. Click the Advanced button and make sure you are restoring junction points (step 9). If you do not go through the advanced menu the restore process will not be successful. 8. Select Original Location in the “Restore Files to” drop down box. 9. In the Advanced Restore Options window, check the boxes for Restore security; Restore junction points, and restore file and folder data under junction points to the original location; Preserve existing volume mount points. See Figure 9 below for an illustration.

Figure 9.

10. Click on the Finish button. 11. Once complete click YES to restart the computer. The system will now reboot and replicate any new information since the last backup with its replication partners. By executing a non-authoritative restore on Active Directory, you automatically execute a non-authoritative restore of SYSVOL—no additional steps are required. For a primary restore of SYSVOL, make sure to check a fourth box in the Advanced Restore Options dialog next to: When restoring replicated data sets, mark the restored data as the primary data for all replicas. A primary restore is only required if the DC you are restoring is the only DC in the domain. Steps Required for an Authoritative Restore You can authoritatively restore the entire directory, a subtree, or an individual object (provided it is a leaf object) in the directory. The examples outlined below will detail how to restore both the entire directory and a subtree of a directory.

Authoritative Restore – Entire Directory

Authoritative restore of the entire directory is a major operation and should only be carried out after consultation with a Microsoft professional. Authoritative restore of the entire directory should not be performed if a DC is the only DC in the domain. Follow the first 10 steps for a non-authoritative restore. When asked to restart the computer, refuse to do so. This is because you would have to restore the System State again to an alternate location. To continue, follow the steps below: 1. Click on the Restore Tab 2. Ensure Alternate Location is selected in the “Restore Files to” drop down list.

9/11/2013 11:27 PM


23 of 40


Selecting Alternate Location will restore the System State to an alternate location. You need not restore the system disk to an alternate location, so you should check the box only for System State. Restoring System State to an alternate location will only restore the SYSVOL, boot files and registry to an alternate location (not Active Directory). This is done to enable an authoritative restore of the SYSVOL. Once the entire directory is restored, you could delete the files in the alternate location.

3. When the restore process has finished, close the backup application. 4. Open a command prompt and type ntdsutil, press Enter. 5. At the next prompt, type authoritative restore and press Enter. 6. At the next prompt, type restore database. 7. At the “Authoritative Restore Confirmation Dialog” box, click OK. 8. Type Quit, and repeat until you exit out of the application. 9. Restart the server. The server is now the authoritative DC for the domain. Changes will be replicated to the other DCs within the environment. 10. Once the system has been rebooted and after the SYSVOL share is published, ( it may take a few minutes before the SYSVOL share and its sub-folders appear on the domain controller) copy the required files/folders from the SYSVOL directory that was copied to the alternate location to the original location. By doing this, the files that were overwritten, are replicated out to the other domain controllers, so that the SYSVOL is the same as that which was present at the time of backup. Below is an example of copying the SYSVOL from the alternate location to the original location. Depending on your system, your drive and folder information may vary. Copy the contents of the scripts directory from: c:\ \sysvol\c_\winnt\Sysvol\Domain\scripts\ And add it to: c:\Winnt\SYSVOL\Sysvol\domain\scripts\ Copy the contents of the policies directory from: c:\ \sysvol\c_\winnt\Sysvol\Domain\policies\ And add it to: c:\Winnt\SYSVOL\Sysvol\domain\policies\ By restoring the SYSVOL authoritatively, the files on the restored DC will be authoritative for the domain and will replicate to other DCs. Changes made to any policy after the backup will be lost. For example, a Group Policy by the name of Finance Policy existed at the time of the last backup, and was referenced by a folder in the SYSVOL directory as: C:\WINNT\SYSVOL\Sysvol\Domain.com\Policies\{31B2F340-016D-11D2-945F-00C04FB984F9} However, shortly after the last backup, an edited the Finance Policy, and although the properties of the policy changed, the GUID of the GPO remained the same. As a result, the policy was still referenced by the same directory name {31B2F340-016D-11D2-945F-00C04FB984F9}. When it came time to authoritatively restore the directory, the folder {31B2F340-016D-11D2-945F-00C04FB984F9} from the alternate SYSVOL location was copied to the original SYSVOL location. This replaced the old folder and thus the changes the had made after the backup were lost. This step is necessary, however, to maintain the synchronization between Active Directory and SYSVOL. Authoritative Restore–Subtree

9/11/2013 11:27 PM


24 of 40


This method of authoritative restore will restore specific components of Active Directory and mark them as authoritative for the directory. It is expected that this will be the most common form of authoritative restore because there are few occasions when the entire directory needs to be restored. To perform the authoritative restore of a subtree, follow the steps in the Authoritative Restore of the Entire Directory section of this paper, but replace Step 6 with the step below:

6. Type RESTORE SUBTREE <path> E.g. RESTORE SUBTREE OU=Sales,OU=Sydney,DC=Whitepaper,DC=com This server is now the authoritative Active Directory domain controller for the path specified. Changes will be replicated out to the other DCs within the domain. Because you are restoring only a portion of the Active Directory, you need not perform an authoritative restore of the SYSVOL. However, if the subtree or object that was authoritatively restored contained elements from the SYSVOL, such as a Group Policy, you should also restore that portion of the SYSVOL authoritatively. If this is required you must complete Step 10 in the Authoritative Restore of the Entire Directory section of this paper.

Note: You can authoritatively restore an individual object in the directory only if it is a leaf object. The object and its children are authoritatively restored together. Even if you restored a subtree of the directory authoritatively, you have to still undergo a complete restoration of the whole directory, using the non-authoritative method, before you can use the ntdsutil tool to mark that subtree authoritative. Verification of a Successful Restore Although there could be a number of tests to see if a domain controller is functioning correctly after a restore, the following are two of the basic tests one should perform to the success of a restore. Reboot in Normal Mode

If the domain controller is able to successfully boot into normal mode, the directory is able to successfully initialize. This is true especially if the DC wasn’t able to boot to normal mode before it was restored. Check the Directory Service Event Log for Any Error Messages

You should check the Directory and System event logs for any error messages pertaining to the restore process. Check if Domain Controller is Able to Authenticate With Its Neighbors

Use the rep tool to this. This is primarily used to check if the restored domain controller can authenticate with another domain controller and replicate in changes to update its copy of the directory. Being able to do so is critical to its task of being a domain controller.

The first check is to obtain the inbound partners for the restored domain controller. The options used are:

D:\>rep /showreps testlab\test-machine3 DSA Options : (none) objectGuid : a07b44e6-76ba-4f03-80c9-5a4a256347bb invocationID: 6037d0c3-2194-4f27-95ed-578b38861414

==== INBOUND NEIGHBORS ======================================

9/11/2013 11:27 PM


25 of 40


CN=Schema,CN=Configuration,DC=testdom,DC=nttest,DC=microsoft,DC=com testlab\test-machine1 via RPC objectGuid: 465848f9-5446-4176-a504-59629c7a8fd8 Last attempt @ 2000-09-08 14:10.16 was successful.

CN=Configuration,DC=testdom,DC=nttest,DC=microsoft,DC=com testlab\test-machine1 via RPC objectGuid: 465848f9-5446-4176-a504-59629c7a8fd8 Last attempt @ 2000-09-08 14:10.16 was successful.

DC=testdom,DC=nttest,DC=microsoft,DC=com testlab\test-machine1 via RPC objectGuid: 465848f9-5446-4176-a504-59629c7a8fd8 Last attempt @ 2000-09-08 14:10.16 was successful.

In the above case, the restored DC is test-machine 3. The test has been run on the restored machine, but the tool can be used on any domain controller. In the example above, test-machine 1 is its inbound partner.

The next command attempts to sync the schema naming context on the restored domain controller with changes from its inbound neighbor:

D:\>rep /sync <destination DSA> <source DSA GUID>

D:\>rep /sync "CN=Schema,CN=Configuration,DC=testdom,DC=nttest,DC=microso ft,DC=com" test-machine3 465848f9-5446-4176-a504-59629c7a8fd8 Sync from 465848f9-5446-4176-a504-59629c7a8fd8 to test-machine3 completed successfully.

You could then try the same command to sync the inbound neighbor with the restored domain controller to check if outbound replication is working:

D:\>rep /sync "CN=Schema,CN=Configuration,DC=testdom,DC=nttest,DC=microso ft,DC=com" test-machine1 a07b44e6-76ba-4f03-80c9-5a4a256347bb Sync from a07b44e6-76ba-4f03-80c9-5a4a256347bb to test-machine1 completed successfully.

If the sync is successful, then the domain controller is able to authenticate with a neighboring domain controller and receive changes from it. If not, then it could mean that the machine on the restored domain controller is old, and the inbound partner cannot authenticate the restored domain controller. You should use the netdom tool to set the on the inbound replica domain controller and restored domain controller to be the same. For more information on the netdom tool, see the Windows 2000 Tools on the Windows 2000 CD. Additional Verification for an Authoritative Restore

In addition to the above steps, you should that the objects you authoritatively restored appear in the directory.

9/11/2013 11:27 PM


26 of 40


You can also use the Rep command-line tool to that the authoritative restore was successful by checking the version number increase on the directory or subtree. Do this by carrying out the /showmeta command followed by the exact distinguished name of the directory or subtree that you authoritatively restored.

Recovery of a Global Catalog Server To recover the global catalog server you can either restore it from backup, or assign a new GC to compensate for the loss of the original. Steps Required for the Restore of a Global Catalog Server Refer to the section Restore From Backup in this paper. Restoring from backup is the only way that a DC (that was functioning as a GC at the time of backup) can automatically be restored to the role of GC. Restoring a DC using the Restore Through Reinstallation method will not automatically reinstate the GC role. While restoring a GC from backup, keep in mind that the time required is greater than for restoring a normal domain controller, in a multi-domain environment. Steps Required to Assign a New GC As there are no real detrimental effects in configuring multiple GCs you may wish to create a new GC in your environment if you anticipate an extended downtime for the GC that has failed. This would be particularly relevant if the community that was being serviced by the original GC no longer had access to a GC, or if the requirement for the GC service was significant in your environment, such as if you were running Exchange 2000. To enable a new GC, follow the steps outlined in the To enable or disable a global catalog section in the Windows 2000 Server Help. Note: Having multiple GCs in a forest increases the availability of the system at the expense of increased replication traffic and database size. If you do reinstall the failed DC and maintain its role as a GC, you may wish to remove any additional GCs you may have configured during its absence.

Recovery of an Operations Master Once an operations master (OM) becomes unavailable, the only way the role can be reinstated is by carrying out one of the following procedures: Restore the failed operations master from backup. ·

Seize the role to another DC within the environment. This must be done only when the original role holder will not be restored from backup.

Note: Restoring an OM server via re-installation will not restore its original role status. After re-installation however, the role could be gracefully transferred back from another DC holding the role. OMs are also referred to as FSMOs (Flexible Single Master operation) role holders. Seizing an Operations Master Role Seizing, or force transfer as it is sometimes referred to, is a process that is carried out without the cooperation of the original role holder. In other words, when the original role holder has suffered a disaster, you can seize the role, forcing it to be moved to another DC within the domain/forest. Although the process required to seize an OM role is similar for all 5 roles, the considerations around their seizure differ. These are discussed later in the white paper. Note: The graceful transfer of an OM role will not be discussed in this paper. If this process can be carried out, it means the original role holder is active and is not involved in a disaster recovery situation. Steps Required to Seize an Operations Master

9/11/2013 11:27 PM


27 of 40


The DC that will be seizing the role must be fully synchronized with respect to updates performed on the previous role holder. This is why it is strongly recommended that the standby role holders specified in your environment be within the same site and direct replication partners with the existing role holder. To ensure that the standby OM that you have selected for your environment is the most appropriate, use the Rep tool (included as part of the tools on the Windows 2000 CD) to check its status. To demonstrate this, suppose that server SYD01 is the operations master of domain Whitepaper.com.au, SYD02 is the specified standby OM, and MEL01 is the only other DC of domain Whitepaper.com.au. Type the following two commands: C:\>rep /showvector dc=whitepaper,dc=com,dc=au SYD02.whitepaper.com.au Sydney\SYD01 @ USN 4023 Melbourne\MEL01 @ USN 4087 C:\>rep /showvector dc=whitepaper,dc=com,dc=au MEL01.whitepaper.com.au Sydney\ SYD01 @ USN 4018 Sydney\SYD02 @ USN 5017 Because SYD01 was the originating operations master, these are the only Update Sequence Numbers (USN) we are concerned with. The USN on SYD02 (4023) is higher than the USN on MEL01 (4018), therefore SYD02 is more up-to-date than MEL01 and is the more appropriate candidate to assume the role. Now that you have determined the best candidate to take on the OM role, follow the steps below to seize the OM role: 1. Open a command prompt. 2. Type NTDSUTIL. 3. At the ntdsutil prompt, type: roles 4. At the FSMO maintenance prompt, type: connections 5. At the server connections prompt, type: connect to server . For example: connect to server syd02.whitepaper.com.au 6. At the server connections prompt, type: quit 7. At the FSMO maintenance prompt, type: seize . For example: seize schema master 8. At the popup window, click Yes to the seizure. 9. At the FSMO maintenance prompt, type: quit 10. At the ntdsutil prompt, type: quit Recovery of the Schema Master Before deciding whether to seize the schema master role, consider the following: Impact on Environment The first thing you must understand is the impact that a failed schema master will have on your environment. The main issues you will see are: Unable to Make Changes to the Schema

When the schema master is unavailable, changes to the Schema will not be possible. If changes to the schema are attempted, a message will be displayed such as the one illustrated in Figure 10 below.

9/11/2013 11:27 PM


28 of 40


Figure 10.

Note: The exact message displayed will depend on the method you are using to make the change. The message above was experienced when attempting to install Exchange 2000 while the schema master was unavailable. In most production environments, changes to the schema should be infrequent and planned well in advance, so a schema master outage should not pose any immediate problems. Considerations for Performing a Seizure On a Schema Master The primary consideration for deciding to seize the schema master role is the longevity or permanence of the outage. Because of the chance of duplicate schema alterations being propagated throughout the environment, a seizure of the schema master role should only be carried out if the failed role holder will never come back online. In most environments, due to both the infrequent requirement for the schema master role and the implications of a seizure, it is likely that you will live with the outage for the period of time it takes to restore the DC holding the role. However, if for some reason you require the immediate use of the schema master role or the original role holder will never be brought back into the Windows 2000 environment, a seizure can be carried out. Recovery of the Domain Naming Master Before deciding whether to seize the domain naming master role, consider: Impact on Environment The first thing you must understand is the impact that a failed domain naming master will have on your environment. The main issues you will see are: Domains Cannot Be Added to the Forest

When this master is not available, domains cannot be added to the Active Directory. Figure 11 below shows the error message received when attempting to run DROMO to add a domain when the domain naming master (Syd01.Whitepaper.com) is not available:

Figure 11.

In a stable production environment, this should not be a significant issue. But in a development, testing or growing production environment, the failure of this master can halt growth until it is restored or seized. Domains Cannot be Removed From the Forest

A similar message is received when attempting to run DROMO to remove a domain when the domain naming master (Syd01.Whitepaper.com) is not available. In that case, for example, the message will read: Binding to serve syd01.Whitepaper.com using the supplied credentials failed. The RPC server is unavailable. Again this should be of limited concern to the vast majority of environments.

9/11/2013 11:27 PM


29 of 40


Considerations for Performing a Seizure on a Domain Naming Master The primary consideration for deciding to seize the domain naming master role is the longevity or permanence of the outage. Because of the chance of duplicate domain alterations being propagated throughout the environment, a seizure of the domain naming master role should only be carried out if the failed role holder will never come back online. In most environments, due to both the infrequent requirement for the domain naming master role and the implications of a seizure, it is likely that you will deal with the outage for the period of time it takes to restore the DC holding the role. However, if for some reason you require the immediate use of the domain naming master role or the original role holder will never be brought back into the Windows 2000 environment, a seizure can be carried out. Recovery of the RID Master Before deciding whether to seize the RID master role, consider: Impact on Environment The first thing you must understand is the impact that a failed RID master will have on your environment. The main issues you will see are: Inability to Create Security Object

The primary issue that you will face in this situation is the inability to add any new security objects, such as s, groups and computers to the domain, resulting in the error message: Windows cannot create the object because: The directory service has exhausted the pool of relative identifiers. In addition, you will receive an error in the event log with an event ID of 16645 on the DC in which you attempted to create the object. The error will explain that the maximum number of identifiers allocated to this domain controller have been assigned. This issue will only surface once the RID pool (512 individual RIDs in a pool) on each of the domain controllers within the domain are depleted (as objects can be created on any DC within the domain). Therefore if you have a domain with 5 remaining DCs you could still theoretically have 2560 (5 x 512) RIDs available to you once the RID master fails. In a typical environment this would provide you with ample RIDs for the creation of security principles until the RID master was repaired/restored using the methods discussed earlier. If however you were in the middle of a mass creation of security objects or inter-domain object moves that required more RIDs than you have in your existing pools, a role seizure could be performed. Failure to Move Security Principles Between Domains

You will not be able to move security principals to a new domain if the RID master in the target domain is not operational. Unlike the above issue, all cross-domain moves would fail immediately, due to the unavailability of the RID master. Considerations for Performing a Seizure on a RID Master Performing a seizure on a RID master is not something you want to do without due consideration. Because of the risk of duplicate RIDs on the network, the server that originally hosted the RID master role should never be brought back online if the seizure is performed. Instead, the original role holder should be completely rebuilt before being introduced back into the production Windows 2000 environment. If, after understanding all the implications of a RID master seizure your situation still requires you to have an active RID master immediately, follow the steps given to seize an operations master role. Recovery of the PDC Emulator Before deciding whether to seize the PDC emulator role, consider: Impact on Environment The first thing you must understand is the impact that a failed PDCE will have on your environment. The main issues you will face are: Mixed Mode Environment

9/11/2013 11:27 PM


30 of 40


If the PDCE role suffers a disaster in a mixed mode environment where Windows NT Server 4.0 backup domain controllers are active, you will witness the same issues you did in Windows NT Server 4.0 when the native PDC was unavailable For example, if you try to ister the NT 4.0 domain using the native NT 4.0 manager for domains tool, you will receive the error message: Could not find domain controller for this domain. If you try to ister the Windows NT Server 4.0 domain using the native Server Manager tool, you will receive the error message: Cannot find the Primary DC for MYDOMAIN. You may ister this domain, but certain domain-wide operations will be disabled. Native Mode Environment

The issues that occur in a native mode environment will also be present (on the Windows 2000 side) in a mixed mode environment. The primary issues you will face when the PDCE suffers a failure in a Windows 2000 environment are: Possible increase in incidents of logon failure. If a is reset, for example when the forgets the and an resets it on a DC that is not the authentication DC, that must wait until the is replicated to the authentication DC before he or she is able to logon. Although the ’s local authentication DC will try and the PDCE to see if the has changed since the last replication, it will fail, because the PDCE is offline. Therefore, the authenticating DC will have no choice but to resort to its local copy of Active Directory, which will still reflect the original forgotten . In this situation you will see no obvious errors, either from the client side or the authenticating DC. Although this can be an issue, the problem can be easily overcome by making the alteration on the s authenticating DC. Error when attempting to edit a Group Policy Object. To help ensure that no data loss occurs during the editing of a GPO, the default DC for changes to a GPO is the one holding the PDCE role. Therefore if the PDCE is unavailable you will receive the message shown in Figure 12 below when trying to edit a GPO within the domain.

Figure 12.

This is a minor issue, and can be quickly rectified by selecting one of the options provided. A brief description of these options is listed below. The one with the operations master token for the PDC emulator. This option is obviously not possible when the role is unavailable. The one used by Active Directory Snap-ins. This uses the domain controller that Active Directory management snap-ins are using. It is the recommended option when the PDCE is unavailable. Use any available domain controller. The third, and least desirable option, allows the Group Policy snap-in to choose any available domain controller. When this option is selected it is likely that a domain controller in the local site will be selected. Note: The changes made here are only made for the single edit. In other words you will be asked this question every

9/11/2013 11:27 PM


31 of 40


time you try to edit a GPO when the PDCE master is unavailable. Considerations for Performing a Seizure on a PDC Emulator Master The role of the PDCE master is not quite as critical as those previously mentioned. Thus, the act of seizing the role does not have the ramifications of the others. If you choose to seize the PDCE role, it is not necessary for the original role holder to be completely rebuilt before it can participate in the Windows 2000 environment again. As a result, the decision to seize the PDCE role will have fewer implications to your environment and would generally be considered as standard practice in the event of a PDCE failure, particularly in a mixed mode environment. The only real issue to consider when seizing the PDCE role is if you are functioning in a mixed mode environment with Windows NT Server 4.0 BDCs. In order for the BDCs to be aware of the changes they will perform, a full synchronization of the Built-in groups with the new PDCE is required. Note: As the issues associated with the seizure of the PDCE role have less impact on the environment, Microsoft has allowed you to seize this role through the Active Directory s And Computers snap-in. To do this, go through the steps you would normally go through for a Role Transfer. To do this, refer to the Windows 2000 Resource Kit Distributed Systems Guide. When the originating PDCE is unavailable, you will see the dialog shown in Figure 13 below.

Figure 13.

Click OK and the role will be seized. Note: A forced transfer is equivalent to a seizure. In addition, you can seize the role using ntdsutil. To do this, follow the steps outlined in the Steps Required to Seize an Operations Master section of this white paper. Replace Step 7 with: 7. At the FSMO maintenance prompt, type: seize pdc Recovery of the Infrastructure Master Before deciding whether to seize the infrastructure master role, consider: Impact on Environment The first thing you must understand is the impact that a failed infrastructure master will have on your environment. The effect of an infrastructure master failure in your environment will be limited. It will not be visible to end s and will only affect s if there has been considerable group manipulation. These group manipulations will typically be in the form of additions and or renames. The only effect of an infrastructure master being down in this situation is a delay in these changes being referenced through the Active Directory management snap-ins. There are very few occasions where your environment could not deal without an infrastructure master for the period it takes to repair/recover the original. However, if you foresee a very long outage a seizure of the role is recommended. Considerations for Performing a Seizure on the Infrastructure Master The primary consideration around the seizure of the infrastructure master role is to ensure that the new DC is not a GC server, but that it has a good connection to a GC, ideally within the same site.

Summary

9/11/2013 11:27 PM


32 of 40


This paper is an attempt to bring together information regarding backup and restore of the Active Directory to help the better understand the issues involved. Using this paper, an can develop a disaster recovery plan for his Windows 2000 environment. For More Information For the latest information on Windows 2000 Server, check out our Web site at http://www.microsoft.com /windows2000 and the Windows 2000/NT Forum at http://computingcentral.msn.com/topics/windowsnt.

Appendix I

Database Integrity Testing There is a limited amount of integrity testing that can be performed against the database, both at the ESE (extensible storage engine) layer and at the DB layer. The layers are illustrated in Figure 14 below.

Figure 14. Active Directory functional layers

These tests could be time consuming. Depending on the urgency of the situation, you could either perform them or go ahead with a restore from backup. Some considerations before performing these tests, include: 1. If you can’t boot into Directory Service Restore mode, then you will not be able to perform these tests so you could ignore this section. 2. The error message in the event log, or the corresponding KB article, should tell you more on the nature of the problem and if an integrity check would resolve the issue. 3. This process could be time consuming. It depends on the size of your database, but could run into hours for large databases (1 GB and above).

The good thing about these tests is that you would not lose any data because of performing them (unlike a repair, which is described in Appendix II). These tests should be performed in the following order: 1. Soft recovery of the log files. 2. File integrity. 3. Semantic analysis. You have to be in Directory Services restore mode to continue further.

9/11/2013 11:27 PM


33 of 40


Performing a Soft Recovery of the Log Files In the event that the power source failed unexpectedly, you can perform a "soft" recovery of the log files. Because transaction data is written to the log files before it is written to the data files, you can re-run the log files to reproduce the effects the transactions would have had if they were made to the data file. The Recover command in the Ntdsutil command line tool is used to perform this "soft" recovery. All of the log files are scanned to ensure that all committed transactions are made to the database file. Soft recovery is performed automatically when the domain controller starts if the previous shutdown was not clean. Following is sample output of running the Recover command: C:\>ntdsutil ntdsutil: files File maintenance: Recover Executing Command: C:\WINNT\System32\esentutl.exe /r /8 /o /l"C:\WINNT\NTDS" /s" C:\WINNT\NTDS" /!10240

Initiating RECOVERY mode... Log files: C:\WINNT\NTDS System files: C:\WINNT\NTDS Performing soft recovery... operation completed successfully in 4.717 seconds. Spawned Process Exit code 0x0(0) If recovery was successful, it is recommended you run semantic database analysis to insure semantic database consistency as well.

Ensuring File Integrity By using the integrity command, you can detect low level (binary level) database corruption. The integrity command reads every byte of the data file. Therefore, depending upon the size of your database, the process might take a considerable amount of time. The integrity command also makes sure that the correct headers exist in the database itself and that all of the tables are functioning and are consistent. This is used while in Directory Services Restore mode. If errors are encountered, they are recorded on the log files. The length of time for the integrity command to complete its operation depends on the type of hardware you are using and the size of your directory database. (In testing environments, the speed of 2 gigabytes (GB) per hour was considered to be normal). However, when you carry out the command, an online graph displays showing the percentage completed. Following is a sample run of an integrity check by using the ntdsutil tool:

C:\>ntdsutil ntdsutil: files file maintenance: Integrity Opening database . Executing Command: C:\WINNT\System32\esentutl.exe /g "C:\WINNT\NTDS\ntds.dit" /! 10240 /8 /v /x /o Initiating INTEGRITY mode... Database: C:\WINNT\NTDS\ntds.dit Temp. Database: INTEG.EDB failed to get 515126 buffers checking database header

9/11/2013 11:27 PM


34 of 40


checking database integrity Scanning Status ( % complete ) 0 10 20 30 40 50 60 70 80 90 100 |----|----|----|----|----|----|----|----|----|----| checking SystemRoot SystemRoot (OE) SystemRoot (AE) checking system table MSysObjectsShadow MSysObjects . Name RootObjects rebuilding and comparing indexes checking table "datatable" (6) checking data ....................... checking long value tree (24) ... checking index "PhantomIndex" (125) . checking index "INDEX_000901FD" (122) checking index "INDEX_000900DE" (121) checking index "INDEX_00090089" (120) checking index "INDEX_00090573" (119) checking index "INDEX_00090073" (118) checking index "INDEX_00090571" (117) checking index "INDEX_0009056C" (116) checking index "INDEX_00090553" (115) checking index "INDEX_0009013A" (114) checking index "INDEX_00090138" (113) checking index "INDEX_00090330" (112) checking index "INDEX_00090030" (111) checking index "INDEX_00090013" (110) checking index "INDEX_00000013" (109) checking index "INDEX_0000000B" (108) checking index "INDEX_00000007" (107) checking index "INDEX_00000003" (106) . checking index "INDEX_00150003" (105) checking index "LCL_ABVIEW_index00000409" (104) checking index "INDEX_00090363" (103) checking index "INDEX_00090303" (102) checking index "INDEX_00090290" (101) checking index "INDEX_000901FF" (100) checking index "INDEX_000900DD" (99) checking index "INDEX_00090085" (98) checking index "INDEX_00090057" (97) checking index "INDEX_0009001C" (96) checking index "INDEX_000201CC" (95) . checking index "INDEX_000200D2" (94) checking index "INDEX_0002000D" (93) checking index "INDEX_0000002A" (92) checking index "INDEX_00000004" (91) checking index "NC_Acc_Type_Name" (90) checking index "PDNT_index" (89) .. checking index "INDEX_00090001" (88) . checking index "INDEX_000901F6" (85) checking index "INDEX_000902EE" (84) checking index "INDEX_000904E1" (83) checking index "INDEX_000201D5" (80) checking index "INDEX_000902BB" (77) checking index "INDEX_000903B4" (76) checking index "INDEX_000200A9" (75) checking index "INDEX_0009039D" (74) checking index "INDEX_0009039A" (73) checking index "INDEX_00090098" (72) checking index "INDEX_00090395" (71) checking index "INDEX_0009028F" (69) checking index "INDEX_00090582" (66) checking index "INDEX_00020078" (65) . checking index "INDEX_00020073" (62) checking index "INDEX_00090171" (60) checking index "INDEX_00090167" (58) checking index "INDEX_00090062" (56) checking index "INDEX_00090261" (55) checking index "INDEX_0009014E" (52) checking index "INDEX_0009014D" (51)

9/11/2013 11:27 PM


35 of 40

.

. . ..


checking index "INDEX_0009014C" (50) checking index "INDEX_00090147" (49) checking index "INDEX_00090141" (48) checking index "INDEX_00090140" (47) checking index "INDEX_0009012E" (42) checking index "INDEX_00020013" (39) checking index "INDEX_0009030E" (36) checking index "INDEX_00090008" (32) checking index "INDEX_00090202" (25) checking index "Ancestors_index" (13) checking index "DRA_USN_CREATED_index" (12) checking index "DRA_USN_index" (11) checking index "del_index" (10) checking index "INDEX_00090002" (9) checking index "NC_Acc_Type_Sid" (8) checking index "INDEX_00090092" (7) rebuilding and comparing indexes checking table "hiddentable" (16) checking data rebuilding and comparing indexes checking table "link_table" (14) checking data checking index "backlink_index" (15) rebuilding and comparing indexes checking table "MSysDefrag1" (123) checking data checking index "TablesToDefrag" (124) rebuilding and comparing indexes checking table "sdproptable" (17) checking data checking index "clientid_index" (19) checking index "trim_index" (18) rebuilding and comparing indexes

integrity check completed. operation completed successfully in 13.640 seconds. Spawned Process Exit code 0x0(0) If integrity was successful, it is recommended you run semantic database analysis to insure semantic database consistency as well.

Ensuring Database Integrity The Ntdsutil tool includes a semantics checker that can be invoked by selecting the semantic database analysis option. The role of the semantic checker is to check the integrity of the contents of the Active Directory database. The tool is run during Directory Service Restore mode. Errors are written into dsdit.dmp .xx log files. A progress indicator indicates the status of the check. The following are examples of the functions that can be performed: Reference count check. Counts all of the references from the data table and the link table to ensure they match the listed counts for the record. (For more information about data and link tables, see the section on Active Directory Data Storage in the Distributed Systems Guide of the Windows 2000 Resource Kit.) This also ensures that each object has a GUID, distinguished name and nonzero reference count. For a deleted object, the check ensures that the object has a deleted time and date, but does not have a GUID or a distinguished name. Deleted object check. Ensures the object has a deleted time and date, and a special relative distinguished name. Ancestor check. Checks to determine if the current distinguished name tag (DNT) is equal to the ancestor list of the parent and the current DNT. Security descriptor check. Checks for a valid descriptor, ensuring that it has a control field, and that the discretionary access control list is not empty. If there are deleted objects without a discretionary control

9/11/2013 11:27 PM


36 of 40


access list, a warning is printed. Replication check. Checks the UpToDate vector in the directory partition head to ensure that the correct number of cursors exists. It also checks to see that every object has property metadata vector. For the instance type of the object, it checks the metadata, the up-to-dateness vectors, the sub references, and partial attribute.

Performing Semantic Database Analysis 1. Back up Active Directory. Windows 2000 Backup natively s backing up Active Directory while online. This occurs automatically when you select the option to back up everything on the computer in the Backup Wizard, or independently by selecting to back up the System State in the wizard. 2. Restart the domain controller, select the appropriate installation from the startup menu, and press F8 to display the Windows 2000 Advanced Option Menu. 3. Select Directory Services Restore Mode, and then press enter. To start the boot process again, press Enter. 4. Log on by using the with the defined for the Local in the offline SAM. 5. From the Start menu, point to Programs and Accessories, and then click Command Prompt. 6. At the command prompt, type ntdsutil and then press Enter. 7. Type Semantic database analysis, and then press Enter. 8. Type Verbose on, and then press Enter. This displays the Semantic Checker. 9. Type go, and then press Enter. The Semantic Checker is started without repairing any errors it encounters. To repair the errors encountered, select the Go Fixup option. 10. Type quit, and then press Enter. To return to the command prompt, type quit again. Following is a sample of running the semantic database analysis option with verbose mode turned on: C:\>ntdsutil ntdsutil: Semantic database analysis semantic checker: Verbose on Verbose mode enabled. semantic checker: Go Opening database . ....Done. Getting record count...2371 records Writing summary into log file dsdit.dmp.0 Records scanned: 2300 Processing records..Done.

Appendix II

Database Repair Repairing the Active Directory should be the last resort. If a valid backup is available, you should always restore that backup. There is no guarantee that repairing Active Directory will work. In fact there is a risk that this process will result in further loss of data. In addition this could be a very time consuming process.

After doing a repair, a semantic integrity check of the database must be done as outlined in Appendix I.

9/11/2013 11:27 PM


37 of 40


To do a repair of the Active Directory, use the repair option in the ntdsutil tool:

C:\>ntdsutil ntdsutil: files ntdsutil: repair

Appendix III

Database: Location, Move, and Offline Defragmentation Determining the Location of Database Files and Log Files To find out the location of the data files, log files, and working directory, you can use the info command, which is part of the ntdsutil command-line tool. This command does the following: Analyzes and reports the free space for all disks installed on the computer. Reads the registry keys that the location of the Active Directory files and reports their values. Reports the sizes of the data file, working directory, and log file. Following is sample output from running the info command: C:\>ntdsutil ntdsutil: files file maintenance: Info

9/11/2013 11:27 PM


38 of 40


Drive Information:

C:\ NTFS (Fixed Drive ) free(2.9 Gb) total(3.9 Gb)

DS Path Information:

Database : C:\WINNT\NTDS\ntds.dit - 12.1 Mb Backup dir : C:\WINNT\NTDS\dsadata.bak Working dir: C:\WINNT\NTDS Log dir : C:\WINNT\NTDS - 40.0 Mb total res2.log - 10.0 Mb res1.log - 10.0 Mb REPAIR.TXT - 0.0 Kb edb00001.log - 10.0 Mb edb.log - 10.0 Mb

Moving the Database When you move the database from one location to another location on the disk, you can use the Ntdsutil command-line tool in Directory Services Restore mode. For example, you might need to move a log file or the Ntds.dit file to another drive if corruption occurs on the previously assigned drive or directory. Specifically, the move db to %s command moves the Ntds.dit data file to the new directory specified by the "%s" and updates the registry keys so that the directory service restarts by using the new location. It is highly recommended that you make a backup before and after the move or else the restore operation does not retain the new file location. You can also move the log files from one location to another. Specifically, the Move logs to %s command moves the directory service log files to the new directory specified by %s and updates the registry keys so that the directory service restarts by using the new location. Offline Defragmentation Active Directory automatically performs online defragmentation of the database at certain intervals (by default, every 12 hours) as part of the garbage collection process. Online defragmentation does not reduce the size of the database file (Ntds.dit), but instead optimizes data storage in the database and reclaims space in the directory for new objects. It prevents data storage problems. Performing offline defragmentation creates a new, compacted version of the database file. Depending on how fragmented the original database file was, the new file might be considerably smaller. To perform offline defragmentation: 1. Back up Active Directory. 2. Restart the domain controller, select the appropriate installation from the startup menu, and press F8 to display the Windows 2000 Advanced Options Menu. 3. Select Directory Services Restore Mode, and then press Enter. To start the boot process again, press Enter. 4. Log on by using the local 5. Click Start, point to Programs and then to Accessories, and then click Command Prompt. 6. At the command prompt, type ntdsutil, and then press Enter. 7. Type files, and then press Enter. 8. Type info, and then press Enter. This displays current information about the path and size of the Active Directory database and its log files. Note the path. 9. Establish a location that has enough drive space for the compacted database to be stored. 10. Type the following, in which and is the path to the location that you established in the previous step. and then press Enter: compact to :\

9/11/2013 11:27 PM


39 of 40


Note: You must specify a directory path. If the path contains any spaces, the entire path must be surrounded by quotation marks (for example, compact to "c:\new folder"). A new database named Ntds.dit is created in the path that you specified. 11. Type quit, and then press Enter. To return to the command prompt, type quit again. 12. Copy the new Ntds.dit file over the old Ntds.dit file in the current Active Directory database path that you noted in Step 8. 13. Restart the computer normally.

Appendix IV

Useful tools for Active Directory Disaster Recovery The following tools have been compiled as a recommended set of tools that are useful in a disaster situation. Note: To obtain tools, do one of the following: ·

Browse the Windows 2000 Server CD for the \\tools folder and run the installer program from there.

Or 1. Click Start, point to Programs, point to istration Tools, click Configure Your Server. 2. In the Configure Your Server Wizard, click Advanced, then click Tools. Follow the instructions. Windows 2000 Domain Manager (NetDom.exe) This tool enables s to manage Windows 2000 domains and trust relationships from the command line. This tool is included in the tools on the Windows 2000 CD. Replication Diagnostics Tool (Rep.exe) This tool allows the to view the replication topology as seen from the perspective of each domain controller. In addition, Rep can be used to force replication events between domain controllers and to view both the replication metadata and up-to-datedness vectors. This tool is included in the tools on the Windows 2000 CD. Active Directory Diagnostics Tool (Ntdsutil.exe) This tool is included with the Windows 2000 Server. It can be accessed from the command line by typing ntdsutil. The use of this tool is described throughout this white paper. Windows 2000 Backup Utility (Ntbackup.exe) This is the default backup application available with Windows 2000. You could use the tool to perform periodic backups of the System State and system drive. Details of tool usage are in the paper. In Windows 2000, Backup is located in System Tools on the Accessories menu. To open a system tools item, click Start, point to Programs, point to Accessories, point to System Tools, and then click the appropriate icon.

9/11/2013 11:27 PM


40 of 40


ADSI Edit ADSI Edit is a Microsoft Management Console (MMC) snap-in that acts as a low-level editor for Active Directory. Using Active Directory Service Interfaces (ADSI), provides a means to add, delete, and move objects within the directory services. The attributes of each object can be viewed, changed, and deleted. This tool is included in the tools on the Windows 2000 CD.

9/11/2013 11:27 PM

Active Directory Disaster Recovery 3s3c1r

Overview 4q3b3c

More details 26j3b

Related Documents 171j1w

Active Directory Disaster Recovery 3s3c1r

Active Directory 6b1t4u

Disaster Recovery Drill 5x3e4j

Practica Active Directory 2003 152i1l

Active Directory Audit Checklist 385l4d

Objetos En Active Directory 4q58u

More Documents from "Kumar Karan" 5q2u5s

Maria Yee 584323

Movement Types In Sap c6f58

Active Directory Disaster Recovery 3s3c1r

Arduino Severino Manual 4x2a1k

Www Tutorialspoint Com Java Java Encapsulation Htm 3l582k

Hans Kelsen Pure Theory Of Law 2w2a55