PST Flight Deck is a solution designed for enterprise-scale migrations. Many of the actions of PST Flight Deck occur on a schedule or timed delay. In production, it is necessary to keep a longer delay for several actions to ensure the system does not get overloaded. When attempting to test or provide a demonstration of Flight Deck, these configurations can result in a lot more manual actions and longer timescales for your test or demonstration. This article is intended as a guide on how to tune a system for use in a demonstration or test environment with a limited number of highly controllable agents reporting back.
*** First It’s important to only use a similar configuration to this in a test or demonstration environment where there are no more than two dozen agents deployed. The more agents deployed, the longer the times recommended in an environment’s configuration. ***
Utilizing this configuration in a busy production environment can result in extreme performance degradation which will cascade into other issues. Please only use this for a test environment as described.
For a responsive system, it is important to perform configuration on the Migration Agent. Configuration for the Migration Agent is found in the PST Flight Deck Admin Console (FDAC) under Settings > Migration Agent.
The Discovery Agent within PST Flight Deck is the portion of the Migration Agent that finds PST files and reports them back to the Core component to be retained within the PST Flight Deck database. Under the File Scanner of the FDAC, there are two settings that can impact the speed that an individual can perform a test.
– Scan Delay
The maximum potential period of time (in hours) that the Discovery Agent process will wait prior to scanning a system for PST files. The actual time executed is a random time within this value. For testing, it is recommended that this value be set to “0”.
– Last Scan
The period of time prior to initiating the next scanning run by the Discovery Agent. The value is set to be dependent upon the extent of your scan and frequency of a systems content changing. In production, a scan is only executed once or twice a day, however, in a test system, this can be set as aggressively as 15 minutes if the scan targets are not complicated or large.
If testing, you may wish to create a folder exclusion to keep sample PST files that you do not wish to be discovered from doing so. It is also possible to force a workstation to perform a scan if you have access to the machine.
The agent is responsible for several operations on a user’s workstation. Perhaps the most important function of the agent component of the Migration Agent is that is responsible and able to query the Core server for information. It also passes work to the local service, facilitates the end-user experiences, and passes work to BITS for uploading. The agent configuration can be managed by accessing the Agent tab. There are two main configuration settings under the agent configuration options that can impact the speed an individual can perform a test:
– Polling Interval
The polling interval is a value in minutes that the agent “phones home” to see if there is work to be done. In production systems, this value may be set to occur after hours. This value can be set very low if there are limited agents reporting back. A value of “5” should be fully sustainable in a test environment with a few agents. If there are dozens reporting back a value closer to “10” may be more appropriate. It is possible to bypass the polling interval with appropriate access to a workstation with the agent installed.
– Initial Sleep
This is a setting in minutes to tell the agent how long to wait prior to doing anything after starting up. The feature is designed to permit adequate time after user login to enable all required resources to be aligned and available prior to starting any actions. In a controlled environment, setting this value to a low figure like “3” minutes should be adequate.
The specific module configuration of a system will vary from environment to environment. After files have been uploaded, the file begins to traverse the modules as configured in the FDAC under Settings > Workflow. Each module has its own configuration which frequently consists of the Update Interval as well as the number of items that will be processed. In non-production systems, you can configure the update intervals of the services to a fairly aggressive setting perhaps “5” minute value. It is typically recommended to increment the value of each module to prevent all modules from executing at the same time.
In a test system where there are no other PST files being processed and the operator has access to the PST Flight Deck server, it is frequently easier to restart the next module’s service to force a request for work. Prior to restarting a module’s service, it is important to note the work being performed by that module. Restarting a service which is presently processing files will likely result in those files failing to successfully process in that module.
There are a few special considerations when processing items for an end to end tests through the modules:
The Park module is used to move ownerless files from the active working area into an alternate storage location while ownership is decided. This behavior is governed by configuration settings within your environment however it is possible that a file being tested gets “Parked” and will require special remediation prior to progressing through the tests.
After a file is successfully processed by the Extraction module, it is no longer in a PST file format and the content has been prepared for ingestion. It is possible that some items fail to successfully extract. If the number of items that failed to extract is greater than the value configured “Failed Item Threshold” as shown below, the file will enter a “Partially Extracted” status and will require remediation to continue testing.
Some modules intentionally delay the processing of files as the default configuration or part of the design. In some instances, these delays can be configured and in other instances, they are hardcoded and can not be altered.
Generally speaking, the deduplication module is not used in aggressive or initial testing. The deduplication module is configured by default and by design to wait for all of a single user’s files prior to evaluating the files for deduplication. If all of a user’s files have not cleared the upload module and all modules prior to the Deduplication module, processing of that user’s files will be delayed until the DeDupe Timeout (as set in Settings > Environment > Advanced section of the FDAC) has been superseded or all files become ready to be processed by the Deduplication module. If testing deduplication, you may disable the feature to “Wait for all files” in Settings > Workflow, selecting the “Deduplication” module and in the window resulting from hitting the Edit button deselect the check. Use caution when changing this setting and only change it depending on the details of your test. If migrating all files for a test user, no modifications to the Deduplication Module is necessary.
All ingestion modules (except Enterprise Vault) have a configuration to limit the number of items processed by one user at a time. These changes were implemented to work around some of the throttling limits introduced by specific targets. For this reason, the ingestion module may only process a single file from a given user at a time. To maximize the speed of testing, several PST files associated with several users are best, real performance testing requires 50-100 users. Frequently, this is challenging to achieve during a test phase.
– Source File Remover
The Source File Remover module is different from the prior modules. It does not run as a service locally but rather is controlled by a scheduled task executed by the Core service. By default, there is a delay built into the SFR executing to permit an opportunity for review and local caches to be populated prior to removing the file from the customer’s machine. Once the delay is met, the request is queued for delivery on the next time the agent where the PST file was discovered checks in with the Core server. After this check-in, the request to delete the source file is sent to the agent and executed.
To ensure the Source File Remover module processes promptly, you will need to remove the “Delay in minutes” value. The value can be found in Settings > Workflow, by selecting the “Source File Remover” module, in the resulting window selecting the Edit button, and setting the value for “Delay in Minutes” to “0”. Once a file has cleared the CleanUp module, you can manually execute the scheduled task and force a poll of the Migration Agent to occur to expedite this final stage of testing.
Upon testing completion, you should revert all these settings to a value appropriate for the environment prior to using the system for any production migration. Having a system aggressively tuned for testing can have a negative impact when a lot of users or data are engaging the system. Please only use this configuration for testing and ensure that after the testing is completed a more standardized group of settings is used.