Overview of Archive Shuttle
This section of the guide gives an overview of Archive Shuttle including key terminology that is used throughout the documentation and user interface. It also presents architectural diagrams to aid with visualizing the components that are deployed in a typical migration.
Key terms
The following table introduces the terminology that is used throughout Archive Shuttle documentation, videos, and the user interface.
| TERM | ITEM + DESCRIPTION |
|---|---|
| Link | Enterprise Vault A Link is a Vault Store Exchange A Link is an Exchange database Office 365 A connection to Office 365 PST A connection to a PST Output Area Proofpoint A connection to a Proofpoint output area EAS A connection to an EAS IIS Server |
| Container | Enterprise Vault A Container is a Vault / Archive Exchange A Container is an Exchange mailbox Office 365 A Container is an Office 365 mailbox or Personal Archive PST A container is a PST file Proofpoint A container currently has no applicable context EAS A container is an archive relating to a user. |
| Item | Enterprise Vault An Item is an archived Item. Exchange An Item is an item in the mailbox Office 365 An Item is an item in the mailbox or personal archive PST An item is an item inside a PST file Proofpoint An item currently has no applicable context EAS An item is a message in the archive |
Architecture
The following diagram shows the Archive Shuttle components:
Archive Shuttle Core
The Archive Shuttle Core consists of three parts:
- Archive Shuttle User Interface
- Archive Shuttle Web Services
- Archive Shuttle Service
These are explained below.
Archive Shuttle User Interface (UI)
The main point of interaction with the administrator is through the web-based user interface. It allows access from anywhere within the enterprise by just using a web browser. It is even possible for archive migrations to be performed remotely from outside of an organization; for example, partner-managed migrations.
Using the wide range of options in the Archive Shuttle User Interface, the administrator can configure and manage archive migrations within an enterprise. The interface also provides at-a-glance progress monitoring, as well as reporting.
Archive Shuttle Web Services
All interaction between Archive Shuttle and Archive Shuttle Modules is done through the Archive Shuttle Web Services. All modules communicate with the Web Services using HTTP(s). If HTTPS is to be used, there is additional configuration that needs to be performed.
Archive Shuttle Service
The Archive Shuttle Service is a Windows Service which periodically executes Archive Shuttle scheduled tasks. These can be database maintenance tasks or other tasks that need periodic execution in Archive Shuttle itself.
Modules
Archive Shuttle has several modules that it uses to communicate with different systems to complete tasks. For best performance, modules are usually installed on the systems they interact with. Information relating to where each module should be installed is given in a later section of this document.
The following table describes each of the Modules which are available with Archive Shuttle:
| MODULE | DESCRIPTION |
|---|---|
| Active Directory Collector Module | The Active Directory Collector Module is responsible for resolving SIDs in AD and for retrieving all necessary data about a user (e.g., department) that might be necessary to filter data. It also collects information about the Exchange configuration of users (For example, on which Exchange Server / Database Availability Group (DAG) the user has a mailbox, if the mailbox has an Exchange Archive Mailbox enabled or not, and so on) |
| Enterprise Vault Collector Module | The Enterprise Vault Collector Module interacts with an Enterprise Vault Directory and is responsible for collecting all necessary metadata about Vault Stores, Archives, Items’ in each Archive and Enterprise Vault shortcuts in a mailbox. |
| Enterprise Vault Provisioning Module | The Enterprise Vault Provisioning Module creates new temporary archives for ingestion. It is able to map a temporary Archive to a User and can enable, disable, rename and ZAP existing Archives. This module utilizes EVPM to perform many tasks. |
| Enterprise Vault Export Module | The Enterprise Vault Export Module is responsible for exporting data from Enterprise Vault. It is multi-threaded and uses the Enterprise Vault APIs for accessing archived items. The exported data is stored on a Windows Network Share (SMB/CIFS). Please note that SMB 2 and SMB 3 are supported. |
| Enterprise Vault Import Module | The Enterprise Vault Import Module is responsible for ingesting data into the newly created or mapped archives in the target Enterprise Vault environment. It is multi-threaded and uses the Enterprise Vault APIs to ingest data. The data to import is read from the export location (SMB/CIFS Share). Please note that SMB 2 and SMB 3 are supported. |
| Enterprise Vault Shortcut processing Module | After migration has finished, the Enterprise Vault shortcuts in a users’ mailbox will be updated so that they refer to the migrated archive. Note: Non-shortcut items, such as calendar items, are also updated. |
| Exchange Import Module | The Exchange Import Module ingests data into a users’ mailbox. This can be the primary mailbox or an Exchange Personal Archive mailbox. |
| Metalogix Export Module | This module collects data from Metalogix Archive Manager and extracts data from that system. |
| Office 365 Module | The Office 365 Module processes data relating to migrations to Office 365. It collects metadata from Office 365, ingests items into Office 365 mailboxes or Personal Archives, and post processes the target to remove Enterprise Vault shortcuts and pending items. |
| Native Format Import Module | The Native Format Import Module allows for data to be exported to PST file format. The resultant files are stored on a configurable destination folder (UNC path) and the files can be split at predetermined sizes. The filenames can also be customized if required. |
| Proofpoint Module | This module takes exported source environment data and prepares the data for Proofpoint ingestion. |
| EAS Module | This module collects data from Zantaz EAS and extracts data from that system. |
| Sherpa Module | This module collects data from Sherpa Mail Attender and extracts the data from that system. |
| PST Export Module | This module collects data from PST files and extracts the data from them. |
| Dell Archive Manager Module | This module collects data from DAM and extracts the data from that system. |
| SourceOne Module | This module collects data from SourceOne and extracts the data from that system. |
Ports used by Archive Shuttle
Archive Shuttle Ports are described here.
Migration Workflow
Enterprise Vault to Enterprise Vault
Stage 1 – Synch
As soon as a user is enabled for migration, the synch process-flow shown above starts.
Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.
The Enterprise Vault Export and Enterprise Vault Ingest process is then started. It runs continuously in the background until the administrator initiates a “Switch” (stage 2) for the user or a specified “external” trigger is fired (e.g., the mailbox has been moved from Exchange 2003 to Exchange 2010)
Stage 2 – Switch
After the administrator has initiated a “Switch” (stage 2) to the target environment (per User), the Metadata Collector determines the gap between the source archive and the last imported items. Archive Shuttle then synchronizes the difference one last time to the target archive. The archive then gets assigned to the user and enabled. The last step is to cleanup shortcuts in the target mailbox.
Enterprise Vault to Exchange
Stage 1 – Synch
As soon as a user is enabled for migration, the synch process-flow shown above starts.
Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.
The Enterprise Vault Export and Exchange Ingest process is then started. It runs continuously in the background until the administrator initiates a “switch” (stage 2) for the user or a specified “external” trigger is fired (e.g., the mailbox has been moved from Exchange 2003 to Exchange 2010)
Stage 2 – Switch
After the administrator has initiated a “Switch” (stage 2) to the target environment, the Metadata Collector determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.
The last step is to cleanup shortcuts in the source mailbox by deleting them.
Enterprise Vault to Office 365
Stage 1 – Synch
As soon as a user is enabled for migration, the synch process-flow shown above starts.
Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.
The Enterprise Vault Export and Office 365 module then begin to work. This runs continuously in the background until the administrator initiates a “Switch” (stage 2).
Stage 2 – Switch
After the administrator has initiated a “Switch” (stage 2) to the target environment, the Metadata Collector determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.
The last step is to ‘Zap’ the mailbox to remove Enterprise Vault settings, cleanup shortcuts in the source mailbox by deleting them, and change any pending-archive items back to normal items.
Enterprise Vault to PST
Stage 1 – Synch
As soon as a user is enabled for migration, the synch process-flow shown above starts.
Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.
The Enterprise Vault Export and Native Format Import module then begin to work. This runs continuously in the background until the administrator initiates a “Switch” (stage 2).
PST files will be created by default of about 2 Gb in size. A lower size can be set in the UI if required
Stage 2 – Switch
After the administrator has initiated a “Switch” (stage 2) to the target environment, the Metadata Collector determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.
Archive Shuttle performs a ‘Zap’ of the mailbox to remove Enterprise Vault settings before closing the PST files and moving them to the PST Output Area. Finally, shortcuts are cleaned up in the source mailbox by deleting them, and any pending-archive items are changed back to normal items.
PST as a Source
Stage 1 – Synch
The first task which has to be performed is the discovery of the PSTs, followed by the assignment of those PSTs to target users.
Then when a PST is enabled for migration, the synch process-flow, shown above, starts.
Archive Shuttle Core sends a command to collect all needed metadata information from each PST file. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.
The PST Export and chosen target environment import module then start to process the items from the PSTs. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the arhcive.
Stage 2 – Switch
PST to Exchange or Office 365
PST to Enterprise Vault
After the administrator has initiated a “Switch” (stage 2) to the target, Archive Shuttle then synchronizes the difference one last time to the target environment. Depending on the target environment there then may be some additional steps (see the diagrams above).
EAS as a Source
Stage 1 – Synch
The first task which has to be performed is the discovery of archives in EAS. One or more of these archives are mapped and enabled for migration, Archive Shuttle Core sends a command to collect all needed metadata information from each archive. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.
The EAS module and chosen target environment import module then start to process the items from the archives. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the archive.
Stage 2 – Switch
EAS to Exchange or Office 365
EAS to PST
Dell Archive Manager as a Source
Stage 1 – Synch
The first task which has to be performed is the discovery of archives in DAM. One or more of these archives are mapped and enabled for migration, Archive Shuttle Core sends a command to collect all needed metadata information from each archive. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.
The DAM module and chosen target environment import module then start to process the items from the archives. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the archive.
Stage 2 – Switch
DAM to Exchange or Office 365
DAM to PST
SourceOne as a Source
Stage 1 – Synch
The first task which has to be performed is the discovery of archives in SourceOne. One or more of these archives are mapped and enabled for migration, Archive Shuttle Core sends a command to collect all needed metadata information from each archive. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.
The SourceOne module and chosen target environment import module then start to process the items from the archives. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the archive.
Stage 2 – Switch
SourceOne to Exchange or Office 365
SourceOne to PST
Planning component installation
Where to set up Archive Shuttle components
The core Archive Shuttle Components include the following:
- Archive Shuttle Core Web Interface
- Archive Shuttle Core Web Services
- Archive Shuttle Core Service
It is recommended to install these components on the same server. For a production environment, this must be a dedicated server (physical or virtual). They can; however, be installed on separate servers.
Please see the Chapter “Hardware requirements for Archive Shuttle Core Server” in the Installation Guide for more information about prerequisites.
Archive Shuttle also has the following Modules
- Archive Shuttle Active Directory Collector Module
- Archive Shuttle Enterprise Vault Collector Module
- Archive Shuttle Enterprise Vault Export Module
- Archive Shuttle Enterprise Vault Import Module
- Archive Shuttle Enterprise Vault Provisioning Module
- Archive Shuttle Enterprise Vault Shortcut Processing Module
- Archive Shuttle Exchange Import Module
- Archive Shuttle Office 365 Module
- Archive Shuttle Native Format Import Module
- Archive Shuttle EAS Module
- Archive Shuttle Sherpa Module
- Archive Shuttle PST Export Module
- Archive Shuttle Dell Archive Manager Module
- Archive Shuttle SourceOne Module
Depending on the migration scenario, some or all of these modules have to be installed.
The following table provides a guideline on where to install these modules:
| MODULE | NUMBER OF MODULES | NOTES |
|---|---|---|
| Active Directory Collector Module | One per Active Directory forest | Needs to be installed on a Server in the Active Directory Forest where data is to be collected from. The account that it is running under must have privileges to read Active Directory and its objects – a normal user account is sufficient. |
| Enterprise Vault Collector Module | Minimum: One per source Enterprise Vault environment Recommended: One per Enterprise Vault Server hosting a Vault Store. Note: If performing an EV to EV migration, this module is also needed in the target environment to gather site settings (HTTP/HTTPS) used when running ‘Fix Shortcuts’. |
There needs to be at least one Enterprise Vault Collector Module installed in the source Enterprise Vault environment. It is recommended to install this component near the SQL Server that hosts the Vault Store databases. Care should be taken when additional Enterprise Vault Collector Modules are installed. Performance might not necessarily be increased, the limiting factor is usually the time it takes to retrieve data from the Vault Store databases. |
| Enterprise Vault Provisioning Module | One per source Enterprise Vault environment, and one per target Enterprise Vault environment. | There needs to be one Enterprise Vault Provisioning Module per Enterprise Vault environment, i.e. per Enterprise Vault Directory Database. This module uses EVPM. It is recommended to install this component on the least busy Enterprise Vault server. |
| Enterprise Vault Export Module | Minimum: One per source Enterprise Vault environment Recommended: One per Enterprise Vault Server hosting a Vault Store. |
There needs to be at least one Enterprise Vault Export Module installed in the source Enterprise Vault environment. It is recommended to install the Enterprise Vault Collector module on each Enterprise Vault server that hosts Vault Store partition data, in the source environment. This module requires the Enterprise Vault API. If this module is to be installed on a non-Enterprise Vault server, then the Enterprise Vault API Runtime needs to be installed. |
| Enterprise Vault Import Module Note: This module is only required if the migration scenario includes migration to an Enterprise Vault environment. |
Minimum: One per target Enterprise Vault environment. Recommended: One per target Enterprise Vault Server hosting a Vault Store. |
There needs to be at least one Enterprise Vault Import Module per target Enterprise Vault environment, i.e., Enterprise Vault Directory. It is recommended to install an Enterprise Vault Import Module on each Enterprise Vault Server where the Vault Store to import to is hosted. This module requires the Enterprise Vault API. If this module is to be installed on a non-Enterprise Vault server, then the Enterprise Vault API Runtime needs to be installed. |
| Shortcut Processing Module | Minimum: One per target environment. | There needs to be at least one Shortcut Processing Module per target environment. This module must be installed on a Server with Microsoft Outlook 2007/2010/2013 (32 bit). Note: An optional change in the System Configuration can change this module to use EWS rather than MAPI (however the installation of the module still requires Outlook) |
| Exchange Import Module Note: This module is only required if the migration scenario includes migration to an Enterprise Vault environment. |
Minimum: One per target Exchange environment. | There needs to be at least one Exchange Import Module per target Exchange environment. This module requires Microsoft Outlook to be installed. It is not supported to install the module on an Exchange Server. It is recommended to install this module on a dedicated physical or virtual machine. To improve performance, multiple Exchange Import Modules can be installed. It is recommended to start with one module and add more modules (up to one per Exchange database) if required. |
| Office 365 Module | One per environment | There needs to be an Office 365 module in order to migrate containers to Office 365. This module collects data about Office 365 mailboxes, as well as migrating data into the target container and post-processing the ingested data to remove Enterprise Vault shortcuts and pending items. The module connects to Office 365 using the credentials specified in the Credential Editor, which is installed by the module. |
| Native Format Import Module | One per environment | One Native Format Import Module is required in order to take extracted data and create PST files. This module will split PST files at predetermined sizes and store them with a name defined by policy into a UNC accessible staging area which is defined per link. This module must be installed on a Server with Microsoft Outlook 2007/2010/2013 (32 bit). |
| Proofpoint Module | One per environment | One Proofpoint Module is required in order to take extracted data and prepare/construct the data required for Proofpoint ingestion. This module must be installed on a Server with Microsoft Outlook 64-bit. This module also requires a 64-bit JRE. Additional information is available in the Installation Overview. |
| EAS Zantaz Module | One per environment | One EAS Module is required on a machine which can communicate with an EASIIS Server which has access to all the required Document Repositories. |
| Sherpa Mail Attender Module | One per source server | One Sherpa Module is required per source server and it collects and extracts data from that server, for migration. |
| PST Export Module | One per environment | One PST Export Module is required, and can process a number of UNC paths to extract data from PST files which are located there. |
| Dell Archive Manager Module | One per environment | One Dell Archive Manager module is required for the source environment. |
| SourceOne Module | One per environment | One SourceOne module is required for the source environment. |
Planning for the Archive Shuttle Databases
Archive Shuttle uses the following Databases:
- The Archive Shuttle Directory database. There is just one of these; it hosts all configuration and non-item based metadata.
- The Archive Shuttle Item database(s). There is one of these for each source Link (e.g. one per Vault Store). These databases do not have to be on the same SQL Server as the Archive Shuttle Directory database. Each Item Database can be on a separate SQL Server, if required.
Microsoft SQL Server must be installed and set up before you install Archive Shuttle. The collation requirement for the SQL Server installation must be case-insensitive, accent-sensitive (CI, AS); case-sensitive and accent-insensitive installations are not supported.
Microsoft SQL Server must be on a dedicated server, either physical or virtual. It is not supported to have it on the same server as the Archive ShuttleCore server. It is not supported to have the Microsoft SQL Server shared with any other software, for production use.
Before installing the Archive Shuttle Core Components, make sure the account that will be used has “dbcreator” rights in Microsoft SQL Server.
SQL Server Version
The following versions of SQL Server are supported with Archive Shuttle:
| Version | Supported? |
|---|---|
| SQL Server 2005 | No |
| SQL Server 2008 | No |
| SQL Server 2008 R2 | No |
| SQL Server 2012 | Yes |
| SQL Server 2014 | Yes |
| SQL Server 2016 | Yes |
| SQL Server 2017 | Yes |
Note: Having the latest service pack installed is recommended.
SQL Editions
Although Enterprise Edition of Microsoft SQL Server is recommended, Standard Edition may be used if the SQL instance uses the recommended (not minimum) resources associated with the size of migration you are performing. Planning for additional time will be required to accommodate regularly required offline maintenance.
Storage Requirements for SQL Databases
Storage is required for the following SQL databases:
- Archive Shuttle Directory database
- Archive Shuttle Item databases
The Directory Database size requirement is 500 Mb. However, to allow for temporary transaction log growth, it is recommended to have at least 5 Gb is available for the database and logs.
Each item database has an initial storage requirement of 2 Gb; 1 Gb for the data file, and 1 GB for the transaction log.
The Archive Shuttle Item databases will grow depending on the items which are being migrated. As a guideline, each item in the item database is 1024 – 1500 bytes.
Planning Export/Import Storage
Sizing the staging location
The Archive Shuttle Export and Import Modules require a location to store the extracted data. Sizing of this location should be carefully considered.
Consider the following options when sizing the storage area:
- Local and Network Attached Storage is supported.
- Modifying the export/import schedules for the required modules can mean that the data to be migrated flows through the storage area with only a small amount being present at any time.
- Modifying the container mappings and ‘batching’ the migration will also lead to a smaller amount of data residing in the export/import location
Note: Export will stop if the free space on the staging area drops to 20 Gb or lower.
It is recommended to allow for between 50 and 100 Gb per source link. For example, with Enterprise Vault that would be 50 to 100 Gb per source Vault Store. Other sources may vary.
Note: If the migration will send the data to PST files, additional space may be required in the staging area since, by default, temporary PSTs are created in the staging area before they’re moved to the PST Output Path. The temporary location can be changed, or the space needed per source link should be increased to at least double the recommendations above.
The System Health dashboard provides an administrator with an overview of storage and other health parameters which may affect the migration being undertaken. The following information is displayed:
| Item | Description |
|---|---|
| Module Health | Displays information relating to all modules in the Archive Shuttle environment. |
| Unmapped Retentions | Any retention categories that are not mapped to a target environment are displayed here. |
| Link Health | Detailed information relating to each of the links will be displayed here. The links can be filtered so that only particular types of link are shown (e.g., Enterprise Vault, Exchange) This part of the dashboard will also show if an Enterprise Vault link is backup mode. This will affect any ingests to that Link (they will be suspended while the link is in backup mode) |
Free space on a staging area location is color coded as shown in the following table:
| Highlighting | Reason |
|---|---|
| No highlighting | Free space is above 100 Gb |
| Yellow | Free space is below 100 Gb |
| Red | Free space is below 50 Gb |
In addition, the System Health page gives an overview of any modules which are disabled or otherwise not running. This is shown at the top left of the screen in the Admin Interface.
The Link information at the bottom of the page also highlights the ‘Used’ space on each link. Used space is defined as the amount of data that has been exported, but not yet imported. This too is color coded for ease of identifying a potential problem, as follows:
| Highlighting | Reason |
|---|---|
| No highlighting | Used space is within normal parameters |
| Yellow | Used space is between 75% and 90% of the maximum allowed |
| Red | Used space is above 90% of the maximum allowed |
If the Used Space reaches 100% on a link, exporting of data will be stopped until data has been imported, even if there is still free disk space available on disk. This is further illustrated as follows:
Permissions/Access requirements for complex deployments
In complex environments crossing domains, organizations and forests, consideration must be given to the permissions / access requirements of the Archive Shuttle export and import modules that are required for the migration.
The basic requirement is that the Active Directory account that the export module runs under needs to be able to read/write to the storage location. In addition, the required import module also requires read/write access to the storage location.
The export/import storage location is specified as a UNC. Hidden shares are supported
Sizing the PST Output location
If Archive Shuttle is being used to extract data to PST files, the sizing of the PST Output location is important and should be carefully considered.
Consider the following options when sizing the storage area:
- It does not have to be locally attached storage to the Native Format Import Module. Network attached storage is supported.
- No data is written to the location until Stage 2 is activated for the chosen containers.
- Typically the space required for a PST file export of an archive will be almost double the size of the source archive.
PST files will be created by default of about 2 GB in size. A lower size can be set in the UI if required
Preserving the Chain of Custody
Archive Shuttle performs a number of operations in order to preserve and verify the Chain of Custody of data items during a migration. This section explains some of those which need to be taken into account in a migration.
Item level hashing
When an item is exported from a source environment. a hash is generated for that item and stored in the Archive Shuttle Item database. Sometime later, when the item is ingested, the hash is recomputed and compared with that stored value.
If the hashes do not match, a Chain of Custody violation is logged and the item will by default be re-exported and re-migrated.
If a significant number of Chain of Custody alerts are reported, it is likely that antivirus is touching the files after they have been stored on the staging area.
Migration-level hashing
As we are performing the hashing on the items as mentioned in the previous section, a hash is generated on the whole message file, in order to avoid tampering during the migration.
Watermarks
When migrating from EV to EV Archive Shuttle adds a custom index entry to migrated item. This entry contains:
- Migration Time UTC
- Source Archive ID
- Source Transaction ID
- Archive Shuttle Version
When migrating from EV to Exchange, Archive Shuttle adds the same information to each migrated item as MAPI properties.
Advanced Logging
The module-level log files that Archive Shuttle generates also track successful migrations when configured to track the module at TRACE level. This logging will show the Archive Shuttle reference for the item along with the source and target item ID values.
Each module logs activity that it performs to a local log file on the server where the module is running. These log files are kept up to a maximum of 10 files.
In addition, each module sends this logging information to the Archive Shuttle Core via a web service. Therefore, the log file is available in two locations.
Planning for Office 365 Migrations
Microsoft throttles connections and bandwidth usage to their cloud service – Office 365.
http://technet.microsoft.com/en-us/library/exchange-online-limits.aspx
There are some steps that can be taken to improve the throughput and performance when migrating to Office 365. These are described in this section. In addition it is recommended that Microsoft be contacted and a request made to increase the throttling limits whilst the data migration is being undertaken.
Planning for Migrations to Exchange
As with the ingestion in to Office 365, ingestion in to Exchange may be subject to throttling limits within Microsoft Exchange. A knowledge base article has been created which can help raise these limits.
By default the Exchange Import Module will attempt to ingest items into the chosen target mailbox or personal archive three times using AIP and then will fall back to trying Exchange Web Services (EWS). If they all fail, then the ItemRoutingErrorCount will be incremented, and the item will be counted as failed (and it will be visible on the Failed Items screen, and retried from time to time)
Planning for Migrations to PST
In order to perform a migration of Enterprise Vault data to PST, a PST File Name policy can be defined. The file name policy is defined with tokens, as shown in the table below:
| Token | Description |
|---|---|
| *username* | Username of the owning user (sAMAccount Name) |
| *firstname* | First name of the owning user |
| *lastname* | Last name of the owning user |
| *fullname* | Full name of the owning user |
| *email* | E-mail address of the owning user |
| *upn* | User principal name of the owning user |
| *pstid* | ID of the PST file; continuous integer over all PST files |
| *pstnumber* | Number of PST file; continuous integer per user |
| *archivename* | Name of the archive |
| *archiveID* | The Enterprise Vault Archive ID associated with the archive |
The tokens can be used to construct filenames and paths.
Planning for Migrations to Enterprise Vault
When migrating data to an Enterprise Vault environment, it is essential that the option in the provisioning group to “Automatically enable mailboxes” is turned off. If it is not turned off, there is the possibility that duplicate archives may be created in the target.
Tuning a Migration
There are many aspects of a migration that can be tuned with Archive Shuttle, some of these are manual tuning mechanisms, others can be automatic if required.
Scheduling Module Operation
Each Archive Shuttle module can be supplied with a specific schedule. For example, if extraction is allowed only during the evening, so as to lessen the load on the source environment, the export modules can be configured to have an evening-only schedule.
Configuring schedules should be done with careful consideration of the impact on the space requirements on the Staging Area. If exports are occurring all night long, but ingests are not scheduled, the staging area may fill up rapidly.
Tuning Module Parallelism
Each of the modules used in an Archive Shuttle migration operates on a number of containers in parallel and within that a number of items in parallel. The defaults for each module have been chosen following testing in many different environments. They can be adjusted to higher or lower values in the System Configuration. The changes made will take effect the next time the module checks in with the Archive Shuttle Core (approximately once per minute)
For reference:
Container level parallelism
For example, Office 365 Mailbox Parallelism. This is the number of containers that will be operated on simultaneously by the module
Item level parallelism
For example, EXC Import Item Parallelism. This is the number of items that will be operated on simultaneously per container.
When changing the parallelism, it is advised to make changes in small steps and then observe the impact that this has had on migration performance.
EV Ingest Specific Tuning
An option in the System Configuration allows the EV Import Module to pause if Enterprise Vault is in a scheduled archiving window. Pausing the ingestion during this time lessens the load on the target environment.
In addition, backup mode is checked per target Vault Store every 5 minutes.
When ingesting data the EV Import module will only send data to a Vault Store that is not in backup mode.
Finally, the number of items that are not yet marked as indexed in the target environment is captured per Vault Store once per minute. This is displayed on the System Health page in the Archive Shuttle Admin Interface.
A large index backlog might indicate that the target environment has a problem relating to indexing is indexing slowly
Native Format Import Specific Tuning
An option in the System Configuration allows the Native Format Import module to rename (i.e., move) temporary PST files from the Staging Area to the PST Output Path during Stage 1. Normally, the finalization of PSTs is not performed until Stage 2 is executed for a container mapping, but this can lead to increased pressure for storage requirements on the Staging Area.
Enabling the option in the System Configuration can mean that this pressure is reduced because completed PST files are moved out of the Staging Area and placed in their final location.
Office 365 Specific Tuning
Additional throughput can be achieved by the Office 365 module by setting up and configuring additional service accounts that have application impersonation rights within Office 365. These accounts can then be added to the credentials editor, and the Office 365 module will use them for ingesting data in a round-robin fashion.
More information on configuring multiple service accounts is provided earlier in this guide
Item Batching
The Exchange and Office 365 modules both utilize an intelligent batching mechanism which groups together items based on size, folder and whether the item has an attachment or not.
This can be disabled on a per-module basis, if required.
