Planning

Overview of Archive Shuttle

This section of the guide gives an overview of Archive Shuttle including key terminology that is used throughout the documentation and user interface. It also presents architectural diagrams to aid with visualizing the components that are deployed in a typical migration.

Key terms

The following table introduces the terminology that is used throughout Archive Shuttle documentation, videos, and the user interface.

TERM DESCRIPTION
Link Enterprise Vault

A Link is a Vault Store

Exchange

A Link is an Exchange database

Office 365

A connection to Office 365

PST

A connection to a PST Output Area

Proofpoint

A connection to a Proofpoint output area

EAS

A connection to an EAS IIS Server

Container Enterprise Vault

A Container is a Vault / Archive

Exchange

A Container is an Exchange mailbox

Office 365

A Container is an Office 365 mailbox or Personal Archive

PST

A container is a PST file

Proofpoint

A container currently has no applicable context

EAS

A container is an archive relating to a user.

Item Enterprise Vault

An Item is an archived Item.

Exchange

An Item is an item in the mailbox

Office 365

An Item is an item in the mailbox or personal archive

PST

An item is an item inside a PST file

Proofpoint

An item currently has no applicable context

EAS

An item is a message in the archive

 

Architecture

The following diagram shows the Archive Shuttle components:

Archive Shuttle Core

The Archive Shuttle Core consists of three parts:

  • Archive Shuttle User Interface
  • Archive Shuttle Web Services
  • Archive Shuttle Service

Archive Shuttle User Interface (UI)

The main point of interaction with the administrator is through the web-based user interface. It allows access from anywhere within the enterprise by just using a web browser. It is even possible for archive migrations to be performed remotely from outside of an organization; for example, partner-managed migrations.

Using the wide range of options in the Archive Shuttle User Interface, the administrator can configure and manage archive migrations within an enterprise. The interface also provides at-a-glance progress monitoring, as well as reporting.

Archive Shuttle Web Services

All interaction between Archive Shuttle and Archive Shuttle Modules is done through the Archive Shuttle Web Services. All modules communicate with the Web Services using HTTP(s). If HTTPS is to be used, there is additional configuration that needs to be performed.

Archive Shuttle Service

The Archive Shuttle Service is a Windows Service which periodically executes Archive Shuttle scheduled tasks. These can be database maintenance tasks or other tasks that need periodic execution in Archive Shuttle itself.

Modules

Archive Shuttle has several modules that it uses to communicate with different systems to complete tasks. For best performance, modules are usually installed on the systems they interact with. Information relating to where each module should be installed is given in a later section of this document.

The following table describes each of the Modules which are available with Archive Shuttle:

MODULE DESCRIPTION
Active Directory Collector Module The Active Directory Collector Module is responsible for resolving SIDs in AD and for retrieving all necessary data about a user (e.g., department) that might be necessary to filter data. It also collects information about the Exchange configuration of users (For example, on which Exchange Server / Database Availability Group (DAG) the user has a mailbox, if the mailbox has an Exchange Archive Mailbox enabled or not, and so on)
Enterprise Vault Collector Module The Enterprise Vault Collector Module interacts with an Enterprise Vault Directory and is responsible for collecting all necessary metadata about Vault Stores, Archives, Items’ in each Archive and Enterprise Vault shortcuts in a mailbox.
Enterprise Vault Provisioning Module The Enterprise Vault Provisioning Module creates new temporary archives for ingestion. It is able to map a temporary Archive to a User and can enable, disable, rename and ZAP existing Archives. This module utilizes EVPM to perform many tasks.
Enterprise Vault Export Module The Enterprise Vault Export Module is responsible for exporting data from Enterprise Vault. It is multi-threaded and uses the Enterprise Vault APIs for accessing archived items. The exported data is stored on a Windows Network Share (SMB/CIFS).
Enterprise Vault Import Module The Enterprise Vault Import Module is responsible for ingesting data into the newly created or mapped archives in the target Enterprise Vault environment. It is multi-threaded and uses the Enterprise Vault APIs to ingest data. The data to import is read from the export location (SMB/CIFS Share).
Enterprise Vault Shortcut processing Module After migration has finished, the Enterprise Vault shortcuts in a users’ mailbox will be updated so that they refer to the migrated archive. Note: Non-shortcut items, such as calendar items, are also updated.
Exchange Import Module The Exchange Import Module ingests data into a users’ mailbox. This can be the primary mailbox or an Exchange Personal Archive mailbox.
Office 365 Module The Office 365 Module processes data relating to migrations to Office 365. It collects metadata from Office 365, ingests items into Office 365 mailboxes or Personal Archives, and post processes the target to remove Enterprise Vault shortcuts and pending items.
Native Format Import Module The Native Format Import Module allows for data to be exported to PST file format. The resultant files are stored on a configurable destination folder (UNC path) and the files can be split at predetermined sizes. The filenames can also be customized if required.
Proofpoint Module This module takes exported source environment data and prepares the data for Proofpoint ingestion.
EAS Module This module collects data from Zantaz EAS and extracts data from that system.
Sherpa Module This module collects data from Sherpa Mail Attender and extracts the data from that system.
PST Export Module This module collects data from PST files and extracts the data from them.
Dell Archive Manager Module This module collects data from DAM and extracts the data from that system.
SourceOne Module This module collects data from SourceOne and extracts the data from that system.

Ports used by Archive Shuttle

The following table shows the network communication ports used by Archive Shuttle. These ports are provided for reference for the situation where a firewall exists between the source and target environments in the archive migration.

SOURCE Destination Port(s) Description
Module Servers Archive Shuttle Server TCP 80 (HTTP)

TCP 443 (HTTPS)

HTTP/S Communication from Modules to Archive Shuttle Servers
Export / Import Module Servers Storage Servers (CIFS Shares) TCP 445 (CIFS) Access to CIFS Shares
Archive ShuttleServer SQL Server TCP 1433 (MSSQL) Access to Microsoft SQL Server
Archive ShuttleServer DNS Servers UDP 53 (DNS) Access to DNS for Windows
Archive ShuttleServer Domain Controllers TCP 88 (Kerberos)

TCP 445 (CIFS)

Access to Domain Controllers for Windows
Enterprise Vault Module Servers Enterprise Vault Servers Please see your Enterprise Vault documentation on what ports are needed for the Enterprise Vault API to talk to Enterprise Vault
Exchange Module Servers Exchange Servers Please see your Microsoft Exchange documentation on what ports are needed to talk to Microsoft Exchange (EWS)
Office 365 Module Servers Office 365 TCP 443 (HTTPS) HTTPS communication from Archive Shuttle Office 365 Module to Office 365

 

Migration Workflow

Enterprise Vault to Enterprise Vault

Stage 1 – Synch

As soon as a user is enabled for migration, the synch process-flow shown above starts.

Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.

The Enterprise Vault Export and Enterprise Vault Ingest process is then started. It runs continuously in the background until the administrator initiates a “Switch” (stage 2) for the user or a specified “external” trigger is fired (e.g., the mailbox has been moved from Exchange 2003 to Exchange 2010)

Stage 2 – Switch

After the administrator has initiated a “Switch” (stage 2) to the target environment (per User), the Metadata Collector determines the gap between the source archive and the last imported items. Archive Shuttle then synchronizes the difference one last time to the target archive. The archive then gets assigned to the user and enabled. The last step is to cleanup shortcuts in the target mailbox.

Enterprise Vault to Exchange

Stage 1 – Synch

As soon as a user is enabled for migration, the synch process-flow shown above starts.

Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.

The Enterprise Vault Export and Exchange Ingest process is then started. It runs continuously in the background until the administrator initiates a “switch” (stage 2) for the user or a specified “external” trigger is fired (e.g., the mailbox has been moved from Exchange 2003 to Exchange 2010)

 

Stage 2 – Switch

After the administrator has initiated a “Switch” (stage 2) to the target environment, the Metadata Collector determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.

The last step is to cleanup shortcuts in the source mailbox by deleting them.

Enterprise Vault to Office 365

Stage 1 – Synch

As soon as a user is enabled for migration, the synch process-flow shown above starts.

Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.

The Enterprise Vault Export and Office 365 module then begin to work. This runs continuously in the background until the administrator initiates a “Switch” (stage 2).

 

Stage 2 – Switch

After the administrator has initiated a “Switch” (stage 2) to the target environment, the Metadata Collector determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.

The last step is to ‘Zap’ the mailbox to remove Enterprise Vault settings, cleanup shortcuts in the source mailbox by deleting them, and change any pending-archive items back to normal items.

Enterprise Vault to PST

Stage 1 – Synch

As soon as a user is enabled for migration, the synch process-flow shown above starts.

Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction ID’s for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.

The Enterprise Vault Export and Native Format Import module then begin to work. This runs continuously in the background until the administrator initiates a “Switch” (stage 2).

PST files will be created by default of about 2 Gb in size. A lower size can be set in the UI if required

 

Stage 2 – Switch

After the administrator has initiated a “Switch” (stage 2) to the target environment, the Metadata Collector determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.

Archive Shuttle performs a ‘Zap’ of the mailbox to remove Enterprise Vault settings before closing the PST files and moving them to the PST Output Area. Finally, shortcuts are cleaned up in the source mailbox by deleting them, and any pending-archive items are changed back to normal items.

PST as a Source

Stage 1 – Synch

The first task which has to be performed is the discovery of the PSTs, followed by the assignment of those PSTs to target users.

 

Then when a PST is enabled for migration, the synch process-flow, shown above, starts.

 

Archive Shuttle Core sends a command to collect all needed metadata information from each PST file. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.

 

The PST Export and chosen target environment import module then start to process the items from the PSTs. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the arhcive.

 

Stage 2 – Switch

PST to Exchange or Office 365

PST to Enterprise Vault

After the administrator has initiated a “Switch” (stage 2) to the target, Archive Shuttle then synchronizes the difference one last time to the target environment. Depending on the target environment there then may be some additional steps (see the diagrams above).

EAS as a Source

Stage 1 – Synch

The first task which has to be performed is the discovery of archives in EAS. One or more of these archives are mapped and enabled for migration, Archive Shuttle Core sends a command to collect all needed metadata information from each archive. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.

 

The EAS module and chosen target environment import module then start to process the items from the archives. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the archive.

 

Stage 2 – Switch

EAS to Exchange or Office 365

EAS to PST

Dell Archive Manager as a Source

Stage 1 – Synch

The first task which has to be performed is the discovery of archives in DAM. One or more of these archives are mapped and enabled for migration, Archive Shuttle Core sends a command to collect all needed metadata information from each archive. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.

 

The DAM module and chosen target environment import module then start to process the items from the archives. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the archive.

 

Stage 2 – Switch

DAM to Exchange or Office 365

DAM to PST

SourceOne as a Source

Stage 1 – Synch

The first task which has to be performed is the discovery of archives in SourceOne. One or more of these archives are mapped and enabled for migration, Archive Shuttle Core sends a command to collect all needed metadata information from each archive. The results are reported back to Archive ShuttleCore to allow item level tracking and auditing.

 

The SourceOne module and chosen target environment import module then start to process the items from the archives. This runs continuously in the background until the administrator initiates a ‘Switch’ (stage 2) for the archive.

 

Stage 2 – Switch

SourceOne to Exchange or Office 365

SourceOne to PST

Planning component installation

Where to set up Archive Shuttle components

The core Archive Shuttle Components include the following:

  • Archive Shuttle Core Web Interface
  • Archive Shuttle Core Web Services
  • Archive Shuttle Core Service

It is recommended to install these components on the same server. For a production environment, this must be a dedicated server (physical or virtual). They can; however, be installed on separate servers.

Please see the Chapter “Hardware requirements for Archive Shuttle Core Server” in the Installation Guide for more information about prerequisites.

Archive Shuttle also has the following Modules

  • Archive Shuttle Active Directory Collector Module
  • Archive Shuttle Enterprise Vault Collector Module
  • Archive Shuttle Enterprise Vault Export Module
  • Archive Shuttle Enterprise Vault Import Module
  • Archive Shuttle Enterprise Vault Provisioning Module
  • Archive Shuttle Enterprise Vault Shortcut Processing Module
  • Archive Shuttle Exchange Import Module
  • Archive Shuttle Office 365 Module
  • Archive Shuttle Native Format Import Module
  • Archive Shuttle EAS Module
  • Archive Shuttle Sherpa Module
  • Archive Shuttle PST Export Module
  • Archive Shuttle Dell Archive Manager Module
  • Archive Shuttle SourceOne Module

Depending on the migration scenario, some or all of these modules have to be installed.

The following table provides a guideline on where to install these modules:

MODULE NUMBER OF MODULES NOTES
Active Directory Collector Module One per Active Directory forest Needs to be installed on a Server in the Active Directory Forest where data is to be collected from. The account that it is running under must have privileges to read Active Directory and its objects – a normal user account is sufficient.
Enterprise Vault Collector Module Minimum: One per source Enterprise Vault environment

Recommended: One per Enterprise Vault Server hosting a Vault Store.

Note: If performing an EV to EV migration, this module is also needed in the target environment to gather site settings (HTTP/HTTPS) used when running ‘Fix Shortcuts’.

There needs to be at least one Enterprise Vault Collector Module installed in the source Enterprise Vault environment. It is recommended to install this component near the SQL Server that hosts the Vault Store databases. Care should be taken when additional Enterprise Vault Collector Modules are installed. Performance might not necessarily be increased, the limiting factor is usually the time it takes to retrieve data from the Vault Store databases.
Enterprise Vault Provisioning Module One per source Enterprise Vault environment, and one per target Enterprise Vault environment. There needs to be one Enterprise Vault Provisioning Module per Enterprise Vault environment, i.e. per Enterprise Vault Directory Database. This module uses EVPM. It is recommended to install this component on the least busy Enterprise Vault server.
Enterprise Vault Export Module Minimum: One per source Enterprise Vault environment

Recommended: One per Enterprise Vault Server hosting a Vault Store.

There needs to be at least one Enterprise Vault Export Module installed in the source Enterprise Vault environment. It is recommended to install the Enterprise Vault Collector module on each Enterprise Vault server that hosts Vault Store partition data, in the source environment. This module requires the Enterprise Vault API. If this module is to be installed on a non-Enterprise Vault server, then the Enterprise Vault API Runtime needs to be installed.
Enterprise Vault Import Module

Note: This module is only required if the migration scenario includes migration to an Enterprise Vault environment.

Minimum: One per target Enterprise Vault environment.

Recommended: One per target Enterprise Vault Server hosting a Vault Store.

There needs to be at least one Enterprise Vault Import Module per target Enterprise Vault environment, i.e., Enterprise Vault Directory. It is recommended to install an Enterprise Vault Import Module on each Enterprise Vault Server where the Vault Store to import to is hosted. This module requires the Enterprise Vault API. If this module is to be installed on a non-Enterprise Vault server, then the Enterprise Vault API Runtime needs to be installed.
Shortcut Processing Module Minimum: One per target environment. There needs to be at least one Shortcut Processing Module per target environment.

This module must be installed on a Server with Microsoft Outlook 2007/2010/2013 (32 bit).

Note: An optional change in the System Configuration can change this module to use EWS rather than MAPI (however the installation of the module still requires Outlook)

Exchange Import Module

Note: This module is only required if the migration scenario includes migration to an Enterprise Vault environment.

Minimum: One per target Exchange environment. There needs to be at least one Exchange Import Module per target Exchange environment. This module requires Microsoft Outlook to be installed. It is not supported to install the module on an Exchange Server. It is recommended to install this module on a dedicated physical or virtual machine.

To improve performance, multiple Exchange Import Modules can be installed. It is recommended to start with one module and add more modules (up to one per Exchange database) if required.

Office 365 Module One per environment There needs to be an Office 365 module in order to migrate containers to Office 365. This module collects data about Office 365 mailboxes, as well as migrating data into the target container and post-processing the ingested data to remove Enterprise Vault shortcuts and pending items.

The module connects to Office 365 using the credentials specified in the Credential Editor, which is installed by the module.

Native Format Import Module One per environment One Native Format Import Module is required in order to take extracted data and create PST files. This module will split PST files at predetermined sizes and store them with a name defined by policy into a UNC accessible staging area which is defined per link.

This module must be installed on a Server with Microsoft Outlook 2007/2010/2013 (32 bit).

Proofpoint Module One per environment One Proofpoint Module is required in order to take extracted data and prepare/construct the data required for Proofpoint ingestion.

This module must be installed on a Server with Microsoft Outlook 64-bit. This module also requires a 64-bit JRE. Additional information is available in the Installation Overview.

EAS Zantaz Module One per environment One EAS Module is required on a machine which can communicate with an EASIIS Server which has access to all the required Document Repositories.
Sherpa Mail Attender Module One per source server One Sherpa Module is required per source server and it collects and extracts data from that server, for migration.
PST Export Module One per environment One PST Export Module is required, and can process a number of UNC paths to extract data from PST files which are located there.
Dell Archive Manager Module One per environment One Dell Archive Manager module is required for the source environment.
SourceOne Module One per environment One SourceOne module is required for the source environment.

Planning for the Archive Shuttle Databases

Archive Shuttle uses the following Databases:

  • The Archive Shuttle Directory database. There is just one of these, it hosts all configuration and non-item based metadata.
  • The Archive Shuttle Item database(s). There is one of these for each source Link (e.g. one per Vault Store). These databases do not have to be on the same SQL Server as the Archive Shuttle Directory database. Each Item Database can be on a separate SQL Server, if required.

Microsoft SQL Server must be installed and set up before you install Archive Shuttle. The collation requirement for the SQL Server installation must be case-insensitive, accent-sensitive (CI, AS); case-sensitive and accent-insensitive installations are not supported.

Microsoft SQL Server must be on a dedicated server, either physical or virtual. It is not supported to have it on the same server as the Archive ShuttleCore server. It is not supported to have the Microsoft SQL Server shared with any other software, for production use.

Before installing the Archive Shuttle Core Components, make sure the account that will be used has “dbcreator” rights in Microsoft SQL Server.

SQL Server Version

The following versions of SQL Server are supported with Archive Shuttle:

Version Supported?
SQL 2005 No
SQL 2008 No
SQL 2008 R2 No
SQL 2012 Yes
SQL 2014 Yes
SQL 2016 Yes

 

It is recommended to have the latest service pack installed.

 

SQL Editions

Although Enterprise Edition of Microsoft SQL Server is recommended, Standard Edition may be used if the SQL instance uses the recommended (not minimum) resources associated with the size of migration you are performing. Planning for additional time will be required to accommodate regularly required offline maintenance.

Storage Requirements for SQL Databases

Storage is required for the following SQL databases:

  • Archive Shuttle Directory database
  • Archive Shuttle Item databases

The Directory Database size requirement is 500 Mb. However, to allow for temporary transaction log growth, it is recommended to have at least 5 Gb is available for the database and logs.

Each item database has an initial storage requirement of 2 Gb; 1 Gb for the data file, and 1 GB for the transaction log.

The Archive Shuttle Item databases will grow depending on the items which are being migrated. As a guideline, each item in the item database is 1024 – 1500 bytes.

Planning Export/Import Storage

Sizing the staging location

The Archive Shuttle Export and Import Modules require a location to store the extracted data. Sizing of this location should be carefully considered.

Consider the following options when sizing the storage area:

  • Local and Network Attached Storage is supported.
  • Modifying the export/import schedules for the required modules can mean that the data to be migrated flows through the storage area with only a small amount being present at any time.
  • Modifying the container mappings and ‘batching’ the migration will also lead to a smaller amount of data residing in the export/import location

Export will stop if the free space on the staging area drops to 20 Gb or lower.

 

It is recommended to allow for between 50 and 100 Gb per source Vault Store for a migration involving Enterprise Vault. For other source environments requirements will vary.

If the migration is to send the data to PST files, the additional space is required on the staging area as temporary PST files are created on there, before being placed on the PST Output Path. It is recommended to use double the normal size of staging area.

The System Health dashboard provides an administrator with an overview of storage and other health parameters which may affect the migration being undertaken. The following information is displayed:

Item Description
Module Health Displays information relating to all modules in the Archive Shuttle environment.
Unmapped Retentions Any retention categories that are not mapped to a target environment are displayed here.
Link Health Detailed information relating to each of the links will be displayed here. The links can be filtered so that only particular types of link are shown (e.g., Enterprise Vault, Exchange)

This part of the dashboard will also show if an Enterprise Vault link is backup mode. This will affect any ingests to that Link (they will be suspended while the link is in backup mode)

Free space on a staging area location is color coded as shown in the following table:

Highlighting Reason
No highlighting Free space is above 100 Gb
Yellow Free space is below 100 Gb
Red Free space is below 50 Gb

In addition, the System Health page gives an overview of any modules which are disabled or otherwise not running. This is shown at the top left of the screen in the Admin Interface.

The Link information at the bottom of the page also highlights the ‘Used’ space on each link. Used space is defined as the amount of data that has been exported, but not yet imported. This too is color coded for ease of identifying a potential problem, as follows:

Highlighting Reason
No highlighting Used space is within normal parameters
Yellow Used space is between 75% and 90% of the maximum allowed
Red Used space is above 90% of the maximum allowed

If the Used Space reaches 100% on a link, exporting of data will be stopped until data has been imported, even if there is still free disk space available on disk. This is further illustrated as follows:

Permissions/Access requirements for complex deployments

In complex environments crossing domains, organizations and forests, consideration must be given to the permissions / access requirements of the Archive Shuttle export and import modules that are required for the migration.

The basic requirement is that the Active Directory account that the export module runs under needs to be able to read/write to the storage location. In addition, the required import module also requires read/write access to the storage location.

The export/import storage location is specified as a UNC. Hidden shares are supported

 

Sizing the PST Output location

If Archive Shuttle is being used to extract data to PST files, the sizing of the PST Output location is important and should be carefully considered.

Consider the following options when sizing the storage area:

  • It does not have to be locally attached storage to the Native Format Import Module. Network attached storage is supported.
  • No data is written to the location until Stage 2 is activated for the chosen containers.
  • Typically the space required for a PST file export of an archive will be almost double the size of the source archive.

PST files will be created by default of about 2 GB in size. A lower size can be set in the UI if required

 

Preserving Chain of Custody

Archive Shuttle performs a number of operations in order to preserve and verify the Chain of Custody of data items during a migration. This section explains some of those which need to be taken into account in a migration.

Item level hashing

When an item is exported from a source environment. a hash is generated for that item and stored in the Archive Shuttle Item database. Sometime later, when the item is ingested, the hash is recomputed and compared with that stored value.

If the hashes do not match, a Chain of Custody violation is logged and the item will by default be re-exported and re-migrated.

If a significant number of Chain of Custody alerts are reported, it is likely that antivirus is touching the files after they have been stored on the staging area.

 

Migration-level hashing

As we are performing the hashing on the items as mentioned in the previous section, a hash is generated on the whole message file, in order to avoid tampering during the migration.

Watermarks

When migrating from EV to EV Archive Shuttle adds a custom index entry to migrated item. This entry contains:

  • Migration Time UTC
  • Source Archive ID
  • Source Transaction ID
  • Archive Shuttle Version

When migrating from EV to Exchange, Archive Shuttle adds the same information to each migrated item as MAPI properties.

Advanced Logging

The module-level log files that Archive Shuttle generates also track successful migrations when configured to track the module at TRACE level. This logging will show the Archive Shuttle reference for the item along with the source and target item ID values.

Each module logs activity that it performs to a local log file on the server where the module is running. These log files are kept up to a maximum of 10 files.

In addition, each module sends this logging information to the Archive Shuttle Core via a web service. Therefore, the log file is available in two locations.

Planning for extraction from Enterprise Vault

In some versions of Enterprise Vault it was possible to archive items above the Inbox, at the level in Outlook where it says: Mailbox – User. This is sometimes referred to as the Top of Information Store. Extracting data from this location is possible using the Enterprise Vault API, but it cannot be ingested into this location in any of the supported target environments.

These items can be migrated into a configurable folder. To specify the name of the folder perform these steps:

  1. Navigate to System Configuration
  2. Enter a value in the text box labelled: Subfolder for folder-less items
  3. Click Save.

If the folder does not exist and there are items which need to be ingested there for a particular container, the folder will be created.

This option will not create the folder in every target container. It will only be created if required in order to ingest data

 

Planning for Office 365 Migrations

Microsoft throttles connections and bandwidth usage to their cloud service – Office 365.

http://technet.microsoft.com/en-us/library/exchange-online-limits.aspx

There are some steps that can be taken to improve the throughput and performance when migrating to Office 365. These are described in this section. In addition it is recommended that Microsoft be contacted and a request made to increase the throttling limits whilst the data migration is being undertaken.

Multiple Service Accounts

Adding multiple service accounts allows Archive Shuttle to round-robin between them, meaning that it is less likely to encountered a throttled connection.

A service account needs to either be:

  • Office 365 Admin Account
  • An Office 365 Account with Application Impersonation rights

The Office 365 module will utilise the additional accounts if defined in the following manners:

If there is one mailbox to migrate and it contains 10,000 items, that will result in 10 import commands being generated for the module, the service accounts will be used in a round robin fashion per import command.

If there are 100 mailboxes with 10 items in each, that will result in 100 commands being sent to the module, the service accounts will also then be used in a round robin fashion per import command.

The additional accounts should be added using the Credentials Editor when logged in as the account that runs the Archive Shuttle Office 365 service.

Planning for Migrations to Exchange

As with the ingestion in to Office 365, ingestion in to Exchange may be subject to throttling limits within Microsoft Exchange. A knowledge base article has been created which can help raise these limits.

By default the Exchange Import Module will attempt to ingest items into the chosen target mailbox or personal archive three times using AIP and then will fall back to trying Exchange Web Services (EWS). If they all fail, then the ItemRoutingErrorCount will be incremented, and the item will be counted as failed (and it will be visible on the Failed Items screen, and retried from time to time)

Planning for Migrations to PST

In order to perform a migration of Enterprise Vault data to PST, a PST File Name policy can be defined. The file name policy is defined with tokens, as shown in the table below:

Token Description
*username* Username of the owning user (sAMAccount Name)
*firstname* First name of the owning user
*lastname* Last name of the owning user
*fullname* Full name of the owning user
*email* E-mail address of the owning user
*upn* User principal name of the owning user
*pstid* ID of the PST file; continuous integer over all PST files
*pstnumber* Number of PST file; continuous integer per user
*archivename* Name of the archive
*archiveID* The Enterprise Vault Archive ID associated with the archive

The tokens can be used to construct filenames and paths.

Planning for Migrations to Enterprise Vault

When migrating data to an Enterprise Vault environment, it is essential that the option in the provisioning group to “Automatically enable mailboxes” is turned off. If it is not turned off, there is the possibility that duplicate archives may be created in the target.

Tuning a Migration

There are many aspects of a migration that can be tuned with Archive Shuttle, some of these are manual tuning mechanisms, others can be automatic if required.

Scheduling Module Operation

Each Archive Shuttle module can be supplied with a specific schedule. For example, if extraction is allowed only during the evening, so as to lessen the load on the source environment, the export modules can be configured to have an evening-only schedule.

Configuring schedules should be done with careful consideration of the impact on the space requirements on the Staging Area. If exports are occurring all night long, but ingests are not scheduled, the staging area may fill up rapidly.

Tuning Module Parallelism

Each of the modules used in an Archive Shuttle migration operates on a number of containers in parallel and within that a number of items in parallel. The defaults for each module have been chosen following testing in many different environments. They can be adjusted to higher or lower values in the System Configuration. The changes made will take effect the next time the module checks in with the Archive Shuttle Core (approximately once per minute)

For reference:

Container level parallelism

For example, Office 365 Mailbox Parallelism. This is the number of containers that will be operated on simultaneously by the module

Item level parallelism

For example, EXC Import Item Parallelism. This is the number of items that will be operated on simultaneously per container.

When changing the parallelism, it is advised to make changes in small steps and then observe the impact that this has had on migration performance.

EV Ingest Specific Tuning

An option in the System Configuration allows the EV Import Module to pause if Enterprise Vault is in a scheduled archiving window. Pausing the ingestion during this time lessens the load on the target environment.

In addition, backup mode is checked per target Vault Store every 5 minutes.

When ingesting data the EV Import module will only send data to a Vault Store that is not in backup mode.

Finally, the number of items that are not yet marked as indexed in the target environment is captured per Vault Store once per minute. This is displayed on the System Health page in the Archive Shuttle Admin Interface.

A large index backlog might indicate that the target environment has a problem relating to indexing is indexing slowly

 

Native Format Import Specific Tuning

An option in the System Configuration allows the Native Format Import module to rename (i.e., move) temporary PST files from the Staging Area to the PST Output Path during Stage 1. Normally, the finalization of PSTs is not performed until Stage 2 is executed for a container mapping, but this can lead to increased pressure for storage requirements on the Staging Area.

Enabling the option in the System Configuration can mean that this pressure is reduced because completed PST files are moved out of the Staging Area and placed in their final location.

Office 365 Specific Tuning

Additional throughput can be achieved by the Office 365 module by setting up and configuring additional service accounts that have application impersonation rights within Office 365. These accounts can then be added to the credentials editor, and the Office 365 module will use them for ingesting data in a round-robin fashion.

More information on configuring multiple service accounts is provided earlier in this guide

 

Item Batching

The Exchange and Office 365 modules both utilize an intelligent batching mechanism which groups together items based on size, folder and whether the item has an attachment or not.

This can be disabled on a per-module basis, if required.

 

 

Print Friendly, PDF & Email
Updated on September 12, 2018

Was this article helpful?

Related Articles