ARX Tiered Storage: Best Practices
Introduction
The ARX has many features for controlling file placement between the servers behind the ARX. ARX File Placement Policy (policy) has many configuration options that allow for fine tuning the behavior of rules. Under most circumstances keeping the configuration simple (using the default values) is the best approach.
This Tech Tip discusses some of the more common features used by place rules and explains how to use the features more effectively.
Problem Statement
When configuring policy it is important to understand a single rule does not make up a complete policy. A tiered storage policy is configured using two rules at a minimum and file placement can be affected by other rules in the volume. Understanding how the ARX places files requires a more holistic view of all rules associated with an ARX volume.
When a rule is configured without understanding its effect on the volume, unexpected behavior may result. The policy subsystem resolves conflicting rule configuration by giving rules precedence within the volume. Rules are applied in order of priority and once a matching rule is found, subsequent rules are ignored. For example, if rule A has highest priority and moves files ending in “.txt” to share 1, and rule B moves old files to share 2, then rule B will never move old files that end in “.txt”.
Adding additional configuration options to rules is also a common source of trouble when configuring a tiering policy. There are many options for rules providing for great flexibility, but in general the simple approach of avoiding optional parameters greatly improves the behavior of the rule and improves the efficiency of the policy subsystem.
Solution
Configuring Tiered Storage
When starting with a volume that does not already contain any tiering policy, it is best to start with the “Tiered Storage Wizard” in the ARX GUI. The “Tiered Storage Wizard” uses the best practices as the default options, so by using the wizard, complexity is automatically reduced in the resulting policy.
Figure 1: Common Operations
The wizard is located on the “Common Operations” page and is accessed via the button titled “Tiered Storage” shown in Figure 1.
Figure 2: Tiered Storage Wizard
The first page (Figure 2) asks for a prefix, namespace, volume and the number of tiers. The prefix will be used in naming all the rules that get created. The wizard creates 1 rule per tier in the specified namespace and volume.
Figure 3: Wizard Share Selection
The next pages (Figure 3) ask for shares or share-farms to use as each tier. The wizard asks for a share or share-farm once for every tier for the number of tiers set on the first page. In this example, the number of tiers is 2, so the wizard asks for 2 shares or share-farms.
Figure 4: Wizard File Matching Criteria
The next page (Figure 4) asks for the age criteria for when files should be moved and for a schedule (both of which are required). The optional section allows for further restrictions on what files should be included in the tiering by allowing an additional file set to be used in the matching criteria (circled). By default the Fileset option is unset and for basic tiering configurations it is not necessary to provide one. All the file are included in the tiering.
Figure 5: Wizard Summary
When the wizard is finished, a summary of the configuration is displayed (Figure 5). Once the “Finish” button is clicked, the wizard creates the rule and file set configuration. Any errors are reported to the user.
Figure 6: Fileset Summary Page
The file sets are visible on the file set page, by expanding the Policy menu tree, select Filesets and a listing of all the file sets is displayed (Figure 6). One file set is created for each tier.
Figure 7: Policy Summary Page
The new rules are visible on the policy page by selecting Policy from the menu tree (Figure 7). One rule is created for each tier.
Now we have a basic tiered storage configuration running in the volume. Let’s look at some of the configuration options available; from the policy summary page, click on a rule name, then the “Edit” button. This brings up the rule edit page (Figure 8):
Figure 8: Edit Place Rule Page For Tier-1 Rule
The “Source” option configures the rule to scan the specified share only. In most cases, the “source” option is not necessary. Before using this option it is important to understand how policy manages file system scans.
An individual rule does not perform file system scans directly. The scan is managed by the policy engine apart from rules so that the scans may be optimized in a tiered storage environment. Once a scan is requested there is a small (10 second) delay before the scan begins. This provides an opportunity for different rules to be incorporated into a single scan. For example, “daily” and “weekly” rules shares the scan on the day that the weekly schedule runs as long as the “daily” and “weekly” schedule run at the same time of day.
Rules maintain a scan version that is incremented whenever a rule is modified. In addition the scan version is incremented whenever a rule that uses an age file set is updated by a schedule. Rules use the scan version to determine when a new scan is necessary and request a scan when the version changes. During a scan, a rule maintains its own information about scan progress so a scan may be resumed after the rule is paused without interfering with another rule even if the two rules started together. When a rule resumes after being paused, the rule requests a new scan using its saved information to pick up from where it left off.
There is only 1 scan allowed for a single volume. Rule priority is not used to order scans since scans are not performed by the rule. Multiple share scans are serialized and performed in the order requested. Share scans are intended as an optimization for migrating entire shares or when scanning the majority of files located on other shares. If the environment calls for the scan of more than half the shares in a volume, a volume scan is recommended.
Rules configured to use a source share evaluate the file location as part of the file set. When a rule is initialized, the source share configuration is merged with the file set configuration. Wen a rule evaluates a file, the location is considered as part of the file set. These rules have an unexpected behavior of removing a matching file from its file set by migrating the file to the target share. Once the share is on the target, it no longer matches the rule because it is not on the source share.
Volume scans are the recommended method for configuring rules. The rule requests a volume scan when it is not configured to use a source share. When rules are modified or run by a schedule at the same time, the policy engine optimizes the scan so rules on the same volume and run at the same time share the scan, which lessens load on backend file servers.
Inline Notification
The next configuration option to look at on the rule edit page is the inline notification option, which is shown in Figure 9.
Figure 9: Edit Place Rule Page For Tier-2 Rule
Notifications are used by rules to detect when a client has accessed or modified a file.
The “last accessed” mode of an age file set causes rules to receive notifications when a file is either read from or written to, while the “last modified” mode only causes notifications to be sent when a file is written to.
Figure 10: Wizard File Matching Criteria
The mode of the file set was specified in the tiered storage wizard.
Inline notifications for non-tier-1 rules
Rules migrating older files to lower tiers of storage use inline notifications to detect when files have had their timestamps restored to a previous point in time. The rule immediately moves the file to lower tiers of storage and avoid using tier-1 storage when a backup is restored.
Because restoring backups is an infrequent event, the inline notification feature for non-tier-1 rules should be disabled. When notifications are enabled, non-tier-1 rules requests notifications for files on tier-1 for proper placement should the restore operation happen. This causes notifications to be triggered for modified or accessed files as part of normal operation, but since the file is already on tier-1, the notification is ignored unless the time stamp of the file is changed. This may generates significant load during peak client usage.
From the edit place rule examples above, the wizard disables notifications for the tier-2 rule (Figure 9) and enables them for the tier-1 rule (Figure 8).
Summary
When configuring policy, there are a couple of points to remember to help reduce unexpected behavior and allowing policy to run more efficiently.
- Keep it simple
- Use the wizard whenever possible.
- Keep the configuration of all rules in the volume in mind when adding or modifying rules. Use the namespace and volume drop down menus at the top of the policy summary page (Figure 7) to limit the displayed rules to a single volume.
Author Biography
Bill is a Principal Software Engineer at F5 Networks and team lead for the ARX policy subsystem. He has been Developing enterprise class and mission critical software for over 16 years, on the ARX software for 8 years, and has been the Policy team lead for 4 years.