If the /etc/opt/SUNWsamfs/archiver.cmd file is configured, it controls the operation of the archiver. No archiver.cmd file exists by default, but sample files are available in /opt/SUNWsamfs/examples.
In the archiver.cmd file the directives to control the behavior of archive requests can be set, and log files defined. Archive sets are declared in this file. In it, archive copies, their archive ages and the VSN associations for those copies are defined for each archive set. It is also possible to set various properties on files in archive sets in the archiver.cmd file. These will be discussed throughout the remainder of this document and those on releasing, staging and recycling.
This file can configure a very large number of SAM functions using statements called directives. In this paper we will focus on the creation of the archiver.cmd file and the most commonly used directives. The documentation provides details on many others.
|
Parameter |
Definition |
Default Value |
Configurable Values |
|
Archive interval |
The
period between the time an archive request is created and the time it is sent
to archive media. |
10
minutes for all file systems |
Any length of time. May be set globally on all
file systems or may be set individually for specific file systems |
|
Archive age |
The
minimum time elapsed after a file is created or modified, and before it is
archived. |
4
minutes |
Any length of time set on individual archive set
copies |
|
Archive copy |
A
copy of the files in an archive set made to archive media. Up to four archive
copies are supported |
One, copy 1 |
One
to four, any of copies 1,2,3 and 4 |
|
Archive log file |
A
file that records all archiving activity |
No
log |
May
be configured for all file systems or for each file system individually |
|
Archive media |
The
media - online disk, magneto-optical disk or tape, to which an archive copy
is sent |
Any
available |
Specific
VSNs, groups of VSNs or online disks |
|
Archive set |
A
group of files defined by size, location in the file system, by file name, or
by ownership. Each archive set has a unique number of archive copies defined,
with an archive age and VSN association for each |
The
archive set named for the file system and containing all files and
directories in the file system |
Any
archive sets, including the non-archiving archive set no_archive. |
All these defaults can be configured to other values in the /etc/opt/SUNWsamfs/archiver.cmd file. If you do not configure a value for archive interval, archive age, archiver log, archive copy number or archive set in the archiver.cmd file, the default will be used. The default of “any available media” cannot be used for VSN associations if the archiver.cmd file exists. If you create this file, you must specify the VSN association for every archive set copy, including those for the default archive set to which directories are archived, if you archive directories (and you should not.) The no_archive archive set does not require a VSN association.
The archiver.cmd file must be reread by the daemon sam-archiverd when changes are made. This happens automatically when the daemon is started at boot. Once the system is up and running, the command samd config can be used to reread the archiver.cmd file. The archiver -lv command checks the syntax of the archiver.cmd file and provides error messages about incorrectly configured directives, but does not cause sam-archiverd to reread the archiver.cmd file. Every time you configure or change the archiver.cmd file, you should run this command. If there are any errors in the archiver.cmd file, archiving will halt completely.
Syntax Rules for
the archiver.cmd File
The archiver.cmd file consists of directives, each of which is located on a single line. No offsets or indentations are required, although they can be used to make the file more readable. Lines contain one or more fields separated by spaces or tabs. Blank lines are ignored. Any text that appears after the pound sign character (#) is treated as a comment. To continue a line onto the next line, end the first line with a backslash.
Some directives used in the archiver.cmd file require units of time or size. Check the man page for the archiver.cmd file for details; it uses standard units such as “s” for seconds and “G” for gigabytes. Do not leave a space between the value and the units: i.e. 10m is 10 minutes and 10M is 10 megabytes.
The general outline of an archiver.cmd file is shown below.
#Comment
#Comment
Global archiving directive #1
Global archiving directive #2
Global Archive Set Assignment #1
Archive Copy Definition #1 for archive set #1
Archive Copy Definition #2 for archive set #1
…
fs = First_File_System
Archive Copy Definition #1 for default archive set
… up to four archive copies
Archiving Directive #1
Archiving Directive #2
…
Configured Archive Set Assignment #1
Archive Copy Definition #1 for archive set #1
…up to four archive copies
Archive Set Assignment #2
Archive Copy Definition #1 for archive set #2
…up to four archive copies
Repeat Archive Set Assignments as necessary
fs = Second_File_System
…
3. Advanced
Directives
start_keyword_for_advanced_directive
Advanced Directives
end_keyword_for_advanced_directive
4. VSN Association Directives
vsns
VSN Association for Copy #1 of First_File_System default archive set
VSN Association for Copy #2 of First_File_System default archive set
…
VSN Association for Copy #1 of Archive Set #1
…
endvsns
The archiver.cmd file can be divided into four sections:
1. Global directives are those that will apply to all file systems. They precede all other directives and typically consist only of archiving directives such as archivemeta=off, which stops archiving of metadata and which is routinely set for all file systems. It may also contain archive set directives for global archive sets. Global directives end when a file system directive or a keyword is reached.
2. File system configurations contain directives applying only to one file system. They immediately follow global directives in the archiver.cmd file. File-system configurations begin with a file system directive of the form fs =<file system name> and continue until a file system directive for a different file system or a keyword is reached. In order, they contain archive copy directives for the default archive set, if that is configured (and it should not be), archiving directives specific to that file system and archive set assignment directives followed by their archive copy definition directives. In case of conflicts, file system-specific directives override global directives.
3. Advanced directives are located after all file system configuration directives and just before VSN associations. They are surrounded by the appropriate keywords. Parameters that configure individual archive set copies are called archive copy parameter directives, and are located between the params and endparams keywords. Archive copy parameter directives are important and commonly configured to override default values for timing archive request closure and for setting up recycling. These will be discussed more later in this paper. For convenience, pools of VSNs may be configured between the keywords vsnpools and endvsnpools. Check the documentation for details.
4. VSN associations are customarily located at the end of the archiver.cmd file. They assign each archive set copy to a VSN. They are surrounded by the keywords vsns and endvsns.
|
Directive Type |
Functions |
General Format |
|
Archiving directives Applicability: global, per-file system |
Set general archiving behavior
Determines when archiving is done Configures log files Prevents directory archiving Sets the maximum size of writes to media Use scan-based archiving, where sam-arfind sleeps for the duration of the archive interval then performs archiving runs |
Variable = value interval = time logfile = path archivemeta = off archmax = tar file size method=scan |
|
File System directives Applicability: per-file system only |
Begins a section of the file containing directives to be applied to that file system, declares the default archive set for the file system |
fs = file_system_name |
|
Archive set assignment directives Applicability: global, per-file system |
Specifies an archive set name and the characteristics of files that are grouped into the set. Optionally sets archiving, releasing, staging and recycling parameters on files in that file system |
Includes archive set
name, base directory, and other optional criteria |
|
Archive copy definition directives Applicability: archive set assignment directives |
Determines how many archive copies are made, and at what archive age. Can also be used to set the time at which copies are unarchived (used for online disk copies). |
Lines begin with a digit (1, 2, 3, or 4) which indicates the copy number. The next value indicates the archive age, and a third value, if present, is the unarchive age. |
|
Archive copy parameter directives Applicability: to archive set copies only |
Used to set special parameters on individual copies of archive sets. |
Directives between keywords params and endparams |
|
VSN association directives Applicability: to archive set copies only |
Determines the destination media of all archive copies. |
Directives between these keywords: vsns and endvsns |
Archiving directives determine general archiving behavior. They may apply globally or per-file system Archiving directives set the archive interval configure log files and prevent directory archiving.. The location of the directive in the archiver.cmd file determines if the directive is global or file system specific. If the archiving directive is at the beginning of the file, in the section containing global directives, it will apply to all file systems. If it is placed after a file system directive, it will apply only to that file system.
To set the archive interval:
interval = time
Where time is the period during which files requiring archiving may be added to an archive request. At the end of the interval, the archive request will be closed and sent to archive media.
Sun Support no longer
recommends the use of the archive interval. It applies to entire file systems, while
archiving behavior is governed by archive sets which commonly span file
systems. Set archive request closure behavior using the directives ‑startage,
‑startcount and ‑startsize in the archive copy parameter directives
section of the archiver.cmd file.
Whether you configure the archive interval or the ‑startage parameter, specify an interval that is short enough that the loss of a disk will not cause the loss of critical information, but that is long enough that a reasonably large number of files will be archived.
To produce a log file that documents archiving action, use the parameter logfile as follows:
logfile = pathname
Where pathname is an absolute path, including the file name for the archive log. This file should go to a UFS file system, normally in /var/opt/SUNWsamfs. It will be automatically created. For security you may want to send it to an NFS mounted file system. Most customers create one logfile per file system to make them more manageable.
To prevent archiving of directories use:
archivemeta
= off
If you want a file system to archive with other than the defaults, you must include a file system directive in the archiver.cmd file at the beginning of any directives that will apply to that file system. File system directives have the format:
fs = file_system_name
Where file_system_name is the family set name of the file system from the mcf file. A file system with no file system directive in the archiver.cmd file will archive using global directives if any exist or defaults if they do not. Whether or not you specify any file system directives, if the archiver.cmd file exists, every archive copy specified or implied must have a VSN association, to tape, magneto-optical disk or online disk, or all archiving will halt.
The following example of the beginning of an archiver.cmd file shows the use of archiving and file system directives. (Note that this is an incomplete archiver.cmd file and would never pass the archiver -lv test!) It is designed to demonstrate the use of archiving directives globally and in file systems:
# Example: archiving Directives
# Global Directives
# Set the global archive interval to 20 minutes
# This archive interval will apply to all archiving file systems not specifying a different
# archive interval
interval = 20m 1)
archivemeta = off
# Declare file system samfs1
fs = samfs1 2)
# Set up an archive log file for samfs1
logfile = /var/opt/SUNWsamfs/samfs1-log 3)
# Declare file system qfs1
fs = qfs1 4)
# Set the archive interval for qfs1 to 10 minutes
interval = 10m 5)
In this archiver.cmd file:
1. The interval between sam-arfind scans of the file system samfs1 is 20 minutes, because that is the global value, and no other value was specified for samfs1. Archiving of directories is set to off for all file systems.
2. Two file systems are declared in this archiver.cmd file: samfs1 and qfs1. Other archiving file systems may be present on the server, but they will archive using the global interval and defaults.
3. Archive activity in the samfs1 file system is logged in /var/opt/SUNWsamfs/samfs1-log
4. Archive activity in the qfs1 file system is not logged because no global or file-system specific log file for qfs1 was declared
5. The archive interval of the file system qfs1 is 10 minutes, because values specified for file systems override global values
Any archiving file system on this server without a file system directive in the archiver.cmd file (for example a SAM-QFS file system called samfs2) would archive using an archive interval of 20 minutes and will not be logged. It would use the default archive set and archive age.
Users can group files into archive sets that share common characteristics, such as size, ownership, or directory location. Each archive set is then assigned copies, and an archive age for each of those copies. Like archiving directives, archive set assignments may also be declared globally at the top of the archiver.cmd file, or declared as part of specific file system configurations.
These rules apply to archive set assignments:
1. Every file and directory in an archiving file system will be archived with one and only one archive set.
2. Every file will be assigned to the first archive set whose criteria it matches.
3. File system specific configured archive sets have first priority, followed by globally configured archive sets, followed by the default archive set for the file system.
4. Directories are always archived as part of the default archive set named for the file system. They may not be assigned to a different archive set
5. Archive sets can include files from multiple SAM-FS and SAM-QFS software file systems, whether they are declared globally or per-file-system. Simply use the same archive set name globally or under one or more file system directives - anywhere you want files to archive as part of that archive set.
To define an archive set, specify the name of the archive set globally, or under one or more file system directives, followed by the characteristics that will define the members of the archive set for that location. The characteristics that define an archive set may vary from one declaration to another. Only the archive set name is constant. The statements defining archive sets use a syntax patterned after that of the find command:
archive_set_name path [search_criteria1 search_criteria2
...] [file_attributes]
For example:
big_files . -minsize 1M
o The maximum archive set name length is 29 characters.
o The acceptable characters are uppercase and lowercase alphabetic characters, the digits 0 through 9, and the underscore character (_). The first character must be an alphabetic character.
o Files in the archive set called no_archive are never archived.
o The -minsize search criteria selects files larger than the specified size.
o The -maxsize search criteria selects files smaller than the specified size.
o The -user search criteria selects files owned by the specified user ID.
o The -group search criteria selects files with the specified group owner.
o The -name search criteria selects files with names that match a pattern defined by a regular expression.
o For example, the archive set declaration: images . -name gif$ assigns to the archive set images all files in the file system that end with gif. The archive set declaration big_files memos,programs -minsize 2M assigns to the archive set big_files all files in the directories memos and programs of at least 2 megabytes.
Example Archive Set
Assignment
The following fragment of an archiver.cmd file expands on the previous example. It shows a fragment of an archiver.cmd file, including archiving directives and archive set assignment directives. This is not a complete archiver.cmd file, nor is it well designed. Well designed archiver.cmd files are available in /opt/SUNWsamfs/examples.
# Example: archive set assignment directives
interval = 20m
archivemeta = off
gif_files . -name \.gif$ 1)
# The backslash, above, causes the “.” to be interpreted literally
fs = samfs1 2)
logfile = /var/opt/SUNWsamfs/samfs1-log
no_archive scratch 3)
big_files . -minsize 2G 4)
marketing memos -group mrkt 5)
all . 6)
fs = qfs1 2)
interval = 10m
big_files . -minsize 1G 7)
all . 6)
This fragment of an archiver.cmd file shows:
1. All data files names ending in .gif are assigned to the globally defined gif_files archive set, unless they are assigned to an archive set in their file systems.
2. Files in samfs1 and qfs1 not archived with big_files and gif_files will be archived with the catch-all archive set “all”. No directories are archived because the directive “archivemeta=off” has been globally set.
3. Files in the directory called scratch in the samfs1 file system are not archived.
4. All regular data files larger than 2 gigabytes in the file system under the samfs1 file system mount point are assigned to the big_files archive set.
5. All data files in the memos directory whose owners belong to the mrkt group are assigned to the marketing archive set except those files that are larger than 2 gigabytes. Those have already been assigned to big_files.
6. All data files in the samfs1 and qfs1 file system are assigned to the “all” archive set, except for files assigned to other archive sets. The “all” archive set is a catch-all archive set. It does not include any search criteria, only the path “.”. It therefore includes any file in a file system. If the “all” archive set had been the first archive set definition in a file system, no other archive set would have contained any regular data files. As the last archive set archived, it can be used to make sure that no data files are overlooked for archiving.
7. Data files in the qfs1 file system of greater than 1 gigabyte are assigned to the big_files archive set. This is the same archive set that holds files of greater than 2 gigabytes in the samfs1 file system. These files will be archived to the same media as those in samfs1, but otherwise need have no features in common.
Each archive set declaration may be assigned one to four archive copy directives, each with specified archive ages and unarchive ages. Archive copy directives are optional, but if they are not set, one copy will be made of any file at an archive age of four minutes. In the archiver.cmd file, copy directives are located immediately below the configured archive set directive to which they apply. The default archive set named for the file system can and should have an archive copy directive assigned as well. It will be located immediately below the file system directive.
Each copy directive has the format:
copy_number [archive_age] [unarchive age]
The value copy_number is a number between 1 and 4 optionally followed by an archive age value and an unarchive age. The copy number indicates only that a copy known by that number should be made, and says nothing about the total number of copies to be made. Thus, if you specified a single copy directive for copy number 4 of the big_files archive set one copy of each file in the archive set would be made at the specified archive age and labeled copy 4 in command output and log files.
It is not particularly organized to make copy 4 your sole archive copy, but it can be done. A more reasonable set of copy directives for the big_files archive set is shown in the following example:
big_files . -minsize 2G
1 1m 1w
2 15m
3 1h
In the preceding example, three copies of each file in the big_files archive set are made. Copy 1 is made using the archive age of 1 minute, copy 2 using the archive age of 15 minutes and copy three using the archive age of one hour. Copy 1 is unarchived after one week. That means that a file in this archive set will be added to an archive request when modifications are 1 minute old. When modifications are 15 minutes old, the file is added to a different archive request. When modifications are one hour old, the file is added to yet another archive request. You are not required to specify an archive age for each copy; if you do not, the default archive age of 4 minutes is used.
Archive sets may span file systems, but the archive copies declared for an archive set after a file system directive are specific to each file system. You do not have to use the same archive copy directives in every file system using a given archive set. The following fragment of an example archiver.cmd file expands on previous examples to show how this works:
# Example: archive copy directives
interval = 20m
archivemeta = off
gif_files . -name \.gif$ 1)
1 10m 1)
2 20m 1)
#
fs = samfs1 2)
logfile = /var/opt/SUNWsamfs/samfs1-log
no_archive scratch
big_files images -minsize 2G 3)
1 1m 1w 3)
2 15m 3)
3 1h 3)
marketing memos -group mrkt 3)
1 20m 1w 3)
2 4h 3)
3 8h 3)
all . 4)
fs = qfs1 5)
interval = 10m
big_files . -minsize 1G 6)
1 25m 6)
2 3h 6)
all . 7)
In this example
1. The global gif_files archive set has two archive copies, assigned: 1 and 2. Copy 1 will be made at an archive age of 10 minutes, copy 2 at an archive age of 20 minutes
2. The default file system archive set samfs1 will not archive because archivemeta has been set to off, and all files have been assigned to a configured archive set.
3. Files in the big_files archive set in the samfs1 file system will be archived once they have met the archive ages of 1 minute, 15 minutes and 1 hour. The marketing archive set will be archived three times after the given archive ages. The first archive copy in both archive sets will be unarchived one week after the last time it is accessed.
4. One copy of each file in the “all” archive set will be made at an archive age of 4 minutes, because no archive copies are specified for this archive set, so it uses the defaults.
5. Any files in file system qfs1 not assigned to a configured archive set will be archived once with the default archive set at an age of four minutes because no "catch-all" archive set has been configured. However, no directories will be archived because archivemeta=off has been globally set.
6. The big_files archive set in the qfs1 file system will be archived at archive ages of 25 minutes and 3 hours.
7. One copy of each file in the “all” archive set will be made at an archive age of 4 minutes as above.
Each archive copy is sent a tape, a magneto-optical disk or a file system on an online disk. The default destination is any available media, but that default only applies if you have not configured an archiver.cmd file. Otherwise you must specify the media to which every archive set copy will be archived. This is true of copies you specify after an archive set definition, or the single copy 1 implied by a file system or an archive set directive that uses the defaults. If you fail to configure VSN association directives for every archive copy, the command archiver -lv will indicate errors, and archiving will stop. VSN associations are the only archiving parameter that will not use defaults if left unconfigured.
VSN association directives serve a number of purposes:
1. They allow different archive copies to be sent to different media for security. If a tape or magneto-optical disk fails, another copy of the file is available on a different tape or disk.
2. They place archive copies of files in the same archive set but from different file systems onto the same VSNs. The same archive set can be used in multiple file systems with different copy numbers and archive ages but all files from a particular copy of an archive set end up on the same media.
3. They allow the administrator to use specific VSNs for archive copies that will later be taken to storage. For example, the third archive copy of all archive sets could be sent to a single tape that is later removed to offsite storage.
4. They allow the administrator to assign the default archive set copies containing file system metadata to their own archive media. This is only necessary if directories are archived. Generally the samfsdump command should be used instead of archiving directories.
5. By default staging is performed from archive copy 1. VSN associations allow you to send the first archive copy to the fastest media, usually an online disk archive, for fastest staging.
All VSN association directives are placed between the vsns and endvsns keywords, each on its own line. The order of VSN association directives is not significant, and does not have to follow the order of archive copy directives in the archiver.cmd file. To specify where to archive a copy of a particular archive set, use the VSN association directive with the following format:
archive_set_name.copy_num media_type vsn_exp
For archive_set_name, specify the name of an archive set defined earlier in an archive set assignment directive.
For copy_num, specify the copy number of an archive copy declared for that archive set. This value must be one of 1, 2, 3, or 4.
For media_type, specify the media type, as defined in the mcf file, field 3, Equipment Type.
For vsn_exp, specify the VSN, or a regular expression that defines a group of VSNs to use for archiving. You can also use the expression “.*” to indicate any available media. If you use this expression, be sure that you also set the parameter “‑reserve set” on all archive set copies so they end up on different VSNs. Customers commonly divide all VSNs into two pools, one that stays in the tape library and one that goes to offsite storage, and configure VSN associations for onsite copies and offsite copies using these pools.
Online Disk Archiving
Archive copies may be sent to “disk VSNs.” A disk VSN is an existing directory in a file system. These directories may be on the same or another host. They may be UFS, VxFS or NFS file systems or other non-archiving file systems. Sending archives to an archiving file system is not a good idea. The directories should be on dedicated file systems, and each should be used only for archiving one archive set copy. The directory to which archiving will occur must be associated with a "disk VSN" in the /etc/opt/SUNWsamfs/diskvols.conf file. The disk VSN is a name selected by the administrator. It is not really a VSN, but it acts as one for the purposes of associating an online disk destination with an archive set copy.
Disk VSNs may also point to remote directories on server hosts running SAM. Those hosts must have at least one SAM-FS or SAM-QFS file system configured, and must authenticate the client host in the diskvols.conf file. The destination directory must exist. Remote online disk archiving transfers archives using port 7105 and the tcp protocol and is implemented on both the client and server end by the daemon sam-rftd.
As of Release 4.4 of SAM disk VSNs are associated with archive set copies the same way as are other VSNs. The media type is “dk.”
Example VSN Association
The following example shows VSN association directives in a complete archiver.cmd file:
# Example: VSN association directives
interval = 20m
archivemeta = off
gif_files . -name \.gif$
1 10m
2 20m
#
fs = samfs1
logfile = /var/opt/SUNWsamfs/samfs1-log
no_archive scratch
big_files images -minsize 2G
1 1m
2 15m
3 1h
marketing memos -group mrkt
1 20m
2 4h
3 8h
all .
fs = qfs1
interval = 10m
big_files . -minsize 1G
1 25m
2 3h
all .
#VSN associations
vsns
big_files.1 lt 100009 1)
big_files.2 .* 2)
big_files.3 lt 100011 2)
marketing.1 dk disk01,disk02 3)
marketing.2 .* 3)
marketing.3 lt 3000[0-9][0-9] 3)
gif_files.1 lt 4000[0-5][0-9] 4)
gif_files.2 lt .* 4)
all.1 lt 6000[6-9][0-9] 5)
endvsns
This example specifies:
1. Copy 1 of the big_files archive set is sent to the DLT tape with the VSN 100009. This copy includes files from the samfs1 and qfs1 file systems. These files have different characteristics - big_files in samfs1 is files greater than 2 gigabytes, while big_files in qfs1 is files greater than 1 gigabyte. Their archive copies are archived at different archive ages. The only thing they have in common is that they are archived to the same tape.
2. Copy 2 of the big_files archive set is sent to any DLT tape. Copy 3 is sent to VSN 100011.
3. Copy 1 of the marketing archive set is sent to the online disk archives with the disk VSNs “disk01” and “disk02”. Copy 2 is sent to any DLT tape. Copy 3 is sent to DLT tapes with VSNs between 300000 and 300099. Although 100 VSNs are assigned to copy 3, all the VSNs in the VSN association do not have to be present in the tape library. You can have only one or two of them if you want. This allows you to plan ahead to replace exported files.
4. Copy 1 of the gif_files archive set is archived on DLT tapes with VSNs 400000 to 400059. Copy 2 is sent to any VSN.
5. Copy 1 of the catch-all archive set “all” is archived on the DLT tapes with VSNs between 600060 and 600099. This archive set copy was never defined in the archiver.cmd file, but it still requires a VSN association.
The archiver.cmd file allows you to customize archiving in very complex ways. Some of this customization is done using advanced parameters. One group of advanced parameters are used to determine the archiving behavior of specific archive copies or of all copies of all archive sets. These are called archive copy parameter directives.
Archive copy parameter directives are placed in the archiver.cmd file after all file system-specific directives, and between the keywords params and endparams. Archive copy parameter directives can be used to determine recycling behavior for online disk archives. Disk archiving will be discussed at the end of this section. Parameters that apply to all archive sets can be defined in a line using the special archive set allsets which is applicable to all archive sets. All allsets directives must occur before those for specific archive copies. Parameters set for individual archive copies override parameters set by the allsets directives.
Each line in between params and endparams consists of an archive set name (or "allsets"), a copy number and those parameters that apply.
For each parameter, use the following format and values:
archive_set_name.copy_number [-param1 argument -param2 argument ...]
The value archive_set_name is the name of an archive set defined earlier in an archive set assignment directive.
The value copy_number is the archive copy number, and must be 1, 2, 3, or 4.
Parameters and arguments are specific to the properties to be set.
Once an archive request has been created, by default the archiver continues to add files to it as they reach their archive ages until the archive interval has elapsed. This can be overridden with the -startcount <number> -startsize <size> or -startage <time> parameters. Since these apply only to a particular archive set copy, the archive interval will continue to apply to any archive set copies on which these parameters have not been set. For example:
big_files.3 -startcount 2
specifies that an archive request for copy 3 of the archive set "big_files" will be closed and sent to media as soon as two files are added to the archive request. The archive interval is irrelevant to this archive set copy.
big_files.2 -startage 1h
specifies that an archive request for copy 2 of the archive set "big_files" will be closed and sent to media after one hour. More than one parameter can be specified:
all.2 -reserve set -startsize 500G -startage 2h
Copy 2 of archive set all will be sent to tape when the archive request contains 500G of data or after two hours, whichever comes first.
VSN associations for removable media work best when you know exactly how much tape each archive set copy will take up. That is not always possible and you can end up with a tape library containing some VSNs that are completely full and others that are empty because an archive set ended up having fewer files than you anticipated. The “reserve -set” directive allows you to overcome this problem. When it is attached to an archive set copy, once that archive set copy has written to a VSN, no other archive set copy can use that VSN. In the VSN associations, you can then send archive set copies to a large set of media using regular expressions, or even to any media, using the regular expression “.*” This allows flexible assignment of VSNs to archive set copies. This is a fairly clever and complicated configuration, but it is very widely used.
Advanced Parameters
Example
The following shows our example archiver.cmd file with archive copy parameters added:
# Example: Archive set parameter directives for disk archiving
interval = 20m
archivemeta = off
gif_files . -name \.gif$
1 10m
2 20m
#
fs = samfs1
logfile = /var/opt/SUNWsamfs/samfs1-log
no_archive scratch
big_files images -minsize 2G
1 1m
2 15m
3 1h
marketing memos -group mrkt
1 20m
2 4h
3 8h
all .
fs = qfs1
interval = 10m
big_files . -minsize 1G
1 25m
2 3h
all .
# Archive set parameter directives
params
allsets.2 -reserve set 1)
big_files.3 -startcount 2 2)
marketing.3 -startage 10m 3)
endparams
#VSN associations
vsns
big_files.1 lt 100009
big_files.2 lt .*
big_files.3 lt 100011
marketing.1 dk disk01,disk02
marketing.2 lt .*
marketing.3 lt 3000[0-9][0-9]
gif_files.1 lt 4000[0-5][0-9]
gif_files.2 lt .*
all.1 lt 6000[6-9][0-9]
endvsns
1) For copy 2 of all archive sets, once a VSN has been used by an archive set, no other archive set copy can use the VSN. Note that copy 2 of each archive set copy is allowed to use any VSN in the tape library.
2) Copy 3 of the big_files archive set will be archived as soon as two files are in the archive request, no matter how long that takes.
3) Copy 3 of the marketing archive will be archived when the archive request has been open for 10 minutes, no matter what the archive interval is.
A Better Example
The example archiver.cmd file above is poorly configured so don’t use it as a template. In reality, copy 1 should probably be sent to online disk archive for all archive sets. According to this archiver.cmd file, copy 2 of each archive set can use any VSN, and once it has used that VSN, no other archive set may use it. As a result it may use one of the VSNs assigned to copy 3 of marketing, after which copy 3 of marketing will not be able to use that VSN. If “-reserve set” is used on one archive set copy sent to tape in a single tape library, it should either be used on all copies sent to tape, or that archive set copy should be assigned to a set of VSNs which it may share with other archive set copies. There are numerous other problems with the file.
The following shows a better configured archiver.cmd file:
# Example: Archive set parameter directives for disk archiving
interval = 1h 1)
archivemeta = off
#gif_files . -name \.gif$ 2)
#1 10m
# 2 1h 3)
#
fs = samfs1
logfile = /var/opt/SUNWsamfs/samfs1-log
no_archive scratch
big_files images -minsize 2G
1 1m
2 15m
3 1h
marketing memos -group mrkt
1 20m
2 4h
3 8h
all .
fs = qfs1
interval = 10m
big_files . -minsize 1G
1 25m
2 3h
all .
# Archive set parameter directives
params
allsets.1 -startage 1h 1)
allsets.2 -reserve set -startage 1h 1)
allsets.3 -reserve set 4)
big_files.3 -startcount 2
marketing.3 -startage 10m
endparams
#VSN associations
vsns
big_files.1 dk disk01,disk02 5)
big_files.2 lt .*
big_files.3 lt .* 4)
marketing.1 dk disk03,disk04
marketing.2 lt .*
marketing.3 lt .* 4)
gif_files.1 dk disk05,disk06 5)
gif_files.2 lt .*
all.1 dk disk07 5)
endvsns
1. One hour is a more reasonable archive interval than 20 minutes for most installations, but in reality Sun Support recommends always overriding the archive interval with one or more of “‑startcount,” “‑startage,” and “‑startsize.”
2. Global archive sets are not commonly configured. You can get the same result with per-file-system archive sets, and that configuration is usually easier to read. Because this archiver.cmd file includes a “catch-all” archive set configured on each file system, this archive set never contained any files.
3. If copy 1 is made at an archive age of 10 minutes, it’s unlikely that copy 2 should be made at an age of 20 minutes. Therefore the archive age for copy 2 has been changed to a more reasonable 1 hour.
4. Archive sets sent to tape usually work better if they are not assigned specific VSNs, but instead if use is allowed to determine how many VSNs are assigned to an archive set copy. The VSN associations for all copies sent to tape can now be changed to “.*”
5. Copy 1 should usually go to an online disk archive.
A disk VSN is defined in the diskvols.conf file on one line with the following format:
VSN_name host_name:path
Where:
VSN_name is the administrator-selected name of the directory where archiving will occur.
host_name is the name of the host where the archive copies are written. This value is required only if original file and archive are on different servers. If files are to be sent to a different host, that host must have an entry in the diskvols.conf file permitting archives to be written to it.
path is the absolute path of the directory to which archive copies are written. If host is specified, a colon is required between host_name and path.
A pound character (#) indicates that all text to the right of the character is a comment, as shown in the following example:
# /etc/opt/SUNWsamfs/diskvols.conf file
#
disk01 /sam_arch1
disk02 gandalf:/sam_arch2/proj_1
These entries set two "disk VSNs" called disk01 and disk02. The disk VSN disk01 is associated with the directory /sam_arch1 on the local host, while disk02 is /sam_arch2/proj1 on the remote host gandalf. The host gandalf must be running at least one SAM-FS or SAM-QFS file system. These entries do not configure online disk archiving, they simply specify the names to use in the archiver.cmd file.
The remote system, in this case "gandalf," must have the following entry in the diskvols.conf file for the client system performing disk archiving:
clients
host_name
endclients
Any time this file is created or edited, samd config must be run.
Configuration
Solutions
This is a summary of commonly used how-tos and the issues they address. How to:
1. Avoid having to predict VSN use by archive sets: Use the “‑reserve set” parameter so that a VSN cannot be written by more than one archive set copy. You can specify overlapping sets of VSNs for different archive set copies without concerning yourself with separating different archive set copies onto different media.
2. Avoid having to edit the archiver.cmd file when VSNs are exported or imported to the library. The tapes listed in the VSN associations do not actually have to be in the tape library. There can be as few as zero, although that might cause a huge unwritten archive queue! That means you can place an inclusive list of all the VSNs you might ever want to use for an archive set copy in the VSN associations, so you never have to edit this section of the file. For example, if you want all copies 3 of all archive sets to be written to a single VSN, so you can export it and send it to offsite storage, you do not need to change the archiver.cmd file every day. You can simply import just one VSN from the list of VSNs in the VSN association every day when you export the VSN to go to offsite storage.
3. Fill a VSN completely before using the next VSN. SAM-QFS file systems are often used as a means of transferring data files to tape. The files placed on the file systems are never modified or deleted and tapes are exported and sent to storage shortly after archiving is complete. For such files, it makes sense to completely fill each VSN prior to using the next VSN. Filled VSNs can then be exported each day and sent to storage, without wasting space. The parameter “‑fillvsns,” set in the archive copy parameters, forces SAM to completely fill one VSN prior to using another, something it does not automatically do. The -fillvsns parameter may also be set on VSNs intended for offsite storage if there is more than one in the library at a time, to economize on tape use.
Filling each VSN is not automatically a good idea, because you may want more than one VSN to be used at a time for archiving or staging the files in an archive request. If your file system is heavily read and your files are typically released, you also want the archives spread over as many tapes as possible to increase the probably that two requests are on different VSNs. In the default writing behavior VSNs are filled to about 1/3 capacity, then another VSN is used. As all VSNs available to an archive set copy exceed 1/3 capacity, SAM begins the rotation again. For VSNs that will stay in the tape library the default is usually preferable.
4. Write tapes as efficiently as possible. The parameter “-fillvsns” uses tape economically, but does not require that it be written efficiently. One way to force efficient use of tape and drives is to combine the parameters that determine how long an archive request stays open. If you use the parameter “-startsize” and set it to the capacity of a tape cartridge, an archive request will close when one full cartridge worth of data has been accumulated. The entire tape will be written at one time requiring only a single load operation. The problem with using “-startsize” is that the archive request may take too long to accumulate a single tape worth of data. If you add the “‑startage” parameter, with a reasonable length of time, the archive request will close when it contains one full tape’s worth of data, or when it reaches the time value set with “‑startage,” whichever comes first.
The following example shows the configurations described above as they might be implemented in the archiver.cmd file. There are three archive sets, set1, set2 and set3. In each archive set, copy 1 of all three archive sets goes to tape and stays in the tape library. Copy 2 and copy 3 go to tape and are exported from the tape libr ary. Their archiving configuration might look like this:
…
params
allsets.1 -reserve set -startage 30m
allsets.2 -reserve set -startsize 500G -startage 4h -fillvsns
allsets.3 -startsize 500G -startage 10h -fillvsns
endparams
vsns
set1.1 li ON[A-Z,0-9] [A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set2.1 li ON[A-Z,0-9] [A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set3.1 li ON[A-Z,0-9] [A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set1.2 li ON[A-Z,0-9] [A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set2.2 li ON[A-Z,0-9] [A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set3.2 li ON[A-Z,0-9] [A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set1.3 li OFF[A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set2.3 li OFF[A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
set3.3 li OFF[A-Z,0-9] [A-Z,0-9] [A-Z,0-9]
endvsns
In this example, copy 1 and copy 2 of all archive sets go to any VSN starting with the letters “ON.” Copy 3 goes to any VSN that starts with the letters “OFF.” Copy 1 remains in the tape library so its archive copies are sent to tape using the default distribution. These copies are the primary backup, so they are sent to tape when an archive request has been open for just 30 minutes. Copy 2 is exported as VSNs fill. Ideally this happens with a single archive request, so the -startsize parameter is set to the capacity of a single tape cartridge. If an archive request doesn’t close in 4 hours, the archive request is written anyway. Copy 3 is exported once a day, but it is an offsite backup copy, so it isn’t urgent that it be written promptly. Archive requests close after 10 hours, so that these requests will be written at night when the system is less active. If a request has one cartridge worth of data, it’s written even if 10 hours have not passed since it opened. Both copy 2 and copy 3 are exported, so it makes sense to fill VSNs for economy. Copy 1 is recycled, so we must include the ‑reserve set parameter for all copies 1. Copy 2 isn’t recycled, and we probably don’t really need the ‑reserve set parameter, but for completeness and clarity it’s included. Copy 3 is placed on its own set of VSNs of which there will be a limited number in the tape library at any time, so the ‑reserve set parameter is not needed.