archive instruction After receiving an archive request, immediately update the ccd_log in /data1/acs16/archive/archive_tools/logs as well as the web version in /data1/acs16/pub_html/ccd/index.content. This means adding comments into the ccd_log, archive request date, etc. Everything is listed by ccd serial number.
Go to each directory designated in the archive request. First, list the contents of that directory into a file in /data1/acs16/archive/archive_tools/practice. Make sure to list the contents complete with directory specification--it can be used later as a list for the collect script. A sample command would be
ls /data2/acs3/site/marco2/NEW/0201/* > 0201_080499
where the file is named according to the last four digits of the ccd serial number and the date of the archive request. Later you'll use this file to verify that no files are missing.
Now determine how much data is in each directory using the script collect. That script is located in /data1/acs16/archive/archive_tools (archive1). If you use the list in archive1/practice, then the collect command is something like
collect --verbose --list=0201_080499
The output of collect shows the name of each file, its size and the subtotal size of the directory up to that file. These sizes are quoted in bytes.
Use the collect output to determine which files to copy over to a working directory in /data2/acs16/archive/ccd_working. Remember that each cd can hold 650 Mbytes of data. Often a large directory is split up over two cd's--the goal is to use as much of a cd-rom as possible and keep good records of where everything is. The naming convention for working directories is the same as that for the cd's--ccd_###. As of 08-17-99 we're on ccd_033. The ccd_### directory is used as the base directory when writing the data to cd-rom, subdirectories are named according to the ccd serial number.
**At this point skip to directions for populating headers if the data was taken at JHU**
After you've collected as close to 650 Mbytes as you can, check each subdirectory to make sure the files were compressed with the most recent version of gzip. Such files have the suffix .gz. Any other files, either uncompressed or with a .z or .Z suffix must be recompressed. In general the gunzip command works on both .z and .Z files to uncompress them.
Now that all the files are in the correct format, start running the data through the idl script imgchk as a quality check. If there are any files that do not check out, talk to whomever sent the archive request about the possiblity of finding an uncorrupted version of the file.
The original of imgchk is in archive1. You can start it in IDL by typing imgchk once you've got the idl path set up correctly in your .cshrc file. You will be prompted for the variables.
Note the date that the images are checked in the ccd_log.
Proceed with writing data to cdrom--write two copies for ccd data.
After writing two copies of the cd, check each one to make sure it is named correctly (readvol) and also to see that the files are readable. To accomplish the second task, copy at least two randomly selected files from each directory on the cd into the archive1/practice directory. Run those files through imgchk. If there are any problems, talk to the author of the archive request and consider rewriting the cd. If there aren't any problems, delete the data files and proceed to step 10.
Do a final comparison check of filenames written to cd and those recorded originally in the practice directory. If everything checks out, delete the listfile. Finally send email back to the person who made the archive request in the first place. Be sure to copy that email to acs-archive for record keeping purposes.
After the CD has been verified to be complete and correct follow the instructions for populating the CCD database.
