Data Retrieval Instructions

IRTF LA Search & Download page provides a script to download all the data found when searching the IRTF Legacy Archive database based on user parameters. The script utilizes GNU wget software, which will need to be installed already. This page provides information on how to use the use the retrieval script, and how wget can be used to download data by itself.

Running the Data Retrieval Script

The data retrieval script is generated on the fly when a new search query is run on Search & Download page. The retrieval Script will download all the FITS files shown on the results page into the directory the script is run from, e.g. irtf-download.

a button labeled is created at the bottom of the results page. Click the button to save the script;
save the script when prompted in a directory, say irtf-download;
run the script to fetch the data.

Run on Unix & Linux systems

Install wget on Mac OS via MacPorts (see the instructions for installtion of MacPorts itself). In Terminal, run:

sudo port install wget

Change to the directory in a terminal (xterm, console, Terminal, etc.) where the script was saved. Then run the script via a bourne shell (sh, bash, zsh, etc.) to actually fetch the data.

cd irtf-download
sh retrievalScript.sh

Run on Microsoft Windows

Install Wget for Windows: Download the "Complete package, except sources" Setup file, named wget-*-setup.exe ("*", wildcard, refers to the version number string). Then run wget-*-setup.exe to install wget.
Update the enviornment variable PATH to include the installation directory (e.g. C:\Program Files (x86)\GnuWin32\bin) reached via System Properties: Advanced: Envrionment Variables: System Variables dialog. This will allow command wget to be executed from any directory for all the users without specifying the full path, e.g. C:\Program Files (x86)\GnuWin32\bin\wget.exe.

Open cmd.exe window. Go in irtf-download directory. Rename the script to retrievalScript.bat. Then, run it.

cd irtf-download
rename retrievalScript.sh retrievalScript.bat
retrievalScript.bat

Using `wget` to Download Files

The data retrieval script utilizes wget to download FIPS data from the IRTF Data Archive Legacy Storage while preserving folder structure. For example the line generated to download s0376.a FITS file from from the 2001A semester is the following:

wget --no-clobber --force-directories http://irtfdata.ifa.hawaii.edu/idals/2001A/data/20010416/data/stockton/apr10/s0376.a

About the wget flags used on this page:

--no-clobber (or, -nc): local files will not be overwritten by downloaded files.
--force-directories (or, -x): maintain the directory structure when downloading the file to the local drive.
--no-parent (or, -np): when recursively fetching files, do not go up the directory tree.
--recursive (or, -r): recursively fetch files, up to default depth of 5.

Using wget to Download Directories

You can also use wget to recursively download entire contents of a directory using the recursive option, for example to download all the fits files in the /idals/2001A/data/20010416/data/stockton/apr10/ directory you can add the --recursive option and run the following command: