This page describes the process of downloading, installing, configuring, and testing the current ETD-db distribution ("distribution" meaning "set of scripts, web pages, and configuration files").
Digital Library and Archives
Steps to Install the ETD-db
- Download the latest distribution
- Create your databases and tables
- Configure your web server
- Customize the scripts for your site
- Test the web-based scripts
- Test and set up the non-web-based scripts
- Set up a backup of your data
- Migrate any existing electronic or paper documents
Download the latest distribution
You must first download and unpack the latest distribution of the ETD-db from http://scholar.lib.vt.edu/ETD-db/developer/download/. The distribution is a gnu-zipped tar file, if you have problem un-tarring the distribution, make sure you have the latest GNU version of tar and gzip (this is especially important for certain Solaris systems, the bundled version of tar is not compatible with files tarred using the more standard GNU tar). It is recommended that you extract all files into the /export directory and create a symbolic link between /export/ETD-db-version and /export/ETD-db, as the distribution is configured to work with /export/ETD-db by default. From this point on, the location you have extracted the scripts, etc. to will be referred to as <ETD-db_root>, all directory and file names described here are relative to that location.
Create your databases and tables
The following are very basic instructions for setting up the MySQL databases, users, tables, and permissions for the ETD database. From here on, you must perform these steps in the order shown. All file names are italic, and are given relative to the directory you unpack the scripts into. Unless otherwise specified, all commands must be run as root, and must be run from the directory in which the scripts and sql command sets are housed.
Edit the perl scripts used to create the databases, tables, users and permissions
Edit <ETD-db_root>/setup-ETD-db/create_etd_users.sql and enter passwords for the different usernames. You can use the defaults, but this is distributed freely, so the default passwords are not secure. You also need to edit <ETD-db_root>/setup-ETD-db/create_etd_users.pl and <ETD-db_root>/setup-ETD-db/create_everything.pl and enter the location of the mysql binaries on your system as well as the password for the mysql user 'root' (can be set to "" if no password is needed).
Creating the databases and tables
Run <ETD-db_root>/setup-ETD-db/create_everything.pl. This script will create all the basic database and tables. It will also prompt you to overwrite existing databases.
Creating the users
Run <ETD-db_root>/setup-ETD-db/create_users.pl. This creates the users etd_submit, etd_review, etd_catalog, and etd_public. This script also sets each user up with enough permissions to connect to the appropriate databases, and sets up etd_available, etd_submitted, etd_review, etd_withheld, and etd_public with enough permissions to select, modify, insert, and delete appropriate data. Don't run this if the users have already been created, as it may alter your permissions.
This script should create all the neccessary users and give them permissions to the tables you created before. If it stops halfway and/or you need to change the user setup, the quick and dirty way to get rid of existing permissions is to pop into mysql using a command like:'mysql mysql --password=ROOT_PASSWORD'
From there, issue the following commands:delete from user where User like '%etd%'; delete from db where User like '%etd%';
Of course, if you want to regenerate permissions in this way, it is imperative that you add any custom permissions you need to the create_users.sql file so that you don't overwrite them the next time you recreate everything. Every time you modify the permissions, you must reload the user and permission information (see below).
Reload the user and permission information
When you add user and/or permission information to the special database mysql, the new permissions will not take effect until the mysql daemon has been either restarted, or has been told to reload its permissions using the following command:'/usr/local/mysql/bin/mysqladmin -uroot -pROOT_PASSWORD reload'
Configure your web server
Once you have the database technology set up, you will need to add additional document and CGI directories for the scripts and support files included here.
- Each of the interfaces to the database (the author interface, reviewer interface, maintainer interface, patent restricted interface, search interface, and browse interface) resides in its own subdirectory relative to <ETD-db_root>/cgi-bin.Your web server must be configured to correctly associate each of these subdirectories with its relative location on the web. In addition, your web server must be configured to associate the document root for the ETD-db documentation and help files with the /ETD-db directory on your server). Here are the lines in our http.conf that configure our Apache web server to associate each of the directories with their location on the web. Both ScriptAlias and Alias commands interpret the first argument as the directory you want web users to be able to visit, and the second argument as the directory on your actual file system.
Please Note: the following lines must be entered in the order shown if you want the cgi directories to show up as subdirectories under /ETD-db.# additional directories for ETD-db (must be added before Alias for ETD-db) ScriptAlias /ETD-db/ETD-submit/ /export/ETD-db/cgi-bin/ETD-submit/ ScriptAlias /ETD-db/ETD-review/ /export/ETD-db/cgi-bin/ETD-review/ ScriptAlias /ETD-db/ETD-withheld/ /export/ETD-db/cgi-bin/ETD-withheld/ ScriptAlias /ETD-db/ETD-maint/ /export/ETD-db/cgi-bin/ETD-maint/ ScriptAlias /ETD-db/ETD-catalog/ /export/ETD-db/cgi-bin/ETD-catalog/ ScriptAlias /ETD-db/ETD-browse/ /export/ETD-db/cgi-bin/ETD-browse/ ScriptAlias /ETD-db/ETD-search/ /export/ETD-db/cgi-bin/ETD-search/ Alias /ETD-db /export/ETD-db/web_root
- Set up the security for all of the non-public directories (ETD-review, ETD-withheld, ETD-maint, and ETD-catalog). Here's a sample from our httpd.conf file:
<Directory /export/ETD-db/cgi-bin/ETD-review> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName review order deny,allow deny from all require user etd-admin etd-grad satisfy any </Directory> <Directory /export/ETD-db/cgi-bin/ETD-catalog> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName catalog order deny,allow deny from all require user etd-admin etd-grad etd-catalog satisfy any </Directory> <Directory /export/ETD-db/cgi-bin/ETD-withheld> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName withheld order deny,allow deny from all require user etd-admin etd-grad satisfy any </Directory> <Directory /export/ETD-db/cgi-bin/ETD-maint> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName maint order deny,allow deny from all require user etd-admin etd-grad satisfy any </Directory>
The configuration variable AuthUserFile indicates where the password file for "Basic" security is stored. You'll need to add each of the users mentioned on the "require user" lines to the password file. This is done using the htpasswd utility that comes with apache.
- Set up the neccessary directories to store your actual ETDs. You will generally need one directory for each of the following types of ETDs:
On our site, we use: /theses/submitted, /theses/available, and /theses/withheld respectively. You will also need a trash folder in the submitted directory, ours is /theses/submitted/trash. Note that each of these directories will also need to be added to your web server as an additional document directory (or you can add their parent directory, which is what we do on our site). The account used by your web server must have write access to these directories. If you haven't done so, you must also add an entry for the directory (or directories) housing your author-submitted files to your web server configuration files. Here's an example from our site:
- submitted ETDs: all ETDs waiting to be approved (should be secured).
- available ETDs: includes restricted (campus) and unrestricted ETDs
- withheld ETDs: includes ETDs withheld for patent or other purposesTo set up access restrictions for the static HTML pages and files associated with your ETDs, you must add an entry for each to your httpd.conf file. Here's an example from our site:# directory the actual author-submitted content lives in Alias /theses /theses_1This example assumes that you have added the users etd-admin, etd-grad, and etd-catalog to a password file using the htpasswd utility that comes with apache. This example sets the access restrictions for campus only ETDs based on the Virginia Tech campus network, you'll want to find out the IP address range for your university as well as your domain suffix and change the line "allow from 128.173 .vt.edu" to something more appropriate. If you want a custom error message for users outside your campus network, you'll also want to make sure the lines "ErrorDocument 401" and "ErrorDocument 403" point to files that actually exist on your system.# security settings for campus only ETDs, including campus only files # that are part of mixed access ETDs <Directory /theses_1/available/*/restricted> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName restricted order deny,allow deny from all allow from 128.173 .vt.edu #require user etd-admin etd-grad etd-catalog ErrorDocument 401 /theses/vt-only.html ErrorDocument 403 /theses/vt-only.html satisfy any </Directory> # security settings for the ETDs that are entirely withheld from access # (i.e. patent restricted) <Directory /theses_1/withheld> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName withheld order deny,allow deny from all require user etd-admin etd-grad etd-catalog satisfy any </Directory> # security settings for the patent restricted files within mixed access ETDs <Directory /theses_1/available/*/withheld> AllowOverride None AuthType Basic AuthUserFile /usr/local/apache/conf/.htpasswd AuthName withheld order deny,allow deny from all require user etd-admin etd-grad etd-catalog satisfy any </Directory>
Customize the scripts for your site
There are a handful of files you need to edit to customize the scripts for use on your site. There is a page describing the process located here. It is also recommended that you us a graphic editor (Photoshop, etc.) to change the "VT ETD Collection" logo provided to reflect the name of your institution.
Test the web-based scripts
Launch at least a few of the web-based scripts (such as <ETD-db_root>/cgi-bin/ETD-submit/login) from the command prompt first, there should be no compilation errors. Once you have verified that the scripts will at least run from the prompt, launch them from the web.
The starting point for each directory is as follows:
If you have problems launching any scripts and can't determine the problem by looking at either your web server's error logs or by running the script from the command prompt, send the complete error message(s) to email@example.com
Next, submit a test ETD for each type of availability (start with link number one on the list above), test adding and removing files, modifying and ETD, deleting and ETD, all from within the submission process. If you are unclear about the use of the submission process, there are help files included with this distribution, which you can either look at over the web as /ETD-db/help/ once you've configured <ETD-db_root>/web_root to point to /ETD-db, or can open manually from the server in the directory <ETD-db_root>/web_root/help.
Once you have submitted a few test ETDs, try reviewing them using link number two on the list above. Test modifying ETDs, adding and removing files, sending notices, deleting an ETD, and finally approving at least one of each type of ETD.
Once you have approved at least one withheld ETD, use the fourth link on the list above to add and remove files, modify information, delete, and finally release an ETD.
Once you have approved at least one available (restricted, unrestricted, or mixed) ETD, use the third link on the list above to add and remove files, modify, information, delete, and change the availability of an ETD. Be sure you change the availability of at least one ETD to withheld to test the transfer of data in that direction.
Test and set up the non-web-based scripts
These scripts, located in <ETD-db_root>/non-cgi-scripts, are used to generate static html pages, perform system maintenance, and even to export data to other formats. More information on the non-cgi scripts.
Like the other subcategories of scripts, these scripts have a master file that controls most of their variables and the look and feel of their headers and footers. To change any of this information, edit <ETD-db_root>/non-cgi-scripts/etd-non-cgi-lib.pl.
Set up a backup of your data
In addition to backing up the author files on your system, you should be prepared to back up the contents of each database on an hourly basis. The databases are stored as files, but simply backing up the database files is no guarantee that all of your data will be preserved. To safely backup up all of your data, you will need to use the mysqldump utility includes with your installation of mysql. You should run one command for each of the three primary databases, something like:mysqldump -uroot -ppassword database_name > database_name.date.dmp
You may also use the sample script <ETD-db_root>/non-cgi-scripts/backup_databases included here. It is suggested that you gzip or tar the dump files and keep them on tape as well as on an entirely unrelated system for safekeeping. On a unix system, you'll want to use the cron facility to have whatever backup you prefer run hourly.
Migrate any existing electronic or paper documents
If you would like to use this system to manage ETDs that were entered using a different system (or that you are creating based on paper documents), you will need to either manually enter all of the ETDs into the new system, or write your own scripts to migrate your existing materials. For help in creating your own migration scripts, please contact firstname.lastname@example.org.