How to download and install needle tools

BMERC : needle tools : Appendices : Needle tools installation


This covers the "needle-tools-1.6" release, which has all of the programs documented by this collection of Web pages, exclusive of the needle software, which is available through the M.I.T. Technology Licensing Office of the Massachusetts Institute of Technology.

Table of contents

  1. How to download and install
    1. Table of contents
    2. System requirements
    3. Installation steps
    4. Multiple system installation

System requirements

needle tools has been installed successfully on the following configurations:

If you manage to bring needle tools up on some other system, please let us know. We would especially appreciate having any changes that were necessary to get it up and running.

In addition to the traditional Unix stuff (e.g. csh, tar, make, etc.), you will need the following compilers and interpreters in order to install and run these programs:

  1. an ANSI-compliant C compiler; we use gcc, the GNU C compiler (see http://www.gnu.org/software/gcc/gcc.html). Our software does not push the envelope of ANSI extensions very hard; the essential thing is that the compiler support prototypes. (The other thing we make use of here and there is the implicit concatenation of adjacent string tokens.)
  2. a Fortran compiler (we use the system's stock f77 compiler).
  3. the perl programming language (see http://language.perl.com/index.html. Most scripts require version 5 of perl.
We also assume you have gzip, which, although not traditional, is close to being standard (if you don't have it yet, see the GNU gzip page at http://www.gnu.org/software/gzip/gzip.html or download from http://www.gzip.org/.) All of these must be available before starting the installation process. (The optional systems listed below may be installed after needle tools.)

needle tools assumes that you will use DSSP secondary structure assignments, and has no other provision for generating them; cores can be produced automatically only from DSSP. So, you will either need to write your own secondary structure definition interface, or install the following additional software, which does not come bundled with the needle tools code.

  1. the DSSP software itself. See [Kabsch, W., and Sander, C., Biopolymers 22(1983) 2577-2637], or the DSSP web pages at http://www.sander.embl-heidelberg.de/dssp/ for more information on DSSP, and how to obtain a copy.
  2. a Lisp system (compiler or interpreter). We generally use cmu-cl (CMU Common Lisp; see http://www.cons.org/cmucl/), or Allegro Common Lisp (by Franz Inc; see http://www.franz.com). If you use something other than these, you must
    1. compile scripts/guess-ss-designations.lisp by hand;
    2. edit scripts/makefile to install the correct binary produced by step 1; and
    3. if not cmu-cl or Allegro, edit scripts/make-ss-designations to call the right Lisp system.
    (Allegro is already supported by scripts/make-ss-designations, but we do not distribute Lisp binaries for Allegro.)
Finally, you will need the following additional packages if you wish to run the corresponding optional parts of needle tools:
  1. If you wish to use pdb-domain-seq.pl or dssp-ss-states.pl, you will also need the globalS program, or equivalent. globalS is part of the U.S.C. Sequence Alignment Package (see http://hto-13.usc.edu/software/seqaln/). If you want the whole package, then follow the installation instructions in the INSTALL file in the distribution's top-level directory. If you only want globalS, then you only need the following steps (using the step numbering in the INSTALL file for version 2.0):

    1. Edit the variables in the top-level Makefile as necessary.
    2. Instead of "make install", do "make all".

    Then, install seqaln-2.0/seqaln/globalS into the needle tools system binary directory. If needle tools has already been installed, you can do this by:

           cd seqaln/bin
           install.pl -show -m 555 globalS $NEEDLE_SYSTEM_BIN
    
  2. If you wish to use BLAST and the related utilities described on the "BLAST Programs" page, you will also need the BLAST software itself, either WU-BLAST (Washington University BLAST) version 2.0 (see http://blast.wustl.edu/blast/README.html) or the earlier NCBI-BLAST version 1.4 (see http://www.ncbi.nlm.nih.gov/BLAST). At BMERC, we prefer WU-BLAST because it can handle gaps, but NCBI-BLAST is in the public domain. Except for generating clique files from sequence similarity data (see the blast-to-clique.pl program documentation), it is not necessary to have BLAST in order to run the rest of needle tools.

Installation steps

These instructions are for Unix systems, and assume you already have the necessary software installed. If you have problems while following these instructions, please contact
Bob Rogers <rogers@darwin.bu.edu> for assistance.
  1. Set the current directory to the parent of the directory that will become the "needle tools root" directory (/usr/local/needle, for instance, or your home directory):
        cd /usr/local
    
    You need to have write access to this directory.

  2. Get the tar file from the BMERC FTP server into this directory. The tar file is available via anonymous FTP on darcy.bu.edu in the /pub/needle/needle-tools-1.6.tar.gz file, and is approximately 687KB. [this doesn't seem to work any more; try http://bmerc-www.bu.edu/needle-doc/latest/needle-tools-1.6.tar.gz instead. -- rgr, 19-Mar-03.]

  3. Unpack and discard the tar file:
       gunzip -c needle-tools-1.6.tar.gz | tar xf -
       rm needle-tools-1.6.tar.gz
    
    This will build a subdirectory called "needle-tools-1.6" that contains the needle tools directory tree, filling it up with approximately 2.5MB of source code, data, and documentation; compiling will push it to about 5MB. The installed software (scripts, programs, and a few data files) requires from 1.0MB to 1.9MB, depending on the system type; the installed HTML documentation is about 700KB.

  4. Change the makefile defaults, if necessary. Depending on your software configuration, you may need to edit needle-tools-1.6/new-stat/makefile and needle-tools-1.6/exposure/makefile to change the identity of or options to the C and/or Fortran compilers.

  5. Compile everything in the needle-tools-1.6 directory:
        cd needle-tools-1.6
        make all
    
    Ignore the following Fortran warnings:
    
    "access.f", line 214: Warning: there is a branch to label 2 from outside block
    "addradii.f", line 70: Warning: ignoring unimplemented "readonly" specifier
    "charge.f", line 42: Warning: ignoring unimplemented "readonly" specifier
    
    and all C "implicit declaration of function" warnings (under gcc). Compiling the Fortran code on the Alphas gives a different set of warnings:
    
    fort: Info: access.f, line 172: Questionable branch into loop or block
          IF(IO.GT.ICT)GO TO 2
    fort: Info: frens.f, line 193: Questionable branch into loop or block
    	IF (NAME(JR).EQ.LRES) GO TO 1400
    fort: Info: frens.f, line 359: Questionable branch into loop or block
    	GO TO 300
           
    But these don't seem to make any difference, either.

  6. Edit the install.pl script in needle-tools-1.6/scripts/install.pl so that it is "self-invoking" at your site. The start of this file looks like this:
    
        #!/usr/local/bin/perl
        #
        # Installation script that is smarter than the install program . . .
    
    The first line essentially hardwires the name of the perl executable, and is therefore site-dependent. (Some other incantation may be necessary depending the level of stupidity of your Unix system; see the Camel Book or the perlrun man page for details.) Change the first line so that it works at your site. Turning it into multiple lines is fine, just as long as it is followed by a line with just a "#" on it. Not only does this allow install.pl itself to run, it tells install.pl how to update these lines for any other perl script it installs.

  7. Choose a name for the binary directory (assuming you only need one; see the "Multiple system installation" section), and set the NEEDLE_SCRIPTS and NEEDLE_SYSTEM_BIN environment variables accordingly.
        setenv NEEDLE_SCRIPTS /usr/local/bin
        setenv NEEDLE_SYSTEM_BIN /usr/local/bin
    
    These definitions are only used by the "make install" targets; you do not need them to run needle tools.

  8. Put the binary directory on your $PATH environment variable.
        set path = ($NEEDLE_SCRIPTS $path)
    
    This must be done for installation, but is also required to run the tools, so you should put something of this nature in your .login script.

  9. Install everything using make:
        make install
    
    This puts read-only copies of all binaries and scripts in the appropriate directories, creating them if necessary. You should get lots of messages, especially on DEC Alpha systems; as long as make completes successfully, you can ignore them.

  10. To test the installation, we recommend trying one of the published examples. (You will need the DSSP software in order to do this.) The needle-tools-1.6/test-cores/ directory contains the makefile and core list from the example in the "Automatic generation of cores via makefiles" section, together with its products as constructed at BMERC. Starting in the same directory from which you just completed the installation, we recommend you try the following (the commands are shown without their output):
        % mkdir new-cores
        % cp test-cores/makefile test-cores/cores.loci new-cores
    
    One must modify new-cores/makefile at this point to replace "../pdb/pdb" with whatever is appropriate at your site to put the PDB directory on the search path. Alternatively, one can link it to needle-tools-1.6/pdb/ by doing
        % ln -s /structure/pdb pdb
    
    (using the BMERC local PDB directory /structure/pdb). This has the advantage of preserving the file names; the generated dependency file will therefore be that much closer to identical to the distributed test version. See the makefile discussion in the "Core library construction examples" section for details.

    Then,

        % cd new-cores
        % touch depends.make
        % make depend
        % make
        % diff . ../test-cores
    
    As described in the "Testing the core generation results" section, diff should find only trivial differences between the two directories.

  11. Optionally, you can install the web pages (the *.html and *.gif files in the needle-tools-1.6/html/ distribution directory) on your local server as a post-installation step. This will allow you to site-customize them if necessary.
Once the needle tools have been installed, it is OK to remove the whole needle-tools-1.6/ directory tree. Or do "make clean" to save space if you want to keep the sources around.

Multiple system installation

If you are installing on multiple operating system or hardware types, you will need several directories in which to put scripts and binaries. If they are someplace that is not cross-mounted on the other systems, as is usually the case for /usr/local/bin, you may be able to get away with just repeating the single-system installation instructions N  times. If they are shared (such as in your home directory tree), then different systems will require different binary directories, and you should use the installation procedure below.

Note that if the systems are very different, or if perl is installed in different places, then the result of editing the install.pl script will be different for different systems, and the perl scripts will have to be treated as if they were binaries (at least to the extent of installing them into the same directories). This can easily be taken care of in the init-needle-tools customization step below.

Modify the single-system recipe as follows.

  1. Perform steps 1, 2, and 3 as described above.

  2. Choose a name for the binary directories, and edit the init-needle-tools script found in the needle-tools-1.6/scripts/init-needle-tools file so that it sets the NEEDLE_SCRIPTS and NEEDLE_SYSTEM_BIN environment variables correctly for all system types. Although init-needle-tools assumes that you want to organize them as siblings in the same tree, this is not necessary; they could be anywhere. It is not necessary to create these directories explicitly; the top-level install target does this automatically, as long as their parent directory (or directories) exist.

    If, for example, you decide to use /usr/local/needle-tools-1.6/bin/scripts for the scripts directory and /usr/local/needle-tools-1.6/bin/* as the system-dependent binary directories, and assuming the init-needle-tools code characterizes the differences between your systems correctly, then all you have to do is change the needle_bin_root initialization to look like this:

    
        . . .
        # [***site***: The following line needs to be site-customized.]
        set needle_bin_root = /usr/local/needle-tools-1.6/bin
        . . .
    
    If you need to install different perl (and/or other) scripts on each system, reset the NEEDLE_SCRIPTS environment variable to the NEEDLE_SYSTEM_BIN value after the latter is initialized:
    
        setenv NEEDLE_SYSTEM_BIN $needle_bin_root/$needle_system
        setenv NEEDLE_SCRIPTS $NEEDLE_SYSTEM_BIN
    

  3. For each system:

    1. "Source" init-needle-tools on the compiling machine so that it defines the new environment variables for the benefit of "make install".
      
        source needle-tools-1.6/scripts/init-needle-tools
      

    2. Edit the makefiles if necessary (the original step 4), and then compile (step 5) as above.

    3. Edit the install.pl script (step 6 above). You only need to do this the first time through, unless you need to install different perl scripts on different systems.

    4. Install the software using "make install" (step 8 above).

    5. Test the installation on this system as by step 10 above. For the first system, set up a test directory:
          % mkdir new-cores
          % cp test-cores/makefile test-cores/cores.loci new-cores
          % cd new-cores/
             [one must modify the makefile at this point.]
          % touch depends.make
      
      (For the second and subsequent system, only the cd command is necessary.) Then, and for all subsequent systems, do:
          % make clean
          % make depend
          % make
          % diff . ../test-cores
          % cd ..
      
      As described in the "Testing the core generation results" section, diff should find only trivial differences between the two directories.

    6. Clean the old binaries in preparation for the next system type.
        make clean
      

  4. Repeat from step 3 above for each system type.

  5. Optionally, you can install the web pages (step 11 above).

  6. Put init-needle-tools in your .login script. As part of the installation procedure, these environment variables are only used to tell "make install" where to put things. However, init-needle-tools also sets up the search path properly for the tools to work, so you probably want to invoke it from your .login script.

Bob Rogers <rogers@darwin.bu.edu>
Last modified: Wed Mar 19 10:38:31 EST 2003