Python for Astronomy

Python for Astronomy

A collection of Python resources for astronomy: packages to install, links to resources for learning Python and for using Python in scientific data analysis.

[Last updated: 2011/05/26]

Basic requirements

Any version of Python >= 2.5 should be compatible with most of the astronomy packages. But Python 3.x series does not have wide support at this time. It is not compatible with Python 2.x, and it will take a while for most packages to become available for Python 3.x. But note that numpy 1.5.0 and asciitable 0.3.1 work with Python 3.x.

The Python distribution that comes with the operating system, should not be un-installed. It is better to leave it untouched, except for installing a few packages that are best installed from the software repositories associated with the operating system. All other packages should be installed in custom locations. This can be done by configuring the PYTHONPATH shell variable or, even better, with virtualenv and virtualenvwrapper tools. Starting with Python version 2.6, Python will also look for modules in ~/.local directory. Usage of these tools and methods are described in the Setting up a “user” Python environment section.

There are a few Python “distributions” that come with many important Python packages and their dependencies, and only the astronomy specific packages need to be installed separately. Another advantage of using these distributions is that the Python installation that comes with the operating system can be left as it is. But on the other hand they may not be available for all operating systems and they may not have specific versions of the libraries we are interested in.

Some Python distributions i.e., Python + libraries, are:

  • EPD

    This is free only for academic usage. This is available for Windows XP, Windows Vista, Windows 7, Mac OS X 10.5+ and several flavors of GNU/Linux.

  • pythonxy

    Available only for Windows.

    EPD and pythonxy are perhaps the easiest way for Windows users to setup a good Python environment for scientific computing.

  • scisoft

    Contains a set of Python libraries, in addition to a very large collection of non-python astronomy software. The ESO version is available only for Fedora 11. I tried this on Fedora 12, but it resulted in some conflicts with X libraries, which prevented me from logging into the system. A version for Mac OS X is available at http://web.mac.com/npirzkal/Scisoft/Scisoft.html

Python and various libraries depend on several non-python programs and libraries. I have made a list of these, based on my experience in installing astronomy packages. Admin privileges are required for installing these libraries. Of-course, not all of these are required; install those that are mentioned in the installation instructions for the packages that you are interested in. When installing non-python packages, make sure that in addition to the package itself, development libraries and header files, if available, are also installed. On Ubuntu these usually have the string “dev” in their names. In Fedora these are indicated with the string “devel” in their names.

Also, add the /usr/local/lib directory to LD_LIBRARY_PATH system variable. Add the following to ~/.bashrc:

export LD_LIBRARY_PATH = $LD_LIBRARY_PATH:/usr/local/lib

The Python libraries that are probably best installed from the operating system repository are:

  • wxPython , pyqt and pygtk

    On Ubuntu these are the libraries that I installed from the repository, using the Ubuntu Software Center tool.

    • libwxgtk2.8-0
    • libwxgtk2.8-dev
    • wx-common
    • python-wxgtk2.8
    • python-wxversion
    • libqtcore4
    • libqtgui4
    • libqt4-dev
    • python-gtk2
    • python-gtk2-dev
  • swig

    In Ubuntu this package is listed simply as “swig”.

    This library is used for interfacing Python with C and C++ libraries.

  • PIL : Python Imaging Library

    This is available as “python-imaging” in Ubuntu.

    PIL is a Python library for performing image manipulations. It can be used to read and write images in various formats, apply various transformations to the image data and other such operations.

Setting up a “user” Python environment

As mentioned before, it is better to leave the system Python installation untouched, as far as possible. But we will install a couple of packages in the system “site-packages” directory. This should make life a lot easier, when installing other packages. If you plan to use these tools, then it is better not to set the PYTHONPATH environment variable.

  • Setuptools, easy_install, distribute and pip

    These tools allow easy installation of Python packages. Many packages can be installed using the commands:

    $ easy_install <package-name>

    or:

    $ pip install <package-name>

    These commands will download the required files and install the packages. Pip can also be used to un-install packages.

    Install the library “python-setuptools” from the system software repository. After installing python-setuptools do the following:

    $ sudo easy_install pip
    $ sudo pip install distribute

    Pip is intended to replace easy_install; so it should be used in favor of the latter.

What if you don’t have admin privileges at all and want to install Python packages? See the Per user site-packages directory and PYTHONPATH environment variable sections.

Virtual environments

  • virtualenv and virtualenvwrapper

    These tools can be used to install Python packages in such a way that they are “isolated” from the system installation. Many such “environments” can be created and say, different versions of a certain library can be installed in each. This is great when testing new versions of libraries. An “environment” can be easily deleted without affecting any other.

    Execute the following to install these:

    $ sudo pip install virtualenv
    $ sudo pip install virtualenvwrapper

    and then add the following lines to ~/.bashrc:

    export WORKON_HOME=$HOME/lib/pyvirtualenv
    source  /usr/local/bin/virtualenvwrapper.sh

Make sure that the directory mentioned in WORKON_HOME exists. We won’t need admin privileges from now on.

Now lets create a virtual environment for our astronomy packages. Open a new terminal and then issue the command:

$ mkvirtualenv astro
...
...
(astro)$

This command will create a virtual environment named “astro”. The shell prompt will change to show that all Python commands will now use this environment instead of the system Python environment.

If we want to go back to the system environment, then we can issue the following command:

(astro)$ deactivate
$

The “astro” label in the will disappear. To go back to “astro”:

$ workon astro
(astro)$

It’s that simple; of-course, this doesn’t solve the problem of having the proper non-python libraries such as HDF, GCC etc., .

Most Python libraries come with a setup.py script that can be used to install the software. Typically one executes:

$ python setup.py install

If a particular “virtual environment” is active, then this command will install the software in that environment.

$ workon astro
.
.
(astro)$ python setup.py install

For installing simple modules, just copy them into $WORKON_HOME/pyvirtualenv/<name>/lib/python2.6/site-packages/, where <name> is the name of the virtual environment.

If you don’t want to use these tools, then there are two methods for installing Python modules, without admin privileges and separate from the system installation. The following two sections describe them.

Per user site-packages directory

With Python versions >= 2.6, Python will also look for modules inside ~/.local directory. Inside this directory there will be version specific sub-directories such as lib/python2.6/site-packages, where users can install modules, without needing admin privileges. Modules can be either copied into the above subdirectory or if there is a setup.py script then the --user option can be used:

$ python setup.py install --user

The easy_install tool also takes the --user option:

$ easy_install --user <package-name>

The easy_install and pip tools themselves can be installed in the ~/.local directory by specifying the --user option while running the setup.py script associated with them. The to use these tools we can do:

$ ~/.local/bin/easy_install --user <package-name>

The advantage of installing packages in ~/.local is that all libraries installed here will also be available from within system Python.

PYTHONPATH environment variable

The second method is to use the PYTHONPATH environment variable; works on versions before 2.6 as well. This method can potentially lead to confusion, especially if you have multiple versions of Python; per-user site packages and “virtual environments” are better options.

Set the environment variable PYTHONPATH to the directory containing Python modules. For example, in bash,

export PYTHONPATH=${HOME}/lib/python/lib/python2.6/site-packages
export PATH=$PATH:${HOME}/lib/python/bin

Any module that you want to install can be simply copied into the ${HOME}/lib/python/lib/python2.6/site-packages directory. With setup.py script we can do:

$ python setup.py install --prefix=${HOME}/lib/python

This will install Python modules and scripts in the “site-packages” directory sub-directory. Create the directory ${HOME}/lib/python before running the setup.py file. Python will create the sub-directories, if needed.

The easy_install tool will also install packages in the PYTHONPATH directory, if run without admin privileges.

All libraries installed in PYTHONPATH are also available from within system Python.

Basic packages

Now that we have installed the main dependencies, we will go ahead and install the basic scientific packages, required by most of the astronomy packages. Be sure to be in the “astro” virtual environment while installing these. Download these packages and follow the instructions provided to install them. All these require several non-python libraries to be installed, as noted in a previous section.

  • Numpy and Scipy

    The main feature provided by numpy is the ability to work with multidimensional arrays in an efficient manner. This forms the basis for carrying out mathematical calculations, such as those related to histograms, correlations, linear algebra etc., . Scipy, for example, builds upon these facilities provided by numpy to create higher level scientific and mathematical routines.

  • Matplotlib

    Matplotlib is a 2D plotting library, with some 3D plotting functionality.

  • IPython

    This is an excellent substitute for the default Python shell. Perhaps the most attractive part of this shell is the ability to list the members of a module or an object. For example, after importing the math module, we can type math. and press the tab key, to get a list of members of the module. The command math.sin? will print the docstring for the function sin in the math module. The %cpaste magic command, allows code to be pasted into the shell, which would otherwise result in indentation errors.

    Screencasts on using iPython are available at: http://showmedo.com/videotutorials/series?name=CnluURUTV

A selection of astronomy libraries

The STScI Python library is a collection of several astronomy libraries. Libraries, pyfits, pywcs, pyraf, pysynphot, and numdisplay are bundled with the stsci_python collection. It also has many other packages, for example the ndimage image package for image processing. So if these satisfy your requirements, then use it instead of installing the libraries one by one. But the latter method is more flexible.

  • pyfits

    This library provides facilities for reading, manipulating and writing FITS files.

  • pywcs

    This is a Python library for handling World Coordinate System information in FITS headers. It uses the wcslib C library written by Mark Calabretta.

    Requires pyfits and wcslib. The latter comes with the pywcs distribution.

  • coords

    This library can be used for managing astronomical coordinate systems. Ultimately, this library aims to provide facilities needed by astronomers for planning and preparing for observing runs.

    It currently provides facilities for coordinate conversions and is under development.

  • astroasciidata, vo.table, asciitable and ATpy

    Together with pyfits, these libraries provide comprehensive facilities for dealing with the variety of tables used in storing astronomical data. For example, asciitable can read data from tables in the CDS format provided by VizieR and ApJ. ATpy can read daophot data files.

  • pyraf, urwid and pmw

    Pyraf provides an interface to the NOAO IRAF system (also see http://www.iraf.net/ for IRAF user support). Libraries urwid and pmw are dependencies for pyraf.

  • numdisplay

    This library provides an interface to the DS9 image display program. Numdisplay can also interact with the ximtool image display program.

    There is another package, pyds9, from the developers of DS9, that provides the similar functionality.

    RO is another library that has a module for interacting with DS9. The library, pyregion, can parse DS9 region files.

  • APLpy

    APLpy builds on matplotlib, and allows creation of production-quality plots of astronomical imaging data in FITS format. Some of the features provided are overlaying coordinate grids, 3-color RGB image display, contours and markers etc., .

  • pywcsgrid2

    The pywcsgrid2 module is used with matplotlib for displaying astronomical fits images. For example, it can be used to easily add RA and DEC labels to FITS images and draw coordinate grid lines.

  • Kapteyn

    A collection of modules for handling World Coordinate System transformations, coordinate conversions, handling FITS files, plotting and others. It also has a module for parsing and interpreting coordinate information entered as strings.

  • PyEphem, pyslalib and pytpm

    PyEphem can be used to get quick answers to questions such as, Which constellation is Jupiter in tonight?, What is the rising/setting time for Moon, at a particular location?, What is the current Julian Date?. PyEphem uses C routines from the XEphem package and a copy of the C library is included with the source code.

    Pyslalib is a Python interface to the SLALIB fortran library. This module comes with a version of the SLALIB fortran library.

    PyTPM is an interface to Jeff Percival‘s Telescope Pointing Machine C library. Pytpm includes a copy of the TPM C library that is part of the coords package. In PyTPM, only one function call, along with appropriate data, is needed for performing coordinate conversions.

    Together with Kapteyn, these three modules provide most of the facilities for performing astrometry calculations.

  • Scipy and ScientificPython

    Modules implementing various algorithms for scientific computing.

  • Sphinx

    Sphinx generates formatted documentation from text documents written in the reStructuredText format. This module can read “docstrings” from Python code and include them in the final document. It incorporates several features such as running “doctests” in Python code, producing plots by running Python code, links to source files and others. Most of these functionalities are provided using “sphinx extensions”. Source code for matplotlib, matplotlib sampledoc tutorial and the sphinxcontrib collection include many such extensions, in addition to those that come with sphinx.

  • nose

    This is a framework for writing tests for Python code. It can find unit tests in Python source files and execute them. It provides various other features for performing test on Python code.

Sphinx and nose are not astronomy specific packages. Sphinx is used for creating documentation and nose is a framework for carrying out unittests. Both of these, documentation and testing, are integral to any software project and hence they are included here.

Other interesting Python libraries for astronomy

  • aipy

    Astronomical Interferometry in Python is a collection of tools for radio astronomical interferometry.

  • astropysics

    This is a library aimed at providing a very large set of astronomy related code. Additionally, it wraps SExtractor and Kcorrect.

  • astLib and pyWCSTools

    Another collection of Python code for astronomy, including coordinate conversions, WCS manipulations and so on.

  • PRESTO

    Software for pulsar search and analysis.

  • RO

    RO is a collection of scripts, written by Russel Owen of University of Washington in Seattle, providing Tkinter GUI extensions, Tkinter-compatible networking, astronomical calculations such as coordinate conversion, time related functions and so on. It includes a module for interacting with DS9.

  • ChiantiPy

    This is a library for accessing the CHIANTI database and calculating various quantities of interest.

  • Sherpa

    Sherpa is a modeling and fitting package that is part of Chandra Interactive Analysis of Observations software suite. It uses IPython as its user interface and is also importable as a module from within Python.

  • yt

    Yt is a package for analyzing and visualizing astrophysical simulation output, generated by simulation platforms such as enzo, orion, flash and others.

  • CosmoloPy and cosmocalc

    Modules for cosmology related calculations, e.g., calculate angular size, given redshift and cosmology.

  • Healpy

    Healpy provides a python package to manipulate healpix maps. It is based on the standard numeric and visualisation tools for Python, Numpy and matplotlib.

  • pyflation

    A Python package for “calculating perturbations during an inflationary expansion of the universe”.

Libraries for interacting with other applications

Performing specific tasks

Read, manipulate and write tabular data in various formats

  • ASCII tables

    Use ATPy, astroasciidata, asciitable and the tabarray module in Kapteyn. A tutorial on using AstroAsciiData is available at this url. Note that ATpy uses asciitable module for reading ASCII tables.

  • SExtractor tables

    Use astroasciidata. The meta-data stored in SExtractor tables are also read in by this module and can be accessed from within Python, using the various facilities provided by AstroAsciiData.

  • FITS tables

    Use pyfits.

  • VO tables

    vo.table and ATpy.

  • Tables from SQLite, MySQL and PostgreSQL.

    Use ATpy. For reading MySQL and PostgreSQL tables, ATpy uses the MySQL-python and PyGreSQL libraries, respectively.

  • HDF5 tables

    Use ATPy. ATpy uses the h5py library for reading HDF5 tables. Also see http://www.pytables.org/.

  • CDS format tables, and Vizier and ApJ machine readable tables

    The asciitable module can read ascii files in these formats.

Plots and annotated images

Performing astrometry calculations

There are several libraries that provide functions for performing astrometry calculations. These are Kapteyn, pytpm, pyslalib , coords, RO, astLib, astropysics and PyEphem. Currently, only PyEphem provides facilities for handling ephemerides of solar system objects and calculating quantities such as rise and set times of these objects.

Mathematical and scientific calculations

Numpy, Scipy and ScientificPython. Sympy is a module for symbolic mathematics. SciTools provides scientific computing tools built on top of NumPy, SciPy, ScientificPython, Matplotlib, Gnuplot, etc.

Calling C, Fortran and IDL code

Photometry

Use pyraf interface to IRAF and pyIDL and pIDLy interfaces to IDL. pysynphot can be used for synthetic photometry and spectrum manipulation. The library tfit is a photometry package, but is currently not maintained.

Spectroscopy

Use pyraf interface to IRAF and pyIDL and pIDLy interfaces to IDL. pysynphot can be used for synthetic photometry and spectrum manipulation.

Creating documentation

Miscellaneous modules

  • Texttable : formatted output for simple tables
  • simpy : object-oriented, process-based discrete-event simulation language
  • suds: lightweight SOAP Python client for consuming Web Services
  • SAMPY: implementation of the Simple Application Messaging Protocol.

Information sources on using Python in scientific data analysis

  • astropython.org

  • AstroPy mailing list

  • Practical Python For Astronomers

    A great collection of documents based on a hands-on workshop at the Smithsonian Astrophysical Observatory, conducted between March 11 and May 6 2011. These documents contain information on installation, basic tutorials on core Python packages, tutorial on plotting and images and plenty more.

    This is a “living document” and all are encouraged to adapt it for their needs. Please do send contributions if they are relevant for other astronomers.

  • Introduction to Numpy

    A good introduction to numpy is available at http://mentat.za.net/numpy/intro/intro.html. The basics sections of the numpy documentation is also a very good place to start: http://docs.scipy.org/doc/numpy/user/basics.html.

  • Practical Scientific Computing with Python (pdf file)

    This is a very good tutorial on using Python for scientific data analysis. This article takes the reader through a whirlwind tour of the Python standard library, iPython, Scipy and Mayavi; great introductory material for anyone with some programming experience.

  • Interactive Data Analysis in Astronomy

    This is an extensive tutorial, by Perry Greenfield and Robert Jedrzejewski, on the usage of Python for interactive data analysis in astronomy. It also serves as a resource for learning Python.

  • Python Scripting for Computational Science

    This is a book on the usage of Python for data analysis and scripting. This book is not directly targeted at astronomical data analysis but is aimed at general scientific data analysis.

    This book is written by Hans Petter Langtangen

  • Software Carpentry

    Not specific to astronomy, but a great resource for learning about methodologies in scientific computing, with focus on Python.

  • Py4Science

    Information on setting up a Python enviroment for scientific calculations, written by Fernando Pérez of UC Berkeley. This document also includes links to books, web pages, videos and other resources for learning Python.

  • http://fossee.in

    FOSSEE stands for Free and Open source Software for Science and Engineering Education. It is a project based at the Indian Institute of Technology, Bombay (IITB) and is funded by the Ministry of Human Resource Department, of the Government of India. The project focus at IITB is on the use of “Scilab and python in the area of numerical computing”.

  • Some courses concentrating on Python for astronomy data analysis

Learning Python

Non-free resources

If you are new to programming, start with Python For Software Design, Byte of Python or Invent with Python.

If you have some programming experience, then start with the official Python tutorial, which is a “must read” item for all Python programmers. After reading this tutorial, read Practical Scientific Computing with Python (pdf file) and Interactive Data Analysis in Astronomy, which focuses on astronomy data analysis and also serves as a guide to Python itself.

Idiomatic Python explains several “pythonic” ways of writing code. Dive into Python explores many Python concepts in depth. For the continued learning of Python use the official documentation for Python standard library. Python essential reference offers a detailed exploration of Python. The various mailing lists and blog aggregates mentioned below are also very useful in learning Python and for keeping up with new developments.

This entry was posted in Astronomy, Python. Bookmark the permalink.

7 Responses to Python for Astronomy

  1. Great article.
    There is so mush libraries and frameworks. I have lost, can you help me to choose right one?
    I need to find a star coordinates (RA and DEC) for current time location. As far as I understood I need to convert current time to Julian time and some how find current DEC for star.

  2. Prasanth says:

    Hello,

    If you want to work with individual objects, then the best option is to use pyephem (http://rhodesmill.org/pyephem/). This allows one to create a star, perform various conversions, assign an observer and then calculate quantities such as rise time, setting time etc., . The pyehem site has very good tutorials and documentation. I have also written one, and is available at https://oneau.wordpress.com/2010/07/04/astrometry-in-python-with-pyephem/.

    The other packages are for users who have a good understanding of astrometry.

    There are currently plans to create a collection of astronomy related Python packages called AstroPy (http://astropy.org/). There are no definite release schedules, but when it is released it will offer a easy way of obtaining Python packages.

    Prasanth

  3. Prasanth says:

    Hello,

    See http://astro.unl.edu/animationsLinks.html for some animations explaining astrometry. Slightly more technical discussions are available at http://star-www.st-and.ac.uk/~fv/webnotes/index.html. Practical Astronomy With Your Calculator is a good book.

    Prasanth

  4. Jonathan Slavin says:

    Just thought I’d let you know that the link for astrobetter is wrong. It’s http://www.astrobetter.com (not .org).

  5. This is awesome, I may nab this for my own blog! I will of course, give full credit! :) Excellent stuff!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s