Enthought Awarded $1M DOE SBIR Grant to Develop Python HPC Framework

Share Article

Software will bring the strengths of Python and NumPy to high-performance computing (HPC) and improve the usability of Trilinos HPC solvers.

Enthought, Inc. today announced that the company has been awarded a $1M Small Business Innovation and Research (SBIR) grant by the United States Department of Energy (DOE) to expand the capabilities of Python and NumPy for high-performance distributed computing.

The open-source Python HPC framework being developed under this Phase II SBIR will help address the growing need to easily access parallel computing resources by bringing the strengths and ease-of-use of the popular Python programming language and NumPy multidimensional arrays to high-performance and parallel computing.

The innovative framework will be comprised of three components, known as “packages” in open-source software parlance. The packages address current issues hindering software computing on HPC systems: accessibility and ease-of use for non-computer scientists to leverage existing codes and resources for developing solutions, distributed array computing, and coding for node-level speed-up of computations.

The first component will improve accessibility to the “The Trilinos Project”, a set of sophisticated algorithms and technologies used for solving large-scale, complex physics, engineering and scientific problems, such as those encountered in ocean modeling, Formula 1 race car design, nuclear engineering, digital dentistry and medical imaging. By wrapping key Trilinos packages in Python, a barrier-to-entry to Trilinos is removed for Python developers.

Under Phase I of the DOE SBIR funded project, Python wrappers, called PyTrilinos, were developed and extended for Trilinos packages to improve their ease of use and accessibility.

“These Trilinos packages, developed primarily at Sandia National Laboratories, allow scientists to solve partial differential equations and large linear, nonlinear, and optimization problems in parallel, from desktops to distributed clusters to supercomputers, with active research on modern architectures such as GPUs,” states Bill Spotz, senior research scientist at Sandia. “This next phase of the project will improve and continue to expand these PyTrilinos interfaces, making Trilinos easier to use.” Spotz will lead the PyTrilinos effort for the Python HPC framework.

The second component in the framework is Optimized Distributed NumPy array, known as ODIN. ODIN will provide a generally useful distributed array object that will be familiar to users of the NumPy array package, providing end users an easy to use array object that can handle large amounts of data in parallel. ODIN will be compatible with PyTrilinos and will further enhance the ease of use of these powerful Trilinos packages, allowing users to easily create and manipulate distributed arrays from Python that work with Trilinos solvers. Enthought scientific software developer Kurt Smith will lead this development.

To maximize interoperability with other libraries, a distributed array protocol will be developed, akin to the PEP-3118 buffer protocol. This distributed protocol will allow interoperability with the Global Arrays and the Global Arrays in NumPy (GAIN) projects based out of Pacific Northwest National Laboratory (PNNL), which provide a “shared memory” programming interface for distributed-memory computers.

Computational scientist Jeff Daily, who leads GAIN development at PNNL, will help in this effort. “The Global Arrays in NumPy project is building a parallel distributed array structure compatible with NumPy’s array structure using the Global Arrays library developed at PNNL. The GAIN project will make it easy for scientists to develop high performance computational software on today’s leading supercomputers as well as on their desktop systems. This will reduce the time to solution in areas such as computational chemistry, fluid dynamics, and climate analysis,” says Daily.

The third component of the Python HPC Framework is Seamless. The Seamless project allows users to easily just-in-time (JIT) -compile Python and to readily interface Python with external languages such as C and C++. Seamless is build on top of the LLVM project, and uses its powerful introspection and code generation capabilities to make cross-language integration nearly effortless.

“This Python HPC framework -- composed of the PyTrilinos wrappers, ODIN, and Seamless -- will make parallel computing easier to use by bringing the user-friendly NumPy interface to HPC,” states Smith. “The framework will make parallel computing more intuitive by reducing the mental overhead required to work with distributed arrays, and more useful by enabling access to HPC solvers. We are extremely grateful that the DOE recognizes the importance of these software innovations for improving the development landscape for high performance computing. We plan to make the three components freely available as open-source software, providing a platform for researchers, analysts, engineers, and scientists to explore large data sets, to prototype and simulate physical models, and to build commercial scientific applications.”

For more details, please see the Enthought blog.

About Enthought

Founded in 2001, Enthought, Inc. has a mission to improve scientific and analytic computing by providing tools and services for data analysis and visualization. Enthought has an exceptional customer base comprised of some of the most respected names in the financial services, oil and gas, consumer products, and technology industries. Headquartered in Austin, Texas, Enthought has offices in New York City, Cambridge (United Kingdom), and Mumbai (India).

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Brett Murphy