Linux Foundation to Host the PaSh Project, Accelerating Shell Scripting with Automated Parallelization for Industrial Use Cases

Rooted in academia, the PaSh project prepares for industrial adoption

News provided by

Sep 21, 2021, 11:00 ET

SAN FRANCISCO, Sept. 21, 2021 /PRNewswire/ -- The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced it will host the PaSh project. PaSh is a system for automatically parallelizing POSIX shell scripts that optimizes programs and speeds up execution times, leading to faster results for data scientists, engineers, biologists, economists, administrators, and programmers.

The project is supported by MIT, Rice University, Stevens Institute of Technology, and the University of Pennsylvania and governed by a Technical Steering Committee that includes Nikos Vasilakis, research scientist at MIT; Michael Greenberg, assistant professor at Stevens Institute of Technology; and Konstantinos Kallas, Ph.D. student at University of Pennsylvania.

"The Linux Foundation offers the technical governance infrastructure and services that PaSh has come to require as it has become more mature," said Nikos Vasilakis, Technical Steering Committee chair for the PaSh Project. "We created the project to improve upon and accelerate the execution of shell scripts in the face of new web crawling, indexing and natural language processing changes."

Konstantinos Kallas, Technical Steering Committee member for the PaSh project, adds "PaSh has received significant attention from the academic community and has the potential for an outsized impact in industry today, due to several synergistic trends. This is the right move at the right time."

The techniques underpinning the project have gathered several awards, including a best paper award at the 16th edition of the EuroSys conference and a graduate research award for Konstantinos by the Association of Computing Machinery.

From succinct data wrangling and everyday processing to system orchestration and administration, shell scripting—the ever-present glue holding commands developed in different languages together—forms the backbone of today's computing userspace. PaSh parallelizes shell scripts, achieving significant performance improvements. On modern multiprocessor computers, PaSh can execute tasks like web crawling and indexing, COVID19-related analyses, natural-language processing, and other workloads in a fraction of their original time with minimal-to-zero developer effort.

"Shell scripts have been used pervasively for half a century, and recent trends towards 'containerization' have only increased their importance," said Michael Greenberg, Technical Steering Committee member for the PaSh Project. "The correct and automated parallelization of shell scripts has been an issue for several decades. PaSh promises a speed boost to shell users of all stripes."

To accelerate shell scripts, PaSh provides a source-to-source parallelizing compiler, a program that takes as input a programmer's shell script and returns a new program that is significantly faster than the original program. Since PaSh is source-to-source, it allows the optimized shell script to be inspected and executed using the same tools, in the same environment, and with the same data as the original script. A small runtime library and associated annotations on programs commonly used in shell scripts complete the picture, providing the PaSh compiler with high-performance primitives and supporting its key functions.

"The PaSh Project represents innovation in computer science and open source software," said Mike Dolan, general manager and senior vice president of Projects at the Linux Foundation. "As software development evolves to address machine learning, containerization, artificial intelligence and more, PaSh shows up to support developers and data scientists who need more out of their scripting tools. We're happy to be able to host this important work at Linux Foundation, a natural home for a project like this."

To learn more and to get involved in the PaSh project, please visit http://binpa.sh/

About the Linux Foundation
Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world's leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation's projects are critical to the world's infrastructure including Linux, Kubernetes, Node.js, and more. The Linux Foundation's methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.

Media Contact
Jennifer Cloer
Story Changes Culture
503-867-2304
[email protected]

SOURCE The Linux Foundation