Jeroen Janssens shows how the command line can make everyday data science work faster and more flexible. Instead of reaching for new technology, the book teaches you to combine small, focused command-line tools, in the spirit of the Unix philosophy, to obtain, scrub, explore, and model data. The specific tools matter less than the underlying way of working with tools, pipes, and data, which is what the book aims to pass on.
The only assumption is that you work with data.
Concepts are explained from the beginning, and a Docker image bundles more than 100 command-line tools so you can follow along on Windows, macOS, or Linux without installing each one yourself.
Some examples use Bash, Python, and R, so programming experience helps but isn’t required.
The second edition is a thorough revision: it swaps in faster or better-maintained tools such as xsv, pup, and Vowpal Wabbit, replaces the virtual machine with Docker, rewrites the project management chapter around make, runs R from the shell with the rush package, and adds a new chapter on combining the command line with Python, R, and Apache Spark.
You’ll learn how to:
- Download data from the web, query databases and APIs, and convert spreadsheets
- Clean plain text, CSV, HTML, XML, and JSON
- Turn one-liners and existing Python or R code into reusable tools
- Manage a data science workflow with
make - Explore data, compute descriptive statistics, and create visualizations
- Run pipelines in parallel and distribute them across machines
- Model data with dimensionality reduction, regression, and classification
- Use the command line from Jupyter, RStudio, and Apache Spark
The book is free to read online.