R works best when data fits in memory, but as datasets grow, analyses can slow down or stop working entirely. Written by developers of the arrow R package, this book shows how to analyze larger-than-memory data directly from R, on your own laptop or workstation, without setting up a database or cluster.

Because arrow implements the dplyr interface, you can query directories of CSV or Parquet files with code you already know, and the package processes the data in chunks rather than loading it all at once. The book also explains the origins and goals of the Apache Arrow project and why it helps connect data science tools with the wider big data ecosystem. It is aimed at R users who want to scale up their data processing without learning a whole new framework.

Topics covered include:

  • The Apache Arrow project and how the arrow R package fits into it
  • Manipulating data with familiar dplyr syntax
  • Choosing and working with file formats such as CSV and Parquet
  • Working with multi-file datasets that are larger than memory
  • Querying data stored in cloud services such as Amazon S3 and Google Cloud Storage
  • Extending arrow with user-defined functions and Acero, Arrow’s query engine
  • Integrating with DuckDB and other tools, and working with geospatial data
  • Sharing data across languages and systems

The full text is available online at the book’s companion website.