ReadStatTables.jl

Read and write Stata, SAS and SPSS data files with Julia tables

ReadStatTables.jl
is a Julia package for reading and writing Stata, SAS and SPSS data files with
Tables.jl-compatible tables.
It utilizes the ReadStat C library
developed by Evan Miller
for parsing and writing the data files.
The same C library is also the backend of popular packages in other languages such as
pyreadstat for Python
and haven for R.
As the Julia counterpart for similar purposes,
ReadStatTables.jl leverages the state-of-the-art Julia ecosystem
for usability and performance.
Its read performance, especially when taking advantage of multiple threads,
surpasses all related packages by a sizable margin
based on the benchmark results
here:

Features

ReadStatTables.jl provides the following features in addition to
wrapping the C interface of ReadStat:

• Fast multi-threaded data collection from ReadStat parsers to a Tables.jl-compatible ReadStatTable
• Interface of file-level and variable-level metadata compatible with DataAPI.jl
• Integration of value labels into data columns via a custom array type LabeledArray
• Translation of date and time values into Julia time types Date and DateTime
• Write support for Tables.jl-compatible tables (experimental)

Supported File Formats

ReadStatTables.jl recognizes data files with the following file extensions at this moment:

• Stata: .dta
• SAS: .sas7bdat and .xpt
• SPSS: .sav and .por

Installation

ReadStatTables.jl can be installed with the Julia package manager
Pkg.
From the Julia REPL, type ] to enter the Pkg REPL and run:

pkg> add ReadStatTables

Quick Start

To read a data file located at data/sample.dta:

julia> using ReadStatTables
julia> tb = readstat("data/sample.dta")
5×7 ReadStatTable:
 Row │  mychar    mynum      mydate                dtime         mylabl           myord               mytime 
     │ String3  Float64       Date?            DateTime?  Labeled{Int8}  Labeled{Int8?}             DateTime 
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │       a      1.1  2018-05-06  2018-05-06T10:10:10           Male             low  1960-01-01T10:10:10
   2 │       b      1.2  1880-05-06  1880-05-06T10:10:10         Female          medium  1960-01-01T23:10:10
   3 │       c  -1000.3  1960-01-01  1960-01-01T00:00:00           Male            high  1960-01-01T00:00:00
   4 │       d     -1.4  1583-01-01  1583-01-01T00:00:00         Female             low  1960-01-01T16:10:10
   5 │       e   1000.3     missing              missing           Male         missing  2000-01-01T00:00:00

To access a column from the above table:

julia> tb.myord
5-element LabeledVector{Union{Missing, Int8}, Vector{Union{Missing, Int8}}, Union{Char, Int32}}:
 1 => low
 2 => medium
 3 => high
 1 => low
 missing => missing

Notice that for data variables with value labels,
both the original values and the value labels are preserved.
For variables representing date/time,
the translation to Julia Date/DateTime is lazy.
One can access the underlying numerical values as follows:

julia> tb.mydate.data
5-element SentinelArrays.SentinelVector{Float64, Float64, Missing, Vector{Float64}}:
   21310.0
  -29093.0
       0.0
 -137696.0
        missing

File-level and variable-level metadata can be retrieved and modified
via methods compatible with DataAPI.jl:

julia> metadata(tb)
ReadStatMeta:
  row count           => 5
  var count           => 7
  modified time       => 2021-04-23T04:36:00
  file format version => 118
  file label          => A test file
  file extension      => .dta
julia> colmetadata(tb, :mylabl)
ReadStatColMeta:
  label         => labeled
  format        => %16.0f
  type          => READSTAT_TYPE_INT8
  value label   => mylabl
  storage width => 1
  display width => 16
  measure       => READSTAT_MEASURE_UNKNOWN
  alignment     => READSTAT_ALIGNMENT_RIGHT

For more details, please see the documentation.