-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Labels
Description
Data is partitioned across 3 axes and making this structure explicit by using an arrow-like partition structure may be helpful:
inst/extdata/
├── 2017
│ ├── all
│ ├── home
│ ├── other
│ ├── school
│ └── work
└── 2020
├── all
│ ├── all
│ ├── rural
│ └── urban
├── home
│ ├── all
│ ├── rural
│ └── urban
├── other
│ ├── all
│ ├── rural
│ └── urban
├── school
│ ├── all
│ ├── rural
│ └── urban
└── work
├── all
├── rural
└── urban
If we made this switch, we may as well store the data as parquet and use arrow for better performance.