BEACON is an application used to collect, manage, and present weather data from the National Water and Climate Center (NWCC) Air and Water Database (AWDB). Data are obtained using the API from nrcs.usda.gov.
Use your favorite version control tool to pull the repository from here
BEACON depends on the following libraries being installed on the
machine the WeatherDataPipeline.py is run from:
| Installed with Python | Additional Install with PIP |
|---|---|
| datetime | dbt |
| json | pandas |
| logging | yaml |
| os | zeep |
BEACON utilizes PostgreSQL as its database management system (DBMS), and Metabase as its data visualization tool. These tools will need to be installed on the machine, along with any required dependencies for those installations.
With the BEACON directory added to your machine, either
manually run the WeatherDataPipeline.py script or schedule it with your favorite
scheduling app.
The final .csv output is written to the /Staging directory by default,
unless manually changed using the stgFolder argument in the config.json
file located in */WeatherDataPipeline/Configuration.
Logs are written to the /Logs directory. Individual logs are retained
by default for 30 days, unless manually changed using the
logRetentionDays argument in the config.json file located in
*/WeatherDataPipeline/Configuration.
To import the daily data into the BEACON database, run the
dbt run command from the */WeatherDataPipeline directory, followed by
the dbt docs generate command.