Skip to content

Releases: caltechlibrary/datatools

new cli, bug fixes and cli option normalization

13 Apr 21:45
Compare
Choose a tag to compare

Changes:

  • csvcols has an option change, -f is now -col

Additions:

  • csvfind will help you find things in a CSV file by searching a single colum
    • search options include exact match, contains and Levenshtein edit distance
  • csvjoin will let you join two CSV files on a column
    • includes support for approximate matching via Lenenshtein edit distance on join column

Documentation has been re-organized. Beginning to add a "How To" section for website. Lots of small BUG fixes.

datatools Go package is beginning to combine common functionality used between the CLI. dotpath continues to be debugged.

Added csv2json cli

11 Apr 18:08
Compare
Choose a tag to compare

Added a new command line tool named csv2json. This will convert a CSV file into a series of JSON blobs output as either a JSON array or one JSON blob per line. Use case is turning a CSV file into a set of JSON blobs that could be easily stored in dataset.

Minor tweaks, bug fixes and improvements

10 Apr 16:25
Compare
Choose a tag to compare
  • Adding leading and trailing new line for CSV to Markdown table conversion (csv2mdtable).
  • Improved docs.
  • Linted and formatted Bash scripts.
  • Improved Makefile

jsonrange refinement

01 Mar 22:22
Compare
Choose a tag to compare

In this release updates were focused on jsonrange. It now returns keys by default, handles input/output like jsoncols and can returns lengths or last key index, and values depending on how you set options.

dotpath replacing jid

28 Feb 23:56
Compare
Choose a tag to compare

The primary purpose of the release is the swapping out of jid for a local submodule named dotpath.
This removes a dependency for an external package and keeps the code minimal (dot path in only about 150 lines). It does mean that interactive mode of jsoncols has been removed. If you need this then use jid which is nice for this type of exploration. There are sure to be bugs introduced by this switch... YMMV

Proof of concept data tools for CSV, JSON and XLSX files

06 Feb 20:48
Compare
Choose a tag to compare

This is a set of tools for working with JSON, CSV and Excel Workbook content. It includes the following
tools

  • csvcols - filters or display CSV row outpout
  • csvjoin - joins two CSV files by match designated column values in each file
  • jsoncols - filters JSON dot paths into columns in CSV format (by default)
  • jsonrange - iterates over JSON object (e.g. an array identified by a dot path) rendering one element per line
  • xlsx2json - converts an Excel Workbook sheet into a JSON 2d array
  • xlsx2csv - converts an Excel Workbook sheet into CSV
  • csv2mdtable - converts CSV into a Github Flavored Markdown table
  • csv2xlsx - convert CSV into a designated Excel Workbook sheet