Merge pull request #238 from ianfixes/2020-12-08_toward_containers

Create a more complete CI environment in a Docker image

Author: ianfixes

.github/workflows/linux.yaml

@@ -11,7 +11,7 @@ jobs:
       - uses: ruby/setup-ruby@v1
         with:
           ruby-version: 2.6
-      - name: Check style, funcionality, and usage
+      - name: Check style, functionality, and usage
         run: |
           g++ -v
           bundle install

.github/workflows/macos.yaml

@@ -11,7 +11,7 @@ jobs:
       - uses: ruby/setup-ruby@v1
         with:
           ruby-version: 2.6
-      - name: Check style, funcionality, and usage
+      - name: Check style, functionality, and usage
         run: |
           g++ -v
           bundle install

.github/workflows/windows.yaml

@@ -11,7 +11,7 @@ jobs:
       - uses: ruby/setup-ruby@v1
         with:
           ruby-version: 2.6
-      - name: Check style, funcionality, and usage
+      - name: Check style, functionality, and usage
         run: |
           g++ -v
           bundle install

.rubocop.yml

@@ -1,5 +1,5 @@
 AllCops:
-  TargetRubyVersion: 2.6
+  TargetRubyVersion: 2.5
   NewCops: enable
   SuggestExtensions: false
   Exclude:
@@ -65,15 +65,15 @@ Layout/LineLength:
 # Configuration parameters: CountComments.
 Metrics/ClassLength:
   Enabled: false
-  Max: 400
 
 Metrics/AbcSize:
   Enabled: false
-  Max: 50
 
 Metrics/MethodLength:
   Enabled: false
-  Max: 50
+
+Metrics/BlockLength:
+  Enabled: false
 
 # Configuration parameters: CountKeywordArgs.
 Metrics/ParameterLists:

CHANGELOG.md

@@ -7,22 +7,35 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased]
 ### Added
+- Environment variable to run a custom initialization script during CI testing: `CUSTOM_INIT_SCRIPT`
+- Environment variable to run from a subdirectory during CI testing: `USE_SUBDIR`
+- `assertComparativeEquivalent()` and `assertComparativeNotEquivalent()` to evaluate equality on an `a - b == 0` basis (and/or `!(a > b) && !(a < b)`)
+- `assertEqualFloat()` and `assertNotEqualFloat()` for comparing floats with epsilon
+- `assertInfinity()` and `assertNotInfinity()` for comparing floats to infinity
+- `assertNAN()` and `assertNotNAN()` for comparing floats to `NaN`
+- `assertion()`, `ReporterTAP.onAssert()`, and `testBehaviorExp` macro to handle simple expression evaluation (is true, is false, etc)
+- `Wire.resetMocks()` and documentation
 
 ### Changed
+- Rubocop expected syntax downgraded from ruby 2.6 to 2.5
+- `assertEqual()` and `assertNotEqual()` use actual `==` and `!=` -- they no longer require a type to be totally ordered just to do equality tests
+- Evaluative assertions (is true/false/null/etc) now produce simpler error messages instead of masquerading as an operation (e.g. "== true")
 
 ### Deprecated
 
 ### Removed
 
 ### Fixed
+- Warnings about directory name mismatches are now based on proper comparison of strings
+- Now using the recommended "stable" URL for the `esp32` board family
 
 ### Security
 
 
 ## [1.1.0] - 2020-12-02
 ### Added
 - `ensure_arduino_installation.rb` now ensures the existence of the library directory as well
-- Environment variables to escalate unit tests or examples not being found during CI testing
+- Environment variables to escalate unit tests or examples not being found during CI testing - `EXPECT_EXAMPLES` and `EXPECT_UNITTESTS`
 
 ### Changed
 - Conserve CI testing minutes by grouping CI into fewer runs

README.md

@@ -3,6 +3,7 @@
 [![Gem Version](https://badge.fury.io/rb/arduino_ci.svg)](https://rubygems.org/gems/arduino_ci)
 [![Documentation](http://img.shields.io/badge/docs-rdoc.info-blue.svg)](http://www.rubydoc.info/gems/arduino_ci/1.1.0)
 [![Gitter](https://badges.gitter.im/Arduino-CI/arduino_ci.svg)](https://gitter.im/Arduino-CI/arduino_ci?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
+[![GitHub Marketplace](https://img.shields.io/badge/Get_it-on_Marketplace-informational.svg)](https://github.com/marketplace/actions/arduino_ci)
 
 You want to run tests on your Arduino library (bonus: without hardware present), but the IDE doesn't support that.  Arduino CI provides that ability.
 
@@ -12,6 +13,8 @@ You want your Arduino library to be automatically built and tested every time so
 
 `arduino_ci` is a cross-platform build/test system, consisting of a Ruby gem and a series of C++ mocks.  It enables tests to be run both locally and as part of a CI service like GitHub Actions, TravisCI, Appveyor, etc.  Any OS that can run the Arduino IDE can run `arduino_ci`.
 
+> Note: for running tests in response to [GitHub events](https://docs.github.com/en/free-pro-team@latest/developers/webhooks-and-events/github-event-types), an [Arduino CI GitHub Action](https://github.com/marketplace/actions/arduino_ci) is available for your convenience.  This method of running `arduino_ci` is driven by Docker, which may also serve your local testing needs (as it does not require a ruby environment to be installed).
+
 
 Platform | CI Status
 ---------|:---------
@@ -20,20 +23,9 @@ Linux    | [![Linux Build Status](https://github.com/Arduino-CI/arduino_ci/workf
 Windows  | [![Windows Build status](https://github.com/Arduino-CI/arduino_ci/workflows/windows/badge.svg)](https://github.com/Arduino-CI/arduino_ci/actions?workflow=windows)
 
 
-## Comparison to Other Arduino Testing Tools
-
-| Project                                                                     | CI | Builds Examples | Unittest | Arduino Mocks | Windows | OSX | Linux | License |
-|-----------------------------------------------------------------------------|:--:|:---------------:|:--------:|:-------------:|:-------:|:---:|:-----:|:--------|
-|[ArduinoCI](https://github.com/Arduino-CI/arduino_ci)                          | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |Free (Apache-2.0)|
-|[ArduinoUnit](https://github.com/mmurdoch/arduinounit)                       | ❌ | ❌ | ⚠️ Hardware-based|❌ | ✅ | ✅ | ✅ |Free (MIT)|
-|[Adafruit `ci-arduino`](https://github.com/adafruit/ci-arduino)| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ |Free (MIT)|
-|[PlatformIO](https://platformio.org)                                         | ✅ | ✅ | ⚠️ Paid only | ❌ | ✅ | ✅ | ✅ |⚠️ EULA|
-|Official [Arduino IDE](https://www.arduino.cc/en/main/software)              | ❌ | ⚠️ Manually | ❌ |N/A 😉| ✅ | ✅ | ✅ |Free (GPLv2)|
-
-
 ## Quick Start
 
-For a bare-bones example that you can copy from, see [SampleProjects/DoSomething](SampleProjects/DoSomething).
+For a fairly minimal practical example that you can copy from, see [the `Arduino-CI/Blink` repository](https://github.com/Arduino-CI/Blink).
 
 The complete set of C++ unit tests for the `arduino_ci` library itself are in the [SampleProjects/TestSomething](SampleProjects/TestSomething) project.  The [test files](SampleProjects/TestSomething/test/) are named after the type of feature being tested.
 
@@ -55,13 +47,20 @@ For unit testing, you will need a compiler; [g++](https://gcc.gnu.org/) is prefe
 * **Windows**: you will need Cygwin, and the `mingw-gcc-g++` package.  A full set of (working) install instructions can be found in `appveyor.yml`, as this is how CI runs for this project.
 
 
+### You _May_ Need `python`
+
+ESP32 and ESP8266 boards have [a dependency on `python` that they don't install themselves](https://github.com/Arduino-CI/arduino_ci/issues/235#issuecomment-739629243).  If you intend to test on these platforms (which are included in the default list of platforms to test against), you will need to make `python` (and possibly `pyserial`) available in the test environment.
+
+Alternately, you might configure `arduino_ci` to simply not test against these.  Consult the reference for those details.
+
+
 ### Changes to Your Repo
 
 Add a file called `Gemfile` (no extension) to your Arduino project:
 
 ```ruby
 source 'https://rubygems.org'
-gem 'arduino_ci'
+gem 'arduino_ci' '~> 1.1'
 ```
 
 It would also make sense to add the following to your `.gitignore`, or copy [the `.gitignore` used by this project](.gitignore):

REFERENCE.md

@@ -39,6 +39,16 @@ This allows a file (or glob) pattern to be executed in your tests directory, cre
 This allows a file (or glob) pattern to be executed in your tests directory, creating a blacklist of files to skip.  E.g. `--testfile-reject=test_animal_*.cpp` would match `test_animal_cat.cpp` and `test_animal_dog.cpp` (skipping those) and test only `test_plant_rose.cpp`, `test_plant_daisy.cpp`, etc.
 
 
+### `CUSTOM_INIT_SCRIPT` environment variable
+
+If set, testing will execute (using `/bin/sh`) the script referred to by this variable -- relative to the current working directory.  This enables use cases like the GitHub action to install custom library versions (i.e. a version of a library that is different than what the library manager would automatically install by name) prior to CI test runs.
+
+
+### `USE_SUBDIR` environment variable
+
+If set, testing will be conducted in this subdirectory (relative to the working directory).  This is for monorepos or other layouts where the library directory and project root directory are different.
+
+
 ### `EXPECT_UNITTESTS` environment variable
 
 If set, testing will fail if no unit test files are detected (or if the directory does not exist).  This is to avoid communicating a passing status in cases where a commit may have accidentally moved or deleted the test files.
@@ -198,15 +208,27 @@ This test defines one `unittest` (a macro provided by `ArduinoUnitTests.h`), cal
 
 The following assertion functions are available in unit tests.
 
-* `assertEqual(expected, actual)`
-* `assertNotEqual(expected, actual)`
-* `assertLess(expected, actual)`
-* `assertMore(expected, actual)`
-* `assertLessOrEqual(expected, actual)`
-* `assertMoreOrEqual(expected, actual)`
-* `assertTrue(actual)`
-* `assertFalse(actual)`
-* `assertNull(actual)`
+```c++
+assertEqual(expected, actual);               // a == b
+assertNotEqual(unwanted, actual);            // a != b
+assertComparativeEquivalent(expected, actual);    // abs(a - b) == 0 or (!(a > b) && !(a < b))
+assertComparativeNotEquivalent(unwanted, actual); // abs(a - b) > 0  or ((a > b) || (a < b))
+assertLess(upperBound, actual);              // a < b
+assertMore(lowerBound, actual);              // a > b
+assertLessOrEqual(upperBound, actual);       // a <= b
+assertMoreOrEqual(lowerBound, actual);       // a >= b
+assertTrue(actual);
+assertFalse(actual);
+assertNull(actual);
+
+// special cases for floats
+assertEqualFloat(expected, actual, epsilon);    // fabs(a - b) <= epsilon
+assertNotEqualFloat(unwanted, actual, epsilon); // fabs(a - b) >= epsilon
+assertInfinity(actual);                         // isinf(a)
+assertNotInfinity(actual);                      // !isinf(a)
+assertNAN(arg);                                 // isnan(a)
+assertNotNAN(arg);                              // !isnan(a)
+```
 
 These functions will report the result of the test to the console, and the testing will continue if they fail.
 
@@ -327,7 +349,7 @@ unittest(pin_history)
   // we expect 6 values in that queue (5 that we set plus one
   // initial value), which we'll hard-code here for convenience.
   // (we'll actually assert those 6 values in the next block)
-  assertEqual(6, state->digitalPin[1].queueSize));
+  assertEqual(6, state->digitalPin[1].queueSize());
   bool expected[6] = {LOW, HIGH, LOW, LOW, HIGH, HIGH};
   bool actual[6];
 
@@ -634,3 +656,47 @@ unittest(eeprom)
   assertEqual(10, a);
 }
 ```
+
+
+### Wire
+
+This library allows communication with I2C / TWI devices.
+
+The interface the library has been fully mocked, with the addition of several functions for debugging
+
+* `Wire.resetMocks()`: Initializes all mocks, and for test repeatability should be called at the top of any unit tests that use Wire.
+* `Wire.didBegin()`: returns whether `Wire.begin()` was called at any point
+* `Wire.getMosi(address)`: returns a pointer to a `deque` that represents the history of data sent to `address`
+* `Wire.getMiso(address)`: returns a pointer to a `deque` that defines what the master will read from `address` (i.e. for you to supply)
+
+```c++
+unittest(wire_basics) {
+  // ensure known starting state
+  Wire.resetMocks();
+
+  // in case you need to check that your library is properly calling .begin()
+  assertFalse(Wire.didBegin());
+  Wire.begin();
+  assertTrue(Wire.didBegin());
+
+  // pick a random device. master write buffer should be empty
+  const uint8_t randomSlaveAddr = 14;
+  deque<uint8_t>* mosi = Wire.getMosi(randomSlaveAddr);
+  assertEqual(0, mosi->size());
+
+  // write some random data to random device
+  const uint8_t randomData[] = { 0x07, 0x0E };
+  Wire.beginTransmission(randomSlaveAddr);
+  Wire.write(randomData[0]);
+  Wire.write(randomData[1]);
+  Wire.endTransmission();
+
+  // check master write buffer values
+  assertEqual(2, mosi->size());
+  assertEqual(randomData[0], mosi->front());
+  mosi->pop_front();
+  assertEqual(randomData[1], mosi->front());
+  mosi->pop_front();
+  assertEqual(0, mosi->size());
+}
+```

SampleProjects/DoSomething/test/bad-errormessages.cpp

@@ -0,0 +1,31 @@
+#include <ArduinoUnitTests.h>
+
+
+#pragma once
+
+
+
+unittest(check_that_assertion_error_messages_are_comprehensible)
+{
+  assertEqual(1, 2);
+  assertNotEqual(2, 2);
+  assertComparativeEquivalent(1, 2);
+  assertComparativeNotEquivalent(2, 2);
+  assertLess(2, 1);
+  assertMore(1, 2);
+  assertLessOrEqual(2, 1);
+  assertMoreOrEqual(1, 2);
+  assertTrue(false);
+  assertFalse(true);
+  assertNull(3);
+  assertNotNull(NULL);
+
+  assertEqualFloat(1.2, 1.0, 0.01);
+  assertNotEqualFloat(1.0, 1.02, 0.1);
+  assertInfinity(42);
+  assertNotInfinity(INFINITY);
+  assertNAN(42);
+  assertNotNAN(0.0/0.0);
+}
+
+unittest_main()

SampleProjects/DoSomething/test/good-assert.cpp

@@ -0,0 +1,57 @@
+#include <ArduinoUnitTests.h>
+#include "../do-something.h"
+
+class NonOrderedType {
+  public:
+    int x; // ehh why not
+    NonOrderedType(int some_x) : x(some_x) {}
+
+    bool operator==(const NonOrderedType &that) const {
+      return that.x == x;
+    }
+
+    bool operator!=(const NonOrderedType &that) const {
+      return that.x != x;
+    }
+};
+inline std::ostream& operator << ( std::ostream& out, const NonOrderedType& n ) {
+  out << "NonOrderedType(" << n.x << ")";
+  return out;
+}
+
+
+unittest(assert_equal_without_total_ordering)
+{
+  NonOrderedType a(3);
+  NonOrderedType b(3);
+  NonOrderedType c(4);
+
+  assertEqual(a, b);
+  assertEqual(a, a);
+  assertNotEqual(a, c);
+
+}
+
+unittest(float_assertions)
+{
+  assertEqualFloat(1.0, 1.02, 0.1);
+  assertNotEqualFloat(1.2, 1.0, 0.01);
+
+  assertInfinity(exp(800));
+  assertInfinity(1.0/0.0);
+  assertNotInfinity(42);
+
+  assertNAN(INFINITY - INFINITY);
+  assertNAN(0.0/0.0);
+  assertNotNAN(42);
+
+  assertComparativeEquivalent(exp(800), INFINITY);
+  assertComparativeEquivalent(0.0/0.0, INFINITY - INFINITY);
+  assertComparativeNotEquivalent(INFINITY,  -INFINITY);
+
+  assertLess(0, INFINITY);
+  assertLess(-INFINITY, 0);
+  assertLess(-INFINITY, INFINITY);
+}
+
+unittest_main()

SampleProjects/TestSomething/test/godmode.cpp

@@ -146,7 +146,7 @@ unittest(analog_pin_write_history) {
 }
 
 unittest(ascii_pin_write_history) {
-  // digitial history as serial data, big-endian
+  // digital history as serial data, big-endian
   bool binaryAscii[24] = {
       0, 1, 0, 1, 1, 0, 0, 1,
       0, 1, 1, 0, 0, 1, 0, 1,
@@ -157,7 +157,7 @@ unittest(ascii_pin_write_history) {
 
   assertEqual("Yes", state->digitalPin[2].toAscii(1, true));
 
-  // digitial history as serial data, little-endian
+  // digital history as serial data, little-endian
   bool binaryAscii2[16] = {
       0, 1, 1, 1, 0, 0, 1, 0,
       1, 1, 1, 1, 0, 1, 1, 0};