Parsing and typechecking for new null-terminated array types.

[secure-sw-dev-bot]

This issue was copied from checkedc/checkedc-clang#394

---

This change adds parsing and type checking for two new kinds of Checked C types: null-terminated arrays and pointers to null-terminated arrays. Null-terminated arrays are written using the nt_checked keyword in place of the checked keyword. An example declaration is int arr nt_checked[10];. Pointers to null-terminated arrays are constructed using the nt_array_ptr keyword instead of the array_ptr keyword (the backward-compatible names for the keywords are _Nt_checked and _Nt_array_ptr).

To represent nt_array_ptrs in the IR, the existing enumeration for the different kinds of pointers is extended. For nt_checked arrays, the boolean indicating whether or not an array is checked is changed to an enumeration. The parsing changes are straightforward. The new keywords are recognized at the points where the existing Checked C keywords are recognized. They are then mapped to the appropriate enumeration tag.

For type checking, the profile methods for type object memorization/construction are changed to take into account the enumeration changes. We also add checks that null-terminated arrays and pointers are only constructed from integer or pointer types (the assumption for now is that the integer 0 will be the null terminator, so only data for which 0 is a valid value can be null-terminated). We do not allow nt_array_ptrs of void (the problem is that conversions through void could allow the null terminator to be partially overwritten when types larger than a character are involved).

With these rules we can construct interesting data structures, such as arrays of null-terminated pointers (i.e. arrays of strings), arrays of null-terminated pointers to null-terminated pointers (and so on), and arrays of null-terminated arrays (arrays of pre-allocated strings of fixed sizes).

Most of the typechecking changes are extending the implicit conversion logic in SemaExpr.cpp to implement the implicit conversion rules for nt_array_ptr. An nt_array_ptr can be converted implicitly to an array_ptr or ptr (bounds checking and checking of bounds declarations will prevent the null terminator from being overwritten). However, implicit conversions in the other direction are not allowed. An unchecked pointer, array_ptr, or ptr cannot be converted implicitly to an array_ptr. The data in memory may not be null-terminated or, if it is, there may be an alias that allows the terminator to be overwritten.

For conditional expressions (e1 ? e2 : e3), the implicit conversion rules are a bit more complex. If the type of one arm of the conditional expression is an nt_array_ptr and the type of the other arm is an unchecked pointer or array_ptr, the resulting type is an array_ptr. Bounds checking will prevent the null terminator from being overwritten.

For pointers to arrays, we allow an unchecked array to be converted to a checked array, but not vice versa. We do not allow an unchecked array to be converted to an nt_array.

This change contains code for one additional issue: we now disallow array_ptrs of function types (checkedc/checkedc#34). This is because functions are indeterminate in size, so no bounds checking is possible. We allow array_ptrs of ptrs to function types.

Testing:

• Extended the existing Checked C typechecking tests to cover nt_array_ptrs and checked arrays. Most of this work for this change went into extending those tests. These tests are pretty comprehensive and cover many situations where types are checked. This will we be covered by a separate pull request for the Checked C repo.
• Passes existing automated testing for Linux and Windows, including the LNT test suite and LNT tests that have been converted to Checked C.