feat: json/curl string operation boundary (#359)

* feat: string comparison as byte-span

* feat: enforce C string semantic in CURL string options

* fix: misc issues

Author: taedlar

docs/efuns/perform_using.md

@@ -22,14 +22,23 @@ that object already has a transfer in progress.
 - `"timeout_ms"` with a number value sets the libcurl timeout in milliseconds.
 - `"follow_location"` with a non-zero number value enables redirect following.
 
+For string-based option values (`"url"`, `"headers"`, and string `"post_data"`/`"body"`),
+embedded NUL bytes are rejected. If binary data with embedded NUL bytes is
+intentional for the request body, pass a `buffer` to `"post_data"`/`"body"`.
+
+This matches the driver's explicit C-string boundary behavior in related efuns:
+`c_str()` and `to_json()` both ensure C-string-safe contents when LPC code
+needs text data routed through C-string-oriented APIs.
+
 Passing `0` clears `"url"`, `"headers"`, or `"post_data"`/`"body"`.
 
 ## ERRORS
 An error is raised when:
 - `opt` is not a string
 - `val` has the wrong type for the selected option
+- a string value for `"url"`, `"headers"`, or string `"post_data"`/`"body"` contains embedded NUL bytes
 - the current object already has an active `perform_to()` transfer
 - the option name is unknown
 
 ## SEE ALSO
-[perform_to()](perform_to.md), [in_perform()](in_perform.md)
+[perform_to()](perform_to.md), [in_perform()](in_perform.md), [c_str()](c_str.md), [to_json()](to_json.md)

docs/plan/counted-string.md

@@ -34,25 +34,27 @@ early as possible (compile-time preferred, runtime as backstop).
 
 ### Near-Term Focus
 
-- Prioritize byte-span readiness for comparison semantics in production paths
-  (equality/inequality complete; next grouping/ordering and related helpers).
+- Keep comparison byte-span coverage stable; production comparison paths are
+  complete for current scope (equality/inequality, ordering operators, shared
+  helper paths).
 - Promote transparent boundary handles to abstract handles so boundary misuse fails at
   compile time.
 - Keep JSON boundary contract coverage stable, including CURL ingress
-  (`buffer -> from_json`) and explicit UTF-8/error-handling behavior.
-- Expand regression coverage to lock in counted-string semantics across LPC,
-  efuns, and JSON boundaries.
+  (`buffer -> from_json`), `perform_using` C-string boundaries, and explicit
+  UTF-8/error-handling behavior.
+- Maintain regression coverage for JSON/CURL efun boundaries (`perform_using`,
+  `c_str()`, `to_json()`) as option surface evolves.
 
 ## Status
 
 | Stage | Status |
 |---|---|
 | Foundation: `blkend` model + shared-table non-NUL lookup + initial safety/tests | complete |
 | Implementation: VM/operator NUL-removal and span API normalization | complete |
-| Implementation: comparison byte-span readiness | in progress (equality/inequality complete; next grouping/ordering semantics) |
+| Implementation: comparison byte-span readiness | complete |
 | Implementation: abstract typed handles + runtime contract enforcement | complete |
 | Implementation: C++ RAII wrapper adoption on exception baseline | complete |
-| Implementation: efun byte-span readiness | in progress (narrowed: defer broad LPC string-efun hardening; prioritize JSON/CURL ingress) |
+| Implementation: efun byte-span readiness | complete for narrowed JSON/CURL ingress scope (broad LPC string-efun hardening remains deferred by plan) |
 | Implementation: JSON boundary contract and tests | complete |
 | Validation: end-to-end LPC/JSON regression matrix | complete |
 
@@ -92,12 +94,52 @@ Completed milestones since the previous handoff:
 - Test strategy was tightened for this milestone: equality tests now execute
   real operator paths (`f_eq`/`f_ne`) on stack operands instead of simulating
   behavior with helper-level `memcmp` assertions.
+- Ordering comparison migration is complete for current scope:
+  `f_lt`/`f_le`/`f_gt`/`f_ge` use byte-span lexical comparison semantics
+  (embedded `\0` participates; length is tie-breaker after equal prefixes),
+  and array helper string comparison paths (`sameval`, builtin sort comparators)
+  now route through the same span-aware compare behavior.
+- Regression coverage now includes direct ordering-operator tests with embedded
+  null bytes (`OrderingOperatorsUseByteSpanForEmbeddedNul`,
+  `OrderingOperatorsUseLengthAfterEqualPrefix`) plus array helper equality
+  semantics (`SamevalUsesByteSpanForEmbeddedNulStrings`).
+- LPC-level ordering validation now includes `sort_array` with embedded-null
+  strings (`LpcSortArrayOrdersEmbeddedNulByByteSpan`), with inputs constructed
+  via `from_json("\"...\\u0000...\"")` to ensure full byte-span payloads are
+  exercised through real efun/operator/runtime paths.
+- Directional coverage for LPC sort ordering is now explicit: both ascending
+  and descending embedded-null cases are regression-tested
+  (`LpcSortArrayOrdersEmbeddedNulByByteSpan`,
+  `LpcSortArrayOrdersEmbeddedNulByByteSpanDescending`).
+- JSON/CURL ingress hardening advanced at the `perform_using` boundary:
+  string values for `url`, `headers`, and string `post_data`/`body` now reject
+  embedded NUL bytes, while binary payload intent remains supported via
+  `buffer` for `post_data`/`body`.
+- C++ wrapper-style read access was expanded in `lib/curl/curl_efuns.cpp`
+  for `perform_using`/`perform_to` validation and string-boundary checks via
+  `lpc::const_svalue_view`, reducing direct raw `svalue_t` field access on
+  those paths.
+- `perform_using` embedded-NUL rejection matrix coverage is now complete for
+  current option scope: regression tests cover `url`, string `body`,
+  string `headers`, and string-array `headers` entry rejection paths, plus the
+  binary body `buffer` allow-path.
+- JSON-derived boundary routing coverage is now explicit for current CURL
+  text-option scope: `perform_using(url, from_json(...))` rejects embedded-NUL
+  strings, while `perform_using(url, c_str(from_json(...)))` succeeds via
+  explicit C-string boundary conversion; similarly,
+  `perform_using(body, to_json(from_json(...)))` succeeds where
+  `perform_using(body, from_json(...))` is rejected.
 
 Validation snapshot:
 
 - Linux validation is green (`ctest --preset ut-linux`).
 - Cross-platform validation for the counted-string hardening set remains green
   (`vs16-x64`, `vs16-win32`, `clang-x64`).
+- As of 2026-04-17, full test runs on `vs16-x64` and `clang-x64` are confirmed
+  green for this milestone.
+- As of 2026-04-17, post-hardening Linux CTest sweep is green after adding
+  `perform_using` header matrix and `c_str()`/`to_json()` boundary-routing
+  regressions.
 
 Still intentional / not candidates for replacement:
 
@@ -114,13 +156,13 @@ Notable bug fix retained for handoff context:
 
 ### Next Focus
 
-- **Comparison byte-span readiness (next milestone)** — finish migrating and
-  validating LPC string comparison semantics end to end under byte-span rules.
-  Equality/inequality production behavior and regression coverage are complete;
-  next target is grouping/ordering paths and shared comparison helpers.
 - **JSON/CURL boundary follow-through** — keep existing JSON/CURL ingress
   coverage stable as counted-string and efun refactors continue, and add
   targeted regression tests only when new boundary behaviors are introduced.
+- **Next high-priority efun hardening (JSON/CURL ingress)** — prioritize
+  keeping `c_str()` / `to_json()` boundary-path behavior stable as future
+  CURL option surface expands, so explicit C-string conversion remains the only
+  truncation boundary and is behaviorally locked.
 
 ### Baseline and Out of Scope
 
@@ -295,8 +337,8 @@ Migration order:
 | P0 | Normalize internal helper APIs | string construction/lookup boundaries | New/updated helpers accept explicit lengths/spans; touched callers stop using sentinel termination as logical length; shared-string boundaries continue to route via `make_shared_string(s,end)` / `findstring(s,end)`. |
 | P0 | Enforce counted-string semantic boundaries | [src/stralloc.h](../../src/stralloc.h), [lib/lpc/types.h](../../lib/lpc/types.h), typed-string boundaries | Boundary-handle mode enabled under `STRING_TYPE_SAFETY`; contract APIs require explicit typed handles or bridge helpers; runtime contract checks remain release-enabled; identifier-class shared strings remain NUL-terminated. |
 | P1 | C++ wrapper adoption on baseline boundaries | C++ wrappers around `svalue_t` | `lpc::svalue_view`/`lpc::svalue` introduced without C ABI layout change; no duplicate exception-boundary rewrites are introduced; wrapper move/dtor are `noexcept`; unit-test-first ownership migration for counted-string targets is complete (no `free_string_svalue` / `free_svalue` in `tests/**/*.cpp`); remaining work is production C++ efun/helper adoption and any targeted perf checks for newly touched hot paths. |
-| P1 | Efun byte-span readiness (deferred broad LPC string paths) | [lib/efuns/string.c](../../lib/efuns/string.c), [lib/efuns/unsorted.c](../../lib/efuns/unsorted.c), [lib/efuns/sprintf.c](../../lib/efuns/sprintf.c), [lib/efuns/sscanf.c](../../lib/efuns/sscanf.c) | Scope is intentionally narrowed for this phase: no broad LPC-side behavioral expansion unless required by JSON/CURL boundary safety. Any touched path must preserve LPC compatibility and existing tests remain green. |
-| P1 | LPC operator semantics vs C-string efuns | [lib/lpc/operator.c](../../lib/lpc/operator.c), [lib/lpc/array.c](../../lib/lpc/array.c), [docs/efuns/strcmp.md](../../docs/efuns/strcmp.md) | Equality/inequality byte-span semantics are complete and regression-covered; remaining work migrates grouping/ordering helpers to full byte-span behavior (including embedded null participation) while `strcmp()` remains explicitly documented and tested as C-string-oriented behavior. |
+| P1 | Efun byte-span readiness (deferred broad LPC string paths) | [lib/efuns/string.c](../../lib/efuns/string.c), [lib/efuns/unsorted.c](../../lib/efuns/unsorted.c), [lib/efuns/sprintf.c](../../lib/efuns/sprintf.c), [lib/efuns/sscanf.c](../../lib/efuns/sscanf.c), [lib/curl/curl_efuns.cpp](../../lib/curl/curl_efuns.cpp) | complete for narrowed JSON/CURL ingress scope: `perform_using` rejects embedded-NUL string values for `url`/`headers`/string body, preserves binary payload path via `buffer`, and regression coverage includes header single/array rejection plus explicit `c_str()`/`to_json()` routing checks for JSON-derived string inputs in CURL text options. Broad LPC string-efun hardening remains deferred by plan. |
+| P1 | LPC operator semantics vs C-string efuns | [lib/lpc/operator.c](../../lib/lpc/operator.c), [lib/lpc/array.c](../../lib/lpc/array.c), [docs/efuns/strcmp.md](../../docs/efuns/strcmp.md) | complete for current scope: equality/inequality and grouping/ordering helper paths use full byte-span behavior (including embedded null participation); `strcmp()` remains explicitly documented and tested as C-string-oriented boundary behavior. |
 | P1 | JSON boundary contract (priority: CURL buffer ingress) | JSON efuns/helpers (`from_json`, `to_json`) and CURL ingress paths | complete: contract docs state LPC byte spans vs JSON text; `from_json` rejects invalid UTF-8 and raises LPC runtime error on invalid sequences; coverage includes explicit UTF-8 pass/fail tests (`fromJsonValidUtf8StringAccepted`, `fromJsonInvalidUtf8StringError`, `fromJsonInvalidUtf8BufferError`) plus buffer ingress success/error/size paths (`fromJsonBuffer`, `fromJsonInvalidBufferError`, `fromJsonLargeBuffer`) and a CURL callback payload integration test (`CurlBufferPayloadParsesViaFromJson`); JSON strings containing embedded null bytes are stored and copied byte-for-byte (full span), not truncated at C-string boundaries. |
 | P1 | Unicode and escape consistency | JSON encode/decode implementation | complete: encoder/decoder symmetry is now covered for control escapes, `\\`, `\"`, `\uXXXX`, surrogate pairs, and non-BMP round-trips via `toJsonControlEscapes`, `fromJsonControlEscapes`, `fromJsonSurrogatePairAccepted`, `fromJsonLoneHighSurrogateError`, and `roundTripNonBmpCharacter`; contract docs are aligned in `docs/efuns/from_json.md` and `docs/efuns/to_json.md`. |
 | P2 | End-to-end regression matrix | LPC-level and efun/unit tests | complete for current hardening scope: dedicated unit suite `tests/test_string_operators` added (21 cases, discovered via CTest), and full matrix validation is passing on Linux, VS16 x64, VS16 win32, and clang x64. Future LPC/JSON round-trip and negative-matrix additions remain follow-on expansion work. |

lib/curl/curl_efuns.cpp

@@ -260,7 +260,8 @@ static int ensure_easy_handle(curl_http_t *handle) {
 }
 
 static int is_clear_value(const svalue_t *value) {
-  return value->type == T_NUMBER && value->u.number == 0;
+  auto view = lpc::const_svalue_view::from(value);
+  return view.is_number() && view.number() == 0;
 }
 
 static void clear_headers(curl_http_t *handle) {
@@ -270,21 +271,45 @@ static void clear_headers(curl_http_t *handle) {
   }
 }
 
+static int has_embedded_nul(const svalue_t *value) {
+  auto view = lpc::const_svalue_view::from(value);
+
+  if (!view.is_string()) {
+    return 0;
+  }
+
+  const char *data = view.c_str();
+  size_t len = view.length();
+  if (len == 0) {
+    return 0;
+  }
+
+  return std::memchr(data, '\0', len) != nullptr;
+}
+
 static void set_headers_from_array(curl_http_t *handle, array_t *headers) {
   struct curl_slist *new_headers = nullptr;
   int index;
 
   for (index = 0; index < headers->size; index++) {
     svalue_t *item = &headers->item[index];
+    auto item_view = lpc::const_svalue_view::from(item);
 
-    if (item->type != T_STRING) {
+    if (!item_view.is_string()) {
       if (new_headers) {
         curl_slist_free_all(new_headers);
       }
       error("perform_using(headers, value) requires a string or string array.\n");
     }
 
-    new_headers = curl_slist_append(new_headers, SVALUE_STRPTR(item));
+    if (has_embedded_nul(item)) {
+      if (new_headers) {
+        curl_slist_free_all(new_headers);
+      }
+      error("perform_using(headers, value) string entries must not contain embedded NUL bytes.\n");
+    }
+
+    new_headers = curl_slist_append(new_headers, item_view.c_str());
     if (!new_headers) {
       error("Out of memory while configuring CURL headers.\n");
     }
@@ -295,6 +320,8 @@ static void set_headers_from_array(curl_http_t *handle, array_t *headers) {
 }
 
 static void configure_option(curl_http_t *handle, const char *option, svalue_t *value) {
+  auto value_view = lpc::const_svalue_view::from(value);
+
   if (!std::strcmp(option, "url")) {
     if (is_clear_value(value)) {
       if (handle->url) {
@@ -304,14 +331,18 @@ static void configure_option(curl_http_t *handle, const char *option, svalue_t *
       return;
     }
 
-    if (value->type != T_STRING) {
+    if (!value_view.is_string()) {
       error("perform_using(url, value) requires a string.\n");
     }
 
+    if (has_embedded_nul(value)) {
+      error("perform_using(url, value) string must not contain embedded NUL bytes.\n");
+    }
+
     if (handle->url) {
       FREE(handle->url);
     }
-    handle->url = dup_bytes(SVALUE_STRPTR(value), SVALUE_STRLEN(value));
+    handle->url = dup_bytes(value_view.c_str(), value_view.length());
     if (!handle->url) {
       error("Out of memory while configuring CURL url.\n");
     }
@@ -324,15 +355,15 @@ static void configure_option(curl_http_t *handle, const char *option, svalue_t *
       return;
     }
 
-    if (value->type == T_STRING) {
+    if (value_view.is_string()) {
       array_t *single = allocate_empty_array(1);
       assign_svalue_no_free(&single->item[0], value);
       set_headers_from_array(handle, single);
       free_array(single);
       return;
     }
 
-    if (value->type != T_ARRAY) {
+    if (!value_view.is_array()) {
       error("perform_using(headers, value) requires a string or string array.\n");
     }
 
@@ -353,9 +384,12 @@ static void configure_option(curl_http_t *handle, const char *option, svalue_t *
       return;
     }
 
-    if (value->type == T_STRING) {
-      size = SVALUE_STRLEN(value);
-      handle->post_data = dup_bytes(SVALUE_STRPTR(value), size);
+    if (value_view.is_string()) {
+      if (has_embedded_nul(value)) {
+        error("perform_using(post_data/body, value) string must not contain embedded NUL bytes; use a buffer for binary payloads.\n");
+      }
+      size = value_view.length();
+      handle->post_data = dup_bytes(value_view.c_str(), size);
       handle->post_size = static_cast<int>(size);
     }
     else if (value->type == T_BUFFER) {
@@ -374,20 +408,21 @@ static void configure_option(curl_http_t *handle, const char *option, svalue_t *
   }
 
   if (!std::strcmp(option, "timeout_ms")) {
-    if (value->type != T_NUMBER) {
+    if (!value_view.is_number()) {
       error("perform_using(timeout_ms, value) requires a number.\n");
     }
 
-    handle->timeout_ms = value->u.number > 0 ? static_cast<long>(value->u.number) : 0L;
+    handle->timeout_ms = value_view.number() > 0 ?
+      std::min(LONG_MAX, value_view.number()) : 0L;
     return;
   }
 
   if (!std::strcmp(option, "follow_location")) {
-    if (value->type != T_NUMBER) {
+    if (!value_view.is_number()) {
       error("perform_using(follow_location, value) requires a number.\n");
     }
 
-    handle->follow_location = value->u.number != 0;
+    handle->follow_location = value_view.number() != 0;
     return;
   }
 
@@ -684,6 +719,7 @@ static size_t curl_response_write_callback(void *ptr, size_t size, size_t nmemb,
 extern "C" void f_perform_using(void) {
   svalue_t *option_value = sp - 1;
   svalue_t *setting_value = sp;
+  auto option_view = lpc::const_svalue_view::from(option_value);
   const char *option;
   int handle_id;
   curl_http_t *handle;
@@ -692,7 +728,7 @@ extern "C" void f_perform_using(void) {
     error("CURL subsystem is not available.\n");
   }
 
-  if (option_value->type != T_STRING) {
+  if (!option_view.is_string()) {
     error("perform_using() option must be a string.\n");
   }
 
@@ -710,7 +746,7 @@ extern "C" void f_perform_using(void) {
     error("Failed to create CURL easy handle.\n");
   }
 
-  option = SVALUE_STRPTR(option_value);
+  option = option_view.c_str();
   configure_option(handle, option, setting_value);
   handle->state = CURL_STATE_CONFIGURED;
   pop_n_elems(2);
@@ -734,7 +770,8 @@ extern "C" void f_perform_to(void) {
 
   callback_value = sp - argc + 1;
   flag_value = callback_value + 1;
-  if (flag_value->type != T_NUMBER) {
+  auto flags_view = lpc::const_svalue_view::from(flag_value);
+  if (!flags_view.is_number()) {
     error("perform_to() flags argument must be a number.\n");
   }
 

lib/lpc/array.c

@@ -14,6 +14,7 @@
 #include "otable.h"
 #include "program.h"
 #include "lpc/include/origin.h"
+#include "svalue.h"
 
 #ifdef ARRAY_STATS
 int num_arrays;
@@ -664,7 +665,7 @@ sameval (svalue_t * arg1, svalue_t * arg2)
     case T_STRING:
       if (string_length_differs (arg1, arg2))
         return 0;
-      return !strcmp (SVALUE_STRPTR(arg1), SVALUE_STRPTR(arg2));
+      return svalue_string_lexcmp (arg1, arg2) == 0;
     case T_OBJECT:
       return arg1->u.ob == arg2->u.ob;
     case T_MAPPING:
@@ -1121,7 +1122,7 @@ static int builtin_sort_array_cmp_fwd (svalue_t * p1, svalue_t * p2) {
     {
     case T_STRING:
       {
-        return strcmp (SVALUE_STRPTR(p1), SVALUE_STRPTR(p2));
+        return svalue_string_lexcmp (p1, p2);
       }
 
     case T_NUMBER:
@@ -1145,7 +1146,7 @@ static int builtin_sort_array_cmp_fwd (svalue_t * p1, svalue_t * p2) {
           {
           case T_STRING:
             {
-              return strcmp (SVALUE_STRPTR(v1->item), SVALUE_STRPTR(v2->item));
+              return svalue_string_lexcmp (v1->item, v2->item);
             }
 
           case T_NUMBER:
@@ -1178,7 +1179,7 @@ static int builtin_sort_array_cmp_rev (svalue_t * p1, svalue_t * p2) {
     {
     case T_STRING:
       {
-        return strcmp (SVALUE_STRPTR(p2), SVALUE_STRPTR(p1));
+        return svalue_string_lexcmp (p2, p1);
       }
 
     case T_NUMBER:
@@ -1202,7 +1203,7 @@ static int builtin_sort_array_cmp_rev (svalue_t * p1, svalue_t * p2) {
           {
           case T_STRING:
             {
-              return strcmp (SVALUE_STRPTR(v2->item), SVALUE_STRPTR(v1->item));
+              return svalue_string_lexcmp (v2->item, v1->item);
             }
 
           case T_NUMBER: