Repository navigation
Add RPM packaging - #59
Closed
jameshickman wants to merge 5 commits into
Closed
jameshickman wants to merge 5 commits into
jameshickman wants to merge 5 commits into
Conversation
Profiling the inner loop with perf shows it is bound purely on instruction count, not on stalls: a 65M dispatch benchmark runs at IPC 3.0 with 13k branch misses out of 2.0e9 branches and 25k L1 misses out of 2.5e9 loads. The jump table the compiler generates for the do_prim() switch does not show up in the profile at all. The time goes into work done per instruction, so this removes two pieces of it. Decode single byte opcodes inline in run(). Most dictionary cells encode as one byte, but every instruction fetch paid for an out of line call into dict_get_cell_typed(), which was 22% of the profile on its own. Only walk the return stack for the trace indent when tracing is actually on. The loop over RSP(ctx) ran once per instruction regardless, and each iteration reloaded RSP through ctx->uservar, which aliases ctx->dict and so cannot be kept in a register. Together these cut the benchmark from 7.44e9 to 5.22e9 instructions, 114 to 80 per dispatch, and the run time from 0.605s to 0.385s at -O2. Output is unchanged on core.zf, mandel.zf, dict.zf, memaccess.zf and misc.zf, and the -t trace output is byte identical. Build the interpreter at -O2 rather than -Os, worth a further 2.2x, and make the AddressSanitizer build opt-in with 'make asan=1' instead of linking it into every build by default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H676rBN7QwnkCHZgBdNKk8
Adds a 'make libs' target building two shared library variants from the same source and the same zfconf.h. libzforth has the tracing code compiled in, libzforth-notrace does not. Compiling tracing in costs about 45% of run time even when tracing is switched off at run time, because every trace site still tests the flag, and reading it goes through ctx->uservar, which aliases ctx->dict and so cannot be kept in a register. On the 65M dispatch benchmark the traced library runs in 0.36s and the untraced one in 0.21s. The untraced library is also 12 kB smaller and does not reference zf_host_trace() at all, so a program embedding it does not have to define that callback. The two carry different sonames and can be installed side by side. They are interchangeable at the API and ABI level, since ZF_ENABLE_TRACE does not appear in struct zf_ctx. Both are built with -fno-semantic-interposition. zf_push() and friends are public API, so under -fPIC the compiler otherwise assumes the dynamic linker could interpose them and emits 70 PLT calls into the inner loop rather than inlining. That alone was costing the library a factor of two against the same code linked statically: 0.45s versus 0.22s. ZF_ENABLE_TRACE in src/linux/zfconf.h is now only defined if not already set, so the two variants can share one configuration header rather than duplicating a file whose dictionary and stack sizes are baked into struct zf_ctx. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H676rBN7QwnkCHZgBdNKk8
Packages zForth for RPM-based systems as four packages built from one spec: zforth (interpreter, Forth standard library, man page), libzforth (the interpreter as a shared library), libzforth-devel and libzforth-static. Upstream has no tagged releases, so the spec builds a git snapshot pinned by a commit macro rather than a release tarball. The spec drives the compiler directly instead of using the upstream Makefile, which appends -fsanitize=address, -Os -g and -Werror to CFLAGS and LDFLAGS unconditionally. Only the distribution build flags apply. libzforth deliberately leaves zf_host_sys(), zf_host_trace() and zf_host_parse_num() undefined for the embedding program to supply, so they only bind at run time if the executable exports its dynamic symbols. zforth.pc therefore carries -Wl,--export-dynamic; without it a linked program builds cleanly and fails at startup. zfconf.h is treated as part of the ABI, since the dictionary and stack sizes it defines are compiled into struct zf_ctx. The libmajor macro is documented as needing a bump alongside any change to that header. %check runs the interpreter against the packaged core.zf, verifies the shared library exports the API in zforth.h while leaving only the three zf_host_* callbacks undefined, and builds the shipped example through pkg-config the way a consumer of the devel package would. The man page and embedding example are written for this package; upstream ships neither. Readline is on by default and can be disabled with rpmbuild --without readline. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EPozsdjamuCqTHpMym75YF
Adds libzforth-notrace and libzforth-notrace-devel, built from the same source and the same zfconf.h as libzforth with ZF_ENABLE_TRACE switched off. Compiling tracing in costs about 45% of interpreter run time even when it is switched off at run time through the 'trace' user variable, because every trace site still has to test the flag. The untraced library is a third smaller and does not reference zf_host_trace(), so a program embedding it does not have to define that callback. The two carry different sonames and can be installed side by side; they are interchangeable at the API and ABI level, since ZF_ENABLE_TRACE does not appear in struct zf_ctx. The untraced build derives its zfconf.h with sed from the one the traced build uses rather than carrying a second copy in the spec, so the two cannot drift apart in the dictionary and stack sizes that are baked into struct zf_ctx. A grep after the sed fails the build if it did not match, so an upstream rename of ZF_ENABLE_TRACE cannot silently produce two identical libraries. Both shared libraries are now built with -fno-semantic-interposition. zf_push() and the other stack calls are public API, so under -fPIC the compiler otherwise assumes the dynamic linker could interpose them and emits PLT calls into the inner interpreter instead of inlining, which was costing the shared library a factor of two against the same code linked statically. %check now runs the symbol and pkg-config checks over both libraries, and asserts that libzforth-notrace really does leave zf_host_trace() out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H676rBN7QwnkCHZgBdNKk8
Moves the pinned commit to 848458c, which carries the inner interpreter work and the shared library build, and resets Release for the new snapshot version. The packaged libraries now show the speedup. Rebuilding both snapshots with the same distribution flags and running the 65M dispatch benchmark against each: libzforth 0.57s -> 0.38s 133 -> 92 instructions per dispatch libzforth-notrace 0.32s -> 0.22s 76 -> 48 instructions per dispatch Both are a little slower than a plain -O2 build of the same source because of the distribution hardening flags, which is the expected trade. The untraced variant is now built with -DZF_ENABLE_TRACE=0 rather than a sed over zfconf.h. The pinned tree only defines ZF_ENABLE_TRACE if it is not already set, so the two variants share the one configuration header directly and the scaffolding that made that safe against the old unconditional define is no longer needed. The zf_host_trace() assertion in %check still covers the case where that guard goes away again. Also drops the note about the upstream Makefile linking AddressSanitizer into every build, which the pinned tree fixed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H676rBN7QwnkCHZgBdNKk8
jameshickman
force-pushed
the
packaging/rpm
branch
from
August 28, 2026 17:21
128e958 to
1627208
Compare
Author
|
Withdrawing this for now. The branch has since been rebased onto some inner interpreter performance work, which I have opened separately as #60, and the spec here now pins a commit that only exists on top of that. So this PR no longer does what its title says, and it is not reviewable on its own in its current state. I would rather not have packaging land ahead of the code it packages anyway. If #60 is something you want, I will reopen this rebased onto whatever it becomes, pinned to a commit that actually exists here. If you would rather not carry distribution packaging in the tree at all, that is a completely reasonable answer too and I will keep it in my fork. Sorry for the churn. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds RPM packaging under
packaging/rpm/. No existing files are touched — thisis additive only, so it should have no effect on anyone not building RPMs.
What it builds
Four packages from one spec:
zforth/usr/bin/zforth, theforth/*.zflibrary in/usr/share/zforth, man pagelibzforthlibzforth.so.0— the interpreter as a shared librarylibzforth-develzforth.h,zfconf.h,zforth.pc, embedding examplelibzforth-staticlibzforth.aSince zForth is aimed at embedding, the library packages seemed at least as
useful as the CLI, so
src/zforth/zforth.cis built as a proper shared objectwith a soname alongside the interpreter.
Two things worth your attention
The spec doesn't use
src/linux/Makefile. It appends-fsanitize=address,-Os -gand-WerrortoCFLAGS/LDFLAGSunconditionally, so a package builtthrough it would ship an ASan-instrumented binary and would break whenever a new
compiler warns about something new. The spec invokes the compiler directly
instead. Happy to send a separate PR making the Makefile respect externally
supplied flags if you'd prefer packaging go through it — that would be the
tidier long-term arrangement.
Versioning is snapshot-based. There are no tags, so
Versionis0^20250815git41db72d, pinned by acommitmacro at the top of the spec. Anytagged release would let this become an ordinary versioned package.
Packaging notes
libzforth.soleaveszf_host_sys(),zf_host_trace()andzf_host_parse_num()undefined for the embedding program to supply, so they onlybind at run time if the executable exports its dynamic symbols.
zforth.pctherefore carries
-Wl,--export-dynamic— without it a linked program buildscleanly and then fails at startup, which is an unpleasant thing to debug.
zfconf.his treated as part of the ABI, sinceZF_DICT_SIZEand the stacksizes are compiled into
struct zf_ctx. The spec documents bumping the sonamemajor alongside any change to that header.
%checkruns the interpreter against the packagedcore.zf, verifies thelibrary exports the API in
zforth.hwhile leaving only the threezf_host_*callbacks undefined, and builds the shipped example through
pkg-configthe waya consumer of the devel package would.
The man page and the embedding example are written for this package, as the
repo ships neither. Readline is on by default and can be turned off with
rpmbuild --without readline.Testing
Built with
rpmbuild -baon Fedora 43 (x86_64, gcc 15.3.1), both with andwithout readline. Installed the resulting packages and confirmed the interpreter
runs against the installed
core.zf, and that the shipped example compiles andruns against both
libzforth.soandlibzforth.avia the installedzforth.pc.🤖 Generated with Claude Code
https://claude.ai/code/session_01EPozsdjamuCqTHpMym75YF