Fix character disappearing and false matching issues in TiddlyWiki 5.4 Freelink plugin by s793016 · Pull Request #9397 · TiddlyWiki/TiddlyWiki5

s793016 · 2025-11-05T15:27:37Z

This PR fixes two critical bugs in the freelinks plugin when running on TiddlyWiki 5.4:

Bug 1: First character disappearing

Symptom:
When freelinking is enabled, the first character of matched words disappears (e.g., "The" becomes "he", "Filter" becomes "ilter", "5.4.0" becomes ".4.0", ).

Root cause:
When the current tiddler's title was being filtered out, it was done too late in the process (during parse tree construction), causing text rendering issues.

Fix:

Move the current tiddler title filtering to the match validation stage (in processTextWithMatches)
Use substring instead of slice for better stability
Add proper case-insensitive comparison for title matching

Bug 2: False positive matches

Symptom:
Words like "it is", "Choose", "Set up" are incorrectly linked to tiddler "FooBar" when a tiddler titled "xxx x FooBar" exists.

Root cause:
The Aho-Corasick algorithm's output merging mechanism in buildFailureLinks caused failure link outputs to be incorrectly merged into intermediate nodes, resulting in false matches.

Fix:

Remove incorrect output merging in buildFailureLinks
Implement proper output collection during search by traversing the failure link chain
Add exact match validation: verify that the matched text exactly equals the pattern before accepting it
Add cycle detection to prevent infinite loops in failure link traversal

Test case
Import attached file to reproduce both bugs:
test2.json

With Word Boundary Check enabled and Ignore Case disabled, both bugs should be fixed.

False positive matches Symptom: Words like "it is", "Choose", "Set up" are incorrectly linked to tiddler "FooBar" when a tiddler titled "xxx x FooBar" exists. Root cause: The Aho-Corasick algorithm's output merging mechanism in buildFailureLinks caused failure link outputs to be incorrectly merged into intermediate nodes, resulting in false matches. Fix: Remove incorrect output merging in buildFailureLinks Implement proper output collection during search by traversing the failure link chain Add exact match validation: verify that the matched text exactly equals the pattern before accepting it Add cycle detection to prevent infinite loops in failure link traversal

First character disappearing Symptom: When freelinking is enabled, the first character of matched words disappears (e.g., "The" becomes "he", "Filter" becomes "ilter"). Root cause: When the current tiddler's title was being filtered out, it was done too late in the process (during parse tree construction), causing text rendering issues. Fix: Move the current tiddler title filtering to the match validation stage (in processTextWithMatches) Use substring instead of slice for better stability Add proper case-insensitive comparison for title matching

add back description

netlify · 2025-11-05T15:27:42Z

✅ Deploy Preview for tiddlywiki-previews ready!

Name	Link
🔨 Latest commit	`2522705`
🔍 Latest deploy log	https://app.netlify.com/projects/tiddlywiki-previews/deploys/6913cc6b7e3e3800082cabb0
😎 Deploy Preview	https://deploy-preview-9397--tiddlywiki-previews.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

github-actions · 2025-11-05T15:27:46Z

Confirmed: s793016 has already signed the Contributor License Agreement (see contributing.md)

add freelinks plugin for testing

restore

add freelinks plugin for test

linonetwo · 2025-11-05T16:20:03Z

-					child.$.push.apply(child.$, failureOutput.$);
-				}
-
+				// Critical fix: remove incorrect output merging


AI like adding these useless comment. we don't need them in the code.

This is legacy feature of AI that used to work as Edit mode. Modern Agent mode AI can directly say these info in the chat, but they still like to add these things into the code.

linonetwo · 2025-11-05T16:21:04Z

I'm not sure how to test this, can you try read existing test edition and try add two tests for this feature and bug issue?

twMat · 2025-11-05T18:59:05Z

Thanks for looking into this!

Symptom:
When freelinking is enabled, the first character of matched words disappears (e.g., "The" becomes "he", "Filter" becomes "ilter", ...

Hm, that is now quite how it appeared for me. My bug demo here shows the topmost tiddler like this:

...when the edit view is actually:

...i.e there it is actually the whole first word of those sentences, not just the first letter, that disappear.

Maybe you PR fixes this but I'm saying it, just to make sure this was not missed.

pmario · 2025-11-05T19:36:34Z

@twMat Did you test with this preview: https://deploy-preview-9397--tiddlywiki-previews.netlify.app/

It contains the freelinks plugin now. So you can directly test from there.

pmario · 2025-11-05T19:42:16Z

I only did a very short test and it looks better now.

erase comment

pmario · 2025-11-05T22:32:18Z

 				var failureLink = (fail && fail[char]) ? fail[char] : root;
 				this.failure[child] = failureLink;
-
-				// Critical fix: remove incorrect output merging


IMO Critical fix should have been removed, but the rest seems to be important to understand the code. so this may stay

pmario · 2025-11-05T22:32:31Z

 		var char = text[i];
 		var transitionCount = 0;

-		// Follow failure links to find a valid transition


iMO should stay

pmario · 2025-11-05T22:32:45Z

 		var visitedNodes = new Set();

 		while(currentNode && collectCount < 10) {
-			// Prevent infinite loops


should stay

pmario · 2025-11-05T22:33:03Z

 			}
 			visitedNodes.add(currentNode);

-			// Only collect outputs from the current node (not merged ones)


should stay

add back some commets

pmario · 2025-11-05T23:19:38Z

 				var failureLink = (fail && fail[char]) ? fail[char] : root;
 				this.failure[child] = failureLink;
+
+				// remove incorrect output merging


As I wrote. The comment // remove incorrect output merging can be removed. The rest can probably stay.

pmario · 2025-11-05T23:20:11Z

 			}
 		}

+		// correctly collect all matches


// correctly collect all matches

clean comment

twMat · 2025-11-06T07:03:06Z

@pmario

@twMat Did you test with this preview:

The point with my post here was merely that the symptom description in the topmost post here, in spite of being very detailed, didn't quite cover the problems I had reported. Regardless, the PR here does seem to resolve everything. All good.

s793016 · 2025-11-06T16:35:07Z

I've prepared another test package here: test by ai3.json

This collection of tiddlers is designed to test the reliability, accuracy, and performance of the Freelinks plugin for TiddlyWiki.

It provides structured test cases to ensure that automatic link generation behaves correctly in various real-world scenarios.

!! 📘 Purpose
The test suite evaluates how Freelinks handles:

Overlapping and nested titles – verifying longest-match priority.
Case sensitivity – ensuring consistency when tv-freelinks-ignore-case is enabled or disabled.
Word boundaries – testing English and multilingual text accuracy.
Algorithm stability and speed – stress-testing the Aho–Corasick matcher under heavy load.

!! 🧩 How to Use

Import this JSON file into your TiddlyWiki.
Open the tiddler “Freelinks Debug Test Index”.
Follow the listed test cases one by one:
- Each page includes bilingual (Chinese–English) explanations.
- Compare the “Expected Result” section to the live rendered text.

!! ⚙️ Notes

Tests cover both English and Chinese content for mixed-language verification.
The Performance Stress Test includes a long 1000× alphabet string to benchmark Aho–Corasick’s efficiency.
No manual setup is required — all needed titles are already included in the dataset.

!! ✅ Expected Outcome
If the plugin is functioning properly:

Only valid titles will auto-link.
Overlaps will be skipped.
No false or missing links should appear.
The stress test should run within a few seconds without errors.

change note TiddlyWiki#9397

pmario · 2025-11-10T09:56:24Z

 		"tiddlywiki/confetti",
 		"tiddlywiki/dynannotate",
 		"tiddlywiki/internals",
+		"tiddlywiki/freelinks",


This plugin will need to be removed once you are sure, that everything works as expected.

reversed to original

update detail

Jermolene · 2025-11-11T16:42:22Z

Thank you @s793016.

I think it's not clear in the documentation that change notes can have multiple GitHub links as shown in this example. In the case of a bugfix like this, you can just add the bugfix PR to the GitHub links for the original PR.

another link added

github-actions · 2025-11-11T23:33:34Z

✅ Change Note Status

All change notes are properly formatted and validated!

📝 $:/changenotes/5.4.0/#9397

Type: bugfix | Category: plugin
Release: 5.4.0

Fix critical freelinks bugs: first character loss and false positive matches in v5.4.0

🔗 #9084 #9397

👥 Contributors: s793016

📖 Change Note Guidelines

Change notes help track and communicate changes effectively. See the full documentation for details.

add "release: 5.4.0"

pmario · 2025-11-11T23:37:01Z

 change-type: bugfix
 change-category: plugin
-github-links: [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397)
+github-links: [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084), [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397)


Suggested change

github-links: [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084), [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397)

github-links: https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084 https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397

Markdown links do not work with TiddlyWiki.

some format modified

pmario · 2025-12-15T20:38:06Z

@Jermolene ... The new freelinks functions have been merged. So IMO this one needs to be merged too.

Jermolene · 2025-12-17T15:02:38Z

Thanks @s793016

update the existing release note with the new PR link instead of creating a new release note.

add link

…t-match, and Unicode safety (#9676) * Update aho-corasick.js fix transition logic; ensure complete outputs (via failure-output merge); clean up stats/build scoping; clarify CJK boundary behavior. * Update text.js implement global longest-match priority with overlap suppression; fix refresh invalidation to ignore $:/state and drafts; handle deletions precisely to avoid rebuilding on draft deletion; add defensive check for cached automaton presence. * Update text.js remove comment * Update aho-corasick.js remove comment * Create #9672.tid * Create #2026-0222.tid * Delete editions/tw5.com/tiddlers/releasenotes/5.4.0/#2026-0222.tid * Update text.js remove \" * Update and rename #9672.tid to #9676.tid change to right number * Update #9397.tid update the existing release note with the new PR link instead of creating a new release note. * Delete editions/tw5.com/tiddlers/releasenotes/5.4.0/#9676.tid update the existing release note with the new PR link instead of creating a new release note. * Rename #9397.tid to #9676.tid update the existing release note with the new PR link instead of creating a new release note. * Update and rename #9676.tid to #9397.tid add link * Rename #9397.tid to #9676.tid * Update tiddlywiki.info add plugin for test build * Update tiddlywiki.info reverse change, ready to be merge.

s793016 added 4 commits November 5, 2025 23:04

Update text.js

abbe53e

add back description

Update aho-corasick.js

b7d45c5

add back description

s793016 mentioned this pull request Nov 5, 2025

[BUG] Freelinks 5.4.0 strange behaviour #9392

Open

s793016 added 3 commits November 5, 2025 23:33

Update tiddlywiki.info

376fe0c

add freelinks plugin for testing

Update tiddlywiki.info

72fb3b5

restore

Update tiddlywiki.info

b532bca

add freelinks plugin for test

linonetwo reviewed Nov 5, 2025

View reviewed changes

s793016 added 2 commits November 6, 2025 06:28

Update aho-corasick.js

0bd13ad

erase comment

Update text.js

f9a0748

erase comment

pmario reviewed Nov 5, 2025

View reviewed changes

Update aho-corasick.js

9c85163

add back some commets

pmario reviewed Nov 5, 2025

View reviewed changes

Update aho-corasick.js

ec122bd

clean comment

s793016 added 2 commits November 10, 2025 13:44

Merge branch 'TiddlyWiki:master' into freelink-bug-fix

e95d313

change note TiddlyWiki#9397

f9a441c

change note TiddlyWiki#9397

pmario reviewed Nov 10, 2025

View reviewed changes

s793016 added 2 commits November 10, 2025 20:42

Update tiddlywiki.info

9f4a83c

reversed to original

Update TiddlyWiki#9397.tid

d10be01

update detail

Update TiddlyWiki#9397.tid

3cdf11b

another link added

Update TiddlyWiki#9397.tid

a329838

add "release: 5.4.0"

pmario reviewed Nov 11, 2025

View reviewed changes

Comment thread editions/tw5.com/tiddlers/releasenotes/5.4.0/#9397.tid Outdated

Update TiddlyWiki#9397.tid

2522705

some format modified

Jermolene merged commit 7c197f6 into TiddlyWiki:master Dec 17, 2025
9 checks passed

s793016 deleted the freelink-bug-fix branch February 21, 2026 10:44

This was referenced Feb 21, 2026

Improve Aho-Corasick search correctness and longest-match priority in Freelinks #9672

Closed

Fix Freelinks Aho-Corasick: failure links, cache invalidation, longest-match, and Unicode safety #9676

Merged

s793016 added a commit to s793016/TiddlyWiki5 that referenced this pull request Feb 22, 2026

Update TiddlyWiki#9397.tid

82b733a

update the existing release note with the new PR link instead of creating a new release note.

s793016 added a commit to s793016/TiddlyWiki5 that referenced this pull request Feb 22, 2026

Rename TiddlyWiki#9397.tid to TiddlyWiki#9676.tid

124ee2c

update the existing release note with the new PR link instead of creating a new release note.

s793016 added a commit to s793016/TiddlyWiki5 that referenced this pull request Feb 22, 2026

Update and rename TiddlyWiki#9676.tid to TiddlyWiki#9397.tid

15597c5

add link

s793016 added a commit to s793016/TiddlyWiki5 that referenced this pull request Feb 22, 2026

Rename TiddlyWiki#9397.tid to TiddlyWiki#9676.tid

0dacc9d

	github-links: [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084), [https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397](https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397)
	github-links: https://github.com/TiddlyWiki/TiddlyWiki5/pull/9084 https://github.com/TiddlyWiki/TiddlyWiki5/pull/9397

Uh oh!

Conversation

s793016 commented Nov 5, 2025

Uh oh!

netlify Bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for tiddlywiki-previews ready!

Uh oh!

github-actions Bot commented Nov 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

linonetwo commented Nov 5, 2025

Uh oh!

twMat commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pmario commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pmario commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

twMat commented Nov 6, 2025

Uh oh!

s793016 commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jermolene commented Nov 11, 2025

Uh oh!

github-actions Bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Change Note Status

📝 $:/changenotes/5.4.0/#9397

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pmario commented Dec 15, 2025

Uh oh!

Jermolene commented Dec 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

netlify Bot commented Nov 5, 2025 •

edited

Loading

twMat commented Nov 5, 2025 •

edited

Loading

pmario commented Nov 5, 2025 •

edited

Loading

pmario commented Nov 5, 2025 •

edited

Loading

s793016 commented Nov 6, 2025 •

edited

Loading

github-actions Bot commented Nov 11, 2025 •

edited

Loading