WIP: Add subdir support for wiki pages

[snoweuph]

Fixes #27858

Description

This PR is about Adding Support for Subdirs in Gitea Wikis like stated in #27858

I Tested that with this PR:

1. Creating Pages in Subdirs Work
2. Visiting Pages in Subdirs Works
3. Deleting Pages in Subdirs Works
4. Editing Pages in Subdirs Works
5. Markdown Links in Wikis Pointing at Subdirs Work
6. Pages Overview Site Works
7. The Dropdown with the Pages List Works

[snoweuph]

I didnt Modify: services/wiki/wiki_path.go
because I didnt know what its really supposed to do

[snoweuph]

Okay, I now understand what the wiki_path.go is for, its for escaping, now I know what to do next

[snoweuph]

Okay, I think I know Understand the Old Escapment:

[snoweuph]

Okay, I tested everything that I can locally run so far, I hope that I pass Migration this Time

[KN4CK3R]

You can run the tests with make test-sqlite. From a first look at the changed tests, they will fail. 😏

[snoweuph]

You can run the tests with make test-sqlite. From a first look at the changed tests, they will fail. 😏

Dont I need to setup an extra test DB for that?

[KN4CK3R]

No, that's the magic word sqlite in the command. It uses a simple Sqlite database.

[snoweuph]

No, that's the magic word sqlite in the command. It uses a simple Sqlite database.

Ahh okay, thanks, executing SQlite Test locally works, gonna use that one to debunk what ive did

[KN4CK3R]

Okay, I think I know Understand the Old Escapment:

Out of curiosity, did you create the graph by hand?

[snoweuph]

Okay, I think I know Understand the Old Escapment:

Out of curiosity, did you create the graph by hand?

I used draw.io, but yes, I put it together myself

[shamefulCake1]

Are there any chances this PR will be merged any time soon?

[snoweuph]

Are there any chances this PR will be merged any time soon?

Nope, Prolly not

[wxiaoguang]

Hi snoweuph. Thank you for your work! The "wiki" feature in Gitea has some longstanding problems, when we would like to improve/fix them, maybe the compatibility/migration for legacy wikis should also be considered together.

For this PR, I think it has finishes the new features. Meanwhile I think it needs to clarify "how to deal with existing legacy wikis".

And I can see there are still some related CI failures, I guess this PR wouldn't be merged into 1.22. To resolve the wiki path problems together, I think it needs a "V2" path standard in the future:

1. Correctly handle slashes and sub directories. Use enough tests to make sure the "sub directories" work well.
2. Only escape necessary characters in the git repo, leave the non-ASCII UTF-8 characters as-is to avoid bloating the filename (a lot of %xx)
3. Migrate the existing data to the new path standard.

[snoweuph]

I'm aware that there is still some stuff, I would need to Fix, but i'm pretty busy currently in Reallife, so I wont be able to work on this for prolly atleast 2 more months.

[snoweuph]

im back (atleast a littlebit)
gonna start doing some work on this again

[snoweuph]

I just did a reset, to be up to date with the latest main, and im redoing the whole thing, with better doc this time. I will need some time though

[snoweuph]

I just started creating a mermaid flowchart to represent the escaping and unespaing to then plan a better thing.
RN I only did the unescaping from the visiting of the wiki file and im allready at over 80 lines of meraid Flowchart code

flowchart TD

  subgraph builtin
    url.QueryEscape[["url.QueryEscape"]]
    url.QueryUnescape[["url.QueryUnescape()"]]
  end

  subgraph wiki_path.go
    WebPathFromRequest[["WebPathFromRequest <a>/</a> => <a>%2F</a>"]]

    subgraph hasDashMarker
        hasDashMarker_input(("input")) --> hasDashMarker_check{{"ends with <a>.-</a> ?"}}
        hasDashMarker_check -->|"YES"| hasDashMarker_return_1>"return <a>TRUE</a>"]
        hasDashMarker_check -->|"NO"| hasDashMarker_return_2>"return <a>FALSE</a>"]
    end

    subgraph trimDashMarker
        trimDashMarker_input(("input")) --> trimDashMarker_return_1>"trim <a>.-</a>"]
    end

    subgraph addDashMarker
        addDashMarker_input(("input")) --> addDashMarker_return_1>"add suffix <a>.-</a>"]
    end

    subgraph unescapeSegment
      unescapeSegment_input(("input")) --> unescapeSegment_call_1(["call"]) -.-> hasDashMarker
      unescapeSegment_call_1 --> unescapeSegment_check_1{{"Was <a>TRUE</a> ?"}}
      unescapeSegment_check_1 -->|"YES"| unescapeSegment_call_2(["call"]) -.-> trimDashMarker
      unescapeSegment_check_1 -->|"NO"| unescapeSegment_action_1["replace all '-' with ' '"]
      unescapeSegment_call_2 & unescapeSegment_action_1 --> unescapeSegment_call_3(["call"]) -.-> url.QueryUnescape

      unescapeSegment_call_3 --> unescapeSegment_check_2{{"<a>error<a> ?"}}
      unescapeSegment_check_2 -->|"YES"| unescapeSegment_return_1>"return <a>error<a>"]
      unescapeSegment_check_2 -->|"NO"| unescapeSegment_return_1["return"]
    end

    subgraph escapeSegToWeb
      escapeSegToWeb_input(("input(<a>had_dashmarker</a>)")) --> escapeSegToWeb_check_1{{"had_dashmarker?"}}
      escapeSegToWeb_check_1  -->|"YES"| escapeSegToWeb_call_1(["call"]) -.-> addDashMarker
      escapeSegToWeb_check_1  -->|"NO"| escapeSegToWeb_check_2{{"contains '-' ?"}}

      escapeSegToWeb_check_2  -->|"YES"| escapeSegToWeb_call_2(["call"]) -.-> addDashMarker
      escapeSegToWeb_check_2  -->|"NO"| escapeSegToWeb_check_3{{"ends with <a>.md</a> ?"}}

      escapeSegToWeb_check_3  -->|"YES"| escapeSegToWeb_call_3(["call"]) -.-> addDashMarker
      escapeSegToWeb_check_3  -->|"NO"| escapeSegToWeb_action_1{{"replace all ' ' with '-'"}}

      escapeSegToWeb_call_1 & escapeSegToWeb_call_2 & escapeSegToWeb_call_3 & escapeSegToWeb_action_1 --> escapeSegToWeb_call_4(["call"]) -.-> url.QueryEscape
      escapeSegToWeb_call_4 --> escapeSegToWeb_return>"return"]
    end

    subgraph GitPathToWebPath
      GitPathToWebPath_input(("input"))
      GitPathToWebPath_check_1{{"ends with <a>.md</a> ?"}}
      GitPathToWebPath_input --> GitPathToWebPath_check_1

      GitPathToWebPath_action_1["trim <a>.md</a>"]
      GitPathToWebPath_return_1>"return <a>ErrWikiInvalidFileName</a>"]
      GitPathToWebPath_check_1 -->|"YES"|GitPathToWebPath_action_1
      GitPathToWebPath_check_1 -->|"NO"|GitPathToWebPath_return_1


      GitPathToWebPath_action_1 --> GitPathToWebPath_action_2["split at <a>/</a>"]
      GitPathToWebPath_action_2 --> GitPathToWebPath_loopstart_1[/"loop over each <a>entry</a>"\]
      GitPathToWebPath_loopstart_1 --> GitPathToWebPath_call_1(["call, write to <a>shouldAddDashMarker</a>"]) -.-> hasDashMarker
      GitPathToWebPath_call_1 --> GitPathToWebPath_call_2([call]) -.-> unescapeSegment
      GitPathToWebPath_call_2 --> GitPathToWebPath_check_2{{"<a>error</a> ?"}}
      GitPathToWebPath_check_2 -->|"YES"| GitPathToWebPath_return_2>"return <a>error</a>"]
      GitPathToWebPath_check_2 --> |"NO"| GitPathToWebPath_call_3(["call with <a>shouldAddDashMarker</a>"]) -.-> escapeSegToWeb
      GitPathToWebPath_call_3 --> GitPathToWebPath_loopend_1[\"loop end"/]
      GitPathToWebPath_loopend_1 --> GitPathToWebPath_action_3["join with <a>/</a>"]
      GitPathToWebPath_action_3 --> GitPathToWebPath_return_3>"return"]

    end
  end

  subgraph wiki.go
    view(("Wiki View")) --> renderViewPage

    subgraph renderViewPage
      renderViewPage_input(("input"))
      renderViewPage_call_1(["call per entry"]) -.-> GitPathToWebPath
      renderViewPage_call_2(["call"]) -.-> WebPathFromRequest
      renderViewPage_input --> renderViewPage_call_1 --> renderViewPage_call_2
    end
  end

Loading

After I understand the whole Process, I will defenitly have to think about a solution to make the whole thing easier to understand and maintain lmao