Reduce memory usage when rendering markdown

[lunny]

This PR removed at least two whole markdown copy when rendering a document.

[delvh]

Took me (surprisingly) way too long to understand where we reduce the memory usage until I noticed it...

[lunny]

Took me (surprisingly) way too long to understand where we reduce the memory usage until I noticed it...

Not only that, I also removed rawHTML, err := io.ReadAll(input) which also copied the whole content. That means, previously there are at least 2 copy of the content in the memory and now they are gone.

[delvh]

rawHTML, err := io.ReadAll(input) - yeah, that was the thing that took me so long.
But what is the other reduction I missed?
The replaceAll(buffer) vs replaceAll(input)?

[lunny]

rawHTML, err := io.ReadAll(input) - yeah, that was the thing that took me so long. But what is the other reduction I missed?

It's different memory usage between convert the whole content and convert part of them, when using a io.Reader to do that, there is only about 1K or 2K when reading the whole content.

The replaceAll(buffer) vs replaceAll(input)?

Yes

[wxiaoguang]

I do not think it is correct.

The content is incomplete in the original buffer, the tagCleaner may not work correctly. And tagCleaner is quite a complex regexp, I am not sure whether it works for incomplete content.

[wxiaoguang]

And one more thing, after the replacing, the returned string may be longer than before, it may overflow the bs buffer.

[Gusted]

The reduced memory is only correct for larger inputs.

1024 bytes (this PR use more memory):

-> % benchstat old.bench new.bench 
name            old time/op    new time/op    delta
PostProcess-12     199µs ± 2%     206µs ± 3%   +3.56%  (p=0.008 n=5+5)

name            old alloc/op   new alloc/op   delta
PostProcess-12    18.7kB ± 0%    22.1kB ± 0%  +17.72%  (p=0.008 n=5+5)

name            old allocs/op  new allocs/op  delta
PostProcess-12      27.0 ± 0%      29.0 ± 0%   +7.41%  (p=0.008 n=5+5)

1024 * 8 bytes (this PR use less memory):

-> % benchstat old.bench new.bench
name            old time/op    new time/op    delta
PostProcess-12    1.76ms ± 4%    1.72ms ± 3%     ~     (p=0.310 n=5+5)

name            old alloc/op   new alloc/op   delta
PostProcess-12     119kB ± 0%      92kB ± 0%  -22.57%  (p=0.008 n=5+5)

name            old allocs/op  new allocs/op  delta
PostProcess-12      35.0 ± 0%      36.0 ± 0%   +2.86%  (p=0.008 n=5+5)

And for reference 1024 * 128 bytes:

-> % benchstat old.bench new.bench
name            old time/op    new time/op    delta
PostProcess-12    27.1ms ± 6%    26.5ms ± 2%     ~     (p=0.421 n=5+5)

name            old alloc/op   new alloc/op   delta
PostProcess-12    2.01MB ± 0%    1.45MB ± 0%  -27.94%  (p=0.016 n=4+5)

name            old allocs/op  new allocs/op  delta
PostProcess-12      52.0 ± 2%      62.2 ± 2%  +19.62%  (p=0.008 n=5+5)

Bench code:

diff --git a/bench/bench_test.go b/bench/bench_test.go
new file mode 100644
index 000000000..33d9279cc
--- /dev/null
+++ b/bench/bench_test.go
@@ -0,0 +1,17 @@
+package bench
+
+import (
+       "io"
+       "strings"
+       "testing"
+
+       "code.gitea.io/gitea/modules/markup"
+)
+
+func BenchmarkPostProcess(b *testing.B) {
+       input := strings.Repeat("a", 1024)
+       b.ReportAllocs()
+       for i := 0; i < b.N; i++ {
+               markup.PostProcess(&markup.RenderContext{}, strings.NewReader(input), io.Discard)
+       }
+}

[6543]

☝️ would be nice to also add this test into this pull

[lunny]

closed as it's not always correct.