UI: Detect and restore encoding and BOM in content #6727
Conversation
Signed-off-by: Andrew Thornton <[email protected]>
|
|
GiteaBot
added
the
lgtm/need 2
Codecov Report
@@ Coverage Diff @@
## master #6727 +/- ##
=========================================
- Coverage 41.03% 41% -0.03%
=========================================
Files 421 421
Lines 57967 58050 +83
=========================================
+ Hits 23784 23804 +20
- Misses 31024 31078 +54
- Partials 3159 3168 +9
Continue to review full report at Codecov.
|
Signed-off-by: Andrew Thornton <[email protected]>
zeripath
changed the title
|
Tested this, can confirm it's working well. BOM is not rendered anymore and preserved if present when editing the file in the web editor. |
GiteaBot
added
lgtm/need 1
GiteaBot
added
lgtm/done
|
Should I provide a backport? |
|
Yes please do so |
modules/templates/helper.go
Outdated
| @@ -267,6 +267,10 @@ func ToUTF8WithErr(content []byte) (string, error) { | |||
| if err != nil { | |||
| return "", err | |||
| } else if charsetLabel == "UTF-8" { | |||
| if len(content) > 2 && bytes.Equal(content[0:3], base.UTF8BOM) { | |||
There was a problem hiding this comment.
How about to create a function named RemoveUTF8BOM(content string) string.
|
@zeripath see my comment. |
|
I also noticed that I wasn't dealing with updating LFSed content so that's done too. |
|
(well it's dealt with in the same way we deal with it on the front end.) |
|
|
|
OK fixed! |
Detect and remove a decoded BOM when showing content. Restore the previous encoding and BOM when updating content. On error keep as UTF-8 encoding. Signed-off-by: Andrew Thornton <[email protected]>
zeripath
changed the title
Detect and remove a decoded BOM when showing content. Restore the previous encoding and BOM when updating content. On error keep as UTF-8 encoding. Signed-off-by: Andrew Thornton <[email protected]>
Fixes #6716
When decoding content, if the first 3 bytes match the UTF-8 BOM remove it.
When updating content through the editor, check the previous content for the encoding and BOM and reencode to that. If we can't encode to that then default to utf8.
Signed-off-by: Andrew Thornton [email protected]