Don't convert buffer to string

[lunny]

should fix #6236

[codecov-io]

Codecov Report

Merging #7115 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #7115      +/-   ##
==========================================
- Coverage   41.46%   41.46%   -0.01%     
==========================================
  Files         478      478              
  Lines       63966    63966              
==========================================
- Hits        26526    26525       -1     
- Misses      33984    33986       +2     
+ Partials     3456     3455       -1

Impacted Files	Coverage Δ
models/issue_comment.go	47.56% <0%> (ø)	⬆️
modules/log/event.go	64.61% <0%> (-1.03%)	⬇️
models/repo_list.go	72.08% <0%> (-1.02%)	⬇️
modules/process/manager.go	81.15% <0%> (+4.34%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09ca391...c2fb205. Read the comment docs.

[zeripath]

I'm not certain that this is enough.

We're still going to be casting these bytes as a string to store them in the database which will cause the same problem. I think Patch needs to be a []byte not string.

however, how do you display a []byte - well most often this is by casting to string or attempting to decode through detection. The trouble is detecting the encoding from such a small sample is going to be very difficult - you really need the rest of the file as context, and it's perfectly reasonable for a file to change encoding.

As per usual file encoding is difficult and complex.

The most complete answer probably involves:

• Storing the bytes only.
• Sending the bytes to the client - and or best guess re-encoding.
• Telling the client the detected encodings of the two sides of the diff
• Allowing the client to override this by requesting the actual bytes and then they re-encode.
• Displaying broken characters in some obvious way - without blowing up.

[lunny]

We could detect the encoding of this file but not the patch and convert it to UTF8 before store it in database.

[lafriks]

Converting patch to utf-8 would be best option and would solve issue on storing into database

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 months. Thank you for your contributions.