Skip to content

Use existing analyzer module for language detection for highlighting#13522

Merged
techknowlogick merged 7 commits into
go-gitea:masterfrom
mrsdizzie:mime-type-highlight
Nov 13, 2020
Merged

Use existing analyzer module for language detection for highlighting#13522
techknowlogick merged 7 commits into
go-gitea:masterfrom
mrsdizzie:mime-type-highlight

Conversation

@mrsdizzie
Copy link
Copy Markdown
Member

@mrsdizzie mrsdizzie commented Nov 11, 2020 •
edited
Loading

Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.

silverwind
silverwind approved these changes Nov 11, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@silverwind silverwind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess it will help a little even thought most of the detectable file types are binary.

@GiteaBot GiteaBot added the lgtm/need 1 This PR needs approval from one additional maintainer to be merged. label Nov 11, 2020
@zeripath zeripath added the topic/ui Change the appearance of the Gitea UI label Nov 12, 2020
@zeripath zeripath added this to the 1.14.0 milestone Nov 12, 2020
@zeripath
Copy link
Copy Markdown
Contributor

zeripath commented Nov 12, 2020

this doesn't seem particularly helpful as most of the mimetypes appear to be binary in the attached library.

@mrsdizzie
Copy link
Copy Markdown
Member Author

mrsdizzie commented Nov 12, 2020

It's for many script languages that are common to not have extensions php/perl/ruby/node/lua etc...

Most people don't use extensions when writing programs in scripting languages. None of these are highlighted now

lafriks
lafriks previously requested changes Nov 12, 2020
View reviewed changes
Comment thread modules/highlight/highlight.go Outdated
@silverwind
Copy link
Copy Markdown
Member

silverwind commented Nov 12, 2020 •
edited
Loading

Here's what that library gives us. I see it's rather incomplete thought. For example I can't find any shells in it, bash, zsh etc which are prime canidates for extensionless files. Still I guess it's a tiny improvement over the previous state.

I guess shell support could be hacked in by checking the first line of the file against a regexp like ^#!.*\b(ba|c|da|k|fi|tc|z)?sh($|\s).

@mrsdizzie
Use existing analyzer module for language detction for highlighting
9a1cef6 
Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.
@mrsdizzie mrsdizzie changed the title Detect mime-type for file highlighting Use existing analyzer module for language detction for highlighting Nov 12, 2020
@mrsdizzie mrsdizzie changed the title Use existing analyzer module for language detction for highlighting Use existing analyzer module for language detection for highlighting Nov 12, 2020
@mrsdizzie
Copy link
Copy Markdown
Member Author

mrsdizzie commented Nov 12, 2020

Changed per @lafriks pointing out we already have an existing module to detect language

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Nov 13, 2020
silverwind
silverwind reviewed Nov 13, 2020
View reviewed changes
Comment thread go.sum
@codecov-io
Copy link
Copy Markdown

codecov-io commented Nov 13, 2020

Codecov Report

Merging #13522 (078dacb) into master (0ae35c6) will increase coverage by 0.01%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #13522      +/-   ##
==========================================
+ Coverage   42.15%   42.17%   +0.01%     
==========================================
  Files         695      695              
  Lines       76395    76394       -1     
==========================================
+ Hits        32207    32219      +12     
+ Misses      38903    38889      -14     
- Partials     5285     5286       +1
Impacted Files Coverage Δ
modules/highlight/highlight.go 25.00% <0.00%> (+0.30%) ⬆️
models/gpg_key.go 53.33% <0.00%> (-0.58%) ⬇️
services/pull/pull.go 40.78% <0.00%> (-0.50%) ⬇️
routers/repo/view.go 38.11% <0.00%> (+0.64%) ⬆️
modules/log/file.go 75.20% <0.00%> (+1.60%) ⬆️
services/pull/patch.go 55.68% <0.00%> (+1.70%) ⬆️
models/unit.go 49.31% <0.00%> (+2.73%) ⬆️
services/pull/check.go 51.82% <0.00%> (+2.91%) ⬆️
services/pull/temp_repo.go 29.78% <0.00%> (+3.19%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ae35c6...078dacb. Read the comment docs.

lafriks
lafriks reviewed Nov 13, 2020
View reviewed changes
Comment thread modules/highlight/highlight.go Outdated
@techknowlogick
Copy link
Copy Markdown
Member

techknowlogick commented Nov 13, 2020

🚀

@techknowlogick techknowlogick merged commit 6280af0 into go-gitea:master Nov 13, 2020
@techknowlogick
Copy link
Copy Markdown
Member

techknowlogick commented Nov 13, 2020

@mrsdizzie please send backport :)

mrsdizzie added a commit to mrsdizzie/gitea that referenced this pull request Nov 13, 2020
@mrsdizzie @zeripath
@lafriks @techknowlogick
Use existing analyzer module for language detection for highlighting (g…
5003eca 
…o-gitea#13522)

* Use existing analyzer module for language detction for highlighting

Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.

* Update modules/highlight/highlight.go

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@mrsdizzie mrsdizzie mentioned this pull request Nov 13, 2020
Merged
techknowlogick added a commit that referenced this pull request Nov 13, 2020
@mrsdizzie @zeripath
@lafriks @techknowlogick
Use existing analyzer module for language detection for highlighting (#…
b4d18da 
…13522) (#13551)

* Use existing analyzer module for language detction for highlighting

Thanks @lafriks for pointing out we can reuse existing code for more reliable language detection here.

* Update modules/highlight/highlight.go

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
@lafriks lafriks added the backport/done All backports for this PR have been created label Nov 13, 2020
@go-gitea go-gitea locked and limited conversation to collaborators Dec 14, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@silverwind silverwind silverwind approved these changes

@lafriks lafriks lafriks left review comments

@lunny lunny lunny approved these changes

Assignees

No one assigned

Labels

backport/done All backports for this PR have been created lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. topic/ui Change the appearance of the Gitea UI

Projects

None yet

Milestone

1.14.0

Development

Successfully merging this pull request may close these issues.

8 participants

@mrsdizzie @zeripath @silverwind @codecov-io @techknowlogick @lunny @lafriks @GiteaBot