Skip to content

Load heatmap data asynchronously#36622

Merged
silverwind merged 16 commits into
go-gitea:mainfrom
silverwind:hm
Feb 17, 2026
Merged

Load heatmap data asynchronously#36622
silverwind merged 16 commits into
go-gitea:mainfrom
silverwind:hm

Conversation

@silverwind
Copy link
Copy Markdown
Member

@silverwind silverwind commented Feb 14, 2026

Fixes: #21045

  • Move heatmap data loading from synchronous server-side rendering to async client-side fetch via dedicated JSON endpoints
  • Dashboard and user profile pages no longer block on the expensive heatmap DB query during HTML generation
  • Use compact [[timestamp,count]] JSON format instead of [{"timestamp":N,"contributions":N}] to reduce payload size
  • Public API (/api/v1/users/{username}/heatmap) remains unchanged
  • Heatmap rendering is unchanged, still shows a spinner as before, which will now spin a litte bit longer.

Benchmark (1M action rows, SQLite)

Scenario Time
Old sync page render 536ms
New async page render 66ms
Heatmap endpoint 464ms (loaded after page is visible)

🤖 Generated with Claude Code

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Feb 14, 2026
@silverwind silverwind added the performance/speed performance issues with slow downs label Feb 14, 2026
@silverwind silverwind changed the title Load heatmap data asynchronously to improve page render Load heatmap data asynchronously Feb 14, 2026
@silverwind silverwind requested a review from Copilot February 14, 2026 07:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request refactors heatmap data loading from synchronous server-side rendering to asynchronous client-side fetching. The change significantly improves page load performance by moving an expensive database query off the critical rendering path. The heatmap data is now fetched via dedicated JSON endpoints after the page becomes visible to the user.

Changes:

  • Introduced three new JSON endpoints for async heatmap data loading: /user/heatmap (personal dashboard), /{org}/dashboard/heatmap (org dashboard), and /{username}/-/heatmap (user profile)
  • Converted heatmap data format from verbose JSON objects [{"timestamp":N,"contributions":N},...] to compact tuple arrays [[timestamp,count],...] to reduce payload size
  • Modified frontend to fetch and render heatmap asynchronously with proper error handling

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
web_src/js/features/heatmap.ts Made initHeatmap async, added data fetching via GET request, and converts compact array format to heatmap display format
templates/user/heatmap.tmpl Replaced inline heatmap data with data-heatmap-url attribute and updated locale string to use placeholder
routers/web/web.go Registered three new JSON endpoints for personal, org, and profile heatmap data
routers/web/user/profile.go Added Heatmap handler for user profile, returns compact JSON format with permission checks
routers/web/user/home.go Modified Dashboard to set heatmap URL instead of data, added DashboardHeatmap handler for async data fetching
models/activities/user_heatmap.go Added heatmapDataJSON helper and two new JSON-returning functions for compact data format

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread routers/web/user/home.go Outdated
Comment thread routers/web/user/profile.go Outdated
Comment thread web_src/js/features/heatmap.ts
Comment thread models/activities/user_heatmap.go Outdated
silverwind and others added 7 commits February 14, 2026 18:39
Move heatmap data loading from synchronous server-side rendering to
async client-side fetch. The dashboard and user profile pages no longer
block on the expensive heatmap DB query during HTML generation. Instead,
the heatmap is loaded via dedicated JSON endpoints after the page is
visible.

Additionally, use a compact [[timestamp,count],...] JSON format instead
of [{"timestamp":N,"contributions":N},...] to reduce payload size.

Benchmark with 1M action rows (SQLite):
- Old sync page render: 535ms
- New async page render:  66ms (8x faster)
- Heatmap endpoint:      464ms (loaded after page is visible)

Fixes: go-gitea#21045

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The heatmap data URLs were built without the /org route prefix,
causing 404 responses and JSON parse errors in the browser.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use OrganisationLink()/HomeLink() helpers which apply url.PathEscape
  for org, team, and user names in heatmap URL construction
- Check resp.ok before parsing JSON response in heatmap.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: silverwind <me@silverwind.io>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move the [][2]int64 conversion inline into the route handlers instead
of adding dedicated model functions for it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
silverwind and others added 2 commits February 14, 2026 18:46
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@silverwind
Copy link
Copy Markdown
Member Author

Diff is minimal now, everything is working.

@silverwind silverwind marked this pull request as ready for review February 14, 2026 17:51
Comment thread routers/web/web.go Outdated
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Feb 16, 2026
Comment thread routers/web/user/home.go Outdated
Comment thread routers/web/user/home.go Outdated
silverwind and others added 2 commits February 17, 2026 11:42
…g dashboard

- Use HomeLink() instead of Req.URL.Path for stable URL construction
- Non-org dashboard reuses /{username}/-/heatmap profile endpoint
- Remove now-unused /user/heatmap route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…HomeLink()

OrganisationLink() returns /org/{name} which matches the route prefix,
while HomeLink() returns /{name} which caused a 404.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@silverwind
Copy link
Copy Markdown
Member Author

silverwind commented Feb 17, 2026

Comments addressed and one regression from that fixed (org dashboard). Tests for the JSON endpoints incoming.

Tests the profile, org dashboard, and org team dashboard heatmap
JSON endpoints for correct status and response structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread tests/integration/heatmap_test.go
silverwind and others added 2 commits February 17, 2026 11:54
Signed-off-by: silverwind <me@silverwind.io>
@wxiaoguang
Copy link
Copy Markdown
Contributor

wxiaoguang commented Feb 17, 2026

The refactoring does:

  1. Remove dead code like GetTotalContributionsInHeatmap
  2. Split prepareHeatmapURL code into its function, to avoid bloating func Dashboard
  3. Don't call getDashboardContextUser in the heatmap endpoint. getDashboardContextUser does too much.
  4. To avoid misuse of getDashboardContextUser, rename it to prepareDashboardContextUserOrgTeams
  5. Remove unnecessary ActivityReadable check in user.Heatmap, GetUserHeatmapDataByUser already does so
  6. Share same logic in DashboardHeatmap, remove duplicate user.Heatmap
  7. Make heatmap entpoints use same route pattern like /issues and /issues/{team}
  8. Fix test name t.Run("UserDashboard" to t.Run("OrgDashboard"
  9. Rename GetUserHeatmapDataByUserTeam to GetUserHeatmapDataByOrgTeam and make it explicitly accept "org" argument

@wxiaoguang
Copy link
Copy Markdown
Contributor

The refactoring does:

When AI reaches this level, I can delegate my work to it.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Feb 17, 2026
@silverwind
Copy link
Copy Markdown
Member Author

Thanks for cleaning up

@silverwind
Copy link
Copy Markdown
Member Author

I tested all heatmaps, all working.

@silverwind silverwind enabled auto-merge (squash) February 17, 2026 13:50
@silverwind silverwind added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Feb 17, 2026
@silverwind silverwind merged commit d6be18e into go-gitea:main Feb 17, 2026
24 checks passed
@GiteaBot GiteaBot added this to the 1.26.0 milestone Feb 17, 2026
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Feb 17, 2026
@wxiaoguang wxiaoguang deleted the hm branch February 17, 2026 14:06
@go-gitea go-gitea locked as resolved and limited conversation to collaborators May 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. performance/speed performance issues with slow downs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Heatmap data needs to be optimized

5 participants