-
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Heatmap: Actions are shown on the previous day #6087
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I noticed this today as well. |
NZST timezone (UTC +12:00/13:00) also does this, except only before midday. |
This probably has to do with the fact the server runs in a different timezone (most likely utc) than your browser. |
This was the first thing I checked in my case, my server and browser are absolutely running in the same timezone. |
I think its because the time is striped from the timestamp. Line 25 in c42bde7
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions. |
This issue is still present on 1.14.0+dev-394-g069acf6a2. It is caused by a disparity in the timezone of the DB (MySQL in my case), and the local browser. My company mainly works out of the EDT timezone, but I am in PST (three hours behind). When I push code, I always see it appear on the prior day, however, if I adjust my local system time to be EDT, the blocks align correctly. As noted, the issue is rooted in the DB query rounding The best solution would likely be to read the timezone from the user browser and use that when globbing the |
I think there are two problems: incorrect heatmap and incorrect commission time. The commission time (the text popup when hovering on the "19 hours ago") can be changed by putting a However, the heatmap uses UTC+0 time zone and there is currently no way to change that. I live in AEDT (UTC+10) and every commission before 10:00 will be marked as yesterday's work. |
The heatmap uses UTC+0 time zone and cannot be changed by any settings (so far). The only way you can make a commission today is to push your work after 19:00 (since you live in UTC-5). |
I think you have that backwards. If I push at 5 PM (17:00) then that should be shown as 1:00 AM on the next day if the heat map is UTC based. It should never show up on the previous day if the user has a negative offset to UTC. The heat map is just wrong. Edit: [{"timestamp":1614211200,"contributions":2}] 1614211200 represents Thursday, February 25, 2021 12:00:00 AM UTC So the timestamp being generated for grouping is causing the issue. gitea/web_src/js/features/heatmap.js Line 11 in 4172b19
use this instead: return {date: new Date((timestamp + ((new Date()).getTimezoneOffset() * 60)) * 1000), count: contributions}; I haven't actually tested out this change, but based on the JSON data and using my javascript console this should work. |
@kins-dev That will not work;
The opposite happens for The server is not aware of the client's timezone, so when it strips out the time, you will get shifted results on the client side. A better solution would be to group to the hour in SQL, then have the JS convert it to local date (strip the time) and summarize it once more. This will send one timestamp per hour instead of every commit's timestamp. |
Really? Did you try it? Did you understand the issue? (The calendar.vue is running in the context of local time, for showing days but the server is providing the date via epoch which evaluates to the previous day when your timezone is behind UTC. You can think of it as taking the time specified and doing dropping the time information but keeping the date information. Using epoch time implies that time is in the UTC zone, so that information is not being stripped when it goes to your browser.) So, I'm PST, but this will work for EST or any other timezone. [{"timestamp":1614211200,"contributions":2}] That's not February 25th where my browser or server is running. It is February 25th UTC time, in fact the very first moment of it. February 25th started at 1614240000 for the PST time zone. So, doing the math 1614240000 - 1614211200 = 28800 seconds. (new Date()).getTimezoneOffset() * 60 + 1614211200 https://jsfiddle.net/3v578s1r/1/ Does that match 1614240000? I think it does, and in fact after putting my code in place, I get this now: This is not perfect, far from it, but it is at least defensible. I'm using math to make the calendar.vue work based on UTC time so it matches the server in terms of what constitutes a single day. Why do you believe that won't work? Edit: Before my change, those contributions all showed up the previous day in the heatmap Edit 2: some wording. Edit 3: Just realized it is just after midnight UTC, so I made a few more commits: The heatmap data is now: [{"timestamp":1614211200,"contributions":12},{"timestamp":1614297600,"contributions":3}] 12 + 3 = 15 contributions and it is obvious the 3 are connected to the next day. heatmap.vue and calendar.vue will not show the next day until it hits the 26th local time, so for now that data is not visible. Like I said not perfect, but better than putting items on the wrong day. |
So I believe the heat map end date can be fixed by:
to endDate: new Date(Date.now() + (new Date()).getTimezoneOffset() * 60000), And this should keep the heatmap in UTC. I won't be able to test it for another day. Of course this all could have been avoided if Line 19 in c42bde7
had kept the date string as a string instead of converting it seconds then milliseconds. (As per the documentation here: https://github.com/julienr114/vue-calendar-heatmap) We'd still need to play games with the end date. That all said there's a better solution. if the function in user_heatmap.go was simplified to: (Remember to remove the unused reference) func getUserHeatmapData(user *User, team *Team, doer *User) ([]*UserHeatmapData, error) {
hdata := make([]*UserHeatmapData, 0)
if !activityReadable(user, doer) {
return hdata, nil
}
cond, err := activityQueryCondition(GetFeedsOptions{
RequestedUser: user,
RequestedTeam: team,
Actor: doer,
IncludePrivate: true, // don't filter by private, as we already filter by repo access
IncludeDeleted: true,
// * Heatmaps for individual users only include actions that the user themself did.
// * For organizations actions by all users that were made in owned
// repositories are counted.
OnlyPerformedBy: !user.IsOrganization(),
})
if err != nil {
return nil, err
}
// Only worry about actions in the last 53 weeks to give a little margin for error
return hdata, x.
Select("created_unix AS timestamp, count(user_id) as contributions").
Table("action").
Where(cond).
And("created_unix > ?", (timeutil.TimeStampNow() - (53*7*60*60))).
GroupBy("created_unix").
OrderBy("timestamp").
Find(&hdata)
} So that everything is just grouped by their unix timestamp. Then in heatmap.js if you did something like: export default async function initHeatmap() {
const el = document.getElementById('user-heatmap');
if (!el) return;
try {
const json_data = JSON.parse(el.dataset.heatmapData).map(({contributions, timestamp}) => {
return {date: new Date(timestamp * 1000).toDateString(), count: contributions};
});
let tmp={};
json_data.forEach(e => {
let val = tmp[e.date]||0;
val += e.count;
tmp[e.date]=val;
});
let heatmap_data = [];
for(let dateVal in tmp) {
let countVal = tmp[dateVal];
heatmap_data.push({date: new Date(dateVal), count: countVal});
}
const values = heatmap_data.map(a => ({...a}));
const View = Vue.extend({
render: (createElement) => createElement(ActivityHeatmap, {props: {values}}),
});
new View().$mount(el);
} catch (err) {
console.error(err);
el.textContent = 'Heatmap failed to load';
}
} Then the heatmap would just work in the web browser's time zone and this entire issue would just go away. Also it reduces the computational load on the server, by making clients sum the number of actions per day, The draw back is, it does increase the amount of data being sent between the client and the server. This may be a problem with a lot of commits, so the person fixing this might want to group by minute, or by a 10 minute interval, but I'll leave that up to them. (The original reason for grouping was not in a comment in the code, so I don't know if there's something I'm missing.) |
I'm in agreement. (This is what I proposed in my previous comment...) There are 30min and 45min timezones (India is UTC +5:30). So grouping by date and 15 min intervals will give you the most accurate times with the least amount of data sent. |
I'm tired of making no contributions every Friday and work overtime every Sunday. Hope this issue can be fixed soon. BTW, Australian Central Daylight Time (ADST) is a +10.5 time. 15 minutes' interval is a great suggestion. |
I started working on a fix for the SQL and used the javascript fix from @kins-dev. SQLite seems to be working so far. |
@siddweiker if you want to go that route you should be able to do some simple math in the SQL Something like: group by created_unix / 900 Probably need that in the select as well. Since leap seconds are ignored by epoch time, this should just work for everyone. Edit: |
@siddweiker Lines 28 to 72 in 83cf1a8
func getUserHeatmapData(user *User, team *Team, doer *User) ([]*UserHeatmapData, error) {
// 53 weeks in seconds, because the heatmap may show up to 53 columns
const WeeksInSec = 53 * 7 * 24 * 60 * 60
heatMapData := make([]*UserHeatmapData, 0)
if !activityReadable(user, doer) {
return heatMapData, nil
}
cond, err := activityQueryCondition(GetFeedsOptions{
RequestedUser: user,
RequestedTeam: team,
Actor: doer,
IncludePrivate: true, // don't filter by private, as we already filter by repo access
IncludeDeleted: true,
// * Heatmaps for individual users only include actions that the user themself did.
// * For organizations actions by all users that were made in owned
// repositories are counted.
OnlyPerformedBy: !user.IsOrganization(),
})
if err != nil {
return nil, err
}
// 900 in the following statements comes from 15 minutes in seconds
// 15 minute groupings was decided because that's the smallest offset
// from an hour that is possible (eg. Nepal)
// https://www.worldtimeserver.com/learn/unusual-time-zones/
return heatMapData, x.
Select("created_unix / 900 AS timestamp, count(user_id) as contributions").
Table("action").
Where(cond).
And("created_unix > ?", (timeutil.TimeStampNow() - (WeeksInSec))).
GroupBy("created_unix / 900").
OrderBy("timestamp").
Find(&heatMapData)
} gitea/web_src/js/features/heatmap.js Lines 5 to 23 in 83cf1a8
export default async function initHeatmap() {
const el = document.getElementById('user-heatmap');
if (!el) return;
try {
const json_data = JSON.parse(el.dataset.heatmapData).map(({ contributions, timestamp }) => {
// groups by 15 minute intervals, (15*60) and we need that in milliseconds, so 900000
// must be kept in sync with user_heatmap.go
return { date: new Date(timestamp * 900000).toDateString(), count: contributions };
});
let tmp = {};
json_data.forEach(e => {
let val = tmp[e.date] || 0;
val += e.count;
tmp[e.date] = val;
});
let heatmap_data = [];
for (let dateVal in tmp) {
let countVal = tmp[dateVal];
heatmap_data.push({ date: new Date(dateVal), count: countVal });
}
const values = heatmap_data.map(a => ({ ...a }));
const View = Vue.extend({
render: (createElement) => createElement(ActivityHeatmap, { props: { values } }),
});
new View().$mount(el);
} catch (err) {
console.error(err);
el.textContent = 'Heatmap failed to load';
}
} Note: In my previous code, I missed the 24 in the calculation of 53 weeks for the I verified the results by looking at the contents of [ {"timestamp":1614267824,"contributions":1},
{"timestamp":1614269550,"contributions":1},
{"timestamp":1614290624,"contributions":1},
{"timestamp":1614290662,"contributions":1},
{"timestamp":1614290794,"contributions":1},
{"timestamp":1614290980,"contributions":1},
{"timestamp":1614291138,"contributions":1},
{"timestamp":1614291523,"contributions":1},
{"timestamp":1614299151,"contributions":1},
{"timestamp":1614299237,"contributions":2},
{"timestamp":1614348696,"contributions":1},
{"timestamp":1614351446,"contributions":1},
{"timestamp":1614355082,"contributions":1},
{"timestamp":1614355346,"contributions":2},
{"timestamp":1614355882,"contributions":1},
{"timestamp":1614356603,"contributions":1},
{"timestamp":1614407454,"contributions":1},
{"timestamp":1614407984,"contributions":1},
{"timestamp":1614408176,"contributions":2},
{"timestamp":1614408536,"contributions":1},
{"timestamp":1614408867,"contributions":1},
{"timestamp":1614448314,"contributions":1},
{"timestamp":1614448315,"contributions":2},
{"timestamp":1614448715,"contributions":1},
{"timestamp":1614453629,"contributions":1},
{"timestamp":1614456736,"contributions":1},
{"timestamp":1614463043,"contributions":1},
{"timestamp":1614464473,"contributions":1},
{"timestamp":1614530671,"contributions":1},
{"timestamp":1614530672,"contributions":1},
{"timestamp":1614532187,"contributions":1},
{"timestamp":1614540754,"contributions":1},
{"timestamp":1614541439,"contributions":1},
{"timestamp":1614541988,"contributions":1},
{"timestamp":1614542189,"contributions":1}] 15 Minute Grouping: [ {"timestamp":1793630,"contributions":1},
{"timestamp":1793632,"contributions":1},
{"timestamp":1793656,"contributions":5},
{"timestamp":1793657,"contributions":1},
{"timestamp":1793665,"contributions":3},
{"timestamp":1793720,"contributions":1},
{"timestamp":1793723,"contributions":1},
{"timestamp":1793727,"contributions":1},
{"timestamp":1793728,"contributions":3},
{"timestamp":1793729,"contributions":1},
{"timestamp":1793786,"contributions":4},
{"timestamp":1793787,"contributions":2},
{"timestamp":1793831,"contributions":4},
{"timestamp":1793837,"contributions":1},
{"timestamp":1793840,"contributions":1},
{"timestamp":1793847,"contributions":1},
{"timestamp":1793849,"contributions":1},
{"timestamp":1793922,"contributions":2},
{"timestamp":1793924,"contributions":1},
{"timestamp":1793934,"contributions":2},
{"timestamp":1793935,"contributions":2}] You can put that data into a spreadsheet if you want to confirm, there are multiple commits in the same 15 minute window, and commits that fall into sequential 15 minute windows. I haven't tested anything but SQLite3 |
Due to segfaults on the master branch when using issue management on the Arm64 platform, I'm going back to 1.13.2. Back porting these changes: gitea/web_src/js/components/ActivityHeatmap.vue Lines 50 to 60 in 0143131
becomes: loadHeatmap(userName) {
const self = this;
$.get(`${this.suburl}/api/v1/users/${userName}/heatmap`, (chartRawData) => {
const chartDataMap = {};
const chartData = [];
for (let i = 0; i < chartRawData.length; i++) {
let dateStr = (new Date(chartRawData[i].timestamp * 1000)).toDateString();
let dayContrib = chartDataMap[dateStr] || 0;
dayContrib += chartRawData[i].contributions;
self.totalContributions += chartRawData[i].contributions;
chartDataMap[dateStr] = dayContrib;
}
for (let dateStr in chartDataMap)
{
chartData.push({date: new Date(dateStr), count: chartDataMap[dateStr]})
}
self.values = chartData;
self.isLoading = false;
}); Lines 18 to 58 in 0143131
becomes: // GetUserHeatmapDataByUser returns an array of UserHeatmapData
func GetUserHeatmapDataByUser(user *User) ([]*UserHeatmapData, error) {
// 53 weeks in seconds, because the heatmap may show up to 53 columns
const WeeksInSec = 53 * 7 * 24 * 60 * 60
hdata := make([]*UserHeatmapData, 0)
if user.KeepActivityPrivate {
return hdata, nil
}
sess := x.Select("(created_unix / 900) * 900 AS timestamp, count(user_id) as contributions").
Table("action").
Where("user_id = ?", user.ID).
And("created_unix > ?", (timeutil.TimeStampNow() - (WeeksInSec)))
// * Heatmaps for individual users only include actions that the user themself
// did.
// * For organizations actions by all users that were made in owned
// repositories are counted.
if user.Type == UserTypeIndividual {
sess = sess.And("act_user_id = ?", user.ID)
}
err := sess.GroupBy("created_unix / 900").
OrderBy("timestamp").
Find(&hdata)
return hdata, err
} There's a couple of bonuses here, one is the API should work the same again (no change in semantics) and the javascript code was cleaned up some. |
@kins-dev @Colin5887 |
Description
Heatmap actions are shown on the previous day for actions in an EST (and beyond?) timezone.
Steps to reproduce:
Screenshots
Here is a screenshot from my own gitea instance, the heatmap should show actions from Monday to Friday, instead they are shifted back one day.

From try.gitea.io, here is a combined screenshot to show the commit time and the heatmap action time:

The text was updated successfully, but these errors were encountered: