Skip to content

fromInfluxDBTimeFormat does not parse milliseconds correct #339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Prebzs opened this issue Jun 16, 2017 · 2 comments
Open

fromInfluxDBTimeFormat does not parse milliseconds correct #339

Prebzs opened this issue Jun 16, 2017 · 2 comments

Comments

@Prebzs
Copy link

Prebzs commented Jun 16, 2017

If a timestamp has a 10-millisecond time (last digit 0), the returned milliseconds time is not correct.

.time(1497600000030L, TimeUnit.MILLISECONDS)

after reading from database:
fromInfluxDBTimeFormat(raw.get(0).toString()) returns "1497600000003"

the database contains "2017-06-16T08:00:00.03Z"

@fmachado
Copy link
Contributor

fmachado commented Jun 26, 2017

TLDR: When you execute a query using the InfluxDB Java client, always specify the TimeUnit parameter and never parse the RFC3339 date format returned by InfluxDB.

This seems to be related with how InfluxDB (in fact, Go lang) uses the RFC3339 for date formatting and how java.text.SimpleDateFormat (used by this library) differs from the first.

<short-version>

  • InfluxDB format "1497600000030L" as "2017-06-16T08:00:00.03Z" (.030 ms was trimmed to .03 ms)
  • SimpleDateFormat using the pattern "yyyy-MM-dd'T'HH:mm:ss.SS'Z'" format "1497600000030L" as "2017-06-16T10:00:00.30Z" (.030 ms was "interpreted" as .30 ms)
  • If you try to parse the "2017-06-16T08:00:00.03Z" using SimpleDateFormat, you will have the wrong value with .03ms.

</short-version>

You can test it by yourself:

root@3e65d276839c:/# influx                   
Connected to http://localhost:8086 version 1.3.0
InfluxDB shell version: 1.3.0
>
> create database mydatabase
>
> use mydatabase
Using database mydatabase
>
> insert bar value=1 1497600000030000000
>
> select * from bar
name: bar
time                value
----                -----
1497600000030000000 1

The time 1497600000030000000 is in nanoseconds and it is the same one provided in the previous comment.

Now let's see how InfluxDB is formatting a date using RFC3339. Accordingly with the InfluxDB documentation:

The -precision argument specifies the format/precision of any returned timestamps. In the example above, rfc3339 tells InfluxDB to return timestamps in RFC3339 format (YYYY-MM-DDTHH:MM:SS.nnnnnnnnnZ).
Source: https://docs.influxdata.com/influxdb/v1.2/introduction/getting_started/#creating-a-database

root@3e65d276839c:/# influx -precision rfc3339
Connected to http://localhost:8086 version 1.3.0
InfluxDB shell version: 1.3.0
>
> use mydatabase
Using database mydatabase
>
> select * from bar
name: bar
time                    value
----                    -----
2017-06-16T08:00:00.03Z 1
> 

Remember, the issue here is with the milliseconds representation (trimmed to .03).

Now try to execute the following code (output added as comment):

public class Main {

  public static void main(String[] args) {
    long time = TimeUnit.NANOSECONDS.toMillis(1497600000030000000L);

    // 2017-06-16T10:00:00.30Z
    System.out.println(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.S'Z'").format(time));

    // 2017-06-16T10:00:00.30Z
    System.out.println(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SS'Z'").format(time));

    // 2017-06-16T10:00:00.030Z
    System.out.println(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'").format(time));

    // 2017-06-16T10:00:00.0030Z
    System.out.println(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSS'Z'").format(time));

    // 2017-06-16T10:00:00.00030Z
    System.out.println(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSS'Z'").format(time));

    // 2017-06-16T10:00:00.000030Z
    System.out.println(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'").format(time));

    // SAME REPRESENTATION AS INFLUXDB: 2017-06-16T08:00:00.03Z
    DateTimeFormatter rfc3339Format = new DateTimeFormatterBuilder()
      .appendPattern("yyyy-MM-dd'T'HH:mm:ss")
      .appendFraction(ChronoField.MILLI_OF_SECOND, 0, 6, true)
      .appendPattern("X")
      .toFormatter();
    System.out.println(rfc3339Format.withZone(ZoneOffset.UTC).format(Instant.ofEpochMilli(time)));
  }
}

@fmachado
Copy link
Contributor

@majst01 IMO, org.influxdb.impl.TimeUtil should be deprecated and removed later. The developer should be responsible for dealing with date/time conversion. What do you think?

Meanwhile, a patch could be applied by replacing SimpleDateFormat with DateTimeFormatterBuilder and fractions of millisecs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants