Blog plugin failing with `TypeError: can't compare offset-naive and offset-aware datetimes`

[perpil]

Context

I have been successfully building my blog using the blog plugin. I'm not sure whether it was a recent insiders update, daylight savings or some filesystem change but I have not changed my mkdocs.yml file lately. When I attempt to build now or serve, I get this error:

INFO    -  Building documentation...
WARNING -  Debug mode is enabled for "social" plugin.
INFO    -  Cleaning site directory
Traceback (most recent call last):
  File "/opt/homebrew/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/mkdocs/__main__.py", line 272, in serve_command
    serve.serve(**kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 85, in serve
    builder(config)
  File "/opt/homebrew/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 67, in builder
    build(config, serve_url=None if is_clean else serve_url, dirty=is_dirty)
  File "/opt/homebrew/lib/python3.10/site-packages/mkdocs/commands/build.py", line 292, in build
    files = config.plugins.on_files(files, config=config)
  File "/opt/homebrew/lib/python3.10/site-packages/mkdocs/plugins.py", line 593, in on_files
    return self.run_event('files', files, config=config)
  File "/opt/homebrew/lib/python3.10/site-packages/mkdocs/plugins.py", line 566, in run_event
    result = method(item, **kwargs)
  File "/Users/david/Documents/GitHub/mkdocs-material/material/plugins/blog/plugin.py", line 134, in on_files
    self.blog.posts = sorted(
TypeError: can't compare offset-naive and offset-aware datetimes

Bug description

The blog plugin appears to choke on the creation dates of some of the files on my filesystem. I'm able to workaround it by modifying line 134 in the blog plugin (material/plugins/blog/plugin.py) from:

        self.blog.posts = sorted(
            self._resolve_posts(files, config),
            key = lambda post: (
                post.config.pin,
                post.config.date.created
            ),
            reverse = True
        )

to (I also add import pytz at the top of the file)

        self.blog.posts = sorted(
            self._resolve_posts(files, config),
            key = lambda post: (
                post.config.pin,
                post.config.date.created.replace(tzinfo=pytz.utc)
            ),
            reverse = True
        )

Related links

• Reporting a bug
• TypeError: can't compare offset-naive and offset-aware datetimes

Reproduction

I was not able to repro this, I tried several things but could not isolate it. The following is the markdown_extensions and plugins sections from the mkdocs.yml where it occurs.

markdown_extensions:
  - attr_list
  - md_in_html
  - pymdownx.emoji:
      emoji_index: !!python/name:material.extensions.emoji.twemoji
      emoji_generator: !!python/name:material.extensions.emoji.to_svg
  - admonition
  - pymdownx.details
  - pymdownx.snippets:
      url_download: true
      restrict_base_path: false
  - pymdownx.superfences
  - pymdownx.tasklist:
      custom_checkbox: true
  - tables
  - footnotes
plugins:
  - meta
  - blog
  - search
  - git-revision-date-localized:
      enable_creation_date: true
      type: timeago
      exclude:
        - blog/*
  - social:
      cards_layout_dir: layouts
      cards_layout: custom
      debug: true
      #debug_on_build: true
  - privacy:
      enabled: true
      cache: true
      cache_dir: .cache/plugins/privacy
      links_attr_map:
        target: _blank
      assets_exclude:
        - platform.twitter.com/widgets.js

And here is the output of my docs/blog directory:

❯ stat -f "%Sc %z %N" *
May 31 15:38:53 2024 4778 2023-year-in-review.md
Oct 28 14:44:33 2024 8815 a-better-readme.md
May 31 15:44:25 2024 7776 bifurcating-lambda-logs.md
May 31 15:44:19 2024 6025 cloudwatch-insights-tricks.md
Aug 14 16:04:14 2024 15969 coldstart-zero-part-duex.md
May 31 15:44:06 2024 9717 coldstart-zero.md
May 31 15:44:01 2024 4142 done-before-the-bass-drops.md
Jun 27 21:06:41 2024 9605 dynamodb-secretmanager.md
Oct 17 12:58:46 2024 8387 edge-metrics.md
May 31 15:44:38 2024 6188 lambda-env-variable-coldstarts.md
May 31 15:44:43 2024 5648 lambda-request-timeline.md
May 31 15:44:52 2024 15597 logging-for-scale.md
Oct 13 17:09:17 2024 988 migrating-to-cdk.md
May 31 15:44:58 2024 2162 mission.md
May 31 15:45:03 2024 23022 optimizing-lambda-coldstarts.md
May 31 15:45:07 2024 8994 stripped-down-coldstarts.md
May 31 15:45:11 2024 5769 the-global-query.md
May 31 15:45:16 2024 4372 totally-async-eventbridge.md
May 31 15:45:21 2024 7874 why-im-spicy-about-coldstarts.md

Steps to reproduce

I tried creating a minimal blog repro both with and without the git-revision-date-localized plugin but was unable to get this issue to reoccur, it only occurs in my main site directory.

plugins:
  - blog
  - git-revision-date-localized:
      enable_creation_date: true
      type: timeago
      exclude:
        - blog/*

I'm using mkdocs-material 9.5.44+insiders.4.53.14

Browser

Firefox

Before submitting

• I have read and followed the bug reporting guidelines.
• I have attached links to the documentation, and possibly related issues and discussions.
• I assure that I have removed all customizations before submitting this bug report.
• I have attached a .zip file with a minimal reproduction using the built-in info plugin.

[kamilkrzyskow]

Hello @perpil,
your stat -f "%Sc %z %N" * isn't really relevant here, as the config.date.created is the date in your meta data front-matter, it's not taken from the filesystem.

If you're willing to modify the the Python sources please try this, before the line with the error:

for post in self._resolve_posts(files, config):
    print(post.file.src_uri)
    print(" ", post.meta["date"])
    print(" ", post.config.date.created)

this might show what the input was that caused the error.

I agree with your solution from the reference there needs to be some normalization.
However, pytz is an external dependency. tzinfo=timezone.utc should also work 🤔

[perpil]

Here's the output if I put the code you suggest prior to the line that causes the error:

blog/posts/2023-year-in-review.md
  2024-01-04
  2024-01-04 00:00:00
blog/posts/a-better-readme.md
  2023-04-20 00:00:00+00:00
  2023-04-20 00:00:00+00:00
blog/posts/bifurcating-lambda-logs.md
  2024-02-23
  2024-02-23 00:00:00
blog/posts/cloudwatch-insights-tricks.md
  2023-05-11
  2023-05-11 00:00:00
blog/posts/coldstart-zero-part-duex.md
  2024-05-22
  2024-05-22 00:00:00
blog/posts/coldstart-zero.md
  2024-04-19
  2024-04-19 00:00:00
blog/posts/done-before-the-bass-drops.md
  2023-12-12
  2023-12-12 00:00:00
blog/posts/dynamodb-secretmanager.md
  2024-06-27
  2024-06-27 00:00:00
blog/posts/edge-metrics.md
  2024-10-16
  2024-10-16 00:00:00
blog/posts/lambda-env-variable-coldstarts.md
  2024-03-13
  2024-03-13 00:00:00
blog/posts/lambda-request-timeline.md
  2024-02-14
  2024-02-14 00:00:00
blog/posts/logging-for-scale.md
  2023-09-08
  2023-09-08 00:00:00
blog/posts/migrating-to-cdk.md
  2024-10-08
  2024-10-08 00:00:00
blog/posts/mission.md
  2023-04-10
  2023-04-10 00:00:00
blog/posts/optimizing-lambda-coldstarts.md
  2023-09-23
  2023-09-23 00:00:00
blog/posts/stripped-down-coldstarts.md
  2024-02-01
  2024-02-01 00:00:00
blog/posts/the-global-query.md
  2023-11-14
  2023-11-14 00:00:00
blog/posts/totally-async-eventbridge.md
  2023-10-12
  2023-10-12 00:00:00
blog/posts/why-im-spicy-about-coldstarts.md
  2024-01-02
  2024-01-02 00:00:00

Surprisingly, mkdocs serve doesn't fail with those additional prints and no .replace(tzinfo=pytz.utc), it succeeds without error. The code:

        for post in self._resolve_posts(files, config):
            print(post.file.src_uri)
            print(" ", post.meta["date"])
            print(" ", post.config.date.created)
        self.blog.posts = sorted(
            self._resolve_posts(files, config),
            key = lambda post: (
                post.config.pin,
                post.config.date.created
            ),
            reverse = True
        )

Putting: .replace(tzinfo=timezone.utc) fails with:

  File "/Users/david/Documents/GitHub/mkdocs-material/material/plugins/blog/plugin.py", line 142, in <lambda>
    post.config.date.created.replace(tzinfo=timezone.utc)
NameError: name 'timezone' is not defined

[kamilkrzyskow]

Thanks for the follow up:

blog/posts/a-better-readme.md
  2023-04-20 00:00:00+00:00
  2023-04-20 00:00:00+00:00

this is the only file with the +00:00, and it appears to be there for post.meta["date"], is the top front-matter in that file any different compared to the other files?

NameError: name 'timezone' is not defined

This needs to be imported via from datetime import timezone, what I meant with pytz, is that it's not included by default during pip install mkdocs-material, so if there is no need let's stick to the built-ins.

[perpil]

That was it! I had a date in the frontmatter that was set to: 2023-04-20T00:00:00Z when I removed the timestamp, it worked fine.

Here's a repro:

9.5.44+insiders.4.53.14-blog-plugin-breaks-with-timestamp-frontmatter.zip

I've also confirmed that adding from datetime import timezone fixes the NameError above.

For now I've removed the timestamp from the frontmatter date on the offending file and am unblocked.

[squidfunk]

Thanks for reporting. Could you PR the necessary change? If so, please send a PR for the community edition, as I believe all functionality that is impacted was already released.

[perpil]

I can do a PR for this change if it's just hardcoding the timezone. A couple of quick questions before I do.

1. Is the blog plugin the only place this change is necessary?
2. Is there a better way than hardcoding to get the timezone? Arguably it is user error to include the timestamp and timezone on the date, but iso8601 is a well known format. Is there some way to process the frontmatter so it respects the timezone if it's present, otherwise uses utc?

[squidfunk]

Ooph, good questions. So, we can only work with what we get from YAML. We're parsing the front matter here:

mkdocs-material/src/plugins/blog/structure/options.py

Lines 53 to 77 in 596aeea

	# Normalize the supported types for post dates to datetime
	def pre_validation(self, config: Config, key_name: str):
	# If the date points to a scalar value, convert it to a dictionary, as
	# we want to allow the author to specify custom and arbitrary dates for
	# posts. Currently, only the `created` date is mandatory, because it's
	# needed to sort posts for views.
	if not isinstance(config[key_name], dict):
	config[key_name] = { "created": config[key_name] }
	# Convert all date values to datetime
	for key, value in config[key_name].items():
	# Handle datetime - since datetime is a subclass of date, we need
	# to check it first, or we lose the time - see https://t.ly/-KG9N
	if isinstance(value, datetime):
	continue
	# Handle date - we set 00:00:00 as the default time, if the author
	# only supplied a date, and convert it to datetime
	if isinstance(value, date):
	config[key_name][key] = datetime.combine(value, time())
	# Initialize date dictionary
	config[key_name] = DateDict(config[key_name])

I'm not sure what the best way forward is, but my educated guess (without diving too deep into this topic, as I'm currently fighting other battles) is that if we would support timezones, we should normalize them to UTC, i.e., +00:00 -> Z + adjustments. I'm not too tied up in this topic right now, so I'm open to discussing things in a PR. If somebody wants to go ahead and dive into the reasoning on this one, the help is very much appreciated.