fix: #337 add file name transfrom

[StringKe]

fix: #337

Use transliteration for filenames in other languages.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 49825f5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

💥 An error occurred when fetching the changed packages and changesets in this PR

Some errors occurred when validating the changesets config:
The package or glob expression "examples-*" is specified in the `ignore` option but it is not found in the project. You may have misspelled the package name or provided an invalid glob expression. Note that glob expressions must be defined according to https://www.npmjs.com/package/micromatch.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[schickling]

Sorry for the delay. I hope to get a chance to look into this soon! 🙏

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[StringKe]

up

[schickling]

Thanks a lot for this PR @StringKe. I agree that we should try to address the underlaying issue (as described in #337), however, the transliteration package seems to add 1.6MB of dependencies which I'm wary to add.

Maybe you (or someone else) is aware of a way to accomplish the same but using more standardised platform APIs?

[StringKe]

hi, @schickling
There is no good way to do this; there are too many languages in the world to use anything but a dictionary-like function to mirror the conversion.

I also have another method

Use unicode as filename, this does not add extra volume but is already unreadable for filenames and the generated url is not friendly enough to read and remember.

origin: 你好.mdx
eng_origin: hello.mdx
transliteration: ni-hao.mdx
unicode: 20320-22909.mdx

origin: 这是一个案例文档
eng_origin: this-is-a-case-document.mdx
transliteration: zhe-shi-yi-ge-an-li-wen-dang.mdx
unicode: 36825-26159-19968-20010-26696-20363-25991-26723.mdx

[schickling]

I see. Let me see whether I can think of some kind of "user-configuration" way to enable the use case above.

[StringKe]

I see. Let me see whether I can think of some kind of "user-configuration" way to enable the use case above.

I favor using this approach by providing a slugify method to allow processing of filenames.

[StringKe]

Close, there has been another result.

[tansongchen]

@StringKe What do you mean by "there has been another result"? Is there any workaround to this problem?
@schickling Any updates on that?

[tansongchen]

I am currently a Flowershow user and I cannot use Chinese in file names because it depends on Contentlayer

[StringKe]

@StringKe What do you mean by "there has been another result"? Is there any workaround to this problem? @schickling Any updates on that?

You can modify the specified files with pnpm patch or npm patch pacakge to solve the problem for a short time.

In the long run, you still need this repository to support custom filename conversions

[huanfe1]

I also meet the same problem and tried some solutions.

Here is my solution.

I add a hash-sum package.

import hash from 'hash-sum'

And change this line of code

contentlayer/packages/@contentlayer/core/src/generation/generate-dotpkg.ts

Line 327 in 024d387

const makeVariableName = flow(idToFileName, (_) => camelCase(_, { stripRegexp: /[^A-Z0-9ㄱ-힣\_]/gi }))

To

  const makeVariableName = flow(hash, idToFileName, (_) => camelCase(_, { stripRegexp: /[^A-Z0-9\_]/gi }))

Or don't add any package.

Change code to

const makeVariableName = flow((_) => Buffer.from(_).toString('base64'), idToFileName, (_) => camelCase(_, { stripRegexp: /[^A-Z0-9\_]/gi }));

But this may lead to variable names that are too long.

Or like #431 change regular expression, add Chinese to it, use Chinese variable names.

const makeVariableName = flow(idToFileName, (_) => camelCase(_, { stripRegexp: /[^\u4e00-\u9fa5A-Z0-9\_]/gi }));