Skip to content

module!: take strings instead of byte strings #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ojeda opened this issue May 6, 2021 · 7 comments · May be fixed by #278
Closed

module!: take strings instead of byte strings #252

ojeda opened this issue May 6, 2021 · 7 comments · May be fixed by #278
Labels
• lib Related to the `rust/` library.

Comments

@ojeda
Copy link
Member

ojeda commented May 6, 2021 •
edited
Loading

Given it is a proc macro, we could take normal strings for the fields and ensure they are ASCII if/where needed.

This would make the interface a bit leaner.

For author, it would be particularly fitting anyway, because we would like to allow names requiring UTF-8, such as non-romanized names. There are kernel modules with MODULE_AUTHORs with non-ASCII characters already (and encoded as UTF-8) e.g.

linux/drivers/phy/ingenic/phy-ingenic-usb.c

Lines 385 to 386 in 428d64a

MODULE_AUTHOR("周琰杰 (Zhou Yanjie) <[email protected]>");
MODULE_AUTHOR("漆鹏振 (Qi Pengzhen) <[email protected]>");

@ojeda ojeda added • lib Related to the `rust/` library. prio: normal good first issue Good for newcomers labels May 6, 2021
@nbdd0121
Copy link
Member

nbdd0121 commented May 9, 2021

This is actually non-trivial, because binary string constants cannot contain non-ASCII characters, so module proc_macro needs to properly parse the string literal and convert it to a binary string.

@bjorn3
Copy link
Member

bjorn3 commented May 9, 2021

str::as_bytes gives the raw UTF-8 bytes of a string. You can directly pass this to Literal::byte_string I believe. Rustc automatically escapes the individual bytes: https://github.com/rust-lang/rust/blob/bba8710616e5e4722215c0d6b27abaedca03ebad/compiler/rustc_expand/src/proc_macro_server.rs#L566-L572

@nbdd0121
Copy link
Member

nbdd0121 commented May 9, 2021

str::as_bytes gives the raw UTF-8 bytes of a string. You can directly pass this to Literal::byte_string I believe. Rustc automatically escapes the individual bytes: https://github.com/rust-lang/rust/blob/bba8710616e5e4722215c0d6b27abaedca03ebad/compiler/rustc_expand/src/proc_macro_server.rs#L566-L572

However there is no way to get a String or Vec<u8> from a Literal other than parsing it yourself. It's easy once once it's parsed. If you look at #258 you'll see that's exactly what I do.

@bjorn3
Copy link
Member

bjorn3 commented May 9, 2021

Ah, I see what you mean. If licensing would allow it copying syn or rustc_ast would be the easiest solutions.

@ojeda ojeda removed the good first issue Good for newcomers label May 11, 2021
@nbdd0121 nbdd0121 mentioned this issue May 11, 2021
Draft
@nbdd0121
Copy link
Member

Are there scenarios that we need to take non-UTF-8 strings?

@ojeda
Copy link
Member Author

ojeda commented May 12, 2021

I do not know, but even if we happen to need it, I do not think we should worry about that for the time being, in particular if it makes things more complex.

@nbdd0121 nbdd0121 mentioned this issue May 19, 2021
Open
@nbdd0121 nbdd0121 mentioned this issue Aug 5, 2022
Merged
nbdd0121 added a commit to nbdd0121/linux that referenced this issue Aug 5, 2022
@nbdd0121
rust: take str literal instead bstr literal in module! macro
ede7317 
For simplicity (avoid parsing), escape sequences are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
nbdd0121 added a commit to nbdd0121/linux that referenced this issue Aug 5, 2022
@nbdd0121
rust: take str literal instead bstr literal in module! macro
593e659 
For simplicity (avoid parsing), escape sequences and raw string literals
are not yet handled.

Module names, aliases and license strings are restricted to ASCII only.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
ojeda added a commit to Rust-for-Linux/rust-out-of-tree-module that referenced this issue Aug 6, 2022
@ojeda
rust_out_of_tree: use string literals in module! macro
b1da1b5 
See commit 593e65924b2e ("rust: take str literal instead bstr literal
in `module!` macro") in the main repository:

    For simplicity (avoid parsing), escape sequences and raw string literals
    are not yet handled.

    Module names, aliases and license strings are restricted to ASCII only.

    Signed-off-by: Gary Guo <[email protected]>

Link: Rust-for-Linux/linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Link: Rust-for-Linux/linux@593e659
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda pushed a commit to ojeda/linux that referenced this issue Nov 10, 2022
@nbdd0121 @ojeda
rust: macros: take string literals in module!
1fce88b 
Instead of taking binary string literals, take string ones instead,
making it easier for users to define a module, i.e. instead of
calling `module!` like:

    module! {
        ...
        name: b"rust_minimal",
        ...
    }

now it is called as:

    module! {
        ...
        name: "rust_minimal",
        ...
    }

Module names, aliases and license strings are restricted to
ASCII only. However, the author and the description allows UTF-8.

For simplicity (avoid parsing), escape sequences and raw string
literals are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
[Reworded, adapted for upstream and applied latest changes]
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda pushed a commit to ojeda/linux that referenced this issue Nov 10, 2022
@nbdd0121 @ojeda
rust: macros: take string literals in module!
9085be7 
Instead of taking binary string literals, take string ones instead,
making it easier for users to define a module, i.e. instead of
calling `module!` like:

    module! {
        ...
        name: b"rust_minimal",
        ...
    }

now it is called as:

    module! {
        ...
        name: "rust_minimal",
        ...
    }

Module names, aliases and license strings are restricted to
ASCII only. However, the author and the description allows UTF-8.

For simplicity (avoid parsing), escape sequences and raw string
literals are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
[Reworded, adapted for upstream and applied latest changes]
Signed-off-by: Miguel Ojeda <[email protected]>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Nov 10, 2022
@nbdd0121 @intel-lab-lkp
rust: macros: take string literals in module!
0779d4a 
Instead of taking binary string literals, take string ones instead,
making it easier for users to define a module, i.e. instead of
calling `module!` like:

    module! {
        ...
        name: b"rust_minimal",
        ...
    }

now it is called as:

    module! {
        ...
        name: "rust_minimal",
        ...
    }

Module names, aliases and license strings are restricted to
ASCII only. However, the author and the description allows UTF-8.

For simplicity (avoid parsing), escape sequences and raw string
literals are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
[Reworded, adapted for upstream and applied latest changes]
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda pushed a commit to ojeda/linux that referenced this issue Dec 2, 2022
@nbdd0121 @ojeda
rust: macros: take string literals in module!
f9bcae8 
Instead of taking binary string literals, take string ones instead,
making it easier for users to define a module, i.e. instead of
calling `module!` like:

    module! {
        ...
        name: b"rust_minimal",
        ...
    }

now it is called as:

    module! {
        ...
        name: "rust_minimal",
        ...
    }

Module names, aliases and license strings are restricted to
ASCII only. However, the author and the description allows UTF-8.

For simplicity (avoid parsing), escape sequences and raw string
literals are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
[Reworded, adapted for upstream and applied latest changes]
Signed-off-by: Miguel Ojeda <[email protected]>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Dec 2, 2022
@nbdd0121 @intel-lab-lkp
rust: macros: take string literals in module!
d48916a 
Instead of taking binary string literals, take string ones instead,
making it easier for users to define a module, i.e. instead of
calling `module!` like:

    module! {
        ...
        name: b"rust_minimal",
        ...
    }

now it is called as:

    module! {
        ...
        name: "rust_minimal",
        ...
    }

Module names, aliases and license strings are restricted to
ASCII only. However, the author and the description allows UTF-8.

For simplicity (avoid parsing), escape sequences and raw string
literals are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
[Reworded, adapted for upstream and applied latest changes]
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda pushed a commit to ojeda/linux that referenced this issue Dec 4, 2022
@nbdd0121 @ojeda
rust: macros: take string literals in module!
b13c988 
Instead of taking binary string literals, take string ones instead,
making it easier for users to define a module, i.e. instead of
calling `module!` like:

    module! {
        ...
        name: b"rust_minimal",
        ...
    }

now it is called as:

    module! {
        ...
        name: "rust_minimal",
        ...
    }

Module names, aliases and license strings are restricted to
ASCII only. However, the author and the description allows UTF-8.

For simplicity (avoid parsing), escape sequences and raw string
literals are not yet handled.

Link: Rust-for-Linux#252
Link: https://lore.kernel.org/lkml/[email protected]/
Signed-off-by: Gary Guo <[email protected]>
[Reworded, adapted for upstream and applied latest changes]
Signed-off-by: Miguel Ojeda <[email protected]>
@y86-dev
Copy link
Member

y86-dev commented Jul 25, 2024

Stale issue: this is already the case.

@y86-dev y86-dev closed this as completed Jul 25, 2024
Ayush1325 pushed a commit to Ayush1325/linux that referenced this issue Apr 22, 2025
@naota @kdave
btrfs: zoned: skip reporting zone for new block group
5605a09 
There is a potential deadlock if we do report zones in an IO context, detailed
in below lockdep report. When one process do a report zones and another process
freezes the block device, the report zones side cannot allocate a tag because
the freeze is already started. This can thus result in new block group creation
to hang forever, blocking the write path.

Thankfully, a new block group should be created on empty zones. So, reporting
the zones is not necessary and we can set the write pointer = 0 and load the
zone capacity from the block layer using bdev_zone_capacity() helper.

 ======================================================
 WARNING: possible circular locking dependency detected
 6.14.0-rc1 Rust-for-Linux#252 Not tainted
 ------------------------------------------------------
 modprobe/1110 is trying to acquire lock:
 ffff888100ac83e0 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: __flush_work+0x38f/0xb60

 but task is already holding lock:
 ffff8881205b6f20 (&q->q_usage_counter(queue)Rust-for-Linux#16){++++}-{0:0}, at: sd_remove+0x85/0x130

 which lock already depends on the new lock.

 the existing dependency chain (in reverse order) is:

 -> Rust-for-Linux#3 (&q->q_usage_counter(queue)Rust-for-Linux#16){++++}-{0:0}:
        blk_queue_enter+0x3d9/0x500
        blk_mq_alloc_request+0x47d/0x8e0
        scsi_execute_cmd+0x14f/0xb80
        sd_zbc_do_report_zones+0x1c1/0x470
        sd_zbc_report_zones+0x362/0xd60
        blkdev_report_zones+0x1b1/0x2e0
        btrfs_get_dev_zones+0x215/0x7e0 [btrfs]
        btrfs_load_block_group_zone_info+0x6d2/0x2c10 [btrfs]
        btrfs_make_block_group+0x36b/0x870 [btrfs]
        btrfs_create_chunk+0x147d/0x2320 [btrfs]
        btrfs_chunk_alloc+0x2ce/0xcf0 [btrfs]
        start_transaction+0xce6/0x1620 [btrfs]
        btrfs_uuid_scan_kthread+0x4ee/0x5b0 [btrfs]
        kthread+0x39d/0x750
        ret_from_fork+0x30/0x70
        ret_from_fork_asm+0x1a/0x30

 -> Rust-for-Linux#2 (&fs_info->dev_replace.rwsem){++++}-{4:4}:
        down_read+0x9b/0x470
        btrfs_map_block+0x2ce/0x2ce0 [btrfs]
        btrfs_submit_chunk+0x2d4/0x16c0 [btrfs]
        btrfs_submit_bbio+0x16/0x30 [btrfs]
        btree_write_cache_pages+0xb5a/0xf90 [btrfs]
        do_writepages+0x17f/0x7b0
        __writeback_single_inode+0x114/0xb00
        writeback_sb_inodes+0x52b/0xe00
        wb_writeback+0x1a7/0x800
        wb_workfn+0x12a/0xbd0
        process_one_work+0x85a/0x1460
        worker_thread+0x5e2/0xfc0
        kthread+0x39d/0x750
        ret_from_fork+0x30/0x70
        ret_from_fork_asm+0x1a/0x30

 -> Rust-for-Linux#1 (&fs_info->zoned_meta_io_lock){+.+.}-{4:4}:
        __mutex_lock+0x1aa/0x1360
        btree_write_cache_pages+0x252/0xf90 [btrfs]
        do_writepages+0x17f/0x7b0
        __writeback_single_inode+0x114/0xb00
        writeback_sb_inodes+0x52b/0xe00
        wb_writeback+0x1a7/0x800
        wb_workfn+0x12a/0xbd0
        process_one_work+0x85a/0x1460
        worker_thread+0x5e2/0xfc0
        kthread+0x39d/0x750
        ret_from_fork+0x30/0x70
        ret_from_fork_asm+0x1a/0x30

 -> #0 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}:
        __lock_acquire+0x2f52/0x5ea0
        lock_acquire+0x1b1/0x540
        __flush_work+0x3ac/0xb60
        wb_shutdown+0x15b/0x1f0
        bdi_unregister+0x172/0x5b0
        del_gendisk+0x841/0xa20
        sd_remove+0x85/0x130
        device_release_driver_internal+0x368/0x520
        bus_remove_device+0x1f1/0x3f0
        device_del+0x3bd/0x9c0
        __scsi_remove_device+0x272/0x340
        scsi_forget_host+0xf7/0x170
        scsi_remove_host+0xd2/0x2a0
        sdebug_driver_remove+0x52/0x2f0 [scsi_debug]
        device_release_driver_internal+0x368/0x520
        bus_remove_device+0x1f1/0x3f0
        device_del+0x3bd/0x9c0
        device_unregister+0x13/0xa0
        sdebug_do_remove_host+0x1fb/0x290 [scsi_debug]
        scsi_debug_exit+0x17/0x70 [scsi_debug]
        __do_sys_delete_module.isra.0+0x321/0x520
        do_syscall_64+0x93/0x180
        entry_SYSCALL_64_after_hwframe+0x76/0x7e

 other info that might help us debug this:

 Chain exists of:
   (work_completion)(&(&wb->dwork)->work) --> &fs_info->dev_replace.rwsem --> &q->q_usage_counter(queue)Rust-for-Linux#16

  Possible unsafe locking scenario:

        CPU0                    CPU1
        ----                    ----
   lock(&q->q_usage_counter(queue)Rust-for-Linux#16);
                                lock(&fs_info->dev_replace.rwsem);
                                lock(&q->q_usage_counter(queue)Rust-for-Linux#16);
   lock((work_completion)(&(&wb->dwork)->work));

  *** DEADLOCK ***

 5 locks held by modprobe/1110:
  #0: ffff88811f7bc108 (&dev->mutex){....}-{4:4}, at: device_release_driver_internal+0x8f/0x520
  Rust-for-Linux#1: ffff8881022ee0e0 (&shost->scan_mutex){+.+.}-{4:4}, at: scsi_remove_host+0x20/0x2a0
  Rust-for-Linux#2: ffff88811b4c4378 (&dev->mutex){....}-{4:4}, at: device_release_driver_internal+0x8f/0x520
  Rust-for-Linux#3: ffff8881205b6f20 (&q->q_usage_counter(queue)Rust-for-Linux#16){++++}-{0:0}, at: sd_remove+0x85/0x130
  Rust-for-Linux#4: ffffffffa3284360 (rcu_read_lock){....}-{1:3}, at: __flush_work+0xda/0xb60

 stack backtrace:
 CPU: 0 UID: 0 PID: 1110 Comm: modprobe Not tainted 6.14.0-rc1 Rust-for-Linux#252
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014
 Call Trace:
  <TASK>
  dump_stack_lvl+0x6a/0x90
  print_circular_bug.cold+0x1e0/0x274
  check_noncircular+0x306/0x3f0
  ? __pfx_check_noncircular+0x10/0x10
  ? mark_lock+0xf5/0x1650
  ? __pfx_check_irq_usage+0x10/0x10
  ? lockdep_lock+0xca/0x1c0
  ? __pfx_lockdep_lock+0x10/0x10
  __lock_acquire+0x2f52/0x5ea0
  ? __pfx___lock_acquire+0x10/0x10
  ? __pfx_mark_lock+0x10/0x10
  lock_acquire+0x1b1/0x540
  ? __flush_work+0x38f/0xb60
  ? __pfx_lock_acquire+0x10/0x10
  ? __pfx_lock_release+0x10/0x10
  ? mark_held_locks+0x94/0xe0
  ? __flush_work+0x38f/0xb60
  __flush_work+0x3ac/0xb60
  ? __flush_work+0x38f/0xb60
  ? __pfx_mark_lock+0x10/0x10
  ? __pfx___flush_work+0x10/0x10
  ? __pfx_wq_barrier_func+0x10/0x10
  ? __pfx___might_resched+0x10/0x10
  ? mark_held_locks+0x94/0xe0
  wb_shutdown+0x15b/0x1f0
  bdi_unregister+0x172/0x5b0
  ? __pfx_bdi_unregister+0x10/0x10
  ? up_write+0x1ba/0x510
  del_gendisk+0x841/0xa20
  ? __pfx_del_gendisk+0x10/0x10
  ? _raw_spin_unlock_irqrestore+0x35/0x60
  ? __pm_runtime_resume+0x79/0x110
  sd_remove+0x85/0x130
  device_release_driver_internal+0x368/0x520
  ? kobject_put+0x5d/0x4a0
  bus_remove_device+0x1f1/0x3f0
  device_del+0x3bd/0x9c0
  ? __pfx_device_del+0x10/0x10
  __scsi_remove_device+0x272/0x340
  scsi_forget_host+0xf7/0x170
  scsi_remove_host+0xd2/0x2a0
  sdebug_driver_remove+0x52/0x2f0 [scsi_debug]
  ? kernfs_remove_by_name_ns+0xc0/0xf0
  device_release_driver_internal+0x368/0x520
  ? kobject_put+0x5d/0x4a0
  bus_remove_device+0x1f1/0x3f0
  device_del+0x3bd/0x9c0
  ? __pfx_device_del+0x10/0x10
  ? __pfx___mutex_unlock_slowpath+0x10/0x10
  device_unregister+0x13/0xa0
  sdebug_do_remove_host+0x1fb/0x290 [scsi_debug]
  scsi_debug_exit+0x17/0x70 [scsi_debug]
  __do_sys_delete_module.isra.0+0x321/0x520
  ? __pfx___do_sys_delete_module.isra.0+0x10/0x10
  ? __pfx_slab_free_after_rcu_debug+0x10/0x10
  ? kasan_save_stack+0x2c/0x50
  ? kasan_record_aux_stack+0xa3/0xb0
  ? __call_rcu_common.constprop.0+0xc4/0xfb0
  ? kmem_cache_free+0x3a0/0x590
  ? __x64_sys_close+0x78/0xd0
  do_syscall_64+0x93/0x180
  ? lock_is_held_type+0xd5/0x130
  ? __call_rcu_common.constprop.0+0x3c0/0xfb0
  ? lockdep_hardirqs_on+0x78/0x100
  ? __call_rcu_common.constprop.0+0x3c0/0xfb0
  ? __pfx___call_rcu_common.constprop.0+0x10/0x10
  ? kmem_cache_free+0x3a0/0x590
  ? lockdep_hardirqs_on_prepare+0x16d/0x400
  ? do_syscall_64+0x9f/0x180
  ? lockdep_hardirqs_on+0x78/0x100
  ? do_syscall_64+0x9f/0x180
  ? __pfx___x64_sys_openat+0x10/0x10
  ? lockdep_hardirqs_on_prepare+0x16d/0x400
  ? do_syscall_64+0x9f/0x180
  ? lockdep_hardirqs_on+0x78/0x100
  ? do_syscall_64+0x9f/0x180
  entry_SYSCALL_64_after_hwframe+0x76/0x7e
 RIP: 0033:0x7f436712b68b
 RSP: 002b:00007ffe9f1a8658 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
 RAX: ffffffffffffffda RBX: 00005559b367fd80 RCX: 00007f436712b68b
 RDX: 0000000000000000 RSI: 0000000000000800 RDI: 00005559b367fde8
 RBP: 00007ffe9f1a8680 R08: 1999999999999999 R09: 0000000000000000
 R10: 00007f43671a5fe0 R11: 0000000000000206 R12: 0000000000000000
 R13: 00007ffe9f1a86b0 R14: 0000000000000000 R15: 0000000000000000
  </TASK>

Reported-by: Shin'ichiro Kawasaki <[email protected]>
CC: <[email protected]> # 6.13+
Tested-by: Shin'ichiro Kawasaki <[email protected]>
Reviewed-by: Damien Le Moal <[email protected]>
Reviewed-by: Johannes Thumshirn <[email protected]>
Signed-off-by: Naohiro Aota <[email protected]>
Signed-off-by: David Sterba <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
• lib Related to the `rust/` library.
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

module! improvements nbdd0121/linux
4 participants
@ojeda @nbdd0121 @bjorn3 @y86-dev