Not every package in the index (for the perl distribution) is in Module::CoreList

[p5pRT]

Migrated from rt.perl.org#125166 (status was 'open')

Searchable as RT125166$

[p5pRT]

From @karenetheridge

This describes a problem discovered by Tatsuhiko Miyagawa and described on #toolchain, #p5p yesterday.

There are some modules that are indexed to the perl distribution in 02packages.details.txt that have no entry in Module​::CoreList. This confuses some tools that try to figure out what distribution to fetch a module from.

The original package that was noticed is Tie​::StdHash (which isn't even require()able on its own, as it exists inside Tie/Hash.pm, but that's another problem); this is the full list. Some look like things that shouldn't be indexed; the others should be added to Module​::CoreList.

grep 5.20.2 ~/.cpanm/sources/http%www.cpan.org/02packages.details.txt \
| perl -MModule​::CoreList -MModule​::Metadata -nlae 'Module​::Metadata->new_from_module($F[0]) or warn "$F[0]​: ", Module​::CoreList​::is_core($F[0]), "\n"'
B​::OBJECT​: 0
B​::Section​: 0
Class​::Struct​::Tie_ISA​: 0
EnableModule​: 0
ExtUtils​::XSSymSet​: 1
Haiku​: 0
Moped​::Msg​: 1
ODBM_File​: 1
OptreeCheck​: 0
OS2​::DLL​: 1
OS2​::DLL​::dll​: 0
OS2​::ExtAttr​: 1
OS2​::localMorphPM​: 0
OS2​::PrfDB​: 1
OS2​::PrfDB​::Hini​: 0
OS2​::PrfDB​::Sub​: 0
OS2​::Process​: 1
OS2​::REXX​: 1
OS2​::REXX​::_ARRAY​: 0
OS2​::REXX​::_HASH​: 0
OS2​::REXX​::_SCALAR​: 0
Pod​::Simple​::XHTML​::LocalPodLinks​: 0
POSIX​::SigAction​: 0
POSIX​::SigRt​: 0
t​::BHK​: 0
t​::Markers​: 0
Testing​: 0
Tie​::ExtraHash​: 0
Tie​::StdArray​: 0
Tie​::StdHash​: 0
Tie​::StdScalar​: 0
VMS​::DCLsym​: 1
VMS​::Filespec​: 1
VMS​::stdio​: 0
Win32CORE​: 1
XS​::APItest​: 1
XS​::Typemap​: 1

[p5pRT]

From @karenetheridge

Since this script was run on OSX, some modules don't exist there that actually
do exist in Module​::CoreList. Below is an amended list.

B​::OBJECT
B​::Section
Class​::Struct​::Tie_ISA
EnableModule
Haiku
OptreeCheck
OS2​::DLL​::dll
OS2​::localMorphPM
OS2​::PrfDB​::Hini
OS2​::PrfDB​::Sub
OS2​::REXX​::_ARRAY
OS2​::REXX​::_HASH
OS2​::REXX​::_SCALAR
Pod​::Simple​::XHTML​::LocalPodLinks
POSIX​::SigAction
POSIX​::SigRt
t​::BHK
t​::Markers
Testing
Tie​::ExtraHash
Tie​::StdArray
Tie​::StdHash
Tie​::StdScalar
VMS​::stdio

[p5pRT]

From @karenetheridge

That list was limited to just modules that were not require()able on their own. Here's the full list​:

grep '/perl-5' ~/.cpanm/02packages.details.txt | perl -MModule​::CoreList -nlae'print $F[0] if not Module​::CoreList​::is_core($F[0])'
attrs
B​::Lint​::Plugin​::Test
B​::OBJECT
B​::Section
Class​::Struct​::Tie_ISA
EnableModule
FindExt
Haiku
in
java​::lang​::String
JNI
JPL​::AutoLoader
JPL​::Class
JPL​::Compile
Maintainers
MyTestModule
of
OptreeCheck
OS2​::DLL​::dll
OS2​::localMorphPM
OS2​::PrfDB​::Hini
OS2​::PrfDB​::Sub
OS2​::REXX​::_ARRAY
OS2​::REXX​::_HASH
OS2​::REXX​::_SCALAR
Pod​::Simple​::XHTML​::LocalPodLinks
POSIX​::SigAction
POSIX​::SigRt
t​::BHK
t​::Markers
Testing
Thread​::Signal
Thread​::Specific
Tie​::ExtraHash
Tie​::StdArray
Tie​::StdHash
Tie​::StdScalar
unicode​::distinct
UnicodeCD
VMS​::stdio

[p5pRT]

From @jkeenan

On Tue May 12 13​:50​:08 2015, ether wrote​:

This describes a problem discovered by Tatsuhiko Miyagawa and
described on #toolchain, #p5p yesterday.

There are some modules that are indexed to the perl distribution in
02packages.details.txt that have no entry in Module​::CoreList. This
confuses some tools that try to figure out what distribution to fetch
a module from.

The original package that was noticed is Tie​::StdHash (which isn't
even require()able on its own, as it exists inside Tie/Hash.pm, but
that's another problem); this is the full list. Some look like things
that shouldn't be indexed; the others should be added to
Module​::CoreList.

ether​:

1. I'm having trouble wrapping my head around the actual problem here. Can you give a fuller description of the "confusion" you mentioned? Also, can you describe the relative severity of the problem?

2. To the extent that I understand the problem, it seems like one that will require cross-jurisdictional activity to resolve. Part of the problem seems to be that www.cpan.org -- something not per se within the jurisdiction of perl5porters -- is indexing things it shouldn't. Another part of the problem seems to be that Module​::CoreList -- which is maintained by perl5porters -- lacks entries for certain modules which really should be there (as distinct from modules which are just in core because they are data for testing purposes). And there may be other parts of the problem besides those two.

Is that the situation we're in? Can you recommend a way forward?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @neilb

[apologies if I’ve missed anything in this thread. on holiday, with rubbish wifi/internet, sitting in the dark outside reception]

On 23 May 2015, at 15​:20, James E Keenan via RT <perlbug-followup@​perl.org> wrote​:

On Tue May 12 13​:50​:08 2015, ether wrote​:

This describes a problem discovered by Tatsuhiko Miyagawa and
described on #toolchain, #p5p yesterday.

There are some modules that are indexed to the perl distribution in
02packages.details.txt that have no entry in Module​::CoreList. […]
ether​:

1. I'm having trouble wrapping my head around the actual problem here. Can you give a fuller description of the "confusion" you mentioned? Also, can you describe the relative severity of the problem?

The (one?) problem is “cuckoo packages” - internal packages defined inside modules (where a “module” is a package found inside a file of the expected name).

Where these happen in Core module, the cuckoo packages aren’t listed in Module​::CoreList. I raised this a year or so ago, and discussed it on #p5p. RJBS and possibly others agreed it would be ok/good for Module​::CoreList to include cuckoo packages. There’s obviously the possibility for confusion, so maybe M​::CL should have additional info identifying cuckoo packages.

Neil [feeling guilty that he didn’t do a patch for that]