import - In python, how to know when importing a submodule vs the main modul is mandatory?

Question

I have a follow up beginners question to this: Do I need to import submodules directly?

How do I know when an import of a submodule is explicit? Can I find it with help() or dir() in the parrent modul? And if yes where?

question from:https://stackoverflow.com/questions/66048756/in-python-how-to-know-when-importing-a-submodule-vs-the-main-modul-is-mandatory

Answer 1

The short answer is that you should be able to use dir(), but not help().

The long answer:

Let's take an example the multiprocessing module of Python 3.8.5 (which is what I have). The directory structure of my installation is in part:

Python38
    Lib
        multiprocessing
            dummy
                __init__.py
                connection.py
            __init__.py
            pool.py

Now I import the multiprocessing module and do a dir against it and observe that neither the dummy nor pool modules appear:

Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing
>>> dir(multiprocessing)
['Array', 'AuthenticationError', 'Barrier', 'BoundedSemaphore', 'BufferTooShort', 'Condition', 'Event', 'JoinableQueue', 'Lock', 'Manager', 'Pipe', 'Pool', 'Process', 'ProcessError', 'Queue', 'RLock', 'RawArray', 'RawValue', 'SUBDEBUG', 'SUBWARNING', 'Semaphore', 'SimpleQueue', 'TimeoutError', 'Value', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'active_children', 'allow_connection_pickling', 'context', 'cpu_count', 'current_process', 'freeze_support', 'get_all_start_methods', 'get_context', 'get_logger', 'get_start_method', 'log_to_stderr', 'parent_process', 'process', 'reducer', 'reduction', 'set_executable', 'set_forkserver_preload', 'set_start_method', 'sys']

And, sure enough, if I try to access those modules, I get an error:

>>> multiprocessing.pool
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'multiprocessing' has no attribute 'pool'
>>> multiprocessing.dummy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'multiprocessing' has no attribute 'dummy'

But if I issue help. I get (in part):

>>> help(multiprocessing)
Help on package multiprocessing:

NAME
    multiprocessing

MODULE REFERENCE
    https://docs.python.org/3.8/library/multiprocessing

    The following documentation is automatically generated from the Python
    source files.  It may be incomplete, incorrect or include features that
    are considered implementation detail and may vary between Python
    implementations.  When in doubt, consult the module reference at the
    location listed above.

DESCRIPTION
    # Package analogous to 'threading.py' but using processes
    #
    # multiprocessing/__init__.py
    #
    # This package is intended to duplicate the functionality (and much of
    # the API) of threading.py but uses processes instead of threads.  A
    # subpackage 'multiprocessing.dummy' has the same API but is a simple
    # wrapper for 'threading'.
    #
    # Copyright (c) 2006-2008, R Oudkerk
    # Licensed to PSF under a Contributor Agreement.
    #

PACKAGE CONTENTS
    connection
    context
    dummy (package)
    forkserver
    heap
    managers
    pool
    popen_fork
    (rest of listing ommitted)

You can see that dummy and pool are included, but we know that I have to import these submodules explicitly.

Now we notice from the dir listing that context is listed and it is also listed among the packages named in the help listing. So I should be able to access it without any further importing, and I can:

>>> multiprocessing.context
<module 'multiprocessing.context' from 'C:\Program Files\Python38\lib\multiprocessing\context.py'>

And finally:

>>> from multiprocessing.pool import Pool
>>> from multiprocessing.dummy import Pool
>>>

Ultimately the documentation should tell you what you need to import if only by presenting examples.