Cannot use Debug mode in INET anymore

[haug-den-lucas]

Hello,

for some reason I cannot use the Debug mode in INET anymore and I cannot figure out why.

For testing I completely reinstalled OMNeT++ 6.0.3 in a new folder to ensure there is not some old configuration issue.
I can use the debug mode for these OMNeT++ samples (e.g. the TicToc sample)

Additionally I created a new workspace in which I installed INET 4.5.2 from the INET v4.5.2 git tag and another new workspace where I've let OMNeT++ install INET automatically.

When I try to run any configuation in debug mode (e.g. timeawareshaper), it builds the project and I'm prompted with the following error:

For every subsequent run where I try to run the configuration in debug mode I get the following error:

When I delete the .metadata folder in my workspace and open the workspace again I jump back to the first error.

I start OMNeT++ in a conda environment:

Conda Environment extract

$ conda list
# packages in environment at /home/haugls/miniconda3/envs/omnet:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
blas                      1.0                         mkl  
bottleneck                1.3.7           py312ha883a20_0  
brotli                    1.0.9                h5eee18b_8  
brotli-bin                1.0.9                h5eee18b_8  
bzip2                     1.0.8                h5eee18b_6  
ca-certificates           2024.3.11            h06a4308_0  
contourpy                 1.2.0           py312hdb19cb5_0  
cycler                    0.11.0             pyhd3eb1b0_0  
cyrus-sasl                2.1.28               h52b45da_1  
dbus                      1.13.18              hb2f20db_0  
expat                     2.6.2                h6a678d5_0  
fontconfig                2.14.1               h4c34cd2_2  
fonttools                 4.51.0          py312h5eee18b_0  
freetype                  2.12.1               h4a9f257_0  
glib                      2.78.4               h6a678d5_0  
glib-tools                2.78.4               h6a678d5_0  
gst-plugins-base          1.14.1               h6a678d5_1  
gstreamer                 1.14.1               h5eee18b_1  
icu                       73.1                 h6a678d5_0  
intel-openmp              2023.1.0         hdb19cb5_46306  
jpeg                      9e                   h5eee18b_1  
kiwisolver                1.4.4           py312h6a678d5_0  
krb5                      1.20.1               h143b758_1  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      3.0                  h295c915_0  
libbrotlicommon           1.0.9                h5eee18b_8  
libbrotlidec              1.0.9                h5eee18b_8  
libbrotlienc              1.0.9                h5eee18b_8  
libclang                  14.0.6          default_hc6dbbc7_1  
libclang13                14.0.6          default_he11475f_1  
libcups                   2.4.2                h2d74bed_1  
libdeflate                1.17                 h5eee18b_1  
libedit                   3.1.20230828         h5eee18b_0  
libffi                    3.4.4                h6a678d5_1  
libgcc                    7.2.0                h69d50b8_2  
libgcc-ng                 11.2.0               h1234567_1  
libgfortran-ng            11.2.0               h00389a5_1  
libgfortran5              11.2.0               h1234567_1  
libglib                   2.78.4               hdc74915_0  
libgomp                   11.2.0               h1234567_1  
libiconv                  1.16                 h5eee18b_3  
libllvm14                 14.0.6               hdb19cb5_3  
libpng                    1.6.39               h5eee18b_0  
libpq                     12.17                hdbd6064_0  
libstdcxx-ng              11.2.0               h1234567_1  
libtiff                   4.5.1                h6a678d5_0  
libuuid                   1.41.5               h5eee18b_0  
libwebp-base              1.3.2                h5eee18b_0  
libxcb                    1.15                 h7f8727e_0  
libxkbcommon              1.0.1                h5eee18b_1  
libxml2                   2.10.4               hfdd30dd_2  
lz4-c                     1.9.4                h6a678d5_1  
matplotlib                3.8.4           py312h06a4308_0  
matplotlib-base           3.8.4           py312h526ad5a_0  
mkl                       2023.1.0         h213fc3f_46344  
mkl-service               2.4.0           py312h5eee18b_1  
mkl_fft                   1.3.8           py312h5eee18b_0  
mkl_random                1.2.4           py312hdb19cb5_0  
mysql                     5.7.24               h721c034_2  
ncurses                   6.4                  h6a678d5_0  
numexpr                   2.8.7           py312hf827012_0  
numpy                     1.26.4          py312hc5e2394_0  
numpy-base                1.26.4          py312h0da6c21_0  
openjpeg                  2.4.0                h3ad879b_0  
openssl                   3.0.14               h5eee18b_0  
packaging                 23.2            py312h06a4308_0  
pandas                    2.2.2           py312h526ad5a_0  
pcre2                     10.42                hebb0a14_1  
pillow                    10.3.0          py312h5eee18b_0  
pip                       24.0            py312h06a4308_0  
ply                       3.11            py312h06a4308_1  
posix-ipc                 1.1.1                    pypi_0    pypi
pybind11-abi              5                    hd3eb1b0_0  
pyparsing                 3.0.9           py312h06a4308_0  
pyqt                      5.15.10         py312h6a678d5_0  
pyqt5-sip                 12.13.0         py312h5eee18b_0  
python                    3.12.3               h996f2a0_1  
python-dateutil           2.9.0post0      py312h06a4308_2  
python-tzdata             2023.3             pyhd3eb1b0_0  
pytz                      2024.1          py312h06a4308_0  
qt-main                   5.15.2              h53bd1ea_10  
readline                  8.2                  h5eee18b_0  
scipy                     1.13.1          py312hc5e2394_0  
seaborn                   0.13.2          py312h06a4308_0  
setuptools                69.5.1          py312h06a4308_0  
sip                       6.7.12          py312h6a678d5_0  
six                       1.16.0             pyhd3eb1b0_1  
sqlite                    3.45.3               h5eee18b_0  
tbb                       2021.8.0             hdb19cb5_0  
tk                        8.6.14               h39e8969_0  
tornado                   6.3.3           py312h5eee18b_0  
tzdata                    2024a                h04d1e81_0  
unicodedata2              15.1.0          py312h5eee18b_0  
wheel                     0.43.0          py312h06a4308_0  
xz                        5.4.6                h5eee18b_1  
zlib                      1.2.13               h5eee18b_1  
zstd                      1.5.5                hc292b87_2

$ pip list
Package         Version
--------------- -----------
Bottleneck      1.3.7
contourpy       1.2.0
cycler          0.11.0
fonttools       4.51.0
kiwisolver      1.4.4
matplotlib      3.8.4
mkl-fft         1.3.8
mkl-random      1.2.4
mkl-service     2.4.0
numexpr         2.8.7
numpy           1.26.4
packaging       23.2
pandas          2.2.2
pillow          10.3.0
pip             24.0
ply             3.11
posix_ipc       1.1.1
pyparsing       3.0.9
PyQt5           5.15.10
PyQt5-sip       12.13.0
python-dateutil 2.9.0.post0
pytz            2024.1
scipy           1.13.1
seaborn         0.13.2
setuptools      69.5.1
sip             6.7.12
six             1.16.0
tornado         6.3.3
tzdata          2023.3
unicodedata2    15.1.0
wheel           0.43.0

Does someone have an idea how to fix this?

[rhornig]

make sure that you source setenv in the INET root directory and start the IDE only after that, from that particular shell. The debugger tries to set up some fomatters and it cannot find them as they are defined in inetroot/python dir which is added only by the setenv script.

[haug-den-lucas]

Sadly that didn't work.
I deleted the .metadata in my workspace again and did a make cleanall in the inet project to start clean again.

Then performed the following steps:

~/workspaces/test$ conda activate omnet
~/workspaces/test$ source ~/omnetpp-6.0.3_2/setenv 
Environment for 'omnetpp-6.0.3' in directory '/home/haugls/omnetpp-6.0.3_2' is ready.
~/workspaces/test$ source inet/setenv 
Environment for INET 4.5.2 in directory '/home/haugls/workspaces/test/inet' is ready.
~/workspaces/test$ omnetpp
Starting the OMNeT++ IDE...

Then I again imported the INET project using the "Open projects from file system" menu and tried to run a simulation in debug mode.
Got the first error again.

[haug-den-lucas]

Some additional information:
I also tried installing OMNeT++ again with the following changes:

• Use X11 instead of Wayland
• Don't use conda at all but instead use the system-wide packages

Getting the same two errors and no clue what else to try.

I think the problem occurs since I moved from Ubuntu 22.04 to 24.04.
It was a clean install, no upgrade

[rhornig]

It's strange as the file reporting the error (gdbinit.py) is in OMNeT++ and does not reference any inet specific package so I don't know why it throws an exeception. I will try to reproduce this. I recently upgraded to ubuntu 24.04, too.

[levy]

Try starting the python repl and import inet, maybe you will see the problem.

[haug-den-lucas]

It's strange as the file reporting the error (gdbinit.py) is in OMNeT++ and does not reference any inet specific package so I don't know why it throws an exeception. I will try to reproduce this. I recently upgraded to ubuntu 24.04, too.

I'm currently setting up a Ubuntu 24.04 VM and I will try to reproduce the issue there.

Try starting the python repl and import inet, maybe you will see the problem.

~/workspaces/test3/inet4.5$ source setenv
Environment for INET 4.5.2 in directory '/home/haugls/workspaces/test3/inet4.5' is ready.
~/workspaces/test3/inet4.5$ python3
Python 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inet
No module named 'py4j': optional inet.common.ide package will not work
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/__init__.py", line 10, in <module>
    from inet.documentation import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/documentation/__init__.py", line 1, in <module>
    from inet.documentation.chart import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/documentation/chart.py", line 10, in <module>
    from inet.simulation.project import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/simulation/__init__.py", line 12, in <module>
    from inet.simulation.config import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/simulation/config.py", line 12, in <module>
    from inet.simulation.project import *
  File "/home/haugls/workspaces/test3/inet4.5/python/inet/simulation/project.py", line 126, in <module>
    ned_exclusions=[s.strip() for s in open(get_workspace_path("inet/.nedexclusions")).readlines()],
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/haugls/workspaces/test3/inet/.nedexclusions'
>>> 

[rhornig]

Hmmm. .nedexclusions file should be optional, so this is an issue. To work aorund just touch .nedexclusions to generate an empty file.

Besides, the following packages are used by the inet module: dask distributed py4j ipython optimparallel sewar. We should properly document this and at least create a proper python/requirements.txt file.

@levy I'm not sure why the debugger's init script wants to import the inet module at all. This prevents debugging if the above modules are not present in the actual python virtual environment.

[haug-den-lucas]

Hmmm. .nedexclusions file should be optional, so this is an issue. To work aorund just touch .nedexclusions to generate an empty file.

Even when the file is present inet tries to load it from [workspace]/inet/.nedexclusions even though the automatic install installs inet in [workspace]/inet4.5

I created a new workspace and pulled the inet 4.5.2 tag manually into [workspace]/inet
Additionally i installed all required python modules by inet. Now I am able to run "import inet" in the python console.
But only if I run source setenv before.
Launching the Python console in omnet even if I launched omnet from a shell which has also executed source setenv for inet it still doesn't work

Also for debugging i still get the same error.

My Ubuntu 24.04 VM is still setting up.
I'll give an update on this later

[haug-den-lucas]

More detailed:
In my bash i execute source inet/setenv and then omnetpp

Within omnet I can rightclick the inet folder and click on "Show in local terminal" i can do the following:

:~/workspaces/test4/inet$ python3
Python 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inet
No module named 'py4j': optional inet.common.ide package will not work
>>> 

However, if I click on the "Open console" in the bottom part and then on "PyDev console" i get the following error:

import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))

/usr/bin/python3 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0]
0.03s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
import inet
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/haugls/omnetpp-6.0.3/ide/plugins/org.python.pydev.core_11.0.3.202310301107/pysrc/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'inet'

So for some reason it feels like I am in the correct environment in one case but I'm not on the other.

Also the PYTHONPATH is different in both cases.
First case:

>>> print(os.getenv("PYTHONPATH"))
/home/haugls/workspaces/test4/inet/python:::/home/haugls/omnetpp-6.0.3/python

Second case:

print(os.getenv("PYTHONPATH"))
/home/haugls/omnetpp-6.0.3/ide/plugins/org.python.pydev.core_11.0.3.202310301107/pysrc/pydev_sitecustomize:/home/haugls/omnetpp-6.0.3/ide/plugins/org.omnetpp.scave.pychart_6.0.3.240223-17fcae5ef3/python:/home/haugls/omnetpp-6.0.3/python:/usr/lib/python3.12:/usr/lib/python3.12/lib-dynload:/home/haugls/.local/lib/python3.12/site-packages:/usr/local/lib/python3.12/dist-packages:/usr/lib/python3/dist-packages

[haug-den-lucas]

So I was able to recreate this issue in a clean Ubuntu 24.04 VM. I can export the VM and provide the VM image if that helps.

Steps to reproduce:
1: Install Ubuntu 24.04 with the default settings.

2: Install all packages as stated in the OMNeT++ installation guide:

sudo apt-get install build-essential clang lld gdb bison flex perl python3 python3-pip qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools libqt5opengl5-dev libxml2-dev zlib1g-dev doxygen graphviz libwebkit2gtk-4.0-37 xdg-utils

This will fail as it libwebkit2gtk-4.0-37 is not found, so I installed everything else

3: Install python modules:

$ python3 -m pip install --user --upgrade numpy pandas matplotlib scipy seaborn posix_ipc

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

So instead install them with apt-get:

sudo apt-get install python3-numpy python3-pandas python3-matplotlib python3-scipy python3-seaborn

posix_ipc does not exist here so in installed that using

pip install --break-system-packages posix_ipc

4: Downlad and setup omnetpp:

source setenv
./configure WITH_OSG=no
make

5: Start omnetpp from console

omnetpp

6: Create an empty workspace (~/workspaces/test in my case)

7: Install inet git clone --branch v4.5.2 https://github.com/inet-framework/inet (needed to install git here)

8: Import the inet folder in OMNeT++ in your workspace

9: Try to debug the timeawareshaper showcase

[haug-den-lucas]

And here the .log file from [workspace]/.metadata/.log:

omnetpp.log

[rhornig]

Normally the debugger should not depend on optional packages, but it seems it is not the case. It is definitely a bug. As a workaround try to install the following packages in the virtual env: dask distributed py4j ipython optimparallel sewar