Running Multiple Containers

[OliverHeilmann]

Hello! I am currently exploring running multiple simulators in parallel for some CICD pipeline testing. I have used the Multi-Vehicle-Simulation-With-Gazebo which has been useful in checking that my agents can connect to multiple assets/ vehicles. Note: this helped me find the correct shell script.

My issue with this is that I cannot run the simulation in a headless environment i.e. on a pipeline test. I was thinking about containerising the multi-vehicle simulation. Gut feel is that three individual docker images may actually be the simplest option but I don't know how to map the UDP:14540 port to other ports as you aren't using the traditional docker -p command for it (noting that this approach would have three instances of Gazebo running which isn't great either).

So my question is two fold:

1. Do you know how to run a mutli-vehicle headless simulation in a docker container? I can handle the mapping of ports 14540, 14541... 1454n using normal docker practices.
2. If this is not possible, how can I modify this docker image to allow for remapping port 14540 to another one as done with the normal docker approach?

[JonasVautherin]

For the first question, I don't think anything prevents you from running a multi-vehicle headless simulation with some changes to this image. I haven't looked into it though, it would require a couple changes.

For the second question, the ports are defined here: https://github.com/JonasVautherin/px4-gazebo-headless/blob/master/edit_rcS.bash#L59C35-L59C58. So you could probably change \$udp_offboard_port_local for the port you want.

[OliverHeilmann]

Thanks for the prompt response! I will look into modifying this image/ dockerfile to support multi-vehicle simulation. I think this is a more elegant solution over running multiple containers, with each running a simulator. Perhaps I will open up a pull request if I arrive to an elegant solution...

Regarding the \$udp_offboard_port_local, I'll give this a try but I am happy to close this issue now. Thanks!

[OliverHeilmann]

Hello again!

Thought I'd reopen this issue instead of making a new one so the discussion stays on one thread... Below are the changes I have made in my attempt to get the multi-vehicle simulation working in a docker container. I used this repo as a baseline and made changes to a couple of the scripts.

entrypoint.sh

Changing the HEADLESS=1 make ... line to the sitl_multiple_run.sh made sense, the issue here is that the shell script uses the pre-built binaries instead of a make command directly.

+ ./Tools/simulation/gazebo-classic/sitl_multiple_run.sh -s "${vehicle}:3" -w "${world}"
-  HEADLESS=1 make px4_sitl gazebo_${vehicle}__${world}

sitl_multiple_run.sh

Add headless option to the px4 referenced binaries to ensure it uses the headless mode instead.

+ HEADLESS=1 $build_path/bin/px4 -i $n -d "$build_path/etc" >out.log 2>err.log &
- $build_path/bin/px4 -i $n -d "$build_path/etc" >out.log 2>err.log &

Issue with REST Request

When I run the below console command, being sure to map the other UDP ports which will be used for other vehicles (following the documentation), I get part of the way there.

oliver@ubuntu:~/<DIR>$ docker run --name sitl -p 14541:14541 -p 14542:14542 --rm -it px4-sitl-headless:latest

From the screenshot below, you can see that all three vehicles are created in the docker container and the headless mode is in fact working. The two errors are relating to an Audio device (which perhaps is not an issue since audio is then disabled) and then the REST request. The programme then hangs on the libcurl print and doesn't do anything else.

I have however tested the connections to the vehicles in the docker containers and it seems to be working which is great however. Any idea on these issues? I suspect if you follow those two changes, you should be able to replicate my issue.

Thanks again for your help!

[JonasVautherin]

I think I wouldn't care about the "Audio will be disabled" part. IMO that could be a warning.

I don't know about the REST error, but the line just above says "Starting gazebo client". I believe it suggests that it is trying to run the frontend, which you don't want. So somehow something is not running headless with your setup.

[IvanC-Gambit]

Looks like there is an issue on PX4 about this as well PX4/PX4-Autopilot#22946; however, albeit a bit old, this fork sounds like it had something running: https://github.com/AngryMeenky/px4-gazebo-headless/tree/master

[JonasVautherin]

Has anyone been using that in the meantime?

[julianoes]

I'm now listening in. I need to simulate more than one drone.

[pushkarprasad007]

@JonasVautherin - as mentioned in link, for me - the requirement is to simulate two vehicle (different behavior - and also type - plane & vtol) at same time, using mavsdk-python code.

I'm inclined to believe if we build the docker image - with the separate ports - should work. Then why do you say this:

What you are trying to do above won't work, though.

Can you please elaborate?

Also, I'm not super confident on how to make this work (possible to help - pretty please):

In your case, maybe you would want to look into simulating two drones in the same px4-gazebo-headless instance (maybe using mavlink-router to separate the streams outside of the container if necessary).

[pushkarprasad007]

Is this what you meant @JonasVautherin

🔹 Dockerfile changes

Add mavlink-router and copy config:

RUN apt-get update && apt-get install -y mavlink-router

COPY main.conf /etc/mavlink-router/main.conf

🔹 /etc/mavlink-router/main.conf

[General]
TcpServerPort = 5760
ReportStats = true

# VTOL output (SYSID 1)
[UdpEndpoint vtol]
Mode = Normal
Address = 0.0.0.0
Port = 14580

# Plane output (SYSID 2)
[UdpEndpoint plane]
Mode = Normal
Address = 0.0.0.0
Port = 14581

🔹 entrypoint.sh

#!/bin/bash
set -e

# --- Start VTOL (SYSID 1, instance 0) ---
HEADLESS=1 PX4_SYS_AUTOSTART=4010 MAV_SYS_ID=1 \
    make px4_sitl gazebo_standard_vtol &

# --- Start Plane (SYSID 2, instance 1) ---
HEADLESS=1 PX4_SYS_AUTOSTART=1001 MAV_SYS_ID=2 PX4_INSTANCE=1 \
    make px4_sitl gazebo_plane &

# Start mavlink-router (wait a bit for PX4s to boot)
sleep 5
mavlink-routerd -c /etc/mavlink-router/main.conf

🔹 Build & Run

docker build -t px4-dual .
docker run --rm -it -p 14580:14580/udp -p 14581:14581/udp px4-dual

[JonasVautherin]

What you are trying to do above won't work, though.

Can you please elaborate?

What you were trying to do was exposing ports of the docker container. Which won't do anything, because the messages don't flow in that direction: it's the docker container that sends MAVLink messages out to the host, and not the other way around.

Is this what you meant

Not quite.

What I meant is that if you change px4-gazebo-headless to simulate multiple drones in the same instance, then the MAVLink messages will still go out of the container the same way (on 14550 and 14540). All the MAVLink messages of all the simulated drones will be sent over the same UDP stream, and each simulated drone will have a different MAVLink sysid.

So from the outside (e.g. in the host), if you have QGC listen on 14550, it will see two drones. If you need to demultiplex the MAVLink streams (i.e. split the streams from the simulated drones into different ports), for instance in order to feed each simulated drone to a separate MAVSDK instance, then on the host (i.e. outside the container) you can use mavlink-router to do that.

[IvanC-Gambit]

Has anyone been using that in the meantime?

I was able to modify the following repo to get multiple vehicles simulated (<10). Required some modifications. Interested in PR upstream to your repo? Still need more work to figure out how to get the port routing working correctly for >10 platforms

[pushkarprasad007]

Yes please @IvanC-Gambit !!