feat(autoware_path_optimizer): reintroducing acados MPT along with changes to linking to acados

[PanConChicharron]

Description

Follow-up to #11479 to propagate changes downstream.

Related links

Parent Issue:

Ticket

How was this PR tested?

Evalautor build succesful: https://evaluation.tier4.jp/evaluation/reports/8a01ff9b-5732-5a87-abd5-931e6d93c6a7?project_id=autoware_dev (with pilot-auto PR: https://github.com/tier4/pilot-auto/pull/2389)

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[soblin]

Hope this fix solves the issue !

[codecov]

Codecov Report

❌ Patch coverage is 21.03321% with 214 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.19%. Comparing base (7302e8c) to head (6a4ee76).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
...ning/autoware_path_optimizer/src/mpt_optimizer.cpp	17.64%	152 Missing and 2 partials ⚠️
..._optimizer/src/acados_mpc/src/acados_interface.cpp	28.91%	58 Missing and 1 partial ⚠️
...imizer/src/acados_mpc/include/acados_interface.hpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #12300      +/-   ##
==========================================
+ Coverage   18.48%   19.19%   +0.71%     
==========================================
  Files        1849     1879      +30     
  Lines      127898   129128    +1230     
  Branches    45490    47955    +2465     
==========================================
+ Hits        23638    24783    +1145     
- Misses      84639    85621     +982     
+ Partials    19621    18724     -897     

Flag	Coverage Δ		*Carryforward flag
daily	20.86% <28.57%> (ø)		Carriedforward from 333302a
daily-cuda	18.37% <28.57%> (ø)		Carriedforward from 333302a
differential	4.17% <21.03%> (?)
total-cuda	18.45% <ø> (-0.01%)	⬇️	Carriedforward from 333302a

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[paulsohn]

Not directly related issue, but since you opened a follow-up, I would like to have one more change before we need another PR:

This is the clean Jazzy CI environment from my personal research purpose, unrelated to AWF. Building this package (with symlink install) overwrites the existing placeholder file, making the repository status dirty. My local build reports the same.

Why did we decided to put the placeholder file at the first place? I recall it was about clang-tidy check, but is it really necessary? I couldn't track on the relevant discussion in the previous PR.

As a solution I want you to investigate on whether either of them is feasible.

• remove the placeholder file, or
• put placeholder file in another directory and let cmake copy when build

[PanConChicharron]

Not directly related issue, but since you opened a follow-up, I would like to have one more change before we need another PR:

This is the clean Jazzy CI environment from my personal research purpose, unrelated to AWF. Building this package (with symlink install) overwrites the existing placeholder file, making the repository status dirty. My local build reports the same.

Why did we decided to put the placeholder file at the first place? I recall it was about clang-tidy check, but is it really necessary? I couldn't track on the relevant discussion in the previous PR.

As a solution I want you to investigate on whether either of them is feasible.

• remove the placeholder file, or
• put placeholder file in another directory and let cmake copy when build

Thank you for the comment!

Unfortunately I had tried many things and including a placeholder was the only way to get clang-tidy to work.
Clang-tidy runs before the build, so even adding a copy command like you had requested would not fix clang-tidy either.

[paulsohn]

@PanConChicharron I would say that the fix is essential to prevent accidental commit updating placeholder.
Which clang-tidy are you referring to? On your local, or in Github CI?

[PanConChicharron]

@PanConChicharron I would say that the fix is essential to prevent accidental commit updating placeholder. Which clang-tidy are you referring to? On your local, or in Github CI?

You are right, and I should definitely figure out an alternative.

I meant Github CI, and unfortunately it seems to run before the file is generated, which causes clang-tidy to fail.
I'm not sure if we can change the workflow so that the build occurs first.

CC: @mitsudome-r @xmfcx

[PanConChicharron]

Committed the generated files as a last resort.

[xmfcx]

Suggested change

-I /home/arjunram/Workspace/autoware/src/universe/autoware_universe/planning/autoware_path_optimizer/src/acados_mpc/c_generated_code \

This has hardcoded user path. We cannot include this kind of auto-generated code in the src folder.

Also because of pre-commit, after each colcon build, we have 15 changes due to code layout refresh. Have to run pre-commit again to reduce them to 1 change. (which is the makefile)

Is it possible to install these auto-generated codes into the install folder and refer to them from there?

[PanConChicharron]

This is the main issue to be dealt with now.

Unfortunately, it's not clear how to proceed. We need autoware_universe/planning/autoware_path_optimizer/src/acados_mpc/c_generated_code/acados_solver_curvilinear_bicycle_model_spatial.h because clang-tidy will fail without it.

Adding a placeholder yields modifications to that file when the code is actually generated, which makes it harder on developers.

And adding the generated code explictly, brings up the above issue (worth noting that changing the code as you had suggested will introduce diffs in every user's workspace after code-generation).

[xmfcx]

what about putting auto-generated code into the install folder?

[PanConChicharron]

Hmm... That could be done, but then I would need some really messy relative directories in the code to point to the install folder.

Instead, I could 'git restore' the code at the end of the CMakeLists.

[xmfcx]

git commands like git restore is not acceptable in cmakelists file. The compile-time generated code should not be present in the source folder.

https://cmake.org/cmake/help/book/mastering-cmake/chapter/Getting%20Started.html#directory-structure

CMake will not write any files to the source directory, only to the binary directory.

Out-of-source builds, where the source and binary directories are different, are strongly encouraged. In-source builds where the source and binary directories are the same are supported but should be avoided if possible. Out-of-source builds make it very easy to maintain a clean source tree and allow quick removal of all of the files generated by a build. Having the build tree differ from the source tree also makes it easy to support having multiple builds of a single source tree.

https://crascit.com/2017/04/18/generated-sources-in-cmake-builds/
This is also worth reading.

Generating source code into the source folder is the root cause of all these issues, let's fix that first.

---

Following post is ai genereated 🤖 but worth trying.

Code Generation Should Target the Binary Directory, Not the Source Tree 🤖

Hey @PanConChicharron — I took a closer look at the CMake changes and I think the current approach introduces a number of issues that will bite contributors and CI down the line. I want to lay out the problems clearly and then propose a concrete alternative that solves the original clang-tidy issue without the trade-offs.

---

Problems with the Current Approach

1. The git checkout POST_BUILD creates an infinite rebuild loop

The add_custom_command(OUTPUT ...) mechanism works on timestamps — CMake checks whether the output files are older than their dependencies, and if so, re-runs the command. By restoring the placeholder header via git checkout after every build, the output is always "stale" relative to the Python generator script. This means the expensive Python codegen will run on every single build, even when nothing has changed. Incremental builds are effectively broken.

# This guarantees the codegen reruns every time:
add_custom_command(TARGET acados_interface POST_BUILD
  COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} checkout -- c_generated_code/acados_solver_${MODEL}.h
  COMMENT "Restore placeholder so git status is clean"
)

2. Generating into the source tree is a CMake anti-pattern

Writing generated files into CMAKE_CURRENT_SOURCE_DIR causes problems in multiple real-world scenarios:

• Read-only source directories — packaging systems (Debian, Nix, Flatpak) and some CI configurations mount the source tree read-only. The build will fail outright.
• Parallel configuration builds — if you build Debug and Release side-by-side from the same source tree, both will try to write the same generated files, causing race conditions.
• Dirty working trees — every build dirties git status, which breaks workflows that check for clean trees, and is confusing for developers.

The idiomatic CMake approach is to always generate into CMAKE_CURRENT_BINARY_DIR.

3. The build depends on git being present at build time

COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} checkout -- c_generated_code/acados_solver_${MODEL}.h

Anyone building from a source tarball, a shallow archive, a vendored copy, or any environment without git installed will get a hard build failure. Build systems should never depend on VCS tooling.

4. The lint exclusion globs are a workaround, not a fix

Because generated code lives in the source tree, we need this in the top-level CMakeLists to prevent ament lint from checking it:

list(APPEND AMENT_LINT_AUTO_FILE_EXCLUDE
  "src/acados_mpc/c_generated_code/**"
  "src/acados_mpc/c_generated_code/**/*"
  "src/acados_mpc/c_generated_code/**/*.*"
)

If the generated code doesn't live in the source tree, this entire block becomes unnecessary.

5. It doesn't fully solve the clang-tidy problem

The original motivation was making acados_solver_curvilinear_bicycle_model_spatial.h available so clang-tidy doesn't fail. But clang-tidy typically runs on the source tree without a full build. After the git checkout restores the placeholder, the placeholder is what clang-tidy sees. If the placeholder doesn't have the full type declarations and prototypes, clang-tidy will still complain — we're paying all this complexity and not even fixing the root cause.

---

Proposed Solution

The fix is straightforward: generate into the binary directory and keep a minimal stub header checked into the source tree for static analysis tools.

Step 1: Generate into CMAKE_CURRENT_BINARY_DIR

In src/acados_mpc/CMakeLists.txt:

set(GEN_DIR "${CMAKE_CURRENT_BINARY_DIR}/c_generated_code")

Update the custom command's working directory so the Python generator writes to the binary tree:

add_custom_command(
  OUTPUT ${GENERATED_MPC_FILES}
  COMMAND ${CMAKE_COMMAND} -E env ${_env_vars}
    ${ACADOS_SOURCE_DIR}/.venv/bin/python3
    ${CMAKE_CURRENT_SOURCE_DIR}/generators/path_tracking_mpc_spatial_with_body_points.py
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/generators/path_tracking_mpc_spatial_with_body_points.py
  COMMENT "Generating acados MPC solver code"
)

Note: If the Python generator hardcodes its output path relative to its CWD, it may need a small patch to accept an output directory argument, or you can symlink the required inputs into the binary dir.

Step 2: Update include directories (binary dir before source dir)

target_include_directories(acados_interface PUBLIC
  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
  $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>   # generated headers found here at build time
  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>    # stub header found here for clang-tidy
  $<INSTALL_INTERFACE:include>
  ${ACADOS_SOURCE_DIR}/include
  ${ACADOS_SOURCE_DIR}/include/acados
  ${ACADOS_SOURCE_DIR}/include/acados_c
  ${ACADOS_SOURCE_DIR}/include/blasfeo/include
  ${ACADOS_SOURCE_DIR}/include/hpipm/include
)

The ordering matters: at build time, the real generated header in the binary dir shadows the stub. When clang-tidy runs outside a full build, it falls back to the stub in the source tree.

Step 3: Keep a minimal stub header for static analysis

Check in c_generated_code/acados_solver_curvilinear_bicycle_model_spatial.h with only the type declarations and function prototypes that clang-tidy (and IDE tooling) need to parse acados_interface.cpp:

// Stub header for static analysis (clang-tidy) and IDE support.
// The real header is generated at build time into CMAKE_CURRENT_BINARY_DIR.
// Do not add implementation details here — keep it minimal.
#ifndef ACADOS_SOLVER_CURVILINEAR_BICYCLE_MODEL_SPATIAL_H_
#define ACADOS_SOLVER_CURVILINEAR_BICYCLE_MODEL_SPATIAL_H_

// ... only the typedefs, struct declarations, and function prototypes
//     that acados_interface.cpp actually references ...

#endif

This is stable, hand-written, passes lint and clang-tidy, and never gets overwritten by the build.

Step 4: Delete the git checkout POST_BUILD hack

Since the source tree is never modified, there's nothing to restore:

# DELETE — no longer needed
# add_custom_command(TARGET acados_interface POST_BUILD
#   COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} checkout -- ...
# )

Step 5: Remove the lint exclusion globs from the top-level CMakeLists

# DELETE — generated code is no longer in the source tree
# list(APPEND AMENT_LINT_AUTO_FILE_EXCLUDE
#   "src/acados_mpc/c_generated_code/**"
#   ...
# )

Step 6: Fix install rules to reference the binary dir

In the top-level CMakeLists.txt, generated headers now live under the binary tree:

set(GEN_DIR "${CMAKE_CURRENT_BINARY_DIR}/src/acados_mpc/c_generated_code")
install(FILES
  ${GEN_DIR}/acados_solver_${MODEL}.h
  ${GEN_DIR}/${MODEL}_model/${MODEL}_model.h
  ${GEN_DIR}/${MODEL}_constraints/${MODEL}_constraints.h
  DESTINATION include/autoware/path_optimizer/c_generated_code
  OPTIONAL
)

---

Summary

Concern	Current approach	Proposed approach
Incremental builds	Broken — codegen reruns every time	Works correctly via timestamp tracking
Source tree cleanliness	Requires git checkout hack	Source tree is never touched
Read-only source dirs	Build fails	Works out of the box
git dependency at build time	Required	Not needed
clang-tidy support	Placeholder may not satisfy analysis	Minimal stub with real prototypes
Lint exclusions	Glob workarounds needed	Not needed
Build from tarball	Fails	Works

The entire complexity in the current implementation — in-tree generation, git restore, lint exclusion globs — stems from one decision: generating into the source tree. Moving that to the binary dir eliminates all of it, and the add_custom_command / add_custom_target / add_dependencies chain you already have is exactly right — it's just pointed at the wrong directory.

[PanConChicharron]

git commands like git restore is not acceptable in cmakelists file. The compile-time generated code should not be present in the source folder.

https://cmake.org/cmake/help/book/mastering-cmake/chapter/Getting%20Started.html#directory-structure

CMake will not write any files to the source directory, only to the binary directory.

Out-of-source builds, where the source and binary directories are different, are strongly encouraged. In-source builds where the source and binary directories are the same are supported but should be avoided if possible. Out-of-source builds make it very easy to maintain a clean source tree and allow quick removal of all of the files generated by a build. Having the build tree differ from the source tree also makes it easy to support having multiple builds of a single source tree.

https://crascit.com/2017/04/18/generated-sources-in-cmake-builds/ This is also worth reading.

Generating source code into the source folder is the root cause of all these issues, let's fix that first.

Following post is ai genereated 🤖 but worth trying.

Code Generation Should Target the Binary Directory, Not the Source Tree 🤖

Hey @PanConChicharron — I took a closer look at the CMake changes and I think the current approach introduces a number of issues that will bite contributors and CI down the line. I want to lay out the problems clearly and then propose a concrete alternative that solves the original clang-tidy issue without the trade-offs.

Problems with the Current Approach

1. The git checkout POST_BUILD creates an infinite rebuild loop

The add_custom_command(OUTPUT ...) mechanism works on timestamps — CMake checks whether the output files are older than their dependencies, and if so, re-runs the command. By restoring the placeholder header via git checkout after every build, the output is always "stale" relative to the Python generator script. This means the expensive Python codegen will run on every single build, even when nothing has changed. Incremental builds are effectively broken.

# This guarantees the codegen reruns every time:
add_custom_command(TARGET acados_interface POST_BUILD
  COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} checkout -- c_generated_code/acados_solver_${MODEL}.h
  COMMENT "Restore placeholder so git status is clean"
)

2. Generating into the source tree is a CMake anti-pattern

Writing generated files into CMAKE_CURRENT_SOURCE_DIR causes problems in multiple real-world scenarios:

• Read-only source directories — packaging systems (Debian, Nix, Flatpak) and some CI configurations mount the source tree read-only. The build will fail outright.
• Parallel configuration builds — if you build Debug and Release side-by-side from the same source tree, both will try to write the same generated files, causing race conditions.
• Dirty working trees — every build dirties git status, which breaks workflows that check for clean trees, and is confusing for developers.

The idiomatic CMake approach is to always generate into CMAKE_CURRENT_BINARY_DIR.

3. The build depends on git being present at build time

COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} checkout -- c_generated_code/acados_solver_${MODEL}.h

Anyone building from a source tarball, a shallow archive, a vendored copy, or any environment without git installed will get a hard build failure. Build systems should never depend on VCS tooling.

4. The lint exclusion globs are a workaround, not a fix

Because generated code lives in the source tree, we need this in the top-level CMakeLists to prevent ament lint from checking it:

list(APPEND AMENT_LINT_AUTO_FILE_EXCLUDE
  "src/acados_mpc/c_generated_code/**"
  "src/acados_mpc/c_generated_code/**/*"
  "src/acados_mpc/c_generated_code/**/*.*"
)

If the generated code doesn't live in the source tree, this entire block becomes unnecessary.

5. It doesn't fully solve the clang-tidy problem

The original motivation was making acados_solver_curvilinear_bicycle_model_spatial.h available so clang-tidy doesn't fail. But clang-tidy typically runs on the source tree without a full build. After the git checkout restores the placeholder, the placeholder is what clang-tidy sees. If the placeholder doesn't have the full type declarations and prototypes, clang-tidy will still complain — we're paying all this complexity and not even fixing the root cause.

Proposed Solution

The fix is straightforward: generate into the binary directory and keep a minimal stub header checked into the source tree for static analysis tools.

Step 1: Generate into CMAKE_CURRENT_BINARY_DIR

In src/acados_mpc/CMakeLists.txt:

set(GEN_DIR "${CMAKE_CURRENT_BINARY_DIR}/c_generated_code")

Update the custom command's working directory so the Python generator writes to the binary tree:

add_custom_command(
  OUTPUT ${GENERATED_MPC_FILES}
  COMMAND ${CMAKE_COMMAND} -E env ${_env_vars}
    ${ACADOS_SOURCE_DIR}/.venv/bin/python3
    ${CMAKE_CURRENT_SOURCE_DIR}/generators/path_tracking_mpc_spatial_with_body_points.py
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/generators/path_tracking_mpc_spatial_with_body_points.py
  COMMENT "Generating acados MPC solver code"
)

Note: If the Python generator hardcodes its output path relative to its CWD, it may need a small patch to accept an output directory argument, or you can symlink the required inputs into the binary dir.

Step 2: Update include directories (binary dir before source dir)

target_include_directories(acados_interface PUBLIC
  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
  $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>   # generated headers found here at build time
  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>    # stub header found here for clang-tidy
  $<INSTALL_INTERFACE:include>
  ${ACADOS_SOURCE_DIR}/include
  ${ACADOS_SOURCE_DIR}/include/acados
  ${ACADOS_SOURCE_DIR}/include/acados_c
  ${ACADOS_SOURCE_DIR}/include/blasfeo/include
  ${ACADOS_SOURCE_DIR}/include/hpipm/include
)

The ordering matters: at build time, the real generated header in the binary dir shadows the stub. When clang-tidy runs outside a full build, it falls back to the stub in the source tree.

Step 3: Keep a minimal stub header for static analysis

Check in c_generated_code/acados_solver_curvilinear_bicycle_model_spatial.h with only the type declarations and function prototypes that clang-tidy (and IDE tooling) need to parse acados_interface.cpp:

// Stub header for static analysis (clang-tidy) and IDE support.
// The real header is generated at build time into CMAKE_CURRENT_BINARY_DIR.
// Do not add implementation details here — keep it minimal.
#ifndef ACADOS_SOLVER_CURVILINEAR_BICYCLE_MODEL_SPATIAL_H_
#define ACADOS_SOLVER_CURVILINEAR_BICYCLE_MODEL_SPATIAL_H_

// ... only the typedefs, struct declarations, and function prototypes
//     that acados_interface.cpp actually references ...

#endif

This is stable, hand-written, passes lint and clang-tidy, and never gets overwritten by the build.

Step 4: Delete the git checkout POST_BUILD hack

Since the source tree is never modified, there's nothing to restore:

# DELETE — no longer needed
# add_custom_command(TARGET acados_interface POST_BUILD
#   COMMAND git -C ${CMAKE_CURRENT_SOURCE_DIR} checkout -- ...
# )

Step 5: Remove the lint exclusion globs from the top-level CMakeLists

# DELETE — generated code is no longer in the source tree
# list(APPEND AMENT_LINT_AUTO_FILE_EXCLUDE
#   "src/acados_mpc/c_generated_code/**"
#   ...
# )

Step 6: Fix install rules to reference the binary dir

In the top-level CMakeLists.txt, generated headers now live under the binary tree:

set(GEN_DIR "${CMAKE_CURRENT_BINARY_DIR}/src/acados_mpc/c_generated_code")
install(FILES
  ${GEN_DIR}/acados_solver_${MODEL}.h
  ${GEN_DIR}/${MODEL}_model/${MODEL}_model.h
  ${GEN_DIR}/${MODEL}_constraints/${MODEL}_constraints.h
  DESTINATION include/autoware/path_optimizer/c_generated_code
  OPTIONAL
)

Summary

Concern Current approach Proposed approach
Incremental builds Broken — codegen reruns every time Works correctly via timestamp tracking
Source tree cleanliness Requires git checkout hack Source tree is never touched
Read-only source dirs Build fails Works out of the box
git dependency at build time Required Not needed
clang-tidy support Placeholder may not satisfy analysis Minimal stub with real prototypes
Lint exclusions Glob workarounds needed Not needed
Build from tarball Fails Works
The entire complexity in the current implementation — in-tree generation, git restore, lint exclusion globs — stems from one decision: generating into the source tree. Moving that to the binary dir eliminates all of it, and the add_custom_command / add_custom_target / add_dependencies chain you already have is exactly right — it's just pointed at the wrong directory.

Fatih, thank you for the very detailed explanation!

I went ahead and made the changes as per your request. Just waiting for CI now.