Skip to content

【Allocator】Add compact func in phi generate code#76430

Merged
liuruyan merged 6 commits intoPaddlePaddle:developfrom
liuruyan:api_gen_compact
Nov 19, 2025
Merged

【Allocator】Add compact func in phi generate code#76430
liuruyan merged 6 commits intoPaddlePaddle:developfrom
liuruyan:api_gen_compact

Conversation

@liuruyan
Copy link
Contributor

@liuruyan liuruyan commented Nov 16, 2025 •
edited
Loading

PR Category

Performance Optimization

PR Types

New features

Description

本PR新增碎片整理相关FLAGS，并在PHI算子库中加入碎片整理分支，具体如下：

  1. 新增以下四个FLAGS用来辅助碎片整理：
FLAGS_enable_compact_mem  // 是否开启碎片整理功能
FLAGS_max_reserved_threshold_in_gb // max_reserved 阈值，小于阈值不进行整理，默认70g
FLAGS_cur_allocated_threshold_in_gb // current_allocated 阈值，小于阈值不进行整理，默认55g
FLAGS_try_allocate // 是否开启TryAlloc功能，开启后碎片整理策略更加激进，对可能OOM情况进行一次尝试分配并判断是否整理
  1. 新增TryAllocBatch功能，可以输入Vector<request_size>，在当前Allocator状态下进行模拟分配，且不会影响原状态
  2. PHI自动生成代码新增碎片整理预测分支，当前预测逻辑如下：
   1. mem_max_reserved < max_reserved_threshold ==> dont need compact
   2. mem_cur_allocated < cur_allocated_threshold ==> dont need compact
   3. max_free_size > req_total_size ==> dont need compact
   5. large_N_free_size < req_total_size ==> need compact
   6. try_allocate result ==> need compact
   7. default ==> dont need compact

其中req_total_size为申请请求大小总和，max_free_size为Allocator最大空闲块，large_N_free_size为为Allocator最大N空闲块大小总和，N为申请请求个数

TODO：Custom api 暂时跳过了碎片整理，后续应该在动态图流程里执行inferShape
Pcard-67164

@paddle-bot
Copy link

paddle-bot bot commented Nov 16, 2025

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@liuruyan liuruyan changed the title add compact in phi generate code 【Allocator】Add compact func in phi generate code Nov 16, 2025
@codecov-commenter
Copy link

codecov-commenter commented Nov 16, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 92.48120% with 10 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@e375b1a). Learn more about missing BASE report.

Files with missing lines Patch % Lines
paddle/phi/api/lib/api_custom_impl.cc 62.50% 6 Missing ⚠️
paddle/phi/api/lib/api_gen_utils.cc 92.30% 4 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             develop   #76430   +/-   ##
==========================================
  Coverage           ?   92.48%           
==========================================
  Files              ?        8           
  Lines              ?      133           
  Branches           ?        0           
==========================================
  Hits               ?      123           
  Misses             ?       10           
  Partials           ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

zyfncg
zyfncg approved these changes Nov 19, 2025
View reviewed changes
paddle/common/flags.cc
/**
* Allocator Compact related FLAG
* Name: FLAGS_enable_compact_mem
* Since Version: 3.2.2
Copy link
Contributor

@zyfncg zyfncg Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.3

Copy link
Contributor Author

@liuruyan liuruyan Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx，下一个PR中进行修改

paddle/fluid/pybind/eager_functions.cc
PyObject* kwargs) {
EAGER_TRY
FLAGS_tensor_operants_mode = "phi";
bool old_flag = FLAGS_enable_compact_mem;
Copy link
Contributor

@zyfncg zyfncg Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

old 这个命名有点糙

Copy link
Contributor Author

@liuruyan liuruyan Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx，下一个PR中修改下命名

paddle/phi/core/memory/allocation/virtual_memory_auto_growth_best_fit_allocator.cc
Comment on lines +483 to +486
VLOG(1) << "Do Memory Compact Large Pool Manual";
size_t compact_free_size = large_allocator->Compact(place);
VLOG(1) << "Memory Compact Large Pool Manual Finish Compact size: "
<< compact_free_size;
Copy link
Contributor

@zyfncg zyfncg Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

日志级别有点低

Copy link
Contributor Author

@liuruyan liuruyan Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里日志级别1应该是合理的，这个日志是函数只有在触发整理时才会打印的必要提示信息

test/cpp/phi/memory/gen_compact_test.cc
@@ -0,0 +1,258 @@
/* Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
Copy link
Contributor

@zyfncg zyfncg Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2025

Copy link
Contributor Author

@liuruyan liuruyan Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx，下一个PR中修改下命名

@liuruyan liuruyan merged commit 4b2810d into PaddlePaddle:develop Nov 19, 2025
158 of 181 checks passed
LittleHeroZZZX pushed a commit to LittleHeroZZZX/Paddle that referenced this pull request Nov 19, 2025
@liuruyan @LittleHeroZZZX
【Allocator】Add compact func in phi generate code (PaddlePaddle#76430)
96b924f 
* add compact in phi generate code

* add try alloc

* fix custom api

* remove std::cout
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zyfncg zyfncg zyfncg approved these changes

@zhangting2020 zhangting2020 zhangting2020 approved these changes

@zhangbo9674 zhangbo9674 zhangbo9674 approved these changes

Assignees

No one assigned

Labels

skip-ci: approval

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@liuruyan @codecov-commenter @zyfncg @zhangting2020 @zhangbo9674 @swgu98