[executorch][flat_tensor] DataMap implementation #7900

lucylq · 2025-01-23T19:53:22Z

Stack from ghstack (oldest at bottom):

DataMap implementation that

Loads a flat_tensor file
Populates a map with {fqn: tensor} and {fqn: TensorLayout}.
Makes tensor information available via the named_data_map.h interface.

For now, DataMap doesn't store the DataLoader.

If/when tensors are in their own segments, DataMap should also store a DataLoader.

Differential Revision: D67064580

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

pytorch-bot · 2025-01-23T19:53:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7900

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

ROCM Infra failures during checkout of PyTorch

✅ No Failures

As of commit e1c396f with merge base 821a2fe ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-01-23T19:53:37Z

This pull request was exported from Phabricator. Differential Revision: D67064580

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) ghstack-source-id: 262756077 Pull Request resolved: #7900

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-01-23T20:06:17Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) ghstack-source-id: 262759344

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-01-23T22:58:36Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. ghstack-source-id: 262759344 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-01-24T18:24:28Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. ghstack-source-id: 262941711 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Populates a map with {key: tensor} and {key: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. ghstack-source-id: 262957949 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

facebook-github-bot · 2025-01-24T19:26:55Z

This pull request was exported from Phabricator. Differential Revision: D67064580

JacobSzwejbka · 2025-01-24T19:53:42Z

extension/flat_tensor/named_data_map/data_map.h

+  executorch::runtime::FreeableBuffer _flat_tensor_data;
+
+  // Map of name to {segment index, offset, TensorLayout}.
+  std::unordered_map<


Im not super opinionated on this, but one thing to consider by having this state as part of the class is you force it to keep dangling around after all the method inits are done which is wasteful. I'm not really super sold on this being a huge latency win either rather then just constructing these things as they are needed.

Ok, this sounds good. Esp. for training case where there aren't many tensors and a linear search (or logn if tensors are sorted) isn't time consuming. If there are latency issues later down the track we can create an impl with a map.

JacobSzwejbka · 2025-01-24T19:55:57Z

extension/flat_tensor/named_data_map/data_map.cpp

+  int segment_size = s_data_segment->Get(0)->size();
+  assert(segment_size <= segment_data_size);
+
+  Result<FreeableBuffer> _data_ro = loader->load(


Ok I was trying to figure out how I wanted to handle this for mutable state. Here is what Ive come up with.

Lazily load segments, but internally keep a list of all the loaded segments. If the first time you need to load a segment is through "get_data" then put it in the preserve lis. If the first time you need to load a segment is through load_into then instead you just call data_loader load into which doesnt keep it hanging around.

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-01-31T20:11:06Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Populates a map with {key: tensor} and {key: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. ghstack-source-id: 264110292 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-02-04T00:18:15Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. ghstack-source-id: 264501132 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-02-05T00:32:00Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. TODO: in a later diff, update the ET runtime to hold onto the FreeableBuffers returned by the NDM. Then, the NDM will not persist the segment. T214294528 ghstack-source-id: 264501132 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-02-05T17:52:02Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. TODO: in a later diff, update the ET runtime to hold onto the FreeableBuffers returned by the NDM. Then, the NDM will not persist the segment. T214294528 ghstack-source-id: 264850001 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-02-05T19:20:32Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. TODO: in a later diff, update the ET runtime to hold onto the FreeableBuffers returned by the NDM. Then, the NDM will not persist the segment. T214294528 ghstack-source-id: 264871691 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-02-05T21:42:30Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. TODO: in a later diff, update the ET runtime to hold onto the FreeableBuffers returned by the NDM. Then, the NDM will not persist the segment. T214294528 ghstack-source-id: 264905837 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

DataMap implementation that * Loads a flat_tensor file * Populates a map with {fqn: tensor} and {fqn: TensorLayout}. * Makes tensor information available via the named_data_map.h interface. For now, DataMap doesn't store the DataLoader. - If/when tensors are in their own segments, DataMap should also store a DataLoader. Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) [ghstack-poisoned]

facebook-github-bot · 2025-02-05T22:25:18Z

This pull request was exported from Phabricator. Differential Revision: D67064580

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. TODO: in a later diff, update the ET runtime to hold onto the FreeableBuffers returned by the NDM. Then, the NDM will not persist the segment. T214294528 ghstack-source-id: 264916795 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/)

Pull Request resolved: #7900 DataMap implementation that * Loads a flat_tensor file * Makes tensor information available via the named_data_map.h interface. TODO: in a later diff, update the ET runtime to hold onto the FreeableBuffers returned by the NDM. Then, the NDM will not persist the segment. T214294528 ghstack-source-id: 264916795 Differential Revision: [D67064580](https://our.internmc.facebook.com/intern/diff/D67064580/) --------- Co-authored-by: lucylq <[email protected]>

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 23, 2025

lucylq mentioned this pull request Jan 23, 2025

[executorch][core] NamedDataMap interface #7763

Merged

facebook-github-bot added the fb-exported label Jan 23, 2025

lucylq added the release notes: api Changes to public facing apis (any interfaces, pybinded runtime methods, etc.) label Jan 23, 2025

JacobSzwejbka reviewed Jan 24, 2025

View reviewed changes

This was referenced Jan 31, 2025

[executorch][flat_tensor] update flat tensor header test #8109

Merged

[executorch] Create target for named_data_map #8110

Merged

lucylq mentioned this pull request Feb 5, 2025

[executorch][runtime] Add NamedDataMap to method load #8200

Merged

lucylq mentioned this pull request Feb 5, 2025

[executorch] Suppress wdeprecated warnings on tensor_layout.h and named_data_map.h #8201

Merged

dbort approved these changes Feb 5, 2025

View reviewed changes

facebook-github-bot merged commit eb94973 into gh/lucylq/32/base Feb 6, 2025
46 of 47 checks passed

facebook-github-bot deleted the gh/lucylq/32/head branch February 6, 2025 01:53

facebook-github-bot had a problem deploying to cherry-pick-bot February 6, 2025 01:53 — with GitHub Actions Failure

kirklandsign mentioned this pull request Feb 6, 2025

[executorch][flat_tensor] DataMap implementation #8280

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[executorch][flat_tensor] DataMap implementation #7900

[executorch][flat_tensor] DataMap implementation #7900

lucylq commented Jan 23, 2025 •

edited

Loading

pytorch-bot bot commented Jan 23, 2025 •

edited

Loading

facebook-github-bot commented Jan 23, 2025

facebook-github-bot commented Jan 23, 2025

facebook-github-bot commented Jan 23, 2025

facebook-github-bot commented Jan 24, 2025

facebook-github-bot commented Jan 24, 2025

JacobSzwejbka Jan 24, 2025

lucylq Jan 24, 2025

JacobSzwejbka Jan 24, 2025

facebook-github-bot commented Jan 31, 2025

facebook-github-bot commented Feb 4, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

[executorch][flat_tensor] DataMap implementation #7900

[executorch][flat_tensor] DataMap implementation #7900

Conversation

lucylq commented Jan 23, 2025 • edited Loading

pytorch-bot bot commented Jan 23, 2025 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7900

❗ 1 Active SEVs

✅ No Failures

facebook-github-bot commented Jan 23, 2025

facebook-github-bot commented Jan 23, 2025

facebook-github-bot commented Jan 23, 2025

facebook-github-bot commented Jan 24, 2025

facebook-github-bot commented Jan 24, 2025

JacobSzwejbka Jan 24, 2025

Choose a reason for hiding this comment

lucylq Jan 24, 2025

Choose a reason for hiding this comment

JacobSzwejbka Jan 24, 2025

Choose a reason for hiding this comment

facebook-github-bot commented Jan 31, 2025

facebook-github-bot commented Feb 4, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

facebook-github-bot commented Feb 5, 2025

lucylq commented Jan 23, 2025 •

edited

Loading

pytorch-bot bot commented Jan 23, 2025 •

edited

Loading