-
Notifications
You must be signed in to change notification settings - Fork 5.9k
Fluid new API scaffolding #10313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fluid new API scaffolding #10313
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| __all__ = ['Inferencer', ] | ||
|
|
||
|
|
||
| class Inferencer(object): | ||
| def __init__(self, network_func, params, place=None): | ||
| # 1. we need to generate a framework.Program by calling | ||
| # network_func. Reference: fluid.program_guard in test_word2vec.py | ||
|
|
||
| # 2. move the default_main_program to self.program. | ||
|
|
||
| # 3. run the default_startup program. | ||
| self.params = params | ||
| self.place = place | ||
|
|
||
| def infer(self, inputs): | ||
| # run self.program | ||
| pass |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| from . import core | ||
|
|
||
| __all__ = ['Params', ] | ||
|
|
||
|
|
||
| class Params(object): | ||
| def __init__(self, path=None): | ||
| self.scope = core.Scope() | ||
|
|
||
| if path: | ||
| self._load(path) | ||
|
|
||
| def _load(self, path): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why load is defined as a private method but save it public?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You are right, this is not optimal. We will remove this class: #10313 (comment)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed params.py in #10354 |
||
| # reference: load_persistables in io.py | ||
| pass | ||
|
|
||
| def save(self, path): | ||
| # reference: save_persistables in io.py | ||
| pass | ||
|
|
||
| def add_params(self, scope): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't quite understand here -- the method name is add params, but the method comment says that it takes keys from the scope. Does it really add something?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We plan to remove this class, and add a
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For the We could use
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jetfuel maybe we can implement
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed params.py in #10354
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @helinwang Yes, it should be in the C++. My approach is to use the LocalVarNames and check one by one.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jetfuel Thanks! Every var in the void Scope::merge(Scope* scope) const {
std::vector<std::string> names = scope.LocalVarNames;
for (auto name_it = names.begin(); name_it != names.end(); name_it ++) {
vars_[*name_it] = scope.Var(*name_it)
}
}Sorry the spec was different from what we discussed yesterday (yesterday's was to not replace if already exists). |
||
| # take the keys from the scope, | ||
| # if not already exists in self.scope, | ||
| # add the key and value into self.scope. | ||
| pass | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| __all__ = [ | ||
| 'Event', | ||
| 'Trainer', | ||
| ] | ||
|
|
||
|
|
||
| class Event(object): | ||
| BEGIN_EPOCH = 0 | ||
| END_EPOCH = 1 | ||
| BEGIN_STEP = 2 | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the "step" here "iteration"?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think step is more commonly used in deep learning. In the same time iteration is more often used in programming languages. Since here is the event for training, I think the user would be more familiar with "step". |
||
| END_STEP = 3 | ||
|
|
||
| def __init__(self): | ||
| self.step = 0 | ||
| self.epoch = 0 | ||
| self.type = Event.BEGIN_EPOCH | ||
|
|
||
|
|
||
| class Trainer(object): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this a base class, which implies that we need derived classes like
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The current plan is "a plain class and both single-node and multi-node training logics should be implemented in it", the logic is controlled by environment variable. Here is an example: #10316 |
||
| def __init__(self, network_func, optimizer, params=None, place=None): | ||
| # 1. we need to generate a framework.Program by calling | ||
| # network_func. Reference: fluid.program_guard in | ||
| # test_word2vec.py | ||
|
|
||
| # 2. move the default_main_program to self.program and run the | ||
| # default_startup program on an empty core.Scope() | ||
|
|
||
| # 3. call self.params.add_vars with the initialized scope, it | ||
| # will add the new vars of the initialized scope into | ||
| # self.params. | ||
| self.network_func = network_func | ||
| self.optimizer = optimizer | ||
| self.params = params | ||
| self.place = place | ||
| # TODO(helin): support distributed training | ||
|
|
||
| def train(self, reader, num_epochs, event_handler): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't quite remember how do we define a pass, and epoch, with our current reader design? If a reader reads gRPC requests, it would never end and would have no pass separations, right?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A call to the reader will return a iterator that contains a single pass of data. Yes, if a reader reads gRPC requests, it would never end and would have no pass separations. |
||
| pass | ||
|
|
||
| def test(self, reader): | ||
| pass | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the relationship between a Params instance and this scope instance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Params is a wrapper on top of scope, and provides additional functionality to save and load parameters.
We think it's not necessary, I will send a PR to remove it.
The new interface of the Trainer constructor will be:
The parameters of the trainer can be optional initialized by the saved parameters in
param_path, or taken fromscope. If nothing is provided, the parameters will be randomly initialized.The
scopeargument is necessary because we want to share the scope between inferencer and trainer. One use case is GAN.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed params.py in #10354