Skip to content

refact: Decide API #314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Feb 1, 2021
Merged

refact: Decide API #314

merged 20 commits into from
Feb 1, 2021

Conversation

oakbani
Copy link
Contributor

@oakbani oakbani commented Jan 22, 2021

Summary

This PR refactors and fixes the decide implementation in another branch.
todo: reasons work. It will be done separately.

Test plan

  • Unit Tests added.
  • Manually verified with Decide API tests in FSC

Issues

Copy link
Contributor

@jaeopt jaeopt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes suggested.
I also see some comments for previous PRs (#305, #309) not addressed yet. Can you fix them as well?

return {
'user_id': self.user_id,
'attributes': self.user_attributes,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a mutex control for setAttribute and getAttributes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about changing public class names to OptimizelyUserContext, OptimizelyDecision and OptimizelyDecideOption to be consistent with other SDKs and docs?


if default_decide_options is None:
self.default_decide_options = []

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if default_decide_option is non-null?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have already placed another check below which ensures that it must be a list when non-null

@@ -1120,7 +1100,7 @@ def decide_all(self, user_context, decide_options=None):

# check if SDK is ready
if not self.is_valid:
self.logger.error(enums.Errors.INVALID_PROJECT_CONFIG.format('decide_all'))
self.logger.error(enums.Errors.INVALID_OPTIMIZELY.format('decide_all'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line #1131: return {} when SDK_NOT_READY

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it already does. Looks like github is not showing you the updated diff.

@@ -1153,7 +1132,7 @@ def decide_for_keys(self, user_context, keys, decide_options=None):

# check if SDK is ready
if not self.is_valid:
self.logger.error(enums.Errors.INVALID_PROJECT_CONFIG.format('decide_for_keys'))
self.logger.error(enums.Errors.INVALID_OPTIMIZELY.format('decide_for_keys'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line #1161: should check with "default_decide_options" as well

@oakbani oakbani mentioned this pull request Jan 25, 2021
@oakbani
Copy link
Contributor Author

oakbani commented Jan 25, 2021

@jaeopt I have tried to address all of your comments both in this PR and the rest. I am yet to take a final look on the unit tests. You may re-review all new changes besides the tests file.

Copy link
Contributor

@jaeopt jaeopt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of changes suggested. Other than that, all looks good.
I didn't review changes in unit tests. I'll take a look at them with your remaining fixes.

@oakbani
Copy link
Contributor Author

oakbani commented Jan 28, 2021

@jaeopt Reasons work has been done. Unit testing is in progress. You may review reasons related work for now.

Copy link
Contributor

@jaeopt jaeopt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it looks good!
Some small changes suggested.

)
decide_reasons.append(message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message can be dropped from decision reasons. Too much info.
Can remove "reasons" from this method return.

@@ -1113,7 +1118,7 @@ def _decide_all(self, user_context, decide_options=None):
if not config:
self.logger.error(enums.Errors.INVALID_PROJECT_CONFIG.format('decide'))
reasons.append(OptimizelyDecisionMessage.SDK_NOT_READY)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for update reasons here. We return {}

Comment on lines 1070 to 1072
should_include_reasons = False
if OptimizelyDecideOption.INCLUDE_REASONS in decide_options:
should_include_reasons = True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
should_include_reasons = False
if OptimizelyDecideOption.INCLUDE_REASONS in decide_options:
should_include_reasons = True
should_include_reasons = OptimizelyDecideOption.INCLUDE_REASONS in decide_options

return None
message = 'User "%s" is not in the forced variation map.' % user_id
self.logger.debug(message)
decide_reasons.append(message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this from reasons? It's the default case.

)
return None
decide_reasons.append(message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this from reasons? It's the default case.


def get_variation(self, project_config, experiment, user_id, attributes, ignore_user_profile=False):
def get_variation(
self, project_config, experiment, user_id, attributes, ignore_user_profile=False, decide_options=[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove "decide_options" not used here. If you want to keep it, then you can remove "ignore_user_profile" from arguments.

@@ -225,25 +242,31 @@ def get_variation(self, project_config, experiment, user_id, attributes, ignore_
user_id: ID for user.
attributes: Dict representing user attributes.
ignore_user_profile: True to ignore the user profile lookup. Defaults to False.
decideOptions: Options to customize evaluation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this line too

if variation:
return variation
message = 'Returning previously activated variation ID "{}" of experiment ' \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message is redundant in reasons (see #223). I think we should remove #223 from reasons and keep this one.


def test_decide__invalid_flag_key(self):
opt_obj = optimizely.Optimizely(json.dumps(self.config_dict_with_features))
user_context = opt_obj.create_user_context('test_user')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add non-empty attributes to this user-context, so we can validate the user-context clone works ok when validating below?

'optimizely.optimizely.Optimizely._send_impression_event'
) as mock_send_event:

user_context = opt_obj.create_user_context('test_user')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here too. Add non-empty attributes to user-context

@oakbani oakbani removed their assignment Jan 29, 2021
Copy link
Contributor

@jaeopt jaeopt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!
Suggested a few more tests for "reasons" validation

mock_decide.assert_called_with(
user_context,
'test_feature_in_experiment',
['EXCLUDE_VARIABLES', 'ENABLED_FLAGS_ONLY']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"decide_for_keys" should not pass the merged options (options + default_decide_options) to "decide()" since decide() itself merge options with default_decide_option. Can you fix this test and the optimizely.

{'browser': 'chrome'}
)

def test_decide__option__include_reasons__feature_test(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests look good.
Can we add a few more tests validating "reasons" messages for:

  • hit on everyone-else rule
  • hit on none of the rules
  • hit variation from "get_forced_variation"
  • hit from "white_list"
  • hit from user-profile-service
  • any other important cases?

@oakbani oakbani requested a review from jaeopt February 1, 2021 11:04
@oakbani oakbani removed their assignment Feb 1, 2021
Copy link
Contributor

@jaeopt jaeopt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@oakbani oakbani merged commit 9517442 into decideApi Feb 1, 2021
@oakbani oakbani deleted the oakbani/decide-api branch February 1, 2021 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants