Migrate mixer API and config to this repo. #2
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,24 +1,15 @@ | ||
| # Istio APIs and Common Configuration Definitions | ||
|
|
||
| This repo defines component-level APIs and common configuration formats for the Istio | ||
| platform. These definitions are specified using the [protobuf](https://github.com/google/protobuf) | ||
| syntax. | ||
|
|
||
| All other Istio repositories can take a dependency on the api | ||
| repository. This repository *will not* depend on any other repos | ||
|
|
||
| We may check-in generated .pb.go and .pb.cc files here. | ||
|
|
||
| ## Standard vocabulary | ||
|
|
||
| All components of an Istio installation operate on a shared vocabulary of attributes. | ||
| A standard vocabulary of attributes including it meaning is available in this repo. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| // Copyright 2016 Google Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| syntax = "proto3"; | ||
|
|
||
| package istio.mixer.v1; | ||
|
|
||
| import "google/protobuf/timestamp.proto"; | ||
|
|
||
| // An instance of this is delivered to the mixer with every | ||
| // API call. | ||
| // | ||
| // The general idea is to leverage the stateful gRPC streams from the | ||
| // proxy to the mixer to keep to a minimum the 'attribute chatter'. | ||
| // Only delta attributes are sent over, multiple concurrent attribute | ||
| // contexts can be used to avoid thrashing, and attribute indices are used to | ||
| // keep the wire protocol maximally efficient. | ||
|
There was a problem hiding this comment. This sounds like a pretty complex protocol. Can we start with one implicit context and add disjoint contexts when we have evidence of need. After all multiple contexts can be handled by having multiple GRPC streams. |
||
| // | ||
| // Producing this message is the responsibility of the mixer's client | ||
| // library which is linked into different proxy implementations. | ||
| // | ||
| // The processing order for this state in the mixer is: | ||
| // | ||
| // * Any new dictionary is applied | ||
| // | ||
| // * The requested attribute context is looked up. If no such context has been defined, a | ||
| // new context is automatically created and initialized to the empty state. When a gRPC | ||
| // stream is first created, there are no attribute contexts for the stream. | ||
| // | ||
| // * If reset_context is true, then the attribute context is reset to the | ||
| // empty state. | ||
| // | ||
| // * All attribute changes are applied to the attribute context. | ||
| // | ||
| // * All deleted attributes are removed from the attribute context. | ||
| // | ||
| message Attributes { | ||
| // A dictionary that provides a mapping of shorthand index values to | ||
| // attribute names. | ||
| // | ||
| // This is intended to leverage the stateful gRPC stream from the | ||
| // proxy to the mixer. This dictionary is sent over only when a | ||
| // stream to the mixer is first established and when the proxy's | ||
| // config changes and different attributes may be produced. | ||
| // | ||
| // Once a dictionary has been sent over, it stays in effect until | ||
| // a new dictionary is sent to replace it. The first request sent on a | ||
| // stream must include a dictionary, otherwise the mixer can't process | ||
| // any attribute updates. | ||
| // | ||
| // Dictionaries are independent of the attribute context and are thus global | ||
| // for the current gRPC stream. | ||
| map<int32, string> dictionary = 1; | ||
|
|
||
| // The attribute context against which to operate. | ||
| // | ||
| // The mixer keeps different contexts live for any proxy gRPC stream. This | ||
| // allows the proxy to maintain multiple concurrent 'bags of attributes' | ||
| // within the mixer. | ||
| // | ||
| // If the proxy doesn't want to leverage multiple contexts, it just passes | ||
| // 0 here for every request. | ||
| // | ||
| // The proxy is configured to use a maximum number of attribute contexts in order | ||
| // to prevent an explosion of contexts in the mixer's memory space. | ||
| // | ||
| // TODO: Consider removing support for this feature. The proxy can achieve | ||
| // the same thing using multiple gRPC streams. The benefit of using streams | ||
| // would be that the mixer would be in control of the maximum number of streams | ||
| // it allows, whereas with the current model the proxy could overwhelm the | ||
| // mixer by creating too many contexts. | ||
| int32 attribute_context = 2; | ||
|
|
||
| // When true, resets the current attribute context to the empty state before | ||
| // applying any incoming attributes. | ||
| // | ||
| // Resetting contexts is useful to constrain the amount of resources used by | ||
| // the mixer. The proxy needs to intelligently manage a pool of contexts. | ||
| // It may be useful to reset a context when certain big events happen, such | ||
| // as when an HTTP2 connection into the proxy terminates. | ||
| bool reset_context = 3; | ||
|
There was a problem hiding this comment. Consider providing guidance about when an attribute context should be reset so that the server can reclaim memory. E.g. If an HTTP2 connection terminated we should be able to reclaim the memory occupied by the context by re-using its id for a new connection later on. |
||
|
|
||
| // Attributes being updated within the specified attribute context. These maps | ||
| // add and/or overwrite the context's current set of attributes. | ||
| map<int32, string> string_attributes = 4; | ||
| map<int32, int64> int64_attributes = 5; | ||
| map<int32, double> double_attributes = 6; | ||
| map<int32, bool> bool_attributes = 7; | ||
| map<int32, google.protobuf.Timestamp> timestamp_attributes = 8; | ||
| map<int32, bytes> bytes_attributes = 9; | ||
|
|
||
| // Attributes that should be removed from the specified attribute context. Deleting | ||
| // attributes which aren't currently in the attribute context is not considered an error. | ||
| repeated int32 deleted_attributes = 10; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| // Copyright 2016 Google Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| syntax = "proto3"; | ||
|
|
||
| package istio.mixer.v1; | ||
|
|
||
| import "google/rpc/status.proto"; | ||
| import "mixer/api/v1/attribute.proto"; | ||
|
|
||
| // Used to verify preconditions before performing an action. | ||
| message CheckRequest { | ||
| // Index within the stream for this request, used to match to responses | ||
| int64 request_index = 1; | ||
|
|
||
| // The attributes to use for this request | ||
| Attributes attribute_update = 2; | ||
| } | ||
|
|
||
| message CheckResponse { | ||
| // Index of the request this response is associated with | ||
| int64 request_index = 1; | ||
|
|
||
| // Indicates whether or not the preconditions succeeded | ||
| google.rpc.Status result = 2; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| // Copyright 2016 Google Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| syntax = "proto3"; | ||
|
|
||
| package istio.mixer.v1; | ||
|
|
||
| import "google/rpc/status.proto"; | ||
| import "mixer/api/v1/attribute.proto"; | ||
|
|
||
| message QuotaRequest { | ||
| // Index within the stream for this request, used to match to responses | ||
| int64 request_index = 1; | ||
|
|
||
| // The attributes to use for this request | ||
| Attributes attribute_update = 2; | ||
|
|
||
| // what kind of quota operation to perform | ||
| OperationKind kind = 3; | ||
|
|
||
| // Used for deduplicating quota allocation/free calls in the case of | ||
| // failed RPCs are retries. This should be a UUID per call, where the same | ||
| // UUID is used for retries of the same quota allocation or release call. | ||
| string deduplication_id = 4; | ||
|
|
||
| enum OperationKind { | ||
| QUOTA_MODE_UNSPECIFIED = 0; | ||
|
|
||
| // Allocate the specified amount, fail if insufficient resources available. | ||
| ALLOC_NORMAL = 1; | ||
|
|
||
| // Allocate from 0 to the specified amount, never fails. | ||
| ALLOC_BEST_EFFORT = 2; | ||
|
|
||
| // Release from 0 to the specified amount, never fails. | ||
| RELEASE_BEST_EFFORT = 3; | ||
|
|
||
| // Return the current content of the quota value cell. | ||
| QUERY = 4; | ||
| } | ||
| } | ||
|
|
||
| // Used to update quota values before and/or after performing an action. | ||
| // | ||
| // A common use case for quotas is to implement rate limits within a service. | ||
| // Quotas can also be used to impose upper limits on the number of specific | ||
| // resources exposed by a service to its consumers. | ||
| message QuotaResponse { | ||
| // Index of the request this response is associated with | ||
| int64 request_index = 1; | ||
|
|
||
| // Results, one for each operation in the request. | ||
| // This map is indexed by the quota_operation_id of the individual quota operations. | ||
| oneof result { | ||
| // The effective amount of quota | ||
| uint64 effective_amount = 2; | ||
|
|
||
| // An error indication in case the quota operation failed | ||
| google.rpc.Status error = 3; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| // Copyright 2016 Google Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| syntax = "proto3"; | ||
|
|
||
| package istio.mixer.v1; | ||
|
|
||
| import "google/rpc/status.proto"; | ||
| import "mixer/api/v1/attribute.proto"; | ||
|
|
||
| // Used to report telemetry after performing an action. | ||
| message ReportRequest { | ||
| // Index within the stream for this request, used to match to responses | ||
| int64 request_index = 1; | ||
|
|
||
| // The attributes to use for this request | ||
| Attributes attribute_update = 2; | ||
| } | ||
|
|
||
| message ReportResponse { | ||
| // Index of the request this response is associated with | ||
| int64 request_index = 1; | ||
|
|
||
| // Indicates whether the report was processed or not | ||
| google.rpc.Status result = 2; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| // Copyright 2016 Google Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| syntax = "proto3"; | ||
|
|
||
| package istio.mixer.v1; | ||
|
|
||
| import "api/mixer/v1/check.proto"; | ||
| import "api/mixer/v1/report.proto"; | ||
| import "api/mixer/v1/quota.proto"; | ||
|
|
||
| // The Istio Mixer API | ||
| service Mixer { | ||
| // Checks preconditions before performing an operation. | ||
| // The preconditions enforced depend on the set of supplied attributes | ||
| // and the active configuration. | ||
| rpc Check(stream CheckRequest) returns (stream CheckResponse) {} | ||
|
There was a problem hiding this comment. Attribute context will span check & report all the time so having them be independent streams seems quite wasteful given all the effort put into dictionary management above. I would either: 1 - punt on dictionary management possibly in favor of a partially static dictionary (see HTTP2 https://http2.github.io/http2-spec/compression.html#static.table.definition) The former seems a lot simpler than the latter frankly There was a problem hiding this comment. I like the lack of special cases in this model, and I like the uniformity. All of the API methods work the same way, both on the server and client, so they'll just be some common code on both ends. The actual runtime overhead of multiple dictionaries is tiny both in CPU time and memory. I'd like to keep this as-is and see how it pans out. There was a problem hiding this comment. I don't strictly mind repeating the same dictionary or attribute context three times so that its available for each operation type for the context at hand. Need to document how to handle the failure mode when an attribute context cannot be allocated by the server, I suspect we should terminate the stream and reset everything and require the client to recover. I'm fine with this, will add some clarifying doc comments. |
||
|
|
||
| // Reports telemetry, such as logs and metrics. | ||
| // The reported information depends on the set of supplied attributes | ||
| // and the active configuration. | ||
| rpc Report(stream ReportRequest) returns (stream ReportResponse) {} | ||
|
|
||
| // Quota allocates and releases quota. | ||
| rpc Quota(stream QuotaRequest) returns (stream QuotaResponse) {} | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i feel like the packages should be mixer.api.v1 instead of istio.mixer.v1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For Google APIs, we never use the term "api". When you have v1 in the package, it is pretty much an api. Adding "api" everywhere doesn't add much value, almost all protos are api anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be using mixer.v1 or is istio.mixer.v1 better?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing api is fine. However, these protos live in mixer/api/v1. It seems a bit appropriate to match that structure in the package.
In any case, I feel like "istio.mixer.v1" is not ideal. The generated go code will lead to some imports like "istio.io/istio/mixer/v1". I'd prefer something like "istio.io/mixer/v1".
We can address this later, once we switch the code over.