Question about using the message-lite in Paddle

[NHZlX]

I’m working on reducing the dynamic libraries of capi for paddle deploying on mobile.

If we use protobuf-lite.a, that is, set option optimize_for = LITE_RUNTIME(defalut is SPEED) in proto. The generated class in *.pb.h will Inherit from Message_lite not the
Message, it will greatly reduce the size of the library.

I haven't found how to dynamically set this option from Official document

I see the option optimize_for = LITE_RUNTIME already setted in OptimizerConfig.proto, but not in other proto files. For examples, ModelConfig.proto. But I find no where use the interface of Message in our paddle code. Can we set all them option optimize_for = LITE_RUNTIME, if not, why?

I think this also should be taken into consideration by the new paddle

[dzhwinter]

We are not sure it is a big issue. The option LITE_RUNTIME can omit the type reflection and heavy io interface implementation and then generate some small size library. The reason behind the option in OptimizerConfig.proto is in comments optimizerconfig.

this is used in SerilazieAsString function calls.
there is two version of protobuf message seriliaze header file of message.h message_lite.h
option will declare which one to include. Only the message_lite.h contains AsString interface, which is more clear.

https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message_lite

这里解释一下：
option optimize_for = LITE_RUNTIME; 有两个side effect, 一个是性能，另一个是会影响include的头文件。用到的接口SerilazieAsString定义在message_lite.h里，如果不写option,那么序列化接口只能这么写:

// include an protobuf header file explicity
#include <google/protobuf/message_lite.h>

std::string blabla;
SerilazieToString(blabla);
// then use the string as before

Have you had a try to metric this library size gain with this option on/off? If so, we can add a configuration in our cmake compile option.

[reyoung]

I think we should always use message_lite, since the speed of generating or loading protobuf is not critical.

[NHZlX]

@dzhwinter
the size of libpaddle_capi_shared.so compiled of armabi-v7a android

type	normal	striped
message	9.7M	4.6M
message-lite	8.8M	4.1M

Is it possible to control on / off in cmake? If so, That would be fine.

[NHZlX]

@reyoung

Using message_lite doesn't affect speed, but it's missing some interfaces

Here is the fragment from the Official Documents.

optimize_for (file option):

Can be set to SPEED, CODE_SIZE, or LITE_RUNTIME. This affects the C++ and Java code generators (and possibly third-party generators) in the following ways:

SPEED (default): The protocol buffer compiler will generate code for serializing, parsing, and performing other common operations on your message types. This code is highly optimized.

CODE_SIZE: The protocol buffer compiler will generate minimal classes and will rely on shared, reflection-based code to implement serialialization, parsing, and various other operations. The generated code will thus be much smaller than with SPEED, but operations will be slower. Classes will still implement exactly the same public API as they do in SPEED mode. This mode is most useful in apps that contain a very large number .proto files and do not need all of them to be blindingly fast.

LITE_RUNTIME: The protocol buffer compiler will generate classes that depend only on the "lite" runtime library (libprotobuf-lite instead of libprotobuf). The lite runtime is much smaller than the full library (around an order of magnitude smaller) but omits certain features like descriptors and reflection. This is particularly useful for apps running on constrained platforms like mobile phones. The compiler will still generate fast implementations of all methods as it does in SPEED mode. Generated classes will only implement the MessageLite interface in each language, which provides only a subset of the methods of the full Message interface.

[NHZlX]

Given that we will use the descriptors and reflectionfeature in the future. So, we should not set option optimize_for = LITE_RUNTIME in any proto files. Thanks to @hedaoyuan 's help and my investigation, we found that the lite mode can be dynamic setted by ./protoc --cpp_out=lite:$OUTPUT_PATH foo.proto in protoc3. More details could be seen here #4638