Performance Question

[WhereRtheInterwebs]

I'm curious about general performance benchmarks for this library. I have a 25 MB file that takes 11 minutes to parse. It seems like it should be faster. As a test I serialized the same data into JSON then deserialized again. The JSON serialization took 4.1 seconds. The JSON deserialization took 5.9 seconds to the same objects with the same data. Is it realistic to compare JSON deserialization to EDI deserialization?

Does anyone else have information on what is considered good time for processing EDI files?

Is there something I need to tweak with this library to get better performance? Is there any general guidance for the models on how to improve performance? Could improper attribute usage cause performance problems?

[cleftheris]

Hi @WhereRtheInterwebs. Although the Serialization logic is quite robust API wise the performance of large file processing has not been tested and optimized.

There are key areas that can be improved drastically like the following

• Do the actual benchmarking. Large EDI files should definitely be a lot faster to parse but first there should be some kind of benchmarking in order to see the hot paths
• the TextReader and TextWriter classes should be rewritten to make use of the dotnet framework language features like Span. This sould make memory and allocations go down.
• Since the forward looking nature of a the EDI parser makes use of internal stack structures (push/pop) this could be potentially optimized. This is done for keeping track of unstructured data that may be nested down the line and in usecases like with conditionals.
• Last but not least there could be some work done to make the attributes generate code on build time using a c# code generator so that the runtime does not need to make use of reflection to figure things out. But this would be the last thing to tackle IMHO.

I have little time to do the work at the moment my self but if someone in the community is interested in breaking the work in smaller PRs I would gladly help out to incorporate any meaningful changes that can make the performance go up.

[WhereRtheInterwebs]

For files that don't require conditionals, would there be any performance advantage to disabling the feature using an option?