Skip to content

Events monitoring stream does not seems to always include CR or LF character at the end of every messages #434

Closed
@simardst

Description

@simardst

Output of dotnet --info:

.NET Core SDK (reflecting any global.json):
 Version:   3.1.101
 Commit:    b377529961

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support):
  Version: 3.1.1
  Commit:  a1388f194c

What version of Docker.DotNet?:

3.125.2

Docker Version:

Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:22:37 2019
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea
  Built:            Wed Nov 13 07:29:19 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Docker Info:

Client:                                                                                                   
 Debug Mode: false                                                                                        
                                                                                                          
Server:                                                                                                   
 Containers: 1                                                                                            
  Running: 1                                                                                              
  Paused: 0                                                                                               
  Stopped: 0                                                                                              
 Images: 10                                                                                               
 Server Version: 19.03.5                                                                                  
 Storage Driver: overlay2                                                                                 
  Backing Filesystem: extfs                                                                               
  Supports d_type: true                                                                                   
  Native Overlay Diff: true                                                                               
 Logging Driver: json-file                                                                                
 Cgroup Driver: cgroupfs                                                                                  
 Plugins:                                                                                                 
  Volume: local                                                                                           
  Network: bridge host ipvlan macvlan null overlay                                                        
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog                     
 Swarm: inactive                                                                                          
 Runtimes: runc                                                                                           
 Default Runtime: runc                                                                                    
 Init Binary: docker-init                                                                                 
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339                                             
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657                                                   
 init version: fec3683                                                                                    
 Security Options:                                                                                        
  seccomp                                                                                                 
   Profile: default                                                                                       
 Kernel Version: 4.19.76-linuxkit                                                                         
 Operating System: Docker Desktop                                                                         
 OSType: linux                                                                                            
 Architecture: x86_64                                                                                     
 CPUs: 2                                                                                                  
 Total Memory: 1.943GiB                                                                                   
 Name: docker-desktop                                                                                     
 ID: XESR:K73P:U465:PX4V:WDXD:HT4Y:H7K3:W7ED:7R2Y:KAEB:VF3G:WM7D                                          
 Docker Root Dir: /var/lib/docker                                                                         
 Debug Mode: true                                                                                         
  File Descriptors: 49                                                                                    
  Goroutines: 67                                                                                          
  System Time: 2020-01-31T17:25:24.631755878Z                                                             
  EventsListeners: 6                                                                                      
 Registry: https://index.docker.io/v1/                                                                    
 Labels:                                                                                                  
 Experimental: false                                                                                      
 Insecure Registries:                                                                                     
  127.0.0.0/8                                                                                             
 Live Restore Enabled: false                                                                              
 Product License: Community Engine                                                                        

Steps to reproduce the issue:

  1. Create DockerClient to local engine (npipe://./pipe/docker_engine)
  2. Monitor system events ()
m_dockerClient.System.MonitorEventsAsync(
    new ContainerEventsParameters(),
    new Progress<Message>(m => Console.WriteLine(JsonConvert.SerializeObject(m, Newtonsoft.Json.Formatting.Indented))));
  1. Using Docker CLI, monitor events (docker events)
  2. Using the docker CLI, start and stops containers
  3. Messages are printed on the Docker CLI console, but not on my app console (see 2.)

What actually happened?:
The HTTP request to get the event stream is accepted (200 - OK) and the stream is correctly opened, but there is no CR or LF character to delimit the messages received. Therefore, the stream reader gets stuck (since it is doing ReadLineAsync(...) calls to read messages and parse them)

What did you expect to happen?:
Find a way to extract messages from the event stream without requiring CR or LF charcaters to detect a complete message.

A solution could be to directly parse the stream to extratc JSON objects/messages.

internal static async Task MonitorStreamForMessagesAsync<T>(Task<Stream> streamTask, DockerClient client, CancellationToken cancel, IProgress<T> progress)
{
    using (var stream = await streamTask)
    {
        // ReadLineAsync must be cancelled by closing the whole stream.
        using (cancel.Register(() => stream.Dispose()))
        {
            using (var reader = new StreamReader(stream, new UTF8Encoding(false))) 
            using (var jsonReader = new JsonTextReader(reader) { SupportMultipleContent = true })
            {
                try
                {
                    while (jsonReader.Read())
                    {
                        var prog = client.JsonSerializer.DeserializeObject<T>(jsonReader);
                        if (prog == null) continue;

                        progress.Report(prog);
                    }
                }
                catch (ObjectDisposedException)
                {
                    // The subsequent call to reader.ReadLineAsync() after cancellation
                    // will fail because we disposed the stream. Just ignore here.
                }
            }
        }
    }
}
internal class JsonSerializer
{
    public T DeserializeObject<T>(JsonReader jsonReader)
    {
        var serializer = Newtonsoft.Json.JsonSerializer.Create(this._settings);
        serializer.CheckAdditionalContent = false;
        return serializer.Deserialize<T>(jsonReader);
    }
   ...
}

Additional information:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions