Heap memory leak when calling client.connect() in 2.3.0, 2.4.1, & tip

[jstuewe]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP-12 on Wemos D1 mini]
• Core Version: [2.3.0, 2.4.1, and latest 5/15/18 code]
• Development Env: [Arduino IDE under Visual Micro\Visual Studio]
• Operating System: [Windows]

Settings in IDE

• Module: [Wemos D1 mini r2]
• Flash Mode: []
• Flash Size: [4MB (1M SPIFFS)]
• lwip Variant: [v2 Lower Memory]
• Reset Method: []
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz]
• Upload Using: [OTA and SERIAL]
• Upload Speed: [921600]

Problem Description

Heap goes down every time I use client.connect() in my application.
There is a bug with 2.4.1 core that will do this but I am using 2.3.0 which doesn't have this bug and i've tested several cores.

-No client.write() is necessary. Just connecting causes the issue.
-Loses 184 bytes per connection with core 2.3.0
-Loses 168 bytes per connection with core 2.4.1 and Arduino-master(05152018) tip
-Does NOT loose memory when connecting to 66.35.40.169 or data.sparkfun.com
-Does loose memory when connecting to 172.217.14.174 (google.com), 192.168.1.254 (Uverse residential gateway), or my IOT device
-Moving 'WiFiClient client;' to global or removing 'client = WiFiClient();' does not affect the issue

MCVE Sketch

#include <ESP8266WiFi.h>

const char* ssid = "Stuewe-GN";
const char* password = "putpasswordhere";

//const char* host = "data.sparkfun.com";
//const char* host = "66.35.40.169";

//const char* host = "google.com";
const char* host = "172.217.14.174";

//const char* host = "192.168.1.254";

const int httpPort = 80;
//const int httpPort = 5577; // the port I want to use. It leaks but I switched to port 80 for debugging
const char LedOffCmd[] = "\x71\x24\x0f\xa4\x00\x00\x00\x00\x00\x00";

WiFiClient client;


void setup() {
	Serial.begin(115200);
	delay(10);
	Serial.println();
	Serial.print("Connecting to ");
	Serial.println(ssid);
	WiFi.mode(WIFI_STA);
	WiFi.begin(ssid, password);
	while (WiFi.status() != WL_CONNECTED) {
		delay(500);
		Serial.print(".");
	}
	Serial.println("");
	Serial.print("WiFi connected at IP address: ");
	Serial.println(WiFi.localIP());
}


void loop() {
	delay(3000);

	client = WiFiClient(); // some forums said this might work around 2.4.1 bug

	if (client.connect(host, httpPort)) {
		Serial.print("Connected to ");
		Serial.println(host);
//		client.write(LedOffCmd, 10); // no need to actually write anything
		delay(500);
		while (client.available()) { 
			char c = client.read();
		}

	}
	else {
		Serial.print("Unable to connect to ");
		Serial.println(host);
	}
	client.stop();
	Serial.print("Heap: ");
	Serial.println(ESP.getFreeHeap());
}

Debug Messages

Output with different cores and targets

Architecture Tools: file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.3.0/variants/d1_mini

Connecting to Stuewe-GN
...
WiFi connected at IP address: 192.168.1.26
Connected to data.sparkfun.com
Heap: 47264
Connected to data.sparkfun.com
Heap: 47264
Connected to data.sparkfun.com
Heap: 47264
Connected to data.sparkfun.com
Heap: 47264


Connecting to Stuewe-GN
....
WiFi connected at IP address: 192.168.1.26
Connected to 66.35.40.169
Heap: 47264
Connected to 66.35.40.169
Heap: 47264
Connected to 66.35.40.169 (data.sparkfun.com)
Heap: 47264

Connecting to Stuewe-GN
........
WiFi connected at IP address: 192.168.1.26
Connected to 172.217.14.174
Heap: 47264
Connected to 172.217.14.174
Heap: 47080
Connected to 172.217.14.174
Heap: 46896
Connected to 172.217.14.174
Heap: 46712
Connected to 172.217.14.174
Heap: 46528
Connected to 172.217.14.174
Heap: 46344
Connected to 172.217.14.174 (google.com)
Heap: 46160  (184 bytes)


Connecting to Stuewe-GN
........................
WiFi connected at IP address: 192.168.1.26
Connected to 192.168.1.254 
Heap: 47264
Connected to 192.168.1.254
Heap: 47080  
Connected to 192.168.1.254
Heap: 46896
Connected to 192.168.1.254
Heap: 46712
Connected to 192.168.1.254
Heap: 46528
Connected to 192.168.1.254 (Uverse residential gateway)
Heap: 46344  (184 bytes)


Architecture Tools: file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/
Sketch Book: file:///C:/Users/john/OneDrive/Arduino_ESP
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.4.1/cores/esp8266
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.4.1/variants/d1_mini

Connecting to Stuewe-GN
.....................
WiFi connected at IP address: 192.168.1.26
Connected to 66.35.40.169
Heap: 45760
Connected to 66.35.40.169
Heap: 45760
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169 (data.sparkfun.com)
Heap: 45256

Connecting to Stuewe-GN
.......................
WiFi connected at IP address: 192.168.1.26
Connected to 192.168.1.254
Heap: 45760 
Connected to 192.168.1.254
Heap: 45592
Connected to 192.168.1.254
Heap: 44920
Connected to 192.168.1.254
Heap: 44752
Connected to 192.168.1.254
Heap: 44584
Connected to 192.168.1.254
Heap: 44416
Connected to 192.168.1.254
Heap: 44248
Connected to 192.168.1.254
Heap: 44080
Connected to 192.168.1.254 (Uverse residential gateway)
Heap: 43912  (168 bytes)

Architecture Tools: file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/
Sketch Book: file:///C:/Users/john/OneDrive/Arduino_ESP
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/Arduino-master05152018/cores/esp8266
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/Arduino-master05152018/variants/d1_mini

 
 Connecting to Stuewe-GN
..
WiFi connected at IP address: 192.168.1.26
Connected to 66.35.40.169 
Heap: 48872
Connected to 66.35.40.169
Heap: 48872
Connected to 66.35.40.169
Heap: 48368
Connected to 66.35.40.169
Heap: 48368
Connected to 66.35.40.169
Heap: 48368
Connected to 66.35.40.169 (data.sparkfun.com)
Heap: 48368 (No memory leak)


 Connecting to Stuewe-GN
.......................
WiFi connected at IP address: 192.168.1.26
Connected to 172.217.14.174
Heap: 48872
Connected to 172.217.14.174
Heap: 48704
Connected to 172.217.14.174
Heap: 48032
Connected to 172.217.14.174
Heap: 47864
Connected to 172.217.14.174
Heap: 47696
Connected to 172.217.14.174
Heap: 47528
Connected to 172.217.14.174 (google.com)
Heap: 47360   (168 bytes)

[5chufti]

with the tcpCleanup() from d.a.v. I get stable heap after first two connects on git from 2018/04/02 for all lwip variants, also for 2.3.0 (only lwip1.4, has one drop after xx connects).
For 2.4.1 I can't get any stable result for any lwip version.
So most probably this was an error in ESP8266WiFi introduced in 2.4.x, allready fixed in recent git (2.5.x).

#include <ESP8266WiFi.h>

const char* ssid = "yourSSID";
const char* password = "yourPASSPHRASE";

//const char* host = "data.sparkfun.com";
//const char* host = "66.35.40.169";

const char* host = "www.google.com";
//const char* host = "172.217.14.174";

//const char* host = "192.168.1.254";

const int httpPort = 80;
//const int httpPort = 5577; // the port I want to use. It leaks but I switched to port 80 for debugging
const char LedOffCmd[] = "\x71\x24\x0f\xa4\x00\x00\x00\x00\x00\x00";

struct tcp_pcb;
extern struct tcp_pcb* tcp_tw_pcbs;
extern "C" void tcp_abort (struct tcp_pcb* pcb);

void tcpCleanup ()
{
  while (tcp_tw_pcbs != NULL)
  {
    tcp_abort(tcp_tw_pcbs);
  }
}


WiFiClient client;

void setup() {
  Serial.begin(74880);
  delay(10);
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("WiFi connected at IP address: ");
  Serial.println(WiFi.localIP());
}


void loop() {
  delay(1500);

//  client = WiFiClient(); // some forums said this might work around 2.4.1 bug

  if (client.connect(host, httpPort)) {
    Serial.print("Connected to ");
    Serial.println(host);
//    client.write(LedOffCmd, 10); // no need to actually write anything
    delay(500);
    while (client.available()) {
      char c = client.read();
    }
  }
  else {
    Serial.print("Unable to connect to ");
    Serial.println(host);
  }
  client.stop();

  tcpCleanup();
  delay(1500);

  Serial.print("Heap: ");
  Serial.println(ESP.getFreeHeap());
}

[d-a-v]

There are several problems

• There is a known leak in 2.4.1 which is fixed in master
• @5chufti is right, tcpCleanup() is needed for that sketch (and current master which has the above leak fixed)
• PR lwip2 fix and update #4729 will definitively remove the need of tcpCleanup() with lwip2 (it was not needed anymore since a while with lwip1.4), see below heap stopping to decrease after about 5 connections.

14:12:50.287 -> WiFi connected at IP address: 10.43.1.117
14:12:53.405 -> Connected to 172.217.14.174
14:12:53.902 -> Heap: 48520
14:12:57.019 -> Connected to 172.217.14.174
14:12:57.549 -> Heap: 48344
14:12:59.902 -> pm open,type:2 0
14:13:00.665 -> Connected to 172.217.14.174
14:13:01.162 -> Heap: 47648
14:13:04.312 -> Connected to 172.217.14.174
14:13:04.809 -> Heap: 47472
14:13:07.925 -> Connected to 172.217.14.174
14:13:08.453 -> Heap: 47296
14:13:11.570 -> Connected to 172.217.14.174
14:13:12.067 -> Heap: 47120
14:13:15.185 -> Connected to 172.217.14.174
14:13:15.716 -> Heap: 46944
14:13:18.832 -> Connected to 172.217.14.174
14:13:19.329 -> Heap: 46944
14:13:22.478 -> Connected to 172.217.14.174
14:13:22.975 -> Heap: 46944
14:13:26.091 -> Connected to 172.217.14.174
14:13:26.588 -> Heap: 46944
14:13:29.736 -> Connected to 172.217.14.174
14:13:30.233 -> Heap: 46944
...

[jstuewe]

Thank you 5chufti and d-a-v.
I did get it working based on your comments but obviously forgot to go back and close the issue.
Closing it now.