Heap memory leak when calling client.connect() in 2.3.0, 2.4.1, & tip #4733

jstuewe · 2018-05-16T02:01:50Z

Basic Infos

This issue complies with the issue POLICY doc.
I have read the documentation at readthedocs and the issue is not addressed there.
I have tested that the issue is present in current master branch (aka latest git).
I have searched the issue tracker for a similar issue.
If there is a stack dump, I have decoded it.
I have filled out all fields below.

Platform

Hardware: [ESP-12 on Wemos D1 mini]
Core Version: [2.3.0, 2.4.1, and latest 5/15/18 code]
Development Env: [Arduino IDE under Visual Micro\Visual Studio]
Operating System: [Windows]

Settings in IDE

Module: [Wemos D1 mini r2]
Flash Mode: []
Flash Size: [4MB (1M SPIFFS)]
lwip Variant: [v2 Lower Memory]
Reset Method: []
Flash Frequency: [40Mhz]
CPU Frequency: [80Mhz]
Upload Using: [OTA and SERIAL]
Upload Speed: [921600]

Problem Description

Heap goes down every time I use client.connect() in my application.
There is a bug with 2.4.1 core that will do this but I am using 2.3.0 which doesn't have this bug and i've tested several cores.

-No client.write() is necessary. Just connecting causes the issue.
-Loses 184 bytes per connection with core 2.3.0
-Loses 168 bytes per connection with core 2.4.1 and Arduino-master(05152018) tip
-Does NOT loose memory when connecting to 66.35.40.169 or data.sparkfun.com
-Does loose memory when connecting to 172.217.14.174 (google.com), 192.168.1.254 (Uverse residential gateway), or my IOT device
-Moving 'WiFiClient client;' to global or removing 'client = WiFiClient();' does not affect the issue

MCVE Sketch

#include <ESP8266WiFi.h>

const char* ssid = "Stuewe-GN";
const char* password = "putpasswordhere";

//const char* host = "data.sparkfun.com";
//const char* host = "66.35.40.169";

//const char* host = "google.com";
const char* host = "172.217.14.174";

//const char* host = "192.168.1.254";

const int httpPort = 80;
//const int httpPort = 5577; // the port I want to use. It leaks but I switched to port 80 for debugging
const char LedOffCmd[] = "\x71\x24\x0f\xa4\x00\x00\x00\x00\x00\x00";

WiFiClient client;


void setup() {
	Serial.begin(115200);
	delay(10);
	Serial.println();
	Serial.print("Connecting to ");
	Serial.println(ssid);
	WiFi.mode(WIFI_STA);
	WiFi.begin(ssid, password);
	while (WiFi.status() != WL_CONNECTED) {
		delay(500);
		Serial.print(".");
	}
	Serial.println("");
	Serial.print("WiFi connected at IP address: ");
	Serial.println(WiFi.localIP());
}


void loop() {
	delay(3000);

	client = WiFiClient(); // some forums said this might work around 2.4.1 bug

	if (client.connect(host, httpPort)) {
		Serial.print("Connected to ");
		Serial.println(host);
//		client.write(LedOffCmd, 10); // no need to actually write anything
		delay(500);
		while (client.available()) { 
			char c = client.read();
		}

	}
	else {
		Serial.print("Unable to connect to ");
		Serial.println(host);
	}
	client.stop();
	Serial.print("Heap: ");
	Serial.println(ESP.getFreeHeap());
}

Debug Messages

Output with different cores and targets

Architecture Tools: file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.3.0/cores/esp8266
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.3.0/variants/d1_mini

Connecting to Stuewe-GN
...
WiFi connected at IP address: 192.168.1.26
Connected to data.sparkfun.com
Heap: 47264
Connected to data.sparkfun.com
Heap: 47264
Connected to data.sparkfun.com
Heap: 47264
Connected to data.sparkfun.com
Heap: 47264


Connecting to Stuewe-GN
....
WiFi connected at IP address: 192.168.1.26
Connected to 66.35.40.169
Heap: 47264
Connected to 66.35.40.169
Heap: 47264
Connected to 66.35.40.169 (data.sparkfun.com)
Heap: 47264

Connecting to Stuewe-GN
........
WiFi connected at IP address: 192.168.1.26
Connected to 172.217.14.174
Heap: 47264
Connected to 172.217.14.174
Heap: 47080
Connected to 172.217.14.174
Heap: 46896
Connected to 172.217.14.174
Heap: 46712
Connected to 172.217.14.174
Heap: 46528
Connected to 172.217.14.174
Heap: 46344
Connected to 172.217.14.174 (google.com)
Heap: 46160  (184 bytes)


Connecting to Stuewe-GN
........................
WiFi connected at IP address: 192.168.1.26
Connected to 192.168.1.254 
Heap: 47264
Connected to 192.168.1.254
Heap: 47080  
Connected to 192.168.1.254
Heap: 46896
Connected to 192.168.1.254
Heap: 46712
Connected to 192.168.1.254
Heap: 46528
Connected to 192.168.1.254 (Uverse residential gateway)
Heap: 46344  (184 bytes)


Architecture Tools: file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/
Sketch Book: file:///C:/Users/john/OneDrive/Arduino_ESP
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.4.1/cores/esp8266
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/2.4.1/variants/d1_mini

Connecting to Stuewe-GN
.....................
WiFi connected at IP address: 192.168.1.26
Connected to 66.35.40.169
Heap: 45760
Connected to 66.35.40.169
Heap: 45760
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169
Heap: 45256
Connected to 66.35.40.169 (data.sparkfun.com)
Heap: 45256

Connecting to Stuewe-GN
.......................
WiFi connected at IP address: 192.168.1.26
Connected to 192.168.1.254
Heap: 45760 
Connected to 192.168.1.254
Heap: 45592
Connected to 192.168.1.254
Heap: 44920
Connected to 192.168.1.254
Heap: 44752
Connected to 192.168.1.254
Heap: 44584
Connected to 192.168.1.254
Heap: 44416
Connected to 192.168.1.254
Heap: 44248
Connected to 192.168.1.254
Heap: 44080
Connected to 192.168.1.254 (Uverse residential gateway)
Heap: 43912  (168 bytes)

Architecture Tools: file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/
Sketch Book: file:///C:/Users/john/OneDrive/Arduino_ESP
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/Arduino-master05152018/cores/esp8266
Include Path file:///C:/Users/john/AppData/Local/arduino15/packages/esp8266/hardware/esp8266/Arduino-master05152018/variants/d1_mini

 
 Connecting to Stuewe-GN
..
WiFi connected at IP address: 192.168.1.26
Connected to 66.35.40.169 
Heap: 48872
Connected to 66.35.40.169
Heap: 48872
Connected to 66.35.40.169
Heap: 48368
Connected to 66.35.40.169
Heap: 48368
Connected to 66.35.40.169
Heap: 48368
Connected to 66.35.40.169 (data.sparkfun.com)
Heap: 48368 (No memory leak)


 Connecting to Stuewe-GN
.......................
WiFi connected at IP address: 192.168.1.26
Connected to 172.217.14.174
Heap: 48872
Connected to 172.217.14.174
Heap: 48704
Connected to 172.217.14.174
Heap: 48032
Connected to 172.217.14.174
Heap: 47864
Connected to 172.217.14.174
Heap: 47696
Connected to 172.217.14.174
Heap: 47528
Connected to 172.217.14.174 (google.com)
Heap: 47360   (168 bytes)

The text was updated successfully, but these errors were encountered:

5chufti · 2018-05-17T11:03:57Z

with the tcpCleanup() from d.a.v. I get stable heap after first two connects on git from 2018/04/02 for all lwip variants, also for 2.3.0 (only lwip1.4, has one drop after xx connects).
For 2.4.1 I can't get any stable result for any lwip version.
So most probably this was an error in ESP8266WiFi introduced in 2.4.x, allready fixed in recent git (2.5.x).

#include <ESP8266WiFi.h>

const char* ssid = "yourSSID";
const char* password = "yourPASSPHRASE";

//const char* host = "data.sparkfun.com";
//const char* host = "66.35.40.169";

const char* host = "www.google.com";
//const char* host = "172.217.14.174";

//const char* host = "192.168.1.254";

const int httpPort = 80;
//const int httpPort = 5577; // the port I want to use. It leaks but I switched to port 80 for debugging
const char LedOffCmd[] = "\x71\x24\x0f\xa4\x00\x00\x00\x00\x00\x00";

struct tcp_pcb;
extern struct tcp_pcb* tcp_tw_pcbs;
extern "C" void tcp_abort (struct tcp_pcb* pcb);

void tcpCleanup ()
{
  while (tcp_tw_pcbs != NULL)
  {
    tcp_abort(tcp_tw_pcbs);
  }
}


WiFiClient client;

void setup() {
  Serial.begin(74880);
  delay(10);
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("WiFi connected at IP address: ");
  Serial.println(WiFi.localIP());
}


void loop() {
  delay(1500);

//  client = WiFiClient(); // some forums said this might work around 2.4.1 bug

  if (client.connect(host, httpPort)) {
    Serial.print("Connected to ");
    Serial.println(host);
//    client.write(LedOffCmd, 10); // no need to actually write anything
    delay(500);
    while (client.available()) {
      char c = client.read();
    }
  }
  else {
    Serial.print("Unable to connect to ");
    Serial.println(host);
  }
  client.stop();

  tcpCleanup();
  delay(1500);

  Serial.print("Heap: ");
  Serial.println(ESP.getFreeHeap());
}

d-a-v · 2018-05-17T12:20:18Z

There are several problems

There is a known leak in 2.4.1 which is fixed in master
@5chufti is right, tcpCleanup() is needed for that sketch (and current master which has the above leak fixed)
PR lwip2 fix and update #4729 will definitively remove the need of tcpCleanup() with lwip2 (it was not needed anymore since a while with lwip1.4), see below heap stopping to decrease after about 5 connections.

14:12:50.287 -> WiFi connected at IP address: 10.43.1.117
14:12:53.405 -> Connected to 172.217.14.174
14:12:53.902 -> Heap: 48520
14:12:57.019 -> Connected to 172.217.14.174
14:12:57.549 -> Heap: 48344
14:12:59.902 -> pm open,type:2 0
14:13:00.665 -> Connected to 172.217.14.174
14:13:01.162 -> Heap: 47648
14:13:04.312 -> Connected to 172.217.14.174
14:13:04.809 -> Heap: 47472
14:13:07.925 -> Connected to 172.217.14.174
14:13:08.453 -> Heap: 47296
14:13:11.570 -> Connected to 172.217.14.174
14:13:12.067 -> Heap: 47120
14:13:15.185 -> Connected to 172.217.14.174
14:13:15.716 -> Heap: 46944
14:13:18.832 -> Connected to 172.217.14.174
14:13:19.329 -> Heap: 46944
14:13:22.478 -> Connected to 172.217.14.174
14:13:22.975 -> Heap: 46944
14:13:26.091 -> Connected to 172.217.14.174
14:13:26.588 -> Heap: 46944
14:13:29.736 -> Connected to 172.217.14.174
14:13:30.233 -> Heap: 46944
...

jstuewe · 2018-10-15T03:49:56Z

Thank you 5chufti and d-a-v.
I did get it working based on your comments but obviously forgot to go back and close the issue.
Closing it now.

gojimmypi mentioned this issue Jul 4, 2018

ESP8266 crashing upon HTTPS-Update when using SSL #2758

Closed

jstuewe closed this as completed Oct 15, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Heap memory leak when calling client.connect() in 2.3.0, 2.4.1, & tip #4733

Heap memory leak when calling client.connect() in 2.3.0, 2.4.1, & tip #4733

jstuewe commented May 16, 2018

5chufti commented May 17, 2018

d-a-v commented May 17, 2018 •

edited

Loading

jstuewe commented Oct 15, 2018

Heap memory leak when calling client.connect() in 2.3.0, 2.4.1, & tip #4733

Heap memory leak when calling client.connect() in 2.3.0, 2.4.1, & tip #4733

Comments

jstuewe commented May 16, 2018

Basic Infos

Platform

Settings in IDE

Problem Description

MCVE Sketch

Debug Messages

5chufti commented May 17, 2018

d-a-v commented May 17, 2018 • edited Loading

jstuewe commented Oct 15, 2018

d-a-v commented May 17, 2018 •

edited

Loading