Netmiko ReadTimeout due to Login Banner on Cisco ASA Devices

[daniel10059]

Description:
We have numerous Cisco ASA firewalls configured with a login and exec banner displaying:

:::::::::::::::> some banner message <:::::::::::::::

This banner occasionally causes Netmiko to misinterpret the prompt, leading to the following error:

/site-packages/netmiko/base_connection.py", line 748, in read_until_pattern
  raise ReadTimeout(msg)
netmiko.exceptions.ReadTimeout:
Pattern not detected: ':::::::.*' in output.

This issue occurs in approximately 10 out of 200 attempts, all on devices with the same banner.

Reproduction Steps:

con = netmiko.Netmiko(**dev)
response = con.send_command(command)

Expected Behavior:
Netmiko should correctly identify the device prompt and execute the command without timeout errors.

Actual Behavior:
Netmiko raises a ReadTimeout exception, failing to detect the prompt due to the banner.

Suggested Solution:
Please provide a fix in Netmiko so that it correctly handles such banners.

[ktbyers]

You probably should change the banner and not use this character in it: >

(i.e. it is problematic to use the prompt terminating character in the banner).

An alternate solution is to use the generic device_type and then use redispatch post login to switch to the cisco_asa device_type...that also might work.

[daniel10059]

Thank you for your response!

We have now removed the characters >, <, and # from the banner on all our ASA firewalls, as suggested.

However, we are still experiencing intermittent issues with Netmiko on ASA devices. It seems the problem occurs randomly — sometimes Netmiko fails to correctly detect the prompt, even though the banners no longer include problematic characters.

For example, during a simple show version, Netmiko incorrectly matches a line such as AnyConnect for Mobile: Enabled as the prompt or expected pattern:

TEST-FW1# show version
Cisco Adaptive Security Appliance Software Version 9.20(3) <context>
SSP Operating System Version 2.14(2.106)
Device Manager Version 7.20(2)

Compiled on Wed 31-Jul-24 00:06 GMT by builders

TEST-FW1 up 92 days 12 hours
failover cluster up 92 days 12 hours
Start-up time 5 secs

Hardware:   FPR-4215

License mode: Smart Licensing

Licensed features for this user context:
Failover                          : Active/Active  
Encryption-DES                    : Enabled        
Encryption-3DES-AES               : Enabled        
Carrier                           : Disabled       
AnyConnect Premium Peers          : 0              
Other VPN Peers                   : 0              
AnyConnect for Mobile             : Enabled
AnyConnect for Cisco VPN Phone    : Enabled        
Advanced Endpoint Assessment      : Enabled        
Cluster                           : Enabled        

Failover cluster licensed features for this user context:
Failover                          : Active/Active  
Encryption-DES                    : Enabled        
Encryption-3DES-AES               : Enabled        
Carrier                           : Disabled       
AnyConnect Premium Peers          : 0              
Other VPN Peers                   : 0              
AnyConnect for Mobile             : Enabled    
AnyConnect for Cisco VPN Phone    : Enabled        
Advanced Endpoint Assessment      : Enabled        
Cluster                           : Enabled        

Configuration last modified by ....

Traceback (most recent call last):
    response = con.send_command(command, read_timeout = 600.0, use_textfsm=use_textfsm, use_genie=use_genie, use_ttp=use_ttp, ttp_template=ttp_template)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/netmiko/cisco/cisco_asa_ssh.py", line 102, in send_command
    output = super().send_command(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/netmiko/base_connection.py", line 111, in wrapper_decorator
    return_val = func(self, *args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/netmiko/utilities.py", line 596, in wrapper_decorator
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/netmiko/base_connection.py", line 1826, in send_command
    raise ReadTimeout(msg)
netmiko.exceptions.ReadTimeout:
Pattern not detected: 'AnyConnect\\ for\\ Mobile\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ :\\ Enabled' in output.
 
Things you might try to fix this:
1. Explicitly set your pattern using the expect_string argument.
2. Increase the read_timeout to a larger value.
 

You can also look at the Netmiko session_log or debug log for more information.

We've also seen similar behavior with other commands like show run, show cpu, etc.
The issue appears to occur sporadically and is not tied to a specific command or device.

Do you have any suggestions on how we can resolve this or make Netmiko's prompt detection more robust in these cases?

Thanks again!

[ktbyers]

Can you post your relevant code?

The output indicates something likely went wrong earlier (i.e. your terminating character is not getting detected correctly).

Probably the easiest solution is to add the expect_string argument to the send_command call.

[daniel10059]

Hi, thank you for your fast reply!

We have been using netmiko for our network automation for years and didnt had any problems. Also the cisco_asa code worked well for years.
It seems like this problem occurs since we moved to RHEL9, but i cannot imagine that this would be a problem.

Here is the relevant code (some code is ommited since it is not essential for this problem):

commands = ["show version", "show cpu", "show run"]
de_type = "cisco_asa"
.....

def get_dev_cfg_ssh(crawlerobj, hostname, context, ip, username, password, enablepassword, commands, de_type, jo_id):
    
	use_textfsm = False
    use_genie = False
    use_ttp = False
    ttp_template = None
	
    if shared.job_db[jo_id]['textparser'] == "textfsm":
        use_textfsm = True
    elif shared.job_db[jo_id]['textparser'] == "genie":
        use_genie = True
    elif shared.job_db[jo_id]['textparser'] == "ttp":
        use_ttp = True
        ttp_template = shared.job_db[jo_id]['ttp_template']

	try:
		dev = {
			"host"                  :   ip,
			"username"              :   username,
			"password"              :   password,
			"secret"                :   enablepassword,
			"device_type"           :   de_type,
			"timeout"               :   200,
			"auth_timeout"          :   20,
			"conn_timeout"          :   20,
			"banner_timeout"        :   40,
		}
		
		con = netmiko.Netmiko(**dev)
		
		realhostname = con.find_prompt().lower()
		modus = realhostname.strip()[-1]
		realhostname = realhostname.strip()[:-1]
		logger.debug("Got realhostname = " + realhostname + " in MODUS: " + modus)

		if modus != "#":
			con.enable()

			realhostname = con.find_prompt().lower()
			modus = realhostname.strip()[-1]
			realhostname = realhostname.strip()[:-1]
			logger.debug("Got realhostname = " + realhostname + " in MODUS: " + modus)
		
		for command in commands:
			try:
				response = con.send_command(command, read_timeout = 600.0, use_textfsm=use_textfsm, use_genie=use_genie, use_ttp=use_ttp, ttp_template=ttp_template)
			except Exception as e:
				logger.exception("Exception netmiko-send_command. de_type=" + de_type + " - Command:" + command)
                continue
		
		if use_genie or use_textfsm or use_ttp:
			response_dict = {}
			response_dict['command']=command
			response_dict['de_type']=de_type
			....
		else:
			response_string += realhostname + modus + " " + command + "\n" + response.strip() + "\n"
			logger.debug("Response[:50] : " + str(response.strip()[:50]))
			
	except Exception as e:
		logger.exception("Exception netmiko_module. (get_dev_cfg)")
		continue

[ktbyers]

I assume you are using recent/current version of Netmiko (version 4.4.0 or 4.5.0)?

You might want to do some check post SSH connection to verify you are fully connected and have read everything--some sort of a check command that you know the output for.

Between commands, you might want to check the state again i.e. verify that the previous command is done and you are back to just getting the prompt when you hit enter (i.e. something along the lines of find_prompt() giving you what you would expect).

You might want to consider using the prompt and explicitly setting it as an the expect_string so add to your send_command() call, the expect_string="TEST-FW#" (this is probably where I would most want to focus). Obviously generalized so that it worked for any device prompt--so something like:

my_prompt = con.find_prompt()
pattern=re.escape(my_prompt)

And then use pattern for the expect_string argument in the send_command() call.

Your error above indicates that the device responded much slower than Netmiko expected--basically it looks like the show version wasn't fully completed and Netmiko used part of the show version output (incorrectly) for trying to determine the prompt for the next command (which is obviously not correct).

There is part of this I don't understand (as Netmiko has mechanisms to try to prevent this getting out of sync)...but that is what it looks like is going on.