python - Broken pipe error selenium webdriver, when there is a gap between commands?

Question

Ubuntu 18.x + selenium webdriver(Firefox)

Facing a weird problem, the following block works if I run all of it together

from selenium import webdriver
url = 'https://indiamart.com'
driver = webdriver.Firefox()
driver.get(url)
driver.find_element_by_xpath(xpath).click()

This is happening with every url I have tried.

However if I execute one line at a time, it gives

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 326, in get
    self.execute(Command.GET, {'url': url})
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/remote_connection.py", line 472, in execute
    return self._request(command_info[0], url, body=data)
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/remote_connection.py", line 495, in _request
    self._conn.request(method, parsed_url.path, body, headers)
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1065, in _send_output
    self.send(chunk)
  File "/usr/lib/python3.6/http/client.py", line 986, in send
    self.sock.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe

This is the error on the code

driver.get(url)

However if I execute the same line again after the Broken Pipe error it works and gets the url.

I am very very confused. Can someone help me out.

Thanks

Answer 1

This is a known bug of the latest build v0.21.0 of geckodriver matched with the latest version of selenium v3.11

To work around this bug either: a) downgrade geckodriver to v0.20.1 or earlier b) wait for the bugfix/mitigations be rolled out in the upcoming versions of selenium and/or geckodriver

This bug originates from newly added support in v 0.21 of Keep-Alive feature. However, the default timeout from geckodriver in 0.21 is set to 5s, which is exceptionally short.

This bug is tracked here for geckodriver and here for selenium.