A socket ResourceWarning in Selenium Python WebDriver

[segevfiner]

🐛 Bug Report

You get a ResourceWarning for a socket from Selenium Python WebDriver.

To Reproduce

Save as "resourcewarning.py":

#!/usr/bin/env python3
import sys
from selenium import webdriver


def main():
    driver = webdriver.Firefox()
    try:
        driver.get("https://www.google.com/")
    finally:
        driver.quit()


if __name__ == "__main__":
    sys.exit(main())

And run it:

C:\>python -Wd -X tracemalloc=25 resourcewarning.py
sys:1: ResourceWarning: unclosed <socket.socket fd=732, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 57420), raddr=('127.0.0.1', 57413)>
Object allocated at (most recent call last):
  File "resourcewarning.py", lineno 15
    sys.exit(main())
  File "resourcewarning.py", lineno 7
    driver = webdriver.Firefox()
  File "<snip>\lib\site-packages\selenium\webdriver\firefox\webdriver.py", lineno 174
    keep_alive=True)
  File "<snip>\lib\site-packages\selenium\webdriver\remote\webdriver.py", lineno 157
    self.start_session(capabilities, browser_profile)
  File "<snip>\lib\site-packages\selenium\webdriver\remote\webdriver.py", lineno 252
    response = self.execute(Command.NEW_SESSION, parameters)
  File "<snip>\lib\site-packages\selenium\webdriver\remote\webdriver.py", lineno 319
    response = self.command_executor.execute(driver_command, params)
  File "<snip>\lib\site-packages\selenium\webdriver\remote\remote_connection.py", lineno 374
    return self._request(command_info[0], url, body=data)
  File "<snip>\lib\site-packages\selenium\webdriver\remote\remote_connection.py", lineno 397
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "<snip>\lib\site-packages\urllib3\request.py", lineno 72
    **urlopen_kw)
  File "<snip>\lib\site-packages\urllib3\request.py", lineno 150
    return self.urlopen(method, url, **extra_kw)
  File "<snip>\lib\site-packages\urllib3\poolmanager.py", lineno 323
    response = conn.urlopen(method, u.request_uri, **kw)
  File "<snip>\lib\site-packages\urllib3\connectionpool.py", lineno 600
    chunked=chunked)
  File "<snip>\lib\site-packages\urllib3\connectionpool.py", lineno 354
    conn.request(method, url, **httplib_request_kw)
  File "<snip>\python37\Lib\http\client.py", lineno 1229
    self._send_request(method, url, body, headers, encode_chunked)
  File <snip>\python37\Lib\http\client.py", lineno 1275
    self.endheaders(body, encode_chunked=encode_chunked)
  File "<snip>\python37\Lib\http\client.py", lineno 1224
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "<snip>\python37\Lib\http\client.py", lineno 1016
    self.send(msg)
  File "<snip>\python37\Lib\http\client.py", lineno 956
    self.connect()
  File "<snip>\lib\site-packages\urllib3\connection.py", lineno 181
    conn = self._new_conn()
  File "<snip>\lib\site-packages\urllib3\connection.py", lineno 159
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "<snip>\lib\site-packages\urllib3\util\connection.py", lineno 61
    sock = socket.socket(af, socktype, proto)

Expected behavior

I expect this script to exit with no output, but it spits out a ResourceWarning.

Analysis

This is caused by not calling clear on the urllib3.PoolManager created at:

selenium/py/selenium/webdriver/remote/remote_connection.py

Line 140 in cac262f

self._conn = urllib3.PoolManager(timeout=self._timeout)

Environment

OS: Windows 10.0.17763.253 x64
Browser: Firefox
Browser version: 64.0.2 (64-bit)
Browser Driver version: GeckoDriver 0.23.0
Language Bindings version: Python 3.141.0

[cgoldberg]

This is caused by not calling clear on the urllib3.PoolManager created at:

the line you are referring to is only conditionally called when keep_alive is True:

if keep_alive:
    self._conn = urllib3.PoolManager(timeout=self._timeout)

so wouldn't your suggestion to call clear on the urllib3.PoolManager close the sockets? (which would disable keep-alive)

I don't think you can have it both ways. If you use keep-alives, a byproduct is that you may leave open connections. or am I missing something?

[segevfiner]

You need to call clear only once you close the remote connection, as in WebDriver.quit. Of course you shouldn't call it while it's still in use as that will drop the keep alove connection which is the whole point of creating the PoolManager in the first place.

[cgoldberg]

can u submit a PR?

[GQAssurance]

I have created a PR for this: #6941

[AutomatedTester]

This appears to have been fixed so closing

[JoanStar]

Seems like this only fix the issue while keep_alive = True, then how about when keep_alive=False i saw the same issue
do we need to clear this as well?
http = urllib3.PoolManager(timeout=self._timeout)

[GQAssurance]

Seems like this only fix the issue while keep_alive = True, then how about when keep_alive=False i saw the same issue
do we need to clear this as well?
http = urllib3.PoolManager(timeout=self._timeout)

Good catch. It looks like a separate PoolManager is being created when KeepAlive=False but it is not being cleared.

I can make a similar PR for this.