Chrome process still running in background after driver.quit()

[MacMarde]

It happens very often to me that the chrome processes are still running in background after I perform a driver.quit() to close chrome.
It is happening about 50% of the time and I can not find any pattern for it. I do also get no error message of any kind.
The chrome process is just still running endless and requires a permanent high cpu usage.
I have no idea what to do.
Selenium, UC and Chrome are all latest versions installed.
I tried downgrading everything. Still no luck.

I found someone having a similar issue here, but it did not help me.

Please help me, I can not use UC in this condition.

[MacMarde]

I have found a workaround solution.
Before I call driver.quit() I am already killing the chrome process and all child processes.

os.system(r'C:\Test\kill.bat ' + str(driver.browser_pid))
driver.quit()

This is the Batch file:

set arg1=%1
taskkill /f /T /pid %arg1%

So I get the process ID from UC and kill this process with taskkill. The parameter /T also kills all child processes.
So other chrome instances should not be affected.

I think something is going wrong with UC and driver.quit(). Would still love to see it fixed.

[ultrafunkamsterdam]

chrome is a bitch with its background processing.

[MacMarde]

chrome is a bitch with its background processing.

ohh yes surely it is

[Mister-Stein]

Unset the --no-sandbox Chrome argument, setting no_sandbox argument of uc.Chrome to False, because it is set to True by default. Example:

uc.Chrome(driver_executable_path=driver_executable_path, options=options, user_data_dir=user_data_dir, no_sandbox=False, user_multi_procs=True, use_subprocess=False)

But some things to consider:
Users say that undetected-chromedriver doesn't work without --no-sandbox Chrome argument on Windows 7 and lower, but works on Windows 8.1 and above. I didn't tested that for myself, but I think it's true.
Also, undetected-chromedriver most of the times doesn't work on Linux without --no-sandbox Chrome argument(when running test script as root user for example). So, that's because no_sandbox argument of uc.Chrome is set to True by default. Concluding that, set no_sandbox argument of uc.Chrome to False for Windows, and set to True for Linux.
But I warn that undetected-chromedriver has breached no_sandbox functionality, so --no-sandbox is always set, whether or not you set it to True or False. I've made pull request to fix it, so we need to wait until it is applied(or download my fork, and use as custom Python module). #1542

[claell]

I am having the same problems. Adding a driver service to the driver (as suggested as an apparently working solution for some cases for Java), didn't help me, either.

So for now, I am having the script kill the remaining processes as a workaround. Not nice, but it seems to work robustly.

[dany-kuznetsov]

Hey guys, I made an investgation on this problem.

I got a problem that such background processes did not close and have a huge CPU consumption, and my machine just fail when parsing.

So I tried to investigate how to differentiate valid chrome processes, and such garbage chrome processes to kill them.

Examples of opened valid working chromes:
(1)

PID: 12272, Name: chrome.exe
Command line arguments: ['C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe', '--type=renderer', '--user-data-dir=C:\\Users\\Admin\\AppData\\Local\\Temp\\scoped_dir14300_478364913', '--no-appcompat-clear', '--no-sandbox', '--remote-debugging-port=0', '--test-type=webdriver', '--allow-pre-commit-input', '--disable-gpu-compositing', '--disable-blink-features=AutomationControlled', '--lang=en-US', '--device-scale-factor=1', '--num-raster-threads=4', '--enable-main-frame-before-activation', '--renderer-client-id=27', '--time-ticks-at-unix-epoch=-1713990424237820', '--launch-time-ticks=13579176720', '--field-trial-handle=4628,i,14198820591187631884,15916917488837001709,262144', '--variations-seed-version', '--enable-logging=handle', '--log-file=4712', '--log-level=0', '--mojo-platform-channel-handle=4856', '/prefetch:1']
User data directory: C:\Users\Admin\AppData\Local\Temp\scoped_dir14300_478364913
Renderer client ID: 27

(2)

PID: 11024, Name: chrome.exe
Command line arguments: ['C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe', '--type=renderer', '--user-data-dir=C:\\Users\\Admin\\AppData\\Local\\Temp\\scoped_dir3240_1902735128', '--no-appcompat-clear', '--no-sandbox', '--remote-debugging-port=0', '--test-type=webdriver', '--allow-pre-commit-input', '--disable-gpu-compositing', '--disable-blink-features=AutomationControlled', '--lang=en-US', '--device-scale-factor=1', '--num-raster-threads=4', '--enable-main-frame-before-activation', '--renderer-client-id=6', '--time-ticks-at-unix-epoch=-1713990424237385', '--launch-time-ticks=13539603038', '--field-trial-handle=2984,i,14322924025893536792,584304756356687822,262144', '--variations-seed-version', '--enable-logging=handle', '--log-file=2996', '--log-level=0', '--mojo-platform-channel-handle=2992', '/prefetch:1']
User data directory: C:\Users\Admin\AppData\Local\Temp\scoped_dir3240_1902735128
Renderer client ID: 6

Example of not valid background high CPU chromes:

! PID: 14704, Name: chrome.exe
Command line arguments: ['C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe', '--type=renderer', '--user-data-dir=C:\\Users\\Admin\\AppData\\Local\\Temp\\scoped_dir2028_1039597504', '--no-appcompat-clear', '--no-sandbox', '--remote-debugging-port=0', '--test-type=webdriver', '--allow-pre-commit-input', '--disable-gpu-compositing', '--disable-blink-features=AutomationControlled', '--lang=en-US', '--device-scale-factor=1', '--num-raster-threads=4', '--enable-main-frame-before-activation', '--renderer-client-id=7', '--time-ticks-at-unix-epoch=-1713990424237678', '--launch-time-ticks=13569893498', '--field-trial-handle=3380,i,2295610200889357938,9837504487891314831,262144', '--variations-seed-version', '--enable-logging=handle', '--log-file=3452', '--log-level=0', '--mojo-platform-channel-handle=3448', '/prefetch:1']
User data directory: C:\Users\Admin\AppData\Local\Temp\scoped_dir2028_1039597504
Renderer client ID: 7

So copilot explain how to diffirinciate these profiles:

From the command line arguments you provided, it's not immediately clear how to distinguish between the "valid" and "not valid" Chrome processes. They all appear to be renderer processes controlled by WebDriver for automated testing.

However, one potential difference is the --renderer-client-id argument. In the "valid" processes, this argument has the values 6 and 29, while in the "not valid" processes, it has the value 7. This argument is likely used by Chrome to distinguish between different renderer processes, but it's not clear how these IDs are assigned or what they mean.

Another potential difference is the --user-data-dir argument. This specifies the directory where user data is stored. If different tests or test suites use different user data directories, you could use this argument to distinguish between them.

Unfortunately, the status 7 actually exists for opened valid chromes. So we need a second filtering attribute that we can derive from processes. I found out that such garbage processes a grouped with a cound of 2, but an opened valid browser has more processes.

And finally here is a code how to find all not valid chrome process with --renderer-client-id as 7. Also it takes only processes where dir group has only 2 processes. And kill them:

import os
import shutil
import schedule
import time
import numpy as np
import sys

import pandas as pd
import psutil
import time
from datetime import datetime, timedelta

import pandas as pd
import psutil

p = psutil.Process(os.getpid())
p.nice(psutil.HIGH_PRIORITY_CLASS)


def extract_process_data(proc):
    # Initialize the data for this process
    data = {
        'pid': proc.info['pid'],
        'name': proc.info['name'],
        'create_time': proc.info['create_time']
    }
    # Extract the renderer client ID and user data directory
    for arg in proc.info['cmdline']:
        if arg.startswith('--renderer-client-id='):
            data['renderer-client-id'] = arg.split('=')[1]
        elif arg.startswith('--user-data-dir='):
            data['user-data-dir'] = arg.split('=')[1]
    return data


def add_dir_group_count(df):
    # Count the number of processes per user-data-dir
    dir_group_count = df.groupby('user-data-dir').size()
    # Add the count to the DataFrame
    df = df.merge(dir_group_count.rename('dir_group_count'), left_on='user-data-dir', right_index=True)
    return df


def kill_processes(pids):
    # Iterate over the process IDs and kill each process
    for pid in pids:
        try:
            proc = psutil.Process(pid)
            proc.kill()
            print(f"Killed process {pid}")
        except psutil.NoSuchProcess:
            print(f"Process {pid} does not exist")
        except Exception as e:
            print(f"Error killing process {pid}: {e}")


def kill_chrome_garbage_processes():
    try:
        print(f'{datetime.now()} Start killing Chrome garbage processes')
        # Initialize an empty list to store process data
        process_data = []
        # Iterate over all running processes
        for proc in psutil.process_iter(['pid', 'name', 'cmdline', 'create_time']):
            # Filter out non-Chrome processes
            if 'chrome' in proc.info['name']:
                # Extract necessary data from each Chrome process
                data = extract_process_data(proc)
                # Add the data to the list
                process_data.append(data)
        if len(process_data) == 0:
            print('No Chrome processes found')
            return
        # Convert the list of process data into a DataFrame
        df = pd.DataFrame(process_data)
        # Add a column to the DataFrame that counts the number of processes per user-data-dir
        df = add_dir_group_count(df)
        df['seconds_exist'] = time.time() - df['create_time']
        print(df['seconds_exist'].mean())
        # Fill missing renderer-client-id values with the first non-null value in the same user-data-dir group
        df['renderer-client-id'] = df['renderer-client-id'].fillna(
            df.groupby('user-data-dir')['renderer-client-id'].transform('first'))
        # Get a list of process IDs that need to be killed
        df = df.sort_values(['dir_group_count', 'user-data-dir']).reset_index(drop=True).copy()
        print(df[['pid', 'dir_group_count', 'user-data-dir', 'renderer-client-id']].shape)
        # & (df['renderer-client-id'].apply(str).isin(['7', '30', '31', '32',])))

        pids_to_kill = df.loc[
            (df['dir_group_count'].apply(str).isin(['1', '2', '3', '4', '5', '1.0', '2.0', '3.0', '4.0', '5.0']))
            , 'pid'].tolist()
        print(f"PIDS to kill all {len(pids_to_kill)}", pids_to_kill)
        pids_to_kill = df.loc[
            (df['dir_group_count'].apply(str).isin(['1', '2', '3', '4', '5', '1.0', '2.0', '3.0', '4.0', '5.0']))
            & (df['seconds_exist'] > 10)
            , 'pid'].tolist()
        print(f"PIDS to kill only old {len(pids_to_kill)}", pids_to_kill)
        # Kill the processes
        kill_processes(pids_to_kill)
    except Exception as e:
        print(e)

schedule.every(2).seconds.do(kill_chrome_garbage_processes, )

# Run the scheduled job indefinitely
while True:
    schedule.run_pending()
    time.sleep(1)

It my first answer comment :) Please like this comment to inspire me to do further public contribution and specifically help you with this exact issue :)

P.S. For the start it helps and close background garbage chromes, but after few iterations such processes stopped to be killed
P.S.S. Resolved, code above updated.
P.S.S.S. But still I do not know the reason why the still running in the background and does not close. If you know how to solve the problem (not just like a work-around above) please let me know.

[DMIYTRO]

"The argument --no-sandbox is used when launching the Chrome browser through Selenium WebDriver. This argument disables the use of the 'sandbox' in Chrome. The 'sandbox' is an additional security layer that isolates the browser from the rest of the system to prevent malicious programs or viruses from affecting your system through the browser.

However, sometimes using this feature can cause issues when launching Chrome through Selenium, especially in environments without a GUI (for example, on a server without a graphical interface). Therefore, adding --no-sandbox to the browser launch arguments via WebDriver can be helpful in such cases.

However, caution should be exercised when using this argument, as it may reduce the security level of your system."

[RileyXX]

You need to call driver.close() before driver.quit() otherwise you get lingering chrome processes with high resource usage. This is an issue with recent chrome versions (124+).

[herickmota]

You need to call driver.close() before driver.quit() otherwise you get lingering chrome processes with high resource usage. This is an issue with recent chrome versions (124+).

this only works if you are running a single test. when you are running multiple tests in sequence, the second scenario will fail with HTTPConnectionPool error:

HOOK-ERROR in after_scenario: MaxRetryError: HTTPConnectionPool(host='localhost', port=45171): Max retries exceeded with url: /session/8d44b82bed7f7b2d65236fe1a0e83290/window (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7d8c5fb91d70>: Failed to establish a new connection: [Errno 111] Connection refused'))