Use Shifter with Selenium
Wire Shifter's residential and ISP proxies into Selenium WebDriver across Python, Java, JavaScript, C#, and Ruby. Works with Chrome, Firefox, and Edge — pair with selenium-wire for full username-password auth in headless mode.
Quick Start
Install
pip install selenium selenium-wireBasic Usage
# selenium-wire is a drop-in replacement for selenium that adds first-class
# support for authenticated proxies in headless mode.
from seleniumwire import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless=new")
seleniumwire_options = {
"proxy": {
"http": "customer-USERNAME-country-us-sid-123ABC:PASSWORD@p.shifter.io:443",
"https": "customer-USERNAME-country-us-sid-123ABC:PASSWORD@p.shifter.io:443",
"no_proxy": "localhost,127.0.0.1",
}
}
driver = webdriver.Chrome(
options=options,
seleniumwire_options=seleniumwire_options,
)
driver.get("https://ipinfo.io/json")
print(driver.find_element("tag name", "body").text)
# {"ip": "154.16.xxx.xxx", "city": "New York", "country": "US", ...}
driver.quit()Features
Examples
Python + selenium-wire (Sticky Session)
selenium-wire forwards traffic through an in-process MITM, which means username-password proxies just work in headless Chrome. Add a `sid` for a sticky residential IP across the whole session.
from seleniumwire import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import secrets
sid = secrets.token_hex(4)
options = Options()
options.add_argument("--headless=new")
options.add_argument(
"--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
)
proxy_url = (
f"customer-USERNAME-country-uk-city-london-sid-{sid}-ttl-300:"
f"PASSWORD@p.shifter.io:443"
)
driver = webdriver.Chrome(
options=options,
seleniumwire_options={"proxy": {"http": proxy_url, "https": proxy_url}},
)
# Multi-step flow — same residential IP across every navigation.
driver.get("https://example.co.uk/login")
driver.find_element(By.ID, "email").send_keys("user@example.com")
driver.find_element(By.ID, "password").send_keys("secret")
driver.find_element(By.CSS_SELECTOR, "button[type=submit]").click()
WebDriverWait(driver, 10).until(EC.url_contains("/dashboard"))
rows = driver.find_elements(By.CSS_SELECTOR, ".order-row")
print(f"Found {len(rows)} orders on dashboard")
driver.quit()Vanilla Selenium (no extras) — IP Whitelist
If you don't want selenium-wire as a dependency, use Shifter's IP whitelist authentication and configure the proxy via standard ChromeOptions. Cleaner deps but you must whitelist your scrape host.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
options = Options()
options.add_argument("--headless=new")
options.add_argument("--proxy-server=http://p.shifter.io:443")
options.add_argument(
"--user-agent=Mozilla/5.0 (Macintosh) AppleWebKit/537.36"
)
# When using IP-whitelist auth, your scrape host's outbound IP is
# pre-authorized — no username/password needed in the URL.
driver = webdriver.Chrome(options=options)
driver.get("https://example.com")
print(driver.title)
products = driver.find_elements(By.CSS_SELECTOR, ".product")
for p in products[:5]:
title = p.find_element(By.CSS_SELECTOR, "h2").text
price = p.find_element(By.CSS_SELECTOR, ".price").text
print(title, price)
driver.quit()Java with selenium-wire-equivalent (BrowserMob)
Java doesn't have selenium-wire, but BrowserMob Proxy fills the same role — start a local proxy that adds Basic auth headers, then point Selenium at it.
import net.lightbody.bmp.BrowserMobProxy;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import java.net.InetSocketAddress;
public class SeleniumShifter {
public static void main(String[] args) {
BrowserMobProxy proxy = new BrowserMobProxyServer();
proxy.setChainedProxy(new InetSocketAddress("p.shifter.io", 443));
proxy.chainedProxyAuthorization(
"customer-USERNAME-country-de-sid-456DEF", "PASSWORD",
net.lightbody.bmp.proxy.auth.AuthType.BASIC);
proxy.start(0);
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
ChromeOptions options = new ChromeOptions();
options.setProxy(seleniumProxy);
options.addArguments("--headless=new");
WebDriver driver = new ChromeDriver(options);
try {
driver.get("https://example.de");
System.out.println(driver.getTitle());
} finally {
driver.quit();
proxy.stop();
}
}
}Selenium Grid (parallel browsers)
When you scale to a Grid hub with multiple nodes, configure Shifter once at the Capabilities level — every node will route through Shifter without per-node config.
from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType
from selenium.webdriver.chrome.options import Options
# Use selenium-wire-equivalent or Shifter IP whitelist for auth.
proxy_str = "p.shifter.io:443"
shared_proxy = Proxy()
shared_proxy.proxy_type = ProxyType.MANUAL
shared_proxy.http_proxy = proxy_str
shared_proxy.ssl_proxy = proxy_str
options = Options()
options.proxy = shared_proxy
options.add_argument("--headless=new")
# Grid hub
HUB_URL = "http://selenium-hub.internal:4444/wd/hub"
driver = webdriver.Remote(command_executor=HUB_URL, options=options)
try:
driver.get("https://example.com")
print(driver.title, len(driver.page_source), "bytes")
finally:
driver.quit()Frequently asked questions
Common questions about using Shifter with Selenium.
Use selenium-wire (Python) or BrowserMob Proxy (Java). Both run a local MITM proxy that injects Basic auth headers — Chrome and Firefox accept the inner proxy without prompting for credentials, even in headless mode. Alternatively, use Shifter's IP-whitelist auth and skip credentials entirely.
Chrome doesn't display proxy auth dialogs in headless mode and there's no built-in way to feed credentials at the command line. selenium-wire works around this by running a local proxy that handles auth itself. The other option is Shifter's IP-whitelist mode — your scrape host's outbound IP is pre-authorized so no credentials are needed.
Pass --proxy-server=http://p.shifter.io:443 in ChromeOptions. For username-password auth in headless mode, use selenium-wire (Python) or BrowserMob (Java). For visible Chrome, page.authenticate() in the underlying CDP works, but most scrapers use selenium-wire as the cleanest option.
Add a session ID to the proxy username — for example `customer-USERNAME-country-us-sid-123ABC`. Every request through the same WebDriver session will reuse the same residential IP. Add `ttl-N` to pin the IP for up to N seconds.
Yes. Configure the proxy at the Capabilities or ChromeOptions level when creating the Remote WebDriver — every Grid node will route through Shifter. Combine with unique sids per worker to scale parallel scraping without sharing IPs.
If you're starting fresh, Playwright is usually easier — proxy auth works natively, multi-browser support is built in, and the API is cleaner. Selenium remains the right choice for legacy test suites, mobile (Appium), or when your team already standardizes on it. Shifter works equally well with both.
Start Using Shifter with Selenium
Drive Selenium WebDriver through Shifter's 205M+ residential and ISP proxies in Python, Java, and beyond. Native --proxy-server flag, selenium-wire for headless auth, and full Selenium Grid support.