java - Running Chromedriver on Ubuntu Server headlessly

Question

I am using Selenium with Chromedriver in Java (1.8) to do some automated web crawling:

System.setProperty("webdriver.chrome.driver", "chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("someurl.com");

I am attempting to migrate to an Ubuntu 16.04 Server. On the server I have installed the Ubuntu chromedriver version 2.37, chrome version 65. According to the chromedriver documentation these versions are compatible. I have changed the code specifying the location of the chromedriver in Ubuntu:

System.setProperty("webdriver.chrome.driver", "/usr/lib/local/chromedriver");

Prior to running my program I start up xvfb:

Xvfb -ac :99 -screen 0 1280x1024x16 &
export DISPLAY=:99

Then execute my program:

java -jar myprogram.jar

The following is printed to the console:

Starting ChromeDriver 2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7) on port 10574
Only local connections are allowed.
Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally
  (Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.4.0-112-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 60.35 seconds
Build info: version: '3.11.0', revision: 'e59cfb3', time: '2018-03-11T20:33:15.31Z'
System info: host: 'ubuntu-s-1vcpu-1gb-nyc1-01', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-112-generic', java.version: '1.8.0_151'
Driver info: driver.version: ChromeDriver
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
        at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
        at org.openqa.selenium.remote.JsonWireProtocolResponse.lambda$new$0(JsonWireProtocolResponse.java:53)
        at org.openqa.selenium.remote.JsonWireProtocolResponse.lambda$getResponseFunction$2(JsonWireProtocolResponse.java:91)
        at org.openqa.selenium.remote.ProtocolHandshake.lambda$createSession$0(ProtocolHandshake.java:123)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
        at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
        at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464)
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:126)
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:73)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:136)
        at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545)
        at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:209)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:132)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:181)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:168)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:123)
        at RHio.Test.main(Test.java:39)

Answer 1

The error does gives us some idea about whats wrong happening :

Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally

As you are trying to initiate Chromedriver and Chrome on Ubuntu Server headless you have to pass a couple of Options as follows :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path='/path/to/chromedriver')
driver.get('http://google.com/')
driver.quit()

Additional Notes

As per the mandatory Requirements and #1341 ensure the following :

• Chromedriver is within /usr/local/bin/
• Chrome Browser is within /usr/bin/google-chrome

---

tl;dr

The story of Sandbox