Python selenium how to scrape list of values from website

Question

I have a list of job roles on this website that I want to scrape. The code I am using is below:

driver.get('https://jobs.ubs.com/TGnewUI/Search/home/HomeWithPreLoad?partnerid=25008&siteid=5012&PageType=searchResults&SearchType=linkquery&LinkID=6017#keyWordSearch=&locationSearch=')
job_roles = driver.find_elements(By.XPATH, '/html/body/div[2]/div[2]/div[1]/div[6]/div[3]/div/div/div[5]/div[2]/div/div[1]/ul/li[1]/div[2]/div[1]/span/a')

for job_roles in job_roles:
    text = job_roles.text
    print(text)

With this code, I am able to retrieve the first role which is: Business Analyst - IB Credit Risk Change

I am unable to retrieve the other roles, can someone kindly assist

Thanks

question from:https://stackoverflow.com/questions/65545642/python-selenium-how-to-scrape-list-of-values-from-website

Answer 1

In this case all the job names have the two CSS classes jobProperty and jobtitle.

So, since you want all the jobs, I recommend selecting by CSS selector.

The following example should work:

driver.get('https://jobs.ubs.com/TGnewUI/Search/home/HomeWithPreLoad?partnerid=25008&siteid=5012&PageType=searchResults&SearchType=linkquery&LinkID=6017#keyWordSearch=&locationSearch=')

job_roles = driver.find_elements_by_css_selector('.jobProperty.jobtitle')
for job_roles in job_roles:
    text = job_roles.text
    print(text)