beautifulsoup - 【NavigableString Does Not Have Find Method】Web Scraping LinkedIn Using Selenium

Question

This is my first post in stack-overflow.

Straight to the point, I was trying to mimic this post:

Extracting job information from LinkedIn Jobs using BeautifulSoup and Selenium

And I also want to sign up, and that is my code:

driver.find_element_by_xpath("/html/body/header/nav/div/a[2]").click()
sleep(3)
driver.find_element_by_xpath("""//*[@id="username"]""").send_keys("xxx")
driver.find_element_by_xpath("""//*[@id="password"]""").send_keys("xxx")
driver.find_element_by_xpath("""//*[@id="app__container"]/main/div[2]/form/div[3]/button""").click()
sleep(5)

driver.get(url) 

sleep(5)

So far so good. However, when I reach this code: "for job in job_container:" The job has a type of NavigableString, but it should be a Tag.

My suspicion is that this line didn't read the correct tag:

job_container = lxml_soup.find('ul', class_ = 'jobs-search-results__list list-style-none')  

But I am not sure what is correct.

Can someone help me, please? Thank you.

question from:https://stackoverflow.com/questions/65929097/navigablestring-does-not-have-find-method-web-scraping-linkedin-using-selenium

Answer 1

Update:

I add one more line before the for loop, and it works:

job_container = job_container.find_all('li', class_ = 'jobs-search-results__list-item')

Don't know why, it just does.