python - Beautiful soup get tag values that follow after the matched tag value

Question

I have a code where I need to look for the div tag that has the value 'Lead' , once that tag is found in the code it looks for the immediate div with value 'name' and then extracts the email id in the immediate next tag, such that my code outputs the name and the email id as follows

John Doe [email protected]

 <div class="badge memberType Member long notAffiliatedContact">Lead</div>
 <div class="name"><a href="/10016/John Doe">John Doe</a></div>
 <div class="posted"><span class="label">Email: </span><span class="value break-word">[email protected]</span></div>

Thanks a lot in advance.

question from:https://stackoverflow.com/questions/65948266/beautiful-soup-get-tag-values-that-follow-after-the-matched-tag-value

Answer 1

You can find the next element using findNext().

from bs4 import BeautifulSoup

html = """
<div class="badge memberType Member long notAffiliatedContact">Lead</div>
 <div class="name"><a href="/10016/John Doe">John Doe</a></div>
 <div class="posted"><span class="label">Email: </span><span class="value break-word">[email protected]</span></div>
 """
soup = BeautifulSoup(html, 'html.parser')
divs = soup.findAll("div", {"class": "badge"})
for div in divs:
    if div.text == "Lead":
        name = div.findNext('div')
        email = name.findNext('span').findNext('span')
print(name.text, email.text)
#John Doe [email protected]