python - Wrong HTML Loading on Etsy -- Can't find element with Selenium

Question

I've been trying to automate a messaging system with Etsy. Basically, it scrapes the link to go to my "messages" page where a dialogue box appears for you to input your subject & message to the seller.

I'm trying to access this dialogue box but Selenium is unable to find the element. I printed out the html code from the page_source, and the html doesn't include the element I'm looking for despite the fact that I'm on the same page.

I don't think this is a separate window/frame because it's a div in the HTML code.

Would really appreciate any help in figuring out how to load the proper source_page code.

Note: There's a message button on the original page, but I was also unable to access it. So I switched to using this redirected page in order to message the seller.

url = 'https://www.etsy.com/ca/listing/839100239/pink-lip-tint-cheek-and-lip-tint-pink?ga_order=most_relevant&ga_search_type=all&ga_view_type=gallery&ga_search_query=green+beauty&ref=sr_gallery-1-2&organic_search_click=1'
wd.get(url)
last_height = wd.execute_script("return document.documentElement.scrollHeight;")  

time.sleep(3)

#first scroll to load 1/3 rd of the page
wd.execute_script('window.scrollTo(0, arguments[0]);',last_height/3)
time.sleep(3)

#scroll to the next 1/3 rd of the page
wd.execute_script('window.scrollTo(0, arguments[0]);',last_height/3 + last_height/3)
time.sleep(3)

wd.execute_script('window.scrollTo(0, arguments[0]);',last_height)
time.sleep(3)


html = wd.page_source

#Click Message Seller & get redirected
message_link = wd.find_element_by_xpath('//*[@id="desktop_shop_owners_parent"]/div/a').get_attribute('href')
wd.get(message_link)
time.sleep(10)

#if the html2 tag is the problem, I've already tried html on it's own and was unsuccessful. 
html2 = wd.page_source
print(html2) #to figure out page source code

#subject heading
subject_box = wd.find_element_by_xpath("//*[@id='subject_id90aa8785-2af1-43f3-ad52-8f6061df3022']")
subject_string = 'hello'
subject_box.send_keys(subject_string)

#content 
content_box = wd.find_element_by_xpath('//*[@id="message0d3ff3ab-4515-469b-bc19-f819f19a03fa"]')
content_string = 'hello'
content_box.send_keys(subject_string) 

seller_name = wd.find_element_by_xpath('//*[@id="wt-modal-container"]/div[3]/div/div/div[2]/div[1]/div[1]/p/strong').get_text(strip=True)

#send
wd.find_element_by_xpath('//*[@id="wt-modal-container"]/div[3]/div/div/div[2]/div[2]/div[2]/button').click()
time(3)

#confirmation
print('messaged: ' + seller_name)

wd.quit()

Thank you!

question from:https://stackoverflow.com/questions/65952976/wrong-html-loading-on-etsy-cant-find-element-with-selenium

Answer 1

Waitting for answers