python - Getting json from HTML

Question

import requests
import json
from bs4 import BeautifulSoup

user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_0) AppleWebKit/537.36 (KHTML, like Gecko) ' 
             'Chrome/87.0.4280.141 Safari/537.36'

params = {
    'userId': 'e1a025bebfce19f66b8c33e19405e565',
    'url': 'https://etherscan.io/tokenholdingsHandler.aspx?&a=0x3f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be&q=&p=1&f=0&h=0&sort=total_price_usd&order=desc&pUsd24hrs=1220.72&pBtc24hrs=0.03852&pUsd=1348.12&fav=&langMsg=A%20total%20of%20XX%20tokenSS%20found&langFilter=Filtered%20by%20XX&langFirst=First&langPage=Page%20X%20of%20Y&langLast=Last&ps=25'
}
with requests.Session() as session:
    session.headers = {
        'User-Agent': user_agent,
        'content-type': 'application/json',
        'Sec-Fetch-Site': 'same-origin',
        'Sec-Fetch-Mode': 'cors',
        'Sec-Fetch-Dest': 'empty',
        'referer': 'https://etherscan.io/tokenholdings?a=0x626D20125da6a371aA48023bF9dad94BD66588F7',
        'Accept-Language': 'en-US,en;q=0.9',
    }
resp = session.get('http://api.ascraper.com/crawl', params=params)
ans = resp.json()
d = json.loads(ans['html_source'])

Trying to parse the json response from the above page. The dict d has different keys, the information I want is in the key recordsfound. The problem is that the value of the key(recordsfound) is all HTML code. Is there a way to get that data as json?

question from:https://stackoverflow.com/questions/65872132/getting-json-from-html

Answer 1

Waitting for answers