python - How to get data from the chart using beautifulsoup - authorization problem?

Question

Welcome To Ask or Share your Answers For Others

python - How to get data from the chart using beautifulsoup - authorization problem?

asked Oct 7, 2021 in Technique[技术] by 深蓝 (71.8m points)

python - How to get data from the chart using beautifulsoup - authorization problem?

I have such a problem. Zeversolar has stopped providing the API about a year ago. And now there is no way to download the data in a simple way for me. It is not possible to generate a csv or other file. You can only rewrite it yourself. I can download this data from zevercloud but I know how to do it from chart. I have created a read-only user in my account so that you can see what I mean.

The data I would like to get from the chart is Generation value and Date.

I found a way to get data from a chart using the link:

https://www.zevercloud.com/data/energy/amountm/2021-01?sid=62923

But i have Message: 403 - Access is forbidden

    root@changeme:/home/root# python3 test.py

<html>
<head>
<title>403 - Access is forbidden</title>
<meta content="width=device-width, initial-scale=1.0" name="viewport"/>
</head>
<body>
<div><h1>Access is forbidden</h1></div>
</body>
</html>

My code:

    import requests
    from bs4 import BeautifulSoup


    login_url = 'https://www.zevercloud.com/login'
    data = {
        'User-Agent': 'Mozilla / 5.0 (Windows NT 10.0; Win64; x64) AppleWebKit / 537.36 (KHTML, jak Gecko) Chrome / 74.0.3729.169 Safari / 537.36',
        'username': 'xxx',
        'password': 'xxx'
    }

    with requests.Session() as s:
        r1 = requests.post(login_url , data) 
        r2= s.get('https://www.zevercloud.com/data/energy/amountm/2021-01?sid=62923')
        soup = BeautifulSoup(r2.text, 'html.parser')
        print(soup)

Can anyone help me how to properly login to the website and download the data ??

question from:https://stackoverflow.com/questions/65938248/how-to-get-data-from-the-chart-using-beautifulsoup-authorization-problem

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…

1 Answer

深蓝 · Answer 1 · 2021-10-06T18:57:25+0000

Try this:

import json

import requests

with requests.Session() as s:
    r1 = s.post(
        'https://www.zevercloud.com/login',
        data=dict(username='[email protected]', password='Password123'),
    )
    r2 = s.get('https://www.zevercloud.com/data/energy/amountm/2021-01?sid=62923').json()
    print(json.dumps(r2, indent=4))

Output:

[
    {
        "isno": null,
        "sid": 62923,
        "byname": "Generation",
        "qdate": "2021-01",
        "unit": "kWh",
        "dataset": [
            {
                "time": "2021-01-01",
                "value": [
                    "2.2"
                ],
                "consump": 0.0,
                "mexport": 0.0,
                "mimport": 0.0,
                "net": 0.0
            },
            {
                "time": "2021-01-02",
                "value": [
                    "1.8"
                ],
                "consump": 0.0,
                "mexport": 0.0,
                "mimport": 0.0,
                "net": 0.0
            },
            {
                "time": "2021-01-03",
                "value": [
                    "2.4"
                ],
                "consump": 0.0,
                "mexport": 0.0,
                "mimport": 0.0,
                "net": 0.0
            },
and so on ...

Categories

python - How to get data from the chart using beautifulsoup - authorization problem?

python - How to get data from the chart using beautifulsoup - authorization problem?

Please log in or register to add a comment.

Please log in or register to answer this question.

1 Answer

Please log in or register to add a comment.

Just Browsing Browsing

Most popular tags