Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
254 views
in Technique[技术] by (71.8m points)

javascript - Can't open many tabs with the same link using express puppeteer

P/S : I change "networkidle0" to "networkidle2" and it works perfect, thanks to vsemozhebuty who help me in a comment of the topic.

I'm coding a web scraping using Express & puppeteer . Below code is a part of my code , it is trying to visit a page and reload if getting an error.

       // getting itemid from many requests and do task below

        let browser = await puppeteer.launch();
        let page = await browser.newPage();

        url = 'https://tiki.vn/' + itemid ;

       await page.goto(url, {waitUntil: 'networkidle0'})
                .catch(function (error) {  throw new Error(error); page.reload({ waitUntil: ["domcontentloaded"]}); 
        });

        const tikiproduct = await page.evaluate( () => { 
            
                 // Doing some actions to scrape data
                
        });

        await browser.close();
       
       return new Promise( resolve => { resolve(tikiproduct); });

But i only visit the link successful 1-2 times. Other visits always throw error : " TimeoutError: Navigation timeout of 30000 ms exceeded " . I tried to set "timeout : 0" but the page is still pending without any errors. Sounds like last visit not finished or cleared but i don't know how to fix it. Hope you guys help me in this problem, thanks for reading.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Here the "TimeoutError" definiton on puppeteer docs https://pptr.dev/#?product=Puppeteer&version=v5.5.0&show=api-class-timeouterror

It is a very simple error but there is not enough code sample to say where is the problem.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...