Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
851 views
in Technique[技术] by (71.8m points)

r - Package "rvest" for web scraping https site with proxy

I want to scrap a https website, but I failed.

Here is my code:

require(rvest)
url <- "https://www.sunnyplayer.com/de/"
content <- read_html(url)

But I have error in console- "Error in open.connection(x, "rb") : Timeout was reached" How I can fix this problem?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

The same thing happens to me on a proxy. To get around this, use download.file and specify a download location. You can then parse the file with read_html.

download.file(url, destfile = 'C://whatever.html')
content <- read_html('C://whatever.html')

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...