r - Retrieve the list of files from a URL

Question

I would like to get a list of all the files available at this address: http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/ (publicly available data from the NOAA).

It would be some sort of "list.files" for the a specific URL. I started to take a look at RCurl but all I could get was the HTML code of the URL.

Answer 1

In this case you can simply use readHTMLTable:

readHTMLTable("http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/", 
              skip.rows=1:2)[[1]]$Name -> file.list

Then to create a list of paths:

paste("http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/", 
      file.list[!is.na(file.list)], sep="") -> path.list