download - Using wildcards in wget or curl query

Question

Is it possible to use wildcards in wget queries when downloading from directories? Basically, I have a site, say, www.download.example.com/dir/version/package.rpm. Howevery, the version directory changes all the time and sometimes contains multiple RPM packages. Is there a single wget query that would get me only and all the packages in the version directory without knowing what that version is? In my ideal world, the query would be something like:

wget www.download.example.com/dir/*/*.rpm

If there is a way to do this with curl, that would work as well.

Answer 1

You can't use wildcards in wget but the -A flag should work. From the wget manpage:

You want to download all the gifs from a directory on an http server. You tried wget http://www.server.com/dir/*.gif, but that didn't work because http retrieval does not support globbing. In that case, use: wget -r -l1 --no-parent -A.gif http://www.server.com/dir/

Edit: found a related question

Regarding directories:

There's a utility called LFTP, which has some support for globbing. Take a look at the manpage. There's another question on Linux & Unix that covers its usage in a scenario similar to yours.