Home:ALL Converter>is there a way to download multiple files using the requests module

is there a way to download multiple files using the requests module

Ask Time:2020-06-29T23:29:53         Author:katkak

Json Formatter

I want to download multiple .hdr files from a website called hdrihaven.com. My knowledge of python is not that great but here is what I have tried so far:

import requests

url = 'https://hdrihaven.com/files/hdris/'
resolution = '4k'
file = 'pump_station' #would need to be every file

url_2k = url + file + '_' + resolution + '.hdr'
print(url_2k)

r = requests.get(url_2k, allow_redirects=True)
open(file + resolution + '.hdr', 'wb').write(r.content)

Idealy file would just loop over every file in the directory.

Thanks for your answers in advance!

EDIT

I found a script on github that does what I need: https://github.com/Alzy/hdrihaven_dl. I edited it to fit my needs here: https://github.com/ktkk/hdrihaven-downloader. It uses the technique of looping through a list of all available files as proposed in the comments.

I have found that the requests module as well as urllib are extremly slow compared to native downloading from eg. Chrome. If anyone has an idea as to how I can speed these up pls let me know.

Author:katkak,eproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/62641516/is-there-a-way-to-download-multiple-files-using-the-requests-module
yy