← Go Back

How to Open and Read URLs

Python's standard library provides the urllib.request module to open and read URLs. Reading the content of a website is quite similar to reading a file:

from urllib.request import urlopen

# Open URL.
r = urlopen("https://pythonassets.com")
# Read the content and print its size.
print(len(r.read()))
# Close to release resources.
r.close()

To save content to a file:

r = urlopen("https://pythonassets.com")
with open("index.html", "wb") as f:
    f.write(r.read())
r.close()

On the other hand, the Python community has adopted the Requests library as the preferred method for handling HTTP input and output. It is a higher level package than urllib and with greater features. Using Requests, the previous code is translated as follows.

import requests

r = requests.get("https://www.recursospython.com")
with open("index.html", "wb") as f:
     f.write(r.content)
r.close()

Requests is simply installed using pip:

pip install requests

url requests http

🐍 You might also find interesting: