What is the Requests Module?

• HTTP/HTTPS library for Python
• fetch files over the internet
• Abstracts connection details
• Akin to a browser without the user interface

Requests Installation

pip3 install requests

Uniform Resource Locator

• also known as URL or web address
• reference to web pages, files and other internet resources

URL Parts

• http is the protocol
• www.berkeley.com is the host name
• /extension/resources.html is the path
• s=python is the query string (allows to send informtion to the server)

Fetching Internet Resources

user » sending a request » server server » sending a response which contains the resource » client

import request module » prepare request >. send request » get response » process response

Import the Requests Module

import requests

Preparing the Request

• prepare the complete URL in string format
• parameters for name-value entries in the form of a dictionary (which tehen will be sent as a query string)

Sending the Request

• send the request using the get()method

Get Method

import requests
requests.get(url, parameters={})

reteieve the url resource and return a response object parameters are mebedded and encouded in the URL.

Getting the Response

• response object is returned by the get() call
• ifthe server is down or it is is an unknown server, get() will throw a ConnectionError » check URL

import request module » prepare request » send request » get response

Processing the Response

if the GET or the POST call receive a response check the status by using response.ok is True:

• response object will contain the contents of the requested resources otherwise
• examine response.status_code

import request module » prepare request >. send request » get response » process response

Response Properties

• response.ok
  • True when the fetch request is successful
• response.status_code
  • the HTTP status code returned by the server

Common HTTP Status Codes

200	Successful Response	nothing more to do
400	Bad Request	400-499 indicate error on client side
401	Unauthorized
404	Reource Not Found	might also be a typo in url
500	Internal Server Error	there is nothing we can do about

Text Contents

• if we expect text, we should check the text files, using response.text
• usually contains the text encoded, Unicode version in string format

Binary Contents

• for binary files, use response.content
• contains the raw, unencoded contents in byte format
• use the ginary files, such as images, sound, encrypted files, etc.

Response Object Properties

response.content

• the raw contents of the response (byte format)

response.text

• the test encoded contents of the respponse (string format)

Requests Example

import request

url = 'https://www.python.org'
response = requests.get(url)
with open ('index.html', 'w') as html_file:
    if response.ok:
        html_file.write(response.text)
        print('finished printing')

import request

url = 'https://upload.wikimedia.org/wikipedia/en/thumb/8/80/Wikipedia-logo-v2.svg/440px-Wikipedia-logo-v2.svg.png'
response = requests.get(url)
with open ('wiki-logo.png', 'wb') as png_file:
    # set the context manager to download a binary file
    if response.ok:
        png_file.write(response.content)
        print('finished printing')