The Internet & Web Requests

CIS1902 Python Programming

Reminders

HW2 has been released! It is due in about 2.5 weeks on Friday November 1.

Agenda

  1. Servers? Clients? Some Definitions
  2. The Lifecycle of a Web Request
  3. Frontend/backend paradigms
  4. Chess.com API Lab

What is a server?

A server is a machine that listens for external requests and responds accordingly. In the context of the web:

  • Typically manages an underlying database
  • External requests follow the HTTP(S) protocol
  • The types of requests allowed follow an API, i.e. a specified set of actions

What is a client?

A client refers to whoever or whatever is initiating the web request. This could be:

  • The human (you!) behind the computer
  • The browser you're using
  • The website currently loaded in your browser (more on this later)
  • Another server making requests
  • Etc

What is HTTP?

HTTP(S) stands for the HyperText Transfer Protocol (Secure), which is a protocol built on top of TCP (Transmission Control Protocol). TCP is a protocol that ensures computers are able to successfully send arbitrary messages between each other.

HTTP simply provides some framework around what types of messages should be sent assuming a Create-Read-Update-Delete (CRUD) model.

Bird's Eye TCP

Consider a scenario where Alice wants to be penpals with Bob. How should she initiate this?

  1. [SYN] Alice sends Bob her address in a letter and asks to be penpals.
  2. [SYN-ACK] Bob agrees to be penpals and sends his acknowledgement as a letter to Alice.
  3. [ACK] Alice sends a final letter to Bob saying she received his acknowledgement and they can now start to be penpals.

This three-step handshake is used in TCP to establish connection.

HTTP Methods

An HTTP request at the very least has two properties: a URL and a method.

The method provides some abstraction to the client to understand what their request should be doing, i.e. reading data or writing data.

Depending on the method, an HTTP request contains parameters or data. Parameters can be thought of as variables used in a function and data is typically more free-form, i.e. a JSON blob.

HTTP Methods

Method Description
GET Retrieves a resource from the server. Should not modify server state.
POST Submits data to be processed by the server, often creating a new resource.
PUT Updates an existing resource on the server with the provided data.
DELETE Removes the specified resource from the server.
HEAD Similar to GET but retrieves only the headers, not the body. Useful for checking resource metadata.
OPTIONS Requests information about the communication options available for the target resource.
PATCH Partially modifies an existing resource with the provided data.
TRACE Performs a message loop-back test along the path to the target resource.
CONNECT Establishes a tunnel to the server identified by the target resource.

API

An API is an Application Programming Interface. In short, it's an abstraction of how to connect to some service. Technically, modules that we import in Python (e.g. pandas or numpy) are an API! However, most people mean web APIs when they use the term.

Web APIs are commonly exposed through HTTP methods that allow clients to do CRUD operations.

Most services have well documented public APIs, so documentation is always the first place to look when exploring.

Lifecycle of a Web Request

Lifecycle of a Web Request

What happens when you type in netflix.com into your browser?

  1. Client formulates HTTP(S) request and sends it to Internet Service Provider (ISP), e.g. Comcast/AT&T/Verizon.
  2. ISP does DNS resolution to determine IP of web address
  3. HTTP(S) request is sent to server
  4. Server does some work and returns a response
  5. Client parses response and displays to user

Lifecycle of a Web Request

lifecycle of web request

https://hackmd.io/@Web-Fundamentals-ML/r115J-Vpu

Frontend & Backend Paradigms

What is a Frontend/Backend?

A frontend is a term used for some GUI-like interface. Typically, these are meant for humans to use. Frontends can make requests to an external source that is typically a backend.

A backend is a term used for some data source, typically exposed through an API. This could be something like a raw SQL-database or a server that provides HTTP methods and queries a database behind the scenes.

Vanilla Website

This is how our course website is made! The backend is a simple filesystem that exposes a directory to the internet. Think of this as an online folder. There is no explicit frontend besides pure HTML files. The client simply GETs the HTML files and they are rendered on the browser.

This is great for serving static content, but can't really do much beyond that. However, it makes it very easy and intuitive to develop new content since one can change the HTML files directly.

Standalone Frontend

A more advanced paradigm is developing a standalone frontend. The backend acts simply as an API that provides CRUD access to a database. The frontend is an entire application typically written in some native language, e.g. Javascript for browsers, Swift for iOS, Kotlin for Android.

The very first request typically delivers the frontend application code to the browser. The frontend then makes API requests on a user's behalf to render content nicely or provide intuitive GUIs for actions. The sky is the limit but requires much more work.

Backend-Driven Frontend

An in-between paradigm can sometimes work well. Here, the backend renders HTML files in real time to deliver to the client. This way, the backend can dynamically change content based on parameters, e.g. user, location, etc. However, can get messy as you typically need to write generic-style HTML or use a templating language. Additionally, it takes extra effort to create a standalone API.

Example: Netflix could have a template for displaying a list of recommendations. They could use this template to render HTML for different webpages, e.g. horror movies, action movies, TV shows, etc.

Chess.com API Lab

https://www.chess.com/news/view/published-data-api