“Network Protocol” 04. Application layer (DNS & DHCP & HTTP)

title: “Network Protocol” 04. Application layer (DNS & DHCP & HTTP)
date: 2022-09-05 14:28:22
updated: 2023-11-12 06:55:52
categories: Learning Record: Network Protocol
excerpt: Application layer, DNS, DHCP, HTTP (URI & URL, ABNF, message format, request method, status code, header field, Form submission, same-origin policy).
comments: false
tags:
top_image: /images/backimg/SunsetClimbing.png

Network Protocol

  • 1: Network interconnection model
    • 1.1: Request process
    • 1.2: Network layering
  • 2: Application layer
  • 3: DNS
    • 3.1: Domain name
    • 3.2: DNS server
    • 3.3: Related commands
  • 4:DHCP
    • 4.1: IP address allocation
    • 4.2: Supplement
    • 4.3: Related commands
  • 5:HTTP
    • 5.1: URN & URI & URL
    • 5.2:ABNF
    • 5.3: Message format
    • 5.4: Request method
    • 5.5: Status code
    • 5.6: Header fields
      • 5.6.1: Request header fields
      • 5.6.2: Response header fields
    • 5.7: Form submission
    • 5.8: Same-origin policy

Network protocols from entry to underlying principles.

1: Network interconnection model

In order to better promote the research and development of Internet networks, the International Organization for Standardization ISO formulated a network interconnection model in 1985: the OSI Reference Model (Open System Interconnect Reference Model).

  • International Standard: OSI Reference Model (7 layers)
  • Practical application: TCP/IP protocol (layer 4)
  • Study and Research: (Level 5)

1.1: Request process

1.2: Network layering

2: Application layer

Common protocols at the application layer include:

  • Hypertext transfer: HTTP, HTTPS
  • File transfer: FTP
  • Email: SMTP, POP3, IMAP
  • Dynamic host configuration: DHCP
  • Domain Name System: DNS

3: DNS

DNS, Domain Name System, domain name system.

  • Using the DNS protocol, domain names can be resolved into corresponding IP addresses.
  • DNS can be based on UDP protocol or TCP protocol. The service occupies port 53.

3.1: Domain name

Because IP addresses are inconvenient to remember and cannot express the name and nature of an organization, people designed domain names (such as baidu.com).

But in fact, in order to access a specific host, you still have to know the IP address of the target host.

Domain name application registration: https://wanwang.aliyun.com/

Why not just use the domain name instead of the IP address?
The IP address is fixed at 4 bytes, and the domain name is at least 10 bytes. This will undoubtedly increase the burden on the router and waste traffic.

According to different levels, domain names can be divided into:

  • Top-Level Domain (TLD)
  • secondary domain
  • Third level domain name
  • …

General Top-level Domain (General Top-level Domain, gTLD):

.com (Company)
.net (network organization)
.org (organization)
.edu (education)
.gov (Government Department)
.int (international organization)
...

Country Code Top-level Domain (Country Code Top-level Domain, ccTLD)

.cn (China)
.jp (Japan)
.uk (UK)
...

New Generic Top-level Domain (New Generic Top-level Domain, New gTLD)

.vip
.xyz
.top
.club
.shop
...

The second-level domain name refers to the domain name below the top-level domain name.

Under a generic top-level domain, it generally refers to the name of the domain name registrant. For example, google, baidu, microsoft, etc. (for example: google.com, baidu.com, microsoft.com)

Under country and region top-level domain names, it generally refers to the registration category. For example, com, edu, gov, net, etc. (Example: china.com.cn)

3.2: DNS Server

  • The client will first access the nearest DNS server (that is, the DNS server configured by the client itself)
  • All DNS servers record the IP address of the DNS root domain name server
  • The upper-level DNS server records the IP address of the lower-level DNS server.

There are a total of 13 IPv4 DNS root name servers and 25 IPv6 DNS root name servers in the world.

3.3: Related commands

DNS related commands under Windows.

View DNS cache records:
ipconfig /displaydns

Clear DNS cache:
ipconfig /flushdns

Resolving DNS:
nslookup

ping will automatically resolve DNS:
ping

4: DHCP

DHCP (Dynamic Host Configuration Protocol), dynamic host configuration protocol.

The DHCP protocol is based on the UDP protocol. The client uses port 68 and the server uses port 67 (both ports are fixed).

The DHCP server will select an IP address from the IP address pool and “lease” it to the client for a period of time, and then reclaim it when the time expires.

The router that connects to the Internet at home can act as a DHCP server.

4.1: IP address allocation

IP addresses can be divided into static IP addresses and dynamic IP addresses according to the allocation method.

Static IP address:

  • manual setting
  • Applicable scenarios: servers, desktops that are not very mobile, etc.

Dynamic IP Address:

  • Automatically obtain an IP address from a DHCP server
  • Applicable scenarios: mobile devices, wireless devices, etc.

Four stages of IP allocation:

  1. DISCOVER: Discovery server
    • Send broadcast packets (source IP is 0.0.0.0, destination IP is 255.255.255.255, destination MAC is FF:FF:FF:FF:FF:FF)
  2. OFFER: Offer a lease
    • The server returns the IP address that can be leased, as well as lease period, subnet mask, gateway, DNS and other information
    • Note: There may be multiple servers providing leases here
  3. REQUEST: Select IP address
    • The client selects an OFFER and sends a broadcast packet in response
  4. ACKNOWLEDGE: Confirm
    • The selected server sends an ACK packet to the client
    • At this point, the IP address allocation is completed

Image:

4.2: Supplement

Can a DHCP server assign IP addresses across network segments? (DHCP server and client are not in the same network segment)

  • You can use the DHCP Relay Agent to allocate IP addresses across network segments.

Automatic renewal:

  • When the lease period is insufficient, the client will automatically send a REQUEST message to the DHCP server to apply for renewal.

4.3: Related commands

Related commands under Windows.

View DHCP-related details, such as lease expiration time, DHCP server address, etc.:
ipconfig /all

Release the lease:
ipconfig /release

Reapply for an IP address and apply for renewal (extension of lease period):
ipconfig /renew

5：HTTP

HTTP (Hyper Text Transfer Protocol), Hypertext Transfer Protocol.

  • One of the most widely used application layer protocols in the Internet
  • The original purpose of designing HTTP: to provide a method to publish and receive HTML pages, using URI to identify specific resources.
  • Since its development, the data format transmitted by HTTP is not only HTML, but has been widely used.

HTML (Hyper Text Markup Language), hypertext markup language. used to write web pages

HTTP version:

  • 1991, HTTP/0.9
    • Only supports GET request method to obtain text data
    • Request headers, response headers, etc. are not supported
  • 1996, HTTP/1.0
    • Supports POST, HEAD and other request methods
    • Support request headers, response headers, etc.
    • Supports more data types, no longer limited to text data
    • Each request requires establishing a TCP connection with the server, and the TCP connection is immediately disconnected after the request is processed.
  • 1997, HTTP/1.1
    • The most classic and widely used version
    • Supports PUT, DELETE and other request methods
    • Using persistent connection (Connection: keep-alive), multiple requests can share the same TCP connection
  • 2015, HTTP/2.0
  • 2018, HTTP/3.0

HTTP standards:

Coordinated and formulated by the World Wide Web Consortium (W3C) and the Internet Engineering Task Force (IETF), a series of RFCs (Request For Comments) were eventually released.

RFC official website:
https://www.rfc-editor.org

Message format diagram:

5.1: URN & amp; URI & amp; URL

URN (Universal Resource Name, unified resource name)
URL (Uniform Resource Locator, Uniform Resource Locator)
URI (Uniform Resource Identifier, Uniform Resource Identifier)

The URL gives where the resource is located (which host, which path, which folder)
URl gives the location path of a specific resource (starting from the resource itself, giving the reachable path)

# URI syntax rules:
scheme://user:[email protected]:port/path/filename?id=1#ch1

# URL syntax rules:
scheme://host.domain:port/path/filename

# explain:
scheme defines the type of Internet service. The most common type is http
user:pass authentication login information
host defines the domain host (the default host for http is www)
domain defines an Internet domain name, such as baidu.com
port defines the port number on the host (the default port number for http is 80)
path defines the path on the server (if omitted, the document must be in the root of the website)
filename defines the name of the document/resource
id=1 query string
ch1 fragment identifier

URLs are URIs.

URL encoding: Some special characters (such as Chinese, spaces) appearing in the URL need to be URL encoded.

When entering a URL into the browser address bar, it is encoded in UTF-8.

URL Online Encoding:
https://tool.oschina.net/encode?type=4

for example:

Before encoding: https://www.baidu.com/s?wd=Baidu
After encoding: https://www.baidu.com/s?wd=Baidu

5.2: ABNF

ABNF (Augmented BNF)

  • It is a modified and enhanced version of BNF (Backus-Naur Form)
  • As stated in RFC 5234: ABNF is used as a definition language for communication protocols in the internet
  • ABNF is the most rigorous description form of HTTP message format. Talking about HTTP message format without ABNF is often not rigorous.

Core Rules:

5.3: Message format

  • Overall format:

Mark Explanation
/ Choose one
* 0 or more. 2 * means at least 2, 3 * 6 means 3 to 6
() form a whole
[] Optional (dispensable)
  • request-line format:

  • status-line format:

  • header-filed format:

  • message-body format:

  • Definition of HTTP message format:

RFC 2616 4.HTTP Message (old)
RFC 7230 3.Message Format (new)

5.4: Request method

There are currently 9 request methods.

RFC 7231, section 4: Request methods: describes 8 request methods:
GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE

RFC 5789, section 2: Patch method: describes the PATCH method

  • GET：
    Commonly used for reading operations, the request parameters are directly spliced after the URL (browsers have length restrictions on URLs)
  • POST：
    Commonly used for add, modify, and delete operations. Request parameters can be placed in the request body (no size limit)
  • HEAD:
    The request gets the same response as the GET request, but without the response body
  • OPTIONS:
    Used to obtain the communication options supported by the destination resource, such as the request methods supported by the server
  • PUT：
    Used to fully cover existing resources
  • PATCH:
    Used to partially modify resources (the resource does not exist, a new resource will be created)
  • DELETE：
    Used to delete specified resources
  • TRACE：
    Requests the server to echo the request information it receives, mainly used for testing or diagnosis of HTTP requests.
  • CONNECT：
    Can open a two-way communication channel between the client and the requested resource, which can be used to create a tunnel.

5.5: Status code

Status Code. Indicates the HTTP request completion status.

Defined in the RFC 2616 10.Status Code Definitions specification.

Status codes can be divided into 5 categories:

  • 1XX: Information response
  • 2XX: Successful response
  • 3XX: Redirect
  • 4XX: Client error
  • 5XX: Server error

Common status codes:

  • 100 Continue：
    The initial part of the request has been received by the server and has not been rejected by the server. The client should continue sending remaining requests. If the request has already been completed, this response is ignored.
    • Before allowing the client to send a request with a request body, determine whether the server is willing to accept the request.
    • In some cases, it is inappropriate or inefficient for the client to send the request body if the server rejects the request without seeing the request body.
  • 200 OK：
    The request was successful.
  • 302 Found：
    The requested resource is temporarily moved to the URL specified by the Location header.
  • 304 Not Modified：
    Indicates that the requested content does not need to be transferred again, which means that the cached content can be used.
  • 400 Bad Request：
    The server cannot understand the request due to invalid syntax.
  • 401 Unauthorized：
    The authentication credentials required by the target resource are missing.
  • 403 Forbidden：
    The server is capable of processing the request, but is denying authorized access.
  • 404 Not Found：
    The requested resource cannot be found on the server side.
  • 405 Method Not Allowed：
    The server blocked the request using the current HTTP method.
  • 406 Not Acceptable：
    The server cannot provide a response that matches the values specified by Accept-Charset and Accept-Language.
  • 408 Request Timeout：
    The server wants to close connections that are not in use.
    • Some servers send this information on an idle connection, even when the client has not sent any requests
  • 500 Internal Server Error：
    The requested server encountered an unexpected condition that prevented it from executing the request.
  • 501 Not Implemented：
    The requested method is not supported by the server and therefore cannot be processed.
    • The only methods that the server must support (that is, methods that will not return this status code) are GET and HEAD
  • 502 Bad Gateway：
    For servers in the gateway or proxy role, responses received from upstream servers (such as tomcat) are invalid.
  • 503 Service Unavailable：
    The server is not yet in a state to accept the request.
    • This is usually caused by the server being down for maintenance or being overloaded.

5.6: Header fields

Header Field can be divided into 4 types:

  • Request Header Fields
    Message headers with information about the resource to be obtained or the client itself
  • Response Header Fields
    Supplementary information about the response, such as headers from the server itself (name, version, etc.)
  • Entity Header Fields
    More information about the entity’s body, such as its Content-Length or its MIME type
  • General Header Fields
    Applicable to both request and response messages, but headers that are not related to the message body

5.6.1: Request header field

================================================ =====================
Header field name description
================================================== ==================
User-Agent The browser's identification string.
-------------------------------------------------- ------------------
Host server domain name and port number.
-------------------------------------------------- ------------------
Date The date and time the message was sent.
-------------------------------------------------- ------------------
Referer represents the previous page visited by the browser,
                   It's a link on that page
                   Brings the browser to the currently requested page.
-------------------------------------------------- ------------------
Content-Type request body type
-------------------------------------------------- ------------------
Content-Length The length of the request body (in bytes)
-------------------------------------------------- ------------------
Accept response content types that can be accepted
-------------------------------------------------- ------------------
Accept-Charset character set that can be accepted
                   The larger the value of q, the higher the priority
                   Defaults to 1.0 (maximum value)
-------------------------------------------------- ------------------
Accept-Encoding list of acceptable encoding methods
-------------------------------------------------- ------------------
Accept-Language A list of natural languages that can be accepted for response content
-------------------------------------------------- ------------------
Range requests only a portion of an entity.
                   Byte offsets start with 0
-------------------------------------------------- ------------------
Origin initiates a request for cross-origin resource sharing
-------------------------------------------------- ------------------
Cookie Cookie previously sent by the server through Set-Cookie
-------------------------------------------------- ------------------
Connection The type of connection this browser wants to use in preference
-------------------------------------------------- ------------------
Cache-Control is used to specify instructions that all caching mechanisms in this request/response chain must comply with.
================================================== ==================

5.6.2: Response header fields

================================================ =====================
Header field name description
================================================== ==================
Date The date and time the message was sent
-------------------------------------------------- ------------------
Last-Modified The last modified date of the requested object
-------------------------------------------------- ------------------
Server The name of the server
-------------------------------------------------- ------------------
Expires specifies a time after which the response is considered to have expired.
-------------------------------------------------- ------------------
Content-Type response body type
-------------------------------------------------- ------------------
Content-Encoding The encoding type used by the content
-------------------------------------------------- ------------------
Content-Length The length of the response body (in bytes)
-------------------------------------------------- ------------------
Content-Disposition A header that allows the client to download the file and suggest a file name
-------------------------------------------------- ------------------
Accept-Ranges What kinds of partial content ranges does the server support?
-------------------------------------------------- ------------------
Content-Range Which part of the complete message does this partial message belong to?
-------------------------------------------------- ------------------
Access-Control-Allow-Origin specifies which websites can participate in cross-domain resource sharing
-------------------------------------------------- ------------------
Location to redirect, or when a new resource is created
-------------------------------------------------- ------------------
Set-Cookie returns a Cookie for the client to save
-------------------------------------------------- ------------------
Connection Options expected for this connection
-------------------------------------------------- ------------------
Cache-Control informs all caching mechanisms from server to client that
                               Whether they can cache this object. (unit is seconds)
================================================== ==================

5.7: Form submission

Common attributes:

  • action: requested URI
  • method: request method (GET, POST)
  • enctype: When making a POST request, the encoding method of the request body
    • application/x-www-form-urlencoded (default)
      Use [& amp;] to separate parameters, use [=] to separate keys and values, and use URL encoding to encode characters.
    • multipart/form-data
      This encoding must be used when uploading files

multipart/form-data: refer to RFC 1521

5.8: Same-origin policy

The browser has a Same-Origin Policy. It stipulates that by default, AJAX requests can only be sent to URLs with the same origin.

The same origin refers to three identical ones: Protocol, Domain name (IP), and Port.

Tags such as img, script, link, iframe, video, and audio are not subject to the same origin policy.

CORS (Cross-Origin Resource Sharing, cross-domain resource sharing), a common method to solve AJAX cross-domain requests.

The implementation of CORS requires support from both the client and the server.

  • Client: basically all browsers support it
  • Server: Need to return the corresponding response header (such as Access-Control-Allow-Origin) to inform the browser that this is a cross-domain request.

It is easy to distinguish between pure and turbid, but difficult to distinguish between evil and righteousness.

–“Bing Jian” (Qing Dynasty) Zeng Guofan