HTTP
Introduction
The Hypertext Transfer Protocol (HTTP) is the foundation for the World Wide Web.
Unless you've been living under a rock for the past 30 years, you have probably been using HTTP pretty much every day (maybe without knowing). If you are reading this through a web browser for example, you are using HTTP.
It is an application communication protocol allowing a client to present a 'request' to a server, which then sends back a 'response'.
HTTP Request/Response cycle
The point of these requests and responses at the end of the day is to enable systems to easily query resources on remote systems.
Methods
HTTP defines a bunch of standard 'methods' (sometimes also called 'verbs') that can be requested against resources:
- GET
- PUT
- POST
- PATCH
- DELETE
- and many more...
The first few are really the heart of the modern web. Viewing a page like the one you are reading right now involves issuing a 'GET' request to the corresponding webserver for the specific resource path (or locator).
Other methods serve different purposes: 'POST' is typically used to request the creation of a new resource, while 'DELETE', as the name implies, is mostly used to request the deletion of a given resource.
Headers
Just as we've seen above, HTTP allows us to access resources across the internet both for reading and writing. As such, we need a way to enforce some kind of authentication. We want to be able to verify that the originator of any request is in fact allowed to carry out the operation.
In addition to this problem, we will also want to add some meta-information about our request to give the server more information about exactly how to process it. Examples of this include the desired response format, management of the underlying connection, or data caching parameters.
To address all of these issues, HTTP was designed so that both requests and responses can (and usually do) contain 'headers' - additional information in the form of key-value pairs joined to the message.
One rather common header that you probably have already heard about is the 'Cookie' header.
URL's
Part of HTTP's power is the ability to access resources through a text-based 'locator' - a kind of address which basically tells us not only where the resource is (the 'locator' part, after the "://"), but also how to access it (the 'protocol' part, before the "://").
For example, let's consider the following URL, which points to the current page on the Dooba website:
We can break it down into smaller parts:
- 'https' - Protocol to use ('how' to access the resource)
- '://' - Delimiter between protocol and location parts
- 'dooba.io' - Host (and port, implicit in this case: 443 for 'https' protocol) on which the resource is to be found
- '/pages/http' - Path to the actual resource on the host
When we combine all of this, we get this simple string of text which allows anyone to easily access the same resource.
Structure of an HTTP request
No time for pointless debates
Over the past years, one of the top technical debates among IT engineers has been the distinction between the concepts of "URL", "URI" and "URN". When it comes to Dooba, let us put an end to this nonsense right now. Addresses in the form 'protocol://resource-location' will be referred to as URLs, that's it. The original definitions from the RFCs are ambiguous and this is why we have been debating their meaning ever since.
Response codes
The key part of the response message is the HTTP response code (sometimes referred to as 'status code'), which indicates to the originator of a request whether it was successful.
Structure of an HTTP response
Some commonly known HTTP response codes are:
- 200 OK
- 301 Moved Permanently
- 302 Found (Moved Temporarily)
- 401 Unauthorized
- 404 Not Found
- 500 Server Error
The http library
Enough with the theory.
How can we use HTTP to access the web with Dooba? The answer lies in the http library.
This library allows us to create both clients and servers that communicate over HTTP. This works by using network sockets - go have a quick look if you're not yet familiar with the concept.
Because it relies on the generic socket system of Dooba, HTTP can be used seamlessly over any network device, provided it integrates with Dooba Sockets (like the ESP8266 WiFi modules for example).
Basic usage
As always, let's start by adding the http library to our dependencies in dfe.conf:
# ... deps: - http
Once this is done, we can start using things from the '<http/client.h>' and '<http/server.h>' header files.
Both client and server features will require us to create a structure to hold information about our client / server. After initializing the structure through calls to http_client_* / http_server_ * functions, we can basically wait for something to happen (either actively or in the background).
The examples below present how to use both the client and server features. To keep things simple, they will assume a socket interface is available as 'wl0' (this should be provided by the substrate).
HTTP client example
1 #include <http/client.h> 2 3 #include "substrate.h" 4 5 // Our HTTP Client 6 struct http_client hc; 7 8 // Main Initialization 9 void init() 10 { 11 struct http_arg *a; 12 uint8_t i; 13 14 // Init Substrate 15 substrate_init(); 16 17 // POST some form data to 'http://eresse.net/example.php' 18 if(http_post(&hc, wl0, "http://eresse.net/example.php?param1=%s", "test-value", 19 20 // Pass in some headers 21 HTTP_CLIENT_HEADER, "User-Agent", "Example HTTP Client V.%i", 1, 22 HTTP_CLIENT_HEADER, "X-Example-Header", "Some %s: %i", "value", 42, 23 24 // Add some form fields 25 HTTP_CLIENT_FORMDATA, "name", "User #%i", 12, 26 HTTP_CLIENT_FORMDATA, "likes", "raccoons", 27 28 // Always end with 0 29 0)) { scli_printf("ERROR! Failed to send HTTP request :(\n"); return; } 30 31 // Actively wait for request to complete & Check Failed 32 http_client_wait(&hc); 33 if(http_client_failed(&hc)) { scli_printf("ERROR! Failed to complete HTTP request :(\n"); return; } 34 35 // Display Response Info 36 scli_printf("Response code: %i (%t)\n", http_client_res_status(&hc), http_client_res_status_txt(&hc), http_client_res_status_txt_l(&hc)); 37 scli_printf("Headers (%i):\n", http_client_res_head_count(&hc)); 38 for(i = 0; i < http_client_res_head_count(&hc); i = i + 1) 39 { 40 a = http_client_res_head_i(&hc, i); 41 scli_printf(" * [%t] -> [%t]\n", http_arg_name(&hc, a), http_arg_name_l(&hc, a), http_arg_val(&hc, a), http_arg_val_l(&hc, a)); 42 } 43 scli_printf("Body: %i bytes\n", http_client_res_body_l(&hc)); 44 scli_printf(" ------------\n%t\n------------\n", http_client_res_body(&hc), http_client_res_body_l(&hc)); 45 } 46 47 // Main Loop 48 void loop() 49 { 50 // Update Substrate 51 substrate_loop(); 52 }
Daemonizing (running in background)
There will be cases when we don't want to waste time actively waiting for the response, but rather continue our program's execution flow while letting the request complete in the background. Thanks to [Discover_Dooba#eloop|eloop] this is trivial. We can even specify a callback function to be executed whenever the request completes.
1 #include <http/client.h> 2 3 #include "substrate.h" 4 5 // Our HTTP Client 6 struct http_client hc; 7 8 // On Request Complete (callback) 9 void on_req_complete(void *user, struct http_client *c) 10 { 11 struct http_arg *a; 12 uint8_t i; 13 14 // Check Failed 15 if(http_client_failed(c)) { scli_printf("ERROR! Failed to complete HTTP request :(\n"); return; } 16 17 // Display Response Info 18 scli_printf("Response code: %i (%t)\n", http_client_res_status(c), http_client_res_status_txt(c), http_client_res_status_txt_l(c)); 19 scli_printf("Headers (%i):\n", http_client_res_head_count(c)); 20 for(i = 0; i < http_client_res_head_count(c); i = i + 1) 21 { 22 a = http_client_res_head_i(c, i); 23 scli_printf(" * [%t] -> [%t]\n", http_arg_name(c, a), http_arg_name_l(c, a), http_arg_val(c, a), http_arg_val_l(c, a)); 24 } 25 scli_printf("Body: %i bytes\n ------------\n%t\n ------------\n", http_client_res_body_l(c), http_client_res_body(c), http_client_res_body_l(c)); 26 } 27 28 // Main Initialization 29 void init() 30 { 31 // Init Substrate 32 substrate_init(); 33 34 // POST some form data to 'http://eresse.net/example.php' 35 if(http_post(&hc, wl0, "http://eresse.net/example.php?param1=%s", "test-value", 36 37 // Pass in some headers 38 HTTP_CLIENT_HEADER, "User-Agent", "Example HTTP Client V.%i", 1, 39 HTTP_CLIENT_HEADER, "X-Example-Header", "Some %s: %i", "value", 42, 40 41 // Add some form fields 42 HTTP_CLIENT_FORMDATA, "name", "User #%i", 12, 43 HTTP_CLIENT_FORMDATA, "likes", "raccoons", 44 45 // Always end with 0 46 0)) { scli_printf("ERROR! Failed to send HTTP request :(\n"); return; } 47 48 // Daemonize (complete request in the background) 49 http_client_daemonize(&hc, on_req_complete, 0); 50 } 51 52 // Main Loop 53 void loop() 54 { 55 // Update Substrate 56 substrate_loop(); 57 }
! WARNING ! When daemonizing a request, we must make sure the client structure (struct http_client hc;
in our case) is a global variable. If we use a local variable (stored on the stack, in the frame of the executing function), it will get trashed as soon as we return from the function, and the background processing of the request will fail.
HTTP server example
1 #include <http/server.h> 2 3 #include "substrate.h" 4 5 // Our HTTP Server 6 struct http_server hs; 7 8 // HTTP Request Handler 9 void req_handler(void *user, struct http_server_client *c) 10 { 11 uint8_t i; 12 char ip[NET_IP_ADDR_LEN_TXT]; 13 uint16_t port; 14 struct http_arg *a; 15 16 // Display Request Info 17 scli_printf("Got HTTP Request! Details below:\n"); 18 19 // Get Client Info 20 if(socket_get_peer(c->conn.sck, ip, &port)) { scli_printf("ERROR: Failed to get peer info :(\n"); } 21 else { scli_printf(" * Client: %s:%i\n", ip, port); } 22 23 // Display Request Details (Method + URL) 24 scli_printf(" * Method: %s (%t)\n", http_code2meth(http_server_req_meth_code(c)), http_server_req_meth(c), http_server_req_meth_l(c)); 25 scli_printf(" * URL: %t\n", http_server_req_url(c), http_server_req_url_l(c)); 26 scli_printf(" - Path: %t\n", http_server_req_url_path(c), http_server_req_url_path_l(c)); 27 if(http_server_req_url_arg_count(c)) { scli_printf(" - Args: %t\n", http_server_req_url_args(c), http_server_req_url_args_l(c)); } 28 for(i = 0; i < http_server_req_url_arg_count(c); i = i + 1) 29 { 30 a = http_server_req_url_arg_i(c, i); 31 scli_printf(" %t -> %t\n", http_arg_name(c, a), http_arg_name_l(c, a), http_arg_val(c, a), http_arg_val_l(c, a)); 32 } 33 34 // Display Request Headers 35 scli_printf(" * Headers:\n"); 36 for(i = 0; i < http_server_req_head_count(c); i = i + 1) 37 { 38 a = http_server_req_head_i(c, i); 39 scli_printf(" - %t -> %t\n", http_arg_name(c, a), http_arg_name_l(c, a), http_arg_val(c, a), http_arg_val_l(c, a)); 40 } 41 42 // Display Form Fields 43 if(http_server_req_field_count(c)) { scli_printf(" * Form Fields:\n"); } 44 for(i = 0; i < http_server_req_field_count(c); i = i + 1) 45 { 46 a = http_server_req_field_i(c, i); 47 scli_printf(" - %t -> %t\n", http_arg_name(c, a), http_arg_name_l(c, a), http_arg_val(c, a), http_arg_val_l(c, a)); 48 } 49 50 // Display Body 51 if(http_server_req_body_l(c)) { scli_printf(" * Raw body: %i bytes\n ------------\n%t\n ------------\n", http_server_req_body_l(c), http_server_req_body(c), http_server_req_body_l(c)); } 52 53 // Respond with 200 - OK 54 http_server_respond(c, HTTP_RES_OK); 55 56 // Set a 'Server' header 57 http_server_header(c, HTTP_HEADER_SERVER, "Example HTTP Server v.%i.%i", 1, 3); 58 59 // Add a body - say hello! 60 http_server_resp_body(c, "Hello world!"); 61 } 62 63 // Main Initialization 64 void init() 65 { 66 // Init Substrate 67 substrate_init(); 68 69 // Create HTTP Server on Port 80 & Daemonize (run it in background) 70 if(http_server_start(&hs, 80, wl0, req_handler, 0)) { scli_printf("ERROR! Failed to create HTTP Server :(\n"); return; } 71 http_server_daemonize(&hs); 72 } 73 74 // Main Loop 75 void loop() 76 { 77 // Update Substrate 78 substrate_loop(); 79 }