Microcontroller system with the HTTP protocol [STM32Nucleo]
Mechanotoro
If the microcontroller is equipped with a TCP server, which I explain in the section "Socket communication with TCP server", it can easily be developed into a web server configured with the HTTP protocol. The advantage of a system equipped with a web server is that it can be accessed from a browser, allowing communication from any terminal, including PCs and smartphones, without the need to install a dedicated application, making it easy to implement IoT, such as remote control management.
This time, I will try to mount a web server with minimum functions on MCU with a small memory like NUCLEO-F103RB.
I also handled a web server with the ESP32 WiFi module, but since I only followed the HTTP driver format provided by the manufacturer, I was able to access it from a browser without being aware of the protocol contents. This time, I would like to build a simplified server with a higher degree of freedom using our knowledge of the general HTTP protocol.
Table of contents
What is the HTTP Protocol?
There are many articles on this subject, so I will not attempt to explain it in detail here. I have tried to summarize the essentials of the minimum necessary knowledge by focusing on the points that are likely to cause stumbling blocks when configuring a web server with MCU.
HTTP is a protocol on top of TCP that exchanges HTTP messages called HTTP requests and HTTP responses, but under the surface, invisible to the eye, data communication is conducted using TCP.
HTTP is a protocol in which a server sends back a response after receiving a request from a client such as a browser. In other words, the server does not return data without a request from the client, even if the data is periodically updated.
Web server to be mounted on MCU
Normally, a web server is a computer system with a file system that stores HTML pages, CGI programs, image files, etc., and responds to the client's instructions by passing pages, executing programs, and responding to the browser.
In contrast, in the case of MCU without a file system, the server cannot create individual files, so instead it encodes all data into a static string (hard coding), stores it in an array, and displays it in the browser via string transmission.
A server without a file system cannot handle image files, so it is limited to a simple, simplified look and feel, but even with minimal functionality, it is significant to be able to access MCU from a browser and manage it remotely, etc.
MCU sends an HTML file to the browser that is hard-coded and stored in an array; the HTML file is essentially static, but it can be combined with dynamic numbers, etc., or a script file can be embedded and executed.
HTTP format
HTTP is a web-only protocol for exchanges between browsers and servers. The client browser sends a request message and the web server sends back a response message.
The HTTP protocol is formatted with a header described in text plus the data you want to send and receive.
① HTTP request
The first line of an HTTP request is the request line, which contains information like a command line with methods. A message header follows, conveying information such as browser type and data type.
An empty line indicates the end of the header, followed by the message body, which contains the data to be conveyed. It may be empty.
The code below is the request message when you enter the URL of the web server you wish to access from your browser. The first line is the request line, starting with the method GET, followed by the URL you want to access, etc.
The message header then conveys information about the browser. There is no data in this request, so the body is empty.
The above code is an access request to the URL 192.168.3.100:50000 using the GET method. The URL may include even the path name of the file to be accessed, but if it is omitted, the HTML of the top page is requested.
If the port number is other than the HTTP protocol's specified port number 80, specify the port number.
② HTTP response
The first line of the HTTP response is the status line, which conveys the results of the web server's processing.
It is followed by a message header that conveys information about the server software, the data type of the response, the compression method, and the length of the data to be passed.
After a blank line, the message body contains HTML and other data.
The HTTP response describes information about the web server. If you want to send a page written in HTML in the header field, set "text/html" in Content-Type. The next Content-Length specifies the data length of the message body. These settings are important for the correct display of the content in the browser.
The message body is the content of a page, etc., described in HTML. The above code is the same as sending an HTML file in the file system, which is a string of HTML encoded (hard-coded) from MCU to the browser.
The GET method is specified (method="get") for the submit button in the input form from the browser.
HTTP method
HTTP method is a process by which a client makes a request to a web server. There are several types of HTTP methods, but two main types are handled in the exchange between the browser and MCU: the GET method and the POST method.
The GET method is used by the browser to retrieve HTML files and other data from the server, and the POST method is used by the browser to send data to the server. The GET method, like the POST method, can also be used to send data, but in that case the data is appended to the URL.
① WEB page access by GET method
Standard GET method usage. When a browser (client) enters the URL of a web page, it first requests a TCP connection to the server.
When the server side accepts a TCP connection request from the client side and establishes a connection (connection), the exchange begins using the HTTP protocol.
The browser sends a request message to the server beginning with the GET method request line.
The server receives a GET request and returns the following response
HTTP/1.1 200 OK
The browser opens an HTML page or accesses a specified file depending on the contents of the GET request.
② Sending data by GET method
Data transmission format using the GET method. Send "Data = xxx" by adding "?" To the end of the URL in the address field of the browser.
The server can get the request message as receive data in the following format, so all you have to do is process and extract the method "GET" and the data "ABC".
Sending data by the GET method is a very easy method, but it is problematic for sending sensitive data because of the history in the URL field.
③ Sending data by POST method
Basically, the POST method is used to send data; unlike the GET method, no browser history is kept.
The server can get request messages as receive data in the following format. The data is in the body after the blank line where the request header ends.
The amount of data received is larger than with "GET" because the method "POST" and the data "ABC" in the body are processed and extracted after all requests are received.
Implementation on MCU
The following is a summary of important points when implementing a web server in MCU system.
WEB server loading program
It is not much different from the server. The data sent/received will have a slight tail peculiar to the HTTP protocol, so additional processing will be required.
This is a program outline of a WEB server on an STM32. HTTP protocol-specific contents are written in the module http_server.c, and the header file http_server.h is read.
Here, one RTOS task prvTask_monitor is added as a demonstration program to display data of the server in a browser, and the data values are counted up by one per second in it.
There are several important points in implementing a web server on MCU, but one of the most important is memory handling, because the difference between a TCP server and a web server is the long strings it handles. When using FreeRTOS, it is important to set a larger stack capacity (about 384 words: at least 1 kbyte) for tasks that handle long strings.
WEB Server Module
The WEB server module below is the core of the WEB server for this demonstration. The top page that is invoked by entering the URL in the browser is the content of the array top_document[]. This is the page when the GET method request does not contain any data.
A separate page that displays messages when the string sent matches or does not match the pre-defined string "ABC" is the contents of the array top_document2[]. This page is for when the GET method request contains data.
The most important point in encoding an HTML file and storing it in an array is the use of the sprintf function to combine the unchanging static code with dynamic code such as "len_top_document," which is code length data that varies depending on the content. This allows HTML, which should be a static file, to become a page with dynamic elements.
The response to a request from a browser is fixed to "HTTP/ 1.1 200 OK" here for simplicity. Strictly speaking, the HTTP protocol returns a status code that is consistent with the browser's request version or assigned a status code according to its status.
If Japanese characters are garbled in the browser when displayed, this should be addressed on the server side. In the above code, I was able to solve this problem by simply changing the character encoding from UTF-8 to UTF-8N, but this is a coping mechanism, so please apply a more fundamental solution.
Receive character processing
Since the request message from the browser contains GET, POST, and other messages and data, it must first be processed to extract them accurately. These are processed in the task prvTask_TCP.
In this demonstration program, the GET method is used to send data, so when "GET" is received as a command, the process branches depending on whether the request has data or not.
Since Nucleo-F103RB does not have a very large memory, it is advisable to review the memory allocation section of the FreeRTOS configuration file FreeRTOSConfig.h if a WEB server is to be installed on FreeRTOS. In particular, configTOTAL_HEAP_SIZE, which specifies the heap size, can be set small if the number of tasks used is small. Adjust it to the extent that it works.
Communication between browser and microcontroller
Now that the Nucleo-F103RB is equipped with a web server, actually enter the URL IP address and port into the browser to access it.
Entering the URL establishes a connection with the server and switches to the following page. The contents are coded in the array top_document[]. The number in the lower left corner shows the data counted up every second in MCU. To check the current value, refresh the browser manually.
Enter text in the following form screen and send it to the server using the GET method. Enter the characters "ABC" and press the Send button. This button is set to the GET method.
To make the input text ambushable so that it does not remain in the history, simply change the HTML input tag from <input type="text"> to <input type="password">. This can be used to create a simple password request screen. However, the sending method must be POST.
When the submit button is pressed, the URL/?cmd=ABC is automatically entered into the address field of the browser and the page switches to the HTML page stored in the array top_document2[]. If the entered characters match "ABC", the following message is displayed.
Now enter "abc" and click the submit button, and the URL/?cmd=abc will automatically be entered into the address field of your browser and the page will switch. If the entered characters do not match "abc", the following message will be displayed.
The above confirmed that MCU can be accessed from a browser to send strings and to display data from MCU. This demo program is simple in its functionality, but if you can handle it freely, you will be able to manage your MCU remotely from your browser, which is an important element. It is also good training for handling strings in C language.
The values sent from the server to the browser are updated manually, but they can also be updated automatically with HTML meta refresh tags. The HTTP protocol basically requires a request from the client to update data, but there are other ways to display data in real time, such as server sent events (SSE) and WebSocket, another protocol. Since it is too difficult for a microcomputer-based web server without a file system, automatic updating may be sufficient for the time being.
In order to handle the HTTP protocol, it is important to check the actual raw data (packets) being exchanged between the client and server. "Fiddler" is a free software tool that monitors the packets exchanged between the browser and the microcontroller, and is useful for program development and troubleshooting.
At this point, I would like to realize something advanced with a file system on a MCU and a program on a web server. As long as there is a file system, I would like to challenge this if there is an opportunity, since it is easier to do advanced things since the general web server method can be adopted.
