FTP File Upload for ESP8266 Arduino

Surprisingly enough, there is no official FTP library for Arduino. I required this functionality for a client’s project and ended up developing a small library that allows to upload files from our file-system to an FTP server.

Implementing this protocol is very educational, as exposes the inner workings of the internet without too much complexity.

We will start with defining what we want.

#include <Arduino.h>

#include <ESP8266WiFi.h>

#include <string>

class ftp_client {
    public:
        using port_t = int;

        ftp_client(
                const IPAddress server_ip, 
                port_t server_port, 
                const std::string user, 
                const std::string password
                );

        bool upload_file(
            const std::string&amp; local_path,  
            const std::string&amp; destination_path
        ) const;
}

Basically we want an interface that allows us to create a ftp_client connected to a server and a upload_file member function.

Let’s checkout how the FTP protocol works. The full specification can be found in: https://tools.ietf.org/html/rfc959

FTP works over TCP. This means we just need to dive up to TCP level of abstraction. Levels of abstractions are the fundamental guideline for developing systems that scale up.

TCP basically allows to send and receive packets of information to a destination. We will need a tool to debug up to this level of abstraction. After some googling, Filezilla and PacketSender seem to be our tools. With
Filezilla we can easily connect to the FTP server and with PacketSender we can establish a TCP connection and manually send and receive packets.

Luckily, there is a TCP client library developed for ESP8266 Arduino. With a quite rather misleading name (WiFiClient), these available functionality should give us all the tools required to implement our FTP client.

We also need an FTP server to tests around. I created one in SiteGround, we could also create one using our local computer. Steps on how to set up a local FTP can be found here

Now that we have all the tools ready, let’s figure out how it works. From the documentation we can see that we basically need to establish a TCP connection in order to be able to send commands. We will use this

PI stands for “Protocol Interpreter” and DTP for “Data Transfer Process”.

As we can see we have two communication channels. The FTP Commands TCP connection is established in the beginning. The data connection will be used for sending the data.

Lets try opening a TCP connection to the FTP server and see what we get. For that we just need to send a blank TCP packet to the server. Make sure to mark the option “Persistent TCP” in PacketSender to keep the connection open:

We get the following message from the server

Nice, the server is greeting us and warning us that we should log in. As we can see, each line is preceded by a response code. In this case, 220 means that everything went alright.

Let’s login. We just need to send the following line. Make sure to un-check the “Append \r” option.

USER user_name\n

Where user_name is your FTP user name. The server replies back:

331 User ftp_user@scalableprototyping.com OK. Password required

The 331 code means just what the message says, password required. To what we can reply with our password:

PASS user_password\n

Which if correct, yields some kind of login successful reply:

230-Your bandwidth usage is restricted
230-OK. Current restricted directory is /
230 0 Kbytes used (0%) - authorized: 102400 Kb

The 230 code means that the user was successfully logged in.

Let’s now upload something! We have different transfer modes. For convenience we will test with the ASCII mode, which allows to simply transfer ASCII characters. To enable this mode we send the following command:

TYPE ASCII\n
200 TYPE is now ASCII

In our Arduino version we will be interested in sending binary files, which we can do with the “image mode” that we enable with TYPE I.

Now we are can establish our data connection. There are two modes available, active mode and passive mode. In active mode the server tries to connect to us (which my be complicated if we are behind a firewall). In the more convenient passive mode, the server will open a port for us that we can connect to. To enable passive mode, we send:

PASV\n
227 Entering Passive Mode (77,104,135,197,135,77)

The passive response will tell us where do we need to establish the data connection. The 4 first digits are the IP: 77.104.135.197 and the last two digits are the high and low bytes of a 16 bit integer that defines the opened port. In our case:

data_port_high_byte = 135;
data_port_low_byte = 77;

data_port = data_port_high_byte << 8 | data_port_low_byte;

So in this case, (77,104,135,197,135,77) yields

IP: 77.104.135.197
PORT: 34637

Now we can command that we will upload a file:

STOR file_name.txt\n

The server won’t reply anything after we send that command. It will be waiting for us to establish the data connection. We can now open the TCP data connection. Once we establish the data connection, the server will reply:

150 Accepted data connection

What ever we send to the data connection will be saved into the file!

Once we are done, we just need to close the data connection and the command connection will report the results:

226-0 Kbytes used (0%) - authorized: 102400 Kb
226-File successfully transferred
226 9.227 seconds (measured here), 1.41 bytes per second

And we are done! We just need to mimic this in our Arduino.

For that I prepared a TCP connection class, that allows us to send and receive text just as we do with PacketSender:

 class connection {
    public:
        using byte_buffer_t = std::vector<char>;

        struct response {
            std::string code = "000";
            std::string body;
        };

        connection(const IPAddress&amp; ip, port_t port);
        response receive();
        bool println(const std::string&amp; message);
        bool print(const byte_buffer_t&amp; buffer);
        bool is_connected();
        void close();
        ~connection(); 

    private:
        WiFiClient tcp_client;
};

This class will allow us to establish the connections and receive the responses.

With the help of some helper functions, now we can mimic what we manually did before:

bool ftp_client::upload_file(const std::string&amp; local_path, const std::string&amp; destination_path) const {

    if (!SPIFFS.exists(local_path)) {
        Serial.println(F("File doesn't exists."));
        Serial.println(local_path);
        return 0;
    }

    ftp_client::file_handler file_handler{path, "r"};

    Serial.println(F("Connecting to FTP server..."));

    ftp_client::connection command_connection{server_ip, server_port};
    if (command_connection.is_connected()) {
        Serial.println(F("FTP connection established!"));
    } else {
        Serial.println(F("FTP connection refused."));
        return 0;
    }

    connection::response response;
    
    response = command_connection.receive();
    if (response.code != "220" ) {
        Serial.println(F("Expected 220 response. Error ocurred"));
        return 0;
    }

    command_connection.println(String("USER ") + user + "\n");
    response = command_connection.receive();
    if (response.code != "331" ) {
        Serial.println(F("Expected 331 response. Error ocurred"));
        return 0;
    }

    command_connection.println(String("PASS ") + password + "\n");
    response = command_connection.receive();
    if (response.code != "230" ) {
        Serial.println(F("Expected 230 response. Error ocurred"));
        return 0;
    }

    // Set Image Data Type RFC 959 3.1.1.3
    command_connection.println(String("TYPE I\n"));
    response = command_connection.receive();
    if (response.code.at(0) != '2' ) {
        Serial.println(F("Expected 2xx response. Error ocurred"));
        return 0;
    }

    // Open FTP pasive Data Port
    command_connection.println(String("PASV\n"));
    response = command_connection.receive();
    if (response.code != "227" ) {
        Serial.println(F("Expected 227 response. Error ocurred"));
        return 0;
    }

    std::vector<int> pasv = parse_pasv_response(response.body);
    unsigned int pasv_port, pasv_port_h, pasv_port_l;
    pasv_port_h = pasv.at(4) << 8;
    pasv_port_l = pasv.at(5);
    pasv_port = pasv_port_h | pasv_port_l;

    ftp_client::connection data_connection(server_ip, pasv_port);
    if (data_connection.is_connected()) {
        Serial.println(F("Data connection established"));
    }
    else {
        Serial.println(F("Data connection refused"));
        return 0;
    }

    command_connection.println(std::string("STOR ") + destination_path + "\n");
    response = command_connection.receive();

#define bufSizeFTP 1460
    uint8_t clientBuf[bufSizeFTP];
    size_t clientCount = 0;

    while (file_handler.file.available()) {
        clientBuf[clientCount] = file_handler.file.read();
        clientCount++;
        if (clientCount > (bufSizeFTP - 1)) {
            auto send_buffer = connection::byte_buffer_t(clientBuf, clientBuf + bufSizeFTP);
            data_connection.print(send_buffer);
            clientCount = 0;
            delay(1);
        }
    }
    if (clientCount > 0) {
        auto send_buffer = connection::byte_buffer_t(clientBuf, clientBuf + clientCount);
        data_connection.print(send_buffer);
    }

    data_connection.close();
    response = command_connection.receive();

    command_connection.println(F("QUIT\n"));
    response = command_connection.receive();

    return 1;
}

The complete version of the code can be found here:

https://github.com/scalableprototyping/ftp-client-arduino-esp8266

Leave a Reply

Close Menu