Introduction

ThorsNisse is RESTful server framework for C++.
It is based on Node.js and its family of associated packages. ThorsNisse uses transparently non-blocking std::streams to achieve high throughput and low complexity of development on a single threaded server.

What does transparently non-blocking: mean?

• To the user of the std::stream it will appear to be standard blocking stream; thus allowing very normal looking C++ code to be written.
• To make the framework efficient any blocking read/write calls transfers control back to the Nisse Framework so that other streams may be processed.

Node.Cxx

To simplify development for those that simply want to write HTTP RESTful endpoints the server Node.Cxx is provided to handle all the boilerplate.

Node.Cxx (Lovingly named after Node.js for the inspiration) is a simple HTTP server implemented using the ThorsNisse framework that will dynamically load Express modules.

The Node.Cxx server takes two flags as arguments on startup:

 > Node.cxx [--debug:<port>] [--load:<SharedLib>:<Host>:<Base>:<port>]
 #
 # --debug:8081
 #      Loads the dynamic loader so it listens on port 8081
 #          To load a libray:
 #              curl --data '{"action": "Load",   "host": "test.com", "base": "", "port": 8080, "lib": "AddBeer.dylib"}' localhost:8081
 #          To unload a library:
 #              curl --data '{"action": "Unload", "host": "test.com", "base": "", "port": 8080}'  localhost:8081
 #
 #
 # --load:AddBeer.dylib:test.com::8080
 #      Loads the AddBeer.dylib express module.
 #          And binds it to the root of http://test.com
 #
 #          Assume AddBeer defines the end pointes /AddBeer
 #          When a request to `http://test.com/AddBeer the associated code will be called.
 #
 # --load:AddDrink.dylib:test.com:/drinks:8080
 #      Loads the AddDrink.dylib express module.
 #          And binds it to http://test.com/drinks
 #
 #          Assume AddDrink defines the end pointes /AddDrink
 #          When a request to `http://test.com/drinks/AddDrink the associated code will be called.
 #
 #          Note we did not bind AddDrink to the root of test.com so you need to add the `Base` to
 #          all routes defined by the library in its code.

Command Line Flag	Description
--debug	Installs the dynamic library loader. This allows shared libraries to be loaded/unloaded at runtime without restarting the server. Useful for building and debugging but should not be used in production. As the command to load/unload library is simply a REST call to the specified port; this can be included into your build environment to un-install the old version and install the newly build version allowing a quick turn around for development.
--load	Loads a shared library that is linked with Express. This flag can be used multiple times. The express module provides the facility to easily associate code with REST end points. This flag uses the same underlying code as the REST call to load a site. So these sites can also be dynamically unloaded if required.

Express

// Must include this header
#include "ThorsExpress/all.h"

namespace HTTP = ThorsAnvil::Nisse::Protocol::HTTP;

// Must define this function once.
void addSite(HTTP::Site& site)
{
    site.get("/listBeer", [](HTTP::Request& request, HTTP::Response& response)
    {
    });
    // You can add as many resources to as site as you need.
}

Express is a simple library that allows the definition of simple HTTP site modules that can be loaded/unload dynamically. Though each site object is simple each one can be bound to a different relative paths on a site.

Core

Utility

if (open("file", O_WRONLY) == -1)
{
    throw std::runtime_error(
        buildErrorMessage("MyClass::", __func__,
                          ": open: ", Utility::systemErrorMessage()));
}

// Utility.h
/*
 * Builds a string for a system error message.
 * uses `errno` to build the name of the error and the associated message into a string.
 */
inline std::string systemErrorMessage();

/*
 * Build an error message from a set of parameters.
 * Slightly more compact than using 'operator<<` very useful for building exception messages.
 */
template<typename... Args>
std::string buildErrorMessage(Args const&... args);

Provides common utility functions for other packages.

NameSpace:

ThorsAnvil::Nisse::Core::Utility

Headers:

ThorsNisseCoreUtility

Utility.h

• std::string buildErrorMessage()
• std::string systemErrorMessage()

buildErrorMessage

systemErrorMessage

Socket

 > g++ Socket.cpp -o Socket -I${THOR_ROOT}/include -L${THOR_ROOT}/lib -lThorsExpress17
 > curl --data "A test message in a bottle :)" http://localhost:8080;echo

// Server Side
#include "ThorsNisseCoreSocket/Socket.h"
#include "ThorsNisseCoreSocket/SocketStream.h"
#include <string>
int main()
{
    namespace Sock = ThorsAnvil::Nisse::Core::Socket;

    Sock::ServerSocket    server(8080);
    while(true)
    {
        Sock::DataSocket    connection = server.accept();
        Sock::ISocketStream input(connection);

        std::string     request;
        std::getline(input, request);

        std::string     header;
        std::size_t     bodySize = 0;
        while(std::getline(input, header) && header != "\r")
        {
            /*
             * Note: This code is still handling the intricacy of the HTTP protocol
             *       so it is brittle. See Protocol/HTTP for help with handling the
             *       details of the protocol.
             */
            if (header.compare(0, 15, "Content-Length:") == 0) {
                bodySize = std::stoi(header.substr(15));
            }
        }

        std::string     message;
        std::string     line;
        while(bodySize > 0 && std::getline(input, line))
        {
            message += line;
            message += "<br>";
            bodySize -= (line.size() + 1);
        }

        Sock::OSocketStream output(connection);
        /*
         * Note: This code is still handling the intricacy of the HTTP protocol
         *       so it is brittle. See Protocol/HTTP for help with handling the
         *       details of the protocol.
         */
        output << "HTTP/1.1 200 OK\r\n"
                  "Content-Length: " << (11 + message.size()) << "\r\n"
                  "\r\n"
                  "It Worked: " << message;
    }
}

NameSpace:

ThorsAnvil::Nisse::Core::Socket

Headers:

ThorsNisseCoreSocket

Socket.h

• BaseSocket
• DataSocket BaseSocket
• ConnectSocket DataSocket
• ServerSocket BaseSocket

SocketStream.h

• SocketStreamBuffer std::streambuf
• ISocketStream std::istream
• OSocketStream std::ostream

BaseSocket

Base of all the socket classes. This class should not be directly created. All socket classes are movable but not copyable.

Move-Constructor

Move-Assignment

swap

close

DataSocket

Data sockets define the read/write interface to a socket. This class should not be directly created

Constructor

getMessageData

putMessageData

putMessageClose

ConnectSocket

Creates a connection to host on port. Note this class inherits from DataSocket so once created you can read/write to the socket.

Constructor

ServerSocket

A server socket that listens on a port for a connection

Constructor

accept

SocketStreamBuffer

This is a wrapper class for a DataSocket that allows the socket to be treated like std::streambuf. This class overrides just enough virtual functions to make the ISocketStream and OSocketStream useful. This class provides no public API and is designed to be used solely with the following stream objects.

ISocketStream

An implementation of std::istream that uses SocketStreamBuffer as the buffer. The Notofer is a primitive event callback mechanism. A blocking read call to these streams calls the Notifier noData. This is used by the Server infrastructure to yield control back to the main event loop. using Notifier = std::function;

Constructor

Constructor

Constructor

Move-Constructor

OSocketStream

An implementation of std::istream that uses SocketStreamBuffer as the buffer. The Notofer is a primitive event callback mechanism. A blocking read call to these streams calls the Notifier noData.cw This is used by the Server infrastructure to yield control back to the main event loop. using Notifier = std::function;

Constructor

Constructor

Move-Constructor

Service

 > g++ -std=c++1z Server.cpp -o Server -I${THOR_ROOT}/include -L${THOR_ROOT}/lib -lThorsExpress17 -lboost_context-mt
 > curl --data "A test message in a bottle :)" http://localhost:8080;echo

#include "ThorsNisseCoreService/Server.h"
#include "ThorsNisseCoreService/Handler.h"
#include "ThorsNisseCoreSocket/Socket.h"
#include "ThorsNisseCoreSocket/SocketStream.h"
#include <string>

namespace Serv = ThorsAnvil::Nisse::Core::Service;
namespace Sock = ThorsAnvil::Nisse::Core::Socket;

class MyHandler: public Serv::HandlerSuspendableWithStream
{
    public:
        MyHandler(Serv::Server& parent, Sock::DataSocket&& stream)
            : HandlerSuspendableWithStream(parent, std::move(stream), EV_READ)
        {}
        virtual bool eventActivateWithStream(std::istream& input, std::ostream& output)
        {
            /*
             * Note: The streams `input` and `output` are transparently non-blocking.
             *       Any blocking read/write will transfer control back to the server
             *       event loop.
             *
             * To the code in this function any read/write requests appear to be blocking
             * So the writting code is still simple and looks like standard C++ code.
             */
            std::string     request;
            std::getline(input, request);

            std::string     header;
            std::size_t     bodySize = 0;
            while(std::getline(input, header) && header != "\r")
            {
                /*
                 * Note: This code is still handling the intricacy of the HTTP protocol
                 *       so it is brittle. See Protocol/HTTP for help with handling the
                 *       details of the protocol.
                 */
                if (header.compare(0, 15, "Content-Length:") == 0) {
                    bodySize = std::stoi(header.substr(15));
                }
            }

            std::string     message;
            std::string     line;
            while(bodySize > 0 && std::getline(input, line))
            {
                message += line;
                message += "<br>";
                bodySize -= (line.size() + 1);
            }

            /*
             * Note: This code is still handling the intricacy of the HTTP protocol
             *       so it is brittle. See Protocol/HTTP for help with handling the
             *       details of the protocol.
             */
            output << "HTTP/1.1 200 OK\r\n"
                      "Content-Length: " << (11 + message.size()) << "\r\n"
                      "\r\n"
                      "It Worked: " << message;
            return true;
        }
};
int main()
{
    Serv::Server      server;
    server.listenOn<MyHandler>(8080);
    server.listenOn<MyHandler>(Serv::ServerConnection(8081, 20));

    // Start the event loop.
    server.start();
}

A simple wrapper around libEvent.

NameSpace:

ThorsAnvil::Nisse::Core::Service

Headers:

ThorsNisseCoreService

Handler.h

• HandlerBase
• HandlerStream HandlerBase
• HandlerNonSuspendable HandlerStream
• HandlerSuspendable HandlerStream
• HandlerSuspendableWithStream HandlerSuspendable

Server.h

• ServerConnection
• Server

ServerHandler.h

• ServerHandler HandlerNonSuspendable
• TimerHandler HandlerNonSuspendable

HandlerBase

Constructor

dropHandler

addHandler

moveHandler

eventActivate

suspendable

close

suspend

HandlerStream

This class is templatized based on the type of stream the socket represents. The class basically defines a common class for holding the stream object and how to close it when required.

Constructor

close

HandlerNonSuspendable

Defines `suspendable()` and `suspend()` for a class that is non suspendable. This is used by some of there server built in handlers that must complete. It is unlikely that this will be useful for a user defined handler.

suspend

suspendable

HandlerSuspendable

Defines a handler that is suspendable. Implements `suspendable`, `suspsend()` and `eventActivate()` as these all work together to define a class that can be suspended. The method `eventActivateNonBlocking()` should be overridden by derived classes to provide functionally.

Constructor

Constructor

suspend

suspendable

eventActivate

eventActivateNonBlocking

HandlerSuspendableWithStream

An implementation of `eventActivateNonBlocking()` that creates input and output stream objects. These stream objects will call `suspend()` if they are about to perform a blocking operation on the underlying socket. Thus we have transparently non-blocking streams.

eventActivateNonBlocking

eventActivateWithStream

ServerConnection

Used to simplify the definition of a port.

Constructor

Server

An object that acts as the main server event loop. One of these objects can handle all the ports your application requires

Default-Constructor

Move-Constructor

Move-Assignment

start

flagShutDown

listenOn

addTimer

ServerHandler

An implementation of HandlerNonSuspendable that is used to accept connections and create other handlers.

TimerHandler

An implementation of HandlerNonSuspendable that is used to implement the timer functionality.

SQL

namespace Express = ThorsAnvil::Nisse::Protocol::HTTP;
namespace SQL     = ThorsAnvil::SQL;

static SQL::Connection connection("mysqlNB://test.com", "test", "testPassword", "test");
static SQL::Statement  listBeers(connection, "SELECT Name, Age FROM Beers");

class MyListBeerHandler: public HandlerSuspendable<DataSocket>
{
    virtual bool eventActivateNonBlocking()
    {
        ISocketStream   input(stream);

        std::string     request;
        std::getline(input, request);

        std::string     header;
        while(std::getline(input, header) && header != "\r")
        {}

        std::stringsttream body;
        body << "<html>"
             << "<head><title>Beer List</title></head>"
             << "<body>"
             << "<h1>Beer List</h1>"
             << "<ol>";

        // Calling execute() suspends the current handler.
        // Control is returned (unsuspends the handler) when the execute() function returns.
        //
        // The execute function returns when all the rows have been returned from the SQL
        // DB. The lambda is executes once for each row that is ruturned this allowing us
        // to build the response as a stream.
        listBeers.execute([&body](std::string const& name, int age)
        {
            // This call to the MySQL server is also non blocking.
            // If the call is going to block control is returned to ThorNisse framework
            // Allowing another connection to be processed while the MySQL server finishes
            // Its work.
            body << "<li>" << name << " : " << age << "</li>";
        });
        body << "</ol></body></html>";

        OSocketStream   output(stream);
        output << "HTTP/1.1 200 OK\r\n"
                  "Content-Length: " << body.size() << "\r\n"
                  "\r\n"
                  << body;
    }
};

Please refer to the ThorSQL library for details.

A PIMPL proxy for the ThorSQL library.
There are no user usable classes in this package. All the classes are used internally by ThorSQL library. There usage is unlocked by using the prefix "mysqlNB" in the connection string.

NameSpace:

ThorsAnvil::Nisse::Core::SQL

Headers:

ThorsNisseCoreSQL/

Protocol

Simple

A very simple protocol. These classes are used to test the functionality of Core::Service without the extra baggage of the HTTP protocol.

NameSpace:

ThorsAnvil::Nisse::Protocol::Simple

Headers:

ThorsNisseProtocolSimple

HTTP

class Site
{
    public:
        Site();
        Site(Site&&) noexcept;
        Site& operator=(Site&&) noexcept;
        void swap(Site&) noexcept;

        void get(std::string&& path, Action&& action)    {add(Method::Get,    std::move(path), std::move(action));}
        void put(std::string&& path, Action&& action)    {add(Method::Put,    std::move(path), std::move(action));}
        void del(std::string&& path, Action&& action)    {add(Method::Delete, std::move(path), std::move(action));}
        void post(std::string&& path, Action&& action)   {add(Method::Post,   std::move(path), std::move(action));}
        void all(std::string&& path, Action&& action)    {add(4,              std::move(path), std::move(action));}
        std::pair<bool, Action> find(Method method, std::string const& path) const;
};

class Binder
{
    public:
        Binder();
        void setCustome404Action(Action&& action);
        void addSite(std::string const& host, std::string const& base, Site&& site);
        std::pair<bool, int> remSite(std::string const& host, std::string const& base);

        Action find(Method method, std::string const& host, std::string const& path) const;
};

class DeveloperHandler: public Core::Service::HandlerNonSuspendable<Core::Socket::DataSocket>
{
    public:
        DeveloperHandler(Core::Service::Server& parent, Core::Socket::DataSocket&& socket, DynamicSiteLoader& loader);
        virtual short eventActivate(Core::Service::LibSocketId sockId, short eventType) override;
};

class DynamicSiteLoader
{
    public:
        DynamicSiteLoader(Core::Service::Server& server);
        std::tuple<bool, int>      load(std::string const& site, int port, std::string const& host, std::string const& base);
        std::tuple<bool, int, int> unload(int port, std::string const& host, std::string const& base);
        void                       setMaxWaitingConnections(int max);
};

class ReadRequestHandler: public Core::Service::HandlerSuspendable<Core::Socket::DataSocket>
{
    public:
        ReadRequestHandler(Core::Service::Server& parent, Core::Socket::DataSocket&& socket, Binder const& binder);
        virtual bool eventActivateNonBlocking() override;
        void setFlusher(Response* f){flusher = f;}
        void flushing()             {if (flusher){flusher->flushing();}}
};

struct HttpParserData
{
    HttpParserData();
    void addCurrentHeader();

        Headers                 headers;
        std::string             currentHead;
        std::string             currentValue;
        std::string             uri;
        char const*             bodyBegin;
        char const*             bodyEnd;
        Method                  method;
        bool                    messageComplete;
        bool                    gotValue;
};

class HttpScanner
{
    public:
        HttpParserData          data;

        HttpScanner();
        void scan(char const* buffer, std::size_t size);
};

class Route
{
    public:
        Route(std::string&& fullRoute);
        Route(Route const&)            = delete;
        Route& operator=(Route const&) = delete;
        Route(Route&&) = default;

        bool operator<(Route const& rhs) const;
        bool operator<(std::string const& rhs) const;
        bool operator<=(std::string const& rhs) const;
        bool operator==(std::string const& rhs) const;
};

struct RouteTester
{
    using is_transparent = std::true_type;

    bool operator()(std::string const& lhs, Route const& rhs) const {return !(rhs <= lhs);}
    bool operator()(Route const& lhs, std::string const& rhs) const {return lhs < rhs;}
    bool operator()(Route const& lhs, Route const& rhs)       const {return lhs < rhs;}
};

class Headers
{
    public:
        typedef ConstIterator const_iterator;
        class Inserter
        {
            ValueStore&  valueStore;
            public:
                Inserter(ValueStore& valueStore);
                void operator=(std::string&& value);
                void operator=(std::string const& value);
        };

        ConstIterator begin() const {return std::cbegin(data);}
        ConstIterator end()   const {return std::cend(data);}
        Inserter operator[](std::string const& key)    {return data[key];}

        std::size_t        getVersions(std::string const& key) const;
        std::string const& get(std::string const& key, std::size_t version = 0) const;
};

class URI
{
    public:
        std::string original;
        std::string normalized;

        std::string schema;
        std::string host;
        std::string path;
        std::string query;
        std::string fragment;
        short       port;

        Headers     queryParam;

        URI(std::string const& hostAndPort, std::string&& pathAndQuery);
};

class Request
{
    public:
        const Method            method;
        const URI               uri;
        const Headers&          headers;
        std::istream&           body;

        Request(Method method,
                URI&& uri,
                Headers& headers,
                std::istream& body);
};

class Response
{
    public:
        short                   resultCode;
        std::string             resultMessage;
        Headers                 headers;
        std::ostream&           body;

        Response(std::ostream& body);
        Response(ReadRequestHandler& flusher,
                 DataSocket& socket,
                 std::ostream& body,
                 short resultCode = 200,
                 std::string const& resultMessage = "OK");
        ~Response();
        void flushing(bool allDone = false);
};

An implementation of the HTTP protocol

NameSpace:

ThorsAnvil::Nisse::Protocol::HTTP

Headers:

ThorsNisseProtocolHTTP

Binder.h

• Site
• Binder

DeveloperHandler.h

• DeveloperHandler Core::Service::HandlerNonSuspendable

DynamicSiteLoader.h

• DynamicSiteLoader

HTTPProtocol.h

• ReadRequestHandler Core::Service::HandlerSuspendable

HttpScanner.h

• HttpParserData
• HttpScanner

Route.h

• Route
• RouteTester

Types.h

• Headers
• URI
• Request
• Response

Site

Binder

DeveloperHandler

DynamicSiteLoader

ReadRequestHandler

HttpParserData

HttpScanner

Route

RouteTester

Headers

URI

Request

Response

Examples

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

AddBeer

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

NameSpace:

ThorsAnvil::Nisse::Example::AddBeer

Headers:

ThorsNisseExampleAddBeer/