Introduction
ThorsNisse is RESTful server framework for C++.
It is based on Node.js and its family of associated packages. ThorsNisse uses transparently non-blocking std::streams
to achieve high throughput and low complexity of development on a single threaded server.
What does transparently non-blocking: mean?
- To the user of the std::stream it will appear to be standard blocking stream; thus allowing very normal looking C++ code to be written.
- To make the framework efficient any blocking read/write calls transfers control back to the Nisse Framework so that other streams may be processed.
Node.Cxx
To simplify development for those that simply want to write HTTP RESTful endpoints the server Node.Cxx is provided to handle all the boilerplate.
Node.Cxx (Lovingly named after Node.js for the inspiration) is a simple HTTP server implemented using the ThorsNisse framework that will dynamically load Express modules.
The Node.Cxx server takes two flags as arguments on startup:
> Node.cxx [--debug:<port>] [--load:<SharedLib>:<Host>:<Base>:<port>]
#
# --debug:8081
# Loads the dynamic loader so it listens on port 8081
# To load a libray:
# curl --data '{"action": "Load", "host": "test.com", "base": "", "port": 8080, "lib": "AddBeer.dylib"}' localhost:8081
# To unload a library:
# curl --data '{"action": "Unload", "host": "test.com", "base": "", "port": 8080}' localhost:8081
#
#
# --load:AddBeer.dylib:test.com::8080
# Loads the AddBeer.dylib express module.
# And binds it to the root of http://test.com
#
# Assume AddBeer defines the end pointes /AddBeer
# When a request to `http://test.com/AddBeer the associated code will be called.
#
# --load:AddDrink.dylib:test.com:/drinks:8080
# Loads the AddDrink.dylib express module.
# And binds it to http://test.com/drinks
#
# Assume AddDrink defines the end pointes /AddDrink
# When a request to `http://test.com/drinks/AddDrink the associated code will be called.
#
# Note we did not bind AddDrink to the root of test.com so you need to add the `Base` to
# all routes defined by the library in its code.
Command Line Flag | Description |
---|---|
--debug | Installs the dynamic library loader. This allows shared libraries to be loaded/unloaded at runtime without restarting the server. Useful for building and debugging but should not be used in production. As the command to load/unload library is simply a REST call to the specified port; this can be included into your build environment to un-install the old version and install the newly build version allowing a quick turn around for development. |
--load | Loads a shared library that is linked with Express. This flag can be used multiple times. The express module provides the facility to easily associate code with REST end points. This flag uses the same underlying code as the REST call to load a site. So these sites can also be dynamically unloaded if required. |
Express
// Must include this header
#include "ThorsExpress/all.h"
namespace HTTP = ThorsAnvil::Nisse::Protocol::HTTP;
// Must define this function once.
void addSite(HTTP::Site& site)
{
site.get("/listBeer", [](HTTP::Request& request, HTTP::Response& response)
{
});
// You can add as many resources to as site as you need.
}
Express is a simple library that allows the definition of simple HTTP site modules that can be loaded/unload dynamically. Though each site object is simple each one can be bound to a different relative paths on a site.
Core
Utility
if (open("file", O_WRONLY) == -1)
{
throw std::runtime_error(
buildErrorMessage("MyClass::", __func__,
": open: ", Utility::systemErrorMessage()));
}
// Utility.h
/*
* Builds a string for a system error message.
* uses `errno` to build the name of the error and the associated message into a string.
*/
inline std::string systemErrorMessage();
/*
* Build an error message from a set of parameters.
* Slightly more compact than using 'operator<<` very useful for building exception messages.
*/
template<typename... Args>
std::string buildErrorMessage(Args const&... args);
Provides common utility functions for other packages.
- NameSpace:
- ThorsAnvil::Nisse::Core::Utility
- Headers:
- ThorsNisseCoreUtility
- Utility.h
-
- std::string buildErrorMessage()
- std::string systemErrorMessage()
buildErrorMessage
systemErrorMessage
Socket
> g++ Socket.cpp -o Socket -I${THOR_ROOT}/include -L${THOR_ROOT}/lib -lThorsExpress17
> curl --data "A test message in a bottle :)" http://localhost:8080;echo
// Server Side
#include "ThorsNisseCoreSocket/Socket.h"
#include "ThorsNisseCoreSocket/SocketStream.h"
#include <string>
int main()
{
namespace Sock = ThorsAnvil::Nisse::Core::Socket;
Sock::ServerSocket server(8080);
while(true)
{
Sock::DataSocket connection = server.accept();
Sock::ISocketStream input(connection);
std::string request;
std::getline(input, request);
std::string header;
std::size_t bodySize = 0;
while(std::getline(input, header) && header != "\r")
{
/*
* Note: This code is still handling the intricacy of the HTTP protocol
* so it is brittle. See Protocol/HTTP for help with handling the
* details of the protocol.
*/
if (header.compare(0, 15, "Content-Length:") == 0) {
bodySize = std::stoi(header.substr(15));
}
}
std::string message;
std::string line;
while(bodySize > 0 && std::getline(input, line))
{
message += line;
message += "<br>";
bodySize -= (line.size() + 1);
}
Sock::OSocketStream output(connection);
/*
* Note: This code is still handling the intricacy of the HTTP protocol
* so it is brittle. See Protocol/HTTP for help with handling the
* details of the protocol.
*/
output << "HTTP/1.1 200 OK\r\n"
"Content-Length: " << (11 + message.size()) << "\r\n"
"\r\n"
"It Worked: " << message;
}
}
- NameSpace:
- ThorsAnvil::Nisse::Core::Socket
- Headers:
- ThorsNisseCoreSocket
- Socket.h
-
- BaseSocket
- DataSocket BaseSocket
- ConnectSocket DataSocket
- ServerSocket BaseSocket
- SocketStream.h
-
- SocketStreamBuffer std::streambuf
- ISocketStream std::istream
- OSocketStream std::ostream
BaseSocket
Base of all the socket classes. This class should not be directly created. All socket classes are movable but not copyable.
Move-Constructor
Move-Assignment
swap
close
DataSocket
Data sockets define the read/write interface to a socket. This class should not be directly created
Constructor
getMessageData
std::pair<bool, std::size_t>
. The first member `bool` indicates if more data can potentially be read from the stream. If the socket was cut or the EOF reached then this value will be false. The second member `std::size_t` indicates exactly how many bytes were read from this stream.std::pair<bool, std::size_t>
. The first member `bool` indicates if more data can potentially be read from the stream. If the socket was cut or the EOF reached then this value will be false. The second member `std::size_t` indicates exactly how many bytes were read from this stream. The size of the buffer. Offset into buffer (and amount size is reduced by) as this amount was read on a previous call).putMessageData
std::pair<bool, std::size_t>
. The first member `bool` indicates if more data can potentially be written to the stream. If the socket was cut or closed then this value will be false. The second member `std::size_t` indicates exactly how many bytes were written to this stream.std::pair<bool, std::size_t>
. The first member `bool` indicates if more data can potentially be written to the stream. If the socket was cut or closed then this value will be false. The second member `std::size_t` indicates exactly how many bytes were written to this stream. The size of the buffer. Offset into buffer (and amount size is reduced by) as this amount was written on a previous call).putMessageClose
ConnectSocket
Creates a connection to host
on port
.
Note this class inherits from DataSocket
so once created you can read/write to the socket.
Constructor
ServerSocket
A server socket that listens on a port for a connection
Constructor
accept
DataSocket
is returned so data can be exchange across the socket. Passed to the constructor of the DataSocket
that is returned.DataSocket
that is returned.SocketStreamBuffer
This is a wrapper class for a DataSocket
that allows the socket to be treated like std::streambuf
.
This class overrides just enough virtual functions to make the ISocketStream
and OSocketStream
useful.
This class provides no public API and is designed to be used solely with the following stream objects.
ISocketStream
An implementation of std::istream
that uses SocketStreamBuffer
as the buffer.
The Notofer
is a primitive event callback mechanism.
A blocking read call to these streams calls the Notifier noData
.
This is used by the Server infrastructure to yield control back to the main event loop.
using Notifier = std::function
Constructor
Constructor
Constructor
Move-Constructor
OSocketStream
An implementation of std::istream
that uses SocketStreamBuffer
as the buffer.
The Notofer
is a primitive event callback mechanism.
A blocking read call to these streams calls the Notifier noData
.cw
This is used by the Server infrastructure to yield control back to the main event loop.
using Notifier = std::function
Constructor
Constructor
Move-Constructor
Service
> g++ -std=c++1z Server.cpp -o Server -I${THOR_ROOT}/include -L${THOR_ROOT}/lib -lThorsExpress17 -lboost_context-mt
> curl --data "A test message in a bottle :)" http://localhost:8080;echo
#include "ThorsNisseCoreService/Server.h"
#include "ThorsNisseCoreService/Handler.h"
#include "ThorsNisseCoreSocket/Socket.h"
#include "ThorsNisseCoreSocket/SocketStream.h"
#include <string>
namespace Serv = ThorsAnvil::Nisse::Core::Service;
namespace Sock = ThorsAnvil::Nisse::Core::Socket;
class MyHandler: public Serv::HandlerSuspendableWithStream
{
public:
MyHandler(Serv::Server& parent, Sock::DataSocket&& stream)
: HandlerSuspendableWithStream(parent, std::move(stream), EV_READ)
{}
virtual bool eventActivateWithStream(std::istream& input, std::ostream& output)
{
/*
* Note: The streams `input` and `output` are transparently non-blocking.
* Any blocking read/write will transfer control back to the server
* event loop.
*
* To the code in this function any read/write requests appear to be blocking
* So the writting code is still simple and looks like standard C++ code.
*/
std::string request;
std::getline(input, request);
std::string header;
std::size_t bodySize = 0;
while(std::getline(input, header) && header != "\r")
{
/*
* Note: This code is still handling the intricacy of the HTTP protocol
* so it is brittle. See Protocol/HTTP for help with handling the
* details of the protocol.
*/
if (header.compare(0, 15, "Content-Length:") == 0) {
bodySize = std::stoi(header.substr(15));
}
}
std::string message;
std::string line;
while(bodySize > 0 && std::getline(input, line))
{
message += line;
message += "<br>";
bodySize -= (line.size() + 1);
}
/*
* Note: This code is still handling the intricacy of the HTTP protocol
* so it is brittle. See Protocol/HTTP for help with handling the
* details of the protocol.
*/
output << "HTTP/1.1 200 OK\r\n"
"Content-Length: " << (11 + message.size()) << "\r\n"
"\r\n"
"It Worked: " << message;
return true;
}
};
int main()
{
Serv::Server server;
server.listenOn<MyHandler>(8080);
server.listenOn<MyHandler>(Serv::ServerConnection(8081, 20));
// Start the event loop.
server.start();
}
A simple wrapper around libEvent.
- NameSpace:
- ThorsAnvil::Nisse::Core::Service
- Headers:
- ThorsNisseCoreService
- Handler.h
-
- HandlerBase
- HandlerStream HandlerBase
- HandlerNonSuspendable HandlerStream
- HandlerSuspendable HandlerStream
- HandlerSuspendableWithStream HandlerSuspendable
- Server.h
-
- ServerConnection
- Server
- ServerHandler.h
-
- ServerHandler HandlerNonSuspendable
- TimerHandler HandlerNonSuspendable
- ServerHandler HandlerNonSuspendable
HandlerBase
Constructor
dropHandler
addHandler
moveHandler
eventActivate
suspendable
close
suspend
HandlerStream
This class is templatized based on the type of stream the socket represents. The class basically defines a common class for holding the stream object and how to close it when required.
Constructor
close
HandlerNonSuspendable
Defines `suspendable()` and `suspend()` for a class that is non suspendable. This is used by some of there server built in handlers that must complete. It is unlikely that this will be useful for a user defined handler.
suspend
suspendable
HandlerSuspendable
Defines a handler that is suspendable. Implements `suspendable`, `suspsend()` and `eventActivate()` as these all work together to define a class that can be suspended. The method `eventActivateNonBlocking()` should be overridden by derived classes to provide functionally.
Constructor
Constructor
suspend
suspendable
eventActivate
eventActivateNonBlocking
HandlerSuspendableWithStream
An implementation of `eventActivateNonBlocking()` that creates input and output stream objects. These stream objects will call `suspend()` if they are about to perform a blocking operation on the underlying socket. Thus we have transparently non-blocking streams.
eventActivateNonBlocking
eventActivateWithStream
ServerConnection
Used to simplify the definition of a port.
Constructor
Server
An object that acts as the main server event loop. One of these objects can handle all the ports your application requires
Default-Constructor
Move-Constructor
Move-Assignment
start
flagShutDown()
will cause the event loop to exit after the current iteration.
flagShutDown
listenOn
eventActivate()
method is called allowing the data to be processed.
For more details see HandlerBase.
addTimer
ServerHandler
An implementation of HandlerNonSuspendable that is used to accept connections and create other handlers.
TimerHandler
An implementation of HandlerNonSuspendable that is used to implement the timer functionality.
SQL
namespace Express = ThorsAnvil::Nisse::Protocol::HTTP;
namespace SQL = ThorsAnvil::SQL;
static SQL::Connection connection("mysqlNB://test.com", "test", "testPassword", "test");
static SQL::Statement listBeers(connection, "SELECT Name, Age FROM Beers");
class MyListBeerHandler: public HandlerSuspendable<DataSocket>
{
virtual bool eventActivateNonBlocking()
{
ISocketStream input(stream);
std::string request;
std::getline(input, request);
std::string header;
while(std::getline(input, header) && header != "\r")
{}
std::stringsttream body;
body << "<html>"
<< "<head><title>Beer List</title></head>"
<< "<body>"
<< "<h1>Beer List</h1>"
<< "<ol>";
// Calling execute() suspends the current handler.
// Control is returned (unsuspends the handler) when the execute() function returns.
//
// The execute function returns when all the rows have been returned from the SQL
// DB. The lambda is executes once for each row that is ruturned this allowing us
// to build the response as a stream.
listBeers.execute([&body](std::string const& name, int age)
{
// This call to the MySQL server is also non blocking.
// If the call is going to block control is returned to ThorNisse framework
// Allowing another connection to be processed while the MySQL server finishes
// Its work.
body << "<li>" << name << " : " << age << "</li>";
});
body << "</ol></body></html>";
OSocketStream output(stream);
output << "HTTP/1.1 200 OK\r\n"
"Content-Length: " << body.size() << "\r\n"
"\r\n"
<< body;
}
};
Please refer to the ThorSQL library for details.
A PIMPL proxy for the ThorSQL library.
There are no user usable classes in this package. All the classes are used internally by ThorSQL library. There usage is unlocked by using the prefix "mysqlNB" in the connection string.
- NameSpace:
- ThorsAnvil::Nisse::Core::SQL
- Headers:
- ThorsNisseCoreSQL/
Protocol
Simple
A very simple protocol. These classes are used to test the functionality of Core::Service without the extra baggage of the HTTP protocol.
- NameSpace:
- ThorsAnvil::Nisse::Protocol::Simple
- Headers:
- ThorsNisseProtocolSimple
HTTP
class Site
{
public:
Site();
Site(Site&&) noexcept;
Site& operator=(Site&&) noexcept;
void swap(Site&) noexcept;
void get(std::string&& path, Action&& action) {add(Method::Get, std::move(path), std::move(action));}
void put(std::string&& path, Action&& action) {add(Method::Put, std::move(path), std::move(action));}
void del(std::string&& path, Action&& action) {add(Method::Delete, std::move(path), std::move(action));}
void post(std::string&& path, Action&& action) {add(Method::Post, std::move(path), std::move(action));}
void all(std::string&& path, Action&& action) {add(4, std::move(path), std::move(action));}
std::pair<bool, Action> find(Method method, std::string const& path) const;
};
class Binder
{
public:
Binder();
void setCustome404Action(Action&& action);
void addSite(std::string const& host, std::string const& base, Site&& site);
std::pair<bool, int> remSite(std::string const& host, std::string const& base);
Action find(Method method, std::string const& host, std::string const& path) const;
};
class DeveloperHandler: public Core::Service::HandlerNonSuspendable<Core::Socket::DataSocket>
{
public:
DeveloperHandler(Core::Service::Server& parent, Core::Socket::DataSocket&& socket, DynamicSiteLoader& loader);
virtual short eventActivate(Core::Service::LibSocketId sockId, short eventType) override;
};
class DynamicSiteLoader
{
public:
DynamicSiteLoader(Core::Service::Server& server);
std::tuple<bool, int> load(std::string const& site, int port, std::string const& host, std::string const& base);
std::tuple<bool, int, int> unload(int port, std::string const& host, std::string const& base);
void setMaxWaitingConnections(int max);
};
class ReadRequestHandler: public Core::Service::HandlerSuspendable<Core::Socket::DataSocket>
{
public:
ReadRequestHandler(Core::Service::Server& parent, Core::Socket::DataSocket&& socket, Binder const& binder);
virtual bool eventActivateNonBlocking() override;
void setFlusher(Response* f){flusher = f;}
void flushing() {if (flusher){flusher->flushing();}}
};
struct HttpParserData
{
HttpParserData();
void addCurrentHeader();
Headers headers;
std::string currentHead;
std::string currentValue;
std::string uri;
char const* bodyBegin;
char const* bodyEnd;
Method method;
bool messageComplete;
bool gotValue;
};
class HttpScanner
{
public:
HttpParserData data;
HttpScanner();
void scan(char const* buffer, std::size_t size);
};
class Route
{
public:
Route(std::string&& fullRoute);
Route(Route const&) = delete;
Route& operator=(Route const&) = delete;
Route(Route&&) = default;
bool operator<(Route const& rhs) const;
bool operator<(std::string const& rhs) const;
bool operator<=(std::string const& rhs) const;
bool operator==(std::string const& rhs) const;
};
struct RouteTester
{
using is_transparent = std::true_type;
bool operator()(std::string const& lhs, Route const& rhs) const {return !(rhs <= lhs);}
bool operator()(Route const& lhs, std::string const& rhs) const {return lhs < rhs;}
bool operator()(Route const& lhs, Route const& rhs) const {return lhs < rhs;}
};
class Headers
{
public:
typedef ConstIterator const_iterator;
class Inserter
{
ValueStore& valueStore;
public:
Inserter(ValueStore& valueStore);
void operator=(std::string&& value);
void operator=(std::string const& value);
};
ConstIterator begin() const {return std::cbegin(data);}
ConstIterator end() const {return std::cend(data);}
Inserter operator[](std::string const& key) {return data[key];}
std::size_t getVersions(std::string const& key) const;
std::string const& get(std::string const& key, std::size_t version = 0) const;
};
class URI
{
public:
std::string original;
std::string normalized;
std::string schema;
std::string host;
std::string path;
std::string query;
std::string fragment;
short port;
Headers queryParam;
URI(std::string const& hostAndPort, std::string&& pathAndQuery);
};
class Request
{
public:
const Method method;
const URI uri;
const Headers& headers;
std::istream& body;
Request(Method method,
URI&& uri,
Headers& headers,
std::istream& body);
};
class Response
{
public:
short resultCode;
std::string resultMessage;
Headers headers;
std::ostream& body;
Response(std::ostream& body);
Response(ReadRequestHandler& flusher,
DataSocket& socket,
std::ostream& body,
short resultCode = 200,
std::string const& resultMessage = "OK");
~Response();
void flushing(bool allDone = false);
};
An implementation of the HTTP protocol
- NameSpace:
- ThorsAnvil::Nisse::Protocol::HTTP
- Headers:
- ThorsNisseProtocolHTTP
- Binder.h
-
- Site
- Binder
- DeveloperHandler.h
-
- DeveloperHandler Core::Service::HandlerNonSuspendable
- DeveloperHandler Core::Service::HandlerNonSuspendable
- DynamicSiteLoader.h
-
- DynamicSiteLoader
- HTTPProtocol.h
-
- ReadRequestHandler Core::Service::HandlerSuspendable
- ReadRequestHandler Core::Service::HandlerSuspendable
- HttpScanner.h
-
- HttpParserData
- HttpScanner
- Route.h
-
- Route
- RouteTester
- Types.h
-
- Headers
- URI
- Request
- Response
Site
Binder
DeveloperHandler
DynamicSiteLoader
ReadRequestHandler
HttpParserData
HttpScanner
Route
RouteTester
Headers
URI
Request
Response
Examples
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum
AddBeer
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum
- NameSpace:
- ThorsAnvil::Nisse::Example::AddBeer
- Headers:
- ThorsNisseExampleAddBeer/