2.7. Concurrency

Joedb offers mechanisms to access a single database from multiple processes on the same machine, or from remote machines over the network.

2.7.1. Principle

Concurrency works by letting each process have a local copy of the central database. Each process can keep data synchronized with 3 operations:

• pull: update local data with new journal entries from the central database.
• lock_pull: get exclusive write access to the central database, and update local data.
• push_unlock: update the central database with the local modifications, and release the lock.

So it works a bit like git, with the significant difference that merging branches is not possible. History must be linear, and a global mutex is used to prevent branches from diverging.

When using a remote network connection, a local copy of the database can be kept in permanent storage between connections. SHA-256 is used at connection time, to check that the contents of the local and remote copies match. Offline modifications to the local copy can be made, and then pushed to the server when connecting later.

2.7.2. Example

The compiler produces code that ensures that locks and unlocks are correctly paired, and modifications to the local database can only occur during a lock.

#include "tutorial.h"
#include "joedb/concurrency/File_Connection.h"
#include "joedb/journal/Memory_File.h"

/////////////////////////////////////////////////////////////////////////////
int main()
/////////////////////////////////////////////////////////////////////////////
{
 //
 // This sets up a configuration with a server and 2 clients.
 //
 joedb::Memory_File server_file;
 joedb::Memory_File client1_file;
 joedb::Memory_File client2_file;

 joedb::File_Connection connection(server_file);

 tutorial::Client client1(connection, client1_file);
 tutorial::Client client2(connection, client2_file);

 //
 // The databases are empty. client1 will add a few cities.
 //
 // All write operations are performed via the transaction function.
 // The transaction function takes a lambda as parameter.
 // The lock_pull operation is performed before the lambda, and the push_unlock
 // operation is performed after the lambda, if no exception was thrown.
 // If any exception was thrown during the lambda, then the changes
 // are not pushed to the server, and the server is unlocked.
 // Writes that occured in a transaction before an exception are not sent to
 // the server, but they are written locally.
 //
 client1.transaction([](tutorial::Generic_File_Database &db)
 {
  db.new_city("Paris");
  db.new_city("New York");
  db.new_city("Tokyo");
 });

 //
 // client1.get_database() gives a read-only access to the local copy
 //
 std::cout << "Number of cities for client1: ";
 std::cout << client1.get_database().get_city_table().get_size() << '\n';

 //
 // Client1 added cities, and they were pushed to the central database.
 // They have not yet reached client2.
 //
 std::cout << "Number of cities for client2 before pulling: ";
 std::cout << client2.get_database().get_city_table().get_size() << '\n';

 //
 // Let's pull to update the database of client2
 //
 client2.pull();
 std::cout << "Number of cities for client2 after pulling: ";
 std::cout << client2.get_database().get_city_table().get_size() << '\n';

 return 0;
}

It produces this output:

Number of cities for client1: 3
Number of cities for client2 before pulling: 0
Number of cities for client2 after pulling: 3

2.7.3. Connections

The constructor of the tutorial::Client class takes two parameters: a connection, and a file for local storage. The connection is an object of the Connection class, that provides the 3 synchronization operations (pull, lock_pull, push_unlock). This section presents the different kinds of available connections.

2.7.3.1. Plain Connection

The Connection superclass does not connect to anything.

One use of this class is to allows generic code that takes a client as parameter to work the same way with either a remote connection or a local file.

Another use of this class is to handle concurrency when opening a local file with Open_Mode::shared_write.

joedbc produces a convenient Local_Client class that creates the connection and the client in a single line of code. Here is an example:

#include "tutorial.h"

#include "joedb/io/main_exception_catcher.h"

#include <chrono>
#include <thread>

/////////////////////////////////////////////////////////////////////////////
static int local_concurrency(int argc, char **argv)
/////////////////////////////////////////////////////////////////////////////
{
 tutorial::Local_Client client("local_concurrency.joedb");

 while (true)
 {
  client.transaction([](tutorial::Generic_File_Database &db)
  {
   db.new_person();
  });

  std::cout << "I have just added one person. Total number of persons: ";
  std::cout << client.get_database().get_person_table().get_size() << '\n';
  std::this_thread::sleep_for(std::chrono::seconds(1));
 }

 return 0;
}

/////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
/////////////////////////////////////////////////////////////////////////////
{
 return joedb::main_exception_catcher(local_concurrency, argc, argv);
}

Multiple instances of this program can safely write to the same database concurrently.

2.7.3.2. File_Connection

File_Connection creates a connection to a file. It is useful for unit testing, and for the connection tutorial. File_Connection can also be used to make a safe and clean copy of a database that is being used or contains a dirty uncheckpointed transaction.

2.7.3.3. Server_Connection

Server_Connection allows connecting to a running joedb_server using the joedb network protocol.

The constructor of Server_Connection takes a Channel parameter. Two channel classes are provided:

• Network_Channel opens a network socket to the server directly.
• ssh::Forward_Channel connects to the server with ssh encryption and authentication.

The code below shows how to connect to a server via ssh:

#ifndef joedb_SSH_Server_Connection_declared
#define joedb_SSH_Server_Connection_declared

#include "joedb/ssh/Forward_Channel.h"
#include "joedb/concurrency/Server_Connection.h"

namespace joedb
{
 /////////////////////////////////////////////////////////////////////////////
 class SSH_Server_Connection:
 /////////////////////////////////////////////////////////////////////////////
  private ssh::Session,
  private ssh::Forward_Channel,
  public Server_Connection
 {
  public:
   SSH_Server_Connection
   (
    const char *user,
    const char *host,
    const uint16_t joedb_port,
    const unsigned ssh_port,
    const int ssh_log_level,
    std::ostream *log
   ):
    ssh::Session(user, host, ssh_port, ssh_log_level),
    ssh::Forward_Channel
    (
     *static_cast<ssh::Session *>(this),
     "localhost",
     joedb_port
    ),
    Server_Connection(*static_cast<ssh::Forward_Channel *>(this), log)
   {
   }
 };
}

#endif

2.7.4. Combining Local and Remote Concurrency

A client is made of two parts: the local part (stored in a file), and the connection part. A client can handle concurrency for both parts simultaneously. That is to say, it is possible for two different clients running on the same machine to share a connection to the same remote server, and also share the same local file.