Shared Memory Locking Strategies in mplib1
There is a set of symbols of which one and only one should be used,
which defines which inter-process locking mechanism to use.
They are currently:
- HAVE_PTHREADS
- defines the use of pthread_mutex_* functions.
- HAVE_MSEM
- defines the use of msem_* functions.
- HP UX
- Sequent (experimental)
- HAVE_MUTEX
- defines the use of mutex_* functions.
- HAVE_GCC_I386
- defines use of GCC inline assembler for i386 family of processors.
- HAVE_SVR4_I386
- defines use of assembler for i386 on SVR4.
- HAVE_SVR4_I386B
- defines use of assembler for i386 on SVR4.
Locking on a New Platform
The choice of which locking mechanism to use is the most
difficult aspect of porting the library to a new platform.
There are two fundamental types of lock that are provided by the library.
These are the bit lock and the pid lock. In both cases, there is no
requirement for ordering on waiting processes.
The Bit Lock
The most basic lock is the bit-lock.
The fundamental operation that needs to be
provided to implement this is a multi-processor safe atomic test-and-set
operation and a suitable clear operation.
Most of the implementations so far have used assembler code to
implement the base operations. In one case (SINIX) there is a suitable
bit-locking library provided with the system (-lmutex) which provides
suitable functions.
The pid Lock
The requirements of the pid lock API are similar to that of an atomic
lock with an added requirement that should the owning
COE of a lock die it should be possible for another
COE to detect this and to take ownership of the lock.
The pid lock then provides the ablity to perform nested locks on the
pid structure by the owning COE and only releasing the lock when the
lock count drops to zero.
In most cases this is achieved by implementing a super-structure based
upon bit locks which tracks use of which pid is owner. In one case
(threads under Solaris) the locking is performed using a
pthread_mutex lock which both the wait mechanism and the
unlock on owner death mechanism.
Lock Functionality
pthreads
The pthreads locking code uses a basic structure like
struct bpo_thread_lock
{
pthread_mutex_t mp;
pthread_mutexattr_t mattr;
};
And uses the following pthread functions.
- pthread_mutexattr_init
- pthread_mutexattr_setpshared
- pthread_mutex_init
- pthread_mutex_lock
- pthread_mutex_unlock
- pthread_mutex_trylock
mutex
The mutex locking code uses the required
library locking type abilock_t. It also
uses the following library functions
- init_lock
- acquire_lock
- release_lock
msem
The use of the msem_* functions is currently under research. The
particular attributes of the msem funcitons that cause problems are that
although the functions provide a queued access to the lock, they do not
provide any protection against ownership death. Thus these functions
cannot be used as the sole lock technique for pid locks. This implies that
they must be used for bit locking only, and that to allow correct unlocking
the code must not call any blocking functions. This could cause
deadlock situations, which is why status of this area of code is
under research.
Having said that, by treating the msemaphore structure as a simple bitlock
with no undo capability, and polling (with waits) a reasonable facsimile
of the required functionality.
assembler
The assembler variations of the bit-locking code rely on three (or two)
simple instructions.
- A Test-And-Set operation or an Exchange operation.
- A Clear operation.
- A Read operation.
If the processor provides an exchange operation then the first two
instructions can be encoded as
XCHG( address, 1 );
XCHG( address, 0 );
Context Of Execution - COE
A Context Of Execution is a cover-all term used to describe
an individual entity in the space that is both processes and threads.
The system as implemented currently does