usr/src/lib/README.mapfiles

#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Copyright (c) 2006, 2010, Oracle and/or its affiliates. All rights reserved.
#

Mapfiles and versioning in illumos
==================================

1.0 Objective of this README

This README describes the engineering practices of creating and updating
visible library interfaces.  It describes various kinds of actions that
typically occur as libraries are evolved, and shows how interface
specifications are affected or updated in accordance.  It tells you what
you must do as a shared library developer if you:

	1. Make interface additions to an existing library
		- add a Public interface
		- add a Private interface
	2. Update an interface in an existing library
		- remove an existing interface
		- promote a Private interface to Public
		- scope a Private interface to local
		- move an interface from one library to another
		- copy interfaces which are part of the standard to a new or
		  existing library
	3. Introduce a new library
		- source directory hierarchy
		- creation of the "mapfile-vers" file
		- Makefiles
	4. Make an entire library obsolete before end-of-life
		- introduce SUNWobsolete to the "mapfile-vers" file

-------------------------------------------------------------------------------

2.0 What's a mapfile?

Mapfiles are used to tell the link-editor ("ld") all sorts of things about
how to generate an executable file or a shared object from a collection of
relocatable objects, such as generated by a compiler.  For all the gory
details, see the Solaris Linker and Libraries Guide.

There are two versions of the mapfile language accepted by the link-editor.
Version 1 derives from AT&T System V Release 4 Unix. Version 2 is a newer
syntax specific to Solaris and derivatives.  All mapfiles in illumos are
required to use version 2 syntax. Note that every mapfile using version 2
syntax must start with the line:

        $mapfile_version 2

Here, we are only concerned with specifying externally-visible interfaces
for shared libraries (shared objects) and with specifying their versions
for ABI (Application Binary Interface) purposes.  For these purposes, we
only need to deal with a subset of the mapfile language.

There should be a "mapfile-vers" file associated with every shared library
and it should reside in the common source directory for that library, most
often in a "common" directory.  This is the usual layout of a library's
top-level directory (usr/src/lib/libwombat):
	Makefile       amd64/         i386/          sparcv9/
	Makefile.com   common/        sparc/

The "common" directory contains the source files and other common files
for the library:
	bat.c              libwombat_impl.h   mapfile-vers       wom.c
	libwombat.h        llib-lwombat       util.c             wombat.c

The mapfile's name is, by convention, "mapfile-vers" because it is used
for only two purposes: to specify externally-visible interface names while
suppressing visibility of all other names, and to specify their respective
unique version names.

-------------------------------------------------------------------------------

3.0 Contents of mapfile-vers

The structure of mapfile-vers is best explained by an example
(the license notification and copyright notice is omitted here
for brevity):

$mapfile_version 2

SYMBOL_VERSION ILLUMOS_0.2 {	# Second interface change in illumos
    global:
	wb_notify;
} ILLUMOS_0.1;

SYMBOL_VERSION ILLUMOS_0.1 {	# First interface change in illumos
    global:
	wb_poll;
} SUNW_1.2;    

SYMBOL_VERSION SUNW_1.2 {	# update to libwombat, Solaris 10
    global:
	wb_readv;
	wb_stat;
	wb_writev;
} SUNW_1.1;

SYMBOL_VERSION SUNW_1.1 {	# first release of libwombat, Solaris 9
    global:
	wb_read;
	wb_write;
};

SYMBOL_VERSION SUNWprivate {	# private libwombat symbols
    global:
	wb_add;
	wb_delete;
	wb_search;
    local:
	*;
};

Each of these sections is a version declaration describing an ABI version of
the library containing the set of symbols exposed by the library to
external callers.

ABI versions must be constant, that is version ILLUMOS_0.2 in a given
library must always describe the same interface such that applications may
safely state their dependencies in terms of these versions and have a
constant and predictable ABI be exposed.  This in effect means that once a
version is publicly visible, it may never be removed or have symbols added
to or removed from it.

ABI versions with the same major number should be upward compatible, that is
ILLUMOS_0.3 of a given library must contain all the interfaces in
ILLUMOS_0.2, and they must be compatible.

The private version, however, is special, and describes any private yet
exposed symbols within a library, and may change at any time (and without
notice).  It exists purely to allow certain symbols to be of global scope
but not Public.  Similarly, any symbols scoped local are private and may
change safely, even if the local statement happens to be within a public
version.

Interface changes made in illumos should be done with ILLUMOS_0.* versions,
introducing one version per interface change.  In illumos, unlike Solaris,
symbol versions are not release-specific because of the requirement that
each be constant.  No change should be made to add or remove symbols from
any pre-existing Public version.

The SUNW_*.* names were the Public version names of the library in Solaris.
There should be at most one version name for each release of Solaris, with
the minor number incremented by one over the previous version.  No changes
should ever be made to SUNW_1.* versions.

So, for example, to add a new interface to libwombat in illumos one would add:

SYMBOL_VERSION ILLUMOS_0.3 {	# Third update to libwombat in illumos
    global:
	wb_lseek;
} ILLUMOS_0.2;

Each version must inherit all symbols from its preceding version, specified at
the ending "}" for each version. The initial public version does not inherit
any symbols.  The private version named either "SUNWprivate" for libraries
with private symbols pre-existing illumos, or "ILLUMOSprivate" otherwise
stands alone, inheriting nothing and being inherited by nothing.

The two lines in SUNWprivate:
    local:
	*;
ensure that no symbols other than those listed in the mapfile are visible to
clients of the library.  If there is no private version, these two lines should
appear in the first public version.

For maintainability, the list of names in each version block should
be sorted in dictionary order (sort -d).  Please comply.

The version 2 mapfile language supports a simple mechanism for conditional
input, in which lines in the mapfile apply only to a specific platform or
ELFCLASS (32/64-bit). This mechanism works very much like the #if/#endif
feature of the C preprocessor. For instance, the following mapfile declares
a version SUNW_1.1 that always exports a symbol foo, and also exports
the symbol bar on 32-bit sparc platforms:

$mapfile_version 2
SYMBOL_VERSION SUNW_1.1 {
        foo;
$if _sparc && _ELF32
	bar;
$endif
};

Conditional input can be used if there are ISA-specific library interfaces
not common to all instances of the library. It is the preferred method for
expressing platform specific items, as long as the differences are simple
(which is almost always the case).  For example, see libproc, or, if you
are masochistic, libc or libnsl.  In general, use of this feature should be
minimal.

In addition to conditional input, there is a second heavier weight mechanism
for expressing ISA-specific differences. In addition to the common mapfile:
	common/mapfile-vers
some libraries may have ISA-specific supplemental mapfiles, one in each
of the ISA directories:
	amd64/mapfile-vers
	i386/mapfile-vers
	sparc/mapfile-vers
	sparcv9/mapfile-vers
The ISA-specific mapfiles look like the common mapfile, except that only
the ISA-specific names appear.  The version names are the same as those
in the common mapfile, but only non-empty version instances are present
and no inheritance specification is present. The link-editor reads the
information from the common and ISA-specific mapfiles and merges them
in memory into a single description used to create the resulting object.

ISA-specific mapfiles were used with the version 1 mapfile language, which
lacked conditional input. Their use is rare now, as conditional input is
generally preferred. However, it is important to use conditional input
carefully, or the resulting mapfile can be extremly difficult to read.

-------------------------------------------------------------------------------

4.0 Making interface additions to an existing library

4.1 Adding a Public interface

Public interfaces should be added to a new ILLUMOS_ symbol version, with the
minor number incremented by one from the current highest version name. If
this is the first Public interface in the shared object, a new ILLUMOS_0.1
version name must be introduced.

The major revision number is incremented whenever an incompatible change is
made to an interface.  This could be the case if an API changes so
dramatically as to invalidate dependencies.  This should almost never occur
in practice.  It also requires changing the suffix of the shared object
from, say, .so.1 to .so.2 and introducing code to continue to ship the .so.1
version of the library.

The minor revision number is incremented whenever one or more new interfaces
is added to a library.  Once a version comes to exist in illumos, it is from
that point onward considered to be immutable.

4.2 Adding a Private interface

Private interfaces are the non-ABI interfaces of the library.  Unlike
introducing a Public interface, a new entry is simply added to the
private version.  No minor number increment is necessary.

If this interface happens to be the first Private interface introduced into
the library, the private version must be created (with no major.minor
version numbers).  It inherits nothing, nothing inherits from it and it
should be named ILLUMOSprivate.

If the library already has Private interfaces in a SUNWprivate version, you
should use that.  They may have numbered version names like SUNWprivate_m.n
(due to errors of the past).  If so, just use the highest numbered private
version name to version the new interface.  There is no need to introduce a
new private version name.  Be careful not to use a lower numbered private
version name; doing so can cause runtime errors (as opposed to load time
errors) when running an application with older versions of the library.

There are also libraries in illumos that contain only private interfaces. In
such libraries, the private versions maybe legitimately be versioned and
they may be incremented to ensure that the programs that depend on them are
built and delivered as a integrated unit. A notable example of this is
libld.so (usr/src/cmd/sgs/libld), which contains the implementation of the
link-editor, the public interface to which is provided by the ld
command. When making a modification to the interface of such a library, you
should follow the convention already in place.

4.3 Historical handling of Solaris update releases.

To aid the understanding of our existing mapfiles, it is useful to note how
interface versions were handled as they interacted with update releases of
Solaris.  Solaris update releases required careful coordination with the full
release currently under development to keep symbol versions constant between
releases.

Multiple update releases were generally shipped during the development of the
next full release of Solaris.  It was impossible to know in advance the full
set of new interfaces in the next full release until it was complete.  Some,
though not all, new interfaces were included in the intervening update
releases between full releases.

Consequently, the new version number for an update cannot be a minor
increment, but must be a micro increment to ensure that was a distinct
version between the two releases.  For example, if Release N had version
number SUNW_1.3 and Release N+1 had SUNW_1.4, then interfaces added to
an update of Release N must have micro numbers such as SUNW_1.3.1,
SUNW_1.3.2, etc.  (note that the micro number is not directly tied to the
update number: SUNW_1.3.1 may have appeared in Update 2).  The micro versions form
an inheritance chain that is inserted between two successive minor versions.
For example, the mapfile-vers file for minor release "N+1" to reflect its
inclusion of micro releases will look like the following:

$mapfile_version 2

SYMBOL_VERSION SUNW_1.4 {	# release N+1
    global:
	...
} SUNW_1.3.2;

SYMBOL_VERSION SUNW_1.3.2 {	# micro release 2 (e.g., release NU3)
    global:
	...
} SUNW_1.3.1;

SYMBOL_VERSION SUNW_1.3.1 {	# micro release 1 (e.g., release NU2)
    global:
	...
} SUNW_1.3;

SYMBOL_VERSION SUNW_1.3 {	# release N
    global:
	...
} SUNW_1.2;

SYMBOL_VERSION SUNW_1.2 {	# release N-1
    global:
	...
} SUNW_1.1;

SYMBOL_VERSION SUNW_1.1 {	# first release
    global:
	...
};

SYMBOL_VERSION SUNW_private {	# same in all releases
    global:
	...
    local:
	*;
};

The corresponding update/patch mapfile-vers file will be identical
except for the exclusion of SUNW_1.4.

Those interfaces which are only present in Release N+1 are always put
into the next minor version set, SUNW_1.4.

Thus when adding a new Public interface to an update release, both the mapfiles
of the update release and next full release should have been modified to be
consistent.

There have been several cases of accidental deviation from this scheme, and
existing mapfiles sometimes reflect this unfortunate state of affairs.

-------------------------------------------------------------------------------

5.0 How to update an interface in an existing library

5.1 Removing an existing interface

5.1.1 Moving a Public interface

No Public interfaces should ever be removed from any mapfile, as this will
break all existing consumers of that interface.

To move an interface from one library to (say) libc, the code has to be
deleted from the library and added to libc, then the mapfile for the
library has to have the interface's entry changed from:
	getfoobar;
to:
	getfoobar       { TYPE = FUNCTION; FILTER = libc.so.1 };
This is an exception to the immutability of public symbol versions.  See,
for example, libnsl's common/mapfile-vers file.

Follow the rules for adding a new interface for the necessary changes
to libc's mapfile to accommodate the moved interface, including creating a
new version in libc for the symbol.

When merging an entire library into libc, the mapfile is changed to specify
the type of each public symbol similarly to the above:
	getfoobar;
to:
	getfoobar       { TYPE = FUNCTION };

But rather than specifying the library on which we filter for each symbol,
the link-editor is invoked with '-F libc.so.1' to specify that our entire
symbol table is a filter on libc.  For examples, see libaio and librt.

5.1.2 Removing a Private interface

Deletion of Private interfaces is allowed, but caution should be taken;
it should first be established that the interface is not being used.
To remove a Private interface, simply delete the corresponding entry
for that symbol from the mapfile's private version section.

Do not forget to delete these Public or Private interfaces from the library's
header files as well as from the code that implements the interfaces.

5.2 Promoting a Private interface to Public

This is similar to what's done when adding a Public interface.  Promoting an
existing Private interface to a Public one only requires a change to the
existing interface definition.  Private interfaces have the symbol version
name "ILLUMOSprivate" or "SUNWprivate" associated with them.  To make the
interface a Public one, the interface must be added as if it were a new
public symbol, following those same rules and removed from the private
version.

As an example, if we were modifying libwombat.so.1 and its existing latest
version were ILLUMOS_0.3, any new ABI would be put into a version called
ILLUMOS_0.4.  Therefore, whether you wish to promote an existing Private
interface to Public, or to introduce a new Public interface, this (next
successive minor numbered version level) would be the version that it would
be associated with.

5.3 Scoping a Private interface local

Any interfaces not present in the mapfile-vers file will be scoped local
due to the presence of the
    local:
	*;
lines discussed earlier. This ensures that such interfaces will not be visible
outside the library.  To move an interface from Private to local scope, simply
remove the Private interface from the mapfile-vers file and the header file
to prevent it from being exported.  This may require moving the Private
interface into a library-private header file.  Scope reduction of Public
interfaces is forbidden.

For the interface to be used in more than one file within the library, it
should be in a header file that can be included by each file in the library
that uses the interface.  For example:

	#include "libprivate.h"

5.4 How to copy interfaces which are part of a standard to a new or existing
    library

SYSVABI and SISCD are reserved version names for interfaces listed in the
System V Interface Definition and the Sparc Compliance Definition.  Avoid using
these version names when copying the implementation of standard interfaces to
another library.  Instead, use ILLUMOS_0.1 for a new library, and ILLUMOS_m.n for
an existing library (where m.n is the next version; i.e., if the
last version was ILLUMOS_0.8, then you should version the interfaces with
ILLUMOS_0.9).

-------------------------------------------------------------------------------

6.0 Introducing a new library

6.1 Directories

The normal discipline for introducing a new library in illumos is to create a
new subdirectory of usr/src/lib.  The interface definition discipline is to
create a common/mapfile-vers file for the new library.  If we were introducing
a new foo library, libfoo, we'd create usr/src/lib/libfoo containing:
	Makefile       amd64/         i386/          sparcv9/
	Makefile.com   common/        sparc/
The common subdirectory would contain the normal source files plus the
mapfile-vers file.  See usr/src/lib/README.Makefiles for directions on
how to organize the Makefiles.

6.2 The mapfile

The new common/mapfile-vers file would contain:

$mapfile_version 2

SYMBOL_VERSION ILLUMOS_0.1 {	# first release of libfoo
    global:
	...
};

SYMBOL_VERSION ILLUMOSprivate {
    global:
	...
    local:
	*;
};

If there are no Public interfaces, the ILLUMOS_0.1 section would be omitted.
If there are no Private interfaces, the ILLUMOSprivate section would be
omitted and the two lines:
    local:
	*;
would be moved into ILLUMOS_0.1.

To decide which interfaces are Public (part of the ABI) and which are
Private (unstable interfaces not intended to be used by third parties), the
heuristic which works to a first approximation is that if it has a man page
then it's Public.

For maintainability, the list of names in each version block should
be sorted in dictionary order (sort -d).  Please comply.

-------------------------------------------------------------------------------

7.0 Make an entire library obsolete

7.1 Introduce SUNWobsolete version

Use this version name not for specific interfaces but for marking an entire
library as obsolete.  The existing public/private version names are left
unchanged, but a new SUNWobsolete version is created with no symbols in it.
This becomes a tag by which the obsolescence of the library can be recognized.
There is no numbering of this version name.

$mapfile_version 2

SYMBOL_VERSION SUNWobsolete {
    global:
	SUNWobsolete;	# This is the only way to do it.
} SUNW_1.2;

SYMBOL_VERSION ILLUMOS_0.2 {
...

You should continue to use the name SUNWobsolete even in illumos.

-------------------------------------------------------------------------------

8.0 Documentation

For further information, please refer to the following documents:

	"Solaris Linker and Libraries Guide"