Added tag 6.55 for changeset 40cc487dc2a0
Removed tag 6.55
REFACTOR Comment some code
pds is a build system for Ocaml that is meant to make it easy to build a project
that follows a particular layout by generating a makefile for the project. The
input to pds is a config file, pds.conf, and a directory structure, which is
always the current working directory, and the output is the build description.
The directory layout looks like:
.
/src
/proj1
/proj2
/proj3
/tests
/test1/test.ml
/test2/test.ml
The pds.conf for this project might look like:
[src.proj1]
type = "exec"
deps = [ "core", "proj2", "proj3" ]
install = true
install_cmd = "cp -vf proj1.native $(PREFIX)/bin/proj1"
remove_cmd = "rm -v $(PREFIX)/bin/proj1"
[src.proj2]
deps = [ "proj3" ]
install = false
[src.proj3]
deps = [ "str" ]
install = false
[tests.test1]
deps = [ "proj2" ]
[tests.test2]
deps = [ "proj1" ]
Finally, by using the standard Makefile definition below, make test will
compile all the code and run the tests. make install will install proj1. And
make remove will remove it.
all:
pds
$(MAKE) -f pds.mk all
%:
pds
$(MAKE) -f pds.mk $*
pds supports multiple types of builds, at this time release, debug,
profile, and docs. The release build must always exist and is what
running make with no target will execute. The debug and profile builds
will inherit all configuration from the release build and options can be
overridden. pds also supports tests, and the existing test targets are: test,
test-debug, and test-profile, which run tests for the respective builds.
pds builds into the build directory and each build type is a sub-directory in
the build directory, for example build/release, build/docs, or
build/test-release.
Ocaml has multiple build systems, so it is probably confusing and frustrating to learn that another one has been created. In the author's experience, many of the Ocaml build tools are more complicated than the value they add. pds is meant to be simple but play nice with other tools. If you've felt frustrated by the existing tooling, then pds might work well for you. If you are happy with the existing tooling then you should be able to safely ignore pds because it meant to be non-intrusive.
pds supports parallel builds. The amount of parallelism is specified with the
-j option in make. A repository with multiple projects in it will
parallelize the build across those projects. That is, if there is proj1 and
proj2 that do not depend on each other, they will be built in parallel.
Inside of an Ocaml project, if every .ml file in the project has an associated
.mli file, the project will be built in parallel. If there is any .ml file
that does not have a .mli file, the project will be built in serial. The
reason for this is when compiling a .ml file that does not have a .mli file,
both ocamlc and ocamlopt generate a .cmi file even if the .cmi file
already exists. With a parallel build, the byte code and native code builds
will be executed in parallel. If the project is built in parallel, the
generation of the .cmi file can overlap, generating compiler errors. This is
the source of the units Foo and Bar make inconsistent assumptions about Baz
error.
pds does some trickery to build changes only if the contents have changed and not based on timestamp.
To accomplish this, pds stores all input files and a hash in a sqlite database and consults this on each run. If the hash of a file is the same but its timestamp has changed, pds sets the timestamp to the value in the database. This tricks Make into only building things that have actually changed.
If the hashes do not match, pds sets the timestamp to that of the current time and stores that value in the database.
The hash stored for every file is both the hash of the file as well as the hash of the build configuration, this ensure that if a pds configuration has changed that causes a rebuild.
src directory. This corresponds to
an entry in the src table in the pds configuration file.release, debug, profile or docs.ocaml or third-party.exec or
library, this applies only to ocaml projects..Projects are divided up between Ocaml projects, which pds knows how to generate
a Makefile for and a third-party project, which pds does not generate a
Makefile for but are expected to have one that corresponds to the pds
interface, which is release, debug, profile, install, remove, docs,
and clean.
Ocaml projects can be either executables or libraries. The name of the output
is derived from the project name with a suffix added depending on the type. In
the case of a library, a META file compatible with ocamlfind will be
generated.
| Project Type | Suffix |
|---|---|
| Library | cma |
| Library | cmxa |
| Executable | byte |
| Executable | native |
Third party projects do not have a Makefile generated by pds but are expected
to have one which corresponds to the pds interface.
The configuration file input to pds specifies dependencies, if any, for each
project. If a dependency has the same name as a project in the src directory
then it is considered an internal dependency and an internal dependency will be
compiled before its dependent project and the emitted output will be specified
as a dependency in the dependents Makefile.
A configuration file is required for all projects. A configuration file path
can be given explicitly or pds will search for pds.conf in the current working
directory. pds.conf is a toml file. All directories in the src directory
must be in the configuration file. pds will error if that is not the case. The
only required attribute for each project is install, which specifies if it
will be installed or not. In the case of an exec project, the install_cmd
and remove_cmd are required.
Projects and tests can have some configuration values overridden by specific build types, the options and precedence is given in the respective sections below.
An example of building the example directory structure.
proj3 requires no special dependencies or settings and projects are assumed
to be library.
The global release build specifies a project should be compiled with
-noassert.
The global debug build specifies that a project should be compiled with
-g.
The project proj2 overrides the global compiler options for release and
debug and compiles with -safe-string.
The debug build options completely override the release build options.
Libraries have a default ocamlfind install directory, however, commands need
to have their installation command specified.
[global.release] extra_compiler_opts = "-noassert"
[global.debug] extra_compiler_opts = "-g"
[src.proj1] deps = ["async", "proj2", "proj3"] type = "exec" install = true install_cmd = "cp proj1.native $(PREFIX)/bin/proj1" remove_cmd = "rm $(PREFIX)/bin/proj1"
[src.proj2] install = false deps = ["core", "async"] type = "library" extra_compiler_opts = "-safe-string -noassert" debug = { extra_compiler_opts = "-safe-string -g" }
[src.proj3] install = false
| Name | Type | Default | Description |
|---|---|---|---|
| deps | string list | [] | List of dependencies both within the project and external |
| type | string | "library" | exec, library - Whether the ocaml project is an executbale or library |
| install | bool | Whether the built artifacts should be installed | |
| installcmd | string | If the type is exec, specify the command to be used to install it | |
| removecmd | string | If the typs is exec, specify the command to be used to remove it | |
| projecttype | string | "ocaml" | third-party or ocaml - defaults to ocaml |
| extracompileropts | string | "" | Extra options to give to the ocaml compiler |
| extraocamldepopts | string | "" | Extra options to give ot ocamldep |
| extramakefilelines | string list | [] | Extra lines to add to the generated Makefile |
| compiledeps | string list | [] | List of builds that need to be built prior to this build but are not automatically linked against |
| metalinkopts | string | "" | Link options to go into the META file. |
| build | bool | true | Whether or not to build the project or test |
The following values can be overridden:
extra_compiler_optsextra_ocamldep_optsdeps - Note that this has no entry in global.compile_deps - Note that this one has no entry in global.The precedence ordering is given below. Selectors are discussed later in the document.
For release builds:
src.<project>.selector.<selector>.<option>src.<project>.<option>global.release.<option>For all other builds:
src.<project>.selector.<selector>.<build_type>.<option>src.<project>.selector.<selector>.<option>src.<project>.<build_type>.<option>global.<build_type>.<option>src.<project>.<option>The same precedence applies to test builds.
Tests only support specifying the deps and extra_compiler_opts
configuration. The deps option cannot be specified in the global section.
The precedence ordering is given below. Note that release builds are unique
in that they search the third precedence first then the second.
tests.<project>.<build_type>.<option>global.test-<build_type>.<option>tests.<project>.<option>This feature is not experimental and will likely change.
pds can output the configuration in a tab delimited format, like a CSV. The
output looks like the following:
| Column | Description |
|---|---|
| Source Type | "src" or "test", the type of build it is |
| Name | The name of the build, "src.<name>" or "test.<name>" |
| Build Type | "exec" or "library", empty for test |
| Project Type | "ocaml" or "third-party", empty for a test |
| Deps | Comma separated list of dependencies |
pds will produce files when it is executed as well as when the build is
performed. This is a list of files that will be produced and can be safely
ignored in the form of a .gitignore.
build/
pds.mk
Ocamlrules.mk.in
pds can generate HTML documentation using ocamldoc. The docs make target is
automatically generated and third-party libraries are expected to implement a
docs target. The docs are put into the builds/docs/<project> of the root
directory of the repository. By default, only mli files are included in
documentation, however this can be modified by setting the DOCS_FILES variable
using the extra_makefile_lines configuration setting. Different options can
also be set for ocamldoc by modifying the OCAMLDOC_OPTS variable through
extra_makefile_lines.
Some builds need to be parameterized over the environment it's being compiled
in. This is accomplished through selectors. A selector is a program which
outputs a single line that matches the regex [a-zA-Z0-9_-]+, any trailing
white space will be removed. The value of the line will then be used to look up
values in src.<project>.selector.<line>.<variable>. For any selection not in
src.<project>.selector.<line> the value at the top-level of a project
definition is used. The empty string is a special selector which utilizes the
default project values.
The only override supported by selectors is setting the variable. It is not possible to unset a variable in a selector.
Take the case where a project should use different dependencies if it is
executed in environment A vs environment B. The selector program,
is_a_or_b will return the line A or B depending on the system.
[global]
selector = ["is_a_or_b"]
[src.foo]
deps = ["dep1", "dep2"]
[src.foo.selector.A]
deps = ["dep1", "dep2", "dep3"]
[src.foo.selector.B]
deps = ["dep1, "dep2", "dep4"]
[src.bar]
deps = ["depX", "depY" ]
[src.bar.selector.A]
deps = ["dep1", "dep2", "dep3"]
Any variable that applies to a project can be set in a selector.
The order of precedence is that the selector for the build type is first checked, then the selector for a release.
.cmi files.pds.conf.docs target to doc because this conflicts with the directory
name of the output..ml files without a complimentary .mli file now build in serial.
This is because building the byte-code for .ml files with no .mli file can
overlap with each other generating the .cmi file which creates a race
condition where the .cmi is in an invalid state when a later step is using
it. This solves it by serializing the build for those cases..cmi files for .mli files. Not it
installs .cmi files for all .ml files.-custom option from default byte code builds.6.20. Instead, split out the linker options and allow
them to be overridden.extra_makefile_lines in tests.OCAML*_LINK_OPTS just like regular builds.