parallel_stereo program is the primary tool of the Ames Stereo
Pipeline. It takes a stereo pair of images that overlap with
corresponding cameras and creates an output point cloud image that can
be processed into a visualizable mesh or a DEM using point2mesh
and point2dem respectively.
This program can distribute the stereo processing over multiple
computing nodes if invoked with the
--nodes-list option. It uses
GNU Parallel to manage the jobs, a tool which is shipped with the
Stereo Pipeline. It expects that all nodes can connect to each other
using ssh without password and that they share the same storage space.
parallel_stereo [options] <images> [<cameras>] <output_file_prefix>
This tool will create a set of output files (Section 19). Internally some of them will be GDAL VRT files, that is, plain text virtual mosaics of files created by individual processes, with the actual files in subdirectories; ASP and GDAL tools are able to use these virtual files in the same way as regular binary TIF files.
See Section 8.16 for how to set up this tool for PBS and SLURM systems.
This program operates only on single channel (grayscale)
images. Multi-channel images need to first be converted to grayscale
or a single channel should be extracted with
16.47.1. Processes and threads¶
It is suggested that after this program is started, one examine how well it uses the CPUs and memory on all nodes, especially at the correlation stage (Section 16.47.2).
Make sure that
--nodes-list is set, otherwise only the head node
will be used.
Note that the SGM and MGM algorithms can be quite memory-intensive. For these, by default, the number of threads is set to 8, and the number of processes is the number of cores divided by the number of threads, on each node. Otherwise, the default is to use as many processes as there are cores.
16.47.2. Entry points¶
parallel_stereo tool is written in Python, and invokes
separate C++ executables for various steps in processing. The
--stop-point options can be used to run only
a portion of these steps.
Only the correlation, blending, subpixel refinement, and triangulation
parallel_stereo are spread over multiple machines, with
the preprocessing and filtering stages using just one node, as they
require global knowledge of the data. In addition, not all stages of
stereo benefit equally from parallelization. Most likely to gain are
stages 1 and 3 (correlation and refinement) which are the most
The invoked C++ executables have their own command-line options
(Section 17). Those options can be passed to
parallel_stereo which will in turn pass them on as needed. By
invoking each executable with no options, it will display the list of
options it accepts.
The steps run by
parallel_stereo are as follows. The output
files created by these steps are described in Section 19.
- Step 0 (Preprocessing)
stereo_pprc. Normalizes the two images and aligns them by locating interest points and matching them in both images. The program is designed to reject outlier interest points. This stage writes out the pre-aligned images and the image masks. It also computes the convergence angle for this stereo pair (for non-mapprojected images and with alignment method
- Step 1 (Stereo correlation)
stereo_corr. Performs correlation using various algorithms which can be specified via
--stereo-algorithm. It writes a disparity map ending in
- Step 2 (Blend)
stereo_blend. Blend the borders of adjacent disparity map tiles obtained during stereo correlation. Needed for all stereo algorithms except the classical
ASP_BMwhen run without local epipolar alignment. The result is the file ending in
- Step 3 (Sub-pixel refinement)
stereo_rfne. Performs sub-pixel correlation that refines the disparity map. Note that all stereo algorithms except
ASP_BMalready do their own refinement at step 1, however further refinement can happen at this step if the
--subpixel-modeoption is set. This produces a file ending in
- Step 4 (Outlier rejection)
stereo_fltr. Performs filtering of the disparity map and (optionally) fills in holes using an inpainting algorithm. It creates
F.tif. Also computes
- Step 5 (Triangulation)
stereo_tri. Generates a 3D triangulated point cloud from the disparity map by intersecting rays traced from the cameras. The output filename ends in
It is important to note that since
parallel_stereo can use a lot
of computational and storage resources, all the intermediate data up
to but not including triangulation can often be reused, if only the
cameras or camera adjustments change (for example, if the cameras got
moved, per Section 16.49.14). Such reuse is discussed in
Section 8.27.11 (in the context of stereo with shallow
If the program failed during correlation, such as because of
insufficient memory, it can be told to resume without recomputing the
existing good partial results with the option
16.47.3. Command-line options¶
- --nodes-list <filename>
The list of computing nodes, one per line. If not provided, run on the local machine. Alternatively, specify the full ssh command for each node, including the port, so one command per line. See examples for PBS and SLURM in Section 8.16.
- -t, --session-type <string>
Select the stereo session type to use for processing. Usually the program can select this automatically by the file extension except for xml cameras. Options and when to use:
pinhole – ground-level cameras, not assuming a datum (Section 8.6)
isis – with planetary images stored in .cub files (Section 4.2)
dg – with Digital Globe exact linescan cameras (Section 5)
rpc – with any RPC cameras (Section 8.19)
spot5 – SPOT 5 exact linescan camera (Section 8.22)
aster – exact ASTER camera model (Section 8.17)
perusat – PeruSat-1 exact linescan camera model (Section 8.20)
opticalbar – Optical Bar cameras (Section 8.24)
csm – Community Sensor Model (Section 8.12)
pleiades – Pleiades satellites (Section 8.21)
The next sessions are for mapprojected images (Section 6.1.7). If
-tis specified as earlier, the sessions will be converted to the entities below automatically.
csmmapcsm / csmmaprpc
- -e, --entry-point <integer (from 0 to 5)>
Stereo Pipeline entry point. Start at this stage. See Section 16.47.2.
- --stop-point <integer (from 1 to 6)>
Stereo Pipeline stop point (stop at the stage right before this).
- --corr-seed-mode <integer (from 0 to 3)>
Correlation seed strategy (Section 17.2).
- --sparse-disp-options <string (default: “”)>
Options to pass directly to sparse_disp (Section 5.6). Use quotes around this string.
- --job-size-w <integer (default: 2048)>
Pixel width of input image tile for a single process. For alignment method
local_epipolaror algorithms apart from
ASP_BM, if not explicitly set, it is overridden by corr-tile-size + 2 * sgm-collar-size. See also Section 17.1.2.
- --job-size-h <integer (default: 2048)>
Pixel height of input image tile for a single process. See also
- --processes <integer>
The number of processes to use per node.
- --threads-multiprocess <integer>
The number of threads to use per process when running multiple processes, for the correlation, subpixel refinement, and triangulation steps (Section 16.47.2).
- --threads-singleprocess <integer>
The number of threads to use when running a single process (for the pre-processing and filtering steps, Section 16.47.2).
Start at the correlation stage and skip recomputing the valid low and full-res disparities for that stage. Do not change
--left-image-crop-win, etc, when running this.
Start at the triangulation stage while reusing the data from this prefix. The new run can use different cameras, bundle adjustment prefix, or bathy planes (if applicable). Do not change crop windows, as that would invalidate the run. See Section 8.27.11 for an example.
- --keep-only <string (default: “all_combined”)>
If set to
all_combined, which is the default, at the end of a successful run combine the results from subdirectories into
.tiffiles with the given output prefix, and delete the subdirectories. If set to
essential, keep only
PC.tifand the files needed to recreate it (those ending with
-F.tif). If set to
unchanged, keep the run directory as it is. For fine-grained control, specify a quoted list of suffixes of files to keep, such as
".exr .match -L.tif -PC.tif".
Display the commands being executed.
Do not launch the jobs, only print the commands that should be run.
- --ssh <filename>
Specify the path to an alternate version of the ssh tool to use.
- --parallel-options <string (default: “”)>
Options to pass directly to GNU Parallel. Example: “
--sshdelay 1 --controlmaster”.
- --cache-size-mb <integer (default = 1024)>
Set the system cache size, in MB.
- --tile-size <integer (default: 256 256)>
Image tile size used for multi-threaded processing.
Tell GDAL to not create bigtiffs.
- --tif-compress <None|LZW|Deflate|Packbits (default: LZW)>
TIFF compression method.
- -v, --version
Display the version of software.
- -h, --help
Display this help message.