mitsuba-visualize: Automates High-Quality Mitsuba Renders

Visualizes meshes, pointclouds and video flythroughs in publication quality

• Collection of scripts automating path-traced rendering with mitsuba (0.6.0), directly using their python API.
• Can also render flythroughs from predefined camera poses using bezier curves and catmull-rom splines

This implementation was used for the visualizations in Dai, Angela, Christian Diller, and Matthias Nießner. “SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans.” (CVPR’20)

You can find more samples, including flythroughs, in our SG-NN video.

Installation

1. Install mitsuba 0.6.0, following the tutorial here: @sreenithyc21/10-steps-to-install-mitsuba-renderer-on-ubuntu-38a9318fbcdf">10 Steps to Install Mitsuba Renderer on Ubuntu
2. Clone this repository and install python dependencies:

   git clone https://github.com/chrdiller/mitsuba-visualize
   cd mitsuba-visualize
   poetry install

3. After building mitsuba, adjust path in set_python_path.py to point to the folder you cloned mitsuba to (has to contain subdirectory dist/python)

   MITSUBA_BASE = Path('/path/to/mitsuba/clone')

Usage

Enter environment: poetry shell

Single images

• Meshes: python mesh.py ...
  ```bash
  usage: mesh.py [-h] -o OUTPUT [—scenes_list SCENES_LIST] [—scene SCENE]

           [--cameras CAMERAS] [--width WIDTH] [--height HEIGHT]
           [--samples SAMPLES] [--workers WORKERS]
           [input_paths [input_paths ...]]

Render directory

positional arguments:
input_paths Path(s) to directory containing all files to render

optional arguments:
-h, —help show this help message and exit
-o OUTPUT, —output OUTPUT
Path to write renderings to
—scenes_list SCENES_LIST
Path to file containing filenames to render in base
path
—scene SCENE One scene. Overrides scenes_list
—cameras CAMERAS XML file containing meshlab cameras
—width WIDTH Width of the resulting image
—height HEIGHT Height of the resulting image
—samples SAMPLES Number of integrator samples per pixel
—workers WORKERS How many concurrent workers to use

- Point Clouds: ``python pointcloud.py ...``
```bash
usage: Render pointcloud with mitsuba by placing little spheres at the points' positions
       [-h] -o OUTPUT [--radius RADIUS] [--width WIDTH] [--height HEIGHT]
       [--samples SAMPLES] [--workers WORKERS]
       input [input ...]
positional arguments:
  input                 Path(s) to the ply file(s) containing a pointcloud(s)
                        to render
optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Path to write renderings to
  --radius RADIUS       Radius of a single point
  --width WIDTH         Width of the resulting image
  --height HEIGHT       Height of the resulting image
  --samples SAMPLES     Number of integrator samples per pixel
  --workers WORKERS     How many concurrent workers to use

Flythroughs

As in the SG-NN video

• (Optional) Export cameras from meshlab
  • Open the mesh in meshlab and press Cmd/Ctrl + C to copy the current camera view
  • Paste into a text editor (will be a few lines of XML)
  • Save as xml file
  • Repeat for every camera keypoint
• (Alternatively) Do not specify cameras; this will render a slightly tilted top-down turntable view
• Usage: python flythough.py ...
  ```bash
  usage: Render a flythrough video of a scene [-h] -o OUTPUT

                                        [--remote [REMOTE [REMOTE ...]]]
                                        [--novideo] [--norender] [--keep]
                                        [--scenes_list SCENES_LIST]
                                        [--frames FRAMES]
                                        [--cameras CAMERAS]
                                        [--shutter_time SHUTTER_TIME]
                                        [--width WIDTH] [--height HEIGHT]
                                        [--fov FOV] [--samples SAMPLES]
                                        [--interpolation {catmullrom,bezier}]
                                        [--workers WORKERS]
                                        input [input ...]

positional arguments:
input Path to the ply file to render

optional arguments:
-h, —help show this help message and exit
-o OUTPUT, —output OUTPUT
Path to write output video to
—remote [REMOTE [REMOTE …]]
Urls of the remote render servers
—novideo Only render frames, do not produce video
—norender Only render video from existing frames, no rendering
—keep Whether to keep the frame images
—scenes_list SCENES_LIST
Path to file containing filenames to render in base
path
—frames FRAMES Number of frames to render (The video file will have
30fps)
—cameras CAMERAS XML file containing meshlab cameras (or path to
directory only containing such files). If set, this is
used for spline interpolation. Otherwise, a rotating
flyover is generated
—shutter_time SHUTTER_TIME
Shutter time of the moving sensor
—width WIDTH Width of the resulting image
—height HEIGHT Height of the resulting image
—fov FOV Field of view of the sensor in degrees (meshlab
default is 60)
—samples SAMPLES Number of integrator samples per pixel
—interpolation {catmullrom,bezier}
Which method to use for interpolation between control
points
—workers WORKERS How many local concurrent workers to use
```