EC - Alembic I/O

To get an idea what the Alembic file format is, please have a look at the definition from “Wikipedia. The Free Encyclopedia”:

“Alembic is an interchange file format for computer graphics used by visual effects and animation professionals. It was announced at SIGGRAPH 2011 and has been widely being adopted in the industry.

Its primary focus is the interchange of geometry (models) between different groups working on the same shots or same assets. This is often different departments in the same company or different studios working on the same projects. Alembic supports the common geometric representations used in the industry, including polygon meshes, subdivision surface, parametric curves, NURBS patches and particles. Alembic also has support for transform hierarchies and cameras. With the latest version comes initial support for materials and lights as well. Alembic is very specifically NOT concerned with storing the complex dependency graph of procedural tools but instead stores the "baked" result.

Alembic was developed as an open source library primarily by Sony Pictures Imageworks and Lucasfilm.”

Currently, Alembic I/O is supported by the following tools and plugins:

Entires with an asterisk ( * ) are approved by Next Limit Technologies and fully support RealFlow data. For software versions, older than the ones displayed in the table above, there is no support for RealFlow's Alembic format. For more information about Alembic, please visit the website:

http://www.alembic.io

If you are interested in the source code – Alembic is an open-source project – go to:

http://code.google.com/p/alembic

The Concept

An Alembic file is comparable to a container where you can store many different data: polygon meshes, points, particles, curves, materials, cameras and so on. It is completely up to the developer which kind of data he/she wants to write to the Alembic file, but the challenge is to make other programs understand what you added to the container. RealFlow, for example, only saves a node's geometry and particles and this data can be read by some 3D programs without problems. Other applications, on the other hand, might not be able to load some of the exported data types. Cameras are another example and they are simply ignored at the moment by RealFlow.

The reason, why it is sometimes not possible to process all of the stored RealFlow data in your 3D program, is that the target applications simply do not “expect” that kind of information – for example particles. Additionally, it is not only necessary to tell which kind of data is stored inside the Alembic container, it also important to put everything into the right place. These differences have to be considered for each platform independently. So, even software packages with integrated Alembic support might not able to read and process all of the provided RealFlow data.

Please bear in mind that development on the Alembic format is still in progress and we will certainly see improvements and changes in the future.

Alembic and Export Central

From a look at your current RealFlow scene it is not apparent which nodes or elements can be exported in Alembic, but the “Export Central” dialogue immediately reveals this information. Only cameras and daemons are not supported by the Alembic format.

Alembic's ABC format can be used like any other available export resource and mixed with other formats as well, for example BIN caches in combination with ABC files. By default, the ABC resource is not active and you have to do this manually for each node you want to export in Alembic format.

SD and MultiBody vs. Alembic

SD is RealFlow's standard exchange format for geometry and motion/animation data and supported by the plugins we offer for the most common 3D applications. SD files can contain multiple, even animated, objects and it is also possible to write animation data to the included nodes. Animations and modifications on a vertex-level are supported as well. Groups or hierarchical animation data, on the other hand, cannot be written to SD files.

When a SD is imported to RealFlow, all included objects are imported as, more or less, a long list of nodes. The main limitation with this import method is that it is not possible to remove selected objects from this list. If you want to delete a certain object, all other nodes will be deleted as well. In order to hide one or more nodes from a simulation it is necessary to change the according “Simulation” option. Additionally, the imported objects could not be cloned or copied.

The MultiBody object introduced many improvements: MultiBodies can contain hundreds or even thousands objects and all these nodes are merged into a single node. Import, for example, is drastically accelerated. The grouped nodes are not controlled individually anymore and the adjustments under “Node Params” affect all objects inside the “MultiBody”.

When you import Alembic files with more than one node you will be asked whether you want to load them as individual objects or as a MultiBody. Loading the objects as individual nodes has some great advantages. It is, for example, possible to remove any element of an imported Alembic scene without affecting the rest of the animation or setup. You can also copy and clone selected parts together with all their properties and animation keys or transformations. Furthermore, objects that have been imported via the Alembic channel can be rotated, scaled and moved even if they contain animation data.

In the case the elements have been imported as a MultiBody you do not have access to the individual objects anymore. This also means that all objects will be removed when you delete the MultiBody. UV coordinates are conserved in both cases. Similar to MultiBodies, the number of imported Alembic is (virtually) unlimited and it is even possible to load the same file several times. To avoid problems, the nodes will be renamed automatically and labelled with a number.

One thing that is not possible with neither SD/MultiBodies nor Alembic I/O, is the creation of nodes during a simulation, for example for dynamics fracturing. However dynamic topology is fully supported.

The Stitcher Tool

In RealFlow, simulation data can be stored in the Alembic format, for example for Hybrido fluids, objects or RealWave splash emitters. All these export resources are stored in separate files and there is one file per frame. The question is why is all this data are stored individually, instead of one ABC file per frame?

The answer is that it is (currently) not possible to open an ABC file, add more data, and close it again. One way to bypass this restriction is to keep the ABC file open until the simulation has been finished. The problem with this method is that is not a very reliable solution: imagine your scene crashes or your hard disk runs out of space. In such cases, all previously simulated data will be simply lost.

Another idea is to let RealFlow merge the different ABC files automatically. Here, RealFlow could write temporary files and stitch them once the simulation is finished. From a technical point of view, this is possible, but not really practicable, because we are dealing with several Gigabytes of data here. The result is a one huge file that is very difficult to handle; additionally it is not very safe to have everything stored in a single file.

For all these reasons we have created a new command line tool that is shipped with your RealFlow distribution and can be found in the application's main folder. It is called “stitcher”. With this handy program you can decide which files you want to merge together. Another advantage is that it is non-destructive. This means that the original files will be kept and the output can be found in a new file.

The “Stitcher” tool is very easy to use. Open a command line application (Terminal, bash or Window's Command Prompt), browse the applications directory with the “cd” command and type in:

Then you should see a help screen:

Usage Example:

./stitcher --in assets/*.abc --out all.abc

Options:

--help produce help message

--compression arg (=1) set compression hint (0-10)

--in arg files to be stitched

--out arg output file

You really only have to launch the tool, specify the path to the folder where the files are stored that you want to merge, and define the name of the output file. Please note that the automatic parameter expansion is made by bash (csh, zsh, etc.) , so it will work in most Linux/Unix environments, but not under Windows. There you have to write everything manually, e.g.

./stitcher --in assets/frame1.abc assets/frame2.abc assets/frame3.abc --out all.abc.