Welcome to Ivy’s documentation!

Introduction

Ivy is a program for image processing. The main idea behind it is the creation of a processing pipeline in the form of a directed acyclic graph, which describes the processing steps performed from the initial input image to the final output image. This provides great flexibility in the processing pipelines and allows for a non-linear processing flow.

_images/preview.png

Each node in the graph represents an image processing operation with specified inputs and outputs. In addition, nodes can have multiple parameters to tune their behavior. These nodes are the building blocks of the pipeline, ranging from simple math operations to advanced multi-scale processes. When arranged in a graph, these operations are executed to obtain the final output image.

Within the editor nodes can be easily placed, connections between nodes created or removed, and parameters of nodes adjusted. With any change, the pipeline is updated, executed and the output is previewed. This allows for easy tuning across the whole pipeline and, depending on the complexity, real-time feedback on the adjustments made.

User Interface

Toolbox

_images/toolbox.png

The toolbox offers the following functionality:

Move Image:When this tool is selected, the image can be dragged with the left mouse button when zoomed in.
Color Picker:This tool allows color sampling from the output image with a left mouse click.
Auto-connect:With this option is enabled, when placing and removing nodes the connections are automatically adjusted to accommodate for the changes.
Add Node:This drop-down menu is used to place new nodes. Right-clicking on the image preview shows the same menu.

Image Panel

The image panel Contains a preview of the currently processed output. It is the main area in which nodes can be positioned, however nodes can be moved outside of it across the whole window. Right-clicking on the image or the surrounding dark-gray area brings up the Add Node menu.

Image Information Panel

_images/image_info.png

The Image Information panel displays the properties of the currently selected input image.

Histogram Panel

_images/histogram.png

The Histogram panel shows a histogram of the output image values. The Visibility drop-down provides a selection of channels to display in the histogram:

Red:The sRGB red channel.
Green:The sRGB green channel.
Blue:The sRGB blue channel.
Lightness:The Lab lightness channel.
_images/color_picker.png

The Color Picker selection is shown above the histogram, together with the corresponding sRGB values. These values are also indicated with vertical lines in the histogram.

Status Bar

The status bar at the bottom of the windows shows information about the program and the current pipeline.

UI:Redraw frequency of the user interface.
Processing:Time elapsed while processing the full pipeline, followed by an indication of the processing device.
Memory used:Amount of memory allocated in image buffers, together with the number of large buffers.

Processing Pipelines

The main component of the user interface is the creation of processing pipelines, this is done in the Image panel. There are several elements that together define the process. Nodes are the basic processing blocks, containing parameters. Links connect the nodes and indicate the flow of data. Together the connected nodes form a graph which specifies the full processing pipeline to be executed.

Nodes

_images/node.png

A pipeline consists of nodes, at the very least one output node which determines the output of the pipeline. In this application, the input node has a special status too. Together with the output node they are always present in a process, and are used to define the input image to be processed and the output containing the processed image. In addition to these, there are a multitude of nodes, which can be added to the process.

Adding nodes:New nodes are accessed through the Add Node menu, subdivided in multiple categories. For further information about the available nodes and their functionality, see the Modules section.
Moving nodes:Once a node is added, it can be moved around by dragging the node title with the left mouse button.
Removing nodes:A node is removed by right-clicking the node title.

The blue line above the node title indicates that this node is active: it is included in the processing pipeline. If a node is disconnected or otherwise does not contribute to the output, this line will be gray.

Nodes contain parameters which adjust the node functionality, and ports which represent the node’s inputs and outputs.

Parameters

A node’s parameters are always shown within the node and can be manipulated at any time. These parameters can have different forms:

Value slider:These represent a floating point value in a pre-defined range. Dragging the slider left or right changes its value. Holding Ctrl while dragging enables stepped mode, in which the adjusted value is rounded to fixed increments. Holding Shift while dragging enables precise mode, in which the slider moves in smaller increments for accurate fine-tuning. These effects can be combined. Lastly, right-clicking on a value resets it to its default value.
Check-box:These toggle a parameter on or off, indicated by a filled or empty box. Right-clicking on a value resets it to its default value.

Note

Some check-boxes enable specific tools, such as a color picker. Such tools can be exclusively enabled, when enabling one tool, all others will be disabled. The check-boxes belonging to the same exclusive group will be momentarily highlighted when one of them is toggled.

Whenever a parameter is changed, the processing pipeline will be updated and output processed to represent the changed parameters.

Often, parameters will allow to be driven by data from another node, this is indicated by Ports next to the parameter. When a parameter’s input port is connected, the manually entered value will be overridden by the value supplied via the port.

Note

A useful feature is driving a parameter with an array input (e.g. mask or image). The parameter will spatially vary according to the input data, allowing for local effects.

Ports

Ports reside on the left and right edge of nodes. These are the inputs to (left) and outputs of (right) the node’s processing function. Each input port can be connected to an output port of a different node using links, or left disconnected. When connected, the output of the output node will be transferred to the input of the input node, facilitating data flow between the processing functions. A disconnected port will use a default value.

Ports can accept and generate data of varying size. In general, the data will be image data containing a 2D array of values across three channels. However, a port can also contain a single value, a single three-channel color, or a single channel 2D mask. These can be used interchangeably, the process will use the smallest possible data size while preserving its full content. When different data sizes are used, the process transparently converts the smaller data to fit the larger data.

Processing Pipeline Execution

The combination of nodes and links describes a graph. This graph represents the processing pipeline as it will be executed. The graph is manipulated by adding or removing nodes, or changing the links between them.

Note

A strictly acyclic graph is not able to represent iteration or recursion. However, the current implementation does not enforce this and cyclic graphs can be created and run in Continuous mode. While this is not advised, a converging loop could be useful in specific circumstances.

Modules

Ivy’s processing pipeline consists of modules chained together, each performing a specific processing step. the available modules are divided into several categories dependent on the type of operation they perform.

Adjust modules

These modules offer the basic tools for global adjustments of the image being processed. All operations are pixel-wise.

Brightness

_images/node.png
Brightness:Modifies image brightness
Preserve Hue:Preserves input image hue in LCH space
Group:Adjust
Color Space:LRGB

This module changes the brightness of the image by adjusting the slope at \(0.0\) with the Brightness parameter, and preserves the white point by compressing the curve toward \(1.0\) (1). The Preserve Hue parameter applies the hue of the input image to that of the output image in LCH space (2).

Warning

The module is intended to preserve the brightened image’s black point and white point. However, color channels might be pushed out of gamut when Preserve Hue is enabled.

Brightness Preserve Hue: On Preserve Hue: Off
-1.0 _images/-1_V.png _images/-1_X.png
+0.0 _images/0.png _images/0.png
+1.0 _images/+1_V.png _images/+1_X.png
_images/plot.png
(1)\[\begin{split}\begin{align} B & = Brightness + 1 \\ Output_{RGB} & = (1-B) \cdot Input_{RGB}^2 + B \cdot Input_{RGB} \end{align}\end{split}\]
(2)\[Output_H = Input_H\]

Contrast

_images/node1.png
Contrast:Modifies image mid-tone contrast
Saturation:Preserves image saturation instead of chroma
Group:Adjust
Color Space:LAB

This module changes the contrast of the image by adjusting the LAB lightness slope at \(0.5\) with the Contrast parameter, and preserves the black point and white point by compressing the extremes (3). The Saturation parameter enables scaling of the chroma channel to preserve constant saturation instead of constant chroma when the lightness channel is modified (3).

This module does not account for changes in saturation perception due to a change in scene luminance contrast, as this is very much scene-dependent. A perceived decrease in saturation when increasing contrast and vice versa is observable. This can be corrected appropriately with the Vibrance or Saturation module.

Warning

The luminance black point and white point are preserved. Color channels may be pushed out of gamut.

Note

Setting the Contrast parameter to \(0.0\) creates a contrast curve with a slope of \(0.0\) in the mid-tones. This results in a very flat output image.

Contrast Saturation: On Saturation: Off
-1.0 _images/-1_V1.png _images/-1_X1.png
+0.0 _images/01.png _images/01.png
+1.0 _images/+1_V1.png _images/+1_X1.png
Contrast Saturation: On Saturation: Off
-1.0 _images/-1_V2.png _images/-1_X2.png
+0.0 _images/02.png _images/02.png
+1.0 _images/+1_V2.png _images/+1_X2.png
_images/plot1.png
(3)\[\begin{split}\begin{align} C &= Contrast + 1 \\ I &= 2 \cdot Input_L - 1 \\ O &= \left\{ \begin{array}{rl} (C-1) \cdot I^2 + I \cdot C & I < 0 \\ (1-C) \cdot I^2 + I \cdot C & I > 0 \end{array} \right. \\ Output_L &= \frac{O + 1}{2} \\ \end{align}\end{split}\]
(4)\[Output_{AB} = Input_{AB} \cdot (Output_L - Input_L)\]

Vibrance

_images/node2.png
Vibrance:Increases color saturation, mainly in less saturated areas
Group:Adjust
Color Space:LCH

This module changes the color saturation such that less saturated colors are boosted. It adjusts the LCH chroma channel slope at \(0.0\) with the Vibrance parameter, and preserves the saturation point by compressing the curve toward \(1.0\) (5). The Vibrance effect is modulated with the image’s lightness channel, such that the effect decreases linearly for darker colors. In addition, the output image lightness is decreased proportional to the increase in chroma \(\times 0.2\) to further enhance color perception.

Warning

While chroma in the LCH space is limited to \(1.0\), the resulting colors at this limit are still outside the sRGB gamut. This module does not necessarily prevent oversaturation.

Value Vibrance Saturation \(\times 0.5\)
-1.0 _images/-1.png _images/-0.5.png
+0.0 _images/03.png _images/04.png
+1.0 _images/+1.png _images/+0.5.png
(5)\[\begin{split}\begin{align} V &= Vibrance \cdot Input_L + 1 \\ Output_C &= (1-V) \cdot Input_C^2 + V \cdot Input_C \\ Output_L &= Input_L \cdot (1 - 0.2 \cdot (Output_C - Input_C)) \end{align}\end{split}\]

Note

The chroma curve for the Vibrance module is equivalent to the Brightness module curve. However, the strength of the Vibrance parameter is in addition modulated by the input image lightness, making the curve dependent on both lightness and chroma.

Saturation

_images/node3.png
Saturation:Increases color saturation linearly
Group:Adjust
Color Space:LCH

This module changes the LCH chroma linearly with the Saturation parameter as multiplication factor (6). It allows for full desaturation of the input image, as well as unbounded oversaturation.

Value Saturation
-1.0 _images/-11.png
-0.5 _images/-0.5.png
+0.0 _images/04.png
+0.5 _images/+0.5.png
+1.0 _images/+11.png
(6)\[Output_C = Input_C \cdot Saturation\]

Temperature

_images/node4.png
Temperature:Source correlated color temperature (K)
Tint:Green tint
Group:Adjust
Color Space:XYZ

This module corrects color cast due to the difference in scene illuminants compared to the white point of the viewing environment. The Temperature parameter indicates the correlated color temperature of the illuminant of the scene. The source color temperature is converted to a source white reference \(RefSource_{LMS}\) (8). The Tint parameter additionally scales the source white reference Y to correct a green cast. The destination white reference \(RefDest_{LMS}\) is computed for \(T = 6500\,K\) illuminant to match the D65 standard illuminant of sRGB. Chromatic adaptation is performed using the Von Kries transform in LMS space (9). A Bradford matrix (7) is used for the conversion from XYZ to LMS.

Note

The conversion from correlated color temperature to chromaticity is performed using the daylight locus as reference instead of the black body locus. This should be more appropriate for naturally occurring light.

Temperature Tint: 0.9 Tint: 1.0 Tint: 1.1
3800 K _images/3800_0.9.png _images/3800_1.png _images/3800_1.1.png
4700K _images/4700_0.9.png _images/4700_1.png _images/4700_1.1.png
5600K _images/5600_0.9.png _images/5600_1.png _images/5600_1.1.png
6500K _images/6500_0.9.png _images/6500_1.png _images/6500_1.1.png
8300K _images/8300_0.9.png _images/8300_1.png _images/8300_1.1.png
11000K _images/11000_0.9.png _images/11000_1.png _images/11000_1.1.png
15500K _images/15500_0.9.png _images/15500_1.png _images/15500_1.1.png
(7)\[\begin{split}M_{Bradford} = \left[ \begin{array}{rrr} 0.8951 & 0.2664 & 0.1614 \\ -0.7502 & 1.7135 & 0.0367 \\ 0.0389 & 0.0685 & 1.0296 \end{array} \right]\end{split}\]
(8)\[\begin{split}\begin{align} T &= Temperature \\ x &= \left\{\begin{array}{rl} \frac{0.27475E9}{T^3} - \frac{0.98598E6}{T^2} + \frac{1.17444E3}{T} + 0.145986 & T < 4000 \\ \frac{-4.6070E9}{T^3} + \frac{2.9678E6}{T^2} + \frac{0.09911E3}{T} + 0.244063 & 4000 < T < 7000 \\ \frac{-2.0064E9}{T^3} + \frac{1.9018E6}{T^2} + \frac{0.24748E3}{T} + 0.237040 & 7000 < T \end{array}\right. \\ y &= -3 \cdot x^2 + 2.87 \cdot x - 0.275 \\ \left[ \begin{array}{c} L \\ M \\ S \end{array} \right] &= M \cdot \left[ \begin{array}{c} X \\ Y \\ Z \end{array} \right] = M \cdot \left[ \begin{array}{c} \frac{x}{y} \\ Tint \\ \frac{1-x-y}{y} \end{array} \right] \end{align}\end{split}\]
(9)\[\begin{split}Output_{XYZ} = M^{-1} \cdot \left[ \begin{array}{ccc} \frac{RefDest_L}{RefSource_L} & 0 & 0 \\ 0 & \frac{RefDest_M}{RefSource_M} & 0 \\ 0 & 0 & \frac{RefDest_S}{RefSource_S} \end{array} \right] \cdot M \cdot Input_{XYZ}\end{split}\]

Curve Modules

The curve modules allow fine control of image adjustments. They define a mapping between one parameter and another specified as a user-defined function. The output of the mapping function can either set an absolute value, or offset or modulate the original value.

Parametric Curve

_images/node5.png
Shadows:Adjustment of the shadows
Darks:Adjustment of the dark tones
Lights:Adjustment of the light tones
Highlights:Adjustment of the highlights
Group:Adjust > Curves
Color Space:LAB

This module adjusts the lightness L in four tonal regions. The overall adjustment is a brightness offset factor \(F_{Tone}\) for each region (10), where \(V_{Tone}\) is one of the Shadows, Darks, Lights, or Highlights parameters. These curves are scaled and linearly modulated depending on the tonal range. They are combined in (11), where \(F_{Shadows}\) and \(F_{Highlights}\) are modulated with the headroom remaining after \(F_{Darks} + F_{Lights}\) is applied. The saturation is preserved in the AB components (12).

Tone -1.0 +1.0
Shadows _images/S-1.png _images/S+1.png
Darks _images/D-1.png _images/D+1.png
Lights _images/L-1.png _images/L+1.png
Highlights _images/H-1.png _images/H+1.png
(10)\[\begin{split}\begin{align} I &= Input_L \\ F_{Shadows} &= \left\{\begin{array}{rl} V_{Shadows} \cdot I \cdot (2 \cdot I - 1)^2 & I<0.5 \\ 0 & I>0.5 \end{array}\right. \\ F_{Darks} &= V_{Darks} \cdot (I - 1)^2 \cdot I \\ F_{Lights} &= -V_{Lights} \cdot (I - 1) \cdot I^2 \\ F_{Highlights} &= \left\{\begin{array}{rl} 0 & I<0.5 \\ - V_{Highlights} \cdot (I-1) \cdot (2 \cdot I - 1)^2 & I>0.5 \end{array}\right. \end{align}\end{split}\]
_images/plot2.png
(11)\[\begin{split}\begin{align} O &= Input_L + F_{Darks} + F_{Lights} \\ Output_L &= O + F_{Shadows} \cdot \frac{O}{Input_L} + F_{Highlights} \cdot \frac{1-O}{1-Input_L} \end{align}\end{split}\]
(12)\[Output_{AB} = Input_{AB} \cdot (Output_L - Input_L)\]

Curve L

_images/curve_L.png
Preserve Saturation:
 Maintain constant saturation
Group:Adjust > Curves
Color Space:LAB

This module sets the output lightness L as function of the input lightness. The implementation is similar to that of the Contrast module (13), but with a user-defined curve \(f\). The Saturation parameter enables scaling of the chroma channel to preserve constant saturation instead of constant chroma when the lightness channel is modified (14).

(13)\[Output_L = f(Input_L)\]
(14)\[Output_{AB} = Input_{AB} \cdot (Output_L - Input_L)\]

Curve Y

_images/curve_Y.png
Preserve Hue:Preserves input image hue in LCH space
Group:Adjust > Curves
Color Space:XYZ

This module sets the output luminance Y as function of the input luminance. The implementation is similar to that of the Brightness module (15), but with a user-defined curve \(f\). The Preserve Hue parameter applies the hue of the input image to that of the output image in LCH space (16).

(15)\[Output_Y = f(Input_Y)\]
(16)\[Output_H = Input_H\]

Advanced Curve modules

Group:Adjust > Curves > Advanced
Color Space:LCH

These curves map one component of the LCH color space against another in the following way, where \(f\) is the curve function:

Curve L-L:\(Output_L = f(Input_L)\)
Curve L-C:\(Output_C = Input_C \cdot f(Input_L)\)
Curve L-H:\(Output_H = Input_H + f(Input_L)\)
Curve C-L:\(Output_L = Input_L \cdot f(Input_C)\)
Curve C-C:\(Output_C = f(Input_C)\)
Curve C-H:\(Output_H = Input_H + f(Input_C)\)
Curve H-L:\(Output_L = Input_L \cdot ((f(Input_H)-1) \cdot Input_C + 1)\)
Curve H-C:\(Output_C = Input_C \cdot f(Input_H)\)
Curve H-H:\(Output_H = Input_H + f(Input_H)\)

Note

The H-L curve effect is modulated with chroma, such that low chroma input with noisy hue does not result in noisy lightness in the output.

Mask Curve Modules

Group:Adjust > Curves > Advanced
Color Space:LCH > Y

These curves create a single-channel mask output based on curve \(f\) applied to the input channel:

Select L:\(Output_Y = f(Input_L)\)
Select C:\(Output_Y = f(Input_C)\)
Select H:\(Output_Y = f(Input_H)\)

Enhance modules

These modules offer tools for local adjustments of the image being processed. The performed operations depend on the neighboring image details at different scales.

Color Spaces

Ivy processes image data which stores color information in a 2-dimensional spatial grid. Color, as characterized by human visual perception, can be mapped to a 3-dimensional space defined by the 3 distinct response spectra of the cone cell photoreceptors in the human eye. These are defined as S (420nm–440nm), M (530nm-540nm), and L (560nm-580nm) corresponding to the peak wavelength they are sensitive to. However, the mapping of color to this LMS space is not always intuitive. Therefor several other color spaces have been defined which more closely match physical or perceptual concepts of light. Transformation between common color spaces are lossless, as they map the same 3-dimensional space.

Note

For capturing and displaying purposes, the color space components are limited to a defined range (gamut). Within Ivy, this restriction is only applied when strictly necessary. The processing pipeline preserves out of gamut colors until export to file or display.

Note

These color spaces do not capture spectrum of visible light in full detail. There is a loss of information when capturing light as a tristimulus color, whether with common imaging devices or in human vision. When looking at most commercially available imaging and display technology, imaging devices map the visible light spectrum to color such that the reproduced light spectrum will be perceived approximately the same as the original spectrum. The loss of spectral information is inherent to this process.

CIE XYZ

The XYZ color space is closely related to human visual perception, where the Y component describes the perceived luminance, the Z component approximates the blue response of the S cones, and the X component is chosen such that all chromaticities of a given luminance Y are represented by positive XZ values.

Internal name:XYZ
Range:\(X, Y, Z \in [0, 1]\)

sRGB

The sRGB color space is primarily used in color displays, mapping to the red, green, and blue sub-pixels. Each of these RGB components is non-linearly transformed, originally to account for the gamma response of CRT displays. Nowadays, this transformation is preserved, as it allows for more efficient storage of human-discernible light levels than a linear mapping. In that sense it is also a (rough) approximation of the non-linear human light intensity perception. The sRGB color space is mapped to a range of \([0, 1]\) instead of the common \([0, 255]\).

Internal name:SRGB
Range:\(R, G, B \in [0, 1]\)

Linear RGB

Internally, a linear light equivalent of the SRGB color space is used. It is useful when dealing with the physical aspects of light, such as additive color mixing. The chromaticity of the RGB components is equivalent to that of the sRGB color space.

Internal name:LRGB
Range:\(R, G, B \in [0, 1]\)

CIE Lab

The LAB color space is designed to approximate human vision. It is perceptually uniform, and the L component matches the human perception of lightness. The A and B components represent the magenta-green axis and the yellow-blue axis respectively. While the definition of L has a range of \([0, 100]\) and AB a range of \([-128, 128]\), these are normalized to a range of \([0, 1]\) and \([-1, 1]\) respectively.

Internal name:LAB
Range:\(L \in [0, 1]; A, B \in [-1, 1]\)

Note

While the LAB color space is based on human visual perception, there are many color appearance phenomena which are not taken into account. As such, it does not fully match human perception in visually complex scenes. It represents a reasonable trade-off between accuracy and complexity.

CIE LCh

Based on the previously described LAB color space, the LCH color space is a radial mapping of the AB coordinates into chroma C the length of the AB vector, and hue H the direction of the AB vector. The L component is the same as in LAB. The H component range is internally changed from \([0, 2 \cdot \pi]\) to \([0, 1]\) for easier manipulation.

Internal name:LCH
Range:\(L, C, H \in [0, 1]\)

Y and L

In addition to the trichromatic color values, Ivy internally uses two monochromatic data representations when there is no color information. This is useful for mask or modulation data. The Y and L representations are derived from the Y and L components of XYZ and LAB respectively. They offer a choice between linear light intensity or perceptual lightness scales.

Internal name:Y
Range:\(Y \in [0, 1]\)
Internal name:L
Range:\(L \in [0, 1]\)

Support

Downloads

  • Windows package

Poject Repository

  • GitLab project page
  • Bug tracker
  • …

Source Installation

Dependencies

Run-time dependencies:

Distribution